2023-10-06T04:57:17.920 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T04:57:17.920 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320 --name yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi --verbose -- /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320/orig.config.yaml 2023-10-06T04:57:17.966 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T04:57:18.054 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320 branch: wip-yuri3-testing-2023-10-05-0943-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} email: yweinste@redhat.com first_in_suite: false job_id: '7412320' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri3-testing-2023-10-05-0943-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-05-0943-quincy sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4002 sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-05-0943-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOCtRlN3Pn051NZ34WfUmcvDmyDYi1mMnnAnxhB2BJY7wk7jHyiXEIm5o3wKOQLH9VdtEhJwxUx5MB1PDqcrXk= smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsWIedGVlr9JwI163R6gtU8pVRhzz0ug/T52ijS54Zev/RwmSYO1NjHrhaiYHJrMWM5FMNsEeqjICUBTlGjX1Q= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:54:57 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T04:57:18.054 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa; will attempt to use it 2023-10-06T04:57:18.055 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks 2023-10-06T04:57:18.055 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T04:57:18.057 INFO:teuthology.task.internal:Checking packages... 2023-10-06T04:57:18.080 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564' 2023-10-06T04:57:18.080 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T04:57:18.080 INFO:teuthology.packaging:ref: None 2023-10-06T04:57:18.081 INFO:teuthology.packaging:tag: None 2023-10-06T04:57:18.081 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T04:57:18.081 INFO:teuthology.packaging:sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T04:57:18.081 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T04:57:18.238 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1274.g68e53a4a 2023-10-06T04:57:18.238 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T04:57:18.244 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T04:57:18.244 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T04:57:18.253 INFO:teuthology.task.internal:Saving configuration 2023-10-06T04:57:18.270 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T04:57:18.275 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T04:57:18.298 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:52:21.445791', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOCtRlN3Pn051NZ34WfUmcvDmyDYi1mMnnAnxhB2BJY7wk7jHyiXEIm5o3wKOQLH9VdtEhJwxUx5MB1PDqcrXk='} 2023-10-06T04:57:18.324 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:52:21.447283', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsWIedGVlr9JwI163R6gtU8pVRhzz0ug/T52ijS54Zev/RwmSYO1NjHrhaiYHJrMWM5FMNsEeqjICUBTlGjX1Q='} 2023-10-06T04:57:18.324 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T04:57:18.330 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T04:57:18.330 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T04:57:18.330 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T04:57:18.400 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc71fa3db80>, signals=[15]) 2023-10-06T04:57:18.401 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T04:57:18.407 INFO:teuthology.task.internal:Opening connections... 2023-10-06T04:57:18.407 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T04:57:18.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:57:18.519 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T04:57:18.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:57:18.586 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T04:57:18.592 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2023-10-06T04:57:18.615 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2023-10-06T04:57:18.616 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-10-06T04:57:18.677 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2023-10-06T04:57:18.677 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="8" 2023-10-06T04:57:18.677 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2023-10-06T04:57:18.677 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2023-10-06T04:57:18.677 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="8" 2023-10-06T04:57:18.677 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el8" 2023-10-06T04:57:18.677 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-06T04:57:18.678 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-10-06T04:57:18.678 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-06T04:57:18.678 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2023-10-06T04:57:18.678 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T04:57:18.678 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T04:57:18.678 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T04:57:18.679 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-10-06T04:57:18.694 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2023-10-06T04:57:18.718 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2023-10-06T04:57:18.718 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-10-06T04:57:18.789 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2023-10-06T04:57:18.789 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8" 2023-10-06T04:57:18.789 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2023-10-06T04:57:18.789 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2023-10-06T04:57:18.789 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8" 2023-10-06T04:57:18.790 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2023-10-06T04:57:18.790 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-06T04:57:18.790 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-10-06T04:57:18.790 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-06T04:57:18.790 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2023-10-06T04:57:18.791 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T04:57:18.791 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T04:57:18.791 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T04:57:18.792 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-10-06T04:57:18.822 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T04:57:18.855 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T04:57:18.954 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T04:57:18.954 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2023-10-06T04:57:18.957 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2023-10-06T04:57:18.979 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T04:57:18.986 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T04:57:18.986 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2023-10-06T04:57:19.021 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2023-10-06T04:57:19.049 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T04:57:19.115 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T04:57:19.134 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-06T04:57:19.135 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-06T04:57:19.135 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-06T04:57:19.135 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T04:57:19.136 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T04:57:19.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:57:19.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:57:19.163 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-06T04:57:19.164 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-06T04:57:19.336 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:57:19.337 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T04:57:19.337 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2023-10-06T04:57:19.363 INFO:teuthology.orchestra.run.smithi031.stdout:4.18.0-489.el8.x86_64 2023-10-06T04:57:19.363 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:57:19.364 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T04:57:19.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum upgrade -y linux-firmware 2023-10-06T04:57:19.399 DEBUG:teuthology.task.kernel:sha1 for client.1 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:57:19.399 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T04:57:19.399 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2023-10-06T04:57:19.422 INFO:teuthology.orchestra.run.smithi062.stdout:4.18.0-489.el8.x86_64 2023-10-06T04:57:19.422 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:57:19.422 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T04:57:19.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum upgrade -y linux-firmware 2023-10-06T04:57:23.322 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - AppStream 27 MB/s | 33 MB 00:01 2023-10-06T04:57:24.174 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 8 - AppStream 14 MB/s | 33 MB 00:02 2023-10-06T04:57:34.547 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - BaseOS 9.9 MB/s | 48 MB 00:04 2023-10-06T04:57:35.182 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 8 - BaseOS 10 MB/s | 48 MB 00:04 2023-10-06T04:57:42.197 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - Extras 56 kB/s | 18 kB 00:00 2023-10-06T04:57:42.797 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2023-10-06T04:57:42.915 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 8 - Extras 58 kB/s | 18 kB 00:00 2023-10-06T04:57:43.199 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-10-06T04:57:43.498 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 8 - Extras common packages 18 kB/s | 6.9 kB 00:00 2023-10-06T04:57:43.808 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-06T04:57:44.776 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-10-06T04:57:45.391 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 16 MB 00:02 2023-10-06T04:57:48.825 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 357 kB/s | 24 kB 00:00 2023-10-06T04:57:49.377 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 379 kB/s | 24 kB 00:00 2023-10-06T04:57:52.823 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-10-06T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2023-10-06T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2023-10-06T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2023-10-06T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2023-10-06T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: linux-firmware noarch 20230824-118.git0e048b06.el8 baseos 286 M 2023-10-06T04:57:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2023-10-06T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2023-10-06T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 1 Package 2023-10-06T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 286 M 2023-10-06T04:57:52.825 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2023-10-06T04:57:53.289 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-10-06T04:57:53.290 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-10-06T04:57:53.290 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2023-10-06T04:57:53.290 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-10-06T04:57:53.290 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2023-10-06T04:57:53.290 INFO:teuthology.orchestra.run.smithi062.stdout: linux-firmware noarch 20230824-118.git0e048b06.el8 baseos 286 M 2023-10-06T04:57:53.290 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T04:57:53.290 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-10-06T04:57:53.291 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-10-06T04:57:53.291 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 1 Package 2023-10-06T04:57:53.291 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T04:57:53.291 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 286 M 2023-10-06T04:57:53.291 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-10-06T04:58:13.508 INFO:teuthology.orchestra.run.smithi031.stdout:linux-firmware-20230824-118.git0e048b06.el8.noa 14 MB/s | 286 MB 00:20 2023-10-06T04:58:13.508 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2023-10-06T04:58:13.508 INFO:teuthology.orchestra.run.smithi031.stdout:Total 14 MB/s | 286 MB 00:20 2023-10-06T04:58:14.041 INFO:teuthology.orchestra.run.smithi062.stdout:linux-firmware-20230824-118.git0e048b06.el8.noa 14 MB/s | 286 MB 00:20 2023-10-06T04:58:14.041 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-10-06T04:58:14.041 INFO:teuthology.orchestra.run.smithi062.stdout:Total 14 MB/s | 286 MB 00:20 2023-10-06T04:58:15.294 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2023-10-06T04:58:15.370 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2023-10-06T04:58:15.370 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2023-10-06T04:58:15.832 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-10-06T04:58:15.904 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-10-06T04:58:15.904 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-10-06T04:58:17.242 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2023-10-06T04:58:17.266 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2023-10-06T04:58:17.791 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-10-06T04:58:17.827 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-10-06T04:58:37.398 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2023-10-06T04:58:37.745 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-10-06T04:58:37.983 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-10-06T04:58:38.034 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-10-06T04:58:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-10-06T04:58:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-10-06T04:58:38.402 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-10-06T04:58:38.613 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-10-06T04:58:38.846 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-10-06T04:58:38.846 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-10-06T04:58:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-10-06T04:58:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T04:58:39.082 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2023-10-06T04:58:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: linux-firmware-20230824-118.git0e048b06.el8.noarch 2023-10-06T04:58:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T04:58:39.082 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-10-06T04:58:39.250 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T04:58:39.251 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:58:39.466 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T04:58:39.466 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T04:58:39.468 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi031.front.sepia.ceph.com to complete... 2023-10-06T04:58:39.636 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-10-06T04:58:39.636 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T04:58:39.636 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2023-10-06T04:58:39.637 INFO:teuthology.orchestra.run.smithi062.stdout: linux-firmware-20230824-118.git0e048b06.el8.noarch 2023-10-06T04:58:39.637 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T04:58:39.637 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-10-06T04:58:39.993 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-06T04:58:39.994 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T04:58:40.136 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T04:58:40.136 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T04:58:40.138 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi062.front.sepia.ceph.com to complete... 2023-10-06T04:58:56.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06 04:58:56 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T04:58:56.556 INFO:teuthology.orchestra.run.smithi062.stderr:FINISHED --2023-10-06 04:58:56-- 2023-10-06T04:58:56.556 INFO:teuthology.orchestra.run.smithi062.stderr:Total wall clock time: 16s 2023-10-06T04:58:56.557 INFO:teuthology.orchestra.run.smithi062.stderr:Downloaded: 1 files, 363M in 16s (22.6 MB/s) 2023-10-06T04:58:56.558 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-06T04:58:56.558 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T04:58:58.417 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying... ######################################## 2023-10-06T04:58:58.463 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing... ######################################## 2023-10-06T04:58:58.579 INFO:teuthology.orchestra.run.smithi062.stdout:Updating / installing... 2023-10-06T04:59:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06 04:59:08 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T04:59:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:FINISHED --2023-10-06 04:59:08-- 2023-10-06T04:59:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:Total wall clock time: 29s 2023-10-06T04:59:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:Downloaded: 1 files, 363M in 29s (12.5 MB/s) 2023-10-06T04:59:08.993 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T04:59:08.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T04:59:10.733 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying... ######################################## 2023-10-06T04:59:10.779 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing... ######################################## 2023-10-06T04:59:10.894 INFO:teuthology.orchestra.run.smithi031.stdout:Updating / installing... 2023-10-06T04:59:39.086 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T04:59:51.530 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T05:00:02.753 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T05:00:02.753 DEBUG:teuthology.orchestra.run.smithi062:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T05:00:02.808 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:02.808 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:02.808 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:02.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:02.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T05:00:02.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T05:00:02.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T05:00:02.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T05:00:02.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T05:00:02.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T05:00:02.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T05:00:02.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T05:00:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T05:00:02.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T05:00:02.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T05:00:02.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T05:00:02.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T05:00:02.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T05:00:02.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T05:00:02.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T05:00:02.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T05:00:02.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T05:00:02.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T05:00:02.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T05:00:02.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T05:00:02.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T05:00:02.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T05:00:02.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T05:00:02.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T05:00:02.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T05:00:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T05:00:02.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T05:00:02.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T05:00:02.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T05:00:02.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T05:00:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T05:00:02.839 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T05:00:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T05:00:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T05:00:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T05:00:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T05:00:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T05:00:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T05:00:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T05:00:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T05:00:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T05:00:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T05:00:02.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T05:00:02.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T05:00:02.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T05:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T05:00:02.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T05:00:02.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T05:00:02.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T05:00:02.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T05:00:02.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T05:00:02.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T05:00:02.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T05:00:02.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T05:00:02.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T05:00:02.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T05:00:02.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T05:00:02.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T05:00:02.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T05:00:02.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T05:00:02.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T05:00:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T05:00:02.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T05:00:02.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T05:00:02.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T05:00:02.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T05:00:02.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T05:00:02.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T05:00:02.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T05:00:02.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T05:00:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T05:00:02.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T05:00:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T05:00:02.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T05:00:02.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T05:00:02.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T05:00:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T05:00:02.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T05:00:02.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T05:00:02.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T05:00:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T05:00:02.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T05:00:02.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T05:00:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T05:00:02.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T05:00:02.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T05:00:02.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T05:00:02.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T05:00:02.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T05:00:02.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T05:00:02.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T05:00:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T05:00:02.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T05:00:02.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T05:00:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T05:00:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T05:00:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T05:00:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T05:00:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T05:00:02.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T05:00:02.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T05:00:02.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T05:00:02.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T05:00:02.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T05:00:02.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T05:00:02.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T05:00:02.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T05:00:02.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T05:00:02.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T05:00:02.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T05:00:02.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T05:00:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T05:00:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T05:00:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T05:00:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T05:00:02.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T05:00:02.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T05:00:02.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T05:00:02.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T05:00:02.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T05:00:02.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T05:00:02.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T05:00:02.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T05:00:02.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T05:00:02.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T05:00:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T05:00:02.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T05:00:02.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T05:00:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T05:00:02.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T05:00:02.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T05:00:02.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T05:00:02.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T05:00:02.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T05:00:02.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T05:00:02.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T05:00:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T05:00:02.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T05:00:02.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T05:00:02.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T05:00:02.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T05:00:02.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T05:00:02.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T05:00:02.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T05:00:02.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T05:00:02.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T05:00:02.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T05:00:02.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T05:00:02.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T05:00:02.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T05:00:02.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T05:00:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T05:00:02.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T05:00:02.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T05:00:02.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T05:00:02.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T05:00:02.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T05:00:02.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T05:00:02.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T05:00:02.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T05:00:02.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T05:00:02.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T05:00:02.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T05:00:02.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T05:00:02.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T05:00:02.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T05:00:02.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T05:00:02.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T05:00:02.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T05:00:02.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T05:00:02.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T05:00:02.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T05:00:02.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T05:00:02.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T05:00:02.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T05:00:02.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T05:00:02.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T05:00:02.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T05:00:02.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T05:00:02.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T05:00:02.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T05:00:02.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T05:00:02.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T05:00:02.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T05:00:02.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T05:00:02.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T05:00:02.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T05:00:02.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T05:00:02.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T05:00:02.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T05:00:02.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T05:00:02.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T05:00:02.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T05:00:02.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T05:00:02.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T05:00:02.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T05:00:02.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T05:00:02.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T05:00:02.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T05:00:02.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T05:00:02.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T05:00:02.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T05:00:02.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T05:00:02.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T05:00:02.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T05:00:02.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T05:00:02.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T05:00:02.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T05:00:02.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T05:00:02.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T05:00:02.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T05:00:02.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T05:00:02.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T05:00:02.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T05:00:02.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T05:00:02.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T05:00:02.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T05:00:02.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T05:00:02.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T05:00:02.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T05:00:02.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T05:00:02.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T05:00:02.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T05:00:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T05:00:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T05:00:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T05:00:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T05:00:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T05:00:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T05:00:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T05:00:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T05:00:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T05:00:02.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T05:00:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T05:00:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T05:00:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T05:00:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T05:00:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T05:00:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T05:00:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T05:00:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T05:00:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T05:00:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T05:00:02.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T05:00:02.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T05:00:02.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T05:00:02.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T05:00:02.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T05:00:02.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T05:00:02.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T05:00:02.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T05:00:02.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T05:00:02.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T05:00:02.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T05:00:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T05:00:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T05:00:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T05:00:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T05:00:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T05:00:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T05:00:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T05:00:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T05:00:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T05:00:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T05:00:02.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T05:00:02.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T05:00:02.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T05:00:02.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T05:00:02.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T05:00:02.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T05:00:02.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T05:00:02.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T05:00:02.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T05:00:02.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T05:00:02.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T05:00:02.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T05:00:02.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T05:00:02.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T05:00:02.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T05:00:02.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T05:00:02.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T05:00:02.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T05:00:02.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T05:00:02.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T05:00:02.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T05:00:02.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T05:00:02.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T05:00:02.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T05:00:02.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T05:00:02.971 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:02.971 DEBUG:teuthology.orchestra.run.smithi062:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T05:00:02.997 INFO:teuthology.orchestra.run.smithi062.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T05:00:02.997 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T05:00:02.997 INFO:teuthology.orchestra.run.smithi062.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T05:00:02.997 INFO:teuthology.orchestra.run.smithi062.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T05:00:02.997 INFO:teuthology.orchestra.run.smithi062.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:02.997 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T05:00:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T05:00:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2023-10-06T05:00:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T05:00:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T05:00:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T05:00:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T05:00:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2023-10-06T05:00:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T05:00:02.998 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T05:00:02.999 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:02.999 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2023-10-06T05:00:03.000 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2023-10-06T05:00:03.076 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2023-10-06T05:00:03.076 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2023-10-06T05:00:03.076 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.02 2023-10-06T05:00:03.076 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 129.el8 2023-10-06T05:00:03.076 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2023-10-06T05:00:03.076 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-06T05:00:03.076 INFO:teuthology.orchestra.run.smithi062.stdout:Group : System Environment/Base 2023-10-06T05:00:03.076 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 9265586 2023-10-06T05:00:03.076 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2023-10-06T05:00:03.077 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-06T05:00:03.077 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-06T05:00:03.077 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-06T05:00:03.077 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-06T05:00:03.077 INFO:teuthology.orchestra.run.smithi062.stdout:Relocations : (not relocatable) 2023-10-06T05:00:03.077 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : CentOS BuildSystem 2023-10-06T05:00:03.077 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : CentOS 2023-10-06T05:00:03.077 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T05:00:03.077 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2023-10-06T05:00:03.077 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:00:03.078 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T05:00:03.079 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T05:00:03.079 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:03.079 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T05:00:07.941 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2023-10-06T05:00:10.180 INFO:teuthology.orchestra.run.smithi062.stderr:done 2023-10-06T05:00:10.182 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2023-10-06T05:00:10.203 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.6aKMuq6BA6 2023-10-06T05:00:10.203 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6aKMuq6BA6 2023-10-06T05:00:10.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.6aKMuq6BA6 2023-10-06T05:00:10.442 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.6aKMuq6BA6 is 6KB 2023-10-06T05:00:10.491 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.6aKMuq6BA6 2023-10-06T05:00:10.509 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T05:00:10.575 INFO:teuthology.orchestra.run.smithi062.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-06T05:00:10.575 INFO:teuthology.orchestra.run.smithi062.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-06T05:00:10.575 INFO:teuthology.orchestra.run.smithi062.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-06T05:00:10.575 INFO:teuthology.orchestra.run.smithi062.stdout:a1b29d6c289747bdb22930598d39874a-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T05:00:10.576 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:10.724 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2023-10-06T05:00:13.913 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T05:00:13.913 DEBUG:teuthology.orchestra.run.smithi031:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T05:00:13.970 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T05:00:13.971 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T05:00:13.972 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T05:00:13.973 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T05:00:13.974 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T05:00:13.975 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T05:00:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T05:00:13.977 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T05:00:13.978 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T05:00:13.979 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T05:00:13.980 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T05:00:13.981 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T05:00:13.982 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T05:00:13.983 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T05:00:13.984 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T05:00:13.985 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T05:00:13.986 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T05:00:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T05:00:13.988 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T05:00:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T05:00:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T05:00:13.991 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T05:00:13.992 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T05:00:13.993 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T05:00:13.994 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T05:00:13.995 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T05:00:13.996 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T05:00:13.997 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T05:00:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T05:00:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T05:00:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T05:00:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T05:00:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T05:00:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T05:00:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T05:00:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T05:00:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T05:00:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T05:00:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T05:00:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T05:00:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T05:00:14.002 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T05:00:14.003 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T05:00:14.004 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T05:00:14.005 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T05:00:14.005 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T05:00:14.005 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T05:00:14.005 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T05:00:14.005 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T05:00:14.005 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T05:00:14.005 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T05:00:14.005 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T05:00:14.005 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T05:00:14.005 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T05:00:14.006 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T05:00:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T05:00:14.008 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T05:00:14.008 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T05:00:14.008 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T05:00:14.008 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T05:00:14.009 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T05:00:14.010 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T05:00:14.010 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T05:00:14.010 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T05:00:14.010 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T05:00:14.010 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T05:00:14.010 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T05:00:14.010 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T05:00:14.010 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T05:00:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T05:00:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T05:00:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T05:00:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T05:00:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T05:00:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T05:00:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T05:00:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T05:00:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T05:00:14.011 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T05:00:14.012 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T05:00:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T05:00:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T05:00:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T05:00:14.016 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T05:00:14.017 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T05:00:14.018 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T05:00:14.018 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T05:00:14.018 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T05:00:14.018 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T05:00:14.018 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T05:00:14.018 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T05:00:14.018 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T05:00:14.018 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T05:00:14.018 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T05:00:14.019 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T05:00:14.020 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T05:00:14.021 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T05:00:14.022 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T05:00:14.023 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T05:00:14.023 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T05:00:14.023 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T05:00:14.023 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T05:00:14.023 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T05:00:14.023 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T05:00:14.023 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T05:00:14.023 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T05:00:14.023 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T05:00:14.023 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T05:00:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T05:00:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T05:00:14.026 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T05:00:14.027 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T05:00:14.028 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T05:00:14.029 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T05:00:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T05:00:14.031 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T05:00:14.032 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T05:00:14.033 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T05:00:14.034 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T05:00:14.034 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T05:00:14.034 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T05:00:14.034 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T05:00:14.034 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T05:00:14.034 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T05:00:14.034 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T05:00:14.034 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T05:00:14.034 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T05:00:14.034 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T05:00:14.035 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T05:00:14.036 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T05:00:14.037 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T05:00:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T05:00:14.039 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T05:00:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T05:00:14.041 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T05:00:14.042 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T05:00:14.043 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T05:00:14.044 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T05:00:14.045 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T05:00:14.046 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T05:00:14.047 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T05:00:14.048 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T05:00:14.049 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T05:00:14.050 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T05:00:14.051 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T05:00:14.052 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T05:00:14.053 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T05:00:14.054 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T05:00:14.055 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T05:00:14.056 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T05:00:14.057 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T05:00:14.058 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T05:00:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T05:00:14.060 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T05:00:14.061 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T05:00:14.062 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T05:00:14.063 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T05:00:14.064 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T05:00:14.065 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T05:00:14.066 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T05:00:14.066 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T05:00:14.066 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T05:00:14.066 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T05:00:14.066 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T05:00:14.066 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T05:00:14.066 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T05:00:14.066 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T05:00:14.066 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T05:00:14.066 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T05:00:14.067 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T05:00:14.068 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T05:00:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T05:00:14.070 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T05:00:14.070 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T05:00:14.070 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T05:00:14.070 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T05:00:14.070 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T05:00:14.070 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T05:00:14.070 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T05:00:14.070 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T05:00:14.072 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T05:00:14.072 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T05:00:14.072 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T05:00:14.072 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T05:00:14.072 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T05:00:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T05:00:14.074 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T05:00:14.074 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T05:00:14.074 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T05:00:14.074 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T05:00:14.074 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T05:00:14.105 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T05:00:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T05:00:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T05:00:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T05:00:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T05:00:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T05:00:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T05:00:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T05:00:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T05:00:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T05:00:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T05:00:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T05:00:14.108 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T05:00:14.109 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T05:00:14.110 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T05:00:14.111 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T05:00:14.112 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T05:00:14.112 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T05:00:14.112 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T05:00:14.112 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T05:00:14.112 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T05:00:14.112 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T05:00:14.113 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T05:00:14.114 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T05:00:14.114 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T05:00:14.114 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T05:00:14.114 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T05:00:14.114 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T05:00:14.114 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T05:00:14.114 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T05:00:14.114 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T05:00:14.114 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T05:00:14.114 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T05:00:14.115 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T05:00:14.116 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T05:00:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T05:00:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T05:00:14.119 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T05:00:14.120 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T05:00:14.120 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T05:00:14.120 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T05:00:14.120 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T05:00:14.121 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T05:00:14.121 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T05:00:14.121 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T05:00:14.121 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T05:00:14.121 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T05:00:14.121 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T05:00:14.121 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T05:00:14.121 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T05:00:14.121 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T05:00:14.122 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T05:00:14.122 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T05:00:14.122 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T05:00:14.122 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T05:00:14.122 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T05:00:14.122 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T05:00:14.122 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T05:00:14.122 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T05:00:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T05:00:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T05:00:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T05:00:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T05:00:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T05:00:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T05:00:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T05:00:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T05:00:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T05:00:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T05:00:14.125 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T05:00:14.126 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T05:00:14.127 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T05:00:14.128 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T05:00:14.129 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T05:00:14.130 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T05:00:14.131 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T05:00:14.132 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T05:00:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T05:00:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T05:00:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T05:00:14.136 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T05:00:14.137 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T05:00:14.138 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T05:00:14.139 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T05:00:14.140 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T05:00:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T05:00:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T05:00:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T05:00:14.144 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T05:00:14.145 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T05:00:14.146 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T05:00:14.147 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T05:00:14.148 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T05:00:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T05:00:14.150 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T05:00:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T05:00:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T05:00:14.153 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T05:00:14.154 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T05:00:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T05:00:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T05:00:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T05:00:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T05:00:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T05:00:14.160 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T05:00:14.161 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T05:00:14.162 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T05:00:14.163 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T05:00:14.164 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T05:00:14.164 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T05:00:14.164 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T05:00:14.164 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T05:00:14.164 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T05:00:14.164 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T05:00:14.164 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T05:00:14.164 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T05:00:14.164 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T05:00:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T05:00:14.166 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T05:00:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T05:00:14.168 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T05:00:14.169 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T05:00:14.170 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T05:00:14.171 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T05:00:14.171 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T05:00:14.171 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T05:00:14.171 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T05:00:14.171 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T05:00:14.171 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T05:00:14.171 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T05:00:14.171 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T05:00:14.171 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T05:00:14.171 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T05:00:14.172 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T05:00:14.172 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T05:00:14.172 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T05:00:14.172 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T05:00:14.172 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T05:00:14.172 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T05:00:14.172 INFO:teuthology.orchestra.run.smithi031.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T05:00:14.173 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:14.173 DEBUG:teuthology.orchestra.run.smithi031:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T05:00:14.201 INFO:teuthology.orchestra.run.smithi031.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T05:00:14.201 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T05:00:14.201 INFO:teuthology.orchestra.run.smithi031.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T05:00:14.201 INFO:teuthology.orchestra.run.smithi031.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T05:00:14.201 INFO:teuthology.orchestra.run.smithi031.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T05:00:14.202 INFO:teuthology.orchestra.run.smithi031.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T05:00:14.203 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:14.203 INFO:teuthology.orchestra.run.smithi031.stdout:fi 2023-10-06T05:00:14.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2023-10-06T05:00:14.284 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2023-10-06T05:00:14.284 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2023-10-06T05:00:14.284 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.02 2023-10-06T05:00:14.284 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 129.el8 2023-10-06T05:00:14.284 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2023-10-06T05:00:14.284 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-06T05:00:14.284 INFO:teuthology.orchestra.run.smithi031.stdout:Group : System Environment/Base 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 9265586 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:Relocations : (not relocatable) 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : CentOS BuildSystem 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2023-10-06T05:00:14.285 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2023-10-06T05:00:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:00:14.286 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T05:00:14.286 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T05:00:14.286 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T05:00:14.286 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2023-10-06T05:00:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:00:14.286 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T05:00:14.287 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T05:00:14.287 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:14.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T05:00:19.006 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2023-10-06T05:00:21.415 INFO:teuthology.orchestra.run.smithi031.stderr:done 2023-10-06T05:00:21.417 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2023-10-06T05:00:21.479 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.U1bBiMkY9W 2023-10-06T05:00:21.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.U1bBiMkY9W 2023-10-06T05:00:21.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.U1bBiMkY9W 2023-10-06T05:00:21.703 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.U1bBiMkY9W is 6KB 2023-10-06T05:00:21.750 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.U1bBiMkY9W 2023-10-06T05:00:21.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T05:00:21.832 INFO:teuthology.orchestra.run.smithi031.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-06T05:00:21.832 INFO:teuthology.orchestra.run.smithi031.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-06T05:00:21.832 INFO:teuthology.orchestra.run.smithi031.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-06T05:00:21.833 INFO:teuthology.orchestra.run.smithi031.stdout:a1b29d6c289747bdb22930598d39874a-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T05:00:21.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.5.0-rc7-gfef64f93da2b 2023-10-06T05:00:22.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2023-10-06T05:00:40.742 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-06T05:00:40.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T05:00:40.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:00:52.050 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T05:00:52.051 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T05:00:52.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:00:59.130 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-06T05:01:08.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T05:01:08.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:01:10.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-10-06T05:01:11.199 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-06T05:01:19.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T05:01:19.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:01:23.202 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T05:01:23.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:01:37.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-10-06T05:01:41.626 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-10-06T05:01:49.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T05:01:49.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:01:52.858 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-10-06T05:01:56.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T05:01:56.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:01:56.929 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-10-06T05:01:57.307 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T05:01:57.308 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T05:01:57.308 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2023-10-06T05:01:57.353 INFO:teuthology.orchestra.run.smithi062.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T05:01:57.353 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T05:01:57.353 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T05:01:57.353 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-06T05:01:58.354 DEBUG:teuthology.parallel:result is None 2023-10-06T05:02:07.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T05:02:07.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:02:08.229 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-10-06T05:02:08.646 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T05:02:08.646 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T05:02:08.647 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2023-10-06T05:02:08.692 INFO:teuthology.orchestra.run.smithi031.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T05:02:08.692 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T05:02:08.692 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T05:02:08.692 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-06T05:02:09.693 DEBUG:teuthology.parallel:result is None 2023-10-06T05:02:09.694 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T05:02:09.703 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T05:02:09.703 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T05:02:09.708 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T05:02:09.760 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T05:02:09.809 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T05:02:09.820 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T05:02:09.821 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T05:02:09.825 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T05:02:09.889 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T05:02:09.897 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T05:02:09.897 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T05:02:09.902 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T05:02:10.023 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T05:02:10.034 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T05:02:10.066 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T05:02:10.085 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T05:02:10.089 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T05:02:10.098 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T05:02:10.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T05:02:10.103 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T05:02:10.174 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T05:02:10.183 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T05:02:10.183 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T05:02:10.188 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T05:02:10.233 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T05:02:10.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T05:02:10.411 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T05:02:10.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T05:02:10.550 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:02:10.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T05:02:10.620 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T05:02:10.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T05:02:10.764 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T05:02:10.838 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T05:02:10.908 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:02:10.908 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T05:02:10.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2023-10-06T05:02:10.982 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2023-10-06T05:02:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:02:11.136 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:02:11.602 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T05:02:11.610 INFO:teuthology.task.internal:Starting timer... 2023-10-06T05:02:11.610 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T05:02:11.633 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T05:02:11.645 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T05:02:11.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2023-10-06T05:02:11.810 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2023-10-06T05:02:11.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2023-10-06T05:02:11.938 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2023-10-06T05:02:11.941 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T05:02:11.942 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2023-10-06T05:02:11.994 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2023-10-06T05:02:11.994 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2023-10-06T05:02:12.041 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2023-10-06T05:02:12.041 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi062.front.sepia.ceph.com': 'permissive'} 2023-10-06T05:02:12.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:02:12.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:02:12.112 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:02:12.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:02:12.183 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T05:02:12.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2023-10-06T05:02:12.253 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2023-10-06T05:02:12.323 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T05:02:12.332 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-06T05:02:12.337 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-06T05:02:12.487 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T05:02:12.502 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T05:02:12.503 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi062.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T05:12:51.782 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com')] 2023-10-06T05:12:51.783 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T05:12:51.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:12:51.872 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-10-06T05:12:51.987 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-06T05:12:51.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T05:12:51.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T05:12:52.073 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-10-06T05:12:52.192 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-10-06T05:12:52.192 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T05:12:52.202 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T05:12:52.202 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T05:12:52.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:12:52.207 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T05:12:52.207 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:12:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T05:12:52.308 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T05:12:52.313 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T05:12:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T05:12:52.405 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2023-10-06T05:12:52.434 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2023-10-06T05:12:52.434 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2023-10-06T05:12:52.463 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2023-10-06T05:12:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T05:12:52.496 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T05:12:52.502 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T05:12:52.529 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T05:12:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2023-10-06T05:12:52.597 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:12:52.597 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2023-10-06T05:12:52.597 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:12:52.598 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:12:52.598 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:12:52.598 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:12:52.619 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2023-10-06T05:12:52.626 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:12:52.627 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2023-10-06T05:12:52.627 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:12:52.627 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:12:52.627 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:12:52.627 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:12:52.628 INFO:teuthology.run_tasks:Running task install... 2023-10-06T05:12:52.637 DEBUG:teuthology.task.install:project ceph 2023-10-06T05:12:52.637 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}} 2023-10-06T05:12:52.637 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'} 2023-10-06T05:12:52.637 INFO:teuthology.task.install:Using flavor: default 2023-10-06T05:12:52.643 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T05:12:52.643 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T05:12:52.643 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T05:12:52.644 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:12:52.646 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T05:12:52.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:12:52.786 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T05:12:52.786 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T05:12:52.794 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T05:12:52.795 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T05:12:52.894 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T05:12:52.894 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:12:52.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T05:12:52.961 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T05:12:52.962 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:12:52.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T05:12:52.965 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T05:12:52.965 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T05:12:53.032 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T05:12:53.033 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T05:12:53.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T05:12:53.145 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T05:12:53.176 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T05:12:53.242 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T05:12:53.246 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2023-10-06T05:12:53.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2023-10-06T05:12:53.312 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2023-10-06T05:12:53.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2023-10-06T05:12:53.623 INFO:teuthology.orchestra.run.smithi031.stdout:76 files removed 2023-10-06T05:12:53.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T05:12:53.693 INFO:teuthology.orchestra.run.smithi062.stdout:76 files removed 2023-10-06T05:12:53.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T05:12:54.492 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 299 kB/s | 78 kB 00:00 2023-10-06T05:12:54.717 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 193 kB/s | 78 kB 00:00 2023-10-06T05:12:54.768 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 39 kB/s | 9.2 kB 00:00 2023-10-06T05:12:55.035 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 33 kB/s | 9.2 kB 00:00 2023-10-06T05:12:55.052 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 5.6 kB/s | 1.4 kB 00:00 2023-10-06T05:12:55.385 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 4.5 kB/s | 1.4 kB 00:00 2023-10-06T05:12:56.719 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 8 - AppStream 20 MB/s | 33 MB 00:01 2023-10-06T05:12:58.336 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - AppStream 11 MB/s | 33 MB 00:02 2023-10-06T05:13:16.052 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 8 - BaseOS 3.9 MB/s | 48 MB 00:12 2023-10-06T05:13:24.076 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 8 - Extras 54 kB/s | 18 kB 00:00 2023-10-06T05:13:24.695 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 8 - Extras common packages 16 kB/s | 6.9 kB 00:00 2023-10-06T05:13:25.886 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 8 - PowerTools 6.3 MB/s | 6.5 MB 00:01 2023-10-06T05:13:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-10-06T05:13:28.405 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-10-06T05:13:31.226 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - BaseOS 1.8 MB/s | 48 MB 00:25 2023-10-06T05:13:33.122 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 364 kB/s | 24 kB 00:00 2023-10-06T05:13:37.797 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:13:37.797 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:13:37.811 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-10-06T05:13:37.811 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-10-06T05:13:37.812 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-10-06T05:13:37.813 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-10-06T05:13:37.814 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T05:13:37.816 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T05:13:37.819 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T05:13:37.820 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T05:13:38.058 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-10-06T05:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout:==================================================================================================== 2023-10-06T05:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2023-10-06T05:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout:==================================================================================================== 2023-10-06T05:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2023-10-06T05:13:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-10-06T05:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T05:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T05:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T05:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T05:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T05:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T05:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T05:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T05:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T05:13:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T05:13:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-10-06T05:13:38.076 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-10-06T05:13:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T05:13:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T05:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T05:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T05:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-06T05:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T05:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-06T05:13:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T05:13:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-10-06T05:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-10-06T05:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T05:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-10-06T05:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-10-06T05:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-06T05:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-06T05:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-06T05:13:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-06T05:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-10-06T05:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-10-06T05:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-10-06T05:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T05:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-10-06T05:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T05:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T05:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T05:13:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T05:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T05:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T05:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-06T05:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T05:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T05:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-06T05:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T05:13:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T05:13:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-06T05:13:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T05:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-06T05:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T05:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T05:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-06T05:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T05:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T05:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T05:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T05:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T05:13:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout:==================================================================================================== 2023-10-06T05:13:38.087 INFO:teuthology.orchestra.run.smithi031.stdout:Install 123 Packages 2023-10-06T05:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 16 Packages 2023-10-06T05:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 209 M 2023-10-06T05:13:38.088 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2023-10-06T05:13:38.432 INFO:teuthology.orchestra.run.smithi031.stdout:(1/139): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2023-10-06T05:13:38.873 INFO:teuthology.orchestra.run.smithi031.stdout:(2/139): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 8.2 MB/s | 5.2 MB 00:00 2023-10-06T05:13:38.932 INFO:teuthology.orchestra.run.smithi031.stdout:(3/139): ceph-immutable-object-cache-17.2.6-127 2.7 MB/s | 162 kB 00:00 2023-10-06T05:13:39.073 INFO:teuthology.orchestra.run.smithi031.stdout:(4/139): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.3 MB/s | 877 kB 00:00 2023-10-06T05:13:39.191 INFO:teuthology.orchestra.run.smithi031.stdout:(5/139): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 8.9 MB/s | 2.3 MB 00:00 2023-10-06T05:13:39.217 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - Extras 54 kB/s | 18 kB 00:00 2023-10-06T05:13:39.591 INFO:teuthology.orchestra.run.smithi031.stdout:(6/139): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 9.8 MB/s | 3.9 MB 00:00 2023-10-06T05:13:39.993 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - Extras common packages 12 kB/s | 6.9 kB 00:00 2023-10-06T05:13:40.034 INFO:teuthology.orchestra.run.smithi031.stdout:(7/139): ceph-common-17.2.6-1274.g68e53a4a.el8. 13 MB/s | 23 MB 00:01 2023-10-06T05:13:40.099 INFO:teuthology.orchestra.run.smithi031.stdout:(8/139): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 1.5 MB/s | 1.6 MB 00:01 2023-10-06T05:13:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:(9/139): ceph-selinux-17.2.6-1274.g68e53a4a.el8 362 kB/s | 24 kB 00:00 2023-10-06T05:13:41.192 INFO:teuthology.orchestra.run.smithi031.stdout:(10/139): ceph-osd-17.2.6-1274.g68e53a4a.el8.x8 10 MB/s | 17 MB 00:01 2023-10-06T05:13:41.251 INFO:teuthology.orchestra.run.smithi031.stdout:(11/139): cephfs-mirror-17.2.6-1274.g68e53a4a.e 4.3 MB/s | 255 kB 00:00 2023-10-06T05:13:41.286 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - PowerTools 5.7 MB/s | 6.5 MB 00:01 2023-10-06T05:13:41.468 INFO:teuthology.orchestra.run.smithi031.stdout:(12/139): ceph-radosgw-17.2.6-1274.g68e53a4a.el 12 MB/s | 17 MB 00:01 2023-10-06T05:13:41.493 INFO:teuthology.orchestra.run.smithi031.stdout:(13/139): libcephfs-devel-17.2.6-1274.g68e53a4a 103 kB/s | 25 kB 00:00 2023-10-06T05:13:41.585 INFO:teuthology.orchestra.run.smithi031.stdout:(14/139): libcephfs2-17.2.6-1274.g68e53a4a.el8. 6.7 MB/s | 797 kB 00:00 2023-10-06T05:13:41.627 INFO:teuthology.orchestra.run.smithi031.stdout:(15/139): libcephsqlite-17.2.6-1274.g68e53a4a.e 1.3 MB/s | 175 kB 00:00 2023-10-06T05:13:41.669 INFO:teuthology.orchestra.run.smithi031.stdout:(16/139): librados-devel-17.2.6-1274.g68e53a4a. 1.5 MB/s | 130 kB 00:00 2023-10-06T05:13:41.719 INFO:teuthology.orchestra.run.smithi031.stdout:(17/139): libradosstriper1-17.2.6-1274.g68e53a4 5.4 MB/s | 506 kB 00:00 2023-10-06T05:13:41.803 INFO:teuthology.orchestra.run.smithi031.stdout:(18/139): python3-ceph-argparse-17.2.6-1274.g68 535 kB/s | 45 kB 00:00 2023-10-06T05:13:41.903 INFO:teuthology.orchestra.run.smithi031.stdout:(19/139): python3-ceph-common-17.2.6-1274.g68e5 1.1 MB/s | 115 kB 00:00 2023-10-06T05:13:42.012 INFO:teuthology.orchestra.run.smithi031.stdout:(20/139): python3-cephfs-17.2.6-1274.g68e53a4a. 1.7 MB/s | 190 kB 00:00 2023-10-06T05:13:42.162 INFO:teuthology.orchestra.run.smithi031.stdout:(21/139): librgw2-17.2.6-1274.g68e53a4a.el8.x86 11 MB/s | 5.6 MB 00:00 2023-10-06T05:13:42.212 INFO:teuthology.orchestra.run.smithi031.stdout:(22/139): python3-rados-17.2.6-1274.g68e53a4a.e 1.8 MB/s | 371 kB 00:00 2023-10-06T05:13:42.271 INFO:teuthology.orchestra.run.smithi031.stdout:(23/139): python3-rbd-17.2.6-1274.g68e53a4a.el8 3.2 MB/s | 358 kB 00:00 2023-10-06T05:13:42.321 INFO:teuthology.orchestra.run.smithi031.stdout:(24/139): python3-rgw-17.2.6-1274.g68e53a4a.el8 983 kB/s | 107 kB 00:00 2023-10-06T05:13:42.363 INFO:teuthology.orchestra.run.smithi031.stdout:(25/139): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 993 kB/s | 91 kB 00:00 2023-10-06T05:13:42.455 INFO:teuthology.orchestra.run.smithi031.stdout:(26/139): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 1.8 MB/s | 173 kB 00:00 2023-10-06T05:13:42.547 INFO:teuthology.orchestra.run.smithi031.stdout:(27/139): ceph-grafana-dashboards-17.2.6-1274.g 252 kB/s | 23 kB 00:00 2023-10-06T05:13:42.664 INFO:teuthology.orchestra.run.smithi031.stdout:(28/139): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 10 MB/s | 3.5 MB 00:00 2023-10-06T05:13:42.689 INFO:teuthology.orchestra.run.smithi031.stdout:(29/139): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 855 kB/s | 121 kB 00:00 2023-10-06T05:13:42.829 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-10-06T05:13:42.923 INFO:teuthology.orchestra.run.smithi031.stdout:(30/139): ceph-mgr-dashboard-17.2.6-1274.g68e53 6.6 MB/s | 1.7 MB 00:00 2023-10-06T05:13:43.098 INFO:teuthology.orchestra.run.smithi031.stdout:(31/139): ceph-mgr-modules-core-17.2.6-1274.g68 1.3 MB/s | 238 kB 00:00 2023-10-06T05:13:43.182 INFO:teuthology.orchestra.run.smithi031.stdout:(32/139): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 568 kB/s | 47 kB 00:00 2023-10-06T05:13:43.249 INFO:teuthology.orchestra.run.smithi031.stdout:(33/139): ceph-prometheus-alerts-17.2.6-1274.g6 218 kB/s | 14 kB 00:00 2023-10-06T05:13:43.424 INFO:teuthology.orchestra.run.smithi031.stdout:(34/139): ceph-mgr-diskprediction-local-17.2.6- 10 MB/s | 7.4 MB 00:00 2023-10-06T05:13:43.449 INFO:teuthology.orchestra.run.smithi031.stdout:(35/139): ceph-volume-17.2.6-1274.g68e53a4a.el8 1.3 MB/s | 267 kB 00:00 2023-10-06T05:13:43.483 INFO:teuthology.orchestra.run.smithi031.stdout:(36/139): cephadm-17.2.6-1274.g68e53a4a.el8.noa 1.3 MB/s | 80 kB 00:00 2023-10-06T05:13:43.508 INFO:teuthology.orchestra.run.smithi031.stdout:(37/139): cephfs-top-17.2.6-1274.g68e53a4a.el8. 305 kB/s | 18 kB 00:00 2023-10-06T05:13:43.767 INFO:teuthology.orchestra.run.smithi031.stdout:(38/139): bison-3.0.4-10.el8.x86_64.rpm 2.6 MB/s | 688 kB 00:00 2023-10-06T05:13:43.800 INFO:teuthology.orchestra.run.smithi031.stdout:(39/139): automake-1.16.1-8.el8.noarch.rpm 2.3 MB/s | 740 kB 00:00 2023-10-06T05:13:43.826 INFO:teuthology.orchestra.run.smithi031.stdout:(40/139): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-10-06T05:13:43.859 INFO:teuthology.orchestra.run.smithi031.stdout:(41/139): jq-1.6-7.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-10-06T05:13:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:(42/139): librdkafka-0.11.4-3.el8.x86_64.rpm 5.2 MB/s | 354 kB 00:00 2023-10-06T05:13:43.960 INFO:teuthology.orchestra.run.smithi031.stdout:(43/139): libtool-2.4.6-25.el8.x86_64.rpm 6.9 MB/s | 709 kB 00:00 2023-10-06T05:13:43.985 INFO:teuthology.orchestra.run.smithi031.stdout:(44/139): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.8 MB/s | 259 kB 00:00 2023-10-06T05:13:44.019 INFO:teuthology.orchestra.run.smithi031.stdout:(45/139): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-10-06T05:13:44.228 INFO:teuthology.orchestra.run.smithi031.stdout:(46/139): openblas-0.3.15-4.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-10-06T05:13:44.303 INFO:teuthology.orchestra.run.smithi031.stdout:(47/139): openblas-threads-0.3.15-4.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-10-06T05:13:44.787 INFO:teuthology.orchestra.run.smithi031.stdout:(48/139): ceph-test-17.2.6-1274.g68e53a4a.el8.x 11 MB/s | 52 MB 00:04 2023-10-06T05:13:44.821 INFO:teuthology.orchestra.run.smithi031.stdout:(49/139): python3-jinja2-2.10.1-3.el8.noarch.rp 1.0 MB/s | 538 kB 00:00 2023-10-06T05:13:44.904 INFO:teuthology.orchestra.run.smithi031.stdout:(50/139): python3-babel-2.5.1-7.el8.noarch.rpm 7.1 MB/s | 4.8 MB 00:00 2023-10-06T05:13:44.929 INFO:teuthology.orchestra.run.smithi031.stdout:(51/139): python3-jsonpatch-1.21-2.el8.noarch.r 187 kB/s | 27 kB 00:00 2023-10-06T05:13:44.954 INFO:teuthology.orchestra.run.smithi031.stdout:(52/139): python3-jsonpointer-1.10-11.el8.noarc 149 kB/s | 20 kB 00:00 2023-10-06T05:13:44.979 INFO:teuthology.orchestra.run.smithi031.stdout:(53/139): python3-mako-1.0.6-14.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2023-10-06T05:13:45.013 INFO:teuthology.orchestra.run.smithi031.stdout:(54/139): python3-markupsafe-0.23-19.el8.x86_64 669 kB/s | 39 kB 00:00 2023-10-06T05:13:45.013 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 7.9 MB/s | 16 MB 00:02 2023-10-06T05:13:45.039 INFO:teuthology.orchestra.run.smithi031.stdout:(55/139): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.7 MB/s | 225 kB 00:00 2023-10-06T05:13:45.072 INFO:teuthology.orchestra.run.smithi031.stdout:(56/139): python3-prettytable-0.7.2-14.el8.noar 749 kB/s | 44 kB 00:00 2023-10-06T05:13:45.097 INFO:teuthology.orchestra.run.smithi031.stdout:(57/139): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.7 MB/s | 103 kB 00:00 2023-10-06T05:13:45.131 INFO:teuthology.orchestra.run.smithi031.stdout:(58/139): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.1 MB/s | 126 kB 00:00 2023-10-06T05:13:45.156 INFO:teuthology.orchestra.run.smithi031.stdout:(59/139): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-10-06T05:13:45.213 INFO:teuthology.orchestra.run.smithi031.stdout:(60/139): python3-pytz-2017.2-11.el8.noarch.rpm 702 kB/s | 57 kB 00:00 2023-10-06T05:13:45.315 INFO:teuthology.orchestra.run.smithi031.stdout:(61/139): python3-werkzeug-0.12.2-4.el8.noarch. 4.4 MB/s | 457 kB 00:00 2023-10-06T05:13:45.424 INFO:teuthology.orchestra.run.smithi031.stdout:(62/139): python3-numpy-1.14.3-10.el8.x86_64.rp 8.0 MB/s | 3.7 MB 00:00 2023-10-06T05:13:45.449 INFO:teuthology.orchestra.run.smithi031.stdout:(63/139): socat-1.7.4.1-1.el8.x86_64.rpm 2.4 MB/s | 323 kB 00:00 2023-10-06T05:13:45.508 INFO:teuthology.orchestra.run.smithi031.stdout:(64/139): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2023-10-06T05:13:45.742 INFO:teuthology.orchestra.run.smithi031.stdout:(65/139): python3-scipy-1.0.0-21.module_el8.5.0 23 MB/s | 14 MB 00:00 2023-10-06T05:13:46.134 INFO:teuthology.orchestra.run.smithi031.stdout:(66/139): elfutils-libelf-devel-0.189-3.el8.x86 90 kB/s | 62 kB 00:00 2023-10-06T05:13:46.292 INFO:teuthology.orchestra.run.smithi031.stdout:(67/139): keyutils-libs-devel-1.5.10-9.el8.x86_ 87 kB/s | 48 kB 00:00 2023-10-06T05:13:46.409 INFO:teuthology.orchestra.run.smithi031.stdout:(68/139): fuse-2.9.7-17.el8.x86_64.rpm 92 kB/s | 83 kB 00:00 2023-10-06T05:13:46.701 INFO:teuthology.orchestra.run.smithi031.stdout:(69/139): libaio-devel-0.3.112-1.el8.x86_64.rpm 46 kB/s | 19 kB 00:00 2023-10-06T05:13:46.926 INFO:teuthology.orchestra.run.smithi031.stdout:(70/139): libcom_err-devel-1.45.6-5.el8.x86_64. 75 kB/s | 39 kB 00:00 2023-10-06T05:13:47.626 INFO:teuthology.orchestra.run.smithi031.stdout:(71/139): krb5-devel-1.18.2-22.el8.x86_64.rpm 376 kB/s | 561 kB 00:01 2023-10-06T05:13:47.960 INFO:teuthology.orchestra.run.smithi031.stdout:(72/139): libquadmath-8.5.0-20.el8.x86_64.rpm 167 kB/s | 172 kB 00:01 2023-10-06T05:13:48.185 INFO:teuthology.orchestra.run.smithi031.stdout:(73/139): librabbitmq-0.9.0-4.el8.x86_64.rpm 84 kB/s | 47 kB 00:00 2023-10-06T05:13:48.294 INFO:teuthology.orchestra.run.smithi031.stdout:(74/139): libgfortran-8.5.0-20.el8.x86_64.rpm 405 kB/s | 645 kB 00:01 2023-10-06T05:13:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:(75/139): libsepol-devel-2.9-3.el8.x86_64.rpm 105 kB/s | 87 kB 00:00 2023-10-06T05:13:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:(76/139): libselinux-devel-2.9-8.el8.x86_64.rpm 182 kB/s | 200 kB 00:01 2023-10-06T05:13:49.253 INFO:teuthology.orchestra.run.smithi031.stdout:(77/139): libuuid-devel-2.32.1-43.el8.x86_64.rp 104 kB/s | 100 kB 00:00 2023-10-06T05:13:49.411 INFO:teuthology.orchestra.run.smithi031.stdout:(78/139): libverto-devel-0.3.2-2.el8.x86_64.rpm 45 kB/s | 18 kB 00:00 2023-10-06T05:13:49.612 INFO:teuthology.orchestra.run.smithi031.stdout:(79/139): libzstd-devel-1.4.4-1.el8.x86_64.rpm 80 kB/s | 44 kB 00:00 2023-10-06T05:13:49.738 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 426 kB/s | 24 kB 00:00 2023-10-06T05:13:50.778 INFO:teuthology.orchestra.run.smithi031.stdout:(80/139): pcre2-utf16-10.32-3.el8.x86_64.rpm 196 kB/s | 229 kB 00:01 2023-10-06T05:13:50.904 INFO:teuthology.orchestra.run.smithi031.stdout:(81/139): pcre2-devel-10.32-3.el8.x86_64.rpm 405 kB/s | 605 kB 00:01 2023-10-06T05:13:51.837 INFO:teuthology.orchestra.run.smithi031.stdout:(82/139): pcre2-utf32-10.32-3.el8.x86_64.rpm 208 kB/s | 220 kB 00:01 2023-10-06T05:13:52.104 INFO:teuthology.orchestra.run.smithi031.stdout:(83/139): python3-cffi-1.11.5-6.el8.x86_64.rpm 207 kB/s | 249 kB 00:01 2023-10-06T05:13:52.563 INFO:teuthology.orchestra.run.smithi031.stdout:(84/139): openssl-devel-1.1.1k-9.el8.x86_64.rpm 983 kB/s | 3.2 MB 00:03 2023-10-06T05:13:52.630 INFO:teuthology.orchestra.run.smithi031.stdout:(85/139): python3-jwt-1.6.1-2.el8.noarch.rpm 81 kB/s | 43 kB 00:00 2023-10-06T05:13:53.430 INFO:teuthology.orchestra.run.smithi031.stdout:(86/139): python3-cryptography-3.2.1-6.el8.x86_ 403 kB/s | 641 kB 00:01 2023-10-06T05:13:53.447 INFO:teuthology.orchestra.run.smithi031.stdout:(87/139): python3-pycparser-2.14-14.el8.noarch. 133 kB/s | 109 kB 00:00 2023-10-06T05:13:53.472 INFO:teuthology.orchestra.run.smithi031.stdout:(88/139): python3-oauthlib-2.1.0-1.el8.noarch.r 170 kB/s | 155 kB 00:00 2023-10-06T05:13:53.956 INFO:teuthology.orchestra.run.smithi031.stdout:(89/139): python3-requests-oauthlib-1.0.0-1.el8 81 kB/s | 43 kB 00:00 2023-10-06T05:13:54.031 INFO:teuthology.orchestra.run.smithi031.stdout:(90/139): gperftools-libs-2.7-9.el8.x86_64.rpm 4.0 MB/s | 306 kB 00:00 2023-10-06T05:13:54.065 INFO:teuthology.orchestra.run.smithi031.stdout:(91/139): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2023-10-06T05:13:54.098 INFO:teuthology.orchestra.run.smithi031.stdout:(92/139): libunwind-1.3.1-3.el8.x86_64.rpm 2.2 MB/s | 75 kB 00:00 2023-10-06T05:13:54.123 INFO:teuthology.orchestra.run.smithi031.stdout:(93/139): zlib-devel-1.2.11-21.el8.x86_64.rpm 89 kB/s | 58 kB 00:00 2023-10-06T05:13:54.157 INFO:teuthology.orchestra.run.smithi031.stdout:(94/139): python3-asyncssh-2.7.0-2.el8.noarch.r 8.0 MB/s | 476 kB 00:00 2023-10-06T05:13:54.182 INFO:teuthology.orchestra.run.smithi031.stdout:(95/139): python3-beautifulsoup4-4.6.3-2.el8.1. 7.3 MB/s | 185 kB 00:00 2023-10-06T05:13:54.208 INFO:teuthology.orchestra.run.smithi031.stdout:(96/139): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 522 kB/s | 44 kB 00:00 2023-10-06T05:13:54.233 INFO:teuthology.orchestra.run.smithi031.stdout:(97/139): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 95 kB/s | 74 kB 00:00 2023-10-06T05:13:54.258 INFO:teuthology.orchestra.run.smithi031.stdout:(98/139): python3-cachetools-3.1.1-4.el8.noarch 438 kB/s | 33 kB 00:00 2023-10-06T05:13:54.283 INFO:teuthology.orchestra.run.smithi031.stdout:(99/139): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2023-10-06T05:13:54.325 INFO:teuthology.orchestra.run.smithi031.stdout:(100/139): python3-cherrypy-18.4.0-1.el8.noarch 9.1 MB/s | 384 kB 00:00 2023-10-06T05:13:54.351 INFO:teuthology.orchestra.run.smithi031.stdout:(101/139): python3-google-auth-1.1.1-10.el8.noa 1.2 MB/s | 81 kB 00:00 2023-10-06T05:13:54.375 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:13:54.376 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T05:13:54.376 INFO:teuthology.orchestra.run.smithi031.stdout:(102/139): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-10-06T05:13:54.390 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-10-06T05:13:54.390 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-10-06T05:13:54.391 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-10-06T05:13:54.392 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-10-06T05:13:54.393 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T05:13:54.395 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T05:13:54.398 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T05:13:54.399 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T05:13:54.422 INFO:teuthology.orchestra.run.smithi031.stdout:(103/139): python3-jaraco-functools-2.0-4.el8.n 249 kB/s | 18 kB 00:00 2023-10-06T05:13:54.452 INFO:teuthology.orchestra.run.smithi031.stdout:(104/139): python3-cheroot-8.5.2-1.el8.noarch.r 791 kB/s | 173 kB 00:00 2023-10-06T05:13:54.477 INFO:teuthology.orchestra.run.smithi031.stdout:(105/139): python3-logutils-0.3.5-11.el8.noarch 896 kB/s | 49 kB 00:00 2023-10-06T05:13:54.502 INFO:teuthology.orchestra.run.smithi031.stdout:(106/139): python3-more-itertools-7.2.0-3.el8.n 1.1 MB/s | 59 kB 00:00 2023-10-06T05:13:54.527 INFO:teuthology.orchestra.run.smithi031.stdout:(107/139): python3-natsort-7.1.1-2.el8.noarch.r 1.2 MB/s | 60 kB 00:00 2023-10-06T05:13:54.553 INFO:teuthology.orchestra.run.smithi031.stdout:(108/139): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-10-06T05:13:54.578 INFO:teuthology.orchestra.run.smithi031.stdout:(109/139): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-10-06T05:13:54.603 INFO:teuthology.orchestra.run.smithi031.stdout:(110/139): python3-repoze-lru-0.7-6.el8.noarch. 672 kB/s | 34 kB 00:00 2023-10-06T05:13:54.638 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-10-06T05:13:54.645 INFO:teuthology.orchestra.run.smithi031.stdout:(111/139): python3-routes-2.4.1-12.el8.noarch.r 2.9 MB/s | 196 kB 00:00 2023-10-06T05:13:54.653 INFO:teuthology.orchestra.run.smithi062.stdout:==================================================================================================== 2023-10-06T05:13:54.653 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout:==================================================================================================== 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T05:13:54.654 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-10-06T05:13:54.655 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:13:54.656 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-10-06T05:13:54.657 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T05:13:54.658 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-10-06T05:13:54.659 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T05:13:54.660 INFO:teuthology.orchestra.run.smithi062.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-10-06T05:13:54.661 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T05:13:54.662 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T05:13:54.663 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T05:13:54.664 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T05:13:54.665 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout:==================================================================================================== 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout:Install 123 Packages 2023-10-06T05:13:54.666 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 16 Packages 2023-10-06T05:13:54.667 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:13:54.667 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 209 M 2023-10-06T05:13:54.667 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-10-06T05:13:54.670 INFO:teuthology.orchestra.run.smithi031.stdout:(112/139): python3-rsa-4.9-2.el8.noarch.rpm 962 kB/s | 64 kB 00:00 2023-10-06T05:13:54.696 INFO:teuthology.orchestra.run.smithi031.stdout:(113/139): python3-simplegeneric-0.8.1-17.el8.n 373 kB/s | 19 kB 00:00 2023-10-06T05:13:54.721 INFO:teuthology.orchestra.run.smithi031.stdout:(114/139): python3-singledispatch-3.4.0.3-18.el 475 kB/s | 24 kB 00:00 2023-10-06T05:13:54.746 INFO:teuthology.orchestra.run.smithi031.stdout:(115/139): python3-tempora-1.14.1-5.el8.noarch. 583 kB/s | 29 kB 00:00 2023-10-06T05:13:54.772 INFO:teuthology.orchestra.run.smithi031.stdout:(116/139): python3-trustme-0.6.0-4.el8.noarch.r 552 kB/s | 27 kB 00:00 2023-10-06T05:13:54.805 INFO:teuthology.orchestra.run.smithi031.stdout:(117/139): python3-kubernetes-11.0.0-6.el8.noar 2.7 MB/s | 1.2 MB 00:00 2023-10-06T05:13:54.831 INFO:teuthology.orchestra.run.smithi031.stdout:(118/139): python3-waitress-1.4.3-1.el8.noarch. 3.0 MB/s | 255 kB 00:00 2023-10-06T05:13:54.856 INFO:teuthology.orchestra.run.smithi031.stdout:(119/139): python3-websocket-client-0.56.0-5.el 1.2 MB/s | 61 kB 00:00 2023-10-06T05:13:54.873 INFO:teuthology.orchestra.run.smithi031.stdout:(120/139): python3-webob-1.8.5-1.el8.1.noarch.r 2.4 MB/s | 251 kB 00:00 2023-10-06T05:13:54.898 INFO:teuthology.orchestra.run.smithi031.stdout:(121/139): python3-webtest-2.0.33-1.el8.noarch. 1.2 MB/s | 85 kB 00:00 2023-10-06T05:13:54.915 INFO:teuthology.orchestra.run.smithi031.stdout:(122/139): python3-zc-lockfile-2.0-2.el8.noarch 391 kB/s | 23 kB 00:00 2023-10-06T05:13:54.965 INFO:teuthology.orchestra.run.smithi031.stdout:(123/139): thrift-0.13.0-2.el8.x86_64.rpm 19 MB/s | 1.7 MB 00:00 2023-10-06T05:13:55.307 INFO:teuthology.orchestra.run.smithi031.stdout:(124/139): librados2-17.2.6-1274.g68e53a4a.el8. 8.3 MB/s | 3.4 MB 00:00 2023-10-06T05:13:55.465 INFO:teuthology.orchestra.run.smithi062.stdout:(1/139): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 32 kB/s | 6.4 kB 00:00 2023-10-06T05:13:55.566 INFO:teuthology.orchestra.run.smithi031.stdout:(125/139): librbd1-17.2.6-1274.g68e53a4a.el8.x8 5.5 MB/s | 3.5 MB 00:00 2023-10-06T05:13:55.883 INFO:teuthology.orchestra.run.smithi031.stdout:(126/139): NetworkManager-initscripts-updown-1. 156 kB/s | 143 kB 00:00 2023-10-06T05:13:55.965 INFO:teuthology.orchestra.run.smithi062.stdout:(2/139): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.7 MB/s | 877 kB 00:00 2023-10-06T05:13:55.966 INFO:teuthology.orchestra.run.smithi031.stdout:(127/139): elfutils-debuginfod-client-0.189-3.e 115 kB/s | 76 kB 00:00 2023-10-06T05:13:56.107 INFO:teuthology.orchestra.run.smithi062.stdout:(3/139): ceph-immutable-object-cache-17.2.6-127 1.1 MB/s | 162 kB 00:00 2023-10-06T05:13:56.224 INFO:teuthology.orchestra.run.smithi062.stdout:(4/139): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 5.4 MB/s | 5.2 MB 00:00 2023-10-06T05:13:56.474 INFO:teuthology.orchestra.run.smithi062.stdout:(5/139): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 6.3 MB/s | 1.6 MB 00:00 2023-10-06T05:13:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:(128/139): elfutils-libelf-0.189-3.el8.x86_64.r 197 kB/s | 232 kB 00:01 2023-10-06T05:13:56.933 INFO:teuthology.orchestra.run.smithi062.stdout:(6/139): ceph-common-17.2.6-1274.g68e53a4a.el8. 14 MB/s | 23 MB 00:01 2023-10-06T05:13:57.009 INFO:teuthology.orchestra.run.smithi062.stdout:(7/139): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 7.4 MB/s | 3.9 MB 00:00 2023-10-06T05:13:57.117 INFO:teuthology.orchestra.run.smithi062.stdout:(8/139): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 2.3 MB/s | 2.3 MB 00:01 2023-10-06T05:13:57.125 INFO:teuthology.orchestra.run.smithi031.stdout:(129/139): elfutils-libs-0.189-3.el8.x86_64.rpm 244 kB/s | 303 kB 00:01 2023-10-06T05:13:57.209 INFO:teuthology.orchestra.run.smithi062.stdout:(9/139): ceph-selinux-17.2.6-1274.g68e53a4a.el8 266 kB/s | 24 kB 00:00 2023-10-06T05:13:57.817 INFO:teuthology.orchestra.run.smithi031.stdout:(130/139): iproute-6.2.0-5.el8.x86_64.rpm 461 kB/s | 854 kB 00:01 2023-10-06T05:13:57.842 INFO:teuthology.orchestra.run.smithi031.stdout:(131/139): libblkid-2.32.1-43.el8.x86_64.rpm 200 kB/s | 220 kB 00:01 2023-10-06T05:13:58.068 INFO:teuthology.orchestra.run.smithi062.stdout:(10/139): ceph-radosgw-17.2.6-1274.g68e53a4a.el 16 MB/s | 17 MB 00:01 2023-10-06T05:13:58.126 INFO:teuthology.orchestra.run.smithi062.stdout:(11/139): cephfs-mirror-17.2.6-1274.g68e53a4a.e 4.3 MB/s | 255 kB 00:00 2023-10-06T05:13:58.193 INFO:teuthology.orchestra.run.smithi062.stdout:(12/139): libcephfs-devel-17.2.6-1274.g68e53a4a 373 kB/s | 25 kB 00:00 2023-10-06T05:13:58.302 INFO:teuthology.orchestra.run.smithi062.stdout:(13/139): libcephfs2-17.2.6-1274.g68e53a4a.el8. 7.2 MB/s | 797 kB 00:00 2023-10-06T05:13:58.334 INFO:teuthology.orchestra.run.smithi031.stdout:(132/139): libfdisk-2.32.1-43.el8.x86_64.rpm 210 kB/s | 254 kB 00:01 2023-10-06T05:13:58.402 INFO:teuthology.orchestra.run.smithi062.stdout:(14/139): libcephsqlite-17.2.6-1274.g68e53a4a.e 1.7 MB/s | 175 kB 00:00 2023-10-06T05:13:58.478 INFO:teuthology.orchestra.run.smithi062.stdout:(15/139): librados-devel-17.2.6-1274.g68e53a4a. 1.7 MB/s | 130 kB 00:00 2023-10-06T05:13:58.569 INFO:teuthology.orchestra.run.smithi062.stdout:(16/139): libradosstriper1-17.2.6-1274.g68e53a4 5.4 MB/s | 506 kB 00:00 2023-10-06T05:13:58.743 INFO:teuthology.orchestra.run.smithi031.stdout:(133/139): libsmartcols-2.32.1-43.el8.x86_64.rp 199 kB/s | 179 kB 00:00 2023-10-06T05:13:58.903 INFO:teuthology.orchestra.run.smithi062.stdout:(17/139): ceph-osd-17.2.6-1274.g68e53a4a.el8.x8 8.5 MB/s | 17 MB 00:01 2023-10-06T05:13:58.945 INFO:teuthology.orchestra.run.smithi062.stdout:(18/139): python3-ceph-argparse-17.2.6-1274.g68 1.1 MB/s | 45 kB 00:00 2023-10-06T05:13:58.951 INFO:teuthology.orchestra.run.smithi031.stdout:(134/139): libmount-2.32.1-43.el8.x86_64.rpm 209 kB/s | 236 kB 00:01 2023-10-06T05:13:59.037 INFO:teuthology.orchestra.run.smithi062.stdout:(19/139): librgw2-17.2.6-1274.g68e53a4a.el8.x86 12 MB/s | 5.6 MB 00:00 2023-10-06T05:13:59.062 INFO:teuthology.orchestra.run.smithi062.stdout:(20/139): python3-ceph-common-17.2.6-1274.g68e5 989 kB/s | 115 kB 00:00 2023-10-06T05:13:59.104 INFO:teuthology.orchestra.run.smithi062.stdout:(21/139): python3-cephfs-17.2.6-1274.g68e53a4a. 2.8 MB/s | 190 kB 00:00 2023-10-06T05:13:59.129 INFO:teuthology.orchestra.run.smithi062.stdout:(22/139): python3-rados-17.2.6-1274.g68e53a4a.e 5.4 MB/s | 371 kB 00:00 2023-10-06T05:13:59.152 INFO:teuthology.orchestra.run.smithi031.stdout:(135/139): libuuid-2.32.1-43.el8.x86_64.rpm 121 kB/s | 99 kB 00:00 2023-10-06T05:13:59.171 INFO:teuthology.orchestra.run.smithi062.stdout:(23/139): python3-rbd-17.2.6-1274.g68e53a4a.el8 5.2 MB/s | 358 kB 00:00 2023-10-06T05:13:59.196 INFO:teuthology.orchestra.run.smithi062.stdout:(24/139): python3-rgw-17.2.6-1274.g68e53a4a.el8 1.6 MB/s | 107 kB 00:00 2023-10-06T05:13:59.230 INFO:teuthology.orchestra.run.smithi062.stdout:(25/139): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.5 MB/s | 91 kB 00:00 2023-10-06T05:13:59.305 INFO:teuthology.orchestra.run.smithi062.stdout:(26/139): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 2.3 MB/s | 173 kB 00:00 2023-10-06T05:13:59.380 INFO:teuthology.orchestra.run.smithi062.stdout:(27/139): ceph-grafana-dashboards-17.2.6-1274.g 308 kB/s | 23 kB 00:00 2023-10-06T05:13:59.439 INFO:teuthology.orchestra.run.smithi062.stdout:(28/139): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 14 MB/s | 3.5 MB 00:00 2023-10-06T05:13:59.464 INFO:teuthology.orchestra.run.smithi062.stdout:(29/139): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 1.4 MB/s | 121 kB 00:00 2023-10-06T05:13:59.631 INFO:teuthology.orchestra.run.smithi062.stdout:(30/139): ceph-mgr-dashboard-17.2.6-1274.g68e53 8.8 MB/s | 1.7 MB 00:00 2023-10-06T05:13:59.698 INFO:teuthology.orchestra.run.smithi062.stdout:(31/139): ceph-mgr-modules-core-17.2.6-1274.g68 3.5 MB/s | 238 kB 00:00 2023-10-06T05:13:59.752 INFO:teuthology.orchestra.run.smithi031.stdout:(136/139): util-linux-user-2.32.1-43.el8.x86_64 128 kB/s | 103 kB 00:00 2023-10-06T05:13:59.765 INFO:teuthology.orchestra.run.smithi062.stdout:(32/139): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 709 kB/s | 47 kB 00:00 2023-10-06T05:13:59.824 INFO:teuthology.orchestra.run.smithi062.stdout:(33/139): ceph-prometheus-alerts-17.2.6-1274.g6 249 kB/s | 14 kB 00:00 2023-10-06T05:13:59.866 INFO:teuthology.orchestra.run.smithi062.stdout:(34/139): ceph-volume-17.2.6-1274.g68e53a4a.el8 6.2 MB/s | 267 kB 00:00 2023-10-06T05:13:59.992 INFO:teuthology.orchestra.run.smithi062.stdout:(35/139): ceph-mgr-diskprediction-local-17.2.6- 14 MB/s | 7.4 MB 00:00 2023-10-06T05:14:00.017 INFO:teuthology.orchestra.run.smithi062.stdout:(36/139): cephadm-17.2.6-1274.g68e53a4a.el8.noa 535 kB/s | 80 kB 00:00 2023-10-06T05:14:00.051 INFO:teuthology.orchestra.run.smithi062.stdout:(37/139): cephfs-top-17.2.6-1274.g68e53a4a.el8. 305 kB/s | 18 kB 00:00 2023-10-06T05:14:00.301 INFO:teuthology.orchestra.run.smithi062.stdout:(38/139): bison-3.0.4-10.el8.x86_64.rpm 2.7 MB/s | 688 kB 00:00 2023-10-06T05:14:00.327 INFO:teuthology.orchestra.run.smithi031.stdout:(137/139): xfsdump-3.1.8-6.el8.x86_64.rpm 284 kB/s | 334 kB 00:01 2023-10-06T05:14:00.335 INFO:teuthology.orchestra.run.smithi062.stdout:(39/139): automake-1.16.1-8.el8.noarch.rpm 2.3 MB/s | 740 kB 00:00 2023-10-06T05:14:00.360 INFO:teuthology.orchestra.run.smithi062.stdout:(40/139): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-10-06T05:14:00.393 INFO:teuthology.orchestra.run.smithi062.stdout:(41/139): jq-1.6-7.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-10-06T05:14:00.419 INFO:teuthology.orchestra.run.smithi062.stdout:(42/139): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-10-06T05:14:00.461 INFO:teuthology.orchestra.run.smithi062.stdout:(43/139): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-06T05:14:00.486 INFO:teuthology.orchestra.run.smithi062.stdout:(44/139): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-10-06T05:14:00.520 INFO:teuthology.orchestra.run.smithi062.stdout:(45/139): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-10-06T05:14:00.703 INFO:teuthology.orchestra.run.smithi062.stdout:(46/139): openblas-0.3.15-4.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-10-06T05:14:00.787 INFO:teuthology.orchestra.run.smithi062.stdout:(47/139): openblas-threads-0.3.15-4.el8.x86_64. 18 MB/s | 4.9 MB 00:00 2023-10-06T05:14:00.854 INFO:teuthology.orchestra.run.smithi062.stdout:(48/139): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-10-06T05:14:00.929 INFO:teuthology.orchestra.run.smithi062.stdout:(49/139): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-10-06T05:14:00.946 INFO:teuthology.orchestra.run.smithi062.stdout:(50/139): python3-jsonpatch-1.21-2.el8.noarch.r 289 kB/s | 27 kB 00:00 2023-10-06T05:14:00.980 INFO:teuthology.orchestra.run.smithi062.stdout:(51/139): python3-jsonpointer-1.10-11.el8.noarc 400 kB/s | 20 kB 00:00 2023-10-06T05:14:01.005 INFO:teuthology.orchestra.run.smithi062.stdout:(52/139): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-06T05:14:01.039 INFO:teuthology.orchestra.run.smithi062.stdout:(53/139): python3-markupsafe-0.23-19.el8.x86_64 659 kB/s | 39 kB 00:00 2023-10-06T05:14:01.089 INFO:teuthology.orchestra.run.smithi062.stdout:(54/139): python3-numpy-f2py-1.14.3-10.el8.x86_ 4.4 MB/s | 225 kB 00:00 2023-10-06T05:14:01.173 INFO:teuthology.orchestra.run.smithi062.stdout:(55/139): python3-prettytable-0.7.2-14.el8.noar 526 kB/s | 44 kB 00:00 2023-10-06T05:14:01.240 INFO:teuthology.orchestra.run.smithi062.stdout:(56/139): python3-numpy-1.14.3-10.el8.x86_64.rp 16 MB/s | 3.7 MB 00:00 2023-10-06T05:14:01.257 INFO:teuthology.orchestra.run.smithi062.stdout:(57/139): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.2 MB/s | 103 kB 00:00 2023-10-06T05:14:01.299 INFO:teuthology.orchestra.run.smithi062.stdout:(58/139): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.1 MB/s | 126 kB 00:00 2023-10-06T05:14:01.324 INFO:teuthology.orchestra.run.smithi062.stdout:(59/139): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-10-06T05:14:01.366 INFO:teuthology.orchestra.run.smithi062.stdout:(60/139): python3-pytz-2017.2-11.el8.noarch.rpm 848 kB/s | 57 kB 00:00 2023-10-06T05:14:01.450 INFO:teuthology.orchestra.run.smithi062.stdout:(61/139): python3-werkzeug-0.12.2-4.el8.noarch. 5.4 MB/s | 457 kB 00:00 2023-10-06T05:14:01.525 INFO:teuthology.orchestra.run.smithi062.stdout:(62/139): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2023-10-06T05:14:01.601 INFO:teuthology.orchestra.run.smithi062.stdout:(63/139): xmlstarlet-1.6.1-20.el8.x86_64.rpm 945 kB/s | 71 kB 00:00 2023-10-06T05:14:01.627 INFO:teuthology.orchestra.run.smithi031.stdout:(138/139): util-linux-2.32.1-43.el8.x86_64.rpm 880 kB/s | 2.5 MB 00:02 2023-10-06T05:14:01.661 INFO:teuthology.orchestra.run.smithi031.stdout:(139/139): xfsprogs-5.0.0-12.el8.x86_64.rpm 580 kB/s | 1.1 MB 00:01 2023-10-06T05:14:01.665 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2023-10-06T05:14:01.665 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.9 MB/s | 209 MB 00:23 2023-10-06T05:14:01.835 INFO:teuthology.orchestra.run.smithi062.stdout:(64/139): python3-scipy-1.0.0-21.module_el8.5.0 27 MB/s | 14 MB 00:00 2023-10-06T05:14:02.393 INFO:teuthology.orchestra.run.smithi062.stdout:(65/139): elfutils-libelf-devel-0.189-3.el8.x86 78 kB/s | 62 kB 00:00 2023-10-06T05:14:02.510 INFO:teuthology.orchestra.run.smithi062.stdout:(66/139): fuse-2.9.7-17.el8.x86_64.rpm 123 kB/s | 83 kB 00:00 2023-10-06T05:14:03.043 INFO:teuthology.orchestra.run.smithi062.stdout:(67/139): keyutils-libs-devel-1.5.10-9.el8.x86_ 74 kB/s | 48 kB 00:00 2023-10-06T05:14:03.412 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2023-10-06T05:14:03.452 INFO:teuthology.orchestra.run.smithi062.stdout:(68/139): libaio-devel-0.3.112-1.el8.x86_64.rpm 46 kB/s | 19 kB 00:00 2023-10-06T05:14:03.847 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2023-10-06T05:14:03.848 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2023-10-06T05:14:03.977 INFO:teuthology.orchestra.run.smithi062.stdout:(69/139): libcom_err-devel-1.45.6-5.el8.x86_64. 74 kB/s | 39 kB 00:00 2023-10-06T05:14:04.128 INFO:teuthology.orchestra.run.smithi062.stdout:(70/139): krb5-devel-1.18.2-22.el8.x86_64.rpm 347 kB/s | 561 kB 00:01 2023-10-06T05:14:05.036 INFO:teuthology.orchestra.run.smithi062.stdout:(71/139): libquadmath-8.5.0-20.el8.x86_64.rpm 190 kB/s | 172 kB 00:00 2023-10-06T05:14:05.344 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2023-10-06T05:14:05.348 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2023-10-06T05:14:05.528 INFO:teuthology.orchestra.run.smithi062.stdout:(72/139): libgfortran-8.5.0-20.el8.x86_64.rpm 416 kB/s | 645 kB 00:01 2023-10-06T05:14:05.570 INFO:teuthology.orchestra.run.smithi062.stdout:(73/139): librabbitmq-0.9.0-4.el8.x86_64.rpm 88 kB/s | 47 kB 00:00 2023-10-06T05:14:06.354 INFO:teuthology.orchestra.run.smithi062.stdout:(74/139): libsepol-devel-2.9-3.el8.x86_64.rpm 111 kB/s | 87 kB 00:00 2023-10-06T05:14:06.679 INFO:teuthology.orchestra.run.smithi062.stdout:(75/139): libselinux-devel-2.9-8.el8.x86_64.rpm 174 kB/s | 200 kB 00:01 2023-10-06T05:14:07.529 INFO:teuthology.orchestra.run.smithi062.stdout:(76/139): ceph-test-17.2.6-1274.g68e53a4a.el8.x 5.0 MB/s | 52 MB 00:10 2023-10-06T05:14:07.546 INFO:teuthology.orchestra.run.smithi062.stdout:(77/139): libverto-devel-0.3.2-2.el8.x86_64.rpm 21 kB/s | 18 kB 00:00 2023-10-06T05:14:07.571 INFO:teuthology.orchestra.run.smithi062.stdout:(78/139): libuuid-devel-2.32.1-43.el8.x86_64.rp 82 kB/s | 100 kB 00:01 2023-10-06T05:14:08.080 INFO:teuthology.orchestra.run.smithi062.stdout:(79/139): libzstd-devel-1.4.4-1.el8.x86_64.rpm 80 kB/s | 44 kB 00:00 2023-10-06T05:14:09.155 INFO:teuthology.orchestra.run.smithi062.stdout:(80/139): pcre2-utf16-10.32-3.el8.x86_64.rpm 213 kB/s | 229 kB 00:01 2023-10-06T05:14:09.189 INFO:teuthology.orchestra.run.smithi062.stdout:(81/139): pcre2-devel-10.32-3.el8.x86_64.rpm 374 kB/s | 605 kB 00:01 2023-10-06T05:14:10.222 INFO:teuthology.orchestra.run.smithi062.stdout:(82/139): python3-cffi-1.11.5-6.el8.x86_64.rpm 241 kB/s | 249 kB 00:01 2023-10-06T05:14:10.381 INFO:teuthology.orchestra.run.smithi062.stdout:(83/139): pcre2-utf32-10.32-3.el8.x86_64.rpm 180 kB/s | 220 kB 00:01 2023-10-06T05:14:10.731 INFO:teuthology.orchestra.run.smithi062.stdout:(84/139): openssl-devel-1.1.1k-9.el8.x86_64.rpm 1.0 MB/s | 3.2 MB 00:03 2023-10-06T05:14:10.737 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2023-10-06T05:14:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/1 2023-10-06T05:14:10.923 INFO:teuthology.orchestra.run.smithi062.stdout:(85/139): python3-jwt-1.6.1-2.el8.noarch.rpm 79 kB/s | 43 kB 00:00 2023-10-06T05:14:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T05:14:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T05:14:11.270 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T05:14:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T05:14:11.551 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/155 2023-10-06T05:14:11.673 INFO:teuthology.orchestra.run.smithi062.stdout:(86/139): python3-oauthlib-2.1.0-1.el8.noarch.r 164 kB/s | 155 kB 00:00 2023-10-06T05:14:11.698 INFO:teuthology.orchestra.run.smithi062.stdout:(87/139): python3-pycparser-2.14-14.el8.noarch. 141 kB/s | 109 kB 00:00 2023-10-06T05:14:11.865 INFO:teuthology.orchestra.run.smithi062.stdout:(88/139): python3-cryptography-3.2.1-6.el8.x86_ 390 kB/s | 641 kB 00:01 2023-10-06T05:14:11.933 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 4/155 2023-10-06T05:14:12.126 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/155 2023-10-06T05:14:12.191 INFO:teuthology.orchestra.run.smithi062.stdout:(89/139): python3-requests-oauthlib-1.0.0-1.el8 82 kB/s | 43 kB 00:00 2023-10-06T05:14:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/155 2023-10-06T05:14:12.349 INFO:teuthology.orchestra.run.smithi062.stdout:(90/139): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 114 kB/s | 74 kB 00:00 2023-10-06T05:14:12.520 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T05:14:12.525 INFO:teuthology.orchestra.run.smithi062.stdout:(91/139): zlib-devel-1.2.11-21.el8.x86_64.rpm 88 kB/s | 58 kB 00:00 2023-10-06T05:14:12.767 INFO:teuthology.orchestra.run.smithi062.stdout:(92/139): liboath-2.6.2-3.el8.x86_64.rpm 141 kB/s | 59 kB 00:00 2023-10-06T05:14:12.792 INFO:teuthology.orchestra.run.smithi062.stdout:(93/139): gperftools-libs-2.7-9.el8.x86_64.rpm 510 kB/s | 306 kB 00:00 2023-10-06T05:14:12.901 INFO:teuthology.orchestra.run.smithi062.stdout:(94/139): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 402 kB/s | 44 kB 00:00 2023-10-06T05:14:12.961 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T05:14:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:14:13.307 INFO:teuthology.orchestra.run.smithi062.stdout:(95/139): libunwind-1.3.1-3.el8.x86_64.rpm 96 kB/s | 75 kB 00:00 2023-10-06T05:14:13.335 INFO:teuthology.orchestra.run.smithi062.stdout:(96/139): python3-beautifulsoup4-4.6.3-2.el8.1. 427 kB/s | 185 kB 00:00 2023-10-06T05:14:13.435 INFO:teuthology.orchestra.run.smithi062.stdout:(97/139): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2023-10-06T05:14:13.460 INFO:teuthology.orchestra.run.smithi062.stdout:(98/139): python3-asyncssh-2.7.0-2.el8.noarch.r 686 kB/s | 476 kB 00:00 2023-10-06T05:14:13.486 INFO:teuthology.orchestra.run.smithi062.stdout:(99/139): python3-cachetools-3.1.1-4.el8.noarch 185 kB/s | 33 kB 00:00 2023-10-06T05:14:13.589 INFO:teuthology.orchestra.run.smithi062.stdout:(100/139): python3-cheroot-8.5.2-1.el8.noarch.r 1.2 MB/s | 173 kB 00:00 2023-10-06T05:14:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:14:13.673 INFO:teuthology.orchestra.run.smithi062.stdout:(101/139): python3-google-auth-1.1.1-10.el8.noa 444 kB/s | 81 kB 00:00 2023-10-06T05:14:13.695 INFO:teuthology.orchestra.run.smithi062.stdout:(102/139): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-10-06T05:14:13.721 INFO:teuthology.orchestra.run.smithi062.stdout:(103/139): python3-cherrypy-18.4.0-1.el8.noarch 1.6 MB/s | 384 kB 00:00 2023-10-06T05:14:13.771 INFO:teuthology.orchestra.run.smithi062.stdout:(104/139): python3-jaraco-functools-2.0-4.el8.n 175 kB/s | 18 kB 00:00 2023-10-06T05:14:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:14:13.854 INFO:teuthology.orchestra.run.smithi062.stdout:(105/139): python3-logutils-0.3.5-11.el8.noarch 365 kB/s | 49 kB 00:00 2023-10-06T05:14:13.913 INFO:teuthology.orchestra.run.smithi062.stdout:(106/139): python3-more-itertools-7.2.0-3.el8.n 414 kB/s | 59 kB 00:00 2023-10-06T05:14:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:14:13.980 INFO:teuthology.orchestra.run.smithi062.stdout:(107/139): python3-kubernetes-11.0.0-6.el8.noar 4.1 MB/s | 1.2 MB 00:00 2023-10-06T05:14:13.997 INFO:teuthology.orchestra.run.smithi062.stdout:(108/139): python3-natsort-7.1.1-2.el8.noarch.r 423 kB/s | 60 kB 00:00 2023-10-06T05:14:14.072 INFO:teuthology.orchestra.run.smithi062.stdout:(109/139): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-10-06T05:14:14.088 INFO:teuthology.orchestra.run.smithi062.stdout:(110/139): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-10-06T05:14:14.122 INFO:teuthology.orchestra.run.smithi062.stdout:(111/139): python3-repoze-lru-0.7-6.el8.noarch. 268 kB/s | 34 kB 00:00 2023-10-06T05:14:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:14:14.198 INFO:teuthology.orchestra.run.smithi062.stdout:(112/139): python3-rsa-4.9-2.el8.noarch.rpm 591 kB/s | 64 kB 00:00 2023-10-06T05:14:14.223 INFO:teuthology.orchestra.run.smithi062.stdout:(113/139): python3-routes-2.4.1-12.el8.noarch.r 1.3 MB/s | 196 kB 00:00 2023-10-06T05:14:14.248 INFO:teuthology.orchestra.run.smithi062.stdout:(114/139): python3-simplegeneric-0.8.1-17.el8.n 149 kB/s | 19 kB 00:00 2023-10-06T05:14:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:14:14.307 INFO:teuthology.orchestra.run.smithi062.stdout:(115/139): python3-singledispatch-3.4.0.3-18.el 219 kB/s | 24 kB 00:00 2023-10-06T05:14:14.332 INFO:teuthology.orchestra.run.smithi062.stdout:(116/139): python3-tempora-1.14.1-5.el8.noarch. 268 kB/s | 29 kB 00:00 2023-10-06T05:14:14.374 INFO:teuthology.orchestra.run.smithi062.stdout:(117/139): python3-trustme-0.6.0-4.el8.noarch.r 218 kB/s | 27 kB 00:00 2023-10-06T05:14:14.433 INFO:teuthology.orchestra.run.smithi062.stdout:(118/139): python3-waitress-1.4.3-1.el8.noarch. 2.0 MB/s | 255 kB 00:00 2023-10-06T05:14:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/155 2023-10-06T05:14:14.483 INFO:teuthology.orchestra.run.smithi062.stdout:(119/139): python3-webob-1.8.5-1.el8.1.noarch.r 1.6 MB/s | 251 kB 00:00 2023-10-06T05:14:14.525 INFO:teuthology.orchestra.run.smithi062.stdout:(120/139): python3-websocket-client-0.56.0-5.el 407 kB/s | 61 kB 00:00 2023-10-06T05:14:14.550 INFO:teuthology.orchestra.run.smithi062.stdout:(121/139): python3-webtest-2.0.33-1.el8.noarch. 729 kB/s | 85 kB 00:00 2023-10-06T05:14:14.592 INFO:teuthology.orchestra.run.smithi062.stdout:(122/139): python3-zc-lockfile-2.0-2.el8.noarch 211 kB/s | 23 kB 00:00 2023-10-06T05:14:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T05:14:15.009 INFO:teuthology.orchestra.run.smithi062.stdout:(123/139): librbd1-17.2.6-1274.g68e53a4a.el8.x8 8.5 MB/s | 3.5 MB 00:00 2023-10-06T05:14:15.093 INFO:teuthology.orchestra.run.smithi062.stdout:(124/139): thrift-0.13.0-2.el8.x86_64.rpm 3.1 MB/s | 1.7 MB 00:00 2023-10-06T05:14:15.218 INFO:teuthology.orchestra.run.smithi062.stdout:(125/139): librados2-17.2.6-1274.g68e53a4a.el8. 5.1 MB/s | 3.4 MB 00:00 2023-10-06T05:14:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T05:14:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:14:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:14:15.877 INFO:teuthology.orchestra.run.smithi062.stdout:(126/139): elfutils-debuginfod-client-0.189-3.e 97 kB/s | 76 kB 00:00 2023-10-06T05:14:16.051 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T05:14:16.077 INFO:teuthology.orchestra.run.smithi062.stdout:(127/139): NetworkManager-initscripts-updown-1. 134 kB/s | 143 kB 00:01 2023-10-06T05:14:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T05:14:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/155 2023-10-06T05:14:16.419 INFO:teuthology.orchestra.run.smithi062.stdout:(128/139): elfutils-libelf-0.189-3.el8.x86_64.r 193 kB/s | 232 kB 00:01 2023-10-06T05:14:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/155 2023-10-06T05:14:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 17/155 2023-10-06T05:14:16.801 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 17/155 2023-10-06T05:14:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:14:17.107 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:14:17.161 INFO:teuthology.orchestra.run.smithi062.stdout:(129/139): elfutils-libs-0.189-3.el8.x86_64.rpm 236 kB/s | 303 kB 00:01 2023-10-06T05:14:17.419 INFO:teuthology.orchestra.run.smithi062.stdout:(130/139): libblkid-2.32.1-43.el8.x86_64.rpm 220 kB/s | 220 kB 00:00 2023-10-06T05:14:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 19/155 2023-10-06T05:14:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 20/155 2023-10-06T05:14:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T05:14:17.897 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T05:14:17.928 INFO:teuthology.orchestra.run.smithi062.stdout:(131/139): iproute-6.2.0-5.el8.x86_64.rpm 461 kB/s | 854 kB 00:01 2023-10-06T05:14:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T05:14:18.295 INFO:teuthology.orchestra.run.smithi062.stdout:(132/139): libfdisk-2.32.1-43.el8.x86_64.rpm 224 kB/s | 254 kB 00:01 2023-10-06T05:14:18.595 INFO:teuthology.orchestra.run.smithi062.stdout:(133/139): libmount-2.32.1-43.el8.x86_64.rpm 201 kB/s | 236 kB 00:01 2023-10-06T05:14:18.786 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T05:14:18.971 INFO:teuthology.orchestra.run.smithi062.stdout:(134/139): libsmartcols-2.32.1-43.el8.x86_64.rp 172 kB/s | 179 kB 00:01 2023-10-06T05:14:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T05:14:19.054 INFO:teuthology.orchestra.run.smithi062.stdout:(135/139): libuuid-2.32.1-43.el8.x86_64.rpm 130 kB/s | 99 kB 00:00 2023-10-06T05:14:19.754 INFO:teuthology.orchestra.run.smithi062.stdout:(136/139): util-linux-user-2.32.1-43.el8.x86_64 131 kB/s | 103 kB 00:00 2023-10-06T05:14:19.766 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T05:14:19.927 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T05:14:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:(137/139): xfsdump-3.1.8-6.el8.x86_64.rpm 262 kB/s | 334 kB 00:01 2023-10-06T05:14:20.809 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T05:14:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/155 2023-10-06T05:14:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 26/155 2023-10-06T05:14:21.488 INFO:teuthology.orchestra.run.smithi062.stdout:(138/139): util-linux-2.32.1-43.el8.x86_64.rpm 877 kB/s | 2.5 MB 00:02 2023-10-06T05:14:21.502 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/155 2023-10-06T05:14:21.555 INFO:teuthology.orchestra.run.smithi062.stdout:(139/139): xfsprogs-5.0.0-12.el8.x86_64.rpm 615 kB/s | 1.1 MB 00:01 2023-10-06T05:14:21.559 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-10-06T05:14:21.559 INFO:teuthology.orchestra.run.smithi062.stdout:Total 7.8 MB/s | 209 MB 00:26 2023-10-06T05:14:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/155 2023-10-06T05:14:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 29/155 2023-10-06T05:14:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 30/155 2023-10-06T05:14:22.328 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 31/155 2023-10-06T05:14:23.307 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-10-06T05:14:23.484 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 32/155 2023-10-06T05:14:23.743 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-10-06T05:14:23.743 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-10-06T05:14:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/155 2023-10-06T05:14:24.160 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/155 2023-10-06T05:14:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T05:14:25.254 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-10-06T05:14:25.258 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-10-06T05:14:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T05:14:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 36/155 2023-10-06T05:14:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/155 2023-10-06T05:14:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/155 2023-10-06T05:14:27.583 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/155 2023-10-06T05:14:27.758 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 40/155 2023-10-06T05:14:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 41/155 2023-10-06T05:14:28.152 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 42/155 2023-10-06T05:14:28.387 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 43/155 2023-10-06T05:14:28.617 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T05:14:28.711 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T05:14:29.259 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T05:14:29.355 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T05:14:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 46/155 2023-10-06T05:14:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T05:14:30.312 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T05:14:30.618 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T05:14:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T05:14:30.665 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-10-06T05:14:30.720 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/1 2023-10-06T05:14:30.843 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/155 2023-10-06T05:14:30.849 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T05:14:30.942 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T05:14:31.082 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T05:14:31.089 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/155 2023-10-06T05:14:31.180 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T05:14:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/155 2023-10-06T05:14:31.409 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/155 2023-10-06T05:14:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/155 2023-10-06T05:14:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/155 2023-10-06T05:14:31.788 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 4/155 2023-10-06T05:14:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/155 2023-10-06T05:14:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/155 2023-10-06T05:14:31.988 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/155 2023-10-06T05:14:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/155 2023-10-06T05:14:32.195 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/155 2023-10-06T05:14:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/155 2023-10-06T05:14:32.365 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T05:14:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/155 2023-10-06T05:14:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/155 2023-10-06T05:14:32.817 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T05:14:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/155 2023-10-06T05:14:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:14:33.031 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/155 2023-10-06T05:14:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/155 2023-10-06T05:14:33.432 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:14:33.518 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/155 2023-10-06T05:14:33.632 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:14:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/155 2023-10-06T05:14:33.789 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:14:33.957 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:14:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/155 2023-10-06T05:14:34.078 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:14:34.269 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/155 2023-10-06T05:14:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/155 2023-10-06T05:14:34.432 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T05:14:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/155 2023-10-06T05:14:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/155 2023-10-06T05:14:35.019 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/155 2023-10-06T05:14:35.102 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T05:14:35.240 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:14:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 70/155 2023-10-06T05:14:35.480 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:14:35.691 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 71/155 2023-10-06T05:14:35.696 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T05:14:35.782 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T05:14:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 72/155 2023-10-06T05:14:36.019 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/155 2023-10-06T05:14:36.159 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 73/155 2023-10-06T05:14:36.199 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/155 2023-10-06T05:14:36.340 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 17/155 2023-10-06T05:14:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 74/155 2023-10-06T05:14:36.464 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 17/155 2023-10-06T05:14:36.606 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:14:36.679 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 75/155 2023-10-06T05:14:36.760 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:14:36.943 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 19/155 2023-10-06T05:14:37.092 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 20/155 2023-10-06T05:14:37.256 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T05:14:37.421 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T05:14:37.548 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T05:14:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/155 2023-10-06T05:14:38.273 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T05:14:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 77/155 2023-10-06T05:14:38.489 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T05:14:38.635 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 78/155 2023-10-06T05:14:38.884 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 79/155 2023-10-06T05:14:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/155 2023-10-06T05:14:39.237 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T05:14:39.403 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 81/155 2023-10-06T05:14:39.406 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T05:14:39.671 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/155 2023-10-06T05:14:40.492 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T05:14:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T05:14:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 84/155 2023-10-06T05:14:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 85/155 2023-10-06T05:14:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 86/155 2023-10-06T05:14:40.623 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/155 2023-10-06T05:14:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 87/155 2023-10-06T05:14:40.841 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 26/155 2023-10-06T05:14:40.998 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/155 2023-10-06T05:14:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 88/155 2023-10-06T05:14:41.366 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/155 2023-10-06T05:14:41.366 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 89/155 2023-10-06T05:14:41.423 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 29/155 2023-10-06T05:14:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 90/155 2023-10-06T05:14:41.570 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 30/155 2023-10-06T05:14:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 91/155 2023-10-06T05:14:41.812 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 31/155 2023-10-06T05:14:41.877 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 92/155 2023-10-06T05:14:42.103 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.7-17.el8.x86_64 93/155 2023-10-06T05:14:42.355 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 94/155 2023-10-06T05:14:42.543 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 95/155 2023-10-06T05:14:42.716 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 96/155 2023-10-06T05:14:42.977 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 32/155 2023-10-06T05:14:43.338 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/155 2023-10-06T05:14:43.631 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/155 2023-10-06T05:14:44.014 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T05:14:45.864 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:14:46.065 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:14:46.272 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T05:14:46.588 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 36/155 2023-10-06T05:14:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:14:46.840 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/155 2023-10-06T05:14:47.002 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/155 2023-10-06T05:14:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 98/155 2023-10-06T05:14:47.213 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/155 2023-10-06T05:14:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 98/155 2023-10-06T05:14:47.396 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 40/155 2023-10-06T05:14:47.417 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 99/155 2023-10-06T05:14:47.585 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 41/155 2023-10-06T05:14:47.764 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 42/155 2023-10-06T05:14:47.950 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 43/155 2023-10-06T05:14:48.155 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T05:14:48.249 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T05:14:48.413 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T05:14:48.500 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T05:14:48.644 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 46/155 2023-10-06T05:14:48.846 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T05:14:49.463 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T05:14:49.714 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T05:14:49.743 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T05:14:49.957 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/155 2023-10-06T05:14:50.254 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/155 2023-10-06T05:14:50.438 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/155 2023-10-06T05:14:50.552 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/155 2023-10-06T05:14:50.685 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/155 2023-10-06T05:14:50.864 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/155 2023-10-06T05:14:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/155 2023-10-06T05:14:51.288 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/155 2023-10-06T05:14:51.467 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/155 2023-10-06T05:14:51.621 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/155 2023-10-06T05:14:51.804 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/155 2023-10-06T05:14:51.971 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/155 2023-10-06T05:14:52.133 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/155 2023-10-06T05:14:52.361 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/155 2023-10-06T05:14:52.613 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/155 2023-10-06T05:14:52.856 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/155 2023-10-06T05:14:53.048 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/155 2023-10-06T05:14:53.378 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/155 2023-10-06T05:14:53.632 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/155 2023-10-06T05:14:53.828 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/155 2023-10-06T05:14:54.099 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/155 2023-10-06T05:14:54.335 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 70/155 2023-10-06T05:14:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 71/155 2023-10-06T05:14:55.021 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 72/155 2023-10-06T05:14:55.188 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 73/155 2023-10-06T05:14:55.430 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 74/155 2023-10-06T05:14:55.633 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 75/155 2023-10-06T05:14:56.982 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/155 2023-10-06T05:14:57.351 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 77/155 2023-10-06T05:14:57.523 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 78/155 2023-10-06T05:14:57.762 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 79/155 2023-10-06T05:14:58.079 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/155 2023-10-06T05:14:58.299 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 81/155 2023-10-06T05:14:58.593 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/155 2023-10-06T05:14:58.819 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T05:14:59.175 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 84/155 2023-10-06T05:14:59.393 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 85/155 2023-10-06T05:14:59.566 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 86/155 2023-10-06T05:14:59.776 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 87/155 2023-10-06T05:15:00.121 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 88/155 2023-10-06T05:15:00.309 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 89/155 2023-10-06T05:15:00.437 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 90/155 2023-10-06T05:15:00.634 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 91/155 2023-10-06T05:15:00.848 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 92/155 2023-10-06T05:15:01.083 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.7-17.el8.x86_64 93/155 2023-10-06T05:15:01.339 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 94/155 2023-10-06T05:15:01.505 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 95/155 2023-10-06T05:15:01.629 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 96/155 2023-10-06T05:15:04.854 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:15:05.103 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:15:05.815 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 97/155 2023-10-06T05:15:06.079 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 98/155 2023-10-06T05:15:06.287 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 98/155 2023-10-06T05:15:06.478 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 99/155 2023-10-06T05:15:08.250 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 99/155 2023-10-06T05:15:08.251 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2023-10-06T05:15:08.251 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2023-10-06T05:15:08.251 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2023-10-06T05:15:08.251 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /run 2023-10-06T05:15:08.251 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2023-10-06T05:15:08.251 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2023-10-06T05:15:08.251 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2023-10-06T05:15:08.252 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2023-10-06T05:15:08.252 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2023-10-06T05:15:08.252 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:15:08.628 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 100/155 2023-10-06T05:15:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 100/155 2023-10-06T05:15:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/155 2023-10-06T05:15:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/155 2023-10-06T05:15:11.064 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 102/155 2023-10-06T05:15:11.545 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 102/155 2023-10-06T05:15:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/155 2023-10-06T05:15:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/155 2023-10-06T05:15:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 104/155 2023-10-06T05:15:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 105/155 2023-10-06T05:15:12.385 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T05:15:12.485 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T05:15:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : jq-1.6-7.el8.x86_64 107/155 2023-10-06T05:15:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.1-8.el8.noarch 108/155 2023-10-06T05:15:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 109/155 2023-10-06T05:15:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 109/155 2023-10-06T05:15:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 110/155 2023-10-06T05:15:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 111/155 2023-10-06T05:15:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 112/155 2023-10-06T05:15:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 112/155 2023-10-06T05:15:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 113/155 2023-10-06T05:15:16.029 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 113/155 2023-10-06T05:15:16.209 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 114/155 2023-10-06T05:15:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 114/155 2023-10-06T05:15:16.746 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 115/155 2023-10-06T05:15:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/155 2023-10-06T05:15:17.028 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/155 2023-10-06T05:15:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 117/155 2023-10-06T05:15:17.238 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 117/155 2023-10-06T05:15:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 118/155 2023-10-06T05:15:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T05:15:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T05:15:25.282 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 120/155 2023-10-06T05:15:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/155 2023-10-06T05:15:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/155 2023-10-06T05:15:25.792 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 122/155 2023-10-06T05:15:25.864 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 122/155 2023-10-06T05:15:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 123/155 2023-10-06T05:15:26.483 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 123/155 2023-10-06T05:15:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:15:26.775 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:15:27.283 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 99/155 2023-10-06T05:15:27.283 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2023-10-06T05:15:27.283 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2023-10-06T05:15:27.283 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-10-06T05:15:27.283 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /run 2023-10-06T05:15:27.283 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2023-10-06T05:15:27.284 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2023-10-06T05:15:27.284 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2023-10-06T05:15:27.284 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2023-10-06T05:15:27.284 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-10-06T05:15:27.284 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:15:27.675 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 100/155 2023-10-06T05:15:28.109 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 125/155 2023-10-06T05:15:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 126/155 2023-10-06T05:15:28.639 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/155 2023-10-06T05:15:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 128/155 2023-10-06T05:15:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 129/155 2023-10-06T05:15:29.317 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 130/155 2023-10-06T05:15:29.449 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 100/155 2023-10-06T05:15:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 131/155 2023-10-06T05:15:29.676 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/155 2023-10-06T05:15:29.804 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 132/155 2023-10-06T05:15:29.966 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/155 2023-10-06T05:15:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 133/155 2023-10-06T05:15:30.158 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 102/155 2023-10-06T05:15:30.407 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 134/155 2023-10-06T05:15:30.648 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 102/155 2023-10-06T05:15:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 135/155 2023-10-06T05:15:30.833 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/155 2023-10-06T05:15:30.851 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T05:15:30.883 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T05:15:30.884 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/155 2023-10-06T05:15:31.084 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 104/155 2023-10-06T05:15:31.088 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 137/155 2023-10-06T05:15:31.290 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 105/155 2023-10-06T05:15:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T05:15:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T05:15:31.432 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T05:15:31.562 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T05:15:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T05:15:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T05:15:31.853 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.6-7.el8.x86_64 107/155 2023-10-06T05:15:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 140/155 2023-10-06T05:15:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 141/155 2023-10-06T05:15:32.066 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.1-8.el8.noarch 108/155 2023-10-06T05:15:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 142/155 2023-10-06T05:15:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T05:15:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T05:15:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T05:15:32.670 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 109/155 2023-10-06T05:15:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T05:15:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T05:15:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T05:15:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T05:15:32.862 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 109/155 2023-10-06T05:15:32.999 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 110/155 2023-10-06T05:15:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T05:15:33.211 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 111/155 2023-10-06T05:15:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 147/155 2023-10-06T05:15:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 148/155 2023-10-06T05:15:33.424 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 112/155 2023-10-06T05:15:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T05:15:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T05:15:33.696 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 112/155 2023-10-06T05:15:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T05:15:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:15:33.941 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 113/155 2023-10-06T05:15:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:15:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T05:15:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T05:15:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T05:15:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T05:15:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T05:15:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T05:15:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T05:15:35.074 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T05:15:35.074 INFO:teuthology.orchestra.run.smithi031.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T05:15:35.074 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:15:35.133 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 113/155 2023-10-06T05:15:35.332 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 114/155 2023-10-06T05:15:35.467 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 114/155 2023-10-06T05:15:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/155 2023-10-06T05:15:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/155 2023-10-06T05:15:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/155 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/155 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/155 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/155 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/155 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/155 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/155 2023-10-06T05:15:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/155 2023-10-06T05:15:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/155 2023-10-06T05:15:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/155 2023-10-06T05:15:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/155 2023-10-06T05:15:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/155 2023-10-06T05:15:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : jq-1.6-7.el8.x86_64 42/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 43/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 44/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 45/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 46/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 47/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 48/155 2023-10-06T05:15:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 49/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 50/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 51/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 52/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 53/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 54/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 55/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 56/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 57/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 58/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 59/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 60/155 2023-10-06T05:15:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 61/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 62/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 63/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 64/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 65/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 66/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 67/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 68/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 69/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 70/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 71/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 72/155 2023-10-06T05:15:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 73/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 74/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 75/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 76/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 77/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 78/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 80/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 81/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 82/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 84/155 2023-10-06T05:15:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 85/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 86/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 87/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 88/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 89/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 90/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 91/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 92/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 93/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 94/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 95/155 2023-10-06T05:15:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 96/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 98/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 99/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 100/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 101/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 102/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 103/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 104/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 105/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 106/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 107/155 2023-10-06T05:15:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 108/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 109/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-2.6-1.el8.noarch 110/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 111/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 112/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 113/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 114/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 115/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 116/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 117/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 118/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 119/155 2023-10-06T05:15:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 120/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 121/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 122/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 123/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 125/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 126/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 127/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 128/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 129/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 130/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 131/155 2023-10-06T05:15:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 132/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 133/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 134/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 135/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 136/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 137/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 138/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 139/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 140/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 141/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 143/155 2023-10-06T05:15:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/155 2023-10-06T05:15:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 145/155 2023-10-06T05:15:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/155 2023-10-06T05:15:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 147/155 2023-10-06T05:15:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/155 2023-10-06T05:15:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T05:15:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/155 2023-10-06T05:15:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:15:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/155 2023-10-06T05:15:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/155 2023-10-06T05:15:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 154/155 2023-10-06T05:15:35.853 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 115/155 2023-10-06T05:15:36.050 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/155 2023-10-06T05:15:36.118 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/155 2023-10-06T05:15:36.307 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 117/155 2023-10-06T05:15:36.335 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 117/155 2023-10-06T05:15:36.549 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 118/155 2023-10-06T05:15:36.698 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T05:15:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 155/155 2023-10-06T05:15:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:15:37.939 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2023-10-06T05:15:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: iproute-6.2.0-5.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: libblkid-2.32.1-43.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: libmount-2.32.1-43.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-2.32.1-43.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux-2.32.1-43.el8.x86_64 2023-10-06T05:15:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.1-8.el8.noarch 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.7-17.el8.x86_64 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: jq-1.6-7.el8.x86_64 2023-10-06T05:15:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T05:15:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-10-06T05:15:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T05:15:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T05:15:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T05:15:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-10-06T05:15:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-10-06T05:15:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-10-06T05:15:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T05:15:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T05:15:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T05:15:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T05:15:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-10-06T05:15:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T05:15:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-10-06T05:15:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:15:37.952 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-10-06T05:15:38.241 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:42.485 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T05:15:44.468 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 120/155 2023-10-06T05:15:44.658 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/155 2023-10-06T05:15:44.786 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/155 2023-10-06T05:15:44.956 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 122/155 2023-10-06T05:15:45.029 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 122/155 2023-10-06T05:15:45.214 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 123/155 2023-10-06T05:15:45.653 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 123/155 2023-10-06T05:15:45.823 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:15:45.959 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:15:47.255 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 125/155 2023-10-06T05:15:47.695 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 126/155 2023-10-06T05:15:47.927 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/155 2023-10-06T05:15:48.130 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 128/155 2023-10-06T05:15:48.269 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 129/155 2023-10-06T05:15:48.496 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 130/155 2023-10-06T05:15:48.687 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 131/155 2023-10-06T05:15:48.891 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 132/155 2023-10-06T05:15:49.125 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 133/155 2023-10-06T05:15:49.516 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 134/155 2023-10-06T05:15:49.724 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 135/155 2023-10-06T05:15:49.855 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T05:15:49.885 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T05:15:50.090 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 137/155 2023-10-06T05:15:50.330 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T05:15:50.448 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T05:15:50.680 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T05:15:50.708 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T05:15:50.882 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 140/155 2023-10-06T05:15:51.017 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 141/155 2023-10-06T05:15:51.169 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 142/155 2023-10-06T05:15:51.169 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T05:15:51.427 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T05:15:51.427 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T05:15:51.658 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T05:15:51.676 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T05:15:51.770 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T05:15:51.771 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T05:15:52.040 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T05:15:52.199 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 147/155 2023-10-06T05:15:52.352 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 148/155 2023-10-06T05:15:52.582 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T05:15:52.582 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T05:15:52.800 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T05:15:52.800 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:15:52.992 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:15:52.992 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T05:15:53.192 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T05:15:53.192 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T05:15:53.392 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T05:15:53.392 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T05:15:53.592 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T05:15:53.815 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T05:15:54.026 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T05:15:54.027 INFO:teuthology.orchestra.run.smithi062.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T05:15:54.027 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:15:54.744 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/155 2023-10-06T05:15:54.744 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/155 2023-10-06T05:15:54.744 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/155 2023-10-06T05:15:54.744 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/155 2023-10-06T05:15:54.744 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/155 2023-10-06T05:15:54.745 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/155 2023-10-06T05:15:54.746 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/155 2023-10-06T05:15:54.757 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/155 2023-10-06T05:15:54.757 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/155 2023-10-06T05:15:54.758 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/155 2023-10-06T05:15:54.758 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/155 2023-10-06T05:15:54.758 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/155 2023-10-06T05:15:54.758 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/155 2023-10-06T05:15:54.758 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/155 2023-10-06T05:15:54.758 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/155 2023-10-06T05:15:54.758 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/155 2023-10-06T05:15:54.758 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/155 2023-10-06T05:15:54.758 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/155 2023-10-06T05:15:54.758 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.6-7.el8.x86_64 42/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 43/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 44/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 45/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 46/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 47/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 48/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 49/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 50/155 2023-10-06T05:15:54.759 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 51/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 52/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 53/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 54/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 55/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 56/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 57/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 58/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 59/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 60/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 61/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 62/155 2023-10-06T05:15:54.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 63/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 64/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 65/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 66/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 67/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 68/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 69/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 70/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 71/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 72/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 73/155 2023-10-06T05:15:54.761 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 74/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 75/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 76/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 77/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 78/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 80/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 81/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 82/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 84/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 85/155 2023-10-06T05:15:54.762 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 86/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 87/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 88/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 89/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 90/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 91/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 92/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 93/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 94/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 95/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 96/155 2023-10-06T05:15:54.763 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 98/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 99/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 100/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 101/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 102/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 103/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 104/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 105/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 106/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 107/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 108/155 2023-10-06T05:15:54.764 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 109/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-2.6-1.el8.noarch 110/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 111/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 112/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 113/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 114/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 115/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 116/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 117/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 118/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 119/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 120/155 2023-10-06T05:15:54.765 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 121/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 122/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 123/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 124/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 125/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 126/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 127/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 128/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 129/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 130/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 131/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 132/155 2023-10-06T05:15:54.766 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 133/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 134/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 135/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 136/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 137/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 138/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 139/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 140/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 141/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 143/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/155 2023-10-06T05:15:54.767 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 145/155 2023-10-06T05:15:54.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/155 2023-10-06T05:15:54.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 147/155 2023-10-06T05:15:54.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/155 2023-10-06T05:15:54.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T05:15:54.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/155 2023-10-06T05:15:54.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 151/155 2023-10-06T05:15:54.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/155 2023-10-06T05:15:54.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/155 2023-10-06T05:15:54.768 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 154/155 2023-10-06T05:15:56.891 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 155/155 2023-10-06T05:15:56.891 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:15:56.891 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2023-10-06T05:15:56.891 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-10-06T05:15:56.891 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-10-06T05:15:56.891 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-10-06T05:15:56.891 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: iproute-6.2.0-5.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid-2.32.1-43.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: libmount-2.32.1-43.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-2.32.1-43.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-2.32.1-43.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-10-06T05:15:56.892 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.1-8.el8.noarch 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.893 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.894 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.7-17.el8.x86_64 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.6-7.el8.x86_64 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-10-06T05:15:56.895 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T05:15:56.896 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-06T05:15:56.897 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.898 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T05:15:56.899 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T05:15:56.900 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.901 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T05:15:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T05:15:56.903 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T05:15:56.903 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.903 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.903 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T05:15:56.903 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T05:15:56.903 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T05:15:56.903 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-10-06T05:15:56.903 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T05:15:56.903 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-10-06T05:15:56.904 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:15:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-10-06T05:15:57.206 DEBUG:teuthology.parallel:result is None 2023-10-06T05:15:57.206 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:15:57.422 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T05:15:57.484 INFO:teuthology.orchestra.run.smithi031.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T05:15:57.485 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T05:15:57.485 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T05:15:57.485 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T05:15:57.623 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T05:15:57.680 INFO:teuthology.orchestra.run.smithi062.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T05:15:57.680 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T05:15:57.680 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T05:15:57.681 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T05:15:57.681 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:15:57.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T05:15:57.759 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:15:57.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T05:15:57.837 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T05:15:57.837 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:15:57.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T05:15:57.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T05:15:58.020 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:15:58.020 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T05:15:58.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T05:15:58.209 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T05:15:58.209 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:15:58.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T05:15:58.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T05:15:58.387 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:15:58.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T05:15:58.461 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T05:15:58.573 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T05:15:58.573 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:15:58.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T05:15:58.644 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T05:15:58.754 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:15:58.754 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T05:15:58.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T05:15:58.938 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T05:15:59.000 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T05:15:59.000 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2023-10-06T05:15:59.005 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2023-10-06T05:15:59.080 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T05:15:59.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T05:15:59.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T05:15:59.156 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T05:15:59.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T05:15:59.161 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T05:15:59.244 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T05:15:59.244 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T05:15:59.244 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T05:15:59.244 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:15:59.244 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T05:15:59.289 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:59.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:59.578 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:15:59.578 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T05:15:59.626 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:59.845 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T05:15:59.924 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T05:15:59.924 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T05:15:59.925 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320', 'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412320', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4002, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOCtRlN3Pn051NZ34WfUmcvDmyDYi1mMnnAnxhB2BJY7wk7jHyiXEIm5o3wKOQLH9VdtEhJwxUx5MB1PDqcrXk=', 'smithi062.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsWIedGVlr9JwI163R6gtU8pVRhzz0ug/T52ijS54Zev/RwmSYO1NjHrhaiYHJrMWM5FMNsEeqjICUBTlGjX1Q='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:54:57', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T05:15:59.925 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T05:15:59.930 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T05:15:59.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T05:16:00.017 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T05:16:00.093 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:00.094 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T05:16:00.139 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T05:16:00.139 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2023-10-06T05:16:00.224 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T05:16:00.224 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:16:00.224 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-10-06T05:16:00.225 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:16:00.225 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:16:00.225 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 05:15:35.301808793 +0000 2023-10-06T05:16:00.225 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 05:12:01.141350001 +0000 2023-10-06T05:16:00.225 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 05:12:01.141350001 +0000 2023-10-06T05:16:00.225 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-10-06 05:12:01.112350462 +0000 2023-10-06T05:16:00.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T05:16:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:16:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:16:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000450227 s, 1.1 MB/s 2023-10-06T05:16:00.335 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T05:16:00.382 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2023-10-06T05:16:00.428 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T05:16:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:16:00.429 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-10-06T05:16:00.429 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:16:00.429 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:16:00.429 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 05:15:35.301808793 +0000 2023-10-06T05:16:00.429 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 05:12:01.996336414 +0000 2023-10-06T05:16:00.429 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 05:12:01.996336414 +0000 2023-10-06T05:16:00.429 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-10-06 05:12:01.968336858 +0000 2023-10-06T05:16:00.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T05:16:00.535 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:16:00.535 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:16:00.535 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.00028723 s, 1.8 MB/s 2023-10-06T05:16:00.537 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T05:16:00.585 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2023-10-06T05:16:00.630 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T05:16:00.631 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:16:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-10-06T05:16:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:16:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:16:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 05:15:35.301808793 +0000 2023-10-06T05:16:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 05:12:02.852322810 +0000 2023-10-06T05:16:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 05:12:02.852322810 +0000 2023-10-06T05:16:00.631 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-10-06 05:12:02.826323223 +0000 2023-10-06T05:16:00.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T05:16:00.737 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:16:00.737 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:16:00.737 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000286911 s, 1.8 MB/s 2023-10-06T05:16:00.740 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T05:16:00.787 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2023-10-06T05:16:00.833 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T05:16:00.833 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:16:00.833 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-10-06T05:16:00.833 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:16:00.833 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:16:00.833 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-10-06 05:15:35.302808777 +0000 2023-10-06T05:16:00.833 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-10-06 05:12:03.687309541 +0000 2023-10-06T05:16:00.833 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-10-06 05:12:03.687309541 +0000 2023-10-06T05:16:00.834 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-10-06 05:12:03.663309922 +0000 2023-10-06T05:16:00.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T05:16:00.940 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:16:00.940 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:16:00.940 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000292973 s, 1.7 MB/s 2023-10-06T05:16:00.942 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T05:16:00.990 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:16:00.991 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:00.991 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T05:16:01.037 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T05:16:01.037 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2023-10-06T05:16:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T05:16:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:16:01.126 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-10-06T05:16:01.126 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:16:01.126 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:16:01.126 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-10-06 05:15:54.246885931 +0000 2023-10-06T05:16:01.126 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-10-06 05:12:01.137242802 +0000 2023-10-06T05:16:01.127 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-10-06 05:12:01.137242802 +0000 2023-10-06T05:16:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2023-10-06 05:12:01.109243209 +0000 2023-10-06T05:16:01.127 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T05:16:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-10-06T05:16:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-10-06T05:16:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000285121 s, 1.8 MB/s 2023-10-06T05:16:01.240 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T05:16:01.290 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2023-10-06T05:16:01.337 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T05:16:01.337 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:16:01.337 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-10-06T05:16:01.338 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:16:01.338 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:16:01.338 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-10-06 05:15:54.246885931 +0000 2023-10-06T05:16:01.338 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-10-06 05:12:01.986230468 +0000 2023-10-06T05:16:01.338 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-10-06 05:12:01.986230468 +0000 2023-10-06T05:16:01.338 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2023-10-06 05:12:01.958230875 +0000 2023-10-06T05:16:01.338 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T05:16:01.447 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-10-06T05:16:01.447 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-10-06T05:16:01.447 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000275041 s, 1.9 MB/s 2023-10-06T05:16:01.449 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T05:16:01.499 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2023-10-06T05:16:01.546 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T05:16:01.546 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:16:01.546 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-10-06T05:16:01.547 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:16:01.547 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:16:01.547 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-10-06 05:15:54.246885931 +0000 2023-10-06T05:16:01.547 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-10-06 05:12:02.828218236 +0000 2023-10-06T05:16:01.547 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-10-06 05:12:02.828218236 +0000 2023-10-06T05:16:01.547 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2023-10-06 05:12:02.803218599 +0000 2023-10-06T05:16:01.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T05:16:01.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-10-06T05:16:01.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-10-06T05:16:01.657 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000273557 s, 1.9 MB/s 2023-10-06T05:16:01.659 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T05:16:01.709 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2023-10-06T05:16:01.757 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T05:16:01.757 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T05:16:01.757 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-10-06T05:16:01.757 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T05:16:01.757 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T05:16:01.757 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-10-06 05:15:54.246885931 +0000 2023-10-06T05:16:01.757 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-10-06 05:12:03.673205960 +0000 2023-10-06T05:16:01.757 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-10-06 05:12:03.673205960 +0000 2023-10-06T05:16:01.758 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2023-10-06 05:12:03.649206309 +0000 2023-10-06T05:16:01.758 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T05:16:01.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-10-06T05:16:01.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-10-06T05:16:01.828 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000277677 s, 1.8 MB/s 2023-10-06T05:16:01.830 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T05:16:01.880 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:16:01.881 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T05:16:01.881 INFO:tasks.ceph:Generating config... 2023-10-06T05:16:01.882 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T05:16:01.882 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T05:16:01.882 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T05:16:01.882 INFO:tasks.ceph:[client] pid file = 2023-10-06T05:16:01.882 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T05:16:01.882 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T05:16:01.882 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-06T05:16:01.882 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T05:16:01.883 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T05:16:01.884 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T05:16:01.884 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T05:16:01.884 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T05:16:01.884 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-06T05:16:01.884 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-06T05:16:01.884 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-06T05:16:01.884 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T05:16:01.884 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T05:16:01.884 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-06T05:16:01.884 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-06T05:16:01.884 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T05:16:01.884 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T05:16:01.884 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-06T05:16:01.885 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T05:16:01.885 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T05:16:01.885 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T05:16:01.885 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T05:16:01.885 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T05:16:01.885 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T05:16:01.885 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T05:16:01.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T05:16:01.993 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T05:16:01.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T05:16:02.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T05:16:02.164 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.b', '172.21.15.62'), ('mon.c', '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]')] 2023-10-06T05:16:02.165 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T05:16:02.166 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:02.166 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T05:16:02.211 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --add b 172.21.15.62 --addv c '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:02.335 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:02.335 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid 86bc8148-d401-41f1-bd7f-edc6b2417154 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = octopus 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 86bc8148-d401-41f1-bd7f-edc6b2417154 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2023-10-06T05:16:02.334710+0000 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:created 2023-10-06T05:16:02.334710+0000 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 15 (octopus) 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] mon.c 2023-10-06T05:16:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T05:16:02.338 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T05:16:02.382 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 86bc8148-d401-41f1-bd7f-edc6b2417154... 2023-10-06T05:16:02.384 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T05:16:02.427 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T05:16:02.541 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2023-10-06T05:16:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2023-10-06T05:16:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T05:16:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2023-10-06T05:16:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2023-10-06T05:16:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2023-10-06T05:16:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]" 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on skipped message = False 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 86bc8148-d401-41f1-bd7f-edc6b2417154 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.545 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2023-10-06T05:16:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: bdev async discard = True 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: bdev enable discard = True 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore allocator = bitmap 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2023-10-06T05:16:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-10-06T05:16:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: debug client = 20 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: debug finisher = 20 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2023-10-06T05:16:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T05:16:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: mon op complaint time = 120 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2023-10-06T05:16:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: client mount timeout = 600 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: debug client = 20 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = "" 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2023-10-06T05:16:02.553 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout:[mds] 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: debug mds = 20 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug frag = True 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug scatterstat = True 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: mds op complaint time = 180 2023-10-06T05:16:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: mds verify scatter = True 2023-10-06T05:16:02.555 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2023-10-06T05:16:02.555 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2023-10-06T05:16:02.555 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2023-10-06T05:16:02.555 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2023-10-06T05:16:02.555 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2023-10-06T05:16:02.555 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T05:16:02.555 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.556 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2023-10-06T05:16:02.557 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2023-10-06T05:16:02.558 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2023-10-06T05:16:02.558 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2023-10-06T05:16:02.558 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2023-10-06T05:16:02.558 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T05:16:02.558 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T05:16:02.558 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2023-10-06T05:16:02.558 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.558 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T05:16:02.558 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "172.21.15.31,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]" 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on skipped message = False 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 86bc8148-d401-41f1-bd7f-edc6b2417154 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.559 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 600 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock profile = high_recovery_ops 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2023-10-06T05:16:02.560 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2023-10-06T05:16:02.561 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2023-10-06T05:16:02.561 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.561 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2023-10-06T05:16:02.561 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2023-10-06T05:16:02.561 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.561 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2023-10-06T05:16:02.561 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.561 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2023-10-06T05:16:02.561 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.561 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2023-10-06T05:16:02.562 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.562 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2023-10-06T05:16:02.562 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2023-10-06T05:16:02.562 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.562 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2023-10-06T05:16:02.562 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2023-10-06T05:16:02.562 INFO:teuthology.orchestra.run.smithi062.stdout: bdev async discard = True 2023-10-06T05:16:02.562 INFO:teuthology.orchestra.run.smithi062.stdout: bdev enable discard = True 2023-10-06T05:16:02.562 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore allocator = bitmap 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 1/20 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 1/20 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 20 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 4/10 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2023-10-06T05:16:02.563 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2023-10-06T05:16:02.564 INFO:teuthology.orchestra.run.smithi062.stdout: debug finisher = 20 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2023-10-06T05:16:02.565 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: # 1m isn't quite enough 2023-10-06T05:16:02.566 INFO:teuthology.orchestra.run.smithi062.stdout: mon_down_mkfs_grace = 2m 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_filestore_osds = false 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout: mon op complaint time = 120 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T05:16:02.567 INFO:teuthology.orchestra.run.smithi062.stdout: client mount timeout = 600 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = "" 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2023-10-06T05:16:02.568 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout:[mds] 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds = 20 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug frag = True 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug scatterstat = True 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout: mds op complaint time = 180 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout: mds verify scatter = True 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2023-10-06T05:16:02.569 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2023-10-06T05:16:02.581 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T05:16:02.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T05:16:02.689 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T05:16:02.689 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:02.689 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T05:16:02.734 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:02.735 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T05:16:02.819 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:16:02.820 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:02.820 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T05:16:02.820 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T05:16:02.955 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:02.955 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:03.041 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:16:03.041 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:03.041 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T05:16:03.041 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T05:16:03.138 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:03.138 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:03.224 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T05:16:03.224 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T05:16:03.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T05:16:03.343 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T05:16:03.348 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T05:16:03.493 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T05:16:03.499 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T05:16:03.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T05:16:03.618 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T05:16:03.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T05:16:03.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T05:16:03.811 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T05:16:03.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T05:16:03.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T05:16:04.006 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T05:16:04.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T05:16:04.081 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T05:16:04.200 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T05:16:04.204 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T05:16:04.274 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T05:16:04.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T05:16:04.369 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T05:16:04.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T05:16:04.492 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T05:16:04.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T05:16:04.615 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T05:16:04.646 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T05:16:04.738 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T05:16:04.768 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T05:16:04.768 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T05:16:04.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T05:16:04.839 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:16:04.840 INFO:tasks.ceph:role: osd.0 2023-10-06T05:16:04.840 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:16:04.840 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T05:16:04.916 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:16:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:16:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:16:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:16:04.917 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:16:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:16:04.917 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:16:04.917 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:16:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:16:04.917 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:16:05.356 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-10-06T05:16:05.359 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-10-06T05:16:05.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T05:16:05.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T05:16:05.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T05:16:05.703 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:16:05.703 INFO:tasks.ceph:role: osd.1 2023-10-06T05:16:05.703 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:16:05.703 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T05:16:05.780 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:16:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:16:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:16:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:16:05.781 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:16:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:16:05.781 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:16:05.781 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:16:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:16:05.781 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:16:06.190 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-10-06T05:16:06.193 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-10-06T05:16:06.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T05:16:06.348 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T05:16:06.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T05:16:06.495 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:16:06.495 INFO:tasks.ceph:role: osd.2 2023-10-06T05:16:06.495 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:16:06.495 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T05:16:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:16:06.572 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:16:06.573 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:16:06.573 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:16:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:16:06.573 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:16:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:16:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:16:06.573 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:16:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:16:07.020 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-10-06T05:16:07.024 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-10-06T05:16:07.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T05:16:07.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T05:16:07.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T05:16:07.352 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T05:16:07.352 INFO:tasks.ceph:role: osd.3 2023-10-06T05:16:07.352 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:16:07.353 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T05:16:07.431 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:16:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:16:07.433 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:16:07.868 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-10-06T05:16:07.872 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-10-06T05:16:07.872 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T05:16:07.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T05:16:08.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:08.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:08.188+0000 7f813686e540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T05:16:08.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:08.189+0000 7f813686e540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T05:16:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:08.189+0000 7f813686e540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T05:16:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:08.189+0000 7f813686e540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T05:16:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:08.189+0000 7f813686e540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T05:16:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:08.191+0000 7f813686e540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T05:16:08.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:08.732+0000 7f813686e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:09.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:09.494+0000 7f813686e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:10.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:10.263+0000 7f813686e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:11.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T05:16:11.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:11.476+0000 7fb6db797540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T05:16:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:11.476+0000 7fb6db797540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T05:16:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:11.476+0000 7fb6db797540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T05:16:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:11.477+0000 7fb6db797540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T05:16:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:11.477+0000 7fb6db797540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T05:16:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:11.478+0000 7fb6db797540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T05:16:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:12.022+0000 7fb6db797540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:12.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:12.785+0000 7fb6db797540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:13.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:13.549+0000 7fb6db797540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:14.369 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T05:16:14.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:14.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:14.546+0000 7f811f4a9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T05:16:14.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:14.547+0000 7f811f4a9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T05:16:14.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:14.547+0000 7f811f4a9540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T05:16:14.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:14.547+0000 7f811f4a9540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T05:16:14.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:14.547+0000 7f811f4a9540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T05:16:14.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:14.549+0000 7f811f4a9540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T05:16:15.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:15.096+0000 7f811f4a9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:15.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:15.859+0000 7f811f4a9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:16.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:16.622+0000 7f811f4a9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:17.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T05:16:17.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:17.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:17.623+0000 7fab759a2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T05:16:17.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:17.623+0000 7fab759a2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T05:16:17.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:17.623+0000 7fab759a2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T05:16:17.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:17.623+0000 7fab759a2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T05:16:17.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:17.623+0000 7fab759a2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T05:16:17.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:17.625+0000 7fab759a2540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T05:16:18.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:18.163+0000 7fab759a2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:18.931+0000 7fab759a2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:19.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:19.694+0000 7fab759a2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:20.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T05:16:20.591 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T05:16:20.670 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:16:20.670 INFO:tasks.ceph:role: osd.4 2023-10-06T05:16:20.670 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:16:20.670 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T05:16:20.750 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:16:20.750 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:16:20.751 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:16:20.751 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:16:20.751 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:16:20.751 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:16:20.751 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:16:20.751 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:16:20.751 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:16:20.751 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:16:21.176 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-10-06T05:16:21.180 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-10-06T05:16:21.180 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T05:16:21.387 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T05:16:21.469 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T05:16:21.539 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:16:21.540 INFO:tasks.ceph:role: osd.5 2023-10-06T05:16:21.540 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:16:21.540 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T05:16:21.616 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:16:21.617 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:16:21.617 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:16:21.617 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:16:21.617 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:16:21.617 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:16:21.617 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:16:21.617 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:16:21.617 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:16:21.617 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:16:22.027 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-10-06T05:16:22.031 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-10-06T05:16:22.031 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T05:16:22.173 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T05:16:22.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T05:16:22.320 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:16:22.320 INFO:tasks.ceph:role: osd.6 2023-10-06T05:16:22.320 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:16:22.320 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T05:16:22.398 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:16:22.398 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:16:22.398 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:16:22.398 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:16:22.399 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:16:22.399 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:16:22.399 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:16:22.399 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:16:22.399 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:16:22.399 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:16:22.849 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-10-06T05:16:22.853 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-10-06T05:16:22.853 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T05:16:23.006 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T05:16:23.086 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T05:16:23.156 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T05:16:23.156 INFO:tasks.ceph:role: osd.7 2023-10-06T05:16:23.156 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:16:23.156 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T05:16:23.233 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T05:16:23.233 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T05:16:23.233 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T05:16:23.233 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T05:16:23.234 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T05:16:23.234 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-10-06T05:16:23.234 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T05:16:23.234 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T05:16:23.234 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T05:16:23.234 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T05:16:23.666 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-10-06T05:16:23.670 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-10-06T05:16:23.670 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T05:16:23.825 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T05:16:23.901 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:24.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:24.016+0000 7f37abb59540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T05:16:24.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:24.017+0000 7f37abb59540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T05:16:24.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:24.017+0000 7f37abb59540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T05:16:24.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:24.017+0000 7f37abb59540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T05:16:24.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:24.017+0000 7f37abb59540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T05:16:24.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:24.019+0000 7f37abb59540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T05:16:24.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:24.556+0000 7f37abb59540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:25.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:25.318+0000 7f37abb59540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:26.085+0000 7f37abb59540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:26.897 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T05:16:26.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:27.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:27.068+0000 7f9a14887540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T05:16:27.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:27.068+0000 7f9a14887540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T05:16:27.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:27.068+0000 7f9a14887540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T05:16:27.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:27.069+0000 7f9a14887540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T05:16:27.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:27.069+0000 7f9a14887540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T05:16:27.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:27.070+0000 7f9a14887540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T05:16:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:27.609+0000 7f9a14887540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:28.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:28.370+0000 7f9a14887540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:29.138+0000 7f9a14887540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:29.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:16:29.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:16:29.955 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T05:16:30.069 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:30.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:30.168+0000 7f623b43f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T05:16:30.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:30.169+0000 7f623b43f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T05:16:30.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:30.169+0000 7f623b43f540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T05:16:30.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:30.169+0000 7f623b43f540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T05:16:30.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:30.169+0000 7f623b43f540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T05:16:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:30.170+0000 7f623b43f540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T05:16:30.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:30.712+0000 7f623b43f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:31.476+0000 7f623b43f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:32.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:32.247+0000 7f623b43f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:33.065 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T05:16:33.139 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:33.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:33.239+0000 7fd5cc9bc540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T05:16:33.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:33.239+0000 7fd5cc9bc540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T05:16:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:33.239+0000 7fd5cc9bc540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T05:16:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:33.239+0000 7fd5cc9bc540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T05:16:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:33.239+0000 7fd5cc9bc540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T05:16:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:33.241+0000 7fd5cc9bc540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T05:16:33.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:33.780+0000 7fd5cc9bc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:34.539+0000 7fd5cc9bc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:35.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:35.303+0000 7fd5cc9bc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T05:16:36.117 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T05:16:36.193 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T05:16:36.193 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:36.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T05:16:36.271 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:36.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T05:16:36.364 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:36.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T05:16:36.434 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:36.434 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T05:16:36.545 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:36.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T05:16:36.654 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:36.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T05:16:36.764 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:36.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T05:16:36.873 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:36.873 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T05:16:36.952 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:36.952 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T05:16:37.024 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:37.025 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T05:16:37.134 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:37.134 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T05:16:37.244 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:37.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T05:16:37.355 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:37.355 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T05:16:37.465 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:37.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T05:16:37.575 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:37.575 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T05:16:37.624 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:37.625 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T05:16:37.672 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:37.672 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T05:16:37.717 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:37.717 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T05:16:37.802 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T05:16:37.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T05:16:37.807 DEBUG:teuthology.orchestra.run.smithi062:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T05:16:37.871 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2023-10-06T05:16:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg== 2023-10-06T05:16:37.871 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2023-10-06T05:16:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw== 2023-10-06T05:16:37.871 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2023-10-06T05:16:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA== 2023-10-06T05:16:37.871 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA== 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ== 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA== 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg== 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg== 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2023-10-06T05:16:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ== 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA== 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw== 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw== 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ== 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ== 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-10-06T05:16:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g== 2023-10-06T05:16:37.874 INFO:teuthology.orchestra.run.smithi031.stdout:[client.2] 2023-10-06T05:16:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ== 2023-10-06T05:16:37.874 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2023-10-06T05:16:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w== 2023-10-06T05:16:37.874 INFO:teuthology.orchestra.run.smithi031.stdout:[client.3] 2023-10-06T05:16:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng== 2023-10-06T05:16:37.909 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.y] 2023-10-06T05:16:37.909 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg== 2023-10-06T05:16:37.909 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2023-10-06T05:16:37.909 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw== 2023-10-06T05:16:37.909 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA== 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.0] 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA== 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.1] 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ== 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.2] 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA== 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.3] 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg== 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.x] 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg== 2023-10-06T05:16:37.910 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ== 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA== 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.4] 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw== 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.5] 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw== 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.6] 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ== 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.7] 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ== 2023-10-06T05:16:37.911 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2023-10-06T05:16:37.912 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g== 2023-10-06T05:16:37.912 INFO:teuthology.orchestra.run.smithi062.stdout:[client.2] 2023-10-06T05:16:37.912 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ== 2023-10-06T05:16:37.912 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2023-10-06T05:16:37.912 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w== 2023-10-06T05:16:37.912 INFO:teuthology.orchestra.run.smithi062.stdout:[client.3] 2023-10-06T05:16:37.912 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng== 2023-10-06T05:16:37.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:16:37.920 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:16:38.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:16:38.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:16:38.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:16:38.173 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:16:38.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:38.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:38.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:38.393 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:38.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:38.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:38.604 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:38.609 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:38.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:16:38.718 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T05:16:38.822 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:16:38.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:16:38.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:16:38.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T05:16:39.044 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:39.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:39.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:39.156 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:39.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:39.263 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:39.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:39.373 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T05:16:39.478 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:16:39.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:16:39.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:16:39.594 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:16:39.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:16:39.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:16:39.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:16:39.817 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T05:16:39.924 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T05:16:39.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T05:16:39.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T05:16:40.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T05:16:40.185 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T05:16:40.255 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T05:16:40.374 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T05:16:40.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T05:16:40.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T05:16:40.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T05:16:40.703 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:40.707 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T05:16:40.752 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T05:16:40.752 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T05:16:40.752 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T05:16:40.757 INFO:tasks.ceph.mon.a:Started 2023-10-06T05:16:40.757 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T05:16:40.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T05:16:40.796 INFO:tasks.ceph.mon.b:Started 2023-10-06T05:16:40.796 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T05:16:40.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T05:16:40.801 INFO:tasks.ceph.mon.c:Started 2023-10-06T05:16:40.801 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T05:16:40.801 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T05:16:40.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T05:16:40.806 INFO:tasks.ceph.mgr.y:Started 2023-10-06T05:16:40.806 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T05:16:40.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T05:16:40.810 INFO:tasks.ceph.mgr.x:Started 2023-10-06T05:16:40.810 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T05:16:40.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T05:16:41.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.003+0000 7f02021ff700 1 Processor -- start 2023-10-06T05:16:41.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.003+0000 7f02021ff700 1 -- start start 2023-10-06T05:16:41.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f02021ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 0x7f01fc0c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f02021ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01fc155430 0x7f01fc1558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f02021ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01fc158190 con 0x7f01fc159c60 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f02021ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01fc157580 con 0x7f01fc155430 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f02021ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc157860 con 0x7f01fc159890 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fbfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01fc155430 0x7f01fc1558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 0x7f01fc0c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fbfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01fc155430 0x7f01fc1558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:35174/0 (socket says 172.21.15.31:35174) 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fbfff700 1 -- 172.21.15.31:0/2463950678 learned_addr learned my addr 172.21.15.31:0/2463950678 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01faffd700 1 --1- 172.21.15.31:0/2463950678 >> v1:172.21.15.62:6789/0 conn(0x7f01fc159c60 0x7f01fc156a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:33738/0 (socket says 172.21.15.31:33738) 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fbfff700 1 -- 172.21.15.31:0/2463950678 >> v2:172.21.15.62:3300/0 conn(0x7f01fc155430 msgr2=0x7f01fc1558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fbfff700 1 -- 172.21.15.31:0/2463950678 >> v2:172.21.15.62:3300/0 conn(0x7f01fc155430 msgr2=0x7f01fc1558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:16:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fbfff700 1 --2- 172.21.15.31:0/2463950678 >> v2:172.21.15.62:3300/0 conn(0x7f01fc155430 0x7f01fc1558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:16:41.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fb7fe700 1 -- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 msgr2=0x7f01fc0c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T05:16:41.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fb7fe700 1 -- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 msgr2=0x7f01fc0c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T05:16:41.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fb7fe700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 0x7f01fc0c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T05:16:41.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fbfff700 1 --2- 172.21.15.31:0/2463950678 >> v2:172.21.15.62:3300/0 conn(0x7f01fc155430 0x7f01fc1558b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:16:41.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.005+0000 7f01fb7fe700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 0x7f01fc0c89e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:16:41.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.156+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603163849 0 0) 0x7f01fc158190 con 0x7f01fc159c60 2023-10-06T05:16:41.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.156+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e4003570 con 0x7f01fc159c60 2023-10-06T05:16:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2200763787 0 0) 0x7f01e4003570 con 0x7f01fc159c60 2023-10-06T05:16:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 msgr2=0x7f01fc0c89e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:16:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 0x7f01fc0c89e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 >> v2:172.21.15.62:3300/0 conn(0x7f01fc155430 msgr2=0x7f01fc1558b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:16:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 --2- 172.21.15.31:0/2463950678 >> v2:172.21.15.62:3300/0 conn(0x7f01fc155430 0x7f01fc1558b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc157b40 con 0x7f01fc159c60 2023-10-06T05:16:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f01f0001470 con 0x7f01fc159c60 2023-10-06T05:16:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 >> v1:172.21.15.62:6789/0 conn(0x7f01fc159c60 legacy=0x7f01fc156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 0x7f01e4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 0x7f01e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01e4004d50 0x7f01e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc157580 con 0x7f01e4003ae0 2023-10-06T05:16:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc157860 con 0x7f01e4004d50 2023-10-06T05:16:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e400bcc0 con 0x7f01fc159890 2023-10-06T05:16:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fbfff700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01e4004d50 0x7f01e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01fb7fe700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 0x7f01e4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.158+0000 7f01faffd700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 0x7f01e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.159+0000 7f01fb7fe700 1 -- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 msgr2=0x7f01e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.159+0000 7f01fb7fe700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 0x7f01e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.159+0000 7f01fb7fe700 1 -- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01e4004d50 msgr2=0x7f01e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:41.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.159+0000 7f01fb7fe700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01e4004d50 0x7f01e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.159+0000 7f01fb7fe700 1 -- 172.21.15.31:0/2463950678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e4003570 con 0x7f01e4003ae0 2023-10-06T05:16:41.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.159+0000 7f01faffd700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc159890 0x7f01e4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:41.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.159+0000 7f01fb7fe700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 0x7f01e4003ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00bb40 tx=0x7f01ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be21f291d4f3a46c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.160+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec018070 con 0x7f01e4003ae0 2023-10-06T05:16:41.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.160+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec00d5d0 con 0x7f01e4003ae0 2023-10-06T05:16:41.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.160+0000 7f01fa7fc700 1 -- 172.21.15.31:0/2463950678 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00d990 con 0x7f01e4003ae0 2023-10-06T05:16:41.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.161+0000 7f02021ff700 1 -- 172.21.15.31:0/2463950678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 msgr2=0x7f01e4003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.161+0000 7f02021ff700 1 --2- 172.21.15.31:0/2463950678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 0x7f01e4003ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00bb40 tx=0x7f01ec008040 comp rx=0 tx=0).stop 2023-10-06T05:16:41.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.161+0000 7f01fbfff700 1 -- 172.21.15.31:0/2463950678 reap_dead start 2023-10-06T05:16:41.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.161+0000 7f02021ff700 1 -- 172.21.15.31:0/2463950678 shutdown_connections 2023-10-06T05:16:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.161+0000 7f02021ff700 1 -- 172.21.15.31:0/2463950678 >> 172.21.15.31:0/2463950678 conn(0x7f01fc067560 msgr2=0x7f01fc067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.161+0000 7f02021ff700 1 -- 172.21.15.31:0/2463950678 shutdown_connections 2023-10-06T05:16:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.161+0000 7f02021ff700 1 -- 172.21.15.31:0/2463950678 wait complete. 2023-10-06T05:16:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.163+0000 7f02021ff700 1 Processor -- start 2023-10-06T05:16:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.163+0000 7f02021ff700 1 -- start start 2023-10-06T05:16:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.165+0000 7f02021ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 0x7f01fc0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.165+0000 7f02021ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc14fe30 0x7f01fc0c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.165+0000 7f02021ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01fc1502a0 0x7f01fc16d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.165+0000 7f02021ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc160550 con 0x7f01e4003ae0 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.165+0000 7f02021ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc16d2d0 con 0x7f01fc1502a0 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.165+0000 7f02021ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0c6f50 con 0x7f01fc14fe30 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.165+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 0x7f01fc0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.165+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 0x7f01fc0c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57618/0 (socket says 172.21.15.31:57618) 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.165+0000 7f01fb7fe700 1 -- 172.21.15.31:0/3556328704 learned_addr learned my addr 172.21.15.31:0/3556328704 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.165+0000 7f01fbfff700 1 --2- 172.21.15.31:0/3556328704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01fc1502a0 0x7f01fc16d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.166+0000 7f01faffd700 1 --2- 172.21.15.31:0/3556328704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc14fe30 0x7f01fc0c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.166+0000 7f01fb7fe700 1 -- 172.21.15.31:0/3556328704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc14fe30 msgr2=0x7f01fc0c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.166+0000 7f01fb7fe700 1 --2- 172.21.15.31:0/3556328704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc14fe30 0x7f01fc0c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.166+0000 7f01fb7fe700 1 -- 172.21.15.31:0/3556328704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01fc1502a0 msgr2=0x7f01fc16d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.166+0000 7f01fb7fe700 1 --2- 172.21.15.31:0/3556328704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01fc1502a0 0x7f01fc16d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.166+0000 7f01fb7fe700 1 -- 172.21.15.31:0/3556328704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc15fcb0 con 0x7f01e4003ae0 2023-10-06T05:16:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.166+0000 7f01fb7fe700 1 --2- 172.21.15.31:0/3556328704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 0x7f01fc0c72c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b5e0 tx=0x7f01ec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.166+0000 7f01e37fe700 1 -- 172.21.15.31:0/3556328704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec01f640 con 0x7f01e4003ae0 2023-10-06T05:16:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.167+0000 7f01e37fe700 1 -- 172.21.15.31:0/3556328704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec01f7c0 con 0x7f01e4003ae0 2023-10-06T05:16:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.167+0000 7f01e37fe700 1 -- 172.21.15.31:0/3556328704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec01fb80 con 0x7f01e4003ae0 2023-10-06T05:16:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.167+0000 7f02021ff700 1 -- 172.21.15.31:0/3556328704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01fc15ff90 con 0x7f01e4003ae0 2023-10-06T05:16:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.167+0000 7f02021ff700 1 -- 172.21.15.31:0/3556328704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc0c7800 con 0x7f01e4003ae0 2023-10-06T05:16:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.167+0000 7f01e37fe700 1 -- 172.21.15.31:0/3556328704 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f01ec014430 con 0x7f01e4003ae0 2023-10-06T05:16:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.167+0000 7f01e37fe700 1 -- 172.21.15.31:0/3556328704 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f01ec027750 con 0x7f01e4003ae0 2023-10-06T05:16:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.168+0000 7f02021ff700 1 -- 172.21.15.31:0/3556328704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01fc16d0a0 con 0x7f01e4003ae0 2023-10-06T05:16:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.171+0000 7f01e37fe700 1 -- 172.21.15.31:0/3556328704 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f01fc16d0a0 con 0x7f01e4003ae0 2023-10-06T05:16:41.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.251+0000 7f02021ff700 1 -- 172.21.15.31:0/3556328704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f01fc04c380 con 0x7f01e4003ae0 2023-10-06T05:16:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.254+0000 7f01e37fe700 1 -- 172.21.15.31:0/3556328704 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f01ec02c020 con 0x7f01e4003ae0 2023-10-06T05:16:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.255+0000 7f02021ff700 1 -- 172.21.15.31:0/3556328704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 msgr2=0x7f01fc0c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.255+0000 7f02021ff700 1 --2- 172.21.15.31:0/3556328704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 0x7f01fc0c72c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b5e0 tx=0x7f01ec01a000 comp rx=0 tx=0).stop 2023-10-06T05:16:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.255+0000 7f02021ff700 1 -- 172.21.15.31:0/3556328704 shutdown_connections 2023-10-06T05:16:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.255+0000 7f02021ff700 1 --2- 172.21.15.31:0/3556328704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01fc1502a0 0x7f01fc16d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.255+0000 7f02021ff700 1 --2- 172.21.15.31:0/3556328704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01e4003ae0 0x7f01fc0c72c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.255+0000 7f02021ff700 1 --2- 172.21.15.31:0/3556328704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01fc14fe30 0x7f01fc0c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.255+0000 7f02021ff700 1 -- 172.21.15.31:0/3556328704 >> 172.21.15.31:0/3556328704 conn(0x7f01fc067560 msgr2=0x7f01fc0aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.255+0000 7f02021ff700 1 -- 172.21.15.31:0/3556328704 shutdown_connections 2023-10-06T05:16:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:41.255+0000 7f02021ff700 1 -- 172.21.15.31:0/3556328704 wait complete. 2023-10-06T05:16:41.258 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2023-10-06T05:16:41.274 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T05:16:41.275 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:41.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T05:16:41.312 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:41.311+0000 7f2eccc7e1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:16:41.324 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:41.322+0000 7efed4e341c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:16:41.351 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:41.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T05:16:41.422 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:41.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T05:16:41.519 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:41.517+0000 7f2eccc7e1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:16:41.539 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:41.537+0000 7efed4e341c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:16:41.554 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:16:41.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T05:16:41.626 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:41.626 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T05:16:41.697 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:41.697 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T05:16:41.807 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:41.808 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T05:16:41.923 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-10-06T05:16:41.923 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T05:16:41.962 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:41.960+0000 7efed4e341c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:16:41.992 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:41.990+0000 7f2eccc7e1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:16:42.041 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new d5ce3e1c-3a1c-41d0-add7-c2c866710e5c 0 2023-10-06T05:16:42.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.272+0000 7f746a9ff700 1 Processor -- start 2023-10-06T05:16:42.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.273+0000 7f746a9ff700 1 -- start start 2023-10-06T05:16:42.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f746a9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 0x7f74640cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f746a9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7464159930 0x7f74640cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f746a9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74640b0d60 0x7f74640b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f746a9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746415d2d0 con 0x7f74640b0d60 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f746a9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746415d5b0 con 0x7f7464159930 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f746a9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746415d890 con 0x7f7464159560 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f74637fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7464159930 0x7f74640cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f7463fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 0x7f74640cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f74637fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7464159930 0x7f74640cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56128/0 (socket says 172.21.15.62:56128) 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f74637fe700 1 -- 172.21.15.62:0/963264022 learned_addr learned my addr 172.21.15.62:0/963264022 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.274+0000 7f7468f9c700 1 --2- 172.21.15.62:0/963264022 >> v2:172.21.15.31:3300/0 conn(0x7f74640b0d60 0x7f74640b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.275+0000 7f74637fe700 1 -- 172.21.15.62:0/963264022 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 msgr2=0x7f74640cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.275+0000 7f74637fe700 1 --2- 172.21.15.62:0/963264022 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 0x7f74640cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.275+0000 7f74637fe700 1 -- 172.21.15.62:0/963264022 >> v2:172.21.15.31:3300/0 conn(0x7f74640b0d60 msgr2=0x7f74640b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.275+0000 7f74637fe700 1 --2- 172.21.15.62:0/963264022 >> v2:172.21.15.31:3300/0 conn(0x7f74640b0d60 0x7f74640b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.275+0000 7f74637fe700 1 -- 172.21.15.62:0/963264022 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746415db70 con 0x7f7464159930 2023-10-06T05:16:42.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.275+0000 7f74637fe700 1 --2- 172.21.15.62:0/963264022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7464159930 0x7f74640cd3b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f74640d03f0 tx=0x7f7458009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f122fb439ab3f57e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.275+0000 7f7462ffd700 1 -- 172.21.15.62:0/963264022 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745800b640 con 0x7f7464159930 2023-10-06T05:16:42.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.275+0000 7f7462ffd700 1 -- 172.21.15.62:0/963264022 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745800b7a0 con 0x7f7464159930 2023-10-06T05:16:42.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.275+0000 7f7462ffd700 1 -- 172.21.15.62:0/963264022 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745800bb60 con 0x7f7464159930 2023-10-06T05:16:42.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.276+0000 7f746a9ff700 1 -- 172.21.15.62:0/963264022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7464159930 msgr2=0x7f74640cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.276+0000 7f746a9ff700 1 --2- 172.21.15.62:0/963264022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7464159930 0x7f74640cd3b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f74640d03f0 tx=0x7f7458009250 comp rx=0 tx=0).stop 2023-10-06T05:16:42.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.276+0000 7f746a9ff700 1 -- 172.21.15.62:0/963264022 shutdown_connections 2023-10-06T05:16:42.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.276+0000 7f746a9ff700 1 --2- 172.21.15.62:0/963264022 >> v2:172.21.15.31:3300/0 conn(0x7f74640b0d60 0x7f74640b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.276+0000 7f746a9ff700 1 --2- 172.21.15.62:0/963264022 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 0x7f74640cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.276+0000 7f746a9ff700 1 --2- 172.21.15.62:0/963264022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7464159930 0x7f74640cd3b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.276+0000 7f746a9ff700 1 -- 172.21.15.62:0/963264022 >> 172.21.15.62:0/963264022 conn(0x7f746406ec90 msgr2=0x7f746406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.276+0000 7f746a9ff700 1 -- 172.21.15.62:0/963264022 shutdown_connections 2023-10-06T05:16:42.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.276+0000 7f746a9ff700 1 -- 172.21.15.62:0/963264022 wait complete. 2023-10-06T05:16:42.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.278+0000 7f746a9ff700 1 Processor -- start 2023-10-06T05:16:42.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.279+0000 7f746a9ff700 1 -- start start 2023-10-06T05:16:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.280+0000 7f746a9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74640b0d60 0x7f7464154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.280+0000 7f746a9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 0x7f746415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.280+0000 7f746a9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7464159930 0x7f7464150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.280+0000 7f746a9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7464154590 con 0x7f74640b0d60 2023-10-06T05:16:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.280+0000 7f746a9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746415acb0 con 0x7f7464159930 2023-10-06T05:16:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.280+0000 7f746a9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74641547c0 con 0x7f7464159560 2023-10-06T05:16:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f74637fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 0x7f746415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f7463fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74640b0d60 0x7f7464154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f7468f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7464159930 0x7f7464150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f74637fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 0x7f746415a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:39678/0 (socket says 172.21.15.62:39678) 2023-10-06T05:16:42.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f74637fe700 1 -- 172.21.15.62:0/410132722 learned_addr learned my addr 172.21.15.62:0/410132722 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:42.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f7463fff700 1 -- 172.21.15.62:0/410132722 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 msgr2=0x7f746415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f7463fff700 1 --2- 172.21.15.62:0/410132722 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 0x7f746415a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f7463fff700 1 -- 172.21.15.62:0/410132722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7464159930 msgr2=0x7f7464150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f7463fff700 1 --2- 172.21.15.62:0/410132722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7464159930 0x7f7464150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f7463fff700 1 -- 172.21.15.62:0/410132722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580094f0 con 0x7f74640b0d60 2023-10-06T05:16:42.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f7468f9c700 1 --2- 172.21.15.62:0/410132722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7464159930 0x7f7464150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:42.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.281+0000 7f7463fff700 1 --2- 172.21.15.62:0/410132722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74640b0d60 0x7f7464154050 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f745400ea50 tx=0x7f74540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.282+0000 7f744bfff700 1 -- 172.21.15.62:0/410132722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745400ee90 con 0x7f74640b0d60 2023-10-06T05:16:42.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.282+0000 7f744bfff700 1 -- 172.21.15.62:0/410132722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745400c7f0 con 0x7f74640b0d60 2023-10-06T05:16:42.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.282+0000 7f744bfff700 1 -- 172.21.15.62:0/410132722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745400cb90 con 0x7f74640b0d60 2023-10-06T05:16:42.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.282+0000 7f746a9ff700 1 -- 172.21.15.62:0/410132722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7464151530 con 0x7f74640b0d60 2023-10-06T05:16:42.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.282+0000 7f746a9ff700 1 -- 172.21.15.62:0/410132722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7464151af0 con 0x7f74640b0d60 2023-10-06T05:16:42.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.282+0000 7f744bfff700 1 -- 172.21.15.62:0/410132722 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7454005c80 con 0x7f74640b0d60 2023-10-06T05:16:42.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.282+0000 7f744bfff700 1 -- 172.21.15.62:0/410132722 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7454013470 con 0x7f74640b0d60 2023-10-06T05:16:42.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.283+0000 7f746a9ff700 1 -- 172.21.15.62:0/410132722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7464152140 con 0x7f74640b0d60 2023-10-06T05:16:42.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.285+0000 7f744bfff700 1 -- 172.21.15.62:0/410132722 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f7454021060 con 0x7f74640b0d60 2023-10-06T05:16:42.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.374+0000 7f746a9ff700 1 -- 172.21.15.62:0/410132722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d5ce3e1c-3a1c-41d0-add7-c2c866710e5c", "id": 0} v 0) v1 -- 0x7f746415de50 con 0x7f74640b0d60 2023-10-06T05:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.381+0000 7f744bfff700 1 -- 172.21.15.62:0/410132722 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d5ce3e1c-3a1c-41d0-add7-c2c866710e5c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7454017070 con 0x7f74640b0d60 2023-10-06T05:16:42.383 INFO:teuthology.orchestra.run.smithi062.stdout:0 2023-10-06T05:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.382+0000 7f746a9ff700 1 -- 172.21.15.62:0/410132722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74640b0d60 msgr2=0x7f7464154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.382+0000 7f746a9ff700 1 --2- 172.21.15.62:0/410132722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74640b0d60 0x7f7464154050 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f745400ea50 tx=0x7f74540091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.383+0000 7f746a9ff700 1 -- 172.21.15.62:0/410132722 shutdown_connections 2023-10-06T05:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.383+0000 7f746a9ff700 1 --2- 172.21.15.62:0/410132722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7464159930 0x7f7464150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.383+0000 7f746a9ff700 1 --2- 172.21.15.62:0/410132722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74640b0d60 0x7f7464154050 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.383+0000 7f746a9ff700 1 --2- 172.21.15.62:0/410132722 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7464159560 0x7f746415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.383+0000 7f746a9ff700 1 -- 172.21.15.62:0/410132722 >> 172.21.15.62:0/410132722 conn(0x7f746406ec90 msgr2=0x7f74640b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.383+0000 7f746a9ff700 1 -- 172.21.15.62:0/410132722 shutdown_connections 2023-10-06T05:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.383+0000 7f746a9ff700 1 -- 172.21.15.62:0/410132722 wait complete. 2023-10-06T05:16:42.403 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 171d8e7f-7775-4532-a010-d9e0a9c28b12 1 2023-10-06T05:16:42.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.590+0000 7f95563ff700 1 Processor -- start 2023-10-06T05:16:42.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.590+0000 7f95563ff700 1 -- start start 2023-10-06T05:16:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.591+0000 7f95563ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95500c52d0 0x7f95500cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.591+0000 7f95563ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500cf110 0x7f95500cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.591+0000 7f95563ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95500cb800 con 0x7f95500af590 2023-10-06T05:16:42.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.591+0000 7f95563ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95500cb280 con 0x7f95500c52d0 2023-10-06T05:16:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.591+0000 7f95563ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500cf590 con 0x7f95500cf110 2023-10-06T05:16:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.591+0000 7f955499c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500cf110 0x7f95500cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.591+0000 7f954f7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95500c52d0 0x7f95500cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.591+0000 7f955499c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500cf110 0x7f95500cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:39690/0 (socket says 172.21.15.62:39690) 2023-10-06T05:16:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.591+0000 7f955499c700 1 -- 172.21.15.62:0/3331943493 learned_addr learned my addr 172.21.15.62:0/3331943493 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:42.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.591+0000 7f954f7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95500c52d0 0x7f95500cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56154/0 (socket says 172.21.15.62:56154) 2023-10-06T05:16:42.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.592+0000 7f955499c700 1 -- 172.21.15.62:0/3331943493 >> v2:172.21.15.62:3300/0 conn(0x7f95500c52d0 msgr2=0x7f95500cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.592+0000 7f955499c700 1 --2- 172.21.15.62:0/3331943493 >> v2:172.21.15.62:3300/0 conn(0x7f95500c52d0 0x7f95500cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.592+0000 7f955499c700 1 -- 172.21.15.62:0/3331943493 >> v1:172.21.15.62:6789/0 conn(0x7f95500af590 legacy=0x7f95500c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.592+0000 7f955499c700 1 -- 172.21.15.62:0/3331943493 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500c6870 con 0x7f95500cf110 2023-10-06T05:16:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.592+0000 7f954f7fe700 1 --2- 172.21.15.62:0/3331943493 >> v2:172.21.15.62:3300/0 conn(0x7f95500c52d0 0x7f95500cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.592+0000 7f955499c700 1 --2- 172.21.15.62:0/3331943493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500cf110 0x7f95500cbc90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9548009f20 tx=0x7f9548009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50c8dfc624945ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.592+0000 7f954effd700 1 -- 172.21.15.62:0/3331943493 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954800e070 con 0x7f95500cf110 2023-10-06T05:16:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.592+0000 7f954effd700 1 -- 172.21.15.62:0/3331943493 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954800b9d0 con 0x7f95500cf110 2023-10-06T05:16:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.593+0000 7f954effd700 1 -- 172.21.15.62:0/3331943493 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95480092e0 con 0x7f95500cf110 2023-10-06T05:16:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.593+0000 7f95563ff700 1 -- 172.21.15.62:0/3331943493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500cf110 msgr2=0x7f95500cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.593+0000 7f95563ff700 1 --2- 172.21.15.62:0/3331943493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500cf110 0x7f95500cbc90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9548009f20 tx=0x7f9548009840 comp rx=0 tx=0).stop 2023-10-06T05:16:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.593+0000 7f95563ff700 1 -- 172.21.15.62:0/3331943493 shutdown_connections 2023-10-06T05:16:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.593+0000 7f95563ff700 1 --2- 172.21.15.62:0/3331943493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500cf110 0x7f95500cbc90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.593+0000 7f95563ff700 1 --2- 172.21.15.62:0/3331943493 >> v2:172.21.15.62:3300/0 conn(0x7f95500c52d0 0x7f95500cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.593+0000 7f95563ff700 1 -- 172.21.15.62:0/3331943493 >> 172.21.15.62:0/3331943493 conn(0x7f95500996f0 msgr2=0x7f955006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.593+0000 7f95563ff700 1 -- 172.21.15.62:0/3331943493 shutdown_connections 2023-10-06T05:16:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.593+0000 7f95563ff700 1 -- 172.21.15.62:0/3331943493 wait complete. 2023-10-06T05:16:42.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.595+0000 7f95563ff700 1 Processor -- start 2023-10-06T05:16:42.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.595+0000 7f95563ff700 1 -- start start 2023-10-06T05:16:42.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f95563ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500af590 0x7f9550169a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f95563ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95500c52d0 0x7f95500b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f95563ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95500cf110 0x7f9550155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f95563ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500c75e0 con 0x7f95500cf110 2023-10-06T05:16:42.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f95563ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500b1070 con 0x7f95500c52d0 2023-10-06T05:16:42.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f95563ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500b1350 con 0x7f95500af590 2023-10-06T05:16:42.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95500c52d0 0x7f95500b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f954ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500af590 0x7f9550169a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95500c52d0 0x7f95500b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56168/0 (socket says 172.21.15.62:56168) 2023-10-06T05:16:42.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f954f7fe700 1 -- 172.21.15.62:0/2689188320 learned_addr learned my addr 172.21.15.62:0/2689188320 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.596+0000 7f955499c700 1 --2- 172.21.15.62:0/2689188320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95500cf110 0x7f9550155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f954ffff700 1 -- 172.21.15.62:0/2689188320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95500c52d0 msgr2=0x7f95500b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f954ffff700 1 --2- 172.21.15.62:0/2689188320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95500c52d0 0x7f95500b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f954ffff700 1 -- 172.21.15.62:0/2689188320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95500cf110 msgr2=0x7f9550155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f954ffff700 1 --2- 172.21.15.62:0/2689188320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95500cf110 0x7f9550155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f954ffff700 1 -- 172.21.15.62:0/2689188320 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9544009420 con 0x7f95500af590 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f955499c700 1 --2- 172.21.15.62:0/2689188320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95500cf110 0x7f9550155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f954f7fe700 1 --2- 172.21.15.62:0/2689188320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95500c52d0 0x7f95500b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f954ffff700 1 --2- 172.21.15.62:0/2689188320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500af590 0x7f9550169a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f954000c4c0 tx=0x7f9540007ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f9537fff700 1 -- 172.21.15.62:0/2689188320 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954000f470 con 0x7f95500af590 2023-10-06T05:16:42.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f9537fff700 1 -- 172.21.15.62:0/2689188320 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954000f5f0 con 0x7f95500af590 2023-10-06T05:16:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f95563ff700 1 -- 172.21.15.62:0/2689188320 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9548009ae0 con 0x7f95500af590 2023-10-06T05:16:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f9537fff700 1 -- 172.21.15.62:0/2689188320 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954000f9b0 con 0x7f95500af590 2023-10-06T05:16:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f95563ff700 1 -- 172.21.15.62:0/2689188320 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95500b1910 con 0x7f95500af590 2023-10-06T05:16:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.597+0000 7f9537fff700 1 -- 172.21.15.62:0/2689188320 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9540012400 con 0x7f95500af590 2023-10-06T05:16:42.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.598+0000 7f9537fff700 1 -- 172.21.15.62:0/2689188320 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f9540012a70 con 0x7f95500af590 2023-10-06T05:16:42.602 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:42.599+0000 7efed4e341c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:16:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.598+0000 7f95563ff700 1 -- 172.21.15.62:0/2689188320 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95500b1b00 con 0x7f95500af590 2023-10-06T05:16:42.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.601+0000 7f9537fff700 1 -- 172.21.15.62:0/2689188320 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f95500b1b00 con 0x7f95500af590 2023-10-06T05:16:42.654 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:42.653+0000 7f2eccc7e1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:16:42.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.688+0000 7f95563ff700 1 -- 172.21.15.62:0/2689188320 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "171d8e7f-7775-4532-a010-d9e0a9c28b12", "id": 1} v 0) v1 -- 0x7f955004c380 con 0x7f95500af590 2023-10-06T05:16:42.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.696+0000 7f9537fff700 1 -- 172.21.15.62:0/2689188320 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "171d8e7f-7775-4532-a010-d9e0a9c28b12", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9540021060 con 0x7f95500af590 2023-10-06T05:16:42.698 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:16:42.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.697+0000 7f95563ff700 1 -- 172.21.15.62:0/2689188320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500af590 msgr2=0x7f9550169a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.697+0000 7f95563ff700 1 --2- 172.21.15.62:0/2689188320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500af590 0x7f9550169a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f954000c4c0 tx=0x7f9540007ea0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.697+0000 7f95563ff700 1 -- 172.21.15.62:0/2689188320 shutdown_connections 2023-10-06T05:16:42.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.697+0000 7f95563ff700 1 --2- 172.21.15.62:0/2689188320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95500cf110 0x7f9550155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.697+0000 7f95563ff700 1 --2- 172.21.15.62:0/2689188320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95500af590 0x7f9550169a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.697+0000 7f95563ff700 1 --2- 172.21.15.62:0/2689188320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95500c52d0 0x7f95500b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.697+0000 7f95563ff700 1 -- 172.21.15.62:0/2689188320 >> 172.21.15.62:0/2689188320 conn(0x7f95500996f0 msgr2=0x7f9550156c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.697+0000 7f95563ff700 1 -- 172.21.15.62:0/2689188320 shutdown_connections 2023-10-06T05:16:42.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.697+0000 7f95563ff700 1 -- 172.21.15.62:0/2689188320 wait complete. 2023-10-06T05:16:42.715 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 128ab23f-d6cc-48db-bd7f-fc80834fcf18 2 2023-10-06T05:16:42.777 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:42.775+0000 7efed4e341c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:16:42.835 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:42.833+0000 7f2eccc7e1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:16:42.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.905+0000 7f4c185ff700 1 Processor -- start 2023-10-06T05:16:42.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.905+0000 7f4c185ff700 1 -- start start 2023-10-06T05:16:42.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.906+0000 7f4c185ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 0x7f4c100af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.906+0000 7f4c185ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4c10159760 0x7f4c100aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.906+0000 7f4c185ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c100ac730 con 0x7f4c100ab510 2023-10-06T05:16:42.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.906+0000 7f4c185ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c100af770 con 0x7f4c10159760 2023-10-06T05:16:42.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.906+0000 7f4c185ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c100abb20 con 0x7f4c10159390 2023-10-06T05:16:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.906+0000 7f4c15b9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4c10159760 0x7f4c100aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.906+0000 7f4c1639b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 0x7f4c100af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.907+0000 7f4c15b9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4c10159760 0x7f4c100aafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56174/0 (socket says 172.21.15.62:56174) 2023-10-06T05:16:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.907+0000 7f4c1639b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 0x7f4c100af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:39706/0 (socket says 172.21.15.62:39706) 2023-10-06T05:16:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.907+0000 7f4c15b9a700 1 -- 172.21.15.62:0/2098141700 learned_addr learned my addr 172.21.15.62:0/2098141700 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.907+0000 7f4c15b9a700 1 -- 172.21.15.62:0/2098141700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 msgr2=0x7f4c100af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.907+0000 7f4c15b9a700 1 --2- 172.21.15.62:0/2098141700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 0x7f4c100af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.907+0000 7f4c15b9a700 1 -- 172.21.15.62:0/2098141700 >> v1:172.21.15.62:6789/0 conn(0x7f4c100ab510 legacy=0x7f4c100cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.907+0000 7f4c15b9a700 1 -- 172.21.15.62:0/2098141700 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c100abe00 con 0x7f4c10159760 2023-10-06T05:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.907+0000 7f4c15b9a700 1 --2- 172.21.15.62:0/2098141700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c10159760 0x7f4c100aafd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4c000099b0 tx=0x7f4c00009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f84de7daa42de724 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.907+0000 7f4c15399700 1 -- 172.21.15.62:0/2098141700 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00009ea0 con 0x7f4c10159760 2023-10-06T05:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.908+0000 7f4c15399700 1 -- 172.21.15.62:0/2098141700 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0000b860 con 0x7f4c10159760 2023-10-06T05:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.908+0000 7f4c15399700 1 -- 172.21.15.62:0/2098141700 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000bc00 con 0x7f4c10159760 2023-10-06T05:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.908+0000 7f4c185ff700 1 -- 172.21.15.62:0/2098141700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c10159760 msgr2=0x7f4c100aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.908+0000 7f4c185ff700 1 --2- 172.21.15.62:0/2098141700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c10159760 0x7f4c100aafd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4c000099b0 tx=0x7f4c00009250 comp rx=0 tx=0).stop 2023-10-06T05:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.908+0000 7f4c185ff700 1 -- 172.21.15.62:0/2098141700 shutdown_connections 2023-10-06T05:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.908+0000 7f4c185ff700 1 --2- 172.21.15.62:0/2098141700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 0x7f4c100af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.908+0000 7f4c185ff700 1 --2- 172.21.15.62:0/2098141700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c10159760 0x7f4c100aafd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.908+0000 7f4c185ff700 1 -- 172.21.15.62:0/2098141700 >> 172.21.15.62:0/2098141700 conn(0x7f4c10067560 msgr2=0x7f4c10067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:42.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.908+0000 7f4c185ff700 1 -- 172.21.15.62:0/2098141700 shutdown_connections 2023-10-06T05:16:42.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.908+0000 7f4c185ff700 1 -- 172.21.15.62:0/2098141700 wait complete. 2023-10-06T05:16:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.910+0000 7f4c185ff700 1 Processor -- start 2023-10-06T05:16:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.910+0000 7f4c185ff700 1 -- start start 2023-10-06T05:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c185ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c100ab510 0x7f4c1014c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c185ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 0x7f4c1016a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c185ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c10159760 0x7f4c1016ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c185ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c10149690 con 0x7f4c10159760 2023-10-06T05:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c185ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1016d770 con 0x7f4c100ab510 2023-10-06T05:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c185ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1016d9e0 con 0x7f4c10159390 2023-10-06T05:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c1639b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c100ab510 0x7f4c1014c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c15b9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 0x7f4c1016a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c1639b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c100ab510 0x7f4c1014c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56186/0 (socket says 172.21.15.62:56186) 2023-10-06T05:16:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c1639b700 1 -- 172.21.15.62:0/3566670211 learned_addr learned my addr 172.21.15.62:0/3566670211 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c16b9c700 1 --2- 172.21.15.62:0/3566670211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c10159760 0x7f4c1016ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c1639b700 1 -- 172.21.15.62:0/3566670211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 msgr2=0x7f4c1016a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.911+0000 7f4c1639b700 1 --2- 172.21.15.62:0/3566670211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 0x7f4c1016a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.912+0000 7f4c1639b700 1 -- 172.21.15.62:0/3566670211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c10159760 msgr2=0x7f4c1016ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.912+0000 7f4c1639b700 1 --2- 172.21.15.62:0/3566670211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c10159760 0x7f4c1016ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.912+0000 7f4c1639b700 1 -- 172.21.15.62:0/3566670211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c000094f0 con 0x7f4c100ab510 2023-10-06T05:16:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.912+0000 7f4c1639b700 1 --2- 172.21.15.62:0/3566670211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c100ab510 0x7f4c1014c990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00ec80 tx=0x7f4c0c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.912+0000 7f4c067fc700 1 -- 172.21.15.62:0/3566670211 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00c850 con 0x7f4c100ab510 2023-10-06T05:16:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.912+0000 7f4c067fc700 1 -- 172.21.15.62:0/3566670211 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c00c9b0 con 0x7f4c100ab510 2023-10-06T05:16:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.912+0000 7f4c067fc700 1 -- 172.21.15.62:0/3566670211 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00cd50 con 0x7f4c100ab510 2023-10-06T05:16:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.912+0000 7f4c185ff700 1 -- 172.21.15.62:0/3566670211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1016dcc0 con 0x7f4c100ab510 2023-10-06T05:16:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.912+0000 7f4c185ff700 1 -- 172.21.15.62:0/3566670211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1014d460 con 0x7f4c100ab510 2023-10-06T05:16:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.912+0000 7f4c067fc700 1 -- 172.21.15.62:0/3566670211 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4c0c013900 con 0x7f4c100ab510 2023-10-06T05:16:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.913+0000 7f4c067fc700 1 -- 172.21.15.62:0/3566670211 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f4c0c005a00 con 0x7f4c100ab510 2023-10-06T05:16:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.913+0000 7f4c185ff700 1 -- 172.21.15.62:0/3566670211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1016e310 con 0x7f4c100ab510 2023-10-06T05:16:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:42.916+0000 7f4c067fc700 1 -- 172.21.15.62:0/3566670211 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4c0c03b6d0 con 0x7f4c100ab510 2023-10-06T05:16:43.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.004+0000 7f4c185ff700 1 -- 172.21.15.62:0/3566670211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "128ab23f-d6cc-48db-bd7f-fc80834fcf18", "id": 2} v 0) v1 -- 0x7f4c1004c380 con 0x7f4c100ab510 2023-10-06T05:16:43.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.010+0000 7f4c067fc700 1 -- 172.21.15.62:0/3566670211 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "128ab23f-d6cc-48db-bd7f-fc80834fcf18", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4c0c02a070 con 0x7f4c100ab510 2023-10-06T05:16:43.012 INFO:teuthology.orchestra.run.smithi062.stdout:2 2023-10-06T05:16:43.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.011+0000 7f4c185ff700 1 -- 172.21.15.62:0/3566670211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c100ab510 msgr2=0x7f4c1014c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.011+0000 7f4c185ff700 1 --2- 172.21.15.62:0/3566670211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c100ab510 0x7f4c1014c990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00ec80 tx=0x7f4c0c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.011+0000 7f4c185ff700 1 -- 172.21.15.62:0/3566670211 shutdown_connections 2023-10-06T05:16:43.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.011+0000 7f4c185ff700 1 --2- 172.21.15.62:0/3566670211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c10159760 0x7f4c1016ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.011+0000 7f4c185ff700 1 --2- 172.21.15.62:0/3566670211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c100ab510 0x7f4c1014c990 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.011+0000 7f4c185ff700 1 --2- 172.21.15.62:0/3566670211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c10159390 0x7f4c1016a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.011+0000 7f4c185ff700 1 -- 172.21.15.62:0/3566670211 >> 172.21.15.62:0/3566670211 conn(0x7f4c10067560 msgr2=0x7f4c101532e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:43.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.011+0000 7f4c185ff700 1 -- 172.21.15.62:0/3566670211 shutdown_connections 2023-10-06T05:16:43.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.011+0000 7f4c185ff700 1 -- 172.21.15.62:0/3566670211 wait complete. 2023-10-06T05:16:43.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new a7b8fafe-138c-48fa-93bf-86376c93ee02 3 2023-10-06T05:16:43.207 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:43.205+0000 7efed4e341c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:16:43.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.220+0000 7f5dd15ff700 1 Processor -- start 2023-10-06T05:16:43.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.220+0000 7f5dd15ff700 1 -- start start 2023-10-06T05:16:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.221+0000 7f5dd15ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5dcc1573a0 0x7f5dcc0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.221+0000 7f5dd15ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc158cf0 con 0x7f5dcc0ae860 2023-10-06T05:16:43.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.221+0000 7f5dd15ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc1580e0 con 0x7f5dcc157770 2023-10-06T05:16:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.221+0000 7f5dd15ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc1583c0 con 0x7f5dcc1573a0 2023-10-06T05:16:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.222+0000 7f5dca7fc700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5dcc157770 0x7f5dcc0b1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:43064/0 (socket says 172.21.15.62:43064) 2023-10-06T05:16:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.222+0000 7f5dca7fc700 1 -- 172.21.15.62:0/1011920606 learned_addr learned my addr 172.21.15.62:0/1011920606 (peer_addr_for_me v1:172.21.15.62:0/0) 2023-10-06T05:16:43.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.222+0000 7f5dcaffd700 1 --2- 172.21.15.62:0/1011920606 >> v2:172.21.15.31:3300/0 conn(0x7f5dcc1573a0 0x7f5dcc0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.222+0000 7f5dc9ffb700 1 -- 172.21.15.62:0/1011920606 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343652454 0 0) 0x7f5dcc1580e0 con 0x7f5dcc157770 2023-10-06T05:16:43.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.222+0000 7f5dc9ffb700 1 -- 172.21.15.62:0/1011920606 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db4003570 con 0x7f5dcc157770 2023-10-06T05:16:43.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.222+0000 7f5dcaffd700 1 -- 172.21.15.62:0/1011920606 >> v1:172.21.15.62:6789/0 conn(0x7f5dcc157770 legacy=0x7f5dcc0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.222+0000 7f5dcaffd700 1 -- 172.21.15.62:0/1011920606 >> v1:172.21.15.31:6789/0 conn(0x7f5dcc0ae860 legacy=0x7f5dcc1611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.222+0000 7f5dcaffd700 1 -- 172.21.15.62:0/1011920606 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc1586a0 con 0x7f5dcc1573a0 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.223+0000 7f5dcaffd700 1 --2- 172.21.15.62:0/1011920606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc1573a0 0x7f5dcc0acb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0098e0 tx=0x7f5dbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e39c8774e4f1e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.223+0000 7f5dc9ffb700 1 -- 172.21.15.62:0/1011920606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00b640 con 0x7f5dcc1573a0 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.223+0000 7f5dc9ffb700 1 -- 172.21.15.62:0/1011920606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc00b7a0 con 0x7f5dcc1573a0 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.223+0000 7f5dc9ffb700 1 -- 172.21.15.62:0/1011920606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00bb40 con 0x7f5dcc1573a0 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.223+0000 7f5dd15ff700 1 -- 172.21.15.62:0/1011920606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc1573a0 msgr2=0x7f5dcc0acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.223+0000 7f5dd15ff700 1 --2- 172.21.15.62:0/1011920606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc1573a0 0x7f5dcc0acb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0098e0 tx=0x7f5dbc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.223+0000 7f5dd15ff700 1 -- 172.21.15.62:0/1011920606 shutdown_connections 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.223+0000 7f5dd15ff700 1 --2- 172.21.15.62:0/1011920606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc1573a0 0x7f5dcc0acb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.224+0000 7f5dd15ff700 1 -- 172.21.15.62:0/1011920606 >> 172.21.15.62:0/1011920606 conn(0x7f5dcc067560 msgr2=0x7f5dcc067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:43.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.224+0000 7f5dd15ff700 1 -- 172.21.15.62:0/1011920606 shutdown_connections 2023-10-06T05:16:43.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.224+0000 7f5dd15ff700 1 -- 172.21.15.62:0/1011920606 wait complete. 2023-10-06T05:16:43.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.226+0000 7f5dd15ff700 1 Processor -- start 2023-10-06T05:16:43.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.226+0000 7f5dd15ff700 1 -- start start 2023-10-06T05:16:43.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.227+0000 7f5dd15ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dcc1573a0 0x7f5dcc1501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.227+0000 7f5dd15ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dcc157770 0x7f5dcc15b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.227+0000 7f5dd15ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc15bdf0 0x7f5dcc155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.227+0000 7f5dd15ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc154db0 con 0x7f5dcc15bdf0 2023-10-06T05:16:43.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.227+0000 7f5dd15ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc150710 con 0x7f5dcc1573a0 2023-10-06T05:16:43.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.227+0000 7f5dd15ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc15c260 con 0x7f5dcc157770 2023-10-06T05:16:43.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.227+0000 7f5dcaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dcc1573a0 0x7f5dcc1501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.227+0000 7f5dcaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dcc1573a0 0x7f5dcc1501d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56194/0 (socket says 172.21.15.62:56194) 2023-10-06T05:16:43.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.228+0000 7f5dca7fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dcc157770 0x7f5dcc15b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.228+0000 7f5dcaffd700 1 -- 172.21.15.62:0/2717871002 learned_addr learned my addr 172.21.15.62:0/2717871002 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:43.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.228+0000 7f5dcb7fe700 1 --2- 172.21.15.62:0/2717871002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc15bdf0 0x7f5dcc155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.228+0000 7f5dcaffd700 1 -- 172.21.15.62:0/2717871002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dcc157770 msgr2=0x7f5dcc15b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.228+0000 7f5dcaffd700 1 --2- 172.21.15.62:0/2717871002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dcc157770 0x7f5dcc15b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.228+0000 7f5dcaffd700 1 -- 172.21.15.62:0/2717871002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc15bdf0 msgr2=0x7f5dcc155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.228+0000 7f5dcaffd700 1 --2- 172.21.15.62:0/2717871002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc15bdf0 0x7f5dcc155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.228+0000 7f5dcaffd700 1 -- 172.21.15.62:0/2717871002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc009420 con 0x7f5dcc1573a0 2023-10-06T05:16:43.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.228+0000 7f5dcaffd700 1 --2- 172.21.15.62:0/2717871002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dcc1573a0 0x7f5dcc1501d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc064290 tx=0x7f5dbc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:43.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.228+0000 7f5db2ffd700 1 -- 172.21.15.62:0/2717871002 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc000d10 con 0x7f5dcc1573a0 2023-10-06T05:16:43.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.229+0000 7f5db2ffd700 1 -- 172.21.15.62:0/2717871002 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc00bb20 con 0x7f5dcc1573a0 2023-10-06T05:16:43.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.229+0000 7f5db2ffd700 1 -- 172.21.15.62:0/2717871002 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc011650 con 0x7f5dcc1573a0 2023-10-06T05:16:43.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.229+0000 7f5dd15ff700 1 -- 172.21.15.62:0/2717871002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc15c540 con 0x7f5dcc1573a0 2023-10-06T05:16:43.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.229+0000 7f5dd15ff700 1 -- 172.21.15.62:0/2717871002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc154a40 con 0x7f5dcc1573a0 2023-10-06T05:16:43.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.229+0000 7f5db2ffd700 1 -- 172.21.15.62:0/2717871002 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5dbc011c40 con 0x7f5dcc1573a0 2023-10-06T05:16:43.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.229+0000 7f5db2ffd700 1 -- 172.21.15.62:0/2717871002 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f5dbc025df0 con 0x7f5dcc1573a0 2023-10-06T05:16:43.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.230+0000 7f5dd15ff700 1 -- 172.21.15.62:0/2717871002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc04c380 con 0x7f5dcc1573a0 2023-10-06T05:16:43.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.232+0000 7f5db2ffd700 1 -- 172.21.15.62:0/2717871002 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5dbc029070 con 0x7f5dcc1573a0 2023-10-06T05:16:43.282 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:43.280+0000 7f2eccc7e1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:16:43.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.320+0000 7f5dd15ff700 1 -- 172.21.15.62:0/2717871002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a7b8fafe-138c-48fa-93bf-86376c93ee02", "id": 3} v 0) v1 -- 0x7f5dcc0b1fb0 con 0x7f5dcc1573a0 2023-10-06T05:16:43.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.328+0000 7f5db2ffd700 1 -- 172.21.15.62:0/2717871002 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7b8fafe-138c-48fa-93bf-86376c93ee02", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5dbc015020 con 0x7f5dcc1573a0 2023-10-06T05:16:43.330 INFO:teuthology.orchestra.run.smithi062.stdout:3 2023-10-06T05:16:43.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.329+0000 7f5dd15ff700 1 -- 172.21.15.62:0/2717871002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dcc1573a0 msgr2=0x7f5dcc1501d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.329+0000 7f5dd15ff700 1 --2- 172.21.15.62:0/2717871002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dcc1573a0 0x7f5dcc1501d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc064290 tx=0x7f5dbc017000 comp rx=0 tx=0).stop 2023-10-06T05:16:43.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.329+0000 7f5dd15ff700 1 -- 172.21.15.62:0/2717871002 shutdown_connections 2023-10-06T05:16:43.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.329+0000 7f5dd15ff700 1 --2- 172.21.15.62:0/2717871002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dcc15bdf0 0x7f5dcc155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.329+0000 7f5dd15ff700 1 --2- 172.21.15.62:0/2717871002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dcc1573a0 0x7f5dcc1501d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.329+0000 7f5dd15ff700 1 --2- 172.21.15.62:0/2717871002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dcc157770 0x7f5dcc15b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.329+0000 7f5dd15ff700 1 -- 172.21.15.62:0/2717871002 >> 172.21.15.62:0/2717871002 conn(0x7f5dcc067560 msgr2=0x7f5dcc069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:43.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.329+0000 7f5dd15ff700 1 -- 172.21.15.62:0/2717871002 shutdown_connections 2023-10-06T05:16:43.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.329+0000 7f5dd15ff700 1 -- 172.21.15.62:0/2717871002 wait complete. 2023-10-06T05:16:43.349 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 19501bab-f99b-4ea0-9a75-455fd46c87ff 4 2023-10-06T05:16:43.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.537+0000 7f86695ff700 1 Processor -- start 2023-10-06T05:16:43.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.537+0000 7f86695ff700 1 -- start start 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.538+0000 7f86695ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8664159930 0x7f86640b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.538+0000 7f86695ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86640cd580 0x7f86640b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.538+0000 7f86695ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866415d2d0 con 0x7f8664159560 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.538+0000 7f86695ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866415d5b0 con 0x7f8664159930 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.538+0000 7f86695ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866415d890 con 0x7f86640cd580 2023-10-06T05:16:43.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.539+0000 7f86637fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86640cd580 0x7f86640b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.539+0000 7f86637fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86640cd580 0x7f86640b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56208/0 (socket says 172.21.15.62:56208) 2023-10-06T05:16:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.539+0000 7f86637fe700 1 -- 172.21.15.62:0/1576450510 learned_addr learned my addr 172.21.15.62:0/1576450510 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.539+0000 7f86627fc700 1 --2- 172.21.15.62:0/1576450510 >> v2:172.21.15.31:3300/0 conn(0x7f8664159930 0x7f86640b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.539+0000 7f86637fe700 1 -- 172.21.15.62:0/1576450510 >> v2:172.21.15.31:3300/0 conn(0x7f8664159930 msgr2=0x7f86640b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.539+0000 7f86637fe700 1 --2- 172.21.15.62:0/1576450510 >> v2:172.21.15.31:3300/0 conn(0x7f8664159930 0x7f86640b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.539+0000 7f86637fe700 1 -- 172.21.15.62:0/1576450510 >> v1:172.21.15.31:6789/0 conn(0x7f8664159560 legacy=0x7f86640cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.539+0000 7f86637fe700 1 -- 172.21.15.62:0/1576450510 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866415db70 con 0x7f86640cd580 2023-10-06T05:16:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.539+0000 7f8661ffb700 1 -- 172.21.15.62:0/1576450510 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771094340 0 0) 0x7f866415d2d0 con 0x7f8664159560 2023-10-06T05:16:43.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.539+0000 7f86637fe700 1 --2- 172.21.15.62:0/1576450510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86640cd580 0x7f86640b29b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f86640b1740 tx=0x7f865c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63b3272cfed70885 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f8661ffb700 1 -- 172.21.15.62:0/1576450510 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c00e070 con 0x7f86640cd580 2023-10-06T05:16:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f8661ffb700 1 -- 172.21.15.62:0/1576450510 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c00ba20 con 0x7f86640cd580 2023-10-06T05:16:43.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f8661ffb700 1 -- 172.21.15.62:0/1576450510 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0092e0 con 0x7f86640cd580 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f86695ff700 1 -- 172.21.15.62:0/1576450510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86640cd580 msgr2=0x7f86640b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f86695ff700 1 --2- 172.21.15.62:0/1576450510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86640cd580 0x7f86640b29b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f86640b1740 tx=0x7f865c009840 comp rx=0 tx=0).stop 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f86695ff700 1 -- 172.21.15.62:0/1576450510 shutdown_connections 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f86695ff700 1 --2- 172.21.15.62:0/1576450510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86640cd580 0x7f86640b29b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f86695ff700 1 --2- 172.21.15.62:0/1576450510 >> v2:172.21.15.31:3300/0 conn(0x7f8664159930 0x7f86640b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f86695ff700 1 -- 172.21.15.62:0/1576450510 >> 172.21.15.62:0/1576450510 conn(0x7f866406ec90 msgr2=0x7f866406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f86695ff700 1 -- 172.21.15.62:0/1576450510 shutdown_connections 2023-10-06T05:16:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.540+0000 7f86695ff700 1 -- 172.21.15.62:0/1576450510 wait complete. 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.543+0000 7f86695ff700 1 Processor -- start 2023-10-06T05:16:43.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.543+0000 7f86695ff700 1 -- start start 2023-10-06T05:16:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f86695ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86640cd580 0x7f86640c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f86695ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8664159560 0x7f86640cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f86695ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664159930 0x7f86640c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f86695ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86640c5230 con 0x7f86640cd580 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f86695ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86640c8e20 con 0x7f8664159930 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f86695ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86640c4990 con 0x7f8664159560 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f8662ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86640cd580 0x7f86640c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f86627fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8664159560 0x7f86640cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f8662ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86640cd580 0x7f86640c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.62:35274/0 (socket says 172.21.15.62:35274) 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f8662ffd700 1 -- 172.21.15.62:0/3588367717 learned_addr learned my addr 172.21.15.62:0/3588367717 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.544+0000 7f86637fe700 1 --2- 172.21.15.62:0/3588367717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664159930 0x7f86640c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.545+0000 7f8662ffd700 1 -- 172.21.15.62:0/3588367717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8664159560 msgr2=0x7f86640cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.545+0000 7f8662ffd700 1 --2- 172.21.15.62:0/3588367717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8664159560 0x7f86640cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.545+0000 7f8662ffd700 1 -- 172.21.15.62:0/3588367717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664159930 msgr2=0x7f86640c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.545+0000 7f8662ffd700 1 --2- 172.21.15.62:0/3588367717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664159930 0x7f86640c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.545+0000 7f8662ffd700 1 -- 172.21.15.62:0/3588367717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c009ae0 con 0x7f86640cd580 2023-10-06T05:16:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.545+0000 7f8662ffd700 1 --2- 172.21.15.62:0/3588367717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86640cd580 0x7f86640c88e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f865400c7e0 tx=0x7f8654007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.545+0000 7f864affd700 1 -- 172.21.15.62:0/3588367717 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654011590 con 0x7f86640cd580 2023-10-06T05:16:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.545+0000 7f864affd700 1 -- 172.21.15.62:0/3588367717 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654011710 con 0x7f86640cd580 2023-10-06T05:16:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.545+0000 7f864affd700 1 -- 172.21.15.62:0/3588367717 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654011ad0 con 0x7f86640cd580 2023-10-06T05:16:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.545+0000 7f86695ff700 1 -- 172.21.15.62:0/3588367717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86640c4c70 con 0x7f86640cd580 2023-10-06T05:16:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.546+0000 7f86695ff700 1 -- 172.21.15.62:0/3588367717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86640c5b20 con 0x7f86640cd580 2023-10-06T05:16:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.546+0000 7f864affd700 1 -- 172.21.15.62:0/3588367717 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f865401c460 con 0x7f86640cd580 2023-10-06T05:16:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.546+0000 7f864affd700 1 -- 172.21.15.62:0/3588367717 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f8654011e40 con 0x7f86640cd580 2023-10-06T05:16:43.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.547+0000 7f86695ff700 1 -- 172.21.15.62:0/3588367717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866404c380 con 0x7f86640cd580 2023-10-06T05:16:43.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.549+0000 7f864affd700 1 -- 172.21.15.62:0/3588367717 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f865401c710 con 0x7f86640cd580 2023-10-06T05:16:43.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.651+0000 7f86695ff700 1 -- 172.21.15.62:0/3588367717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "19501bab-f99b-4ea0-9a75-455fd46c87ff", "id": 4} v 0) v1 -- 0x7f86640b11d0 con 0x7f86640cd580 2023-10-06T05:16:43.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.658+0000 7f864affd700 1 -- 172.21.15.62:0/3588367717 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19501bab-f99b-4ea0-9a75-455fd46c87ff", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f865402b0b0 con 0x7f86640cd580 2023-10-06T05:16:43.660 INFO:teuthology.orchestra.run.smithi062.stdout:4 2023-10-06T05:16:43.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.659+0000 7f86695ff700 1 -- 172.21.15.62:0/3588367717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86640cd580 msgr2=0x7f86640c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.659+0000 7f86695ff700 1 --2- 172.21.15.62:0/3588367717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86640cd580 0x7f86640c88e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f865400c7e0 tx=0x7f8654007df0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.659+0000 7f86695ff700 1 -- 172.21.15.62:0/3588367717 shutdown_connections 2023-10-06T05:16:43.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.659+0000 7f86695ff700 1 --2- 172.21.15.62:0/3588367717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664159930 0x7f86640c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.659+0000 7f86695ff700 1 --2- 172.21.15.62:0/3588367717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86640cd580 0x7f86640c88e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.659+0000 7f86695ff700 1 --2- 172.21.15.62:0/3588367717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8664159560 0x7f86640cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.659+0000 7f86695ff700 1 -- 172.21.15.62:0/3588367717 >> 172.21.15.62:0/3588367717 conn(0x7f866406ec90 msgr2=0x7f866415c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:43.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.659+0000 7f86695ff700 1 -- 172.21.15.62:0/3588367717 shutdown_connections 2023-10-06T05:16:43.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.659+0000 7f86695ff700 1 -- 172.21.15.62:0/3588367717 wait complete. 2023-10-06T05:16:43.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 370a0c4f-8324-451d-b90f-607c426a9c45 5 2023-10-06T05:16:43.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.865+0000 7fd7f17ff700 1 Processor -- start 2023-10-06T05:16:43.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.865+0000 7fd7f17ff700 1 -- start start 2023-10-06T05:16:43.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.866+0000 7fd7f17ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd7ec159560 0x7fd7ec0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.866+0000 7fd7f17ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd7ec0b2b80 0x7fd7ec0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.866+0000 7fd7f17ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ec15d2d0 con 0x7fd7ec159930 2023-10-06T05:16:43.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.866+0000 7fd7f17ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ec15d5b0 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.866+0000 7fd7f17ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ec15d890 con 0x7fd7ec159560 2023-10-06T05:16:43.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.866+0000 7fd7eaffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd7ec159560 0x7fd7ec0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.866+0000 7fd7eaffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd7ec159560 0x7fd7ec0cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56232/0 (socket says 172.21.15.62:56232) 2023-10-06T05:16:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.866+0000 7fd7eaffd700 1 -- 172.21.15.62:0/898423544 learned_addr learned my addr 172.21.15.62:0/898423544 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.866+0000 7fd7eb7fe700 1 --2- 172.21.15.62:0/898423544 >> v2:172.21.15.31:3300/0 conn(0x7fd7ec0b2b80 0x7fd7ec0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.867+0000 7fd7eaffd700 1 -- 172.21.15.62:0/898423544 >> v2:172.21.15.31:3300/0 conn(0x7fd7ec0b2b80 msgr2=0x7fd7ec0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.867+0000 7fd7eaffd700 1 --2- 172.21.15.62:0/898423544 >> v2:172.21.15.31:3300/0 conn(0x7fd7ec0b2b80 0x7fd7ec0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.867+0000 7fd7eaffd700 1 -- 172.21.15.62:0/898423544 >> v1:172.21.15.62:6789/0 conn(0x7fd7ec159930 legacy=0x7fd7ec0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.867+0000 7fd7eaffd700 1 -- 172.21.15.62:0/898423544 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec15db70 con 0x7fd7ec159560 2023-10-06T05:16:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.867+0000 7fd7eaffd700 1 --2- 172.21.15.62:0/898423544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec159560 0x7fd7ec0cce70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec063d80 tx=0x7fd7dc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f98aa2841344b45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.867+0000 7fd7e9ffb700 1 -- 172.21.15.62:0/898423544 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc009e70 con 0x7fd7ec159560 2023-10-06T05:16:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.867+0000 7fd7e9ffb700 1 -- 172.21.15.62:0/898423544 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc00b860 con 0x7fd7ec159560 2023-10-06T05:16:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.867+0000 7fd7e9ffb700 1 -- 172.21.15.62:0/898423544 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00bc00 con 0x7fd7ec159560 2023-10-06T05:16:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.868+0000 7fd7f17ff700 1 -- 172.21.15.62:0/898423544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec159560 msgr2=0x7fd7ec0cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.868+0000 7fd7f17ff700 1 --2- 172.21.15.62:0/898423544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec159560 0x7fd7ec0cce70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec063d80 tx=0x7fd7dc009250 comp rx=0 tx=0).stop 2023-10-06T05:16:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.868+0000 7fd7f17ff700 1 -- 172.21.15.62:0/898423544 shutdown_connections 2023-10-06T05:16:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.868+0000 7fd7f17ff700 1 --2- 172.21.15.62:0/898423544 >> v2:172.21.15.31:3300/0 conn(0x7fd7ec0b2b80 0x7fd7ec0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.868+0000 7fd7f17ff700 1 --2- 172.21.15.62:0/898423544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec159560 0x7fd7ec0cce70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.868+0000 7fd7f17ff700 1 -- 172.21.15.62:0/898423544 >> 172.21.15.62:0/898423544 conn(0x7fd7ec06ec90 msgr2=0x7fd7ec06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.868+0000 7fd7f17ff700 1 -- 172.21.15.62:0/898423544 shutdown_connections 2023-10-06T05:16:43.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.868+0000 7fd7f17ff700 1 -- 172.21.15.62:0/898423544 wait complete. 2023-10-06T05:16:43.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.869+0000 7fd7f17ff700 1 Processor -- start 2023-10-06T05:16:43.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.870+0000 7fd7f17ff700 1 -- start start 2023-10-06T05:16:43.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.870+0000 7fd7f17ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0b2b80 0x7fd7ec169810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.870+0000 7fd7f17ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7ec159560 0x7fd7ec0c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.870+0000 7fd7f17ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7ec159930 0x7fd7ec0c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7f17ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0cf610 con 0x7fd7ec159930 2023-10-06T05:16:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7f17ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0ce910 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7f17ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0c99f0 con 0x7fd7ec159560 2023-10-06T05:16:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7eaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0b2b80 0x7fd7ec169810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7eaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0b2b80 0x7fd7ec169810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56236/0 (socket says 172.21.15.62:56236) 2023-10-06T05:16:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7eaffd700 1 -- 172.21.15.62:0/1403803627 learned_addr learned my addr 172.21.15.62:0/1403803627 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7ea7fc700 1 --2- 172.21.15.62:0/1403803627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7ec159560 0x7fd7ec0c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7eb7fe700 1 --2- 172.21.15.62:0/1403803627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7ec159930 0x7fd7ec0c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7eaffd700 1 -- 172.21.15.62:0/1403803627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7ec159560 msgr2=0x7fd7ec0c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7eaffd700 1 --2- 172.21.15.62:0/1403803627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7ec159560 0x7fd7ec0c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7eaffd700 1 -- 172.21.15.62:0/1403803627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7ec159930 msgr2=0x7fd7ec0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7eaffd700 1 --2- 172.21.15.62:0/1403803627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7ec159930 0x7fd7ec0c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.871+0000 7fd7eaffd700 1 -- 172.21.15.62:0/1403803627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc0094f0 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.872+0000 7fd7eaffd700 1 --2- 172.21.15.62:0/1403803627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0b2b80 0x7fd7ec169810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0cd980 tx=0x7fd7dc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.872+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1403803627 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00bbe0 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.872+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1403803627 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc011420 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.872+0000 7fd7f17ff700 1 -- 172.21.15.62:0/1403803627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ec0c9cd0 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.872+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1403803627 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0117c0 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.872+0000 7fd7f17ff700 1 -- 172.21.15.62:0/1403803627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ec0cf2a0 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.872+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1403803627 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd7dc011bf0 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.872+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1403803627 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7fd7dc023970 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.873+0000 7fd7f17ff700 1 -- 172.21.15.62:0/1403803627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ec04c380 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.876+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1403803627 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd7dc028070 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.963+0000 7fd7f17ff700 1 -- 172.21.15.62:0/1403803627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "370a0c4f-8324-451d-b90f-607c426a9c45", "id": 5} v 0) v1 -- 0x7fd7ec15de50 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.971+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1403803627 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "370a0c4f-8324-451d-b90f-607c426a9c45", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd7dc015020 con 0x7fd7ec0b2b80 2023-10-06T05:16:43.973 INFO:teuthology.orchestra.run.smithi062.stdout:5 2023-10-06T05:16:43.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.972+0000 7fd7f17ff700 1 -- 172.21.15.62:0/1403803627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0b2b80 msgr2=0x7fd7ec169810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:43.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.972+0000 7fd7f17ff700 1 --2- 172.21.15.62:0/1403803627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0b2b80 0x7fd7ec169810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0cd980 tx=0x7fd7dc017000 comp rx=0 tx=0).stop 2023-10-06T05:16:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.972+0000 7fd7f17ff700 1 -- 172.21.15.62:0/1403803627 shutdown_connections 2023-10-06T05:16:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.972+0000 7fd7f17ff700 1 --2- 172.21.15.62:0/1403803627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7ec159930 0x7fd7ec0c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.972+0000 7fd7f17ff700 1 --2- 172.21.15.62:0/1403803627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0b2b80 0x7fd7ec169810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.972+0000 7fd7f17ff700 1 --2- 172.21.15.62:0/1403803627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7ec159560 0x7fd7ec0c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.972+0000 7fd7f17ff700 1 -- 172.21.15.62:0/1403803627 >> 172.21.15.62:0/1403803627 conn(0x7fd7ec06ec90 msgr2=0x7fd7ec0cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.972+0000 7fd7f17ff700 1 -- 172.21.15.62:0/1403803627 shutdown_connections 2023-10-06T05:16:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:43.973+0000 7fd7f17ff700 1 -- 172.21.15.62:0/1403803627 wait complete. 2023-10-06T05:16:43.990 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new ad324207-ac27-4cab-ab02-1c0b5fa7de64 6 2023-10-06T05:16:44.099 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:44.097+0000 7efed4e341c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:16:44.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.187+0000 7f9ad8fff700 1 Processor -- start 2023-10-06T05:16:44.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.188+0000 7f9ad8fff700 1 -- start start 2023-10-06T05:16:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.189+0000 7f9ad8fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ad415ca40 0x7f9ad4068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.189+0000 7f9ad8fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad406a300 0x7f9ad40c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.189+0000 7f9ad8fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad4068da0 con 0x7f9ad40c8d70 2023-10-06T05:16:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.189+0000 7f9ad8fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad40cabc0 con 0x7f9ad415ca40 2023-10-06T05:16:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.189+0000 7f9ad8fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad4068bb0 con 0x7f9ad406a300 2023-10-06T05:16:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.189+0000 7f9ad1d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad406a300 0x7f9ad40c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.189+0000 7f9ad2d9d700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9ad40c8d70 0x7f9ad406cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:43088/0 (socket says 172.21.15.62:43088) 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.189+0000 7f9ad2d9d700 1 -- 172.21.15.62:0/3833902344 learned_addr learned my addr 172.21.15.62:0/3833902344 (peer_addr_for_me v1:172.21.15.62:0/0) 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.189+0000 7f9ad259c700 1 --2- 172.21.15.62:0/3833902344 >> v2:172.21.15.31:3300/0 conn(0x7f9ad415ca40 0x7f9ad4068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.190+0000 7f9ad1d9b700 1 -- 172.21.15.62:0/3833902344 >> v2:172.21.15.31:3300/0 conn(0x7f9ad415ca40 msgr2=0x7f9ad4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.190+0000 7f9ad159a700 1 -- 172.21.15.62:0/3833902344 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162556587 0 0) 0x7f9ad4068da0 con 0x7f9ad40c8d70 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.190+0000 7f9ad1d9b700 1 --2- 172.21.15.62:0/3833902344 >> v2:172.21.15.31:3300/0 conn(0x7f9ad415ca40 0x7f9ad4068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.190+0000 7f9ad1d9b700 1 -- 172.21.15.62:0/3833902344 >> v1:172.21.15.62:6789/0 conn(0x7f9ad40c8d70 legacy=0x7f9ad406cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.190+0000 7f9ad1d9b700 1 -- 172.21.15.62:0/3833902344 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4069080 con 0x7f9ad406a300 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.190+0000 7f9ad259c700 1 --2- 172.21.15.62:0/3833902344 >> v2:172.21.15.31:3300/0 conn(0x7f9ad415ca40 0x7f9ad4068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.190+0000 7f9ad1d9b700 1 --2- 172.21.15.62:0/3833902344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad406a300 0x7f9ad40c8830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80098e0 tx=0x7f9ac80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fa9a96838bd3ce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.190+0000 7f9ad159a700 1 -- 172.21.15.62:0/3833902344 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac800b640 con 0x7f9ad406a300 2023-10-06T05:16:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.190+0000 7f9ad159a700 1 -- 172.21.15.62:0/3833902344 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac800b7a0 con 0x7f9ad406a300 2023-10-06T05:16:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.190+0000 7f9ad159a700 1 -- 172.21.15.62:0/3833902344 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac800bb40 con 0x7f9ad406a300 2023-10-06T05:16:44.194 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:44.190+0000 7efed4e341c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:16:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.191+0000 7f9ad8fff700 1 -- 172.21.15.62:0/3833902344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad406a300 msgr2=0x7f9ad40c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.191+0000 7f9ad8fff700 1 --2- 172.21.15.62:0/3833902344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad406a300 0x7f9ad40c8830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80098e0 tx=0x7f9ac80091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.191+0000 7f9ad8fff700 1 -- 172.21.15.62:0/3833902344 shutdown_connections 2023-10-06T05:16:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.191+0000 7f9ad8fff700 1 --2- 172.21.15.62:0/3833902344 >> v2:172.21.15.31:3300/0 conn(0x7f9ad415ca40 0x7f9ad4068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.191+0000 7f9ad8fff700 1 --2- 172.21.15.62:0/3833902344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad406a300 0x7f9ad40c8830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.191+0000 7f9ad8fff700 1 -- 172.21.15.62:0/3833902344 >> 172.21.15.62:0/3833902344 conn(0x7f9ad40ce290 msgr2=0x7f9ad40ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.191+0000 7f9ad8fff700 1 -- 172.21.15.62:0/3833902344 shutdown_connections 2023-10-06T05:16:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.191+0000 7f9ad8fff700 1 -- 172.21.15.62:0/3833902344 wait complete. 2023-10-06T05:16:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.193+0000 7f9ad8fff700 1 Processor -- start 2023-10-06T05:16:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.193+0000 7f9ad8fff700 1 -- start start 2023-10-06T05:16:44.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad8fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ad406a300 0x7f9ad40b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad8fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad40c8d70 0x7f9ad40b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad8fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad415ca40 0x7f9ad40c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad8fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad4159d70 con 0x7f9ad415ca40 2023-10-06T05:16:44.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad8fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad41592d0 con 0x7f9ad406a300 2023-10-06T05:16:44.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad8fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40b3f60 con 0x7f9ad40c8d70 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ad406a300 0x7f9ad40b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ad406a300 0x7f9ad40b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56242/0 (socket says 172.21.15.62:56242) 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad1d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad40c8d70 0x7f9ad40b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad259c700 1 -- 172.21.15.62:0/695652864 learned_addr learned my addr 172.21.15.62:0/695652864 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad2d9d700 1 --2- 172.21.15.62:0/695652864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad415ca40 0x7f9ad40c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad259c700 1 -- 172.21.15.62:0/695652864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad40c8d70 msgr2=0x7f9ad40b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad259c700 1 --2- 172.21.15.62:0/695652864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad40c8d70 0x7f9ad40b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad259c700 1 -- 172.21.15.62:0/695652864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad415ca40 msgr2=0x7f9ad40c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad259c700 1 --2- 172.21.15.62:0/695652864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad415ca40 0x7f9ad40c64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad259c700 1 -- 172.21.15.62:0/695652864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4009420 con 0x7f9ad406a300 2023-10-06T05:16:44.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad1d9b700 1 --2- 172.21.15.62:0/695652864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad40c8d70 0x7f9ad40b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:44.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.195+0000 7f9ad2d9d700 1 --2- 172.21.15.62:0/695652864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad415ca40 0x7f9ad40c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:44.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.196+0000 7f9ad259c700 1 --2- 172.21.15.62:0/695652864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ad406a300 0x7f9ad40b1a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400ecb0 tx=0x7f9ac40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:44.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.196+0000 7f9ac27fc700 1 -- 172.21.15.62:0/695652864 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400c850 con 0x7f9ad406a300 2023-10-06T05:16:44.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.196+0000 7f9ad8fff700 1 -- 172.21.15.62:0/695652864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac8009420 con 0x7f9ad406a300 2023-10-06T05:16:44.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.196+0000 7f9ac27fc700 1 -- 172.21.15.62:0/695652864 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac400c9b0 con 0x7f9ad406a300 2023-10-06T05:16:44.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.196+0000 7f9ac27fc700 1 -- 172.21.15.62:0/695652864 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400cd30 con 0x7f9ad406a300 2023-10-06T05:16:44.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.196+0000 7f9ad8fff700 1 -- 172.21.15.62:0/695652864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad40c59d0 con 0x7f9ad406a300 2023-10-06T05:16:44.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.197+0000 7f9ac27fc700 1 -- 172.21.15.62:0/695652864 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9ac4005c80 con 0x7f9ad406a300 2023-10-06T05:16:44.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.197+0000 7f9ac27fc700 1 -- 172.21.15.62:0/695652864 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f9ac4006620 con 0x7f9ad406a300 2023-10-06T05:16:44.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.197+0000 7f9ad8fff700 1 -- 172.21.15.62:0/695652864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad40c6300 con 0x7f9ad406a300 2023-10-06T05:16:44.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.200+0000 7f9ac27fc700 1 -- 172.21.15.62:0/695652864 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9ac4021060 con 0x7f9ad406a300 2023-10-06T05:16:44.226 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:44.224+0000 7f2eccc7e1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:16:44.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.287+0000 7f9ad8fff700 1 -- 172.21.15.62:0/695652864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ad324207-ac27-4cab-ab02-1c0b5fa7de64", "id": 6} v 0) v1 -- 0x7f9ad404c380 con 0x7f9ad406a300 2023-10-06T05:16:44.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.291+0000 7f9ac27fc700 1 -- 172.21.15.62:0/695652864 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad324207-ac27-4cab-ab02-1c0b5fa7de64", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9ac4017070 con 0x7f9ad406a300 2023-10-06T05:16:44.293 INFO:teuthology.orchestra.run.smithi062.stdout:6 2023-10-06T05:16:44.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.292+0000 7f9ad8fff700 1 -- 172.21.15.62:0/695652864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ad406a300 msgr2=0x7f9ad40b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.292+0000 7f9ad8fff700 1 --2- 172.21.15.62:0/695652864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ad406a300 0x7f9ad40b1a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400ecb0 tx=0x7f9ac40091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.293+0000 7f9ad8fff700 1 -- 172.21.15.62:0/695652864 shutdown_connections 2023-10-06T05:16:44.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.293+0000 7f9ad8fff700 1 --2- 172.21.15.62:0/695652864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad415ca40 0x7f9ad40c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.293+0000 7f9ad8fff700 1 --2- 172.21.15.62:0/695652864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ad406a300 0x7f9ad40b1a50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.293+0000 7f9ad8fff700 1 --2- 172.21.15.62:0/695652864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ad40c8d70 0x7f9ad40b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.293+0000 7f9ad8fff700 1 -- 172.21.15.62:0/695652864 >> 172.21.15.62:0/695652864 conn(0x7f9ad40ce290 msgr2=0x7f9ad4069880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:44.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.293+0000 7f9ad8fff700 1 -- 172.21.15.62:0/695652864 shutdown_connections 2023-10-06T05:16:44.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.293+0000 7f9ad8fff700 1 -- 172.21.15.62:0/695652864 wait complete. 2023-10-06T05:16:44.311 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 44fd2503-f934-4417-85de-16dcfb16d465 7 2023-10-06T05:16:44.320 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:44.318+0000 7f2eccc7e1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:16:44.365 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:44.363+0000 7efed4e341c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:16:44.455 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:44.453+0000 7efed4e341c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:16:44.494 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:44.493+0000 7f2eccc7e1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:16:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.500+0000 7f743b3ff700 1 Processor -- start 2023-10-06T05:16:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.500+0000 7f743b3ff700 1 -- start start 2023-10-06T05:16:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.501+0000 7f743b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7434159560 0x7f74340cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.501+0000 7f743b3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743415d2d0 con 0x7f74340b2b80 2023-10-06T05:16:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.501+0000 7f743b3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743415d5b0 con 0x7f7434159930 2023-10-06T05:16:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.501+0000 7f743b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743415d890 con 0x7f7434159560 2023-10-06T05:16:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.501+0000 7f743919b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7434159560 0x7f74340cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.501+0000 7f743899a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7434159930 0x7f74340b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:43104/0 (socket says 172.21.15.62:43104) 2023-10-06T05:16:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.501+0000 7f743899a700 1 -- 172.21.15.62:0/855707473 learned_addr learned my addr 172.21.15.62:0/855707473 (peer_addr_for_me v1:172.21.15.62:0/0) 2023-10-06T05:16:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.502+0000 7f743919b700 1 -- 172.21.15.62:0/855707473 >> v1:172.21.15.62:6789/0 conn(0x7f7434159930 legacy=0x7f74340b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.502+0000 7f743919b700 1 -- 172.21.15.62:0/855707473 >> v1:172.21.15.31:6789/0 conn(0x7f74340b2b80 legacy=0x7f74340cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.502+0000 7f743919b700 1 -- 172.21.15.62:0/855707473 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743415db70 con 0x7f7434159560 2023-10-06T05:16:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.502+0000 7f742bfff700 1 -- 172.21.15.62:0/855707473 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614403664 0 0) 0x7f743415d5b0 con 0x7f7434159930 2023-10-06T05:16:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.502+0000 7f743919b700 1 --2- 172.21.15.62:0/855707473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7434159560 0x7f74340cce70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7430009910 tx=0x7f74300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f10561476c61300 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.502+0000 7f742bfff700 1 -- 172.21.15.62:0/855707473 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743000b860 con 0x7f7434159560 2023-10-06T05:16:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.502+0000 7f742bfff700 1 -- 172.21.15.62:0/855707473 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743000b9c0 con 0x7f7434159560 2023-10-06T05:16:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.502+0000 7f742bfff700 1 -- 172.21.15.62:0/855707473 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743000bd60 con 0x7f7434159560 2023-10-06T05:16:44.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.503+0000 7f743b3ff700 1 -- 172.21.15.62:0/855707473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7434159560 msgr2=0x7f74340cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.503+0000 7f743b3ff700 1 --2- 172.21.15.62:0/855707473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7434159560 0x7f74340cce70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7430009910 tx=0x7f74300091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.503+0000 7f743b3ff700 1 -- 172.21.15.62:0/855707473 shutdown_connections 2023-10-06T05:16:44.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.503+0000 7f743b3ff700 1 --2- 172.21.15.62:0/855707473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7434159560 0x7f74340cce70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.503+0000 7f743b3ff700 1 -- 172.21.15.62:0/855707473 >> 172.21.15.62:0/855707473 conn(0x7f743406ec90 msgr2=0x7f743406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.503+0000 7f743b3ff700 1 -- 172.21.15.62:0/855707473 shutdown_connections 2023-10-06T05:16:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.503+0000 7f743b3ff700 1 -- 172.21.15.62:0/855707473 wait complete. 2023-10-06T05:16:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.505+0000 7f743b3ff700 1 Processor -- start 2023-10-06T05:16:44.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.506+0000 7f743b3ff700 1 -- start start 2023-10-06T05:16:44.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74340b2b80 0x7f74340c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743b3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7434159560 0x7f7434145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743b3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7434159930 0x7f74340c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743b3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340c3ef0 con 0x7f7434159930 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743b3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340c9bc0 con 0x7f7434159560 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74340c9ea0 con 0x7f74340b2b80 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743919b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74340b2b80 0x7f74340c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743919b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74340b2b80 0x7f74340c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:39776/0 (socket says 172.21.15.62:39776) 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743919b700 1 -- 172.21.15.62:0/1184606627 learned_addr learned my addr 172.21.15.62:0/1184606627 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743899a700 1 --2- 172.21.15.62:0/1184606627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7434159560 0x7f7434145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743999c700 1 --2- 172.21.15.62:0/1184606627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7434159930 0x7f74340c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743919b700 1 -- 172.21.15.62:0/1184606627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7434159560 msgr2=0x7f7434145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743919b700 1 --2- 172.21.15.62:0/1184606627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7434159560 0x7f7434145420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.507+0000 7f743919b700 1 -- 172.21.15.62:0/1184606627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7434159930 msgr2=0x7f74340c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.508+0000 7f743919b700 1 --2- 172.21.15.62:0/1184606627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7434159930 0x7f74340c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.508+0000 7f743919b700 1 -- 172.21.15.62:0/1184606627 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7430009420 con 0x7f74340b2b80 2023-10-06T05:16:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.508+0000 7f743919b700 1 --2- 172.21.15.62:0/1184606627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74340b2b80 0x7f74340c7a00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f74340893d0 tx=0x7f7430017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.508+0000 7f74297fa700 1 -- 172.21.15.62:0/1184606627 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743001f070 con 0x7f74340b2b80 2023-10-06T05:16:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.508+0000 7f74297fa700 1 -- 172.21.15.62:0/1184606627 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7430002f70 con 0x7f74340b2b80 2023-10-06T05:16:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.508+0000 7f743b3ff700 1 -- 172.21.15.62:0/1184606627 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74340ca180 con 0x7f74340b2b80 2023-10-06T05:16:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.508+0000 7f74297fa700 1 -- 172.21.15.62:0/1184606627 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430025420 con 0x7f74340b2b80 2023-10-06T05:16:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.508+0000 7f743b3ff700 1 -- 172.21.15.62:0/1184606627 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74340c9700 con 0x7f74340b2b80 2023-10-06T05:16:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.509+0000 7f74297fa700 1 -- 172.21.15.62:0/1184606627 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7430023070 con 0x7f74340b2b80 2023-10-06T05:16:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.509+0000 7f74297fa700 1 -- 172.21.15.62:0/1184606627 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f743001aa90 con 0x7f74340b2b80 2023-10-06T05:16:44.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.509+0000 7f743b3ff700 1 -- 172.21.15.62:0/1184606627 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74340c4380 con 0x7f74340b2b80 2023-10-06T05:16:44.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.512+0000 7f74297fa700 1 -- 172.21.15.62:0/1184606627 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f74340c4380 con 0x7f74340b2b80 2023-10-06T05:16:44.587 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:44.586+0000 7f2eccc7e1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:16:44.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.601+0000 7f743b3ff700 1 -- 172.21.15.62:0/1184606627 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "44fd2503-f934-4417-85de-16dcfb16d465", "id": 7} v 0) v1 -- 0x7f743404c380 con 0x7f74340b2b80 2023-10-06T05:16:44.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.608+0000 7f74297fa700 1 -- 172.21.15.62:0/1184606627 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "44fd2503-f934-4417-85de-16dcfb16d465", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f743002a070 con 0x7f74340b2b80 2023-10-06T05:16:44.609 INFO:teuthology.orchestra.run.smithi062.stdout:7 2023-10-06T05:16:44.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.609+0000 7f743b3ff700 1 -- 172.21.15.62:0/1184606627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74340b2b80 msgr2=0x7f74340c7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.609+0000 7f743b3ff700 1 --2- 172.21.15.62:0/1184606627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74340b2b80 0x7f74340c7a00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f74340893d0 tx=0x7f7430017000 comp rx=0 tx=0).stop 2023-10-06T05:16:44.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.609+0000 7f743b3ff700 1 -- 172.21.15.62:0/1184606627 shutdown_connections 2023-10-06T05:16:44.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.609+0000 7f743b3ff700 1 --2- 172.21.15.62:0/1184606627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7434159930 0x7f74340c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.609+0000 7f743b3ff700 1 --2- 172.21.15.62:0/1184606627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74340b2b80 0x7f74340c7a00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.609+0000 7f743b3ff700 1 --2- 172.21.15.62:0/1184606627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7434159560 0x7f7434145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.609+0000 7f743b3ff700 1 -- 172.21.15.62:0/1184606627 >> 172.21.15.62:0/1184606627 conn(0x7f743406ec90 msgr2=0x7f7434158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:44.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.609+0000 7f743b3ff700 1 -- 172.21.15.62:0/1184606627 shutdown_connections 2023-10-06T05:16:44.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-10-06T05:16:44.609+0000 7f743b3ff700 1 -- 172.21.15.62:0/1184606627 wait complete. 2023-10-06T05:16:44.627 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T05:16:44.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T05:16:44.634 INFO:tasks.ceph.osd.0:Started 2023-10-06T05:16:44.634 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T05:16:44.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T05:16:44.640 INFO:tasks.ceph.osd.1:Started 2023-10-06T05:16:44.640 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T05:16:44.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T05:16:44.646 INFO:tasks.ceph.osd.2:Started 2023-10-06T05:16:44.646 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T05:16:44.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T05:16:44.652 INFO:tasks.ceph.osd.3:Started 2023-10-06T05:16:44.652 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T05:16:44.653 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T05:16:44.674 INFO:tasks.ceph.osd.4:Started 2023-10-06T05:16:44.674 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T05:16:44.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T05:16:44.680 INFO:tasks.ceph.osd.5:Started 2023-10-06T05:16:44.680 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T05:16:44.680 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T05:16:44.685 INFO:tasks.ceph.osd.6:Started 2023-10-06T05:16:44.685 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T05:16:44.685 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T05:16:44.690 INFO:tasks.ceph.osd.7:Started 2023-10-06T05:16:44.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:44.696 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:44.691+0000 7efed4e341c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:16:44.830 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:44.828+0000 7f2eccc7e1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:16:44.843 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:44.841+0000 7efed4e341c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:16:44.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.938+0000 7f12039ff700 1 Processor -- start 2023-10-06T05:16:44.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.938+0000 7f12039ff700 1 -- start start 2023-10-06T05:16:44.941 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:44.939+0000 7f2eccc7e1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:16:44.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.939+0000 7f12039ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f11fc0cd480 0x7f11fc0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.939+0000 7f12039ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc0cedc0 con 0x7f11fc0cac90 2023-10-06T05:16:44.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f12039ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc0c7a30 con 0x7f11fc1595b0 2023-10-06T05:16:44.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f12039ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc0ce1b0 con 0x7f11fc0cd480 2023-10-06T05:16:44.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f1200f9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f11fc0cac90 0x7f11fc0ccd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53408/0 (socket says 172.21.15.31:53408) 2023-10-06T05:16:44.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f1200f9a700 1 -- 172.21.15.31:0/2087216420 learned_addr learned my addr 172.21.15.31:0/2087216420 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:16:44.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f1201f9c700 1 --2- 172.21.15.31:0/2087216420 >> v2:172.21.15.62:3300/0 conn(0x7f11fc0cd480 0x7f11fc0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f11f3fff700 1 -- 172.21.15.31:0/2087216420 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313193560 0 0) 0x7f11fc0c7a30 con 0x7f11fc1595b0 2023-10-06T05:16:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f11f3fff700 1 -- 172.21.15.31:0/2087216420 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11e8003570 con 0x7f11fc1595b0 2023-10-06T05:16:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f11f3fff700 1 -- 172.21.15.31:0/2087216420 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369853502 0 0) 0x7f11fc0cedc0 con 0x7f11fc0cac90 2023-10-06T05:16:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f11f3fff700 1 -- 172.21.15.31:0/2087216420 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11fc0c7a30 con 0x7f11fc0cac90 2023-10-06T05:16:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f1201f9c700 1 -- 172.21.15.31:0/2087216420 >> v1:172.21.15.62:6789/0 conn(0x7f11fc1595b0 legacy=0x7f11fc0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f1201f9c700 1 -- 172.21.15.31:0/2087216420 >> v1:172.21.15.31:6789/0 conn(0x7f11fc0cac90 legacy=0x7f11fc0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.940+0000 7f1201f9c700 1 -- 172.21.15.31:0/2087216420 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0ce490 con 0x7f11fc0cd480 2023-10-06T05:16:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.941+0000 7f1201f9c700 1 --2- 172.21.15.31:0/2087216420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11fc0cd480 0x7f11fc0c74f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009e60 tx=0x7f11f4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8187fe4c089f139 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:44.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.941+0000 7f11f3fff700 1 -- 172.21.15.31:0/2087216420 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f400e070 con 0x7f11fc0cd480 2023-10-06T05:16:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.941+0000 7f11f3fff700 1 -- 172.21.15.31:0/2087216420 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f400bab0 con 0x7f11fc0cd480 2023-10-06T05:16:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.941+0000 7f11f3fff700 1 -- 172.21.15.31:0/2087216420 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f400be70 con 0x7f11fc0cd480 2023-10-06T05:16:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.941+0000 7f12039ff700 1 -- 172.21.15.31:0/2087216420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11fc0cd480 msgr2=0x7f11fc0c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.941+0000 7f12039ff700 1 --2- 172.21.15.31:0/2087216420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11fc0cd480 0x7f11fc0c74f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009e60 tx=0x7f11f4009780 comp rx=0 tx=0).stop 2023-10-06T05:16:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.942+0000 7f12039ff700 1 -- 172.21.15.31:0/2087216420 shutdown_connections 2023-10-06T05:16:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.942+0000 7f12039ff700 1 --2- 172.21.15.31:0/2087216420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11fc0cd480 0x7f11fc0c74f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.942+0000 7f12039ff700 1 -- 172.21.15.31:0/2087216420 >> 172.21.15.31:0/2087216420 conn(0x7f11fc099740 msgr2=0x7f11fc06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.942+0000 7f12039ff700 1 -- 172.21.15.31:0/2087216420 shutdown_connections 2023-10-06T05:16:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.942+0000 7f12039ff700 1 -- 172.21.15.31:0/2087216420 wait complete. 2023-10-06T05:16:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.944+0000 7f12039ff700 1 Processor -- start 2023-10-06T05:16:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.944+0000 7f12039ff700 1 -- start start 2023-10-06T05:16:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f12039ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11fc0cac90 0x7f11fc15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f12039ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11fc0cd480 0x7f11fc0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f12039ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11fc1595b0 0x7f11fc15f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f12039ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15c1d0 con 0x7f11fc0cd480 2023-10-06T05:16:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f12039ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15c4b0 con 0x7f11fc1595b0 2023-10-06T05:16:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f12039ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15c790 con 0x7f11fc0cac90 2023-10-06T05:16:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f1200f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11fc0cd480 0x7f11fc0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.948 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:44.945+0000 7efed4e341c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:16:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f120179b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11fc0cac90 0x7f11fc15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f1200f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11fc0cd480 0x7f11fc0b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57672/0 (socket says 172.21.15.31:57672) 2023-10-06T05:16:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f1200f9a700 1 -- 172.21.15.31:0/29319917 learned_addr learned my addr 172.21.15.31:0/29319917 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.945+0000 7f1201f9c700 1 --2- 172.21.15.31:0/29319917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11fc1595b0 0x7f11fc15f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f1200f9a700 1 -- 172.21.15.31:0/29319917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11fc0cac90 msgr2=0x7f11fc15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f1200f9a700 1 --2- 172.21.15.31:0/29319917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11fc0cac90 0x7f11fc15b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f1200f9a700 1 -- 172.21.15.31:0/29319917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11fc1595b0 msgr2=0x7f11fc15f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f1200f9a700 1 --2- 172.21.15.31:0/29319917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11fc1595b0 0x7f11fc15f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f1200f9a700 1 -- 172.21.15.31:0/29319917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f4009a20 con 0x7f11fc0cd480 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f120179b700 1 --2- 172.21.15.31:0/29319917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11fc0cac90 0x7f11fc15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f1200f9a700 1 --2- 172.21.15.31:0/29319917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11fc0cd480 0x7f11fc0b54f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00c7c0 tx=0x7f11ec007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f11f17fa700 1 -- 172.21.15.31:0/29319917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec005c00 con 0x7f11fc0cd480 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f11f17fa700 1 -- 172.21.15.31:0/29319917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec005d60 con 0x7f11fc0cd480 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f12039ff700 1 -- 172.21.15.31:0/29319917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc15ca70 con 0x7f11fc0cd480 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f11f17fa700 1 -- 172.21.15.31:0/29319917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec007300 con 0x7f11fc0cd480 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f12039ff700 1 -- 172.21.15.31:0/29319917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc16c6d0 con 0x7f11fc0cd480 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.946+0000 7f11f17fa700 1 -- 172.21.15.31:0/29319917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f11ec01c440 con 0x7f11fc0cd480 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.947+0000 7f11f17fa700 1 -- 172.21.15.31:0/29319917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f11ec012ae0 con 0x7f11fc0cd480 2023-10-06T05:16:44.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.947+0000 7f12039ff700 1 -- 172.21.15.31:0/29319917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc15cf90 con 0x7f11fc0cd480 2023-10-06T05:16:44.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:44.950+0000 7f11f17fa700 1 -- 172.21.15.31:0/29319917 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f11fc15cf90 con 0x7f11fc0cd480 2023-10-06T05:16:45.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.023+0000 7f12039ff700 1 -- 172.21.15.31:0/29319917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11fc04c380 con 0x7f11fc0cd480 2023-10-06T05:16:45.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.024+0000 7f11f17fa700 1 -- 172.21.15.31:0/29319917 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f11ec017070 con 0x7f11fc0cd480 2023-10-06T05:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:45.026 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:16:44.603355+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:45.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.025+0000 7f12039ff700 1 -- 172.21.15.31:0/29319917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11fc0cd480 msgr2=0x7f11fc0b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:45.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.025+0000 7f12039ff700 1 --2- 172.21.15.31:0/29319917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11fc0cd480 0x7f11fc0b54f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00c7c0 tx=0x7f11ec007e80 comp rx=0 tx=0).stop 2023-10-06T05:16:45.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.025+0000 7f12039ff700 1 -- 172.21.15.31:0/29319917 shutdown_connections 2023-10-06T05:16:45.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.025+0000 7f12039ff700 1 --2- 172.21.15.31:0/29319917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11fc1595b0 0x7f11fc15f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:45.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.025+0000 7f12039ff700 1 --2- 172.21.15.31:0/29319917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11fc0cac90 0x7f11fc15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:45.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.025+0000 7f12039ff700 1 --2- 172.21.15.31:0/29319917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11fc0cd480 0x7f11fc0b54f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:45.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.025+0000 7f12039ff700 1 -- 172.21.15.31:0/29319917 >> 172.21.15.31:0/29319917 conn(0x7f11fc099740 msgr2=0x7f11fc0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:45.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.025+0000 7f12039ff700 1 -- 172.21.15.31:0/29319917 shutdown_connections 2023-10-06T05:16:45.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:45.025+0000 7f12039ff700 1 -- 172.21.15.31:0/29319917 wait complete. 2023-10-06T05:16:45.041 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:45.039+0000 7f2eccc7e1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:16:45.044 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T05:16:45.044 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T05:16:45.065 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:45.063+0000 7efed4e341c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:16:45.152 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:45.150+0000 7f2eccc7e1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:16:45.294 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:45.292+0000 7efed4e341c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:16:45.360 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-10-06T05:16:45.358+0000 7f2daa4c0540 -1 Falling back to public interface 2023-10-06T05:16:45.361 INFO:tasks.ceph.osd.4.smithi062.stderr:2023-10-06T05:16:45.360+0000 7f7d2f41b540 -1 Falling back to public interface 2023-10-06T05:16:45.365 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-10-06T05:16:45.364+0000 7fe44e6a2540 -1 Falling back to public interface 2023-10-06T05:16:45.379 INFO:tasks.ceph.osd.7.smithi062.stderr:2023-10-06T05:16:45.377+0000 7f2dde8e5540 -1 Falling back to public interface 2023-10-06T05:16:45.379 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-10-06T05:16:45.377+0000 7f0b4a1a3540 -1 Falling back to public interface 2023-10-06T05:16:45.384 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-10-06T05:16:45.382+0000 7fd4214b1540 -1 Falling back to public interface 2023-10-06T05:16:45.384 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:45.382+0000 7f2eccc7e1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:16:45.406 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:45.405+0000 7efed4e341c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:16:45.421 INFO:tasks.ceph.osd.5.smithi062.stderr:2023-10-06T05:16:45.419+0000 7f029c710540 -1 Falling back to public interface 2023-10-06T05:16:45.425 INFO:tasks.ceph.osd.6.smithi062.stderr:2023-10-06T05:16:45.423+0000 7fdab09e3540 -1 Falling back to public interface 2023-10-06T05:16:45.488 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:45.486+0000 7f2eccc7e1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:16:45.759 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:45.758+0000 7efed4e341c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:16:45.842 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:45.840+0000 7f2eccc7e1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:16:45.849 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:45.847+0000 7efed4e341c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:16:45.849 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:45.935 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:45.933+0000 7f2eccc7e1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:16:46.027 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.025+0000 7f74249ff700 1 Processor -- start 2023-10-06T05:16:46.027 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.025+0000 7f74249ff700 1 -- start start 2023-10-06T05:16:46.028 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f74249ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200cea00 0x7f74200cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:46.028 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f74249ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7420068ac0 0x7f7420159130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:46.029 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f74249ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74200cceb0 con 0x7f74200686f0 2023-10-06T05:16:46.030 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f74249ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74200cf290 con 0x7f7420068ac0 2023-10-06T05:16:46.030 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f74249ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74200cc2a0 con 0x7f74200cea00 2023-10-06T05:16:46.030 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741e59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f74200686f0 0x7f74200cba10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53416/0 (socket says 172.21.15.31:53416) 2023-10-06T05:16:46.030 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741e59c700 1 -- 172.21.15.31:0/3261769278 learned_addr learned my addr 172.21.15.31:0/3261769278 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:16:46.032 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741dd9b700 1 --2- 172.21.15.31:0/3261769278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200cea00 0x7f74200cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:46.032 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741ed9d700 1 --2- 172.21.15.31:0/3261769278 >> v2:172.21.15.31:3300/0 conn(0x7f7420068ac0 0x7f7420159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:46.032 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741d59a700 1 -- 172.21.15.31:0/3261769278 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286090387 0 0) 0x7f74200cceb0 con 0x7f74200686f0 2023-10-06T05:16:46.032 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741d59a700 1 -- 172.21.15.31:0/3261769278 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7408003570 con 0x7f74200686f0 2023-10-06T05:16:46.032 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741dd9b700 1 -- 172.21.15.31:0/3261769278 >> v2:172.21.15.31:3300/0 conn(0x7f7420068ac0 msgr2=0x7f7420159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741dd9b700 1 --2- 172.21.15.31:0/3261769278 >> v2:172.21.15.31:3300/0 conn(0x7f7420068ac0 0x7f7420159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741dd9b700 1 -- 172.21.15.31:0/3261769278 >> v1:172.21.15.31:6789/0 conn(0x7f74200686f0 legacy=0x7f74200cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741dd9b700 1 -- 172.21.15.31:0/3261769278 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74200cc580 con 0x7f74200cea00 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.027+0000 7f741ed9d700 1 --2- 172.21.15.31:0/3261769278 >> v2:172.21.15.31:3300/0 conn(0x7f7420068ac0 0x7f7420159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.028+0000 7f741dd9b700 1 --2- 172.21.15.31:0/3261769278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200cea00 0x7f74200cd340 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f74140098e0 tx=0x7f74140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5e2c7c1ffc9c205 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.028+0000 7f741d59a700 1 -- 172.21.15.31:0/3261769278 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414011630 con 0x7f74200cea00 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.028+0000 7f741d59a700 1 -- 172.21.15.31:0/3261769278 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7414011790 con 0x7f74200cea00 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.028+0000 7f741d59a700 1 -- 172.21.15.31:0/3261769278 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414011b30 con 0x7f74200cea00 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.028+0000 7f74249ff700 1 -- 172.21.15.31:0/3261769278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200cea00 msgr2=0x7f74200cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.028+0000 7f74249ff700 1 --2- 172.21.15.31:0/3261769278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200cea00 0x7f74200cd340 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f74140098e0 tx=0x7f74140091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.029+0000 7f74249ff700 1 -- 172.21.15.31:0/3261769278 shutdown_connections 2023-10-06T05:16:46.033 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.029+0000 7f74249ff700 1 --2- 172.21.15.31:0/3261769278 >> v2:172.21.15.31:3300/0 conn(0x7f7420068ac0 0x7f7420159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:46.034 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.029+0000 7f74249ff700 1 --2- 172.21.15.31:0/3261769278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200cea00 0x7f74200cd340 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:46.034 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.029+0000 7f74249ff700 1 -- 172.21.15.31:0/3261769278 >> 172.21.15.31:0/3261769278 conn(0x7f742006ecd0 msgr2=0x7f742006f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:46.034 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.029+0000 7f74249ff700 1 -- 172.21.15.31:0/3261769278 shutdown_connections 2023-10-06T05:16:46.034 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.029+0000 7f74249ff700 1 -- 172.21.15.31:0/3261769278 wait complete. 2023-10-06T05:16:46.034 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.030+0000 7f74249ff700 1 Processor -- start 2023-10-06T05:16:46.034 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.030+0000 7f74249ff700 1 -- start start 2023-10-06T05:16:46.036 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.031+0000 7f74249ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200686f0 0x7f7420156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:46.036 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.031+0000 7f74249ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7420068ac0 0x7f742016ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:46.036 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.031+0000 7f74249ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74200cea00 0x7f742016d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:46.037 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.031+0000 7f74249ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74200b4ce0 con 0x7f7420068ac0 2023-10-06T05:16:46.037 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.031+0000 7f74249ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742015c190 con 0x7f74200cea00 2023-10-06T05:16:46.037 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.031+0000 7f74249ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742016da00 con 0x7f74200686f0 2023-10-06T05:16:46.037 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.031+0000 7f741dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7420068ac0 0x7f742016ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:46.038 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f741dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7420068ac0 0x7f742016ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57694/0 (socket says 172.21.15.31:57694) 2023-10-06T05:16:46.038 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f741dd9b700 1 -- 172.21.15.31:0/2543068027 learned_addr learned my addr 172.21.15.31:0/2543068027 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:46.038 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f741e59c700 1 --2- 172.21.15.31:0/2543068027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200686f0 0x7f7420156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:46.038 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f741ed9d700 1 --2- 172.21.15.31:0/2543068027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74200cea00 0x7f742016d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:46.038 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f741dd9b700 1 -- 172.21.15.31:0/2543068027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200686f0 msgr2=0x7f7420156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:46.039 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f741dd9b700 1 --2- 172.21.15.31:0/2543068027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200686f0 0x7f7420156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:46.039 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f741dd9b700 1 -- 172.21.15.31:0/2543068027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74200cea00 msgr2=0x7f742016d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:46.039 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f741dd9b700 1 --2- 172.21.15.31:0/2543068027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74200cea00 0x7f742016d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:46.039 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f741dd9b700 1 -- 172.21.15.31:0/2543068027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7418009a20 con 0x7f7420068ac0 2023-10-06T05:16:46.039 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f741dd9b700 1 --2- 172.21.15.31:0/2543068027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7420068ac0 0x7f742016ce90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f74200cc3e0 tx=0x7f7414017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:46.040 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.032+0000 7f740e7fc700 1 -- 172.21.15.31:0/2543068027 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414011b10 con 0x7f7420068ac0 2023-10-06T05:16:46.040 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.033+0000 7f740e7fc700 1 -- 172.21.15.31:0/2543068027 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7414000cc0 con 0x7f7420068ac0 2023-10-06T05:16:46.040 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.033+0000 7f74249ff700 1 -- 172.21.15.31:0/2543068027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7414009420 con 0x7f7420068ac0 2023-10-06T05:16:46.040 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.033+0000 7f740e7fc700 1 -- 172.21.15.31:0/2543068027 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414025620 con 0x7f7420068ac0 2023-10-06T05:16:46.040 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.033+0000 7f74249ff700 1 -- 172.21.15.31:0/2543068027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74200b4970 con 0x7f7420068ac0 2023-10-06T05:16:46.040 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.033+0000 7f740e7fc700 1 -- 172.21.15.31:0/2543068027 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7414025c90 con 0x7f7420068ac0 2023-10-06T05:16:46.041 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.033+0000 7f740e7fc700 1 -- 172.21.15.31:0/2543068027 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f74140193b0 con 0x7f7420068ac0 2023-10-06T05:16:46.041 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.034+0000 7f74249ff700 1 -- 172.21.15.31:0/2543068027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74201566c0 con 0x7f7420068ac0 2023-10-06T05:16:46.041 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.036+0000 7f740e7fc700 1 -- 172.21.15.31:0/2543068027 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f74201566c0 con 0x7f7420068ac0 2023-10-06T05:16:46.111 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.109+0000 7f74249ff700 1 -- 172.21.15.31:0/2543068027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74200b4e50 con 0x7f7420068ac0 2023-10-06T05:16:46.112 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.110+0000 7f740e7fc700 1 -- 172.21.15.31:0/2543068027 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f7414033060 con 0x7f7420068ac0 2023-10-06T05:16:46.112 INFO:teuthology.misc.health.smithi031.stdout: 2023-10-06T05:16:46.112 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:16:44.603355+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:46.113 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.111+0000 7f74249ff700 1 -- 172.21.15.31:0/2543068027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7420068ac0 msgr2=0x7f742016ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:46.114 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.111+0000 7f74249ff700 1 --2- 172.21.15.31:0/2543068027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7420068ac0 0x7f742016ce90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f74200cc3e0 tx=0x7f7414017000 comp rx=0 tx=0).stop 2023-10-06T05:16:46.114 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.111+0000 7f74249ff700 1 -- 172.21.15.31:0/2543068027 shutdown_connections 2023-10-06T05:16:46.114 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.111+0000 7f74249ff700 1 --2- 172.21.15.31:0/2543068027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74200cea00 0x7f742016d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:46.114 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.111+0000 7f74249ff700 1 --2- 172.21.15.31:0/2543068027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74200686f0 0x7f7420156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:46.114 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.111+0000 7f74249ff700 1 --2- 172.21.15.31:0/2543068027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7420068ac0 0x7f742016ce90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:46.114 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.111+0000 7f74249ff700 1 -- 172.21.15.31:0/2543068027 >> 172.21.15.31:0/2543068027 conn(0x7f742006ecd0 msgr2=0x7f74200b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:46.114 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.111+0000 7f74249ff700 1 -- 172.21.15.31:0/2543068027 shutdown_connections 2023-10-06T05:16:46.114 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:46.111+0000 7f74249ff700 1 -- 172.21.15.31:0/2543068027 wait complete. 2023-10-06T05:16:46.127 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T05:16:46.501 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:46.499+0000 7efed4e341c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:16:46.574 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:46.572+0000 7f2eccc7e1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:16:46.884 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:46.882+0000 7efed4e341c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:16:46.958 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:46.956+0000 7f2eccc7e1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:16:47.065 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:47.063+0000 7efed4e341c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:16:47.142 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:47.140+0000 7f2eccc7e1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:16:47.517 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:47.515+0000 7efed4e341c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:16:47.596 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:47.595+0000 7f2eccc7e1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:16:47.700 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:47.699+0000 7efed4e341c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:16:47.783 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:47.781+0000 7f2eccc7e1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:16:47.919 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:47.917+0000 7efed4e341c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:16:48.008 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:48.006+0000 7f2eccc7e1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:16:48.047 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:48.046+0000 7efed4e341c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:16:48.150 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:16:48.148+0000 7f2eccc7e1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:16:48.871 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-10-06T05:16:48.869+0000 7f2daa4c0540 -1 osd.0 0 log_to_monitors true 2023-10-06T05:16:48.884 INFO:tasks.ceph.osd.4.smithi062.stderr:2023-10-06T05:16:48.882+0000 7f7d2f41b540 -1 osd.4 0 log_to_monitors true 2023-10-06T05:16:48.924 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-10-06T05:16:48.923+0000 7fe44e6a2540 -1 osd.2 0 log_to_monitors true 2023-10-06T05:16:48.952 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-10-06T05:16:48.951+0000 7f0b4a1a3540 -1 osd.1 0 log_to_monitors true 2023-10-06T05:16:48.956 INFO:tasks.ceph.osd.7.smithi062.stderr:2023-10-06T05:16:48.954+0000 7f2dde8e5540 -1 osd.7 0 log_to_monitors true 2023-10-06T05:16:48.983 INFO:tasks.ceph.osd.5.smithi062.stderr:2023-10-06T05:16:48.982+0000 7f029c710540 -1 osd.5 0 log_to_monitors true 2023-10-06T05:16:48.990 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-10-06T05:16:48.988+0000 7fd4214b1540 -1 osd.3 0 log_to_monitors true 2023-10-06T05:16:49.022 INFO:tasks.ceph.osd.6.smithi062.stderr:2023-10-06T05:16:49.021+0000 7fdab09e3540 -1 osd.6 0 log_to_monitors true 2023-10-06T05:16:49.067 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:49.065+0000 7efe943fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T05:16:49.068 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:16:49.067+0000 7efe943fa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T05:16:52.249 INFO:tasks.ceph.osd.6.smithi062.stderr:2023-10-06T05:16:52.247+0000 7fdaa5bf9700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T05:16:52.258 INFO:tasks.ceph.osd.6.smithi062.stderr:2023-10-06T05:16:52.256+0000 7fda9d3e8700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:16:52.282 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-10-06T05:16:52.280+0000 7f2d9dff6700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T05:16:52.288 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-10-06T05:16:52.286+0000 7f2d96fe8700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:16:52.289 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-10-06T05:16:52.287+0000 7f0b3dbf6700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T05:16:52.297 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-10-06T05:16:52.294+0000 7f0b36be8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:16:52.311 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-10-06T05:16:52.309+0000 7fd414ff6700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T05:16:52.313 INFO:tasks.ceph.osd.5.smithi062.stderr:2023-10-06T05:16:52.312+0000 7f02901f6700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T05:16:52.317 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-10-06T05:16:52.316+0000 7fd40dfe8700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:16:52.326 INFO:tasks.ceph.osd.5.smithi062.stderr:2023-10-06T05:16:52.324+0000 7f02891e8700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:16:52.335 INFO:tasks.ceph.osd.4.smithi062.stderr:2023-10-06T05:16:52.333+0000 7f7d22ff6700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T05:16:52.343 INFO:tasks.ceph.osd.4.smithi062.stderr:2023-10-06T05:16:52.341+0000 7f7d1bfe8700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:16:52.349 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-10-06T05:16:52.348+0000 7fe4421f6700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T05:16:52.350 INFO:tasks.ceph.osd.7.smithi062.stderr:2023-10-06T05:16:52.349+0000 7f2dd23f6700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T05:16:52.354 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-10-06T05:16:52.352+0000 7fe43b1e8700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:16:52.359 INFO:tasks.ceph.osd.7.smithi062.stderr:2023-10-06T05:16:52.358+0000 7f2dcb3e8700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T05:16:52.934 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:53.116 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.114+0000 7f09195ff700 1 Processor -- start 2023-10-06T05:16:53.116 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.114+0000 7f09195ff700 1 -- start start 2023-10-06T05:16:53.117 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09195ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09140ca220 0x7f09140686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.118 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09195ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 0x7f09140c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.118 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09195ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09140c8d50 con 0x7f091406ba30 2023-10-06T05:16:53.118 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09195ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091415daf0 con 0x7f09140ca220 2023-10-06T05:16:53.118 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09195ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091415dce0 con 0x7f0914068bf0 2023-10-06T05:16:53.118 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f0912ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f091406ba30 0x7f091415d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43480/0 (socket says 172.21.15.31:43480) 2023-10-06T05:16:53.118 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f0912ffd700 1 -- 172.21.15.31:0/1460383885 learned_addr learned my addr 172.21.15.31:0/1460383885 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:16:53.118 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09127fc700 1 --2- 172.21.15.31:0/1460383885 >> v2:172.21.15.31:3300/0 conn(0x7f09140ca220 0x7f09140686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.120 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09137fe700 1 --2- 172.21.15.31:0/1460383885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 0x7f09140c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.120 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f0911ffb700 1 -- 172.21.15.31:0/1460383885 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866821820 0 0) 0x7f09140c8d50 con 0x7f091406ba30 2023-10-06T05:16:53.120 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09127fc700 1 -- 172.21.15.31:0/1460383885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 msgr2=0x7f09140c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:53.120 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09127fc700 1 --2- 172.21.15.31:0/1460383885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 0x7f09140c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.120 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09127fc700 1 -- 172.21.15.31:0/1460383885 >> v1:172.21.15.31:6789/0 conn(0x7f091406ba30 legacy=0x7f091415d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:53.120 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.116+0000 7f09127fc700 1 -- 172.21.15.31:0/1460383885 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091415dfc0 con 0x7f09140ca220 2023-10-06T05:16:53.120 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.117+0000 7f09127fc700 1 --2- 172.21.15.31:0/1460383885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09140ca220 0x7f09140686b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f09080098e0 tx=0x7f09080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8c66fdcf4a1d55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:53.120 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.117+0000 7f0911ffb700 1 -- 172.21.15.31:0/1460383885 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908009e80 con 0x7f09140ca220 2023-10-06T05:16:53.121 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.117+0000 7f0911ffb700 1 -- 172.21.15.31:0/1460383885 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090800b860 con 0x7f09140ca220 2023-10-06T05:16:53.121 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.117+0000 7f0911ffb700 1 -- 172.21.15.31:0/1460383885 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090800bbe0 con 0x7f09140ca220 2023-10-06T05:16:53.121 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.117+0000 7f09195ff700 1 -- 172.21.15.31:0/1460383885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09140ca220 msgr2=0x7f09140686b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:53.121 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.117+0000 7f09195ff700 1 --2- 172.21.15.31:0/1460383885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09140ca220 0x7f09140686b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f09080098e0 tx=0x7f09080091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.121 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.118+0000 7f09195ff700 1 -- 172.21.15.31:0/1460383885 shutdown_connections 2023-10-06T05:16:53.121 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.118+0000 7f09195ff700 1 --2- 172.21.15.31:0/1460383885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 0x7f09140c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.121 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.118+0000 7f09195ff700 1 --2- 172.21.15.31:0/1460383885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09140ca220 0x7f09140686b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.121 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.118+0000 7f09195ff700 1 -- 172.21.15.31:0/1460383885 >> 172.21.15.31:0/1460383885 conn(0x7f09140ce300 msgr2=0x7f09140ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:53.121 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.118+0000 7f09195ff700 1 -- 172.21.15.31:0/1460383885 shutdown_connections 2023-10-06T05:16:53.121 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.118+0000 7f09195ff700 1 -- 172.21.15.31:0/1460383885 wait complete. 2023-10-06T05:16:53.124 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.119+0000 7f09195ff700 1 Processor -- start 2023-10-06T05:16:53.124 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.119+0000 7f09195ff700 1 -- start start 2023-10-06T05:16:53.124 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.120+0000 7f09195ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 0x7f09140b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.124 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.120+0000 7f09195ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f091406ba30 0x7f0914169c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.124 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.120+0000 7f09195ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09140ca220 0x7f0914153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.124 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.120+0000 7f09195ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091416ac10 con 0x7f091406ba30 2023-10-06T05:16:53.124 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.120+0000 7f09195ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091416a140 con 0x7f09140ca220 2023-10-06T05:16:53.124 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.120+0000 7f09195ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09140b2450 con 0x7f0914068bf0 2023-10-06T05:16:53.124 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.120+0000 7f09127fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f091406ba30 0x7f0914169c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.124 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.120+0000 7f09127fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f091406ba30 0x7f0914169c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49196/0 (socket says 172.21.15.31:49196) 2023-10-06T05:16:53.125 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.120+0000 7f09127fc700 1 -- 172.21.15.31:0/902431512 learned_addr learned my addr 172.21.15.31:0/902431512 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:53.125 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f09137fe700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09140ca220 0x7f0914153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.125 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f09127fc700 1 -- 172.21.15.31:0/902431512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 msgr2=0x7f09140b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:53.125 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f0912ffd700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 0x7f09140b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.125 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f09127fc700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 0x7f09140b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.125 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f09127fc700 1 -- 172.21.15.31:0/902431512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09140ca220 msgr2=0x7f0914153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:53.125 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f09127fc700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09140ca220 0x7f0914153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.126 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f09127fc700 1 -- 172.21.15.31:0/902431512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908009420 con 0x7f091406ba30 2023-10-06T05:16:53.126 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f09137fe700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09140ca220 0x7f0914153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:53.126 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f0912ffd700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 0x7f09140b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:53.126 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f09127fc700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f091406ba30 0x7f0914169c00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f091406b2a0 tx=0x7f0908017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:53.126 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f08faffd700 1 -- 172.21.15.31:0/902431512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090800bbe0 con 0x7f091406ba30 2023-10-06T05:16:53.126 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f08faffd700 1 -- 172.21.15.31:0/902431512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090801a430 con 0x7f091406ba30 2023-10-06T05:16:53.126 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f09195ff700 1 -- 172.21.15.31:0/902431512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091416a370 con 0x7f091406ba30 2023-10-06T05:16:53.126 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.121+0000 7f08faffd700 1 -- 172.21.15.31:0/902431512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090801a7f0 con 0x7f091406ba30 2023-10-06T05:16:53.126 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.122+0000 7f09195ff700 1 -- 172.21.15.31:0/902431512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091416a930 con 0x7f091406ba30 2023-10-06T05:16:53.127 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.122+0000 7f09195ff700 1 -- 172.21.15.31:0/902431512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09140b2220 con 0x7f091406ba30 2023-10-06T05:16:53.127 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.123+0000 7f08faffd700 1 -- 172.21.15.31:0/902431512 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f090801abe0 con 0x7f091406ba30 2023-10-06T05:16:53.127 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.124+0000 7f08faffd700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0900066ea0 0x7f0900069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.127 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.124+0000 7f08faffd700 1 -- 172.21.15.31:0/902431512 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7f0908085440 con 0x7f091406ba30 2023-10-06T05:16:53.127 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.124+0000 7f0912ffd700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0900066ea0 0x7f0900069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.128 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.125+0000 7f0912ffd700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0900066ea0 0x7f0900069350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09040018d0 tx=0x7f0904009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:53.131 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.130+0000 7f08faffd700 1 -- 172.21.15.31:0/902431512 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0908053480 con 0x7f091406ba30 2023-10-06T05:16:53.344 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.343+0000 7f09195ff700 1 -- 172.21.15.31:0/902431512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f091404c380 con 0x7f091406ba30 2023-10-06T05:16:53.345 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.343+0000 7f08faffd700 1 -- 172.21.15.31:0/902431512 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10444 (secure 0 0 0) 0x7f0908015070 con 0x7f091406ba30 2023-10-06T05:16:53.345 INFO:teuthology.misc.health.smithi031.stdout: 2023-10-06T05:16:53.346 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":13,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:16:53.258246+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:53.348 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 -- 172.21.15.31:0/902431512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0900066ea0 msgr2=0x7f0900069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:53.348 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0900066ea0 0x7f0900069350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09040018d0 tx=0x7f0904009040 comp rx=0 tx=0).stop 2023-10-06T05:16:53.348 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 -- 172.21.15.31:0/902431512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f091406ba30 msgr2=0x7f0914169c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:53.348 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f091406ba30 0x7f0914169c00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f091406b2a0 tx=0x7f0908017000 comp rx=0 tx=0).stop 2023-10-06T05:16:53.348 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 -- 172.21.15.31:0/902431512 shutdown_connections 2023-10-06T05:16:53.348 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09140ca220 0x7f0914153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.348 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0914068bf0 0x7f09140b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.348 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0900066ea0 0x7f0900069350 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.348 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 --2- 172.21.15.31:0/902431512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f091406ba30 0x7f0914169c00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.349 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 -- 172.21.15.31:0/902431512 >> 172.21.15.31:0/902431512 conn(0x7f09140ce300 msgr2=0x7f0914152cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:53.349 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 -- 172.21.15.31:0/902431512 shutdown_connections 2023-10-06T05:16:53.349 INFO:teuthology.misc.health.smithi031.stderr:2023-10-06T05:16:53.346+0000 7f09195ff700 1 -- 172.21.15.31:0/902431512 wait complete. 2023-10-06T05:16:53.363 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T05:16:53.363 INFO:tasks.ceph:Creating RBD pool 2023-10-06T05:16:53.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T05:16:53.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.551+0000 7fcb861ff700 1 Processor -- start 2023-10-06T05:16:53.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.551+0000 7fcb861ff700 1 -- start start 2023-10-06T05:16:53.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.552+0000 7fcb861ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcb80159930 0x7fcb800b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.552+0000 7fcb861ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 0x7fcb800b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.552+0000 7fcb861ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8015d2d0 con 0x7fcb80159560 2023-10-06T05:16:53.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.552+0000 7fcb861ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8015d5b0 con 0x7fcb80159930 2023-10-06T05:16:53.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.552+0000 7fcb861ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8015d890 con 0x7fcb800cd580 2023-10-06T05:16:53.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 0x7fcb800b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7effd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcb80159930 0x7fcb800b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7effd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcb80159930 0x7fcb800b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:54216/0 (socket says 172.21.15.31:54216) 2023-10-06T05:16:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 0x7fcb800b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56064/0 (socket says 172.21.15.31:56064) 2023-10-06T05:16:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7effd700 1 -- 172.21.15.31:0/2974208843 learned_addr learned my addr 172.21.15.31:0/2974208843 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7f7fe700 1 --1- 172.21.15.31:0/2974208843 >> v1:172.21.15.62:6789/0 conn(0x7fcb80159560 0x7fcb800cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:53954/0 (socket says 172.21.15.31:53954) 2023-10-06T05:16:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7effd700 1 -- 172.21.15.31:0/2974208843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 msgr2=0x7fcb800b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7effd700 1 --2- 172.21.15.31:0/2974208843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 0x7fcb800b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7effd700 1 -- 172.21.15.31:0/2974208843 >> v1:172.21.15.62:6789/0 conn(0x7fcb80159560 legacy=0x7fcb800cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7effd700 1 -- 172.21.15.31:0/2974208843 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8015db70 con 0x7fcb80159930 2023-10-06T05:16:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7ffff700 1 --2- 172.21.15.31:0/2974208843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 0x7fcb800b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.553+0000 7fcb7effd700 1 --2- 172.21.15.31:0/2974208843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb80159930 0x7fcb800b2470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcb740099b0 tx=0x7fcb74009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef72125c9696bc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb7e7fc700 1 -- 172.21.15.31:0/2974208843 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74009ea0 con 0x7fcb80159930 2023-10-06T05:16:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb7e7fc700 1 -- 172.21.15.31:0/2974208843 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7400b860 con 0x7fcb80159930 2023-10-06T05:16:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb7e7fc700 1 -- 172.21.15.31:0/2974208843 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400bc00 con 0x7fcb80159930 2023-10-06T05:16:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb861ff700 1 -- 172.21.15.31:0/2974208843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb80159930 msgr2=0x7fcb800b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb861ff700 1 --2- 172.21.15.31:0/2974208843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb80159930 0x7fcb800b2470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcb740099b0 tx=0x7fcb74009250 comp rx=0 tx=0).stop 2023-10-06T05:16:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb861ff700 1 -- 172.21.15.31:0/2974208843 shutdown_connections 2023-10-06T05:16:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb861ff700 1 --2- 172.21.15.31:0/2974208843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 0x7fcb800b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb861ff700 1 --2- 172.21.15.31:0/2974208843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb80159930 0x7fcb800b2470 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb861ff700 1 -- 172.21.15.31:0/2974208843 >> 172.21.15.31:0/2974208843 conn(0x7fcb8006ec90 msgr2=0x7fcb8006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb861ff700 1 -- 172.21.15.31:0/2974208843 shutdown_connections 2023-10-06T05:16:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.554+0000 7fcb861ff700 1 -- 172.21.15.31:0/2974208843 wait complete. 2023-10-06T05:16:53.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.556+0000 7fcb861ff700 1 Processor -- start 2023-10-06T05:16:53.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.556+0000 7fcb861ff700 1 -- start start 2023-10-06T05:16:53.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb861ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 0x7fcb800c3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb861ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb80159560 0x7fcb8015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb861ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb80159930 0x7fcb8015c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb861ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800ced00 con 0x7fcb80159560 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb861ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800ce0f0 con 0x7fcb80159930 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb861ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800ce3d0 con 0x7fcb800cd580 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb7effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb80159560 0x7fcb8015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb7effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb80159560 0x7fcb8015bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49198/0 (socket says 172.21.15.31:49198) 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb7effd700 1 -- 172.21.15.31:0/1147354930 learned_addr learned my addr 172.21.15.31:0/1147354930 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb7f7fe700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 0x7fcb800c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.557+0000 7fcb7ffff700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb80159930 0x7fcb8015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb7effd700 1 -- 172.21.15.31:0/1147354930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 msgr2=0x7fcb800c3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb7effd700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 0x7fcb800c3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb7effd700 1 -- 172.21.15.31:0/1147354930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb80159930 msgr2=0x7fcb8015c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb7effd700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb80159930 0x7fcb8015c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb7effd700 1 -- 172.21.15.31:0/1147354930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb78009a20 con 0x7fcb80159560 2023-10-06T05:16:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb7effd700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb80159560 0x7fcb8015bb50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcb800893d0 tx=0x7fcb74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb677fe700 1 -- 172.21.15.31:0/1147354930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400bbe0 con 0x7fcb80159560 2023-10-06T05:16:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb677fe700 1 -- 172.21.15.31:0/1147354930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb74011420 con 0x7fcb80159560 2023-10-06T05:16:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb677fe700 1 -- 172.21.15.31:0/1147354930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb740117c0 con 0x7fcb80159560 2023-10-06T05:16:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb861ff700 1 -- 172.21.15.31:0/1147354930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb740094f0 con 0x7fcb80159560 2023-10-06T05:16:53.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.558+0000 7fcb861ff700 1 -- 172.21.15.31:0/1147354930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb800ce990 con 0x7fcb80159560 2023-10-06T05:16:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.560+0000 7fcb861ff700 1 -- 172.21.15.31:0/1147354930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb800ceeb0 con 0x7fcb80159560 2023-10-06T05:16:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.560+0000 7fcb677fe700 1 -- 172.21.15.31:0/1147354930 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fcb74011bf0 con 0x7fcb80159560 2023-10-06T05:16:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.562+0000 7fcb677fe700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcb6c066ea0 0x7fcb6c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.562+0000 7fcb677fe700 1 -- 172.21.15.31:0/1147354930 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fcb74085370 con 0x7fcb80159560 2023-10-06T05:16:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.562+0000 7fcb7f7fe700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcb6c066ea0 0x7fcb6c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.562+0000 7fcb7f7fe700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcb6c066ea0 0x7fcb6c069350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb70004fd0 tx=0x7fcb70007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:53.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.566+0000 7fcb677fe700 1 -- 172.21.15.31:0/1147354930 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb800ceeb0 con 0x7fcb80159560 2023-10-06T05:16:53.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:53.767+0000 7fcb861ff700 1 -- 172.21.15.31:0/1147354930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fcb8004c380 con 0x7fcb80159560 2023-10-06T05:16:54.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.382+0000 7fcb677fe700 1 -- 172.21.15.31:0/1147354930 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7fcb74052cd0 con 0x7fcb80159560 2023-10-06T05:16:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 -- 172.21.15.31:0/1147354930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcb6c066ea0 msgr2=0x7fcb6c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcb6c066ea0 0x7fcb6c069350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb70004fd0 tx=0x7fcb70007eb0 comp rx=0 tx=0).stop 2023-10-06T05:16:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 -- 172.21.15.31:0/1147354930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb80159560 msgr2=0x7fcb8015bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb80159560 0x7fcb8015bb50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcb800893d0 tx=0x7fcb74017000 comp rx=0 tx=0).stop 2023-10-06T05:16:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 -- 172.21.15.31:0/1147354930 shutdown_connections 2023-10-06T05:16:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb80159930 0x7fcb8015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb800cd580 0x7fcb800c3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:54.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcb6c066ea0 0x7fcb6c069350 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:54.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 --2- 172.21.15.31:0/1147354930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb80159560 0x7fcb8015bb50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:54.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 -- 172.21.15.31:0/1147354930 >> 172.21.15.31:0/1147354930 conn(0x7fcb8006ec90 msgr2=0x7fcb8015a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:54.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 -- 172.21.15.31:0/1147354930 shutdown_connections 2023-10-06T05:16:54.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.385+0000 7fcb861ff700 1 -- 172.21.15.31:0/1147354930 wait complete. 2023-10-06T05:16:54.388 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'rbd' created 2023-10-06T05:16:54.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T05:16:54.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.594+0000 7fa6635ff700 1 Processor -- start 2023-10-06T05:16:54.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.594+0000 7fa6635ff700 1 -- start start 2023-10-06T05:16:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.595+0000 7fa6635ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa65c153d20 0x7fa65c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.595+0000 7fa6635ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa65c15d750 0x7fa65c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.595+0000 7fa6635ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c15dbb0 con 0x7fa65c153950 2023-10-06T05:16:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.595+0000 7fa6635ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c154170 con 0x7fa65c15d750 2023-10-06T05:16:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.595+0000 7fa6635ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c15bb20 con 0x7fa65c153d20 2023-10-06T05:16:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.595+0000 7fa66139b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa65c153950 0x7fa65c0cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43492/0 (socket says 172.21.15.31:43492) 2023-10-06T05:16:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.595+0000 7fa66139b700 1 -- 172.21.15.31:0/2359882461 learned_addr learned my addr 172.21.15.31:0/2359882461 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:16:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.596+0000 7fa661b9c700 1 --2- 172.21.15.31:0/2359882461 >> v2:172.21.15.31:3300/0 conn(0x7fa65c15d750 0x7fa65c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:54.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.596+0000 7fa660b9a700 1 --2- 172.21.15.31:0/2359882461 >> v2:172.21.15.62:3300/0 conn(0x7fa65c153d20 0x7fa65c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.596+0000 7fa661b9c700 1 -- 172.21.15.31:0/2359882461 >> v2:172.21.15.62:3300/0 conn(0x7fa65c153d20 msgr2=0x7fa65c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.596+0000 7fa661b9c700 1 --2- 172.21.15.31:0/2359882461 >> v2:172.21.15.62:3300/0 conn(0x7fa65c153d20 0x7fa65c15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.596+0000 7fa661b9c700 1 -- 172.21.15.31:0/2359882461 >> v1:172.21.15.31:6789/0 conn(0x7fa65c153950 legacy=0x7fa65c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.596+0000 7fa661b9c700 1 -- 172.21.15.31:0/2359882461 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c15af10 con 0x7fa65c15d750 2023-10-06T05:16:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.596+0000 7fa660b9a700 1 --2- 172.21.15.31:0/2359882461 >> v2:172.21.15.62:3300/0 conn(0x7fa65c153d20 0x7fa65c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.596+0000 7fa661b9c700 1 --2- 172.21.15.31:0/2359882461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c15d750 0x7fa65c15bfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa65c157800 tx=0x7fa654009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65694ac6b1b4e125 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.596+0000 7fa653fff700 1 -- 172.21.15.31:0/2359882461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400e070 con 0x7fa65c15d750 2023-10-06T05:16:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.597+0000 7fa653fff700 1 -- 172.21.15.31:0/2359882461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65400bc30 con 0x7fa65c15d750 2023-10-06T05:16:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.597+0000 7fa653fff700 1 -- 172.21.15.31:0/2359882461 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654012430 con 0x7fa65c15d750 2023-10-06T05:16:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.597+0000 7fa6635ff700 1 -- 172.21.15.31:0/2359882461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c15d750 msgr2=0x7fa65c15bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.597+0000 7fa6635ff700 1 --2- 172.21.15.31:0/2359882461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c15d750 0x7fa65c15bfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa65c157800 tx=0x7fa654009780 comp rx=0 tx=0).stop 2023-10-06T05:16:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.597+0000 7fa6635ff700 1 -- 172.21.15.31:0/2359882461 shutdown_connections 2023-10-06T05:16:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.597+0000 7fa6635ff700 1 --2- 172.21.15.31:0/2359882461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c15d750 0x7fa65c15bfb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.597+0000 7fa6635ff700 1 --2- 172.21.15.31:0/2359882461 >> v2:172.21.15.62:3300/0 conn(0x7fa65c153d20 0x7fa65c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.597+0000 7fa6635ff700 1 -- 172.21.15.31:0/2359882461 >> 172.21.15.31:0/2359882461 conn(0x7fa65c0681b0 msgr2=0x7fa65c0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.597+0000 7fa6635ff700 1 -- 172.21.15.31:0/2359882461 shutdown_connections 2023-10-06T05:16:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.597+0000 7fa6635ff700 1 -- 172.21.15.31:0/2359882461 wait complete. 2023-10-06T05:16:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.599+0000 7fa6635ff700 1 Processor -- start 2023-10-06T05:16:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.599+0000 7fa6635ff700 1 -- start start 2023-10-06T05:16:54.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa6635ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c153950 0x7fa65c0ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:54.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa6635ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa65c153d20 0x7fa65c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:54.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa6635ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa65c15d750 0x7fa65c16d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:54.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa6635ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c14f4f0 con 0x7fa65c153950 2023-10-06T05:16:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa6635ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c0b55d0 con 0x7fa65c15d750 2023-10-06T05:16:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa6635ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c0b58a0 con 0x7fa65c153d20 2023-10-06T05:16:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa66139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c153950 0x7fa65c0ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa66139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c153950 0x7fa65c0ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49228/0 (socket says 172.21.15.31:49228) 2023-10-06T05:16:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa66139b700 1 -- 172.21.15.31:0/3667555094 learned_addr learned my addr 172.21.15.31:0/3667555094 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa661b9c700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa65c15d750 0x7fa65c16d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa660b9a700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa65c153d20 0x7fa65c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa66139b700 1 -- 172.21.15.31:0/3667555094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa65c153d20 msgr2=0x7fa65c0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa66139b700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa65c153d20 0x7fa65c0b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa66139b700 1 -- 172.21.15.31:0/3667555094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa65c15d750 msgr2=0x7fa65c16d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa66139b700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa65c15d750 0x7fa65c16d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa66139b700 1 -- 172.21.15.31:0/3667555094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa654009a20 con 0x7fa65c153950 2023-10-06T05:16:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.600+0000 7fa661b9c700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa65c15d750 0x7fa65c16d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:16:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.601+0000 7fa66139b700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c153950 0x7fa65c0ca150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c590 tx=0x7fa658007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.601+0000 7fa6517fa700 1 -- 172.21.15.31:0/3667555094 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6580072e0 con 0x7fa65c153950 2023-10-06T05:16:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.601+0000 7fa6517fa700 1 -- 172.21.15.31:0/3667555094 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa658007460 con 0x7fa65c153950 2023-10-06T05:16:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.601+0000 7fa6635ff700 1 -- 172.21.15.31:0/3667555094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65c0b5b80 con 0x7fa65c153950 2023-10-06T05:16:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.601+0000 7fa6517fa700 1 -- 172.21.15.31:0/3667555094 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6580057f0 con 0x7fa65c153950 2023-10-06T05:16:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.601+0000 7fa6635ff700 1 -- 172.21.15.31:0/3667555094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c14ec90 con 0x7fa65c153950 2023-10-06T05:16:54.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.602+0000 7fa6635ff700 1 -- 172.21.15.31:0/3667555094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65c0b03d0 con 0x7fa65c153950 2023-10-06T05:16:54.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.603+0000 7fa6517fa700 1 -- 172.21.15.31:0/3667555094 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fa658005be0 con 0x7fa65c153950 2023-10-06T05:16:54.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.604+0000 7fa6517fa700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa640066ea0 0x7fa640069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:54.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.604+0000 7fa6517fa700 1 -- 172.21.15.31:0/3667555094 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5184+0+0 (secure 0 0 0) 0x7fa6580834e0 con 0x7fa65c153950 2023-10-06T05:16:54.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.604+0000 7fa660b9a700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa640066ea0 0x7fa640069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:54.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.605+0000 7fa660b9a700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa640066ea0 0x7fa640069350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa64c006f00 tx=0x7fa64c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:54.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.609+0000 7fa6517fa700 1 -- 172.21.15.31:0/3667555094 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa65804b5a0 con 0x7fa65c153950 2023-10-06T05:16:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:54.813+0000 7fa6635ff700 1 -- 172.21.15.31:0/3667555094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa65c15dbb0 con 0x7fa65c153950 2023-10-06T05:16:55.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.383+0000 7fa6517fa700 1 -- 172.21.15.31:0/3667555094 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa65c15dbb0 con 0x7fa65c153950 2023-10-06T05:16:55.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 -- 172.21.15.31:0/3667555094 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa640066ea0 msgr2=0x7fa640069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:55.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa640066ea0 0x7fa640069350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa64c006f00 tx=0x7fa64c008040 comp rx=0 tx=0).stop 2023-10-06T05:16:55.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 -- 172.21.15.31:0/3667555094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c153950 msgr2=0x7fa65c0ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:55.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c153950 0x7fa65c0ca150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c590 tx=0x7fa658007e40 comp rx=0 tx=0).stop 2023-10-06T05:16:55.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 -- 172.21.15.31:0/3667555094 shutdown_connections 2023-10-06T05:16:55.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa65c15d750 0x7fa65c16d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa640066ea0 0x7fa640069350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa65c153950 0x7fa65c0ca150 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 --2- 172.21.15.31:0/3667555094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa65c153d20 0x7fa65c0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 -- 172.21.15.31:0/3667555094 >> 172.21.15.31:0/3667555094 conn(0x7fa65c0681b0 msgr2=0x7fa65c0c6850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:55.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 -- 172.21.15.31:0/3667555094 shutdown_connections 2023-10-06T05:16:55.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.387+0000 7fa6635ff700 1 -- 172.21.15.31:0/3667555094 wait complete. 2023-10-06T05:16:55.391 INFO:teuthology.orchestra.run.smithi031.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T05:16:55.412 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T05:16:55.413 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:16:55.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:16:55.417 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:16:55.417 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:16:55.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:16:55.422 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:16:55.422 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:16:55.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:16:55.427 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:16:55.427 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:16:55.427 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:16:55.432 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:16:55.432 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T05:16:55.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:55.587 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:16:55.587 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:16:55.588 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:16:55.592 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:16:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.663+0000 7f35ab5ff700 1 Processor -- start 2023-10-06T05:16:55.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.663+0000 7f35ab5ff700 1 -- start start 2023-10-06T05:16:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.664+0000 7f35ab5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35a4159500 0x7f35a40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.664+0000 7f35ab5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35a40cd510 0x7f35a40aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:55.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.664+0000 7f35ab5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a40c3f40 con 0x7f35a415d2d0 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.664+0000 7f35ab5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a40b2860 con 0x7f35a4159500 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.664+0000 7f35ab5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a415d7d0 con 0x7f35a40cd510 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.665+0000 7f35a939b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35a4159500 0x7f35a40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.665+0000 7f35a939b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35a4159500 0x7f35a40b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49272/0 (socket says 172.21.15.31:49272) 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.665+0000 7f35a939b700 1 -- 172.21.15.31:0/526106279 learned_addr learned my addr 172.21.15.31:0/526106279 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.665+0000 7f35a9b9c700 1 --2- 172.21.15.31:0/526106279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35a40cd510 0x7f35a40aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.665+0000 7f35a939b700 1 -- 172.21.15.31:0/526106279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35a40cd510 msgr2=0x7f35a40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.665+0000 7f35a939b700 1 --2- 172.21.15.31:0/526106279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35a40cd510 0x7f35a40aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.665+0000 7f35a939b700 1 -- 172.21.15.31:0/526106279 >> v1:172.21.15.31:6789/0 conn(0x7f35a415d2d0 legacy=0x7f35a40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.665+0000 7f35a939b700 1 -- 172.21.15.31:0/526106279 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40cb000 con 0x7f35a4159500 2023-10-06T05:16:55.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.665+0000 7f359bfff700 1 -- 172.21.15.31:0/526106279 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508883983 0 0) 0x7f35a40c3f40 con 0x7f35a415d2d0 2023-10-06T05:16:55.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.665+0000 7f35a939b700 1 --2- 172.21.15.31:0/526106279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35a4159500 0x7f35a40b0ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f35a00099b0 tx=0x7f35a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73a6959bacdf918 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:55.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f359bfff700 1 -- 172.21.15.31:0/526106279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0009ea0 con 0x7f35a4159500 2023-10-06T05:16:55.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f359bfff700 1 -- 172.21.15.31:0/526106279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a000b860 con 0x7f35a4159500 2023-10-06T05:16:55.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f359bfff700 1 -- 172.21.15.31:0/526106279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000bc00 con 0x7f35a4159500 2023-10-06T05:16:55.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f35ab5ff700 1 -- 172.21.15.31:0/526106279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35a4159500 msgr2=0x7f35a40b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:55.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f35ab5ff700 1 --2- 172.21.15.31:0/526106279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35a4159500 0x7f35a40b0ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f35a00099b0 tx=0x7f35a0009250 comp rx=0 tx=0).stop 2023-10-06T05:16:55.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f35ab5ff700 1 -- 172.21.15.31:0/526106279 shutdown_connections 2023-10-06T05:16:55.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f35ab5ff700 1 --2- 172.21.15.31:0/526106279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35a4159500 0x7f35a40b0ab0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f35ab5ff700 1 --2- 172.21.15.31:0/526106279 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35a40cd510 0x7f35a40aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f35ab5ff700 1 -- 172.21.15.31:0/526106279 >> 172.21.15.31:0/526106279 conn(0x7f35a406ec90 msgr2=0x7f35a406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:55.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f35ab5ff700 1 -- 172.21.15.31:0/526106279 shutdown_connections 2023-10-06T05:16:55.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.666+0000 7f35ab5ff700 1 -- 172.21.15.31:0/526106279 wait complete. 2023-10-06T05:16:55.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.668+0000 7f35ab5ff700 1 Processor -- start 2023-10-06T05:16:55.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.669+0000 7f35ab5ff700 1 -- start start 2023-10-06T05:16:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35ab5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a40cd510 0x7f35a416cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35ab5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35a4159500 0x7f35a416b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35ab5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35a415d2d0 0x7f35a40c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35ab5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40c9c90 con 0x7f35a415d2d0 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35ab5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40b3da0 con 0x7f35a40cd510 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35ab5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40b3470 con 0x7f35a4159500 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35a939b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a40cd510 0x7f35a416cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35a9b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35a415d2d0 0x7f35a40c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35a939b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a40cd510 0x7f35a416cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:54320/0 (socket says 172.21.15.31:54320) 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35a939b700 1 -- 172.21.15.31:0/3548023401 learned_addr learned my addr 172.21.15.31:0/3548023401 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.670+0000 7f35a8b9a700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35a4159500 0x7f35a416b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35a9b9c700 1 -- 172.21.15.31:0/3548023401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35a4159500 msgr2=0x7f35a416b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35a9b9c700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35a4159500 0x7f35a416b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35a9b9c700 1 -- 172.21.15.31:0/3548023401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a40cd510 msgr2=0x7f35a416cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:55.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35a9b9c700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a40cd510 0x7f35a416cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35a9b9c700 1 -- 172.21.15.31:0/3548023401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a00094f0 con 0x7f35a415d2d0 2023-10-06T05:16:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35a939b700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a40cd510 0x7f35a416cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35a9b9c700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35a415d2d0 0x7f35a40c9240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f359c009e50 tx=0x7f359c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35997fa700 1 -- 172.21.15.31:0/3548023401 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00caa0 con 0x7f35a415d2d0 2023-10-06T05:16:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35997fa700 1 -- 172.21.15.31:0/3548023401 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c00f040 con 0x7f35a415d2d0 2023-10-06T05:16:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35997fa700 1 -- 172.21.15.31:0/3548023401 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00ce70 con 0x7f35a415d2d0 2023-10-06T05:16:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35ab5ff700 1 -- 172.21.15.31:0/3548023401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a40b3750 con 0x7f35a415d2d0 2023-10-06T05:16:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.671+0000 7f35ab5ff700 1 -- 172.21.15.31:0/3548023401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a416d4a0 con 0x7f35a415d2d0 2023-10-06T05:16:55.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.672+0000 7f35ab5ff700 1 -- 172.21.15.31:0/3548023401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a404c380 con 0x7f35a415d2d0 2023-10-06T05:16:55.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.673+0000 7f35997fa700 1 -- 172.21.15.31:0/3548023401 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f359c014690 con 0x7f35a415d2d0 2023-10-06T05:16:55.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.674+0000 7f35997fa700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3588066ea0 0x7f3588069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:55.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.674+0000 7f35997fa700 1 -- 172.21.15.31:0/3548023401 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f359c084e20 con 0x7f35a415d2d0 2023-10-06T05:16:55.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.674+0000 7f35a939b700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3588066ea0 0x7f3588069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:55.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.675+0000 7f35a939b700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3588066ea0 0x7f3588069350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f35a0003080 tx=0x7f35a0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:55.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.682+0000 7f35997fa700 1 -- 172.21.15.31:0/3548023401 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f359c0522c0 con 0x7f35a415d2d0 2023-10-06T05:16:55.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.903+0000 7f35ab5ff700 1 -- 172.21.15.31:0/3548023401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35a40cade0 con 0x7f35a415d2d0 2023-10-06T05:16:55.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.904+0000 7f35997fa700 1 -- 172.21.15.31:0/3548023401 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+13722 (secure 0 0 0) 0x7f359c0558e0 con 0x7f35a415d2d0 2023-10-06T05:16:55.906 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:55.906 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":15,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:16:55.378495+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:55.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.906+0000 7f35ab5ff700 1 -- 172.21.15.31:0/3548023401 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3588066ea0 msgr2=0x7f3588069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:55.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.906+0000 7f35ab5ff700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3588066ea0 0x7f3588069350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f35a0003080 tx=0x7f35a0017000 comp rx=0 tx=0).stop 2023-10-06T05:16:55.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.906+0000 7f35ab5ff700 1 -- 172.21.15.31:0/3548023401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35a415d2d0 msgr2=0x7f35a40c9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:55.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.906+0000 7f35ab5ff700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35a415d2d0 0x7f35a40c9240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f359c009e50 tx=0x7f359c00b040 comp rx=0 tx=0).stop 2023-10-06T05:16:55.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.906+0000 7f35ab5ff700 1 -- 172.21.15.31:0/3548023401 shutdown_connections 2023-10-06T05:16:55.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.906+0000 7f35ab5ff700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3588066ea0 0x7f3588069350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.906+0000 7f35ab5ff700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35a415d2d0 0x7f35a40c9240 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.906+0000 7f35ab5ff700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a40cd510 0x7f35a416cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.906+0000 7f35ab5ff700 1 --2- 172.21.15.31:0/3548023401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35a4159500 0x7f35a416b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:55.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.906+0000 7f35ab5ff700 1 -- 172.21.15.31:0/3548023401 >> 172.21.15.31:0/3548023401 conn(0x7f35a406ec90 msgr2=0x7f35a40af080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:55.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.907+0000 7f35ab5ff700 1 -- 172.21.15.31:0/3548023401 shutdown_connections 2023-10-06T05:16:55.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:55.907+0000 7f35ab5ff700 1 -- 172.21.15.31:0/3548023401 wait complete. 2023-10-06T05:16:55.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:16:55.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:16:56.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.130+0000 7fca179ff700 1 Processor -- start 2023-10-06T05:16:56.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.130+0000 7fca179ff700 1 -- start start 2023-10-06T05:16:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.131+0000 7fca179ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca10159560 0x7fca100cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.131+0000 7fca179ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca100b2b80 0x7fca100cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.131+0000 7fca179ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1015d2d0 con 0x7fca10159930 2023-10-06T05:16:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.131+0000 7fca179ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1015d5b0 con 0x7fca100b2b80 2023-10-06T05:16:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.131+0000 7fca179ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1015d890 con 0x7fca10159560 2023-10-06T05:16:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca15f9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca100b2b80 0x7fca100cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca1579b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca10159560 0x7fca100cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca15f9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca100b2b80 0x7fca100cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:54326/0 (socket says 172.21.15.31:54326) 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca15f9c700 1 -- 172.21.15.31:0/3690540964 learned_addr learned my addr 172.21.15.31:0/3690540964 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca14f9a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fca10159930 0x7fca100b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:53974/0 (socket says 172.21.15.31:53974) 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca07fff700 1 -- 172.21.15.31:0/3690540964 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417817200 0 0) 0x7fca1015d2d0 con 0x7fca10159930 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca15f9c700 1 -- 172.21.15.31:0/3690540964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca10159560 msgr2=0x7fca100cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca15f9c700 1 --2- 172.21.15.31:0/3690540964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca10159560 0x7fca100cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca15f9c700 1 -- 172.21.15.31:0/3690540964 >> v1:172.21.15.62:6789/0 conn(0x7fca10159930 legacy=0x7fca100b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca15f9c700 1 -- 172.21.15.31:0/3690540964 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1015db70 con 0x7fca100b2b80 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.132+0000 7fca15f9c700 1 --2- 172.21.15.31:0/3690540964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca100b2b80 0x7fca100cd3b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca08009f20 tx=0x7fca08009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=612f8ac5ebc2d985 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.133+0000 7fca07fff700 1 -- 172.21.15.31:0/3690540964 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0800e070 con 0x7fca100b2b80 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.133+0000 7fca07fff700 1 -- 172.21.15.31:0/3690540964 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0800b9d0 con 0x7fca100b2b80 2023-10-06T05:16:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.133+0000 7fca07fff700 1 -- 172.21.15.31:0/3690540964 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca080092e0 con 0x7fca100b2b80 2023-10-06T05:16:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.133+0000 7fca179ff700 1 -- 172.21.15.31:0/3690540964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca100b2b80 msgr2=0x7fca100cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.133+0000 7fca179ff700 1 --2- 172.21.15.31:0/3690540964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca100b2b80 0x7fca100cd3b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca08009f20 tx=0x7fca08009840 comp rx=0 tx=0).stop 2023-10-06T05:16:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.133+0000 7fca179ff700 1 -- 172.21.15.31:0/3690540964 shutdown_connections 2023-10-06T05:16:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.133+0000 7fca179ff700 1 --2- 172.21.15.31:0/3690540964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca10159560 0x7fca100cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.133+0000 7fca179ff700 1 --2- 172.21.15.31:0/3690540964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca100b2b80 0x7fca100cd3b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.134+0000 7fca179ff700 1 -- 172.21.15.31:0/3690540964 >> 172.21.15.31:0/3690540964 conn(0x7fca1006ec90 msgr2=0x7fca1006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.134+0000 7fca179ff700 1 -- 172.21.15.31:0/3690540964 shutdown_connections 2023-10-06T05:16:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.134+0000 7fca179ff700 1 -- 172.21.15.31:0/3690540964 wait complete. 2023-10-06T05:16:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.135+0000 7fca179ff700 1 Processor -- start 2023-10-06T05:16:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.135+0000 7fca179ff700 1 -- start start 2023-10-06T05:16:56.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca179ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100b2b80 0x7fca100c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca179ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159560 0x7fca1016c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca179ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca10159930 0x7fca10158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca179ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca100c9410 con 0x7fca10159930 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca179ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca100c5d00 con 0x7fca10159560 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca179ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca100c8800 con 0x7fca100b2b80 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca15f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca10159930 0x7fca10158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca1579b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100b2b80 0x7fca100c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca14f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159560 0x7fca1016c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca1579b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100b2b80 0x7fca100c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56176/0 (socket says 172.21.15.31:56176) 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.136+0000 7fca1579b700 1 -- 172.21.15.31:0/287947092 learned_addr learned my addr 172.21.15.31:0/287947092 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.137+0000 7fca1579b700 1 -- 172.21.15.31:0/287947092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159560 msgr2=0x7fca1016c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.137+0000 7fca1579b700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159560 0x7fca1016c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.137+0000 7fca1579b700 1 -- 172.21.15.31:0/287947092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca10159930 msgr2=0x7fca10158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.137+0000 7fca1579b700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca10159930 0x7fca10158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.137+0000 7fca1579b700 1 -- 172.21.15.31:0/287947092 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca08009ae0 con 0x7fca100b2b80 2023-10-06T05:16:56.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.137+0000 7fca14f9a700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159560 0x7fca1016c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:56.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.137+0000 7fca1579b700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100b2b80 0x7fca100c57c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00ecb0 tx=0x7fca0c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:56.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.137+0000 7fca057fa700 1 -- 172.21.15.31:0/287947092 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00c850 con 0x7fca100b2b80 2023-10-06T05:16:56.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.138+0000 7fca057fa700 1 -- 172.21.15.31:0/287947092 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c00c9b0 con 0x7fca100b2b80 2023-10-06T05:16:56.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.138+0000 7fca057fa700 1 -- 172.21.15.31:0/287947092 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00cd50 con 0x7fca100b2b80 2023-10-06T05:16:56.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.138+0000 7fca179ff700 1 -- 172.21.15.31:0/287947092 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca100c8ae0 con 0x7fca100b2b80 2023-10-06T05:16:56.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.138+0000 7fca179ff700 1 -- 172.21.15.31:0/287947092 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca100c90a0 con 0x7fca100b2b80 2023-10-06T05:16:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.139+0000 7fca179ff700 1 -- 172.21.15.31:0/287947092 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca10157d60 con 0x7fca100b2b80 2023-10-06T05:16:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.140+0000 7fca057fa700 1 -- 172.21.15.31:0/287947092 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fca0c013680 con 0x7fca100b2b80 2023-10-06T05:16:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.140+0000 7fca057fa700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9f4066f20 0x7fc9f40693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.141+0000 7fca057fa700 1 -- 172.21.15.31:0/287947092 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fca0c084520 con 0x7fca100b2b80 2023-10-06T05:16:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.141+0000 7fca14f9a700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9f4066f20 0x7fc9f40693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.141+0000 7fca14f9a700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9f4066f20 0x7fc9f40693d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca100b61d0 tx=0x7fca00007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.145+0000 7fca057fa700 1 -- 172.21.15.31:0/287947092 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca0c0519c0 con 0x7fca100b2b80 2023-10-06T05:16:56.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.349+0000 7fca179ff700 1 -- 172.21.15.31:0/287947092 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fca1004c380 con 0x7fca100b2b80 2023-10-06T05:16:56.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.350+0000 7fca057fa700 1 -- 172.21.15.31:0/287947092 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7fca0c054fe0 con 0x7fca100b2b80 2023-10-06T05:16:56.352 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:16:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 -- 172.21.15.31:0/287947092 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9f4066f20 msgr2=0x7fc9f40693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9f4066f20 0x7fc9f40693d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca100b61d0 tx=0x7fca00007ef0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 -- 172.21.15.31:0/287947092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100b2b80 msgr2=0x7fca100c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100b2b80 0x7fca100c57c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00ecb0 tx=0x7fca0c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 -- 172.21.15.31:0/287947092 shutdown_connections 2023-10-06T05:16:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9f4066f20 0x7fc9f40693d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca10159930 0x7fca10158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100b2b80 0x7fca100c57c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 --2- 172.21.15.31:0/287947092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159560 0x7fca1016c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 -- 172.21.15.31:0/287947092 >> 172.21.15.31:0/287947092 conn(0x7fca1006ec90 msgr2=0x7fca100b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 -- 172.21.15.31:0/287947092 shutdown_connections 2023-10-06T05:16:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.352+0000 7fca179ff700 1 -- 172.21.15.31:0/287947092 wait complete. 2023-10-06T05:16:56.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:16:56.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.572+0000 7fece39ff700 1 Processor -- start 2023-10-06T05:16:56.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.573+0000 7fece39ff700 1 -- start start 2023-10-06T05:16:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.574+0000 7fece39ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fecdc15ca40 0x7fecdc068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.574+0000 7fece39ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fecdc0c8f40 0x7fecdc0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.574+0000 7fece39ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecdc068da0 con 0x7fecdc06a300 2023-10-06T05:16:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.574+0000 7fece39ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc0cabc0 con 0x7fecdc0c8f40 2023-10-06T05:16:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.574+0000 7fece39ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc068bb0 con 0x7fecdc15ca40 2023-10-06T05:16:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.574+0000 7fece179b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fecdc15ca40 0x7fecdc068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.574+0000 7fece0f9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fecdc06a300 0x7fecdc0c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43534/0 (socket says 172.21.15.31:43534) 2023-10-06T05:16:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.574+0000 7fece0f9a700 1 -- 172.21.15.31:0/702107563 learned_addr learned my addr 172.21.15.31:0/702107563 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:16:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.574+0000 7fece1f9c700 1 --2- 172.21.15.31:0/702107563 >> v2:172.21.15.31:3300/0 conn(0x7fecdc0c8f40 0x7fecdc0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.575+0000 7fece179b700 1 -- 172.21.15.31:0/702107563 >> v2:172.21.15.31:3300/0 conn(0x7fecdc0c8f40 msgr2=0x7fecdc0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:56.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.575+0000 7fece179b700 1 --2- 172.21.15.31:0/702107563 >> v2:172.21.15.31:3300/0 conn(0x7fecdc0c8f40 0x7fecdc0adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.575+0000 7fecd3fff700 1 -- 172.21.15.31:0/702107563 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346819786 0 0) 0x7fecdc068da0 con 0x7fecdc06a300 2023-10-06T05:16:56.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.575+0000 7fece179b700 1 -- 172.21.15.31:0/702107563 >> v1:172.21.15.31:6789/0 conn(0x7fecdc06a300 legacy=0x7fecdc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.575+0000 7fece179b700 1 -- 172.21.15.31:0/702107563 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc069080 con 0x7fecdc15ca40 2023-10-06T05:16:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.575+0000 7fece1f9c700 1 --2- 172.21.15.31:0/702107563 >> v2:172.21.15.31:3300/0 conn(0x7fecdc0c8f40 0x7fecdc0adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.575+0000 7fece179b700 1 --2- 172.21.15.31:0/702107563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc15ca40 0x7fecdc068670 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fecdc067f80 tx=0x7fecd80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62939aaf06d3f0f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.575+0000 7fecd3fff700 1 -- 172.21.15.31:0/702107563 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8011630 con 0x7fecdc15ca40 2023-10-06T05:16:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.575+0000 7fecd3fff700 1 -- 172.21.15.31:0/702107563 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd8011790 con 0x7fecdc15ca40 2023-10-06T05:16:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.575+0000 7fecd3fff700 1 -- 172.21.15.31:0/702107563 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8011b30 con 0x7fecdc15ca40 2023-10-06T05:16:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.576+0000 7fece39ff700 1 -- 172.21.15.31:0/702107563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc15ca40 msgr2=0x7fecdc068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.576+0000 7fece39ff700 1 --2- 172.21.15.31:0/702107563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc15ca40 0x7fecdc068670 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fecdc067f80 tx=0x7fecd80091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.576+0000 7fece39ff700 1 -- 172.21.15.31:0/702107563 shutdown_connections 2023-10-06T05:16:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.576+0000 7fece39ff700 1 --2- 172.21.15.31:0/702107563 >> v2:172.21.15.31:3300/0 conn(0x7fecdc0c8f40 0x7fecdc0adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.576+0000 7fece39ff700 1 --2- 172.21.15.31:0/702107563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc15ca40 0x7fecdc068670 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.576+0000 7fece39ff700 1 -- 172.21.15.31:0/702107563 >> 172.21.15.31:0/702107563 conn(0x7fecdc0ce290 msgr2=0x7fecdc0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:56.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.576+0000 7fece39ff700 1 -- 172.21.15.31:0/702107563 shutdown_connections 2023-10-06T05:16:56.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.576+0000 7fece39ff700 1 -- 172.21.15.31:0/702107563 wait complete. 2023-10-06T05:16:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.577+0000 7fece39ff700 1 Processor -- start 2023-10-06T05:16:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.578+0000 7fece39ff700 1 -- start start 2023-10-06T05:16:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.578+0000 7fece39ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc06a300 0x7fecdc0c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.578+0000 7fece39ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecdc0c8f40 0x7fecdc14d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.578+0000 7fece39ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15ca40 0x7fecdc0c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.578+0000 7fece39ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc14dcd0 con 0x7fecdc0c8f40 2023-10-06T05:16:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.578+0000 7fece39ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0c35d0 con 0x7fecdc06a300 2023-10-06T05:16:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.578+0000 7fece39ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0ae400 con 0x7fecdc15ca40 2023-10-06T05:16:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece179b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc06a300 0x7fecdc0c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece0f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecdc0c8f40 0x7fecdc14d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece179b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc06a300 0x7fecdc0c2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:54356/0 (socket says 172.21.15.31:54356) 2023-10-06T05:16:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece179b700 1 -- 172.21.15.31:0/1955918288 learned_addr learned my addr 172.21.15.31:0/1955918288 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece1f9c700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15ca40 0x7fecdc0c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece179b700 1 -- 172.21.15.31:0/1955918288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15ca40 msgr2=0x7fecdc0c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece179b700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15ca40 0x7fecdc0c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece179b700 1 -- 172.21.15.31:0/1955918288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecdc0c8f40 msgr2=0x7fecdc14d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece179b700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecdc0c8f40 0x7fecdc14d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece179b700 1 -- 172.21.15.31:0/1955918288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd4009a20 con 0x7fecdc06a300 2023-10-06T05:16:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.579+0000 7fece179b700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc06a300 0x7fecdc0c2c10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fecd8011fa0 tx=0x7fecd8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.580+0000 7fecd17fa700 1 -- 172.21.15.31:0/1955918288 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8000cc0 con 0x7fecdc06a300 2023-10-06T05:16:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.580+0000 7fecd17fa700 1 -- 172.21.15.31:0/1955918288 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd8011900 con 0x7fecdc06a300 2023-10-06T05:16:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.580+0000 7fecd17fa700 1 -- 172.21.15.31:0/1955918288 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8025420 con 0x7fecdc06a300 2023-10-06T05:16:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.580+0000 7fece39ff700 1 -- 172.21.15.31:0/1955918288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd8009420 con 0x7fecdc06a300 2023-10-06T05:16:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.580+0000 7fece39ff700 1 -- 172.21.15.31:0/1955918288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc0c67b0 con 0x7fecdc06a300 2023-10-06T05:16:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.581+0000 7fece39ff700 1 -- 172.21.15.31:0/1955918288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc04c380 con 0x7fecdc06a300 2023-10-06T05:16:56.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.582+0000 7fecd17fa700 1 -- 172.21.15.31:0/1955918288 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fecd8023070 con 0x7fecdc06a300 2023-10-06T05:16:56.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.582+0000 7fecd17fa700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecc0066ea0 0x7fecc0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:56.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.583+0000 7fecd17fa700 1 -- 172.21.15.31:0/1955918288 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fecd8085770 con 0x7fecdc06a300 2023-10-06T05:16:56.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.583+0000 7fece0f9a700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecc0066ea0 0x7fecc0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:56.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.583+0000 7fece0f9a700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecc0066ea0 0x7fecc0069350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0c3930 tx=0x7feccc007f50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:56.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.588+0000 7fecd17fa700 1 -- 172.21.15.31:0/1955918288 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecd8052cd0 con 0x7fecdc06a300 2023-10-06T05:16:56.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.796+0000 7fece39ff700 1 -- 172.21.15.31:0/1955918288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecdc0ca1a0 con 0x7fecdc06a300 2023-10-06T05:16:56.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.796+0000 7fecd17fa700 1 -- 172.21.15.31:0/1955918288 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fecd802a0a0 con 0x7fecdc06a300 2023-10-06T05:16:56.798 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:16:56.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.798+0000 7fece39ff700 1 -- 172.21.15.31:0/1955918288 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecc0066ea0 msgr2=0x7fecc0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.798+0000 7fece39ff700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecc0066ea0 0x7fecc0069350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0c3930 tx=0x7feccc007f50 comp rx=0 tx=0).stop 2023-10-06T05:16:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.798+0000 7fece39ff700 1 -- 172.21.15.31:0/1955918288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc06a300 msgr2=0x7fecdc0c2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.798+0000 7fece39ff700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc06a300 0x7fecdc0c2c10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fecd8011fa0 tx=0x7fecd8017000 comp rx=0 tx=0).stop 2023-10-06T05:16:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.798+0000 7fece39ff700 1 -- 172.21.15.31:0/1955918288 shutdown_connections 2023-10-06T05:16:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.798+0000 7fece39ff700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecdc15ca40 0x7fecdc0c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.798+0000 7fece39ff700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc06a300 0x7fecdc0c2c10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.798+0000 7fece39ff700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecc0066ea0 0x7fecc0069350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.798+0000 7fece39ff700 1 --2- 172.21.15.31:0/1955918288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecdc0c8f40 0x7fecdc14d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.798+0000 7fece39ff700 1 -- 172.21.15.31:0/1955918288 >> 172.21.15.31:0/1955918288 conn(0x7fecdc0ce290 msgr2=0x7fecdc0b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.799+0000 7fece39ff700 1 -- 172.21.15.31:0/1955918288 shutdown_connections 2023-10-06T05:16:56.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:56.799+0000 7fece39ff700 1 -- 172.21.15.31:0/1955918288 wait complete. 2023-10-06T05:16:56.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:16:57.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.012+0000 7f257a3ff700 1 Processor -- start 2023-10-06T05:16:57.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.012+0000 7f257a3ff700 1 -- start start 2023-10-06T05:16:57.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.014+0000 7f257a3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f2574159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.014+0000 7f257a3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f25740cc0d0 0x7f2574068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.014+0000 7f257a3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2574069010 con 0x7f2574068bb0 2023-10-06T05:16:57.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.014+0000 7f257a3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25740c95b0 con 0x7f25740cc0d0 2023-10-06T05:16:57.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.014+0000 7f257a3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25740c9890 con 0x7f25741595a0 2023-10-06T05:16:57.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.014+0000 7f2573fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f2574159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.014+0000 7f25737fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f25740cc0d0 0x7f2574068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.014+0000 7f2573fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f2574159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:53616/0 (socket says 172.21.15.31:53616) 2023-10-06T05:16:57.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.014+0000 7f2573fff700 1 -- 172.21.15.31:0/1104077062 learned_addr learned my addr 172.21.15.31:0/1104077062 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:57.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.015+0000 7f2573fff700 1 -- 172.21.15.31:0/1104077062 >> v2:172.21.15.62:3300/0 conn(0x7f25740cc0d0 msgr2=0x7f2574068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:57.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.015+0000 7f2573fff700 1 --2- 172.21.15.31:0/1104077062 >> v2:172.21.15.62:3300/0 conn(0x7f25740cc0d0 0x7f2574068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.015+0000 7f2573fff700 1 -- 172.21.15.31:0/1104077062 >> v1:172.21.15.62:6789/0 conn(0x7f2574068bb0 legacy=0x7f25740c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.015+0000 7f2573fff700 1 -- 172.21.15.31:0/1104077062 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25740c9b70 con 0x7f25741595a0 2023-10-06T05:16:57.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.015+0000 7f2572ffd700 1 -- 172.21.15.31:0/1104077062 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323683925 0 0) 0x7f256c001490 con 0x7f2574068bb0 2023-10-06T05:16:57.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.015+0000 7f2573fff700 1 --2- 172.21.15.31:0/1104077062 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f2574159970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25640099b0 tx=0x7f2564009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae7bf6505ee324f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:57.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.015+0000 7f2572ffd700 1 -- 172.21.15.31:0/1104077062 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564009ea0 con 0x7f25741595a0 2023-10-06T05:16:57.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.015+0000 7f2572ffd700 1 -- 172.21.15.31:0/1104077062 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256400b860 con 0x7f25741595a0 2023-10-06T05:16:57.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.015+0000 7f2572ffd700 1 -- 172.21.15.31:0/1104077062 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256400bc00 con 0x7f25741595a0 2023-10-06T05:16:57.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.016+0000 7f257a3ff700 1 -- 172.21.15.31:0/1104077062 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 msgr2=0x7f2574159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.016+0000 7f257a3ff700 1 --2- 172.21.15.31:0/1104077062 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f2574159970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25640099b0 tx=0x7f2564009250 comp rx=0 tx=0).stop 2023-10-06T05:16:57.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.016+0000 7f257a3ff700 1 -- 172.21.15.31:0/1104077062 shutdown_connections 2023-10-06T05:16:57.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.016+0000 7f257a3ff700 1 --2- 172.21.15.31:0/1104077062 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f2574159970 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.016+0000 7f257a3ff700 1 --2- 172.21.15.31:0/1104077062 >> v2:172.21.15.62:3300/0 conn(0x7f25740cc0d0 0x7f2574068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.016+0000 7f257a3ff700 1 -- 172.21.15.31:0/1104077062 >> 172.21.15.31:0/1104077062 conn(0x7f257406b620 msgr2=0x7f257406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:57.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.016+0000 7f257a3ff700 1 -- 172.21.15.31:0/1104077062 shutdown_connections 2023-10-06T05:16:57.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.016+0000 7f257a3ff700 1 -- 172.21.15.31:0/1104077062 wait complete. 2023-10-06T05:16:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.018+0000 7f257a3ff700 1 Processor -- start 2023-10-06T05:16:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.018+0000 7f257a3ff700 1 -- start start 2023-10-06T05:16:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f257a3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2574068bb0 0x7f25740b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f257a3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25740cc0d0 0x7f257414e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f257a3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f25740b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f257a3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257414eb90 con 0x7f2574068bb0 2023-10-06T05:16:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f257a3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740b3ae0 con 0x7f25740cc0d0 2023-10-06T05:16:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f257a3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25740b1e20 con 0x7f25741595a0 2023-10-06T05:16:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f25737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25740cc0d0 0x7f257414e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f257899c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f25740b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f25737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25740cc0d0 0x7f257414e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:45006/0 (socket says 172.21.15.31:45006) 2023-10-06T05:16:57.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f25737fe700 1 -- 172.21.15.31:0/4018840460 learned_addr learned my addr 172.21.15.31:0/4018840460 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:57.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.019+0000 7f2573fff700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2574068bb0 0x7f25740b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f25737fe700 1 -- 172.21.15.31:0/4018840460 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 msgr2=0x7f25740b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f25737fe700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f25740b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f25737fe700 1 -- 172.21.15.31:0/4018840460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2574068bb0 msgr2=0x7f25740b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f25737fe700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2574068bb0 0x7f25740b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f25737fe700 1 -- 172.21.15.31:0/4018840460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25640094f0 con 0x7f25740cc0d0 2023-10-06T05:16:57.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f257899c700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f25740b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:57.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f25737fe700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25740cc0d0 0x7f257414e650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f256800ec80 tx=0x7f25680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:57.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f255bfff700 1 -- 172.21.15.31:0/4018840460 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256800c850 con 0x7f25740cc0d0 2023-10-06T05:16:57.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f255bfff700 1 -- 172.21.15.31:0/4018840460 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256800c9b0 con 0x7f25740cc0d0 2023-10-06T05:16:57.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f257a3ff700 1 -- 172.21.15.31:0/4018840460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25740b3800 con 0x7f25740cc0d0 2023-10-06T05:16:57.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f255bfff700 1 -- 172.21.15.31:0/4018840460 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256800cd50 con 0x7f25740cc0d0 2023-10-06T05:16:57.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.020+0000 7f257a3ff700 1 -- 172.21.15.31:0/4018840460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257415c580 con 0x7f25740cc0d0 2023-10-06T05:16:57.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.021+0000 7f257a3ff700 1 -- 172.21.15.31:0/4018840460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25740ce0c0 con 0x7f25740cc0d0 2023-10-06T05:16:57.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.022+0000 7f255bfff700 1 -- 172.21.15.31:0/4018840460 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f2568013680 con 0x7f25740cc0d0 2023-10-06T05:16:57.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.023+0000 7f255bfff700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2560066ea0 0x7f2560069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.023+0000 7f255bfff700 1 -- 172.21.15.31:0/4018840460 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f25680845c0 con 0x7f25740cc0d0 2023-10-06T05:16:57.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.023+0000 7f2573fff700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2560066ea0 0x7f2560069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.024+0000 7f2573fff700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2560066ea0 0x7f2560069350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25640099b0 tx=0x7f2564017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:57.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.027+0000 7f255bfff700 1 -- 172.21.15.31:0/4018840460 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2568051a60 con 0x7f25740cc0d0 2023-10-06T05:16:57.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.202+0000 7f255bfff700 1 -- 172.21.15.31:0/4018840460 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2568051e30 con 0x7f25740cc0d0 2023-10-06T05:16:57.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.285+0000 7f257a3ff700 1 -- 172.21.15.31:0/4018840460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f257404c380 con 0x7f25740cc0d0 2023-10-06T05:16:57.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.285+0000 7f255bfff700 1 -- 172.21.15.31:0/4018840460 <== mon.1 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7f2568017de0 con 0x7f25740cc0d0 2023-10-06T05:16:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:57.288 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1001195925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1001195925},{"type":"v1","addr":"172.21.15.62:6835","nonce":1001195925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1336279472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1336279472},{"type":"v1","addr":"172.21.15.31:6837","nonce":1336279472}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/577485679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":577485679},{"type":"v1","addr":"172.21.15.62:6834","nonce":577485679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4169737410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169737410},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169737410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-06T05:16:57.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 -- 172.21.15.31:0/4018840460 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2560066ea0 msgr2=0x7f2560069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2560066ea0 0x7f2560069350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25640099b0 tx=0x7f2564017000 comp rx=0 tx=0).stop 2023-10-06T05:16:57.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 -- 172.21.15.31:0/4018840460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25740cc0d0 msgr2=0x7f257414e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25740cc0d0 0x7f257414e650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f256800ec80 tx=0x7f25680091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 -- 172.21.15.31:0/4018840460 shutdown_connections 2023-10-06T05:16:57.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25741595a0 0x7f25740b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2560066ea0 0x7f2560069350 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2574068bb0 0x7f25740b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 --2- 172.21.15.31:0/4018840460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25740cc0d0 0x7f257414e650 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 -- 172.21.15.31:0/4018840460 >> 172.21.15.31:0/4018840460 conn(0x7f257406b620 msgr2=0x7f2574154a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:57.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 -- 172.21.15.31:0/4018840460 shutdown_connections 2023-10-06T05:16:57.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.288+0000 7f257a3ff700 1 -- 172.21.15.31:0/4018840460 wait complete. 2023-10-06T05:16:57.293 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2023-10-06T05:16:57.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:16:57.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.502+0000 7fcc3a9ff700 1 Processor -- start 2023-10-06T05:16:57.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.503+0000 7fcc3a9ff700 1 -- start start 2023-10-06T05:16:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.504+0000 7fcc3a9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc340cd440 0x7fcc34164250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.504+0000 7fcc3a9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc340ce090 con 0x7fcc340cf5a0 2023-10-06T05:16:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.504+0000 7fcc3a9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc340cb6f0 con 0x7fcc3415a930 2023-10-06T05:16:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.504+0000 7fcc3a9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340cb9d0 con 0x7fcc340cd440 2023-10-06T05:16:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.504+0000 7fcc337fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcc340cf5a0 0x7fcc340ccd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34012/0 (socket says 172.21.15.31:34012) 2023-10-06T05:16:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.504+0000 7fcc337fe700 1 -- 172.21.15.31:0/324483301 learned_addr learned my addr 172.21.15.31:0/324483301 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:16:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.504+0000 7fcc38f9c700 1 --2- 172.21.15.31:0/324483301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc340cd440 0x7fcc34164250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.504+0000 7fcc32ffd700 1 -- 172.21.15.31:0/324483301 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250475287 0 0) 0x7fcc340ce090 con 0x7fcc340cf5a0 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.505+0000 7fcc32ffd700 1 -- 172.21.15.31:0/324483301 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc1c003570 con 0x7fcc340cf5a0 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.505+0000 7fcc32ffd700 1 -- 172.21.15.31:0/324483301 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294869041 0 0) 0x7fcc340cb6f0 con 0x7fcc3415a930 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.505+0000 7fcc32ffd700 1 -- 172.21.15.31:0/324483301 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc340ce090 con 0x7fcc3415a930 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.505+0000 7fcc38f9c700 1 -- 172.21.15.31:0/324483301 >> v1:172.21.15.62:6789/0 conn(0x7fcc3415a930 legacy=0x7fcc340cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.505+0000 7fcc38f9c700 1 -- 172.21.15.31:0/324483301 >> v1:172.21.15.31:6789/0 conn(0x7fcc340cf5a0 legacy=0x7fcc340ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.505+0000 7fcc38f9c700 1 -- 172.21.15.31:0/324483301 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc340cbcb0 con 0x7fcc340cd440 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.505+0000 7fcc38f9c700 1 --2- 172.21.15.31:0/324483301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc340cd440 0x7fcc34164250 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009e90 tx=0x7fcc2c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9f255d8a1fd5598 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.505+0000 7fcc32ffd700 1 -- 172.21.15.31:0/324483301 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00e070 con 0x7fcc340cd440 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.505+0000 7fcc32ffd700 1 -- 172.21.15.31:0/324483301 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2c00bc10 con 0x7fcc340cd440 2023-10-06T05:16:57.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.505+0000 7fcc32ffd700 1 -- 172.21.15.31:0/324483301 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c012430 con 0x7fcc340cd440 2023-10-06T05:16:57.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.506+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/324483301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc340cd440 msgr2=0x7fcc34164250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.506+0000 7fcc3a9ff700 1 --2- 172.21.15.31:0/324483301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc340cd440 0x7fcc34164250 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009e90 tx=0x7fcc2c009780 comp rx=0 tx=0).stop 2023-10-06T05:16:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.506+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/324483301 shutdown_connections 2023-10-06T05:16:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.506+0000 7fcc3a9ff700 1 --2- 172.21.15.31:0/324483301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc340cd440 0x7fcc34164250 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.506+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/324483301 >> 172.21.15.31:0/324483301 conn(0x7fcc34099720 msgr2=0x7fcc3406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.506+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/324483301 shutdown_connections 2023-10-06T05:16:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.506+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/324483301 wait complete. 2023-10-06T05:16:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.507+0000 7fcc3a9ff700 1 Processor -- start 2023-10-06T05:16:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.508+0000 7fcc3a9ff700 1 -- start start 2023-10-06T05:16:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.508+0000 7fcc3a9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc340cd440 0x7fcc34152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.508+0000 7fcc3a9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc340cf5a0 0x7fcc340c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.508+0000 7fcc3a9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc3415a930 0x7fcc340d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.508+0000 7fcc3a9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340c6bf0 con 0x7fcc340cd440 2023-10-06T05:16:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.508+0000 7fcc3a9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340c5fe0 con 0x7fcc340cf5a0 2023-10-06T05:16:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.508+0000 7fcc3a9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340c62c0 con 0x7fcc3415a930 2023-10-06T05:16:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc340cd440 0x7fcc34152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc340cd440 0x7fcc34152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48044/0 (socket says 172.21.15.31:48044) 2023-10-06T05:16:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc33fff700 1 -- 172.21.15.31:0/140115212 learned_addr learned my addr 172.21.15.31:0/140115212 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc337fe700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc340cf5a0 0x7fcc340c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc38f9c700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc3415a930 0x7fcc340d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc33fff700 1 -- 172.21.15.31:0/140115212 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc3415a930 msgr2=0x7fcc340d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc33fff700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc3415a930 0x7fcc340d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc33fff700 1 -- 172.21.15.31:0/140115212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc340cf5a0 msgr2=0x7fcc340c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc33fff700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc340cf5a0 0x7fcc340c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc33fff700 1 -- 172.21.15.31:0/140115212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c009a20 con 0x7fcc340cd440 2023-10-06T05:16:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.509+0000 7fcc33fff700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc340cd440 0x7fcc34152520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400c560 tx=0x7fcc24007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.510+0000 7fcc1bfff700 1 -- 172.21.15.31:0/140115212 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24010510 con 0x7fcc340cd440 2023-10-06T05:16:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.510+0000 7fcc1bfff700 1 -- 172.21.15.31:0/140115212 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24010670 con 0x7fcc340cd440 2023-10-06T05:16:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.510+0000 7fcc1bfff700 1 -- 172.21.15.31:0/140115212 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24010a30 con 0x7fcc340cd440 2023-10-06T05:16:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.510+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/140115212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc340c65a0 con 0x7fcc340cd440 2023-10-06T05:16:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.510+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/140115212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc34152a60 con 0x7fcc340cd440 2023-10-06T05:16:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.511+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/140115212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3404c380 con 0x7fcc340cd440 2023-10-06T05:16:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.512+0000 7fcc1bfff700 1 -- 172.21.15.31:0/140115212 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc24002050 con 0x7fcc340cd440 2023-10-06T05:16:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.512+0000 7fcc1bfff700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc20066f30 0x7fcc200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.513+0000 7fcc1bfff700 1 -- 172.21.15.31:0/140115212 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fcc24083d10 con 0x7fcc340cd440 2023-10-06T05:16:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.513+0000 7fcc337fe700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc20066f30 0x7fcc200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.513+0000 7fcc337fe700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc20066f30 0x7fcc200693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcc340d1140 tx=0x7fcc28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:57.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.518+0000 7fcc1bfff700 1 -- 172.21.15.31:0/140115212 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc24089020 con 0x7fcc340cd440 2023-10-06T05:16:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.725+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/140115212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc34159ce0 con 0x7fcc340cd440 2023-10-06T05:16:57.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.726+0000 7fcc1bfff700 1 -- 172.21.15.31:0/140115212 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13696 (secure 0 0 0) 0x7fcc240510d0 con 0x7fcc340cd440 2023-10-06T05:16:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:16:56.387831+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:16:57.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/140115212 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc20066f30 msgr2=0x7fcc200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc20066f30 0x7fcc200693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcc340d1140 tx=0x7fcc28009040 comp rx=0 tx=0).stop 2023-10-06T05:16:57.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/140115212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc340cd440 msgr2=0x7fcc34152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc340cd440 0x7fcc34152520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400c560 tx=0x7fcc24007e00 comp rx=0 tx=0).stop 2023-10-06T05:16:57.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/140115212 shutdown_connections 2023-10-06T05:16:57.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc3415a930 0x7fcc340d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc20066f30 0x7fcc200693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc340cd440 0x7fcc34152520 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 --2- 172.21.15.31:0/140115212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc340cf5a0 0x7fcc340c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/140115212 >> 172.21.15.31:0/140115212 conn(0x7fcc34099720 msgr2=0x7fcc34153970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:57.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/140115212 shutdown_connections 2023-10-06T05:16:57.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.728+0000 7fcc3a9ff700 1 -- 172.21.15.31:0/140115212 wait complete. 2023-10-06T05:16:57.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:16:57.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:16:57.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.944+0000 7fd4499ff700 1 Processor -- start 2023-10-06T05:16:57.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.945+0000 7fd4499ff700 1 -- start start 2023-10-06T05:16:57.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.946+0000 7fd4499ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 0x7fd4440c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.946+0000 7fd4499ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd4440aef00 0x7fd4440af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.946+0000 7fd4499ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd444068250 con 0x7fd4440afcf0 2023-10-06T05:16:57.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.946+0000 7fd4499ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44406ac80 con 0x7fd4440aef00 2023-10-06T05:16:57.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.946+0000 7fd4499ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd444067f20 con 0x7fd4440c7190 2023-10-06T05:16:57.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.946+0000 7fd4437fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd4440afcf0 0x7fd4440686e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34024/0 (socket says 172.21.15.31:34024) 2023-10-06T05:16:57.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.946+0000 7fd4437fe700 1 -- 172.21.15.31:0/4039456664 learned_addr learned my addr 172.21.15.31:0/4039456664 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:16:57.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.946+0000 7fd4427fc700 1 --2- 172.21.15.31:0/4039456664 >> v2:172.21.15.62:3300/0 conn(0x7fd4440aef00 0x7fd4440af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.946+0000 7fd442ffd700 1 --2- 172.21.15.31:0/4039456664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 0x7fd4440c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.946+0000 7fd441ffb700 1 -- 172.21.15.31:0/4039456664 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769423163 0 0) 0x7fd444068250 con 0x7fd4440afcf0 2023-10-06T05:16:57.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.947+0000 7fd441ffb700 1 -- 172.21.15.31:0/4039456664 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd42c003570 con 0x7fd4440afcf0 2023-10-06T05:16:57.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.947+0000 7fd442ffd700 1 -- 172.21.15.31:0/4039456664 >> v2:172.21.15.62:3300/0 conn(0x7fd4440aef00 msgr2=0x7fd4440af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:57.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.947+0000 7fd442ffd700 1 --2- 172.21.15.31:0/4039456664 >> v2:172.21.15.62:3300/0 conn(0x7fd4440aef00 0x7fd4440af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.947+0000 7fd442ffd700 1 -- 172.21.15.31:0/4039456664 >> v1:172.21.15.31:6789/0 conn(0x7fd4440afcf0 legacy=0x7fd4440686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.947+0000 7fd442ffd700 1 -- 172.21.15.31:0/4039456664 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44406aed0 con 0x7fd4440c7190 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.947+0000 7fd442ffd700 1 --2- 172.21.15.31:0/4039456664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 0x7fd4440c9d40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd4340098e0 tx=0x7fd4340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55ea30f9acc374fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.947+0000 7fd441ffb700 1 -- 172.21.15.31:0/4039456664 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434011630 con 0x7fd4440c7190 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.947+0000 7fd441ffb700 1 -- 172.21.15.31:0/4039456664 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434011790 con 0x7fd4440c7190 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.947+0000 7fd441ffb700 1 -- 172.21.15.31:0/4039456664 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434011b30 con 0x7fd4440c7190 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.948+0000 7fd4499ff700 1 -- 172.21.15.31:0/4039456664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 msgr2=0x7fd4440c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.948+0000 7fd4499ff700 1 --2- 172.21.15.31:0/4039456664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 0x7fd4440c9d40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd4340098e0 tx=0x7fd4340091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.948+0000 7fd4499ff700 1 -- 172.21.15.31:0/4039456664 shutdown_connections 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.948+0000 7fd4499ff700 1 --2- 172.21.15.31:0/4039456664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 0x7fd4440c9d40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.948+0000 7fd4499ff700 1 --2- 172.21.15.31:0/4039456664 >> v2:172.21.15.62:3300/0 conn(0x7fd4440aef00 0x7fd4440af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.948+0000 7fd4499ff700 1 -- 172.21.15.31:0/4039456664 >> 172.21.15.31:0/4039456664 conn(0x7fd4440cd3d0 msgr2=0x7fd4440c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:57.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.948+0000 7fd4499ff700 1 -- 172.21.15.31:0/4039456664 shutdown_connections 2023-10-06T05:16:57.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.948+0000 7fd4499ff700 1 -- 172.21.15.31:0/4039456664 wait complete. 2023-10-06T05:16:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.949+0000 7fd4499ff700 1 Processor -- start 2023-10-06T05:16:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.950+0000 7fd4499ff700 1 -- start start 2023-10-06T05:16:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.950+0000 7fd4499ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4440aef00 0x7fd4440c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.950+0000 7fd4499ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4440afcf0 0x7fd44415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4499ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 0x7fd44415fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4499ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44415b090 con 0x7fd4440afcf0 2023-10-06T05:16:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4499ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4440c1e30 con 0x7fd4440aef00 2023-10-06T05:16:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4499ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4440c1870 con 0x7fd4440c7190 2023-10-06T05:16:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4427fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4440afcf0 0x7fd44415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4427fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4440afcf0 0x7fd44415f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48058/0 (socket says 172.21.15.31:48058) 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4427fc700 1 -- 172.21.15.31:0/3828824908 learned_addr learned my addr 172.21.15.31:0/3828824908 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd442ffd700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4440aef00 0x7fd4440c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4437fe700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 0x7fd44415fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4427fc700 1 -- 172.21.15.31:0/3828824908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 msgr2=0x7fd44415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4427fc700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 0x7fd44415fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4427fc700 1 -- 172.21.15.31:0/3828824908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4440aef00 msgr2=0x7fd4440c0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4427fc700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4440aef00 0x7fd4440c0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.951+0000 7fd4427fc700 1 -- 172.21.15.31:0/3828824908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd434009420 con 0x7fd4440afcf0 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.952+0000 7fd4427fc700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4440afcf0 0x7fd44415f6b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd43800e930 tx=0x7fd4380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.952+0000 7fd42affd700 1 -- 172.21.15.31:0/3828824908 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438013680 con 0x7fd4440afcf0 2023-10-06T05:16:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.952+0000 7fd42affd700 1 -- 172.21.15.31:0/3828824908 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4380137e0 con 0x7fd4440afcf0 2023-10-06T05:16:57.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.952+0000 7fd4499ff700 1 -- 172.21.15.31:0/3828824908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4440c1b50 con 0x7fd4440afcf0 2023-10-06T05:16:57.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.952+0000 7fd42affd700 1 -- 172.21.15.31:0/3828824908 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438013b80 con 0x7fd4440afcf0 2023-10-06T05:16:57.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.952+0000 7fd4499ff700 1 -- 172.21.15.31:0/3828824908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4440c2c20 con 0x7fd4440afcf0 2023-10-06T05:16:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.953+0000 7fd4499ff700 1 -- 172.21.15.31:0/3828824908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44415ab80 con 0x7fd4440afcf0 2023-10-06T05:16:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.954+0000 7fd42affd700 1 -- 172.21.15.31:0/3828824908 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd43800c3e0 con 0x7fd4440afcf0 2023-10-06T05:16:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.954+0000 7fd42affd700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd430066f30 0x7fd4300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.955+0000 7fd42affd700 1 -- 172.21.15.31:0/3828824908 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd4380844e0 con 0x7fd4440afcf0 2023-10-06T05:16:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.955+0000 7fd442ffd700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd430066f30 0x7fd4300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.955+0000 7fd442ffd700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd430066f30 0x7fd4300693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4340091e0 tx=0x7fd434017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:57.959+0000 7fd42affd700 1 -- 172.21.15.31:0/3828824908 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd44415ab80 con 0x7fd4440afcf0 2023-10-06T05:16:58.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.161+0000 7fd4499ff700 1 -- 172.21.15.31:0/3828824908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd44415ab80 con 0x7fd4440afcf0 2023-10-06T05:16:58.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.161+0000 7fd42affd700 1 -- 172.21.15.31:0/3828824908 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd44415ab80 con 0x7fd4440afcf0 2023-10-06T05:16:58.163 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:16:58.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.163+0000 7fd4499ff700 1 -- 172.21.15.31:0/3828824908 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd430066f30 msgr2=0x7fd4300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.163+0000 7fd4499ff700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd430066f30 0x7fd4300693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4340091e0 tx=0x7fd434017000 comp rx=0 tx=0).stop 2023-10-06T05:16:58.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.163+0000 7fd4499ff700 1 -- 172.21.15.31:0/3828824908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4440afcf0 msgr2=0x7fd44415f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.163+0000 7fd4499ff700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4440afcf0 0x7fd44415f6b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd43800e930 tx=0x7fd4380091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.164+0000 7fd4499ff700 1 -- 172.21.15.31:0/3828824908 shutdown_connections 2023-10-06T05:16:58.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.164+0000 7fd4499ff700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4440c7190 0x7fd44415fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.164+0000 7fd4499ff700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4440aef00 0x7fd4440c0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.164+0000 7fd4499ff700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd430066f30 0x7fd4300693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.164+0000 7fd4499ff700 1 --2- 172.21.15.31:0/3828824908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4440afcf0 0x7fd44415f6b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.164+0000 7fd4499ff700 1 -- 172.21.15.31:0/3828824908 >> 172.21.15.31:0/3828824908 conn(0x7fd4440cd3d0 msgr2=0x7fd444159430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:58.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.164+0000 7fd4499ff700 1 -- 172.21.15.31:0/3828824908 shutdown_connections 2023-10-06T05:16:58.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.164+0000 7fd4499ff700 1 -- 172.21.15.31:0/3828824908 wait complete. 2023-10-06T05:16:58.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:16:58.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.379+0000 7f45915ff700 1 Processor -- start 2023-10-06T05:16:58.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.379+0000 7f45915ff700 1 -- start start 2023-10-06T05:16:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.380+0000 7f45915ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f458c0c7190 0x7f458c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.380+0000 7f45915ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f458c0c7560 0x7f458c0686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.380+0000 7f45915ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c068250 con 0x7f458c0aef00 2023-10-06T05:16:58.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.380+0000 7f45915ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c06ac80 con 0x7f458c0c7560 2023-10-06T05:16:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.380+0000 7f45915ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c067f20 con 0x7f458c0c7190 2023-10-06T05:16:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f458b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f458c0c7560 0x7f458c0686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f458b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f458c0c7560 0x7f458c0686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48060/0 (socket says 172.21.15.31:48060) 2023-10-06T05:16:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f458a7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f458c0aef00 0x7f458c0af7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34034/0 (socket says 172.21.15.31:34034) 2023-10-06T05:16:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f458b7fe700 1 -- 172.21.15.31:0/1344436454 learned_addr learned my addr 172.21.15.31:0/1344436454 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f458affd700 1 --2- 172.21.15.31:0/1344436454 >> v2:172.21.15.62:3300/0 conn(0x7f458c0c7190 0x7f458c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f4589ffb700 1 -- 172.21.15.31:0/1344436454 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376005877 0 0) 0x7f458c068250 con 0x7f458c0aef00 2023-10-06T05:16:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f458b7fe700 1 -- 172.21.15.31:0/1344436454 >> v2:172.21.15.62:3300/0 conn(0x7f458c0c7190 msgr2=0x7f458c0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f458b7fe700 1 --2- 172.21.15.31:0/1344436454 >> v2:172.21.15.62:3300/0 conn(0x7f458c0c7190 0x7f458c0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f458b7fe700 1 -- 172.21.15.31:0/1344436454 >> v1:172.21.15.31:6789/0 conn(0x7f458c0aef00 legacy=0x7f458c0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f458b7fe700 1 -- 172.21.15.31:0/1344436454 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c06aed0 con 0x7f458c0c7560 2023-10-06T05:16:58.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.381+0000 7f458b7fe700 1 --2- 172.21.15.31:0/1344436454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 0x7f458c0686e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f458c06f4e0 tx=0x7f4584009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed23d70de38f3b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:58.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f4589ffb700 1 -- 172.21.15.31:0/1344436454 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400e070 con 0x7f458c0c7560 2023-10-06T05:16:58.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f4589ffb700 1 -- 172.21.15.31:0/1344436454 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458400ba20 con 0x7f458c0c7560 2023-10-06T05:16:58.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f4589ffb700 1 -- 172.21.15.31:0/1344436454 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45840092e0 con 0x7f458c0c7560 2023-10-06T05:16:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f45915ff700 1 -- 172.21.15.31:0/1344436454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 msgr2=0x7f458c0686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f45915ff700 1 --2- 172.21.15.31:0/1344436454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 0x7f458c0686e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f458c06f4e0 tx=0x7f4584009840 comp rx=0 tx=0).stop 2023-10-06T05:16:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f45915ff700 1 -- 172.21.15.31:0/1344436454 shutdown_connections 2023-10-06T05:16:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f45915ff700 1 --2- 172.21.15.31:0/1344436454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 0x7f458c0686e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f45915ff700 1 --2- 172.21.15.31:0/1344436454 >> v2:172.21.15.62:3300/0 conn(0x7f458c0c7190 0x7f458c0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f45915ff700 1 -- 172.21.15.31:0/1344436454 >> 172.21.15.31:0/1344436454 conn(0x7f458c0cd3d0 msgr2=0x7f458c0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f45915ff700 1 -- 172.21.15.31:0/1344436454 shutdown_connections 2023-10-06T05:16:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.382+0000 7f45915ff700 1 -- 172.21.15.31:0/1344436454 wait complete. 2023-10-06T05:16:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.384+0000 7f45915ff700 1 Processor -- start 2023-10-06T05:16:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.384+0000 7f45915ff700 1 -- start start 2023-10-06T05:16:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f45915ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c0aef00 0x7f458c159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f45915ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f458c0c7190 0x7f458c0c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f45915ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 0x7f458c150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f45915ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c15a900 con 0x7f458c0c7560 2023-10-06T05:16:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f45915ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c151ec0 con 0x7f458c0c7190 2023-10-06T05:16:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f45915ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1521a0 con 0x7f458c0aef00 2023-10-06T05:16:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f458b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 0x7f458c150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f458b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 0x7f458c150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48068/0 (socket says 172.21.15.31:48068) 2023-10-06T05:16:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f458b7fe700 1 -- 172.21.15.31:0/1039827317 learned_addr learned my addr 172.21.15.31:0/1039827317 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f458a7fc700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f458c0c7190 0x7f458c0c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.385+0000 7f458affd700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c0aef00 0x7f458c159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.386+0000 7f458b7fe700 1 -- 172.21.15.31:0/1039827317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c0aef00 msgr2=0x7f458c159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.386+0000 7f458b7fe700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c0aef00 0x7f458c159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.386+0000 7f458b7fe700 1 -- 172.21.15.31:0/1039827317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f458c0c7190 msgr2=0x7f458c0c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.386+0000 7f458b7fe700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f458c0c7190 0x7f458c0c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.386+0000 7f458b7fe700 1 -- 172.21.15.31:0/1039827317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4584009ae0 con 0x7f458c0c7560 2023-10-06T05:16:58.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.386+0000 7f458b7fe700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 0x7f458c150f10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4584009880 tx=0x7f4584018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.386+0000 7f4572ffd700 1 -- 172.21.15.31:0/1039827317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584008a80 con 0x7f458c0c7560 2023-10-06T05:16:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.386+0000 7f4572ffd700 1 -- 172.21.15.31:0/1039827317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4584008c00 con 0x7f458c0c7560 2023-10-06T05:16:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.386+0000 7f4572ffd700 1 -- 172.21.15.31:0/1039827317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584026460 con 0x7f458c0c7560 2023-10-06T05:16:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.386+0000 7f45915ff700 1 -- 172.21.15.31:0/1039827317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c152480 con 0x7f458c0c7560 2023-10-06T05:16:58.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.387+0000 7f45915ff700 1 -- 172.21.15.31:0/1039827317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c15a4b0 con 0x7f458c0c7560 2023-10-06T05:16:58.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.387+0000 7f45915ff700 1 -- 172.21.15.31:0/1039827317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c04c380 con 0x7f458c0c7560 2023-10-06T05:16:58.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.388+0000 7f4572ffd700 1 -- 172.21.15.31:0/1039827317 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4584003040 con 0x7f458c0c7560 2023-10-06T05:16:58.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.389+0000 7f4572ffd700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4578066f30 0x7f45780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.390+0000 7f4572ffd700 1 -- 172.21.15.31:0/1039827317 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4584086270 con 0x7f458c0c7560 2023-10-06T05:16:58.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.390+0000 7f458affd700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4578066f30 0x7f45780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.390+0000 7f458affd700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4578066f30 0x7f45780693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f457c009db0 tx=0x7f457c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.394+0000 7f4572ffd700 1 -- 172.21.15.31:0/1039827317 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4584053630 con 0x7f458c0c7560 2023-10-06T05:16:58.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.595+0000 7f45915ff700 1 -- 172.21.15.31:0/1039827317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f458c15a750 con 0x7f458c0c7560 2023-10-06T05:16:58.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.596+0000 7f4572ffd700 1 -- 172.21.15.31:0/1039827317 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4584009940 con 0x7f458c0c7560 2023-10-06T05:16:58.598 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:16:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 -- 172.21.15.31:0/1039827317 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4578066f30 msgr2=0x7f45780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4578066f30 0x7f45780693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f457c009db0 tx=0x7f457c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 -- 172.21.15.31:0/1039827317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 msgr2=0x7f458c150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 0x7f458c150f10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4584009880 tx=0x7f4584018000 comp rx=0 tx=0).stop 2023-10-06T05:16:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 -- 172.21.15.31:0/1039827317 shutdown_connections 2023-10-06T05:16:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4578066f30 0x7f45780693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0c7560 0x7f458c150f10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c0aef00 0x7f458c159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 --2- 172.21.15.31:0/1039827317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f458c0c7190 0x7f458c0c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 -- 172.21.15.31:0/1039827317 >> 172.21.15.31:0/1039827317 conn(0x7f458c0cd3d0 msgr2=0x7f458c0686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:58.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 -- 172.21.15.31:0/1039827317 shutdown_connections 2023-10-06T05:16:58.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.598+0000 7f45915ff700 1 -- 172.21.15.31:0/1039827317 wait complete. 2023-10-06T05:16:58.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:16:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.809+0000 7fdbb03ff700 1 Processor -- start 2023-10-06T05:16:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.810+0000 7fdbb03ff700 1 -- start start 2023-10-06T05:16:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.811+0000 7fdbb03ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdba8159500 0x7fdba80b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.811+0000 7fdbb03ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdba80cd510 0x7fdba80aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.811+0000 7fdbb03ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba80c3f40 con 0x7fdba815d2d0 2023-10-06T05:16:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.811+0000 7fdbb03ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba80b2860 con 0x7fdba80cd510 2023-10-06T05:16:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbb03ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba815d7d0 con 0x7fdba8159500 2023-10-06T05:16:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbad99a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdba815d2d0 0x7fdba80cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34038/0 (socket says 172.21.15.31:34038) 2023-10-06T05:16:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbad99a700 1 -- 172.21.15.31:0/2455007438 learned_addr learned my addr 172.21.15.31:0/2455007438 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:16:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbae19b700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdba8159500 0x7fdba80b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbae99c700 1 --2- 172.21.15.31:0/2455007438 >> v2:172.21.15.62:3300/0 conn(0x7fdba80cd510 0x7fdba80aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642089127 0 0) 0x7fdba80c3f40 con 0x7fdba815d2d0 2023-10-06T05:16:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb94003570 con 0x7fdba815d2d0 2023-10-06T05:16:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1887062348 0 0) 0x7fdb94003570 con 0x7fdba815d2d0 2023-10-06T05:16:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdba8159500 msgr2=0x7fdba80b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbad199700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdba8159500 0x7fdba80b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 >> v2:172.21.15.62:3300/0 conn(0x7fdba80cd510 msgr2=0x7fdba80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.812+0000 7fdbad199700 1 --2- 172.21.15.31:0/2455007438 >> v2:172.21.15.62:3300/0 conn(0x7fdba80cd510 0x7fdba80aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba80cb000 con 0x7fdba815d2d0 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fdb980015f0 con 0x7fdba815d2d0 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 >> v1:172.21.15.31:6789/0 conn(0x7fdba815d2d0 legacy=0x7fdba80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad199700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb94003a60 0x7fdb94003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad199700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb94004810 0x7fdb94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad199700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb940051c0 0x7fdb9400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b2860 con 0x7fdb94004810 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba815d7d0 con 0x7fdb94003a60 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9400bfa0 con 0x7fdb940051c0 2023-10-06T05:16:58.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbae19b700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdba8159500 0x7fdba80b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad99a700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb94004810 0x7fdb94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbae99c700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb940051c0 0x7fdb9400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbae19b700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb94003a60 0x7fdb94003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad99a700 1 -- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb940051c0 msgr2=0x7fdb9400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad99a700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb940051c0 0x7fdb9400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad99a700 1 -- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb94003a60 msgr2=0x7fdb94003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad99a700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb94003a60 0x7fdb94003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.813+0000 7fdbad99a700 1 -- 172.21.15.31:0/2455007438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94003570 con 0x7fdb94004810 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.814+0000 7fdbad99a700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb94004810 0x7fdb94004c80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c390 tx=0x7fdb98007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc83a0d05959fc54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.814+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800cc60 con 0x7fdb94004810 2023-10-06T05:16:58.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.814+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9800cdc0 con 0x7fdb94004810 2023-10-06T05:16:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.814+0000 7fdbad199700 1 -- 172.21.15.31:0/2455007438 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98007320 con 0x7fdb94004810 2023-10-06T05:16:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.814+0000 7fdbb03ff700 1 -- 172.21.15.31:0/2455007438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb94004810 msgr2=0x7fdb94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.814+0000 7fdbb03ff700 1 --2- 172.21.15.31:0/2455007438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb94004810 0x7fdb94004c80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c390 tx=0x7fdb98007920 comp rx=0 tx=0).stop 2023-10-06T05:16:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.814+0000 7fdbae99c700 1 -- 172.21.15.31:0/2455007438 reap_dead start 2023-10-06T05:16:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.814+0000 7fdbb03ff700 1 -- 172.21.15.31:0/2455007438 shutdown_connections 2023-10-06T05:16:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.814+0000 7fdbb03ff700 1 -- 172.21.15.31:0/2455007438 >> 172.21.15.31:0/2455007438 conn(0x7fdba806ec90 msgr2=0x7fdba806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.815+0000 7fdbb03ff700 1 -- 172.21.15.31:0/2455007438 shutdown_connections 2023-10-06T05:16:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.815+0000 7fdbb03ff700 1 -- 172.21.15.31:0/2455007438 wait complete. 2023-10-06T05:16:58.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.816+0000 7fdbb03ff700 1 Processor -- start 2023-10-06T05:16:58.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.816+0000 7fdbb03ff700 1 -- start start 2023-10-06T05:16:58.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbb03ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb94004810 0x7fdba81559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbb03ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba80c8350 0x7fdba80b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbb03ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba816a380 0x7fdba80c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbb03ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b3290 con 0x7fdba80c8350 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbb03ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b3e10 con 0x7fdba816a380 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbb03ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b34e0 con 0x7fdb94004810 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbad99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba80c8350 0x7fdba80b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbad99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba80c8350 0x7fdba80b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48076/0 (socket says 172.21.15.31:48076) 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbad99a700 1 -- 172.21.15.31:0/4062944161 learned_addr learned my addr 172.21.15.31:0/4062944161 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbae99c700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba816a380 0x7fdba80c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.817+0000 7fdbae19b700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb94004810 0x7fdba81559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdbad99a700 1 -- 172.21.15.31:0/4062944161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb94004810 msgr2=0x7fdba81559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdbad99a700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb94004810 0x7fdba81559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdbad99a700 1 -- 172.21.15.31:0/4062944161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba816a380 msgr2=0x7fdba80c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdbad99a700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba816a380 0x7fdba80c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdbad99a700 1 -- 172.21.15.31:0/4062944161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba80b37c0 con 0x7fdba80c8350 2023-10-06T05:16:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdbae99c700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba816a380 0x7fdba80c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdbad99a700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba80c8350 0x7fdba80b5960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdb940046f0 tx=0x7fdb98015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdb9e7fc700 1 -- 172.21.15.31:0/4062944161 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800cc60 con 0x7fdba80c8350 2023-10-06T05:16:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdb9e7fc700 1 -- 172.21.15.31:0/4062944161 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98007690 con 0x7fdba80c8350 2023-10-06T05:16:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdbb03ff700 1 -- 172.21.15.31:0/4062944161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba80b3aa0 con 0x7fdba80c8350 2023-10-06T05:16:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdb9e7fc700 1 -- 172.21.15.31:0/4062944161 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98005c50 con 0x7fdba80c8350 2023-10-06T05:16:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.818+0000 7fdbb03ff700 1 -- 172.21.15.31:0/4062944161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba80b5ea0 con 0x7fdba80c8350 2023-10-06T05:16:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.819+0000 7fdbb03ff700 1 -- 172.21.15.31:0/4062944161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba80b3fa0 con 0x7fdba80c8350 2023-10-06T05:16:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.820+0000 7fdb9e7fc700 1 -- 172.21.15.31:0/4062944161 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb98019030 con 0x7fdba80c8350 2023-10-06T05:16:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.821+0000 7fdb9e7fc700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb8c066f30 0x7fdb8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.821+0000 7fdb9e7fc700 1 -- 172.21.15.31:0/4062944161 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdb98085f40 con 0x7fdba80c8350 2023-10-06T05:16:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.821+0000 7fdbae19b700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb8c066f30 0x7fdb8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.822+0000 7fdbae19b700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb8c066f30 0x7fdb8c0693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdba4006d80 tx=0x7fdba4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:58.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:58.826+0000 7fdb9e7fc700 1 -- 172.21.15.31:0/4062944161 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb98053300 con 0x7fdba80c8350 2023-10-06T05:16:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.029+0000 7fdbb03ff700 1 -- 172.21.15.31:0/4062944161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdba80b3290 con 0x7fdba80c8350 2023-10-06T05:16:59.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.029+0000 7fdb9e7fc700 1 -- 172.21.15.31:0/4062944161 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7fdb98002d40 con 0x7fdba80c8350 2023-10-06T05:16:59.031 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:16:59.031 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:16:59.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:16:59.032 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:16:59.032 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:16:59.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:16:59.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:16:59.032 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:16:59.032 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:16:59.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:16:59.032 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:16:59.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 -- 172.21.15.31:0/4062944161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb8c066f30 msgr2=0x7fdb8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:59.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb8c066f30 0x7fdb8c0693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdba4006d80 tx=0x7fdba4008040 comp rx=0 tx=0).stop 2023-10-06T05:16:59.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 -- 172.21.15.31:0/4062944161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba80c8350 msgr2=0x7fdba80b5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:59.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba80c8350 0x7fdba80b5960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdb940046f0 tx=0x7fdb98015000 comp rx=0 tx=0).stop 2023-10-06T05:16:59.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 -- 172.21.15.31:0/4062944161 shutdown_connections 2023-10-06T05:16:59.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba816a380 0x7fdba80c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:59.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb94004810 0x7fdba81559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:59.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb8c066f30 0x7fdb8c0693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:59.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 --2- 172.21.15.31:0/4062944161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdba80c8350 0x7fdba80b5960 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:59.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 -- 172.21.15.31:0/4062944161 >> 172.21.15.31:0/4062944161 conn(0x7fdba806ec90 msgr2=0x7fdba80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:59.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 -- 172.21.15.31:0/4062944161 shutdown_connections 2023-10-06T05:16:59.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.032+0000 7fdbb03ff700 1 -- 172.21.15.31:0/4062944161 wait complete. 2023-10-06T05:16:59.051 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:16:59.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:16:59.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.248+0000 7f52589ff700 1 Processor -- start 2023-10-06T05:16:59.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.248+0000 7f52589ff700 1 -- start start 2023-10-06T05:16:59.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.249+0000 7f52589ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f52541550a0 0x7f52540ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:59.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.249+0000 7f52589ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525406ee10 0x7f525406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:59.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.249+0000 7f52589ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525406f640 con 0x7f5254154cd0 2023-10-06T05:16:59.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.249+0000 7f52589ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525406e610 con 0x7f52541550a0 2023-10-06T05:16:59.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.249+0000 7f52589ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52540adf60 con 0x7f525406ee10 2023-10-06T05:16:59.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.250+0000 7f5252d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525406ee10 0x7f525406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.250+0000 7f5251d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f52541550a0 0x7f52540ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.250+0000 7f5252d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525406ee10 0x7f525406d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:53720/0 (socket says 172.21.15.31:53720) 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.250+0000 7f5252d9d700 1 -- 172.21.15.31:0/451676911 learned_addr learned my addr 172.21.15.31:0/451676911 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.250+0000 7f5252d9d700 1 -- 172.21.15.31:0/451676911 >> v2:172.21.15.62:3300/0 conn(0x7f52541550a0 msgr2=0x7f52540ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.250+0000 7f5252d9d700 1 --2- 172.21.15.31:0/451676911 >> v2:172.21.15.62:3300/0 conn(0x7f52541550a0 0x7f52540ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.250+0000 7f5252d9d700 1 -- 172.21.15.31:0/451676911 >> v1:172.21.15.62:6789/0 conn(0x7f5254154cd0 legacy=0x7f52541583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.250+0000 7f5252d9d700 1 -- 172.21.15.31:0/451676911 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525406f360 con 0x7f525406ee10 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.250+0000 7f5252d9d700 1 --2- 172.21.15.31:0/451676911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525406ee10 0x7f525406d7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f524c009f20 tx=0x7f524c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8741f5a659c74a44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.251+0000 7f525159a700 1 -- 172.21.15.31:0/451676911 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00e070 con 0x7f525406ee10 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.251+0000 7f525159a700 1 -- 172.21.15.31:0/451676911 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c00b9d0 con 0x7f525406ee10 2023-10-06T05:16:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.251+0000 7f525159a700 1 -- 172.21.15.31:0/451676911 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c0092e0 con 0x7f525406ee10 2023-10-06T05:16:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.251+0000 7f52589ff700 1 -- 172.21.15.31:0/451676911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525406ee10 msgr2=0x7f525406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.251+0000 7f52589ff700 1 --2- 172.21.15.31:0/451676911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525406ee10 0x7f525406d7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f524c009f20 tx=0x7f524c009840 comp rx=0 tx=0).stop 2023-10-06T05:16:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.251+0000 7f52589ff700 1 -- 172.21.15.31:0/451676911 shutdown_connections 2023-10-06T05:16:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.251+0000 7f52589ff700 1 --2- 172.21.15.31:0/451676911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525406ee10 0x7f525406d7c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.251+0000 7f52589ff700 1 --2- 172.21.15.31:0/451676911 >> v2:172.21.15.62:3300/0 conn(0x7f52541550a0 0x7f52540ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.251+0000 7f52589ff700 1 -- 172.21.15.31:0/451676911 >> 172.21.15.31:0/451676911 conn(0x7f5254068030 msgr2=0x7f52540cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:16:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.252+0000 7f52589ff700 1 -- 172.21.15.31:0/451676911 shutdown_connections 2023-10-06T05:16:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.252+0000 7f52589ff700 1 -- 172.21.15.31:0/451676911 wait complete. 2023-10-06T05:16:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.253+0000 7f52589ff700 1 Processor -- start 2023-10-06T05:16:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.253+0000 7f52589ff700 1 -- start start 2023-10-06T05:16:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.254+0000 7f52589ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525406ee10 0x7f525414bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.254+0000 7f52589ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5254154cd0 0x7f52540c3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.254+0000 7f52589ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52541550a0 0x7f52540af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.254+0000 7f52589ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540c84d0 con 0x7f5254154cd0 2023-10-06T05:16:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.254+0000 7f52589ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540c35d0 con 0x7f525406ee10 2023-10-06T05:16:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.254+0000 7f52589ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52540c73d0 con 0x7f52541550a0 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.254+0000 7f5251d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5254154cd0 0x7f52540c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f5251d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5254154cd0 0x7f52540c3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48078/0 (socket says 172.21.15.31:48078) 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f5251d9b700 1 -- 172.21.15.31:0/3641523524 learned_addr learned my addr 172.21.15.31:0/3641523524 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f5252d9d700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52541550a0 0x7f52540af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f525259c700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525406ee10 0x7f525414bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f5251d9b700 1 -- 172.21.15.31:0/3641523524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52541550a0 msgr2=0x7f52540af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f5251d9b700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52541550a0 0x7f52540af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f5251d9b700 1 -- 172.21.15.31:0/3641523524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525406ee10 msgr2=0x7f525414bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f5251d9b700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525406ee10 0x7f525414bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f5251d9b700 1 -- 172.21.15.31:0/3641523524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c009ae0 con 0x7f5254154cd0 2023-10-06T05:16:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f525259c700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525406ee10 0x7f525414bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:16:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f5251d9b700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5254154cd0 0x7f52540c3090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f524800ec80 tx=0x7f52480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f52427fc700 1 -- 172.21.15.31:0/3641523524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800c9d0 con 0x7f5254154cd0 2023-10-06T05:16:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f52427fc700 1 -- 172.21.15.31:0/3641523524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524800cb30 con 0x7f5254154cd0 2023-10-06T05:16:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f52589ff700 1 -- 172.21.15.31:0/3641523524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52540c76b0 con 0x7f5254154cd0 2023-10-06T05:16:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.255+0000 7f52427fc700 1 -- 172.21.15.31:0/3641523524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248013470 con 0x7f5254154cd0 2023-10-06T05:16:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.256+0000 7f52589ff700 1 -- 172.21.15.31:0/3641523524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52540c8160 con 0x7f5254154cd0 2023-10-06T05:16:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.256+0000 7f52589ff700 1 -- 172.21.15.31:0/3641523524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525404c380 con 0x7f5254154cd0 2023-10-06T05:16:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.257+0000 7f52427fc700 1 -- 172.21.15.31:0/3641523524 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5248013820 con 0x7f5254154cd0 2023-10-06T05:16:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.259+0000 7f52427fc700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5234066f30 0x7f52340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:16:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.259+0000 7f52427fc700 1 -- 172.21.15.31:0/3641523524 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f52480846d0 con 0x7f5254154cd0 2023-10-06T05:16:59.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.259+0000 7f525259c700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5234066f30 0x7f52340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:16:59.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.259+0000 7f525259c700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5234066f30 0x7f52340693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f525416c990 tx=0x7f5244009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:16:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.263+0000 7f52427fc700 1 -- 172.21.15.31:0/3641523524 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5248051b30 con 0x7f5254154cd0 2023-10-06T05:16:59.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:16:59.468+0000 7f52589ff700 1 -- 172.21.15.31:0/3641523524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f52540c8610 con 0x7f5254154cd0 2023-10-06T05:17:00.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:00.098 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:00.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.415+0000 7f52427fc700 1 -- 172.21.15.31:0/3641523524 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f52540c8610 con 0x7f5254154cd0 2023-10-06T05:17:00.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.420+0000 7f52589ff700 1 -- 172.21.15.31:0/3641523524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5234066f30 msgr2=0x7f52340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:00.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.420+0000 7f52589ff700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5234066f30 0x7f52340693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f525416c990 tx=0x7f5244009040 comp rx=0 tx=0).stop 2023-10-06T05:17:00.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.420+0000 7f52589ff700 1 -- 172.21.15.31:0/3641523524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5254154cd0 msgr2=0x7f52540c3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:00.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.420+0000 7f52589ff700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5254154cd0 0x7f52540c3090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f524800ec80 tx=0x7f52480091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.420+0000 7f52589ff700 1 -- 172.21.15.31:0/3641523524 shutdown_connections 2023-10-06T05:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.420+0000 7f52589ff700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52541550a0 0x7f52540af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.420+0000 7f52589ff700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525406ee10 0x7f525414bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.420+0000 7f52589ff700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5234066f30 0x7f52340693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.420+0000 7f52589ff700 1 --2- 172.21.15.31:0/3641523524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5254154cd0 0x7f52540c3090 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.421+0000 7f52589ff700 1 -- 172.21.15.31:0/3641523524 >> 172.21.15.31:0/3641523524 conn(0x7f5254068030 msgr2=0x7f52540b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.421+0000 7f52589ff700 1 -- 172.21.15.31:0/3641523524 shutdown_connections 2023-10-06T05:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.421+0000 7f52589ff700 1 -- 172.21.15.31:0/3641523524 wait complete. 2023-10-06T05:17:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:17:00.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:17:00.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.649+0000 7f9033bff700 1 Processor -- start 2023-10-06T05:17:00.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.650+0000 7f9033bff700 1 -- start start 2023-10-06T05:17:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.651+0000 7f9033bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0c7190 0x7f902c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.651+0000 7f9033bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f902c0aef00 0x7f902c0af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.651+0000 7f9033bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902c068250 con 0x7f902c0afcf0 2023-10-06T05:17:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.651+0000 7f9033bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902c06ac80 con 0x7f902c0aef00 2023-10-06T05:17:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.651+0000 7f9033bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c067f20 con 0x7f902c0c7190 2023-10-06T05:17:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.651+0000 7f903219c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f902c0afcf0 0x7f902c0686e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34052/0 (socket says 172.21.15.31:34052) 2023-10-06T05:17:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.651+0000 7f903219c700 1 -- 172.21.15.31:0/1864209001 learned_addr learned my addr 172.21.15.31:0/1864209001 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.651+0000 7f903119a700 1 --2- 172.21.15.31:0/1864209001 >> v2:172.21.15.62:3300/0 conn(0x7f902c0aef00 0x7f902c0af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.651+0000 7f903199b700 1 --2- 172.21.15.31:0/1864209001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0c7190 0x7f902c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:00.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.652+0000 7f9030999700 1 -- 172.21.15.31:0/1864209001 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199526823 0 0) 0x7f902c068250 con 0x7f902c0afcf0 2023-10-06T05:17:00.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.652+0000 7f9030999700 1 -- 172.21.15.31:0/1864209001 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9018003570 con 0x7f902c0afcf0 2023-10-06T05:17:00.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.652+0000 7f903119a700 1 -- 172.21.15.31:0/1864209001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0c7190 msgr2=0x7f902c0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:00.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.652+0000 7f903119a700 1 --2- 172.21.15.31:0/1864209001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0c7190 0x7f902c0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.652+0000 7f903119a700 1 -- 172.21.15.31:0/1864209001 >> v1:172.21.15.31:6789/0 conn(0x7f902c0afcf0 legacy=0x7f902c0686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:00.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.652+0000 7f903119a700 1 -- 172.21.15.31:0/1864209001 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c06aed0 con 0x7f902c0aef00 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.652+0000 7f903199b700 1 --2- 172.21.15.31:0/1864209001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0c7190 0x7f902c0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.652+0000 7f903119a700 1 --2- 172.21.15.31:0/1864209001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f902c0aef00 0x7f902c0af7b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f901c0098e0 tx=0x7f901c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f65a37b7609ad060 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.652+0000 7f9030999700 1 -- 172.21.15.31:0/1864209001 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c011630 con 0x7f902c0aef00 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.653+0000 7f9030999700 1 -- 172.21.15.31:0/1864209001 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901c011790 con 0x7f902c0aef00 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.653+0000 7f9030999700 1 -- 172.21.15.31:0/1864209001 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c011b30 con 0x7f902c0aef00 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.653+0000 7f9033bff700 1 -- 172.21.15.31:0/1864209001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f902c0aef00 msgr2=0x7f902c0af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.653+0000 7f9033bff700 1 --2- 172.21.15.31:0/1864209001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f902c0aef00 0x7f902c0af7b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f901c0098e0 tx=0x7f901c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.653+0000 7f9033bff700 1 -- 172.21.15.31:0/1864209001 shutdown_connections 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.653+0000 7f9033bff700 1 --2- 172.21.15.31:0/1864209001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0c7190 0x7f902c0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.653+0000 7f9033bff700 1 --2- 172.21.15.31:0/1864209001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f902c0aef00 0x7f902c0af7b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.653+0000 7f9033bff700 1 -- 172.21.15.31:0/1864209001 >> 172.21.15.31:0/1864209001 conn(0x7f902c0cd3d0 msgr2=0x7f902c0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.653+0000 7f9033bff700 1 -- 172.21.15.31:0/1864209001 shutdown_connections 2023-10-06T05:17:00.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.653+0000 7f9033bff700 1 -- 172.21.15.31:0/1864209001 wait complete. 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.655+0000 7f9033bff700 1 Processor -- start 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.655+0000 7f9033bff700 1 -- start start 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f9033bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f902c0aef00 0x7f902c0c0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f9033bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0afcf0 0x7f902c0c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f9033bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f902c0c7190 0x7f902c0c2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f9033bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0c2000 con 0x7f902c0aef00 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f9033bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0c10d0 con 0x7f902c0c7190 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f9033bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0c1380 con 0x7f902c0afcf0 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f903199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f902c0aef00 0x7f902c0c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f903199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f902c0aef00 0x7f902c0c0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48088/0 (socket says 172.21.15.31:48088) 2023-10-06T05:17:00.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f903199b700 1 -- 172.21.15.31:0/2756150956 learned_addr learned my addr 172.21.15.31:0/2756150956 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f903119a700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0afcf0 0x7f902c0c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f903219c700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f902c0c7190 0x7f902c0c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f903199b700 1 -- 172.21.15.31:0/2756150956 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0afcf0 msgr2=0x7f902c0c0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f903199b700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0afcf0 0x7f902c0c0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f903199b700 1 -- 172.21.15.31:0/2756150956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f902c0c7190 msgr2=0x7f902c0c2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f903199b700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f902c0c7190 0x7f902c0c2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.656+0000 7f903199b700 1 -- 172.21.15.31:0/2756150956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9028009420 con 0x7f902c0aef00 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.657+0000 7f903199b700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f902c0aef00 0x7f902c0c0650 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f902800ec80 tx=0x7f90280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.657+0000 7f9021ffb700 1 -- 172.21.15.31:0/2756150956 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902800c9d0 con 0x7f902c0aef00 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.657+0000 7f9021ffb700 1 -- 172.21.15.31:0/2756150956 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902800cb30 con 0x7f902c0aef00 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.657+0000 7f9021ffb700 1 -- 172.21.15.31:0/2756150956 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902800ceb0 con 0x7f902c0aef00 2023-10-06T05:17:00.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.657+0000 7f9033bff700 1 -- 172.21.15.31:0/2756150956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c009420 con 0x7f902c0aef00 2023-10-06T05:17:00.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.657+0000 7f9033bff700 1 -- 172.21.15.31:0/2756150956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902c0c1940 con 0x7f902c0aef00 2023-10-06T05:17:00.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.658+0000 7f9033bff700 1 -- 172.21.15.31:0/2756150956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902c04c380 con 0x7f902c0aef00 2023-10-06T05:17:00.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.659+0000 7f9021ffb700 1 -- 172.21.15.31:0/2756150956 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9028013680 con 0x7f902c0aef00 2023-10-06T05:17:00.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.660+0000 7f9021ffb700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9010066f30 0x7f90100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:00.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.660+0000 7f9021ffb700 1 -- 172.21.15.31:0/2756150956 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f90280849a0 con 0x7f902c0aef00 2023-10-06T05:17:00.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.660+0000 7f903119a700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9010066f30 0x7f90100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:00.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.661+0000 7f903119a700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9010066f30 0x7f90100693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f901c0091e0 tx=0x7f901c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.665+0000 7f9021ffb700 1 -- 172.21.15.31:0/2756150956 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9028051bf0 con 0x7f902c0aef00 2023-10-06T05:17:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:00.866+0000 7f9033bff700 1 -- 172.21.15.31:0/2756150956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f902c0c9220 con 0x7f902c0aef00 2023-10-06T05:17:01.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.423+0000 7f9021ffb700 1 -- 172.21.15.31:0/2756150956 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9028055210 con 0x7f902c0aef00 2023-10-06T05:17:01.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.429+0000 7f9033bff700 1 -- 172.21.15.31:0/2756150956 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9010066f30 msgr2=0x7f90100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:01.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.429+0000 7f9033bff700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9010066f30 0x7f90100693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f901c0091e0 tx=0x7f901c017000 comp rx=0 tx=0).stop 2023-10-06T05:17:01.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.429+0000 7f9033bff700 1 -- 172.21.15.31:0/2756150956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f902c0aef00 msgr2=0x7f902c0c0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:01.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.429+0000 7f9033bff700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f902c0aef00 0x7f902c0c0650 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f902800ec80 tx=0x7f90280091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.431+0000 7f9033bff700 1 -- 172.21.15.31:0/2756150956 shutdown_connections 2023-10-06T05:17:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.431+0000 7f9033bff700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f902c0c7190 0x7f902c0c2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.431+0000 7f9033bff700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9010066f30 0x7f90100693e0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f901c0091e0 tx=0x7f901c017000 comp rx=0 tx=0).stop 2023-10-06T05:17:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.431+0000 7f9033bff700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f902c0aef00 0x7f902c0c0650 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.431+0000 7f9033bff700 1 --2- 172.21.15.31:0/2756150956 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f902c0afcf0 0x7f902c0c0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.431+0000 7f9033bff700 1 -- 172.21.15.31:0/2756150956 >> 172.21.15.31:0/2756150956 conn(0x7f902c0cd3d0 msgr2=0x7f902c0686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.433+0000 7f9033bff700 1 -- 172.21.15.31:0/2756150956 shutdown_connections 2023-10-06T05:17:01.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.435+0000 7f9033bff700 1 -- 172.21.15.31:0/2756150956 wait complete. 2023-10-06T05:17:01.447 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:17:01.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:17:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.665+0000 7f6eecbff700 1 Processor -- start 2023-10-06T05:17:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.665+0000 7f6eecbff700 1 -- start start 2023-10-06T05:17:01.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.666+0000 7f6eecbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ee815ca40 0x7f6ee8068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:01.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.666+0000 7f6eecbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6ee80c8f40 0x7f6ee80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:01.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.666+0000 7f6eecbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee8068da0 con 0x7f6ee806a300 2023-10-06T05:17:01.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.666+0000 7f6eecbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee80cabc0 con 0x7f6ee815ca40 2023-10-06T05:17:01.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.666+0000 7f6eecbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee8068bb0 con 0x7f6ee80c8f40 2023-10-06T05:17:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.666+0000 7f6ee659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ee815ca40 0x7f6ee8068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.666+0000 7f6ee659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ee815ca40 0x7f6ee8068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48098/0 (socket says 172.21.15.31:48098) 2023-10-06T05:17:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.666+0000 7f6ee659c700 1 -- 172.21.15.31:0/1686026197 learned_addr learned my addr 172.21.15.31:0/1686026197 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.667+0000 7f6ee6d9d700 1 --2- 172.21.15.31:0/1686026197 >> v2:172.21.15.62:3300/0 conn(0x7f6ee80c8f40 0x7f6ee80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:01.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.667+0000 7f6ee659c700 1 -- 172.21.15.31:0/1686026197 >> v2:172.21.15.62:3300/0 conn(0x7f6ee80c8f40 msgr2=0x7f6ee80adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:01.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.667+0000 7f6ee659c700 1 --2- 172.21.15.31:0/1686026197 >> v2:172.21.15.62:3300/0 conn(0x7f6ee80c8f40 0x7f6ee80adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:01.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.667+0000 7f6ee659c700 1 -- 172.21.15.31:0/1686026197 >> v1:172.21.15.62:6789/0 conn(0x7f6ee806a300 legacy=0x7f6ee80c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.667+0000 7f6ee659c700 1 -- 172.21.15.31:0/1686026197 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee8069080 con 0x7f6ee815ca40 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.667+0000 7f6ee659c700 1 --2- 172.21.15.31:0/1686026197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee815ca40 0x7f6ee8068670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80c19b0 tx=0x7f6ed8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd14d4b7d0e27454 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.667+0000 7f6ee5d9b700 1 -- 172.21.15.31:0/1686026197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8009ea0 con 0x7f6ee815ca40 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.667+0000 7f6ee5d9b700 1 -- 172.21.15.31:0/1686026197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed800b820 con 0x7f6ee815ca40 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.667+0000 7f6ee5d9b700 1 -- 172.21.15.31:0/1686026197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed800bbc0 con 0x7f6ee815ca40 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.668+0000 7f6eecbff700 1 -- 172.21.15.31:0/1686026197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee815ca40 msgr2=0x7f6ee8068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.668+0000 7f6eecbff700 1 --2- 172.21.15.31:0/1686026197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee815ca40 0x7f6ee8068670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80c19b0 tx=0x7f6ed8009250 comp rx=0 tx=0).stop 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.668+0000 7f6eecbff700 1 -- 172.21.15.31:0/1686026197 shutdown_connections 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.668+0000 7f6eecbff700 1 --2- 172.21.15.31:0/1686026197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee815ca40 0x7f6ee8068670 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.668+0000 7f6eecbff700 1 --2- 172.21.15.31:0/1686026197 >> v2:172.21.15.62:3300/0 conn(0x7f6ee80c8f40 0x7f6ee80adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:01.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.668+0000 7f6eecbff700 1 -- 172.21.15.31:0/1686026197 >> 172.21.15.31:0/1686026197 conn(0x7f6ee80ce290 msgr2=0x7f6ee80ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:01.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.668+0000 7f6eecbff700 1 -- 172.21.15.31:0/1686026197 shutdown_connections 2023-10-06T05:17:01.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.668+0000 7f6eecbff700 1 -- 172.21.15.31:0/1686026197 wait complete. 2023-10-06T05:17:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.669+0000 7f6eecbff700 1 Processor -- start 2023-10-06T05:17:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.670+0000 7f6eecbff700 1 -- start start 2023-10-06T05:17:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.670+0000 7f6eecbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee806a300 0x7f6ee8159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6eecbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee80c8f40 0x7f6ee80ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6eecbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee815ca40 0x7f6ee815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6eecbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee815a050 con 0x7f6ee80c8f40 2023-10-06T05:17:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6eecbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee816a840 con 0x7f6ee815ca40 2023-10-06T05:17:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6eecbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee8169c20 con 0x7f6ee806a300 2023-10-06T05:17:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6edffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee80c8f40 0x7f6ee80ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6edffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee80c8f40 0x7f6ee80ad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48114/0 (socket says 172.21.15.31:48114) 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6edffff700 1 -- 172.21.15.31:0/3321374665 learned_addr learned my addr 172.21.15.31:0/3321374665 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6ee6d9d700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee815ca40 0x7f6ee815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6ee659c700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee806a300 0x7f6ee8159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6edffff700 1 -- 172.21.15.31:0/3321374665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee806a300 msgr2=0x7f6ee8159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6edffff700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee806a300 0x7f6ee8159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6edffff700 1 -- 172.21.15.31:0/3321374665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee815ca40 msgr2=0x7f6ee815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6edffff700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee815ca40 0x7f6ee815b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6edffff700 1 -- 172.21.15.31:0/3321374665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80094f0 con 0x7f6ee80c8f40 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6ee659c700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee806a300 0x7f6ee8159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.671+0000 7f6edffff700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee80c8f40 0x7f6ee80ad190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000c590 tx=0x7f6ed000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.672+0000 7f6ede7fc700 1 -- 172.21.15.31:0/3321374665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0011590 con 0x7f6ee80c8f40 2023-10-06T05:17:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.672+0000 7f6ede7fc700 1 -- 172.21.15.31:0/3321374665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed0011710 con 0x7f6ee80c8f40 2023-10-06T05:17:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.672+0000 7f6ede7fc700 1 -- 172.21.15.31:0/3321374665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0011ad0 con 0x7f6ee80c8f40 2023-10-06T05:17:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.672+0000 7f6eecbff700 1 -- 172.21.15.31:0/3321374665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee8169f00 con 0x7f6ee80c8f40 2023-10-06T05:17:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.672+0000 7f6eecbff700 1 -- 172.21.15.31:0/3321374665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee816a4c0 con 0x7f6ee80c8f40 2023-10-06T05:17:01.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.673+0000 7f6eecbff700 1 -- 172.21.15.31:0/3321374665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee804c380 con 0x7f6ee80c8f40 2023-10-06T05:17:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.674+0000 7f6ede7fc700 1 -- 172.21.15.31:0/3321374665 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ed001b3b0 con 0x7f6ee80c8f40 2023-10-06T05:17:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.675+0000 7f6ede7fc700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ec8066f60 0x7f6ec8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.675+0000 7f6ede7fc700 1 -- 172.21.15.31:0/3321374665 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f6ed0083c00 con 0x7f6ee80c8f40 2023-10-06T05:17:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.675+0000 7f6ee659c700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ec8066f60 0x7f6ec8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:01.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.676+0000 7f6ee659c700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ec8066f60 0x7f6ec8069410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80c19b0 tx=0x7f6ed8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:01.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.680+0000 7f6ede7fc700 1 -- 172.21.15.31:0/3321374665 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ed0050cd0 con 0x7f6ee80c8f40 2023-10-06T05:17:01.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:01.928+0000 7f6eecbff700 1 -- 172.21.15.31:0/3321374665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6ee80ca1a0 con 0x7f6ee80c8f40 2023-10-06T05:17:02.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.446+0000 7f6ede7fc700 1 -- 172.21.15.31:0/3321374665 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6ed0017070 con 0x7f6ee80c8f40 2023-10-06T05:17:02.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.460+0000 7f6eecbff700 1 -- 172.21.15.31:0/3321374665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ec8066f60 msgr2=0x7f6ec8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:02.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.461+0000 7f6eecbff700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ec8066f60 0x7f6ec8069410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80c19b0 tx=0x7f6ed8017000 comp rx=0 tx=0).stop 2023-10-06T05:17:02.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.461+0000 7f6eecbff700 1 -- 172.21.15.31:0/3321374665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee80c8f40 msgr2=0x7f6ee80ad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:02.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.461+0000 7f6eecbff700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee80c8f40 0x7f6ee80ad190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000c590 tx=0x7f6ed000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.461+0000 7f6eecbff700 1 -- 172.21.15.31:0/3321374665 shutdown_connections 2023-10-06T05:17:02.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.461+0000 7f6eecbff700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee815ca40 0x7f6ee815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.461+0000 7f6eecbff700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee806a300 0x7f6ee8159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.461+0000 7f6eecbff700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ec8066f60 0x7f6ec8069410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.461+0000 7f6eecbff700 1 --2- 172.21.15.31:0/3321374665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee80c8f40 0x7f6ee80ad190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.461+0000 7f6eecbff700 1 -- 172.21.15.31:0/3321374665 >> 172.21.15.31:0/3321374665 conn(0x7f6ee80ce290 msgr2=0x7f6ee80cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:02.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.461+0000 7f6eecbff700 1 -- 172.21.15.31:0/3321374665 shutdown_connections 2023-10-06T05:17:02.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.462+0000 7f6eecbff700 1 -- 172.21.15.31:0/3321374665 wait complete. 2023-10-06T05:17:02.465 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T05:17:02.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:02.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.680+0000 7f4fef9ff700 1 Processor -- start 2023-10-06T05:17:02.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.680+0000 7f4fef9ff700 1 -- start start 2023-10-06T05:17:02.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.681+0000 7f4fef9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fe80cd510 0x7f4fe80b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:02.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.681+0000 7f4fef9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fe815d2d0 0x7f4fe80aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:02.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.681+0000 7f4fef9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe80c3f40 con 0x7f4fe8159500 2023-10-06T05:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.681+0000 7f4fef9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe80b2860 con 0x7f4fe80cd510 2023-10-06T05:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.681+0000 7f4fef9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe815d7d0 con 0x7f4fe815d2d0 2023-10-06T05:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.681+0000 7f4fecf9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fe80cd510 0x7f4fe80b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.681+0000 7f4fed79b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4fe8159500 0x7f4fe80cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34056/0 (socket says 172.21.15.31:34056) 2023-10-06T05:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.681+0000 7f4fecf9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fe80cd510 0x7f4fe80b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48122/0 (socket says 172.21.15.31:48122) 2023-10-06T05:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.681+0000 7f4fed79b700 1 -- 172.21.15.31:0/2679732390 learned_addr learned my addr 172.21.15.31:0/2679732390 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.682+0000 7f4fecf9a700 1 -- 172.21.15.31:0/2679732390 >> v2:172.21.15.62:3300/0 conn(0x7f4fe815d2d0 msgr2=0x7f4fe80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.682+0000 7f4fecf9a700 1 --2- 172.21.15.31:0/2679732390 >> v2:172.21.15.62:3300/0 conn(0x7f4fe815d2d0 0x7f4fe80aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.682+0000 7f4fecf9a700 1 -- 172.21.15.31:0/2679732390 >> v1:172.21.15.31:6789/0 conn(0x7f4fe8159500 legacy=0x7f4fe80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.682+0000 7f4fecf9a700 1 -- 172.21.15.31:0/2679732390 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80cb000 con 0x7f4fe80cd510 2023-10-06T05:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.682+0000 7f4fecf9a700 1 --2- 172.21.15.31:0/2679732390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe80cd510 0x7f4fe80b0ab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80ca9f0 tx=0x7f4fd8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7ceb0ed266fc03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.682+0000 7f4fdffff700 1 -- 172.21.15.31:0/2679732390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8005c10 con 0x7f4fe80cd510 2023-10-06T05:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.682+0000 7f4fdffff700 1 -- 172.21.15.31:0/2679732390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd8005d70 con 0x7f4fe80cd510 2023-10-06T05:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.682+0000 7f4fdffff700 1 -- 172.21.15.31:0/2679732390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8006260 con 0x7f4fe80cd510 2023-10-06T05:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.683+0000 7f4fef9ff700 1 -- 172.21.15.31:0/2679732390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe80cd510 msgr2=0x7f4fe80b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.683+0000 7f4fef9ff700 1 --2- 172.21.15.31:0/2679732390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe80cd510 0x7f4fe80b0ab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80ca9f0 tx=0x7f4fd8009120 comp rx=0 tx=0).stop 2023-10-06T05:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.683+0000 7f4fef9ff700 1 -- 172.21.15.31:0/2679732390 shutdown_connections 2023-10-06T05:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.683+0000 7f4fef9ff700 1 --2- 172.21.15.31:0/2679732390 >> v2:172.21.15.62:3300/0 conn(0x7f4fe815d2d0 0x7f4fe80aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.683+0000 7f4fef9ff700 1 --2- 172.21.15.31:0/2679732390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe80cd510 0x7f4fe80b0ab0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.683+0000 7f4fef9ff700 1 -- 172.21.15.31:0/2679732390 >> 172.21.15.31:0/2679732390 conn(0x7f4fe806ec90 msgr2=0x7f4fe806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:02.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.683+0000 7f4fef9ff700 1 -- 172.21.15.31:0/2679732390 shutdown_connections 2023-10-06T05:17:02.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.683+0000 7f4fef9ff700 1 -- 172.21.15.31:0/2679732390 wait complete. 2023-10-06T05:17:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.684+0000 7f4fef9ff700 1 Processor -- start 2023-10-06T05:17:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.685+0000 7f4fef9ff700 1 -- start start 2023-10-06T05:17:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.685+0000 7f4fef9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fe80cd510 0x7f4fe8155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fef9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe8159500 0x7f4fe80b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fef9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe815d2d0 0x7f4fe80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fef9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80b4440 con 0x7f4fe815d2d0 2023-10-06T05:17:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fef9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80b3830 con 0x7f4fe8159500 2023-10-06T05:17:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fef9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80b3b10 con 0x7f4fe80cd510 2023-10-06T05:17:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fedf9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe815d2d0 0x7f4fe80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fed79b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fe80cd510 0x7f4fe8155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fedf9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe815d2d0 0x7f4fe80ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48136/0 (socket says 172.21.15.31:48136) 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fedf9c700 1 -- 172.21.15.31:0/4123158185 learned_addr learned my addr 172.21.15.31:0/4123158185 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fecf9a700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe8159500 0x7f4fe80b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fedf9c700 1 -- 172.21.15.31:0/4123158185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fe80cd510 msgr2=0x7f4fe8155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fedf9c700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fe80cd510 0x7f4fe8155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fedf9c700 1 -- 172.21.15.31:0/4123158185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe8159500 msgr2=0x7f4fe80b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fedf9c700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe8159500 0x7f4fe80b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fedf9c700 1 -- 172.21.15.31:0/4123158185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd80093c0 con 0x7f4fe815d2d0 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.686+0000 7f4fedf9c700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe815d2d0 0x7f4fe80ceac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000bc30 tx=0x7f4fe0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.687+0000 7f4fdd7fa700 1 -- 172.21.15.31:0/4123158185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00092e0 con 0x7f4fe815d2d0 2023-10-06T05:17:02.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.687+0000 7f4fdd7fa700 1 -- 172.21.15.31:0/4123158185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe000c6f0 con 0x7f4fe815d2d0 2023-10-06T05:17:02.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.687+0000 7f4fdd7fa700 1 -- 172.21.15.31:0/4123158185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe000cad0 con 0x7f4fe815d2d0 2023-10-06T05:17:02.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.687+0000 7f4fef9ff700 1 -- 172.21.15.31:0/4123158185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe80b3df0 con 0x7f4fe815d2d0 2023-10-06T05:17:02.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.687+0000 7f4fef9ff700 1 -- 172.21.15.31:0/4123158185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe8156320 con 0x7f4fe815d2d0 2023-10-06T05:17:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.688+0000 7f4fef9ff700 1 -- 172.21.15.31:0/4123158185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe804c380 con 0x7f4fe815d2d0 2023-10-06T05:17:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.689+0000 7f4fdd7fa700 1 -- 172.21.15.31:0/4123158185 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4fe0012070 con 0x7f4fe815d2d0 2023-10-06T05:17:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.689+0000 7f4fdd7fa700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4fcc066f30 0x7f4fcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.690+0000 7f4fdd7fa700 1 -- 172.21.15.31:0/4123158185 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4fe0084360 con 0x7f4fe815d2d0 2023-10-06T05:17:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.690+0000 7f4fed79b700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4fcc066f30 0x7f4fcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.690+0000 7f4fed79b700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4fcc066f30 0x7f4fcc0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4007130 tx=0x7f4fe4007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:02.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.694+0000 7f4fdd7fa700 1 -- 172.21.15.31:0/4123158185 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fe00513d0 con 0x7f4fe815d2d0 2023-10-06T05:17:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.899+0000 7f4fef9ff700 1 -- 172.21.15.31:0/4123158185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fe80ca7d0 con 0x7f4fe815d2d0 2023-10-06T05:17:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.901+0000 7f4fdd7fa700 1 -- 172.21.15.31:0/4123158185 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16927 (secure 0 0 0) 0x7f4fe00549f0 con 0x7f4fe815d2d0 2023-10-06T05:17:02.903 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:02.903 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:01.929973+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 -- 172.21.15.31:0/4123158185 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4fcc066f30 msgr2=0x7f4fcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4fcc066f30 0x7f4fcc0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4007130 tx=0x7f4fe4007eb0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 -- 172.21.15.31:0/4123158185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe815d2d0 msgr2=0x7f4fe80ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe815d2d0 0x7f4fe80ceac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000bc30 tx=0x7f4fe0009840 comp rx=0 tx=0).stop 2023-10-06T05:17:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 -- 172.21.15.31:0/4123158185 shutdown_connections 2023-10-06T05:17:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4fcc066f30 0x7f4fcc0693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe815d2d0 0x7f4fe80ceac0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fe80cd510 0x7f4fe8155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 --2- 172.21.15.31:0/4123158185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fe8159500 0x7f4fe80b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 -- 172.21.15.31:0/4123158185 >> 172.21.15.31:0/4123158185 conn(0x7f4fe806ec90 msgr2=0x7f4fe80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 -- 172.21.15.31:0/4123158185 shutdown_connections 2023-10-06T05:17:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:02.903+0000 7f4fef9ff700 1 -- 172.21.15.31:0/4123158185 wait complete. 2023-10-06T05:17:02.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.117+0000 7f80ae3ff700 1 Processor -- start 2023-10-06T05:17:03.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.117+0000 7f80ae3ff700 1 -- start start 2023-10-06T05:17:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.118+0000 7f80ae3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a80cd510 0x7f80a80b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.118+0000 7f80ae3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f80a815d2d0 0x7f80a80aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.118+0000 7f80ae3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a80c3f40 con 0x7f80a8159500 2023-10-06T05:17:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.118+0000 7f80ae3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a80b2860 con 0x7f80a815d2d0 2023-10-06T05:17:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.118+0000 7f80ae3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a815d7d0 con 0x7f80a80cd510 2023-10-06T05:17:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.119+0000 7f80a77fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a80cd510 0x7f80a80b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.119+0000 7f80a77fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a80cd510 0x7f80a80b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:53762/0 (socket says 172.21.15.31:53762) 2023-10-06T05:17:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.119+0000 7f80a77fe700 1 -- 172.21.15.31:0/1026977344 learned_addr learned my addr 172.21.15.31:0/1026977344 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:03.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.119+0000 7f80ac99c700 1 --2- 172.21.15.31:0/1026977344 >> v2:172.21.15.62:3300/0 conn(0x7f80a815d2d0 0x7f80a80aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.119+0000 7f80a77fe700 1 -- 172.21.15.31:0/1026977344 >> v2:172.21.15.62:3300/0 conn(0x7f80a815d2d0 msgr2=0x7f80a80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:03.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.119+0000 7f80a77fe700 1 --2- 172.21.15.31:0/1026977344 >> v2:172.21.15.62:3300/0 conn(0x7f80a815d2d0 0x7f80a80aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.119+0000 7f80a77fe700 1 -- 172.21.15.31:0/1026977344 >> v1:172.21.15.62:6789/0 conn(0x7f80a8159500 legacy=0x7f80a80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.119+0000 7f80a77fe700 1 -- 172.21.15.31:0/1026977344 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80cb000 con 0x7f80a80cd510 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.119+0000 7f80a77fe700 1 --2- 172.21.15.31:0/1026977344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a80cd510 0x7f80a80b0ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f809c009990 tx=0x7f809c009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3743c8cdace88e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80a6ffd700 1 -- 172.21.15.31:0/1026977344 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c009e80 con 0x7f80a80cd510 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80a6ffd700 1 -- 172.21.15.31:0/1026977344 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c005c10 con 0x7f80a80cd510 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80a6ffd700 1 -- 172.21.15.31:0/1026977344 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c00b650 con 0x7f80a80cd510 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80ae3ff700 1 -- 172.21.15.31:0/1026977344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a80cd510 msgr2=0x7f80a80b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80ae3ff700 1 --2- 172.21.15.31:0/1026977344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a80cd510 0x7f80a80b0ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f809c009990 tx=0x7f809c009220 comp rx=0 tx=0).stop 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80ae3ff700 1 -- 172.21.15.31:0/1026977344 shutdown_connections 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80ae3ff700 1 --2- 172.21.15.31:0/1026977344 >> v2:172.21.15.62:3300/0 conn(0x7f80a815d2d0 0x7f80a80aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80ae3ff700 1 --2- 172.21.15.31:0/1026977344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a80cd510 0x7f80a80b0ab0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80ae3ff700 1 -- 172.21.15.31:0/1026977344 >> 172.21.15.31:0/1026977344 conn(0x7f80a806ec90 msgr2=0x7f80a806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80ae3ff700 1 -- 172.21.15.31:0/1026977344 shutdown_connections 2023-10-06T05:17:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.120+0000 7f80ae3ff700 1 -- 172.21.15.31:0/1026977344 wait complete. 2023-10-06T05:17:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.122+0000 7f80ae3ff700 1 Processor -- start 2023-10-06T05:17:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.122+0000 7f80ae3ff700 1 -- start start 2023-10-06T05:17:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.123+0000 7f80ae3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a80cd510 0x7f80a815a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.123+0000 7f80ae3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a8159500 0x7f80a8158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.123+0000 7f80ae3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80a815d2d0 0x7f80a816a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.123+0000 7f80ae3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a815a860 con 0x7f80a80cd510 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.123+0000 7f80ae3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a81598d0 con 0x7f80a815d2d0 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.123+0000 7f80ae3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8159e90 con 0x7f80a8159500 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.123+0000 7f80a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a80cd510 0x7f80a815a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.123+0000 7f80a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a80cd510 0x7f80a815a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48152/0 (socket says 172.21.15.31:48152) 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.123+0000 7f80a7fff700 1 -- 172.21.15.31:0/3507492576 learned_addr learned my addr 172.21.15.31:0/3507492576 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f80a77fe700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a8159500 0x7f80a8158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f80ac99c700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80a815d2d0 0x7f80a816a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f80a7fff700 1 -- 172.21.15.31:0/3507492576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a8159500 msgr2=0x7f80a8158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f80a7fff700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a8159500 0x7f80a8158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f80a7fff700 1 -- 172.21.15.31:0/3507492576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80a815d2d0 msgr2=0x7f80a816a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f80a7fff700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80a815d2d0 0x7f80a816a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f80a7fff700 1 -- 172.21.15.31:0/3507492576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c009480 con 0x7f80a80cd510 2023-10-06T05:17:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f80a7fff700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a80cd510 0x7f80a815a320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f809800c5c0 tx=0x7f809800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f808ffff700 1 -- 172.21.15.31:0/3507492576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098011590 con 0x7f80a80cd510 2023-10-06T05:17:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f808ffff700 1 -- 172.21.15.31:0/3507492576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8098011710 con 0x7f80a80cd510 2023-10-06T05:17:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f80ae3ff700 1 -- 172.21.15.31:0/3507492576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a8158640 con 0x7f80a80cd510 2023-10-06T05:17:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f808ffff700 1 -- 172.21.15.31:0/3507492576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098011ad0 con 0x7f80a80cd510 2023-10-06T05:17:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.124+0000 7f80ae3ff700 1 -- 172.21.15.31:0/3507492576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a816aef0 con 0x7f80a80cd510 2023-10-06T05:17:03.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.125+0000 7f80ae3ff700 1 -- 172.21.15.31:0/3507492576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a804c380 con 0x7f80a80cd510 2023-10-06T05:17:03.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.126+0000 7f808ffff700 1 -- 172.21.15.31:0/3507492576 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f809801b3b0 con 0x7f80a80cd510 2023-10-06T05:17:03.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.127+0000 7f808ffff700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8094066f30 0x7f80940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.127+0000 7f808ffff700 1 -- 172.21.15.31:0/3507492576 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8098083d70 con 0x7f80a80cd510 2023-10-06T05:17:03.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.127+0000 7f80a77fe700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8094066f30 0x7f80940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.128+0000 7f80a77fe700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8094066f30 0x7f80940693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f809c009260 tx=0x7f809c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.131+0000 7f808ffff700 1 -- 172.21.15.31:0/3507492576 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8098050de0 con 0x7f80a80cd510 2023-10-06T05:17:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.334+0000 7f80ae3ff700 1 -- 172.21.15.31:0/3507492576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80a80cade0 con 0x7f80a80cd510 2023-10-06T05:17:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.335+0000 7f808ffff700 1 -- 172.21.15.31:0/3507492576 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16927 (secure 0 0 0) 0x7f8098017070 con 0x7f80a80cd510 2023-10-06T05:17:03.337 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:03.337 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:01.929973+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 -- 172.21.15.31:0/3507492576 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8094066f30 msgr2=0x7f80940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8094066f30 0x7f80940693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f809c009260 tx=0x7f809c016000 comp rx=0 tx=0).stop 2023-10-06T05:17:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 -- 172.21.15.31:0/3507492576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a80cd510 msgr2=0x7f80a815a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a80cd510 0x7f80a815a320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f809800c5c0 tx=0x7f809800c980 comp rx=0 tx=0).stop 2023-10-06T05:17:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 -- 172.21.15.31:0/3507492576 shutdown_connections 2023-10-06T05:17:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80a815d2d0 0x7f80a816a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8094066f30 0x7f80940693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a80cd510 0x7f80a815a320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 --2- 172.21.15.31:0/3507492576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80a8159500 0x7f80a8158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 -- 172.21.15.31:0/3507492576 >> 172.21.15.31:0/3507492576 conn(0x7f80a806ec90 msgr2=0x7f80a81589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 -- 172.21.15.31:0/3507492576 shutdown_connections 2023-10-06T05:17:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.337+0000 7f80ae3ff700 1 -- 172.21.15.31:0/3507492576 wait complete. 2023-10-06T05:17:03.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:17:03.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.556+0000 7fa4e7dff700 1 Processor -- start 2023-10-06T05:17:03.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.556+0000 7fa4e7dff700 1 -- start start 2023-10-06T05:17:03.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.557+0000 7fa4e7dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4e0153d40 0x7fa4e00b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.557+0000 7fa4e7dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e00af890 0x7fa4e00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e7dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e00b0e40 con 0x7fa4e00b09c0 2023-10-06T05:17:03.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e7dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e00afcd0 con 0x7fa4e0153d40 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e7dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e00ad690 con 0x7fa4e00af890 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e5b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4e0153d40 0x7fa4e00b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e5b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4e0153d40 0x7fa4e00b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48156/0 (socket says 172.21.15.31:48156) 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e5b9b700 1 -- 172.21.15.31:0/1600779986 learned_addr learned my addr 172.21.15.31:0/1600779986 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e539a700 1 --1- 172.21.15.31:0/1600779986 >> v1:172.21.15.31:6789/0 conn(0x7fa4e00b09c0 0x7fa4e00af180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34068/0 (socket says 172.21.15.31:34068) 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e639c700 1 --2- 172.21.15.31:0/1600779986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e00af890 0x7fa4e00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e5b9b700 1 -- 172.21.15.31:0/1600779986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e00af890 msgr2=0x7fa4e00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e5b9b700 1 --2- 172.21.15.31:0/1600779986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e00af890 0x7fa4e00adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e5b9b700 1 -- 172.21.15.31:0/1600779986 >> v1:172.21.15.31:6789/0 conn(0x7fa4e00b09c0 legacy=0x7fa4e00af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e5b9b700 1 -- 172.21.15.31:0/1600779986 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e00acd60 con 0x7fa4e0153d40 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e4b99700 1 -- 172.21.15.31:0/1600779986 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116249857 0 0) 0x7fa4d0001110 con 0x7fa4e00b09c0 2023-10-06T05:17:03.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.558+0000 7fa4e5b9b700 1 --2- 172.21.15.31:0/1600779986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 0x7fa4e00b0480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0099b0 tx=0x7fa4dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c561f01e2c321da0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.559+0000 7fa4e4b99700 1 -- 172.21.15.31:0/1600779986 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00b860 con 0x7fa4e0153d40 2023-10-06T05:17:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.559+0000 7fa4e4b99700 1 -- 172.21.15.31:0/1600779986 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4dc00b9c0 con 0x7fa4e0153d40 2023-10-06T05:17:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.559+0000 7fa4e4b99700 1 -- 172.21.15.31:0/1600779986 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00bd60 con 0x7fa4e0153d40 2023-10-06T05:17:03.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.559+0000 7fa4e7dff700 1 -- 172.21.15.31:0/1600779986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 msgr2=0x7fa4e00b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.559+0000 7fa4e7dff700 1 --2- 172.21.15.31:0/1600779986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 0x7fa4e00b0480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0099b0 tx=0x7fa4dc009250 comp rx=0 tx=0).stop 2023-10-06T05:17:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.559+0000 7fa4e7dff700 1 -- 172.21.15.31:0/1600779986 shutdown_connections 2023-10-06T05:17:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.559+0000 7fa4e7dff700 1 --2- 172.21.15.31:0/1600779986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 0x7fa4e00b0480 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.559+0000 7fa4e7dff700 1 --2- 172.21.15.31:0/1600779986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e00af890 0x7fa4e00adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.559+0000 7fa4e7dff700 1 -- 172.21.15.31:0/1600779986 >> 172.21.15.31:0/1600779986 conn(0x7fa4e0067560 msgr2=0x7fa4e0067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.559+0000 7fa4e7dff700 1 -- 172.21.15.31:0/1600779986 shutdown_connections 2023-10-06T05:17:03.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.560+0000 7fa4e7dff700 1 -- 172.21.15.31:0/1600779986 wait complete. 2023-10-06T05:17:03.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.562+0000 7fa4e7dff700 1 Processor -- start 2023-10-06T05:17:03.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.562+0000 7fa4e7dff700 1 -- start start 2023-10-06T05:17:03.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e7dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e00af890 0x7fa4e00ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e7dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00b09c0 0x7fa4e014dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e7dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 0x7fa4e00aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e7dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e014e220 con 0x7fa4e0153d40 2023-10-06T05:17:03.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e7dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e00ab480 con 0x7fa4e00b09c0 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e7dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e00cb500 con 0x7fa4e00af890 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 0x7fa4e00aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 0x7fa4e00aaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48168/0 (socket says 172.21.15.31:48168) 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e639c700 1 -- 172.21.15.31:0/3759837890 learned_addr learned my addr 172.21.15.31:0/3759837890 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e5b9b700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e00af890 0x7fa4e00ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.563+0000 7fa4e539a700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00b09c0 0x7fa4e014dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4e639c700 1 -- 172.21.15.31:0/3759837890 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e00af890 msgr2=0x7fa4e00ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4e639c700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e00af890 0x7fa4e00ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4e639c700 1 -- 172.21.15.31:0/3759837890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00b09c0 msgr2=0x7fa4e014dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4e639c700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00b09c0 0x7fa4e014dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4e639c700 1 -- 172.21.15.31:0/3759837890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc0094f0 con 0x7fa4e0153d40 2023-10-06T05:17:03.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4e539a700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00b09c0 0x7fa4e014dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:03.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4e639c700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 0x7fa4e00aaf40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009fa0 tx=0x7fa4d8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:03.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4d5ffb700 1 -- 172.21.15.31:0/3759837890 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800ec70 con 0x7fa4e0153d40 2023-10-06T05:17:03.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4d5ffb700 1 -- 172.21.15.31:0/3759837890 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d800edd0 con 0x7fa4e0153d40 2023-10-06T05:17:03.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4e7dff700 1 -- 172.21.15.31:0/3759837890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e00ca8f0 con 0x7fa4e0153d40 2023-10-06T05:17:03.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.564+0000 7fa4d5ffb700 1 -- 172.21.15.31:0/3759837890 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8014670 con 0x7fa4e0153d40 2023-10-06T05:17:03.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.565+0000 7fa4e7dff700 1 -- 172.21.15.31:0/3759837890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e00caeb0 con 0x7fa4e0153d40 2023-10-06T05:17:03.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.565+0000 7fa4e7dff700 1 -- 172.21.15.31:0/3759837890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e00cb6b0 con 0x7fa4e0153d40 2023-10-06T05:17:03.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.566+0000 7fa4d5ffb700 1 -- 172.21.15.31:0/3759837890 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4d80149e0 con 0x7fa4e0153d40 2023-10-06T05:17:03.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.567+0000 7fa4d5ffb700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4c4066f30 0x7fa4c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:03.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.567+0000 7fa4d5ffb700 1 -- 172.21.15.31:0/3759837890 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa4d8084e10 con 0x7fa4e0153d40 2023-10-06T05:17:03.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.568+0000 7fa4e5b9b700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4c4066f30 0x7fa4c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:03.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.568+0000 7fa4e5b9b700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4c4066f30 0x7fa4c40693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0099b0 tx=0x7fa4dc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:03.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.571+0000 7fa4d5ffb700 1 -- 172.21.15.31:0/3759837890 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4e00cb6b0 con 0x7fa4e0153d40 2023-10-06T05:17:03.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:03.817+0000 7fa4e7dff700 1 -- 172.21.15.31:0/3759837890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa4e00cb670 con 0x7fa4e0153d40 2023-10-06T05:17:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.460+0000 7fa4d5ffb700 1 -- 172.21.15.31:0/3759837890 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa4d8088030 con 0x7fa4e0153d40 2023-10-06T05:17:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 -- 172.21.15.31:0/3759837890 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4c4066f30 msgr2=0x7fa4c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4c4066f30 0x7fa4c40693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0099b0 tx=0x7fa4dc017000 comp rx=0 tx=0).stop 2023-10-06T05:17:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 -- 172.21.15.31:0/3759837890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 msgr2=0x7fa4e00aaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 0x7fa4e00aaf40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009fa0 tx=0x7fa4d8009900 comp rx=0 tx=0).stop 2023-10-06T05:17:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 -- 172.21.15.31:0/3759837890 shutdown_connections 2023-10-06T05:17:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4c4066f30 0x7fa4c40693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0153d40 0x7fa4e00aaf40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e00af890 0x7fa4e00ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 --2- 172.21.15.31:0/3759837890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00b09c0 0x7fa4e014dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:04.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 -- 172.21.15.31:0/3759837890 >> 172.21.15.31:0/3759837890 conn(0x7fa4e0067560 msgr2=0x7fa4e0151e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:04.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 -- 172.21.15.31:0/3759837890 shutdown_connections 2023-10-06T05:17:04.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.463+0000 7fa4e7dff700 1 -- 172.21.15.31:0/3759837890 wait complete. 2023-10-06T05:17:04.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:17:04.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.686+0000 7f6de87ff700 1 Processor -- start 2023-10-06T05:17:04.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.686+0000 7f6de87ff700 1 -- start start 2023-10-06T05:17:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.687+0000 7f6de87ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6de006a300 0x7f6de0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:04.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.687+0000 7f6de87ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de0068da0 con 0x7f6de00c8f40 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.687+0000 7f6de87ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de0069080 con 0x7f6de015ca40 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.687+0000 7f6de87ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de00cabc0 con 0x7f6de006a300 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.687+0000 7f6de5d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6de00c8f40 0x7f6de00adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34078/0 (socket says 172.21.15.31:34078) 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.687+0000 7f6de5d9a700 1 -- 172.21.15.31:0/1203399266 learned_addr learned my addr 172.21.15.31:0/1203399266 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.688+0000 7f6de6d9c700 1 --2- 172.21.15.31:0/1203399266 >> v2:172.21.15.62:3300/0 conn(0x7f6de006a300 0x7f6de0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.688+0000 7f6de5599700 1 -- 172.21.15.31:0/1203399266 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161157248 0 0) 0x7f6de0068da0 con 0x7f6de00c8f40 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.688+0000 7f6de5599700 1 -- 172.21.15.31:0/1203399266 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dcc003570 con 0x7f6de00c8f40 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.688+0000 7f6de6d9c700 1 -- 172.21.15.31:0/1203399266 >> v1:172.21.15.62:6789/0 conn(0x7f6de015ca40 legacy=0x7f6de00c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.688+0000 7f6de5599700 1 -- 172.21.15.31:0/1203399266 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2653514888 0 0) 0x7f6dcc003570 con 0x7f6de00c8f40 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.688+0000 7f6de6d9c700 1 -- 172.21.15.31:0/1203399266 >> v1:172.21.15.31:6789/0 conn(0x7f6de00c8f40 legacy=0x7f6de00adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:04.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.688+0000 7f6de6d9c700 1 -- 172.21.15.31:0/1203399266 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0069360 con 0x7f6de006a300 2023-10-06T05:17:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.688+0000 7f6de6d9c700 1 --2- 172.21.15.31:0/1203399266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de006a300 0x7f6de0068670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009e60 tx=0x7f6dd8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf00cc741edd884a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.689+0000 7f6de5599700 1 -- 172.21.15.31:0/1203399266 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800e070 con 0x7f6de006a300 2023-10-06T05:17:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.689+0000 7f6de5599700 1 -- 172.21.15.31:0/1203399266 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd800b850 con 0x7f6de006a300 2023-10-06T05:17:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.689+0000 7f6de5599700 1 -- 172.21.15.31:0/1203399266 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0001470 con 0x7f6de006a300 2023-10-06T05:17:04.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.689+0000 7f6de87ff700 1 -- 172.21.15.31:0/1203399266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de006a300 msgr2=0x7f6de0068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:04.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.689+0000 7f6de87ff700 1 --2- 172.21.15.31:0/1203399266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de006a300 0x7f6de0068670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009e60 tx=0x7f6dd8009780 comp rx=0 tx=0).stop 2023-10-06T05:17:04.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.689+0000 7f6de87ff700 1 -- 172.21.15.31:0/1203399266 shutdown_connections 2023-10-06T05:17:04.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.689+0000 7f6de87ff700 1 --2- 172.21.15.31:0/1203399266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de006a300 0x7f6de0068670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:04.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.689+0000 7f6de87ff700 1 -- 172.21.15.31:0/1203399266 >> 172.21.15.31:0/1203399266 conn(0x7f6de00ce290 msgr2=0x7f6de00ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:04.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.689+0000 7f6de87ff700 1 -- 172.21.15.31:0/1203399266 shutdown_connections 2023-10-06T05:17:04.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.689+0000 7f6de87ff700 1 -- 172.21.15.31:0/1203399266 wait complete. 2023-10-06T05:17:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.691+0000 7f6de87ff700 1 Processor -- start 2023-10-06T05:17:04.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.691+0000 7f6de87ff700 1 -- start start 2023-10-06T05:17:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de87ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de006a300 0x7f6de00c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de87ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6de00c8f40 0x7f6de00c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de87ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de015ca40 0x7f6de0158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de87ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00c44b0 con 0x7f6de006a300 2023-10-06T05:17:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de87ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de016def0 con 0x7f6de015ca40 2023-10-06T05:17:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de87ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01588b0 con 0x7f6de00c8f40 2023-10-06T05:17:04.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de006a300 0x7f6de00c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de006a300 0x7f6de00c26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48170/0 (socket says 172.21.15.31:48170) 2023-10-06T05:17:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de659b700 1 -- 172.21.15.31:0/1152601696 learned_addr learned my addr 172.21.15.31:0/1152601696 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de5d9a700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6de00c8f40 0x7f6de00c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.692+0000 7f6de6d9c700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de015ca40 0x7f6de0158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6de659b700 1 -- 172.21.15.31:0/1152601696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6de00c8f40 msgr2=0x7f6de00c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6de659b700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6de00c8f40 0x7f6de00c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6de659b700 1 -- 172.21.15.31:0/1152601696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de015ca40 msgr2=0x7f6de0158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6de659b700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de015ca40 0x7f6de0158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6de659b700 1 -- 172.21.15.31:0/1152601696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd8009a20 con 0x7f6de006a300 2023-10-06T05:17:04.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6de5d9a700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6de00c8f40 0x7f6de00c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6de6d9c700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de015ca40 0x7f6de0158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6de659b700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de006a300 0x7f6de00c26b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c540 tx=0x7f6ddc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6dd67fc700 1 -- 172.21.15.31:0/1152601696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc011590 con 0x7f6de006a300 2023-10-06T05:17:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6dd67fc700 1 -- 172.21.15.31:0/1152601696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc011710 con 0x7f6de006a300 2023-10-06T05:17:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6dd67fc700 1 -- 172.21.15.31:0/1152601696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc011ad0 con 0x7f6de006a300 2023-10-06T05:17:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6de87ff700 1 -- 172.21.15.31:0/1152601696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de00c46a0 con 0x7f6de006a300 2023-10-06T05:17:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.693+0000 7f6de87ff700 1 -- 172.21.15.31:0/1152601696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de016da60 con 0x7f6de006a300 2023-10-06T05:17:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.694+0000 7f6de87ff700 1 -- 172.21.15.31:0/1152601696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de004c380 con 0x7f6de006a300 2023-10-06T05:17:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.695+0000 7f6dd67fc700 1 -- 172.21.15.31:0/1152601696 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ddc01b3b0 con 0x7f6de006a300 2023-10-06T05:17:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.696+0000 7f6dd67fc700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6dc4066f30 0x7f6dc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.696+0000 7f6dd67fc700 1 -- 172.21.15.31:0/1152601696 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ddc083be0 con 0x7f6de006a300 2023-10-06T05:17:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.696+0000 7f6de5d9a700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6dc4066f30 0x7f6dc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:04.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.697+0000 7f6de5d9a700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6dc4066f30 0x7f6dc40693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00011e0 tx=0x7f6dd0001940 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:04.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.700+0000 7f6dd67fc700 1 -- 172.21.15.31:0/1152601696 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ddc050c50 con 0x7f6de006a300 2023-10-06T05:17:04.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:04.950+0000 7f6de87ff700 1 -- 172.21.15.31:0/1152601696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6de00ca1a0 con 0x7f6de006a300 2023-10-06T05:17:05.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.462+0000 7f6dd67fc700 1 -- 172.21.15.31:0/1152601696 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ddc017020 con 0x7f6de006a300 2023-10-06T05:17:05.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 -- 172.21.15.31:0/1152601696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6dc4066f30 msgr2=0x7f6dc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:05.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6dc4066f30 0x7f6dc40693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00011e0 tx=0x7f6dd0001940 comp rx=0 tx=0).stop 2023-10-06T05:17:05.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 -- 172.21.15.31:0/1152601696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de006a300 msgr2=0x7f6de00c26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:05.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de006a300 0x7f6de00c26b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c540 tx=0x7f6ddc00c900 comp rx=0 tx=0).stop 2023-10-06T05:17:05.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 -- 172.21.15.31:0/1152601696 shutdown_connections 2023-10-06T05:17:05.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de015ca40 0x7f6de0158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6dc4066f30 0x7f6dc40693e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de006a300 0x7f6de00c26b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 --2- 172.21.15.31:0/1152601696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6de00c8f40 0x7f6de00c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 -- 172.21.15.31:0/1152601696 >> 172.21.15.31:0/1152601696 conn(0x7f6de00ce290 msgr2=0x7f6de015a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:05.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 -- 172.21.15.31:0/1152601696 shutdown_connections 2023-10-06T05:17:05.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.465+0000 7f6de87ff700 1 -- 172.21.15.31:0/1152601696 wait complete. 2023-10-06T05:17:05.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:05.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.685+0000 7f39883ff700 1 Processor -- start 2023-10-06T05:17:05.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.685+0000 7f39883ff700 1 -- start start 2023-10-06T05:17:05.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.686+0000 7f39883ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 0x7f39800cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:05.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.686+0000 7f39883ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39800cb1b0 0x7f39800ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.686+0000 7f39883ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39800ce090 con 0x7f39800cd270 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.686+0000 7f39883ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39800cb6f0 con 0x7f39800cb1b0 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.686+0000 7f39883ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39800cb9d0 con 0x7f398015a930 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.686+0000 7f398599a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39800cb1b0 0x7f39800ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f398599a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39800cb1b0 0x7f39800ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48180/0 (socket says 172.21.15.31:48180) 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f398599a700 1 -- 172.21.15.31:0/1683281037 learned_addr learned my addr 172.21.15.31:0/1683281037 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f398619b700 1 --2- 172.21.15.31:0/1683281037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 0x7f39800cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f398599a700 1 -- 172.21.15.31:0/1683281037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 msgr2=0x7f39800cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f398599a700 1 --2- 172.21.15.31:0/1683281037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 0x7f39800cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f398599a700 1 -- 172.21.15.31:0/1683281037 >> v1:172.21.15.62:6789/0 conn(0x7f39800cd270 legacy=0x7f39800ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f398599a700 1 -- 172.21.15.31:0/1683281037 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39800cbcb0 con 0x7f39800cb1b0 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f398599a700 1 --2- 172.21.15.31:0/1683281037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 0x7f39800ccd30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39700099b0 tx=0x7f3970009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7eb05604dc51f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f3985199700 1 -- 172.21.15.31:0/1683281037 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397000b8f0 con 0x7f39800cb1b0 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f3985199700 1 -- 172.21.15.31:0/1683281037 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397000ba50 con 0x7f39800cb1b0 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.687+0000 7f3985199700 1 -- 172.21.15.31:0/1683281037 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397000bdf0 con 0x7f39800cb1b0 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.688+0000 7f39883ff700 1 -- 172.21.15.31:0/1683281037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 msgr2=0x7f39800ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.688+0000 7f39883ff700 1 --2- 172.21.15.31:0/1683281037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 0x7f39800ccd30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39700099b0 tx=0x7f3970009250 comp rx=0 tx=0).stop 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.688+0000 7f39883ff700 1 -- 172.21.15.31:0/1683281037 shutdown_connections 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.688+0000 7f39883ff700 1 --2- 172.21.15.31:0/1683281037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 0x7f39800cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.688+0000 7f39883ff700 1 --2- 172.21.15.31:0/1683281037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 0x7f39800ccd30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.688+0000 7f39883ff700 1 -- 172.21.15.31:0/1683281037 >> 172.21.15.31:0/1683281037 conn(0x7f3980099720 msgr2=0x7f398006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.688+0000 7f39883ff700 1 -- 172.21.15.31:0/1683281037 shutdown_connections 2023-10-06T05:17:05.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.688+0000 7f39883ff700 1 -- 172.21.15.31:0/1683281037 wait complete. 2023-10-06T05:17:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.690+0000 7f39883ff700 1 Processor -- start 2023-10-06T05:17:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.690+0000 7f39883ff700 1 -- start start 2023-10-06T05:17:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f39883ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 0x7f398009c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f39883ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39800cd270 0x7f3980152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f39883ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 0x7f398016cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f39883ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398009c030 con 0x7f39800cb1b0 2023-10-06T05:17:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f39883ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398009ca00 con 0x7f39800cd270 2023-10-06T05:17:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f39883ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3980153300 con 0x7f398015a930 2023-10-06T05:17:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398619b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 0x7f398009c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398619b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 0x7f398009c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48192/0 (socket says 172.21.15.31:48192) 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398619b700 1 -- 172.21.15.31:0/1661170343 learned_addr learned my addr 172.21.15.31:0/1661170343 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398599a700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39800cd270 0x7f3980152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398699c700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 0x7f398016cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398619b700 1 -- 172.21.15.31:0/1661170343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 msgr2=0x7f398016cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398619b700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 0x7f398016cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398619b700 1 -- 172.21.15.31:0/1661170343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39800cd270 msgr2=0x7f3980152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398619b700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39800cd270 0x7f3980152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398619b700 1 -- 172.21.15.31:0/1661170343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700094f0 con 0x7f39800cb1b0 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.691+0000 7f398699c700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 0x7f398016cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.692+0000 7f398619b700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 0x7f398009c4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f397c00d840 tx=0x7f397c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.692+0000 7f39767fc700 1 -- 172.21.15.31:0/1661170343 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c014620 con 0x7f39800cb1b0 2023-10-06T05:17:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.692+0000 7f39767fc700 1 -- 172.21.15.31:0/1661170343 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c014780 con 0x7f39800cb1b0 2023-10-06T05:17:05.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.692+0000 7f39767fc700 1 -- 172.21.15.31:0/1661170343 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c014b20 con 0x7f39800cb1b0 2023-10-06T05:17:05.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.692+0000 7f39883ff700 1 -- 172.21.15.31:0/1661170343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398016d900 con 0x7f39800cb1b0 2023-10-06T05:17:05.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.692+0000 7f39883ff700 1 -- 172.21.15.31:0/1661170343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398009bcc0 con 0x7f39800cb1b0 2023-10-06T05:17:05.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.693+0000 7f39883ff700 1 -- 172.21.15.31:0/1661170343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398009c310 con 0x7f39800cb1b0 2023-10-06T05:17:05.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.694+0000 7f39767fc700 1 -- 172.21.15.31:0/1661170343 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f397c01d400 con 0x7f39800cb1b0 2023-10-06T05:17:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.695+0000 7f39767fc700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3964066f30 0x7f39640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.695+0000 7f39767fc700 1 -- 172.21.15.31:0/1661170343 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f397c084e80 con 0x7f39800cb1b0 2023-10-06T05:17:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.695+0000 7f398599a700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3964066f30 0x7f39640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.696+0000 7f398599a700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3964066f30 0x7f39640693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f39700099b0 tx=0x7f3970017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:05.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.699+0000 7f39767fc700 1 -- 172.21.15.31:0/1661170343 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f397c051f90 con 0x7f39800cb1b0 2023-10-06T05:17:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.947+0000 7f39883ff700 1 -- 172.21.15.31:0/1661170343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f398004c380 con 0x7f39800cb1b0 2023-10-06T05:17:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.947+0000 7f39767fc700 1 -- 172.21.15.31:0/1661170343 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f397c0555b0 con 0x7f39800cb1b0 2023-10-06T05:17:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:05.949 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1001195925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1001195925},{"type":"v1","addr":"172.21.15.62:6835","nonce":1001195925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1336279472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1336279472},{"type":"v1","addr":"172.21.15.31:6837","nonce":1336279472}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/577485679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":577485679},{"type":"v1","addr":"172.21.15.62:6834","nonce":577485679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:01.930460+0000","modified":"2023-10-06T05:17:05.459675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4169737410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169737410},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169737410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:17:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 -- 172.21.15.31:0/1661170343 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3964066f30 msgr2=0x7f39640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3964066f30 0x7f39640693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f39700099b0 tx=0x7f3970017000 comp rx=0 tx=0).stop 2023-10-06T05:17:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 -- 172.21.15.31:0/1661170343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 msgr2=0x7f398009c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 0x7f398009c4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f397c00d840 tx=0x7f397c008040 comp rx=0 tx=0).stop 2023-10-06T05:17:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 -- 172.21.15.31:0/1661170343 shutdown_connections 2023-10-06T05:17:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f398015a930 0x7f398016cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3964066f30 0x7f39640693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39800cb1b0 0x7f398009c4c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 --2- 172.21.15.31:0/1661170343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39800cd270 0x7f3980152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 -- 172.21.15.31:0/1661170343 >> 172.21.15.31:0/1661170343 conn(0x7f3980099720 msgr2=0x7f3980159f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 -- 172.21.15.31:0/1661170343 shutdown_connections 2023-10-06T05:17:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:05.950+0000 7f39883ff700 1 -- 172.21.15.31:0/1661170343 wait complete. 2023-10-06T05:17:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T05:17:05.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.163+0000 7fe45dbff700 1 Processor -- start 2023-10-06T05:17:06.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.163+0000 7fe45dbff700 1 -- start start 2023-10-06T05:17:06.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.164+0000 7fe45dbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe45813a5b0 0x7fe45806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.164+0000 7fe45dbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45806ddb0 con 0x7fe45806d980 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.164+0000 7fe45dbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4580c71b0 con 0x7fe45813a1e0 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.164+0000 7fe45dbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580c7490 con 0x7fe45813a5b0 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.164+0000 7fe456ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe45813a5b0 0x7fe45806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.164+0000 7fe456ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe45813a5b0 0x7fe45806d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48202/0 (socket says 172.21.15.31:48202) 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.164+0000 7fe457fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe45806d980 0x7fe458069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34086/0 (socket says 172.21.15.31:34086) 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.164+0000 7fe456ffd700 1 -- 172.21.15.31:0/2282820655 learned_addr learned my addr 172.21.15.31:0/2282820655 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.165+0000 7fe456ffd700 1 -- 172.21.15.31:0/2282820655 >> v1:172.21.15.62:6789/0 conn(0x7fe45813a1e0 legacy=0x7fe4580c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.165+0000 7fe4567fc700 1 -- 172.21.15.31:0/2282820655 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184889086 0 0) 0x7fe45806ddb0 con 0x7fe45806d980 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.165+0000 7fe456ffd700 1 -- 172.21.15.31:0/2282820655 >> v1:172.21.15.31:6789/0 conn(0x7fe45806d980 legacy=0x7fe458069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.165+0000 7fe456ffd700 1 -- 172.21.15.31:0/2282820655 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4580c7770 con 0x7fe45813a5b0 2023-10-06T05:17:06.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.165+0000 7fe456ffd700 1 --2- 172.21.15.31:0/2282820655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45813a5b0 0x7fe45806d440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0099b0 tx=0x7fe44c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c0006677b69f17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:06.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.165+0000 7fe4567fc700 1 -- 172.21.15.31:0/2282820655 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00b860 con 0x7fe45813a5b0 2023-10-06T05:17:06.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.165+0000 7fe4567fc700 1 -- 172.21.15.31:0/2282820655 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c00b9c0 con 0x7fe45813a5b0 2023-10-06T05:17:06.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.165+0000 7fe4567fc700 1 -- 172.21.15.31:0/2282820655 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00bd60 con 0x7fe45813a5b0 2023-10-06T05:17:06.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.166+0000 7fe45dbff700 1 -- 172.21.15.31:0/2282820655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45813a5b0 msgr2=0x7fe45806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.166+0000 7fe45dbff700 1 --2- 172.21.15.31:0/2282820655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45813a5b0 0x7fe45806d440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0099b0 tx=0x7fe44c009250 comp rx=0 tx=0).stop 2023-10-06T05:17:06.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.166+0000 7fe45dbff700 1 -- 172.21.15.31:0/2282820655 shutdown_connections 2023-10-06T05:17:06.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.166+0000 7fe45dbff700 1 --2- 172.21.15.31:0/2282820655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45813a5b0 0x7fe45806d440 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.166+0000 7fe45dbff700 1 -- 172.21.15.31:0/2282820655 >> 172.21.15.31:0/2282820655 conn(0x7fe4580cc0b0 msgr2=0x7fe4580cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:06.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.166+0000 7fe45dbff700 1 -- 172.21.15.31:0/2282820655 shutdown_connections 2023-10-06T05:17:06.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.166+0000 7fe45dbff700 1 -- 172.21.15.31:0/2282820655 wait complete. 2023-10-06T05:17:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.167+0000 7fe45dbff700 1 Processor -- start 2023-10-06T05:17:06.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.168+0000 7fe45dbff700 1 -- start start 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.168+0000 7fe45dbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45806d980 0x7fe4580c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.168+0000 7fe45dbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe45813a1e0 0x7fe458156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.168+0000 7fe45dbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe45813a5b0 0x7fe4581555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe45dbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe458150410 con 0x7fe45806d980 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe45dbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580c6e90 con 0x7fe45813a5b0 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe45dbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe458150ce0 con 0x7fe45813a1e0 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe4577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45806d980 0x7fe4580c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe4577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45806d980 0x7fe4580c6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48214/0 (socket says 172.21.15.31:48214) 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe4577fe700 1 -- 172.21.15.31:0/2722871451 learned_addr learned my addr 172.21.15.31:0/2722871451 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe457fff700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe45813a5b0 0x7fe4581555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe456ffd700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe45813a1e0 0x7fe458156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe4577fe700 1 -- 172.21.15.31:0/2722871451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe45813a1e0 msgr2=0x7fe458156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe4577fe700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe45813a1e0 0x7fe458156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe4577fe700 1 -- 172.21.15.31:0/2722871451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe45813a5b0 msgr2=0x7fe4581555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe4577fe700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe45813a5b0 0x7fe4581555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe4577fe700 1 -- 172.21.15.31:0/2722871451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c0094f0 con 0x7fe45806d980 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe457fff700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe45813a5b0 0x7fe4581555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe456ffd700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe45813a1e0 0x7fe458156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.169+0000 7fe4577fe700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45806d980 0x7fe4580c6950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c7c0 tx=0x7fe448007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.170+0000 7fe43f7fe700 1 -- 172.21.15.31:0/2722871451 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448011590 con 0x7fe45806d980 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.170+0000 7fe43f7fe700 1 -- 172.21.15.31:0/2722871451 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe448011710 con 0x7fe45806d980 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.170+0000 7fe43f7fe700 1 -- 172.21.15.31:0/2722871451 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448011ad0 con 0x7fe45806d980 2023-10-06T05:17:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.170+0000 7fe45dbff700 1 -- 172.21.15.31:0/2722871451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe458150fc0 con 0x7fe45806d980 2023-10-06T05:17:06.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.170+0000 7fe45dbff700 1 -- 172.21.15.31:0/2722871451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe458151580 con 0x7fe45806d980 2023-10-06T05:17:06.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.171+0000 7fe45dbff700 1 -- 172.21.15.31:0/2722871451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45804c380 con 0x7fe45806d980 2023-10-06T05:17:06.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.172+0000 7fe43f7fe700 1 -- 172.21.15.31:0/2722871451 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe44801b3b0 con 0x7fe45806d980 2023-10-06T05:17:06.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.173+0000 7fe43f7fe700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe444066f30 0x7fe4440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.173+0000 7fe43f7fe700 1 -- 172.21.15.31:0/2722871451 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe448083ba0 con 0x7fe45806d980 2023-10-06T05:17:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.173+0000 7fe456ffd700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe444066f30 0x7fe4440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.173+0000 7fe456ffd700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe444066f30 0x7fe4440693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe45813dba0 tx=0x7fe44c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.177+0000 7fe43f7fe700 1 -- 172.21.15.31:0/2722871451 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe448050d60 con 0x7fe45806d980 2023-10-06T05:17:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.380+0000 7fe45dbff700 1 -- 172.21.15.31:0/2722871451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4580c7a50 con 0x7fe45806d980 2023-10-06T05:17:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.381+0000 7fe43f7fe700 1 -- 172.21.15.31:0/2722871451 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fe448017070 con 0x7fe45806d980 2023-10-06T05:17:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:06.383 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:06.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 -- 172.21.15.31:0/2722871451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe444066f30 msgr2=0x7fe4440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe444066f30 0x7fe4440693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe45813dba0 tx=0x7fe44c017000 comp rx=0 tx=0).stop 2023-10-06T05:17:06.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 -- 172.21.15.31:0/2722871451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45806d980 msgr2=0x7fe4580c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45806d980 0x7fe4580c6950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c7c0 tx=0x7fe448007f70 comp rx=0 tx=0).stop 2023-10-06T05:17:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 -- 172.21.15.31:0/2722871451 shutdown_connections 2023-10-06T05:17:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe45813a5b0 0x7fe4581555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe444066f30 0x7fe4440693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe45806d980 0x7fe4580c6950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 --2- 172.21.15.31:0/2722871451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe45813a1e0 0x7fe458156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 -- 172.21.15.31:0/2722871451 >> 172.21.15.31:0/2722871451 conn(0x7fe4580cc0b0 msgr2=0x7fe458069460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 -- 172.21.15.31:0/2722871451 shutdown_connections 2023-10-06T05:17:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.383+0000 7fe45dbff700 1 -- 172.21.15.31:0/2722871451 wait complete. 2023-10-06T05:17:06.403 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:17:06.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:17:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.598+0000 7f4632bff700 1 Processor -- start 2023-10-06T05:17:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.598+0000 7f4632bff700 1 -- start start 2023-10-06T05:17:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f4632bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f462c1596d0 0x7f462c159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f4632bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f462c0c6f80 0x7f462c06a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f4632bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462c067f70 con 0x7f462c0c16e0 2023-10-06T05:17:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f4632bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462c06e040 con 0x7f462c1596d0 2023-10-06T05:17:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f4632bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c0681c0 con 0x7f462c0c6f80 2023-10-06T05:17:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f463099b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f462c1596d0 0x7f462c159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f463119c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f462c0c16e0 0x7f462c15d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34090/0 (socket says 172.21.15.31:34090) 2023-10-06T05:17:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f463119c700 1 -- 172.21.15.31:0/863798360 learned_addr learned my addr 172.21.15.31:0/863798360 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f462bfff700 1 --2- 172.21.15.31:0/863798360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f462c0c6f80 0x7f462c06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f462b7fe700 1 -- 172.21.15.31:0/863798360 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220001638 0 0) 0x7f462c067f70 con 0x7f462c0c16e0 2023-10-06T05:17:06.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f463099b700 1 -- 172.21.15.31:0/863798360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f462c0c6f80 msgr2=0x7f462c06a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:06.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.600+0000 7f463099b700 1 --2- 172.21.15.31:0/863798360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f462c0c6f80 0x7f462c06a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.601+0000 7f463099b700 1 -- 172.21.15.31:0/863798360 >> v1:172.21.15.31:6789/0 conn(0x7f462c0c16e0 legacy=0x7f462c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.601+0000 7f463099b700 1 -- 172.21.15.31:0/863798360 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c0684a0 con 0x7f462c1596d0 2023-10-06T05:17:06.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.601+0000 7f463099b700 1 --2- 172.21.15.31:0/863798360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 0x7f462c159aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46200098e0 tx=0x7f46200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f0ccf9a68039ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:06.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.601+0000 7f462b7fe700 1 -- 172.21.15.31:0/863798360 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000b640 con 0x7f462c1596d0 2023-10-06T05:17:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.601+0000 7f462b7fe700 1 -- 172.21.15.31:0/863798360 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462000b7a0 con 0x7f462c1596d0 2023-10-06T05:17:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.601+0000 7f462b7fe700 1 -- 172.21.15.31:0/863798360 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000bb40 con 0x7f462c1596d0 2023-10-06T05:17:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.601+0000 7f4632bff700 1 -- 172.21.15.31:0/863798360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 msgr2=0x7f462c159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.601+0000 7f4632bff700 1 --2- 172.21.15.31:0/863798360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 0x7f462c159aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46200098e0 tx=0x7f46200091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.602+0000 7f4632bff700 1 -- 172.21.15.31:0/863798360 shutdown_connections 2023-10-06T05:17:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.602+0000 7f4632bff700 1 --2- 172.21.15.31:0/863798360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 0x7f462c159aa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.602+0000 7f4632bff700 1 --2- 172.21.15.31:0/863798360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f462c0c6f80 0x7f462c06a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.602+0000 7f4632bff700 1 -- 172.21.15.31:0/863798360 >> 172.21.15.31:0/863798360 conn(0x7f462c0c9e80 msgr2=0x7f462c06a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.602+0000 7f4632bff700 1 -- 172.21.15.31:0/863798360 shutdown_connections 2023-10-06T05:17:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.602+0000 7f4632bff700 1 -- 172.21.15.31:0/863798360 wait complete. 2023-10-06T05:17:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.603+0000 7f4632bff700 1 Processor -- start 2023-10-06T05:17:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.603+0000 7f4632bff700 1 -- start start 2023-10-06T05:17:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.604+0000 7f4632bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f462c0c16e0 0x7f462c16be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.604+0000 7f4632bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c0c6f80 0x7f462c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.604+0000 7f4632bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 0x7f462c156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.604+0000 7f4632bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c0afa60 con 0x7f462c1596d0 2023-10-06T05:17:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.604+0000 7f4632bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c0afcb0 con 0x7f462c0c6f80 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.604+0000 7f4632bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c0aff90 con 0x7f462c0c16e0 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.604+0000 7f463119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 0x7f462c156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.604+0000 7f463119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 0x7f462c156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48236/0 (socket says 172.21.15.31:48236) 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.604+0000 7f463119c700 1 -- 172.21.15.31:0/3308986248 learned_addr learned my addr 172.21.15.31:0/3308986248 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f463099b700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f462c0c16e0 0x7f462c16be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f462bfff700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c0c6f80 0x7f462c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f463119c700 1 -- 172.21.15.31:0/3308986248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f462c0c16e0 msgr2=0x7f462c16be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f463119c700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f462c0c16e0 0x7f462c16be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f463119c700 1 -- 172.21.15.31:0/3308986248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c0c6f80 msgr2=0x7f462c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f463119c700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c0c6f80 0x7f462c15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f463119c700 1 -- 172.21.15.31:0/3308986248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620009420 con 0x7f462c1596d0 2023-10-06T05:17:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f463119c700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 0x7f462c156d70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f462400cbf0 tx=0x7f46240019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f4628ff9700 1 -- 172.21.15.31:0/3308986248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400a9c0 con 0x7f462c1596d0 2023-10-06T05:17:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f4628ff9700 1 -- 172.21.15.31:0/3308986248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462400ab20 con 0x7f462c1596d0 2023-10-06T05:17:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f4628ff9700 1 -- 172.21.15.31:0/3308986248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46240070f0 con 0x7f462c1596d0 2023-10-06T05:17:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.605+0000 7f4632bff700 1 -- 172.21.15.31:0/3308986248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c0b0270 con 0x7f462c1596d0 2023-10-06T05:17:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.606+0000 7f4632bff700 1 -- 172.21.15.31:0/3308986248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c16c3a0 con 0x7f462c1596d0 2023-10-06T05:17:06.610 INFO:tasks.ceph.mds.a.smithi031.stderr:2023-10-06T05:17:06.606+0000 7fcb381f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:17:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.607+0000 7f4632bff700 1 -- 172.21.15.31:0/3308986248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c04c380 con 0x7f462c1596d0 2023-10-06T05:17:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.607+0000 7f4628ff9700 1 -- 172.21.15.31:0/3308986248 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4624007520 con 0x7f462c1596d0 2023-10-06T05:17:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.608+0000 7f4628ff9700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4610066f30 0x7f46100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.608+0000 7f4628ff9700 1 -- 172.21.15.31:0/3308986248 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4624084b30 con 0x7f462c1596d0 2023-10-06T05:17:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.609+0000 7f463099b700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4610066f30 0x7f46100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.609+0000 7f463099b700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4610066f30 0x7f46100693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f46200093f0 tx=0x7f4620017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:06.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.612+0000 7f4628ff9700 1 -- 172.21.15.31:0/3308986248 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4624051c20 con 0x7f462c1596d0 2023-10-06T05:17:06.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.812+0000 7f4632bff700 1 -- 172.21.15.31:0/3308986248 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f462c0afe20 con 0x7f4610066f30 2023-10-06T05:17:06.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.822+0000 7f4628ff9700 1 -- 172.21.15.31:0/3308986248 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286934 (secure 0 0 0) 0x7f462c0afe20 con 0x7f4610066f30 2023-10-06T05:17:06.828 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:06.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.826+0000 7f4632bff700 1 -- 172.21.15.31:0/3308986248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4610066f30 msgr2=0x7f46100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.826+0000 7f4632bff700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4610066f30 0x7f46100693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f46200093f0 tx=0x7f4620017000 comp rx=0 tx=0).stop 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.826+0000 7f4632bff700 1 -- 172.21.15.31:0/3308986248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 msgr2=0x7f462c156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.826+0000 7f4632bff700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 0x7f462c156d70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f462400cbf0 tx=0x7f46240019b0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.827+0000 7f4632bff700 1 -- 172.21.15.31:0/3308986248 shutdown_connections 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.827+0000 7f4632bff700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4610066f30 0x7f46100693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.827+0000 7f4632bff700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f462c1596d0 0x7f462c156d70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.827+0000 7f4632bff700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f462c0c16e0 0x7f462c16be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.827+0000 7f4632bff700 1 --2- 172.21.15.31:0/3308986248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c0c6f80 0x7f462c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.827+0000 7f4632bff700 1 -- 172.21.15.31:0/3308986248 >> 172.21.15.31:0/3308986248 conn(0x7f462c0c9e80 msgr2=0x7f462c0c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.827+0000 7f4632bff700 1 -- 172.21.15.31:0/3308986248 shutdown_connections 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:06.827+0000 7f4632bff700 1 -- 172.21.15.31:0/3308986248 wait complete. 2023-10-06T05:17:06.831 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:17:06.846 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-06T05:17:06.061061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381068,"kb_used_data":2764,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593652,"statfs":{"total":773094113280,"available":770655899648,"internally_reserved":0,"allocated":2830336,"data_stored":1745615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.644982"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464636+0000","last_change":"2023-10-06T05:17:02.463452+0000","last_active":"2023-10-06T05:17:02.464636+0000","last_peered":"2023-10-06T05:17:02.464636+0000","last_clean":"2023-10-06T05:17:02.464636+0000","last_became_active":"2023-10-06T05:17:02.463139+0000","last_became_peered":"2023-10-06T05:17:02.463139+0000","last_unstale":"2023-10-06T05:17:02.464636+0000","last_undegraded":"2023-10-06T05:17:02.464636+0000","last_fullsized":"2023-10-06T05:17:02.464636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:52.286394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456371+0000","last_change":"2023-10-06T05:17:02.451303+0000","last_active":"2023-10-06T05:17:02.456371+0000","last_peered":"2023-10-06T05:17:02.456371+0000","last_clean":"2023-10-06T05:17:02.456371+0000","last_became_active":"2023-10-06T05:17:02.450985+0000","last_became_peered":"2023-10-06T05:17:02.450985+0000","last_unstale":"2023-10-06T05:17:02.456371+0000","last_undegraded":"2023-10-06T05:17:02.456371+0000","last_fullsized":"2023-10-06T05:17:02.456371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453158+0000","last_change":"2023-10-06T05:17:02.451335+0000","last_active":"2023-10-06T05:17:02.453158+0000","last_peered":"2023-10-06T05:17:02.453158+0000","last_clean":"2023-10-06T05:17:02.453158+0000","last_became_active":"2023-10-06T05:17:02.451119+0000","last_became_peered":"2023-10-06T05:17:02.451119+0000","last_unstale":"2023-10-06T05:17:02.453158+0000","last_undegraded":"2023-10-06T05:17:02.453158+0000","last_fullsized":"2023-10-06T05:17:02.453158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:59.439667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.458136+0000","last_change":"2023-10-06T05:17:02.454559+0000","last_active":"2023-10-06T05:17:02.458136+0000","last_peered":"2023-10-06T05:17:02.458136+0000","last_clean":"2023-10-06T05:17:02.458136+0000","last_became_active":"2023-10-06T05:17:02.453985+0000","last_became_peered":"2023-10-06T05:17:02.453985+0000","last_unstale":"2023-10-06T05:17:02.458136+0000","last_undegraded":"2023-10-06T05:17:02.458136+0000","last_fullsized":"2023-10-06T05:17:02.458136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:57.117094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465742+0000","last_change":"2023-10-06T05:17:02.460869+0000","last_active":"2023-10-06T05:17:02.465742+0000","last_peered":"2023-10-06T05:17:02.465742+0000","last_clean":"2023-10-06T05:17:02.465742+0000","last_became_active":"2023-10-06T05:17:02.460257+0000","last_became_peered":"2023-10-06T05:17:02.460257+0000","last_unstale":"2023-10-06T05:17:02.465742+0000","last_undegraded":"2023-10-06T05:17:02.465742+0000","last_fullsized":"2023-10-06T05:17:02.465742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453122+0000","last_change":"2023-10-06T05:17:02.451827+0000","last_active":"2023-10-06T05:17:02.453122+0000","last_peered":"2023-10-06T05:17:02.453122+0000","last_clean":"2023-10-06T05:17:02.453122+0000","last_became_active":"2023-10-06T05:17:02.451573+0000","last_became_peered":"2023-10-06T05:17:02.451573+0000","last_unstale":"2023-10-06T05:17:02.453122+0000","last_undegraded":"2023-10-06T05:17:02.453122+0000","last_fullsized":"2023-10-06T05:17:02.453122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:50.530278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.454214+0000","last_change":"2023-10-06T05:17:02.451805+0000","last_active":"2023-10-06T05:17:02.454214+0000","last_peered":"2023-10-06T05:17:02.454214+0000","last_clean":"2023-10-06T05:17:02.454214+0000","last_became_active":"2023-10-06T05:17:02.451504+0000","last_became_peered":"2023-10-06T05:17:02.451504+0000","last_unstale":"2023-10-06T05:17:02.454214+0000","last_undegraded":"2023-10-06T05:17:02.454214+0000","last_fullsized":"2023-10-06T05:17:02.454214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448446+0000","last_change":"2023-10-06T05:17:02.440866+0000","last_active":"2023-10-06T05:17:02.448446+0000","last_peered":"2023-10-06T05:17:02.448446+0000","last_clean":"2023-10-06T05:17:02.448446+0000","last_became_active":"2023-10-06T05:17:02.440548+0000","last_became_peered":"2023-10-06T05:17:02.440548+0000","last_unstale":"2023-10-06T05:17:02.448446+0000","last_undegraded":"2023-10-06T05:17:02.448446+0000","last_fullsized":"2023-10-06T05:17:02.448446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.466997+0000","last_change":"2023-10-06T05:17:02.461715+0000","last_active":"2023-10-06T05:17:02.466997+0000","last_peered":"2023-10-06T05:17:02.466997+0000","last_clean":"2023-10-06T05:17:02.466997+0000","last_became_active":"2023-10-06T05:17:02.461152+0000","last_became_peered":"2023-10-06T05:17:02.461152+0000","last_unstale":"2023-10-06T05:17:02.466997+0000","last_undegraded":"2023-10-06T05:17:02.466997+0000","last_fullsized":"2023-10-06T05:17:02.466997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453073+0000","last_change":"2023-10-06T05:17:02.448948+0000","last_active":"2023-10-06T05:17:02.453073+0000","last_peered":"2023-10-06T05:17:02.453073+0000","last_clean":"2023-10-06T05:17:02.453073+0000","last_became_active":"2023-10-06T05:17:02.448524+0000","last_became_peered":"2023-10-06T05:17:02.448524+0000","last_unstale":"2023-10-06T05:17:02.453073+0000","last_undegraded":"2023-10-06T05:17:02.453073+0000","last_fullsized":"2023-10-06T05:17:02.453073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452570+0000","last_change":"2023-10-06T05:17:02.446198+0000","last_active":"2023-10-06T05:17:02.452570+0000","last_peered":"2023-10-06T05:17:02.452570+0000","last_clean":"2023-10-06T05:17:02.452570+0000","last_became_active":"2023-10-06T05:17:02.445918+0000","last_became_peered":"2023-10-06T05:17:02.445918+0000","last_unstale":"2023-10-06T05:17:02.452570+0000","last_undegraded":"2023-10-06T05:17:02.452570+0000","last_fullsized":"2023-10-06T05:17:02.452570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:45.361281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450359+0000","last_change":"2023-10-06T05:17:02.447094+0000","last_active":"2023-10-06T05:17:02.450359+0000","last_peered":"2023-10-06T05:17:02.450359+0000","last_clean":"2023-10-06T05:17:02.450359+0000","last_became_active":"2023-10-06T05:17:02.446800+0000","last_became_peered":"2023-10-06T05:17:02.446800+0000","last_unstale":"2023-10-06T05:17:02.450359+0000","last_undegraded":"2023-10-06T05:17:02.450359+0000","last_fullsized":"2023-10-06T05:17:02.450359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:46.452504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464672+0000","last_change":"2023-10-06T05:17:02.463414+0000","last_active":"2023-10-06T05:17:02.464672+0000","last_peered":"2023-10-06T05:17:02.464672+0000","last_clean":"2023-10-06T05:17:02.464672+0000","last_became_active":"2023-10-06T05:17:02.463125+0000","last_became_peered":"2023-10-06T05:17:02.463125+0000","last_unstale":"2023-10-06T05:17:02.464672+0000","last_undegraded":"2023-10-06T05:17:02.464672+0000","last_fullsized":"2023-10-06T05:17:02.464672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:29.979974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450580+0000","last_change":"2023-10-06T05:17:02.447113+0000","last_active":"2023-10-06T05:17:02.450580+0000","last_peered":"2023-10-06T05:17:02.450580+0000","last_clean":"2023-10-06T05:17:02.450580+0000","last_became_active":"2023-10-06T05:17:02.446878+0000","last_became_peered":"2023-10-06T05:17:02.446878+0000","last_unstale":"2023-10-06T05:17:02.450580+0000","last_undegraded":"2023-10-06T05:17:02.450580+0000","last_fullsized":"2023-10-06T05:17:02.450580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.454233+0000","last_change":"2023-10-06T05:17:02.451949+0000","last_active":"2023-10-06T05:17:02.454233+0000","last_peered":"2023-10-06T05:17:02.454233+0000","last_clean":"2023-10-06T05:17:02.454233+0000","last_became_active":"2023-10-06T05:17:02.451640+0000","last_became_peered":"2023-10-06T05:17:02.451640+0000","last_unstale":"2023-10-06T05:17:02.454233+0000","last_undegraded":"2023-10-06T05:17:02.454233+0000","last_fullsized":"2023-10-06T05:17:02.454233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452479+0000","last_change":"2023-10-06T05:17:02.445441+0000","last_active":"2023-10-06T05:17:02.452479+0000","last_peered":"2023-10-06T05:17:02.452479+0000","last_clean":"2023-10-06T05:17:02.452479+0000","last_became_active":"2023-10-06T05:17:02.445132+0000","last_became_peered":"2023-10-06T05:17:02.445132+0000","last_unstale":"2023-10-06T05:17:02.452479+0000","last_undegraded":"2023-10-06T05:17:02.452479+0000","last_fullsized":"2023-10-06T05:17:02.452479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:10.259711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464487+0000","last_change":"2023-10-06T05:17:02.461554+0000","last_active":"2023-10-06T05:17:02.464487+0000","last_peered":"2023-10-06T05:17:02.464487+0000","last_clean":"2023-10-06T05:17:02.464487+0000","last_became_active":"2023-10-06T05:17:02.460803+0000","last_became_peered":"2023-10-06T05:17:02.460803+0000","last_unstale":"2023-10-06T05:17:02.464487+0000","last_undegraded":"2023-10-06T05:17:02.464487+0000","last_fullsized":"2023-10-06T05:17:02.464487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452431+0000","last_change":"2023-10-06T05:17:02.448249+0000","last_active":"2023-10-06T05:17:02.452431+0000","last_peered":"2023-10-06T05:17:02.452431+0000","last_clean":"2023-10-06T05:17:02.452431+0000","last_became_active":"2023-10-06T05:17:02.447977+0000","last_became_peered":"2023-10-06T05:17:02.447977+0000","last_unstale":"2023-10-06T05:17:02.452431+0000","last_undegraded":"2023-10-06T05:17:02.452431+0000","last_fullsized":"2023-10-06T05:17:02.452431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:10.643350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456974+0000","last_change":"2023-10-06T05:17:02.451653+0000","last_active":"2023-10-06T05:17:02.456974+0000","last_peered":"2023-10-06T05:17:02.456974+0000","last_clean":"2023-10-06T05:17:02.456974+0000","last_became_active":"2023-10-06T05:17:02.451347+0000","last_became_peered":"2023-10-06T05:17:02.451347+0000","last_unstale":"2023-10-06T05:17:02.456974+0000","last_undegraded":"2023-10-06T05:17:02.456974+0000","last_fullsized":"2023-10-06T05:17:02.456974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:20.132880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452508+0000","last_change":"2023-10-06T05:17:02.446775+0000","last_active":"2023-10-06T05:17:02.452508+0000","last_peered":"2023-10-06T05:17:02.452508+0000","last_clean":"2023-10-06T05:17:02.452508+0000","last_became_active":"2023-10-06T05:17:02.446482+0000","last_became_peered":"2023-10-06T05:17:02.446482+0000","last_unstale":"2023-10-06T05:17:02.452508+0000","last_undegraded":"2023-10-06T05:17:02.452508+0000","last_fullsized":"2023-10-06T05:17:02.452508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:14.399779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468275+0000","last_change":"2023-10-06T05:17:02.463237+0000","last_active":"2023-10-06T05:17:02.468275+0000","last_peered":"2023-10-06T05:17:02.468275+0000","last_clean":"2023-10-06T05:17:02.468275+0000","last_became_active":"2023-10-06T05:17:02.462891+0000","last_became_peered":"2023-10-06T05:17:02.462891+0000","last_unstale":"2023-10-06T05:17:02.468275+0000","last_undegraded":"2023-10-06T05:17:02.468275+0000","last_fullsized":"2023-10-06T05:17:02.468275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462403+0000","last_change":"2023-10-06T05:17:02.458157+0000","last_active":"2023-10-06T05:17:02.462403+0000","last_peered":"2023-10-06T05:17:02.462403+0000","last_clean":"2023-10-06T05:17:02.462403+0000","last_became_active":"2023-10-06T05:17:02.455535+0000","last_became_peered":"2023-10-06T05:17:02.455535+0000","last_unstale":"2023-10-06T05:17:02.462403+0000","last_undegraded":"2023-10-06T05:17:02.462403+0000","last_fullsized":"2023-10-06T05:17:02.462403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461887+0000","last_change":"2023-10-06T05:17:02.449068+0000","last_active":"2023-10-06T05:17:02.461887+0000","last_peered":"2023-10-06T05:17:02.461887+0000","last_clean":"2023-10-06T05:17:02.461887+0000","last_became_active":"2023-10-06T05:17:02.448751+0000","last_became_peered":"2023-10-06T05:17:02.448751+0000","last_unstale":"2023-10-06T05:17:02.461887+0000","last_undegraded":"2023-10-06T05:17:02.461887+0000","last_fullsized":"2023-10-06T05:17:02.461887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468443+0000","last_change":"2023-10-06T05:17:02.465489+0000","last_active":"2023-10-06T05:17:02.468443+0000","last_peered":"2023-10-06T05:17:02.468443+0000","last_clean":"2023-10-06T05:17:02.468443+0000","last_became_active":"2023-10-06T05:17:02.465329+0000","last_became_peered":"2023-10-06T05:17:02.465329+0000","last_unstale":"2023-10-06T05:17:02.468443+0000","last_undegraded":"2023-10-06T05:17:02.468443+0000","last_fullsized":"2023-10-06T05:17:02.468443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453337+0000","last_change":"2023-10-06T05:17:02.444229+0000","last_active":"2023-10-06T05:17:02.453337+0000","last_peered":"2023-10-06T05:17:02.453337+0000","last_clean":"2023-10-06T05:17:02.453337+0000","last_became_active":"2023-10-06T05:17:02.444064+0000","last_became_peered":"2023-10-06T05:17:02.444064+0000","last_unstale":"2023-10-06T05:17:02.453337+0000","last_undegraded":"2023-10-06T05:17:02.453337+0000","last_fullsized":"2023-10-06T05:17:02.453337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:39.861351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453265+0000","last_change":"2023-10-06T05:17:02.447686+0000","last_active":"2023-10-06T05:17:02.453265+0000","last_peered":"2023-10-06T05:17:02.453265+0000","last_clean":"2023-10-06T05:17:02.453265+0000","last_became_active":"2023-10-06T05:17:02.447364+0000","last_became_peered":"2023-10-06T05:17:02.447364+0000","last_unstale":"2023-10-06T05:17:02.453265+0000","last_undegraded":"2023-10-06T05:17:02.453265+0000","last_fullsized":"2023-10-06T05:17:02.453265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448537+0000","last_change":"2023-10-06T05:17:02.442095+0000","last_active":"2023-10-06T05:17:02.448537+0000","last_peered":"2023-10-06T05:17:02.448537+0000","last_clean":"2023-10-06T05:17:02.448537+0000","last_became_active":"2023-10-06T05:17:02.441645+0000","last_became_peered":"2023-10-06T05:17:02.441645+0000","last_unstale":"2023-10-06T05:17:02.448537+0000","last_undegraded":"2023-10-06T05:17:02.448537+0000","last_fullsized":"2023-10-06T05:17:02.448537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461441+0000","last_change":"2023-10-06T05:17:01.449274+0000","last_active":"2023-10-06T05:17:02.461441+0000","last_peered":"2023-10-06T05:17:02.461441+0000","last_clean":"2023-10-06T05:17:02.461441+0000","last_became_active":"2023-10-06T05:17:01.448908+0000","last_became_peered":"2023-10-06T05:17:01.448908+0000","last_unstale":"2023-10-06T05:17:02.461441+0000","last_undegraded":"2023-10-06T05:17:02.461441+0000","last_fullsized":"2023-10-06T05:17:02.461441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462280+0000","last_change":"2023-10-06T05:17:02.443949+0000","last_active":"2023-10-06T05:17:02.462280+0000","last_peered":"2023-10-06T05:17:02.462280+0000","last_clean":"2023-10-06T05:17:02.462280+0000","last_became_active":"2023-10-06T05:17:02.443677+0000","last_became_peered":"2023-10-06T05:17:02.443677+0000","last_unstale":"2023-10-06T05:17:02.462280+0000","last_undegraded":"2023-10-06T05:17:02.462280+0000","last_fullsized":"2023-10-06T05:17:02.462280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428951+0000","last_change":"2023-10-06T05:17:01.438479+0000","last_active":"2023-10-06T05:17:02.428951+0000","last_peered":"2023-10-06T05:17:02.428951+0000","last_clean":"2023-10-06T05:17:02.428951+0000","last_became_active":"2023-10-06T05:17:01.438182+0000","last_became_peered":"2023-10-06T05:17:01.438182+0000","last_unstale":"2023-10-06T05:17:02.428951+0000","last_undegraded":"2023-10-06T05:17:02.428951+0000","last_fullsized":"2023-10-06T05:17:02.428951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:42.243614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453470+0000","last_change":"2023-10-06T05:17:02.447288+0000","last_active":"2023-10-06T05:17:02.453470+0000","last_peered":"2023-10-06T05:17:02.453470+0000","last_clean":"2023-10-06T05:17:02.453470+0000","last_became_active":"2023-10-06T05:17:02.446914+0000","last_became_peered":"2023-10-06T05:17:02.446914+0000","last_unstale":"2023-10-06T05:17:02.453470+0000","last_undegraded":"2023-10-06T05:17:02.453470+0000","last_fullsized":"2023-10-06T05:17:02.453470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428771+0000","last_change":"2023-10-06T05:17:01.444309+0000","last_active":"2023-10-06T05:17:02.428771+0000","last_peered":"2023-10-06T05:17:02.428771+0000","last_clean":"2023-10-06T05:17:02.428771+0000","last_became_active":"2023-10-06T05:17:01.444026+0000","last_became_peered":"2023-10-06T05:17:01.444026+0000","last_unstale":"2023-10-06T05:17:02.428771+0000","last_undegraded":"2023-10-06T05:17:02.428771+0000","last_fullsized":"2023-10-06T05:17:02.428771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.458208+0000","last_change":"2023-10-06T05:17:02.455822+0000","last_active":"2023-10-06T05:17:02.458208+0000","last_peered":"2023-10-06T05:17:02.458208+0000","last_clean":"2023-10-06T05:17:02.458208+0000","last_became_active":"2023-10-06T05:17:02.455539+0000","last_became_peered":"2023-10-06T05:17:02.455539+0000","last_unstale":"2023-10-06T05:17:02.458208+0000","last_undegraded":"2023-10-06T05:17:02.458208+0000","last_fullsized":"2023-10-06T05:17:02.458208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434617+0000","last_change":"2023-10-06T05:17:01.453079+0000","last_active":"2023-10-06T05:17:02.434617+0000","last_peered":"2023-10-06T05:17:02.434617+0000","last_clean":"2023-10-06T05:17:02.434617+0000","last_became_active":"2023-10-06T05:17:01.452800+0000","last_became_peered":"2023-10-06T05:17:01.452800+0000","last_unstale":"2023-10-06T05:17:02.434617+0000","last_undegraded":"2023-10-06T05:17:02.434617+0000","last_fullsized":"2023-10-06T05:17:02.434617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.457891+0000","last_change":"2023-10-06T05:17:02.439162+0000","last_active":"2023-10-06T05:17:02.457891+0000","last_peered":"2023-10-06T05:17:02.457891+0000","last_clean":"2023-10-06T05:17:02.457891+0000","last_became_active":"2023-10-06T05:17:02.438810+0000","last_became_peered":"2023-10-06T05:17:02.438810+0000","last_unstale":"2023-10-06T05:17:02.457891+0000","last_undegraded":"2023-10-06T05:17:02.457891+0000","last_fullsized":"2023-10-06T05:17:02.457891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435781+0000","last_change":"2023-10-06T05:17:01.449427+0000","last_active":"2023-10-06T05:17:02.435781+0000","last_peered":"2023-10-06T05:17:02.435781+0000","last_clean":"2023-10-06T05:17:02.435781+0000","last_became_active":"2023-10-06T05:17:01.449100+0000","last_became_peered":"2023-10-06T05:17:01.449100+0000","last_unstale":"2023-10-06T05:17:02.435781+0000","last_undegraded":"2023-10-06T05:17:02.435781+0000","last_fullsized":"2023-10-06T05:17:02.435781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461821+0000","last_change":"2023-10-06T05:17:02.441035+0000","last_active":"2023-10-06T05:17:02.461821+0000","last_peered":"2023-10-06T05:17:02.461821+0000","last_clean":"2023-10-06T05:17:02.461821+0000","last_became_active":"2023-10-06T05:17:02.440422+0000","last_became_peered":"2023-10-06T05:17:02.440422+0000","last_unstale":"2023-10-06T05:17:02.461821+0000","last_undegraded":"2023-10-06T05:17:02.461821+0000","last_fullsized":"2023-10-06T05:17:02.461821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468652+0000","last_change":"2023-10-06T05:17:01.445468+0000","last_active":"2023-10-06T05:17:02.468652+0000","last_peered":"2023-10-06T05:17:02.468652+0000","last_clean":"2023-10-06T05:17:02.468652+0000","last_became_active":"2023-10-06T05:17:01.445179+0000","last_became_peered":"2023-10-06T05:17:01.445179+0000","last_unstale":"2023-10-06T05:17:02.468652+0000","last_undegraded":"2023-10-06T05:17:02.468652+0000","last_fullsized":"2023-10-06T05:17:02.468652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:01.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453479+0000","last_change":"2023-10-06T05:17:02.444872+0000","last_active":"2023-10-06T05:17:02.453479+0000","last_peered":"2023-10-06T05:17:02.453479+0000","last_clean":"2023-10-06T05:17:02.453479+0000","last_became_active":"2023-10-06T05:17:02.444565+0000","last_became_peered":"2023-10-06T05:17:02.444565+0000","last_unstale":"2023-10-06T05:17:02.453479+0000","last_undegraded":"2023-10-06T05:17:02.453479+0000","last_fullsized":"2023-10-06T05:17:02.453479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.470982+0000","last_change":"2023-10-06T05:17:01.448641+0000","last_active":"2023-10-06T05:17:02.470982+0000","last_peered":"2023-10-06T05:17:02.470982+0000","last_clean":"2023-10-06T05:17:02.470982+0000","last_became_active":"2023-10-06T05:17:01.448232+0000","last_became_peered":"2023-10-06T05:17:01.448232+0000","last_unstale":"2023-10-06T05:17:02.470982+0000","last_undegraded":"2023-10-06T05:17:02.470982+0000","last_fullsized":"2023-10-06T05:17:02.470982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447551+0000","last_change":"2023-10-06T05:17:02.443125+0000","last_active":"2023-10-06T05:17:02.447551+0000","last_peered":"2023-10-06T05:17:02.447551+0000","last_clean":"2023-10-06T05:17:02.447551+0000","last_became_active":"2023-10-06T05:17:02.442847+0000","last_became_peered":"2023-10-06T05:17:02.442847+0000","last_unstale":"2023-10-06T05:17:02.447551+0000","last_undegraded":"2023-10-06T05:17:02.447551+0000","last_fullsized":"2023-10-06T05:17:02.447551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471314+0000","last_change":"2023-10-06T05:17:01.451184+0000","last_active":"2023-10-06T05:17:02.471314+0000","last_peered":"2023-10-06T05:17:02.471314+0000","last_clean":"2023-10-06T05:17:02.471314+0000","last_became_active":"2023-10-06T05:17:01.450521+0000","last_became_peered":"2023-10-06T05:17:01.450521+0000","last_unstale":"2023-10-06T05:17:02.471314+0000","last_undegraded":"2023-10-06T05:17:02.471314+0000","last_fullsized":"2023-10-06T05:17:02.471314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:56.109170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462307+0000","last_change":"2023-10-06T05:17:02.444967+0000","last_active":"2023-10-06T05:17:02.462307+0000","last_peered":"2023-10-06T05:17:02.462307+0000","last_clean":"2023-10-06T05:17:02.462307+0000","last_became_active":"2023-10-06T05:17:02.444639+0000","last_became_peered":"2023-10-06T05:17:02.444639+0000","last_unstale":"2023-10-06T05:17:02.462307+0000","last_undegraded":"2023-10-06T05:17:02.462307+0000","last_fullsized":"2023-10-06T05:17:02.462307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:59.439667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.439263+0000","last_change":"2023-10-06T05:17:01.446625+0000","last_active":"2023-10-06T05:17:02.439263+0000","last_peered":"2023-10-06T05:17:02.439263+0000","last_clean":"2023-10-06T05:17:02.439263+0000","last_became_active":"2023-10-06T05:17:01.446249+0000","last_became_peered":"2023-10-06T05:17:01.446249+0000","last_unstale":"2023-10-06T05:17:02.439263+0000","last_undegraded":"2023-10-06T05:17:02.439263+0000","last_fullsized":"2023-10-06T05:17:02.439263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448517+0000","last_change":"2023-10-06T05:17:02.441887+0000","last_active":"2023-10-06T05:17:02.448517+0000","last_peered":"2023-10-06T05:17:02.448517+0000","last_clean":"2023-10-06T05:17:02.448517+0000","last_became_active":"2023-10-06T05:17:02.441377+0000","last_became_peered":"2023-10-06T05:17:02.441377+0000","last_unstale":"2023-10-06T05:17:02.448517+0000","last_undegraded":"2023-10-06T05:17:02.448517+0000","last_fullsized":"2023-10-06T05:17:02.448517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434243+0000","last_change":"2023-10-06T05:17:01.450424+0000","last_active":"2023-10-06T05:17:02.434243+0000","last_peered":"2023-10-06T05:17:02.434243+0000","last_clean":"2023-10-06T05:17:02.434243+0000","last_became_active":"2023-10-06T05:17:01.450072+0000","last_became_peered":"2023-10-06T05:17:01.450072+0000","last_unstale":"2023-10-06T05:17:02.434243+0000","last_undegraded":"2023-10-06T05:17:02.434243+0000","last_fullsized":"2023-10-06T05:17:02.434243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:19.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464457+0000","last_change":"2023-10-06T05:17:02.460678+0000","last_active":"2023-10-06T05:17:02.464457+0000","last_peered":"2023-10-06T05:17:02.464457+0000","last_clean":"2023-10-06T05:17:02.464457+0000","last_became_active":"2023-10-06T05:17:02.460159+0000","last_became_peered":"2023-10-06T05:17:02.460159+0000","last_unstale":"2023-10-06T05:17:02.464457+0000","last_undegraded":"2023-10-06T05:17:02.464457+0000","last_fullsized":"2023-10-06T05:17:02.464457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.467405+0000","last_change":"2023-10-06T05:17:01.447124+0000","last_active":"2023-10-06T05:17:02.467405+0000","last_peered":"2023-10-06T05:17:02.467405+0000","last_clean":"2023-10-06T05:17:02.467405+0000","last_became_active":"2023-10-06T05:17:01.446810+0000","last_became_peered":"2023-10-06T05:17:01.446810+0000","last_unstale":"2023-10-06T05:17:02.467405+0000","last_undegraded":"2023-10-06T05:17:02.467405+0000","last_fullsized":"2023-10-06T05:17:02.467405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462426+0000","last_change":"2023-10-06T05:17:02.440973+0000","last_active":"2023-10-06T05:17:02.462426+0000","last_peered":"2023-10-06T05:17:02.462426+0000","last_clean":"2023-10-06T05:17:02.462426+0000","last_became_active":"2023-10-06T05:17:02.440594+0000","last_became_peered":"2023-10-06T05:17:02.440594+0000","last_unstale":"2023-10-06T05:17:02.462426+0000","last_undegraded":"2023-10-06T05:17:02.462426+0000","last_fullsized":"2023-10-06T05:17:02.462426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:45.361281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471211+0000","last_change":"2023-10-06T05:17:01.454983+0000","last_active":"2023-10-06T05:17:02.471211+0000","last_peered":"2023-10-06T05:17:02.471211+0000","last_clean":"2023-10-06T05:17:02.471211+0000","last_became_active":"2023-10-06T05:17:01.454580+0000","last_became_peered":"2023-10-06T05:17:01.454580+0000","last_unstale":"2023-10-06T05:17:02.471211+0000","last_undegraded":"2023-10-06T05:17:02.471211+0000","last_fullsized":"2023-10-06T05:17:02.471211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:53.875873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448468+0000","last_change":"2023-10-06T05:17:02.442909+0000","last_active":"2023-10-06T05:17:02.448468+0000","last_peered":"2023-10-06T05:17:02.448468+0000","last_clean":"2023-10-06T05:17:02.448468+0000","last_became_active":"2023-10-06T05:17:02.442619+0000","last_became_peered":"2023-10-06T05:17:02.442619+0000","last_unstale":"2023-10-06T05:17:02.448468+0000","last_undegraded":"2023-10-06T05:17:02.448468+0000","last_fullsized":"2023-10-06T05:17:02.448468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471248+0000","last_change":"2023-10-06T05:17:01.454784+0000","last_active":"2023-10-06T05:17:02.471248+0000","last_peered":"2023-10-06T05:17:02.471248+0000","last_clean":"2023-10-06T05:17:02.471248+0000","last_became_active":"2023-10-06T05:17:01.454498+0000","last_became_peered":"2023-10-06T05:17:01.454498+0000","last_unstale":"2023-10-06T05:17:02.471248+0000","last_undegraded":"2023-10-06T05:17:02.471248+0000","last_fullsized":"2023-10-06T05:17:02.471248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:41.021745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452621+0000","last_change":"2023-10-06T05:17:02.446992+0000","last_active":"2023-10-06T05:17:02.452621+0000","last_peered":"2023-10-06T05:17:02.452621+0000","last_clean":"2023-10-06T05:17:02.452621+0000","last_became_active":"2023-10-06T05:17:02.445148+0000","last_became_peered":"2023-10-06T05:17:02.445148+0000","last_unstale":"2023-10-06T05:17:02.452621+0000","last_undegraded":"2023-10-06T05:17:02.452621+0000","last_fullsized":"2023-10-06T05:17:02.452621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:31.781450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.469131+0000","last_change":"2023-10-06T05:17:01.446494+0000","last_active":"2023-10-06T05:17:02.469131+0000","last_peered":"2023-10-06T05:17:02.469131+0000","last_clean":"2023-10-06T05:17:02.469131+0000","last_became_active":"2023-10-06T05:17:01.446164+0000","last_became_peered":"2023-10-06T05:17:01.446164+0000","last_unstale":"2023-10-06T05:17:02.469131+0000","last_undegraded":"2023-10-06T05:17:02.469131+0000","last_fullsized":"2023-10-06T05:17:02.469131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:45.078207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456419+0000","last_change":"2023-10-06T05:17:02.451322+0000","last_active":"2023-10-06T05:17:02.456419+0000","last_peered":"2023-10-06T05:17:02.456419+0000","last_clean":"2023-10-06T05:17:02.456419+0000","last_became_active":"2023-10-06T05:17:02.451064+0000","last_became_peered":"2023-10-06T05:17:02.451064+0000","last_unstale":"2023-10-06T05:17:02.456419+0000","last_undegraded":"2023-10-06T05:17:02.456419+0000","last_fullsized":"2023-10-06T05:17:02.456419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:46.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.432653+0000","last_change":"2023-10-06T05:17:01.443417+0000","last_active":"2023-10-06T05:17:02.432653+0000","last_peered":"2023-10-06T05:17:02.432653+0000","last_clean":"2023-10-06T05:17:02.432653+0000","last_became_active":"2023-10-06T05:17:01.443114+0000","last_became_peered":"2023-10-06T05:17:01.443114+0000","last_unstale":"2023-10-06T05:17:02.432653+0000","last_undegraded":"2023-10-06T05:17:02.432653+0000","last_fullsized":"2023-10-06T05:17:02.432653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461876+0000","last_change":"2023-10-06T05:17:02.456471+0000","last_active":"2023-10-06T05:17:02.461876+0000","last_peered":"2023-10-06T05:17:02.461876+0000","last_clean":"2023-10-06T05:17:02.461876+0000","last_became_active":"2023-10-06T05:17:02.456142+0000","last_became_peered":"2023-10-06T05:17:02.456142+0000","last_unstale":"2023-10-06T05:17:02.461876+0000","last_undegraded":"2023-10-06T05:17:02.461876+0000","last_fullsized":"2023-10-06T05:17:02.461876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.466429+0000","last_change":"2023-10-06T05:17:01.444030+0000","last_active":"2023-10-06T05:17:02.466429+0000","last_peered":"2023-10-06T05:17:02.466429+0000","last_clean":"2023-10-06T05:17:02.466429+0000","last_became_active":"2023-10-06T05:17:01.443754+0000","last_became_peered":"2023-10-06T05:17:01.443754+0000","last_unstale":"2023-10-06T05:17:02.466429+0000","last_undegraded":"2023-10-06T05:17:02.466429+0000","last_fullsized":"2023-10-06T05:17:02.466429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.440368+0000","last_change":"2023-10-06T05:17:02.437195+0000","last_active":"2023-10-06T05:17:02.440368+0000","last_peered":"2023-10-06T05:17:02.440368+0000","last_clean":"2023-10-06T05:17:02.440368+0000","last_became_active":"2023-10-06T05:17:02.436917+0000","last_became_peered":"2023-10-06T05:17:02.436917+0000","last_unstale":"2023-10-06T05:17:02.440368+0000","last_undegraded":"2023-10-06T05:17:02.440368+0000","last_fullsized":"2023-10-06T05:17:02.440368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435134+0000","last_change":"2023-10-06T05:17:01.454676+0000","last_active":"2023-10-06T05:17:02.435134+0000","last_peered":"2023-10-06T05:17:02.435134+0000","last_clean":"2023-10-06T05:17:02.435134+0000","last_became_active":"2023-10-06T05:17:01.454385+0000","last_became_peered":"2023-10-06T05:17:01.454385+0000","last_unstale":"2023-10-06T05:17:02.435134+0000","last_undegraded":"2023-10-06T05:17:02.435134+0000","last_fullsized":"2023-10-06T05:17:02.435134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453140+0000","last_change":"2023-10-06T05:17:02.449388+0000","last_active":"2023-10-06T05:17:02.453140+0000","last_peered":"2023-10-06T05:17:02.453140+0000","last_clean":"2023-10-06T05:17:02.453140+0000","last_became_active":"2023-10-06T05:17:02.448868+0000","last_became_peered":"2023-10-06T05:17:02.448868+0000","last_unstale":"2023-10-06T05:17:02.453140+0000","last_undegraded":"2023-10-06T05:17:02.453140+0000","last_fullsized":"2023-10-06T05:17:02.453140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428450+0000","last_change":"2023-10-06T05:17:01.448698+0000","last_active":"2023-10-06T05:17:02.428450+0000","last_peered":"2023-10-06T05:17:02.428450+0000","last_clean":"2023-10-06T05:17:02.428450+0000","last_became_active":"2023-10-06T05:17:01.448084+0000","last_became_peered":"2023-10-06T05:17:01.448084+0000","last_unstale":"2023-10-06T05:17:02.428450+0000","last_undegraded":"2023-10-06T05:17:02.428450+0000","last_fullsized":"2023-10-06T05:17:02.428450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452412+0000","last_change":"2023-10-06T05:17:02.446522+0000","last_active":"2023-10-06T05:17:02.452412+0000","last_peered":"2023-10-06T05:17:02.452412+0000","last_clean":"2023-10-06T05:17:02.452412+0000","last_became_active":"2023-10-06T05:17:02.446203+0000","last_became_peered":"2023-10-06T05:17:02.446203+0000","last_unstale":"2023-10-06T05:17:02.452412+0000","last_undegraded":"2023-10-06T05:17:02.452412+0000","last_fullsized":"2023-10-06T05:17:02.452412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:28.806289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428705+0000","last_change":"2023-10-06T05:17:01.441195+0000","last_active":"2023-10-06T05:17:02.428705+0000","last_peered":"2023-10-06T05:17:02.428705+0000","last_clean":"2023-10-06T05:17:02.428705+0000","last_became_active":"2023-10-06T05:17:01.440903+0000","last_became_peered":"2023-10-06T05:17:01.440903+0000","last_unstale":"2023-10-06T05:17:02.428705+0000","last_undegraded":"2023-10-06T05:17:02.428705+0000","last_fullsized":"2023-10-06T05:17:02.428705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453318+0000","last_change":"2023-10-06T05:17:02.444240+0000","last_active":"2023-10-06T05:17:02.453318+0000","last_peered":"2023-10-06T05:17:02.453318+0000","last_clean":"2023-10-06T05:17:02.453318+0000","last_became_active":"2023-10-06T05:17:02.443717+0000","last_became_peered":"2023-10-06T05:17:02.443717+0000","last_unstale":"2023-10-06T05:17:02.453318+0000","last_undegraded":"2023-10-06T05:17:02.453318+0000","last_fullsized":"2023-10-06T05:17:02.453318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464271+0000","last_change":"2023-10-06T05:17:01.443363+0000","last_active":"2023-10-06T05:17:02.464271+0000","last_peered":"2023-10-06T05:17:02.464271+0000","last_clean":"2023-10-06T05:17:02.464271+0000","last_became_active":"2023-10-06T05:17:01.443131+0000","last_became_peered":"2023-10-06T05:17:01.443131+0000","last_unstale":"2023-10-06T05:17:02.464271+0000","last_undegraded":"2023-10-06T05:17:02.464271+0000","last_fullsized":"2023-10-06T05:17:02.464271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428924+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:17:02.428924+0000","last_peered":"2023-10-06T05:17:02.428924+0000","last_clean":"2023-10-06T05:17:02.428924+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:17:02.428924+0000","last_undegraded":"2023-10-06T05:17:02.428924+0000","last_fullsized":"2023-10-06T05:17:02.428924+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464122+0000","last_change":"2023-10-06T05:17:02.442449+0000","last_active":"2023-10-06T05:17:02.464122+0000","last_peered":"2023-10-06T05:17:02.464122+0000","last_clean":"2023-10-06T05:17:02.464122+0000","last_became_active":"2023-10-06T05:17:02.442135+0000","last_became_peered":"2023-10-06T05:17:02.442135+0000","last_unstale":"2023-10-06T05:17:02.464122+0000","last_undegraded":"2023-10-06T05:17:02.464122+0000","last_fullsized":"2023-10-06T05:17:02.464122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:31.781450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.427977+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:17:02.427977+0000","last_peered":"2023-10-06T05:17:02.427977+0000","last_clean":"2023-10-06T05:17:02.427977+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:17:02.427977+0000","last_undegraded":"2023-10-06T05:17:02.427977+0000","last_fullsized":"2023-10-06T05:17:02.427977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.436400+0000","last_change":"2023-10-06T05:17:01.448082+0000","last_active":"2023-10-06T05:17:02.436400+0000","last_peered":"2023-10-06T05:17:02.436400+0000","last_clean":"2023-10-06T05:17:02.436400+0000","last_became_active":"2023-10-06T05:17:01.447418+0000","last_became_peered":"2023-10-06T05:17:01.447418+0000","last_unstale":"2023-10-06T05:17:02.436400+0000","last_undegraded":"2023-10-06T05:17:02.436400+0000","last_fullsized":"2023-10-06T05:17:02.436400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452602+0000","last_change":"2023-10-06T05:17:02.445401+0000","last_active":"2023-10-06T05:17:02.452602+0000","last_peered":"2023-10-06T05:17:02.452602+0000","last_clean":"2023-10-06T05:17:02.452602+0000","last_became_active":"2023-10-06T05:17:02.444770+0000","last_became_peered":"2023-10-06T05:17:02.444770+0000","last_unstale":"2023-10-06T05:17:02.452602+0000","last_undegraded":"2023-10-06T05:17:02.452602+0000","last_fullsized":"2023-10-06T05:17:02.452602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429051+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:17:02.429051+0000","last_peered":"2023-10-06T05:17:02.429051+0000","last_clean":"2023-10-06T05:17:02.429051+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:17:02.429051+0000","last_undegraded":"2023-10-06T05:17:02.429051+0000","last_fullsized":"2023-10-06T05:17:02.429051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.436179+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:17:02.436179+0000","last_peered":"2023-10-06T05:17:02.436179+0000","last_clean":"2023-10-06T05:17:02.436179+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:17:02.436179+0000","last_undegraded":"2023-10-06T05:17:02.436179+0000","last_fullsized":"2023-10-06T05:17:02.436179+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.432715+0000","last_change":"2023-10-06T05:17:01.447882+0000","last_active":"2023-10-06T05:17:02.432715+0000","last_peered":"2023-10-06T05:17:02.432715+0000","last_clean":"2023-10-06T05:17:02.432715+0000","last_became_active":"2023-10-06T05:17:01.447617+0000","last_became_peered":"2023-10-06T05:17:01.447617+0000","last_unstale":"2023-10-06T05:17:02.432715+0000","last_undegraded":"2023-10-06T05:17:02.432715+0000","last_fullsized":"2023-10-06T05:17:02.432715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.460775+0000","last_change":"2023-10-06T05:17:02.455716+0000","last_active":"2023-10-06T05:17:02.460775+0000","last_peered":"2023-10-06T05:17:02.460775+0000","last_clean":"2023-10-06T05:17:02.460775+0000","last_became_active":"2023-10-06T05:17:02.455388+0000","last_became_peered":"2023-10-06T05:17:02.455388+0000","last_unstale":"2023-10-06T05:17:02.460775+0000","last_undegraded":"2023-10-06T05:17:02.460775+0000","last_fullsized":"2023-10-06T05:17:02.460775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434749+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:17:02.434749+0000","last_peered":"2023-10-06T05:17:02.434749+0000","last_clean":"2023-10-06T05:17:02.434749+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:17:02.434749+0000","last_undegraded":"2023-10-06T05:17:02.434749+0000","last_fullsized":"2023-10-06T05:17:02.434749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429080+0000","last_change":"2023-10-06T05:17:01.449606+0000","last_active":"2023-10-06T05:17:02.429080+0000","last_peered":"2023-10-06T05:17:02.429080+0000","last_clean":"2023-10-06T05:17:02.429080+0000","last_became_active":"2023-10-06T05:17:01.449380+0000","last_became_peered":"2023-10-06T05:17:01.449380+0000","last_unstale":"2023-10-06T05:17:02.429080+0000","last_undegraded":"2023-10-06T05:17:02.429080+0000","last_fullsized":"2023-10-06T05:17:02.429080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453300+0000","last_change":"2023-10-06T05:17:02.444044+0000","last_active":"2023-10-06T05:17:02.453300+0000","last_peered":"2023-10-06T05:17:02.453300+0000","last_clean":"2023-10-06T05:17:02.453300+0000","last_became_active":"2023-10-06T05:17:02.443659+0000","last_became_peered":"2023-10-06T05:17:02.443659+0000","last_unstale":"2023-10-06T05:17:02.453300+0000","last_undegraded":"2023-10-06T05:17:02.453300+0000","last_fullsized":"2023-10-06T05:17:02.453300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:46.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435753+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:17:02.435753+0000","last_peered":"2023-10-06T05:17:02.435753+0000","last_clean":"2023-10-06T05:17:02.435753+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:17:02.435753+0000","last_undegraded":"2023-10-06T05:17:02.435753+0000","last_fullsized":"2023-10-06T05:17:02.435753+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.427375+0000","last_change":"2023-10-06T05:17:01.441700+0000","last_active":"2023-10-06T05:17:02.427375+0000","last_peered":"2023-10-06T05:17:02.427375+0000","last_clean":"2023-10-06T05:17:02.427375+0000","last_became_active":"2023-10-06T05:17:01.441370+0000","last_became_peered":"2023-10-06T05:17:01.441370+0000","last_unstale":"2023-10-06T05:17:02.427375+0000","last_undegraded":"2023-10-06T05:17:02.427375+0000","last_fullsized":"2023-10-06T05:17:02.427375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:01.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450934+0000","last_change":"2023-10-06T05:17:02.446567+0000","last_active":"2023-10-06T05:17:02.450934+0000","last_peered":"2023-10-06T05:17:02.450934+0000","last_clean":"2023-10-06T05:17:02.450934+0000","last_became_active":"2023-10-06T05:17:02.446263+0000","last_became_peered":"2023-10-06T05:17:02.446263+0000","last_unstale":"2023-10-06T05:17:02.450934+0000","last_undegraded":"2023-10-06T05:17:02.450934+0000","last_fullsized":"2023-10-06T05:17:02.450934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.433849+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:17:02.433849+0000","last_peered":"2023-10-06T05:17:02.433849+0000","last_clean":"2023-10-06T05:17:02.433849+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:17:02.433849+0000","last_undegraded":"2023-10-06T05:17:02.433849+0000","last_fullsized":"2023-10-06T05:17:02.433849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429218+0000","last_change":"2023-10-06T05:17:01.448636+0000","last_active":"2023-10-06T05:17:02.429218+0000","last_peered":"2023-10-06T05:17:02.429218+0000","last_clean":"2023-10-06T05:17:02.429218+0000","last_became_active":"2023-10-06T05:17:01.446188+0000","last_became_peered":"2023-10-06T05:17:01.446188+0000","last_unstale":"2023-10-06T05:17:02.429218+0000","last_undegraded":"2023-10-06T05:17:02.429218+0000","last_fullsized":"2023-10-06T05:17:02.429218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453488+0000","last_change":"2023-10-06T05:17:02.447390+0000","last_active":"2023-10-06T05:17:02.453488+0000","last_peered":"2023-10-06T05:17:02.453488+0000","last_clean":"2023-10-06T05:17:02.453488+0000","last_became_active":"2023-10-06T05:17:02.447118+0000","last_became_peered":"2023-10-06T05:17:02.447118+0000","last_unstale":"2023-10-06T05:17:02.453488+0000","last_undegraded":"2023-10-06T05:17:02.453488+0000","last_fullsized":"2023-10-06T05:17:02.453488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.439722+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:17:02.439722+0000","last_peered":"2023-10-06T05:17:02.439722+0000","last_clean":"2023-10-06T05:17:02.439722+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:17:02.439722+0000","last_undegraded":"2023-10-06T05:17:02.439722+0000","last_fullsized":"2023-10-06T05:17:02.439722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.431999+0000","last_change":"2023-10-06T05:17:01.449227+0000","last_active":"2023-10-06T05:17:02.431999+0000","last_peered":"2023-10-06T05:17:02.431999+0000","last_clean":"2023-10-06T05:17:02.431999+0000","last_became_active":"2023-10-06T05:17:01.448956+0000","last_became_peered":"2023-10-06T05:17:01.448956+0000","last_unstale":"2023-10-06T05:17:02.431999+0000","last_undegraded":"2023-10-06T05:17:02.431999+0000","last_fullsized":"2023-10-06T05:17:02.431999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450001+0000","last_change":"2023-10-06T05:17:02.442854+0000","last_active":"2023-10-06T05:17:02.450001+0000","last_peered":"2023-10-06T05:17:02.450001+0000","last_clean":"2023-10-06T05:17:02.450001+0000","last_became_active":"2023-10-06T05:17:02.442424+0000","last_became_peered":"2023-10-06T05:17:02.442424+0000","last_unstale":"2023-10-06T05:17:02.450001+0000","last_undegraded":"2023-10-06T05:17:02.450001+0000","last_fullsized":"2023-10-06T05:17:02.450001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.437608+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:17:02.437608+0000","last_peered":"2023-10-06T05:17:02.437608+0000","last_clean":"2023-10-06T05:17:02.437608+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:17:02.437608+0000","last_undegraded":"2023-10-06T05:17:02.437608+0000","last_fullsized":"2023-10-06T05:17:02.437608+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.433246+0000","last_change":"2023-10-06T05:17:01.444318+0000","last_active":"2023-10-06T05:17:02.433246+0000","last_peered":"2023-10-06T05:17:02.433246+0000","last_clean":"2023-10-06T05:17:02.433246+0000","last_became_active":"2023-10-06T05:17:01.444042+0000","last_became_peered":"2023-10-06T05:17:01.444042+0000","last_unstale":"2023-10-06T05:17:02.433246+0000","last_undegraded":"2023-10-06T05:17:02.433246+0000","last_fullsized":"2023-10-06T05:17:02.433246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453458+0000","last_change":"2023-10-06T05:17:02.443796+0000","last_active":"2023-10-06T05:17:02.453458+0000","last_peered":"2023-10-06T05:17:02.453458+0000","last_clean":"2023-10-06T05:17:02.453458+0000","last_became_active":"2023-10-06T05:17:02.443364+0000","last_became_peered":"2023-10-06T05:17:02.443364+0000","last_unstale":"2023-10-06T05:17:02.453458+0000","last_undegraded":"2023-10-06T05:17:02.453458+0000","last_fullsized":"2023-10-06T05:17:02.453458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:07.184397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465585+0000","last_change":"2023-10-06T05:17:01.451251+0000","last_active":"2023-10-06T05:17:02.465585+0000","last_peered":"2023-10-06T05:17:02.465585+0000","last_clean":"2023-10-06T05:17:02.465585+0000","last_became_active":"2023-10-06T05:17:01.450983+0000","last_became_peered":"2023-10-06T05:17:01.450983+0000","last_unstale":"2023-10-06T05:17:02.465585+0000","last_undegraded":"2023-10-06T05:17:02.465585+0000","last_fullsized":"2023-10-06T05:17:02.465585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456082+0000","last_change":"2023-10-06T05:17:02.451794+0000","last_active":"2023-10-06T05:17:02.456082+0000","last_peered":"2023-10-06T05:17:02.456082+0000","last_clean":"2023-10-06T05:17:02.456082+0000","last_became_active":"2023-10-06T05:17:02.451533+0000","last_became_peered":"2023-10-06T05:17:02.451533+0000","last_unstale":"2023-10-06T05:17:02.456082+0000","last_undegraded":"2023-10-06T05:17:02.456082+0000","last_fullsized":"2023-10-06T05:17:02.456082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:23.939592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471267+0000","last_change":"2023-10-06T05:17:01.449818+0000","last_active":"2023-10-06T05:17:02.471267+0000","last_peered":"2023-10-06T05:17:02.471267+0000","last_clean":"2023-10-06T05:17:02.471267+0000","last_became_active":"2023-10-06T05:17:01.449477+0000","last_became_peered":"2023-10-06T05:17:01.449477+0000","last_unstale":"2023-10-06T05:17:02.471267+0000","last_undegraded":"2023-10-06T05:17:02.471267+0000","last_fullsized":"2023-10-06T05:17:02.471267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447570+0000","last_change":"2023-10-06T05:17:02.443687+0000","last_active":"2023-10-06T05:17:02.447570+0000","last_peered":"2023-10-06T05:17:02.447570+0000","last_clean":"2023-10-06T05:17:02.447570+0000","last_became_active":"2023-10-06T05:17:02.443433+0000","last_became_peered":"2023-10-06T05:17:02.443433+0000","last_unstale":"2023-10-06T05:17:02.447570+0000","last_undegraded":"2023-10-06T05:17:02.447570+0000","last_fullsized":"2023-10-06T05:17:02.447570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:29.361752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471253+0000","last_change":"2023-10-06T05:17:01.449867+0000","last_active":"2023-10-06T05:17:02.471253+0000","last_peered":"2023-10-06T05:17:02.471253+0000","last_clean":"2023-10-06T05:17:02.471253+0000","last_became_active":"2023-10-06T05:17:01.449452+0000","last_became_peered":"2023-10-06T05:17:01.449452+0000","last_unstale":"2023-10-06T05:17:02.471253+0000","last_undegraded":"2023-10-06T05:17:02.471253+0000","last_fullsized":"2023-10-06T05:17:02.471253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447425+0000","last_change":"2023-10-06T05:17:02.439184+0000","last_active":"2023-10-06T05:17:02.447425+0000","last_peered":"2023-10-06T05:17:02.447425+0000","last_clean":"2023-10-06T05:17:02.447425+0000","last_became_active":"2023-10-06T05:17:02.438893+0000","last_became_peered":"2023-10-06T05:17:02.438893+0000","last_unstale":"2023-10-06T05:17:02.447425+0000","last_undegraded":"2023-10-06T05:17:02.447425+0000","last_fullsized":"2023-10-06T05:17:02.447425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:28.005164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.463116+0000","last_change":"2023-10-06T05:17:01.441308+0000","last_active":"2023-10-06T05:17:02.463116+0000","last_peered":"2023-10-06T05:17:02.463116+0000","last_clean":"2023-10-06T05:17:02.463116+0000","last_became_active":"2023-10-06T05:17:01.440904+0000","last_became_peered":"2023-10-06T05:17:01.440904+0000","last_unstale":"2023-10-06T05:17:02.463116+0000","last_undegraded":"2023-10-06T05:17:02.463116+0000","last_fullsized":"2023-10-06T05:17:02.463116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:52.916083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461869+0000","last_change":"2023-10-06T05:17:02.447801+0000","last_active":"2023-10-06T05:17:02.461869+0000","last_peered":"2023-10-06T05:17:02.461869+0000","last_clean":"2023-10-06T05:17:02.461869+0000","last_became_active":"2023-10-06T05:17:02.447361+0000","last_became_peered":"2023-10-06T05:17:02.447361+0000","last_unstale":"2023-10-06T05:17:02.461869+0000","last_undegraded":"2023-10-06T05:17:02.461869+0000","last_fullsized":"2023-10-06T05:17:02.461869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465330+0000","last_change":"2023-10-06T05:17:01.446169+0000","last_active":"2023-10-06T05:17:02.465330+0000","last_peered":"2023-10-06T05:17:02.465330+0000","last_clean":"2023-10-06T05:17:02.465330+0000","last_became_active":"2023-10-06T05:17:01.445836+0000","last_became_peered":"2023-10-06T05:17:01.445836+0000","last_unstale":"2023-10-06T05:17:02.465330+0000","last_undegraded":"2023-10-06T05:17:02.465330+0000","last_fullsized":"2023-10-06T05:17:02.465330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:52.916083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452461+0000","last_change":"2023-10-06T05:17:02.447171+0000","last_active":"2023-10-06T05:17:02.452461+0000","last_peered":"2023-10-06T05:17:02.452461+0000","last_clean":"2023-10-06T05:17:02.452461+0000","last_became_active":"2023-10-06T05:17:02.446884+0000","last_became_peered":"2023-10-06T05:17:02.446884+0000","last_unstale":"2023-10-06T05:17:02.452461+0000","last_undegraded":"2023-10-06T05:17:02.452461+0000","last_fullsized":"2023-10-06T05:17:02.452461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456523+0000","last_change":"2023-10-06T05:17:02.451494+0000","last_active":"2023-10-06T05:17:02.456523+0000","last_peered":"2023-10-06T05:17:02.456523+0000","last_clean":"2023-10-06T05:17:02.456523+0000","last_became_active":"2023-10-06T05:17:02.451213+0000","last_became_peered":"2023-10-06T05:17:02.451213+0000","last_unstale":"2023-10-06T05:17:02.456523+0000","last_undegraded":"2023-10-06T05:17:02.456523+0000","last_fullsized":"2023-10-06T05:17:02.456523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:54.883797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461850+0000","last_change":"2023-10-06T05:17:02.444575+0000","last_active":"2023-10-06T05:17:02.461850+0000","last_peered":"2023-10-06T05:17:02.461850+0000","last_clean":"2023-10-06T05:17:02.461850+0000","last_became_active":"2023-10-06T05:17:02.444226+0000","last_became_peered":"2023-10-06T05:17:02.444226+0000","last_unstale":"2023-10-06T05:17:02.461850+0000","last_undegraded":"2023-10-06T05:17:02.461850+0000","last_fullsized":"2023-10-06T05:17:02.461850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452618+0000","last_change":"2023-10-06T05:17:02.447774+0000","last_active":"2023-10-06T05:17:02.452618+0000","last_peered":"2023-10-06T05:17:02.452618+0000","last_clean":"2023-10-06T05:17:02.452618+0000","last_became_active":"2023-10-06T05:17:02.447500+0000","last_became_peered":"2023-10-06T05:17:02.447500+0000","last_unstale":"2023-10-06T05:17:02.452618+0000","last_undegraded":"2023-10-06T05:17:02.452618+0000","last_fullsized":"2023-10-06T05:17:02.452618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:01.131607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462293+0000","last_change":"2023-10-06T05:17:02.441565+0000","last_active":"2023-10-06T05:17:02.462293+0000","last_peered":"2023-10-06T05:17:02.462293+0000","last_clean":"2023-10-06T05:17:02.462293+0000","last_became_active":"2023-10-06T05:17:02.441272+0000","last_became_peered":"2023-10-06T05:17:02.441272+0000","last_unstale":"2023-10-06T05:17:02.462293+0000","last_undegraded":"2023-10-06T05:17:02.462293+0000","last_fullsized":"2023-10-06T05:17:02.462293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:28.806289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453469+0000","last_change":"2023-10-06T05:17:02.443555+0000","last_active":"2023-10-06T05:17:02.453469+0000","last_peered":"2023-10-06T05:17:02.453469+0000","last_clean":"2023-10-06T05:17:02.453469+0000","last_became_active":"2023-10-06T05:17:02.443268+0000","last_became_peered":"2023-10-06T05:17:02.443268+0000","last_unstale":"2023-10-06T05:17:02.453469+0000","last_undegraded":"2023-10-06T05:17:02.453469+0000","last_fullsized":"2023-10-06T05:17:02.453469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":659663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":67543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298068,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073772,"statfs":{"total":96636764160,"available":96331542528,"internally_reserved":0,"allocated":798720,"data_stored":664552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297480,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074360,"statfs":{"total":96636764160,"available":96332144640,"internally_reserved":0,"allocated":196608,"data_stored":68097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":67543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":200704,"data_stored":68167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:17:06.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:17:07.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.078+0000 7f141bbff700 1 Processor -- start 2023-10-06T05:17:07.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.079+0000 7f141bbff700 1 -- start start 2023-10-06T05:17:07.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.080+0000 7f141bbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1414159500 0x7f14140b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.080+0000 7f141bbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f141415d2d0 0x7f14140cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.080+0000 7f141bbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14140c3f40 con 0x7f14141598d0 2023-10-06T05:17:07.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.080+0000 7f141bbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14140b2860 con 0x7f141415d2d0 2023-10-06T05:17:07.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.080+0000 7f141bbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141415d7d0 con 0x7f1414159500 2023-10-06T05:17:07.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.080+0000 7f141919a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f141415d2d0 0x7f14140cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.080+0000 7f141919a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f141415d2d0 0x7f14140cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34126/0 (socket says 172.21.15.31:34126) 2023-10-06T05:17:07.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.080+0000 7f141919a700 1 -- 172.21.15.31:0/4021378011 learned_addr learned my addr 172.21.15.31:0/4021378011 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:07.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.080+0000 7f141999b700 1 --2- 172.21.15.31:0/4021378011 >> v2:172.21.15.62:3300/0 conn(0x7f1414159500 0x7f14140b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.081+0000 7f141919a700 1 -- 172.21.15.31:0/4021378011 >> v2:172.21.15.62:3300/0 conn(0x7f1414159500 msgr2=0x7f14140b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:07.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.081+0000 7f141919a700 1 --2- 172.21.15.31:0/4021378011 >> v2:172.21.15.62:3300/0 conn(0x7f1414159500 0x7f14140b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.081+0000 7f141919a700 1 -- 172.21.15.31:0/4021378011 >> v1:172.21.15.62:6789/0 conn(0x7f14141598d0 legacy=0x7f14140aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.081+0000 7f141919a700 1 -- 172.21.15.31:0/4021378011 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14140cb000 con 0x7f141415d2d0 2023-10-06T05:17:07.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.081+0000 7f141919a700 1 --2- 172.21.15.31:0/4021378011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141415d2d0 0x7f14140cce00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1414089840 tx=0x7f1404009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554faee26f87d314 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:07.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.081+0000 7f1418999700 1 -- 172.21.15.31:0/4021378011 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404005c10 con 0x7f141415d2d0 2023-10-06T05:17:07.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.081+0000 7f1418999700 1 -- 172.21.15.31:0/4021378011 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1404005d70 con 0x7f141415d2d0 2023-10-06T05:17:07.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.081+0000 7f1418999700 1 -- 172.21.15.31:0/4021378011 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404006260 con 0x7f141415d2d0 2023-10-06T05:17:07.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.082+0000 7f141bbff700 1 -- 172.21.15.31:0/4021378011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141415d2d0 msgr2=0x7f14140cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.082+0000 7f141bbff700 1 --2- 172.21.15.31:0/4021378011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141415d2d0 0x7f14140cce00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1414089840 tx=0x7f1404009220 comp rx=0 tx=0).stop 2023-10-06T05:17:07.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.082+0000 7f141bbff700 1 -- 172.21.15.31:0/4021378011 shutdown_connections 2023-10-06T05:17:07.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.082+0000 7f141bbff700 1 --2- 172.21.15.31:0/4021378011 >> v2:172.21.15.62:3300/0 conn(0x7f1414159500 0x7f14140b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.082+0000 7f141bbff700 1 --2- 172.21.15.31:0/4021378011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141415d2d0 0x7f14140cce00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.082+0000 7f141bbff700 1 -- 172.21.15.31:0/4021378011 >> 172.21.15.31:0/4021378011 conn(0x7f141406ec90 msgr2=0x7f141406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:07.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.082+0000 7f141bbff700 1 -- 172.21.15.31:0/4021378011 shutdown_connections 2023-10-06T05:17:07.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.082+0000 7f141bbff700 1 -- 172.21.15.31:0/4021378011 wait complete. 2023-10-06T05:17:07.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.083+0000 7f141bbff700 1 Processor -- start 2023-10-06T05:17:07.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.083+0000 7f141bbff700 1 -- start start 2023-10-06T05:17:07.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.084+0000 7f141bbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1414159500 0x7f1414157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.084+0000 7f141bbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141598d0 0x7f14140b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.084+0000 7f141bbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f141415d2d0 0x7f14140b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.084+0000 7f141bbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14140b6220 con 0x7f14141598d0 2023-10-06T05:17:07.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.084+0000 7f141bbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1414146440 con 0x7f1414159500 2023-10-06T05:17:07.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.084+0000 7f141bbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14140c89d0 con 0x7f141415d2d0 2023-10-06T05:17:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141598d0 0x7f14140b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141598d0 0x7f14140b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34142/0 (socket says 172.21.15.31:34142) 2023-10-06T05:17:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141919a700 1 -- 172.21.15.31:0/1114555367 learned_addr learned my addr 172.21.15.31:0/1114555367 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141999b700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1414159500 0x7f1414157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141a19c700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f141415d2d0 0x7f14140b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141919a700 1 -- 172.21.15.31:0/1114555367 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f141415d2d0 msgr2=0x7f14140b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141919a700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f141415d2d0 0x7f14140b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141919a700 1 -- 172.21.15.31:0/1114555367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1414159500 msgr2=0x7f1414157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141919a700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1414159500 0x7f1414157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141919a700 1 -- 172.21.15.31:0/1114555367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1404009480 con 0x7f14141598d0 2023-10-06T05:17:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.085+0000 7f141919a700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141598d0 0x7f14140b5ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1404006360 tx=0x7f1404016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.086+0000 7f1409ffb700 1 -- 172.21.15.31:0/1114555367 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400b8d0 con 0x7f14141598d0 2023-10-06T05:17:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.086+0000 7f1409ffb700 1 -- 172.21.15.31:0/1114555367 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140400ba30 con 0x7f14141598d0 2023-10-06T05:17:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.086+0000 7f141bbff700 1 -- 172.21.15.31:0/1114555367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14140c7dc0 con 0x7f14141598d0 2023-10-06T05:17:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.086+0000 7f1409ffb700 1 -- 172.21.15.31:0/1114555367 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400bdf0 con 0x7f14141598d0 2023-10-06T05:17:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.086+0000 7f141bbff700 1 -- 172.21.15.31:0/1114555367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14140c8380 con 0x7f14141598d0 2023-10-06T05:17:07.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.087+0000 7f141bbff700 1 -- 172.21.15.31:0/1114555367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141404c380 con 0x7f14141598d0 2023-10-06T05:17:07.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.088+0000 7f1409ffb700 1 -- 172.21.15.31:0/1114555367 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f140401b640 con 0x7f14141598d0 2023-10-06T05:17:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.089+0000 7f1409ffb700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f13f8066f30 0x7f13f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.089+0000 7f1409ffb700 1 -- 172.21.15.31:0/1114555367 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1404085f60 con 0x7f14141598d0 2023-10-06T05:17:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.089+0000 7f141999b700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f13f8066f30 0x7f13f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.089+0000 7f141999b700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f13f8066f30 0x7f13f80693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14140cc5f0 tx=0x7f14100091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:07.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.093+0000 7f1409ffb700 1 -- 172.21.15.31:0/1114555367 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1404052fd0 con 0x7f14141598d0 2023-10-06T05:17:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.293+0000 7f141bbff700 1 -- 172.21.15.31:0/1114555367 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14140c8b10 con 0x7f13f8066f30 2023-10-06T05:17:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.302+0000 7f1409ffb700 1 -- 172.21.15.31:0/1114555367 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286934 (secure 0 0 0) 0x7f14140c8b10 con 0x7f13f8066f30 2023-10-06T05:17:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 -- 172.21.15.31:0/1114555367 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f13f8066f30 msgr2=0x7f13f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f13f8066f30 0x7f13f80693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14140cc5f0 tx=0x7f14100091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 -- 172.21.15.31:0/1114555367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141598d0 msgr2=0x7f14140b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141598d0 0x7f14140b5ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1404006360 tx=0x7f1404016000 comp rx=0 tx=0).stop 2023-10-06T05:17:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 -- 172.21.15.31:0/1114555367 shutdown_connections 2023-10-06T05:17:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f141415d2d0 0x7f14140b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1414159500 0x7f1414157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f13f8066f30 0x7f13f80693e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 --2- 172.21.15.31:0/1114555367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14141598d0 0x7f14140b5ce0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 -- 172.21.15.31:0/1114555367 >> 172.21.15.31:0/1114555367 conn(0x7f141406ec90 msgr2=0x7f14140b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 -- 172.21.15.31:0/1114555367 shutdown_connections 2023-10-06T05:17:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.306+0000 7f141bbff700 1 -- 172.21.15.31:0/1114555367 wait complete. 2023-10-06T05:17:07.312 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:17:07.325 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-06T05:17:06.061061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381068,"kb_used_data":2764,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593652,"statfs":{"total":773094113280,"available":770655899648,"internally_reserved":0,"allocated":2830336,"data_stored":1745615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.644982"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464636+0000","last_change":"2023-10-06T05:17:02.463452+0000","last_active":"2023-10-06T05:17:02.464636+0000","last_peered":"2023-10-06T05:17:02.464636+0000","last_clean":"2023-10-06T05:17:02.464636+0000","last_became_active":"2023-10-06T05:17:02.463139+0000","last_became_peered":"2023-10-06T05:17:02.463139+0000","last_unstale":"2023-10-06T05:17:02.464636+0000","last_undegraded":"2023-10-06T05:17:02.464636+0000","last_fullsized":"2023-10-06T05:17:02.464636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:52.286394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456371+0000","last_change":"2023-10-06T05:17:02.451303+0000","last_active":"2023-10-06T05:17:02.456371+0000","last_peered":"2023-10-06T05:17:02.456371+0000","last_clean":"2023-10-06T05:17:02.456371+0000","last_became_active":"2023-10-06T05:17:02.450985+0000","last_became_peered":"2023-10-06T05:17:02.450985+0000","last_unstale":"2023-10-06T05:17:02.456371+0000","last_undegraded":"2023-10-06T05:17:02.456371+0000","last_fullsized":"2023-10-06T05:17:02.456371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453158+0000","last_change":"2023-10-06T05:17:02.451335+0000","last_active":"2023-10-06T05:17:02.453158+0000","last_peered":"2023-10-06T05:17:02.453158+0000","last_clean":"2023-10-06T05:17:02.453158+0000","last_became_active":"2023-10-06T05:17:02.451119+0000","last_became_peered":"2023-10-06T05:17:02.451119+0000","last_unstale":"2023-10-06T05:17:02.453158+0000","last_undegraded":"2023-10-06T05:17:02.453158+0000","last_fullsized":"2023-10-06T05:17:02.453158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:59.439667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.458136+0000","last_change":"2023-10-06T05:17:02.454559+0000","last_active":"2023-10-06T05:17:02.458136+0000","last_peered":"2023-10-06T05:17:02.458136+0000","last_clean":"2023-10-06T05:17:02.458136+0000","last_became_active":"2023-10-06T05:17:02.453985+0000","last_became_peered":"2023-10-06T05:17:02.453985+0000","last_unstale":"2023-10-06T05:17:02.458136+0000","last_undegraded":"2023-10-06T05:17:02.458136+0000","last_fullsized":"2023-10-06T05:17:02.458136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:57.117094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465742+0000","last_change":"2023-10-06T05:17:02.460869+0000","last_active":"2023-10-06T05:17:02.465742+0000","last_peered":"2023-10-06T05:17:02.465742+0000","last_clean":"2023-10-06T05:17:02.465742+0000","last_became_active":"2023-10-06T05:17:02.460257+0000","last_became_peered":"2023-10-06T05:17:02.460257+0000","last_unstale":"2023-10-06T05:17:02.465742+0000","last_undegraded":"2023-10-06T05:17:02.465742+0000","last_fullsized":"2023-10-06T05:17:02.465742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453122+0000","last_change":"2023-10-06T05:17:02.451827+0000","last_active":"2023-10-06T05:17:02.453122+0000","last_peered":"2023-10-06T05:17:02.453122+0000","last_clean":"2023-10-06T05:17:02.453122+0000","last_became_active":"2023-10-06T05:17:02.451573+0000","last_became_peered":"2023-10-06T05:17:02.451573+0000","last_unstale":"2023-10-06T05:17:02.453122+0000","last_undegraded":"2023-10-06T05:17:02.453122+0000","last_fullsized":"2023-10-06T05:17:02.453122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:50.530278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.454214+0000","last_change":"2023-10-06T05:17:02.451805+0000","last_active":"2023-10-06T05:17:02.454214+0000","last_peered":"2023-10-06T05:17:02.454214+0000","last_clean":"2023-10-06T05:17:02.454214+0000","last_became_active":"2023-10-06T05:17:02.451504+0000","last_became_peered":"2023-10-06T05:17:02.451504+0000","last_unstale":"2023-10-06T05:17:02.454214+0000","last_undegraded":"2023-10-06T05:17:02.454214+0000","last_fullsized":"2023-10-06T05:17:02.454214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448446+0000","last_change":"2023-10-06T05:17:02.440866+0000","last_active":"2023-10-06T05:17:02.448446+0000","last_peered":"2023-10-06T05:17:02.448446+0000","last_clean":"2023-10-06T05:17:02.448446+0000","last_became_active":"2023-10-06T05:17:02.440548+0000","last_became_peered":"2023-10-06T05:17:02.440548+0000","last_unstale":"2023-10-06T05:17:02.448446+0000","last_undegraded":"2023-10-06T05:17:02.448446+0000","last_fullsized":"2023-10-06T05:17:02.448446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.466997+0000","last_change":"2023-10-06T05:17:02.461715+0000","last_active":"2023-10-06T05:17:02.466997+0000","last_peered":"2023-10-06T05:17:02.466997+0000","last_clean":"2023-10-06T05:17:02.466997+0000","last_became_active":"2023-10-06T05:17:02.461152+0000","last_became_peered":"2023-10-06T05:17:02.461152+0000","last_unstale":"2023-10-06T05:17:02.466997+0000","last_undegraded":"2023-10-06T05:17:02.466997+0000","last_fullsized":"2023-10-06T05:17:02.466997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453073+0000","last_change":"2023-10-06T05:17:02.448948+0000","last_active":"2023-10-06T05:17:02.453073+0000","last_peered":"2023-10-06T05:17:02.453073+0000","last_clean":"2023-10-06T05:17:02.453073+0000","last_became_active":"2023-10-06T05:17:02.448524+0000","last_became_peered":"2023-10-06T05:17:02.448524+0000","last_unstale":"2023-10-06T05:17:02.453073+0000","last_undegraded":"2023-10-06T05:17:02.453073+0000","last_fullsized":"2023-10-06T05:17:02.453073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452570+0000","last_change":"2023-10-06T05:17:02.446198+0000","last_active":"2023-10-06T05:17:02.452570+0000","last_peered":"2023-10-06T05:17:02.452570+0000","last_clean":"2023-10-06T05:17:02.452570+0000","last_became_active":"2023-10-06T05:17:02.445918+0000","last_became_peered":"2023-10-06T05:17:02.445918+0000","last_unstale":"2023-10-06T05:17:02.452570+0000","last_undegraded":"2023-10-06T05:17:02.452570+0000","last_fullsized":"2023-10-06T05:17:02.452570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:45.361281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450359+0000","last_change":"2023-10-06T05:17:02.447094+0000","last_active":"2023-10-06T05:17:02.450359+0000","last_peered":"2023-10-06T05:17:02.450359+0000","last_clean":"2023-10-06T05:17:02.450359+0000","last_became_active":"2023-10-06T05:17:02.446800+0000","last_became_peered":"2023-10-06T05:17:02.446800+0000","last_unstale":"2023-10-06T05:17:02.450359+0000","last_undegraded":"2023-10-06T05:17:02.450359+0000","last_fullsized":"2023-10-06T05:17:02.450359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:46.452504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464672+0000","last_change":"2023-10-06T05:17:02.463414+0000","last_active":"2023-10-06T05:17:02.464672+0000","last_peered":"2023-10-06T05:17:02.464672+0000","last_clean":"2023-10-06T05:17:02.464672+0000","last_became_active":"2023-10-06T05:17:02.463125+0000","last_became_peered":"2023-10-06T05:17:02.463125+0000","last_unstale":"2023-10-06T05:17:02.464672+0000","last_undegraded":"2023-10-06T05:17:02.464672+0000","last_fullsized":"2023-10-06T05:17:02.464672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:29.979974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450580+0000","last_change":"2023-10-06T05:17:02.447113+0000","last_active":"2023-10-06T05:17:02.450580+0000","last_peered":"2023-10-06T05:17:02.450580+0000","last_clean":"2023-10-06T05:17:02.450580+0000","last_became_active":"2023-10-06T05:17:02.446878+0000","last_became_peered":"2023-10-06T05:17:02.446878+0000","last_unstale":"2023-10-06T05:17:02.450580+0000","last_undegraded":"2023-10-06T05:17:02.450580+0000","last_fullsized":"2023-10-06T05:17:02.450580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.454233+0000","last_change":"2023-10-06T05:17:02.451949+0000","last_active":"2023-10-06T05:17:02.454233+0000","last_peered":"2023-10-06T05:17:02.454233+0000","last_clean":"2023-10-06T05:17:02.454233+0000","last_became_active":"2023-10-06T05:17:02.451640+0000","last_became_peered":"2023-10-06T05:17:02.451640+0000","last_unstale":"2023-10-06T05:17:02.454233+0000","last_undegraded":"2023-10-06T05:17:02.454233+0000","last_fullsized":"2023-10-06T05:17:02.454233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452479+0000","last_change":"2023-10-06T05:17:02.445441+0000","last_active":"2023-10-06T05:17:02.452479+0000","last_peered":"2023-10-06T05:17:02.452479+0000","last_clean":"2023-10-06T05:17:02.452479+0000","last_became_active":"2023-10-06T05:17:02.445132+0000","last_became_peered":"2023-10-06T05:17:02.445132+0000","last_unstale":"2023-10-06T05:17:02.452479+0000","last_undegraded":"2023-10-06T05:17:02.452479+0000","last_fullsized":"2023-10-06T05:17:02.452479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:10.259711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464487+0000","last_change":"2023-10-06T05:17:02.461554+0000","last_active":"2023-10-06T05:17:02.464487+0000","last_peered":"2023-10-06T05:17:02.464487+0000","last_clean":"2023-10-06T05:17:02.464487+0000","last_became_active":"2023-10-06T05:17:02.460803+0000","last_became_peered":"2023-10-06T05:17:02.460803+0000","last_unstale":"2023-10-06T05:17:02.464487+0000","last_undegraded":"2023-10-06T05:17:02.464487+0000","last_fullsized":"2023-10-06T05:17:02.464487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452431+0000","last_change":"2023-10-06T05:17:02.448249+0000","last_active":"2023-10-06T05:17:02.452431+0000","last_peered":"2023-10-06T05:17:02.452431+0000","last_clean":"2023-10-06T05:17:02.452431+0000","last_became_active":"2023-10-06T05:17:02.447977+0000","last_became_peered":"2023-10-06T05:17:02.447977+0000","last_unstale":"2023-10-06T05:17:02.452431+0000","last_undegraded":"2023-10-06T05:17:02.452431+0000","last_fullsized":"2023-10-06T05:17:02.452431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:10.643350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456974+0000","last_change":"2023-10-06T05:17:02.451653+0000","last_active":"2023-10-06T05:17:02.456974+0000","last_peered":"2023-10-06T05:17:02.456974+0000","last_clean":"2023-10-06T05:17:02.456974+0000","last_became_active":"2023-10-06T05:17:02.451347+0000","last_became_peered":"2023-10-06T05:17:02.451347+0000","last_unstale":"2023-10-06T05:17:02.456974+0000","last_undegraded":"2023-10-06T05:17:02.456974+0000","last_fullsized":"2023-10-06T05:17:02.456974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:20.132880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452508+0000","last_change":"2023-10-06T05:17:02.446775+0000","last_active":"2023-10-06T05:17:02.452508+0000","last_peered":"2023-10-06T05:17:02.452508+0000","last_clean":"2023-10-06T05:17:02.452508+0000","last_became_active":"2023-10-06T05:17:02.446482+0000","last_became_peered":"2023-10-06T05:17:02.446482+0000","last_unstale":"2023-10-06T05:17:02.452508+0000","last_undegraded":"2023-10-06T05:17:02.452508+0000","last_fullsized":"2023-10-06T05:17:02.452508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:14.399779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468275+0000","last_change":"2023-10-06T05:17:02.463237+0000","last_active":"2023-10-06T05:17:02.468275+0000","last_peered":"2023-10-06T05:17:02.468275+0000","last_clean":"2023-10-06T05:17:02.468275+0000","last_became_active":"2023-10-06T05:17:02.462891+0000","last_became_peered":"2023-10-06T05:17:02.462891+0000","last_unstale":"2023-10-06T05:17:02.468275+0000","last_undegraded":"2023-10-06T05:17:02.468275+0000","last_fullsized":"2023-10-06T05:17:02.468275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462403+0000","last_change":"2023-10-06T05:17:02.458157+0000","last_active":"2023-10-06T05:17:02.462403+0000","last_peered":"2023-10-06T05:17:02.462403+0000","last_clean":"2023-10-06T05:17:02.462403+0000","last_became_active":"2023-10-06T05:17:02.455535+0000","last_became_peered":"2023-10-06T05:17:02.455535+0000","last_unstale":"2023-10-06T05:17:02.462403+0000","last_undegraded":"2023-10-06T05:17:02.462403+0000","last_fullsized":"2023-10-06T05:17:02.462403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461887+0000","last_change":"2023-10-06T05:17:02.449068+0000","last_active":"2023-10-06T05:17:02.461887+0000","last_peered":"2023-10-06T05:17:02.461887+0000","last_clean":"2023-10-06T05:17:02.461887+0000","last_became_active":"2023-10-06T05:17:02.448751+0000","last_became_peered":"2023-10-06T05:17:02.448751+0000","last_unstale":"2023-10-06T05:17:02.461887+0000","last_undegraded":"2023-10-06T05:17:02.461887+0000","last_fullsized":"2023-10-06T05:17:02.461887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468443+0000","last_change":"2023-10-06T05:17:02.465489+0000","last_active":"2023-10-06T05:17:02.468443+0000","last_peered":"2023-10-06T05:17:02.468443+0000","last_clean":"2023-10-06T05:17:02.468443+0000","last_became_active":"2023-10-06T05:17:02.465329+0000","last_became_peered":"2023-10-06T05:17:02.465329+0000","last_unstale":"2023-10-06T05:17:02.468443+0000","last_undegraded":"2023-10-06T05:17:02.468443+0000","last_fullsized":"2023-10-06T05:17:02.468443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453337+0000","last_change":"2023-10-06T05:17:02.444229+0000","last_active":"2023-10-06T05:17:02.453337+0000","last_peered":"2023-10-06T05:17:02.453337+0000","last_clean":"2023-10-06T05:17:02.453337+0000","last_became_active":"2023-10-06T05:17:02.444064+0000","last_became_peered":"2023-10-06T05:17:02.444064+0000","last_unstale":"2023-10-06T05:17:02.453337+0000","last_undegraded":"2023-10-06T05:17:02.453337+0000","last_fullsized":"2023-10-06T05:17:02.453337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:39.861351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453265+0000","last_change":"2023-10-06T05:17:02.447686+0000","last_active":"2023-10-06T05:17:02.453265+0000","last_peered":"2023-10-06T05:17:02.453265+0000","last_clean":"2023-10-06T05:17:02.453265+0000","last_became_active":"2023-10-06T05:17:02.447364+0000","last_became_peered":"2023-10-06T05:17:02.447364+0000","last_unstale":"2023-10-06T05:17:02.453265+0000","last_undegraded":"2023-10-06T05:17:02.453265+0000","last_fullsized":"2023-10-06T05:17:02.453265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448537+0000","last_change":"2023-10-06T05:17:02.442095+0000","last_active":"2023-10-06T05:17:02.448537+0000","last_peered":"2023-10-06T05:17:02.448537+0000","last_clean":"2023-10-06T05:17:02.448537+0000","last_became_active":"2023-10-06T05:17:02.441645+0000","last_became_peered":"2023-10-06T05:17:02.441645+0000","last_unstale":"2023-10-06T05:17:02.448537+0000","last_undegraded":"2023-10-06T05:17:02.448537+0000","last_fullsized":"2023-10-06T05:17:02.448537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461441+0000","last_change":"2023-10-06T05:17:01.449274+0000","last_active":"2023-10-06T05:17:02.461441+0000","last_peered":"2023-10-06T05:17:02.461441+0000","last_clean":"2023-10-06T05:17:02.461441+0000","last_became_active":"2023-10-06T05:17:01.448908+0000","last_became_peered":"2023-10-06T05:17:01.448908+0000","last_unstale":"2023-10-06T05:17:02.461441+0000","last_undegraded":"2023-10-06T05:17:02.461441+0000","last_fullsized":"2023-10-06T05:17:02.461441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462280+0000","last_change":"2023-10-06T05:17:02.443949+0000","last_active":"2023-10-06T05:17:02.462280+0000","last_peered":"2023-10-06T05:17:02.462280+0000","last_clean":"2023-10-06T05:17:02.462280+0000","last_became_active":"2023-10-06T05:17:02.443677+0000","last_became_peered":"2023-10-06T05:17:02.443677+0000","last_unstale":"2023-10-06T05:17:02.462280+0000","last_undegraded":"2023-10-06T05:17:02.462280+0000","last_fullsized":"2023-10-06T05:17:02.462280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428951+0000","last_change":"2023-10-06T05:17:01.438479+0000","last_active":"2023-10-06T05:17:02.428951+0000","last_peered":"2023-10-06T05:17:02.428951+0000","last_clean":"2023-10-06T05:17:02.428951+0000","last_became_active":"2023-10-06T05:17:01.438182+0000","last_became_peered":"2023-10-06T05:17:01.438182+0000","last_unstale":"2023-10-06T05:17:02.428951+0000","last_undegraded":"2023-10-06T05:17:02.428951+0000","last_fullsized":"2023-10-06T05:17:02.428951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:42.243614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453470+0000","last_change":"2023-10-06T05:17:02.447288+0000","last_active":"2023-10-06T05:17:02.453470+0000","last_peered":"2023-10-06T05:17:02.453470+0000","last_clean":"2023-10-06T05:17:02.453470+0000","last_became_active":"2023-10-06T05:17:02.446914+0000","last_became_peered":"2023-10-06T05:17:02.446914+0000","last_unstale":"2023-10-06T05:17:02.453470+0000","last_undegraded":"2023-10-06T05:17:02.453470+0000","last_fullsized":"2023-10-06T05:17:02.453470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428771+0000","last_change":"2023-10-06T05:17:01.444309+0000","last_active":"2023-10-06T05:17:02.428771+0000","last_peered":"2023-10-06T05:17:02.428771+0000","last_clean":"2023-10-06T05:17:02.428771+0000","last_became_active":"2023-10-06T05:17:01.444026+0000","last_became_peered":"2023-10-06T05:17:01.444026+0000","last_unstale":"2023-10-06T05:17:02.428771+0000","last_undegraded":"2023-10-06T05:17:02.428771+0000","last_fullsized":"2023-10-06T05:17:02.428771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.458208+0000","last_change":"2023-10-06T05:17:02.455822+0000","last_active":"2023-10-06T05:17:02.458208+0000","last_peered":"2023-10-06T05:17:02.458208+0000","last_clean":"2023-10-06T05:17:02.458208+0000","last_became_active":"2023-10-06T05:17:02.455539+0000","last_became_peered":"2023-10-06T05:17:02.455539+0000","last_unstale":"2023-10-06T05:17:02.458208+0000","last_undegraded":"2023-10-06T05:17:02.458208+0000","last_fullsized":"2023-10-06T05:17:02.458208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434617+0000","last_change":"2023-10-06T05:17:01.453079+0000","last_active":"2023-10-06T05:17:02.434617+0000","last_peered":"2023-10-06T05:17:02.434617+0000","last_clean":"2023-10-06T05:17:02.434617+0000","last_became_active":"2023-10-06T05:17:01.452800+0000","last_became_peered":"2023-10-06T05:17:01.452800+0000","last_unstale":"2023-10-06T05:17:02.434617+0000","last_undegraded":"2023-10-06T05:17:02.434617+0000","last_fullsized":"2023-10-06T05:17:02.434617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.457891+0000","last_change":"2023-10-06T05:17:02.439162+0000","last_active":"2023-10-06T05:17:02.457891+0000","last_peered":"2023-10-06T05:17:02.457891+0000","last_clean":"2023-10-06T05:17:02.457891+0000","last_became_active":"2023-10-06T05:17:02.438810+0000","last_became_peered":"2023-10-06T05:17:02.438810+0000","last_unstale":"2023-10-06T05:17:02.457891+0000","last_undegraded":"2023-10-06T05:17:02.457891+0000","last_fullsized":"2023-10-06T05:17:02.457891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435781+0000","last_change":"2023-10-06T05:17:01.449427+0000","last_active":"2023-10-06T05:17:02.435781+0000","last_peered":"2023-10-06T05:17:02.435781+0000","last_clean":"2023-10-06T05:17:02.435781+0000","last_became_active":"2023-10-06T05:17:01.449100+0000","last_became_peered":"2023-10-06T05:17:01.449100+0000","last_unstale":"2023-10-06T05:17:02.435781+0000","last_undegraded":"2023-10-06T05:17:02.435781+0000","last_fullsized":"2023-10-06T05:17:02.435781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461821+0000","last_change":"2023-10-06T05:17:02.441035+0000","last_active":"2023-10-06T05:17:02.461821+0000","last_peered":"2023-10-06T05:17:02.461821+0000","last_clean":"2023-10-06T05:17:02.461821+0000","last_became_active":"2023-10-06T05:17:02.440422+0000","last_became_peered":"2023-10-06T05:17:02.440422+0000","last_unstale":"2023-10-06T05:17:02.461821+0000","last_undegraded":"2023-10-06T05:17:02.461821+0000","last_fullsized":"2023-10-06T05:17:02.461821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468652+0000","last_change":"2023-10-06T05:17:01.445468+0000","last_active":"2023-10-06T05:17:02.468652+0000","last_peered":"2023-10-06T05:17:02.468652+0000","last_clean":"2023-10-06T05:17:02.468652+0000","last_became_active":"2023-10-06T05:17:01.445179+0000","last_became_peered":"2023-10-06T05:17:01.445179+0000","last_unstale":"2023-10-06T05:17:02.468652+0000","last_undegraded":"2023-10-06T05:17:02.468652+0000","last_fullsized":"2023-10-06T05:17:02.468652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:01.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453479+0000","last_change":"2023-10-06T05:17:02.444872+0000","last_active":"2023-10-06T05:17:02.453479+0000","last_peered":"2023-10-06T05:17:02.453479+0000","last_clean":"2023-10-06T05:17:02.453479+0000","last_became_active":"2023-10-06T05:17:02.444565+0000","last_became_peered":"2023-10-06T05:17:02.444565+0000","last_unstale":"2023-10-06T05:17:02.453479+0000","last_undegraded":"2023-10-06T05:17:02.453479+0000","last_fullsized":"2023-10-06T05:17:02.453479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.470982+0000","last_change":"2023-10-06T05:17:01.448641+0000","last_active":"2023-10-06T05:17:02.470982+0000","last_peered":"2023-10-06T05:17:02.470982+0000","last_clean":"2023-10-06T05:17:02.470982+0000","last_became_active":"2023-10-06T05:17:01.448232+0000","last_became_peered":"2023-10-06T05:17:01.448232+0000","last_unstale":"2023-10-06T05:17:02.470982+0000","last_undegraded":"2023-10-06T05:17:02.470982+0000","last_fullsized":"2023-10-06T05:17:02.470982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447551+0000","last_change":"2023-10-06T05:17:02.443125+0000","last_active":"2023-10-06T05:17:02.447551+0000","last_peered":"2023-10-06T05:17:02.447551+0000","last_clean":"2023-10-06T05:17:02.447551+0000","last_became_active":"2023-10-06T05:17:02.442847+0000","last_became_peered":"2023-10-06T05:17:02.442847+0000","last_unstale":"2023-10-06T05:17:02.447551+0000","last_undegraded":"2023-10-06T05:17:02.447551+0000","last_fullsized":"2023-10-06T05:17:02.447551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471314+0000","last_change":"2023-10-06T05:17:01.451184+0000","last_active":"2023-10-06T05:17:02.471314+0000","last_peered":"2023-10-06T05:17:02.471314+0000","last_clean":"2023-10-06T05:17:02.471314+0000","last_became_active":"2023-10-06T05:17:01.450521+0000","last_became_peered":"2023-10-06T05:17:01.450521+0000","last_unstale":"2023-10-06T05:17:02.471314+0000","last_undegraded":"2023-10-06T05:17:02.471314+0000","last_fullsized":"2023-10-06T05:17:02.471314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:56.109170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462307+0000","last_change":"2023-10-06T05:17:02.444967+0000","last_active":"2023-10-06T05:17:02.462307+0000","last_peered":"2023-10-06T05:17:02.462307+0000","last_clean":"2023-10-06T05:17:02.462307+0000","last_became_active":"2023-10-06T05:17:02.444639+0000","last_became_peered":"2023-10-06T05:17:02.444639+0000","last_unstale":"2023-10-06T05:17:02.462307+0000","last_undegraded":"2023-10-06T05:17:02.462307+0000","last_fullsized":"2023-10-06T05:17:02.462307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:59.439667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.439263+0000","last_change":"2023-10-06T05:17:01.446625+0000","last_active":"2023-10-06T05:17:02.439263+0000","last_peered":"2023-10-06T05:17:02.439263+0000","last_clean":"2023-10-06T05:17:02.439263+0000","last_became_active":"2023-10-06T05:17:01.446249+0000","last_became_peered":"2023-10-06T05:17:01.446249+0000","last_unstale":"2023-10-06T05:17:02.439263+0000","last_undegraded":"2023-10-06T05:17:02.439263+0000","last_fullsized":"2023-10-06T05:17:02.439263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448517+0000","last_change":"2023-10-06T05:17:02.441887+0000","last_active":"2023-10-06T05:17:02.448517+0000","last_peered":"2023-10-06T05:17:02.448517+0000","last_clean":"2023-10-06T05:17:02.448517+0000","last_became_active":"2023-10-06T05:17:02.441377+0000","last_became_peered":"2023-10-06T05:17:02.441377+0000","last_unstale":"2023-10-06T05:17:02.448517+0000","last_undegraded":"2023-10-06T05:17:02.448517+0000","last_fullsized":"2023-10-06T05:17:02.448517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434243+0000","last_change":"2023-10-06T05:17:01.450424+0000","last_active":"2023-10-06T05:17:02.434243+0000","last_peered":"2023-10-06T05:17:02.434243+0000","last_clean":"2023-10-06T05:17:02.434243+0000","last_became_active":"2023-10-06T05:17:01.450072+0000","last_became_peered":"2023-10-06T05:17:01.450072+0000","last_unstale":"2023-10-06T05:17:02.434243+0000","last_undegraded":"2023-10-06T05:17:02.434243+0000","last_fullsized":"2023-10-06T05:17:02.434243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:19.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464457+0000","last_change":"2023-10-06T05:17:02.460678+0000","last_active":"2023-10-06T05:17:02.464457+0000","last_peered":"2023-10-06T05:17:02.464457+0000","last_clean":"2023-10-06T05:17:02.464457+0000","last_became_active":"2023-10-06T05:17:02.460159+0000","last_became_peered":"2023-10-06T05:17:02.460159+0000","last_unstale":"2023-10-06T05:17:02.464457+0000","last_undegraded":"2023-10-06T05:17:02.464457+0000","last_fullsized":"2023-10-06T05:17:02.464457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.467405+0000","last_change":"2023-10-06T05:17:01.447124+0000","last_active":"2023-10-06T05:17:02.467405+0000","last_peered":"2023-10-06T05:17:02.467405+0000","last_clean":"2023-10-06T05:17:02.467405+0000","last_became_active":"2023-10-06T05:17:01.446810+0000","last_became_peered":"2023-10-06T05:17:01.446810+0000","last_unstale":"2023-10-06T05:17:02.467405+0000","last_undegraded":"2023-10-06T05:17:02.467405+0000","last_fullsized":"2023-10-06T05:17:02.467405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462426+0000","last_change":"2023-10-06T05:17:02.440973+0000","last_active":"2023-10-06T05:17:02.462426+0000","last_peered":"2023-10-06T05:17:02.462426+0000","last_clean":"2023-10-06T05:17:02.462426+0000","last_became_active":"2023-10-06T05:17:02.440594+0000","last_became_peered":"2023-10-06T05:17:02.440594+0000","last_unstale":"2023-10-06T05:17:02.462426+0000","last_undegraded":"2023-10-06T05:17:02.462426+0000","last_fullsized":"2023-10-06T05:17:02.462426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:45.361281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471211+0000","last_change":"2023-10-06T05:17:01.454983+0000","last_active":"2023-10-06T05:17:02.471211+0000","last_peered":"2023-10-06T05:17:02.471211+0000","last_clean":"2023-10-06T05:17:02.471211+0000","last_became_active":"2023-10-06T05:17:01.454580+0000","last_became_peered":"2023-10-06T05:17:01.454580+0000","last_unstale":"2023-10-06T05:17:02.471211+0000","last_undegraded":"2023-10-06T05:17:02.471211+0000","last_fullsized":"2023-10-06T05:17:02.471211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:53.875873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448468+0000","last_change":"2023-10-06T05:17:02.442909+0000","last_active":"2023-10-06T05:17:02.448468+0000","last_peered":"2023-10-06T05:17:02.448468+0000","last_clean":"2023-10-06T05:17:02.448468+0000","last_became_active":"2023-10-06T05:17:02.442619+0000","last_became_peered":"2023-10-06T05:17:02.442619+0000","last_unstale":"2023-10-06T05:17:02.448468+0000","last_undegraded":"2023-10-06T05:17:02.448468+0000","last_fullsized":"2023-10-06T05:17:02.448468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471248+0000","last_change":"2023-10-06T05:17:01.454784+0000","last_active":"2023-10-06T05:17:02.471248+0000","last_peered":"2023-10-06T05:17:02.471248+0000","last_clean":"2023-10-06T05:17:02.471248+0000","last_became_active":"2023-10-06T05:17:01.454498+0000","last_became_peered":"2023-10-06T05:17:01.454498+0000","last_unstale":"2023-10-06T05:17:02.471248+0000","last_undegraded":"2023-10-06T05:17:02.471248+0000","last_fullsized":"2023-10-06T05:17:02.471248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:41.021745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452621+0000","last_change":"2023-10-06T05:17:02.446992+0000","last_active":"2023-10-06T05:17:02.452621+0000","last_peered":"2023-10-06T05:17:02.452621+0000","last_clean":"2023-10-06T05:17:02.452621+0000","last_became_active":"2023-10-06T05:17:02.445148+0000","last_became_peered":"2023-10-06T05:17:02.445148+0000","last_unstale":"2023-10-06T05:17:02.452621+0000","last_undegraded":"2023-10-06T05:17:02.452621+0000","last_fullsized":"2023-10-06T05:17:02.452621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:31.781450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.469131+0000","last_change":"2023-10-06T05:17:01.446494+0000","last_active":"2023-10-06T05:17:02.469131+0000","last_peered":"2023-10-06T05:17:02.469131+0000","last_clean":"2023-10-06T05:17:02.469131+0000","last_became_active":"2023-10-06T05:17:01.446164+0000","last_became_peered":"2023-10-06T05:17:01.446164+0000","last_unstale":"2023-10-06T05:17:02.469131+0000","last_undegraded":"2023-10-06T05:17:02.469131+0000","last_fullsized":"2023-10-06T05:17:02.469131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:45.078207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456419+0000","last_change":"2023-10-06T05:17:02.451322+0000","last_active":"2023-10-06T05:17:02.456419+0000","last_peered":"2023-10-06T05:17:02.456419+0000","last_clean":"2023-10-06T05:17:02.456419+0000","last_became_active":"2023-10-06T05:17:02.451064+0000","last_became_peered":"2023-10-06T05:17:02.451064+0000","last_unstale":"2023-10-06T05:17:02.456419+0000","last_undegraded":"2023-10-06T05:17:02.456419+0000","last_fullsized":"2023-10-06T05:17:02.456419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:46.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.432653+0000","last_change":"2023-10-06T05:17:01.443417+0000","last_active":"2023-10-06T05:17:02.432653+0000","last_peered":"2023-10-06T05:17:02.432653+0000","last_clean":"2023-10-06T05:17:02.432653+0000","last_became_active":"2023-10-06T05:17:01.443114+0000","last_became_peered":"2023-10-06T05:17:01.443114+0000","last_unstale":"2023-10-06T05:17:02.432653+0000","last_undegraded":"2023-10-06T05:17:02.432653+0000","last_fullsized":"2023-10-06T05:17:02.432653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461876+0000","last_change":"2023-10-06T05:17:02.456471+0000","last_active":"2023-10-06T05:17:02.461876+0000","last_peered":"2023-10-06T05:17:02.461876+0000","last_clean":"2023-10-06T05:17:02.461876+0000","last_became_active":"2023-10-06T05:17:02.456142+0000","last_became_peered":"2023-10-06T05:17:02.456142+0000","last_unstale":"2023-10-06T05:17:02.461876+0000","last_undegraded":"2023-10-06T05:17:02.461876+0000","last_fullsized":"2023-10-06T05:17:02.461876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.466429+0000","last_change":"2023-10-06T05:17:01.444030+0000","last_active":"2023-10-06T05:17:02.466429+0000","last_peered":"2023-10-06T05:17:02.466429+0000","last_clean":"2023-10-06T05:17:02.466429+0000","last_became_active":"2023-10-06T05:17:01.443754+0000","last_became_peered":"2023-10-06T05:17:01.443754+0000","last_unstale":"2023-10-06T05:17:02.466429+0000","last_undegraded":"2023-10-06T05:17:02.466429+0000","last_fullsized":"2023-10-06T05:17:02.466429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.440368+0000","last_change":"2023-10-06T05:17:02.437195+0000","last_active":"2023-10-06T05:17:02.440368+0000","last_peered":"2023-10-06T05:17:02.440368+0000","last_clean":"2023-10-06T05:17:02.440368+0000","last_became_active":"2023-10-06T05:17:02.436917+0000","last_became_peered":"2023-10-06T05:17:02.436917+0000","last_unstale":"2023-10-06T05:17:02.440368+0000","last_undegraded":"2023-10-06T05:17:02.440368+0000","last_fullsized":"2023-10-06T05:17:02.440368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435134+0000","last_change":"2023-10-06T05:17:01.454676+0000","last_active":"2023-10-06T05:17:02.435134+0000","last_peered":"2023-10-06T05:17:02.435134+0000","last_clean":"2023-10-06T05:17:02.435134+0000","last_became_active":"2023-10-06T05:17:01.454385+0000","last_became_peered":"2023-10-06T05:17:01.454385+0000","last_unstale":"2023-10-06T05:17:02.435134+0000","last_undegraded":"2023-10-06T05:17:02.435134+0000","last_fullsized":"2023-10-06T05:17:02.435134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453140+0000","last_change":"2023-10-06T05:17:02.449388+0000","last_active":"2023-10-06T05:17:02.453140+0000","last_peered":"2023-10-06T05:17:02.453140+0000","last_clean":"2023-10-06T05:17:02.453140+0000","last_became_active":"2023-10-06T05:17:02.448868+0000","last_became_peered":"2023-10-06T05:17:02.448868+0000","last_unstale":"2023-10-06T05:17:02.453140+0000","last_undegraded":"2023-10-06T05:17:02.453140+0000","last_fullsized":"2023-10-06T05:17:02.453140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428450+0000","last_change":"2023-10-06T05:17:01.448698+0000","last_active":"2023-10-06T05:17:02.428450+0000","last_peered":"2023-10-06T05:17:02.428450+0000","last_clean":"2023-10-06T05:17:02.428450+0000","last_became_active":"2023-10-06T05:17:01.448084+0000","last_became_peered":"2023-10-06T05:17:01.448084+0000","last_unstale":"2023-10-06T05:17:02.428450+0000","last_undegraded":"2023-10-06T05:17:02.428450+0000","last_fullsized":"2023-10-06T05:17:02.428450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452412+0000","last_change":"2023-10-06T05:17:02.446522+0000","last_active":"2023-10-06T05:17:02.452412+0000","last_peered":"2023-10-06T05:17:02.452412+0000","last_clean":"2023-10-06T05:17:02.452412+0000","last_became_active":"2023-10-06T05:17:02.446203+0000","last_became_peered":"2023-10-06T05:17:02.446203+0000","last_unstale":"2023-10-06T05:17:02.452412+0000","last_undegraded":"2023-10-06T05:17:02.452412+0000","last_fullsized":"2023-10-06T05:17:02.452412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:28.806289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428705+0000","last_change":"2023-10-06T05:17:01.441195+0000","last_active":"2023-10-06T05:17:02.428705+0000","last_peered":"2023-10-06T05:17:02.428705+0000","last_clean":"2023-10-06T05:17:02.428705+0000","last_became_active":"2023-10-06T05:17:01.440903+0000","last_became_peered":"2023-10-06T05:17:01.440903+0000","last_unstale":"2023-10-06T05:17:02.428705+0000","last_undegraded":"2023-10-06T05:17:02.428705+0000","last_fullsized":"2023-10-06T05:17:02.428705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453318+0000","last_change":"2023-10-06T05:17:02.444240+0000","last_active":"2023-10-06T05:17:02.453318+0000","last_peered":"2023-10-06T05:17:02.453318+0000","last_clean":"2023-10-06T05:17:02.453318+0000","last_became_active":"2023-10-06T05:17:02.443717+0000","last_became_peered":"2023-10-06T05:17:02.443717+0000","last_unstale":"2023-10-06T05:17:02.453318+0000","last_undegraded":"2023-10-06T05:17:02.453318+0000","last_fullsized":"2023-10-06T05:17:02.453318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464271+0000","last_change":"2023-10-06T05:17:01.443363+0000","last_active":"2023-10-06T05:17:02.464271+0000","last_peered":"2023-10-06T05:17:02.464271+0000","last_clean":"2023-10-06T05:17:02.464271+0000","last_became_active":"2023-10-06T05:17:01.443131+0000","last_became_peered":"2023-10-06T05:17:01.443131+0000","last_unstale":"2023-10-06T05:17:02.464271+0000","last_undegraded":"2023-10-06T05:17:02.464271+0000","last_fullsized":"2023-10-06T05:17:02.464271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428924+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:17:02.428924+0000","last_peered":"2023-10-06T05:17:02.428924+0000","last_clean":"2023-10-06T05:17:02.428924+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:17:02.428924+0000","last_undegraded":"2023-10-06T05:17:02.428924+0000","last_fullsized":"2023-10-06T05:17:02.428924+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464122+0000","last_change":"2023-10-06T05:17:02.442449+0000","last_active":"2023-10-06T05:17:02.464122+0000","last_peered":"2023-10-06T05:17:02.464122+0000","last_clean":"2023-10-06T05:17:02.464122+0000","last_became_active":"2023-10-06T05:17:02.442135+0000","last_became_peered":"2023-10-06T05:17:02.442135+0000","last_unstale":"2023-10-06T05:17:02.464122+0000","last_undegraded":"2023-10-06T05:17:02.464122+0000","last_fullsized":"2023-10-06T05:17:02.464122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:31.781450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.427977+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:17:02.427977+0000","last_peered":"2023-10-06T05:17:02.427977+0000","last_clean":"2023-10-06T05:17:02.427977+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:17:02.427977+0000","last_undegraded":"2023-10-06T05:17:02.427977+0000","last_fullsized":"2023-10-06T05:17:02.427977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.436400+0000","last_change":"2023-10-06T05:17:01.448082+0000","last_active":"2023-10-06T05:17:02.436400+0000","last_peered":"2023-10-06T05:17:02.436400+0000","last_clean":"2023-10-06T05:17:02.436400+0000","last_became_active":"2023-10-06T05:17:01.447418+0000","last_became_peered":"2023-10-06T05:17:01.447418+0000","last_unstale":"2023-10-06T05:17:02.436400+0000","last_undegraded":"2023-10-06T05:17:02.436400+0000","last_fullsized":"2023-10-06T05:17:02.436400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452602+0000","last_change":"2023-10-06T05:17:02.445401+0000","last_active":"2023-10-06T05:17:02.452602+0000","last_peered":"2023-10-06T05:17:02.452602+0000","last_clean":"2023-10-06T05:17:02.452602+0000","last_became_active":"2023-10-06T05:17:02.444770+0000","last_became_peered":"2023-10-06T05:17:02.444770+0000","last_unstale":"2023-10-06T05:17:02.452602+0000","last_undegraded":"2023-10-06T05:17:02.452602+0000","last_fullsized":"2023-10-06T05:17:02.452602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429051+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:17:02.429051+0000","last_peered":"2023-10-06T05:17:02.429051+0000","last_clean":"2023-10-06T05:17:02.429051+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:17:02.429051+0000","last_undegraded":"2023-10-06T05:17:02.429051+0000","last_fullsized":"2023-10-06T05:17:02.429051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.436179+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:17:02.436179+0000","last_peered":"2023-10-06T05:17:02.436179+0000","last_clean":"2023-10-06T05:17:02.436179+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:17:02.436179+0000","last_undegraded":"2023-10-06T05:17:02.436179+0000","last_fullsized":"2023-10-06T05:17:02.436179+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.432715+0000","last_change":"2023-10-06T05:17:01.447882+0000","last_active":"2023-10-06T05:17:02.432715+0000","last_peered":"2023-10-06T05:17:02.432715+0000","last_clean":"2023-10-06T05:17:02.432715+0000","last_became_active":"2023-10-06T05:17:01.447617+0000","last_became_peered":"2023-10-06T05:17:01.447617+0000","last_unstale":"2023-10-06T05:17:02.432715+0000","last_undegraded":"2023-10-06T05:17:02.432715+0000","last_fullsized":"2023-10-06T05:17:02.432715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.460775+0000","last_change":"2023-10-06T05:17:02.455716+0000","last_active":"2023-10-06T05:17:02.460775+0000","last_peered":"2023-10-06T05:17:02.460775+0000","last_clean":"2023-10-06T05:17:02.460775+0000","last_became_active":"2023-10-06T05:17:02.455388+0000","last_became_peered":"2023-10-06T05:17:02.455388+0000","last_unstale":"2023-10-06T05:17:02.460775+0000","last_undegraded":"2023-10-06T05:17:02.460775+0000","last_fullsized":"2023-10-06T05:17:02.460775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434749+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:17:02.434749+0000","last_peered":"2023-10-06T05:17:02.434749+0000","last_clean":"2023-10-06T05:17:02.434749+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:17:02.434749+0000","last_undegraded":"2023-10-06T05:17:02.434749+0000","last_fullsized":"2023-10-06T05:17:02.434749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429080+0000","last_change":"2023-10-06T05:17:01.449606+0000","last_active":"2023-10-06T05:17:02.429080+0000","last_peered":"2023-10-06T05:17:02.429080+0000","last_clean":"2023-10-06T05:17:02.429080+0000","last_became_active":"2023-10-06T05:17:01.449380+0000","last_became_peered":"2023-10-06T05:17:01.449380+0000","last_unstale":"2023-10-06T05:17:02.429080+0000","last_undegraded":"2023-10-06T05:17:02.429080+0000","last_fullsized":"2023-10-06T05:17:02.429080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453300+0000","last_change":"2023-10-06T05:17:02.444044+0000","last_active":"2023-10-06T05:17:02.453300+0000","last_peered":"2023-10-06T05:17:02.453300+0000","last_clean":"2023-10-06T05:17:02.453300+0000","last_became_active":"2023-10-06T05:17:02.443659+0000","last_became_peered":"2023-10-06T05:17:02.443659+0000","last_unstale":"2023-10-06T05:17:02.453300+0000","last_undegraded":"2023-10-06T05:17:02.453300+0000","last_fullsized":"2023-10-06T05:17:02.453300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:46.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435753+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:17:02.435753+0000","last_peered":"2023-10-06T05:17:02.435753+0000","last_clean":"2023-10-06T05:17:02.435753+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:17:02.435753+0000","last_undegraded":"2023-10-06T05:17:02.435753+0000","last_fullsized":"2023-10-06T05:17:02.435753+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.427375+0000","last_change":"2023-10-06T05:17:01.441700+0000","last_active":"2023-10-06T05:17:02.427375+0000","last_peered":"2023-10-06T05:17:02.427375+0000","last_clean":"2023-10-06T05:17:02.427375+0000","last_became_active":"2023-10-06T05:17:01.441370+0000","last_became_peered":"2023-10-06T05:17:01.441370+0000","last_unstale":"2023-10-06T05:17:02.427375+0000","last_undegraded":"2023-10-06T05:17:02.427375+0000","last_fullsized":"2023-10-06T05:17:02.427375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:01.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450934+0000","last_change":"2023-10-06T05:17:02.446567+0000","last_active":"2023-10-06T05:17:02.450934+0000","last_peered":"2023-10-06T05:17:02.450934+0000","last_clean":"2023-10-06T05:17:02.450934+0000","last_became_active":"2023-10-06T05:17:02.446263+0000","last_became_peered":"2023-10-06T05:17:02.446263+0000","last_unstale":"2023-10-06T05:17:02.450934+0000","last_undegraded":"2023-10-06T05:17:02.450934+0000","last_fullsized":"2023-10-06T05:17:02.450934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.433849+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:17:02.433849+0000","last_peered":"2023-10-06T05:17:02.433849+0000","last_clean":"2023-10-06T05:17:02.433849+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:17:02.433849+0000","last_undegraded":"2023-10-06T05:17:02.433849+0000","last_fullsized":"2023-10-06T05:17:02.433849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429218+0000","last_change":"2023-10-06T05:17:01.448636+0000","last_active":"2023-10-06T05:17:02.429218+0000","last_peered":"2023-10-06T05:17:02.429218+0000","last_clean":"2023-10-06T05:17:02.429218+0000","last_became_active":"2023-10-06T05:17:01.446188+0000","last_became_peered":"2023-10-06T05:17:01.446188+0000","last_unstale":"2023-10-06T05:17:02.429218+0000","last_undegraded":"2023-10-06T05:17:02.429218+0000","last_fullsized":"2023-10-06T05:17:02.429218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453488+0000","last_change":"2023-10-06T05:17:02.447390+0000","last_active":"2023-10-06T05:17:02.453488+0000","last_peered":"2023-10-06T05:17:02.453488+0000","last_clean":"2023-10-06T05:17:02.453488+0000","last_became_active":"2023-10-06T05:17:02.447118+0000","last_became_peered":"2023-10-06T05:17:02.447118+0000","last_unstale":"2023-10-06T05:17:02.453488+0000","last_undegraded":"2023-10-06T05:17:02.453488+0000","last_fullsized":"2023-10-06T05:17:02.453488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.439722+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:17:02.439722+0000","last_peered":"2023-10-06T05:17:02.439722+0000","last_clean":"2023-10-06T05:17:02.439722+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:17:02.439722+0000","last_undegraded":"2023-10-06T05:17:02.439722+0000","last_fullsized":"2023-10-06T05:17:02.439722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.431999+0000","last_change":"2023-10-06T05:17:01.449227+0000","last_active":"2023-10-06T05:17:02.431999+0000","last_peered":"2023-10-06T05:17:02.431999+0000","last_clean":"2023-10-06T05:17:02.431999+0000","last_became_active":"2023-10-06T05:17:01.448956+0000","last_became_peered":"2023-10-06T05:17:01.448956+0000","last_unstale":"2023-10-06T05:17:02.431999+0000","last_undegraded":"2023-10-06T05:17:02.431999+0000","last_fullsized":"2023-10-06T05:17:02.431999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450001+0000","last_change":"2023-10-06T05:17:02.442854+0000","last_active":"2023-10-06T05:17:02.450001+0000","last_peered":"2023-10-06T05:17:02.450001+0000","last_clean":"2023-10-06T05:17:02.450001+0000","last_became_active":"2023-10-06T05:17:02.442424+0000","last_became_peered":"2023-10-06T05:17:02.442424+0000","last_unstale":"2023-10-06T05:17:02.450001+0000","last_undegraded":"2023-10-06T05:17:02.450001+0000","last_fullsized":"2023-10-06T05:17:02.450001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.437608+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:17:02.437608+0000","last_peered":"2023-10-06T05:17:02.437608+0000","last_clean":"2023-10-06T05:17:02.437608+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:17:02.437608+0000","last_undegraded":"2023-10-06T05:17:02.437608+0000","last_fullsized":"2023-10-06T05:17:02.437608+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.433246+0000","last_change":"2023-10-06T05:17:01.444318+0000","last_active":"2023-10-06T05:17:02.433246+0000","last_peered":"2023-10-06T05:17:02.433246+0000","last_clean":"2023-10-06T05:17:02.433246+0000","last_became_active":"2023-10-06T05:17:01.444042+0000","last_became_peered":"2023-10-06T05:17:01.444042+0000","last_unstale":"2023-10-06T05:17:02.433246+0000","last_undegraded":"2023-10-06T05:17:02.433246+0000","last_fullsized":"2023-10-06T05:17:02.433246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453458+0000","last_change":"2023-10-06T05:17:02.443796+0000","last_active":"2023-10-06T05:17:02.453458+0000","last_peered":"2023-10-06T05:17:02.453458+0000","last_clean":"2023-10-06T05:17:02.453458+0000","last_became_active":"2023-10-06T05:17:02.443364+0000","last_became_peered":"2023-10-06T05:17:02.443364+0000","last_unstale":"2023-10-06T05:17:02.453458+0000","last_undegraded":"2023-10-06T05:17:02.453458+0000","last_fullsized":"2023-10-06T05:17:02.453458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:07.184397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465585+0000","last_change":"2023-10-06T05:17:01.451251+0000","last_active":"2023-10-06T05:17:02.465585+0000","last_peered":"2023-10-06T05:17:02.465585+0000","last_clean":"2023-10-06T05:17:02.465585+0000","last_became_active":"2023-10-06T05:17:01.450983+0000","last_became_peered":"2023-10-06T05:17:01.450983+0000","last_unstale":"2023-10-06T05:17:02.465585+0000","last_undegraded":"2023-10-06T05:17:02.465585+0000","last_fullsized":"2023-10-06T05:17:02.465585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456082+0000","last_change":"2023-10-06T05:17:02.451794+0000","last_active":"2023-10-06T05:17:02.456082+0000","last_peered":"2023-10-06T05:17:02.456082+0000","last_clean":"2023-10-06T05:17:02.456082+0000","last_became_active":"2023-10-06T05:17:02.451533+0000","last_became_peered":"2023-10-06T05:17:02.451533+0000","last_unstale":"2023-10-06T05:17:02.456082+0000","last_undegraded":"2023-10-06T05:17:02.456082+0000","last_fullsized":"2023-10-06T05:17:02.456082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:23.939592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471267+0000","last_change":"2023-10-06T05:17:01.449818+0000","last_active":"2023-10-06T05:17:02.471267+0000","last_peered":"2023-10-06T05:17:02.471267+0000","last_clean":"2023-10-06T05:17:02.471267+0000","last_became_active":"2023-10-06T05:17:01.449477+0000","last_became_peered":"2023-10-06T05:17:01.449477+0000","last_unstale":"2023-10-06T05:17:02.471267+0000","last_undegraded":"2023-10-06T05:17:02.471267+0000","last_fullsized":"2023-10-06T05:17:02.471267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447570+0000","last_change":"2023-10-06T05:17:02.443687+0000","last_active":"2023-10-06T05:17:02.447570+0000","last_peered":"2023-10-06T05:17:02.447570+0000","last_clean":"2023-10-06T05:17:02.447570+0000","last_became_active":"2023-10-06T05:17:02.443433+0000","last_became_peered":"2023-10-06T05:17:02.443433+0000","last_unstale":"2023-10-06T05:17:02.447570+0000","last_undegraded":"2023-10-06T05:17:02.447570+0000","last_fullsized":"2023-10-06T05:17:02.447570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:29.361752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471253+0000","last_change":"2023-10-06T05:17:01.449867+0000","last_active":"2023-10-06T05:17:02.471253+0000","last_peered":"2023-10-06T05:17:02.471253+0000","last_clean":"2023-10-06T05:17:02.471253+0000","last_became_active":"2023-10-06T05:17:01.449452+0000","last_became_peered":"2023-10-06T05:17:01.449452+0000","last_unstale":"2023-10-06T05:17:02.471253+0000","last_undegraded":"2023-10-06T05:17:02.471253+0000","last_fullsized":"2023-10-06T05:17:02.471253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447425+0000","last_change":"2023-10-06T05:17:02.439184+0000","last_active":"2023-10-06T05:17:02.447425+0000","last_peered":"2023-10-06T05:17:02.447425+0000","last_clean":"2023-10-06T05:17:02.447425+0000","last_became_active":"2023-10-06T05:17:02.438893+0000","last_became_peered":"2023-10-06T05:17:02.438893+0000","last_unstale":"2023-10-06T05:17:02.447425+0000","last_undegraded":"2023-10-06T05:17:02.447425+0000","last_fullsized":"2023-10-06T05:17:02.447425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:28.005164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.463116+0000","last_change":"2023-10-06T05:17:01.441308+0000","last_active":"2023-10-06T05:17:02.463116+0000","last_peered":"2023-10-06T05:17:02.463116+0000","last_clean":"2023-10-06T05:17:02.463116+0000","last_became_active":"2023-10-06T05:17:01.440904+0000","last_became_peered":"2023-10-06T05:17:01.440904+0000","last_unstale":"2023-10-06T05:17:02.463116+0000","last_undegraded":"2023-10-06T05:17:02.463116+0000","last_fullsized":"2023-10-06T05:17:02.463116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:52.916083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461869+0000","last_change":"2023-10-06T05:17:02.447801+0000","last_active":"2023-10-06T05:17:02.461869+0000","last_peered":"2023-10-06T05:17:02.461869+0000","last_clean":"2023-10-06T05:17:02.461869+0000","last_became_active":"2023-10-06T05:17:02.447361+0000","last_became_peered":"2023-10-06T05:17:02.447361+0000","last_unstale":"2023-10-06T05:17:02.461869+0000","last_undegraded":"2023-10-06T05:17:02.461869+0000","last_fullsized":"2023-10-06T05:17:02.461869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465330+0000","last_change":"2023-10-06T05:17:01.446169+0000","last_active":"2023-10-06T05:17:02.465330+0000","last_peered":"2023-10-06T05:17:02.465330+0000","last_clean":"2023-10-06T05:17:02.465330+0000","last_became_active":"2023-10-06T05:17:01.445836+0000","last_became_peered":"2023-10-06T05:17:01.445836+0000","last_unstale":"2023-10-06T05:17:02.465330+0000","last_undegraded":"2023-10-06T05:17:02.465330+0000","last_fullsized":"2023-10-06T05:17:02.465330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:52.916083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452461+0000","last_change":"2023-10-06T05:17:02.447171+0000","last_active":"2023-10-06T05:17:02.452461+0000","last_peered":"2023-10-06T05:17:02.452461+0000","last_clean":"2023-10-06T05:17:02.452461+0000","last_became_active":"2023-10-06T05:17:02.446884+0000","last_became_peered":"2023-10-06T05:17:02.446884+0000","last_unstale":"2023-10-06T05:17:02.452461+0000","last_undegraded":"2023-10-06T05:17:02.452461+0000","last_fullsized":"2023-10-06T05:17:02.452461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456523+0000","last_change":"2023-10-06T05:17:02.451494+0000","last_active":"2023-10-06T05:17:02.456523+0000","last_peered":"2023-10-06T05:17:02.456523+0000","last_clean":"2023-10-06T05:17:02.456523+0000","last_became_active":"2023-10-06T05:17:02.451213+0000","last_became_peered":"2023-10-06T05:17:02.451213+0000","last_unstale":"2023-10-06T05:17:02.456523+0000","last_undegraded":"2023-10-06T05:17:02.456523+0000","last_fullsized":"2023-10-06T05:17:02.456523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:54.883797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461850+0000","last_change":"2023-10-06T05:17:02.444575+0000","last_active":"2023-10-06T05:17:02.461850+0000","last_peered":"2023-10-06T05:17:02.461850+0000","last_clean":"2023-10-06T05:17:02.461850+0000","last_became_active":"2023-10-06T05:17:02.444226+0000","last_became_peered":"2023-10-06T05:17:02.444226+0000","last_unstale":"2023-10-06T05:17:02.461850+0000","last_undegraded":"2023-10-06T05:17:02.461850+0000","last_fullsized":"2023-10-06T05:17:02.461850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452618+0000","last_change":"2023-10-06T05:17:02.447774+0000","last_active":"2023-10-06T05:17:02.452618+0000","last_peered":"2023-10-06T05:17:02.452618+0000","last_clean":"2023-10-06T05:17:02.452618+0000","last_became_active":"2023-10-06T05:17:02.447500+0000","last_became_peered":"2023-10-06T05:17:02.447500+0000","last_unstale":"2023-10-06T05:17:02.452618+0000","last_undegraded":"2023-10-06T05:17:02.452618+0000","last_fullsized":"2023-10-06T05:17:02.452618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:01.131607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462293+0000","last_change":"2023-10-06T05:17:02.441565+0000","last_active":"2023-10-06T05:17:02.462293+0000","last_peered":"2023-10-06T05:17:02.462293+0000","last_clean":"2023-10-06T05:17:02.462293+0000","last_became_active":"2023-10-06T05:17:02.441272+0000","last_became_peered":"2023-10-06T05:17:02.441272+0000","last_unstale":"2023-10-06T05:17:02.462293+0000","last_undegraded":"2023-10-06T05:17:02.462293+0000","last_fullsized":"2023-10-06T05:17:02.462293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:28.806289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453469+0000","last_change":"2023-10-06T05:17:02.443555+0000","last_active":"2023-10-06T05:17:02.453469+0000","last_peered":"2023-10-06T05:17:02.453469+0000","last_clean":"2023-10-06T05:17:02.453469+0000","last_became_active":"2023-10-06T05:17:02.443268+0000","last_became_peered":"2023-10-06T05:17:02.443268+0000","last_unstale":"2023-10-06T05:17:02.453469+0000","last_undegraded":"2023-10-06T05:17:02.453469+0000","last_fullsized":"2023-10-06T05:17:02.453469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":659663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":67543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298068,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073772,"statfs":{"total":96636764160,"available":96331542528,"internally_reserved":0,"allocated":798720,"data_stored":664552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297480,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074360,"statfs":{"total":96636764160,"available":96332144640,"internally_reserved":0,"allocated":196608,"data_stored":68097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":67543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297484,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074356,"statfs":{"total":96636764160,"available":96332140544,"internally_reserved":0,"allocated":200704,"data_stored":68167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:17:07.333 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:17:07.333 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T05:17:07.333 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T05:17:07.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T05:17:07.334 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T05:17:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.559+0000 7fa9fe9ff700 1 Processor -- start 2023-10-06T05:17:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.559+0000 7fa9fe9ff700 1 -- start start 2023-10-06T05:17:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.560+0000 7fa9fe9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa9f815a930 0x7fa9f80cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.560+0000 7fa9fe9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f80ce090 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.560+0000 7fa9fe9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f80cb6f0 con 0x7fa9f80cb1b0 2023-10-06T05:17:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.560+0000 7fa9fe9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f80cb9d0 con 0x7fa9f815a930 2023-10-06T05:17:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.560+0000 7fa9fcf9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa9f80cf5a0 0x7fa9f8163240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44680/0 (socket says 172.21.15.31:44680) 2023-10-06T05:17:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.560+0000 7fa9fcf9c700 1 -- 172.21.15.31:0/3854426994 learned_addr learned my addr 172.21.15.31:0/3854426994 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.561+0000 7fa9f7fff700 1 --2- 172.21.15.31:0/3854426994 >> v2:172.21.15.62:3300/0 conn(0x7fa9f815a930 0x7fa9f80cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.561+0000 7fa9f6ffd700 1 -- 172.21.15.31:0/3854426994 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234653270 0 0) 0x7fa9f80ce090 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.561+0000 7fa9f6ffd700 1 -- 172.21.15.31:0/3854426994 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9e0003570 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.561+0000 7fa9f6ffd700 1 -- 172.21.15.31:0/3854426994 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2542313663 0 0) 0x7fa9e0003570 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.561+0000 7fa9f7fff700 1 -- 172.21.15.31:0/3854426994 >> v1:172.21.15.62:6789/0 conn(0x7fa9f80cb1b0 legacy=0x7fa9f80ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.561+0000 7fa9f7fff700 1 -- 172.21.15.31:0/3854426994 >> v1:172.21.15.31:6789/0 conn(0x7fa9f80cf5a0 legacy=0x7fa9f8163240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.561+0000 7fa9f7fff700 1 -- 172.21.15.31:0/3854426994 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80cbcb0 con 0x7fa9f815a930 2023-10-06T05:17:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.561+0000 7fa9f7fff700 1 --2- 172.21.15.31:0/3854426994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9f815a930 0x7fa9f80cac70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80098d0 tx=0x7fa9e8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a62397043ddfc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.562+0000 7fa9f6ffd700 1 -- 172.21.15.31:0/3854426994 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00017f0 con 0x7fa9f815a930 2023-10-06T05:17:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.562+0000 7fa9f6ffd700 1 -- 172.21.15.31:0/3854426994 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e8011420 con 0x7fa9f815a930 2023-10-06T05:17:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.562+0000 7fa9f6ffd700 1 -- 172.21.15.31:0/3854426994 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80117c0 con 0x7fa9f815a930 2023-10-06T05:17:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.562+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/3854426994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9f815a930 msgr2=0x7fa9f80cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.562+0000 7fa9fe9ff700 1 --2- 172.21.15.31:0/3854426994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9f815a930 0x7fa9f80cac70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80098d0 tx=0x7fa9e8009120 comp rx=0 tx=0).stop 2023-10-06T05:17:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.562+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/3854426994 shutdown_connections 2023-10-06T05:17:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.562+0000 7fa9fe9ff700 1 --2- 172.21.15.31:0/3854426994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9f815a930 0x7fa9f80cac70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.562+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/3854426994 >> 172.21.15.31:0/3854426994 conn(0x7fa9f8099720 msgr2=0x7fa9f806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.562+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/3854426994 shutdown_connections 2023-10-06T05:17:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.563+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/3854426994 wait complete. 2023-10-06T05:17:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.564+0000 7fa9fe9ff700 1 Processor -- start 2023-10-06T05:17:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.564+0000 7fa9fe9ff700 1 -- start start 2023-10-06T05:17:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.565+0000 7fa9fe9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9f80cb1b0 0x7fa9f80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.565+0000 7fa9fe9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f80cf5a0 0x7fa9f816a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.565+0000 7fa9fe9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9f815a930 0x7fa9f809d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.565+0000 7fa9fe9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f80b48f0 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.565+0000 7fa9fe9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f816ac00 con 0x7fa9f80cb1b0 2023-10-06T05:17:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.565+0000 7fa9fe9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f80c6a30 con 0x7fa9f815a930 2023-10-06T05:17:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.565+0000 7fa9f77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f80cf5a0 0x7fa9f816a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.565+0000 7fa9f77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f80cf5a0 0x7fa9f816a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34154/0 (socket says 172.21.15.31:34154) 2023-10-06T05:17:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.565+0000 7fa9f77fe700 1 -- 172.21.15.31:0/1264692318 learned_addr learned my addr 172.21.15.31:0/1264692318 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9f7fff700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9f80cb1b0 0x7fa9f80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9fcf9c700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9f815a930 0x7fa9f809d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9f77fe700 1 -- 172.21.15.31:0/1264692318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9f815a930 msgr2=0x7fa9f809d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9f77fe700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9f815a930 0x7fa9f809d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9f77fe700 1 -- 172.21.15.31:0/1264692318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9f80cb1b0 msgr2=0x7fa9f80b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9f77fe700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9f80cb1b0 0x7fa9f80b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9f77fe700 1 -- 172.21.15.31:0/1264692318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80093c0 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9fcf9c700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9f815a930 0x7fa9f809d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9f77fe700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f80cf5a0 0x7fa9f816a6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c590 tx=0x7fa9ec00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9dffff700 1 -- 172.21.15.31:0/1264692318 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec011590 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9dffff700 1 -- 172.21.15.31:0/1264692318 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec0116f0 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9dffff700 1 -- 172.21.15.31:0/1264692318 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec011ab0 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/1264692318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f809de90 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.566+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/1264692318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f80b4dc0 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.567+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/1264692318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f804c380 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.569+0000 7fa9dffff700 1 -- 172.21.15.31:0/1264692318 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9ec01b3b0 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.570+0000 7fa9dffff700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa9e4066f30 0x7fa9e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.570+0000 7fa9dffff700 1 -- 172.21.15.31:0/1264692318 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa9ec083ce0 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.570+0000 7fa9f7fff700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa9e4066f30 0x7fa9e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.570+0000 7fa9f7fff700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa9e4066f30 0x7fa9e40693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009390 tx=0x7fa9e8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:07.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.574+0000 7fa9dffff700 1 -- 172.21.15.31:0/1264692318 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9ec050d50 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.830+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/1264692318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa9f80b2be0 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.834+0000 7fa9dffff700 1 -- 172.21.15.31:0/1264692318 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163702 (secure 0 0 0) 0x7fa9ec017020 con 0x7fa9f80cf5a0 2023-10-06T05:17:07.836 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/1264692318 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa9e4066f30 msgr2=0x7fa9e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa9e4066f30 0x7fa9e40693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009390 tx=0x7fa9e8016000 comp rx=0 tx=0).stop 2023-10-06T05:17:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/1264692318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f80cf5a0 msgr2=0x7fa9f816a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f80cf5a0 0x7fa9f816a6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c590 tx=0x7fa9ec00c950 comp rx=0 tx=0).stop 2023-10-06T05:17:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/1264692318 shutdown_connections 2023-10-06T05:17:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9f815a930 0x7fa9f809d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9f80cb1b0 0x7fa9f80b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa9e4066f30 0x7fa9e40693e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 --2- 172.21.15.31:0/1264692318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f80cf5a0 0x7fa9f816a6c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/1264692318 >> 172.21.15.31:0/1264692318 conn(0x7fa9f8099720 msgr2=0x7fa9f8153950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/1264692318 shutdown_connections 2023-10-06T05:17:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:07.837+0000 7fa9fe9ff700 1 -- 172.21.15.31:0/1264692318 wait complete. 2023-10-06T05:17:07.856 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":114556},{"type":"v1","addr":"172.21.15.31:6833","nonce":114556}]},"active_addr":"172.21.15.31:6833/114556","active_change":"2023-10-06T05:16:48.048502+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4101,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1652493878}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3647032433}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1563541111}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2247400373}]}]} 2023-10-06T05:17:07.860 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T05:17:07.860 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T05:17:07.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.055+0000 7f55ed7ff700 1 Processor -- start 2023-10-06T05:17:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.055+0000 7f55ed7ff700 1 -- start start 2023-10-06T05:17:08.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.056+0000 7f55ed7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e8159500 0x7f55e80b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.056+0000 7f55ed7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f55e815d2d0 0x7f55e80cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.056+0000 7f55ed7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e80c3f40 con 0x7f55e81598d0 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.056+0000 7f55ed7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e80b2860 con 0x7f55e815d2d0 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.056+0000 7f55ed7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e815d7d0 con 0x7f55e8159500 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e8159500 0x7f55e80b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e67fc700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f55e815d2d0 0x7f55e80cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e8159500 0x7f55e80b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49516/0 (socket says 172.21.15.31:49516) 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e6ffd700 1 -- 172.21.15.31:0/2147056215 learned_addr learned my addr 172.21.15.31:0/2147056215 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e77fe700 1 --1- 172.21.15.31:0/2147056215 >> v1:172.21.15.62:6789/0 conn(0x7f55e81598d0 0x7f55e80aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:34068/0 (socket says 172.21.15.31:34068) 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e6ffd700 1 -- 172.21.15.31:0/2147056215 >> v2:172.21.15.62:3300/0 conn(0x7f55e815d2d0 msgr2=0x7f55e80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e6ffd700 1 --2- 172.21.15.31:0/2147056215 >> v2:172.21.15.62:3300/0 conn(0x7f55e815d2d0 0x7f55e80cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e6ffd700 1 -- 172.21.15.31:0/2147056215 >> v1:172.21.15.62:6789/0 conn(0x7f55e81598d0 legacy=0x7f55e80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e6ffd700 1 -- 172.21.15.31:0/2147056215 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e80cb000 con 0x7f55e8159500 2023-10-06T05:17:08.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e5ffb700 1 -- 172.21.15.31:0/2147056215 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273224081 0 0) 0x7f55e80c3f40 con 0x7f55e81598d0 2023-10-06T05:17:08.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.057+0000 7f55e6ffd700 1 --2- 172.21.15.31:0/2147056215 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e8159500 0x7f55e80b0ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f55d80099b0 tx=0x7f55d8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc1c5a2ee264ab6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:08.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55e5ffb700 1 -- 172.21.15.31:0/2147056215 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8009ea0 con 0x7f55e8159500 2023-10-06T05:17:08.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55e5ffb700 1 -- 172.21.15.31:0/2147056215 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d800b860 con 0x7f55e8159500 2023-10-06T05:17:08.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55e5ffb700 1 -- 172.21.15.31:0/2147056215 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d800bc00 con 0x7f55e8159500 2023-10-06T05:17:08.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55ed7ff700 1 -- 172.21.15.31:0/2147056215 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e8159500 msgr2=0x7f55e80b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55ed7ff700 1 --2- 172.21.15.31:0/2147056215 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e8159500 0x7f55e80b0ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f55d80099b0 tx=0x7f55d8009250 comp rx=0 tx=0).stop 2023-10-06T05:17:08.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55ed7ff700 1 -- 172.21.15.31:0/2147056215 shutdown_connections 2023-10-06T05:17:08.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55ed7ff700 1 --2- 172.21.15.31:0/2147056215 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e8159500 0x7f55e80b0ab0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55ed7ff700 1 --2- 172.21.15.31:0/2147056215 >> v2:172.21.15.62:3300/0 conn(0x7f55e815d2d0 0x7f55e80cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55ed7ff700 1 -- 172.21.15.31:0/2147056215 >> 172.21.15.31:0/2147056215 conn(0x7f55e806ec90 msgr2=0x7f55e806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:08.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55ed7ff700 1 -- 172.21.15.31:0/2147056215 shutdown_connections 2023-10-06T05:17:08.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.058+0000 7f55ed7ff700 1 -- 172.21.15.31:0/2147056215 wait complete. 2023-10-06T05:17:08.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.060+0000 7f55ed7ff700 1 Processor -- start 2023-10-06T05:17:08.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.060+0000 7f55ed7ff700 1 -- start start 2023-10-06T05:17:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55ed7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e8159500 0x7f55e8158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55ed7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81598d0 0x7f55e80c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55ed7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e815d2d0 0x7f55e80c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55ed7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80c7780 con 0x7f55e81598d0 2023-10-06T05:17:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55ed7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80c8f60 con 0x7f55e8159500 2023-10-06T05:17:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55ed7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80c9240 con 0x7f55e815d2d0 2023-10-06T05:17:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55e67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81598d0 0x7f55e80c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e8159500 0x7f55e8158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55e67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81598d0 0x7f55e80c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34158/0 (socket says 172.21.15.31:34158) 2023-10-06T05:17:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55e67fc700 1 -- 172.21.15.31:0/3280171803 learned_addr learned my addr 172.21.15.31:0/3280171803 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e8159500 0x7f55e8158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53286/0 (socket says 172.21.15.31:53286) 2023-10-06T05:17:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.061+0000 7f55e77fe700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e815d2d0 0x7f55e80c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55e67fc700 1 -- 172.21.15.31:0/3280171803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e815d2d0 msgr2=0x7f55e80c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55e67fc700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e815d2d0 0x7f55e80c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55e67fc700 1 -- 172.21.15.31:0/3280171803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e8159500 msgr2=0x7f55e8158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55e67fc700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e8159500 0x7f55e8158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55e67fc700 1 -- 172.21.15.31:0/3280171803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d80094f0 con 0x7f55e81598d0 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55e6ffd700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e8159500 0x7f55e8158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55e67fc700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81598d0 0x7f55e80c83a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00e9d0 tx=0x7f55dc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55ceffd700 1 -- 172.21.15.31:0/3280171803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc013680 con 0x7f55e81598d0 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55ceffd700 1 -- 172.21.15.31:0/3280171803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc0137e0 con 0x7f55e81598d0 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55ed7ff700 1 -- 172.21.15.31:0/3280171803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e80c9520 con 0x7f55e81598d0 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55ceffd700 1 -- 172.21.15.31:0/3280171803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc013b80 con 0x7f55e81598d0 2023-10-06T05:17:08.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.062+0000 7f55ed7ff700 1 -- 172.21.15.31:0/3280171803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e815aec0 con 0x7f55e81598d0 2023-10-06T05:17:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.063+0000 7f55ed7ff700 1 -- 172.21.15.31:0/3280171803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e80c99c0 con 0x7f55e81598d0 2023-10-06T05:17:08.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.064+0000 7f55ceffd700 1 -- 172.21.15.31:0/3280171803 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55dc00c490 con 0x7f55e81598d0 2023-10-06T05:17:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.065+0000 7f55ceffd700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.065+0000 7f55ceffd700 1 -- 172.21.15.31:0/3280171803 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f55dc084ca0 con 0x7f55e81598d0 2023-10-06T05:17:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.065+0000 7f55e6ffd700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.066+0000 7f55e6ffd700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f55d4066f30 0x7f55d40693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55d80099b0 tx=0x7f55d8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.070+0000 7f55ceffd700 1 -- 172.21.15.31:0/3280171803 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55dc051d10 con 0x7f55e81598d0 2023-10-06T05:17:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.271+0000 7f55ed7ff700 1 -- 172.21.15.31:0/3280171803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55e804c380 con 0x7f55e81598d0 2023-10-06T05:17:08.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.272+0000 7f55ceffd700 1 -- 172.21.15.31:0/3280171803 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f55dc017090 con 0x7f55e81598d0 2023-10-06T05:17:08.274 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:08.275 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:08.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.274+0000 7f55ed7ff700 1 -- 172.21.15.31:0/3280171803 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f55d4066f30 msgr2=0x7f55d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.274+0000 7f55ed7ff700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f55d4066f30 0x7f55d40693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55d80099b0 tx=0x7f55d8017000 comp rx=0 tx=0).stop 2023-10-06T05:17:08.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.274+0000 7f55ed7ff700 1 -- 172.21.15.31:0/3280171803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81598d0 msgr2=0x7f55e80c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.275+0000 7f55ed7ff700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81598d0 0x7f55e80c83a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00e9d0 tx=0x7f55dc009120 comp rx=0 tx=0).stop 2023-10-06T05:17:08.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.275+0000 7f55ed7ff700 1 -- 172.21.15.31:0/3280171803 shutdown_connections 2023-10-06T05:17:08.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.275+0000 7f55ed7ff700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e815d2d0 0x7f55e80c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.275+0000 7f55ed7ff700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e8159500 0x7f55e8158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.275+0000 7f55ed7ff700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.275+0000 7f55ed7ff700 1 --2- 172.21.15.31:0/3280171803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81598d0 0x7f55e80c83a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.275+0000 7f55ed7ff700 1 -- 172.21.15.31:0/3280171803 >> 172.21.15.31:0/3280171803 conn(0x7f55e806ec90 msgr2=0x7f55e80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:08.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.275+0000 7f55ed7ff700 1 -- 172.21.15.31:0/3280171803 shutdown_connections 2023-10-06T05:17:08.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.275+0000 7f55ed7ff700 1 -- 172.21.15.31:0/3280171803 wait complete. 2023-10-06T05:17:08.294 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T05:17:08.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:08.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.490+0000 7fd9da1ff700 1 Processor -- start 2023-10-06T05:17:08.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.490+0000 7fd9da1ff700 1 -- start start 2023-10-06T05:17:08.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9da1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9d41595a0 0x7fd9d4159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9da1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd9d4068d80 0x7fd9d40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9da1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d40c95b0 con 0x7fd9d40cc0d0 2023-10-06T05:17:08.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9da1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40c9890 con 0x7fd9d41595a0 2023-10-06T05:17:08.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9da1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40c9b70 con 0x7fd9d4068d80 2023-10-06T05:17:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9d41595a0 0x7fd9d4159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9d41595a0 0x7fd9d4159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34160/0 (socket says 172.21.15.31:34160) 2023-10-06T05:17:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d37fe700 1 -- 172.21.15.31:0/1345110477 learned_addr learned my addr 172.21.15.31:0/1345110477 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d2ffd700 1 --1- 172.21.15.31:0/1345110477 >> v1:172.21.15.31:6789/0 conn(0x7fd9d40cc0d0 0x7fd9d4068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44692/0 (socket says 172.21.15.31:44692) 2023-10-06T05:17:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d3fff700 1 --2- 172.21.15.31:0/1345110477 >> v2:172.21.15.62:3300/0 conn(0x7fd9d4068d80 0x7fd9d40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d37fe700 1 -- 172.21.15.31:0/1345110477 >> v2:172.21.15.62:3300/0 conn(0x7fd9d4068d80 msgr2=0x7fd9d40c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d37fe700 1 --2- 172.21.15.31:0/1345110477 >> v2:172.21.15.62:3300/0 conn(0x7fd9d4068d80 0x7fd9d40c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d37fe700 1 -- 172.21.15.31:0/1345110477 >> v1:172.21.15.31:6789/0 conn(0x7fd9d40cc0d0 legacy=0x7fd9d4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d37fe700 1 -- 172.21.15.31:0/1345110477 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40ca130 con 0x7fd9d41595a0 2023-10-06T05:17:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d3fff700 1 --2- 172.21.15.31:0/1345110477 >> v2:172.21.15.62:3300/0 conn(0x7fd9d4068d80 0x7fd9d40c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:08.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.492+0000 7fd9d27fc700 1 -- 172.21.15.31:0/1345110477 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298146127 0 0) 0x7fd9d40c95b0 con 0x7fd9d40cc0d0 2023-10-06T05:17:08.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.493+0000 7fd9d37fe700 1 --2- 172.21.15.31:0/1345110477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 0x7fd9d4159970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40099b0 tx=0x7fd9c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5d6d952015f203 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.493+0000 7fd9d27fc700 1 -- 172.21.15.31:0/1345110477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c400b820 con 0x7fd9d41595a0 2023-10-06T05:17:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.493+0000 7fd9d27fc700 1 -- 172.21.15.31:0/1345110477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c400b980 con 0x7fd9d41595a0 2023-10-06T05:17:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.493+0000 7fd9d27fc700 1 -- 172.21.15.31:0/1345110477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c400bd20 con 0x7fd9d41595a0 2023-10-06T05:17:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.493+0000 7fd9da1ff700 1 -- 172.21.15.31:0/1345110477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 msgr2=0x7fd9d4159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.493+0000 7fd9da1ff700 1 --2- 172.21.15.31:0/1345110477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 0x7fd9d4159970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40099b0 tx=0x7fd9c4009250 comp rx=0 tx=0).stop 2023-10-06T05:17:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.494+0000 7fd9da1ff700 1 -- 172.21.15.31:0/1345110477 shutdown_connections 2023-10-06T05:17:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.494+0000 7fd9da1ff700 1 --2- 172.21.15.31:0/1345110477 >> v2:172.21.15.62:3300/0 conn(0x7fd9d4068d80 0x7fd9d40c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.494+0000 7fd9da1ff700 1 --2- 172.21.15.31:0/1345110477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 0x7fd9d4159970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.494+0000 7fd9da1ff700 1 -- 172.21.15.31:0/1345110477 >> 172.21.15.31:0/1345110477 conn(0x7fd9d406b620 msgr2=0x7fd9d406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.494+0000 7fd9da1ff700 1 -- 172.21.15.31:0/1345110477 shutdown_connections 2023-10-06T05:17:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.494+0000 7fd9da1ff700 1 -- 172.21.15.31:0/1345110477 wait complete. 2023-10-06T05:17:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.495+0000 7fd9da1ff700 1 Processor -- start 2023-10-06T05:17:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.495+0000 7fd9da1ff700 1 -- start start 2023-10-06T05:17:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.496+0000 7fd9da1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9d4068d80 0x7fd9d40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.496+0000 7fd9da1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d40cc0d0 0x7fd9d415e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.496+0000 7fd9da1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 0x7fd9d415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.496+0000 7fd9da1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40b3cf0 con 0x7fd9d41595a0 2023-10-06T05:17:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.496+0000 7fd9da1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d415f190 con 0x7fd9d40cc0d0 2023-10-06T05:17:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.496+0000 7fd9da1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d415f470 con 0x7fd9d4068d80 2023-10-06T05:17:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.496+0000 7fd9d3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 0x7fd9d415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.496+0000 7fd9d3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 0x7fd9d415ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34172/0 (socket says 172.21.15.31:34172) 2023-10-06T05:17:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.496+0000 7fd9d3fff700 1 -- 172.21.15.31:0/3925316423 learned_addr learned my addr 172.21.15.31:0/3925316423 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9d2ffd700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d40cc0d0 0x7fd9d415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9d37fe700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9d4068d80 0x7fd9d40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9d3fff700 1 -- 172.21.15.31:0/3925316423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9d4068d80 msgr2=0x7fd9d40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9d3fff700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9d4068d80 0x7fd9d40b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9d3fff700 1 -- 172.21.15.31:0/3925316423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d40cc0d0 msgr2=0x7fd9d415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9d3fff700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d40cc0d0 0x7fd9d415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9d3fff700 1 -- 172.21.15.31:0/3925316423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c40094f0 con 0x7fd9d41595a0 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9d2ffd700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d40cc0d0 0x7fd9d415e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9d3fff700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 0x7fd9d415ea80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009f90 tx=0x7fd9cc0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9bb7fe700 1 -- 172.21.15.31:0/3925316423 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00dd40 con 0x7fd9d41595a0 2023-10-06T05:17:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9bb7fe700 1 -- 172.21.15.31:0/3925316423 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc00f040 con 0x7fd9d41595a0 2023-10-06T05:17:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9bb7fe700 1 -- 172.21.15.31:0/3925316423 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc015690 con 0x7fd9d41595a0 2023-10-06T05:17:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.497+0000 7fd9da1ff700 1 -- 172.21.15.31:0/3925316423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d415f750 con 0x7fd9d41595a0 2023-10-06T05:17:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.498+0000 7fd9da1ff700 1 -- 172.21.15.31:0/3925316423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d415fd10 con 0x7fd9d41595a0 2023-10-06T05:17:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.498+0000 7fd9da1ff700 1 -- 172.21.15.31:0/3925316423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d404c380 con 0x7fd9d41595a0 2023-10-06T05:17:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.499+0000 7fd9bb7fe700 1 -- 172.21.15.31:0/3925316423 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9cc01e3b0 con 0x7fd9d41595a0 2023-10-06T05:17:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.500+0000 7fd9bb7fe700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd9c0066f30 0x7fd9c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.500+0000 7fd9bb7fe700 1 -- 172.21.15.31:0/3925316423 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd9cc084c60 con 0x7fd9d41595a0 2023-10-06T05:17:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.500+0000 7fd9d37fe700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd9c0066f30 0x7fd9c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.501+0000 7fd9d37fe700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd9c0066f30 0x7fd9c00693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40099b0 tx=0x7fd9c4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.504+0000 7fd9bb7fe700 1 -- 172.21.15.31:0/3925316423 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9cc051cd0 con 0x7fd9d41595a0 2023-10-06T05:17:08.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.709+0000 7fd9da1ff700 1 -- 172.21.15.31:0/3925316423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9d4069b60 con 0x7fd9d41595a0 2023-10-06T05:17:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.710+0000 7fd9bb7fe700 1 -- 172.21.15.31:0/3925316423 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fd9cc01a090 con 0x7fd9d41595a0 2023-10-06T05:17:08.712 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:08.713 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 -- 172.21.15.31:0/3925316423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd9c0066f30 msgr2=0x7fd9c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd9c0066f30 0x7fd9c00693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40099b0 tx=0x7fd9c4017000 comp rx=0 tx=0).stop 2023-10-06T05:17:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 -- 172.21.15.31:0/3925316423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 msgr2=0x7fd9d415ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 0x7fd9d415ea80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009f90 tx=0x7fd9cc0099b0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 -- 172.21.15.31:0/3925316423 shutdown_connections 2023-10-06T05:17:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd9c0066f30 0x7fd9c00693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d41595a0 0x7fd9d415ea80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9d4068d80 0x7fd9d40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 --2- 172.21.15.31:0/3925316423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d40cc0d0 0x7fd9d415e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 -- 172.21.15.31:0/3925316423 >> 172.21.15.31:0/3925316423 conn(0x7fd9d406b620 msgr2=0x7fd9d406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 -- 172.21.15.31:0/3925316423 shutdown_connections 2023-10-06T05:17:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:08.713+0000 7fd9da1ff700 1 -- 172.21.15.31:0/3925316423 wait complete. 2023-10-06T05:17:08.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T05:17:08.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T05:17:08.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T05:17:08.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T05:17:08.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T05:17:08.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T05:17:08.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T05:17:08.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T05:17:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.091+0000 7fda891ff700 1 Processor -- start 2023-10-06T05:17:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.091+0000 7fe20a7ff700 1 Processor -- start 2023-10-06T05:17:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.092+0000 7fda891ff700 1 -- start start 2023-10-06T05:17:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.093+0000 7fe20a7ff700 1 -- start start 2023-10-06T05:17:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.094+0000 7fda891ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda84069560 0x7fda8406aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.094+0000 7fda891ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 0x7fda8406e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.094+0000 7fda891ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8406d5e0 con 0x7fda8406eef0 2023-10-06T05:17:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.094+0000 7fda891ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8406a5e0 con 0x7fda84069560 2023-10-06T05:17:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.094+0000 7fda891ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda84069cb0 con 0x7fda840ccbc0 2023-10-06T05:17:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.096+0000 7fda8259c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 0x7fda8406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.096+0000 7fda8259c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 0x7fda8406e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49530/0 (socket says 172.21.15.31:49530) 2023-10-06T05:17:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.096+0000 7fda8259c700 1 -- 172.21.15.31:0/931218601 learned_addr learned my addr 172.21.15.31:0/931218601 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.097+0000 7fe20a7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe20406f140 0x7fe20406aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.097+0000 7fe20a7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe204069560 0x7fe20406e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.097+0000 7fe20a7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20406d5e0 con 0x7fe20406f510 2023-10-06T05:17:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.097+0000 7fe20a7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20406a5e0 con 0x7fe20406f140 2023-10-06T05:17:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.097+0000 7fe20a7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040699d0 con 0x7fe204069560 2023-10-06T05:17:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.098+0000 7fe2037fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe204069560 0x7fe20406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.098+0000 7fe2037fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe204069560 0x7fe20406e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49542/0 (socket says 172.21.15.31:49542) 2023-10-06T05:17:09.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.098+0000 7fe2037fe700 1 -- 172.21.15.31:0/3065430704 learned_addr learned my addr 172.21.15.31:0/3065430704 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.098+0000 7fe2037fe700 1 -- 172.21.15.31:0/3065430704 >> v2:172.21.15.62:3300/0 conn(0x7fe20406f140 msgr2=0x7fe20406aa70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:17:09.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.098+0000 7fe2037fe700 1 --2- 172.21.15.31:0/3065430704 >> v2:172.21.15.62:3300/0 conn(0x7fe20406f140 0x7fe20406aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.098+0000 7fe2037fe700 1 -- 172.21.15.31:0/3065430704 >> v1:172.21.15.62:6789/0 conn(0x7fe20406f510 legacy=0x7fe2040ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.098+0000 7fe2037fe700 1 -- 172.21.15.31:0/3065430704 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204069cb0 con 0x7fe204069560 2023-10-06T05:17:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.100+0000 7fda82d9d700 1 --2- 172.21.15.31:0/931218601 >> v2:172.21.15.31:3300/0 conn(0x7fda84069560 0x7fda8406aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.100+0000 7f8d1d7ff700 1 Processor -- start 2023-10-06T05:17:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.100+0000 7fda8259c700 1 -- 172.21.15.31:0/931218601 >> v2:172.21.15.31:3300/0 conn(0x7fda84069560 msgr2=0x7fda8406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.100+0000 7fda8259c700 1 --2- 172.21.15.31:0/931218601 >> v2:172.21.15.31:3300/0 conn(0x7fda84069560 0x7fda8406aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.100+0000 7fda8259c700 1 -- 172.21.15.31:0/931218601 >> v1:172.21.15.31:6789/0 conn(0x7fda8406eef0 legacy=0x7fda840ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.100+0000 7fda8259c700 1 -- 172.21.15.31:0/931218601 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda84069f90 con 0x7fda840ccbc0 2023-10-06T05:17:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.100+0000 7f8d1d7ff700 1 -- start start 2023-10-06T05:17:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.101+0000 7fe2037fe700 1 --2- 172.21.15.31:0/3065430704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe204069560 0x7fe20406e340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009f20 tx=0x7fe1fc009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97892d6b000f087 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.101+0000 7fe08bfff700 1 Processor -- start 2023-10-06T05:17:09.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.101+0000 7fe08bfff700 1 -- start start 2023-10-06T05:17:09.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.102+0000 7f8d1d7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d18069e80 0x7f8d180c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.102+0000 7f8d1d7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8d180c6ca0 0x7f8d180ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.102+0000 7f8d1d7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d180c74d0 con 0x7f8d18147380 2023-10-06T05:17:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.102+0000 7f8d1d7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d180c9c80 con 0x7f8d180c6ca0 2023-10-06T05:17:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.102+0000 7f8d1d7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180c77b0 con 0x7f8d18069e80 2023-10-06T05:17:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.103+0000 7f8d0e7fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d18069e80 0x7f8d180c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.103+0000 7f8d0e7fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d18069e80 0x7f8d180c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49552/0 (socket says 172.21.15.31:49552) 2023-10-06T05:17:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.103+0000 7f8d0e7fc700 1 -- 172.21.15.31:0/3729873845 learned_addr learned my addr 172.21.15.31:0/3729873845 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.103+0000 7f8d177fe700 1 --2- 172.21.15.31:0/3729873845 >> v2:172.21.15.62:3300/0 conn(0x7f8d180c6ca0 0x7f8d180ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7f8d0e7fc700 1 -- 172.21.15.31:0/3729873845 >> v2:172.21.15.62:3300/0 conn(0x7f8d180c6ca0 msgr2=0x7f8d180ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7f8d0e7fc700 1 --2- 172.21.15.31:0/3729873845 >> v2:172.21.15.62:3300/0 conn(0x7f8d180c6ca0 0x7f8d180ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7f8d0e7fc700 1 -- 172.21.15.31:0/3729873845 >> v1:172.21.15.31:6789/0 conn(0x7f8d18147380 legacy=0x7f8d18069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7f8d0e7fc700 1 -- 172.21.15.31:0/3729873845 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d180c7a90 con 0x7f8d18069e80 2023-10-06T05:17:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7fe08bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe084069e80 0x7fe0840c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7fe08bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0840c74d0 con 0x7fe0840c6ca0 2023-10-06T05:17:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7fe08bfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0840c77b0 con 0x7fe084147380 2023-10-06T05:17:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7fe08bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0840c9c80 con 0x7fe084069e80 2023-10-06T05:17:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7fda8259c700 1 --2- 172.21.15.31:0/931218601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 0x7fda8406e340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fda740099b0 tx=0x7fda74009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71e6f9f21eae9ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.102+0000 7fe202ffd700 1 -- 172.21.15.31:0/3065430704 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00e070 con 0x7fe204069560 2023-10-06T05:17:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.102+0000 7fe202ffd700 1 -- 172.21.15.31:0/3065430704 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc00bb10 con 0x7fe204069560 2023-10-06T05:17:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.102+0000 7fe202ffd700 1 -- 172.21.15.31:0/3065430704 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc0092e0 con 0x7fe204069560 2023-10-06T05:17:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.103+0000 7fe20a7ff700 1 -- 172.21.15.31:0/3065430704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe204069560 msgr2=0x7fe20406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.103+0000 7fe20a7ff700 1 --2- 172.21.15.31:0/3065430704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe204069560 0x7fe20406e340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009f20 tx=0x7fe1fc009840 comp rx=0 tx=0).stop 2023-10-06T05:17:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.103+0000 7fe20a7ff700 1 -- 172.21.15.31:0/3065430704 shutdown_connections 2023-10-06T05:17:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.103+0000 7fe20a7ff700 1 --2- 172.21.15.31:0/3065430704 >> v2:172.21.15.62:3300/0 conn(0x7fe20406f140 0x7fe20406aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.103+0000 7fe20a7ff700 1 --2- 172.21.15.31:0/3065430704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe204069560 0x7fe20406e340 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.103+0000 7fe20a7ff700 1 -- 172.21.15.31:0/3065430704 >> 172.21.15.31:0/3065430704 conn(0x7fe2040cc0b0 msgr2=0x7fe2040cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7f8d0e7fc700 1 --2- 172.21.15.31:0/3729873845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d18069e80 0x7f8d180c6760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d080099b0 tx=0x7f8d08009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec73367cce337bcf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7fda81d9b700 1 -- 172.21.15.31:0/931218601 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7400b8f0 con 0x7fda840ccbc0 2023-10-06T05:17:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.105+0000 7fda81d9b700 1 -- 172.21.15.31:0/931218601 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7400ba50 con 0x7fda840ccbc0 2023-10-06T05:17:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.105+0000 7fda81d9b700 1 -- 172.21.15.31:0/931218601 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7400bdf0 con 0x7fda840ccbc0 2023-10-06T05:17:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7fe089d9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe084147380 0x7fe084069770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:34098/0 (socket says 172.21.15.31:34098) 2023-10-06T05:17:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.104+0000 7fe089d9b700 1 -- 172.21.15.31:0/1176768462 learned_addr learned my addr 172.21.15.31:0/1176768462 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.105+0000 7fe08959a700 1 --2- 172.21.15.31:0/1176768462 >> v2:172.21.15.62:3300/0 conn(0x7fe084069e80 0x7fe0840c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.105+0000 7f8d167fc700 1 -- 172.21.15.31:0/3729873845 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08009ea0 con 0x7f8d18069e80 2023-10-06T05:17:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.105+0000 7f8d167fc700 1 -- 172.21.15.31:0/3729873845 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0800b820 con 0x7f8d18069e80 2023-10-06T05:17:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.105+0000 7f8d167fc700 1 -- 172.21.15.31:0/3729873845 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0800bbe0 con 0x7f8d18069e80 2023-10-06T05:17:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.105+0000 7fda891ff700 1 -- 172.21.15.31:0/931218601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 msgr2=0x7fda8406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.105+0000 7fda891ff700 1 --2- 172.21.15.31:0/931218601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 0x7fda8406e340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fda740099b0 tx=0x7fda74009250 comp rx=0 tx=0).stop 2023-10-06T05:17:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fda891ff700 1 -- 172.21.15.31:0/931218601 shutdown_connections 2023-10-06T05:17:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fda891ff700 1 --2- 172.21.15.31:0/931218601 >> v2:172.21.15.31:3300/0 conn(0x7fda84069560 0x7fda8406aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fda891ff700 1 --2- 172.21.15.31:0/931218601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 0x7fda8406e340 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fda891ff700 1 -- 172.21.15.31:0/931218601 >> 172.21.15.31:0/931218601 conn(0x7fda840cc0b0 msgr2=0x7fda840cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.105+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100145605 0 0) 0x7fe0840c74d0 con 0x7fe0840c6ca0 2023-10-06T05:17:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe070003570 con 0x7fe0840c6ca0 2023-10-06T05:17:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434628107 0 0) 0x7fe0840c77b0 con 0x7fe084147380 2023-10-06T05:17:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0840c74d0 con 0x7fe084147380 2023-10-06T05:17:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/3729873845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d18069e80 msgr2=0x7f8d180c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7f8d1d7ff700 1 --2- 172.21.15.31:0/3729873845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d18069e80 0x7f8d180c6760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d080099b0 tx=0x7f8d08009250 comp rx=0 tx=0).stop 2023-10-06T05:17:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2406800374 0 0) 0x7fe0840c74d0 con 0x7fe084147380 2023-10-06T05:17:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 >> v2:172.21.15.62:3300/0 conn(0x7fe084069e80 msgr2=0x7fe0840c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 --2- 172.21.15.31:0/1176768462 >> v2:172.21.15.62:3300/0 conn(0x7fe084069e80 0x7fe0840c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 >> v1:172.21.15.31:6789/0 conn(0x7fe0840c6ca0 legacy=0x7fe0840ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840c7a90 con 0x7fe084147380 2023-10-06T05:17:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fe0740014c0 con 0x7fe084147380 2023-10-06T05:17:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe20a7ff700 1 -- 172.21.15.31:0/3065430704 shutdown_connections 2023-10-06T05:17:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe20a7ff700 1 -- 172.21.15.31:0/3065430704 wait complete. 2023-10-06T05:17:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 >> v1:172.21.15.62:6789/0 conn(0x7fe084147380 legacy=0x7fe084069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 --2- 172.21.15.31:0/1176768462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe070004ca0 0x7fe070005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 --2- 172.21.15.31:0/1176768462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0700037b0 0x7fe070003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 --2- 172.21.15.31:0/1176768462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe070004280 0x7fe07000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07000c220 con 0x7fe070004ca0 2023-10-06T05:17:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07000c390 con 0x7fe070004280 2023-10-06T05:17:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07000c550 con 0x7fe0700037b0 2023-10-06T05:17:09.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/3729873845 shutdown_connections 2023-10-06T05:17:09.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7f8d1d7ff700 1 --2- 172.21.15.31:0/3729873845 >> v2:172.21.15.62:3300/0 conn(0x7f8d180c6ca0 0x7f8d180ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7f8d1d7ff700 1 --2- 172.21.15.31:0/3729873845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d18069e80 0x7f8d180c6760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.106+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/3729873845 >> 172.21.15.31:0/3729873845 conn(0x7f8d180cc0b0 msgr2=0x7f8d180cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.107+0000 7fe089d9b700 1 --2- 172.21.15.31:0/1176768462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe070004ca0 0x7fe070005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.107+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/3729873845 shutdown_connections 2023-10-06T05:17:09.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.107+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/3729873845 wait complete. 2023-10-06T05:17:09.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.108+0000 7fe089d9b700 1 -- 172.21.15.31:0/1176768462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0700037b0 msgr2=0x7fe070003c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:17:09.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.108+0000 7fe089d9b700 1 --2- 172.21.15.31:0/1176768462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0700037b0 0x7fe070003c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.108+0000 7fe089d9b700 1 -- 172.21.15.31:0/1176768462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe070004280 msgr2=0x7fe07000ba60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:17:09.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.108+0000 7fe089d9b700 1 --2- 172.21.15.31:0/1176768462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe070004280 0x7fe07000ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.108+0000 7fe089d9b700 1 -- 172.21.15.31:0/1176768462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840c74d0 con 0x7fe070004ca0 2023-10-06T05:17:09.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.108+0000 7fe089d9b700 1 --2- 172.21.15.31:0/1176768462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe070004ca0 0x7fe070005090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c210 tx=0x7fe074001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9bfe3504331ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.108+0000 7f8d1d7ff700 1 Processor -- start 2023-10-06T05:17:09.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.108+0000 7fe20a7ff700 1 Processor -- start 2023-10-06T05:17:09.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.109+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074001de0 con 0x7fe070004ca0 2023-10-06T05:17:09.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.109+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07400cae0 con 0x7fe070004ca0 2023-10-06T05:17:09.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.109+0000 7fe088d99700 1 -- 172.21.15.31:0/1176768462 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07400ce80 con 0x7fe070004ca0 2023-10-06T05:17:09.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.109+0000 7fe20a7ff700 1 -- start start 2023-10-06T05:17:09.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.109+0000 7f8d1d7ff700 1 -- start start 2023-10-06T05:17:09.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.110+0000 7f8d1d7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18069e80 0x7f8d18155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.110+0000 7f8d1d7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d180c6ca0 0x7f8d180c1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.110+0000 7f8d1d7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d18147380 0x7f8d180c1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.110+0000 7f8d1d7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18157c80 con 0x7f8d18069e80 2023-10-06T05:17:09.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.110+0000 7f8d1d7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18172440 con 0x7f8d18147380 2023-10-06T05:17:09.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.110+0000 7f8d1d7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18171680 con 0x7f8d180c6ca0 2023-10-06T05:17:09.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.110+0000 7fe20a7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe204069560 0x7fe2040add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.110+0000 7fe20a7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe20406f140 0x7fe2040ab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7fe20a7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe20406f510 0x7fe2040c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7fe20a7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040c5430 con 0x7fe20406f140 2023-10-06T05:17:09.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7fe20a7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040c4b00 con 0x7fe204069560 2023-10-06T05:17:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7fe20a7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040c4de0 con 0x7fe20406f510 2023-10-06T05:17:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.110+0000 7fe08bfff700 1 -- 172.21.15.31:0/1176768462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe070004ca0 msgr2=0x7fe070005090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.110+0000 7fe08bfff700 1 --2- 172.21.15.31:0/1176768462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe070004ca0 0x7fe070005090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c210 tx=0x7fe074001d70 comp rx=0 tx=0).stop 2023-10-06T05:17:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7f8d0e7fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d180c6ca0 0x7f8d180c1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7f8d16ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18069e80 0x7f8d18155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7f8d16ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18069e80 0x7f8d18155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34200/0 (socket says 172.21.15.31:34200) 2023-10-06T05:17:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7f8d16ffd700 1 -- 172.21.15.31:0/806661727 learned_addr learned my addr 172.21.15.31:0/806661727 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7f8d177fe700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d18147380 0x7f8d180c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7fe203fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe204069560 0x7fe2040add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7fe203fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe204069560 0x7fe2040add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53342/0 (socket says 172.21.15.31:53342) 2023-10-06T05:17:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7fe203fff700 1 -- 172.21.15.31:0/919955586 learned_addr learned my addr 172.21.15.31:0/919955586 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.111+0000 7fe208d9c700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe20406f510 0x7fe2040c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7fe2037fe700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe20406f140 0x7fe2040ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7fe203fff700 1 -- 172.21.15.31:0/919955586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe20406f510 msgr2=0x7fe2040c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7fe203fff700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe20406f510 0x7fe2040c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7fe203fff700 1 -- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe20406f140 msgr2=0x7fe2040ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7f8d0e7fc700 1 -- 172.21.15.31:0/806661727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d18147380 msgr2=0x7f8d180c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7f8d0e7fc700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d18147380 0x7f8d180c1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7f8d0e7fc700 1 -- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18069e80 msgr2=0x7f8d18155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7f8d0e7fc700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18069e80 0x7f8d18155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7f8d0e7fc700 1 -- 172.21.15.31:0/806661727 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d080094f0 con 0x7f8d180c6ca0 2023-10-06T05:17:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7f8d0e7fc700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d180c6ca0 0x7f8d180c1350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d1806c960 tx=0x7f8d08016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7fe203fff700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe20406f140 0x7fe2040ab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7fe203fff700 1 -- 172.21.15.31:0/919955586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc009ae0 con 0x7fe204069560 2023-10-06T05:17:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7fe08a59c700 1 -- 172.21.15.31:0/1176768462 reap_dead start 2023-10-06T05:17:09.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7fe08a59c700 1 -- 172.21.15.31:0/1176768462 reap_dead start 2023-10-06T05:17:09.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.112+0000 7fe2037fe700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe20406f140 0x7fe2040ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:09.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7fe08bfff700 1 -- 172.21.15.31:0/1176768462 shutdown_connections 2023-10-06T05:17:09.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7fe08bfff700 1 -- 172.21.15.31:0/1176768462 >> 172.21.15.31:0/1176768462 conn(0x7fe0840cc0b0 msgr2=0x7fe0840cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7fda891ff700 1 -- 172.21.15.31:0/931218601 shutdown_connections 2023-10-06T05:17:09.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7f8d0f7fe700 1 -- 172.21.15.31:0/806661727 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08006ba0 con 0x7f8d180c6ca0 2023-10-06T05:17:09.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7f8d0f7fe700 1 -- 172.21.15.31:0/806661727 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d08020070 con 0x7f8d180c6ca0 2023-10-06T05:17:09.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7f8d0f7fe700 1 -- 172.21.15.31:0/806661727 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08011cb0 con 0x7f8d180c6ca0 2023-10-06T05:17:09.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d18171960 con 0x7f8d180c6ca0 2023-10-06T05:17:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7fda891ff700 1 -- 172.21.15.31:0/931218601 wait complete. 2023-10-06T05:17:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7fe203fff700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe204069560 0x7fe2040add50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800b600 tx=0x7fe1f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7fe1ebfff700 1 -- 172.21.15.31:0/919955586 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800f510 con 0x7fe204069560 2023-10-06T05:17:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.114+0000 7fe1ebfff700 1 -- 172.21.15.31:0/919955586 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f800f690 con 0x7fe204069560 2023-10-06T05:17:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.114+0000 7fe1ebfff700 1 -- 172.21.15.31:0/919955586 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800fa50 con 0x7fe204069560 2023-10-06T05:17:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.114+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2040c50c0 con 0x7fe204069560 2023-10-06T05:17:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.114+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20416e190 con 0x7fe204069560 2023-10-06T05:17:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.113+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d18171f20 con 0x7f8d180c6ca0 2023-10-06T05:17:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.115+0000 7fda891ff700 1 Processor -- start 2023-10-06T05:17:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.115+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d1804c380 con 0x7f8d180c6ca0 2023-10-06T05:17:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.116+0000 7fda891ff700 1 -- start start 2023-10-06T05:17:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.116+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe20404c380 con 0x7fe204069560 2023-10-06T05:17:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.116+0000 7fe08bfff700 1 -- 172.21.15.31:0/1176768462 shutdown_connections 2023-10-06T05:17:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.116+0000 7ff3385ff700 1 Processor -- start 2023-10-06T05:17:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.116+0000 7ff3385ff700 1 -- start start 2023-10-06T05:17:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.117+0000 7f8d0f7fe700 1 -- 172.21.15.31:0/806661727 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d08022030 con 0x7f8d180c6ca0 2023-10-06T05:17:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.118+0000 7f8d0f7fe700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8cf8067080 0x7f8cf8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.118+0000 7f8d0f7fe700 1 -- 172.21.15.31:0/806661727 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8d080863f0 con 0x7f8d180c6ca0 2023-10-06T05:17:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7f8d0f7fe700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] conn(0x7f8cf806f930 0x7f8cf8071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7f8d0f7fe700 1 -- 172.21.15.31:0/806661727 --> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cf80723f0 con 0x7f8cf806f930 2023-10-06T05:17:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7f8d0f7fe700 1 -- 172.21.15.31:0/806661727 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d08088990 con 0x7f8d180c6ca0 2023-10-06T05:17:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7fda891ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda84069560 0x7fda84146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7fda891ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda8406eef0 0x7fda840c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fda891ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 0x7fda84154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fda891ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda840c53d0 con 0x7fda8406eef0 2023-10-06T05:17:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fda891ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda84147470 con 0x7fda84069560 2023-10-06T05:17:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fda891ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda840ae5a0 con 0x7fda840ccbc0 2023-10-06T05:17:09.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7f8d16ffd700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8cf8067080 0x7f8cf8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fda82d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda84069560 0x7fda84146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fda82d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda84069560 0x7fda84146c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53352/0 (socket says 172.21.15.31:53352) 2023-10-06T05:17:09.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fda82d9d700 1 -- 172.21.15.31:0/996730017 learned_addr learned my addr 172.21.15.31:0/996730017 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fda8259c700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda8406eef0 0x7fda840c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fda8359e700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 0x7fda84154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7ff3385ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff330147380 0x7ff330069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7ff3385ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 0x7ff3300c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7ff3385ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33006a100 con 0x7ff3300c6ca0 2023-10-06T05:17:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7ff3385ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3300c9c80 con 0x7ff330147380 2023-10-06T05:17:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7ff3385ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300c74d0 con 0x7ff330069cb0 2023-10-06T05:17:09.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7fe1ebfff700 1 -- 172.21.15.31:0/919955586 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1f801a400 con 0x7fe204069560 2023-10-06T05:17:09.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fe1ebfff700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe1f0067080 0x7fe1f0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.120+0000 7fe1ebfff700 1 -- 172.21.15.31:0/919955586 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe1f8084100 con 0x7fe204069560 2023-10-06T05:17:09.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fe1ebfff700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] conn(0x7fe1f006f8b0 0x7fe1f0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fe1ebfff700 1 -- 172.21.15.31:0/919955586 --> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1f0072370 con 0x7fe1f006f8b0 2023-10-06T05:17:09.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fda82d9d700 1 -- 172.21.15.31:0/996730017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 msgr2=0x7fda84154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fda82d9d700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda840ccbc0 0x7fda84154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fda82d9d700 1 -- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda8406eef0 msgr2=0x7fda840c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fda82d9d700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda8406eef0 0x7fda840c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fda82d9d700 1 -- 172.21.15.31:0/996730017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740094f0 con 0x7fda84069560 2023-10-06T05:17:09.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fe208d9c700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] conn(0x7fe1f006f8b0 0x7fe1f0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.117+0000 7fe08bfff700 1 -- 172.21.15.31:0/1176768462 wait complete. 2023-10-06T05:17:09.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7fe08bfff700 1 Processor -- start 2023-10-06T05:17:09.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.119+0000 7fe08bfff700 1 -- start start 2023-10-06T05:17:09.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fe08bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0840c5950 0x7fe0840c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fe08bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0841624a0 0x7fe084162910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fe08bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0840b1150 0x7fe0840b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fe08bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084087880 con 0x7fe0841624a0 2023-10-06T05:17:09.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fe08bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840c6260 con 0x7fe0840b1150 2023-10-06T05:17:09.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fe08bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840b00b0 con 0x7fe0840c5950 2023-10-06T05:17:09.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7ff335b9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 0x7ff3300c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7ff335b9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 0x7ff3300c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49600/0 (socket says 172.21.15.31:49600) 2023-10-06T05:17:09.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7ff335b9a700 1 -- 172.21.15.31:0/596340030 learned_addr learned my addr 172.21.15.31:0/596340030 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7f8d177fe700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] conn(0x7f8cf806f930 0x7f8cf8071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe1ebfff700 1 -- 172.21.15.31:0/919955586 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f8051270 con 0x7fe204069560 2023-10-06T05:17:09.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.121+0000 7fe089d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0840c5950 0x7fe0840c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe089d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0840c5950 0x7fe0840c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49604/0 (socket says 172.21.15.31:49604) 2023-10-06T05:17:09.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe089d9b700 1 -- 172.21.15.31:0/167537689 learned_addr learned my addr 172.21.15.31:0/167537689 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe08a59c700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0840b1150 0x7fe0840b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fda82d9d700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda84069560 0x7fda84146c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009fa0 tx=0x7fda7c009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7f8d16ffd700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8cf8067080 0x7f8cf8069530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d00007fd0 tx=0x7f8d00007f60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fda6affd700 1 -- 172.21.15.31:0/996730017 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c00ebc0 con 0x7fda84069560 2023-10-06T05:17:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fda6affd700 1 -- 172.21.15.31:0/996730017 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c00ed20 con 0x7fda84069560 2023-10-06T05:17:09.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fda6affd700 1 -- 172.21.15.31:0/996730017 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c014670 con 0x7fda84069560 2023-10-06T05:17:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe2037fe700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe1f0067080 0x7fe1f0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe08959a700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0841624a0 0x7fe084162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe089d9b700 1 -- 172.21.15.31:0/167537689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0840b1150 msgr2=0x7fe0840b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe089d9b700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0840b1150 0x7fe0840b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe089d9b700 1 -- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0841624a0 msgr2=0x7fe084162910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe089d9b700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0841624a0 0x7fe084162910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fe089d9b700 1 -- 172.21.15.31:0/167537689 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840b0390 con 0x7fe0840c5950 2023-10-06T05:17:09.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.122+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda840acc30 con 0x7fda84069560 2023-10-06T05:17:09.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.123+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda84147a10 con 0x7fda84069560 2023-10-06T05:17:09.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.123+0000 7f8d177fe700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] conn(0x7f8cf806f930 0x7f8cf8071d40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda8406f370 con 0x7fda84069560 2023-10-06T05:17:09.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7fe208d9c700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] conn(0x7fe1f006f8b0 0x7fe1f0071cc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7fe2037fe700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe1f0067080 0x7fe1f0069530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009ab0 tx=0x7fe1fc018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.123+0000 7fe089d9b700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0840c5950 0x7fe0840c5d20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe074007b90 tx=0x7fe074015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7fe079ffb700 1 -- 172.21.15.31:0/167537689 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0740072e0 con 0x7fe0840c5950 2023-10-06T05:17:09.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7fe079ffb700 1 -- 172.21.15.31:0/167537689 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0740074a0 con 0x7fe0840c5950 2023-10-06T05:17:09.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7fe079ffb700 1 -- 172.21.15.31:0/167537689 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07401e400 con 0x7fe0840c5950 2023-10-06T05:17:09.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0840b0670 con 0x7fe0840c5950 2023-10-06T05:17:09.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0840b0c30 con 0x7fe0840c5950 2023-10-06T05:17:09.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7ff33639b700 1 --2- 172.21.15.31:0/596340030 >> v2:172.21.15.31:3300/0 conn(0x7ff330147380 0x7ff330069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7ff335b9a700 1 -- 172.21.15.31:0/596340030 >> v2:172.21.15.31:3300/0 conn(0x7ff330147380 msgr2=0x7ff330069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7ff335b9a700 1 --2- 172.21.15.31:0/596340030 >> v2:172.21.15.31:3300/0 conn(0x7ff330147380 0x7ff330069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7ff335b9a700 1 -- 172.21.15.31:0/596340030 >> v1:172.21.15.62:6789/0 conn(0x7ff3300c6ca0 legacy=0x7ff3301615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.124+0000 7ff335b9a700 1 -- 172.21.15.31:0/596340030 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3300c77b0 con 0x7ff330069cb0 2023-10-06T05:17:09.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.125+0000 7ff335b9a700 1 --2- 172.21.15.31:0/596340030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 0x7ff3300c6760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff3200099b0 tx=0x7ff320009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51642a1dc882e0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.125+0000 7f0f74bff700 1 Processor -- start 2023-10-06T05:17:09.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.125+0000 7fe1ebfff700 1 -- 172.21.15.31:0/919955586 <== osd.1 v2:172.21.15.31:6816/114859 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe1f0072370 con 0x7fe1f006f8b0 2023-10-06T05:17:09.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.125+0000 7ff335399700 1 -- 172.21.15.31:0/596340030 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32000b860 con 0x7ff330069cb0 2023-10-06T05:17:09.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.125+0000 7ff335399700 1 -- 172.21.15.31:0/596340030 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32000b9c0 con 0x7ff330069cb0 2023-10-06T05:17:09.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.125+0000 7ff335399700 1 -- 172.21.15.31:0/596340030 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32000bd80 con 0x7ff330069cb0 2023-10-06T05:17:09.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.127+0000 7fda6affd700 1 -- 172.21.15.31:0/996730017 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda7c0149e0 con 0x7fda84069560 2023-10-06T05:17:09.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.126+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe064000fd0 con 0x7fe0840c5950 2023-10-06T05:17:09.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.126+0000 7f8d0f7fe700 1 -- 172.21.15.31:0/806661727 <== osd.3 v2:172.21.15.31:6823/114865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8cf80723f0 con 0x7f8cf806f930 2023-10-06T05:17:09.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.127+0000 7ff3385ff700 1 -- 172.21.15.31:0/596340030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 msgr2=0x7ff3300c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.127+0000 7ff3385ff700 1 --2- 172.21.15.31:0/596340030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 0x7ff3300c6760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff3200099b0 tx=0x7ff320009250 comp rx=0 tx=0).stop 2023-10-06T05:17:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.126+0000 7f0f74bff700 1 -- start start 2023-10-06T05:17:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.127+0000 7ff3385ff700 1 -- 172.21.15.31:0/596340030 shutdown_connections 2023-10-06T05:17:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.127+0000 7ff3385ff700 1 --2- 172.21.15.31:0/596340030 >> v2:172.21.15.31:3300/0 conn(0x7ff330147380 0x7ff330069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.127+0000 7ff3385ff700 1 --2- 172.21.15.31:0/596340030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 0x7ff3300c6760 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.127+0000 7ff3385ff700 1 -- 172.21.15.31:0/596340030 >> 172.21.15.31:0/596340030 conn(0x7ff3300cc0b0 msgr2=0x7ff3300cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.128+0000 7ff3385ff700 1 -- 172.21.15.31:0/596340030 shutdown_connections 2023-10-06T05:17:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.129+0000 7fda6affd700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fda700670b0 0x7fda70069560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.129+0000 7fda6affd700 1 -- 172.21.15.31:0/996730017 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fda7c085b80 con 0x7fda84069560 2023-10-06T05:17:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.129+0000 7fda6affd700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] conn(0x7fda7006f960 0x7fda70071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.129+0000 7fda6affd700 1 -- 172.21.15.31:0/996730017 --> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda70072420 con 0x7fda7006f960 2023-10-06T05:17:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.129+0000 7fda6affd700 1 -- 172.21.15.31:0/996730017 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda7c085eb0 con 0x7fda84069560 2023-10-06T05:17:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.130+0000 7fda8359e700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] conn(0x7fda7006f960 0x7fda70071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7fda8359e700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] conn(0x7fda7006f960 0x7fda70071d70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7fda8259c700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fda700670b0 0x7fda70069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.128+0000 7f4decbff700 1 Processor -- start 2023-10-06T05:17:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.129+0000 7f4decbff700 1 -- start start 2023-10-06T05:17:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7f4decbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4de8147380 0x7f4de8069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7f4decbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4de80c6e70 0x7f4de80ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7f4decbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de806a100 con 0x7f4de8069cb0 2023-10-06T05:17:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7f4decbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de80c9c80 con 0x7f4de80c6e70 2023-10-06T05:17:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7f4decbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de80c72d0 con 0x7f4de8147380 2023-10-06T05:17:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.128+0000 7ff3385ff700 1 -- 172.21.15.31:0/596340030 wait complete. 2023-10-06T05:17:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.130+0000 7ff3385ff700 1 Processor -- start 2023-10-06T05:17:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.129+0000 7f0f74bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f70147380 0x7f0f70069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.130+0000 7f0f74bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7006a100 con 0x7f0f70069cb0 2023-10-06T05:17:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.130+0000 7f0f74bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f700c74d0 con 0x7f0f700c6e70 2023-10-06T05:17:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.130+0000 7f0f74bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700c9c80 con 0x7f0f70147380 2023-10-06T05:17:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.130+0000 7f0f6e59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f70147380 0x7f0f70069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.130+0000 7f0f6e59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f70147380 0x7f0f70069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49616/0 (socket says 172.21.15.31:49616) 2023-10-06T05:17:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.130+0000 7f0f6e59c700 1 -- 172.21.15.31:0/1026928643 learned_addr learned my addr 172.21.15.31:0/1026928643 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.130+0000 7f0f67fff700 1 --1- 172.21.15.31:0/1026928643 >> v1:172.21.15.31:6789/0 conn(0x7f0f70069cb0 0x7f0f700c6760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44732/0 (socket says 172.21.15.31:44732) 2023-10-06T05:17:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7f0f6e59c700 1 -- 172.21.15.31:0/1026928643 >> v1:172.21.15.62:6789/0 conn(0x7f0f700c6e70 legacy=0x7f0f700ca0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:17:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7f0f6e59c700 1 -- 172.21.15.31:0/1026928643 >> v1:172.21.15.31:6789/0 conn(0x7f0f70069cb0 legacy=0x7f0f700c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7f0f6e59c700 1 -- 172.21.15.31:0/1026928643 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f700c77b0 con 0x7f0f70147380 2023-10-06T05:17:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7f0f6e59c700 1 --2- 172.21.15.31:0/1026928643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f70147380 0x7f0f70069770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f600099c0 tx=0x7f0f60009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86553aaaf1aacbc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7f6ba2dff700 1 Processor -- start 2023-10-06T05:17:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7ff3385ff700 1 -- start start 2023-10-06T05:17:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7ff3385ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 0x7ff330145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7ff3385ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3300c6ca0 0x7ff330157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7ff3385ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff330147380 0x7ff3300c1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7ff3385ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300b1920 con 0x7ff3300c6ca0 2023-10-06T05:17:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7ff3385ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300af260 con 0x7ff330147380 2023-10-06T05:17:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7ff3385ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff330155ab0 con 0x7ff330069cb0 2023-10-06T05:17:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.133+0000 7f0f6dd9b700 1 -- 172.21.15.31:0/1026928643 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60005c10 con 0x7f0f70147380 2023-10-06T05:17:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.133+0000 7f0f6dd9b700 1 -- 172.21.15.31:0/1026928643 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f60005d70 con 0x7f0f70147380 2023-10-06T05:17:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.133+0000 7f0f6dd9b700 1 -- 172.21.15.31:0/1026928643 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6000b860 con 0x7f0f70147380 2023-10-06T05:17:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7f6ba2dff700 1 -- start start 2023-10-06T05:17:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.133+0000 7fda6affd700 1 -- 172.21.15.31:0/996730017 <== osd.0 v2:172.21.15.31:6800/114853 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fda70072420 con 0x7fda7006f960 2023-10-06T05:17:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.133+0000 7ff33639b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 0x7ff330145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.133+0000 7ff33639b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 0x7ff330145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49622/0 (socket says 172.21.15.31:49622) 2023-10-06T05:17:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.133+0000 7ff33639b700 1 -- 172.21.15.31:0/3758827514 learned_addr learned my addr 172.21.15.31:0/3758827514 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.133+0000 7ff335b9a700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3300c6ca0 0x7ff330157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7ff336b9c700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff330147380 0x7ff3300c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7fda8259c700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fda700670b0 0x7fda70069560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda740099b0 tx=0x7fda74017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.128+0000 7fe079ffb700 1 -- 172.21.15.31:0/167537689 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe074019030 con 0x7fe0840c5950 2023-10-06T05:17:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7fe079ffb700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe068067080 0x7fe068069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.131+0000 7fe079ffb700 1 -- 172.21.15.31:0/167537689 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe0740862e0 con 0x7fe0840c5950 2023-10-06T05:17:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7fe079ffb700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] conn(0x7fe06806f930 0x7fe068071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7fe079ffb700 1 -- 172.21.15.31:0/167537689 --> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0680723f0 con 0x7fe06806f930 2023-10-06T05:17:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7fe079ffb700 1 -- 172.21.15.31:0/167537689 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe074086610 con 0x7fe0840c5950 2023-10-06T05:17:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7fe08a59c700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] conn(0x7fe06806f930 0x7fe068071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.133+0000 7fe08a59c700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] conn(0x7fe06806f930 0x7fe068071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7fe08959a700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe068067080 0x7fe068069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7f4de6d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4de80c6e70 0x7f4de80ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7f4de6d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4de80c6e70 0x7f4de80ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34254/0 (socket says 172.21.15.31:34254) 2023-10-06T05:17:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7f4de6d9d700 1 -- 172.21.15.31:0/1956126293 learned_addr learned my addr 172.21.15.31:0/1956126293 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7f4de6d9d700 1 -- 172.21.15.31:0/1956126293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4de8147380 msgr2=0x7f4de8069770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:17:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7f4de6d9d700 1 --2- 172.21.15.31:0/1956126293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4de8147380 0x7f4de8069770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7f4de6d9d700 1 -- 172.21.15.31:0/1956126293 >> v1:172.21.15.31:6789/0 conn(0x7f4de8069cb0 legacy=0x7f4de80c6760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:17:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.132+0000 7f4de6d9d700 1 -- 172.21.15.31:0/1956126293 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de80c74d0 con 0x7f4de80c6e70 2023-10-06T05:17:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7f4de6d9d700 1 --2- 172.21.15.31:0/1956126293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de80c6e70 0x7f4de80ca0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4de00099e0 tx=0x7f4de0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae40f1261a2ee6ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f4de559a700 1 -- 172.21.15.31:0/1956126293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000b7e0 con 0x7f4de80c6e70 2023-10-06T05:17:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f4de559a700 1 -- 172.21.15.31:0/1956126293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de000b940 con 0x7f4de80c6e70 2023-10-06T05:17:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f4de559a700 1 -- 172.21.15.31:0/1956126293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000bd00 con 0x7f4de80c6e70 2023-10-06T05:17:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7ff335b9a700 1 -- 172.21.15.31:0/3758827514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 msgr2=0x7ff330145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7ff335b9a700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff330069cb0 0x7ff330145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7ff335b9a700 1 -- 172.21.15.31:0/3758827514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff330147380 msgr2=0x7ff3300c1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7ff335b9a700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff330147380 0x7ff3300c1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7ff335b9a700 1 -- 172.21.15.31:0/3758827514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3200094f0 con 0x7ff3300c6ca0 2023-10-06T05:17:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7f0f74bff700 1 -- 172.21.15.31:0/1026928643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f70147380 msgr2=0x7f0f70069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.134+0000 7f0f74bff700 1 --2- 172.21.15.31:0/1026928643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f70147380 0x7f0f70069770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f600099c0 tx=0x7f0f60009220 comp rx=0 tx=0).stop 2023-10-06T05:17:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7ff335b9a700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3300c6ca0 0x7ff330157620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff33015dfc0 tx=0x7ff320017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f6ba2dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0cf5a0 0x7f6b9c0ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f6ba2dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b9c0cd270 0x7f6b9c163240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f6ba2dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9c0ce090 con 0x7f6b9c15a930 2023-10-06T05:17:09.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f6ba2dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9c0cb6f0 con 0x7f6b9c0cd270 2023-10-06T05:17:09.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f6ba2dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c0cb9d0 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f0f74bff700 1 -- 172.21.15.31:0/1026928643 shutdown_connections 2023-10-06T05:17:09.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f0f74bff700 1 --2- 172.21.15.31:0/1026928643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f70147380 0x7f0f70069770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f0f74bff700 1 -- 172.21.15.31:0/1026928643 >> 172.21.15.31:0/1026928643 conn(0x7f0f700cc0b0 msgr2=0x7f0f700cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f6ba139c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b9c0cd270 0x7f6b9c163240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f6ba0b9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6b9c15a930 0x7f6b9c0cac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:34126/0 (socket says 172.21.15.31:34126) 2023-10-06T05:17:09.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.135+0000 7f6ba0b9b700 1 -- 172.21.15.31:0/2301981722 learned_addr learned my addr 172.21.15.31:0/2301981722 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:09.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f6b9bfff700 1 --2- 172.21.15.31:0/2301981722 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0cf5a0 0x7f6b9c0ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7ff3267fc700 1 -- 172.21.15.31:0/3758827514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32001f070 con 0x7ff3300c6ca0 2023-10-06T05:17:09.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7ff3267fc700 1 -- 172.21.15.31:0/3758827514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff320002f30 con 0x7ff3300c6ca0 2023-10-06T05:17:09.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7ff3267fc700 1 -- 172.21.15.31:0/3758827514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff320011bd0 con 0x7ff3300c6ca0 2023-10-06T05:17:09.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f6ba139c700 1 -- 172.21.15.31:0/2301981722 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0cf5a0 msgr2=0x7f6b9c0ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f6ba139c700 1 --2- 172.21.15.31:0/2301981722 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0cf5a0 0x7f6b9c0ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f6ba139c700 1 -- 172.21.15.31:0/2301981722 >> v1:172.21.15.62:6789/0 conn(0x7f6b9c15a930 legacy=0x7f6b9c0cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f6b9b7fe700 1 -- 172.21.15.31:0/2301981722 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058097404 0 0) 0x7f6b9c0ce090 con 0x7f6b9c15a930 2023-10-06T05:17:09.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7fe08959a700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe068067080 0x7fe068069530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe080006fd0 tx=0x7fe080008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f6ba139c700 1 -- 172.21.15.31:0/2301981722 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c0cbcb0 con 0x7f6b9c0cd270 2023-10-06T05:17:09.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f4decbff700 1 -- 172.21.15.31:0/1956126293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de80c6e70 msgr2=0x7f4de80ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f4decbff700 1 --2- 172.21.15.31:0/1956126293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de80c6e70 0x7f4de80ca0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4de00099e0 tx=0x7f4de0009250 comp rx=0 tx=0).stop 2023-10-06T05:17:09.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7fe079ffb700 1 -- 172.21.15.31:0/167537689 <== osd.2 v2:172.21.15.31:6808/114858 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe0680723f0 con 0x7fe06806f930 2023-10-06T05:17:09.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f4decbff700 1 -- 172.21.15.31:0/1956126293 shutdown_connections 2023-10-06T05:17:09.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f4decbff700 1 --2- 172.21.15.31:0/1956126293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de80c6e70 0x7f4de80ca0d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f4decbff700 1 --2- 172.21.15.31:0/1956126293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4de8147380 0x7f4de8069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff330155d90 con 0x7ff3300c6ca0 2023-10-06T05:17:09.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3300aeef0 con 0x7ff3300c6ca0 2023-10-06T05:17:09.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.136+0000 7f4decbff700 1 -- 172.21.15.31:0/1956126293 >> 172.21.15.31:0/1956126293 conn(0x7f4de80cc0b0 msgr2=0x7f4de80cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.137+0000 7f4decbff700 1 -- 172.21.15.31:0/1956126293 shutdown_connections 2023-10-06T05:17:09.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.137+0000 7f6ba139c700 1 --2- 172.21.15.31:0/2301981722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0cd270 0x7f6b9c163240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0098e0 tx=0x7f6b8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33dd4c513c3a8b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.137+0000 7f4decbff700 1 -- 172.21.15.31:0/1956126293 wait complete. 2023-10-06T05:17:09.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.137+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff318000fd0 con 0x7ff3300c6ca0 2023-10-06T05:17:09.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.137+0000 7f0f74bff700 1 -- 172.21.15.31:0/1026928643 shutdown_connections 2023-10-06T05:17:09.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.137+0000 7f0f74bff700 1 -- 172.21.15.31:0/1026928643 wait complete. 2023-10-06T05:17:09.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.139+0000 7f4decbff700 1 Processor -- start 2023-10-06T05:17:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.140+0000 7f0f74bff700 1 Processor -- start 2023-10-06T05:17:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.140+0000 7f4decbff700 1 -- start start 2023-10-06T05:17:09.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.140+0000 7ff3267fc700 1 -- 172.21.15.31:0/3758827514 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff320023070 con 0x7ff3300c6ca0 2023-10-06T05:17:09.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.141+0000 7f4decbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4de8069cb0 0x7f4de8137a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.141+0000 7f4decbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4de80c6e70 0x7f4de80c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.141+0000 7f4decbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8147380 0x7f4de80c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.141+0000 7f4decbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8138280 con 0x7f4de8147380 2023-10-06T05:17:09.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.141+0000 7f4decbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de80acbe0 con 0x7f4de80c6e70 2023-10-06T05:17:09.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.141+0000 7f4decbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8137fb0 con 0x7f4de8069cb0 2023-10-06T05:17:09.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.140+0000 7f0f74bff700 1 -- start start 2023-10-06T05:17:09.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.141+0000 7f0f74bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f70069cb0 0x7f0f70176e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f0f74bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f700c6e70 0x7f0f701738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f0f74bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f70147380 0x7f0f700ab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f0f74bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700ae680 con 0x7f0f70147380 2023-10-06T05:17:09.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f0f74bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700af670 con 0x7f0f700c6e70 2023-10-06T05:17:09.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f0f74bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700adb00 con 0x7f0f70069cb0 2023-10-06T05:17:09.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.139+0000 7f6b9b7fe700 1 -- 172.21.15.31:0/2301981722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c009e80 con 0x7f6b9c0cd270 2023-10-06T05:17:09.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.139+0000 7f6b9b7fe700 1 -- 172.21.15.31:0/2301981722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c00b8f0 con 0x7f6b9c0cd270 2023-10-06T05:17:09.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.139+0000 7f6b9b7fe700 1 -- 172.21.15.31:0/2301981722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c00bc70 con 0x7f6b9c0cd270 2023-10-06T05:17:09.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.140+0000 7f6ba2dff700 1 -- 172.21.15.31:0/2301981722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0cd270 msgr2=0x7f6b9c163240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.140+0000 7f6ba2dff700 1 --2- 172.21.15.31:0/2301981722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0cd270 0x7f6b9c163240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0098e0 tx=0x7f6b8c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.140+0000 7f6ba2dff700 1 -- 172.21.15.31:0/2301981722 shutdown_connections 2023-10-06T05:17:09.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.140+0000 7f6ba2dff700 1 --2- 172.21.15.31:0/2301981722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0cd270 0x7f6b9c163240 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.140+0000 7f6ba2dff700 1 --2- 172.21.15.31:0/2301981722 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0cf5a0 0x7f6b9c0ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.140+0000 7f6ba2dff700 1 -- 172.21.15.31:0/2301981722 >> 172.21.15.31:0/2301981722 conn(0x7f6b9c099720 msgr2=0x7f6b9c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f4de659c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4de8069cb0 0x7f4de8137a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f4de659c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4de8069cb0 0x7f4de8137a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49638/0 (socket says 172.21.15.31:49638) 2023-10-06T05:17:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f4de659c700 1 -- 172.21.15.31:0/1309159735 learned_addr learned my addr 172.21.15.31:0/1309159735 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.141+0000 7ff3267fc700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff314067080 0x7ff314069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7ff3267fc700 1 -- 172.21.15.31:0/3758827514 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff3200862d0 con 0x7ff3300c6ca0 2023-10-06T05:17:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7ff3267fc700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] conn(0x7ff31406f930 0x7ff314071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7ff3267fc700 1 -- 172.21.15.31:0/3758827514 --> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3140723f0 con 0x7ff31406f930 2023-10-06T05:17:09.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7ff3267fc700 1 -- 172.21.15.31:0/3758827514 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff320086600 con 0x7ff3300c6ca0 2023-10-06T05:17:09.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f4de5d9b700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4de80c6e70 0x7f4de80c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f0f6ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f70147380 0x7f0f700ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f0f6ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f70147380 0x7f0f700ab700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34284/0 (socket says 172.21.15.31:34284) 2023-10-06T05:17:09.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f0f6ed9d700 1 -- 172.21.15.31:0/1833231017 learned_addr learned my addr 172.21.15.31:0/1833231017 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7f4de659c700 1 -- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4de80c6e70 msgr2=0x7f4de80c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7f4de659c700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4de80c6e70 0x7f4de80c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7f4de659c700 1 -- 172.21.15.31:0/1309159735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8147380 msgr2=0x7f4de80c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7f4de659c700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8147380 0x7f4de80c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7f4de659c700 1 -- 172.21.15.31:0/1309159735 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de00094f0 con 0x7f4de8069cb0 2023-10-06T05:17:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7ff33639b700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff314067080 0x7ff314069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7ff336b9c700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] conn(0x7ff31406f930 0x7ff314071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7f0f6ed9d700 1 -- 172.21.15.31:0/1833231017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f70069cb0 msgr2=0x7f0f70176e30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:17:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7f0f6ed9d700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f70069cb0 0x7f0f70176e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7f0f6ed9d700 1 -- 172.21.15.31:0/1833231017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f700c6e70 msgr2=0x7f0f701738e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:17:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7f0f6ed9d700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f700c6e70 0x7f0f701738e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.143+0000 7f0f6ed9d700 1 -- 172.21.15.31:0/1833231017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f60009480 con 0x7f0f70147380 2023-10-06T05:17:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.144+0000 7f4de659c700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4de8069cb0 0x7f4de8137a70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800bc60 tx=0x7f4dd8009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.144+0000 7f0f6ed9d700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f70147380 0x7f0f700ab700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800bbb0 tx=0x7f0f68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.144+0000 7f4dde7fc700 1 -- 172.21.15.31:0/1309159735 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800c8f0 con 0x7f4de8069cb0 2023-10-06T05:17:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.144+0000 7f4dde7fc700 1 -- 172.21.15.31:0/1309159735 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd800ca50 con 0x7f4de8069cb0 2023-10-06T05:17:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.144+0000 7f4dde7fc700 1 -- 172.21.15.31:0/1309159735 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800ce10 con 0x7f4de8069cb0 2023-10-06T05:17:09.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.144+0000 7f0f667fc700 1 -- 172.21.15.31:0/1833231017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680092e0 con 0x7f0f70147380 2023-10-06T05:17:09.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.144+0000 7f0f667fc700 1 -- 172.21.15.31:0/1833231017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6800c7a0 con 0x7f0f70147380 2023-10-06T05:17:09.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.144+0000 7f0f667fc700 1 -- 172.21.15.31:0/1833231017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6800cb40 con 0x7f0f70147380 2023-10-06T05:17:09.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.142+0000 7f6ba2dff700 1 -- 172.21.15.31:0/2301981722 shutdown_connections 2023-10-06T05:17:09.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.145+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de80ac2b0 con 0x7f4de8069cb0 2023-10-06T05:17:09.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.144+0000 7ff33639b700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff314067080 0x7ff314069530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff32c007ce0 tx=0x7ff32c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.145+0000 7ff336b9c700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] conn(0x7ff31406f930 0x7ff314071d40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.145+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de80ac870 con 0x7f4de8069cb0 2023-10-06T05:17:09.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.145+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f700adde0 con 0x7f0f70147380 2023-10-06T05:17:09.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.145+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f700ae3a0 con 0x7f0f70147380 2023-10-06T05:17:09.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.146+0000 7f6ba2dff700 1 -- 172.21.15.31:0/2301981722 wait complete. 2023-10-06T05:17:09.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.146+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4de804c380 con 0x7f4de8069cb0 2023-10-06T05:17:09.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.147+0000 7ff3267fc700 1 -- 172.21.15.31:0/3758827514 <== osd.5 v2:172.21.15.62:6816/115134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff3140723f0 con 0x7ff31406f930 2023-10-06T05:17:09.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.148+0000 7f6ba2dff700 1 Processor -- start 2023-10-06T05:17:09.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.149+0000 7f6ba2dff700 1 -- start start 2023-10-06T05:17:09.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.149+0000 7f4dde7fc700 1 -- 172.21.15.31:0/1309159735 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4dd8014620 con 0x7f4de8069cb0 2023-10-06T05:17:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.150+0000 7f4dde7fc700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4dc8067080 0x7f4dc8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.150+0000 7f4dde7fc700 1 -- 172.21.15.31:0/1309159735 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4dd8084830 con 0x7f4de8069cb0 2023-10-06T05:17:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.150+0000 7f4dde7fc700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] conn(0x7f4dc806f8b0 0x7f4dc8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.151+0000 7f4dde7fc700 1 -- 172.21.15.31:0/1309159735 --> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4dc8072370 con 0x7f4dc806f8b0 2023-10-06T05:17:09.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.151+0000 7f4dde7fc700 1 -- 172.21.15.31:0/1309159735 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dd8051920 con 0x7f4de8069cb0 2023-10-06T05:17:09.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.151+0000 7f4de5d9b700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4dc8067080 0x7f4dc8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.151+0000 7f4de6d9d700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] conn(0x7f4dc806f8b0 0x7f4dc8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.153+0000 7f4de6d9d700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] conn(0x7f4dc806f8b0 0x7f4dc8071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.149+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f54000fd0 con 0x7f0f70147380 2023-10-06T05:17:09.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.150+0000 7f0f667fc700 1 -- 172.21.15.31:0/1833231017 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f68012070 con 0x7f0f70147380 2023-10-06T05:17:09.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f0f667fc700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0f50067050 0x7f0f50069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f0f667fc700 1 -- 172.21.15.31:0/1833231017 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0f680844a0 con 0x7f0f70147380 2023-10-06T05:17:09.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f0f667fc700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] conn(0x7f0f5006f880 0x7f0f50071c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f0f667fc700 1 -- 172.21.15.31:0/1833231017 --> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f50072340 con 0x7f0f5006f880 2023-10-06T05:17:09.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f0f6e59c700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0f50067050 0x7f0f50069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.153+0000 7f0f667fc700 1 -- 172.21.15.31:0/1833231017 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f68051590 con 0x7f0f70147380 2023-10-06T05:17:09.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.154+0000 7f0f6e59c700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0f50067050 0x7f0f50069500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009910 tx=0x7f0f60015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.154+0000 7f0f67fff700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] conn(0x7f0f5006f880 0x7f0f50071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.150+0000 7f6ba2dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0cd270 0x7f6b9c0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.150+0000 7f6ba2dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0cf5a0 0x7f6b9c0caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.150+0000 7f6ba2dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b9c15a930 0x7f6b9c175eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.150+0000 7f6ba2dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c09b7f0 con 0x7f6b9c0cd270 2023-10-06T05:17:09.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.150+0000 7f6ba2dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c06ce80 con 0x7f6b9c15a930 2023-10-06T05:17:09.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.150+0000 7f6ba2dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c06d160 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.151+0000 7f6ba0b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0cd270 0x7f6b9c0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.151+0000 7f6ba0b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0cd270 0x7f6b9c0caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34288/0 (socket says 172.21.15.31:34288) 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.151+0000 7f6ba0b9b700 1 -- 172.21.15.31:0/3016801305 learned_addr learned my addr 172.21.15.31:0/3016801305 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f6ba139c700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b9c15a930 0x7f6b9c175eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f6b9bfff700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0cf5a0 0x7f6b9c0caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f6b9bfff700 1 -- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b9c15a930 msgr2=0x7f6b9c175eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f6b9bfff700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b9c15a930 0x7f6b9c175eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f6b9bfff700 1 -- 172.21.15.31:0/3016801305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0cd270 msgr2=0x7f6b9c0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f6b9bfff700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9c0cd270 0x7f6b9c0caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.152+0000 7f6b9bfff700 1 -- 172.21.15.31:0/3016801305 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c009420 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.153+0000 7f6b9bfff700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0cf5a0 0x7f6b9c0caf80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400cf00 tx=0x7f6b940077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.154+0000 7f6b98ff9700 1 -- 172.21.15.31:0/3016801305 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9400ad70 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.154+0000 7f6b98ff9700 1 -- 172.21.15.31:0/3016801305 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b94016430 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.154+0000 7f6b98ff9700 1 -- 172.21.15.31:0/3016801305 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b940167f0 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.154+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9c06d440 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.154+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9c1763f0 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.155+0000 7f4dde7fc700 1 -- 172.21.15.31:0/1309159735 <== osd.4 v2:172.21.15.62:6800/115118 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4dc8072370 con 0x7f4dc806f8b0 2023-10-06T05:17:09.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.156+0000 7f4de5d9b700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4dc8067080 0x7f4dc8069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0006fd0 tx=0x7f4dd0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.155+0000 7f0f67fff700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] conn(0x7f0f5006f880 0x7f0f50071c90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.157+0000 7f0f667fc700 1 -- 172.21.15.31:0/1833231017 <== osd.6 v2:172.21.15.62:6821/115137 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0f50072340 con 0x7f0f5006f880 2023-10-06T05:17:09.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.157+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6b9c04c380 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.158+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 --> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8d18068570 con 0x7f8cf806f930 2023-10-06T05:17:09.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.158+0000 7f6b98ff9700 1 -- 172.21.15.31:0/3016801305 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b94003020 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.158+0000 7f8d0f7fe700 1 -- 172.21.15.31:0/806661727 <== osd.3 v2:172.21.15.31:6823/114865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8d18068570 con 0x7f8cf806f930 2023-10-06T05:17:09.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.159+0000 7f6b98ff9700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b88067000 0x7f6b880694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.159+0000 7f6b98ff9700 1 -- 172.21.15.31:0/3016801305 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6b94012070 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.159+0000 7f6b98ff9700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] conn(0x7f6b8806f8b0 0x7f6b88071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.159+0000 7f6b98ff9700 1 -- 172.21.15.31:0/3016801305 --> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b88072370 con 0x7f6b8806f8b0 2023-10-06T05:17:09.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.159+0000 7f6b98ff9700 1 -- 172.21.15.31:0/3016801305 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b94087800 con 0x7f6b9c0cf5a0 2023-10-06T05:17:09.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.159+0000 7f6ba0b9b700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b88067000 0x7f6b880694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.160+0000 7f6ba139c700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] conn(0x7f6b8806f8b0 0x7f6b88071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.160+0000 7f6ba0b9b700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b88067000 0x7f6b880694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007fd0 tx=0x7f6b90007d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.161+0000 7f6ba139c700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] conn(0x7f6b8806f8b0 0x7f6b88071cc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.161+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 --> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe2040c5250 con 0x7fe1f006f8b0 2023-10-06T05:17:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.161+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] conn(0x7f8cf806f930 msgr2=0x7f8cf8071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.161+0000 7f8d1d7ff700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] conn(0x7f8cf806f930 0x7f8cf8071d40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.161+0000 7fe1ebfff700 1 -- 172.21.15.31:0/919955586 <== osd.1 v2:172.21.15.31:6816/114859 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe2040c5250 con 0x7fe1f006f8b0 2023-10-06T05:17:09.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.162+0000 7f6b98ff9700 1 -- 172.21.15.31:0/3016801305 <== osd.7 v2:172.21.15.62:6808/115119 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6b88072370 con 0x7f6b8806f8b0 2023-10-06T05:17:09.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.162+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8cf8067080 msgr2=0x7f8cf8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.162+0000 7f8d1d7ff700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8cf8067080 0x7f8cf8069530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d00007fd0 tx=0x7f8d00007f60 comp rx=0 tx=0).stop 2023-10-06T05:17:09.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.162+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d180c6ca0 msgr2=0x7f8d180c1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.162+0000 7f8d1d7ff700 1 --2- 172.21.15.31:0/806661727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d180c6ca0 0x7f8d180c1350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d1806c960 tx=0x7f8d08016000 comp rx=0 tx=0).stop 2023-10-06T05:17:09.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.162+0000 7f8d177fe700 1 -- 172.21.15.31:0/806661727 reap_dead start 2023-10-06T05:17:09.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.162+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 shutdown_connections 2023-10-06T05:17:09.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.162+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 >> 172.21.15.31:0/806661727 conn(0x7f8d180cc0b0 msgr2=0x7f8d18160bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.163+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] conn(0x7fe1f006f8b0 msgr2=0x7fe1f0071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.163+0000 7fe20a7ff700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] conn(0x7fe1f006f8b0 0x7fe1f0071cc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.163+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 shutdown_connections 2023-10-06T05:17:09.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.163+0000 7f8d1d7ff700 1 -- 172.21.15.31:0/806661727 wait complete. 2023-10-06T05:17:09.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.164+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe1f0067080 msgr2=0x7fe1f0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.164+0000 7fe20a7ff700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe1f0067080 0x7fe1f0069530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009ab0 tx=0x7fe1fc018000 comp rx=0 tx=0).stop 2023-10-06T05:17:09.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.164+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe204069560 msgr2=0x7fe2040add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.164+0000 7fe20a7ff700 1 --2- 172.21.15.31:0/919955586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe204069560 0x7fe2040add50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800b600 tx=0x7fe1f80091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.164+0000 7fe208d9c700 1 -- 172.21.15.31:0/919955586 reap_dead start 2023-10-06T05:17:09.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.165+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 shutdown_connections 2023-10-06T05:17:09.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.165+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 >> 172.21.15.31:0/919955586 conn(0x7fe2040cc0b0 msgr2=0x7fe2040ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.165+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 shutdown_connections 2023-10-06T05:17:09.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.166+0000 7fe20a7ff700 1 -- 172.21.15.31:0/919955586 wait complete. 2023-10-06T05:17:09.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.173+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 --> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe064002d30 con 0x7fe06806f930 2023-10-06T05:17:09.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.174+0000 7fe079ffb700 1 -- 172.21.15.31:0/167537689 <== osd.2 v2:172.21.15.31:6808/114858 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe064002d30 con 0x7fe06806f930 2023-10-06T05:17:09.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.175+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] conn(0x7fe06806f930 msgr2=0x7fe068071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.175+0000 7fe08bfff700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] conn(0x7fe06806f930 0x7fe068071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.176+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe068067080 msgr2=0x7fe068069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.176+0000 7fe08bfff700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe068067080 0x7fe068069530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe080006fd0 tx=0x7fe080008040 comp rx=0 tx=0).stop 2023-10-06T05:17:09.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.176+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0840c5950 msgr2=0x7fe0840c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.176+0000 7fe08bfff700 1 --2- 172.21.15.31:0/167537689 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0840c5950 0x7fe0840c5d20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe074007b90 tx=0x7fe074015000 comp rx=0 tx=0).stop 2023-10-06T05:17:09.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.176+0000 7fe08a59c700 1 -- 172.21.15.31:0/167537689 reap_dead start 2023-10-06T05:17:09.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.176+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 shutdown_connections 2023-10-06T05:17:09.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.176+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 >> 172.21.15.31:0/167537689 conn(0x7fe0840cc0b0 msgr2=0x7fe0840cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.177+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 --> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda841475b0 con 0x7fda7006f960 2023-10-06T05:17:09.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.177+0000 7fda6affd700 1 -- 172.21.15.31:0/996730017 <== osd.0 v2:172.21.15.31:6800/114853 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fda78003670 con 0x7fda7006f960 2023-10-06T05:17:09.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.180+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 shutdown_connections 2023-10-06T05:17:09.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.180+0000 7fe08bfff700 1 -- 172.21.15.31:0/167537689 wait complete. 2023-10-06T05:17:09.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.181+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] conn(0x7fda7006f960 msgr2=0x7fda70071d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.181+0000 7fda891ff700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] conn(0x7fda7006f960 0x7fda70071d70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.181+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fda700670b0 msgr2=0x7fda70069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.181+0000 7fda891ff700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fda700670b0 0x7fda70069560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda740099b0 tx=0x7fda74017000 comp rx=0 tx=0).stop 2023-10-06T05:17:09.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.181+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda84069560 msgr2=0x7fda84146c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.181+0000 7fda891ff700 1 --2- 172.21.15.31:0/996730017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda84069560 0x7fda84146c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009fa0 tx=0x7fda7c009880 comp rx=0 tx=0).stop 2023-10-06T05:17:09.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.181+0000 7fda8359e700 1 -- 172.21.15.31:0/996730017 reap_dead start 2023-10-06T05:17:09.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.182+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 shutdown_connections 2023-10-06T05:17:09.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.182+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 >> 172.21.15.31:0/996730017 conn(0x7fda840cc0b0 msgr2=0x7fda8406e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.182+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 shutdown_connections 2023-10-06T05:17:09.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.182+0000 7fda891ff700 1 -- 172.21.15.31:0/996730017 wait complete. 2023-10-06T05:17:09.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.188+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 --> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff318002d30 con 0x7ff31406f930 2023-10-06T05:17:09.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.189+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 --> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0f54002d30 con 0x7f0f5006f880 2023-10-06T05:17:09.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.189+0000 7f0f667fc700 1 -- 172.21.15.31:0/1833231017 <== osd.6 v2:172.21.15.62:6821/115137 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0f54002d30 con 0x7f0f5006f880 2023-10-06T05:17:09.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.190+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 --> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4de80ca160 con 0x7f4dc806f8b0 2023-10-06T05:17:09.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.190+0000 7ff3267fc700 1 -- 172.21.15.31:0/3758827514 <== osd.5 v2:172.21.15.62:6816/115134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff318002d30 con 0x7ff31406f930 2023-10-06T05:17:09.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.190+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 >> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] conn(0x7f0f5006f880 msgr2=0x7f0f50071c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.190+0000 7f0f74bff700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] conn(0x7f0f5006f880 0x7f0f50071c90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.191+0000 7f4dde7fc700 1 -- 172.21.15.31:0/1309159735 <== osd.4 v2:172.21.15.62:6800/115118 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4de80ca160 con 0x7f4dc806f8b0 2023-10-06T05:17:09.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.191+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0f50067050 msgr2=0x7f0f50069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.191+0000 7f0f74bff700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0f50067050 0x7f0f50069500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009910 tx=0x7f0f60015000 comp rx=0 tx=0).stop 2023-10-06T05:17:09.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.191+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f70147380 msgr2=0x7f0f700ab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.191+0000 7f0f74bff700 1 --2- 172.21.15.31:0/1833231017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f70147380 0x7f0f700ab700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800bbb0 tx=0x7f0f68009840 comp rx=0 tx=0).stop 2023-10-06T05:17:09.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.191+0000 7f0f6ed9d700 1 -- 172.21.15.31:0/1833231017 reap_dead start 2023-10-06T05:17:09.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.191+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 shutdown_connections 2023-10-06T05:17:09.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.191+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 >> 172.21.15.31:0/1833231017 conn(0x7f0f700cc0b0 msgr2=0x7f0f700ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.191+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 >> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] conn(0x7ff31406f930 msgr2=0x7ff314071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.191+0000 7ff3385ff700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] conn(0x7ff31406f930 0x7ff314071d40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 shutdown_connections 2023-10-06T05:17:09.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] conn(0x7f4dc806f8b0 msgr2=0x7f4dc8071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7f4decbff700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] conn(0x7f4dc806f8b0 0x7f4dc8071cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff314067080 msgr2=0x7ff314069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7ff3385ff700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff314067080 0x7ff314069530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff32c007ce0 tx=0x7ff32c008040 comp rx=0 tx=0).stop 2023-10-06T05:17:09.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3300c6ca0 msgr2=0x7ff330157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7ff3385ff700 1 --2- 172.21.15.31:0/3758827514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3300c6ca0 0x7ff330157620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff33015dfc0 tx=0x7ff320017000 comp rx=0 tx=0).stop 2023-10-06T05:17:09.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7ff336b9c700 1 -- 172.21.15.31:0/3758827514 reap_dead start 2023-10-06T05:17:09.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 shutdown_connections 2023-10-06T05:17:09.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 >> 172.21.15.31:0/3758827514 conn(0x7ff3300cc0b0 msgr2=0x7ff3300c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.192+0000 7f0f74bff700 1 -- 172.21.15.31:0/1833231017 wait complete. 2023-10-06T05:17:09.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 shutdown_connections 2023-10-06T05:17:09.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7ff3385ff700 1 -- 172.21.15.31:0/3758827514 wait complete. 2023-10-06T05:17:09.211 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-10-06T05:17:09.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T05:17:09.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4dc8067080 msgr2=0x7f4dc8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7f4decbff700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4dc8067080 0x7f4dc8069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0006fd0 tx=0x7f4dd0009060 comp rx=0 tx=0).stop 2023-10-06T05:17:09.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4de8069cb0 msgr2=0x7f4de8137a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7f4decbff700 1 --2- 172.21.15.31:0/1309159735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4de8069cb0 0x7f4de8137a70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800bc60 tx=0x7f4dd8009820 comp rx=0 tx=0).stop 2023-10-06T05:17:09.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7f4de6d9d700 1 -- 172.21.15.31:0/1309159735 reap_dead start 2023-10-06T05:17:09.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 shutdown_connections 2023-10-06T05:17:09.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 >> 172.21.15.31:0/1309159735 conn(0x7f4de80cc0b0 msgr2=0x7f4de806f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 shutdown_connections 2023-10-06T05:17:09.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.193+0000 7f4decbff700 1 -- 172.21.15.31:0/1309159735 wait complete. 2023-10-06T05:17:09.215 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:09.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T05:17:09.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.195+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 --> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6b9c1645e0 con 0x7f6b8806f8b0 2023-10-06T05:17:09.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.196+0000 7f6b98ff9700 1 -- 172.21.15.31:0/3016801305 <== osd.7 v2:172.21.15.62:6808/115119 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6b9c1645e0 con 0x7f6b8806f8b0 2023-10-06T05:17:09.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.197+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] conn(0x7f6b8806f8b0 msgr2=0x7f6b88071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.197+0000 7f6ba2dff700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] conn(0x7f6b8806f8b0 0x7f6b88071cc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.197+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b88067000 msgr2=0x7f6b880694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.197+0000 7f6ba2dff700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b88067000 0x7f6b880694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b90007fd0 tx=0x7f6b90007d70 comp rx=0 tx=0).stop 2023-10-06T05:17:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.197+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0cf5a0 msgr2=0x7f6b9c0caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.197+0000 7f6ba2dff700 1 --2- 172.21.15.31:0/3016801305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b9c0cf5a0 0x7f6b9c0caf80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400cf00 tx=0x7f6b940077f0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.197+0000 7f6ba139c700 1 -- 172.21.15.31:0/3016801305 reap_dead start 2023-10-06T05:17:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.203+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 shutdown_connections 2023-10-06T05:17:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.203+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 >> 172.21.15.31:0/3016801305 conn(0x7f6b9c099720 msgr2=0x7f6b9c0cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.203+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 shutdown_connections 2023-10-06T05:17:09.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.203+0000 7f6ba2dff700 1 -- 172.21.15.31:0/3016801305 wait complete. 2023-10-06T05:17:09.220 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:09.221 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T05:17:09.221 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:09.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T05:17:09.225 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:09.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T05:17:09.231 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:09.231 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:09.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T05:17:09.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T05:17:09.234 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:09.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T05:17:09.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.573+0000 7fc0749ff700 1 Processor -- start 2023-10-06T05:17:09.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.573+0000 7fc0749ff700 1 -- start start 2023-10-06T05:17:09.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.575+0000 7fc0749ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc07015d840 0x7fc0700c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.575+0000 7fc0749ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc070069560 con 0x7fc07015dc10 2023-10-06T05:17:09.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.575+0000 7fc0749ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc070069840 con 0x7fc07006ace0 2023-10-06T05:17:09.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.575+0000 7fc0749ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0700cac00 con 0x7fc07015d840 2023-10-06T05:17:09.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.576+0000 7fc06e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc07015d840 0x7fc0700c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.576+0000 7fc06e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc07015d840 0x7fc0700c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34302/0 (socket says 172.21.15.31:34302) 2023-10-06T05:17:09.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.576+0000 7fc06e59c700 1 -- 172.21.15.31:0/3968359396 learned_addr learned my addr 172.21.15.31:0/3968359396 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.577+0000 7fc06e59c700 1 -- 172.21.15.31:0/3968359396 >> v1:172.21.15.62:6789/0 conn(0x7fc07006ace0 legacy=0x7fc0701675a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T05:17:09.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.577+0000 7fc06e59c700 1 -- 172.21.15.31:0/3968359396 >> v1:172.21.15.31:6789/0 conn(0x7fc07015dc10 legacy=0x7fc0700c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.577+0000 7fc06e59c700 1 -- 172.21.15.31:0/3968359396 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0700caee0 con 0x7fc07015d840 2023-10-06T05:17:09.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.578+0000 7fc06e59c700 1 --2- 172.21.15.31:0/3968359396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07015d840 0x7fc0700c6900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc0600099b0 tx=0x7fc060009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97c9f8dc3af4909 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.578+0000 7fc06d59a700 1 -- 172.21.15.31:0/3968359396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06000b7e0 con 0x7fc07015d840 2023-10-06T05:17:09.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.579+0000 7fc06d59a700 1 -- 172.21.15.31:0/3968359396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06000b940 con 0x7fc07015d840 2023-10-06T05:17:09.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.579+0000 7fc06d59a700 1 -- 172.21.15.31:0/3968359396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06000bce0 con 0x7fc07015d840 2023-10-06T05:17:09.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.580+0000 7fc0749ff700 1 -- 172.21.15.31:0/3968359396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07015d840 msgr2=0x7fc0700c6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.580+0000 7fc0749ff700 1 --2- 172.21.15.31:0/3968359396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07015d840 0x7fc0700c6900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc0600099b0 tx=0x7fc060009250 comp rx=0 tx=0).stop 2023-10-06T05:17:09.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.580+0000 7fc0749ff700 1 -- 172.21.15.31:0/3968359396 shutdown_connections 2023-10-06T05:17:09.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.580+0000 7fc0749ff700 1 --2- 172.21.15.31:0/3968359396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07015d840 0x7fc0700c6900 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.580+0000 7fc0749ff700 1 -- 172.21.15.31:0/3968359396 >> 172.21.15.31:0/3968359396 conn(0x7fc0700cc0b0 msgr2=0x7fc0700cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.580+0000 7f8768bff700 1 Processor -- start 2023-10-06T05:17:09.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.580+0000 7f8768bff700 1 -- start start 2023-10-06T05:17:09.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.582+0000 7fc0749ff700 1 -- 172.21.15.31:0/3968359396 shutdown_connections 2023-10-06T05:17:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.582+0000 7f8768bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8764069560 0x7f876406aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.582+0000 7f8768bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876406d5e0 con 0x7f87640ccbc0 2023-10-06T05:17:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.582+0000 7f8768bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876406a5e0 con 0x7f876406eef0 2023-10-06T05:17:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.582+0000 7f8768bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876406ea50 con 0x7f8764069560 2023-10-06T05:17:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.582+0000 7fc0749ff700 1 -- 172.21.15.31:0/3968359396 wait complete. 2023-10-06T05:17:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.584+0000 7f8761d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f87640ccbc0 0x7f876406e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44746/0 (socket says 172.21.15.31:44746) 2023-10-06T05:17:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.584+0000 7f8761d9b700 1 -- 172.21.15.31:0/1923747602 learned_addr learned my addr 172.21.15.31:0/1923747602 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:09.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.584+0000 7f876259c700 1 --2- 172.21.15.31:0/1923747602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8764069560 0x7f876406aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.584+0000 7fc0749ff700 1 Processor -- start 2023-10-06T05:17:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.584+0000 7fc0749ff700 1 -- start start 2023-10-06T05:17:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.585+0000 7f876159a700 1 -- 172.21.15.31:0/1923747602 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747013657 0 0) 0x7f876406d5e0 con 0x7f87640ccbc0 2023-10-06T05:17:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.585+0000 7f876159a700 1 -- 172.21.15.31:0/1923747602 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f874c003570 con 0x7f87640ccbc0 2023-10-06T05:17:09.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.585+0000 7f876259c700 1 -- 172.21.15.31:0/1923747602 >> v1:172.21.15.62:6789/0 conn(0x7f876406eef0 legacy=0x7f87640ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.585+0000 7f876259c700 1 -- 172.21.15.31:0/1923747602 >> v1:172.21.15.31:6789/0 conn(0x7f87640ccbc0 legacy=0x7f876406e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.585+0000 7f876259c700 1 -- 172.21.15.31:0/1923747602 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764069cb0 con 0x7f8764069560 2023-10-06T05:17:09.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.586+0000 7fc0749ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07006ace0 0x7fc07013e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.586+0000 7fc0749ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07015d840 0x7fc070142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.586+0000 7fc0749ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07015dc10 0x7fc0701604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.586+0000 7fc0749ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0700b0430 con 0x7fc07015dc10 2023-10-06T05:17:09.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.586+0000 7fc0749ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc070143420 con 0x7fc07015d840 2023-10-06T05:17:09.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.586+0000 7fc0749ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0700af820 con 0x7fc07006ace0 2023-10-06T05:17:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.586+0000 7f876259c700 1 --2- 172.21.15.31:0/1923747602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8764069560 0x7f876406aa70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8758009910 tx=0x7f87580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=783ff2927b9ed970 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7fc06dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07015d840 0x7fc070142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7fc06dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07015d840 0x7fc070142ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53406/0 (socket says 172.21.15.31:53406) 2023-10-06T05:17:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7fc06dd9b700 1 -- 172.21.15.31:0/1299452751 learned_addr learned my addr 172.21.15.31:0/1299452751 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7fc06e59c700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07006ace0 0x7fc07013e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7fc06dd9b700 1 -- 172.21.15.31:0/1299452751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07006ace0 msgr2=0x7fc07013e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7fc06dd9b700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07006ace0 0x7fc07013e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7fc06dd9b700 1 -- 172.21.15.31:0/1299452751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07015dc10 msgr2=0x7fc0701604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7fc06dd9b700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07015dc10 0x7fc0701604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7fc06dd9b700 1 -- 172.21.15.31:0/1299452751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0600094f0 con 0x7fc07015d840 2023-10-06T05:17:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.586+0000 7f876159a700 1 -- 172.21.15.31:0/1923747602 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875800b6d0 con 0x7f8764069560 2023-10-06T05:17:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7f876159a700 1 -- 172.21.15.31:0/1923747602 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875800b830 con 0x7f8764069560 2023-10-06T05:17:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.587+0000 7f876159a700 1 -- 172.21.15.31:0/1923747602 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875800bbf0 con 0x7f8764069560 2023-10-06T05:17:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.588+0000 7f8768bff700 1 -- 172.21.15.31:0/1923747602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8764069560 msgr2=0x7f876406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.588+0000 7f8768bff700 1 --2- 172.21.15.31:0/1923747602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8764069560 0x7f876406aa70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8758009910 tx=0x7f87580091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.588+0000 7fc06dd9b700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07015d840 0x7fc070142ee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc06800bbb0 tx=0x7fc068009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.588+0000 7f8768bff700 1 -- 172.21.15.31:0/1923747602 shutdown_connections 2023-10-06T05:17:09.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.588+0000 7f8768bff700 1 --2- 172.21.15.31:0/1923747602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8764069560 0x7f876406aa70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.588+0000 7f8768bff700 1 -- 172.21.15.31:0/1923747602 >> 172.21.15.31:0/1923747602 conn(0x7f87640cc0b0 msgr2=0x7f87640cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.589+0000 7fc05e7fc700 1 -- 172.21.15.31:0/1299452751 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0680092e0 con 0x7fc07015d840 2023-10-06T05:17:09.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.589+0000 7fc05e7fc700 1 -- 172.21.15.31:0/1299452751 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06800c7a0 con 0x7fc07015d840 2023-10-06T05:17:09.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.589+0000 7fc05e7fc700 1 -- 172.21.15.31:0/1299452751 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06800cb60 con 0x7fc07015d840 2023-10-06T05:17:09.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.588+0000 7f062efff700 1 Processor -- start 2023-10-06T05:17:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.589+0000 7fc0749ff700 1 -- 172.21.15.31:0/1299452751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0700afb00 con 0x7fc07015d840 2023-10-06T05:17:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.589+0000 7fc0749ff700 1 -- 172.21.15.31:0/1299452751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0700b00c0 con 0x7fc07015d840 2023-10-06T05:17:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.589+0000 7f8768bff700 1 -- 172.21.15.31:0/1923747602 shutdown_connections 2023-10-06T05:17:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.591+0000 7f062efff700 1 -- start start 2023-10-06T05:17:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.589+0000 7f8768bff700 1 -- 172.21.15.31:0/1923747602 wait complete. 2023-10-06T05:17:09.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.591+0000 7f8768bff700 1 Processor -- start 2023-10-06T05:17:09.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.593+0000 7f062efff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0628147380 0x7f0628069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.593+0000 7f062efff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06280c6e70 0x7f06280ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.593+0000 7f062efff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062806a100 con 0x7f0628069cb0 2023-10-06T05:17:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.593+0000 7f062efff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06280c9c80 con 0x7f06280c6e70 2023-10-06T05:17:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.593+0000 7f062efff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06280c72b0 con 0x7f0628147380 2023-10-06T05:17:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.592+0000 7f8768bff700 1 -- start start 2023-10-06T05:17:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.593+0000 7f8768bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8764069560 0x7f8764154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.593+0000 7f8768bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f876406eef0 0x7f8764170f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.593+0000 7f8768bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640ccbc0 0x7f8764171460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.594+0000 7f8768bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87641487a0 con 0x7f8764069560 2023-10-06T05:17:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.594+0000 7f8768bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8764148a80 con 0x7f87640ccbc0 2023-10-06T05:17:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.594+0000 7f8768bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8764148d60 con 0x7f876406eef0 2023-10-06T05:17:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.595+0000 7f876259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8764069560 0x7f8764154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.595+0000 7f876259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8764069560 0x7f8764154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34342/0 (socket says 172.21.15.31:34342) 2023-10-06T05:17:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.595+0000 7f876259c700 1 -- 172.21.15.31:0/257530331 learned_addr learned my addr 172.21.15.31:0/257530331 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.594+0000 7fc05e7fc700 1 -- 172.21.15.31:0/1299452751 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc068012070 con 0x7fc07015d840 2023-10-06T05:17:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.595+0000 7fc05e7fc700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc050067080 0x7fc050069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.595+0000 7fc05e7fc700 1 -- 172.21.15.31:0/1299452751 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc068084310 con 0x7fc07015d840 2023-10-06T05:17:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.595+0000 7f8761d9b700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f876406eef0 0x7f8764170f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f8762d9d700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640ccbc0 0x7f8764171460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f876259c700 1 -- 172.21.15.31:0/257530331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f876406eef0 msgr2=0x7f8764170f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f876259c700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f876406eef0 0x7f8764170f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f876259c700 1 -- 172.21.15.31:0/257530331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640ccbc0 msgr2=0x7f8764171460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f876259c700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640ccbc0 0x7f8764171460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f876259c700 1 -- 172.21.15.31:0/257530331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758009420 con 0x7f8764069560 2023-10-06T05:17:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f8762d9d700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640ccbc0 0x7f8764171460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f876259c700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8764069560 0x7f8764154e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f87580030c0 tx=0x7f8758016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f062d59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06280c6e70 0x7f06280ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f062d59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06280c6e70 0x7f06280ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34332/0 (socket says 172.21.15.31:34332) 2023-10-06T05:17:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.596+0000 7f062d59c700 1 -- 172.21.15.31:0/3265637704 learned_addr learned my addr 172.21.15.31:0/3265637704 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.597+0000 7f87527fc700 1 -- 172.21.15.31:0/257530331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875800b9a0 con 0x7f8764069560 2023-10-06T05:17:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.597+0000 7f87527fc700 1 -- 172.21.15.31:0/257530331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758020070 con 0x7f8764069560 2023-10-06T05:17:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.597+0000 7f87527fc700 1 -- 172.21.15.31:0/257530331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875801b450 con 0x7f8764069560 2023-10-06T05:17:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.597+0000 7f8768bff700 1 -- 172.21.15.31:0/257530331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8764149040 con 0x7f8764069560 2023-10-06T05:17:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.597+0000 7fc0749ff700 1 -- 172.21.15.31:0/1299452751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0540052b0 con 0x7fc07015d840 2023-10-06T05:17:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.597+0000 7f062d59c700 1 -- 172.21.15.31:0/3265637704 >> v2:172.21.15.62:3300/0 conn(0x7f0628147380 msgr2=0x7f0628069770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:17:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.597+0000 7f062d59c700 1 --2- 172.21.15.31:0/3265637704 >> v2:172.21.15.62:3300/0 conn(0x7f0628147380 0x7f0628069770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.597+0000 7f062d59c700 1 -- 172.21.15.31:0/3265637704 >> v1:172.21.15.31:6789/0 conn(0x7f0628069cb0 legacy=0x7f06280c6760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:17:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.597+0000 7f062d59c700 1 -- 172.21.15.31:0/3265637704 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06280c74d0 con 0x7f06280c6e70 2023-10-06T05:17:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.598+0000 7fc06e59c700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc050067080 0x7fc050069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.599+0000 7f062d59c700 1 --2- 172.21.15.31:0/3265637704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06280c6e70 0x7f06280ca0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f062806c960 tx=0x7f061c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b77fcea371c2da6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.598+0000 7f8768bff700 1 -- 172.21.15.31:0/257530331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87641483c0 con 0x7f8764069560 2023-10-06T05:17:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.599+0000 7fc06e59c700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc050067080 0x7fc050069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc0600099b0 tx=0x7fc060016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.600+0000 7f06277fe700 1 -- 172.21.15.31:0/3265637704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c00b7e0 con 0x7f06280c6e70 2023-10-06T05:17:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.600+0000 7f06277fe700 1 -- 172.21.15.31:0/3265637704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c00b940 con 0x7f06280c6e70 2023-10-06T05:17:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.600+0000 7f06277fe700 1 -- 172.21.15.31:0/3265637704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c00bce0 con 0x7f06280c6e70 2023-10-06T05:17:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.600+0000 7f874bfff700 1 -- 172.21.15.31:0/257530331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87380052b0 con 0x7f8764069560 2023-10-06T05:17:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.600+0000 7f062efff700 1 -- 172.21.15.31:0/3265637704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06280c6e70 msgr2=0x7f06280ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.600+0000 7f062efff700 1 --2- 172.21.15.31:0/3265637704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06280c6e70 0x7f06280ca0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f062806c960 tx=0x7f061c009250 comp rx=0 tx=0).stop 2023-10-06T05:17:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.601+0000 7f062efff700 1 -- 172.21.15.31:0/3265637704 shutdown_connections 2023-10-06T05:17:09.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.601+0000 7f062efff700 1 --2- 172.21.15.31:0/3265637704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06280c6e70 0x7f06280ca0d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.601+0000 7f062efff700 1 --2- 172.21.15.31:0/3265637704 >> v2:172.21.15.62:3300/0 conn(0x7f0628147380 0x7f0628069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.601+0000 7f062efff700 1 -- 172.21.15.31:0/3265637704 >> 172.21.15.31:0/3265637704 conn(0x7f06280cc0b0 msgr2=0x7f06280cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.601+0000 7f062efff700 1 -- 172.21.15.31:0/3265637704 shutdown_connections 2023-10-06T05:17:09.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.601+0000 7f062efff700 1 -- 172.21.15.31:0/3265637704 wait complete. 2023-10-06T05:17:09.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.603+0000 7f062efff700 1 Processor -- start 2023-10-06T05:17:09.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.603+0000 7fc05e7fc700 1 -- 172.21.15.31:0/1299452751 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc068051400 con 0x7fc07015d840 2023-10-06T05:17:09.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.604+0000 7f062efff700 1 -- start start 2023-10-06T05:17:09.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.605+0000 7f062efff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0628069cb0 0x7f0628146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.605+0000 7f062efff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06280c6e70 0x7f06280c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.605+0000 7f062efff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0628147380 0x7f06281570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.605+0000 7f062efff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06280c5600 con 0x7f0628069cb0 2023-10-06T05:17:09.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.605+0000 7f062efff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06280c58e0 con 0x7f0628147380 2023-10-06T05:17:09.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.605+0000 7f062efff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06280c5bc0 con 0x7f06280c6e70 2023-10-06T05:17:09.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.607+0000 7f062d59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0628147380 0x7f06281570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.607+0000 7fe31d1ff700 1 Processor -- start 2023-10-06T05:17:09.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.608+0000 7f0627fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06280c6e70 0x7f06280c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.609+0000 7f0627fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06280c6e70 0x7f06280c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49712/0 (socket says 172.21.15.31:49712) 2023-10-06T05:17:09.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.609+0000 7f0627fff700 1 -- 172.21.15.31:0/1032396887 learned_addr learned my addr 172.21.15.31:0/1032396887 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.606+0000 7f26afbff700 1 Processor -- start 2023-10-06T05:17:09.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.606+0000 7f26afbff700 1 -- start start 2023-10-06T05:17:09.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.608+0000 7f26afbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f26a815d840 0x7f26a80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.608+0000 7f26afbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26a806ace0 0x7f26a80ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.608+0000 7f26afbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a8069560 con 0x7f26a815dc10 2023-10-06T05:17:09.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.608+0000 7f26afbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a8069840 con 0x7f26a815d840 2023-10-06T05:17:09.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.608+0000 7f26afbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a80cac00 con 0x7f26a806ace0 2023-10-06T05:17:09.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.609+0000 7f26ad19a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f26a815dc10 0x7f26a80c9eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:34174/0 (socket says 172.21.15.31:34174) 2023-10-06T05:17:09.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.609+0000 7f26ad19a700 1 -- 172.21.15.31:0/623929384 learned_addr learned my addr 172.21.15.31:0/623929384 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:09.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.608+0000 7fe31d1ff700 1 -- start start 2023-10-06T05:17:09.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.609+0000 7f0627fff700 1 -- 172.21.15.31:0/1032396887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0628147380 msgr2=0x7f06281570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.609+0000 7f0627fff700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0628147380 0x7f06281570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.609+0000 7f0627fff700 1 -- 172.21.15.31:0/1032396887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0628069cb0 msgr2=0x7f0628146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.609+0000 7f0627fff700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0628069cb0 0x7f0628146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.609+0000 7f0627fff700 1 -- 172.21.15.31:0/1032396887 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c0094f0 con 0x7f06280c6e70 2023-10-06T05:17:09.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.610+0000 7f0627fff700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06280c6e70 0x7f06280c4b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f061800b5e0 tx=0x7f06180090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.610+0000 7fe31d1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe318069560 0x7fe31806e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.610+0000 7fe31d1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180ccbc0 0x7fe318155000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.610+0000 7fe31d1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31806d5e0 con 0x7fe31806f140 2023-10-06T05:17:09.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.610+0000 7fe31d1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31806b180 con 0x7fe318069560 2023-10-06T05:17:09.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.610+0000 7fe31d1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31806a5e0 con 0x7fe3180ccbc0 2023-10-06T05:17:09.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.610+0000 7fe31759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180ccbc0 0x7fe318155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7fe31759e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180ccbc0 0x7fe318155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49726/0 (socket says 172.21.15.31:49726) 2023-10-06T05:17:09.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7fe31759e700 1 -- 172.21.15.31:0/4192123199 learned_addr learned my addr 172.21.15.31:0/4192123199 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7f0624ff9700 1 -- 172.21.15.31:0/1032396887 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061800f510 con 0x7f06280c6e70 2023-10-06T05:17:09.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7f0624ff9700 1 -- 172.21.15.31:0/1032396887 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061800f670 con 0x7f06280c6e70 2023-10-06T05:17:09.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7f0624ff9700 1 -- 172.21.15.31:0/1032396887 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061800fa30 con 0x7f06280c6e70 2023-10-06T05:17:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7f26ae19c700 1 --2- 172.21.15.31:0/623929384 >> v2:172.21.15.62:3300/0 conn(0x7f26a806ace0 0x7f26a80ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7f99197ff700 1 Processor -- start 2023-10-06T05:17:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7f062efff700 1 -- 172.21.15.31:0/1032396887 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06280c5ea0 con 0x7f06280c6e70 2023-10-06T05:17:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7f062efff700 1 -- 172.21.15.31:0/1032396887 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06280b1290 con 0x7f06280c6e70 2023-10-06T05:17:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7fe31759e700 1 -- 172.21.15.31:0/4192123199 >> v2:172.21.15.31:3300/0 conn(0x7fe318069560 msgr2=0x7fe31806e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7fe31759e700 1 --2- 172.21.15.31:0/4192123199 >> v2:172.21.15.31:3300/0 conn(0x7fe318069560 0x7fe31806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7fe31759e700 1 -- 172.21.15.31:0/4192123199 >> v1:172.21.15.31:6789/0 conn(0x7fe31806f140 legacy=0x7fe31806aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7fe31759e700 1 -- 172.21.15.31:0/4192123199 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3180699d0 con 0x7fe3180ccbc0 2023-10-06T05:17:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.611+0000 7fe31759e700 1 --2- 172.21.15.31:0/4192123199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180ccbc0 0x7fe318155000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe310009f20 tx=0x7fe310009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abc4d086dd9c1968 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7f26ae19c700 1 -- 172.21.15.31:0/623929384 >> v2:172.21.15.31:3300/0 conn(0x7f26a815d840 msgr2=0x7f26a80c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7f26ae19c700 1 --2- 172.21.15.31:0/623929384 >> v2:172.21.15.31:3300/0 conn(0x7f26a815d840 0x7f26a80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7f26ae19c700 1 -- 172.21.15.31:0/623929384 >> v1:172.21.15.62:6789/0 conn(0x7f26a815dc10 legacy=0x7f26a80c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7f26ae19c700 1 -- 172.21.15.31:0/623929384 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80caee0 con 0x7f26a806ace0 2023-10-06T05:17:09.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7f99197ff700 1 -- start start 2023-10-06T05:17:09.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7f8677dff700 1 Processor -- start 2023-10-06T05:17:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7f26ae19c700 1 --2- 172.21.15.31:0/623929384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26a806ace0 0x7f26a80ca5c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f26a80697b0 tx=0x7f26a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc3076b5d2ffcfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7f062efff700 1 -- 172.21.15.31:0/1032396887 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062804c380 con 0x7f06280c6e70 2023-10-06T05:17:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7f26ac999700 1 -- 172.21.15.31:0/623929384 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a400b670 con 0x7f26a806ace0 2023-10-06T05:17:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.613+0000 7f26ac999700 1 -- 172.21.15.31:0/623929384 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a400b7d0 con 0x7f26a806ace0 2023-10-06T05:17:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.613+0000 7f26ac999700 1 -- 172.21.15.31:0/623929384 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a400bb70 con 0x7f26a806ace0 2023-10-06T05:17:09.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7fe315d9b700 1 -- 172.21.15.31:0/4192123199 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31000e070 con 0x7fe3180ccbc0 2023-10-06T05:17:09.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7fe315d9b700 1 -- 172.21.15.31:0/4192123199 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31000bad0 con 0x7fe3180ccbc0 2023-10-06T05:17:09.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7fe315d9b700 1 -- 172.21.15.31:0/4192123199 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3100092e0 con 0x7fe3180ccbc0 2023-10-06T05:17:09.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.613+0000 7fe31d1ff700 1 -- 172.21.15.31:0/4192123199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180ccbc0 msgr2=0x7fe318155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.613+0000 7fe31d1ff700 1 --2- 172.21.15.31:0/4192123199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180ccbc0 0x7fe318155000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe310009f20 tx=0x7fe310009840 comp rx=0 tx=0).stop 2023-10-06T05:17:09.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.613+0000 7f1e09bff700 1 Processor -- start 2023-10-06T05:17:09.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f8677dff700 1 -- start start 2023-10-06T05:17:09.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f26afbff700 1 -- 172.21.15.31:0/623929384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26a806ace0 msgr2=0x7f26a80ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f26afbff700 1 --2- 172.21.15.31:0/623929384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26a806ace0 0x7f26a80ca5c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f26a80697b0 tx=0x7f26a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f1e09bff700 1 -- start start 2023-10-06T05:17:09.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f26afbff700 1 -- 172.21.15.31:0/623929384 shutdown_connections 2023-10-06T05:17:09.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f26afbff700 1 --2- 172.21.15.31:0/623929384 >> v2:172.21.15.31:3300/0 conn(0x7f26a815d840 0x7f26a80c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f26afbff700 1 --2- 172.21.15.31:0/623929384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26a806ace0 0x7f26a80ca5c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f26afbff700 1 -- 172.21.15.31:0/623929384 >> 172.21.15.31:0/623929384 conn(0x7f26a80cc0b0 msgr2=0x7f26a80cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f99197ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9914147380 0x7f9914069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f99197ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9914069cb0 0x7f99140c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f26afbff700 1 -- 172.21.15.31:0/623929384 shutdown_connections 2023-10-06T05:17:09.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f26afbff700 1 -- 172.21.15.31:0/623929384 wait complete. 2023-10-06T05:17:09.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.612+0000 7f87527fc700 1 -- 172.21.15.31:0/257530331 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8758022030 con 0x7f8764069560 2023-10-06T05:17:09.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f87527fc700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f874c067540 0x7f874c0699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f87527fc700 1 -- 172.21.15.31:0/257530331 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8758086430 con 0x7f8764069560 2023-10-06T05:17:09.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f87527fc700 1 -- 172.21.15.31:0/257530331 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87580868c0 con 0x7f8764069560 2023-10-06T05:17:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f99197ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991406a100 con 0x7f99140c6ca0 2023-10-06T05:17:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f99197ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99140c9c80 con 0x7f9914069cb0 2023-10-06T05:17:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.614+0000 7f99197ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99140c74d0 con 0x7f9914147380 2023-10-06T05:17:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.615+0000 7f9912ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9914147380 0x7f9914069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.615+0000 7f9912ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9914147380 0x7f9914069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53438/0 (socket says 172.21.15.31:53438) 2023-10-06T05:17:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.615+0000 7f9912ffd700 1 -- 172.21.15.31:0/1367322954 learned_addr learned my addr 172.21.15.31:0/1367322954 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.615+0000 7fe31d1ff700 1 -- 172.21.15.31:0/4192123199 shutdown_connections 2023-10-06T05:17:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.615+0000 7fe31d1ff700 1 --2- 172.21.15.31:0/4192123199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3180ccbc0 0x7fe318155000 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.615+0000 7fe31d1ff700 1 --2- 172.21.15.31:0/4192123199 >> v2:172.21.15.31:3300/0 conn(0x7fe318069560 0x7fe31806e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.615+0000 7fe31d1ff700 1 -- 172.21.15.31:0/4192123199 >> 172.21.15.31:0/4192123199 conn(0x7fe3180cc0b0 msgr2=0x7fe3180cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.615+0000 7f99127fc700 1 --2- 172.21.15.31:0/1367322954 >> v2:172.21.15.31:3300/0 conn(0x7f9914069cb0 0x7f99140c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.616+0000 7f99127fc700 1 -- 172.21.15.31:0/1367322954 >> v2:172.21.15.62:3300/0 conn(0x7f9914147380 msgr2=0x7f9914069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.616+0000 7f99127fc700 1 --2- 172.21.15.31:0/1367322954 >> v2:172.21.15.62:3300/0 conn(0x7f9914147380 0x7f9914069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.616+0000 7f99127fc700 1 -- 172.21.15.31:0/1367322954 >> v1:172.21.15.62:6789/0 conn(0x7f99140c6ca0 legacy=0x7f99141615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.616+0000 7f99127fc700 1 -- 172.21.15.31:0/1367322954 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99140c77b0 con 0x7f9914069cb0 2023-10-06T05:17:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.616+0000 7f8677dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86700ccbc0 0x7f867006e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.616+0000 7f8677dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867006d5e0 con 0x7f8670069560 2023-10-06T05:17:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.616+0000 7f8677dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867006a5e0 con 0x7f867006eef0 2023-10-06T05:17:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.616+0000 7f8677dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867006b180 con 0x7f86700ccbc0 2023-10-06T05:17:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.617+0000 7f99127fc700 1 --2- 172.21.15.31:0/1367322954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9914069cb0 0x7f99140c6760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f991406c960 tx=0x7f99080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bd681b410690c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.617+0000 7fe31d1ff700 1 -- 172.21.15.31:0/4192123199 shutdown_connections 2023-10-06T05:17:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.616+0000 7f867539a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86700ccbc0 0x7f867006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.617+0000 7f867539a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86700ccbc0 0x7f867006e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53444/0 (socket says 172.21.15.31:53444) 2023-10-06T05:17:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.617+0000 7f867539a700 1 -- 172.21.15.31:0/1108123515 learned_addr learned my addr 172.21.15.31:0/1108123515 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.617+0000 7f26afbff700 1 Processor -- start 2023-10-06T05:17:09.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.617+0000 7fe31d1ff700 1 -- 172.21.15.31:0/4192123199 wait complete. 2023-10-06T05:17:09.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.617+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247180847 0 0) 0x7f867006a5e0 con 0x7f867006eef0 2023-10-06T05:17:09.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.617+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f865c003570 con 0x7f867006eef0 2023-10-06T05:17:09.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.615+0000 7f8761d9b700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f874c067540 0x7f874c0699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.617+0000 7f8761d9b700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f874c067540 0x7f874c0699f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f87641581c0 tx=0x7f875c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599611585 0 0) 0x7f867006d5e0 con 0x7f8670069560 2023-10-06T05:17:09.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f867006a5e0 con 0x7f8670069560 2023-10-06T05:17:09.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.617+0000 7f26afbff700 1 -- start start 2023-10-06T05:17:09.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f9911ffb700 1 -- 172.21.15.31:0/1367322954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800b670 con 0x7f9914069cb0 2023-10-06T05:17:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f9911ffb700 1 -- 172.21.15.31:0/1367322954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800b7d0 con 0x7f9914069cb0 2023-10-06T05:17:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f9911ffb700 1 -- 172.21.15.31:0/1367322954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800bb70 con 0x7f9914069cb0 2023-10-06T05:17:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1235370235 0 0) 0x7f865c003570 con 0x7f867006eef0 2023-10-06T05:17:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 >> v2:172.21.15.62:3300/0 conn(0x7f86700ccbc0 msgr2=0x7f867006e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 --2- 172.21.15.31:0/1108123515 >> v2:172.21.15.62:3300/0 conn(0x7f86700ccbc0 0x7f867006e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 >> v1:172.21.15.31:6789/0 conn(0x7f8670069560 legacy=0x7f867006aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670069cb0 con 0x7f867006eef0 2023-10-06T05:17:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f86680017f0 con 0x7f867006eef0 2023-10-06T05:17:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 >> v1:172.21.15.62:6789/0 conn(0x7f867006eef0 legacy=0x7f86700ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 --2- 172.21.15.31:0/1108123515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c003c40 0x7f865c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 --2- 172.21.15.31:0/1108123515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f865c004810 0x7f865c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 --2- 172.21.15.31:0/1108123515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0052e0 0x7f865c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867006b180 con 0x7f865c0052e0 2023-10-06T05:17:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c00c1b0 con 0x7f865c003c40 2023-10-06T05:17:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.618+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c00c370 con 0x7f865c004810 2023-10-06T05:17:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f99197ff700 1 -- 172.21.15.31:0/1367322954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9914069cb0 msgr2=0x7f99140c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f99197ff700 1 --2- 172.21.15.31:0/1367322954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9914069cb0 0x7f99140c6760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f991406c960 tx=0x7f99080091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f99197ff700 1 -- 172.21.15.31:0/1367322954 shutdown_connections 2023-10-06T05:17:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f99197ff700 1 --2- 172.21.15.31:0/1367322954 >> v2:172.21.15.62:3300/0 conn(0x7f9914147380 0x7f9914069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f99197ff700 1 --2- 172.21.15.31:0/1367322954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9914069cb0 0x7f99140c6760 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f99197ff700 1 -- 172.21.15.31:0/1367322954 >> 172.21.15.31:0/1367322954 conn(0x7f99140cc0b0 msgr2=0x7f99140cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f26afbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26a806ace0 0x7f26a815c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f26afbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a815d840 0x7f26a80b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f26afbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26a815dc10 0x7f26a814ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f26afbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a8171f20 con 0x7f26a815d840 2023-10-06T05:17:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f26afbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80b0d20 con 0x7f26a815dc10 2023-10-06T05:17:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f26afbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80b0fc0 con 0x7f26a806ace0 2023-10-06T05:17:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f26ad99b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26a806ace0 0x7f26a815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f26ad99b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26a806ace0 0x7f26a815c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49742/0 (socket says 172.21.15.31:49742) 2023-10-06T05:17:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f26ad99b700 1 -- 172.21.15.31:0/3582465266 learned_addr learned my addr 172.21.15.31:0/3582465266 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f26ae19c700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26a815dc10 0x7f26a814ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f1e09bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1e04147380 0x7f1e04069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f1e09bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e04069cb0 0x7f1e040c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f1e09bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0406a100 con 0x7f1e040c6ca0 2023-10-06T05:17:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f1e09bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e040c9c80 con 0x7f1e04147380 2023-10-06T05:17:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f1e09bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040c74d0 con 0x7f1e04069cb0 2023-10-06T05:17:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7fe31d1ff700 1 Processor -- start 2023-10-06T05:17:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f1e037fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1e04147380 0x7f1e04069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f1e037fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1e04147380 0x7f1e04069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53450/0 (socket says 172.21.15.31:53450) 2023-10-06T05:17:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.619+0000 7f1e037fe700 1 -- 172.21.15.31:0/903205413 learned_addr learned my addr 172.21.15.31:0/903205413 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f1e02ffd700 1 --2- 172.21.15.31:0/903205413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e04069cb0 0x7f1e040c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f1e037fe700 1 -- 172.21.15.31:0/903205413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e04069cb0 msgr2=0x7f1e040c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f1e037fe700 1 --2- 172.21.15.31:0/903205413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e04069cb0 0x7f1e040c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f1e037fe700 1 -- 172.21.15.31:0/903205413 >> v1:172.21.15.31:6789/0 conn(0x7f1e040c6ca0 legacy=0x7f1e041615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f1e037fe700 1 -- 172.21.15.31:0/903205413 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e040c77b0 con 0x7f1e04147380 2023-10-06T05:17:09.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f867639c700 1 --2- 172.21.15.31:0/1108123515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0052e0 0x7f865c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f26ad19a700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a815d840 0x7f26a80b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f26ad99b700 1 -- 172.21.15.31:0/3582465266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26a815dc10 msgr2=0x7f26a814ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f26ad99b700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26a815dc10 0x7f26a814ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f26ad99b700 1 -- 172.21.15.31:0/3582465266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a815d840 msgr2=0x7f26a80b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f26ad99b700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a815d840 0x7f26a80b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f26ad99b700 1 -- 172.21.15.31:0/3582465266 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a4009420 con 0x7f26a806ace0 2023-10-06T05:17:09.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f8675b9b700 1 --2- 172.21.15.31:0/1108123515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c003c40 0x7f865c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f26ad99b700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26a806ace0 0x7f26a815c950 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f26a000bc30 tx=0x7f26a0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f867639c700 1 -- 172.21.15.31:0/1108123515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f865c004810 msgr2=0x7f865c004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T05:17:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f867639c700 1 --2- 172.21.15.31:0/1108123515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f865c004810 0x7f865c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f867639c700 1 -- 172.21.15.31:0/1108123515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c003c40 msgr2=0x7f865c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f867639c700 1 --2- 172.21.15.31:0/1108123515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c003c40 0x7f865c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f867639c700 1 -- 172.21.15.31:0/1108123515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670069cb0 con 0x7f865c0052e0 2023-10-06T05:17:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f269dffb700 1 -- 172.21.15.31:0/3582465266 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a00092e0 con 0x7f26a806ace0 2023-10-06T05:17:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f26afbff700 1 -- 172.21.15.31:0/3582465266 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a80b12a0 con 0x7f26a806ace0 2023-10-06T05:17:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f867639c700 1 -- 172.21.15.31:0/1108123515 reap_dead start 2023-10-06T05:17:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f26afbff700 1 -- 172.21.15.31:0/3582465266 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a8171bb0 con 0x7f26a806ace0 2023-10-06T05:17:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f269dffb700 1 -- 172.21.15.31:0/3582465266 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a000c6f0 con 0x7f26a806ace0 2023-10-06T05:17:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f269dffb700 1 -- 172.21.15.31:0/3582465266 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a000cad0 con 0x7f26a806ace0 2023-10-06T05:17:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7f867639c700 1 --2- 172.21.15.31:0/1108123515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0052e0 0x7f865c00ba60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f866800c640 tx=0x7f8668002e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728680deeed51b2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7f0624ff9700 1 -- 172.21.15.31:0/1032396887 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0618024430 con 0x7f06280c6e70 2023-10-06T05:17:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f0624ff9700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0614067080 0x7f0614069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.621+0000 7f0624ff9700 1 -- 172.21.15.31:0/1032396887 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0618084e00 con 0x7f06280c6e70 2023-10-06T05:17:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7f062cd9b700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0614067080 0x7f0614069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7f0624ff9700 1 -- 172.21.15.31:0/1032396887 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0618052020 con 0x7f06280c6e70 2023-10-06T05:17:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.620+0000 7fe31d1ff700 1 -- start start 2023-10-06T05:17:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7fe31d1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe318069560 0x7fe31814f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7fe31d1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31806f140 0x7fe3180ac750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7fe31d1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180ccbc0 0x7fe3180c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7fe31d1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31814f960 con 0x7fe318069560 2023-10-06T05:17:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7fe31d1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3181541d0 con 0x7fe3180ccbc0 2023-10-06T05:17:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7fe31d1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe318150bc0 con 0x7fe31806f140 2023-10-06T05:17:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7fe31659c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31806f140 0x7fe3180ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7fe31659c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31806f140 0x7fe3180ac750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49770/0 (socket says 172.21.15.31:49770) 2023-10-06T05:17:09.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7fe31659c700 1 -- 172.21.15.31:0/2469627349 learned_addr learned my addr 172.21.15.31:0/2469627349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866800ce90 con 0x7f865c0052e0 2023-10-06T05:17:09.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668006ba0 con 0x7f865c0052e0 2023-10-06T05:17:09.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.622+0000 7f8674b99700 1 -- 172.21.15.31:0/1108123515 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668007020 con 0x7f865c0052e0 2023-10-06T05:17:09.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.623+0000 7f1e037fe700 1 --2- 172.21.15.31:0/903205413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04147380 0x7f1e04069770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009990 tx=0x7f1df4009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90f55a37f4191c6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.623+0000 7fe31759e700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180ccbc0 0x7fe3180c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.623+0000 7fe316d9d700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe318069560 0x7fe31814f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.623+0000 7fe31659c700 1 -- 172.21.15.31:0/2469627349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180ccbc0 msgr2=0x7fe3180c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.623+0000 7fe31659c700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180ccbc0 0x7fe3180c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.623+0000 7fe31659c700 1 -- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe318069560 msgr2=0x7fe31814f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.623+0000 7fe31659c700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe318069560 0x7fe31814f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.623+0000 7fe31659c700 1 -- 172.21.15.31:0/2469627349 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe310009ae0 con 0x7fe31806f140 2023-10-06T05:17:09.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.623+0000 7fe316d9d700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe318069560 0x7fe31814f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.623+0000 7f062cd9b700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0614067080 0x7f0614069530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f062000acf0 tx=0x7f0620009350 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.624+0000 7f99197ff700 1 -- 172.21.15.31:0/1367322954 shutdown_connections 2023-10-06T05:17:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.624+0000 7f99197ff700 1 -- 172.21.15.31:0/1367322954 wait complete. 2023-10-06T05:17:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7f269dffb700 1 -- 172.21.15.31:0/3582465266 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26a0012070 con 0x7f26a806ace0 2023-10-06T05:17:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.624+0000 7f1e027fc700 1 -- 172.21.15.31:0/903205413 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4005c10 con 0x7f1e04147380 2023-10-06T05:17:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.624+0000 7f1e027fc700 1 -- 172.21.15.31:0/903205413 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4005d70 con 0x7f1e04147380 2023-10-06T05:17:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.624+0000 7f1e027fc700 1 -- 172.21.15.31:0/903205413 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df400b860 con 0x7f1e04147380 2023-10-06T05:17:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7f1e09bff700 1 -- 172.21.15.31:0/903205413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04147380 msgr2=0x7f1e04069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7f1e09bff700 1 --2- 172.21.15.31:0/903205413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04147380 0x7f1e04069770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009990 tx=0x7f1df4009220 comp rx=0 tx=0).stop 2023-10-06T05:17:09.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.624+0000 7fe31659c700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31806f140 0x7fe3180ac750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00b810 tx=0x7fe30c0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.624+0000 7f8677dff700 1 -- 172.21.15.31:0/1108123515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0052e0 msgr2=0x7f865c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.624+0000 7f8677dff700 1 --2- 172.21.15.31:0/1108123515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0052e0 0x7f865c00ba60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f866800c640 tx=0x7f8668002e20 comp rx=0 tx=0).stop 2023-10-06T05:17:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7f8677dff700 1 -- 172.21.15.31:0/1108123515 shutdown_connections 2023-10-06T05:17:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7f8677dff700 1 --2- 172.21.15.31:0/1108123515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0052e0 0x7f865c00ba60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7f8677dff700 1 -- 172.21.15.31:0/1108123515 >> 172.21.15.31:0/1108123515 conn(0x7f86700cc0b0 msgr2=0x7f86700cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7f26afbff700 1 -- 172.21.15.31:0/3582465266 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26880052b0 con 0x7f26a806ace0 2023-10-06T05:17:09.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.626+0000 7f269dffb700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f268c066fb0 0x7f268c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.627+0000 7f269dffb700 1 -- 172.21.15.31:0/3582465266 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26a0084270 con 0x7f26a806ace0 2023-10-06T05:17:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7fe2feffd700 1 -- 172.21.15.31:0/2469627349 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00f570 con 0x7fe31806f140 2023-10-06T05:17:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7fe2feffd700 1 -- 172.21.15.31:0/2469627349 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c00f6d0 con 0x7fe31806f140 2023-10-06T05:17:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7fe2feffd700 1 -- 172.21.15.31:0/2469627349 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00fa70 con 0x7fe31806f140 2023-10-06T05:17:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.625+0000 7fe31d1ff700 1 -- 172.21.15.31:0/2469627349 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31814fcd0 con 0x7fe31806f140 2023-10-06T05:17:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.626+0000 7fe31d1ff700 1 -- 172.21.15.31:0/2469627349 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe318150290 con 0x7fe31806f140 2023-10-06T05:17:09.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.626+0000 7f8677dff700 1 -- 172.21.15.31:0/1108123515 shutdown_connections 2023-10-06T05:17:09.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.627+0000 7f99197ff700 1 Processor -- start 2023-10-06T05:17:09.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.627+0000 7f26ad19a700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f268c066fb0 0x7f268c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.627+0000 7f1e09bff700 1 -- 172.21.15.31:0/903205413 shutdown_connections 2023-10-06T05:17:09.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.627+0000 7f1e09bff700 1 --2- 172.21.15.31:0/903205413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04147380 0x7f1e04069770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.627+0000 7f1e09bff700 1 --2- 172.21.15.31:0/903205413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e04069cb0 0x7f1e040c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.627+0000 7f1e09bff700 1 -- 172.21.15.31:0/903205413 >> 172.21.15.31:0/903205413 conn(0x7f1e040cc0b0 msgr2=0x7f1e040cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:09.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.628+0000 7f99197ff700 1 -- start start 2023-10-06T05:17:09.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.627+0000 7fe31d1ff700 1 -- 172.21.15.31:0/2469627349 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe318150d50 con 0x7fe31806f140 2023-10-06T05:17:09.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.628+0000 7f26ad19a700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f268c066fb0 0x7f268c069460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2698006fd0 tx=0x7f2698008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.627+0000 7f8677dff700 1 -- 172.21.15.31:0/1108123515 wait complete. 2023-10-06T05:17:09.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f1e09bff700 1 -- 172.21.15.31:0/903205413 shutdown_connections 2023-10-06T05:17:09.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f1e09bff700 1 -- 172.21.15.31:0/903205413 wait complete. 2023-10-06T05:17:09.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f99197ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9914069cb0 0x7f9914160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f99197ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99140c6ca0 0x7f991406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f99197ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9914147380 0x7f991406f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f99197ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99140c5c00 con 0x7f9914147380 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f99197ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99140c4f80 con 0x7f9914069cb0 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f99197ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99140c5260 con 0x7f99140c6ca0 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f9912ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9914069cb0 0x7f9914160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f99127fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99140c6ca0 0x7f991406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f99127fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99140c6ca0 0x7f991406ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49776/0 (socket says 172.21.15.31:49776) 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.629+0000 7f99127fc700 1 -- 172.21.15.31:0/3534288603 learned_addr learned my addr 172.21.15.31:0/3534288603 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.630+0000 7f99137fe700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9914147380 0x7f991406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.630+0000 7f99127fc700 1 -- 172.21.15.31:0/3534288603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9914069cb0 msgr2=0x7f9914160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.630+0000 7f99127fc700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9914069cb0 0x7f9914160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.630+0000 7f99127fc700 1 -- 172.21.15.31:0/3534288603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9914147380 msgr2=0x7f991406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.630+0000 7f99127fc700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9914147380 0x7f991406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.630+0000 7f99127fc700 1 -- 172.21.15.31:0/3534288603 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9908009420 con 0x7f99140c6ca0 2023-10-06T05:17:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.630+0000 7f99127fc700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99140c6ca0 0x7f991406ed40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f99080030c0 tx=0x7f9908016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.631+0000 7f8677dff700 1 Processor -- start 2023-10-06T05:17:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.631+0000 7f1e09bff700 1 Processor -- start 2023-10-06T05:17:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.632+0000 7f269dffb700 1 -- 172.21.15.31:0/3582465266 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26a00512e0 con 0x7f26a806ace0 2023-10-06T05:17:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.632+0000 7f98faffd700 1 -- 172.21.15.31:0/3534288603 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908011dd0 con 0x7f99140c6ca0 2023-10-06T05:17:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.632+0000 7f98faffd700 1 -- 172.21.15.31:0/3534288603 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990801b470 con 0x7f99140c6ca0 2023-10-06T05:17:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.632+0000 7f98faffd700 1 -- 172.21.15.31:0/3534288603 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990801b7f0 con 0x7f99140c6ca0 2023-10-06T05:17:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.632+0000 7f99197ff700 1 -- 172.21.15.31:0/3534288603 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99140c5540 con 0x7f99140c6ca0 2023-10-06T05:17:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.632+0000 7f99197ff700 1 -- 172.21.15.31:0/3534288603 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9914160d50 con 0x7f99140c6ca0 2023-10-06T05:17:09.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.631+0000 7f1e09bff700 1 -- start start 2023-10-06T05:17:09.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.631+0000 7f8677dff700 1 -- start start 2023-10-06T05:17:09.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.633+0000 7f8677dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0052e0 0x7f86701544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.633+0000 7f8677dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86701779a0 0x7f86700ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.633+0000 7f8677dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86700ae290 0x7f86700c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.633+0000 7f8677dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8670179bb0 con 0x7f86701779a0 2023-10-06T05:17:09.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.633+0000 7f8677dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8670154dd0 con 0x7f865c0052e0 2023-10-06T05:17:09.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.633+0000 7f8677dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8670155670 con 0x7f86700ae290 2023-10-06T05:17:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.634+0000 7f99197ff700 1 -- 172.21.15.31:0/3534288603 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f40052b0 con 0x7f99140c6ca0 2023-10-06T05:17:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f1e09bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04069cb0 0x7f1e04145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f1e09bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e040c6ca0 0x7f1e040adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f1e09bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e04147380 0x7f1e040c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f1e09bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e04156ed0 con 0x7f1e04147380 2023-10-06T05:17:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f1e09bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040ada50 con 0x7f1e04069cb0 2023-10-06T05:17:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f1e09bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040ace40 con 0x7f1e040c6ca0 2023-10-06T05:17:09.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f1e037fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04069cb0 0x7f1e04145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f1e037fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04069cb0 0x7f1e04145130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53504/0 (socket says 172.21.15.31:53504) 2023-10-06T05:17:09.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f1e037fe700 1 -- 172.21.15.31:0/2786822076 learned_addr learned my addr 172.21.15.31:0/2786822076 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.634+0000 7f867639c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86700ae290 0x7f86700c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.634+0000 7f8675b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0052e0 0x7f86701544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.634+0000 7f867639c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86700ae290 0x7f86700c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49782/0 (socket says 172.21.15.31:49782) 2023-10-06T05:17:09.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.634+0000 7f8675b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0052e0 0x7f86701544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53490/0 (socket says 172.21.15.31:53490) 2023-10-06T05:17:09.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.634+0000 7f867639c700 1 -- 172.21.15.31:0/2731189451 learned_addr learned my addr 172.21.15.31:0/2731189451 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f867539a700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86701779a0 0x7f86700ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f867639c700 1 -- 172.21.15.31:0/2731189451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0052e0 msgr2=0x7f86701544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f867639c700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0052e0 0x7f86701544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f867639c700 1 -- 172.21.15.31:0/2731189451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86701779a0 msgr2=0x7f86700ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f867639c700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86701779a0 0x7f86700ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.635+0000 7f867639c700 1 -- 172.21.15.31:0/2731189451 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86700ad7f0 con 0x7f86700ae290 2023-10-06T05:17:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.636+0000 7f1e03fff700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e04147380 0x7f1e040c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.636+0000 7f1e037fe700 1 -- 172.21.15.31:0/2786822076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e040c6ca0 msgr2=0x7f1e040adee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:17:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.636+0000 7f1e037fe700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e040c6ca0 0x7f1e040adee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.636+0000 7f1e037fe700 1 -- 172.21.15.31:0/2786822076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e04147380 msgr2=0x7f1e040c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.636+0000 7f1e037fe700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e04147380 0x7f1e040c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.636+0000 7f1e037fe700 1 -- 172.21.15.31:0/2786822076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df4009480 con 0x7f1e04069cb0 2023-10-06T05:17:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.636+0000 7f1e037fe700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04069cb0 0x7f1e04145130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1df4002000 tx=0x7f1df4015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.637+0000 7fe2feffd700 1 -- 172.21.15.31:0/2469627349 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe30c024430 con 0x7fe31806f140 2023-10-06T05:17:09.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.637+0000 7f1deb7fe700 1 -- 172.21.15.31:0/2786822076 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4011aa0 con 0x7f1e04069cb0 2023-10-06T05:17:09.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.637+0000 7f1deb7fe700 1 -- 172.21.15.31:0/2786822076 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4011c00 con 0x7f1e04069cb0 2023-10-06T05:17:09.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.637+0000 7f1deb7fe700 1 -- 172.21.15.31:0/2786822076 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df401d430 con 0x7f1e04069cb0 2023-10-06T05:17:09.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.637+0000 7f1e09bff700 1 -- 172.21.15.31:0/2786822076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e040ad120 con 0x7f1e04069cb0 2023-10-06T05:17:09.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.637+0000 7f1e09bff700 1 -- 172.21.15.31:0/2786822076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e040ad6e0 con 0x7f1e04069cb0 2023-10-06T05:17:09.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.639+0000 7f1e09bff700 1 -- 172.21.15.31:0/2786822076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0404c380 con 0x7f1e04069cb0 2023-10-06T05:17:09.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.640+0000 7f98faffd700 1 -- 172.21.15.31:0/3534288603 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f990801bba0 con 0x7f99140c6ca0 2023-10-06T05:17:09.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.639+0000 7fe2feffd700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe304067080 0x7fe304069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.640+0000 7fe2feffd700 1 -- 172.21.15.31:0/2469627349 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe30c084e40 con 0x7fe31806f140 2023-10-06T05:17:09.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.641+0000 7fe316d9d700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe304067080 0x7fe304069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.641+0000 7fe2feffd700 1 -- 172.21.15.31:0/2469627349 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe30c051fb0 con 0x7fe31806f140 2023-10-06T05:17:09.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.642+0000 7fe316d9d700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe304067080 0x7fe304069530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe318153b70 tx=0x7fe308008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.642+0000 7f98faffd700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9900067000 0x7f99000694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.642+0000 7f98faffd700 1 -- 172.21.15.31:0/3534288603 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f99080866b0 con 0x7f99140c6ca0 2023-10-06T05:17:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.643+0000 7f9912ffd700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9900067000 0x7f99000694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.643+0000 7f98faffd700 1 -- 172.21.15.31:0/3534288603 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9908053820 con 0x7f99140c6ca0 2023-10-06T05:17:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.644+0000 7f9912ffd700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9900067000 0x7f99000694b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9904006ce0 tx=0x7f9904008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.642+0000 7f1deb7fe700 1 -- 172.21.15.31:0/2786822076 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1df4043b80 con 0x7f1e04069cb0 2023-10-06T05:17:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.644+0000 7f1deb7fe700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0067080 0x7f1df0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.644+0000 7f1deb7fe700 1 -- 172.21.15.31:0/2786822076 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1df40869a0 con 0x7f1e04069cb0 2023-10-06T05:17:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.644+0000 7f867639c700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86700ae290 0x7f86700c52b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86700aebf0 tx=0x7f8668015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.645+0000 7f1e02ffd700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0067080 0x7f1df0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.645+0000 7f8665ffb700 1 -- 172.21.15.31:0/2731189451 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668022070 con 0x7f86700ae290 2023-10-06T05:17:09.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.645+0000 7f8665ffb700 1 -- 172.21.15.31:0/2731189451 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668001350 con 0x7f86700ae290 2023-10-06T05:17:09.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.645+0000 7f8665ffb700 1 -- 172.21.15.31:0/2731189451 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668006d10 con 0x7f86700ae290 2023-10-06T05:17:09.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.646+0000 7f1deb7fe700 1 -- 172.21.15.31:0/2786822076 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1df4053b10 con 0x7f1e04069cb0 2023-10-06T05:17:09.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.645+0000 7f8677dff700 1 -- 172.21.15.31:0/2731189451 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86700ae720 con 0x7f86700ae290 2023-10-06T05:17:09.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.645+0000 7f8677dff700 1 -- 172.21.15.31:0/2731189451 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8670155300 con 0x7f86700ae290 2023-10-06T05:17:09.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.647+0000 7f8677dff700 1 -- 172.21.15.31:0/2731189451 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8670155800 con 0x7f86700ae290 2023-10-06T05:17:09.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.647+0000 7f1e02ffd700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0067080 0x7f1df0069530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1df8006fd0 tx=0x7f1df8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:09.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.652+0000 7f8665ffb700 1 -- 172.21.15.31:0/2731189451 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f866801a040 con 0x7f86700ae290 2023-10-06T05:17:09.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.654+0000 7f8665ffb700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8654067080 0x7f8654069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:09.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.654+0000 7f8665ffb700 1 -- 172.21.15.31:0/2731189451 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8668086640 con 0x7f86700ae290 2023-10-06T05:17:09.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.654+0000 7f8665ffb700 1 -- 172.21.15.31:0/2731189451 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8668086ad0 con 0x7f86700ae290 2023-10-06T05:17:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.655+0000 7f8675b9b700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8654067080 0x7f8654069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:09.656+0000 7f8675b9b700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8654067080 0x7f8654069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f866c007e20 tx=0x7f866c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:10.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.062+0000 7fc0749ff700 1 -- 172.21.15.31:0/1299452751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc054005560 con 0x7fc07015d840 2023-10-06T05:17:10.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.064+0000 7f874bfff700 1 -- 172.21.15.31:0/257530331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8738005830 con 0x7f8764069560 2023-10-06T05:17:10.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.065+0000 7f87527fc700 1 -- 172.21.15.31:0/257530331 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f87580535a0 con 0x7f8764069560 2023-10-06T05:17:10.067 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2023-10-06T05:17:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.067+0000 7fc05e7fc700 1 -- 172.21.15.31:0/1299452751 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc068058020 con 0x7fc07015d840 2023-10-06T05:17:10.074 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-10-06T05:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.071+0000 7f874bfff700 1 -- 172.21.15.31:0/257530331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f874c067540 msgr2=0x7f874c0699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.071+0000 7f874bfff700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f874c067540 0x7f874c0699f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f87641581c0 tx=0x7f875c009040 comp rx=0 tx=0).stop 2023-10-06T05:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.071+0000 7f874bfff700 1 -- 172.21.15.31:0/257530331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8764069560 msgr2=0x7f8764154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.071+0000 7f874bfff700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8764069560 0x7f8764154e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f87580030c0 tx=0x7f8758016000 comp rx=0 tx=0).stop 2023-10-06T05:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.072+0000 7f874bfff700 1 -- 172.21.15.31:0/257530331 shutdown_connections 2023-10-06T05:17:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.072+0000 7f874bfff700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87640ccbc0 0x7f8764171460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.072+0000 7f874bfff700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f874c067540 0x7f874c0699f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.072+0000 7f874bfff700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8764069560 0x7f8764154e40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.072+0000 7f874bfff700 1 --2- 172.21.15.31:0/257530331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f876406eef0 0x7f8764170f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.072+0000 7f874bfff700 1 -- 172.21.15.31:0/257530331 >> 172.21.15.31:0/257530331 conn(0x7f87640cc0b0 msgr2=0x7f876414d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7fc0749ff700 1 -- 172.21.15.31:0/1299452751 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc050067080 msgr2=0x7fc050069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7fc0749ff700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc050067080 0x7fc050069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc0600099b0 tx=0x7fc060016000 comp rx=0 tx=0).stop 2023-10-06T05:17:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7fc0749ff700 1 -- 172.21.15.31:0/1299452751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07015d840 msgr2=0x7fc070142ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7fc0749ff700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07015d840 0x7fc070142ee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc06800bbb0 tx=0x7fc068009840 comp rx=0 tx=0).stop 2023-10-06T05:17:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7fc0749ff700 1 -- 172.21.15.31:0/1299452751 shutdown_connections 2023-10-06T05:17:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7fc0749ff700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc050067080 0x7fc050069530 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc0600099b0 tx=0x7fc060016000 comp rx=0 tx=0).stop 2023-10-06T05:17:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7fc0749ff700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07015dc10 0x7fc0701604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7fc0749ff700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07006ace0 0x7fc07013e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7fc0749ff700 1 --2- 172.21.15.31:0/1299452751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07015d840 0x7fc070142ee0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc06800bbb0 tx=0x7fc068009840 comp rx=0 tx=0).stop 2023-10-06T05:17:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7fc0749ff700 1 -- 172.21.15.31:0/1299452751 >> 172.21.15.31:0/1299452751 conn(0x7fc0700cc0b0 msgr2=0x7fc0700c26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7f874bfff700 1 -- 172.21.15.31:0/257530331 shutdown_connections 2023-10-06T05:17:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.073+0000 7f874bfff700 1 -- 172.21.15.31:0/257530331 wait complete. 2023-10-06T05:17:10.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.074+0000 7fc0749ff700 1 -- 172.21.15.31:0/1299452751 shutdown_connections 2023-10-06T05:17:10.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.075+0000 7fc0749ff700 1 -- 172.21.15.31:0/1299452751 wait complete. 2023-10-06T05:17:10.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.076+0000 7f062efff700 1 -- 172.21.15.31:0/1032396887 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0628161320 con 0x7f06280c6e70 2023-10-06T05:17:10.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.076+0000 7f0624ff9700 1 -- 172.21.15.31:0/1032396887 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0618055640 con 0x7f06280c6e70 2023-10-06T05:17:10.078 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.084+0000 7f062efff700 1 -- 172.21.15.31:0/1032396887 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0614067080 msgr2=0x7f0614069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.084+0000 7f062efff700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0614067080 0x7f0614069530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f062000acf0 tx=0x7f0620009350 comp rx=0 tx=0).stop 2023-10-06T05:17:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.084+0000 7f062efff700 1 -- 172.21.15.31:0/1032396887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06280c6e70 msgr2=0x7f06280c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.084+0000 7f062efff700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06280c6e70 0x7f06280c4b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f061800b5e0 tx=0x7f06180090f0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.090+0000 7f062efff700 1 -- 172.21.15.31:0/1032396887 shutdown_connections 2023-10-06T05:17:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.090+0000 7f062efff700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0628147380 0x7f06281570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.090+0000 7f062efff700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0614067080 0x7f0614069530 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f062000acf0 tx=0x7f0620009350 comp rx=0 tx=0).stop 2023-10-06T05:17:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.090+0000 7f062efff700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0628069cb0 0x7f0628146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.090+0000 7f062efff700 1 --2- 172.21.15.31:0/1032396887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06280c6e70 0x7f06280c4b40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.090+0000 7f062efff700 1 -- 172.21.15.31:0/1032396887 >> 172.21.15.31:0/1032396887 conn(0x7f06280cc0b0 msgr2=0x7f06280cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:10.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.091+0000 7f26afbff700 1 -- 172.21.15.31:0/3582465266 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2688005560 con 0x7f26a806ace0 2023-10-06T05:17:10.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.092+0000 7f062efff700 1 -- 172.21.15.31:0/1032396887 shutdown_connections 2023-10-06T05:17:10.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.092+0000 7f062efff700 1 -- 172.21.15.31:0/1032396887 wait complete. 2023-10-06T05:17:10.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.093+0000 7fe31d1ff700 1 -- 172.21.15.31:0/2469627349 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe31804c380 con 0x7fe31806f140 2023-10-06T05:17:10.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.099+0000 7fe2feffd700 1 -- 172.21.15.31:0/2469627349 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe30c0555d0 con 0x7fe31806f140 2023-10-06T05:17:10.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.099+0000 7f8677dff700 1 -- 172.21.15.31:0/2731189451 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f867004c380 con 0x7f86700ae290 2023-10-06T05:17:10.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.099+0000 7f1e09bff700 1 -- 172.21.15.31:0/2786822076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1e040c1430 con 0x7f1e04069cb0 2023-10-06T05:17:10.102 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:10.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.100+0000 7f1deb7fe700 1 -- 172.21.15.31:0/2786822076 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1df40069c0 con 0x7f1e04069cb0 2023-10-06T05:17:10.102 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:10.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.101+0000 7f8665ffb700 1 -- 172.21.15.31:0/2731189451 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8668016090 con 0x7f86700ae290 2023-10-06T05:17:10.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.101+0000 7f269dffb700 1 -- 172.21.15.31:0/3582465266 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f26a0054900 con 0x7f26a806ace0 2023-10-06T05:17:10.103 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:10.104 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:10.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.101+0000 7f99197ff700 1 -- 172.21.15.31:0/3534288603 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f98f4005830 con 0x7f99140c6ca0 2023-10-06T05:17:10.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.103+0000 7f98faffd700 1 -- 172.21.15.31:0/3534288603 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9908056e40 con 0x7f99140c6ca0 2023-10-06T05:17:10.106 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:10.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.104+0000 7f8677dff700 1 -- 172.21.15.31:0/2731189451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8654067080 msgr2=0x7f8654069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.104+0000 7f8677dff700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8654067080 0x7f8654069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f866c007e20 tx=0x7f866c009040 comp rx=0 tx=0).stop 2023-10-06T05:17:10.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.104+0000 7f8677dff700 1 -- 172.21.15.31:0/2731189451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86700ae290 msgr2=0x7f86700c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.104+0000 7f8677dff700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86700ae290 0x7f86700c52b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86700aebf0 tx=0x7f8668015000 comp rx=0 tx=0).stop 2023-10-06T05:17:10.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.104+0000 7f1e09bff700 1 -- 172.21.15.31:0/2786822076 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0067080 msgr2=0x7f1df0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.104+0000 7f1e09bff700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0067080 0x7f1df0069530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1df8006fd0 tx=0x7f1df8008040 comp rx=0 tx=0).stop 2023-10-06T05:17:10.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.104+0000 7f1e09bff700 1 -- 172.21.15.31:0/2786822076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04069cb0 msgr2=0x7f1e04145130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.104+0000 7f1e09bff700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04069cb0 0x7f1e04145130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1df4002000 tx=0x7f1df4015000 comp rx=0 tx=0).stop 2023-10-06T05:17:10.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f8677dff700 1 -- 172.21.15.31:0/2731189451 shutdown_connections 2023-10-06T05:17:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f8677dff700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86700ae290 0x7f86700c52b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f8677dff700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0052e0 0x7f86701544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f8677dff700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8654067080 0x7f8654069530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f8677dff700 1 --2- 172.21.15.31:0/2731189451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86701779a0 0x7f86700ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f8677dff700 1 -- 172.21.15.31:0/2731189451 >> 172.21.15.31:0/2731189451 conn(0x7f86700cc0b0 msgr2=0x7f86700b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:10.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.3 2023-10-06T05:17:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f1e09bff700 1 -- 172.21.15.31:0/2786822076 shutdown_connections 2023-10-06T05:17:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f1e09bff700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0067080 0x7f1df0069530 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f1e09bff700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e04147380 0x7f1e040c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f1e09bff700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e04069cb0 0x7f1e04145130 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f1e09bff700 1 --2- 172.21.15.31:0/2786822076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e040c6ca0 0x7f1e040adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f1e09bff700 1 -- 172.21.15.31:0/2786822076 >> 172.21.15.31:0/2786822076 conn(0x7f1e040cc0b0 msgr2=0x7f1e04162ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 -- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe304067080 msgr2=0x7fe304069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe304067080 0x7fe304069530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe318153b70 tx=0x7fe308008040 comp rx=0 tx=0).stop 2023-10-06T05:17:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 -- 172.21.15.31:0/2469627349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31806f140 msgr2=0x7fe3180ac750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31806f140 0x7fe3180ac750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00b810 tx=0x7fe30c0091d0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f1e09bff700 1 -- 172.21.15.31:0/2786822076 shutdown_connections 2023-10-06T05:17:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 -- 172.21.15.31:0/2469627349 shutdown_connections 2023-10-06T05:17:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3180ccbc0 0x7fe3180c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe304067080 0x7fe304069530 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe318153b70 tx=0x7fe308008040 comp rx=0 tx=0).stop 2023-10-06T05:17:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe318069560 0x7fe31814f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 --2- 172.21.15.31:0/2469627349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31806f140 0x7fe3180ac750 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 -- 172.21.15.31:0/2469627349 >> 172.21.15.31:0/2469627349 conn(0x7fe3180cc0b0 msgr2=0x7fe31806d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f1e09bff700 1 -- 172.21.15.31:0/2786822076 wait complete. 2023-10-06T05:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f8677dff700 1 -- 172.21.15.31:0/2731189451 shutdown_connections 2023-10-06T05:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7f26afbff700 1 -- 172.21.15.31:0/3582465266 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f268c066fb0 msgr2=0x7f268c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7f26afbff700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f268c066fb0 0x7f268c069460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2698006fd0 tx=0x7f2698008040 comp rx=0 tx=0).stop 2023-10-06T05:17:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7fe31d1ff700 1 -- 172.21.15.31:0/2469627349 shutdown_connections 2023-10-06T05:17:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.105+0000 7f8677dff700 1 -- 172.21.15.31:0/2731189451 wait complete. 2023-10-06T05:17:10.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7f26afbff700 1 -- 172.21.15.31:0/3582465266 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26a806ace0 msgr2=0x7f26a815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7f26afbff700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26a806ace0 0x7f26a815c950 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f26a000bc30 tx=0x7f26a0009840 comp rx=0 tx=0).stop 2023-10-06T05:17:10.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7fe31d1ff700 1 -- 172.21.15.31:0/2469627349 wait complete. 2023-10-06T05:17:10.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.2 2023-10-06T05:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7f26afbff700 1 -- 172.21.15.31:0/3582465266 shutdown_connections 2023-10-06T05:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7f26afbff700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26a815dc10 0x7f26a814ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7f26afbff700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26a806ace0 0x7f26a815c950 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7f26afbff700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f268c066fb0 0x7f268c069460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7f26afbff700 1 --2- 172.21.15.31:0/3582465266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a815d840 0x7f26a80b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.106+0000 7f26afbff700 1 -- 172.21.15.31:0/3582465266 >> 172.21.15.31:0/3582465266 conn(0x7f26a80cc0b0 msgr2=0x7f26a80c9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.107+0000 7f26afbff700 1 -- 172.21.15.31:0/3582465266 shutdown_connections 2023-10-06T05:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 -- 172.21.15.31:0/3534288603 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9900067000 msgr2=0x7f99000694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9900067000 0x7f99000694b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9904006ce0 tx=0x7f9904008040 comp rx=0 tx=0).stop 2023-10-06T05:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 -- 172.21.15.31:0/3534288603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99140c6ca0 msgr2=0x7f991406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99140c6ca0 0x7f991406ed40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f99080030c0 tx=0x7f9908016000 comp rx=0 tx=0).stop 2023-10-06T05:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 -- 172.21.15.31:0/3534288603 shutdown_connections 2023-10-06T05:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9900067000 0x7f99000694b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9914147380 0x7f991406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9914069cb0 0x7f9914160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 --2- 172.21.15.31:0/3534288603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99140c6ca0 0x7f991406ed40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 -- 172.21.15.31:0/3534288603 >> 172.21.15.31:0/3534288603 conn(0x7f99140cc0b0 msgr2=0x7f9914145b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:10.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 -- 172.21.15.31:0/3534288603 shutdown_connections 2023-10-06T05:17:10.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.108+0000 7f99197ff700 1 -- 172.21.15.31:0/3534288603 wait complete. 2023-10-06T05:17:10.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:10.107+0000 7f26afbff700 1 -- 172.21.15.31:0/3582465266 wait complete. 2023-10-06T05:17:10.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-10-06T05:17:10.121 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:10.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-10-06T05:17:10.133 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:10.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-06T05:17:10.142 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:10.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-06T05:17:10.143 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:10.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-10-06T05:17:10.144 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:10.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-10-06T05:17:10.145 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:11.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T05:17:11.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T05:17:11.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.318+0000 7f152cfff700 1 Processor -- start 2023-10-06T05:17:11.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.318+0000 7f152cfff700 1 -- start start 2023-10-06T05:17:11.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.320+0000 7f152cfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f152815d3f0 0x7f15280aab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.320+0000 7f152cfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15280c77f0 con 0x7f152815d7c0 2023-10-06T05:17:11.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.320+0000 7f152cfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15280a9ab0 con 0x7f1528159040 2023-10-06T05:17:11.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.320+0000 7f152cfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15280a98a0 con 0x7f152815d3f0 2023-10-06T05:17:11.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.320+0000 7f152659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f152815d3f0 0x7f15280aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.320+0000 7f152659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f152815d3f0 0x7f15280aab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34440/0 (socket says 172.21.15.31:34440) 2023-10-06T05:17:11.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.320+0000 7f152659c700 1 -- 172.21.15.31:0/3023086909 learned_addr learned my addr 172.21.15.31:0/3023086909 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:11.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.320+0000 7f1525d9b700 1 --1- 172.21.15.31:0/3023086909 >> v1:172.21.15.31:6789/0 conn(0x7f152815d7c0 0x7f15280c70e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44790/0 (socket says 172.21.15.31:44790) 2023-10-06T05:17:11.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.321+0000 7f152659c700 1 -- 172.21.15.31:0/3023086909 >> v1:172.21.15.62:6789/0 conn(0x7f1528159040 legacy=0x7f1528159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:11.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.321+0000 7f152659c700 1 -- 172.21.15.31:0/3023086909 >> v1:172.21.15.31:6789/0 conn(0x7f152815d7c0 legacy=0x7f15280c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.321+0000 7f152659c700 1 -- 172.21.15.31:0/3023086909 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15280a9d90 con 0x7f152815d3f0 2023-10-06T05:17:11.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.321+0000 7f152659c700 1 --2- 172.21.15.31:0/3023086909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d3f0 0x7f15280aab50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1518009990 tx=0x7f1518009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a98531642a92ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:11.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.321+0000 7f152559a700 1 -- 172.21.15.31:0/3023086909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518005c10 con 0x7f152815d3f0 2023-10-06T05:17:11.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.321+0000 7f152559a700 1 -- 172.21.15.31:0/3023086909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1518005d70 con 0x7f152815d3f0 2023-10-06T05:17:11.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.321+0000 7f152559a700 1 -- 172.21.15.31:0/3023086909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151800b840 con 0x7f152815d3f0 2023-10-06T05:17:11.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.322+0000 7f152cfff700 1 -- 172.21.15.31:0/3023086909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d3f0 msgr2=0x7f15280aab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.322+0000 7f152cfff700 1 --2- 172.21.15.31:0/3023086909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d3f0 0x7f15280aab50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1518009990 tx=0x7f1518009220 comp rx=0 tx=0).stop 2023-10-06T05:17:11.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.322+0000 7f152cfff700 1 -- 172.21.15.31:0/3023086909 shutdown_connections 2023-10-06T05:17:11.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.322+0000 7f152cfff700 1 --2- 172.21.15.31:0/3023086909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d3f0 0x7f15280aab50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.322+0000 7f152cfff700 1 -- 172.21.15.31:0/3023086909 >> 172.21.15.31:0/3023086909 conn(0x7f15280cc0b0 msgr2=0x7f15280cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:11.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.322+0000 7f152cfff700 1 -- 172.21.15.31:0/3023086909 shutdown_connections 2023-10-06T05:17:11.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.322+0000 7f152cfff700 1 -- 172.21.15.31:0/3023086909 wait complete. 2023-10-06T05:17:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.323+0000 7f152cfff700 1 Processor -- start 2023-10-06T05:17:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.324+0000 7f152cfff700 1 -- start start 2023-10-06T05:17:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.324+0000 7f152cfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1528159040 0x7f152815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.324+0000 7f152cfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f152815d3f0 0x7f152816d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.324+0000 7f152cfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d7c0 0x7f152806e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f152cfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152816a080 con 0x7f152815d7c0 2023-10-06T05:17:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f152cfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152816d870 con 0x7f152815d3f0 2023-10-06T05:17:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f152cfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1528169d30 con 0x7f1528159040 2023-10-06T05:17:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f1526d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d7c0 0x7f152806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f1526d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d7c0 0x7f152806e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34456/0 (socket says 172.21.15.31:34456) 2023-10-06T05:17:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f1526d9d700 1 -- 172.21.15.31:0/1356600067 learned_addr learned my addr 172.21.15.31:0/1356600067 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f1525d9b700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f152815d3f0 0x7f152816d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f152659c700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1528159040 0x7f152815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f1526d9d700 1 -- 172.21.15.31:0/1356600067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1528159040 msgr2=0x7f152815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f1526d9d700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1528159040 0x7f152815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f1526d9d700 1 -- 172.21.15.31:0/1356600067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f152815d3f0 msgr2=0x7f152816d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f1526d9d700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f152815d3f0 0x7f152816d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f1526d9d700 1 -- 172.21.15.31:0/1356600067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518009480 con 0x7f152815d7c0 2023-10-06T05:17:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.325+0000 7f1526d9d700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d7c0 0x7f152806e100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f152000bc30 tx=0x7f1520009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.326+0000 7f15167fc700 1 -- 172.21.15.31:0/1356600067 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152000c9b0 con 0x7f152815d7c0 2023-10-06T05:17:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.326+0000 7f15167fc700 1 -- 172.21.15.31:0/1356600067 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152000cb50 con 0x7f152815d7c0 2023-10-06T05:17:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.326+0000 7f152cfff700 1 -- 172.21.15.31:0/1356600067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152816daf0 con 0x7f152815d7c0 2023-10-06T05:17:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.326+0000 7f15167fc700 1 -- 172.21.15.31:0/1356600067 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520014410 con 0x7f152815d7c0 2023-10-06T05:17:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.326+0000 7f152cfff700 1 -- 172.21.15.31:0/1356600067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152816e0b0 con 0x7f152815d7c0 2023-10-06T05:17:11.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.327+0000 7f152cfff700 1 -- 172.21.15.31:0/1356600067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152816a1f0 con 0x7f152815d7c0 2023-10-06T05:17:11.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.328+0000 7f15167fc700 1 -- 172.21.15.31:0/1356600067 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1520014800 con 0x7f152815d7c0 2023-10-06T05:17:11.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.329+0000 7f15167fc700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1508066f30 0x7f15080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.329+0000 7f15167fc700 1 -- 172.21.15.31:0/1356600067 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f15200847e0 con 0x7f152815d7c0 2023-10-06T05:17:11.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.329+0000 7f152659c700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1508066f30 0x7f15080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.330+0000 7f152659c700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1508066f30 0x7f15080693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1518009260 tx=0x7f1518015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.338+0000 7f15167fc700 1 -- 172.21.15.31:0/1356600067 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f152004c5a0 con 0x7f152815d7c0 2023-10-06T05:17:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.363+0000 7f9189dff700 1 Processor -- start 2023-10-06T05:17:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.363+0000 7f9189dff700 1 -- start start 2023-10-06T05:17:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.364+0000 7f9189dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9184140a80 0x7f91840cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.364+0000 7f9189dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 0x7f91840cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.364+0000 7f9189dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91840cbec0 con 0x7f91840b10b0 2023-10-06T05:17:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.364+0000 7f9189dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91840cca40 con 0x7f9184140a80 2023-10-06T05:17:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.364+0000 7f9189dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91840cc110 con 0x7f91840c9040 2023-10-06T05:17:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.364+0000 7f91837fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9184140a80 0x7f91840cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.364+0000 7f91837fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9184140a80 0x7f91840cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34458/0 (socket says 172.21.15.31:34458) 2023-10-06T05:17:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.364+0000 7f91837fe700 1 -- 172.21.15.31:0/3335445881 learned_addr learned my addr 172.21.15.31:0/3335445881 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.364+0000 7f9182ffd700 1 --1- 172.21.15.31:0/3335445881 >> v1:172.21.15.31:6789/0 conn(0x7f91840b10b0 0x7f91840c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44802/0 (socket says 172.21.15.31:44802) 2023-10-06T05:17:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f91837fe700 1 -- 172.21.15.31:0/3335445881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 msgr2=0x7f91840cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f9183fff700 1 --2- 172.21.15.31:0/3335445881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 0x7f91840cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f91837fe700 1 --2- 172.21.15.31:0/3335445881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 0x7f91840cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f91837fe700 1 -- 172.21.15.31:0/3335445881 >> v1:172.21.15.31:6789/0 conn(0x7f91840b10b0 legacy=0x7f91840c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f91837fe700 1 -- 172.21.15.31:0/3335445881 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91840cc3f0 con 0x7f9184140a80 2023-10-06T05:17:11.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f9183fff700 1 --2- 172.21.15.31:0/3335445881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 0x7f91840cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f91827fc700 1 -- 172.21.15.31:0/3335445881 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384053572 0 0) 0x7f91840cbec0 con 0x7f91840b10b0 2023-10-06T05:17:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f91837fe700 1 --2- 172.21.15.31:0/3335445881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 0x7f91840cb120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f91740099b0 tx=0x7f9174009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5701f4d52707c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f91827fc700 1 -- 172.21.15.31:0/3335445881 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400b860 con 0x7f9184140a80 2023-10-06T05:17:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f91827fc700 1 -- 172.21.15.31:0/3335445881 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917400b9c0 con 0x7f9184140a80 2023-10-06T05:17:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.365+0000 7f91827fc700 1 -- 172.21.15.31:0/3335445881 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400bd60 con 0x7f9184140a80 2023-10-06T05:17:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.366+0000 7f9189dff700 1 -- 172.21.15.31:0/3335445881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 msgr2=0x7f91840cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.366+0000 7f9189dff700 1 --2- 172.21.15.31:0/3335445881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 0x7f91840cb120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f91740099b0 tx=0x7f9174009250 comp rx=0 tx=0).stop 2023-10-06T05:17:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.366+0000 7f9189dff700 1 -- 172.21.15.31:0/3335445881 shutdown_connections 2023-10-06T05:17:11.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.366+0000 7f9189dff700 1 --2- 172.21.15.31:0/3335445881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 0x7f91840cb120 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.366+0000 7f9189dff700 1 --2- 172.21.15.31:0/3335445881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 0x7f91840cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.366+0000 7f9189dff700 1 -- 172.21.15.31:0/3335445881 >> 172.21.15.31:0/3335445881 conn(0x7f918406b620 msgr2=0x7f918406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.366+0000 7f9189dff700 1 -- 172.21.15.31:0/3335445881 shutdown_connections 2023-10-06T05:17:11.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.366+0000 7f9189dff700 1 -- 172.21.15.31:0/3335445881 wait complete. 2023-10-06T05:17:11.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.368+0000 7f9189dff700 1 Processor -- start 2023-10-06T05:17:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.368+0000 7f9189dff700 1 -- start start 2023-10-06T05:17:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f9189dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91840b10b0 0x7f91840c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f9189dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 0x7f91840c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f9189dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 0x7f9184159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f9189dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91840c61d0 con 0x7f9184140a80 2023-10-06T05:17:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f9189dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91840c5550 con 0x7f91840b10b0 2023-10-06T05:17:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f9189dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91840c5830 con 0x7f91840c9040 2023-10-06T05:17:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f9183fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 0x7f9184159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f9183fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 0x7f9184159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34472/0 (socket says 172.21.15.31:34472) 2023-10-06T05:17:11.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f9183fff700 1 -- 172.21.15.31:0/1143017248 learned_addr learned my addr 172.21.15.31:0/1143017248 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:11.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f91837fe700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91840b10b0 0x7f91840c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.369+0000 7f9182ffd700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 0x7f91840c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f9183fff700 1 -- 172.21.15.31:0/1143017248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 msgr2=0x7f91840c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f9183fff700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 0x7f91840c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f9183fff700 1 -- 172.21.15.31:0/1143017248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91840b10b0 msgr2=0x7f91840c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f9183fff700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91840b10b0 0x7f91840c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f9183fff700 1 -- 172.21.15.31:0/1143017248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91740094f0 con 0x7f9184140a80 2023-10-06T05:17:11.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f91837fe700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91840b10b0 0x7f91840c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:11.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f9183fff700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 0x7f9184159470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f917c00a380 tx=0x7f917c007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:11.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f916b7fe700 1 -- 172.21.15.31:0/1143017248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c015440 con 0x7f9184140a80 2023-10-06T05:17:11.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f916b7fe700 1 -- 172.21.15.31:0/1143017248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c0155a0 con 0x7f9184140a80 2023-10-06T05:17:11.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f916b7fe700 1 -- 172.21.15.31:0/1143017248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c015960 con 0x7f9184140a80 2023-10-06T05:17:11.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f9189dff700 1 -- 172.21.15.31:0/1143017248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91840c5b10 con 0x7f9184140a80 2023-10-06T05:17:11.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.370+0000 7f9189dff700 1 -- 172.21.15.31:0/1143017248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91841599b0 con 0x7f9184140a80 2023-10-06T05:17:11.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.371+0000 7f9189dff700 1 -- 172.21.15.31:0/1143017248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91840ca970 con 0x7f9184140a80 2023-10-06T05:17:11.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.373+0000 7f916b7fe700 1 -- 172.21.15.31:0/1143017248 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f917c00d440 con 0x7f9184140a80 2023-10-06T05:17:11.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.374+0000 7f916b7fe700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9170066f30 0x7f91700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.374+0000 7f916b7fe700 1 -- 172.21.15.31:0/1143017248 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f917c085950 con 0x7f9184140a80 2023-10-06T05:17:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.374+0000 7f91837fe700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9170066f30 0x7f91700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.375+0000 7f91837fe700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9170066f30 0x7f91700693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91740099b0 tx=0x7f9174017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:11.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.377+0000 7f916b7fe700 1 -- 172.21.15.31:0/1143017248 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f917c052a40 con 0x7f9184140a80 2023-10-06T05:17:11.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.543+0000 7f152cfff700 1 -- 172.21.15.31:0/1356600067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f152804c380 con 0x7f152815d7c0 2023-10-06T05:17:11.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.544+0000 7f15167fc700 1 -- 172.21.15.31:0/1356600067 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1520051850 con 0x7f152815d7c0 2023-10-06T05:17:11.546 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-10-06T05:17:11.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.546+0000 7f152cfff700 1 -- 172.21.15.31:0/1356600067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1508066f30 msgr2=0x7f15080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.546+0000 7f152cfff700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1508066f30 0x7f15080693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1518009260 tx=0x7f1518015000 comp rx=0 tx=0).stop 2023-10-06T05:17:11.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.546+0000 7f152cfff700 1 -- 172.21.15.31:0/1356600067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d7c0 msgr2=0x7f152806e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.546+0000 7f152cfff700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d7c0 0x7f152806e100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f152000bc30 tx=0x7f1520009820 comp rx=0 tx=0).stop 2023-10-06T05:17:11.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.547+0000 7f152cfff700 1 -- 172.21.15.31:0/1356600067 shutdown_connections 2023-10-06T05:17:11.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.547+0000 7f152cfff700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1508066f30 0x7f15080693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.547+0000 7f152cfff700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152815d7c0 0x7f152806e100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.547+0000 7f152cfff700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1528159040 0x7f152815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.547+0000 7f152cfff700 1 --2- 172.21.15.31:0/1356600067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f152815d3f0 0x7f152816d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.547+0000 7f152cfff700 1 -- 172.21.15.31:0/1356600067 >> 172.21.15.31:0/1356600067 conn(0x7f15280cc0b0 msgr2=0x7f152806a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:11.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.547+0000 7f152cfff700 1 -- 172.21.15.31:0/1356600067 shutdown_connections 2023-10-06T05:17:11.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.547+0000 7f152cfff700 1 -- 172.21.15.31:0/1356600067 wait complete. 2023-10-06T05:17:11.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-10-06T05:17:11.566 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:11.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.581+0000 7f9189dff700 1 -- 172.21.15.31:0/1143017248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f91840c6310 con 0x7f9184140a80 2023-10-06T05:17:11.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.581+0000 7f916b7fe700 1 -- 172.21.15.31:0/1143017248 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f917c01a020 con 0x7f9184140a80 2023-10-06T05:17:11.583 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-10-06T05:17:11.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 -- 172.21.15.31:0/1143017248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9170066f30 msgr2=0x7f91700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9170066f30 0x7f91700693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91740099b0 tx=0x7f9174017000 comp rx=0 tx=0).stop 2023-10-06T05:17:11.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 -- 172.21.15.31:0/1143017248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 msgr2=0x7f9184159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 0x7f9184159470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f917c00a380 tx=0x7f917c007900 comp rx=0 tx=0).stop 2023-10-06T05:17:11.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 -- 172.21.15.31:0/1143017248 shutdown_connections 2023-10-06T05:17:11.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9170066f30 0x7f91700693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9184140a80 0x7f9184159470 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91840b10b0 0x7f91840c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 --2- 172.21.15.31:0/1143017248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91840c9040 0x7f91840c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 -- 172.21.15.31:0/1143017248 >> 172.21.15.31:0/1143017248 conn(0x7f918406b620 msgr2=0x7f918413ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 -- 172.21.15.31:0/1143017248 shutdown_connections 2023-10-06T05:17:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.584+0000 7f9189dff700 1 -- 172.21.15.31:0/1143017248 wait complete. 2023-10-06T05:17:11.603 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-10-06T05:17:11.603 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:11.603 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T05:17:11.604 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:17:11.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.837+0000 7fbda6fff700 1 Processor -- start 2023-10-06T05:17:11.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.837+0000 7fbda6fff700 1 -- start start 2023-10-06T05:17:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.838+0000 7fbda6fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbda0068d80 0x7fbda0159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.838+0000 7fbda6fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00cc0d0 0x7fbda00c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.838+0000 7fbda6fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda00c95b0 con 0x7fbda01595a0 2023-10-06T05:17:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.838+0000 7fbda6fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda00c9890 con 0x7fbda0068d80 2023-10-06T05:17:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.838+0000 7fbda6fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00c9b70 con 0x7fbda00cc0d0 2023-10-06T05:17:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.838+0000 7fbda4d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbda01595a0 0x7fbda0068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44816/0 (socket says 172.21.15.31:44816) 2023-10-06T05:17:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbda4d9b700 1 -- 172.21.15.31:0/4259484032 learned_addr learned my addr 172.21.15.31:0/4259484032 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9ffff700 1 --2- 172.21.15.31:0/4259484032 >> v2:172.21.15.62:3300/0 conn(0x7fbda0068d80 0x7fbda0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611780009 0 0) 0x7fbda00c95b0 con 0x7fbda01595a0 2023-10-06T05:17:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbda559c700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00cc0d0 0x7fbda00c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd90003570 con 0x7fbda01595a0 2023-10-06T05:17:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 877687108 0 0) 0x7fbd90003570 con 0x7fbda01595a0 2023-10-06T05:17:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00cc0d0 msgr2=0x7fbda00c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00cc0d0 0x7fbda00c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 >> v2:172.21.15.62:3300/0 conn(0x7fbda0068d80 msgr2=0x7fbda0159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:11.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 --2- 172.21.15.31:0/4259484032 >> v2:172.21.15.62:3300/0 conn(0x7fbda0068d80 0x7fbda0159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda00ca130 con 0x7fbda01595a0 2023-10-06T05:17:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fbd94001470 con 0x7fbda01595a0 2023-10-06T05:17:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 >> v1:172.21.15.31:6789/0 conn(0x7fbda01595a0 legacy=0x7fbda0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90003a60 0x7fbd90003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00cc0d0 0x7fbd90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd90004d50 0x7fbd900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00c9890 con 0x7fbd90004d50 2023-10-06T05:17:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00c9b70 con 0x7fbd90003a60 2023-10-06T05:17:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.839+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9000ba60 con 0x7fbda00cc0d0 2023-10-06T05:17:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.840+0000 7fbda559c700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd90004d50 0x7fbd900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.840+0000 7fbd9ffff700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00cc0d0 0x7fbd90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.840+0000 7fbda4d9b700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90003a60 0x7fbd90003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.840+0000 7fbda559c700 1 -- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00cc0d0 msgr2=0x7fbd90004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.840+0000 7fbda559c700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00cc0d0 0x7fbd90004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.840+0000 7fbda559c700 1 -- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90003a60 msgr2=0x7fbd90003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.840+0000 7fbda559c700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90003a60 0x7fbd90003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.840+0000 7fbda559c700 1 -- 172.21.15.31:0/4259484032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90003570 con 0x7fbd90004d50 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.840+0000 7fbda4d9b700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90003a60 0x7fbd90003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.840+0000 7fbda559c700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd90004d50 0x7fbd900051c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800e210 tx=0x7fbd9800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f7d52a15bbdb31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.841+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800ea60 con 0x7fbd90004d50 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.841+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9800f040 con 0x7fbd90004d50 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.841+0000 7fbd9f7fe700 1 -- 172.21.15.31:0/4259484032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800ee10 con 0x7fbd90004d50 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.841+0000 7fbda6fff700 1 -- 172.21.15.31:0/4259484032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd90004d50 msgr2=0x7fbd900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.841+0000 7fbda6fff700 1 --2- 172.21.15.31:0/4259484032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd90004d50 0x7fbd900051c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800e210 tx=0x7fbd9800b040 comp rx=0 tx=0).stop 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.841+0000 7fbda559c700 1 -- 172.21.15.31:0/4259484032 reap_dead start 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.841+0000 7fbda6fff700 1 -- 172.21.15.31:0/4259484032 shutdown_connections 2023-10-06T05:17:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.841+0000 7fbda6fff700 1 -- 172.21.15.31:0/4259484032 >> 172.21.15.31:0/4259484032 conn(0x7fbda006b620 msgr2=0x7fbda006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.841+0000 7fbda6fff700 1 -- 172.21.15.31:0/4259484032 shutdown_connections 2023-10-06T05:17:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.841+0000 7fbda6fff700 1 -- 172.21.15.31:0/4259484032 wait complete. 2023-10-06T05:17:11.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.843+0000 7fbda6fff700 1 Processor -- start 2023-10-06T05:17:11.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.843+0000 7fbda6fff700 1 -- start start 2023-10-06T05:17:11.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda6fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90004d50 0x7fbda0154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda6fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00b2a20 0x7fbda016a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda6fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda0157a50 0x7fbda0157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda6fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda0158430 con 0x7fbda0157a50 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda6fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda015d9a0 con 0x7fbd90004d50 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda6fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda016a8b0 con 0x7fbda00b2a20 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda559c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda0157a50 0x7fbda0157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda559c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda0157a50 0x7fbda0157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34498/0 (socket says 172.21.15.31:34498) 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda559c700 1 -- 172.21.15.31:0/2827581746 learned_addr learned my addr 172.21.15.31:0/2827581746 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda4d9b700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90004d50 0x7fbda0154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbd9ffff700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00b2a20 0x7fbda016a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda559c700 1 -- 172.21.15.31:0/2827581746 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00b2a20 msgr2=0x7fbda016a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda559c700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00b2a20 0x7fbda016a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda559c700 1 -- 172.21.15.31:0/2827581746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90004d50 msgr2=0x7fbda0154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda559c700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90004d50 0x7fbda0154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbda559c700 1 -- 172.21.15.31:0/2827581746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda00b2fe0 con 0x7fbda0157a50 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.844+0000 7fbd9ffff700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00b2a20 0x7fbda016a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.845+0000 7fbda559c700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda0157a50 0x7fbda0157ef0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800e800 tx=0x7fbd9801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.845+0000 7fbd9cff9700 1 -- 172.21.15.31:0/2827581746 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800eee0 con 0x7fbda0157a50 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.845+0000 7fbd9cff9700 1 -- 172.21.15.31:0/2827581746 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9801d630 con 0x7fbda0157a50 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.845+0000 7fbd9cff9700 1 -- 172.21.15.31:0/2827581746 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9801d9f0 con 0x7fbda0157a50 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.845+0000 7fbda6fff700 1 -- 172.21.15.31:0/2827581746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda015d070 con 0x7fbda0157a50 2023-10-06T05:17:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.845+0000 7fbda6fff700 1 -- 172.21.15.31:0/2827581746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda015d630 con 0x7fbda0157a50 2023-10-06T05:17:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.846+0000 7fbda6fff700 1 -- 172.21.15.31:0/2827581746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda015db30 con 0x7fbda0157a50 2023-10-06T05:17:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.847+0000 7fbd9cff9700 1 -- 172.21.15.31:0/2827581746 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd98003040 con 0x7fbda0157a50 2023-10-06T05:17:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.848+0000 7fbd9cff9700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbd84066f30 0x7fbd840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.848+0000 7fbd9cff9700 1 -- 172.21.15.31:0/2827581746 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbd98085bd0 con 0x7fbda0157a50 2023-10-06T05:17:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.848+0000 7fbda4d9b700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbd84066f30 0x7fbd840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.848+0000 7fbda4d9b700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbd84066f30 0x7fbd840693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbda0158720 tx=0x7fbd94007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:11.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:11.852+0000 7fbd9cff9700 1 -- 172.21.15.31:0/2827581746 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd98052c40 con 0x7fbda0157a50 2023-10-06T05:17:12.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.052+0000 7fbda6fff700 1 -- 172.21.15.31:0/2827581746 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbda004c380 con 0x7fbd84066f30 2023-10-06T05:17:12.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.061+0000 7fbd9cff9700 1 -- 172.21.15.31:0/2827581746 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286813 (secure 0 0 0) 0x7fbda004c380 con 0x7fbd84066f30 2023-10-06T05:17:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:12.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 -- 172.21.15.31:0/2827581746 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbd84066f30 msgr2=0x7fbd840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbd84066f30 0x7fbd840693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbda0158720 tx=0x7fbd94007040 comp rx=0 tx=0).stop 2023-10-06T05:17:12.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 -- 172.21.15.31:0/2827581746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda0157a50 msgr2=0x7fbda0157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda0157a50 0x7fbda0157ef0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800e800 tx=0x7fbd9801a000 comp rx=0 tx=0).stop 2023-10-06T05:17:12.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 -- 172.21.15.31:0/2827581746 shutdown_connections 2023-10-06T05:17:12.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbd84066f30 0x7fbd840693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbda0157a50 0x7fbda0157ef0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90004d50 0x7fbda0154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 --2- 172.21.15.31:0/2827581746 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbda00b2a20 0x7fbda016a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 -- 172.21.15.31:0/2827581746 >> 172.21.15.31:0/2827581746 conn(0x7fbda006b620 msgr2=0x7fbda0068670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.065+0000 7fbda6fff700 1 -- 172.21.15.31:0/2827581746 shutdown_connections 2023-10-06T05:17:12.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.066+0000 7fbda6fff700 1 -- 172.21.15.31:0/2827581746 wait complete. 2023-10-06T05:17:12.070 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:17:12.085 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T05:17:10.062471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381128,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593592,"statfs":{"total":773094113280,"available":770655838208,"internally_reserved":0,"allocated":2891776,"data_stored":1778820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002434"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464636+0000","last_change":"2023-10-06T05:17:02.463452+0000","last_active":"2023-10-06T05:17:02.464636+0000","last_peered":"2023-10-06T05:17:02.464636+0000","last_clean":"2023-10-06T05:17:02.464636+0000","last_became_active":"2023-10-06T05:17:02.463139+0000","last_became_peered":"2023-10-06T05:17:02.463139+0000","last_unstale":"2023-10-06T05:17:02.464636+0000","last_undegraded":"2023-10-06T05:17:02.464636+0000","last_fullsized":"2023-10-06T05:17:02.464636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:52.286394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456371+0000","last_change":"2023-10-06T05:17:02.451303+0000","last_active":"2023-10-06T05:17:02.456371+0000","last_peered":"2023-10-06T05:17:02.456371+0000","last_clean":"2023-10-06T05:17:02.456371+0000","last_became_active":"2023-10-06T05:17:02.450985+0000","last_became_peered":"2023-10-06T05:17:02.450985+0000","last_unstale":"2023-10-06T05:17:02.456371+0000","last_undegraded":"2023-10-06T05:17:02.456371+0000","last_fullsized":"2023-10-06T05:17:02.456371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453158+0000","last_change":"2023-10-06T05:17:02.451335+0000","last_active":"2023-10-06T05:17:02.453158+0000","last_peered":"2023-10-06T05:17:02.453158+0000","last_clean":"2023-10-06T05:17:02.453158+0000","last_became_active":"2023-10-06T05:17:02.451119+0000","last_became_peered":"2023-10-06T05:17:02.451119+0000","last_unstale":"2023-10-06T05:17:02.453158+0000","last_undegraded":"2023-10-06T05:17:02.453158+0000","last_fullsized":"2023-10-06T05:17:02.453158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:59.439667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.458136+0000","last_change":"2023-10-06T05:17:02.454559+0000","last_active":"2023-10-06T05:17:02.458136+0000","last_peered":"2023-10-06T05:17:02.458136+0000","last_clean":"2023-10-06T05:17:02.458136+0000","last_became_active":"2023-10-06T05:17:02.453985+0000","last_became_peered":"2023-10-06T05:17:02.453985+0000","last_unstale":"2023-10-06T05:17:02.458136+0000","last_undegraded":"2023-10-06T05:17:02.458136+0000","last_fullsized":"2023-10-06T05:17:02.458136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:57.117094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465742+0000","last_change":"2023-10-06T05:17:02.460869+0000","last_active":"2023-10-06T05:17:02.465742+0000","last_peered":"2023-10-06T05:17:02.465742+0000","last_clean":"2023-10-06T05:17:02.465742+0000","last_became_active":"2023-10-06T05:17:02.460257+0000","last_became_peered":"2023-10-06T05:17:02.460257+0000","last_unstale":"2023-10-06T05:17:02.465742+0000","last_undegraded":"2023-10-06T05:17:02.465742+0000","last_fullsized":"2023-10-06T05:17:02.465742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453122+0000","last_change":"2023-10-06T05:17:02.451827+0000","last_active":"2023-10-06T05:17:02.453122+0000","last_peered":"2023-10-06T05:17:02.453122+0000","last_clean":"2023-10-06T05:17:02.453122+0000","last_became_active":"2023-10-06T05:17:02.451573+0000","last_became_peered":"2023-10-06T05:17:02.451573+0000","last_unstale":"2023-10-06T05:17:02.453122+0000","last_undegraded":"2023-10-06T05:17:02.453122+0000","last_fullsized":"2023-10-06T05:17:02.453122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:50.530278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.454214+0000","last_change":"2023-10-06T05:17:02.451805+0000","last_active":"2023-10-06T05:17:02.454214+0000","last_peered":"2023-10-06T05:17:02.454214+0000","last_clean":"2023-10-06T05:17:02.454214+0000","last_became_active":"2023-10-06T05:17:02.451504+0000","last_became_peered":"2023-10-06T05:17:02.451504+0000","last_unstale":"2023-10-06T05:17:02.454214+0000","last_undegraded":"2023-10-06T05:17:02.454214+0000","last_fullsized":"2023-10-06T05:17:02.454214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448446+0000","last_change":"2023-10-06T05:17:02.440866+0000","last_active":"2023-10-06T05:17:02.448446+0000","last_peered":"2023-10-06T05:17:02.448446+0000","last_clean":"2023-10-06T05:17:02.448446+0000","last_became_active":"2023-10-06T05:17:02.440548+0000","last_became_peered":"2023-10-06T05:17:02.440548+0000","last_unstale":"2023-10-06T05:17:02.448446+0000","last_undegraded":"2023-10-06T05:17:02.448446+0000","last_fullsized":"2023-10-06T05:17:02.448446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.466997+0000","last_change":"2023-10-06T05:17:02.461715+0000","last_active":"2023-10-06T05:17:02.466997+0000","last_peered":"2023-10-06T05:17:02.466997+0000","last_clean":"2023-10-06T05:17:02.466997+0000","last_became_active":"2023-10-06T05:17:02.461152+0000","last_became_peered":"2023-10-06T05:17:02.461152+0000","last_unstale":"2023-10-06T05:17:02.466997+0000","last_undegraded":"2023-10-06T05:17:02.466997+0000","last_fullsized":"2023-10-06T05:17:02.466997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453073+0000","last_change":"2023-10-06T05:17:02.448948+0000","last_active":"2023-10-06T05:17:02.453073+0000","last_peered":"2023-10-06T05:17:02.453073+0000","last_clean":"2023-10-06T05:17:02.453073+0000","last_became_active":"2023-10-06T05:17:02.448524+0000","last_became_peered":"2023-10-06T05:17:02.448524+0000","last_unstale":"2023-10-06T05:17:02.453073+0000","last_undegraded":"2023-10-06T05:17:02.453073+0000","last_fullsized":"2023-10-06T05:17:02.453073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452570+0000","last_change":"2023-10-06T05:17:02.446198+0000","last_active":"2023-10-06T05:17:02.452570+0000","last_peered":"2023-10-06T05:17:02.452570+0000","last_clean":"2023-10-06T05:17:02.452570+0000","last_became_active":"2023-10-06T05:17:02.445918+0000","last_became_peered":"2023-10-06T05:17:02.445918+0000","last_unstale":"2023-10-06T05:17:02.452570+0000","last_undegraded":"2023-10-06T05:17:02.452570+0000","last_fullsized":"2023-10-06T05:17:02.452570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:45.361281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450359+0000","last_change":"2023-10-06T05:17:02.447094+0000","last_active":"2023-10-06T05:17:02.450359+0000","last_peered":"2023-10-06T05:17:02.450359+0000","last_clean":"2023-10-06T05:17:02.450359+0000","last_became_active":"2023-10-06T05:17:02.446800+0000","last_became_peered":"2023-10-06T05:17:02.446800+0000","last_unstale":"2023-10-06T05:17:02.450359+0000","last_undegraded":"2023-10-06T05:17:02.450359+0000","last_fullsized":"2023-10-06T05:17:02.450359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:46.452504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464672+0000","last_change":"2023-10-06T05:17:02.463414+0000","last_active":"2023-10-06T05:17:02.464672+0000","last_peered":"2023-10-06T05:17:02.464672+0000","last_clean":"2023-10-06T05:17:02.464672+0000","last_became_active":"2023-10-06T05:17:02.463125+0000","last_became_peered":"2023-10-06T05:17:02.463125+0000","last_unstale":"2023-10-06T05:17:02.464672+0000","last_undegraded":"2023-10-06T05:17:02.464672+0000","last_fullsized":"2023-10-06T05:17:02.464672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:29.979974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450580+0000","last_change":"2023-10-06T05:17:02.447113+0000","last_active":"2023-10-06T05:17:02.450580+0000","last_peered":"2023-10-06T05:17:02.450580+0000","last_clean":"2023-10-06T05:17:02.450580+0000","last_became_active":"2023-10-06T05:17:02.446878+0000","last_became_peered":"2023-10-06T05:17:02.446878+0000","last_unstale":"2023-10-06T05:17:02.450580+0000","last_undegraded":"2023-10-06T05:17:02.450580+0000","last_fullsized":"2023-10-06T05:17:02.450580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.454233+0000","last_change":"2023-10-06T05:17:02.451949+0000","last_active":"2023-10-06T05:17:02.454233+0000","last_peered":"2023-10-06T05:17:02.454233+0000","last_clean":"2023-10-06T05:17:02.454233+0000","last_became_active":"2023-10-06T05:17:02.451640+0000","last_became_peered":"2023-10-06T05:17:02.451640+0000","last_unstale":"2023-10-06T05:17:02.454233+0000","last_undegraded":"2023-10-06T05:17:02.454233+0000","last_fullsized":"2023-10-06T05:17:02.454233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452479+0000","last_change":"2023-10-06T05:17:02.445441+0000","last_active":"2023-10-06T05:17:02.452479+0000","last_peered":"2023-10-06T05:17:02.452479+0000","last_clean":"2023-10-06T05:17:02.452479+0000","last_became_active":"2023-10-06T05:17:02.445132+0000","last_became_peered":"2023-10-06T05:17:02.445132+0000","last_unstale":"2023-10-06T05:17:02.452479+0000","last_undegraded":"2023-10-06T05:17:02.452479+0000","last_fullsized":"2023-10-06T05:17:02.452479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:10.259711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464487+0000","last_change":"2023-10-06T05:17:02.461554+0000","last_active":"2023-10-06T05:17:02.464487+0000","last_peered":"2023-10-06T05:17:02.464487+0000","last_clean":"2023-10-06T05:17:02.464487+0000","last_became_active":"2023-10-06T05:17:02.460803+0000","last_became_peered":"2023-10-06T05:17:02.460803+0000","last_unstale":"2023-10-06T05:17:02.464487+0000","last_undegraded":"2023-10-06T05:17:02.464487+0000","last_fullsized":"2023-10-06T05:17:02.464487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452431+0000","last_change":"2023-10-06T05:17:02.448249+0000","last_active":"2023-10-06T05:17:02.452431+0000","last_peered":"2023-10-06T05:17:02.452431+0000","last_clean":"2023-10-06T05:17:02.452431+0000","last_became_active":"2023-10-06T05:17:02.447977+0000","last_became_peered":"2023-10-06T05:17:02.447977+0000","last_unstale":"2023-10-06T05:17:02.452431+0000","last_undegraded":"2023-10-06T05:17:02.452431+0000","last_fullsized":"2023-10-06T05:17:02.452431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:10.643350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456974+0000","last_change":"2023-10-06T05:17:02.451653+0000","last_active":"2023-10-06T05:17:02.456974+0000","last_peered":"2023-10-06T05:17:02.456974+0000","last_clean":"2023-10-06T05:17:02.456974+0000","last_became_active":"2023-10-06T05:17:02.451347+0000","last_became_peered":"2023-10-06T05:17:02.451347+0000","last_unstale":"2023-10-06T05:17:02.456974+0000","last_undegraded":"2023-10-06T05:17:02.456974+0000","last_fullsized":"2023-10-06T05:17:02.456974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:20.132880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452508+0000","last_change":"2023-10-06T05:17:02.446775+0000","last_active":"2023-10-06T05:17:02.452508+0000","last_peered":"2023-10-06T05:17:02.452508+0000","last_clean":"2023-10-06T05:17:02.452508+0000","last_became_active":"2023-10-06T05:17:02.446482+0000","last_became_peered":"2023-10-06T05:17:02.446482+0000","last_unstale":"2023-10-06T05:17:02.452508+0000","last_undegraded":"2023-10-06T05:17:02.452508+0000","last_fullsized":"2023-10-06T05:17:02.452508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:14.399779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468275+0000","last_change":"2023-10-06T05:17:02.463237+0000","last_active":"2023-10-06T05:17:02.468275+0000","last_peered":"2023-10-06T05:17:02.468275+0000","last_clean":"2023-10-06T05:17:02.468275+0000","last_became_active":"2023-10-06T05:17:02.462891+0000","last_became_peered":"2023-10-06T05:17:02.462891+0000","last_unstale":"2023-10-06T05:17:02.468275+0000","last_undegraded":"2023-10-06T05:17:02.468275+0000","last_fullsized":"2023-10-06T05:17:02.468275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462403+0000","last_change":"2023-10-06T05:17:02.458157+0000","last_active":"2023-10-06T05:17:02.462403+0000","last_peered":"2023-10-06T05:17:02.462403+0000","last_clean":"2023-10-06T05:17:02.462403+0000","last_became_active":"2023-10-06T05:17:02.455535+0000","last_became_peered":"2023-10-06T05:17:02.455535+0000","last_unstale":"2023-10-06T05:17:02.462403+0000","last_undegraded":"2023-10-06T05:17:02.462403+0000","last_fullsized":"2023-10-06T05:17:02.462403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461887+0000","last_change":"2023-10-06T05:17:02.449068+0000","last_active":"2023-10-06T05:17:02.461887+0000","last_peered":"2023-10-06T05:17:02.461887+0000","last_clean":"2023-10-06T05:17:02.461887+0000","last_became_active":"2023-10-06T05:17:02.448751+0000","last_became_peered":"2023-10-06T05:17:02.448751+0000","last_unstale":"2023-10-06T05:17:02.461887+0000","last_undegraded":"2023-10-06T05:17:02.461887+0000","last_fullsized":"2023-10-06T05:17:02.461887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468443+0000","last_change":"2023-10-06T05:17:02.465489+0000","last_active":"2023-10-06T05:17:02.468443+0000","last_peered":"2023-10-06T05:17:02.468443+0000","last_clean":"2023-10-06T05:17:02.468443+0000","last_became_active":"2023-10-06T05:17:02.465329+0000","last_became_peered":"2023-10-06T05:17:02.465329+0000","last_unstale":"2023-10-06T05:17:02.468443+0000","last_undegraded":"2023-10-06T05:17:02.468443+0000","last_fullsized":"2023-10-06T05:17:02.468443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453337+0000","last_change":"2023-10-06T05:17:02.444229+0000","last_active":"2023-10-06T05:17:02.453337+0000","last_peered":"2023-10-06T05:17:02.453337+0000","last_clean":"2023-10-06T05:17:02.453337+0000","last_became_active":"2023-10-06T05:17:02.444064+0000","last_became_peered":"2023-10-06T05:17:02.444064+0000","last_unstale":"2023-10-06T05:17:02.453337+0000","last_undegraded":"2023-10-06T05:17:02.453337+0000","last_fullsized":"2023-10-06T05:17:02.453337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:39.861351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453265+0000","last_change":"2023-10-06T05:17:02.447686+0000","last_active":"2023-10-06T05:17:02.453265+0000","last_peered":"2023-10-06T05:17:02.453265+0000","last_clean":"2023-10-06T05:17:02.453265+0000","last_became_active":"2023-10-06T05:17:02.447364+0000","last_became_peered":"2023-10-06T05:17:02.447364+0000","last_unstale":"2023-10-06T05:17:02.453265+0000","last_undegraded":"2023-10-06T05:17:02.453265+0000","last_fullsized":"2023-10-06T05:17:02.453265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448537+0000","last_change":"2023-10-06T05:17:02.442095+0000","last_active":"2023-10-06T05:17:02.448537+0000","last_peered":"2023-10-06T05:17:02.448537+0000","last_clean":"2023-10-06T05:17:02.448537+0000","last_became_active":"2023-10-06T05:17:02.441645+0000","last_became_peered":"2023-10-06T05:17:02.441645+0000","last_unstale":"2023-10-06T05:17:02.448537+0000","last_undegraded":"2023-10-06T05:17:02.448537+0000","last_fullsized":"2023-10-06T05:17:02.448537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461441+0000","last_change":"2023-10-06T05:17:01.449274+0000","last_active":"2023-10-06T05:17:02.461441+0000","last_peered":"2023-10-06T05:17:02.461441+0000","last_clean":"2023-10-06T05:17:02.461441+0000","last_became_active":"2023-10-06T05:17:01.448908+0000","last_became_peered":"2023-10-06T05:17:01.448908+0000","last_unstale":"2023-10-06T05:17:02.461441+0000","last_undegraded":"2023-10-06T05:17:02.461441+0000","last_fullsized":"2023-10-06T05:17:02.461441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462280+0000","last_change":"2023-10-06T05:17:02.443949+0000","last_active":"2023-10-06T05:17:02.462280+0000","last_peered":"2023-10-06T05:17:02.462280+0000","last_clean":"2023-10-06T05:17:02.462280+0000","last_became_active":"2023-10-06T05:17:02.443677+0000","last_became_peered":"2023-10-06T05:17:02.443677+0000","last_unstale":"2023-10-06T05:17:02.462280+0000","last_undegraded":"2023-10-06T05:17:02.462280+0000","last_fullsized":"2023-10-06T05:17:02.462280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428951+0000","last_change":"2023-10-06T05:17:01.438479+0000","last_active":"2023-10-06T05:17:02.428951+0000","last_peered":"2023-10-06T05:17:02.428951+0000","last_clean":"2023-10-06T05:17:02.428951+0000","last_became_active":"2023-10-06T05:17:01.438182+0000","last_became_peered":"2023-10-06T05:17:01.438182+0000","last_unstale":"2023-10-06T05:17:02.428951+0000","last_undegraded":"2023-10-06T05:17:02.428951+0000","last_fullsized":"2023-10-06T05:17:02.428951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:42.243614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453470+0000","last_change":"2023-10-06T05:17:02.447288+0000","last_active":"2023-10-06T05:17:02.453470+0000","last_peered":"2023-10-06T05:17:02.453470+0000","last_clean":"2023-10-06T05:17:02.453470+0000","last_became_active":"2023-10-06T05:17:02.446914+0000","last_became_peered":"2023-10-06T05:17:02.446914+0000","last_unstale":"2023-10-06T05:17:02.453470+0000","last_undegraded":"2023-10-06T05:17:02.453470+0000","last_fullsized":"2023-10-06T05:17:02.453470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428771+0000","last_change":"2023-10-06T05:17:01.444309+0000","last_active":"2023-10-06T05:17:02.428771+0000","last_peered":"2023-10-06T05:17:02.428771+0000","last_clean":"2023-10-06T05:17:02.428771+0000","last_became_active":"2023-10-06T05:17:01.444026+0000","last_became_peered":"2023-10-06T05:17:01.444026+0000","last_unstale":"2023-10-06T05:17:02.428771+0000","last_undegraded":"2023-10-06T05:17:02.428771+0000","last_fullsized":"2023-10-06T05:17:02.428771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.458208+0000","last_change":"2023-10-06T05:17:02.455822+0000","last_active":"2023-10-06T05:17:02.458208+0000","last_peered":"2023-10-06T05:17:02.458208+0000","last_clean":"2023-10-06T05:17:02.458208+0000","last_became_active":"2023-10-06T05:17:02.455539+0000","last_became_peered":"2023-10-06T05:17:02.455539+0000","last_unstale":"2023-10-06T05:17:02.458208+0000","last_undegraded":"2023-10-06T05:17:02.458208+0000","last_fullsized":"2023-10-06T05:17:02.458208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434617+0000","last_change":"2023-10-06T05:17:01.453079+0000","last_active":"2023-10-06T05:17:02.434617+0000","last_peered":"2023-10-06T05:17:02.434617+0000","last_clean":"2023-10-06T05:17:02.434617+0000","last_became_active":"2023-10-06T05:17:01.452800+0000","last_became_peered":"2023-10-06T05:17:01.452800+0000","last_unstale":"2023-10-06T05:17:02.434617+0000","last_undegraded":"2023-10-06T05:17:02.434617+0000","last_fullsized":"2023-10-06T05:17:02.434617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.457891+0000","last_change":"2023-10-06T05:17:02.439162+0000","last_active":"2023-10-06T05:17:02.457891+0000","last_peered":"2023-10-06T05:17:02.457891+0000","last_clean":"2023-10-06T05:17:02.457891+0000","last_became_active":"2023-10-06T05:17:02.438810+0000","last_became_peered":"2023-10-06T05:17:02.438810+0000","last_unstale":"2023-10-06T05:17:02.457891+0000","last_undegraded":"2023-10-06T05:17:02.457891+0000","last_fullsized":"2023-10-06T05:17:02.457891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435781+0000","last_change":"2023-10-06T05:17:01.449427+0000","last_active":"2023-10-06T05:17:02.435781+0000","last_peered":"2023-10-06T05:17:02.435781+0000","last_clean":"2023-10-06T05:17:02.435781+0000","last_became_active":"2023-10-06T05:17:01.449100+0000","last_became_peered":"2023-10-06T05:17:01.449100+0000","last_unstale":"2023-10-06T05:17:02.435781+0000","last_undegraded":"2023-10-06T05:17:02.435781+0000","last_fullsized":"2023-10-06T05:17:02.435781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461821+0000","last_change":"2023-10-06T05:17:02.441035+0000","last_active":"2023-10-06T05:17:02.461821+0000","last_peered":"2023-10-06T05:17:02.461821+0000","last_clean":"2023-10-06T05:17:02.461821+0000","last_became_active":"2023-10-06T05:17:02.440422+0000","last_became_peered":"2023-10-06T05:17:02.440422+0000","last_unstale":"2023-10-06T05:17:02.461821+0000","last_undegraded":"2023-10-06T05:17:02.461821+0000","last_fullsized":"2023-10-06T05:17:02.461821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468652+0000","last_change":"2023-10-06T05:17:01.445468+0000","last_active":"2023-10-06T05:17:02.468652+0000","last_peered":"2023-10-06T05:17:02.468652+0000","last_clean":"2023-10-06T05:17:02.468652+0000","last_became_active":"2023-10-06T05:17:01.445179+0000","last_became_peered":"2023-10-06T05:17:01.445179+0000","last_unstale":"2023-10-06T05:17:02.468652+0000","last_undegraded":"2023-10-06T05:17:02.468652+0000","last_fullsized":"2023-10-06T05:17:02.468652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:01.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453479+0000","last_change":"2023-10-06T05:17:02.444872+0000","last_active":"2023-10-06T05:17:02.453479+0000","last_peered":"2023-10-06T05:17:02.453479+0000","last_clean":"2023-10-06T05:17:02.453479+0000","last_became_active":"2023-10-06T05:17:02.444565+0000","last_became_peered":"2023-10-06T05:17:02.444565+0000","last_unstale":"2023-10-06T05:17:02.453479+0000","last_undegraded":"2023-10-06T05:17:02.453479+0000","last_fullsized":"2023-10-06T05:17:02.453479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.470982+0000","last_change":"2023-10-06T05:17:01.448641+0000","last_active":"2023-10-06T05:17:02.470982+0000","last_peered":"2023-10-06T05:17:02.470982+0000","last_clean":"2023-10-06T05:17:02.470982+0000","last_became_active":"2023-10-06T05:17:01.448232+0000","last_became_peered":"2023-10-06T05:17:01.448232+0000","last_unstale":"2023-10-06T05:17:02.470982+0000","last_undegraded":"2023-10-06T05:17:02.470982+0000","last_fullsized":"2023-10-06T05:17:02.470982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447551+0000","last_change":"2023-10-06T05:17:02.443125+0000","last_active":"2023-10-06T05:17:02.447551+0000","last_peered":"2023-10-06T05:17:02.447551+0000","last_clean":"2023-10-06T05:17:02.447551+0000","last_became_active":"2023-10-06T05:17:02.442847+0000","last_became_peered":"2023-10-06T05:17:02.442847+0000","last_unstale":"2023-10-06T05:17:02.447551+0000","last_undegraded":"2023-10-06T05:17:02.447551+0000","last_fullsized":"2023-10-06T05:17:02.447551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471314+0000","last_change":"2023-10-06T05:17:01.451184+0000","last_active":"2023-10-06T05:17:02.471314+0000","last_peered":"2023-10-06T05:17:02.471314+0000","last_clean":"2023-10-06T05:17:02.471314+0000","last_became_active":"2023-10-06T05:17:01.450521+0000","last_became_peered":"2023-10-06T05:17:01.450521+0000","last_unstale":"2023-10-06T05:17:02.471314+0000","last_undegraded":"2023-10-06T05:17:02.471314+0000","last_fullsized":"2023-10-06T05:17:02.471314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:56.109170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462307+0000","last_change":"2023-10-06T05:17:02.444967+0000","last_active":"2023-10-06T05:17:02.462307+0000","last_peered":"2023-10-06T05:17:02.462307+0000","last_clean":"2023-10-06T05:17:02.462307+0000","last_became_active":"2023-10-06T05:17:02.444639+0000","last_became_peered":"2023-10-06T05:17:02.444639+0000","last_unstale":"2023-10-06T05:17:02.462307+0000","last_undegraded":"2023-10-06T05:17:02.462307+0000","last_fullsized":"2023-10-06T05:17:02.462307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:59.439667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.439263+0000","last_change":"2023-10-06T05:17:01.446625+0000","last_active":"2023-10-06T05:17:02.439263+0000","last_peered":"2023-10-06T05:17:02.439263+0000","last_clean":"2023-10-06T05:17:02.439263+0000","last_became_active":"2023-10-06T05:17:01.446249+0000","last_became_peered":"2023-10-06T05:17:01.446249+0000","last_unstale":"2023-10-06T05:17:02.439263+0000","last_undegraded":"2023-10-06T05:17:02.439263+0000","last_fullsized":"2023-10-06T05:17:02.439263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448517+0000","last_change":"2023-10-06T05:17:02.441887+0000","last_active":"2023-10-06T05:17:02.448517+0000","last_peered":"2023-10-06T05:17:02.448517+0000","last_clean":"2023-10-06T05:17:02.448517+0000","last_became_active":"2023-10-06T05:17:02.441377+0000","last_became_peered":"2023-10-06T05:17:02.441377+0000","last_unstale":"2023-10-06T05:17:02.448517+0000","last_undegraded":"2023-10-06T05:17:02.448517+0000","last_fullsized":"2023-10-06T05:17:02.448517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434243+0000","last_change":"2023-10-06T05:17:01.450424+0000","last_active":"2023-10-06T05:17:02.434243+0000","last_peered":"2023-10-06T05:17:02.434243+0000","last_clean":"2023-10-06T05:17:02.434243+0000","last_became_active":"2023-10-06T05:17:01.450072+0000","last_became_peered":"2023-10-06T05:17:01.450072+0000","last_unstale":"2023-10-06T05:17:02.434243+0000","last_undegraded":"2023-10-06T05:17:02.434243+0000","last_fullsized":"2023-10-06T05:17:02.434243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:19.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464457+0000","last_change":"2023-10-06T05:17:02.460678+0000","last_active":"2023-10-06T05:17:02.464457+0000","last_peered":"2023-10-06T05:17:02.464457+0000","last_clean":"2023-10-06T05:17:02.464457+0000","last_became_active":"2023-10-06T05:17:02.460159+0000","last_became_peered":"2023-10-06T05:17:02.460159+0000","last_unstale":"2023-10-06T05:17:02.464457+0000","last_undegraded":"2023-10-06T05:17:02.464457+0000","last_fullsized":"2023-10-06T05:17:02.464457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.467405+0000","last_change":"2023-10-06T05:17:01.447124+0000","last_active":"2023-10-06T05:17:02.467405+0000","last_peered":"2023-10-06T05:17:02.467405+0000","last_clean":"2023-10-06T05:17:02.467405+0000","last_became_active":"2023-10-06T05:17:01.446810+0000","last_became_peered":"2023-10-06T05:17:01.446810+0000","last_unstale":"2023-10-06T05:17:02.467405+0000","last_undegraded":"2023-10-06T05:17:02.467405+0000","last_fullsized":"2023-10-06T05:17:02.467405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462426+0000","last_change":"2023-10-06T05:17:02.440973+0000","last_active":"2023-10-06T05:17:02.462426+0000","last_peered":"2023-10-06T05:17:02.462426+0000","last_clean":"2023-10-06T05:17:02.462426+0000","last_became_active":"2023-10-06T05:17:02.440594+0000","last_became_peered":"2023-10-06T05:17:02.440594+0000","last_unstale":"2023-10-06T05:17:02.462426+0000","last_undegraded":"2023-10-06T05:17:02.462426+0000","last_fullsized":"2023-10-06T05:17:02.462426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:45.361281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471211+0000","last_change":"2023-10-06T05:17:01.454983+0000","last_active":"2023-10-06T05:17:02.471211+0000","last_peered":"2023-10-06T05:17:02.471211+0000","last_clean":"2023-10-06T05:17:02.471211+0000","last_became_active":"2023-10-06T05:17:01.454580+0000","last_became_peered":"2023-10-06T05:17:01.454580+0000","last_unstale":"2023-10-06T05:17:02.471211+0000","last_undegraded":"2023-10-06T05:17:02.471211+0000","last_fullsized":"2023-10-06T05:17:02.471211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:53.875873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448468+0000","last_change":"2023-10-06T05:17:02.442909+0000","last_active":"2023-10-06T05:17:02.448468+0000","last_peered":"2023-10-06T05:17:02.448468+0000","last_clean":"2023-10-06T05:17:02.448468+0000","last_became_active":"2023-10-06T05:17:02.442619+0000","last_became_peered":"2023-10-06T05:17:02.442619+0000","last_unstale":"2023-10-06T05:17:02.448468+0000","last_undegraded":"2023-10-06T05:17:02.448468+0000","last_fullsized":"2023-10-06T05:17:02.448468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471248+0000","last_change":"2023-10-06T05:17:01.454784+0000","last_active":"2023-10-06T05:17:02.471248+0000","last_peered":"2023-10-06T05:17:02.471248+0000","last_clean":"2023-10-06T05:17:02.471248+0000","last_became_active":"2023-10-06T05:17:01.454498+0000","last_became_peered":"2023-10-06T05:17:01.454498+0000","last_unstale":"2023-10-06T05:17:02.471248+0000","last_undegraded":"2023-10-06T05:17:02.471248+0000","last_fullsized":"2023-10-06T05:17:02.471248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:41.021745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452621+0000","last_change":"2023-10-06T05:17:02.446992+0000","last_active":"2023-10-06T05:17:02.452621+0000","last_peered":"2023-10-06T05:17:02.452621+0000","last_clean":"2023-10-06T05:17:02.452621+0000","last_became_active":"2023-10-06T05:17:02.445148+0000","last_became_peered":"2023-10-06T05:17:02.445148+0000","last_unstale":"2023-10-06T05:17:02.452621+0000","last_undegraded":"2023-10-06T05:17:02.452621+0000","last_fullsized":"2023-10-06T05:17:02.452621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:31.781450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.469131+0000","last_change":"2023-10-06T05:17:01.446494+0000","last_active":"2023-10-06T05:17:02.469131+0000","last_peered":"2023-10-06T05:17:02.469131+0000","last_clean":"2023-10-06T05:17:02.469131+0000","last_became_active":"2023-10-06T05:17:01.446164+0000","last_became_peered":"2023-10-06T05:17:01.446164+0000","last_unstale":"2023-10-06T05:17:02.469131+0000","last_undegraded":"2023-10-06T05:17:02.469131+0000","last_fullsized":"2023-10-06T05:17:02.469131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:45.078207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456419+0000","last_change":"2023-10-06T05:17:02.451322+0000","last_active":"2023-10-06T05:17:02.456419+0000","last_peered":"2023-10-06T05:17:02.456419+0000","last_clean":"2023-10-06T05:17:02.456419+0000","last_became_active":"2023-10-06T05:17:02.451064+0000","last_became_peered":"2023-10-06T05:17:02.451064+0000","last_unstale":"2023-10-06T05:17:02.456419+0000","last_undegraded":"2023-10-06T05:17:02.456419+0000","last_fullsized":"2023-10-06T05:17:02.456419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:46.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.432653+0000","last_change":"2023-10-06T05:17:01.443417+0000","last_active":"2023-10-06T05:17:02.432653+0000","last_peered":"2023-10-06T05:17:02.432653+0000","last_clean":"2023-10-06T05:17:02.432653+0000","last_became_active":"2023-10-06T05:17:01.443114+0000","last_became_peered":"2023-10-06T05:17:01.443114+0000","last_unstale":"2023-10-06T05:17:02.432653+0000","last_undegraded":"2023-10-06T05:17:02.432653+0000","last_fullsized":"2023-10-06T05:17:02.432653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461876+0000","last_change":"2023-10-06T05:17:02.456471+0000","last_active":"2023-10-06T05:17:02.461876+0000","last_peered":"2023-10-06T05:17:02.461876+0000","last_clean":"2023-10-06T05:17:02.461876+0000","last_became_active":"2023-10-06T05:17:02.456142+0000","last_became_peered":"2023-10-06T05:17:02.456142+0000","last_unstale":"2023-10-06T05:17:02.461876+0000","last_undegraded":"2023-10-06T05:17:02.461876+0000","last_fullsized":"2023-10-06T05:17:02.461876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.466429+0000","last_change":"2023-10-06T05:17:01.444030+0000","last_active":"2023-10-06T05:17:02.466429+0000","last_peered":"2023-10-06T05:17:02.466429+0000","last_clean":"2023-10-06T05:17:02.466429+0000","last_became_active":"2023-10-06T05:17:01.443754+0000","last_became_peered":"2023-10-06T05:17:01.443754+0000","last_unstale":"2023-10-06T05:17:02.466429+0000","last_undegraded":"2023-10-06T05:17:02.466429+0000","last_fullsized":"2023-10-06T05:17:02.466429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.440368+0000","last_change":"2023-10-06T05:17:02.437195+0000","last_active":"2023-10-06T05:17:02.440368+0000","last_peered":"2023-10-06T05:17:02.440368+0000","last_clean":"2023-10-06T05:17:02.440368+0000","last_became_active":"2023-10-06T05:17:02.436917+0000","last_became_peered":"2023-10-06T05:17:02.436917+0000","last_unstale":"2023-10-06T05:17:02.440368+0000","last_undegraded":"2023-10-06T05:17:02.440368+0000","last_fullsized":"2023-10-06T05:17:02.440368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435134+0000","last_change":"2023-10-06T05:17:01.454676+0000","last_active":"2023-10-06T05:17:02.435134+0000","last_peered":"2023-10-06T05:17:02.435134+0000","last_clean":"2023-10-06T05:17:02.435134+0000","last_became_active":"2023-10-06T05:17:01.454385+0000","last_became_peered":"2023-10-06T05:17:01.454385+0000","last_unstale":"2023-10-06T05:17:02.435134+0000","last_undegraded":"2023-10-06T05:17:02.435134+0000","last_fullsized":"2023-10-06T05:17:02.435134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453140+0000","last_change":"2023-10-06T05:17:02.449388+0000","last_active":"2023-10-06T05:17:02.453140+0000","last_peered":"2023-10-06T05:17:02.453140+0000","last_clean":"2023-10-06T05:17:02.453140+0000","last_became_active":"2023-10-06T05:17:02.448868+0000","last_became_peered":"2023-10-06T05:17:02.448868+0000","last_unstale":"2023-10-06T05:17:02.453140+0000","last_undegraded":"2023-10-06T05:17:02.453140+0000","last_fullsized":"2023-10-06T05:17:02.453140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428450+0000","last_change":"2023-10-06T05:17:01.448698+0000","last_active":"2023-10-06T05:17:02.428450+0000","last_peered":"2023-10-06T05:17:02.428450+0000","last_clean":"2023-10-06T05:17:02.428450+0000","last_became_active":"2023-10-06T05:17:01.448084+0000","last_became_peered":"2023-10-06T05:17:01.448084+0000","last_unstale":"2023-10-06T05:17:02.428450+0000","last_undegraded":"2023-10-06T05:17:02.428450+0000","last_fullsized":"2023-10-06T05:17:02.428450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452412+0000","last_change":"2023-10-06T05:17:02.446522+0000","last_active":"2023-10-06T05:17:02.452412+0000","last_peered":"2023-10-06T05:17:02.452412+0000","last_clean":"2023-10-06T05:17:02.452412+0000","last_became_active":"2023-10-06T05:17:02.446203+0000","last_became_peered":"2023-10-06T05:17:02.446203+0000","last_unstale":"2023-10-06T05:17:02.452412+0000","last_undegraded":"2023-10-06T05:17:02.452412+0000","last_fullsized":"2023-10-06T05:17:02.452412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:28.806289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428705+0000","last_change":"2023-10-06T05:17:01.441195+0000","last_active":"2023-10-06T05:17:02.428705+0000","last_peered":"2023-10-06T05:17:02.428705+0000","last_clean":"2023-10-06T05:17:02.428705+0000","last_became_active":"2023-10-06T05:17:01.440903+0000","last_became_peered":"2023-10-06T05:17:01.440903+0000","last_unstale":"2023-10-06T05:17:02.428705+0000","last_undegraded":"2023-10-06T05:17:02.428705+0000","last_fullsized":"2023-10-06T05:17:02.428705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453318+0000","last_change":"2023-10-06T05:17:02.444240+0000","last_active":"2023-10-06T05:17:02.453318+0000","last_peered":"2023-10-06T05:17:02.453318+0000","last_clean":"2023-10-06T05:17:02.453318+0000","last_became_active":"2023-10-06T05:17:02.443717+0000","last_became_peered":"2023-10-06T05:17:02.443717+0000","last_unstale":"2023-10-06T05:17:02.453318+0000","last_undegraded":"2023-10-06T05:17:02.453318+0000","last_fullsized":"2023-10-06T05:17:02.453318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464271+0000","last_change":"2023-10-06T05:17:01.443363+0000","last_active":"2023-10-06T05:17:02.464271+0000","last_peered":"2023-10-06T05:17:02.464271+0000","last_clean":"2023-10-06T05:17:02.464271+0000","last_became_active":"2023-10-06T05:17:01.443131+0000","last_became_peered":"2023-10-06T05:17:01.443131+0000","last_unstale":"2023-10-06T05:17:02.464271+0000","last_undegraded":"2023-10-06T05:17:02.464271+0000","last_fullsized":"2023-10-06T05:17:02.464271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428924+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:17:02.428924+0000","last_peered":"2023-10-06T05:17:02.428924+0000","last_clean":"2023-10-06T05:17:02.428924+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:17:02.428924+0000","last_undegraded":"2023-10-06T05:17:02.428924+0000","last_fullsized":"2023-10-06T05:17:02.428924+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464122+0000","last_change":"2023-10-06T05:17:02.442449+0000","last_active":"2023-10-06T05:17:02.464122+0000","last_peered":"2023-10-06T05:17:02.464122+0000","last_clean":"2023-10-06T05:17:02.464122+0000","last_became_active":"2023-10-06T05:17:02.442135+0000","last_became_peered":"2023-10-06T05:17:02.442135+0000","last_unstale":"2023-10-06T05:17:02.464122+0000","last_undegraded":"2023-10-06T05:17:02.464122+0000","last_fullsized":"2023-10-06T05:17:02.464122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:31.781450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.427977+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:17:02.427977+0000","last_peered":"2023-10-06T05:17:02.427977+0000","last_clean":"2023-10-06T05:17:02.427977+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:17:02.427977+0000","last_undegraded":"2023-10-06T05:17:02.427977+0000","last_fullsized":"2023-10-06T05:17:02.427977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T05:17:04.091912+0000","last_change":"2023-10-06T05:17:01.448082+0000","last_active":"2023-10-06T05:17:04.091912+0000","last_peered":"2023-10-06T05:17:04.091912+0000","last_clean":"2023-10-06T05:17:04.091912+0000","last_became_active":"2023-10-06T05:17:01.447418+0000","last_became_peered":"2023-10-06T05:17:01.447418+0000","last_unstale":"2023-10-06T05:17:04.091912+0000","last_undegraded":"2023-10-06T05:17:04.091912+0000","last_fullsized":"2023-10-06T05:17:04.091912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452602+0000","last_change":"2023-10-06T05:17:02.445401+0000","last_active":"2023-10-06T05:17:02.452602+0000","last_peered":"2023-10-06T05:17:02.452602+0000","last_clean":"2023-10-06T05:17:02.452602+0000","last_became_active":"2023-10-06T05:17:02.444770+0000","last_became_peered":"2023-10-06T05:17:02.444770+0000","last_unstale":"2023-10-06T05:17:02.452602+0000","last_undegraded":"2023-10-06T05:17:02.452602+0000","last_fullsized":"2023-10-06T05:17:02.452602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429051+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:17:02.429051+0000","last_peered":"2023-10-06T05:17:02.429051+0000","last_clean":"2023-10-06T05:17:02.429051+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:17:02.429051+0000","last_undegraded":"2023-10-06T05:17:02.429051+0000","last_fullsized":"2023-10-06T05:17:02.429051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.436179+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:17:02.436179+0000","last_peered":"2023-10-06T05:17:02.436179+0000","last_clean":"2023-10-06T05:17:02.436179+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:17:02.436179+0000","last_undegraded":"2023-10-06T05:17:02.436179+0000","last_fullsized":"2023-10-06T05:17:02.436179+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.432715+0000","last_change":"2023-10-06T05:17:01.447882+0000","last_active":"2023-10-06T05:17:02.432715+0000","last_peered":"2023-10-06T05:17:02.432715+0000","last_clean":"2023-10-06T05:17:02.432715+0000","last_became_active":"2023-10-06T05:17:01.447617+0000","last_became_peered":"2023-10-06T05:17:01.447617+0000","last_unstale":"2023-10-06T05:17:02.432715+0000","last_undegraded":"2023-10-06T05:17:02.432715+0000","last_fullsized":"2023-10-06T05:17:02.432715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.460775+0000","last_change":"2023-10-06T05:17:02.455716+0000","last_active":"2023-10-06T05:17:02.460775+0000","last_peered":"2023-10-06T05:17:02.460775+0000","last_clean":"2023-10-06T05:17:02.460775+0000","last_became_active":"2023-10-06T05:17:02.455388+0000","last_became_peered":"2023-10-06T05:17:02.455388+0000","last_unstale":"2023-10-06T05:17:02.460775+0000","last_undegraded":"2023-10-06T05:17:02.460775+0000","last_fullsized":"2023-10-06T05:17:02.460775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434749+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:17:02.434749+0000","last_peered":"2023-10-06T05:17:02.434749+0000","last_clean":"2023-10-06T05:17:02.434749+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:17:02.434749+0000","last_undegraded":"2023-10-06T05:17:02.434749+0000","last_fullsized":"2023-10-06T05:17:02.434749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429080+0000","last_change":"2023-10-06T05:17:01.449606+0000","last_active":"2023-10-06T05:17:02.429080+0000","last_peered":"2023-10-06T05:17:02.429080+0000","last_clean":"2023-10-06T05:17:02.429080+0000","last_became_active":"2023-10-06T05:17:01.449380+0000","last_became_peered":"2023-10-06T05:17:01.449380+0000","last_unstale":"2023-10-06T05:17:02.429080+0000","last_undegraded":"2023-10-06T05:17:02.429080+0000","last_fullsized":"2023-10-06T05:17:02.429080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453300+0000","last_change":"2023-10-06T05:17:02.444044+0000","last_active":"2023-10-06T05:17:02.453300+0000","last_peered":"2023-10-06T05:17:02.453300+0000","last_clean":"2023-10-06T05:17:02.453300+0000","last_became_active":"2023-10-06T05:17:02.443659+0000","last_became_peered":"2023-10-06T05:17:02.443659+0000","last_unstale":"2023-10-06T05:17:02.453300+0000","last_undegraded":"2023-10-06T05:17:02.453300+0000","last_fullsized":"2023-10-06T05:17:02.453300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:46.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435753+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:17:02.435753+0000","last_peered":"2023-10-06T05:17:02.435753+0000","last_clean":"2023-10-06T05:17:02.435753+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:17:02.435753+0000","last_undegraded":"2023-10-06T05:17:02.435753+0000","last_fullsized":"2023-10-06T05:17:02.435753+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.427375+0000","last_change":"2023-10-06T05:17:01.441700+0000","last_active":"2023-10-06T05:17:02.427375+0000","last_peered":"2023-10-06T05:17:02.427375+0000","last_clean":"2023-10-06T05:17:02.427375+0000","last_became_active":"2023-10-06T05:17:01.441370+0000","last_became_peered":"2023-10-06T05:17:01.441370+0000","last_unstale":"2023-10-06T05:17:02.427375+0000","last_undegraded":"2023-10-06T05:17:02.427375+0000","last_fullsized":"2023-10-06T05:17:02.427375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:01.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450934+0000","last_change":"2023-10-06T05:17:02.446567+0000","last_active":"2023-10-06T05:17:02.450934+0000","last_peered":"2023-10-06T05:17:02.450934+0000","last_clean":"2023-10-06T05:17:02.450934+0000","last_became_active":"2023-10-06T05:17:02.446263+0000","last_became_peered":"2023-10-06T05:17:02.446263+0000","last_unstale":"2023-10-06T05:17:02.450934+0000","last_undegraded":"2023-10-06T05:17:02.450934+0000","last_fullsized":"2023-10-06T05:17:02.450934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.433849+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:17:02.433849+0000","last_peered":"2023-10-06T05:17:02.433849+0000","last_clean":"2023-10-06T05:17:02.433849+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:17:02.433849+0000","last_undegraded":"2023-10-06T05:17:02.433849+0000","last_fullsized":"2023-10-06T05:17:02.433849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429218+0000","last_change":"2023-10-06T05:17:01.448636+0000","last_active":"2023-10-06T05:17:02.429218+0000","last_peered":"2023-10-06T05:17:02.429218+0000","last_clean":"2023-10-06T05:17:02.429218+0000","last_became_active":"2023-10-06T05:17:01.446188+0000","last_became_peered":"2023-10-06T05:17:01.446188+0000","last_unstale":"2023-10-06T05:17:02.429218+0000","last_undegraded":"2023-10-06T05:17:02.429218+0000","last_fullsized":"2023-10-06T05:17:02.429218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453488+0000","last_change":"2023-10-06T05:17:02.447390+0000","last_active":"2023-10-06T05:17:02.453488+0000","last_peered":"2023-10-06T05:17:02.453488+0000","last_clean":"2023-10-06T05:17:02.453488+0000","last_became_active":"2023-10-06T05:17:02.447118+0000","last_became_peered":"2023-10-06T05:17:02.447118+0000","last_unstale":"2023-10-06T05:17:02.453488+0000","last_undegraded":"2023-10-06T05:17:02.453488+0000","last_fullsized":"2023-10-06T05:17:02.453488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.439722+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:17:02.439722+0000","last_peered":"2023-10-06T05:17:02.439722+0000","last_clean":"2023-10-06T05:17:02.439722+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:17:02.439722+0000","last_undegraded":"2023-10-06T05:17:02.439722+0000","last_fullsized":"2023-10-06T05:17:02.439722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.431999+0000","last_change":"2023-10-06T05:17:01.449227+0000","last_active":"2023-10-06T05:17:02.431999+0000","last_peered":"2023-10-06T05:17:02.431999+0000","last_clean":"2023-10-06T05:17:02.431999+0000","last_became_active":"2023-10-06T05:17:01.448956+0000","last_became_peered":"2023-10-06T05:17:01.448956+0000","last_unstale":"2023-10-06T05:17:02.431999+0000","last_undegraded":"2023-10-06T05:17:02.431999+0000","last_fullsized":"2023-10-06T05:17:02.431999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450001+0000","last_change":"2023-10-06T05:17:02.442854+0000","last_active":"2023-10-06T05:17:02.450001+0000","last_peered":"2023-10-06T05:17:02.450001+0000","last_clean":"2023-10-06T05:17:02.450001+0000","last_became_active":"2023-10-06T05:17:02.442424+0000","last_became_peered":"2023-10-06T05:17:02.442424+0000","last_unstale":"2023-10-06T05:17:02.450001+0000","last_undegraded":"2023-10-06T05:17:02.450001+0000","last_fullsized":"2023-10-06T05:17:02.450001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.437608+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:17:02.437608+0000","last_peered":"2023-10-06T05:17:02.437608+0000","last_clean":"2023-10-06T05:17:02.437608+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:17:02.437608+0000","last_undegraded":"2023-10-06T05:17:02.437608+0000","last_fullsized":"2023-10-06T05:17:02.437608+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.433246+0000","last_change":"2023-10-06T05:17:01.444318+0000","last_active":"2023-10-06T05:17:02.433246+0000","last_peered":"2023-10-06T05:17:02.433246+0000","last_clean":"2023-10-06T05:17:02.433246+0000","last_became_active":"2023-10-06T05:17:01.444042+0000","last_became_peered":"2023-10-06T05:17:01.444042+0000","last_unstale":"2023-10-06T05:17:02.433246+0000","last_undegraded":"2023-10-06T05:17:02.433246+0000","last_fullsized":"2023-10-06T05:17:02.433246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453458+0000","last_change":"2023-10-06T05:17:02.443796+0000","last_active":"2023-10-06T05:17:02.453458+0000","last_peered":"2023-10-06T05:17:02.453458+0000","last_clean":"2023-10-06T05:17:02.453458+0000","last_became_active":"2023-10-06T05:17:02.443364+0000","last_became_peered":"2023-10-06T05:17:02.443364+0000","last_unstale":"2023-10-06T05:17:02.453458+0000","last_undegraded":"2023-10-06T05:17:02.453458+0000","last_fullsized":"2023-10-06T05:17:02.453458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:07.184397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465585+0000","last_change":"2023-10-06T05:17:01.451251+0000","last_active":"2023-10-06T05:17:02.465585+0000","last_peered":"2023-10-06T05:17:02.465585+0000","last_clean":"2023-10-06T05:17:02.465585+0000","last_became_active":"2023-10-06T05:17:01.450983+0000","last_became_peered":"2023-10-06T05:17:01.450983+0000","last_unstale":"2023-10-06T05:17:02.465585+0000","last_undegraded":"2023-10-06T05:17:02.465585+0000","last_fullsized":"2023-10-06T05:17:02.465585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456082+0000","last_change":"2023-10-06T05:17:02.451794+0000","last_active":"2023-10-06T05:17:02.456082+0000","last_peered":"2023-10-06T05:17:02.456082+0000","last_clean":"2023-10-06T05:17:02.456082+0000","last_became_active":"2023-10-06T05:17:02.451533+0000","last_became_peered":"2023-10-06T05:17:02.451533+0000","last_unstale":"2023-10-06T05:17:02.456082+0000","last_undegraded":"2023-10-06T05:17:02.456082+0000","last_fullsized":"2023-10-06T05:17:02.456082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:23.939592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471267+0000","last_change":"2023-10-06T05:17:01.449818+0000","last_active":"2023-10-06T05:17:02.471267+0000","last_peered":"2023-10-06T05:17:02.471267+0000","last_clean":"2023-10-06T05:17:02.471267+0000","last_became_active":"2023-10-06T05:17:01.449477+0000","last_became_peered":"2023-10-06T05:17:01.449477+0000","last_unstale":"2023-10-06T05:17:02.471267+0000","last_undegraded":"2023-10-06T05:17:02.471267+0000","last_fullsized":"2023-10-06T05:17:02.471267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447570+0000","last_change":"2023-10-06T05:17:02.443687+0000","last_active":"2023-10-06T05:17:02.447570+0000","last_peered":"2023-10-06T05:17:02.447570+0000","last_clean":"2023-10-06T05:17:02.447570+0000","last_became_active":"2023-10-06T05:17:02.443433+0000","last_became_peered":"2023-10-06T05:17:02.443433+0000","last_unstale":"2023-10-06T05:17:02.447570+0000","last_undegraded":"2023-10-06T05:17:02.447570+0000","last_fullsized":"2023-10-06T05:17:02.447570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:29.361752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471253+0000","last_change":"2023-10-06T05:17:01.449867+0000","last_active":"2023-10-06T05:17:02.471253+0000","last_peered":"2023-10-06T05:17:02.471253+0000","last_clean":"2023-10-06T05:17:02.471253+0000","last_became_active":"2023-10-06T05:17:01.449452+0000","last_became_peered":"2023-10-06T05:17:01.449452+0000","last_unstale":"2023-10-06T05:17:02.471253+0000","last_undegraded":"2023-10-06T05:17:02.471253+0000","last_fullsized":"2023-10-06T05:17:02.471253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447425+0000","last_change":"2023-10-06T05:17:02.439184+0000","last_active":"2023-10-06T05:17:02.447425+0000","last_peered":"2023-10-06T05:17:02.447425+0000","last_clean":"2023-10-06T05:17:02.447425+0000","last_became_active":"2023-10-06T05:17:02.438893+0000","last_became_peered":"2023-10-06T05:17:02.438893+0000","last_unstale":"2023-10-06T05:17:02.447425+0000","last_undegraded":"2023-10-06T05:17:02.447425+0000","last_fullsized":"2023-10-06T05:17:02.447425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:28.005164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.463116+0000","last_change":"2023-10-06T05:17:01.441308+0000","last_active":"2023-10-06T05:17:02.463116+0000","last_peered":"2023-10-06T05:17:02.463116+0000","last_clean":"2023-10-06T05:17:02.463116+0000","last_became_active":"2023-10-06T05:17:01.440904+0000","last_became_peered":"2023-10-06T05:17:01.440904+0000","last_unstale":"2023-10-06T05:17:02.463116+0000","last_undegraded":"2023-10-06T05:17:02.463116+0000","last_fullsized":"2023-10-06T05:17:02.463116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:52.916083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461869+0000","last_change":"2023-10-06T05:17:02.447801+0000","last_active":"2023-10-06T05:17:02.461869+0000","last_peered":"2023-10-06T05:17:02.461869+0000","last_clean":"2023-10-06T05:17:02.461869+0000","last_became_active":"2023-10-06T05:17:02.447361+0000","last_became_peered":"2023-10-06T05:17:02.447361+0000","last_unstale":"2023-10-06T05:17:02.461869+0000","last_undegraded":"2023-10-06T05:17:02.461869+0000","last_fullsized":"2023-10-06T05:17:02.461869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465330+0000","last_change":"2023-10-06T05:17:01.446169+0000","last_active":"2023-10-06T05:17:02.465330+0000","last_peered":"2023-10-06T05:17:02.465330+0000","last_clean":"2023-10-06T05:17:02.465330+0000","last_became_active":"2023-10-06T05:17:01.445836+0000","last_became_peered":"2023-10-06T05:17:01.445836+0000","last_unstale":"2023-10-06T05:17:02.465330+0000","last_undegraded":"2023-10-06T05:17:02.465330+0000","last_fullsized":"2023-10-06T05:17:02.465330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:52.916083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452461+0000","last_change":"2023-10-06T05:17:02.447171+0000","last_active":"2023-10-06T05:17:02.452461+0000","last_peered":"2023-10-06T05:17:02.452461+0000","last_clean":"2023-10-06T05:17:02.452461+0000","last_became_active":"2023-10-06T05:17:02.446884+0000","last_became_peered":"2023-10-06T05:17:02.446884+0000","last_unstale":"2023-10-06T05:17:02.452461+0000","last_undegraded":"2023-10-06T05:17:02.452461+0000","last_fullsized":"2023-10-06T05:17:02.452461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456523+0000","last_change":"2023-10-06T05:17:02.451494+0000","last_active":"2023-10-06T05:17:02.456523+0000","last_peered":"2023-10-06T05:17:02.456523+0000","last_clean":"2023-10-06T05:17:02.456523+0000","last_became_active":"2023-10-06T05:17:02.451213+0000","last_became_peered":"2023-10-06T05:17:02.451213+0000","last_unstale":"2023-10-06T05:17:02.456523+0000","last_undegraded":"2023-10-06T05:17:02.456523+0000","last_fullsized":"2023-10-06T05:17:02.456523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:54.883797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461850+0000","last_change":"2023-10-06T05:17:02.444575+0000","last_active":"2023-10-06T05:17:02.461850+0000","last_peered":"2023-10-06T05:17:02.461850+0000","last_clean":"2023-10-06T05:17:02.461850+0000","last_became_active":"2023-10-06T05:17:02.444226+0000","last_became_peered":"2023-10-06T05:17:02.444226+0000","last_unstale":"2023-10-06T05:17:02.461850+0000","last_undegraded":"2023-10-06T05:17:02.461850+0000","last_fullsized":"2023-10-06T05:17:02.461850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452618+0000","last_change":"2023-10-06T05:17:02.447774+0000","last_active":"2023-10-06T05:17:02.452618+0000","last_peered":"2023-10-06T05:17:02.452618+0000","last_clean":"2023-10-06T05:17:02.452618+0000","last_became_active":"2023-10-06T05:17:02.447500+0000","last_became_peered":"2023-10-06T05:17:02.447500+0000","last_unstale":"2023-10-06T05:17:02.452618+0000","last_undegraded":"2023-10-06T05:17:02.452618+0000","last_fullsized":"2023-10-06T05:17:02.452618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:01.131607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462293+0000","last_change":"2023-10-06T05:17:02.441565+0000","last_active":"2023-10-06T05:17:02.462293+0000","last_peered":"2023-10-06T05:17:02.462293+0000","last_clean":"2023-10-06T05:17:02.462293+0000","last_became_active":"2023-10-06T05:17:02.441272+0000","last_became_peered":"2023-10-06T05:17:02.441272+0000","last_unstale":"2023-10-06T05:17:02.462293+0000","last_undegraded":"2023-10-06T05:17:02.462293+0000","last_fullsized":"2023-10-06T05:17:02.462293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:28.806289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453469+0000","last_change":"2023-10-06T05:17:02.443555+0000","last_active":"2023-10-06T05:17:02.453469+0000","last_peered":"2023-10-06T05:17:02.453469+0000","last_clean":"2023-10-06T05:17:02.453469+0000","last_became_active":"2023-10-06T05:17:02.443268+0000","last_became_peered":"2023-10-06T05:17:02.443268+0000","last_unstale":"2023-10-06T05:17:02.453469+0000","last_undegraded":"2023-10-06T05:17:02.453469+0000","last_fullsized":"2023-10-06T05:17:02.453469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298096,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073744,"statfs":{"total":96636764160,"available":96331513856,"internally_reserved":0,"allocated":827392,"data_stored":666304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":74184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298068,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073772,"statfs":{"total":96636764160,"available":96331542528,"internally_reserved":0,"allocated":798720,"data_stored":664552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":74184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":74808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:17:12.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:17:12.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.319+0000 7f08f6dff700 1 Processor -- start 2023-10-06T05:17:12.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.319+0000 7f08f6dff700 1 -- start start 2023-10-06T05:17:12.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.320+0000 7f08f6dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f08f0159500 0x7f08f00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.320+0000 7f08f6dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f00c3f40 con 0x7f08f015d2d0 2023-10-06T05:17:12.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.320+0000 7f08f6dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f00cb000 con 0x7f08f00cd510 2023-10-06T05:17:12.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.320+0000 7f08f6dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f00b2860 con 0x7f08f0159500 2023-10-06T05:17:12.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.320+0000 7f08effff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f08f015d2d0 0x7f08f00cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44818/0 (socket says 172.21.15.31:44818) 2023-10-06T05:17:12.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.320+0000 7f08effff700 1 -- 172.21.15.31:0/1030055557 learned_addr learned my addr 172.21.15.31:0/1030055557 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:12.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.320+0000 7f08f4b9b700 1 --2- 172.21.15.31:0/1030055557 >> v2:172.21.15.62:3300/0 conn(0x7f08f0159500 0x7f08f00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914730522 0 0) 0x7f08f00c3f40 con 0x7f08f015d2d0 2023-10-06T05:17:12.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08e0003570 con 0x7f08f015d2d0 2023-10-06T05:17:12.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515684600 0 0) 0x7f08f00cb000 con 0x7f08f00cd510 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08f00c3f40 con 0x7f08f00cd510 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2252379882 0 0) 0x7f08e0003570 con 0x7f08f015d2d0 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 >> v2:172.21.15.62:3300/0 conn(0x7f08f0159500 msgr2=0x7f08f00b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 --2- 172.21.15.31:0/1030055557 >> v2:172.21.15.62:3300/0 conn(0x7f08f0159500 0x7f08f00b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 >> v1:172.21.15.62:6789/0 conn(0x7f08f00cd510 legacy=0x7f08f00aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f015d7d0 con 0x7f08f015d2d0 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f08dc0015f0 con 0x7f08f015d2d0 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 >> v1:172.21.15.31:6789/0 conn(0x7f08f015d2d0 legacy=0x7f08f00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08e0003bf0 0x7f08e0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0004810 0x7f08e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e00051c0 0x7f08e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00b2860 con 0x7f08e00051c0 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e000c160 con 0x7f08e0004810 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e000c320 con 0x7f08e0003bf0 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.321+0000 7f08f539c700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e00051c0 0x7f08e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08effff700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0004810 0x7f08e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08f4b9b700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08e0003bf0 0x7f08e0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08f539c700 1 -- 172.21.15.31:0/1030055557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08e0003bf0 msgr2=0x7f08e0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08f539c700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08e0003bf0 0x7f08e0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08f539c700 1 -- 172.21.15.31:0/1030055557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0004810 msgr2=0x7f08e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08f539c700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0004810 0x7f08e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08f539c700 1 -- 172.21.15.31:0/1030055557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e0003570 con 0x7f08e00051c0 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08f539c700 1 -- 172.21.15.31:0/1030055557 reap_dead start 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08f539c700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e00051c0 0x7f08e000ba60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08e80020f0 tx=0x7f08e80017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7311376713b382 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8001ab0 con 0x7f08e00051c0 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e800c8b0 con 0x7f08e00051c0 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.322+0000 7f08ef7fe700 1 -- 172.21.15.31:0/1030055557 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800cc70 con 0x7f08e00051c0 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.323+0000 7f08f6dff700 1 -- 172.21.15.31:0/1030055557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e00051c0 msgr2=0x7f08e000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.323+0000 7f08f6dff700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e00051c0 0x7f08e000ba60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08e80020f0 tx=0x7f08e80017a0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.323+0000 7f08f6dff700 1 -- 172.21.15.31:0/1030055557 shutdown_connections 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.323+0000 7f08f6dff700 1 --2- 172.21.15.31:0/1030055557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e00051c0 0x7f08e000ba60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.323+0000 7f08f6dff700 1 -- 172.21.15.31:0/1030055557 >> 172.21.15.31:0/1030055557 conn(0x7f08f006ec90 msgr2=0x7f08f006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.323+0000 7f08f6dff700 1 -- 172.21.15.31:0/1030055557 shutdown_connections 2023-10-06T05:17:12.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.323+0000 7f08f6dff700 1 -- 172.21.15.31:0/1030055557 wait complete. 2023-10-06T05:17:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.324+0000 7f08f6dff700 1 Processor -- start 2023-10-06T05:17:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.325+0000 7f08f6dff700 1 -- start start 2023-10-06T05:17:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.325+0000 7f08f6dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e00051c0 0x7f08f00c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.325+0000 7f08f6dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08f00c7ee0 0x7f08f0158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.325+0000 7f08f6dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f00b3290 0x7f08f00b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.325+0000 7f08f6dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00b3c70 con 0x7f08f00b3290 2023-10-06T05:17:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.325+0000 7f08f6dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00c9bc0 con 0x7f08e00051c0 2023-10-06T05:17:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.325+0000 7f08f6dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00c83e0 con 0x7f08f00c7ee0 2023-10-06T05:17:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08f539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f00b3290 0x7f08f00b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08f539c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f00b3290 0x7f08f00b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34512/0 (socket says 172.21.15.31:34512) 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08f539c700 1 -- 172.21.15.31:0/3204979007 learned_addr learned my addr 172.21.15.31:0/3204979007 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08f4b9b700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e00051c0 0x7f08f00c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08f539c700 1 -- 172.21.15.31:0/3204979007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08f00c7ee0 msgr2=0x7f08f0158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08f539c700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08f00c7ee0 0x7f08f0158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08f539c700 1 -- 172.21.15.31:0/3204979007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e00051c0 msgr2=0x7f08f00c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08f539c700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e00051c0 0x7f08f00c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08f539c700 1 -- 172.21.15.31:0/3204979007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f00c86c0 con 0x7f08f00b3290 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08f539c700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f00b3290 0x7f08f00b3730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f08f00b3010 tx=0x7f08e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.326+0000 7f08ecff9700 1 -- 172.21.15.31:0/3204979007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800c6d0 con 0x7f08f00b3290 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.327+0000 7f08ecff9700 1 -- 172.21.15.31:0/3204979007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e8012c40 con 0x7f08f00b3290 2023-10-06T05:17:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.327+0000 7f08f6dff700 1 -- 172.21.15.31:0/3204979007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f00c89a0 con 0x7f08f00b3290 2023-10-06T05:17:12.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.327+0000 7f08ecff9700 1 -- 172.21.15.31:0/3204979007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e801e430 con 0x7f08f00b3290 2023-10-06T05:17:12.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.327+0000 7f08f6dff700 1 -- 172.21.15.31:0/3204979007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f015a450 con 0x7f08f00b3290 2023-10-06T05:17:12.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.328+0000 7f08f6dff700 1 -- 172.21.15.31:0/3204979007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f00c8e40 con 0x7f08f00b3290 2023-10-06T05:17:12.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.328+0000 7f08ecff9700 1 -- 172.21.15.31:0/3204979007 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08e801e7e0 con 0x7f08f00b3290 2023-10-06T05:17:12.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.329+0000 7f08ecff9700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f08d4066f30 0x7f08d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.329+0000 7f08ecff9700 1 -- 172.21.15.31:0/3204979007 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f08e80867f0 con 0x7f08f00b3290 2023-10-06T05:17:12.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.329+0000 7f08f4b9b700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f08d4066f30 0x7f08d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.330+0000 7f08f4b9b700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f08d4066f30 0x7f08d40693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08e4006f00 tx=0x7f08e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.334+0000 7f08ecff9700 1 -- 172.21.15.31:0/3204979007 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08e8053860 con 0x7f08f00b3290 2023-10-06T05:17:12.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.533+0000 7f08f6dff700 1 -- 172.21.15.31:0/3204979007 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08f015a640 con 0x7f08d4066f30 2023-10-06T05:17:12.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.542+0000 7f08ecff9700 1 -- 172.21.15.31:0/3204979007 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286813 (secure 0 0 0) 0x7f08f015a640 con 0x7f08d4066f30 2023-10-06T05:17:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 -- 172.21.15.31:0/3204979007 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f08d4066f30 msgr2=0x7f08d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f08d4066f30 0x7f08d40693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08e4006f00 tx=0x7f08e4008040 comp rx=0 tx=0).stop 2023-10-06T05:17:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 -- 172.21.15.31:0/3204979007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f00b3290 msgr2=0x7f08f00b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f00b3290 0x7f08f00b3730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f08f00b3010 tx=0x7f08e8016000 comp rx=0 tx=0).stop 2023-10-06T05:17:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 -- 172.21.15.31:0/3204979007 shutdown_connections 2023-10-06T05:17:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f08d4066f30 0x7f08d40693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f00b3290 0x7f08f00b3730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e00051c0 0x7f08f00c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 --2- 172.21.15.31:0/3204979007 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08f00c7ee0 0x7f08f0158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 -- 172.21.15.31:0/3204979007 >> 172.21.15.31:0/3204979007 conn(0x7f08f006ec90 msgr2=0x7f08f00cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 -- 172.21.15.31:0/3204979007 shutdown_connections 2023-10-06T05:17:12.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.546+0000 7f08f6dff700 1 -- 172.21.15.31:0/3204979007 wait complete. 2023-10-06T05:17:12.551 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:17:12.565 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T05:17:12.063046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381128,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593592,"statfs":{"total":773094113280,"available":770655838208,"internally_reserved":0,"allocated":2891776,"data_stored":1778820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.638324"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464636+0000","last_change":"2023-10-06T05:17:02.463452+0000","last_active":"2023-10-06T05:17:02.464636+0000","last_peered":"2023-10-06T05:17:02.464636+0000","last_clean":"2023-10-06T05:17:02.464636+0000","last_became_active":"2023-10-06T05:17:02.463139+0000","last_became_peered":"2023-10-06T05:17:02.463139+0000","last_unstale":"2023-10-06T05:17:02.464636+0000","last_undegraded":"2023-10-06T05:17:02.464636+0000","last_fullsized":"2023-10-06T05:17:02.464636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:52.286394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456371+0000","last_change":"2023-10-06T05:17:02.451303+0000","last_active":"2023-10-06T05:17:02.456371+0000","last_peered":"2023-10-06T05:17:02.456371+0000","last_clean":"2023-10-06T05:17:02.456371+0000","last_became_active":"2023-10-06T05:17:02.450985+0000","last_became_peered":"2023-10-06T05:17:02.450985+0000","last_unstale":"2023-10-06T05:17:02.456371+0000","last_undegraded":"2023-10-06T05:17:02.456371+0000","last_fullsized":"2023-10-06T05:17:02.456371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453158+0000","last_change":"2023-10-06T05:17:02.451335+0000","last_active":"2023-10-06T05:17:02.453158+0000","last_peered":"2023-10-06T05:17:02.453158+0000","last_clean":"2023-10-06T05:17:02.453158+0000","last_became_active":"2023-10-06T05:17:02.451119+0000","last_became_peered":"2023-10-06T05:17:02.451119+0000","last_unstale":"2023-10-06T05:17:02.453158+0000","last_undegraded":"2023-10-06T05:17:02.453158+0000","last_fullsized":"2023-10-06T05:17:02.453158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:59.439667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.458136+0000","last_change":"2023-10-06T05:17:02.454559+0000","last_active":"2023-10-06T05:17:02.458136+0000","last_peered":"2023-10-06T05:17:02.458136+0000","last_clean":"2023-10-06T05:17:02.458136+0000","last_became_active":"2023-10-06T05:17:02.453985+0000","last_became_peered":"2023-10-06T05:17:02.453985+0000","last_unstale":"2023-10-06T05:17:02.458136+0000","last_undegraded":"2023-10-06T05:17:02.458136+0000","last_fullsized":"2023-10-06T05:17:02.458136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:57.117094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465742+0000","last_change":"2023-10-06T05:17:02.460869+0000","last_active":"2023-10-06T05:17:02.465742+0000","last_peered":"2023-10-06T05:17:02.465742+0000","last_clean":"2023-10-06T05:17:02.465742+0000","last_became_active":"2023-10-06T05:17:02.460257+0000","last_became_peered":"2023-10-06T05:17:02.460257+0000","last_unstale":"2023-10-06T05:17:02.465742+0000","last_undegraded":"2023-10-06T05:17:02.465742+0000","last_fullsized":"2023-10-06T05:17:02.465742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453122+0000","last_change":"2023-10-06T05:17:02.451827+0000","last_active":"2023-10-06T05:17:02.453122+0000","last_peered":"2023-10-06T05:17:02.453122+0000","last_clean":"2023-10-06T05:17:02.453122+0000","last_became_active":"2023-10-06T05:17:02.451573+0000","last_became_peered":"2023-10-06T05:17:02.451573+0000","last_unstale":"2023-10-06T05:17:02.453122+0000","last_undegraded":"2023-10-06T05:17:02.453122+0000","last_fullsized":"2023-10-06T05:17:02.453122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:56:50.530278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.454214+0000","last_change":"2023-10-06T05:17:02.451805+0000","last_active":"2023-10-06T05:17:02.454214+0000","last_peered":"2023-10-06T05:17:02.454214+0000","last_clean":"2023-10-06T05:17:02.454214+0000","last_became_active":"2023-10-06T05:17:02.451504+0000","last_became_peered":"2023-10-06T05:17:02.451504+0000","last_unstale":"2023-10-06T05:17:02.454214+0000","last_undegraded":"2023-10-06T05:17:02.454214+0000","last_fullsized":"2023-10-06T05:17:02.454214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448446+0000","last_change":"2023-10-06T05:17:02.440866+0000","last_active":"2023-10-06T05:17:02.448446+0000","last_peered":"2023-10-06T05:17:02.448446+0000","last_clean":"2023-10-06T05:17:02.448446+0000","last_became_active":"2023-10-06T05:17:02.440548+0000","last_became_peered":"2023-10-06T05:17:02.440548+0000","last_unstale":"2023-10-06T05:17:02.448446+0000","last_undegraded":"2023-10-06T05:17:02.448446+0000","last_fullsized":"2023-10-06T05:17:02.448446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.466997+0000","last_change":"2023-10-06T05:17:02.461715+0000","last_active":"2023-10-06T05:17:02.466997+0000","last_peered":"2023-10-06T05:17:02.466997+0000","last_clean":"2023-10-06T05:17:02.466997+0000","last_became_active":"2023-10-06T05:17:02.461152+0000","last_became_peered":"2023-10-06T05:17:02.461152+0000","last_unstale":"2023-10-06T05:17:02.466997+0000","last_undegraded":"2023-10-06T05:17:02.466997+0000","last_fullsized":"2023-10-06T05:17:02.466997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453073+0000","last_change":"2023-10-06T05:17:02.448948+0000","last_active":"2023-10-06T05:17:02.453073+0000","last_peered":"2023-10-06T05:17:02.453073+0000","last_clean":"2023-10-06T05:17:02.453073+0000","last_became_active":"2023-10-06T05:17:02.448524+0000","last_became_peered":"2023-10-06T05:17:02.448524+0000","last_unstale":"2023-10-06T05:17:02.453073+0000","last_undegraded":"2023-10-06T05:17:02.453073+0000","last_fullsized":"2023-10-06T05:17:02.453073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452570+0000","last_change":"2023-10-06T05:17:02.446198+0000","last_active":"2023-10-06T05:17:02.452570+0000","last_peered":"2023-10-06T05:17:02.452570+0000","last_clean":"2023-10-06T05:17:02.452570+0000","last_became_active":"2023-10-06T05:17:02.445918+0000","last_became_peered":"2023-10-06T05:17:02.445918+0000","last_unstale":"2023-10-06T05:17:02.452570+0000","last_undegraded":"2023-10-06T05:17:02.452570+0000","last_fullsized":"2023-10-06T05:17:02.452570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:45.361281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450359+0000","last_change":"2023-10-06T05:17:02.447094+0000","last_active":"2023-10-06T05:17:02.450359+0000","last_peered":"2023-10-06T05:17:02.450359+0000","last_clean":"2023-10-06T05:17:02.450359+0000","last_became_active":"2023-10-06T05:17:02.446800+0000","last_became_peered":"2023-10-06T05:17:02.446800+0000","last_unstale":"2023-10-06T05:17:02.450359+0000","last_undegraded":"2023-10-06T05:17:02.450359+0000","last_fullsized":"2023-10-06T05:17:02.450359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:46.452504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464672+0000","last_change":"2023-10-06T05:17:02.463414+0000","last_active":"2023-10-06T05:17:02.464672+0000","last_peered":"2023-10-06T05:17:02.464672+0000","last_clean":"2023-10-06T05:17:02.464672+0000","last_became_active":"2023-10-06T05:17:02.463125+0000","last_became_peered":"2023-10-06T05:17:02.463125+0000","last_unstale":"2023-10-06T05:17:02.464672+0000","last_undegraded":"2023-10-06T05:17:02.464672+0000","last_fullsized":"2023-10-06T05:17:02.464672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:29.979974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450580+0000","last_change":"2023-10-06T05:17:02.447113+0000","last_active":"2023-10-06T05:17:02.450580+0000","last_peered":"2023-10-06T05:17:02.450580+0000","last_clean":"2023-10-06T05:17:02.450580+0000","last_became_active":"2023-10-06T05:17:02.446878+0000","last_became_peered":"2023-10-06T05:17:02.446878+0000","last_unstale":"2023-10-06T05:17:02.450580+0000","last_undegraded":"2023-10-06T05:17:02.450580+0000","last_fullsized":"2023-10-06T05:17:02.450580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.454233+0000","last_change":"2023-10-06T05:17:02.451949+0000","last_active":"2023-10-06T05:17:02.454233+0000","last_peered":"2023-10-06T05:17:02.454233+0000","last_clean":"2023-10-06T05:17:02.454233+0000","last_became_active":"2023-10-06T05:17:02.451640+0000","last_became_peered":"2023-10-06T05:17:02.451640+0000","last_unstale":"2023-10-06T05:17:02.454233+0000","last_undegraded":"2023-10-06T05:17:02.454233+0000","last_fullsized":"2023-10-06T05:17:02.454233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452479+0000","last_change":"2023-10-06T05:17:02.445441+0000","last_active":"2023-10-06T05:17:02.452479+0000","last_peered":"2023-10-06T05:17:02.452479+0000","last_clean":"2023-10-06T05:17:02.452479+0000","last_became_active":"2023-10-06T05:17:02.445132+0000","last_became_peered":"2023-10-06T05:17:02.445132+0000","last_unstale":"2023-10-06T05:17:02.452479+0000","last_undegraded":"2023-10-06T05:17:02.452479+0000","last_fullsized":"2023-10-06T05:17:02.452479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:10.259711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464487+0000","last_change":"2023-10-06T05:17:02.461554+0000","last_active":"2023-10-06T05:17:02.464487+0000","last_peered":"2023-10-06T05:17:02.464487+0000","last_clean":"2023-10-06T05:17:02.464487+0000","last_became_active":"2023-10-06T05:17:02.460803+0000","last_became_peered":"2023-10-06T05:17:02.460803+0000","last_unstale":"2023-10-06T05:17:02.464487+0000","last_undegraded":"2023-10-06T05:17:02.464487+0000","last_fullsized":"2023-10-06T05:17:02.464487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452431+0000","last_change":"2023-10-06T05:17:02.448249+0000","last_active":"2023-10-06T05:17:02.452431+0000","last_peered":"2023-10-06T05:17:02.452431+0000","last_clean":"2023-10-06T05:17:02.452431+0000","last_became_active":"2023-10-06T05:17:02.447977+0000","last_became_peered":"2023-10-06T05:17:02.447977+0000","last_unstale":"2023-10-06T05:17:02.452431+0000","last_undegraded":"2023-10-06T05:17:02.452431+0000","last_fullsized":"2023-10-06T05:17:02.452431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:10.643350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456974+0000","last_change":"2023-10-06T05:17:02.451653+0000","last_active":"2023-10-06T05:17:02.456974+0000","last_peered":"2023-10-06T05:17:02.456974+0000","last_clean":"2023-10-06T05:17:02.456974+0000","last_became_active":"2023-10-06T05:17:02.451347+0000","last_became_peered":"2023-10-06T05:17:02.451347+0000","last_unstale":"2023-10-06T05:17:02.456974+0000","last_undegraded":"2023-10-06T05:17:02.456974+0000","last_fullsized":"2023-10-06T05:17:02.456974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:20.132880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452508+0000","last_change":"2023-10-06T05:17:02.446775+0000","last_active":"2023-10-06T05:17:02.452508+0000","last_peered":"2023-10-06T05:17:02.452508+0000","last_clean":"2023-10-06T05:17:02.452508+0000","last_became_active":"2023-10-06T05:17:02.446482+0000","last_became_peered":"2023-10-06T05:17:02.446482+0000","last_unstale":"2023-10-06T05:17:02.452508+0000","last_undegraded":"2023-10-06T05:17:02.452508+0000","last_fullsized":"2023-10-06T05:17:02.452508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:14.399779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468275+0000","last_change":"2023-10-06T05:17:02.463237+0000","last_active":"2023-10-06T05:17:02.468275+0000","last_peered":"2023-10-06T05:17:02.468275+0000","last_clean":"2023-10-06T05:17:02.468275+0000","last_became_active":"2023-10-06T05:17:02.462891+0000","last_became_peered":"2023-10-06T05:17:02.462891+0000","last_unstale":"2023-10-06T05:17:02.468275+0000","last_undegraded":"2023-10-06T05:17:02.468275+0000","last_fullsized":"2023-10-06T05:17:02.468275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462403+0000","last_change":"2023-10-06T05:17:02.458157+0000","last_active":"2023-10-06T05:17:02.462403+0000","last_peered":"2023-10-06T05:17:02.462403+0000","last_clean":"2023-10-06T05:17:02.462403+0000","last_became_active":"2023-10-06T05:17:02.455535+0000","last_became_peered":"2023-10-06T05:17:02.455535+0000","last_unstale":"2023-10-06T05:17:02.462403+0000","last_undegraded":"2023-10-06T05:17:02.462403+0000","last_fullsized":"2023-10-06T05:17:02.462403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461887+0000","last_change":"2023-10-06T05:17:02.449068+0000","last_active":"2023-10-06T05:17:02.461887+0000","last_peered":"2023-10-06T05:17:02.461887+0000","last_clean":"2023-10-06T05:17:02.461887+0000","last_became_active":"2023-10-06T05:17:02.448751+0000","last_became_peered":"2023-10-06T05:17:02.448751+0000","last_unstale":"2023-10-06T05:17:02.461887+0000","last_undegraded":"2023-10-06T05:17:02.461887+0000","last_fullsized":"2023-10-06T05:17:02.461887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468443+0000","last_change":"2023-10-06T05:17:02.465489+0000","last_active":"2023-10-06T05:17:02.468443+0000","last_peered":"2023-10-06T05:17:02.468443+0000","last_clean":"2023-10-06T05:17:02.468443+0000","last_became_active":"2023-10-06T05:17:02.465329+0000","last_became_peered":"2023-10-06T05:17:02.465329+0000","last_unstale":"2023-10-06T05:17:02.468443+0000","last_undegraded":"2023-10-06T05:17:02.468443+0000","last_fullsized":"2023-10-06T05:17:02.468443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453337+0000","last_change":"2023-10-06T05:17:02.444229+0000","last_active":"2023-10-06T05:17:02.453337+0000","last_peered":"2023-10-06T05:17:02.453337+0000","last_clean":"2023-10-06T05:17:02.453337+0000","last_became_active":"2023-10-06T05:17:02.444064+0000","last_became_peered":"2023-10-06T05:17:02.444064+0000","last_unstale":"2023-10-06T05:17:02.453337+0000","last_undegraded":"2023-10-06T05:17:02.453337+0000","last_fullsized":"2023-10-06T05:17:02.453337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:39.861351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453265+0000","last_change":"2023-10-06T05:17:02.447686+0000","last_active":"2023-10-06T05:17:02.453265+0000","last_peered":"2023-10-06T05:17:02.453265+0000","last_clean":"2023-10-06T05:17:02.453265+0000","last_became_active":"2023-10-06T05:17:02.447364+0000","last_became_peered":"2023-10-06T05:17:02.447364+0000","last_unstale":"2023-10-06T05:17:02.453265+0000","last_undegraded":"2023-10-06T05:17:02.453265+0000","last_fullsized":"2023-10-06T05:17:02.453265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448537+0000","last_change":"2023-10-06T05:17:02.442095+0000","last_active":"2023-10-06T05:17:02.448537+0000","last_peered":"2023-10-06T05:17:02.448537+0000","last_clean":"2023-10-06T05:17:02.448537+0000","last_became_active":"2023-10-06T05:17:02.441645+0000","last_became_peered":"2023-10-06T05:17:02.441645+0000","last_unstale":"2023-10-06T05:17:02.448537+0000","last_undegraded":"2023-10-06T05:17:02.448537+0000","last_fullsized":"2023-10-06T05:17:02.448537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461441+0000","last_change":"2023-10-06T05:17:01.449274+0000","last_active":"2023-10-06T05:17:02.461441+0000","last_peered":"2023-10-06T05:17:02.461441+0000","last_clean":"2023-10-06T05:17:02.461441+0000","last_became_active":"2023-10-06T05:17:01.448908+0000","last_became_peered":"2023-10-06T05:17:01.448908+0000","last_unstale":"2023-10-06T05:17:02.461441+0000","last_undegraded":"2023-10-06T05:17:02.461441+0000","last_fullsized":"2023-10-06T05:17:02.461441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462280+0000","last_change":"2023-10-06T05:17:02.443949+0000","last_active":"2023-10-06T05:17:02.462280+0000","last_peered":"2023-10-06T05:17:02.462280+0000","last_clean":"2023-10-06T05:17:02.462280+0000","last_became_active":"2023-10-06T05:17:02.443677+0000","last_became_peered":"2023-10-06T05:17:02.443677+0000","last_unstale":"2023-10-06T05:17:02.462280+0000","last_undegraded":"2023-10-06T05:17:02.462280+0000","last_fullsized":"2023-10-06T05:17:02.462280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428951+0000","last_change":"2023-10-06T05:17:01.438479+0000","last_active":"2023-10-06T05:17:02.428951+0000","last_peered":"2023-10-06T05:17:02.428951+0000","last_clean":"2023-10-06T05:17:02.428951+0000","last_became_active":"2023-10-06T05:17:01.438182+0000","last_became_peered":"2023-10-06T05:17:01.438182+0000","last_unstale":"2023-10-06T05:17:02.428951+0000","last_undegraded":"2023-10-06T05:17:02.428951+0000","last_fullsized":"2023-10-06T05:17:02.428951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:42.243614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453470+0000","last_change":"2023-10-06T05:17:02.447288+0000","last_active":"2023-10-06T05:17:02.453470+0000","last_peered":"2023-10-06T05:17:02.453470+0000","last_clean":"2023-10-06T05:17:02.453470+0000","last_became_active":"2023-10-06T05:17:02.446914+0000","last_became_peered":"2023-10-06T05:17:02.446914+0000","last_unstale":"2023-10-06T05:17:02.453470+0000","last_undegraded":"2023-10-06T05:17:02.453470+0000","last_fullsized":"2023-10-06T05:17:02.453470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428771+0000","last_change":"2023-10-06T05:17:01.444309+0000","last_active":"2023-10-06T05:17:02.428771+0000","last_peered":"2023-10-06T05:17:02.428771+0000","last_clean":"2023-10-06T05:17:02.428771+0000","last_became_active":"2023-10-06T05:17:01.444026+0000","last_became_peered":"2023-10-06T05:17:01.444026+0000","last_unstale":"2023-10-06T05:17:02.428771+0000","last_undegraded":"2023-10-06T05:17:02.428771+0000","last_fullsized":"2023-10-06T05:17:02.428771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.458208+0000","last_change":"2023-10-06T05:17:02.455822+0000","last_active":"2023-10-06T05:17:02.458208+0000","last_peered":"2023-10-06T05:17:02.458208+0000","last_clean":"2023-10-06T05:17:02.458208+0000","last_became_active":"2023-10-06T05:17:02.455539+0000","last_became_peered":"2023-10-06T05:17:02.455539+0000","last_unstale":"2023-10-06T05:17:02.458208+0000","last_undegraded":"2023-10-06T05:17:02.458208+0000","last_fullsized":"2023-10-06T05:17:02.458208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434617+0000","last_change":"2023-10-06T05:17:01.453079+0000","last_active":"2023-10-06T05:17:02.434617+0000","last_peered":"2023-10-06T05:17:02.434617+0000","last_clean":"2023-10-06T05:17:02.434617+0000","last_became_active":"2023-10-06T05:17:01.452800+0000","last_became_peered":"2023-10-06T05:17:01.452800+0000","last_unstale":"2023-10-06T05:17:02.434617+0000","last_undegraded":"2023-10-06T05:17:02.434617+0000","last_fullsized":"2023-10-06T05:17:02.434617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.457891+0000","last_change":"2023-10-06T05:17:02.439162+0000","last_active":"2023-10-06T05:17:02.457891+0000","last_peered":"2023-10-06T05:17:02.457891+0000","last_clean":"2023-10-06T05:17:02.457891+0000","last_became_active":"2023-10-06T05:17:02.438810+0000","last_became_peered":"2023-10-06T05:17:02.438810+0000","last_unstale":"2023-10-06T05:17:02.457891+0000","last_undegraded":"2023-10-06T05:17:02.457891+0000","last_fullsized":"2023-10-06T05:17:02.457891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435781+0000","last_change":"2023-10-06T05:17:01.449427+0000","last_active":"2023-10-06T05:17:02.435781+0000","last_peered":"2023-10-06T05:17:02.435781+0000","last_clean":"2023-10-06T05:17:02.435781+0000","last_became_active":"2023-10-06T05:17:01.449100+0000","last_became_peered":"2023-10-06T05:17:01.449100+0000","last_unstale":"2023-10-06T05:17:02.435781+0000","last_undegraded":"2023-10-06T05:17:02.435781+0000","last_fullsized":"2023-10-06T05:17:02.435781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461821+0000","last_change":"2023-10-06T05:17:02.441035+0000","last_active":"2023-10-06T05:17:02.461821+0000","last_peered":"2023-10-06T05:17:02.461821+0000","last_clean":"2023-10-06T05:17:02.461821+0000","last_became_active":"2023-10-06T05:17:02.440422+0000","last_became_peered":"2023-10-06T05:17:02.440422+0000","last_unstale":"2023-10-06T05:17:02.461821+0000","last_undegraded":"2023-10-06T05:17:02.461821+0000","last_fullsized":"2023-10-06T05:17:02.461821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.468652+0000","last_change":"2023-10-06T05:17:01.445468+0000","last_active":"2023-10-06T05:17:02.468652+0000","last_peered":"2023-10-06T05:17:02.468652+0000","last_clean":"2023-10-06T05:17:02.468652+0000","last_became_active":"2023-10-06T05:17:01.445179+0000","last_became_peered":"2023-10-06T05:17:01.445179+0000","last_unstale":"2023-10-06T05:17:02.468652+0000","last_undegraded":"2023-10-06T05:17:02.468652+0000","last_fullsized":"2023-10-06T05:17:02.468652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:01.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453479+0000","last_change":"2023-10-06T05:17:02.444872+0000","last_active":"2023-10-06T05:17:02.453479+0000","last_peered":"2023-10-06T05:17:02.453479+0000","last_clean":"2023-10-06T05:17:02.453479+0000","last_became_active":"2023-10-06T05:17:02.444565+0000","last_became_peered":"2023-10-06T05:17:02.444565+0000","last_unstale":"2023-10-06T05:17:02.453479+0000","last_undegraded":"2023-10-06T05:17:02.453479+0000","last_fullsized":"2023-10-06T05:17:02.453479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.470982+0000","last_change":"2023-10-06T05:17:01.448641+0000","last_active":"2023-10-06T05:17:02.470982+0000","last_peered":"2023-10-06T05:17:02.470982+0000","last_clean":"2023-10-06T05:17:02.470982+0000","last_became_active":"2023-10-06T05:17:01.448232+0000","last_became_peered":"2023-10-06T05:17:01.448232+0000","last_unstale":"2023-10-06T05:17:02.470982+0000","last_undegraded":"2023-10-06T05:17:02.470982+0000","last_fullsized":"2023-10-06T05:17:02.470982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447551+0000","last_change":"2023-10-06T05:17:02.443125+0000","last_active":"2023-10-06T05:17:02.447551+0000","last_peered":"2023-10-06T05:17:02.447551+0000","last_clean":"2023-10-06T05:17:02.447551+0000","last_became_active":"2023-10-06T05:17:02.442847+0000","last_became_peered":"2023-10-06T05:17:02.442847+0000","last_unstale":"2023-10-06T05:17:02.447551+0000","last_undegraded":"2023-10-06T05:17:02.447551+0000","last_fullsized":"2023-10-06T05:17:02.447551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:35.283660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471314+0000","last_change":"2023-10-06T05:17:01.451184+0000","last_active":"2023-10-06T05:17:02.471314+0000","last_peered":"2023-10-06T05:17:02.471314+0000","last_clean":"2023-10-06T05:17:02.471314+0000","last_became_active":"2023-10-06T05:17:01.450521+0000","last_became_peered":"2023-10-06T05:17:01.450521+0000","last_unstale":"2023-10-06T05:17:02.471314+0000","last_undegraded":"2023-10-06T05:17:02.471314+0000","last_fullsized":"2023-10-06T05:17:02.471314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:56.109170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462307+0000","last_change":"2023-10-06T05:17:02.444967+0000","last_active":"2023-10-06T05:17:02.462307+0000","last_peered":"2023-10-06T05:17:02.462307+0000","last_clean":"2023-10-06T05:17:02.462307+0000","last_became_active":"2023-10-06T05:17:02.444639+0000","last_became_peered":"2023-10-06T05:17:02.444639+0000","last_unstale":"2023-10-06T05:17:02.462307+0000","last_undegraded":"2023-10-06T05:17:02.462307+0000","last_fullsized":"2023-10-06T05:17:02.462307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:59.439667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.439263+0000","last_change":"2023-10-06T05:17:01.446625+0000","last_active":"2023-10-06T05:17:02.439263+0000","last_peered":"2023-10-06T05:17:02.439263+0000","last_clean":"2023-10-06T05:17:02.439263+0000","last_became_active":"2023-10-06T05:17:01.446249+0000","last_became_peered":"2023-10-06T05:17:01.446249+0000","last_unstale":"2023-10-06T05:17:02.439263+0000","last_undegraded":"2023-10-06T05:17:02.439263+0000","last_fullsized":"2023-10-06T05:17:02.439263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448517+0000","last_change":"2023-10-06T05:17:02.441887+0000","last_active":"2023-10-06T05:17:02.448517+0000","last_peered":"2023-10-06T05:17:02.448517+0000","last_clean":"2023-10-06T05:17:02.448517+0000","last_became_active":"2023-10-06T05:17:02.441377+0000","last_became_peered":"2023-10-06T05:17:02.441377+0000","last_unstale":"2023-10-06T05:17:02.448517+0000","last_undegraded":"2023-10-06T05:17:02.448517+0000","last_fullsized":"2023-10-06T05:17:02.448517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:36.448276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434243+0000","last_change":"2023-10-06T05:17:01.450424+0000","last_active":"2023-10-06T05:17:02.434243+0000","last_peered":"2023-10-06T05:17:02.434243+0000","last_clean":"2023-10-06T05:17:02.434243+0000","last_became_active":"2023-10-06T05:17:01.450072+0000","last_became_peered":"2023-10-06T05:17:01.450072+0000","last_unstale":"2023-10-06T05:17:02.434243+0000","last_undegraded":"2023-10-06T05:17:02.434243+0000","last_fullsized":"2023-10-06T05:17:02.434243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:19.933200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464457+0000","last_change":"2023-10-06T05:17:02.460678+0000","last_active":"2023-10-06T05:17:02.464457+0000","last_peered":"2023-10-06T05:17:02.464457+0000","last_clean":"2023-10-06T05:17:02.464457+0000","last_became_active":"2023-10-06T05:17:02.460159+0000","last_became_peered":"2023-10-06T05:17:02.460159+0000","last_unstale":"2023-10-06T05:17:02.464457+0000","last_undegraded":"2023-10-06T05:17:02.464457+0000","last_fullsized":"2023-10-06T05:17:02.464457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.467405+0000","last_change":"2023-10-06T05:17:01.447124+0000","last_active":"2023-10-06T05:17:02.467405+0000","last_peered":"2023-10-06T05:17:02.467405+0000","last_clean":"2023-10-06T05:17:02.467405+0000","last_became_active":"2023-10-06T05:17:01.446810+0000","last_became_peered":"2023-10-06T05:17:01.446810+0000","last_unstale":"2023-10-06T05:17:02.467405+0000","last_undegraded":"2023-10-06T05:17:02.467405+0000","last_fullsized":"2023-10-06T05:17:02.467405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462426+0000","last_change":"2023-10-06T05:17:02.440973+0000","last_active":"2023-10-06T05:17:02.462426+0000","last_peered":"2023-10-06T05:17:02.462426+0000","last_clean":"2023-10-06T05:17:02.462426+0000","last_became_active":"2023-10-06T05:17:02.440594+0000","last_became_peered":"2023-10-06T05:17:02.440594+0000","last_unstale":"2023-10-06T05:17:02.462426+0000","last_undegraded":"2023-10-06T05:17:02.462426+0000","last_fullsized":"2023-10-06T05:17:02.462426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:45.361281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471211+0000","last_change":"2023-10-06T05:17:01.454983+0000","last_active":"2023-10-06T05:17:02.471211+0000","last_peered":"2023-10-06T05:17:02.471211+0000","last_clean":"2023-10-06T05:17:02.471211+0000","last_became_active":"2023-10-06T05:17:01.454580+0000","last_became_peered":"2023-10-06T05:17:01.454580+0000","last_unstale":"2023-10-06T05:17:02.471211+0000","last_undegraded":"2023-10-06T05:17:02.471211+0000","last_fullsized":"2023-10-06T05:17:02.471211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:53.875873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.448468+0000","last_change":"2023-10-06T05:17:02.442909+0000","last_active":"2023-10-06T05:17:02.448468+0000","last_peered":"2023-10-06T05:17:02.448468+0000","last_clean":"2023-10-06T05:17:02.448468+0000","last_became_active":"2023-10-06T05:17:02.442619+0000","last_became_peered":"2023-10-06T05:17:02.442619+0000","last_unstale":"2023-10-06T05:17:02.448468+0000","last_undegraded":"2023-10-06T05:17:02.448468+0000","last_fullsized":"2023-10-06T05:17:02.448468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471248+0000","last_change":"2023-10-06T05:17:01.454784+0000","last_active":"2023-10-06T05:17:02.471248+0000","last_peered":"2023-10-06T05:17:02.471248+0000","last_clean":"2023-10-06T05:17:02.471248+0000","last_became_active":"2023-10-06T05:17:01.454498+0000","last_became_peered":"2023-10-06T05:17:01.454498+0000","last_unstale":"2023-10-06T05:17:02.471248+0000","last_undegraded":"2023-10-06T05:17:02.471248+0000","last_fullsized":"2023-10-06T05:17:02.471248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:41.021745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452621+0000","last_change":"2023-10-06T05:17:02.446992+0000","last_active":"2023-10-06T05:17:02.452621+0000","last_peered":"2023-10-06T05:17:02.452621+0000","last_clean":"2023-10-06T05:17:02.452621+0000","last_became_active":"2023-10-06T05:17:02.445148+0000","last_became_peered":"2023-10-06T05:17:02.445148+0000","last_unstale":"2023-10-06T05:17:02.452621+0000","last_undegraded":"2023-10-06T05:17:02.452621+0000","last_fullsized":"2023-10-06T05:17:02.452621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:31.781450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.469131+0000","last_change":"2023-10-06T05:17:01.446494+0000","last_active":"2023-10-06T05:17:02.469131+0000","last_peered":"2023-10-06T05:17:02.469131+0000","last_clean":"2023-10-06T05:17:02.469131+0000","last_became_active":"2023-10-06T05:17:01.446164+0000","last_became_peered":"2023-10-06T05:17:01.446164+0000","last_unstale":"2023-10-06T05:17:02.469131+0000","last_undegraded":"2023-10-06T05:17:02.469131+0000","last_fullsized":"2023-10-06T05:17:02.469131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:45.078207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456419+0000","last_change":"2023-10-06T05:17:02.451322+0000","last_active":"2023-10-06T05:17:02.456419+0000","last_peered":"2023-10-06T05:17:02.456419+0000","last_clean":"2023-10-06T05:17:02.456419+0000","last_became_active":"2023-10-06T05:17:02.451064+0000","last_became_peered":"2023-10-06T05:17:02.451064+0000","last_unstale":"2023-10-06T05:17:02.456419+0000","last_undegraded":"2023-10-06T05:17:02.456419+0000","last_fullsized":"2023-10-06T05:17:02.456419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:46.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.432653+0000","last_change":"2023-10-06T05:17:01.443417+0000","last_active":"2023-10-06T05:17:02.432653+0000","last_peered":"2023-10-06T05:17:02.432653+0000","last_clean":"2023-10-06T05:17:02.432653+0000","last_became_active":"2023-10-06T05:17:01.443114+0000","last_became_peered":"2023-10-06T05:17:01.443114+0000","last_unstale":"2023-10-06T05:17:02.432653+0000","last_undegraded":"2023-10-06T05:17:02.432653+0000","last_fullsized":"2023-10-06T05:17:02.432653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461876+0000","last_change":"2023-10-06T05:17:02.456471+0000","last_active":"2023-10-06T05:17:02.461876+0000","last_peered":"2023-10-06T05:17:02.461876+0000","last_clean":"2023-10-06T05:17:02.461876+0000","last_became_active":"2023-10-06T05:17:02.456142+0000","last_became_peered":"2023-10-06T05:17:02.456142+0000","last_unstale":"2023-10-06T05:17:02.461876+0000","last_undegraded":"2023-10-06T05:17:02.461876+0000","last_fullsized":"2023-10-06T05:17:02.461876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.466429+0000","last_change":"2023-10-06T05:17:01.444030+0000","last_active":"2023-10-06T05:17:02.466429+0000","last_peered":"2023-10-06T05:17:02.466429+0000","last_clean":"2023-10-06T05:17:02.466429+0000","last_became_active":"2023-10-06T05:17:01.443754+0000","last_became_peered":"2023-10-06T05:17:01.443754+0000","last_unstale":"2023-10-06T05:17:02.466429+0000","last_undegraded":"2023-10-06T05:17:02.466429+0000","last_fullsized":"2023-10-06T05:17:02.466429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:45.444580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.440368+0000","last_change":"2023-10-06T05:17:02.437195+0000","last_active":"2023-10-06T05:17:02.440368+0000","last_peered":"2023-10-06T05:17:02.440368+0000","last_clean":"2023-10-06T05:17:02.440368+0000","last_became_active":"2023-10-06T05:17:02.436917+0000","last_became_peered":"2023-10-06T05:17:02.436917+0000","last_unstale":"2023-10-06T05:17:02.440368+0000","last_undegraded":"2023-10-06T05:17:02.440368+0000","last_fullsized":"2023-10-06T05:17:02.440368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435134+0000","last_change":"2023-10-06T05:17:01.454676+0000","last_active":"2023-10-06T05:17:02.435134+0000","last_peered":"2023-10-06T05:17:02.435134+0000","last_clean":"2023-10-06T05:17:02.435134+0000","last_became_active":"2023-10-06T05:17:01.454385+0000","last_became_peered":"2023-10-06T05:17:01.454385+0000","last_unstale":"2023-10-06T05:17:02.435134+0000","last_undegraded":"2023-10-06T05:17:02.435134+0000","last_fullsized":"2023-10-06T05:17:02.435134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453140+0000","last_change":"2023-10-06T05:17:02.449388+0000","last_active":"2023-10-06T05:17:02.453140+0000","last_peered":"2023-10-06T05:17:02.453140+0000","last_clean":"2023-10-06T05:17:02.453140+0000","last_became_active":"2023-10-06T05:17:02.448868+0000","last_became_peered":"2023-10-06T05:17:02.448868+0000","last_unstale":"2023-10-06T05:17:02.453140+0000","last_undegraded":"2023-10-06T05:17:02.453140+0000","last_fullsized":"2023-10-06T05:17:02.453140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:28.371670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428450+0000","last_change":"2023-10-06T05:17:01.448698+0000","last_active":"2023-10-06T05:17:02.428450+0000","last_peered":"2023-10-06T05:17:02.428450+0000","last_clean":"2023-10-06T05:17:02.428450+0000","last_became_active":"2023-10-06T05:17:01.448084+0000","last_became_peered":"2023-10-06T05:17:01.448084+0000","last_unstale":"2023-10-06T05:17:02.428450+0000","last_undegraded":"2023-10-06T05:17:02.428450+0000","last_fullsized":"2023-10-06T05:17:02.428450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452412+0000","last_change":"2023-10-06T05:17:02.446522+0000","last_active":"2023-10-06T05:17:02.452412+0000","last_peered":"2023-10-06T05:17:02.452412+0000","last_clean":"2023-10-06T05:17:02.452412+0000","last_became_active":"2023-10-06T05:17:02.446203+0000","last_became_peered":"2023-10-06T05:17:02.446203+0000","last_unstale":"2023-10-06T05:17:02.452412+0000","last_undegraded":"2023-10-06T05:17:02.452412+0000","last_fullsized":"2023-10-06T05:17:02.452412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:28.806289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428705+0000","last_change":"2023-10-06T05:17:01.441195+0000","last_active":"2023-10-06T05:17:02.428705+0000","last_peered":"2023-10-06T05:17:02.428705+0000","last_clean":"2023-10-06T05:17:02.428705+0000","last_became_active":"2023-10-06T05:17:01.440903+0000","last_became_peered":"2023-10-06T05:17:01.440903+0000","last_unstale":"2023-10-06T05:17:02.428705+0000","last_undegraded":"2023-10-06T05:17:02.428705+0000","last_fullsized":"2023-10-06T05:17:02.428705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:19.124956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453318+0000","last_change":"2023-10-06T05:17:02.444240+0000","last_active":"2023-10-06T05:17:02.453318+0000","last_peered":"2023-10-06T05:17:02.453318+0000","last_clean":"2023-10-06T05:17:02.453318+0000","last_became_active":"2023-10-06T05:17:02.443717+0000","last_became_peered":"2023-10-06T05:17:02.443717+0000","last_unstale":"2023-10-06T05:17:02.453318+0000","last_undegraded":"2023-10-06T05:17:02.453318+0000","last_fullsized":"2023-10-06T05:17:02.453318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:50.695132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464271+0000","last_change":"2023-10-06T05:17:01.443363+0000","last_active":"2023-10-06T05:17:02.464271+0000","last_peered":"2023-10-06T05:17:02.464271+0000","last_clean":"2023-10-06T05:17:02.464271+0000","last_became_active":"2023-10-06T05:17:01.443131+0000","last_became_peered":"2023-10-06T05:17:01.443131+0000","last_unstale":"2023-10-06T05:17:02.464271+0000","last_undegraded":"2023-10-06T05:17:02.464271+0000","last_fullsized":"2023-10-06T05:17:02.464271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.428924+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:17:02.428924+0000","last_peered":"2023-10-06T05:17:02.428924+0000","last_clean":"2023-10-06T05:17:02.428924+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:17:02.428924+0000","last_undegraded":"2023-10-06T05:17:02.428924+0000","last_fullsized":"2023-10-06T05:17:02.428924+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.464122+0000","last_change":"2023-10-06T05:17:02.442449+0000","last_active":"2023-10-06T05:17:02.464122+0000","last_peered":"2023-10-06T05:17:02.464122+0000","last_clean":"2023-10-06T05:17:02.464122+0000","last_became_active":"2023-10-06T05:17:02.442135+0000","last_became_peered":"2023-10-06T05:17:02.442135+0000","last_unstale":"2023-10-06T05:17:02.464122+0000","last_undegraded":"2023-10-06T05:17:02.464122+0000","last_fullsized":"2023-10-06T05:17:02.464122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:31.781450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.427977+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:17:02.427977+0000","last_peered":"2023-10-06T05:17:02.427977+0000","last_clean":"2023-10-06T05:17:02.427977+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:17:02.427977+0000","last_undegraded":"2023-10-06T05:17:02.427977+0000","last_fullsized":"2023-10-06T05:17:02.427977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T05:17:04.091912+0000","last_change":"2023-10-06T05:17:01.448082+0000","last_active":"2023-10-06T05:17:04.091912+0000","last_peered":"2023-10-06T05:17:04.091912+0000","last_clean":"2023-10-06T05:17:04.091912+0000","last_became_active":"2023-10-06T05:17:01.447418+0000","last_became_peered":"2023-10-06T05:17:01.447418+0000","last_unstale":"2023-10-06T05:17:04.091912+0000","last_undegraded":"2023-10-06T05:17:04.091912+0000","last_fullsized":"2023-10-06T05:17:04.091912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452602+0000","last_change":"2023-10-06T05:17:02.445401+0000","last_active":"2023-10-06T05:17:02.452602+0000","last_peered":"2023-10-06T05:17:02.452602+0000","last_clean":"2023-10-06T05:17:02.452602+0000","last_became_active":"2023-10-06T05:17:02.444770+0000","last_became_peered":"2023-10-06T05:17:02.444770+0000","last_unstale":"2023-10-06T05:17:02.452602+0000","last_undegraded":"2023-10-06T05:17:02.452602+0000","last_fullsized":"2023-10-06T05:17:02.452602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429051+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:17:02.429051+0000","last_peered":"2023-10-06T05:17:02.429051+0000","last_clean":"2023-10-06T05:17:02.429051+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:17:02.429051+0000","last_undegraded":"2023-10-06T05:17:02.429051+0000","last_fullsized":"2023-10-06T05:17:02.429051+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.436179+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:17:02.436179+0000","last_peered":"2023-10-06T05:17:02.436179+0000","last_clean":"2023-10-06T05:17:02.436179+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:17:02.436179+0000","last_undegraded":"2023-10-06T05:17:02.436179+0000","last_fullsized":"2023-10-06T05:17:02.436179+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.432715+0000","last_change":"2023-10-06T05:17:01.447882+0000","last_active":"2023-10-06T05:17:02.432715+0000","last_peered":"2023-10-06T05:17:02.432715+0000","last_clean":"2023-10-06T05:17:02.432715+0000","last_became_active":"2023-10-06T05:17:01.447617+0000","last_became_peered":"2023-10-06T05:17:01.447617+0000","last_unstale":"2023-10-06T05:17:02.432715+0000","last_undegraded":"2023-10-06T05:17:02.432715+0000","last_fullsized":"2023-10-06T05:17:02.432715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.460775+0000","last_change":"2023-10-06T05:17:02.455716+0000","last_active":"2023-10-06T05:17:02.460775+0000","last_peered":"2023-10-06T05:17:02.460775+0000","last_clean":"2023-10-06T05:17:02.460775+0000","last_became_active":"2023-10-06T05:17:02.455388+0000","last_became_peered":"2023-10-06T05:17:02.455388+0000","last_unstale":"2023-10-06T05:17:02.460775+0000","last_undegraded":"2023-10-06T05:17:02.460775+0000","last_fullsized":"2023-10-06T05:17:02.460775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:26.713943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.434749+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:17:02.434749+0000","last_peered":"2023-10-06T05:17:02.434749+0000","last_clean":"2023-10-06T05:17:02.434749+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:17:02.434749+0000","last_undegraded":"2023-10-06T05:17:02.434749+0000","last_fullsized":"2023-10-06T05:17:02.434749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429080+0000","last_change":"2023-10-06T05:17:01.449606+0000","last_active":"2023-10-06T05:17:02.429080+0000","last_peered":"2023-10-06T05:17:02.429080+0000","last_clean":"2023-10-06T05:17:02.429080+0000","last_became_active":"2023-10-06T05:17:01.449380+0000","last_became_peered":"2023-10-06T05:17:01.449380+0000","last_unstale":"2023-10-06T05:17:02.429080+0000","last_undegraded":"2023-10-06T05:17:02.429080+0000","last_fullsized":"2023-10-06T05:17:02.429080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:38.884815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453300+0000","last_change":"2023-10-06T05:17:02.444044+0000","last_active":"2023-10-06T05:17:02.453300+0000","last_peered":"2023-10-06T05:17:02.453300+0000","last_clean":"2023-10-06T05:17:02.453300+0000","last_became_active":"2023-10-06T05:17:02.443659+0000","last_became_peered":"2023-10-06T05:17:02.443659+0000","last_unstale":"2023-10-06T05:17:02.453300+0000","last_undegraded":"2023-10-06T05:17:02.453300+0000","last_fullsized":"2023-10-06T05:17:02.453300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:46.086131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.435753+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:17:02.435753+0000","last_peered":"2023-10-06T05:17:02.435753+0000","last_clean":"2023-10-06T05:17:02.435753+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:17:02.435753+0000","last_undegraded":"2023-10-06T05:17:02.435753+0000","last_fullsized":"2023-10-06T05:17:02.435753+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.427375+0000","last_change":"2023-10-06T05:17:01.441700+0000","last_active":"2023-10-06T05:17:02.427375+0000","last_peered":"2023-10-06T05:17:02.427375+0000","last_clean":"2023-10-06T05:17:02.427375+0000","last_became_active":"2023-10-06T05:17:01.441370+0000","last_became_peered":"2023-10-06T05:17:01.441370+0000","last_unstale":"2023-10-06T05:17:02.427375+0000","last_undegraded":"2023-10-06T05:17:02.427375+0000","last_fullsized":"2023-10-06T05:17:02.427375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:01.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450934+0000","last_change":"2023-10-06T05:17:02.446567+0000","last_active":"2023-10-06T05:17:02.450934+0000","last_peered":"2023-10-06T05:17:02.450934+0000","last_clean":"2023-10-06T05:17:02.450934+0000","last_became_active":"2023-10-06T05:17:02.446263+0000","last_became_peered":"2023-10-06T05:17:02.446263+0000","last_unstale":"2023-10-06T05:17:02.450934+0000","last_undegraded":"2023-10-06T05:17:02.450934+0000","last_fullsized":"2023-10-06T05:17:02.450934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:20.941124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.433849+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:17:02.433849+0000","last_peered":"2023-10-06T05:17:02.433849+0000","last_clean":"2023-10-06T05:17:02.433849+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:17:02.433849+0000","last_undegraded":"2023-10-06T05:17:02.433849+0000","last_fullsized":"2023-10-06T05:17:02.433849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.429218+0000","last_change":"2023-10-06T05:17:01.448636+0000","last_active":"2023-10-06T05:17:02.429218+0000","last_peered":"2023-10-06T05:17:02.429218+0000","last_clean":"2023-10-06T05:17:02.429218+0000","last_became_active":"2023-10-06T05:17:01.446188+0000","last_became_peered":"2023-10-06T05:17:01.446188+0000","last_unstale":"2023-10-06T05:17:02.429218+0000","last_undegraded":"2023-10-06T05:17:02.429218+0000","last_fullsized":"2023-10-06T05:17:02.429218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453488+0000","last_change":"2023-10-06T05:17:02.447390+0000","last_active":"2023-10-06T05:17:02.453488+0000","last_peered":"2023-10-06T05:17:02.453488+0000","last_clean":"2023-10-06T05:17:02.453488+0000","last_became_active":"2023-10-06T05:17:02.447118+0000","last_became_peered":"2023-10-06T05:17:02.447118+0000","last_unstale":"2023-10-06T05:17:02.453488+0000","last_undegraded":"2023-10-06T05:17:02.453488+0000","last_fullsized":"2023-10-06T05:17:02.453488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:17.996091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.439722+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:17:02.439722+0000","last_peered":"2023-10-06T05:17:02.439722+0000","last_clean":"2023-10-06T05:17:02.439722+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:17:02.439722+0000","last_undegraded":"2023-10-06T05:17:02.439722+0000","last_fullsized":"2023-10-06T05:17:02.439722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.431999+0000","last_change":"2023-10-06T05:17:01.449227+0000","last_active":"2023-10-06T05:17:02.431999+0000","last_peered":"2023-10-06T05:17:02.431999+0000","last_clean":"2023-10-06T05:17:02.431999+0000","last_became_active":"2023-10-06T05:17:01.448956+0000","last_became_peered":"2023-10-06T05:17:01.448956+0000","last_unstale":"2023-10-06T05:17:02.431999+0000","last_undegraded":"2023-10-06T05:17:02.431999+0000","last_fullsized":"2023-10-06T05:17:02.431999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.450001+0000","last_change":"2023-10-06T05:17:02.442854+0000","last_active":"2023-10-06T05:17:02.450001+0000","last_peered":"2023-10-06T05:17:02.450001+0000","last_clean":"2023-10-06T05:17:02.450001+0000","last_became_active":"2023-10-06T05:17:02.442424+0000","last_became_peered":"2023-10-06T05:17:02.442424+0000","last_unstale":"2023-10-06T05:17:02.450001+0000","last_undegraded":"2023-10-06T05:17:02.450001+0000","last_fullsized":"2023-10-06T05:17:02.450001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:45.220431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.437608+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:17:02.437608+0000","last_peered":"2023-10-06T05:17:02.437608+0000","last_clean":"2023-10-06T05:17:02.437608+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:17:02.437608+0000","last_undegraded":"2023-10-06T05:17:02.437608+0000","last_fullsized":"2023-10-06T05:17:02.437608+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.433246+0000","last_change":"2023-10-06T05:17:01.444318+0000","last_active":"2023-10-06T05:17:02.433246+0000","last_peered":"2023-10-06T05:17:02.433246+0000","last_clean":"2023-10-06T05:17:02.433246+0000","last_became_active":"2023-10-06T05:17:01.444042+0000","last_became_peered":"2023-10-06T05:17:01.444042+0000","last_unstale":"2023-10-06T05:17:02.433246+0000","last_undegraded":"2023-10-06T05:17:02.433246+0000","last_fullsized":"2023-10-06T05:17:02.433246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:22.931668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453458+0000","last_change":"2023-10-06T05:17:02.443796+0000","last_active":"2023-10-06T05:17:02.453458+0000","last_peered":"2023-10-06T05:17:02.453458+0000","last_clean":"2023-10-06T05:17:02.453458+0000","last_became_active":"2023-10-06T05:17:02.443364+0000","last_became_peered":"2023-10-06T05:17:02.443364+0000","last_unstale":"2023-10-06T05:17:02.453458+0000","last_undegraded":"2023-10-06T05:17:02.453458+0000","last_fullsized":"2023-10-06T05:17:02.453458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:07.184397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465585+0000","last_change":"2023-10-06T05:17:01.451251+0000","last_active":"2023-10-06T05:17:02.465585+0000","last_peered":"2023-10-06T05:17:02.465585+0000","last_clean":"2023-10-06T05:17:02.465585+0000","last_became_active":"2023-10-06T05:17:01.450983+0000","last_became_peered":"2023-10-06T05:17:01.450983+0000","last_unstale":"2023-10-06T05:17:02.465585+0000","last_undegraded":"2023-10-06T05:17:02.465585+0000","last_fullsized":"2023-10-06T05:17:02.465585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456082+0000","last_change":"2023-10-06T05:17:02.451794+0000","last_active":"2023-10-06T05:17:02.456082+0000","last_peered":"2023-10-06T05:17:02.456082+0000","last_clean":"2023-10-06T05:17:02.456082+0000","last_became_active":"2023-10-06T05:17:02.451533+0000","last_became_peered":"2023-10-06T05:17:02.451533+0000","last_unstale":"2023-10-06T05:17:02.456082+0000","last_undegraded":"2023-10-06T05:17:02.456082+0000","last_fullsized":"2023-10-06T05:17:02.456082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:23.939592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471267+0000","last_change":"2023-10-06T05:17:01.449818+0000","last_active":"2023-10-06T05:17:02.471267+0000","last_peered":"2023-10-06T05:17:02.471267+0000","last_clean":"2023-10-06T05:17:02.471267+0000","last_became_active":"2023-10-06T05:17:01.449477+0000","last_became_peered":"2023-10-06T05:17:01.449477+0000","last_unstale":"2023-10-06T05:17:02.471267+0000","last_undegraded":"2023-10-06T05:17:02.471267+0000","last_fullsized":"2023-10-06T05:17:02.471267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:51.278470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447570+0000","last_change":"2023-10-06T05:17:02.443687+0000","last_active":"2023-10-06T05:17:02.447570+0000","last_peered":"2023-10-06T05:17:02.447570+0000","last_clean":"2023-10-06T05:17:02.447570+0000","last_became_active":"2023-10-06T05:17:02.443433+0000","last_became_peered":"2023-10-06T05:17:02.443433+0000","last_unstale":"2023-10-06T05:17:02.447570+0000","last_undegraded":"2023-10-06T05:17:02.447570+0000","last_fullsized":"2023-10-06T05:17:02.447570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:29.361752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.471253+0000","last_change":"2023-10-06T05:17:01.449867+0000","last_active":"2023-10-06T05:17:02.471253+0000","last_peered":"2023-10-06T05:17:02.471253+0000","last_clean":"2023-10-06T05:17:02.471253+0000","last_became_active":"2023-10-06T05:17:01.449452+0000","last_became_peered":"2023-10-06T05:17:01.449452+0000","last_unstale":"2023-10-06T05:17:02.471253+0000","last_undegraded":"2023-10-06T05:17:02.471253+0000","last_fullsized":"2023-10-06T05:17:02.471253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:28.972050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.447425+0000","last_change":"2023-10-06T05:17:02.439184+0000","last_active":"2023-10-06T05:17:02.447425+0000","last_peered":"2023-10-06T05:17:02.447425+0000","last_clean":"2023-10-06T05:17:02.447425+0000","last_became_active":"2023-10-06T05:17:02.438893+0000","last_became_peered":"2023-10-06T05:17:02.438893+0000","last_unstale":"2023-10-06T05:17:02.447425+0000","last_undegraded":"2023-10-06T05:17:02.447425+0000","last_fullsized":"2023-10-06T05:17:02.447425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:28.005164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.463116+0000","last_change":"2023-10-06T05:17:01.441308+0000","last_active":"2023-10-06T05:17:02.463116+0000","last_peered":"2023-10-06T05:17:02.463116+0000","last_clean":"2023-10-06T05:17:02.463116+0000","last_became_active":"2023-10-06T05:17:01.440904+0000","last_became_peered":"2023-10-06T05:17:01.440904+0000","last_unstale":"2023-10-06T05:17:02.463116+0000","last_undegraded":"2023-10-06T05:17:02.463116+0000","last_fullsized":"2023-10-06T05:17:02.463116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:52.916083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461869+0000","last_change":"2023-10-06T05:17:02.447801+0000","last_active":"2023-10-06T05:17:02.461869+0000","last_peered":"2023-10-06T05:17:02.461869+0000","last_clean":"2023-10-06T05:17:02.461869+0000","last_became_active":"2023-10-06T05:17:02.447361+0000","last_became_peered":"2023-10-06T05:17:02.447361+0000","last_unstale":"2023-10-06T05:17:02.461869+0000","last_undegraded":"2023-10-06T05:17:02.461869+0000","last_fullsized":"2023-10-06T05:17:02.461869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:29.674836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.465330+0000","last_change":"2023-10-06T05:17:01.446169+0000","last_active":"2023-10-06T05:17:02.465330+0000","last_peered":"2023-10-06T05:17:02.465330+0000","last_clean":"2023-10-06T05:17:02.465330+0000","last_became_active":"2023-10-06T05:17:01.445836+0000","last_became_peered":"2023-10-06T05:17:01.445836+0000","last_unstale":"2023-10-06T05:17:02.465330+0000","last_undegraded":"2023-10-06T05:17:02.465330+0000","last_fullsized":"2023-10-06T05:17:02.465330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:00.406554+0000","last_clean_scrub_stamp":"2023-10-06T05:17:00.406554+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:52.916083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452461+0000","last_change":"2023-10-06T05:17:02.447171+0000","last_active":"2023-10-06T05:17:02.452461+0000","last_peered":"2023-10-06T05:17:02.452461+0000","last_clean":"2023-10-06T05:17:02.452461+0000","last_became_active":"2023-10-06T05:17:02.446884+0000","last_became_peered":"2023-10-06T05:17:02.446884+0000","last_unstale":"2023-10-06T05:17:02.452461+0000","last_undegraded":"2023-10-06T05:17:02.452461+0000","last_fullsized":"2023-10-06T05:17:02.452461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:07.148284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.456523+0000","last_change":"2023-10-06T05:17:02.451494+0000","last_active":"2023-10-06T05:17:02.456523+0000","last_peered":"2023-10-06T05:17:02.456523+0000","last_clean":"2023-10-06T05:17:02.456523+0000","last_became_active":"2023-10-06T05:17:02.451213+0000","last_became_peered":"2023-10-06T05:17:02.451213+0000","last_unstale":"2023-10-06T05:17:02.456523+0000","last_undegraded":"2023-10-06T05:17:02.456523+0000","last_fullsized":"2023-10-06T05:17:02.456523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:54.883797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.461850+0000","last_change":"2023-10-06T05:17:02.444575+0000","last_active":"2023-10-06T05:17:02.461850+0000","last_peered":"2023-10-06T05:17:02.461850+0000","last_clean":"2023-10-06T05:17:02.461850+0000","last_became_active":"2023-10-06T05:17:02.444226+0000","last_became_peered":"2023-10-06T05:17:02.444226+0000","last_unstale":"2023-10-06T05:17:02.461850+0000","last_undegraded":"2023-10-06T05:17:02.461850+0000","last_fullsized":"2023-10-06T05:17:02.461850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:03.555988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.452618+0000","last_change":"2023-10-06T05:17:02.447774+0000","last_active":"2023-10-06T05:17:02.452618+0000","last_peered":"2023-10-06T05:17:02.452618+0000","last_clean":"2023-10-06T05:17:02.452618+0000","last_became_active":"2023-10-06T05:17:02.447500+0000","last_became_peered":"2023-10-06T05:17:02.447500+0000","last_unstale":"2023-10-06T05:17:02.452618+0000","last_undegraded":"2023-10-06T05:17:02.452618+0000","last_fullsized":"2023-10-06T05:17:02.452618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:01.131607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.462293+0000","last_change":"2023-10-06T05:17:02.441565+0000","last_active":"2023-10-06T05:17:02.462293+0000","last_peered":"2023-10-06T05:17:02.462293+0000","last_clean":"2023-10-06T05:17:02.462293+0000","last_became_active":"2023-10-06T05:17:02.441272+0000","last_became_peered":"2023-10-06T05:17:02.441272+0000","last_unstale":"2023-10-06T05:17:02.462293+0000","last_undegraded":"2023-10-06T05:17:02.462293+0000","last_fullsized":"2023-10-06T05:17:02.462293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:28.806289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T05:17:02.453469+0000","last_change":"2023-10-06T05:17:02.443555+0000","last_active":"2023-10-06T05:17:02.453469+0000","last_peered":"2023-10-06T05:17:02.453469+0000","last_clean":"2023-10-06T05:17:02.453469+0000","last_became_active":"2023-10-06T05:17:02.443268+0000","last_became_peered":"2023-10-06T05:17:02.443268+0000","last_unstale":"2023-10-06T05:17:02.453469+0000","last_undegraded":"2023-10-06T05:17:02.453469+0000","last_fullsized":"2023-10-06T05:17:02.453469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:01.414478+0000","last_clean_scrub_stamp":"2023-10-06T05:17:01.414478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:29.554988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298096,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073744,"statfs":{"total":96636764160,"available":96331513856,"internally_reserved":0,"allocated":827392,"data_stored":666304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":74184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298068,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073772,"statfs":{"total":96636764160,"available":96331542528,"internally_reserved":0,"allocated":798720,"data_stored":664552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":74184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":74274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":74808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:17:12.572 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T05:17:12.572 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T05:17:12.572 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T05:17:12.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T05:17:12.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.797+0000 7f1e13dff700 1 Processor -- start 2023-10-06T05:17:12.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.798+0000 7f1e13dff700 1 -- start start 2023-10-06T05:17:12.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.799+0000 7f1e13dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c15d2d0 0x7f1e0c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.799+0000 7f1e13dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0c3f40 con 0x7f1e0c159500 2023-10-06T05:17:12.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.799+0000 7f1e13dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0cb000 con 0x7f1e0c0cd510 2023-10-06T05:17:12.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.799+0000 7f1e13dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0b2860 con 0x7f1e0c15d2d0 2023-10-06T05:17:12.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.799+0000 7f1e11b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1e0c159500 0x7f1e0c0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44824/0 (socket says 172.21.15.31:44824) 2023-10-06T05:17:12.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.799+0000 7f1e11b9b700 1 -- 172.21.15.31:0/3264314047 learned_addr learned my addr 172.21.15.31:0/3264314047 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:12.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.799+0000 7f1e1239c700 1 --2- 172.21.15.31:0/3264314047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c15d2d0 0x7f1e0c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.800+0000 7f1e10b99700 1 -- 172.21.15.31:0/3264314047 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448503605 0 0) 0x7f1e0c0c3f40 con 0x7f1e0c159500 2023-10-06T05:17:12.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.800+0000 7f1e10b99700 1 -- 172.21.15.31:0/3264314047 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df8003570 con 0x7f1e0c159500 2023-10-06T05:17:12.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.800+0000 7f1e10b99700 1 -- 172.21.15.31:0/3264314047 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816721544 0 0) 0x7f1e0c0cb000 con 0x7f1e0c0cd510 2023-10-06T05:17:12.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.800+0000 7f1e10b99700 1 -- 172.21.15.31:0/3264314047 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c0c3f40 con 0x7f1e0c0cd510 2023-10-06T05:17:12.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.800+0000 7f1e1239c700 1 -- 172.21.15.31:0/3264314047 >> v1:172.21.15.62:6789/0 conn(0x7f1e0c0cd510 legacy=0x7f1e0c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.800+0000 7f1e1239c700 1 -- 172.21.15.31:0/3264314047 >> v1:172.21.15.31:6789/0 conn(0x7f1e0c159500 legacy=0x7f1e0c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.800+0000 7f1e1239c700 1 -- 172.21.15.31:0/3264314047 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c15d7d0 con 0x7f1e0c15d2d0 2023-10-06T05:17:12.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.800+0000 7f1e1239c700 1 --2- 172.21.15.31:0/3264314047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c15d2d0 0x7f1e0c0b0ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009e90 tx=0x7f1e04009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2261ac5eeb70d561 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.801+0000 7f1e10b99700 1 -- 172.21.15.31:0/3264314047 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400e070 con 0x7f1e0c15d2d0 2023-10-06T05:17:12.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.801+0000 7f1e10b99700 1 -- 172.21.15.31:0/3264314047 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0400bb80 con 0x7f1e0c15d2d0 2023-10-06T05:17:12.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.801+0000 7f1e10b99700 1 -- 172.21.15.31:0/3264314047 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04012430 con 0x7f1e0c15d2d0 2023-10-06T05:17:12.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.801+0000 7f1e13dff700 1 -- 172.21.15.31:0/3264314047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c15d2d0 msgr2=0x7f1e0c0b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.801+0000 7f1e13dff700 1 --2- 172.21.15.31:0/3264314047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c15d2d0 0x7f1e0c0b0ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009e90 tx=0x7f1e04009780 comp rx=0 tx=0).stop 2023-10-06T05:17:12.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.801+0000 7f1e13dff700 1 -- 172.21.15.31:0/3264314047 shutdown_connections 2023-10-06T05:17:12.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.801+0000 7f1e13dff700 1 --2- 172.21.15.31:0/3264314047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c15d2d0 0x7f1e0c0b0ab0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.801+0000 7f1e13dff700 1 -- 172.21.15.31:0/3264314047 >> 172.21.15.31:0/3264314047 conn(0x7f1e0c06ec90 msgr2=0x7f1e0c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:12.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.801+0000 7f1e13dff700 1 -- 172.21.15.31:0/3264314047 shutdown_connections 2023-10-06T05:17:12.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.801+0000 7f1e13dff700 1 -- 172.21.15.31:0/3264314047 wait complete. 2023-10-06T05:17:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.803+0000 7f1e13dff700 1 Processor -- start 2023-10-06T05:17:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.803+0000 7f1e13dff700 1 -- start start 2023-10-06T05:17:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e13dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e0c0cd510 0x7f1e0c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e13dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c159500 0x7f1e0c1598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e13dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e0c15d2d0 0x7f1e0c159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e13dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c15b030 con 0x7f1e0c0cd510 2023-10-06T05:17:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e13dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c169ba0 con 0x7f1e0c15d2d0 2023-10-06T05:17:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e13dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c15a4b0 con 0x7f1e0c159500 2023-10-06T05:17:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e11b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e0c0cd510 0x7f1e0c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e11b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e0c0cd510 0x7f1e0c0b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34522/0 (socket says 172.21.15.31:34522) 2023-10-06T05:17:12.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e11b9b700 1 -- 172.21.15.31:0/2018571467 learned_addr learned my addr 172.21.15.31:0/2018571467 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:12.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e1239c700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e0c15d2d0 0x7f1e0c159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e1139a700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c159500 0x7f1e0c1598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e11b9b700 1 -- 172.21.15.31:0/2018571467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c159500 msgr2=0x7f1e0c1598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e11b9b700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c159500 0x7f1e0c1598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e11b9b700 1 -- 172.21.15.31:0/2018571467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e0c15d2d0 msgr2=0x7f1e0c159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:12.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e11b9b700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e0c15d2d0 0x7f1e0c159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:12.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e11b9b700 1 -- 172.21.15.31:0/2018571467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04009a20 con 0x7f1e0c0cd510 2023-10-06T05:17:12.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.804+0000 7f1e1139a700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c159500 0x7f1e0c1598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:12.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.805+0000 7f1e11b9b700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e0c0cd510 0x7f1e0c0b54f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c790 tx=0x7f1e08007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.805+0000 7f1e01ffb700 1 -- 172.21.15.31:0/2018571467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08005b30 con 0x7f1e0c0cd510 2023-10-06T05:17:12.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.805+0000 7f1e01ffb700 1 -- 172.21.15.31:0/2018571467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e08005c90 con 0x7f1e0c0cd510 2023-10-06T05:17:12.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.805+0000 7f1e01ffb700 1 -- 172.21.15.31:0/2018571467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08007300 con 0x7f1e0c0cd510 2023-10-06T05:17:12.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.805+0000 7f1e13dff700 1 -- 172.21.15.31:0/2018571467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c15a790 con 0x7f1e0c0cd510 2023-10-06T05:17:12.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.805+0000 7f1e13dff700 1 -- 172.21.15.31:0/2018571467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c15ad50 con 0x7f1e0c0cd510 2023-10-06T05:17:12.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.806+0000 7f1e13dff700 1 -- 172.21.15.31:0/2018571467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c04c380 con 0x7f1e0c0cd510 2023-10-06T05:17:12.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.807+0000 7f1e01ffb700 1 -- 172.21.15.31:0/2018571467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e0801b3b0 con 0x7f1e0c0cd510 2023-10-06T05:17:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.808+0000 7f1e01ffb700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0066f30 0x7f1df00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.808+0000 7f1e01ffb700 1 -- 172.21.15.31:0/2018571467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1e08083cc0 con 0x7f1e0c0cd510 2023-10-06T05:17:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.808+0000 7f1e1139a700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0066f30 0x7f1df00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.809+0000 7f1e1139a700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0066f30 0x7f1df00693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0cd910 tx=0x7f1dfc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:12.812+0000 7f1e01ffb700 1 -- 172.21.15.31:0/2018571467 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e0808a080 con 0x7f1e0c0cd510 2023-10-06T05:17:13.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.055+0000 7f1e13dff700 1 -- 172.21.15.31:0/2018571467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1e0c0cade0 con 0x7f1e0c0cd510 2023-10-06T05:17:13.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.056+0000 7f1e01ffb700 1 -- 172.21.15.31:0/2018571467 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1e08017080 con 0x7f1e0c0cd510 2023-10-06T05:17:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:13.058 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T05:17:13.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 -- 172.21.15.31:0/2018571467 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0066f30 msgr2=0x7f1df00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0066f30 0x7f1df00693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0cd910 tx=0x7f1dfc009040 comp rx=0 tx=0).stop 2023-10-06T05:17:13.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 -- 172.21.15.31:0/2018571467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e0c0cd510 msgr2=0x7f1e0c0b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e0c0cd510 0x7f1e0c0b54f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c790 tx=0x7f1e08007ea0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 -- 172.21.15.31:0/2018571467 shutdown_connections 2023-10-06T05:17:13.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e0c15d2d0 0x7f1e0c159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1df0066f30 0x7f1df00693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e0c0cd510 0x7f1e0c0b54f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 --2- 172.21.15.31:0/2018571467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e0c159500 0x7f1e0c1598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 -- 172.21.15.31:0/2018571467 >> 172.21.15.31:0/2018571467 conn(0x7f1e0c06ec90 msgr2=0x7f1e0c0ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:13.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 -- 172.21.15.31:0/2018571467 shutdown_connections 2023-10-06T05:17:13.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.058+0000 7f1e13dff700 1 -- 172.21.15.31:0/2018571467 wait complete. 2023-10-06T05:17:13.077 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T05:17:13.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:13.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.271+0000 7f05807ff700 1 Processor -- start 2023-10-06T05:17:13.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.271+0000 7f05807ff700 1 -- start start 2023-10-06T05:17:13.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f05807ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f057813fb00 0x7f057806c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f05807ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f057813fed0 0x7f057806e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f05807ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057806ea90 0x7f05780cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f05807ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057806ef30 con 0x7f057813fb00 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f05807ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057806be70 con 0x7f057813fed0 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f05807ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057806f130 con 0x7f057806ea90 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f057e59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f057813fb00 0x7f057806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f057e59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f057813fb00 0x7f057806c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34528/0 (socket says 172.21.15.31:34528) 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f057e59b700 1 -- 172.21.15.31:0/2356172683 learned_addr learned my addr 172.21.15.31:0/2356172683 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f057dd9a700 1 --2- 172.21.15.31:0/2356172683 >> v2:172.21.15.62:3300/0 conn(0x7f057813fed0 0x7f057806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f057ed9c700 1 --2- 172.21.15.31:0/2356172683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057806ea90 0x7f05780cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f057e59b700 1 -- 172.21.15.31:0/2356172683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057806ea90 msgr2=0x7f05780cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:13.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.273+0000 7f057e59b700 1 --2- 172.21.15.31:0/2356172683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057806ea90 0x7f05780cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.274+0000 7f057e59b700 1 -- 172.21.15.31:0/2356172683 >> v2:172.21.15.62:3300/0 conn(0x7f057813fed0 msgr2=0x7f057806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:13.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.274+0000 7f057e59b700 1 --2- 172.21.15.31:0/2356172683 >> v2:172.21.15.62:3300/0 conn(0x7f057813fed0 0x7f057806e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.274+0000 7f057e59b700 1 -- 172.21.15.31:0/2356172683 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057806f410 con 0x7f057813fb00 2023-10-06T05:17:13.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.274+0000 7f057dd9a700 1 --2- 172.21.15.31:0/2356172683 >> v2:172.21.15.62:3300/0 conn(0x7f057813fed0 0x7f057806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:13.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.274+0000 7f057e59b700 1 --2- 172.21.15.31:0/2356172683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 0x7f057806c300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f05780710f0 tx=0x7f0574009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0325a0092cd1fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:13.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.274+0000 7f057d599700 1 -- 172.21.15.31:0/2356172683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400b860 con 0x7f057813fb00 2023-10-06T05:17:13.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.274+0000 7f057d599700 1 -- 172.21.15.31:0/2356172683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057400b9c0 con 0x7f057813fb00 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.274+0000 7f057d599700 1 -- 172.21.15.31:0/2356172683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400bd60 con 0x7f057813fb00 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.275+0000 7f05807ff700 1 -- 172.21.15.31:0/2356172683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 msgr2=0x7f057806c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.275+0000 7f05807ff700 1 --2- 172.21.15.31:0/2356172683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 0x7f057806c300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f05780710f0 tx=0x7f0574009250 comp rx=0 tx=0).stop 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.275+0000 7f05807ff700 1 -- 172.21.15.31:0/2356172683 shutdown_connections 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.275+0000 7f05807ff700 1 --2- 172.21.15.31:0/2356172683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057806ea90 0x7f05780cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.275+0000 7f05807ff700 1 --2- 172.21.15.31:0/2356172683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 0x7f057806c300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.275+0000 7f05807ff700 1 --2- 172.21.15.31:0/2356172683 >> v2:172.21.15.62:3300/0 conn(0x7f057813fed0 0x7f057806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.275+0000 7f05807ff700 1 -- 172.21.15.31:0/2356172683 >> 172.21.15.31:0/2356172683 conn(0x7f05780cdcb0 msgr2=0x7f05780c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.275+0000 7f05807ff700 1 -- 172.21.15.31:0/2356172683 shutdown_connections 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.275+0000 7f05807ff700 1 -- 172.21.15.31:0/2356172683 wait complete. 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.276+0000 7f05807ff700 1 Processor -- start 2023-10-06T05:17:13.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.276+0000 7f05807ff700 1 -- start start 2023-10-06T05:17:13.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.277+0000 7f05807ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057806ea90 0x7f0578156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.277+0000 7f05807ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 0x7f057815d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.277+0000 7f05807ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057813fed0 0x7f057815dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.277+0000 7f05807ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057815e180 con 0x7f057813fb00 2023-10-06T05:17:13.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.277+0000 7f05807ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057815e3f0 con 0x7f057813fed0 2023-10-06T05:17:13.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.277+0000 7f05807ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057815e6d0 con 0x7f057806ea90 2023-10-06T05:17:13.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 0x7f057815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 0x7f057815d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34540/0 (socket says 172.21.15.31:34540) 2023-10-06T05:17:13.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057dd9a700 1 -- 172.21.15.31:0/3665489123 learned_addr learned my addr 172.21.15.31:0/3665489123 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:13.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057ed9c700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057813fed0 0x7f057815dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057e59b700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057806ea90 0x7f0578156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057dd9a700 1 -- 172.21.15.31:0/3665489123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057806ea90 msgr2=0x7f0578156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057dd9a700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057806ea90 0x7f0578156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057dd9a700 1 -- 172.21.15.31:0/3665489123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057813fed0 msgr2=0x7f057815dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057dd9a700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057813fed0 0x7f057815dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057dd9a700 1 -- 172.21.15.31:0/3665489123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740094f0 con 0x7f057813fb00 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f057dd9a700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 0x7f057815d700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f056800e8f0 tx=0x7f0568008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f056e7fc700 1 -- 172.21.15.31:0/3665489123 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056800ee90 con 0x7f057813fb00 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f056e7fc700 1 -- 172.21.15.31:0/3665489123 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568005a00 con 0x7f057813fb00 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f05807ff700 1 -- 172.21.15.31:0/3665489123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057815e9b0 con 0x7f057813fb00 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.278+0000 7f056e7fc700 1 -- 172.21.15.31:0/3665489123 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568005d80 con 0x7f057813fb00 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.279+0000 7f05807ff700 1 -- 172.21.15.31:0/3665489123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057815ef70 con 0x7f057813fb00 2023-10-06T05:17:13.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.280+0000 7f05807ff700 1 -- 172.21.15.31:0/3665489123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057804c380 con 0x7f057813fb00 2023-10-06T05:17:13.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.280+0000 7f056e7fc700 1 -- 172.21.15.31:0/3665489123 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0568006230 con 0x7f057813fb00 2023-10-06T05:17:13.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.281+0000 7f056e7fc700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f055c066f30 0x7f055c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.281+0000 7f056e7fc700 1 -- 172.21.15.31:0/3665489123 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0568084880 con 0x7f057813fb00 2023-10-06T05:17:13.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.281+0000 7f057e59b700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f055c066f30 0x7f055c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.282+0000 7f057e59b700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f055c066f30 0x7f055c0693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f05780710f0 tx=0x7f0574017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:13.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.285+0000 7f056e7fc700 1 -- 172.21.15.31:0/3665489123 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05680518f0 con 0x7f057813fb00 2023-10-06T05:17:13.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.487+0000 7f05807ff700 1 -- 172.21.15.31:0/3665489123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f057813da20 con 0x7f057813fb00 2023-10-06T05:17:13.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.488+0000 7f056e7fc700 1 -- 172.21.15.31:0/3665489123 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f0568054f10 con 0x7f057813fb00 2023-10-06T05:17:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:13.491 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:13.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.490+0000 7f05807ff700 1 -- 172.21.15.31:0/3665489123 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f055c066f30 msgr2=0x7f055c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.490+0000 7f05807ff700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f055c066f30 0x7f055c0693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f05780710f0 tx=0x7f0574017000 comp rx=0 tx=0).stop 2023-10-06T05:17:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.490+0000 7f05807ff700 1 -- 172.21.15.31:0/3665489123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 msgr2=0x7f057815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.490+0000 7f05807ff700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 0x7f057815d700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f056800e8f0 tx=0x7f0568008040 comp rx=0 tx=0).stop 2023-10-06T05:17:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.491+0000 7f05807ff700 1 -- 172.21.15.31:0/3665489123 shutdown_connections 2023-10-06T05:17:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.491+0000 7f05807ff700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057813fed0 0x7f057815dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.491+0000 7f05807ff700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057806ea90 0x7f0578156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.491+0000 7f05807ff700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f055c066f30 0x7f055c0693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.491+0000 7f05807ff700 1 --2- 172.21.15.31:0/3665489123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057813fb00 0x7f057815d700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.491+0000 7f05807ff700 1 -- 172.21.15.31:0/3665489123 >> 172.21.15.31:0/3665489123 conn(0x7f05780cdcb0 msgr2=0x7f05780c2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:13.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.491+0000 7f05807ff700 1 -- 172.21.15.31:0/3665489123 shutdown_connections 2023-10-06T05:17:13.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.491+0000 7f05807ff700 1 -- 172.21.15.31:0/3665489123 wait complete. 2023-10-06T05:17:13.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:16:59.507957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:17:00.905826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:17:13.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.706+0000 7fdcc77ff700 1 Processor -- start 2023-10-06T05:17:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.706+0000 7fdcc77ff700 1 -- start start 2023-10-06T05:17:13.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc77ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdcc00ae860 0x7fdcc0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc77ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdcc00b1260 0x7fdcc00b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc77ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc0158cf0 con 0x7fdcc01573a0 2023-10-06T05:17:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc77ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc00b1c40 con 0x7fdcc00b1260 2023-10-06T05:17:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc77ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc01580e0 con 0x7fdcc00ae860 2023-10-06T05:17:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc5d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdcc00b1260 0x7fdcc00b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc4d9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdcc00ae860 0x7fdcc0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc5d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdcc00b1260 0x7fdcc00b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34546/0 (socket says 172.21.15.31:34546) 2023-10-06T05:17:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc5d9c700 1 -- 172.21.15.31:0/593936603 learned_addr learned my addr 172.21.15.31:0/593936603 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc5d9c700 1 -- 172.21.15.31:0/593936603 >> v2:172.21.15.62:3300/0 conn(0x7fdcc00ae860 msgr2=0x7fdcc0157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc5d9c700 1 --2- 172.21.15.31:0/593936603 >> v2:172.21.15.62:3300/0 conn(0x7fdcc00ae860 0x7fdcc0157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc5d9c700 1 -- 172.21.15.31:0/593936603 >> v1:172.21.15.62:6789/0 conn(0x7fdcc01573a0 legacy=0x7fdcc00acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.708+0000 7fdcc5d9c700 1 -- 172.21.15.31:0/593936603 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc01583c0 con 0x7fdcc00b1260 2023-10-06T05:17:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcc5d9c700 1 --2- 172.21.15.31:0/593936603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc00b1260 0x7fdcc00b1700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00c9c30 tx=0x7fdcb8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43514b748f1db8d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcb7fff700 1 -- 172.21.15.31:0/593936603 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800e070 con 0x7fdcc00b1260 2023-10-06T05:17:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcb7fff700 1 -- 172.21.15.31:0/593936603 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb800ba20 con 0x7fdcc00b1260 2023-10-06T05:17:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcb7fff700 1 -- 172.21.15.31:0/593936603 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80092e0 con 0x7fdcc00b1260 2023-10-06T05:17:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcc77ff700 1 -- 172.21.15.31:0/593936603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc00b1260 msgr2=0x7fdcc00b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcc77ff700 1 --2- 172.21.15.31:0/593936603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc00b1260 0x7fdcc00b1700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00c9c30 tx=0x7fdcb8009840 comp rx=0 tx=0).stop 2023-10-06T05:17:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcc77ff700 1 -- 172.21.15.31:0/593936603 shutdown_connections 2023-10-06T05:17:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcc77ff700 1 --2- 172.21.15.31:0/593936603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc00b1260 0x7fdcc00b1700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcc77ff700 1 --2- 172.21.15.31:0/593936603 >> v2:172.21.15.62:3300/0 conn(0x7fdcc00ae860 0x7fdcc0157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcc77ff700 1 -- 172.21.15.31:0/593936603 >> 172.21.15.31:0/593936603 conn(0x7fdcc0067560 msgr2=0x7fdcc0067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.709+0000 7fdcc77ff700 1 -- 172.21.15.31:0/593936603 shutdown_connections 2023-10-06T05:17:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.710+0000 7fdcc77ff700 1 -- 172.21.15.31:0/593936603 wait complete. 2023-10-06T05:17:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.711+0000 7fdcc77ff700 1 Processor -- start 2023-10-06T05:17:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.711+0000 7fdcc77ff700 1 -- start start 2023-10-06T05:17:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.712+0000 7fdcc77ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcc00b1260 0x7fdcc0146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.712+0000 7fdcc77ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc01573a0 0x7fdcc00afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc77ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc015d110 0x7fdcc01475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc77ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc00b02b0 con 0x7fdcc015d110 2023-10-06T05:17:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc77ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc00a9a40 con 0x7fdcc01573a0 2023-10-06T05:17:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc77ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc00a9d20 con 0x7fdcc00b1260 2023-10-06T05:17:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc5d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc015d110 0x7fdcc01475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc5d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc015d110 0x7fdcc01475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34550/0 (socket says 172.21.15.31:34550) 2023-10-06T05:17:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc5d9c700 1 -- 172.21.15.31:0/965898152 learned_addr learned my addr 172.21.15.31:0/965898152 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc559b700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcc00b1260 0x7fdcc0146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc4d9a700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc01573a0 0x7fdcc00afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc5d9c700 1 -- 172.21.15.31:0/965898152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcc00b1260 msgr2=0x7fdcc0146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc5d9c700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcc00b1260 0x7fdcc0146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc5d9c700 1 -- 172.21.15.31:0/965898152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc01573a0 msgr2=0x7fdcc00afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc5d9c700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc01573a0 0x7fdcc00afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc5d9c700 1 -- 172.21.15.31:0/965898152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8009ae0 con 0x7fdcc015d110 2023-10-06T05:17:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.713+0000 7fdcc559b700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcc00b1260 0x7fdcc0146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.714+0000 7fdcc5d9c700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc015d110 0x7fdcc01475d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009880 tx=0x7fdcb8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.714+0000 7fdcb57fa700 1 -- 172.21.15.31:0/965898152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8008a80 con 0x7fdcc015d110 2023-10-06T05:17:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.714+0000 7fdcb57fa700 1 -- 172.21.15.31:0/965898152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb8008c00 con 0x7fdcc015d110 2023-10-06T05:17:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.714+0000 7fdcc77ff700 1 -- 172.21.15.31:0/965898152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc00aa000 con 0x7fdcc015d110 2023-10-06T05:17:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.714+0000 7fdcb57fa700 1 -- 172.21.15.31:0/965898152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8026460 con 0x7fdcc015d110 2023-10-06T05:17:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.714+0000 7fdcc77ff700 1 -- 172.21.15.31:0/965898152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc0147020 con 0x7fdcc015d110 2023-10-06T05:17:13.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.715+0000 7fdcc77ff700 1 -- 172.21.15.31:0/965898152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc004c380 con 0x7fdcc015d110 2023-10-06T05:17:13.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.716+0000 7fdcb57fa700 1 -- 172.21.15.31:0/965898152 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcb8003040 con 0x7fdcc015d110 2023-10-06T05:17:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.717+0000 7fdcb57fa700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdca4066f30 0x7fdca40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.717+0000 7fdcb57fa700 1 -- 172.21.15.31:0/965898152 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdcb8085ff0 con 0x7fdcc015d110 2023-10-06T05:17:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.717+0000 7fdcc559b700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdca4066f30 0x7fdca40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.718+0000 7fdcc559b700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdca4066f30 0x7fdca40693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc001d20 tx=0x7fdcbc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.721+0000 7fdcb57fa700 1 -- 172.21.15.31:0/965898152 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcb8053080 con 0x7fdcc015d110 2023-10-06T05:17:13.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.924+0000 7fdcc77ff700 1 -- 172.21.15.31:0/965898152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdcc0157770 con 0x7fdcc015d110 2023-10-06T05:17:13.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.924+0000 7fdcb57fa700 1 -- 172.21.15.31:0/965898152 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdcb80566a0 con 0x7fdcc015d110 2023-10-06T05:17:13.926 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:17:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.926+0000 7fdcc77ff700 1 -- 172.21.15.31:0/965898152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdca4066f30 msgr2=0x7fdca40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.926+0000 7fdcc77ff700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdca4066f30 0x7fdca40693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc001d20 tx=0x7fdcbc009040 comp rx=0 tx=0).stop 2023-10-06T05:17:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.926+0000 7fdcc77ff700 1 -- 172.21.15.31:0/965898152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc015d110 msgr2=0x7fdcc01475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.926+0000 7fdcc77ff700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc015d110 0x7fdcc01475d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009880 tx=0x7fdcb8018000 comp rx=0 tx=0).stop 2023-10-06T05:17:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.927+0000 7fdcc77ff700 1 -- 172.21.15.31:0/965898152 shutdown_connections 2023-10-06T05:17:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.927+0000 7fdcc77ff700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdca4066f30 0x7fdca40693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.927+0000 7fdcc77ff700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcc015d110 0x7fdcc01475d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.927+0000 7fdcc77ff700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcc00b1260 0x7fdcc0146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.927+0000 7fdcc77ff700 1 --2- 172.21.15.31:0/965898152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc01573a0 0x7fdcc00afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.927+0000 7fdcc77ff700 1 -- 172.21.15.31:0/965898152 >> 172.21.15.31:0/965898152 conn(0x7fdcc0067560 msgr2=0x7fdcc00cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.927+0000 7fdcc77ff700 1 -- 172.21.15.31:0/965898152 shutdown_connections 2023-10-06T05:17:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:13.927+0000 7fdcc77ff700 1 -- 172.21.15.31:0/965898152 wait complete. 2023-10-06T05:17:13.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:14.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.145+0000 7f708a3ff700 1 Processor -- start 2023-10-06T05:17:14.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.145+0000 7f708a3ff700 1 -- start start 2023-10-06T05:17:14.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f708a3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840ae860 0x7f7084157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f708a3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7084158cf0 con 0x7f70840b1260 2023-10-06T05:17:14.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f708a3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70841580e0 con 0x7f70841573a0 2023-10-06T05:17:14.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f708a3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70841583c0 con 0x7f70840ae860 2023-10-06T05:17:14.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f708899c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f70840b1260 0x7f70841611e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44826/0 (socket says 172.21.15.31:44826) 2023-10-06T05:17:14.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f708899c700 1 -- 172.21.15.31:0/683362798 learned_addr learned my addr 172.21.15.31:0/683362798 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:14.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f70837fe700 1 --2- 172.21.15.31:0/683362798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840ae860 0x7f7084157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f7082ffd700 1 -- 172.21.15.31:0/683362798 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007485171 0 0) 0x7f7084158cf0 con 0x7f70840b1260 2023-10-06T05:17:14.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f7082ffd700 1 -- 172.21.15.31:0/683362798 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f706c003570 con 0x7f70840b1260 2023-10-06T05:17:14.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f7082ffd700 1 -- 172.21.15.31:0/683362798 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856910438 0 0) 0x7f70841580e0 con 0x7f70841573a0 2023-10-06T05:17:14.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.147+0000 7f7082ffd700 1 -- 172.21.15.31:0/683362798 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7084158cf0 con 0x7f70841573a0 2023-10-06T05:17:14.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.148+0000 7f70837fe700 1 -- 172.21.15.31:0/683362798 >> v1:172.21.15.62:6789/0 conn(0x7f70841573a0 legacy=0x7f70840acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.148+0000 7f70837fe700 1 -- 172.21.15.31:0/683362798 >> v1:172.21.15.31:6789/0 conn(0x7f70840b1260 legacy=0x7f70841611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.148+0000 7f70837fe700 1 -- 172.21.15.31:0/683362798 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70841586a0 con 0x7f70840ae860 2023-10-06T05:17:14.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.148+0000 7f70837fe700 1 --2- 172.21.15.31:0/683362798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840ae860 0x7f7084157770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7078009910 tx=0x7f70780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53d2096e358ba205 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.148+0000 7f7082ffd700 1 -- 172.21.15.31:0/683362798 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078011630 con 0x7f70840ae860 2023-10-06T05:17:14.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.148+0000 7f7082ffd700 1 -- 172.21.15.31:0/683362798 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7078011790 con 0x7f70840ae860 2023-10-06T05:17:14.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.148+0000 7f7082ffd700 1 -- 172.21.15.31:0/683362798 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078011b30 con 0x7f70840ae860 2023-10-06T05:17:14.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.149+0000 7f708a3ff700 1 -- 172.21.15.31:0/683362798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840ae860 msgr2=0x7f7084157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.149+0000 7f708a3ff700 1 --2- 172.21.15.31:0/683362798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840ae860 0x7f7084157770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7078009910 tx=0x7f70780091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.149+0000 7f708a3ff700 1 -- 172.21.15.31:0/683362798 shutdown_connections 2023-10-06T05:17:14.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.149+0000 7f708a3ff700 1 --2- 172.21.15.31:0/683362798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840ae860 0x7f7084157770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.149+0000 7f708a3ff700 1 -- 172.21.15.31:0/683362798 >> 172.21.15.31:0/683362798 conn(0x7f7084067560 msgr2=0x7f7084067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:14.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.149+0000 7f708a3ff700 1 -- 172.21.15.31:0/683362798 shutdown_connections 2023-10-06T05:17:14.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.149+0000 7f708a3ff700 1 -- 172.21.15.31:0/683362798 wait complete. 2023-10-06T05:17:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.150+0000 7f708a3ff700 1 Processor -- start 2023-10-06T05:17:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.150+0000 7f708a3ff700 1 -- start start 2023-10-06T05:17:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.151+0000 7f708a3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840b1260 0x7f70840a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.151+0000 7f708a3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70841573a0 0x7f70841491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.151+0000 7f708a3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f708414d630 0x7f708414dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.151+0000 7f708a3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70840a5b80 con 0x7f708414d630 2023-10-06T05:17:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.151+0000 7f708a3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084149730 con 0x7f70841573a0 2023-10-06T05:17:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.151+0000 7f708a3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840cc0a0 con 0x7f70840b1260 2023-10-06T05:17:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f708899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f708414d630 0x7f708414dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f708899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f708414d630 0x7f708414dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34554/0 (socket says 172.21.15.31:34554) 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f708899c700 1 -- 172.21.15.31:0/1867563783 learned_addr learned my addr 172.21.15.31:0/1867563783 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f70837fe700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70841573a0 0x7f70841491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f7083fff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840b1260 0x7f70840a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f708899c700 1 -- 172.21.15.31:0/1867563783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840b1260 msgr2=0x7f70840a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f708899c700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840b1260 0x7f70840a83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f708899c700 1 -- 172.21.15.31:0/1867563783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70841573a0 msgr2=0x7f70841491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f708899c700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70841573a0 0x7f70841491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f708899c700 1 -- 172.21.15.31:0/1867563783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078009420 con 0x7f708414d630 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f7083fff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840b1260 0x7f70840a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f708899c700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f708414d630 0x7f708414dab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f707c00cc80 tx=0x7f707c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.152+0000 7f706bfff700 1 -- 172.21.15.31:0/1867563783 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c00ce90 con 0x7f708414d630 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.153+0000 7f706bfff700 1 -- 172.21.15.31:0/1867563783 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c00ab70 con 0x7f708414d630 2023-10-06T05:17:14.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.153+0000 7f706bfff700 1 -- 172.21.15.31:0/1867563783 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0070f0 con 0x7f708414d630 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.153+0000 7f708a3ff700 1 -- 172.21.15.31:0/1867563783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70840cc380 con 0x7f708414d630 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.153+0000 7f708a3ff700 1 -- 172.21.15.31:0/1867563783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70840a5810 con 0x7f708414d630 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.154+0000 7f708a3ff700 1 -- 172.21.15.31:0/1867563783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70840a5cf0 con 0x7f708414d630 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.155+0000 7f706bfff700 1 -- 172.21.15.31:0/1867563783 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f707c0074a0 con 0x7f708414d630 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.155+0000 7f706bfff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7070066f30 0x7f70700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.156+0000 7f706bfff700 1 -- 172.21.15.31:0/1867563783 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f707c084ba0 con 0x7f708414d630 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.156+0000 7f7083fff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7070066f30 0x7f70700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.156+0000 7f7083fff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7070066f30 0x7f70700693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7074007fd0 tx=0x7f7074009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.160+0000 7f706bfff700 1 -- 172.21.15.31:0/1867563783 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f707c051d30 con 0x7f708414d630 2023-10-06T05:17:14.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.365+0000 7f708a3ff700 1 -- 172.21.15.31:0/1867563783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f708404c380 con 0x7f708414d630 2023-10-06T05:17:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.365+0000 7f706bfff700 1 -- 172.21.15.31:0/1867563783 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f707c055350 con 0x7f708414d630 2023-10-06T05:17:14.367 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:17:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 -- 172.21.15.31:0/1867563783 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7070066f30 msgr2=0x7f70700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7070066f30 0x7f70700693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7074007fd0 tx=0x7f7074009040 comp rx=0 tx=0).stop 2023-10-06T05:17:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 -- 172.21.15.31:0/1867563783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f708414d630 msgr2=0x7f708414dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f708414d630 0x7f708414dab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f707c00cc80 tx=0x7f707c001ae0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 -- 172.21.15.31:0/1867563783 shutdown_connections 2023-10-06T05:17:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7070066f30 0x7f70700693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f708414d630 0x7f708414dab0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70840b1260 0x7f70840a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 --2- 172.21.15.31:0/1867563783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70841573a0 0x7f70841491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 -- 172.21.15.31:0/1867563783 >> 172.21.15.31:0/1867563783 conn(0x7f7084067560 msgr2=0x7f70840aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 -- 172.21.15.31:0/1867563783 shutdown_connections 2023-10-06T05:17:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.368+0000 7f708a3ff700 1 -- 172.21.15.31:0/1867563783 wait complete. 2023-10-06T05:17:14.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:17:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.583+0000 7f76a75ff700 1 Processor -- start 2023-10-06T05:17:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.583+0000 7f76a75ff700 1 -- start start 2023-10-06T05:17:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f76a75ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a00ae860 0x7f76a0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f76a75ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a0158cf0 con 0x7f76a00b1260 2023-10-06T05:17:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f76a75ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a01580e0 con 0x7f76a01573a0 2023-10-06T05:17:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f76a75ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01583c0 con 0x7f76a00ae860 2023-10-06T05:17:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f76a5b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f76a00b1260 0x7f76a01611e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44830/0 (socket says 172.21.15.31:44830) 2023-10-06T05:17:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f76a5b9c700 1 -- 172.21.15.31:0/546260043 learned_addr learned my addr 172.21.15.31:0/546260043 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f76a4b9a700 1 --2- 172.21.15.31:0/546260043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a00ae860 0x7f76a0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f7697fff700 1 -- 172.21.15.31:0/546260043 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252060488 0 0) 0x7f76a0158cf0 con 0x7f76a00b1260 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f7697fff700 1 -- 172.21.15.31:0/546260043 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f768c003570 con 0x7f76a00b1260 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f7697fff700 1 -- 172.21.15.31:0/546260043 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233964959 0 0) 0x7f76a01580e0 con 0x7f76a01573a0 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.585+0000 7f7697fff700 1 -- 172.21.15.31:0/546260043 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76a0158cf0 con 0x7f76a01573a0 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.586+0000 7f76a4b9a700 1 -- 172.21.15.31:0/546260043 >> v1:172.21.15.62:6789/0 conn(0x7f76a01573a0 legacy=0x7f76a00acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.586+0000 7f76a4b9a700 1 -- 172.21.15.31:0/546260043 >> v1:172.21.15.31:6789/0 conn(0x7f76a00b1260 legacy=0x7f76a01611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.586+0000 7f76a4b9a700 1 -- 172.21.15.31:0/546260043 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a01586a0 con 0x7f76a00ae860 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.586+0000 7f76a4b9a700 1 --2- 172.21.15.31:0/546260043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a00ae860 0x7f76a0157770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7690009910 tx=0x7f76900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c656af7fb0e220e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.586+0000 7f7697fff700 1 -- 172.21.15.31:0/546260043 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690011630 con 0x7f76a00ae860 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.586+0000 7f7697fff700 1 -- 172.21.15.31:0/546260043 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7690011790 con 0x7f76a00ae860 2023-10-06T05:17:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.586+0000 7f7697fff700 1 -- 172.21.15.31:0/546260043 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690011b30 con 0x7f76a00ae860 2023-10-06T05:17:14.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.587+0000 7f76a75ff700 1 -- 172.21.15.31:0/546260043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a00ae860 msgr2=0x7f76a0157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.587+0000 7f76a75ff700 1 --2- 172.21.15.31:0/546260043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a00ae860 0x7f76a0157770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7690009910 tx=0x7f76900091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.587+0000 7f76a75ff700 1 -- 172.21.15.31:0/546260043 shutdown_connections 2023-10-06T05:17:14.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.587+0000 7f76a75ff700 1 --2- 172.21.15.31:0/546260043 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a00ae860 0x7f76a0157770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.587+0000 7f76a75ff700 1 -- 172.21.15.31:0/546260043 >> 172.21.15.31:0/546260043 conn(0x7f76a0067560 msgr2=0x7f76a0067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:14.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.587+0000 7f76a75ff700 1 -- 172.21.15.31:0/546260043 shutdown_connections 2023-10-06T05:17:14.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.587+0000 7f76a75ff700 1 -- 172.21.15.31:0/546260043 wait complete. 2023-10-06T05:17:14.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.589+0000 7f76a75ff700 1 Processor -- start 2023-10-06T05:17:14.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.590+0000 7f76a75ff700 1 -- start start 2023-10-06T05:17:14.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a75ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a00b1260 0x7f76a00ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a75ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a01573a0 0x7f76a00acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a75ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a00a9e20 0x7f76a0167de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a75ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00aaf10 con 0x7f76a00b1260 2023-10-06T05:17:14.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a75ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00a8080 con 0x7f76a00a9e20 2023-10-06T05:17:14.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a75ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00aa300 con 0x7f76a01573a0 2023-10-06T05:17:14.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a00b1260 0x7f76a00ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a00b1260 0x7f76a00ac5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34560/0 (socket says 172.21.15.31:34560) 2023-10-06T05:17:14.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a539b700 1 -- 172.21.15.31:0/2656940836 learned_addr learned my addr 172.21.15.31:0/2656940836 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:14.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a5b9c700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a00a9e20 0x7f76a0167de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.591+0000 7f76a4b9a700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a01573a0 0x7f76a00acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76a539b700 1 -- 172.21.15.31:0/2656940836 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a01573a0 msgr2=0x7f76a00acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76a539b700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a01573a0 0x7f76a00acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76a539b700 1 -- 172.21.15.31:0/2656940836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a00a9e20 msgr2=0x7f76a0167de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76a539b700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a00a9e20 0x7f76a0167de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76a539b700 1 -- 172.21.15.31:0/2656940836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009420 con 0x7f76a00b1260 2023-10-06T05:17:14.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76a4b9a700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a01573a0 0x7f76a00acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:14.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76a539b700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a00b1260 0x7f76a00ac5e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c580 tx=0x7f769c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76957fa700 1 -- 172.21.15.31:0/2656940836 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c010510 con 0x7f76a00b1260 2023-10-06T05:17:14.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76957fa700 1 -- 172.21.15.31:0/2656940836 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c010670 con 0x7f76a00b1260 2023-10-06T05:17:14.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76a75ff700 1 -- 172.21.15.31:0/2656940836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a00aa5e0 con 0x7f76a00b1260 2023-10-06T05:17:14.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76957fa700 1 -- 172.21.15.31:0/2656940836 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c010a30 con 0x7f76a00b1260 2023-10-06T05:17:14.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.592+0000 7f76a75ff700 1 -- 172.21.15.31:0/2656940836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a00aaba0 con 0x7f76a00b1260 2023-10-06T05:17:14.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.593+0000 7f76a75ff700 1 -- 172.21.15.31:0/2656940836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a004c380 con 0x7f76a00b1260 2023-10-06T05:17:14.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.594+0000 7f76957fa700 1 -- 172.21.15.31:0/2656940836 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f769c01b400 con 0x7f76a00b1260 2023-10-06T05:17:14.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.595+0000 7f76957fa700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7684066f30 0x7f76840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:14.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.595+0000 7f76957fa700 1 -- 172.21.15.31:0/2656940836 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f769c084350 con 0x7f76a00b1260 2023-10-06T05:17:14.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.595+0000 7f76a4b9a700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7684066f30 0x7f76840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:14.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.596+0000 7f76a4b9a700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7684066f30 0x7f76840693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76a00c9ee0 tx=0x7f7690017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.600+0000 7f76957fa700 1 -- 172.21.15.31:0/2656940836 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f769c0513c0 con 0x7f76a00b1260 2023-10-06T05:17:14.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.802+0000 7f76a75ff700 1 -- 172.21.15.31:0/2656940836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76a00c8880 con 0x7f76a00b1260 2023-10-06T05:17:14.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.803+0000 7f76957fa700 1 -- 172.21.15.31:0/2656940836 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f769c0549e0 con 0x7f76a00b1260 2023-10-06T05:17:14.805 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:17:14.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 -- 172.21.15.31:0/2656940836 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7684066f30 msgr2=0x7f76840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7684066f30 0x7f76840693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76a00c9ee0 tx=0x7f7690017000 comp rx=0 tx=0).stop 2023-10-06T05:17:14.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 -- 172.21.15.31:0/2656940836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a00b1260 msgr2=0x7f76a00ac5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:14.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a00b1260 0x7f76a00ac5e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c580 tx=0x7f769c007e10 comp rx=0 tx=0).stop 2023-10-06T05:17:14.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 -- 172.21.15.31:0/2656940836 shutdown_connections 2023-10-06T05:17:14.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a00a9e20 0x7f76a0167de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7684066f30 0x7f76840693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a00b1260 0x7f76a00ac5e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 --2- 172.21.15.31:0/2656940836 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a01573a0 0x7f76a00acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:14.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 -- 172.21.15.31:0/2656940836 >> 172.21.15.31:0/2656940836 conn(0x7f76a0067560 msgr2=0x7f76a0155290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:14.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 -- 172.21.15.31:0/2656940836 shutdown_connections 2023-10-06T05:17:14.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:14.805+0000 7f76a75ff700 1 -- 172.21.15.31:0/2656940836 wait complete. 2023-10-06T05:17:14.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:17:15.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.020+0000 7feccddff700 1 Processor -- start 2023-10-06T05:17:15.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.021+0000 7feccddff700 1 -- start start 2023-10-06T05:17:15.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.022+0000 7feccddff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fecc80cd580 0x7fecc81645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.022+0000 7feccddff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc815d2d0 con 0x7fecc8159930 2023-10-06T05:17:15.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.022+0000 7feccddff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc815d5b0 con 0x7fecc8159560 2023-10-06T05:17:15.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.022+0000 7feccddff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc815d890 con 0x7fecc80cd580 2023-10-06T05:17:15.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.022+0000 7fecc7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fecc80cd580 0x7fecc81645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.022+0000 7fecc7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fecc80cd580 0x7fecc81645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34572/0 (socket says 172.21.15.31:34572) 2023-10-06T05:17:15.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.022+0000 7fecc7fff700 1 -- 172.21.15.31:0/1760053455 learned_addr learned my addr 172.21.15.31:0/1760053455 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:15.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.022+0000 7fecc6ffd700 1 --1- 172.21.15.31:0/1760053455 >> v1:172.21.15.31:6789/0 conn(0x7fecc8159930 0x7fecc80b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44840/0 (socket says 172.21.15.31:44840) 2023-10-06T05:17:15.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.023+0000 7fecc7fff700 1 -- 172.21.15.31:0/1760053455 >> v1:172.21.15.62:6789/0 conn(0x7fecc8159560 legacy=0x7fecc80cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.023+0000 7fecc7fff700 1 -- 172.21.15.31:0/1760053455 >> v1:172.21.15.31:6789/0 conn(0x7fecc8159930 legacy=0x7fecc80b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.023+0000 7fecc7fff700 1 -- 172.21.15.31:0/1760053455 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc815db70 con 0x7fecc80cd580 2023-10-06T05:17:15.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.023+0000 7fecc7fff700 1 --2- 172.21.15.31:0/1760053455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80cd580 0x7fecc81645d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009f20 tx=0x7fecc0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc49afb54239cb5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.023+0000 7fecc67fc700 1 -- 172.21.15.31:0/1760053455 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc000e070 con 0x7fecc80cd580 2023-10-06T05:17:15.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.023+0000 7fecc67fc700 1 -- 172.21.15.31:0/1760053455 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc000ba80 con 0x7fecc80cd580 2023-10-06T05:17:15.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.023+0000 7fecc67fc700 1 -- 172.21.15.31:0/1760053455 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc00092e0 con 0x7fecc80cd580 2023-10-06T05:17:15.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.024+0000 7feccddff700 1 -- 172.21.15.31:0/1760053455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80cd580 msgr2=0x7fecc81645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.024+0000 7feccddff700 1 --2- 172.21.15.31:0/1760053455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80cd580 0x7fecc81645d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009f20 tx=0x7fecc0009840 comp rx=0 tx=0).stop 2023-10-06T05:17:15.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.024+0000 7feccddff700 1 -- 172.21.15.31:0/1760053455 shutdown_connections 2023-10-06T05:17:15.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.024+0000 7feccddff700 1 --2- 172.21.15.31:0/1760053455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80cd580 0x7fecc81645d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.024+0000 7feccddff700 1 -- 172.21.15.31:0/1760053455 >> 172.21.15.31:0/1760053455 conn(0x7fecc806ec90 msgr2=0x7fecc806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:15.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.024+0000 7feccddff700 1 -- 172.21.15.31:0/1760053455 shutdown_connections 2023-10-06T05:17:15.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.024+0000 7feccddff700 1 -- 172.21.15.31:0/1760053455 wait complete. 2023-10-06T05:17:15.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.026+0000 7feccddff700 1 Processor -- start 2023-10-06T05:17:15.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.026+0000 7feccddff700 1 -- start start 2023-10-06T05:17:15.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.027+0000 7feccddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecc80cd580 0x7fecc815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.027+0000 7feccddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc8159560 0x7fecc8155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.027+0000 7feccddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecc8159930 0x7fecc80c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.027+0000 7feccddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc815a6e0 con 0x7fecc8159560 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.027+0000 7feccddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc815c930 con 0x7fecc80cd580 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.027+0000 7feccddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc815cc10 con 0x7fecc8159930 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc8159560 0x7fecc8155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc8159560 0x7fecc8155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34580/0 (socket says 172.21.15.31:34580) 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc6ffd700 1 -- 172.21.15.31:0/3133935368 learned_addr learned my addr 172.21.15.31:0/3133935368 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc7fff700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecc8159930 0x7fecc80c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc77fe700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecc80cd580 0x7fecc815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc6ffd700 1 -- 172.21.15.31:0/3133935368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecc8159930 msgr2=0x7fecc80c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc6ffd700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecc8159930 0x7fecc80c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc6ffd700 1 -- 172.21.15.31:0/3133935368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecc80cd580 msgr2=0x7fecc815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc6ffd700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecc80cd580 0x7fecc815c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc6ffd700 1 -- 172.21.15.31:0/3133935368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0009ae0 con 0x7fecc8159560 2023-10-06T05:17:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc77fe700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecc80cd580 0x7fecc815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.028+0000 7fecc6ffd700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc8159560 0x7fecc8155cb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c560 tx=0x7fecbc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.029+0000 7fecaf7fe700 1 -- 172.21.15.31:0/3133935368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc005a70 con 0x7fecc8159560 2023-10-06T05:17:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.029+0000 7fecaf7fe700 1 -- 172.21.15.31:0/3133935368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc005bd0 con 0x7fecc8159560 2023-10-06T05:17:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.029+0000 7fecaf7fe700 1 -- 172.21.15.31:0/3133935368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0070d0 con 0x7fecc8159560 2023-10-06T05:17:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.029+0000 7feccddff700 1 -- 172.21.15.31:0/3133935368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc815cef0 con 0x7fecc8159560 2023-10-06T05:17:15.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.029+0000 7feccddff700 1 -- 172.21.15.31:0/3133935368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc815c570 con 0x7fecc8159560 2023-10-06T05:17:15.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.030+0000 7feccddff700 1 -- 172.21.15.31:0/3133935368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc815a850 con 0x7fecc8159560 2023-10-06T05:17:15.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.031+0000 7fecaf7fe700 1 -- 172.21.15.31:0/3133935368 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fecbc007440 con 0x7fecc8159560 2023-10-06T05:17:15.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.031+0000 7fecaf7fe700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb4066f30 0x7fecb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.032+0000 7fecaf7fe700 1 -- 172.21.15.31:0/3133935368 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fecbc083b80 con 0x7fecc8159560 2023-10-06T05:17:15.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.032+0000 7fecc77fe700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb4066f30 0x7fecb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.032+0000 7fecc77fe700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb4066f30 0x7fecb40693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fecb8007fd0 tx=0x7fecb8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.036+0000 7fecaf7fe700 1 -- 172.21.15.31:0/3133935368 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecbc050bf0 con 0x7fecc8159560 2023-10-06T05:17:15.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.238+0000 7feccddff700 1 -- 172.21.15.31:0/3133935368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fecc804c380 con 0x7fecc8159560 2023-10-06T05:17:15.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.238+0000 7fecaf7fe700 1 -- 172.21.15.31:0/3133935368 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fecbc083e60 con 0x7fecc8159560 2023-10-06T05:17:15.240 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:17:15.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 -- 172.21.15.31:0/3133935368 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb4066f30 msgr2=0x7fecb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb4066f30 0x7fecb40693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fecb8007fd0 tx=0x7fecb8009040 comp rx=0 tx=0).stop 2023-10-06T05:17:15.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 -- 172.21.15.31:0/3133935368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc8159560 msgr2=0x7fecc8155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc8159560 0x7fecc8155cb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c560 tx=0x7fecbc007f70 comp rx=0 tx=0).stop 2023-10-06T05:17:15.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 -- 172.21.15.31:0/3133935368 shutdown_connections 2023-10-06T05:17:15.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fecc8159930 0x7fecc80c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecc80cd580 0x7fecc815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb4066f30 0x7fecb40693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 --2- 172.21.15.31:0/3133935368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc8159560 0x7fecc8155cb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 -- 172.21.15.31:0/3133935368 >> 172.21.15.31:0/3133935368 conn(0x7fecc806ec90 msgr2=0x7fecc80b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:15.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 -- 172.21.15.31:0/3133935368 shutdown_connections 2023-10-06T05:17:15.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.241+0000 7feccddff700 1 -- 172.21.15.31:0/3133935368 wait complete. 2023-10-06T05:17:15.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:15.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.453+0000 7f3f329ff700 1 Processor -- start 2023-10-06T05:17:15.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.453+0000 7f3f329ff700 1 -- start start 2023-10-06T05:17:15.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.454+0000 7f3f329ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3f2c15a930 0x7f3f2c0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.454+0000 7f3f329ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f2c0cf5a0 0x7f3f2c163240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.454+0000 7f3f329ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c0ce090 con 0x7f3f2c0cb1b0 2023-10-06T05:17:15.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.454+0000 7f3f329ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2c0cb6f0 con 0x7f3f2c15a930 2023-10-06T05:17:15.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.454+0000 7f3f329ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0cb9d0 con 0x7f3f2c0cf5a0 2023-10-06T05:17:15.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.454+0000 7f3f2b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3f2c0cb1b0 0x7f3f2c0ccd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44856/0 (socket says 172.21.15.31:44856) 2023-10-06T05:17:15.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.454+0000 7f3f2b7fe700 1 -- 172.21.15.31:0/1458575994 learned_addr learned my addr 172.21.15.31:0/1458575994 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:15.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.455+0000 7f3f2bfff700 1 --2- 172.21.15.31:0/1458575994 >> v2:172.21.15.62:3300/0 conn(0x7f3f2c15a930 0x7f3f2c0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.455+0000 7f3f30f9c700 1 --2- 172.21.15.31:0/1458575994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f2c0cf5a0 0x7f3f2c163240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.455+0000 7f3f2affd700 1 -- 172.21.15.31:0/1458575994 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403825419 0 0) 0x7f3f2c0ce090 con 0x7f3f2c0cb1b0 2023-10-06T05:17:15.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.455+0000 7f3f2affd700 1 -- 172.21.15.31:0/1458575994 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f14003570 con 0x7f3f2c0cb1b0 2023-10-06T05:17:15.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.455+0000 7f3f2affd700 1 -- 172.21.15.31:0/1458575994 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2623185137 0 0) 0x7f3f14003570 con 0x7f3f2c0cb1b0 2023-10-06T05:17:15.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.455+0000 7f3f2bfff700 1 -- 172.21.15.31:0/1458575994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f2c0cf5a0 msgr2=0x7f3f2c163240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:15.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.455+0000 7f3f2bfff700 1 --2- 172.21.15.31:0/1458575994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f2c0cf5a0 0x7f3f2c163240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.455+0000 7f3f2bfff700 1 -- 172.21.15.31:0/1458575994 >> v1:172.21.15.31:6789/0 conn(0x7f3f2c0cb1b0 legacy=0x7f3f2c0ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.455+0000 7f3f2bfff700 1 -- 172.21.15.31:0/1458575994 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0cbcb0 con 0x7f3f2c15a930 2023-10-06T05:17:15.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.455+0000 7f3f2bfff700 1 --2- 172.21.15.31:0/1458575994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f2c15a930 0x7f3f2c0cac70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0098d0 tx=0x7f3f1c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9328da3e71aa890 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.456+0000 7f3f2affd700 1 -- 172.21.15.31:0/1458575994 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20001470 con 0x7f3f2c15a930 2023-10-06T05:17:15.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.456+0000 7f3f2affd700 1 -- 172.21.15.31:0/1458575994 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c011420 con 0x7f3f2c15a930 2023-10-06T05:17:15.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.456+0000 7f3f2affd700 1 -- 172.21.15.31:0/1458575994 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0117c0 con 0x7f3f2c15a930 2023-10-06T05:17:15.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.456+0000 7f3f329ff700 1 -- 172.21.15.31:0/1458575994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f2c15a930 msgr2=0x7f3f2c0cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.456+0000 7f3f329ff700 1 --2- 172.21.15.31:0/1458575994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f2c15a930 0x7f3f2c0cac70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0098d0 tx=0x7f3f1c009120 comp rx=0 tx=0).stop 2023-10-06T05:17:15.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.456+0000 7f3f329ff700 1 -- 172.21.15.31:0/1458575994 shutdown_connections 2023-10-06T05:17:15.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.456+0000 7f3f329ff700 1 --2- 172.21.15.31:0/1458575994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f2c15a930 0x7f3f2c0cac70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.456+0000 7f3f329ff700 1 --2- 172.21.15.31:0/1458575994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f2c0cf5a0 0x7f3f2c163240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.456+0000 7f3f329ff700 1 -- 172.21.15.31:0/1458575994 >> 172.21.15.31:0/1458575994 conn(0x7f3f2c099720 msgr2=0x7f3f2c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:15.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.456+0000 7f3f329ff700 1 -- 172.21.15.31:0/1458575994 shutdown_connections 2023-10-06T05:17:15.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.457+0000 7f3f329ff700 1 -- 172.21.15.31:0/1458575994 wait complete. 2023-10-06T05:17:15.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.458+0000 7f3f329ff700 1 Processor -- start 2023-10-06T05:17:15.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.458+0000 7f3f329ff700 1 -- start start 2023-10-06T05:17:15.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f329ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f2c0cb1b0 0x7f3f2c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f329ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f2c0cf5a0 0x7f3f2c09d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f329ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f2c15a930 0x7f3f2c151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f329ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c09a500 con 0x7f3f2c15a930 2023-10-06T05:17:15.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f329ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c09a7e0 con 0x7f3f2c0cf5a0 2023-10-06T05:17:15.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f329ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c09aac0 con 0x7f3f2c0cb1b0 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f30f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f2c15a930 0x7f3f2c151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f30f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f2c15a930 0x7f3f2c151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34590/0 (socket says 172.21.15.31:34590) 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f30f9c700 1 -- 172.21.15.31:0/1184157571 learned_addr learned my addr 172.21.15.31:0/1184157571 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f2bfff700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f2c0cb1b0 0x7f3f2c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.459+0000 7f3f2b7fe700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f2c0cf5a0 0x7f3f2c09d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f30f9c700 1 -- 172.21.15.31:0/1184157571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f2c0cb1b0 msgr2=0x7f3f2c0b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f30f9c700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f2c0cb1b0 0x7f3f2c0b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f30f9c700 1 -- 172.21.15.31:0/1184157571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f2c0cf5a0 msgr2=0x7f3f2c09d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f30f9c700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f2c0cf5a0 0x7f3f2c09d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f30f9c700 1 -- 172.21.15.31:0/1184157571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0093c0 con 0x7f3f2c15a930 2023-10-06T05:17:15.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f2b7fe700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f2c0cf5a0 0x7f3f2c09d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:15.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f30f9c700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f2c15a930 0x7f3f2c151720 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009e90 tx=0x7f3f2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f13fff700 1 -- 172.21.15.31:0/1184157571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400cc20 con 0x7f3f2c15a930 2023-10-06T05:17:15.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f13fff700 1 -- 172.21.15.31:0/1184157571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2400f040 con 0x7f3f2c15a930 2023-10-06T05:17:15.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f329ff700 1 -- 172.21.15.31:0/1184157571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2c09ada0 con 0x7f3f2c15a930 2023-10-06T05:17:15.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f13fff700 1 -- 172.21.15.31:0/1184157571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24014480 con 0x7f3f2c15a930 2023-10-06T05:17:15.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.460+0000 7f3f329ff700 1 -- 172.21.15.31:0/1184157571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2c09a190 con 0x7f3f2c15a930 2023-10-06T05:17:15.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.461+0000 7f3f329ff700 1 -- 172.21.15.31:0/1184157571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2c0635c0 con 0x7f3f2c15a930 2023-10-06T05:17:15.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.462+0000 7f3f13fff700 1 -- 172.21.15.31:0/1184157571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f24014830 con 0x7f3f2c15a930 2023-10-06T05:17:15.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.463+0000 7f3f13fff700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f18066f30 0x7f3f180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.463+0000 7f3f13fff700 1 -- 172.21.15.31:0/1184157571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3f24085410 con 0x7f3f2c15a930 2023-10-06T05:17:15.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.463+0000 7f3f2bfff700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f18066f30 0x7f3f180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.464+0000 7f3f2bfff700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f18066f30 0x7f3f180693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009390 tx=0x7f3f1c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.467+0000 7f3f13fff700 1 -- 172.21.15.31:0/1184157571 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f24055f10 con 0x7f3f2c15a930 2023-10-06T05:17:15.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.716+0000 7f3f329ff700 1 -- 172.21.15.31:0/1184157571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f2c04c380 con 0x7f3f2c15a930 2023-10-06T05:17:15.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.717+0000 7f3f13fff700 1 -- 172.21.15.31:0/1184157571 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f3f24055aa0 con 0x7f3f2c15a930 2023-10-06T05:17:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:15.719 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1001195925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1001195925},{"type":"v1","addr":"172.21.15.62:6835","nonce":1001195925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1336279472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1336279472},{"type":"v1","addr":"172.21.15.31:6837","nonce":1336279472}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/577485679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":577485679},{"type":"v1","addr":"172.21.15.62:6834","nonce":577485679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:01.930460+0000","modified":"2023-10-06T05:17:05.459675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4169737410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169737410},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169737410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:17:15.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.719+0000 7f3f329ff700 1 -- 172.21.15.31:0/1184157571 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f18066f30 msgr2=0x7f3f180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.719+0000 7f3f329ff700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f18066f30 0x7f3f180693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009390 tx=0x7f3f1c016000 comp rx=0 tx=0).stop 2023-10-06T05:17:15.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.719+0000 7f3f329ff700 1 -- 172.21.15.31:0/1184157571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f2c15a930 msgr2=0x7f3f2c151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.719+0000 7f3f329ff700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f2c15a930 0x7f3f2c151720 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009e90 tx=0x7f3f2400b040 comp rx=0 tx=0).stop 2023-10-06T05:17:15.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.719+0000 7f3f329ff700 1 -- 172.21.15.31:0/1184157571 shutdown_connections 2023-10-06T05:17:15.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.719+0000 7f3f329ff700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f18066f30 0x7f3f180693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.719+0000 7f3f329ff700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f2c15a930 0x7f3f2c151720 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.719+0000 7f3f329ff700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f2c0cb1b0 0x7f3f2c0b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.719+0000 7f3f329ff700 1 --2- 172.21.15.31:0/1184157571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f2c0cf5a0 0x7f3f2c09d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.719+0000 7f3f329ff700 1 -- 172.21.15.31:0/1184157571 >> 172.21.15.31:0/1184157571 conn(0x7f3f2c099720 msgr2=0x7f3f2c1539e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:15.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.720+0000 7f3f329ff700 1 -- 172.21.15.31:0/1184157571 shutdown_connections 2023-10-06T05:17:15.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.720+0000 7f3f329ff700 1 -- 172.21.15.31:0/1184157571 wait complete. 2023-10-06T05:17:15.723 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T05:17:15.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:15.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.931+0000 7f92ec5ff700 1 Processor -- start 2023-10-06T05:17:15.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.931+0000 7f92ec5ff700 1 -- start start 2023-10-06T05:17:15.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92ec5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f92e41595a0 0x7f92e4159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92ec5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92ec5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e40c95b0 con 0x7f92e40cc0d0 2023-10-06T05:17:15.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92ec5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e40c9890 con 0x7f92e41595a0 2023-10-06T05:17:15.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92ec5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40c9b70 con 0x7f92e4068d80 2023-10-06T05:17:15.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92eab9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92ea39b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f92e41595a0 0x7f92e4159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92eab9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e40c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49972/0 (socket says 172.21.15.31:49972) 2023-10-06T05:17:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92eab9c700 1 -- 172.21.15.31:0/2749365294 learned_addr learned my addr 172.21.15.31:0/2749365294 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92ea39b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f92e41595a0 0x7f92e4159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53682/0 (socket says 172.21.15.31:53682) 2023-10-06T05:17:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92e9b9a700 1 --1- 172.21.15.31:0/2749365294 >> v1:172.21.15.62:6789/0 conn(0x7f92e40cc0d0 0x7f92e4068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:34262/0 (socket says 172.21.15.31:34262) 2023-10-06T05:17:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92eab9c700 1 -- 172.21.15.31:0/2749365294 >> v2:172.21.15.62:3300/0 conn(0x7f92e41595a0 msgr2=0x7f92e4159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92eab9c700 1 --2- 172.21.15.31:0/2749365294 >> v2:172.21.15.62:3300/0 conn(0x7f92e41595a0 0x7f92e4159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92eab9c700 1 -- 172.21.15.31:0/2749365294 >> v1:172.21.15.62:6789/0 conn(0x7f92e40cc0d0 legacy=0x7f92e4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.933+0000 7f92eab9c700 1 -- 172.21.15.31:0/2749365294 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40ca130 con 0x7f92e4068d80 2023-10-06T05:17:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.934+0000 7f92eab9c700 1 --2- 172.21.15.31:0/2749365294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e40c8930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009f20 tx=0x7f92dc009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaed9fc481757f6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.934+0000 7f92e9399700 1 -- 172.21.15.31:0/2749365294 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc00e070 con 0x7f92e4068d80 2023-10-06T05:17:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.934+0000 7f92e9399700 1 -- 172.21.15.31:0/2749365294 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc00b9d0 con 0x7f92e4068d80 2023-10-06T05:17:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.934+0000 7f92e9399700 1 -- 172.21.15.31:0/2749365294 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0092e0 con 0x7f92e4068d80 2023-10-06T05:17:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.935+0000 7f92ec5ff700 1 -- 172.21.15.31:0/2749365294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 msgr2=0x7f92e40c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.935+0000 7f92ec5ff700 1 --2- 172.21.15.31:0/2749365294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e40c8930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009f20 tx=0x7f92dc009840 comp rx=0 tx=0).stop 2023-10-06T05:17:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.935+0000 7f92ec5ff700 1 -- 172.21.15.31:0/2749365294 shutdown_connections 2023-10-06T05:17:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.935+0000 7f92ec5ff700 1 --2- 172.21.15.31:0/2749365294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e40c8930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.935+0000 7f92ec5ff700 1 --2- 172.21.15.31:0/2749365294 >> v2:172.21.15.62:3300/0 conn(0x7f92e41595a0 0x7f92e4159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.935+0000 7f92ec5ff700 1 -- 172.21.15.31:0/2749365294 >> 172.21.15.31:0/2749365294 conn(0x7f92e406b620 msgr2=0x7f92e406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.935+0000 7f92ec5ff700 1 -- 172.21.15.31:0/2749365294 shutdown_connections 2023-10-06T05:17:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.935+0000 7f92ec5ff700 1 -- 172.21.15.31:0/2749365294 wait complete. 2023-10-06T05:17:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.936+0000 7f92ec5ff700 1 Processor -- start 2023-10-06T05:17:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.936+0000 7f92ec5ff700 1 -- start start 2023-10-06T05:17:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.937+0000 7f92ec5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e4145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.937+0000 7f92ec5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92e40cc0d0 0x7f92e40b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.937+0000 7f92ec5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e41595a0 0x7f92e40b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.937+0000 7f92ec5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40c9100 con 0x7f92e41595a0 2023-10-06T05:17:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.937+0000 7f92ec5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40b1c70 con 0x7f92e40cc0d0 2023-10-06T05:17:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.937+0000 7f92ec5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40b1f50 con 0x7f92e4068d80 2023-10-06T05:17:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92eab9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e41595a0 0x7f92e40b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92eab9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e41595a0 0x7f92e40b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34594/0 (socket says 172.21.15.31:34594) 2023-10-06T05:17:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92eab9c700 1 -- 172.21.15.31:0/1106974140 learned_addr learned my addr 172.21.15.31:0/1106974140 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92ea39b700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e4145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92e9b9a700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92e40cc0d0 0x7f92e40b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92eab9c700 1 -- 172.21.15.31:0/1106974140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 msgr2=0x7f92e4145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92eab9c700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e4145ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92eab9c700 1 -- 172.21.15.31:0/1106974140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92e40cc0d0 msgr2=0x7f92e40b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92eab9c700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92e40cc0d0 0x7f92e40b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92eab9c700 1 -- 172.21.15.31:0/1106974140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc009ae0 con 0x7f92e41595a0 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92ea39b700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e4145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92e9b9a700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92e40cc0d0 0x7f92e40b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92eab9c700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e41595a0 0x7f92e40b15d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f92e40c9cb0 tx=0x7f92dc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.938+0000 7f92da7fc700 1 -- 172.21.15.31:0/1106974140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc008c30 con 0x7f92e41595a0 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.939+0000 7f92da7fc700 1 -- 172.21.15.31:0/1106974140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc008d90 con 0x7f92e41595a0 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.939+0000 7f92ec5ff700 1 -- 172.21.15.31:0/1106974140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e40b2230 con 0x7f92e41595a0 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.939+0000 7f92da7fc700 1 -- 172.21.15.31:0/1106974140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc01c660 con 0x7f92e41595a0 2023-10-06T05:17:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.939+0000 7f92ec5ff700 1 -- 172.21.15.31:0/1106974140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e40b27f0 con 0x7f92e41595a0 2023-10-06T05:17:15.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.940+0000 7f92ec5ff700 1 -- 172.21.15.31:0/1106974140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e404c380 con 0x7f92e41595a0 2023-10-06T05:17:15.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.941+0000 7f92da7fc700 1 -- 172.21.15.31:0/1106974140 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f92dc003040 con 0x7f92e41595a0 2023-10-06T05:17:15.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.941+0000 7f92da7fc700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f92c8066f30 0x7f92c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.941+0000 7f92da7fc700 1 -- 172.21.15.31:0/1106974140 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f92dc085f90 con 0x7f92e41595a0 2023-10-06T05:17:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.942+0000 7f92ea39b700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f92c8066f30 0x7f92c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.942+0000 7f92ea39b700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f92c8066f30 0x7f92c80693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009d80 tx=0x7f92e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:15.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:15.946+0000 7f92da7fc700 1 -- 172.21.15.31:0/1106974140 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92dc053000 con 0x7f92e41595a0 2023-10-06T05:17:16.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.147+0000 7f92ec5ff700 1 -- 172.21.15.31:0/1106974140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92e4069880 con 0x7f92e41595a0 2023-10-06T05:17:16.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.148+0000 7f92da7fc700 1 -- 172.21.15.31:0/1106974140 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f92dc056620 con 0x7f92e41595a0 2023-10-06T05:17:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:16.150 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:16.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 -- 172.21.15.31:0/1106974140 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f92c8066f30 msgr2=0x7f92c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f92c8066f30 0x7f92c80693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009d80 tx=0x7f92e0008040 comp rx=0 tx=0).stop 2023-10-06T05:17:16.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 -- 172.21.15.31:0/1106974140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e41595a0 msgr2=0x7f92e40b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e41595a0 0x7f92e40b15d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f92e40c9cb0 tx=0x7f92dc018000 comp rx=0 tx=0).stop 2023-10-06T05:17:16.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 -- 172.21.15.31:0/1106974140 shutdown_connections 2023-10-06T05:17:16.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f92c8066f30 0x7f92c80693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e41595a0 0x7f92e40b15d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92e4068d80 0x7f92e4145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 --2- 172.21.15.31:0/1106974140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92e40cc0d0 0x7f92e40b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 -- 172.21.15.31:0/1106974140 >> 172.21.15.31:0/1106974140 conn(0x7f92e406b620 msgr2=0x7f92e40ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:16.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 -- 172.21.15.31:0/1106974140 shutdown_connections 2023-10-06T05:17:16.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.150+0000 7f92ec5ff700 1 -- 172.21.15.31:0/1106974140 wait complete. 2023-10-06T05:17:16.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:16:59.507957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:17:00.905826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:17:16.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.366+0000 7f9284dff700 1 Processor -- start 2023-10-06T05:17:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.367+0000 7f9284dff700 1 -- start start 2023-10-06T05:17:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.368+0000 7f9284dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9280159560 0x7f92800cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.368+0000 7f9284dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9280159930 0x7f92800cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.368+0000 7f9284dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928015d2d0 con 0x7f92800b0d60 2023-10-06T05:17:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.368+0000 7f9284dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928015d5b0 con 0x7f9280159560 2023-10-06T05:17:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.368+0000 7f9284dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928015d890 con 0x7f9280159930 2023-10-06T05:17:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.368+0000 7f927e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9280159560 0x7f92800cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.368+0000 7f927e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9280159560 0x7f92800cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34606/0 (socket says 172.21.15.31:34606) 2023-10-06T05:17:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.368+0000 7f927e59c700 1 -- 172.21.15.31:0/2065838283 learned_addr learned my addr 172.21.15.31:0/2065838283 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.368+0000 7f927dd9b700 1 --2- 172.21.15.31:0/2065838283 >> v2:172.21.15.62:3300/0 conn(0x7f9280159930 0x7f92800cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.369+0000 7f927e59c700 1 -- 172.21.15.31:0/2065838283 >> v2:172.21.15.62:3300/0 conn(0x7f9280159930 msgr2=0x7f92800cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:16.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.369+0000 7f927e59c700 1 --2- 172.21.15.31:0/2065838283 >> v2:172.21.15.62:3300/0 conn(0x7f9280159930 0x7f92800cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.369+0000 7f927e59c700 1 -- 172.21.15.31:0/2065838283 >> v1:172.21.15.31:6789/0 conn(0x7f92800b0d60 legacy=0x7f92800b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.369+0000 7f927e59c700 1 -- 172.21.15.31:0/2065838283 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928015db70 con 0x7f9280159560 2023-10-06T05:17:16.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.369+0000 7f927d59a700 1 -- 172.21.15.31:0/2065838283 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642452623 0 0) 0x7f928015d2d0 con 0x7f92800b0d60 2023-10-06T05:17:16.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.369+0000 7f927e59c700 1 --2- 172.21.15.31:0/2065838283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9280159560 0x7f92800cce70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9280089160 tx=0x7f9270009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfafa369896d94e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.369+0000 7f927d59a700 1 -- 172.21.15.31:0/2065838283 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270009e70 con 0x7f9280159560 2023-10-06T05:17:16.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.369+0000 7f927d59a700 1 -- 172.21.15.31:0/2065838283 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927000b860 con 0x7f9280159560 2023-10-06T05:17:16.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.369+0000 7f927d59a700 1 -- 172.21.15.31:0/2065838283 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927000bc00 con 0x7f9280159560 2023-10-06T05:17:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.370+0000 7f9284dff700 1 -- 172.21.15.31:0/2065838283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9280159560 msgr2=0x7f92800cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.370+0000 7f9284dff700 1 --2- 172.21.15.31:0/2065838283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9280159560 0x7f92800cce70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9280089160 tx=0x7f9270009250 comp rx=0 tx=0).stop 2023-10-06T05:17:16.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.370+0000 7f9284dff700 1 -- 172.21.15.31:0/2065838283 shutdown_connections 2023-10-06T05:17:16.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.370+0000 7f9284dff700 1 --2- 172.21.15.31:0/2065838283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9280159560 0x7f92800cce70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.370+0000 7f9284dff700 1 --2- 172.21.15.31:0/2065838283 >> v2:172.21.15.62:3300/0 conn(0x7f9280159930 0x7f92800cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.370+0000 7f9284dff700 1 -- 172.21.15.31:0/2065838283 >> 172.21.15.31:0/2065838283 conn(0x7f928006ec90 msgr2=0x7f928006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:16.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.370+0000 7f9284dff700 1 -- 172.21.15.31:0/2065838283 shutdown_connections 2023-10-06T05:17:16.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.370+0000 7f9284dff700 1 -- 172.21.15.31:0/2065838283 wait complete. 2023-10-06T05:17:16.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.371+0000 7f9284dff700 1 Processor -- start 2023-10-06T05:17:16.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.371+0000 7f9284dff700 1 -- start start 2023-10-06T05:17:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.372+0000 7f9284dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92800b0d60 0x7f92800c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.372+0000 7f9284dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9280159560 0x7f92800c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.372+0000 7f9284dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9280159930 0x7f928016e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.372+0000 7f9284dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92800c4c60 con 0x7f92800b0d60 2023-10-06T05:17:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.372+0000 7f9284dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92800c7b70 con 0x7f9280159930 2023-10-06T05:17:16.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.372+0000 7f9284dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928016e840 con 0x7f9280159560 2023-10-06T05:17:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92800b0d60 0x7f92800c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92800b0d60 0x7f92800c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34616/0 (socket says 172.21.15.31:34616) 2023-10-06T05:17:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927e59c700 1 -- 172.21.15.31:0/2351428895 learned_addr learned my addr 172.21.15.31:0/2351428895 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927dd9b700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9280159560 0x7f92800c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927ed9d700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9280159930 0x7f928016e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927e59c700 1 -- 172.21.15.31:0/2351428895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9280159560 msgr2=0x7f92800c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927e59c700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9280159560 0x7f92800c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927e59c700 1 -- 172.21.15.31:0/2351428895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9280159930 msgr2=0x7f928016e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927e59c700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9280159930 0x7f928016e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927e59c700 1 -- 172.21.15.31:0/2351428895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92700094f0 con 0x7f92800b0d60 2023-10-06T05:17:16.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927ed9d700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9280159930 0x7f928016e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:16.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f927e59c700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92800b0d60 0x7f92800c9e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f92700094c0 tx=0x7f9270017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f926e7fc700 1 -- 172.21.15.31:0/2351428895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927000bbe0 con 0x7f92800b0d60 2023-10-06T05:17:16.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f926e7fc700 1 -- 172.21.15.31:0/2351428895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9270011420 con 0x7f92800b0d60 2023-10-06T05:17:16.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f9284dff700 1 -- 172.21.15.31:0/2351428895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928016eae0 con 0x7f92800b0d60 2023-10-06T05:17:16.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.373+0000 7f926e7fc700 1 -- 172.21.15.31:0/2351428895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270011800 con 0x7f92800b0d60 2023-10-06T05:17:16.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.374+0000 7f9284dff700 1 -- 172.21.15.31:0/2351428895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928016f0a0 con 0x7f92800b0d60 2023-10-06T05:17:16.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.375+0000 7f9284dff700 1 -- 172.21.15.31:0/2351428895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92800c4f40 con 0x7f92800b0d60 2023-10-06T05:17:16.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.375+0000 7f926e7fc700 1 -- 172.21.15.31:0/2351428895 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9270011bb0 con 0x7f92800b0d60 2023-10-06T05:17:16.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.376+0000 7f926e7fc700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9260066f30 0x7f92600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.377+0000 7f926e7fc700 1 -- 172.21.15.31:0/2351428895 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9270085f90 con 0x7f92800b0d60 2023-10-06T05:17:16.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.377+0000 7f927dd9b700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9260066f30 0x7f92600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.377+0000 7f927dd9b700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9260066f30 0x7f92600693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f928006ec60 tx=0x7f9274008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.381+0000 7f926e7fc700 1 -- 172.21.15.31:0/2351428895 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9270053000 con 0x7f92800b0d60 2023-10-06T05:17:16.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.582+0000 7f9284dff700 1 -- 172.21.15.31:0/2351428895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f928016f290 con 0x7f92800b0d60 2023-10-06T05:17:16.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.582+0000 7f926e7fc700 1 -- 172.21.15.31:0/2351428895 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9270011e60 con 0x7f92800b0d60 2023-10-06T05:17:16.585 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:17:16.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.584+0000 7f9284dff700 1 -- 172.21.15.31:0/2351428895 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9260066f30 msgr2=0x7f92600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.584+0000 7f9284dff700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9260066f30 0x7f92600693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f928006ec60 tx=0x7f9274008040 comp rx=0 tx=0).stop 2023-10-06T05:17:16.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.584+0000 7f9284dff700 1 -- 172.21.15.31:0/2351428895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92800b0d60 msgr2=0x7f92800c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.584+0000 7f9284dff700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92800b0d60 0x7f92800c9e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f92700094c0 tx=0x7f9270017000 comp rx=0 tx=0).stop 2023-10-06T05:17:16.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.585+0000 7f9284dff700 1 -- 172.21.15.31:0/2351428895 shutdown_connections 2023-10-06T05:17:16.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.585+0000 7f9284dff700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9280159930 0x7f928016e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.585+0000 7f9284dff700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9260066f30 0x7f92600693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.585+0000 7f9284dff700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92800b0d60 0x7f92800c9e60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.585+0000 7f9284dff700 1 --2- 172.21.15.31:0/2351428895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9280159560 0x7f92800c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.585+0000 7f9284dff700 1 -- 172.21.15.31:0/2351428895 >> 172.21.15.31:0/2351428895 conn(0x7f928006ec90 msgr2=0x7f9280158540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:16.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.585+0000 7f9284dff700 1 -- 172.21.15.31:0/2351428895 shutdown_connections 2023-10-06T05:17:16.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.585+0000 7f9284dff700 1 -- 172.21.15.31:0/2351428895 wait complete. 2023-10-06T05:17:16.604 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:16.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.800+0000 7f3f189ff700 1 Processor -- start 2023-10-06T05:17:16.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.800+0000 7f3f189ff700 1 -- start start 2023-10-06T05:17:16.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.801+0000 7f3f189ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f14159560 0x7f3f140cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.801+0000 7f3f189ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3f140b2b80 0x7f3f140cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.801+0000 7f3f189ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1415d2d0 con 0x7f3f14159930 2023-10-06T05:17:16.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.801+0000 7f3f189ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1415d5b0 con 0x7f3f14159560 2023-10-06T05:17:16.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.801+0000 7f3f189ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1415d890 con 0x7f3f140b2b80 2023-10-06T05:17:16.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.801+0000 7f3f1259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f14159560 0x7f3f140cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.801+0000 7f3f11d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3f14159930 0x7f3f140b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44878/0 (socket says 172.21.15.31:44878) 2023-10-06T05:17:16.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.801+0000 7f3f11d9b700 1 -- 172.21.15.31:0/831106882 learned_addr learned my addr 172.21.15.31:0/831106882 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:16.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.801+0000 7f3f12d9d700 1 --2- 172.21.15.31:0/831106882 >> v2:172.21.15.62:3300/0 conn(0x7f3f140b2b80 0x7f3f140cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.802+0000 7f3f1259c700 1 -- 172.21.15.31:0/831106882 >> v2:172.21.15.62:3300/0 conn(0x7f3f140b2b80 msgr2=0x7f3f140cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:16.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.802+0000 7f3f1259c700 1 --2- 172.21.15.31:0/831106882 >> v2:172.21.15.62:3300/0 conn(0x7f3f140b2b80 0x7f3f140cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.802+0000 7f3f1259c700 1 -- 172.21.15.31:0/831106882 >> v1:172.21.15.31:6789/0 conn(0x7f3f14159930 legacy=0x7f3f140b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.802+0000 7f3f1259c700 1 -- 172.21.15.31:0/831106882 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1415db70 con 0x7f3f14159560 2023-10-06T05:17:16.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.802+0000 7f3f12d9d700 1 --2- 172.21.15.31:0/831106882 >> v2:172.21.15.62:3300/0 conn(0x7f3f140b2b80 0x7f3f140cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.802+0000 7f3f1159a700 1 -- 172.21.15.31:0/831106882 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280894993 0 0) 0x7f3f1415d2d0 con 0x7f3f14159930 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.802+0000 7f3f1259c700 1 --2- 172.21.15.31:0/831106882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159560 0x7f3f140cce70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f140b1740 tx=0x7f3f040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64635b7ddc8b59fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.802+0000 7f3f1159a700 1 -- 172.21.15.31:0/831106882 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0400b6b0 con 0x7f3f14159560 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.802+0000 7f3f1159a700 1 -- 172.21.15.31:0/831106882 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0400b810 con 0x7f3f14159560 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.802+0000 7f3f1159a700 1 -- 172.21.15.31:0/831106882 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0400bbb0 con 0x7f3f14159560 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.803+0000 7f3f189ff700 1 -- 172.21.15.31:0/831106882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159560 msgr2=0x7f3f140cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.803+0000 7f3f189ff700 1 --2- 172.21.15.31:0/831106882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159560 0x7f3f140cce70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f140b1740 tx=0x7f3f040091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.803+0000 7f3f189ff700 1 -- 172.21.15.31:0/831106882 shutdown_connections 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.803+0000 7f3f189ff700 1 --2- 172.21.15.31:0/831106882 >> v2:172.21.15.62:3300/0 conn(0x7f3f140b2b80 0x7f3f140cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.803+0000 7f3f189ff700 1 --2- 172.21.15.31:0/831106882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159560 0x7f3f140cce70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.803+0000 7f3f189ff700 1 -- 172.21.15.31:0/831106882 >> 172.21.15.31:0/831106882 conn(0x7f3f1406ec90 msgr2=0x7f3f1406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:16.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.803+0000 7f3f189ff700 1 -- 172.21.15.31:0/831106882 shutdown_connections 2023-10-06T05:17:16.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.803+0000 7f3f189ff700 1 -- 172.21.15.31:0/831106882 wait complete. 2023-10-06T05:17:16.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.804+0000 7f3f189ff700 1 Processor -- start 2023-10-06T05:17:16.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.805+0000 7f3f189ff700 1 -- start start 2023-10-06T05:17:16.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.805+0000 7f3f189ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f140b2b80 0x7f3f14155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.805+0000 7f3f189ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f14159560 0x7f3f140c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.805+0000 7f3f189ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159930 0x7f3f1416a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.805+0000 7f3f189ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140c56e0 con 0x7f3f14159930 2023-10-06T05:17:16.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.805+0000 7f3f189ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1415bf40 con 0x7f3f14159560 2023-10-06T05:17:16.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.805+0000 7f3f189ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140c43c0 con 0x7f3f140b2b80 2023-10-06T05:17:16.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f12d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159930 0x7f3f1416a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f12d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159930 0x7f3f1416a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34630/0 (socket says 172.21.15.31:34630) 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f12d9d700 1 -- 172.21.15.31:0/3980458703 learned_addr learned my addr 172.21.15.31:0/3980458703 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f11d9b700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f14159560 0x7f3f140c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f12d9d700 1 -- 172.21.15.31:0/3980458703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f140b2b80 msgr2=0x7f3f14155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f12d9d700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f140b2b80 0x7f3f14155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f12d9d700 1 -- 172.21.15.31:0/3980458703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f14159560 msgr2=0x7f3f140c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f12d9d700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f14159560 0x7f3f140c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f12d9d700 1 -- 172.21.15.31:0/3980458703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04009420 con 0x7f3f14159930 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f11d9b700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f14159560 0x7f3f140c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.806+0000 7f3f12d9d700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159930 0x7f3f1416a530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00d0d0 tx=0x7f3f0c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.807+0000 7f3f027fc700 1 -- 172.21.15.31:0/3980458703 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00dd70 con 0x7f3f14159930 2023-10-06T05:17:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.807+0000 7f3f027fc700 1 -- 172.21.15.31:0/3980458703 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c00f040 con 0x7f3f14159930 2023-10-06T05:17:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.807+0000 7f3f027fc700 1 -- 172.21.15.31:0/3980458703 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c015670 con 0x7f3f14159930 2023-10-06T05:17:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.807+0000 7f3f189ff700 1 -- 172.21.15.31:0/3980458703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1415b8f0 con 0x7f3f14159930 2023-10-06T05:17:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.807+0000 7f3f189ff700 1 -- 172.21.15.31:0/3980458703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1416aa70 con 0x7f3f14159930 2023-10-06T05:17:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.808+0000 7f3f189ff700 1 -- 172.21.15.31:0/3980458703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1404c380 con 0x7f3f14159930 2023-10-06T05:17:16.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.809+0000 7f3f027fc700 1 -- 172.21.15.31:0/3980458703 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f0c01e3b0 con 0x7f3f14159930 2023-10-06T05:17:16.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.809+0000 7f3f027fc700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 0x7f3ef40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:16.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.810+0000 7f3f027fc700 1 -- 172.21.15.31:0/3980458703 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3f0c084cb0 con 0x7f3f14159930 2023-10-06T05:17:16.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.810+0000 7f3f1259c700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 0x7f3ef40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:16.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.810+0000 7f3f1259c700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 0x7f3ef40693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f140c9740 tx=0x7f3f04017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:16.814+0000 7f3f027fc700 1 -- 172.21.15.31:0/3980458703 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f0c051d20 con 0x7f3f14159930 2023-10-06T05:17:17.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.016+0000 7f3f189ff700 1 -- 172.21.15.31:0/3980458703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f1415de50 con 0x7f3f14159930 2023-10-06T05:17:17.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.016+0000 7f3f027fc700 1 -- 172.21.15.31:0/3980458703 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f0c01a0a0 con 0x7f3f14159930 2023-10-06T05:17:17.018 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:17:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.018+0000 7f3f189ff700 1 -- 172.21.15.31:0/3980458703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 msgr2=0x7f3ef40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.018+0000 7f3f189ff700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 0x7f3ef40693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f140c9740 tx=0x7f3f04017000 comp rx=0 tx=0).stop 2023-10-06T05:17:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.019+0000 7f3f189ff700 1 -- 172.21.15.31:0/3980458703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159930 msgr2=0x7f3f1416a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.019+0000 7f3f189ff700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159930 0x7f3f1416a530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00d0d0 tx=0x7f3f0c009a50 comp rx=0 tx=0).stop 2023-10-06T05:17:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.019+0000 7f3f189ff700 1 -- 172.21.15.31:0/3980458703 shutdown_connections 2023-10-06T05:17:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.019+0000 7f3f189ff700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 0x7f3ef40693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.019+0000 7f3f189ff700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f14159930 0x7f3f1416a530 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.019+0000 7f3f189ff700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f140b2b80 0x7f3f14155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.019+0000 7f3f189ff700 1 --2- 172.21.15.31:0/3980458703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f14159560 0x7f3f140c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.019+0000 7f3f189ff700 1 -- 172.21.15.31:0/3980458703 >> 172.21.15.31:0/3980458703 conn(0x7f3f1406ec90 msgr2=0x7f3f141566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.019+0000 7f3f189ff700 1 -- 172.21.15.31:0/3980458703 shutdown_connections 2023-10-06T05:17:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.019+0000 7f3f189ff700 1 -- 172.21.15.31:0/3980458703 wait complete. 2023-10-06T05:17:17.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:17:17.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.235+0000 7fdf321ff700 1 Processor -- start 2023-10-06T05:17:17.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.235+0000 7fdf321ff700 1 -- start start 2023-10-06T05:17:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.237+0000 7fdf321ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdf2c155b40 0x7fdf2c0c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.237+0000 7fdf321ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c158190 con 0x7fdf2c159890 2023-10-06T05:17:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.237+0000 7fdf321ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c157580 con 0x7fdf2c159c60 2023-10-06T05:17:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.237+0000 7fdf321ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c157860 con 0x7fdf2c155b40 2023-10-06T05:17:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.237+0000 7fdf2b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdf2c159890 0x7fdf2c156a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58666/0 (socket says 172.21.15.31:58666) 2023-10-06T05:17:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.237+0000 7fdf2b7fe700 1 -- 172.21.15.31:0/916854381 learned_addr learned my addr 172.21.15.31:0/916854381 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.237+0000 7fdf2bfff700 1 --2- 172.21.15.31:0/916854381 >> v2:172.21.15.31:3300/0 conn(0x7fdf2c155b40 0x7fdf2c0c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.237+0000 7fdf2a7fc700 1 -- 172.21.15.31:0/916854381 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593549171 0 0) 0x7fdf2c158190 con 0x7fdf2c159890 2023-10-06T05:17:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.238+0000 7fdf2a7fc700 1 -- 172.21.15.31:0/916854381 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf14003570 con 0x7fdf2c159890 2023-10-06T05:17:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.238+0000 7fdf2bfff700 1 -- 172.21.15.31:0/916854381 >> v1:172.21.15.62:6789/0 conn(0x7fdf2c159c60 legacy=0x7fdf2c155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.238+0000 7fdf2bfff700 1 -- 172.21.15.31:0/916854381 >> v1:172.21.15.31:6789/0 conn(0x7fdf2c159890 legacy=0x7fdf2c156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.238+0000 7fdf2bfff700 1 -- 172.21.15.31:0/916854381 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c157b40 con 0x7fdf2c155b40 2023-10-06T05:17:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.238+0000 7fdf2bfff700 1 --2- 172.21.15.31:0/916854381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c155b40 0x7fdf2c0c89e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400af70 tx=0x7fdf2400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9901feae4014676 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.238+0000 7fdf2a7fc700 1 -- 172.21.15.31:0/916854381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400f070 con 0x7fdf2c155b40 2023-10-06T05:17:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.238+0000 7fdf2a7fc700 1 -- 172.21.15.31:0/916854381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2400a2e0 con 0x7fdf2c155b40 2023-10-06T05:17:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.238+0000 7fdf2a7fc700 1 -- 172.21.15.31:0/916854381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400cc70 con 0x7fdf2c155b40 2023-10-06T05:17:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.239+0000 7fdf321ff700 1 -- 172.21.15.31:0/916854381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c155b40 msgr2=0x7fdf2c0c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.239+0000 7fdf321ff700 1 --2- 172.21.15.31:0/916854381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c155b40 0x7fdf2c0c89e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400af70 tx=0x7fdf2400a890 comp rx=0 tx=0).stop 2023-10-06T05:17:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.239+0000 7fdf321ff700 1 -- 172.21.15.31:0/916854381 shutdown_connections 2023-10-06T05:17:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.239+0000 7fdf321ff700 1 --2- 172.21.15.31:0/916854381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c155b40 0x7fdf2c0c89e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.239+0000 7fdf321ff700 1 -- 172.21.15.31:0/916854381 >> 172.21.15.31:0/916854381 conn(0x7fdf2c067560 msgr2=0x7fdf2c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.239+0000 7fdf321ff700 1 -- 172.21.15.31:0/916854381 shutdown_connections 2023-10-06T05:17:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.239+0000 7fdf321ff700 1 -- 172.21.15.31:0/916854381 wait complete. 2023-10-06T05:17:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.240+0000 7fdf321ff700 1 Processor -- start 2023-10-06T05:17:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.240+0000 7fdf321ff700 1 -- start start 2023-10-06T05:17:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.241+0000 7fdf321ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c155b40 0x7fdf2c0c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.241+0000 7fdf321ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf2c159890 0x7fdf2c166ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.241+0000 7fdf321ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c159c60 0x7fdf2c0c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.241+0000 7fdf321ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c15b200 con 0x7fdf2c159c60 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.241+0000 7fdf321ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c15b4e0 con 0x7fdf2c155b40 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.241+0000 7fdf321ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c15b7c0 con 0x7fdf2c159890 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.241+0000 7fdf2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c159c60 0x7fdf2c0c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c159c60 0x7fdf2c0c1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51056/0 (socket says 172.21.15.31:51056) 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf2bfff700 1 -- 172.21.15.31:0/1443528839 learned_addr learned my addr 172.21.15.31:0/1443528839 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf2b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c155b40 0x7fdf2c0c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf2affd700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf2c159890 0x7fdf2c166ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf2bfff700 1 -- 172.21.15.31:0/1443528839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf2c159890 msgr2=0x7fdf2c166ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf2bfff700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf2c159890 0x7fdf2c166ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf2bfff700 1 -- 172.21.15.31:0/1443528839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c155b40 msgr2=0x7fdf2c0c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf2bfff700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c155b40 0x7fdf2c0c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf2bfff700 1 -- 172.21.15.31:0/1443528839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2400ab30 con 0x7fdf2c159c60 2023-10-06T05:17:17.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf2bfff700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c159c60 0x7fdf2c0c1b10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400ab00 tx=0x7fdf24019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf137fe700 1 -- 172.21.15.31:0/1443528839 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24009ad0 con 0x7fdf2c159c60 2023-10-06T05:17:17.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf137fe700 1 -- 172.21.15.31:0/1443528839 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf24009c70 con 0x7fdf2c159c60 2023-10-06T05:17:17.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf321ff700 1 -- 172.21.15.31:0/1443528839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c15baa0 con 0x7fdf2c159c60 2023-10-06T05:17:17.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.242+0000 7fdf137fe700 1 -- 172.21.15.31:0/1443528839 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2401d440 con 0x7fdf2c159c60 2023-10-06T05:17:17.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.243+0000 7fdf321ff700 1 -- 172.21.15.31:0/1443528839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c1673e0 con 0x7fdf2c159c60 2023-10-06T05:17:17.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.244+0000 7fdf321ff700 1 -- 172.21.15.31:0/1443528839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c0ae730 con 0x7fdf2c159c60 2023-10-06T05:17:17.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.244+0000 7fdf137fe700 1 -- 172.21.15.31:0/1443528839 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf2401d7b0 con 0x7fdf2c159c60 2023-10-06T05:17:17.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.245+0000 7fdf137fe700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf18066f30 0x7fdf180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.245+0000 7fdf137fe700 1 -- 172.21.15.31:0/1443528839 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdf240860c0 con 0x7fdf2c159c60 2023-10-06T05:17:17.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.246+0000 7fdf2b7fe700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf18066f30 0x7fdf180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.246+0000 7fdf2b7fe700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf18066f30 0x7fdf180693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c007fd0 tx=0x7fdf1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.250+0000 7fdf137fe700 1 -- 172.21.15.31:0/1443528839 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf24053130 con 0x7fdf2c159c60 2023-10-06T05:17:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.453+0000 7fdf321ff700 1 -- 172.21.15.31:0/1443528839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf2c04c380 con 0x7fdf2c159c60 2023-10-06T05:17:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.454+0000 7fdf137fe700 1 -- 172.21.15.31:0/1443528839 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf24056750 con 0x7fdf2c159c60 2023-10-06T05:17:17.455 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:17:17.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 -- 172.21.15.31:0/1443528839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf18066f30 msgr2=0x7fdf180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf18066f30 0x7fdf180693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c007fd0 tx=0x7fdf1c009040 comp rx=0 tx=0).stop 2023-10-06T05:17:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 -- 172.21.15.31:0/1443528839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c159c60 msgr2=0x7fdf2c0c1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c159c60 0x7fdf2c0c1b10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400ab00 tx=0x7fdf24019000 comp rx=0 tx=0).stop 2023-10-06T05:17:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 -- 172.21.15.31:0/1443528839 shutdown_connections 2023-10-06T05:17:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf18066f30 0x7fdf180693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf2c159c60 0x7fdf2c0c1b10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c155b40 0x7fdf2c0c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 --2- 172.21.15.31:0/1443528839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf2c159890 0x7fdf2c166ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 -- 172.21.15.31:0/1443528839 >> 172.21.15.31:0/1443528839 conn(0x7fdf2c067560 msgr2=0x7fdf2c158620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 -- 172.21.15.31:0/1443528839 shutdown_connections 2023-10-06T05:17:17.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.456+0000 7fdf321ff700 1 -- 172.21.15.31:0/1443528839 wait complete. 2023-10-06T05:17:17.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:17:17.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.672+0000 7f7c091ff700 1 Processor -- start 2023-10-06T05:17:17.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.672+0000 7f7c091ff700 1 -- start start 2023-10-06T05:17:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.673+0000 7f7c091ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159560 0x7f7c040cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.673+0000 7f7c091ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7c04159930 0x7f7c040cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.673+0000 7f7c091ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0415d2d0 con 0x7f7c040b0d60 2023-10-06T05:17:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.673+0000 7f7c091ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0415d5b0 con 0x7f7c04159930 2023-10-06T05:17:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.673+0000 7f7c091ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0415d890 con 0x7f7c04159560 2023-10-06T05:17:17.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c0259c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7c04159930 0x7f7c040cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c02d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159560 0x7f7c040cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c0359e700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7c040b0d60 0x7f7c040b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:57664/0 (socket says 172.21.15.31:57664) 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c0259c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7c04159930 0x7f7c040cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:56050/0 (socket says 172.21.15.31:56050) 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c0359e700 1 -- 172.21.15.31:0/2281949945 learned_addr learned my addr 172.21.15.31:0/2281949945 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c01d9b700 1 -- 172.21.15.31:0/2281949945 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583044941 0 0) 0x7f7c0415d2d0 con 0x7f7c040b0d60 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c01d9b700 1 -- 172.21.15.31:0/2281949945 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bec003570 con 0x7f7c040b0d60 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c02d9d700 1 -- 172.21.15.31:0/2281949945 >> v2:172.21.15.62:3300/0 conn(0x7f7c04159930 msgr2=0x7f7c040cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c02d9d700 1 --2- 172.21.15.31:0/2281949945 >> v2:172.21.15.62:3300/0 conn(0x7f7c04159930 0x7f7c040cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c02d9d700 1 -- 172.21.15.31:0/2281949945 >> v1:172.21.15.62:6789/0 conn(0x7f7c040b0d60 legacy=0x7f7c040b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c02d9d700 1 -- 172.21.15.31:0/2281949945 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0415db70 con 0x7f7c04159560 2023-10-06T05:17:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.674+0000 7f7c0259c700 1 --2- 172.21.15.31:0/2281949945 >> v2:172.21.15.62:3300/0 conn(0x7f7c04159930 0x7f7c040cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.675+0000 7f7c02d9d700 1 --2- 172.21.15.31:0/2281949945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159560 0x7f7c040cce70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40098e0 tx=0x7f7bf40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4565c7f0a4bd2acd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.675+0000 7f7c01d9b700 1 -- 172.21.15.31:0/2281949945 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4011630 con 0x7f7c04159560 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.675+0000 7f7c01d9b700 1 -- 172.21.15.31:0/2281949945 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf4011790 con 0x7f7c04159560 2023-10-06T05:17:17.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.675+0000 7f7c01d9b700 1 -- 172.21.15.31:0/2281949945 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4011b30 con 0x7f7c04159560 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.675+0000 7f7c091ff700 1 -- 172.21.15.31:0/2281949945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159560 msgr2=0x7f7c040cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.675+0000 7f7c091ff700 1 --2- 172.21.15.31:0/2281949945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159560 0x7f7c040cce70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40098e0 tx=0x7f7bf40091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.676+0000 7f7c091ff700 1 -- 172.21.15.31:0/2281949945 shutdown_connections 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.676+0000 7f7c091ff700 1 --2- 172.21.15.31:0/2281949945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159560 0x7f7c040cce70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.676+0000 7f7c091ff700 1 --2- 172.21.15.31:0/2281949945 >> v2:172.21.15.62:3300/0 conn(0x7f7c04159930 0x7f7c040cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.676+0000 7f7c091ff700 1 -- 172.21.15.31:0/2281949945 >> 172.21.15.31:0/2281949945 conn(0x7f7c0406ec90 msgr2=0x7f7c0406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.676+0000 7f7c091ff700 1 -- 172.21.15.31:0/2281949945 shutdown_connections 2023-10-06T05:17:17.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.676+0000 7f7c091ff700 1 -- 172.21.15.31:0/2281949945 wait complete. 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.678+0000 7f7c091ff700 1 Processor -- start 2023-10-06T05:17:17.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.678+0000 7f7c091ff700 1 -- start start 2023-10-06T05:17:17.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.679+0000 7f7c091ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c040b0d60 0x7f7c0415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.679+0000 7f7c091ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c04159560 0x7f7c04154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.679+0000 7f7c091ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159930 0x7f7c041553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.679+0000 7f7c091ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04155dc0 con 0x7f7c04159560 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.679+0000 7f7c091ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040ce8f0 con 0x7f7c040b0d60 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.679+0000 7f7c091ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040c4280 con 0x7f7c04159930 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.679+0000 7f7c0259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c04159560 0x7f7c04154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c0259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c04159560 0x7f7c04154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51060/0 (socket says 172.21.15.31:51060) 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c0259c700 1 -- 172.21.15.31:0/194526641 learned_addr learned my addr 172.21.15.31:0/194526641 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c02d9d700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c040b0d60 0x7f7c0415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c0359e700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159930 0x7f7c041553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c0259c700 1 -- 172.21.15.31:0/194526641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159930 msgr2=0x7f7c041553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c0259c700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159930 0x7f7c041553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c0259c700 1 -- 172.21.15.31:0/194526641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c040b0d60 msgr2=0x7f7c0415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c0259c700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c040b0d60 0x7f7c0415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c0259c700 1 -- 172.21.15.31:0/194526641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf8009420 con 0x7f7c04159560 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c0359e700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159930 0x7f7c041553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7c0259c700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c04159560 0x7f7c04154e80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800ec80 tx=0x7f7bf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.680+0000 7f7beaffd700 1 -- 172.21.15.31:0/194526641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800c850 con 0x7f7c04159560 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.681+0000 7f7beaffd700 1 -- 172.21.15.31:0/194526641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf800c9b0 con 0x7f7c04159560 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.681+0000 7f7beaffd700 1 -- 172.21.15.31:0/194526641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800cd30 con 0x7f7c04159560 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.681+0000 7f7c091ff700 1 -- 172.21.15.31:0/194526641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf4009420 con 0x7f7c04159560 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.681+0000 7f7c091ff700 1 -- 172.21.15.31:0/194526641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c040ce2a0 con 0x7f7c04159560 2023-10-06T05:17:17.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.682+0000 7f7c091ff700 1 -- 172.21.15.31:0/194526641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c040ceaa0 con 0x7f7c04159560 2023-10-06T05:17:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.683+0000 7f7beaffd700 1 -- 172.21.15.31:0/194526641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7bf8013680 con 0x7f7c04159560 2023-10-06T05:17:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.684+0000 7f7beaffd700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7bf0066f30 0x7f7bf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.684+0000 7f7beaffd700 1 -- 172.21.15.31:0/194526641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7bf8084d30 con 0x7f7c04159560 2023-10-06T05:17:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.684+0000 7f7c02d9d700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7bf0066f30 0x7f7bf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:17.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.684+0000 7f7c02d9d700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7bf0066f30 0x7f7bf00693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40091e0 tx=0x7f7bf4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.687+0000 7f7beaffd700 1 -- 172.21.15.31:0/194526641 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c040ceaa0 con 0x7f7c04159560 2023-10-06T05:17:17.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.888+0000 7f7c091ff700 1 -- 172.21.15.31:0/194526641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c040cea60 con 0x7f7c04159560 2023-10-06T05:17:17.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.888+0000 7f7beaffd700 1 -- 172.21.15.31:0/194526641 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7bf80170a0 con 0x7f7c04159560 2023-10-06T05:17:17.890 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:17:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 -- 172.21.15.31:0/194526641 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7bf0066f30 msgr2=0x7f7bf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7bf0066f30 0x7f7bf00693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40091e0 tx=0x7f7bf4017000 comp rx=0 tx=0).stop 2023-10-06T05:17:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 -- 172.21.15.31:0/194526641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c04159560 msgr2=0x7f7c04154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c04159560 0x7f7c04154e80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800ec80 tx=0x7f7bf80091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 -- 172.21.15.31:0/194526641 shutdown_connections 2023-10-06T05:17:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c04159930 0x7f7c041553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c040b0d60 0x7f7c0415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7bf0066f30 0x7f7bf00693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 --2- 172.21.15.31:0/194526641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c04159560 0x7f7c04154e80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:17.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 -- 172.21.15.31:0/194526641 >> 172.21.15.31:0/194526641 conn(0x7f7c0406ec90 msgr2=0x7f7c040b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:17.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 -- 172.21.15.31:0/194526641 shutdown_connections 2023-10-06T05:17:17.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:17.891+0000 7f7c091ff700 1 -- 172.21.15.31:0/194526641 wait complete. 2023-10-06T05:17:17.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:18.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.105+0000 7f059719d700 1 Processor -- start 2023-10-06T05:17:18.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.105+0000 7f059719d700 1 -- start start 2023-10-06T05:17:18.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.107+0000 7f059719d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05900ceae0 0x7f05900b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.107+0000 7f059719d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05900cbf40 con 0x7f05901597b0 2023-10-06T05:17:18.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.107+0000 7f059719d700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05900cc220 con 0x7f05900ceeb0 2023-10-06T05:17:18.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.107+0000 7f059719d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05900cc500 con 0x7f05900ceae0 2023-10-06T05:17:18.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.107+0000 7f059619b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05900ceae0 0x7f05900b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.107+0000 7f059699c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f05901597b0 0x7f05900cb1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58676/0 (socket says 172.21.15.31:58676) 2023-10-06T05:17:18.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.107+0000 7f059619b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05900ceae0 0x7f05900b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51066/0 (socket says 172.21.15.31:51066) 2023-10-06T05:17:18.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.107+0000 7f059699c700 1 -- 172.21.15.31:0/3327763489 learned_addr learned my addr 172.21.15.31:0/3327763489 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.107+0000 7f059619b700 1 -- 172.21.15.31:0/3327763489 >> v1:172.21.15.62:6789/0 conn(0x7f05900ceeb0 legacy=0x7f05901590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.108+0000 7f059619b700 1 -- 172.21.15.31:0/3327763489 >> v1:172.21.15.31:6789/0 conn(0x7f05901597b0 legacy=0x7f05900cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.108+0000 7f0595199700 1 -- 172.21.15.31:0/3327763489 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471942142 0 0) 0x7f05900cbf40 con 0x7f05901597b0 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.108+0000 7f059619b700 1 -- 172.21.15.31:0/3327763489 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05900cc7e0 con 0x7f05900ceae0 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.108+0000 7f059619b700 1 --2- 172.21.15.31:0/3327763489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceae0 0x7f05900b30e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f058c0098d0 tx=0x7f058c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29021cfdef9fa7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.108+0000 7f0595199700 1 -- 172.21.15.31:0/3327763489 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c009e70 con 0x7f05900ceae0 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.108+0000 7f0595199700 1 -- 172.21.15.31:0/3327763489 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c005c10 con 0x7f05900ceae0 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.108+0000 7f0595199700 1 -- 172.21.15.31:0/3327763489 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00b650 con 0x7f05900ceae0 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.109+0000 7f059719d700 1 -- 172.21.15.31:0/3327763489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceae0 msgr2=0x7f05900b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.109+0000 7f059719d700 1 --2- 172.21.15.31:0/3327763489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceae0 0x7f05900b30e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f058c0098d0 tx=0x7f058c009120 comp rx=0 tx=0).stop 2023-10-06T05:17:18.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.109+0000 7f059719d700 1 -- 172.21.15.31:0/3327763489 shutdown_connections 2023-10-06T05:17:18.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.109+0000 7f059719d700 1 --2- 172.21.15.31:0/3327763489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceae0 0x7f05900b30e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.109+0000 7f059719d700 1 -- 172.21.15.31:0/3327763489 >> 172.21.15.31:0/3327763489 conn(0x7f0590099560 msgr2=0x7f059006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:18.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.109+0000 7f059719d700 1 -- 172.21.15.31:0/3327763489 shutdown_connections 2023-10-06T05:17:18.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.109+0000 7f059719d700 1 -- 172.21.15.31:0/3327763489 wait complete. 2023-10-06T05:17:18.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.110+0000 7f059719d700 1 Processor -- start 2023-10-06T05:17:18.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.111+0000 7f059719d700 1 -- start start 2023-10-06T05:17:18.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.111+0000 7f059719d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05900ceae0 0x7f05900b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059719d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceeb0 0x7f059016a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059719d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05901597b0 0x7f0590143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059719d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059016a9b0 con 0x7f05900ceeb0 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059719d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05901569b0 con 0x7f05900ceae0 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059719d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05900b3c40 con 0x7f05901597b0 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceeb0 0x7f059016a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceeb0 0x7f059016a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51072/0 (socket says 172.21.15.31:51072) 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059599a700 1 -- 172.21.15.31:0/4210427479 learned_addr learned my addr 172.21.15.31:0/4210427479 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059619b700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05900ceae0 0x7f05900b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059699c700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05901597b0 0x7f0590143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059599a700 1 -- 172.21.15.31:0/4210427479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05901597b0 msgr2=0x7f0590143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059599a700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05901597b0 0x7f0590143c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059599a700 1 -- 172.21.15.31:0/4210427479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05900ceae0 msgr2=0x7f05900b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059599a700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05900ceae0 0x7f05900b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.112+0000 7f059599a700 1 -- 172.21.15.31:0/4210427479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c0093c0 con 0x7f05900ceeb0 2023-10-06T05:17:18.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.113+0000 7f059599a700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceeb0 0x7f059016a470 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f058000c5c0 tx=0x7f058000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.113+0000 7f05867fc700 1 -- 172.21.15.31:0/4210427479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580011590 con 0x7f05900ceeb0 2023-10-06T05:17:18.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.113+0000 7f05867fc700 1 -- 172.21.15.31:0/4210427479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0580011710 con 0x7f05900ceeb0 2023-10-06T05:17:18.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.113+0000 7f059719d700 1 -- 172.21.15.31:0/4210427479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05900b3f20 con 0x7f05900ceeb0 2023-10-06T05:17:18.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.113+0000 7f05867fc700 1 -- 172.21.15.31:0/4210427479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580011ad0 con 0x7f05900ceeb0 2023-10-06T05:17:18.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.113+0000 7f059719d700 1 -- 172.21.15.31:0/4210427479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05900b42b0 con 0x7f05900ceeb0 2023-10-06T05:17:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.114+0000 7f059719d700 1 -- 172.21.15.31:0/4210427479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0590063720 con 0x7f05900ceeb0 2023-10-06T05:17:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.115+0000 7f05867fc700 1 -- 172.21.15.31:0/4210427479 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f058001b3b0 con 0x7f05900ceeb0 2023-10-06T05:17:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.116+0000 7f05867fc700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0574066f30 0x7f05740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.116+0000 7f05867fc700 1 -- 172.21.15.31:0/4210427479 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0580083ba0 con 0x7f05900ceeb0 2023-10-06T05:17:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.116+0000 7f059619b700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0574066f30 0x7f05740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.116+0000 7f059619b700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0574066f30 0x7f05740693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f058c009160 tx=0x7f058c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.120+0000 7f05867fc700 1 -- 172.21.15.31:0/4210427479 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0580050c10 con 0x7f05900ceeb0 2023-10-06T05:17:18.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.367+0000 7f059719d700 1 -- 172.21.15.31:0/4210427479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f059004c380 con 0x7f05900ceeb0 2023-10-06T05:17:18.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.368+0000 7f05867fc700 1 -- 172.21.15.31:0/4210427479 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f0580017070 con 0x7f05900ceeb0 2023-10-06T05:17:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:18.370 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1001195925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1001195925},{"type":"v1","addr":"172.21.15.62:6835","nonce":1001195925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1336279472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1336279472},{"type":"v1","addr":"172.21.15.31:6837","nonce":1336279472}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/577485679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":577485679},{"type":"v1","addr":"172.21.15.62:6834","nonce":577485679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:01.930460+0000","modified":"2023-10-06T05:17:05.459675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4169737410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169737410},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169737410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:17:18.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.370+0000 7f059719d700 1 -- 172.21.15.31:0/4210427479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0574066f30 msgr2=0x7f05740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.370+0000 7f059719d700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0574066f30 0x7f05740693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f058c009160 tx=0x7f058c016000 comp rx=0 tx=0).stop 2023-10-06T05:17:18.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.370+0000 7f059719d700 1 -- 172.21.15.31:0/4210427479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceeb0 msgr2=0x7f059016a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.370+0000 7f059719d700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceeb0 0x7f059016a470 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f058000c5c0 tx=0x7f058000c8d0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.370+0000 7f059719d700 1 -- 172.21.15.31:0/4210427479 shutdown_connections 2023-10-06T05:17:18.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.370+0000 7f059719d700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05901597b0 0x7f0590143c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.370+0000 7f059719d700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05900ceae0 0x7f05900b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.370+0000 7f059719d700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0574066f30 0x7f05740693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.370+0000 7f059719d700 1 --2- 172.21.15.31:0/4210427479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05900ceeb0 0x7f059016a470 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.370+0000 7f059719d700 1 -- 172.21.15.31:0/4210427479 >> 172.21.15.31:0/4210427479 conn(0x7f0590099560 msgr2=0x7f05900680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:18.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.371+0000 7f059719d700 1 -- 172.21.15.31:0/4210427479 shutdown_connections 2023-10-06T05:17:18.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.371+0000 7f059719d700 1 -- 172.21.15.31:0/4210427479 wait complete. 2023-10-06T05:17:18.374 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T05:17:18.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:18.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.585+0000 7f06dc9ff700 1 Processor -- start 2023-10-06T05:17:18.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.585+0000 7f06dc9ff700 1 -- start start 2023-10-06T05:17:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.586+0000 7f06dc9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f06d81595a0 0x7f06d8159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.586+0000 7f06dc9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d80c95b0 con 0x7f06d80cc0d0 2023-10-06T05:17:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.586+0000 7f06dc9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d80c9890 con 0x7f06d8068d80 2023-10-06T05:17:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.586+0000 7f06dc9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d80c9b70 con 0x7f06d81595a0 2023-10-06T05:17:18.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.586+0000 7f06d5d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f06d80cc0d0 0x7f06d8068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58680/0 (socket says 172.21.15.31:58680) 2023-10-06T05:17:18.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.586+0000 7f06d5d9b700 1 -- 172.21.15.31:0/1380633747 learned_addr learned my addr 172.21.15.31:0/1380633747 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:18.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d659c700 1 --2- 172.21.15.31:0/1380633747 >> v2:172.21.15.62:3300/0 conn(0x7f06d81595a0 0x7f06d8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280256271 0 0) 0x7f06d80c95b0 con 0x7f06d80cc0d0 2023-10-06T05:17:18.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06c0003570 con 0x7f06d80cc0d0 2023-10-06T05:17:18.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468672944 0 0) 0x7f06d80c9890 con 0x7f06d8068d80 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06d80c95b0 con 0x7f06d8068d80 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 599568097 0 0) 0x7f06c0003570 con 0x7f06d80cc0d0 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 >> v2:172.21.15.62:3300/0 conn(0x7f06d81595a0 msgr2=0x7f06d8159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 --2- 172.21.15.31:0/1380633747 >> v2:172.21.15.62:3300/0 conn(0x7f06d81595a0 0x7f06d8159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 >> v1:172.21.15.62:6789/0 conn(0x7f06d8068d80 legacy=0x7f06d80c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d80ca130 con 0x7f06d80cc0d0 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f06cc001470 con 0x7f06d80cc0d0 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 >> v1:172.21.15.31:6789/0 conn(0x7f06d80cc0d0 legacy=0x7f06d8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c0003c40 0x7f06c0004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c0004810 0x7f06c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06c0005210 0x7f06c000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80c9b70 con 0x7f06c0003c40 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c000c130 con 0x7f06c0004810 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.587+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c000c2c0 con 0x7f06c0005210 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d659c700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c0003c40 0x7f06c0004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d5d9b700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c0004810 0x7f06c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d6d9d700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06c0005210 0x7f06c000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d659c700 1 -- 172.21.15.31:0/1380633747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06c0005210 msgr2=0x7f06c000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d659c700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06c0005210 0x7f06c000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d659c700 1 -- 172.21.15.31:0/1380633747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c0004810 msgr2=0x7f06c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d659c700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c0004810 0x7f06c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d659c700 1 -- 172.21.15.31:0/1380633747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0003570 con 0x7f06c0003c40 2023-10-06T05:17:18.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d6d9d700 1 -- 172.21.15.31:0/1380633747 reap_dead start 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d659c700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c0003c40 0x7f06c0004030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009c30 tx=0x7f06c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cc188efb929e33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.588+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80095b0 con 0x7f06c0003c40 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.589+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c800e710 con 0x7f06c0003c40 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.589+0000 7f06d559a700 1 -- 172.21.15.31:0/1380633747 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c800ead0 con 0x7f06c0003c40 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.589+0000 7f06dc9ff700 1 -- 172.21.15.31:0/1380633747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c0003c40 msgr2=0x7f06c0004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.589+0000 7f06dc9ff700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c0003c40 0x7f06c0004030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009c30 tx=0x7f06c80091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.589+0000 7f06dc9ff700 1 -- 172.21.15.31:0/1380633747 shutdown_connections 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.589+0000 7f06dc9ff700 1 --2- 172.21.15.31:0/1380633747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c0003c40 0x7f06c0004030 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.589+0000 7f06dc9ff700 1 -- 172.21.15.31:0/1380633747 >> 172.21.15.31:0/1380633747 conn(0x7f06d806b620 msgr2=0x7f06d806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.589+0000 7f06dc9ff700 1 -- 172.21.15.31:0/1380633747 shutdown_connections 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.589+0000 7f06dc9ff700 1 -- 172.21.15.31:0/1380633747 wait complete. 2023-10-06T05:17:18.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.591+0000 7f06dc9ff700 1 Processor -- start 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.591+0000 7f06dc9ff700 1 -- start start 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06dc9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06c0003c40 0x7f06d814b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06dc9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06d814acb0 0x7f06d81583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06dc9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06d81563f0 0x7f06d8156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06dc9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8158cd0 con 0x7f06d814acb0 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06dc9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80b2700 con 0x7f06d81563f0 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06dc9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d814b120 con 0x7f06c0003c40 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06d814acb0 0x7f06d81583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06d814acb0 0x7f06d81583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51090/0 (socket says 172.21.15.31:51090) 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d5d9b700 1 -- 172.21.15.31:0/2079741331 learned_addr learned my addr 172.21.15.31:0/2079741331 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d659c700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06c0003c40 0x7f06d814b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d6d9d700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06d81563f0 0x7f06d8156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d5d9b700 1 -- 172.21.15.31:0/2079741331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06c0003c40 msgr2=0x7f06d814b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d5d9b700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06c0003c40 0x7f06d814b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d5d9b700 1 -- 172.21.15.31:0/2079741331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06d81563f0 msgr2=0x7f06d8156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d5d9b700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06d81563f0 0x7f06d8156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d5d9b700 1 -- 172.21.15.31:0/2079741331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d8156da0 con 0x7f06d814acb0 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d6d9d700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06d81563f0 0x7f06d8156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.592+0000 7f06d5d9b700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06d814acb0 0x7f06d81583e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06cc007390 tx=0x7f06cc001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.593+0000 7f06c67fc700 1 -- 172.21.15.31:0/2079741331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc018030 con 0x7f06d814acb0 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.593+0000 7f06c67fc700 1 -- 172.21.15.31:0/2079741331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06cc00f8d0 con 0x7f06d814acb0 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.593+0000 7f06dc9ff700 1 -- 172.21.15.31:0/2079741331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d816ac50 con 0x7f06d814acb0 2023-10-06T05:17:18.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.593+0000 7f06c67fc700 1 -- 172.21.15.31:0/2079741331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00fcb0 con 0x7f06d814acb0 2023-10-06T05:17:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.593+0000 7f06dc9ff700 1 -- 172.21.15.31:0/2079741331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d816a7c0 con 0x7f06d814acb0 2023-10-06T05:17:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.594+0000 7f06dc9ff700 1 -- 172.21.15.31:0/2079741331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d816a950 con 0x7f06d814acb0 2023-10-06T05:17:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.595+0000 7f06c67fc700 1 -- 172.21.15.31:0/2079741331 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06cc01b070 con 0x7f06d814acb0 2023-10-06T05:17:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.596+0000 7f06c67fc700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06b8066f30 0x7f06b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.596+0000 7f06c67fc700 1 -- 172.21.15.31:0/2079741331 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f06cc019070 con 0x7f06d814acb0 2023-10-06T05:17:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.596+0000 7f06d659c700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06b8066f30 0x7f06b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.596+0000 7f06d659c700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06b8066f30 0x7f06b80693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f06c80091e0 tx=0x7f06c801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.600+0000 7f06c67fc700 1 -- 172.21.15.31:0/2079741331 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06cc0513d0 con 0x7f06d814acb0 2023-10-06T05:17:18.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.800+0000 7f06dc9ff700 1 -- 172.21.15.31:0/2079741331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06d816ac50 con 0x7f06d814acb0 2023-10-06T05:17:18.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.801+0000 7f06c67fc700 1 -- 172.21.15.31:0/2079741331 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f06d816ac50 con 0x7f06d814acb0 2023-10-06T05:17:18.803 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:18.803 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.803+0000 7f06dc9ff700 1 -- 172.21.15.31:0/2079741331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06b8066f30 msgr2=0x7f06b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.803+0000 7f06dc9ff700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06b8066f30 0x7f06b80693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f06c80091e0 tx=0x7f06c801a000 comp rx=0 tx=0).stop 2023-10-06T05:17:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.803+0000 7f06dc9ff700 1 -- 172.21.15.31:0/2079741331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06d814acb0 msgr2=0x7f06d81583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.803+0000 7f06dc9ff700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06d814acb0 0x7f06d81583e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06cc007390 tx=0x7f06cc001d40 comp rx=0 tx=0).stop 2023-10-06T05:17:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.804+0000 7f06dc9ff700 1 -- 172.21.15.31:0/2079741331 shutdown_connections 2023-10-06T05:17:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.804+0000 7f06dc9ff700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06d81563f0 0x7f06d8156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.804+0000 7f06dc9ff700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06c0003c40 0x7f06d814b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.804+0000 7f06dc9ff700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06b8066f30 0x7f06b80693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.804+0000 7f06dc9ff700 1 --2- 172.21.15.31:0/2079741331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06d814acb0 0x7f06d81583e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.804+0000 7f06dc9ff700 1 -- 172.21.15.31:0/2079741331 >> 172.21.15.31:0/2079741331 conn(0x7f06d806b620 msgr2=0x7f06d815b340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.804+0000 7f06dc9ff700 1 -- 172.21.15.31:0/2079741331 shutdown_connections 2023-10-06T05:17:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:18.804+0000 7f06dc9ff700 1 -- 172.21.15.31:0/2079741331 wait complete. 2023-10-06T05:17:18.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:19.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.015+0000 7f82213ff700 1 Processor -- start 2023-10-06T05:17:19.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.016+0000 7f82213ff700 1 -- start start 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.017+0000 7f82213ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f821c1595a0 0x7f821c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.017+0000 7f82213ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f821c068d80 0x7f821c0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.017+0000 7f82213ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c0c95b0 con 0x7f821c0cc0d0 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.017+0000 7f82213ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c0c9890 con 0x7f821c068d80 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.017+0000 7f82213ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c0c9b70 con 0x7f821c1595a0 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.017+0000 7f821b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f821c068d80 0x7f821c0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.017+0000 7f821b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f821c068d80 0x7f821c0c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51102/0 (socket says 172.21.15.31:51102) 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.017+0000 7f821b7fe700 1 -- 172.21.15.31:0/4220026734 learned_addr learned my addr 172.21.15.31:0/4220026734 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.017+0000 7f821affd700 1 --2- 172.21.15.31:0/4220026734 >> v2:172.21.15.62:3300/0 conn(0x7f821c1595a0 0x7f821c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.018+0000 7f821b7fe700 1 -- 172.21.15.31:0/4220026734 >> v2:172.21.15.62:3300/0 conn(0x7f821c1595a0 msgr2=0x7f821c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.018+0000 7f821b7fe700 1 --2- 172.21.15.31:0/4220026734 >> v2:172.21.15.62:3300/0 conn(0x7f821c1595a0 0x7f821c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.018+0000 7f821b7fe700 1 -- 172.21.15.31:0/4220026734 >> v1:172.21.15.62:6789/0 conn(0x7f821c0cc0d0 legacy=0x7f821c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.018+0000 7f821b7fe700 1 -- 172.21.15.31:0/4220026734 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0ca130 con 0x7f821c068d80 2023-10-06T05:17:19.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.018+0000 7f821b7fe700 1 --2- 172.21.15.31:0/4220026734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c068d80 0x7f821c0c8930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8214009f20 tx=0x7f8214009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62626d9c4e8c3041 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.018+0000 7f8219ffb700 1 -- 172.21.15.31:0/4220026734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400e070 con 0x7f821c068d80 2023-10-06T05:17:19.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.018+0000 7f8219ffb700 1 -- 172.21.15.31:0/4220026734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821400ba40 con 0x7f821c068d80 2023-10-06T05:17:19.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.018+0000 7f8219ffb700 1 -- 172.21.15.31:0/4220026734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140092e0 con 0x7f821c068d80 2023-10-06T05:17:19.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.019+0000 7f82213ff700 1 -- 172.21.15.31:0/4220026734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c068d80 msgr2=0x7f821c0c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.019+0000 7f82213ff700 1 --2- 172.21.15.31:0/4220026734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c068d80 0x7f821c0c8930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8214009f20 tx=0x7f8214009840 comp rx=0 tx=0).stop 2023-10-06T05:17:19.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.019+0000 7f82213ff700 1 -- 172.21.15.31:0/4220026734 shutdown_connections 2023-10-06T05:17:19.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.019+0000 7f82213ff700 1 --2- 172.21.15.31:0/4220026734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c068d80 0x7f821c0c8930 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.019+0000 7f82213ff700 1 --2- 172.21.15.31:0/4220026734 >> v2:172.21.15.62:3300/0 conn(0x7f821c1595a0 0x7f821c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.019+0000 7f82213ff700 1 -- 172.21.15.31:0/4220026734 >> 172.21.15.31:0/4220026734 conn(0x7f821c06b620 msgr2=0x7f821c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.019+0000 7f82213ff700 1 -- 172.21.15.31:0/4220026734 shutdown_connections 2023-10-06T05:17:19.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.019+0000 7f82213ff700 1 -- 172.21.15.31:0/4220026734 wait complete. 2023-10-06T05:17:19.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.020+0000 7f82213ff700 1 Processor -- start 2023-10-06T05:17:19.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.020+0000 7f82213ff700 1 -- start start 2023-10-06T05:17:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.021+0000 7f82213ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c068d80 0x7f821c14f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.021+0000 7f82213ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c0cc0d0 0x7f821c16a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.021+0000 7f82213ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c1595a0 0x7f821c0c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.021+0000 7f82213ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0b1090 con 0x7f821c0cc0d0 2023-10-06T05:17:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.021+0000 7f82213ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0b1650 con 0x7f821c1595a0 2023-10-06T05:17:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.021+0000 7f82213ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c14fa60 con 0x7f821c068d80 2023-10-06T05:17:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c0cc0d0 0x7f821c16a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c0cc0d0 0x7f821c16a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51106/0 (socket says 172.21.15.31:51106) 2023-10-06T05:17:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821a7fc700 1 -- 172.21.15.31:0/1905184204 learned_addr learned my addr 172.21.15.31:0/1905184204 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821b7fe700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c1595a0 0x7f821c0c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821affd700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c068d80 0x7f821c14f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821a7fc700 1 -- 172.21.15.31:0/1905184204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c068d80 msgr2=0x7f821c14f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821a7fc700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c068d80 0x7f821c14f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821a7fc700 1 -- 172.21.15.31:0/1905184204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c1595a0 msgr2=0x7f821c0c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821a7fc700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c1595a0 0x7f821c0c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821a7fc700 1 -- 172.21.15.31:0/1905184204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214009ae0 con 0x7f821c0cc0d0 2023-10-06T05:17:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821b7fe700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c1595a0 0x7f821c0c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.022+0000 7f821a7fc700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c0cc0d0 0x7f821c16a860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f821000c5c0 tx=0x7f821000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.023+0000 7f8202ffd700 1 -- 172.21.15.31:0/1905184204 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210011590 con 0x7f821c0cc0d0 2023-10-06T05:17:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.023+0000 7f8202ffd700 1 -- 172.21.15.31:0/1905184204 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8210011710 con 0x7f821c0cc0d0 2023-10-06T05:17:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.023+0000 7f82213ff700 1 -- 172.21.15.31:0/1905184204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c16ada0 con 0x7f821c0cc0d0 2023-10-06T05:17:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.023+0000 7f8202ffd700 1 -- 172.21.15.31:0/1905184204 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210011ad0 con 0x7f821c0cc0d0 2023-10-06T05:17:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.023+0000 7f82213ff700 1 -- 172.21.15.31:0/1905184204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c0b3620 con 0x7f821c0cc0d0 2023-10-06T05:17:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.024+0000 7f82213ff700 1 -- 172.21.15.31:0/1905184204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c0b17c0 con 0x7f821c0cc0d0 2023-10-06T05:17:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.025+0000 7f8202ffd700 1 -- 172.21.15.31:0/1905184204 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f821001b3b0 con 0x7f821c0cc0d0 2023-10-06T05:17:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.025+0000 7f8202ffd700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8208066f30 0x7f82080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.026+0000 7f8202ffd700 1 -- 172.21.15.31:0/1905184204 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8210083bb0 con 0x7f821c0cc0d0 2023-10-06T05:17:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.026+0000 7f821affd700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8208066f30 0x7f82080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.026+0000 7f821affd700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8208066f30 0x7f82080693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f820c006fd0 tx=0x7f820c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.030+0000 7f8202ffd700 1 -- 172.21.15.31:0/1905184204 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8210050c20 con 0x7f821c0cc0d0 2023-10-06T05:17:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.276+0000 7f82213ff700 1 -- 172.21.15.31:0/1905184204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f821c04c380 con 0x7f821c0cc0d0 2023-10-06T05:17:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.277+0000 7f8202ffd700 1 -- 172.21.15.31:0/1905184204 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f8210017070 con 0x7f821c0cc0d0 2023-10-06T05:17:19.279 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:19.279 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1001195925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1001195925},{"type":"v1","addr":"172.21.15.62:6835","nonce":1001195925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1336279472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1336279472},{"type":"v1","addr":"172.21.15.31:6837","nonce":1336279472}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/577485679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":577485679},{"type":"v1","addr":"172.21.15.62:6834","nonce":577485679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:01.930460+0000","modified":"2023-10-06T05:17:05.459675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4169737410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169737410},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169737410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:17:19.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 -- 172.21.15.31:0/1905184204 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8208066f30 msgr2=0x7f82080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8208066f30 0x7f82080693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f820c006fd0 tx=0x7f820c008040 comp rx=0 tx=0).stop 2023-10-06T05:17:19.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 -- 172.21.15.31:0/1905184204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c0cc0d0 msgr2=0x7f821c16a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c0cc0d0 0x7f821c16a860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f821000c5c0 tx=0x7f821000c8d0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 -- 172.21.15.31:0/1905184204 shutdown_connections 2023-10-06T05:17:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c1595a0 0x7f821c0c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c068d80 0x7f821c14f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8208066f30 0x7f82080693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 --2- 172.21.15.31:0/1905184204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f821c0cc0d0 0x7f821c16a860 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 -- 172.21.15.31:0/1905184204 >> 172.21.15.31:0/1905184204 conn(0x7f821c06b620 msgr2=0x7f821c06cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 -- 172.21.15.31:0/1905184204 shutdown_connections 2023-10-06T05:17:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.279+0000 7f82213ff700 1 -- 172.21.15.31:0/1905184204 wait complete. 2023-10-06T05:17:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T05:17:19.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:17:01.930460+0000', 'modified': '2023-10-06T05:17:05.459675+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/4169737410', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6834', 'nonce': 4169737410}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 4169737410}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:17:19.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:17:19.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:17:19.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.899+0000 7fcde69ff700 1 Processor -- start 2023-10-06T05:17:19.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.899+0000 7fcde69ff700 1 -- start start 2023-10-06T05:17:19.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.900+0000 7fcde69ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcde013a1e0 0x7fcde00c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.900+0000 7fcde69ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a5b0 0x7fcde006d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.900+0000 7fcde69ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde00c71b0 con 0x7fcde00c6c30 2023-10-06T05:17:19.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.900+0000 7fcde69ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde00c7490 con 0x7fcde013a1e0 2023-10-06T05:17:19.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.900+0000 7fcde69ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde00c7770 con 0x7fcde013a5b0 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.901+0000 7fcddffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcde013a1e0 0x7fcde00c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.901+0000 7fcddf7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a5b0 0x7fcde006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.901+0000 7fcddf7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a5b0 0x7fcde006d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:47092/0 (socket says 172.21.15.31:47092) 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.901+0000 7fcddf7fe700 1 -- 172.21.15.31:0/3059112373 learned_addr learned my addr 172.21.15.31:0/3059112373 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.901+0000 7fcde4f9c700 1 --1- 172.21.15.31:0/3059112373 >> v1:172.21.15.62:6789/0 conn(0x7fcde00c6c30 0x7fcde015afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:57706/0 (socket says 172.21.15.31:57706) 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.901+0000 7fcddf7fe700 1 -- 172.21.15.31:0/3059112373 >> v2:172.21.15.62:3300/0 conn(0x7fcde013a1e0 msgr2=0x7fcde00c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.901+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/3059112373 >> v2:172.21.15.62:3300/0 conn(0x7fcde013a1e0 0x7fcde00c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.901+0000 7fcddf7fe700 1 -- 172.21.15.31:0/3059112373 >> v1:172.21.15.62:6789/0 conn(0x7fcde00c6c30 legacy=0x7fcde015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.901+0000 7fcddf7fe700 1 -- 172.21.15.31:0/3059112373 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde00c7a50 con 0x7fcde013a5b0 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.901+0000 7fcddffff700 1 --2- 172.21.15.31:0/3059112373 >> v2:172.21.15.62:3300/0 conn(0x7fcde013a1e0 0x7fcde00c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.902+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/3059112373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a5b0 0x7fcde006d440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40099b0 tx=0x7fcdd4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a936944bd7b2a45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.902+0000 7fcddeffd700 1 -- 172.21.15.31:0/3059112373 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4009ea0 con 0x7fcde013a5b0 2023-10-06T05:17:19.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.902+0000 7fcddeffd700 1 -- 172.21.15.31:0/3059112373 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd400b860 con 0x7fcde013a5b0 2023-10-06T05:17:19.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.902+0000 7fcddeffd700 1 -- 172.21.15.31:0/3059112373 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400bc00 con 0x7fcde013a5b0 2023-10-06T05:17:19.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.902+0000 7fcde69ff700 1 -- 172.21.15.31:0/3059112373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a5b0 msgr2=0x7fcde006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.902+0000 7fcde69ff700 1 --2- 172.21.15.31:0/3059112373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a5b0 0x7fcde006d440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40099b0 tx=0x7fcdd4009250 comp rx=0 tx=0).stop 2023-10-06T05:17:19.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.903+0000 7fcde69ff700 1 -- 172.21.15.31:0/3059112373 shutdown_connections 2023-10-06T05:17:19.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.903+0000 7fcde69ff700 1 --2- 172.21.15.31:0/3059112373 >> v2:172.21.15.62:3300/0 conn(0x7fcde013a1e0 0x7fcde00c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.903+0000 7fcde69ff700 1 --2- 172.21.15.31:0/3059112373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a5b0 0x7fcde006d440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.903+0000 7fcde69ff700 1 -- 172.21.15.31:0/3059112373 >> 172.21.15.31:0/3059112373 conn(0x7fcde00cc0b0 msgr2=0x7fcde00cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.903+0000 7fcde69ff700 1 -- 172.21.15.31:0/3059112373 shutdown_connections 2023-10-06T05:17:19.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.903+0000 7fcde69ff700 1 -- 172.21.15.31:0/3059112373 wait complete. 2023-10-06T05:17:19.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.904+0000 7fcde69ff700 1 Processor -- start 2023-10-06T05:17:19.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.904+0000 7fcde69ff700 1 -- start start 2023-10-06T05:17:19.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.905+0000 7fcde69ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde00c6c30 0x7fcde00c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.905+0000 7fcde69ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a1e0 0x7fcde006d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.905+0000 7fcde69ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde013a5b0 0x7fcde0139630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.905+0000 7fcde69ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde0157120 con 0x7fcde00c6c30 2023-10-06T05:17:19.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.905+0000 7fcde69ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde0156510 con 0x7fcde013a5b0 2023-10-06T05:17:19.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.905+0000 7fcde69ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde01567f0 con 0x7fcde013a1e0 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.905+0000 7fcddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde00c6c30 0x7fcde00c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde00c6c30 0x7fcde00c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51112/0 (socket says 172.21.15.31:51112) 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcddffff700 1 -- 172.21.15.31:0/1975296521 learned_addr learned my addr 172.21.15.31:0/1975296521 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a1e0 0x7fcde006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcde4f9c700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde013a5b0 0x7fcde0139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcddffff700 1 -- 172.21.15.31:0/1975296521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a1e0 msgr2=0x7fcde006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcddffff700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a1e0 0x7fcde006d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcddffff700 1 -- 172.21.15.31:0/1975296521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde013a5b0 msgr2=0x7fcde0139630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcddffff700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde013a5b0 0x7fcde0139630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcddffff700 1 -- 172.21.15.31:0/1975296521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd00093c0 con 0x7fcde00c6c30 2023-10-06T05:17:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a1e0 0x7fcde006d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:19.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcddffff700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde00c6c30 0x7fcde00c6370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000ec80 tx=0x7fcdd0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcdc7fff700 1 -- 172.21.15.31:0/1975296521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0005a00 con 0x7fcde00c6c30 2023-10-06T05:17:19.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcdc7fff700 1 -- 172.21.15.31:0/1975296521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd0005b60 con 0x7fcde00c6c30 2023-10-06T05:17:19.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcde69ff700 1 -- 172.21.15.31:0/1975296521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd40094f0 con 0x7fcde00c6c30 2023-10-06T05:17:19.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.906+0000 7fcdc7fff700 1 -- 172.21.15.31:0/1975296521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000c9d0 con 0x7fcde00c6c30 2023-10-06T05:17:19.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.907+0000 7fcde69ff700 1 -- 172.21.15.31:0/1975296521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde0156db0 con 0x7fcde00c6c30 2023-10-06T05:17:19.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.909+0000 7fcdc7fff700 1 -- 172.21.15.31:0/1975296521 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdd000cd80 con 0x7fcde00c6c30 2023-10-06T05:17:19.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.909+0000 7fcdc7fff700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcdcc066f30 0x7fcdcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.909+0000 7fcdc7fff700 1 -- 172.21.15.31:0/1975296521 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcdd0084d60 con 0x7fcde00c6c30 2023-10-06T05:17:19.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.910+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcdcc066f30 0x7fcdcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.910+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcdcc066f30 0x7fcdcc0693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40099b0 tx=0x7fcdd4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.911+0000 7fcdf50afb80 1 Processor -- start 2023-10-06T05:17:19.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.912+0000 7fcdf50afb80 1 -- start start 2023-10-06T05:17:19.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.912+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ef820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.912+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9d4f40 0x564a4c9f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.912+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9f8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.912+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4c8c2310 con 0x564a4c9ef450 2023-10-06T05:17:19.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.912+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4c8ff130 con 0x564a4c9d4f40 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.912+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a4c8fcaf0 con 0x564a4c9f84d0 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ef820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ef820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51118/0 (socket says 172.21.15.31:51118) 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcddffff700 1 -- 172.21.15.31:0/1409697405 learned_addr learned my addr 172.21.15.31:0/1409697405 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcde4f9c700 1 --2- 172.21.15.31:0/1409697405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/1409697405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9d4f40 0x564a4c9f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcddffff700 1 -- 172.21.15.31:0/1409697405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 msgr2=0x564a4c9f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcddffff700 1 --2- 172.21.15.31:0/1409697405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9f8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcddffff700 1 -- 172.21.15.31:0/1409697405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9d4f40 msgr2=0x564a4c9f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcddffff700 1 --2- 172.21.15.31:0/1409697405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9d4f40 0x564a4c9f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcddffff700 1 -- 172.21.15.31:0/1409697405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a4c6fe790 con 0x564a4c9ef450 2023-10-06T05:17:19.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcddffff700 1 --2- 172.21.15.31:0/1409697405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ef820 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0051f90 tx=0x7fcdd0082fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e55ad4f1c763570 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcdc57fa700 1 -- 172.21.15.31:0/1409697405 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0058ce0 con 0x564a4c9ef450 2023-10-06T05:17:19.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcdc57fa700 1 -- 172.21.15.31:0/1409697405 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd005ca30 con 0x564a4c9ef450 2023-10-06T05:17:19.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.913+0000 7fcdc57fa700 1 -- 172.21.15.31:0/1409697405 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0053e30 con 0x564a4c9ef450 2023-10-06T05:17:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.914+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1409697405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 msgr2=0x564a4c9ef820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.914+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/1409697405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ef820 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0051f90 tx=0x7fcdd0082fe0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.914+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1409697405 shutdown_connections 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.914+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/1409697405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.914+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/1409697405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ef820 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.914+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/1409697405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9d4f40 0x564a4c9f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.914+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1409697405 >> 172.21.15.31:0/1409697405 conn(0x564a4c9f2880 msgr2=0x564a4c9f2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.914+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1409697405 shutdown_connections 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.914+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1409697405 wait complete. 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.915+0000 7fcdf50afb80 1 -- start start 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.915+0000 7fcdf50afb80 1 Processor -- start 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.916+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9d4f40 0x564a4c9ee990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.916+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 0x564a4c9c9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.916+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.916+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4c8f88e0 con 0x564a4c9d4f40 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.916+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4c8a4980 con 0x564a4c9ef450 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.916+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a4c8a4620 con 0x564a4c9f84d0 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.916+0000 7fcdf50afb80 10 client.0 _ll_register_callbacks cb 0x564a4c707a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9d4f40 0x564a4c9ee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9d4f40 0x564a4c9ee990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51126/0 (socket says 172.21.15.31:51126) 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcddffff700 1 -- 172.21.15.31:0/1365179575 learned_addr learned my addr 172.21.15.31:0/1365179575 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 0x564a4c9c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcde4f9c700 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcddffff700 1 -- 172.21.15.31:0/1365179575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 msgr2=0x564a4c9a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcddffff700 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9a97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcddffff700 1 -- 172.21.15.31:0/1365179575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 msgr2=0x564a4c9c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcddffff700 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 0x564a4c9c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcddffff700 1 -- 172.21.15.31:0/1365179575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde0156db0 con 0x564a4c9d4f40 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcddffff700 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9d4f40 0x564a4c9ee990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0083110 tx=0x7fcdd006b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d383db891d77fa0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcdbe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/1365179575 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0053ca0 con 0x564a4c9d4f40 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/1365179575 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcdd0054f80 con 0x564a4c9d4f40 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.917+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1365179575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a4c99cd60 con 0x564a4c9d4f40 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/1365179575 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdd005c5a0 con 0x564a4c9d4f40 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdf50afb80 10 client.4623 fetch_fsmap learned FSMap version 8 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1365179575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x564a4c7eb0a0 con 0x564a4c9d4f40 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/1365179575 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564a4c99cd60 con 0x564a4c9d4f40 2023-10-06T05:17:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/1365179575 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fcdd0051880 con 0x564a4c9d4f40 2023-10-06T05:17:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdf50afb80 10 client.4623 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T05:17:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdf50afb80 10 client.4623 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:17:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdf50afb80 20 client.4623 populate_metadata read hostname 'smithi031' 2023-10-06T05:17:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.31:6834/4169737410,v1:172.21.15.31:6835/4169737410] conn(0x564a4c9c9cf0 0x564a4c9ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdf50afb80 4 client.4623 mds_command: new command op to 4288 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:17:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1365179575 --> [v2:172.21.15.31:6834/4169737410,v1:172.21.15.31:6835/4169737410] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a4c979b30 con 0x564a4c9c9cf0 2023-10-06T05:17:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.918+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.31:6834/4169737410,v1:172.21.15.31:6835/4169737410] conn(0x564a4c9c9cf0 0x564a4c9ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.919+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.31:6834/4169737410,v1:172.21.15.31:6835/4169737410] conn(0x564a4c9c9cf0 0x564a4c9ca0c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5289151a016dc2c6 server_cookie=6be8dac859458e64 in_seq=0 out_seq=0 2023-10-06T05:17:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.919+0000 7fcdbe7fc700 10 client.4623 ms_handle_connect on v2:172.21.15.31:6834/4169737410 2023-10-06T05:17:19.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.919+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/1365179575 <== mds.0 v2:172.21.15.31:6834/4169737410 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564a4c979b30 con 0x564a4c9c9cf0 2023-10-06T05:17:19.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.919+0000 7fcdbe7fc700 10 client.4623 handle_command_reply: tid=0 2023-10-06T05:17:19.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 client.4623 shutdown 2023-10-06T05:17:19.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1365179575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9d4f40 msgr2=0x564a4c9ee990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9d4f40 0x564a4c9ee990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0083110 tx=0x7fcdd006b000 comp rx=0 tx=0).stop 2023-10-06T05:17:19.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1365179575 shutdown_connections 2023-10-06T05:17:19.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.31:6834/4169737410,v1:172.21.15.31:6835/4169737410] conn(0x564a4c9c9cf0 0x564a4c9ca0c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9d4f40 0x564a4c9ee990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdbe7fc700 0 client.4623 ms_handle_reset on v2:172.21.15.31:6834/4169737410 2023-10-06T05:17:19.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/1365179575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 0x564a4c9c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1365179575 >> 172.21.15.31:0/1365179575 conn(0x564a4c9f2880 msgr2=0x564a4c9a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1365179575 shutdown_connections 2023-10-06T05:17:19.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 1 -- 172.21.15.31:0/1365179575 wait complete. 2023-10-06T05:17:19.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.920+0000 7fcdf50afb80 20 client.4623 trim_cache size 0 max 16384 2023-10-06T05:17:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.937+0000 7fcdf50afb80 1 Processor -- start 2023-10-06T05:17:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.938+0000 7fcdf50afb80 1 -- start start 2023-10-06T05:17:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.938+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9d4f40 0x564a4c9f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.938+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 0x564a4c9a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.938+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9f84d0 0x564a4c9ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.938+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4c7901e0 con 0x564a4c9f84d0 2023-10-06T05:17:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.938+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4c9f8180 con 0x564a4c9ef450 2023-10-06T05:17:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.938+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a4c9a9a80 con 0x564a4c9d4f40 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcde4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9f84d0 0x564a4c9ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcde4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9f84d0 0x564a4c9ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51136/0 (socket says 172.21.15.31:51136) 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcde4f9c700 1 -- 172.21.15.31:0/237818534 learned_addr learned my addr 172.21.15.31:0/237818534 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcddffff700 1 --2- 172.21.15.31:0/237818534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9d4f40 0x564a4c9f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/237818534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 0x564a4c9a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcde4f9c700 1 -- 172.21.15.31:0/237818534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9d4f40 msgr2=0x564a4c9f0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcde4f9c700 1 --2- 172.21.15.31:0/237818534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9d4f40 0x564a4c9f0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcde4f9c700 1 -- 172.21.15.31:0/237818534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 msgr2=0x564a4c9a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcde4f9c700 1 --2- 172.21.15.31:0/237818534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 0x564a4c9a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcde4f9c700 1 -- 172.21.15.31:0/237818534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a4c8c73c0 con 0x564a4c9f84d0 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/237818534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 0x564a4c9a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcde4f9c700 1 --2- 172.21.15.31:0/237818534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9f84d0 0x564a4c9ca290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8001c40 tx=0x7fcdd8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b152e644add4465 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcdc5ffb700 1 -- 172.21.15.31:0/237818534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80074c0 con 0x564a4c9f84d0 2023-10-06T05:17:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcdc5ffb700 1 -- 172.21.15.31:0/237818534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8007640 con 0x564a4c9f84d0 2023-10-06T05:17:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.939+0000 7fcdc5ffb700 1 -- 172.21.15.31:0/237818534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8007a00 con 0x564a4c9f84d0 2023-10-06T05:17:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.940+0000 7fcdf50afb80 1 -- 172.21.15.31:0/237818534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9f84d0 msgr2=0x564a4c9ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.940+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/237818534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9f84d0 0x564a4c9ca290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8001c40 tx=0x7fcdd8009040 comp rx=0 tx=0).stop 2023-10-06T05:17:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.940+0000 7fcdf50afb80 1 -- 172.21.15.31:0/237818534 shutdown_connections 2023-10-06T05:17:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.940+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/237818534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9f84d0 0x564a4c9ca290 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.940+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/237818534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9d4f40 0x564a4c9f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.940+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/237818534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9ef450 0x564a4c9a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.940+0000 7fcdf50afb80 1 -- 172.21.15.31:0/237818534 >> 172.21.15.31:0/237818534 conn(0x564a4c9f2880 msgr2=0x564a4c9ee990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.940+0000 7fcdf50afb80 1 -- 172.21.15.31:0/237818534 shutdown_connections 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.940+0000 7fcdf50afb80 1 -- 172.21.15.31:0/237818534 wait complete. 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.941+0000 7fcdf50afb80 1 -- start start 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.941+0000 7fcdf50afb80 1 Processor -- start 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.942+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9d4f40 0x564a4c9a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.942+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.942+0000 7fcdf50afb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.942+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4c9f6e60 con 0x564a4c9ef450 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.942+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4c9ecee0 con 0x564a4c9d4f40 2023-10-06T05:17:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.942+0000 7fcdf50afb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a4c9edd40 con 0x564a4c9f84d0 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.942+0000 7fcdf50afb80 10 client.0 _ll_register_callbacks cb 0x564a4c707a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.942+0000 7fcddf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.942+0000 7fcddf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51146/0 (socket says 172.21.15.31:51146) 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.942+0000 7fcddf7fe700 1 -- 172.21.15.31:0/3212227133 learned_addr learned my addr 172.21.15.31:0/3212227133 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcddffff700 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9d4f40 0x564a4c9a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcde4f9c700 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcddf7fe700 1 -- 172.21.15.31:0/3212227133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 msgr2=0x564a4c9ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcddf7fe700 1 -- 172.21.15.31:0/3212227133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9d4f40 msgr2=0x564a4c9a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9d4f40 0x564a4c9a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcddf7fe700 1 -- 172.21.15.31:0/3212227133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd800f040 con 0x564a4c9ef450 2023-10-06T05:17:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ca290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcde00c78b0 tx=0x7fcdd402b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ccbdfaa667bc37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcdbe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/3212227133 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd402f050 con 0x564a4c9ef450 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcdf50afb80 1 -- 172.21.15.31:0/3212227133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdd005ca30 con 0x564a4c9ef450 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/3212227133 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcdd400bb90 con 0x564a4c9ef450 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/3212227133 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdd4033250 con 0x564a4c9ef450 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcdf50afb80 10 client.4629 fetch_fsmap learned FSMap version 8 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.943+0000 7fcdf50afb80 1 -- 172.21.15.31:0/3212227133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x564a4c75f1d0 con 0x564a4c9ef450 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/3212227133 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd005ca30 con 0x564a4c9ef450 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/3212227133 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fcdd4034660 con 0x564a4c9ef450 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcdf50afb80 10 client.4629 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcdf50afb80 10 client.4629 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:17:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcdf50afb80 20 client.4629 populate_metadata read hostname 'smithi031' 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.31:6834/4169737410,v1:172.21.15.31:6835/4169737410] conn(0x564a4c9ceb40 0x564a4c9cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcdf50afb80 4 client.4629 mds_command: new command op to 4288 tid=0[{"prefix": "status"}] 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcdf50afb80 1 -- 172.21.15.31:0/3212227133 --> [v2:172.21.15.31:6834/4169737410,v1:172.21.15.31:6835/4169737410] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564a4c89b8d0 con 0x564a4c9ceb40 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcddffff700 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.31:6834/4169737410,v1:172.21.15.31:6835/4169737410] conn(0x564a4c9ceb40 0x564a4c9cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcdbe7fc700 10 client.4629 ms_handle_connect on v2:172.21.15.31:6834/4169737410 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.944+0000 7fcddffff700 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.31:6834/4169737410,v1:172.21.15.31:6835/4169737410] conn(0x564a4c9ceb40 0x564a4c9cef10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349cec2c023781fc server_cookie=2dfcf0e5ba648a5 in_seq=0 out_seq=0 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdbe7fc700 1 -- 172.21.15.31:0/3212227133 <== mds.0 v2:172.21.15.31:6834/4169737410 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x564a4c89b8d0 con 0x564a4c9ceb40 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdbe7fc700 10 client.4629 handle_command_reply: tid=0 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 client.4629 shutdown 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 -- 172.21.15.31:0/3212227133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 msgr2=0x564a4c9ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ca290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcde00c78b0 tx=0x7fcdd402b040 comp rx=0 tx=0).stop 2023-10-06T05:17:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 -- 172.21.15.31:0/3212227133 shutdown_connections 2023-10-06T05:17:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.31:6834/4169737410,v1:172.21.15.31:6835/4169737410] conn(0x564a4c9ceb40 0x564a4c9cef10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564a4c9f84d0 0x564a4c9ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a4c9d4f40 0x564a4c9a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 --2- 172.21.15.31:0/3212227133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564a4c9ef450 0x564a4c9ca290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdbe7fc700 0 client.4629 ms_handle_reset on v2:172.21.15.31:6834/4169737410 2023-10-06T05:17:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 -- 172.21.15.31:0/3212227133 >> 172.21.15.31:0/3212227133 conn(0x564a4c9f2880 msgr2=0x564a4c9cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 -- 172.21.15.31:0/3212227133 shutdown_connections 2023-10-06T05:17:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 1 -- 172.21.15.31:0/3212227133 wait complete. 2023-10-06T05:17:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.945+0000 7fcdf50afb80 20 client.4629 trim_cache size 0 max 16384 2023-10-06T05:17:19.953 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:17:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:17:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:17:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4288, 2023-10-06T05:17:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:17:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:17:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:17:19.953 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 17.499483335000001, 2023-10-06T05:17:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 8, 2023-10-06T05:17:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 20, 2023-10-06T05:17:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 20, 2023-10-06T05:17:19.954 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 24.355449863 2023-10-06T05:17:19.954 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:17:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 -- 172.21.15.31:0/1975296521 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcdcc066f30 msgr2=0x7fcdcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcdcc066f30 0x7fcdcc0693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40099b0 tx=0x7fcdd4017000 comp rx=0 tx=0).stop 2023-10-06T05:17:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 -- 172.21.15.31:0/1975296521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde00c6c30 msgr2=0x7fcde00c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde00c6c30 0x7fcde00c6370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000ec80 tx=0x7fcdd0009120 comp rx=0 tx=0).stop 2023-10-06T05:17:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 -- 172.21.15.31:0/1975296521 shutdown_connections 2023-10-06T05:17:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde013a5b0 0x7fcde0139630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcdcc066f30 0x7fcdcc0693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde00c6c30 0x7fcde00c6370 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 --2- 172.21.15.31:0/1975296521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde013a1e0 0x7fcde006d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 -- 172.21.15.31:0/1975296521 >> 172.21.15.31:0/1975296521 conn(0x7fcde00cc0b0 msgr2=0x7fcde013c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 -- 172.21.15.31:0/1975296521 shutdown_connections 2023-10-06T05:17:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:19.946+0000 7fcde69ff700 1 -- 172.21.15.31:0/1975296521 wait complete. 2023-10-06T05:17:19.965 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T05:17:19.987 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T05:17:19.987 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T05:17:19.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:19.988 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:17:20.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:20.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-10-06T05:17:20.109 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-10-06T05:17:20.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:20.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:20.114 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:17:20.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:20.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-10-06T05:17:20.235 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-10-06T05:17:20.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:20.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:20.239 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:17:20.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:20.294 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2023-10-06T05:17:20.373 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2023-10-06T05:17:20.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:20.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:20.376 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:17:20.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:20.424 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2023-10-06T05:17:20.497 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2023-10-06T05:17:20.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:17:20.500 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-06T05:17:20.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:17:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.694+0000 7f8b481ff700 1 Processor -- start 2023-10-06T05:17:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.695+0000 7f8b481ff700 1 -- start start 2023-10-06T05:17:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.696+0000 7f8b481ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400c7b60 0x7f8b40069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.696+0000 7f8b481ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b400aee70 0x7f8b4013f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.696+0000 7f8b481ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b400ca5b0 con 0x7f8b400c7f30 2023-10-06T05:17:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.696+0000 7f8b481ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b400af2d0 con 0x7f8b400aee70 2023-10-06T05:17:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.696+0000 7f8b481ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4006f740 con 0x7f8b400c7b60 2023-10-06T05:17:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.696+0000 7f8b4679c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b400aee70 0x7f8b4013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.696+0000 7f8b4679c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b400aee70 0x7f8b4013f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51156/0 (socket says 172.21.15.31:51156) 2023-10-06T05:17:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.696+0000 7f8b4679c700 1 -- 172.21.15.31:0/2685558151 learned_addr learned my addr 172.21.15.31:0/2685558151 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.696+0000 7f8b45f9b700 1 --2- 172.21.15.31:0/2685558151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400c7b60 0x7f8b40069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b4679c700 1 -- 172.21.15.31:0/2685558151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400c7b60 msgr2=0x7f8b40069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b4679c700 1 --2- 172.21.15.31:0/2685558151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400c7b60 0x7f8b40069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b4679c700 1 -- 172.21.15.31:0/2685558151 >> v1:172.21.15.31:6789/0 conn(0x7f8b400c7f30 legacy=0x7f8b4006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b4679c700 1 -- 172.21.15.31:0/2685558151 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b400c9c80 con 0x7f8b400aee70 2023-10-06T05:17:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b44f99700 1 -- 172.21.15.31:0/2685558151 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674836356 0 0) 0x7f8b400ca5b0 con 0x7f8b400c7f30 2023-10-06T05:17:20.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b4679c700 1 --2- 172.21.15.31:0/2685558151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b400aee70 0x7f8b4013f690 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009f20 tx=0x7f8b38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c72fc42306cc53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b44f99700 1 -- 172.21.15.31:0/2685558151 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3800e070 con 0x7f8b400aee70 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b44f99700 1 -- 172.21.15.31:0/2685558151 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3800ba80 con 0x7f8b400aee70 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b44f99700 1 -- 172.21.15.31:0/2685558151 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b380092e0 con 0x7f8b400aee70 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b481ff700 1 -- 172.21.15.31:0/2685558151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b400aee70 msgr2=0x7f8b4013f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.697+0000 7f8b481ff700 1 --2- 172.21.15.31:0/2685558151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b400aee70 0x7f8b4013f690 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009f20 tx=0x7f8b38009840 comp rx=0 tx=0).stop 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.698+0000 7f8b481ff700 1 -- 172.21.15.31:0/2685558151 shutdown_connections 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.698+0000 7f8b481ff700 1 --2- 172.21.15.31:0/2685558151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b400aee70 0x7f8b4013f690 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.698+0000 7f8b481ff700 1 --2- 172.21.15.31:0/2685558151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400c7b60 0x7f8b40069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.698+0000 7f8b481ff700 1 -- 172.21.15.31:0/2685558151 >> 172.21.15.31:0/2685558151 conn(0x7f8b400cc0b0 msgr2=0x7f8b400cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.698+0000 7f8b481ff700 1 -- 172.21.15.31:0/2685558151 shutdown_connections 2023-10-06T05:17:20.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.698+0000 7f8b481ff700 1 -- 172.21.15.31:0/2685558151 wait complete. 2023-10-06T05:17:20.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.700+0000 7f8b481ff700 1 Processor -- start 2023-10-06T05:17:20.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.700+0000 7f8b481ff700 1 -- start start 2023-10-06T05:17:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.701+0000 7f8b481ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400aee70 0x7f8b400c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.701+0000 7f8b481ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b400c7b60 0x7f8b400a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.701+0000 7f8b481ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b400c7f30 0x7f8b400a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.701+0000 7f8b481ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400c3510 con 0x7f8b400c7f30 2023-10-06T05:17:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.701+0000 7f8b481ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400c52c0 con 0x7f8b400c7b60 2023-10-06T05:17:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.701+0000 7f8b481ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400a6be0 con 0x7f8b400aee70 2023-10-06T05:17:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.701+0000 7f8b45f9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400aee70 0x7f8b400c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b45f9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400aee70 0x7f8b400c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:47150/0 (socket says 172.21.15.31:47150) 2023-10-06T05:17:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b45f9b700 1 -- 172.21.15.31:0/4192894668 learned_addr learned my addr 172.21.15.31:0/4192894668 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b4679c700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b400c7f30 0x7f8b400a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b4579a700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b400c7b60 0x7f8b400a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b4679c700 1 -- 172.21.15.31:0/4192894668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400aee70 msgr2=0x7f8b400c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b4679c700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400aee70 0x7f8b400c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b4679c700 1 -- 172.21.15.31:0/4192894668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b400c7b60 msgr2=0x7f8b400a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b4679c700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b400c7b60 0x7f8b400a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b4679c700 1 -- 172.21.15.31:0/4192894668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38009ae0 con 0x7f8b400c7f30 2023-10-06T05:17:20.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b45f9b700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400aee70 0x7f8b400c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:20.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b4679c700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b400c7f30 0x7f8b400a5fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8b38015000 tx=0x7f8b38018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.702+0000 7f8b35ffb700 1 -- 172.21.15.31:0/4192894668 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b380032f0 con 0x7f8b400c7f30 2023-10-06T05:17:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.703+0000 7f8b35ffb700 1 -- 172.21.15.31:0/4192894668 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b38008d60 con 0x7f8b400c7f30 2023-10-06T05:17:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.703+0000 7f8b481ff700 1 -- 172.21.15.31:0/4192894668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b400a6ec0 con 0x7f8b400c7f30 2023-10-06T05:17:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.703+0000 7f8b35ffb700 1 -- 172.21.15.31:0/4192894668 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38026690 con 0x7f8b400c7f30 2023-10-06T05:17:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.703+0000 7f8b481ff700 1 -- 172.21.15.31:0/4192894668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b400c4f50 con 0x7f8b400c7f30 2023-10-06T05:17:20.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.704+0000 7f8b481ff700 1 -- 172.21.15.31:0/4192894668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4004c380 con 0x7f8b400c7f30 2023-10-06T05:17:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.704+0000 7f8b35ffb700 1 -- 172.21.15.31:0/4192894668 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b38003040 con 0x7f8b400c7f30 2023-10-06T05:17:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.705+0000 7f8b35ffb700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b24066f30 0x7f8b240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.705+0000 7f8b35ffb700 1 -- 172.21.15.31:0/4192894668 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8b38085fd0 con 0x7f8b400c7f30 2023-10-06T05:17:20.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.706+0000 7f8b45f9b700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b24066f30 0x7f8b240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:20.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.706+0000 7f8b45f9b700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b24066f30 0x7f8b240693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c010370 tx=0x7f8b3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:20.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.710+0000 7f8b35ffb700 1 -- 172.21.15.31:0/4192894668 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b38053190 con 0x7f8b400c7f30 2023-10-06T05:17:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.960+0000 7f8b481ff700 1 -- 172.21.15.31:0/4192894668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b400c5400 con 0x7f8b400c7f30 2023-10-06T05:17:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.960+0000 7f8b35ffb700 1 -- 172.21.15.31:0/4192894668 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b380567b0 con 0x7f8b400c7f30 2023-10-06T05:17:20.962 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:17:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 -- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b24066f30 msgr2=0x7f8b240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b24066f30 0x7f8b240693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c010370 tx=0x7f8b3c008040 comp rx=0 tx=0).stop 2023-10-06T05:17:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 -- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b400c7f30 msgr2=0x7f8b400a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b400c7f30 0x7f8b400a5fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8b38015000 tx=0x7f8b38018000 comp rx=0 tx=0).stop 2023-10-06T05:17:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 -- 172.21.15.31:0/4192894668 shutdown_connections 2023-10-06T05:17:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b24066f30 0x7f8b240693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b400c7f30 0x7f8b400a5fd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b400aee70 0x7f8b400c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 --2- 172.21.15.31:0/4192894668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b400c7b60 0x7f8b400a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 -- 172.21.15.31:0/4192894668 >> 172.21.15.31:0/4192894668 conn(0x7f8b400cc0b0 msgr2=0x7f8b4013e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 -- 172.21.15.31:0/4192894668 shutdown_connections 2023-10-06T05:17:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:20.963+0000 7f8b481ff700 1 -- 172.21.15.31:0/4192894668 wait complete. 2023-10-06T05:17:20.982 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:17:20.982 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:17:20.982 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:17:20.982 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:17:20.982 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:20.982 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:17:20.982 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:17:20.983 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:17:20.983 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:17:20.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:20.983 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:17:21.031 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:17:21.032 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:17:21.032 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:17:21.032 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:21.032 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:17:21.032 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:21.033 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:17:21.033 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:17:21.034 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:17:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:17:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:17:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42273sec preferred_lft 42273sec 2023-10-06T05:17:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:17:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:21.035 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:17:21.036 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T05:17:21.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:21.037 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:17:21.037 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:17:21.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2023-10-06T05:17:21.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2023-10-06T05:17:21.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2023-10-06T05:17:21.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T05:17:21.038 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:17:21.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:21.464 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:17:21.538 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:17:21.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:21.541 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:17:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:17:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:17:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:17:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:17:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:17:21.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:21.621 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:17:21.621 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:17:21.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:17:21.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:17:21.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:17:21.621 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:17:22.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:22.340 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:17:22.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:22.388 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-10-06T05:17:22.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:22.435 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:17:22.435 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:17:22.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:17:22.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T05:17:22.435 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:17:22.580 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T05:17:22.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:22.580 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:17:22.581 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:17:22.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T05:17:22.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T05:17:22.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T05:17:22.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T05:17:22.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T05:17:22.581 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:17:22.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:22.861 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:17:22.861 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:17:22.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:17:22.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T05:17:22.861 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:17:23.035 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:23.035 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:23.082 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:17:23.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:23.082 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:23.168 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:17:23.168 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:17:23.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:23.309 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:23.309 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:17:23.743 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:23.743 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:17:23.743 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:17:23.743 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:17:23.743 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:17:23.748 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:17:23.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:23.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:17:23.820 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-06T05:17:23.820 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:17:24.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.013+0000 7fb8cddff700 1 Processor -- start 2023-10-06T05:17:24.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.013+0000 7fb8cddff700 1 -- start start 2023-10-06T05:17:24.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8cddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8c8159580 0x7fb8c80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8cddff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8c80ccc80 0x7fb8c80cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8cddff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c80cf280 con 0x7fb8c80cb9f0 2023-10-06T05:17:24.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8cddff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c80cfe20 con 0x7fb8c80ccc80 2023-10-06T05:17:24.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8cddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80ce670 con 0x7fb8c8159580 2023-10-06T05:17:24.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8c7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8c80ccc80 0x7fb8c80cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8c7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8c80ccc80 0x7fb8c80cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51164/0 (socket says 172.21.15.31:51164) 2023-10-06T05:17:24.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8c7fff700 1 -- 172.21.15.31:0/3153780872 learned_addr learned my addr 172.21.15.31:0/3153780872 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:24.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8c6ffd700 1 --1- 172.21.15.31:0/3153780872 >> v1:172.21.15.31:6789/0 conn(0x7fb8c80cb9f0 0x7fb8c80cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58696/0 (socket says 172.21.15.31:58696) 2023-10-06T05:17:24.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8c77fe700 1 --2- 172.21.15.31:0/3153780872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8c8159580 0x7fb8c80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8c7fff700 1 -- 172.21.15.31:0/3153780872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8c8159580 msgr2=0x7fb8c80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:24.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8c7fff700 1 --2- 172.21.15.31:0/3153780872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8c8159580 0x7fb8c80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8c7fff700 1 -- 172.21.15.31:0/3153780872 >> v1:172.21.15.31:6789/0 conn(0x7fb8c80cb9f0 legacy=0x7fb8c80cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8c7fff700 1 -- 172.21.15.31:0/3153780872 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c80ce950 con 0x7fb8c80ccc80 2023-10-06T05:17:24.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.015+0000 7fb8c67fc700 1 -- 172.21.15.31:0/3153780872 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120235216 0 0) 0x7fb8c80cf280 con 0x7fb8c80cb9f0 2023-10-06T05:17:24.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.016+0000 7fb8c7fff700 1 --2- 172.21.15.31:0/3153780872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c80ccc80 0x7fb8c80cd0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009f20 tx=0x7fb8c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7b89834716db17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:24.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.016+0000 7fb8c67fc700 1 -- 172.21.15.31:0/3153780872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c000e070 con 0x7fb8c80ccc80 2023-10-06T05:17:24.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.016+0000 7fb8c67fc700 1 -- 172.21.15.31:0/3153780872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c000ba80 con 0x7fb8c80ccc80 2023-10-06T05:17:24.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.016+0000 7fb8c67fc700 1 -- 172.21.15.31:0/3153780872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c00092e0 con 0x7fb8c80ccc80 2023-10-06T05:17:24.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.016+0000 7fb8cddff700 1 -- 172.21.15.31:0/3153780872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c80ccc80 msgr2=0x7fb8c80cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.016+0000 7fb8cddff700 1 --2- 172.21.15.31:0/3153780872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c80ccc80 0x7fb8c80cd0c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009f20 tx=0x7fb8c0009840 comp rx=0 tx=0).stop 2023-10-06T05:17:24.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.017+0000 7fb8cddff700 1 -- 172.21.15.31:0/3153780872 shutdown_connections 2023-10-06T05:17:24.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.017+0000 7fb8cddff700 1 --2- 172.21.15.31:0/3153780872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c80ccc80 0x7fb8c80cd0c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.017+0000 7fb8cddff700 1 --2- 172.21.15.31:0/3153780872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8c8159580 0x7fb8c80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.017+0000 7fb8cddff700 1 -- 172.21.15.31:0/3153780872 >> 172.21.15.31:0/3153780872 conn(0x7fb8c8099710 msgr2=0x7fb8c806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:24.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.017+0000 7fb8cddff700 1 -- 172.21.15.31:0/3153780872 shutdown_connections 2023-10-06T05:17:24.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.017+0000 7fb8cddff700 1 -- 172.21.15.31:0/3153780872 wait complete. 2023-10-06T05:17:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.019+0000 7fb8cddff700 1 Processor -- start 2023-10-06T05:17:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.019+0000 7fb8cddff700 1 -- start start 2023-10-06T05:17:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.020+0000 7fb8cddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8c80cb9f0 0x7fb8c80c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.020+0000 7fb8cddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c80ccc80 0x7fb8c816d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.020+0000 7fb8cddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c8159580 0x7fb8c80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.020+0000 7fb8cddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80ca800 con 0x7fb8c8159580 2023-10-06T05:17:24.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.020+0000 7fb8cddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c816da60 con 0x7fb8c80ccc80 2023-10-06T05:17:24.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.020+0000 7fb8cddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80c4fb0 con 0x7fb8c80cb9f0 2023-10-06T05:17:24.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c8159580 0x7fb8c80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c8159580 0x7fb8c80c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51178/0 (socket says 172.21.15.31:51178) 2023-10-06T05:17:24.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c7fff700 1 -- 172.21.15.31:0/2945210408 learned_addr learned my addr 172.21.15.31:0/2945210408 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:24.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c77fe700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8c80cb9f0 0x7fb8c80c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c6ffd700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c80ccc80 0x7fb8c816d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c7fff700 1 -- 172.21.15.31:0/2945210408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8c80cb9f0 msgr2=0x7fb8c80c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c7fff700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8c80cb9f0 0x7fb8c80c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c7fff700 1 -- 172.21.15.31:0/2945210408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c80ccc80 msgr2=0x7fb8c816d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c7fff700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c80ccc80 0x7fb8c816d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c7fff700 1 -- 172.21.15.31:0/2945210408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c0009ae0 con 0x7fb8c8159580 2023-10-06T05:17:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.021+0000 7fb8c7fff700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c8159580 0x7fb8c80c3660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000b6c0 tx=0x7fb8c0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.022+0000 7fb8af7fe700 1 -- 172.21.15.31:0/2945210408 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c00032f0 con 0x7fb8c8159580 2023-10-06T05:17:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.022+0000 7fb8af7fe700 1 -- 172.21.15.31:0/2945210408 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0008d50 con 0x7fb8c8159580 2023-10-06T05:17:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.022+0000 7fb8af7fe700 1 -- 172.21.15.31:0/2945210408 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0026690 con 0x7fb8c8159580 2023-10-06T05:17:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.022+0000 7fb8cddff700 1 -- 172.21.15.31:0/2945210408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c80c3ba0 con 0x7fb8c8159580 2023-10-06T05:17:24.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.022+0000 7fb8cddff700 1 -- 172.21.15.31:0/2945210408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c816df70 con 0x7fb8c8159580 2023-10-06T05:17:24.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.023+0000 7fb8cddff700 1 -- 172.21.15.31:0/2945210408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c81520e0 con 0x7fb8c8159580 2023-10-06T05:17:24.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.024+0000 7fb8af7fe700 1 -- 172.21.15.31:0/2945210408 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8c0003040 con 0x7fb8c8159580 2023-10-06T05:17:24.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.024+0000 7fb8af7fe700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb8b4066f30 0x7fb8b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:24.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.025+0000 7fb8af7fe700 1 -- 172.21.15.31:0/2945210408 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb8c00863f0 con 0x7fb8c8159580 2023-10-06T05:17:24.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.025+0000 7fb8c77fe700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb8b4066f30 0x7fb8b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:24.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.025+0000 7fb8c77fe700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb8b4066f30 0x7fb8b40693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80061f0 tx=0x7fb8b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:24.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.029+0000 7fb8af7fe700 1 -- 172.21.15.31:0/2945210408 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8c0053510 con 0x7fb8c8159580 2023-10-06T05:17:24.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.278+0000 7fb8cddff700 1 -- 172.21.15.31:0/2945210408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb8c804c3b0 con 0x7fb8c8159580 2023-10-06T05:17:24.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.279+0000 7fb8af7fe700 1 -- 172.21.15.31:0/2945210408 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb8c0056b30 con 0x7fb8c8159580 2023-10-06T05:17:24.282 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:17:24.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 -- 172.21.15.31:0/2945210408 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb8b4066f30 msgr2=0x7fb8b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb8b4066f30 0x7fb8b40693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80061f0 tx=0x7fb8b8008040 comp rx=0 tx=0).stop 2023-10-06T05:17:24.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 -- 172.21.15.31:0/2945210408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c8159580 msgr2=0x7fb8c80c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:24.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c8159580 0x7fb8c80c3660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000b6c0 tx=0x7fb8c0018000 comp rx=0 tx=0).stop 2023-10-06T05:17:24.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 -- 172.21.15.31:0/2945210408 shutdown_connections 2023-10-06T05:17:24.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb8b4066f30 0x7fb8b40693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8c8159580 0x7fb8c80c3660 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8c80cb9f0 0x7fb8c80c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 --2- 172.21.15.31:0/2945210408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c80ccc80 0x7fb8c816d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:24.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 -- 172.21.15.31:0/2945210408 >> 172.21.15.31:0/2945210408 conn(0x7fb8c8099710 msgr2=0x7fb8c81519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:24.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 -- 172.21.15.31:0/2945210408 shutdown_connections 2023-10-06T05:17:24.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:24.281+0000 7fb8cddff700 1 -- 172.21.15.31:0/2945210408 wait complete. 2023-10-06T05:17:24.300 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:17:24.300 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:17:24.300 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:17:24.300 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T05:17:24.300 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T05:17:24.300 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:17:24.300 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:17:24.300 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:17:24.300 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T05:17:24.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:24.301 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:17:24.347 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:17:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:17:24.348 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42270sec preferred_lft 42270sec 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:17:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:17:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:24.350 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:17:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:17:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link tentative 2023-10-06T05:17:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:24.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:24.351 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:17:24.460 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:17:24.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:24.463 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:17:24.512 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:17:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:17:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:17:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:17:24.513 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:17:24.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:24.513 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:17:24.513 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:17:24.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:17:24.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:17:24.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:17:24.514 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:17:24.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:24.685 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:17:24.732 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:17:24.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:24.733 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-10-06T05:17:24.779 INFO:teuthology.orchestra.run.smithi031.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T05:17:24.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:24.780 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:17:24.780 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:17:24.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:17:24.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T05:17:24.781 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:17:24.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:24.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T05:17:24.995 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:85:5c:9e:e2:40 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c485:5cff:fe9e:e240/64 scope link 2023-10-06T05:17:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:24.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:24.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T05:17:25.070 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:17:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:17:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:17:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:17:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:25.070 INFO:teuthology.orchestra.run.smithi031.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:17:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:85:5c:9e:e2:40 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:17:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:17:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c485:5cff:fe9e:e240/64 scope link 2023-10-06T05:17:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:25.073 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T05:17:25.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:25.073 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:17:25.073 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:17:25.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T05:17:25.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T05:17:25.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T05:17:25.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T05:17:25.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T05:17:25.074 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:17:25.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:25.321 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:17:25.321 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:17:25.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.1 up 2023-10-06T05:17:25.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T05:17:25.321 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:17:25.497 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:25.497 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T05:17:25.543 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T05:17:25.543 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:25.544 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T05:17:25.589 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-06T05:17:25.589 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:17:25.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:25.729 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:25.730 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:17:25.756 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:25.756 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:17:25.756 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:17:25.756 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 2@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:17:25.756 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:17:25.760 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:17:25.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:25.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T05:17:25.831 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-06T05:17:25.831 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:17:26.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.025+0000 7f4ac03ff700 1 Processor -- start 2023-10-06T05:17:26.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.025+0000 7f4ac03ff700 1 -- start start 2023-10-06T05:17:26.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.026+0000 7f4ac03ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4ab80c7f30 0x7f4ab8069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.026+0000 7f4ac03ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab80ca5b0 con 0x7f4ab80aee70 2023-10-06T05:17:26.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.026+0000 7f4ac03ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab80af2f0 con 0x7f4ab80c7b60 2023-10-06T05:17:26.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.026+0000 7f4ac03ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab806f740 con 0x7f4ab80c7f30 2023-10-06T05:17:26.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.027+0000 7f4abd99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4ab80c7f30 0x7f4ab8069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.027+0000 7f4abd99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4ab80c7f30 0x7f4ab8069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51192/0 (socket says 172.21.15.31:51192) 2023-10-06T05:17:26.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.027+0000 7f4abd99a700 1 -- 172.21.15.31:0/1540320262 learned_addr learned my addr 172.21.15.31:0/1540320262 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:26.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.027+0000 7f4abe99c700 1 --1- 172.21.15.31:0/1540320262 >> v1:172.21.15.31:6789/0 conn(0x7f4ab80aee70 0x7f4ab813f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58706/0 (socket says 172.21.15.31:58706) 2023-10-06T05:17:26.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.027+0000 7f4abd99a700 1 -- 172.21.15.31:0/1540320262 >> v1:172.21.15.62:6789/0 conn(0x7f4ab80c7b60 legacy=0x7f4ab806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.027+0000 7f4abd99a700 1 -- 172.21.15.31:0/1540320262 >> v1:172.21.15.31:6789/0 conn(0x7f4ab80aee70 legacy=0x7f4ab813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.027+0000 7f4abd99a700 1 -- 172.21.15.31:0/1540320262 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab80c9c80 con 0x7f4ab80c7f30 2023-10-06T05:17:26.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.027+0000 7f4abd199700 1 -- 172.21.15.31:0/1540320262 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124778881 0 0) 0x7f4ab80ca5b0 con 0x7f4ab80aee70 2023-10-06T05:17:26.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.027+0000 7f4abd99a700 1 --2- 172.21.15.31:0/1540320262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 0x7f4ab8069880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80099b0 tx=0x7f4aa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b146a8b9e6c6b6cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:26.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.028+0000 7f4abd199700 1 -- 172.21.15.31:0/1540320262 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800b860 con 0x7f4ab80c7f30 2023-10-06T05:17:26.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.028+0000 7f4abd199700 1 -- 172.21.15.31:0/1540320262 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa800b9c0 con 0x7f4ab80c7f30 2023-10-06T05:17:26.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.028+0000 7f4abd199700 1 -- 172.21.15.31:0/1540320262 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800bd60 con 0x7f4ab80c7f30 2023-10-06T05:17:26.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.028+0000 7f4ac03ff700 1 -- 172.21.15.31:0/1540320262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 msgr2=0x7f4ab8069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.028+0000 7f4ac03ff700 1 --2- 172.21.15.31:0/1540320262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 0x7f4ab8069880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80099b0 tx=0x7f4aa8009250 comp rx=0 tx=0).stop 2023-10-06T05:17:26.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.028+0000 7f4ac03ff700 1 -- 172.21.15.31:0/1540320262 shutdown_connections 2023-10-06T05:17:26.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.028+0000 7f4ac03ff700 1 --2- 172.21.15.31:0/1540320262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 0x7f4ab8069880 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.028+0000 7f4ac03ff700 1 -- 172.21.15.31:0/1540320262 >> 172.21.15.31:0/1540320262 conn(0x7f4ab80cc0b0 msgr2=0x7f4ab80cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:26.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.028+0000 7f4ac03ff700 1 -- 172.21.15.31:0/1540320262 shutdown_connections 2023-10-06T05:17:26.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.028+0000 7f4ac03ff700 1 -- 172.21.15.31:0/1540320262 wait complete. 2023-10-06T05:17:26.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.030+0000 7f4ac03ff700 1 Processor -- start 2023-10-06T05:17:26.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.030+0000 7f4ac03ff700 1 -- start start 2023-10-06T05:17:26.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4ac03ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab80aee70 0x7f4ab813ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4ac03ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ab80c7b60 0x7f4ab815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4ac03ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 0x7f4ab80b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4ac03ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab815a940 con 0x7f4ab80c7f30 2023-10-06T05:17:26.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4ac03ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab8159d30 con 0x7f4ab80aee70 2023-10-06T05:17:26.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4ac03ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab815a010 con 0x7f4ab80c7b60 2023-10-06T05:17:26.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4abe99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 0x7f4ab80b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4abe99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 0x7f4ab80b0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51198/0 (socket says 172.21.15.31:51198) 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4abe99c700 1 -- 172.21.15.31:0/207109078 learned_addr learned my addr 172.21.15.31:0/207109078 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4abd99a700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ab80c7b60 0x7f4ab815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.031+0000 7f4abe19b700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab80aee70 0x7f4ab813ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4abe99c700 1 -- 172.21.15.31:0/207109078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ab80c7b60 msgr2=0x7f4ab815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4abe99c700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ab80c7b60 0x7f4ab815b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4abe99c700 1 -- 172.21.15.31:0/207109078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab80aee70 msgr2=0x7f4ab813ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4abe99c700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab80aee70 0x7f4ab813ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4abe99c700 1 -- 172.21.15.31:0/207109078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80094f0 con 0x7f4ab80c7f30 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4abe19b700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab80aee70 0x7f4ab813ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4abe99c700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 0x7f4ab80b0850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000cd50 tx=0x7f4ab0001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:26.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4aae7fc700 1 -- 172.21.15.31:0/207109078 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0005df0 con 0x7f4ab80c7f30 2023-10-06T05:17:26.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4aae7fc700 1 -- 172.21.15.31:0/207109078 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab000a960 con 0x7f4ab80c7f30 2023-10-06T05:17:26.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4aae7fc700 1 -- 172.21.15.31:0/207109078 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000ad20 con 0x7f4ab80c7f30 2023-10-06T05:17:26.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4ac03ff700 1 -- 172.21.15.31:0/207109078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab815a2f0 con 0x7f4ab80c7f30 2023-10-06T05:17:26.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.032+0000 7f4ac03ff700 1 -- 172.21.15.31:0/207109078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab815b880 con 0x7f4ab80c7f30 2023-10-06T05:17:26.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.033+0000 7f4ac03ff700 1 -- 172.21.15.31:0/207109078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab815a790 con 0x7f4ab80c7f30 2023-10-06T05:17:26.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.034+0000 7f4aae7fc700 1 -- 172.21.15.31:0/207109078 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ab0007300 con 0x7f4ab80c7f30 2023-10-06T05:17:26.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.035+0000 7f4aae7fc700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4a9c066f30 0x7f4a9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:26.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.035+0000 7f4aae7fc700 1 -- 172.21.15.31:0/207109078 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4ab00849a0 con 0x7f4ab80c7f30 2023-10-06T05:17:26.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.035+0000 7f4abe19b700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4a9c066f30 0x7f4a9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:26.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.036+0000 7f4abe19b700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4a9c066f30 0x7f4a9c0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40018f0 tx=0x7f4ab4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:26.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.040+0000 7f4aae7fc700 1 -- 172.21.15.31:0/207109078 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ab0051a90 con 0x7f4ab80c7f30 2023-10-06T05:17:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.288+0000 7f4ac03ff700 1 -- 172.21.15.31:0/207109078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4ab80ccbd0 con 0x7f4ab80c7f30 2023-10-06T05:17:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.289+0000 7f4aae7fc700 1 -- 172.21.15.31:0/207109078 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4ab00550b0 con 0x7f4ab80c7f30 2023-10-06T05:17:26.291 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 -- 172.21.15.31:0/207109078 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4a9c066f30 msgr2=0x7f4a9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4a9c066f30 0x7f4a9c0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40018f0 tx=0x7f4ab4009040 comp rx=0 tx=0).stop 2023-10-06T05:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 -- 172.21.15.31:0/207109078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 msgr2=0x7f4ab80b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 0x7f4ab80b0850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000cd50 tx=0x7f4ab0001b10 comp rx=0 tx=0).stop 2023-10-06T05:17:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 -- 172.21.15.31:0/207109078 shutdown_connections 2023-10-06T05:17:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4a9c066f30 0x7f4a9c0693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ab80c7f30 0x7f4ab80b0850 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ab80aee70 0x7f4ab813ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 --2- 172.21.15.31:0/207109078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ab80c7b60 0x7f4ab815b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 -- 172.21.15.31:0/207109078 >> 172.21.15.31:0/207109078 conn(0x7f4ab80cc0b0 msgr2=0x7f4ab813eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 -- 172.21.15.31:0/207109078 shutdown_connections 2023-10-06T05:17:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:26.291+0000 7f4ac03ff700 1 -- 172.21.15.31:0/207109078 wait complete. 2023-10-06T05:17:26.310 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:17:26.310 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:17:26.310 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:17:26.310 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:17:26.311 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:17:26.311 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:17:26.311 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:17:26.311 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:17:26.311 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:17:26.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:26.311 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42268sec preferred_lft 42268sec 2023-10-06T05:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:26.365 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T05:17:26.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:26.366 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:17:26.366 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:17:26.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2023-10-06T05:17:26.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2023-10-06T05:17:26.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2023-10-06T05:17:26.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T05:17:26.366 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:17:26.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:26.805 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:17:26.878 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:17:26.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:26.881 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:17:26.961 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:17:26.962 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:17:26.962 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:17:26.962 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:17:26.962 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:17:26.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:26.963 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:17:26.963 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:17:26.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:17:26.964 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:17:26.964 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:17:26.964 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:17:27.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:27.673 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:17:27.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:27.719 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2023-10-06T05:17:27.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:27.766 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:17:27.766 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:17:27.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:17:27.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T05:17:27.767 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:17:27.952 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T05:17:27.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:27.953 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:17:27.953 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:17:27.953 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T05:17:27.953 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T05:17:27.953 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T05:17:27.953 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T05:17:27.953 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T05:17:27.953 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:17:28.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:28.225 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:17:28.225 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:17:28.225 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:17:28.225 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T05:17:28.225 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:17:28.400 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:28.400 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:17:28.447 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:17:28.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:28.447 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:17:28.532 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:17:28.533 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:17:28.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:28.673 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:28.673 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:17:29.105 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:29.106 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:17:29.106 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:17:29.106 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:17:29.106 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:17:29.110 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:17:29.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:29.110 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:17:29.183 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-06T05:17:29.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:17:29.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.388+0000 7f2c7d9ff700 1 Processor -- start 2023-10-06T05:17:29.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.388+0000 7f2c7d9ff700 1 -- start start 2023-10-06T05:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.389+0000 7f2c7d9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2c78159500 0x7f2c780b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.389+0000 7f2c7d9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c780c3f40 con 0x7f2c780cd510 2023-10-06T05:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.389+0000 7f2c7d9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c780cb000 con 0x7f2c7815d2d0 2023-10-06T05:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.389+0000 7f2c7d9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c780b2860 con 0x7f2c78159500 2023-10-06T05:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.389+0000 7f2c777fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2c780cd510 0x7f2c780aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46172/0 (socket says 172.21.15.31:46172) 2023-10-06T05:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.389+0000 7f2c777fe700 1 -- 172.21.15.31:0/3497269686 learned_addr learned my addr 172.21.15.31:0/3497269686 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.389+0000 7f2c76ffd700 1 --2- 172.21.15.31:0/3497269686 >> v2:172.21.15.62:3300/0 conn(0x7f2c78159500 0x7f2c780b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.390+0000 7f2c75ffb700 1 -- 172.21.15.31:0/3497269686 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402129638 0 0) 0x7f2c780c3f40 con 0x7f2c780cd510 2023-10-06T05:17:29.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.390+0000 7f2c75ffb700 1 -- 172.21.15.31:0/3497269686 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c60003570 con 0x7f2c780cd510 2023-10-06T05:17:29.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.390+0000 7f2c75ffb700 1 -- 172.21.15.31:0/3497269686 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882543284 0 0) 0x7f2c780cb000 con 0x7f2c7815d2d0 2023-10-06T05:17:29.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.390+0000 7f2c75ffb700 1 -- 172.21.15.31:0/3497269686 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c780c3f40 con 0x7f2c7815d2d0 2023-10-06T05:17:29.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.390+0000 7f2c76ffd700 1 -- 172.21.15.31:0/3497269686 >> v1:172.21.15.62:6789/0 conn(0x7f2c7815d2d0 legacy=0x7f2c780cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.390+0000 7f2c76ffd700 1 -- 172.21.15.31:0/3497269686 >> v1:172.21.15.31:6789/0 conn(0x7f2c780cd510 legacy=0x7f2c780aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.390+0000 7f2c76ffd700 1 -- 172.21.15.31:0/3497269686 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7815d7d0 con 0x7f2c78159500 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.390+0000 7f2c76ffd700 1 --2- 172.21.15.31:0/3497269686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c78159500 0x7f2c780b0ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2c680098e0 tx=0x7f2c680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d79ce2739b4e257 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.390+0000 7f2c75ffb700 1 -- 172.21.15.31:0/3497269686 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68011630 con 0x7f2c78159500 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.390+0000 7f2c75ffb700 1 -- 172.21.15.31:0/3497269686 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c68011790 con 0x7f2c78159500 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.391+0000 7f2c75ffb700 1 -- 172.21.15.31:0/3497269686 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68011b30 con 0x7f2c78159500 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.391+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3497269686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c78159500 msgr2=0x7f2c780b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.391+0000 7f2c7d9ff700 1 --2- 172.21.15.31:0/3497269686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c78159500 0x7f2c780b0ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2c680098e0 tx=0x7f2c680091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.391+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3497269686 shutdown_connections 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.391+0000 7f2c7d9ff700 1 --2- 172.21.15.31:0/3497269686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c78159500 0x7f2c780b0ab0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.391+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3497269686 >> 172.21.15.31:0/3497269686 conn(0x7f2c7806ec90 msgr2=0x7f2c7806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.391+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3497269686 shutdown_connections 2023-10-06T05:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.391+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3497269686 wait complete. 2023-10-06T05:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.393+0000 7f2c7d9ff700 1 Processor -- start 2023-10-06T05:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.393+0000 7f2c7d9ff700 1 -- start start 2023-10-06T05:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c7d9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c780cd510 0x7f2c780c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c7d9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c78159500 0x7f2c780ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c7d9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c7815d2d0 0x7f2c78157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c7d9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780c4610 con 0x7f2c780cd510 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c7d9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780b3330 con 0x7f2c7815d2d0 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c7d9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780ca790 con 0x7f2c78159500 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c780cd510 0x7f2c780c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c780cd510 0x7f2c780c40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53820/0 (socket says 172.21.15.31:53820) 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c76ffd700 1 -- 172.21.15.31:0/3347653970 learned_addr learned my addr 172.21.15.31:0/3347653970 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c767fc700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c78159500 0x7f2c780ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c777fe700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c7815d2d0 0x7f2c78157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c76ffd700 1 -- 172.21.15.31:0/3347653970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c78159500 msgr2=0x7f2c780ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c76ffd700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c78159500 0x7f2c780ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c76ffd700 1 -- 172.21.15.31:0/3347653970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c7815d2d0 msgr2=0x7f2c78157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c76ffd700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c7815d2d0 0x7f2c78157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c76ffd700 1 -- 172.21.15.31:0/3347653970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c68009420 con 0x7f2c780cd510 2023-10-06T05:17:29.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c777fe700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c7815d2d0 0x7f2c78157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:29.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.394+0000 7f2c767fc700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c78159500 0x7f2c780ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:29.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.395+0000 7f2c76ffd700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c780cd510 0x7f2c780c40d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2c78067c50 tx=0x7f2c68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:29.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.395+0000 7f2c5effd700 1 -- 172.21.15.31:0/3347653970 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6801f070 con 0x7f2c780cd510 2023-10-06T05:17:29.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.395+0000 7f2c5effd700 1 -- 172.21.15.31:0/3347653970 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c68011b10 con 0x7f2c780cd510 2023-10-06T05:17:29.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.395+0000 7f2c5effd700 1 -- 172.21.15.31:0/3347653970 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68023640 con 0x7f2c780cd510 2023-10-06T05:17:29.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.395+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3347653970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c780c4840 con 0x7f2c780cd510 2023-10-06T05:17:29.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.395+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3347653970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c780b3880 con 0x7f2c780cd510 2023-10-06T05:17:29.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.396+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3347653970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7804c380 con 0x7f2c780cd510 2023-10-06T05:17:29.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.397+0000 7f2c5effd700 1 -- 172.21.15.31:0/3347653970 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c6802c3b0 con 0x7f2c780cd510 2023-10-06T05:17:29.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.398+0000 7f2c5effd700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2c64066f30 0x7f2c640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:29.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.398+0000 7f2c5effd700 1 -- 172.21.15.31:0/3347653970 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2c68085a70 con 0x7f2c780cd510 2023-10-06T05:17:29.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.398+0000 7f2c767fc700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2c64066f30 0x7f2c640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:29.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.399+0000 7f2c767fc700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2c64066f30 0x7f2c640693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2c780af250 tx=0x7f2c6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:29.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.402+0000 7f2c5effd700 1 -- 172.21.15.31:0/3347653970 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c68052ae0 con 0x7f2c780cd510 2023-10-06T05:17:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.647+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3347653970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2c780cade0 con 0x7f2c780cd510 2023-10-06T05:17:29.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.647+0000 7f2c5effd700 1 -- 172.21.15.31:0/3347653970 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2c68015070 con 0x7f2c780cd510 2023-10-06T05:17:29.650 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.649+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3347653970 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2c64066f30 msgr2=0x7f2c640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.649+0000 7f2c7d9ff700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2c64066f30 0x7f2c640693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2c780af250 tx=0x7f2c6c009040 comp rx=0 tx=0).stop 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.649+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3347653970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c780cd510 msgr2=0x7f2c780c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.649+0000 7f2c7d9ff700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c780cd510 0x7f2c780c40d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2c78067c50 tx=0x7f2c68017000 comp rx=0 tx=0).stop 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.650+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3347653970 shutdown_connections 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.650+0000 7f2c7d9ff700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c7815d2d0 0x7f2c78157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.650+0000 7f2c7d9ff700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2c64066f30 0x7f2c640693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.650+0000 7f2c7d9ff700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c780cd510 0x7f2c780c40d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.650+0000 7f2c7d9ff700 1 --2- 172.21.15.31:0/3347653970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c78159500 0x7f2c780ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.650+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3347653970 >> 172.21.15.31:0/3347653970 conn(0x7f2c7806ec90 msgr2=0x7f2c78158890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.650+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3347653970 shutdown_connections 2023-10-06T05:17:29.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:29.650+0000 7f2c7d9ff700 1 -- 172.21.15.31:0/3347653970 wait complete. 2023-10-06T05:17:29.670 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T05:17:29.670 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:17:29.670 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:17:29.670 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T05:17:29.670 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T05:17:29.670 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T05:17:29.670 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T05:17:29.670 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:17:29.671 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T05:17:29.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:29.671 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:17:29.720 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:17:29.720 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:17:29.720 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:17:29.720 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:17:29.721 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42265sec preferred_lft 42265sec 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:29.722 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:17:29.723 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:17:29.723 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link tentative 2023-10-06T05:17:29.723 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:29.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:29.724 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:17:29.793 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:17:29.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:29.796 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:17:29.845 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:17:29.845 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:17:29.845 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:17:29.845 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:17:29.845 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:17:29.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:29.846 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:17:29.846 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:17:29.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:17:29.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:17:29.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:17:29.847 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:17:30.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:30.020 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:17:30.066 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:17:30.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:30.067 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2023-10-06T05:17:30.113 INFO:teuthology.orchestra.run.smithi062.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T05:17:30.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:30.114 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:17:30.114 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:17:30.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:17:30.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T05:17:30.115 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:17:30.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:30.235 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:17:30.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:30.301 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T05:17:30.373 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:17:30.373 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:17:30.373 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:17:30.373 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:30.373 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:17:30.373 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:30.374 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:17:30.374 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether d6:d0:14:6e:d0:8d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:17:30.374 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:17:30.374 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:30.374 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d4d0:14ff:fe6e:d08d/64 scope link 2023-10-06T05:17:30.374 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:30.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:30.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T05:17:30.487 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:17:30.487 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:17:30.487 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:17:30.488 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:30.488 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:17:30.488 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:30.488 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:17:30.488 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether d6:d0:14:6e:d0:8d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T05:17:30.488 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T05:17:30.488 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:30.488 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d4d0:14ff:fe6e:d08d/64 scope link 2023-10-06T05:17:30.488 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:17:30.490 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T05:17:30.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:30.491 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:17:30.491 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:17:30.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T05:17:30.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T05:17:30.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T05:17:30.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T05:17:30.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T05:17:30.491 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:17:30.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:30.748 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:17:30.748 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:17:30.748 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.1 up 2023-10-06T05:17:30.748 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T05:17:30.748 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:17:30.922 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:30.922 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T05:17:30.968 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T05:17:30.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:17:30.968 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T05:17:31.013 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-06T05:17:31.013 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:17:31.013 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:31.156 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:31.156 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:17:31.183 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:17:31.183 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:17:31.183 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:17:31.183 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 3@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:17:31.183 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:17:31.187 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:17:31.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:31.187 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T05:17:31.257 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T05:17:31.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:31.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.477+0000 7f22a77ff700 1 Processor -- start 2023-10-06T05:17:31.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.477+0000 7f22a77ff700 1 -- start start 2023-10-06T05:17:31.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a77ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f22a013a5b0 0x7f22a006d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a77ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22a006d980 0x7f22a0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a77ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a006ddb0 con 0x7f22a013a1e0 2023-10-06T05:17:31.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a77ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a00c71b0 con 0x7f22a006d980 2023-10-06T05:17:31.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a77ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a00c7490 con 0x7f22a013a5b0 2023-10-06T05:17:31.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a5d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22a006d980 0x7f22a0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a559b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f22a013a1e0 0x7f22a00c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46188/0 (socket says 172.21.15.31:46188) 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a559b700 1 -- 172.21.15.31:0/2022895905 learned_addr learned my addr 172.21.15.31:0/2022895905 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a4d9a700 1 --2- 172.21.15.31:0/2022895905 >> v2:172.21.15.62:3300/0 conn(0x7f22a013a5b0 0x7f22a006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a5d9c700 1 -- 172.21.15.31:0/2022895905 >> v2:172.21.15.62:3300/0 conn(0x7f22a013a5b0 msgr2=0x7f22a006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a5d9c700 1 --2- 172.21.15.31:0/2022895905 >> v2:172.21.15.62:3300/0 conn(0x7f22a013a5b0 0x7f22a006d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a5d9c700 1 -- 172.21.15.31:0/2022895905 >> v1:172.21.15.31:6789/0 conn(0x7f22a013a1e0 legacy=0x7f22a00c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.479+0000 7f22a5d9c700 1 -- 172.21.15.31:0/2022895905 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a00c7770 con 0x7f22a006d980 2023-10-06T05:17:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.480+0000 7f22a5d9c700 1 --2- 172.21.15.31:0/2022895905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 0x7f22a0069560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f22a006d130 tx=0x7f2298009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af8ac3a4c057ecb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.480+0000 7f2297fff700 1 -- 172.21.15.31:0/2022895905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800e070 con 0x7f22a006d980 2023-10-06T05:17:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.480+0000 7f2297fff700 1 -- 172.21.15.31:0/2022895905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229800bc30 con 0x7f22a006d980 2023-10-06T05:17:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.480+0000 7f2297fff700 1 -- 172.21.15.31:0/2022895905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298012430 con 0x7f22a006d980 2023-10-06T05:17:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.480+0000 7f22a77ff700 1 -- 172.21.15.31:0/2022895905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 msgr2=0x7f22a0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.480+0000 7f22a77ff700 1 --2- 172.21.15.31:0/2022895905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 0x7f22a0069560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f22a006d130 tx=0x7f2298009780 comp rx=0 tx=0).stop 2023-10-06T05:17:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.481+0000 7f22a77ff700 1 -- 172.21.15.31:0/2022895905 shutdown_connections 2023-10-06T05:17:31.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.481+0000 7f22a77ff700 1 --2- 172.21.15.31:0/2022895905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 0x7f22a0069560 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.481+0000 7f22a77ff700 1 --2- 172.21.15.31:0/2022895905 >> v2:172.21.15.62:3300/0 conn(0x7f22a013a5b0 0x7f22a006d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.481+0000 7f22a77ff700 1 -- 172.21.15.31:0/2022895905 >> 172.21.15.31:0/2022895905 conn(0x7f22a00cc0b0 msgr2=0x7f22a00cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:31.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.481+0000 7f22a77ff700 1 -- 172.21.15.31:0/2022895905 shutdown_connections 2023-10-06T05:17:31.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.481+0000 7f22a77ff700 1 -- 172.21.15.31:0/2022895905 wait complete. 2023-10-06T05:17:31.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.482+0000 7f22a77ff700 1 Processor -- start 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.482+0000 7f22a77ff700 1 -- start start 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.483+0000 7f22a77ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 0x7f22a00b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.483+0000 7f22a77ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22a013a1e0 0x7f22a00af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.483+0000 7f22a77ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22a013a5b0 0x7f22a014dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.483+0000 7f22a77ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00c4fd0 con 0x7f22a006d980 2023-10-06T05:17:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.483+0000 7f22a77ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00c52b0 con 0x7f22a013a1e0 2023-10-06T05:17:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.483+0000 7f22a77ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00c5590 con 0x7f22a013a5b0 2023-10-06T05:17:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a559b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 0x7f22a00b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a559b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 0x7f22a00b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53832/0 (socket says 172.21.15.31:53832) 2023-10-06T05:17:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a559b700 1 -- 172.21.15.31:0/3191297754 learned_addr learned my addr 172.21.15.31:0/3191297754 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a4d9a700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22a013a1e0 0x7f22a00af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a5d9c700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22a013a5b0 0x7f22a014dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a559b700 1 -- 172.21.15.31:0/3191297754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22a013a5b0 msgr2=0x7f22a014dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a559b700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22a013a5b0 0x7f22a014dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a559b700 1 -- 172.21.15.31:0/3191297754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22a013a1e0 msgr2=0x7f22a00af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a559b700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22a013a1e0 0x7f22a00af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a559b700 1 -- 172.21.15.31:0/3191297754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298009a20 con 0x7f22a006d980 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.484+0000 7f22a559b700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 0x7f22a00b1610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c7c0 tx=0x7f229c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:31.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.485+0000 7f22957fa700 1 -- 172.21.15.31:0/3191297754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c001ea0 con 0x7f22a006d980 2023-10-06T05:17:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.485+0000 7f22957fa700 1 -- 172.21.15.31:0/3191297754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c0073b0 con 0x7f22a006d980 2023-10-06T05:17:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.485+0000 7f22957fa700 1 -- 172.21.15.31:0/3191297754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0057f0 con 0x7f22a006d980 2023-10-06T05:17:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.485+0000 7f22a77ff700 1 -- 172.21.15.31:0/3191297754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a00c5870 con 0x7f22a006d980 2023-10-06T05:17:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.485+0000 7f22a77ff700 1 -- 172.21.15.31:0/3191297754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a00af7f0 con 0x7f22a006d980 2023-10-06T05:17:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.486+0000 7f22a77ff700 1 -- 172.21.15.31:0/3191297754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a00afa10 con 0x7f22a006d980 2023-10-06T05:17:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.487+0000 7f22957fa700 1 -- 172.21.15.31:0/3191297754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f229c005c20 con 0x7f22a006d980 2023-10-06T05:17:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.488+0000 7f22957fa700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.488+0000 7f22957fa700 1 -- 172.21.15.31:0/3191297754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f229c083a40 con 0x7f22a006d980 2023-10-06T05:17:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.488+0000 7f22a4d9a700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.489+0000 7f22a4d9a700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2284066f30 0x7f22840693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2290006fd0 tx=0x7f2290008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:31.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.492+0000 7f22957fa700 1 -- 172.21.15.31:0/3191297754 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f229c04b5a0 con 0x7f22a006d980 2023-10-06T05:17:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.699+0000 7f22a77ff700 1 -- 172.21.15.31:0/3191297754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22a004c380 con 0x7f22a006d980 2023-10-06T05:17:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.700+0000 7f22957fa700 1 -- 172.21.15.31:0/3191297754 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f229c050ab0 con 0x7f22a006d980 2023-10-06T05:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:31.703 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:31.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 -- 172.21.15.31:0/3191297754 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2284066f30 msgr2=0x7f22840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2284066f30 0x7f22840693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2290006fd0 tx=0x7f2290008040 comp rx=0 tx=0).stop 2023-10-06T05:17:31.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 -- 172.21.15.31:0/3191297754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 msgr2=0x7f22a00b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 0x7f22a00b1610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c7c0 tx=0x7f229c007de0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 -- 172.21.15.31:0/3191297754 shutdown_connections 2023-10-06T05:17:31.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22a013a5b0 0x7f22a014dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22a006d980 0x7f22a00b1610 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 --2- 172.21.15.31:0/3191297754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22a013a1e0 0x7f22a00af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 -- 172.21.15.31:0/3191297754 >> 172.21.15.31:0/3191297754 conn(0x7f22a00cc0b0 msgr2=0x7f22a00c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:31.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 -- 172.21.15.31:0/3191297754 shutdown_connections 2023-10-06T05:17:31.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.702+0000 7f22a77ff700 1 -- 172.21.15.31:0/3191297754 wait complete. 2023-10-06T05:17:31.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:16:59.507957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:17:00.905826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:17:31.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:31.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.918+0000 7fdb293ff700 1 Processor -- start 2023-10-06T05:17:31.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.919+0000 7fdb293ff700 1 -- start start 2023-10-06T05:17:31.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.920+0000 7fdb293ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdb2406b790 0x7fdb240b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.920+0000 7fdb293ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb240b0970 con 0x7fdb240ca5f0 2023-10-06T05:17:31.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.920+0000 7fdb293ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb240b0ba0 con 0x7fdb2406bb60 2023-10-06T05:17:31.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.920+0000 7fdb293ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb240af150 con 0x7fdb2406b790 2023-10-06T05:17:31.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.920+0000 7fdb227fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdb240ca5f0 0x7fdb240c9690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46196/0 (socket says 172.21.15.31:46196) 2023-10-06T05:17:31.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.920+0000 7fdb227fc700 1 -- 172.21.15.31:0/3560372616 learned_addr learned my addr 172.21.15.31:0/3560372616 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:31.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.920+0000 7fdb22ffd700 1 --2- 172.21.15.31:0/3560372616 >> v2:172.21.15.62:3300/0 conn(0x7fdb2406b790 0x7fdb240b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.920+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309320484 0 0) 0x7fdb240b0970 con 0x7fdb240ca5f0 2023-10-06T05:17:31.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb0c003570 con 0x7fdb240ca5f0 2023-10-06T05:17:31.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025641484 0 0) 0x7fdb240b0ba0 con 0x7fdb2406bb60 2023-10-06T05:17:31.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb240b0970 con 0x7fdb2406bb60 2023-10-06T05:17:31.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2065889023 0 0) 0x7fdb0c003570 con 0x7fdb240ca5f0 2023-10-06T05:17:31.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 >> v2:172.21.15.62:3300/0 conn(0x7fdb2406b790 msgr2=0x7fdb240b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:31.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 --2- 172.21.15.31:0/3560372616 >> v2:172.21.15.62:3300/0 conn(0x7fdb2406b790 0x7fdb240b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 >> v1:172.21.15.62:6789/0 conn(0x7fdb2406bb60 legacy=0x7fdb2415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb240c88d0 con 0x7fdb240ca5f0 2023-10-06T05:17:31.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fdb18001470 con 0x7fdb240ca5f0 2023-10-06T05:17:31.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb22ffd700 1 --2- 172.21.15.31:0/3560372616 >> v2:172.21.15.62:3300/0 conn(0x7fdb2406b790 0x7fdb240b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:31.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 >> v1:172.21.15.31:6789/0 conn(0x7fdb240ca5f0 legacy=0x7fdb240c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb0c003c40 0x7fdb0c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb0c004810 0x7fdb0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb0c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb240af150 con 0x7fdb0c005210 2023-10-06T05:17:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c00c130 con 0x7fdb0c003c40 2023-10-06T05:17:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c00c2c0 con 0x7fdb0c004810 2023-10-06T05:17:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.921+0000 7fdb237fe700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb0c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb22ffd700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb0c003c40 0x7fdb0c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb227fc700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb0c004810 0x7fdb0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb237fe700 1 -- 172.21.15.31:0/3560372616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb0c004810 msgr2=0x7fdb0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb237fe700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb0c004810 0x7fdb0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb237fe700 1 -- 172.21.15.31:0/3560372616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb0c003c40 msgr2=0x7fdb0c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb237fe700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb0c003c40 0x7fdb0c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb237fe700 1 -- 172.21.15.31:0/3560372616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c003570 con 0x7fdb0c005210 2023-10-06T05:17:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb237fe700 1 -- 172.21.15.31:0/3560372616 reap_dead start 2023-10-06T05:17:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb237fe700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb0c00ba60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c002150 tx=0x7fdb1c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf34bda5428913c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00c940 con 0x7fdb0c005210 2023-10-06T05:17:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c00caa0 con 0x7fdb0c005210 2023-10-06T05:17:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.922+0000 7fdb21ffb700 1 -- 172.21.15.31:0/3560372616 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00cc10 con 0x7fdb0c005210 2023-10-06T05:17:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.923+0000 7fdb293ff700 1 -- 172.21.15.31:0/3560372616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 msgr2=0x7fdb0c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.923+0000 7fdb293ff700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb0c00ba60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c002150 tx=0x7fdb1c0017a0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.923+0000 7fdb293ff700 1 -- 172.21.15.31:0/3560372616 shutdown_connections 2023-10-06T05:17:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.923+0000 7fdb293ff700 1 --2- 172.21.15.31:0/3560372616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb0c00ba60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.923+0000 7fdb293ff700 1 -- 172.21.15.31:0/3560372616 >> 172.21.15.31:0/3560372616 conn(0x7fdb240ce290 msgr2=0x7fdb240ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:31.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.923+0000 7fdb293ff700 1 -- 172.21.15.31:0/3560372616 shutdown_connections 2023-10-06T05:17:31.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.923+0000 7fdb293ff700 1 -- 172.21.15.31:0/3560372616 wait complete. 2023-10-06T05:17:31.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.924+0000 7fdb293ff700 1 Processor -- start 2023-10-06T05:17:31.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.925+0000 7fdb293ff700 1 -- start start 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.925+0000 7fdb293ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb240cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.925+0000 7fdb293ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb24156500 0x7fdb24156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.925+0000 7fdb293ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb240ac8c0 0x7fdb240acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb293ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb240aa1e0 con 0x7fdb0c005210 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb293ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb240c6b60 con 0x7fdb240ac8c0 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb293ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb24156eb0 con 0x7fdb24156500 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb22ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb240cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb22ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb240cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53842/0 (socket says 172.21.15.31:53842) 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb22ffd700 1 -- 172.21.15.31:0/2819589080 learned_addr learned my addr 172.21.15.31:0/2819589080 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb227fc700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb24156500 0x7fdb24156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb237fe700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb240ac8c0 0x7fdb240acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb22ffd700 1 -- 172.21.15.31:0/2819589080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb24156500 msgr2=0x7fdb24156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb22ffd700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb24156500 0x7fdb24156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb22ffd700 1 -- 172.21.15.31:0/2819589080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb240ac8c0 msgr2=0x7fdb240acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb22ffd700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb240ac8c0 0x7fdb240acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb22ffd700 1 -- 172.21.15.31:0/2819589080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb240ad270 con 0x7fdb0c005210 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.926+0000 7fdb237fe700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb240ac8c0 0x7fdb240acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.927+0000 7fdb22ffd700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb240cb050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdb14011da0 tx=0x7fdb140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.927+0000 7fdb0affd700 1 -- 172.21.15.31:0/2819589080 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14011ea0 con 0x7fdb0c005210 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.927+0000 7fdb0affd700 1 -- 172.21.15.31:0/2819589080 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1400fd60 con 0x7fdb0c005210 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.927+0000 7fdb0affd700 1 -- 172.21.15.31:0/2819589080 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1401d6d0 con 0x7fdb0c005210 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.927+0000 7fdb293ff700 1 -- 172.21.15.31:0/2819589080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb240ad510 con 0x7fdb0c005210 2023-10-06T05:17:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.927+0000 7fdb293ff700 1 -- 172.21.15.31:0/2819589080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb240c6510 con 0x7fdb0c005210 2023-10-06T05:17:31.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.928+0000 7fdb293ff700 1 -- 172.21.15.31:0/2819589080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2404c380 con 0x7fdb0c005210 2023-10-06T05:17:31.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.929+0000 7fdb0affd700 1 -- 172.21.15.31:0/2819589080 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb1401da80 con 0x7fdb0c005210 2023-10-06T05:17:31.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.931+0000 7fdb0affd700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb10066f30 0x7fdb100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:31.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.931+0000 7fdb0affd700 1 -- 172.21.15.31:0/2819589080 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdb14018070 con 0x7fdb0c005210 2023-10-06T05:17:31.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.931+0000 7fdb227fc700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb10066f30 0x7fdb100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:31.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.931+0000 7fdb227fc700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb10066f30 0x7fdb100693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdb18000e50 tx=0x7fdb18001d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:31.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:31.935+0000 7fdb0affd700 1 -- 172.21.15.31:0/2819589080 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb140526f0 con 0x7fdb0c005210 2023-10-06T05:17:32.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.140+0000 7fdb293ff700 1 -- 172.21.15.31:0/2819589080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdb24071060 con 0x7fdb0c005210 2023-10-06T05:17:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.140+0000 7fdb0affd700 1 -- 172.21.15.31:0/2819589080 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdb14055d10 con 0x7fdb0c005210 2023-10-06T05:17:32.143 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:17:32.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 -- 172.21.15.31:0/2819589080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb10066f30 msgr2=0x7fdb100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb10066f30 0x7fdb100693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdb18000e50 tx=0x7fdb18001d20 comp rx=0 tx=0).stop 2023-10-06T05:17:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 -- 172.21.15.31:0/2819589080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 msgr2=0x7fdb240cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb240cb050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdb14011da0 tx=0x7fdb140091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 -- 172.21.15.31:0/2819589080 shutdown_connections 2023-10-06T05:17:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb240ac8c0 0x7fdb240acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdb10066f30 0x7fdb100693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb0c005210 0x7fdb240cb050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 --2- 172.21.15.31:0/2819589080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdb24156500 0x7fdb24156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 -- 172.21.15.31:0/2819589080 >> 172.21.15.31:0/2819589080 conn(0x7fdb240ce290 msgr2=0x7fdb24069890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:32.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 -- 172.21.15.31:0/2819589080 shutdown_connections 2023-10-06T05:17:32.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.143+0000 7fdb293ff700 1 -- 172.21.15.31:0/2819589080 wait complete. 2023-10-06T05:17:32.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.359+0000 7f5265bff700 1 Processor -- start 2023-10-06T05:17:32.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.359+0000 7f5265bff700 1 -- start start 2023-10-06T05:17:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.361+0000 7f5265bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f52600c7190 0x7f52600c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.361+0000 7f5265bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52600aef00 0x7f52600af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.361+0000 7f5265bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5260068250 con 0x7f52600afcf0 2023-10-06T05:17:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.361+0000 7f5265bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f526006ac80 con 0x7f52600c7190 2023-10-06T05:17:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.361+0000 7f5265bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5260067f20 con 0x7f52600aef00 2023-10-06T05:17:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.361+0000 7f525ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52600afcf0 0x7f52600686e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46200/0 (socket says 172.21.15.31:46200) 2023-10-06T05:17:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.361+0000 7f525ffff700 1 -- 172.21.15.31:0/258600603 learned_addr learned my addr 172.21.15.31:0/258600603 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.361+0000 7f525effd700 1 --2- 172.21.15.31:0/258600603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52600aef00 0x7f52600af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.361+0000 7f525f7fe700 1 --2- 172.21.15.31:0/258600603 >> v2:172.21.15.62:3300/0 conn(0x7f52600c7190 0x7f52600c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.361+0000 7f525e7fc700 1 -- 172.21.15.31:0/258600603 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069922791 0 0) 0x7f5260068250 con 0x7f52600afcf0 2023-10-06T05:17:32.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.362+0000 7f525e7fc700 1 -- 172.21.15.31:0/258600603 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5248003570 con 0x7f52600afcf0 2023-10-06T05:17:32.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.362+0000 7f525f7fe700 1 -- 172.21.15.31:0/258600603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52600aef00 msgr2=0x7f52600af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:32.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.362+0000 7f525f7fe700 1 --2- 172.21.15.31:0/258600603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52600aef00 0x7f52600af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.362+0000 7f525f7fe700 1 -- 172.21.15.31:0/258600603 >> v1:172.21.15.31:6789/0 conn(0x7f52600afcf0 legacy=0x7f52600686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.362+0000 7f525f7fe700 1 -- 172.21.15.31:0/258600603 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526006aed0 con 0x7f52600c7190 2023-10-06T05:17:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.362+0000 7f525f7fe700 1 --2- 172.21.15.31:0/258600603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52600c7190 0x7f52600c9d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52500098e0 tx=0x7f52500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d3b95c1f0dbefb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.362+0000 7f525e7fc700 1 -- 172.21.15.31:0/258600603 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250011630 con 0x7f52600c7190 2023-10-06T05:17:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.362+0000 7f525e7fc700 1 -- 172.21.15.31:0/258600603 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5250011790 con 0x7f52600c7190 2023-10-06T05:17:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.362+0000 7f525e7fc700 1 -- 172.21.15.31:0/258600603 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250011b30 con 0x7f52600c7190 2023-10-06T05:17:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.363+0000 7f5265bff700 1 -- 172.21.15.31:0/258600603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52600c7190 msgr2=0x7f52600c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.363+0000 7f5265bff700 1 --2- 172.21.15.31:0/258600603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52600c7190 0x7f52600c9d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52500098e0 tx=0x7f52500091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.363+0000 7f5265bff700 1 -- 172.21.15.31:0/258600603 shutdown_connections 2023-10-06T05:17:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.363+0000 7f5265bff700 1 --2- 172.21.15.31:0/258600603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52600c7190 0x7f52600c9d40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.363+0000 7f5265bff700 1 --2- 172.21.15.31:0/258600603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52600aef00 0x7f52600af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.363+0000 7f5265bff700 1 -- 172.21.15.31:0/258600603 >> 172.21.15.31:0/258600603 conn(0x7f52600cd3d0 msgr2=0x7f52600c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.363+0000 7f5265bff700 1 -- 172.21.15.31:0/258600603 shutdown_connections 2023-10-06T05:17:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.363+0000 7f5265bff700 1 -- 172.21.15.31:0/258600603 wait complete. 2023-10-06T05:17:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.364+0000 7f5265bff700 1 Processor -- start 2023-10-06T05:17:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.365+0000 7f5265bff700 1 -- start start 2023-10-06T05:17:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f5265bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52600aef00 0x7f5260159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f5265bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52600afcf0 0x7f526015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f5265bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52600c7190 0x7f5260157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f5265bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526015bbc0 con 0x7f52600c7190 2023-10-06T05:17:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f5265bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5260169d00 con 0x7f52600afcf0 2023-10-06T05:17:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f5265bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526015ae40 con 0x7f52600aef00 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f525ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52600c7190 0x7f5260157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f525ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52600c7190 0x7f5260157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53854/0 (socket says 172.21.15.31:53854) 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f525ffff700 1 -- 172.21.15.31:0/845231642 learned_addr learned my addr 172.21.15.31:0/845231642 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f525f7fe700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52600aef00 0x7f5260159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f525effd700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52600afcf0 0x7f526015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f525ffff700 1 -- 172.21.15.31:0/845231642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52600aef00 msgr2=0x7f5260159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f525ffff700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52600aef00 0x7f5260159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f525ffff700 1 -- 172.21.15.31:0/845231642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52600afcf0 msgr2=0x7f526015bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f525ffff700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52600afcf0 0x7f526015bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.366+0000 7f525ffff700 1 -- 172.21.15.31:0/845231642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5250009420 con 0x7f52600c7190 2023-10-06T05:17:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.367+0000 7f525ffff700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52600c7190 0x7f5260157d30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5258001460 tx=0x7f5258001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.367+0000 7f52477fe700 1 -- 172.21.15.31:0/845231642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525800a890 con 0x7f52600c7190 2023-10-06T05:17:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.367+0000 7f52477fe700 1 -- 172.21.15.31:0/845231642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525800a9f0 con 0x7f52600c7190 2023-10-06T05:17:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.367+0000 7f5265bff700 1 -- 172.21.15.31:0/845231642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526015a5a0 con 0x7f52600c7190 2023-10-06T05:17:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.367+0000 7f52477fe700 1 -- 172.21.15.31:0/845231642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525800ad70 con 0x7f52600c7190 2023-10-06T05:17:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.367+0000 7f5265bff700 1 -- 172.21.15.31:0/845231642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526015ab60 con 0x7f52600c7190 2023-10-06T05:17:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.368+0000 7f5265bff700 1 -- 172.21.15.31:0/845231642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526004c380 con 0x7f52600c7190 2023-10-06T05:17:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.369+0000 7f52477fe700 1 -- 172.21.15.31:0/845231642 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5258007300 con 0x7f52600c7190 2023-10-06T05:17:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.370+0000 7f52477fe700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f524c066f30 0x7f524c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.371+0000 7f52477fe700 1 -- 172.21.15.31:0/845231642 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5258084a50 con 0x7f52600c7190 2023-10-06T05:17:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.371+0000 7f525f7fe700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f524c066f30 0x7f524c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.371+0000 7f525f7fe700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f524c066f30 0x7f524c0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f52500091e0 tx=0x7f5250017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.375+0000 7f52477fe700 1 -- 172.21.15.31:0/845231642 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5258051b40 con 0x7f52600c7190 2023-10-06T05:17:32.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.577+0000 7f5265bff700 1 -- 172.21.15.31:0/845231642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f526015c4c0 con 0x7f52600c7190 2023-10-06T05:17:32.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.578+0000 7f52477fe700 1 -- 172.21.15.31:0/845231642 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5258055160 con 0x7f52600c7190 2023-10-06T05:17:32.581 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:17:32.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 -- 172.21.15.31:0/845231642 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f524c066f30 msgr2=0x7f524c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f524c066f30 0x7f524c0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f52500091e0 tx=0x7f5250017000 comp rx=0 tx=0).stop 2023-10-06T05:17:32.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 -- 172.21.15.31:0/845231642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52600c7190 msgr2=0x7f5260157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52600c7190 0x7f5260157d30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5258001460 tx=0x7f5258001ae0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 -- 172.21.15.31:0/845231642 shutdown_connections 2023-10-06T05:17:32.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f524c066f30 0x7f524c0693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52600c7190 0x7f5260157d30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52600aef00 0x7f5260159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 --2- 172.21.15.31:0/845231642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52600afcf0 0x7f526015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 -- 172.21.15.31:0/845231642 >> 172.21.15.31:0/845231642 conn(0x7f52600cd3d0 msgr2=0x7f52600b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:32.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 -- 172.21.15.31:0/845231642 shutdown_connections 2023-10-06T05:17:32.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.580+0000 7f5265bff700 1 -- 172.21.15.31:0/845231642 wait complete. 2023-10-06T05:17:32.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:17:32.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.797+0000 7fdd1a5ff700 1 Processor -- start 2023-10-06T05:17:32.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.797+0000 7fdd1a5ff700 1 -- start start 2023-10-06T05:17:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.797+0000 7fdd1a5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd140c8f40 0x7fdd14068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.797+0000 7fdd1a5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd1406a300 0x7fdd140adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.797+0000 7fdd1a5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd14068da0 con 0x7fdd1415ca40 2023-10-06T05:17:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.797+0000 7fdd1a5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd140cabc0 con 0x7fdd1406a300 2023-10-06T05:17:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.797+0000 7fdd1a5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd14068bb0 con 0x7fdd140c8f40 2023-10-06T05:17:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd137fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd140c8f40 0x7fdd14068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd18b9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd1406a300 0x7fdd140adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd18b9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd1406a300 0x7fdd140adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53204/0 (socket says 172.21.15.31:53204) 2023-10-06T05:17:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd137fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd140c8f40 0x7fdd14068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:53960/0 (socket says 172.21.15.31:53960) 2023-10-06T05:17:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd18b9c700 1 -- 172.21.15.31:0/606159901 learned_addr learned my addr 172.21.15.31:0/606159901 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:32.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd13fff700 1 --1- 172.21.15.31:0/606159901 >> v1:172.21.15.62:6789/0 conn(0x7fdd1415ca40 0x7fdd140c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:48944/0 (socket says 172.21.15.31:48944) 2023-10-06T05:17:32.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd18b9c700 1 -- 172.21.15.31:0/606159901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd140c8f40 msgr2=0x7fdd14068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:32.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd18b9c700 1 --2- 172.21.15.31:0/606159901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd140c8f40 0x7fdd14068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd18b9c700 1 -- 172.21.15.31:0/606159901 >> v1:172.21.15.62:6789/0 conn(0x7fdd1415ca40 legacy=0x7fdd140c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd18b9c700 1 -- 172.21.15.31:0/606159901 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14069080 con 0x7fdd1406a300 2023-10-06T05:17:32.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.798+0000 7fdd18b9c700 1 --2- 172.21.15.31:0/606159901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1406a300 0x7fdd140adde0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009f20 tx=0x7fdd0c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9b2114b4c556dfe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.799+0000 7fdd12ffd700 1 -- 172.21.15.31:0/606159901 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c00e070 con 0x7fdd1406a300 2023-10-06T05:17:32.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.799+0000 7fdd12ffd700 1 -- 172.21.15.31:0/606159901 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c00b9d0 con 0x7fdd1406a300 2023-10-06T05:17:32.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.799+0000 7fdd12ffd700 1 -- 172.21.15.31:0/606159901 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0092e0 con 0x7fdd1406a300 2023-10-06T05:17:32.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.799+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/606159901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1406a300 msgr2=0x7fdd140adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.799+0000 7fdd1a5ff700 1 --2- 172.21.15.31:0/606159901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1406a300 0x7fdd140adde0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009f20 tx=0x7fdd0c009840 comp rx=0 tx=0).stop 2023-10-06T05:17:32.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.799+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/606159901 shutdown_connections 2023-10-06T05:17:32.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.799+0000 7fdd1a5ff700 1 --2- 172.21.15.31:0/606159901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1406a300 0x7fdd140adde0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.799+0000 7fdd1a5ff700 1 --2- 172.21.15.31:0/606159901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd140c8f40 0x7fdd14068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.799+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/606159901 >> 172.21.15.31:0/606159901 conn(0x7fdd140ce290 msgr2=0x7fdd140ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:32.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.800+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/606159901 shutdown_connections 2023-10-06T05:17:32.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.800+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/606159901 wait complete. 2023-10-06T05:17:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.802+0000 7fdd1a5ff700 1 Processor -- start 2023-10-06T05:17:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.802+0000 7fdd1a5ff700 1 -- start start 2023-10-06T05:17:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd1a5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1406a300 0x7fdd14158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd1a5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140c8f40 0x7fdd140b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd1a5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd1415ca40 0x7fdd140afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd1a5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd141591e0 con 0x7fdd140c8f40 2023-10-06T05:17:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd1a5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140ad760 con 0x7fdd1406a300 2023-10-06T05:17:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd1a5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1416d2d0 con 0x7fdd1415ca40 2023-10-06T05:17:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140c8f40 0x7fdd140b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140c8f40 0x7fdd140b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53858/0 (socket says 172.21.15.31:53858) 2023-10-06T05:17:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd137fe700 1 -- 172.21.15.31:0/3786405612 learned_addr learned my addr 172.21.15.31:0/3786405612 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd18b9c700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd1415ca40 0x7fdd140afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd13fff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1406a300 0x7fdd14158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd137fe700 1 -- 172.21.15.31:0/3786405612 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd1415ca40 msgr2=0x7fdd140afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd137fe700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd1415ca40 0x7fdd140afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd137fe700 1 -- 172.21.15.31:0/3786405612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1406a300 msgr2=0x7fdd14158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd137fe700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1406a300 0x7fdd14158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd137fe700 1 -- 172.21.15.31:0/3786405612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c009ae0 con 0x7fdd140c8f40 2023-10-06T05:17:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd18b9c700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd1415ca40 0x7fdd140afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd13fff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1406a300 0x7fdd14158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd137fe700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140c8f40 0x7fdd140b2f30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800ec80 tx=0x7fdd080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdcfbfff700 1 -- 172.21.15.31:0/3786405612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0800c9d0 con 0x7fdd140c8f40 2023-10-06T05:17:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdcfbfff700 1 -- 172.21.15.31:0/3786405612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0800cb30 con 0x7fdd140c8f40 2023-10-06T05:17:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/3786405612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1416d5b0 con 0x7fdd140c8f40 2023-10-06T05:17:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.803+0000 7fdcfbfff700 1 -- 172.21.15.31:0/3786405612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08013470 con 0x7fdd140c8f40 2023-10-06T05:17:32.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.804+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/3786405612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1416db70 con 0x7fdd140c8f40 2023-10-06T05:17:32.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.805+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/3786405612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1404c380 con 0x7fdd140c8f40 2023-10-06T05:17:32.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.805+0000 7fdcfbfff700 1 -- 172.21.15.31:0/3786405612 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd08013820 con 0x7fdd140c8f40 2023-10-06T05:17:32.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.806+0000 7fdcfbfff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd00066f30 0x7fdd000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:32.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.806+0000 7fdcfbfff700 1 -- 172.21.15.31:0/3786405612 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd08084b40 con 0x7fdd140c8f40 2023-10-06T05:17:32.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.807+0000 7fdd13fff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd00066f30 0x7fdd000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:32.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.807+0000 7fdd13fff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd00066f30 0x7fdd000693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdd1406fa50 tx=0x7fdd04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:32.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:32.810+0000 7fdcfbfff700 1 -- 172.21.15.31:0/3786405612 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd08051bb0 con 0x7fdd140c8f40 2023-10-06T05:17:33.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.009+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/3786405612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd140ca1a0 con 0x7fdd140c8f40 2023-10-06T05:17:33.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.010+0000 7fdcfbfff700 1 -- 172.21.15.31:0/3786405612 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd080551d0 con 0x7fdd140c8f40 2023-10-06T05:17:33.013 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:17:33.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/3786405612 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd00066f30 msgr2=0x7fdd000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd00066f30 0x7fdd000693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdd1406fa50 tx=0x7fdd04009040 comp rx=0 tx=0).stop 2023-10-06T05:17:33.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/3786405612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140c8f40 msgr2=0x7fdd140b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140c8f40 0x7fdd140b2f30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800ec80 tx=0x7fdd080091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/3786405612 shutdown_connections 2023-10-06T05:17:33.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd1415ca40 0x7fdd140afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1406a300 0x7fdd14158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd00066f30 0x7fdd000693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 --2- 172.21.15.31:0/3786405612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd140c8f40 0x7fdd140b2f30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/3786405612 >> 172.21.15.31:0/3786405612 conn(0x7fdd140ce290 msgr2=0x7fdd1406cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:33.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/3786405612 shutdown_connections 2023-10-06T05:17:33.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.013+0000 7fdd1a5ff700 1 -- 172.21.15.31:0/3786405612 wait complete. 2023-10-06T05:17:33.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:17:33.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.229+0000 7f2212dff700 1 Processor -- start 2023-10-06T05:17:33.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.229+0000 7f2212dff700 1 -- start start 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.230+0000 7f2212dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f220c159540 0x7f220c068930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.230+0000 7f2212dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f220c06a2a0 0x7f220c0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.230+0000 7f2212dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220c0adb00 con 0x7f220c159910 2023-10-06T05:17:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.230+0000 7f2212dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220c068f10 con 0x7f220c159540 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.230+0000 7f2212dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0691f0 con 0x7f220c06a2a0 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f2210b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f220c159540 0x7f220c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f221139c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f220c159910 0x7f220c06cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46208/0 (socket says 172.21.15.31:46208) 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f2210b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f220c159540 0x7f220c068930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53860/0 (socket says 172.21.15.31:53860) 2023-10-06T05:17:33.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f221139c700 1 -- 172.21.15.31:0/1982248451 learned_addr learned my addr 172.21.15.31:0/1982248451 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:33.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f220bfff700 1 --2- 172.21.15.31:0/1982248451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f220c06a2a0 0x7f220c0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f2210b9b700 1 -- 172.21.15.31:0/1982248451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f220c06a2a0 msgr2=0x7f220c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f2210b9b700 1 --2- 172.21.15.31:0/1982248451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f220c06a2a0 0x7f220c0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f2210b9b700 1 -- 172.21.15.31:0/1982248451 >> v1:172.21.15.31:6789/0 conn(0x7f220c159910 legacy=0x7f220c06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f2210b9b700 1 -- 172.21.15.31:0/1982248451 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c0694d0 con 0x7f220c159540 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f220b7fe700 1 -- 172.21.15.31:0/1982248451 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449929317 0 0) 0x7f220c0adb00 con 0x7f220c159910 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.231+0000 7f2210b9b700 1 --2- 172.21.15.31:0/1982248451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c159540 0x7f220c068930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22000098e0 tx=0x7f22000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f191a7c35ecea9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f220b7fe700 1 -- 172.21.15.31:0/1982248451 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200009e80 con 0x7f220c159540 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f220b7fe700 1 -- 172.21.15.31:0/1982248451 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220000b860 con 0x7f220c159540 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f220b7fe700 1 -- 172.21.15.31:0/1982248451 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220000bbe0 con 0x7f220c159540 2023-10-06T05:17:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f2212dff700 1 -- 172.21.15.31:0/1982248451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c159540 msgr2=0x7f220c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f2212dff700 1 --2- 172.21.15.31:0/1982248451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c159540 0x7f220c068930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22000098e0 tx=0x7f22000091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f2212dff700 1 -- 172.21.15.31:0/1982248451 shutdown_connections 2023-10-06T05:17:33.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f2212dff700 1 --2- 172.21.15.31:0/1982248451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c159540 0x7f220c068930 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f2212dff700 1 --2- 172.21.15.31:0/1982248451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f220c06a2a0 0x7f220c0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f2212dff700 1 -- 172.21.15.31:0/1982248451 >> 172.21.15.31:0/1982248451 conn(0x7f220c0ce290 msgr2=0x7f220c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:33.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f2212dff700 1 -- 172.21.15.31:0/1982248451 shutdown_connections 2023-10-06T05:17:33.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.232+0000 7f2212dff700 1 -- 172.21.15.31:0/1982248451 wait complete. 2023-10-06T05:17:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.234+0000 7f2212dff700 1 Processor -- start 2023-10-06T05:17:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.234+0000 7f2212dff700 1 -- start start 2023-10-06T05:17:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f2212dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c06a2a0 0x7f220c152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f2212dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f220c159540 0x7f220c0c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f2212dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f220c159910 0x7f220c0c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f2212dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c158670 con 0x7f220c06a2a0 2023-10-06T05:17:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f2212dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0c91b0 con 0x7f220c159540 2023-10-06T05:17:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f2212dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c158390 con 0x7f220c159910 2023-10-06T05:17:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f2210b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c06a2a0 0x7f220c152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f2210b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c06a2a0 0x7f220c152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53870/0 (socket says 172.21.15.31:53870) 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f2210b9b700 1 -- 172.21.15.31:0/2575312396 learned_addr learned my addr 172.21.15.31:0/2575312396 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f220bfff700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f220c159540 0x7f220c0c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.235+0000 7f221139c700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f220c159910 0x7f220c0c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2210b9b700 1 -- 172.21.15.31:0/2575312396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f220c159910 msgr2=0x7f220c0c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2210b9b700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f220c159910 0x7f220c0c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2210b9b700 1 -- 172.21.15.31:0/2575312396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f220c159540 msgr2=0x7f220c0c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2210b9b700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f220c159540 0x7f220c0c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2210b9b700 1 -- 172.21.15.31:0/2575312396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2200009420 con 0x7f220c06a2a0 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2210b9b700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c06a2a0 0x7f220c152c40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f220000b3c0 tx=0x7f2200017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2208ff9700 1 -- 172.21.15.31:0/2575312396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220000bbe0 con 0x7f220c06a2a0 2023-10-06T05:17:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2208ff9700 1 -- 172.21.15.31:0/2575312396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200000c90 con 0x7f220c06a2a0 2023-10-06T05:17:33.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2208ff9700 1 -- 172.21.15.31:0/2575312396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200011690 con 0x7f220c06a2a0 2023-10-06T05:17:33.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2212dff700 1 -- 172.21.15.31:0/2575312396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220c155450 con 0x7f220c06a2a0 2023-10-06T05:17:33.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.236+0000 7f2212dff700 1 -- 172.21.15.31:0/2575312396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c155a10 con 0x7f220c06a2a0 2023-10-06T05:17:33.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.237+0000 7f2212dff700 1 -- 172.21.15.31:0/2575312396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220c1552b0 con 0x7f220c06a2a0 2023-10-06T05:17:33.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.238+0000 7f2208ff9700 1 -- 172.21.15.31:0/2575312396 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2200011a80 con 0x7f220c06a2a0 2023-10-06T05:17:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.239+0000 7f2208ff9700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f21f0066f30 0x7f21f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.239+0000 7f2208ff9700 1 -- 172.21.15.31:0/2575312396 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2200085fe0 con 0x7f220c06a2a0 2023-10-06T05:17:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.239+0000 7f220bfff700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f21f0066f30 0x7f21f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.240+0000 7f220bfff700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f21f0066f30 0x7f21f00693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f220c0b0870 tx=0x7f21f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:33.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.243+0000 7f2208ff9700 1 -- 172.21.15.31:0/2575312396 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f220c1552b0 con 0x7f220c06a2a0 2023-10-06T05:17:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.450+0000 7f2212dff700 1 -- 172.21.15.31:0/2575312396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f220c1552b0 con 0x7f220c06a2a0 2023-10-06T05:17:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.450+0000 7f2208ff9700 1 -- 172.21.15.31:0/2575312396 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f220c1552b0 con 0x7f220c06a2a0 2023-10-06T05:17:33.452 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:17:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.452+0000 7f2212dff700 1 -- 172.21.15.31:0/2575312396 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f21f0066f30 msgr2=0x7f21f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.452+0000 7f2212dff700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f21f0066f30 0x7f21f00693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f220c0b0870 tx=0x7f21f8008040 comp rx=0 tx=0).stop 2023-10-06T05:17:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.452+0000 7f2212dff700 1 -- 172.21.15.31:0/2575312396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c06a2a0 msgr2=0x7f220c152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.452+0000 7f2212dff700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c06a2a0 0x7f220c152c40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f220000b3c0 tx=0x7f2200017000 comp rx=0 tx=0).stop 2023-10-06T05:17:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.453+0000 7f2212dff700 1 -- 172.21.15.31:0/2575312396 shutdown_connections 2023-10-06T05:17:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.453+0000 7f2212dff700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f220c159910 0x7f220c0c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.453+0000 7f2212dff700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f21f0066f30 0x7f21f00693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.453+0000 7f2212dff700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f220c06a2a0 0x7f220c152c40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.453+0000 7f2212dff700 1 --2- 172.21.15.31:0/2575312396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f220c159540 0x7f220c0c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.453+0000 7f2212dff700 1 -- 172.21.15.31:0/2575312396 >> 172.21.15.31:0/2575312396 conn(0x7f220c0ce290 msgr2=0x7f220c0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.453+0000 7f2212dff700 1 -- 172.21.15.31:0/2575312396 shutdown_connections 2023-10-06T05:17:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.453+0000 7f2212dff700 1 -- 172.21.15.31:0/2575312396 wait complete. 2023-10-06T05:17:33.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:33.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.666+0000 7f30227ff700 1 Processor -- start 2023-10-06T05:17:33.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.666+0000 7f30227ff700 1 -- start start 2023-10-06T05:17:33.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f30227ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c0cd510 0x7f301c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f30227ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301c0c3f40 con 0x7f301c15d2d0 2023-10-06T05:17:33.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f30227ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301c0cb000 con 0x7f301c159500 2023-10-06T05:17:33.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f30227ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0b2860 con 0x7f301c0cd510 2023-10-06T05:17:33.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f3020d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f301c15d2d0 0x7f301c0aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46218/0 (socket says 172.21.15.31:46218) 2023-10-06T05:17:33.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f3020d9c700 1 -- 172.21.15.31:0/1771630401 learned_addr learned my addr 172.21.15.31:0/1771630401 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:33.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f301b7fe700 1 --2- 172.21.15.31:0/1771630401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c0cd510 0x7f301c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f301affd700 1 -- 172.21.15.31:0/1771630401 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145960022 0 0) 0x7f301c0c3f40 con 0x7f301c15d2d0 2023-10-06T05:17:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f301affd700 1 -- 172.21.15.31:0/1771630401 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3004003570 con 0x7f301c15d2d0 2023-10-06T05:17:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f301affd700 1 -- 172.21.15.31:0/1771630401 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060153946 0 0) 0x7f301c0cb000 con 0x7f301c159500 2023-10-06T05:17:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.668+0000 7f301affd700 1 -- 172.21.15.31:0/1771630401 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f301c0c3f40 con 0x7f301c159500 2023-10-06T05:17:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.669+0000 7f301b7fe700 1 -- 172.21.15.31:0/1771630401 >> v1:172.21.15.62:6789/0 conn(0x7f301c159500 legacy=0x7f301c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.669+0000 7f301b7fe700 1 -- 172.21.15.31:0/1771630401 >> v1:172.21.15.31:6789/0 conn(0x7f301c15d2d0 legacy=0x7f301c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.669+0000 7f301b7fe700 1 -- 172.21.15.31:0/1771630401 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c15d7d0 con 0x7f301c0cd510 2023-10-06T05:17:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.669+0000 7f301b7fe700 1 --2- 172.21.15.31:0/1771630401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c0cd510 0x7f301c0b0ab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3010009900 tx=0x7f3010009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=151726a0f175e4d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:33.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.669+0000 7f301affd700 1 -- 172.21.15.31:0/1771630401 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010005c10 con 0x7f301c0cd510 2023-10-06T05:17:33.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.669+0000 7f301affd700 1 -- 172.21.15.31:0/1771630401 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3010005d70 con 0x7f301c0cd510 2023-10-06T05:17:33.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.669+0000 7f301affd700 1 -- 172.21.15.31:0/1771630401 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010006260 con 0x7f301c0cd510 2023-10-06T05:17:33.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.670+0000 7f30227ff700 1 -- 172.21.15.31:0/1771630401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c0cd510 msgr2=0x7f301c0b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.670+0000 7f30227ff700 1 --2- 172.21.15.31:0/1771630401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c0cd510 0x7f301c0b0ab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3010009900 tx=0x7f3010009120 comp rx=0 tx=0).stop 2023-10-06T05:17:33.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.670+0000 7f30227ff700 1 -- 172.21.15.31:0/1771630401 shutdown_connections 2023-10-06T05:17:33.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.670+0000 7f30227ff700 1 --2- 172.21.15.31:0/1771630401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c0cd510 0x7f301c0b0ab0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.670+0000 7f30227ff700 1 -- 172.21.15.31:0/1771630401 >> 172.21.15.31:0/1771630401 conn(0x7f301c06ec90 msgr2=0x7f301c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:33.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.670+0000 7f30227ff700 1 -- 172.21.15.31:0/1771630401 shutdown_connections 2023-10-06T05:17:33.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.670+0000 7f30227ff700 1 -- 172.21.15.31:0/1771630401 wait complete. 2023-10-06T05:17:33.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.671+0000 7f30227ff700 1 Processor -- start 2023-10-06T05:17:33.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.671+0000 7f30227ff700 1 -- start start 2023-10-06T05:17:33.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.672+0000 7f30227ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f301c0cd510 0x7f301c0ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.672+0000 7f30227ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c159500 0x7f301c0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.672+0000 7f30227ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f301c15d2d0 0x7f301c0c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.672+0000 7f30227ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c16e310 con 0x7f301c0cd510 2023-10-06T05:17:33.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.672+0000 7f30227ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0aea40 con 0x7f301c15d2d0 2023-10-06T05:17:33.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.672+0000 7f30227ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301c16d700 con 0x7f301c159500 2023-10-06T05:17:33.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f301c0cd510 0x7f301c0ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f301c0cd510 0x7f301c0ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53876/0 (socket says 172.21.15.31:53876) 2023-10-06T05:17:33.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301bfff700 1 -- 172.21.15.31:0/3535751024 learned_addr learned my addr 172.21.15.31:0/3535751024 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:33.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301b7fe700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c159500 0x7f301c0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f3020d9c700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f301c15d2d0 0x7f301c0c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301bfff700 1 -- 172.21.15.31:0/3535751024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c159500 msgr2=0x7f301c0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301bfff700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c159500 0x7f301c0aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301bfff700 1 -- 172.21.15.31:0/3535751024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f301c15d2d0 msgr2=0x7f301c0c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301bfff700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f301c15d2d0 0x7f301c0c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301bfff700 1 -- 172.21.15.31:0/3535751024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30100093c0 con 0x7f301c0cd510 2023-10-06T05:17:33.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301b7fe700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c159500 0x7f301c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:33.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f3020d9c700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f301c15d2d0 0x7f301c0c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:33.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.673+0000 7f301bfff700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f301c0cd510 0x7f301c0ae500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f300c00c790 tx=0x7f300c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:33.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.674+0000 7f3003fff700 1 -- 172.21.15.31:0/3535751024 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c010510 con 0x7f301c0cd510 2023-10-06T05:17:33.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.674+0000 7f3003fff700 1 -- 172.21.15.31:0/3535751024 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c010670 con 0x7f301c0cd510 2023-10-06T05:17:33.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.674+0000 7f30227ff700 1 -- 172.21.15.31:0/3535751024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301c16d9e0 con 0x7f301c0cd510 2023-10-06T05:17:33.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.674+0000 7f3003fff700 1 -- 172.21.15.31:0/3535751024 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c010a10 con 0x7f301c0cd510 2023-10-06T05:17:33.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.674+0000 7f30227ff700 1 -- 172.21.15.31:0/3535751024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301c16dfa0 con 0x7f301c0cd510 2023-10-06T05:17:33.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.675+0000 7f30227ff700 1 -- 172.21.15.31:0/3535751024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301c16e5f0 con 0x7f301c0cd510 2023-10-06T05:17:33.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.676+0000 7f3003fff700 1 -- 172.21.15.31:0/3535751024 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f300c01b400 con 0x7f301c0cd510 2023-10-06T05:17:33.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.677+0000 7f3003fff700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3008066f30 0x7f30080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:33.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.677+0000 7f3003fff700 1 -- 172.21.15.31:0/3535751024 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f300c084500 con 0x7f301c0cd510 2023-10-06T05:17:33.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.677+0000 7f301b7fe700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3008066f30 0x7f30080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:33.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.677+0000 7f301b7fe700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3008066f30 0x7f30080693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3010009390 tx=0x7f3010016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:33.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.681+0000 7f3003fff700 1 -- 172.21.15.31:0/3535751024 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f300c051570 con 0x7f301c0cd510 2023-10-06T05:17:33.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.885+0000 7f30227ff700 1 -- 172.21.15.31:0/3535751024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f301c04c380 con 0x7f301c0cd510 2023-10-06T05:17:33.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.886+0000 7f3003fff700 1 -- 172.21.15.31:0/3535751024 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f300c054b90 con 0x7f301c0cd510 2023-10-06T05:17:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:33.888 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 -- 172.21.15.31:0/3535751024 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3008066f30 msgr2=0x7f30080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3008066f30 0x7f30080693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3010009390 tx=0x7f3010016000 comp rx=0 tx=0).stop 2023-10-06T05:17:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 -- 172.21.15.31:0/3535751024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f301c0cd510 msgr2=0x7f301c0ae500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f301c0cd510 0x7f301c0ae500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f300c00c790 tx=0x7f300c007e10 comp rx=0 tx=0).stop 2023-10-06T05:17:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 -- 172.21.15.31:0/3535751024 shutdown_connections 2023-10-06T05:17:33.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f301c15d2d0 0x7f301c0c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3008066f30 0x7f30080693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f301c0cd510 0x7f301c0ae500 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 --2- 172.21.15.31:0/3535751024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f301c159500 0x7f301c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:33.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 -- 172.21.15.31:0/3535751024 >> 172.21.15.31:0/3535751024 conn(0x7f301c06ec90 msgr2=0x7f301c0cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:33.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 -- 172.21.15.31:0/3535751024 shutdown_connections 2023-10-06T05:17:33.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:33.888+0000 7f30227ff700 1 -- 172.21.15.31:0/3535751024 wait complete. 2023-10-06T05:17:33.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:16:59.507957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:17:00.905826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:17:33.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.106+0000 7f58379ff700 1 Processor -- start 2023-10-06T05:17:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.106+0000 7f58379ff700 1 -- start start 2023-10-06T05:17:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.107+0000 7f58379ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f583015ca40 0x7f5830068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.107+0000 7f58379ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f583006a300 0x7f58300c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.107+0000 7f58379ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5830068da0 con 0x7f58300c8d70 2023-10-06T05:17:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.107+0000 7f58379ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58300cabc0 con 0x7f583015ca40 2023-10-06T05:17:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.107+0000 7f58379ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5830068bb0 con 0x7f583006a300 2023-10-06T05:17:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.108+0000 7f583579b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f583015ca40 0x7f5830068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.108+0000 7f583579b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f583015ca40 0x7f5830068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53890/0 (socket says 172.21.15.31:53890) 2023-10-06T05:17:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.108+0000 7f583579b700 1 -- 172.21.15.31:0/3427238288 learned_addr learned my addr 172.21.15.31:0/3427238288 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.108+0000 7f5834f9a700 1 --2- 172.21.15.31:0/3427238288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f583006a300 0x7f58300c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.108+0000 7f583579b700 1 -- 172.21.15.31:0/3427238288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f583006a300 msgr2=0x7f58300c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.108+0000 7f583579b700 1 --2- 172.21.15.31:0/3427238288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f583006a300 0x7f58300c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.108+0000 7f583579b700 1 -- 172.21.15.31:0/3427238288 >> v1:172.21.15.62:6789/0 conn(0x7f58300c8d70 legacy=0x7f583006cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.108+0000 7f583579b700 1 -- 172.21.15.31:0/3427238288 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5830069080 con 0x7f583015ca40 2023-10-06T05:17:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.108+0000 7f583579b700 1 --2- 172.21.15.31:0/3427238288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583015ca40 0x7f5830068670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f582c0099b0 tx=0x7f582c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe890223afbb3db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f5827fff700 1 -- 172.21.15.31:0/3427238288 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c00b860 con 0x7f583015ca40 2023-10-06T05:17:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f5827fff700 1 -- 172.21.15.31:0/3427238288 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c00b9c0 con 0x7f583015ca40 2023-10-06T05:17:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f5827fff700 1 -- 172.21.15.31:0/3427238288 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c00bd60 con 0x7f583015ca40 2023-10-06T05:17:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f58379ff700 1 -- 172.21.15.31:0/3427238288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583015ca40 msgr2=0x7f5830068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f58379ff700 1 --2- 172.21.15.31:0/3427238288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583015ca40 0x7f5830068670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f582c0099b0 tx=0x7f582c009250 comp rx=0 tx=0).stop 2023-10-06T05:17:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f58379ff700 1 -- 172.21.15.31:0/3427238288 shutdown_connections 2023-10-06T05:17:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f58379ff700 1 --2- 172.21.15.31:0/3427238288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583015ca40 0x7f5830068670 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f58379ff700 1 --2- 172.21.15.31:0/3427238288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f583006a300 0x7f58300c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f58379ff700 1 -- 172.21.15.31:0/3427238288 >> 172.21.15.31:0/3427238288 conn(0x7f58300ce290 msgr2=0x7f58300ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f58379ff700 1 -- 172.21.15.31:0/3427238288 shutdown_connections 2023-10-06T05:17:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.109+0000 7f58379ff700 1 -- 172.21.15.31:0/3427238288 wait complete. 2023-10-06T05:17:34.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.111+0000 7f58379ff700 1 Processor -- start 2023-10-06T05:17:34.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.111+0000 7f58379ff700 1 -- start start 2023-10-06T05:17:34.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f58379ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583006a300 0x7f58300ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f58379ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58300c8d70 0x7f58300ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f58379ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f583015ca40 0x7f58300aa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f58379ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58300ad0c0 con 0x7f583006a300 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f58379ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583016a700 con 0x7f583015ca40 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f58379ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58300aca70 con 0x7f58300c8d70 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f583579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583006a300 0x7f58300ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f583579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583006a300 0x7f58300ad550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53900/0 (socket says 172.21.15.31:53900) 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f583579b700 1 -- 172.21.15.31:0/113109426 learned_addr learned my addr 172.21.15.31:0/113109426 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f5835f9c700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f583015ca40 0x7f58300aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.112+0000 7f5834f9a700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58300c8d70 0x7f58300ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f583579b700 1 -- 172.21.15.31:0/113109426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58300c8d70 msgr2=0x7f58300ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f583579b700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58300c8d70 0x7f58300ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f583579b700 1 -- 172.21.15.31:0/113109426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f583015ca40 msgr2=0x7f58300aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f583579b700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f583015ca40 0x7f58300aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f583579b700 1 -- 172.21.15.31:0/113109426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0094f0 con 0x7f583006a300 2023-10-06T05:17:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f5835f9c700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f583015ca40 0x7f58300aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f5834f9a700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58300c8d70 0x7f58300ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f583579b700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583006a300 0x7f58300ad550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f583006e040 tx=0x7f582c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f58257fa700 1 -- 172.21.15.31:0/113109426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c024440 con 0x7f583006a300 2023-10-06T05:17:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f58257fa700 1 -- 172.21.15.31:0/113109426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c0245c0 con 0x7f583006a300 2023-10-06T05:17:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f58379ff700 1 -- 172.21.15.31:0/113109426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58300acd50 con 0x7f583006a300 2023-10-06T05:17:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f58257fa700 1 -- 172.21.15.31:0/113109426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c024980 con 0x7f583006a300 2023-10-06T05:17:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.113+0000 7f58379ff700 1 -- 172.21.15.31:0/113109426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58300cd010 con 0x7f583006a300 2023-10-06T05:17:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.114+0000 7f58379ff700 1 -- 172.21.15.31:0/113109426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583004c380 con 0x7f583006a300 2023-10-06T05:17:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.115+0000 7f58257fa700 1 -- 172.21.15.31:0/113109426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f582c01a420 con 0x7f583006a300 2023-10-06T05:17:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.116+0000 7f58257fa700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5814066f30 0x7f58140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.116+0000 7f58257fa700 1 -- 172.21.15.31:0/113109426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f582c0863d0 con 0x7f583006a300 2023-10-06T05:17:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.116+0000 7f5834f9a700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5814066f30 0x7f58140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.117+0000 7f5834f9a700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5814066f30 0x7f58140693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58300c8af0 tx=0x7f5820008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.120+0000 7f58257fa700 1 -- 172.21.15.31:0/113109426 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f582c053440 con 0x7f583006a300 2023-10-06T05:17:34.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.322+0000 7f58379ff700 1 -- 172.21.15.31:0/113109426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f58300ad3a0 con 0x7f583006a300 2023-10-06T05:17:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.323+0000 7f58257fa700 1 -- 172.21.15.31:0/113109426 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f582c056a60 con 0x7f583006a300 2023-10-06T05:17:34.325 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:17:34.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 -- 172.21.15.31:0/113109426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5814066f30 msgr2=0x7f58140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5814066f30 0x7f58140693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58300c8af0 tx=0x7f5820008040 comp rx=0 tx=0).stop 2023-10-06T05:17:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 -- 172.21.15.31:0/113109426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583006a300 msgr2=0x7f58300ad550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583006a300 0x7f58300ad550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f583006e040 tx=0x7f582c017000 comp rx=0 tx=0).stop 2023-10-06T05:17:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 -- 172.21.15.31:0/113109426 shutdown_connections 2023-10-06T05:17:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f583015ca40 0x7f58300aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5814066f30 0x7f58140693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f583006a300 0x7f58300ad550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 --2- 172.21.15.31:0/113109426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58300c8d70 0x7f58300ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 -- 172.21.15.31:0/113109426 >> 172.21.15.31:0/113109426 conn(0x7f58300ce290 msgr2=0x7f583015a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.325+0000 7f58379ff700 1 -- 172.21.15.31:0/113109426 shutdown_connections 2023-10-06T05:17:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.326+0000 7f58379ff700 1 -- 172.21.15.31:0/113109426 wait complete. 2023-10-06T05:17:34.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.544+0000 7fc9fadff700 1 Processor -- start 2023-10-06T05:17:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.544+0000 7fc9fadff700 1 -- start start 2023-10-06T05:17:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.545+0000 7fc9fadff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc9f4153d40 0x7fc9f40b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.545+0000 7fc9fadff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9f40b09c0 0x7fc9f40af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.545+0000 7fc9fadff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f40b0e40 con 0x7fc9f40af6c0 2023-10-06T05:17:34.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.545+0000 7fc9fadff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f40ad690 con 0x7fc9f4153d40 2023-10-06T05:17:34.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.545+0000 7fc9fadff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f40acd60 con 0x7fc9f40b09c0 2023-10-06T05:17:34.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f8b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc9f4153d40 0x7fc9f40b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f8b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc9f4153d40 0x7fc9f40b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53902/0 (socket says 172.21.15.31:53902) 2023-10-06T05:17:34.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f939c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc9f40af6c0 0x7fc9f4159ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46226/0 (socket says 172.21.15.31:46226) 2023-10-06T05:17:34.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f8b9b700 1 -- 172.21.15.31:0/2193312987 learned_addr learned my addr 172.21.15.31:0/2193312987 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:34.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f3fff700 1 --2- 172.21.15.31:0/2193312987 >> v2:172.21.15.62:3300/0 conn(0x7fc9f40b09c0 0x7fc9f40af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f8b9b700 1 -- 172.21.15.31:0/2193312987 >> v2:172.21.15.62:3300/0 conn(0x7fc9f40b09c0 msgr2=0x7fc9f40af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f8b9b700 1 --2- 172.21.15.31:0/2193312987 >> v2:172.21.15.62:3300/0 conn(0x7fc9f40b09c0 0x7fc9f40af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f8b9b700 1 -- 172.21.15.31:0/2193312987 >> v1:172.21.15.31:6789/0 conn(0x7fc9f40af6c0 legacy=0x7fc9f4159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f8b9b700 1 -- 172.21.15.31:0/2193312987 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f40ad040 con 0x7fc9f4153d40 2023-10-06T05:17:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f37fe700 1 -- 172.21.15.31:0/2193312987 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820886301 0 0) 0x7fc9ec001490 con 0x7fc9f40af6c0 2023-10-06T05:17:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.546+0000 7fc9f8b9b700 1 --2- 172.21.15.31:0/2193312987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f4153d40 0x7fc9f40b0480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4064120 tx=0x7fc9e800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ff1365c4ac98e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9f37fe700 1 -- 172.21.15.31:0/2193312987 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800dba0 con 0x7fc9f4153d40 2023-10-06T05:17:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9f37fe700 1 -- 172.21.15.31:0/2193312987 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e800dd00 con 0x7fc9f4153d40 2023-10-06T05:17:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9f37fe700 1 -- 172.21.15.31:0/2193312987 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e80196d0 con 0x7fc9f4153d40 2023-10-06T05:17:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9fadff700 1 -- 172.21.15.31:0/2193312987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f4153d40 msgr2=0x7fc9f40b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9fadff700 1 --2- 172.21.15.31:0/2193312987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f4153d40 0x7fc9f40b0480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4064120 tx=0x7fc9e800b4f0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9fadff700 1 -- 172.21.15.31:0/2193312987 shutdown_connections 2023-10-06T05:17:34.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9fadff700 1 --2- 172.21.15.31:0/2193312987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f4153d40 0x7fc9f40b0480 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9fadff700 1 --2- 172.21.15.31:0/2193312987 >> v2:172.21.15.62:3300/0 conn(0x7fc9f40b09c0 0x7fc9f40af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9fadff700 1 -- 172.21.15.31:0/2193312987 >> 172.21.15.31:0/2193312987 conn(0x7fc9f4067560 msgr2=0x7fc9f4067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:34.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9fadff700 1 -- 172.21.15.31:0/2193312987 shutdown_connections 2023-10-06T05:17:34.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.547+0000 7fc9fadff700 1 -- 172.21.15.31:0/2193312987 wait complete. 2023-10-06T05:17:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.549+0000 7fc9fadff700 1 Processor -- start 2023-10-06T05:17:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.549+0000 7fc9fadff700 1 -- start start 2023-10-06T05:17:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9fadff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9f40af6c0 0x7fc9f414e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9fadff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f40b09c0 0x7fc9f40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9fadff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9f4153d40 0x7fc9f40cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9fadff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f4137570 con 0x7fc9f40b09c0 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9fadff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f414f260 con 0x7fc9f4153d40 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9fadff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f4151c40 con 0x7fc9f40af6c0 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f40b09c0 0x7fc9f40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f40b09c0 0x7fc9f40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53908/0 (socket says 172.21.15.31:53908) 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9f3fff700 1 -- 172.21.15.31:0/1418697506 learned_addr learned my addr 172.21.15.31:0/1418697506 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9f8b9b700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9f40af6c0 0x7fc9f414e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.550+0000 7fc9f939c700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9f4153d40 0x7fc9f40cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9f3fff700 1 -- 172.21.15.31:0/1418697506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9f40af6c0 msgr2=0x7fc9f414e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9f3fff700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9f40af6c0 0x7fc9f414e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9f3fff700 1 -- 172.21.15.31:0/1418697506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9f4153d40 msgr2=0x7fc9f40cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9f3fff700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9f4153d40 0x7fc9f40cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9f3fff700 1 -- 172.21.15.31:0/1418697506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e800b790 con 0x7fc9f40b09c0 2023-10-06T05:17:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9f8b9b700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9f40af6c0 0x7fc9f414e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9f3fff700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f40b09c0 0x7fc9f40cae70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000ecb0 tx=0x7fc9e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9f0ff9700 1 -- 172.21.15.31:0/1418697506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000c970 con 0x7fc9f40b09c0 2023-10-06T05:17:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9f0ff9700 1 -- 172.21.15.31:0/1418697506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e000cad0 con 0x7fc9f40b09c0 2023-10-06T05:17:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9f0ff9700 1 -- 172.21.15.31:0/1418697506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000ce70 con 0x7fc9f40b09c0 2023-10-06T05:17:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9fadff700 1 -- 172.21.15.31:0/1418697506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f4151310 con 0x7fc9f40b09c0 2023-10-06T05:17:34.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.551+0000 7fc9fadff700 1 -- 172.21.15.31:0/1418697506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f41518d0 con 0x7fc9f40b09c0 2023-10-06T05:17:34.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.552+0000 7fc9fadff700 1 -- 172.21.15.31:0/1418697506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f40afc20 con 0x7fc9f40b09c0 2023-10-06T05:17:34.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.553+0000 7fc9f0ff9700 1 -- 172.21.15.31:0/1418697506 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9e0013680 con 0x7fc9f40b09c0 2023-10-06T05:17:34.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.554+0000 7fc9f0ff9700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9d8066f30 0x7fc9d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.554+0000 7fc9f0ff9700 1 -- 172.21.15.31:0/1418697506 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc9e0084ae0 con 0x7fc9f40b09c0 2023-10-06T05:17:34.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.554+0000 7fc9f8b9b700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9d8066f30 0x7fc9d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.555+0000 7fc9f8b9b700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9d8066f30 0x7fc9d80693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4064120 tx=0x7fc9e801b000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.558+0000 7fc9f0ff9700 1 -- 172.21.15.31:0/1418697506 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9e0051b50 con 0x7fc9f40b09c0 2023-10-06T05:17:34.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.761+0000 7fc9fadff700 1 -- 172.21.15.31:0/1418697506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9f404c380 con 0x7fc9f40b09c0 2023-10-06T05:17:34.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.761+0000 7fc9f0ff9700 1 -- 172.21.15.31:0/1418697506 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9e0055170 con 0x7fc9f40b09c0 2023-10-06T05:17:34.763 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.763+0000 7fc9fadff700 1 -- 172.21.15.31:0/1418697506 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9d8066f30 msgr2=0x7fc9d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.763+0000 7fc9fadff700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9d8066f30 0x7fc9d80693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4064120 tx=0x7fc9e801b000 comp rx=0 tx=0).stop 2023-10-06T05:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.764+0000 7fc9fadff700 1 -- 172.21.15.31:0/1418697506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f40b09c0 msgr2=0x7fc9f40cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.764+0000 7fc9fadff700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f40b09c0 0x7fc9f40cae70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000ecb0 tx=0x7fc9e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.764+0000 7fc9fadff700 1 -- 172.21.15.31:0/1418697506 shutdown_connections 2023-10-06T05:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.764+0000 7fc9fadff700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9f4153d40 0x7fc9f40cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.764+0000 7fc9fadff700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9f40af6c0 0x7fc9f414e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.764+0000 7fc9fadff700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9d8066f30 0x7fc9d80693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.764+0000 7fc9fadff700 1 --2- 172.21.15.31:0/1418697506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9f40b09c0 0x7fc9f40cae70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.764+0000 7fc9fadff700 1 -- 172.21.15.31:0/1418697506 >> 172.21.15.31:0/1418697506 conn(0x7fc9f4067560 msgr2=0x7fc9f40cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.764+0000 7fc9fadff700 1 -- 172.21.15.31:0/1418697506 shutdown_connections 2023-10-06T05:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.764+0000 7fc9fadff700 1 -- 172.21.15.31:0/1418697506 wait complete. 2023-10-06T05:17:34.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:17:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.979+0000 7fb7355ff700 1 Processor -- start 2023-10-06T05:17:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.979+0000 7fb7355ff700 1 -- start start 2023-10-06T05:17:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.980+0000 7fb7355ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7300c7190 0x7fb7300c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.980+0000 7fb7355ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7300c7560 0x7fb7300686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.980+0000 7fb7355ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb730068250 con 0x7fb7300aef00 2023-10-06T05:17:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.980+0000 7fb7355ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73006ac80 con 0x7fb7300c7560 2023-10-06T05:17:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.980+0000 7fb7355ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb730067f20 con 0x7fb7300c7190 2023-10-06T05:17:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.981+0000 7fb72f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7300c7560 0x7fb7300686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.981+0000 7fb72f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7300c7560 0x7fb7300686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53910/0 (socket says 172.21.15.31:53910) 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.981+0000 7fb72f7fe700 1 -- 172.21.15.31:0/1949842065 learned_addr learned my addr 172.21.15.31:0/1949842065 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.981+0000 7fb72effd700 1 --2- 172.21.15.31:0/1949842065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7300c7190 0x7fb7300c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.981+0000 7fb72f7fe700 1 -- 172.21.15.31:0/1949842065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7300c7190 msgr2=0x7fb7300c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.981+0000 7fb72f7fe700 1 --2- 172.21.15.31:0/1949842065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7300c7190 0x7fb7300c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.981+0000 7fb72f7fe700 1 -- 172.21.15.31:0/1949842065 >> v1:172.21.15.62:6789/0 conn(0x7fb7300aef00 legacy=0x7fb7300af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.981+0000 7fb72f7fe700 1 -- 172.21.15.31:0/1949842065 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73006aed0 con 0x7fb7300c7560 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.982+0000 7fb72f7fe700 1 --2- 172.21.15.31:0/1949842065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7560 0x7fb7300686e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb728009f20 tx=0x7fb728009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f50efeb782c44f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.982+0000 7fb72dffb700 1 -- 172.21.15.31:0/1949842065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72800e070 con 0x7fb7300c7560 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.982+0000 7fb72dffb700 1 -- 172.21.15.31:0/1949842065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72800ba80 con 0x7fb7300c7560 2023-10-06T05:17:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.982+0000 7fb72dffb700 1 -- 172.21.15.31:0/1949842065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7280092e0 con 0x7fb7300c7560 2023-10-06T05:17:34.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.982+0000 7fb7355ff700 1 -- 172.21.15.31:0/1949842065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7560 msgr2=0x7fb7300686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.982+0000 7fb7355ff700 1 --2- 172.21.15.31:0/1949842065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7560 0x7fb7300686e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb728009f20 tx=0x7fb728009840 comp rx=0 tx=0).stop 2023-10-06T05:17:34.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.983+0000 7fb7355ff700 1 -- 172.21.15.31:0/1949842065 shutdown_connections 2023-10-06T05:17:34.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.983+0000 7fb7355ff700 1 --2- 172.21.15.31:0/1949842065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7560 0x7fb7300686e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.983+0000 7fb7355ff700 1 --2- 172.21.15.31:0/1949842065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7300c7190 0x7fb7300c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.983+0000 7fb7355ff700 1 -- 172.21.15.31:0/1949842065 >> 172.21.15.31:0/1949842065 conn(0x7fb7300cd3d0 msgr2=0x7fb7300c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.983+0000 7fb7355ff700 1 -- 172.21.15.31:0/1949842065 shutdown_connections 2023-10-06T05:17:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.983+0000 7fb7355ff700 1 -- 172.21.15.31:0/1949842065 wait complete. 2023-10-06T05:17:34.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.984+0000 7fb7355ff700 1 Processor -- start 2023-10-06T05:17:34.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.984+0000 7fb7355ff700 1 -- start start 2023-10-06T05:17:34.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.985+0000 7fb7355ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7300aef00 0x7fb7300c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.985+0000 7fb7355ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7190 0x7fb7300c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.985+0000 7fb7355ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7300c7560 0x7fb730169280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.985+0000 7fb7355ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73015d780 con 0x7fb7300c7190 2023-10-06T05:17:34.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.985+0000 7fb7355ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300c5420 con 0x7fb7300c7560 2023-10-06T05:17:34.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.985+0000 7fb7355ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300c5700 con 0x7fb7300aef00 2023-10-06T05:17:34.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.985+0000 7fb72e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7190 0x7fb7300c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.985+0000 7fb72e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7190 0x7fb7300c4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53926/0 (socket says 172.21.15.31:53926) 2023-10-06T05:17:34.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.985+0000 7fb72e7fc700 1 -- 172.21.15.31:0/2217098445 learned_addr learned my addr 172.21.15.31:0/2217098445 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:34.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb72f7fe700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7300c7560 0x7fb730169280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb72effd700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7300aef00 0x7fb7300c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb72e7fc700 1 -- 172.21.15.31:0/2217098445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7300aef00 msgr2=0x7fb7300c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb72e7fc700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7300aef00 0x7fb7300c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb72e7fc700 1 -- 172.21.15.31:0/2217098445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7300c7560 msgr2=0x7fb730169280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb72e7fc700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7300c7560 0x7fb730169280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb72e7fc700 1 -- 172.21.15.31:0/2217098445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb728009ae0 con 0x7fb7300c7190 2023-10-06T05:17:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb72e7fc700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7190 0x7fb7300c4bb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb72400c4e0 tx=0x7fb72400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb716ffd700 1 -- 172.21.15.31:0/2217098445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724011590 con 0x7fb7300c7190 2023-10-06T05:17:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb716ffd700 1 -- 172.21.15.31:0/2217098445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb724011710 con 0x7fb7300c7190 2023-10-06T05:17:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb7355ff700 1 -- 172.21.15.31:0/2217098445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73015d9d0 con 0x7fb7300c7190 2023-10-06T05:17:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.986+0000 7fb716ffd700 1 -- 172.21.15.31:0/2217098445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724011ad0 con 0x7fb7300c7190 2023-10-06T05:17:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.987+0000 7fb7355ff700 1 -- 172.21.15.31:0/2217098445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7300c5170 con 0x7fb7300c7190 2023-10-06T05:17:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.988+0000 7fb7355ff700 1 -- 172.21.15.31:0/2217098445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73015def0 con 0x7fb7300c7190 2023-10-06T05:17:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.989+0000 7fb716ffd700 1 -- 172.21.15.31:0/2217098445 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb72401b3b0 con 0x7fb7300c7190 2023-10-06T05:17:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.989+0000 7fb716ffd700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.989+0000 7fb716ffd700 1 -- 172.21.15.31:0/2217098445 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb724083b60 con 0x7fb7300c7190 2023-10-06T05:17:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.990+0000 7fb72effd700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.990+0000 7fb72effd700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb71c066f30 0x7fb71c0693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb720006fd0 tx=0x7fb720008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:34.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:34.993+0000 7fb716ffd700 1 -- 172.21.15.31:0/2217098445 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb73015def0 con 0x7fb7300c7190 2023-10-06T05:17:35.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.192+0000 7fb7355ff700 1 -- 172.21.15.31:0/2217098445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb73015dec0 con 0x7fb7300c7190 2023-10-06T05:17:35.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.193+0000 7fb716ffd700 1 -- 172.21.15.31:0/2217098445 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb724017020 con 0x7fb7300c7190 2023-10-06T05:17:35.195 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:17:35.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 -- 172.21.15.31:0/2217098445 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb71c066f30 msgr2=0x7fb71c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb71c066f30 0x7fb71c0693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb720006fd0 tx=0x7fb720008040 comp rx=0 tx=0).stop 2023-10-06T05:17:35.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 -- 172.21.15.31:0/2217098445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7190 msgr2=0x7fb7300c4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7190 0x7fb7300c4bb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb72400c4e0 tx=0x7fb72400c7f0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 -- 172.21.15.31:0/2217098445 shutdown_connections 2023-10-06T05:17:35.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7300c7560 0x7fb730169280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb7300aef00 0x7fb7300c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 --2- 172.21.15.31:0/2217098445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7300c7190 0x7fb7300c4bb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 -- 172.21.15.31:0/2217098445 >> 172.21.15.31:0/2217098445 conn(0x7fb7300cd3d0 msgr2=0x7fb73006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:35.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 -- 172.21.15.31:0/2217098445 shutdown_connections 2023-10-06T05:17:35.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.195+0000 7fb7355ff700 1 -- 172.21.15.31:0/2217098445 wait complete. 2023-10-06T05:17:35.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:17:35.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.410+0000 7fc186fff700 1 Processor -- start 2023-10-06T05:17:35.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.411+0000 7fc186fff700 1 -- start start 2023-10-06T05:17:35.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc186fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc180159560 0x7fc1800cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc186fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc180159930 0x7fc1800cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc186fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1800b0d60 0x7fc1800b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc186fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18015d2d0 con 0x7fc1800b0d60 2023-10-06T05:17:35.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc186fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18015d5b0 con 0x7fc180159560 2023-10-06T05:17:35.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc186fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18015d890 con 0x7fc180159930 2023-10-06T05:17:35.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc18559c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1800b0d60 0x7fc1800b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc18559c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1800b0d60 0x7fc1800b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53940/0 (socket says 172.21.15.31:53940) 2023-10-06T05:17:35.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc18559c700 1 -- 172.21.15.31:0/2868566260 learned_addr learned my addr 172.21.15.31:0/2868566260 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:35.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc17ffff700 1 --2- 172.21.15.31:0/2868566260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc180159930 0x7fc1800cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.412+0000 7fc184d9b700 1 --2- 172.21.15.31:0/2868566260 >> v2:172.21.15.62:3300/0 conn(0x7fc180159560 0x7fc1800cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.413+0000 7fc18559c700 1 -- 172.21.15.31:0/2868566260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc180159930 msgr2=0x7fc1800cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.413+0000 7fc18559c700 1 --2- 172.21.15.31:0/2868566260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc180159930 0x7fc1800cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.413+0000 7fc18559c700 1 -- 172.21.15.31:0/2868566260 >> v2:172.21.15.62:3300/0 conn(0x7fc180159560 msgr2=0x7fc1800cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.413+0000 7fc18559c700 1 --2- 172.21.15.31:0/2868566260 >> v2:172.21.15.62:3300/0 conn(0x7fc180159560 0x7fc1800cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.413+0000 7fc18559c700 1 -- 172.21.15.31:0/2868566260 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18015db70 con 0x7fc1800b0d60 2023-10-06T05:17:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.413+0000 7fc18559c700 1 --2- 172.21.15.31:0/2868566260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1800b0d60 0x7fc1800b2470 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc180063d80 tx=0x7fc178009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394d928b426c488 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.413+0000 7fc17f7fe700 1 -- 172.21.15.31:0/2868566260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800e070 con 0x7fc1800b0d60 2023-10-06T05:17:35.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.413+0000 7fc17f7fe700 1 -- 172.21.15.31:0/2868566260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17800b970 con 0x7fc1800b0d60 2023-10-06T05:17:35.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.413+0000 7fc17f7fe700 1 -- 172.21.15.31:0/2868566260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780092e0 con 0x7fc1800b0d60 2023-10-06T05:17:35.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.414+0000 7fc186fff700 1 -- 172.21.15.31:0/2868566260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1800b0d60 msgr2=0x7fc1800b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.414+0000 7fc186fff700 1 --2- 172.21.15.31:0/2868566260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1800b0d60 0x7fc1800b2470 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc180063d80 tx=0x7fc178009840 comp rx=0 tx=0).stop 2023-10-06T05:17:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.414+0000 7fc186fff700 1 -- 172.21.15.31:0/2868566260 shutdown_connections 2023-10-06T05:17:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.414+0000 7fc186fff700 1 --2- 172.21.15.31:0/2868566260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1800b0d60 0x7fc1800b2470 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.414+0000 7fc186fff700 1 --2- 172.21.15.31:0/2868566260 >> v2:172.21.15.62:3300/0 conn(0x7fc180159560 0x7fc1800cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.414+0000 7fc186fff700 1 --2- 172.21.15.31:0/2868566260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc180159930 0x7fc1800cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.414+0000 7fc186fff700 1 -- 172.21.15.31:0/2868566260 >> 172.21.15.31:0/2868566260 conn(0x7fc18006ec90 msgr2=0x7fc18006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.414+0000 7fc186fff700 1 -- 172.21.15.31:0/2868566260 shutdown_connections 2023-10-06T05:17:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.414+0000 7fc186fff700 1 -- 172.21.15.31:0/2868566260 wait complete. 2023-10-06T05:17:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.415+0000 7fc186fff700 1 Processor -- start 2023-10-06T05:17:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.416+0000 7fc186fff700 1 -- start start 2023-10-06T05:17:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc186fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1800b0d60 0x7fc18014a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc186fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc180159560 0x7fc1800d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc186fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc180159930 0x7fc1800c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc186fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800ce5f0 con 0x7fc180159560 2023-10-06T05:17:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc186fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800cdfc0 con 0x7fc180159930 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc186fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc180156b60 con 0x7fc1800b0d60 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc17ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc180159560 0x7fc1800d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc17ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc180159560 0x7fc1800d0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53942/0 (socket says 172.21.15.31:53942) 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc17ffff700 1 -- 172.21.15.31:0/3846189660 learned_addr learned my addr 172.21.15.31:0/3846189660 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc184d9b700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1800b0d60 0x7fc18014a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc18559c700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc180159930 0x7fc1800c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc17ffff700 1 -- 172.21.15.31:0/3846189660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1800b0d60 msgr2=0x7fc18014a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc17ffff700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1800b0d60 0x7fc18014a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc17ffff700 1 -- 172.21.15.31:0/3846189660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc180159930 msgr2=0x7fc1800c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc17ffff700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc180159930 0x7fc1800c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.417+0000 7fc17ffff700 1 -- 172.21.15.31:0/3846189660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc178009ae0 con 0x7fc180159560 2023-10-06T05:17:35.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.418+0000 7fc17ffff700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc180159560 0x7fc1800d0730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00e930 tx=0x7fc16c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.418+0000 7fc17cff9700 1 -- 172.21.15.31:0/3846189660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c013680 con 0x7fc180159560 2023-10-06T05:17:35.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.418+0000 7fc17cff9700 1 -- 172.21.15.31:0/3846189660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c0137e0 con 0x7fc180159560 2023-10-06T05:17:35.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.418+0000 7fc17cff9700 1 -- 172.21.15.31:0/3846189660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c013b80 con 0x7fc180159560 2023-10-06T05:17:35.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.418+0000 7fc186fff700 1 -- 172.21.15.31:0/3846189660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1800c5100 con 0x7fc180159560 2023-10-06T05:17:35.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.418+0000 7fc186fff700 1 -- 172.21.15.31:0/3846189660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1800ce280 con 0x7fc180159560 2023-10-06T05:17:35.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.419+0000 7fc186fff700 1 -- 172.21.15.31:0/3846189660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1800ce760 con 0x7fc180159560 2023-10-06T05:17:35.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.420+0000 7fc17cff9700 1 -- 172.21.15.31:0/3846189660 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc16c00c3e0 con 0x7fc180159560 2023-10-06T05:17:35.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.421+0000 7fc17cff9700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc164066f30 0x7fc1640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.421+0000 7fc17cff9700 1 -- 172.21.15.31:0/3846189660 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc16c0847e0 con 0x7fc180159560 2023-10-06T05:17:35.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.421+0000 7fc184d9b700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc164066f30 0x7fc1640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.422+0000 7fc184d9b700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc164066f30 0x7fc1640693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc174009db0 tx=0x7fc1740091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.425+0000 7fc17cff9700 1 -- 172.21.15.31:0/3846189660 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc16c051850 con 0x7fc180159560 2023-10-06T05:17:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.628+0000 7fc186fff700 1 -- 172.21.15.31:0/3846189660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc1800ce440 con 0x7fc180159560 2023-10-06T05:17:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.628+0000 7fc17cff9700 1 -- 172.21.15.31:0/3846189660 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc16c054e70 con 0x7fc180159560 2023-10-06T05:17:35.630 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:17:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 -- 172.21.15.31:0/3846189660 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc164066f30 msgr2=0x7fc1640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc164066f30 0x7fc1640693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc174009db0 tx=0x7fc1740091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 -- 172.21.15.31:0/3846189660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc180159560 msgr2=0x7fc1800d0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc180159560 0x7fc1800d0730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00e930 tx=0x7fc16c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 -- 172.21.15.31:0/3846189660 shutdown_connections 2023-10-06T05:17:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc180159930 0x7fc1800c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1800b0d60 0x7fc18014a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc164066f30 0x7fc1640693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 --2- 172.21.15.31:0/3846189660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc180159560 0x7fc1800d0730 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 -- 172.21.15.31:0/3846189660 >> 172.21.15.31:0/3846189660 conn(0x7fc18006ec90 msgr2=0x7fc1800b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 -- 172.21.15.31:0/3846189660 shutdown_connections 2023-10-06T05:17:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.631+0000 7fc186fff700 1 -- 172.21.15.31:0/3846189660 wait complete. 2023-10-06T05:17:35.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:35.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.843+0000 7f6cd9dff700 1 Processor -- start 2023-10-06T05:17:35.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.844+0000 7f6cd9dff700 1 -- start start 2023-10-06T05:17:35.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.845+0000 7f6cd9dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6cd413a1e0 0x7f6cd40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.845+0000 7f6cd9dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6cd413a5b0 0x7f6cd406d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.845+0000 7f6cd9dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd40c71b0 con 0x7f6cd40c6c30 2023-10-06T05:17:35.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.845+0000 7f6cd9dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd40c7490 con 0x7f6cd413a5b0 2023-10-06T05:17:35.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.845+0000 7f6cd9dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd40c7770 con 0x7f6cd413a1e0 2023-10-06T05:17:35.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.845+0000 7f6cd2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6cd413a5b0 0x7f6cd406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.845+0000 7f6cd2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6cd413a5b0 0x7f6cd406d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53958/0 (socket says 172.21.15.31:53958) 2023-10-06T05:17:35.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.845+0000 7f6cd2ffd700 1 -- 172.21.15.31:0/1457299220 learned_addr learned my addr 172.21.15.31:0/1457299220 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:35.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.845+0000 7f6cd3fff700 1 --1- 172.21.15.31:0/1457299220 >> v1:172.21.15.31:6789/0 conn(0x7f6cd40c6c30 0x7f6cd415afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46238/0 (socket says 172.21.15.31:46238) 2023-10-06T05:17:35.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.846+0000 7f6cd37fe700 1 --2- 172.21.15.31:0/1457299220 >> v2:172.21.15.62:3300/0 conn(0x7f6cd413a1e0 0x7f6cd40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.846+0000 7f6cd2ffd700 1 -- 172.21.15.31:0/1457299220 >> v2:172.21.15.62:3300/0 conn(0x7f6cd413a1e0 msgr2=0x7f6cd40c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:35.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.846+0000 7f6cd2ffd700 1 --2- 172.21.15.31:0/1457299220 >> v2:172.21.15.62:3300/0 conn(0x7f6cd413a1e0 0x7f6cd40c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.846+0000 7f6cd2ffd700 1 -- 172.21.15.31:0/1457299220 >> v1:172.21.15.31:6789/0 conn(0x7f6cd40c6c30 legacy=0x7f6cd415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.846+0000 7f6cd2ffd700 1 -- 172.21.15.31:0/1457299220 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd40c7a50 con 0x7f6cd413a5b0 2023-10-06T05:17:35.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.846+0000 7f6cd37fe700 1 --2- 172.21.15.31:0/1457299220 >> v2:172.21.15.62:3300/0 conn(0x7f6cd413a1e0 0x7f6cd40c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:35.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.846+0000 7f6cd2ffd700 1 --2- 172.21.15.31:0/1457299220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd413a5b0 0x7f6cd406d440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40cbf90 tx=0x7f6cc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aec75c5c8a0c7b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.846+0000 7f6cd27fc700 1 -- 172.21.15.31:0/1457299220 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800b860 con 0x7f6cd413a5b0 2023-10-06T05:17:35.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.846+0000 7f6cd27fc700 1 -- 172.21.15.31:0/1457299220 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc800b9c0 con 0x7f6cd413a5b0 2023-10-06T05:17:35.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.846+0000 7f6cd27fc700 1 -- 172.21.15.31:0/1457299220 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800bd60 con 0x7f6cd413a5b0 2023-10-06T05:17:35.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.847+0000 7f6cd9dff700 1 -- 172.21.15.31:0/1457299220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd413a5b0 msgr2=0x7f6cd406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.847+0000 7f6cd9dff700 1 --2- 172.21.15.31:0/1457299220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd413a5b0 0x7f6cd406d440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40cbf90 tx=0x7f6cc8009250 comp rx=0 tx=0).stop 2023-10-06T05:17:35.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.847+0000 7f6cd9dff700 1 -- 172.21.15.31:0/1457299220 shutdown_connections 2023-10-06T05:17:35.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.847+0000 7f6cd9dff700 1 --2- 172.21.15.31:0/1457299220 >> v2:172.21.15.62:3300/0 conn(0x7f6cd413a1e0 0x7f6cd40c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.847+0000 7f6cd9dff700 1 --2- 172.21.15.31:0/1457299220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd413a5b0 0x7f6cd406d440 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.847+0000 7f6cd9dff700 1 -- 172.21.15.31:0/1457299220 >> 172.21.15.31:0/1457299220 conn(0x7f6cd40cc0b0 msgr2=0x7f6cd40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:35.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.847+0000 7f6cd9dff700 1 -- 172.21.15.31:0/1457299220 shutdown_connections 2023-10-06T05:17:35.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.847+0000 7f6cd9dff700 1 -- 172.21.15.31:0/1457299220 wait complete. 2023-10-06T05:17:35.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.849+0000 7f6cd9dff700 1 Processor -- start 2023-10-06T05:17:35.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.849+0000 7f6cd9dff700 1 -- start start 2023-10-06T05:17:35.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd9dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd40c6c30 0x7f6cd40c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd9dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cd413a1e0 0x7f6cd4139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd9dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd413a5b0 0x7f6cd40c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd9dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd40c5ee0 con 0x7f6cd40c6c30 2023-10-06T05:17:35.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd9dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd40c52d0 con 0x7f6cd413a5b0 2023-10-06T05:17:35.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd9dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd40c55b0 con 0x7f6cd413a1e0 2023-10-06T05:17:35.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd40c6c30 0x7f6cd40c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd40c6c30 0x7f6cd40c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53966/0 (socket says 172.21.15.31:53966) 2023-10-06T05:17:35.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd37fe700 1 -- 172.21.15.31:0/2565462676 learned_addr learned my addr 172.21.15.31:0/2565462676 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:35.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd2ffd700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cd413a1e0 0x7f6cd4139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd3fff700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd413a5b0 0x7f6cd40c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd37fe700 1 -- 172.21.15.31:0/2565462676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cd413a1e0 msgr2=0x7f6cd4139bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd37fe700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cd413a1e0 0x7f6cd4139bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd37fe700 1 -- 172.21.15.31:0/2565462676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd413a5b0 msgr2=0x7f6cd40c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:35.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd37fe700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd413a5b0 0x7f6cd40c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:35.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.850+0000 7f6cd37fe700 1 -- 172.21.15.31:0/2565462676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80094f0 con 0x7f6cd40c6c30 2023-10-06T05:17:35.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.851+0000 7f6cd37fe700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd40c6c30 0x7f6cd40c4c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400e920 tx=0x7f6cc4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.851+0000 7f6cbb7fe700 1 -- 172.21.15.31:0/2565462676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4005a00 con 0x7f6cd40c6c30 2023-10-06T05:17:35.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.851+0000 7f6cbb7fe700 1 -- 172.21.15.31:0/2565462676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc4005b60 con 0x7f6cd40c6c30 2023-10-06T05:17:35.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.851+0000 7f6cbb7fe700 1 -- 172.21.15.31:0/2565462676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4006020 con 0x7f6cd40c6c30 2023-10-06T05:17:35.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.851+0000 7f6cd9dff700 1 -- 172.21.15.31:0/2565462676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd40c5890 con 0x7f6cd40c6c30 2023-10-06T05:17:35.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.851+0000 7f6cd9dff700 1 -- 172.21.15.31:0/2565462676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd40b1b70 con 0x7f6cd40c6c30 2023-10-06T05:17:35.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.852+0000 7f6cd9dff700 1 -- 172.21.15.31:0/2565462676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd404c380 con 0x7f6cd40c6c30 2023-10-06T05:17:35.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.853+0000 7f6cbb7fe700 1 -- 172.21.15.31:0/2565462676 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cc4006410 con 0x7f6cd40c6c30 2023-10-06T05:17:35.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.854+0000 7f6cbb7fe700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cc0066fb0 0x7f6cc0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:35.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.854+0000 7f6cbb7fe700 1 -- 172.21.15.31:0/2565462676 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6cc4084940 con 0x7f6cd40c6c30 2023-10-06T05:17:35.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.854+0000 7f6cd2ffd700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cc0066fb0 0x7f6cc0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:35.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.854+0000 7f6cd2ffd700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cc0066fb0 0x7f6cc0069460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6cd413dba0 tx=0x7f6cc8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:35.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:35.858+0000 7f6cbb7fe700 1 -- 172.21.15.31:0/2565462676 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cc40519b0 con 0x7f6cd40c6c30 2023-10-06T05:17:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.062+0000 7f6cd9dff700 1 -- 172.21.15.31:0/2565462676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cd40c7d30 con 0x7f6cd40c6c30 2023-10-06T05:17:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.063+0000 7f6cbb7fe700 1 -- 172.21.15.31:0/2565462676 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f6cc4054fd0 con 0x7f6cd40c6c30 2023-10-06T05:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:36.065 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:36.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 -- 172.21.15.31:0/2565462676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cc0066fb0 msgr2=0x7f6cc0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cc0066fb0 0x7f6cc0069460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6cd413dba0 tx=0x7f6cc8017000 comp rx=0 tx=0).stop 2023-10-06T05:17:36.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 -- 172.21.15.31:0/2565462676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd40c6c30 msgr2=0x7f6cd40c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd40c6c30 0x7f6cd40c4c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400e920 tx=0x7f6cc4008040 comp rx=0 tx=0).stop 2023-10-06T05:17:36.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 -- 172.21.15.31:0/2565462676 shutdown_connections 2023-10-06T05:17:36.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd413a5b0 0x7f6cd40c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cc0066fb0 0x7f6cc0069460 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd40c6c30 0x7f6cd40c4c80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 --2- 172.21.15.31:0/2565462676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cd413a1e0 0x7f6cd4139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 -- 172.21.15.31:0/2565462676 >> 172.21.15.31:0/2565462676 conn(0x7f6cd40cc0b0 msgr2=0x7f6cd406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:36.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 -- 172.21.15.31:0/2565462676 shutdown_connections 2023-10-06T05:17:36.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.065+0000 7f6cd9dff700 1 -- 172.21.15.31:0/2565462676 wait complete. 2023-10-06T05:17:36.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:16:59.507957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:17:00.905826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:17:36.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.283+0000 7f50615ff700 1 Processor -- start 2023-10-06T05:17:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.283+0000 7f50615ff700 1 -- start start 2023-10-06T05:17:36.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.284+0000 7f50615ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f505c15a930 0x7f505c0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.284+0000 7f50615ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f505c0cf470 0x7f505c0cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.284+0000 7f50615ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c0cc010 con 0x7f505c15a560 2023-10-06T05:17:36.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.284+0000 7f50615ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c0cd6e0 con 0x7f505c15a930 2023-10-06T05:17:36.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.284+0000 7f50615ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0cdfd0 con 0x7f505c0cf470 2023-10-06T05:17:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.284+0000 7f505affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f505c15a560 0x7f505c0caa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46240/0 (socket says 172.21.15.31:46240) 2023-10-06T05:17:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.284+0000 7f505affd700 1 -- 172.21.15.31:0/3378999536 learned_addr learned my addr 172.21.15.31:0/3378999536 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:36.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021352561 0 0) 0x7f505c0cc010 con 0x7f505c15a560 2023-10-06T05:17:36.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f505b7fe700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f505c0cf470 0x7f505c0cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f505a7fc700 1 --2- 172.21.15.31:0/3378999536 >> v2:172.21.15.62:3300/0 conn(0x7f505c15a930 0x7f505c0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5044003570 con 0x7f505c15a560 2023-10-06T05:17:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3098744171 0 0) 0x7f5044003570 con 0x7f505c15a560 2023-10-06T05:17:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f505c0cf470 msgr2=0x7f505c0cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f505c0cf470 0x7f505c0cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 >> v2:172.21.15.62:3300/0 conn(0x7f505c15a930 msgr2=0x7f505c0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 --2- 172.21.15.31:0/3378999536 >> v2:172.21.15.62:3300/0 conn(0x7f505c15a930 0x7f505c0ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c0cb490 con 0x7f505c15a560 2023-10-06T05:17:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f504c0015f0 con 0x7f505c15a560 2023-10-06T05:17:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 >> v1:172.21.15.31:6789/0 conn(0x7f505c15a560 legacy=0x7f505c0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f505c0cf470 0x7f5044003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5044003fe0 0x7f5044004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5044004d50 0x7f50440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0cd6e0 con 0x7f5044003fe0 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0cdfd0 con 0x7f5044004d50 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.285+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504400ba90 con 0x7f505c0cf470 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.286+0000 7f505a7fc700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5044003fe0 0x7f5044004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.286+0000 7f505affd700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f505c0cf470 0x7f5044003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.286+0000 7f505b7fe700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5044004d50 0x7f50440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.286+0000 7f505a7fc700 1 -- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f505c0cf470 msgr2=0x7f5044003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.286+0000 7f505a7fc700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f505c0cf470 0x7f5044003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.286+0000 7f505a7fc700 1 -- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5044004d50 msgr2=0x7f50440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.286+0000 7f505a7fc700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5044004d50 0x7f50440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.286+0000 7f505a7fc700 1 -- 172.21.15.31:0/3378999536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5044003570 con 0x7f5044003fe0 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.286+0000 7f505a7fc700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5044003fe0 0x7f5044004810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f505c0cdc50 tx=0x7f50500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec30633f0903e7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.287+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000e680 con 0x7f5044003fe0 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.287+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505000e7e0 con 0x7f5044003fe0 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.287+0000 7f5059ffb700 1 -- 172.21.15.31:0/3378999536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000eba0 con 0x7f5044003fe0 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.287+0000 7f50615ff700 1 -- 172.21.15.31:0/3378999536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5044003fe0 msgr2=0x7f5044004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.287+0000 7f50615ff700 1 --2- 172.21.15.31:0/3378999536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5044003fe0 0x7f5044004810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f505c0cdc50 tx=0x7f50500091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.287+0000 7f505b7fe700 1 -- 172.21.15.31:0/3378999536 reap_dead start 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.287+0000 7f50615ff700 1 -- 172.21.15.31:0/3378999536 shutdown_connections 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.287+0000 7f50615ff700 1 -- 172.21.15.31:0/3378999536 >> 172.21.15.31:0/3378999536 conn(0x7f505c0996f0 msgr2=0x7f505c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.287+0000 7f50615ff700 1 -- 172.21.15.31:0/3378999536 shutdown_connections 2023-10-06T05:17:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.287+0000 7f50615ff700 1 -- 172.21.15.31:0/3378999536 wait complete. 2023-10-06T05:17:36.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.289+0000 7f50615ff700 1 Processor -- start 2023-10-06T05:17:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.289+0000 7f50615ff700 1 -- start start 2023-10-06T05:17:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f50615ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5044003fe0 0x7f505c068470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f50615ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f505c151910 0x7f505c151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f50615ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505c0c8240 0x7f505c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f50615ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c1522c0 con 0x7f505c0c8240 2023-10-06T05:17:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f50615ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0b07b0 con 0x7f505c151910 2023-10-06T05:17:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f50615ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0afba0 con 0x7f5044003fe0 2023-10-06T05:17:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f505b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505c0c8240 0x7f505c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f505b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505c0c8240 0x7f505c0c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53980/0 (socket says 172.21.15.31:53980) 2023-10-06T05:17:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f505b7fe700 1 -- 172.21.15.31:0/2161362027 learned_addr learned my addr 172.21.15.31:0/2161362027 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f505affd700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5044003fe0 0x7f505c068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f505a7fc700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f505c151910 0x7f505c151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f505b7fe700 1 -- 172.21.15.31:0/2161362027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5044003fe0 msgr2=0x7f505c068470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f505b7fe700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5044003fe0 0x7f505c068470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f505b7fe700 1 -- 172.21.15.31:0/2161362027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f505c151910 msgr2=0x7f505c151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.290+0000 7f505b7fe700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f505c151910 0x7f505c151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.291+0000 7f505b7fe700 1 -- 172.21.15.31:0/2161362027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c0afe80 con 0x7f505c0c8240 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.291+0000 7f505b7fe700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505c0c8240 0x7f505c0c86b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f505c15ad00 tx=0x7f505400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.291+0000 7f5042ffd700 1 -- 172.21.15.31:0/2161362027 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505400e070 con 0x7f505c0c8240 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.291+0000 7f5042ffd700 1 -- 172.21.15.31:0/2161362027 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5054011b80 con 0x7f505c0c8240 2023-10-06T05:17:36.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.291+0000 7f50615ff700 1 -- 172.21.15.31:0/2161362027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c0b0160 con 0x7f505c0c8240 2023-10-06T05:17:36.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.291+0000 7f5042ffd700 1 -- 172.21.15.31:0/2161362027 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054016470 con 0x7f505c0c8240 2023-10-06T05:17:36.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.291+0000 7f50615ff700 1 -- 172.21.15.31:0/2161362027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c0c8bf0 con 0x7f505c0c8240 2023-10-06T05:17:36.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.292+0000 7f50615ff700 1 -- 172.21.15.31:0/2161362027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c04c380 con 0x7f505c0c8240 2023-10-06T05:17:36.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.293+0000 7f5042ffd700 1 -- 172.21.15.31:0/2161362027 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5054016820 con 0x7f505c0c8240 2023-10-06T05:17:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.294+0000 7f5042ffd700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5048066f30 0x7f50480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.294+0000 7f5042ffd700 1 -- 172.21.15.31:0/2161362027 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5054085260 con 0x7f505c0c8240 2023-10-06T05:17:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.294+0000 7f505affd700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5048066f30 0x7f50480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.295+0000 7f505affd700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5048066f30 0x7f50480693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f504c0011d0 tx=0x7f504c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.298+0000 7f5042ffd700 1 -- 172.21.15.31:0/2161362027 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50540522d0 con 0x7f505c0c8240 2023-10-06T05:17:36.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.500+0000 7f50615ff700 1 -- 172.21.15.31:0/2161362027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f505c0cb770 con 0x7f505c0c8240 2023-10-06T05:17:36.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.501+0000 7f5042ffd700 1 -- 172.21.15.31:0/2161362027 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f50540558f0 con 0x7f505c0c8240 2023-10-06T05:17:36.503 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:17:36.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.503+0000 7f50615ff700 1 -- 172.21.15.31:0/2161362027 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5048066f30 msgr2=0x7f50480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.503+0000 7f50615ff700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5048066f30 0x7f50480693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f504c0011d0 tx=0x7f504c009040 comp rx=0 tx=0).stop 2023-10-06T05:17:36.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.503+0000 7f50615ff700 1 -- 172.21.15.31:0/2161362027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505c0c8240 msgr2=0x7f505c0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.503+0000 7f50615ff700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505c0c8240 0x7f505c0c86b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f505c15ad00 tx=0x7f505400b040 comp rx=0 tx=0).stop 2023-10-06T05:17:36.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.503+0000 7f50615ff700 1 -- 172.21.15.31:0/2161362027 shutdown_connections 2023-10-06T05:17:36.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.503+0000 7f50615ff700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5048066f30 0x7f50480693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.503+0000 7f50615ff700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505c0c8240 0x7f505c0c86b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.503+0000 7f50615ff700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5044003fe0 0x7f505c068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.503+0000 7f50615ff700 1 --2- 172.21.15.31:0/2161362027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f505c151910 0x7f505c151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.503+0000 7f50615ff700 1 -- 172.21.15.31:0/2161362027 >> 172.21.15.31:0/2161362027 conn(0x7f505c0996f0 msgr2=0x7f505c0ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:36.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.504+0000 7f50615ff700 1 -- 172.21.15.31:0/2161362027 shutdown_connections 2023-10-06T05:17:36.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.504+0000 7f50615ff700 1 -- 172.21.15.31:0/2161362027 wait complete. 2023-10-06T05:17:36.523 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.720+0000 7f3ef1dff700 1 Processor -- start 2023-10-06T05:17:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.720+0000 7f3ef1dff700 1 -- start start 2023-10-06T05:17:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.721+0000 7f3ef1dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3eec159560 0x7f3eec0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.721+0000 7f3ef1dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3eec0b2b80 0x7f3eec0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.721+0000 7f3ef1dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec15d2d0 con 0x7f3eec159930 2023-10-06T05:17:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.721+0000 7f3ef1dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eec15d5b0 con 0x7f3eec0b2b80 2023-10-06T05:17:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.721+0000 7f3ef1dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eec15d890 con 0x7f3eec159560 2023-10-06T05:17:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.722+0000 7f3eebfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3eec0b2b80 0x7f3eec0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.722+0000 7f3eebfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3eec0b2b80 0x7f3eec0cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53982/0 (socket says 172.21.15.31:53982) 2023-10-06T05:17:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.722+0000 7f3eebfff700 1 -- 172.21.15.31:0/3283031413 learned_addr learned my addr 172.21.15.31:0/3283031413 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.722+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/3283031413 >> v2:172.21.15.62:3300/0 conn(0x7f3eec159560 0x7f3eec0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.722+0000 7f3eebfff700 1 -- 172.21.15.31:0/3283031413 >> v2:172.21.15.62:3300/0 conn(0x7f3eec159560 msgr2=0x7f3eec0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:36.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.722+0000 7f3eebfff700 1 --2- 172.21.15.31:0/3283031413 >> v2:172.21.15.62:3300/0 conn(0x7f3eec159560 0x7f3eec0cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.722+0000 7f3eebfff700 1 -- 172.21.15.31:0/3283031413 >> v1:172.21.15.62:6789/0 conn(0x7f3eec159930 legacy=0x7f3eec0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.722+0000 7f3eebfff700 1 -- 172.21.15.31:0/3283031413 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec15db70 con 0x7f3eec0b2b80 2023-10-06T05:17:36.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.722+0000 7f3eebfff700 1 --2- 172.21.15.31:0/3283031413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0b2b80 0x7f3eec0cd3b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3eec063d80 tx=0x7f3ee4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953e49a283b1d51a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3eea7fc700 1 -- 172.21.15.31:0/3283031413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400e070 con 0x7f3eec0b2b80 2023-10-06T05:17:36.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3eea7fc700 1 -- 172.21.15.31:0/3283031413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee400ba50 con 0x7f3eec0b2b80 2023-10-06T05:17:36.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3eea7fc700 1 -- 172.21.15.31:0/3283031413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40092e0 con 0x7f3eec0b2b80 2023-10-06T05:17:36.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3ef1dff700 1 -- 172.21.15.31:0/3283031413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0b2b80 msgr2=0x7f3eec0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3ef1dff700 1 --2- 172.21.15.31:0/3283031413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0b2b80 0x7f3eec0cd3b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3eec063d80 tx=0x7f3ee4009840 comp rx=0 tx=0).stop 2023-10-06T05:17:36.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3ef1dff700 1 -- 172.21.15.31:0/3283031413 shutdown_connections 2023-10-06T05:17:36.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3ef1dff700 1 --2- 172.21.15.31:0/3283031413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0b2b80 0x7f3eec0cd3b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3ef1dff700 1 --2- 172.21.15.31:0/3283031413 >> v2:172.21.15.62:3300/0 conn(0x7f3eec159560 0x7f3eec0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3ef1dff700 1 -- 172.21.15.31:0/3283031413 >> 172.21.15.31:0/3283031413 conn(0x7f3eec06ec90 msgr2=0x7f3eec06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3ef1dff700 1 -- 172.21.15.31:0/3283031413 shutdown_connections 2023-10-06T05:17:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.723+0000 7f3ef1dff700 1 -- 172.21.15.31:0/3283031413 wait complete. 2023-10-06T05:17:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.725+0000 7f3ef1dff700 1 Processor -- start 2023-10-06T05:17:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.725+0000 7f3ef1dff700 1 -- start start 2023-10-06T05:17:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3ef1dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3eec0b2b80 0x7f3eec0c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3ef1dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eec159560 0x7f3eec0c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3ef1dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec159930 0x7f3eec16a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3ef1dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0c8de0 con 0x7f3eec159930 2023-10-06T05:17:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3ef1dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0c5930 con 0x7f3eec159560 2023-10-06T05:17:36.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3ef1dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec16ab90 con 0x7f3eec0b2b80 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec159930 0x7f3eec16a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec159930 0x7f3eec16a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53992/0 (socket says 172.21.15.31:53992) 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3eebfff700 1 -- 172.21.15.31:0/2758704000 learned_addr learned my addr 172.21.15.31:0/2758704000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3eec0b2b80 0x7f3eec0c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.726+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eec159560 0x7f3eec0c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3eebfff700 1 -- 172.21.15.31:0/2758704000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3eec0b2b80 msgr2=0x7f3eec0c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3eebfff700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3eec0b2b80 0x7f3eec0c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3eebfff700 1 -- 172.21.15.31:0/2758704000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eec159560 msgr2=0x7f3eec0c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3eebfff700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eec159560 0x7f3eec0c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3eebfff700 1 -- 172.21.15.31:0/2758704000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4009ae0 con 0x7f3eec159930 2023-10-06T05:17:36.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eec159560 0x7f3eec0c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3eebfff700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec159930 0x7f3eec16a090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0893d0 tx=0x7f3ee4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3ed37fe700 1 -- 172.21.15.31:0/2758704000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400be60 con 0x7f3eec159930 2023-10-06T05:17:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3ed37fe700 1 -- 172.21.15.31:0/2758704000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee4008dd0 con 0x7f3eec159930 2023-10-06T05:17:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3ed37fe700 1 -- 172.21.15.31:0/2758704000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40266b0 con 0x7f3eec159930 2023-10-06T05:17:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3ef1dff700 1 -- 172.21.15.31:0/2758704000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec16ae70 con 0x7f3eec159930 2023-10-06T05:17:36.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.727+0000 7f3ef1dff700 1 -- 172.21.15.31:0/2758704000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec0c8a00 con 0x7f3eec159930 2023-10-06T05:17:36.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.728+0000 7f3ef1dff700 1 -- 172.21.15.31:0/2758704000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec16b0b0 con 0x7f3eec159930 2023-10-06T05:17:36.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.729+0000 7f3ed37fe700 1 -- 172.21.15.31:0/2758704000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ee4003040 con 0x7f3eec159930 2023-10-06T05:17:36.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.730+0000 7f3ed37fe700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ed8066f30 0x7f3ed80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:36.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.730+0000 7f3ed37fe700 1 -- 172.21.15.31:0/2758704000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3ee40866b0 con 0x7f3eec159930 2023-10-06T05:17:36.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.730+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ed8066f30 0x7f3ed80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:36.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.731+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ed8066f30 0x7f3ed80693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3edc006fd0 tx=0x7f3edc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:36.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.735+0000 7f3ed37fe700 1 -- 172.21.15.31:0/2758704000 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ee4053720 con 0x7f3eec159930 2023-10-06T05:17:36.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.937+0000 7f3ef1dff700 1 -- 172.21.15.31:0/2758704000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3eec15de50 con 0x7f3eec159930 2023-10-06T05:17:36.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.937+0000 7f3ed37fe700 1 -- 172.21.15.31:0/2758704000 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ee4009940 con 0x7f3eec159930 2023-10-06T05:17:36.940 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:17:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 -- 172.21.15.31:0/2758704000 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ed8066f30 msgr2=0x7f3ed80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ed8066f30 0x7f3ed80693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3edc006fd0 tx=0x7f3edc008040 comp rx=0 tx=0).stop 2023-10-06T05:17:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 -- 172.21.15.31:0/2758704000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec159930 msgr2=0x7f3eec16a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec159930 0x7f3eec16a090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0893d0 tx=0x7f3ee4018000 comp rx=0 tx=0).stop 2023-10-06T05:17:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 -- 172.21.15.31:0/2758704000 shutdown_connections 2023-10-06T05:17:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ed8066f30 0x7f3ed80693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec159930 0x7f3eec16a090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3eec0b2b80 0x7f3eec0c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 --2- 172.21.15.31:0/2758704000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3eec159560 0x7f3eec0c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 -- 172.21.15.31:0/2758704000 >> 172.21.15.31:0/2758704000 conn(0x7f3eec06ec90 msgr2=0x7f3eec0b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 -- 172.21.15.31:0/2758704000 shutdown_connections 2023-10-06T05:17:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:36.940+0000 7f3ef1dff700 1 -- 172.21.15.31:0/2758704000 wait complete. 2023-10-06T05:17:36.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:17:37.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.156+0000 7fdd567ff700 1 Processor -- start 2023-10-06T05:17:37.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.156+0000 7fdd567ff700 1 -- start start 2023-10-06T05:17:37.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd567ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd50159930 0x7fdd500b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd567ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500cd580 0x7fdd500b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd567ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5015d2d0 con 0x7fdd50159560 2023-10-06T05:17:37.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd567ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5015d5b0 con 0x7fdd50159930 2023-10-06T05:17:37.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd567ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5015d890 con 0x7fdd500cd580 2023-10-06T05:17:37.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd4ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdd50159560 0x7fdd500cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49254/0 (socket says 172.21.15.31:49254) 2023-10-06T05:17:37.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd4ffff700 1 -- 172.21.15.31:0/1313485905 learned_addr learned my addr 172.21.15.31:0/1313485905 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:37.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd54d9c700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500cd580 0x7fdd500b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd4f7fe700 1 --2- 172.21.15.31:0/1313485905 >> v2:172.21.15.62:3300/0 conn(0x7fdd50159930 0x7fdd500b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103800243 0 0) 0x7fdd5015d2d0 con 0x7fdd50159560 2023-10-06T05:17:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.158+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd38003570 con 0x7fdd50159560 2023-10-06T05:17:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1416180404 0 0) 0x7fdd38003570 con 0x7fdd50159560 2023-10-06T05:17:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500cd580 msgr2=0x7fdd500b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500cd580 0x7fdd500b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 >> v2:172.21.15.62:3300/0 conn(0x7fdd50159930 msgr2=0x7fdd500b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 --2- 172.21.15.31:0/1313485905 >> v2:172.21.15.62:3300/0 conn(0x7fdd50159930 0x7fdd500b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5015db70 con 0x7fdd50159560 2023-10-06T05:17:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fdd40001470 con 0x7fdd50159560 2023-10-06T05:17:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 >> v1:172.21.15.31:6789/0 conn(0x7fdd50159560 legacy=0x7fdd500cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500cd580 0x7fdd38003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd380040b0 0x7fdd38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd38004d50 0x7fdd380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5015d5b0 con 0x7fdd38004d50 2023-10-06T05:17:37.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5015d890 con 0x7fdd380040b0 2023-10-06T05:17:37.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3800bb60 con 0x7fdd500cd580 2023-10-06T05:17:37.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd54d9c700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd38004d50 0x7fdd380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4f7fe700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd380040b0 0x7fdd38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.159+0000 7fdd4ffff700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500cd580 0x7fdd38003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.160+0000 7fdd54d9c700 1 -- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500cd580 msgr2=0x7fdd38003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.160+0000 7fdd54d9c700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500cd580 0x7fdd38003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.160+0000 7fdd54d9c700 1 -- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd380040b0 msgr2=0x7fdd38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.160+0000 7fdd54d9c700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd380040b0 0x7fdd38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.160+0000 7fdd54d9c700 1 -- 172.21.15.31:0/1313485905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38003570 con 0x7fdd38004d50 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.160+0000 7fdd54d9c700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd38004d50 0x7fdd380051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd480094d0 tx=0x7fdd4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb8c9497a6c9700 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.160+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800ea60 con 0x7fdd38004d50 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.160+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4800f040 con 0x7fdd38004d50 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.160+0000 7fdd4effd700 1 -- 172.21.15.31:0/1313485905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800edf0 con 0x7fdd38004d50 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.161+0000 7fdd567ff700 1 -- 172.21.15.31:0/1313485905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd38004d50 msgr2=0x7fdd380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.161+0000 7fdd567ff700 1 --2- 172.21.15.31:0/1313485905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd38004d50 0x7fdd380051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd480094d0 tx=0x7fdd4800b040 comp rx=0 tx=0).stop 2023-10-06T05:17:37.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.161+0000 7fdd54d9c700 1 -- 172.21.15.31:0/1313485905 reap_dead start 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.161+0000 7fdd567ff700 1 -- 172.21.15.31:0/1313485905 shutdown_connections 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.161+0000 7fdd567ff700 1 -- 172.21.15.31:0/1313485905 >> 172.21.15.31:0/1313485905 conn(0x7fdd5006ec90 msgr2=0x7fdd5006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.161+0000 7fdd567ff700 1 -- 172.21.15.31:0/1313485905 shutdown_connections 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.161+0000 7fdd567ff700 1 -- 172.21.15.31:0/1313485905 wait complete. 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.162+0000 7fdd567ff700 1 Processor -- start 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.162+0000 7fdd567ff700 1 -- start start 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.163+0000 7fdd567ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd50159d00 0x7fdd500c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.163+0000 7fdd567ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500c48a0 0x7fdd5015b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.163+0000 7fdd567ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd5015a0d0 0x7fdd5015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.163+0000 7fdd567ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500c5a00 con 0x7fdd50159d00 2023-10-06T05:17:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.163+0000 7fdd567ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500c4df0 con 0x7fdd5015a0d0 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.163+0000 7fdd567ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500c50d0 con 0x7fdd500c48a0 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.163+0000 7fdd4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd50159d00 0x7fdd500c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.163+0000 7fdd4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd50159d00 0x7fdd500c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57476/0 (socket says 172.21.15.31:57476) 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.163+0000 7fdd4ffff700 1 -- 172.21.15.31:0/3630438117 learned_addr learned my addr 172.21.15.31:0/3630438117 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd4f7fe700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500c48a0 0x7fdd5015b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd4ffff700 1 -- 172.21.15.31:0/3630438117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500c48a0 msgr2=0x7fdd5015b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd54d9c700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd5015a0d0 0x7fdd5015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd4ffff700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500c48a0 0x7fdd5015b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd4ffff700 1 -- 172.21.15.31:0/3630438117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd5015a0d0 msgr2=0x7fdd5015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd4ffff700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd5015a0d0 0x7fdd5015a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd4ffff700 1 -- 172.21.15.31:0/3630438117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd500c53b0 con 0x7fdd50159d00 2023-10-06T05:17:37.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd4f7fe700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500c48a0 0x7fdd5015b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:37.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd54d9c700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd5015a0d0 0x7fdd5015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:37.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd4ffff700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd50159d00 0x7fdd500c5e90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000eb00 tx=0x7fdd40000d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd37fff700 1 -- 172.21.15.31:0/3630438117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40001d40 con 0x7fdd50159d00 2023-10-06T05:17:37.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd37fff700 1 -- 172.21.15.31:0/3630438117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd40010040 con 0x7fdd50159d00 2023-10-06T05:17:37.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd37fff700 1 -- 172.21.15.31:0/3630438117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4000a9b0 con 0x7fdd50159d00 2023-10-06T05:17:37.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.164+0000 7fdd567ff700 1 -- 172.21.15.31:0/3630438117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd500c5690 con 0x7fdd50159d00 2023-10-06T05:17:37.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.165+0000 7fdd567ff700 1 -- 172.21.15.31:0/3630438117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5015aab0 con 0x7fdd50159d00 2023-10-06T05:17:37.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.166+0000 7fdd567ff700 1 -- 172.21.15.31:0/3630438117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5004c380 con 0x7fdd50159d00 2023-10-06T05:17:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.167+0000 7fdd37fff700 1 -- 172.21.15.31:0/3630438117 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd4000ad20 con 0x7fdd50159d00 2023-10-06T05:17:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.167+0000 7fdd37fff700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd3c066fb0 0x7fdd3c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.167+0000 7fdd37fff700 1 -- 172.21.15.31:0/3630438117 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd40083c60 con 0x7fdd50159d00 2023-10-06T05:17:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.168+0000 7fdd4f7fe700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd3c066fb0 0x7fdd3c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.168+0000 7fdd4f7fe700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd3c066fb0 0x7fdd3c069460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd44006c80 tx=0x7fdd44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.171+0000 7fdd37fff700 1 -- 172.21.15.31:0/3630438117 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd40050cd0 con 0x7fdd50159d00 2023-10-06T05:17:37.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.375+0000 7fdd567ff700 1 -- 172.21.15.31:0/3630438117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd500c5ce0 con 0x7fdd50159d00 2023-10-06T05:17:37.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.375+0000 7fdd37fff700 1 -- 172.21.15.31:0/3630438117 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd40019020 con 0x7fdd50159d00 2023-10-06T05:17:37.377 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:17:37.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 -- 172.21.15.31:0/3630438117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd3c066fb0 msgr2=0x7fdd3c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd3c066fb0 0x7fdd3c069460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd44006c80 tx=0x7fdd44008040 comp rx=0 tx=0).stop 2023-10-06T05:17:37.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 -- 172.21.15.31:0/3630438117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd50159d00 msgr2=0x7fdd500c5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd50159d00 0x7fdd500c5e90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000eb00 tx=0x7fdd40000d30 comp rx=0 tx=0).stop 2023-10-06T05:17:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 -- 172.21.15.31:0/3630438117 shutdown_connections 2023-10-06T05:17:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd5015a0d0 0x7fdd5015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd3c066fb0 0x7fdd3c069460 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd50159d00 0x7fdd500c5e90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 --2- 172.21.15.31:0/3630438117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd500c48a0 0x7fdd5015b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 -- 172.21.15.31:0/3630438117 >> 172.21.15.31:0/3630438117 conn(0x7fdd5006ec90 msgr2=0x7fdd500b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 -- 172.21.15.31:0/3630438117 shutdown_connections 2023-10-06T05:17:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.377+0000 7fdd567ff700 1 -- 172.21.15.31:0/3630438117 wait complete. 2023-10-06T05:17:37.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:17:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.592+0000 7f57e49ff700 1 Processor -- start 2023-10-06T05:17:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.592+0000 7f57e49ff700 1 -- start start 2023-10-06T05:17:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.593+0000 7f57e49ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f57e015ca40 0x7f57e0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.593+0000 7f57e49ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f57e00c8f40 0x7f57e00adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.593+0000 7f57e49ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e0068da0 con 0x7f57e006a300 2023-10-06T05:17:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.593+0000 7f57e49ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e00cabc0 con 0x7f57e015ca40 2023-10-06T05:17:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.593+0000 7f57e49ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e0068bb0 con 0x7f57e00c8f40 2023-10-06T05:17:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57de59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f57e015ca40 0x7f57e0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57ddd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f57e006a300 0x7f57e00c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49256/0 (socket says 172.21.15.31:49256) 2023-10-06T05:17:37.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57ddd9b700 1 -- 172.21.15.31:0/2583264293 learned_addr learned my addr 172.21.15.31:0/2583264293 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:37.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57ded9d700 1 --2- 172.21.15.31:0/2583264293 >> v2:172.21.15.62:3300/0 conn(0x7f57e00c8f40 0x7f57e00adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57de59c700 1 -- 172.21.15.31:0/2583264293 >> v2:172.21.15.62:3300/0 conn(0x7f57e00c8f40 msgr2=0x7f57e00adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:37.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57de59c700 1 --2- 172.21.15.31:0/2583264293 >> v2:172.21.15.62:3300/0 conn(0x7f57e00c8f40 0x7f57e00adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57de59c700 1 -- 172.21.15.31:0/2583264293 >> v1:172.21.15.31:6789/0 conn(0x7f57e006a300 legacy=0x7f57e00c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57de59c700 1 -- 172.21.15.31:0/2583264293 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0069080 con 0x7f57e015ca40 2023-10-06T05:17:37.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57ded9d700 1 --2- 172.21.15.31:0/2583264293 >> v2:172.21.15.62:3300/0 conn(0x7f57e00c8f40 0x7f57e00adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:37.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57dd59a700 1 -- 172.21.15.31:0/2583264293 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331313123 0 0) 0x7f57e0068da0 con 0x7f57e006a300 2023-10-06T05:17:37.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.594+0000 7f57de59c700 1 --2- 172.21.15.31:0/2583264293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e015ca40 0x7f57e0068670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f57e00c19b0 tx=0x7f57d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f951390121aa10cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57dd59a700 1 -- 172.21.15.31:0/2583264293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d0009e80 con 0x7f57e015ca40 2023-10-06T05:17:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57dd59a700 1 -- 172.21.15.31:0/2583264293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d000b820 con 0x7f57e015ca40 2023-10-06T05:17:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57dd59a700 1 -- 172.21.15.31:0/2583264293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d000bba0 con 0x7f57e015ca40 2023-10-06T05:17:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57e49ff700 1 -- 172.21.15.31:0/2583264293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e015ca40 msgr2=0x7f57e0068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57e49ff700 1 --2- 172.21.15.31:0/2583264293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e015ca40 0x7f57e0068670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f57e00c19b0 tx=0x7f57d00091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57e49ff700 1 -- 172.21.15.31:0/2583264293 shutdown_connections 2023-10-06T05:17:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57e49ff700 1 --2- 172.21.15.31:0/2583264293 >> v2:172.21.15.62:3300/0 conn(0x7f57e00c8f40 0x7f57e00adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57e49ff700 1 --2- 172.21.15.31:0/2583264293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e015ca40 0x7f57e0068670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57e49ff700 1 -- 172.21.15.31:0/2583264293 >> 172.21.15.31:0/2583264293 conn(0x7f57e00ce290 msgr2=0x7f57e00ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57e49ff700 1 -- 172.21.15.31:0/2583264293 shutdown_connections 2023-10-06T05:17:37.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.595+0000 7f57e49ff700 1 -- 172.21.15.31:0/2583264293 wait complete. 2023-10-06T05:17:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.597+0000 7f57e49ff700 1 Processor -- start 2023-10-06T05:17:37.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.598+0000 7f57e49ff700 1 -- start start 2023-10-06T05:17:37.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57e49ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e006a300 0x7f57e014c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57e49ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00c8f40 0x7f57e00b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57e49ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57e015ca40 0x7f57e00b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57e49ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00c3670 con 0x7f57e006a300 2023-10-06T05:17:37.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57e49ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00c2a60 con 0x7f57e00c8f40 2023-10-06T05:17:37.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57e49ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00c2d40 con 0x7f57e015ca40 2023-10-06T05:17:37.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57de59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e006a300 0x7f57e014c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57de59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e006a300 0x7f57e014c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57492/0 (socket says 172.21.15.31:57492) 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57de59c700 1 -- 172.21.15.31:0/248101316 learned_addr learned my addr 172.21.15.31:0/248101316 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57ddd9b700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00c8f40 0x7f57e00b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.599+0000 7f57ded9d700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57e015ca40 0x7f57e00b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57de59c700 1 -- 172.21.15.31:0/248101316 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57e015ca40 msgr2=0x7f57e00b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57de59c700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57e015ca40 0x7f57e00b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57de59c700 1 -- 172.21.15.31:0/248101316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00c8f40 msgr2=0x7f57e00b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57de59c700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00c8f40 0x7f57e00b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57de59c700 1 -- 172.21.15.31:0/248101316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d0009420 con 0x7f57e006a300 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57ddd9b700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00c8f40 0x7f57e00b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57de59c700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e006a300 0x7f57e014c810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f57e006e040 tx=0x7f57d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57ce7fc700 1 -- 172.21.15.31:0/248101316 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d000bba0 con 0x7f57e006a300 2023-10-06T05:17:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57ce7fc700 1 -- 172.21.15.31:0/248101316 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d0011420 con 0x7f57e006a300 2023-10-06T05:17:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57ce7fc700 1 -- 172.21.15.31:0/248101316 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d00117c0 con 0x7f57e006a300 2023-10-06T05:17:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57e49ff700 1 -- 172.21.15.31:0/248101316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e00c3020 con 0x7f57e006a300 2023-10-06T05:17:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.600+0000 7f57e49ff700 1 -- 172.21.15.31:0/248101316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e00c1cf0 con 0x7f57e006a300 2023-10-06T05:17:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.601+0000 7f57e49ff700 1 -- 172.21.15.31:0/248101316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e004c380 con 0x7f57e006a300 2023-10-06T05:17:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.602+0000 7f57ce7fc700 1 -- 172.21.15.31:0/248101316 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57d0011b30 con 0x7f57e006a300 2023-10-06T05:17:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.604+0000 7f57ce7fc700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57c0066f30 0x7f57c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.604+0000 7f57ce7fc700 1 -- 172.21.15.31:0/248101316 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f57d00860d0 con 0x7f57e006a300 2023-10-06T05:17:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.604+0000 7f57ddd9b700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57c0066f30 0x7f57c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.605+0000 7f57ddd9b700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57c0066f30 0x7f57c00693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f57e00b0cd0 tx=0x7f57d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:37.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.608+0000 7f57ce7fc700 1 -- 172.21.15.31:0/248101316 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57d0053140 con 0x7f57e006a300 2023-10-06T05:17:37.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.811+0000 7f57e49ff700 1 -- 172.21.15.31:0/248101316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57e00ca1a0 con 0x7f57e006a300 2023-10-06T05:17:37.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.812+0000 7f57ce7fc700 1 -- 172.21.15.31:0/248101316 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57d0056760 con 0x7f57e006a300 2023-10-06T05:17:37.815 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:17:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.814+0000 7f57e49ff700 1 -- 172.21.15.31:0/248101316 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57c0066f30 msgr2=0x7f57c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.814+0000 7f57e49ff700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57c0066f30 0x7f57c00693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f57e00b0cd0 tx=0x7f57d4009040 comp rx=0 tx=0).stop 2023-10-06T05:17:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.814+0000 7f57e49ff700 1 -- 172.21.15.31:0/248101316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e006a300 msgr2=0x7f57e014c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.814+0000 7f57e49ff700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e006a300 0x7f57e014c810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f57e006e040 tx=0x7f57d0017000 comp rx=0 tx=0).stop 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.814+0000 7f57e49ff700 1 -- 172.21.15.31:0/248101316 shutdown_connections 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.814+0000 7f57e49ff700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57e015ca40 0x7f57e00b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.814+0000 7f57e49ff700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57c0066f30 0x7f57c00693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.814+0000 7f57e49ff700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57e006a300 0x7f57e014c810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.814+0000 7f57e49ff700 1 --2- 172.21.15.31:0/248101316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00c8f40 0x7f57e00b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.814+0000 7f57e49ff700 1 -- 172.21.15.31:0/248101316 >> 172.21.15.31:0/248101316 conn(0x7f57e00ce290 msgr2=0x7f57e015a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.815+0000 7f57e49ff700 1 -- 172.21.15.31:0/248101316 shutdown_connections 2023-10-06T05:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:37.815+0000 7f57e49ff700 1 -- 172.21.15.31:0/248101316 wait complete. 2023-10-06T05:17:37.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:38.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.029+0000 7fddf2dff700 1 Processor -- start 2023-10-06T05:17:38.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.029+0000 7fddf2dff700 1 -- start start 2023-10-06T05:17:38.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.030+0000 7fddf2dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddec13a5b0 0x7fddec06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.030+0000 7fddf2dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddec06ddb0 con 0x7fddec13a1e0 2023-10-06T05:17:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.030+0000 7fddf2dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddec0c71b0 con 0x7fddec06d980 2023-10-06T05:17:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.030+0000 7fddf2dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddec0c7490 con 0x7fddec13a5b0 2023-10-06T05:17:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.031+0000 7fddebfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddec13a5b0 0x7fddec06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.031+0000 7fddf0b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fddec13a1e0 0x7fddec0c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49270/0 (socket says 172.21.15.31:49270) 2023-10-06T05:17:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.031+0000 7fddf0b9b700 1 -- 172.21.15.31:0/125608802 learned_addr learned my addr 172.21.15.31:0/125608802 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.031+0000 7fddebfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddec13a5b0 0x7fddec06d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57496/0 (socket says 172.21.15.31:57496) 2023-10-06T05:17:38.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.031+0000 7fddebfff700 1 -- 172.21.15.31:0/125608802 >> v1:172.21.15.62:6789/0 conn(0x7fddec06d980 legacy=0x7fddec069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.031+0000 7fddebfff700 1 -- 172.21.15.31:0/125608802 >> v1:172.21.15.31:6789/0 conn(0x7fddec13a1e0 legacy=0x7fddec0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.031+0000 7fddebfff700 1 -- 172.21.15.31:0/125608802 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec0c7770 con 0x7fddec13a5b0 2023-10-06T05:17:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.031+0000 7fddebfff700 1 --2- 172.21.15.31:0/125608802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a5b0 0x7fddec06d440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fddd80098e0 tx=0x7fddd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d988181f85c4bc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.032+0000 7fddeb7fe700 1 -- 172.21.15.31:0/125608802 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8009e80 con 0x7fddec13a5b0 2023-10-06T05:17:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.032+0000 7fddeb7fe700 1 -- 172.21.15.31:0/125608802 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd800b860 con 0x7fddec13a5b0 2023-10-06T05:17:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.032+0000 7fddeb7fe700 1 -- 172.21.15.31:0/125608802 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd800bbe0 con 0x7fddec13a5b0 2023-10-06T05:17:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.032+0000 7fddf2dff700 1 -- 172.21.15.31:0/125608802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a5b0 msgr2=0x7fddec06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.032+0000 7fddf2dff700 1 --2- 172.21.15.31:0/125608802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a5b0 0x7fddec06d440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fddd80098e0 tx=0x7fddd80091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.032+0000 7fddf2dff700 1 -- 172.21.15.31:0/125608802 shutdown_connections 2023-10-06T05:17:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.032+0000 7fddf2dff700 1 --2- 172.21.15.31:0/125608802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a5b0 0x7fddec06d440 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.032+0000 7fddf2dff700 1 -- 172.21.15.31:0/125608802 >> 172.21.15.31:0/125608802 conn(0x7fddec0cc0b0 msgr2=0x7fddec0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.032+0000 7fddf2dff700 1 -- 172.21.15.31:0/125608802 shutdown_connections 2023-10-06T05:17:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.032+0000 7fddf2dff700 1 -- 172.21.15.31:0/125608802 wait complete. 2023-10-06T05:17:38.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.034+0000 7fddf2dff700 1 Processor -- start 2023-10-06T05:17:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.034+0000 7fddf2dff700 1 -- start start 2023-10-06T05:17:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddf2dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddec06d980 0x7fddec14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddf2dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a1e0 0x7fddec0c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddf2dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddec13a5b0 0x7fddec152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddf2dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec1507d0 con 0x7fddec13a1e0 2023-10-06T05:17:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddf2dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec14fd00 con 0x7fddec06d980 2023-10-06T05:17:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddf2dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddec1522b0 con 0x7fddec13a5b0 2023-10-06T05:17:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a1e0 0x7fddec0c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a1e0 0x7fddec0c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57512/0 (socket says 172.21.15.31:57512) 2023-10-06T05:17:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddebfff700 1 -- 172.21.15.31:0/1133260461 learned_addr learned my addr 172.21.15.31:0/1133260461 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:38.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddf0b9b700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddec06d980 0x7fddec14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.035+0000 7fddf139c700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddec13a5b0 0x7fddec152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fddebfff700 1 -- 172.21.15.31:0/1133260461 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddec13a5b0 msgr2=0x7fddec152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fddebfff700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddec13a5b0 0x7fddec152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fddebfff700 1 -- 172.21.15.31:0/1133260461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddec06d980 msgr2=0x7fddec14f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fddebfff700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddec06d980 0x7fddec14f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fddebfff700 1 -- 172.21.15.31:0/1133260461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd8009420 con 0x7fddec13a1e0 2023-10-06T05:17:38.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fddf0b9b700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddec06d980 0x7fddec14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:38.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fddebfff700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a1e0 0x7fddec0c68d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fddd8013000 tx=0x7fddd8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fdde8ff9700 1 -- 172.21.15.31:0/1133260461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd800bb20 con 0x7fddec13a1e0 2023-10-06T05:17:38.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fdde8ff9700 1 -- 172.21.15.31:0/1133260461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd8025420 con 0x7fddec13a1e0 2023-10-06T05:17:38.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fddf2dff700 1 -- 172.21.15.31:0/1133260461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddec14ff30 con 0x7fddec13a1e0 2023-10-06T05:17:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fdde8ff9700 1 -- 172.21.15.31:0/1133260461 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd80257c0 con 0x7fddec13a1e0 2023-10-06T05:17:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.036+0000 7fddf2dff700 1 -- 172.21.15.31:0/1133260461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddec1504f0 con 0x7fddec13a1e0 2023-10-06T05:17:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.037+0000 7fddf2dff700 1 -- 172.21.15.31:0/1133260461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddec152040 con 0x7fddec13a1e0 2023-10-06T05:17:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.038+0000 7fdde8ff9700 1 -- 172.21.15.31:0/1133260461 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fddd8023070 con 0x7fddec13a1e0 2023-10-06T05:17:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.039+0000 7fdde8ff9700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fddd0066f30 0x7fddd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.039+0000 7fdde8ff9700 1 -- 172.21.15.31:0/1133260461 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fddd8085d50 con 0x7fddec13a1e0 2023-10-06T05:17:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.039+0000 7fddf0b9b700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fddd0066f30 0x7fddd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.040+0000 7fddf0b9b700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fddd0066f30 0x7fddd00693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdde00010e0 tx=0x7fdde0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.044+0000 7fdde8ff9700 1 -- 172.21.15.31:0/1133260461 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddd8052dc0 con 0x7fddec13a1e0 2023-10-06T05:17:38.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.253+0000 7fddf2dff700 1 -- 172.21.15.31:0/1133260461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddec152440 con 0x7fddec13a1e0 2023-10-06T05:17:38.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.254+0000 7fdde8ff9700 1 -- 172.21.15.31:0/1133260461 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fddd80563e0 con 0x7fddec13a1e0 2023-10-06T05:17:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:38.256 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:38.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 -- 172.21.15.31:0/1133260461 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fddd0066f30 msgr2=0x7fddd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fddd0066f30 0x7fddd00693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdde00010e0 tx=0x7fdde0009040 comp rx=0 tx=0).stop 2023-10-06T05:17:38.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 -- 172.21.15.31:0/1133260461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a1e0 msgr2=0x7fddec0c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a1e0 0x7fddec0c68d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fddd8013000 tx=0x7fddd8017000 comp rx=0 tx=0).stop 2023-10-06T05:17:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 -- 172.21.15.31:0/1133260461 shutdown_connections 2023-10-06T05:17:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddec13a5b0 0x7fddec152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddec06d980 0x7fddec14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fddd0066f30 0x7fddd00693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 --2- 172.21.15.31:0/1133260461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec13a1e0 0x7fddec0c68d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 -- 172.21.15.31:0/1133260461 >> 172.21.15.31:0/1133260461 conn(0x7fddec0cc0b0 msgr2=0x7fddec0cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 -- 172.21.15.31:0/1133260461 shutdown_connections 2023-10-06T05:17:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.256+0000 7fddf2dff700 1 -- 172.21.15.31:0/1133260461 wait complete. 2023-10-06T05:17:38.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:16:59.507957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:17:00.905826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:17:38.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:38.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.472+0000 7fb4df1ff700 1 Processor -- start 2023-10-06T05:17:38.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.472+0000 7fb4df1ff700 1 -- start start 2023-10-06T05:17:38.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.473+0000 7fb4df1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb4d80c8f40 0x7fb4d8068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.473+0000 7fb4df1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4d806a300 0x7fb4d80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.473+0000 7fb4df1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d8068da0 con 0x7fb4d815ca40 2023-10-06T05:17:38.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.473+0000 7fb4df1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d80cabc0 con 0x7fb4d806a300 2023-10-06T05:17:38.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.473+0000 7fb4df1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d8068bb0 con 0x7fb4d80c8f40 2023-10-06T05:17:38.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.473+0000 7fb4dd79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4d806a300 0x7fb4d80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.473+0000 7fb4d7fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb4d80c8f40 0x7fb4d8068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.474+0000 7fb4dd79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4d806a300 0x7fb4d80adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57520/0 (socket says 172.21.15.31:57520) 2023-10-06T05:17:38.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.474+0000 7fb4dd79c700 1 -- 172.21.15.31:0/2847807965 learned_addr learned my addr 172.21.15.31:0/2847807965 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:38.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.474+0000 7fb4dcf9b700 1 --1- 172.21.15.31:0/2847807965 >> v1:172.21.15.62:6789/0 conn(0x7fb4d815ca40 0x7fb4d80c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:39982/0 (socket says 172.21.15.31:39982) 2023-10-06T05:17:38.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.474+0000 7fb4dd79c700 1 -- 172.21.15.31:0/2847807965 >> v2:172.21.15.62:3300/0 conn(0x7fb4d80c8f40 msgr2=0x7fb4d8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.474+0000 7fb4dd79c700 1 --2- 172.21.15.31:0/2847807965 >> v2:172.21.15.62:3300/0 conn(0x7fb4d80c8f40 0x7fb4d8068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.474+0000 7fb4dd79c700 1 -- 172.21.15.31:0/2847807965 >> v1:172.21.15.62:6789/0 conn(0x7fb4d815ca40 legacy=0x7fb4d80c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.474+0000 7fb4dd79c700 1 -- 172.21.15.31:0/2847807965 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d8069080 con 0x7fb4d806a300 2023-10-06T05:17:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.474+0000 7fb4dd79c700 1 --2- 172.21.15.31:0/2847807965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d806a300 0x7fb4d80adde0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb4d806e950 tx=0x7fb4d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4773dfb372f8d447 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.474+0000 7fb4d77fe700 1 -- 172.21.15.31:0/2847807965 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d000e070 con 0x7fb4d806a300 2023-10-06T05:17:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.474+0000 7fb4d77fe700 1 -- 172.21.15.31:0/2847807965 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d000ba20 con 0x7fb4d806a300 2023-10-06T05:17:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.475+0000 7fb4d77fe700 1 -- 172.21.15.31:0/2847807965 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00092e0 con 0x7fb4d806a300 2023-10-06T05:17:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.475+0000 7fb4df1ff700 1 -- 172.21.15.31:0/2847807965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d806a300 msgr2=0x7fb4d80adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.475+0000 7fb4df1ff700 1 --2- 172.21.15.31:0/2847807965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d806a300 0x7fb4d80adde0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb4d806e950 tx=0x7fb4d0009840 comp rx=0 tx=0).stop 2023-10-06T05:17:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.475+0000 7fb4df1ff700 1 -- 172.21.15.31:0/2847807965 shutdown_connections 2023-10-06T05:17:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.475+0000 7fb4df1ff700 1 --2- 172.21.15.31:0/2847807965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d806a300 0x7fb4d80adde0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.475+0000 7fb4df1ff700 1 --2- 172.21.15.31:0/2847807965 >> v2:172.21.15.62:3300/0 conn(0x7fb4d80c8f40 0x7fb4d8068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.475+0000 7fb4df1ff700 1 -- 172.21.15.31:0/2847807965 >> 172.21.15.31:0/2847807965 conn(0x7fb4d80ce290 msgr2=0x7fb4d80ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.475+0000 7fb4df1ff700 1 -- 172.21.15.31:0/2847807965 shutdown_connections 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.475+0000 7fb4df1ff700 1 -- 172.21.15.31:0/2847807965 wait complete. 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.477+0000 7fb4df1ff700 1 Processor -- start 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.477+0000 7fb4df1ff700 1 -- start start 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4df1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d806a300 0x7fb4d8159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4df1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d80c8f40 0x7fb4d80cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4df1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4d815ca40 0x7fb4d80cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4df1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d80c6650 con 0x7fb4d80c8f40 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4df1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d80c7490 con 0x7fb4d806a300 2023-10-06T05:17:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4df1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d80c6880 con 0x7fb4d815ca40 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d80c8f40 0x7fb4d80cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d80c8f40 0x7fb4d80cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57532/0 (socket says 172.21.15.31:57532) 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4d7fff700 1 -- 172.21.15.31:0/831835721 learned_addr learned my addr 172.21.15.31:0/831835721 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4dcf9b700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d806a300 0x7fb4d8159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4dd79c700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4d815ca40 0x7fb4d80cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4d7fff700 1 -- 172.21.15.31:0/831835721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4d815ca40 msgr2=0x7fb4d80cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4d7fff700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4d815ca40 0x7fb4d80cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4d7fff700 1 -- 172.21.15.31:0/831835721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d806a300 msgr2=0x7fb4d8159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4d7fff700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d806a300 0x7fb4d8159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.478+0000 7fb4d7fff700 1 -- 172.21.15.31:0/831835721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d0009ae0 con 0x7fb4d80c8f40 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.479+0000 7fb4d7fff700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d80c8f40 0x7fb4d80cbdc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400ec80 tx=0x7fb4c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.479+0000 7fb4d4ff9700 1 -- 172.21.15.31:0/831835721 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c400c850 con 0x7fb4d80c8f40 2023-10-06T05:17:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.479+0000 7fb4d4ff9700 1 -- 172.21.15.31:0/831835721 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c400c9b0 con 0x7fb4d80c8f40 2023-10-06T05:17:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.479+0000 7fb4df1ff700 1 -- 172.21.15.31:0/831835721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d80c6b60 con 0x7fb4d80c8f40 2023-10-06T05:17:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.479+0000 7fb4d4ff9700 1 -- 172.21.15.31:0/831835721 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c400cd50 con 0x7fb4d80c8f40 2023-10-06T05:17:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.479+0000 7fb4df1ff700 1 -- 172.21.15.31:0/831835721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d80c7120 con 0x7fb4d80c8f40 2023-10-06T05:17:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.480+0000 7fb4df1ff700 1 -- 172.21.15.31:0/831835721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d80c7600 con 0x7fb4d80c8f40 2023-10-06T05:17:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.481+0000 7fb4d4ff9700 1 -- 172.21.15.31:0/831835721 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4c4013680 con 0x7fb4d80c8f40 2023-10-06T05:17:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.482+0000 7fb4d4ff9700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb4bc066f30 0x7fb4bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.482+0000 7fb4d4ff9700 1 -- 172.21.15.31:0/831835721 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb4c4084920 con 0x7fb4d80c8f40 2023-10-06T05:17:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.482+0000 7fb4dcf9b700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb4bc066f30 0x7fb4bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.483+0000 7fb4dcf9b700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb4bc066f30 0x7fb4bc0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc004fd0 tx=0x7fb4cc007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.486+0000 7fb4d4ff9700 1 -- 172.21.15.31:0/831835721 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4c4051990 con 0x7fb4d80c8f40 2023-10-06T05:17:38.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.689+0000 7fb4df1ff700 1 -- 172.21.15.31:0/831835721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb4d804c380 con 0x7fb4d80c8f40 2023-10-06T05:17:38.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.689+0000 7fb4d4ff9700 1 -- 172.21.15.31:0/831835721 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb4c4054fb0 con 0x7fb4d80c8f40 2023-10-06T05:17:38.691 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:17:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 -- 172.21.15.31:0/831835721 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb4bc066f30 msgr2=0x7fb4bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb4bc066f30 0x7fb4bc0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc004fd0 tx=0x7fb4cc007eb0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 -- 172.21.15.31:0/831835721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d80c8f40 msgr2=0x7fb4d80cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d80c8f40 0x7fb4d80cbdc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400ec80 tx=0x7fb4c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 -- 172.21.15.31:0/831835721 shutdown_connections 2023-10-06T05:17:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4d815ca40 0x7fb4d80cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d806a300 0x7fb4d8159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb4bc066f30 0x7fb4bc0693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 --2- 172.21.15.31:0/831835721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d80c8f40 0x7fb4d80cbdc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 -- 172.21.15.31:0/831835721 >> 172.21.15.31:0/831835721 conn(0x7fb4d80ce290 msgr2=0x7fb4d80b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 -- 172.21.15.31:0/831835721 shutdown_connections 2023-10-06T05:17:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.691+0000 7fb4df1ff700 1 -- 172.21.15.31:0/831835721 wait complete. 2023-10-06T05:17:38.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:38.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.906+0000 7f06f3fff700 1 Processor -- start 2023-10-06T05:17:38.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.906+0000 7f06f3fff700 1 -- start start 2023-10-06T05:17:38.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.907+0000 7f06f3fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f06ec15ca40 0x7f06ec068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.907+0000 7f06f3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06ec06a300 0x7f06ec0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.907+0000 7f06f3fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ec068da0 con 0x7f06ec0c8d70 2023-10-06T05:17:38.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.907+0000 7f06f3fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ec0cabc0 con 0x7f06ec15ca40 2023-10-06T05:17:38.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.907+0000 7f06f3fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec068bb0 con 0x7f06ec06a300 2023-10-06T05:17:38.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.908+0000 7f06f259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f06ec0c8d70 0x7f06ec06cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49272/0 (socket says 172.21.15.31:49272) 2023-10-06T05:17:38.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.908+0000 7f06f259c700 1 -- 172.21.15.31:0/3590739263 learned_addr learned my addr 172.21.15.31:0/3590739263 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:38.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.908+0000 7f06f1d9b700 1 --2- 172.21.15.31:0/3590739263 >> v2:172.21.15.62:3300/0 conn(0x7f06ec15ca40 0x7f06ec068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.908+0000 7f06f159a700 1 --2- 172.21.15.31:0/3590739263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06ec06a300 0x7f06ec0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.908+0000 7f06f0d99700 1 -- 172.21.15.31:0/3590739263 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096395996 0 0) 0x7f06ec068da0 con 0x7f06ec0c8d70 2023-10-06T05:17:38.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.908+0000 7f06f0d99700 1 -- 172.21.15.31:0/3590739263 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06d8003570 con 0x7f06ec0c8d70 2023-10-06T05:17:38.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.908+0000 7f06f1d9b700 1 -- 172.21.15.31:0/3590739263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06ec06a300 msgr2=0x7f06ec0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:38.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.908+0000 7f06f1d9b700 1 --2- 172.21.15.31:0/3590739263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06ec06a300 0x7f06ec0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.908+0000 7f06f1d9b700 1 -- 172.21.15.31:0/3590739263 >> v1:172.21.15.31:6789/0 conn(0x7f06ec0c8d70 legacy=0x7f06ec06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.908+0000 7f06f1d9b700 1 -- 172.21.15.31:0/3590739263 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec069080 con 0x7f06ec15ca40 2023-10-06T05:17:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.909+0000 7f06f1d9b700 1 --2- 172.21.15.31:0/3590739263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06ec15ca40 0x7f06ec068670 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06e80098e0 tx=0x7f06e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c2444cc11b781a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.909+0000 7f06f0d99700 1 -- 172.21.15.31:0/3590739263 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8011630 con 0x7f06ec15ca40 2023-10-06T05:17:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.909+0000 7f06f0d99700 1 -- 172.21.15.31:0/3590739263 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8011790 con 0x7f06ec15ca40 2023-10-06T05:17:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.909+0000 7f06f0d99700 1 -- 172.21.15.31:0/3590739263 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8011b30 con 0x7f06ec15ca40 2023-10-06T05:17:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.909+0000 7f06f3fff700 1 -- 172.21.15.31:0/3590739263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06ec15ca40 msgr2=0x7f06ec068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.909+0000 7f06f3fff700 1 --2- 172.21.15.31:0/3590739263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06ec15ca40 0x7f06ec068670 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06e80098e0 tx=0x7f06e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.910+0000 7f06f3fff700 1 -- 172.21.15.31:0/3590739263 shutdown_connections 2023-10-06T05:17:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.910+0000 7f06f3fff700 1 --2- 172.21.15.31:0/3590739263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06ec15ca40 0x7f06ec068670 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.910+0000 7f06f3fff700 1 --2- 172.21.15.31:0/3590739263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06ec06a300 0x7f06ec0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.910+0000 7f06f3fff700 1 -- 172.21.15.31:0/3590739263 >> 172.21.15.31:0/3590739263 conn(0x7f06ec0ce290 msgr2=0x7f06ec0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.910+0000 7f06f3fff700 1 -- 172.21.15.31:0/3590739263 shutdown_connections 2023-10-06T05:17:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.910+0000 7f06f3fff700 1 -- 172.21.15.31:0/3590739263 wait complete. 2023-10-06T05:17:38.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.912+0000 7f06f3fff700 1 Processor -- start 2023-10-06T05:17:38.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.912+0000 7f06f3fff700 1 -- start start 2023-10-06T05:17:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.913+0000 7f06f3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06ec06a300 0x7f06ec1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.913+0000 7f06f3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06ec0c8d70 0x7f06ec0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.913+0000 7f06f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06ec15ca40 0x7f06ec0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.913+0000 7f06f3fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec0a98b0 con 0x7f06ec15ca40 2023-10-06T05:17:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.913+0000 7f06f3fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec1695d0 con 0x7f06ec0c8d70 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.913+0000 7f06f3fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec16a400 con 0x7f06ec06a300 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.913+0000 7f06f259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06ec15ca40 0x7f06ec0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06ec15ca40 0x7f06ec0cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57544/0 (socket says 172.21.15.31:57544) 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f259c700 1 -- 172.21.15.31:0/3013759923 learned_addr learned my addr 172.21.15.31:0/3013759923 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f159a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06ec0c8d70 0x7f06ec0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f1d9b700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06ec06a300 0x7f06ec1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f259c700 1 -- 172.21.15.31:0/3013759923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06ec06a300 msgr2=0x7f06ec1588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f259c700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06ec06a300 0x7f06ec1588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f259c700 1 -- 172.21.15.31:0/3013759923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06ec0c8d70 msgr2=0x7f06ec0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f259c700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06ec0c8d70 0x7f06ec0cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f259c700 1 -- 172.21.15.31:0/3013759923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e8009420 con 0x7f06ec15ca40 2023-10-06T05:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f259c700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06ec15ca40 0x7f06ec0cc370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f06e400cca0 tx=0x7f06e4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06e1ffb700 1 -- 172.21.15.31:0/3013759923 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e400a960 con 0x7f06ec15ca40 2023-10-06T05:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06e1ffb700 1 -- 172.21.15.31:0/3013759923 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e400aac0 con 0x7f06ec15ca40 2023-10-06T05:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.914+0000 7f06f3fff700 1 -- 172.21.15.31:0/3013759923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ec158e10 con 0x7f06ec15ca40 2023-10-06T05:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.915+0000 7f06e1ffb700 1 -- 172.21.15.31:0/3013759923 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e400ae40 con 0x7f06ec15ca40 2023-10-06T05:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.915+0000 7f06f3fff700 1 -- 172.21.15.31:0/3013759923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ec169ad0 con 0x7f06ec15ca40 2023-10-06T05:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.916+0000 7f06f3fff700 1 -- 172.21.15.31:0/3013759923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ec04c380 con 0x7f06ec15ca40 2023-10-06T05:17:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.917+0000 7f06e1ffb700 1 -- 172.21.15.31:0/3013759923 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06e4007300 con 0x7f06ec15ca40 2023-10-06T05:17:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.917+0000 7f06e1ffb700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06d0066f60 0x7f06d0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.918+0000 7f06e1ffb700 1 -- 172.21.15.31:0/3013759923 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f06e4084b80 con 0x7f06ec15ca40 2023-10-06T05:17:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.918+0000 7f06f1d9b700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06d0066f60 0x7f06d0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:38.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.918+0000 7f06f1d9b700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06d0066f60 0x7f06d0069410 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f06e80091e0 tx=0x7f06e8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:38.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:38.921+0000 7f06e1ffb700 1 -- 172.21.15.31:0/3013759923 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06e4051be0 con 0x7f06ec15ca40 2023-10-06T05:17:39.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.124+0000 7f06f3fff700 1 -- 172.21.15.31:0/3013759923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06ec16a540 con 0x7f06ec15ca40 2023-10-06T05:17:39.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.125+0000 7f06e1ffb700 1 -- 172.21.15.31:0/3013759923 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06e4084e60 con 0x7f06ec15ca40 2023-10-06T05:17:39.127 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:17:39.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 -- 172.21.15.31:0/3013759923 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06d0066f60 msgr2=0x7f06d0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06d0066f60 0x7f06d0069410 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f06e80091e0 tx=0x7f06e8017000 comp rx=0 tx=0).stop 2023-10-06T05:17:39.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 -- 172.21.15.31:0/3013759923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06ec15ca40 msgr2=0x7f06ec0cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06ec15ca40 0x7f06ec0cc370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f06e400cca0 tx=0x7f06e4001ae0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 -- 172.21.15.31:0/3013759923 shutdown_connections 2023-10-06T05:17:39.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f06d0066f60 0x7f06d0069410 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06ec15ca40 0x7f06ec0cc370 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06ec06a300 0x7f06ec1588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 --2- 172.21.15.31:0/3013759923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06ec0c8d70 0x7f06ec0cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 -- 172.21.15.31:0/3013759923 >> 172.21.15.31:0/3013759923 conn(0x7f06ec0ce290 msgr2=0x7f06ec15b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:39.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 -- 172.21.15.31:0/3013759923 shutdown_connections 2023-10-06T05:17:39.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.127+0000 7f06f3fff700 1 -- 172.21.15.31:0/3013759923 wait complete. 2023-10-06T05:17:39.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.343+0000 7fbcd2bff700 1 Processor -- start 2023-10-06T05:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.343+0000 7fbcd2bff700 1 -- start start 2023-10-06T05:17:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.344+0000 7fbcd2bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbccc0c7190 0x7fbccc0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbcd2bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc068250 con 0x7fbccc0aef00 2023-10-06T05:17:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbcd2bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc06ac80 con 0x7fbccc0c7560 2023-10-06T05:17:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbcd2bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc067f20 con 0x7fbccc0c7190 2023-10-06T05:17:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbccbfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbccc0aef00 0x7fbccc0af7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49284/0 (socket says 172.21.15.31:49284) 2023-10-06T05:17:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbccbfff700 1 -- 172.21.15.31:0/3425150383 learned_addr learned my addr 172.21.15.31:0/3425150383 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbcd099b700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbccc0c7190 0x7fbccc0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668534450 0 0) 0x7fbccc068250 con 0x7fbccc0aef00 2023-10-06T05:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcbc003570 con 0x7fbccc0aef00 2023-10-06T05:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898728155 0 0) 0x7fbccc06ac80 con 0x7fbccc0c7560 2023-10-06T05:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbccc068250 con 0x7fbccc0c7560 2023-10-06T05:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3085718806 0 0) 0x7fbcbc003570 con 0x7fbccc0aef00 2023-10-06T05:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbccc0c7190 msgr2=0x7fbccc0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.345+0000 7fbccb7fe700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbccc0c7190 0x7fbccc0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 >> v1:172.21.15.62:6789/0 conn(0x7fbccc0c7560 legacy=0x7fbccc0686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc06aed0 con 0x7fbccc0aef00 2023-10-06T05:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fbcb8001470 con 0x7fbccc0aef00 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 >> v1:172.21.15.31:6789/0 conn(0x7fbccc0aef00 legacy=0x7fbccc0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccb7fe700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbccc0c7190 0x7fbcbc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccb7fe700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc0041d0 0x7fbcbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccb7fe700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc004d50 0x7fbcbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc067f20 con 0x7fbcbc004d50 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc00bbf0 con 0x7fbcbc0041d0 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc00bdb0 con 0x7fbccc0c7190 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbcd119c700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc004d50 0x7fbcbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbcd099b700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbccc0c7190 0x7fbcbc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbccbfff700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc0041d0 0x7fbcbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbcd119c700 1 -- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbccc0c7190 msgr2=0x7fbcbc003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbcd119c700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbccc0c7190 0x7fbcbc003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbcd119c700 1 -- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc0041d0 msgr2=0x7fbcbc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbcd119c700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcbc0041d0 0x7fbcbc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.346+0000 7fbcd119c700 1 -- 172.21.15.31:0/3425150383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc003570 con 0x7fbcbc004d50 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.347+0000 7fbcd119c700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc004d50 0x7fbcbc0051c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4002120 tx=0x7fbcc40017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5eb75deb38e995 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.347+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4001ab0 con 0x7fbcbc004d50 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.347+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc400c980 con 0x7fbcbc004d50 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.347+0000 7fbccb7fe700 1 -- 172.21.15.31:0/3425150383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400cd20 con 0x7fbcbc004d50 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.347+0000 7fbcd2bff700 1 -- 172.21.15.31:0/3425150383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc004d50 msgr2=0x7fbcbc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.347+0000 7fbcd2bff700 1 --2- 172.21.15.31:0/3425150383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc004d50 0x7fbcbc0051c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4002120 tx=0x7fbcc40017a0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.347+0000 7fbcd119c700 1 -- 172.21.15.31:0/3425150383 reap_dead start 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.348+0000 7fbcd2bff700 1 -- 172.21.15.31:0/3425150383 shutdown_connections 2023-10-06T05:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.348+0000 7fbcd2bff700 1 -- 172.21.15.31:0/3425150383 >> 172.21.15.31:0/3425150383 conn(0x7fbccc0cd3d0 msgr2=0x7fbccc0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:39.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.348+0000 7fbcd2bff700 1 -- 172.21.15.31:0/3425150383 shutdown_connections 2023-10-06T05:17:39.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.348+0000 7fbcd2bff700 1 -- 172.21.15.31:0/3425150383 wait complete. 2023-10-06T05:17:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.349+0000 7fbcd2bff700 1 Processor -- start 2023-10-06T05:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.349+0000 7fbcd2bff700 1 -- start start 2023-10-06T05:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.350+0000 7fbcd2bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcbc004d50 0x7fbccc0ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.350+0000 7fbcd2bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbccc0aa270 0x7fbccc0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.350+0000 7fbcd2bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbccc0ad050 0x7fbccc0ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.350+0000 7fbcd2bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0ada00 con 0x7fbccc0ad050 2023-10-06T05:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.350+0000 7fbcd2bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0aaad0 con 0x7fbccc0aa270 2023-10-06T05:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.350+0000 7fbcd2bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0aadb0 con 0x7fbcbc004d50 2023-10-06T05:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.350+0000 7fbcd119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbccc0ad050 0x7fbccc0ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.350+0000 7fbcd119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbccc0ad050 0x7fbccc0ad4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57566/0 (socket says 172.21.15.31:57566) 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcd119c700 1 -- 172.21.15.31:0/1435193138 learned_addr learned my addr 172.21.15.31:0/1435193138 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcd099b700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcbc004d50 0x7fbccc0ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbccbfff700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbccc0aa270 0x7fbccc0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcd119c700 1 -- 172.21.15.31:0/1435193138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcbc004d50 msgr2=0x7fbccc0ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcd119c700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcbc004d50 0x7fbccc0ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcd119c700 1 -- 172.21.15.31:0/1435193138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbccc0aa270 msgr2=0x7fbccc0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcd119c700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbccc0aa270 0x7fbccc0acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcd119c700 1 -- 172.21.15.31:0/1435193138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0ab090 con 0x7fbccc0ad050 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcd099b700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcbc004d50 0x7fbccc0ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbccbfff700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbccc0aa270 0x7fbccc0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcd119c700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbccc0ad050 0x7fbccc0ad4c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0057f0 tx=0x7fbcc4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcc8ff9700 1 -- 172.21.15.31:0/1435193138 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4001ab0 con 0x7fbccc0ad050 2023-10-06T05:17:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcc8ff9700 1 -- 172.21.15.31:0/1435193138 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc400c6d0 con 0x7fbccc0ad050 2023-10-06T05:17:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcc8ff9700 1 -- 172.21.15.31:0/1435193138 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4006db0 con 0x7fbccc0ad050 2023-10-06T05:17:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.351+0000 7fbcd2bff700 1 -- 172.21.15.31:0/1435193138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc0ab370 con 0x7fbccc0ad050 2023-10-06T05:17:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.352+0000 7fbcd2bff700 1 -- 172.21.15.31:0/1435193138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc0aa6e0 con 0x7fbccc0ad050 2023-10-06T05:17:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.353+0000 7fbcd2bff700 1 -- 172.21.15.31:0/1435193138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc0aa870 con 0x7fbccc0ad050 2023-10-06T05:17:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.354+0000 7fbcc8ff9700 1 -- 172.21.15.31:0/1435193138 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbcc4003030 con 0x7fbccc0ad050 2023-10-06T05:17:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.354+0000 7fbcc8ff9700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbcb0067030 0x7fbcb00694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.354+0000 7fbcc8ff9700 1 -- 172.21.15.31:0/1435193138 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbcc40866e0 con 0x7fbccc0ad050 2023-10-06T05:17:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.355+0000 7fbcd099b700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbcb0067030 0x7fbcb00694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.355+0000 7fbcd099b700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbcb0067030 0x7fbcb00694e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00093f0 tx=0x7fbcc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:39.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.359+0000 7fbcc8ff9700 1 -- 172.21.15.31:0/1435193138 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcc4053870 con 0x7fbccc0ad050 2023-10-06T05:17:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.566+0000 7fbcd2bff700 1 -- 172.21.15.31:0/1435193138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbccc04c380 con 0x7fbccc0ad050 2023-10-06T05:17:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.566+0000 7fbcc8ff9700 1 -- 172.21.15.31:0/1435193138 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbcc4056e90 con 0x7fbccc0ad050 2023-10-06T05:17:39.568 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 -- 172.21.15.31:0/1435193138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbcb0067030 msgr2=0x7fbcb00694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbcb0067030 0x7fbcb00694e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00093f0 tx=0x7fbcc0008040 comp rx=0 tx=0).stop 2023-10-06T05:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 -- 172.21.15.31:0/1435193138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbccc0ad050 msgr2=0x7fbccc0ad4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbccc0ad050 0x7fbccc0ad4c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0057f0 tx=0x7fbcc4016000 comp rx=0 tx=0).stop 2023-10-06T05:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 -- 172.21.15.31:0/1435193138 shutdown_connections 2023-10-06T05:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbcb0067030 0x7fbcb00694e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbccc0ad050 0x7fbccc0ad4c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcbc004d50 0x7fbccc0ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 --2- 172.21.15.31:0/1435193138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbccc0aa270 0x7fbccc0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 -- 172.21.15.31:0/1435193138 >> 172.21.15.31:0/1435193138 conn(0x7fbccc0cd3d0 msgr2=0x7fbccc0afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 -- 172.21.15.31:0/1435193138 shutdown_connections 2023-10-06T05:17:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.569+0000 7fbcd2bff700 1 -- 172.21.15.31:0/1435193138 wait complete. 2023-10-06T05:17:39.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:17:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.785+0000 7ff488fff700 1 Processor -- start 2023-10-06T05:17:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.785+0000 7ff488fff700 1 -- start start 2023-10-06T05:17:39.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.786+0000 7ff488fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff4840cd580 0x7ff4841645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.786+0000 7ff488fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48415d2d0 con 0x7ff484159930 2023-10-06T05:17:39.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.786+0000 7ff488fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48415d5b0 con 0x7ff484159560 2023-10-06T05:17:39.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.786+0000 7ff488fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48415d890 con 0x7ff4840cd580 2023-10-06T05:17:39.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.786+0000 7ff482d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff4840cd580 0x7ff4841645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.786+0000 7ff481d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff484159930 0x7ff4840b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49292/0 (socket says 172.21.15.31:49292) 2023-10-06T05:17:39.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.786+0000 7ff482d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff4840cd580 0x7ff4841645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57574/0 (socket says 172.21.15.31:57574) 2023-10-06T05:17:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.786+0000 7ff481d9b700 1 -- 172.21.15.31:0/2385303896 learned_addr learned my addr 172.21.15.31:0/2385303896 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.787+0000 7ff482d9d700 1 -- 172.21.15.31:0/2385303896 >> v1:172.21.15.62:6789/0 conn(0x7ff484159560 legacy=0x7ff4840cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.787+0000 7ff482d9d700 1 -- 172.21.15.31:0/2385303896 >> v1:172.21.15.31:6789/0 conn(0x7ff484159930 legacy=0x7ff4840b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.787+0000 7ff482d9d700 1 -- 172.21.15.31:0/2385303896 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48415db70 con 0x7ff4840cd580 2023-10-06T05:17:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.787+0000 7ff48159a700 1 -- 172.21.15.31:0/2385303896 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662635124 0 0) 0x7ff48415d2d0 con 0x7ff484159930 2023-10-06T05:17:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.787+0000 7ff482d9d700 1 --2- 172.21.15.31:0/2385303896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4840cd580 0x7ff4841645d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009e60 tx=0x7ff47c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e94bb6472e2f992 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.787+0000 7ff48159a700 1 -- 172.21.15.31:0/2385303896 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00e070 con 0x7ff4840cd580 2023-10-06T05:17:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.787+0000 7ff48159a700 1 -- 172.21.15.31:0/2385303896 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c00bc30 con 0x7ff4840cd580 2023-10-06T05:17:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.787+0000 7ff48159a700 1 -- 172.21.15.31:0/2385303896 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c012430 con 0x7ff4840cd580 2023-10-06T05:17:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.788+0000 7ff488fff700 1 -- 172.21.15.31:0/2385303896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4840cd580 msgr2=0x7ff4841645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.788+0000 7ff488fff700 1 --2- 172.21.15.31:0/2385303896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4840cd580 0x7ff4841645d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009e60 tx=0x7ff47c009780 comp rx=0 tx=0).stop 2023-10-06T05:17:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.788+0000 7ff488fff700 1 -- 172.21.15.31:0/2385303896 shutdown_connections 2023-10-06T05:17:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.788+0000 7ff488fff700 1 --2- 172.21.15.31:0/2385303896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4840cd580 0x7ff4841645d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.788+0000 7ff488fff700 1 -- 172.21.15.31:0/2385303896 >> 172.21.15.31:0/2385303896 conn(0x7ff48406ec90 msgr2=0x7ff48406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.788+0000 7ff488fff700 1 -- 172.21.15.31:0/2385303896 shutdown_connections 2023-10-06T05:17:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.788+0000 7ff488fff700 1 -- 172.21.15.31:0/2385303896 wait complete. 2023-10-06T05:17:39.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.790+0000 7ff488fff700 1 Processor -- start 2023-10-06T05:17:39.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.790+0000 7ff488fff700 1 -- start start 2023-10-06T05:17:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff488fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4840cd580 0x7ff48415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff488fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff484159560 0x7ff48415e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff488fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484159930 0x7ff48415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff488fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48415f2b0 con 0x7ff484159930 2023-10-06T05:17:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff488fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c4b00 con 0x7ff4840cd580 2023-10-06T05:17:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff488fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c3ef0 con 0x7ff484159560 2023-10-06T05:17:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff482d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484159930 0x7ff48415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff482d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484159930 0x7ff48415ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57582/0 (socket says 172.21.15.31:57582) 2023-10-06T05:17:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff482d9d700 1 -- 172.21.15.31:0/1199347041 learned_addr learned my addr 172.21.15.31:0/1199347041 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff481d9b700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff484159560 0x7ff48415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff48259c700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4840cd580 0x7ff48415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff482d9d700 1 -- 172.21.15.31:0/1199347041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff484159560 msgr2=0x7ff48415e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff482d9d700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff484159560 0x7ff48415e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff482d9d700 1 -- 172.21.15.31:0/1199347041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4840cd580 msgr2=0x7ff48415fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff482d9d700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4840cd580 0x7ff48415fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.791+0000 7ff482d9d700 1 -- 172.21.15.31:0/1199347041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c009a20 con 0x7ff484159930 2023-10-06T05:17:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.792+0000 7ff482d9d700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484159930 0x7ff48415ed70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff47c013000 tx=0x7ff47c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.792+0000 7ff4727fc700 1 -- 172.21.15.31:0/1199347041 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c01b630 con 0x7ff484159930 2023-10-06T05:17:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.792+0000 7ff4727fc700 1 -- 172.21.15.31:0/1199347041 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c01b790 con 0x7ff484159930 2023-10-06T05:17:39.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.792+0000 7ff488fff700 1 -- 172.21.15.31:0/1199347041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4840c41d0 con 0x7ff484159930 2023-10-06T05:17:39.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.792+0000 7ff4727fc700 1 -- 172.21.15.31:0/1199347041 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c01bb30 con 0x7ff484159930 2023-10-06T05:17:39.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.792+0000 7ff488fff700 1 -- 172.21.15.31:0/1199347041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4840c4810 con 0x7ff484159930 2023-10-06T05:17:39.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.793+0000 7ff488fff700 1 -- 172.21.15.31:0/1199347041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4840c4c90 con 0x7ff484159930 2023-10-06T05:17:39.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.794+0000 7ff4727fc700 1 -- 172.21.15.31:0/1199347041 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff47c01c400 con 0x7ff484159930 2023-10-06T05:17:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.795+0000 7ff4727fc700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff464066f30 0x7ff4640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.795+0000 7ff4727fc700 1 -- 172.21.15.31:0/1199347041 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff47c086670 con 0x7ff484159930 2023-10-06T05:17:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.795+0000 7ff48259c700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff464066f30 0x7ff4640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.796+0000 7ff48259c700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff464066f30 0x7ff4640693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff474003860 tx=0x7ff474009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:39.799+0000 7ff4727fc700 1 -- 172.21.15.31:0/1199347041 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff47c0536e0 con 0x7ff484159930 2023-10-06T05:17:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.002+0000 7ff488fff700 1 -- 172.21.15.31:0/1199347041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff48404c380 con 0x7ff484159930 2023-10-06T05:17:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.003+0000 7ff4727fc700 1 -- 172.21.15.31:0/1199347041 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff47c056d00 con 0x7ff484159930 2023-10-06T05:17:40.005 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:17:40.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.005+0000 7ff488fff700 1 -- 172.21.15.31:0/1199347041 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff464066f30 msgr2=0x7ff4640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.005+0000 7ff488fff700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff464066f30 0x7ff4640693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff474003860 tx=0x7ff474009040 comp rx=0 tx=0).stop 2023-10-06T05:17:40.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.005+0000 7ff488fff700 1 -- 172.21.15.31:0/1199347041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484159930 msgr2=0x7ff48415ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.005+0000 7ff488fff700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484159930 0x7ff48415ed70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff47c013000 tx=0x7ff47c019000 comp rx=0 tx=0).stop 2023-10-06T05:17:40.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.005+0000 7ff488fff700 1 -- 172.21.15.31:0/1199347041 shutdown_connections 2023-10-06T05:17:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.005+0000 7ff488fff700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff464066f30 0x7ff4640693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.005+0000 7ff488fff700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484159930 0x7ff48415ed70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.005+0000 7ff488fff700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4840cd580 0x7ff48415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.005+0000 7ff488fff700 1 --2- 172.21.15.31:0/1199347041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff484159560 0x7ff48415e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.006+0000 7ff488fff700 1 -- 172.21.15.31:0/1199347041 >> 172.21.15.31:0/1199347041 conn(0x7ff48406ec90 msgr2=0x7ff4840b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.006+0000 7ff488fff700 1 -- 172.21.15.31:0/1199347041 shutdown_connections 2023-10-06T05:17:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.006+0000 7ff488fff700 1 -- 172.21.15.31:0/1199347041 wait complete. 2023-10-06T05:17:40.092 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2023-10-06T05:17:40.092 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:17:40.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-10-06T05:17:40.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.289+0000 7f803a5ff700 1 Processor -- start 2023-10-06T05:17:40.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.289+0000 7f803a5ff700 1 -- start start 2023-10-06T05:17:40.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.290+0000 7f803a5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80340cd510 0x7f80340b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f803a5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f803415d2d0 0x7f80340aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f803a5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340c3f40 con 0x7f8034159500 2023-10-06T05:17:40.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f803a5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80340b2860 con 0x7f80340cd510 2023-10-06T05:17:40.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f803a5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803415d7d0 con 0x7f803415d2d0 2023-10-06T05:17:40.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f80337fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80340cd510 0x7f80340b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f80337fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80340cd510 0x7f80340b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57590/0 (socket says 172.21.15.31:57590) 2023-10-06T05:17:40.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f80337fe700 1 -- 172.21.15.31:0/2209962102 learned_addr learned my addr 172.21.15.31:0/2209962102 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:40.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f8038b9c700 1 --2- 172.21.15.31:0/2209962102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f803415d2d0 0x7f80340aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f80337fe700 1 -- 172.21.15.31:0/2209962102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f803415d2d0 msgr2=0x7f80340aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:40.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f80337fe700 1 --2- 172.21.15.31:0/2209962102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f803415d2d0 0x7f80340aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f80337fe700 1 -- 172.21.15.31:0/2209962102 >> v1:172.21.15.62:6789/0 conn(0x7f8034159500 legacy=0x7f80340cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.291+0000 7f80337fe700 1 -- 172.21.15.31:0/2209962102 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340cb000 con 0x7f80340cd510 2023-10-06T05:17:40.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.292+0000 7f80337fe700 1 --2- 172.21.15.31:0/2209962102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80340cd510 0x7f80340b0ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f80280099a0 tx=0x7f8028009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf2fc123f3d24e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.292+0000 7f8032ffd700 1 -- 172.21.15.31:0/2209962102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028009e90 con 0x7f80340cd510 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.292+0000 7f8032ffd700 1 -- 172.21.15.31:0/2209962102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802800b860 con 0x7f80340cd510 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.292+0000 7f8032ffd700 1 -- 172.21.15.31:0/2209962102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802800bc20 con 0x7f80340cd510 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.292+0000 7f803a5ff700 1 -- 172.21.15.31:0/2209962102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80340cd510 msgr2=0x7f80340b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.292+0000 7f803a5ff700 1 --2- 172.21.15.31:0/2209962102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80340cd510 0x7f80340b0ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f80280099a0 tx=0x7f8028009250 comp rx=0 tx=0).stop 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.293+0000 7f803a5ff700 1 -- 172.21.15.31:0/2209962102 shutdown_connections 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.293+0000 7f803a5ff700 1 --2- 172.21.15.31:0/2209962102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f803415d2d0 0x7f80340aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.293+0000 7f803a5ff700 1 --2- 172.21.15.31:0/2209962102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80340cd510 0x7f80340b0ab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.293+0000 7f803a5ff700 1 -- 172.21.15.31:0/2209962102 >> 172.21.15.31:0/2209962102 conn(0x7f803406ec90 msgr2=0x7f803406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.293+0000 7f803a5ff700 1 -- 172.21.15.31:0/2209962102 shutdown_connections 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.293+0000 7f803a5ff700 1 -- 172.21.15.31:0/2209962102 wait complete. 2023-10-06T05:17:40.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.294+0000 7f803a5ff700 1 Processor -- start 2023-10-06T05:17:40.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.294+0000 7f803a5ff700 1 -- start start 2023-10-06T05:17:40.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.295+0000 7f803a5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80340cd510 0x7f80340b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.295+0000 7f803a5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8034159500 0x7f803416a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.295+0000 7f803a5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f803415d2d0 0x7f803416d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.295+0000 7f803a5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340b49b0 con 0x7f8034159500 2023-10-06T05:17:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.295+0000 7f803a5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340b4110 con 0x7f803415d2d0 2023-10-06T05:17:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.295+0000 7f803a5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80340b43f0 con 0x7f80340cd510 2023-10-06T05:17:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f80337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8034159500 0x7f803416a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f80337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8034159500 0x7f803416a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57592/0 (socket says 172.21.15.31:57592) 2023-10-06T05:17:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f80337fe700 1 -- 172.21.15.31:0/1688873599 learned_addr learned my addr 172.21.15.31:0/1688873599 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f8033fff700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80340cd510 0x7f80340b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f8038b9c700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f803415d2d0 0x7f803416d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f80337fe700 1 -- 172.21.15.31:0/1688873599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80340cd510 msgr2=0x7f80340b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f80337fe700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80340cd510 0x7f80340b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f80337fe700 1 -- 172.21.15.31:0/1688873599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f803415d2d0 msgr2=0x7f803416d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:40.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f80337fe700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f803415d2d0 0x7f803416d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:40.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f80337fe700 1 -- 172.21.15.31:0/1688873599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028009490 con 0x7f8034159500 2023-10-06T05:17:40.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.296+0000 7f80337fe700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8034159500 0x7f803416a450 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8028009290 tx=0x7f8028017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.297+0000 7f801bfff700 1 -- 172.21.15.31:0/1688873599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802801f070 con 0x7f8034159500 2023-10-06T05:17:40.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.297+0000 7f801bfff700 1 -- 172.21.15.31:0/1688873599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8028000bc0 con 0x7f8034159500 2023-10-06T05:17:40.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.297+0000 7f801bfff700 1 -- 172.21.15.31:0/1688873599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802800bc00 con 0x7f8034159500 2023-10-06T05:17:40.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.297+0000 7f803a5ff700 1 -- 172.21.15.31:0/1688873599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80340b46d0 con 0x7f8034159500 2023-10-06T05:17:40.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.297+0000 7f803a5ff700 1 -- 172.21.15.31:0/1688873599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80340c7680 con 0x7f8034159500 2023-10-06T05:17:40.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.298+0000 7f803a5ff700 1 -- 172.21.15.31:0/1688873599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8034169d90 con 0x7f8034159500 2023-10-06T05:17:40.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.299+0000 7f801bfff700 1 -- 172.21.15.31:0/1688873599 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8028023070 con 0x7f8034159500 2023-10-06T05:17:40.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.299+0000 7f801bfff700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8020066f30 0x7f80200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:40.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.300+0000 7f801bfff700 1 -- 172.21.15.31:0/1688873599 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8028085770 con 0x7f8034159500 2023-10-06T05:17:40.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.300+0000 7f8033fff700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8020066f30 0x7f80200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:40.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.300+0000 7f8033fff700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8020066f30 0x7f80200693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8024003860 tx=0x7f8024009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:40.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.304+0000 7f801bfff700 1 -- 172.21.15.31:0/1688873599 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8034169d90 con 0x7f8034159500 2023-10-06T05:17:40.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:40.585+0000 7f803a5ff700 1 -- 172.21.15.31:0/1688873599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f8034169d90 con 0x7f8034159500 2023-10-06T05:17:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.164+0000 7f801bfff700 1 -- 172.21.15.31:0/1688873599 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8034169d90 con 0x7f8034159500 2023-10-06T05:17:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 -- 172.21.15.31:0/1688873599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8020066f30 msgr2=0x7f80200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8020066f30 0x7f80200693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8024003860 tx=0x7f8024009040 comp rx=0 tx=0).stop 2023-10-06T05:17:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 -- 172.21.15.31:0/1688873599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8034159500 msgr2=0x7f803416a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8034159500 0x7f803416a450 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8028009290 tx=0x7f8028017000 comp rx=0 tx=0).stop 2023-10-06T05:17:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 -- 172.21.15.31:0/1688873599 shutdown_connections 2023-10-06T05:17:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f803415d2d0 0x7f803416d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80340cd510 0x7f80340b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8020066f30 0x7f80200693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 --2- 172.21.15.31:0/1688873599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8034159500 0x7f803416a450 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 -- 172.21.15.31:0/1688873599 >> 172.21.15.31:0/1688873599 conn(0x7f803406ec90 msgr2=0x7f80340af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 -- 172.21.15.31:0/1688873599 shutdown_connections 2023-10-06T05:17:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.167+0000 7f803a5ff700 1 -- 172.21.15.31:0/1688873599 wait complete. 2023-10-06T05:17:41.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:17:41.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.390+0000 7f42c0fff700 1 Processor -- start 2023-10-06T05:17:41.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.390+0000 7f42c0fff700 1 -- start start 2023-10-06T05:17:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.391+0000 7f42c0fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42bc0c7190 0x7f42bc0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.391+0000 7f42c0fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f42bc0aef00 0x7f42bc0af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.391+0000 7f42c0fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42bc0afcf0 0x7f42bc0686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.391+0000 7f42c0fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc068250 con 0x7f42bc0afcf0 2023-10-06T05:17:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.391+0000 7f42c0fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc067f20 con 0x7f42bc0aef00 2023-10-06T05:17:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.391+0000 7f42c0fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc06aed0 con 0x7f42bc0c7190 2023-10-06T05:17:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42bad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42bc0afcf0 0x7f42bc0686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42b9d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f42bc0aef00 0x7f42bc0af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42bad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42bc0afcf0 0x7f42bc0686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57602/0 (socket says 172.21.15.31:57602) 2023-10-06T05:17:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42bad9d700 1 -- 172.21.15.31:0/2007920134 learned_addr learned my addr 172.21.15.31:0/2007920134 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42ba59c700 1 --2- 172.21.15.31:0/2007920134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42bc0c7190 0x7f42bc0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42bad9d700 1 -- 172.21.15.31:0/2007920134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42bc0c7190 msgr2=0x7f42bc0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42bad9d700 1 --2- 172.21.15.31:0/2007920134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42bc0c7190 0x7f42bc0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42bad9d700 1 -- 172.21.15.31:0/2007920134 >> v2:172.21.15.62:3300/0 conn(0x7f42bc0aef00 msgr2=0x7f42bc0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42bad9d700 1 --2- 172.21.15.31:0/2007920134 >> v2:172.21.15.62:3300/0 conn(0x7f42bc0aef00 0x7f42bc0af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42bad9d700 1 -- 172.21.15.31:0/2007920134 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc06b1b0 con 0x7f42bc0afcf0 2023-10-06T05:17:41.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42bad9d700 1 --2- 172.21.15.31:0/2007920134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0afcf0 0x7f42bc0686e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f42bc06d4b0 tx=0x7f42b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6d81f2ad2a183 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:41.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.392+0000 7f42b959a700 1 -- 172.21.15.31:0/2007920134 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b400e070 con 0x7f42bc0afcf0 2023-10-06T05:17:41.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42b959a700 1 -- 172.21.15.31:0/2007920134 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b400b970 con 0x7f42bc0afcf0 2023-10-06T05:17:41.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42b959a700 1 -- 172.21.15.31:0/2007920134 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b40092e0 con 0x7f42bc0afcf0 2023-10-06T05:17:41.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42c0fff700 1 -- 172.21.15.31:0/2007920134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0afcf0 msgr2=0x7f42bc0686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42c0fff700 1 --2- 172.21.15.31:0/2007920134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0afcf0 0x7f42bc0686e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f42bc06d4b0 tx=0x7f42b4009840 comp rx=0 tx=0).stop 2023-10-06T05:17:41.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42c0fff700 1 -- 172.21.15.31:0/2007920134 shutdown_connections 2023-10-06T05:17:41.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42c0fff700 1 --2- 172.21.15.31:0/2007920134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0afcf0 0x7f42bc0686e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42c0fff700 1 --2- 172.21.15.31:0/2007920134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42bc0c7190 0x7f42bc0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42c0fff700 1 --2- 172.21.15.31:0/2007920134 >> v2:172.21.15.62:3300/0 conn(0x7f42bc0aef00 0x7f42bc0af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42c0fff700 1 -- 172.21.15.31:0/2007920134 >> 172.21.15.31:0/2007920134 conn(0x7f42bc0cd3d0 msgr2=0x7f42bc0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:41.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42c0fff700 1 -- 172.21.15.31:0/2007920134 shutdown_connections 2023-10-06T05:17:41.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.393+0000 7f42c0fff700 1 -- 172.21.15.31:0/2007920134 wait complete. 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.395+0000 7f42c0fff700 1 Processor -- start 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.396+0000 7f42c0fff700 1 -- start start 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42c0fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0aef00 0x7f42bc0c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42c0fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc0afcf0 0x7f42bc16a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42c0fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42bc0c7190 0x7f42bc151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42c0fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0c4070 con 0x7f42bc0aef00 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42c0fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0c4350 con 0x7f42bc0afcf0 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42c0fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0c4630 con 0x7f42bc0c7190 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42ba59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0aef00 0x7f42bc0c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42ba59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0aef00 0x7f42bc0c39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57604/0 (socket says 172.21.15.31:57604) 2023-10-06T05:17:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42ba59c700 1 -- 172.21.15.31:0/1081745583 learned_addr learned my addr 172.21.15.31:0/1081745583 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42b9d9b700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc0afcf0 0x7f42bc16a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42bad9d700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42bc0c7190 0x7f42bc151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42ba59c700 1 -- 172.21.15.31:0/1081745583 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42bc0c7190 msgr2=0x7f42bc151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42ba59c700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42bc0c7190 0x7f42bc151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42ba59c700 1 -- 172.21.15.31:0/1081745583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc0afcf0 msgr2=0x7f42bc16a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42ba59c700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc0afcf0 0x7f42bc16a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42ba59c700 1 -- 172.21.15.31:0/1081745583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b4009ae0 con 0x7f42bc0aef00 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.397+0000 7f42b9d9b700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc0afcf0 0x7f42bc16a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.398+0000 7f42ba59c700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0aef00 0x7f42bc0c39b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00ec80 tx=0x7f42ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.398+0000 7f42aa7fc700 1 -- 172.21.15.31:0/1081745583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00c9d0 con 0x7f42bc0aef00 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.398+0000 7f42aa7fc700 1 -- 172.21.15.31:0/1081745583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac00cb50 con 0x7f42bc0aef00 2023-10-06T05:17:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.398+0000 7f42c0fff700 1 -- 172.21.15.31:0/1081745583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc0c4910 con 0x7f42bc0aef00 2023-10-06T05:17:41.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.398+0000 7f42aa7fc700 1 -- 172.21.15.31:0/1081745583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac013470 con 0x7f42bc0aef00 2023-10-06T05:17:41.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.398+0000 7f42c0fff700 1 -- 172.21.15.31:0/1081745583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc1524d0 con 0x7f42bc0aef00 2023-10-06T05:17:41.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.399+0000 7f42c0fff700 1 -- 172.21.15.31:0/1081745583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc06ed60 con 0x7f42bc0aef00 2023-10-06T05:17:41.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.400+0000 7f42aa7fc700 1 -- 172.21.15.31:0/1081745583 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42ac013860 con 0x7f42bc0aef00 2023-10-06T05:17:41.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.401+0000 7f42aa7fc700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f429c066f60 0x7f429c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:41.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.401+0000 7f42aa7fc700 1 -- 172.21.15.31:0/1081745583 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f42ac084a70 con 0x7f42bc0aef00 2023-10-06T05:17:41.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.402+0000 7f42b9d9b700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f429c066f60 0x7f429c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:41.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.402+0000 7f42b9d9b700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f429c066f60 0x7f429c069410 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f42bc15d4b0 tx=0x7f42b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:41.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.405+0000 7f42aa7fc700 1 -- 172.21.15.31:0/1081745583 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42ac051ae0 con 0x7f42bc0aef00 2023-10-06T05:17:41.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.613+0000 7f42c0fff700 1 -- 172.21.15.31:0/1081745583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f42bc04c380 con 0x7f42bc0aef00 2023-10-06T05:17:41.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.617+0000 7f42aa7fc700 1 -- 172.21.15.31:0/1081745583 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f42ac055100 con 0x7f42bc0aef00 2023-10-06T05:17:41.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 -- 172.21.15.31:0/1081745583 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f429c066f60 msgr2=0x7f429c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f429c066f60 0x7f429c069410 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f42bc15d4b0 tx=0x7f42b0008040 comp rx=0 tx=0).stop 2023-10-06T05:17:41.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 -- 172.21.15.31:0/1081745583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0aef00 msgr2=0x7f42bc0c39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:41.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0aef00 0x7f42bc0c39b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00ec80 tx=0x7f42ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 -- 172.21.15.31:0/1081745583 shutdown_connections 2023-10-06T05:17:41.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42bc0c7190 0x7f42bc151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f429c066f60 0x7f429c069410 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42bc0aef00 0x7f42bc0c39b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 --2- 172.21.15.31:0/1081745583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc0afcf0 0x7f42bc16a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:41.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 -- 172.21.15.31:0/1081745583 >> 172.21.15.31:0/1081745583 conn(0x7f42bc0cd3d0 msgr2=0x7f42bc158820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:41.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 -- 172.21.15.31:0/1081745583 shutdown_connections 2023-10-06T05:17:41.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:41.619+0000 7f42c0fff700 1 -- 172.21.15.31:0/1081745583 wait complete. 2023-10-06T05:17:41.639 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:17:41.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:17:41.750 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:17:41.853 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:17:41.853 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:17:41.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:41.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T05:17:41.944 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:17:41.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:41.944 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:17:41.993 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:17:41.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:41.993 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-06T05:17:42.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:17:42.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:42.088 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:17:42.145 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-06T05:17:42.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:42.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-06T05:17:42.233 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:17:42.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:42.234 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T05:17:42.280 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-06T05:17:42.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:17:42.281 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-06T05:17:42.366 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:17:42.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:17:42.366 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T05:17:42.412 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:17:42.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:42.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.609+0000 7fae567ff700 1 Processor -- start 2023-10-06T05:17:42.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.609+0000 7fae567ff700 1 -- start start 2023-10-06T05:17:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.610+0000 7fae567ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fae5013fda0 0x7fae50069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.610+0000 7fae567ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae500ca5b0 con 0x7fae500c7f30 2023-10-06T05:17:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.610+0000 7fae567ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5006f740 con 0x7fae500c7b60 2023-10-06T05:17:42.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.610+0000 7fae567ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae500c9c80 con 0x7fae5013fda0 2023-10-06T05:17:42.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fae500c7f30 0x7fae5013f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49306/0 (socket says 172.21.15.31:49306) 2023-10-06T05:17:42.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4f7fe700 1 -- 172.21.15.31:0/841900070 learned_addr learned my addr 172.21.15.31:0/841900070 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:42.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae54d9c700 1 --2- 172.21.15.31:0/841900070 >> v2:172.21.15.62:3300/0 conn(0x7fae5013fda0 0x7fae50069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895035376 0 0) 0x7fae500ca5b0 con 0x7fae500c7f30 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae38003570 con 0x7fae500c7f30 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925674499 0 0) 0x7fae5006f740 con 0x7fae500c7b60 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae500ca5b0 con 0x7fae500c7b60 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2709544107 0 0) 0x7fae38003570 con 0x7fae500c7f30 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 >> v2:172.21.15.62:3300/0 conn(0x7fae5013fda0 msgr2=0x7fae50069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 --2- 172.21.15.31:0/841900070 >> v2:172.21.15.62:3300/0 conn(0x7fae5013fda0 0x7fae50069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 >> v1:172.21.15.62:6789/0 conn(0x7fae500c7b60 legacy=0x7fae5006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500c9f60 con 0x7fae500c7f30 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fae44001470 con 0x7fae500c7f30 2023-10-06T05:17:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.611+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 >> v1:172.21.15.31:6789/0 conn(0x7fae500c7f30 legacy=0x7fae5013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4effd700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae38003c40 0x7fae38004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4effd700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae38004810 0x7fae38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4effd700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38005210 0x7fae3800dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae500c9c80 con 0x7fae38003c40 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3800e340 con 0x7fae38004810 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3800e4d0 con 0x7fae38005210 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae54d9c700 1 --2- 172.21.15.31:0/841900070 >> v2:172.21.15.62:3300/0 conn(0x7fae5013fda0 0x7fae50069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4ffff700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae38003c40 0x7fae38004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4f7fe700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae38004810 0x7fae38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae54d9c700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38005210 0x7fae3800dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4ffff700 1 -- 172.21.15.31:0/841900070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38005210 msgr2=0x7fae3800dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:42.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4ffff700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38005210 0x7fae3800dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4ffff700 1 -- 172.21.15.31:0/841900070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae38004810 msgr2=0x7fae38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4ffff700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae38004810 0x7fae38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae4ffff700 1 -- 172.21.15.31:0/841900070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae38003570 con 0x7fae38003c40 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.612+0000 7fae54d9c700 1 -- 172.21.15.31:0/841900070 reap_dead start 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae4ffff700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae38003c40 0x7fae38004030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae40001a60 tx=0x7fae40007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdd3057a7af7e4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40001ea0 con 0x7fae38003c40 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4000c550 con 0x7fae38003c40 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae4effd700 1 -- 172.21.15.31:0/841900070 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40001420 con 0x7fae38003c40 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae567ff700 1 -- 172.21.15.31:0/841900070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae38003c40 msgr2=0x7fae38004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae567ff700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae38003c40 0x7fae38004030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae40001a60 tx=0x7fae40007d60 comp rx=0 tx=0).stop 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae567ff700 1 -- 172.21.15.31:0/841900070 shutdown_connections 2023-10-06T05:17:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae567ff700 1 --2- 172.21.15.31:0/841900070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae38003c40 0x7fae38004030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae567ff700 1 -- 172.21.15.31:0/841900070 >> 172.21.15.31:0/841900070 conn(0x7fae500cc0b0 msgr2=0x7fae500cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae567ff700 1 -- 172.21.15.31:0/841900070 shutdown_connections 2023-10-06T05:17:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.613+0000 7fae567ff700 1 -- 172.21.15.31:0/841900070 wait complete. 2023-10-06T05:17:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.615+0000 7fae567ff700 1 Processor -- start 2023-10-06T05:17:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.615+0000 7fae567ff700 1 -- start start 2023-10-06T05:17:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae567ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38003c40 0x7fae500ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae567ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae500c0e50 0x7fae500c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae567ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae50157af0 0x7fae50157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae567ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae500c1800 con 0x7fae50157af0 2023-10-06T05:17:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae567ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae50158820 con 0x7fae500c0e50 2023-10-06T05:17:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae567ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae50158b00 con 0x7fae38003c40 2023-10-06T05:17:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae4ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38003c40 0x7fae500ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae54d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae50157af0 0x7fae50157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae4ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38003c40 0x7fae500ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:52316/0 (socket says 172.21.15.31:52316) 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae4f7fe700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae500c0e50 0x7fae500c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.616+0000 7fae4ffff700 1 -- 172.21.15.31:0/2918717560 learned_addr learned my addr 172.21.15.31:0/2918717560 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.617+0000 7fae54d9c700 1 -- 172.21.15.31:0/2918717560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38003c40 msgr2=0x7fae500ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.617+0000 7fae54d9c700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38003c40 0x7fae500ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.617+0000 7fae54d9c700 1 -- 172.21.15.31:0/2918717560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae500c0e50 msgr2=0x7fae500c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.617+0000 7fae54d9c700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae500c0e50 0x7fae500c12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.617+0000 7fae54d9c700 1 -- 172.21.15.31:0/2918717560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae50158de0 con 0x7fae50157af0 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.617+0000 7fae4f7fe700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae500c0e50 0x7fae500c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.617+0000 7fae54d9c700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae50157af0 0x7fae50157f60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae500ca520 tx=0x7fae4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:42.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.618+0000 7fae37fff700 1 -- 172.21.15.31:0/2918717560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4800e040 con 0x7fae50157af0 2023-10-06T05:17:42.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.618+0000 7fae37fff700 1 -- 172.21.15.31:0/2918717560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4800f040 con 0x7fae50157af0 2023-10-06T05:17:42.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.618+0000 7fae567ff700 1 -- 172.21.15.31:0/2918717560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae501590c0 con 0x7fae50157af0 2023-10-06T05:17:42.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.618+0000 7fae37fff700 1 -- 172.21.15.31:0/2918717560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4800fe80 con 0x7fae50157af0 2023-10-06T05:17:42.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.618+0000 7fae567ff700 1 -- 172.21.15.31:0/2918717560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae501584a0 con 0x7fae50157af0 2023-10-06T05:17:42.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.619+0000 7fae567ff700 1 -- 172.21.15.31:0/2918717560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae50158c40 con 0x7fae50157af0 2023-10-06T05:17:42.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.620+0000 7fae37fff700 1 -- 172.21.15.31:0/2918717560 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae48015640 con 0x7fae50157af0 2023-10-06T05:17:42.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.621+0000 7fae37fff700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fae3c066f30 0x7fae3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:42.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.621+0000 7fae37fff700 1 -- 172.21.15.31:0/2918717560 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fae48084eb0 con 0x7fae50157af0 2023-10-06T05:17:42.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.621+0000 7fae4ffff700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fae3c066f30 0x7fae3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:42.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.622+0000 7fae4ffff700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fae3c066f30 0x7fae3c0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fae40001870 tx=0x7fae40015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:42.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.625+0000 7fae37fff700 1 -- 172.21.15.31:0/2918717560 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae50158c40 con 0x7fae50157af0 2023-10-06T05:17:42.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.872+0000 7fae567ff700 1 -- 172.21.15.31:0/2918717560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae50158c40 con 0x7fae50157af0 2023-10-06T05:17:42.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.873+0000 7fae37fff700 1 -- 172.21.15.31:0/2918717560 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7fae50158c40 con 0x7fae50157af0 2023-10-06T05:17:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:42.875 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1001195925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1001195925},{"type":"v1","addr":"172.21.15.62:6835","nonce":1001195925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1336279472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1336279472},{"type":"v1","addr":"172.21.15.31:6837","nonce":1336279472}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/577485679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":577485679},{"type":"v1","addr":"172.21.15.62:6834","nonce":577485679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:01.930460+0000","modified":"2023-10-06T05:17:05.459675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4169737410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169737410},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169737410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:17:42.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.875+0000 7fae567ff700 1 -- 172.21.15.31:0/2918717560 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fae3c066f30 msgr2=0x7fae3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:42.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.875+0000 7fae567ff700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fae3c066f30 0x7fae3c0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fae40001870 tx=0x7fae40015000 comp rx=0 tx=0).stop 2023-10-06T05:17:42.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.875+0000 7fae567ff700 1 -- 172.21.15.31:0/2918717560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae50157af0 msgr2=0x7fae50157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:42.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.875+0000 7fae567ff700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae50157af0 0x7fae50157f60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae500ca520 tx=0x7fae4800b040 comp rx=0 tx=0).stop 2023-10-06T05:17:42.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.875+0000 7fae567ff700 1 -- 172.21.15.31:0/2918717560 shutdown_connections 2023-10-06T05:17:42.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.875+0000 7fae567ff700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fae3c066f30 0x7fae3c0693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:42.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.875+0000 7fae567ff700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae50157af0 0x7fae50157f60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:42.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.875+0000 7fae567ff700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38003c40 0x7fae500ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:42.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.875+0000 7fae567ff700 1 --2- 172.21.15.31:0/2918717560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae500c0e50 0x7fae500c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:42.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.875+0000 7fae567ff700 1 -- 172.21.15.31:0/2918717560 >> 172.21.15.31:0/2918717560 conn(0x7fae500cc0b0 msgr2=0x7fae5006e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:42.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.876+0000 7fae567ff700 1 -- 172.21.15.31:0/2918717560 shutdown_connections 2023-10-06T05:17:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:42.876+0000 7fae567ff700 1 -- 172.21.15.31:0/2918717560 wait complete. 2023-10-06T05:17:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T05:17:42.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:43.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.089+0000 7fdd13dff700 1 Processor -- start 2023-10-06T05:17:43.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.089+0000 7fdd13dff700 1 -- start start 2023-10-06T05:17:43.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.090+0000 7fdd13dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c13a1e0 0x7fdd0c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.090+0000 7fdd13dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd0c13a5b0 0x7fdd0c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.090+0000 7fdd13dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c0c71b0 con 0x7fdd0c0c6c30 2023-10-06T05:17:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.090+0000 7fdd13dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0c0c7490 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.090+0000 7fdd13dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0c7770 con 0x7fdd0c13a1e0 2023-10-06T05:17:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd1139a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd0c13a5b0 0x7fdd0c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd1239c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdd0c0c6c30 0x7fdd0c15afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49322/0 (socket says 172.21.15.31:49322) 2023-10-06T05:17:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd1239c700 1 -- 172.21.15.31:0/3077771563 learned_addr learned my addr 172.21.15.31:0/3077771563 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd1139a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd0c13a5b0 0x7fdd0c06d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57620/0 (socket says 172.21.15.31:57620) 2023-10-06T05:17:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd11b9b700 1 --2- 172.21.15.31:0/3077771563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c13a1e0 0x7fdd0c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd1139a700 1 -- 172.21.15.31:0/3077771563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c13a1e0 msgr2=0x7fdd0c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd1139a700 1 --2- 172.21.15.31:0/3077771563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c13a1e0 0x7fdd0c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd1139a700 1 -- 172.21.15.31:0/3077771563 >> v1:172.21.15.31:6789/0 conn(0x7fdd0c0c6c30 legacy=0x7fdd0c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd1139a700 1 -- 172.21.15.31:0/3077771563 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c0c7a50 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd10b99700 1 -- 172.21.15.31:0/3077771563 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009220566 0 0) 0x7fdd0c0c71b0 con 0x7fdd0c0c6c30 2023-10-06T05:17:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.091+0000 7fdd1139a700 1 --2- 172.21.15.31:0/3077771563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 0x7fdd0c06d440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0098e0 tx=0x7fdcfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee38096f89437d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd10b99700 1 -- 172.21.15.31:0/3077771563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc009e80 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd10b99700 1 -- 172.21.15.31:0/3077771563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc00b860 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd10b99700 1 -- 172.21.15.31:0/3077771563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc00bbe0 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd13dff700 1 -- 172.21.15.31:0/3077771563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 msgr2=0x7fdd0c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd13dff700 1 --2- 172.21.15.31:0/3077771563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 0x7fdd0c06d440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0098e0 tx=0x7fdcfc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd13dff700 1 -- 172.21.15.31:0/3077771563 shutdown_connections 2023-10-06T05:17:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd13dff700 1 --2- 172.21.15.31:0/3077771563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c13a1e0 0x7fdd0c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd13dff700 1 --2- 172.21.15.31:0/3077771563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 0x7fdd0c06d440 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd13dff700 1 -- 172.21.15.31:0/3077771563 >> 172.21.15.31:0/3077771563 conn(0x7fdd0c0cc0b0 msgr2=0x7fdd0c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd13dff700 1 -- 172.21.15.31:0/3077771563 shutdown_connections 2023-10-06T05:17:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.092+0000 7fdd13dff700 1 -- 172.21.15.31:0/3077771563 wait complete. 2023-10-06T05:17:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.094+0000 7fdd13dff700 1 Processor -- start 2023-10-06T05:17:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.094+0000 7fdd13dff700 1 -- start start 2023-10-06T05:17:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.095+0000 7fdd13dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c0c6c30 0x7fdd0c15b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.095+0000 7fdd13dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd0c13a1e0 0x7fdd0c169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.095+0000 7fdd13dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 0x7fdd0c150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.095+0000 7fdd13dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c16d1a0 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.095+0000 7fdd13dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c16d480 con 0x7fdd0c13a1e0 2023-10-06T05:17:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.095+0000 7fdd13dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c16a0e0 con 0x7fdd0c0c6c30 2023-10-06T05:17:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.095+0000 7fdd1239c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 0x7fdd0c150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.095+0000 7fdd1239c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 0x7fdd0c150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57622/0 (socket says 172.21.15.31:57622) 2023-10-06T05:17:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.095+0000 7fdd1239c700 1 -- 172.21.15.31:0/2130551530 learned_addr learned my addr 172.21.15.31:0/2130551530 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd11b9b700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c0c6c30 0x7fdd0c15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd1139a700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd0c13a1e0 0x7fdd0c169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd1239c700 1 -- 172.21.15.31:0/2130551530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c0c6c30 msgr2=0x7fdd0c15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd1239c700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c0c6c30 0x7fdd0c15b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd1239c700 1 -- 172.21.15.31:0/2130551530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd0c13a1e0 msgr2=0x7fdd0c169ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd1239c700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd0c13a1e0 0x7fdd0c169ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd1239c700 1 -- 172.21.15.31:0/2130551530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc009420 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd1239c700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 0x7fdd0c150f50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400ccc0 tx=0x7fdd04001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2130551530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400ab30 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2130551530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0400ac90 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd13dff700 1 -- 172.21.15.31:0/2130551530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c16a380 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2130551530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04007340 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.096+0000 7fdd13dff700 1 -- 172.21.15.31:0/2130551530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c16a940 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.097+0000 7fdd13dff700 1 -- 172.21.15.31:0/2130551530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0c04c380 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.099+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2130551530 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd040076f0 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.099+0000 7fdd01ffb700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdcf0066f30 0x7fdcf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.099+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2130551530 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd04084be0 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.100+0000 7fdd11b9b700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdcf0066f30 0x7fdcf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.100+0000 7fdd11b9b700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdcf0066f30 0x7fdcf00693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdd08006aa0 tx=0x7fdd08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:43.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.104+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2130551530 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd04051cd0 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.311+0000 7fdd13dff700 1 -- 172.21.15.31:0/2130551530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd0c0c7d30 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.312+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2130551530 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fdd040552f0 con 0x7fdd0c13a5b0 2023-10-06T05:17:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:43.314 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:43.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 -- 172.21.15.31:0/2130551530 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdcf0066f30 msgr2=0x7fdcf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdcf0066f30 0x7fdcf00693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdd08006aa0 tx=0x7fdd08008040 comp rx=0 tx=0).stop 2023-10-06T05:17:43.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 -- 172.21.15.31:0/2130551530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 msgr2=0x7fdd0c150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 0x7fdd0c150f50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400ccc0 tx=0x7fdd04001a80 comp rx=0 tx=0).stop 2023-10-06T05:17:43.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 -- 172.21.15.31:0/2130551530 shutdown_connections 2023-10-06T05:17:43.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdcf0066f30 0x7fdcf00693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c13a5b0 0x7fdd0c150f50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c0c6c30 0x7fdd0c15b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 --2- 172.21.15.31:0/2130551530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd0c13a1e0 0x7fdd0c169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 -- 172.21.15.31:0/2130551530 >> 172.21.15.31:0/2130551530 conn(0x7fdd0c0cc0b0 msgr2=0x7fdd0c0cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 -- 172.21.15.31:0/2130551530 shutdown_connections 2023-10-06T05:17:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.314+0000 7fdd13dff700 1 -- 172.21.15.31:0/2130551530 wait complete. 2023-10-06T05:17:43.333 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:16:59.507957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:17:00.905826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:17:43.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:43.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.530+0000 7fc2bcdff700 1 Processor -- start 2023-10-06T05:17:43.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.530+0000 7fc2bcdff700 1 -- start start 2023-10-06T05:17:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.531+0000 7fc2bcdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 0x7fc2b80c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.531+0000 7fc2bcdff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b8068250 con 0x7fc2b80c7560 2023-10-06T05:17:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.531+0000 7fc2bcdff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b806ac80 con 0x7fc2b80c7190 2023-10-06T05:17:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.531+0000 7fc2bcdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8067f20 con 0x7fc2b80aef00 2023-10-06T05:17:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.531+0000 7fc2b6d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc2b80c7560 0x7fc2b80686e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49338/0 (socket says 172.21.15.31:49338) 2023-10-06T05:17:43.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.531+0000 7fc2b6d9d700 1 -- 172.21.15.31:0/3274538033 learned_addr learned my addr 172.21.15.31:0/3274538033 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:43.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.532+0000 7fc2b5d9b700 1 --2- 172.21.15.31:0/3274538033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 0x7fc2b80c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.532+0000 7fc2b559a700 1 -- 172.21.15.31:0/3274538033 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459265152 0 0) 0x7fc2b8068250 con 0x7fc2b80c7560 2023-10-06T05:17:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.532+0000 7fc2b559a700 1 -- 172.21.15.31:0/3274538033 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a0003570 con 0x7fc2b80c7560 2023-10-06T05:17:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.532+0000 7fc2b559a700 1 -- 172.21.15.31:0/3274538033 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944855565 0 0) 0x7fc2b806ac80 con 0x7fc2b80c7190 2023-10-06T05:17:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.532+0000 7fc2b559a700 1 -- 172.21.15.31:0/3274538033 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b8068250 con 0x7fc2b80c7190 2023-10-06T05:17:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.532+0000 7fc2b5d9b700 1 -- 172.21.15.31:0/3274538033 >> v1:172.21.15.62:6789/0 conn(0x7fc2b80c7190 legacy=0x7fc2b80af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.532+0000 7fc2b5d9b700 1 -- 172.21.15.31:0/3274538033 >> v1:172.21.15.31:6789/0 conn(0x7fc2b80c7560 legacy=0x7fc2b80686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.532+0000 7fc2b5d9b700 1 -- 172.21.15.31:0/3274538033 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b806aed0 con 0x7fc2b80aef00 2023-10-06T05:17:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.532+0000 7fc2b5d9b700 1 --2- 172.21.15.31:0/3274538033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 0x7fc2b80c9d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009910 tx=0x7fc2ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7f0cdeaa5252f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.533+0000 7fc2b559a700 1 -- 172.21.15.31:0/3274538033 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac011630 con 0x7fc2b80aef00 2023-10-06T05:17:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.533+0000 7fc2b559a700 1 -- 172.21.15.31:0/3274538033 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac011790 con 0x7fc2b80aef00 2023-10-06T05:17:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.533+0000 7fc2b559a700 1 -- 172.21.15.31:0/3274538033 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac011b30 con 0x7fc2b80aef00 2023-10-06T05:17:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.533+0000 7fc2bcdff700 1 -- 172.21.15.31:0/3274538033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 msgr2=0x7fc2b80c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.533+0000 7fc2bcdff700 1 --2- 172.21.15.31:0/3274538033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 0x7fc2b80c9d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009910 tx=0x7fc2ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.533+0000 7fc2bcdff700 1 -- 172.21.15.31:0/3274538033 shutdown_connections 2023-10-06T05:17:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.533+0000 7fc2bcdff700 1 --2- 172.21.15.31:0/3274538033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 0x7fc2b80c9d40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.533+0000 7fc2bcdff700 1 -- 172.21.15.31:0/3274538033 >> 172.21.15.31:0/3274538033 conn(0x7fc2b80cd3d0 msgr2=0x7fc2b80c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.533+0000 7fc2bcdff700 1 -- 172.21.15.31:0/3274538033 shutdown_connections 2023-10-06T05:17:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.534+0000 7fc2bcdff700 1 -- 172.21.15.31:0/3274538033 wait complete. 2023-10-06T05:17:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.535+0000 7fc2bcdff700 1 Processor -- start 2023-10-06T05:17:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.535+0000 7fc2bcdff700 1 -- start start 2023-10-06T05:17:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2bcdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 0x7fc2b816d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2bcdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b80c7190 0x7fc2b8160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2bcdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b80c7560 0x7fc2b8169800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2bcdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8160760 con 0x7fc2b80c7190 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2bcdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b816ce00 con 0x7fc2b80c7560 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2bcdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b816a310 con 0x7fc2b80aef00 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2b5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b80c7190 0x7fc2b8160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2b5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b80c7190 0x7fc2b8160220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57630/0 (socket says 172.21.15.31:57630) 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2b5d9b700 1 -- 172.21.15.31:0/2572769979 learned_addr learned my addr 172.21.15.31:0/2572769979 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2b659c700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 0x7fc2b816d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.536+0000 7fc2b6d9d700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b80c7560 0x7fc2b8169800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2b5d9b700 1 -- 172.21.15.31:0/2572769979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 msgr2=0x7fc2b816d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2b5d9b700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 0x7fc2b816d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2b5d9b700 1 -- 172.21.15.31:0/2572769979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b80c7560 msgr2=0x7fc2b8169800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2b5d9b700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b80c7560 0x7fc2b8169800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2b5d9b700 1 -- 172.21.15.31:0/2572769979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac009420 con 0x7fc2b80c7190 2023-10-06T05:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2b6d9d700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b80c7560 0x7fc2b8169800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2b5d9b700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b80c7190 0x7fc2b8160220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009f00 tx=0x7fc2ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2a67fc700 1 -- 172.21.15.31:0/2572769979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac01f070 con 0x7fc2b80c7190 2023-10-06T05:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2a67fc700 1 -- 172.21.15.31:0/2572769979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac000bc0 con 0x7fc2b80c7190 2023-10-06T05:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2a67fc700 1 -- 172.21.15.31:0/2572769979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac025420 con 0x7fc2b80c7190 2023-10-06T05:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2bcdff700 1 -- 172.21.15.31:0/2572769979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b816a5f0 con 0x7fc2b80c7190 2023-10-06T05:17:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.537+0000 7fc2bcdff700 1 -- 172.21.15.31:0/2572769979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b816d380 con 0x7fc2b80c7190 2023-10-06T05:17:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.538+0000 7fc2bcdff700 1 -- 172.21.15.31:0/2572769979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b816d5a0 con 0x7fc2b80c7190 2023-10-06T05:17:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.539+0000 7fc2a67fc700 1 -- 172.21.15.31:0/2572769979 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2ac023070 con 0x7fc2b80c7190 2023-10-06T05:17:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.540+0000 7fc2a67fc700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc298066f30 0x7fc2980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.540+0000 7fc2a67fc700 1 -- 172.21.15.31:0/2572769979 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc2ac085960 con 0x7fc2b80c7190 2023-10-06T05:17:43.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.541+0000 7fc2b659c700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc298066f30 0x7fc2980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.541+0000 7fc2b659c700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc298066f30 0x7fc2980693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc2b806b460 tx=0x7fc2a8007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.544+0000 7fc2a67fc700 1 -- 172.21.15.31:0/2572769979 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2ac0529d0 con 0x7fc2b80c7190 2023-10-06T05:17:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.750+0000 7fc2bcdff700 1 -- 172.21.15.31:0/2572769979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc2b804c380 con 0x7fc2b80c7190 2023-10-06T05:17:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.750+0000 7fc2a67fc700 1 -- 172.21.15.31:0/2572769979 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc2ac02a0a0 con 0x7fc2b80c7190 2023-10-06T05:17:43.753 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:17:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 -- 172.21.15.31:0/2572769979 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc298066f30 msgr2=0x7fc2980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc298066f30 0x7fc2980693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc2b806b460 tx=0x7fc2a8007d50 comp rx=0 tx=0).stop 2023-10-06T05:17:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 -- 172.21.15.31:0/2572769979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b80c7190 msgr2=0x7fc2b8160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b80c7190 0x7fc2b8160220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009f00 tx=0x7fc2ac017000 comp rx=0 tx=0).stop 2023-10-06T05:17:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 -- 172.21.15.31:0/2572769979 shutdown_connections 2023-10-06T05:17:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2b80c7560 0x7fc2b8169800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2b80aef00 0x7fc2b816d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc298066f30 0x7fc2980693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 --2- 172.21.15.31:0/2572769979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2b80c7190 0x7fc2b8160220 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 -- 172.21.15.31:0/2572769979 >> 172.21.15.31:0/2572769979 conn(0x7fc2b80cd3d0 msgr2=0x7fc2b80cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 -- 172.21.15.31:0/2572769979 shutdown_connections 2023-10-06T05:17:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.753+0000 7fc2bcdff700 1 -- 172.21.15.31:0/2572769979 wait complete. 2023-10-06T05:17:43.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:43.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.968+0000 7f57f19ff700 1 Processor -- start 2023-10-06T05:17:43.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.968+0000 7f57f19ff700 1 -- start start 2023-10-06T05:17:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.969+0000 7f57f19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 0x7f57ec0c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.969+0000 7f57f19ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f57ec159c60 0x7f57ec156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57f19ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f57ec156fa0 0x7f57ec155430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57f19ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec158190 con 0x7f57ec156fa0 2023-10-06T05:17:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57f19ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec157580 con 0x7f57ec159c60 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57f19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec157860 con 0x7f57ec159890 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57eb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f57ec156fa0 0x7f57ec155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57eb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f57ec156fa0 0x7f57ec155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57632/0 (socket says 172.21.15.31:57632) 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57eb7fe700 1 -- 172.21.15.31:0/2833262863 learned_addr learned my addr 172.21.15.31:0/2833262863 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57ea7fc700 1 --2- 172.21.15.31:0/2833262863 >> v2:172.21.15.62:3300/0 conn(0x7f57ec159c60 0x7f57ec156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57eaffd700 1 --2- 172.21.15.31:0/2833262863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 0x7f57ec0c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57eb7fe700 1 -- 172.21.15.31:0/2833262863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 msgr2=0x7f57ec0c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57eb7fe700 1 --2- 172.21.15.31:0/2833262863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 0x7f57ec0c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57eb7fe700 1 -- 172.21.15.31:0/2833262863 >> v2:172.21.15.62:3300/0 conn(0x7f57ec159c60 msgr2=0x7f57ec156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57eb7fe700 1 --2- 172.21.15.31:0/2833262863 >> v2:172.21.15.62:3300/0 conn(0x7f57ec159c60 0x7f57ec156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57eb7fe700 1 -- 172.21.15.31:0/2833262863 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec157b40 con 0x7f57ec156fa0 2023-10-06T05:17:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.970+0000 7f57ea7fc700 1 --2- 172.21.15.31:0/2833262863 >> v2:172.21.15.62:3300/0 conn(0x7f57ec159c60 0x7f57ec156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:43.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.971+0000 7f57eb7fe700 1 --2- 172.21.15.31:0/2833262863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 0x7f57ec155430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009f20 tx=0x7f57e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cfd5b6287040a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:43.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.971+0000 7f57e9ffb700 1 -- 172.21.15.31:0/2833262863 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e400e070 con 0x7f57ec156fa0 2023-10-06T05:17:43.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.971+0000 7f57e9ffb700 1 -- 172.21.15.31:0/2833262863 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e400b990 con 0x7f57ec156fa0 2023-10-06T05:17:43.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.971+0000 7f57e9ffb700 1 -- 172.21.15.31:0/2833262863 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e40092e0 con 0x7f57ec156fa0 2023-10-06T05:17:43.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.971+0000 7f57f19ff700 1 -- 172.21.15.31:0/2833262863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 msgr2=0x7f57ec155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.971+0000 7f57f19ff700 1 --2- 172.21.15.31:0/2833262863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 0x7f57ec155430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009f20 tx=0x7f57e4009840 comp rx=0 tx=0).stop 2023-10-06T05:17:43.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.972+0000 7f57f19ff700 1 -- 172.21.15.31:0/2833262863 shutdown_connections 2023-10-06T05:17:43.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.972+0000 7f57f19ff700 1 --2- 172.21.15.31:0/2833262863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 0x7f57ec155430 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.972+0000 7f57f19ff700 1 --2- 172.21.15.31:0/2833262863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 0x7f57ec0c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.972+0000 7f57f19ff700 1 --2- 172.21.15.31:0/2833262863 >> v2:172.21.15.62:3300/0 conn(0x7f57ec159c60 0x7f57ec156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.972+0000 7f57f19ff700 1 -- 172.21.15.31:0/2833262863 >> 172.21.15.31:0/2833262863 conn(0x7f57ec067560 msgr2=0x7f57ec067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:43.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.972+0000 7f57f19ff700 1 -- 172.21.15.31:0/2833262863 shutdown_connections 2023-10-06T05:17:43.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.972+0000 7f57f19ff700 1 -- 172.21.15.31:0/2833262863 wait complete. 2023-10-06T05:17:43.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.973+0000 7f57f19ff700 1 Processor -- start 2023-10-06T05:17:43.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.973+0000 7f57f19ff700 1 -- start start 2023-10-06T05:17:43.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.974+0000 7f57f19ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 0x7f57ec168640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.974+0000 7f57f19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 0x7f57ec167ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.974+0000 7f57f19ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ec159c60 0x7f57ec16a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.974+0000 7f57f19ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec168b80 con 0x7f57ec156fa0 2023-10-06T05:17:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.974+0000 7f57f19ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec168400 con 0x7f57ec159c60 2023-10-06T05:17:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.974+0000 7f57f19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0ab990 con 0x7f57ec159890 2023-10-06T05:17:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.974+0000 7f57eaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 0x7f57ec168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.974+0000 7f57eaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 0x7f57ec168640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57642/0 (socket says 172.21.15.31:57642) 2023-10-06T05:17:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.974+0000 7f57eaffd700 1 -- 172.21.15.31:0/2038233951 learned_addr learned my addr 172.21.15.31:0/2038233951 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57eb7fe700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ec159c60 0x7f57ec16a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57ea7fc700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 0x7f57ec167ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57eaffd700 1 -- 172.21.15.31:0/2038233951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 msgr2=0x7f57ec167ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57eaffd700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 0x7f57ec167ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57eaffd700 1 -- 172.21.15.31:0/2038233951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ec159c60 msgr2=0x7f57ec16a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57eaffd700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ec159c60 0x7f57ec16a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57eaffd700 1 -- 172.21.15.31:0/2038233951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4009ae0 con 0x7f57ec156fa0 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57eb7fe700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ec159c60 0x7f57ec16a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57ea7fc700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 0x7f57ec167ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57eaffd700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 0x7f57ec168640 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00ec80 tx=0x7f57dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57d2ffd700 1 -- 172.21.15.31:0/2038233951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00c9d0 con 0x7f57ec156fa0 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57d2ffd700 1 -- 172.21.15.31:0/2038233951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc00cb50 con 0x7f57ec156fa0 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57d2ffd700 1 -- 172.21.15.31:0/2038233951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc013470 con 0x7f57ec156fa0 2023-10-06T05:17:43.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.975+0000 7f57f19ff700 1 -- 172.21.15.31:0/2038233951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec0abc70 con 0x7f57ec156fa0 2023-10-06T05:17:43.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.976+0000 7f57f19ff700 1 -- 172.21.15.31:0/2038233951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec0ac230 con 0x7f57ec156fa0 2023-10-06T05:17:43.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.976+0000 7f57f19ff700 1 -- 172.21.15.31:0/2038233951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec04c380 con 0x7f57ec156fa0 2023-10-06T05:17:43.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.977+0000 7f57d2ffd700 1 -- 172.21.15.31:0/2038233951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57dc013860 con 0x7f57ec156fa0 2023-10-06T05:17:43.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.978+0000 7f57d2ffd700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57d8066f30 0x7f57d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.979+0000 7f57d2ffd700 1 -- 172.21.15.31:0/2038233951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f57dc084b00 con 0x7f57ec156fa0 2023-10-06T05:17:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.979+0000 7f57ea7fc700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57d8066f30 0x7f57d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.979+0000 7f57ea7fc700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57d8066f30 0x7f57d80693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f57e0006f00 tx=0x7f57e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:43.982+0000 7f57d2ffd700 1 -- 172.21.15.31:0/2038233951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57dc086e60 con 0x7f57ec156fa0 2023-10-06T05:17:44.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.186+0000 7f57f19ff700 1 -- 172.21.15.31:0/2038233951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57ec157e20 con 0x7f57ec156fa0 2023-10-06T05:17:44.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.187+0000 7f57d2ffd700 1 -- 172.21.15.31:0/2038233951 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57dc051b70 con 0x7f57ec156fa0 2023-10-06T05:17:44.189 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:17:44.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 -- 172.21.15.31:0/2038233951 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57d8066f30 msgr2=0x7f57d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57d8066f30 0x7f57d80693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f57e0006f00 tx=0x7f57e0008040 comp rx=0 tx=0).stop 2023-10-06T05:17:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 -- 172.21.15.31:0/2038233951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 msgr2=0x7f57ec168640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 0x7f57ec168640 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00ec80 tx=0x7f57dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 -- 172.21.15.31:0/2038233951 shutdown_connections 2023-10-06T05:17:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57ec159c60 0x7f57ec16a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f57d8066f30 0x7f57d80693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec156fa0 0x7f57ec168640 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 --2- 172.21.15.31:0/2038233951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57ec159890 0x7f57ec167ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 -- 172.21.15.31:0/2038233951 >> 172.21.15.31:0/2038233951 conn(0x7f57ec067560 msgr2=0x7f57ec152330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 -- 172.21.15.31:0/2038233951 shutdown_connections 2023-10-06T05:17:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.189+0000 7f57f19ff700 1 -- 172.21.15.31:0/2038233951 wait complete. 2023-10-06T05:17:44.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:17:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.405+0000 7efd23fff700 1 Processor -- start 2023-10-06T05:17:44.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.406+0000 7efd23fff700 1 -- start start 2023-10-06T05:17:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.407+0000 7efd23fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd1c06a300 0x7efd1c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.407+0000 7efd23fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1c068da0 con 0x7efd1c15ca40 2023-10-06T05:17:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.407+0000 7efd23fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1c069080 con 0x7efd1c0c8f40 2023-10-06T05:17:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.407+0000 7efd23fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0cabc0 con 0x7efd1c06a300 2023-10-06T05:17:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.407+0000 7efd21d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efd1c15ca40 0x7efd1c0c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49342/0 (socket says 172.21.15.31:49342) 2023-10-06T05:17:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.407+0000 7efd21d9b700 1 -- 172.21.15.31:0/2702378857 learned_addr learned my addr 172.21.15.31:0/2702378857 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.407+0000 7efd2259c700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd1c06a300 0x7efd1c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633090875 0 0) 0x7efd1c068da0 con 0x7efd1c15ca40 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd08003570 con 0x7efd1c15ca40 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806370821 0 0) 0x7efd1c069080 con 0x7efd1c0c8f40 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd1c068da0 con 0x7efd1c0c8f40 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 173635275 0 0) 0x7efd08003570 con 0x7efd1c15ca40 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd1c06a300 msgr2=0x7efd1c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd1c06a300 0x7efd1c068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 >> v1:172.21.15.62:6789/0 conn(0x7efd1c0c8f40 legacy=0x7efd1c0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c069360 con 0x7efd1c15ca40 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7efd18001470 con 0x7efd1c15ca40 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 >> v1:172.21.15.31:6789/0 conn(0x7efd1c15ca40 legacy=0x7efd1c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd08003c20 0x7efd08004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd08004810 0x7efd08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd1c06a300 0x7efd08005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0cabc0 con 0x7efd08003c20 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0800bbf0 con 0x7efd08004810 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0800bdb0 con 0x7efd1c06a300 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.408+0000 7efd21d9b700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd08003c20 0x7efd08004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd2259c700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd1c06a300 0x7efd08005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd2159a700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd08004810 0x7efd08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd21d9b700 1 -- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd1c06a300 msgr2=0x7efd08005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd21d9b700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd1c06a300 0x7efd08005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd21d9b700 1 -- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd08004810 msgr2=0x7efd08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:44.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd21d9b700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd08004810 0x7efd08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd21d9b700 1 -- 172.21.15.31:0/2702378857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd08003570 con 0x7efd08003c20 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd21d9b700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd08003c20 0x7efd08004010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c1e0 tx=0x7efd18001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b5930efebff1c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1800cb10 con 0x7efd08003c20 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1800cc90 con 0x7efd08003c20 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.409+0000 7efd20d99700 1 -- 172.21.15.31:0/2702378857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18005b70 con 0x7efd08003c20 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.410+0000 7efd23fff700 1 -- 172.21.15.31:0/2702378857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd08003c20 msgr2=0x7efd08004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.410+0000 7efd23fff700 1 --2- 172.21.15.31:0/2702378857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd08003c20 0x7efd08004010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c1e0 tx=0x7efd18001d40 comp rx=0 tx=0).stop 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.410+0000 7efd2259c700 1 -- 172.21.15.31:0/2702378857 reap_dead start 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.410+0000 7efd23fff700 1 -- 172.21.15.31:0/2702378857 shutdown_connections 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.410+0000 7efd23fff700 1 -- 172.21.15.31:0/2702378857 >> 172.21.15.31:0/2702378857 conn(0x7efd1c0ce290 msgr2=0x7efd1c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.410+0000 7efd23fff700 1 -- 172.21.15.31:0/2702378857 shutdown_connections 2023-10-06T05:17:44.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.410+0000 7efd23fff700 1 -- 172.21.15.31:0/2702378857 wait complete. 2023-10-06T05:17:44.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.412+0000 7efd23fff700 1 Processor -- start 2023-10-06T05:17:44.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.412+0000 7efd23fff700 1 -- start start 2023-10-06T05:17:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd23fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd08003c20 0x7efd1c16a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd23fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd1c0c29a0 0x7efd1c0c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd1c0c7580 0x7efd1c0c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd23fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0c4500 con 0x7efd1c0c29a0 2023-10-06T05:17:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd23fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0c7f40 con 0x7efd1c0c7580 2023-10-06T05:17:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd23fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c16a800 con 0x7efd08003c20 2023-10-06T05:17:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd2159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd1c0c29a0 0x7efd1c0c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd2159a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd1c0c29a0 0x7efd1c0c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57666/0 (socket says 172.21.15.31:57666) 2023-10-06T05:17:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd2159a700 1 -- 172.21.15.31:0/817069008 learned_addr learned my addr 172.21.15.31:0/817069008 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:44.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd21d9b700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd08003c20 0x7efd1c16a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd2259c700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd1c0c7580 0x7efd1c0c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd2159a700 1 -- 172.21.15.31:0/817069008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd08003c20 msgr2=0x7efd1c16a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd2159a700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd08003c20 0x7efd1c16a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd2159a700 1 -- 172.21.15.31:0/817069008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd1c0c7580 msgr2=0x7efd1c0c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd2159a700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd1c0c7580 0x7efd1c0c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.413+0000 7efd2159a700 1 -- 172.21.15.31:0/817069008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c0c3460 con 0x7efd1c0c29a0 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.414+0000 7efd2159a700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd1c0c29a0 0x7efd1c0c2e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd0c002c00 tx=0x7efd0c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.414+0000 7efd11ffb700 1 -- 172.21.15.31:0/817069008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c00fa30 con 0x7efd1c0c29a0 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.414+0000 7efd11ffb700 1 -- 172.21.15.31:0/817069008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c00fb90 con 0x7efd1c0c29a0 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.414+0000 7efd23fff700 1 -- 172.21.15.31:0/817069008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c0c3740 con 0x7efd1c0c29a0 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.414+0000 7efd11ffb700 1 -- 172.21.15.31:0/817069008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c0133f0 con 0x7efd1c0c29a0 2023-10-06T05:17:44.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.414+0000 7efd23fff700 1 -- 172.21.15.31:0/817069008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c0c3d00 con 0x7efd1c0c29a0 2023-10-06T05:17:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.415+0000 7efd23fff700 1 -- 172.21.15.31:0/817069008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c04c380 con 0x7efd1c0c29a0 2023-10-06T05:17:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.416+0000 7efd11ffb700 1 -- 172.21.15.31:0/817069008 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd0c013760 con 0x7efd1c0c29a0 2023-10-06T05:17:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.417+0000 7efd11ffb700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efd00066f30 0x7efd000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.417+0000 7efd11ffb700 1 -- 172.21.15.31:0/817069008 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efd0c084ec0 con 0x7efd1c0c29a0 2023-10-06T05:17:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.417+0000 7efd21d9b700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efd00066f30 0x7efd000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.418+0000 7efd21d9b700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efd00066f30 0x7efd000693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0b05d0 tx=0x7efd18015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:44.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.421+0000 7efd11ffb700 1 -- 172.21.15.31:0/817069008 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd0c051f30 con 0x7efd1c0c29a0 2023-10-06T05:17:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.625+0000 7efd23fff700 1 -- 172.21.15.31:0/817069008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd1c0ca1a0 con 0x7efd1c0c29a0 2023-10-06T05:17:44.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.626+0000 7efd11ffb700 1 -- 172.21.15.31:0/817069008 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd0c055550 con 0x7efd1c0c29a0 2023-10-06T05:17:44.628 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:17:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.628+0000 7efd23fff700 1 -- 172.21.15.31:0/817069008 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efd00066f30 msgr2=0x7efd000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.628+0000 7efd23fff700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efd00066f30 0x7efd000693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0b05d0 tx=0x7efd18015000 comp rx=0 tx=0).stop 2023-10-06T05:17:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.628+0000 7efd23fff700 1 -- 172.21.15.31:0/817069008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd1c0c29a0 msgr2=0x7efd1c0c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.628+0000 7efd23fff700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd1c0c29a0 0x7efd1c0c2e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd0c002c00 tx=0x7efd0c007d30 comp rx=0 tx=0).stop 2023-10-06T05:17:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.628+0000 7efd23fff700 1 -- 172.21.15.31:0/817069008 shutdown_connections 2023-10-06T05:17:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.628+0000 7efd23fff700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd1c0c7580 0x7efd1c0c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.628+0000 7efd23fff700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd08003c20 0x7efd1c16a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.628+0000 7efd23fff700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efd00066f30 0x7efd000693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.628+0000 7efd23fff700 1 --2- 172.21.15.31:0/817069008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd1c0c29a0 0x7efd1c0c2e10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.628+0000 7efd23fff700 1 -- 172.21.15.31:0/817069008 >> 172.21.15.31:0/817069008 conn(0x7efd1c0ce290 msgr2=0x7efd1c0cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.629+0000 7efd23fff700 1 -- 172.21.15.31:0/817069008 shutdown_connections 2023-10-06T05:17:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.629+0000 7efd23fff700 1 -- 172.21.15.31:0/817069008 wait complete. 2023-10-06T05:17:44.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:17:44.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.843+0000 7f7e439ff700 1 Processor -- start 2023-10-06T05:17:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.843+0000 7f7e439ff700 1 -- start start 2023-10-06T05:17:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.844+0000 7f7e439ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e3c159560 0x7f7e3c0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.844+0000 7f7e439ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159930 0x7f7e3c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.844+0000 7f7e439ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c15d2d0 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.844+0000 7f7e439ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c15d5b0 con 0x7f7e3c159560 2023-10-06T05:17:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.844+0000 7f7e439ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c15d890 con 0x7f7e3c159930 2023-10-06T05:17:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.844+0000 7f7e4179b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e3c159560 0x7f7e3c0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.844+0000 7f7e4179b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e3c159560 0x7f7e3c0cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:37162/0 (socket says 172.21.15.31:37162) 2023-10-06T05:17:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.844+0000 7f7e40f9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159930 0x7f7e3c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.844+0000 7f7e4179b700 1 -- 172.21.15.31:0/3814770046 learned_addr learned my addr 172.21.15.31:0/3814770046 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:44.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.845+0000 7f7e33fff700 1 -- 172.21.15.31:0/3814770046 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289575904 0 0) 0x7f7e3c15d2d0 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.845+0000 7f7e33fff700 1 -- 172.21.15.31:0/3814770046 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e28003570 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.845+0000 7f7e40f9a700 1 -- 172.21.15.31:0/3814770046 >> v2:172.21.15.62:3300/0 conn(0x7f7e3c159560 msgr2=0x7f7e3c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:44.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.845+0000 7f7e40f9a700 1 --2- 172.21.15.31:0/3814770046 >> v2:172.21.15.62:3300/0 conn(0x7f7e3c159560 0x7f7e3c0cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.845+0000 7f7e40f9a700 1 -- 172.21.15.31:0/3814770046 >> v1:172.21.15.62:6789/0 conn(0x7f7e3c0b0d60 legacy=0x7f7e3c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.845+0000 7f7e40f9a700 1 -- 172.21.15.31:0/3814770046 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c15db70 con 0x7f7e3c159930 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.845+0000 7f7e40f9a700 1 --2- 172.21.15.31:0/3814770046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159930 0x7f7e3c0cd3b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0098e0 tx=0x7f7e2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c167661151218ac9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e33fff700 1 -- 172.21.15.31:0/3814770046 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c011630 con 0x7f7e3c159930 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e33fff700 1 -- 172.21.15.31:0/3814770046 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c011790 con 0x7f7e3c159930 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e33fff700 1 -- 172.21.15.31:0/3814770046 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c011b30 con 0x7f7e3c159930 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e439ff700 1 -- 172.21.15.31:0/3814770046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159930 msgr2=0x7f7e3c0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e439ff700 1 --2- 172.21.15.31:0/3814770046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159930 0x7f7e3c0cd3b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0098e0 tx=0x7f7e2c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e439ff700 1 -- 172.21.15.31:0/3814770046 shutdown_connections 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e439ff700 1 --2- 172.21.15.31:0/3814770046 >> v2:172.21.15.62:3300/0 conn(0x7f7e3c159560 0x7f7e3c0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e439ff700 1 --2- 172.21.15.31:0/3814770046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159930 0x7f7e3c0cd3b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e439ff700 1 -- 172.21.15.31:0/3814770046 >> 172.21.15.31:0/3814770046 conn(0x7f7e3c06ec90 msgr2=0x7f7e3c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e439ff700 1 -- 172.21.15.31:0/3814770046 shutdown_connections 2023-10-06T05:17:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.846+0000 7f7e439ff700 1 -- 172.21.15.31:0/3814770046 wait complete. 2023-10-06T05:17:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.848+0000 7f7e439ff700 1 Processor -- start 2023-10-06T05:17:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.848+0000 7f7e439ff700 1 -- start start 2023-10-06T05:17:44.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e439ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c0b0d60 0x7f7e3c0c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e439ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159560 0x7f7e3c0c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e439ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e3c159930 0x7f7e3c155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e439ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0ce950 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e439ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0ce060 con 0x7f7e3c159930 2023-10-06T05:17:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e439ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0ce300 con 0x7f7e3c159560 2023-10-06T05:17:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e4179b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c0b0d60 0x7f7e3c0c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e4179b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c0b0d60 0x7f7e3c0c4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57674/0 (socket says 172.21.15.31:57674) 2023-10-06T05:17:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e4179b700 1 -- 172.21.15.31:0/1672041802 learned_addr learned my addr 172.21.15.31:0/1672041802 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e41f9c700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e3c159930 0x7f7e3c155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e4179b700 1 -- 172.21.15.31:0/1672041802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159560 msgr2=0x7f7e3c0c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.849+0000 7f7e40f9a700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159560 0x7f7e3c0c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e4179b700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159560 0x7f7e3c0c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e4179b700 1 -- 172.21.15.31:0/1672041802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e3c159930 msgr2=0x7f7e3c155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e4179b700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e3c159930 0x7f7e3c155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e4179b700 1 -- 172.21.15.31:0/1672041802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c009420 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e41f9c700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e3c159930 0x7f7e3c155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e40f9a700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159560 0x7f7e3c0c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e4179b700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c0b0d60 0x7f7e3c0c4de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800ea50 tx=0x7f7e38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e317fa700 1 -- 172.21.15.31:0/1672041802 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800c850 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e317fa700 1 -- 172.21.15.31:0/1672041802 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3800c9b0 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e317fa700 1 -- 172.21.15.31:0/1672041802 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800cd30 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e439ff700 1 -- 172.21.15.31:0/1672041802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c0ce5e0 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.850+0000 7f7e439ff700 1 -- 172.21.15.31:0/1672041802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c0c5320 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.851+0000 7f7e439ff700 1 -- 172.21.15.31:0/1672041802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c04c380 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.852+0000 7f7e317fa700 1 -- 172.21.15.31:0/1672041802 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e38013680 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.853+0000 7f7e317fa700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7e20066f30 0x7f7e200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:44.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.853+0000 7f7e317fa700 1 -- 172.21.15.31:0/1672041802 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7e38084950 con 0x7f7e3c0b0d60 2023-10-06T05:17:44.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.853+0000 7f7e40f9a700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7e20066f30 0x7f7e200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:44.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.854+0000 7f7e40f9a700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7e20066f30 0x7f7e200693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0c7170 tx=0x7f7e2c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:44.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:44.857+0000 7f7e317fa700 1 -- 172.21.15.31:0/1672041802 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e38051a60 con 0x7f7e3c0b0d60 2023-10-06T05:17:45.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.059+0000 7f7e439ff700 1 -- 172.21.15.31:0/1672041802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e3c15de50 con 0x7f7e3c0b0d60 2023-10-06T05:17:45.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.059+0000 7f7e317fa700 1 -- 172.21.15.31:0/1672041802 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e38055080 con 0x7f7e3c0b0d60 2023-10-06T05:17:45.061 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:17:45.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.061+0000 7f7e439ff700 1 -- 172.21.15.31:0/1672041802 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7e20066f30 msgr2=0x7f7e200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.061+0000 7f7e439ff700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7e20066f30 0x7f7e200693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0c7170 tx=0x7f7e2c017000 comp rx=0 tx=0).stop 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.061+0000 7f7e439ff700 1 -- 172.21.15.31:0/1672041802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c0b0d60 msgr2=0x7f7e3c0c4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.061+0000 7f7e439ff700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c0b0d60 0x7f7e3c0c4de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800ea50 tx=0x7f7e38009250 comp rx=0 tx=0).stop 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.062+0000 7f7e439ff700 1 -- 172.21.15.31:0/1672041802 shutdown_connections 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.062+0000 7f7e439ff700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e3c159930 0x7f7e3c155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.062+0000 7f7e439ff700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7e20066f30 0x7f7e200693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.062+0000 7f7e439ff700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e3c0b0d60 0x7f7e3c0c4de0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.062+0000 7f7e439ff700 1 --2- 172.21.15.31:0/1672041802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e3c159560 0x7f7e3c0c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.062+0000 7f7e439ff700 1 -- 172.21.15.31:0/1672041802 >> 172.21.15.31:0/1672041802 conn(0x7f7e3c06ec90 msgr2=0x7f7e3c0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.062+0000 7f7e439ff700 1 -- 172.21.15.31:0/1672041802 shutdown_connections 2023-10-06T05:17:45.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.062+0000 7f7e439ff700 1 -- 172.21.15.31:0/1672041802 wait complete. 2023-10-06T05:17:45.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:45.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.275+0000 7ff3b6fff700 1 Processor -- start 2023-10-06T05:17:45.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.275+0000 7ff3b6fff700 1 -- start start 2023-10-06T05:17:45.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.276+0000 7ff3b6fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3b00cd510 0x7ff3b00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.276+0000 7ff3b6fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b015d2d0 0x7ff3b00aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.276+0000 7ff3b6fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b00c3f40 con 0x7ff3b0159500 2023-10-06T05:17:45.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.276+0000 7ff3b6fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b00b2860 con 0x7ff3b00cd510 2023-10-06T05:17:45.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.277+0000 7ff3b6fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b015d7d0 con 0x7ff3b015d2d0 2023-10-06T05:17:45.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.277+0000 7ff3affff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3b00cd510 0x7ff3b00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.277+0000 7ff3affff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3b00cd510 0x7ff3b00b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57682/0 (socket says 172.21.15.31:57682) 2023-10-06T05:17:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.277+0000 7ff3affff700 1 -- 172.21.15.31:0/405780376 learned_addr learned my addr 172.21.15.31:0/405780376 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.277+0000 7ff3b559c700 1 --2- 172.21.15.31:0/405780376 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b015d2d0 0x7ff3b00aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.277+0000 7ff3affff700 1 -- 172.21.15.31:0/405780376 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b015d2d0 msgr2=0x7ff3b00aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.277+0000 7ff3affff700 1 --2- 172.21.15.31:0/405780376 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b015d2d0 0x7ff3b00aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.277+0000 7ff3affff700 1 -- 172.21.15.31:0/405780376 >> v1:172.21.15.62:6789/0 conn(0x7ff3b0159500 legacy=0x7ff3b00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.277+0000 7ff3affff700 1 -- 172.21.15.31:0/405780376 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b00cb000 con 0x7ff3b00cd510 2023-10-06T05:17:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.278+0000 7ff3affff700 1 --2- 172.21.15.31:0/405780376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b00cd510 0x7ff3b00b0ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009990 tx=0x7ff39c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2118180de54681a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:45.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.278+0000 7ff3af7fe700 1 -- 172.21.15.31:0/405780376 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c005c10 con 0x7ff3b00cd510 2023-10-06T05:17:45.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.278+0000 7ff3af7fe700 1 -- 172.21.15.31:0/405780376 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c005d70 con 0x7ff3b00cd510 2023-10-06T05:17:45.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.278+0000 7ff3af7fe700 1 -- 172.21.15.31:0/405780376 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00b880 con 0x7ff3b00cd510 2023-10-06T05:17:45.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.278+0000 7ff3b6fff700 1 -- 172.21.15.31:0/405780376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b00cd510 msgr2=0x7ff3b00b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.278+0000 7ff3b6fff700 1 --2- 172.21.15.31:0/405780376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b00cd510 0x7ff3b00b0ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009990 tx=0x7ff39c009220 comp rx=0 tx=0).stop 2023-10-06T05:17:45.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.278+0000 7ff3b6fff700 1 -- 172.21.15.31:0/405780376 shutdown_connections 2023-10-06T05:17:45.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.278+0000 7ff3b6fff700 1 --2- 172.21.15.31:0/405780376 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b015d2d0 0x7ff3b00aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.278+0000 7ff3b6fff700 1 --2- 172.21.15.31:0/405780376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b00cd510 0x7ff3b00b0ab0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.278+0000 7ff3b6fff700 1 -- 172.21.15.31:0/405780376 >> 172.21.15.31:0/405780376 conn(0x7ff3b006ec90 msgr2=0x7ff3b006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:45.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.279+0000 7ff3b6fff700 1 -- 172.21.15.31:0/405780376 shutdown_connections 2023-10-06T05:17:45.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.279+0000 7ff3b6fff700 1 -- 172.21.15.31:0/405780376 wait complete. 2023-10-06T05:17:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.280+0000 7ff3b6fff700 1 Processor -- start 2023-10-06T05:17:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.280+0000 7ff3b6fff700 1 -- start start 2023-10-06T05:17:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.281+0000 7ff3b6fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b00cd510 0x7ff3b00b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.281+0000 7ff3b6fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b0159500 0x7ff3b015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.281+0000 7ff3b6fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b015d2d0 0x7ff3b00c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.281+0000 7ff3b6fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b015b8c0 con 0x7ff3b0159500 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.281+0000 7ff3b6fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00ca0e0 con 0x7ff3b00cd510 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.281+0000 7ff3b6fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00c7a10 con 0x7ff3b015d2d0 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.281+0000 7ff3affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b0159500 0x7ff3b015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.281+0000 7ff3affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b0159500 0x7ff3b015bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57698/0 (socket says 172.21.15.31:57698) 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.281+0000 7ff3affff700 1 -- 172.21.15.31:0/4133975910 learned_addr learned my addr 172.21.15.31:0/4133975910 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3b559c700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b015d2d0 0x7ff3b00c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3b4d9b700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b00cd510 0x7ff3b00b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3affff700 1 -- 172.21.15.31:0/4133975910 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b015d2d0 msgr2=0x7ff3b00c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3affff700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b015d2d0 0x7ff3b00c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3affff700 1 -- 172.21.15.31:0/4133975910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b00cd510 msgr2=0x7ff3b00b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3affff700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b00cd510 0x7ff3b00b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3affff700 1 -- 172.21.15.31:0/4133975910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c009480 con 0x7ff3b0159500 2023-10-06T05:17:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3b4d9b700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b00cd510 0x7ff3b00b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3affff700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b0159500 0x7ff3b015bd50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff39c005d40 tx=0x7ff39c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3acff9700 1 -- 172.21.15.31:0/4133975910 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c011a60 con 0x7ff3b0159500 2023-10-06T05:17:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3acff9700 1 -- 172.21.15.31:0/4133975910 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c011bc0 con 0x7ff3b0159500 2023-10-06T05:17:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3acff9700 1 -- 172.21.15.31:0/4133975910 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c01b430 con 0x7ff3b0159500 2023-10-06T05:17:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.282+0000 7ff3b6fff700 1 -- 172.21.15.31:0/4133975910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b00ca380 con 0x7ff3b0159500 2023-10-06T05:17:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.283+0000 7ff3b6fff700 1 -- 172.21.15.31:0/4133975910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b015b260 con 0x7ff3b0159500 2023-10-06T05:17:45.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.283+0000 7ff3b6fff700 1 -- 172.21.15.31:0/4133975910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b004c380 con 0x7ff3b0159500 2023-10-06T05:17:45.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.284+0000 7ff3acff9700 1 -- 172.21.15.31:0/4133975910 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff39c01b860 con 0x7ff3b0159500 2023-10-06T05:17:45.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.285+0000 7ff3acff9700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff394066f30 0x7ff3940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.285+0000 7ff3acff9700 1 -- 172.21.15.31:0/4133975910 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff39c086430 con 0x7ff3b0159500 2023-10-06T05:17:45.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.286+0000 7ff3b4d9b700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff394066f30 0x7ff3940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.286+0000 7ff3b4d9b700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff394066f30 0x7ff3940693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4007fd0 tx=0x7ff3a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.289+0000 7ff3acff9700 1 -- 172.21.15.31:0/4133975910 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff39c0534a0 con 0x7ff3b0159500 2023-10-06T05:17:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.534+0000 7ff3b6fff700 1 -- 172.21.15.31:0/4133975910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3b00cade0 con 0x7ff3b0159500 2023-10-06T05:17:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.535+0000 7ff3acff9700 1 -- 172.21.15.31:0/4133975910 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7ff39c056ac0 con 0x7ff3b0159500 2023-10-06T05:17:45.537 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:45.537 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1001195925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1001195925},{"type":"v1","addr":"172.21.15.62:6835","nonce":1001195925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1336279472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1336279472},{"type":"v1","addr":"172.21.15.31:6837","nonce":1336279472}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/577485679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":577485679},{"type":"v1","addr":"172.21.15.62:6834","nonce":577485679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:01.930460+0000","modified":"2023-10-06T05:17:05.459675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4169737410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169737410},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169737410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:17:45.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 -- 172.21.15.31:0/4133975910 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff394066f30 msgr2=0x7ff3940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff394066f30 0x7ff3940693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4007fd0 tx=0x7ff3a4009040 comp rx=0 tx=0).stop 2023-10-06T05:17:45.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 -- 172.21.15.31:0/4133975910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b0159500 msgr2=0x7ff3b015bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b0159500 0x7ff3b015bd50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff39c005d40 tx=0x7ff39c016000 comp rx=0 tx=0).stop 2023-10-06T05:17:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 -- 172.21.15.31:0/4133975910 shutdown_connections 2023-10-06T05:17:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3b015d2d0 0x7ff3b00c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b00cd510 0x7ff3b00b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff394066f30 0x7ff3940693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 --2- 172.21.15.31:0/4133975910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b0159500 0x7ff3b015bd50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 -- 172.21.15.31:0/4133975910 >> 172.21.15.31:0/4133975910 conn(0x7ff3b006ec90 msgr2=0x7ff3b00cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 -- 172.21.15.31:0/4133975910 shutdown_connections 2023-10-06T05:17:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.537+0000 7ff3b6fff700 1 -- 172.21.15.31:0/4133975910 wait complete. 2023-10-06T05:17:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T05:17:45.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:45.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.749+0000 7f10ee1ff700 1 Processor -- start 2023-10-06T05:17:45.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.750+0000 7f10ee1ff700 1 -- start start 2023-10-06T05:17:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.751+0000 7f10ee1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10e80c7b60 0x7f10e8069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.751+0000 7f10ee1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f10e80c7f30 0x7f10e80aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.751+0000 7f10ee1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e80ca5b0 con 0x7f10e806ef50 2023-10-06T05:17:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.751+0000 7f10ee1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e806f740 con 0x7f10e80c7b60 2023-10-06T05:17:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.751+0000 7f10ee1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e80c9c80 con 0x7f10e80c7f30 2023-10-06T05:17:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.751+0000 7f10e77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10e80c7b60 0x7f10e8069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.751+0000 7f10e77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10e80c7b60 0x7f10e8069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57704/0 (socket says 172.21.15.31:57704) 2023-10-06T05:17:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.751+0000 7f10e77fe700 1 -- 172.21.15.31:0/2651521650 learned_addr learned my addr 172.21.15.31:0/2651521650 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.751+0000 7f10e6ffd700 1 --2- 172.21.15.31:0/2651521650 >> v2:172.21.15.62:3300/0 conn(0x7f10e80c7f30 0x7f10e80aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.752+0000 7f10e77fe700 1 -- 172.21.15.31:0/2651521650 >> v2:172.21.15.62:3300/0 conn(0x7f10e80c7f30 msgr2=0x7f10e80aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.752+0000 7f10e77fe700 1 --2- 172.21.15.31:0/2651521650 >> v2:172.21.15.62:3300/0 conn(0x7f10e80c7f30 0x7f10e80aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.752+0000 7f10e77fe700 1 -- 172.21.15.31:0/2651521650 >> v1:172.21.15.62:6789/0 conn(0x7f10e806ef50 legacy=0x7f10e813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.752+0000 7f10e77fe700 1 -- 172.21.15.31:0/2651521650 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e80c9f60 con 0x7f10e80c7b60 2023-10-06T05:17:45.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.752+0000 7f10e77fe700 1 --2- 172.21.15.31:0/2651521650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 0x7f10e8069880 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009990 tx=0x7f10d8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c151eb5acd2c38a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:45.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.752+0000 7f10e67fc700 1 -- 172.21.15.31:0/2651521650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d8005c10 con 0x7f10e80c7b60 2023-10-06T05:17:45.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.752+0000 7f10e67fc700 1 -- 172.21.15.31:0/2651521650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d8005d70 con 0x7f10e80c7b60 2023-10-06T05:17:45.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.752+0000 7f10e67fc700 1 -- 172.21.15.31:0/2651521650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d800b840 con 0x7f10e80c7b60 2023-10-06T05:17:45.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.753+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2651521650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 msgr2=0x7f10e8069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.753+0000 7f10ee1ff700 1 --2- 172.21.15.31:0/2651521650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 0x7f10e8069880 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009990 tx=0x7f10d8009220 comp rx=0 tx=0).stop 2023-10-06T05:17:45.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.753+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2651521650 shutdown_connections 2023-10-06T05:17:45.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.753+0000 7f10ee1ff700 1 --2- 172.21.15.31:0/2651521650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 0x7f10e8069880 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.753+0000 7f10ee1ff700 1 --2- 172.21.15.31:0/2651521650 >> v2:172.21.15.62:3300/0 conn(0x7f10e80c7f30 0x7f10e80aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.753+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2651521650 >> 172.21.15.31:0/2651521650 conn(0x7f10e80cc0b0 msgr2=0x7f10e80cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.753+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2651521650 shutdown_connections 2023-10-06T05:17:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.753+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2651521650 wait complete. 2023-10-06T05:17:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.754+0000 7f10ee1ff700 1 Processor -- start 2023-10-06T05:17:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.755+0000 7f10ee1ff700 1 -- start start 2023-10-06T05:17:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.755+0000 7f10ee1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10e806ef50 0x7f10e80c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.755+0000 7f10ee1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 0x7f10e80c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.755+0000 7f10ee1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10e80c7f30 0x7f10e80afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.755+0000 7f10ee1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80c5c90 con 0x7f10e80c7b60 2023-10-06T05:17:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.755+0000 7f10ee1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80c5f70 con 0x7f10e806ef50 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.755+0000 7f10ee1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80c6250 con 0x7f10e80c7f30 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 0x7f10e80c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 0x7f10e80c2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57718/0 (socket says 172.21.15.31:57718) 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e6ffd700 1 -- 172.21.15.31:0/2885568591 learned_addr learned my addr 172.21.15.31:0/2885568591 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e77fe700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10e806ef50 0x7f10e80c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e7fff700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10e80c7f30 0x7f10e80afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e6ffd700 1 -- 172.21.15.31:0/2885568591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10e80c7f30 msgr2=0x7f10e80afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e6ffd700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10e80c7f30 0x7f10e80afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e6ffd700 1 -- 172.21.15.31:0/2885568591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10e806ef50 msgr2=0x7f10e80c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e6ffd700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10e806ef50 0x7f10e80c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e6ffd700 1 -- 172.21.15.31:0/2885568591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d8009480 con 0x7f10e80c7b60 2023-10-06T05:17:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e7fff700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10e80c7f30 0x7f10e80afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.756+0000 7f10e6ffd700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 0x7f10e80c2630 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f10dc010e30 tx=0x7f10dc00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.757+0000 7f10cf7fe700 1 -- 172.21.15.31:0/2885568591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc00ed90 con 0x7f10e80c7b60 2023-10-06T05:17:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.757+0000 7f10cf7fe700 1 -- 172.21.15.31:0/2885568591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc019440 con 0x7f10e80c7b60 2023-10-06T05:17:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.757+0000 7f10cf7fe700 1 -- 172.21.15.31:0/2885568591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc019800 con 0x7f10e80c7b60 2023-10-06T05:17:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.757+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2885568591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e80c6530 con 0x7f10e80c7b60 2023-10-06T05:17:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.757+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2885568591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e80c36a0 con 0x7f10e80c7b60 2023-10-06T05:17:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.758+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2885568591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e804c380 con 0x7f10e80c7b60 2023-10-06T05:17:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.759+0000 7f10cf7fe700 1 -- 172.21.15.31:0/2885568591 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10dc0113f0 con 0x7f10e80c7b60 2023-10-06T05:17:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.759+0000 7f10cf7fe700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10d4066f30 0x7f10d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.760+0000 7f10cf7fe700 1 -- 172.21.15.31:0/2885568591 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f10dc086f30 con 0x7f10e80c7b60 2023-10-06T05:17:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.760+0000 7f10e77fe700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10d4066f30 0x7f10d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.760+0000 7f10e77fe700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10d4066f30 0x7f10d40693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009260 tx=0x7f10d8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:45.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.764+0000 7f10cf7fe700 1 -- 172.21.15.31:0/2885568591 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10dc053fa0 con 0x7f10e80c7b60 2023-10-06T05:17:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.964+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2885568591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10e80c0ef0 con 0x7f10e80c7b60 2023-10-06T05:17:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.965+0000 7f10cf7fe700 1 -- 172.21.15.31:0/2885568591 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f10dc0575c0 con 0x7f10e80c7b60 2023-10-06T05:17:45.967 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:45.967 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2885568591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10d4066f30 msgr2=0x7f10d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10d4066f30 0x7f10d40693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009260 tx=0x7f10d8016000 comp rx=0 tx=0).stop 2023-10-06T05:17:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2885568591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 msgr2=0x7f10e80c2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 0x7f10e80c2630 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f10dc010e30 tx=0x7f10dc00b630 comp rx=0 tx=0).stop 2023-10-06T05:17:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2885568591 shutdown_connections 2023-10-06T05:17:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10e80c7f30 0x7f10e80afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10e806ef50 0x7f10e80c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10d4066f30 0x7f10d40693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 --2- 172.21.15.31:0/2885568591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10e80c7b60 0x7f10e80c2630 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2885568591 >> 172.21.15.31:0/2885568591 conn(0x7f10e80cc0b0 msgr2=0x7f10e813dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2885568591 shutdown_connections 2023-10-06T05:17:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:45.967+0000 7f10ee1ff700 1 -- 172.21.15.31:0/2885568591 wait complete. 2023-10-06T05:17:45.987 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:17:45.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:46.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.183+0000 7f41239ff700 1 Processor -- start 2023-10-06T05:17:46.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.183+0000 7f41239ff700 1 -- start start 2023-10-06T05:17:46.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.185+0000 7f41239ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f411c0cd510 0x7f411c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.185+0000 7f41239ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f411c15d2d0 0x7f411c0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.185+0000 7f41239ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c0c3f40 con 0x7f411c159500 2023-10-06T05:17:46.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.185+0000 7f41239ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c0b2860 con 0x7f411c15d2d0 2023-10-06T05:17:46.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.185+0000 7f41239ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c15d7d0 con 0x7f411c0cd510 2023-10-06T05:17:46.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4121f9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f411c15d2d0 0x7f411c0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4120f9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f411c0cd510 0x7f411c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4121f9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f411c15d2d0 0x7f411c0aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:37218/0 (socket says 172.21.15.31:37218) 2023-10-06T05:17:46.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4121f9c700 1 -- 172.21.15.31:0/3569906589 learned_addr learned my addr 172.21.15.31:0/3569906589 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:46.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f412179b700 1 --1- 172.21.15.31:0/3569906589 >> v1:172.21.15.62:6789/0 conn(0x7f411c159500 0x7f411c0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:40094/0 (socket says 172.21.15.31:40094) 2023-10-06T05:17:46.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4113fff700 1 -- 172.21.15.31:0/3569906589 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80055842 0 0) 0x7f411c0c3f40 con 0x7f411c159500 2023-10-06T05:17:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4113fff700 1 -- 172.21.15.31:0/3569906589 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4108003570 con 0x7f411c159500 2023-10-06T05:17:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4121f9c700 1 -- 172.21.15.31:0/3569906589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f411c0cd510 msgr2=0x7f411c0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4121f9c700 1 --2- 172.21.15.31:0/3569906589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f411c0cd510 0x7f411c0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4121f9c700 1 -- 172.21.15.31:0/3569906589 >> v1:172.21.15.62:6789/0 conn(0x7f411c159500 legacy=0x7f411c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4121f9c700 1 -- 172.21.15.31:0/3569906589 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0cb000 con 0x7f411c15d2d0 2023-10-06T05:17:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.186+0000 7f4120f9a700 1 --2- 172.21.15.31:0/3569906589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f411c0cd510 0x7f411c0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.187+0000 7f4121f9c700 1 --2- 172.21.15.31:0/3569906589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f411c15d2d0 0x7f411c0aedd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4114009f20 tx=0x7f4114009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb7f1c4d3e3dab03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.187+0000 7f4113fff700 1 -- 172.21.15.31:0/3569906589 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411400e070 con 0x7f411c15d2d0 2023-10-06T05:17:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.187+0000 7f4113fff700 1 -- 172.21.15.31:0/3569906589 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411400b970 con 0x7f411c15d2d0 2023-10-06T05:17:46.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.187+0000 7f4113fff700 1 -- 172.21.15.31:0/3569906589 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41140092e0 con 0x7f411c15d2d0 2023-10-06T05:17:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.187+0000 7f41239ff700 1 -- 172.21.15.31:0/3569906589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f411c15d2d0 msgr2=0x7f411c0aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.187+0000 7f41239ff700 1 --2- 172.21.15.31:0/3569906589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f411c15d2d0 0x7f411c0aedd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4114009f20 tx=0x7f4114009840 comp rx=0 tx=0).stop 2023-10-06T05:17:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.187+0000 7f41239ff700 1 -- 172.21.15.31:0/3569906589 shutdown_connections 2023-10-06T05:17:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.187+0000 7f41239ff700 1 --2- 172.21.15.31:0/3569906589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f411c15d2d0 0x7f411c0aedd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.187+0000 7f41239ff700 1 --2- 172.21.15.31:0/3569906589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f411c0cd510 0x7f411c0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.187+0000 7f41239ff700 1 -- 172.21.15.31:0/3569906589 >> 172.21.15.31:0/3569906589 conn(0x7f411c06ec90 msgr2=0x7f411c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.188+0000 7f41239ff700 1 -- 172.21.15.31:0/3569906589 shutdown_connections 2023-10-06T05:17:46.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.188+0000 7f41239ff700 1 -- 172.21.15.31:0/3569906589 wait complete. 2023-10-06T05:17:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.190+0000 7f41239ff700 1 Processor -- start 2023-10-06T05:17:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.190+0000 7f41239ff700 1 -- start start 2023-10-06T05:17:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.191+0000 7f41239ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f411c0cd510 0x7f411c0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.191+0000 7f41239ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f411c159500 0x7f411c1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.191+0000 7f41239ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f411c15d2d0 0x7f411c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.191+0000 7f41239ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c15bf80 con 0x7f411c159500 2023-10-06T05:17:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.191+0000 7f41239ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c158f30 con 0x7f411c15d2d0 2023-10-06T05:17:46.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.191+0000 7f41239ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c15b0c0 con 0x7f411c0cd510 2023-10-06T05:17:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.191+0000 7f4120f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f411c159500 0x7f411c1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.191+0000 7f4120f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f411c159500 0x7f411c1589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57726/0 (socket says 172.21.15.31:57726) 2023-10-06T05:17:46.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.191+0000 7f4120f9a700 1 -- 172.21.15.31:0/2310388578 learned_addr learned my addr 172.21.15.31:0/2310388578 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f4120f9a700 1 -- 172.21.15.31:0/2310388578 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f411c0cd510 msgr2=0x7f411c0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f4120f9a700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f411c0cd510 0x7f411c0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f4120f9a700 1 -- 172.21.15.31:0/2310388578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f411c15d2d0 msgr2=0x7f411c15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f4121f9c700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f411c15d2d0 0x7f411c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f4120f9a700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f411c15d2d0 0x7f411c15ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f4120f9a700 1 -- 172.21.15.31:0/2310388578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0093c0 con 0x7f411c159500 2023-10-06T05:17:46.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f4121f9c700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f411c15d2d0 0x7f411c15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f4120f9a700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f411c159500 0x7f411c1589f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f410c00ec80 tx=0x7f410c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f41117fa700 1 -- 172.21.15.31:0/2310388578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c005a00 con 0x7f411c159500 2023-10-06T05:17:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f41117fa700 1 -- 172.21.15.31:0/2310388578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c005b60 con 0x7f411c159500 2023-10-06T05:17:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f41117fa700 1 -- 172.21.15.31:0/2310388578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c00c9d0 con 0x7f411c159500 2023-10-06T05:17:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.192+0000 7f41239ff700 1 -- 172.21.15.31:0/2310388578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4114009ae0 con 0x7f411c159500 2023-10-06T05:17:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.193+0000 7f41239ff700 1 -- 172.21.15.31:0/2310388578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c15b650 con 0x7f411c159500 2023-10-06T05:17:46.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.194+0000 7f41239ff700 1 -- 172.21.15.31:0/2310388578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c04c380 con 0x7f411c159500 2023-10-06T05:17:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.194+0000 7f41117fa700 1 -- 172.21.15.31:0/2310388578 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f410c00cd80 con 0x7f411c159500 2023-10-06T05:17:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.196+0000 7f41117fa700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4100066f30 0x7f41000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.196+0000 7f41117fa700 1 -- 172.21.15.31:0/2310388578 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f410c084d80 con 0x7f411c159500 2023-10-06T05:17:46.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.196+0000 7f412179b700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4100066f30 0x7f41000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.197+0000 7f412179b700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4100066f30 0x7f41000693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f41180016e0 tx=0x7f41180079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:46.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.200+0000 7f41117fa700 1 -- 172.21.15.31:0/2310388578 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f410c051df0 con 0x7f411c159500 2023-10-06T05:17:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.447+0000 7f41239ff700 1 -- 172.21.15.31:0/2310388578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f411c15c0c0 con 0x7f411c159500 2023-10-06T05:17:46.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.447+0000 7f41117fa700 1 -- 172.21.15.31:0/2310388578 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f410c055410 con 0x7f411c159500 2023-10-06T05:17:46.449 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:46.449 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1001195925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1001195925},{"type":"v1","addr":"172.21.15.62:6835","nonce":1001195925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1336279472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1336279472},{"type":"v1","addr":"172.21.15.31:6837","nonce":1336279472}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/577485679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":577485679},{"type":"v1","addr":"172.21.15.62:6834","nonce":577485679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:01.930460+0000","modified":"2023-10-06T05:17:05.459675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4169737410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169737410},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169737410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:17:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.449+0000 7f41239ff700 1 -- 172.21.15.31:0/2310388578 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4100066f30 msgr2=0x7f41000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.449+0000 7f41239ff700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4100066f30 0x7f41000693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f41180016e0 tx=0x7f41180079c0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.449+0000 7f41239ff700 1 -- 172.21.15.31:0/2310388578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f411c159500 msgr2=0x7f411c1589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.449+0000 7f41239ff700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f411c159500 0x7f411c1589f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f410c00ec80 tx=0x7f410c009120 comp rx=0 tx=0).stop 2023-10-06T05:17:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.450+0000 7f41239ff700 1 -- 172.21.15.31:0/2310388578 shutdown_connections 2023-10-06T05:17:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.450+0000 7f41239ff700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f411c15d2d0 0x7f411c15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.450+0000 7f41239ff700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f411c0cd510 0x7f411c0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.450+0000 7f41239ff700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4100066f30 0x7f41000693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.450+0000 7f41239ff700 1 --2- 172.21.15.31:0/2310388578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f411c159500 0x7f411c1589f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.450+0000 7f41239ff700 1 -- 172.21.15.31:0/2310388578 >> 172.21.15.31:0/2310388578 conn(0x7f411c06ec90 msgr2=0x7f411c0cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:46.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.450+0000 7f41239ff700 1 -- 172.21.15.31:0/2310388578 shutdown_connections 2023-10-06T05:17:46.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.450+0000 7f41239ff700 1 -- 172.21.15.31:0/2310388578 wait complete. 2023-10-06T05:17:46.453 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T05:17:46.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:17:46.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.662+0000 7f8e69fff700 1 Processor -- start 2023-10-06T05:17:46.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.662+0000 7f8e69fff700 1 -- start start 2023-10-06T05:17:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e69fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e640c7f30 0x7f8e64069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e69fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640aee70 0x7f8e6413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e69fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e640ca5b0 con 0x7f8e640c7b60 2023-10-06T05:17:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e69fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e640af2f0 con 0x7f8e640c7f30 2023-10-06T05:17:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e69fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6406f740 con 0x7f8e640aee70 2023-10-06T05:17:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e62ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e640c7f30 0x7f8e64069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e62ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e640c7f30 0x7f8e64069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57742/0 (socket says 172.21.15.31:57742) 2023-10-06T05:17:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e62ffd700 1 -- 172.21.15.31:0/2378273142 learned_addr learned my addr 172.21.15.31:0/2378273142 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e637fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8e640c7b60 0x7f8e6406ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49350/0 (socket says 172.21.15.31:49350) 2023-10-06T05:17:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e63fff700 1 --2- 172.21.15.31:0/2378273142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640aee70 0x7f8e6413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e62ffd700 1 -- 172.21.15.31:0/2378273142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640aee70 msgr2=0x7f8e6413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e62ffd700 1 --2- 172.21.15.31:0/2378273142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640aee70 0x7f8e6413f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e62ffd700 1 -- 172.21.15.31:0/2378273142 >> v1:172.21.15.31:6789/0 conn(0x7f8e640c7b60 legacy=0x7f8e6406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.665+0000 7f8e62ffd700 1 -- 172.21.15.31:0/2378273142 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e640c9c80 con 0x7f8e640c7f30 2023-10-06T05:17:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.664+0000 7f8e627fc700 1 -- 172.21.15.31:0/2378273142 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988462230 0 0) 0x7f8e640ca5b0 con 0x7f8e640c7b60 2023-10-06T05:17:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.665+0000 7f8e62ffd700 1 --2- 172.21.15.31:0/2378273142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640c7f30 0x7f8e64069880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8e580099b0 tx=0x7f8e58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5566bedf4360e94e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.665+0000 7f8e627fc700 1 -- 172.21.15.31:0/2378273142 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58009ea0 con 0x7f8e640c7f30 2023-10-06T05:17:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.665+0000 7f8e627fc700 1 -- 172.21.15.31:0/2378273142 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5800b860 con 0x7f8e640c7f30 2023-10-06T05:17:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.665+0000 7f8e627fc700 1 -- 172.21.15.31:0/2378273142 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800bc00 con 0x7f8e640c7f30 2023-10-06T05:17:46.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.666+0000 7f8e69fff700 1 -- 172.21.15.31:0/2378273142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640c7f30 msgr2=0x7f8e64069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.666+0000 7f8e69fff700 1 --2- 172.21.15.31:0/2378273142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640c7f30 0x7f8e64069880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8e580099b0 tx=0x7f8e58009250 comp rx=0 tx=0).stop 2023-10-06T05:17:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.666+0000 7f8e69fff700 1 -- 172.21.15.31:0/2378273142 shutdown_connections 2023-10-06T05:17:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.666+0000 7f8e69fff700 1 --2- 172.21.15.31:0/2378273142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640aee70 0x7f8e6413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.666+0000 7f8e69fff700 1 --2- 172.21.15.31:0/2378273142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640c7f30 0x7f8e64069880 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.666+0000 7f8e69fff700 1 -- 172.21.15.31:0/2378273142 >> 172.21.15.31:0/2378273142 conn(0x7f8e640cc0b0 msgr2=0x7f8e640cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.666+0000 7f8e69fff700 1 -- 172.21.15.31:0/2378273142 shutdown_connections 2023-10-06T05:17:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.666+0000 7f8e69fff700 1 -- 172.21.15.31:0/2378273142 wait complete. 2023-10-06T05:17:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.667+0000 7f8e69fff700 1 Processor -- start 2023-10-06T05:17:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.667+0000 7f8e69fff700 1 -- start start 2023-10-06T05:17:46.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.668+0000 7f8e69fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640aee70 0x7f8e6413d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.668+0000 7f8e69fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e640c7b60 0x7f8e641557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.668+0000 7f8e69fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640c7f30 0x7f8e6416b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.668+0000 7f8e69fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6413de00 con 0x7f8e640aee70 2023-10-06T05:17:46.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.668+0000 7f8e69fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e64155d30 con 0x7f8e640c7b60 2023-10-06T05:17:46.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.668+0000 7f8e69fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6413adc0 con 0x7f8e640c7f30 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640aee70 0x7f8e6413d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640aee70 0x7f8e6413d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57754/0 (socket says 172.21.15.31:57754) 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e637fe700 1 -- 172.21.15.31:0/2470099044 learned_addr learned my addr 172.21.15.31:0/2470099044 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e63fff700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640c7f30 0x7f8e6416b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e62ffd700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e640c7b60 0x7f8e641557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e637fe700 1 -- 172.21.15.31:0/2470099044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640c7f30 msgr2=0x7f8e6416b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e637fe700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640c7f30 0x7f8e6416b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e637fe700 1 -- 172.21.15.31:0/2470099044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e640c7b60 msgr2=0x7f8e641557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e637fe700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e640c7b60 0x7f8e641557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e637fe700 1 -- 172.21.15.31:0/2470099044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e580094f0 con 0x7f8e640aee70 2023-10-06T05:17:46.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e63fff700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640c7f30 0x7f8e6416b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:46.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.669+0000 7f8e637fe700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640aee70 0x7f8e6413d8c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400c810 tx=0x7f8e54007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:46.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.670+0000 7f8e4b7fe700 1 -- 172.21.15.31:0/2470099044 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54007300 con 0x7f8e640aee70 2023-10-06T05:17:46.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.670+0000 7f8e4b7fe700 1 -- 172.21.15.31:0/2470099044 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e54007460 con 0x7f8e640aee70 2023-10-06T05:17:46.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.670+0000 7f8e69fff700 1 -- 172.21.15.31:0/2470099044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6413b0a0 con 0x7f8e640aee70 2023-10-06T05:17:46.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.670+0000 7f8e4b7fe700 1 -- 172.21.15.31:0/2470099044 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e540057f0 con 0x7f8e640aee70 2023-10-06T05:17:46.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.670+0000 7f8e69fff700 1 -- 172.21.15.31:0/2470099044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6416afa0 con 0x7f8e640aee70 2023-10-06T05:17:46.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.671+0000 7f8e69fff700 1 -- 172.21.15.31:0/2470099044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6416b190 con 0x7f8e640aee70 2023-10-06T05:17:46.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.672+0000 7f8e4b7fe700 1 -- 172.21.15.31:0/2470099044 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e54005bc0 con 0x7f8e640aee70 2023-10-06T05:17:46.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.672+0000 7f8e4b7fe700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8e50066f30 0x7f8e500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:46.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.673+0000 7f8e4b7fe700 1 -- 172.21.15.31:0/2470099044 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8e540842b0 con 0x7f8e640aee70 2023-10-06T05:17:46.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.673+0000 7f8e62ffd700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8e50066f30 0x7f8e500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:46.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.674+0000 7f8e62ffd700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8e50066f30 0x7f8e500693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8e64069dc0 tx=0x7f8e58017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:46.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.677+0000 7f8e4b7fe700 1 -- 172.21.15.31:0/2470099044 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e540513a0 con 0x7f8e640aee70 2023-10-06T05:17:46.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.923+0000 7f8e69fff700 1 -- 172.21.15.31:0/2470099044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e6404c380 con 0x7f8e640aee70 2023-10-06T05:17:46.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.924+0000 7f8e4b7fe700 1 -- 172.21.15.31:0/2470099044 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f8e540549c0 con 0x7f8e640aee70 2023-10-06T05:17:46.925 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:46.926 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1001195925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1001195925},{"type":"v1","addr":"172.21.15.62:6835","nonce":1001195925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1336279472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1336279472},{"type":"v1","addr":"172.21.15.31:6837","nonce":1336279472}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/577485679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":577485679},{"type":"v1","addr":"172.21.15.62:6834","nonce":577485679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:17:01.930460+0000","modified":"2023-10-06T05:17:05.459675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4169737410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4169737410},{"type":"v1","addr":"172.21.15.31:6835","nonce":4169737410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T05:17:46.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 -- 172.21.15.31:0/2470099044 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8e50066f30 msgr2=0x7f8e500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8e50066f30 0x7f8e500693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8e64069dc0 tx=0x7f8e58017000 comp rx=0 tx=0).stop 2023-10-06T05:17:46.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 -- 172.21.15.31:0/2470099044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640aee70 msgr2=0x7f8e6413d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:46.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640aee70 0x7f8e6413d8c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400c810 tx=0x7f8e54007d20 comp rx=0 tx=0).stop 2023-10-06T05:17:46.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 -- 172.21.15.31:0/2470099044 shutdown_connections 2023-10-06T05:17:46.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e640c7f30 0x7f8e6416b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8e50066f30 0x7f8e500693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e640aee70 0x7f8e6413d8c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 --2- 172.21.15.31:0/2470099044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e640c7b60 0x7f8e641557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:46.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 -- 172.21.15.31:0/2470099044 >> 172.21.15.31:0/2470099044 conn(0x7f8e640cc0b0 msgr2=0x7f8e640ad910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:46.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 -- 172.21.15.31:0/2470099044 shutdown_connections 2023-10-06T05:17:46.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:46.926+0000 7f8e69fff700 1 -- 172.21.15.31:0/2470099044 wait complete. 2023-10-06T05:17:46.929 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-10-06T05:17:46.945 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:47.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.138+0000 7fe7e95ff700 1 Processor -- start 2023-10-06T05:17:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.139+0000 7fe7e95ff700 1 -- start start 2023-10-06T05:17:47.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.140+0000 7fe7e95ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e4159500 0x7fe7e40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.140+0000 7fe7e95ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7e40cd510 0x7fe7e40aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.140+0000 7fe7e95ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e40c3f40 con 0x7fe7e415d2d0 2023-10-06T05:17:47.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.140+0000 7fe7e95ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e40b2860 con 0x7fe7e40cd510 2023-10-06T05:17:47.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.140+0000 7fe7e95ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e415d7d0 con 0x7fe7e4159500 2023-10-06T05:17:47.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.140+0000 7fe7e37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7e40cd510 0x7fe7e40aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.140+0000 7fe7e37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7e40cd510 0x7fe7e40aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39116/0 (socket says 172.21.15.31:39116) 2023-10-06T05:17:47.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.140+0000 7fe7e37fe700 1 -- 172.21.15.31:0/1650818698 learned_addr learned my addr 172.21.15.31:0/1650818698 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:47.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.140+0000 7fe7e2ffd700 1 --2- 172.21.15.31:0/1650818698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e4159500 0x7fe7e40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.141+0000 7fe7e37fe700 1 -- 172.21.15.31:0/1650818698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e4159500 msgr2=0x7fe7e40b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:47.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.141+0000 7fe7e37fe700 1 --2- 172.21.15.31:0/1650818698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e4159500 0x7fe7e40b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.141+0000 7fe7e37fe700 1 -- 172.21.15.31:0/1650818698 >> v1:172.21.15.62:6789/0 conn(0x7fe7e415d2d0 legacy=0x7fe7e40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.141+0000 7fe7e37fe700 1 -- 172.21.15.31:0/1650818698 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e40cb000 con 0x7fe7e40cd510 2023-10-06T05:17:47.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.141+0000 7fe7e37fe700 1 --2- 172.21.15.31:0/1650818698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 0x7fe7e40aedd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009f20 tx=0x7fe7dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceef70f05806641 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.141+0000 7fe7e1ffb700 1 -- 172.21.15.31:0/1650818698 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc00e070 con 0x7fe7e40cd510 2023-10-06T05:17:47.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.141+0000 7fe7e1ffb700 1 -- 172.21.15.31:0/1650818698 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc00ba80 con 0x7fe7e40cd510 2023-10-06T05:17:47.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.141+0000 7fe7e1ffb700 1 -- 172.21.15.31:0/1650818698 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc0092e0 con 0x7fe7e40cd510 2023-10-06T05:17:47.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.142+0000 7fe7e95ff700 1 -- 172.21.15.31:0/1650818698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 msgr2=0x7fe7e40aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.142+0000 7fe7e95ff700 1 --2- 172.21.15.31:0/1650818698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 0x7fe7e40aedd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009f20 tx=0x7fe7dc009840 comp rx=0 tx=0).stop 2023-10-06T05:17:47.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.142+0000 7fe7e95ff700 1 -- 172.21.15.31:0/1650818698 shutdown_connections 2023-10-06T05:17:47.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.142+0000 7fe7e95ff700 1 --2- 172.21.15.31:0/1650818698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 0x7fe7e40aedd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.142+0000 7fe7e95ff700 1 --2- 172.21.15.31:0/1650818698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e4159500 0x7fe7e40b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.142+0000 7fe7e95ff700 1 -- 172.21.15.31:0/1650818698 >> 172.21.15.31:0/1650818698 conn(0x7fe7e406ec90 msgr2=0x7fe7e406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:47.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.142+0000 7fe7e95ff700 1 -- 172.21.15.31:0/1650818698 shutdown_connections 2023-10-06T05:17:47.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.142+0000 7fe7e95ff700 1 -- 172.21.15.31:0/1650818698 wait complete. 2023-10-06T05:17:47.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.143+0000 7fe7e95ff700 1 Processor -- start 2023-10-06T05:17:47.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.143+0000 7fe7e95ff700 1 -- start start 2023-10-06T05:17:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.144+0000 7fe7e95ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 0x7fe7e41536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.144+0000 7fe7e95ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e4159500 0x7fe7e40c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.144+0000 7fe7e95ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e415d2d0 0x7fe7e40c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.144+0000 7fe7e95ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e40c8f20 con 0x7fe7e40cd510 2023-10-06T05:17:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.144+0000 7fe7e95ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e4153c30 con 0x7fe7e4159500 2023-10-06T05:17:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.144+0000 7fe7e95ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e40c85f0 con 0x7fe7e415d2d0 2023-10-06T05:17:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.144+0000 7fe7e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 0x7fe7e41536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 0x7fe7e41536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39128/0 (socket says 172.21.15.31:39128) 2023-10-06T05:17:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e2ffd700 1 -- 172.21.15.31:0/873830320 learned_addr learned my addr 172.21.15.31:0/873830320 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e27fc700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e4159500 0x7fe7e40c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e37fe700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e415d2d0 0x7fe7e40c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e2ffd700 1 -- 172.21.15.31:0/873830320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e415d2d0 msgr2=0x7fe7e40c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e2ffd700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e415d2d0 0x7fe7e40c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e2ffd700 1 -- 172.21.15.31:0/873830320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e4159500 msgr2=0x7fe7e40c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e2ffd700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e4159500 0x7fe7e40c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e2ffd700 1 -- 172.21.15.31:0/873830320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc009ae0 con 0x7fe7e40cd510 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e37fe700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e415d2d0 0x7fe7e40c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e27fc700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e4159500 0x7fe7e40c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e2ffd700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 0x7fe7e41536f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400d890 tx=0x7fe7d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7caffd700 1 -- 172.21.15.31:0/873830320 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d400b600 con 0x7fe7e40cd510 2023-10-06T05:17:47.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7caffd700 1 -- 172.21.15.31:0/873830320 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d400b760 con 0x7fe7e40cd510 2023-10-06T05:17:47.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7caffd700 1 -- 172.21.15.31:0/873830320 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d400bae0 con 0x7fe7e40cd510 2023-10-06T05:17:47.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.145+0000 7fe7e95ff700 1 -- 172.21.15.31:0/873830320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e40c88d0 con 0x7fe7e40cd510 2023-10-06T05:17:47.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.146+0000 7fe7e95ff700 1 -- 172.21.15.31:0/873830320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e40c8230 con 0x7fe7e40cd510 2023-10-06T05:17:47.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.147+0000 7fe7e95ff700 1 -- 172.21.15.31:0/873830320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e404c380 con 0x7fe7e40cd510 2023-10-06T05:17:47.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.147+0000 7fe7caffd700 1 -- 172.21.15.31:0/873830320 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7d401e460 con 0x7fe7e40cd510 2023-10-06T05:17:47.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.148+0000 7fe7caffd700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe7d0066f60 0x7fe7d0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.149+0000 7fe7caffd700 1 -- 172.21.15.31:0/873830320 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe7d4084de0 con 0x7fe7e40cd510 2023-10-06T05:17:47.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.149+0000 7fe7e27fc700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe7d0066f60 0x7fe7d0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.149+0000 7fe7e27fc700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe7d0066f60 0x7fe7d0069410 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe7e416cf60 tx=0x7fe7d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.152+0000 7fe7caffd700 1 -- 172.21.15.31:0/873830320 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7d4051e50 con 0x7fe7e40cd510 2023-10-06T05:17:47.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.357+0000 7fe7e95ff700 1 -- 172.21.15.31:0/873830320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7e40cade0 con 0x7fe7e40cd510 2023-10-06T05:17:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.358+0000 7fe7caffd700 1 -- 172.21.15.31:0/873830320 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fe7d4055470 con 0x7fe7e40cd510 2023-10-06T05:17:47.360 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:47.361 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:03.439992+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:16:59.507957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:00.905826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 -- 172.21.15.31:0/873830320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe7d0066f60 msgr2=0x7fe7d0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe7d0066f60 0x7fe7d0069410 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe7e416cf60 tx=0x7fe7d8009040 comp rx=0 tx=0).stop 2023-10-06T05:17:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 -- 172.21.15.31:0/873830320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 msgr2=0x7fe7e41536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 0x7fe7e41536f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400d890 tx=0x7fe7d4008040 comp rx=0 tx=0).stop 2023-10-06T05:17:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 -- 172.21.15.31:0/873830320 shutdown_connections 2023-10-06T05:17:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e415d2d0 0x7fe7e40c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe7d0066f60 0x7fe7d0069410 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40cd510 0x7fe7e41536f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 --2- 172.21.15.31:0/873830320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e4159500 0x7fe7e40c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 -- 172.21.15.31:0/873830320 >> 172.21.15.31:0/873830320 conn(0x7fe7e406ec90 msgr2=0x7fe7e40cfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:47.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 -- 172.21.15.31:0/873830320 shutdown_connections 2023-10-06T05:17:47.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.361+0000 7fe7e95ff700 1 -- 172.21.15.31:0/873830320 wait complete. 2023-10-06T05:17:47.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:17:47.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.574+0000 7f4924fff700 1 Processor -- start 2023-10-06T05:17:47.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.574+0000 7f4924fff700 1 -- start start 2023-10-06T05:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.575+0000 7f4924fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4920159500 0x7f49200b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.575+0000 7f4924fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492015d2d0 0x7f49200cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.575+0000 7f4924fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49200c3f40 con 0x7f49201598d0 2023-10-06T05:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.575+0000 7f4924fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49200b2860 con 0x7f4920159500 2023-10-06T05:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.575+0000 7f4924fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492015d7d0 con 0x7f492015d2d0 2023-10-06T05:17:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.576+0000 7f491dd9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492015d2d0 0x7f49200cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.576+0000 7f491e59c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4920159500 0x7f49200b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.576+0000 7f491e59c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4920159500 0x7f49200b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:56600/0 (socket says 172.21.15.31:56600) 2023-10-06T05:17:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.576+0000 7f491e59c700 1 -- 172.21.15.31:0/513798676 learned_addr learned my addr 172.21.15.31:0/513798676 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.576+0000 7f491dd9b700 1 --2- 172.21.15.31:0/513798676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492015d2d0 0x7f49200cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56582/0 (socket says 172.21.15.31:56582) 2023-10-06T05:17:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.576+0000 7f491ed9d700 1 --1- 172.21.15.31:0/513798676 >> v1:172.21.15.62:6789/0 conn(0x7f49201598d0 0x7f49200aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:51472/0 (socket says 172.21.15.31:51472) 2023-10-06T05:17:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.577+0000 7f491dd9b700 1 -- 172.21.15.31:0/513798676 >> v2:172.21.15.62:3300/0 conn(0x7f4920159500 msgr2=0x7f49200b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:47.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.577+0000 7f491dd9b700 1 --2- 172.21.15.31:0/513798676 >> v2:172.21.15.62:3300/0 conn(0x7f4920159500 0x7f49200b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.577+0000 7f491d59a700 1 -- 172.21.15.31:0/513798676 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697769542 0 0) 0x7f49200c3f40 con 0x7f49201598d0 2023-10-06T05:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.577+0000 7f491dd9b700 1 -- 172.21.15.31:0/513798676 >> v1:172.21.15.62:6789/0 conn(0x7f49201598d0 legacy=0x7f49200aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.577+0000 7f491dd9b700 1 -- 172.21.15.31:0/513798676 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49200cb000 con 0x7f492015d2d0 2023-10-06T05:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.577+0000 7f491dd9b700 1 --2- 172.21.15.31:0/513798676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492015d2d0 0x7f49200cce00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49140098d0 tx=0x7f4914009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7309e7edd28d930b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.577+0000 7f491d59a700 1 -- 172.21.15.31:0/513798676 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914005c10 con 0x7f492015d2d0 2023-10-06T05:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.577+0000 7f491d59a700 1 -- 172.21.15.31:0/513798676 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4914005d70 con 0x7f492015d2d0 2023-10-06T05:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.577+0000 7f491d59a700 1 -- 172.21.15.31:0/513798676 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914006280 con 0x7f492015d2d0 2023-10-06T05:17:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.578+0000 7f4924fff700 1 -- 172.21.15.31:0/513798676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492015d2d0 msgr2=0x7f49200cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.578+0000 7f4924fff700 1 --2- 172.21.15.31:0/513798676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492015d2d0 0x7f49200cce00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49140098d0 tx=0x7f4914009120 comp rx=0 tx=0).stop 2023-10-06T05:17:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.578+0000 7f4924fff700 1 -- 172.21.15.31:0/513798676 shutdown_connections 2023-10-06T05:17:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.578+0000 7f4924fff700 1 --2- 172.21.15.31:0/513798676 >> v2:172.21.15.62:3300/0 conn(0x7f4920159500 0x7f49200b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.578+0000 7f4924fff700 1 --2- 172.21.15.31:0/513798676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492015d2d0 0x7f49200cce00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.578+0000 7f4924fff700 1 -- 172.21.15.31:0/513798676 >> 172.21.15.31:0/513798676 conn(0x7f492006ec90 msgr2=0x7f492006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.578+0000 7f4924fff700 1 -- 172.21.15.31:0/513798676 shutdown_connections 2023-10-06T05:17:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.578+0000 7f4924fff700 1 -- 172.21.15.31:0/513798676 wait complete. 2023-10-06T05:17:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.579+0000 7f4924fff700 1 Processor -- start 2023-10-06T05:17:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.580+0000 7f4924fff700 1 -- start start 2023-10-06T05:17:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.580+0000 7f4924fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4920159500 0x7f49200b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.580+0000 7f4924fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49201598d0 0x7f492016b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.580+0000 7f4924fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492015d2d0 0x7f492016ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.580+0000 7f4924fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49200b33c0 con 0x7f492015d2d0 2023-10-06T05:17:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.580+0000 7f4924fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49200b36a0 con 0x7f4920159500 2023-10-06T05:17:47.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.580+0000 7f4924fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49200b3980 con 0x7f49201598d0 2023-10-06T05:17:47.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492015d2d0 0x7f492016ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492015d2d0 0x7f492016ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39142/0 (socket says 172.21.15.31:39142) 2023-10-06T05:17:47.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491ed9d700 1 -- 172.21.15.31:0/1264287102 learned_addr learned my addr 172.21.15.31:0/1264287102 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:47.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491dd9b700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49201598d0 0x7f492016b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491e59c700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4920159500 0x7f49200b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491ed9d700 1 -- 172.21.15.31:0/1264287102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49201598d0 msgr2=0x7f492016b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491ed9d700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49201598d0 0x7f492016b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491ed9d700 1 -- 172.21.15.31:0/1264287102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4920159500 msgr2=0x7f49200b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491ed9d700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4920159500 0x7f49200b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491ed9d700 1 -- 172.21.15.31:0/1264287102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49140093c0 con 0x7f492015d2d0 2023-10-06T05:17:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.581+0000 7f491dd9b700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49201598d0 0x7f492016b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.582+0000 7f491ed9d700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492015d2d0 0x7f492016ba20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f491800cd00 tx=0x7f4918001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.582+0000 7f490e7fc700 1 -- 172.21.15.31:0/1264287102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491800ce60 con 0x7f492015d2d0 2023-10-06T05:17:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.582+0000 7f490e7fc700 1 -- 172.21.15.31:0/1264287102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4918005df0 con 0x7f492015d2d0 2023-10-06T05:17:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.582+0000 7f490e7fc700 1 -- 172.21.15.31:0/1264287102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491800aae0 con 0x7f492015d2d0 2023-10-06T05:17:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.582+0000 7f4924fff700 1 -- 172.21.15.31:0/1264287102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49200b3c60 con 0x7f492015d2d0 2023-10-06T05:17:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.582+0000 7f4924fff700 1 -- 172.21.15.31:0/1264287102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49200b49a0 con 0x7f492015d2d0 2023-10-06T05:17:47.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.583+0000 7f4924fff700 1 -- 172.21.15.31:0/1264287102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492004c380 con 0x7f492015d2d0 2023-10-06T05:17:47.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.584+0000 7f490e7fc700 1 -- 172.21.15.31:0/1264287102 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49180070f0 con 0x7f492015d2d0 2023-10-06T05:17:47.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.585+0000 7f490e7fc700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4900066f30 0x7f49000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:47.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.585+0000 7f490e7fc700 1 -- 172.21.15.31:0/1264287102 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4918084750 con 0x7f492015d2d0 2023-10-06T05:17:47.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.585+0000 7f491e59c700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4900066f30 0x7f49000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:47.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.586+0000 7f491e59c700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4900066f30 0x7f49000693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f49200b4f90 tx=0x7f4910009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:47.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.589+0000 7f490e7fc700 1 -- 172.21.15.31:0/1264287102 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4918051840 con 0x7f492015d2d0 2023-10-06T05:17:47.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.844+0000 7f4924fff700 1 -- 172.21.15.31:0/1264287102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f49200cade0 con 0x7f492015d2d0 2023-10-06T05:17:47.846 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:17:47.845+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:17:47.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.850+0000 7f490e7fc700 1 -- 172.21.15.31:0/1264287102 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4918054e60 con 0x7f492015d2d0 2023-10-06T05:17:47.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.853+0000 7f4924fff700 1 -- 172.21.15.31:0/1264287102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4900066f30 msgr2=0x7f49000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.853+0000 7f4924fff700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4900066f30 0x7f49000693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f49200b4f90 tx=0x7f4910009250 comp rx=0 tx=0).stop 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.853+0000 7f4924fff700 1 -- 172.21.15.31:0/1264287102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492015d2d0 msgr2=0x7f492016ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.853+0000 7f4924fff700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492015d2d0 0x7f492016ba20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f491800cd00 tx=0x7f4918001ac0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.854+0000 7f4924fff700 1 -- 172.21.15.31:0/1264287102 shutdown_connections 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.854+0000 7f4924fff700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4900066f30 0x7f49000693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.854+0000 7f4924fff700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492015d2d0 0x7f492016ba20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.854+0000 7f4924fff700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4920159500 0x7f49200b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.854+0000 7f4924fff700 1 --2- 172.21.15.31:0/1264287102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49201598d0 0x7f492016b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.854+0000 7f4924fff700 1 -- 172.21.15.31:0/1264287102 >> 172.21.15.31:0/1264287102 conn(0x7f492006ec90 msgr2=0x7f49200c52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.854+0000 7f4924fff700 1 -- 172.21.15.31:0/1264287102 shutdown_connections 2023-10-06T05:17:47.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:47.854+0000 7f4924fff700 1 -- 172.21.15.31:0/1264287102 wait complete. 2023-10-06T05:17:47.857 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:17:47.861 INFO:tasks.ceph.mds.a.smithi031.stderr: -2185> 2023-10-06T05:17:06.606+0000 7fcb381f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:17:47.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:17:47.911 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:17:48.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.077+0000 7fb3defff700 1 Processor -- start 2023-10-06T05:17:48.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.077+0000 7fb3defff700 1 -- start start 2023-10-06T05:17:48.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.078+0000 7fb3defff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d8153d40 0x7fb3d80b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.078+0000 7fb3defff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3d80af890 0x7fb3d80adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.078+0000 7fb3defff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d80ad690 con 0x7fb3d80b09c0 2023-10-06T05:17:48.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.078+0000 7fb3defff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d80afcd0 con 0x7fb3d80af890 2023-10-06T05:17:48.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.078+0000 7fb3defff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80b0e30 con 0x7fb3d8153d40 2023-10-06T05:17:48.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.079+0000 7fb3d7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb3d80b09c0 0x7fb3d80af180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60620/0 (socket says 172.21.15.31:60620) 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.079+0000 7fb3d7fff700 1 -- 172.21.15.31:0/2840467411 learned_addr learned my addr 172.21.15.31:0/2840467411 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.079+0000 7fb3dd59c700 1 --2- 172.21.15.31:0/2840467411 >> v2:172.21.15.31:3300/0 conn(0x7fb3d80af890 0x7fb3d80adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.079+0000 7fb3dcd9b700 1 --2- 172.21.15.31:0/2840467411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d8153d40 0x7fb3d80b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.079+0000 7fb3d77fe700 1 -- 172.21.15.31:0/2840467411 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475446639 0 0) 0x7fb3d80ad690 con 0x7fb3d80b09c0 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.079+0000 7fb3d77fe700 1 -- 172.21.15.31:0/2840467411 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3c8003570 con 0x7fb3d80b09c0 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.079+0000 7fb3dd59c700 1 -- 172.21.15.31:0/2840467411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d8153d40 msgr2=0x7fb3d80b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.079+0000 7fb3dd59c700 1 --2- 172.21.15.31:0/2840467411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d8153d40 0x7fb3d80b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.079+0000 7fb3dd59c700 1 -- 172.21.15.31:0/2840467411 >> v1:172.21.15.31:6789/0 conn(0x7fb3d80b09c0 legacy=0x7fb3d80af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.079+0000 7fb3dd59c700 1 -- 172.21.15.31:0/2840467411 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d80acd60 con 0x7fb3d80af890 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.080+0000 7fb3dd59c700 1 --2- 172.21.15.31:0/2840467411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80af890 0x7fb3d80adb20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009ec0 tx=0x7fb3d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7790d009ad0a39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:48.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.080+0000 7fb3d77fe700 1 -- 172.21.15.31:0/2840467411 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000bb00 con 0x7fb3d80af890 2023-10-06T05:17:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.080+0000 7fb3d77fe700 1 -- 172.21.15.31:0/2840467411 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d000bc60 con 0x7fb3d80af890 2023-10-06T05:17:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.080+0000 7fb3d77fe700 1 -- 172.21.15.31:0/2840467411 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0011460 con 0x7fb3d80af890 2023-10-06T05:17:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.081+0000 7fb3defff700 1 -- 172.21.15.31:0/2840467411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80af890 msgr2=0x7fb3d80adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.081+0000 7fb3defff700 1 --2- 172.21.15.31:0/2840467411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80af890 0x7fb3d80adb20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009ec0 tx=0x7fb3d0009780 comp rx=0 tx=0).stop 2023-10-06T05:17:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.081+0000 7fb3defff700 1 -- 172.21.15.31:0/2840467411 shutdown_connections 2023-10-06T05:17:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.081+0000 7fb3defff700 1 --2- 172.21.15.31:0/2840467411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80af890 0x7fb3d80adb20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.081+0000 7fb3defff700 1 --2- 172.21.15.31:0/2840467411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d8153d40 0x7fb3d80b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.081+0000 7fb3defff700 1 -- 172.21.15.31:0/2840467411 >> 172.21.15.31:0/2840467411 conn(0x7fb3d8067560 msgr2=0x7fb3d8067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.081+0000 7fb3defff700 1 -- 172.21.15.31:0/2840467411 shutdown_connections 2023-10-06T05:17:48.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.081+0000 7fb3defff700 1 -- 172.21.15.31:0/2840467411 wait complete. 2023-10-06T05:17:48.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.083+0000 7fb3defff700 1 Processor -- start 2023-10-06T05:17:48.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.084+0000 7fb3defff700 1 -- start start 2023-10-06T05:17:48.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.085+0000 7fb3defff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d80af890 0x7fb3d806eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.085+0000 7fb3defff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80b09c0 0x7fb3d80c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.085+0000 7fb3defff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d8153d40 0x7fb3d80c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.085+0000 7fb3defff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80c7590 con 0x7fb3d80b09c0 2023-10-06T05:17:48.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.085+0000 7fb3defff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80c6710 con 0x7fb3d8153d40 2023-10-06T05:17:48.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.085+0000 7fb3defff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80c6980 con 0x7fb3d80af890 2023-10-06T05:17:48.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.085+0000 7fb3d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80b09c0 0x7fb3d80c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.085+0000 7fb3d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80b09c0 0x7fb3d80c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39180/0 (socket says 172.21.15.31:39180) 2023-10-06T05:17:48.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.085+0000 7fb3d7fff700 1 -- 172.21.15.31:0/2637763392 learned_addr learned my addr 172.21.15.31:0/2637763392 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:48.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.086+0000 7fb3dcd9b700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d80af890 0x7fb3d806eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.086+0000 7fb3dd59c700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d8153d40 0x7fb3d80c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.086+0000 7fb3d7fff700 1 -- 172.21.15.31:0/2637763392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d80af890 msgr2=0x7fb3d806eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.086+0000 7fb3d7fff700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d80af890 0x7fb3d806eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.086+0000 7fb3d7fff700 1 -- 172.21.15.31:0/2637763392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d8153d40 msgr2=0x7fb3d80c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.086+0000 7fb3d7fff700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d8153d40 0x7fb3d80c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.086+0000 7fb3d7fff700 1 -- 172.21.15.31:0/2637763392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d0009a20 con 0x7fb3d80b09c0 2023-10-06T05:17:48.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.086+0000 7fb3dd59c700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d8153d40 0x7fb3d80c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:48.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.086+0000 7fb3d7fff700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80b09c0 0x7fb3d80c5c90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400c7f0 tx=0x7fb3c4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:48.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.086+0000 7fb3d4ff9700 1 -- 172.21.15.31:0/2637763392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4005b30 con 0x7fb3d80b09c0 2023-10-06T05:17:48.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.087+0000 7fb3d4ff9700 1 -- 172.21.15.31:0/2637763392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c4005cb0 con 0x7fb3d80b09c0 2023-10-06T05:17:48.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.087+0000 7fb3defff700 1 -- 172.21.15.31:0/2637763392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d80c6c60 con 0x7fb3d80b09c0 2023-10-06T05:17:48.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.087+0000 7fb3d4ff9700 1 -- 172.21.15.31:0/2637763392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4007320 con 0x7fb3d80b09c0 2023-10-06T05:17:48.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.087+0000 7fb3defff700 1 -- 172.21.15.31:0/2637763392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d80c7220 con 0x7fb3d80b09c0 2023-10-06T05:17:48.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.088+0000 7fb3defff700 1 -- 172.21.15.31:0/2637763392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d80c7740 con 0x7fb3d80b09c0 2023-10-06T05:17:48.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.089+0000 7fb3d4ff9700 1 -- 172.21.15.31:0/2637763392 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3c401b3b0 con 0x7fb3d80b09c0 2023-10-06T05:17:48.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.089+0000 7fb3d4ff9700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb3bc066f30 0x7fb3bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:48.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.089+0000 7fb3d4ff9700 1 -- 172.21.15.31:0/2637763392 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb3c4083d50 con 0x7fb3d80b09c0 2023-10-06T05:17:48.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.090+0000 7fb3dcd9b700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb3bc066f30 0x7fb3bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:48.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.090+0000 7fb3dcd9b700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb3bc066f30 0x7fb3bc0693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009d10 tx=0x7fb3cc0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:48.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.093+0000 7fb3d4ff9700 1 -- 172.21.15.31:0/2637763392 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3d80c7740 con 0x7fb3d80b09c0 2023-10-06T05:17:48.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.342+0000 7fb3defff700 1 -- 172.21.15.31:0/2637763392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb3d80c7740 con 0x7fb3d80b09c0 2023-10-06T05:17:48.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.852+0000 7fb3d4ff9700 1 -- 172.21.15.31:0/2637763392 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb3d80c7740 con 0x7fb3d80b09c0 2023-10-06T05:17:48.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 -- 172.21.15.31:0/2637763392 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb3bc066f30 msgr2=0x7fb3bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb3bc066f30 0x7fb3bc0693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009d10 tx=0x7fb3cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 -- 172.21.15.31:0/2637763392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80b09c0 msgr2=0x7fb3d80c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:48.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80b09c0 0x7fb3d80c5c90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400c7f0 tx=0x7fb3c4007e00 comp rx=0 tx=0).stop 2023-10-06T05:17:48.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 -- 172.21.15.31:0/2637763392 shutdown_connections 2023-10-06T05:17:48.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d8153d40 0x7fb3d80c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d80af890 0x7fb3d806eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb3bc066f30 0x7fb3bc0693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 --2- 172.21.15.31:0/2637763392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d80b09c0 0x7fb3d80c5c90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:48.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 -- 172.21.15.31:0/2637763392 >> 172.21.15.31:0/2637763392 conn(0x7fb3d8067560 msgr2=0x7fb3d80aa420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:48.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.855+0000 7fb3defff700 1 -- 172.21.15.31:0/2637763392 shutdown_connections 2023-10-06T05:17:48.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:48.856+0000 7fb3defff700 1 -- 172.21.15.31:0/2637763392 wait complete. 2023-10-06T05:17:48.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:17:49.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.123+0000 7fd5bf3ff700 1 Processor -- start 2023-10-06T05:17:49.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.123+0000 7fd5bf3ff700 1 -- start start 2023-10-06T05:17:49.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5bf3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd5b8153950 0x7fd5b8153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5bf3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5b815d920 0x7fd5b815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5bf3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b815bb20 con 0x7fd5b80cd5d0 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5bf3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b815af10 con 0x7fd5b815d920 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5bf3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b815b1f0 con 0x7fd5b8153950 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5bc99a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd5b80cd5d0 0x7fd5b815d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60636/0 (socket says 172.21.15.31:60636) 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5bc99a700 1 -- 172.21.15.31:0/4062336882 learned_addr learned my addr 172.21.15.31:0/4062336882 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5bd99c700 1 --2- 172.21.15.31:0/4062336882 >> v2:172.21.15.31:3300/0 conn(0x7fd5b815d920 0x7fd5b815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5bd19b700 1 --2- 172.21.15.31:0/4062336882 >> v2:172.21.15.62:3300/0 conn(0x7fd5b8153950 0x7fd5b8153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5affff700 1 -- 172.21.15.31:0/4062336882 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132798008 0 0) 0x7fd5b815bb20 con 0x7fd5b80cd5d0 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.125+0000 7fd5affff700 1 -- 172.21.15.31:0/4062336882 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a4003570 con 0x7fd5b80cd5d0 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.126+0000 7fd5bd99c700 1 -- 172.21.15.31:0/4062336882 >> v2:172.21.15.62:3300/0 conn(0x7fd5b8153950 msgr2=0x7fd5b8153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.126+0000 7fd5bd99c700 1 --2- 172.21.15.31:0/4062336882 >> v2:172.21.15.62:3300/0 conn(0x7fd5b8153950 0x7fd5b8153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.126+0000 7fd5bd99c700 1 -- 172.21.15.31:0/4062336882 >> v1:172.21.15.31:6789/0 conn(0x7fd5b80cd5d0 legacy=0x7fd5b815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.126+0000 7fd5bd99c700 1 -- 172.21.15.31:0/4062336882 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b815b4d0 con 0x7fd5b815d920 2023-10-06T05:17:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.126+0000 7fd5bd19b700 1 --2- 172.21.15.31:0/4062336882 >> v2:172.21.15.62:3300/0 conn(0x7fd5b8153950 0x7fd5b8153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.126+0000 7fd5bd99c700 1 --2- 172.21.15.31:0/4062336882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b815d920 0x7fd5b815bfb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80d15b0 tx=0x7fd5b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173ae01bd7b37593 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.126+0000 7fd5affff700 1 -- 172.21.15.31:0/4062336882 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000e070 con 0x7fd5b815d920 2023-10-06T05:17:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.126+0000 7fd5affff700 1 -- 172.21.15.31:0/4062336882 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b000bb60 con 0x7fd5b815d920 2023-10-06T05:17:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.126+0000 7fd5affff700 1 -- 172.21.15.31:0/4062336882 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0012430 con 0x7fd5b815d920 2023-10-06T05:17:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.127+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/4062336882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b815d920 msgr2=0x7fd5b815bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.127+0000 7fd5bf3ff700 1 --2- 172.21.15.31:0/4062336882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b815d920 0x7fd5b815bfb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80d15b0 tx=0x7fd5b0009780 comp rx=0 tx=0).stop 2023-10-06T05:17:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.127+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/4062336882 shutdown_connections 2023-10-06T05:17:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.127+0000 7fd5bf3ff700 1 --2- 172.21.15.31:0/4062336882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b815d920 0x7fd5b815bfb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.127+0000 7fd5bf3ff700 1 --2- 172.21.15.31:0/4062336882 >> v2:172.21.15.62:3300/0 conn(0x7fd5b8153950 0x7fd5b8153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.127+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/4062336882 >> 172.21.15.31:0/4062336882 conn(0x7fd5b80681b0 msgr2=0x7fd5b80685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.127+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/4062336882 shutdown_connections 2023-10-06T05:17:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.127+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/4062336882 wait complete. 2023-10-06T05:17:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.128+0000 7fd5bf3ff700 1 Processor -- start 2023-10-06T05:17:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.128+0000 7fd5bf3ff700 1 -- start start 2023-10-06T05:17:49.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.129+0000 7fd5bf3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b80cd5d0 0x7fd5b80b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.129+0000 7fd5bf3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b8153950 0x7fd5b80ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.129+0000 7fd5bf3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b815d920 0x7fd5b81509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.129+0000 7fd5bf3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b80b6060 con 0x7fd5b8153950 2023-10-06T05:17:49.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.129+0000 7fd5bf3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b81588e0 con 0x7fd5b80cd5d0 2023-10-06T05:17:49.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.129+0000 7fd5bf3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8151a60 con 0x7fd5b815d920 2023-10-06T05:17:49.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bc99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b8153950 0x7fd5b80ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bc99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b8153950 0x7fd5b80ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39204/0 (socket says 172.21.15.31:39204) 2023-10-06T05:17:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bc99a700 1 -- 172.21.15.31:0/1808123196 learned_addr learned my addr 172.21.15.31:0/1808123196 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bd19b700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b80cd5d0 0x7fd5b80b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bd99c700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b815d920 0x7fd5b81509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bc99a700 1 -- 172.21.15.31:0/1808123196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b815d920 msgr2=0x7fd5b81509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bc99a700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b815d920 0x7fd5b81509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bc99a700 1 -- 172.21.15.31:0/1808123196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b80cd5d0 msgr2=0x7fd5b80b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bc99a700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b80cd5d0 0x7fd5b80b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bc99a700 1 -- 172.21.15.31:0/1808123196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b0009a20 con 0x7fd5b8153950 2023-10-06T05:17:49.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bd99c700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b815d920 0x7fd5b81509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:49.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.130+0000 7fd5bc99a700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b8153950 0x7fd5b80ca000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800c590 tx=0x7fd5a8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:49.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.131+0000 7fd5ad7fa700 1 -- 172.21.15.31:0/1808123196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8005cd0 con 0x7fd5b8153950 2023-10-06T05:17:49.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.131+0000 7fd5ad7fa700 1 -- 172.21.15.31:0/1808123196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a8005e50 con 0x7fd5b8153950 2023-10-06T05:17:49.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.131+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/1808123196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b80ca540 con 0x7fd5b8153950 2023-10-06T05:17:49.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.131+0000 7fd5ad7fa700 1 -- 172.21.15.31:0/1808123196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8007320 con 0x7fd5b8153950 2023-10-06T05:17:49.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.131+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/1808123196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b80b5aa0 con 0x7fd5b8153950 2023-10-06T05:17:49.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.132+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/1808123196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b804c380 con 0x7fd5b8153950 2023-10-06T05:17:49.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.133+0000 7fd5ad7fa700 1 -- 172.21.15.31:0/1808123196 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5a801b3b0 con 0x7fd5b8153950 2023-10-06T05:17:49.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.134+0000 7fd5ad7fa700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd59c066f30 0x7fd59c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:49.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.134+0000 7fd5ad7fa700 1 -- 172.21.15.31:0/1808123196 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fd5a8083eb0 con 0x7fd5b8153950 2023-10-06T05:17:49.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.134+0000 7fd5bd19b700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd59c066f30 0x7fd59c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:49.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.135+0000 7fd5bd19b700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd59c066f30 0x7fd59c0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80cb920 tx=0x7fd5b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:49.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.139+0000 7fd5ad7fa700 1 -- 172.21.15.31:0/1808123196 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5a8050e90 con 0x7fd5b8153950 2023-10-06T05:17:49.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.344+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/1808123196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd5b815b7b0 con 0x7fd5b8153950 2023-10-06T05:17:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.859+0000 7fd5ad7fa700 1 -- 172.21.15.31:0/1808123196 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd5a8017070 con 0x7fd5b8153950 2023-10-06T05:17:49.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.864+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/1808123196 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd59c066f30 msgr2=0x7fd59c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.864+0000 7fd5bf3ff700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd59c066f30 0x7fd59c0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80cb920 tx=0x7fd5b4008040 comp rx=0 tx=0).stop 2023-10-06T05:17:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.864+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/1808123196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b8153950 msgr2=0x7fd5b80ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.864+0000 7fd5bf3ff700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b8153950 0x7fd5b80ca000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800c590 tx=0x7fd5a8007e00 comp rx=0 tx=0).stop 2023-10-06T05:17:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.865+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/1808123196 shutdown_connections 2023-10-06T05:17:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.865+0000 7fd5bf3ff700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b815d920 0x7fd5b81509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.865+0000 7fd5bf3ff700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b80cd5d0 0x7fd5b80b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.865+0000 7fd5bf3ff700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd59c066f30 0x7fd59c0693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.865+0000 7fd5bf3ff700 1 --2- 172.21.15.31:0/1808123196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b8153950 0x7fd5b80ca000 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.865+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/1808123196 >> 172.21.15.31:0/1808123196 conn(0x7fd5b80681b0 msgr2=0x7fd5b8152460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.865+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/1808123196 shutdown_connections 2023-10-06T05:17:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:49.865+0000 7fd5bf3ff700 1 -- 172.21.15.31:0/1808123196 wait complete. 2023-10-06T05:17:49.872 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:17:49.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:17:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.097+0000 7f7941bff700 1 Processor -- start 2023-10-06T05:17:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.097+0000 7f7941bff700 1 -- start start 2023-10-06T05:17:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.098+0000 7f7941bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f793c153950 0x7f793c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.098+0000 7f7941bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f793c0cd5d0 0x7f793c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.098+0000 7f7941bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f793c0cda20 0x7f793c15d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.098+0000 7f7941bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c15bb20 con 0x7f793c0cd5d0 2023-10-06T05:17:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.098+0000 7f7941bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c15af10 con 0x7f793c153950 2023-10-06T05:17:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.098+0000 7f7941bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c15b1f0 con 0x7f793c0cda20 2023-10-06T05:17:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f793c0cd5d0 0x7f793c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f793c0cd5d0 0x7f793c15d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39216/0 (socket says 172.21.15.31:39216) 2023-10-06T05:17:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793affd700 1 -- 172.21.15.31:0/2474691183 learned_addr learned my addr 172.21.15.31:0/2474691183 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793b7fe700 1 --2- 172.21.15.31:0/2474691183 >> v2:172.21.15.62:3300/0 conn(0x7f793c153950 0x7f793c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793bfff700 1 --2- 172.21.15.31:0/2474691183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f793c0cda20 0x7f793c15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793affd700 1 -- 172.21.15.31:0/2474691183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f793c0cda20 msgr2=0x7f793c15d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793affd700 1 --2- 172.21.15.31:0/2474691183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f793c0cda20 0x7f793c15d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793affd700 1 -- 172.21.15.31:0/2474691183 >> v2:172.21.15.62:3300/0 conn(0x7f793c153950 msgr2=0x7f793c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793affd700 1 --2- 172.21.15.31:0/2474691183 >> v2:172.21.15.62:3300/0 conn(0x7f793c153950 0x7f793c153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793affd700 1 -- 172.21.15.31:0/2474691183 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c15b4d0 con 0x7f793c0cd5d0 2023-10-06T05:17:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.099+0000 7f793b7fe700 1 --2- 172.21.15.31:0/2474691183 >> v2:172.21.15.62:3300/0 conn(0x7f793c153950 0x7f793c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.100+0000 7f793affd700 1 --2- 172.21.15.31:0/2474691183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 0x7f793c15d210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f793c0d15b0 tx=0x7f7930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50097f31fed6b326 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.100+0000 7f793a7fc700 1 -- 172.21.15.31:0/2474691183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793000b640 con 0x7f793c0cd5d0 2023-10-06T05:17:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.100+0000 7f793a7fc700 1 -- 172.21.15.31:0/2474691183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793000b7a0 con 0x7f793c0cd5d0 2023-10-06T05:17:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.100+0000 7f793a7fc700 1 -- 172.21.15.31:0/2474691183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793000bb60 con 0x7f793c0cd5d0 2023-10-06T05:17:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.100+0000 7f7941bff700 1 -- 172.21.15.31:0/2474691183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 msgr2=0x7f793c15d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.100+0000 7f7941bff700 1 --2- 172.21.15.31:0/2474691183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 0x7f793c15d210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f793c0d15b0 tx=0x7f7930009250 comp rx=0 tx=0).stop 2023-10-06T05:17:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.100+0000 7f7941bff700 1 -- 172.21.15.31:0/2474691183 shutdown_connections 2023-10-06T05:17:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.100+0000 7f7941bff700 1 --2- 172.21.15.31:0/2474691183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f793c0cda20 0x7f793c15d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.101+0000 7f7941bff700 1 --2- 172.21.15.31:0/2474691183 >> v2:172.21.15.62:3300/0 conn(0x7f793c153950 0x7f793c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.101+0000 7f7941bff700 1 --2- 172.21.15.31:0/2474691183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 0x7f793c15d210 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.101+0000 7f7941bff700 1 -- 172.21.15.31:0/2474691183 >> 172.21.15.31:0/2474691183 conn(0x7f793c0681b0 msgr2=0x7f793c0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:50.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.101+0000 7f7941bff700 1 -- 172.21.15.31:0/2474691183 shutdown_connections 2023-10-06T05:17:50.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.101+0000 7f7941bff700 1 -- 172.21.15.31:0/2474691183 wait complete. 2023-10-06T05:17:50.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.102+0000 7f7941bff700 1 Processor -- start 2023-10-06T05:17:50.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.102+0000 7f7941bff700 1 -- start start 2023-10-06T05:17:50.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.103+0000 7f7941bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 0x7f793c14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.103+0000 7f7941bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793c0cda20 0x7f793c16b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.103+0000 7f7941bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f793c153950 0x7f793c0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.103+0000 7f7941bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c14f5f0 con 0x7f793c0cd5d0 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.103+0000 7f7941bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0b3ff0 con 0x7f793c0cda20 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.103+0000 7f7941bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0b33e0 con 0x7f793c153950 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.103+0000 7f793b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 0x7f793c14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.103+0000 7f793b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 0x7f793c14f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39222/0 (socket says 172.21.15.31:39222) 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f793b7fe700 1 -- 172.21.15.31:0/804631380 learned_addr learned my addr 172.21.15.31:0/804631380 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f793bfff700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f793c153950 0x7f793c0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f793affd700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793c0cda20 0x7f793c16b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f793b7fe700 1 -- 172.21.15.31:0/804631380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f793c153950 msgr2=0x7f793c0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f793b7fe700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f793c153950 0x7f793c0c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f793b7fe700 1 -- 172.21.15.31:0/804631380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793c0cda20 msgr2=0x7f793c16b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f793b7fe700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793c0cda20 0x7f793c16b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f793b7fe700 1 -- 172.21.15.31:0/804631380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c0093c0 con 0x7f793c0cd5d0 2023-10-06T05:17:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f793b7fe700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 0x7f793c14f0b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f792c00ec80 tx=0x7f792c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f79237fe700 1 -- 172.21.15.31:0/804631380 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c005a00 con 0x7f793c0cd5d0 2023-10-06T05:17:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f79237fe700 1 -- 172.21.15.31:0/804631380 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c005b60 con 0x7f793c0cd5d0 2023-10-06T05:17:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f7941bff700 1 -- 172.21.15.31:0/804631380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79300094f0 con 0x7f793c0cd5d0 2023-10-06T05:17:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.104+0000 7f79237fe700 1 -- 172.21.15.31:0/804631380 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c00c9d0 con 0x7f793c0cd5d0 2023-10-06T05:17:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.105+0000 7f7941bff700 1 -- 172.21.15.31:0/804631380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c0b39a0 con 0x7f793c0cd5d0 2023-10-06T05:17:50.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.106+0000 7f7941bff700 1 -- 172.21.15.31:0/804631380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c04c380 con 0x7f793c0cd5d0 2023-10-06T05:17:50.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.107+0000 7f79237fe700 1 -- 172.21.15.31:0/804631380 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f792c00cd80 con 0x7f793c0cd5d0 2023-10-06T05:17:50.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.107+0000 7f79237fe700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7928066f30 0x7f79280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:50.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.107+0000 7f79237fe700 1 -- 172.21.15.31:0/804631380 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f792c084c40 con 0x7f793c0cd5d0 2023-10-06T05:17:50.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.108+0000 7f793affd700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7928066f30 0x7f79280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:50.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.108+0000 7f793affd700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7928066f30 0x7f79280693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f793c0d15b0 tx=0x7f7930017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.111+0000 7f79237fe700 1 -- 172.21.15.31:0/804631380 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f792c051dd0 con 0x7f793c0cd5d0 2023-10-06T05:17:50.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:50.319+0000 7f7941bff700 1 -- 172.21.15.31:0/804631380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f793c15b7b0 con 0x7f793c0cd5d0 2023-10-06T05:17:51.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.189+0000 7f79237fe700 1 -- 172.21.15.31:0/804631380 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f792c0553f0 con 0x7f793c0cd5d0 2023-10-06T05:17:51.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.194+0000 7f7941bff700 1 -- 172.21.15.31:0/804631380 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7928066f30 msgr2=0x7f79280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.194+0000 7f7941bff700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7928066f30 0x7f79280693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f793c0d15b0 tx=0x7f7930017000 comp rx=0 tx=0).stop 2023-10-06T05:17:51.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.194+0000 7f7941bff700 1 -- 172.21.15.31:0/804631380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 msgr2=0x7f793c14f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.194+0000 7f7941bff700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 0x7f793c14f0b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f792c00ec80 tx=0x7f792c009120 comp rx=0 tx=0).stop 2023-10-06T05:17:51.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.194+0000 7f7941bff700 1 -- 172.21.15.31:0/804631380 shutdown_connections 2023-10-06T05:17:51.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.194+0000 7f7941bff700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f793c153950 0x7f793c0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.194+0000 7f7941bff700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7928066f30 0x7f79280693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.194+0000 7f7941bff700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f793c0cd5d0 0x7f793c14f0b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.194+0000 7f7941bff700 1 --2- 172.21.15.31:0/804631380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f793c0cda20 0x7f793c16b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.194+0000 7f7941bff700 1 -- 172.21.15.31:0/804631380 >> 172.21.15.31:0/804631380 conn(0x7f793c0681b0 msgr2=0x7f793c0cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:51.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.195+0000 7f7941bff700 1 -- 172.21.15.31:0/804631380 shutdown_connections 2023-10-06T05:17:51.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.195+0000 7f7941bff700 1 -- 172.21.15.31:0/804631380 wait complete. 2023-10-06T05:17:51.202 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:17:51.223 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:17:51.223 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:17:51.224 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:17:51.224 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:17:51.224 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:17:51.224 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:17:51.224 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:17:51.225 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:17:51.225 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:17:51.225 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:17:51.225 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:17:51.225 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:17:51.225 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:17:51.225 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:17:51.226 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:17:51.226 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:17:51.241 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:17:51.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:17:51.242 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:17:51.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:17:51.243 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:17:51.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:17:51.244 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:17:51.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:17:51.286 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:17:51.286 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:51.288 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:17:51.288 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:51.289 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:17:51.289 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:51.291 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:17:51.292 DEBUG:teuthology.parallel:result is None 2023-10-06T05:17:51.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:17:51.438 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:17:51.438 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:17:51.439 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:17:51.439 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:17:51.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.498+0000 7f0bc81ff700 1 Processor -- start 2023-10-06T05:17:51.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.498+0000 7f0bc81ff700 1 -- start start 2023-10-06T05:17:51.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.499+0000 7f0bc81ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 0x7f0bc00c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.499+0000 7f0bc81ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0bc006db50 0x7f0bc0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc81ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc00c71b0 con 0x7f0bc013a5b0 2023-10-06T05:17:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc81ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc00c7490 con 0x7f0bc006db50 2023-10-06T05:17:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc81ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00c7770 con 0x7f0bc013a1e0 2023-10-06T05:17:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc679c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0bc006db50 0x7f0bc0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc679c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0bc006db50 0x7f0bc0069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39240/0 (socket says 172.21.15.31:39240) 2023-10-06T05:17:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc679c700 1 -- 172.21.15.31:0/3208883710 learned_addr learned my addr 172.21.15.31:0/3208883710 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc5f9b700 1 --2- 172.21.15.31:0/3208883710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 0x7f0bc00c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc679c700 1 -- 172.21.15.31:0/3208883710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 msgr2=0x7f0bc00c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:51.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc679c700 1 --2- 172.21.15.31:0/3208883710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 0x7f0bc00c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc679c700 1 -- 172.21.15.31:0/3208883710 >> v1:172.21.15.62:6789/0 conn(0x7f0bc013a5b0 legacy=0x7f0bc006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.500+0000 7f0bc679c700 1 -- 172.21.15.31:0/3208883710 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc00c7a50 con 0x7f0bc006db50 2023-10-06T05:17:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.501+0000 7f0bc679c700 1 --2- 172.21.15.31:0/3208883710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc006db50 0x7f0bc0069560 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009f20 tx=0x7f0bb8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a876877b59b609e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.501+0000 7f0bc4f99700 1 -- 172.21.15.31:0/3208883710 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800e070 con 0x7f0bc006db50 2023-10-06T05:17:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.501+0000 7f0bc4f99700 1 -- 172.21.15.31:0/3208883710 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb800b9d0 con 0x7f0bc006db50 2023-10-06T05:17:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.501+0000 7f0bc4f99700 1 -- 172.21.15.31:0/3208883710 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80092e0 con 0x7f0bc006db50 2023-10-06T05:17:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.501+0000 7f0bc81ff700 1 -- 172.21.15.31:0/3208883710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc006db50 msgr2=0x7f0bc0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.501+0000 7f0bc81ff700 1 --2- 172.21.15.31:0/3208883710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc006db50 0x7f0bc0069560 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009f20 tx=0x7f0bb8009840 comp rx=0 tx=0).stop 2023-10-06T05:17:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.501+0000 7f0bc81ff700 1 -- 172.21.15.31:0/3208883710 shutdown_connections 2023-10-06T05:17:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.501+0000 7f0bc81ff700 1 --2- 172.21.15.31:0/3208883710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc006db50 0x7f0bc0069560 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.501+0000 7f0bc81ff700 1 --2- 172.21.15.31:0/3208883710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 0x7f0bc00c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.502+0000 7f0bc81ff700 1 -- 172.21.15.31:0/3208883710 >> 172.21.15.31:0/3208883710 conn(0x7f0bc00cc0b0 msgr2=0x7f0bc00cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.502+0000 7f0bc81ff700 1 -- 172.21.15.31:0/3208883710 shutdown_connections 2023-10-06T05:17:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.502+0000 7f0bc81ff700 1 -- 172.21.15.31:0/3208883710 wait complete. 2023-10-06T05:17:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.504+0000 7f0bc81ff700 1 Processor -- start 2023-10-06T05:17:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.504+0000 7f0bc81ff700 1 -- start start 2023-10-06T05:17:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc81ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc006db50 0x7f0bc0158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc81ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 0x7f0bc00c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc81ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc013a5b0 0x7f0bc0151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc81ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc0159d70 con 0x7f0bc013a5b0 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc81ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00c67e0 con 0x7f0bc006db50 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc81ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc0159160 con 0x7f0bc013a1e0 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc679c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc013a5b0 0x7f0bc0151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc679c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc013a5b0 0x7f0bc0151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39244/0 (socket says 172.21.15.31:39244) 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc679c700 1 -- 172.21.15.31:0/1059080435 learned_addr learned my addr 172.21.15.31:0/1059080435 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc5f9b700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc006db50 0x7f0bc0158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.505+0000 7f0bc579a700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 0x7f0bc00c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bc679c700 1 -- 172.21.15.31:0/1059080435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 msgr2=0x7f0bc00c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bc679c700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 0x7f0bc00c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bc679c700 1 -- 172.21.15.31:0/1059080435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc006db50 msgr2=0x7f0bc0158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:51.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bc679c700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc006db50 0x7f0bc0158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bc679c700 1 -- 172.21.15.31:0/1059080435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8009ae0 con 0x7f0bc013a5b0 2023-10-06T05:17:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bc579a700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 0x7f0bc00c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bc679c700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc013a5b0 0x7f0bc0151630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8012000 tx=0x7f0bb8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bb5ffb700 1 -- 172.21.15.31:0/1059080435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8008bc0 con 0x7f0bc013a5b0 2023-10-06T05:17:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bb5ffb700 1 -- 172.21.15.31:0/1059080435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb8008d60 con 0x7f0bc013a5b0 2023-10-06T05:17:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bc81ff700 1 -- 172.21.15.31:0/1059080435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc0159440 con 0x7f0bc013a5b0 2023-10-06T05:17:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.506+0000 7f0bb5ffb700 1 -- 172.21.15.31:0/1059080435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80266b0 con 0x7f0bc013a5b0 2023-10-06T05:17:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.507+0000 7f0bc81ff700 1 -- 172.21.15.31:0/1059080435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc0159a00 con 0x7f0bc013a5b0 2023-10-06T05:17:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.508+0000 7f0bc81ff700 1 -- 172.21.15.31:0/1059080435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc0159ee0 con 0x7f0bc013a5b0 2023-10-06T05:17:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.509+0000 7f0bb5ffb700 1 -- 172.21.15.31:0/1059080435 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0bb8003040 con 0x7f0bc013a5b0 2023-10-06T05:17:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.509+0000 7f0bb5ffb700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ba4066f30 0x7f0ba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.509+0000 7f0bb5ffb700 1 -- 172.21.15.31:0/1059080435 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f0bb8085d40 con 0x7f0bc013a5b0 2023-10-06T05:17:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.510+0000 7f0bc5f9b700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ba4066f30 0x7f0ba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.510+0000 7f0bc5f9b700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ba4066f30 0x7f0ba40693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009da0 tx=0x7f0bbc009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:51.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.514+0000 7f0bb5ffb700 1 -- 172.21.15.31:0/1059080435 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bb8053200 con 0x7f0bc013a5b0 2023-10-06T05:17:51.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:51.730+0000 7f0bc81ff700 1 -- 172.21.15.31:0/1059080435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0bc004c380 con 0x7f0bc013a5b0 2023-10-06T05:17:52.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.209+0000 7f0bb5ffb700 1 -- 172.21.15.31:0/1059080435 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0bb8056820 con 0x7f0bc013a5b0 2023-10-06T05:17:52.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 -- 172.21.15.31:0/1059080435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ba4066f30 msgr2=0x7f0ba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ba4066f30 0x7f0ba40693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009da0 tx=0x7f0bbc009120 comp rx=0 tx=0).stop 2023-10-06T05:17:52.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 -- 172.21.15.31:0/1059080435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc013a5b0 msgr2=0x7f0bc0151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc013a5b0 0x7f0bc0151630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8012000 tx=0x7f0bb8018000 comp rx=0 tx=0).stop 2023-10-06T05:17:52.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 -- 172.21.15.31:0/1059080435 shutdown_connections 2023-10-06T05:17:52.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ba4066f30 0x7f0ba40693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bc013a5b0 0x7f0bc0151630 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc006db50 0x7f0bc0158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 --2- 172.21.15.31:0/1059080435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc013a1e0 0x7f0bc00c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 -- 172.21.15.31:0/1059080435 >> 172.21.15.31:0/1059080435 conn(0x7f0bc00cc0b0 msgr2=0x7f0bc0139860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:52.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.212+0000 7f0bc81ff700 1 -- 172.21.15.31:0/1059080435 shutdown_connections 2023-10-06T05:17:52.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.213+0000 7f0bc81ff700 1 -- 172.21.15.31:0/1059080435 wait complete. 2023-10-06T05:17:52.216 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:17:52.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:17:52.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.436+0000 7f4d731ff700 1 Processor -- start 2023-10-06T05:17:52.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.436+0000 7f4d731ff700 1 -- start start 2023-10-06T05:17:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.437+0000 7f4d731ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c13fb00 0x7f4d6c06c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d731ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d6c13fed0 0x7f4d6c06e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d731ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c06be70 con 0x7f4d6c06ea90 2023-10-06T05:17:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d731ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6c06ef30 con 0x7f4d6c13fed0 2023-10-06T05:17:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d731ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c06f130 con 0x7f4d6c13fb00 2023-10-06T05:17:52.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d6bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d6c13fed0 0x7f4d6c06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d7179c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4d6c06ea90 0x7f4d6c0cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60670/0 (socket says 172.21.15.31:60670) 2023-10-06T05:17:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d7179c700 1 -- 172.21.15.31:0/1748920805 learned_addr learned my addr 172.21.15.31:0/1748920805 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d70f9b700 1 --2- 172.21.15.31:0/1748920805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c13fb00 0x7f4d6c06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d6bfff700 1 -- 172.21.15.31:0/1748920805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c13fb00 msgr2=0x7f4d6c06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d6bfff700 1 --2- 172.21.15.31:0/1748920805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c13fb00 0x7f4d6c06c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d6bfff700 1 -- 172.21.15.31:0/1748920805 >> v1:172.21.15.31:6789/0 conn(0x7f4d6c06ea90 legacy=0x7f4d6c0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d6bfff700 1 -- 172.21.15.31:0/1748920805 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c06f410 con 0x7f4d6c13fed0 2023-10-06T05:17:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.438+0000 7f4d70f9b700 1 --2- 172.21.15.31:0/1748920805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c13fb00 0x7f4d6c06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:52.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.439+0000 7f4d6bfff700 1 --2- 172.21.15.31:0/1748920805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fed0 0x7f4d6c06e550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4d580098d0 tx=0x7f4d58009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511d8ef912b618c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.439+0000 7f4d6b7fe700 1 -- 172.21.15.31:0/1748920805 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58009e70 con 0x7f4d6c13fed0 2023-10-06T05:17:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.439+0000 7f4d6b7fe700 1 -- 172.21.15.31:0/1748920805 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58005c10 con 0x7f4d6c13fed0 2023-10-06T05:17:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.439+0000 7f4d6b7fe700 1 -- 172.21.15.31:0/1748920805 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800b650 con 0x7f4d6c13fed0 2023-10-06T05:17:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.439+0000 7f4d731ff700 1 -- 172.21.15.31:0/1748920805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fed0 msgr2=0x7f4d6c06e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.439+0000 7f4d731ff700 1 --2- 172.21.15.31:0/1748920805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fed0 0x7f4d6c06e550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4d580098d0 tx=0x7f4d58009120 comp rx=0 tx=0).stop 2023-10-06T05:17:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.440+0000 7f4d731ff700 1 -- 172.21.15.31:0/1748920805 shutdown_connections 2023-10-06T05:17:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.440+0000 7f4d731ff700 1 --2- 172.21.15.31:0/1748920805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c13fb00 0x7f4d6c06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.440+0000 7f4d731ff700 1 --2- 172.21.15.31:0/1748920805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fed0 0x7f4d6c06e550 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.440+0000 7f4d731ff700 1 -- 172.21.15.31:0/1748920805 >> 172.21.15.31:0/1748920805 conn(0x7f4d6c0cdcb0 msgr2=0x7f4d6c0c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:52.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.440+0000 7f4d731ff700 1 -- 172.21.15.31:0/1748920805 shutdown_connections 2023-10-06T05:17:52.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.440+0000 7f4d731ff700 1 -- 172.21.15.31:0/1748920805 wait complete. 2023-10-06T05:17:52.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.441+0000 7f4d731ff700 1 Processor -- start 2023-10-06T05:17:52.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.441+0000 7f4d731ff700 1 -- start start 2023-10-06T05:17:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.442+0000 7f4d731ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c06ea90 0x7f4d6c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.442+0000 7f4d731ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fb00 0x7f4d6c159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.442+0000 7f4d731ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c13fed0 0x7f4d6c140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.442+0000 7f4d731ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c159570 con 0x7f4d6c13fb00 2023-10-06T05:17:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.442+0000 7f4d731ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c142110 con 0x7f4d6c13fed0 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.442+0000 7f4d731ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c141500 con 0x7f4d6c06ea90 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fb00 0x7f4d6c159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fb00 0x7f4d6c159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39260/0 (socket says 172.21.15.31:39260) 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d6bfff700 1 -- 172.21.15.31:0/1668911274 learned_addr learned my addr 172.21.15.31:0/1668911274 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d70f9b700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c06ea90 0x7f4d6c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d6bfff700 1 -- 172.21.15.31:0/1668911274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c06ea90 msgr2=0x7f4d6c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d7179c700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c13fed0 0x7f4d6c140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d6bfff700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c06ea90 0x7f4d6c15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d6bfff700 1 -- 172.21.15.31:0/1668911274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c13fed0 msgr2=0x7f4d6c140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d6bfff700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c13fed0 0x7f4d6c140ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d6bfff700 1 -- 172.21.15.31:0/1668911274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d580093c0 con 0x7f4d6c13fb00 2023-10-06T05:17:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.443+0000 7f4d6bfff700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fb00 0x7f4d6c159030 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800b3c0 tx=0x7f4d58016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.444+0000 7f4d68ff9700 1 -- 172.21.15.31:0/1668911274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58006310 con 0x7f4d6c13fb00 2023-10-06T05:17:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.444+0000 7f4d68ff9700 1 -- 172.21.15.31:0/1668911274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58006470 con 0x7f4d6c13fb00 2023-10-06T05:17:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.444+0000 7f4d68ff9700 1 -- 172.21.15.31:0/1668911274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58006830 con 0x7f4d6c13fb00 2023-10-06T05:17:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.444+0000 7f4d731ff700 1 -- 172.21.15.31:0/1668911274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c1417e0 con 0x7f4d6c13fb00 2023-10-06T05:17:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.444+0000 7f4d731ff700 1 -- 172.21.15.31:0/1668911274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c141da0 con 0x7f4d6c13fb00 2023-10-06T05:17:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.445+0000 7f4d731ff700 1 -- 172.21.15.31:0/1668911274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c1422c0 con 0x7f4d6c13fb00 2023-10-06T05:17:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.446+0000 7f4d68ff9700 1 -- 172.21.15.31:0/1668911274 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d5801b690 con 0x7f4d6c13fb00 2023-10-06T05:17:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.447+0000 7f4d68ff9700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.447+0000 7f4d68ff9700 1 -- 172.21.15.31:0/1668911274 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4d580864e0 con 0x7f4d6c13fb00 2023-10-06T05:17:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.447+0000 7f4d70f9b700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.447+0000 7f4d70f9b700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d50066f30 0x7f4d500693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4d60006f00 tx=0x7f4d60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.451+0000 7f4d68ff9700 1 -- 172.21.15.31:0/1668911274 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d6c1422c0 con 0x7f4d6c13fb00 2023-10-06T05:17:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.731+0000 7f4d731ff700 1 -- 172.21.15.31:0/1668911274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4d6c142280 con 0x7f4d6c13fb00 2023-10-06T05:17:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.731+0000 7f4d68ff9700 1 -- 172.21.15.31:0/1668911274 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f4d580538a0 con 0x7f4d6c13fb00 2023-10-06T05:17:52.733 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:52.733 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.735 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:17:52.736 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.737 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:17:52.737 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:17:52.737 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:17:52.737 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:17:52.739 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:52.740 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:17:52.742 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:17:52.743 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:17:52.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:17:52.746 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:17:52.747 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.748 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:17:52.748 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:17:52.748 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:17:52.748 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-mds", 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQA5mB9lIh4ZCBAAW9Yss6DUnlWOBfbHl/XkuA==", 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQA5mB9l0CkZCBAAcRCIK3BjzvVOCpn8sxmdfg==", 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T05:17:52.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-osd", 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQA5mB9lpzIZCBAAiHdqYlr3Z7oCauMZpJFP4g==", 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQA5mB9lUD0ZCBAADIfODKTgtBXpQfCIuJA1sg==", 2023-10-06T05:17:52.750 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQA5mB9lAkYZCBAAj7IMauMWGAcL+D1uJcGYMw==", 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.751 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQA5mB9l7k4ZCBAAs1X1bup6xAGReVgxO5ebDw==", 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:17:52.752 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:52.754 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:17:52.754 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:17:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 -- 172.21.15.31:0/1668911274 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d50066f30 msgr2=0x7f4d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d50066f30 0x7f4d500693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4d60006f00 tx=0x7f4d60008040 comp rx=0 tx=0).stop 2023-10-06T05:17:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 -- 172.21.15.31:0/1668911274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fb00 msgr2=0x7f4d6c159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fb00 0x7f4d6c159030 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800b3c0 tx=0x7f4d58016000 comp rx=0 tx=0).stop 2023-10-06T05:17:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 -- 172.21.15.31:0/1668911274 shutdown_connections 2023-10-06T05:17:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d6c13fed0 0x7f4d6c140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d6c06ea90 0x7f4d6c15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 --2- 172.21.15.31:0/1668911274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d6c13fb00 0x7f4d6c159030 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 -- 172.21.15.31:0/1668911274 >> 172.21.15.31:0/1668911274 conn(0x7f4d6c0cdcb0 msgr2=0x7f4d6c0c3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 -- 172.21.15.31:0/1668911274 shutdown_connections 2023-10-06T05:17:52.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.734+0000 7f4d731ff700 1 -- 172.21.15.31:0/1668911274 wait complete. 2023-10-06T05:17:52.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T05:17:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.951+0000 7ff6a5fff700 1 Processor -- start 2023-10-06T05:17:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.951+0000 7ff6a5fff700 1 -- start start 2023-10-06T05:17:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.952+0000 7ff6a5fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a0068d80 0x7ff6a0159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.952+0000 7ff6a5fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6a00cc0d0 0x7ff6a00c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.952+0000 7ff6a5fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a00c95b0 con 0x7ff6a01595a0 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.952+0000 7ff6a5fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a00c9890 con 0x7ff6a00cc0d0 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.952+0000 7ff6a5fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00c9b70 con 0x7ff6a0068d80 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6a00cc0d0 0x7ff6a00c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff6a01595a0 0x7ff6a0068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60684/0 (socket says 172.21.15.31:60684) 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69f7fe700 1 -- 172.21.15.31:0/1065645551 learned_addr learned my addr 172.21.15.31:0/1065645551 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69ffff700 1 -- 172.21.15.31:0/1065645551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a0068d80 msgr2=0x7ff6a0159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69effd700 1 --2- 172.21.15.31:0/1065645551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a0068d80 0x7ff6a0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69ffff700 1 --2- 172.21.15.31:0/1065645551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a0068d80 0x7ff6a0159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69e7fc700 1 -- 172.21.15.31:0/1065645551 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570286948 0 0) 0x7ff6a00c95b0 con 0x7ff6a01595a0 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69ffff700 1 -- 172.21.15.31:0/1065645551 >> v1:172.21.15.31:6789/0 conn(0x7ff6a01595a0 legacy=0x7ff6a0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69ffff700 1 -- 172.21.15.31:0/1065645551 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a00ca130 con 0x7ff6a00cc0d0 2023-10-06T05:17:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69effd700 1 --2- 172.21.15.31:0/1065645551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a0068d80 0x7ff6a0159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.953+0000 7ff69ffff700 1 --2- 172.21.15.31:0/1065645551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a00cc0d0 0x7ff6a00c8930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff698009e60 tx=0x7ff698009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f48862687adb8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff69e7fc700 1 -- 172.21.15.31:0/1065645551 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800e070 con 0x7ff6a00cc0d0 2023-10-06T05:17:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff69e7fc700 1 -- 172.21.15.31:0/1065645551 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69800bc30 con 0x7ff6a00cc0d0 2023-10-06T05:17:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff69e7fc700 1 -- 172.21.15.31:0/1065645551 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698012430 con 0x7ff6a00cc0d0 2023-10-06T05:17:52.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1065645551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a00cc0d0 msgr2=0x7ff6a00c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff6a5fff700 1 --2- 172.21.15.31:0/1065645551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a00cc0d0 0x7ff6a00c8930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff698009e60 tx=0x7ff698009780 comp rx=0 tx=0).stop 2023-10-06T05:17:52.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1065645551 shutdown_connections 2023-10-06T05:17:52.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff6a5fff700 1 --2- 172.21.15.31:0/1065645551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a00cc0d0 0x7ff6a00c8930 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff6a5fff700 1 --2- 172.21.15.31:0/1065645551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a0068d80 0x7ff6a0159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1065645551 >> 172.21.15.31:0/1065645551 conn(0x7ff6a006b620 msgr2=0x7ff6a006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:52.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1065645551 shutdown_connections 2023-10-06T05:17:52.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.954+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1065645551 wait complete. 2023-10-06T05:17:52.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.956+0000 7ff6a5fff700 1 Processor -- start 2023-10-06T05:17:52.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.956+0000 7ff6a5fff700 1 -- start start 2023-10-06T05:17:52.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff6a5fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6a0068d80 0x7ff6a00c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff6a5fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a00cc0d0 0x7ff6a006cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff6a5fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a01595a0 0x7ff6a006d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff6a5fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00c53a0 con 0x7ff6a01595a0 2023-10-06T05:17:52.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff6a5fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00c61c0 con 0x7ff6a0068d80 2023-10-06T05:17:52.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff6a5fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00c55b0 con 0x7ff6a00cc0d0 2023-10-06T05:17:52.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff69ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a01595a0 0x7ff6a006d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff69ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a01595a0 0x7ff6a006d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39278/0 (socket says 172.21.15.31:39278) 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff69ffff700 1 -- 172.21.15.31:0/1435672652 learned_addr learned my addr 172.21.15.31:0/1435672652 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff69effd700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a00cc0d0 0x7ff6a006cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.957+0000 7ff69f7fe700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6a0068d80 0x7ff6a00c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.958+0000 7ff69ffff700 1 -- 172.21.15.31:0/1435672652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a00cc0d0 msgr2=0x7ff6a006cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.958+0000 7ff69ffff700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a00cc0d0 0x7ff6a006cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.958+0000 7ff69ffff700 1 -- 172.21.15.31:0/1435672652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6a0068d80 msgr2=0x7ff6a00c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.958+0000 7ff69ffff700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6a0068d80 0x7ff6a00c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.958+0000 7ff69ffff700 1 -- 172.21.15.31:0/1435672652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698009a20 con 0x7ff6a01595a0 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.958+0000 7ff69ffff700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a01595a0 0x7ff6a006d2e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff6a006ee00 tx=0x7ff698019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.958+0000 7ff6877fe700 1 -- 172.21.15.31:0/1435672652 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698027470 con 0x7ff6a01595a0 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.958+0000 7ff6877fe700 1 -- 172.21.15.31:0/1435672652 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6980275f0 con 0x7ff6a01595a0 2023-10-06T05:17:52.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.958+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1435672652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a00c5890 con 0x7ff6a01595a0 2023-10-06T05:17:52.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.958+0000 7ff6877fe700 1 -- 172.21.15.31:0/1435672652 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6980279b0 con 0x7ff6a01595a0 2023-10-06T05:17:52.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.959+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1435672652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a00c5e50 con 0x7ff6a01595a0 2023-10-06T05:17:52.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.960+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1435672652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a004c380 con 0x7ff6a01595a0 2023-10-06T05:17:52.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.961+0000 7ff6877fe700 1 -- 172.21.15.31:0/1435672652 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff698024070 con 0x7ff6a01595a0 2023-10-06T05:17:52.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.961+0000 7ff6877fe700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff68c066f30 0x7ff68c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:52.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.961+0000 7ff6877fe700 1 -- 172.21.15.31:0/1435672652 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff698086b20 con 0x7ff6a01595a0 2023-10-06T05:17:52.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.962+0000 7ff69f7fe700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff68c066f30 0x7ff68c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:52.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.962+0000 7ff69f7fe700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff68c066f30 0x7ff68c0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6a006dc70 tx=0x7ff690007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:52.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:52.966+0000 7ff6877fe700 1 -- 172.21.15.31:0/1435672652 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff698053f80 con 0x7ff6a01595a0 2023-10-06T05:17:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.250+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1435672652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7ff6a00c6300 con 0x7ff6a01595a0 2023-10-06T05:17:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.254+0000 7ff6877fe700 1 -- 172.21.15.31:0/1435672652 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff6980575a0 con 0x7ff6a01595a0 2023-10-06T05:17:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1435672652 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff68c066f30 msgr2=0x7ff68c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff68c066f30 0x7ff68c0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6a006dc70 tx=0x7ff690007d30 comp rx=0 tx=0).stop 2023-10-06T05:17:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1435672652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a01595a0 msgr2=0x7ff6a006d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a01595a0 0x7ff6a006d2e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff6a006ee00 tx=0x7ff698019000 comp rx=0 tx=0).stop 2023-10-06T05:17:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1435672652 shutdown_connections 2023-10-06T05:17:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff68c066f30 0x7ff68c0693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a01595a0 0x7ff6a006d2e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6a0068d80 0x7ff6a00c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 --2- 172.21.15.31:0/1435672652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6a00cc0d0 0x7ff6a006cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1435672652 >> 172.21.15.31:0/1435672652 conn(0x7ff6a006b620 msgr2=0x7ff6a0157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:53.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1435672652 shutdown_connections 2023-10-06T05:17:53.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.257+0000 7ff6a5fff700 1 -- 172.21.15.31:0/1435672652 wait complete. 2023-10-06T05:17:53.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T05:17:53.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.470+0000 7ff4e0bff700 1 Processor -- start 2023-10-06T05:17:53.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.470+0000 7ff4e0bff700 1 -- start start 2023-10-06T05:17:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4e0bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 0x7ff4dc0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4e0bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff4dc15d2d0 0x7ff4dc0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4e0bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc0c3f40 con 0x7ff4dc159500 2023-10-06T05:17:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4e0bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4dc0b2860 con 0x7ff4dc15d2d0 2023-10-06T05:17:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4e0bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc15d7d0 con 0x7ff4dc0cd510 2023-10-06T05:17:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4da59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff4dc159500 0x7ff4dc0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60688/0 (socket says 172.21.15.31:60688) 2023-10-06T05:17:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4da59c700 1 -- 172.21.15.31:0/895352546 learned_addr learned my addr 172.21.15.31:0/895352546 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4dad9d700 1 --2- 172.21.15.31:0/895352546 >> v2:172.21.15.62:3300/0 conn(0x7ff4dc15d2d0 0x7ff4dc0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4d9d9b700 1 --2- 172.21.15.31:0/895352546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 0x7ff4dc0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4d959a700 1 -- 172.21.15.31:0/895352546 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50170068 0 0) 0x7ff4dc0c3f40 con 0x7ff4dc159500 2023-10-06T05:17:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.472+0000 7ff4d959a700 1 -- 172.21.15.31:0/895352546 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4c4003570 con 0x7ff4dc159500 2023-10-06T05:17:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.473+0000 7ff4dad9d700 1 -- 172.21.15.31:0/895352546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 msgr2=0x7ff4dc0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.473+0000 7ff4dad9d700 1 --2- 172.21.15.31:0/895352546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 0x7ff4dc0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.473+0000 7ff4dad9d700 1 -- 172.21.15.31:0/895352546 >> v1:172.21.15.31:6789/0 conn(0x7ff4dc159500 legacy=0x7ff4dc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.473+0000 7ff4dad9d700 1 -- 172.21.15.31:0/895352546 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc0cb000 con 0x7ff4dc15d2d0 2023-10-06T05:17:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.473+0000 7ff4d9d9b700 1 --2- 172.21.15.31:0/895352546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 0x7ff4dc0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.473+0000 7ff4dad9d700 1 --2- 172.21.15.31:0/895352546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4dc15d2d0 0x7ff4dc0aedd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009e60 tx=0x7ff4d4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523a4c4419e23299 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.473+0000 7ff4d959a700 1 -- 172.21.15.31:0/895352546 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d400e070 con 0x7ff4dc15d2d0 2023-10-06T05:17:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.473+0000 7ff4d959a700 1 -- 172.21.15.31:0/895352546 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d400bab0 con 0x7ff4dc15d2d0 2023-10-06T05:17:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.473+0000 7ff4d959a700 1 -- 172.21.15.31:0/895352546 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d400be70 con 0x7ff4dc15d2d0 2023-10-06T05:17:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.474+0000 7ff4e0bff700 1 -- 172.21.15.31:0/895352546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4dc15d2d0 msgr2=0x7ff4dc0aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.474+0000 7ff4e0bff700 1 --2- 172.21.15.31:0/895352546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4dc15d2d0 0x7ff4dc0aedd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009e60 tx=0x7ff4d4009780 comp rx=0 tx=0).stop 2023-10-06T05:17:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.474+0000 7ff4e0bff700 1 -- 172.21.15.31:0/895352546 shutdown_connections 2023-10-06T05:17:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.474+0000 7ff4e0bff700 1 --2- 172.21.15.31:0/895352546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4dc15d2d0 0x7ff4dc0aedd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.474+0000 7ff4e0bff700 1 --2- 172.21.15.31:0/895352546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 0x7ff4dc0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.474+0000 7ff4e0bff700 1 -- 172.21.15.31:0/895352546 >> 172.21.15.31:0/895352546 conn(0x7ff4dc06ec90 msgr2=0x7ff4dc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.474+0000 7ff4e0bff700 1 -- 172.21.15.31:0/895352546 shutdown_connections 2023-10-06T05:17:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.474+0000 7ff4e0bff700 1 -- 172.21.15.31:0/895352546 wait complete. 2023-10-06T05:17:53.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.476+0000 7ff4e0bff700 1 Processor -- start 2023-10-06T05:17:53.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.476+0000 7ff4e0bff700 1 -- start start 2023-10-06T05:17:53.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.477+0000 7ff4e0bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 0x7ff4dc0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.477+0000 7ff4e0bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4dc159500 0x7ff4dc0af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.477+0000 7ff4e0bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4dc15d2d0 0x7ff4dc0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4e0bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc0c7600 con 0x7ff4dc159500 2023-10-06T05:17:53.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4e0bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc0c78e0 con 0x7ff4dc15d2d0 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4e0bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc0c7bc0 con 0x7ff4dc0cd510 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4d9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4dc159500 0x7ff4dc0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4d9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4dc159500 0x7ff4dc0af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39288/0 (socket says 172.21.15.31:39288) 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4d9d9b700 1 -- 172.21.15.31:0/377106981 learned_addr learned my addr 172.21.15.31:0/377106981 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4da59c700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 0x7ff4dc0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4dad9d700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4dc15d2d0 0x7ff4dc0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4d9d9b700 1 -- 172.21.15.31:0/377106981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 msgr2=0x7ff4dc0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4d9d9b700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 0x7ff4dc0c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4d9d9b700 1 -- 172.21.15.31:0/377106981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4dc15d2d0 msgr2=0x7ff4dc0afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4d9d9b700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4dc15d2d0 0x7ff4dc0afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4d9d9b700 1 -- 172.21.15.31:0/377106981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d00093c0 con 0x7ff4dc159500 2023-10-06T05:17:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4dad9d700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4dc15d2d0 0x7ff4dc0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.478+0000 7ff4d9d9b700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4dc159500 0x7ff4dc0af380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ec80 tx=0x7ff4d0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.479+0000 7ff4ca7fc700 1 -- 172.21.15.31:0/377106981 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0005a00 con 0x7ff4dc159500 2023-10-06T05:17:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.479+0000 7ff4ca7fc700 1 -- 172.21.15.31:0/377106981 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d0005b60 con 0x7ff4dc159500 2023-10-06T05:17:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.479+0000 7ff4ca7fc700 1 -- 172.21.15.31:0/377106981 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000c9d0 con 0x7ff4dc159500 2023-10-06T05:17:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.479+0000 7ff4e0bff700 1 -- 172.21.15.31:0/377106981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d4009a20 con 0x7ff4dc159500 2023-10-06T05:17:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.479+0000 7ff4e0bff700 1 -- 172.21.15.31:0/377106981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc0c8180 con 0x7ff4dc159500 2023-10-06T05:17:53.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.480+0000 7ff4e0bff700 1 -- 172.21.15.31:0/377106981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4dc04c380 con 0x7ff4dc159500 2023-10-06T05:17:53.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.481+0000 7ff4ca7fc700 1 -- 172.21.15.31:0/377106981 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4d000cd80 con 0x7ff4dc159500 2023-10-06T05:17:53.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.482+0000 7ff4ca7fc700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff4bc066f30 0x7ff4bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.482+0000 7ff4ca7fc700 1 -- 172.21.15.31:0/377106981 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff4d0084a10 con 0x7ff4dc159500 2023-10-06T05:17:53.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.482+0000 7ff4da59c700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff4bc066f30 0x7ff4bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.483+0000 7ff4da59c700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff4bc066f30 0x7ff4bc0693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0058d0 tx=0x7ff4cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.486+0000 7ff4ca7fc700 1 -- 172.21.15.31:0/377106981 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4d0051dd0 con 0x7ff4dc159500 2023-10-06T05:17:53.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.771+0000 7ff4e0bff700 1 -- 172.21.15.31:0/377106981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7ff4dc0cade0 con 0x7ff4dc159500 2023-10-06T05:17:53.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.775+0000 7ff4ca7fc700 1 -- 172.21.15.31:0/377106981 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff4d00553f0 con 0x7ff4dc159500 2023-10-06T05:17:53.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 -- 172.21.15.31:0/377106981 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff4bc066f30 msgr2=0x7ff4bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff4bc066f30 0x7ff4bc0693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0058d0 tx=0x7ff4cc009040 comp rx=0 tx=0).stop 2023-10-06T05:17:53.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 -- 172.21.15.31:0/377106981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4dc159500 msgr2=0x7ff4dc0af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4dc159500 0x7ff4dc0af380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ec80 tx=0x7ff4d0009120 comp rx=0 tx=0).stop 2023-10-06T05:17:53.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 -- 172.21.15.31:0/377106981 shutdown_connections 2023-10-06T05:17:53.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4dc15d2d0 0x7ff4dc0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4dc0cd510 0x7ff4dc0c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff4bc066f30 0x7ff4bc0693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 --2- 172.21.15.31:0/377106981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4dc159500 0x7ff4dc0af380 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 -- 172.21.15.31:0/377106981 >> 172.21.15.31:0/377106981 conn(0x7ff4dc06ec90 msgr2=0x7ff4dc0ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 -- 172.21.15.31:0/377106981 shutdown_connections 2023-10-06T05:17:53.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.777+0000 7ff4e0bff700 1 -- 172.21.15.31:0/377106981 wait complete. 2023-10-06T05:17:53.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T05:17:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.991+0000 7f8100dff700 1 Processor -- start 2023-10-06T05:17:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.991+0000 7f8100dff700 1 -- start start 2023-10-06T05:17:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.992+0000 7f8100dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80fc0c7f30 0x7f80fc069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.992+0000 7f8100dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f80fc0aee70 0x7f80fc13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.992+0000 7f8100dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80fc0ca5b0 con 0x7f80fc0c7b60 2023-10-06T05:17:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.992+0000 7f8100dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80fc0af2f0 con 0x7f80fc0c7f30 2023-10-06T05:17:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.992+0000 7f8100dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80fc06f740 con 0x7f80fc0aee70 2023-10-06T05:17:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.993+0000 7f80f9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80fc0c7f30 0x7f80fc069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.993+0000 7f80f9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f80fc0c7f30 0x7f80fc069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39294/0 (socket says 172.21.15.31:39294) 2023-10-06T05:17:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.993+0000 7f80f9d9b700 1 -- 172.21.15.31:0/3015164077 learned_addr learned my addr 172.21.15.31:0/3015164077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.993+0000 7f80fad9d700 1 --2- 172.21.15.31:0/3015164077 >> v2:172.21.15.62:3300/0 conn(0x7f80fc0aee70 0x7f80fc13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.993+0000 7f80f9d9b700 1 -- 172.21.15.31:0/3015164077 >> v2:172.21.15.62:3300/0 conn(0x7f80fc0aee70 msgr2=0x7f80fc13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.993+0000 7f80f9d9b700 1 --2- 172.21.15.31:0/3015164077 >> v2:172.21.15.62:3300/0 conn(0x7f80fc0aee70 0x7f80fc13f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.993+0000 7f80f9d9b700 1 -- 172.21.15.31:0/3015164077 >> v1:172.21.15.62:6789/0 conn(0x7f80fc0c7b60 legacy=0x7f80fc06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.993+0000 7f80f9d9b700 1 -- 172.21.15.31:0/3015164077 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc0c9c80 con 0x7f80fc0c7f30 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.993+0000 7f80fad9d700 1 --2- 172.21.15.31:0/3015164077 >> v2:172.21.15.62:3300/0 conn(0x7f80fc0aee70 0x7f80fc13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.993+0000 7f80f9d9b700 1 --2- 172.21.15.31:0/3015164077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7f30 0x7f80fc069880 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f80fc06d6e0 tx=0x7f80f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658bb10bd5fbc03a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.994+0000 7f80f959a700 1 -- 172.21.15.31:0/3015164077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f000b860 con 0x7f80fc0c7f30 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.994+0000 7f80f959a700 1 -- 172.21.15.31:0/3015164077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f000b9c0 con 0x7f80fc0c7f30 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.994+0000 7f80f959a700 1 -- 172.21.15.31:0/3015164077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f000bd60 con 0x7f80fc0c7f30 2023-10-06T05:17:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.994+0000 7f8100dff700 1 -- 172.21.15.31:0/3015164077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7f30 msgr2=0x7f80fc069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.994+0000 7f8100dff700 1 --2- 172.21.15.31:0/3015164077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7f30 0x7f80fc069880 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f80fc06d6e0 tx=0x7f80f0009250 comp rx=0 tx=0).stop 2023-10-06T05:17:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.994+0000 7f8100dff700 1 -- 172.21.15.31:0/3015164077 shutdown_connections 2023-10-06T05:17:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.994+0000 7f8100dff700 1 --2- 172.21.15.31:0/3015164077 >> v2:172.21.15.62:3300/0 conn(0x7f80fc0aee70 0x7f80fc13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.994+0000 7f8100dff700 1 --2- 172.21.15.31:0/3015164077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7f30 0x7f80fc069880 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.994+0000 7f8100dff700 1 -- 172.21.15.31:0/3015164077 >> 172.21.15.31:0/3015164077 conn(0x7f80fc0cc0b0 msgr2=0x7f80fc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.994+0000 7f8100dff700 1 -- 172.21.15.31:0/3015164077 shutdown_connections 2023-10-06T05:17:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.995+0000 7f8100dff700 1 -- 172.21.15.31:0/3015164077 wait complete. 2023-10-06T05:17:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.996+0000 7f8100dff700 1 Processor -- start 2023-10-06T05:17:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.996+0000 7f8100dff700 1 -- start start 2023-10-06T05:17:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.997+0000 7f8100dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80fc0aee70 0x7f80fc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.997+0000 7f8100dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7b60 0x7f80fc0acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.997+0000 7f8100dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80fc0c7f30 0x7f80fc0c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.997+0000 7f8100dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0ab9a0 con 0x7f80fc0c7b60 2023-10-06T05:17:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.997+0000 7f8100dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0abbf0 con 0x7f80fc0aee70 2023-10-06T05:17:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.997+0000 7f8100dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0abed0 con 0x7f80fc0c7f30 2023-10-06T05:17:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.997+0000 7f80f9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7b60 0x7f80fc0acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.997+0000 7f80f9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7b60 0x7f80fc0acc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39302/0 (socket says 172.21.15.31:39302) 2023-10-06T05:17:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.997+0000 7f80f9d9b700 1 -- 172.21.15.31:0/3399977125 learned_addr learned my addr 172.21.15.31:0/3399977125 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80fa59c700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80fc0aee70 0x7f80fc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80fad9d700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80fc0c7f30 0x7f80fc0c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80f9d9b700 1 -- 172.21.15.31:0/3399977125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80fc0c7f30 msgr2=0x7f80fc0c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80f9d9b700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80fc0c7f30 0x7f80fc0c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80f9d9b700 1 -- 172.21.15.31:0/3399977125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80fc0aee70 msgr2=0x7f80fc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80f9d9b700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80fc0aee70 0x7f80fc0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80f9d9b700 1 -- 172.21.15.31:0/3399977125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f00094f0 con 0x7f80fc0c7b60 2023-10-06T05:17:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80fa59c700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80fc0aee70 0x7f80fc0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80f9d9b700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7b60 0x7f80fc0acc90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f80f00090b0 tx=0x7f80f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80ea7fc700 1 -- 172.21.15.31:0/3399977125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0011e60 con 0x7f80fc0c7b60 2023-10-06T05:17:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80ea7fc700 1 -- 172.21.15.31:0/3399977125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f001a630 con 0x7f80fc0c7b60 2023-10-06T05:17:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f80ea7fc700 1 -- 172.21.15.31:0/3399977125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f001aa10 con 0x7f80fc0c7b60 2023-10-06T05:17:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f8100dff700 1 -- 172.21.15.31:0/3399977125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80fc0ac1b0 con 0x7f80fc0c7b60 2023-10-06T05:17:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.998+0000 7f8100dff700 1 -- 172.21.15.31:0/3399977125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80fc0ac770 con 0x7f80fc0c7b60 2023-10-06T05:17:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:53.999+0000 7f8100dff700 1 -- 172.21.15.31:0/3399977125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80fc0ac960 con 0x7f80fc0c7b60 2023-10-06T05:17:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.000+0000 7f80ea7fc700 1 -- 172.21.15.31:0/3399977125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80f002d440 con 0x7f80fc0c7b60 2023-10-06T05:17:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.001+0000 7f80ea7fc700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80dc066f30 0x7f80dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.001+0000 7f80ea7fc700 1 -- 172.21.15.31:0/3399977125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f80f0085e50 con 0x7f80fc0c7b60 2023-10-06T05:17:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.002+0000 7f80fa59c700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80dc066f30 0x7f80dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.002+0000 7f80fa59c700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80dc066f30 0x7f80dc0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80ec0018f0 tx=0x7f80ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.005+0000 7f80ea7fc700 1 -- 172.21.15.31:0/3399977125 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80f0053210 con 0x7f80fc0c7b60 2023-10-06T05:17:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.289+0000 7f8100dff700 1 -- 172.21.15.31:0/3399977125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f80fc04c380 con 0x7f80fc0c7b60 2023-10-06T05:17:54.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.293+0000 7f80ea7fc700 1 -- 172.21.15.31:0/3399977125 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f80f005ae60 con 0x7f80fc0c7b60 2023-10-06T05:17:54.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 -- 172.21.15.31:0/3399977125 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80dc066f30 msgr2=0x7f80dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80dc066f30 0x7f80dc0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80ec0018f0 tx=0x7f80ec009040 comp rx=0 tx=0).stop 2023-10-06T05:17:54.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 -- 172.21.15.31:0/3399977125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7b60 msgr2=0x7f80fc0acc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7b60 0x7f80fc0acc90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f80f00090b0 tx=0x7f80f0017000 comp rx=0 tx=0).stop 2023-10-06T05:17:54.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 -- 172.21.15.31:0/3399977125 shutdown_connections 2023-10-06T05:17:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80fc0c7f30 0x7f80fc0c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80fc0aee70 0x7f80fc0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80dc066f30 0x7f80dc0693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 --2- 172.21.15.31:0/3399977125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc0c7b60 0x7f80fc0acc90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 -- 172.21.15.31:0/3399977125 >> 172.21.15.31:0/3399977125 conn(0x7f80fc0cc0b0 msgr2=0x7f80fc0b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 -- 172.21.15.31:0/3399977125 shutdown_connections 2023-10-06T05:17:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.295+0000 7f8100dff700 1 -- 172.21.15.31:0/3399977125 wait complete. 2023-10-06T05:17:54.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T05:17:54.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.508+0000 7fd2b73ff700 1 Processor -- start 2023-10-06T05:17:54.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.509+0000 7fd2b73ff700 1 -- start start 2023-10-06T05:17:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.510+0000 7fd2b73ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b00cd510 0x7fd2b00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.510+0000 7fd2b73ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2b015d2d0 0x7fd2b00aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.510+0000 7fd2b73ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b00c3f40 con 0x7fd2b0159500 2023-10-06T05:17:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.510+0000 7fd2b73ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b00b2860 con 0x7fd2b015d2d0 2023-10-06T05:17:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.510+0000 7fd2b73ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b015d7d0 con 0x7fd2b00cd510 2023-10-06T05:17:54.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.510+0000 7fd2b599c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2b015d2d0 0x7fd2b00aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.510+0000 7fd2b599c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2b015d2d0 0x7fd2b00aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39304/0 (socket says 172.21.15.31:39304) 2023-10-06T05:17:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.510+0000 7fd2b599c700 1 -- 172.21.15.31:0/3083973747 learned_addr learned my addr 172.21.15.31:0/3083973747 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.510+0000 7fd2b499a700 1 --2- 172.21.15.31:0/3083973747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b00cd510 0x7fd2b00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.511+0000 7fd2b599c700 1 -- 172.21.15.31:0/3083973747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b00cd510 msgr2=0x7fd2b00b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.511+0000 7fd2b599c700 1 --2- 172.21.15.31:0/3083973747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b00cd510 0x7fd2b00b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.511+0000 7fd2b599c700 1 -- 172.21.15.31:0/3083973747 >> v1:172.21.15.62:6789/0 conn(0x7fd2b0159500 legacy=0x7fd2b00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.511+0000 7fd2b599c700 1 -- 172.21.15.31:0/3083973747 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00cb000 con 0x7fd2b015d2d0 2023-10-06T05:17:54.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.511+0000 7fd2b599c700 1 --2- 172.21.15.31:0/3083973747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b015d2d0 0x7fd2b00aedd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009f20 tx=0x7fd2a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b88e8a0249ed4cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.511+0000 7fd2a7fff700 1 -- 172.21.15.31:0/3083973747 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800e070 con 0x7fd2b015d2d0 2023-10-06T05:17:54.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.511+0000 7fd2a7fff700 1 -- 172.21.15.31:0/3083973747 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a800b9d0 con 0x7fd2b015d2d0 2023-10-06T05:17:54.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.511+0000 7fd2a7fff700 1 -- 172.21.15.31:0/3083973747 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80092e0 con 0x7fd2b015d2d0 2023-10-06T05:17:54.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.512+0000 7fd2b73ff700 1 -- 172.21.15.31:0/3083973747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b015d2d0 msgr2=0x7fd2b00aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.512+0000 7fd2b73ff700 1 --2- 172.21.15.31:0/3083973747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b015d2d0 0x7fd2b00aedd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009f20 tx=0x7fd2a8009840 comp rx=0 tx=0).stop 2023-10-06T05:17:54.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.512+0000 7fd2b73ff700 1 -- 172.21.15.31:0/3083973747 shutdown_connections 2023-10-06T05:17:54.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.512+0000 7fd2b73ff700 1 --2- 172.21.15.31:0/3083973747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b015d2d0 0x7fd2b00aedd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.512+0000 7fd2b73ff700 1 --2- 172.21.15.31:0/3083973747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b00cd510 0x7fd2b00b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.512+0000 7fd2b73ff700 1 -- 172.21.15.31:0/3083973747 >> 172.21.15.31:0/3083973747 conn(0x7fd2b006ec90 msgr2=0x7fd2b006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:54.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.512+0000 7fd2b73ff700 1 -- 172.21.15.31:0/3083973747 shutdown_connections 2023-10-06T05:17:54.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.512+0000 7fd2b73ff700 1 -- 172.21.15.31:0/3083973747 wait complete. 2023-10-06T05:17:54.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.513+0000 7fd2b73ff700 1 Processor -- start 2023-10-06T05:17:54.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.513+0000 7fd2b73ff700 1 -- start start 2023-10-06T05:17:54.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.514+0000 7fd2b73ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b00cd510 0x7fd2b015a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.514+0000 7fd2b73ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b0159500 0x7fd2b00c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.514+0000 7fd2b73ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2b015d2d0 0x7fd2b016d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.514+0000 7fd2b73ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b016e0b0 con 0x7fd2b0159500 2023-10-06T05:17:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.514+0000 7fd2b73ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b016d810 con 0x7fd2b015d2d0 2023-10-06T05:17:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.514+0000 7fd2b73ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b016daf0 con 0x7fd2b00cd510 2023-10-06T05:17:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b499a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b0159500 0x7fd2b00c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b499a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b0159500 0x7fd2b00c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39316/0 (socket says 172.21.15.31:39316) 2023-10-06T05:17:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b499a700 1 -- 172.21.15.31:0/2645159209 learned_addr learned my addr 172.21.15.31:0/2645159209 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b599c700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2b015d2d0 0x7fd2b016d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b519b700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b00cd510 0x7fd2b015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b499a700 1 -- 172.21.15.31:0/2645159209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b00cd510 msgr2=0x7fd2b015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b499a700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b00cd510 0x7fd2b015a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b499a700 1 -- 172.21.15.31:0/2645159209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2b015d2d0 msgr2=0x7fd2b016d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b499a700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2b015d2d0 0x7fd2b016d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b499a700 1 -- 172.21.15.31:0/2645159209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8009ae0 con 0x7fd2b0159500 2023-10-06T05:17:54.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.515+0000 7fd2b499a700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b0159500 0x7fd2b00c4a90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000ec80 tx=0x7fd2a0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.516+0000 7fd2a57fa700 1 -- 172.21.15.31:0/2645159209 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000c850 con 0x7fd2b0159500 2023-10-06T05:17:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.516+0000 7fd2a57fa700 1 -- 172.21.15.31:0/2645159209 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a000c9b0 con 0x7fd2b0159500 2023-10-06T05:17:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.516+0000 7fd2a57fa700 1 -- 172.21.15.31:0/2645159209 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000cd30 con 0x7fd2b0159500 2023-10-06T05:17:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.516+0000 7fd2b73ff700 1 -- 172.21.15.31:0/2645159209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b016ddd0 con 0x7fd2b0159500 2023-10-06T05:17:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.516+0000 7fd2b73ff700 1 -- 172.21.15.31:0/2645159209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b0151d10 con 0x7fd2b0159500 2023-10-06T05:17:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.517+0000 7fd2b73ff700 1 -- 172.21.15.31:0/2645159209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b00b44b0 con 0x7fd2b0159500 2023-10-06T05:17:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.518+0000 7fd2a57fa700 1 -- 172.21.15.31:0/2645159209 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2a0013680 con 0x7fd2b0159500 2023-10-06T05:17:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.518+0000 7fd2a57fa700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd294066f30 0x7fd2940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.519+0000 7fd2a57fa700 1 -- 172.21.15.31:0/2645159209 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd2a00849b0 con 0x7fd2b0159500 2023-10-06T05:17:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.519+0000 7fd2b519b700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd294066f30 0x7fd2940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.519+0000 7fd2b519b700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd294066f30 0x7fd2940693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac003a80 tx=0x7fd2ac007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:54.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.523+0000 7fd2a57fa700 1 -- 172.21.15.31:0/2645159209 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2b00b44b0 con 0x7fd2b0159500 2023-10-06T05:17:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.804+0000 7fd2b73ff700 1 -- 172.21.15.31:0/2645159209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fd2b00b4890 con 0x7fd2b0159500 2023-10-06T05:17:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.808+0000 7fd2a57fa700 1 -- 172.21.15.31:0/2645159209 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd2b00b4890 con 0x7fd2b0159500 2023-10-06T05:17:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 -- 172.21.15.31:0/2645159209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd294066f30 msgr2=0x7fd2940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd294066f30 0x7fd2940693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac003a80 tx=0x7fd2ac007ec0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 -- 172.21.15.31:0/2645159209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b0159500 msgr2=0x7fd2b00c4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b0159500 0x7fd2b00c4a90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000ec80 tx=0x7fd2a0009120 comp rx=0 tx=0).stop 2023-10-06T05:17:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 -- 172.21.15.31:0/2645159209 shutdown_connections 2023-10-06T05:17:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2b015d2d0 0x7fd2b016d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b00cd510 0x7fd2b015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd294066f30 0x7fd2940693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 --2- 172.21.15.31:0/2645159209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b0159500 0x7fd2b00c4a90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 -- 172.21.15.31:0/2645159209 >> 172.21.15.31:0/2645159209 conn(0x7fd2b006ec90 msgr2=0x7fd2b00cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.810+0000 7fd2b73ff700 1 -- 172.21.15.31:0/2645159209 shutdown_connections 2023-10-06T05:17:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:54.811+0000 7fd2b73ff700 1 -- 172.21.15.31:0/2645159209 wait complete. 2023-10-06T05:17:54.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T05:17:55.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.025+0000 7fbcbb7ff700 1 Processor -- start 2023-10-06T05:17:55.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.025+0000 7fbcbb7ff700 1 -- start start 2023-10-06T05:17:55.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.026+0000 7fbcbb7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbcb406ec30 0x7fbcb414ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.026+0000 7fbcbb7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb40c6df0 con 0x7fbcb406b510 2023-10-06T05:17:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.026+0000 7fbcbb7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb40c70d0 con 0x7fbcb40ca770 2023-10-06T05:17:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.026+0000 7fbcbb7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb40c7a70 con 0x7fbcb406ec30 2023-10-06T05:17:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.026+0000 7fbcb8d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbcb406b510 0x7fbcb414c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60692/0 (socket says 172.21.15.31:60692) 2023-10-06T05:17:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.026+0000 7fbcb8d9a700 1 -- 172.21.15.31:0/3544556260 learned_addr learned my addr 172.21.15.31:0/3544556260 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcb9d9c700 1 --2- 172.21.15.31:0/3544556260 >> v2:172.21.15.62:3300/0 conn(0x7fbcb406ec30 0x7fbcb414ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430231785 0 0) 0x7fbcb40c6df0 con 0x7fbcb406b510 2023-10-06T05:17:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca0004680 con 0x7fbcb406b510 2023-10-06T05:17:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 92867491 0 0) 0x7fbca0004680 con 0x7fbcb406b510 2023-10-06T05:17:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 >> v2:172.21.15.62:3300/0 conn(0x7fbcb406ec30 msgr2=0x7fbcb414ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 --2- 172.21.15.31:0/3544556260 >> v2:172.21.15.62:3300/0 conn(0x7fbcb406ec30 0x7fbcb414ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 >> v1:172.21.15.62:6789/0 conn(0x7fbcb40ca770 legacy=0x7fbcb406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb40c73b0 con 0x7fbcb406b510 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fbca4001470 con 0x7fbcb406b510 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 >> v1:172.21.15.31:6789/0 conn(0x7fbcb406b510 legacy=0x7fbcb414c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbca0004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbca0005920 0x7fbca0005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0006320 0x7fbca000cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40c7a70 con 0x7fbca0004bc0 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0003570 con 0x7fbca0006320 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.027+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0003730 con 0x7fbca0005920 2023-10-06T05:17:55.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb959b700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbca0004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb9d9c700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0006320 0x7fbca000cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb8d9a700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbca0005920 0x7fbca0005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb959b700 1 -- 172.21.15.31:0/3544556260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbca0005920 msgr2=0x7fbca0005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:55.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb959b700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbca0005920 0x7fbca0005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb959b700 1 -- 172.21.15.31:0/3544556260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0006320 msgr2=0x7fbca000cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:55.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb959b700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca0006320 0x7fbca000cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb959b700 1 -- 172.21.15.31:0/3544556260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca0004680 con 0x7fbca0004bc0 2023-10-06T05:17:55.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb9d9c700 1 -- 172.21.15.31:0/3544556260 reap_dead start 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb8d9a700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbca0005920 0x7fbca0005d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.028+0000 7fbcb959b700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbca0004fd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00018d0 tx=0x7fbcb0007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7575b33e2ec08c4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.029+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0001290 con 0x7fbca0004bc0 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.029+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb0012420 con 0x7fbca0004bc0 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.029+0000 7fbcabfff700 1 -- 172.21.15.31:0/3544556260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00127e0 con 0x7fbca0004bc0 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.029+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/3544556260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 msgr2=0x7fbca0004fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.029+0000 7fbcbb7ff700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbca0004fd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00018d0 tx=0x7fbcb0007d80 comp rx=0 tx=0).stop 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.029+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/3544556260 shutdown_connections 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.029+0000 7fbcbb7ff700 1 --2- 172.21.15.31:0/3544556260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbca0004fd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.029+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/3544556260 >> 172.21.15.31:0/3544556260 conn(0x7fbcb40cc0b0 msgr2=0x7fbcb40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.029+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/3544556260 shutdown_connections 2023-10-06T05:17:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.029+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/3544556260 wait complete. 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.031+0000 7fbcbb7ff700 1 Processor -- start 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.031+0000 7fbcbb7ff700 1 -- start start 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcbb7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbcb416f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcbb7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcb414a1a0 0x7fbcb40af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcbb7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcb40afc30 0x7fbcb416e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcbb7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb414af60 con 0x7fbca0004bc0 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcbb7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb416f060 con 0x7fbcb40afc30 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcbb7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40aee50 con 0x7fbcb414a1a0 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcb959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbcb416f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcb959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbcb416f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39332/0 (socket says 172.21.15.31:39332) 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcb959b700 1 -- 172.21.15.31:0/1185641524 learned_addr learned my addr 172.21.15.31:0/1185641524 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:55.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcb8d9a700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcb414a1a0 0x7fbcb40af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.032+0000 7fbcb9d9c700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcb40afc30 0x7fbcb416e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbcb959b700 1 -- 172.21.15.31:0/1185641524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcb414a1a0 msgr2=0x7fbcb40af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbcb959b700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcb414a1a0 0x7fbcb40af6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbcb959b700 1 -- 172.21.15.31:0/1185641524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcb40afc30 msgr2=0x7fbcb416e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbcb959b700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcb40afc30 0x7fbcb416e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbcb959b700 1 -- 172.21.15.31:0/1185641524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb40af130 con 0x7fbca0004bc0 2023-10-06T05:17:55.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbcb959b700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbcb416f380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbca00068e0 tx=0x7fbcb0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbca97fa700 1 -- 172.21.15.31:0/1185641524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0001230 con 0x7fbca0004bc0 2023-10-06T05:17:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbca97fa700 1 -- 172.21.15.31:0/1185641524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb0011b60 con 0x7fbca0004bc0 2023-10-06T05:17:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/1185641524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb40af410 con 0x7fbca0004bc0 2023-10-06T05:17:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbca97fa700 1 -- 172.21.15.31:0/1185641524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb001e400 con 0x7fbca0004bc0 2023-10-06T05:17:55.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.033+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/1185641524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb414a610 con 0x7fbca0004bc0 2023-10-06T05:17:55.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.034+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/1185641524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb414b240 con 0x7fbca0004bc0 2023-10-06T05:17:55.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.035+0000 7fbca97fa700 1 -- 172.21.15.31:0/1185641524 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbcb0019030 con 0x7fbca0004bc0 2023-10-06T05:17:55.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.036+0000 7fbca97fa700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbc98066f30 0x7fbc980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.036+0000 7fbca97fa700 1 -- 172.21.15.31:0/1185641524 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbcb0085900 con 0x7fbca0004bc0 2023-10-06T05:17:55.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.036+0000 7fbcb8d9a700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbc98066f30 0x7fbc980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.037+0000 7fbcb8d9a700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbc98066f30 0x7fbc980693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbca4000e10 tx=0x7fbca4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.040+0000 7fbca97fa700 1 -- 172.21.15.31:0/1185641524 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcb0052cc0 con 0x7fbca0004bc0 2023-10-06T05:17:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.324+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/1185641524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fbcb404c380 con 0x7fbca0004bc0 2023-10-06T05:17:55.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.330+0000 7fbca97fa700 1 -- 172.21.15.31:0/1185641524 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbcb002c0a0 con 0x7fbca0004bc0 2023-10-06T05:17:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.332+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/1185641524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbc98066f30 msgr2=0x7fbc980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.332+0000 7fbcbb7ff700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbc98066f30 0x7fbc980693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbca4000e10 tx=0x7fbca4009040 comp rx=0 tx=0).stop 2023-10-06T05:17:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.333+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/1185641524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 msgr2=0x7fbcb416f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.333+0000 7fbcbb7ff700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbcb416f380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbca00068e0 tx=0x7fbcb0015000 comp rx=0 tx=0).stop 2023-10-06T05:17:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.333+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/1185641524 shutdown_connections 2023-10-06T05:17:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.333+0000 7fbcbb7ff700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcb40afc30 0x7fbcb416e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.333+0000 7fbcbb7ff700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbc98066f30 0x7fbc980693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.333+0000 7fbcbb7ff700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbca0004bc0 0x7fbcb416f380 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.333+0000 7fbcbb7ff700 1 --2- 172.21.15.31:0/1185641524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcb414a1a0 0x7fbcb40af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.333+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/1185641524 >> 172.21.15.31:0/1185641524 conn(0x7fbcb40cc0b0 msgr2=0x7fbcb40cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:55.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.333+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/1185641524 shutdown_connections 2023-10-06T05:17:55.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.333+0000 7fbcbb7ff700 1 -- 172.21.15.31:0/1185641524 wait complete. 2023-10-06T05:17:55.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T05:17:55.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.546+0000 7f3b263ff700 1 Processor -- start 2023-10-06T05:17:55.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.546+0000 7f3b263ff700 1 -- start start 2023-10-06T05:17:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.547+0000 7f3b263ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b20159580 0x7f3b200cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.547+0000 7f3b263ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b200cf280 con 0x7f3b200cb9f0 2023-10-06T05:17:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.547+0000 7f3b263ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b200cfe20 con 0x7f3b200ccc80 2023-10-06T05:17:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.547+0000 7f3b263ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b200ce670 con 0x7f3b20159580 2023-10-06T05:17:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.548+0000 7f3b1f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3b200cb9f0 0x7f3b200cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60704/0 (socket says 172.21.15.31:60704) 2023-10-06T05:17:55.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.548+0000 7f3b1f7fe700 1 -- 172.21.15.31:0/1605673433 learned_addr learned my addr 172.21.15.31:0/1605673433 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.548+0000 7f3b1ffff700 1 --2- 172.21.15.31:0/1605673433 >> v2:172.21.15.62:3300/0 conn(0x7f3b20159580 0x7f3b200cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.548+0000 7f3b1effd700 1 -- 172.21.15.31:0/1605673433 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407981807 0 0) 0x7f3b200cf280 con 0x7f3b200cb9f0 2023-10-06T05:17:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.548+0000 7f3b1effd700 1 -- 172.21.15.31:0/1605673433 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b08003570 con 0x7f3b200cb9f0 2023-10-06T05:17:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.548+0000 7f3b1ffff700 1 -- 172.21.15.31:0/1605673433 >> v1:172.21.15.62:6789/0 conn(0x7f3b200ccc80 legacy=0x7f3b200cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.548+0000 7f3b1ffff700 1 -- 172.21.15.31:0/1605673433 >> v1:172.21.15.31:6789/0 conn(0x7f3b200cb9f0 legacy=0x7f3b200cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.548+0000 7f3b1ffff700 1 -- 172.21.15.31:0/1605673433 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200ce950 con 0x7f3b20159580 2023-10-06T05:17:55.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.549+0000 7f3b1ffff700 1 --2- 172.21.15.31:0/1605673433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b20159580 0x7f3b200cdfe0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3b100098d0 tx=0x7f3b10009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da3f02d554b054c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.549+0000 7f3b1effd700 1 -- 172.21.15.31:0/1605673433 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10005c10 con 0x7f3b20159580 2023-10-06T05:17:55.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.549+0000 7f3b1effd700 1 -- 172.21.15.31:0/1605673433 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10005d70 con 0x7f3b20159580 2023-10-06T05:17:55.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.549+0000 7f3b1effd700 1 -- 172.21.15.31:0/1605673433 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10006280 con 0x7f3b20159580 2023-10-06T05:17:55.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.550+0000 7f3b263ff700 1 -- 172.21.15.31:0/1605673433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b20159580 msgr2=0x7f3b200cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.550+0000 7f3b263ff700 1 --2- 172.21.15.31:0/1605673433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b20159580 0x7f3b200cdfe0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3b100098d0 tx=0x7f3b10009120 comp rx=0 tx=0).stop 2023-10-06T05:17:55.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.550+0000 7f3b263ff700 1 -- 172.21.15.31:0/1605673433 shutdown_connections 2023-10-06T05:17:55.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.550+0000 7f3b263ff700 1 --2- 172.21.15.31:0/1605673433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b20159580 0x7f3b200cdfe0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.550+0000 7f3b263ff700 1 -- 172.21.15.31:0/1605673433 >> 172.21.15.31:0/1605673433 conn(0x7f3b20099710 msgr2=0x7f3b2006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:55.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.550+0000 7f3b263ff700 1 -- 172.21.15.31:0/1605673433 shutdown_connections 2023-10-06T05:17:55.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.550+0000 7f3b263ff700 1 -- 172.21.15.31:0/1605673433 wait complete. 2023-10-06T05:17:55.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.551+0000 7f3b263ff700 1 Processor -- start 2023-10-06T05:17:55.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.551+0000 7f3b263ff700 1 -- start start 2023-10-06T05:17:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.552+0000 7f3b263ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b200cb9f0 0x7f3b200b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.552+0000 7f3b263ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b200ccc80 0x7f3b200c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.552+0000 7f3b263ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b20159580 0x7f3b200c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.552+0000 7f3b263ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200c6d30 con 0x7f3b200cb9f0 2023-10-06T05:17:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.552+0000 7f3b263ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200b0ae0 con 0x7f3b20159580 2023-10-06T05:17:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.552+0000 7f3b263ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200b0dc0 con 0x7f3b200ccc80 2023-10-06T05:17:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b1ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b200cb9f0 0x7f3b200b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b1ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b200cb9f0 0x7f3b200b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39338/0 (socket says 172.21.15.31:39338) 2023-10-06T05:17:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b1ffff700 1 -- 172.21.15.31:0/2990871014 learned_addr learned my addr 172.21.15.31:0/2990871014 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:55.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b1f7fe700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b200ccc80 0x7f3b200c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b2499c700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b20159580 0x7f3b200c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b1ffff700 1 -- 172.21.15.31:0/2990871014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b200ccc80 msgr2=0x7f3b200c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b1ffff700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b200ccc80 0x7f3b200c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b1ffff700 1 -- 172.21.15.31:0/2990871014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b20159580 msgr2=0x7f3b200c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b1ffff700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b20159580 0x7f3b200c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b1ffff700 1 -- 172.21.15.31:0/2990871014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b100093c0 con 0x7f3b200cb9f0 2023-10-06T05:17:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b1ffff700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b200cb9f0 0x7f3b200b2470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3b200888e0 tx=0x7f3b10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b07fff700 1 -- 172.21.15.31:0/2990871014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1001d880 con 0x7f3b200cb9f0 2023-10-06T05:17:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.553+0000 7f3b07fff700 1 -- 172.21.15.31:0/2990871014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1001da00 con 0x7f3b200cb9f0 2023-10-06T05:17:55.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.554+0000 7f3b263ff700 1 -- 172.21.15.31:0/2990871014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b200b10a0 con 0x7f3b200cb9f0 2023-10-06T05:17:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.554+0000 7f3b07fff700 1 -- 172.21.15.31:0/2990871014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1001ddc0 con 0x7f3b200cb9f0 2023-10-06T05:17:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.554+0000 7f3b263ff700 1 -- 172.21.15.31:0/2990871014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b200b1660 con 0x7f3b200cb9f0 2023-10-06T05:17:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.555+0000 7f3b263ff700 1 -- 172.21.15.31:0/2990871014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2004c380 con 0x7f3b200cb9f0 2023-10-06T05:17:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.556+0000 7f3b07fff700 1 -- 172.21.15.31:0/2990871014 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b1001b690 con 0x7f3b200cb9f0 2023-10-06T05:17:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.556+0000 7f3b07fff700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3b0c066f30 0x7f3b0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.556+0000 7f3b07fff700 1 -- 172.21.15.31:0/2990871014 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3b10086120 con 0x7f3b200cb9f0 2023-10-06T05:17:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.557+0000 7f3b1f7fe700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3b0c066f30 0x7f3b0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.557+0000 7f3b1f7fe700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3b0c066f30 0x7f3b0c0693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b200cd230 tx=0x7f3b14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.561+0000 7f3b07fff700 1 -- 172.21.15.31:0/2990871014 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b10053520 con 0x7f3b200cb9f0 2023-10-06T05:17:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.839+0000 7f3b263ff700 1 -- 172.21.15.31:0/2990871014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f3b200cec30 con 0x7f3b200cb9f0 2023-10-06T05:17:55.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.843+0000 7f3b07fff700 1 -- 172.21.15.31:0/2990871014 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3b10056b00 con 0x7f3b200cb9f0 2023-10-06T05:17:55.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 -- 172.21.15.31:0/2990871014 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3b0c066f30 msgr2=0x7f3b0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3b0c066f30 0x7f3b0c0693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b200cd230 tx=0x7f3b14009040 comp rx=0 tx=0).stop 2023-10-06T05:17:55.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 -- 172.21.15.31:0/2990871014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b200cb9f0 msgr2=0x7f3b200b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b200cb9f0 0x7f3b200b2470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3b200888e0 tx=0x7f3b10016000 comp rx=0 tx=0).stop 2023-10-06T05:17:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 -- 172.21.15.31:0/2990871014 shutdown_connections 2023-10-06T05:17:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b20159580 0x7f3b200c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3b0c066f30 0x7f3b0c0693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b200cb9f0 0x7f3b200b2470 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 --2- 172.21.15.31:0/2990871014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3b200ccc80 0x7f3b200c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 -- 172.21.15.31:0/2990871014 >> 172.21.15.31:0/2990871014 conn(0x7f3b20099710 msgr2=0x7f3b20158980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 -- 172.21.15.31:0/2990871014 shutdown_connections 2023-10-06T05:17:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:55.845+0000 7f3b263ff700 1 -- 172.21.15.31:0/2990871014 wait complete. 2023-10-06T05:17:55.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:17:56.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.060+0000 7fe4e7dff700 1 Processor -- start 2023-10-06T05:17:56.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.060+0000 7fe4e7dff700 1 -- start start 2023-10-06T05:17:56.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.062+0000 7fe4e7dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 0x7fe4e0159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.062+0000 7fe4e7dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe4e00cc0d0 0x7fe4e00c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.062+0000 7fe4e7dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e00c95b0 con 0x7fe4e01595a0 2023-10-06T05:17:56.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.062+0000 7fe4e7dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e00c9890 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.062+0000 7fe4e7dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00c9b70 con 0x7fe4e0068d80 2023-10-06T05:17:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e539a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 0x7fe4e0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e639c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe4e00cc0d0 0x7fe4e00c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e539a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 0x7fe4e0159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56844/0 (socket says 172.21.15.31:56844) 2023-10-06T05:17:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e639c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe4e00cc0d0 0x7fe4e00c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:56822/0 (socket says 172.21.15.31:56822) 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e539a700 1 -- 172.21.15.31:0/3789389066 learned_addr learned my addr 172.21.15.31:0/3789389066 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e5b9b700 1 --1- 172.21.15.31:0/3789389066 >> v1:172.21.15.62:6789/0 conn(0x7fe4e01595a0 0x7fe4e0068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:51530/0 (socket says 172.21.15.31:51530) 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e4b99700 1 -- 172.21.15.31:0/3789389066 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844631436 0 0) 0x7fe4e00c95b0 con 0x7fe4e01595a0 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e4b99700 1 -- 172.21.15.31:0/3789389066 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4cc003570 con 0x7fe4e01595a0 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e539a700 1 -- 172.21.15.31:0/3789389066 >> v2:172.21.15.62:3300/0 conn(0x7fe4e00cc0d0 msgr2=0x7fe4e00c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e539a700 1 --2- 172.21.15.31:0/3789389066 >> v2:172.21.15.62:3300/0 conn(0x7fe4e00cc0d0 0x7fe4e00c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e539a700 1 -- 172.21.15.31:0/3789389066 >> v1:172.21.15.62:6789/0 conn(0x7fe4e01595a0 legacy=0x7fe4e0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e539a700 1 -- 172.21.15.31:0/3789389066 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e00ca130 con 0x7fe4e0068d80 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.063+0000 7fe4e639c700 1 --2- 172.21.15.31:0/3789389066 >> v2:172.21.15.62:3300/0 conn(0x7fe4e00cc0d0 0x7fe4e00c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.064+0000 7fe4e539a700 1 --2- 172.21.15.31:0/3789389066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 0x7fe4e0159970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00099b0 tx=0x7fe4d0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a69a48399b910534 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.064+0000 7fe4e4b99700 1 -- 172.21.15.31:0/3789389066 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000b680 con 0x7fe4e0068d80 2023-10-06T05:17:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.064+0000 7fe4e4b99700 1 -- 172.21.15.31:0/3789389066 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d000b7e0 con 0x7fe4e0068d80 2023-10-06T05:17:56.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.064+0000 7fe4e4b99700 1 -- 172.21.15.31:0/3789389066 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000bb80 con 0x7fe4e0068d80 2023-10-06T05:17:56.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.064+0000 7fe4e7dff700 1 -- 172.21.15.31:0/3789389066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 msgr2=0x7fe4e0159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.064+0000 7fe4e7dff700 1 --2- 172.21.15.31:0/3789389066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 0x7fe4e0159970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00099b0 tx=0x7fe4d0009250 comp rx=0 tx=0).stop 2023-10-06T05:17:56.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.064+0000 7fe4e7dff700 1 -- 172.21.15.31:0/3789389066 shutdown_connections 2023-10-06T05:17:56.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.064+0000 7fe4e7dff700 1 --2- 172.21.15.31:0/3789389066 >> v2:172.21.15.62:3300/0 conn(0x7fe4e00cc0d0 0x7fe4e00c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.064+0000 7fe4e7dff700 1 --2- 172.21.15.31:0/3789389066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 0x7fe4e0159970 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.064+0000 7fe4e7dff700 1 -- 172.21.15.31:0/3789389066 >> 172.21.15.31:0/3789389066 conn(0x7fe4e006b620 msgr2=0x7fe4e006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:56.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.065+0000 7fe4e7dff700 1 -- 172.21.15.31:0/3789389066 shutdown_connections 2023-10-06T05:17:56.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.065+0000 7fe4e7dff700 1 -- 172.21.15.31:0/3789389066 wait complete. 2023-10-06T05:17:56.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.066+0000 7fe4e7dff700 1 Processor -- start 2023-10-06T05:17:56.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.066+0000 7fe4e7dff700 1 -- start start 2023-10-06T05:17:56.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.067+0000 7fe4e7dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 0x7fe4e00b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.067+0000 7fe4e7dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e00cc0d0 0x7fe4e00c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.067+0000 7fe4e7dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e01595a0 0x7fe4e00c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.067+0000 7fe4e7dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00c8d00 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.067+0000 7fe4e7dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00b40f0 con 0x7fe4e01595a0 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.067+0000 7fe4e7dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00b21d0 con 0x7fe4e0068d80 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.067+0000 7fe4e539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e00cc0d0 0x7fe4e00c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.067+0000 7fe4e539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e00cc0d0 0x7fe4e00c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39350/0 (socket says 172.21.15.31:39350) 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.067+0000 7fe4e539a700 1 -- 172.21.15.31:0/1772978579 learned_addr learned my addr 172.21.15.31:0/1772978579 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.067+0000 7fe4e5b9b700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 0x7fe4e00b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4e639c700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e01595a0 0x7fe4e00c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4e539a700 1 -- 172.21.15.31:0/1772978579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 msgr2=0x7fe4e00b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4e539a700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 0x7fe4e00b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4e539a700 1 -- 172.21.15.31:0/1772978579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e01595a0 msgr2=0x7fe4e00c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4e539a700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e01595a0 0x7fe4e00c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4e539a700 1 -- 172.21.15.31:0/1772978579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d8009a20 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4e539a700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e00cc0d0 0x7fe4e00c3d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00c9cb0 tx=0x7fe4d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4d5ffb700 1 -- 172.21.15.31:0/1772978579 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000b950 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4d5ffb700 1 -- 172.21.15.31:0/1772978579 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d000bab0 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4d5ffb700 1 -- 172.21.15.31:0/1772978579 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001a680 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.068+0000 7fe4e7dff700 1 -- 172.21.15.31:0/1772978579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d00094f0 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.069+0000 7fe4e7dff700 1 -- 172.21.15.31:0/1772978579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e00b1410 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.069+0000 7fe4e7dff700 1 -- 172.21.15.31:0/1772978579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e015aff0 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.071+0000 7fe4d5ffb700 1 -- 172.21.15.31:0/1772978579 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe4d001aa70 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.071+0000 7fe4d5ffb700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4c4066f30 0x7fe4c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.072+0000 7fe4d5ffb700 1 -- 172.21.15.31:0/1772978579 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe4d0085ee0 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.072+0000 7fe4e5b9b700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4c4066f30 0x7fe4c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.072+0000 7fe4e5b9b700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4c4066f30 0x7fe4c40693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc004f70 tx=0x7fe4dc007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.076+0000 7fe4d5ffb700 1 -- 172.21.15.31:0/1772978579 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4d0053360 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.285+0000 7fe4e7dff700 1 -- 172.21.15.31:0/1772978579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4e004c380 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.286+0000 7fe4d5ffb700 1 -- 172.21.15.31:0/1772978579 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+13698 (secure 0 0 0) 0x7fe4d0056980 con 0x7fe4e00cc0d0 2023-10-06T05:17:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:56.288 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:17:52.214679+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:17:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.288+0000 7fe4e7dff700 1 -- 172.21.15.31:0/1772978579 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4c4066f30 msgr2=0x7fe4c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.288+0000 7fe4e7dff700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4c4066f30 0x7fe4c40693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc004f70 tx=0x7fe4dc007d40 comp rx=0 tx=0).stop 2023-10-06T05:17:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.288+0000 7fe4e7dff700 1 -- 172.21.15.31:0/1772978579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e00cc0d0 msgr2=0x7fe4e00c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.288+0000 7fe4e7dff700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e00cc0d0 0x7fe4e00c3d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00c9cb0 tx=0x7fe4d0017000 comp rx=0 tx=0).stop 2023-10-06T05:17:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.289+0000 7fe4e7dff700 1 -- 172.21.15.31:0/1772978579 shutdown_connections 2023-10-06T05:17:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.289+0000 7fe4e7dff700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e01595a0 0x7fe4e00c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.289+0000 7fe4e7dff700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4e0068d80 0x7fe4e00b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.289+0000 7fe4e7dff700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4c4066f30 0x7fe4c40693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.289+0000 7fe4e7dff700 1 --2- 172.21.15.31:0/1772978579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e00cc0d0 0x7fe4e00c3d80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.289+0000 7fe4e7dff700 1 -- 172.21.15.31:0/1772978579 >> 172.21.15.31:0/1772978579 conn(0x7fe4e006b620 msgr2=0x7fe4e0155080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.289+0000 7fe4e7dff700 1 -- 172.21.15.31:0/1772978579 shutdown_connections 2023-10-06T05:17:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.289+0000 7fe4e7dff700 1 -- 172.21.15.31:0/1772978579 wait complete. 2023-10-06T05:17:56.308 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:17:56.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:17:56.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.504+0000 7ff35cfff700 1 Processor -- start 2023-10-06T05:17:56.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.505+0000 7ff35cfff700 1 -- start start 2023-10-06T05:17:56.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.506+0000 7ff35cfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3580af590 0x7ff3580c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.506+0000 7ff35cfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3580cf2e0 0x7ff3580cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.506+0000 7ff35cfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3580cb800 con 0x7ff3580c52d0 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.506+0000 7ff35cfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3580cb280 con 0x7ff3580af590 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.506+0000 7ff35cfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3580c7090 con 0x7ff3580cf2e0 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.506+0000 7ff35659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3580af590 0x7ff3580c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.506+0000 7ff355d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff3580c52d0 0x7ff3580cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60708/0 (socket says 172.21.15.31:60708) 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.506+0000 7ff35659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3580af590 0x7ff3580c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39364/0 (socket says 172.21.15.31:39364) 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.506+0000 7ff355d9b700 1 -- 172.21.15.31:0/1783696997 learned_addr learned my addr 172.21.15.31:0/1783696997 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.506+0000 7ff356d9d700 1 --2- 172.21.15.31:0/1783696997 >> v2:172.21.15.62:3300/0 conn(0x7ff3580cf2e0 0x7ff3580cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.507+0000 7ff35659c700 1 -- 172.21.15.31:0/1783696997 >> v2:172.21.15.62:3300/0 conn(0x7ff3580cf2e0 msgr2=0x7ff3580cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.507+0000 7ff35659c700 1 --2- 172.21.15.31:0/1783696997 >> v2:172.21.15.62:3300/0 conn(0x7ff3580cf2e0 0x7ff3580cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.507+0000 7ff35659c700 1 -- 172.21.15.31:0/1783696997 >> v1:172.21.15.31:6789/0 conn(0x7ff3580c52d0 legacy=0x7ff3580cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.507+0000 7ff35659c700 1 -- 172.21.15.31:0/1783696997 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3580af960 con 0x7ff3580af590 2023-10-06T05:17:56.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.507+0000 7ff35559a700 1 -- 172.21.15.31:0/1783696997 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174820096 0 0) 0x7ff3580cb800 con 0x7ff3580c52d0 2023-10-06T05:17:56.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.507+0000 7ff35659c700 1 --2- 172.21.15.31:0/1783696997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580af590 0x7ff3580c6160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff358089820 tx=0x7ff3480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f7be554e0535df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.507+0000 7ff35559a700 1 -- 172.21.15.31:0/1783696997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34800b6b0 con 0x7ff3580af590 2023-10-06T05:17:56.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.507+0000 7ff35559a700 1 -- 172.21.15.31:0/1783696997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34800b810 con 0x7ff3580af590 2023-10-06T05:17:56.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.507+0000 7ff35559a700 1 -- 172.21.15.31:0/1783696997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34800bbb0 con 0x7ff3580af590 2023-10-06T05:17:56.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.508+0000 7ff35cfff700 1 -- 172.21.15.31:0/1783696997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580af590 msgr2=0x7ff3580c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.508+0000 7ff35cfff700 1 --2- 172.21.15.31:0/1783696997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580af590 0x7ff3580c6160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff358089820 tx=0x7ff3480091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.508+0000 7ff35cfff700 1 -- 172.21.15.31:0/1783696997 shutdown_connections 2023-10-06T05:17:56.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.508+0000 7ff35cfff700 1 --2- 172.21.15.31:0/1783696997 >> v2:172.21.15.62:3300/0 conn(0x7ff3580cf2e0 0x7ff3580cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.508+0000 7ff35cfff700 1 --2- 172.21.15.31:0/1783696997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580af590 0x7ff3580c6160 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.508+0000 7ff35cfff700 1 -- 172.21.15.31:0/1783696997 >> 172.21.15.31:0/1783696997 conn(0x7ff3580996f0 msgr2=0x7ff35806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:56.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.508+0000 7ff35cfff700 1 -- 172.21.15.31:0/1783696997 shutdown_connections 2023-10-06T05:17:56.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.508+0000 7ff35cfff700 1 -- 172.21.15.31:0/1783696997 wait complete. 2023-10-06T05:17:56.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.509+0000 7ff35cfff700 1 Processor -- start 2023-10-06T05:17:56.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.510+0000 7ff35cfff700 1 -- start start 2023-10-06T05:17:56.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.510+0000 7ff35cfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3580af590 0x7ff35816df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.510+0000 7ff35cfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580c52d0 0x7ff3580b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff35cfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3580cf2e0 0x7ff358069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff35cfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35816e480 con 0x7ff3580c52d0 2023-10-06T05:17:56.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff35cfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580b4130 con 0x7ff3580cf2e0 2023-10-06T05:17:56.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff35cfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580b3240 con 0x7ff3580af590 2023-10-06T05:17:56.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff355d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580c52d0 0x7ff3580b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff355d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580c52d0 0x7ff3580b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39380/0 (socket says 172.21.15.31:39380) 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff355d9b700 1 -- 172.21.15.31:0/1680230688 learned_addr learned my addr 172.21.15.31:0/1680230688 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff35659c700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3580af590 0x7ff35816df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff356d9d700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3580cf2e0 0x7ff358069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff355d9b700 1 -- 172.21.15.31:0/1680230688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3580af590 msgr2=0x7ff35816df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff355d9b700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3580af590 0x7ff35816df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff355d9b700 1 -- 172.21.15.31:0/1680230688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3580cf2e0 msgr2=0x7ff358069610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff355d9b700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3580cf2e0 0x7ff358069610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff355d9b700 1 -- 172.21.15.31:0/1680230688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3480093c0 con 0x7ff3580c52d0 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff356d9d700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3580cf2e0 0x7ff358069610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff35659c700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3580af590 0x7ff35816df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.511+0000 7ff355d9b700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580c52d0 0x7ff3580b1970 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c810 tx=0x7ff34c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.512+0000 7ff3467fc700 1 -- 172.21.15.31:0/1680230688 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c005b30 con 0x7ff3580c52d0 2023-10-06T05:17:56.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.512+0000 7ff3467fc700 1 -- 172.21.15.31:0/1680230688 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c005c90 con 0x7ff3580c52d0 2023-10-06T05:17:56.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.512+0000 7ff3467fc700 1 -- 172.21.15.31:0/1680230688 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c007300 con 0x7ff3580c52d0 2023-10-06T05:17:56.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.512+0000 7ff35cfff700 1 -- 172.21.15.31:0/1680230688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3580b3520 con 0x7ff3580c52d0 2023-10-06T05:17:56.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.512+0000 7ff35cfff700 1 -- 172.21.15.31:0/1680230688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3580b3ae0 con 0x7ff3580c52d0 2023-10-06T05:17:56.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.513+0000 7ff35cfff700 1 -- 172.21.15.31:0/1680230688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3580b42c0 con 0x7ff3580c52d0 2023-10-06T05:17:56.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.514+0000 7ff3467fc700 1 -- 172.21.15.31:0/1680230688 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff34c01b400 con 0x7ff3580c52d0 2023-10-06T05:17:56.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.515+0000 7ff3467fc700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff338066f30 0x7ff3380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.515+0000 7ff3467fc700 1 -- 172.21.15.31:0/1680230688 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff34c083e40 con 0x7ff3580c52d0 2023-10-06T05:17:56.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.515+0000 7ff35659c700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff338066f30 0x7ff3380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.516+0000 7ff35659c700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff338066f30 0x7ff3380693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff348009390 tx=0x7ff348017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.519+0000 7ff3467fc700 1 -- 172.21.15.31:0/1680230688 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff34c051320 con 0x7ff3580c52d0 2023-10-06T05:17:56.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.722+0000 7ff35cfff700 1 -- 172.21.15.31:0/1680230688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff35804c3b0 con 0x7ff3580c52d0 2023-10-06T05:17:56.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.722+0000 7ff3467fc700 1 -- 172.21.15.31:0/1680230688 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff34c054940 con 0x7ff3580c52d0 2023-10-06T05:17:56.724 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:17:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 -- 172.21.15.31:0/1680230688 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff338066f30 msgr2=0x7ff3380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff338066f30 0x7ff3380693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff348009390 tx=0x7ff348017000 comp rx=0 tx=0).stop 2023-10-06T05:17:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 -- 172.21.15.31:0/1680230688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580c52d0 msgr2=0x7ff3580b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580c52d0 0x7ff3580b1970 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c810 tx=0x7ff34c007df0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 -- 172.21.15.31:0/1680230688 shutdown_connections 2023-10-06T05:17:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3580cf2e0 0x7ff358069610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3580af590 0x7ff35816df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff338066f30 0x7ff3380693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 --2- 172.21.15.31:0/1680230688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3580c52d0 0x7ff3580b1970 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 -- 172.21.15.31:0/1680230688 >> 172.21.15.31:0/1680230688 conn(0x7ff3580996f0 msgr2=0x7ff358068220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 -- 172.21.15.31:0/1680230688 shutdown_connections 2023-10-06T05:17:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.725+0000 7ff35cfff700 1 -- 172.21.15.31:0/1680230688 wait complete. 2023-10-06T05:17:56.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:17:56.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.940+0000 7f3e779ff700 1 Processor -- start 2023-10-06T05:17:56.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.940+0000 7f3e779ff700 1 -- start start 2023-10-06T05:17:56.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.941+0000 7f3e779ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e700af890 0x7f3e70161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.941+0000 7f3e779ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e700afc90 con 0x7f3e70153d40 2023-10-06T05:17:56.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.941+0000 7f3e779ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e700ad690 con 0x7f3e700b0b90 2023-10-06T05:17:56.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.941+0000 7f3e779ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e700acd60 con 0x7f3e700af890 2023-10-06T05:17:56.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.942+0000 7f3e75f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e700af890 0x7f3e70161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.942+0000 7f3e7579b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3e70153d40 0x7f3e700b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44336/0 (socket says 172.21.15.31:44336) 2023-10-06T05:17:56.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.942+0000 7f3e7579b700 1 -- 172.21.15.31:0/1401716456 learned_addr learned my addr 172.21.15.31:0/1401716456 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:56.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.942+0000 7f3e67fff700 1 -- 172.21.15.31:0/1401716456 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107074115 0 0) 0x7f3e6c001110 con 0x7f3e70153d40 2023-10-06T05:17:56.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.942+0000 7f3e67fff700 1 -- 172.21.15.31:0/1401716456 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e5c003570 con 0x7f3e70153d40 2023-10-06T05:17:56.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.942+0000 7f3e75f9c700 1 -- 172.21.15.31:0/1401716456 >> v1:172.21.15.62:6789/0 conn(0x7f3e700b0b90 legacy=0x7f3e700af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.942+0000 7f3e75f9c700 1 -- 172.21.15.31:0/1401716456 >> v1:172.21.15.31:6789/0 conn(0x7f3e70153d40 legacy=0x7f3e700b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.942+0000 7f3e75f9c700 1 -- 172.21.15.31:0/1401716456 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e700ad040 con 0x7f3e700af890 2023-10-06T05:17:56.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.942+0000 7f3e75f9c700 1 --2- 172.21.15.31:0/1401716456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 0x7f3e70161240 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009e60 tx=0x7f3e68009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fd6d2da820d70e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.943+0000 7f3e67fff700 1 -- 172.21.15.31:0/1401716456 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6800e070 con 0x7f3e700af890 2023-10-06T05:17:56.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.943+0000 7f3e67fff700 1 -- 172.21.15.31:0/1401716456 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6800bb60 con 0x7f3e700af890 2023-10-06T05:17:56.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.943+0000 7f3e67fff700 1 -- 172.21.15.31:0/1401716456 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68012430 con 0x7f3e700af890 2023-10-06T05:17:56.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.943+0000 7f3e779ff700 1 -- 172.21.15.31:0/1401716456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 msgr2=0x7f3e70161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.943+0000 7f3e779ff700 1 --2- 172.21.15.31:0/1401716456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 0x7f3e70161240 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009e60 tx=0x7f3e68009780 comp rx=0 tx=0).stop 2023-10-06T05:17:56.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.943+0000 7f3e779ff700 1 -- 172.21.15.31:0/1401716456 shutdown_connections 2023-10-06T05:17:56.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.943+0000 7f3e779ff700 1 --2- 172.21.15.31:0/1401716456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 0x7f3e70161240 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.943+0000 7f3e779ff700 1 -- 172.21.15.31:0/1401716456 >> 172.21.15.31:0/1401716456 conn(0x7f3e70067560 msgr2=0x7f3e70067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:56.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.943+0000 7f3e779ff700 1 -- 172.21.15.31:0/1401716456 shutdown_connections 2023-10-06T05:17:56.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.943+0000 7f3e779ff700 1 -- 172.21.15.31:0/1401716456 wait complete. 2023-10-06T05:17:56.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.945+0000 7f3e779ff700 1 Processor -- start 2023-10-06T05:17:56.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.945+0000 7f3e779ff700 1 -- start start 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e779ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 0x7f3e700c1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e779ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e700b0b90 0x7f3e700a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e779ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e70153d40 0x7f3e700aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e779ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7016a8d0 con 0x7f3e700af890 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e779ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7016d720 con 0x7f3e70153d40 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e779ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e70151190 con 0x7f3e700b0b90 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e7579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 0x7f3e700c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e7579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 0x7f3e700c1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59476/0 (socket says 172.21.15.31:59476) 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e7579b700 1 -- 172.21.15.31:0/2088046385 learned_addr learned my addr 172.21.15.31:0/2088046385 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e75f9c700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e70153d40 0x7f3e700aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.946+0000 7f3e74f9a700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e700b0b90 0x7f3e700a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e7579b700 1 -- 172.21.15.31:0/2088046385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e700b0b90 msgr2=0x7f3e700a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e7579b700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e700b0b90 0x7f3e700a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e7579b700 1 -- 172.21.15.31:0/2088046385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e70153d40 msgr2=0x7f3e700aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:56.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e7579b700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e70153d40 0x7f3e700aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:56.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e7579b700 1 -- 172.21.15.31:0/2088046385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68009a20 con 0x7f3e700af890 2023-10-06T05:17:56.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e74f9a700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e700b0b90 0x7f3e700a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:17:56.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e7579b700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 0x7f3e700c1580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c5c0 tx=0x7f3e6c007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e657fa700 1 -- 172.21.15.31:0/2088046385 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c005b30 con 0x7f3e700af890 2023-10-06T05:17:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e657fa700 1 -- 172.21.15.31:0/2088046385 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c005cb0 con 0x7f3e700af890 2023-10-06T05:17:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e657fa700 1 -- 172.21.15.31:0/2088046385 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c007320 con 0x7f3e700af890 2023-10-06T05:17:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e779ff700 1 -- 172.21.15.31:0/2088046385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e700c1ac0 con 0x7f3e700af890 2023-10-06T05:17:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.947+0000 7f3e779ff700 1 -- 172.21.15.31:0/2088046385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7016d0d0 con 0x7f3e700af890 2023-10-06T05:17:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.948+0000 7f3e779ff700 1 -- 172.21.15.31:0/2088046385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7004c380 con 0x7f3e700af890 2023-10-06T05:17:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.949+0000 7f3e657fa700 1 -- 172.21.15.31:0/2088046385 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3e6c01b3b0 con 0x7f3e700af890 2023-10-06T05:17:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.950+0000 7f3e657fa700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e54066f60 0x7f3e54069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.950+0000 7f3e657fa700 1 -- 172.21.15.31:0/2088046385 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3e6c083740 con 0x7f3e700af890 2023-10-06T05:17:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.950+0000 7f3e74f9a700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e54066f60 0x7f3e54069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.951+0000 7f3e74f9a700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e54066f60 0x7f3e54069410 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e60007fd0 tx=0x7f3e60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:56.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:56.955+0000 7f3e657fa700 1 -- 172.21.15.31:0/2088046385 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e6c050b00 con 0x7f3e700af890 2023-10-06T05:17:57.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.156+0000 7f3e779ff700 1 -- 172.21.15.31:0/2088046385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e700afc90 con 0x7f3e700af890 2023-10-06T05:17:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.157+0000 7f3e657fa700 1 -- 172.21.15.31:0/2088046385 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e700afc90 con 0x7f3e700af890 2023-10-06T05:17:57.159 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:17:57.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 -- 172.21.15.31:0/2088046385 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e54066f60 msgr2=0x7f3e54069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e54066f60 0x7f3e54069410 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e60007fd0 tx=0x7f3e60009040 comp rx=0 tx=0).stop 2023-10-06T05:17:57.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 -- 172.21.15.31:0/2088046385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 msgr2=0x7f3e700c1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 0x7f3e700c1580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c5c0 tx=0x7f3e6c007c70 comp rx=0 tx=0).stop 2023-10-06T05:17:57.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 -- 172.21.15.31:0/2088046385 shutdown_connections 2023-10-06T05:17:57.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e70153d40 0x7f3e700aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e54066f60 0x7f3e54069410 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e700af890 0x7f3e700c1580 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 --2- 172.21.15.31:0/2088046385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e700b0b90 0x7f3e700a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 -- 172.21.15.31:0/2088046385 >> 172.21.15.31:0/2088046385 conn(0x7f3e70067560 msgr2=0x7f3e7006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 -- 172.21.15.31:0/2088046385 shutdown_connections 2023-10-06T05:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.159+0000 7f3e779ff700 1 -- 172.21.15.31:0/2088046385 wait complete. 2023-10-06T05:17:57.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:17:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.371+0000 7f41fe3ff700 1 Processor -- start 2023-10-06T05:17:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.372+0000 7f41fe3ff700 1 -- start start 2023-10-06T05:17:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.373+0000 7f41fe3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41f81595a0 0x7f41f8159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.373+0000 7f41fe3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f8068d80 0x7f41f80c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.373+0000 7f41fe3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f80c95b0 con 0x7f41f80cc0d0 2023-10-06T05:17:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.373+0000 7f41fe3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f80c9890 con 0x7f41f81595a0 2023-10-06T05:17:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.373+0000 7f41fe3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f80c9b70 con 0x7f41f8068d80 2023-10-06T05:17:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.373+0000 7f41f7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41f81595a0 0x7f41f8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.373+0000 7f41f7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41f81595a0 0x7f41f8159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59486/0 (socket says 172.21.15.31:59486) 2023-10-06T05:17:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.373+0000 7f41f7fff700 1 -- 172.21.15.31:0/1210146786 learned_addr learned my addr 172.21.15.31:0/1210146786 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.374+0000 7f41fc99c700 1 --2- 172.21.15.31:0/1210146786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f8068d80 0x7f41f80c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.374+0000 7f41f7fff700 1 -- 172.21.15.31:0/1210146786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f8068d80 msgr2=0x7f41f80c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.374+0000 7f41f7fff700 1 --2- 172.21.15.31:0/1210146786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f8068d80 0x7f41f80c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.374+0000 7f41f7fff700 1 -- 172.21.15.31:0/1210146786 >> v1:172.21.15.62:6789/0 conn(0x7f41f80cc0d0 legacy=0x7f41f8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.374+0000 7f41f7fff700 1 -- 172.21.15.31:0/1210146786 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f80ca130 con 0x7f41f81595a0 2023-10-06T05:17:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.374+0000 7f41fc99c700 1 --2- 172.21.15.31:0/1210146786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f8068d80 0x7f41f80c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:17:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.374+0000 7f41f7fff700 1 --2- 172.21.15.31:0/1210146786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f81595a0 0x7f41f8159970 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f41e80099b0 tx=0x7f41e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1805c8216715587a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.374+0000 7f41f6ffd700 1 -- 172.21.15.31:0/1210146786 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e800b820 con 0x7f41f81595a0 2023-10-06T05:17:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.374+0000 7f41f6ffd700 1 -- 172.21.15.31:0/1210146786 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e800b980 con 0x7f41f81595a0 2023-10-06T05:17:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.374+0000 7f41f6ffd700 1 -- 172.21.15.31:0/1210146786 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e800bd20 con 0x7f41f81595a0 2023-10-06T05:17:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.375+0000 7f41fe3ff700 1 -- 172.21.15.31:0/1210146786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f81595a0 msgr2=0x7f41f8159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.375+0000 7f41fe3ff700 1 --2- 172.21.15.31:0/1210146786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f81595a0 0x7f41f8159970 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f41e80099b0 tx=0x7f41e8009250 comp rx=0 tx=0).stop 2023-10-06T05:17:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.375+0000 7f41fe3ff700 1 -- 172.21.15.31:0/1210146786 shutdown_connections 2023-10-06T05:17:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.375+0000 7f41fe3ff700 1 --2- 172.21.15.31:0/1210146786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f8068d80 0x7f41f80c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.375+0000 7f41fe3ff700 1 --2- 172.21.15.31:0/1210146786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f81595a0 0x7f41f8159970 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.375+0000 7f41fe3ff700 1 -- 172.21.15.31:0/1210146786 >> 172.21.15.31:0/1210146786 conn(0x7f41f806b620 msgr2=0x7f41f806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.375+0000 7f41fe3ff700 1 -- 172.21.15.31:0/1210146786 shutdown_connections 2023-10-06T05:17:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.375+0000 7f41fe3ff700 1 -- 172.21.15.31:0/1210146786 wait complete. 2023-10-06T05:17:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.377+0000 7f41fe3ff700 1 Processor -- start 2023-10-06T05:17:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.377+0000 7f41fe3ff700 1 -- start start 2023-10-06T05:17:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41fe3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f8068d80 0x7f41f81582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41fe3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f80cc0d0 0x7f41f8169c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41fe3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f81595a0 0x7f41f8152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41fe3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f816ac50 con 0x7f41f80cc0d0 2023-10-06T05:17:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41fe3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f816a180 con 0x7f41f8068d80 2023-10-06T05:17:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41fe3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f8153eb0 con 0x7f41f81595a0 2023-10-06T05:17:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41f77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f80cc0d0 0x7f41f8169c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41f77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f80cc0d0 0x7f41f8169c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59500/0 (socket says 172.21.15.31:59500) 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41f77fe700 1 -- 172.21.15.31:0/2674756182 learned_addr learned my addr 172.21.15.31:0/2674756182 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41f7fff700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f8068d80 0x7f41f81582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.378+0000 7f41fc99c700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f81595a0 0x7f41f8152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41f77fe700 1 -- 172.21.15.31:0/2674756182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f81595a0 msgr2=0x7f41f8152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41f77fe700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f81595a0 0x7f41f8152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41f77fe700 1 -- 172.21.15.31:0/2674756182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f8068d80 msgr2=0x7f41f81582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41f77fe700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f8068d80 0x7f41f81582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41f77fe700 1 -- 172.21.15.31:0/2674756182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80094f0 con 0x7f41f80cc0d0 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41f77fe700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f80cc0d0 0x7f41f8169c40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00c600 tx=0x7f41ec007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41dffff700 1 -- 172.21.15.31:0/2674756182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec011590 con 0x7f41f80cc0d0 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41dffff700 1 -- 172.21.15.31:0/2674756182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec011710 con 0x7f41f80cc0d0 2023-10-06T05:17:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41fe3ff700 1 -- 172.21.15.31:0/2674756182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f815c5f0 con 0x7f41f80cc0d0 2023-10-06T05:17:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41dffff700 1 -- 172.21.15.31:0/2674756182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec011ad0 con 0x7f41f80cc0d0 2023-10-06T05:17:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.379+0000 7f41fe3ff700 1 -- 172.21.15.31:0/2674756182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f816a3b0 con 0x7f41f80cc0d0 2023-10-06T05:17:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.380+0000 7f41fe3ff700 1 -- 172.21.15.31:0/2674756182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f815c780 con 0x7f41f80cc0d0 2023-10-06T05:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.381+0000 7f41dffff700 1 -- 172.21.15.31:0/2674756182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f41ec01c460 con 0x7f41f80cc0d0 2023-10-06T05:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.382+0000 7f41dffff700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41e4066f30 0x7f41e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.382+0000 7f41dffff700 1 -- 172.21.15.31:0/2674756182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f41ec083fe0 con 0x7f41f80cc0d0 2023-10-06T05:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.382+0000 7f41f7fff700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41e4066f30 0x7f41e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.383+0000 7f41f7fff700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41e4066f30 0x7f41e40693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f41e80099b0 tx=0x7f41e8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:57.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.386+0000 7f41dffff700 1 -- 172.21.15.31:0/2674756182 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41ec0513c0 con 0x7f41f80cc0d0 2023-10-06T05:17:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.593+0000 7f41fe3ff700 1 -- 172.21.15.31:0/2674756182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f41f804c380 con 0x7f41f80cc0d0 2023-10-06T05:17:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.594+0000 7f41dffff700 1 -- 172.21.15.31:0/2674756182 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+126 (secure 0 0 0) 0x7f41ec0549e0 con 0x7f41f80cc0d0 2023-10-06T05:17:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:17:57.596 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:17:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:17:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:17:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:17:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:17:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:17:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:17:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:17:57.597 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:17:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 -- 172.21.15.31:0/2674756182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41e4066f30 msgr2=0x7f41e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41e4066f30 0x7f41e40693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f41e80099b0 tx=0x7f41e8016000 comp rx=0 tx=0).stop 2023-10-06T05:17:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 -- 172.21.15.31:0/2674756182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f80cc0d0 msgr2=0x7f41f8169c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f80cc0d0 0x7f41f8169c40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00c600 tx=0x7f41ec007f60 comp rx=0 tx=0).stop 2023-10-06T05:17:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 -- 172.21.15.31:0/2674756182 shutdown_connections 2023-10-06T05:17:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f81595a0 0x7f41f8152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f8068d80 0x7f41f81582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41e4066f30 0x7f41e40693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 --2- 172.21.15.31:0/2674756182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41f80cc0d0 0x7f41f8169c40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 -- 172.21.15.31:0/2674756182 >> 172.21.15.31:0/2674756182 conn(0x7f41f806b620 msgr2=0x7f41f815b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 -- 172.21.15.31:0/2674756182 shutdown_connections 2023-10-06T05:17:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.596+0000 7f41fe3ff700 1 -- 172.21.15.31:0/2674756182 wait complete. 2023-10-06T05:17:57.615 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:17:57.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:17:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.814+0000 7f05a31ff700 1 Processor -- start 2023-10-06T05:17:57.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.814+0000 7f05a31ff700 1 -- start start 2023-10-06T05:17:57.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.815+0000 7f05a31ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f059c153950 0x7f059c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.815+0000 7f05a31ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c15d920 0x7f059c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:57.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.815+0000 7f05a31ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c15bb20 con 0x7f059c0cd5d0 2023-10-06T05:17:57.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.815+0000 7f05a31ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c15af10 con 0x7f059c153950 2023-10-06T05:17:57.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.815+0000 7f05a31ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15b1f0 con 0x7f059c15d920 2023-10-06T05:17:57.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.816+0000 7f05a179c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c15d920 0x7f059c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.816+0000 7f05a179c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c15d920 0x7f059c15bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:43688/0 (socket says 172.21.15.31:43688) 2023-10-06T05:17:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.816+0000 7f05a179c700 1 -- 172.21.15.31:0/3391476405 learned_addr learned my addr 172.21.15.31:0/3391476405 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.816+0000 7f05a0f9b700 1 --2- 172.21.15.31:0/3391476405 >> v2:172.21.15.62:3300/0 conn(0x7f059c153950 0x7f059c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.816+0000 7f05a179c700 1 -- 172.21.15.31:0/3391476405 >> v2:172.21.15.62:3300/0 conn(0x7f059c153950 msgr2=0x7f059c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.816+0000 7f05a179c700 1 --2- 172.21.15.31:0/3391476405 >> v2:172.21.15.62:3300/0 conn(0x7f059c153950 0x7f059c153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.816+0000 7f05a179c700 1 -- 172.21.15.31:0/3391476405 >> v1:172.21.15.62:6789/0 conn(0x7f059c0cd5d0 legacy=0x7f059c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.816+0000 7f05a179c700 1 -- 172.21.15.31:0/3391476405 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c15b4d0 con 0x7f059c15d920 2023-10-06T05:17:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.816+0000 7f05a0f9b700 1 --2- 172.21.15.31:0/3391476405 >> v2:172.21.15.62:3300/0 conn(0x7f059c153950 0x7f059c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.817+0000 7f05a179c700 1 --2- 172.21.15.31:0/3391476405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c15d920 0x7f059c15bfb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0594009f20 tx=0x7f0594009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28881a7c5ce5c8b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.817+0000 7f059b7fe700 1 -- 172.21.15.31:0/3391476405 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059400e070 con 0x7f059c15d920 2023-10-06T05:17:58.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.817+0000 7f059b7fe700 1 -- 172.21.15.31:0/3391476405 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059400b9d0 con 0x7f059c15d920 2023-10-06T05:17:58.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.817+0000 7f059b7fe700 1 -- 172.21.15.31:0/3391476405 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05940092e0 con 0x7f059c15d920 2023-10-06T05:17:58.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.817+0000 7f05a31ff700 1 -- 172.21.15.31:0/3391476405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c15d920 msgr2=0x7f059c15bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.817+0000 7f05a31ff700 1 --2- 172.21.15.31:0/3391476405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c15d920 0x7f059c15bfb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0594009f20 tx=0x7f0594009840 comp rx=0 tx=0).stop 2023-10-06T05:17:58.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.818+0000 7f05a31ff700 1 -- 172.21.15.31:0/3391476405 shutdown_connections 2023-10-06T05:17:58.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.818+0000 7f05a31ff700 1 --2- 172.21.15.31:0/3391476405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c15d920 0x7f059c15bfb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.818+0000 7f05a31ff700 1 --2- 172.21.15.31:0/3391476405 >> v2:172.21.15.62:3300/0 conn(0x7f059c153950 0x7f059c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.818+0000 7f05a31ff700 1 -- 172.21.15.31:0/3391476405 >> 172.21.15.31:0/3391476405 conn(0x7f059c0681b0 msgr2=0x7f059c0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.818+0000 7f05a31ff700 1 -- 172.21.15.31:0/3391476405 shutdown_connections 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.818+0000 7f05a31ff700 1 -- 172.21.15.31:0/3391476405 wait complete. 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.819+0000 7f05a31ff700 1 Processor -- start 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.819+0000 7f05a31ff700 1 -- start start 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.820+0000 7f05a31ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c0cd5d0 0x7f059c0b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.820+0000 7f05a31ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c153950 0x7f059c155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.820+0000 7f05a31ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059c15d920 0x7f059c16d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.820+0000 7f05a31ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0b3960 con 0x7f059c15d920 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.820+0000 7f05a31ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c7680 con 0x7f059c153950 2023-10-06T05:17:58.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.820+0000 7f05a31ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c7920 con 0x7f059c0cd5d0 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.820+0000 7f05a179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059c15d920 0x7f059c16d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f05a179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059c15d920 0x7f059c16d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59506/0 (socket says 172.21.15.31:59506) 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f05a179c700 1 -- 172.21.15.31:0/2444702888 learned_addr learned my addr 172.21.15.31:0/2444702888 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f05a0f9b700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c0cd5d0 0x7f059c0b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f059bfff700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c153950 0x7f059c155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f05a179c700 1 -- 172.21.15.31:0/2444702888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c0cd5d0 msgr2=0x7f059c0b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f05a179c700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c0cd5d0 0x7f059c0b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f05a179c700 1 -- 172.21.15.31:0/2444702888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c153950 msgr2=0x7f059c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f05a179c700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c153950 0x7f059c155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f05a179c700 1 -- 172.21.15.31:0/2444702888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05900093c0 con 0x7f059c15d920 2023-10-06T05:17:58.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f05a179c700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059c15d920 0x7f059c16d200 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0594012000 tx=0x7f0594018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.821+0000 7f0598ff9700 1 -- 172.21.15.31:0/2444702888 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05940094e0 con 0x7f059c15d920 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.822+0000 7f0598ff9700 1 -- 172.21.15.31:0/2444702888 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0594008bc0 con 0x7f059c15d920 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.822+0000 7f0598ff9700 1 -- 172.21.15.31:0/2444702888 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594026460 con 0x7f059c15d920 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.822+0000 7f05a31ff700 1 -- 172.21.15.31:0/2444702888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0594009ae0 con 0x7f059c15d920 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.822+0000 7f05a31ff700 1 -- 172.21.15.31:0/2444702888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c0c81c0 con 0x7f059c15d920 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.823+0000 7f05a31ff700 1 -- 172.21.15.31:0/2444702888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c04c380 con 0x7f059c15d920 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.824+0000 7f0598ff9700 1 -- 172.21.15.31:0/2444702888 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0594003040 con 0x7f059c15d920 2023-10-06T05:17:58.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.824+0000 7f0598ff9700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0580066f30 0x7f05800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.824+0000 7f0598ff9700 1 -- 172.21.15.31:0/2444702888 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0594086260 con 0x7f059c15d920 2023-10-06T05:17:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.825+0000 7f05a0f9b700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0580066f30 0x7f05800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.825+0000 7f05a0f9b700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0580066f30 0x7f05800693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0590009f30 tx=0x7f0590009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:57.828+0000 7f0598ff9700 1 -- 172.21.15.31:0/2444702888 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0594053620 con 0x7f059c15d920 2023-10-06T05:17:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.035+0000 7f05a31ff700 1 -- 172.21.15.31:0/2444702888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f059c153730 con 0x7f059c15d920 2023-10-06T05:17:58.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.222+0000 7f0598ff9700 1 -- 172.21.15.31:0/2444702888 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0594056c40 con 0x7f059c15d920 2023-10-06T05:17:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 -- 172.21.15.31:0/2444702888 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0580066f30 msgr2=0x7f05800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0580066f30 0x7f05800693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0590009f30 tx=0x7f0590009120 comp rx=0 tx=0).stop 2023-10-06T05:17:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 -- 172.21.15.31:0/2444702888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059c15d920 msgr2=0x7f059c16d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059c15d920 0x7f059c16d200 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0594012000 tx=0x7f0594018000 comp rx=0 tx=0).stop 2023-10-06T05:17:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 -- 172.21.15.31:0/2444702888 shutdown_connections 2023-10-06T05:17:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0580066f30 0x7f05800693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059c15d920 0x7f059c16d200 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f059c0cd5d0 0x7f059c0b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 --2- 172.21.15.31:0/2444702888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f059c153950 0x7f059c155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 -- 172.21.15.31:0/2444702888 >> 172.21.15.31:0/2444702888 conn(0x7f059c0681b0 msgr2=0x7f059c0d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.226+0000 7f05a31ff700 1 -- 172.21.15.31:0/2444702888 shutdown_connections 2023-10-06T05:17:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.227+0000 7f05a31ff700 1 -- 172.21.15.31:0/2444702888 wait complete. 2023-10-06T05:17:58.395 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:17:58.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:17:58.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.596+0000 7f1ff8bff700 1 Processor -- start 2023-10-06T05:17:58.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.596+0000 7f1ff8bff700 1 -- start start 2023-10-06T05:17:58.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.598+0000 7f1ff8bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 0x7f1ff40c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:58.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.598+0000 7f1ff8bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff4068250 con 0x7f1ff40c7190 2023-10-06T05:17:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.598+0000 7f1ff8bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff406ac80 con 0x7f1ff40c7560 2023-10-06T05:17:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.598+0000 7f1ff8bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4067f20 con 0x7f1ff40aef00 2023-10-06T05:17:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.598+0000 7f1ff259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1ff40c7190 0x7f1ff40af7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44346/0 (socket says 172.21.15.31:44346) 2023-10-06T05:17:58.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.598+0000 7f1ff259c700 1 -- 172.21.15.31:0/4181325422 learned_addr learned my addr 172.21.15.31:0/4181325422 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:58.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.598+0000 7f1ff1d9b700 1 --2- 172.21.15.31:0/4181325422 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 0x7f1ff40c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:58.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.598+0000 7f1ff159a700 1 -- 172.21.15.31:0/4181325422 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766775127 0 0) 0x7f1ff4068250 con 0x7f1ff40c7190 2023-10-06T05:17:58.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.598+0000 7f1ff159a700 1 -- 172.21.15.31:0/4181325422 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fdc003570 con 0x7f1ff40c7190 2023-10-06T05:17:58.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.599+0000 7f1ff159a700 1 -- 172.21.15.31:0/4181325422 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170785993 0 0) 0x7f1ff406ac80 con 0x7f1ff40c7560 2023-10-06T05:17:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.599+0000 7f1ff159a700 1 -- 172.21.15.31:0/4181325422 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ff4068250 con 0x7f1ff40c7560 2023-10-06T05:17:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.599+0000 7f1ff1d9b700 1 -- 172.21.15.31:0/4181325422 >> v1:172.21.15.62:6789/0 conn(0x7f1ff40c7560 legacy=0x7f1ff40686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.599+0000 7f1ff159a700 1 -- 172.21.15.31:0/4181325422 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3261441274 0 0) 0x7f1fdc003570 con 0x7f1ff40c7190 2023-10-06T05:17:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.599+0000 7f1ff1d9b700 1 -- 172.21.15.31:0/4181325422 >> v1:172.21.15.31:6789/0 conn(0x7f1ff40c7190 legacy=0x7f1ff40af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.599+0000 7f1ff1d9b700 1 -- 172.21.15.31:0/4181325422 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff406aed0 con 0x7f1ff40aef00 2023-10-06T05:17:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.599+0000 7f1ff1d9b700 1 --2- 172.21.15.31:0/4181325422 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 0x7f1ff40c9d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009910 tx=0x7f1fe80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c1175bcec83f51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.599+0000 7f1ff159a700 1 -- 172.21.15.31:0/4181325422 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4001470 con 0x7f1ff40aef00 2023-10-06T05:17:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.599+0000 7f1ff159a700 1 -- 172.21.15.31:0/4181325422 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe8011420 con 0x7f1ff40aef00 2023-10-06T05:17:58.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.599+0000 7f1ff159a700 1 -- 172.21.15.31:0/4181325422 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80117c0 con 0x7f1ff40aef00 2023-10-06T05:17:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.600+0000 7f1ff8bff700 1 -- 172.21.15.31:0/4181325422 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 msgr2=0x7f1ff40c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.600+0000 7f1ff8bff700 1 --2- 172.21.15.31:0/4181325422 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 0x7f1ff40c9d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009910 tx=0x7f1fe80091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.600+0000 7f1ff8bff700 1 -- 172.21.15.31:0/4181325422 shutdown_connections 2023-10-06T05:17:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.600+0000 7f1ff8bff700 1 --2- 172.21.15.31:0/4181325422 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 0x7f1ff40c9d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.600+0000 7f1ff8bff700 1 -- 172.21.15.31:0/4181325422 >> 172.21.15.31:0/4181325422 conn(0x7f1ff40cd3d0 msgr2=0x7f1ff40c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.600+0000 7f1ff8bff700 1 -- 172.21.15.31:0/4181325422 shutdown_connections 2023-10-06T05:17:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.600+0000 7f1ff8bff700 1 -- 172.21.15.31:0/4181325422 wait complete. 2023-10-06T05:17:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.602+0000 7f1ff8bff700 1 Processor -- start 2023-10-06T05:17:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.602+0000 7f1ff8bff700 1 -- start start 2023-10-06T05:17:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff8bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 0x7f1ff414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff8bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff40c7190 0x7f1ff415eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff8bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff40c7560 0x7f1ff415f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff8bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40c1c10 con 0x7f1ff40c7190 2023-10-06T05:17:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff8bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40c0d90 con 0x7f1ff40c7560 2023-10-06T05:17:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff8bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40c1000 con 0x7f1ff40aef00 2023-10-06T05:17:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff40c7190 0x7f1ff415eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff40c7190 0x7f1ff415eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59514/0 (socket says 172.21.15.31:59514) 2023-10-06T05:17:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff1d9b700 1 -- 172.21.15.31:0/1391034045 learned_addr learned my addr 172.21.15.31:0/1391034045 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff259c700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 0x7f1ff414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff2d9d700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff40c7560 0x7f1ff415f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff1d9b700 1 -- 172.21.15.31:0/1391034045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 msgr2=0x7f1ff414eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff1d9b700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 0x7f1ff414eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff1d9b700 1 -- 172.21.15.31:0/1391034045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff40c7560 msgr2=0x7f1ff415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff1d9b700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff40c7560 0x7f1ff415f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.603+0000 7f1ff1d9b700 1 -- 172.21.15.31:0/1391034045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe8009420 con 0x7f1ff40c7190 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.604+0000 7f1ff1d9b700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff40c7190 0x7f1ff415eba0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80030c0 tx=0x7f1fe8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.604+0000 7f1fe27fc700 1 -- 172.21.15.31:0/1391034045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe801cba0 con 0x7f1ff40c7190 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.604+0000 7f1fe27fc700 1 -- 172.21.15.31:0/1391034045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe801cd00 con 0x7f1ff40c7190 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.604+0000 7f1fe27fc700 1 -- 172.21.15.31:0/1391034045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe801a6a0 con 0x7f1ff40c7190 2023-10-06T05:17:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.604+0000 7f1ff8bff700 1 -- 172.21.15.31:0/1391034045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff40c12e0 con 0x7f1ff40c7190 2023-10-06T05:17:59.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.604+0000 7f1ff8bff700 1 -- 172.21.15.31:0/1391034045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff40c18a0 con 0x7f1ff40c7190 2023-10-06T05:17:59.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.605+0000 7f1ff8bff700 1 -- 172.21.15.31:0/1391034045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff404c380 con 0x7f1ff40c7190 2023-10-06T05:17:59.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.606+0000 7f1fe27fc700 1 -- 172.21.15.31:0/1391034045 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1fe801aad0 con 0x7f1ff40c7190 2023-10-06T05:17:59.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.607+0000 7f1fe27fc700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fd4066f30 0x7f1fd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.607+0000 7f1fe27fc700 1 -- 172.21.15.31:0/1391034045 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f1fe80862f0 con 0x7f1ff40c7190 2023-10-06T05:17:59.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.608+0000 7f1ff259c700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fd4066f30 0x7f1fd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.608+0000 7f1ff259c700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fd4066f30 0x7f1fd40693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4001ed0 tx=0x7f1fe4000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.612+0000 7f1fe27fc700 1 -- 172.21.15.31:0/1391034045 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1fe8053540 con 0x7f1ff40c7190 2023-10-06T05:17:59.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:58.814+0000 7f1ff8bff700 1 -- 172.21.15.31:0/1391034045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1ff40cc6e0 con 0x7f1ff40c7190 2023-10-06T05:17:59.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.225+0000 7f1fe27fc700 1 -- 172.21.15.31:0/1391034045 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1fe8056b60 con 0x7f1ff40c7190 2023-10-06T05:17:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.230+0000 7f1ff8bff700 1 -- 172.21.15.31:0/1391034045 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fd4066f30 msgr2=0x7f1fd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.230+0000 7f1ff8bff700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fd4066f30 0x7f1fd40693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4001ed0 tx=0x7f1fe4000bc0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.230+0000 7f1ff8bff700 1 -- 172.21.15.31:0/1391034045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff40c7190 msgr2=0x7f1ff415eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.230+0000 7f1ff8bff700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff40c7190 0x7f1ff415eba0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80030c0 tx=0x7f1fe8016000 comp rx=0 tx=0).stop 2023-10-06T05:17:59.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.230+0000 7f1ff8bff700 1 -- 172.21.15.31:0/1391034045 shutdown_connections 2023-10-06T05:17:59.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.230+0000 7f1ff8bff700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff40c7560 0x7f1ff415f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.230+0000 7f1ff8bff700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff40aef00 0x7f1ff414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.230+0000 7f1ff8bff700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fd4066f30 0x7f1fd40693e0 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4001ed0 tx=0x7f1fe4000bc0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.230+0000 7f1ff8bff700 1 --2- 172.21.15.31:0/1391034045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff40c7190 0x7f1ff415eba0 secure :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80030c0 tx=0x7f1fe8016000 comp rx=0 tx=0).stop 2023-10-06T05:17:59.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.230+0000 7f1ff8bff700 1 -- 172.21.15.31:0/1391034045 >> 172.21.15.31:0/1391034045 conn(0x7f1ff40cd3d0 msgr2=0x7f1ff406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:59.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.235+0000 7f1ff8bff700 1 -- 172.21.15.31:0/1391034045 shutdown_connections 2023-10-06T05:17:59.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.239+0000 7f1ff8bff700 1 -- 172.21.15.31:0/1391034045 wait complete. 2023-10-06T05:17:59.244 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:17:59.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:17:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.823+0000 7f6c783ff700 1 Processor -- start 2023-10-06T05:17:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.824+0000 7f6c783ff700 1 -- start start 2023-10-06T05:17:59.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.825+0000 7f6c783ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6c700c7190 0x7f6c700c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.825+0000 7f6c783ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 0x7f6c700af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.825+0000 7f6c783ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c70068250 con 0x7f6c700afcf0 2023-10-06T05:17:59.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.825+0000 7f6c783ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7006ac80 con 0x7f6c700c7190 2023-10-06T05:17:59.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.825+0000 7f6c783ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c70067f20 con 0x7f6c700aef00 2023-10-06T05:17:59.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.825+0000 7f6c7699c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6c700afcf0 0x7f6c700686e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44360/0 (socket says 172.21.15.31:44360) 2023-10-06T05:17:59.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.825+0000 7f6c7699c700 1 -- 172.21.15.31:0/1652729088 learned_addr learned my addr 172.21.15.31:0/1652729088 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:17:59.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.825+0000 7f6c7619b700 1 --2- 172.21.15.31:0/1652729088 >> v2:172.21.15.62:3300/0 conn(0x7f6c700c7190 0x7f6c700c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.825+0000 7f6c7599a700 1 --2- 172.21.15.31:0/1652729088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 0x7f6c700af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.826+0000 7f6c75199700 1 -- 172.21.15.31:0/1652729088 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886811936 0 0) 0x7f6c70068250 con 0x7f6c700afcf0 2023-10-06T05:17:59.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.826+0000 7f6c75199700 1 -- 172.21.15.31:0/1652729088 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c5c003570 con 0x7f6c700afcf0 2023-10-06T05:17:59.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.826+0000 7f6c7619b700 1 -- 172.21.15.31:0/1652729088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 msgr2=0x7f6c700af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:17:59.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.826+0000 7f6c7619b700 1 --2- 172.21.15.31:0/1652729088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 0x7f6c700af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.826+0000 7f6c7619b700 1 -- 172.21.15.31:0/1652729088 >> v1:172.21.15.31:6789/0 conn(0x7f6c700afcf0 legacy=0x7f6c700686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.826+0000 7f6c7619b700 1 -- 172.21.15.31:0/1652729088 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7006aed0 con 0x7f6c700c7190 2023-10-06T05:17:59.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.826+0000 7f6c7599a700 1 --2- 172.21.15.31:0/1652729088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 0x7f6c700af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:17:59.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.826+0000 7f6c7619b700 1 --2- 172.21.15.31:0/1652729088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c700c7190 0x7f6c700c9d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0098e0 tx=0x7f6c6c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d482241e063135c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.827+0000 7f6c75199700 1 -- 172.21.15.31:0/1652729088 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c011630 con 0x7f6c700c7190 2023-10-06T05:17:59.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.827+0000 7f6c75199700 1 -- 172.21.15.31:0/1652729088 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c011790 con 0x7f6c700c7190 2023-10-06T05:17:59.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.827+0000 7f6c75199700 1 -- 172.21.15.31:0/1652729088 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c011b30 con 0x7f6c700c7190 2023-10-06T05:17:59.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.827+0000 7f6c783ff700 1 -- 172.21.15.31:0/1652729088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c700c7190 msgr2=0x7f6c700c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.827+0000 7f6c783ff700 1 --2- 172.21.15.31:0/1652729088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c700c7190 0x7f6c700c9d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0098e0 tx=0x7f6c6c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.828+0000 7f6c783ff700 1 -- 172.21.15.31:0/1652729088 shutdown_connections 2023-10-06T05:17:59.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.828+0000 7f6c783ff700 1 --2- 172.21.15.31:0/1652729088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c700c7190 0x7f6c700c9d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.828+0000 7f6c783ff700 1 --2- 172.21.15.31:0/1652729088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 0x7f6c700af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.828+0000 7f6c783ff700 1 -- 172.21.15.31:0/1652729088 >> 172.21.15.31:0/1652729088 conn(0x7f6c700cd3d0 msgr2=0x7f6c700c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:17:59.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.828+0000 7f6c783ff700 1 -- 172.21.15.31:0/1652729088 shutdown_connections 2023-10-06T05:17:59.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.828+0000 7f6c783ff700 1 -- 172.21.15.31:0/1652729088 wait complete. 2023-10-06T05:17:59.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.829+0000 7f6c783ff700 1 Processor -- start 2023-10-06T05:17:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.829+0000 7f6c783ff700 1 -- start start 2023-10-06T05:17:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.830+0000 7f6c783ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 0x7f6c70157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.830+0000 7f6c783ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c700afcf0 0x7f6c7016d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.830+0000 7f6c783ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c700c7190 0x7f6c7016da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.830+0000 7f6c783ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c70158350 con 0x7f6c700afcf0 2023-10-06T05:17:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.830+0000 7f6c783ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7016b730 con 0x7f6c700c7190 2023-10-06T05:17:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.830+0000 7f6c783ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7016b150 con 0x7f6c700aef00 2023-10-06T05:17:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.830+0000 7f6c7599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c700afcf0 0x7f6c7016d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c7599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c700afcf0 0x7f6c7016d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59526/0 (socket says 172.21.15.31:59526) 2023-10-06T05:17:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c7599a700 1 -- 172.21.15.31:0/3137192115 learned_addr learned my addr 172.21.15.31:0/3137192115 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c7619b700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 0x7f6c70157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c7699c700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c700c7190 0x7f6c7016da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c7599a700 1 -- 172.21.15.31:0/3137192115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 msgr2=0x7f6c70157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c7599a700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 0x7f6c70157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c7599a700 1 -- 172.21.15.31:0/3137192115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c700c7190 msgr2=0x7f6c7016da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c7599a700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c700c7190 0x7f6c7016da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c7599a700 1 -- 172.21.15.31:0/3137192115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c60009420 con 0x7f6c700afcf0 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c7599a700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c700afcf0 0x7f6c7016d4e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000ecb0 tx=0x7f6c600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c667fc700 1 -- 172.21.15.31:0/3137192115 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000c920 con 0x7f6c700afcf0 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c667fc700 1 -- 172.21.15.31:0/3137192115 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6000ca80 con 0x7f6c700afcf0 2023-10-06T05:17:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.831+0000 7f6c783ff700 1 -- 172.21.15.31:0/3137192115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c009420 con 0x7f6c700afcf0 2023-10-06T05:17:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.832+0000 7f6c667fc700 1 -- 172.21.15.31:0/3137192115 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000ce00 con 0x7f6c700afcf0 2023-10-06T05:17:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.832+0000 7f6c783ff700 1 -- 172.21.15.31:0/3137192115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c700c50c0 con 0x7f6c700afcf0 2023-10-06T05:17:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.833+0000 7f6c783ff700 1 -- 172.21.15.31:0/3137192115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c700c53e0 con 0x7f6c700afcf0 2023-10-06T05:17:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.834+0000 7f6c667fc700 1 -- 172.21.15.31:0/3137192115 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c60013680 con 0x7f6c700afcf0 2023-10-06T05:17:59.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.835+0000 7f6c667fc700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c54066f30 0x7f6c540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:17:59.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.835+0000 7f6c667fc700 1 -- 172.21.15.31:0/3137192115 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f6c600849a0 con 0x7f6c700afcf0 2023-10-06T05:17:59.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.835+0000 7f6c7619b700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c54066f30 0x7f6c540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:17:59.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.836+0000 7f6c7619b700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c54066f30 0x7f6c540693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0091e0 tx=0x7f6c6c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:17:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:17:59.840+0000 7f6c667fc700 1 -- 172.21.15.31:0/3137192115 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c60051a70 con 0x7f6c700afcf0 2023-10-06T05:18:00.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.088+0000 7f6c783ff700 1 -- 172.21.15.31:0/3137192115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6c7016b8c0 con 0x7f6c700afcf0 2023-10-06T05:18:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.249+0000 7f6c667fc700 1 -- 172.21.15.31:0/3137192115 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6c60055090 con 0x7f6c700afcf0 2023-10-06T05:18:00.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.256+0000 7f6c783ff700 1 -- 172.21.15.31:0/3137192115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c54066f30 msgr2=0x7f6c540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.256+0000 7f6c783ff700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c54066f30 0x7f6c540693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0091e0 tx=0x7f6c6c017000 comp rx=0 tx=0).stop 2023-10-06T05:18:00.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.256+0000 7f6c783ff700 1 -- 172.21.15.31:0/3137192115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c700afcf0 msgr2=0x7f6c7016d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.256+0000 7f6c783ff700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c700afcf0 0x7f6c7016d4e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000ecb0 tx=0x7f6c600091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.259+0000 7f6c783ff700 1 -- 172.21.15.31:0/3137192115 shutdown_connections 2023-10-06T05:18:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.259+0000 7f6c783ff700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c700c7190 0x7f6c7016da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.259+0000 7f6c783ff700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c700aef00 0x7f6c70157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.259+0000 7f6c783ff700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c54066f30 0x7f6c540693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.259+0000 7f6c783ff700 1 --2- 172.21.15.31:0/3137192115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c700afcf0 0x7f6c7016d4e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.259+0000 7f6c783ff700 1 -- 172.21.15.31:0/3137192115 >> 172.21.15.31:0/3137192115 conn(0x7f6c700cd3d0 msgr2=0x7f6c700ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.261+0000 7f6c783ff700 1 -- 172.21.15.31:0/3137192115 shutdown_connections 2023-10-06T05:18:00.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.263+0000 7f6c783ff700 1 -- 172.21.15.31:0/3137192115 wait complete. 2023-10-06T05:18:00.270 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 5 and data pool 6 2023-10-06T05:18:00.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:00.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:00.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:00.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.676+0000 7f2f0f5ff700 1 Processor -- start 2023-10-06T05:18:00.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.676+0000 7f2f0f5ff700 1 -- start start 2023-10-06T05:18:00.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.677+0000 7f2f0f5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2f080c7b60 0x7f2f08069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.677+0000 7f2f0f5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080aee70 0x7f2f0813f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.677+0000 7f2f0f5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f080ca5b0 con 0x7f2f080c7f30 2023-10-06T05:18:00.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.677+0000 7f2f0f5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f080af2d0 con 0x7f2f080c7b60 2023-10-06T05:18:00.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.677+0000 7f2f0f5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0806f740 con 0x7f2f080aee70 2023-10-06T05:18:00.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.677+0000 7f2f0d39b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2f080c7b60 0x7f2f08069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.678+0000 7f2f0d39b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2f080c7b60 0x7f2f08069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59538/0 (socket says 172.21.15.31:59538) 2023-10-06T05:18:00.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.678+0000 7f2f0d39b700 1 -- 172.21.15.31:0/2851631922 learned_addr learned my addr 172.21.15.31:0/2851631922 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:00.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.678+0000 7f2f0db9c700 1 --2- 172.21.15.31:0/2851631922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080aee70 0x7f2f0813f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.678+0000 7f2f0d39b700 1 -- 172.21.15.31:0/2851631922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080aee70 msgr2=0x7f2f0813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:00.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.678+0000 7f2f0d39b700 1 --2- 172.21.15.31:0/2851631922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080aee70 0x7f2f0813f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.678+0000 7f2f0d39b700 1 -- 172.21.15.31:0/2851631922 >> v1:172.21.15.31:6789/0 conn(0x7f2f080c7f30 legacy=0x7f2f0806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.678+0000 7f2f0d39b700 1 -- 172.21.15.31:0/2851631922 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080c9c80 con 0x7f2f080c7b60 2023-10-06T05:18:00.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.678+0000 7f2f0d39b700 1 --2- 172.21.15.31:0/2851631922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 0x7f2f08069880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009990 tx=0x7f2f04009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f68ead9a482876c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:00.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.678+0000 7f2efffff700 1 -- 172.21.15.31:0/2851631922 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04005c10 con 0x7f2f080c7b60 2023-10-06T05:18:00.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.679+0000 7f2efffff700 1 -- 172.21.15.31:0/2851631922 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f04005d70 con 0x7f2f080c7b60 2023-10-06T05:18:00.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.679+0000 7f2efffff700 1 -- 172.21.15.31:0/2851631922 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400b880 con 0x7f2f080c7b60 2023-10-06T05:18:00.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.679+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/2851631922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 msgr2=0x7f2f08069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.679+0000 7f2f0f5ff700 1 --2- 172.21.15.31:0/2851631922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 0x7f2f08069880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009990 tx=0x7f2f04009220 comp rx=0 tx=0).stop 2023-10-06T05:18:00.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.679+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/2851631922 shutdown_connections 2023-10-06T05:18:00.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.679+0000 7f2f0f5ff700 1 --2- 172.21.15.31:0/2851631922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 0x7f2f08069880 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.679+0000 7f2f0f5ff700 1 --2- 172.21.15.31:0/2851631922 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080aee70 0x7f2f0813f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.679+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/2851631922 >> 172.21.15.31:0/2851631922 conn(0x7f2f080cc0b0 msgr2=0x7f2f080cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:00.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.679+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/2851631922 shutdown_connections 2023-10-06T05:18:00.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.679+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/2851631922 wait complete. 2023-10-06T05:18:00.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.681+0000 7f2f0f5ff700 1 Processor -- start 2023-10-06T05:18:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.681+0000 7f2f0f5ff700 1 -- start start 2023-10-06T05:18:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0f5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f080aee70 0x7f2f081698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0f5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 0x7f2f0813eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0f5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080c7f30 0x7f2f0815bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0f5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080ab700 con 0x7f2f080c7b60 2023-10-06T05:18:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0f5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080ab9e0 con 0x7f2f080aee70 2023-10-06T05:18:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0f5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0815c570 con 0x7f2f080c7f30 2023-10-06T05:18:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0cb9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 0x7f2f0813eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0cb9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 0x7f2f0813eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59542/0 (socket says 172.21.15.31:59542) 2023-10-06T05:18:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0cb9a700 1 -- 172.21.15.31:0/3491759781 learned_addr learned my addr 172.21.15.31:0/3491759781 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0d39b700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f080aee70 0x7f2f081698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0db9c700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080c7f30 0x7f2f0815bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0cb9a700 1 -- 172.21.15.31:0/3491759781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080c7f30 msgr2=0x7f2f0815bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0cb9a700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080c7f30 0x7f2f0815bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0cb9a700 1 -- 172.21.15.31:0/3491759781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f080aee70 msgr2=0x7f2f081698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0cb9a700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f080aee70 0x7f2f081698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.682+0000 7f2f0cb9a700 1 -- 172.21.15.31:0/3491759781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f04009480 con 0x7f2f080c7b60 2023-10-06T05:18:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.683+0000 7f2f0cb9a700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 0x7f2f0813eae0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800c510 tx=0x7f2ef800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.683+0000 7f2efd7fa700 1 -- 172.21.15.31:0/3491759781 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8011590 con 0x7f2f080c7b60 2023-10-06T05:18:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.683+0000 7f2efd7fa700 1 -- 172.21.15.31:0/3491759781 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef8011710 con 0x7f2f080c7b60 2023-10-06T05:18:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.683+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/3491759781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0815c850 con 0x7f2f080c7b60 2023-10-06T05:18:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.683+0000 7f2efd7fa700 1 -- 172.21.15.31:0/3491759781 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8011ad0 con 0x7f2f080c7b60 2023-10-06T05:18:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.683+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/3491759781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f08169e20 con 0x7f2f080c7b60 2023-10-06T05:18:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.684+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/3491759781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0804c380 con 0x7f2f080c7b60 2023-10-06T05:18:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.685+0000 7f2efd7fa700 1 -- 172.21.15.31:0/3491759781 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ef801b3b0 con 0x7f2f080c7b60 2023-10-06T05:18:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.686+0000 7f2efd7fa700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eec066f30 0x7f2eec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.686+0000 7f2efd7fa700 1 -- 172.21.15.31:0/3491759781 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2ef8083af0 con 0x7f2f080c7b60 2023-10-06T05:18:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.687+0000 7f2f0d39b700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eec066f30 0x7f2eec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.687+0000 7f2f0d39b700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eec066f30 0x7f2eec0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f04007fd0 tx=0x7f2f04016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.691+0000 7f2efd7fa700 1 -- 172.21.15.31:0/3491759781 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ef8050b60 con 0x7f2f080c7b60 2023-10-06T05:18:00.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.893+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/3491759781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f080abb20 con 0x7f2f080c7b60 2023-10-06T05:18:00.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.894+0000 7f2efd7fa700 1 -- 172.21.15.31:0/3491759781 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16927 (secure 0 0 0) 0x7f2ef8017070 con 0x7f2f080c7b60 2023-10-06T05:18:00.901 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:00.902 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":31,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:00.089680+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:58.074919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:58.854233+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:00.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/3491759781 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eec066f30 msgr2=0x7f2eec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eec066f30 0x7f2eec0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f04007fd0 tx=0x7f2f04016000 comp rx=0 tx=0).stop 2023-10-06T05:18:00.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/3491759781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 msgr2=0x7f2f0813eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:00.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 0x7f2f0813eae0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800c510 tx=0x7f2ef800c820 comp rx=0 tx=0).stop 2023-10-06T05:18:00.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/3491759781 shutdown_connections 2023-10-06T05:18:00.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080c7f30 0x7f2f0815bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f080aee70 0x7f2f081698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eec066f30 0x7f2eec0693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 --2- 172.21.15.31:0/3491759781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f080c7b60 0x7f2f0813eae0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:00.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/3491759781 >> 172.21.15.31:0/3491759781 conn(0x7f2f080cc0b0 msgr2=0x7f2f080c1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:00.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/3491759781 shutdown_connections 2023-10-06T05:18:00.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:00.897+0000 7f2f0f5ff700 1 -- 172.21.15.31:0/3491759781 wait complete. 2023-10-06T05:18:00.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.112+0000 7f91d1fff700 1 Processor -- start 2023-10-06T05:18:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.112+0000 7f91d1fff700 1 -- start start 2023-10-06T05:18:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.113+0000 7f91d1fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91cc1595a0 0x7f91cc159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.113+0000 7f91d1fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91cc0cc0d0 0x7f91cc068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91d1fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc069010 con 0x7f91cc068bb0 2023-10-06T05:18:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91d1fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc0c95b0 con 0x7f91cc0cc0d0 2023-10-06T05:18:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91d1fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0c9890 con 0x7f91cc1595a0 2023-10-06T05:18:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91caffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91cc0cc0d0 0x7f91cc068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91caffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91cc0cc0d0 0x7f91cc068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59550/0 (socket says 172.21.15.31:59550) 2023-10-06T05:18:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91caffd700 1 -- 172.21.15.31:0/2913697883 learned_addr learned my addr 172.21.15.31:0/2913697883 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91cb7fe700 1 --2- 172.21.15.31:0/2913697883 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91cc1595a0 0x7f91cc159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91caffd700 1 -- 172.21.15.31:0/2913697883 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91cc1595a0 msgr2=0x7f91cc159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91caffd700 1 --2- 172.21.15.31:0/2913697883 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91cc1595a0 0x7f91cc159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91caffd700 1 -- 172.21.15.31:0/2913697883 >> v1:172.21.15.62:6789/0 conn(0x7f91cc068bb0 legacy=0x7f91cc0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.114+0000 7f91caffd700 1 -- 172.21.15.31:0/2913697883 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0c9b70 con 0x7f91cc0cc0d0 2023-10-06T05:18:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.115+0000 7f91caffd700 1 --2- 172.21.15.31:0/2913697883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc0cc0d0 0x7f91cc068670 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f91c00099b0 tx=0x7f91c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a649c4a1d0f432f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:01.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.115+0000 7f91ca7fc700 1 -- 172.21.15.31:0/2913697883 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0009ea0 con 0x7f91cc0cc0d0 2023-10-06T05:18:01.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.115+0000 7f91ca7fc700 1 -- 172.21.15.31:0/2913697883 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c000b860 con 0x7f91cc0cc0d0 2023-10-06T05:18:01.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.115+0000 7f91ca7fc700 1 -- 172.21.15.31:0/2913697883 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c000bc00 con 0x7f91cc0cc0d0 2023-10-06T05:18:01.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.115+0000 7f91d1fff700 1 -- 172.21.15.31:0/2913697883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc0cc0d0 msgr2=0x7f91cc068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.115+0000 7f91d1fff700 1 --2- 172.21.15.31:0/2913697883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc0cc0d0 0x7f91cc068670 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f91c00099b0 tx=0x7f91c0009250 comp rx=0 tx=0).stop 2023-10-06T05:18:01.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.116+0000 7f91d1fff700 1 -- 172.21.15.31:0/2913697883 shutdown_connections 2023-10-06T05:18:01.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.116+0000 7f91d1fff700 1 --2- 172.21.15.31:0/2913697883 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91cc1595a0 0x7f91cc159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.116+0000 7f91d1fff700 1 --2- 172.21.15.31:0/2913697883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc0cc0d0 0x7f91cc068670 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.116+0000 7f91d1fff700 1 -- 172.21.15.31:0/2913697883 >> 172.21.15.31:0/2913697883 conn(0x7f91cc06b620 msgr2=0x7f91cc06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:01.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.116+0000 7f91d1fff700 1 -- 172.21.15.31:0/2913697883 shutdown_connections 2023-10-06T05:18:01.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.116+0000 7f91d1fff700 1 -- 172.21.15.31:0/2913697883 wait complete. 2023-10-06T05:18:01.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.117+0000 7f91d1fff700 1 Processor -- start 2023-10-06T05:18:01.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.117+0000 7f91d1fff700 1 -- start start 2023-10-06T05:18:01.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91d1fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91cc068bb0 0x7f91cc0b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91d1fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91cc0cc0d0 0x7f91cc137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91d1fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc1595a0 0x7f91cc138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91d1fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc138880 con 0x7f91cc1595a0 2023-10-06T05:18:01.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91d1fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc138ad0 con 0x7f91cc068bb0 2023-10-06T05:18:01.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91d1fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0b28a0 con 0x7f91cc0cc0d0 2023-10-06T05:18:01.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc1595a0 0x7f91cc138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc1595a0 0x7f91cc138340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59566/0 (socket says 172.21.15.31:59566) 2023-10-06T05:18:01.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91cbfff700 1 -- 172.21.15.31:0/2902415826 learned_addr learned my addr 172.21.15.31:0/2902415826 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:01.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91cb7fe700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91cc068bb0 0x7f91cc0b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.118+0000 7f91caffd700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91cc0cc0d0 0x7f91cc137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.119+0000 7f91cbfff700 1 -- 172.21.15.31:0/2902415826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91cc0cc0d0 msgr2=0x7f91cc137e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.119+0000 7f91cbfff700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91cc0cc0d0 0x7f91cc137e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.119+0000 7f91cbfff700 1 -- 172.21.15.31:0/2902415826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91cc068bb0 msgr2=0x7f91cc0b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.119+0000 7f91cbfff700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91cc068bb0 0x7f91cc0b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.119+0000 7f91cbfff700 1 -- 172.21.15.31:0/2902415826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00094f0 con 0x7f91cc1595a0 2023-10-06T05:18:01.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.119+0000 7f91cbfff700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc1595a0 0x7f91cc138340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f91c400c8e0 tx=0x7f91c4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:01.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.119+0000 7f91b37fe700 1 -- 172.21.15.31:0/2902415826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4012630 con 0x7f91cc1595a0 2023-10-06T05:18:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.119+0000 7f91b37fe700 1 -- 172.21.15.31:0/2902415826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c4012790 con 0x7f91cc1595a0 2023-10-06T05:18:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.119+0000 7f91b37fe700 1 -- 172.21.15.31:0/2902415826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4012b50 con 0x7f91cc1595a0 2023-10-06T05:18:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.119+0000 7f91d1fff700 1 -- 172.21.15.31:0/2902415826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc0b1c90 con 0x7f91cc1595a0 2023-10-06T05:18:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.120+0000 7f91d1fff700 1 -- 172.21.15.31:0/2902415826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc0b2250 con 0x7f91cc1595a0 2023-10-06T05:18:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.120+0000 7f91d1fff700 1 -- 172.21.15.31:0/2902415826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc0b2a30 con 0x7f91cc1595a0 2023-10-06T05:18:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.121+0000 7f91b37fe700 1 -- 172.21.15.31:0/2902415826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91c401a3a0 con 0x7f91cc1595a0 2023-10-06T05:18:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.122+0000 7f91b37fe700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f91b8066f60 0x7f91b8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.122+0000 7f91b37fe700 1 -- 172.21.15.31:0/2902415826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91c40847b0 con 0x7f91cc1595a0 2023-10-06T05:18:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.122+0000 7f91cb7fe700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f91b8066f60 0x7f91b8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.123+0000 7f91cb7fe700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f91b8066f60 0x7f91b8069410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009d10 tx=0x7f91bc0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.127+0000 7f91b37fe700 1 -- 172.21.15.31:0/2902415826 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91c40552b0 con 0x7f91cc1595a0 2023-10-06T05:18:01.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.335+0000 7f91d1fff700 1 -- 172.21.15.31:0/2902415826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91cc069010 con 0x7f91cc1595a0 2023-10-06T05:18:01.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.336+0000 7f91b37fe700 1 -- 172.21.15.31:0/2902415826 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f91cc069010 con 0x7f91cc1595a0 2023-10-06T05:18:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:01.374 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:01.245088+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:58.074919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:58.854233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 -- 172.21.15.31:0/2902415826 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f91b8066f60 msgr2=0x7f91b8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f91b8066f60 0x7f91b8069410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009d10 tx=0x7f91bc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 -- 172.21.15.31:0/2902415826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc1595a0 msgr2=0x7f91cc138340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc1595a0 0x7f91cc138340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f91c400c8e0 tx=0x7f91c4001710 comp rx=0 tx=0).stop 2023-10-06T05:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 -- 172.21.15.31:0/2902415826 shutdown_connections 2023-10-06T05:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f91b8066f60 0x7f91b8069410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91cc1595a0 0x7f91cc138340 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91cc068bb0 0x7f91cc0b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 --2- 172.21.15.31:0/2902415826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91cc0cc0d0 0x7f91cc137e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 -- 172.21.15.31:0/2902415826 >> 172.21.15.31:0/2902415826 conn(0x7f91cc06b620 msgr2=0x7f91cc15b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:01.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 -- 172.21.15.31:0/2902415826 shutdown_connections 2023-10-06T05:18:01.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.338+0000 7f91d1fff700 1 -- 172.21.15.31:0/2902415826 wait complete. 2023-10-06T05:18:01.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:18:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.586+0000 7fc33abff700 1 Processor -- start 2023-10-06T05:18:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.586+0000 7fc33abff700 1 -- start start 2023-10-06T05:18:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.588+0000 7fc33abff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340b2820 0x7fc3340c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.588+0000 7fc33abff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3340cbae0 con 0x7fc3340cce60 2023-10-06T05:18:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.588+0000 7fc33abff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3340cf120 con 0x7fc3340b2bf0 2023-10-06T05:18:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.588+0000 7fc33abff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340cb280 con 0x7fc3340b2820 2023-10-06T05:18:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.588+0000 7fc33919c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc3340cce60 0x7fc3340cbf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44388/0 (socket says 172.21.15.31:44388) 2023-10-06T05:18:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.588+0000 7fc33919c700 1 -- 172.21.15.31:0/405122664 learned_addr learned my addr 172.21.15.31:0/405122664 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:01.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc33899b700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340b2820 0x7fc3340c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124465388 0 0) 0x7fc3340cbae0 con 0x7fc3340cce60 2023-10-06T05:18:01.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc324003570 con 0x7fc3340cce60 2023-10-06T05:18:01.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375245676 0 0) 0x7fc3340cf120 con 0x7fc3340b2bf0 2023-10-06T05:18:01.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3340cbae0 con 0x7fc3340b2bf0 2023-10-06T05:18:01.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 908051713 0 0) 0x7fc324003570 con 0x7fc3340cce60 2023-10-06T05:18:01.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340b2820 msgr2=0x7fc3340c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:01.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340b2820 0x7fc3340c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 >> v1:172.21.15.62:6789/0 conn(0x7fc3340b2bf0 legacy=0x7fc3340cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3340cf3f0 con 0x7fc3340cce60 2023-10-06T05:18:01.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fc32c0017f0 con 0x7fc3340cce60 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc33899b700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340b2820 0x7fc3340c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 >> v1:172.21.15.31:6789/0 conn(0x7fc3340cce60 legacy=0x7fc3340cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc324003c20 0x7fc324004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340b2820 0x7fc324004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc324004d50 0x7fc3240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340cb280 con 0x7fc324003c20 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32400bb70 con 0x7fc324004d50 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32400bd30 con 0x7fc3340b2820 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.589+0000 7fc33899b700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc324003c20 0x7fc324004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc333fff700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340b2820 0x7fc324004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc33919c700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc324004d50 0x7fc3240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc33899b700 1 -- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340b2820 msgr2=0x7fc324004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc33899b700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340b2820 0x7fc324004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc33899b700 1 -- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc324004d50 msgr2=0x7fc3240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc33899b700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc324004d50 0x7fc3240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc33899b700 1 -- 172.21.15.31:0/405122664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280093c0 con 0x7fc324003c20 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc333fff700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340b2820 0x7fc324004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc33899b700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc324003c20 0x7fc324004010 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc328009de0 tx=0x7fc328009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce3b278be279c64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328009730 con 0x7fc324003c20 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc328005a00 con 0x7fc324003c20 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.590+0000 7fc3337fe700 1 -- 172.21.15.31:0/405122664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328005dc0 con 0x7fc324003c20 2023-10-06T05:18:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.591+0000 7fc33abff700 1 -- 172.21.15.31:0/405122664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc324003c20 msgr2=0x7fc324004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.591+0000 7fc33abff700 1 --2- 172.21.15.31:0/405122664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc324003c20 0x7fc324004010 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc328009de0 tx=0x7fc328009120 comp rx=0 tx=0).stop 2023-10-06T05:18:01.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.591+0000 7fc33919c700 1 -- 172.21.15.31:0/405122664 reap_dead start 2023-10-06T05:18:01.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.591+0000 7fc33abff700 1 -- 172.21.15.31:0/405122664 shutdown_connections 2023-10-06T05:18:01.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.591+0000 7fc33abff700 1 -- 172.21.15.31:0/405122664 >> 172.21.15.31:0/405122664 conn(0x7fc3340996f0 msgr2=0x7fc33406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:01.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.591+0000 7fc33abff700 1 -- 172.21.15.31:0/405122664 shutdown_connections 2023-10-06T05:18:01.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.591+0000 7fc33abff700 1 -- 172.21.15.31:0/405122664 wait complete. 2023-10-06T05:18:01.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.593+0000 7fc33abff700 1 Processor -- start 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.593+0000 7fc33abff700 1 -- start start 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc33abff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc324003c20 0x7fc33416c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc33abff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc33416aa10 0x7fc3340c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc33abff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340c7a90 0x7fc3340b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc33abff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340c85b0 con 0x7fc33416aa10 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc33abff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33416cc90 con 0x7fc324003c20 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc33abff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340c95b0 con 0x7fc3340c7a90 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc333fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc33416aa10 0x7fc3340c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc333fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc33416aa10 0x7fc3340c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59590/0 (socket says 172.21.15.31:59590) 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc333fff700 1 -- 172.21.15.31:0/959504255 learned_addr learned my addr 172.21.15.31:0/959504255 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:01.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc33899b700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc324003c20 0x7fc33416c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc33919c700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340c7a90 0x7fc3340b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc333fff700 1 -- 172.21.15.31:0/959504255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340c7a90 msgr2=0x7fc3340b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc333fff700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340c7a90 0x7fc3340b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc333fff700 1 -- 172.21.15.31:0/959504255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc324003c20 msgr2=0x7fc33416c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc333fff700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc324003c20 0x7fc33416c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.595+0000 7fc333fff700 1 -- 172.21.15.31:0/959504255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3340c7ff0 con 0x7fc33416aa10 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.596+0000 7fc333fff700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc33416aa10 0x7fc3340c7550 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc32000e810 tx=0x7fc320009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.596+0000 7fc330ff9700 1 -- 172.21.15.31:0/959504255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320006820 con 0x7fc33416aa10 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.596+0000 7fc330ff9700 1 -- 172.21.15.31:0/959504255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc320006980 con 0x7fc33416aa10 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.596+0000 7fc33abff700 1 -- 172.21.15.31:0/959504255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3340c82d0 con 0x7fc33416aa10 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.596+0000 7fc330ff9700 1 -- 172.21.15.31:0/959504255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320006d00 con 0x7fc33416aa10 2023-10-06T05:18:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.596+0000 7fc33abff700 1 -- 172.21.15.31:0/959504255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc334169ae0 con 0x7fc33416aa10 2023-10-06T05:18:01.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.597+0000 7fc33abff700 1 -- 172.21.15.31:0/959504255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3340c90a0 con 0x7fc33416aa10 2023-10-06T05:18:01.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.598+0000 7fc330ff9700 1 -- 172.21.15.31:0/959504255 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc320007230 con 0x7fc33416aa10 2023-10-06T05:18:01.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.599+0000 7fc330ff9700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc318066f30 0x7fc3180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:01.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.599+0000 7fc330ff9700 1 -- 172.21.15.31:0/959504255 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc320084930 con 0x7fc33416aa10 2023-10-06T05:18:01.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.599+0000 7fc33899b700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc318066f30 0x7fc3180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:01.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.600+0000 7fc33899b700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc318066f30 0x7fc3180693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc328009390 tx=0x7fc328019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:01.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.603+0000 7fc330ff9700 1 -- 172.21.15.31:0/959504255 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3340c90a0 con 0x7fc33416aa10 2023-10-06T05:18:01.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:01.853+0000 7fc33abff700 1 -- 172.21.15.31:0/959504255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc3340c90a0 con 0x7fc33416aa10 2023-10-06T05:18:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.258+0000 7fc330ff9700 1 -- 172.21.15.31:0/959504255 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc3340c90a0 con 0x7fc33416aa10 2023-10-06T05:18:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 -- 172.21.15.31:0/959504255 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc318066f30 msgr2=0x7fc3180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc318066f30 0x7fc3180693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc328009390 tx=0x7fc328019000 comp rx=0 tx=0).stop 2023-10-06T05:18:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 -- 172.21.15.31:0/959504255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc33416aa10 msgr2=0x7fc3340c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc33416aa10 0x7fc3340c7550 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc32000e810 tx=0x7fc320009040 comp rx=0 tx=0).stop 2023-10-06T05:18:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 -- 172.21.15.31:0/959504255 shutdown_connections 2023-10-06T05:18:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc3340c7a90 0x7fc3340b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc324003c20 0x7fc33416c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc318066f30 0x7fc3180693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 --2- 172.21.15.31:0/959504255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc33416aa10 0x7fc3340c7550 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 -- 172.21.15.31:0/959504255 >> 172.21.15.31:0/959504255 conn(0x7fc3340996f0 msgr2=0x7fc3340697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 -- 172.21.15.31:0/959504255 shutdown_connections 2023-10-06T05:18:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.261+0000 7fc33abff700 1 -- 172.21.15.31:0/959504255 wait complete. 2023-10-06T05:18:02.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:18:02.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.489+0000 7f6d3fbff700 1 Processor -- start 2023-10-06T05:18:02.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.490+0000 7f6d3fbff700 1 -- start start 2023-10-06T05:18:02.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.491+0000 7f6d3fbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6d380cd570 0x7f6d380c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.491+0000 7f6d3fbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d380cbae0 con 0x7f6d380b2820 2023-10-06T05:18:02.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.491+0000 7f6d3fbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d380cf120 con 0x7f6d380b2bf0 2023-10-06T05:18:02.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.491+0000 7f6d3fbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d380cb280 con 0x7f6d380cd570 2023-10-06T05:18:02.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.491+0000 7f6d3d99b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6d380b2820 0x7f6d380cea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44396/0 (socket says 172.21.15.31:44396) 2023-10-06T05:18:02.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.491+0000 7f6d3d99b700 1 -- 172.21.15.31:0/2114453103 learned_addr learned my addr 172.21.15.31:0/2114453103 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:02.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.491+0000 7f6d3e19c700 1 --2- 172.21.15.31:0/2114453103 >> v2:172.21.15.62:3300/0 conn(0x7f6d380cd570 0x7f6d380c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774583095 0 0) 0x7f6d380cbae0 con 0x7f6d380b2820 2023-10-06T05:18:02.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d24003570 con 0x7f6d380b2820 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871949936 0 0) 0x7f6d380cf120 con 0x7f6d380b2bf0 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d380cbae0 con 0x7f6d380b2bf0 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2917697863 0 0) 0x7f6d24003570 con 0x7f6d380b2820 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 >> v2:172.21.15.62:3300/0 conn(0x7f6d380cd570 msgr2=0x7f6d380c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 --2- 172.21.15.31:0/2114453103 >> v2:172.21.15.62:3300/0 conn(0x7f6d380cd570 0x7f6d380c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 >> v1:172.21.15.62:6789/0 conn(0x7f6d380b2bf0 legacy=0x7f6d380cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380cf3f0 con 0x7f6d380b2820 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f6d340015f0 con 0x7f6d380b2820 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 >> v1:172.21.15.31:6789/0 conn(0x7f6d380b2820 legacy=0x7f6d380cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 --2- 172.21.15.31:0/2114453103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d24004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 --2- 172.21.15.31:0/2114453103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d24004810 0x7f6d24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 --2- 172.21.15.31:0/2114453103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d24005210 0x7f6d2400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380cb280 con 0x7f6d24003c40 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2400c1b0 con 0x7f6d24004810 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2400c370 con 0x7f6d24005210 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3e19c700 1 --2- 172.21.15.31:0/2114453103 >> v2:172.21.15.62:3300/0 conn(0x7f6d380cd570 0x7f6d380c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.492+0000 7f6d3d99b700 1 --2- 172.21.15.31:0/2114453103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d24004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3d19a700 1 --2- 172.21.15.31:0/2114453103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d24004810 0x7f6d24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3d99b700 1 -- 172.21.15.31:0/2114453103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d24005210 msgr2=0x7f6d2400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3d99b700 1 --2- 172.21.15.31:0/2114453103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d24005210 0x7f6d2400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3d99b700 1 -- 172.21.15.31:0/2114453103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d24004810 msgr2=0x7f6d24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3d99b700 1 --2- 172.21.15.31:0/2114453103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d24004810 0x7f6d24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3d99b700 1 -- 172.21.15.31:0/2114453103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d24003570 con 0x7f6d24003c40 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3e19c700 1 -- 172.21.15.31:0/2114453103 reap_dead start 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3d99b700 1 --2- 172.21.15.31:0/2114453103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d24004030 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400c1e0 tx=0x7f6d34007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e06ca0a27941b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3400cad0 con 0x7f6d24003c40 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3400cc50 con 0x7f6d24003c40 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.493+0000 7f6d3c999700 1 -- 172.21.15.31:0/2114453103 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d340070d0 con 0x7f6d24003c40 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.494+0000 7f6d3fbff700 1 -- 172.21.15.31:0/2114453103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 msgr2=0x7f6d24004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.494+0000 7f6d3fbff700 1 --2- 172.21.15.31:0/2114453103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d24004030 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400c1e0 tx=0x7f6d34007810 comp rx=0 tx=0).stop 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.494+0000 7f6d3fbff700 1 -- 172.21.15.31:0/2114453103 shutdown_connections 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.494+0000 7f6d3fbff700 1 --2- 172.21.15.31:0/2114453103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d24004030 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.494+0000 7f6d3fbff700 1 -- 172.21.15.31:0/2114453103 >> 172.21.15.31:0/2114453103 conn(0x7f6d380996f0 msgr2=0x7f6d3806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:02.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.494+0000 7f6d3fbff700 1 -- 172.21.15.31:0/2114453103 shutdown_connections 2023-10-06T05:18:02.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.494+0000 7f6d3fbff700 1 -- 172.21.15.31:0/2114453103 wait complete. 2023-10-06T05:18:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.495+0000 7f6d3fbff700 1 Processor -- start 2023-10-06T05:18:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.496+0000 7f6d3fbff700 1 -- start start 2023-10-06T05:18:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.496+0000 7f6d3fbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d3809c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.496+0000 7f6d3fbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d380b5840 0x7f6d38169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.496+0000 7f6d3fbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d3816c8f0 0x7f6d3809a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3fbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380b61f0 con 0x7f6d24003c40 2023-10-06T05:18:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3fbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380b3bd0 con 0x7f6d380b5840 2023-10-06T05:18:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3fbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3816a130 con 0x7f6d3816c8f0 2023-10-06T05:18:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d3809c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3d99b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d3809c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59616/0 (socket says 172.21.15.31:59616) 2023-10-06T05:18:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3d99b700 1 -- 172.21.15.31:0/3966282550 learned_addr learned my addr 172.21.15.31:0/3966282550 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3d19a700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d380b5840 0x7f6d38169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3e19c700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d3816c8f0 0x7f6d3809a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3d99b700 1 -- 172.21.15.31:0/3966282550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d3816c8f0 msgr2=0x7f6d3809a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3d99b700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d3816c8f0 0x7f6d3809a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3d99b700 1 -- 172.21.15.31:0/3966282550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d380b5840 msgr2=0x7f6d38169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3d99b700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d380b5840 0x7f6d38169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3d99b700 1 -- 172.21.15.31:0/3966282550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380b5cb0 con 0x7f6d24003c40 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3e19c700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d3816c8f0 0x7f6d3809a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.497+0000 7f6d3d99b700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d3809c3c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d34011000 tx=0x7f6d34015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.498+0000 7f6d2dffb700 1 -- 172.21.15.31:0/3966282550 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34001700 con 0x7f6d24003c40 2023-10-06T05:18:02.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.498+0000 7f6d2dffb700 1 -- 172.21.15.31:0/3966282550 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d34001860 con 0x7f6d24003c40 2023-10-06T05:18:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.498+0000 7f6d3fbff700 1 -- 172.21.15.31:0/3966282550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d380b32e0 con 0x7f6d24003c40 2023-10-06T05:18:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.498+0000 7f6d2dffb700 1 -- 172.21.15.31:0/3966282550 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34001d10 con 0x7f6d24003c40 2023-10-06T05:18:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.498+0000 7f6d3fbff700 1 -- 172.21.15.31:0/3966282550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d380b3860 con 0x7f6d24003c40 2023-10-06T05:18:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.499+0000 7f6d3fbff700 1 -- 172.21.15.31:0/3966282550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3804c380 con 0x7f6d24003c40 2023-10-06T05:18:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.500+0000 7f6d2dffb700 1 -- 172.21.15.31:0/3966282550 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d34005a00 con 0x7f6d24003c40 2023-10-06T05:18:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.501+0000 7f6d2dffb700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d1c066f30 0x7f6d1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.501+0000 7f6d2dffb700 1 -- 172.21.15.31:0/3966282550 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6d34085920 con 0x7f6d24003c40 2023-10-06T05:18:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.501+0000 7f6d3d19a700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d1c066f30 0x7f6d1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.502+0000 7f6d3d19a700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d1c066f30 0x7f6d1c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d38069cb0 tx=0x7f6d28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.506+0000 7f6d2dffb700 1 -- 172.21.15.31:0/3966282550 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d34052990 con 0x7f6d24003c40 2023-10-06T05:18:02.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:02.750+0000 7f6d3fbff700 1 -- 172.21.15.31:0/3966282550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6d380cb490 con 0x7f6d24003c40 2023-10-06T05:18:03.211 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-10-06T05:18:03.209+0000 7f985cff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:18:03.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.264+0000 7f6d2dffb700 1 -- 172.21.15.31:0/3966282550 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6d34005cb0 con 0x7f6d24003c40 2023-10-06T05:18:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 -- 172.21.15.31:0/3966282550 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d1c066f30 msgr2=0x7f6d1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d1c066f30 0x7f6d1c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d38069cb0 tx=0x7f6d28009040 comp rx=0 tx=0).stop 2023-10-06T05:18:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 -- 172.21.15.31:0/3966282550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 msgr2=0x7f6d3809c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d3809c3c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d34011000 tx=0x7f6d34015000 comp rx=0 tx=0).stop 2023-10-06T05:18:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 -- 172.21.15.31:0/3966282550 shutdown_connections 2023-10-06T05:18:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d3816c8f0 0x7f6d3809a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d1c066f30 0x7f6d1c0693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d24003c40 0x7f6d3809c3c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 --2- 172.21.15.31:0/3966282550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d380b5840 0x7f6d38169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 -- 172.21.15.31:0/3966282550 >> 172.21.15.31:0/3966282550 conn(0x7f6d380996f0 msgr2=0x7f6d380c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 -- 172.21.15.31:0/3966282550 shutdown_connections 2023-10-06T05:18:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.267+0000 7f6d3fbff700 1 -- 172.21.15.31:0/3966282550 wait complete. 2023-10-06T05:18:03.289 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.546+0000 7f2cc37ff700 1 Processor -- start 2023-10-06T05:18:03.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.546+0000 7f2cc37ff700 1 -- start start 2023-10-06T05:18:03.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc37ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cbc0cd510 0x7f2cbc0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc37ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2cbc15d2d0 0x7f2cbc0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc37ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc0c3f40 con 0x7f2cbc159500 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc37ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b2860 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc37ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc15d7d0 con 0x7f2cbc0cd510 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc1d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2cbc15d2d0 0x7f2cbc0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc1d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2cbc15d2d0 0x7f2cbc0aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59632/0 (socket says 172.21.15.31:59632) 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc1d9c700 1 -- 172.21.15.31:0/549212275 learned_addr learned my addr 172.21.15.31:0/549212275 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc0d9a700 1 --2- 172.21.15.31:0/549212275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cbc0cd510 0x7f2cbc0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc1d9c700 1 -- 172.21.15.31:0/549212275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cbc0cd510 msgr2=0x7f2cbc0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc1d9c700 1 --2- 172.21.15.31:0/549212275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cbc0cd510 0x7f2cbc0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.548+0000 7f2cc1d9c700 1 -- 172.21.15.31:0/549212275 >> v1:172.21.15.62:6789/0 conn(0x7f2cbc159500 legacy=0x7f2cbc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.549+0000 7f2cc1d9c700 1 -- 172.21.15.31:0/549212275 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0cb000 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.549+0000 7f2cc1d9c700 1 --2- 172.21.15.31:0/549212275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 0x7f2cbc0aedd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009f20 tx=0x7f2cb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7122facd9aa6a0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.549+0000 7f2cb3fff700 1 -- 172.21.15.31:0/549212275 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb400e070 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.549+0000 7f2cb3fff700 1 -- 172.21.15.31:0/549212275 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb400ba80 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.549+0000 7f2cb3fff700 1 -- 172.21.15.31:0/549212275 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40092e0 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.550+0000 7f2cc37ff700 1 -- 172.21.15.31:0/549212275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 msgr2=0x7f2cbc0aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.550+0000 7f2cc37ff700 1 --2- 172.21.15.31:0/549212275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 0x7f2cbc0aedd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009f20 tx=0x7f2cb4009840 comp rx=0 tx=0).stop 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.550+0000 7f2cc37ff700 1 -- 172.21.15.31:0/549212275 shutdown_connections 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.550+0000 7f2cc37ff700 1 --2- 172.21.15.31:0/549212275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 0x7f2cbc0aedd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.550+0000 7f2cc37ff700 1 --2- 172.21.15.31:0/549212275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cbc0cd510 0x7f2cbc0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.550+0000 7f2cc37ff700 1 -- 172.21.15.31:0/549212275 >> 172.21.15.31:0/549212275 conn(0x7f2cbc06ec90 msgr2=0x7f2cbc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.550+0000 7f2cc37ff700 1 -- 172.21.15.31:0/549212275 shutdown_connections 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.550+0000 7f2cc37ff700 1 -- 172.21.15.31:0/549212275 wait complete. 2023-10-06T05:18:03.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.551+0000 7f2cc37ff700 1 Processor -- start 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.551+0000 7f2cc37ff700 1 -- start start 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.552+0000 7f2cc37ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cbc0cd510 0x7f2cbc0b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.552+0000 7f2cc37ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc159500 0x7f2cbc0c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.552+0000 7f2cc37ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 0x7f2cbc158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.552+0000 7f2cc37ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0c7c50 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.552+0000 7f2cc37ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc159020 con 0x7f2cbc159500 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.552+0000 7f2cc37ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b5e40 con 0x7f2cbc0cd510 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc1d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 0x7f2cbc158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc1d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 0x7f2cbc158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59648/0 (socket says 172.21.15.31:59648) 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc1d9c700 1 -- 172.21.15.31:0/1011916332 learned_addr learned my addr 172.21.15.31:0/1011916332 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:03.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc159b700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cbc0cd510 0x7f2cbc0b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc0d9a700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc159500 0x7f2cbc0c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc1d9c700 1 -- 172.21.15.31:0/1011916332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cbc0cd510 msgr2=0x7f2cbc0b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc1d9c700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cbc0cd510 0x7f2cbc0b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc1d9c700 1 -- 172.21.15.31:0/1011916332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc159500 msgr2=0x7f2cbc0c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc1d9c700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc159500 0x7f2cbc0c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc1d9c700 1 -- 172.21.15.31:0/1011916332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4009ae0 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc0d9a700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc159500 0x7f2cbc0c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.553+0000 7f2cc1d9c700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 0x7f2cbc158ae0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400b6c0 tx=0x7f2cb4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.554+0000 7f2cb17fa700 1 -- 172.21.15.31:0/1011916332 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40032f0 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.554+0000 7f2cb17fa700 1 -- 172.21.15.31:0/1011916332 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb4008dd0 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.554+0000 7f2cb17fa700 1 -- 172.21.15.31:0/1011916332 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4025670 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.554+0000 7f2cc37ff700 1 -- 172.21.15.31:0/1011916332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc0b6120 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.554+0000 7f2cc37ff700 1 -- 172.21.15.31:0/1011916332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc15b4f0 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.555+0000 7f2ca6ffd700 1 -- 172.21.15.31:0/1011916332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc15b070 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.556+0000 7f2cb17fa700 1 -- 172.21.15.31:0/1011916332 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2cb4003040 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.557+0000 7f2cb17fa700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ca0066f30 0x7f2ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:03.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.557+0000 7f2cb17fa700 1 -- 172.21.15.31:0/1011916332 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2cb40860a0 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.557+0000 7f2cc159b700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ca0066f30 0x7f2ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:03.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.557+0000 7f2cc159b700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ca0066f30 0x7f2ca00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8003a80 tx=0x7f2cb8007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:03.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.561+0000 7f2cb17fa700 1 -- 172.21.15.31:0/1011916332 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cbc15b070 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.806+0000 7f2ca6ffd700 1 -- 172.21.15.31:0/1011916332 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cbc15b070 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.806+0000 7f2cb17fa700 1 -- 172.21.15.31:0/1011916332 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f2cbc15b070 con 0x7f2cbc15d2d0 2023-10-06T05:18:03.849 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:03.850 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/746584225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":746584225},{"type":"v1","addr":"172.21.15.62:6834","nonce":746584225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/584374139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":584374139},{"type":"v1","addr":"172.21.15.62:6835","nonce":584374139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4926,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3794168057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3794168057},{"type":"v1","addr":"172.21.15.31:6836","nonce":3794168057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:00.090015+0000","modified":"2023-10-06T05:18:03.257587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/3886723511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3886723511},{"type":"v1","addr":"172.21.15.31:6837","nonce":3886723511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:18:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 -- 172.21.15.31:0/1011916332 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ca0066f30 msgr2=0x7f2ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ca0066f30 0x7f2ca00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8003a80 tx=0x7f2cb8007ec0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 -- 172.21.15.31:0/1011916332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 msgr2=0x7f2cbc158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 0x7f2cbc158ae0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400b6c0 tx=0x7f2cb4018000 comp rx=0 tx=0).stop 2023-10-06T05:18:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 -- 172.21.15.31:0/1011916332 shutdown_connections 2023-10-06T05:18:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ca0066f30 0x7f2ca00693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cbc15d2d0 0x7f2cbc158ae0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cbc0cd510 0x7f2cbc0b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 --2- 172.21.15.31:0/1011916332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc159500 0x7f2cbc0c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 -- 172.21.15.31:0/1011916332 >> 172.21.15.31:0/1011916332 conn(0x7f2cbc06ec90 msgr2=0x7f2cbc0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 -- 172.21.15.31:0/1011916332 shutdown_connections 2023-10-06T05:18:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:03.809+0000 7f2ca6ffd700 1 -- 172.21.15.31:0/1011916332 wait complete. 2023-10-06T05:18:03.853 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 16 2023-10-06T05:18:03.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:04.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.046+0000 7fc87f1ff700 1 Processor -- start 2023-10-06T05:18:04.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.046+0000 7fc87f1ff700 1 -- start start 2023-10-06T05:18:04.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.048+0000 7fc87f1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc878068d80 0x7fc878159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.048+0000 7fc87f1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc8780cc0d0 0x7fc8780c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.048+0000 7fc87f1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8780c95b0 con 0x7fc8781595a0 2023-10-06T05:18:04.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.048+0000 7fc87f1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8780c9890 con 0x7fc878068d80 2023-10-06T05:18:04.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.048+0000 7fc87f1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8780c9b70 con 0x7fc8780cc0d0 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.049+0000 7fc877fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc878068d80 0x7fc878159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.049+0000 7fc877fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc878068d80 0x7fc878159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59654/0 (socket says 172.21.15.31:59654) 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.049+0000 7fc877fff700 1 -- 172.21.15.31:0/2468551811 learned_addr learned my addr 172.21.15.31:0/2468551811 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.049+0000 7fc87d79c700 1 --2- 172.21.15.31:0/2468551811 >> v2:172.21.15.62:3300/0 conn(0x7fc8780cc0d0 0x7fc8780c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.049+0000 7fc877fff700 1 -- 172.21.15.31:0/2468551811 >> v2:172.21.15.62:3300/0 conn(0x7fc8780cc0d0 msgr2=0x7fc8780c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.049+0000 7fc877fff700 1 --2- 172.21.15.31:0/2468551811 >> v2:172.21.15.62:3300/0 conn(0x7fc8780cc0d0 0x7fc8780c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.049+0000 7fc877fff700 1 -- 172.21.15.31:0/2468551811 >> v1:172.21.15.62:6789/0 conn(0x7fc8781595a0 legacy=0x7fc878068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.049+0000 7fc877fff700 1 -- 172.21.15.31:0/2468551811 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780ca130 con 0x7fc878068d80 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.049+0000 7fc877fff700 1 --2- 172.21.15.31:0/2468551811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc878068d80 0x7fc878159970 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8640099b0 tx=0x7fc864009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f26ee27cd20a75a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc8777fe700 1 -- 172.21.15.31:0/2468551811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400b820 con 0x7fc878068d80 2023-10-06T05:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc8777fe700 1 -- 172.21.15.31:0/2468551811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86400b980 con 0x7fc878068d80 2023-10-06T05:18:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc8777fe700 1 -- 172.21.15.31:0/2468551811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400bd20 con 0x7fc878068d80 2023-10-06T05:18:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2468551811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc878068d80 msgr2=0x7fc878159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc87f1ff700 1 --2- 172.21.15.31:0/2468551811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc878068d80 0x7fc878159970 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8640099b0 tx=0x7fc864009250 comp rx=0 tx=0).stop 2023-10-06T05:18:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2468551811 shutdown_connections 2023-10-06T05:18:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc87f1ff700 1 --2- 172.21.15.31:0/2468551811 >> v2:172.21.15.62:3300/0 conn(0x7fc8780cc0d0 0x7fc8780c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc87f1ff700 1 --2- 172.21.15.31:0/2468551811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc878068d80 0x7fc878159970 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2468551811 >> 172.21.15.31:0/2468551811 conn(0x7fc87806b620 msgr2=0x7fc87806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2468551811 shutdown_connections 2023-10-06T05:18:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.050+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2468551811 wait complete. 2023-10-06T05:18:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.052+0000 7fc87f1ff700 1 Processor -- start 2023-10-06T05:18:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.052+0000 7fc87f1ff700 1 -- start start 2023-10-06T05:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc87f1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc878068d80 0x7fc87815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc87f1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8780cc0d0 0x7fc87815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc87f1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8781595a0 0x7fc87816d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc87f1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87815bb60 con 0x7fc8781595a0 2023-10-06T05:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc87f1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87815e1c0 con 0x7fc8780cc0d0 2023-10-06T05:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc87f1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87815e4a0 con 0x7fc878068d80 2023-10-06T05:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc87d79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8781595a0 0x7fc87816d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc87d79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8781595a0 0x7fc87816d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59668/0 (socket says 172.21.15.31:59668) 2023-10-06T05:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc87d79c700 1 -- 172.21.15.31:0/2135838115 learned_addr learned my addr 172.21.15.31:0/2135838115 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc87cf9b700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc878068d80 0x7fc87815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.053+0000 7fc877fff700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8780cc0d0 0x7fc87815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc87d79c700 1 -- 172.21.15.31:0/2135838115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc878068d80 msgr2=0x7fc87815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc87d79c700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc878068d80 0x7fc87815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc87d79c700 1 -- 172.21.15.31:0/2135838115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8780cc0d0 msgr2=0x7fc87815b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc87d79c700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8780cc0d0 0x7fc87815b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc87d79c700 1 -- 172.21.15.31:0/2135838115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640094f0 con 0x7fc8781595a0 2023-10-06T05:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc87d79c700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8781595a0 0x7fc87816d090 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc870009fa0 tx=0x7fc870009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc874ff9700 1 -- 172.21.15.31:0/2135838115 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000ebc0 con 0x7fc8781595a0 2023-10-06T05:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc874ff9700 1 -- 172.21.15.31:0/2135838115 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000ed20 con 0x7fc8781595a0 2023-10-06T05:18:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc874ff9700 1 -- 172.21.15.31:0/2135838115 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870014670 con 0x7fc8781595a0 2023-10-06T05:18:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2135838115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87815e780 con 0x7fc8781595a0 2023-10-06T05:18:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.054+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2135838115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87815ed40 con 0x7fc8781595a0 2023-10-06T05:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.055+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2135838115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87815ef60 con 0x7fc8781595a0 2023-10-06T05:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.056+0000 7fc874ff9700 1 -- 172.21.15.31:0/2135838115 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc8700149e0 con 0x7fc8781595a0 2023-10-06T05:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.057+0000 7fc874ff9700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc85c066f30 0x7fc85c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.057+0000 7fc874ff9700 1 -- 172.21.15.31:0/2135838115 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc870084d90 con 0x7fc8781595a0 2023-10-06T05:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.057+0000 7fc87cf9b700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc85c066f30 0x7fc85c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.058+0000 7fc87cf9b700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc85c066f30 0x7fc85c0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc86c003a80 tx=0x7fc86c007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:04.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.061+0000 7fc874ff9700 1 -- 172.21.15.31:0/2135838115 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc870051e00 con 0x7fc8781595a0 2023-10-06T05:18:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.266+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2135838115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc87804c380 con 0x7fc8781595a0 2023-10-06T05:18:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.267+0000 7fc874ff9700 1 -- 172.21.15.31:0/2135838115 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fc870055420 con 0x7fc8781595a0 2023-10-06T05:18:04.319 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:04.320 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:01.245088+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:58.074919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:58.854233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2135838115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc85c066f30 msgr2=0x7fc85c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc85c066f30 0x7fc85c0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc86c003a80 tx=0x7fc86c007ec0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2135838115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8781595a0 msgr2=0x7fc87816d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8781595a0 0x7fc87816d090 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc870009fa0 tx=0x7fc870009900 comp rx=0 tx=0).stop 2023-10-06T05:18:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2135838115 shutdown_connections 2023-10-06T05:18:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc85c066f30 0x7fc85c0693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8781595a0 0x7fc87816d090 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc878068d80 0x7fc87815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 --2- 172.21.15.31:0/2135838115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8780cc0d0 0x7fc87815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2135838115 >> 172.21.15.31:0/2135838115 conn(0x7fc87806b620 msgr2=0x7fc878153740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2135838115 shutdown_connections 2023-10-06T05:18:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.270+0000 7fc87f1ff700 1 -- 172.21.15.31:0/2135838115 wait complete. 2023-10-06T05:18:04.323 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:18:04.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.517+0000 7efef3fff700 1 Processor -- start 2023-10-06T05:18:04.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.517+0000 7efef3fff700 1 -- start start 2023-10-06T05:18:04.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.518+0000 7efef3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efeec0c7b60 0x7efeec069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.518+0000 7efef3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efeec0c7f30 0x7efeec0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef3fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeec0ca5b0 con 0x7efeec06ef50 2023-10-06T05:18:04.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef3fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec06f740 con 0x7efeec0c7b60 2023-10-06T05:18:04.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef3fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec0c9c80 con 0x7efeec0c7f30 2023-10-06T05:18:04.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef1d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efeec0c7b60 0x7efeec069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efeec06ef50 0x7efeec13f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44400/0 (socket says 172.21.15.31:44400) 2023-10-06T05:18:04.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef259c700 1 -- 172.21.15.31:0/1843697557 learned_addr learned my addr 172.21.15.31:0/1843697557 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:04.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef159a700 1 --2- 172.21.15.31:0/1843697557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efeec0c7f30 0x7efeec0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef1d9b700 1 -- 172.21.15.31:0/1843697557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efeec0c7f30 msgr2=0x7efeec0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:04.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef1d9b700 1 --2- 172.21.15.31:0/1843697557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efeec0c7f30 0x7efeec0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef1d9b700 1 -- 172.21.15.31:0/1843697557 >> v1:172.21.15.31:6789/0 conn(0x7efeec06ef50 legacy=0x7efeec13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef1d9b700 1 -- 172.21.15.31:0/1843697557 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec0c9f60 con 0x7efeec0c7b60 2023-10-06T05:18:04.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.519+0000 7efef0d99700 1 -- 172.21.15.31:0/1843697557 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756451326 0 0) 0x7efeec0ca5b0 con 0x7efeec06ef50 2023-10-06T05:18:04.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.520+0000 7efef1d9b700 1 --2- 172.21.15.31:0/1843697557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec0c7b60 0x7efeec069880 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efee80098d0 tx=0x7efee8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178b9817bd2e98be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.520+0000 7efef0d99700 1 -- 172.21.15.31:0/1843697557 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8005c10 con 0x7efeec0c7b60 2023-10-06T05:18:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.520+0000 7efef0d99700 1 -- 172.21.15.31:0/1843697557 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee8005d70 con 0x7efeec0c7b60 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.520+0000 7efef0d99700 1 -- 172.21.15.31:0/1843697557 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8006280 con 0x7efeec0c7b60 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.520+0000 7efef3fff700 1 -- 172.21.15.31:0/1843697557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec0c7b60 msgr2=0x7efeec069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.520+0000 7efef3fff700 1 --2- 172.21.15.31:0/1843697557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec0c7b60 0x7efeec069880 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efee80098d0 tx=0x7efee8009120 comp rx=0 tx=0).stop 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.521+0000 7efef3fff700 1 -- 172.21.15.31:0/1843697557 shutdown_connections 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.521+0000 7efef3fff700 1 --2- 172.21.15.31:0/1843697557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec0c7b60 0x7efeec069880 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.521+0000 7efef3fff700 1 --2- 172.21.15.31:0/1843697557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efeec0c7f30 0x7efeec0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.521+0000 7efef3fff700 1 -- 172.21.15.31:0/1843697557 >> 172.21.15.31:0/1843697557 conn(0x7efeec0cc0b0 msgr2=0x7efeec0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.521+0000 7efef3fff700 1 -- 172.21.15.31:0/1843697557 shutdown_connections 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.521+0000 7efef3fff700 1 -- 172.21.15.31:0/1843697557 wait complete. 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.522+0000 7efef3fff700 1 Processor -- start 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.523+0000 7efef3fff700 1 -- start start 2023-10-06T05:18:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.523+0000 7efef3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec06ef50 0x7efeec0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec0c7b60 0x7efeec0a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efeec0c7f30 0x7efeec16a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef3fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec169ca0 con 0x7efeec06ef50 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef3fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec169f10 con 0x7efeec0c7b60 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef3fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec0abae0 con 0x7efeec0c7f30 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec06ef50 0x7efeec0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec06ef50 0x7efeec0ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59690/0 (socket says 172.21.15.31:59690) 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef1d9b700 1 -- 172.21.15.31:0/543529846 learned_addr learned my addr 172.21.15.31:0/543529846 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef159a700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec0c7b60 0x7efeec0a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef259c700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efeec0c7f30 0x7efeec16a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef1d9b700 1 -- 172.21.15.31:0/543529846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efeec0c7f30 msgr2=0x7efeec16a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef1d9b700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efeec0c7f30 0x7efeec16a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef1d9b700 1 -- 172.21.15.31:0/543529846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec0c7b60 msgr2=0x7efeec0a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef1d9b700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec0c7b60 0x7efeec0a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef1d9b700 1 -- 172.21.15.31:0/543529846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee80093c0 con 0x7efeec06ef50 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.524+0000 7efef1d9b700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec06ef50 0x7efeec0ae200 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efee8009f70 tx=0x7efee8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.525+0000 7efee1ffb700 1 -- 172.21.15.31:0/543529846 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee801d880 con 0x7efeec06ef50 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.525+0000 7efee1ffb700 1 -- 172.21.15.31:0/543529846 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee801da00 con 0x7efeec06ef50 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.525+0000 7efee1ffb700 1 -- 172.21.15.31:0/543529846 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee801ddc0 con 0x7efeec06ef50 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.525+0000 7efef3fff700 1 -- 172.21.15.31:0/543529846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec0abdc0 con 0x7efeec06ef50 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.525+0000 7efef3fff700 1 -- 172.21.15.31:0/543529846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec0ac380 con 0x7efeec06ef50 2023-10-06T05:18:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.526+0000 7efef3fff700 1 -- 172.21.15.31:0/543529846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeec04c380 con 0x7efeec06ef50 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.527+0000 7efee1ffb700 1 -- 172.21.15.31:0/543529846 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efee801b690 con 0x7efeec06ef50 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.527+0000 7efee1ffb700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efed0066f30 0x7efed00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.528+0000 7efee1ffb700 1 -- 172.21.15.31:0/543529846 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efee8086460 con 0x7efeec06ef50 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.528+0000 7efef159a700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efed0066f30 0x7efed00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.528+0000 7efef159a700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efed0066f30 0x7efed00693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efeec139bb0 tx=0x7efedc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.532+0000 7efee1ffb700 1 -- 172.21.15.31:0/543529846 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efee80534d0 con 0x7efeec06ef50 2023-10-06T05:18:04.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.732+0000 7efef3fff700 1 -- 172.21.15.31:0/543529846 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efeec0b1a30 con 0x7efed0066f30 2023-10-06T05:18:04.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.742+0000 7efee1ffb700 1 -- 172.21.15.31:0/543529846 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300553 (secure 0 0 0) 0x7efeec0b1a30 con 0x7efed0066f30 2023-10-06T05:18:04.773 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 -- 172.21.15.31:0/543529846 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efed0066f30 msgr2=0x7efed00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efed0066f30 0x7efed00693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efeec139bb0 tx=0x7efedc008040 comp rx=0 tx=0).stop 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 -- 172.21.15.31:0/543529846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec06ef50 msgr2=0x7efeec0ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec06ef50 0x7efeec0ae200 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efee8009f70 tx=0x7efee8016000 comp rx=0 tx=0).stop 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 -- 172.21.15.31:0/543529846 shutdown_connections 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efeec0c7f30 0x7efeec16a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efed0066f30 0x7efed00693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeec06ef50 0x7efeec0ae200 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 --2- 172.21.15.31:0/543529846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec0c7b60 0x7efeec0a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 -- 172.21.15.31:0/543529846 >> 172.21.15.31:0/543529846 conn(0x7efeec0cc0b0 msgr2=0x7efeec13e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 -- 172.21.15.31:0/543529846 shutdown_connections 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:04.747+0000 7efef3fff700 1 -- 172.21.15.31:0/543529846 wait complete. 2023-10-06T05:18:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:18:04.777 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-06T05:18:04.073783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381992,"kb_used_data":3688,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592728,"statfs":{"total":773094113280,"available":770654953472,"internally_reserved":0,"allocated":3776512,"data_stored":2199312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.853925"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226925+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:52.601888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.224836+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.224480+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:43.961009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.222379+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:05.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.225299+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:42.051694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.228244+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:55.094026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.224345+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:41.223117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.222130+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:59.265675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.221911+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:14.113766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.225317+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:55.352926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.225580+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:30.375958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.221916+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.667357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.221970+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226297+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:55.352926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226195+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:40.606016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.227287+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:56.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226172+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:18.659937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.222065+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.233699+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:08.242881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226250+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:42.051694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.232856+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:20.768693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.232989+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:28.452676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.227666+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:04.636437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.232795+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:22.033143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.222142+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:23.488521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.219461+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:17:52.219461+0000","last_peered":"2023-10-06T05:17:52.219461+0000","last_clean":"2023-10-06T05:17:52.219461+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:17:52.219461+0000","last_undegraded":"2023-10-06T05:17:52.219461+0000","last_fullsized":"2023-10-06T05:17:52.219461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.221916+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:23.102068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.219141+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:17:52.219141+0000","last_peered":"2023-10-06T05:17:52.219141+0000","last_clean":"2023-10-06T05:17:52.219141+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:17:52.219141+0000","last_undegraded":"2023-10-06T05:17:52.219141+0000","last_fullsized":"2023-10-06T05:17:52.219141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.225484+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:43.680592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.218901+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:17:52.218901+0000","last_peered":"2023-10-06T05:17:52.218901+0000","last_clean":"2023-10-06T05:17:52.218901+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:17:52.218901+0000","last_undegraded":"2023-10-06T05:17:52.218901+0000","last_fullsized":"2023-10-06T05:17:52.218901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":169,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.600704+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:17:52.600704+0000","last_peered":"2023-10-06T05:17:52.600704+0000","last_clean":"2023-10-06T05:17:52.600704+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:17:52.600704+0000","last_undegraded":"2023-10-06T05:17:52.600704+0000","last_fullsized":"2023-10-06T05:17:52.600704+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.232865+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:07.830657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.600670+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:17:52.600670+0000","last_peered":"2023-10-06T05:17:52.600670+0000","last_clean":"2023-10-06T05:17:52.600670+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:17:52.600670+0000","last_undegraded":"2023-10-06T05:17:52.600670+0000","last_fullsized":"2023-10-06T05:17:52.600670+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226934+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:18.659937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.600663+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:17:52.600663+0000","last_peered":"2023-10-06T05:17:52.600663+0000","last_clean":"2023-10-06T05:17:52.600663+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:17:52.600663+0000","last_undegraded":"2023-10-06T05:17:52.600663+0000","last_fullsized":"2023-10-06T05:17:52.600663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.224749+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:23.102068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.600906+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:17:52.600906+0000","last_peered":"2023-10-06T05:17:52.600906+0000","last_clean":"2023-10-06T05:17:52.600906+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:17:52.600906+0000","last_undegraded":"2023-10-06T05:17:52.600906+0000","last_fullsized":"2023-10-06T05:17:52.600906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226355+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:39.271735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.532195+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:17:52.532195+0000","last_peered":"2023-10-06T05:17:52.532195+0000","last_clean":"2023-10-06T05:17:52.532195+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:17:52.532195+0000","last_undegraded":"2023-10-06T05:17:52.532195+0000","last_fullsized":"2023-10-06T05:17:52.532195+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.224769+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.667357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.532240+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:17:52.532240+0000","last_peered":"2023-10-06T05:17:52.532240+0000","last_clean":"2023-10-06T05:17:52.532240+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:17:52.532240+0000","last_undegraded":"2023-10-06T05:17:52.532240+0000","last_fullsized":"2023-10-06T05:17:52.532240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298184,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073656,"statfs":{"total":96636764160,"available":96331423744,"internally_reserved":0,"allocated":917504,"data_stored":717690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.755}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.88200000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.86399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.93000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.76700000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.752},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.79500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298184,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073656,"statfs":{"total":96636764160,"available":96331423744,"internally_reserved":0,"allocated":917504,"data_stored":717690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.373}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.93300000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.86799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.5},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.872}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45000000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.79000000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.876}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.374},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.435}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.495}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.751},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.82199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.78500000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.629},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.71199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.55200000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":7.742,"5min":7.742,"15min":7.742},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.76300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.90500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2780000000000005,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":9.25,"5min":9.25,"15min":9.25},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.108000000000001,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":10.384,"5min":10.384,"15min":10.384},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":9.1699999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.205,"5min":10.205,"15min":10.205},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.62}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.83399999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.78900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:05.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.330+0000 7f5fbb7ff700 1 Processor -- start 2023-10-06T05:18:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.330+0000 7f5fbb7ff700 1 -- start start 2023-10-06T05:18:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.332+0000 7f5fbb7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5fb406a570 0x7f5fb4160560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.332+0000 7f5fbb7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb406e840 con 0x7f5fb4155390 2023-10-06T05:18:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.332+0000 7f5fbb7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb406dc30 con 0x7f5fb4155760 2023-10-06T05:18:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.332+0000 7f5fbb7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb406df10 con 0x7f5fb406a570 2023-10-06T05:18:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.332+0000 7f5fb9d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5fb406a570 0x7f5fb4160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.332+0000 7f5fb9d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5fb406a570 0x7f5fb4160560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59696/0 (socket says 172.21.15.31:59696) 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.332+0000 7f5fb959b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5fb4155390 0x7f5fb40c7170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44406/0 (socket says 172.21.15.31:44406) 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.332+0000 7f5fb9d9c700 1 -- 172.21.15.31:0/2035068706 learned_addr learned my addr 172.21.15.31:0/2035068706 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.332+0000 7f5fb9d9c700 1 -- 172.21.15.31:0/2035068706 >> v1:172.21.15.62:6789/0 conn(0x7f5fb4155760 legacy=0x7f5fb4155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.333+0000 7f5fb9d9c700 1 -- 172.21.15.31:0/2035068706 >> v1:172.21.15.31:6789/0 conn(0x7f5fb4155390 legacy=0x7f5fb40c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.333+0000 7f5fb9d9c700 1 -- 172.21.15.31:0/2035068706 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb406e1f0 con 0x7f5fb406a570 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.333+0000 7f5fabfff700 1 -- 172.21.15.31:0/2035068706 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913827733 0 0) 0x7f5fb406e840 con 0x7f5fb4155390 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.333+0000 7f5fb9d9c700 1 --2- 172.21.15.31:0/2035068706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb406a570 0x7f5fb4160560 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00d1a0 tx=0x7f5fac00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24606b425ac26fb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.333+0000 7f5fabfff700 1 -- 172.21.15.31:0/2035068706 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac010070 con 0x7f5fb406a570 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.333+0000 7f5fabfff700 1 -- 172.21.15.31:0/2035068706 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac00dcb0 con 0x7f5fb406a570 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.333+0000 7f5fabfff700 1 -- 172.21.15.31:0/2035068706 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac016630 con 0x7f5fb406a570 2023-10-06T05:18:05.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.334+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/2035068706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb406a570 msgr2=0x7f5fb4160560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.334+0000 7f5fbb7ff700 1 --2- 172.21.15.31:0/2035068706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb406a570 0x7f5fb4160560 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00d1a0 tx=0x7f5fac00ba50 comp rx=0 tx=0).stop 2023-10-06T05:18:05.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.334+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/2035068706 shutdown_connections 2023-10-06T05:18:05.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.334+0000 7f5fbb7ff700 1 --2- 172.21.15.31:0/2035068706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb406a570 0x7f5fb4160560 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.334+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/2035068706 >> 172.21.15.31:0/2035068706 conn(0x7f5fb40cc0b0 msgr2=0x7f5fb40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:05.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.334+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/2035068706 shutdown_connections 2023-10-06T05:18:05.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.334+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/2035068706 wait complete. 2023-10-06T05:18:05.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.335+0000 7f5fbb7ff700 1 Processor -- start 2023-10-06T05:18:05.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.335+0000 7f5fbb7ff700 1 -- start start 2023-10-06T05:18:05.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.336+0000 7f5fbb7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb406a570 0x7f5fb4151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.336+0000 7f5fbb7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fb4155390 0x7f5fb40c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.336+0000 7f5fbb7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb4155760 0x7f5fb4152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.336+0000 7f5fbb7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb41496a0 con 0x7f5fb4155760 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.336+0000 7f5fbb7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb41520b0 con 0x7f5fb406a570 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.336+0000 7f5fbb7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40aa270 con 0x7f5fb4155390 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb9d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb4155760 0x7f5fb4152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb9d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb4155760 0x7f5fb4152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59704/0 (socket says 172.21.15.31:59704) 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb9d9c700 1 -- 172.21.15.31:0/3811673326 learned_addr learned my addr 172.21.15.31:0/3811673326 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb959b700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb406a570 0x7f5fb4151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb8d9a700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fb4155390 0x7f5fb40c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb9d9c700 1 -- 172.21.15.31:0/3811673326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fb4155390 msgr2=0x7f5fb40c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb9d9c700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fb4155390 0x7f5fb40c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb9d9c700 1 -- 172.21.15.31:0/3811673326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb406a570 msgr2=0x7f5fb4151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb9d9c700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb406a570 0x7f5fb4151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb9d9c700 1 -- 172.21.15.31:0/3811673326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac00bcf0 con 0x7f5fb4155760 2023-10-06T05:18:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.337+0000 7f5fb9d9c700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb4155760 0x7f5fb4152e30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5fac016000 tx=0x7f5fac01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.338+0000 7f5fa97fa700 1 -- 172.21.15.31:0/3811673326 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac008410 con 0x7f5fb4155760 2023-10-06T05:18:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.338+0000 7f5fa97fa700 1 -- 172.21.15.31:0/3811673326 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac0085b0 con 0x7f5fb4155760 2023-10-06T05:18:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.338+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/3811673326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb416df20 con 0x7f5fb4155760 2023-10-06T05:18:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.338+0000 7f5fa97fa700 1 -- 172.21.15.31:0/3811673326 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac008970 con 0x7f5fb4155760 2023-10-06T05:18:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.338+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/3811673326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb40a9940 con 0x7f5fb4155760 2023-10-06T05:18:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.339+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/3811673326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb404c380 con 0x7f5fb4155760 2023-10-06T05:18:05.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.340+0000 7f5fa97fa700 1 -- 172.21.15.31:0/3811673326 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5fac01f3f0 con 0x7f5fb4155760 2023-10-06T05:18:05.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.340+0000 7f5fa97fa700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5f98066f30 0x7f5f980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:05.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.341+0000 7f5fa97fa700 1 -- 172.21.15.31:0/3811673326 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5fac086690 con 0x7f5fb4155760 2023-10-06T05:18:05.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.341+0000 7f5fb959b700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5f98066f30 0x7f5f980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:05.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.341+0000 7f5fb959b700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5f98066f30 0x7f5f980693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00012a0 tx=0x7f5fb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:05.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.345+0000 7f5fa97fa700 1 -- 172.21.15.31:0/3811673326 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fac053700 con 0x7f5fb4155760 2023-10-06T05:18:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.542+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/3811673326 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5fb40c9f90 con 0x7f5f98066f30 2023-10-06T05:18:05.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.551+0000 7f5fa97fa700 1 -- 172.21.15.31:0/3811673326 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300553 (secure 0 0 0) 0x7f5fb40c9f90 con 0x7f5f98066f30 2023-10-06T05:18:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:05.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.555+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/3811673326 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5f98066f30 msgr2=0x7f5f980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.555+0000 7f5fbb7ff700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5f98066f30 0x7f5f980693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00012a0 tx=0x7f5fb0009040 comp rx=0 tx=0).stop 2023-10-06T05:18:05.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.556+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/3811673326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb4155760 msgr2=0x7f5fb4152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:05.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.556+0000 7f5fbb7ff700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb4155760 0x7f5fb4152e30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5fac016000 tx=0x7f5fac01b000 comp rx=0 tx=0).stop 2023-10-06T05:18:05.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.556+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/3811673326 shutdown_connections 2023-10-06T05:18:05.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.556+0000 7f5fbb7ff700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5f98066f30 0x7f5f980693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.556+0000 7f5fbb7ff700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb4155760 0x7f5fb4152e30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.556+0000 7f5fbb7ff700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fb406a570 0x7f5fb4151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.556+0000 7f5fbb7ff700 1 --2- 172.21.15.31:0/3811673326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5fb4155390 0x7f5fb40c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:05.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.556+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/3811673326 >> 172.21.15.31:0/3811673326 conn(0x7f5fb40cc0b0 msgr2=0x7f5fb4156d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:05.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.556+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/3811673326 shutdown_connections 2023-10-06T05:18:05.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:05.556+0000 7f5fbb7ff700 1 -- 172.21.15.31:0/3811673326 wait complete. 2023-10-06T05:18:05.586 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:18:05.586 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-06T05:18:04.073783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381992,"kb_used_data":3688,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592728,"statfs":{"total":773094113280,"available":770654953472,"internally_reserved":0,"allocated":3776512,"data_stored":2199312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.853925"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226925+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:52.601888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.224836+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.224480+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:43.961009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.222379+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:05.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.225299+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:42.051694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.228244+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:55.094026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.224345+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:41.223117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.222130+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:59.265675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.221911+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:14.113766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.225317+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:55.352926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.225580+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:30.375958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.221916+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.667357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.221970+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226297+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:55.352926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226195+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:40.606016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.227287+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:56.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226172+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:18.659937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.222065+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.613898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.233699+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:08.242881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226250+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:42.051694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.232856+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:20.768693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.232989+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:28.452676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.227666+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:04.636437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.232795+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:22.033143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.222142+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:23.488521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.219461+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:17:52.219461+0000","last_peered":"2023-10-06T05:17:52.219461+0000","last_clean":"2023-10-06T05:17:52.219461+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:17:52.219461+0000","last_undegraded":"2023-10-06T05:17:52.219461+0000","last_fullsized":"2023-10-06T05:17:52.219461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.221916+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:23.102068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.219141+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:17:52.219141+0000","last_peered":"2023-10-06T05:17:52.219141+0000","last_clean":"2023-10-06T05:17:52.219141+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:17:52.219141+0000","last_undegraded":"2023-10-06T05:17:52.219141+0000","last_fullsized":"2023-10-06T05:17:52.219141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.225484+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:43.680592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.218901+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:17:52.218901+0000","last_peered":"2023-10-06T05:17:52.218901+0000","last_clean":"2023-10-06T05:17:52.218901+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:17:52.218901+0000","last_undegraded":"2023-10-06T05:17:52.218901+0000","last_fullsized":"2023-10-06T05:17:52.218901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":169,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.600704+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:17:52.600704+0000","last_peered":"2023-10-06T05:17:52.600704+0000","last_clean":"2023-10-06T05:17:52.600704+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:17:52.600704+0000","last_undegraded":"2023-10-06T05:17:52.600704+0000","last_fullsized":"2023-10-06T05:17:52.600704+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.232865+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:07.830657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.600670+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:17:52.600670+0000","last_peered":"2023-10-06T05:17:52.600670+0000","last_clean":"2023-10-06T05:17:52.600670+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:17:52.600670+0000","last_undegraded":"2023-10-06T05:17:52.600670+0000","last_fullsized":"2023-10-06T05:17:52.600670+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226934+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:18.659937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.600663+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:17:52.600663+0000","last_peered":"2023-10-06T05:17:52.600663+0000","last_clean":"2023-10-06T05:17:52.600663+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:17:52.600663+0000","last_undegraded":"2023-10-06T05:17:52.600663+0000","last_fullsized":"2023-10-06T05:17:52.600663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.224749+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:23.102068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.600906+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:17:52.600906+0000","last_peered":"2023-10-06T05:17:52.600906+0000","last_clean":"2023-10-06T05:17:52.600906+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:17:52.600906+0000","last_undegraded":"2023-10-06T05:17:52.600906+0000","last_fullsized":"2023-10-06T05:17:52.600906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.226355+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:39.271735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.532195+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:17:52.532195+0000","last_peered":"2023-10-06T05:17:52.532195+0000","last_clean":"2023-10-06T05:17:52.532195+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:17:52.532195+0000","last_undegraded":"2023-10-06T05:17:52.532195+0000","last_fullsized":"2023-10-06T05:17:52.532195+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:17:59.218571+0000","last_change":"2023-10-06T05:17:59.218571+0000","last_active":"2023-10-06T05:17:59.218571+0000","last_peered":"2023-10-06T05:17:59.218571+0000","last_clean":"2023-10-06T05:17:59.218571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:59.218571+0000","last_undegraded":"2023-10-06T05:17:59.218571+0000","last_fullsized":"2023-10-06T05:17:59.218571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:17:58.211999+0000","last_change":"2023-10-06T05:17:58.224769+0000","last_active":"2023-10-06T05:17:58.211999+0000","last_peered":"2023-10-06T05:17:58.211999+0000","last_clean":"2023-10-06T05:17:58.211999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:17:58.211999+0000","last_undegraded":"2023-10-06T05:17:58.211999+0000","last_fullsized":"2023-10-06T05:17:58.211999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.667357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T05:17:52.532240+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:17:52.532240+0000","last_peered":"2023-10-06T05:17:52.532240+0000","last_clean":"2023-10-06T05:17:52.532240+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:17:52.532240+0000","last_undegraded":"2023-10-06T05:17:52.532240+0000","last_fullsized":"2023-10-06T05:17:52.532240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298184,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073656,"statfs":{"total":96636764160,"available":96331423744,"internally_reserved":0,"allocated":917504,"data_stored":717690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.755}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.88200000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.86399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.93000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.76700000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.752},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.79500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298184,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073656,"statfs":{"total":96636764160,"available":96331423744,"internally_reserved":0,"allocated":917504,"data_stored":717690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.373}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.93300000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.86799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.5},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.872}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45000000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.79000000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.876}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.374},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.435}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.495}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.751},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.82199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.78500000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.629},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.71199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.55200000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":7.742,"5min":7.742,"15min":7.742},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.76300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.90500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":127322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2780000000000005,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":9.25,"5min":9.25,"15min":9.25},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.108000000000001,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":10.384,"5min":10.384,"15min":10.384},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":9.1699999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.205,"5min":10.205,"15min":10.205},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.62}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.83399999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.78900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:05.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.199+0000 7f6dff3ff700 1 Processor -- start 2023-10-06T05:18:06.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.199+0000 7f6dff3ff700 1 -- start start 2023-10-06T05:18:06.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.200+0000 7f6dff3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df8159500 0x7f6df80b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.201+0000 7f6dff3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df80c3f40 con 0x7f6df815d2d0 2023-10-06T05:18:06.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.201+0000 7f6dff3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df80cb000 con 0x7f6df80cd510 2023-10-06T05:18:06.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.201+0000 7f6dff3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80b2860 con 0x7f6df8159500 2023-10-06T05:18:06.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.201+0000 7f6dfc99a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6df815d2d0 0x7f6df80cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44414/0 (socket says 172.21.15.31:44414) 2023-10-06T05:18:06.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.201+0000 7f6dfc99a700 1 -- 172.21.15.31:0/2699945475 learned_addr learned my addr 172.21.15.31:0/2699945475 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:06.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.201+0000 7f6dfd19b700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df8159500 0x7f6df80b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.201+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123718736 0 0) 0x7f6df80c3f40 con 0x7f6df815d2d0 2023-10-06T05:18:06.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.201+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6de4003570 con 0x7f6df815d2d0 2023-10-06T05:18:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111039164 0 0) 0x7f6df80cb000 con 0x7f6df80cd510 2023-10-06T05:18:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6df80c3f40 con 0x7f6df80cd510 2023-10-06T05:18:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 472935142 0 0) 0x7f6de4003570 con 0x7f6df815d2d0 2023-10-06T05:18:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df8159500 msgr2=0x7f6df80b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df8159500 0x7f6df80b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 >> v1:172.21.15.62:6789/0 conn(0x7f6df80cd510 legacy=0x7f6df80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df815d7d0 con 0x7f6df815d2d0 2023-10-06T05:18:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f6de80015f0 con 0x7f6df815d2d0 2023-10-06T05:18:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 >> v1:172.21.15.31:6789/0 conn(0x7f6df815d2d0 legacy=0x7f6df80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de4003c20 0x7f6de4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df8159500 0x7f6de4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de4004d50 0x7f6de40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80b2860 con 0x7f6de4004d50 2023-10-06T05:18:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de400bbf0 con 0x7f6de4003c20 2023-10-06T05:18:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de400bd60 con 0x7f6df8159500 2023-10-06T05:18:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6dfd19b700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de4003c20 0x7f6de4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6dfd99c700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de4004d50 0x7f6de40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.202+0000 7f6dfc99a700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df8159500 0x7f6de4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.203+0000 7f6dfd99c700 1 -- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df8159500 msgr2=0x7f6de4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.203+0000 7f6dfd99c700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df8159500 0x7f6de4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.203+0000 7f6dfd99c700 1 -- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de4003c20 msgr2=0x7f6de4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.203+0000 7f6dfd99c700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de4003c20 0x7f6de4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.203+0000 7f6dfd99c700 1 -- 172.21.15.31:0/2699945475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4003570 con 0x7f6de4004d50 2023-10-06T05:18:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.203+0000 7f6dfd19b700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de4003c20 0x7f6de4004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.203+0000 7f6dfd99c700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de4004d50 0x7f6de40051c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6df80b3030 tx=0x7f6df00017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913e286b4aa9121a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:06.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.203+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0001ab0 con 0x7f6de4004d50 2023-10-06T05:18:06.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.203+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df000c880 con 0x7f6de4004d50 2023-10-06T05:18:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.203+0000 7f6deffff700 1 -- 172.21.15.31:0/2699945475 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df000cc40 con 0x7f6de4004d50 2023-10-06T05:18:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.204+0000 7f6dff3ff700 1 -- 172.21.15.31:0/2699945475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de4004d50 msgr2=0x7f6de40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.204+0000 7f6dff3ff700 1 --2- 172.21.15.31:0/2699945475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6de4004d50 0x7f6de40051c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6df80b3030 tx=0x7f6df00017a0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.204+0000 7f6dfd99c700 1 -- 172.21.15.31:0/2699945475 reap_dead start 2023-10-06T05:18:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.204+0000 7f6dff3ff700 1 -- 172.21.15.31:0/2699945475 shutdown_connections 2023-10-06T05:18:06.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.204+0000 7f6dff3ff700 1 -- 172.21.15.31:0/2699945475 >> 172.21.15.31:0/2699945475 conn(0x7f6df806ec90 msgr2=0x7f6df806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.204+0000 7f6dff3ff700 1 -- 172.21.15.31:0/2699945475 shutdown_connections 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.204+0000 7f6dff3ff700 1 -- 172.21.15.31:0/2699945475 wait complete. 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.205+0000 7f6dff3ff700 1 Processor -- start 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.206+0000 7f6dff3ff700 1 -- start start 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dff3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de4004d50 0x7f6df8157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dff3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df816a8f0 0x7f6df815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dff3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df81598d0 0x7f6df8159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dff3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df815a2b0 con 0x7f6df816a8f0 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dff3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8158250 con 0x7f6de4004d50 2023-10-06T05:18:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dff3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df815d060 con 0x7f6df81598d0 2023-10-06T05:18:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dfc99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df816a8f0 0x7f6df815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dfc99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df816a8f0 0x7f6df815b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59716/0 (socket says 172.21.15.31:59716) 2023-10-06T05:18:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dfc99a700 1 -- 172.21.15.31:0/4226081443 learned_addr learned my addr 172.21.15.31:0/4226081443 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dfd19b700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de4004d50 0x7f6df8157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.207+0000 7f6dfd99c700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df81598d0 0x7f6df8159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.208+0000 7f6dfc99a700 1 -- 172.21.15.31:0/4226081443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df81598d0 msgr2=0x7f6df8159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.208+0000 7f6dfc99a700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df81598d0 0x7f6df8159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.208+0000 7f6dfc99a700 1 -- 172.21.15.31:0/4226081443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de4004d50 msgr2=0x7f6df8157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.208+0000 7f6dfc99a700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de4004d50 0x7f6df8157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.208+0000 7f6dfc99a700 1 -- 172.21.15.31:0/4226081443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df815cbd0 con 0x7f6df816a8f0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.208+0000 7f6dfc99a700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df816a8f0 0x7f6df815b7e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6de800f930 tx=0x7f6de8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.208+0000 7f6ded7fa700 1 -- 172.21.15.31:0/4226081443 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de80072e0 con 0x7f6df816a8f0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.208+0000 7f6ded7fa700 1 -- 172.21.15.31:0/4226081443 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de8007460 con 0x7f6df816a8f0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.208+0000 7f6ded7fa700 1 -- 172.21.15.31:0/4226081443 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de80057f0 con 0x7f6df816a8f0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.208+0000 7f6dff3ff700 1 -- 172.21.15.31:0/4226081443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df815bd20 con 0x7f6df816a8f0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.209+0000 7f6dff3ff700 1 -- 172.21.15.31:0/4226081443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df815c2a0 con 0x7f6df816a8f0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.210+0000 7f6dff3ff700 1 -- 172.21.15.31:0/4226081443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df815cd60 con 0x7f6df816a8f0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.210+0000 7f6ded7fa700 1 -- 172.21.15.31:0/4226081443 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6de8005ba0 con 0x7f6df816a8f0 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.211+0000 7f6ded7fa700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ddc066f30 0x7f6ddc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.211+0000 7f6ded7fa700 1 -- 172.21.15.31:0/4226081443 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6de80845f0 con 0x7f6df816a8f0 2023-10-06T05:18:06.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.212+0000 7f6dfd19b700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ddc066f30 0x7f6ddc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:06.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.212+0000 7f6dfd19b700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ddc066f30 0x7f6ddc0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6df80af730 tx=0x7f6df40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:06.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.216+0000 7f6ded7fa700 1 -- 172.21.15.31:0/4226081443 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6de8051660 con 0x7f6df816a8f0 2023-10-06T05:18:06.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.414+0000 7f6dff3ff700 1 -- 172.21.15.31:0/4226081443 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6df804c380 con 0x7f6ddc066f30 2023-10-06T05:18:06.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.424+0000 7f6ded7fa700 1 -- 172.21.15.31:0/4226081443 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311214 (secure 0 0 0) 0x7f6df804c380 con 0x7f6ddc066f30 2023-10-06T05:18:06.479 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.428+0000 7f6dff3ff700 1 -- 172.21.15.31:0/4226081443 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ddc066f30 msgr2=0x7f6ddc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.428+0000 7f6dff3ff700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ddc066f30 0x7f6ddc0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6df80af730 tx=0x7f6df40091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.428+0000 7f6dff3ff700 1 -- 172.21.15.31:0/4226081443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df816a8f0 msgr2=0x7f6df815b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.428+0000 7f6dff3ff700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df816a8f0 0x7f6df815b7e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6de800f930 tx=0x7f6de8009040 comp rx=0 tx=0).stop 2023-10-06T05:18:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.428+0000 7f6dff3ff700 1 -- 172.21.15.31:0/4226081443 shutdown_connections 2023-10-06T05:18:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.428+0000 7f6dff3ff700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6df81598d0 0x7f6df8159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.428+0000 7f6dff3ff700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6de4004d50 0x7f6df8157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.428+0000 7f6dff3ff700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ddc066f30 0x7f6ddc0693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.428+0000 7f6dff3ff700 1 --2- 172.21.15.31:0/4226081443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df816a8f0 0x7f6df815b7e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.428+0000 7f6dff3ff700 1 -- 172.21.15.31:0/4226081443 >> 172.21.15.31:0/4226081443 conn(0x7f6df806ec90 msgr2=0x7f6df80c5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.429+0000 7f6dff3ff700 1 -- 172.21.15.31:0/4226081443 shutdown_connections 2023-10-06T05:18:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:06.429+0000 7f6dff3ff700 1 -- 172.21.15.31:0/4226081443 wait complete. 2023-10-06T05:18:06.510 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:18:06.510 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-10-06T05:18:06.074074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382344,"kb_used_data":4040,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592376,"statfs":{"total":773094113280,"available":770654593024,"internally_reserved":0,"allocated":4136960,"data_stored":2366908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.851052"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.256952+0000","last_change":"2023-10-06T05:18:00.248587+0000","last_active":"2023-10-06T05:18:00.256952+0000","last_peered":"2023-10-06T05:18:00.256952+0000","last_clean":"2023-10-06T05:18:00.256952+0000","last_became_active":"2023-10-06T05:18:00.248347+0000","last_became_peered":"2023-10-06T05:18:00.248347+0000","last_unstale":"2023-10-06T05:18:00.256952+0000","last_undegraded":"2023-10-06T05:18:00.256952+0000","last_fullsized":"2023-10-06T05:18:00.256952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:52.838407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247823+0000","last_change":"2023-10-06T05:18:00.245720+0000","last_active":"2023-10-06T05:18:00.247823+0000","last_peered":"2023-10-06T05:18:00.247823+0000","last_clean":"2023-10-06T05:18:00.247823+0000","last_became_active":"2023-10-06T05:18:00.245480+0000","last_became_peered":"2023-10-06T05:18:00.245480+0000","last_unstale":"2023-10-06T05:18:00.247823+0000","last_undegraded":"2023-10-06T05:18:00.247823+0000","last_fullsized":"2023-10-06T05:18:00.247823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:54.805238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267755+0000","last_change":"2023-10-06T05:18:00.266938+0000","last_active":"2023-10-06T05:18:00.267755+0000","last_peered":"2023-10-06T05:18:00.267755+0000","last_clean":"2023-10-06T05:18:00.267755+0000","last_became_active":"2023-10-06T05:18:00.266629+0000","last_became_peered":"2023-10-06T05:18:00.266629+0000","last_unstale":"2023-10-06T05:18:00.267755+0000","last_undegraded":"2023-10-06T05:18:00.267755+0000","last_fullsized":"2023-10-06T05:18:00.267755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264848+0000","last_change":"2023-10-06T05:18:00.260400+0000","last_active":"2023-10-06T05:18:00.264848+0000","last_peered":"2023-10-06T05:18:00.264848+0000","last_clean":"2023-10-06T05:18:00.264848+0000","last_became_active":"2023-10-06T05:18:00.259981+0000","last_became_peered":"2023-10-06T05:18:00.259981+0000","last_unstale":"2023-10-06T05:18:00.264848+0000","last_undegraded":"2023-10-06T05:18:00.264848+0000","last_fullsized":"2023-10-06T05:18:00.264848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:31.382530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.266572+0000","last_change":"2023-10-06T05:18:00.263393+0000","last_active":"2023-10-06T05:18:00.266572+0000","last_peered":"2023-10-06T05:18:00.266572+0000","last_clean":"2023-10-06T05:18:00.266572+0000","last_became_active":"2023-10-06T05:18:00.263138+0000","last_became_peered":"2023-10-06T05:18:00.263138+0000","last_unstale":"2023-10-06T05:18:00.266572+0000","last_undegraded":"2023-10-06T05:18:00.266572+0000","last_fullsized":"2023-10-06T05:18:00.266572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:56.359498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.251146+0000","last_change":"2023-10-06T05:18:00.245969+0000","last_active":"2023-10-06T05:18:00.251146+0000","last_peered":"2023-10-06T05:18:00.251146+0000","last_clean":"2023-10-06T05:18:00.251146+0000","last_became_active":"2023-10-06T05:18:00.245686+0000","last_became_peered":"2023-10-06T05:18:00.245686+0000","last_unstale":"2023-10-06T05:18:00.251146+0000","last_undegraded":"2023-10-06T05:18:00.251146+0000","last_fullsized":"2023-10-06T05:18:00.251146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:43.620470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276255+0000","last_change":"2023-10-06T05:18:00.269398+0000","last_active":"2023-10-06T05:18:00.276255+0000","last_peered":"2023-10-06T05:18:00.276255+0000","last_clean":"2023-10-06T05:18:00.276255+0000","last_became_active":"2023-10-06T05:18:00.268716+0000","last_became_peered":"2023-10-06T05:18:00.268716+0000","last_unstale":"2023-10-06T05:18:00.276255+0000","last_undegraded":"2023-10-06T05:18:00.276255+0000","last_fullsized":"2023-10-06T05:18:00.276255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268330+0000","last_change":"2023-10-06T05:18:00.259226+0000","last_active":"2023-10-06T05:18:00.268330+0000","last_peered":"2023-10-06T05:18:00.268330+0000","last_clean":"2023-10-06T05:18:00.268330+0000","last_became_active":"2023-10-06T05:18:00.258668+0000","last_became_peered":"2023-10-06T05:18:00.258668+0000","last_unstale":"2023-10-06T05:18:00.268330+0000","last_undegraded":"2023-10-06T05:18:00.268330+0000","last_fullsized":"2023-10-06T05:18:00.268330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270755+0000","last_change":"2023-10-06T05:18:00.269432+0000","last_active":"2023-10-06T05:18:00.270755+0000","last_peered":"2023-10-06T05:18:00.270755+0000","last_clean":"2023-10-06T05:18:00.270755+0000","last_became_active":"2023-10-06T05:18:00.269160+0000","last_became_peered":"2023-10-06T05:18:00.269160+0000","last_unstale":"2023-10-06T05:18:00.270755+0000","last_undegraded":"2023-10-06T05:18:00.270755+0000","last_fullsized":"2023-10-06T05:18:00.270755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276341+0000","last_change":"2023-10-06T05:18:00.270059+0000","last_active":"2023-10-06T05:18:00.276341+0000","last_peered":"2023-10-06T05:18:00.276341+0000","last_clean":"2023-10-06T05:18:00.276341+0000","last_became_active":"2023-10-06T05:18:00.269731+0000","last_became_peered":"2023-10-06T05:18:00.269731+0000","last_unstale":"2023-10-06T05:18:00.276341+0000","last_undegraded":"2023-10-06T05:18:00.276341+0000","last_fullsized":"2023-10-06T05:18:00.276341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:39.723364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276464+0000","last_change":"2023-10-06T05:18:00.271238+0000","last_active":"2023-10-06T05:18:00.276464+0000","last_peered":"2023-10-06T05:18:00.276464+0000","last_clean":"2023-10-06T05:18:00.276464+0000","last_became_active":"2023-10-06T05:18:00.270924+0000","last_became_peered":"2023-10-06T05:18:00.270924+0000","last_unstale":"2023-10-06T05:18:00.276464+0000","last_undegraded":"2023-10-06T05:18:00.276464+0000","last_fullsized":"2023-10-06T05:18:00.276464+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:59.068875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.256429+0000","last_change":"2023-10-06T05:18:00.252619+0000","last_active":"2023-10-06T05:18:00.256429+0000","last_peered":"2023-10-06T05:18:00.256429+0000","last_clean":"2023-10-06T05:18:00.256429+0000","last_became_active":"2023-10-06T05:18:00.252356+0000","last_became_peered":"2023-10-06T05:18:00.252356+0000","last_unstale":"2023-10-06T05:18:00.256429+0000","last_undegraded":"2023-10-06T05:18:00.256429+0000","last_fullsized":"2023-10-06T05:18:00.256429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:51.085783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276386+0000","last_change":"2023-10-06T05:18:00.270371+0000","last_active":"2023-10-06T05:18:00.276386+0000","last_peered":"2023-10-06T05:18:00.276386+0000","last_clean":"2023-10-06T05:18:00.276386+0000","last_became_active":"2023-10-06T05:18:00.270057+0000","last_became_peered":"2023-10-06T05:18:00.270057+0000","last_unstale":"2023-10-06T05:18:00.276386+0000","last_undegraded":"2023-10-06T05:18:00.276386+0000","last_fullsized":"2023-10-06T05:18:00.276386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:09.782005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267999+0000","last_change":"2023-10-06T05:18:00.257240+0000","last_active":"2023-10-06T05:18:00.267999+0000","last_peered":"2023-10-06T05:18:00.267999+0000","last_clean":"2023-10-06T05:18:00.267999+0000","last_became_active":"2023-10-06T05:18:00.256724+0000","last_became_peered":"2023-10-06T05:18:00.256724+0000","last_unstale":"2023-10-06T05:18:00.267999+0000","last_undegraded":"2023-10-06T05:18:00.267999+0000","last_fullsized":"2023-10-06T05:18:00.267999+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276488+0000","last_change":"2023-10-06T05:18:00.271414+0000","last_active":"2023-10-06T05:18:00.276488+0000","last_peered":"2023-10-06T05:18:00.276488+0000","last_clean":"2023-10-06T05:18:00.276488+0000","last_became_active":"2023-10-06T05:18:00.269313+0000","last_became_peered":"2023-10-06T05:18:00.269313+0000","last_unstale":"2023-10-06T05:18:00.276488+0000","last_undegraded":"2023-10-06T05:18:00.276488+0000","last_fullsized":"2023-10-06T05:18:00.276488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:14.311697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.273450+0000","last_change":"2023-10-06T05:18:00.271086+0000","last_active":"2023-10-06T05:18:00.273450+0000","last_peered":"2023-10-06T05:18:00.273450+0000","last_clean":"2023-10-06T05:18:00.273450+0000","last_became_active":"2023-10-06T05:18:00.270779+0000","last_became_peered":"2023-10-06T05:18:00.270779+0000","last_unstale":"2023-10-06T05:18:00.273450+0000","last_undegraded":"2023-10-06T05:18:00.273450+0000","last_fullsized":"2023-10-06T05:18:00.273450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.278360+0000","last_change":"2023-10-06T05:18:00.274034+0000","last_active":"2023-10-06T05:18:00.278360+0000","last_peered":"2023-10-06T05:18:00.278360+0000","last_clean":"2023-10-06T05:18:00.278360+0000","last_became_active":"2023-10-06T05:18:00.273761+0000","last_became_peered":"2023-10-06T05:18:00.273761+0000","last_unstale":"2023-10-06T05:18:00.278360+0000","last_undegraded":"2023-10-06T05:18:00.278360+0000","last_fullsized":"2023-10-06T05:18:00.278360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.266117+0000","last_change":"2023-10-06T05:18:00.257811+0000","last_active":"2023-10-06T05:18:00.266117+0000","last_peered":"2023-10-06T05:18:00.266117+0000","last_clean":"2023-10-06T05:18:00.266117+0000","last_became_active":"2023-10-06T05:18:00.257397+0000","last_became_peered":"2023-10-06T05:18:00.257397+0000","last_unstale":"2023-10-06T05:18:00.266117+0000","last_undegraded":"2023-10-06T05:18:00.266117+0000","last_fullsized":"2023-10-06T05:18:00.266117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:05.643009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264562+0000","last_change":"2023-10-06T05:18:00.253362+0000","last_active":"2023-10-06T05:18:00.264562+0000","last_peered":"2023-10-06T05:18:00.264562+0000","last_clean":"2023-10-06T05:18:00.264562+0000","last_became_active":"2023-10-06T05:18:00.253099+0000","last_became_peered":"2023-10-06T05:18:00.253099+0000","last_unstale":"2023-10-06T05:18:00.264562+0000","last_undegraded":"2023-10-06T05:18:00.264562+0000","last_fullsized":"2023-10-06T05:18:00.264562+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:43.058266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264993+0000","last_change":"2023-10-06T05:18:00.260492+0000","last_active":"2023-10-06T05:18:00.264993+0000","last_peered":"2023-10-06T05:18:00.264993+0000","last_clean":"2023-10-06T05:18:00.264993+0000","last_became_active":"2023-10-06T05:18:00.259904+0000","last_became_peered":"2023-10-06T05:18:00.259904+0000","last_unstale":"2023-10-06T05:18:00.264993+0000","last_undegraded":"2023-10-06T05:18:00.264993+0000","last_fullsized":"2023-10-06T05:18:00.264993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:31.382530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276229+0000","last_change":"2023-10-06T05:18:00.269188+0000","last_active":"2023-10-06T05:18:00.276229+0000","last_peered":"2023-10-06T05:18:00.276229+0000","last_clean":"2023-10-06T05:18:00.276229+0000","last_became_active":"2023-10-06T05:18:00.268797+0000","last_became_peered":"2023-10-06T05:18:00.268797+0000","last_unstale":"2023-10-06T05:18:00.276229+0000","last_undegraded":"2023-10-06T05:18:00.276229+0000","last_fullsized":"2023-10-06T05:18:00.276229+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268053+0000","last_change":"2023-10-06T05:18:00.259932+0000","last_active":"2023-10-06T05:18:00.268053+0000","last_peered":"2023-10-06T05:18:00.268053+0000","last_clean":"2023-10-06T05:18:00.268053+0000","last_became_active":"2023-10-06T05:18:00.259624+0000","last_became_peered":"2023-10-06T05:18:00.259624+0000","last_unstale":"2023-10-06T05:18:00.268053+0000","last_undegraded":"2023-10-06T05:18:00.268053+0000","last_fullsized":"2023-10-06T05:18:00.268053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:52.838407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.271986+0000","last_change":"2023-10-06T05:18:00.266626+0000","last_active":"2023-10-06T05:18:00.271986+0000","last_peered":"2023-10-06T05:18:00.271986+0000","last_clean":"2023-10-06T05:18:00.271986+0000","last_became_active":"2023-10-06T05:18:00.266309+0000","last_became_peered":"2023-10-06T05:18:00.266309+0000","last_unstale":"2023-10-06T05:18:00.271986+0000","last_undegraded":"2023-10-06T05:18:00.271986+0000","last_fullsized":"2023-10-06T05:18:00.271986+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:22.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264747+0000","last_change":"2023-10-06T05:18:00.261859+0000","last_active":"2023-10-06T05:18:00.264747+0000","last_peered":"2023-10-06T05:18:00.264747+0000","last_clean":"2023-10-06T05:18:00.264747+0000","last_became_active":"2023-10-06T05:18:00.257780+0000","last_became_peered":"2023-10-06T05:18:00.257780+0000","last_unstale":"2023-10-06T05:18:00.264747+0000","last_undegraded":"2023-10-06T05:18:00.264747+0000","last_fullsized":"2023-10-06T05:18:00.264747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:57.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258110+0000","last_change":"2023-10-06T05:18:00.252247+0000","last_active":"2023-10-06T05:18:00.258110+0000","last_peered":"2023-10-06T05:18:00.258110+0000","last_clean":"2023-10-06T05:18:00.258110+0000","last_became_active":"2023-10-06T05:18:00.251944+0000","last_became_peered":"2023-10-06T05:18:00.251944+0000","last_unstale":"2023-10-06T05:18:00.258110+0000","last_undegraded":"2023-10-06T05:18:00.258110+0000","last_fullsized":"2023-10-06T05:18:00.258110+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:56.100599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270520+0000","last_change":"2023-10-06T05:18:00.266347+0000","last_active":"2023-10-06T05:18:00.270520+0000","last_peered":"2023-10-06T05:18:00.270520+0000","last_clean":"2023-10-06T05:18:00.270520+0000","last_became_active":"2023-10-06T05:18:00.266110+0000","last_became_peered":"2023-10-06T05:18:00.266110+0000","last_unstale":"2023-10-06T05:18:00.270520+0000","last_undegraded":"2023-10-06T05:18:00.270520+0000","last_fullsized":"2023-10-06T05:18:00.270520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:24.495094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.254068+0000","last_change":"2023-10-06T05:18:00.251450+0000","last_active":"2023-10-06T05:18:00.254068+0000","last_peered":"2023-10-06T05:18:00.254068+0000","last_clean":"2023-10-06T05:18:00.254068+0000","last_became_active":"2023-10-06T05:18:00.251179+0000","last_became_peered":"2023-10-06T05:18:00.251179+0000","last_unstale":"2023-10-06T05:18:00.254068+0000","last_undegraded":"2023-10-06T05:18:00.254068+0000","last_fullsized":"2023-10-06T05:18:00.254068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:57.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275097+0000","last_change":"2023-10-06T05:18:00.258527+0000","last_active":"2023-10-06T05:18:00.275097+0000","last_peered":"2023-10-06T05:18:00.275097+0000","last_clean":"2023-10-06T05:18:00.275097+0000","last_became_active":"2023-10-06T05:18:00.258187+0000","last_became_peered":"2023-10-06T05:18:00.258187+0000","last_unstale":"2023-10-06T05:18:00.275097+0000","last_undegraded":"2023-10-06T05:18:00.275097+0000","last_fullsized":"2023-10-06T05:18:00.275097+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258080+0000","last_change":"2023-10-06T05:18:00.253028+0000","last_active":"2023-10-06T05:18:00.258080+0000","last_peered":"2023-10-06T05:18:00.258080+0000","last_clean":"2023-10-06T05:18:00.258080+0000","last_became_active":"2023-10-06T05:18:00.252747+0000","last_became_peered":"2023-10-06T05:18:00.252747+0000","last_unstale":"2023-10-06T05:18:00.258080+0000","last_undegraded":"2023-10-06T05:18:00.258080+0000","last_fullsized":"2023-10-06T05:18:00.258080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:44.687164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.253919+0000","last_change":"2023-10-06T05:18:00.251407+0000","last_active":"2023-10-06T05:18:00.253919+0000","last_peered":"2023-10-06T05:18:00.253919+0000","last_clean":"2023-10-06T05:18:00.253919+0000","last_became_active":"2023-10-06T05:18:00.251112+0000","last_became_peered":"2023-10-06T05:18:00.251112+0000","last_unstale":"2023-10-06T05:18:00.253919+0000","last_undegraded":"2023-10-06T05:18:00.253919+0000","last_fullsized":"2023-10-06T05:18:00.253919+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:19.666509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.271966+0000","last_change":"2023-10-06T05:18:00.266564+0000","last_active":"2023-10-06T05:18:00.271966+0000","last_peered":"2023-10-06T05:18:00.271966+0000","last_clean":"2023-10-06T05:18:00.271966+0000","last_became_active":"2023-10-06T05:18:00.266255+0000","last_became_peered":"2023-10-06T05:18:00.266255+0000","last_unstale":"2023-10-06T05:18:00.271966+0000","last_undegraded":"2023-10-06T05:18:00.271966+0000","last_fullsized":"2023-10-06T05:18:00.271966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.248417+0000","last_change":"2023-10-06T05:18:00.246355+0000","last_active":"2023-10-06T05:18:00.248417+0000","last_peered":"2023-10-06T05:18:00.248417+0000","last_clean":"2023-10-06T05:18:00.248417+0000","last_became_active":"2023-10-06T05:18:00.246083+0000","last_became_peered":"2023-10-06T05:18:00.246083+0000","last_unstale":"2023-10-06T05:18:00.248417+0000","last_undegraded":"2023-10-06T05:18:00.248417+0000","last_fullsized":"2023-10-06T05:18:00.248417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:00.272247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.273676+0000","last_change":"2023-10-06T05:18:00.270787+0000","last_active":"2023-10-06T05:18:00.273676+0000","last_peered":"2023-10-06T05:18:00.273676+0000","last_clean":"2023-10-06T05:18:00.273676+0000","last_became_active":"2023-10-06T05:18:00.270455+0000","last_became_peered":"2023-10-06T05:18:00.270455+0000","last_unstale":"2023-10-06T05:18:00.273676+0000","last_undegraded":"2023-10-06T05:18:00.273676+0000","last_fullsized":"2023-10-06T05:18:00.273676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275240+0000","last_change":"2023-10-06T05:17:59.243498+0000","last_active":"2023-10-06T05:18:00.275240+0000","last_peered":"2023-10-06T05:18:00.275240+0000","last_clean":"2023-10-06T05:18:00.275240+0000","last_became_active":"2023-10-06T05:17:59.243236+0000","last_became_peered":"2023-10-06T05:17:59.243236+0000","last_unstale":"2023-10-06T05:18:00.275240+0000","last_undegraded":"2023-10-06T05:18:00.275240+0000","last_fullsized":"2023-10-06T05:18:00.275240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:05.824830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270372+0000","last_change":"2023-10-06T05:18:00.267044+0000","last_active":"2023-10-06T05:18:00.270372+0000","last_peered":"2023-10-06T05:18:00.270372+0000","last_clean":"2023-10-06T05:18:00.270372+0000","last_became_active":"2023-10-06T05:18:00.266737+0000","last_became_peered":"2023-10-06T05:18:00.266737+0000","last_unstale":"2023-10-06T05:18:00.270372+0000","last_undegraded":"2023-10-06T05:18:00.270372+0000","last_fullsized":"2023-10-06T05:18:00.270372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:06.177164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270511+0000","last_change":"2023-10-06T05:17:59.245349+0000","last_active":"2023-10-06T05:18:00.270511+0000","last_peered":"2023-10-06T05:18:00.270511+0000","last_clean":"2023-10-06T05:18:00.270511+0000","last_became_active":"2023-10-06T05:17:59.245007+0000","last_became_peered":"2023-10-06T05:17:59.245007+0000","last_unstale":"2023-10-06T05:18:00.270511+0000","last_undegraded":"2023-10-06T05:18:00.270511+0000","last_fullsized":"2023-10-06T05:18:00.270511+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:05.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267808+0000","last_change":"2023-10-06T05:18:00.257009+0000","last_active":"2023-10-06T05:18:00.267808+0000","last_peered":"2023-10-06T05:18:00.267808+0000","last_clean":"2023-10-06T05:18:00.267808+0000","last_became_active":"2023-10-06T05:18:00.256695+0000","last_became_peered":"2023-10-06T05:18:00.256695+0000","last_unstale":"2023-10-06T05:18:00.267808+0000","last_undegraded":"2023-10-06T05:18:00.267808+0000","last_fullsized":"2023-10-06T05:18:00.267808+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.273396+0000","last_change":"2023-10-06T05:17:59.237465+0000","last_active":"2023-10-06T05:18:00.273396+0000","last_peered":"2023-10-06T05:18:00.273396+0000","last_clean":"2023-10-06T05:18:00.273396+0000","last_became_active":"2023-10-06T05:17:59.237181+0000","last_became_peered":"2023-10-06T05:17:59.237181+0000","last_unstale":"2023-10-06T05:18:00.273396+0000","last_undegraded":"2023-10-06T05:18:00.273396+0000","last_fullsized":"2023-10-06T05:18:00.273396+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:16.338535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258373+0000","last_change":"2023-10-06T05:18:00.256807+0000","last_active":"2023-10-06T05:18:00.258373+0000","last_peered":"2023-10-06T05:18:00.258373+0000","last_clean":"2023-10-06T05:18:00.258373+0000","last_became_active":"2023-10-06T05:18:00.256532+0000","last_became_peered":"2023-10-06T05:18:00.256532+0000","last_unstale":"2023-10-06T05:18:00.258373+0000","last_undegraded":"2023-10-06T05:18:00.258373+0000","last_fullsized":"2023-10-06T05:18:00.258373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272849+0000","last_change":"2023-10-06T05:17:59.239309+0000","last_active":"2023-10-06T05:18:00.272849+0000","last_peered":"2023-10-06T05:18:00.272849+0000","last_clean":"2023-10-06T05:18:00.272849+0000","last_became_active":"2023-10-06T05:17:59.239008+0000","last_became_peered":"2023-10-06T05:17:59.239008+0000","last_unstale":"2023-10-06T05:18:00.272849+0000","last_undegraded":"2023-10-06T05:18:00.272849+0000","last_fullsized":"2023-10-06T05:18:00.272849+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:53.729179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.253951+0000","last_change":"2023-10-06T05:18:00.249760+0000","last_active":"2023-10-06T05:18:00.253951+0000","last_peered":"2023-10-06T05:18:00.253951+0000","last_clean":"2023-10-06T05:18:00.253951+0000","last_became_active":"2023-10-06T05:18:00.249217+0000","last_became_peered":"2023-10-06T05:18:00.249217+0000","last_unstale":"2023-10-06T05:18:00.253951+0000","last_undegraded":"2023-10-06T05:18:00.253951+0000","last_fullsized":"2023-10-06T05:18:00.253951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:22.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276296+0000","last_change":"2023-10-06T05:17:59.250696+0000","last_active":"2023-10-06T05:18:00.276296+0000","last_peered":"2023-10-06T05:18:00.276296+0000","last_clean":"2023-10-06T05:18:00.276296+0000","last_became_active":"2023-10-06T05:17:59.250383+0000","last_became_peered":"2023-10-06T05:17:59.250383+0000","last_unstale":"2023-10-06T05:18:00.276296+0000","last_undegraded":"2023-10-06T05:18:00.276296+0000","last_fullsized":"2023-10-06T05:18:00.276296+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:04.636437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276278+0000","last_change":"2023-10-06T05:18:00.269612+0000","last_active":"2023-10-06T05:18:00.276278+0000","last_peered":"2023-10-06T05:18:00.276278+0000","last_clean":"2023-10-06T05:18:00.276278+0000","last_became_active":"2023-10-06T05:18:00.268780+0000","last_became_peered":"2023-10-06T05:18:00.268780+0000","last_unstale":"2023-10-06T05:18:00.276278+0000","last_undegraded":"2023-10-06T05:18:00.276278+0000","last_fullsized":"2023-10-06T05:18:00.276278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:22.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.274334+0000","last_change":"2023-10-06T05:17:59.247904+0000","last_active":"2023-10-06T05:18:00.274334+0000","last_peered":"2023-10-06T05:18:00.274334+0000","last_clean":"2023-10-06T05:18:00.274334+0000","last_became_active":"2023-10-06T05:17:59.247645+0000","last_became_peered":"2023-10-06T05:17:59.247645+0000","last_unstale":"2023-10-06T05:18:00.274334+0000","last_undegraded":"2023-10-06T05:18:00.274334+0000","last_fullsized":"2023-10-06T05:18:00.274334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:23.102068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276341+0000","last_change":"2023-10-06T05:18:00.272334+0000","last_active":"2023-10-06T05:18:00.276341+0000","last_peered":"2023-10-06T05:18:00.276341+0000","last_clean":"2023-10-06T05:18:00.276341+0000","last_became_active":"2023-10-06T05:18:00.272051+0000","last_became_peered":"2023-10-06T05:18:00.272051+0000","last_unstale":"2023-10-06T05:18:00.276341+0000","last_undegraded":"2023-10-06T05:18:00.276341+0000","last_fullsized":"2023-10-06T05:18:00.276341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:54.805238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.236476+0000","last_change":"2023-10-06T05:17:59.248353+0000","last_active":"2023-10-06T05:18:00.236476+0000","last_peered":"2023-10-06T05:18:00.236476+0000","last_clean":"2023-10-06T05:18:00.236476+0000","last_became_active":"2023-10-06T05:17:59.247998+0000","last_became_peered":"2023-10-06T05:17:59.247998+0000","last_unstale":"2023-10-06T05:18:00.236476+0000","last_undegraded":"2023-10-06T05:18:00.236476+0000","last_fullsized":"2023-10-06T05:18:00.236476+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:25.646387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.259108+0000","last_change":"2023-10-06T05:18:00.257458+0000","last_active":"2023-10-06T05:18:00.259108+0000","last_peered":"2023-10-06T05:18:00.259108+0000","last_clean":"2023-10-06T05:18:00.259108+0000","last_became_active":"2023-10-06T05:18:00.257165+0000","last_became_peered":"2023-10-06T05:18:00.257165+0000","last_unstale":"2023-10-06T05:18:00.259108+0000","last_undegraded":"2023-10-06T05:18:00.259108+0000","last_fullsized":"2023-10-06T05:18:00.259108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:24.836220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244601+0000","last_change":"2023-10-06T05:17:59.241941+0000","last_active":"2023-10-06T05:18:00.244601+0000","last_peered":"2023-10-06T05:18:00.244601+0000","last_clean":"2023-10-06T05:18:00.244601+0000","last_became_active":"2023-10-06T05:17:59.241520+0000","last_became_peered":"2023-10-06T05:17:59.241520+0000","last_unstale":"2023-10-06T05:18:00.244601+0000","last_undegraded":"2023-10-06T05:18:00.244601+0000","last_fullsized":"2023-10-06T05:18:00.244601+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:18.915215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264800+0000","last_change":"2023-10-06T05:18:00.260233+0000","last_active":"2023-10-06T05:18:00.264800+0000","last_peered":"2023-10-06T05:18:00.264800+0000","last_clean":"2023-10-06T05:18:00.264800+0000","last_became_active":"2023-10-06T05:18:00.259546+0000","last_became_peered":"2023-10-06T05:18:00.259546+0000","last_unstale":"2023-10-06T05:18:00.264800+0000","last_undegraded":"2023-10-06T05:18:00.264800+0000","last_fullsized":"2023-10-06T05:18:00.264800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:44.687164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.242225+0000","last_change":"2023-10-06T05:17:59.240300+0000","last_active":"2023-10-06T05:18:00.242225+0000","last_peered":"2023-10-06T05:18:00.242225+0000","last_clean":"2023-10-06T05:18:00.242225+0000","last_became_active":"2023-10-06T05:17:59.240027+0000","last_became_peered":"2023-10-06T05:17:59.240027+0000","last_unstale":"2023-10-06T05:18:00.242225+0000","last_undegraded":"2023-10-06T05:18:00.242225+0000","last_fullsized":"2023-10-06T05:18:00.242225+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:23.488521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.259525+0000","last_change":"2023-10-06T05:18:00.255781+0000","last_active":"2023-10-06T05:18:00.259525+0000","last_peered":"2023-10-06T05:18:00.259525+0000","last_clean":"2023-10-06T05:18:00.259525+0000","last_became_active":"2023-10-06T05:18:00.255447+0000","last_became_peered":"2023-10-06T05:18:00.255447+0000","last_unstale":"2023-10-06T05:18:00.259525+0000","last_undegraded":"2023-10-06T05:18:00.259525+0000","last_fullsized":"2023-10-06T05:18:00.259525+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.255928+0000","last_change":"2023-10-06T05:17:59.248906+0000","last_active":"2023-10-06T05:18:00.255928+0000","last_peered":"2023-10-06T05:18:00.255928+0000","last_clean":"2023-10-06T05:18:00.255928+0000","last_became_active":"2023-10-06T05:17:59.248594+0000","last_became_peered":"2023-10-06T05:17:59.248594+0000","last_unstale":"2023-10-06T05:18:00.255928+0000","last_undegraded":"2023-10-06T05:18:00.255928+0000","last_fullsized":"2023-10-06T05:18:00.255928+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:56.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.262995+0000","last_change":"2023-10-06T05:18:00.259360+0000","last_active":"2023-10-06T05:18:00.262995+0000","last_peered":"2023-10-06T05:18:00.262995+0000","last_clean":"2023-10-06T05:18:00.262995+0000","last_became_active":"2023-10-06T05:18:00.259041+0000","last_became_peered":"2023-10-06T05:18:00.259041+0000","last_unstale":"2023-10-06T05:18:00.262995+0000","last_undegraded":"2023-10-06T05:18:00.262995+0000","last_fullsized":"2023-10-06T05:18:00.262995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275998+0000","last_change":"2023-10-06T05:17:59.251452+0000","last_active":"2023-10-06T05:18:00.275998+0000","last_peered":"2023-10-06T05:18:00.275998+0000","last_clean":"2023-10-06T05:18:00.275998+0000","last_became_active":"2023-10-06T05:17:59.251156+0000","last_became_peered":"2023-10-06T05:17:59.251156+0000","last_unstale":"2023-10-06T05:18:00.275998+0000","last_undegraded":"2023-10-06T05:18:00.275998+0000","last_fullsized":"2023-10-06T05:18:00.275998+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:55.094026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268676+0000","last_change":"2023-10-06T05:18:00.259652+0000","last_active":"2023-10-06T05:18:00.268676+0000","last_peered":"2023-10-06T05:18:00.268676+0000","last_clean":"2023-10-06T05:18:00.268676+0000","last_became_active":"2023-10-06T05:18:00.259345+0000","last_became_peered":"2023-10-06T05:18:00.259345+0000","last_unstale":"2023-10-06T05:18:00.268676+0000","last_undegraded":"2023-10-06T05:18:00.268676+0000","last_fullsized":"2023-10-06T05:18:00.268676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.279064+0000","last_change":"2023-10-06T05:17:59.250002+0000","last_active":"2023-10-06T05:18:00.279064+0000","last_peered":"2023-10-06T05:18:00.279064+0000","last_clean":"2023-10-06T05:18:00.279064+0000","last_became_active":"2023-10-06T05:17:59.249731+0000","last_became_peered":"2023-10-06T05:17:59.249731+0000","last_unstale":"2023-10-06T05:18:00.279064+0000","last_undegraded":"2023-10-06T05:18:00.279064+0000","last_fullsized":"2023-10-06T05:18:00.279064+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:08.884333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268400+0000","last_change":"2023-10-06T05:18:00.267476+0000","last_active":"2023-10-06T05:18:00.268400+0000","last_peered":"2023-10-06T05:18:00.268400+0000","last_clean":"2023-10-06T05:18:00.268400+0000","last_became_active":"2023-10-06T05:18:00.267246+0000","last_became_peered":"2023-10-06T05:18:00.267246+0000","last_unstale":"2023-10-06T05:18:00.268400+0000","last_undegraded":"2023-10-06T05:18:00.268400+0000","last_fullsized":"2023-10-06T05:18:00.268400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267608+0000","last_change":"2023-10-06T05:17:59.243961+0000","last_active":"2023-10-06T05:18:00.267608+0000","last_peered":"2023-10-06T05:18:00.267608+0000","last_clean":"2023-10-06T05:18:00.267608+0000","last_became_active":"2023-10-06T05:17:59.243642+0000","last_became_peered":"2023-10-06T05:17:59.243642+0000","last_unstale":"2023-10-06T05:18:00.267608+0000","last_undegraded":"2023-10-06T05:18:00.267608+0000","last_fullsized":"2023-10-06T05:18:00.267608+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:05.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.257564+0000","last_change":"2023-10-06T05:18:00.248610+0000","last_active":"2023-10-06T05:18:00.257564+0000","last_peered":"2023-10-06T05:18:00.257564+0000","last_clean":"2023-10-06T05:18:00.257564+0000","last_became_active":"2023-10-06T05:18:00.248327+0000","last_became_peered":"2023-10-06T05:18:00.248327+0000","last_unstale":"2023-10-06T05:18:00.257564+0000","last_undegraded":"2023-10-06T05:18:00.257564+0000","last_fullsized":"2023-10-06T05:18:00.257564+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.236677+0000","last_change":"2023-10-06T05:17:59.248279+0000","last_active":"2023-10-06T05:18:00.236677+0000","last_peered":"2023-10-06T05:18:00.236677+0000","last_clean":"2023-10-06T05:18:00.236677+0000","last_became_active":"2023-10-06T05:17:59.247982+0000","last_became_peered":"2023-10-06T05:17:59.247982+0000","last_unstale":"2023-10-06T05:18:00.236677+0000","last_undegraded":"2023-10-06T05:18:00.236677+0000","last_fullsized":"2023-10-06T05:18:00.236677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:43.680592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272025+0000","last_change":"2023-10-06T05:18:00.266361+0000","last_active":"2023-10-06T05:18:00.272025+0000","last_peered":"2023-10-06T05:18:00.272025+0000","last_clean":"2023-10-06T05:18:00.272025+0000","last_became_active":"2023-10-06T05:18:00.266063+0000","last_became_peered":"2023-10-06T05:18:00.266063+0000","last_unstale":"2023-10-06T05:18:00.272025+0000","last_undegraded":"2023-10-06T05:18:00.272025+0000","last_fullsized":"2023-10-06T05:18:00.272025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:51.085783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241370+0000","last_change":"2023-10-06T05:17:59.242838+0000","last_active":"2023-10-06T05:18:00.241370+0000","last_peered":"2023-10-06T05:18:00.241370+0000","last_clean":"2023-10-06T05:18:00.241370+0000","last_became_active":"2023-10-06T05:17:59.242560+0000","last_became_peered":"2023-10-06T05:17:59.242560+0000","last_unstale":"2023-10-06T05:18:00.241370+0000","last_undegraded":"2023-10-06T05:18:00.241370+0000","last_fullsized":"2023-10-06T05:18:00.241370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:34.986106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258218+0000","last_change":"2023-10-06T05:18:00.248546+0000","last_active":"2023-10-06T05:18:00.258218+0000","last_peered":"2023-10-06T05:18:00.258218+0000","last_clean":"2023-10-06T05:18:00.258218+0000","last_became_active":"2023-10-06T05:18:00.248280+0000","last_became_peered":"2023-10-06T05:18:00.248280+0000","last_unstale":"2023-10-06T05:18:00.258218+0000","last_undegraded":"2023-10-06T05:18:00.258218+0000","last_fullsized":"2023-10-06T05:18:00.258218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:51.687265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275494+0000","last_change":"2023-10-06T05:17:59.242977+0000","last_active":"2023-10-06T05:18:00.275494+0000","last_peered":"2023-10-06T05:18:00.275494+0000","last_clean":"2023-10-06T05:18:00.275494+0000","last_became_active":"2023-10-06T05:17:59.242700+0000","last_became_peered":"2023-10-06T05:17:59.242700+0000","last_unstale":"2023-10-06T05:18:00.275494+0000","last_undegraded":"2023-10-06T05:18:00.275494+0000","last_fullsized":"2023-10-06T05:18:00.275494+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:14.113766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258815+0000","last_change":"2023-10-06T05:18:00.256835+0000","last_active":"2023-10-06T05:18:00.258815+0000","last_peered":"2023-10-06T05:18:00.258815+0000","last_clean":"2023-10-06T05:18:00.258815+0000","last_became_active":"2023-10-06T05:18:00.256557+0000","last_became_peered":"2023-10-06T05:18:00.256557+0000","last_unstale":"2023-10-06T05:18:00.258815+0000","last_undegraded":"2023-10-06T05:18:00.258815+0000","last_fullsized":"2023-10-06T05:18:00.258815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276895+0000","last_change":"2023-10-06T05:17:59.249226+0000","last_active":"2023-10-06T05:18:00.276895+0000","last_peered":"2023-10-06T05:18:00.276895+0000","last_clean":"2023-10-06T05:18:00.276895+0000","last_became_active":"2023-10-06T05:17:59.248875+0000","last_became_peered":"2023-10-06T05:17:59.248875+0000","last_unstale":"2023-10-06T05:18:00.276895+0000","last_undegraded":"2023-10-06T05:18:00.276895+0000","last_fullsized":"2023-10-06T05:18:00.276895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:14.113766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276443+0000","last_change":"2023-10-06T05:18:00.270683+0000","last_active":"2023-10-06T05:18:00.276443+0000","last_peered":"2023-10-06T05:18:00.276443+0000","last_clean":"2023-10-06T05:18:00.276443+0000","last_became_active":"2023-10-06T05:18:00.268778+0000","last_became_peered":"2023-10-06T05:18:00.268778+0000","last_unstale":"2023-10-06T05:18:00.276443+0000","last_undegraded":"2023-10-06T05:18:00.276443+0000","last_fullsized":"2023-10-06T05:18:00.276443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276681+0000","last_change":"2023-10-06T05:17:59.249619+0000","last_active":"2023-10-06T05:18:00.276681+0000","last_peered":"2023-10-06T05:18:00.276681+0000","last_clean":"2023-10-06T05:18:00.276681+0000","last_became_active":"2023-10-06T05:17:59.249268+0000","last_became_peered":"2023-10-06T05:17:59.249268+0000","last_unstale":"2023-10-06T05:18:00.276681+0000","last_undegraded":"2023-10-06T05:18:00.276681+0000","last_fullsized":"2023-10-06T05:18:00.276681+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:50.044225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.271497+0000","last_change":"2023-10-06T05:18:00.261160+0000","last_active":"2023-10-06T05:18:00.271497+0000","last_peered":"2023-10-06T05:18:00.271497+0000","last_clean":"2023-10-06T05:18:00.271497+0000","last_became_active":"2023-10-06T05:18:00.260836+0000","last_became_peered":"2023-10-06T05:18:00.260836+0000","last_unstale":"2023-10-06T05:18:00.271497+0000","last_undegraded":"2023-10-06T05:18:00.271497+0000","last_fullsized":"2023-10-06T05:18:00.271497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272530+0000","last_change":"2023-10-06T05:17:59.245361+0000","last_active":"2023-10-06T05:18:00.272530+0000","last_peered":"2023-10-06T05:18:00.272530+0000","last_clean":"2023-10-06T05:18:00.272530+0000","last_became_active":"2023-10-06T05:17:59.245059+0000","last_became_peered":"2023-10-06T05:17:59.245059+0000","last_unstale":"2023-10-06T05:18:00.272530+0000","last_undegraded":"2023-10-06T05:18:00.272530+0000","last_fullsized":"2023-10-06T05:18:00.272530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:20.151146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.249226+0000","last_change":"2023-10-06T05:18:00.247084+0000","last_active":"2023-10-06T05:18:00.249226+0000","last_peered":"2023-10-06T05:18:00.249226+0000","last_clean":"2023-10-06T05:18:00.249226+0000","last_became_active":"2023-10-06T05:18:00.246814+0000","last_became_peered":"2023-10-06T05:18:00.246814+0000","last_unstale":"2023-10-06T05:18:00.249226+0000","last_undegraded":"2023-10-06T05:18:00.249226+0000","last_fullsized":"2023-10-06T05:18:00.249226+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:19.666509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.237231+0000","last_change":"2023-10-06T05:17:59.237296+0000","last_active":"2023-10-06T05:18:00.237231+0000","last_peered":"2023-10-06T05:18:00.237231+0000","last_clean":"2023-10-06T05:18:00.237231+0000","last_became_active":"2023-10-06T05:17:59.237025+0000","last_became_peered":"2023-10-06T05:17:59.237025+0000","last_unstale":"2023-10-06T05:18:00.237231+0000","last_undegraded":"2023-10-06T05:18:00.237231+0000","last_fullsized":"2023-10-06T05:18:00.237231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:30.375958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272053+0000","last_change":"2023-10-06T05:18:00.266960+0000","last_active":"2023-10-06T05:18:00.272053+0000","last_peered":"2023-10-06T05:18:00.272053+0000","last_clean":"2023-10-06T05:18:00.272053+0000","last_became_active":"2023-10-06T05:18:00.266690+0000","last_became_peered":"2023-10-06T05:18:00.266690+0000","last_unstale":"2023-10-06T05:18:00.272053+0000","last_undegraded":"2023-10-06T05:18:00.272053+0000","last_fullsized":"2023-10-06T05:18:00.272053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:09.782005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268511+0000","last_change":"2023-10-06T05:17:59.243307+0000","last_active":"2023-10-06T05:18:00.268511+0000","last_peered":"2023-10-06T05:18:00.268511+0000","last_clean":"2023-10-06T05:18:00.268511+0000","last_became_active":"2023-10-06T05:17:59.243006+0000","last_became_peered":"2023-10-06T05:17:59.243006+0000","last_unstale":"2023-10-06T05:18:00.268511+0000","last_undegraded":"2023-10-06T05:18:00.268511+0000","last_fullsized":"2023-10-06T05:18:00.268511+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:43.085633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264171+0000","last_change":"2023-10-06T05:18:00.260265+0000","last_active":"2023-10-06T05:18:00.264171+0000","last_peered":"2023-10-06T05:18:00.264171+0000","last_clean":"2023-10-06T05:18:00.264171+0000","last_became_active":"2023-10-06T05:18:00.259993+0000","last_became_peered":"2023-10-06T05:18:00.259993+0000","last_unstale":"2023-10-06T05:18:00.264171+0000","last_undegraded":"2023-10-06T05:18:00.264171+0000","last_fullsized":"2023-10-06T05:18:00.264171+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.240737+0000","last_change":"2023-10-06T05:17:59.242903+0000","last_active":"2023-10-06T05:18:00.240737+0000","last_peered":"2023-10-06T05:18:00.240737+0000","last_clean":"2023-10-06T05:18:00.240737+0000","last_became_active":"2023-10-06T05:17:59.242658+0000","last_became_peered":"2023-10-06T05:17:59.242658+0000","last_unstale":"2023-10-06T05:18:00.240737+0000","last_undegraded":"2023-10-06T05:18:00.240737+0000","last_fullsized":"2023-10-06T05:18:00.240737+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:39.271735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275355+0000","last_change":"2023-10-06T05:18:00.262962+0000","last_active":"2023-10-06T05:18:00.275355+0000","last_peered":"2023-10-06T05:18:00.275355+0000","last_clean":"2023-10-06T05:18:00.275355+0000","last_became_active":"2023-10-06T05:18:00.262667+0000","last_became_peered":"2023-10-06T05:18:00.262667+0000","last_unstale":"2023-10-06T05:18:00.275355+0000","last_undegraded":"2023-10-06T05:18:00.275355+0000","last_fullsized":"2023-10-06T05:18:00.275355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:51.085783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247893+0000","last_change":"2023-10-06T05:17:59.248713+0000","last_active":"2023-10-06T05:18:00.247893+0000","last_peered":"2023-10-06T05:18:00.247893+0000","last_clean":"2023-10-06T05:18:00.247893+0000","last_became_active":"2023-10-06T05:17:59.248483+0000","last_became_peered":"2023-10-06T05:17:59.248483+0000","last_unstale":"2023-10-06T05:18:00.247893+0000","last_undegraded":"2023-10-06T05:18:00.247893+0000","last_fullsized":"2023-10-06T05:18:00.247893+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:40.124236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.263773+0000","last_change":"2023-10-06T05:18:00.258121+0000","last_active":"2023-10-06T05:18:00.263773+0000","last_peered":"2023-10-06T05:18:00.263773+0000","last_clean":"2023-10-06T05:18:00.263773+0000","last_became_active":"2023-10-06T05:18:00.257865+0000","last_became_peered":"2023-10-06T05:18:00.257865+0000","last_unstale":"2023-10-06T05:18:00.263773+0000","last_undegraded":"2023-10-06T05:18:00.263773+0000","last_fullsized":"2023-10-06T05:18:00.263773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:59.262865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.240737+0000","last_change":"2023-10-06T05:17:59.242377+0000","last_active":"2023-10-06T05:18:00.240737+0000","last_peered":"2023-10-06T05:18:00.240737+0000","last_clean":"2023-10-06T05:18:00.240737+0000","last_became_active":"2023-10-06T05:17:59.242098+0000","last_became_peered":"2023-10-06T05:17:59.242098+0000","last_unstale":"2023-10-06T05:18:00.240737+0000","last_undegraded":"2023-10-06T05:18:00.240737+0000","last_fullsized":"2023-10-06T05:18:00.240737+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:52.601888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268035+0000","last_change":"2023-10-06T05:18:00.258948+0000","last_active":"2023-10-06T05:18:00.268035+0000","last_peered":"2023-10-06T05:18:00.268035+0000","last_clean":"2023-10-06T05:18:00.268035+0000","last_became_active":"2023-10-06T05:18:00.258675+0000","last_became_peered":"2023-10-06T05:18:00.258675+0000","last_unstale":"2023-10-06T05:18:00.268035+0000","last_undegraded":"2023-10-06T05:18:00.268035+0000","last_fullsized":"2023-10-06T05:18:00.268035+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244631+0000","last_change":"2023-10-06T05:17:59.238167+0000","last_active":"2023-10-06T05:18:00.244631+0000","last_peered":"2023-10-06T05:18:00.244631+0000","last_clean":"2023-10-06T05:18:00.244631+0000","last_became_active":"2023-10-06T05:17:59.237881+0000","last_became_peered":"2023-10-06T05:17:59.237881+0000","last_unstale":"2023-10-06T05:18:00.244631+0000","last_undegraded":"2023-10-06T05:18:00.244631+0000","last_fullsized":"2023-10-06T05:18:00.244631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:08.884333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.245822+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:18:00.245822+0000","last_peered":"2023-10-06T05:18:00.245822+0000","last_clean":"2023-10-06T05:18:00.245822+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:18:00.245822+0000","last_undegraded":"2023-10-06T05:18:00.245822+0000","last_fullsized":"2023-10-06T05:18:00.245822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.259572+0000","last_change":"2023-10-06T05:18:00.256434+0000","last_active":"2023-10-06T05:18:00.259572+0000","last_peered":"2023-10-06T05:18:00.259572+0000","last_clean":"2023-10-06T05:18:00.259572+0000","last_became_active":"2023-10-06T05:18:00.255363+0000","last_became_peered":"2023-10-06T05:18:00.255363+0000","last_unstale":"2023-10-06T05:18:00.259572+0000","last_undegraded":"2023-10-06T05:18:00.259572+0000","last_fullsized":"2023-10-06T05:18:00.259572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.240921+0000","last_change":"2023-10-06T05:17:59.240304+0000","last_active":"2023-10-06T05:18:00.240921+0000","last_peered":"2023-10-06T05:18:00.240921+0000","last_clean":"2023-10-06T05:18:00.240921+0000","last_became_active":"2023-10-06T05:17:59.240043+0000","last_became_peered":"2023-10-06T05:17:59.240043+0000","last_unstale":"2023-10-06T05:18:00.240921+0000","last_undegraded":"2023-10-06T05:18:00.240921+0000","last_fullsized":"2023-10-06T05:18:00.240921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:59.265675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241247+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:18:00.241247+0000","last_peered":"2023-10-06T05:18:00.241247+0000","last_clean":"2023-10-06T05:18:00.241247+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:18:00.241247+0000","last_undegraded":"2023-10-06T05:18:00.241247+0000","last_fullsized":"2023-10-06T05:18:00.241247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.269306+0000","last_change":"2023-10-06T05:18:00.265724+0000","last_active":"2023-10-06T05:18:00.269306+0000","last_peered":"2023-10-06T05:18:00.269306+0000","last_clean":"2023-10-06T05:18:00.269306+0000","last_became_active":"2023-10-06T05:18:00.265150+0000","last_became_peered":"2023-10-06T05:18:00.265150+0000","last_unstale":"2023-10-06T05:18:00.269306+0000","last_undegraded":"2023-10-06T05:18:00.269306+0000","last_fullsized":"2023-10-06T05:18:00.269306+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:09.890905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276291+0000","last_change":"2023-10-06T05:17:59.251787+0000","last_active":"2023-10-06T05:18:00.276291+0000","last_peered":"2023-10-06T05:18:00.276291+0000","last_clean":"2023-10-06T05:18:00.276291+0000","last_became_active":"2023-10-06T05:17:59.251455+0000","last_became_peered":"2023-10-06T05:17:59.251455+0000","last_unstale":"2023-10-06T05:18:00.276291+0000","last_undegraded":"2023-10-06T05:18:00.276291+0000","last_fullsized":"2023-10-06T05:18:00.276291+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:40.124236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.236583+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:18:00.236583+0000","last_peered":"2023-10-06T05:18:00.236583+0000","last_clean":"2023-10-06T05:18:00.236583+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:18:00.236583+0000","last_undegraded":"2023-10-06T05:18:00.236583+0000","last_fullsized":"2023-10-06T05:18:00.236583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247061+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:18:00.247061+0000","last_peered":"2023-10-06T05:18:00.247061+0000","last_clean":"2023-10-06T05:18:00.247061+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:18:00.247061+0000","last_undegraded":"2023-10-06T05:18:00.247061+0000","last_fullsized":"2023-10-06T05:18:00.247061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264673+0000","last_change":"2023-10-06T05:18:00.257655+0000","last_active":"2023-10-06T05:18:00.264673+0000","last_peered":"2023-10-06T05:18:00.264673+0000","last_clean":"2023-10-06T05:18:00.264673+0000","last_became_active":"2023-10-06T05:18:00.257358+0000","last_became_peered":"2023-10-06T05:18:00.257358+0000","last_unstale":"2023-10-06T05:18:00.264673+0000","last_undegraded":"2023-10-06T05:18:00.264673+0000","last_fullsized":"2023-10-06T05:18:00.264673+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:39.723364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T05:18:01.250153+0000","last_change":"2023-10-06T05:17:59.247681+0000","last_active":"2023-10-06T05:18:01.250153+0000","last_peered":"2023-10-06T05:18:01.250153+0000","last_clean":"2023-10-06T05:18:01.250153+0000","last_became_active":"2023-10-06T05:17:59.247406+0000","last_became_peered":"2023-10-06T05:17:59.247406+0000","last_unstale":"2023-10-06T05:18:01.250153+0000","last_undegraded":"2023-10-06T05:18:01.250153+0000","last_fullsized":"2023-10-06T05:18:01.250153+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:40.606016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.246267+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:18:00.246267+0000","last_peered":"2023-10-06T05:18:00.246267+0000","last_clean":"2023-10-06T05:18:00.246267+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:18:00.246267+0000","last_undegraded":"2023-10-06T05:18:00.246267+0000","last_fullsized":"2023-10-06T05:18:00.246267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276821+0000","last_change":"2023-10-06T05:18:00.272933+0000","last_active":"2023-10-06T05:18:00.276821+0000","last_peered":"2023-10-06T05:18:00.276821+0000","last_clean":"2023-10-06T05:18:00.276821+0000","last_became_active":"2023-10-06T05:18:00.272397+0000","last_became_peered":"2023-10-06T05:18:00.272397+0000","last_unstale":"2023-10-06T05:18:00.276821+0000","last_undegraded":"2023-10-06T05:18:00.276821+0000","last_fullsized":"2023-10-06T05:18:00.276821+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247188+0000","last_change":"2023-10-06T05:17:59.247516+0000","last_active":"2023-10-06T05:18:00.247188+0000","last_peered":"2023-10-06T05:18:00.247188+0000","last_clean":"2023-10-06T05:18:00.247188+0000","last_became_active":"2023-10-06T05:17:59.247223+0000","last_became_peered":"2023-10-06T05:17:59.247223+0000","last_unstale":"2023-10-06T05:18:00.247188+0000","last_undegraded":"2023-10-06T05:18:00.247188+0000","last_fullsized":"2023-10-06T05:18:00.247188+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.667357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.246919+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:18:00.246919+0000","last_peered":"2023-10-06T05:18:00.246919+0000","last_clean":"2023-10-06T05:18:00.246919+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:18:00.246919+0000","last_undegraded":"2023-10-06T05:18:00.246919+0000","last_fullsized":"2023-10-06T05:18:00.246919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264308+0000","last_change":"2023-10-06T05:18:00.258394+0000","last_active":"2023-10-06T05:18:00.264308+0000","last_peered":"2023-10-06T05:18:00.264308+0000","last_clean":"2023-10-06T05:18:00.264308+0000","last_became_active":"2023-10-06T05:18:00.258118+0000","last_became_peered":"2023-10-06T05:18:00.258118+0000","last_unstale":"2023-10-06T05:18:00.264308+0000","last_undegraded":"2023-10-06T05:18:00.264308+0000","last_fullsized":"2023-10-06T05:18:00.264308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244236+0000","last_change":"2023-10-06T05:17:59.241749+0000","last_active":"2023-10-06T05:18:00.244236+0000","last_peered":"2023-10-06T05:18:00.244236+0000","last_clean":"2023-10-06T05:18:00.244236+0000","last_became_active":"2023-10-06T05:17:59.241482+0000","last_became_peered":"2023-10-06T05:17:59.241482+0000","last_unstale":"2023-10-06T05:18:00.244236+0000","last_undegraded":"2023-10-06T05:18:00.244236+0000","last_fullsized":"2023-10-06T05:18:00.244236+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:59.265675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.252157+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:18:00.252157+0000","last_peered":"2023-10-06T05:18:00.252157+0000","last_clean":"2023-10-06T05:18:00.252157+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:18:00.252157+0000","last_undegraded":"2023-10-06T05:18:00.252157+0000","last_fullsized":"2023-10-06T05:18:00.252157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.260566+0000","last_change":"2023-10-06T05:18:00.254085+0000","last_active":"2023-10-06T05:18:00.260566+0000","last_peered":"2023-10-06T05:18:00.260566+0000","last_clean":"2023-10-06T05:18:00.260566+0000","last_became_active":"2023-10-06T05:18:00.253803+0000","last_became_peered":"2023-10-06T05:18:00.253803+0000","last_unstale":"2023-10-06T05:18:00.260566+0000","last_undegraded":"2023-10-06T05:18:00.260566+0000","last_fullsized":"2023-10-06T05:18:00.260566+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.673929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241666+0000","last_change":"2023-10-06T05:17:59.245152+0000","last_active":"2023-10-06T05:18:00.241666+0000","last_peered":"2023-10-06T05:18:00.241666+0000","last_clean":"2023-10-06T05:18:00.241666+0000","last_became_active":"2023-10-06T05:17:59.244828+0000","last_became_peered":"2023-10-06T05:17:59.244828+0000","last_unstale":"2023-10-06T05:18:00.241666+0000","last_undegraded":"2023-10-06T05:18:00.241666+0000","last_fullsized":"2023-10-06T05:18:00.241666+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:20.151146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241245+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:18:00.241245+0000","last_peered":"2023-10-06T05:18:00.241245+0000","last_clean":"2023-10-06T05:18:00.241245+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:18:00.241245+0000","last_undegraded":"2023-10-06T05:18:00.241245+0000","last_fullsized":"2023-10-06T05:18:00.241245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.251333+0000","last_change":"2023-10-06T05:18:00.248737+0000","last_active":"2023-10-06T05:18:00.251333+0000","last_peered":"2023-10-06T05:18:00.251333+0000","last_clean":"2023-10-06T05:18:00.251333+0000","last_became_active":"2023-10-06T05:18:00.248458+0000","last_became_peered":"2023-10-06T05:18:00.248458+0000","last_unstale":"2023-10-06T05:18:00.251333+0000","last_undegraded":"2023-10-06T05:18:00.251333+0000","last_fullsized":"2023-10-06T05:18:00.251333+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:09.782005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244107+0000","last_change":"2023-10-06T05:17:59.247105+0000","last_active":"2023-10-06T05:18:00.244107+0000","last_peered":"2023-10-06T05:18:00.244107+0000","last_clean":"2023-10-06T05:18:00.244107+0000","last_became_active":"2023-10-06T05:17:59.246878+0000","last_became_peered":"2023-10-06T05:17:59.246878+0000","last_unstale":"2023-10-06T05:18:00.244107+0000","last_undegraded":"2023-10-06T05:18:00.244107+0000","last_fullsized":"2023-10-06T05:18:00.244107+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:23.488521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241510+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:18:00.241510+0000","last_peered":"2023-10-06T05:18:00.241510+0000","last_clean":"2023-10-06T05:18:00.241510+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:18:00.241510+0000","last_undegraded":"2023-10-06T05:18:00.241510+0000","last_fullsized":"2023-10-06T05:18:00.241510+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":739669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.83099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":148249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.872}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.754}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.94599999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.0129999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":0.91500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.99299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":738153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.439},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.84899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.745}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":147729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.371},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.33900000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.434}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.83799999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":1.0700000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.03}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":147765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.64000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":149265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.875}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.84699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.752},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.78900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":148271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.77200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.753}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":7.742,"5min":7.742,"15min":7.742},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.83099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":147807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2780000000000005,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.80300000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":9.25,"5min":9.25,"15min":9.25},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.108000000000001,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":10.384,"5min":10.384,"15min":10.384},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":9.1699999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.90500000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.205,"5min":10.205,"15min":10.205},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.70999999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.98099999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.1000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.056},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.83999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:06.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T05:18:07.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.029+0000 7f72db7ff700 1 Processor -- start 2023-10-06T05:18:07.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.029+0000 7f72db7ff700 1 -- start start 2023-10-06T05:18:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.030+0000 7f72db7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f72d406ec30 0x7f72d414ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.030+0000 7f72db7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d40c6df0 con 0x7f72d406b510 2023-10-06T05:18:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.030+0000 7f72db7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d40c70d0 con 0x7f72d40ca770 2023-10-06T05:18:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.030+0000 7f72db7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d40c7a70 con 0x7f72d406ec30 2023-10-06T05:18:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.031+0000 7f72d9d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f72d406ec30 0x7f72d414ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.031+0000 7f72d8d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f72d406b510 0x7f72d414c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35010/0 (socket says 172.21.15.31:35010) 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.031+0000 7f72d8d9a700 1 -- 172.21.15.31:0/1743525713 learned_addr learned my addr 172.21.15.31:0/1743525713 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.031+0000 7f72cbfff700 1 -- 172.21.15.31:0/1743525713 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401721793 0 0) 0x7f72d40c6df0 con 0x7f72d406b510 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.031+0000 7f72cbfff700 1 -- 172.21.15.31:0/1743525713 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c0003570 con 0x7f72d406b510 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.031+0000 7f72d9d9c700 1 -- 172.21.15.31:0/1743525713 >> v1:172.21.15.62:6789/0 conn(0x7f72d40ca770 legacy=0x7f72d406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.031+0000 7f72cbfff700 1 -- 172.21.15.31:0/1743525713 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588557422 0 0) 0x7f72d40c70d0 con 0x7f72d40ca770 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.031+0000 7f72d9d9c700 1 -- 172.21.15.31:0/1743525713 >> v1:172.21.15.31:6789/0 conn(0x7f72d406b510 legacy=0x7f72d414c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.031+0000 7f72d9d9c700 1 -- 172.21.15.31:0/1743525713 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40c73b0 con 0x7f72d406ec30 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.031+0000 7f72d9d9c700 1 --2- 172.21.15.31:0/1743525713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406ec30 0x7f72d414ca10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00af70 tx=0x7f72cc00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba21f1ad2cb302c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.032+0000 7f72cbfff700 1 -- 172.21.15.31:0/1743525713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00f070 con 0x7f72d406ec30 2023-10-06T05:18:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.032+0000 7f72cbfff700 1 -- 172.21.15.31:0/1743525713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc00a2e0 con 0x7f72d406ec30 2023-10-06T05:18:07.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.032+0000 7f72cbfff700 1 -- 172.21.15.31:0/1743525713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00cc70 con 0x7f72d406ec30 2023-10-06T05:18:07.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.032+0000 7f72db7ff700 1 -- 172.21.15.31:0/1743525713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406ec30 msgr2=0x7f72d414ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.032+0000 7f72db7ff700 1 --2- 172.21.15.31:0/1743525713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406ec30 0x7f72d414ca10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00af70 tx=0x7f72cc00a890 comp rx=0 tx=0).stop 2023-10-06T05:18:07.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.032+0000 7f72db7ff700 1 -- 172.21.15.31:0/1743525713 shutdown_connections 2023-10-06T05:18:07.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.032+0000 7f72db7ff700 1 --2- 172.21.15.31:0/1743525713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406ec30 0x7f72d414ca10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.032+0000 7f72db7ff700 1 -- 172.21.15.31:0/1743525713 >> 172.21.15.31:0/1743525713 conn(0x7f72d40cc0b0 msgr2=0x7f72d40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.032+0000 7f72db7ff700 1 -- 172.21.15.31:0/1743525713 shutdown_connections 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.032+0000 7f72db7ff700 1 -- 172.21.15.31:0/1743525713 wait complete. 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.034+0000 7f72db7ff700 1 Processor -- start 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.034+0000 7f72db7ff700 1 -- start start 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72db7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406b510 0x7f72d40b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72db7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72d406ec30 0x7f72d416a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72db7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d40ca770 0x7f72d40b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72db7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d416ad40 con 0x7f72d406b510 2023-10-06T05:18:07.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72db7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40b0580 con 0x7f72d40ca770 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72db7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40b07d0 con 0x7f72d406ec30 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72d959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406b510 0x7f72d40b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72d959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406b510 0x7f72d40b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34668/0 (socket says 172.21.15.31:34668) 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72d959b700 1 -- 172.21.15.31:0/3372916882 learned_addr learned my addr 172.21.15.31:0/3372916882 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72d9d9c700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d40ca770 0x7f72d40b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.035+0000 7f72d8d9a700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72d406ec30 0x7f72d416a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.036+0000 7f72d959b700 1 -- 172.21.15.31:0/3372916882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72d406ec30 msgr2=0x7f72d416a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.036+0000 7f72d959b700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72d406ec30 0x7f72d416a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.036+0000 7f72d959b700 1 -- 172.21.15.31:0/3372916882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d40ca770 msgr2=0x7f72d40b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.036+0000 7f72d959b700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d40ca770 0x7f72d40b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.036+0000 7f72d959b700 1 -- 172.21.15.31:0/3372916882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc00ab30 con 0x7f72d406b510 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.036+0000 7f72d959b700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406b510 0x7f72d40b1850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f72d0001f00 tx=0x7f72d0001060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.036+0000 7f72c97fa700 1 -- 172.21.15.31:0/3372916882 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0011590 con 0x7f72d406b510 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.036+0000 7f72c97fa700 1 -- 172.21.15.31:0/3372916882 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d00116f0 con 0x7f72d406b510 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.036+0000 7f72c97fa700 1 -- 172.21.15.31:0/3372916882 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0011ab0 con 0x7f72d406b510 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.037+0000 7f72db7ff700 1 -- 172.21.15.31:0/3372916882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d40b0b10 con 0x7f72d406b510 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.037+0000 7f72db7ff700 1 -- 172.21.15.31:0/3372916882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d40b1350 con 0x7f72d406b510 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.038+0000 7f72db7ff700 1 -- 172.21.15.31:0/3372916882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d40b16c0 con 0x7f72d406b510 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.039+0000 7f72c97fa700 1 -- 172.21.15.31:0/3372916882 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72d001b3b0 con 0x7f72d406b510 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.039+0000 7f72c97fa700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f72b8066f30 0x7f72b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.039+0000 7f72c97fa700 1 -- 172.21.15.31:0/3372916882 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f72d0083af0 con 0x7f72d406b510 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.040+0000 7f72d8d9a700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f72b8066f30 0x7f72b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.040+0000 7f72d8d9a700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f72b8066f30 0x7f72b80693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f72c4007fd0 tx=0x7f72c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.044+0000 7f72c97fa700 1 -- 172.21.15.31:0/3372916882 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72d0050c10 con 0x7f72d406b510 2023-10-06T05:18:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.328+0000 7f72db7ff700 1 -- 172.21.15.31:0/3372916882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f72d404c380 con 0x7f72d406b510 2023-10-06T05:18:07.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.329+0000 7f72c97fa700 1 -- 172.21.15.31:0/3372916882 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1564 (secure 0 0 0) 0x7f72d0017020 con 0x7f72d406b510 2023-10-06T05:18:07.350 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:07.350 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":86,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":32,"num_osds":8,"num_up_osds":8,"osd_up_since":1696569413,"num_in_osds":8,"osd_in_since":1696569404,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":24,"data_bytes":592674,"bytes_used":2439520256,"bytes_avail":770654593024,"bytes_total":773094113280,"write_bytes_sec":1943,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4935}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-10-06T05:17:56.072424+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T05:18:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 -- 172.21.15.31:0/3372916882 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f72b8066f30 msgr2=0x7f72b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f72b8066f30 0x7f72b80693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f72c4007fd0 tx=0x7f72c4009040 comp rx=0 tx=0).stop 2023-10-06T05:18:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 -- 172.21.15.31:0/3372916882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406b510 msgr2=0x7f72d40b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406b510 0x7f72d40b1850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f72d0001f00 tx=0x7f72d0001060 comp rx=0 tx=0).stop 2023-10-06T05:18:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 -- 172.21.15.31:0/3372916882 shutdown_connections 2023-10-06T05:18:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72d40ca770 0x7f72d40b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f72b8066f30 0x7f72b80693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72d406b510 0x7f72d40b1850 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 --2- 172.21.15.31:0/3372916882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72d406ec30 0x7f72d416a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 -- 172.21.15.31:0/3372916882 >> 172.21.15.31:0/3372916882 conn(0x7f72d40cc0b0 msgr2=0x7f72d406f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 -- 172.21.15.31:0/3372916882 shutdown_connections 2023-10-06T05:18:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.331+0000 7f72db7ff700 1 -- 172.21.15.31:0/3372916882 wait complete. 2023-10-06T05:18:07.354 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T05:18:07.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:07.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.548+0000 7f0f139ff700 1 Processor -- start 2023-10-06T05:18:07.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.548+0000 7f0f139ff700 1 -- start start 2023-10-06T05:18:07.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.549+0000 7f0f139ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0f0c13fda0 0x7f0f0c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.549+0000 7f0f139ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c0ca5b0 con 0x7f0f0c0c7b60 2023-10-06T05:18:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.549+0000 7f0f139ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c06f740 con 0x7f0f0c0c7f30 2023-10-06T05:18:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.549+0000 7f0f139ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c9c80 con 0x7f0f0c13fda0 2023-10-06T05:18:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f1179b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0f0c0c7b60 0x7f0f0c06ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35024/0 (socket says 172.21.15.31:35024) 2023-10-06T05:18:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f1179b700 1 -- 172.21.15.31:0/4272474248 learned_addr learned my addr 172.21.15.31:0/4272474248 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f11f9c700 1 --2- 172.21.15.31:0/4272474248 >> v2:172.21.15.62:3300/0 conn(0x7f0f0c13fda0 0x7f0f0c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489673002 0 0) 0x7f0f0c0ca5b0 con 0x7f0f0c0c7b60 2023-10-06T05:18:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef8003570 con 0x7f0f0c0c7b60 2023-10-06T05:18:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931083698 0 0) 0x7f0f0c06f740 con 0x7f0f0c0c7f30 2023-10-06T05:18:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f0c0ca5b0 con 0x7f0f0c0c7f30 2023-10-06T05:18:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 578914401 0 0) 0x7f0ef8003570 con 0x7f0f0c0c7b60 2023-10-06T05:18:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 >> v2:172.21.15.62:3300/0 conn(0x7f0f0c13fda0 msgr2=0x7f0f0c069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f03fff700 1 --2- 172.21.15.31:0/4272474248 >> v2:172.21.15.62:3300/0 conn(0x7f0f0c13fda0 0x7f0f0c069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 >> v1:172.21.15.62:6789/0 conn(0x7f0f0c0c7f30 legacy=0x7f0f0c13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.550+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0c9f60 con 0x7f0f0c0c7b60 2023-10-06T05:18:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f0f080015f0 con 0x7f0f0c0c7b60 2023-10-06T05:18:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 >> v1:172.21.15.31:6789/0 conn(0x7f0f0c0c7b60 legacy=0x7f0f0c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f03fff700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ef8003c40 0x7f0ef8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f03fff700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ef8004810 0x7f0ef8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f03fff700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ef8005210 0x7f0ef800dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c9c80 con 0x7f0ef8005210 2023-10-06T05:18:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef800e340 con 0x7f0ef8004810 2023-10-06T05:18:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef800e4d0 con 0x7f0ef8003c40 2023-10-06T05:18:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f11f9c700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ef8005210 0x7f0ef800dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f1179b700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ef8003c40 0x7f0ef8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f10f9a700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ef8004810 0x7f0ef8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f11f9c700 1 -- 172.21.15.31:0/4272474248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ef8003c40 msgr2=0x7f0ef8004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f11f9c700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ef8003c40 0x7f0ef8004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f11f9c700 1 -- 172.21.15.31:0/4272474248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ef8004810 msgr2=0x7f0ef8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f11f9c700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ef8004810 0x7f0ef8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.551+0000 7f0f11f9c700 1 -- 172.21.15.31:0/4272474248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef8003570 con 0x7f0ef8005210 2023-10-06T05:18:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.552+0000 7f0f11f9c700 1 -- 172.21.15.31:0/4272474248 reap_dead start 2023-10-06T05:18:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.552+0000 7f0f11f9c700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ef8005210 0x7f0ef800dc70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400e210 tx=0x7f0f0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e554f68d44bd2921 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.552+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04009ac0 con 0x7f0ef8005210 2023-10-06T05:18:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.552+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0400f040 con 0x7f0ef8005210 2023-10-06T05:18:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.552+0000 7f0f03fff700 1 -- 172.21.15.31:0/4272474248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0400ecf0 con 0x7f0ef8005210 2023-10-06T05:18:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.552+0000 7f0f139ff700 1 -- 172.21.15.31:0/4272474248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ef8005210 msgr2=0x7f0ef800dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.552+0000 7f0f139ff700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ef8005210 0x7f0ef800dc70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400e210 tx=0x7f0f0400b040 comp rx=0 tx=0).stop 2023-10-06T05:18:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.553+0000 7f0f139ff700 1 -- 172.21.15.31:0/4272474248 shutdown_connections 2023-10-06T05:18:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.553+0000 7f0f139ff700 1 --2- 172.21.15.31:0/4272474248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ef8005210 0x7f0ef800dc70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.553+0000 7f0f139ff700 1 -- 172.21.15.31:0/4272474248 >> 172.21.15.31:0/4272474248 conn(0x7f0f0c0cc0b0 msgr2=0x7f0f0c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.553+0000 7f0f139ff700 1 -- 172.21.15.31:0/4272474248 shutdown_connections 2023-10-06T05:18:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.553+0000 7f0f139ff700 1 -- 172.21.15.31:0/4272474248 wait complete. 2023-10-06T05:18:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.554+0000 7f0f139ff700 1 Processor -- start 2023-10-06T05:18:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.554+0000 7f0f139ff700 1 -- start start 2023-10-06T05:18:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.555+0000 7f0f139ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ef8005210 0x7f0f0c13de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.555+0000 7f0f139ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f0c15a6b0 0x7f0f0c0c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.555+0000 7f0f139ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f0c0ac4f0 0x7f0f0c0ac990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.555+0000 7f0f139ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c5960 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.555+0000 7f0f139ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c4260 con 0x7f0ef8005210 2023-10-06T05:18:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.555+0000 7f0f139ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0c2c30 con 0x7f0f0c0ac4f0 2023-10-06T05:18:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.555+0000 7f0f10f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f0c15a6b0 0x7f0f0c0c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.555+0000 7f0f10f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f0c15a6b0 0x7f0f0c0c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34682/0 (socket says 172.21.15.31:34682) 2023-10-06T05:18:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.555+0000 7f0f10f9a700 1 -- 172.21.15.31:0/3094062319 learned_addr learned my addr 172.21.15.31:0/3094062319 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f10f9a700 1 -- 172.21.15.31:0/3094062319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f0c0ac4f0 msgr2=0x7f0f0c0ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f10f9a700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f0c0ac4f0 0x7f0f0c0ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f10f9a700 1 -- 172.21.15.31:0/3094062319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ef8005210 msgr2=0x7f0f0c13de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f10f9a700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ef8005210 0x7f0f0c13de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f10f9a700 1 -- 172.21.15.31:0/3094062319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0c5b50 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f10f9a700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f0c15a6b0 0x7f0f0c0c5420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0efc002b50 tx=0x7f0efc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f017fa700 1 -- 172.21.15.31:0/3094062319 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc00fa00 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f017fa700 1 -- 172.21.15.31:0/3094062319 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0efc00fb80 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f139ff700 1 -- 172.21.15.31:0/3094062319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c0c5e30 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f017fa700 1 -- 172.21.15.31:0/3094062319 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc0133f0 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.556+0000 7f0f139ff700 1 -- 172.21.15.31:0/3094062319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c0c3ef0 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.557+0000 7f0f139ff700 1 -- 172.21.15.31:0/3094062319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c04c380 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.558+0000 7f0f017fa700 1 -- 172.21.15.31:0/3094062319 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0efc013760 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.560+0000 7f0f017fa700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ef0066f30 0x7f0ef00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.560+0000 7f0f017fa700 1 -- 172.21.15.31:0/3094062319 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0efc084f00 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.560+0000 7f0f1179b700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ef0066f30 0x7f0ef00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:07.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.560+0000 7f0f1179b700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ef0066f30 0x7f0ef00693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f08002d60 tx=0x7f0f08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:07.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.564+0000 7f0f017fa700 1 -- 172.21.15.31:0/3094062319 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0efc051f70 con 0x7f0f0c15a6b0 2023-10-06T05:18:07.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.763+0000 7f0f139ff700 1 -- 172.21.15.31:0/3094062319 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f0c0c43a0 con 0x7f0ef0066f30 2023-10-06T05:18:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.773+0000 7f0f017fa700 1 -- 172.21.15.31:0/3094062319 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311214 (secure 0 0 0) 0x7f0f0c0c43a0 con 0x7f0ef0066f30 2023-10-06T05:18:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 -- 172.21.15.31:0/3094062319 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ef0066f30 msgr2=0x7f0ef00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ef0066f30 0x7f0ef00693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f08002d60 tx=0x7f0f08009040 comp rx=0 tx=0).stop 2023-10-06T05:18:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 -- 172.21.15.31:0/3094062319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f0c15a6b0 msgr2=0x7f0f0c0c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f0c15a6b0 0x7f0f0c0c5420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0efc002b50 tx=0x7f0efc007d30 comp rx=0 tx=0).stop 2023-10-06T05:18:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 -- 172.21.15.31:0/3094062319 shutdown_connections 2023-10-06T05:18:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0f0c0ac4f0 0x7f0f0c0ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ef8005210 0x7f0f0c13de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0ef0066f30 0x7f0ef00693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 --2- 172.21.15.31:0/3094062319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f0c15a6b0 0x7f0f0c0c5420 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 -- 172.21.15.31:0/3094062319 >> 172.21.15.31:0/3094062319 conn(0x7f0f0c0cc0b0 msgr2=0x7f0f0c15b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 -- 172.21.15.31:0/3094062319 shutdown_connections 2023-10-06T05:18:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:07.777+0000 7f0f139ff700 1 -- 172.21.15.31:0/3094062319 wait complete. 2023-10-06T05:18:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:18:07.796 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-10-06T05:18:06.074074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382344,"kb_used_data":4040,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592376,"statfs":{"total":773094113280,"available":770654593024,"internally_reserved":0,"allocated":4136960,"data_stored":2366908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.851052"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.256952+0000","last_change":"2023-10-06T05:18:00.248587+0000","last_active":"2023-10-06T05:18:00.256952+0000","last_peered":"2023-10-06T05:18:00.256952+0000","last_clean":"2023-10-06T05:18:00.256952+0000","last_became_active":"2023-10-06T05:18:00.248347+0000","last_became_peered":"2023-10-06T05:18:00.248347+0000","last_unstale":"2023-10-06T05:18:00.256952+0000","last_undegraded":"2023-10-06T05:18:00.256952+0000","last_fullsized":"2023-10-06T05:18:00.256952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:52.838407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247823+0000","last_change":"2023-10-06T05:18:00.245720+0000","last_active":"2023-10-06T05:18:00.247823+0000","last_peered":"2023-10-06T05:18:00.247823+0000","last_clean":"2023-10-06T05:18:00.247823+0000","last_became_active":"2023-10-06T05:18:00.245480+0000","last_became_peered":"2023-10-06T05:18:00.245480+0000","last_unstale":"2023-10-06T05:18:00.247823+0000","last_undegraded":"2023-10-06T05:18:00.247823+0000","last_fullsized":"2023-10-06T05:18:00.247823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:54.805238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267755+0000","last_change":"2023-10-06T05:18:00.266938+0000","last_active":"2023-10-06T05:18:00.267755+0000","last_peered":"2023-10-06T05:18:00.267755+0000","last_clean":"2023-10-06T05:18:00.267755+0000","last_became_active":"2023-10-06T05:18:00.266629+0000","last_became_peered":"2023-10-06T05:18:00.266629+0000","last_unstale":"2023-10-06T05:18:00.267755+0000","last_undegraded":"2023-10-06T05:18:00.267755+0000","last_fullsized":"2023-10-06T05:18:00.267755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264848+0000","last_change":"2023-10-06T05:18:00.260400+0000","last_active":"2023-10-06T05:18:00.264848+0000","last_peered":"2023-10-06T05:18:00.264848+0000","last_clean":"2023-10-06T05:18:00.264848+0000","last_became_active":"2023-10-06T05:18:00.259981+0000","last_became_peered":"2023-10-06T05:18:00.259981+0000","last_unstale":"2023-10-06T05:18:00.264848+0000","last_undegraded":"2023-10-06T05:18:00.264848+0000","last_fullsized":"2023-10-06T05:18:00.264848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:31.382530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.266572+0000","last_change":"2023-10-06T05:18:00.263393+0000","last_active":"2023-10-06T05:18:00.266572+0000","last_peered":"2023-10-06T05:18:00.266572+0000","last_clean":"2023-10-06T05:18:00.266572+0000","last_became_active":"2023-10-06T05:18:00.263138+0000","last_became_peered":"2023-10-06T05:18:00.263138+0000","last_unstale":"2023-10-06T05:18:00.266572+0000","last_undegraded":"2023-10-06T05:18:00.266572+0000","last_fullsized":"2023-10-06T05:18:00.266572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:56.359498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.251146+0000","last_change":"2023-10-06T05:18:00.245969+0000","last_active":"2023-10-06T05:18:00.251146+0000","last_peered":"2023-10-06T05:18:00.251146+0000","last_clean":"2023-10-06T05:18:00.251146+0000","last_became_active":"2023-10-06T05:18:00.245686+0000","last_became_peered":"2023-10-06T05:18:00.245686+0000","last_unstale":"2023-10-06T05:18:00.251146+0000","last_undegraded":"2023-10-06T05:18:00.251146+0000","last_fullsized":"2023-10-06T05:18:00.251146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:43.620470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276255+0000","last_change":"2023-10-06T05:18:00.269398+0000","last_active":"2023-10-06T05:18:00.276255+0000","last_peered":"2023-10-06T05:18:00.276255+0000","last_clean":"2023-10-06T05:18:00.276255+0000","last_became_active":"2023-10-06T05:18:00.268716+0000","last_became_peered":"2023-10-06T05:18:00.268716+0000","last_unstale":"2023-10-06T05:18:00.276255+0000","last_undegraded":"2023-10-06T05:18:00.276255+0000","last_fullsized":"2023-10-06T05:18:00.276255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268330+0000","last_change":"2023-10-06T05:18:00.259226+0000","last_active":"2023-10-06T05:18:00.268330+0000","last_peered":"2023-10-06T05:18:00.268330+0000","last_clean":"2023-10-06T05:18:00.268330+0000","last_became_active":"2023-10-06T05:18:00.258668+0000","last_became_peered":"2023-10-06T05:18:00.258668+0000","last_unstale":"2023-10-06T05:18:00.268330+0000","last_undegraded":"2023-10-06T05:18:00.268330+0000","last_fullsized":"2023-10-06T05:18:00.268330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270755+0000","last_change":"2023-10-06T05:18:00.269432+0000","last_active":"2023-10-06T05:18:00.270755+0000","last_peered":"2023-10-06T05:18:00.270755+0000","last_clean":"2023-10-06T05:18:00.270755+0000","last_became_active":"2023-10-06T05:18:00.269160+0000","last_became_peered":"2023-10-06T05:18:00.269160+0000","last_unstale":"2023-10-06T05:18:00.270755+0000","last_undegraded":"2023-10-06T05:18:00.270755+0000","last_fullsized":"2023-10-06T05:18:00.270755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276341+0000","last_change":"2023-10-06T05:18:00.270059+0000","last_active":"2023-10-06T05:18:00.276341+0000","last_peered":"2023-10-06T05:18:00.276341+0000","last_clean":"2023-10-06T05:18:00.276341+0000","last_became_active":"2023-10-06T05:18:00.269731+0000","last_became_peered":"2023-10-06T05:18:00.269731+0000","last_unstale":"2023-10-06T05:18:00.276341+0000","last_undegraded":"2023-10-06T05:18:00.276341+0000","last_fullsized":"2023-10-06T05:18:00.276341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:39.723364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276464+0000","last_change":"2023-10-06T05:18:00.271238+0000","last_active":"2023-10-06T05:18:00.276464+0000","last_peered":"2023-10-06T05:18:00.276464+0000","last_clean":"2023-10-06T05:18:00.276464+0000","last_became_active":"2023-10-06T05:18:00.270924+0000","last_became_peered":"2023-10-06T05:18:00.270924+0000","last_unstale":"2023-10-06T05:18:00.276464+0000","last_undegraded":"2023-10-06T05:18:00.276464+0000","last_fullsized":"2023-10-06T05:18:00.276464+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:59.068875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.256429+0000","last_change":"2023-10-06T05:18:00.252619+0000","last_active":"2023-10-06T05:18:00.256429+0000","last_peered":"2023-10-06T05:18:00.256429+0000","last_clean":"2023-10-06T05:18:00.256429+0000","last_became_active":"2023-10-06T05:18:00.252356+0000","last_became_peered":"2023-10-06T05:18:00.252356+0000","last_unstale":"2023-10-06T05:18:00.256429+0000","last_undegraded":"2023-10-06T05:18:00.256429+0000","last_fullsized":"2023-10-06T05:18:00.256429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:51.085783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276386+0000","last_change":"2023-10-06T05:18:00.270371+0000","last_active":"2023-10-06T05:18:00.276386+0000","last_peered":"2023-10-06T05:18:00.276386+0000","last_clean":"2023-10-06T05:18:00.276386+0000","last_became_active":"2023-10-06T05:18:00.270057+0000","last_became_peered":"2023-10-06T05:18:00.270057+0000","last_unstale":"2023-10-06T05:18:00.276386+0000","last_undegraded":"2023-10-06T05:18:00.276386+0000","last_fullsized":"2023-10-06T05:18:00.276386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:09.782005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267999+0000","last_change":"2023-10-06T05:18:00.257240+0000","last_active":"2023-10-06T05:18:00.267999+0000","last_peered":"2023-10-06T05:18:00.267999+0000","last_clean":"2023-10-06T05:18:00.267999+0000","last_became_active":"2023-10-06T05:18:00.256724+0000","last_became_peered":"2023-10-06T05:18:00.256724+0000","last_unstale":"2023-10-06T05:18:00.267999+0000","last_undegraded":"2023-10-06T05:18:00.267999+0000","last_fullsized":"2023-10-06T05:18:00.267999+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276488+0000","last_change":"2023-10-06T05:18:00.271414+0000","last_active":"2023-10-06T05:18:00.276488+0000","last_peered":"2023-10-06T05:18:00.276488+0000","last_clean":"2023-10-06T05:18:00.276488+0000","last_became_active":"2023-10-06T05:18:00.269313+0000","last_became_peered":"2023-10-06T05:18:00.269313+0000","last_unstale":"2023-10-06T05:18:00.276488+0000","last_undegraded":"2023-10-06T05:18:00.276488+0000","last_fullsized":"2023-10-06T05:18:00.276488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:14.311697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.273450+0000","last_change":"2023-10-06T05:18:00.271086+0000","last_active":"2023-10-06T05:18:00.273450+0000","last_peered":"2023-10-06T05:18:00.273450+0000","last_clean":"2023-10-06T05:18:00.273450+0000","last_became_active":"2023-10-06T05:18:00.270779+0000","last_became_peered":"2023-10-06T05:18:00.270779+0000","last_unstale":"2023-10-06T05:18:00.273450+0000","last_undegraded":"2023-10-06T05:18:00.273450+0000","last_fullsized":"2023-10-06T05:18:00.273450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.278360+0000","last_change":"2023-10-06T05:18:00.274034+0000","last_active":"2023-10-06T05:18:00.278360+0000","last_peered":"2023-10-06T05:18:00.278360+0000","last_clean":"2023-10-06T05:18:00.278360+0000","last_became_active":"2023-10-06T05:18:00.273761+0000","last_became_peered":"2023-10-06T05:18:00.273761+0000","last_unstale":"2023-10-06T05:18:00.278360+0000","last_undegraded":"2023-10-06T05:18:00.278360+0000","last_fullsized":"2023-10-06T05:18:00.278360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.266117+0000","last_change":"2023-10-06T05:18:00.257811+0000","last_active":"2023-10-06T05:18:00.266117+0000","last_peered":"2023-10-06T05:18:00.266117+0000","last_clean":"2023-10-06T05:18:00.266117+0000","last_became_active":"2023-10-06T05:18:00.257397+0000","last_became_peered":"2023-10-06T05:18:00.257397+0000","last_unstale":"2023-10-06T05:18:00.266117+0000","last_undegraded":"2023-10-06T05:18:00.266117+0000","last_fullsized":"2023-10-06T05:18:00.266117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:05.643009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264562+0000","last_change":"2023-10-06T05:18:00.253362+0000","last_active":"2023-10-06T05:18:00.264562+0000","last_peered":"2023-10-06T05:18:00.264562+0000","last_clean":"2023-10-06T05:18:00.264562+0000","last_became_active":"2023-10-06T05:18:00.253099+0000","last_became_peered":"2023-10-06T05:18:00.253099+0000","last_unstale":"2023-10-06T05:18:00.264562+0000","last_undegraded":"2023-10-06T05:18:00.264562+0000","last_fullsized":"2023-10-06T05:18:00.264562+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:43.058266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264993+0000","last_change":"2023-10-06T05:18:00.260492+0000","last_active":"2023-10-06T05:18:00.264993+0000","last_peered":"2023-10-06T05:18:00.264993+0000","last_clean":"2023-10-06T05:18:00.264993+0000","last_became_active":"2023-10-06T05:18:00.259904+0000","last_became_peered":"2023-10-06T05:18:00.259904+0000","last_unstale":"2023-10-06T05:18:00.264993+0000","last_undegraded":"2023-10-06T05:18:00.264993+0000","last_fullsized":"2023-10-06T05:18:00.264993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:31.382530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276229+0000","last_change":"2023-10-06T05:18:00.269188+0000","last_active":"2023-10-06T05:18:00.276229+0000","last_peered":"2023-10-06T05:18:00.276229+0000","last_clean":"2023-10-06T05:18:00.276229+0000","last_became_active":"2023-10-06T05:18:00.268797+0000","last_became_peered":"2023-10-06T05:18:00.268797+0000","last_unstale":"2023-10-06T05:18:00.276229+0000","last_undegraded":"2023-10-06T05:18:00.276229+0000","last_fullsized":"2023-10-06T05:18:00.276229+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268053+0000","last_change":"2023-10-06T05:18:00.259932+0000","last_active":"2023-10-06T05:18:00.268053+0000","last_peered":"2023-10-06T05:18:00.268053+0000","last_clean":"2023-10-06T05:18:00.268053+0000","last_became_active":"2023-10-06T05:18:00.259624+0000","last_became_peered":"2023-10-06T05:18:00.259624+0000","last_unstale":"2023-10-06T05:18:00.268053+0000","last_undegraded":"2023-10-06T05:18:00.268053+0000","last_fullsized":"2023-10-06T05:18:00.268053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:52.838407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.271986+0000","last_change":"2023-10-06T05:18:00.266626+0000","last_active":"2023-10-06T05:18:00.271986+0000","last_peered":"2023-10-06T05:18:00.271986+0000","last_clean":"2023-10-06T05:18:00.271986+0000","last_became_active":"2023-10-06T05:18:00.266309+0000","last_became_peered":"2023-10-06T05:18:00.266309+0000","last_unstale":"2023-10-06T05:18:00.271986+0000","last_undegraded":"2023-10-06T05:18:00.271986+0000","last_fullsized":"2023-10-06T05:18:00.271986+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:22.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264747+0000","last_change":"2023-10-06T05:18:00.261859+0000","last_active":"2023-10-06T05:18:00.264747+0000","last_peered":"2023-10-06T05:18:00.264747+0000","last_clean":"2023-10-06T05:18:00.264747+0000","last_became_active":"2023-10-06T05:18:00.257780+0000","last_became_peered":"2023-10-06T05:18:00.257780+0000","last_unstale":"2023-10-06T05:18:00.264747+0000","last_undegraded":"2023-10-06T05:18:00.264747+0000","last_fullsized":"2023-10-06T05:18:00.264747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:57.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258110+0000","last_change":"2023-10-06T05:18:00.252247+0000","last_active":"2023-10-06T05:18:00.258110+0000","last_peered":"2023-10-06T05:18:00.258110+0000","last_clean":"2023-10-06T05:18:00.258110+0000","last_became_active":"2023-10-06T05:18:00.251944+0000","last_became_peered":"2023-10-06T05:18:00.251944+0000","last_unstale":"2023-10-06T05:18:00.258110+0000","last_undegraded":"2023-10-06T05:18:00.258110+0000","last_fullsized":"2023-10-06T05:18:00.258110+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:56.100599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270520+0000","last_change":"2023-10-06T05:18:00.266347+0000","last_active":"2023-10-06T05:18:00.270520+0000","last_peered":"2023-10-06T05:18:00.270520+0000","last_clean":"2023-10-06T05:18:00.270520+0000","last_became_active":"2023-10-06T05:18:00.266110+0000","last_became_peered":"2023-10-06T05:18:00.266110+0000","last_unstale":"2023-10-06T05:18:00.270520+0000","last_undegraded":"2023-10-06T05:18:00.270520+0000","last_fullsized":"2023-10-06T05:18:00.270520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:24.495094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.254068+0000","last_change":"2023-10-06T05:18:00.251450+0000","last_active":"2023-10-06T05:18:00.254068+0000","last_peered":"2023-10-06T05:18:00.254068+0000","last_clean":"2023-10-06T05:18:00.254068+0000","last_became_active":"2023-10-06T05:18:00.251179+0000","last_became_peered":"2023-10-06T05:18:00.251179+0000","last_unstale":"2023-10-06T05:18:00.254068+0000","last_undegraded":"2023-10-06T05:18:00.254068+0000","last_fullsized":"2023-10-06T05:18:00.254068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:57.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275097+0000","last_change":"2023-10-06T05:18:00.258527+0000","last_active":"2023-10-06T05:18:00.275097+0000","last_peered":"2023-10-06T05:18:00.275097+0000","last_clean":"2023-10-06T05:18:00.275097+0000","last_became_active":"2023-10-06T05:18:00.258187+0000","last_became_peered":"2023-10-06T05:18:00.258187+0000","last_unstale":"2023-10-06T05:18:00.275097+0000","last_undegraded":"2023-10-06T05:18:00.275097+0000","last_fullsized":"2023-10-06T05:18:00.275097+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258080+0000","last_change":"2023-10-06T05:18:00.253028+0000","last_active":"2023-10-06T05:18:00.258080+0000","last_peered":"2023-10-06T05:18:00.258080+0000","last_clean":"2023-10-06T05:18:00.258080+0000","last_became_active":"2023-10-06T05:18:00.252747+0000","last_became_peered":"2023-10-06T05:18:00.252747+0000","last_unstale":"2023-10-06T05:18:00.258080+0000","last_undegraded":"2023-10-06T05:18:00.258080+0000","last_fullsized":"2023-10-06T05:18:00.258080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:44.687164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.253919+0000","last_change":"2023-10-06T05:18:00.251407+0000","last_active":"2023-10-06T05:18:00.253919+0000","last_peered":"2023-10-06T05:18:00.253919+0000","last_clean":"2023-10-06T05:18:00.253919+0000","last_became_active":"2023-10-06T05:18:00.251112+0000","last_became_peered":"2023-10-06T05:18:00.251112+0000","last_unstale":"2023-10-06T05:18:00.253919+0000","last_undegraded":"2023-10-06T05:18:00.253919+0000","last_fullsized":"2023-10-06T05:18:00.253919+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:19.666509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.271966+0000","last_change":"2023-10-06T05:18:00.266564+0000","last_active":"2023-10-06T05:18:00.271966+0000","last_peered":"2023-10-06T05:18:00.271966+0000","last_clean":"2023-10-06T05:18:00.271966+0000","last_became_active":"2023-10-06T05:18:00.266255+0000","last_became_peered":"2023-10-06T05:18:00.266255+0000","last_unstale":"2023-10-06T05:18:00.271966+0000","last_undegraded":"2023-10-06T05:18:00.271966+0000","last_fullsized":"2023-10-06T05:18:00.271966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.248417+0000","last_change":"2023-10-06T05:18:00.246355+0000","last_active":"2023-10-06T05:18:00.248417+0000","last_peered":"2023-10-06T05:18:00.248417+0000","last_clean":"2023-10-06T05:18:00.248417+0000","last_became_active":"2023-10-06T05:18:00.246083+0000","last_became_peered":"2023-10-06T05:18:00.246083+0000","last_unstale":"2023-10-06T05:18:00.248417+0000","last_undegraded":"2023-10-06T05:18:00.248417+0000","last_fullsized":"2023-10-06T05:18:00.248417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:00.272247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.273676+0000","last_change":"2023-10-06T05:18:00.270787+0000","last_active":"2023-10-06T05:18:00.273676+0000","last_peered":"2023-10-06T05:18:00.273676+0000","last_clean":"2023-10-06T05:18:00.273676+0000","last_became_active":"2023-10-06T05:18:00.270455+0000","last_became_peered":"2023-10-06T05:18:00.270455+0000","last_unstale":"2023-10-06T05:18:00.273676+0000","last_undegraded":"2023-10-06T05:18:00.273676+0000","last_fullsized":"2023-10-06T05:18:00.273676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275240+0000","last_change":"2023-10-06T05:17:59.243498+0000","last_active":"2023-10-06T05:18:00.275240+0000","last_peered":"2023-10-06T05:18:00.275240+0000","last_clean":"2023-10-06T05:18:00.275240+0000","last_became_active":"2023-10-06T05:17:59.243236+0000","last_became_peered":"2023-10-06T05:17:59.243236+0000","last_unstale":"2023-10-06T05:18:00.275240+0000","last_undegraded":"2023-10-06T05:18:00.275240+0000","last_fullsized":"2023-10-06T05:18:00.275240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:05.824830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270372+0000","last_change":"2023-10-06T05:18:00.267044+0000","last_active":"2023-10-06T05:18:00.270372+0000","last_peered":"2023-10-06T05:18:00.270372+0000","last_clean":"2023-10-06T05:18:00.270372+0000","last_became_active":"2023-10-06T05:18:00.266737+0000","last_became_peered":"2023-10-06T05:18:00.266737+0000","last_unstale":"2023-10-06T05:18:00.270372+0000","last_undegraded":"2023-10-06T05:18:00.270372+0000","last_fullsized":"2023-10-06T05:18:00.270372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:06.177164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270511+0000","last_change":"2023-10-06T05:17:59.245349+0000","last_active":"2023-10-06T05:18:00.270511+0000","last_peered":"2023-10-06T05:18:00.270511+0000","last_clean":"2023-10-06T05:18:00.270511+0000","last_became_active":"2023-10-06T05:17:59.245007+0000","last_became_peered":"2023-10-06T05:17:59.245007+0000","last_unstale":"2023-10-06T05:18:00.270511+0000","last_undegraded":"2023-10-06T05:18:00.270511+0000","last_fullsized":"2023-10-06T05:18:00.270511+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:05.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267808+0000","last_change":"2023-10-06T05:18:00.257009+0000","last_active":"2023-10-06T05:18:00.267808+0000","last_peered":"2023-10-06T05:18:00.267808+0000","last_clean":"2023-10-06T05:18:00.267808+0000","last_became_active":"2023-10-06T05:18:00.256695+0000","last_became_peered":"2023-10-06T05:18:00.256695+0000","last_unstale":"2023-10-06T05:18:00.267808+0000","last_undegraded":"2023-10-06T05:18:00.267808+0000","last_fullsized":"2023-10-06T05:18:00.267808+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.273396+0000","last_change":"2023-10-06T05:17:59.237465+0000","last_active":"2023-10-06T05:18:00.273396+0000","last_peered":"2023-10-06T05:18:00.273396+0000","last_clean":"2023-10-06T05:18:00.273396+0000","last_became_active":"2023-10-06T05:17:59.237181+0000","last_became_peered":"2023-10-06T05:17:59.237181+0000","last_unstale":"2023-10-06T05:18:00.273396+0000","last_undegraded":"2023-10-06T05:18:00.273396+0000","last_fullsized":"2023-10-06T05:18:00.273396+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:16.338535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258373+0000","last_change":"2023-10-06T05:18:00.256807+0000","last_active":"2023-10-06T05:18:00.258373+0000","last_peered":"2023-10-06T05:18:00.258373+0000","last_clean":"2023-10-06T05:18:00.258373+0000","last_became_active":"2023-10-06T05:18:00.256532+0000","last_became_peered":"2023-10-06T05:18:00.256532+0000","last_unstale":"2023-10-06T05:18:00.258373+0000","last_undegraded":"2023-10-06T05:18:00.258373+0000","last_fullsized":"2023-10-06T05:18:00.258373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272849+0000","last_change":"2023-10-06T05:17:59.239309+0000","last_active":"2023-10-06T05:18:00.272849+0000","last_peered":"2023-10-06T05:18:00.272849+0000","last_clean":"2023-10-06T05:18:00.272849+0000","last_became_active":"2023-10-06T05:17:59.239008+0000","last_became_peered":"2023-10-06T05:17:59.239008+0000","last_unstale":"2023-10-06T05:18:00.272849+0000","last_undegraded":"2023-10-06T05:18:00.272849+0000","last_fullsized":"2023-10-06T05:18:00.272849+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:53.729179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.253951+0000","last_change":"2023-10-06T05:18:00.249760+0000","last_active":"2023-10-06T05:18:00.253951+0000","last_peered":"2023-10-06T05:18:00.253951+0000","last_clean":"2023-10-06T05:18:00.253951+0000","last_became_active":"2023-10-06T05:18:00.249217+0000","last_became_peered":"2023-10-06T05:18:00.249217+0000","last_unstale":"2023-10-06T05:18:00.253951+0000","last_undegraded":"2023-10-06T05:18:00.253951+0000","last_fullsized":"2023-10-06T05:18:00.253951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:22.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276296+0000","last_change":"2023-10-06T05:17:59.250696+0000","last_active":"2023-10-06T05:18:00.276296+0000","last_peered":"2023-10-06T05:18:00.276296+0000","last_clean":"2023-10-06T05:18:00.276296+0000","last_became_active":"2023-10-06T05:17:59.250383+0000","last_became_peered":"2023-10-06T05:17:59.250383+0000","last_unstale":"2023-10-06T05:18:00.276296+0000","last_undegraded":"2023-10-06T05:18:00.276296+0000","last_fullsized":"2023-10-06T05:18:00.276296+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:04.636437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276278+0000","last_change":"2023-10-06T05:18:00.269612+0000","last_active":"2023-10-06T05:18:00.276278+0000","last_peered":"2023-10-06T05:18:00.276278+0000","last_clean":"2023-10-06T05:18:00.276278+0000","last_became_active":"2023-10-06T05:18:00.268780+0000","last_became_peered":"2023-10-06T05:18:00.268780+0000","last_unstale":"2023-10-06T05:18:00.276278+0000","last_undegraded":"2023-10-06T05:18:00.276278+0000","last_fullsized":"2023-10-06T05:18:00.276278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:22.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.274334+0000","last_change":"2023-10-06T05:17:59.247904+0000","last_active":"2023-10-06T05:18:00.274334+0000","last_peered":"2023-10-06T05:18:00.274334+0000","last_clean":"2023-10-06T05:18:00.274334+0000","last_became_active":"2023-10-06T05:17:59.247645+0000","last_became_peered":"2023-10-06T05:17:59.247645+0000","last_unstale":"2023-10-06T05:18:00.274334+0000","last_undegraded":"2023-10-06T05:18:00.274334+0000","last_fullsized":"2023-10-06T05:18:00.274334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:23.102068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276341+0000","last_change":"2023-10-06T05:18:00.272334+0000","last_active":"2023-10-06T05:18:00.276341+0000","last_peered":"2023-10-06T05:18:00.276341+0000","last_clean":"2023-10-06T05:18:00.276341+0000","last_became_active":"2023-10-06T05:18:00.272051+0000","last_became_peered":"2023-10-06T05:18:00.272051+0000","last_unstale":"2023-10-06T05:18:00.276341+0000","last_undegraded":"2023-10-06T05:18:00.276341+0000","last_fullsized":"2023-10-06T05:18:00.276341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:54.805238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.236476+0000","last_change":"2023-10-06T05:17:59.248353+0000","last_active":"2023-10-06T05:18:00.236476+0000","last_peered":"2023-10-06T05:18:00.236476+0000","last_clean":"2023-10-06T05:18:00.236476+0000","last_became_active":"2023-10-06T05:17:59.247998+0000","last_became_peered":"2023-10-06T05:17:59.247998+0000","last_unstale":"2023-10-06T05:18:00.236476+0000","last_undegraded":"2023-10-06T05:18:00.236476+0000","last_fullsized":"2023-10-06T05:18:00.236476+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:25.646387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.259108+0000","last_change":"2023-10-06T05:18:00.257458+0000","last_active":"2023-10-06T05:18:00.259108+0000","last_peered":"2023-10-06T05:18:00.259108+0000","last_clean":"2023-10-06T05:18:00.259108+0000","last_became_active":"2023-10-06T05:18:00.257165+0000","last_became_peered":"2023-10-06T05:18:00.257165+0000","last_unstale":"2023-10-06T05:18:00.259108+0000","last_undegraded":"2023-10-06T05:18:00.259108+0000","last_fullsized":"2023-10-06T05:18:00.259108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:24.836220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244601+0000","last_change":"2023-10-06T05:17:59.241941+0000","last_active":"2023-10-06T05:18:00.244601+0000","last_peered":"2023-10-06T05:18:00.244601+0000","last_clean":"2023-10-06T05:18:00.244601+0000","last_became_active":"2023-10-06T05:17:59.241520+0000","last_became_peered":"2023-10-06T05:17:59.241520+0000","last_unstale":"2023-10-06T05:18:00.244601+0000","last_undegraded":"2023-10-06T05:18:00.244601+0000","last_fullsized":"2023-10-06T05:18:00.244601+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:18.915215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264800+0000","last_change":"2023-10-06T05:18:00.260233+0000","last_active":"2023-10-06T05:18:00.264800+0000","last_peered":"2023-10-06T05:18:00.264800+0000","last_clean":"2023-10-06T05:18:00.264800+0000","last_became_active":"2023-10-06T05:18:00.259546+0000","last_became_peered":"2023-10-06T05:18:00.259546+0000","last_unstale":"2023-10-06T05:18:00.264800+0000","last_undegraded":"2023-10-06T05:18:00.264800+0000","last_fullsized":"2023-10-06T05:18:00.264800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:44.687164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.242225+0000","last_change":"2023-10-06T05:17:59.240300+0000","last_active":"2023-10-06T05:18:00.242225+0000","last_peered":"2023-10-06T05:18:00.242225+0000","last_clean":"2023-10-06T05:18:00.242225+0000","last_became_active":"2023-10-06T05:17:59.240027+0000","last_became_peered":"2023-10-06T05:17:59.240027+0000","last_unstale":"2023-10-06T05:18:00.242225+0000","last_undegraded":"2023-10-06T05:18:00.242225+0000","last_fullsized":"2023-10-06T05:18:00.242225+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:23.488521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.259525+0000","last_change":"2023-10-06T05:18:00.255781+0000","last_active":"2023-10-06T05:18:00.259525+0000","last_peered":"2023-10-06T05:18:00.259525+0000","last_clean":"2023-10-06T05:18:00.259525+0000","last_became_active":"2023-10-06T05:18:00.255447+0000","last_became_peered":"2023-10-06T05:18:00.255447+0000","last_unstale":"2023-10-06T05:18:00.259525+0000","last_undegraded":"2023-10-06T05:18:00.259525+0000","last_fullsized":"2023-10-06T05:18:00.259525+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.255928+0000","last_change":"2023-10-06T05:17:59.248906+0000","last_active":"2023-10-06T05:18:00.255928+0000","last_peered":"2023-10-06T05:18:00.255928+0000","last_clean":"2023-10-06T05:18:00.255928+0000","last_became_active":"2023-10-06T05:17:59.248594+0000","last_became_peered":"2023-10-06T05:17:59.248594+0000","last_unstale":"2023-10-06T05:18:00.255928+0000","last_undegraded":"2023-10-06T05:18:00.255928+0000","last_fullsized":"2023-10-06T05:18:00.255928+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:56.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.262995+0000","last_change":"2023-10-06T05:18:00.259360+0000","last_active":"2023-10-06T05:18:00.262995+0000","last_peered":"2023-10-06T05:18:00.262995+0000","last_clean":"2023-10-06T05:18:00.262995+0000","last_became_active":"2023-10-06T05:18:00.259041+0000","last_became_peered":"2023-10-06T05:18:00.259041+0000","last_unstale":"2023-10-06T05:18:00.262995+0000","last_undegraded":"2023-10-06T05:18:00.262995+0000","last_fullsized":"2023-10-06T05:18:00.262995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275998+0000","last_change":"2023-10-06T05:17:59.251452+0000","last_active":"2023-10-06T05:18:00.275998+0000","last_peered":"2023-10-06T05:18:00.275998+0000","last_clean":"2023-10-06T05:18:00.275998+0000","last_became_active":"2023-10-06T05:17:59.251156+0000","last_became_peered":"2023-10-06T05:17:59.251156+0000","last_unstale":"2023-10-06T05:18:00.275998+0000","last_undegraded":"2023-10-06T05:18:00.275998+0000","last_fullsized":"2023-10-06T05:18:00.275998+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:55.094026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268676+0000","last_change":"2023-10-06T05:18:00.259652+0000","last_active":"2023-10-06T05:18:00.268676+0000","last_peered":"2023-10-06T05:18:00.268676+0000","last_clean":"2023-10-06T05:18:00.268676+0000","last_became_active":"2023-10-06T05:18:00.259345+0000","last_became_peered":"2023-10-06T05:18:00.259345+0000","last_unstale":"2023-10-06T05:18:00.268676+0000","last_undegraded":"2023-10-06T05:18:00.268676+0000","last_fullsized":"2023-10-06T05:18:00.268676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.279064+0000","last_change":"2023-10-06T05:17:59.250002+0000","last_active":"2023-10-06T05:18:00.279064+0000","last_peered":"2023-10-06T05:18:00.279064+0000","last_clean":"2023-10-06T05:18:00.279064+0000","last_became_active":"2023-10-06T05:17:59.249731+0000","last_became_peered":"2023-10-06T05:17:59.249731+0000","last_unstale":"2023-10-06T05:18:00.279064+0000","last_undegraded":"2023-10-06T05:18:00.279064+0000","last_fullsized":"2023-10-06T05:18:00.279064+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:08.884333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268400+0000","last_change":"2023-10-06T05:18:00.267476+0000","last_active":"2023-10-06T05:18:00.268400+0000","last_peered":"2023-10-06T05:18:00.268400+0000","last_clean":"2023-10-06T05:18:00.268400+0000","last_became_active":"2023-10-06T05:18:00.267246+0000","last_became_peered":"2023-10-06T05:18:00.267246+0000","last_unstale":"2023-10-06T05:18:00.268400+0000","last_undegraded":"2023-10-06T05:18:00.268400+0000","last_fullsized":"2023-10-06T05:18:00.268400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267608+0000","last_change":"2023-10-06T05:17:59.243961+0000","last_active":"2023-10-06T05:18:00.267608+0000","last_peered":"2023-10-06T05:18:00.267608+0000","last_clean":"2023-10-06T05:18:00.267608+0000","last_became_active":"2023-10-06T05:17:59.243642+0000","last_became_peered":"2023-10-06T05:17:59.243642+0000","last_unstale":"2023-10-06T05:18:00.267608+0000","last_undegraded":"2023-10-06T05:18:00.267608+0000","last_fullsized":"2023-10-06T05:18:00.267608+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:05.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.257564+0000","last_change":"2023-10-06T05:18:00.248610+0000","last_active":"2023-10-06T05:18:00.257564+0000","last_peered":"2023-10-06T05:18:00.257564+0000","last_clean":"2023-10-06T05:18:00.257564+0000","last_became_active":"2023-10-06T05:18:00.248327+0000","last_became_peered":"2023-10-06T05:18:00.248327+0000","last_unstale":"2023-10-06T05:18:00.257564+0000","last_undegraded":"2023-10-06T05:18:00.257564+0000","last_fullsized":"2023-10-06T05:18:00.257564+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.236677+0000","last_change":"2023-10-06T05:17:59.248279+0000","last_active":"2023-10-06T05:18:00.236677+0000","last_peered":"2023-10-06T05:18:00.236677+0000","last_clean":"2023-10-06T05:18:00.236677+0000","last_became_active":"2023-10-06T05:17:59.247982+0000","last_became_peered":"2023-10-06T05:17:59.247982+0000","last_unstale":"2023-10-06T05:18:00.236677+0000","last_undegraded":"2023-10-06T05:18:00.236677+0000","last_fullsized":"2023-10-06T05:18:00.236677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:43.680592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272025+0000","last_change":"2023-10-06T05:18:00.266361+0000","last_active":"2023-10-06T05:18:00.272025+0000","last_peered":"2023-10-06T05:18:00.272025+0000","last_clean":"2023-10-06T05:18:00.272025+0000","last_became_active":"2023-10-06T05:18:00.266063+0000","last_became_peered":"2023-10-06T05:18:00.266063+0000","last_unstale":"2023-10-06T05:18:00.272025+0000","last_undegraded":"2023-10-06T05:18:00.272025+0000","last_fullsized":"2023-10-06T05:18:00.272025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:51.085783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241370+0000","last_change":"2023-10-06T05:17:59.242838+0000","last_active":"2023-10-06T05:18:00.241370+0000","last_peered":"2023-10-06T05:18:00.241370+0000","last_clean":"2023-10-06T05:18:00.241370+0000","last_became_active":"2023-10-06T05:17:59.242560+0000","last_became_peered":"2023-10-06T05:17:59.242560+0000","last_unstale":"2023-10-06T05:18:00.241370+0000","last_undegraded":"2023-10-06T05:18:00.241370+0000","last_fullsized":"2023-10-06T05:18:00.241370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:34.986106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258218+0000","last_change":"2023-10-06T05:18:00.248546+0000","last_active":"2023-10-06T05:18:00.258218+0000","last_peered":"2023-10-06T05:18:00.258218+0000","last_clean":"2023-10-06T05:18:00.258218+0000","last_became_active":"2023-10-06T05:18:00.248280+0000","last_became_peered":"2023-10-06T05:18:00.248280+0000","last_unstale":"2023-10-06T05:18:00.258218+0000","last_undegraded":"2023-10-06T05:18:00.258218+0000","last_fullsized":"2023-10-06T05:18:00.258218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:51.687265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275494+0000","last_change":"2023-10-06T05:17:59.242977+0000","last_active":"2023-10-06T05:18:00.275494+0000","last_peered":"2023-10-06T05:18:00.275494+0000","last_clean":"2023-10-06T05:18:00.275494+0000","last_became_active":"2023-10-06T05:17:59.242700+0000","last_became_peered":"2023-10-06T05:17:59.242700+0000","last_unstale":"2023-10-06T05:18:00.275494+0000","last_undegraded":"2023-10-06T05:18:00.275494+0000","last_fullsized":"2023-10-06T05:18:00.275494+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:14.113766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258815+0000","last_change":"2023-10-06T05:18:00.256835+0000","last_active":"2023-10-06T05:18:00.258815+0000","last_peered":"2023-10-06T05:18:00.258815+0000","last_clean":"2023-10-06T05:18:00.258815+0000","last_became_active":"2023-10-06T05:18:00.256557+0000","last_became_peered":"2023-10-06T05:18:00.256557+0000","last_unstale":"2023-10-06T05:18:00.258815+0000","last_undegraded":"2023-10-06T05:18:00.258815+0000","last_fullsized":"2023-10-06T05:18:00.258815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276895+0000","last_change":"2023-10-06T05:17:59.249226+0000","last_active":"2023-10-06T05:18:00.276895+0000","last_peered":"2023-10-06T05:18:00.276895+0000","last_clean":"2023-10-06T05:18:00.276895+0000","last_became_active":"2023-10-06T05:17:59.248875+0000","last_became_peered":"2023-10-06T05:17:59.248875+0000","last_unstale":"2023-10-06T05:18:00.276895+0000","last_undegraded":"2023-10-06T05:18:00.276895+0000","last_fullsized":"2023-10-06T05:18:00.276895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:14.113766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276443+0000","last_change":"2023-10-06T05:18:00.270683+0000","last_active":"2023-10-06T05:18:00.276443+0000","last_peered":"2023-10-06T05:18:00.276443+0000","last_clean":"2023-10-06T05:18:00.276443+0000","last_became_active":"2023-10-06T05:18:00.268778+0000","last_became_peered":"2023-10-06T05:18:00.268778+0000","last_unstale":"2023-10-06T05:18:00.276443+0000","last_undegraded":"2023-10-06T05:18:00.276443+0000","last_fullsized":"2023-10-06T05:18:00.276443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276681+0000","last_change":"2023-10-06T05:17:59.249619+0000","last_active":"2023-10-06T05:18:00.276681+0000","last_peered":"2023-10-06T05:18:00.276681+0000","last_clean":"2023-10-06T05:18:00.276681+0000","last_became_active":"2023-10-06T05:17:59.249268+0000","last_became_peered":"2023-10-06T05:17:59.249268+0000","last_unstale":"2023-10-06T05:18:00.276681+0000","last_undegraded":"2023-10-06T05:18:00.276681+0000","last_fullsized":"2023-10-06T05:18:00.276681+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:50.044225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.271497+0000","last_change":"2023-10-06T05:18:00.261160+0000","last_active":"2023-10-06T05:18:00.271497+0000","last_peered":"2023-10-06T05:18:00.271497+0000","last_clean":"2023-10-06T05:18:00.271497+0000","last_became_active":"2023-10-06T05:18:00.260836+0000","last_became_peered":"2023-10-06T05:18:00.260836+0000","last_unstale":"2023-10-06T05:18:00.271497+0000","last_undegraded":"2023-10-06T05:18:00.271497+0000","last_fullsized":"2023-10-06T05:18:00.271497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272530+0000","last_change":"2023-10-06T05:17:59.245361+0000","last_active":"2023-10-06T05:18:00.272530+0000","last_peered":"2023-10-06T05:18:00.272530+0000","last_clean":"2023-10-06T05:18:00.272530+0000","last_became_active":"2023-10-06T05:17:59.245059+0000","last_became_peered":"2023-10-06T05:17:59.245059+0000","last_unstale":"2023-10-06T05:18:00.272530+0000","last_undegraded":"2023-10-06T05:18:00.272530+0000","last_fullsized":"2023-10-06T05:18:00.272530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:20.151146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.249226+0000","last_change":"2023-10-06T05:18:00.247084+0000","last_active":"2023-10-06T05:18:00.249226+0000","last_peered":"2023-10-06T05:18:00.249226+0000","last_clean":"2023-10-06T05:18:00.249226+0000","last_became_active":"2023-10-06T05:18:00.246814+0000","last_became_peered":"2023-10-06T05:18:00.246814+0000","last_unstale":"2023-10-06T05:18:00.249226+0000","last_undegraded":"2023-10-06T05:18:00.249226+0000","last_fullsized":"2023-10-06T05:18:00.249226+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:19.666509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.237231+0000","last_change":"2023-10-06T05:17:59.237296+0000","last_active":"2023-10-06T05:18:00.237231+0000","last_peered":"2023-10-06T05:18:00.237231+0000","last_clean":"2023-10-06T05:18:00.237231+0000","last_became_active":"2023-10-06T05:17:59.237025+0000","last_became_peered":"2023-10-06T05:17:59.237025+0000","last_unstale":"2023-10-06T05:18:00.237231+0000","last_undegraded":"2023-10-06T05:18:00.237231+0000","last_fullsized":"2023-10-06T05:18:00.237231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:30.375958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272053+0000","last_change":"2023-10-06T05:18:00.266960+0000","last_active":"2023-10-06T05:18:00.272053+0000","last_peered":"2023-10-06T05:18:00.272053+0000","last_clean":"2023-10-06T05:18:00.272053+0000","last_became_active":"2023-10-06T05:18:00.266690+0000","last_became_peered":"2023-10-06T05:18:00.266690+0000","last_unstale":"2023-10-06T05:18:00.272053+0000","last_undegraded":"2023-10-06T05:18:00.272053+0000","last_fullsized":"2023-10-06T05:18:00.272053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:09.782005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268511+0000","last_change":"2023-10-06T05:17:59.243307+0000","last_active":"2023-10-06T05:18:00.268511+0000","last_peered":"2023-10-06T05:18:00.268511+0000","last_clean":"2023-10-06T05:18:00.268511+0000","last_became_active":"2023-10-06T05:17:59.243006+0000","last_became_peered":"2023-10-06T05:17:59.243006+0000","last_unstale":"2023-10-06T05:18:00.268511+0000","last_undegraded":"2023-10-06T05:18:00.268511+0000","last_fullsized":"2023-10-06T05:18:00.268511+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:43.085633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264171+0000","last_change":"2023-10-06T05:18:00.260265+0000","last_active":"2023-10-06T05:18:00.264171+0000","last_peered":"2023-10-06T05:18:00.264171+0000","last_clean":"2023-10-06T05:18:00.264171+0000","last_became_active":"2023-10-06T05:18:00.259993+0000","last_became_peered":"2023-10-06T05:18:00.259993+0000","last_unstale":"2023-10-06T05:18:00.264171+0000","last_undegraded":"2023-10-06T05:18:00.264171+0000","last_fullsized":"2023-10-06T05:18:00.264171+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.240737+0000","last_change":"2023-10-06T05:17:59.242903+0000","last_active":"2023-10-06T05:18:00.240737+0000","last_peered":"2023-10-06T05:18:00.240737+0000","last_clean":"2023-10-06T05:18:00.240737+0000","last_became_active":"2023-10-06T05:17:59.242658+0000","last_became_peered":"2023-10-06T05:17:59.242658+0000","last_unstale":"2023-10-06T05:18:00.240737+0000","last_undegraded":"2023-10-06T05:18:00.240737+0000","last_fullsized":"2023-10-06T05:18:00.240737+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:39.271735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275355+0000","last_change":"2023-10-06T05:18:00.262962+0000","last_active":"2023-10-06T05:18:00.275355+0000","last_peered":"2023-10-06T05:18:00.275355+0000","last_clean":"2023-10-06T05:18:00.275355+0000","last_became_active":"2023-10-06T05:18:00.262667+0000","last_became_peered":"2023-10-06T05:18:00.262667+0000","last_unstale":"2023-10-06T05:18:00.275355+0000","last_undegraded":"2023-10-06T05:18:00.275355+0000","last_fullsized":"2023-10-06T05:18:00.275355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:51.085783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247893+0000","last_change":"2023-10-06T05:17:59.248713+0000","last_active":"2023-10-06T05:18:00.247893+0000","last_peered":"2023-10-06T05:18:00.247893+0000","last_clean":"2023-10-06T05:18:00.247893+0000","last_became_active":"2023-10-06T05:17:59.248483+0000","last_became_peered":"2023-10-06T05:17:59.248483+0000","last_unstale":"2023-10-06T05:18:00.247893+0000","last_undegraded":"2023-10-06T05:18:00.247893+0000","last_fullsized":"2023-10-06T05:18:00.247893+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:40.124236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.263773+0000","last_change":"2023-10-06T05:18:00.258121+0000","last_active":"2023-10-06T05:18:00.263773+0000","last_peered":"2023-10-06T05:18:00.263773+0000","last_clean":"2023-10-06T05:18:00.263773+0000","last_became_active":"2023-10-06T05:18:00.257865+0000","last_became_peered":"2023-10-06T05:18:00.257865+0000","last_unstale":"2023-10-06T05:18:00.263773+0000","last_undegraded":"2023-10-06T05:18:00.263773+0000","last_fullsized":"2023-10-06T05:18:00.263773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:59.262865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.240737+0000","last_change":"2023-10-06T05:17:59.242377+0000","last_active":"2023-10-06T05:18:00.240737+0000","last_peered":"2023-10-06T05:18:00.240737+0000","last_clean":"2023-10-06T05:18:00.240737+0000","last_became_active":"2023-10-06T05:17:59.242098+0000","last_became_peered":"2023-10-06T05:17:59.242098+0000","last_unstale":"2023-10-06T05:18:00.240737+0000","last_undegraded":"2023-10-06T05:18:00.240737+0000","last_fullsized":"2023-10-06T05:18:00.240737+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:52.601888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268035+0000","last_change":"2023-10-06T05:18:00.258948+0000","last_active":"2023-10-06T05:18:00.268035+0000","last_peered":"2023-10-06T05:18:00.268035+0000","last_clean":"2023-10-06T05:18:00.268035+0000","last_became_active":"2023-10-06T05:18:00.258675+0000","last_became_peered":"2023-10-06T05:18:00.258675+0000","last_unstale":"2023-10-06T05:18:00.268035+0000","last_undegraded":"2023-10-06T05:18:00.268035+0000","last_fullsized":"2023-10-06T05:18:00.268035+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244631+0000","last_change":"2023-10-06T05:17:59.238167+0000","last_active":"2023-10-06T05:18:00.244631+0000","last_peered":"2023-10-06T05:18:00.244631+0000","last_clean":"2023-10-06T05:18:00.244631+0000","last_became_active":"2023-10-06T05:17:59.237881+0000","last_became_peered":"2023-10-06T05:17:59.237881+0000","last_unstale":"2023-10-06T05:18:00.244631+0000","last_undegraded":"2023-10-06T05:18:00.244631+0000","last_fullsized":"2023-10-06T05:18:00.244631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:08.884333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.245822+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:18:00.245822+0000","last_peered":"2023-10-06T05:18:00.245822+0000","last_clean":"2023-10-06T05:18:00.245822+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:18:00.245822+0000","last_undegraded":"2023-10-06T05:18:00.245822+0000","last_fullsized":"2023-10-06T05:18:00.245822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.259572+0000","last_change":"2023-10-06T05:18:00.256434+0000","last_active":"2023-10-06T05:18:00.259572+0000","last_peered":"2023-10-06T05:18:00.259572+0000","last_clean":"2023-10-06T05:18:00.259572+0000","last_became_active":"2023-10-06T05:18:00.255363+0000","last_became_peered":"2023-10-06T05:18:00.255363+0000","last_unstale":"2023-10-06T05:18:00.259572+0000","last_undegraded":"2023-10-06T05:18:00.259572+0000","last_fullsized":"2023-10-06T05:18:00.259572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.240921+0000","last_change":"2023-10-06T05:17:59.240304+0000","last_active":"2023-10-06T05:18:00.240921+0000","last_peered":"2023-10-06T05:18:00.240921+0000","last_clean":"2023-10-06T05:18:00.240921+0000","last_became_active":"2023-10-06T05:17:59.240043+0000","last_became_peered":"2023-10-06T05:17:59.240043+0000","last_unstale":"2023-10-06T05:18:00.240921+0000","last_undegraded":"2023-10-06T05:18:00.240921+0000","last_fullsized":"2023-10-06T05:18:00.240921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:59.265675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241247+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:18:00.241247+0000","last_peered":"2023-10-06T05:18:00.241247+0000","last_clean":"2023-10-06T05:18:00.241247+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:18:00.241247+0000","last_undegraded":"2023-10-06T05:18:00.241247+0000","last_fullsized":"2023-10-06T05:18:00.241247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.269306+0000","last_change":"2023-10-06T05:18:00.265724+0000","last_active":"2023-10-06T05:18:00.269306+0000","last_peered":"2023-10-06T05:18:00.269306+0000","last_clean":"2023-10-06T05:18:00.269306+0000","last_became_active":"2023-10-06T05:18:00.265150+0000","last_became_peered":"2023-10-06T05:18:00.265150+0000","last_unstale":"2023-10-06T05:18:00.269306+0000","last_undegraded":"2023-10-06T05:18:00.269306+0000","last_fullsized":"2023-10-06T05:18:00.269306+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:09.890905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276291+0000","last_change":"2023-10-06T05:17:59.251787+0000","last_active":"2023-10-06T05:18:00.276291+0000","last_peered":"2023-10-06T05:18:00.276291+0000","last_clean":"2023-10-06T05:18:00.276291+0000","last_became_active":"2023-10-06T05:17:59.251455+0000","last_became_peered":"2023-10-06T05:17:59.251455+0000","last_unstale":"2023-10-06T05:18:00.276291+0000","last_undegraded":"2023-10-06T05:18:00.276291+0000","last_fullsized":"2023-10-06T05:18:00.276291+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:40.124236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.236583+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:18:00.236583+0000","last_peered":"2023-10-06T05:18:00.236583+0000","last_clean":"2023-10-06T05:18:00.236583+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:18:00.236583+0000","last_undegraded":"2023-10-06T05:18:00.236583+0000","last_fullsized":"2023-10-06T05:18:00.236583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247061+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:18:00.247061+0000","last_peered":"2023-10-06T05:18:00.247061+0000","last_clean":"2023-10-06T05:18:00.247061+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:18:00.247061+0000","last_undegraded":"2023-10-06T05:18:00.247061+0000","last_fullsized":"2023-10-06T05:18:00.247061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264673+0000","last_change":"2023-10-06T05:18:00.257655+0000","last_active":"2023-10-06T05:18:00.264673+0000","last_peered":"2023-10-06T05:18:00.264673+0000","last_clean":"2023-10-06T05:18:00.264673+0000","last_became_active":"2023-10-06T05:18:00.257358+0000","last_became_peered":"2023-10-06T05:18:00.257358+0000","last_unstale":"2023-10-06T05:18:00.264673+0000","last_undegraded":"2023-10-06T05:18:00.264673+0000","last_fullsized":"2023-10-06T05:18:00.264673+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:39.723364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T05:18:01.250153+0000","last_change":"2023-10-06T05:17:59.247681+0000","last_active":"2023-10-06T05:18:01.250153+0000","last_peered":"2023-10-06T05:18:01.250153+0000","last_clean":"2023-10-06T05:18:01.250153+0000","last_became_active":"2023-10-06T05:17:59.247406+0000","last_became_peered":"2023-10-06T05:17:59.247406+0000","last_unstale":"2023-10-06T05:18:01.250153+0000","last_undegraded":"2023-10-06T05:18:01.250153+0000","last_fullsized":"2023-10-06T05:18:01.250153+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:40.606016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.246267+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:18:00.246267+0000","last_peered":"2023-10-06T05:18:00.246267+0000","last_clean":"2023-10-06T05:18:00.246267+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:18:00.246267+0000","last_undegraded":"2023-10-06T05:18:00.246267+0000","last_fullsized":"2023-10-06T05:18:00.246267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276821+0000","last_change":"2023-10-06T05:18:00.272933+0000","last_active":"2023-10-06T05:18:00.276821+0000","last_peered":"2023-10-06T05:18:00.276821+0000","last_clean":"2023-10-06T05:18:00.276821+0000","last_became_active":"2023-10-06T05:18:00.272397+0000","last_became_peered":"2023-10-06T05:18:00.272397+0000","last_unstale":"2023-10-06T05:18:00.276821+0000","last_undegraded":"2023-10-06T05:18:00.276821+0000","last_fullsized":"2023-10-06T05:18:00.276821+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247188+0000","last_change":"2023-10-06T05:17:59.247516+0000","last_active":"2023-10-06T05:18:00.247188+0000","last_peered":"2023-10-06T05:18:00.247188+0000","last_clean":"2023-10-06T05:18:00.247188+0000","last_became_active":"2023-10-06T05:17:59.247223+0000","last_became_peered":"2023-10-06T05:17:59.247223+0000","last_unstale":"2023-10-06T05:18:00.247188+0000","last_undegraded":"2023-10-06T05:18:00.247188+0000","last_fullsized":"2023-10-06T05:18:00.247188+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.667357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.246919+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:18:00.246919+0000","last_peered":"2023-10-06T05:18:00.246919+0000","last_clean":"2023-10-06T05:18:00.246919+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:18:00.246919+0000","last_undegraded":"2023-10-06T05:18:00.246919+0000","last_fullsized":"2023-10-06T05:18:00.246919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264308+0000","last_change":"2023-10-06T05:18:00.258394+0000","last_active":"2023-10-06T05:18:00.264308+0000","last_peered":"2023-10-06T05:18:00.264308+0000","last_clean":"2023-10-06T05:18:00.264308+0000","last_became_active":"2023-10-06T05:18:00.258118+0000","last_became_peered":"2023-10-06T05:18:00.258118+0000","last_unstale":"2023-10-06T05:18:00.264308+0000","last_undegraded":"2023-10-06T05:18:00.264308+0000","last_fullsized":"2023-10-06T05:18:00.264308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244236+0000","last_change":"2023-10-06T05:17:59.241749+0000","last_active":"2023-10-06T05:18:00.244236+0000","last_peered":"2023-10-06T05:18:00.244236+0000","last_clean":"2023-10-06T05:18:00.244236+0000","last_became_active":"2023-10-06T05:17:59.241482+0000","last_became_peered":"2023-10-06T05:17:59.241482+0000","last_unstale":"2023-10-06T05:18:00.244236+0000","last_undegraded":"2023-10-06T05:18:00.244236+0000","last_fullsized":"2023-10-06T05:18:00.244236+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:59.265675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.252157+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:18:00.252157+0000","last_peered":"2023-10-06T05:18:00.252157+0000","last_clean":"2023-10-06T05:18:00.252157+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:18:00.252157+0000","last_undegraded":"2023-10-06T05:18:00.252157+0000","last_fullsized":"2023-10-06T05:18:00.252157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.260566+0000","last_change":"2023-10-06T05:18:00.254085+0000","last_active":"2023-10-06T05:18:00.260566+0000","last_peered":"2023-10-06T05:18:00.260566+0000","last_clean":"2023-10-06T05:18:00.260566+0000","last_became_active":"2023-10-06T05:18:00.253803+0000","last_became_peered":"2023-10-06T05:18:00.253803+0000","last_unstale":"2023-10-06T05:18:00.260566+0000","last_undegraded":"2023-10-06T05:18:00.260566+0000","last_fullsized":"2023-10-06T05:18:00.260566+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.673929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241666+0000","last_change":"2023-10-06T05:17:59.245152+0000","last_active":"2023-10-06T05:18:00.241666+0000","last_peered":"2023-10-06T05:18:00.241666+0000","last_clean":"2023-10-06T05:18:00.241666+0000","last_became_active":"2023-10-06T05:17:59.244828+0000","last_became_peered":"2023-10-06T05:17:59.244828+0000","last_unstale":"2023-10-06T05:18:00.241666+0000","last_undegraded":"2023-10-06T05:18:00.241666+0000","last_fullsized":"2023-10-06T05:18:00.241666+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:20.151146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241245+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:18:00.241245+0000","last_peered":"2023-10-06T05:18:00.241245+0000","last_clean":"2023-10-06T05:18:00.241245+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:18:00.241245+0000","last_undegraded":"2023-10-06T05:18:00.241245+0000","last_fullsized":"2023-10-06T05:18:00.241245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.251333+0000","last_change":"2023-10-06T05:18:00.248737+0000","last_active":"2023-10-06T05:18:00.251333+0000","last_peered":"2023-10-06T05:18:00.251333+0000","last_clean":"2023-10-06T05:18:00.251333+0000","last_became_active":"2023-10-06T05:18:00.248458+0000","last_became_peered":"2023-10-06T05:18:00.248458+0000","last_unstale":"2023-10-06T05:18:00.251333+0000","last_undegraded":"2023-10-06T05:18:00.251333+0000","last_fullsized":"2023-10-06T05:18:00.251333+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:09.782005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244107+0000","last_change":"2023-10-06T05:17:59.247105+0000","last_active":"2023-10-06T05:18:00.244107+0000","last_peered":"2023-10-06T05:18:00.244107+0000","last_clean":"2023-10-06T05:18:00.244107+0000","last_became_active":"2023-10-06T05:17:59.246878+0000","last_became_peered":"2023-10-06T05:17:59.246878+0000","last_unstale":"2023-10-06T05:18:00.244107+0000","last_undegraded":"2023-10-06T05:18:00.244107+0000","last_fullsized":"2023-10-06T05:18:00.244107+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:23.488521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241510+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:18:00.241510+0000","last_peered":"2023-10-06T05:18:00.241510+0000","last_clean":"2023-10-06T05:18:00.241510+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:18:00.241510+0000","last_undegraded":"2023-10-06T05:18:00.241510+0000","last_fullsized":"2023-10-06T05:18:00.241510+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":739669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.83099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":148249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.872}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.754}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.94599999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.0129999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":0.91500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.99299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":738153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.439},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.84899999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.745}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":147729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.371},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.33900000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.434}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.83799999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":1.0700000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.03}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":147765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.64000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":149265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.875}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.84699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.752},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.78900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":148271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.77200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.753}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":7.742,"5min":7.742,"15min":7.742},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.83099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":147807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2780000000000005,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.80300000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":9.25,"5min":9.25,"15min":9.25},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.108000000000001,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":10.384,"5min":10.384,"15min":10.384},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":9.1699999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.90500000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.205,"5min":10.205,"15min":10.205},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.70999999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.98099999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.1000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.056},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.83999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:11.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.217+0000 7fdd1e5ff700 1 Processor -- start 2023-10-06T05:18:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.217+0000 7fdd1e5ff700 1 -- start start 2023-10-06T05:18:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.218+0000 7fdd1e5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd180c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.218+0000 7fdd1e5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd180b01b0 0x7fdd180b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.218+0000 7fdd1e5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd18069170 con 0x7fdd180cae30 2023-10-06T05:18:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.218+0000 7fdd1e5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd180685f0 con 0x7fdd180b01b0 2023-10-06T05:18:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.218+0000 7fdd1e5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180b0b70 con 0x7fdd180cb200 2023-10-06T05:18:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd177fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd180c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd1cb9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd180b01b0 0x7fdd180b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd177fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd180c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:38384/0 (socket says 172.21.15.31:38384) 2023-10-06T05:18:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd177fe700 1 -- 172.21.15.31:0/483961988 learned_addr learned my addr 172.21.15.31:0/483961988 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd1cb9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd180b01b0 0x7fdd180b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:60394/0 (socket says 172.21.15.31:60394) 2023-10-06T05:18:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd17fff700 1 --1- 172.21.15.31:0/483961988 >> v1:172.21.15.62:6789/0 conn(0x7fdd180cae30 0x7fdd180ca090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:47464/0 (socket says 172.21.15.31:47464) 2023-10-06T05:18:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd177fe700 1 -- 172.21.15.31:0/483961988 >> v2:172.21.15.62:3300/0 conn(0x7fdd180b01b0 msgr2=0x7fdd180b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd177fe700 1 --2- 172.21.15.31:0/483961988 >> v2:172.21.15.62:3300/0 conn(0x7fdd180b01b0 0x7fdd180b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd177fe700 1 -- 172.21.15.31:0/483961988 >> v1:172.21.15.62:6789/0 conn(0x7fdd180cae30 legacy=0x7fdd180ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd177fe700 1 -- 172.21.15.31:0/483961988 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd18068840 con 0x7fdd180cb200 2023-10-06T05:18:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd1cb9c700 1 --2- 172.21.15.31:0/483961988 >> v2:172.21.15.62:3300/0 conn(0x7fdd180b01b0 0x7fdd180b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.219+0000 7fdd16ffd700 1 -- 172.21.15.31:0/483961988 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19046783 0 0) 0x7fdd18069170 con 0x7fdd180cae30 2023-10-06T05:18:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.220+0000 7fdd177fe700 1 --2- 172.21.15.31:0/483961988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd180c90a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0099b0 tx=0x7fdd0c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fb2850603af36bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.220+0000 7fdd16ffd700 1 -- 172.21.15.31:0/483961988 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c009ea0 con 0x7fdd180cb200 2023-10-06T05:18:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.220+0000 7fdd16ffd700 1 -- 172.21.15.31:0/483961988 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c00b860 con 0x7fdd180cb200 2023-10-06T05:18:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.220+0000 7fdd16ffd700 1 -- 172.21.15.31:0/483961988 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c00bc00 con 0x7fdd180cb200 2023-10-06T05:18:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.220+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/483961988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 msgr2=0x7fdd180c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.220+0000 7fdd1e5ff700 1 --2- 172.21.15.31:0/483961988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd180c90a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0099b0 tx=0x7fdd0c009250 comp rx=0 tx=0).stop 2023-10-06T05:18:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.221+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/483961988 shutdown_connections 2023-10-06T05:18:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.221+0000 7fdd1e5ff700 1 --2- 172.21.15.31:0/483961988 >> v2:172.21.15.62:3300/0 conn(0x7fdd180b01b0 0x7fdd180b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.221+0000 7fdd1e5ff700 1 --2- 172.21.15.31:0/483961988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd180c90a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.221+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/483961988 >> 172.21.15.31:0/483961988 conn(0x7fdd18087fc0 msgr2=0x7fdd180639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.221+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/483961988 shutdown_connections 2023-10-06T05:18:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.221+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/483961988 wait complete. 2023-10-06T05:18:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.222+0000 7fdd1e5ff700 1 Processor -- start 2023-10-06T05:18:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.222+0000 7fdd1e5ff700 1 -- start start 2023-10-06T05:18:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.223+0000 7fdd1e5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd180b01b0 0x7fdd18153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.223+0000 7fdd1e5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd180cae30 0x7fdd180b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.223+0000 7fdd1e5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd18069600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.223+0000 7fdd1e5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd18154050 con 0x7fdd180b01b0 2023-10-06T05:18:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.223+0000 7fdd1e5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180b3010 con 0x7fdd180cae30 2023-10-06T05:18:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.223+0000 7fdd1e5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180b32f0 con 0x7fdd180cb200 2023-10-06T05:18:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd17fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd180b01b0 0x7fdd18153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd17fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd180b01b0 0x7fdd18153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34698/0 (socket says 172.21.15.31:34698) 2023-10-06T05:18:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd17fff700 1 -- 172.21.15.31:0/1205232820 learned_addr learned my addr 172.21.15.31:0/1205232820 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd177fe700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd180cae30 0x7fdd180b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd1cb9c700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd18069600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd17fff700 1 -- 172.21.15.31:0/1205232820 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 msgr2=0x7fdd18069600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd17fff700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd18069600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd17fff700 1 -- 172.21.15.31:0/1205232820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd180cae30 msgr2=0x7fdd180b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd17fff700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd180cae30 0x7fdd180b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd17fff700 1 -- 172.21.15.31:0/1205232820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd10009a20 con 0x7fdd180b01b0 2023-10-06T05:18:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd177fe700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd180cae30 0x7fdd180b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd1cb9c700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd18069600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:11.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.224+0000 7fdd17fff700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd180b01b0 0x7fdd18153b10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c5e0 tx=0x7fdd08001030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:11.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.225+0000 7fdcfffff700 1 -- 172.21.15.31:0/1205232820 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08005ac0 con 0x7fdd180b01b0 2023-10-06T05:18:11.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.225+0000 7fdcfffff700 1 -- 172.21.15.31:0/1205232820 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd08005c20 con 0x7fdd180b01b0 2023-10-06T05:18:11.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.225+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/1205232820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c0094f0 con 0x7fdd180b01b0 2023-10-06T05:18:11.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.225+0000 7fdcfffff700 1 -- 172.21.15.31:0/1205232820 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd080070d0 con 0x7fdd180b01b0 2023-10-06T05:18:11.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.225+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/1205232820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd180b38b0 con 0x7fdd180b01b0 2023-10-06T05:18:11.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.226+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/1205232820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1804c380 con 0x7fdd180b01b0 2023-10-06T05:18:11.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.227+0000 7fdcfffff700 1 -- 172.21.15.31:0/1205232820 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd08007480 con 0x7fdd180b01b0 2023-10-06T05:18:11.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.227+0000 7fdcfffff700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd04066f30 0x7fdd040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.228+0000 7fdcfffff700 1 -- 172.21.15.31:0/1205232820 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd08083ce0 con 0x7fdd180b01b0 2023-10-06T05:18:11.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.228+0000 7fdd177fe700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd04066f30 0x7fdd040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.228+0000 7fdd177fe700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd04066f30 0x7fdd040693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0099b0 tx=0x7fdd0c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:11.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.232+0000 7fdcfffff700 1 -- 172.21.15.31:0/1205232820 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd08050d50 con 0x7fdd180b01b0 2023-10-06T05:18:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.428+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/1205232820 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd1806f880 con 0x7fdd04066f30 2023-10-06T05:18:11.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.437+0000 7fdcfffff700 1 -- 172.21.15.31:0/1205232820 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311103 (secure 0 0 0) 0x7fdd1806f880 con 0x7fdd04066f30 2023-10-06T05:18:11.443 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.441+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/1205232820 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd04066f30 msgr2=0x7fdd040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.441+0000 7fdd1e5ff700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd04066f30 0x7fdd040693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0099b0 tx=0x7fdd0c017000 comp rx=0 tx=0).stop 2023-10-06T05:18:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.441+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/1205232820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd180b01b0 msgr2=0x7fdd18153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.441+0000 7fdd1e5ff700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd180b01b0 0x7fdd18153b10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c5e0 tx=0x7fdd08001030 comp rx=0 tx=0).stop 2023-10-06T05:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.442+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/1205232820 shutdown_connections 2023-10-06T05:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.442+0000 7fdd1e5ff700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd180cb200 0x7fdd18069600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.442+0000 7fdd1e5ff700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdd04066f30 0x7fdd040693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.442+0000 7fdd1e5ff700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd180b01b0 0x7fdd18153b10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.442+0000 7fdd1e5ff700 1 --2- 172.21.15.31:0/1205232820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd180cae30 0x7fdd180b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.442+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/1205232820 >> 172.21.15.31:0/1205232820 conn(0x7fdd18087fc0 msgr2=0x7fdd1806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:11.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.442+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/1205232820 shutdown_connections 2023-10-06T05:18:11.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.442+0000 7fdd1e5ff700 1 -- 172.21.15.31:0/1205232820 wait complete. 2023-10-06T05:18:11.449 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:18:11.461 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-10-06T05:18:10.075146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382344,"kb_used_data":4040,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592376,"statfs":{"total":773094113280,"available":770654593024,"internally_reserved":0,"allocated":4136960,"data_stored":2366908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.836907"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.256952+0000","last_change":"2023-10-06T05:18:00.248587+0000","last_active":"2023-10-06T05:18:00.256952+0000","last_peered":"2023-10-06T05:18:00.256952+0000","last_clean":"2023-10-06T05:18:00.256952+0000","last_became_active":"2023-10-06T05:18:00.248347+0000","last_became_peered":"2023-10-06T05:18:00.248347+0000","last_unstale":"2023-10-06T05:18:00.256952+0000","last_undegraded":"2023-10-06T05:18:00.256952+0000","last_fullsized":"2023-10-06T05:18:00.256952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:52.838407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247823+0000","last_change":"2023-10-06T05:18:00.245720+0000","last_active":"2023-10-06T05:18:00.247823+0000","last_peered":"2023-10-06T05:18:00.247823+0000","last_clean":"2023-10-06T05:18:00.247823+0000","last_became_active":"2023-10-06T05:18:00.245480+0000","last_became_peered":"2023-10-06T05:18:00.245480+0000","last_unstale":"2023-10-06T05:18:00.247823+0000","last_undegraded":"2023-10-06T05:18:00.247823+0000","last_fullsized":"2023-10-06T05:18:00.247823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:54.805238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267755+0000","last_change":"2023-10-06T05:18:00.266938+0000","last_active":"2023-10-06T05:18:00.267755+0000","last_peered":"2023-10-06T05:18:00.267755+0000","last_clean":"2023-10-06T05:18:00.267755+0000","last_became_active":"2023-10-06T05:18:00.266629+0000","last_became_peered":"2023-10-06T05:18:00.266629+0000","last_unstale":"2023-10-06T05:18:00.267755+0000","last_undegraded":"2023-10-06T05:18:00.267755+0000","last_fullsized":"2023-10-06T05:18:00.267755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264848+0000","last_change":"2023-10-06T05:18:00.260400+0000","last_active":"2023-10-06T05:18:00.264848+0000","last_peered":"2023-10-06T05:18:00.264848+0000","last_clean":"2023-10-06T05:18:00.264848+0000","last_became_active":"2023-10-06T05:18:00.259981+0000","last_became_peered":"2023-10-06T05:18:00.259981+0000","last_unstale":"2023-10-06T05:18:00.264848+0000","last_undegraded":"2023-10-06T05:18:00.264848+0000","last_fullsized":"2023-10-06T05:18:00.264848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:31.382530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.266572+0000","last_change":"2023-10-06T05:18:00.263393+0000","last_active":"2023-10-06T05:18:00.266572+0000","last_peered":"2023-10-06T05:18:00.266572+0000","last_clean":"2023-10-06T05:18:00.266572+0000","last_became_active":"2023-10-06T05:18:00.263138+0000","last_became_peered":"2023-10-06T05:18:00.263138+0000","last_unstale":"2023-10-06T05:18:00.266572+0000","last_undegraded":"2023-10-06T05:18:00.266572+0000","last_fullsized":"2023-10-06T05:18:00.266572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:56.359498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.251146+0000","last_change":"2023-10-06T05:18:00.245969+0000","last_active":"2023-10-06T05:18:00.251146+0000","last_peered":"2023-10-06T05:18:00.251146+0000","last_clean":"2023-10-06T05:18:00.251146+0000","last_became_active":"2023-10-06T05:18:00.245686+0000","last_became_peered":"2023-10-06T05:18:00.245686+0000","last_unstale":"2023-10-06T05:18:00.251146+0000","last_undegraded":"2023-10-06T05:18:00.251146+0000","last_fullsized":"2023-10-06T05:18:00.251146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:43.620470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276255+0000","last_change":"2023-10-06T05:18:00.269398+0000","last_active":"2023-10-06T05:18:00.276255+0000","last_peered":"2023-10-06T05:18:00.276255+0000","last_clean":"2023-10-06T05:18:00.276255+0000","last_became_active":"2023-10-06T05:18:00.268716+0000","last_became_peered":"2023-10-06T05:18:00.268716+0000","last_unstale":"2023-10-06T05:18:00.276255+0000","last_undegraded":"2023-10-06T05:18:00.276255+0000","last_fullsized":"2023-10-06T05:18:00.276255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268330+0000","last_change":"2023-10-06T05:18:00.259226+0000","last_active":"2023-10-06T05:18:00.268330+0000","last_peered":"2023-10-06T05:18:00.268330+0000","last_clean":"2023-10-06T05:18:00.268330+0000","last_became_active":"2023-10-06T05:18:00.258668+0000","last_became_peered":"2023-10-06T05:18:00.258668+0000","last_unstale":"2023-10-06T05:18:00.268330+0000","last_undegraded":"2023-10-06T05:18:00.268330+0000","last_fullsized":"2023-10-06T05:18:00.268330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270755+0000","last_change":"2023-10-06T05:18:00.269432+0000","last_active":"2023-10-06T05:18:00.270755+0000","last_peered":"2023-10-06T05:18:00.270755+0000","last_clean":"2023-10-06T05:18:00.270755+0000","last_became_active":"2023-10-06T05:18:00.269160+0000","last_became_peered":"2023-10-06T05:18:00.269160+0000","last_unstale":"2023-10-06T05:18:00.270755+0000","last_undegraded":"2023-10-06T05:18:00.270755+0000","last_fullsized":"2023-10-06T05:18:00.270755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276341+0000","last_change":"2023-10-06T05:18:00.270059+0000","last_active":"2023-10-06T05:18:00.276341+0000","last_peered":"2023-10-06T05:18:00.276341+0000","last_clean":"2023-10-06T05:18:00.276341+0000","last_became_active":"2023-10-06T05:18:00.269731+0000","last_became_peered":"2023-10-06T05:18:00.269731+0000","last_unstale":"2023-10-06T05:18:00.276341+0000","last_undegraded":"2023-10-06T05:18:00.276341+0000","last_fullsized":"2023-10-06T05:18:00.276341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:39.723364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276464+0000","last_change":"2023-10-06T05:18:00.271238+0000","last_active":"2023-10-06T05:18:00.276464+0000","last_peered":"2023-10-06T05:18:00.276464+0000","last_clean":"2023-10-06T05:18:00.276464+0000","last_became_active":"2023-10-06T05:18:00.270924+0000","last_became_peered":"2023-10-06T05:18:00.270924+0000","last_unstale":"2023-10-06T05:18:00.276464+0000","last_undegraded":"2023-10-06T05:18:00.276464+0000","last_fullsized":"2023-10-06T05:18:00.276464+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:59.068875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.256429+0000","last_change":"2023-10-06T05:18:00.252619+0000","last_active":"2023-10-06T05:18:00.256429+0000","last_peered":"2023-10-06T05:18:00.256429+0000","last_clean":"2023-10-06T05:18:00.256429+0000","last_became_active":"2023-10-06T05:18:00.252356+0000","last_became_peered":"2023-10-06T05:18:00.252356+0000","last_unstale":"2023-10-06T05:18:00.256429+0000","last_undegraded":"2023-10-06T05:18:00.256429+0000","last_fullsized":"2023-10-06T05:18:00.256429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:51.085783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276386+0000","last_change":"2023-10-06T05:18:00.270371+0000","last_active":"2023-10-06T05:18:00.276386+0000","last_peered":"2023-10-06T05:18:00.276386+0000","last_clean":"2023-10-06T05:18:00.276386+0000","last_became_active":"2023-10-06T05:18:00.270057+0000","last_became_peered":"2023-10-06T05:18:00.270057+0000","last_unstale":"2023-10-06T05:18:00.276386+0000","last_undegraded":"2023-10-06T05:18:00.276386+0000","last_fullsized":"2023-10-06T05:18:00.276386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:09.782005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267999+0000","last_change":"2023-10-06T05:18:00.257240+0000","last_active":"2023-10-06T05:18:00.267999+0000","last_peered":"2023-10-06T05:18:00.267999+0000","last_clean":"2023-10-06T05:18:00.267999+0000","last_became_active":"2023-10-06T05:18:00.256724+0000","last_became_peered":"2023-10-06T05:18:00.256724+0000","last_unstale":"2023-10-06T05:18:00.267999+0000","last_undegraded":"2023-10-06T05:18:00.267999+0000","last_fullsized":"2023-10-06T05:18:00.267999+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276488+0000","last_change":"2023-10-06T05:18:00.271414+0000","last_active":"2023-10-06T05:18:00.276488+0000","last_peered":"2023-10-06T05:18:00.276488+0000","last_clean":"2023-10-06T05:18:00.276488+0000","last_became_active":"2023-10-06T05:18:00.269313+0000","last_became_peered":"2023-10-06T05:18:00.269313+0000","last_unstale":"2023-10-06T05:18:00.276488+0000","last_undegraded":"2023-10-06T05:18:00.276488+0000","last_fullsized":"2023-10-06T05:18:00.276488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:14.311697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.273450+0000","last_change":"2023-10-06T05:18:00.271086+0000","last_active":"2023-10-06T05:18:00.273450+0000","last_peered":"2023-10-06T05:18:00.273450+0000","last_clean":"2023-10-06T05:18:00.273450+0000","last_became_active":"2023-10-06T05:18:00.270779+0000","last_became_peered":"2023-10-06T05:18:00.270779+0000","last_unstale":"2023-10-06T05:18:00.273450+0000","last_undegraded":"2023-10-06T05:18:00.273450+0000","last_fullsized":"2023-10-06T05:18:00.273450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.278360+0000","last_change":"2023-10-06T05:18:00.274034+0000","last_active":"2023-10-06T05:18:00.278360+0000","last_peered":"2023-10-06T05:18:00.278360+0000","last_clean":"2023-10-06T05:18:00.278360+0000","last_became_active":"2023-10-06T05:18:00.273761+0000","last_became_peered":"2023-10-06T05:18:00.273761+0000","last_unstale":"2023-10-06T05:18:00.278360+0000","last_undegraded":"2023-10-06T05:18:00.278360+0000","last_fullsized":"2023-10-06T05:18:00.278360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.266117+0000","last_change":"2023-10-06T05:18:00.257811+0000","last_active":"2023-10-06T05:18:00.266117+0000","last_peered":"2023-10-06T05:18:00.266117+0000","last_clean":"2023-10-06T05:18:00.266117+0000","last_became_active":"2023-10-06T05:18:00.257397+0000","last_became_peered":"2023-10-06T05:18:00.257397+0000","last_unstale":"2023-10-06T05:18:00.266117+0000","last_undegraded":"2023-10-06T05:18:00.266117+0000","last_fullsized":"2023-10-06T05:18:00.266117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:05.643009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264562+0000","last_change":"2023-10-06T05:18:00.253362+0000","last_active":"2023-10-06T05:18:00.264562+0000","last_peered":"2023-10-06T05:18:00.264562+0000","last_clean":"2023-10-06T05:18:00.264562+0000","last_became_active":"2023-10-06T05:18:00.253099+0000","last_became_peered":"2023-10-06T05:18:00.253099+0000","last_unstale":"2023-10-06T05:18:00.264562+0000","last_undegraded":"2023-10-06T05:18:00.264562+0000","last_fullsized":"2023-10-06T05:18:00.264562+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:43.058266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264993+0000","last_change":"2023-10-06T05:18:00.260492+0000","last_active":"2023-10-06T05:18:00.264993+0000","last_peered":"2023-10-06T05:18:00.264993+0000","last_clean":"2023-10-06T05:18:00.264993+0000","last_became_active":"2023-10-06T05:18:00.259904+0000","last_became_peered":"2023-10-06T05:18:00.259904+0000","last_unstale":"2023-10-06T05:18:00.264993+0000","last_undegraded":"2023-10-06T05:18:00.264993+0000","last_fullsized":"2023-10-06T05:18:00.264993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:31.382530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276229+0000","last_change":"2023-10-06T05:18:00.269188+0000","last_active":"2023-10-06T05:18:00.276229+0000","last_peered":"2023-10-06T05:18:00.276229+0000","last_clean":"2023-10-06T05:18:00.276229+0000","last_became_active":"2023-10-06T05:18:00.268797+0000","last_became_peered":"2023-10-06T05:18:00.268797+0000","last_unstale":"2023-10-06T05:18:00.276229+0000","last_undegraded":"2023-10-06T05:18:00.276229+0000","last_fullsized":"2023-10-06T05:18:00.276229+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268053+0000","last_change":"2023-10-06T05:18:00.259932+0000","last_active":"2023-10-06T05:18:00.268053+0000","last_peered":"2023-10-06T05:18:00.268053+0000","last_clean":"2023-10-06T05:18:00.268053+0000","last_became_active":"2023-10-06T05:18:00.259624+0000","last_became_peered":"2023-10-06T05:18:00.259624+0000","last_unstale":"2023-10-06T05:18:00.268053+0000","last_undegraded":"2023-10-06T05:18:00.268053+0000","last_fullsized":"2023-10-06T05:18:00.268053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:52.838407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.271986+0000","last_change":"2023-10-06T05:18:00.266626+0000","last_active":"2023-10-06T05:18:00.271986+0000","last_peered":"2023-10-06T05:18:00.271986+0000","last_clean":"2023-10-06T05:18:00.271986+0000","last_became_active":"2023-10-06T05:18:00.266309+0000","last_became_peered":"2023-10-06T05:18:00.266309+0000","last_unstale":"2023-10-06T05:18:00.271986+0000","last_undegraded":"2023-10-06T05:18:00.271986+0000","last_fullsized":"2023-10-06T05:18:00.271986+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:22.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264747+0000","last_change":"2023-10-06T05:18:00.261859+0000","last_active":"2023-10-06T05:18:00.264747+0000","last_peered":"2023-10-06T05:18:00.264747+0000","last_clean":"2023-10-06T05:18:00.264747+0000","last_became_active":"2023-10-06T05:18:00.257780+0000","last_became_peered":"2023-10-06T05:18:00.257780+0000","last_unstale":"2023-10-06T05:18:00.264747+0000","last_undegraded":"2023-10-06T05:18:00.264747+0000","last_fullsized":"2023-10-06T05:18:00.264747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:57.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258110+0000","last_change":"2023-10-06T05:18:00.252247+0000","last_active":"2023-10-06T05:18:00.258110+0000","last_peered":"2023-10-06T05:18:00.258110+0000","last_clean":"2023-10-06T05:18:00.258110+0000","last_became_active":"2023-10-06T05:18:00.251944+0000","last_became_peered":"2023-10-06T05:18:00.251944+0000","last_unstale":"2023-10-06T05:18:00.258110+0000","last_undegraded":"2023-10-06T05:18:00.258110+0000","last_fullsized":"2023-10-06T05:18:00.258110+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:56.100599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270520+0000","last_change":"2023-10-06T05:18:00.266347+0000","last_active":"2023-10-06T05:18:00.270520+0000","last_peered":"2023-10-06T05:18:00.270520+0000","last_clean":"2023-10-06T05:18:00.270520+0000","last_became_active":"2023-10-06T05:18:00.266110+0000","last_became_peered":"2023-10-06T05:18:00.266110+0000","last_unstale":"2023-10-06T05:18:00.270520+0000","last_undegraded":"2023-10-06T05:18:00.270520+0000","last_fullsized":"2023-10-06T05:18:00.270520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:24.495094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.254068+0000","last_change":"2023-10-06T05:18:00.251450+0000","last_active":"2023-10-06T05:18:00.254068+0000","last_peered":"2023-10-06T05:18:00.254068+0000","last_clean":"2023-10-06T05:18:00.254068+0000","last_became_active":"2023-10-06T05:18:00.251179+0000","last_became_peered":"2023-10-06T05:18:00.251179+0000","last_unstale":"2023-10-06T05:18:00.254068+0000","last_undegraded":"2023-10-06T05:18:00.254068+0000","last_fullsized":"2023-10-06T05:18:00.254068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:57.725200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275097+0000","last_change":"2023-10-06T05:18:00.258527+0000","last_active":"2023-10-06T05:18:00.275097+0000","last_peered":"2023-10-06T05:18:00.275097+0000","last_clean":"2023-10-06T05:18:00.275097+0000","last_became_active":"2023-10-06T05:18:00.258187+0000","last_became_peered":"2023-10-06T05:18:00.258187+0000","last_unstale":"2023-10-06T05:18:00.275097+0000","last_undegraded":"2023-10-06T05:18:00.275097+0000","last_fullsized":"2023-10-06T05:18:00.275097+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258080+0000","last_change":"2023-10-06T05:18:00.253028+0000","last_active":"2023-10-06T05:18:00.258080+0000","last_peered":"2023-10-06T05:18:00.258080+0000","last_clean":"2023-10-06T05:18:00.258080+0000","last_became_active":"2023-10-06T05:18:00.252747+0000","last_became_peered":"2023-10-06T05:18:00.252747+0000","last_unstale":"2023-10-06T05:18:00.258080+0000","last_undegraded":"2023-10-06T05:18:00.258080+0000","last_fullsized":"2023-10-06T05:18:00.258080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:44.687164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.253919+0000","last_change":"2023-10-06T05:18:00.251407+0000","last_active":"2023-10-06T05:18:00.253919+0000","last_peered":"2023-10-06T05:18:00.253919+0000","last_clean":"2023-10-06T05:18:00.253919+0000","last_became_active":"2023-10-06T05:18:00.251112+0000","last_became_peered":"2023-10-06T05:18:00.251112+0000","last_unstale":"2023-10-06T05:18:00.253919+0000","last_undegraded":"2023-10-06T05:18:00.253919+0000","last_fullsized":"2023-10-06T05:18:00.253919+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:19.666509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.271966+0000","last_change":"2023-10-06T05:18:00.266564+0000","last_active":"2023-10-06T05:18:00.271966+0000","last_peered":"2023-10-06T05:18:00.271966+0000","last_clean":"2023-10-06T05:18:00.271966+0000","last_became_active":"2023-10-06T05:18:00.266255+0000","last_became_peered":"2023-10-06T05:18:00.266255+0000","last_unstale":"2023-10-06T05:18:00.271966+0000","last_undegraded":"2023-10-06T05:18:00.271966+0000","last_fullsized":"2023-10-06T05:18:00.271966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.248417+0000","last_change":"2023-10-06T05:18:00.246355+0000","last_active":"2023-10-06T05:18:00.248417+0000","last_peered":"2023-10-06T05:18:00.248417+0000","last_clean":"2023-10-06T05:18:00.248417+0000","last_became_active":"2023-10-06T05:18:00.246083+0000","last_became_peered":"2023-10-06T05:18:00.246083+0000","last_unstale":"2023-10-06T05:18:00.248417+0000","last_undegraded":"2023-10-06T05:18:00.248417+0000","last_fullsized":"2023-10-06T05:18:00.248417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:00.272247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.273676+0000","last_change":"2023-10-06T05:18:00.270787+0000","last_active":"2023-10-06T05:18:00.273676+0000","last_peered":"2023-10-06T05:18:00.273676+0000","last_clean":"2023-10-06T05:18:00.273676+0000","last_became_active":"2023-10-06T05:18:00.270455+0000","last_became_peered":"2023-10-06T05:18:00.270455+0000","last_unstale":"2023-10-06T05:18:00.273676+0000","last_undegraded":"2023-10-06T05:18:00.273676+0000","last_fullsized":"2023-10-06T05:18:00.273676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275240+0000","last_change":"2023-10-06T05:17:59.243498+0000","last_active":"2023-10-06T05:18:00.275240+0000","last_peered":"2023-10-06T05:18:00.275240+0000","last_clean":"2023-10-06T05:18:00.275240+0000","last_became_active":"2023-10-06T05:17:59.243236+0000","last_became_peered":"2023-10-06T05:17:59.243236+0000","last_unstale":"2023-10-06T05:18:00.275240+0000","last_undegraded":"2023-10-06T05:18:00.275240+0000","last_fullsized":"2023-10-06T05:18:00.275240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:05.824830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270372+0000","last_change":"2023-10-06T05:18:00.267044+0000","last_active":"2023-10-06T05:18:00.270372+0000","last_peered":"2023-10-06T05:18:00.270372+0000","last_clean":"2023-10-06T05:18:00.270372+0000","last_became_active":"2023-10-06T05:18:00.266737+0000","last_became_peered":"2023-10-06T05:18:00.266737+0000","last_unstale":"2023-10-06T05:18:00.270372+0000","last_undegraded":"2023-10-06T05:18:00.270372+0000","last_fullsized":"2023-10-06T05:18:00.270372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:06.177164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.270511+0000","last_change":"2023-10-06T05:17:59.245349+0000","last_active":"2023-10-06T05:18:00.270511+0000","last_peered":"2023-10-06T05:18:00.270511+0000","last_clean":"2023-10-06T05:18:00.270511+0000","last_became_active":"2023-10-06T05:17:59.245007+0000","last_became_peered":"2023-10-06T05:17:59.245007+0000","last_unstale":"2023-10-06T05:18:00.270511+0000","last_undegraded":"2023-10-06T05:18:00.270511+0000","last_fullsized":"2023-10-06T05:18:00.270511+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:05.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267808+0000","last_change":"2023-10-06T05:18:00.257009+0000","last_active":"2023-10-06T05:18:00.267808+0000","last_peered":"2023-10-06T05:18:00.267808+0000","last_clean":"2023-10-06T05:18:00.267808+0000","last_became_active":"2023-10-06T05:18:00.256695+0000","last_became_peered":"2023-10-06T05:18:00.256695+0000","last_unstale":"2023-10-06T05:18:00.267808+0000","last_undegraded":"2023-10-06T05:18:00.267808+0000","last_fullsized":"2023-10-06T05:18:00.267808+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:34.357320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.273396+0000","last_change":"2023-10-06T05:17:59.237465+0000","last_active":"2023-10-06T05:18:00.273396+0000","last_peered":"2023-10-06T05:18:00.273396+0000","last_clean":"2023-10-06T05:18:00.273396+0000","last_became_active":"2023-10-06T05:17:59.237181+0000","last_became_peered":"2023-10-06T05:17:59.237181+0000","last_unstale":"2023-10-06T05:18:00.273396+0000","last_undegraded":"2023-10-06T05:18:00.273396+0000","last_fullsized":"2023-10-06T05:18:00.273396+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:16.338535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258373+0000","last_change":"2023-10-06T05:18:00.256807+0000","last_active":"2023-10-06T05:18:00.258373+0000","last_peered":"2023-10-06T05:18:00.258373+0000","last_clean":"2023-10-06T05:18:00.258373+0000","last_became_active":"2023-10-06T05:18:00.256532+0000","last_became_peered":"2023-10-06T05:18:00.256532+0000","last_unstale":"2023-10-06T05:18:00.258373+0000","last_undegraded":"2023-10-06T05:18:00.258373+0000","last_fullsized":"2023-10-06T05:18:00.258373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272849+0000","last_change":"2023-10-06T05:17:59.239309+0000","last_active":"2023-10-06T05:18:00.272849+0000","last_peered":"2023-10-06T05:18:00.272849+0000","last_clean":"2023-10-06T05:18:00.272849+0000","last_became_active":"2023-10-06T05:17:59.239008+0000","last_became_peered":"2023-10-06T05:17:59.239008+0000","last_unstale":"2023-10-06T05:18:00.272849+0000","last_undegraded":"2023-10-06T05:18:00.272849+0000","last_fullsized":"2023-10-06T05:18:00.272849+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:53.729179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.253951+0000","last_change":"2023-10-06T05:18:00.249760+0000","last_active":"2023-10-06T05:18:00.253951+0000","last_peered":"2023-10-06T05:18:00.253951+0000","last_clean":"2023-10-06T05:18:00.253951+0000","last_became_active":"2023-10-06T05:18:00.249217+0000","last_became_peered":"2023-10-06T05:18:00.249217+0000","last_unstale":"2023-10-06T05:18:00.253951+0000","last_undegraded":"2023-10-06T05:18:00.253951+0000","last_fullsized":"2023-10-06T05:18:00.253951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:22.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276296+0000","last_change":"2023-10-06T05:17:59.250696+0000","last_active":"2023-10-06T05:18:00.276296+0000","last_peered":"2023-10-06T05:18:00.276296+0000","last_clean":"2023-10-06T05:18:00.276296+0000","last_became_active":"2023-10-06T05:17:59.250383+0000","last_became_peered":"2023-10-06T05:17:59.250383+0000","last_unstale":"2023-10-06T05:18:00.276296+0000","last_undegraded":"2023-10-06T05:18:00.276296+0000","last_fullsized":"2023-10-06T05:18:00.276296+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:04.636437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276278+0000","last_change":"2023-10-06T05:18:00.269612+0000","last_active":"2023-10-06T05:18:00.276278+0000","last_peered":"2023-10-06T05:18:00.276278+0000","last_clean":"2023-10-06T05:18:00.276278+0000","last_became_active":"2023-10-06T05:18:00.268780+0000","last_became_peered":"2023-10-06T05:18:00.268780+0000","last_unstale":"2023-10-06T05:18:00.276278+0000","last_undegraded":"2023-10-06T05:18:00.276278+0000","last_fullsized":"2023-10-06T05:18:00.276278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:22.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.274334+0000","last_change":"2023-10-06T05:17:59.247904+0000","last_active":"2023-10-06T05:18:00.274334+0000","last_peered":"2023-10-06T05:18:00.274334+0000","last_clean":"2023-10-06T05:18:00.274334+0000","last_became_active":"2023-10-06T05:17:59.247645+0000","last_became_peered":"2023-10-06T05:17:59.247645+0000","last_unstale":"2023-10-06T05:18:00.274334+0000","last_undegraded":"2023-10-06T05:18:00.274334+0000","last_fullsized":"2023-10-06T05:18:00.274334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:23.102068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276341+0000","last_change":"2023-10-06T05:18:00.272334+0000","last_active":"2023-10-06T05:18:00.276341+0000","last_peered":"2023-10-06T05:18:00.276341+0000","last_clean":"2023-10-06T05:18:00.276341+0000","last_became_active":"2023-10-06T05:18:00.272051+0000","last_became_peered":"2023-10-06T05:18:00.272051+0000","last_unstale":"2023-10-06T05:18:00.276341+0000","last_undegraded":"2023-10-06T05:18:00.276341+0000","last_fullsized":"2023-10-06T05:18:00.276341+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:54.805238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.236476+0000","last_change":"2023-10-06T05:17:59.248353+0000","last_active":"2023-10-06T05:18:00.236476+0000","last_peered":"2023-10-06T05:18:00.236476+0000","last_clean":"2023-10-06T05:18:00.236476+0000","last_became_active":"2023-10-06T05:17:59.247998+0000","last_became_peered":"2023-10-06T05:17:59.247998+0000","last_unstale":"2023-10-06T05:18:00.236476+0000","last_undegraded":"2023-10-06T05:18:00.236476+0000","last_fullsized":"2023-10-06T05:18:00.236476+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:25.646387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.259108+0000","last_change":"2023-10-06T05:18:00.257458+0000","last_active":"2023-10-06T05:18:00.259108+0000","last_peered":"2023-10-06T05:18:00.259108+0000","last_clean":"2023-10-06T05:18:00.259108+0000","last_became_active":"2023-10-06T05:18:00.257165+0000","last_became_peered":"2023-10-06T05:18:00.257165+0000","last_unstale":"2023-10-06T05:18:00.259108+0000","last_undegraded":"2023-10-06T05:18:00.259108+0000","last_fullsized":"2023-10-06T05:18:00.259108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:24.836220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244601+0000","last_change":"2023-10-06T05:17:59.241941+0000","last_active":"2023-10-06T05:18:00.244601+0000","last_peered":"2023-10-06T05:18:00.244601+0000","last_clean":"2023-10-06T05:18:00.244601+0000","last_became_active":"2023-10-06T05:17:59.241520+0000","last_became_peered":"2023-10-06T05:17:59.241520+0000","last_unstale":"2023-10-06T05:18:00.244601+0000","last_undegraded":"2023-10-06T05:18:00.244601+0000","last_fullsized":"2023-10-06T05:18:00.244601+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:18.915215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264800+0000","last_change":"2023-10-06T05:18:00.260233+0000","last_active":"2023-10-06T05:18:00.264800+0000","last_peered":"2023-10-06T05:18:00.264800+0000","last_clean":"2023-10-06T05:18:00.264800+0000","last_became_active":"2023-10-06T05:18:00.259546+0000","last_became_peered":"2023-10-06T05:18:00.259546+0000","last_unstale":"2023-10-06T05:18:00.264800+0000","last_undegraded":"2023-10-06T05:18:00.264800+0000","last_fullsized":"2023-10-06T05:18:00.264800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:44.687164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.242225+0000","last_change":"2023-10-06T05:17:59.240300+0000","last_active":"2023-10-06T05:18:00.242225+0000","last_peered":"2023-10-06T05:18:00.242225+0000","last_clean":"2023-10-06T05:18:00.242225+0000","last_became_active":"2023-10-06T05:17:59.240027+0000","last_became_peered":"2023-10-06T05:17:59.240027+0000","last_unstale":"2023-10-06T05:18:00.242225+0000","last_undegraded":"2023-10-06T05:18:00.242225+0000","last_fullsized":"2023-10-06T05:18:00.242225+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:23.488521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.259525+0000","last_change":"2023-10-06T05:18:00.255781+0000","last_active":"2023-10-06T05:18:00.259525+0000","last_peered":"2023-10-06T05:18:00.259525+0000","last_clean":"2023-10-06T05:18:00.259525+0000","last_became_active":"2023-10-06T05:18:00.255447+0000","last_became_peered":"2023-10-06T05:18:00.255447+0000","last_unstale":"2023-10-06T05:18:00.259525+0000","last_undegraded":"2023-10-06T05:18:00.259525+0000","last_fullsized":"2023-10-06T05:18:00.259525+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.255928+0000","last_change":"2023-10-06T05:17:59.248906+0000","last_active":"2023-10-06T05:18:00.255928+0000","last_peered":"2023-10-06T05:18:00.255928+0000","last_clean":"2023-10-06T05:18:00.255928+0000","last_became_active":"2023-10-06T05:17:59.248594+0000","last_became_peered":"2023-10-06T05:17:59.248594+0000","last_unstale":"2023-10-06T05:18:00.255928+0000","last_undegraded":"2023-10-06T05:18:00.255928+0000","last_fullsized":"2023-10-06T05:18:00.255928+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:56.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.262995+0000","last_change":"2023-10-06T05:18:00.259360+0000","last_active":"2023-10-06T05:18:00.262995+0000","last_peered":"2023-10-06T05:18:00.262995+0000","last_clean":"2023-10-06T05:18:00.262995+0000","last_became_active":"2023-10-06T05:18:00.259041+0000","last_became_peered":"2023-10-06T05:18:00.259041+0000","last_unstale":"2023-10-06T05:18:00.262995+0000","last_undegraded":"2023-10-06T05:18:00.262995+0000","last_fullsized":"2023-10-06T05:18:00.262995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275998+0000","last_change":"2023-10-06T05:17:59.251452+0000","last_active":"2023-10-06T05:18:00.275998+0000","last_peered":"2023-10-06T05:18:00.275998+0000","last_clean":"2023-10-06T05:18:00.275998+0000","last_became_active":"2023-10-06T05:17:59.251156+0000","last_became_peered":"2023-10-06T05:17:59.251156+0000","last_unstale":"2023-10-06T05:18:00.275998+0000","last_undegraded":"2023-10-06T05:18:00.275998+0000","last_fullsized":"2023-10-06T05:18:00.275998+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:55.094026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268676+0000","last_change":"2023-10-06T05:18:00.259652+0000","last_active":"2023-10-06T05:18:00.268676+0000","last_peered":"2023-10-06T05:18:00.268676+0000","last_clean":"2023-10-06T05:18:00.268676+0000","last_became_active":"2023-10-06T05:18:00.259345+0000","last_became_peered":"2023-10-06T05:18:00.259345+0000","last_unstale":"2023-10-06T05:18:00.268676+0000","last_undegraded":"2023-10-06T05:18:00.268676+0000","last_fullsized":"2023-10-06T05:18:00.268676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.279064+0000","last_change":"2023-10-06T05:17:59.250002+0000","last_active":"2023-10-06T05:18:00.279064+0000","last_peered":"2023-10-06T05:18:00.279064+0000","last_clean":"2023-10-06T05:18:00.279064+0000","last_became_active":"2023-10-06T05:17:59.249731+0000","last_became_peered":"2023-10-06T05:17:59.249731+0000","last_unstale":"2023-10-06T05:18:00.279064+0000","last_undegraded":"2023-10-06T05:18:00.279064+0000","last_fullsized":"2023-10-06T05:18:00.279064+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:08.884333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268400+0000","last_change":"2023-10-06T05:18:00.267476+0000","last_active":"2023-10-06T05:18:00.268400+0000","last_peered":"2023-10-06T05:18:00.268400+0000","last_clean":"2023-10-06T05:18:00.268400+0000","last_became_active":"2023-10-06T05:18:00.267246+0000","last_became_peered":"2023-10-06T05:18:00.267246+0000","last_unstale":"2023-10-06T05:18:00.268400+0000","last_undegraded":"2023-10-06T05:18:00.268400+0000","last_fullsized":"2023-10-06T05:18:00.268400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.267608+0000","last_change":"2023-10-06T05:17:59.243961+0000","last_active":"2023-10-06T05:18:00.267608+0000","last_peered":"2023-10-06T05:18:00.267608+0000","last_clean":"2023-10-06T05:18:00.267608+0000","last_became_active":"2023-10-06T05:17:59.243642+0000","last_became_peered":"2023-10-06T05:17:59.243642+0000","last_unstale":"2023-10-06T05:18:00.267608+0000","last_undegraded":"2023-10-06T05:18:00.267608+0000","last_fullsized":"2023-10-06T05:18:00.267608+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:05.170591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.257564+0000","last_change":"2023-10-06T05:18:00.248610+0000","last_active":"2023-10-06T05:18:00.257564+0000","last_peered":"2023-10-06T05:18:00.257564+0000","last_clean":"2023-10-06T05:18:00.257564+0000","last_became_active":"2023-10-06T05:18:00.248327+0000","last_became_peered":"2023-10-06T05:18:00.248327+0000","last_unstale":"2023-10-06T05:18:00.257564+0000","last_undegraded":"2023-10-06T05:18:00.257564+0000","last_fullsized":"2023-10-06T05:18:00.257564+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.236677+0000","last_change":"2023-10-06T05:17:59.248279+0000","last_active":"2023-10-06T05:18:00.236677+0000","last_peered":"2023-10-06T05:18:00.236677+0000","last_clean":"2023-10-06T05:18:00.236677+0000","last_became_active":"2023-10-06T05:17:59.247982+0000","last_became_peered":"2023-10-06T05:17:59.247982+0000","last_unstale":"2023-10-06T05:18:00.236677+0000","last_undegraded":"2023-10-06T05:18:00.236677+0000","last_fullsized":"2023-10-06T05:18:00.236677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:43.680592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272025+0000","last_change":"2023-10-06T05:18:00.266361+0000","last_active":"2023-10-06T05:18:00.272025+0000","last_peered":"2023-10-06T05:18:00.272025+0000","last_clean":"2023-10-06T05:18:00.272025+0000","last_became_active":"2023-10-06T05:18:00.266063+0000","last_became_peered":"2023-10-06T05:18:00.266063+0000","last_unstale":"2023-10-06T05:18:00.272025+0000","last_undegraded":"2023-10-06T05:18:00.272025+0000","last_fullsized":"2023-10-06T05:18:00.272025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:51.085783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241370+0000","last_change":"2023-10-06T05:17:59.242838+0000","last_active":"2023-10-06T05:18:00.241370+0000","last_peered":"2023-10-06T05:18:00.241370+0000","last_clean":"2023-10-06T05:18:00.241370+0000","last_became_active":"2023-10-06T05:17:59.242560+0000","last_became_peered":"2023-10-06T05:17:59.242560+0000","last_unstale":"2023-10-06T05:18:00.241370+0000","last_undegraded":"2023-10-06T05:18:00.241370+0000","last_fullsized":"2023-10-06T05:18:00.241370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:34.986106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258218+0000","last_change":"2023-10-06T05:18:00.248546+0000","last_active":"2023-10-06T05:18:00.258218+0000","last_peered":"2023-10-06T05:18:00.258218+0000","last_clean":"2023-10-06T05:18:00.258218+0000","last_became_active":"2023-10-06T05:18:00.248280+0000","last_became_peered":"2023-10-06T05:18:00.248280+0000","last_unstale":"2023-10-06T05:18:00.258218+0000","last_undegraded":"2023-10-06T05:18:00.258218+0000","last_fullsized":"2023-10-06T05:18:00.258218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:51.687265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275494+0000","last_change":"2023-10-06T05:17:59.242977+0000","last_active":"2023-10-06T05:18:00.275494+0000","last_peered":"2023-10-06T05:18:00.275494+0000","last_clean":"2023-10-06T05:18:00.275494+0000","last_became_active":"2023-10-06T05:17:59.242700+0000","last_became_peered":"2023-10-06T05:17:59.242700+0000","last_unstale":"2023-10-06T05:18:00.275494+0000","last_undegraded":"2023-10-06T05:18:00.275494+0000","last_fullsized":"2023-10-06T05:18:00.275494+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:14.113766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.258815+0000","last_change":"2023-10-06T05:18:00.256835+0000","last_active":"2023-10-06T05:18:00.258815+0000","last_peered":"2023-10-06T05:18:00.258815+0000","last_clean":"2023-10-06T05:18:00.258815+0000","last_became_active":"2023-10-06T05:18:00.256557+0000","last_became_peered":"2023-10-06T05:18:00.256557+0000","last_unstale":"2023-10-06T05:18:00.258815+0000","last_undegraded":"2023-10-06T05:18:00.258815+0000","last_fullsized":"2023-10-06T05:18:00.258815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276895+0000","last_change":"2023-10-06T05:17:59.249226+0000","last_active":"2023-10-06T05:18:00.276895+0000","last_peered":"2023-10-06T05:18:00.276895+0000","last_clean":"2023-10-06T05:18:00.276895+0000","last_became_active":"2023-10-06T05:17:59.248875+0000","last_became_peered":"2023-10-06T05:17:59.248875+0000","last_unstale":"2023-10-06T05:18:00.276895+0000","last_undegraded":"2023-10-06T05:18:00.276895+0000","last_fullsized":"2023-10-06T05:18:00.276895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:14.113766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276443+0000","last_change":"2023-10-06T05:18:00.270683+0000","last_active":"2023-10-06T05:18:00.276443+0000","last_peered":"2023-10-06T05:18:00.276443+0000","last_clean":"2023-10-06T05:18:00.276443+0000","last_became_active":"2023-10-06T05:18:00.268778+0000","last_became_peered":"2023-10-06T05:18:00.268778+0000","last_unstale":"2023-10-06T05:18:00.276443+0000","last_undegraded":"2023-10-06T05:18:00.276443+0000","last_fullsized":"2023-10-06T05:18:00.276443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276681+0000","last_change":"2023-10-06T05:17:59.249619+0000","last_active":"2023-10-06T05:18:00.276681+0000","last_peered":"2023-10-06T05:18:00.276681+0000","last_clean":"2023-10-06T05:18:00.276681+0000","last_became_active":"2023-10-06T05:17:59.249268+0000","last_became_peered":"2023-10-06T05:17:59.249268+0000","last_unstale":"2023-10-06T05:18:00.276681+0000","last_undegraded":"2023-10-06T05:18:00.276681+0000","last_fullsized":"2023-10-06T05:18:00.276681+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:50.044225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.271497+0000","last_change":"2023-10-06T05:18:00.261160+0000","last_active":"2023-10-06T05:18:00.271497+0000","last_peered":"2023-10-06T05:18:00.271497+0000","last_clean":"2023-10-06T05:18:00.271497+0000","last_became_active":"2023-10-06T05:18:00.260836+0000","last_became_peered":"2023-10-06T05:18:00.260836+0000","last_unstale":"2023-10-06T05:18:00.271497+0000","last_undegraded":"2023-10-06T05:18:00.271497+0000","last_fullsized":"2023-10-06T05:18:00.271497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272530+0000","last_change":"2023-10-06T05:17:59.245361+0000","last_active":"2023-10-06T05:18:00.272530+0000","last_peered":"2023-10-06T05:18:00.272530+0000","last_clean":"2023-10-06T05:18:00.272530+0000","last_became_active":"2023-10-06T05:17:59.245059+0000","last_became_peered":"2023-10-06T05:17:59.245059+0000","last_unstale":"2023-10-06T05:18:00.272530+0000","last_undegraded":"2023-10-06T05:18:00.272530+0000","last_fullsized":"2023-10-06T05:18:00.272530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:20.151146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.249226+0000","last_change":"2023-10-06T05:18:00.247084+0000","last_active":"2023-10-06T05:18:00.249226+0000","last_peered":"2023-10-06T05:18:00.249226+0000","last_clean":"2023-10-06T05:18:00.249226+0000","last_became_active":"2023-10-06T05:18:00.246814+0000","last_became_peered":"2023-10-06T05:18:00.246814+0000","last_unstale":"2023-10-06T05:18:00.249226+0000","last_undegraded":"2023-10-06T05:18:00.249226+0000","last_fullsized":"2023-10-06T05:18:00.249226+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:19.666509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.237231+0000","last_change":"2023-10-06T05:17:59.237296+0000","last_active":"2023-10-06T05:18:00.237231+0000","last_peered":"2023-10-06T05:18:00.237231+0000","last_clean":"2023-10-06T05:18:00.237231+0000","last_became_active":"2023-10-06T05:17:59.237025+0000","last_became_peered":"2023-10-06T05:17:59.237025+0000","last_unstale":"2023-10-06T05:18:00.237231+0000","last_undegraded":"2023-10-06T05:18:00.237231+0000","last_fullsized":"2023-10-06T05:18:00.237231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:30.375958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.272053+0000","last_change":"2023-10-06T05:18:00.266960+0000","last_active":"2023-10-06T05:18:00.272053+0000","last_peered":"2023-10-06T05:18:00.272053+0000","last_clean":"2023-10-06T05:18:00.272053+0000","last_became_active":"2023-10-06T05:18:00.266690+0000","last_became_peered":"2023-10-06T05:18:00.266690+0000","last_unstale":"2023-10-06T05:18:00.272053+0000","last_undegraded":"2023-10-06T05:18:00.272053+0000","last_fullsized":"2023-10-06T05:18:00.272053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:09.782005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268511+0000","last_change":"2023-10-06T05:17:59.243307+0000","last_active":"2023-10-06T05:18:00.268511+0000","last_peered":"2023-10-06T05:18:00.268511+0000","last_clean":"2023-10-06T05:18:00.268511+0000","last_became_active":"2023-10-06T05:17:59.243006+0000","last_became_peered":"2023-10-06T05:17:59.243006+0000","last_unstale":"2023-10-06T05:18:00.268511+0000","last_undegraded":"2023-10-06T05:18:00.268511+0000","last_fullsized":"2023-10-06T05:18:00.268511+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:43.085633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264171+0000","last_change":"2023-10-06T05:18:00.260265+0000","last_active":"2023-10-06T05:18:00.264171+0000","last_peered":"2023-10-06T05:18:00.264171+0000","last_clean":"2023-10-06T05:18:00.264171+0000","last_became_active":"2023-10-06T05:18:00.259993+0000","last_became_peered":"2023-10-06T05:18:00.259993+0000","last_unstale":"2023-10-06T05:18:00.264171+0000","last_undegraded":"2023-10-06T05:18:00.264171+0000","last_fullsized":"2023-10-06T05:18:00.264171+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.834774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.240737+0000","last_change":"2023-10-06T05:17:59.242903+0000","last_active":"2023-10-06T05:18:00.240737+0000","last_peered":"2023-10-06T05:18:00.240737+0000","last_clean":"2023-10-06T05:18:00.240737+0000","last_became_active":"2023-10-06T05:17:59.242658+0000","last_became_peered":"2023-10-06T05:17:59.242658+0000","last_unstale":"2023-10-06T05:18:00.240737+0000","last_undegraded":"2023-10-06T05:18:00.240737+0000","last_fullsized":"2023-10-06T05:18:00.240737+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:39.271735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.275355+0000","last_change":"2023-10-06T05:18:00.262962+0000","last_active":"2023-10-06T05:18:00.275355+0000","last_peered":"2023-10-06T05:18:00.275355+0000","last_clean":"2023-10-06T05:18:00.275355+0000","last_became_active":"2023-10-06T05:18:00.262667+0000","last_became_peered":"2023-10-06T05:18:00.262667+0000","last_unstale":"2023-10-06T05:18:00.275355+0000","last_undegraded":"2023-10-06T05:18:00.275355+0000","last_fullsized":"2023-10-06T05:18:00.275355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:51.085783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247893+0000","last_change":"2023-10-06T05:17:59.248713+0000","last_active":"2023-10-06T05:18:00.247893+0000","last_peered":"2023-10-06T05:18:00.247893+0000","last_clean":"2023-10-06T05:18:00.247893+0000","last_became_active":"2023-10-06T05:17:59.248483+0000","last_became_peered":"2023-10-06T05:17:59.248483+0000","last_unstale":"2023-10-06T05:18:00.247893+0000","last_undegraded":"2023-10-06T05:18:00.247893+0000","last_fullsized":"2023-10-06T05:18:00.247893+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:40.124236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.263773+0000","last_change":"2023-10-06T05:18:00.258121+0000","last_active":"2023-10-06T05:18:00.263773+0000","last_peered":"2023-10-06T05:18:00.263773+0000","last_clean":"2023-10-06T05:18:00.263773+0000","last_became_active":"2023-10-06T05:18:00.257865+0000","last_became_peered":"2023-10-06T05:18:00.257865+0000","last_unstale":"2023-10-06T05:18:00.263773+0000","last_undegraded":"2023-10-06T05:18:00.263773+0000","last_fullsized":"2023-10-06T05:18:00.263773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:59.262865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.240737+0000","last_change":"2023-10-06T05:17:59.242377+0000","last_active":"2023-10-06T05:18:00.240737+0000","last_peered":"2023-10-06T05:18:00.240737+0000","last_clean":"2023-10-06T05:18:00.240737+0000","last_became_active":"2023-10-06T05:17:59.242098+0000","last_became_peered":"2023-10-06T05:17:59.242098+0000","last_unstale":"2023-10-06T05:18:00.240737+0000","last_undegraded":"2023-10-06T05:18:00.240737+0000","last_fullsized":"2023-10-06T05:18:00.240737+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:52.601888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.268035+0000","last_change":"2023-10-06T05:18:00.258948+0000","last_active":"2023-10-06T05:18:00.268035+0000","last_peered":"2023-10-06T05:18:00.268035+0000","last_clean":"2023-10-06T05:18:00.268035+0000","last_became_active":"2023-10-06T05:18:00.258675+0000","last_became_peered":"2023-10-06T05:18:00.258675+0000","last_unstale":"2023-10-06T05:18:00.268035+0000","last_undegraded":"2023-10-06T05:18:00.268035+0000","last_fullsized":"2023-10-06T05:18:00.268035+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244631+0000","last_change":"2023-10-06T05:17:59.238167+0000","last_active":"2023-10-06T05:18:00.244631+0000","last_peered":"2023-10-06T05:18:00.244631+0000","last_clean":"2023-10-06T05:18:00.244631+0000","last_became_active":"2023-10-06T05:17:59.237881+0000","last_became_peered":"2023-10-06T05:17:59.237881+0000","last_unstale":"2023-10-06T05:18:00.244631+0000","last_undegraded":"2023-10-06T05:18:00.244631+0000","last_fullsized":"2023-10-06T05:18:00.244631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:08.884333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.245822+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:18:00.245822+0000","last_peered":"2023-10-06T05:18:00.245822+0000","last_clean":"2023-10-06T05:18:00.245822+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:18:00.245822+0000","last_undegraded":"2023-10-06T05:18:00.245822+0000","last_fullsized":"2023-10-06T05:18:00.245822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.259572+0000","last_change":"2023-10-06T05:18:00.256434+0000","last_active":"2023-10-06T05:18:00.259572+0000","last_peered":"2023-10-06T05:18:00.259572+0000","last_clean":"2023-10-06T05:18:00.259572+0000","last_became_active":"2023-10-06T05:18:00.255363+0000","last_became_peered":"2023-10-06T05:18:00.255363+0000","last_unstale":"2023-10-06T05:18:00.259572+0000","last_undegraded":"2023-10-06T05:18:00.259572+0000","last_fullsized":"2023-10-06T05:18:00.259572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:40.860998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.240921+0000","last_change":"2023-10-06T05:17:59.240304+0000","last_active":"2023-10-06T05:18:00.240921+0000","last_peered":"2023-10-06T05:18:00.240921+0000","last_clean":"2023-10-06T05:18:00.240921+0000","last_became_active":"2023-10-06T05:17:59.240043+0000","last_became_peered":"2023-10-06T05:17:59.240043+0000","last_unstale":"2023-10-06T05:18:00.240921+0000","last_undegraded":"2023-10-06T05:18:00.240921+0000","last_fullsized":"2023-10-06T05:18:00.240921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:59.265675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241247+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:18:00.241247+0000","last_peered":"2023-10-06T05:18:00.241247+0000","last_clean":"2023-10-06T05:18:00.241247+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:18:00.241247+0000","last_undegraded":"2023-10-06T05:18:00.241247+0000","last_fullsized":"2023-10-06T05:18:00.241247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.269306+0000","last_change":"2023-10-06T05:18:00.265724+0000","last_active":"2023-10-06T05:18:00.269306+0000","last_peered":"2023-10-06T05:18:00.269306+0000","last_clean":"2023-10-06T05:18:00.269306+0000","last_became_active":"2023-10-06T05:18:00.265150+0000","last_became_peered":"2023-10-06T05:18:00.265150+0000","last_unstale":"2023-10-06T05:18:00.269306+0000","last_undegraded":"2023-10-06T05:18:00.269306+0000","last_fullsized":"2023-10-06T05:18:00.269306+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:09.890905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276291+0000","last_change":"2023-10-06T05:17:59.251787+0000","last_active":"2023-10-06T05:18:00.276291+0000","last_peered":"2023-10-06T05:18:00.276291+0000","last_clean":"2023-10-06T05:18:00.276291+0000","last_became_active":"2023-10-06T05:17:59.251455+0000","last_became_peered":"2023-10-06T05:17:59.251455+0000","last_unstale":"2023-10-06T05:18:00.276291+0000","last_undegraded":"2023-10-06T05:18:00.276291+0000","last_fullsized":"2023-10-06T05:18:00.276291+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:40.124236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.236583+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:18:00.236583+0000","last_peered":"2023-10-06T05:18:00.236583+0000","last_clean":"2023-10-06T05:18:00.236583+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:18:00.236583+0000","last_undegraded":"2023-10-06T05:18:00.236583+0000","last_fullsized":"2023-10-06T05:18:00.236583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247061+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:18:00.247061+0000","last_peered":"2023-10-06T05:18:00.247061+0000","last_clean":"2023-10-06T05:18:00.247061+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:18:00.247061+0000","last_undegraded":"2023-10-06T05:18:00.247061+0000","last_fullsized":"2023-10-06T05:18:00.247061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264673+0000","last_change":"2023-10-06T05:18:00.257655+0000","last_active":"2023-10-06T05:18:00.264673+0000","last_peered":"2023-10-06T05:18:00.264673+0000","last_clean":"2023-10-06T05:18:00.264673+0000","last_became_active":"2023-10-06T05:18:00.257358+0000","last_became_peered":"2023-10-06T05:18:00.257358+0000","last_unstale":"2023-10-06T05:18:00.264673+0000","last_undegraded":"2023-10-06T05:18:00.264673+0000","last_fullsized":"2023-10-06T05:18:00.264673+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:39.723364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T05:18:01.250153+0000","last_change":"2023-10-06T05:17:59.247681+0000","last_active":"2023-10-06T05:18:01.250153+0000","last_peered":"2023-10-06T05:18:01.250153+0000","last_clean":"2023-10-06T05:18:01.250153+0000","last_became_active":"2023-10-06T05:17:59.247406+0000","last_became_peered":"2023-10-06T05:17:59.247406+0000","last_unstale":"2023-10-06T05:18:01.250153+0000","last_undegraded":"2023-10-06T05:18:01.250153+0000","last_fullsized":"2023-10-06T05:18:01.250153+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:40.606016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.246267+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:18:00.246267+0000","last_peered":"2023-10-06T05:18:00.246267+0000","last_clean":"2023-10-06T05:18:00.246267+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:18:00.246267+0000","last_undegraded":"2023-10-06T05:18:00.246267+0000","last_fullsized":"2023-10-06T05:18:00.246267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.276821+0000","last_change":"2023-10-06T05:18:00.272933+0000","last_active":"2023-10-06T05:18:00.276821+0000","last_peered":"2023-10-06T05:18:00.276821+0000","last_clean":"2023-10-06T05:18:00.276821+0000","last_became_active":"2023-10-06T05:18:00.272397+0000","last_became_peered":"2023-10-06T05:18:00.272397+0000","last_unstale":"2023-10-06T05:18:00.276821+0000","last_undegraded":"2023-10-06T05:18:00.276821+0000","last_fullsized":"2023-10-06T05:18:00.276821+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:20.674448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.247188+0000","last_change":"2023-10-06T05:17:59.247516+0000","last_active":"2023-10-06T05:18:00.247188+0000","last_peered":"2023-10-06T05:18:00.247188+0000","last_clean":"2023-10-06T05:18:00.247188+0000","last_became_active":"2023-10-06T05:17:59.247223+0000","last_became_peered":"2023-10-06T05:17:59.247223+0000","last_unstale":"2023-10-06T05:18:00.247188+0000","last_undegraded":"2023-10-06T05:18:00.247188+0000","last_fullsized":"2023-10-06T05:18:00.247188+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:59.667357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.246919+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:18:00.246919+0000","last_peered":"2023-10-06T05:18:00.246919+0000","last_clean":"2023-10-06T05:18:00.246919+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:18:00.246919+0000","last_undegraded":"2023-10-06T05:18:00.246919+0000","last_fullsized":"2023-10-06T05:18:00.246919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.264308+0000","last_change":"2023-10-06T05:18:00.258394+0000","last_active":"2023-10-06T05:18:00.264308+0000","last_peered":"2023-10-06T05:18:00.264308+0000","last_clean":"2023-10-06T05:18:00.264308+0000","last_became_active":"2023-10-06T05:18:00.258118+0000","last_became_peered":"2023-10-06T05:18:00.258118+0000","last_unstale":"2023-10-06T05:18:00.264308+0000","last_undegraded":"2023-10-06T05:18:00.264308+0000","last_fullsized":"2023-10-06T05:18:00.264308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:37.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244236+0000","last_change":"2023-10-06T05:17:59.241749+0000","last_active":"2023-10-06T05:18:00.244236+0000","last_peered":"2023-10-06T05:18:00.244236+0000","last_clean":"2023-10-06T05:18:00.244236+0000","last_became_active":"2023-10-06T05:17:59.241482+0000","last_became_peered":"2023-10-06T05:17:59.241482+0000","last_unstale":"2023-10-06T05:18:00.244236+0000","last_undegraded":"2023-10-06T05:18:00.244236+0000","last_fullsized":"2023-10-06T05:18:00.244236+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:59.265675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.252157+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:18:00.252157+0000","last_peered":"2023-10-06T05:18:00.252157+0000","last_clean":"2023-10-06T05:18:00.252157+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:18:00.252157+0000","last_undegraded":"2023-10-06T05:18:00.252157+0000","last_fullsized":"2023-10-06T05:18:00.252157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.260566+0000","last_change":"2023-10-06T05:18:00.254085+0000","last_active":"2023-10-06T05:18:00.260566+0000","last_peered":"2023-10-06T05:18:00.260566+0000","last_clean":"2023-10-06T05:18:00.260566+0000","last_became_active":"2023-10-06T05:18:00.253803+0000","last_became_peered":"2023-10-06T05:18:00.253803+0000","last_unstale":"2023-10-06T05:18:00.260566+0000","last_undegraded":"2023-10-06T05:18:00.260566+0000","last_fullsized":"2023-10-06T05:18:00.260566+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.673929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241666+0000","last_change":"2023-10-06T05:17:59.245152+0000","last_active":"2023-10-06T05:18:00.241666+0000","last_peered":"2023-10-06T05:18:00.241666+0000","last_clean":"2023-10-06T05:18:00.241666+0000","last_became_active":"2023-10-06T05:17:59.244828+0000","last_became_peered":"2023-10-06T05:17:59.244828+0000","last_unstale":"2023-10-06T05:18:00.241666+0000","last_undegraded":"2023-10-06T05:18:00.241666+0000","last_fullsized":"2023-10-06T05:18:00.241666+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:20.151146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241245+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:18:00.241245+0000","last_peered":"2023-10-06T05:18:00.241245+0000","last_clean":"2023-10-06T05:18:00.241245+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:18:00.241245+0000","last_undegraded":"2023-10-06T05:18:00.241245+0000","last_fullsized":"2023-10-06T05:18:00.241245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.251333+0000","last_change":"2023-10-06T05:18:00.248737+0000","last_active":"2023-10-06T05:18:00.251333+0000","last_peered":"2023-10-06T05:18:00.251333+0000","last_clean":"2023-10-06T05:18:00.251333+0000","last_became_active":"2023-10-06T05:18:00.248458+0000","last_became_peered":"2023-10-06T05:18:00.248458+0000","last_unstale":"2023-10-06T05:18:00.251333+0000","last_undegraded":"2023-10-06T05:18:00.251333+0000","last_fullsized":"2023-10-06T05:18:00.251333+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:59.218571+0000","last_clean_scrub_stamp":"2023-10-06T05:17:59.218571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:09.782005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.244107+0000","last_change":"2023-10-06T05:17:59.247105+0000","last_active":"2023-10-06T05:18:00.244107+0000","last_peered":"2023-10-06T05:18:00.244107+0000","last_clean":"2023-10-06T05:18:00.244107+0000","last_became_active":"2023-10-06T05:17:59.246878+0000","last_became_peered":"2023-10-06T05:17:59.246878+0000","last_unstale":"2023-10-06T05:18:00.244107+0000","last_undegraded":"2023-10-06T05:18:00.244107+0000","last_fullsized":"2023-10-06T05:18:00.244107+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:17:58.211999+0000","last_clean_scrub_stamp":"2023-10-06T05:17:58.211999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:23.488521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T05:18:00.241510+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:18:00.241510+0000","last_peered":"2023-10-06T05:18:00.241510+0000","last_clean":"2023-10-06T05:18:00.241510+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:18:00.241510+0000","last_undegraded":"2023-10-06T05:18:00.241510+0000","last_fullsized":"2023-10-06T05:18:00.241510+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":739669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.88},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.95199999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.98099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.89400000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.92000000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.93600000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.96899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":148249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.70799999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.67600000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.752},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.63700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":738153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.624}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.92600000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.87},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.89200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":147729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54800000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.86499999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":147765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.502},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.55200000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":149265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.434},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.80400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":148271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.65600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.85799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":7.742,"5min":7.742,"15min":7.742},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.83799999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.91300000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":147807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2780000000000005,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.0049999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":9.25,"5min":9.25,"15min":9.25},"last":1.014},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.108000000000001,"5min":10.108000000000001,"15min":10.108000000000001},"last":1.032}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":10.384,"5min":10.384,"15min":10.384},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":9.1699999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.98399999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.205,"5min":10.205,"15min":10.205},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.99399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.85399999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.877},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.92000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:11.473 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:18:11.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:18:11.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.701+0000 7f2eea5ff700 1 Processor -- start 2023-10-06T05:18:11.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.702+0000 7f2eea5ff700 1 -- start start 2023-10-06T05:18:11.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.703+0000 7f2eea5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ee415ca40 0x7f2ee4068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.703+0000 7f2eea5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee4068da0 con 0x7f2ee40c8f40 2023-10-06T05:18:11.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.703+0000 7f2eea5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee4069080 con 0x7f2ee406a300 2023-10-06T05:18:11.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.703+0000 7f2eea5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee40cabc0 con 0x7f2ee415ca40 2023-10-06T05:18:11.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.704+0000 7f2ee3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ee415ca40 0x7f2ee4068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.704+0000 7f2ee3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ee415ca40 0x7f2ee4068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34706/0 (socket says 172.21.15.31:34706) 2023-10-06T05:18:11.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.704+0000 7f2ee3fff700 1 -- 172.21.15.31:0/1886130608 learned_addr learned my addr 172.21.15.31:0/1886130608 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:11.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.704+0000 7f2ee8b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2ee40c8f40 0x7f2ee40adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35036/0 (socket says 172.21.15.31:35036) 2023-10-06T05:18:11.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.704+0000 7f2ee3fff700 1 -- 172.21.15.31:0/1886130608 >> v1:172.21.15.62:6789/0 conn(0x7f2ee406a300 legacy=0x7f2ee40c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.704+0000 7f2ee3fff700 1 -- 172.21.15.31:0/1886130608 >> v1:172.21.15.31:6789/0 conn(0x7f2ee40c8f40 legacy=0x7f2ee40adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.704+0000 7f2ee3fff700 1 -- 172.21.15.31:0/1886130608 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4069360 con 0x7f2ee415ca40 2023-10-06T05:18:11.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.704+0000 7f2ee2ffd700 1 -- 172.21.15.31:0/1886130608 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405803893 0 0) 0x7f2ee4068da0 con 0x7f2ee40c8f40 2023-10-06T05:18:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.704+0000 7f2ee3fff700 1 --2- 172.21.15.31:0/1886130608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee415ca40 0x7f2ee4068670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40099b0 tx=0x7f2ed4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da98598025da115c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.705+0000 7f2ee2ffd700 1 -- 172.21.15.31:0/1886130608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400b860 con 0x7f2ee415ca40 2023-10-06T05:18:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.705+0000 7f2ee2ffd700 1 -- 172.21.15.31:0/1886130608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed400b9c0 con 0x7f2ee415ca40 2023-10-06T05:18:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.705+0000 7f2ee2ffd700 1 -- 172.21.15.31:0/1886130608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400bd60 con 0x7f2ee415ca40 2023-10-06T05:18:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.705+0000 7f2eea5ff700 1 -- 172.21.15.31:0/1886130608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee415ca40 msgr2=0x7f2ee4068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.705+0000 7f2eea5ff700 1 --2- 172.21.15.31:0/1886130608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee415ca40 0x7f2ee4068670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40099b0 tx=0x7f2ed4009250 comp rx=0 tx=0).stop 2023-10-06T05:18:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.705+0000 7f2eea5ff700 1 -- 172.21.15.31:0/1886130608 shutdown_connections 2023-10-06T05:18:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.705+0000 7f2eea5ff700 1 --2- 172.21.15.31:0/1886130608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee415ca40 0x7f2ee4068670 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.705+0000 7f2eea5ff700 1 -- 172.21.15.31:0/1886130608 >> 172.21.15.31:0/1886130608 conn(0x7f2ee40ce290 msgr2=0x7f2ee40ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.705+0000 7f2eea5ff700 1 -- 172.21.15.31:0/1886130608 shutdown_connections 2023-10-06T05:18:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.705+0000 7f2eea5ff700 1 -- 172.21.15.31:0/1886130608 wait complete. 2023-10-06T05:18:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.707+0000 7f2eea5ff700 1 Processor -- start 2023-10-06T05:18:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.707+0000 7f2eea5ff700 1 -- start start 2023-10-06T05:18:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2eea5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee406a300 0x7f2ee40c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2eea5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40c8f40 0x7f2ee40c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2eea5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee415ca40 0x7f2ee41596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2eea5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40c31c0 con 0x7f2ee406a300 2023-10-06T05:18:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2eea5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40adfa0 con 0x7f2ee40c8f40 2023-10-06T05:18:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2eea5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40ae840 con 0x7f2ee415ca40 2023-10-06T05:18:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee406a300 0x7f2ee40c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee406a300 0x7f2ee40c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34712/0 (socket says 172.21.15.31:34712) 2023-10-06T05:18:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee3fff700 1 -- 172.21.15.31:0/914128138 learned_addr learned my addr 172.21.15.31:0/914128138 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee37fe700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40c8f40 0x7f2ee40c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee8b9c700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee415ca40 0x7f2ee41596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee3fff700 1 -- 172.21.15.31:0/914128138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee415ca40 msgr2=0x7f2ee41596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee3fff700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee415ca40 0x7f2ee41596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee3fff700 1 -- 172.21.15.31:0/914128138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40c8f40 msgr2=0x7f2ee40c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee3fff700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40c8f40 0x7f2ee40c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee3fff700 1 -- 172.21.15.31:0/914128138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed40094f0 con 0x7f2ee406a300 2023-10-06T05:18:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.708+0000 7f2ee37fe700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40c8f40 0x7f2ee40c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.709+0000 7f2ee3fff700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee406a300 0x7f2ee40c2c80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40094c0 tx=0x7f2ed4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.709+0000 7f2ecbfff700 1 -- 172.21.15.31:0/914128138 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed401f070 con 0x7f2ee406a300 2023-10-06T05:18:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.709+0000 7f2ecbfff700 1 -- 172.21.15.31:0/914128138 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4011d10 con 0x7f2ee406a300 2023-10-06T05:18:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.709+0000 7f2ecbfff700 1 -- 172.21.15.31:0/914128138 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4025650 con 0x7f2ee406a300 2023-10-06T05:18:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.709+0000 7f2eea5ff700 1 -- 172.21.15.31:0/914128138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee40c6d20 con 0x7f2ee406a300 2023-10-06T05:18:11.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.709+0000 7f2eea5ff700 1 -- 172.21.15.31:0/914128138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee40ae1f0 con 0x7f2ee406a300 2023-10-06T05:18:11.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.710+0000 7f2eea5ff700 1 -- 172.21.15.31:0/914128138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee404c380 con 0x7f2ee406a300 2023-10-06T05:18:11.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.711+0000 7f2ecbfff700 1 -- 172.21.15.31:0/914128138 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ed4023070 con 0x7f2ee406a300 2023-10-06T05:18:11.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.712+0000 7f2ecbfff700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed0066f30 0x7f2ed00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:11.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.713+0000 7f2ecbfff700 1 -- 172.21.15.31:0/914128138 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2ed4085c30 con 0x7f2ee406a300 2023-10-06T05:18:11.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.713+0000 7f2ee37fe700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed0066f30 0x7f2ed00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:11.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.713+0000 7f2ee37fe700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed0066f30 0x7f2ed00693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8007fd0 tx=0x7f2ed8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:11.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.717+0000 7f2ecbfff700 1 -- 172.21.15.31:0/914128138 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ed4052ca0 con 0x7f2ee406a300 2023-10-06T05:18:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.996+0000 7f2eea5ff700 1 -- 172.21.15.31:0/914128138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2ee40ca1a0 con 0x7f2ee406a300 2023-10-06T05:18:12.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:11.999+0000 7f2ecbfff700 1 -- 172.21.15.31:0/914128138 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2ed40150e0 con 0x7f2ee406a300 2023-10-06T05:18:12.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 -- 172.21.15.31:0/914128138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed0066f30 msgr2=0x7f2ed00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed0066f30 0x7f2ed00693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8007fd0 tx=0x7f2ed8009040 comp rx=0 tx=0).stop 2023-10-06T05:18:12.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 -- 172.21.15.31:0/914128138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee406a300 msgr2=0x7f2ee40c2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee406a300 0x7f2ee40c2c80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40094c0 tx=0x7f2ed4017000 comp rx=0 tx=0).stop 2023-10-06T05:18:12.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 -- 172.21.15.31:0/914128138 shutdown_connections 2023-10-06T05:18:12.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee415ca40 0x7f2ee41596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed0066f30 0x7f2ed00693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee406a300 0x7f2ee40c2c80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 --2- 172.21.15.31:0/914128138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40c8f40 0x7f2ee40c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 -- 172.21.15.31:0/914128138 >> 172.21.15.31:0/914128138 conn(0x7f2ee40ce290 msgr2=0x7f2ee415e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:12.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 -- 172.21.15.31:0/914128138 shutdown_connections 2023-10-06T05:18:12.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.002+0000 7f2eea5ff700 1 -- 172.21.15.31:0/914128138 wait complete. 2023-10-06T05:18:12.005 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:18:12.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:12.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.216+0000 7fc98d7ff700 1 Processor -- start 2023-10-06T05:18:12.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.216+0000 7fc98d7ff700 1 -- start start 2023-10-06T05:18:12.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.217+0000 7fc98d7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98813a1e0 0x7fc9880c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.217+0000 7fc98d7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc98806db50 0x7fc988069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.217+0000 7fc98d7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9880c71b0 con 0x7fc98813a5b0 2023-10-06T05:18:12.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.217+0000 7fc98d7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9880c7490 con 0x7fc98806db50 2023-10-06T05:18:12.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.217+0000 7fc98d7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880c7770 con 0x7fc98813a1e0 2023-10-06T05:18:12.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.218+0000 7fc9877fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc98806db50 0x7fc988069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.218+0000 7fc9877fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc98806db50 0x7fc988069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:60420/0 (socket says 172.21.15.31:60420) 2023-10-06T05:18:12.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.218+0000 7fc9877fe700 1 -- 172.21.15.31:0/1253772283 learned_addr learned my addr 172.21.15.31:0/1253772283 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:12.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.218+0000 7fc986ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98813a1e0 0x7fc9880c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.218+0000 7fc9867fc700 1 --1- 172.21.15.31:0/1253772283 >> v1:172.21.15.62:6789/0 conn(0x7fc98813a5b0 0x7fc98806d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:47474/0 (socket says 172.21.15.31:47474) 2023-10-06T05:18:12.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.219+0000 7fc986ffd700 1 -- 172.21.15.31:0/1253772283 >> v2:172.21.15.62:3300/0 conn(0x7fc98806db50 msgr2=0x7fc988069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:12.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.219+0000 7fc986ffd700 1 --2- 172.21.15.31:0/1253772283 >> v2:172.21.15.62:3300/0 conn(0x7fc98806db50 0x7fc988069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.219+0000 7fc986ffd700 1 -- 172.21.15.31:0/1253772283 >> v1:172.21.15.62:6789/0 conn(0x7fc98813a5b0 legacy=0x7fc98806d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.219+0000 7fc986ffd700 1 -- 172.21.15.31:0/1253772283 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880c7a50 con 0x7fc98813a1e0 2023-10-06T05:18:12.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.219+0000 7fc9877fe700 1 --2- 172.21.15.31:0/1253772283 >> v2:172.21.15.62:3300/0 conn(0x7fc98806db50 0x7fc988069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:12.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.219+0000 7fc985ffb700 1 -- 172.21.15.31:0/1253772283 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622158425 0 0) 0x7fc9880c71b0 con 0x7fc98813a5b0 2023-10-06T05:18:12.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.219+0000 7fc986ffd700 1 --2- 172.21.15.31:0/1253772283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98813a1e0 0x7fc9880c66f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc9780098d0 tx=0x7fc978009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38241e1c35da06a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:12.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.219+0000 7fc985ffb700 1 -- 172.21.15.31:0/1253772283 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978005c10 con 0x7fc98813a1e0 2023-10-06T05:18:12.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.219+0000 7fc985ffb700 1 -- 172.21.15.31:0/1253772283 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc978005d70 con 0x7fc98813a1e0 2023-10-06T05:18:12.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.219+0000 7fc985ffb700 1 -- 172.21.15.31:0/1253772283 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978006280 con 0x7fc98813a1e0 2023-10-06T05:18:12.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.220+0000 7fc98d7ff700 1 -- 172.21.15.31:0/1253772283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98813a1e0 msgr2=0x7fc9880c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.220+0000 7fc98d7ff700 1 --2- 172.21.15.31:0/1253772283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98813a1e0 0x7fc9880c66f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc9780098d0 tx=0x7fc978009120 comp rx=0 tx=0).stop 2023-10-06T05:18:12.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.220+0000 7fc98d7ff700 1 -- 172.21.15.31:0/1253772283 shutdown_connections 2023-10-06T05:18:12.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.220+0000 7fc98d7ff700 1 --2- 172.21.15.31:0/1253772283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98813a1e0 0x7fc9880c66f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.220+0000 7fc98d7ff700 1 --2- 172.21.15.31:0/1253772283 >> v2:172.21.15.62:3300/0 conn(0x7fc98806db50 0x7fc988069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.220+0000 7fc98d7ff700 1 -- 172.21.15.31:0/1253772283 >> 172.21.15.31:0/1253772283 conn(0x7fc9880cc0b0 msgr2=0x7fc9880cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:12.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.220+0000 7fc98d7ff700 1 -- 172.21.15.31:0/1253772283 shutdown_connections 2023-10-06T05:18:12.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.220+0000 7fc98d7ff700 1 -- 172.21.15.31:0/1253772283 wait complete. 2023-10-06T05:18:12.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.221+0000 7fc98d7ff700 1 Processor -- start 2023-10-06T05:18:12.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.222+0000 7fc98d7ff700 1 -- start start 2023-10-06T05:18:12.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.222+0000 7fc98d7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98806db50 0x7fc9880c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.222+0000 7fc98d7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98813a1e0 0x7fc9880caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.222+0000 7fc98d7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc98813a5b0 0x7fc9880c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.222+0000 7fc98d7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880c6010 con 0x7fc98813a5b0 2023-10-06T05:18:12.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.222+0000 7fc98d7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880c5640 con 0x7fc98813a1e0 2023-10-06T05:18:12.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc98d7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880c4790 con 0x7fc98806db50 2023-10-06T05:18:12.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc9877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc98813a5b0 0x7fc9880c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc9877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc98813a5b0 0x7fc9880c3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34722/0 (socket says 172.21.15.31:34722) 2023-10-06T05:18:12.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc9877fe700 1 -- 172.21.15.31:0/3411495572 learned_addr learned my addr 172.21.15.31:0/3411495572 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:12.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc9867fc700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98813a1e0 0x7fc9880caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc986ffd700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98806db50 0x7fc9880c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc9877fe700 1 -- 172.21.15.31:0/3411495572 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98806db50 msgr2=0x7fc9880c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc9877fe700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98806db50 0x7fc9880c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc9877fe700 1 -- 172.21.15.31:0/3411495572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98813a1e0 msgr2=0x7fc9880caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc9877fe700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98813a1e0 0x7fc9880caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.223+0000 7fc9877fe700 1 -- 172.21.15.31:0/3411495572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980009ae0 con 0x7fc98813a5b0 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.224+0000 7fc9877fe700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc98813a5b0 0x7fc9880c3b20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc980009fa0 tx=0x7fc98000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.224+0000 7fc96effd700 1 -- 172.21.15.31:0/3411495572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980018070 con 0x7fc98813a5b0 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.224+0000 7fc96effd700 1 -- 172.21.15.31:0/3411495572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98000f040 con 0x7fc98813a5b0 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.224+0000 7fc96effd700 1 -- 172.21.15.31:0/3411495572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800092e0 con 0x7fc98813a5b0 2023-10-06T05:18:12.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.224+0000 7fc98d7ff700 1 -- 172.21.15.31:0/3411495572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9780093c0 con 0x7fc98813a5b0 2023-10-06T05:18:12.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.224+0000 7fc98d7ff700 1 -- 172.21.15.31:0/3411495572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9880c4d10 con 0x7fc98813a5b0 2023-10-06T05:18:12.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.225+0000 7fc98d7ff700 1 -- 172.21.15.31:0/3411495572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9880c57d0 con 0x7fc98813a5b0 2023-10-06T05:18:12.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.226+0000 7fc96effd700 1 -- 172.21.15.31:0/3411495572 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc980003020 con 0x7fc98813a5b0 2023-10-06T05:18:12.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.227+0000 7fc96effd700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc974066f30 0x7fc9740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:12.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.227+0000 7fc96effd700 1 -- 172.21.15.31:0/3411495572 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc980084ac0 con 0x7fc98813a5b0 2023-10-06T05:18:12.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.227+0000 7fc986ffd700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc974066f30 0x7fc9740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:12.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.228+0000 7fc986ffd700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc974066f30 0x7fc9740693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc978009160 tx=0x7fc978016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.231+0000 7fc96effd700 1 -- 172.21.15.31:0/3411495572 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc980051b30 con 0x7fc98813a5b0 2023-10-06T05:18:12.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.474+0000 7fc98d7ff700 1 -- 172.21.15.31:0/3411495572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc98804c380 con 0x7fc98813a5b0 2023-10-06T05:18:12.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.474+0000 7fc96effd700 1 -- 172.21.15.31:0/3411495572 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fc980055150 con 0x7fc98813a5b0 2023-10-06T05:18:12.476 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:12.476 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/746584225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":746584225},{"type":"v1","addr":"172.21.15.62:6834","nonce":746584225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/584374139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":584374139},{"type":"v1","addr":"172.21.15.62:6835","nonce":584374139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4926,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3794168057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3794168057},{"type":"v1","addr":"172.21.15.31:6836","nonce":3794168057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:00.090015+0000","modified":"2023-10-06T05:18:03.257587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/3886723511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3886723511},{"type":"v1","addr":"172.21.15.31:6837","nonce":3886723511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.476+0000 7fc98d7ff700 1 -- 172.21.15.31:0/3411495572 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc974066f30 msgr2=0x7fc9740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.476+0000 7fc98d7ff700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc974066f30 0x7fc9740693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc978009160 tx=0x7fc978016000 comp rx=0 tx=0).stop 2023-10-06T05:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.476+0000 7fc98d7ff700 1 -- 172.21.15.31:0/3411495572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc98813a5b0 msgr2=0x7fc9880c3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.476+0000 7fc98d7ff700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc98813a5b0 0x7fc9880c3b20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc980009fa0 tx=0x7fc98000b040 comp rx=0 tx=0).stop 2023-10-06T05:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.477+0000 7fc98d7ff700 1 -- 172.21.15.31:0/3411495572 shutdown_connections 2023-10-06T05:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.477+0000 7fc98d7ff700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc974066f30 0x7fc9740693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.477+0000 7fc98d7ff700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc98813a5b0 0x7fc9880c3b20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.477+0000 7fc98d7ff700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc98806db50 0x7fc9880c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.477+0000 7fc98d7ff700 1 --2- 172.21.15.31:0/3411495572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98813a1e0 0x7fc9880caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.477+0000 7fc98d7ff700 1 -- 172.21.15.31:0/3411495572 >> 172.21.15.31:0/3411495572 conn(0x7fc9880cc0b0 msgr2=0x7fc98813cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:12.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.477+0000 7fc98d7ff700 1 -- 172.21.15.31:0/3411495572 shutdown_connections 2023-10-06T05:18:12.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:12.477+0000 7fc98d7ff700 1 -- 172.21.15.31:0/3411495572 wait complete. 2023-10-06T05:18:12.481 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 16 2023-10-06T05:18:12.898 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:18:00.090015+0000', 'modified': '2023-10-06T05:18:03.257587+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4935}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4935': {'gid': 4935, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.31:6837/3886723511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6835', 'nonce': 3886723511}, {'type': 'v1', 'addr': '172.21.15.31:6837', 'nonce': 3886723511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:18:12.898 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:18:12.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:18:13.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.098+0000 7f3f16fff700 1 Processor -- start 2023-10-06T05:18:13.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.098+0000 7f3f16fff700 1 -- start start 2023-10-06T05:18:13.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.099+0000 7f3f16fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f10159500 0x7f3f100b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.100+0000 7f3f16fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f100c3f40 con 0x7f3f100cd510 2023-10-06T05:18:13.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.100+0000 7f3f16fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f100cb000 con 0x7f3f1015d2d0 2023-10-06T05:18:13.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.100+0000 7f3f16fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100b2860 con 0x7f3f10159500 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.100+0000 7f3f1559c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3f100cd510 0x7f3f100aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35040/0 (socket says 172.21.15.31:35040) 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.100+0000 7f3f1559c700 1 -- 172.21.15.31:0/1801815086 learned_addr learned my addr 172.21.15.31:0/1801815086 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.100+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f10159500 0x7f3f100b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.100+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813150877 0 0) 0x7f3f100c3f40 con 0x7f3f100cd510 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.100+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f00003570 con 0x7f3f100cd510 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964593093 0 0) 0x7f3f100cb000 con 0x7f3f1015d2d0 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f100c3f40 con 0x7f3f1015d2d0 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3639295330 0 0) 0x7f3f00003570 con 0x7f3f100cd510 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f10159500 msgr2=0x7f3f100b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f10159500 0x7f3f100b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 >> v1:172.21.15.62:6789/0 conn(0x7f3f1015d2d0 legacy=0x7f3f100cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1015d7d0 con 0x7f3f100cd510 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f3f080017f0 con 0x7f3f100cd510 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 >> v1:172.21.15.31:6789/0 conn(0x7f3f100cd510 legacy=0x7f3f100aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f10159500 0x7f3f00003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f00004180 0x7f3f00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 0x7f3f000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100b2860 con 0x7f3f00004d50 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0000bba0 con 0x7f3f00004180 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0000bd60 con 0x7f3f10159500 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f1559c700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 0x7f3f000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f10159500 0x7f3f00003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.101+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f00004180 0x7f3f00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.102+0000 7f3f1559c700 1 -- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f10159500 msgr2=0x7f3f00003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.102+0000 7f3f1559c700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f10159500 0x7f3f00003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.102+0000 7f3f1559c700 1 -- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f00004180 msgr2=0x7f3f00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.102+0000 7f3f1559c700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f00004180 0x7f3f00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.102+0000 7f3f1559c700 1 -- 172.21.15.31:0/1801815086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f00003570 con 0x7f3f00004d50 2023-10-06T05:18:13.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.102+0000 7f3f1559c700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 0x7f3f000051c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3f100cd910 tx=0x7f3f08002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a46e289be6e808 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.102+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080070d0 con 0x7f3f00004d50 2023-10-06T05:18:13.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.102+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08007230 con 0x7f3f00004d50 2023-10-06T05:18:13.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.102+0000 7f3f0f7fe700 1 -- 172.21.15.31:0/1801815086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080075f0 con 0x7f3f00004d50 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.103+0000 7f3f16fff700 1 -- 172.21.15.31:0/1801815086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 msgr2=0x7f3f000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.103+0000 7f3f16fff700 1 --2- 172.21.15.31:0/1801815086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 0x7f3f000051c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3f100cd910 tx=0x7f3f08002190 comp rx=0 tx=0).stop 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.103+0000 7f3f1559c700 1 -- 172.21.15.31:0/1801815086 reap_dead start 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.103+0000 7f3f16fff700 1 -- 172.21.15.31:0/1801815086 shutdown_connections 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.103+0000 7f3f16fff700 1 -- 172.21.15.31:0/1801815086 >> 172.21.15.31:0/1801815086 conn(0x7f3f1006ec90 msgr2=0x7f3f1006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.103+0000 7f3f16fff700 1 -- 172.21.15.31:0/1801815086 shutdown_connections 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.103+0000 7f3f16fff700 1 -- 172.21.15.31:0/1801815086 wait complete. 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.104+0000 7f3f16fff700 1 Processor -- start 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.104+0000 7f3f16fff700 1 -- start start 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.105+0000 7f3f16fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 0x7f3f10158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.105+0000 7f3f16fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f1016aad0 0x7f3f100cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.105+0000 7f3f16fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f100ce530 0x7f3f100c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.105+0000 7f3f16fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100ce9b0 con 0x7f3f00004d50 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.105+0000 7f3f16fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100c74d0 con 0x7f3f100ce530 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.105+0000 7f3f16fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f10158f10 con 0x7f3f1016aad0 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f14d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 0x7f3f10158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f14d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 0x7f3f10158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34738/0 (socket says 172.21.15.31:34738) 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f14d9b700 1 -- 172.21.15.31:0/1071995232 learned_addr learned my addr 172.21.15.31:0/1071995232 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f1016aad0 0x7f3f100cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f1559c700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f100ce530 0x7f3f100c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f14d9b700 1 -- 172.21.15.31:0/1071995232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f1016aad0 msgr2=0x7f3f100cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f1016aad0 0x7f3f100cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f14d9b700 1 -- 172.21.15.31:0/1071995232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f100ce530 msgr2=0x7f3f100c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f100ce530 0x7f3f100c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f14d9b700 1 -- 172.21.15.31:0/1071995232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f100cebd0 con 0x7f3f00004d50 2023-10-06T05:18:13.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 0x7f3f10158350 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006ac0 tx=0x7f3f040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f0cff9700 1 -- 172.21.15.31:0/1071995232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0400fa10 con 0x7f3f00004d50 2023-10-06T05:18:13.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f0cff9700 1 -- 172.21.15.31:0/1071995232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0400fb70 con 0x7f3f00004d50 2023-10-06T05:18:13.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f0cff9700 1 -- 172.21.15.31:0/1071995232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04014440 con 0x7f3f00004d50 2023-10-06T05:18:13.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.106+0000 7f3f16fff700 1 -- 172.21.15.31:0/1071995232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f100ceeb0 con 0x7f3f00004d50 2023-10-06T05:18:13.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.107+0000 7f3f16fff700 1 -- 172.21.15.31:0/1071995232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f100c7a00 con 0x7f3f00004d50 2023-10-06T05:18:13.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.110+0000 7f3f0cff9700 1 -- 172.21.15.31:0/1071995232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f04014830 con 0x7f3f00004d50 2023-10-06T05:18:13.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.111+0000 7f3f0cff9700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 0x7f3ef40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.111+0000 7f3f0cff9700 1 -- 172.21.15.31:0/1071995232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3f04018030 con 0x7f3f00004d50 2023-10-06T05:18:13.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.111+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 0x7f3ef40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.111+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 0x7f3ef40693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3f100c94a0 tx=0x7f3efc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.112+0000 7f3f2574cb80 1 Processor -- start 2023-10-06T05:18:13.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.112+0000 7f3f2574cb80 1 -- start start 2023-10-06T05:18:13.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.113+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d4012e450 0x558d4012e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.113+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d40113f40 0x558d40136f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.113+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d40137940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.113+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d40001310 con 0x558d4012e450 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.113+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d4003e130 con 0x558d401374d0 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.113+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d4003baf0 con 0x558d40113f40 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.113+0000 7f3f14d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d4012e450 0x558d4012e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3f14d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d4012e450 0x558d4012e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34746/0 (socket says 172.21.15.31:34746) 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3f14d9b700 1 -- 172.21.15.31:0/2227976399 learned_addr learned my addr 172.21.15.31:0/2227976399 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3f1559c700 1 --2- 172.21.15.31:0/2227976399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d40137940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/2227976399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d40113f40 0x558d40136f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3f14d9b700 1 -- 172.21.15.31:0/2227976399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d40113f40 msgr2=0x558d40136f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/2227976399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d40113f40 0x558d40136f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3f14d9b700 1 -- 172.21.15.31:0/2227976399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 msgr2=0x558d40137940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/2227976399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d40137940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3f14d9b700 1 -- 172.21.15.31:0/2227976399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d3fe3d790 con 0x558d4012e450 2023-10-06T05:18:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/2227976399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d4012e450 0x558d4012e820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f04012010 tx=0x7f3f04083400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc0b7282b6fc1f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3ef9ffb700 1 -- 172.21.15.31:0/2227976399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04059180 con 0x558d4012e450 2023-10-06T05:18:13.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3ef9ffb700 1 -- 172.21.15.31:0/2227976399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0405ced0 con 0x558d4012e450 2023-10-06T05:18:13.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.114+0000 7f3ef9ffb700 1 -- 172.21.15.31:0/2227976399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040542d0 con 0x558d4012e450 2023-10-06T05:18:13.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.115+0000 7f3f2574cb80 1 -- 172.21.15.31:0/2227976399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d4012e450 msgr2=0x558d4012e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.115+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/2227976399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d4012e450 0x558d4012e820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f04012010 tx=0x7f3f04083400 comp rx=0 tx=0).stop 2023-10-06T05:18:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.115+0000 7f3f2574cb80 1 -- 172.21.15.31:0/2227976399 shutdown_connections 2023-10-06T05:18:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.115+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/2227976399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d40137940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.115+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/2227976399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d4012e450 0x558d4012e820 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.115+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/2227976399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d40113f40 0x558d40136f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.115+0000 7f3f2574cb80 1 -- 172.21.15.31:0/2227976399 >> 172.21.15.31:0/2227976399 conn(0x558d40131880 msgr2=0x558d40131c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.115+0000 7f3f2574cb80 1 -- 172.21.15.31:0/2227976399 shutdown_connections 2023-10-06T05:18:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.115+0000 7f3f2574cb80 1 -- 172.21.15.31:0/2227976399 wait complete. 2023-10-06T05:18:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.116+0000 7f3f2574cb80 1 -- start start 2023-10-06T05:18:13.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.116+0000 7f3f2574cb80 1 Processor -- start 2023-10-06T05:18:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.117+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400e9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.117+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d4012e450 0x558d4010b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.117+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d401374d0 0x558d4012dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.117+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d400378e0 con 0x558d40113f40 2023-10-06T05:18:13.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.117+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3ffe3980 con 0x558d4012e450 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.117+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d3ffe3620 con 0x558d401374d0 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.117+0000 7f3f2574cb80 10 client.0 _ll_register_callbacks cb 0x558d3fe46a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.117+0000 7f3f14d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400e9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.117+0000 7f3f14d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400e9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34750/0 (socket says 172.21.15.31:34750) 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.117+0000 7f3f14d9b700 1 -- 172.21.15.31:0/1020581031 learned_addr learned my addr 172.21.15.31:0/1020581031 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3f1559c700 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d401374d0 0x558d4012dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d4012e450 0x558d4010b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3f14d9b700 1 -- 172.21.15.31:0/1020581031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d401374d0 msgr2=0x558d4012dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d401374d0 0x558d4012dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3f14d9b700 1 -- 172.21.15.31:0/1020581031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d4012e450 msgr2=0x558d4010b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d4012e450 0x558d4010b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3f14d9b700 1 -- 172.21.15.31:0/1020581031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f100c7a00 con 0x558d40113f40 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400e9a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f040567e0 tx=0x7f3f04008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efb4a358b9f843b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3eeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1020581031 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04055f30 con 0x558d40113f40 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1020581031 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3f04059180 con 0x558d40113f40 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.118+0000 7f3f2574cb80 1 -- 172.21.15.31:0/1020581031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d400dbd60 con 0x558d40113f40 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1020581031 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f040145a0 con 0x558d40113f40 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3f2574cb80 10 client.5136 fetch_fsmap learned FSMap version 16 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3f2574cb80 1 -- 172.21.15.31:0/1020581031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x558d3ff2a0a0 con 0x558d40113f40 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1020581031 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558d400dbd60 con 0x558d40113f40 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1020581031 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3f0406c170 con 0x558d40113f40 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3f2574cb80 10 client.5136 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T05:18:13.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3f2574cb80 10 client.5136 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:18:13.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3f2574cb80 20 client.5136 populate_metadata read hostname 'smithi031' 2023-10-06T05:18:13.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.31:6835/3886723511,v1:172.21.15.31:6837/3886723511] conn(0x558d4010a120 0x558d4010a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3f2574cb80 4 client.5136 mds_command: new command op to 4935 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:18:13.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3f2574cb80 1 -- 172.21.15.31:0/1020581031 --> [v2:172.21.15.31:6835/3886723511,v1:172.21.15.31:6837/3886723511] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d400b8b30 con 0x558d4010a120 2023-10-06T05:18:13.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.119+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.31:6835/3886723511,v1:172.21.15.31:6837/3886723511] conn(0x558d4010a120 0x558d4010a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.120+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.31:6835/3886723511,v1:172.21.15.31:6837/3886723511] conn(0x558d4010a120 0x558d4010a4f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db6902e028e9e05 server_cookie=156389721e1b688b in_seq=0 out_seq=0 2023-10-06T05:18:13.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.120+0000 7f3eeeffd700 10 client.5136 ms_handle_connect on v2:172.21.15.31:6835/3886723511 2023-10-06T05:18:13.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1020581031 <== mds.0 v2:172.21.15.31:6835/3886723511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558d400b8b30 con 0x558d4010a120 2023-10-06T05:18:13.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3eeeffd700 10 client.5136 handle_command_reply: tid=0 2023-10-06T05:18:13.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 client.5136 shutdown 2023-10-06T05:18:13.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 -- 172.21.15.31:0/1020581031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 msgr2=0x558d400e9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400e9a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f040567e0 tx=0x7f3f04008000 comp rx=0 tx=0).stop 2023-10-06T05:18:13.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 -- 172.21.15.31:0/1020581031 shutdown_connections 2023-10-06T05:18:13.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d401374d0 0x558d4012dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400e9a50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.31:6835/3886723511,v1:172.21.15.31:6837/3886723511] conn(0x558d4010a120 0x558d4010a4f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/1020581031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d4012e450 0x558d4010b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 -- 172.21.15.31:0/1020581031 >> 172.21.15.31:0/1020581031 conn(0x558d40131880 msgr2=0x558d400e3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3eeeffd700 0 client.5136 ms_handle_reset on v2:172.21.15.31:6835/3886723511 2023-10-06T05:18:13.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 -- 172.21.15.31:0/1020581031 shutdown_connections 2023-10-06T05:18:13.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 1 -- 172.21.15.31:0/1020581031 wait complete. 2023-10-06T05:18:13.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.121+0000 7f3f2574cb80 20 client.5136 trim_cache size 0 max 16384 2023-10-06T05:18:13.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.141+0000 7f3f2574cb80 1 Processor -- start 2023-10-06T05:18:13.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.141+0000 7f3f2574cb80 1 -- start start 2023-10-06T05:18:13.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400dd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 0x558d400dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d4010f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d4012f540 con 0x558d40113f40 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3fecf1e0 con 0x558d401374d0 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d400e6020 con 0x558d4012e450 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f14d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400dd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f14d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400dd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34766/0 (socket says 172.21.15.31:34766) 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f14d9b700 1 -- 172.21.15.31:0/243655783 learned_addr learned my addr 172.21.15.31:0/243655783 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f1559c700 1 --2- 172.21.15.31:0/243655783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d4010f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/243655783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 0x558d400dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f14d9b700 1 -- 172.21.15.31:0/243655783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 msgr2=0x558d400dddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/243655783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 0x558d400dddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f14d9b700 1 -- 172.21.15.31:0/243655783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 msgr2=0x558d4010f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/243655783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d4010f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.142+0000 7f3f14d9b700 1 -- 172.21.15.31:0/243655783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d400063c0 con 0x558d40113f40 2023-10-06T05:18:13.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/243655783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400dd870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3f04054280 tx=0x7f3f0407f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaf6fa5c6dba9d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3efa7fc700 1 -- 172.21.15.31:0/243655783 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04055f30 con 0x558d40113f40 2023-10-06T05:18:13.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3efa7fc700 1 -- 172.21.15.31:0/243655783 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04059180 con 0x558d40113f40 2023-10-06T05:18:13.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3efa7fc700 1 -- 172.21.15.31:0/243655783 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040145a0 con 0x558d40113f40 2023-10-06T05:18:13.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3f2574cb80 1 -- 172.21.15.31:0/243655783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 msgr2=0x558d400dd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/243655783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400dd870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3f04054280 tx=0x7f3f0407f040 comp rx=0 tx=0).stop 2023-10-06T05:18:13.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3f2574cb80 1 -- 172.21.15.31:0/243655783 shutdown_connections 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/243655783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d4010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/243655783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d400dd870 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/243655783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 0x558d400dddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.143+0000 7f3f2574cb80 1 -- 172.21.15.31:0/243655783 >> 172.21.15.31:0/243655783 conn(0x558d40131880 msgr2=0x558d400e3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.144+0000 7f3f2574cb80 1 -- 172.21.15.31:0/243655783 shutdown_connections 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.144+0000 7f3f2574cb80 1 -- 172.21.15.31:0/243655783 wait complete. 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.144+0000 7f3f2574cb80 1 -- start start 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.144+0000 7f3f2574cb80 1 Processor -- start 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.145+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d4010c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.145+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 0x558d4010c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.145+0000 7f3f2574cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d400dd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.145+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d40116f30 con 0x558d40113f40 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.145+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d40133ff0 con 0x558d401374d0 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.145+0000 7f3f2574cb80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d40135f60 con 0x558d4012e450 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.145+0000 7f3f2574cb80 10 client.0 _ll_register_callbacks cb 0x558d3fe46a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f14d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d4010c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f14d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d4010c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34778/0 (socket says 172.21.15.31:34778) 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f14d9b700 1 -- 172.21.15.31:0/3999355174 learned_addr learned my addr 172.21.15.31:0/3999355174 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 0x558d4010c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f1559c700 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d400dd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f14d9b700 1 -- 172.21.15.31:0/3999355174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 msgr2=0x558d4010c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 0x558d4010c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f14d9b700 1 -- 172.21.15.31:0/3999355174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 msgr2=0x558d400dd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d400dd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f14d9b700 1 -- 172.21.15.31:0/3999355174 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d3ff2a0a0 con 0x558d40113f40 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 0x558d4010c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f1559c700 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d400dd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3f14d9b700 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d4010c130 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3f04056590 tx=0x7f3f0400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce17c6afbd8ea11e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.146+0000 7f3eeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3eeeffd700 1 -- 172.21.15.31:0/3999355174 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04055f30 con 0x558d40113f40 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3f2574cb80 1 -- 172.21.15.31:0/3999355174 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f0405ced0 con 0x558d40113f40 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3eeeffd700 1 -- 172.21.15.31:0/3999355174 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3f040835b0 con 0x558d40113f40 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3eeeffd700 1 -- 172.21.15.31:0/3999355174 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f0407dbe0 con 0x558d40113f40 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3f2574cb80 10 client.5142 fetch_fsmap learned FSMap version 16 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3f2574cb80 1 -- 172.21.15.31:0/3999355174 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x558d3fe9e1d0 con 0x558d40113f40 2023-10-06T05:18:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3eeeffd700 1 -- 172.21.15.31:0/3999355174 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0402b080 con 0x558d40113f40 2023-10-06T05:18:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3eeeffd700 1 -- 172.21.15.31:0/3999355174 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3f04055420 con 0x558d40113f40 2023-10-06T05:18:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3f2574cb80 10 client.5142 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T05:18:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3f2574cb80 10 client.5142 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:18:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3f2574cb80 20 client.5142 populate_metadata read hostname 'smithi031' 2023-10-06T05:18:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.31:6835/3886723511,v1:172.21.15.31:6837/3886723511] conn(0x558d401363f0 0x558d400dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3f2574cb80 4 client.5142 mds_command: new command op to 4935 tid=0[{"prefix": "status"}] 2023-10-06T05:18:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3f2574cb80 1 -- 172.21.15.31:0/3999355174 --> [v2:172.21.15.31:6835/3886723511,v1:172.21.15.31:6837/3886723511] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558d3ffda8d0 con 0x558d401363f0 2023-10-06T05:18:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.147+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.31:6835/3886723511,v1:172.21.15.31:6837/3886723511] conn(0x558d401363f0 0x558d400dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3f0ffff700 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.31:6835/3886723511,v1:172.21.15.31:6837/3886723511] conn(0x558d401363f0 0x558d400dddb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1bfde5b9042e30f server_cookie=2bb4854a0266a61 in_seq=0 out_seq=0 2023-10-06T05:18:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3eeeffd700 10 client.5142 ms_handle_connect on v2:172.21.15.31:6835/3886723511 2023-10-06T05:18:13.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3eeeffd700 1 -- 172.21.15.31:0/3999355174 <== mds.0 v2:172.21.15.31:6835/3886723511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x558d3ffda8d0 con 0x558d401363f0 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3eeeffd700 10 client.5142 handle_command_reply: tid=0 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3f2574cb80 1 client.5142 shutdown 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3f2574cb80 1 -- 172.21.15.31:0/3999355174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 msgr2=0x558d4010c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d4010c130 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3f04056590 tx=0x7f3f0400d000 comp rx=0 tx=0).stop 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3f2574cb80 1 -- 172.21.15.31:0/3999355174 shutdown_connections 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.31:6835/3886723511,v1:172.21.15.31:6837/3886723511] conn(0x558d401363f0 0x558d400dddb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558d401374d0 0x558d400dd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d40113f40 0x558d4010c130 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3f2574cb80 1 --2- 172.21.15.31:0/3999355174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558d4012e450 0x558d4010c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3eeeffd700 0 client.5142 ms_handle_reset on v2:172.21.15.31:6835/3886723511 2023-10-06T05:18:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.148+0000 7f3f2574cb80 1 -- 172.21.15.31:0/3999355174 >> 172.21.15.31:0/3999355174 conn(0x558d40131880 msgr2=0x558d401308c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f2574cb80 1 -- 172.21.15.31:0/3999355174 shutdown_connections 2023-10-06T05:18:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f2574cb80 1 -- 172.21.15.31:0/3999355174 wait complete. 2023-10-06T05:18:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f2574cb80 20 client.5142 trim_cache size 0 max 16384 2023-10-06T05:18:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:18:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:18:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:18:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4935, 2023-10-06T05:18:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:18:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:18:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:18:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 12.902120516, 2023-10-06T05:18:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 16, 2023-10-06T05:18:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 32, 2023-10-06T05:18:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 32, 2023-10-06T05:18:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 21.705414906000001 2023-10-06T05:18:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:18:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f16fff700 1 -- 172.21.15.31:0/1071995232 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 msgr2=0x7f3ef40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f16fff700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 0x7f3ef40693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3f100c94a0 tx=0x7f3efc009040 comp rx=0 tx=0).stop 2023-10-06T05:18:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f16fff700 1 -- 172.21.15.31:0/1071995232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 msgr2=0x7f3f10158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f16fff700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 0x7f3f10158350 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006ac0 tx=0x7f3f040091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f16fff700 1 -- 172.21.15.31:0/1071995232 shutdown_connections 2023-10-06T05:18:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f16fff700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f100ce530 0x7f3f100c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f16fff700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3ef4066f30 0x7f3ef40693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f16fff700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00004d50 0x7f3f10158350 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f16fff700 1 --2- 172.21.15.31:0/1071995232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f1016aad0 0x7f3f100cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.149+0000 7f3f16fff700 1 -- 172.21.15.31:0/1071995232 >> 172.21.15.31:0/1071995232 conn(0x7f3f1006ec90 msgr2=0x7f3f100cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.150+0000 7f3f16fff700 1 -- 172.21.15.31:0/1071995232 shutdown_connections 2023-10-06T05:18:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.150+0000 7f3f16fff700 1 -- 172.21.15.31:0/1071995232 wait complete. 2023-10-06T05:18:13.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:18:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.362+0000 7f82701ff700 1 Processor -- start 2023-10-06T05:18:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.362+0000 7f82701ff700 1 -- start start 2023-10-06T05:18:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.363+0000 7f82701ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8268159850 0x7f82680b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.363+0000 7f82701ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8268068f50 con 0x7f82680b19e0 2023-10-06T05:18:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.363+0000 7f82701ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82680b3ab0 con 0x7f82680b1610 2023-10-06T05:18:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.363+0000 7f82701ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82680b3d90 con 0x7f8268159850 2023-10-06T05:18:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.363+0000 7f826d79a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f82680b19e0 0x7f8268159140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35056/0 (socket says 172.21.15.31:35056) 2023-10-06T05:18:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.363+0000 7f826d79a700 1 -- 172.21.15.31:0/618728166 learned_addr learned my addr 172.21.15.31:0/618728166 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.364+0000 7f826e79c700 1 --2- 172.21.15.31:0/618728166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8268159850 0x7f82680b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.364+0000 7f826cf99700 1 -- 172.21.15.31:0/618728166 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836771809 0 0) 0x7f8268068f50 con 0x7f82680b19e0 2023-10-06T05:18:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.364+0000 7f826cf99700 1 -- 172.21.15.31:0/618728166 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8254003570 con 0x7f82680b19e0 2023-10-06T05:18:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.364+0000 7f826cf99700 1 -- 172.21.15.31:0/618728166 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749286968 0 0) 0x7f82680b3ab0 con 0x7f82680b1610 2023-10-06T05:18:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.364+0000 7f826cf99700 1 -- 172.21.15.31:0/618728166 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8268068f50 con 0x7f82680b1610 2023-10-06T05:18:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.364+0000 7f826cf99700 1 -- 172.21.15.31:0/618728166 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2515658586 0 0) 0x7f8254003570 con 0x7f82680b19e0 2023-10-06T05:18:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.364+0000 7f826e79c700 1 -- 172.21.15.31:0/618728166 >> v1:172.21.15.62:6789/0 conn(0x7f82680b1610 legacy=0x7f82680cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.364+0000 7f826e79c700 1 -- 172.21.15.31:0/618728166 >> v1:172.21.15.31:6789/0 conn(0x7f82680b19e0 legacy=0x7f8268159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.364+0000 7f826e79c700 1 -- 172.21.15.31:0/618728166 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268068be0 con 0x7f8268159850 2023-10-06T05:18:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.365+0000 7f826e79c700 1 --2- 172.21.15.31:0/618728166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8268159850 0x7f82680b33d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8260009e90 tx=0x7f8260009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3dc272425ad8216 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.365+0000 7f826cf99700 1 -- 172.21.15.31:0/618728166 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826000e070 con 0x7f8268159850 2023-10-06T05:18:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.365+0000 7f826cf99700 1 -- 172.21.15.31:0/618728166 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826000b870 con 0x7f8268159850 2023-10-06T05:18:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.365+0000 7f826cf99700 1 -- 172.21.15.31:0/618728166 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258001470 con 0x7f8268159850 2023-10-06T05:18:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.365+0000 7f82701ff700 1 -- 172.21.15.31:0/618728166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8268159850 msgr2=0x7f82680b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.365+0000 7f82701ff700 1 --2- 172.21.15.31:0/618728166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8268159850 0x7f82680b33d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8260009e90 tx=0x7f8260009780 comp rx=0 tx=0).stop 2023-10-06T05:18:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.365+0000 7f82701ff700 1 -- 172.21.15.31:0/618728166 shutdown_connections 2023-10-06T05:18:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.365+0000 7f82701ff700 1 --2- 172.21.15.31:0/618728166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8268159850 0x7f82680b33d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.365+0000 7f82701ff700 1 -- 172.21.15.31:0/618728166 >> 172.21.15.31:0/618728166 conn(0x7f826806b5b0 msgr2=0x7f826806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.366+0000 7f82701ff700 1 -- 172.21.15.31:0/618728166 shutdown_connections 2023-10-06T05:18:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.366+0000 7f82701ff700 1 -- 172.21.15.31:0/618728166 wait complete. 2023-10-06T05:18:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.367+0000 7f82701ff700 1 Processor -- start 2023-10-06T05:18:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.367+0000 7f82701ff700 1 -- start start 2023-10-06T05:18:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f82701ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82680b1610 0x7f826815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f82701ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82680b19e0 0x7f82680c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f82701ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268159850 0x7f8268153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f82701ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826815ac10 con 0x7f82680b1610 2023-10-06T05:18:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f82701ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826815f290 con 0x7f8268159850 2023-10-06T05:18:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f82701ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826815f570 con 0x7f82680b19e0 2023-10-06T05:18:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f826df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82680b1610 0x7f826815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f826df9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82680b1610 0x7f826815ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34794/0 (socket says 172.21.15.31:34794) 2023-10-06T05:18:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f826df9b700 1 -- 172.21.15.31:0/3624823703 learned_addr learned my addr 172.21.15.31:0/3624823703 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f826d79a700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82680b19e0 0x7f82680c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.368+0000 7f826e79c700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268159850 0x7f8268153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f826df9b700 1 -- 172.21.15.31:0/3624823703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82680b19e0 msgr2=0x7f82680c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f826df9b700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82680b19e0 0x7f82680c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f826df9b700 1 -- 172.21.15.31:0/3624823703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268159850 msgr2=0x7f8268153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f826df9b700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268159850 0x7f8268153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f826df9b700 1 -- 172.21.15.31:0/3624823703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8260009a20 con 0x7f82680b1610 2023-10-06T05:18:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f826df9b700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82680b1610 0x7f826815ebe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f826400c7c0 tx=0x7f8264007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f825dffb700 1 -- 172.21.15.31:0/3624823703 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264002ea0 con 0x7f82680b1610 2023-10-06T05:18:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f825dffb700 1 -- 172.21.15.31:0/3624823703 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8264010510 con 0x7f82680b1610 2023-10-06T05:18:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f82701ff700 1 -- 172.21.15.31:0/3624823703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826815f850 con 0x7f82680b1610 2023-10-06T05:18:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f825dffb700 1 -- 172.21.15.31:0/3624823703 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82640108b0 con 0x7f82680b1610 2023-10-06T05:18:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.369+0000 7f82701ff700 1 -- 172.21.15.31:0/3624823703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82680c7c80 con 0x7f82680b1610 2023-10-06T05:18:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.371+0000 7f82701ff700 1 -- 172.21.15.31:0/3624823703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826804c380 con 0x7f82680b1610 2023-10-06T05:18:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.371+0000 7f825dffb700 1 -- 172.21.15.31:0/3624823703 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8264010c60 con 0x7f82680b1610 2023-10-06T05:18:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.372+0000 7f825dffb700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f824c066f30 0x7f824c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.372+0000 7f825dffb700 1 -- 172.21.15.31:0/3624823703 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8264083dc0 con 0x7f82680b1610 2023-10-06T05:18:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.372+0000 7f826d79a700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f824c066f30 0x7f824c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.373+0000 7f826d79a700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f824c066f30 0x7f824c0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8258001ed0 tx=0x7f82580019d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.376+0000 7f825dffb700 1 -- 172.21.15.31:0/3624823703 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8264050e30 con 0x7f82680b1610 2023-10-06T05:18:13.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.624+0000 7f82701ff700 1 -- 172.21.15.31:0/3624823703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f82680b4070 con 0x7f82680b1610 2023-10-06T05:18:13.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.625+0000 7f825dffb700 1 -- 172.21.15.31:0/3624823703 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8264017070 con 0x7f82680b1610 2023-10-06T05:18:13.627 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:18:13.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 -- 172.21.15.31:0/3624823703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f824c066f30 msgr2=0x7f824c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f824c066f30 0x7f824c0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8258001ed0 tx=0x7f82580019d0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 -- 172.21.15.31:0/3624823703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82680b1610 msgr2=0x7f826815ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:13.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82680b1610 0x7f826815ebe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f826400c7c0 tx=0x7f8264007dc0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 -- 172.21.15.31:0/3624823703 shutdown_connections 2023-10-06T05:18:13.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268159850 0x7f8268153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f824c066f30 0x7f824c0693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82680b1610 0x7f826815ebe0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 --2- 172.21.15.31:0/3624823703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82680b19e0 0x7f82680c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:13.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 -- 172.21.15.31:0/3624823703 >> 172.21.15.31:0/3624823703 conn(0x7f826806b5b0 msgr2=0x7f82680af060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:13.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 -- 172.21.15.31:0/3624823703 shutdown_connections 2023-10-06T05:18:13.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:13.627+0000 7f82701ff700 1 -- 172.21.15.31:0/3624823703 wait complete. 2023-10-06T05:18:13.646 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:18:13.646 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:18:13.646 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:18:13.646 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:18:13.646 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:13.646 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:18:13.647 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:18:13.647 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:18:13.647 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:13.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:13.647 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:18:13.694 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:18:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:18:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:18:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:18:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:13.695 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:18:13.696 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:18:13.696 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:18:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:18:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:18:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42220sec preferred_lft 42220sec 2023-10-06T05:18:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:18:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:13.697 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:18:13.697 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:18:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:18:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:18:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:13.698 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:18:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:18:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:13.698 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:18:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:18:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:13.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:13.699 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:18:13.803 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:18:13.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:13.806 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:18:13.855 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:18:13.855 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:18:13.855 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:18:13.855 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:18:13.855 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:18:13.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:13.857 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:18:13.857 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:18:13.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:18:13.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:18:13.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:18:13.857 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:18:14.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:14.028 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:18:14.075 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:18:14.075 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:18:14.076 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:14.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:14.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:18:14.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:14.190 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:14.237 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:14.237 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:14.283 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:18:14.283 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:18:14.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:14.424 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:14.424 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:18:14.450 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:14.451 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:18:14.451 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:18:14.451 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:18:14.451 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:18:14.455 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:18:14.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:14.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:14.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:18:14.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.719+0000 7f1d4b3ff700 1 Processor -- start 2023-10-06T05:18:14.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.719+0000 7f1d4b3ff700 1 -- start start 2023-10-06T05:18:14.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.721+0000 7f1d4b3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1d44140a80 0x7f1d440cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.721+0000 7f1d4b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 0x7f1d440c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.721+0000 7f1d4b3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d440cbec0 con 0x7f1d44140e50 2023-10-06T05:18:14.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.721+0000 7f1d4b3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d440cca40 con 0x7f1d44140a80 2023-10-06T05:18:14.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.721+0000 7f1d4b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440cc110 con 0x7f1d440b10b0 2023-10-06T05:18:14.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.721+0000 7f1d4919b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1d44140a80 0x7f1d440cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.721+0000 7f1d4899a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 0x7f1d440c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.721+0000 7f1d4919b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1d44140a80 0x7f1d440cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:60498/0 (socket says 172.21.15.31:60498) 2023-10-06T05:18:14.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.721+0000 7f1d4919b700 1 -- 172.21.15.31:0/919837071 learned_addr learned my addr 172.21.15.31:0/919837071 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:14.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.722+0000 7f1d4919b700 1 -- 172.21.15.31:0/919837071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 msgr2=0x7f1d440c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:14.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.722+0000 7f1d4919b700 1 --2- 172.21.15.31:0/919837071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 0x7f1d440c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.722+0000 7f1d4919b700 1 -- 172.21.15.31:0/919837071 >> v1:172.21.15.62:6789/0 conn(0x7f1d44140e50 legacy=0x7f1d4406cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.722+0000 7f1d4919b700 1 -- 172.21.15.31:0/919837071 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d440cc3f0 con 0x7f1d44140a80 2023-10-06T05:18:14.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.722+0000 7f1d4899a700 1 --2- 172.21.15.31:0/919837071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 0x7f1d440c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:14.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.722+0000 7f1d4919b700 1 --2- 172.21.15.31:0/919837071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d44140a80 0x7f1d440cb120 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d400099b0 tx=0x7f1d40009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8681b82942e8591 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.722+0000 7f1d3bfff700 1 -- 172.21.15.31:0/919837071 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d40009ea0 con 0x7f1d44140a80 2023-10-06T05:18:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.723+0000 7f1d3bfff700 1 -- 172.21.15.31:0/919837071 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4000b860 con 0x7f1d44140a80 2023-10-06T05:18:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.723+0000 7f1d3bfff700 1 -- 172.21.15.31:0/919837071 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4000bc20 con 0x7f1d44140a80 2023-10-06T05:18:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.723+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/919837071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d44140a80 msgr2=0x7f1d440cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.723+0000 7f1d4b3ff700 1 --2- 172.21.15.31:0/919837071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d44140a80 0x7f1d440cb120 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d400099b0 tx=0x7f1d40009250 comp rx=0 tx=0).stop 2023-10-06T05:18:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.723+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/919837071 shutdown_connections 2023-10-06T05:18:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.723+0000 7f1d4b3ff700 1 --2- 172.21.15.31:0/919837071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d44140a80 0x7f1d440cb120 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.723+0000 7f1d4b3ff700 1 --2- 172.21.15.31:0/919837071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 0x7f1d440c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.723+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/919837071 >> 172.21.15.31:0/919837071 conn(0x7f1d4406b620 msgr2=0x7f1d4406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.723+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/919837071 shutdown_connections 2023-10-06T05:18:14.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.723+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/919837071 wait complete. 2023-10-06T05:18:14.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.725+0000 7f1d4b3ff700 1 Processor -- start 2023-10-06T05:18:14.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.726+0000 7f1d4b3ff700 1 -- start start 2023-10-06T05:18:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 0x7f1d44158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4b3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d44140a80 0x7f1d440b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4b3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d44140e50 0x7f1d44156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4b3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d44138920 con 0x7f1d44140e50 2023-10-06T05:18:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4b3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4413b080 con 0x7f1d44140a80 2023-10-06T05:18:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4413b360 con 0x7f1d440b10b0 2023-10-06T05:18:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d44140e50 0x7f1d44156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4919b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 0x7f1d44158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4899a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d44140a80 0x7f1d440b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d44140e50 0x7f1d44156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34800/0 (socket says 172.21.15.31:34800) 2023-10-06T05:18:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4999c700 1 -- 172.21.15.31:0/4055909450 learned_addr learned my addr 172.21.15.31:0/4055909450 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.727+0000 7f1d4919b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 0x7f1d44158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:38502/0 (socket says 172.21.15.31:38502) 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.728+0000 7f1d4999c700 1 -- 172.21.15.31:0/4055909450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 msgr2=0x7f1d44158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.728+0000 7f1d4999c700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 0x7f1d44158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.728+0000 7f1d4999c700 1 -- 172.21.15.31:0/4055909450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d44140a80 msgr2=0x7f1d440b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.728+0000 7f1d4999c700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d44140a80 0x7f1d440b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.728+0000 7f1d4999c700 1 -- 172.21.15.31:0/4055909450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34009420 con 0x7f1d44140e50 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.728+0000 7f1d4999c700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d44140e50 0x7f1d44156a50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c910 tx=0x7f1d3c0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.728+0000 7f1d397fa700 1 -- 172.21.15.31:0/4055909450 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c012630 con 0x7f1d44140e50 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.728+0000 7f1d397fa700 1 -- 172.21.15.31:0/4055909450 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c012790 con 0x7f1d44140e50 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.728+0000 7f1d397fa700 1 -- 172.21.15.31:0/4055909450 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c012b50 con 0x7f1d44140e50 2023-10-06T05:18:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.728+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/4055909450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d400094f0 con 0x7f1d44140e50 2023-10-06T05:18:14.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.729+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/4055909450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4413b920 con 0x7f1d44140e50 2023-10-06T05:18:14.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.730+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/4055909450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4404c380 con 0x7f1d44140e50 2023-10-06T05:18:14.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.730+0000 7f1d397fa700 1 -- 172.21.15.31:0/4055909450 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d3c01b450 con 0x7f1d44140e50 2023-10-06T05:18:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.731+0000 7f1d397fa700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1d28066f30 0x7f1d280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.731+0000 7f1d397fa700 1 -- 172.21.15.31:0/4055909450 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1d3c084820 con 0x7f1d44140e50 2023-10-06T05:18:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.732+0000 7f1d4919b700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1d28066f30 0x7f1d280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.732+0000 7f1d4919b700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1d28066f30 0x7f1d280693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d400099b0 tx=0x7f1d40017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:14.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.740+0000 7f1d397fa700 1 -- 172.21.15.31:0/4055909450 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d3c0553c0 con 0x7f1d44140e50 2023-10-06T05:18:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.991+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/4055909450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1d440cc6d0 con 0x7f1d44140e50 2023-10-06T05:18:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.991+0000 7f1d397fa700 1 -- 172.21.15.31:0/4055909450 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1d3c016060 con 0x7f1d44140e50 2023-10-06T05:18:14.993 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:18:14.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/4055909450 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1d28066f30 msgr2=0x7f1d280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1d28066f30 0x7f1d280693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d400099b0 tx=0x7f1d40017000 comp rx=0 tx=0).stop 2023-10-06T05:18:14.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/4055909450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d44140e50 msgr2=0x7f1d44156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:14.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d44140e50 0x7f1d44156a50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c910 tx=0x7f1d3c0016f0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/4055909450 shutdown_connections 2023-10-06T05:18:14.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1d28066f30 0x7f1d280693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d44140e50 0x7f1d44156a50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d440b10b0 0x7f1d44158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 --2- 172.21.15.31:0/4055909450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d44140a80 0x7f1d440b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:14.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/4055909450 >> 172.21.15.31:0/4055909450 conn(0x7f1d4406b620 msgr2=0x7f1d440c4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:14.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/4055909450 shutdown_connections 2023-10-06T05:18:14.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:14.994+0000 7f1d4b3ff700 1 -- 172.21.15.31:0/4055909450 wait complete. 2023-10-06T05:18:15.013 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:18:15.013 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:18:15.013 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:18:15.013 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:18:15.013 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:18:15.013 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:18:15.013 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:18:15.014 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:18:15.014 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:18:15.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:15.014 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:18:15.067 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:15.068 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42219sec preferred_lft 42219sec 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:18:15.069 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:15.070 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:18:15.070 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:15.070 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:18:15.070 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:18:15.070 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:15.070 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:15.070 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:18:15.070 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:18:15.070 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:15.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:15.071 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:18:15.145 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:18:15.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:15.149 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:18:15.199 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:18:15.199 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:18:15.199 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:18:15.199 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:18:15.199 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:18:15.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:15.200 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:18:15.200 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:18:15.200 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:18:15.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:18:15.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:18:15.201 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:18:15.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:15.374 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:18:15.420 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:18:15.421 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:18:15.421 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:18:15.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:15.421 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:18:15.533 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:15.533 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:18:15.578 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:18:15.579 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:15.579 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:18:15.663 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:18:15.664 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:18:15.664 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:15.803 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:15.803 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:18:15.831 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:15.831 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:18:15.831 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:18:15.831 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:18:15.831 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:18:15.835 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:18:15.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:15.835 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:18:15.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:18:16.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.116+0000 7f3f691ff700 1 Processor -- start 2023-10-06T05:18:16.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.116+0000 7f3f691ff700 1 -- start start 2023-10-06T05:18:16.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.117+0000 7f3f691ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 0x7f3f6406d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.117+0000 7f3f691ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f6406d830 0x7f3f64159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.117+0000 7f3f691ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3f6406e300 0x7f3f64159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.117+0000 7f3f691ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f640aba60 con 0x7f3f6406d830 2023-10-06T05:18:16.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.117+0000 7f3f691ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f640c72b0 con 0x7f3f6406e300 2023-10-06T05:18:16.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.117+0000 7f3f691ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640c7590 con 0x7f3f6406df30 2023-10-06T05:18:16.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.118+0000 7f3f6259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f6406d830 0x7f3f64159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.118+0000 7f3f6259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f6406d830 0x7f3f64159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34816/0 (socket says 172.21.15.31:34816) 2023-10-06T05:18:16.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.118+0000 7f3f6259c700 1 -- 172.21.15.31:0/1796534470 learned_addr learned my addr 172.21.15.31:0/1796534470 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:16.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.118+0000 7f3f6359e700 1 --2- 172.21.15.31:0/1796534470 >> v2:172.21.15.62:3300/0 conn(0x7f3f6406e300 0x7f3f64159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.118+0000 7f3f62d9d700 1 --2- 172.21.15.31:0/1796534470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 0x7f3f6406d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.118+0000 7f3f6259c700 1 -- 172.21.15.31:0/1796534470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 msgr2=0x7f3f6406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.118+0000 7f3f6259c700 1 --2- 172.21.15.31:0/1796534470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 0x7f3f6406d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.118+0000 7f3f6259c700 1 -- 172.21.15.31:0/1796534470 >> v2:172.21.15.62:3300/0 conn(0x7f3f6406e300 msgr2=0x7f3f64159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.118+0000 7f3f6259c700 1 --2- 172.21.15.31:0/1796534470 >> v2:172.21.15.62:3300/0 conn(0x7f3f6406e300 0x7f3f64159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.118+0000 7f3f6259c700 1 -- 172.21.15.31:0/1796534470 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640c7870 con 0x7f3f6406d830 2023-10-06T05:18:16.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.119+0000 7f3f6259c700 1 --2- 172.21.15.31:0/1796534470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 0x7f3f64159660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3f641571c0 tx=0x7f3f58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff858abf32c6769 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.119+0000 7f3f61d9b700 1 -- 172.21.15.31:0/1796534470 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800b640 con 0x7f3f6406d830 2023-10-06T05:18:16.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.119+0000 7f3f61d9b700 1 -- 172.21.15.31:0/1796534470 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5800b7a0 con 0x7f3f6406d830 2023-10-06T05:18:16.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.119+0000 7f3f61d9b700 1 -- 172.21.15.31:0/1796534470 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800bb60 con 0x7f3f6406d830 2023-10-06T05:18:16.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.119+0000 7f3f691ff700 1 -- 172.21.15.31:0/1796534470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 msgr2=0x7f3f64159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.119+0000 7f3f691ff700 1 --2- 172.21.15.31:0/1796534470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 0x7f3f64159660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3f641571c0 tx=0x7f3f58009250 comp rx=0 tx=0).stop 2023-10-06T05:18:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.120+0000 7f3f691ff700 1 -- 172.21.15.31:0/1796534470 shutdown_connections 2023-10-06T05:18:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.120+0000 7f3f691ff700 1 --2- 172.21.15.31:0/1796534470 >> v2:172.21.15.62:3300/0 conn(0x7f3f6406e300 0x7f3f64159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.120+0000 7f3f691ff700 1 --2- 172.21.15.31:0/1796534470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 0x7f3f6406d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.120+0000 7f3f691ff700 1 --2- 172.21.15.31:0/1796534470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 0x7f3f64159660 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.120+0000 7f3f691ff700 1 -- 172.21.15.31:0/1796534470 >> 172.21.15.31:0/1796534470 conn(0x7f3f64068030 msgr2=0x7f3f640cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:16.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.120+0000 7f3f691ff700 1 -- 172.21.15.31:0/1796534470 shutdown_connections 2023-10-06T05:18:16.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.120+0000 7f3f691ff700 1 -- 172.21.15.31:0/1796534470 wait complete. 2023-10-06T05:18:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.121+0000 7f3f691ff700 1 Processor -- start 2023-10-06T05:18:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.121+0000 7f3f691ff700 1 -- start start 2023-10-06T05:18:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.122+0000 7f3f691ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 0x7f3f640c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.122+0000 7f3f691ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 0x7f3f640ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.122+0000 7f3f691ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f6406e300 0x7f3f64146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.122+0000 7f3f691ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64154de0 con 0x7f3f6406d830 2023-10-06T05:18:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.122+0000 7f3f691ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64147130 con 0x7f3f6406e300 2023-10-06T05:18:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.122+0000 7f3f691ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640ad620 con 0x7f3f6406df30 2023-10-06T05:18:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.122+0000 7f3f62d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 0x7f3f640c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.122+0000 7f3f62d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 0x7f3f640c2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34824/0 (socket says 172.21.15.31:34824) 2023-10-06T05:18:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.122+0000 7f3f62d9d700 1 -- 172.21.15.31:0/3136096537 learned_addr learned my addr 172.21.15.31:0/3136096537 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f6359e700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f6406e300 0x7f3f64146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f6259c700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 0x7f3f640ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f62d9d700 1 -- 172.21.15.31:0/3136096537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 msgr2=0x7f3f640ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f62d9d700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 0x7f3f640ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f62d9d700 1 -- 172.21.15.31:0/3136096537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f6406e300 msgr2=0x7f3f64146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:16.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f62d9d700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f6406e300 0x7f3f64146bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:16.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f62d9d700 1 -- 172.21.15.31:0/3136096537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580094f0 con 0x7f3f6406d830 2023-10-06T05:18:16.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f6359e700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f6406e300 0x7f3f64146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:16.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f6259c700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 0x7f3f640ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:16.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f62d9d700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 0x7f3f640c2750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400ec80 tx=0x7f3f54009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f4affd700 1 -- 172.21.15.31:0/3136096537 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400c9d0 con 0x7f3f6406d830 2023-10-06T05:18:16.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f4affd700 1 -- 172.21.15.31:0/3136096537 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5400cb30 con 0x7f3f6406d830 2023-10-06T05:18:16.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f4affd700 1 -- 172.21.15.31:0/3136096537 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54013470 con 0x7f3f6406d830 2023-10-06T05:18:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.123+0000 7f3f691ff700 1 -- 172.21.15.31:0/3136096537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f640aca10 con 0x7f3f6406d830 2023-10-06T05:18:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.124+0000 7f3f691ff700 1 -- 172.21.15.31:0/3136096537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f640acfd0 con 0x7f3f6406d830 2023-10-06T05:18:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.125+0000 7f3f691ff700 1 -- 172.21.15.31:0/3136096537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6404c380 con 0x7f3f6406d830 2023-10-06T05:18:16.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.125+0000 7f3f4affd700 1 -- 172.21.15.31:0/3136096537 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f54013820 con 0x7f3f6406d830 2023-10-06T05:18:16.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.127+0000 7f3f4affd700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:16.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.127+0000 7f3f4affd700 1 -- 172.21.15.31:0/3136096537 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3f54084ed0 con 0x7f3f6406d830 2023-10-06T05:18:16.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.127+0000 7f3f6259c700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:16.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.128+0000 7f3f6259c700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f50066f30 0x7f3f500693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f641571c0 tx=0x7f3f58017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:16.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.132+0000 7f3f4affd700 1 -- 172.21.15.31:0/3136096537 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f54051f40 con 0x7f3f6406d830 2023-10-06T05:18:16.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:16.383+0000 7f3f691ff700 1 -- 172.21.15.31:0/3136096537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3f640c7b50 con 0x7f3f6406d830 2023-10-06T05:18:17.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.033+0000 7f3f4affd700 1 -- 172.21.15.31:0/3136096537 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3f54055560 con 0x7f3f6406d830 2023-10-06T05:18:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.036+0000 7f3f691ff700 1 -- 172.21.15.31:0/3136096537 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f50066f30 msgr2=0x7f3f500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.036+0000 7f3f691ff700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f50066f30 0x7f3f500693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f641571c0 tx=0x7f3f58017000 comp rx=0 tx=0).stop 2023-10-06T05:18:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.036+0000 7f3f691ff700 1 -- 172.21.15.31:0/3136096537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 msgr2=0x7f3f640c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.036+0000 7f3f691ff700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 0x7f3f640c2750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400ec80 tx=0x7f3f54009120 comp rx=0 tx=0).stop 2023-10-06T05:18:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.036+0000 7f3f691ff700 1 -- 172.21.15.31:0/3136096537 shutdown_connections 2023-10-06T05:18:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.036+0000 7f3f691ff700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f6406e300 0x7f3f64146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.036+0000 7f3f691ff700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f50066f30 0x7f3f500693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.036+0000 7f3f691ff700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6406d830 0x7f3f640c2750 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.036+0000 7f3f691ff700 1 --2- 172.21.15.31:0/3136096537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f6406df30 0x7f3f640ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.036+0000 7f3f691ff700 1 -- 172.21.15.31:0/3136096537 >> 172.21.15.31:0/3136096537 conn(0x7f3f64068030 msgr2=0x7f3f64158460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.037+0000 7f3f691ff700 1 -- 172.21.15.31:0/3136096537 shutdown_connections 2023-10-06T05:18:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.037+0000 7f3f691ff700 1 -- 172.21.15.31:0/3136096537 wait complete. 2023-10-06T05:18:17.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:18:17.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.257+0000 7f349c7ff700 1 Processor -- start 2023-10-06T05:18:17.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.257+0000 7f349c7ff700 1 -- start start 2023-10-06T05:18:17.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.258+0000 7f349c7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34940c66f0 0x7f34941569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.258+0000 7f349c7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3494069560 0x7f34940ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.258+0000 7f349c7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34940699d0 0x7f349406ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.258+0000 7f349c7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34940c7a70 con 0x7f34940c66f0 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.258+0000 7f349c7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3494156ef0 con 0x7f3494069560 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.258+0000 7f349c7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34940c6df0 con 0x7f34940699d0 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f349a59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34940c66f0 0x7f34941569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f349a59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34940c66f0 0x7f34941569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58782/0 (socket says 172.21.15.31:58782) 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f349a59b700 1 -- 172.21.15.31:0/741174726 learned_addr learned my addr 172.21.15.31:0/741174726 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f349ad9c700 1 --2- 172.21.15.31:0/741174726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34940699d0 0x7f349406ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f3499d9a700 1 --2- 172.21.15.31:0/741174726 >> v2:172.21.15.62:3300/0 conn(0x7f3494069560 0x7f34940ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f349a59b700 1 -- 172.21.15.31:0/741174726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34940699d0 msgr2=0x7f349406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f349a59b700 1 --2- 172.21.15.31:0/741174726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34940699d0 0x7f349406ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f349a59b700 1 -- 172.21.15.31:0/741174726 >> v2:172.21.15.62:3300/0 conn(0x7f3494069560 msgr2=0x7f34940ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f349a59b700 1 --2- 172.21.15.31:0/741174726 >> v2:172.21.15.62:3300/0 conn(0x7f3494069560 0x7f34940ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f349a59b700 1 -- 172.21.15.31:0/741174726 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34940c70d0 con 0x7f34940c66f0 2023-10-06T05:18:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.259+0000 7f349a59b700 1 --2- 172.21.15.31:0/741174726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34940c66f0 0x7f34941569b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f34940ca030 tx=0x7f3490009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba019084917a87b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f3499599700 1 -- 172.21.15.31:0/741174726 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490005c10 con 0x7f34940c66f0 2023-10-06T05:18:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f3499599700 1 -- 172.21.15.31:0/741174726 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3490005d70 con 0x7f34940c66f0 2023-10-06T05:18:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f3499599700 1 -- 172.21.15.31:0/741174726 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000b880 con 0x7f34940c66f0 2023-10-06T05:18:17.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f349c7ff700 1 -- 172.21.15.31:0/741174726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34940c66f0 msgr2=0x7f34941569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f349c7ff700 1 --2- 172.21.15.31:0/741174726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34940c66f0 0x7f34941569b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f34940ca030 tx=0x7f3490009220 comp rx=0 tx=0).stop 2023-10-06T05:18:17.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f349c7ff700 1 -- 172.21.15.31:0/741174726 shutdown_connections 2023-10-06T05:18:17.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f349c7ff700 1 --2- 172.21.15.31:0/741174726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34940699d0 0x7f349406ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f349c7ff700 1 --2- 172.21.15.31:0/741174726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34940c66f0 0x7f34941569b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f349c7ff700 1 --2- 172.21.15.31:0/741174726 >> v2:172.21.15.62:3300/0 conn(0x7f3494069560 0x7f34940ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f349c7ff700 1 -- 172.21.15.31:0/741174726 >> 172.21.15.31:0/741174726 conn(0x7f34940cc0b0 msgr2=0x7f34940cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f349c7ff700 1 -- 172.21.15.31:0/741174726 shutdown_connections 2023-10-06T05:18:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.260+0000 7f349c7ff700 1 -- 172.21.15.31:0/741174726 wait complete. 2023-10-06T05:18:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.262+0000 7f349c7ff700 1 Processor -- start 2023-10-06T05:18:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.262+0000 7f349c7ff700 1 -- start start 2023-10-06T05:18:17.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f349c7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3494069560 0x7f349414a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f349c7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940699d0 0x7f34940b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f349c7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34940c66f0 0x7f34940c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f349c7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494148b00 con 0x7f3494069560 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f349c7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34940b1c50 con 0x7f34940699d0 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f349c7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34941481d0 con 0x7f34940c66f0 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f349a59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3494069560 0x7f349414a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f349a59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3494069560 0x7f349414a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58794/0 (socket says 172.21.15.31:58794) 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f349a59b700 1 -- 172.21.15.31:0/3954800683 learned_addr learned my addr 172.21.15.31:0/3954800683 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f3499d9a700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940699d0 0x7f34940b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.263+0000 7f349ad9c700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34940c66f0 0x7f34940c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f349a59b700 1 -- 172.21.15.31:0/3954800683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34940c66f0 msgr2=0x7f34940c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f349a59b700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34940c66f0 0x7f34940c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f349a59b700 1 -- 172.21.15.31:0/3954800683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940699d0 msgr2=0x7f34940b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f349a59b700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940699d0 0x7f34940b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f349a59b700 1 -- 172.21.15.31:0/3954800683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3490009480 con 0x7f3494069560 2023-10-06T05:18:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f349a59b700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3494069560 0x7f349414a380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3490007fd0 tx=0x7f3490016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f348a7fc700 1 -- 172.21.15.31:0/3954800683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490011b30 con 0x7f3494069560 2023-10-06T05:18:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f348a7fc700 1 -- 172.21.15.31:0/3954800683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3490011cb0 con 0x7f3494069560 2023-10-06T05:18:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f349c7ff700 1 -- 172.21.15.31:0/3954800683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34941484b0 con 0x7f3494069560 2023-10-06T05:18:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f348a7fc700 1 -- 172.21.15.31:0/3954800683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349001b680 con 0x7f3494069560 2023-10-06T05:18:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.264+0000 7f349c7ff700 1 -- 172.21.15.31:0/3954800683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34940c46b0 con 0x7f3494069560 2023-10-06T05:18:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.265+0000 7f349c7ff700 1 -- 172.21.15.31:0/3954800683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349404c380 con 0x7f3494069560 2023-10-06T05:18:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.266+0000 7f348a7fc700 1 -- 172.21.15.31:0/3954800683 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f349001ba70 con 0x7f3494069560 2023-10-06T05:18:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.267+0000 7f348a7fc700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3478066f30 0x7f34780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.267+0000 7f348a7fc700 1 -- 172.21.15.31:0/3954800683 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3490086640 con 0x7f3494069560 2023-10-06T05:18:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.267+0000 7f3499d9a700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3478066f30 0x7f34780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.268+0000 7f3499d9a700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3478066f30 0x7f34780693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f34940710a0 tx=0x7f3484008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:17.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.271+0000 7f348a7fc700 1 -- 172.21.15.31:0/3954800683 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34900536b0 con 0x7f3494069560 2023-10-06T05:18:17.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.501+0000 7f349c7ff700 1 -- 172.21.15.31:0/3954800683 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3494148c40 con 0x7f3478066f30 2023-10-06T05:18:17.505 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:18:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:18:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:17.505 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:18:17.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.503+0000 7f348a7fc700 1 -- 172.21.15.31:0/3954800683 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f348400f6e0 con 0x7f3478066f30 2023-10-06T05:18:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.505+0000 7f349c7ff700 1 -- 172.21.15.31:0/3954800683 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3478066f30 msgr2=0x7f34780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.505+0000 7f349c7ff700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3478066f30 0x7f34780693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f34940710a0 tx=0x7f3484008040 comp rx=0 tx=0).stop 2023-10-06T05:18:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.505+0000 7f349c7ff700 1 -- 172.21.15.31:0/3954800683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3494069560 msgr2=0x7f349414a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.505+0000 7f349c7ff700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3494069560 0x7f349414a380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3490007fd0 tx=0x7f3490016000 comp rx=0 tx=0).stop 2023-10-06T05:18:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.505+0000 7f349c7ff700 1 -- 172.21.15.31:0/3954800683 shutdown_connections 2023-10-06T05:18:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.505+0000 7f349c7ff700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34940c66f0 0x7f34940c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.505+0000 7f349c7ff700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3478066f30 0x7f34780693e0 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f34940710a0 tx=0x7f3484008040 comp rx=0 tx=0).stop 2023-10-06T05:18:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.505+0000 7f349c7ff700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3494069560 0x7f349414a380 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.505+0000 7f349c7ff700 1 --2- 172.21.15.31:0/3954800683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940699d0 0x7f34940b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.506+0000 7f349c7ff700 1 -- 172.21.15.31:0/3954800683 >> 172.21.15.31:0/3954800683 conn(0x7f34940cc0b0 msgr2=0x7f349406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:17.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.506+0000 7f349c7ff700 1 -- 172.21.15.31:0/3954800683 shutdown_connections 2023-10-06T05:18:17.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.506+0000 7f349c7ff700 1 -- 172.21.15.31:0/3954800683 wait complete. 2023-10-06T05:18:17.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:18:17.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.720+0000 7f6af17ff700 1 Processor -- start 2023-10-06T05:18:17.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.721+0000 7f6af17ff700 1 -- start start 2023-10-06T05:18:17.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.722+0000 7f6af17ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6aec0b2540 0x7f6aec0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.722+0000 7f6af17ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec15a4b0 0x7f6aec0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.722+0000 7f6af17ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec15a020 con 0x7f6aec0cc340 2023-10-06T05:18:17.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.722+0000 7f6af17ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec0ccc50 con 0x7f6aec0b2540 2023-10-06T05:18:17.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.722+0000 7f6af17ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0b07e0 con 0x7f6aec15a4b0 2023-10-06T05:18:17.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.722+0000 7f6aea7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6aec0b2540 0x7f6aec0cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.722+0000 7f6aea7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6aec0b2540 0x7f6aec0cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58806/0 (socket says 172.21.15.31:58806) 2023-10-06T05:18:17.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.722+0000 7f6aeaffd700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6aec0cc340 0x7f6aec0b00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:53210/0 (socket says 172.21.15.31:53210) 2023-10-06T05:18:17.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.722+0000 7f6aea7fc700 1 -- 172.21.15.31:0/2721656561 learned_addr learned my addr 172.21.15.31:0/2721656561 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:17.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.722+0000 7f6aeb7fe700 1 --2- 172.21.15.31:0/2721656561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec15a4b0 0x7f6aec0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.723+0000 7f6aea7fc700 1 -- 172.21.15.31:0/2721656561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec15a4b0 msgr2=0x7f6aec0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:17.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.723+0000 7f6aea7fc700 1 --2- 172.21.15.31:0/2721656561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec15a4b0 0x7f6aec0aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.723+0000 7f6ae9ffb700 1 -- 172.21.15.31:0/2721656561 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987917259 0 0) 0x7f6aec15a020 con 0x7f6aec0cc340 2023-10-06T05:18:17.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.723+0000 7f6aea7fc700 1 -- 172.21.15.31:0/2721656561 >> v1:172.21.15.62:6789/0 conn(0x7f6aec0cc340 legacy=0x7f6aec0b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.723+0000 7f6aea7fc700 1 -- 172.21.15.31:0/2721656561 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec159cf0 con 0x7f6aec0b2540 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.723+0000 7f6aea7fc700 1 --2- 172.21.15.31:0/2721656561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0b2540 0x7f6aec0cc710 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00099e0 tx=0x7f6ae0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493775920b3303e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.723+0000 7f6ae9ffb700 1 -- 172.21.15.31:0/2721656561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000b820 con 0x7f6aec0b2540 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.723+0000 7f6ae9ffb700 1 -- 172.21.15.31:0/2721656561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae000b980 con 0x7f6aec0b2540 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.723+0000 7f6ae9ffb700 1 -- 172.21.15.31:0/2721656561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000bd20 con 0x7f6aec0b2540 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.724+0000 7f6af17ff700 1 -- 172.21.15.31:0/2721656561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0b2540 msgr2=0x7f6aec0cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.724+0000 7f6af17ff700 1 --2- 172.21.15.31:0/2721656561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0b2540 0x7f6aec0cc710 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00099e0 tx=0x7f6ae0009250 comp rx=0 tx=0).stop 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.724+0000 7f6af17ff700 1 -- 172.21.15.31:0/2721656561 shutdown_connections 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.724+0000 7f6af17ff700 1 --2- 172.21.15.31:0/2721656561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec15a4b0 0x7f6aec0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.724+0000 7f6af17ff700 1 --2- 172.21.15.31:0/2721656561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0b2540 0x7f6aec0cc710 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.724+0000 7f6af17ff700 1 -- 172.21.15.31:0/2721656561 >> 172.21.15.31:0/2721656561 conn(0x7f6aec068670 msgr2=0x7f6aec068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.724+0000 7f6af17ff700 1 -- 172.21.15.31:0/2721656561 shutdown_connections 2023-10-06T05:18:17.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.724+0000 7f6af17ff700 1 -- 172.21.15.31:0/2721656561 wait complete. 2023-10-06T05:18:17.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.726+0000 7f6af17ff700 1 Processor -- start 2023-10-06T05:18:17.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.726+0000 7f6af17ff700 1 -- start start 2023-10-06T05:18:17.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6af17ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec0b2540 0x7f6aec0af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6af17ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec0cc340 0x7f6aec16ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6af17ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15a4b0 0x7f6aec158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6af17ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec15cc50 con 0x7f6aec15a4b0 2023-10-06T05:18:17.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6af17ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec16ab40 con 0x7f6aec0b2540 2023-10-06T05:18:17.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6af17ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec16a860 con 0x7f6aec0cc340 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6aeb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15a4b0 0x7f6aec158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6aeb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15a4b0 0x7f6aec158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58820/0 (socket says 172.21.15.31:58820) 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6aeb7fe700 1 -- 172.21.15.31:0/423411757 learned_addr learned my addr 172.21.15.31:0/423411757 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6aeaffd700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec0b2540 0x7f6aec0af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6aea7fc700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec0cc340 0x7f6aec16ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6aeb7fe700 1 -- 172.21.15.31:0/423411757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec0cc340 msgr2=0x7f6aec16ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6aeb7fe700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec0cc340 0x7f6aec16ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6aeb7fe700 1 -- 172.21.15.31:0/423411757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec0b2540 msgr2=0x7f6aec0af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6aeb7fe700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec0b2540 0x7f6aec0af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.728+0000 7f6aeb7fe700 1 -- 172.21.15.31:0/423411757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae00094f0 con 0x7f6aec15a4b0 2023-10-06T05:18:17.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.729+0000 7f6aeb7fe700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15a4b0 0x7f6aec158b80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009e90 tx=0x7f6ae400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:17.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.729+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/423411757 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae400cb70 con 0x7f6aec15a4b0 2023-10-06T05:18:17.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.729+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/423411757 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae400f040 con 0x7f6aec15a4b0 2023-10-06T05:18:17.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.729+0000 7f6af17ff700 1 -- 172.21.15.31:0/423411757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec0acd50 con 0x7f6aec15a4b0 2023-10-06T05:18:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.729+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/423411757 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4014480 con 0x7f6aec15a4b0 2023-10-06T05:18:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.729+0000 7f6af17ff700 1 -- 172.21.15.31:0/423411757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec0ad310 con 0x7f6aec15a4b0 2023-10-06T05:18:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.730+0000 7f6af17ff700 1 -- 172.21.15.31:0/423411757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec0cc180 con 0x7f6aec15a4b0 2023-10-06T05:18:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.731+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/423411757 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ae4014830 con 0x7f6aec15a4b0 2023-10-06T05:18:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.732+0000 7f6ad2ffd700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ad8066f30 0x7f6ad80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:17.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.732+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/423411757 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ae4085160 con 0x7f6aec15a4b0 2023-10-06T05:18:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.732+0000 7f6aeaffd700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ad8066f30 0x7f6ad80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:17.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.733+0000 7f6aeaffd700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ad8066f30 0x7f6ad80693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6adc000fb0 tx=0x7f6adc007ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:17.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.736+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/423411757 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ae40521d0 con 0x7f6aec15a4b0 2023-10-06T05:18:17.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.939+0000 7f6af17ff700 1 -- 172.21.15.31:0/423411757 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6aec04c380 con 0x7f6aec15a4b0 2023-10-06T05:18:17.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.940+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/423411757 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ae40557f0 con 0x7f6aec15a4b0 2023-10-06T05:18:17.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 -- 172.21.15.31:0/423411757 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ad8066f30 msgr2=0x7f6ad80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ad8066f30 0x7f6ad80693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6adc000fb0 tx=0x7f6adc007ce0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 -- 172.21.15.31:0/423411757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15a4b0 msgr2=0x7f6aec158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:17.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15a4b0 0x7f6aec158b80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009e90 tx=0x7f6ae400b040 comp rx=0 tx=0).stop 2023-10-06T05:18:17.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 -- 172.21.15.31:0/423411757 shutdown_connections 2023-10-06T05:18:17.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ad8066f30 0x7f6ad80693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15a4b0 0x7f6aec158b80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec0b2540 0x7f6aec0af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 --2- 172.21.15.31:0/423411757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6aec0cc340 0x7f6aec16ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:17.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 -- 172.21.15.31:0/423411757 >> 172.21.15.31:0/423411757 conn(0x7f6aec068670 msgr2=0x7f6aec1547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:17.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 -- 172.21.15.31:0/423411757 shutdown_connections 2023-10-06T05:18:17.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:17.942+0000 7f6af17ff700 1 -- 172.21.15.31:0/423411757 wait complete. 2023-10-06T05:18:17.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:17.962 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/group_0000000000618160/subvolume_0000000000721169) 2023-10-06T05:18:18.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:18.042 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T05:18:18.099 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T05:18:18.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:18.099 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/group_0000000000618160/subvolume_0000000000721169) 2023-10-06T05:18:18.189 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:18:18.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/group_0000000000618160/subvolume_0000000000721169/.meta 2023-10-06T05:18:18.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000721169 group_0000000000618160 2023-10-06T05:18:18.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.504+0000 7fee24fff700 1 Processor -- start 2023-10-06T05:18:18.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.504+0000 7fee24fff700 1 -- start start 2023-10-06T05:18:18.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.505+0000 7fee24fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee2006e300 0x7fee20159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.505+0000 7fee24fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee20159fe0 con 0x7fee20159ba0 2023-10-06T05:18:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.505+0000 7fee24fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee200c72b0 con 0x7fee2006df30 2023-10-06T05:18:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.505+0000 7fee24fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee200aba60 con 0x7fee2006e300 2023-10-06T05:18:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1ed9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fee20159ba0 0x7fee20152cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56244/0 (socket says 172.21.15.31:56244) 2023-10-06T05:18:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1ed9d700 1 -- 172.21.15.31:0/193088851 learned_addr learned my addr 172.21.15.31:0/193088851 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1dd9b700 1 --2- 172.21.15.31:0/193088851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee2006e300 0x7fee20159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1d59a700 1 -- 172.21.15.31:0/193088851 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111576239 0 0) 0x7fee20159fe0 con 0x7fee20159ba0 2023-10-06T05:18:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1d59a700 1 -- 172.21.15.31:0/193088851 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee08003570 con 0x7fee20159ba0 2023-10-06T05:18:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1d59a700 1 -- 172.21.15.31:0/193088851 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219855235 0 0) 0x7fee200c72b0 con 0x7fee2006df30 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1d59a700 1 -- 172.21.15.31:0/193088851 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee20159fe0 con 0x7fee2006df30 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1d59a700 1 -- 172.21.15.31:0/193088851 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 488712621 0 0) 0x7fee08003570 con 0x7fee20159ba0 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1dd9b700 1 -- 172.21.15.31:0/193088851 >> v1:172.21.15.62:6789/0 conn(0x7fee2006df30 legacy=0x7fee2006d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1dd9b700 1 -- 172.21.15.31:0/193088851 >> v1:172.21.15.31:6789/0 conn(0x7fee20159ba0 legacy=0x7fee20152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.506+0000 7fee1dd9b700 1 -- 172.21.15.31:0/193088851 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200c7590 con 0x7fee2006e300 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.507+0000 7fee1dd9b700 1 --2- 172.21.15.31:0/193088851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee2006e300 0x7fee20159660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee1400aac0 tx=0x7fee1400a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4007dfeb09dd4335 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.507+0000 7fee1d59a700 1 -- 172.21.15.31:0/193088851 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee180017f0 con 0x7fee2006e300 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.507+0000 7fee1d59a700 1 -- 172.21.15.31:0/193088851 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1400c660 con 0x7fee2006e300 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.507+0000 7fee1d59a700 1 -- 172.21.15.31:0/193088851 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1400ca00 con 0x7fee2006e300 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.508+0000 7fee24fff700 1 -- 172.21.15.31:0/193088851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee2006e300 msgr2=0x7fee20159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.508+0000 7fee24fff700 1 --2- 172.21.15.31:0/193088851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee2006e300 0x7fee20159660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee1400aac0 tx=0x7fee1400a330 comp rx=0 tx=0).stop 2023-10-06T05:18:18.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.508+0000 7fee24fff700 1 -- 172.21.15.31:0/193088851 shutdown_connections 2023-10-06T05:18:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.508+0000 7fee24fff700 1 --2- 172.21.15.31:0/193088851 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee2006e300 0x7fee20159660 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.508+0000 7fee24fff700 1 -- 172.21.15.31:0/193088851 >> 172.21.15.31:0/193088851 conn(0x7fee20068030 msgr2=0x7fee200cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.508+0000 7fee24fff700 1 -- 172.21.15.31:0/193088851 shutdown_connections 2023-10-06T05:18:18.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.508+0000 7fee24fff700 1 -- 172.21.15.31:0/193088851 wait complete. 2023-10-06T05:18:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.509+0000 7fee24fff700 1 Processor -- start 2023-10-06T05:18:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.509+0000 7fee24fff700 1 -- start start 2023-10-06T05:18:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.510+0000 7fee24fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee2006df30 0x7fee200ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.510+0000 7fee24fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee2006e300 0x7fee20158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.510+0000 7fee24fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee20159ba0 0x7fee200aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.510+0000 7fee24fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20147050 con 0x7fee2006df30 2023-10-06T05:18:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.510+0000 7fee24fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee200c0890 con 0x7fee2006e300 2023-10-06T05:18:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.510+0000 7fee24fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee200bfc80 con 0x7fee20159ba0 2023-10-06T05:18:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee2006df30 0x7fee200ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee2006df30 0x7fee200ad640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58834/0 (socket says 172.21.15.31:58834) 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1e59c700 1 -- 172.21.15.31:0/3558005665 learned_addr learned my addr 172.21.15.31:0/3558005665 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1dd9b700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee2006e300 0x7fee20158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1ed9d700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee20159ba0 0x7fee200aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1e59c700 1 -- 172.21.15.31:0/3558005665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee20159ba0 msgr2=0x7fee200aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1e59c700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee20159ba0 0x7fee200aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1e59c700 1 -- 172.21.15.31:0/3558005665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee2006e300 msgr2=0x7fee20158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1e59c700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee2006e300 0x7fee20158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1e59c700 1 -- 172.21.15.31:0/3558005665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1400a5d0 con 0x7fee2006df30 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1ed9d700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee20159ba0 0x7fee200aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee1e59c700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee2006df30 0x7fee200ad640 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fee1000c7f0 tx=0x7fee10007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.511+0000 7fee0e7fc700 1 -- 172.21.15.31:0/3558005665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10011680 con 0x7fee2006df30 2023-10-06T05:18:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.512+0000 7fee0e7fc700 1 -- 172.21.15.31:0/3558005665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee100117e0 con 0x7fee2006df30 2023-10-06T05:18:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.512+0000 7fee0e7fc700 1 -- 172.21.15.31:0/3558005665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10011ba0 con 0x7fee2006df30 2023-10-06T05:18:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.512+0000 7fee24fff700 1 -- 172.21.15.31:0/3558005665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee200bff60 con 0x7fee2006df30 2023-10-06T05:18:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.512+0000 7fee24fff700 1 -- 172.21.15.31:0/3558005665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee200c0520 con 0x7fee2006df30 2023-10-06T05:18:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.513+0000 7fee24fff700 1 -- 172.21.15.31:0/3558005665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee200bf9d0 con 0x7fee2006df30 2023-10-06T05:18:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.514+0000 7fee0e7fc700 1 -- 172.21.15.31:0/3558005665 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee10012440 con 0x7fee2006df30 2023-10-06T05:18:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.515+0000 7fee0e7fc700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fee00066f30 0x7fee000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.515+0000 7fee0e7fc700 1 -- 172.21.15.31:0/3558005665 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fee10084610 con 0x7fee2006df30 2023-10-06T05:18:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.516+0000 7fee1dd9b700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fee00066f30 0x7fee000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.516+0000 7fee1dd9b700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fee00066f30 0x7fee000693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee1400aa10 tx=0x7fee14019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.519+0000 7fee0e7fc700 1 -- 172.21.15.31:0/3558005665 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee10051680 con 0x7fee2006df30 2023-10-06T05:18:18.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.754+0000 7fee24fff700 1 -- 172.21.15.31:0/3558005665 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000721169", "group_name": "group_0000000000618160", "target": ["mon-mgr", ""]}) v1 -- 0x7fee200c09d0 con 0x7fee00066f30 2023-10-06T05:18:18.762 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:18.760+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:18.762 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:18.760+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:18.763 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:18.760+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:18.763 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:18.760+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:18.763 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:18.760+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.804+0000 7fee0e7fc700 1 -- 172.21.15.31:0/3558005665 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fee14000c50 con 0x7fee00066f30 2023-10-06T05:18:18.806 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000618160/subvolume_0000000000721169 2023-10-06T05:18:18.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 -- 172.21.15.31:0/3558005665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fee00066f30 msgr2=0x7fee000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fee00066f30 0x7fee000693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee1400aa10 tx=0x7fee14019000 comp rx=0 tx=0).stop 2023-10-06T05:18:18.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 -- 172.21.15.31:0/3558005665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee2006df30 msgr2=0x7fee200ad640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:18.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee2006df30 0x7fee200ad640 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fee1000c7f0 tx=0x7fee10007e00 comp rx=0 tx=0).stop 2023-10-06T05:18:18.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 -- 172.21.15.31:0/3558005665 shutdown_connections 2023-10-06T05:18:18.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee20159ba0 0x7fee200aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fee00066f30 0x7fee000693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee2006df30 0x7fee200ad640 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 --2- 172.21.15.31:0/3558005665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee2006e300 0x7fee20158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:18.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 -- 172.21.15.31:0/3558005665 >> 172.21.15.31:0/3558005665 conn(0x7fee20068030 msgr2=0x7fee20155000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:18.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 -- 172.21.15.31:0/3558005665 shutdown_connections 2023-10-06T05:18:18.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:18.807+0000 7fee24fff700 1 -- 172.21.15.31:0/3558005665 wait complete. 2023-10-06T05:18:18.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000721169 group_0000000000618160 2023-10-06T05:18:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.023+0000 7fca169ff700 1 Processor -- start 2023-10-06T05:18:19.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.023+0000 7fca169ff700 1 -- start start 2023-10-06T05:18:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca169ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca100cf7a0 0x7fca1015a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca169ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca1015a580 0x7fca10163230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca169ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca100cd050 con 0x7fca10159c70 2023-10-06T05:18:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca169ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca100cde90 con 0x7fca100cf7a0 2023-10-06T05:18:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca169ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca100ce020 con 0x7fca1015a580 2023-10-06T05:18:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca14f9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca1015a580 0x7fca10163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca0f7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca100cf7a0 0x7fca1015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca14f9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca1015a580 0x7fca10163230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:45662/0 (socket says 172.21.15.31:45662) 2023-10-06T05:18:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca0f7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca100cf7a0 0x7fca1015a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:50450/0 (socket says 172.21.15.31:50450) 2023-10-06T05:18:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca14f9c700 1 -- 172.21.15.31:0/2319490417 learned_addr learned my addr 172.21.15.31:0/2319490417 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.025+0000 7fca0ffff700 1 --1- 172.21.15.31:0/2319490417 >> v1:172.21.15.62:6789/0 conn(0x7fca10159c70 0x7fca100caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:53232/0 (socket says 172.21.15.31:53232) 2023-10-06T05:18:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.026+0000 7fca0f7fe700 1 -- 172.21.15.31:0/2319490417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca1015a580 msgr2=0x7fca10163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.026+0000 7fca0f7fe700 1 --2- 172.21.15.31:0/2319490417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca1015a580 0x7fca10163230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.026+0000 7fca0f7fe700 1 -- 172.21.15.31:0/2319490417 >> v1:172.21.15.62:6789/0 conn(0x7fca10159c70 legacy=0x7fca100caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.026+0000 7fca0f7fe700 1 -- 172.21.15.31:0/2319490417 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca100ce300 con 0x7fca100cf7a0 2023-10-06T05:18:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.026+0000 7fca14f9c700 1 --2- 172.21.15.31:0/2319490417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca1015a580 0x7fca10163230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.026+0000 7fca0effd700 1 -- 172.21.15.31:0/2319490417 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536218044 0 0) 0x7fca100cd050 con 0x7fca10159c70 2023-10-06T05:18:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.026+0000 7fca0f7fe700 1 --2- 172.21.15.31:0/2319490417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca100cf7a0 0x7fca1015a040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fca040098e0 tx=0x7fca040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d970106b32cf311a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca0effd700 1 -- 172.21.15.31:0/2319490417 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400b640 con 0x7fca100cf7a0 2023-10-06T05:18:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca0effd700 1 -- 172.21.15.31:0/2319490417 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0400b7a0 con 0x7fca100cf7a0 2023-10-06T05:18:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca0effd700 1 -- 172.21.15.31:0/2319490417 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400bb40 con 0x7fca100cf7a0 2023-10-06T05:18:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca169ff700 1 -- 172.21.15.31:0/2319490417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca100cf7a0 msgr2=0x7fca1015a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca169ff700 1 --2- 172.21.15.31:0/2319490417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca100cf7a0 0x7fca1015a040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fca040098e0 tx=0x7fca040091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca169ff700 1 -- 172.21.15.31:0/2319490417 shutdown_connections 2023-10-06T05:18:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca169ff700 1 --2- 172.21.15.31:0/2319490417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca1015a580 0x7fca10163230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca169ff700 1 --2- 172.21.15.31:0/2319490417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca100cf7a0 0x7fca1015a040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca169ff700 1 -- 172.21.15.31:0/2319490417 >> 172.21.15.31:0/2319490417 conn(0x7fca10099710 msgr2=0x7fca1006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca169ff700 1 -- 172.21.15.31:0/2319490417 shutdown_connections 2023-10-06T05:18:19.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.027+0000 7fca169ff700 1 -- 172.21.15.31:0/2319490417 wait complete. 2023-10-06T05:18:19.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.029+0000 7fca169ff700 1 Processor -- start 2023-10-06T05:18:19.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.030+0000 7fca169ff700 1 -- start start 2023-10-06T05:18:19.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca169ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100cf7a0 0x7fca10157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca169ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159c70 0x7fca100b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca169ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1015a580 0x7fca10155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca169ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca100b5ec0 con 0x7fca1015a580 2023-10-06T05:18:19.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca169ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca100b5340 con 0x7fca10159c70 2023-10-06T05:18:19.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca169ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca100b5620 con 0x7fca100cf7a0 2023-10-06T05:18:19.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca14f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1015a580 0x7fca10155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca14f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1015a580 0x7fca10155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58860/0 (socket says 172.21.15.31:58860) 2023-10-06T05:18:19.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca14f9c700 1 -- 172.21.15.31:0/1014658038 learned_addr learned my addr 172.21.15.31:0/1014658038 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:19.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca0ffff700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100cf7a0 0x7fca10157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.031+0000 7fca0f7fe700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159c70 0x7fca100b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fca14f9c700 1 -- 172.21.15.31:0/1014658038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100cf7a0 msgr2=0x7fca10157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fca14f9c700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100cf7a0 0x7fca10157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fca14f9c700 1 -- 172.21.15.31:0/1014658038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159c70 msgr2=0x7fca100b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fca14f9c700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159c70 0x7fca100b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fca14f9c700 1 -- 172.21.15.31:0/1014658038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca08009ae0 con 0x7fca1015a580 2023-10-06T05:18:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fca14f9c700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1015a580 0x7fca10155e10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fca08009fd0 tx=0x7fca0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fc9f7fff700 1 -- 172.21.15.31:0/1014658038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08018070 con 0x7fca1015a580 2023-10-06T05:18:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fc9f7fff700 1 -- 172.21.15.31:0/1014658038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0800f040 con 0x7fca1015a580 2023-10-06T05:18:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fc9f7fff700 1 -- 172.21.15.31:0/1014658038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0800cd50 con 0x7fca1015a580 2023-10-06T05:18:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fca169ff700 1 -- 172.21.15.31:0/1014658038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca04009420 con 0x7fca1015a580 2023-10-06T05:18:19.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.032+0000 7fca169ff700 1 -- 172.21.15.31:0/1014658038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca100b5be0 con 0x7fca1015a580 2023-10-06T05:18:19.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.033+0000 7fca169ff700 1 -- 172.21.15.31:0/1014658038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca100c7c90 con 0x7fca1015a580 2023-10-06T05:18:19.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.035+0000 7fc9f7fff700 1 -- 172.21.15.31:0/1014658038 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca08003020 con 0x7fca1015a580 2023-10-06T05:18:19.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.035+0000 7fc9f7fff700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9fc066f30 0x7fc9fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.036+0000 7fc9f7fff700 1 -- 172.21.15.31:0/1014658038 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca080846c0 con 0x7fca1015a580 2023-10-06T05:18:19.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.036+0000 7fca0ffff700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9fc066f30 0x7fc9fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.036+0000 7fca0ffff700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9fc066f30 0x7fc9fc0693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fca00001230 tx=0x7fca00007b30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:19.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.040+0000 7fc9f7fff700 1 -- 172.21.15.31:0/1014658038 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca0804d5a0 con 0x7fca1015a580 2023-10-06T05:18:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.274+0000 7fca169ff700 1 -- 172.21.15.31:0/1014658038 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000721169", "group_name": "group_0000000000618160", "target": ["mon-mgr", ""]}) v1 -- 0x7fca1004c380 con 0x7fc9fc066f30 2023-10-06T05:18:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.279+0000 7fc9f7fff700 1 -- 172.21.15.31:0/1014658038 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fca1004c380 con 0x7fc9fc066f30 2023-10-06T05:18:19.281 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000618160/subvolume_0000000000721169 2023-10-06T05:18:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 -- 172.21.15.31:0/1014658038 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9fc066f30 msgr2=0x7fc9fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9fc066f30 0x7fc9fc0693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fca00001230 tx=0x7fca00007b30 comp rx=0 tx=0).stop 2023-10-06T05:18:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 -- 172.21.15.31:0/1014658038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1015a580 msgr2=0x7fca10155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1015a580 0x7fca10155e10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fca08009fd0 tx=0x7fca0800b040 comp rx=0 tx=0).stop 2023-10-06T05:18:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 -- 172.21.15.31:0/1014658038 shutdown_connections 2023-10-06T05:18:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc9fc066f30 0x7fc9fc0693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca1015a580 0x7fca10155e10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca100cf7a0 0x7fca10157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 --2- 172.21.15.31:0/1014658038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca10159c70 0x7fca100b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 -- 172.21.15.31:0/1014658038 >> 172.21.15.31:0/1014658038 conn(0x7fca10099710 msgr2=0x7fca10068240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.281+0000 7fca169ff700 1 -- 172.21.15.31:0/1014658038 shutdown_connections 2023-10-06T05:18:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.282+0000 7fca169ff700 1 -- 172.21.15.31:0/1014658038 wait complete. 2023-10-06T05:18:19.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:19.300 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/4979057dae2a71444ee78c3b55d20246.meta) 2023-10-06T05:18:19.377 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:18:19.377 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T05:18:19.377 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T05:18:19.377 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/group_0000000000618160/subvolume_0000000000721169 2023-10-06T05:18:19.378 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:18:19.378 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:19.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000721169 group_0000000000618160 2023-10-06T05:18:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.578+0000 7fce933ff700 1 Processor -- start 2023-10-06T05:18:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.578+0000 7fce933ff700 1 -- start start 2023-10-06T05:18:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.579+0000 7fce933ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fce8c153950 0x7fce8c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.579+0000 7fce933ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce8c0cd5d0 0x7fce8c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.579+0000 7fce933ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8c15cfc0 con 0x7fce8c0cda20 2023-10-06T05:18:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.579+0000 7fce933ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c15bb20 con 0x7fce8c153950 2023-10-06T05:18:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.580+0000 7fce933ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c15af10 con 0x7fce8c0cd5d0 2023-10-06T05:18:19.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.580+0000 7fce9119b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fce8c153950 0x7fce8c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.580+0000 7fce9119b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fce8c153950 0x7fce8c153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58874/0 (socket says 172.21.15.31:58874) 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.580+0000 7fce9119b700 1 -- 172.21.15.31:0/825535245 learned_addr learned my addr 172.21.15.31:0/825535245 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.580+0000 7fce9099a700 1 --2- 172.21.15.31:0/825535245 >> v2:172.21.15.62:3300/0 conn(0x7fce8c0cd5d0 0x7fce8c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.580+0000 7fce9119b700 1 -- 172.21.15.31:0/825535245 >> v2:172.21.15.62:3300/0 conn(0x7fce8c0cd5d0 msgr2=0x7fce8c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.580+0000 7fce9119b700 1 --2- 172.21.15.31:0/825535245 >> v2:172.21.15.62:3300/0 conn(0x7fce8c0cd5d0 0x7fce8c15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.580+0000 7fce9119b700 1 -- 172.21.15.31:0/825535245 >> v1:172.21.15.62:6789/0 conn(0x7fce8c0cda20 legacy=0x7fce8c159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.580+0000 7fce9119b700 1 -- 172.21.15.31:0/825535245 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c15b1f0 con 0x7fce8c153950 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.581+0000 7fce9119b700 1 --2- 172.21.15.31:0/825535245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c153950 0x7fce8c153d20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fce8c157800 tx=0x7fce88009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9582f0653cafd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.581+0000 7fce83fff700 1 -- 172.21.15.31:0/825535245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88005c10 con 0x7fce8c153950 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.581+0000 7fce83fff700 1 -- 172.21.15.31:0/825535245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce88005d70 con 0x7fce8c153950 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.581+0000 7fce83fff700 1 -- 172.21.15.31:0/825535245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88006260 con 0x7fce8c153950 2023-10-06T05:18:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.581+0000 7fce933ff700 1 -- 172.21.15.31:0/825535245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c153950 msgr2=0x7fce8c153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.581+0000 7fce933ff700 1 --2- 172.21.15.31:0/825535245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c153950 0x7fce8c153d20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fce8c157800 tx=0x7fce88009220 comp rx=0 tx=0).stop 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.582+0000 7fce933ff700 1 -- 172.21.15.31:0/825535245 shutdown_connections 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.582+0000 7fce933ff700 1 --2- 172.21.15.31:0/825535245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c153950 0x7fce8c153d20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.582+0000 7fce933ff700 1 --2- 172.21.15.31:0/825535245 >> v2:172.21.15.62:3300/0 conn(0x7fce8c0cd5d0 0x7fce8c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.582+0000 7fce933ff700 1 -- 172.21.15.31:0/825535245 >> 172.21.15.31:0/825535245 conn(0x7fce8c0681b0 msgr2=0x7fce8c0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.582+0000 7fce933ff700 1 -- 172.21.15.31:0/825535245 shutdown_connections 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.582+0000 7fce933ff700 1 -- 172.21.15.31:0/825535245 wait complete. 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.583+0000 7fce933ff700 1 Processor -- start 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.583+0000 7fce933ff700 1 -- start start 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.584+0000 7fce933ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8c0cd5d0 0x7fce8c0b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.584+0000 7fce933ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c0cda20 0x7fce8c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.584+0000 7fce933ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8c153950 0x7fce8c1540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.584+0000 7fce933ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c15e860 con 0x7fce8c0cda20 2023-10-06T05:18:19.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.584+0000 7fce933ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c15ecf0 con 0x7fce8c153950 2023-10-06T05:18:19.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.584+0000 7fce933ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c15dc50 con 0x7fce8c0cd5d0 2023-10-06T05:18:19.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.584+0000 7fce9099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c0cda20 0x7fce8c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.584+0000 7fce9099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c0cda20 0x7fce8c15d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58876/0 (socket says 172.21.15.31:58876) 2023-10-06T05:18:19.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.584+0000 7fce9099a700 1 -- 172.21.15.31:0/2106966636 learned_addr learned my addr 172.21.15.31:0/2106966636 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:19.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce9119b700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8c0cd5d0 0x7fce8c0b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce9099a700 1 -- 172.21.15.31:0/2106966636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8c0cd5d0 msgr2=0x7fce8c0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce9099a700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8c0cd5d0 0x7fce8c0b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce9099a700 1 -- 172.21.15.31:0/2106966636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8c153950 msgr2=0x7fce8c1540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce9199c700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8c153950 0x7fce8c1540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce9099a700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8c153950 0x7fce8c1540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce9099a700 1 -- 172.21.15.31:0/2106966636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce88009480 con 0x7fce8c0cda20 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce9119b700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8c0cd5d0 0x7fce8c0b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce9199c700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8c153950 0x7fce8c1540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce9099a700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c0cda20 0x7fce8c15d2e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00e930 tx=0x7fce7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce817fa700 1 -- 172.21.15.31:0/2106966636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c00eed0 con 0x7fce8c0cda20 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce817fa700 1 -- 172.21.15.31:0/2106966636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c00c850 con 0x7fce8c0cda20 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce933ff700 1 -- 172.21.15.31:0/2106966636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c15df30 con 0x7fce8c0cda20 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce817fa700 1 -- 172.21.15.31:0/2106966636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c00cbf0 con 0x7fce8c0cda20 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.585+0000 7fce933ff700 1 -- 172.21.15.31:0/2106966636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c15e4f0 con 0x7fce8c0cda20 2023-10-06T05:18:19.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.586+0000 7fce933ff700 1 -- 172.21.15.31:0/2106966636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c04c380 con 0x7fce8c0cda20 2023-10-06T05:18:19.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.587+0000 7fce817fa700 1 -- 172.21.15.31:0/2106966636 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce7c013470 con 0x7fce8c0cda20 2023-10-06T05:18:19.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.588+0000 7fce817fa700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fce70066f30 0x7fce700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:19.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.588+0000 7fce817fa700 1 -- 172.21.15.31:0/2106966636 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce7c0848f0 con 0x7fce8c0cda20 2023-10-06T05:18:19.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.588+0000 7fce9119b700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fce70066f30 0x7fce700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:19.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.589+0000 7fce9119b700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fce70066f30 0x7fce700693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fce88009260 tx=0x7fce88016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:19.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.592+0000 7fce817fa700 1 -- 172.21.15.31:0/2106966636 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce7c0519e0 con 0x7fce8c0cda20 2023-10-06T05:18:19.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.829+0000 7fce933ff700 1 -- 172.21.15.31:0/2106966636 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000721169", "group_name": "group_0000000000618160", "target": ["mon-mgr", ""]}) v1 -- 0x7fce8c15e9a0 con 0x7fce70066f30 2023-10-06T05:18:19.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.844+0000 7fce817fa700 1 -- 172.21.15.31:0/2106966636 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fce8c15e9a0 con 0x7fce70066f30 2023-10-06T05:18:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 -- 172.21.15.31:0/2106966636 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fce70066f30 msgr2=0x7fce700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fce70066f30 0x7fce700693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fce88009260 tx=0x7fce88016000 comp rx=0 tx=0).stop 2023-10-06T05:18:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 -- 172.21.15.31:0/2106966636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c0cda20 msgr2=0x7fce8c15d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c0cda20 0x7fce8c15d2e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00e930 tx=0x7fce7c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 -- 172.21.15.31:0/2106966636 shutdown_connections 2023-10-06T05:18:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8c153950 0x7fce8c1540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8c0cd5d0 0x7fce8c0b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fce70066f30 0x7fce700693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 --2- 172.21.15.31:0/2106966636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce8c0cda20 0x7fce8c15d2e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 -- 172.21.15.31:0/2106966636 >> 172.21.15.31:0/2106966636 conn(0x7fce8c0681b0 msgr2=0x7fce8c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:19.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 -- 172.21.15.31:0/2106966636 shutdown_connections 2023-10-06T05:18:19.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:19.847+0000 7fce933ff700 1 -- 172.21.15.31:0/2106966636 wait complete. 2023-10-06T05:18:19.851 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:19.849+0000 7efe4fe04700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:19.851 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:19.849+0000 7efe4fe04700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:19.851 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:19.849+0000 7efe4fe04700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:19.851 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:19.849+0000 7efe4fe04700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:19.851 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:19.849+0000 7efe4fe04700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:19.871 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:19.869+0000 7efe4f603700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:19.871 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:19.869+0000 7efe4f603700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:19.871 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:19.869+0000 7efe4f603700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:19.871 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:19.869+0000 7efe4f603700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:19.871 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:19.869+0000 7efe4f603700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:19.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:19.872 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T05:18:19.963 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T05:18:19.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000618160 2023-10-06T05:18:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.163+0000 7f4d22fff700 1 Processor -- start 2023-10-06T05:18:20.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.163+0000 7f4d22fff700 1 -- start start 2023-10-06T05:18:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.164+0000 7f4d22fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4d1c159540 0x7f4d1c068930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.164+0000 7f4d22fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d1c0c8f40 0x7f4d1c0adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.164+0000 7f4d22fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c0adb00 con 0x7f4d1c06a2a0 2023-10-06T05:18:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.164+0000 7f4d22fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c068f10 con 0x7f4d1c159540 2023-10-06T05:18:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.164+0000 7f4d22fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0691f0 con 0x7f4d1c0c8f40 2023-10-06T05:18:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.164+0000 7f4d1bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4d1c06a2a0 0x7f4d1c0c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56254/0 (socket says 172.21.15.31:56254) 2023-10-06T05:18:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.164+0000 7f4d1bfff700 1 -- 172.21.15.31:0/162805282 learned_addr learned my addr 172.21.15.31:0/162805282 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d20d9b700 1 --2- 172.21.15.31:0/162805282 >> v2:172.21.15.62:3300/0 conn(0x7f4d1c159540 0x7f4d1c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d2159c700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d1c0c8f40 0x7f4d1c0adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855535604 0 0) 0x7f4d1c0adb00 con 0x7f4d1c06a2a0 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d0c003570 con 0x7f4d1c06a2a0 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 132146833 0 0) 0x7f4d0c003570 con 0x7f4d1c06a2a0 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d1c0c8f40 msgr2=0x7f4d1c0adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d1c0c8f40 0x7f4d1c0adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 >> v2:172.21.15.62:3300/0 conn(0x7f4d1c159540 msgr2=0x7f4d1c068930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 --2- 172.21.15.31:0/162805282 >> v2:172.21.15.62:3300/0 conn(0x7f4d1c159540 0x7f4d1c068930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0694d0 con 0x7f4d1c06a2a0 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f4d08001470 con 0x7f4d1c06a2a0 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d2159c700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d1c0c8f40 0x7f4d1c0adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 >> v1:172.21.15.31:6789/0 conn(0x7f4d1c06a2a0 legacy=0x7f4d1c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:20.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c003ae0 0x7f4d0c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d1c0c8f40 0x7f4d0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0c004d50 0x7f4d0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c068f10 con 0x7f4d0c004d50 2023-10-06T05:18:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0691f0 con 0x7f4d0c003ae0 2023-10-06T05:18:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c00bab0 con 0x7f4d1c0c8f40 2023-10-06T05:18:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.165+0000 7f4d20d9b700 1 --2- 172.21.15.31:0/162805282 >> v2:172.21.15.62:3300/0 conn(0x7f4d1c159540 0x7f4d1c068930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.166+0000 7f4d2159c700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0c004d50 0x7f4d0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.166+0000 7f4d1bfff700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d1c0c8f40 0x7f4d0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.166+0000 7f4d20d9b700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c003ae0 0x7f4d0c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.166+0000 7f4d2159c700 1 -- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d1c0c8f40 msgr2=0x7f4d0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.166+0000 7f4d2159c700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d1c0c8f40 0x7f4d0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.166+0000 7f4d2159c700 1 -- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c003ae0 msgr2=0x7f4d0c003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.166+0000 7f4d2159c700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d0c003ae0 0x7f4d0c003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.166+0000 7f4d2159c700 1 -- 172.21.15.31:0/162805282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14009420 con 0x7f4d0c004d50 2023-10-06T05:18:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.166+0000 7f4d2159c700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0c004d50 0x7f4d0c0051c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0c9340 tx=0x7f4d140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eea54f0593506fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.166+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14009740 con 0x7f4d0c004d50 2023-10-06T05:18:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.167+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1400e8f0 con 0x7f4d0c004d50 2023-10-06T05:18:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.167+0000 7f4d1b7fe700 1 -- 172.21.15.31:0/162805282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400ec90 con 0x7f4d0c004d50 2023-10-06T05:18:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.167+0000 7f4d22fff700 1 -- 172.21.15.31:0/162805282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0c004d50 msgr2=0x7f4d0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.167+0000 7f4d22fff700 1 --2- 172.21.15.31:0/162805282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d0c004d50 0x7f4d0c0051c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0c9340 tx=0x7f4d140091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.167+0000 7f4d2159c700 1 -- 172.21.15.31:0/162805282 reap_dead start 2023-10-06T05:18:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.167+0000 7f4d22fff700 1 -- 172.21.15.31:0/162805282 shutdown_connections 2023-10-06T05:18:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.167+0000 7f4d22fff700 1 -- 172.21.15.31:0/162805282 >> 172.21.15.31:0/162805282 conn(0x7f4d1c0ce290 msgr2=0x7f4d1c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.167+0000 7f4d22fff700 1 -- 172.21.15.31:0/162805282 shutdown_connections 2023-10-06T05:18:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.167+0000 7f4d22fff700 1 -- 172.21.15.31:0/162805282 wait complete. 2023-10-06T05:18:20.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.169+0000 7f4d22fff700 1 Processor -- start 2023-10-06T05:18:20.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.169+0000 7f4d22fff700 1 -- start start 2023-10-06T05:18:20.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d22fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d0c004d50 0x7f4d1c158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d22fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d1c0c5830 0x7f4d1c0c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d22fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c0cd3e0 0x7f4d1c0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d22fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c4b30 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d22fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c8650 con 0x7f4d1c0c5830 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d22fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0cdd90 con 0x7f4d0c004d50 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d2159c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c0cd3e0 0x7f4d1c0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d2159c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c0cd3e0 0x7f4d1c0cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58920/0 (socket says 172.21.15.31:58920) 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d2159c700 1 -- 172.21.15.31:0/3828183020 learned_addr learned my addr 172.21.15.31:0/3828183020 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d1bfff700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d1c0c5830 0x7f4d1c0c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d20d9b700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d0c004d50 0x7f4d1c158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d2159c700 1 -- 172.21.15.31:0/3828183020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d0c004d50 msgr2=0x7f4d1c158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d2159c700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d0c004d50 0x7f4d1c158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d2159c700 1 -- 172.21.15.31:0/3828183020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d1c0c5830 msgr2=0x7f4d1c0c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d2159c700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d1c0c5830 0x7f4d1c0c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d2159c700 1 -- 172.21.15.31:0/3828183020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c15a8a0 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.170+0000 7f4d20d9b700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d0c004d50 0x7f4d1c158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.171+0000 7f4d2159c700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c0cd3e0 0x7f4d1c0cd850 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4d14013ef0 tx=0x7f4d1401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.171+0000 7f4d18ff9700 1 -- 172.21.15.31:0/3828183020 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14022070 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.171+0000 7f4d18ff9700 1 -- 172.21.15.31:0/3828183020 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1400ec70 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.171+0000 7f4d18ff9700 1 -- 172.21.15.31:0/3828183020 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14005c50 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.171+0000 7f4d22fff700 1 -- 172.21.15.31:0/3828183020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c15a290 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.171+0000 7f4d22fff700 1 -- 172.21.15.31:0/3828183020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c15a530 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.172+0000 7f4d22fff700 1 -- 172.21.15.31:0/3828183020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c15a0f0 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.173+0000 7f4d18ff9700 1 -- 172.21.15.31:0/3828183020 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d14006230 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.174+0000 7f4d18ff9700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d00066f30 0x7f4d000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:20.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.174+0000 7f4d18ff9700 1 -- 172.21.15.31:0/3828183020 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4d14000e40 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.174+0000 7f4d20d9b700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d00066f30 0x7f4d000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:20.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.175+0000 7f4d20d9b700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d00066f30 0x7f4d000693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0cb0f0 tx=0x7f4d1000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:20.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.178+0000 7f4d18ff9700 1 -- 172.21.15.31:0/3828183020 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d1c15a0f0 con 0x7f4d1c0cd3e0 2023-10-06T05:18:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.409+0000 7f4d22fff700 1 -- 172.21.15.31:0/3828183020 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "group_0000000000618160", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d1c068480 con 0x7f4d00066f30 2023-10-06T05:18:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.413+0000 7f4d18ff9700 1 -- 172.21.15.31:0/3828183020 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4d1c068480 con 0x7f4d00066f30 2023-10-06T05:18:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 -- 172.21.15.31:0/3828183020 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d00066f30 msgr2=0x7f4d000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d00066f30 0x7f4d000693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0cb0f0 tx=0x7f4d1000b040 comp rx=0 tx=0).stop 2023-10-06T05:18:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 -- 172.21.15.31:0/3828183020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c0cd3e0 msgr2=0x7f4d1c0cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c0cd3e0 0x7f4d1c0cd850 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4d14013ef0 tx=0x7f4d1401a000 comp rx=0 tx=0).stop 2023-10-06T05:18:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 -- 172.21.15.31:0/3828183020 shutdown_connections 2023-10-06T05:18:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d00066f30 0x7f4d000693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d1c0cd3e0 0x7f4d1c0cd850 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d0c004d50 0x7f4d1c158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 --2- 172.21.15.31:0/3828183020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d1c0c5830 0x7f4d1c0c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 -- 172.21.15.31:0/3828183020 >> 172.21.15.31:0/3828183020 conn(0x7f4d1c0ce290 msgr2=0x7f4d1c070360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:20.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 -- 172.21.15.31:0/3828183020 shutdown_connections 2023-10-06T05:18:20.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:20.415+0000 7f4d22fff700 1 -- 172.21.15.31:0/3828183020 wait complete. 2023-10-06T05:18:20.434 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:18:20.434 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:18:20.534 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:18:20.638 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:18:20.638 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:18:20.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:20.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:20.749 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:18:20.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:20.749 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:18:20.801 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:18:20.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:20.801 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:18:20.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:18:20.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:20.897 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:18:20.944 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:21.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.143+0000 7fc4267ff700 1 Processor -- start 2023-10-06T05:18:21.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.143+0000 7fc4267ff700 1 -- start start 2023-10-06T05:18:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.144+0000 7fc4267ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc4200c7f30 0x7fc420069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.144+0000 7fc4267ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42013f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.144+0000 7fc4267ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4200ca5b0 con 0x7fc4200c7b60 2023-10-06T05:18:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.144+0000 7fc4267ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4200af2f0 con 0x7fc4200c7f30 2023-10-06T05:18:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.144+0000 7fc4267ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42006f740 con 0x7fc4200aee70 2023-10-06T05:18:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc41ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc4200c7b60 0x7fc42006ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56256/0 (socket says 172.21.15.31:56256) 2023-10-06T05:18:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc41ffff700 1 -- 172.21.15.31:0/4027608121 learned_addr learned my addr 172.21.15.31:0/4027608121 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc424d9c700 1 --2- 172.21.15.31:0/4027608121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc41f7fe700 1 --2- 172.21.15.31:0/4027608121 >> v2:172.21.15.62:3300/0 conn(0x7fc4200c7f30 0x7fc420069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc41effd700 1 -- 172.21.15.31:0/4027608121 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714342285 0 0) 0x7fc4200ca5b0 con 0x7fc4200c7b60 2023-10-06T05:18:21.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc41effd700 1 -- 172.21.15.31:0/4027608121 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc408003570 con 0x7fc4200c7b60 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc424d9c700 1 -- 172.21.15.31:0/4027608121 >> v2:172.21.15.62:3300/0 conn(0x7fc4200c7f30 msgr2=0x7fc420069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc424d9c700 1 --2- 172.21.15.31:0/4027608121 >> v2:172.21.15.62:3300/0 conn(0x7fc4200c7f30 0x7fc420069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc424d9c700 1 -- 172.21.15.31:0/4027608121 >> v1:172.21.15.31:6789/0 conn(0x7fc4200c7b60 legacy=0x7fc42006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc424d9c700 1 -- 172.21.15.31:0/4027608121 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4200c9c80 con 0x7fc4200aee70 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.145+0000 7fc41f7fe700 1 --2- 172.21.15.31:0/4027608121 >> v2:172.21.15.62:3300/0 conn(0x7fc4200c7f30 0x7fc420069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.146+0000 7fc424d9c700 1 --2- 172.21.15.31:0/4027608121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42013f690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc418009e60 tx=0x7fc418009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=737c78fc93e5ed31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.146+0000 7fc41effd700 1 -- 172.21.15.31:0/4027608121 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41800e070 con 0x7fc4200aee70 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.146+0000 7fc41effd700 1 -- 172.21.15.31:0/4027608121 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41800bab0 con 0x7fc4200aee70 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.146+0000 7fc41effd700 1 -- 172.21.15.31:0/4027608121 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41800be70 con 0x7fc4200aee70 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.146+0000 7fc4267ff700 1 -- 172.21.15.31:0/4027608121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 msgr2=0x7fc42013f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.146+0000 7fc4267ff700 1 --2- 172.21.15.31:0/4027608121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42013f690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc418009e60 tx=0x7fc418009780 comp rx=0 tx=0).stop 2023-10-06T05:18:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.147+0000 7fc4267ff700 1 -- 172.21.15.31:0/4027608121 shutdown_connections 2023-10-06T05:18:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.147+0000 7fc4267ff700 1 --2- 172.21.15.31:0/4027608121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42013f690 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.147+0000 7fc4267ff700 1 --2- 172.21.15.31:0/4027608121 >> v2:172.21.15.62:3300/0 conn(0x7fc4200c7f30 0x7fc420069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.147+0000 7fc4267ff700 1 -- 172.21.15.31:0/4027608121 >> 172.21.15.31:0/4027608121 conn(0x7fc4200cc0b0 msgr2=0x7fc4200cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.147+0000 7fc4267ff700 1 -- 172.21.15.31:0/4027608121 shutdown_connections 2023-10-06T05:18:21.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.147+0000 7fc4267ff700 1 -- 172.21.15.31:0/4027608121 wait complete. 2023-10-06T05:18:21.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.148+0000 7fc4267ff700 1 Processor -- start 2023-10-06T05:18:21.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.148+0000 7fc4267ff700 1 -- start start 2023-10-06T05:18:21.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc4267ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42016a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc4267ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4200c7b60 0x7fc420133ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc4267ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4200c7f30 0x7fc42016cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc4267ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42016aa70 con 0x7fc4200c7b60 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc4267ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc420134010 con 0x7fc4200c7f30 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc4267ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42016da80 con 0x7fc4200aee70 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc41ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42016a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc41f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4200c7b60 0x7fc420133ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc41ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42016a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:45746/0 (socket says 172.21.15.31:45746) 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc41ffff700 1 -- 172.21.15.31:0/3600731804 learned_addr learned my addr 172.21.15.31:0/3600731804 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc41f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4200c7b60 0x7fc420133ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58930/0 (socket says 172.21.15.31:58930) 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.149+0000 7fc424d9c700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4200c7f30 0x7fc42016cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc41ffff700 1 -- 172.21.15.31:0/3600731804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4200c7f30 msgr2=0x7fc42016cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc41ffff700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4200c7f30 0x7fc42016cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc41ffff700 1 -- 172.21.15.31:0/3600731804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4200c7b60 msgr2=0x7fc420133ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc41ffff700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4200c7b60 0x7fc420133ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc41ffff700 1 -- 172.21.15.31:0/3600731804 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414009420 con 0x7fc4200aee70 2023-10-06T05:18:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc41f7fe700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4200c7b60 0x7fc420133ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc41ffff700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42016a530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc41000c790 tx=0x7fc410007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc407fff700 1 -- 172.21.15.31:0/3600731804 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4100072e0 con 0x7fc4200aee70 2023-10-06T05:18:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc407fff700 1 -- 172.21.15.31:0/3600731804 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc410007440 con 0x7fc4200aee70 2023-10-06T05:18:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc4267ff700 1 -- 172.21.15.31:0/3600731804 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc418009a20 con 0x7fc4200aee70 2023-10-06T05:18:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.150+0000 7fc407fff700 1 -- 172.21.15.31:0/3600731804 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4100057f0 con 0x7fc4200aee70 2023-10-06T05:18:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.151+0000 7fc4267ff700 1 -- 172.21.15.31:0/3600731804 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc420169d30 con 0x7fc4200aee70 2023-10-06T05:18:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.151+0000 7fc4267ff700 1 -- 172.21.15.31:0/3600731804 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42004c380 con 0x7fc4200aee70 2023-10-06T05:18:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.152+0000 7fc407fff700 1 -- 172.21.15.31:0/3600731804 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fc410005be0 con 0x7fc4200aee70 2023-10-06T05:18:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.153+0000 7fc407fff700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc40c0670e0 0x7fc40c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.153+0000 7fc407fff700 1 -- 172.21.15.31:0/3600731804 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc410083db0 con 0x7fc4200aee70 2023-10-06T05:18:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.154+0000 7fc41f7fe700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc40c0670e0 0x7fc40c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.154+0000 7fc41f7fe700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc40c0670e0 0x7fc40c069590 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc4200ccfe0 tx=0x7fc4140091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:21.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.158+0000 7fc407fff700 1 -- 172.21.15.31:0/3600731804 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc41004b5a0 con 0x7fc4200aee70 2023-10-06T05:18:21.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.406+0000 7fc4267ff700 1 -- 172.21.15.31:0/3600731804 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4200c9f60 con 0x7fc4200aee70 2023-10-06T05:18:21.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.407+0000 7fc407fff700 1 -- 172.21.15.31:0/3600731804 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fc410017070 con 0x7fc4200aee70 2023-10-06T05:18:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:21.409 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/746584225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":746584225},{"type":"v1","addr":"172.21.15.62:6834","nonce":746584225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/584374139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":584374139},{"type":"v1","addr":"172.21.15.62:6835","nonce":584374139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4926,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3794168057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3794168057},{"type":"v1","addr":"172.21.15.31:6836","nonce":3794168057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:00.090015+0000","modified":"2023-10-06T05:18:03.257587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/3886723511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3886723511},{"type":"v1","addr":"172.21.15.31:6837","nonce":3886723511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:18:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 -- 172.21.15.31:0/3600731804 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc40c0670e0 msgr2=0x7fc40c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc40c0670e0 0x7fc40c069590 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc4200ccfe0 tx=0x7fc4140091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 -- 172.21.15.31:0/3600731804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 msgr2=0x7fc42016a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42016a530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc41000c790 tx=0x7fc410007c80 comp rx=0 tx=0).stop 2023-10-06T05:18:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 -- 172.21.15.31:0/3600731804 shutdown_connections 2023-10-06T05:18:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4200c7f30 0x7fc42016cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4200aee70 0x7fc42016a530 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc40c0670e0 0x7fc40c069590 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 --2- 172.21.15.31:0/3600731804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4200c7b60 0x7fc420133ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 -- 172.21.15.31:0/3600731804 >> 172.21.15.31:0/3600731804 conn(0x7fc4200cc0b0 msgr2=0x7fc4200c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:21.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 -- 172.21.15.31:0/3600731804 shutdown_connections 2023-10-06T05:18:21.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.409+0000 7fc4267ff700 1 -- 172.21.15.31:0/3600731804 wait complete. 2023-10-06T05:18:21.412 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2023-10-06T05:18:21.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.622+0000 7f85227ff700 1 Processor -- start 2023-10-06T05:18:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.622+0000 7f85227ff700 1 -- start start 2023-10-06T05:18:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.623+0000 7f85227ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f851c13a1e0 0x7f851c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.623+0000 7f85227ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f851c06db50 0x7f851c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.623+0000 7f85227ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851c0c71b0 con 0x7f851c13a5b0 2023-10-06T05:18:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.623+0000 7f85227ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c0c7490 con 0x7f851c13a1e0 2023-10-06T05:18:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.623+0000 7f85227ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c0c7770 con 0x7f851c06db50 2023-10-06T05:18:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.624+0000 7f851bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f851c13a1e0 0x7f851c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.624+0000 7f851bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f851c13a1e0 0x7f851c0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58944/0 (socket says 172.21.15.31:58944) 2023-10-06T05:18:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.624+0000 7f851bfff700 1 -- 172.21.15.31:0/2724825127 learned_addr learned my addr 172.21.15.31:0/2724825127 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.624+0000 7f851b7fe700 1 --1- 172.21.15.31:0/2724825127 >> v1:172.21.15.31:6789/0 conn(0x7f851c13a5b0 0x7f851c06d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56260/0 (socket says 172.21.15.31:56260) 2023-10-06T05:18:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.624+0000 7f8520d9c700 1 --2- 172.21.15.31:0/2724825127 >> v2:172.21.15.62:3300/0 conn(0x7f851c06db50 0x7f851c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.624+0000 7f851bfff700 1 -- 172.21.15.31:0/2724825127 >> v2:172.21.15.62:3300/0 conn(0x7f851c06db50 msgr2=0x7f851c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.624+0000 7f851bfff700 1 --2- 172.21.15.31:0/2724825127 >> v2:172.21.15.62:3300/0 conn(0x7f851c06db50 0x7f851c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.624+0000 7f851bfff700 1 -- 172.21.15.31:0/2724825127 >> v1:172.21.15.31:6789/0 conn(0x7f851c13a5b0 legacy=0x7f851c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.624+0000 7f851bfff700 1 -- 172.21.15.31:0/2724825127 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0c7a50 con 0x7f851c13a1e0 2023-10-06T05:18:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.624+0000 7f851affd700 1 -- 172.21.15.31:0/2724825127 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045871408 0 0) 0x7f851c0c71b0 con 0x7f851c13a5b0 2023-10-06T05:18:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.625+0000 7f851bfff700 1 --2- 172.21.15.31:0/2724825127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 0x7f851c0c66f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f851c06b920 tx=0x7f850c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875d23c2ca3b341d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.625+0000 7f851affd700 1 -- 172.21.15.31:0/2724825127 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c005c10 con 0x7f851c13a1e0 2023-10-06T05:18:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.625+0000 7f851affd700 1 -- 172.21.15.31:0/2724825127 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c005d70 con 0x7f851c13a1e0 2023-10-06T05:18:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.625+0000 7f851affd700 1 -- 172.21.15.31:0/2724825127 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c006260 con 0x7f851c13a1e0 2023-10-06T05:18:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.625+0000 7f85227ff700 1 -- 172.21.15.31:0/2724825127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 msgr2=0x7f851c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.625+0000 7f85227ff700 1 --2- 172.21.15.31:0/2724825127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 0x7f851c0c66f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f851c06b920 tx=0x7f850c009220 comp rx=0 tx=0).stop 2023-10-06T05:18:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.625+0000 7f85227ff700 1 -- 172.21.15.31:0/2724825127 shutdown_connections 2023-10-06T05:18:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.625+0000 7f85227ff700 1 --2- 172.21.15.31:0/2724825127 >> v2:172.21.15.62:3300/0 conn(0x7f851c06db50 0x7f851c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.625+0000 7f85227ff700 1 --2- 172.21.15.31:0/2724825127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 0x7f851c0c66f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.625+0000 7f85227ff700 1 -- 172.21.15.31:0/2724825127 >> 172.21.15.31:0/2724825127 conn(0x7f851c0cc0b0 msgr2=0x7f851c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.626+0000 7f85227ff700 1 -- 172.21.15.31:0/2724825127 shutdown_connections 2023-10-06T05:18:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.626+0000 7f85227ff700 1 -- 172.21.15.31:0/2724825127 wait complete. 2023-10-06T05:18:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.627+0000 7f85227ff700 1 Processor -- start 2023-10-06T05:18:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.627+0000 7f85227ff700 1 -- start start 2023-10-06T05:18:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.628+0000 7f85227ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c06db50 0x7f851c0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.628+0000 7f85227ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 0x7f851c151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.628+0000 7f85227ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f851c13a5b0 0x7f851c0c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.628+0000 7f85227ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b13e0 con 0x7f851c13a1e0 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.628+0000 7f85227ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b0ab0 con 0x7f851c06db50 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.628+0000 7f85227ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b0d90 con 0x7f851c13a5b0 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.628+0000 7f851b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 0x7f851c151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.628+0000 7f851b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 0x7f851c151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58960/0 (socket says 172.21.15.31:58960) 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.628+0000 7f851b7fe700 1 -- 172.21.15.31:0/1552029556 learned_addr learned my addr 172.21.15.31:0/1552029556 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.628+0000 7f851bfff700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c06db50 0x7f851c0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f8520d9c700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f851c13a5b0 0x7f851c0c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f851b7fe700 1 -- 172.21.15.31:0/1552029556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f851c13a5b0 msgr2=0x7f851c0c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f851b7fe700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f851c13a5b0 0x7f851c0c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f851b7fe700 1 -- 172.21.15.31:0/1552029556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c06db50 msgr2=0x7f851c0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f851b7fe700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c06db50 0x7f851c0b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f851b7fe700 1 -- 172.21.15.31:0/1552029556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c009480 con 0x7f851c13a1e0 2023-10-06T05:18:21.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f851b7fe700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 0x7f851c151000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f851000c510 tx=0x7f851000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:21.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f8503fff700 1 -- 172.21.15.31:0/1552029556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510005a70 con 0x7f851c13a1e0 2023-10-06T05:18:21.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f8503fff700 1 -- 172.21.15.31:0/1552029556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8510005bf0 con 0x7f851c13a1e0 2023-10-06T05:18:21.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f85227ff700 1 -- 172.21.15.31:0/1552029556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c0b1070 con 0x7f851c13a1e0 2023-10-06T05:18:21.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f8503fff700 1 -- 172.21.15.31:0/1552029556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100070d0 con 0x7f851c13a1e0 2023-10-06T05:18:21.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.629+0000 7f85227ff700 1 -- 172.21.15.31:0/1552029556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c0aef70 con 0x7f851c13a1e0 2023-10-06T05:18:21.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.630+0000 7f85227ff700 1 -- 172.21.15.31:0/1552029556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c0af190 con 0x7f851c13a1e0 2023-10-06T05:18:21.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.631+0000 7f8503fff700 1 -- 172.21.15.31:0/1552029556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8510007440 con 0x7f851c13a1e0 2023-10-06T05:18:21.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.632+0000 7f8503fff700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f85080670e0 0x7f8508069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:21.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.632+0000 7f8503fff700 1 -- 172.21.15.31:0/1552029556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8510083f70 con 0x7f851c13a1e0 2023-10-06T05:18:21.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.632+0000 7f851bfff700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f85080670e0 0x7f8508069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:21.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.633+0000 7f851bfff700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f85080670e0 0x7f8508069590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f850c009260 tx=0x7f850c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:21.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.637+0000 7f8503fff700 1 -- 172.21.15.31:0/1552029556 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f851c0af190 con 0x7f851c13a1e0 2023-10-06T05:18:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.841+0000 7f85227ff700 1 -- 172.21.15.31:0/1552029556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f851c0af190 con 0x7f851c13a1e0 2023-10-06T05:18:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.842+0000 7f8503fff700 1 -- 172.21.15.31:0/1552029556 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f851c0af190 con 0x7f851c13a1e0 2023-10-06T05:18:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:21.845 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:01.245088+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:58.074919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:58.854233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 -- 172.21.15.31:0/1552029556 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f85080670e0 msgr2=0x7f8508069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f85080670e0 0x7f8508069590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f850c009260 tx=0x7f850c016000 comp rx=0 tx=0).stop 2023-10-06T05:18:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 -- 172.21.15.31:0/1552029556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 msgr2=0x7f851c151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 0x7f851c151000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f851000c510 tx=0x7f851000c8d0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 -- 172.21.15.31:0/1552029556 shutdown_connections 2023-10-06T05:18:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f851c13a5b0 0x7f851c0c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f851c06db50 0x7f851c0b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f85080670e0 0x7f8508069590 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 --2- 172.21.15.31:0/1552029556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f851c13a1e0 0x7f851c151000 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 -- 172.21.15.31:0/1552029556 >> 172.21.15.31:0/1552029556 conn(0x7f851c0cc0b0 msgr2=0x7f851c0c12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 -- 172.21.15.31:0/1552029556 shutdown_connections 2023-10-06T05:18:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:21.845+0000 7f85227ff700 1 -- 172.21.15.31:0/1552029556 wait complete. 2023-10-06T05:18:21.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:17:58.074919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:17:58.854233+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:18:21.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:18:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.063+0000 7f8920fff700 1 Processor -- start 2023-10-06T05:18:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.064+0000 7f8920fff700 1 -- start start 2023-10-06T05:18:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.065+0000 7f8920fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159560 0x7f891c0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.065+0000 7f8920fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f891c0b2b80 0x7f891c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.065+0000 7f8920fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891c15d2d0 con 0x7f891c159930 2023-10-06T05:18:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.065+0000 7f8920fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c15d5b0 con 0x7f891c0b2b80 2023-10-06T05:18:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.065+0000 7f8920fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c15d890 con 0x7f891c159560 2023-10-06T05:18:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.065+0000 7f891ad9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f891c0b2b80 0x7f891c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.065+0000 7f891a59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159560 0x7f891c0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.065+0000 7f891a59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159560 0x7f891c0cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:45762/0 (socket says 172.21.15.31:45762) 2023-10-06T05:18:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.065+0000 7f891a59c700 1 -- 172.21.15.31:0/1733370544 learned_addr learned my addr 172.21.15.31:0/1733370544 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.065+0000 7f8919d9b700 1 --1- 172.21.15.31:0/1733370544 >> v1:172.21.15.62:6789/0 conn(0x7f891c159930 0x7f891c0b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:53256/0 (socket says 172.21.15.31:53256) 2023-10-06T05:18:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.066+0000 7f891959a700 1 -- 172.21.15.31:0/1733370544 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455938722 0 0) 0x7f891c15d2d0 con 0x7f891c159930 2023-10-06T05:18:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.066+0000 7f891959a700 1 -- 172.21.15.31:0/1733370544 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8904003570 con 0x7f891c159930 2023-10-06T05:18:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.066+0000 7f891a59c700 1 -- 172.21.15.31:0/1733370544 >> v2:172.21.15.62:3300/0 conn(0x7f891c0b2b80 msgr2=0x7f891c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.066+0000 7f891a59c700 1 --2- 172.21.15.31:0/1733370544 >> v2:172.21.15.62:3300/0 conn(0x7f891c0b2b80 0x7f891c0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.066+0000 7f891a59c700 1 -- 172.21.15.31:0/1733370544 >> v1:172.21.15.62:6789/0 conn(0x7f891c159930 legacy=0x7f891c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.066+0000 7f891a59c700 1 -- 172.21.15.31:0/1733370544 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c15db70 con 0x7f891c159560 2023-10-06T05:18:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.066+0000 7f891a59c700 1 --2- 172.21.15.31:0/1733370544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159560 0x7f891c0cce70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f890c0099b0 tx=0x7f890c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91497d331a51aa50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.066+0000 7f891959a700 1 -- 172.21.15.31:0/1733370544 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c00b680 con 0x7f891c159560 2023-10-06T05:18:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.067+0000 7f891959a700 1 -- 172.21.15.31:0/1733370544 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890c00b7e0 con 0x7f891c159560 2023-10-06T05:18:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.067+0000 7f891959a700 1 -- 172.21.15.31:0/1733370544 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c00bb80 con 0x7f891c159560 2023-10-06T05:18:22.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.067+0000 7f8920fff700 1 -- 172.21.15.31:0/1733370544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159560 msgr2=0x7f891c0cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.067+0000 7f8920fff700 1 --2- 172.21.15.31:0/1733370544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159560 0x7f891c0cce70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f890c0099b0 tx=0x7f890c009250 comp rx=0 tx=0).stop 2023-10-06T05:18:22.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.067+0000 7f8920fff700 1 -- 172.21.15.31:0/1733370544 shutdown_connections 2023-10-06T05:18:22.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.067+0000 7f8920fff700 1 --2- 172.21.15.31:0/1733370544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159560 0x7f891c0cce70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.067+0000 7f8920fff700 1 --2- 172.21.15.31:0/1733370544 >> v2:172.21.15.62:3300/0 conn(0x7f891c0b2b80 0x7f891c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.067+0000 7f8920fff700 1 -- 172.21.15.31:0/1733370544 >> 172.21.15.31:0/1733370544 conn(0x7f891c06ec90 msgr2=0x7f891c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.067+0000 7f8920fff700 1 -- 172.21.15.31:0/1733370544 shutdown_connections 2023-10-06T05:18:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.067+0000 7f8920fff700 1 -- 172.21.15.31:0/1733370544 wait complete. 2023-10-06T05:18:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.069+0000 7f8920fff700 1 Processor -- start 2023-10-06T05:18:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.069+0000 7f8920fff700 1 -- start start 2023-10-06T05:18:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f8920fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f891c0b2b80 0x7f891c0c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f8920fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f891c159560 0x7f891c15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f8920fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159930 0x7f891c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f8920fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0c4bc0 con 0x7f891c0b2b80 2023-10-06T05:18:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f8920fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0ca2d0 con 0x7f891c159560 2023-10-06T05:18:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f8920fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0c3fb0 con 0x7f891c159930 2023-10-06T05:18:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f891a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f891c0b2b80 0x7f891c0c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f891a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f891c0b2b80 0x7f891c0c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58972/0 (socket says 172.21.15.31:58972) 2023-10-06T05:18:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f891a59c700 1 -- 172.21.15.31:0/3922264637 learned_addr learned my addr 172.21.15.31:0/3922264637 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f8919d9b700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f891c159560 0x7f891c15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f891ad9d700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159930 0x7f891c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f891a59c700 1 -- 172.21.15.31:0/3922264637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159930 msgr2=0x7f891c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f891a59c700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159930 0x7f891c0cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f891a59c700 1 -- 172.21.15.31:0/3922264637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f891c159560 msgr2=0x7f891c15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f891a59c700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f891c159560 0x7f891c15c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.070+0000 7f891a59c700 1 -- 172.21.15.31:0/3922264637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c0094f0 con 0x7f891c0b2b80 2023-10-06T05:18:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.071+0000 7f891a59c700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f891c0b2b80 0x7f891c0c9d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f890c009290 tx=0x7f890c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.071+0000 7f890a7fc700 1 -- 172.21.15.31:0/3922264637 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c002f90 con 0x7f891c0b2b80 2023-10-06T05:18:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.071+0000 7f890a7fc700 1 -- 172.21.15.31:0/3922264637 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890c00b950 con 0x7f891c0b2b80 2023-10-06T05:18:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.071+0000 7f890a7fc700 1 -- 172.21.15.31:0/3922264637 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c01a430 con 0x7f891c0b2b80 2023-10-06T05:18:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.071+0000 7f8920fff700 1 -- 172.21.15.31:0/3922264637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c0c4290 con 0x7f891c0b2b80 2023-10-06T05:18:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.071+0000 7f8920fff700 1 -- 172.21.15.31:0/3922264637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c0c4850 con 0x7f891c0b2b80 2023-10-06T05:18:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.072+0000 7f8920fff700 1 -- 172.21.15.31:0/3922264637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c04c380 con 0x7f891c0b2b80 2023-10-06T05:18:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.073+0000 7f890a7fc700 1 -- 172.21.15.31:0/3922264637 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f890c01a7e0 con 0x7f891c0b2b80 2023-10-06T05:18:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.074+0000 7f890a7fc700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f88fc0670e0 0x7f88fc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.074+0000 7f890a7fc700 1 -- 172.21.15.31:0/3922264637 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f890c0865c0 con 0x7f891c0b2b80 2023-10-06T05:18:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.074+0000 7f8919d9b700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f88fc0670e0 0x7f88fc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.075+0000 7f8919d9b700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f88fc0670e0 0x7f88fc069590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f891c16cf60 tx=0x7f8910007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.079+0000 7f890a7fc700 1 -- 172.21.15.31:0/3922264637 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f890c0533b0 con 0x7f891c0b2b80 2023-10-06T05:18:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.285+0000 7f8920fff700 1 -- 172.21.15.31:0/3922264637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f891c15de50 con 0x7f891c0b2b80 2023-10-06T05:18:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.286+0000 7f890a7fc700 1 -- 172.21.15.31:0/3922264637 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f890c0569d0 con 0x7f891c0b2b80 2023-10-06T05:18:22.287 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:18:22.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 -- 172.21.15.31:0/3922264637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f88fc0670e0 msgr2=0x7f88fc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f88fc0670e0 0x7f88fc069590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f891c16cf60 tx=0x7f8910007d40 comp rx=0 tx=0).stop 2023-10-06T05:18:22.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 -- 172.21.15.31:0/3922264637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f891c0b2b80 msgr2=0x7f891c0c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f891c0b2b80 0x7f891c0c9d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f890c009290 tx=0x7f890c017000 comp rx=0 tx=0).stop 2023-10-06T05:18:22.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 -- 172.21.15.31:0/3922264637 shutdown_connections 2023-10-06T05:18:22.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f891c159930 0x7f891c0cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f88fc0670e0 0x7f88fc069590 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f891c0b2b80 0x7f891c0c9d90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 --2- 172.21.15.31:0/3922264637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f891c159560 0x7f891c15c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 -- 172.21.15.31:0/3922264637 >> 172.21.15.31:0/3922264637 conn(0x7f891c06ec90 msgr2=0x7f891c156560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:22.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 -- 172.21.15.31:0/3922264637 shutdown_connections 2023-10-06T05:18:22.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.288+0000 7f8920fff700 1 -- 172.21.15.31:0/3922264637 wait complete. 2023-10-06T05:18:22.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:18:22.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.504+0000 7fd4203ff700 1 Processor -- start 2023-10-06T05:18:22.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.504+0000 7fd4203ff700 1 -- start start 2023-10-06T05:18:22.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.505+0000 7fd4203ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd41815ca40 0x7fd418068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.505+0000 7fd4203ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd418068da0 con 0x7fd4180c8f40 2023-10-06T05:18:22.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.505+0000 7fd4203ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd418069080 con 0x7fd41806a300 2023-10-06T05:18:22.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.505+0000 7fd4203ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4180cabc0 con 0x7fd41815ca40 2023-10-06T05:18:22.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.506+0000 7fd41e99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd4180c8f40 0x7fd4180adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56266/0 (socket says 172.21.15.31:56266) 2023-10-06T05:18:22.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.506+0000 7fd41e99c700 1 -- 172.21.15.31:0/1486350186 learned_addr learned my addr 172.21.15.31:0/1486350186 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:22.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.506+0000 7fd41e19b700 1 --2- 172.21.15.31:0/1486350186 >> v2:172.21.15.62:3300/0 conn(0x7fd41815ca40 0x7fd418068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.506+0000 7fd41d199700 1 -- 172.21.15.31:0/1486350186 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848116332 0 0) 0x7fd418068da0 con 0x7fd4180c8f40 2023-10-06T05:18:22.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.506+0000 7fd41d199700 1 -- 172.21.15.31:0/1486350186 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd404003570 con 0x7fd4180c8f40 2023-10-06T05:18:22.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.506+0000 7fd41d199700 1 -- 172.21.15.31:0/1486350186 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659550827 0 0) 0x7fd418069080 con 0x7fd41806a300 2023-10-06T05:18:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.506+0000 7fd41d199700 1 -- 172.21.15.31:0/1486350186 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd418068da0 con 0x7fd41806a300 2023-10-06T05:18:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.506+0000 7fd41e19b700 1 -- 172.21.15.31:0/1486350186 >> v1:172.21.15.62:6789/0 conn(0x7fd41806a300 legacy=0x7fd4180c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.506+0000 7fd41e19b700 1 -- 172.21.15.31:0/1486350186 >> v1:172.21.15.31:6789/0 conn(0x7fd4180c8f40 legacy=0x7fd4180adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.507+0000 7fd41e19b700 1 -- 172.21.15.31:0/1486350186 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd418069360 con 0x7fd41815ca40 2023-10-06T05:18:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.507+0000 7fd41e19b700 1 --2- 172.21.15.31:0/1486350186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd41815ca40 0x7fd418068670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd4140098e0 tx=0x7fd4140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73c4d5e9b05e10fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.507+0000 7fd41d199700 1 -- 172.21.15.31:0/1486350186 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414011630 con 0x7fd41815ca40 2023-10-06T05:18:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.507+0000 7fd41d199700 1 -- 172.21.15.31:0/1486350186 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd414011790 con 0x7fd41815ca40 2023-10-06T05:18:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.507+0000 7fd41d199700 1 -- 172.21.15.31:0/1486350186 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414011b30 con 0x7fd41815ca40 2023-10-06T05:18:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.508+0000 7fd4203ff700 1 -- 172.21.15.31:0/1486350186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd41815ca40 msgr2=0x7fd418068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.508+0000 7fd4203ff700 1 --2- 172.21.15.31:0/1486350186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd41815ca40 0x7fd418068670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd4140098e0 tx=0x7fd4140091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.508+0000 7fd4203ff700 1 -- 172.21.15.31:0/1486350186 shutdown_connections 2023-10-06T05:18:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.508+0000 7fd4203ff700 1 --2- 172.21.15.31:0/1486350186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd41815ca40 0x7fd418068670 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.508+0000 7fd4203ff700 1 -- 172.21.15.31:0/1486350186 >> 172.21.15.31:0/1486350186 conn(0x7fd4180ce290 msgr2=0x7fd4180ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.508+0000 7fd4203ff700 1 -- 172.21.15.31:0/1486350186 shutdown_connections 2023-10-06T05:18:22.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.508+0000 7fd4203ff700 1 -- 172.21.15.31:0/1486350186 wait complete. 2023-10-06T05:18:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.509+0000 7fd4203ff700 1 Processor -- start 2023-10-06T05:18:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.510+0000 7fd4203ff700 1 -- start start 2023-10-06T05:18:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.510+0000 7fd4203ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41806a300 0x7fd4180aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd4203ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4180c8f40 0x7fd4180c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd4203ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd41815ca40 0x7fd4180c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd4203ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180c4d00 con 0x7fd41806a300 2023-10-06T05:18:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd4203ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180c8c00 con 0x7fd4180c8f40 2023-10-06T05:18:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd4203ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180af120 con 0x7fd41815ca40 2023-10-06T05:18:22.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41806a300 0x7fd4180aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41806a300 0x7fd4180aeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58976/0 (socket says 172.21.15.31:58976) 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41e19b700 1 -- 172.21.15.31:0/4027646434 learned_addr learned my addr 172.21.15.31:0/4027646434 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41d99a700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4180c8f40 0x7fd4180c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41e99c700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd41815ca40 0x7fd4180c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41e19b700 1 -- 172.21.15.31:0/4027646434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd41815ca40 msgr2=0x7fd4180c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41e19b700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd41815ca40 0x7fd4180c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41e19b700 1 -- 172.21.15.31:0/4027646434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4180c8f40 msgr2=0x7fd4180c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41e19b700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4180c8f40 0x7fd4180c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41e19b700 1 -- 172.21.15.31:0/4027646434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd414009420 con 0x7fd41806a300 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.511+0000 7fd41e19b700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41806a300 0x7fd4180aeab0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd41806e040 tx=0x7fd414017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:22.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.512+0000 7fd40e7fc700 1 -- 172.21.15.31:0/4027646434 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41401f070 con 0x7fd41806a300 2023-10-06T05:18:22.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.512+0000 7fd40e7fc700 1 -- 172.21.15.31:0/4027646434 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd414000c90 con 0x7fd41806a300 2023-10-06T05:18:22.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.512+0000 7fd4203ff700 1 -- 172.21.15.31:0/4027646434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4180af400 con 0x7fd41806a300 2023-10-06T05:18:22.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.512+0000 7fd40e7fc700 1 -- 172.21.15.31:0/4027646434 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414025420 con 0x7fd41806a300 2023-10-06T05:18:22.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.512+0000 7fd4203ff700 1 -- 172.21.15.31:0/4027646434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4180af9c0 con 0x7fd41806a300 2023-10-06T05:18:22.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.513+0000 7fd4203ff700 1 -- 172.21.15.31:0/4027646434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41804c380 con 0x7fd41806a300 2023-10-06T05:18:22.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.514+0000 7fd40e7fc700 1 -- 172.21.15.31:0/4027646434 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd414023070 con 0x7fd41806a300 2023-10-06T05:18:22.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.515+0000 7fd40e7fc700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd3fc0670e0 0x7fd3fc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.515+0000 7fd40e7fc700 1 -- 172.21.15.31:0/4027646434 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd414085cf0 con 0x7fd41806a300 2023-10-06T05:18:22.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.515+0000 7fd41d99a700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd3fc0670e0 0x7fd3fc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.516+0000 7fd41d99a700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd3fc0670e0 0x7fd3fc069590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd41814c090 tx=0x7fd408009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:22.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.519+0000 7fd40e7fc700 1 -- 172.21.15.31:0/4027646434 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd414052ae0 con 0x7fd41806a300 2023-10-06T05:18:22.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.722+0000 7fd4203ff700 1 -- 172.21.15.31:0/4027646434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4180ca1a0 con 0x7fd41806a300 2023-10-06T05:18:22.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.723+0000 7fd40e7fc700 1 -- 172.21.15.31:0/4027646434 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd41402a0a0 con 0x7fd41806a300 2023-10-06T05:18:22.725 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:18:22.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.725+0000 7fd4203ff700 1 -- 172.21.15.31:0/4027646434 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd3fc0670e0 msgr2=0x7fd3fc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.725+0000 7fd4203ff700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd3fc0670e0 0x7fd3fc069590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd41814c090 tx=0x7fd408009040 comp rx=0 tx=0).stop 2023-10-06T05:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.725+0000 7fd4203ff700 1 -- 172.21.15.31:0/4027646434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41806a300 msgr2=0x7fd4180aeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.725+0000 7fd4203ff700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41806a300 0x7fd4180aeab0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd41806e040 tx=0x7fd414017000 comp rx=0 tx=0).stop 2023-10-06T05:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.725+0000 7fd4203ff700 1 -- 172.21.15.31:0/4027646434 shutdown_connections 2023-10-06T05:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.725+0000 7fd4203ff700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd41815ca40 0x7fd4180c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.725+0000 7fd4203ff700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd3fc0670e0 0x7fd3fc069590 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.725+0000 7fd4203ff700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd41806a300 0x7fd4180aeab0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.725+0000 7fd4203ff700 1 --2- 172.21.15.31:0/4027646434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4180c8f40 0x7fd4180c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.725+0000 7fd4203ff700 1 -- 172.21.15.31:0/4027646434 >> 172.21.15.31:0/4027646434 conn(0x7fd4180ce290 msgr2=0x7fd418159790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.726+0000 7fd4203ff700 1 -- 172.21.15.31:0/4027646434 shutdown_connections 2023-10-06T05:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.726+0000 7fd4203ff700 1 -- 172.21.15.31:0/4027646434 wait complete. 2023-10-06T05:18:22.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:18:22.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.941+0000 7f9d683ff700 1 Processor -- start 2023-10-06T05:18:22.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.941+0000 7f9d683ff700 1 -- start start 2023-10-06T05:18:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d683ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d600cc340 0x7f9d600b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d683ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 0x7f9d600aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d683ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6015a020 con 0x7f9d600cc710 2023-10-06T05:18:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d683ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d600b07e0 con 0x7f9d600cc340 2023-10-06T05:18:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d683ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d60159cf0 con 0x7f9d6015a4b0 2023-10-06T05:18:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d6619b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d600cc340 0x7f9d600b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d6619b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d600cc340 0x7f9d600b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58988/0 (socket says 172.21.15.31:58988) 2023-10-06T05:18:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d6619b700 1 -- 172.21.15.31:0/2640206931 learned_addr learned my addr 172.21.15.31:0/2640206931 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d6699c700 1 --2- 172.21.15.31:0/2640206931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 0x7f9d600aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d6619b700 1 -- 172.21.15.31:0/2640206931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 msgr2=0x7f9d600aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:22.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d6619b700 1 --2- 172.21.15.31:0/2640206931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 0x7f9d600aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d6619b700 1 -- 172.21.15.31:0/2640206931 >> v1:172.21.15.62:6789/0 conn(0x7f9d600cc710 legacy=0x7f9d600b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.943+0000 7f9d6619b700 1 -- 172.21.15.31:0/2640206931 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d600ccb90 con 0x7f9d600cc340 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.944+0000 7f9d6619b700 1 --2- 172.21.15.31:0/2640206931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 0x7f9d600b2540 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009990 tx=0x7f9d5c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dadac0a733f87d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.944+0000 7f9d65199700 1 -- 172.21.15.31:0/2640206931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c009e80 con 0x7f9d600cc340 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.944+0000 7f9d65199700 1 -- 172.21.15.31:0/2640206931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c005c10 con 0x7f9d600cc340 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.944+0000 7f9d65199700 1 -- 172.21.15.31:0/2640206931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c00b650 con 0x7f9d600cc340 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.945+0000 7f9d683ff700 1 -- 172.21.15.31:0/2640206931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 msgr2=0x7f9d600b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.945+0000 7f9d683ff700 1 --2- 172.21.15.31:0/2640206931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 0x7f9d600b2540 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009990 tx=0x7f9d5c009220 comp rx=0 tx=0).stop 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.945+0000 7f9d683ff700 1 -- 172.21.15.31:0/2640206931 shutdown_connections 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.945+0000 7f9d683ff700 1 --2- 172.21.15.31:0/2640206931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 0x7f9d600aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.945+0000 7f9d683ff700 1 --2- 172.21.15.31:0/2640206931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 0x7f9d600b2540 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.945+0000 7f9d683ff700 1 -- 172.21.15.31:0/2640206931 >> 172.21.15.31:0/2640206931 conn(0x7f9d60068670 msgr2=0x7f9d60068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.945+0000 7f9d683ff700 1 -- 172.21.15.31:0/2640206931 shutdown_connections 2023-10-06T05:18:22.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.945+0000 7f9d683ff700 1 -- 172.21.15.31:0/2640206931 wait complete. 2023-10-06T05:18:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.946+0000 7f9d683ff700 1 Processor -- start 2023-10-06T05:18:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.946+0000 7f9d683ff700 1 -- start start 2023-10-06T05:18:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.947+0000 7f9d683ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 0x7f9d6016a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.947+0000 7f9d683ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d600cc710 0x7f9d600ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.947+0000 7f9d683ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 0x7f9d6016b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.947+0000 7f9d683ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6016b000 con 0x7f9d600cc340 2023-10-06T05:18:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.947+0000 7f9d683ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600af270 con 0x7f9d600cc710 2023-10-06T05:18:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.947+0000 7f9d683ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6016d890 con 0x7f9d6015a4b0 2023-10-06T05:18:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6619b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 0x7f9d6016a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6619b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 0x7f9d6016a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59004/0 (socket says 172.21.15.31:59004) 2023-10-06T05:18:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6619b700 1 -- 172.21.15.31:0/2633207786 learned_addr learned my addr 172.21.15.31:0/2633207786 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6599a700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d600cc710 0x7f9d600ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6699c700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 0x7f9d6016b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6619b700 1 -- 172.21.15.31:0/2633207786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 msgr2=0x7f9d6016b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6619b700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 0x7f9d6016b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6619b700 1 -- 172.21.15.31:0/2633207786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d600cc710 msgr2=0x7f9d600ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6619b700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d600cc710 0x7f9d600ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6619b700 1 -- 172.21.15.31:0/2633207786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c009480 con 0x7f9d600cc340 2023-10-06T05:18:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6599a700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d600cc710 0x7f9d600ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6699c700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 0x7f9d6016b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d6619b700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 0x7f9d6016a060 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b3c0 tx=0x7f9d5c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.948+0000 7f9d567fc700 1 -- 172.21.15.31:0/2633207786 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c011d80 con 0x7f9d600cc340 2023-10-06T05:18:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.949+0000 7f9d567fc700 1 -- 172.21.15.31:0/2633207786 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c01b430 con 0x7f9d600cc340 2023-10-06T05:18:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.949+0000 7f9d567fc700 1 -- 172.21.15.31:0/2633207786 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c01b7d0 con 0x7f9d600cc340 2023-10-06T05:18:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.949+0000 7f9d683ff700 1 -- 172.21.15.31:0/2633207786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6016a760 con 0x7f9d600cc340 2023-10-06T05:18:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.949+0000 7f9d683ff700 1 -- 172.21.15.31:0/2633207786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6016ad20 con 0x7f9d600cc340 2023-10-06T05:18:22.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.950+0000 7f9d683ff700 1 -- 172.21.15.31:0/2633207786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6016d660 con 0x7f9d600cc340 2023-10-06T05:18:22.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.950+0000 7f9d567fc700 1 -- 172.21.15.31:0/2633207786 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9d5c01bbc0 con 0x7f9d600cc340 2023-10-06T05:18:22.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.951+0000 7f9d567fc700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9d440670e0 0x7f9d44069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.951+0000 7f9d567fc700 1 -- 172.21.15.31:0/2633207786 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9d5c086480 con 0x7f9d600cc340 2023-10-06T05:18:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.952+0000 7f9d6599a700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9d440670e0 0x7f9d44069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.952+0000 7f9d6599a700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9d440670e0 0x7f9d44069590 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9d601530a0 tx=0x7f9d50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:22.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:22.956+0000 7f9d567fc700 1 -- 172.21.15.31:0/2633207786 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d6016d660 con 0x7f9d600cc340 2023-10-06T05:18:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.162+0000 7f9d683ff700 1 -- 172.21.15.31:0/2633207786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d6016db70 con 0x7f9d600cc340 2023-10-06T05:18:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.162+0000 7f9d567fc700 1 -- 172.21.15.31:0/2633207786 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d5c053270 con 0x7f9d600cc340 2023-10-06T05:18:23.164 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:18:23.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.164+0000 7f9d683ff700 1 -- 172.21.15.31:0/2633207786 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9d440670e0 msgr2=0x7f9d44069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.164+0000 7f9d683ff700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9d440670e0 0x7f9d44069590 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9d601530a0 tx=0x7f9d50009040 comp rx=0 tx=0).stop 2023-10-06T05:18:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.164+0000 7f9d683ff700 1 -- 172.21.15.31:0/2633207786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 msgr2=0x7f9d6016a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.164+0000 7f9d683ff700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 0x7f9d6016a060 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b3c0 tx=0x7f9d5c016000 comp rx=0 tx=0).stop 2023-10-06T05:18:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.165+0000 7f9d683ff700 1 -- 172.21.15.31:0/2633207786 shutdown_connections 2023-10-06T05:18:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.165+0000 7f9d683ff700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d6015a4b0 0x7f9d6016b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.165+0000 7f9d683ff700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9d440670e0 0x7f9d44069590 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.165+0000 7f9d683ff700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d600cc340 0x7f9d6016a060 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.165+0000 7f9d683ff700 1 --2- 172.21.15.31:0/2633207786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d600cc710 0x7f9d600ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.165+0000 7f9d683ff700 1 -- 172.21.15.31:0/2633207786 >> 172.21.15.31:0/2633207786 conn(0x7f9d60068670 msgr2=0x7f9d60158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.165+0000 7f9d683ff700 1 -- 172.21.15.31:0/2633207786 shutdown_connections 2023-10-06T05:18:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.165+0000 7f9d683ff700 1 -- 172.21.15.31:0/2633207786 wait complete. 2023-10-06T05:18:23.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:18:23.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.382+0000 7fbedf3ff700 1 Processor -- start 2023-10-06T05:18:23.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.382+0000 7fbedf3ff700 1 -- start start 2023-10-06T05:18:23.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedf3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbed8159930 0x7fbed80b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedf3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed80cd580 0x7fbed80b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedf3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed815d2d0 con 0x7fbed8159560 2023-10-06T05:18:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedf3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed815d5b0 con 0x7fbed8159930 2023-10-06T05:18:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedf3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed815d890 con 0x7fbed80cd580 2023-10-06T05:18:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedc99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbed8159930 0x7fbed80b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedc99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbed8159930 0x7fbed80b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59006/0 (socket says 172.21.15.31:59006) 2023-10-06T05:18:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedc99a700 1 -- 172.21.15.31:0/671864614 learned_addr learned my addr 172.21.15.31:0/671864614 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:23.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedd99c700 1 --2- 172.21.15.31:0/671864614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed80cd580 0x7fbed80b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedc99a700 1 -- 172.21.15.31:0/671864614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed80cd580 msgr2=0x7fbed80b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedc99a700 1 --2- 172.21.15.31:0/671864614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed80cd580 0x7fbed80b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedc99a700 1 -- 172.21.15.31:0/671864614 >> v1:172.21.15.62:6789/0 conn(0x7fbed8159560 legacy=0x7fbed80cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedc99a700 1 -- 172.21.15.31:0/671864614 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed815db70 con 0x7fbed8159930 2023-10-06T05:18:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.384+0000 7fbedd99c700 1 --2- 172.21.15.31:0/671864614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed80cd580 0x7fbed80b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.385+0000 7fbedc99a700 1 --2- 172.21.15.31:0/671864614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed8159930 0x7fbed80b2470 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbec80099b0 tx=0x7fbec8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf718a01f073bc24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.385+0000 7fbecffff700 1 -- 172.21.15.31:0/671864614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800b820 con 0x7fbed8159930 2023-10-06T05:18:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.385+0000 7fbecffff700 1 -- 172.21.15.31:0/671864614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec800b980 con 0x7fbed8159930 2023-10-06T05:18:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.385+0000 7fbecffff700 1 -- 172.21.15.31:0/671864614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800bd20 con 0x7fbed8159930 2023-10-06T05:18:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.385+0000 7fbedf3ff700 1 -- 172.21.15.31:0/671864614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed8159930 msgr2=0x7fbed80b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.385+0000 7fbedf3ff700 1 --2- 172.21.15.31:0/671864614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed8159930 0x7fbed80b2470 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbec80099b0 tx=0x7fbec8009250 comp rx=0 tx=0).stop 2023-10-06T05:18:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.385+0000 7fbedf3ff700 1 -- 172.21.15.31:0/671864614 shutdown_connections 2023-10-06T05:18:23.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.385+0000 7fbedf3ff700 1 --2- 172.21.15.31:0/671864614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed80cd580 0x7fbed80b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.385+0000 7fbedf3ff700 1 --2- 172.21.15.31:0/671864614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed8159930 0x7fbed80b2470 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.385+0000 7fbedf3ff700 1 -- 172.21.15.31:0/671864614 >> 172.21.15.31:0/671864614 conn(0x7fbed806ec90 msgr2=0x7fbed806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.386+0000 7fbedf3ff700 1 -- 172.21.15.31:0/671864614 shutdown_connections 2023-10-06T05:18:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.386+0000 7fbedf3ff700 1 -- 172.21.15.31:0/671864614 wait complete. 2023-10-06T05:18:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.387+0000 7fbedf3ff700 1 Processor -- start 2023-10-06T05:18:23.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.387+0000 7fbedf3ff700 1 -- start start 2023-10-06T05:18:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.388+0000 7fbedf3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed80cd580 0x7fbed80c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.388+0000 7fbedf3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8159560 0x7fbed80b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.388+0000 7fbedf3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed8159930 0x7fbed80b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.388+0000 7fbedf3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80c8e20 con 0x7fbed80cd580 2023-10-06T05:18:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.388+0000 7fbedf3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80c8210 con 0x7fbed8159560 2023-10-06T05:18:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.388+0000 7fbedf3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80c84f0 con 0x7fbed8159930 2023-10-06T05:18:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.388+0000 7fbedd19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed80cd580 0x7fbed80c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.388+0000 7fbedd19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed80cd580 0x7fbed80c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59018/0 (socket says 172.21.15.31:59018) 2023-10-06T05:18:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.388+0000 7fbedd19b700 1 -- 172.21.15.31:0/2984571383 learned_addr learned my addr 172.21.15.31:0/2984571383 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.389+0000 7fbedd99c700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed8159930 0x7fbed80b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.389+0000 7fbedc99a700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8159560 0x7fbed80b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.389+0000 7fbedd19b700 1 -- 172.21.15.31:0/2984571383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed8159930 msgr2=0x7fbed80b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.389+0000 7fbedd19b700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed8159930 0x7fbed80b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.389+0000 7fbedd19b700 1 -- 172.21.15.31:0/2984571383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8159560 msgr2=0x7fbed80b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.389+0000 7fbedd19b700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8159560 0x7fbed80b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.389+0000 7fbedd19b700 1 -- 172.21.15.31:0/2984571383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80094f0 con 0x7fbed80cd580 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.389+0000 7fbedd99c700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed8159930 0x7fbed80b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.389+0000 7fbedc99a700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8159560 0x7fbed80b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.389+0000 7fbedd19b700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed80cd580 0x7fbed80c78a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbed400c5d0 tx=0x7fbed4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.390+0000 7fbecd7fa700 1 -- 172.21.15.31:0/2984571383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4011590 con 0x7fbed80cd580 2023-10-06T05:18:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.390+0000 7fbecd7fa700 1 -- 172.21.15.31:0/2984571383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed4011710 con 0x7fbed80cd580 2023-10-06T05:18:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.390+0000 7fbedf3ff700 1 -- 172.21.15.31:0/2984571383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed80c87d0 con 0x7fbed80cd580 2023-10-06T05:18:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.390+0000 7fbecd7fa700 1 -- 172.21.15.31:0/2984571383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4011ad0 con 0x7fbed80cd580 2023-10-06T05:18:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.390+0000 7fbedf3ff700 1 -- 172.21.15.31:0/2984571383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed80c7de0 con 0x7fbed80cd580 2023-10-06T05:18:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.391+0000 7fbedf3ff700 1 -- 172.21.15.31:0/2984571383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed80c9100 con 0x7fbed80cd580 2023-10-06T05:18:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.392+0000 7fbecd7fa700 1 -- 172.21.15.31:0/2984571383 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fbed401c460 con 0x7fbed80cd580 2023-10-06T05:18:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.393+0000 7fbecd7fa700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbebc0670e0 0x7fbebc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.393+0000 7fbecd7fa700 1 -- 172.21.15.31:0/2984571383 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbed4084880 con 0x7fbed80cd580 2023-10-06T05:18:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.393+0000 7fbedc99a700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbebc0670e0 0x7fbebc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.394+0000 7fbedc99a700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbebc0670e0 0x7fbebc069590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbec80099b0 tx=0x7fbec8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:23.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.397+0000 7fbecd7fa700 1 -- 172.21.15.31:0/2984571383 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbed4051670 con 0x7fbed80cd580 2023-10-06T05:18:23.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.599+0000 7fbedf3ff700 1 -- 172.21.15.31:0/2984571383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbed804c380 con 0x7fbed80cd580 2023-10-06T05:18:23.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.600+0000 7fbecd7fa700 1 -- 172.21.15.31:0/2984571383 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbed4054c90 con 0x7fbed80cd580 2023-10-06T05:18:23.602 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:18:23.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 -- 172.21.15.31:0/2984571383 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbebc0670e0 msgr2=0x7fbebc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:23.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbebc0670e0 0x7fbebc069590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbec80099b0 tx=0x7fbec8016000 comp rx=0 tx=0).stop 2023-10-06T05:18:23.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 -- 172.21.15.31:0/2984571383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed80cd580 msgr2=0x7fbed80c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:23.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed80cd580 0x7fbed80c78a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbed400c5d0 tx=0x7fbed4007f60 comp rx=0 tx=0).stop 2023-10-06T05:18:23.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 -- 172.21.15.31:0/2984571383 shutdown_connections 2023-10-06T05:18:23.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbed8159930 0x7fbed80b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbebc0670e0 0x7fbebc069590 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed80cd580 0x7fbed80c78a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 --2- 172.21.15.31:0/2984571383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed8159560 0x7fbed80b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 -- 172.21.15.31:0/2984571383 >> 172.21.15.31:0/2984571383 conn(0x7fbed806ec90 msgr2=0x7fbed815caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 -- 172.21.15.31:0/2984571383 shutdown_connections 2023-10-06T05:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.602+0000 7fbedf3ff700 1 -- 172.21.15.31:0/2984571383 wait complete. 2023-10-06T05:18:23.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:23.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.815+0000 7ff6e19ff700 1 Processor -- start 2023-10-06T05:18:23.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.815+0000 7ff6e19ff700 1 -- start start 2023-10-06T05:18:23.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.816+0000 7ff6e19ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff6dc159500 0x7ff6dc0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.816+0000 7ff6e19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc15d2d0 0x7ff6dc0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.816+0000 7ff6e19ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6dc1598d0 0x7ff6dc0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.816+0000 7ff6e19ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc0c3f40 con 0x7ff6dc1598d0 2023-10-06T05:18:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.816+0000 7ff6e19ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc0cb000 con 0x7ff6dc159500 2023-10-06T05:18:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.816+0000 7ff6e19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0b2860 con 0x7ff6dc15d2d0 2023-10-06T05:18:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.817+0000 7ff6db7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6dc1598d0 0x7ff6dc0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.817+0000 7ff6db7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6dc1598d0 0x7ff6dc0cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59024/0 (socket says 172.21.15.31:59024) 2023-10-06T05:18:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.817+0000 7ff6db7fe700 1 -- 172.21.15.31:0/2022060656 learned_addr learned my addr 172.21.15.31:0/2022060656 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.817+0000 7ff6da7fc700 1 --2- 172.21.15.31:0/2022060656 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc15d2d0 0x7ff6dc0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.817+0000 7ff6daffd700 1 --2- 172.21.15.31:0/2022060656 >> v2:172.21.15.62:3300/0 conn(0x7ff6dc159500 0x7ff6dc0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.817+0000 7ff6db7fe700 1 -- 172.21.15.31:0/2022060656 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc15d2d0 msgr2=0x7ff6dc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.817+0000 7ff6db7fe700 1 --2- 172.21.15.31:0/2022060656 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc15d2d0 0x7ff6dc0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.817+0000 7ff6db7fe700 1 -- 172.21.15.31:0/2022060656 >> v2:172.21.15.62:3300/0 conn(0x7ff6dc159500 msgr2=0x7ff6dc0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.817+0000 7ff6db7fe700 1 --2- 172.21.15.31:0/2022060656 >> v2:172.21.15.62:3300/0 conn(0x7ff6dc159500 0x7ff6dc0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.817+0000 7ff6db7fe700 1 -- 172.21.15.31:0/2022060656 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc15d7d0 con 0x7ff6dc1598d0 2023-10-06T05:18:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.818+0000 7ff6db7fe700 1 --2- 172.21.15.31:0/2022060656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 0x7ff6dc0cd340 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc089840 tx=0x7ff6d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e166fa7059ef6db7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.818+0000 7ff6d9ffb700 1 -- 172.21.15.31:0/2022060656 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400e070 con 0x7ff6dc1598d0 2023-10-06T05:18:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.818+0000 7ff6d9ffb700 1 -- 172.21.15.31:0/2022060656 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d400b970 con 0x7ff6dc1598d0 2023-10-06T05:18:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.818+0000 7ff6d9ffb700 1 -- 172.21.15.31:0/2022060656 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40092e0 con 0x7ff6dc1598d0 2023-10-06T05:18:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.818+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2022060656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 msgr2=0x7ff6dc0cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.818+0000 7ff6e19ff700 1 --2- 172.21.15.31:0/2022060656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 0x7ff6dc0cd340 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc089840 tx=0x7ff6d4009840 comp rx=0 tx=0).stop 2023-10-06T05:18:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.819+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2022060656 shutdown_connections 2023-10-06T05:18:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.819+0000 7ff6e19ff700 1 --2- 172.21.15.31:0/2022060656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 0x7ff6dc0cd340 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.819+0000 7ff6e19ff700 1 --2- 172.21.15.31:0/2022060656 >> v2:172.21.15.62:3300/0 conn(0x7ff6dc159500 0x7ff6dc0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.819+0000 7ff6e19ff700 1 --2- 172.21.15.31:0/2022060656 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc15d2d0 0x7ff6dc0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.819+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2022060656 >> 172.21.15.31:0/2022060656 conn(0x7ff6dc06ec90 msgr2=0x7ff6dc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.819+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2022060656 shutdown_connections 2023-10-06T05:18:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.819+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2022060656 wait complete. 2023-10-06T05:18:23.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.820+0000 7ff6e19ff700 1 Processor -- start 2023-10-06T05:18:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.820+0000 7ff6e19ff700 1 -- start start 2023-10-06T05:18:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.821+0000 7ff6e19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc159500 0x7ff6dc157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.821+0000 7ff6e19ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 0x7ff6dc0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.821+0000 7ff6e19ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6dc15d2d0 0x7ff6dc0c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.821+0000 7ff6e19ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc152260 con 0x7ff6dc1598d0 2023-10-06T05:18:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.821+0000 7ff6e19ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0ccc00 con 0x7ff6dc15d2d0 2023-10-06T05:18:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.821+0000 7ff6e19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0cbff0 con 0x7ff6dc159500 2023-10-06T05:18:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.822+0000 7ff6da7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 0x7ff6dc0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.822+0000 7ff6da7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 0x7ff6dc0cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59040/0 (socket says 172.21.15.31:59040) 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.822+0000 7ff6da7fc700 1 -- 172.21.15.31:0/2021100624 learned_addr learned my addr 172.21.15.31:0/2021100624 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.822+0000 7ff6da7fc700 1 -- 172.21.15.31:0/2021100624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc159500 msgr2=0x7ff6dc157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.822+0000 7ff6da7fc700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc159500 0x7ff6dc157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.822+0000 7ff6da7fc700 1 -- 172.21.15.31:0/2021100624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6dc15d2d0 msgr2=0x7ff6dc0c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.822+0000 7ff6da7fc700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6dc15d2d0 0x7ff6dc0c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.822+0000 7ff6da7fc700 1 -- 172.21.15.31:0/2021100624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4009ae0 con 0x7ff6dc1598d0 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.822+0000 7ff6da7fc700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 0x7ff6dc0cb7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000ec50 tx=0x7ff6d0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.823+0000 7ff6c2ffd700 1 -- 172.21.15.31:0/2021100624 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000c9d0 con 0x7ff6dc1598d0 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.823+0000 7ff6c2ffd700 1 -- 172.21.15.31:0/2021100624 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d000cb30 con 0x7ff6dc1598d0 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.823+0000 7ff6c2ffd700 1 -- 172.21.15.31:0/2021100624 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0013470 con 0x7ff6dc1598d0 2023-10-06T05:18:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.823+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2021100624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc0cc2d0 con 0x7ff6dc1598d0 2023-10-06T05:18:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.823+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2021100624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc0cc890 con 0x7ff6dc1598d0 2023-10-06T05:18:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.824+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2021100624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6dc04c380 con 0x7ff6dc1598d0 2023-10-06T05:18:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.825+0000 7ff6c2ffd700 1 -- 172.21.15.31:0/2021100624 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7ff6d0013820 con 0x7ff6dc1598d0 2023-10-06T05:18:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.826+0000 7ff6c2ffd700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff6c80670e0 0x7ff6c8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.826+0000 7ff6c2ffd700 1 -- 172.21.15.31:0/2021100624 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff6d00850f0 con 0x7ff6dc1598d0 2023-10-06T05:18:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.826+0000 7ff6daffd700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff6c80670e0 0x7ff6c8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.827+0000 7ff6daffd700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff6c80670e0 0x7ff6c8069590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009b10 tx=0x7ff6cc0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:23.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:23.829+0000 7ff6c2ffd700 1 -- 172.21.15.31:0/2021100624 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6d0051ee0 con 0x7ff6dc1598d0 2023-10-06T05:18:24.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.077+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2021100624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6dc0ccd70 con 0x7ff6dc1598d0 2023-10-06T05:18:24.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.077+0000 7ff6c2ffd700 1 -- 172.21.15.31:0/2021100624 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4591 (secure 0 0 0) 0x7ff6d0055500 con 0x7ff6dc1598d0 2023-10-06T05:18:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:24.079 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/746584225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":746584225},{"type":"v1","addr":"172.21.15.62:6834","nonce":746584225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/584374139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":584374139},{"type":"v1","addr":"172.21.15.62:6835","nonce":584374139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4926,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3794168057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3794168057},{"type":"v1","addr":"172.21.15.31:6836","nonce":3794168057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:00.090015+0000","modified":"2023-10-06T05:18:03.257587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/3886723511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3886723511},{"type":"v1","addr":"172.21.15.31:6837","nonce":3886723511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:18:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2021100624 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff6c80670e0 msgr2=0x7ff6c8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff6c80670e0 0x7ff6c8069590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009b10 tx=0x7ff6cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2021100624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 msgr2=0x7ff6dc0cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 0x7ff6dc0cb7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000ec50 tx=0x7ff6d0009120 comp rx=0 tx=0).stop 2023-10-06T05:18:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2021100624 shutdown_connections 2023-10-06T05:18:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6dc15d2d0 0x7ff6dc0c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc159500 0x7ff6dc157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff6c80670e0 0x7ff6c8069590 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 --2- 172.21.15.31:0/2021100624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6dc1598d0 0x7ff6dc0cb7a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2021100624 >> 172.21.15.31:0/2021100624 conn(0x7ff6dc06ec90 msgr2=0x7ff6dc0c40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2021100624 shutdown_connections 2023-10-06T05:18:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.080+0000 7ff6e19ff700 1 -- 172.21.15.31:0/2021100624 wait complete. 2023-10-06T05:18:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2023-10-06T05:18:24.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:24.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.297+0000 7f3f589ff700 1 Processor -- start 2023-10-06T05:18:24.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.297+0000 7f3f589ff700 1 -- start start 2023-10-06T05:18:24.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.298+0000 7f3f589ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3f540cd510 0x7f3f540b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.298+0000 7f3f589ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f540c3f40 con 0x7f3f5415d2d0 2023-10-06T05:18:24.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.298+0000 7f3f589ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f540cb000 con 0x7f3f54159500 2023-10-06T05:18:24.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.298+0000 7f3f589ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f540b2860 con 0x7f3f540cd510 2023-10-06T05:18:24.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.299+0000 7f3f52d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3f5415d2d0 0x7f3f540aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56278/0 (socket says 172.21.15.31:56278) 2023-10-06T05:18:24.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.299+0000 7f3f52d9d700 1 -- 172.21.15.31:0/2802207766 learned_addr learned my addr 172.21.15.31:0/2802207766 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:24.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.299+0000 7f3f51d9b700 1 --2- 172.21.15.31:0/2802207766 >> v2:172.21.15.62:3300/0 conn(0x7f3f540cd510 0x7f3f540b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.299+0000 7f3f5159a700 1 -- 172.21.15.31:0/2802207766 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056284217 0 0) 0x7f3f540c3f40 con 0x7f3f5415d2d0 2023-10-06T05:18:24.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.299+0000 7f3f5159a700 1 -- 172.21.15.31:0/2802207766 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f3c003570 con 0x7f3f5415d2d0 2023-10-06T05:18:24.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.299+0000 7f3f51d9b700 1 -- 172.21.15.31:0/2802207766 >> v1:172.21.15.62:6789/0 conn(0x7f3f54159500 legacy=0x7f3f540cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.299+0000 7f3f51d9b700 1 -- 172.21.15.31:0/2802207766 >> v1:172.21.15.31:6789/0 conn(0x7f3f5415d2d0 legacy=0x7f3f540aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.299+0000 7f3f51d9b700 1 -- 172.21.15.31:0/2802207766 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5415d7d0 con 0x7f3f540cd510 2023-10-06T05:18:24.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.300+0000 7f3f51d9b700 1 --2- 172.21.15.31:0/2802207766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f540cd510 0x7f3f540b0ab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f480098d0 tx=0x7f3f48009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da54d8da8781ef1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:24.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.300+0000 7f3f5159a700 1 -- 172.21.15.31:0/2802207766 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48005c10 con 0x7f3f540cd510 2023-10-06T05:18:24.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.300+0000 7f3f5159a700 1 -- 172.21.15.31:0/2802207766 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48005d70 con 0x7f3f540cd510 2023-10-06T05:18:24.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.300+0000 7f3f5159a700 1 -- 172.21.15.31:0/2802207766 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48006280 con 0x7f3f540cd510 2023-10-06T05:18:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.300+0000 7f3f589ff700 1 -- 172.21.15.31:0/2802207766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f540cd510 msgr2=0x7f3f540b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.300+0000 7f3f589ff700 1 --2- 172.21.15.31:0/2802207766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f540cd510 0x7f3f540b0ab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f480098d0 tx=0x7f3f48009120 comp rx=0 tx=0).stop 2023-10-06T05:18:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.301+0000 7f3f589ff700 1 -- 172.21.15.31:0/2802207766 shutdown_connections 2023-10-06T05:18:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.301+0000 7f3f589ff700 1 --2- 172.21.15.31:0/2802207766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f540cd510 0x7f3f540b0ab0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.301+0000 7f3f589ff700 1 -- 172.21.15.31:0/2802207766 >> 172.21.15.31:0/2802207766 conn(0x7f3f5406ec90 msgr2=0x7f3f5406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.301+0000 7f3f589ff700 1 -- 172.21.15.31:0/2802207766 shutdown_connections 2023-10-06T05:18:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.301+0000 7f3f589ff700 1 -- 172.21.15.31:0/2802207766 wait complete. 2023-10-06T05:18:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.302+0000 7f3f589ff700 1 Processor -- start 2023-10-06T05:18:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.302+0000 7f3f589ff700 1 -- start start 2023-10-06T05:18:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.303+0000 7f3f589ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f540cd510 0x7f3f54159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.303+0000 7f3f589ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f54159500 0x7f3f5415b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.303+0000 7f3f589ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f5415d2d0 0x7f3f54150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.303+0000 7f3f589ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540b3b60 con 0x7f3f5415d2d0 2023-10-06T05:18:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.303+0000 7f3f589ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540b3e40 con 0x7f3f54159500 2023-10-06T05:18:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.303+0000 7f3f589ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540b4120 con 0x7f3f540cd510 2023-10-06T05:18:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.303+0000 7f3f52d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f5415d2d0 0x7f3f54150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.304+0000 7f3f52d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f5415d2d0 0x7f3f54150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59046/0 (socket says 172.21.15.31:59046) 2023-10-06T05:18:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.304+0000 7f3f52d9d700 1 -- 172.21.15.31:0/1233075958 learned_addr learned my addr 172.21.15.31:0/1233075958 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.304+0000 7f3f51d9b700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f54159500 0x7f3f5415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.304+0000 7f3f5259c700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f540cd510 0x7f3f54159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.304+0000 7f3f52d9d700 1 -- 172.21.15.31:0/1233075958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f540cd510 msgr2=0x7f3f54159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.304+0000 7f3f52d9d700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f540cd510 0x7f3f54159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.304+0000 7f3f52d9d700 1 -- 172.21.15.31:0/1233075958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f54159500 msgr2=0x7f3f5415b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.304+0000 7f3f52d9d700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f54159500 0x7f3f5415b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.304+0000 7f3f52d9d700 1 -- 172.21.15.31:0/1233075958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f480093c0 con 0x7f3f5415d2d0 2023-10-06T05:18:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.304+0000 7f3f52d9d700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f5415d2d0 0x7f3f54150560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c001460 tx=0x7f3f4c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.305+0000 7f3f427fc700 1 -- 172.21.15.31:0/1233075958 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00a870 con 0x7f3f5415d2d0 2023-10-06T05:18:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.305+0000 7f3f427fc700 1 -- 172.21.15.31:0/1233075958 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c00a9d0 con 0x7f3f5415d2d0 2023-10-06T05:18:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.305+0000 7f3f427fc700 1 -- 172.21.15.31:0/1233075958 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00ad50 con 0x7f3f5415d2d0 2023-10-06T05:18:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.305+0000 7f3f589ff700 1 -- 172.21.15.31:0/1233075958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f540b4400 con 0x7f3f5415d2d0 2023-10-06T05:18:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.305+0000 7f3f589ff700 1 -- 172.21.15.31:0/1233075958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f54159fa0 con 0x7f3f5415d2d0 2023-10-06T05:18:24.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.306+0000 7f3f589ff700 1 -- 172.21.15.31:0/1233075958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f540b4920 con 0x7f3f5415d2d0 2023-10-06T05:18:24.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.307+0000 7f3f427fc700 1 -- 172.21.15.31:0/1233075958 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f3f4c007300 con 0x7f3f5415d2d0 2023-10-06T05:18:24.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.309+0000 7f3f427fc700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f340670e0 0x7f3f34069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.309+0000 7f3f427fc700 1 -- 172.21.15.31:0/1233075958 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3f4c084d70 con 0x7f3f5415d2d0 2023-10-06T05:18:24.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.309+0000 7f3f5259c700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f340670e0 0x7f3f34069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.309+0000 7f3f5259c700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f340670e0 0x7f3f34069590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f44003a80 tx=0x7f3f44007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:24.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.313+0000 7f3f427fc700 1 -- 172.21.15.31:0/1233075958 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f540b4920 con 0x7f3f5415d2d0 2023-10-06T05:18:24.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.515+0000 7f3f589ff700 1 -- 172.21.15.31:0/1233075958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f540b4920 con 0x7f3f5415d2d0 2023-10-06T05:18:24.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.516+0000 7f3f427fc700 1 -- 172.21.15.31:0/1233075958 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f3f540b4920 con 0x7f3f5415d2d0 2023-10-06T05:18:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:24.518 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:01.245088+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:58.074919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:58.854233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:24.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 -- 172.21.15.31:0/1233075958 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f340670e0 msgr2=0x7f3f34069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f340670e0 0x7f3f34069590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f44003a80 tx=0x7f3f44007ec0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 -- 172.21.15.31:0/1233075958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f5415d2d0 msgr2=0x7f3f54150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f5415d2d0 0x7f3f54150560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c001460 tx=0x7f3f4c001a40 comp rx=0 tx=0).stop 2023-10-06T05:18:24.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 -- 172.21.15.31:0/1233075958 shutdown_connections 2023-10-06T05:18:24.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3f340670e0 0x7f3f34069590 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f5415d2d0 0x7f3f54150560 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f540cd510 0x7f3f54159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 --2- 172.21.15.31:0/1233075958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f54159500 0x7f3f5415b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 -- 172.21.15.31:0/1233075958 >> 172.21.15.31:0/1233075958 conn(0x7f3f5406ec90 msgr2=0x7f3f540c5130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:24.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 -- 172.21.15.31:0/1233075958 shutdown_connections 2023-10-06T05:18:24.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.518+0000 7f3f589ff700 1 -- 172.21.15.31:0/1233075958 wait complete. 2023-10-06T05:18:24.538 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:18:24.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:24.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.732+0000 7fb1dd9ff700 1 Processor -- start 2023-10-06T05:18:24.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.732+0000 7fb1dd9ff700 1 -- start start 2023-10-06T05:18:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.733+0000 7fb1dd9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1d81595a0 0x7fb1d8159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.733+0000 7fb1dd9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1d8068d80 0x7fb1d80c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.733+0000 7fb1dd9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d80c95b0 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.733+0000 7fb1dd9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d80c9890 con 0x7fb1d8068d80 2023-10-06T05:18:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.733+0000 7fb1dd9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d80c9b70 con 0x7fb1d81595a0 2023-10-06T05:18:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.733+0000 7fb1d77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1d8068d80 0x7fb1d80c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.733+0000 7fb1d77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1d8068d80 0x7fb1d80c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59054/0 (socket says 172.21.15.31:59054) 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.733+0000 7fb1d77fe700 1 -- 172.21.15.31:0/861258778 learned_addr learned my addr 172.21.15.31:0/861258778 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.733+0000 7fb1d67fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb1d80cc0d0 0x7fb1d8068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56280/0 (socket says 172.21.15.31:56280) 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.734+0000 7fb1d6ffd700 1 --2- 172.21.15.31:0/861258778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1d81595a0 0x7fb1d8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.734+0000 7fb1d77fe700 1 -- 172.21.15.31:0/861258778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1d81595a0 msgr2=0x7fb1d8159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.734+0000 7fb1d77fe700 1 --2- 172.21.15.31:0/861258778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1d81595a0 0x7fb1d8159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.734+0000 7fb1d77fe700 1 -- 172.21.15.31:0/861258778 >> v1:172.21.15.31:6789/0 conn(0x7fb1d80cc0d0 legacy=0x7fb1d8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.734+0000 7fb1d77fe700 1 -- 172.21.15.31:0/861258778 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d80ca130 con 0x7fb1d8068d80 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.734+0000 7fb1d5ffb700 1 -- 172.21.15.31:0/861258778 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508997225 0 0) 0x7fb1d80c95b0 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.734+0000 7fb1d77fe700 1 --2- 172.21.15.31:0/861258778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8068d80 0x7fb1d80c8930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009f20 tx=0x7fb1d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ad023034233893 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.734+0000 7fb1d5ffb700 1 -- 172.21.15.31:0/861258778 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d000e070 con 0x7fb1d8068d80 2023-10-06T05:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.734+0000 7fb1d5ffb700 1 -- 172.21.15.31:0/861258778 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d000ba80 con 0x7fb1d8068d80 2023-10-06T05:18:24.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.734+0000 7fb1d5ffb700 1 -- 172.21.15.31:0/861258778 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00092e0 con 0x7fb1d8068d80 2023-10-06T05:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.735+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/861258778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8068d80 msgr2=0x7fb1d80c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.735+0000 7fb1dd9ff700 1 --2- 172.21.15.31:0/861258778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8068d80 0x7fb1d80c8930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009f20 tx=0x7fb1d0009840 comp rx=0 tx=0).stop 2023-10-06T05:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.735+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/861258778 shutdown_connections 2023-10-06T05:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.735+0000 7fb1dd9ff700 1 --2- 172.21.15.31:0/861258778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d8068d80 0x7fb1d80c8930 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.735+0000 7fb1dd9ff700 1 --2- 172.21.15.31:0/861258778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1d81595a0 0x7fb1d8159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.735+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/861258778 >> 172.21.15.31:0/861258778 conn(0x7fb1d806b620 msgr2=0x7fb1d806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.735+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/861258778 shutdown_connections 2023-10-06T05:18:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.735+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/861258778 wait complete. 2023-10-06T05:18:24.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.736+0000 7fb1dd9ff700 1 Processor -- start 2023-10-06T05:18:24.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.737+0000 7fb1dd9ff700 1 -- start start 2023-10-06T05:18:24.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.737+0000 7fb1dd9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1d8068d80 0x7fb1d8156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.737+0000 7fb1dd9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d80cc0d0 0x7fb1d81546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1dd9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1d81595a0 0x7fb1d8154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1dd9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d80c6760 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1dd9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d81552c0 con 0x7fb1d8068d80 2023-10-06T05:18:24.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1dd9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d81555a0 con 0x7fb1d81595a0 2023-10-06T05:18:24.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d80cc0d0 0x7fb1d81546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d80cc0d0 0x7fb1d81546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59070/0 (socket says 172.21.15.31:59070) 2023-10-06T05:18:24.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d67fc700 1 -- 172.21.15.31:0/284180917 learned_addr learned my addr 172.21.15.31:0/284180917 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:24.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d6ffd700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1d8068d80 0x7fb1d8156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d77fe700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1d81595a0 0x7fb1d8154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d67fc700 1 -- 172.21.15.31:0/284180917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1d81595a0 msgr2=0x7fb1d8154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d67fc700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1d81595a0 0x7fb1d8154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d67fc700 1 -- 172.21.15.31:0/284180917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1d8068d80 msgr2=0x7fb1d8156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:24.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d67fc700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1d8068d80 0x7fb1d8156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:24.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d67fc700 1 -- 172.21.15.31:0/284180917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0009ae0 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.738+0000 7fb1d67fc700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d80cc0d0 0x7fb1d81546b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00c5b0 tx=0x7fb1cc001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:24.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.739+0000 7fb1beffd700 1 -- 172.21.15.31:0/284180917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc005a70 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.739+0000 7fb1beffd700 1 -- 172.21.15.31:0/284180917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc005bd0 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.739+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/284180917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d8155880 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.739+0000 7fb1beffd700 1 -- 172.21.15.31:0/284180917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0070d0 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.739+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/284180917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d8155e40 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.740+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/284180917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d8156060 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.741+0000 7fb1beffd700 1 -- 172.21.15.31:0/284180917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fb1cc007440 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.743+0000 7fb1beffd700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb1c40670e0 0x7fb1c4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:24.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.743+0000 7fb1beffd700 1 -- 172.21.15.31:0/284180917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb1cc083f30 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.743+0000 7fb1d6ffd700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb1c40670e0 0x7fb1c4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:24.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.743+0000 7fb1d6ffd700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb1c40670e0 0x7fb1c4069590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8006fd0 tx=0x7fb1c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:24.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.747+0000 7fb1beffd700 1 -- 172.21.15.31:0/284180917 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1cc050d20 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.995+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/284180917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1d804c380 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.996+0000 7fb1beffd700 1 -- 172.21.15.31:0/284180917 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fb1cc017020 con 0x7fb1d80cc0d0 2023-10-06T05:18:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:24.999 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/746584225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":746584225},{"type":"v1","addr":"172.21.15.62:6834","nonce":746584225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/584374139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":584374139},{"type":"v1","addr":"172.21.15.62:6835","nonce":584374139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4926,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3794168057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3794168057},{"type":"v1","addr":"172.21.15.31:6836","nonce":3794168057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:00.090015+0000","modified":"2023-10-06T05:18:03.257587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/3886723511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3886723511},{"type":"v1","addr":"172.21.15.31:6837","nonce":3886723511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:18:25.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/284180917 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb1c40670e0 msgr2=0x7fb1c4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb1c40670e0 0x7fb1c4069590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8006fd0 tx=0x7fb1c8008040 comp rx=0 tx=0).stop 2023-10-06T05:18:25.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/284180917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d80cc0d0 msgr2=0x7fb1d81546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d80cc0d0 0x7fb1d81546b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00c5b0 tx=0x7fb1cc001040 comp rx=0 tx=0).stop 2023-10-06T05:18:25.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/284180917 shutdown_connections 2023-10-06T05:18:25.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1d81595a0 0x7fb1d8154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1d8068d80 0x7fb1d8156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb1c40670e0 0x7fb1c4069590 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 --2- 172.21.15.31:0/284180917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d80cc0d0 0x7fb1d81546b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/284180917 >> 172.21.15.31:0/284180917 conn(0x7fb1d806b620 msgr2=0x7fb1d806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:25.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/284180917 shutdown_connections 2023-10-06T05:18:25.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:24.998+0000 7fb1dd9ff700 1 -- 172.21.15.31:0/284180917 wait complete. 2023-10-06T05:18:25.002 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2023-10-06T05:18:25.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:25.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.213+0000 7f9773fff700 1 Processor -- start 2023-10-06T05:18:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.213+0000 7f9773fff700 1 -- start start 2023-10-06T05:18:25.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.214+0000 7f9773fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f976c159500 0x7f976c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.214+0000 7f9773fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c15d2d0 0x7f976c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.214+0000 7f9773fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976c0c3f40 con 0x7f976c1598d0 2023-10-06T05:18:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.214+0000 7f9773fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976c0b2860 con 0x7f976c159500 2023-10-06T05:18:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.214+0000 7f9773fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c15d7d0 con 0x7f976c15d2d0 2023-10-06T05:18:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.215+0000 7f9771d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f976c159500 0x7f976c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.215+0000 7f977159a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c15d2d0 0x7f976c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.215+0000 7f977159a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c15d2d0 0x7f976c0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:45830/0 (socket says 172.21.15.31:45830) 2023-10-06T05:18:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.215+0000 7f977159a700 1 -- 172.21.15.31:0/986155624 learned_addr learned my addr 172.21.15.31:0/986155624 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.215+0000 7f9771d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f976c159500 0x7f976c0b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:50696/0 (socket says 172.21.15.31:50696) 2023-10-06T05:18:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.215+0000 7f977259c700 1 --1- 172.21.15.31:0/986155624 >> v1:172.21.15.62:6789/0 conn(0x7f976c1598d0 0x7f976c0aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:53300/0 (socket says 172.21.15.31:53300) 2023-10-06T05:18:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.215+0000 7f977159a700 1 -- 172.21.15.31:0/986155624 >> v2:172.21.15.62:3300/0 conn(0x7f976c159500 msgr2=0x7f976c0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.216+0000 7f977159a700 1 --2- 172.21.15.31:0/986155624 >> v2:172.21.15.62:3300/0 conn(0x7f976c159500 0x7f976c0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.216+0000 7f977159a700 1 -- 172.21.15.31:0/986155624 >> v1:172.21.15.62:6789/0 conn(0x7f976c1598d0 legacy=0x7f976c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.216+0000 7f9770d99700 1 -- 172.21.15.31:0/986155624 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612598601 0 0) 0x7f976c0c3f40 con 0x7f976c1598d0 2023-10-06T05:18:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.216+0000 7f977159a700 1 -- 172.21.15.31:0/986155624 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c0cb000 con 0x7f976c15d2d0 2023-10-06T05:18:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.216+0000 7f977159a700 1 --2- 172.21.15.31:0/986155624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c15d2d0 0x7f976c0cce00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f975c009990 tx=0x7f975c009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de4c8e2feee71e63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.216+0000 7f9770d99700 1 -- 172.21.15.31:0/986155624 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c009e80 con 0x7f976c15d2d0 2023-10-06T05:18:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.216+0000 7f9770d99700 1 -- 172.21.15.31:0/986155624 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c005c10 con 0x7f976c15d2d0 2023-10-06T05:18:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.216+0000 7f9770d99700 1 -- 172.21.15.31:0/986155624 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00b650 con 0x7f976c15d2d0 2023-10-06T05:18:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.217+0000 7f9773fff700 1 -- 172.21.15.31:0/986155624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c15d2d0 msgr2=0x7f976c0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.217+0000 7f9773fff700 1 --2- 172.21.15.31:0/986155624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c15d2d0 0x7f976c0cce00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f975c009990 tx=0x7f975c009220 comp rx=0 tx=0).stop 2023-10-06T05:18:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.217+0000 7f9773fff700 1 -- 172.21.15.31:0/986155624 shutdown_connections 2023-10-06T05:18:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.217+0000 7f9773fff700 1 --2- 172.21.15.31:0/986155624 >> v2:172.21.15.62:3300/0 conn(0x7f976c159500 0x7f976c0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.217+0000 7f9773fff700 1 --2- 172.21.15.31:0/986155624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c15d2d0 0x7f976c0cce00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.217+0000 7f9773fff700 1 -- 172.21.15.31:0/986155624 >> 172.21.15.31:0/986155624 conn(0x7f976c06ec90 msgr2=0x7f976c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:25.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.217+0000 7f9773fff700 1 -- 172.21.15.31:0/986155624 shutdown_connections 2023-10-06T05:18:25.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.217+0000 7f9773fff700 1 -- 172.21.15.31:0/986155624 wait complete. 2023-10-06T05:18:25.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.218+0000 7f9773fff700 1 Processor -- start 2023-10-06T05:18:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.218+0000 7f9773fff700 1 -- start start 2023-10-06T05:18:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.219+0000 7f9773fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c159500 0x7f976c157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.219+0000 7f9773fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f976c1598d0 0x7f976c0c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.219+0000 7f9773fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f976c15d2d0 0x7f976c0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.219+0000 7f9773fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c158030 con 0x7f976c15d2d0 2023-10-06T05:18:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.219+0000 7f9773fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c0c9200 con 0x7f976c1598d0 2023-10-06T05:18:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.219+0000 7f9773fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c0c7fa0 con 0x7f976c159500 2023-10-06T05:18:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f977259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f976c15d2d0 0x7f976c0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f977259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f976c15d2d0 0x7f976c0c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59086/0 (socket says 172.21.15.31:59086) 2023-10-06T05:18:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f977259c700 1 -- 172.21.15.31:0/3592491493 learned_addr learned my addr 172.21.15.31:0/3592491493 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f977159a700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f976c1598d0 0x7f976c0c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f9771d9b700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c159500 0x7f976c157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f977259c700 1 -- 172.21.15.31:0/3592491493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c159500 msgr2=0x7f976c157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f977259c700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c159500 0x7f976c157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f977259c700 1 -- 172.21.15.31:0/3592491493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f976c1598d0 msgr2=0x7f976c0c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f977259c700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f976c1598d0 0x7f976c0c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f977259c700 1 -- 172.21.15.31:0/3592491493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c009480 con 0x7f976c15d2d0 2023-10-06T05:18:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.220+0000 7f977259c700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f976c15d2d0 0x7f976c0c8b30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f976400cd50 tx=0x7f9764001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.221+0000 7f9761ffb700 1 -- 172.21.15.31:0/3592491493 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764005df0 con 0x7f976c15d2d0 2023-10-06T05:18:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.221+0000 7f9761ffb700 1 -- 172.21.15.31:0/3592491493 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976400a990 con 0x7f976c15d2d0 2023-10-06T05:18:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.221+0000 7f9761ffb700 1 -- 172.21.15.31:0/3592491493 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976400ad50 con 0x7f976c15d2d0 2023-10-06T05:18:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.221+0000 7f9773fff700 1 -- 172.21.15.31:0/3592491493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976c0b4df0 con 0x7f976c15d2d0 2023-10-06T05:18:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.221+0000 7f9773fff700 1 -- 172.21.15.31:0/3592491493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976c0c7950 con 0x7f976c15d2d0 2023-10-06T05:18:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.222+0000 7f9773fff700 1 -- 172.21.15.31:0/3592491493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976c04c380 con 0x7f976c15d2d0 2023-10-06T05:18:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.223+0000 7f9761ffb700 1 -- 172.21.15.31:0/3592491493 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9764007300 con 0x7f976c15d2d0 2023-10-06T05:18:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.224+0000 7f9761ffb700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f97500670e0 0x7f9750069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.224+0000 7f9761ffb700 1 -- 172.21.15.31:0/3592491493 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9764084c20 con 0x7f976c15d2d0 2023-10-06T05:18:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.224+0000 7f9771d9b700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f97500670e0 0x7f9750069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.224+0000 7f9771d9b700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f97500670e0 0x7f9750069590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9768009d40 tx=0x7f9768008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.228+0000 7f9761ffb700 1 -- 172.21.15.31:0/3592491493 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9764051a90 con 0x7f976c15d2d0 2023-10-06T05:18:25.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.473+0000 7f9773fff700 1 -- 172.21.15.31:0/3592491493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f976c0c80e0 con 0x7f976c15d2d0 2023-10-06T05:18:25.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.474+0000 7f9761ffb700 1 -- 172.21.15.31:0/3592491493 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f97640550b0 con 0x7f976c15d2d0 2023-10-06T05:18:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:25.476 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/746584225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":746584225},{"type":"v1","addr":"172.21.15.62:6834","nonce":746584225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/584374139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":584374139},{"type":"v1","addr":"172.21.15.62:6835","nonce":584374139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4926,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3794168057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3794168057},{"type":"v1","addr":"172.21.15.31:6836","nonce":3794168057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:00.090015+0000","modified":"2023-10-06T05:18:03.257587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/3886723511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3886723511},{"type":"v1","addr":"172.21.15.31:6837","nonce":3886723511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T05:18:25.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.476+0000 7f9773fff700 1 -- 172.21.15.31:0/3592491493 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f97500670e0 msgr2=0x7f9750069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.476+0000 7f9773fff700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f97500670e0 0x7f9750069590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9768009d40 tx=0x7f9768008040 comp rx=0 tx=0).stop 2023-10-06T05:18:25.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.476+0000 7f9773fff700 1 -- 172.21.15.31:0/3592491493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f976c15d2d0 msgr2=0x7f976c0c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.476+0000 7f9773fff700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f976c15d2d0 0x7f976c0c8b30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f976400cd50 tx=0x7f9764001b10 comp rx=0 tx=0).stop 2023-10-06T05:18:25.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.476+0000 7f9773fff700 1 -- 172.21.15.31:0/3592491493 shutdown_connections 2023-10-06T05:18:25.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.476+0000 7f9773fff700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f97500670e0 0x7f9750069590 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.476+0000 7f9773fff700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f976c15d2d0 0x7f976c0c8b30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.476+0000 7f9773fff700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f976c159500 0x7f976c157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.476+0000 7f9773fff700 1 --2- 172.21.15.31:0/3592491493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f976c1598d0 0x7f976c0c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.476+0000 7f9773fff700 1 -- 172.21.15.31:0/3592491493 >> 172.21.15.31:0/3592491493 conn(0x7f976c06ec90 msgr2=0x7f976c0c5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.477+0000 7f9773fff700 1 -- 172.21.15.31:0/3592491493 shutdown_connections 2023-10-06T05:18:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.477+0000 7f9773fff700 1 -- 172.21.15.31:0/3592491493 wait complete. 2023-10-06T05:18:25.481 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2023-10-06T05:18:25.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:25.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.690+0000 7f2a4b5ff700 1 Processor -- start 2023-10-06T05:18:25.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.690+0000 7f2a4b5ff700 1 -- start start 2023-10-06T05:18:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.692+0000 7f2a4b5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2a440ca770 0x7f2a440c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.692+0000 7f2a4b5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2a440cab40 0x7f2a441569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.692+0000 7f2a4b5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a44156ef0 con 0x7f2a4406ae00 2023-10-06T05:18:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.692+0000 7f2a4b5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a440c7a70 con 0x7f2a440ca770 2023-10-06T05:18:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.692+0000 7f2a4b5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a440c6df0 con 0x7f2a440cab40 2023-10-06T05:18:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.693+0000 7f2a4939b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2a440ca770 0x7f2a440c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.693+0000 7f2a4939b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2a440ca770 0x7f2a440c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59096/0 (socket says 172.21.15.31:59096) 2023-10-06T05:18:25.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.693+0000 7f2a4939b700 1 -- 172.21.15.31:0/1093607689 learned_addr learned my addr 172.21.15.31:0/1093607689 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:25.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.693+0000 7f2a48b9a700 1 --2- 172.21.15.31:0/1093607689 >> v2:172.21.15.62:3300/0 conn(0x7f2a440cab40 0x7f2a441569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.693+0000 7f2a4939b700 1 -- 172.21.15.31:0/1093607689 >> v2:172.21.15.62:3300/0 conn(0x7f2a440cab40 msgr2=0x7f2a441569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:25.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.693+0000 7f2a4939b700 1 --2- 172.21.15.31:0/1093607689 >> v2:172.21.15.62:3300/0 conn(0x7f2a440cab40 0x7f2a441569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.693+0000 7f2a4939b700 1 -- 172.21.15.31:0/1093607689 >> v1:172.21.15.62:6789/0 conn(0x7f2a4406ae00 legacy=0x7f2a4414c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.693+0000 7f2a4939b700 1 -- 172.21.15.31:0/1093607689 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440c70d0 con 0x7f2a440ca770 2023-10-06T05:18:25.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.693+0000 7f2a4939b700 1 --2- 172.21.15.31:0/1093607689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440ca770 0x7f2a440c66f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2a440ca030 tx=0x7f2a4000b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb7f8b571732413 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a3bfff700 1 -- 172.21.15.31:0/1093607689 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4000da60 con 0x7f2a440ca770 2023-10-06T05:18:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a3bfff700 1 -- 172.21.15.31:0/1093607689 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4000dbe0 con 0x7f2a440ca770 2023-10-06T05:18:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a3bfff700 1 -- 172.21.15.31:0/1093607689 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40005a00 con 0x7f2a440ca770 2023-10-06T05:18:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/1093607689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440ca770 msgr2=0x7f2a440c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a4b5ff700 1 --2- 172.21.15.31:0/1093607689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440ca770 0x7f2a440c66f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2a440ca030 tx=0x7f2a4000b4b0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/1093607689 shutdown_connections 2023-10-06T05:18:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a4b5ff700 1 --2- 172.21.15.31:0/1093607689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440ca770 0x7f2a440c66f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a4b5ff700 1 --2- 172.21.15.31:0/1093607689 >> v2:172.21.15.62:3300/0 conn(0x7f2a440cab40 0x7f2a441569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/1093607689 >> 172.21.15.31:0/1093607689 conn(0x7f2a440cc0b0 msgr2=0x7f2a440cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/1093607689 shutdown_connections 2023-10-06T05:18:25.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.694+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/1093607689 wait complete. 2023-10-06T05:18:25.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.697+0000 7f2a4b5ff700 1 Processor -- start 2023-10-06T05:18:25.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.697+0000 7f2a4b5ff700 1 -- start start 2023-10-06T05:18:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a4b5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a4406ae00 0x7f2a44149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a4b5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a440ca770 0x7f2a440ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a4b5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440cab40 0x7f2a440b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a4b5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440aee50 con 0x7f2a440cab40 2023-10-06T05:18:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a4b5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440af0a0 con 0x7f2a440ca770 2023-10-06T05:18:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a4b5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440af380 con 0x7f2a4406ae00 2023-10-06T05:18:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a49b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440cab40 0x7f2a440b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a49b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440cab40 0x7f2a440b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59110/0 (socket says 172.21.15.31:59110) 2023-10-06T05:18:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a49b9c700 1 -- 172.21.15.31:0/772216814 learned_addr learned my addr 172.21.15.31:0/772216814 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:25.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a4939b700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a4406ae00 0x7f2a44149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.698+0000 7f2a48b9a700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a440ca770 0x7f2a440ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a49b9c700 1 -- 172.21.15.31:0/772216814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a4406ae00 msgr2=0x7f2a44149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a49b9c700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a4406ae00 0x7f2a44149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a49b9c700 1 -- 172.21.15.31:0/772216814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a440ca770 msgr2=0x7f2a440ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a49b9c700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a440ca770 0x7f2a440ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a49b9c700 1 -- 172.21.15.31:0/772216814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4000b730 con 0x7f2a440cab40 2023-10-06T05:18:25.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a49b9c700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440cab40 0x7f2a440b0140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00c840 tx=0x7f2a3c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:25.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a397fa700 1 -- 172.21.15.31:0/772216814 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c012630 con 0x7f2a440cab40 2023-10-06T05:18:25.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a397fa700 1 -- 172.21.15.31:0/772216814 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c0127b0 con 0x7f2a440cab40 2023-10-06T05:18:25.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a397fa700 1 -- 172.21.15.31:0/772216814 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c012b70 con 0x7f2a440cab40 2023-10-06T05:18:25.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/772216814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a440af660 con 0x7f2a440cab40 2023-10-06T05:18:25.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.699+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/772216814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a440afc20 con 0x7f2a440cab40 2023-10-06T05:18:25.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.700+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/772216814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a440afe40 con 0x7f2a440cab40 2023-10-06T05:18:25.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.701+0000 7f2a397fa700 1 -- 172.21.15.31:0/772216814 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2a3c01a3a0 con 0x7f2a440cab40 2023-10-06T05:18:25.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.702+0000 7f2a397fa700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a280670e0 0x7f2a28069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:25.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.702+0000 7f2a397fa700 1 -- 172.21.15.31:0/772216814 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2a3c084a80 con 0x7f2a440cab40 2023-10-06T05:18:25.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.702+0000 7f2a4939b700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a280670e0 0x7f2a28069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:25.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.703+0000 7f2a4939b700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a280670e0 0x7f2a28069590 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a440ca030 tx=0x7f2a40018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:25.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.706+0000 7f2a397fa700 1 -- 172.21.15.31:0/772216814 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a3c051910 con 0x7f2a440cab40 2023-10-06T05:18:25.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.909+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/772216814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a4404c380 con 0x7f2a440cab40 2023-10-06T05:18:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.910+0000 7f2a397fa700 1 -- 172.21.15.31:0/772216814 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f2a3c054f30 con 0x7f2a440cab40 2023-10-06T05:18:25.912 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:25.912 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:01.245088+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:17:58.074919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T05:17:58.854233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.912+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/772216814 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a280670e0 msgr2=0x7f2a28069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.912+0000 7f2a4b5ff700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a280670e0 0x7f2a28069590 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a440ca030 tx=0x7f2a40018000 comp rx=0 tx=0).stop 2023-10-06T05:18:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.912+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/772216814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440cab40 msgr2=0x7f2a440b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.912+0000 7f2a4b5ff700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440cab40 0x7f2a440b0140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00c840 tx=0x7f2a3c001670 comp rx=0 tx=0).stop 2023-10-06T05:18:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.913+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/772216814 shutdown_connections 2023-10-06T05:18:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.913+0000 7f2a4b5ff700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a280670e0 0x7f2a28069590 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.913+0000 7f2a4b5ff700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a440cab40 0x7f2a440b0140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.913+0000 7f2a4b5ff700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a4406ae00 0x7f2a44149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.913+0000 7f2a4b5ff700 1 --2- 172.21.15.31:0/772216814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a440ca770 0x7f2a440ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:25.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.913+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/772216814 >> 172.21.15.31:0/772216814 conn(0x7f2a440cc0b0 msgr2=0x7f2a4406dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:25.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.913+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/772216814 shutdown_connections 2023-10-06T05:18:25.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:25.913+0000 7f2a4b5ff700 1 -- 172.21.15.31:0/772216814 wait complete. 2023-10-06T05:18:25.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:18:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.129+0000 7fca2fdff700 1 Processor -- start 2023-10-06T05:18:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.129+0000 7fca2fdff700 1 -- start start 2023-10-06T05:18:26.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.130+0000 7fca2fdff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca280cd510 0x7fca280b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.131+0000 7fca2fdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca2815d2d0 0x7fca280aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.131+0000 7fca2fdff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca280c3f40 con 0x7fca28159500 2023-10-06T05:18:26.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.131+0000 7fca2fdff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca280b2860 con 0x7fca280cd510 2023-10-06T05:18:26.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.131+0000 7fca2fdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2815d7d0 con 0x7fca2815d2d0 2023-10-06T05:18:26.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.131+0000 7fca2db9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fca28159500 0x7fca280cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56294/0 (socket says 172.21.15.31:56294) 2023-10-06T05:18:26.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.131+0000 7fca2db9b700 1 -- 172.21.15.31:0/2769344693 learned_addr learned my addr 172.21.15.31:0/2769344693 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:26.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.131+0000 7fca2e39c700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca2815d2d0 0x7fca280aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.131+0000 7fca2d39a700 1 --2- 172.21.15.31:0/2769344693 >> v2:172.21.15.62:3300/0 conn(0x7fca280cd510 0x7fca280b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.131+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468491303 0 0) 0x7fca280c3f40 con 0x7fca28159500 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.131+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca14003570 con 0x7fca28159500 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1347194188 0 0) 0x7fca14003570 con 0x7fca28159500 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca2815d2d0 msgr2=0x7fca280aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca2815d2d0 0x7fca280aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 >> v2:172.21.15.62:3300/0 conn(0x7fca280cd510 msgr2=0x7fca280b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 --2- 172.21.15.31:0/2769344693 >> v2:172.21.15.62:3300/0 conn(0x7fca280cd510 0x7fca280b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca280cb000 con 0x7fca28159500 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fca24001470 con 0x7fca28159500 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 >> v1:172.21.15.31:6789/0 conn(0x7fca28159500 legacy=0x7fca280cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca14003ae0 0x7fca14003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 0x7fca14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca2815d2d0 0x7fca140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280b2860 con 0x7fca14004810 2023-10-06T05:18:26.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2815d7d0 con 0x7fca14003ae0 2023-10-06T05:18:26.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1400bb10 con 0x7fca2815d2d0 2023-10-06T05:18:26.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2d39a700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 0x7fca14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2db9b700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca14003ae0 0x7fca14003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.132+0000 7fca2e39c700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca2815d2d0 0x7fca140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.133+0000 7fca2d39a700 1 -- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca2815d2d0 msgr2=0x7fca140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:26.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.133+0000 7fca2d39a700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca2815d2d0 0x7fca140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.133+0000 7fca2d39a700 1 -- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca14003ae0 msgr2=0x7fca14003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.133+0000 7fca2d39a700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca14003ae0 0x7fca14003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.133+0000 7fca2d39a700 1 -- 172.21.15.31:0/2769344693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca14003570 con 0x7fca14004810 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.133+0000 7fca2e39c700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca2815d2d0 0x7fca140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.133+0000 7fca2d39a700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 0x7fca14004c80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fca280cd910 tx=0x7fca18009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca97b9f6b408450d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.133+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18005a00 con 0x7fca14004810 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.133+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca18005b60 con 0x7fca14004810 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.133+0000 7fca2cb99700 1 -- 172.21.15.31:0/2769344693 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1800e640 con 0x7fca14004810 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.134+0000 7fca2fdff700 1 -- 172.21.15.31:0/2769344693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 msgr2=0x7fca14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.134+0000 7fca2fdff700 1 --2- 172.21.15.31:0/2769344693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 0x7fca14004c80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fca280cd910 tx=0x7fca18009120 comp rx=0 tx=0).stop 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.134+0000 7fca2e39c700 1 -- 172.21.15.31:0/2769344693 reap_dead start 2023-10-06T05:18:26.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.134+0000 7fca2fdff700 1 -- 172.21.15.31:0/2769344693 shutdown_connections 2023-10-06T05:18:26.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.134+0000 7fca2fdff700 1 -- 172.21.15.31:0/2769344693 >> 172.21.15.31:0/2769344693 conn(0x7fca2806ec90 msgr2=0x7fca2806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:26.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.134+0000 7fca2fdff700 1 -- 172.21.15.31:0/2769344693 shutdown_connections 2023-10-06T05:18:26.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.134+0000 7fca2fdff700 1 -- 172.21.15.31:0/2769344693 wait complete. 2023-10-06T05:18:26.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.135+0000 7fca2fdff700 1 Processor -- start 2023-10-06T05:18:26.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.136+0000 7fca2fdff700 1 -- start start 2023-10-06T05:18:26.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.136+0000 7fca2fdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 0x7fca280c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2fdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7e80 0x7fca281509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2fdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280b4260 0x7fca280b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2fdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c9fa0 con 0x7fca14004810 2023-10-06T05:18:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2fdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280b4c10 con 0x7fca280c7e80 2023-10-06T05:18:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2fdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c9b10 con 0x7fca280b4260 2023-10-06T05:18:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2db9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 0x7fca280c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2db9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 0x7fca280c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59120/0 (socket says 172.21.15.31:59120) 2023-10-06T05:18:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2db9b700 1 -- 172.21.15.31:0/3772171878 learned_addr learned my addr 172.21.15.31:0/3772171878 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2d39a700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7e80 0x7fca281509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2e39c700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280b4260 0x7fca280b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2db9b700 1 -- 172.21.15.31:0/3772171878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280b4260 msgr2=0x7fca280b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2db9b700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280b4260 0x7fca280b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2db9b700 1 -- 172.21.15.31:0/3772171878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7e80 msgr2=0x7fca281509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2db9b700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7e80 0x7fca281509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2db9b700 1 -- 172.21.15.31:0/3772171878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca280c8f00 con 0x7fca14004810 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2d39a700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7e80 0x7fca281509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2e39c700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280b4260 0x7fca280b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.137+0000 7fca2db9b700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 0x7fca280c8920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fca240073f0 tx=0x7fca24001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.138+0000 7fca1dffb700 1 -- 172.21.15.31:0/3772171878 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24018030 con 0x7fca14004810 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.138+0000 7fca1dffb700 1 -- 172.21.15.31:0/3772171878 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca240014e0 con 0x7fca14004810 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.138+0000 7fca1dffb700 1 -- 172.21.15.31:0/3772171878 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400fb00 con 0x7fca14004810 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.138+0000 7fca2fdff700 1 -- 172.21.15.31:0/3772171878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca280c91e0 con 0x7fca14004810 2023-10-06T05:18:26.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.138+0000 7fca2fdff700 1 -- 172.21.15.31:0/3772171878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca280c97a0 con 0x7fca14004810 2023-10-06T05:18:26.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.139+0000 7fca2fdff700 1 -- 172.21.15.31:0/3772171878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2804c380 con 0x7fca14004810 2023-10-06T05:18:26.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.140+0000 7fca1dffb700 1 -- 172.21.15.31:0/3772171878 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fca2401b070 con 0x7fca14004810 2023-10-06T05:18:26.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.140+0000 7fca1dffb700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 0x7fca0c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.141+0000 7fca1dffb700 1 -- 172.21.15.31:0/3772171878 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca24019070 con 0x7fca14004810 2023-10-06T05:18:26.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.141+0000 7fca2d39a700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 0x7fca0c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.142+0000 7fca2d39a700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 0x7fca0c069590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fca18009160 tx=0x7fca18019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:26.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.145+0000 7fca1dffb700 1 -- 172.21.15.31:0/3772171878 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca240516b0 con 0x7fca14004810 2023-10-06T05:18:26.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.392+0000 7fca2fdff700 1 -- 172.21.15.31:0/3772171878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fca280cade0 con 0x7fca14004810 2023-10-06T05:18:26.395 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:18:26.393+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:18:26.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.401+0000 7fca1dffb700 1 -- 172.21.15.31:0/3772171878 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fca24054cd0 con 0x7fca14004810 2023-10-06T05:18:26.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.405+0000 7fca2fdff700 1 -- 172.21.15.31:0/3772171878 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 msgr2=0x7fca0c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.405+0000 7fca2fdff700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 0x7fca0c069590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fca18009160 tx=0x7fca18019000 comp rx=0 tx=0).stop 2023-10-06T05:18:26.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.405+0000 7fca2fdff700 1 -- 172.21.15.31:0/3772171878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 msgr2=0x7fca280c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.405+0000 7fca2fdff700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 0x7fca280c8920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fca240073f0 tx=0x7fca24001dc0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.405+0000 7fca2fdff700 1 -- 172.21.15.31:0/3772171878 shutdown_connections 2023-10-06T05:18:26.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.405+0000 7fca2fdff700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280b4260 0x7fca280b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.405+0000 7fca2fdff700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 0x7fca0c069590 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.405+0000 7fca2fdff700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca14004810 0x7fca280c8920 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.405+0000 7fca2fdff700 1 --2- 172.21.15.31:0/3772171878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7e80 0x7fca281509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.405+0000 7fca2fdff700 1 -- 172.21.15.31:0/3772171878 >> 172.21.15.31:0/3772171878 conn(0x7fca2806ec90 msgr2=0x7fca280cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:26.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.406+0000 7fca2fdff700 1 -- 172.21.15.31:0/3772171878 shutdown_connections 2023-10-06T05:18:26.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.406+0000 7fca2fdff700 1 -- 172.21.15.31:0/3772171878 wait complete. 2023-10-06T05:18:26.409 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:18:26.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:18:26.476 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:18:26.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.630+0000 7f4e989ff700 1 Processor -- start 2023-10-06T05:18:26.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.630+0000 7f4e989ff700 1 -- start start 2023-10-06T05:18:26.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.631+0000 7f4e989ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4e94159930 0x7f4e940b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.631+0000 7f4e989ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9415d2d0 con 0x7f4e94159560 2023-10-06T05:18:26.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.631+0000 7f4e989ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9415d5b0 con 0x7f4e940cd580 2023-10-06T05:18:26.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.631+0000 7f4e989ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9415d890 con 0x7f4e94159930 2023-10-06T05:18:26.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.632+0000 7f4e9259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4e94159560 0x7f4e940cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56304/0 (socket says 172.21.15.31:56304) 2023-10-06T05:18:26.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.632+0000 7f4e9259c700 1 -- 172.21.15.31:0/2289898137 learned_addr learned my addr 172.21.15.31:0/2289898137 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:26.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.632+0000 7f4e91d9b700 1 --2- 172.21.15.31:0/2289898137 >> v2:172.21.15.62:3300/0 conn(0x7f4e94159930 0x7f4e940b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.632+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798546124 0 0) 0x7f4e9415d2d0 con 0x7f4e94159560 2023-10-06T05:18:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.632+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e7c003570 con 0x7f4e94159560 2023-10-06T05:18:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.632+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609292796 0 0) 0x7f4e9415d5b0 con 0x7f4e940cd580 2023-10-06T05:18:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e9415d2d0 con 0x7f4e940cd580 2023-10-06T05:18:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1146472674 0 0) 0x7f4e7c003570 con 0x7f4e94159560 2023-10-06T05:18:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 >> v2:172.21.15.62:3300/0 conn(0x7f4e94159930 msgr2=0x7f4e940b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 --2- 172.21.15.31:0/2289898137 >> v2:172.21.15.62:3300/0 conn(0x7f4e94159930 0x7f4e940b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 >> v1:172.21.15.62:6789/0 conn(0x7f4e940cd580 legacy=0x7f4e940cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9415db70 con 0x7f4e94159560 2023-10-06T05:18:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f4e84001470 con 0x7f4e94159560 2023-10-06T05:18:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e91d9b700 1 --2- 172.21.15.31:0/2289898137 >> v2:172.21.15.62:3300/0 conn(0x7f4e94159930 0x7f4e940b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 >> v1:172.21.15.31:6789/0 conn(0x7f4e94159560 legacy=0x7f4e940cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7c003c40 0x7f4e7c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e7c004810 0x7f4e7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e7c005210 0x7f4e7c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9415d890 con 0x7f4e7c004810 2023-10-06T05:18:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c00c1b0 con 0x7f4e7c003c40 2023-10-06T05:18:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c00c370 con 0x7f4e7c005210 2023-10-06T05:18:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e91d9b700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e7c004810 0x7f4e7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e9259c700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7c003c40 0x7f4e7c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e92d9d700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e7c005210 0x7f4e7c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e91d9b700 1 -- 172.21.15.31:0/2289898137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e7c005210 msgr2=0x7f4e7c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e91d9b700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e7c005210 0x7f4e7c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e91d9b700 1 -- 172.21.15.31:0/2289898137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7c003c40 msgr2=0x7f4e7c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e91d9b700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7c003c40 0x7f4e7c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e91d9b700 1 -- 172.21.15.31:0/2289898137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c003570 con 0x7f4e7c004810 2023-10-06T05:18:26.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.633+0000 7f4e92d9d700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e7c005210 0x7f4e7c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.634+0000 7f4e92d9d700 1 -- 172.21.15.31:0/2289898137 reap_dead start 2023-10-06T05:18:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.634+0000 7f4e91d9b700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e7c004810 0x7f4e7c004c80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4e88006f00 tx=0x7f4e880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1b7fec466b6d96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.634+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88009740 con 0x7f4e7c004810 2023-10-06T05:18:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.634+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8800e920 con 0x7f4e7c004810 2023-10-06T05:18:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.634+0000 7f4e9159a700 1 -- 172.21.15.31:0/2289898137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8800ece0 con 0x7f4e7c004810 2023-10-06T05:18:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.634+0000 7f4e989ff700 1 -- 172.21.15.31:0/2289898137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e7c004810 msgr2=0x7f4e7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.634+0000 7f4e989ff700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e7c004810 0x7f4e7c004c80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4e88006f00 tx=0x7f4e880091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.635+0000 7f4e989ff700 1 -- 172.21.15.31:0/2289898137 shutdown_connections 2023-10-06T05:18:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.635+0000 7f4e989ff700 1 --2- 172.21.15.31:0/2289898137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e7c004810 0x7f4e7c004c80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.635+0000 7f4e989ff700 1 -- 172.21.15.31:0/2289898137 >> 172.21.15.31:0/2289898137 conn(0x7f4e9406ec90 msgr2=0x7f4e9406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.635+0000 7f4e989ff700 1 -- 172.21.15.31:0/2289898137 shutdown_connections 2023-10-06T05:18:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.635+0000 7f4e989ff700 1 -- 172.21.15.31:0/2289898137 wait complete. 2023-10-06T05:18:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.636+0000 7f4e989ff700 1 Processor -- start 2023-10-06T05:18:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.636+0000 7f4e989ff700 1 -- start start 2023-10-06T05:18:26.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.637+0000 7f4e989ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7c004810 0x7f4e9415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.637+0000 7f4e989ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9415ae50 0x7f4e940cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.637+0000 7f4e989ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e940b01a0 0x7f4e940cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.637+0000 7f4e989ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940c3ef0 con 0x7f4e9415ae50 2023-10-06T05:18:26.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.637+0000 7f4e989ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9415c580 con 0x7f4e7c004810 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.637+0000 7f4e989ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940c4470 con 0x7f4e940b01a0 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.637+0000 7f4e91d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9415ae50 0x7f4e940cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.637+0000 7f4e91d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9415ae50 0x7f4e940cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59154/0 (socket says 172.21.15.31:59154) 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.637+0000 7f4e91d9b700 1 -- 172.21.15.31:0/98007117 learned_addr learned my addr 172.21.15.31:0/98007117 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e9259c700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7c004810 0x7f4e9415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e92d9d700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e940b01a0 0x7f4e940cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e91d9b700 1 -- 172.21.15.31:0/98007117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e940b01a0 msgr2=0x7f4e940cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e91d9b700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e940b01a0 0x7f4e940cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e91d9b700 1 -- 172.21.15.31:0/98007117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7c004810 msgr2=0x7f4e9415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e91d9b700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7c004810 0x7f4e9415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e91d9b700 1 -- 172.21.15.31:0/98007117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9415c7b0 con 0x7f4e9415ae50 2023-10-06T05:18:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e91d9b700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9415ae50 0x7f4e940cf270 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c180 tx=0x7f4e8801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e827fc700 1 -- 172.21.15.31:0/98007117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88022070 con 0x7f4e9415ae50 2023-10-06T05:18:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e827fc700 1 -- 172.21.15.31:0/98007117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e88013be0 con 0x7f4e9415ae50 2023-10-06T05:18:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e989ff700 1 -- 172.21.15.31:0/98007117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9415ca90 con 0x7f4e9415ae50 2023-10-06T05:18:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.638+0000 7f4e827fc700 1 -- 172.21.15.31:0/98007117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8801c450 con 0x7f4e9415ae50 2023-10-06T05:18:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.639+0000 7f4e989ff700 1 -- 172.21.15.31:0/98007117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e94159eb0 con 0x7f4e9415ae50 2023-10-06T05:18:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.640+0000 7f4e989ff700 1 -- 172.21.15.31:0/98007117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e940c42c0 con 0x7f4e9415ae50 2023-10-06T05:18:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.640+0000 7f4e827fc700 1 -- 172.21.15.31:0/98007117 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f4e88026070 con 0x7f4e9415ae50 2023-10-06T05:18:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.641+0000 7f4e827fc700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4e740670e0 0x7f4e74069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.642+0000 7f4e827fc700 1 -- 172.21.15.31:0/98007117 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f4e880862f0 con 0x7f4e9415ae50 2023-10-06T05:18:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.642+0000 7f4e9259c700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4e740670e0 0x7f4e74069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.642+0000 7f4e9259c700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4e740670e0 0x7f4e74069590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4e84000e50 tx=0x7f4e84001d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.645+0000 7f4e827fc700 1 -- 172.21.15.31:0/98007117 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e880530f0 con 0x7f4e9415ae50 2023-10-06T05:18:26.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:26.890+0000 7f4e989ff700 1 -- 172.21.15.31:0/98007117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4e940c45e0 con 0x7f4e9415ae50 2023-10-06T05:18:27.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.400+0000 7f4e827fc700 1 -- 172.21.15.31:0/98007117 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4e880566d0 con 0x7f4e9415ae50 2023-10-06T05:18:27.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.402+0000 7f4e989ff700 1 -- 172.21.15.31:0/98007117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4e740670e0 msgr2=0x7f4e74069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.402+0000 7f4e989ff700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4e740670e0 0x7f4e74069590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4e84000e50 tx=0x7f4e84001d40 comp rx=0 tx=0).stop 2023-10-06T05:18:27.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.402+0000 7f4e989ff700 1 -- 172.21.15.31:0/98007117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9415ae50 msgr2=0x7f4e940cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.402+0000 7f4e989ff700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9415ae50 0x7f4e940cf270 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c180 tx=0x7f4e8801a000 comp rx=0 tx=0).stop 2023-10-06T05:18:27.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.403+0000 7f4e989ff700 1 -- 172.21.15.31:0/98007117 shutdown_connections 2023-10-06T05:18:27.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.403+0000 7f4e989ff700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e940b01a0 0x7f4e940cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.403+0000 7f4e989ff700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e7c004810 0x7f4e9415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.403+0000 7f4e989ff700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4e740670e0 0x7f4e74069590 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.403+0000 7f4e989ff700 1 --2- 172.21.15.31:0/98007117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9415ae50 0x7f4e940cf270 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.403+0000 7f4e989ff700 1 -- 172.21.15.31:0/98007117 >> 172.21.15.31:0/98007117 conn(0x7f4e9406ec90 msgr2=0x7f4e94157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.403+0000 7f4e989ff700 1 -- 172.21.15.31:0/98007117 shutdown_connections 2023-10-06T05:18:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.403+0000 7f4e989ff700 1 -- 172.21.15.31:0/98007117 wait complete. 2023-10-06T05:18:27.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:18:27.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.663+0000 7f1179fff700 1 Processor -- start 2023-10-06T05:18:27.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.664+0000 7f1179fff700 1 -- start start 2023-10-06T05:18:27.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.665+0000 7f1179fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1174154cd0 0x7f11741583e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.665+0000 7f1179fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11741550a0 0x7f11740ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.665+0000 7f1179fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117406f640 con 0x7f117406ee10 2023-10-06T05:18:27.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.665+0000 7f1179fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117406e610 con 0x7f11741550a0 2023-10-06T05:18:27.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.665+0000 7f1179fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11740adf60 con 0x7f1174154cd0 2023-10-06T05:18:27.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.665+0000 7f1172ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11741550a0 0x7f11740ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.665+0000 7f1172ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11741550a0 0x7f11740ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43328/0 (socket says 172.21.15.31:43328) 2023-10-06T05:18:27.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.665+0000 7f1172ffd700 1 -- 172.21.15.31:0/3472372829 learned_addr learned my addr 172.21.15.31:0/3472372829 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:27.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.665+0000 7f11737fe700 1 --2- 172.21.15.31:0/3472372829 >> v2:172.21.15.62:3300/0 conn(0x7f1174154cd0 0x7f11741583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.666+0000 7f1172ffd700 1 -- 172.21.15.31:0/3472372829 >> v2:172.21.15.62:3300/0 conn(0x7f1174154cd0 msgr2=0x7f11741583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:27.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.666+0000 7f1172ffd700 1 --2- 172.21.15.31:0/3472372829 >> v2:172.21.15.62:3300/0 conn(0x7f1174154cd0 0x7f11741583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.666+0000 7f1172ffd700 1 -- 172.21.15.31:0/3472372829 >> v1:172.21.15.62:6789/0 conn(0x7f117406ee10 legacy=0x7f117406d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.666+0000 7f1172ffd700 1 -- 172.21.15.31:0/3472372829 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1174158920 con 0x7f11741550a0 2023-10-06T05:18:27.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.666+0000 7f1172ffd700 1 --2- 172.21.15.31:0/3472372829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11741550a0 0x7f11740ae3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f11740aebb0 tx=0x7f1168009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794eb329d6700cd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:27.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.666+0000 7f11727fc700 1 -- 172.21.15.31:0/3472372829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800b860 con 0x7f11741550a0 2023-10-06T05:18:27.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.666+0000 7f11727fc700 1 -- 172.21.15.31:0/3472372829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116800b9c0 con 0x7f11741550a0 2023-10-06T05:18:27.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.666+0000 7f11727fc700 1 -- 172.21.15.31:0/3472372829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800bd60 con 0x7f11741550a0 2023-10-06T05:18:27.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.667+0000 7f1179fff700 1 -- 172.21.15.31:0/3472372829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11741550a0 msgr2=0x7f11740ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.667+0000 7f1179fff700 1 --2- 172.21.15.31:0/3472372829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11741550a0 0x7f11740ae3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f11740aebb0 tx=0x7f1168009250 comp rx=0 tx=0).stop 2023-10-06T05:18:27.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.667+0000 7f1179fff700 1 -- 172.21.15.31:0/3472372829 shutdown_connections 2023-10-06T05:18:27.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.667+0000 7f1179fff700 1 --2- 172.21.15.31:0/3472372829 >> v2:172.21.15.62:3300/0 conn(0x7f1174154cd0 0x7f11741583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.667+0000 7f1179fff700 1 --2- 172.21.15.31:0/3472372829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11741550a0 0x7f11740ae3f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.667+0000 7f1179fff700 1 -- 172.21.15.31:0/3472372829 >> 172.21.15.31:0/3472372829 conn(0x7f1174068030 msgr2=0x7f11740cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:27.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.667+0000 7f1179fff700 1 -- 172.21.15.31:0/3472372829 shutdown_connections 2023-10-06T05:18:27.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.667+0000 7f1179fff700 1 -- 172.21.15.31:0/3472372829 wait complete. 2023-10-06T05:18:27.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.669+0000 7f1179fff700 1 Processor -- start 2023-10-06T05:18:27.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.669+0000 7f1179fff700 1 -- start start 2023-10-06T05:18:27.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1179fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f117406ee10 0x7f11740c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1179fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174154cd0 0x7f117416cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1179fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11741550a0 0x7f117416d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1179fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11740a9d30 con 0x7f1174154cd0 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1179fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11740c8040 con 0x7f11741550a0 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1179fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11740c7a80 con 0x7f117406ee10 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1172ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174154cd0 0x7f117416cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1172ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174154cd0 0x7f117416cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43332/0 (socket says 172.21.15.31:43332) 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1172ffd700 1 -- 172.21.15.31:0/1742998152 learned_addr learned my addr 172.21.15.31:0/1742998152 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f11737fe700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f117406ee10 0x7f11740c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1173fff700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11741550a0 0x7f117416d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1172ffd700 1 -- 172.21.15.31:0/1742998152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f117406ee10 msgr2=0x7f11740c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1172ffd700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f117406ee10 0x7f11740c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1172ffd700 1 -- 172.21.15.31:0/1742998152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11741550a0 msgr2=0x7f117416d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:27.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.670+0000 7f1172ffd700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11741550a0 0x7f117416d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:27.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.671+0000 7f1172ffd700 1 -- 172.21.15.31:0/1742998152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11680094f0 con 0x7f1174154cd0 2023-10-06T05:18:27.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.671+0000 7f1172ffd700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174154cd0 0x7f117416cec0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1168006fd0 tx=0x7f1168017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:27.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.671+0000 7f115b7fe700 1 -- 172.21.15.31:0/1742998152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168024440 con 0x7f1174154cd0 2023-10-06T05:18:27.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.671+0000 7f115b7fe700 1 -- 172.21.15.31:0/1742998152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11680245a0 con 0x7f1174154cd0 2023-10-06T05:18:27.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.671+0000 7f1179fff700 1 -- 172.21.15.31:0/1742998152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11740c7d60 con 0x7f1174154cd0 2023-10-06T05:18:27.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.671+0000 7f115b7fe700 1 -- 172.21.15.31:0/1742998152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168024960 con 0x7f1174154cd0 2023-10-06T05:18:27.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.671+0000 7f1179fff700 1 -- 172.21.15.31:0/1742998152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11740c5fd0 con 0x7f1174154cd0 2023-10-06T05:18:27.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.672+0000 7f1179fff700 1 -- 172.21.15.31:0/1742998152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11740a9ee0 con 0x7f1174154cd0 2023-10-06T05:18:27.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.673+0000 7f115b7fe700 1 -- 172.21.15.31:0/1742998152 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f116801a420 con 0x7f1174154cd0 2023-10-06T05:18:27.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.674+0000 7f115b7fe700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f11600670e0 0x7f1160069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:27.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.674+0000 7f115b7fe700 1 -- 172.21.15.31:0/1742998152 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1168086840 con 0x7f1174154cd0 2023-10-06T05:18:27.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.674+0000 7f11737fe700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f11600670e0 0x7f1160069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:27.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.675+0000 7f11737fe700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f11600670e0 0x7f1160069590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1164006aa0 tx=0x7f1164008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:27.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.678+0000 7f115b7fe700 1 -- 172.21.15.31:0/1742998152 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11740a9ee0 con 0x7f1174154cd0 2023-10-06T05:18:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:27.882+0000 7f1179fff700 1 -- 172.21.15.31:0/1742998152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11740a9ee0 con 0x7f1174154cd0 2023-10-06T05:18:28.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.406+0000 7f115b7fe700 1 -- 172.21.15.31:0/1742998152 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) v1 ==== 181+0+0 (secure 0 0 0) 0x7f11740a9ee0 con 0x7f1174154cd0 2023-10-06T05:18:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.411+0000 7f1179fff700 1 -- 172.21.15.31:0/1742998152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f11600670e0 msgr2=0x7f1160069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.411+0000 7f1179fff700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f11600670e0 0x7f1160069590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1164006aa0 tx=0x7f1164008040 comp rx=0 tx=0).stop 2023-10-06T05:18:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.411+0000 7f1179fff700 1 -- 172.21.15.31:0/1742998152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174154cd0 msgr2=0x7f117416cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.411+0000 7f1179fff700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174154cd0 0x7f117416cec0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1168006fd0 tx=0x7f1168017000 comp rx=0 tx=0).stop 2023-10-06T05:18:28.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.411+0000 7f1179fff700 1 -- 172.21.15.31:0/1742998152 shutdown_connections 2023-10-06T05:18:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.411+0000 7f1179fff700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11741550a0 0x7f117416d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.411+0000 7f1179fff700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f117406ee10 0x7f11740c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.411+0000 7f1179fff700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f11600670e0 0x7f1160069590 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.411+0000 7f1179fff700 1 --2- 172.21.15.31:0/1742998152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1174154cd0 0x7f117416cec0 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1168006fd0 tx=0x7f1168017000 comp rx=0 tx=0).stop 2023-10-06T05:18:28.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.411+0000 7f1179fff700 1 -- 172.21.15.31:0/1742998152 >> 172.21.15.31:0/1742998152 conn(0x7f1174068030 msgr2=0x7f11740cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.412+0000 7f1179fff700 1 -- 172.21.15.31:0/1742998152 shutdown_connections 2023-10-06T05:18:28.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.412+0000 7f1179fff700 1 -- 172.21.15.31:0/1742998152 wait complete. 2023-10-06T05:18:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:18:28.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:18:28.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.641+0000 7f0cf41ff700 1 Processor -- start 2023-10-06T05:18:28.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.641+0000 7f0cf41ff700 1 -- start start 2023-10-06T05:18:28.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf41ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0cec153950 0x7f0cec153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:28.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf41ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0cec0cd5d0 0x7f0cec15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:28.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf41ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 0x7f0cec15d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:28.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf41ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cec15bb20 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf41ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cec15af10 con 0x7f0cec153950 2023-10-06T05:18:28.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf41ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec15b1f0 con 0x7f0cec0cda20 2023-10-06T05:18:28.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0cec0cd5d0 0x7f0cec15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:28.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0cec0cd5d0 0x7f0cec15d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43346/0 (socket says 172.21.15.31:43346) 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf179a700 1 -- 172.21.15.31:0/2737173881 learned_addr learned my addr 172.21.15.31:0/2737173881 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf279c700 1 --2- 172.21.15.31:0/2737173881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 0x7f0cec15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.643+0000 7f0cf1f9b700 1 --2- 172.21.15.31:0/2737173881 >> v2:172.21.15.62:3300/0 conn(0x7f0cec153950 0x7f0cec153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.644+0000 7f0cf179a700 1 -- 172.21.15.31:0/2737173881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 msgr2=0x7f0cec15d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.644+0000 7f0cf179a700 1 --2- 172.21.15.31:0/2737173881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 0x7f0cec15d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.644+0000 7f0cf179a700 1 -- 172.21.15.31:0/2737173881 >> v2:172.21.15.62:3300/0 conn(0x7f0cec153950 msgr2=0x7f0cec153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.644+0000 7f0cf179a700 1 --2- 172.21.15.31:0/2737173881 >> v2:172.21.15.62:3300/0 conn(0x7f0cec153950 0x7f0cec153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.644+0000 7f0cf179a700 1 -- 172.21.15.31:0/2737173881 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec15b4d0 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.644+0000 7f0cf179a700 1 --2- 172.21.15.31:0/2737173881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 0x7f0cec15d210 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0d15b0 tx=0x7f0cdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142c3daa78586921 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.644+0000 7f0cf0f99700 1 -- 172.21.15.31:0/2737173881 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc00b640 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.644+0000 7f0cf0f99700 1 -- 172.21.15.31:0/2737173881 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc00b7a0 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.644+0000 7f0cf0f99700 1 -- 172.21.15.31:0/2737173881 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc00bb60 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.645+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2737173881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 msgr2=0x7f0cec15d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.645+0000 7f0cf41ff700 1 --2- 172.21.15.31:0/2737173881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 0x7f0cec15d210 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0d15b0 tx=0x7f0cdc009250 comp rx=0 tx=0).stop 2023-10-06T05:18:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.645+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2737173881 shutdown_connections 2023-10-06T05:18:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.645+0000 7f0cf41ff700 1 --2- 172.21.15.31:0/2737173881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 0x7f0cec15d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.645+0000 7f0cf41ff700 1 --2- 172.21.15.31:0/2737173881 >> v2:172.21.15.62:3300/0 conn(0x7f0cec153950 0x7f0cec153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.645+0000 7f0cf41ff700 1 --2- 172.21.15.31:0/2737173881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 0x7f0cec15d210 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.645+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2737173881 >> 172.21.15.31:0/2737173881 conn(0x7f0cec0681b0 msgr2=0x7f0cec0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.645+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2737173881 shutdown_connections 2023-10-06T05:18:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.645+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2737173881 wait complete. 2023-10-06T05:18:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.646+0000 7f0cf41ff700 1 Processor -- start 2023-10-06T05:18:28.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.647+0000 7f0cf41ff700 1 -- start start 2023-10-06T05:18:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.647+0000 7f0cf41ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 0x7f0cec153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.647+0000 7f0cf41ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 0x7f0cec0c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.647+0000 7f0cf41ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cec153950 0x7f0cec0cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.647+0000 7f0cf41ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0ca0a0 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.647+0000 7f0cf41ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0ca340 con 0x7f0cec153950 2023-10-06T05:18:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.647+0000 7f0cf41ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0b4920 con 0x7f0cec0cda20 2023-10-06T05:18:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf1f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 0x7f0cec153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf1f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 0x7f0cec153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43352/0 (socket says 172.21.15.31:43352) 2023-10-06T05:18:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf1f9b700 1 -- 172.21.15.31:0/2507713221 learned_addr learned my addr 172.21.15.31:0/2507713221 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf279c700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cec153950 0x7f0cec0cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf179a700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 0x7f0cec0c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf1f9b700 1 -- 172.21.15.31:0/2507713221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 msgr2=0x7f0cec0c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf1f9b700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 0x7f0cec0c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf1f9b700 1 -- 172.21.15.31:0/2507713221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cec153950 msgr2=0x7f0cec0cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf1f9b700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cec153950 0x7f0cec0cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf1f9b700 1 -- 172.21.15.31:0/2507713221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc0094f0 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf179a700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 0x7f0cec0c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.648+0000 7f0cf1f9b700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 0x7f0cec153d20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800ec80 tx=0x7f0ce8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.649+0000 7f0ce1ffb700 1 -- 172.21.15.31:0/2507713221 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800c850 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.649+0000 7f0ce1ffb700 1 -- 172.21.15.31:0/2507713221 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce800c9b0 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.649+0000 7f0ce1ffb700 1 -- 172.21.15.31:0/2507713221 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800cd30 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.649+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2507713221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cec0b4c00 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.649+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2507713221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cec0cfde0 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.650+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2507713221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cec0b5120 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.651+0000 7f0ce1ffb700 1 -- 172.21.15.31:0/2507713221 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f0ce8013680 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.652+0000 7f0ce1ffb700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0cd00670e0 0x7f0cd0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:28.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.652+0000 7f0ce1ffb700 1 -- 172.21.15.31:0/2507713221 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f0ce8084f60 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.652+0000 7f0cf179a700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0cd00670e0 0x7f0cd0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:28.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.652+0000 7f0cf179a700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0cd00670e0 0x7f0cd0069590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0d15b0 tx=0x7f0cdc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:28.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.656+0000 7f0ce1ffb700 1 -- 172.21.15.31:0/2507713221 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cec0b5120 con 0x7f0cec0cd5d0 2023-10-06T05:18:28.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:28.859+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2507713221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0cec0b5120 con 0x7f0cec0cd5d0 2023-10-06T05:18:29.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.413+0000 7f0ce1ffb700 1 -- 172.21.15.31:0/2507713221 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0cec0b5120 con 0x7f0cec0cd5d0 2023-10-06T05:18:29.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2507713221 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0cd00670e0 msgr2=0x7f0cd0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0cd00670e0 0x7f0cd0069590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0d15b0 tx=0x7f0cdc017000 comp rx=0 tx=0).stop 2023-10-06T05:18:29.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2507713221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 msgr2=0x7f0cec153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 0x7f0cec153d20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800ec80 tx=0x7f0ce8009120 comp rx=0 tx=0).stop 2023-10-06T05:18:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2507713221 shutdown_connections 2023-10-06T05:18:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cec153950 0x7f0cec0cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0cd00670e0 0x7f0cd0069590 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cec0cd5d0 0x7f0cec153d20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 --2- 172.21.15.31:0/2507713221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0cec0cda20 0x7f0cec0c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2507713221 >> 172.21.15.31:0/2507713221 conn(0x7f0cec0681b0 msgr2=0x7f0cec0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2507713221 shutdown_connections 2023-10-06T05:18:29.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.419+0000 7f0cf41ff700 1 -- 172.21.15.31:0/2507713221 wait complete. 2023-10-06T05:18:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:18:29.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:18:29.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.640+0000 7f4242fff700 1 Processor -- start 2023-10-06T05:18:29.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.640+0000 7f4242fff700 1 -- start start 2023-10-06T05:18:29.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.642+0000 7f4242fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f423c0c8f40 0x7f423c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.642+0000 7f4242fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f423c06a300 0x7f423c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.642+0000 7f4242fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423c068da0 con 0x7f423c15ca40 2023-10-06T05:18:29.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.642+0000 7f4242fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c0cabc0 con 0x7f423c0c8f40 2023-10-06T05:18:29.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.642+0000 7f4242fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c068bb0 con 0x7f423c06a300 2023-10-06T05:18:29.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.642+0000 7f423bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f423c0c8f40 0x7f423c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.642+0000 7f423bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f423c0c8f40 0x7f423c068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43364/0 (socket says 172.21.15.31:43364) 2023-10-06T05:18:29.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.642+0000 7f423bfff700 1 -- 172.21.15.31:0/1189617272 learned_addr learned my addr 172.21.15.31:0/1189617272 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:29.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.642+0000 7f424159c700 1 --2- 172.21.15.31:0/1189617272 >> v2:172.21.15.62:3300/0 conn(0x7f423c06a300 0x7f423c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.643+0000 7f423bfff700 1 -- 172.21.15.31:0/1189617272 >> v2:172.21.15.62:3300/0 conn(0x7f423c06a300 msgr2=0x7f423c0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:29.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.643+0000 7f423bfff700 1 --2- 172.21.15.31:0/1189617272 >> v2:172.21.15.62:3300/0 conn(0x7f423c06a300 0x7f423c0adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.643+0000 7f423bfff700 1 -- 172.21.15.31:0/1189617272 >> v1:172.21.15.62:6789/0 conn(0x7f423c15ca40 legacy=0x7f423c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.643+0000 7f423bfff700 1 -- 172.21.15.31:0/1189617272 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c069080 con 0x7f423c0c8f40 2023-10-06T05:18:29.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.643+0000 7f423bfff700 1 --2- 172.21.15.31:0/1189617272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 0x7f423c068670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f423c06c5f0 tx=0x7f4228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57e4e53b89ab9b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:29.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.643+0000 7f423b7fe700 1 -- 172.21.15.31:0/1189617272 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228009e70 con 0x7f423c0c8f40 2023-10-06T05:18:29.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.643+0000 7f423b7fe700 1 -- 172.21.15.31:0/1189617272 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422800b860 con 0x7f423c0c8f40 2023-10-06T05:18:29.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.643+0000 7f423b7fe700 1 -- 172.21.15.31:0/1189617272 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422800bc00 con 0x7f423c0c8f40 2023-10-06T05:18:29.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.644+0000 7f4242fff700 1 -- 172.21.15.31:0/1189617272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 msgr2=0x7f423c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.644+0000 7f4242fff700 1 --2- 172.21.15.31:0/1189617272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 0x7f423c068670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f423c06c5f0 tx=0x7f4228009250 comp rx=0 tx=0).stop 2023-10-06T05:18:29.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.644+0000 7f4242fff700 1 -- 172.21.15.31:0/1189617272 shutdown_connections 2023-10-06T05:18:29.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.644+0000 7f4242fff700 1 --2- 172.21.15.31:0/1189617272 >> v2:172.21.15.62:3300/0 conn(0x7f423c06a300 0x7f423c0adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.644+0000 7f4242fff700 1 --2- 172.21.15.31:0/1189617272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 0x7f423c068670 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.644+0000 7f4242fff700 1 -- 172.21.15.31:0/1189617272 >> 172.21.15.31:0/1189617272 conn(0x7f423c0ce290 msgr2=0x7f423c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:29.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.644+0000 7f4242fff700 1 -- 172.21.15.31:0/1189617272 shutdown_connections 2023-10-06T05:18:29.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.644+0000 7f4242fff700 1 -- 172.21.15.31:0/1189617272 wait complete. 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.645+0000 7f4242fff700 1 Processor -- start 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.645+0000 7f4242fff700 1 -- start start 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.646+0000 7f4242fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423c06a300 0x7f423c16d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.646+0000 7f4242fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 0x7f423c0c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.646+0000 7f4242fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c15ca40 0x7f423c169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.646+0000 7f4242fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c16df70 con 0x7f423c0c8f40 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.646+0000 7f4242fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0c1eb0 con 0x7f423c15ca40 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.646+0000 7f4242fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0b3fc0 con 0x7f423c06a300 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f423bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 0x7f423c0c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f423bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 0x7f423c0c3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43374/0 (socket says 172.21.15.31:43374) 2023-10-06T05:18:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f423bfff700 1 -- 172.21.15.31:0/2692018328 learned_addr learned my addr 172.21.15.31:0/2692018328 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:29.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f424159c700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c15ca40 0x7f423c169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f4240d9b700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423c06a300 0x7f423c16d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f423bfff700 1 -- 172.21.15.31:0/2692018328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423c06a300 msgr2=0x7f423c16d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f423bfff700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423c06a300 0x7f423c16d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f423bfff700 1 -- 172.21.15.31:0/2692018328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c15ca40 msgr2=0x7f423c169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f423bfff700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c15ca40 0x7f423c169a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f423bfff700 1 -- 172.21.15.31:0/2692018328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42280094f0 con 0x7f423c0c8f40 2023-10-06T05:18:29.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f4240d9b700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423c06a300 0x7f423c16d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:29.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.647+0000 7f423bfff700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 0x7f423c0c3950 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4228006fd0 tx=0x7f4228017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:29.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.648+0000 7f4238ff9700 1 -- 172.21.15.31:0/2692018328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422800b9d0 con 0x7f423c0c8f40 2023-10-06T05:18:29.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.648+0000 7f4238ff9700 1 -- 172.21.15.31:0/2692018328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422800bb30 con 0x7f423c0c8f40 2023-10-06T05:18:29.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.648+0000 7f4242fff700 1 -- 172.21.15.31:0/2692018328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c16d6d0 con 0x7f423c0c8f40 2023-10-06T05:18:29.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.648+0000 7f4238ff9700 1 -- 172.21.15.31:0/2692018328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228025670 con 0x7f423c0c8f40 2023-10-06T05:18:29.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.648+0000 7f4242fff700 1 -- 172.21.15.31:0/2692018328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c16dc90 con 0x7f423c0c8f40 2023-10-06T05:18:29.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.649+0000 7f4242fff700 1 -- 172.21.15.31:0/2692018328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c0c2060 con 0x7f423c0c8f40 2023-10-06T05:18:29.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.650+0000 7f4238ff9700 1 -- 172.21.15.31:0/2692018328 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f4228023070 con 0x7f423c0c8f40 2023-10-06T05:18:29.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.650+0000 7f4238ff9700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f42200670e0 0x7f4220069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.651+0000 7f4238ff9700 1 -- 172.21.15.31:0/2692018328 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4228085b30 con 0x7f423c0c8f40 2023-10-06T05:18:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.651+0000 7f4240d9b700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f42200670e0 0x7f4220069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:29.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.651+0000 7f4240d9b700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f42200670e0 0x7f4220069590 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4230007fd0 tx=0x7f4230009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:29.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.655+0000 7f4238ff9700 1 -- 172.21.15.31:0/2692018328 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f423c0c2060 con 0x7f423c0c8f40 2023-10-06T05:18:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.855+0000 7f4242fff700 1 -- 172.21.15.31:0/2692018328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f423c0c2060 con 0x7f423c0c8f40 2023-10-06T05:18:29.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.859+0000 7f4238ff9700 1 -- 172.21.15.31:0/2692018328 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f423c0c2060 con 0x7f423c0c8f40 2023-10-06T05:18:29.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 -- 172.21.15.31:0/2692018328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f42200670e0 msgr2=0x7f4220069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f42200670e0 0x7f4220069590 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4230007fd0 tx=0x7f4230009040 comp rx=0 tx=0).stop 2023-10-06T05:18:29.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 -- 172.21.15.31:0/2692018328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 msgr2=0x7f423c0c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:29.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 0x7f423c0c3950 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4228006fd0 tx=0x7f4228017000 comp rx=0 tx=0).stop 2023-10-06T05:18:29.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 -- 172.21.15.31:0/2692018328 shutdown_connections 2023-10-06T05:18:29.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f423c15ca40 0x7f423c169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f423c06a300 0x7f423c16d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f42200670e0 0x7f4220069590 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 --2- 172.21.15.31:0/2692018328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c0c8f40 0x7f423c0c3950 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:29.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 -- 172.21.15.31:0/2692018328 >> 172.21.15.31:0/2692018328 conn(0x7f423c0ce290 msgr2=0x7f423c0aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:29.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 -- 172.21.15.31:0/2692018328 shutdown_connections 2023-10-06T05:18:29.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:29.861+0000 7f4242fff700 1 -- 172.21.15.31:0/2692018328 wait complete. 2023-10-06T05:18:29.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-10-06T05:18:30.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.073+0000 7f35111ff700 1 Processor -- start 2023-10-06T05:18:30.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.074+0000 7f35111ff700 1 -- start start 2023-10-06T05:18:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f35111ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f350c068d80 0x7f350c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f35111ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c0cc0d0 0x7f350c0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f35111ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350c0c95b0 con 0x7f350c1595a0 2023-10-06T05:18:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f35111ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350c0c9890 con 0x7f350c068d80 2023-10-06T05:18:30.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f35111ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0c9b70 con 0x7f350c0cc0d0 2023-10-06T05:18:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f350a59c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f350c068d80 0x7f350c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f350b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c0cc0d0 0x7f350c0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f350ad9d700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f350c1595a0 0x7f350c068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:55804/0 (socket says 172.21.15.31:55804) 2023-10-06T05:18:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f350ad9d700 1 -- 172.21.15.31:0/3002538040 learned_addr learned my addr 172.21.15.31:0/3002538040 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f350b59e700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c0cc0d0 0x7f350c0c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:50620/0 (socket says 172.21.15.31:50620) 2023-10-06T05:18:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.075+0000 7f350a59c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f350c068d80 0x7f350c159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33700/0 (socket says 172.21.15.31:33700) 2023-10-06T05:18:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.076+0000 7f3509d9b700 1 -- 172.21.15.31:0/3002538040 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209001942 0 0) 0x7f350c0c95b0 con 0x7f350c1595a0 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.076+0000 7f3509d9b700 1 -- 172.21.15.31:0/3002538040 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34f4003570 con 0x7f350c1595a0 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.076+0000 7f350b59e700 1 -- 172.21.15.31:0/3002538040 >> v2:172.21.15.62:3300/0 conn(0x7f350c068d80 msgr2=0x7f350c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.076+0000 7f350b59e700 1 --2- 172.21.15.31:0/3002538040 >> v2:172.21.15.62:3300/0 conn(0x7f350c068d80 0x7f350c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.076+0000 7f350b59e700 1 -- 172.21.15.31:0/3002538040 >> v1:172.21.15.62:6789/0 conn(0x7f350c1595a0 legacy=0x7f350c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.076+0000 7f350b59e700 1 -- 172.21.15.31:0/3002538040 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c0ca130 con 0x7f350c0cc0d0 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.076+0000 7f350a59c700 1 --2- 172.21.15.31:0/3002538040 >> v2:172.21.15.62:3300/0 conn(0x7f350c068d80 0x7f350c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.076+0000 7f350b59e700 1 --2- 172.21.15.31:0/3002538040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c0cc0d0 0x7f350c0c8930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3504009e90 tx=0x7f3504009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47201a4ffcfb592 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f3509d9b700 1 -- 172.21.15.31:0/3002538040 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350400e070 con 0x7f350c0cc0d0 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f3509d9b700 1 -- 172.21.15.31:0/3002538040 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350400bb40 con 0x7f350c0cc0d0 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f3509d9b700 1 -- 172.21.15.31:0/3002538040 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504012430 con 0x7f350c0cc0d0 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f35111ff700 1 -- 172.21.15.31:0/3002538040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c0cc0d0 msgr2=0x7f350c0c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f35111ff700 1 --2- 172.21.15.31:0/3002538040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c0cc0d0 0x7f350c0c8930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3504009e90 tx=0x7f3504009780 comp rx=0 tx=0).stop 2023-10-06T05:18:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f35111ff700 1 -- 172.21.15.31:0/3002538040 shutdown_connections 2023-10-06T05:18:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f35111ff700 1 --2- 172.21.15.31:0/3002538040 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c0cc0d0 0x7f350c0c8930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f35111ff700 1 --2- 172.21.15.31:0/3002538040 >> v2:172.21.15.62:3300/0 conn(0x7f350c068d80 0x7f350c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f35111ff700 1 -- 172.21.15.31:0/3002538040 >> 172.21.15.31:0/3002538040 conn(0x7f350c06b620 msgr2=0x7f350c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f35111ff700 1 -- 172.21.15.31:0/3002538040 shutdown_connections 2023-10-06T05:18:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.077+0000 7f35111ff700 1 -- 172.21.15.31:0/3002538040 wait complete. 2023-10-06T05:18:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.079+0000 7f35111ff700 1 Processor -- start 2023-10-06T05:18:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.079+0000 7f35111ff700 1 -- start start 2023-10-06T05:18:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.080+0000 7f35111ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c068d80 0x7f350c1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.080+0000 7f35111ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0cc0d0 0x7f350c0c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.080+0000 7f35111ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f350c1595a0 0x7f350c1602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.080+0000 7f35111ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0b1f30 con 0x7f350c0cc0d0 2023-10-06T05:18:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.080+0000 7f35111ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0b2210 con 0x7f350c1595a0 2023-10-06T05:18:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.080+0000 7f35111ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0b24f0 con 0x7f350c068d80 2023-10-06T05:18:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0cc0d0 0x7f350c0c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350ad9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c068d80 0x7f350c1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0cc0d0 0x7f350c0c4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43376/0 (socket says 172.21.15.31:43376) 2023-10-06T05:18:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350a59c700 1 -- 172.21.15.31:0/1729551161 learned_addr learned my addr 172.21.15.31:0/1729551161 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350b59e700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f350c1595a0 0x7f350c1602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350a59c700 1 -- 172.21.15.31:0/1729551161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c068d80 msgr2=0x7f350c1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350a59c700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c068d80 0x7f350c1578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350a59c700 1 -- 172.21.15.31:0/1729551161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f350c1595a0 msgr2=0x7f350c1602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350a59c700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f350c1595a0 0x7f350c1602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350a59c700 1 -- 172.21.15.31:0/1729551161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500009420 con 0x7f350c0cc0d0 2023-10-06T05:18:30.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f350a59c700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0cc0d0 0x7f350c0c4490 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f350000ec80 tx=0x7f35000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.081+0000 7f34f2ffd700 1 -- 172.21.15.31:0/1729551161 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350000c850 con 0x7f350c0cc0d0 2023-10-06T05:18:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.082+0000 7f34f2ffd700 1 -- 172.21.15.31:0/1729551161 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350000c9b0 con 0x7f350c0cc0d0 2023-10-06T05:18:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.082+0000 7f35111ff700 1 -- 172.21.15.31:0/1729551161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3504009a20 con 0x7f350c0cc0d0 2023-10-06T05:18:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.082+0000 7f34f2ffd700 1 -- 172.21.15.31:0/1729551161 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350000cd50 con 0x7f350c0cc0d0 2023-10-06T05:18:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.082+0000 7f35111ff700 1 -- 172.21.15.31:0/1729551161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c0b2ab0 con 0x7f350c0cc0d0 2023-10-06T05:18:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.083+0000 7f35111ff700 1 -- 172.21.15.31:0/1729551161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c04c380 con 0x7f350c0cc0d0 2023-10-06T05:18:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.084+0000 7f34f2ffd700 1 -- 172.21.15.31:0/1729551161 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f3500013680 con 0x7f350c0cc0d0 2023-10-06T05:18:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.085+0000 7f34f2ffd700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f34f80670e0 0x7f34f8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.085+0000 7f34f2ffd700 1 -- 172.21.15.31:0/1729551161 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f35000846e0 con 0x7f350c0cc0d0 2023-10-06T05:18:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.085+0000 7f350ad9d700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f34f80670e0 0x7f34f8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.086+0000 7f350ad9d700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f34f80670e0 0x7f34f8069590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f34fc007fd0 tx=0x7f34fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:30.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.089+0000 7f34f2ffd700 1 -- 172.21.15.31:0/1729551161 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35000517d0 con 0x7f350c0cc0d0 2023-10-06T05:18:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.371+0000 7f35111ff700 1 -- 172.21.15.31:0/1729551161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f350c0b40c0 con 0x7f350c0cc0d0 2023-10-06T05:18:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.411+0000 7f34f2ffd700 1 -- 172.21.15.31:0/1729551161 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v102) v1 ==== 158+0+0 (secure 0 0 0) 0x7f3500054df0 con 0x7f350c0cc0d0 2023-10-06T05:18:30.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.413+0000 7f35111ff700 1 -- 172.21.15.31:0/1729551161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f34f80670e0 msgr2=0x7f34f8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.413+0000 7f35111ff700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f34f80670e0 0x7f34f8069590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f34fc007fd0 tx=0x7f34fc009040 comp rx=0 tx=0).stop 2023-10-06T05:18:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.413+0000 7f35111ff700 1 -- 172.21.15.31:0/1729551161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0cc0d0 msgr2=0x7f350c0c4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.413+0000 7f35111ff700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0cc0d0 0x7f350c0c4490 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f350000ec80 tx=0x7f35000091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.413+0000 7f35111ff700 1 -- 172.21.15.31:0/1729551161 shutdown_connections 2023-10-06T05:18:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.413+0000 7f35111ff700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f350c1595a0 0x7f350c1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.413+0000 7f35111ff700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f350c068d80 0x7f350c1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.413+0000 7f35111ff700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f34f80670e0 0x7f34f8069590 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.413+0000 7f35111ff700 1 --2- 172.21.15.31:0/1729551161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0cc0d0 0x7f350c0c4490 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.413+0000 7f35111ff700 1 -- 172.21.15.31:0/1729551161 >> 172.21.15.31:0/1729551161 conn(0x7f350c06b620 msgr2=0x7f350c153b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.414+0000 7f35111ff700 1 -- 172.21.15.31:0/1729551161 shutdown_connections 2023-10-06T05:18:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.414+0000 7f35111ff700 1 -- 172.21.15.31:0/1729551161 wait complete. 2023-10-06T05:18:30.433 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2023-10-06T05:18:30.433 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:18:30.433 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-10-06T05:18:30.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:30.591 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:18:30.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.627+0000 7f183abff700 1 Processor -- start 2023-10-06T05:18:30.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.627+0000 7f183abff700 1 -- start start 2023-10-06T05:18:30.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.628+0000 7f183abff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1834159500 0x7f18340b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.628+0000 7f183abff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f183415d2d0 0x7f18340cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.628+0000 7f183abff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18340c3f40 con 0x7f18341598d0 2023-10-06T05:18:30.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.628+0000 7f183abff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18340b2860 con 0x7f183415d2d0 2023-10-06T05:18:30.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.629+0000 7f183abff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183415d7d0 con 0x7f1834159500 2023-10-06T05:18:30.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.629+0000 7f1833fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f183415d2d0 0x7f18340cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.629+0000 7f183899b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1834159500 0x7f18340b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.629+0000 7f1833fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f183415d2d0 0x7f18340cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43384/0 (socket says 172.21.15.31:43384) 2023-10-06T05:18:30.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.629+0000 7f1833fff700 1 -- 172.21.15.31:0/278612179 learned_addr learned my addr 172.21.15.31:0/278612179 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:30.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.629+0000 7f183919c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f18341598d0 0x7f18340aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:55814/0 (socket says 172.21.15.31:55814) 2023-10-06T05:18:30.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.629+0000 7f1833fff700 1 -- 172.21.15.31:0/278612179 >> v2:172.21.15.62:3300/0 conn(0x7f1834159500 msgr2=0x7f18340b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:30.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.629+0000 7f1833fff700 1 --2- 172.21.15.31:0/278612179 >> v2:172.21.15.62:3300/0 conn(0x7f1834159500 0x7f18340b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.629+0000 7f1833fff700 1 -- 172.21.15.31:0/278612179 >> v1:172.21.15.62:6789/0 conn(0x7f18341598d0 legacy=0x7f18340aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.629+0000 7f1833fff700 1 -- 172.21.15.31:0/278612179 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18340cb000 con 0x7f183415d2d0 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.630+0000 7f1833fff700 1 --2- 172.21.15.31:0/278612179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183415d2d0 0x7f18340cce00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1834089840 tx=0x7f1820009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f010e08c07d6d87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.630+0000 7f18337fe700 1 -- 172.21.15.31:0/278612179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820005c10 con 0x7f183415d2d0 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.630+0000 7f18337fe700 1 -- 172.21.15.31:0/278612179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1820005d70 con 0x7f183415d2d0 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.630+0000 7f18337fe700 1 -- 172.21.15.31:0/278612179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820006260 con 0x7f183415d2d0 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.631+0000 7f183abff700 1 -- 172.21.15.31:0/278612179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183415d2d0 msgr2=0x7f18340cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.631+0000 7f183abff700 1 --2- 172.21.15.31:0/278612179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183415d2d0 0x7f18340cce00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1834089840 tx=0x7f1820009220 comp rx=0 tx=0).stop 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.631+0000 7f183abff700 1 -- 172.21.15.31:0/278612179 shutdown_connections 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.631+0000 7f183abff700 1 --2- 172.21.15.31:0/278612179 >> v2:172.21.15.62:3300/0 conn(0x7f1834159500 0x7f18340b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.631+0000 7f183abff700 1 --2- 172.21.15.31:0/278612179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183415d2d0 0x7f18340cce00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.631+0000 7f183abff700 1 -- 172.21.15.31:0/278612179 >> 172.21.15.31:0/278612179 conn(0x7f183406ec90 msgr2=0x7f183406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.631+0000 7f183abff700 1 -- 172.21.15.31:0/278612179 shutdown_connections 2023-10-06T05:18:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.631+0000 7f183abff700 1 -- 172.21.15.31:0/278612179 wait complete. 2023-10-06T05:18:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.632+0000 7f183abff700 1 Processor -- start 2023-10-06T05:18:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.632+0000 7f183abff700 1 -- start start 2023-10-06T05:18:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.633+0000 7f183abff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1834159500 0x7f18340c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.633+0000 7f183abff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18341598d0 0x7f18340c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.633+0000 7f183abff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183415d2d0 0x7f1834156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.633+0000 7f183abff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183416a740 con 0x7f18341598d0 2023-10-06T05:18:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.633+0000 7f183abff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834157330 con 0x7f1834159500 2023-10-06T05:18:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.633+0000 7f183abff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1834169b70 con 0x7f183415d2d0 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f1833fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18341598d0 0x7f18340c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f1833fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18341598d0 0x7f18340c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43392/0 (socket says 172.21.15.31:43392) 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f1833fff700 1 -- 172.21.15.31:0/1604435158 learned_addr learned my addr 172.21.15.31:0/1604435158 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f183899b700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1834159500 0x7f18340c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f183919c700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183415d2d0 0x7f1834156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f1833fff700 1 -- 172.21.15.31:0/1604435158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183415d2d0 msgr2=0x7f1834156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f1833fff700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183415d2d0 0x7f1834156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f1833fff700 1 -- 172.21.15.31:0/1604435158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1834159500 msgr2=0x7f18340c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f1833fff700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1834159500 0x7f18340c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f1833fff700 1 -- 172.21.15.31:0/1604435158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820009480 con 0x7f18341598d0 2023-10-06T05:18:30.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.634+0000 7f1833fff700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18341598d0 0x7f18340c9e90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f182000b0e0 tx=0x7f1820016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.635+0000 7f1830ff9700 1 -- 172.21.15.31:0/1604435158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182000ba60 con 0x7f18341598d0 2023-10-06T05:18:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.635+0000 7f183abff700 1 -- 172.21.15.31:0/1604435158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1834169e10 con 0x7f18341598d0 2023-10-06T05:18:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.635+0000 7f183abff700 1 -- 172.21.15.31:0/1604435158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183416a3d0 con 0x7f18341598d0 2023-10-06T05:18:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.635+0000 7f1830ff9700 1 -- 172.21.15.31:0/1604435158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182000bdd0 con 0x7f18341598d0 2023-10-06T05:18:30.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.635+0000 7f1830ff9700 1 -- 172.21.15.31:0/1604435158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182001b430 con 0x7f18341598d0 2023-10-06T05:18:30.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.636+0000 7f183abff700 1 -- 172.21.15.31:0/1604435158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18140052b0 con 0x7f18341598d0 2023-10-06T05:18:30.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.637+0000 7f1830ff9700 1 -- 172.21.15.31:0/1604435158 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f182001b590 con 0x7f18341598d0 2023-10-06T05:18:30.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.638+0000 7f1830ff9700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f18180670e0 0x7f1818069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:30.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.638+0000 7f1830ff9700 1 -- 172.21.15.31:0/1604435158 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1820085d50 con 0x7f18341598d0 2023-10-06T05:18:30.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.638+0000 7f183899b700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f18180670e0 0x7f1818069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:30.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.639+0000 7f183899b700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f18180670e0 0x7f1818069590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f18340ca4d0 tx=0x7f18280091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.646+0000 7f1830ff9700 1 -- 172.21.15.31:0/1604435158 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1820052e40 con 0x7f18341598d0 2023-10-06T05:18:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:30.934+0000 7f183abff700 1 -- 172.21.15.31:0/1604435158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f1814005560 con 0x7f18341598d0 2023-10-06T05:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.412+0000 7f1830ff9700 1 -- 172.21.15.31:0/1604435158 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v103) v1 ==== 143+0+0 (secure 0 0 0) 0x7f1820056460 con 0x7f18341598d0 2023-10-06T05:18:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 -- 172.21.15.31:0/1604435158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f18180670e0 msgr2=0x7f1818069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f18180670e0 0x7f1818069590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f18340ca4d0 tx=0x7f18280091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 -- 172.21.15.31:0/1604435158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18341598d0 msgr2=0x7f18340c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18341598d0 0x7f18340c9e90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f182000b0e0 tx=0x7f1820016000 comp rx=0 tx=0).stop 2023-10-06T05:18:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 -- 172.21.15.31:0/1604435158 shutdown_connections 2023-10-06T05:18:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183415d2d0 0x7f1834156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1834159500 0x7f18340c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f18180670e0 0x7f1818069590 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 --2- 172.21.15.31:0/1604435158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18341598d0 0x7f18340c9e90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 -- 172.21.15.31:0/1604435158 >> 172.21.15.31:0/1604435158 conn(0x7f183406ec90 msgr2=0x7f18340cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.415+0000 7f183abff700 1 -- 172.21.15.31:0/1604435158 shutdown_connections 2023-10-06T05:18:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.416+0000 7f183abff700 1 -- 172.21.15.31:0/1604435158 wait complete. 2023-10-06T05:18:31.438 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:18:31.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.636+0000 7f9330bff700 1 Processor -- start 2023-10-06T05:18:31.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.636+0000 7f9330bff700 1 -- start start 2023-10-06T05:18:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.637+0000 7f9330bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f932c153d40 0x7f932c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.637+0000 7f9330bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 0x7f932c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.638+0000 7f9330bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c0ad690 con 0x7f932c0af6c0 2023-10-06T05:18:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.638+0000 7f9330bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c0b0e10 con 0x7f932c153d40 2023-10-06T05:18:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.638+0000 7f9330bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c0acd60 con 0x7f932c0b09c0 2023-10-06T05:18:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.638+0000 7f932ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f932c0af6c0 0x7f932c159ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57546/0 (socket says 172.21.15.31:57546) 2023-10-06T05:18:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.638+0000 7f932ad9d700 1 -- 172.21.15.31:0/886262468 learned_addr learned my addr 172.21.15.31:0/886262468 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.638+0000 7f9329d9b700 1 --2- 172.21.15.31:0/886262468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 0x7f932c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.638+0000 7f932a59c700 1 --2- 172.21.15.31:0/886262468 >> v2:172.21.15.62:3300/0 conn(0x7f932c153d40 0x7f932c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.638+0000 7f932959a700 1 -- 172.21.15.31:0/886262468 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938003745 0 0) 0x7f932c0ad690 con 0x7f932c0af6c0 2023-10-06T05:18:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.638+0000 7f932959a700 1 -- 172.21.15.31:0/886262468 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9314003570 con 0x7f932c0af6c0 2023-10-06T05:18:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.638+0000 7f932a59c700 1 -- 172.21.15.31:0/886262468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 msgr2=0x7f932c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.639+0000 7f932a59c700 1 --2- 172.21.15.31:0/886262468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 0x7f932c0af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.639+0000 7f932a59c700 1 -- 172.21.15.31:0/886262468 >> v1:172.21.15.31:6789/0 conn(0x7f932c0af6c0 legacy=0x7f932c159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.639+0000 7f932a59c700 1 -- 172.21.15.31:0/886262468 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c0ad040 con 0x7f932c153d40 2023-10-06T05:18:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.639+0000 7f9329d9b700 1 --2- 172.21.15.31:0/886262468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 0x7f932c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.639+0000 7f932a59c700 1 --2- 172.21.15.31:0/886262468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932c153d40 0x7f932c0b0480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098e0 tx=0x7f931c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc6c493e60ebca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.639+0000 7f932959a700 1 -- 172.21.15.31:0/886262468 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c011630 con 0x7f932c153d40 2023-10-06T05:18:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.639+0000 7f932959a700 1 -- 172.21.15.31:0/886262468 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c011790 con 0x7f932c153d40 2023-10-06T05:18:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.639+0000 7f932959a700 1 -- 172.21.15.31:0/886262468 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c011b30 con 0x7f932c153d40 2023-10-06T05:18:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.640+0000 7f9330bff700 1 -- 172.21.15.31:0/886262468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932c153d40 msgr2=0x7f932c0b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.640+0000 7f9330bff700 1 --2- 172.21.15.31:0/886262468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932c153d40 0x7f932c0b0480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098e0 tx=0x7f931c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.640+0000 7f9330bff700 1 -- 172.21.15.31:0/886262468 shutdown_connections 2023-10-06T05:18:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.640+0000 7f9330bff700 1 --2- 172.21.15.31:0/886262468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932c153d40 0x7f932c0b0480 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.640+0000 7f9330bff700 1 --2- 172.21.15.31:0/886262468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 0x7f932c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.640+0000 7f9330bff700 1 -- 172.21.15.31:0/886262468 >> 172.21.15.31:0/886262468 conn(0x7f932c067560 msgr2=0x7f932c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.640+0000 7f9330bff700 1 -- 172.21.15.31:0/886262468 shutdown_connections 2023-10-06T05:18:31.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.640+0000 7f9330bff700 1 -- 172.21.15.31:0/886262468 wait complete. 2023-10-06T05:18:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.641+0000 7f9330bff700 1 Processor -- start 2023-10-06T05:18:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.642+0000 7f9330bff700 1 -- start start 2023-10-06T05:18:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.642+0000 7f9330bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932c0af6c0 0x7f932c1698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.642+0000 7f9330bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 0x7f932c1492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f9330bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932c153d40 0x7f932c152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f9330bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c0cb500 con 0x7f932c153d40 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f9330bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c1478a0 con 0x7f932c0af6c0 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f9330bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c169e20 con 0x7f932c0b09c0 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f932ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932c153d40 0x7f932c152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f932ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932c153d40 0x7f932c152870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43408/0 (socket says 172.21.15.31:43408) 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f932ad9d700 1 -- 172.21.15.31:0/2467186837 learned_addr learned my addr 172.21.15.31:0/2467186837 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f932a59c700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932c0af6c0 0x7f932c1698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f9329d9b700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 0x7f932c1492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f932ad9d700 1 -- 172.21.15.31:0/2467186837 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 msgr2=0x7f932c1492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f932ad9d700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 0x7f932c1492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f932ad9d700 1 -- 172.21.15.31:0/2467186837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932c0af6c0 msgr2=0x7f932c1698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f932ad9d700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932c0af6c0 0x7f932c1698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.643+0000 7f932ad9d700 1 -- 172.21.15.31:0/2467186837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9320009420 con 0x7f932c153d40 2023-10-06T05:18:31.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.644+0000 7f932ad9d700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932c153d40 0x7f932c152870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f932400cca0 tx=0x7f9324001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:31.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.644+0000 7f931a7fc700 1 -- 172.21.15.31:0/2467186837 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932400ab30 con 0x7f932c153d40 2023-10-06T05:18:31.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.644+0000 7f931a7fc700 1 -- 172.21.15.31:0/2467186837 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932400ac90 con 0x7f932c153d40 2023-10-06T05:18:31.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.644+0000 7f9330bff700 1 -- 172.21.15.31:0/2467186837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c009420 con 0x7f932c153d40 2023-10-06T05:18:31.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.644+0000 7f931a7fc700 1 -- 172.21.15.31:0/2467186837 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324007300 con 0x7f932c153d40 2023-10-06T05:18:31.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.644+0000 7f9330bff700 1 -- 172.21.15.31:0/2467186837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c146d40 con 0x7f932c153d40 2023-10-06T05:18:31.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.645+0000 7f9330bff700 1 -- 172.21.15.31:0/2467186837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932c147a50 con 0x7f932c153d40 2023-10-06T05:18:31.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.646+0000 7f931a7fc700 1 -- 172.21.15.31:0/2467186837 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9324007730 con 0x7f932c153d40 2023-10-06T05:18:31.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.647+0000 7f931a7fc700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f930c0670e0 0x7f930c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:31.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.647+0000 7f931a7fc700 1 -- 172.21.15.31:0/2467186837 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9324084bf0 con 0x7f932c153d40 2023-10-06T05:18:31.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.647+0000 7f932a59c700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f930c0670e0 0x7f930c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:31.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.648+0000 7f932a59c700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f930c0670e0 0x7f930c069590 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f931c0091e0 tx=0x7f931c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:31.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.651+0000 7f931a7fc700 1 -- 172.21.15.31:0/2467186837 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9324051cd0 con 0x7f932c153d40 2023-10-06T05:18:31.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.859+0000 7f9330bff700 1 -- 172.21.15.31:0/2467186837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f932c0ad320 con 0x7f932c153d40 2023-10-06T05:18:31.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.864+0000 7f931a7fc700 1 -- 172.21.15.31:0/2467186837 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f93240552f0 con 0x7f932c153d40 2023-10-06T05:18:31.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 -- 172.21.15.31:0/2467186837 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f930c0670e0 msgr2=0x7f930c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f930c0670e0 0x7f930c069590 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f931c0091e0 tx=0x7f931c017000 comp rx=0 tx=0).stop 2023-10-06T05:18:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 -- 172.21.15.31:0/2467186837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932c153d40 msgr2=0x7f932c152870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932c153d40 0x7f932c152870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f932400cca0 tx=0x7f9324001ae0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 -- 172.21.15.31:0/2467186837 shutdown_connections 2023-10-06T05:18:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f930c0670e0 0x7f930c069590 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932c153d40 0x7f932c152870 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f932c0af6c0 0x7f932c1698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 --2- 172.21.15.31:0/2467186837 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f932c0b09c0 0x7f932c1492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 -- 172.21.15.31:0/2467186837 >> 172.21.15.31:0/2467186837 conn(0x7f932c067560 msgr2=0x7f932c0abe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 -- 172.21.15.31:0/2467186837 shutdown_connections 2023-10-06T05:18:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:31.866+0000 7f9330bff700 1 -- 172.21.15.31:0/2467186837 wait complete. 2023-10-06T05:18:31.886 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:18:31.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:18:31.985 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:18:32.090 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:18:32.090 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:18:32.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:32.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.284+0000 7f1ff01ff700 1 Processor -- start 2023-10-06T05:18:32.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.284+0000 7f1ff01ff700 1 -- start start 2023-10-06T05:18:32.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.285+0000 7f1ff01ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1fe813a1e0 0x7f1fe80c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.285+0000 7f1ff01ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fe813a5b0 0x7f1fe806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.285+0000 7f1ff01ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1fe80c6c30 0x7f1fe806d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.285+0000 7f1ff01ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe80c71b0 con 0x7f1fe813a1e0 2023-10-06T05:18:32.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.285+0000 7f1ff01ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe80c7490 con 0x7f1fe80c6c30 2023-10-06T05:18:32.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.285+0000 7f1ff01ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c7770 con 0x7f1fe813a5b0 2023-10-06T05:18:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fedf9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1fe813a1e0 0x7f1fe80c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fedf9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1fe813a1e0 0x7f1fe80c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43410/0 (socket says 172.21.15.31:43410) 2023-10-06T05:18:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fedf9b700 1 -- 172.21.15.31:0/3019765021 learned_addr learned my addr 172.21.15.31:0/3019765021 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fed79a700 1 --2- 172.21.15.31:0/3019765021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fe813a5b0 0x7f1fe806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fee79c700 1 --2- 172.21.15.31:0/3019765021 >> v2:172.21.15.62:3300/0 conn(0x7f1fe80c6c30 0x7f1fe806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fedf9b700 1 -- 172.21.15.31:0/3019765021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fe813a5b0 msgr2=0x7f1fe806d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fedf9b700 1 --2- 172.21.15.31:0/3019765021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fe813a5b0 0x7f1fe806d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fedf9b700 1 -- 172.21.15.31:0/3019765021 >> v2:172.21.15.62:3300/0 conn(0x7f1fe80c6c30 msgr2=0x7f1fe806d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fedf9b700 1 --2- 172.21.15.31:0/3019765021 >> v2:172.21.15.62:3300/0 conn(0x7f1fe80c6c30 0x7f1fe806d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fedf9b700 1 -- 172.21.15.31:0/3019765021 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80c7a50 con 0x7f1fe813a1e0 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.286+0000 7f1fedf9b700 1 --2- 172.21.15.31:0/3019765021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 0x7f1fe80c66f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1fe806b530 tx=0x7f1fe4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470c7c5c6d4097b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1fecf99700 1 -- 172.21.15.31:0/3019765021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400b860 con 0x7f1fe813a1e0 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1fecf99700 1 -- 172.21.15.31:0/3019765021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe400b9c0 con 0x7f1fe813a1e0 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1fecf99700 1 -- 172.21.15.31:0/3019765021 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400bd60 con 0x7f1fe813a1e0 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1ff01ff700 1 -- 172.21.15.31:0/3019765021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 msgr2=0x7f1fe80c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1ff01ff700 1 --2- 172.21.15.31:0/3019765021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 0x7f1fe80c66f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1fe806b530 tx=0x7f1fe4009250 comp rx=0 tx=0).stop 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1ff01ff700 1 -- 172.21.15.31:0/3019765021 shutdown_connections 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1ff01ff700 1 --2- 172.21.15.31:0/3019765021 >> v2:172.21.15.62:3300/0 conn(0x7f1fe80c6c30 0x7f1fe806d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1ff01ff700 1 --2- 172.21.15.31:0/3019765021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 0x7f1fe80c66f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1ff01ff700 1 --2- 172.21.15.31:0/3019765021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fe813a5b0 0x7f1fe806d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1ff01ff700 1 -- 172.21.15.31:0/3019765021 >> 172.21.15.31:0/3019765021 conn(0x7f1fe80cc0b0 msgr2=0x7f1fe80cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1ff01ff700 1 -- 172.21.15.31:0/3019765021 shutdown_connections 2023-10-06T05:18:32.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.287+0000 7f1ff01ff700 1 -- 172.21.15.31:0/3019765021 wait complete. 2023-10-06T05:18:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.289+0000 7f1ff01ff700 1 Processor -- start 2023-10-06T05:18:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.290+0000 7f1ff01ff700 1 -- start start 2023-10-06T05:18:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1ff01ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe80c6c30 0x7f1fe8133820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1ff01ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 0x7f1fe8156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1ff01ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fe813a5b0 0x7f1fe8157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1ff01ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8157670 con 0x7f1fe813a1e0 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1ff01ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c43e0 con 0x7f1fe80c6c30 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1ff01ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c37d0 con 0x7f1fe813a5b0 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1fed79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 0x7f1fe8156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1fed79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 0x7f1fe8156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43424/0 (socket says 172.21.15.31:43424) 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1fed79a700 1 -- 172.21.15.31:0/241300609 learned_addr learned my addr 172.21.15.31:0/241300609 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1fedf9b700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe80c6c30 0x7f1fe8133820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.291+0000 7f1fee79c700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fe813a5b0 0x7f1fe8157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1fed79a700 1 -- 172.21.15.31:0/241300609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fe813a5b0 msgr2=0x7f1fe8157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1fed79a700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fe813a5b0 0x7f1fe8157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1fed79a700 1 -- 172.21.15.31:0/241300609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe80c6c30 msgr2=0x7f1fe8133820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1fed79a700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe80c6c30 0x7f1fe8133820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1fed79a700 1 -- 172.21.15.31:0/241300609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe4009490 con 0x7f1fe813a1e0 2023-10-06T05:18:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1fed79a700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 0x7f1fe8156bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d860 tx=0x7f1fd8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1fddffb700 1 -- 172.21.15.31:0/241300609 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80125f0 con 0x7f1fe813a1e0 2023-10-06T05:18:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1fddffb700 1 -- 172.21.15.31:0/241300609 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8012770 con 0x7f1fe813a1e0 2023-10-06T05:18:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1fddffb700 1 -- 172.21.15.31:0/241300609 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8012b30 con 0x7f1fe813a1e0 2023-10-06T05:18:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1ff01ff700 1 -- 172.21.15.31:0/241300609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe80c3ab0 con 0x7f1fe813a1e0 2023-10-06T05:18:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.292+0000 7f1ff01ff700 1 -- 172.21.15.31:0/241300609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe80c4070 con 0x7f1fe813a1e0 2023-10-06T05:18:32.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.293+0000 7f1ff01ff700 1 -- 172.21.15.31:0/241300609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe80c4590 con 0x7f1fe813a1e0 2023-10-06T05:18:32.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.294+0000 7f1fddffb700 1 -- 172.21.15.31:0/241300609 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f1fd8013480 con 0x7f1fe813a1e0 2023-10-06T05:18:32.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.295+0000 7f1fddffb700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fcc0670e0 0x7f1fcc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.295+0000 7f1fddffb700 1 -- 172.21.15.31:0/241300609 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1fd8084a60 con 0x7f1fe813a1e0 2023-10-06T05:18:32.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.295+0000 7f1fedf9b700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fcc0670e0 0x7f1fcc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.296+0000 7f1fedf9b700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fcc0670e0 0x7f1fcc069590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1fe806b530 tx=0x7f1fe4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.299+0000 7f1fddffb700 1 -- 172.21.15.31:0/241300609 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1fe80c4590 con 0x7f1fe813a1e0 2023-10-06T05:18:32.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.551+0000 7f1ff01ff700 1 -- 172.21.15.31:0/241300609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fe80c4590 con 0x7f1fe813a1e0 2023-10-06T05:18:32.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.552+0000 7f1fddffb700 1 -- 172.21.15.31:0/241300609 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f1fe80c4590 con 0x7f1fe813a1e0 2023-10-06T05:18:32.555 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:32.555 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/746584225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":746584225},{"type":"v1","addr":"172.21.15.62:6834","nonce":746584225}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/584374139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":584374139},{"type":"v1","addr":"172.21.15.62:6835","nonce":584374139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4926,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3794168057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3794168057},{"type":"v1","addr":"172.21.15.31:6836","nonce":3794168057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4276891855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":4276891855},{"type":"v1","addr":"172.21.15.31:6837","nonce":4276891855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-10-06T05:18:32.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 -- 172.21.15.31:0/241300609 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fcc0670e0 msgr2=0x7f1fcc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fcc0670e0 0x7f1fcc069590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1fe806b530 tx=0x7f1fe4017000 comp rx=0 tx=0).stop 2023-10-06T05:18:32.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 -- 172.21.15.31:0/241300609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 msgr2=0x7f1fe8156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 0x7f1fe8156bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d860 tx=0x7f1fd8008040 comp rx=0 tx=0).stop 2023-10-06T05:18:32.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 -- 172.21.15.31:0/241300609 shutdown_connections 2023-10-06T05:18:32.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1fe813a5b0 0x7f1fe8157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe80c6c30 0x7f1fe8133820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1fcc0670e0 0x7f1fcc069590 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 --2- 172.21.15.31:0/241300609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe813a1e0 0x7f1fe8156bf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 -- 172.21.15.31:0/241300609 >> 172.21.15.31:0/241300609 conn(0x7f1fe80cc0b0 msgr2=0x7f1fe80b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 -- 172.21.15.31:0/241300609 shutdown_connections 2023-10-06T05:18:32.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.554+0000 7f1ff01ff700 1 -- 172.21.15.31:0/241300609 wait complete. 2023-10-06T05:18:32.558 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2023-10-06T05:18:32.575 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:18:32.575 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:18:32.575 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:18:32.575 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:18:32.575 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:18:32.576 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:18:32.576 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:18:32.576 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:18:32.576 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:18:32.576 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:18:32.576 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:18:32.577 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:18:32.577 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:18:32.577 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:18:32.577 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:18:32.577 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:18:32.590 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:18:32.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:18:32.591 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:18:32.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:18:32.592 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:18:32.593 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:18:32.593 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:18:32.594 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:18:32.635 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:18:32.635 DEBUG:teuthology.parallel:result is None 2023-10-06T05:18:32.637 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:18:32.637 DEBUG:teuthology.parallel:result is None 2023-10-06T05:18:32.638 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:18:32.638 DEBUG:teuthology.parallel:result is None 2023-10-06T05:18:32.640 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:18:32.640 DEBUG:teuthology.parallel:result is None 2023-10-06T05:18:32.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:18:32.783 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:18:32.783 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:18:32.791 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:18:32.791 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:18:32.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.842+0000 7f9428fff700 1 Processor -- start 2023-10-06T05:18:32.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.842+0000 7f9428fff700 1 -- start start 2023-10-06T05:18:32.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.844+0000 7f9428fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f942406b510 0x7f94240c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.844+0000 7f9428fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94240cab40 0x7f94241569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.844+0000 7f9428fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9424156ef0 con 0x7f94240ca770 2023-10-06T05:18:32.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.844+0000 7f9428fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94240c7a70 con 0x7f94240cab40 2023-10-06T05:18:32.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.844+0000 7f9428fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94240c6df0 con 0x7f942406b510 2023-10-06T05:18:32.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.844+0000 7f9422d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94240cab40 0x7f94241569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.844+0000 7f9422d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94240cab40 0x7f94241569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43448/0 (socket says 172.21.15.31:43448) 2023-10-06T05:18:32.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.844+0000 7f9422d9d700 1 -- 172.21.15.31:0/530285503 learned_addr learned my addr 172.21.15.31:0/530285503 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:32.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.844+0000 7f9421d9b700 1 --2- 172.21.15.31:0/530285503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f942406b510 0x7f94240c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.845+0000 7f9422d9d700 1 -- 172.21.15.31:0/530285503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f942406b510 msgr2=0x7f94240c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:32.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.845+0000 7f9422d9d700 1 --2- 172.21.15.31:0/530285503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f942406b510 0x7f94240c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.845+0000 7f9422d9d700 1 -- 172.21.15.31:0/530285503 >> v1:172.21.15.62:6789/0 conn(0x7f94240ca770 legacy=0x7f942406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.845+0000 7f9422d9d700 1 -- 172.21.15.31:0/530285503 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94240c70d0 con 0x7f94240cab40 2023-10-06T05:18:32.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.845+0000 7f942159a700 1 -- 172.21.15.31:0/530285503 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443232309 0 0) 0x7f9424156ef0 con 0x7f94240ca770 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.845+0000 7f9422d9d700 1 --2- 172.21.15.31:0/530285503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 0x7f94241569b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f941c009f20 tx=0x7f941c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33117e89b6cfe993 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.845+0000 7f942159a700 1 -- 172.21.15.31:0/530285503 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c00e070 con 0x7f94240cab40 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.845+0000 7f942159a700 1 -- 172.21.15.31:0/530285503 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c00b9d0 con 0x7f94240cab40 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.845+0000 7f942159a700 1 -- 172.21.15.31:0/530285503 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0092e0 con 0x7f94240cab40 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.846+0000 7f9428fff700 1 -- 172.21.15.31:0/530285503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 msgr2=0x7f94241569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.846+0000 7f9428fff700 1 --2- 172.21.15.31:0/530285503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 0x7f94241569b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f941c009f20 tx=0x7f941c009840 comp rx=0 tx=0).stop 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.846+0000 7f9428fff700 1 -- 172.21.15.31:0/530285503 shutdown_connections 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.846+0000 7f9428fff700 1 --2- 172.21.15.31:0/530285503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 0x7f94241569b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.846+0000 7f9428fff700 1 --2- 172.21.15.31:0/530285503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f942406b510 0x7f94240c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.846+0000 7f9428fff700 1 -- 172.21.15.31:0/530285503 >> 172.21.15.31:0/530285503 conn(0x7f94240cc0b0 msgr2=0x7f94240cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.846+0000 7f9428fff700 1 -- 172.21.15.31:0/530285503 shutdown_connections 2023-10-06T05:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.846+0000 7f9428fff700 1 -- 172.21.15.31:0/530285503 wait complete. 2023-10-06T05:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.848+0000 7f9428fff700 1 Processor -- start 2023-10-06T05:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.848+0000 7f9428fff700 1 -- start start 2023-10-06T05:18:32.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.849+0000 7f9428fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f942406b510 0x7f942416aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9428fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94240ca770 0x7f942416af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9428fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 0x7f9424167430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9428fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94241692b0 con 0x7f94240cab40 2023-10-06T05:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9428fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94241686a0 con 0x7f942406b510 2023-10-06T05:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9428fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9424168980 con 0x7f94240ca770 2023-10-06T05:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9422d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 0x7f9424167430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9421d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94240ca770 0x7f942416af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9422d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 0x7f9424167430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43462/0 (socket says 172.21.15.31:43462) 2023-10-06T05:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9422d9d700 1 -- 172.21.15.31:0/1704279615 learned_addr learned my addr 172.21.15.31:0/1704279615 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f942259c700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f942406b510 0x7f942416aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9421d9b700 1 -- 172.21.15.31:0/1704279615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f942406b510 msgr2=0x7f942416aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9421d9b700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f942406b510 0x7f942416aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9421d9b700 1 -- 172.21.15.31:0/1704279615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 msgr2=0x7f9424167430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:32.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9421d9b700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 0x7f9424167430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:32.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9421d9b700 1 -- 172.21.15.31:0/1704279615 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c009ae0 con 0x7f94240ca770 2023-10-06T05:18:32.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.850+0000 7f9422d9d700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 0x7f9424167430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:32.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.851+0000 7f9421d9b700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94240ca770 0x7f942416af70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f941800e900 tx=0x7f94180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.851+0000 7f94127fc700 1 -- 172.21.15.31:0/1704279615 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418013680 con 0x7f94240ca770 2023-10-06T05:18:32.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.851+0000 7f94127fc700 1 -- 172.21.15.31:0/1704279615 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94180137e0 con 0x7f94240ca770 2023-10-06T05:18:32.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.851+0000 7f94127fc700 1 -- 172.21.15.31:0/1704279615 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418013ba0 con 0x7f94240ca770 2023-10-06T05:18:32.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.851+0000 7f9428fff700 1 -- 172.21.15.31:0/1704279615 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9424168c60 con 0x7f94240ca770 2023-10-06T05:18:32.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.851+0000 7f9428fff700 1 -- 172.21.15.31:0/1704279615 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9424165070 con 0x7f94240ca770 2023-10-06T05:18:32.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.852+0000 7f9428fff700 1 -- 172.21.15.31:0/1704279615 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942404c380 con 0x7f94240ca770 2023-10-06T05:18:32.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.853+0000 7f94127fc700 1 -- 172.21.15.31:0/1704279615 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f94180123d0 con 0x7f94240ca770 2023-10-06T05:18:32.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.854+0000 7f94127fc700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f94040670e0 0x7f9404069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:32.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.854+0000 7f94127fc700 1 -- 172.21.15.31:0/1704279615 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9418084770 con 0x7f94240ca770 2023-10-06T05:18:32.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.854+0000 7f942259c700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f94040670e0 0x7f9404069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:32.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.855+0000 7f942259c700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f94040670e0 0x7f9404069590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9414007fd0 tx=0x7f9414009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:32.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:32.860+0000 7f94127fc700 1 -- 172.21.15.31:0/1704279615 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9418051860 con 0x7f94240ca770 2023-10-06T05:18:33.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.083+0000 7f9428fff700 1 -- 172.21.15.31:0/1704279615 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9424168ac0 con 0x7f94240ca770 2023-10-06T05:18:33.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.883+0000 7f94127fc700 1 -- 172.21.15.31:0/1704279615 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9424168ac0 con 0x7f94240ca770 2023-10-06T05:18:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.886+0000 7f9428fff700 1 -- 172.21.15.31:0/1704279615 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f94040670e0 msgr2=0x7f9404069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f94040670e0 0x7f9404069590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9414007fd0 tx=0x7f9414009040 comp rx=0 tx=0).stop 2023-10-06T05:18:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 -- 172.21.15.31:0/1704279615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94240ca770 msgr2=0x7f942416af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94240ca770 0x7f942416af70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f941800e900 tx=0x7f94180091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 -- 172.21.15.31:0/1704279615 shutdown_connections 2023-10-06T05:18:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f94040670e0 0x7f9404069590 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94240cab40 0x7f9424167430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f942406b510 0x7f942416aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 --2- 172.21.15.31:0/1704279615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94240ca770 0x7f942416af70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 -- 172.21.15.31:0/1704279615 >> 172.21.15.31:0/1704279615 conn(0x7f94240cc0b0 msgr2=0x7f942406e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:33.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 -- 172.21.15.31:0/1704279615 shutdown_connections 2023-10-06T05:18:33.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:33.887+0000 7f9428fff700 1 -- 172.21.15.31:0/1704279615 wait complete. 2023-10-06T05:18:33.891 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:18:33.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:18:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.108+0000 7f1bc11ff700 1 Processor -- start 2023-10-06T05:18:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.108+0000 7f1bc11ff700 1 -- start start 2023-10-06T05:18:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.109+0000 7f1bc11ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc0cd510 0x7f1bbc0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.109+0000 7f1bc11ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1bbc15d2d0 0x7f1bbc0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.109+0000 7f1bc11ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bbc0c3f40 con 0x7f1bbc159500 2023-10-06T05:18:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.109+0000 7f1bc11ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bbc0b2860 con 0x7f1bbc15d2d0 2023-10-06T05:18:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.109+0000 7f1bc11ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc15d7d0 con 0x7f1bbc0cd510 2023-10-06T05:18:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.109+0000 7f1bbad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1bbc159500 0x7f1bbc0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57562/0 (socket says 172.21.15.31:57562) 2023-10-06T05:18:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bbad9d700 1 -- 172.21.15.31:0/105183125 learned_addr learned my addr 172.21.15.31:0/105183125 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:34.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bba59c700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc0cd510 0x7f1bbc0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bbb59e700 1 --2- 172.21.15.31:0/105183125 >> v2:172.21.15.62:3300/0 conn(0x7f1bbc15d2d0 0x7f1bbc0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307502347 0 0) 0x7f1bbc0c3f40 con 0x7f1bbc159500 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ba4003570 con 0x7f1bbc159500 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1060386582 0 0) 0x7f1ba4003570 con 0x7f1bbc159500 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc0cd510 msgr2=0x7f1bbc0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bb9d9b700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc0cd510 0x7f1bbc0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 >> v2:172.21.15.62:3300/0 conn(0x7f1bbc15d2d0 msgr2=0x7f1bbc0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bb9d9b700 1 --2- 172.21.15.31:0/105183125 >> v2:172.21.15.62:3300/0 conn(0x7f1bbc15d2d0 0x7f1bbc0aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc0cb000 con 0x7f1bbc159500 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f1bac001470 con 0x7f1bbc159500 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.110+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 >> v1:172.21.15.31:6789/0 conn(0x7f1bbc159500 legacy=0x7f1bbc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bb9d9b700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba4003a60 0x7f1ba4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bb9d9b700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 0x7f1ba4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bb9d9b700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc0cd510 0x7f1ba40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc0b2860 con 0x7f1ba4004810 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc15d7d0 con 0x7f1ba4003a60 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba400ba60 con 0x7f1bbc0cd510 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bba59c700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 0x7f1ba4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bbb59e700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc0cd510 0x7f1ba40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bbad9d700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba4003a60 0x7f1ba4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bba59c700 1 -- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc0cd510 msgr2=0x7f1ba40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bba59c700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc0cd510 0x7f1ba40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bba59c700 1 -- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba4003a60 msgr2=0x7f1ba4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bba59c700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ba4003a60 0x7f1ba4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.111+0000 7f1bba59c700 1 -- 172.21.15.31:0/105183125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba4003570 con 0x7f1ba4004810 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.112+0000 7f1bba59c700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 0x7f1ba4004c80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009c20 tx=0x7f1bb0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c77461b239457d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.112+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0005a00 con 0x7f1ba4004810 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.112+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb0005b60 con 0x7f1ba4004810 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.112+0000 7f1bb9d9b700 1 -- 172.21.15.31:0/105183125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000e710 con 0x7f1ba4004810 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.112+0000 7f1bc11ff700 1 -- 172.21.15.31:0/105183125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 msgr2=0x7f1ba4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.112+0000 7f1bc11ff700 1 --2- 172.21.15.31:0/105183125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 0x7f1ba4004c80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009c20 tx=0x7f1bb0009120 comp rx=0 tx=0).stop 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.112+0000 7f1bbb59e700 1 -- 172.21.15.31:0/105183125 reap_dead start 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.113+0000 7f1bc11ff700 1 -- 172.21.15.31:0/105183125 shutdown_connections 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.113+0000 7f1bc11ff700 1 -- 172.21.15.31:0/105183125 >> 172.21.15.31:0/105183125 conn(0x7f1bbc06ec90 msgr2=0x7f1bbc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:34.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.113+0000 7f1bc11ff700 1 -- 172.21.15.31:0/105183125 shutdown_connections 2023-10-06T05:18:34.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.113+0000 7f1bc11ff700 1 -- 172.21.15.31:0/105183125 wait complete. 2023-10-06T05:18:34.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.114+0000 7f1bc11ff700 1 Processor -- start 2023-10-06T05:18:34.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.114+0000 7f1bc11ff700 1 -- start start 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.115+0000 7f1bc11ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 0x7f1bbc150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.115+0000 7f1bc11ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc15b5d0 0x7f1bbc1598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.115+0000 7f1bc11ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc159e10 0x7f1bbc0d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.115+0000 7f1bc11ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc15c770 con 0x7f1ba4004810 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.115+0000 7f1bc11ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc15ab80 con 0x7f1bbc15b5d0 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.115+0000 7f1bc11ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc15a320 con 0x7f1bbc159e10 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.115+0000 7f1bbad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 0x7f1bbc150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bbad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 0x7f1bbc150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43482/0 (socket says 172.21.15.31:43482) 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bbad9d700 1 -- 172.21.15.31:0/546872949 learned_addr learned my addr 172.21.15.31:0/546872949 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bbb59e700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc159e10 0x7f1bbc0d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bbad9d700 1 -- 172.21.15.31:0/546872949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc159e10 msgr2=0x7f1bbc0d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bba59c700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc15b5d0 0x7f1bbc1598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bbad9d700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc159e10 0x7f1bbc0d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bbad9d700 1 -- 172.21.15.31:0/546872949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc15b5d0 msgr2=0x7f1bbc1598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bbad9d700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc15b5d0 0x7f1bbc1598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bbad9d700 1 -- 172.21.15.31:0/546872949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc15a600 con 0x7f1ba4004810 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bbb59e700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc159e10 0x7f1bbc0d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bba59c700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc15b5d0 0x7f1bbc1598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bbad9d700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 0x7f1bbc150270 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0073c0 tx=0x7f1bac001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1ba2ffd700 1 -- 172.21.15.31:0/546872949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac018030 con 0x7f1ba4004810 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1ba2ffd700 1 -- 172.21.15.31:0/546872949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac00f8d0 con 0x7f1ba4004810 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1bc11ff700 1 -- 172.21.15.31:0/546872949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bbc15add0 con 0x7f1ba4004810 2023-10-06T05:18:34.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.116+0000 7f1ba2ffd700 1 -- 172.21.15.31:0/546872949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00fc90 con 0x7f1ba4004810 2023-10-06T05:18:34.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.117+0000 7f1bc11ff700 1 -- 172.21.15.31:0/546872949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc150d30 con 0x7f1ba4004810 2023-10-06T05:18:34.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.118+0000 7f1bc11ff700 1 -- 172.21.15.31:0/546872949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bbc15b2f0 con 0x7f1ba4004810 2023-10-06T05:18:34.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.118+0000 7f1ba2ffd700 1 -- 172.21.15.31:0/546872949 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f1bac01b070 con 0x7f1ba4004810 2023-10-06T05:18:34.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.119+0000 7f1ba2ffd700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ba80670e0 0x7f1ba8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.119+0000 7f1ba2ffd700 1 -- 172.21.15.31:0/546872949 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1bac019070 con 0x7f1ba4004810 2023-10-06T05:18:34.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.119+0000 7f1bba59c700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ba80670e0 0x7f1ba8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.120+0000 7f1bba59c700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ba80670e0 0x7f1ba8069590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0cc880 tx=0x7f1bb0019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.124+0000 7f1ba2ffd700 1 -- 172.21.15.31:0/546872949 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1bbc15b2f0 con 0x7f1ba4004810 2023-10-06T05:18:34.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.408+0000 7f1bc11ff700 1 -- 172.21.15.31:0/546872949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1bbc15b2f0 con 0x7f1ba4004810 2023-10-06T05:18:34.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.409+0000 7f1ba2ffd700 1 -- 172.21.15.31:0/546872949 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1bbc15b2f0 con 0x7f1ba4004810 2023-10-06T05:18:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:34.412 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:18:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:18:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:18:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:18:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:18:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:18:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:18:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:18:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:18:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:18:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:18:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:18:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:18:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:18:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:18:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:18:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:18:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:18:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:18:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:18:34.428 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:18:34.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 -- 172.21.15.31:0/546872949 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ba80670e0 msgr2=0x7f1ba8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ba80670e0 0x7f1ba8069590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0cc880 tx=0x7f1bb0019000 comp rx=0 tx=0).stop 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 -- 172.21.15.31:0/546872949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 msgr2=0x7f1bbc150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 0x7f1bbc150270 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0073c0 tx=0x7f1bac001e90 comp rx=0 tx=0).stop 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 -- 172.21.15.31:0/546872949 shutdown_connections 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1bbc159e10 0x7f1bbc0d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ba80670e0 0x7f1ba8069590 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ba4004810 0x7f1bbc150270 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 --2- 172.21.15.31:0/546872949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc15b5d0 0x7f1bbc1598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 -- 172.21.15.31:0/546872949 >> 172.21.15.31:0/546872949 conn(0x7f1bbc06ec90 msgr2=0x7f1bbc0cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 -- 172.21.15.31:0/546872949 shutdown_connections 2023-10-06T05:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.412+0000 7f1bc11ff700 1 -- 172.21.15.31:0/546872949 wait complete. 2023-10-06T05:18:34.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:34.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.627+0000 7fd91f5ff700 1 Processor -- start 2023-10-06T05:18:34.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.627+0000 7fd91f5ff700 1 -- start start 2023-10-06T05:18:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91f5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd91813a5b0 0x7fd91806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91f5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd91806d980 0x7fd918069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91f5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91806ddb0 con 0x7fd91813a1e0 2023-10-06T05:18:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91f5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9180c71b0 con 0x7fd91813a5b0 2023-10-06T05:18:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91f5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180c7490 con 0x7fd91806d980 2023-10-06T05:18:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91cb9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd91813a5b0 0x7fd91806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91cb9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd91813a5b0 0x7fd91806d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43490/0 (socket says 172.21.15.31:43490) 2023-10-06T05:18:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91cb9a700 1 -- 172.21.15.31:0/419631021 learned_addr learned my addr 172.21.15.31:0/419631021 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91d39b700 1 --1- 172.21.15.31:0/419631021 >> v1:172.21.15.31:6789/0 conn(0x7fd91813a1e0 0x7fd9180c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57574/0 (socket says 172.21.15.31:57574) 2023-10-06T05:18:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91db9c700 1 --2- 172.21.15.31:0/419631021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd91806d980 0x7fd918069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91cb9a700 1 -- 172.21.15.31:0/419631021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd91806d980 msgr2=0x7fd918069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91cb9a700 1 --2- 172.21.15.31:0/419631021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd91806d980 0x7fd918069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91cb9a700 1 -- 172.21.15.31:0/419631021 >> v1:172.21.15.31:6789/0 conn(0x7fd91813a1e0 legacy=0x7fd9180c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91cb9a700 1 -- 172.21.15.31:0/419631021 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9180c7770 con 0x7fd91813a5b0 2023-10-06T05:18:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd90ffff700 1 -- 172.21.15.31:0/419631021 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256919903 0 0) 0x7fd91806ddb0 con 0x7fd91813a1e0 2023-10-06T05:18:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91cb9a700 1 --2- 172.21.15.31:0/419631021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91813a5b0 0x7fd91806d440 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd9080099b0 tx=0x7fd908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae57d21b55ff5aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd90ffff700 1 -- 172.21.15.31:0/419631021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90800b860 con 0x7fd91813a5b0 2023-10-06T05:18:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd90ffff700 1 -- 172.21.15.31:0/419631021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90800b9c0 con 0x7fd91813a5b0 2023-10-06T05:18:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd90ffff700 1 -- 172.21.15.31:0/419631021 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90800bd60 con 0x7fd91813a5b0 2023-10-06T05:18:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91f5ff700 1 -- 172.21.15.31:0/419631021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91813a5b0 msgr2=0x7fd91806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.629+0000 7fd91f5ff700 1 --2- 172.21.15.31:0/419631021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91813a5b0 0x7fd91806d440 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd9080099b0 tx=0x7fd908009250 comp rx=0 tx=0).stop 2023-10-06T05:18:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.630+0000 7fd91f5ff700 1 -- 172.21.15.31:0/419631021 shutdown_connections 2023-10-06T05:18:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.630+0000 7fd91f5ff700 1 --2- 172.21.15.31:0/419631021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd91806d980 0x7fd918069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.630+0000 7fd91f5ff700 1 --2- 172.21.15.31:0/419631021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91813a5b0 0x7fd91806d440 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.630+0000 7fd91f5ff700 1 -- 172.21.15.31:0/419631021 >> 172.21.15.31:0/419631021 conn(0x7fd9180cc0b0 msgr2=0x7fd9180cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.630+0000 7fd91f5ff700 1 -- 172.21.15.31:0/419631021 shutdown_connections 2023-10-06T05:18:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.630+0000 7fd91f5ff700 1 -- 172.21.15.31:0/419631021 wait complete. 2023-10-06T05:18:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.631+0000 7fd91f5ff700 1 Processor -- start 2023-10-06T05:18:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.631+0000 7fd91f5ff700 1 -- start start 2023-10-06T05:18:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.632+0000 7fd91f5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91806d980 0x7fd918154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.632+0000 7fd91f5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd91813a1e0 0x7fd91814f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.632+0000 7fd91f5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd91813a5b0 0x7fd918150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.632+0000 7fd91f5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180cb060 con 0x7fd91806d980 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.632+0000 7fd91f5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91814f780 con 0x7fd91813a5b0 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.632+0000 7fd91f5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180ca730 con 0x7fd91813a1e0 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.632+0000 7fd91d39b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91806d980 0x7fd918154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.632+0000 7fd91d39b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91806d980 0x7fd918154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43498/0 (socket says 172.21.15.31:43498) 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd91d39b700 1 -- 172.21.15.31:0/150603965 learned_addr learned my addr 172.21.15.31:0/150603965 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd91d39b700 1 -- 172.21.15.31:0/150603965 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd91813a1e0 msgr2=0x7fd91814f0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd91d39b700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd91813a1e0 0x7fd91814f0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd91d39b700 1 -- 172.21.15.31:0/150603965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd91813a5b0 msgr2=0x7fd918150d30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd91d39b700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd91813a5b0 0x7fd918150d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd91d39b700 1 -- 172.21.15.31:0/150603965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080094f0 con 0x7fd91806d980 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd91d39b700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91806d980 0x7fd918154df0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd91400c810 tx=0x7fd914007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd90d7fa700 1 -- 172.21.15.31:0/150603965 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914007300 con 0x7fd91806d980 2023-10-06T05:18:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd90d7fa700 1 -- 172.21.15.31:0/150603965 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd914007460 con 0x7fd91806d980 2023-10-06T05:18:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd90d7fa700 1 -- 172.21.15.31:0/150603965 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9140057f0 con 0x7fd91806d980 2023-10-06T05:18:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.633+0000 7fd91f5ff700 1 -- 172.21.15.31:0/150603965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9180caa10 con 0x7fd91806d980 2023-10-06T05:18:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.634+0000 7fd91f5ff700 1 -- 172.21.15.31:0/150603965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9180cb4f0 con 0x7fd91806d980 2023-10-06T05:18:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.635+0000 7fd91f5ff700 1 -- 172.21.15.31:0/150603965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91804c380 con 0x7fd91806d980 2023-10-06T05:18:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.635+0000 7fd90d7fa700 1 -- 172.21.15.31:0/150603965 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd914005bc0 con 0x7fd91806d980 2023-10-06T05:18:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.636+0000 7fd90d7fa700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8fc0670e0 0x7fd8fc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.636+0000 7fd90d7fa700 1 -- 172.21.15.31:0/150603965 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd914084270 con 0x7fd91806d980 2023-10-06T05:18:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.637+0000 7fd91cb9a700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8fc0670e0 0x7fd8fc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.637+0000 7fd91cb9a700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8fc0670e0 0x7fd8fc069590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd91813dba0 tx=0x7fd908017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.641+0000 7fd90d7fa700 1 -- 172.21.15.31:0/150603965 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd914051430 con 0x7fd91806d980 2023-10-06T05:18:34.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.845+0000 7fd91f5ff700 1 -- 172.21.15.31:0/150603965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9180c7a50 con 0x7fd91806d980 2023-10-06T05:18:34.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.846+0000 7fd90d7fa700 1 -- 172.21.15.31:0/150603965 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13698 (secure 0 0 0) 0x7fd914054a50 con 0x7fd91806d980 2023-10-06T05:18:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:34.849 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:33.879338+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 -- 172.21.15.31:0/150603965 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8fc0670e0 msgr2=0x7fd8fc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8fc0670e0 0x7fd8fc069590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd91813dba0 tx=0x7fd908017000 comp rx=0 tx=0).stop 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 -- 172.21.15.31:0/150603965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91806d980 msgr2=0x7fd918154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91806d980 0x7fd918154df0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd91400c810 tx=0x7fd914007d20 comp rx=0 tx=0).stop 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 -- 172.21.15.31:0/150603965 shutdown_connections 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd91813a5b0 0x7fd918150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8fc0670e0 0x7fd8fc069590 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd91806d980 0x7fd918154df0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 --2- 172.21.15.31:0/150603965 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd91813a1e0 0x7fd91814f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 -- 172.21.15.31:0/150603965 >> 172.21.15.31:0/150603965 conn(0x7fd9180cc0b0 msgr2=0x7fd9180b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 -- 172.21.15.31:0/150603965 shutdown_connections 2023-10-06T05:18:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:34.849+0000 7fd91f5ff700 1 -- 172.21.15.31:0/150603965 wait complete. 2023-10-06T05:18:34.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:18:34.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:18:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.067+0000 7fa2289ff700 1 Processor -- start 2023-10-06T05:18:35.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.067+0000 7fa2289ff700 1 -- start start 2023-10-06T05:18:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.068+0000 7fa2289ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa224159560 0x7fa2240cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.068+0000 7fa2289ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2240b2b80 0x7fa2240cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.068+0000 7fa2289ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22415d2d0 con 0x7fa224159930 2023-10-06T05:18:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.068+0000 7fa2289ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22415d5b0 con 0x7fa224159560 2023-10-06T05:18:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.068+0000 7fa2289ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22415d890 con 0x7fa2240b2b80 2023-10-06T05:18:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.068+0000 7fa22259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa224159560 0x7fa2240cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.068+0000 7fa22259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa224159560 0x7fa2240cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43504/0 (socket says 172.21.15.31:43504) 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.068+0000 7fa22259c700 1 -- 172.21.15.31:0/2232366398 learned_addr learned my addr 172.21.15.31:0/2232366398 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.068+0000 7fa222d9d700 1 --2- 172.21.15.31:0/2232366398 >> v2:172.21.15.62:3300/0 conn(0x7fa2240b2b80 0x7fa2240cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.069+0000 7fa22259c700 1 -- 172.21.15.31:0/2232366398 >> v2:172.21.15.62:3300/0 conn(0x7fa2240b2b80 msgr2=0x7fa2240cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.069+0000 7fa22259c700 1 --2- 172.21.15.31:0/2232366398 >> v2:172.21.15.62:3300/0 conn(0x7fa2240b2b80 0x7fa2240cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.069+0000 7fa22259c700 1 -- 172.21.15.31:0/2232366398 >> v1:172.21.15.62:6789/0 conn(0x7fa224159930 legacy=0x7fa2240b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.069+0000 7fa22259c700 1 -- 172.21.15.31:0/2232366398 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22415db70 con 0x7fa224159560 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.069+0000 7fa22259c700 1 --2- 172.21.15.31:0/2232366398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159560 0x7fa2240cce70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa2240b1740 tx=0x7fa214009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad193e55f44842f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.069+0000 7fa22159a700 1 -- 172.21.15.31:0/2232366398 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400b860 con 0x7fa224159560 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.069+0000 7fa22159a700 1 -- 172.21.15.31:0/2232366398 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21400b9c0 con 0x7fa224159560 2023-10-06T05:18:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.069+0000 7fa22159a700 1 -- 172.21.15.31:0/2232366398 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400bd60 con 0x7fa224159560 2023-10-06T05:18:35.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.070+0000 7fa2289ff700 1 -- 172.21.15.31:0/2232366398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159560 msgr2=0x7fa2240cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.070+0000 7fa2289ff700 1 --2- 172.21.15.31:0/2232366398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159560 0x7fa2240cce70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa2240b1740 tx=0x7fa214009250 comp rx=0 tx=0).stop 2023-10-06T05:18:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.070+0000 7fa2289ff700 1 -- 172.21.15.31:0/2232366398 shutdown_connections 2023-10-06T05:18:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.070+0000 7fa2289ff700 1 --2- 172.21.15.31:0/2232366398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159560 0x7fa2240cce70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.070+0000 7fa2289ff700 1 --2- 172.21.15.31:0/2232366398 >> v2:172.21.15.62:3300/0 conn(0x7fa2240b2b80 0x7fa2240cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.070+0000 7fa2289ff700 1 -- 172.21.15.31:0/2232366398 >> 172.21.15.31:0/2232366398 conn(0x7fa22406ec90 msgr2=0x7fa22406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.070+0000 7fa2289ff700 1 -- 172.21.15.31:0/2232366398 shutdown_connections 2023-10-06T05:18:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.070+0000 7fa2289ff700 1 -- 172.21.15.31:0/2232366398 wait complete. 2023-10-06T05:18:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.071+0000 7fa2289ff700 1 Processor -- start 2023-10-06T05:18:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.072+0000 7fa2289ff700 1 -- start start 2023-10-06T05:18:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.072+0000 7fa2289ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2240b2b80 0x7fa2240b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.072+0000 7fa2289ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224159560 0x7fa2241451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.072+0000 7fa2289ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159930 0x7fa22416a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.072+0000 7fa2289ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240c6e40 con 0x7fa224159930 2023-10-06T05:18:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.072+0000 7fa2289ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa224169880 con 0x7fa224159560 2023-10-06T05:18:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa2289ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2240c4180 con 0x7fa2240b2b80 2023-10-06T05:18:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa222d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159930 0x7fa22416a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa222d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159930 0x7fa22416a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43506/0 (socket says 172.21.15.31:43506) 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa222d9d700 1 -- 172.21.15.31:0/2472483426 learned_addr learned my addr 172.21.15.31:0/2472483426 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa22259c700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2240b2b80 0x7fa2240b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa221d9b700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224159560 0x7fa2241451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa222d9d700 1 -- 172.21.15.31:0/2472483426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2240b2b80 msgr2=0x7fa2240b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa222d9d700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2240b2b80 0x7fa2240b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa222d9d700 1 -- 172.21.15.31:0/2472483426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224159560 msgr2=0x7fa2241451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa222d9d700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224159560 0x7fa2241451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.073+0000 7fa222d9d700 1 -- 172.21.15.31:0/2472483426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2140094f0 con 0x7fa224159930 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.074+0000 7fa222d9d700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159930 0x7fa22416a890 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009fa0 tx=0x7fa21c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.074+0000 7fa2127fc700 1 -- 172.21.15.31:0/2472483426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c00ebc0 con 0x7fa224159930 2023-10-06T05:18:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.074+0000 7fa2127fc700 1 -- 172.21.15.31:0/2472483426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c00ed20 con 0x7fa224159930 2023-10-06T05:18:35.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.074+0000 7fa2289ff700 1 -- 172.21.15.31:0/2472483426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2240c4460 con 0x7fa224159930 2023-10-06T05:18:35.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.074+0000 7fa2127fc700 1 -- 172.21.15.31:0/2472483426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c014670 con 0x7fa224159930 2023-10-06T05:18:35.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.074+0000 7fa2289ff700 1 -- 172.21.15.31:0/2472483426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2240c4a20 con 0x7fa224159930 2023-10-06T05:18:35.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.075+0000 7fa2289ff700 1 -- 172.21.15.31:0/2472483426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2240c4be0 con 0x7fa224159930 2023-10-06T05:18:35.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.076+0000 7fa2127fc700 1 -- 172.21.15.31:0/2472483426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fa21c0149e0 con 0x7fa224159930 2023-10-06T05:18:35.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.077+0000 7fa2127fc700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa2040670e0 0x7fa204069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.077+0000 7fa2127fc700 1 -- 172.21.15.31:0/2472483426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa21c084ce0 con 0x7fa224159930 2023-10-06T05:18:35.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.077+0000 7fa22259c700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa2040670e0 0x7fa204069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.078+0000 7fa22259c700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa2040670e0 0x7fa204069590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa2240b1740 tx=0x7fa214017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:35.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.081+0000 7fa2127fc700 1 -- 172.21.15.31:0/2472483426 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa21c051e20 con 0x7fa224159930 2023-10-06T05:18:35.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.280+0000 7fa2289ff700 1 -- 172.21.15.31:0/2472483426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa22404c380 con 0x7fa224159930 2023-10-06T05:18:35.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.281+0000 7fa2127fc700 1 -- 172.21.15.31:0/2472483426 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa21c055440 con 0x7fa224159930 2023-10-06T05:18:35.284 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:18:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 -- 172.21.15.31:0/2472483426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa2040670e0 msgr2=0x7fa204069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa2040670e0 0x7fa204069590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa2240b1740 tx=0x7fa214017000 comp rx=0 tx=0).stop 2023-10-06T05:18:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 -- 172.21.15.31:0/2472483426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159930 msgr2=0x7fa22416a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159930 0x7fa22416a890 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009fa0 tx=0x7fa21c009900 comp rx=0 tx=0).stop 2023-10-06T05:18:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 -- 172.21.15.31:0/2472483426 shutdown_connections 2023-10-06T05:18:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa2040670e0 0x7fa204069590 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa224159930 0x7fa22416a890 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2240b2b80 0x7fa2240b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 --2- 172.21.15.31:0/2472483426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa224159560 0x7fa2241451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 -- 172.21.15.31:0/2472483426 >> 172.21.15.31:0/2472483426 conn(0x7fa22406ec90 msgr2=0x7fa2241586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:35.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 -- 172.21.15.31:0/2472483426 shutdown_connections 2023-10-06T05:18:35.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.283+0000 7fa2289ff700 1 -- 172.21.15.31:0/2472483426 wait complete. 2023-10-06T05:18:35.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:18:35.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.501+0000 7f7c015ff700 1 Processor -- start 2023-10-06T05:18:35.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.501+0000 7f7c015ff700 1 -- start start 2023-10-06T05:18:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.502+0000 7f7c015ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7bfc15ca40 0x7f7bfc068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.502+0000 7f7c015ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bfc06a300 0x7f7bfc0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.502+0000 7f7c015ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc068da0 con 0x7f7bfc0c8d70 2023-10-06T05:18:35.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.502+0000 7f7c015ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bfc0cabc0 con 0x7f7bfc15ca40 2023-10-06T05:18:35.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.502+0000 7f7c015ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc068bb0 con 0x7f7bfc06a300 2023-10-06T05:18:35.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bfb7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7bfc0c8d70 0x7f7bfc06cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57582/0 (socket says 172.21.15.31:57582) 2023-10-06T05:18:35.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bfb7fe700 1 -- 172.21.15.31:0/3433410390 learned_addr learned my addr 172.21.15.31:0/3433410390 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:35.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bfa7fc700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bfc06a300 0x7f7bfc0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bfaffd700 1 --2- 172.21.15.31:0/3433410390 >> v2:172.21.15.62:3300/0 conn(0x7f7bfc15ca40 0x7f7bfc068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307745555 0 0) 0x7f7bfc068da0 con 0x7f7bfc0c8d70 2023-10-06T05:18:35.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7be4003570 con 0x7f7bfc0c8d70 2023-10-06T05:18:35.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3711592394 0 0) 0x7f7be4003570 con 0x7f7bfc0c8d70 2023-10-06T05:18:35.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bfc06a300 msgr2=0x7f7bfc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:35.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bfc06a300 0x7f7bfc0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 >> v2:172.21.15.62:3300/0 conn(0x7f7bfc15ca40 msgr2=0x7f7bfc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:35.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 --2- 172.21.15.31:0/3433410390 >> v2:172.21.15.62:3300/0 conn(0x7f7bfc15ca40 0x7f7bfc068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc069080 con 0x7f7bfc0c8d70 2023-10-06T05:18:35.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f7bf40017f0 con 0x7f7bfc0c8d70 2023-10-06T05:18:35.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 >> v1:172.21.15.31:6789/0 conn(0x7f7bfc0c8d70 legacy=0x7f7bfc06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7be4003a60 0x7f7be4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7be4004810 0x7f7be4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7be40051c0 0x7f7be400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc0cabc0 con 0x7f7be40051c0 2023-10-06T05:18:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be4004680 con 0x7f7be4004810 2023-10-06T05:18:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be400c020 con 0x7f7be4003a60 2023-10-06T05:18:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bfa7fc700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bfc06a300 0x7f7bfc0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.503+0000 7f7bfaffd700 1 --2- 172.21.15.31:0/3433410390 >> v2:172.21.15.62:3300/0 conn(0x7f7bfc15ca40 0x7f7bfc068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.504+0000 7f7bfb7fe700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7be40051c0 0x7f7be400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.504+0000 7f7bfa7fc700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7be4004810 0x7f7be4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.504+0000 7f7bfaffd700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7be4003a60 0x7f7be4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.504+0000 7f7bfb7fe700 1 -- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7be4003a60 msgr2=0x7f7be4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.504+0000 7f7bfb7fe700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7be4003a60 0x7f7be4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.504+0000 7f7bfb7fe700 1 -- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7be4004810 msgr2=0x7f7be4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.504+0000 7f7bfb7fe700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7be4004810 0x7f7be4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.504+0000 7f7bfb7fe700 1 -- 172.21.15.31:0/3433410390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be4003570 con 0x7f7be40051c0 2023-10-06T05:18:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.504+0000 7f7bfb7fe700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7be40051c0 0x7f7be400ba60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400a970 tx=0x7f7bfc06a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bec2c61ecfd65cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.505+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40070d0 con 0x7f7be40051c0 2023-10-06T05:18:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.505+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf4007270 con 0x7f7be40051c0 2023-10-06T05:18:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.505+0000 7f7bf9ffb700 1 -- 172.21.15.31:0/3433410390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4007630 con 0x7f7be40051c0 2023-10-06T05:18:35.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.505+0000 7f7c015ff700 1 -- 172.21.15.31:0/3433410390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7be40051c0 msgr2=0x7f7be400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.505+0000 7f7c015ff700 1 --2- 172.21.15.31:0/3433410390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7be40051c0 0x7f7be400ba60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400a970 tx=0x7f7bfc06a770 comp rx=0 tx=0).stop 2023-10-06T05:18:35.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.505+0000 7f7bfb7fe700 1 -- 172.21.15.31:0/3433410390 reap_dead start 2023-10-06T05:18:35.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.505+0000 7f7c015ff700 1 -- 172.21.15.31:0/3433410390 shutdown_connections 2023-10-06T05:18:35.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.505+0000 7f7c015ff700 1 -- 172.21.15.31:0/3433410390 >> 172.21.15.31:0/3433410390 conn(0x7f7bfc0ce290 msgr2=0x7f7bfc0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:35.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.505+0000 7f7c015ff700 1 -- 172.21.15.31:0/3433410390 shutdown_connections 2023-10-06T05:18:35.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.505+0000 7f7c015ff700 1 -- 172.21.15.31:0/3433410390 wait complete. 2023-10-06T05:18:35.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.507+0000 7f7c015ff700 1 Processor -- start 2023-10-06T05:18:35.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.507+0000 7f7c015ff700 1 -- start start 2023-10-06T05:18:35.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7c015ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bfc1696a0 0x7f7bfc169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7c015ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bfc0cc8d0 0x7f7bfc0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7c015ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bfc15b900 0x7f7bfc15bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7c015ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc0c6a50 con 0x7f7bfc15b900 2023-10-06T05:18:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7c015ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc15c2b0 con 0x7f7bfc0cc8d0 2023-10-06T05:18:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7c015ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc0c5dd0 con 0x7f7bfc1696a0 2023-10-06T05:18:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bfc15b900 0x7f7bfc15bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bfc15b900 0x7f7bfc15bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43534/0 (socket says 172.21.15.31:43534) 2023-10-06T05:18:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfb7fe700 1 -- 172.21.15.31:0/827318472 learned_addr learned my addr 172.21.15.31:0/827318472 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfa7fc700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bfc0cc8d0 0x7f7bfc0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfaffd700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bfc1696a0 0x7f7bfc169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfb7fe700 1 -- 172.21.15.31:0/827318472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bfc1696a0 msgr2=0x7f7bfc169a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfb7fe700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bfc1696a0 0x7f7bfc169a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfb7fe700 1 -- 172.21.15.31:0/827318472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bfc0cc8d0 msgr2=0x7f7bfc0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfb7fe700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bfc0cc8d0 0x7f7bfc0ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfb7fe700 1 -- 172.21.15.31:0/827318472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc0c60b0 con 0x7f7bfc15b900 2023-10-06T05:18:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.508+0000 7f7bfa7fc700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bfc0cc8d0 0x7f7bfc0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.509+0000 7f7bfb7fe700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bfc15b900 0x7f7bfc15bd70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400a430 tx=0x7f7bf4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.509+0000 7f7be2ffd700 1 -- 172.21.15.31:0/827318472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf401e640 con 0x7f7bfc15b900 2023-10-06T05:18:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.509+0000 7f7be2ffd700 1 -- 172.21.15.31:0/827318472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf401e7e0 con 0x7f7bfc15b900 2023-10-06T05:18:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.509+0000 7f7be2ffd700 1 -- 172.21.15.31:0/827318472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf401eba0 con 0x7f7bfc15b900 2023-10-06T05:18:35.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.509+0000 7f7c015ff700 1 -- 172.21.15.31:0/827318472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc0c6390 con 0x7f7bfc15b900 2023-10-06T05:18:35.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.509+0000 7f7c015ff700 1 -- 172.21.15.31:0/827318472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc16c4f0 con 0x7f7bfc15b900 2023-10-06T05:18:35.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.510+0000 7f7c015ff700 1 -- 172.21.15.31:0/827318472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc0c6b90 con 0x7f7bfc15b900 2023-10-06T05:18:35.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.511+0000 7f7be2ffd700 1 -- 172.21.15.31:0/827318472 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7bf4029460 con 0x7f7bfc15b900 2023-10-06T05:18:35.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.512+0000 7f7be2ffd700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7be80670e0 0x7f7be8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.512+0000 7f7be2ffd700 1 -- 172.21.15.31:0/827318472 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7bf4086d60 con 0x7f7bfc15b900 2023-10-06T05:18:35.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.512+0000 7f7bfaffd700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7be80670e0 0x7f7be8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.513+0000 7f7bfaffd700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7be80670e0 0x7f7be8069590 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0093f0 tx=0x7f7bec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:35.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.516+0000 7f7be2ffd700 1 -- 172.21.15.31:0/827318472 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bfc0c6b90 con 0x7f7bfc15b900 2023-10-06T05:18:35.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.721+0000 7f7c015ff700 1 -- 172.21.15.31:0/827318472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bfc0c6b90 con 0x7f7bfc15b900 2023-10-06T05:18:35.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.721+0000 7f7be2ffd700 1 -- 172.21.15.31:0/827318472 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bfc0c6b90 con 0x7f7bfc15b900 2023-10-06T05:18:35.723 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:18:35.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 -- 172.21.15.31:0/827318472 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7be80670e0 msgr2=0x7f7be8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7be80670e0 0x7f7be8069590 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0093f0 tx=0x7f7bec008040 comp rx=0 tx=0).stop 2023-10-06T05:18:35.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 -- 172.21.15.31:0/827318472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bfc15b900 msgr2=0x7f7bfc15bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bfc15b900 0x7f7bfc15bd70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400a430 tx=0x7f7bf4016000 comp rx=0 tx=0).stop 2023-10-06T05:18:35.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 -- 172.21.15.31:0/827318472 shutdown_connections 2023-10-06T05:18:35.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7be80670e0 0x7f7be8069590 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bfc15b900 0x7f7bfc15bd70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bfc1696a0 0x7f7bfc169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 --2- 172.21.15.31:0/827318472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bfc0cc8d0 0x7f7bfc0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 -- 172.21.15.31:0/827318472 >> 172.21.15.31:0/827318472 conn(0x7f7bfc0ce290 msgr2=0x7f7bfc159160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:35.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 -- 172.21.15.31:0/827318472 shutdown_connections 2023-10-06T05:18:35.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.724+0000 7f7c015ff700 1 -- 172.21.15.31:0/827318472 wait complete. 2023-10-06T05:18:35.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:18:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.938+0000 7f2a8dfff700 1 Processor -- start 2023-10-06T05:18:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.938+0000 7f2a8dfff700 1 -- start start 2023-10-06T05:18:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a8dfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a88068d80 0x7f2a88159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a8dfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2a880cc0d0 0x7f2a880c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a8dfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a880c95b0 con 0x7f2a881595a0 2023-10-06T05:18:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a8dfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a880c9890 con 0x7f2a880cc0d0 2023-10-06T05:18:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a8dfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880c9b70 con 0x7f2a88068d80 2023-10-06T05:18:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a86ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a88068d80 0x7f2a88159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a87fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2a880cc0d0 0x7f2a880c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a86ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a88068d80 0x7f2a88159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:50814/0 (socket says 172.21.15.31:50814) 2023-10-06T05:18:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a86ffd700 1 -- 172.21.15.31:0/2145791139 learned_addr learned my addr 172.21.15.31:0/2145791139 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:35.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a87fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2a880cc0d0 0x7f2a880c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33892/0 (socket says 172.21.15.31:33892) 2023-10-06T05:18:35.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.940+0000 7f2a877fe700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2a881595a0 0x7f2a88068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:55846/0 (socket says 172.21.15.31:55846) 2023-10-06T05:18:35.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.941+0000 7f2a86ffd700 1 -- 172.21.15.31:0/2145791139 >> v2:172.21.15.62:3300/0 conn(0x7f2a880cc0d0 msgr2=0x7f2a880c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:35.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.941+0000 7f2a86ffd700 1 --2- 172.21.15.31:0/2145791139 >> v2:172.21.15.62:3300/0 conn(0x7f2a880cc0d0 0x7f2a880c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.941+0000 7f2a86ffd700 1 -- 172.21.15.31:0/2145791139 >> v1:172.21.15.62:6789/0 conn(0x7f2a881595a0 legacy=0x7f2a88068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.941+0000 7f2a86ffd700 1 -- 172.21.15.31:0/2145791139 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a880ca130 con 0x7f2a88068d80 2023-10-06T05:18:35.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.941+0000 7f2a87fff700 1 --2- 172.21.15.31:0/2145791139 >> v2:172.21.15.62:3300/0 conn(0x7f2a880cc0d0 0x7f2a880c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:35.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.941+0000 7f2a86ffd700 1 --2- 172.21.15.31:0/2145791139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a88068d80 0x7f2a88159970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0099b0 tx=0x7f2a7c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb3af6166afc06f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:35.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.941+0000 7f2a867fc700 1 -- 172.21.15.31:0/2145791139 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c009ea0 con 0x7f2a88068d80 2023-10-06T05:18:35.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.942+0000 7f2a867fc700 1 -- 172.21.15.31:0/2145791139 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c00b860 con 0x7f2a88068d80 2023-10-06T05:18:35.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.942+0000 7f2a867fc700 1 -- 172.21.15.31:0/2145791139 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00bc00 con 0x7f2a88068d80 2023-10-06T05:18:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.942+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2145791139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a88068d80 msgr2=0x7f2a88159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.942+0000 7f2a8dfff700 1 --2- 172.21.15.31:0/2145791139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a88068d80 0x7f2a88159970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0099b0 tx=0x7f2a7c009250 comp rx=0 tx=0).stop 2023-10-06T05:18:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.942+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2145791139 shutdown_connections 2023-10-06T05:18:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.942+0000 7f2a8dfff700 1 --2- 172.21.15.31:0/2145791139 >> v2:172.21.15.62:3300/0 conn(0x7f2a880cc0d0 0x7f2a880c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.942+0000 7f2a8dfff700 1 --2- 172.21.15.31:0/2145791139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a88068d80 0x7f2a88159970 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.942+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2145791139 >> 172.21.15.31:0/2145791139 conn(0x7f2a8806b620 msgr2=0x7f2a8806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.942+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2145791139 shutdown_connections 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.942+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2145791139 wait complete. 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.944+0000 7f2a8dfff700 1 Processor -- start 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.944+0000 7f2a8dfff700 1 -- start start 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a8dfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a88068d80 0x7f2a8815ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a8dfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a880cc0d0 0x7f2a8815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a8dfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a881595a0 0x7f2a88144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a8dfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815e8e0 con 0x7f2a88068d80 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a8dfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815da30 con 0x7f2a880cc0d0 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a8dfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815dcd0 con 0x7f2a881595a0 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a88068d80 0x7f2a8815ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a88068d80 0x7f2a8815ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43546/0 (socket says 172.21.15.31:43546) 2023-10-06T05:18:35.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a877fe700 1 -- 172.21.15.31:0/2939621257 learned_addr learned my addr 172.21.15.31:0/2939621257 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a86ffd700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a880cc0d0 0x7f2a8815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.945+0000 7f2a87fff700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a881595a0 0x7f2a88144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a877fe700 1 -- 172.21.15.31:0/2939621257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a881595a0 msgr2=0x7f2a88144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a877fe700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a881595a0 0x7f2a88144d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a877fe700 1 -- 172.21.15.31:0/2939621257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a880cc0d0 msgr2=0x7f2a8815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a877fe700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a880cc0d0 0x7f2a8815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a877fe700 1 -- 172.21.15.31:0/2939621257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a80009a20 con 0x7f2a88068d80 2023-10-06T05:18:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a877fe700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a88068d80 0x7f2a8815ed70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800c7c0 tx=0x7f2a78007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/2939621257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78005a70 con 0x7f2a88068d80 2023-10-06T05:18:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2939621257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c0094f0 con 0x7f2a88068d80 2023-10-06T05:18:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/2939621257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a78005bd0 con 0x7f2a88068d80 2023-10-06T05:18:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/2939621257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a780070d0 con 0x7f2a88068d80 2023-10-06T05:18:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.946+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2939621257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8815e290 con 0x7f2a88068d80 2023-10-06T05:18:35.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.947+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2939621257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8804c380 con 0x7f2a88068d80 2023-10-06T05:18:35.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.948+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/2939621257 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2a78007480 con 0x7f2a88068d80 2023-10-06T05:18:35.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.949+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a740670e0 0x7f2a74069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:35.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.949+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/2939621257 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2a78083bd0 con 0x7f2a88068d80 2023-10-06T05:18:35.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.949+0000 7f2a86ffd700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a740670e0 0x7f2a74069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:35.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.950+0000 7f2a86ffd700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a740670e0 0x7f2a74069590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0099b0 tx=0x7f2a7c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:35.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:35.953+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/2939621257 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a78050dd0 con 0x7f2a88068d80 2023-10-06T05:18:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.158+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2939621257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2a88160150 con 0x7f2a88068d80 2023-10-06T05:18:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.158+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/2939621257 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2a78017070 con 0x7f2a88068d80 2023-10-06T05:18:36.160 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:36.160 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:18:36.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:36.160 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:18:36.160 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:18:36.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:18:36.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:36.161 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:18:36.161 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:18:36.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:36.161 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:18:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.160+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2939621257 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a740670e0 msgr2=0x7f2a74069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.160+0000 7f2a8dfff700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a740670e0 0x7f2a74069590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0099b0 tx=0x7f2a7c017000 comp rx=0 tx=0).stop 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.160+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2939621257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a88068d80 msgr2=0x7f2a8815ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.160+0000 7f2a8dfff700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a88068d80 0x7f2a8815ed70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800c7c0 tx=0x7f2a78007f70 comp rx=0 tx=0).stop 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.160+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2939621257 shutdown_connections 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.160+0000 7f2a8dfff700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a881595a0 0x7f2a88144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.160+0000 7f2a8dfff700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a740670e0 0x7f2a74069590 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.160+0000 7f2a8dfff700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a88068d80 0x7f2a8815ed70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.160+0000 7f2a8dfff700 1 --2- 172.21.15.31:0/2939621257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a880cc0d0 0x7f2a8815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.161+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2939621257 >> 172.21.15.31:0/2939621257 conn(0x7f2a8806b620 msgr2=0x7f2a8806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.161+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2939621257 shutdown_connections 2023-10-06T05:18:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.161+0000 7f2a8dfff700 1 -- 172.21.15.31:0/2939621257 wait complete. 2023-10-06T05:18:36.180 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:18:36.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:18:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.378+0000 7fd432dff700 1 Processor -- start 2023-10-06T05:18:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.378+0000 7fd432dff700 1 -- start start 2023-10-06T05:18:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.380+0000 7fd432dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c153950 0x7fd42c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.380+0000 7fd432dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd42c0cd5d0 0x7fd42c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.380+0000 7fd432dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42c15cfc0 con 0x7fd42c0cda40 2023-10-06T05:18:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.380+0000 7fd432dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42c15bb20 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.380+0000 7fd432dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c15af10 con 0x7fd42c153950 2023-10-06T05:18:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.380+0000 7fd430b9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c153950 0x7fd42c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.380+0000 7fd42bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd42c0cd5d0 0x7fd42c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.380+0000 7fd43139c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd42c0cda40 0x7fd42c159c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:55858/0 (socket says 172.21.15.31:55858) 2023-10-06T05:18:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.380+0000 7fd43139c700 1 -- 172.21.15.31:0/3275893447 learned_addr learned my addr 172.21.15.31:0/3275893447 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.381+0000 7fd42bfff700 1 -- 172.21.15.31:0/3275893447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c153950 msgr2=0x7fd42c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.381+0000 7fd42bfff700 1 --2- 172.21.15.31:0/3275893447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c153950 0x7fd42c153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.381+0000 7fd42bfff700 1 -- 172.21.15.31:0/3275893447 >> v1:172.21.15.62:6789/0 conn(0x7fd42c0cda40 legacy=0x7fd42c159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.381+0000 7fd42b7fe700 1 -- 172.21.15.31:0/3275893447 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158030637 0 0) 0x7fd42c15cfc0 con 0x7fd42c0cda40 2023-10-06T05:18:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.381+0000 7fd42bfff700 1 -- 172.21.15.31:0/3275893447 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c15b1f0 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.381+0000 7fd430b9b700 1 --2- 172.21.15.31:0/3275893447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c153950 0x7fd42c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:36.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.381+0000 7fd42bfff700 1 --2- 172.21.15.31:0/3275893447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 0x7fd42c15d210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd4180098e0 tx=0x7fd4180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56c2fea0dcab5d22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:36.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.381+0000 7fd42b7fe700 1 -- 172.21.15.31:0/3275893447 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41800b640 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.381+0000 7fd42b7fe700 1 -- 172.21.15.31:0/3275893447 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41800b7a0 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.381+0000 7fd42b7fe700 1 -- 172.21.15.31:0/3275893447 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41800bb40 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.382+0000 7fd432dff700 1 -- 172.21.15.31:0/3275893447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 msgr2=0x7fd42c15d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:36.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.382+0000 7fd432dff700 1 --2- 172.21.15.31:0/3275893447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 0x7fd42c15d210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd4180098e0 tx=0x7fd4180091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:36.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.382+0000 7fd432dff700 1 -- 172.21.15.31:0/3275893447 shutdown_connections 2023-10-06T05:18:36.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.382+0000 7fd432dff700 1 --2- 172.21.15.31:0/3275893447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c153950 0x7fd42c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:36.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.382+0000 7fd432dff700 1 --2- 172.21.15.31:0/3275893447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 0x7fd42c15d210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:36.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.382+0000 7fd432dff700 1 -- 172.21.15.31:0/3275893447 >> 172.21.15.31:0/3275893447 conn(0x7fd42c0681b0 msgr2=0x7fd42c0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:36.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.382+0000 7fd432dff700 1 -- 172.21.15.31:0/3275893447 shutdown_connections 2023-10-06T05:18:36.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.382+0000 7fd432dff700 1 -- 172.21.15.31:0/3275893447 wait complete. 2023-10-06T05:18:36.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.384+0000 7fd432dff700 1 Processor -- start 2023-10-06T05:18:36.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.384+0000 7fd432dff700 1 -- start start 2023-10-06T05:18:36.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd432dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 0x7fd42c150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:36.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd432dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c0cda40 0x7fd42c16a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:36.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd432dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd42c153950 0x7fd42c1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd432dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c16dbb0 con 0x7fd42c153950 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd432dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c16a7a0 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd432dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c16cfc0 con 0x7fd42c0cda40 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd43139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd42c153950 0x7fd42c1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd430b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 0x7fd42c150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd430b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 0x7fd42c150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:33916/0 (socket says 172.21.15.31:33916) 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd430b9b700 1 -- 172.21.15.31:0/3070405850 learned_addr learned my addr 172.21.15.31:0/3070405850 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd42bfff700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c0cda40 0x7fd42c16a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd430b9b700 1 -- 172.21.15.31:0/3070405850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c0cda40 msgr2=0x7fd42c16a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd430b9b700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c0cda40 0x7fd42c16a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd430b9b700 1 -- 172.21.15.31:0/3070405850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd42c153950 msgr2=0x7fd42c1501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:36.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd430b9b700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd42c153950 0x7fd42c1501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:36.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.385+0000 7fd430b9b700 1 -- 172.21.15.31:0/3070405850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4200093c0 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.386+0000 7fd43139c700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd42c153950 0x7fd42c1501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:36.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.386+0000 7fd430b9b700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 0x7fd42c150a40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd42000ec80 tx=0x7fd420009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:36.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.386+0000 7fd428ff9700 1 -- 172.21.15.31:0/3070405850 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd420005a00 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.386+0000 7fd428ff9700 1 -- 172.21.15.31:0/3070405850 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd420005b60 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.386+0000 7fd428ff9700 1 -- 172.21.15.31:0/3070405850 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42000c850 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.386+0000 7fd432dff700 1 -- 172.21.15.31:0/3070405850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd418009420 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.386+0000 7fd432dff700 1 -- 172.21.15.31:0/3070405850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c16d560 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.387+0000 7fd432dff700 1 -- 172.21.15.31:0/3070405850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42c04c380 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.388+0000 7fd428ff9700 1 -- 172.21.15.31:0/3070405850 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd42000cbc0 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.389+0000 7fd428ff9700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd4100670e0 0x7fd410069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:36.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.389+0000 7fd428ff9700 1 -- 172.21.15.31:0/3070405850 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd420084b40 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.390+0000 7fd42bfff700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd4100670e0 0x7fd410069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.390+0000 7fd42bfff700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd4100670e0 0x7fd410069590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd4180093f0 tx=0x7fd418017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.395+0000 7fd428ff9700 1 -- 172.21.15.31:0/3070405850 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd420051c80 con 0x7fd42c0cd5d0 2023-10-06T05:18:36.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:36.599+0000 7fd432dff700 1 -- 172.21.15.31:0/3070405850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd42c15b4d0 con 0x7fd42c0cd5d0 2023-10-06T05:18:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.435+0000 7fd428ff9700 1 -- 172.21.15.31:0/3070405850 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd4200552a0 con 0x7fd42c0cd5d0 2023-10-06T05:18:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 -- 172.21.15.31:0/3070405850 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd4100670e0 msgr2=0x7fd410069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd4100670e0 0x7fd410069590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd4180093f0 tx=0x7fd418017000 comp rx=0 tx=0).stop 2023-10-06T05:18:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 -- 172.21.15.31:0/3070405850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 msgr2=0x7fd42c150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 0x7fd42c150a40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd42000ec80 tx=0x7fd420009120 comp rx=0 tx=0).stop 2023-10-06T05:18:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 -- 172.21.15.31:0/3070405850 shutdown_connections 2023-10-06T05:18:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd4100670e0 0x7fd410069590 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd42c153950 0x7fd42c1501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c0cd5d0 0x7fd42c150a40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 --2- 172.21.15.31:0/3070405850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42c0cda40 0x7fd42c16a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 -- 172.21.15.31:0/3070405850 >> 172.21.15.31:0/3070405850 conn(0x7fd42c0681b0 msgr2=0x7fd42c152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:37.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 -- 172.21.15.31:0/3070405850 shutdown_connections 2023-10-06T05:18:37.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.440+0000 7fd432dff700 1 -- 172.21.15.31:0/3070405850 wait complete. 2023-10-06T05:18:37.445 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:18:37.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:18:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.671+0000 7fb2523ff700 1 Processor -- start 2023-10-06T05:18:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.671+0000 7fb2523ff700 1 -- start start 2023-10-06T05:18:37.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.672+0000 7fb2523ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb24c0c7190 0x7fb24c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:37.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.672+0000 7fb2523ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb24c0aef00 0x7fb24c0af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:37.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.672+0000 7fb2523ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24c068250 con 0x7fb24c0afcf0 2023-10-06T05:18:37.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.672+0000 7fb2523ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24c06ac80 con 0x7fb24c0aef00 2023-10-06T05:18:37.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.672+0000 7fb2523ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c067f20 con 0x7fb24c0c7190 2023-10-06T05:18:37.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.673+0000 7fb24b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb24c0aef00 0x7fb24c0af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:37.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.673+0000 7fb24b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb24c0aef00 0x7fb24c0af7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56772/0 (socket says 172.21.15.31:56772) 2023-10-06T05:18:37.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.673+0000 7fb24b7fe700 1 -- 172.21.15.31:0/2044122165 learned_addr learned my addr 172.21.15.31:0/2044122165 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:37.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.673+0000 7fb24bfff700 1 --2- 172.21.15.31:0/2044122165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb24c0c7190 0x7fb24c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:37.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.673+0000 7fb24b7fe700 1 -- 172.21.15.31:0/2044122165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb24c0c7190 msgr2=0x7fb24c0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:37.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.673+0000 7fb24b7fe700 1 --2- 172.21.15.31:0/2044122165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb24c0c7190 0x7fb24c0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:37.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.673+0000 7fb24b7fe700 1 -- 172.21.15.31:0/2044122165 >> v1:172.21.15.62:6789/0 conn(0x7fb24c0afcf0 legacy=0x7fb24c0686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:37.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.673+0000 7fb24b7fe700 1 -- 172.21.15.31:0/2044122165 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c06aed0 con 0x7fb24c0aef00 2023-10-06T05:18:37.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.673+0000 7fb24b7fe700 1 --2- 172.21.15.31:0/2044122165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0aef00 0x7fb24c0af7b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb2400099b0 tx=0x7fb240009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4291c5fdd6753ea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:37.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.673+0000 7fb24affd700 1 -- 172.21.15.31:0/2044122165 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24000b860 con 0x7fb24c0aef00 2023-10-06T05:18:37.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.674+0000 7fb24affd700 1 -- 172.21.15.31:0/2044122165 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24000b9c0 con 0x7fb24c0aef00 2023-10-06T05:18:37.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.674+0000 7fb24affd700 1 -- 172.21.15.31:0/2044122165 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24000bd60 con 0x7fb24c0aef00 2023-10-06T05:18:37.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.674+0000 7fb2523ff700 1 -- 172.21.15.31:0/2044122165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0aef00 msgr2=0x7fb24c0af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:37.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.674+0000 7fb2523ff700 1 --2- 172.21.15.31:0/2044122165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0aef00 0x7fb24c0af7b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb2400099b0 tx=0x7fb240009250 comp rx=0 tx=0).stop 2023-10-06T05:18:37.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.674+0000 7fb2523ff700 1 -- 172.21.15.31:0/2044122165 shutdown_connections 2023-10-06T05:18:37.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.674+0000 7fb2523ff700 1 --2- 172.21.15.31:0/2044122165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb24c0c7190 0x7fb24c0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:37.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.674+0000 7fb2523ff700 1 --2- 172.21.15.31:0/2044122165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0aef00 0x7fb24c0af7b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:37.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.674+0000 7fb2523ff700 1 -- 172.21.15.31:0/2044122165 >> 172.21.15.31:0/2044122165 conn(0x7fb24c0cd3d0 msgr2=0x7fb24c0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:37.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.674+0000 7fb2523ff700 1 -- 172.21.15.31:0/2044122165 shutdown_connections 2023-10-06T05:18:37.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.674+0000 7fb2523ff700 1 -- 172.21.15.31:0/2044122165 wait complete. 2023-10-06T05:18:37.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.676+0000 7fb2523ff700 1 Processor -- start 2023-10-06T05:18:37.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.676+0000 7fb2523ff700 1 -- start start 2023-10-06T05:18:37.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb2523ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24c0aef00 0x7fb24c14f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:37.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb2523ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb24c0afcf0 0x7fb24c169e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:37.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb2523ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0c7190 0x7fb24c16a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:37.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb2523ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c152ce0 con 0x7fb24c0c7190 2023-10-06T05:18:37.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb2523ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c14f410 con 0x7fb24c0aef00 2023-10-06T05:18:37.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb2523ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c1520d0 con 0x7fb24c0afcf0 2023-10-06T05:18:37.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb25099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0c7190 0x7fb24c16a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:37.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb24bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24c0aef00 0x7fb24c14f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:37.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb24b7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb24c0afcf0 0x7fb24c169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:37.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb25099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0c7190 0x7fb24c16a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56774/0 (socket says 172.21.15.31:56774) 2023-10-06T05:18:37.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.677+0000 7fb25099c700 1 -- 172.21.15.31:0/1721415630 learned_addr learned my addr 172.21.15.31:0/1721415630 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:37.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.678+0000 7fb25099c700 1 -- 172.21.15.31:0/1721415630 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb24c0afcf0 msgr2=0x7fb24c169e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:37.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.678+0000 7fb25099c700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb24c0afcf0 0x7fb24c169e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:37.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.678+0000 7fb25099c700 1 -- 172.21.15.31:0/1721415630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24c0aef00 msgr2=0x7fb24c14f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:37.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.678+0000 7fb25099c700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24c0aef00 0x7fb24c14f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:37.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.678+0000 7fb25099c700 1 -- 172.21.15.31:0/1721415630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2400094f0 con 0x7fb24c0c7190 2023-10-06T05:18:37.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.678+0000 7fb25099c700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0c7190 0x7fb24c16a350 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb24400c8e0 tx=0x7fb244001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:37.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.678+0000 7fb233fff700 1 -- 172.21.15.31:0/1721415630 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244012630 con 0x7fb24c0c7190 2023-10-06T05:18:37.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.678+0000 7fb233fff700 1 -- 172.21.15.31:0/1721415630 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb244012790 con 0x7fb24c0c7190 2023-10-06T05:18:37.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.678+0000 7fb2523ff700 1 -- 172.21.15.31:0/1721415630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24c1523b0 con 0x7fb24c0c7190 2023-10-06T05:18:37.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.678+0000 7fb233fff700 1 -- 172.21.15.31:0/1721415630 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244012b50 con 0x7fb24c0c7190 2023-10-06T05:18:37.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.679+0000 7fb2523ff700 1 -- 172.21.15.31:0/1721415630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24c152970 con 0x7fb24c0c7190 2023-10-06T05:18:37.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.679+0000 7fb2523ff700 1 -- 172.21.15.31:0/1721415630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24c04c380 con 0x7fb24c0c7190 2023-10-06T05:18:37.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.681+0000 7fb233fff700 1 -- 172.21.15.31:0/1721415630 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fb24401a3a0 con 0x7fb24c0c7190 2023-10-06T05:18:37.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.681+0000 7fb233fff700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2380670e0 0x7fb238069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:37.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.681+0000 7fb233fff700 1 -- 172.21.15.31:0/1721415630 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fb244084930 con 0x7fb24c0c7190 2023-10-06T05:18:37.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.682+0000 7fb24bfff700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2380670e0 0x7fb238069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:37.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.682+0000 7fb24bfff700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2380670e0 0x7fb238069590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb23c006fd0 tx=0x7fb23c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:37.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.686+0000 7fb233fff700 1 -- 172.21.15.31:0/1721415630 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2440519a0 con 0x7fb24c0c7190 2023-10-06T05:18:37.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:37.890+0000 7fb2523ff700 1 -- 172.21.15.31:0/1721415630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb24c0cc6e0 con 0x7fb24c0c7190 2023-10-06T05:18:38.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.448+0000 7fb233fff700 1 -- 172.21.15.31:0/1721415630 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb244054fc0 con 0x7fb24c0c7190 2023-10-06T05:18:38.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.457+0000 7fb2523ff700 1 -- 172.21.15.31:0/1721415630 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2380670e0 msgr2=0x7fb238069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:38.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.457+0000 7fb2523ff700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2380670e0 0x7fb238069590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb23c006fd0 tx=0x7fb23c008040 comp rx=0 tx=0).stop 2023-10-06T05:18:38.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.457+0000 7fb2523ff700 1 -- 172.21.15.31:0/1721415630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0c7190 msgr2=0x7fb24c16a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:38.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.457+0000 7fb2523ff700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0c7190 0x7fb24c16a350 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb24400c8e0 tx=0x7fb244001710 comp rx=0 tx=0).stop 2023-10-06T05:18:38.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.458+0000 7fb2523ff700 1 -- 172.21.15.31:0/1721415630 shutdown_connections 2023-10-06T05:18:38.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.458+0000 7fb2523ff700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2380670e0 0x7fb238069590 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:38.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.458+0000 7fb2523ff700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb24c0c7190 0x7fb24c16a350 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:38.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.458+0000 7fb2523ff700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb24c0aef00 0x7fb24c14f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:38.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.458+0000 7fb2523ff700 1 --2- 172.21.15.31:0/1721415630 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb24c0afcf0 0x7fb24c169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:38.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.458+0000 7fb2523ff700 1 -- 172.21.15.31:0/1721415630 >> 172.21.15.31:0/1721415630 conn(0x7fb24c0cd3d0 msgr2=0x7fb24c0ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:38.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.460+0000 7fb2523ff700 1 -- 172.21.15.31:0/1721415630 shutdown_connections 2023-10-06T05:18:38.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.460+0000 7fb2523ff700 1 -- 172.21.15.31:0/1721415630 wait complete. 2023-10-06T05:18:38.468 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:18:38.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:18:38.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.683+0000 7fca2f3ff700 1 Processor -- start 2023-10-06T05:18:38.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.683+0000 7fca2f3ff700 1 -- start start 2023-10-06T05:18:38.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.684+0000 7fca2f3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 0x7fca280c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:38.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.685+0000 7fca2f3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca28068250 con 0x7fca280c7190 2023-10-06T05:18:38.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.685+0000 7fca2f3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2806ac80 con 0x7fca280c7560 2023-10-06T05:18:38.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.685+0000 7fca2f3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca28067f20 con 0x7fca280aef00 2023-10-06T05:18:38.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.685+0000 7fca2d19b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fca280c7190 0x7fca280af7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52184/0 (socket says 172.21.15.31:52184) 2023-10-06T05:18:38.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.685+0000 7fca2d19b700 1 -- 172.21.15.31:0/514676730 learned_addr learned my addr 172.21.15.31:0/514676730 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:38.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.685+0000 7fca2c99a700 1 --2- 172.21.15.31:0/514676730 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 0x7fca280c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.685+0000 7fca1ffff700 1 -- 172.21.15.31:0/514676730 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907965808 0 0) 0x7fca28068250 con 0x7fca280c7190 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.685+0000 7fca1ffff700 1 -- 172.21.15.31:0/514676730 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca14003570 con 0x7fca280c7190 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.686+0000 7fca1ffff700 1 -- 172.21.15.31:0/514676730 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937564590 0 0) 0x7fca2806ac80 con 0x7fca280c7560 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.686+0000 7fca1ffff700 1 -- 172.21.15.31:0/514676730 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca28068250 con 0x7fca280c7560 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.686+0000 7fca2c99a700 1 -- 172.21.15.31:0/514676730 >> v1:172.21.15.62:6789/0 conn(0x7fca280c7560 legacy=0x7fca280686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.686+0000 7fca2c99a700 1 -- 172.21.15.31:0/514676730 >> v1:172.21.15.31:6789/0 conn(0x7fca280c7190 legacy=0x7fca280af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.686+0000 7fca2c99a700 1 -- 172.21.15.31:0/514676730 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2806aed0 con 0x7fca280aef00 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.686+0000 7fca2c99a700 1 --2- 172.21.15.31:0/514676730 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 0x7fca280c9d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fca18009910 tx=0x7fca180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d9cebd67c0ba197 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.686+0000 7fca1ffff700 1 -- 172.21.15.31:0/514676730 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18011630 con 0x7fca280aef00 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.686+0000 7fca1ffff700 1 -- 172.21.15.31:0/514676730 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca18011790 con 0x7fca280aef00 2023-10-06T05:18:38.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.686+0000 7fca1ffff700 1 -- 172.21.15.31:0/514676730 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18011b10 con 0x7fca280aef00 2023-10-06T05:18:38.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.687+0000 7fca2f3ff700 1 -- 172.21.15.31:0/514676730 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 msgr2=0x7fca280c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:38.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.687+0000 7fca2f3ff700 1 --2- 172.21.15.31:0/514676730 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 0x7fca280c9d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fca18009910 tx=0x7fca180091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:38.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.687+0000 7fca2f3ff700 1 -- 172.21.15.31:0/514676730 shutdown_connections 2023-10-06T05:18:38.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.687+0000 7fca2f3ff700 1 --2- 172.21.15.31:0/514676730 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 0x7fca280c9d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:38.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.687+0000 7fca2f3ff700 1 -- 172.21.15.31:0/514676730 >> 172.21.15.31:0/514676730 conn(0x7fca280cd3d0 msgr2=0x7fca280c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:38.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.687+0000 7fca2f3ff700 1 -- 172.21.15.31:0/514676730 shutdown_connections 2023-10-06T05:18:38.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.687+0000 7fca2f3ff700 1 -- 172.21.15.31:0/514676730 wait complete. 2023-10-06T05:18:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.689+0000 7fca2f3ff700 1 Processor -- start 2023-10-06T05:18:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.689+0000 7fca2f3ff700 1 -- start start 2023-10-06T05:18:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2f3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 0x7fca280ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2f3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7190 0x7fca28157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2f3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7560 0x7fca281582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2f3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c4a70 con 0x7fca280c7560 2023-10-06T05:18:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2f3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c4d10 con 0x7fca280c7190 2023-10-06T05:18:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2f3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c4ff0 con 0x7fca280aef00 2023-10-06T05:18:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7560 0x7fca281582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2d19b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 0x7fca280ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2c99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7190 0x7fca28157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7560 0x7fca281582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56780/0 (socket says 172.21.15.31:56780) 2023-10-06T05:18:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2d99c700 1 -- 172.21.15.31:0/3907264067 learned_addr learned my addr 172.21.15.31:0/3907264067 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2d99c700 1 -- 172.21.15.31:0/3907264067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 msgr2=0x7fca280ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2d99c700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 0x7fca280ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2d99c700 1 -- 172.21.15.31:0/3907264067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7190 msgr2=0x7fca28157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2d99c700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7190 0x7fca28157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:38.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.690+0000 7fca2d99c700 1 -- 172.21.15.31:0/3907264067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca18009420 con 0x7fca280c7560 2023-10-06T05:18:38.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.691+0000 7fca2d99c700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7560 0x7fca281582e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c860 tx=0x7fca20002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:38.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.691+0000 7fca1d7fa700 1 -- 172.21.15.31:0/3907264067 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20002030 con 0x7fca280c7560 2023-10-06T05:18:38.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.691+0000 7fca1d7fa700 1 -- 172.21.15.31:0/3907264067 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca200021b0 con 0x7fca280c7560 2023-10-06T05:18:38.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.691+0000 7fca2f3ff700 1 -- 172.21.15.31:0/3907264067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca280c52d0 con 0x7fca280c7560 2023-10-06T05:18:38.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.691+0000 7fca1d7fa700 1 -- 172.21.15.31:0/3907264067 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20012630 con 0x7fca280c7560 2023-10-06T05:18:38.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.691+0000 7fca2f3ff700 1 -- 172.21.15.31:0/3907264067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca280cac30 con 0x7fca280c7560 2023-10-06T05:18:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.692+0000 7fca2f3ff700 1 -- 172.21.15.31:0/3907264067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca280cae80 con 0x7fca280c7560 2023-10-06T05:18:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.693+0000 7fca1d7fa700 1 -- 172.21.15.31:0/3907264067 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fca20012a20 con 0x7fca280c7560 2023-10-06T05:18:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.694+0000 7fca1d7fa700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 0x7fca0c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.694+0000 7fca1d7fa700 1 -- 172.21.15.31:0/3907264067 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fca200848f0 con 0x7fca280c7560 2023-10-06T05:18:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.694+0000 7fca2d19b700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 0x7fca0c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:38.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.695+0000 7fca2d19b700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 0x7fca0c069590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fca24007fd0 tx=0x7fca24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:38.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.699+0000 7fca1d7fa700 1 -- 172.21.15.31:0/3907264067 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca20051740 con 0x7fca280c7560 2023-10-06T05:18:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:38.944+0000 7fca2f3ff700 1 -- 172.21.15.31:0/3907264067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fca2804c380 con 0x7fca280c7560 2023-10-06T05:18:39.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.445+0000 7fca1d7fa700 1 -- 172.21.15.31:0/3907264067 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fca20054d60 con 0x7fca280c7560 2023-10-06T05:18:39.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.451+0000 7fca2f3ff700 1 -- 172.21.15.31:0/3907264067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 msgr2=0x7fca0c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.452+0000 7fca2f3ff700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 0x7fca0c069590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fca24007fd0 tx=0x7fca24009040 comp rx=0 tx=0).stop 2023-10-06T05:18:39.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.452+0000 7fca2f3ff700 1 -- 172.21.15.31:0/3907264067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7560 msgr2=0x7fca281582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.452+0000 7fca2f3ff700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7560 0x7fca281582e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c860 tx=0x7fca20002700 comp rx=0 tx=0).stop 2023-10-06T05:18:39.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.454+0000 7fca2f3ff700 1 -- 172.21.15.31:0/3907264067 shutdown_connections 2023-10-06T05:18:39.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.454+0000 7fca2f3ff700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fca0c0670e0 0x7fca0c069590 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.454+0000 7fca2f3ff700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca280c7560 0x7fca281582e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.454+0000 7fca2f3ff700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca280aef00 0x7fca280ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.454+0000 7fca2f3ff700 1 --2- 172.21.15.31:0/3907264067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca280c7190 0x7fca28157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.454+0000 7fca2f3ff700 1 -- 172.21.15.31:0/3907264067 >> 172.21.15.31:0/3907264067 conn(0x7fca280cd3d0 msgr2=0x7fca28158950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.457+0000 7fca2f3ff700 1 -- 172.21.15.31:0/3907264067 shutdown_connections 2023-10-06T05:18:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.457+0000 7fca2f3ff700 1 -- 172.21.15.31:0/3907264067 wait complete. 2023-10-06T05:18:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 7 and data pool 8 2023-10-06T05:18:39.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.691+0000 7f10aa7ff700 1 Processor -- start 2023-10-06T05:18:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.691+0000 7f10aa7ff700 1 -- start start 2023-10-06T05:18:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.692+0000 7f10aa7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 0x7f10a40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.692+0000 7f10aa7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10a415d2d0 0x7f10a40cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.692+0000 7f10aa7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40c3f40 con 0x7f10a41598d0 2023-10-06T05:18:39.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.692+0000 7f10aa7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40b2860 con 0x7f10a415d2d0 2023-10-06T05:18:39.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.692+0000 7f10aa7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a415d7d0 con 0x7f10a4159500 2023-10-06T05:18:39.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.692+0000 7f10a37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10a415d2d0 0x7f10a40cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.693+0000 7f10a8d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f10a41598d0 0x7f10a40aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52190/0 (socket says 172.21.15.31:52190) 2023-10-06T05:18:39.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.693+0000 7f10a8d9c700 1 -- 172.21.15.31:0/3229371398 learned_addr learned my addr 172.21.15.31:0/3229371398 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:39.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.693+0000 7f10a3fff700 1 --2- 172.21.15.31:0/3229371398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 0x7f10a40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.693+0000 7f10a2ffd700 1 -- 172.21.15.31:0/3229371398 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23528873 0 0) 0x7f10a40c3f40 con 0x7f10a41598d0 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.693+0000 7f10a37fe700 1 -- 172.21.15.31:0/3229371398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 msgr2=0x7f10a40b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.693+0000 7f10a37fe700 1 --2- 172.21.15.31:0/3229371398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 0x7f10a40b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.693+0000 7f10a37fe700 1 -- 172.21.15.31:0/3229371398 >> v1:172.21.15.31:6789/0 conn(0x7f10a41598d0 legacy=0x7f10a40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.693+0000 7f10a37fe700 1 -- 172.21.15.31:0/3229371398 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40cb000 con 0x7f10a415d2d0 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.693+0000 7f10a3fff700 1 --2- 172.21.15.31:0/3229371398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 0x7f10a40b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.693+0000 7f10a37fe700 1 --2- 172.21.15.31:0/3229371398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a415d2d0 0x7f10a40cce00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f10980098d0 tx=0x7f1098009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a3241850a1527b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10a2ffd700 1 -- 172.21.15.31:0/3229371398 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098009e70 con 0x7f10a415d2d0 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10a2ffd700 1 -- 172.21.15.31:0/3229371398 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098005c10 con 0x7f10a415d2d0 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10a2ffd700 1 -- 172.21.15.31:0/3229371398 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109800b650 con 0x7f10a415d2d0 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3229371398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a415d2d0 msgr2=0x7f10a40cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10aa7ff700 1 --2- 172.21.15.31:0/3229371398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a415d2d0 0x7f10a40cce00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f10980098d0 tx=0x7f1098009120 comp rx=0 tx=0).stop 2023-10-06T05:18:39.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3229371398 shutdown_connections 2023-10-06T05:18:39.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10aa7ff700 1 --2- 172.21.15.31:0/3229371398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 0x7f10a40b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10aa7ff700 1 --2- 172.21.15.31:0/3229371398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a415d2d0 0x7f10a40cce00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3229371398 >> 172.21.15.31:0/3229371398 conn(0x7f10a406ec90 msgr2=0x7f10a406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:39.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3229371398 shutdown_connections 2023-10-06T05:18:39.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.694+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3229371398 wait complete. 2023-10-06T05:18:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.696+0000 7f10aa7ff700 1 Processor -- start 2023-10-06T05:18:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.696+0000 7f10aa7ff700 1 -- start start 2023-10-06T05:18:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10aa7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 0x7f10a40c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10aa7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41598d0 0x7f10a4157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10aa7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a415d2d0 0x7f10a4158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10aa7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b4750 con 0x7f10a41598d0 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10aa7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b3b40 con 0x7f10a415d2d0 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10aa7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b3e20 con 0x7f10a4159500 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41598d0 0x7f10a4157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41598d0 0x7f10a4157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56792/0 (socket says 172.21.15.31:56792) 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a37fe700 1 -- 172.21.15.31:0/3545052256 learned_addr learned my addr 172.21.15.31:0/3545052256 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a8d9c700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a415d2d0 0x7f10a4158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a3fff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 0x7f10a40c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a37fe700 1 -- 172.21.15.31:0/3545052256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 msgr2=0x7f10a40c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a37fe700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 0x7f10a40c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a37fe700 1 -- 172.21.15.31:0/3545052256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a415d2d0 msgr2=0x7f10a4158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a37fe700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a415d2d0 0x7f10a4158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a37fe700 1 -- 172.21.15.31:0/3545052256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980093c0 con 0x7f10a41598d0 2023-10-06T05:18:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.697+0000 7f10a3fff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 0x7f10a40c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.698+0000 7f10a8d9c700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a415d2d0 0x7f10a4158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.698+0000 7f10a37fe700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41598d0 0x7f10a4157c50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f10a40ca9f0 tx=0x7f1098016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.698+0000 7f108bfff700 1 -- 172.21.15.31:0/3545052256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10980062c0 con 0x7f10a41598d0 2023-10-06T05:18:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.698+0000 7f108bfff700 1 -- 172.21.15.31:0/3545052256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098006420 con 0x7f10a41598d0 2023-10-06T05:18:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.698+0000 7f108bfff700 1 -- 172.21.15.31:0/3545052256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10980067e0 con 0x7f10a41598d0 2023-10-06T05:18:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.698+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3545052256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a40b4100 con 0x7f10a41598d0 2023-10-06T05:18:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.698+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3545052256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a40b36f0 con 0x7f10a41598d0 2023-10-06T05:18:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.699+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3545052256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a40b3910 con 0x7f10a41598d0 2023-10-06T05:18:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.700+0000 7f108bfff700 1 -- 172.21.15.31:0/3545052256 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f109801b690 con 0x7f10a41598d0 2023-10-06T05:18:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.701+0000 7f108bfff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10900670e0 0x7f1090069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.701+0000 7f108bfff700 1 -- 172.21.15.31:0/3545052256 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1098086b00 con 0x7f10a41598d0 2023-10-06T05:18:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.701+0000 7f10a3fff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10900670e0 0x7f1090069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:39.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.702+0000 7f10a3fff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10900670e0 0x7f1090069590 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1094006f00 tx=0x7f1094008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.705+0000 7f108bfff700 1 -- 172.21.15.31:0/3545052256 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1098088e60 con 0x7f10a41598d0 2023-10-06T05:18:39.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.906+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3545052256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10a404c380 con 0x7f10a41598d0 2023-10-06T05:18:39.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.907+0000 7f108bfff700 1 -- 172.21.15.31:0/3545052256 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16927 (secure 0 0 0) 0x7f10980538f0 con 0x7f10a41598d0 2023-10-06T05:18:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:39.910 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":43,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:38.945738+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:36.639713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:37.929704+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:39.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3545052256 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10900670e0 msgr2=0x7f1090069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10900670e0 0x7f1090069590 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1094006f00 tx=0x7f1094008040 comp rx=0 tx=0).stop 2023-10-06T05:18:39.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3545052256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41598d0 msgr2=0x7f10a4157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:39.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41598d0 0x7f10a4157c50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f10a40ca9f0 tx=0x7f1098016000 comp rx=0 tx=0).stop 2023-10-06T05:18:39.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3545052256 shutdown_connections 2023-10-06T05:18:39.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a415d2d0 0x7f10a4158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a4159500 0x7f10a40c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10900670e0 0x7f1090069590 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 --2- 172.21.15.31:0/3545052256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41598d0 0x7f10a4157c50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:39.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3545052256 >> 172.21.15.31:0/3545052256 conn(0x7f10a406ec90 msgr2=0x7f10a40cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:39.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3545052256 shutdown_connections 2023-10-06T05:18:39.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:39.910+0000 7f10aa7ff700 1 -- 172.21.15.31:0/3545052256 wait complete. 2023-10-06T05:18:39.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:40.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.123+0000 7fd85b7ff700 1 Processor -- start 2023-10-06T05:18:40.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.123+0000 7fd85b7ff700 1 -- start start 2023-10-06T05:18:40.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.125+0000 7fd85b7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8541595a0 0x7fd854159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.125+0000 7fd85b7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8540c95b0 con 0x7fd854068d80 2023-10-06T05:18:40.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.125+0000 7fd85b7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8540c9890 con 0x7fd8540cc0d0 2023-10-06T05:18:40.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.125+0000 7fd85b7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540c9b70 con 0x7fd8541595a0 2023-10-06T05:18:40.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.125+0000 7fd859d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd854068d80 0x7fd8540c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52198/0 (socket says 172.21.15.31:52198) 2023-10-06T05:18:40.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.125+0000 7fd859d9c700 1 -- 172.21.15.31:0/3928915853 learned_addr learned my addr 172.21.15.31:0/3928915853 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:40.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.125+0000 7fd85959b700 1 --2- 172.21.15.31:0/3928915853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8541595a0 0x7fd854159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.126+0000 7fd84bfff700 1 -- 172.21.15.31:0/3928915853 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482600188 0 0) 0x7fd8540c95b0 con 0x7fd854068d80 2023-10-06T05:18:40.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.126+0000 7fd84bfff700 1 -- 172.21.15.31:0/3928915853 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd840003570 con 0x7fd854068d80 2023-10-06T05:18:40.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.126+0000 7fd84bfff700 1 -- 172.21.15.31:0/3928915853 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595720314 0 0) 0x7fd8540c9890 con 0x7fd8540cc0d0 2023-10-06T05:18:40.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.126+0000 7fd84bfff700 1 -- 172.21.15.31:0/3928915853 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8540c95b0 con 0x7fd8540cc0d0 2023-10-06T05:18:40.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.126+0000 7fd85959b700 1 -- 172.21.15.31:0/3928915853 >> v1:172.21.15.62:6789/0 conn(0x7fd8540cc0d0 legacy=0x7fd854068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.126+0000 7fd85959b700 1 -- 172.21.15.31:0/3928915853 >> v1:172.21.15.31:6789/0 conn(0x7fd854068d80 legacy=0x7fd8540c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.126+0000 7fd85959b700 1 -- 172.21.15.31:0/3928915853 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540ca130 con 0x7fd8541595a0 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.126+0000 7fd85959b700 1 --2- 172.21.15.31:0/3928915853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8541595a0 0x7fd854159970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd850009910 tx=0x7fd8500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6a5a98571cf9ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.126+0000 7fd84bfff700 1 -- 172.21.15.31:0/3928915853 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850011630 con 0x7fd8541595a0 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.126+0000 7fd84bfff700 1 -- 172.21.15.31:0/3928915853 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd850011790 con 0x7fd8541595a0 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.127+0000 7fd84bfff700 1 -- 172.21.15.31:0/3928915853 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850011b30 con 0x7fd8541595a0 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.127+0000 7fd85b7ff700 1 -- 172.21.15.31:0/3928915853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8541595a0 msgr2=0x7fd854159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.127+0000 7fd85b7ff700 1 --2- 172.21.15.31:0/3928915853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8541595a0 0x7fd854159970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd850009910 tx=0x7fd8500091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.127+0000 7fd85b7ff700 1 -- 172.21.15.31:0/3928915853 shutdown_connections 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.127+0000 7fd85b7ff700 1 --2- 172.21.15.31:0/3928915853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8541595a0 0x7fd854159970 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.127+0000 7fd85b7ff700 1 -- 172.21.15.31:0/3928915853 >> 172.21.15.31:0/3928915853 conn(0x7fd85406b620 msgr2=0x7fd85406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:40.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.127+0000 7fd85b7ff700 1 -- 172.21.15.31:0/3928915853 shutdown_connections 2023-10-06T05:18:40.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.127+0000 7fd85b7ff700 1 -- 172.21.15.31:0/3928915853 wait complete. 2023-10-06T05:18:40.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.129+0000 7fd85b7ff700 1 Processor -- start 2023-10-06T05:18:40.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.130+0000 7fd85b7ff700 1 -- start start 2023-10-06T05:18:40.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd85b7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd854068d80 0x7fd8540b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd85b7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8540cc0d0 0x7fd854151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd85b7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8541595a0 0x7fd8540ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd85b7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540b15d0 con 0x7fd8541595a0 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd85b7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540b1800 con 0x7fd854068d80 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd85b7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540b1ae0 con 0x7fd8540cc0d0 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd859d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8541595a0 0x7fd8540ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd859d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8541595a0 0x7fd8540ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56794/0 (socket says 172.21.15.31:56794) 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd85959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd854068d80 0x7fd8540b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd859d9c700 1 -- 172.21.15.31:0/1394058289 learned_addr learned my addr 172.21.15.31:0/1394058289 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.131+0000 7fd858d9a700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8540cc0d0 0x7fd854151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd859d9c700 1 -- 172.21.15.31:0/1394058289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8540cc0d0 msgr2=0x7fd854151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd859d9c700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8540cc0d0 0x7fd854151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd859d9c700 1 -- 172.21.15.31:0/1394058289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd854068d80 msgr2=0x7fd8540b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd859d9c700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd854068d80 0x7fd8540b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd859d9c700 1 -- 172.21.15.31:0/1394058289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850009420 con 0x7fd8541595a0 2023-10-06T05:18:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd859d9c700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8541595a0 0x7fd8540ccac0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd84c001460 tx=0x7fd84c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd8497fa700 1 -- 172.21.15.31:0/1394058289 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00a7e0 con 0x7fd8541595a0 2023-10-06T05:18:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd8497fa700 1 -- 172.21.15.31:0/1394058289 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c00a980 con 0x7fd8541595a0 2023-10-06T05:18:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd8497fa700 1 -- 172.21.15.31:0/1394058289 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00ad40 con 0x7fd8541595a0 2023-10-06T05:18:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd85b7ff700 1 -- 172.21.15.31:0/1394058289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8540b1dc0 con 0x7fd8541595a0 2023-10-06T05:18:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.132+0000 7fd85b7ff700 1 -- 172.21.15.31:0/1394058289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8540b2380 con 0x7fd8541595a0 2023-10-06T05:18:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.133+0000 7fd85b7ff700 1 -- 172.21.15.31:0/1394058289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85404c380 con 0x7fd8541595a0 2023-10-06T05:18:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.134+0000 7fd8497fa700 1 -- 172.21.15.31:0/1394058289 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd84c007300 con 0x7fd8541595a0 2023-10-06T05:18:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.135+0000 7fd8497fa700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8380670e0 0x7fd838069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.135+0000 7fd8497fa700 1 -- 172.21.15.31:0/1394058289 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd84c084ca0 con 0x7fd8541595a0 2023-10-06T05:18:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.136+0000 7fd85959b700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8380670e0 0x7fd838069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.136+0000 7fd85959b700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8380670e0 0x7fd838069590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd8500091e0 tx=0x7fd850017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.140+0000 7fd8497fa700 1 -- 172.21.15.31:0/1394058289 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd84c051b10 con 0x7fd8541595a0 2023-10-06T05:18:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.341+0000 7fd85b7ff700 1 -- 172.21.15.31:0/1394058289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd85406f0f0 con 0x7fd8541595a0 2023-10-06T05:18:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.342+0000 7fd8497fa700 1 -- 172.21.15.31:0/1394058289 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16927 (secure 0 0 0) 0x7fd84c055130 con 0x7fd8541595a0 2023-10-06T05:18:40.344 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:40.344 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":43,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:38.945738+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:36.639713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:37.929704+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:40.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.344+0000 7fd85b7ff700 1 -- 172.21.15.31:0/1394058289 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8380670e0 msgr2=0x7fd838069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.344+0000 7fd85b7ff700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8380670e0 0x7fd838069590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd8500091e0 tx=0x7fd850017000 comp rx=0 tx=0).stop 2023-10-06T05:18:40.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.344+0000 7fd85b7ff700 1 -- 172.21.15.31:0/1394058289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8541595a0 msgr2=0x7fd8540ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.344+0000 7fd85b7ff700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8541595a0 0x7fd8540ccac0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd84c001460 tx=0x7fd84c001a00 comp rx=0 tx=0).stop 2023-10-06T05:18:40.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.345+0000 7fd85b7ff700 1 -- 172.21.15.31:0/1394058289 shutdown_connections 2023-10-06T05:18:40.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.345+0000 7fd85b7ff700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd8380670e0 0x7fd838069590 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.345+0000 7fd85b7ff700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8541595a0 0x7fd8540ccac0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.345+0000 7fd85b7ff700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd854068d80 0x7fd8540b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.345+0000 7fd85b7ff700 1 --2- 172.21.15.31:0/1394058289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8540cc0d0 0x7fd854151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.345+0000 7fd85b7ff700 1 -- 172.21.15.31:0/1394058289 >> 172.21.15.31:0/1394058289 conn(0x7fd85406b620 msgr2=0x7fd854154da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:40.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.345+0000 7fd85b7ff700 1 -- 172.21.15.31:0/1394058289 shutdown_connections 2023-10-06T05:18:40.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.345+0000 7fd85b7ff700 1 -- 172.21.15.31:0/1394058289 wait complete. 2023-10-06T05:18:40.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:18:40.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.565+0000 7f014fbff700 1 Processor -- start 2023-10-06T05:18:40.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.565+0000 7f014fbff700 1 -- start start 2023-10-06T05:18:40.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.567+0000 7f014fbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f014815ca40 0x7f0148068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.567+0000 7f014fbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0148068da0 con 0x7f01480c8f40 2023-10-06T05:18:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.567+0000 7f014fbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0148069080 con 0x7f014806a300 2023-10-06T05:18:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.567+0000 7f014fbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01480cabc0 con 0x7f014815ca40 2023-10-06T05:18:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.567+0000 7f014e19c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f01480c8f40 0x7f01480adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52214/0 (socket says 172.21.15.31:52214) 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.567+0000 7f014e19c700 1 -- 172.21.15.31:0/4178042634 learned_addr learned my addr 172.21.15.31:0/4178042634 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.567+0000 7f014d99b700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f014815ca40 0x7f0148068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774963197 0 0) 0x7f0148068da0 con 0x7f01480c8f40 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0134003570 con 0x7f01480c8f40 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797805437 0 0) 0x7f0148069080 con 0x7f014806a300 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0148068da0 con 0x7f014806a300 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3270978512 0 0) 0x7f0134003570 con 0x7f01480c8f40 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f014815ca40 msgr2=0x7f0148068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f014815ca40 0x7f0148068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 >> v1:172.21.15.62:6789/0 conn(0x7f014806a300 legacy=0x7f01480c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0148069360 con 0x7f01480c8f40 2023-10-06T05:18:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f01400017f0 con 0x7f01480c8f40 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 >> v1:172.21.15.31:6789/0 conn(0x7f01480c8f40 legacy=0x7f01480adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f014815ca40 0x7f0134003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01340041d0 0x7f0134004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134004d50 0x7f01340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01480cabc0 con 0x7f01340041d0 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013400bbf0 con 0x7f0134004d50 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013400bdb0 con 0x7f014815ca40 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.568+0000 7f014d19a700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01340041d0 0x7f0134004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014e19c700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134004d50 0x7f01340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014d99b700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f014815ca40 0x7f0134003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014d19a700 1 -- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f014815ca40 msgr2=0x7f0134003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014d19a700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f014815ca40 0x7f0134003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014d19a700 1 -- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134004d50 msgr2=0x7f01340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014d19a700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134004d50 0x7f01340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014d19a700 1 -- 172.21.15.31:0/4178042634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0134003570 con 0x7f01340041d0 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014d19a700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01340041d0 0x7f0134004810 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0138001a10 tx=0x7f0138007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f30bb4cf42cba3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800c5d0 con 0x7f01340041d0 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013800c730 con 0x7f01340041d0 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.569+0000 7f014c999700 1 -- 172.21.15.31:0/4178042634 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800cad0 con 0x7f01340041d0 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.570+0000 7f014fbff700 1 -- 172.21.15.31:0/4178042634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01340041d0 msgr2=0x7f0134004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.570+0000 7f014fbff700 1 --2- 172.21.15.31:0/4178042634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01340041d0 0x7f0134004810 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0138001a10 tx=0x7f0138007d30 comp rx=0 tx=0).stop 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.570+0000 7f014e19c700 1 -- 172.21.15.31:0/4178042634 reap_dead start 2023-10-06T05:18:40.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.570+0000 7f014fbff700 1 -- 172.21.15.31:0/4178042634 shutdown_connections 2023-10-06T05:18:40.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.570+0000 7f014fbff700 1 -- 172.21.15.31:0/4178042634 >> 172.21.15.31:0/4178042634 conn(0x7f01480ce290 msgr2=0x7f01480ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:40.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.570+0000 7f014fbff700 1 -- 172.21.15.31:0/4178042634 shutdown_connections 2023-10-06T05:18:40.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.570+0000 7f014fbff700 1 -- 172.21.15.31:0/4178042634 wait complete. 2023-10-06T05:18:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.572+0000 7f014fbff700 1 Processor -- start 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.572+0000 7f014fbff700 1 -- start start 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.573+0000 7f014fbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01340041d0 0x7f01480b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.573+0000 7f014fbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01480b32f0 0x7f0148153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.573+0000 7f014fbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01480c27c0 0x7f0148153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.573+0000 7f014fbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01480c2330 con 0x7f01480b32f0 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.573+0000 7f014fbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0148154eb0 con 0x7f01340041d0 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.573+0000 7f014fbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01480c1a00 con 0x7f01480c27c0 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01480b32f0 0x7f0148153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01480b32f0 0x7f0148153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56820/0 (socket says 172.21.15.31:56820) 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014d19a700 1 -- 172.21.15.31:0/3405337839 learned_addr learned my addr 172.21.15.31:0/3405337839 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014e19c700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01480c27c0 0x7f0148153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014d99b700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01340041d0 0x7f01480b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014d19a700 1 -- 172.21.15.31:0/3405337839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01480c27c0 msgr2=0x7f0148153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014d19a700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01480c27c0 0x7f0148153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014d19a700 1 -- 172.21.15.31:0/3405337839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01340041d0 msgr2=0x7f01480b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014d19a700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01340041d0 0x7f01480b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014d19a700 1 -- 172.21.15.31:0/3405337839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01480c1ce0 con 0x7f01480b32f0 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014e19c700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01480c27c0 0x7f0148153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.574+0000 7f014d19a700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01480b32f0 0x7f0148153300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f013800c3c0 tx=0x7f0138015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.575+0000 7f013dffb700 1 -- 172.21.15.31:0/3405337839 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800c5d0 con 0x7f01480b32f0 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.575+0000 7f013dffb700 1 -- 172.21.15.31:0/3405337839 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0138001360 con 0x7f01480b32f0 2023-10-06T05:18:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.575+0000 7f014fbff700 1 -- 172.21.15.31:0/3405337839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01480c1fc0 con 0x7f01480b32f0 2023-10-06T05:18:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.575+0000 7f013dffb700 1 -- 172.21.15.31:0/3405337839 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138006a70 con 0x7f01480b32f0 2023-10-06T05:18:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.575+0000 7f014fbff700 1 -- 172.21.15.31:0/3405337839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0148154380 con 0x7f01480b32f0 2023-10-06T05:18:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.576+0000 7f014fbff700 1 -- 172.21.15.31:0/3405337839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014804c380 con 0x7f01480b32f0 2023-10-06T05:18:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.577+0000 7f013dffb700 1 -- 172.21.15.31:0/3405337839 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f0138019070 con 0x7f01480b32f0 2023-10-06T05:18:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.578+0000 7f013dffb700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f012c0670e0 0x7f012c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.578+0000 7f013dffb700 1 -- 172.21.15.31:0/3405337839 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0138085cb0 con 0x7f01480b32f0 2023-10-06T05:18:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.578+0000 7f014d99b700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f012c0670e0 0x7f012c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.579+0000 7f014d99b700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f012c0670e0 0x7f012c069590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0144006dc0 tx=0x7f0144008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:40.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.582+0000 7f013dffb700 1 -- 172.21.15.31:0/3405337839 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0138052aa0 con 0x7f01480b32f0 2023-10-06T05:18:40.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:40.828+0000 7f014fbff700 1 -- 172.21.15.31:0/3405337839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f01480ca1a0 con 0x7f01480b32f0 2023-10-06T05:18:41.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.479+0000 7f013dffb700 1 -- 172.21.15.31:0/3405337839 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0138006e60 con 0x7f01480b32f0 2023-10-06T05:18:41.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.482+0000 7f014fbff700 1 -- 172.21.15.31:0/3405337839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f012c0670e0 msgr2=0x7f012c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:41.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.482+0000 7f014fbff700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f012c0670e0 0x7f012c069590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0144006dc0 tx=0x7f0144008040 comp rx=0 tx=0).stop 2023-10-06T05:18:41.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.482+0000 7f014fbff700 1 -- 172.21.15.31:0/3405337839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01480b32f0 msgr2=0x7f0148153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:41.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.482+0000 7f014fbff700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01480b32f0 0x7f0148153300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f013800c3c0 tx=0x7f0138015000 comp rx=0 tx=0).stop 2023-10-06T05:18:41.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.482+0000 7f014fbff700 1 -- 172.21.15.31:0/3405337839 shutdown_connections 2023-10-06T05:18:41.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.482+0000 7f014fbff700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01480c27c0 0x7f0148153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.482+0000 7f014fbff700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01340041d0 0x7f01480b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.482+0000 7f014fbff700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f012c0670e0 0x7f012c069590 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.482+0000 7f014fbff700 1 --2- 172.21.15.31:0/3405337839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01480b32f0 0x7f0148153300 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.482+0000 7f014fbff700 1 -- 172.21.15.31:0/3405337839 >> 172.21.15.31:0/3405337839 conn(0x7f01480ce290 msgr2=0x7f01480c4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:41.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.483+0000 7f014fbff700 1 -- 172.21.15.31:0/3405337839 shutdown_connections 2023-10-06T05:18:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.483+0000 7f014fbff700 1 -- 172.21.15.31:0/3405337839 wait complete. 2023-10-06T05:18:41.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:18:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.707+0000 7f99735ff700 1 Processor -- start 2023-10-06T05:18:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.707+0000 7f99735ff700 1 -- start start 2023-10-06T05:18:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f99735ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f996c159930 0x7f996c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f99735ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f996c0cd580 0x7f996c0b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f99735ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996c15d2d0 con 0x7f996c159560 2023-10-06T05:18:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f99735ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996c15d5b0 con 0x7f996c159930 2023-10-06T05:18:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f99735ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c15d890 con 0x7f996c0cd580 2023-10-06T05:18:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f997139b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f996c159560 0x7f996c0cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52218/0 (socket says 172.21.15.31:52218) 2023-10-06T05:18:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f997139b700 1 -- 172.21.15.31:0/3426105140 learned_addr learned my addr 172.21.15.31:0/3426105140 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f9971b9c700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f996c0cd580 0x7f996c0b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f9970b9a700 1 --2- 172.21.15.31:0/3426105140 >> v2:172.21.15.62:3300/0 conn(0x7f996c159930 0x7f996c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700078517 0 0) 0x7f996c15d2d0 con 0x7f996c159560 2023-10-06T05:18:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.709+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9958003570 con 0x7f996c159560 2023-10-06T05:18:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2002904151 0 0) 0x7f9958003570 con 0x7f996c159560 2023-10-06T05:18:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f996c0cd580 msgr2=0x7f996c0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f996c0cd580 0x7f996c0b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 >> v2:172.21.15.62:3300/0 conn(0x7f996c159930 msgr2=0x7f996c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 --2- 172.21.15.31:0/3426105140 >> v2:172.21.15.62:3300/0 conn(0x7f996c159930 0x7f996c0b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c15db70 con 0x7f996c159560 2023-10-06T05:18:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f9968001470 con 0x7f996c159560 2023-10-06T05:18:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 >> v1:172.21.15.31:6789/0 conn(0x7f996c159560 legacy=0x7f996c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f996c0cd580 0x7f9958003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99580040b0 0x7f9958004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9958004d50 0x7f99580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c15d5b0 con 0x7f99580040b0 2023-10-06T05:18:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c15d890 con 0x7f9958004d50 2023-10-06T05:18:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995800bb60 con 0x7f996c0cd580 2023-10-06T05:18:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9970b9a700 1 --2- 172.21.15.31:0/3426105140 >> v2:172.21.15.62:3300/0 conn(0x7f996c159930 0x7f996c0b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9970b9a700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99580040b0 0x7f9958004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f9971b9c700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9958004d50 0x7f99580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.710+0000 7f997139b700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f996c0cd580 0x7f9958003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.711+0000 7f9970b9a700 1 -- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f996c0cd580 msgr2=0x7f9958003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.711+0000 7f9970b9a700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f996c0cd580 0x7f9958003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.711+0000 7f9970b9a700 1 -- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9958004d50 msgr2=0x7f99580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.711+0000 7f9970b9a700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9958004d50 0x7f99580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.711+0000 7f9970b9a700 1 -- 172.21.15.31:0/3426105140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9958003570 con 0x7f99580040b0 2023-10-06T05:18:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.711+0000 7f9971b9c700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9958004d50 0x7f99580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.711+0000 7f9970b9a700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99580040b0 0x7f9958004810 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f995c006f00 tx=0x7f995c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a17691805b71a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:41.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.711+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c009740 con 0x7f99580040b0 2023-10-06T05:18:41.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.711+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c00e920 con 0x7f99580040b0 2023-10-06T05:18:41.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.711+0000 7f9963fff700 1 -- 172.21.15.31:0/3426105140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c00ecc0 con 0x7f99580040b0 2023-10-06T05:18:41.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.712+0000 7f99735ff700 1 -- 172.21.15.31:0/3426105140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99580040b0 msgr2=0x7f9958004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:41.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.712+0000 7f99735ff700 1 --2- 172.21.15.31:0/3426105140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99580040b0 0x7f9958004810 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f995c006f00 tx=0x7f995c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.712+0000 7f9971b9c700 1 -- 172.21.15.31:0/3426105140 reap_dead start 2023-10-06T05:18:41.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.712+0000 7f99735ff700 1 -- 172.21.15.31:0/3426105140 shutdown_connections 2023-10-06T05:18:41.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.712+0000 7f99735ff700 1 -- 172.21.15.31:0/3426105140 >> 172.21.15.31:0/3426105140 conn(0x7f996c06ec90 msgr2=0x7f996c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:41.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.712+0000 7f99735ff700 1 -- 172.21.15.31:0/3426105140 shutdown_connections 2023-10-06T05:18:41.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.712+0000 7f99735ff700 1 -- 172.21.15.31:0/3426105140 wait complete. 2023-10-06T05:18:41.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.714+0000 7f99735ff700 1 Processor -- start 2023-10-06T05:18:41.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.714+0000 7f99735ff700 1 -- start start 2023-10-06T05:18:41.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f99735ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99580040b0 0x7f996c0c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:41.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f99735ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f996c15a710 0x7f996c0af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:41.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f99735ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c155a70 0x7f996c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f99735ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0c8610 con 0x7f996c155a70 2023-10-06T05:18:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f99735ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0afd40 con 0x7f996c15a710 2023-10-06T05:18:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f99735ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c15acb0 con 0x7f99580040b0 2023-10-06T05:18:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f9971b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c155a70 0x7f996c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f9971b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c155a70 0x7f996c15ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56836/0 (socket says 172.21.15.31:56836) 2023-10-06T05:18:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f9971b9c700 1 -- 172.21.15.31:0/3495472458 learned_addr learned my addr 172.21.15.31:0/3495472458 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f997139b700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99580040b0 0x7f996c0c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f9970b9a700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f996c15a710 0x7f996c0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f9971b9c700 1 -- 172.21.15.31:0/3495472458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99580040b0 msgr2=0x7f996c0c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f9971b9c700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99580040b0 0x7f996c0c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f9971b9c700 1 -- 172.21.15.31:0/3495472458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f996c15a710 msgr2=0x7f996c0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f9971b9c700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f996c15a710 0x7f996c0af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.716+0000 7f9971b9c700 1 -- 172.21.15.31:0/3495472458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c15af90 con 0x7f996c155a70 2023-10-06T05:18:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.717+0000 7f9971b9c700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c155a70 0x7f996c15ba30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f99640094d0 tx=0x7f996400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.717+0000 7f99617fa700 1 -- 172.21.15.31:0/3495472458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996400e040 con 0x7f996c155a70 2023-10-06T05:18:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.717+0000 7f99617fa700 1 -- 172.21.15.31:0/3495472458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996400f040 con 0x7f996c155a70 2023-10-06T05:18:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.717+0000 7f99617fa700 1 -- 172.21.15.31:0/3495472458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996400fdb0 con 0x7f996c155a70 2023-10-06T05:18:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.717+0000 7f99735ff700 1 -- 172.21.15.31:0/3495472458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c0c4f20 con 0x7f996c155a70 2023-10-06T05:18:41.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.717+0000 7f99735ff700 1 -- 172.21.15.31:0/3495472458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c0c54e0 con 0x7f996c155a70 2023-10-06T05:18:41.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.718+0000 7f99735ff700 1 -- 172.21.15.31:0/3495472458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c0c5700 con 0x7f996c155a70 2023-10-06T05:18:41.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.719+0000 7f99617fa700 1 -- 172.21.15.31:0/3495472458 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9964015640 con 0x7f996c155a70 2023-10-06T05:18:41.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.720+0000 7f99617fa700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f99500670e0 0x7f9950069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:41.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.720+0000 7f99617fa700 1 -- 172.21.15.31:0/3495472458 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9964085440 con 0x7f996c155a70 2023-10-06T05:18:41.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.720+0000 7f997139b700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f99500670e0 0x7f9950069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:41.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.721+0000 7f997139b700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f99500670e0 0x7f9950069590 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9968000fd0 tx=0x7f9968001dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:41.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.724+0000 7f99617fa700 1 -- 172.21.15.31:0/3495472458 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9964052120 con 0x7f996c155a70 2023-10-06T05:18:41.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:41.970+0000 7f99735ff700 1 -- 172.21.15.31:0/3495472458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f996c04c380 con 0x7f996c155a70 2023-10-06T05:18:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.489+0000 7f99617fa700 1 -- 172.21.15.31:0/3495472458 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9964055740 con 0x7f996c155a70 2023-10-06T05:18:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.491+0000 7f99735ff700 1 -- 172.21.15.31:0/3495472458 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f99500670e0 msgr2=0x7f9950069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.491+0000 7f99735ff700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f99500670e0 0x7f9950069590 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9968000fd0 tx=0x7f9968001dc0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.491+0000 7f99735ff700 1 -- 172.21.15.31:0/3495472458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c155a70 msgr2=0x7f996c15ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.491+0000 7f99735ff700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c155a70 0x7f996c15ba30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f99640094d0 tx=0x7f996400b040 comp rx=0 tx=0).stop 2023-10-06T05:18:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.491+0000 7f99735ff700 1 -- 172.21.15.31:0/3495472458 shutdown_connections 2023-10-06T05:18:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.491+0000 7f99735ff700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f99500670e0 0x7f9950069590 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.491+0000 7f99735ff700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c155a70 0x7f996c15ba30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.491+0000 7f99735ff700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99580040b0 0x7f996c0c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.491+0000 7f99735ff700 1 --2- 172.21.15.31:0/3495472458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f996c15a710 0x7f996c0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.491+0000 7f99735ff700 1 -- 172.21.15.31:0/3495472458 >> 172.21.15.31:0/3495472458 conn(0x7f996c06ec90 msgr2=0x7f996c156580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.492+0000 7f99735ff700 1 -- 172.21.15.31:0/3495472458 shutdown_connections 2023-10-06T05:18:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.492+0000 7f99735ff700 1 -- 172.21.15.31:0/3495472458 wait complete. 2023-10-06T05:18:42.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:42.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.708+0000 7f36029ff700 1 Processor -- start 2023-10-06T05:18:42.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.709+0000 7f36029ff700 1 -- start start 2023-10-06T05:18:42.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.710+0000 7f36029ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35fc1595a0 0x7f35fc159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:42.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.710+0000 7f36029ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc0c95b0 con 0x7f35fc0cc0d0 2023-10-06T05:18:42.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.710+0000 7f36029ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc0c9890 con 0x7f35fc068d80 2023-10-06T05:18:42.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.710+0000 7f36029ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0c9b70 con 0x7f35fc1595a0 2023-10-06T05:18:42.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.710+0000 7f35fb7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f35fc0cc0d0 0x7f35fc068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52228/0 (socket says 172.21.15.31:52228) 2023-10-06T05:18:42.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.710+0000 7f35fb7fe700 1 -- 172.21.15.31:0/2910822508 learned_addr learned my addr 172.21.15.31:0/2910822508 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:42.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.710+0000 7f35fbfff700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35fc1595a0 0x7f35fc159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:42.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183663845 0 0) 0x7f35fc0c95b0 con 0x7f35fc0cc0d0 2023-10-06T05:18:42.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35e4003570 con 0x7f35fc0cc0d0 2023-10-06T05:18:42.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889454280 0 0) 0x7f35fc0c9890 con 0x7f35fc068d80 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35fc0c95b0 con 0x7f35fc068d80 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 591204365 0 0) 0x7f35e4003570 con 0x7f35fc0cc0d0 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35fc1595a0 msgr2=0x7f35fc159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35fc1595a0 0x7f35fc159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 >> v1:172.21.15.62:6789/0 conn(0x7f35fc068d80 legacy=0x7f35fc0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0ca130 con 0x7f35fc0cc0d0 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f35f0001470 con 0x7f35fc0cc0d0 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 >> v1:172.21.15.31:6789/0 conn(0x7f35fc0cc0d0 legacy=0x7f35fc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35e4003c20 0x7f35e4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35e4004810 0x7f35e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e40051c0 0x7f35e400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0c9b70 con 0x7f35e40051c0 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e400c020 con 0x7f35e4003c20 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e400c1e0 con 0x7f35e4004810 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.711+0000 7f35fbfff700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35fc1595a0 0x7f35fc159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.712+0000 7f3600f9c700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e40051c0 0x7f35e400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.712+0000 7f3600f9c700 1 -- 172.21.15.31:0/2910822508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35e4004810 msgr2=0x7f35e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.712+0000 7f3600f9c700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35e4004810 0x7f35e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.712+0000 7f3600f9c700 1 -- 172.21.15.31:0/2910822508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35e4003c20 msgr2=0x7f35e4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.712+0000 7f3600f9c700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35e4003c20 0x7f35e4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.712+0000 7f3600f9c700 1 -- 172.21.15.31:0/2910822508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4003570 con 0x7f35e40051c0 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.712+0000 7f3600f9c700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e40051c0 0x7f35e400ba60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f35f4002130 tx=0x7f35f40017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1fc53b6d1f44e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:42.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.712+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4001af0 con 0x7f35e40051c0 2023-10-06T05:18:42.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.712+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f400c980 con 0x7f35e40051c0 2023-10-06T05:18:42.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.712+0000 7f35faffd700 1 -- 172.21.15.31:0/2910822508 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400cd20 con 0x7f35e40051c0 2023-10-06T05:18:42.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.713+0000 7f36029ff700 1 -- 172.21.15.31:0/2910822508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e40051c0 msgr2=0x7f35e400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:42.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.713+0000 7f36029ff700 1 --2- 172.21.15.31:0/2910822508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35e40051c0 0x7f35e400ba60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f35f4002130 tx=0x7f35f40017e0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.713+0000 7f3600f9c700 1 -- 172.21.15.31:0/2910822508 reap_dead start 2023-10-06T05:18:42.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.713+0000 7f36029ff700 1 -- 172.21.15.31:0/2910822508 shutdown_connections 2023-10-06T05:18:42.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.713+0000 7f36029ff700 1 -- 172.21.15.31:0/2910822508 >> 172.21.15.31:0/2910822508 conn(0x7f35fc06b620 msgr2=0x7f35fc06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:42.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.713+0000 7f36029ff700 1 -- 172.21.15.31:0/2910822508 shutdown_connections 2023-10-06T05:18:42.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.713+0000 7f36029ff700 1 -- 172.21.15.31:0/2910822508 wait complete. 2023-10-06T05:18:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.714+0000 7f36029ff700 1 Processor -- start 2023-10-06T05:18:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.715+0000 7f36029ff700 1 -- start start 2023-10-06T05:18:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.715+0000 7f36029ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35e40051c0 0x7f35fc0c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f36029ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35fc0c61f0 0x7f35fc0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f36029ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35fc151c00 0x7f35fc0c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f36029ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0c50a0 con 0x7f35fc151c00 2023-10-06T05:18:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f36029ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0c4520 con 0x7f35fc0c61f0 2023-10-06T05:18:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f36029ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0c4800 con 0x7f35e40051c0 2023-10-06T05:18:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f3600f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35fc151c00 0x7f35fc0c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f3600f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35fc151c00 0x7f35fc0c7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56858/0 (socket says 172.21.15.31:56858) 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f3600f9c700 1 -- 172.21.15.31:0/974617118 learned_addr learned my addr 172.21.15.31:0/974617118 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f35fb7fe700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35fc0c61f0 0x7f35fc0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f35fbfff700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35e40051c0 0x7f35fc0c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f3600f9c700 1 -- 172.21.15.31:0/974617118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35e40051c0 msgr2=0x7f35fc0c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f3600f9c700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35e40051c0 0x7f35fc0c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f3600f9c700 1 -- 172.21.15.31:0/974617118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35fc0c61f0 msgr2=0x7f35fc0c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f3600f9c700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35fc0c61f0 0x7f35fc0c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.716+0000 7f3600f9c700 1 -- 172.21.15.31:0/974617118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0c4ae0 con 0x7f35fc151c00 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.717+0000 7f3600f9c700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35fc151c00 0x7f35fc0c7e50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f35e400bff0 tx=0x7f35f4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.717+0000 7f35e3fff700 1 -- 172.21.15.31:0/974617118 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4001af0 con 0x7f35fc151c00 2023-10-06T05:18:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.717+0000 7f35e3fff700 1 -- 172.21.15.31:0/974617118 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f400c6d0 con 0x7f35fc151c00 2023-10-06T05:18:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.717+0000 7f36029ff700 1 -- 172.21.15.31:0/974617118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc0c4dc0 con 0x7f35fc151c00 2023-10-06T05:18:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.717+0000 7f35e3fff700 1 -- 172.21.15.31:0/974617118 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4012cf0 con 0x7f35fc151c00 2023-10-06T05:18:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.717+0000 7f36029ff700 1 -- 172.21.15.31:0/974617118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc0c8390 con 0x7f35fc151c00 2023-10-06T05:18:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.718+0000 7f36029ff700 1 -- 172.21.15.31:0/974617118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc155700 con 0x7f35fc151c00 2023-10-06T05:18:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.719+0000 7f35e3fff700 1 -- 172.21.15.31:0/974617118 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f35f4025020 con 0x7f35fc151c00 2023-10-06T05:18:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.720+0000 7f35e3fff700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f35e80670e0 0x7f35e8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.720+0000 7f35e3fff700 1 -- 172.21.15.31:0/974617118 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f35f4086740 con 0x7f35fc151c00 2023-10-06T05:18:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.721+0000 7f35fbfff700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f35e80670e0 0x7f35e8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.721+0000 7f35fbfff700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f35e80670e0 0x7f35e8069590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0093f0 tx=0x7f35ec0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.725+0000 7f35e3fff700 1 -- 172.21.15.31:0/974617118 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35fc155700 con 0x7f35fc151c00 2023-10-06T05:18:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.969+0000 7f36029ff700 1 -- 172.21.15.31:0/974617118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35fc155700 con 0x7f35fc151c00 2023-10-06T05:18:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.970+0000 7f35e3fff700 1 -- 172.21.15.31:0/974617118 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f35fc155700 con 0x7f35fc151c00 2023-10-06T05:18:42.972 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:42.972 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:18:42.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.972+0000 7f36029ff700 1 -- 172.21.15.31:0/974617118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f35e80670e0 msgr2=0x7f35e8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:42.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.972+0000 7f36029ff700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f35e80670e0 0x7f35e8069590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0093f0 tx=0x7f35ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.972+0000 7f36029ff700 1 -- 172.21.15.31:0/974617118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35fc151c00 msgr2=0x7f35fc0c7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.972+0000 7f36029ff700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35fc151c00 0x7f35fc0c7e50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f35e400bff0 tx=0x7f35f4016000 comp rx=0 tx=0).stop 2023-10-06T05:18:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.972+0000 7f36029ff700 1 -- 172.21.15.31:0/974617118 shutdown_connections 2023-10-06T05:18:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.972+0000 7f36029ff700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f35e80670e0 0x7f35e8069590 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.972+0000 7f36029ff700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35fc151c00 0x7f35fc0c7e50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.972+0000 7f36029ff700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35e40051c0 0x7f35fc0c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.972+0000 7f36029ff700 1 --2- 172.21.15.31:0/974617118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35fc0c61f0 0x7f35fc0c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.972+0000 7f36029ff700 1 -- 172.21.15.31:0/974617118 >> 172.21.15.31:0/974617118 conn(0x7f35fc06b620 msgr2=0x7f35fc0ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.973+0000 7f36029ff700 1 -- 172.21.15.31:0/974617118 shutdown_connections 2023-10-06T05:18:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:42.973+0000 7f36029ff700 1 -- 172.21.15.31:0/974617118 wait complete. 2023-10-06T05:18:42.976 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2023-10-06T05:18:42.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:18:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.186+0000 7f80703ff700 1 Processor -- start 2023-10-06T05:18:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.186+0000 7f80703ff700 1 -- start start 2023-10-06T05:18:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.187+0000 7f80703ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80680cd510 0x7f80680b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.187+0000 7f80703ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f806815d2d0 0x7f80680aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.187+0000 7f80703ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80680c3f40 con 0x7f8068159500 2023-10-06T05:18:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.187+0000 7f80703ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80680b2860 con 0x7f806815d2d0 2023-10-06T05:18:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.187+0000 7f80703ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806815d7d0 con 0x7f80680cd510 2023-10-06T05:18:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.188+0000 7f806e99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f806815d2d0 0x7f80680aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.188+0000 7f806e99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f806815d2d0 0x7f80680aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56868/0 (socket says 172.21.15.31:56868) 2023-10-06T05:18:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.188+0000 7f806e99c700 1 -- 172.21.15.31:0/2509037976 learned_addr learned my addr 172.21.15.31:0/2509037976 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.188+0000 7f806d99a700 1 --2- 172.21.15.31:0/2509037976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80680cd510 0x7f80680b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.188+0000 7f806e99c700 1 -- 172.21.15.31:0/2509037976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80680cd510 msgr2=0x7f80680b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.188+0000 7f806e99c700 1 --2- 172.21.15.31:0/2509037976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80680cd510 0x7f80680b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.188+0000 7f806e99c700 1 -- 172.21.15.31:0/2509037976 >> v1:172.21.15.62:6789/0 conn(0x7f8068159500 legacy=0x7f80680cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.188+0000 7f806e99c700 1 -- 172.21.15.31:0/2509037976 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680cb000 con 0x7f806815d2d0 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.188+0000 7f806d99a700 1 --2- 172.21.15.31:0/2509037976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80680cd510 0x7f80680b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.188+0000 7f806e99c700 1 --2- 172.21.15.31:0/2509037976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f806815d2d0 0x7f80680aedd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8060009f20 tx=0x7f8060009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67a9c5c7d785733 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f806d199700 1 -- 172.21.15.31:0/2509037976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806000e070 con 0x7f806815d2d0 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f806d199700 1 -- 172.21.15.31:0/2509037976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806000ba80 con 0x7f806815d2d0 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f806d199700 1 -- 172.21.15.31:0/2509037976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80600092e0 con 0x7f806815d2d0 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f80703ff700 1 -- 172.21.15.31:0/2509037976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f806815d2d0 msgr2=0x7f80680aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f80703ff700 1 --2- 172.21.15.31:0/2509037976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f806815d2d0 0x7f80680aedd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8060009f20 tx=0x7f8060009840 comp rx=0 tx=0).stop 2023-10-06T05:18:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f80703ff700 1 -- 172.21.15.31:0/2509037976 shutdown_connections 2023-10-06T05:18:43.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f80703ff700 1 --2- 172.21.15.31:0/2509037976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f806815d2d0 0x7f80680aedd0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f80703ff700 1 --2- 172.21.15.31:0/2509037976 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80680cd510 0x7f80680b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f80703ff700 1 -- 172.21.15.31:0/2509037976 >> 172.21.15.31:0/2509037976 conn(0x7f806806ec90 msgr2=0x7f806806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:43.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f80703ff700 1 -- 172.21.15.31:0/2509037976 shutdown_connections 2023-10-06T05:18:43.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.189+0000 7f80703ff700 1 -- 172.21.15.31:0/2509037976 wait complete. 2023-10-06T05:18:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.191+0000 7f80703ff700 1 Processor -- start 2023-10-06T05:18:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.191+0000 7f80703ff700 1 -- start start 2023-10-06T05:18:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f80703ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80680cd510 0x7f80680c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f80703ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8068159500 0x7f80680c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f80703ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f806815d2d0 0x7f80680c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f80703ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80680b5160 con 0x7f80680cd510 2023-10-06T05:18:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f80703ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80680b42f0 con 0x7f806815d2d0 2023-10-06T05:18:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f80703ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80680b44e0 con 0x7f8068159500 2023-10-06T05:18:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f806e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80680cd510 0x7f80680c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f806e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80680cd510 0x7f80680c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56882/0 (socket says 172.21.15.31:56882) 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f806e19b700 1 -- 172.21.15.31:0/4125260712 learned_addr learned my addr 172.21.15.31:0/4125260712 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f806d99a700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8068159500 0x7f80680c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.192+0000 7f806e19b700 1 -- 172.21.15.31:0/4125260712 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8068159500 msgr2=0x7f80680c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f806e99c700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f806815d2d0 0x7f80680c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f806e19b700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8068159500 0x7f80680c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f806e19b700 1 -- 172.21.15.31:0/4125260712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f806815d2d0 msgr2=0x7f80680c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f806e19b700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f806815d2d0 0x7f80680c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f806e19b700 1 -- 172.21.15.31:0/4125260712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060009ae0 con 0x7f80680cd510 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f806d99a700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8068159500 0x7f80680c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f806e99c700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f806815d2d0 0x7f80680c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f806e19b700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80680cd510 0x7f80680c74d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f806400c590 tx=0x7f806400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f805e7fc700 1 -- 172.21.15.31:0/4125260712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8064011590 con 0x7f80680cd510 2023-10-06T05:18:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f805e7fc700 1 -- 172.21.15.31:0/4125260712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80640116f0 con 0x7f80680cd510 2023-10-06T05:18:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f805e7fc700 1 -- 172.21.15.31:0/4125260712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8064011ab0 con 0x7f80680cd510 2023-10-06T05:18:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f80703ff700 1 -- 172.21.15.31:0/4125260712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80680b47c0 con 0x7f80680cd510 2023-10-06T05:18:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.193+0000 7f80703ff700 1 -- 172.21.15.31:0/4125260712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8068157fa0 con 0x7f80680cd510 2023-10-06T05:18:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.194+0000 7f80703ff700 1 -- 172.21.15.31:0/4125260712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80680b52a0 con 0x7f80680cd510 2023-10-06T05:18:43.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.195+0000 7f805e7fc700 1 -- 172.21.15.31:0/4125260712 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f806401b3b0 con 0x7f80680cd510 2023-10-06T05:18:43.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.196+0000 7f805e7fc700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f804c0670e0 0x7f804c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.196+0000 7f805e7fc700 1 -- 172.21.15.31:0/4125260712 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8064083fa0 con 0x7f80680cd510 2023-10-06T05:18:43.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.196+0000 7f806d99a700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f804c0670e0 0x7f804c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.197+0000 7f806d99a700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f804c0670e0 0x7f804c069590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8058006ea0 tx=0x7f8058008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:43.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.200+0000 7f805e7fc700 1 -- 172.21.15.31:0/4125260712 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80680b52a0 con 0x7f80680cd510 2023-10-06T05:18:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.404+0000 7f80703ff700 1 -- 172.21.15.31:0/4125260712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80680b52a0 con 0x7f80680cd510 2023-10-06T05:18:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.405+0000 7f805e7fc700 1 -- 172.21.15.31:0/4125260712 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f80680b52a0 con 0x7f80680cd510 2023-10-06T05:18:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:43.407 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":44,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:40.441086+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:36.639713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:37.929704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:18:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 -- 172.21.15.31:0/4125260712 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f804c0670e0 msgr2=0x7f804c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f804c0670e0 0x7f804c069590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8058006ea0 tx=0x7f8058008040 comp rx=0 tx=0).stop 2023-10-06T05:18:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 -- 172.21.15.31:0/4125260712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80680cd510 msgr2=0x7f80680c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80680cd510 0x7f80680c74d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f806400c590 tx=0x7f806400c950 comp rx=0 tx=0).stop 2023-10-06T05:18:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 -- 172.21.15.31:0/4125260712 shutdown_connections 2023-10-06T05:18:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f806815d2d0 0x7f80680c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f804c0670e0 0x7f804c069590 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80680cd510 0x7f80680c74d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 --2- 172.21.15.31:0/4125260712 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8068159500 0x7f80680c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 -- 172.21.15.31:0/4125260712 >> 172.21.15.31:0/4125260712 conn(0x7f806806ec90 msgr2=0x7f8068158890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 -- 172.21.15.31:0/4125260712 shutdown_connections 2023-10-06T05:18:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.407+0000 7f80703ff700 1 -- 172.21.15.31:0/4125260712 wait complete. 2023-10-06T05:18:43.427 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:18:43.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:43.428 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-10-06T05:18:43.426+0000 7ff2e61f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:18:43.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.621+0000 7f1cfb9ff700 1 Processor -- start 2023-10-06T05:18:43.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.622+0000 7f1cfb9ff700 1 -- start start 2023-10-06T05:18:43.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.623+0000 7f1cfb9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cf415d2d0 0x7f1cf40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.623+0000 7f1cfb9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf40c3f40 con 0x7f1cf4159500 2023-10-06T05:18:43.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.623+0000 7f1cfb9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf40cb000 con 0x7f1cf40cd510 2023-10-06T05:18:43.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.623+0000 7f1cfb9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf40b2860 con 0x7f1cf415d2d0 2023-10-06T05:18:43.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cf979b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1cf4159500 0x7f1cf40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52244/0 (socket says 172.21.15.31:52244) 2023-10-06T05:18:43.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cf979b700 1 -- 172.21.15.31:0/1595219733 learned_addr learned my addr 172.21.15.31:0/1595219733 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:43.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cf9f9c700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cf415d2d0 0x7f1cf40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492918608 0 0) 0x7f1cf40c3f40 con 0x7f1cf4159500 2023-10-06T05:18:43.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce0003570 con 0x7f1cf4159500 2023-10-06T05:18:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748368396 0 0) 0x7f1cf40cb000 con 0x7f1cf40cd510 2023-10-06T05:18:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf40c3f40 con 0x7f1cf40cd510 2023-10-06T05:18:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1861308273 0 0) 0x7f1ce0003570 con 0x7f1cf4159500 2023-10-06T05:18:43.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cf415d2d0 msgr2=0x7f1cf40b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cebfff700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cf415d2d0 0x7f1cf40b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 >> v1:172.21.15.62:6789/0 conn(0x7f1cf40cd510 legacy=0x7f1cf40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf415d7d0 con 0x7f1cf4159500 2023-10-06T05:18:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.624+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f1cf0001470 con 0x7f1cf4159500 2023-10-06T05:18:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 >> v1:172.21.15.31:6789/0 conn(0x7f1cf4159500 legacy=0x7f1cf40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cebfff700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ce0003c20 0x7f1ce0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cebfff700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce0004810 0x7f1ce0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cebfff700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cf415d2d0 0x7f1ce00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf40b2860 con 0x7f1ce0004810 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce000bb40 con 0x7f1ce0003c20 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce000bd00 con 0x7f1cf415d2d0 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cf8f9a700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce0004810 0x7f1ce0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cf979b700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ce0003c20 0x7f1ce0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cf9f9c700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cf415d2d0 0x7f1ce00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cf8f9a700 1 -- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cf415d2d0 msgr2=0x7f1ce00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cf8f9a700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1cf415d2d0 0x7f1ce00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cf8f9a700 1 -- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ce0003c20 msgr2=0x7f1ce0004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cf8f9a700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ce0003c20 0x7f1ce0004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.625+0000 7f1cf8f9a700 1 -- 172.21.15.31:0/1595219733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce0003570 con 0x7f1ce0004810 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cf8f9a700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce0004810 0x7f1ce0004c80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40cd910 tx=0x7f1ce4007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a029d83e35c2ed66 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40013f0 con 0x7f1ce0004810 2023-10-06T05:18:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce400c5d0 con 0x7f1ce0004810 2023-10-06T05:18:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cebfff700 1 -- 172.21.15.31:0/1595219733 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400c990 con 0x7f1ce0004810 2023-10-06T05:18:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/1595219733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce0004810 msgr2=0x7f1ce0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cfb9ff700 1 --2- 172.21.15.31:0/1595219733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce0004810 0x7f1ce0004c80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40cd910 tx=0x7f1ce4007d80 comp rx=0 tx=0).stop 2023-10-06T05:18:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cf9f9c700 1 -- 172.21.15.31:0/1595219733 reap_dead start 2023-10-06T05:18:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/1595219733 shutdown_connections 2023-10-06T05:18:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/1595219733 >> 172.21.15.31:0/1595219733 conn(0x7f1cf406ec90 msgr2=0x7f1cf406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/1595219733 shutdown_connections 2023-10-06T05:18:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.626+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/1595219733 wait complete. 2023-10-06T05:18:43.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.628+0000 7f1cfb9ff700 1 Processor -- start 2023-10-06T05:18:43.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.628+0000 7f1cfb9ff700 1 -- start start 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.629+0000 7f1cfb9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ce0004810 0x7f1cf40b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.629+0000 7f1cfb9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf40b59b0 0x7f1cf4150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.629+0000 7f1cfb9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf415a8a0 0x7f1cf415b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.629+0000 7f1cfb9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf40b35c0 con 0x7f1cf415a8a0 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.629+0000 7f1cfb9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41598d0 con 0x7f1cf40b59b0 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.629+0000 7f1cfb9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf40b5e20 con 0x7f1ce0004810 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.629+0000 7f1cf9f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf415a8a0 0x7f1cf415b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.629+0000 7f1cf9f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf415a8a0 0x7f1cf415b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56892/0 (socket says 172.21.15.31:56892) 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.629+0000 7f1cf9f9c700 1 -- 172.21.15.31:0/3751364822 learned_addr learned my addr 172.21.15.31:0/3751364822 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1cf8f9a700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf40b59b0 0x7f1cf4150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1cf979b700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ce0004810 0x7f1cf40b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1cf9f9c700 1 -- 172.21.15.31:0/3751364822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ce0004810 msgr2=0x7f1cf40b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1cf9f9c700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ce0004810 0x7f1cf40b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1cf9f9c700 1 -- 172.21.15.31:0/3751364822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf40b59b0 msgr2=0x7f1cf4150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1cf9f9c700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf40b59b0 0x7f1cf4150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1cf9f9c700 1 -- 172.21.15.31:0/3751364822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf415a410 con 0x7f1cf415a8a0 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1cf9f9c700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf415a8a0 0x7f1cf415b8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1cf415d200 tx=0x7f1cec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1ce97fa700 1 -- 172.21.15.31:0/3751364822 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00e070 con 0x7f1cf415a8a0 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1ce97fa700 1 -- 172.21.15.31:0/3751364822 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec011b80 con 0x7f1cf415a8a0 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1ce97fa700 1 -- 172.21.15.31:0/3751364822 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec016470 con 0x7f1cf415a8a0 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.630+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/3751364822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf41511d0 con 0x7f1cf415a8a0 2023-10-06T05:18:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.631+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/3751364822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf4159ae0 con 0x7f1cf415a8a0 2023-10-06T05:18:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.632+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/3751364822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf415a5a0 con 0x7f1cf415a8a0 2023-10-06T05:18:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.633+0000 7f1ce97fa700 1 -- 172.21.15.31:0/3751364822 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f1cec0168a0 con 0x7f1cf415a8a0 2023-10-06T05:18:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.633+0000 7f1ce97fa700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1cd80670e0 0x7f1cd8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.633+0000 7f1ce97fa700 1 -- 172.21.15.31:0/3751364822 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1cec0854c0 con 0x7f1cf415a8a0 2023-10-06T05:18:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.634+0000 7f1cf979b700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1cd80670e0 0x7f1cd8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.634+0000 7f1cf979b700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1cd80670e0 0x7f1cd8069590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000e10 tx=0x7f1cf0001df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.637+0000 7f1ce97fa700 1 -- 172.21.15.31:0/3751364822 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1cec0521a0 con 0x7f1cf415a8a0 2023-10-06T05:18:43.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.833+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/3751364822 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1cf404c380 con 0x7f1cd80670e0 2023-10-06T05:18:43.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.843+0000 7f1ce97fa700 1 -- 172.21.15.31:0/3751364822 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308313 (secure 0 0 0) 0x7f1cf404c380 con 0x7f1cd80670e0 2023-10-06T05:18:43.851 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:43.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.847+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/3751364822 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1cd80670e0 msgr2=0x7f1cd8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.847+0000 7f1cfb9ff700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1cd80670e0 0x7f1cd8069590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000e10 tx=0x7f1cf0001df0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.847+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/3751364822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf415a8a0 msgr2=0x7f1cf415b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.847+0000 7f1cfb9ff700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf415a8a0 0x7f1cf415b8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1cf415d200 tx=0x7f1cec00b040 comp rx=0 tx=0).stop 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.847+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/3751364822 shutdown_connections 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.847+0000 7f1cfb9ff700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1cd80670e0 0x7f1cd8069590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.847+0000 7f1cfb9ff700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cf415a8a0 0x7f1cf415b8d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.847+0000 7f1cfb9ff700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ce0004810 0x7f1cf40b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.847+0000 7f1cfb9ff700 1 --2- 172.21.15.31:0/3751364822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf40b59b0 0x7f1cf4150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.847+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/3751364822 >> 172.21.15.31:0/3751364822 conn(0x7f1cf406ec90 msgr2=0x7f1cf40af100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.848+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/3751364822 shutdown_connections 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:43.848+0000 7f1cfb9ff700 1 -- 172.21.15.31:0/3751364822 wait complete. 2023-10-06T05:18:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:18:43.867 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-10-06T05:18:42.081724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383280,"kb_used_data":4976,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591440,"statfs":{"total":773094113280,"available":770653634560,"internally_reserved":0,"allocated":5095424,"data_stored":2819864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.648167"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453619+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:33.498919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448171+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:24.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.449706+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447164+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.444195+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455710+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:17.100290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448958+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446650+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:27.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446143+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:10.283521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453249+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:05.369497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455141+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:34.252333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445402+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455343+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:52.371640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451248+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:10.283521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447669+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454977+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:50.655754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452837+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:19.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446943+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448342+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.456888+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:33.498919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451223+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:56.384273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445385+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451398+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:56.125739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.449458+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452400+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:06.022509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.444580+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454129+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:39.631263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.456353+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:05.369497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450548+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447835+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:36.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451478+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:52.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451233+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446889+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464327+0000","last_change":"2023-10-06T05:18:38.464327+0000","last_active":"2023-10-06T05:18:38.464327+0000","last_peered":"2023-10-06T05:18:38.464327+0000","last_clean":"2023-10-06T05:18:38.464327+0000","last_became_active":"2023-10-06T05:18:38.464063+0000","last_became_peered":"2023-10-06T05:18:38.464063+0000","last_unstale":"2023-10-06T05:18:38.464327+0000","last_undegraded":"2023-10-06T05:18:38.464327+0000","last_fullsized":"2023-10-06T05:18:38.464327+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:47.847969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450735+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:28.365507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467137+0000","last_change":"2023-10-06T05:18:38.467137+0000","last_active":"2023-10-06T05:18:38.467137+0000","last_peered":"2023-10-06T05:18:38.467137+0000","last_clean":"2023-10-06T05:18:38.467137+0000","last_became_active":"2023-10-06T05:18:38.466871+0000","last_became_peered":"2023-10-06T05:18:38.466871+0000","last_unstale":"2023-10-06T05:18:38.467137+0000","last_undegraded":"2023-10-06T05:18:38.467137+0000","last_fullsized":"2023-10-06T05:18:38.467137+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:26.910024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.457317+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:39.631263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.465625+0000","last_change":"2023-10-06T05:18:38.465625+0000","last_active":"2023-10-06T05:18:38.465625+0000","last_peered":"2023-10-06T05:18:38.465625+0000","last_clean":"2023-10-06T05:18:38.465625+0000","last_became_active":"2023-10-06T05:18:38.465367+0000","last_became_peered":"2023-10-06T05:18:38.465367+0000","last_unstale":"2023-10-06T05:18:38.465625+0000","last_undegraded":"2023-10-06T05:18:38.465625+0000","last_fullsized":"2023-10-06T05:18:38.465625+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:55.377855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445717+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466328+0000","last_change":"2023-10-06T05:18:38.466328+0000","last_active":"2023-10-06T05:18:38.466328+0000","last_peered":"2023-10-06T05:18:38.466328+0000","last_clean":"2023-10-06T05:18:38.466328+0000","last_became_active":"2023-10-06T05:18:38.466033+0000","last_became_peered":"2023-10-06T05:18:38.466033+0000","last_unstale":"2023-10-06T05:18:38.466328+0000","last_undegraded":"2023-10-06T05:18:38.466328+0000","last_fullsized":"2023-10-06T05:18:38.466328+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:23.634297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443349+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464675+0000","last_change":"2023-10-06T05:18:38.464675+0000","last_active":"2023-10-06T05:18:38.464675+0000","last_peered":"2023-10-06T05:18:38.464675+0000","last_clean":"2023-10-06T05:18:38.464675+0000","last_became_active":"2023-10-06T05:18:38.464409+0000","last_became_peered":"2023-10-06T05:18:38.464409+0000","last_unstale":"2023-10-06T05:18:38.464675+0000","last_undegraded":"2023-10-06T05:18:38.464675+0000","last_fullsized":"2023-10-06T05:18:38.464675+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:09.080286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447205+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.465029+0000","last_change":"2023-10-06T05:18:38.465029+0000","last_active":"2023-10-06T05:18:38.465029+0000","last_peered":"2023-10-06T05:18:38.465029+0000","last_clean":"2023-10-06T05:18:38.465029+0000","last_became_active":"2023-10-06T05:18:38.464703+0000","last_became_peered":"2023-10-06T05:18:38.464703+0000","last_unstale":"2023-10-06T05:18:38.465029+0000","last_undegraded":"2023-10-06T05:18:38.465029+0000","last_fullsized":"2023-10-06T05:18:38.465029+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446502+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460564+0000","last_change":"2023-10-06T05:18:38.460564+0000","last_active":"2023-10-06T05:18:38.460564+0000","last_peered":"2023-10-06T05:18:38.460564+0000","last_clean":"2023-10-06T05:18:38.460564+0000","last_became_active":"2023-10-06T05:18:38.460269+0000","last_became_peered":"2023-10-06T05:18:38.460269+0000","last_unstale":"2023-10-06T05:18:38.460564+0000","last_undegraded":"2023-10-06T05:18:38.460564+0000","last_fullsized":"2023-10-06T05:18:38.460564+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:33.245915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447681+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:13.626073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.461183+0000","last_change":"2023-10-06T05:18:38.461183+0000","last_active":"2023-10-06T05:18:38.461183+0000","last_peered":"2023-10-06T05:18:38.461183+0000","last_clean":"2023-10-06T05:18:38.461183+0000","last_became_active":"2023-10-06T05:18:38.460893+0000","last_became_peered":"2023-10-06T05:18:38.460893+0000","last_unstale":"2023-10-06T05:18:38.461183+0000","last_undegraded":"2023-10-06T05:18:38.461183+0000","last_fullsized":"2023-10-06T05:18:38.461183+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:12.619655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443147+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458370+0000","last_change":"2023-10-06T05:18:38.458370+0000","last_active":"2023-10-06T05:18:38.458370+0000","last_peered":"2023-10-06T05:18:38.458370+0000","last_clean":"2023-10-06T05:18:38.458370+0000","last_became_active":"2023-10-06T05:18:38.458104+0000","last_became_peered":"2023-10-06T05:18:38.458104+0000","last_unstale":"2023-10-06T05:18:38.458370+0000","last_undegraded":"2023-10-06T05:18:38.458370+0000","last_fullsized":"2023-10-06T05:18:38.458370+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:41.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451319+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463433+0000","last_change":"2023-10-06T05:18:38.463433+0000","last_active":"2023-10-06T05:18:38.463433+0000","last_peered":"2023-10-06T05:18:38.463433+0000","last_clean":"2023-10-06T05:18:38.463433+0000","last_became_active":"2023-10-06T05:18:38.463176+0000","last_became_peered":"2023-10-06T05:18:38.463176+0000","last_unstale":"2023-10-06T05:18:38.463433+0000","last_undegraded":"2023-10-06T05:18:38.463433+0000","last_fullsized":"2023-10-06T05:18:38.463433+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:41.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450043+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.461448+0000","last_change":"2023-10-06T05:18:38.461448+0000","last_active":"2023-10-06T05:18:38.461448+0000","last_peered":"2023-10-06T05:18:38.461448+0000","last_clean":"2023-10-06T05:18:38.461448+0000","last_became_active":"2023-10-06T05:18:38.461184+0000","last_became_peered":"2023-10-06T05:18:38.461184+0000","last_unstale":"2023-10-06T05:18:38.461448+0000","last_undegraded":"2023-10-06T05:18:38.461448+0000","last_fullsized":"2023-10-06T05:18:38.461448+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:34.215548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443576+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460848+0000","last_change":"2023-10-06T05:18:38.460848+0000","last_active":"2023-10-06T05:18:38.460848+0000","last_peered":"2023-10-06T05:18:38.460848+0000","last_clean":"2023-10-06T05:18:38.460848+0000","last_became_active":"2023-10-06T05:18:38.460598+0000","last_became_peered":"2023-10-06T05:18:38.460598+0000","last_unstale":"2023-10-06T05:18:38.460848+0000","last_undegraded":"2023-10-06T05:18:38.460848+0000","last_fullsized":"2023-10-06T05:18:38.460848+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:12.765494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454251+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463872+0000","last_change":"2023-10-06T05:18:38.463872+0000","last_active":"2023-10-06T05:18:38.463872+0000","last_peered":"2023-10-06T05:18:38.463872+0000","last_clean":"2023-10-06T05:18:38.463872+0000","last_became_active":"2023-10-06T05:18:38.463587+0000","last_became_peered":"2023-10-06T05:18:38.463587+0000","last_unstale":"2023-10-06T05:18:38.463872+0000","last_undegraded":"2023-10-06T05:18:38.463872+0000","last_fullsized":"2023-10-06T05:18:38.463872+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:27.960703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448983+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466701+0000","last_change":"2023-10-06T05:18:38.466701+0000","last_active":"2023-10-06T05:18:38.466701+0000","last_peered":"2023-10-06T05:18:38.466701+0000","last_clean":"2023-10-06T05:18:38.466701+0000","last_became_active":"2023-10-06T05:18:38.466409+0000","last_became_peered":"2023-10-06T05:18:38.466409+0000","last_unstale":"2023-10-06T05:18:38.466701+0000","last_undegraded":"2023-10-06T05:18:38.466701+0000","last_fullsized":"2023-10-06T05:18:38.466701+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:26.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446996+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:52.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464573+0000","last_change":"2023-10-06T05:18:38.464573+0000","last_active":"2023-10-06T05:18:38.464573+0000","last_peered":"2023-10-06T05:18:38.464573+0000","last_clean":"2023-10-06T05:18:38.464573+0000","last_became_active":"2023-10-06T05:18:38.464303+0000","last_became_peered":"2023-10-06T05:18:38.464303+0000","last_unstale":"2023-10-06T05:18:38.464573+0000","last_undegraded":"2023-10-06T05:18:38.464573+0000","last_fullsized":"2023-10-06T05:18:38.464573+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451911+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:41.261742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458013+0000","last_change":"2023-10-06T05:18:38.458013+0000","last_active":"2023-10-06T05:18:38.458013+0000","last_peered":"2023-10-06T05:18:38.458013+0000","last_clean":"2023-10-06T05:18:38.458013+0000","last_became_active":"2023-10-06T05:18:38.457708+0000","last_became_peered":"2023-10-06T05:18:38.457708+0000","last_unstale":"2023-10-06T05:18:38.458013+0000","last_undegraded":"2023-10-06T05:18:38.458013+0000","last_fullsized":"2023-10-06T05:18:38.458013+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:35.907402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445801+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466371+0000","last_change":"2023-10-06T05:18:38.466371+0000","last_active":"2023-10-06T05:18:38.466371+0000","last_peered":"2023-10-06T05:18:38.466371+0000","last_clean":"2023-10-06T05:18:38.466371+0000","last_became_active":"2023-10-06T05:18:38.466099+0000","last_became_peered":"2023-10-06T05:18:38.466099+0000","last_unstale":"2023-10-06T05:18:38.466371+0000","last_undegraded":"2023-10-06T05:18:38.466371+0000","last_fullsized":"2023-10-06T05:18:38.466371+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:12.765494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454588+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:51.516975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.456427+0000","last_change":"2023-10-06T05:18:38.456427+0000","last_active":"2023-10-06T05:18:38.456427+0000","last_peered":"2023-10-06T05:18:38.456427+0000","last_clean":"2023-10-06T05:18:38.456427+0000","last_became_active":"2023-10-06T05:18:38.456172+0000","last_became_peered":"2023-10-06T05:18:38.456172+0000","last_unstale":"2023-10-06T05:18:38.456427+0000","last_undegraded":"2023-10-06T05:18:38.456427+0000","last_fullsized":"2023-10-06T05:18:38.456427+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:13.476914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443171+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466443+0000","last_change":"2023-10-06T05:18:38.466443+0000","last_active":"2023-10-06T05:18:38.466443+0000","last_peered":"2023-10-06T05:18:38.466443+0000","last_clean":"2023-10-06T05:18:38.466443+0000","last_became_active":"2023-10-06T05:18:38.465603+0000","last_became_peered":"2023-10-06T05:18:38.465603+0000","last_unstale":"2023-10-06T05:18:38.466443+0000","last_undegraded":"2023-10-06T05:18:38.466443+0000","last_fullsized":"2023-10-06T05:18:38.466443+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:08.665979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454720+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:09.410169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467188+0000","last_change":"2023-10-06T05:18:38.467188+0000","last_active":"2023-10-06T05:18:38.467188+0000","last_peered":"2023-10-06T05:18:38.467188+0000","last_clean":"2023-10-06T05:18:38.467188+0000","last_became_active":"2023-10-06T05:18:38.466475+0000","last_became_peered":"2023-10-06T05:18:38.466475+0000","last_unstale":"2023-10-06T05:18:38.467188+0000","last_undegraded":"2023-10-06T05:18:38.467188+0000","last_fullsized":"2023-10-06T05:18:38.467188+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:22.230340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451137+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:11.273331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.468654+0000","last_change":"2023-10-06T05:18:38.468654+0000","last_active":"2023-10-06T05:18:38.468654+0000","last_peered":"2023-10-06T05:18:38.468654+0000","last_clean":"2023-10-06T05:18:38.468654+0000","last_became_active":"2023-10-06T05:18:38.468394+0000","last_became_peered":"2023-10-06T05:18:38.468394+0000","last_unstale":"2023-10-06T05:18:38.468654+0000","last_undegraded":"2023-10-06T05:18:38.468654+0000","last_fullsized":"2023-10-06T05:18:38.468654+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:56.721340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452992+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:43.862504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460519+0000","last_change":"2023-10-06T05:18:38.460519+0000","last_active":"2023-10-06T05:18:38.460519+0000","last_peered":"2023-10-06T05:18:38.460519+0000","last_clean":"2023-10-06T05:18:38.460519+0000","last_became_active":"2023-10-06T05:18:38.460215+0000","last_became_peered":"2023-10-06T05:18:38.460215+0000","last_unstale":"2023-10-06T05:18:38.460519+0000","last_undegraded":"2023-10-06T05:18:38.460519+0000","last_fullsized":"2023-10-06T05:18:38.460519+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:12.619655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451676+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:27.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.462389+0000","last_change":"2023-10-06T05:18:38.462389+0000","last_active":"2023-10-06T05:18:38.462389+0000","last_peered":"2023-10-06T05:18:38.462389+0000","last_clean":"2023-10-06T05:18:38.462389+0000","last_became_active":"2023-10-06T05:18:38.462121+0000","last_became_peered":"2023-10-06T05:18:38.462121+0000","last_unstale":"2023-10-06T05:18:38.462389+0000","last_undegraded":"2023-10-06T05:18:38.462389+0000","last_fullsized":"2023-10-06T05:18:38.462389+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:23.179454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453051+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:15.086481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.457211+0000","last_change":"2023-10-06T05:18:38.457211+0000","last_active":"2023-10-06T05:18:38.457211+0000","last_peered":"2023-10-06T05:18:38.457211+0000","last_clean":"2023-10-06T05:18:38.457211+0000","last_became_active":"2023-10-06T05:18:38.456921+0000","last_became_peered":"2023-10-06T05:18:38.456921+0000","last_unstale":"2023-10-06T05:18:38.457211+0000","last_undegraded":"2023-10-06T05:18:38.457211+0000","last_fullsized":"2023-10-06T05:18:38.457211+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:11.790017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448636+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460312+0000","last_change":"2023-10-06T05:18:38.460312+0000","last_active":"2023-10-06T05:18:38.460312+0000","last_peered":"2023-10-06T05:18:38.460312+0000","last_clean":"2023-10-06T05:18:38.460312+0000","last_became_active":"2023-10-06T05:18:38.460043+0000","last_became_peered":"2023-10-06T05:18:38.460043+0000","last_unstale":"2023-10-06T05:18:38.460312+0000","last_undegraded":"2023-10-06T05:18:38.460312+0000","last_fullsized":"2023-10-06T05:18:38.460312+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:09.080286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.444681+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:18:37.444681+0000","last_peered":"2023-10-06T05:18:37.444681+0000","last_clean":"2023-10-06T05:18:37.444681+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:18:37.444681+0000","last_undegraded":"2023-10-06T05:18:37.444681+0000","last_fullsized":"2023-10-06T05:18:37.444681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452130+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458978+0000","last_change":"2023-10-06T05:18:38.458978+0000","last_active":"2023-10-06T05:18:38.458978+0000","last_peered":"2023-10-06T05:18:38.458978+0000","last_clean":"2023-10-06T05:18:38.458978+0000","last_became_active":"2023-10-06T05:18:38.458700+0000","last_became_peered":"2023-10-06T05:18:38.458700+0000","last_unstale":"2023-10-06T05:18:38.458978+0000","last_undegraded":"2023-10-06T05:18:38.458978+0000","last_fullsized":"2023-10-06T05:18:38.458978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:23.179454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.436719+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:18:37.436719+0000","last_peered":"2023-10-06T05:18:37.436719+0000","last_clean":"2023-10-06T05:18:37.436719+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:18:37.436719+0000","last_undegraded":"2023-10-06T05:18:37.436719+0000","last_fullsized":"2023-10-06T05:18:37.436719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455206+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:44.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460411+0000","last_change":"2023-10-06T05:18:38.460411+0000","last_active":"2023-10-06T05:18:38.460411+0000","last_peered":"2023-10-06T05:18:38.460411+0000","last_clean":"2023-10-06T05:18:38.460411+0000","last_became_active":"2023-10-06T05:18:38.460142+0000","last_became_peered":"2023-10-06T05:18:38.460142+0000","last_unstale":"2023-10-06T05:18:38.460411+0000","last_undegraded":"2023-10-06T05:18:38.460411+0000","last_fullsized":"2023-10-06T05:18:38.460411+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:10.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.434247+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:18:37.434247+0000","last_peered":"2023-10-06T05:18:37.434247+0000","last_clean":"2023-10-06T05:18:37.434247+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:18:37.434247+0000","last_undegraded":"2023-10-06T05:18:37.434247+0000","last_fullsized":"2023-10-06T05:18:37.434247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":180,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437257+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:18:37.437257+0000","last_peered":"2023-10-06T05:18:37.437257+0000","last_clean":"2023-10-06T05:18:37.437257+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:18:37.437257+0000","last_undegraded":"2023-10-06T05:18:37.437257+0000","last_fullsized":"2023-10-06T05:18:37.437257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445051+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463399+0000","last_change":"2023-10-06T05:18:38.463399+0000","last_active":"2023-10-06T05:18:38.463399+0000","last_peered":"2023-10-06T05:18:38.463399+0000","last_clean":"2023-10-06T05:18:38.463399+0000","last_became_active":"2023-10-06T05:18:38.463130+0000","last_became_peered":"2023-10-06T05:18:38.463130+0000","last_unstale":"2023-10-06T05:18:38.463399+0000","last_undegraded":"2023-10-06T05:18:38.463399+0000","last_fullsized":"2023-10-06T05:18:38.463399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:35.907402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437352+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:18:37.437352+0000","last_peered":"2023-10-06T05:18:37.437352+0000","last_clean":"2023-10-06T05:18:37.437352+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:18:37.437352+0000","last_undegraded":"2023-10-06T05:18:37.437352+0000","last_fullsized":"2023-10-06T05:18:37.437352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.457853+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:51.516975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463382+0000","last_change":"2023-10-06T05:18:38.463382+0000","last_active":"2023-10-06T05:18:38.463382+0000","last_peered":"2023-10-06T05:18:38.463382+0000","last_clean":"2023-10-06T05:18:38.463382+0000","last_became_active":"2023-10-06T05:18:38.463112+0000","last_became_peered":"2023-10-06T05:18:38.463112+0000","last_unstale":"2023-10-06T05:18:38.463382+0000","last_undegraded":"2023-10-06T05:18:38.463382+0000","last_fullsized":"2023-10-06T05:18:38.463382+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:37.839836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437251+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:18:37.437251+0000","last_peered":"2023-10-06T05:18:37.437251+0000","last_clean":"2023-10-06T05:18:37.437251+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:18:37.437251+0000","last_undegraded":"2023-10-06T05:18:37.437251+0000","last_fullsized":"2023-10-06T05:18:37.437251+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450529+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:42.744594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466799+0000","last_change":"2023-10-06T05:18:38.466799+0000","last_active":"2023-10-06T05:18:38.466799+0000","last_peered":"2023-10-06T05:18:38.466799+0000","last_clean":"2023-10-06T05:18:38.466799+0000","last_became_active":"2023-10-06T05:18:38.466486+0000","last_became_peered":"2023-10-06T05:18:38.466486+0000","last_unstale":"2023-10-06T05:18:38.466799+0000","last_undegraded":"2023-10-06T05:18:38.466799+0000","last_fullsized":"2023-10-06T05:18:38.466799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:04.252646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.439395+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:18:37.439395+0000","last_peered":"2023-10-06T05:18:37.439395+0000","last_clean":"2023-10-06T05:18:37.439395+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:18:37.439395+0000","last_undegraded":"2023-10-06T05:18:37.439395+0000","last_fullsized":"2023-10-06T05:18:37.439395+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445017+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467529+0000","last_change":"2023-10-06T05:18:38.467529+0000","last_active":"2023-10-06T05:18:38.467529+0000","last_peered":"2023-10-06T05:18:38.467529+0000","last_clean":"2023-10-06T05:18:38.467529+0000","last_became_active":"2023-10-06T05:18:38.467262+0000","last_became_peered":"2023-10-06T05:18:38.467262+0000","last_unstale":"2023-10-06T05:18:38.467529+0000","last_undegraded":"2023-10-06T05:18:38.467529+0000","last_fullsized":"2023-10-06T05:18:38.467529+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:34.215548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.440499+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:18:37.440499+0000","last_peered":"2023-10-06T05:18:37.440499+0000","last_clean":"2023-10-06T05:18:37.440499+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:18:37.440499+0000","last_undegraded":"2023-10-06T05:18:37.440499+0000","last_fullsized":"2023-10-06T05:18:37.440499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452193+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:39.628432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460634+0000","last_change":"2023-10-06T05:18:38.460634+0000","last_active":"2023-10-06T05:18:38.460634+0000","last_peered":"2023-10-06T05:18:38.460634+0000","last_clean":"2023-10-06T05:18:38.460634+0000","last_became_active":"2023-10-06T05:18:38.459850+0000","last_became_peered":"2023-10-06T05:18:38.459850+0000","last_unstale":"2023-10-06T05:18:38.460634+0000","last_undegraded":"2023-10-06T05:18:38.460634+0000","last_fullsized":"2023-10-06T05:18:38.460634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:10.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.440631+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:18:37.440631+0000","last_peered":"2023-10-06T05:18:37.440631+0000","last_clean":"2023-10-06T05:18:37.440631+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:18:37.440631+0000","last_undegraded":"2023-10-06T05:18:37.440631+0000","last_fullsized":"2023-10-06T05:18:37.440631+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298348,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073492,"statfs":{"total":96636764160,"available":96331255808,"internally_reserved":0,"allocated":1085440,"data_stored":796913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78300000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.871},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.62},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77400000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298336,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073504,"statfs":{"total":96636764160,"available":96331268096,"internally_reserved":0,"allocated":1073152,"data_stored":790297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.433}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.95999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.94099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.006}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.82399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297756,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074084,"statfs":{"total":96636764160,"available":96331862016,"internally_reserved":0,"allocated":479232,"data_stored":199929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.51300000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.749},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.81899999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.91700000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.84599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.004}]},{"osd":7,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.0169999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.371}]},{"osd":1,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.754},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.78100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":7.742,"5min":7.742,"15min":7.742},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.88500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.87},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.85799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2780000000000005,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":9.25,"5min":9.25,"15min":9.25},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.108000000000001,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.95799999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":10.384,"5min":10.384,"15min":10.384},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":9.1699999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.205,"5min":10.205,"15min":10.205},"last":1.038},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.83299999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":1.113}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.0920000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:43.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:44.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.101+0000 7f12727ff700 1 Processor -- start 2023-10-06T05:18:44.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.101+0000 7f12727ff700 1 -- start start 2023-10-06T05:18:44.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.102+0000 7f12727ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f126c1595a0 0x7f126c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f12727ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f126c0cc0d0 0x7f126c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f12727ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f126c068bb0 0x7f126c0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f12727ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126c069010 con 0x7f126c068bb0 2023-10-06T05:18:44.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f12727ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126c0c95b0 con 0x7f126c0cc0d0 2023-10-06T05:18:44.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f12727ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c0c9890 con 0x7f126c1595a0 2023-10-06T05:18:44.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f1270d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f126c068bb0 0x7f126c0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f1270d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f126c068bb0 0x7f126c0c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56900/0 (socket says 172.21.15.31:56900) 2023-10-06T05:18:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f1270d9c700 1 -- 172.21.15.31:0/663169092 learned_addr learned my addr 172.21.15.31:0/663169092 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f126b7fe700 1 --2- 172.21.15.31:0/663169092 >> v2:172.21.15.62:3300/0 conn(0x7f126c0cc0d0 0x7f126c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f126bfff700 1 --2- 172.21.15.31:0/663169092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f126c1595a0 0x7f126c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f1270d9c700 1 -- 172.21.15.31:0/663169092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f126c1595a0 msgr2=0x7f126c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f1270d9c700 1 --2- 172.21.15.31:0/663169092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f126c1595a0 0x7f126c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f1270d9c700 1 -- 172.21.15.31:0/663169092 >> v2:172.21.15.62:3300/0 conn(0x7f126c0cc0d0 msgr2=0x7f126c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f1270d9c700 1 --2- 172.21.15.31:0/663169092 >> v2:172.21.15.62:3300/0 conn(0x7f126c0cc0d0 0x7f126c068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f1270d9c700 1 -- 172.21.15.31:0/663169092 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c0c9b70 con 0x7f126c068bb0 2023-10-06T05:18:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.103+0000 7f126b7fe700 1 --2- 172.21.15.31:0/663169092 >> v2:172.21.15.62:3300/0 conn(0x7f126c0cc0d0 0x7f126c068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.104+0000 7f1270d9c700 1 --2- 172.21.15.31:0/663169092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 0x7f126c0c8930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f126c06f290 tx=0x7f1264009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96e6211561abd14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:44.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.104+0000 7f126affd700 1 -- 172.21.15.31:0/663169092 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126400e070 con 0x7f126c068bb0 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.104+0000 7f126affd700 1 -- 172.21.15.31:0/663169092 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126400ba50 con 0x7f126c068bb0 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.104+0000 7f126affd700 1 -- 172.21.15.31:0/663169092 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12640092e0 con 0x7f126c068bb0 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.104+0000 7f12727ff700 1 -- 172.21.15.31:0/663169092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 msgr2=0x7f126c0c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.104+0000 7f12727ff700 1 --2- 172.21.15.31:0/663169092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 0x7f126c0c8930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f126c06f290 tx=0x7f1264009840 comp rx=0 tx=0).stop 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.105+0000 7f12727ff700 1 -- 172.21.15.31:0/663169092 shutdown_connections 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.105+0000 7f12727ff700 1 --2- 172.21.15.31:0/663169092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 0x7f126c0c8930 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.105+0000 7f12727ff700 1 --2- 172.21.15.31:0/663169092 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f126c1595a0 0x7f126c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.105+0000 7f12727ff700 1 --2- 172.21.15.31:0/663169092 >> v2:172.21.15.62:3300/0 conn(0x7f126c0cc0d0 0x7f126c068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.105+0000 7f12727ff700 1 -- 172.21.15.31:0/663169092 >> 172.21.15.31:0/663169092 conn(0x7f126c06b620 msgr2=0x7f126c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.105+0000 7f12727ff700 1 -- 172.21.15.31:0/663169092 shutdown_connections 2023-10-06T05:18:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.105+0000 7f12727ff700 1 -- 172.21.15.31:0/663169092 wait complete. 2023-10-06T05:18:44.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.106+0000 7f12727ff700 1 Processor -- start 2023-10-06T05:18:44.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.106+0000 7f12727ff700 1 -- start start 2023-10-06T05:18:44.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.107+0000 7f12727ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 0x7f126c150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.107+0000 7f12727ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f126c0cc0d0 0x7f126c16cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.107+0000 7f12727ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f126c1595a0 0x7f126c158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.107+0000 7f12727ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c16d510 con 0x7f126c068bb0 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.107+0000 7f12727ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c16d770 con 0x7f126c1595a0 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.107+0000 7f12727ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c16da50 con 0x7f126c0cc0d0 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.107+0000 7f126bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 0x7f126c150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.107+0000 7f126bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 0x7f126c150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56912/0 (socket says 172.21.15.31:56912) 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.107+0000 7f126bfff700 1 -- 172.21.15.31:0/4073380702 learned_addr learned my addr 172.21.15.31:0/4073380702 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f1270d9c700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f126c1595a0 0x7f126c158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f126b7fe700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f126c0cc0d0 0x7f126c16cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f126bfff700 1 -- 172.21.15.31:0/4073380702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f126c0cc0d0 msgr2=0x7f126c16cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f126bfff700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f126c0cc0d0 0x7f126c16cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f126bfff700 1 -- 172.21.15.31:0/4073380702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f126c1595a0 msgr2=0x7f126c158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f126bfff700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f126c1595a0 0x7f126c158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f126bfff700 1 -- 172.21.15.31:0/4073380702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1264009ae0 con 0x7f126c068bb0 2023-10-06T05:18:44.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f1270d9c700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f126c1595a0 0x7f126c158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:44.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f126bfff700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 0x7f126c150530 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f125c00d860 tx=0x7f125c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:44.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f1253fff700 1 -- 172.21.15.31:0/4073380702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c009610 con 0x7f126c068bb0 2023-10-06T05:18:44.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f1253fff700 1 -- 172.21.15.31:0/4073380702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c009770 con 0x7f126c068bb0 2023-10-06T05:18:44.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f12727ff700 1 -- 172.21.15.31:0/4073380702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c16dd30 con 0x7f126c068bb0 2023-10-06T05:18:44.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.108+0000 7f1253fff700 1 -- 172.21.15.31:0/4073380702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c009b30 con 0x7f126c068bb0 2023-10-06T05:18:44.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.109+0000 7f12727ff700 1 -- 172.21.15.31:0/4073380702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c150e40 con 0x7f126c068bb0 2023-10-06T05:18:44.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.110+0000 7f12727ff700 1 -- 172.21.15.31:0/4073380702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126c1511b0 con 0x7f126c068bb0 2023-10-06T05:18:44.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.110+0000 7f1253fff700 1 -- 172.21.15.31:0/4073380702 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f125c01e460 con 0x7f126c068bb0 2023-10-06T05:18:44.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.111+0000 7f1253fff700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f12580670e0 0x7f1258069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.111+0000 7f1253fff700 1 -- 172.21.15.31:0/4073380702 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f125c084fc0 con 0x7f126c068bb0 2023-10-06T05:18:44.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.112+0000 7f126b7fe700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f12580670e0 0x7f1258069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.112+0000 7f126b7fe700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f12580670e0 0x7f1258069590 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1260006f00 tx=0x7f1260008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:44.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.116+0000 7f1253fff700 1 -- 172.21.15.31:0/4073380702 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f125c051e50 con 0x7f126c068bb0 2023-10-06T05:18:44.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.317+0000 7f12727ff700 1 -- 172.21.15.31:0/4073380702 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f126c04c380 con 0x7f12580670e0 2023-10-06T05:18:44.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.326+0000 7f1253fff700 1 -- 172.21.15.31:0/4073380702 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308313 (secure 0 0 0) 0x7f126c04c380 con 0x7f12580670e0 2023-10-06T05:18:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:44.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 -- 172.21.15.31:0/4073380702 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f12580670e0 msgr2=0x7f1258069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f12580670e0 0x7f1258069590 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1260006f00 tx=0x7f1260008040 comp rx=0 tx=0).stop 2023-10-06T05:18:44.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 -- 172.21.15.31:0/4073380702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 msgr2=0x7f126c150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 0x7f126c150530 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f125c00d860 tx=0x7f125c008040 comp rx=0 tx=0).stop 2023-10-06T05:18:44.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 -- 172.21.15.31:0/4073380702 shutdown_connections 2023-10-06T05:18:44.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f126c1595a0 0x7f126c158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f12580670e0 0x7f1258069590 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f126c068bb0 0x7f126c150530 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 --2- 172.21.15.31:0/4073380702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f126c0cc0d0 0x7f126c16cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 -- 172.21.15.31:0/4073380702 >> 172.21.15.31:0/4073380702 conn(0x7f126c06b620 msgr2=0x7f126c155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:44.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 -- 172.21.15.31:0/4073380702 shutdown_connections 2023-10-06T05:18:44.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.330+0000 7f12727ff700 1 -- 172.21.15.31:0/4073380702 wait complete. 2023-10-06T05:18:44.335 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:18:44.349 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-10-06T05:18:44.082072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383280,"kb_used_data":4976,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591440,"statfs":{"total":773094113280,"available":770653634560,"internally_reserved":0,"allocated":5095424,"data_stored":2819864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001132"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453619+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:33.498919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448171+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:24.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.449706+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447164+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.444195+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455710+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:17.100290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448958+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446650+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:27.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446143+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:10.283521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453249+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:05.369497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455141+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:34.252333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445402+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455343+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:52.371640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451248+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:10.283521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447669+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454977+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:50.655754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452837+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:19.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446943+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448342+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.456888+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:33.498919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451223+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:56.384273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445385+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451398+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:56.125739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.449458+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452400+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:06.022509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.444580+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454129+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:39.631263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.456353+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:05.369497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450548+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447835+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:36.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451478+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:52.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451233+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446889+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464327+0000","last_change":"2023-10-06T05:18:38.464327+0000","last_active":"2023-10-06T05:18:38.464327+0000","last_peered":"2023-10-06T05:18:38.464327+0000","last_clean":"2023-10-06T05:18:38.464327+0000","last_became_active":"2023-10-06T05:18:38.464063+0000","last_became_peered":"2023-10-06T05:18:38.464063+0000","last_unstale":"2023-10-06T05:18:38.464327+0000","last_undegraded":"2023-10-06T05:18:38.464327+0000","last_fullsized":"2023-10-06T05:18:38.464327+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:47.847969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450735+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:28.365507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467137+0000","last_change":"2023-10-06T05:18:38.467137+0000","last_active":"2023-10-06T05:18:38.467137+0000","last_peered":"2023-10-06T05:18:38.467137+0000","last_clean":"2023-10-06T05:18:38.467137+0000","last_became_active":"2023-10-06T05:18:38.466871+0000","last_became_peered":"2023-10-06T05:18:38.466871+0000","last_unstale":"2023-10-06T05:18:38.467137+0000","last_undegraded":"2023-10-06T05:18:38.467137+0000","last_fullsized":"2023-10-06T05:18:38.467137+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:26.910024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.457317+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:39.631263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.465625+0000","last_change":"2023-10-06T05:18:38.465625+0000","last_active":"2023-10-06T05:18:38.465625+0000","last_peered":"2023-10-06T05:18:38.465625+0000","last_clean":"2023-10-06T05:18:38.465625+0000","last_became_active":"2023-10-06T05:18:38.465367+0000","last_became_peered":"2023-10-06T05:18:38.465367+0000","last_unstale":"2023-10-06T05:18:38.465625+0000","last_undegraded":"2023-10-06T05:18:38.465625+0000","last_fullsized":"2023-10-06T05:18:38.465625+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:55.377855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445717+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466328+0000","last_change":"2023-10-06T05:18:38.466328+0000","last_active":"2023-10-06T05:18:38.466328+0000","last_peered":"2023-10-06T05:18:38.466328+0000","last_clean":"2023-10-06T05:18:38.466328+0000","last_became_active":"2023-10-06T05:18:38.466033+0000","last_became_peered":"2023-10-06T05:18:38.466033+0000","last_unstale":"2023-10-06T05:18:38.466328+0000","last_undegraded":"2023-10-06T05:18:38.466328+0000","last_fullsized":"2023-10-06T05:18:38.466328+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:23.634297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443349+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464675+0000","last_change":"2023-10-06T05:18:38.464675+0000","last_active":"2023-10-06T05:18:38.464675+0000","last_peered":"2023-10-06T05:18:38.464675+0000","last_clean":"2023-10-06T05:18:38.464675+0000","last_became_active":"2023-10-06T05:18:38.464409+0000","last_became_peered":"2023-10-06T05:18:38.464409+0000","last_unstale":"2023-10-06T05:18:38.464675+0000","last_undegraded":"2023-10-06T05:18:38.464675+0000","last_fullsized":"2023-10-06T05:18:38.464675+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:09.080286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447205+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.465029+0000","last_change":"2023-10-06T05:18:38.465029+0000","last_active":"2023-10-06T05:18:38.465029+0000","last_peered":"2023-10-06T05:18:38.465029+0000","last_clean":"2023-10-06T05:18:38.465029+0000","last_became_active":"2023-10-06T05:18:38.464703+0000","last_became_peered":"2023-10-06T05:18:38.464703+0000","last_unstale":"2023-10-06T05:18:38.465029+0000","last_undegraded":"2023-10-06T05:18:38.465029+0000","last_fullsized":"2023-10-06T05:18:38.465029+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446502+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460564+0000","last_change":"2023-10-06T05:18:38.460564+0000","last_active":"2023-10-06T05:18:38.460564+0000","last_peered":"2023-10-06T05:18:38.460564+0000","last_clean":"2023-10-06T05:18:38.460564+0000","last_became_active":"2023-10-06T05:18:38.460269+0000","last_became_peered":"2023-10-06T05:18:38.460269+0000","last_unstale":"2023-10-06T05:18:38.460564+0000","last_undegraded":"2023-10-06T05:18:38.460564+0000","last_fullsized":"2023-10-06T05:18:38.460564+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:33.245915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447681+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:13.626073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.461183+0000","last_change":"2023-10-06T05:18:38.461183+0000","last_active":"2023-10-06T05:18:38.461183+0000","last_peered":"2023-10-06T05:18:38.461183+0000","last_clean":"2023-10-06T05:18:38.461183+0000","last_became_active":"2023-10-06T05:18:38.460893+0000","last_became_peered":"2023-10-06T05:18:38.460893+0000","last_unstale":"2023-10-06T05:18:38.461183+0000","last_undegraded":"2023-10-06T05:18:38.461183+0000","last_fullsized":"2023-10-06T05:18:38.461183+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:12.619655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443147+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458370+0000","last_change":"2023-10-06T05:18:38.458370+0000","last_active":"2023-10-06T05:18:38.458370+0000","last_peered":"2023-10-06T05:18:38.458370+0000","last_clean":"2023-10-06T05:18:38.458370+0000","last_became_active":"2023-10-06T05:18:38.458104+0000","last_became_peered":"2023-10-06T05:18:38.458104+0000","last_unstale":"2023-10-06T05:18:38.458370+0000","last_undegraded":"2023-10-06T05:18:38.458370+0000","last_fullsized":"2023-10-06T05:18:38.458370+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:41.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451319+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463433+0000","last_change":"2023-10-06T05:18:38.463433+0000","last_active":"2023-10-06T05:18:38.463433+0000","last_peered":"2023-10-06T05:18:38.463433+0000","last_clean":"2023-10-06T05:18:38.463433+0000","last_became_active":"2023-10-06T05:18:38.463176+0000","last_became_peered":"2023-10-06T05:18:38.463176+0000","last_unstale":"2023-10-06T05:18:38.463433+0000","last_undegraded":"2023-10-06T05:18:38.463433+0000","last_fullsized":"2023-10-06T05:18:38.463433+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:41.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450043+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.461448+0000","last_change":"2023-10-06T05:18:38.461448+0000","last_active":"2023-10-06T05:18:38.461448+0000","last_peered":"2023-10-06T05:18:38.461448+0000","last_clean":"2023-10-06T05:18:38.461448+0000","last_became_active":"2023-10-06T05:18:38.461184+0000","last_became_peered":"2023-10-06T05:18:38.461184+0000","last_unstale":"2023-10-06T05:18:38.461448+0000","last_undegraded":"2023-10-06T05:18:38.461448+0000","last_fullsized":"2023-10-06T05:18:38.461448+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:34.215548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443576+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460848+0000","last_change":"2023-10-06T05:18:38.460848+0000","last_active":"2023-10-06T05:18:38.460848+0000","last_peered":"2023-10-06T05:18:38.460848+0000","last_clean":"2023-10-06T05:18:38.460848+0000","last_became_active":"2023-10-06T05:18:38.460598+0000","last_became_peered":"2023-10-06T05:18:38.460598+0000","last_unstale":"2023-10-06T05:18:38.460848+0000","last_undegraded":"2023-10-06T05:18:38.460848+0000","last_fullsized":"2023-10-06T05:18:38.460848+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:12.765494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454251+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463872+0000","last_change":"2023-10-06T05:18:38.463872+0000","last_active":"2023-10-06T05:18:38.463872+0000","last_peered":"2023-10-06T05:18:38.463872+0000","last_clean":"2023-10-06T05:18:38.463872+0000","last_became_active":"2023-10-06T05:18:38.463587+0000","last_became_peered":"2023-10-06T05:18:38.463587+0000","last_unstale":"2023-10-06T05:18:38.463872+0000","last_undegraded":"2023-10-06T05:18:38.463872+0000","last_fullsized":"2023-10-06T05:18:38.463872+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:27.960703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448983+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466701+0000","last_change":"2023-10-06T05:18:38.466701+0000","last_active":"2023-10-06T05:18:38.466701+0000","last_peered":"2023-10-06T05:18:38.466701+0000","last_clean":"2023-10-06T05:18:38.466701+0000","last_became_active":"2023-10-06T05:18:38.466409+0000","last_became_peered":"2023-10-06T05:18:38.466409+0000","last_unstale":"2023-10-06T05:18:38.466701+0000","last_undegraded":"2023-10-06T05:18:38.466701+0000","last_fullsized":"2023-10-06T05:18:38.466701+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:26.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446996+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:52.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464573+0000","last_change":"2023-10-06T05:18:38.464573+0000","last_active":"2023-10-06T05:18:38.464573+0000","last_peered":"2023-10-06T05:18:38.464573+0000","last_clean":"2023-10-06T05:18:38.464573+0000","last_became_active":"2023-10-06T05:18:38.464303+0000","last_became_peered":"2023-10-06T05:18:38.464303+0000","last_unstale":"2023-10-06T05:18:38.464573+0000","last_undegraded":"2023-10-06T05:18:38.464573+0000","last_fullsized":"2023-10-06T05:18:38.464573+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451911+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:41.261742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458013+0000","last_change":"2023-10-06T05:18:38.458013+0000","last_active":"2023-10-06T05:18:38.458013+0000","last_peered":"2023-10-06T05:18:38.458013+0000","last_clean":"2023-10-06T05:18:38.458013+0000","last_became_active":"2023-10-06T05:18:38.457708+0000","last_became_peered":"2023-10-06T05:18:38.457708+0000","last_unstale":"2023-10-06T05:18:38.458013+0000","last_undegraded":"2023-10-06T05:18:38.458013+0000","last_fullsized":"2023-10-06T05:18:38.458013+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:35.907402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445801+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466371+0000","last_change":"2023-10-06T05:18:38.466371+0000","last_active":"2023-10-06T05:18:38.466371+0000","last_peered":"2023-10-06T05:18:38.466371+0000","last_clean":"2023-10-06T05:18:38.466371+0000","last_became_active":"2023-10-06T05:18:38.466099+0000","last_became_peered":"2023-10-06T05:18:38.466099+0000","last_unstale":"2023-10-06T05:18:38.466371+0000","last_undegraded":"2023-10-06T05:18:38.466371+0000","last_fullsized":"2023-10-06T05:18:38.466371+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:12.765494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454588+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:51.516975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.456427+0000","last_change":"2023-10-06T05:18:38.456427+0000","last_active":"2023-10-06T05:18:38.456427+0000","last_peered":"2023-10-06T05:18:38.456427+0000","last_clean":"2023-10-06T05:18:38.456427+0000","last_became_active":"2023-10-06T05:18:38.456172+0000","last_became_peered":"2023-10-06T05:18:38.456172+0000","last_unstale":"2023-10-06T05:18:38.456427+0000","last_undegraded":"2023-10-06T05:18:38.456427+0000","last_fullsized":"2023-10-06T05:18:38.456427+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:13.476914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443171+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466443+0000","last_change":"2023-10-06T05:18:38.466443+0000","last_active":"2023-10-06T05:18:38.466443+0000","last_peered":"2023-10-06T05:18:38.466443+0000","last_clean":"2023-10-06T05:18:38.466443+0000","last_became_active":"2023-10-06T05:18:38.465603+0000","last_became_peered":"2023-10-06T05:18:38.465603+0000","last_unstale":"2023-10-06T05:18:38.466443+0000","last_undegraded":"2023-10-06T05:18:38.466443+0000","last_fullsized":"2023-10-06T05:18:38.466443+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:08.665979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454720+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:09.410169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467188+0000","last_change":"2023-10-06T05:18:38.467188+0000","last_active":"2023-10-06T05:18:38.467188+0000","last_peered":"2023-10-06T05:18:38.467188+0000","last_clean":"2023-10-06T05:18:38.467188+0000","last_became_active":"2023-10-06T05:18:38.466475+0000","last_became_peered":"2023-10-06T05:18:38.466475+0000","last_unstale":"2023-10-06T05:18:38.467188+0000","last_undegraded":"2023-10-06T05:18:38.467188+0000","last_fullsized":"2023-10-06T05:18:38.467188+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:22.230340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451137+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:11.273331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.468654+0000","last_change":"2023-10-06T05:18:38.468654+0000","last_active":"2023-10-06T05:18:38.468654+0000","last_peered":"2023-10-06T05:18:38.468654+0000","last_clean":"2023-10-06T05:18:38.468654+0000","last_became_active":"2023-10-06T05:18:38.468394+0000","last_became_peered":"2023-10-06T05:18:38.468394+0000","last_unstale":"2023-10-06T05:18:38.468654+0000","last_undegraded":"2023-10-06T05:18:38.468654+0000","last_fullsized":"2023-10-06T05:18:38.468654+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:56.721340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452992+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:43.862504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460519+0000","last_change":"2023-10-06T05:18:38.460519+0000","last_active":"2023-10-06T05:18:38.460519+0000","last_peered":"2023-10-06T05:18:38.460519+0000","last_clean":"2023-10-06T05:18:38.460519+0000","last_became_active":"2023-10-06T05:18:38.460215+0000","last_became_peered":"2023-10-06T05:18:38.460215+0000","last_unstale":"2023-10-06T05:18:38.460519+0000","last_undegraded":"2023-10-06T05:18:38.460519+0000","last_fullsized":"2023-10-06T05:18:38.460519+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:12.619655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451676+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:27.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.462389+0000","last_change":"2023-10-06T05:18:38.462389+0000","last_active":"2023-10-06T05:18:38.462389+0000","last_peered":"2023-10-06T05:18:38.462389+0000","last_clean":"2023-10-06T05:18:38.462389+0000","last_became_active":"2023-10-06T05:18:38.462121+0000","last_became_peered":"2023-10-06T05:18:38.462121+0000","last_unstale":"2023-10-06T05:18:38.462389+0000","last_undegraded":"2023-10-06T05:18:38.462389+0000","last_fullsized":"2023-10-06T05:18:38.462389+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:23.179454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453051+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:15.086481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.457211+0000","last_change":"2023-10-06T05:18:38.457211+0000","last_active":"2023-10-06T05:18:38.457211+0000","last_peered":"2023-10-06T05:18:38.457211+0000","last_clean":"2023-10-06T05:18:38.457211+0000","last_became_active":"2023-10-06T05:18:38.456921+0000","last_became_peered":"2023-10-06T05:18:38.456921+0000","last_unstale":"2023-10-06T05:18:38.457211+0000","last_undegraded":"2023-10-06T05:18:38.457211+0000","last_fullsized":"2023-10-06T05:18:38.457211+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:11.790017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448636+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460312+0000","last_change":"2023-10-06T05:18:38.460312+0000","last_active":"2023-10-06T05:18:38.460312+0000","last_peered":"2023-10-06T05:18:38.460312+0000","last_clean":"2023-10-06T05:18:38.460312+0000","last_became_active":"2023-10-06T05:18:38.460043+0000","last_became_peered":"2023-10-06T05:18:38.460043+0000","last_unstale":"2023-10-06T05:18:38.460312+0000","last_undegraded":"2023-10-06T05:18:38.460312+0000","last_fullsized":"2023-10-06T05:18:38.460312+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:09.080286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.444681+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:18:37.444681+0000","last_peered":"2023-10-06T05:18:37.444681+0000","last_clean":"2023-10-06T05:18:37.444681+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:18:37.444681+0000","last_undegraded":"2023-10-06T05:18:37.444681+0000","last_fullsized":"2023-10-06T05:18:37.444681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452130+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458978+0000","last_change":"2023-10-06T05:18:38.458978+0000","last_active":"2023-10-06T05:18:38.458978+0000","last_peered":"2023-10-06T05:18:38.458978+0000","last_clean":"2023-10-06T05:18:38.458978+0000","last_became_active":"2023-10-06T05:18:38.458700+0000","last_became_peered":"2023-10-06T05:18:38.458700+0000","last_unstale":"2023-10-06T05:18:38.458978+0000","last_undegraded":"2023-10-06T05:18:38.458978+0000","last_fullsized":"2023-10-06T05:18:38.458978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:23.179454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.436719+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:18:37.436719+0000","last_peered":"2023-10-06T05:18:37.436719+0000","last_clean":"2023-10-06T05:18:37.436719+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:18:37.436719+0000","last_undegraded":"2023-10-06T05:18:37.436719+0000","last_fullsized":"2023-10-06T05:18:37.436719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455206+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:44.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460411+0000","last_change":"2023-10-06T05:18:38.460411+0000","last_active":"2023-10-06T05:18:38.460411+0000","last_peered":"2023-10-06T05:18:38.460411+0000","last_clean":"2023-10-06T05:18:38.460411+0000","last_became_active":"2023-10-06T05:18:38.460142+0000","last_became_peered":"2023-10-06T05:18:38.460142+0000","last_unstale":"2023-10-06T05:18:38.460411+0000","last_undegraded":"2023-10-06T05:18:38.460411+0000","last_fullsized":"2023-10-06T05:18:38.460411+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:10.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.434247+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:18:37.434247+0000","last_peered":"2023-10-06T05:18:37.434247+0000","last_clean":"2023-10-06T05:18:37.434247+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:18:37.434247+0000","last_undegraded":"2023-10-06T05:18:37.434247+0000","last_fullsized":"2023-10-06T05:18:37.434247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":180,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437257+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:18:37.437257+0000","last_peered":"2023-10-06T05:18:37.437257+0000","last_clean":"2023-10-06T05:18:37.437257+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:18:37.437257+0000","last_undegraded":"2023-10-06T05:18:37.437257+0000","last_fullsized":"2023-10-06T05:18:37.437257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445051+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463399+0000","last_change":"2023-10-06T05:18:38.463399+0000","last_active":"2023-10-06T05:18:38.463399+0000","last_peered":"2023-10-06T05:18:38.463399+0000","last_clean":"2023-10-06T05:18:38.463399+0000","last_became_active":"2023-10-06T05:18:38.463130+0000","last_became_peered":"2023-10-06T05:18:38.463130+0000","last_unstale":"2023-10-06T05:18:38.463399+0000","last_undegraded":"2023-10-06T05:18:38.463399+0000","last_fullsized":"2023-10-06T05:18:38.463399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:35.907402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437352+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:18:37.437352+0000","last_peered":"2023-10-06T05:18:37.437352+0000","last_clean":"2023-10-06T05:18:37.437352+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:18:37.437352+0000","last_undegraded":"2023-10-06T05:18:37.437352+0000","last_fullsized":"2023-10-06T05:18:37.437352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.457853+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:51.516975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463382+0000","last_change":"2023-10-06T05:18:38.463382+0000","last_active":"2023-10-06T05:18:38.463382+0000","last_peered":"2023-10-06T05:18:38.463382+0000","last_clean":"2023-10-06T05:18:38.463382+0000","last_became_active":"2023-10-06T05:18:38.463112+0000","last_became_peered":"2023-10-06T05:18:38.463112+0000","last_unstale":"2023-10-06T05:18:38.463382+0000","last_undegraded":"2023-10-06T05:18:38.463382+0000","last_fullsized":"2023-10-06T05:18:38.463382+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:37.839836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437251+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:18:37.437251+0000","last_peered":"2023-10-06T05:18:37.437251+0000","last_clean":"2023-10-06T05:18:37.437251+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:18:37.437251+0000","last_undegraded":"2023-10-06T05:18:37.437251+0000","last_fullsized":"2023-10-06T05:18:37.437251+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450529+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:42.744594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466799+0000","last_change":"2023-10-06T05:18:38.466799+0000","last_active":"2023-10-06T05:18:38.466799+0000","last_peered":"2023-10-06T05:18:38.466799+0000","last_clean":"2023-10-06T05:18:38.466799+0000","last_became_active":"2023-10-06T05:18:38.466486+0000","last_became_peered":"2023-10-06T05:18:38.466486+0000","last_unstale":"2023-10-06T05:18:38.466799+0000","last_undegraded":"2023-10-06T05:18:38.466799+0000","last_fullsized":"2023-10-06T05:18:38.466799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:04.252646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.439395+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:18:37.439395+0000","last_peered":"2023-10-06T05:18:37.439395+0000","last_clean":"2023-10-06T05:18:37.439395+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:18:37.439395+0000","last_undegraded":"2023-10-06T05:18:37.439395+0000","last_fullsized":"2023-10-06T05:18:37.439395+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445017+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467529+0000","last_change":"2023-10-06T05:18:38.467529+0000","last_active":"2023-10-06T05:18:38.467529+0000","last_peered":"2023-10-06T05:18:38.467529+0000","last_clean":"2023-10-06T05:18:38.467529+0000","last_became_active":"2023-10-06T05:18:38.467262+0000","last_became_peered":"2023-10-06T05:18:38.467262+0000","last_unstale":"2023-10-06T05:18:38.467529+0000","last_undegraded":"2023-10-06T05:18:38.467529+0000","last_fullsized":"2023-10-06T05:18:38.467529+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:34.215548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.440499+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:18:37.440499+0000","last_peered":"2023-10-06T05:18:37.440499+0000","last_clean":"2023-10-06T05:18:37.440499+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:18:37.440499+0000","last_undegraded":"2023-10-06T05:18:37.440499+0000","last_fullsized":"2023-10-06T05:18:37.440499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452193+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:39.628432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460634+0000","last_change":"2023-10-06T05:18:38.460634+0000","last_active":"2023-10-06T05:18:38.460634+0000","last_peered":"2023-10-06T05:18:38.460634+0000","last_clean":"2023-10-06T05:18:38.460634+0000","last_became_active":"2023-10-06T05:18:38.459850+0000","last_became_peered":"2023-10-06T05:18:38.459850+0000","last_unstale":"2023-10-06T05:18:38.460634+0000","last_undegraded":"2023-10-06T05:18:38.460634+0000","last_fullsized":"2023-10-06T05:18:38.460634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:10.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.440631+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:18:37.440631+0000","last_peered":"2023-10-06T05:18:37.440631+0000","last_clean":"2023-10-06T05:18:37.440631+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:18:37.440631+0000","last_undegraded":"2023-10-06T05:18:37.440631+0000","last_fullsized":"2023-10-06T05:18:37.440631+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298348,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073492,"statfs":{"total":96636764160,"available":96331255808,"internally_reserved":0,"allocated":1085440,"data_stored":796913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78300000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.871},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.62},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77400000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298336,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073504,"statfs":{"total":96636764160,"available":96331268096,"internally_reserved":0,"allocated":1073152,"data_stored":790297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.433}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.95999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.94099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.006}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.82399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297756,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074084,"statfs":{"total":96636764160,"available":96331862016,"internally_reserved":0,"allocated":479232,"data_stored":199929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.51300000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.749},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.81899999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.91700000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.84599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.004}]},{"osd":7,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.0169999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.371}]},{"osd":1,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.754},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.78100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":7.742,"5min":7.742,"15min":7.742},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.88500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.87},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.85799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2780000000000005,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":9.25,"5min":9.25,"15min":9.25},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.108000000000001,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.95799999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":10.384,"5min":10.384,"15min":10.384},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":9.1699999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.205,"5min":10.205,"15min":10.205},"last":1.038},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.83299999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":1.113}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.0920000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:44.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:44.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.585+0000 7f41851ff700 1 Processor -- start 2023-10-06T05:18:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.585+0000 7f41851ff700 1 -- start start 2023-10-06T05:18:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.586+0000 7f41851ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41801596d0 0x7f4180159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.586+0000 7f41851ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 0x7f418006a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.586+0000 7f41851ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4180067f70 con 0x7f41800c16e0 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.586+0000 7f41851ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418006e040 con 0x7f41801596d0 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.586+0000 7f41851ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41800681c0 con 0x7f41800c6f80 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.586+0000 7f417ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41801596d0 0x7f4180159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.586+0000 7f417ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41801596d0 0x7f4180159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56922/0 (socket says 172.21.15.31:56922) 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.586+0000 7f417ed9d700 1 -- 172.21.15.31:0/2009104139 learned_addr learned my addr 172.21.15.31:0/2009104139 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.586+0000 7f417f59e700 1 --1- 172.21.15.31:0/2009104139 >> v1:172.21.15.31:6789/0 conn(0x7f41800c16e0 0x7f418015d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52260/0 (socket says 172.21.15.31:52260) 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.587+0000 7f417e59c700 1 --2- 172.21.15.31:0/2009104139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 0x7f418006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.587+0000 7f417ed9d700 1 -- 172.21.15.31:0/2009104139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 msgr2=0x7f418006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.587+0000 7f417ed9d700 1 --2- 172.21.15.31:0/2009104139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 0x7f418006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.587+0000 7f417ed9d700 1 -- 172.21.15.31:0/2009104139 >> v1:172.21.15.31:6789/0 conn(0x7f41800c16e0 legacy=0x7f418015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.587+0000 7f417ed9d700 1 -- 172.21.15.31:0/2009104139 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800684a0 con 0x7f41801596d0 2023-10-06T05:18:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.587+0000 7f417e59c700 1 --2- 172.21.15.31:0/2009104139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 0x7f418006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.587+0000 7f417ed9d700 1 --2- 172.21.15.31:0/2009104139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41801596d0 0x7f4180159aa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f41700099b0 tx=0x7f4170009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ec0fdf6c5e5b75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.587+0000 7f417dd9b700 1 -- 172.21.15.31:0/2009104139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417000b860 con 0x7f41801596d0 2023-10-06T05:18:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.587+0000 7f417dd9b700 1 -- 172.21.15.31:0/2009104139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417000b9c0 con 0x7f41801596d0 2023-10-06T05:18:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.587+0000 7f417dd9b700 1 -- 172.21.15.31:0/2009104139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417000bd60 con 0x7f41801596d0 2023-10-06T05:18:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.588+0000 7f41851ff700 1 -- 172.21.15.31:0/2009104139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41801596d0 msgr2=0x7f4180159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.588+0000 7f41851ff700 1 --2- 172.21.15.31:0/2009104139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41801596d0 0x7f4180159aa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f41700099b0 tx=0x7f4170009250 comp rx=0 tx=0).stop 2023-10-06T05:18:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.588+0000 7f41851ff700 1 -- 172.21.15.31:0/2009104139 shutdown_connections 2023-10-06T05:18:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.588+0000 7f41851ff700 1 --2- 172.21.15.31:0/2009104139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41801596d0 0x7f4180159aa0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.588+0000 7f41851ff700 1 --2- 172.21.15.31:0/2009104139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 0x7f418006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.588+0000 7f41851ff700 1 -- 172.21.15.31:0/2009104139 >> 172.21.15.31:0/2009104139 conn(0x7f41800c9e80 msgr2=0x7f418006a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.588+0000 7f41851ff700 1 -- 172.21.15.31:0/2009104139 shutdown_connections 2023-10-06T05:18:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.588+0000 7f41851ff700 1 -- 172.21.15.31:0/2009104139 wait complete. 2023-10-06T05:18:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.590+0000 7f41851ff700 1 Processor -- start 2023-10-06T05:18:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.590+0000 7f41851ff700 1 -- start start 2023-10-06T05:18:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f41851ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41800c16e0 0x7f418016a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f41851ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 0x7f41800b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f41851ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41801596d0 0x7f41800c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f41851ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800c4e30 con 0x7f41800c16e0 2023-10-06T05:18:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f41851ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800b1640 con 0x7f41801596d0 2023-10-06T05:18:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f41851ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41800c52c0 con 0x7f41800c6f80 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41800c16e0 0x7f418016a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41800c16e0 0x7f418016a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56932/0 (socket says 172.21.15.31:56932) 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417ed9d700 1 -- 172.21.15.31:0/2063326517 learned_addr learned my addr 172.21.15.31:0/2063326517 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417e59c700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 0x7f41800b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417f59e700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41801596d0 0x7f41800c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417ed9d700 1 -- 172.21.15.31:0/2063326517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 msgr2=0x7f41800b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417ed9d700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 0x7f41800b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417ed9d700 1 -- 172.21.15.31:0/2063326517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41801596d0 msgr2=0x7f41800c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417ed9d700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41801596d0 0x7f41800c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417ed9d700 1 -- 172.21.15.31:0/2063326517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700094f0 con 0x7f41800c16e0 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.591+0000 7f417f59e700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41801596d0 0x7f41800c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.592+0000 7f417ed9d700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41800c16e0 0x7f418016a300 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4170009290 tx=0x7f4170017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.592+0000 7f4166ffd700 1 -- 172.21.15.31:0/2063326517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170026480 con 0x7f41800c16e0 2023-10-06T05:18:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.592+0000 7f4166ffd700 1 -- 172.21.15.31:0/2063326517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417001f070 con 0x7f41800c16e0 2023-10-06T05:18:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.592+0000 7f4166ffd700 1 -- 172.21.15.31:0/2063326517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170026850 con 0x7f41800c16e0 2023-10-06T05:18:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.592+0000 7f41851ff700 1 -- 172.21.15.31:0/2063326517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41800c4220 con 0x7f41800c16e0 2023-10-06T05:18:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.592+0000 7f41851ff700 1 -- 172.21.15.31:0/2063326517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41800c47e0 con 0x7f41800c16e0 2023-10-06T05:18:44.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.593+0000 7f41851ff700 1 -- 172.21.15.31:0/2063326517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418004c380 con 0x7f41800c16e0 2023-10-06T05:18:44.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.594+0000 7f4166ffd700 1 -- 172.21.15.31:0/2063326517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f417001a430 con 0x7f41800c16e0 2023-10-06T05:18:44.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.595+0000 7f4166ffd700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f416c0670e0 0x7f416c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:44.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.595+0000 7f4166ffd700 1 -- 172.21.15.31:0/2063326517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4170021070 con 0x7f41800c16e0 2023-10-06T05:18:44.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.595+0000 7f417e59c700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f416c0670e0 0x7f416c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:44.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.596+0000 7f417e59c700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f416c0670e0 0x7f416c069590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f418016a840 tx=0x7f4174008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:44.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.599+0000 7f4166ffd700 1 -- 172.21.15.31:0/2063326517 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41700536d0 con 0x7f41800c16e0 2023-10-06T05:18:44.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.799+0000 7f41851ff700 1 -- 172.21.15.31:0/2063326517 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41800c4f70 con 0x7f416c0670e0 2023-10-06T05:18:44.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.808+0000 7f4166ffd700 1 -- 172.21.15.31:0/2063326517 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308313 (secure 0 0 0) 0x7f4174011700 con 0x7f416c0670e0 2023-10-06T05:18:44.813 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 -- 172.21.15.31:0/2063326517 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f416c0670e0 msgr2=0x7f416c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f416c0670e0 0x7f416c069590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f418016a840 tx=0x7f4174008040 comp rx=0 tx=0).stop 2023-10-06T05:18:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 -- 172.21.15.31:0/2063326517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41800c16e0 msgr2=0x7f418016a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41800c16e0 0x7f418016a300 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4170009290 tx=0x7f4170017000 comp rx=0 tx=0).stop 2023-10-06T05:18:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 -- 172.21.15.31:0/2063326517 shutdown_connections 2023-10-06T05:18:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41801596d0 0x7f41800c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f416c0670e0 0x7f416c069590 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41800c16e0 0x7f418016a300 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 --2- 172.21.15.31:0/2063326517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41800c6f80 0x7f41800b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 -- 172.21.15.31:0/2063326517 >> 172.21.15.31:0/2063326517 conn(0x7f41800c9e80 msgr2=0x7f4180067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 -- 172.21.15.31:0/2063326517 shutdown_connections 2023-10-06T05:18:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:44.812+0000 7f41851ff700 1 -- 172.21.15.31:0/2063326517 wait complete. 2023-10-06T05:18:44.817 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:18:44.831 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-10-06T05:18:44.082072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383280,"kb_used_data":4976,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591440,"statfs":{"total":773094113280,"available":770653634560,"internally_reserved":0,"allocated":5095424,"data_stored":2819864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001132"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453619+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:33.498919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448171+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:24.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.449706+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447164+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.444195+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455710+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:17.100290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448958+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446650+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:27.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446143+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:10.283521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453249+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:05.369497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455141+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:34.252333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445402+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455343+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:52.371640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451248+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:10.283521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447669+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454977+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:50.655754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452837+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:19.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446943+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448342+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.456888+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:33.498919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451223+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:56.384273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445385+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451398+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:56.125739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.449458+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452400+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:06.022509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.444580+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454129+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:39.631263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.456353+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:05.369497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450548+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447835+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:36.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451478+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:52.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451233+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446889+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464327+0000","last_change":"2023-10-06T05:18:38.464327+0000","last_active":"2023-10-06T05:18:38.464327+0000","last_peered":"2023-10-06T05:18:38.464327+0000","last_clean":"2023-10-06T05:18:38.464327+0000","last_became_active":"2023-10-06T05:18:38.464063+0000","last_became_peered":"2023-10-06T05:18:38.464063+0000","last_unstale":"2023-10-06T05:18:38.464327+0000","last_undegraded":"2023-10-06T05:18:38.464327+0000","last_fullsized":"2023-10-06T05:18:38.464327+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:47.847969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450735+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:28.365507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467137+0000","last_change":"2023-10-06T05:18:38.467137+0000","last_active":"2023-10-06T05:18:38.467137+0000","last_peered":"2023-10-06T05:18:38.467137+0000","last_clean":"2023-10-06T05:18:38.467137+0000","last_became_active":"2023-10-06T05:18:38.466871+0000","last_became_peered":"2023-10-06T05:18:38.466871+0000","last_unstale":"2023-10-06T05:18:38.467137+0000","last_undegraded":"2023-10-06T05:18:38.467137+0000","last_fullsized":"2023-10-06T05:18:38.467137+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:26.910024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.457317+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:39.631263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.465625+0000","last_change":"2023-10-06T05:18:38.465625+0000","last_active":"2023-10-06T05:18:38.465625+0000","last_peered":"2023-10-06T05:18:38.465625+0000","last_clean":"2023-10-06T05:18:38.465625+0000","last_became_active":"2023-10-06T05:18:38.465367+0000","last_became_peered":"2023-10-06T05:18:38.465367+0000","last_unstale":"2023-10-06T05:18:38.465625+0000","last_undegraded":"2023-10-06T05:18:38.465625+0000","last_fullsized":"2023-10-06T05:18:38.465625+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:55.377855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445717+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466328+0000","last_change":"2023-10-06T05:18:38.466328+0000","last_active":"2023-10-06T05:18:38.466328+0000","last_peered":"2023-10-06T05:18:38.466328+0000","last_clean":"2023-10-06T05:18:38.466328+0000","last_became_active":"2023-10-06T05:18:38.466033+0000","last_became_peered":"2023-10-06T05:18:38.466033+0000","last_unstale":"2023-10-06T05:18:38.466328+0000","last_undegraded":"2023-10-06T05:18:38.466328+0000","last_fullsized":"2023-10-06T05:18:38.466328+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:23.634297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443349+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464675+0000","last_change":"2023-10-06T05:18:38.464675+0000","last_active":"2023-10-06T05:18:38.464675+0000","last_peered":"2023-10-06T05:18:38.464675+0000","last_clean":"2023-10-06T05:18:38.464675+0000","last_became_active":"2023-10-06T05:18:38.464409+0000","last_became_peered":"2023-10-06T05:18:38.464409+0000","last_unstale":"2023-10-06T05:18:38.464675+0000","last_undegraded":"2023-10-06T05:18:38.464675+0000","last_fullsized":"2023-10-06T05:18:38.464675+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:09.080286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447205+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.465029+0000","last_change":"2023-10-06T05:18:38.465029+0000","last_active":"2023-10-06T05:18:38.465029+0000","last_peered":"2023-10-06T05:18:38.465029+0000","last_clean":"2023-10-06T05:18:38.465029+0000","last_became_active":"2023-10-06T05:18:38.464703+0000","last_became_peered":"2023-10-06T05:18:38.464703+0000","last_unstale":"2023-10-06T05:18:38.465029+0000","last_undegraded":"2023-10-06T05:18:38.465029+0000","last_fullsized":"2023-10-06T05:18:38.465029+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446502+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460564+0000","last_change":"2023-10-06T05:18:38.460564+0000","last_active":"2023-10-06T05:18:38.460564+0000","last_peered":"2023-10-06T05:18:38.460564+0000","last_clean":"2023-10-06T05:18:38.460564+0000","last_became_active":"2023-10-06T05:18:38.460269+0000","last_became_peered":"2023-10-06T05:18:38.460269+0000","last_unstale":"2023-10-06T05:18:38.460564+0000","last_undegraded":"2023-10-06T05:18:38.460564+0000","last_fullsized":"2023-10-06T05:18:38.460564+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:33.245915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447681+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:13.626073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.461183+0000","last_change":"2023-10-06T05:18:38.461183+0000","last_active":"2023-10-06T05:18:38.461183+0000","last_peered":"2023-10-06T05:18:38.461183+0000","last_clean":"2023-10-06T05:18:38.461183+0000","last_became_active":"2023-10-06T05:18:38.460893+0000","last_became_peered":"2023-10-06T05:18:38.460893+0000","last_unstale":"2023-10-06T05:18:38.461183+0000","last_undegraded":"2023-10-06T05:18:38.461183+0000","last_fullsized":"2023-10-06T05:18:38.461183+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:12.619655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443147+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458370+0000","last_change":"2023-10-06T05:18:38.458370+0000","last_active":"2023-10-06T05:18:38.458370+0000","last_peered":"2023-10-06T05:18:38.458370+0000","last_clean":"2023-10-06T05:18:38.458370+0000","last_became_active":"2023-10-06T05:18:38.458104+0000","last_became_peered":"2023-10-06T05:18:38.458104+0000","last_unstale":"2023-10-06T05:18:38.458370+0000","last_undegraded":"2023-10-06T05:18:38.458370+0000","last_fullsized":"2023-10-06T05:18:38.458370+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:41.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451319+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463433+0000","last_change":"2023-10-06T05:18:38.463433+0000","last_active":"2023-10-06T05:18:38.463433+0000","last_peered":"2023-10-06T05:18:38.463433+0000","last_clean":"2023-10-06T05:18:38.463433+0000","last_became_active":"2023-10-06T05:18:38.463176+0000","last_became_peered":"2023-10-06T05:18:38.463176+0000","last_unstale":"2023-10-06T05:18:38.463433+0000","last_undegraded":"2023-10-06T05:18:38.463433+0000","last_fullsized":"2023-10-06T05:18:38.463433+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:41.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450043+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.461448+0000","last_change":"2023-10-06T05:18:38.461448+0000","last_active":"2023-10-06T05:18:38.461448+0000","last_peered":"2023-10-06T05:18:38.461448+0000","last_clean":"2023-10-06T05:18:38.461448+0000","last_became_active":"2023-10-06T05:18:38.461184+0000","last_became_peered":"2023-10-06T05:18:38.461184+0000","last_unstale":"2023-10-06T05:18:38.461448+0000","last_undegraded":"2023-10-06T05:18:38.461448+0000","last_fullsized":"2023-10-06T05:18:38.461448+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:34.215548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443576+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460848+0000","last_change":"2023-10-06T05:18:38.460848+0000","last_active":"2023-10-06T05:18:38.460848+0000","last_peered":"2023-10-06T05:18:38.460848+0000","last_clean":"2023-10-06T05:18:38.460848+0000","last_became_active":"2023-10-06T05:18:38.460598+0000","last_became_peered":"2023-10-06T05:18:38.460598+0000","last_unstale":"2023-10-06T05:18:38.460848+0000","last_undegraded":"2023-10-06T05:18:38.460848+0000","last_fullsized":"2023-10-06T05:18:38.460848+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:12.765494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454251+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463872+0000","last_change":"2023-10-06T05:18:38.463872+0000","last_active":"2023-10-06T05:18:38.463872+0000","last_peered":"2023-10-06T05:18:38.463872+0000","last_clean":"2023-10-06T05:18:38.463872+0000","last_became_active":"2023-10-06T05:18:38.463587+0000","last_became_peered":"2023-10-06T05:18:38.463587+0000","last_unstale":"2023-10-06T05:18:38.463872+0000","last_undegraded":"2023-10-06T05:18:38.463872+0000","last_fullsized":"2023-10-06T05:18:38.463872+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:27.960703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448983+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466701+0000","last_change":"2023-10-06T05:18:38.466701+0000","last_active":"2023-10-06T05:18:38.466701+0000","last_peered":"2023-10-06T05:18:38.466701+0000","last_clean":"2023-10-06T05:18:38.466701+0000","last_became_active":"2023-10-06T05:18:38.466409+0000","last_became_peered":"2023-10-06T05:18:38.466409+0000","last_unstale":"2023-10-06T05:18:38.466701+0000","last_undegraded":"2023-10-06T05:18:38.466701+0000","last_fullsized":"2023-10-06T05:18:38.466701+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:26.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446996+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:52.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464573+0000","last_change":"2023-10-06T05:18:38.464573+0000","last_active":"2023-10-06T05:18:38.464573+0000","last_peered":"2023-10-06T05:18:38.464573+0000","last_clean":"2023-10-06T05:18:38.464573+0000","last_became_active":"2023-10-06T05:18:38.464303+0000","last_became_peered":"2023-10-06T05:18:38.464303+0000","last_unstale":"2023-10-06T05:18:38.464573+0000","last_undegraded":"2023-10-06T05:18:38.464573+0000","last_fullsized":"2023-10-06T05:18:38.464573+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451911+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:41.261742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458013+0000","last_change":"2023-10-06T05:18:38.458013+0000","last_active":"2023-10-06T05:18:38.458013+0000","last_peered":"2023-10-06T05:18:38.458013+0000","last_clean":"2023-10-06T05:18:38.458013+0000","last_became_active":"2023-10-06T05:18:38.457708+0000","last_became_peered":"2023-10-06T05:18:38.457708+0000","last_unstale":"2023-10-06T05:18:38.458013+0000","last_undegraded":"2023-10-06T05:18:38.458013+0000","last_fullsized":"2023-10-06T05:18:38.458013+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:35.907402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445801+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466371+0000","last_change":"2023-10-06T05:18:38.466371+0000","last_active":"2023-10-06T05:18:38.466371+0000","last_peered":"2023-10-06T05:18:38.466371+0000","last_clean":"2023-10-06T05:18:38.466371+0000","last_became_active":"2023-10-06T05:18:38.466099+0000","last_became_peered":"2023-10-06T05:18:38.466099+0000","last_unstale":"2023-10-06T05:18:38.466371+0000","last_undegraded":"2023-10-06T05:18:38.466371+0000","last_fullsized":"2023-10-06T05:18:38.466371+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:12.765494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454588+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:51.516975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.456427+0000","last_change":"2023-10-06T05:18:38.456427+0000","last_active":"2023-10-06T05:18:38.456427+0000","last_peered":"2023-10-06T05:18:38.456427+0000","last_clean":"2023-10-06T05:18:38.456427+0000","last_became_active":"2023-10-06T05:18:38.456172+0000","last_became_peered":"2023-10-06T05:18:38.456172+0000","last_unstale":"2023-10-06T05:18:38.456427+0000","last_undegraded":"2023-10-06T05:18:38.456427+0000","last_fullsized":"2023-10-06T05:18:38.456427+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:13.476914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443171+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466443+0000","last_change":"2023-10-06T05:18:38.466443+0000","last_active":"2023-10-06T05:18:38.466443+0000","last_peered":"2023-10-06T05:18:38.466443+0000","last_clean":"2023-10-06T05:18:38.466443+0000","last_became_active":"2023-10-06T05:18:38.465603+0000","last_became_peered":"2023-10-06T05:18:38.465603+0000","last_unstale":"2023-10-06T05:18:38.466443+0000","last_undegraded":"2023-10-06T05:18:38.466443+0000","last_fullsized":"2023-10-06T05:18:38.466443+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:08.665979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454720+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:09.410169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467188+0000","last_change":"2023-10-06T05:18:38.467188+0000","last_active":"2023-10-06T05:18:38.467188+0000","last_peered":"2023-10-06T05:18:38.467188+0000","last_clean":"2023-10-06T05:18:38.467188+0000","last_became_active":"2023-10-06T05:18:38.466475+0000","last_became_peered":"2023-10-06T05:18:38.466475+0000","last_unstale":"2023-10-06T05:18:38.467188+0000","last_undegraded":"2023-10-06T05:18:38.467188+0000","last_fullsized":"2023-10-06T05:18:38.467188+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:22.230340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451137+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:11.273331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.468654+0000","last_change":"2023-10-06T05:18:38.468654+0000","last_active":"2023-10-06T05:18:38.468654+0000","last_peered":"2023-10-06T05:18:38.468654+0000","last_clean":"2023-10-06T05:18:38.468654+0000","last_became_active":"2023-10-06T05:18:38.468394+0000","last_became_peered":"2023-10-06T05:18:38.468394+0000","last_unstale":"2023-10-06T05:18:38.468654+0000","last_undegraded":"2023-10-06T05:18:38.468654+0000","last_fullsized":"2023-10-06T05:18:38.468654+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:56.721340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452992+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:43.862504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460519+0000","last_change":"2023-10-06T05:18:38.460519+0000","last_active":"2023-10-06T05:18:38.460519+0000","last_peered":"2023-10-06T05:18:38.460519+0000","last_clean":"2023-10-06T05:18:38.460519+0000","last_became_active":"2023-10-06T05:18:38.460215+0000","last_became_peered":"2023-10-06T05:18:38.460215+0000","last_unstale":"2023-10-06T05:18:38.460519+0000","last_undegraded":"2023-10-06T05:18:38.460519+0000","last_fullsized":"2023-10-06T05:18:38.460519+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:12.619655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451676+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:27.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.462389+0000","last_change":"2023-10-06T05:18:38.462389+0000","last_active":"2023-10-06T05:18:38.462389+0000","last_peered":"2023-10-06T05:18:38.462389+0000","last_clean":"2023-10-06T05:18:38.462389+0000","last_became_active":"2023-10-06T05:18:38.462121+0000","last_became_peered":"2023-10-06T05:18:38.462121+0000","last_unstale":"2023-10-06T05:18:38.462389+0000","last_undegraded":"2023-10-06T05:18:38.462389+0000","last_fullsized":"2023-10-06T05:18:38.462389+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:23.179454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453051+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:15.086481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.457211+0000","last_change":"2023-10-06T05:18:38.457211+0000","last_active":"2023-10-06T05:18:38.457211+0000","last_peered":"2023-10-06T05:18:38.457211+0000","last_clean":"2023-10-06T05:18:38.457211+0000","last_became_active":"2023-10-06T05:18:38.456921+0000","last_became_peered":"2023-10-06T05:18:38.456921+0000","last_unstale":"2023-10-06T05:18:38.457211+0000","last_undegraded":"2023-10-06T05:18:38.457211+0000","last_fullsized":"2023-10-06T05:18:38.457211+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:11.790017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448636+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460312+0000","last_change":"2023-10-06T05:18:38.460312+0000","last_active":"2023-10-06T05:18:38.460312+0000","last_peered":"2023-10-06T05:18:38.460312+0000","last_clean":"2023-10-06T05:18:38.460312+0000","last_became_active":"2023-10-06T05:18:38.460043+0000","last_became_peered":"2023-10-06T05:18:38.460043+0000","last_unstale":"2023-10-06T05:18:38.460312+0000","last_undegraded":"2023-10-06T05:18:38.460312+0000","last_fullsized":"2023-10-06T05:18:38.460312+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:09.080286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.444681+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:18:37.444681+0000","last_peered":"2023-10-06T05:18:37.444681+0000","last_clean":"2023-10-06T05:18:37.444681+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:18:37.444681+0000","last_undegraded":"2023-10-06T05:18:37.444681+0000","last_fullsized":"2023-10-06T05:18:37.444681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452130+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458978+0000","last_change":"2023-10-06T05:18:38.458978+0000","last_active":"2023-10-06T05:18:38.458978+0000","last_peered":"2023-10-06T05:18:38.458978+0000","last_clean":"2023-10-06T05:18:38.458978+0000","last_became_active":"2023-10-06T05:18:38.458700+0000","last_became_peered":"2023-10-06T05:18:38.458700+0000","last_unstale":"2023-10-06T05:18:38.458978+0000","last_undegraded":"2023-10-06T05:18:38.458978+0000","last_fullsized":"2023-10-06T05:18:38.458978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:23.179454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.436719+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:18:37.436719+0000","last_peered":"2023-10-06T05:18:37.436719+0000","last_clean":"2023-10-06T05:18:37.436719+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:18:37.436719+0000","last_undegraded":"2023-10-06T05:18:37.436719+0000","last_fullsized":"2023-10-06T05:18:37.436719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455206+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:44.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460411+0000","last_change":"2023-10-06T05:18:38.460411+0000","last_active":"2023-10-06T05:18:38.460411+0000","last_peered":"2023-10-06T05:18:38.460411+0000","last_clean":"2023-10-06T05:18:38.460411+0000","last_became_active":"2023-10-06T05:18:38.460142+0000","last_became_peered":"2023-10-06T05:18:38.460142+0000","last_unstale":"2023-10-06T05:18:38.460411+0000","last_undegraded":"2023-10-06T05:18:38.460411+0000","last_fullsized":"2023-10-06T05:18:38.460411+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:10.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.434247+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:18:37.434247+0000","last_peered":"2023-10-06T05:18:37.434247+0000","last_clean":"2023-10-06T05:18:37.434247+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:18:37.434247+0000","last_undegraded":"2023-10-06T05:18:37.434247+0000","last_fullsized":"2023-10-06T05:18:37.434247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":180,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437257+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:18:37.437257+0000","last_peered":"2023-10-06T05:18:37.437257+0000","last_clean":"2023-10-06T05:18:37.437257+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:18:37.437257+0000","last_undegraded":"2023-10-06T05:18:37.437257+0000","last_fullsized":"2023-10-06T05:18:37.437257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445051+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463399+0000","last_change":"2023-10-06T05:18:38.463399+0000","last_active":"2023-10-06T05:18:38.463399+0000","last_peered":"2023-10-06T05:18:38.463399+0000","last_clean":"2023-10-06T05:18:38.463399+0000","last_became_active":"2023-10-06T05:18:38.463130+0000","last_became_peered":"2023-10-06T05:18:38.463130+0000","last_unstale":"2023-10-06T05:18:38.463399+0000","last_undegraded":"2023-10-06T05:18:38.463399+0000","last_fullsized":"2023-10-06T05:18:38.463399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:35.907402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437352+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:18:37.437352+0000","last_peered":"2023-10-06T05:18:37.437352+0000","last_clean":"2023-10-06T05:18:37.437352+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:18:37.437352+0000","last_undegraded":"2023-10-06T05:18:37.437352+0000","last_fullsized":"2023-10-06T05:18:37.437352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.457853+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:51.516975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463382+0000","last_change":"2023-10-06T05:18:38.463382+0000","last_active":"2023-10-06T05:18:38.463382+0000","last_peered":"2023-10-06T05:18:38.463382+0000","last_clean":"2023-10-06T05:18:38.463382+0000","last_became_active":"2023-10-06T05:18:38.463112+0000","last_became_peered":"2023-10-06T05:18:38.463112+0000","last_unstale":"2023-10-06T05:18:38.463382+0000","last_undegraded":"2023-10-06T05:18:38.463382+0000","last_fullsized":"2023-10-06T05:18:38.463382+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:37.839836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437251+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:18:37.437251+0000","last_peered":"2023-10-06T05:18:37.437251+0000","last_clean":"2023-10-06T05:18:37.437251+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:18:37.437251+0000","last_undegraded":"2023-10-06T05:18:37.437251+0000","last_fullsized":"2023-10-06T05:18:37.437251+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450529+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:42.744594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466799+0000","last_change":"2023-10-06T05:18:38.466799+0000","last_active":"2023-10-06T05:18:38.466799+0000","last_peered":"2023-10-06T05:18:38.466799+0000","last_clean":"2023-10-06T05:18:38.466799+0000","last_became_active":"2023-10-06T05:18:38.466486+0000","last_became_peered":"2023-10-06T05:18:38.466486+0000","last_unstale":"2023-10-06T05:18:38.466799+0000","last_undegraded":"2023-10-06T05:18:38.466799+0000","last_fullsized":"2023-10-06T05:18:38.466799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:04.252646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.439395+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:18:37.439395+0000","last_peered":"2023-10-06T05:18:37.439395+0000","last_clean":"2023-10-06T05:18:37.439395+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:18:37.439395+0000","last_undegraded":"2023-10-06T05:18:37.439395+0000","last_fullsized":"2023-10-06T05:18:37.439395+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445017+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467529+0000","last_change":"2023-10-06T05:18:38.467529+0000","last_active":"2023-10-06T05:18:38.467529+0000","last_peered":"2023-10-06T05:18:38.467529+0000","last_clean":"2023-10-06T05:18:38.467529+0000","last_became_active":"2023-10-06T05:18:38.467262+0000","last_became_peered":"2023-10-06T05:18:38.467262+0000","last_unstale":"2023-10-06T05:18:38.467529+0000","last_undegraded":"2023-10-06T05:18:38.467529+0000","last_fullsized":"2023-10-06T05:18:38.467529+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:34.215548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.440499+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:18:37.440499+0000","last_peered":"2023-10-06T05:18:37.440499+0000","last_clean":"2023-10-06T05:18:37.440499+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:18:37.440499+0000","last_undegraded":"2023-10-06T05:18:37.440499+0000","last_fullsized":"2023-10-06T05:18:37.440499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452193+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:39.628432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460634+0000","last_change":"2023-10-06T05:18:38.460634+0000","last_active":"2023-10-06T05:18:38.460634+0000","last_peered":"2023-10-06T05:18:38.460634+0000","last_clean":"2023-10-06T05:18:38.460634+0000","last_became_active":"2023-10-06T05:18:38.459850+0000","last_became_peered":"2023-10-06T05:18:38.459850+0000","last_unstale":"2023-10-06T05:18:38.460634+0000","last_undegraded":"2023-10-06T05:18:38.460634+0000","last_fullsized":"2023-10-06T05:18:38.460634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:10.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.440631+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:18:37.440631+0000","last_peered":"2023-10-06T05:18:37.440631+0000","last_clean":"2023-10-06T05:18:37.440631+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:18:37.440631+0000","last_undegraded":"2023-10-06T05:18:37.440631+0000","last_fullsized":"2023-10-06T05:18:37.440631+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298348,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073492,"statfs":{"total":96636764160,"available":96331255808,"internally_reserved":0,"allocated":1085440,"data_stored":796913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78300000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.871},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.62},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77400000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298336,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073504,"statfs":{"total":96636764160,"available":96331268096,"internally_reserved":0,"allocated":1073152,"data_stored":790297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.433}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.95999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.94099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.006}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.82399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297756,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074084,"statfs":{"total":96636764160,"available":96331862016,"internally_reserved":0,"allocated":479232,"data_stored":199929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.51300000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.749},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.81899999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.91700000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.84599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.004}]},{"osd":7,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.0169999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.371}]},{"osd":1,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.754},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.78100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":7.742,"5min":7.742,"15min":7.742},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.88500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.87},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.85799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2780000000000005,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":9.25,"5min":9.25,"15min":9.25},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.108000000000001,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.95799999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":10.384,"5min":10.384,"15min":10.384},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":9.1699999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.205,"5min":10.205,"15min":10.205},"last":1.038},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.83299999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":1.113}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.0920000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:44.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2023-10-06T05:18:44.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.453619+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:29:33.498919+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:18:44.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2023-10-06T05:18:44.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.448171+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:50:24.194372+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:18:44.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2023-10-06T05:18:44.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.449706+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2023-10-06T05:18:44.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.447164+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:18:44.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2023-10-06T05:18:44.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.444195+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:18:44.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2023-10-06T05:18:44.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.455710+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:29:17.100290+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:18:44.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2023-10-06T05:18:44.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.448958+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2023-10-06T05:18:44.839 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.446650+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:04:27.404627+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2023-10-06T05:18:44.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.446143+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:30:10.283521+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2023-10-06T05:18:44.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.453249+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:37:05.369497+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:18:44.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2023-10-06T05:18:44.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.455141+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:55:34.252333+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:18:44.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2023-10-06T05:18:44.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.445402+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2023-10-06T05:18:44.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.455343+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:52:52.371640+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:18:44.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2023-10-06T05:18:44.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.451248+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:30:10.283521+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2023-10-06T05:18:44.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.447669+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:18:44.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2023-10-06T05:18:44.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.454977+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:25:50.655754+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:18:44.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2023-10-06T05:18:44.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.452837+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:28:19.930105+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2023-10-06T05:18:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.446943+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:18:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2023-10-06T05:18:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.448342+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2023-10-06T05:18:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.456888+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:29:33.498919+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:18:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2023-10-06T05:18:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.451223+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:00:56.384273+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:18:44.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2023-10-06T05:18:44.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.445385+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2023-10-06T05:18:44.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.451398+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:14:56.125739+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:18:44.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2023-10-06T05:18:44.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.449458+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2023-10-06T05:18:44.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.452400+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:45:06.022509+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2023-10-06T05:18:44.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.444580+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2023-10-06T05:18:44.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.454129+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:57:39.631263+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:18:44.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2023-10-06T05:18:44.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.456353+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:37:05.369497+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:18:44.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2023-10-06T05:18:44.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.450548+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2023-10-06T05:18:44.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.447835+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:14:36.913820+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:18:44.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2023-10-06T05:18:44.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.451478+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T17:10:52.082838+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2023-10-06T05:18:44.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.451233+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:18:44.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-10-06T05:18:44.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.446889+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:18:44.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-10-06T05:18:44.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.450735+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:33:28.365507+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:18:44.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-10-06T05:18:44.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.457317+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:57:39.631263+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:18:44.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-10-06T05:18:44.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.445717+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-10-06T05:18:44.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.443349+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:18:44.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-10-06T05:18:44.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.447205+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:18:44.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-10-06T05:18:44.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.446502+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-10-06T05:18:44.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.447681+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:40:13.626073+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:18:44.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-10-06T05:18:44.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.443147+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:18:44.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-10-06T05:18:44.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.451319+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:18:44.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-10-06T05:18:44.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.450043+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-10-06T05:18:44.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.443576+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:18:44.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-10-06T05:18:44.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.454251+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:18:44.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-10-06T05:18:44.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.448983+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-10-06T05:18:44.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.446996+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T17:10:52.082838+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-10-06T05:18:44.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.451911+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:17:41.261742+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-10-06T05:18:44.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.445801+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-10-06T05:18:44.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.454588+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:27:51.516975+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:18:44.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-10-06T05:18:44.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.443171+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:18:44.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-10-06T05:18:44.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.454720+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:06:09.410169+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:18:44.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-10-06T05:18:44.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.451137+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:53:11.273331+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:18:44.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-10-06T05:18:44.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.452992+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:21:43.862504+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:18:44.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-10-06T05:18:44.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.451676+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:04:27.404627+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-10-06T05:18:44.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.453051+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:49:15.086481+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:18:44.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-10-06T05:18:44.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.448636+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-10-06T05:18:44.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.452130+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:18:44.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-10-06T05:18:44.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.455206+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:20:44.210443+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:18:44.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-10-06T05:18:44.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.445051+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-10-06T05:18:44.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.457853+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:27:51.516975+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:18:44.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-10-06T05:18:44.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.450529+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:20:42.744594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:18:44.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-10-06T05:18:44.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.445017+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:18:44.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-10-06T05:18:44.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:18:38.431948+0000', 'last_change': '2023-10-06T05:18:38.452193+0000', 'last_active': '2023-10-06T05:18:38.431948+0000', 'last_peered': '2023-10-06T05:18:38.431948+0000', 'last_clean': '2023-10-06T05:18:38.431948+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:18:38.431948+0000', 'last_undegraded': '2023-10-06T05:18:38.431948+0000', 'last_fullsized': '2023-10-06T05:18:38.431948+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'last_clean_scrub_stamp': '2023-10-06T05:18:38.431948+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:11:39.628432+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:18:44.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T05:18:45.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.064+0000 7f3e349ff700 1 Processor -- start 2023-10-06T05:18:45.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.064+0000 7f3e349ff700 1 -- start start 2023-10-06T05:18:45.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e349ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3e30159500 0x7f3e300b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e349ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e3015d2d0 0x7f3e300cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e349ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e301598d0 0x7f3e300cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e349ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e300c3f40 con 0x7f3e301598d0 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e349ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e300cb000 con 0x7f3e30159500 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e349ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e300b2860 con 0x7f3e3015d2d0 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e2ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e301598d0 0x7f3e300cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e2ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e301598d0 0x7f3e300cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56940/0 (socket says 172.21.15.31:56940) 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e2ed9d700 1 -- 172.21.15.31:0/689620858 learned_addr learned my addr 172.21.15.31:0/689620858 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e2e59c700 1 --2- 172.21.15.31:0/689620858 >> v2:172.21.15.62:3300/0 conn(0x7f3e30159500 0x7f3e300b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.065+0000 7f3e2dd9b700 1 --2- 172.21.15.31:0/689620858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e3015d2d0 0x7f3e300cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.066+0000 7f3e2ed9d700 1 -- 172.21.15.31:0/689620858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e3015d2d0 msgr2=0x7f3e300cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:45.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.066+0000 7f3e2ed9d700 1 --2- 172.21.15.31:0/689620858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e3015d2d0 0x7f3e300cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.066+0000 7f3e2ed9d700 1 -- 172.21.15.31:0/689620858 >> v2:172.21.15.62:3300/0 conn(0x7f3e30159500 msgr2=0x7f3e300b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:45.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.066+0000 7f3e2ed9d700 1 --2- 172.21.15.31:0/689620858 >> v2:172.21.15.62:3300/0 conn(0x7f3e30159500 0x7f3e300b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.066+0000 7f3e2ed9d700 1 -- 172.21.15.31:0/689620858 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3015d7d0 con 0x7f3e301598d0 2023-10-06T05:18:45.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.066+0000 7f3e2ed9d700 1 --2- 172.21.15.31:0/689620858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 0x7f3e300cd340 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3e30089840 tx=0x7f3e28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bcce220e116511 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:45.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.066+0000 7f3e2d59a700 1 -- 172.21.15.31:0/689620858 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800e070 con 0x7f3e301598d0 2023-10-06T05:18:45.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.066+0000 7f3e2d59a700 1 -- 172.21.15.31:0/689620858 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2800b970 con 0x7f3e301598d0 2023-10-06T05:18:45.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.066+0000 7f3e2d59a700 1 -- 172.21.15.31:0/689620858 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e280092e0 con 0x7f3e301598d0 2023-10-06T05:18:45.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.067+0000 7f3e349ff700 1 -- 172.21.15.31:0/689620858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 msgr2=0x7f3e300cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.067+0000 7f3e349ff700 1 --2- 172.21.15.31:0/689620858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 0x7f3e300cd340 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3e30089840 tx=0x7f3e28009840 comp rx=0 tx=0).stop 2023-10-06T05:18:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.067+0000 7f3e349ff700 1 -- 172.21.15.31:0/689620858 shutdown_connections 2023-10-06T05:18:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.067+0000 7f3e349ff700 1 --2- 172.21.15.31:0/689620858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 0x7f3e300cd340 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.067+0000 7f3e349ff700 1 --2- 172.21.15.31:0/689620858 >> v2:172.21.15.62:3300/0 conn(0x7f3e30159500 0x7f3e300b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.067+0000 7f3e349ff700 1 --2- 172.21.15.31:0/689620858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e3015d2d0 0x7f3e300cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.067+0000 7f3e349ff700 1 -- 172.21.15.31:0/689620858 >> 172.21.15.31:0/689620858 conn(0x7f3e3006ec90 msgr2=0x7f3e3006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.067+0000 7f3e349ff700 1 -- 172.21.15.31:0/689620858 shutdown_connections 2023-10-06T05:18:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.067+0000 7f3e349ff700 1 -- 172.21.15.31:0/689620858 wait complete. 2023-10-06T05:18:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.069+0000 7f3e349ff700 1 Processor -- start 2023-10-06T05:18:45.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.069+0000 7f3e349ff700 1 -- start start 2023-10-06T05:18:45.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e349ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e30159500 0x7f3e300c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e349ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 0x7f3e30169a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e349ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e3015d2d0 0x7f3e30169f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e349ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e300c8850 con 0x7f3e301598d0 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e349ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e30158a50 con 0x7f3e3015d2d0 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e349ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e300c74d0 con 0x7f3e30159500 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 0x7f3e30169a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 0x7f3e30169a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56942/0 (socket says 172.21.15.31:56942) 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2dd9b700 1 -- 172.21.15.31:0/97607504 learned_addr learned my addr 172.21.15.31:0/97607504 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2ed9d700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e3015d2d0 0x7f3e30169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2e59c700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e30159500 0x7f3e300c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2dd9b700 1 -- 172.21.15.31:0/97607504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e30159500 msgr2=0x7f3e300c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2dd9b700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e30159500 0x7f3e300c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2dd9b700 1 -- 172.21.15.31:0/97607504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e3015d2d0 msgr2=0x7f3e30169f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2dd9b700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e3015d2d0 0x7f3e30169f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2dd9b700 1 -- 172.21.15.31:0/97607504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e28009ae0 con 0x7f3e301598d0 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.070+0000 7f3e2ed9d700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e3015d2d0 0x7f3e30169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.071+0000 7f3e2dd9b700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 0x7f3e30169a30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400ea50 tx=0x7f3e24009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.071+0000 7f3e1e7fc700 1 -- 172.21.15.31:0/97607504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2400c850 con 0x7f3e301598d0 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.071+0000 7f3e1e7fc700 1 -- 172.21.15.31:0/97607504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2400c9b0 con 0x7f3e301598d0 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.071+0000 7f3e349ff700 1 -- 172.21.15.31:0/97607504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e300c77b0 con 0x7f3e301598d0 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.071+0000 7f3e1e7fc700 1 -- 172.21.15.31:0/97607504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2400cd70 con 0x7f3e301598d0 2023-10-06T05:18:45.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.071+0000 7f3e349ff700 1 -- 172.21.15.31:0/97607504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e300c7d70 con 0x7f3e301598d0 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.072+0000 7f3e349ff700 1 -- 172.21.15.31:0/97607504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e300ccf40 con 0x7f3e301598d0 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.073+0000 7f3e1e7fc700 1 -- 172.21.15.31:0/97607504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f3e24013680 con 0x7f3e301598d0 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.074+0000 7f3e1e7fc700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e100670e0 0x7f3e10069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.074+0000 7f3e1e7fc700 1 -- 172.21.15.31:0/97607504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3e24085090 con 0x7f3e301598d0 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.074+0000 7f3e2e59c700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e100670e0 0x7f3e10069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.075+0000 7f3e2e59c700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e100670e0 0x7f3e10069590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3e300ced60 tx=0x7f3e20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:45.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.078+0000 7f3e1e7fc700 1 -- 172.21.15.31:0/97607504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e24051e80 con 0x7f3e301598d0 2023-10-06T05:18:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.361+0000 7f3e349ff700 1 -- 172.21.15.31:0/97607504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3e3004c380 con 0x7f3e301598d0 2023-10-06T05:18:45.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.361+0000 7f3e1e7fc700 1 -- 172.21.15.31:0/97607504 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1585 (secure 0 0 0) 0x7f3e24017650 con 0x7f3e301598d0 2023-10-06T05:18:45.364 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:45.364 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":124,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":44,"num_osds":8,"num_up_osds":8,"osd_up_since":1696569413,"num_in_osds":8,"osd_in_since":1696569404,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":2440478720,"bytes_avail":770653634560,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5355}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-10-06T05:17:56.072424+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T05:18:45.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.363+0000 7f3e349ff700 1 -- 172.21.15.31:0/97607504 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e100670e0 msgr2=0x7f3e10069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.363+0000 7f3e349ff700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e100670e0 0x7f3e10069590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3e300ced60 tx=0x7f3e20008040 comp rx=0 tx=0).stop 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.363+0000 7f3e349ff700 1 -- 172.21.15.31:0/97607504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 msgr2=0x7f3e30169a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.363+0000 7f3e349ff700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 0x7f3e30169a30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400ea50 tx=0x7f3e24009120 comp rx=0 tx=0).stop 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.363+0000 7f3e349ff700 1 -- 172.21.15.31:0/97607504 shutdown_connections 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.363+0000 7f3e349ff700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e3015d2d0 0x7f3e30169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.363+0000 7f3e349ff700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e30159500 0x7f3e300c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.363+0000 7f3e349ff700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3e100670e0 0x7f3e10069590 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.363+0000 7f3e349ff700 1 --2- 172.21.15.31:0/97607504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e301598d0 0x7f3e30169a30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.364+0000 7f3e349ff700 1 -- 172.21.15.31:0/97607504 >> 172.21.15.31:0/97607504 conn(0x7f3e3006ec90 msgr2=0x7f3e300cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.364+0000 7f3e349ff700 1 -- 172.21.15.31:0/97607504 shutdown_connections 2023-10-06T05:18:45.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.364+0000 7f3e349ff700 1 -- 172.21.15.31:0/97607504 wait complete. 2023-10-06T05:18:45.383 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T05:18:45.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:45.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.577+0000 7fb6f13ff700 1 Processor -- start 2023-10-06T05:18:45.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.577+0000 7fb6f13ff700 1 -- start start 2023-10-06T05:18:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.578+0000 7fb6f13ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6ec1595a0 0x7fb6ec159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.578+0000 7fb6f13ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6ec0cc0d0 0x7fb6ec068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.578+0000 7fb6f13ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ec069010 con 0x7fb6ec068bb0 2023-10-06T05:18:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.578+0000 7fb6f13ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec0c95b0 con 0x7fb6ec0cc0d0 2023-10-06T05:18:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.578+0000 7fb6f13ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0c9890 con 0x7fb6ec1595a0 2023-10-06T05:18:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.579+0000 7fb6ea7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6ec0cc0d0 0x7fb6ec068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.579+0000 7fb6ea7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6ec0cc0d0 0x7fb6ec068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56958/0 (socket says 172.21.15.31:56958) 2023-10-06T05:18:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.579+0000 7fb6ea7fc700 1 -- 172.21.15.31:0/2282245660 learned_addr learned my addr 172.21.15.31:0/2282245660 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:45.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.579+0000 7fb6eb7fe700 1 --1- 172.21.15.31:0/2282245660 >> v1:172.21.15.62:6789/0 conn(0x7fb6ec068bb0 0x7fb6ec0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:59602/0 (socket says 172.21.15.31:59602) 2023-10-06T05:18:45.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.579+0000 7fb6eaffd700 1 --2- 172.21.15.31:0/2282245660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6ec1595a0 0x7fb6ec159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.579+0000 7fb6ea7fc700 1 -- 172.21.15.31:0/2282245660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6ec1595a0 msgr2=0x7fb6ec159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:45.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.579+0000 7fb6ea7fc700 1 --2- 172.21.15.31:0/2282245660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6ec1595a0 0x7fb6ec159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.579+0000 7fb6ea7fc700 1 -- 172.21.15.31:0/2282245660 >> v1:172.21.15.62:6789/0 conn(0x7fb6ec068bb0 legacy=0x7fb6ec0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.579+0000 7fb6ea7fc700 1 -- 172.21.15.31:0/2282245660 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0c9b70 con 0x7fb6ec0cc0d0 2023-10-06T05:18:45.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.580+0000 7fb6ea7fc700 1 --2- 172.21.15.31:0/2282245660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec0cc0d0 0x7fb6ec068670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00099b0 tx=0x7fb6e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a3148ba2aaa8e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:45.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.580+0000 7fb6e9ffb700 1 -- 172.21.15.31:0/2282245660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0009ea0 con 0x7fb6ec0cc0d0 2023-10-06T05:18:45.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.580+0000 7fb6e9ffb700 1 -- 172.21.15.31:0/2282245660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e000b860 con 0x7fb6ec0cc0d0 2023-10-06T05:18:45.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.580+0000 7fb6e9ffb700 1 -- 172.21.15.31:0/2282245660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e000bc00 con 0x7fb6ec0cc0d0 2023-10-06T05:18:45.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.580+0000 7fb6f13ff700 1 -- 172.21.15.31:0/2282245660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec0cc0d0 msgr2=0x7fb6ec068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.580+0000 7fb6f13ff700 1 --2- 172.21.15.31:0/2282245660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec0cc0d0 0x7fb6ec068670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00099b0 tx=0x7fb6e0009250 comp rx=0 tx=0).stop 2023-10-06T05:18:45.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.580+0000 7fb6f13ff700 1 -- 172.21.15.31:0/2282245660 shutdown_connections 2023-10-06T05:18:45.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.580+0000 7fb6f13ff700 1 --2- 172.21.15.31:0/2282245660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6ec1595a0 0x7fb6ec159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.580+0000 7fb6f13ff700 1 --2- 172.21.15.31:0/2282245660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec0cc0d0 0x7fb6ec068670 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.580+0000 7fb6f13ff700 1 -- 172.21.15.31:0/2282245660 >> 172.21.15.31:0/2282245660 conn(0x7fb6ec06b620 msgr2=0x7fb6ec06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:45.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.581+0000 7fb6f13ff700 1 -- 172.21.15.31:0/2282245660 shutdown_connections 2023-10-06T05:18:45.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.581+0000 7fb6f13ff700 1 -- 172.21.15.31:0/2282245660 wait complete. 2023-10-06T05:18:45.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.583+0000 7fb6f13ff700 1 Processor -- start 2023-10-06T05:18:45.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.583+0000 7fb6f13ff700 1 -- start start 2023-10-06T05:18:45.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.584+0000 7fb6f13ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec068bb0 0x7fb6ec0b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.584+0000 7fb6f13ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6ec0cc0d0 0x7fb6ec137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.584+0000 7fb6f13ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6ec1595a0 0x7fb6ec138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.584+0000 7fb6f13ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec138880 con 0x7fb6ec068bb0 2023-10-06T05:18:45.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.584+0000 7fb6f13ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec138ad0 con 0x7fb6ec0cc0d0 2023-10-06T05:18:45.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.584+0000 7fb6f13ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0b28a0 con 0x7fb6ec1595a0 2023-10-06T05:18:45.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.584+0000 7fb6eaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec068bb0 0x7fb6ec0b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.584+0000 7fb6eaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec068bb0 0x7fb6ec0b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56974/0 (socket says 172.21.15.31:56974) 2023-10-06T05:18:45.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.584+0000 7fb6eaffd700 1 -- 172.21.15.31:0/1384573276 learned_addr learned my addr 172.21.15.31:0/1384573276 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:45.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6ea7fc700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6ec0cc0d0 0x7fb6ec137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6eaffd700 1 -- 172.21.15.31:0/1384573276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6ec1595a0 msgr2=0x7fb6ec138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6eaffd700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6ec1595a0 0x7fb6ec138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6eaffd700 1 -- 172.21.15.31:0/1384573276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6ec0cc0d0 msgr2=0x7fb6ec137e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6eaffd700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6ec0cc0d0 0x7fb6ec137e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6eaffd700 1 -- 172.21.15.31:0/1384573276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e00094f0 con 0x7fb6ec068bb0 2023-10-06T05:18:45.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6ea7fc700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6ec0cc0d0 0x7fb6ec137e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:45.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6eaffd700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec068bb0 0x7fb6ec0b1440 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00e930 tx=0x7fb6dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6d2ffd700 1 -- 172.21.15.31:0/1384573276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00eed0 con 0x7fb6ec068bb0 2023-10-06T05:18:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6d2ffd700 1 -- 172.21.15.31:0/1384573276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc00c8e0 con 0x7fb6ec068bb0 2023-10-06T05:18:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6d2ffd700 1 -- 172.21.15.31:0/1384573276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00cc80 con 0x7fb6ec068bb0 2023-10-06T05:18:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6f13ff700 1 -- 172.21.15.31:0/1384573276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ec0b1c90 con 0x7fb6ec068bb0 2023-10-06T05:18:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.585+0000 7fb6f13ff700 1 -- 172.21.15.31:0/1384573276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec0b2250 con 0x7fb6ec068bb0 2023-10-06T05:18:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.586+0000 7fb6f13ff700 1 -- 172.21.15.31:0/1384573276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ec04c380 con 0x7fb6ec068bb0 2023-10-06T05:18:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.587+0000 7fb6d2ffd700 1 -- 172.21.15.31:0/1384573276 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fb6dc013680 con 0x7fb6ec068bb0 2023-10-06T05:18:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.588+0000 7fb6d2ffd700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb6d8067110 0x7fb6d80695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.588+0000 7fb6d2ffd700 1 -- 172.21.15.31:0/1384573276 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb6dc084c40 con 0x7fb6ec068bb0 2023-10-06T05:18:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.589+0000 7fb6ea7fc700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb6d8067110 0x7fb6d80695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.589+0000 7fb6ea7fc700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb6d8067110 0x7fb6d80695c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00099b0 tx=0x7fb6e0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.592+0000 7fb6d2ffd700 1 -- 172.21.15.31:0/1384573276 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6dc051ad0 con 0x7fb6ec068bb0 2023-10-06T05:18:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.791+0000 7fb6f13ff700 1 -- 172.21.15.31:0/1384573276 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6ec0b29e0 con 0x7fb6d8067110 2023-10-06T05:18:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.800+0000 7fb6d2ffd700 1 -- 172.21.15.31:0/1384573276 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308313 (secure 0 0 0) 0x7fb6e000b9d0 con 0x7fb6d8067110 2023-10-06T05:18:45.805 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:45.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 -- 172.21.15.31:0/1384573276 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb6d8067110 msgr2=0x7fb6d80695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb6d8067110 0x7fb6d80695c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00099b0 tx=0x7fb6e0017000 comp rx=0 tx=0).stop 2023-10-06T05:18:45.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 -- 172.21.15.31:0/1384573276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec068bb0 msgr2=0x7fb6ec0b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:45.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec068bb0 0x7fb6ec0b1440 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00e930 tx=0x7fb6dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 -- 172.21.15.31:0/1384573276 shutdown_connections 2023-10-06T05:18:45.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb6ec1595a0 0x7fb6ec138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb6d8067110 0x7fb6d80695c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec068bb0 0x7fb6ec0b1440 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 --2- 172.21.15.31:0/1384573276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6ec0cc0d0 0x7fb6ec137e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 -- 172.21.15.31:0/1384573276 >> 172.21.15.31:0/1384573276 conn(0x7fb6ec06b620 msgr2=0x7fb6ec15b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 -- 172.21.15.31:0/1384573276 shutdown_connections 2023-10-06T05:18:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:45.804+0000 7fb6f13ff700 1 -- 172.21.15.31:0/1384573276 wait complete. 2023-10-06T05:18:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:18:45.823 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-10-06T05:18:44.082072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383280,"kb_used_data":4976,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591440,"statfs":{"total":773094113280,"available":770653634560,"internally_reserved":0,"allocated":5095424,"data_stored":2819864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001132"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453619+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:33.498919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448171+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:24.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.449706+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447164+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.444195+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455710+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:17.100290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448958+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446650+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:27.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446143+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:10.283521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453249+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:05.369497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455141+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:34.252333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445402+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455343+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:52.371640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451248+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:10.283521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447669+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454977+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:50.655754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452837+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:19.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446943+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448342+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.456888+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:33.498919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451223+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:56.384273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445385+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451398+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:56.125739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.449458+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452400+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:06.022509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.444580+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454129+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:39.631263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.456353+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:05.369497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450548+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447835+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:36.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451478+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:52.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451233+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446889+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464327+0000","last_change":"2023-10-06T05:18:38.464327+0000","last_active":"2023-10-06T05:18:38.464327+0000","last_peered":"2023-10-06T05:18:38.464327+0000","last_clean":"2023-10-06T05:18:38.464327+0000","last_became_active":"2023-10-06T05:18:38.464063+0000","last_became_peered":"2023-10-06T05:18:38.464063+0000","last_unstale":"2023-10-06T05:18:38.464327+0000","last_undegraded":"2023-10-06T05:18:38.464327+0000","last_fullsized":"2023-10-06T05:18:38.464327+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:47.847969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450735+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:28.365507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467137+0000","last_change":"2023-10-06T05:18:38.467137+0000","last_active":"2023-10-06T05:18:38.467137+0000","last_peered":"2023-10-06T05:18:38.467137+0000","last_clean":"2023-10-06T05:18:38.467137+0000","last_became_active":"2023-10-06T05:18:38.466871+0000","last_became_peered":"2023-10-06T05:18:38.466871+0000","last_unstale":"2023-10-06T05:18:38.467137+0000","last_undegraded":"2023-10-06T05:18:38.467137+0000","last_fullsized":"2023-10-06T05:18:38.467137+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:26.910024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.457317+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:39.631263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.465625+0000","last_change":"2023-10-06T05:18:38.465625+0000","last_active":"2023-10-06T05:18:38.465625+0000","last_peered":"2023-10-06T05:18:38.465625+0000","last_clean":"2023-10-06T05:18:38.465625+0000","last_became_active":"2023-10-06T05:18:38.465367+0000","last_became_peered":"2023-10-06T05:18:38.465367+0000","last_unstale":"2023-10-06T05:18:38.465625+0000","last_undegraded":"2023-10-06T05:18:38.465625+0000","last_fullsized":"2023-10-06T05:18:38.465625+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:55.377855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445717+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466328+0000","last_change":"2023-10-06T05:18:38.466328+0000","last_active":"2023-10-06T05:18:38.466328+0000","last_peered":"2023-10-06T05:18:38.466328+0000","last_clean":"2023-10-06T05:18:38.466328+0000","last_became_active":"2023-10-06T05:18:38.466033+0000","last_became_peered":"2023-10-06T05:18:38.466033+0000","last_unstale":"2023-10-06T05:18:38.466328+0000","last_undegraded":"2023-10-06T05:18:38.466328+0000","last_fullsized":"2023-10-06T05:18:38.466328+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:23.634297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443349+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464675+0000","last_change":"2023-10-06T05:18:38.464675+0000","last_active":"2023-10-06T05:18:38.464675+0000","last_peered":"2023-10-06T05:18:38.464675+0000","last_clean":"2023-10-06T05:18:38.464675+0000","last_became_active":"2023-10-06T05:18:38.464409+0000","last_became_peered":"2023-10-06T05:18:38.464409+0000","last_unstale":"2023-10-06T05:18:38.464675+0000","last_undegraded":"2023-10-06T05:18:38.464675+0000","last_fullsized":"2023-10-06T05:18:38.464675+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:09.080286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447205+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.465029+0000","last_change":"2023-10-06T05:18:38.465029+0000","last_active":"2023-10-06T05:18:38.465029+0000","last_peered":"2023-10-06T05:18:38.465029+0000","last_clean":"2023-10-06T05:18:38.465029+0000","last_became_active":"2023-10-06T05:18:38.464703+0000","last_became_peered":"2023-10-06T05:18:38.464703+0000","last_unstale":"2023-10-06T05:18:38.465029+0000","last_undegraded":"2023-10-06T05:18:38.465029+0000","last_fullsized":"2023-10-06T05:18:38.465029+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446502+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460564+0000","last_change":"2023-10-06T05:18:38.460564+0000","last_active":"2023-10-06T05:18:38.460564+0000","last_peered":"2023-10-06T05:18:38.460564+0000","last_clean":"2023-10-06T05:18:38.460564+0000","last_became_active":"2023-10-06T05:18:38.460269+0000","last_became_peered":"2023-10-06T05:18:38.460269+0000","last_unstale":"2023-10-06T05:18:38.460564+0000","last_undegraded":"2023-10-06T05:18:38.460564+0000","last_fullsized":"2023-10-06T05:18:38.460564+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:33.245915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.447681+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:13.626073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.461183+0000","last_change":"2023-10-06T05:18:38.461183+0000","last_active":"2023-10-06T05:18:38.461183+0000","last_peered":"2023-10-06T05:18:38.461183+0000","last_clean":"2023-10-06T05:18:38.461183+0000","last_became_active":"2023-10-06T05:18:38.460893+0000","last_became_peered":"2023-10-06T05:18:38.460893+0000","last_unstale":"2023-10-06T05:18:38.461183+0000","last_undegraded":"2023-10-06T05:18:38.461183+0000","last_fullsized":"2023-10-06T05:18:38.461183+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:12.619655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443147+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458370+0000","last_change":"2023-10-06T05:18:38.458370+0000","last_active":"2023-10-06T05:18:38.458370+0000","last_peered":"2023-10-06T05:18:38.458370+0000","last_clean":"2023-10-06T05:18:38.458370+0000","last_became_active":"2023-10-06T05:18:38.458104+0000","last_became_peered":"2023-10-06T05:18:38.458104+0000","last_unstale":"2023-10-06T05:18:38.458370+0000","last_undegraded":"2023-10-06T05:18:38.458370+0000","last_fullsized":"2023-10-06T05:18:38.458370+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:41.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451319+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:24.185872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463433+0000","last_change":"2023-10-06T05:18:38.463433+0000","last_active":"2023-10-06T05:18:38.463433+0000","last_peered":"2023-10-06T05:18:38.463433+0000","last_clean":"2023-10-06T05:18:38.463433+0000","last_became_active":"2023-10-06T05:18:38.463176+0000","last_became_peered":"2023-10-06T05:18:38.463176+0000","last_unstale":"2023-10-06T05:18:38.463433+0000","last_undegraded":"2023-10-06T05:18:38.463433+0000","last_fullsized":"2023-10-06T05:18:38.463433+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:41.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450043+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.461448+0000","last_change":"2023-10-06T05:18:38.461448+0000","last_active":"2023-10-06T05:18:38.461448+0000","last_peered":"2023-10-06T05:18:38.461448+0000","last_clean":"2023-10-06T05:18:38.461448+0000","last_became_active":"2023-10-06T05:18:38.461184+0000","last_became_peered":"2023-10-06T05:18:38.461184+0000","last_unstale":"2023-10-06T05:18:38.461448+0000","last_undegraded":"2023-10-06T05:18:38.461448+0000","last_fullsized":"2023-10-06T05:18:38.461448+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:34.215548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443576+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460848+0000","last_change":"2023-10-06T05:18:38.460848+0000","last_active":"2023-10-06T05:18:38.460848+0000","last_peered":"2023-10-06T05:18:38.460848+0000","last_clean":"2023-10-06T05:18:38.460848+0000","last_became_active":"2023-10-06T05:18:38.460598+0000","last_became_peered":"2023-10-06T05:18:38.460598+0000","last_unstale":"2023-10-06T05:18:38.460848+0000","last_undegraded":"2023-10-06T05:18:38.460848+0000","last_fullsized":"2023-10-06T05:18:38.460848+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:12.765494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454251+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.086704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463872+0000","last_change":"2023-10-06T05:18:38.463872+0000","last_active":"2023-10-06T05:18:38.463872+0000","last_peered":"2023-10-06T05:18:38.463872+0000","last_clean":"2023-10-06T05:18:38.463872+0000","last_became_active":"2023-10-06T05:18:38.463587+0000","last_became_peered":"2023-10-06T05:18:38.463587+0000","last_unstale":"2023-10-06T05:18:38.463872+0000","last_undegraded":"2023-10-06T05:18:38.463872+0000","last_fullsized":"2023-10-06T05:18:38.463872+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:27.960703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448983+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466701+0000","last_change":"2023-10-06T05:18:38.466701+0000","last_active":"2023-10-06T05:18:38.466701+0000","last_peered":"2023-10-06T05:18:38.466701+0000","last_clean":"2023-10-06T05:18:38.466701+0000","last_became_active":"2023-10-06T05:18:38.466409+0000","last_became_peered":"2023-10-06T05:18:38.466409+0000","last_unstale":"2023-10-06T05:18:38.466701+0000","last_undegraded":"2023-10-06T05:18:38.466701+0000","last_fullsized":"2023-10-06T05:18:38.466701+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:26.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.446996+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:52.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.464573+0000","last_change":"2023-10-06T05:18:38.464573+0000","last_active":"2023-10-06T05:18:38.464573+0000","last_peered":"2023-10-06T05:18:38.464573+0000","last_clean":"2023-10-06T05:18:38.464573+0000","last_became_active":"2023-10-06T05:18:38.464303+0000","last_became_peered":"2023-10-06T05:18:38.464303+0000","last_unstale":"2023-10-06T05:18:38.464573+0000","last_undegraded":"2023-10-06T05:18:38.464573+0000","last_fullsized":"2023-10-06T05:18:38.464573+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451911+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:41.261742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458013+0000","last_change":"2023-10-06T05:18:38.458013+0000","last_active":"2023-10-06T05:18:38.458013+0000","last_peered":"2023-10-06T05:18:38.458013+0000","last_clean":"2023-10-06T05:18:38.458013+0000","last_became_active":"2023-10-06T05:18:38.457708+0000","last_became_peered":"2023-10-06T05:18:38.457708+0000","last_unstale":"2023-10-06T05:18:38.458013+0000","last_undegraded":"2023-10-06T05:18:38.458013+0000","last_fullsized":"2023-10-06T05:18:38.458013+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:35.907402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445801+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466371+0000","last_change":"2023-10-06T05:18:38.466371+0000","last_active":"2023-10-06T05:18:38.466371+0000","last_peered":"2023-10-06T05:18:38.466371+0000","last_clean":"2023-10-06T05:18:38.466371+0000","last_became_active":"2023-10-06T05:18:38.466099+0000","last_became_peered":"2023-10-06T05:18:38.466099+0000","last_unstale":"2023-10-06T05:18:38.466371+0000","last_undegraded":"2023-10-06T05:18:38.466371+0000","last_fullsized":"2023-10-06T05:18:38.466371+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:12.765494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454588+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:51.516975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.456427+0000","last_change":"2023-10-06T05:18:38.456427+0000","last_active":"2023-10-06T05:18:38.456427+0000","last_peered":"2023-10-06T05:18:38.456427+0000","last_clean":"2023-10-06T05:18:38.456427+0000","last_became_active":"2023-10-06T05:18:38.456172+0000","last_became_peered":"2023-10-06T05:18:38.456172+0000","last_unstale":"2023-10-06T05:18:38.456427+0000","last_undegraded":"2023-10-06T05:18:38.456427+0000","last_fullsized":"2023-10-06T05:18:38.456427+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:13.476914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.443171+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:28.967122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466443+0000","last_change":"2023-10-06T05:18:38.466443+0000","last_active":"2023-10-06T05:18:38.466443+0000","last_peered":"2023-10-06T05:18:38.466443+0000","last_clean":"2023-10-06T05:18:38.466443+0000","last_became_active":"2023-10-06T05:18:38.465603+0000","last_became_peered":"2023-10-06T05:18:38.465603+0000","last_unstale":"2023-10-06T05:18:38.466443+0000","last_undegraded":"2023-10-06T05:18:38.466443+0000","last_fullsized":"2023-10-06T05:18:38.466443+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:08.665979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.454720+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:09.410169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467188+0000","last_change":"2023-10-06T05:18:38.467188+0000","last_active":"2023-10-06T05:18:38.467188+0000","last_peered":"2023-10-06T05:18:38.467188+0000","last_clean":"2023-10-06T05:18:38.467188+0000","last_became_active":"2023-10-06T05:18:38.466475+0000","last_became_peered":"2023-10-06T05:18:38.466475+0000","last_unstale":"2023-10-06T05:18:38.467188+0000","last_undegraded":"2023-10-06T05:18:38.467188+0000","last_fullsized":"2023-10-06T05:18:38.467188+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:22.230340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451137+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:11.273331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.468654+0000","last_change":"2023-10-06T05:18:38.468654+0000","last_active":"2023-10-06T05:18:38.468654+0000","last_peered":"2023-10-06T05:18:38.468654+0000","last_clean":"2023-10-06T05:18:38.468654+0000","last_became_active":"2023-10-06T05:18:38.468394+0000","last_became_peered":"2023-10-06T05:18:38.468394+0000","last_unstale":"2023-10-06T05:18:38.468654+0000","last_undegraded":"2023-10-06T05:18:38.468654+0000","last_fullsized":"2023-10-06T05:18:38.468654+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:56.721340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452992+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:43.862504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460519+0000","last_change":"2023-10-06T05:18:38.460519+0000","last_active":"2023-10-06T05:18:38.460519+0000","last_peered":"2023-10-06T05:18:38.460519+0000","last_clean":"2023-10-06T05:18:38.460519+0000","last_became_active":"2023-10-06T05:18:38.460215+0000","last_became_peered":"2023-10-06T05:18:38.460215+0000","last_unstale":"2023-10-06T05:18:38.460519+0000","last_undegraded":"2023-10-06T05:18:38.460519+0000","last_fullsized":"2023-10-06T05:18:38.460519+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:12.619655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.451676+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:27.404627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.462389+0000","last_change":"2023-10-06T05:18:38.462389+0000","last_active":"2023-10-06T05:18:38.462389+0000","last_peered":"2023-10-06T05:18:38.462389+0000","last_clean":"2023-10-06T05:18:38.462389+0000","last_became_active":"2023-10-06T05:18:38.462121+0000","last_became_peered":"2023-10-06T05:18:38.462121+0000","last_unstale":"2023-10-06T05:18:38.462389+0000","last_undegraded":"2023-10-06T05:18:38.462389+0000","last_fullsized":"2023-10-06T05:18:38.462389+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:23.179454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.453051+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:15.086481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.457211+0000","last_change":"2023-10-06T05:18:38.457211+0000","last_active":"2023-10-06T05:18:38.457211+0000","last_peered":"2023-10-06T05:18:38.457211+0000","last_clean":"2023-10-06T05:18:38.457211+0000","last_became_active":"2023-10-06T05:18:38.456921+0000","last_became_peered":"2023-10-06T05:18:38.456921+0000","last_unstale":"2023-10-06T05:18:38.457211+0000","last_undegraded":"2023-10-06T05:18:38.457211+0000","last_fullsized":"2023-10-06T05:18:38.457211+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:11.790017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.448636+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460312+0000","last_change":"2023-10-06T05:18:38.460312+0000","last_active":"2023-10-06T05:18:38.460312+0000","last_peered":"2023-10-06T05:18:38.460312+0000","last_clean":"2023-10-06T05:18:38.460312+0000","last_became_active":"2023-10-06T05:18:38.460043+0000","last_became_peered":"2023-10-06T05:18:38.460043+0000","last_unstale":"2023-10-06T05:18:38.460312+0000","last_undegraded":"2023-10-06T05:18:38.460312+0000","last_fullsized":"2023-10-06T05:18:38.460312+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:09.080286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.444681+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:18:37.444681+0000","last_peered":"2023-10-06T05:18:37.444681+0000","last_clean":"2023-10-06T05:18:37.444681+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:18:37.444681+0000","last_undegraded":"2023-10-06T05:18:37.444681+0000","last_fullsized":"2023-10-06T05:18:37.444681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452130+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.458978+0000","last_change":"2023-10-06T05:18:38.458978+0000","last_active":"2023-10-06T05:18:38.458978+0000","last_peered":"2023-10-06T05:18:38.458978+0000","last_clean":"2023-10-06T05:18:38.458978+0000","last_became_active":"2023-10-06T05:18:38.458700+0000","last_became_peered":"2023-10-06T05:18:38.458700+0000","last_unstale":"2023-10-06T05:18:38.458978+0000","last_undegraded":"2023-10-06T05:18:38.458978+0000","last_fullsized":"2023-10-06T05:18:38.458978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:23.179454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.436719+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:18:37.436719+0000","last_peered":"2023-10-06T05:18:37.436719+0000","last_clean":"2023-10-06T05:18:37.436719+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:18:37.436719+0000","last_undegraded":"2023-10-06T05:18:37.436719+0000","last_fullsized":"2023-10-06T05:18:37.436719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.455206+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:44.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460411+0000","last_change":"2023-10-06T05:18:38.460411+0000","last_active":"2023-10-06T05:18:38.460411+0000","last_peered":"2023-10-06T05:18:38.460411+0000","last_clean":"2023-10-06T05:18:38.460411+0000","last_became_active":"2023-10-06T05:18:38.460142+0000","last_became_peered":"2023-10-06T05:18:38.460142+0000","last_unstale":"2023-10-06T05:18:38.460411+0000","last_undegraded":"2023-10-06T05:18:38.460411+0000","last_fullsized":"2023-10-06T05:18:38.460411+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:10.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.434247+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:18:37.434247+0000","last_peered":"2023-10-06T05:18:37.434247+0000","last_clean":"2023-10-06T05:18:37.434247+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:18:37.434247+0000","last_undegraded":"2023-10-06T05:18:37.434247+0000","last_fullsized":"2023-10-06T05:18:37.434247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":180,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437257+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:18:37.437257+0000","last_peered":"2023-10-06T05:18:37.437257+0000","last_clean":"2023-10-06T05:18:37.437257+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:18:37.437257+0000","last_undegraded":"2023-10-06T05:18:37.437257+0000","last_fullsized":"2023-10-06T05:18:37.437257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445051+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463399+0000","last_change":"2023-10-06T05:18:38.463399+0000","last_active":"2023-10-06T05:18:38.463399+0000","last_peered":"2023-10-06T05:18:38.463399+0000","last_clean":"2023-10-06T05:18:38.463399+0000","last_became_active":"2023-10-06T05:18:38.463130+0000","last_became_peered":"2023-10-06T05:18:38.463130+0000","last_unstale":"2023-10-06T05:18:38.463399+0000","last_undegraded":"2023-10-06T05:18:38.463399+0000","last_fullsized":"2023-10-06T05:18:38.463399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:35.907402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437352+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:18:37.437352+0000","last_peered":"2023-10-06T05:18:37.437352+0000","last_clean":"2023-10-06T05:18:37.437352+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:18:37.437352+0000","last_undegraded":"2023-10-06T05:18:37.437352+0000","last_fullsized":"2023-10-06T05:18:37.437352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.457853+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:51.516975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.463382+0000","last_change":"2023-10-06T05:18:38.463382+0000","last_active":"2023-10-06T05:18:38.463382+0000","last_peered":"2023-10-06T05:18:38.463382+0000","last_clean":"2023-10-06T05:18:38.463382+0000","last_became_active":"2023-10-06T05:18:38.463112+0000","last_became_peered":"2023-10-06T05:18:38.463112+0000","last_unstale":"2023-10-06T05:18:38.463382+0000","last_undegraded":"2023-10-06T05:18:38.463382+0000","last_fullsized":"2023-10-06T05:18:38.463382+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:37.839836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.437251+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:18:37.437251+0000","last_peered":"2023-10-06T05:18:37.437251+0000","last_clean":"2023-10-06T05:18:37.437251+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:18:37.437251+0000","last_undegraded":"2023-10-06T05:18:37.437251+0000","last_fullsized":"2023-10-06T05:18:37.437251+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.450529+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:42.744594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.466799+0000","last_change":"2023-10-06T05:18:38.466799+0000","last_active":"2023-10-06T05:18:38.466799+0000","last_peered":"2023-10-06T05:18:38.466799+0000","last_clean":"2023-10-06T05:18:38.466799+0000","last_became_active":"2023-10-06T05:18:38.466486+0000","last_became_peered":"2023-10-06T05:18:38.466486+0000","last_unstale":"2023-10-06T05:18:38.466799+0000","last_undegraded":"2023-10-06T05:18:38.466799+0000","last_fullsized":"2023-10-06T05:18:38.466799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:04.252646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.439395+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:18:37.439395+0000","last_peered":"2023-10-06T05:18:37.439395+0000","last_clean":"2023-10-06T05:18:37.439395+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:18:37.439395+0000","last_undegraded":"2023-10-06T05:18:37.439395+0000","last_fullsized":"2023-10-06T05:18:37.439395+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.445017+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.467529+0000","last_change":"2023-10-06T05:18:38.467529+0000","last_active":"2023-10-06T05:18:38.467529+0000","last_peered":"2023-10-06T05:18:38.467529+0000","last_clean":"2023-10-06T05:18:38.467529+0000","last_became_active":"2023-10-06T05:18:38.467262+0000","last_became_peered":"2023-10-06T05:18:38.467262+0000","last_unstale":"2023-10-06T05:18:38.467529+0000","last_undegraded":"2023-10-06T05:18:38.467529+0000","last_fullsized":"2023-10-06T05:18:38.467529+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:34.215548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.440499+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:18:37.440499+0000","last_peered":"2023-10-06T05:18:37.440499+0000","last_clean":"2023-10-06T05:18:37.440499+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:18:37.440499+0000","last_undegraded":"2023-10-06T05:18:37.440499+0000","last_fullsized":"2023-10-06T05:18:37.440499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:18:38.431948+0000","last_change":"2023-10-06T05:18:38.452193+0000","last_active":"2023-10-06T05:18:38.431948+0000","last_peered":"2023-10-06T05:18:38.431948+0000","last_clean":"2023-10-06T05:18:38.431948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:18:38.431948+0000","last_undegraded":"2023-10-06T05:18:38.431948+0000","last_fullsized":"2023-10-06T05:18:38.431948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:39.628432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":8,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-06T05:18:38.460634+0000","last_change":"2023-10-06T05:18:38.460634+0000","last_active":"2023-10-06T05:18:38.460634+0000","last_peered":"2023-10-06T05:18:38.460634+0000","last_clean":"2023-10-06T05:18:38.460634+0000","last_became_active":"2023-10-06T05:18:38.459850+0000","last_became_peered":"2023-10-06T05:18:38.459850+0000","last_unstale":"2023-10-06T05:18:38.460634+0000","last_undegraded":"2023-10-06T05:18:38.460634+0000","last_fullsized":"2023-10-06T05:18:38.460634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:10.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T05:18:37.440631+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:18:37.440631+0000","last_peered":"2023-10-06T05:18:37.440631+0000","last_clean":"2023-10-06T05:18:37.440631+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:18:37.440631+0000","last_undegraded":"2023-10-06T05:18:37.440631+0000","last_fullsized":"2023-10-06T05:18:37.440631+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298348,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073492,"statfs":{"total":96636764160,"available":96331255808,"internally_reserved":0,"allocated":1085440,"data_stored":796913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78300000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.871},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.62},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77400000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298336,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073504,"statfs":{"total":96636764160,"available":96331268096,"internally_reserved":0,"allocated":1073152,"data_stored":790297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.433}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.95999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.94099999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.006}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.82399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297756,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074084,"statfs":{"total":96636764160,"available":96331862016,"internally_reserved":0,"allocated":479232,"data_stored":199929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.51300000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.749},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.81899999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.91700000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.84599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.004}]},{"osd":7,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.0169999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.371}]},{"osd":1,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.44},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.754},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.78100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":7.742,"5min":7.742,"15min":7.742},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.88500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.87},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.85799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297768,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074072,"statfs":{"total":96636764160,"available":96331849728,"internally_reserved":0,"allocated":491520,"data_stored":206545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2780000000000005,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":9.25,"5min":9.25,"15min":9.25},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.108000000000001,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.95799999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":10.384,"5min":10.384,"15min":10.384},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":9.1699999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.205,"5min":10.205,"15min":10.205},"last":1.038},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.83299999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":1.113}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.0920000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:48.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:18:49.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.065+0000 7f2939bff700 1 Processor -- start 2023-10-06T05:18:49.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.065+0000 7f2939bff700 1 -- start start 2023-10-06T05:18:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.066+0000 7f2939bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2934068d80 0x7f2934159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.066+0000 7f2939bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29340cc0d0 0x7f29340c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.066+0000 7f2939bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29340c95b0 con 0x7f29341595a0 2023-10-06T05:18:49.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.066+0000 7f2939bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29340c9890 con 0x7f2934068d80 2023-10-06T05:18:49.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.066+0000 7f2939bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29340c9b70 con 0x7f29340cc0d0 2023-10-06T05:18:49.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f2933fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29340cc0d0 0x7f29340c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f2932ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2934068d80 0x7f2934159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f2933fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29340cc0d0 0x7f29340c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56664/0 (socket says 172.21.15.31:56664) 2023-10-06T05:18:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f2933fff700 1 -- 172.21.15.31:0/104031614 learned_addr learned my addr 172.21.15.31:0/104031614 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f29337fe700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f29341595a0 0x7f2934068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:35952/0 (socket says 172.21.15.31:35952) 2023-10-06T05:18:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f29327fc700 1 -- 172.21.15.31:0/104031614 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568443488 0 0) 0x7f29340c95b0 con 0x7f29341595a0 2023-10-06T05:18:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f29327fc700 1 -- 172.21.15.31:0/104031614 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f291c003570 con 0x7f29341595a0 2023-10-06T05:18:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f2932ffd700 1 -- 172.21.15.31:0/104031614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29340cc0d0 msgr2=0x7f29340c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f2932ffd700 1 --2- 172.21.15.31:0/104031614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29340cc0d0 0x7f29340c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f2932ffd700 1 -- 172.21.15.31:0/104031614 >> v1:172.21.15.62:6789/0 conn(0x7f29341595a0 legacy=0x7f2934068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f2932ffd700 1 -- 172.21.15.31:0/104031614 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29340ca130 con 0x7f2934068d80 2023-10-06T05:18:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.067+0000 7f2933fff700 1 --2- 172.21.15.31:0/104031614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29340cc0d0 0x7f29340c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.068+0000 7f2932ffd700 1 --2- 172.21.15.31:0/104031614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2934068d80 0x7f2934159970 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2928009910 tx=0x7f29280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b7204a267552e56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.068+0000 7f29327fc700 1 -- 172.21.15.31:0/104031614 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928011630 con 0x7f2934068d80 2023-10-06T05:18:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.068+0000 7f29327fc700 1 -- 172.21.15.31:0/104031614 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2928011790 con 0x7f2934068d80 2023-10-06T05:18:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.068+0000 7f29327fc700 1 -- 172.21.15.31:0/104031614 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928011b30 con 0x7f2934068d80 2023-10-06T05:18:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.068+0000 7f2939bff700 1 -- 172.21.15.31:0/104031614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2934068d80 msgr2=0x7f2934159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.068+0000 7f2939bff700 1 --2- 172.21.15.31:0/104031614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2934068d80 0x7f2934159970 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2928009910 tx=0x7f29280091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.068+0000 7f2939bff700 1 -- 172.21.15.31:0/104031614 shutdown_connections 2023-10-06T05:18:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.068+0000 7f2939bff700 1 --2- 172.21.15.31:0/104031614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29340cc0d0 0x7f29340c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.068+0000 7f2939bff700 1 --2- 172.21.15.31:0/104031614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2934068d80 0x7f2934159970 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.068+0000 7f2939bff700 1 -- 172.21.15.31:0/104031614 >> 172.21.15.31:0/104031614 conn(0x7f293406b620 msgr2=0x7f293406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.069+0000 7f2939bff700 1 -- 172.21.15.31:0/104031614 shutdown_connections 2023-10-06T05:18:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.069+0000 7f2939bff700 1 -- 172.21.15.31:0/104031614 wait complete. 2023-10-06T05:18:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.070+0000 7f2939bff700 1 Processor -- start 2023-10-06T05:18:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.070+0000 7f2939bff700 1 -- start start 2023-10-06T05:18:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.071+0000 7f2939bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2934068d80 0x7f29340ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.071+0000 7f2939bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29340cc0d0 0x7f29340b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.071+0000 7f2939bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29341595a0 0x7f293414a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.071+0000 7f2939bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29340c59f0 con 0x7f29340cc0d0 2023-10-06T05:18:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.071+0000 7f2939bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29340c8e70 con 0x7f29341595a0 2023-10-06T05:18:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.071+0000 7f2939bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29340c9150 con 0x7f2934068d80 2023-10-06T05:18:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.071+0000 7f2932ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29340cc0d0 0x7f29340b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.071+0000 7f2932ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29340cc0d0 0x7f29340b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33938/0 (socket says 172.21.15.31:33938) 2023-10-06T05:18:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.071+0000 7f2932ffd700 1 -- 172.21.15.31:0/1998175473 learned_addr learned my addr 172.21.15.31:0/1998175473 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f29337fe700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2934068d80 0x7f29340ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f2933fff700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29341595a0 0x7f293414a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f2932ffd700 1 -- 172.21.15.31:0/1998175473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2934068d80 msgr2=0x7f29340ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f2932ffd700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2934068d80 0x7f29340ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f2932ffd700 1 -- 172.21.15.31:0/1998175473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29341595a0 msgr2=0x7f293414a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f2932ffd700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29341595a0 0x7f293414a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f2932ffd700 1 -- 172.21.15.31:0/1998175473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c009ae0 con 0x7f29340cc0d0 2023-10-06T05:18:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f29337fe700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2934068d80 0x7f29340ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f2932ffd700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29340cc0d0 0x7f29340b1aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2928013000 tx=0x7f2928017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f291b7fe700 1 -- 172.21.15.31:0/1998175473 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928011900 con 0x7f29340cc0d0 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f291b7fe700 1 -- 172.21.15.31:0/1998175473 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2928011a80 con 0x7f29340cc0d0 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f2939bff700 1 -- 172.21.15.31:0/1998175473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2928009420 con 0x7f29340cc0d0 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.072+0000 7f291b7fe700 1 -- 172.21.15.31:0/1998175473 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928025670 con 0x7f29340cc0d0 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.073+0000 7f2939bff700 1 -- 172.21.15.31:0/1998175473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29340c5680 con 0x7f29340cc0d0 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.074+0000 7f2939bff700 1 -- 172.21.15.31:0/1998175473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293404c380 con 0x7f29340cc0d0 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.075+0000 7f291b7fe700 1 -- 172.21.15.31:0/1998175473 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2928023070 con 0x7f29340cc0d0 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.075+0000 7f291b7fe700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f29200670e0 0x7f2920069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.076+0000 7f291b7fe700 1 -- 172.21.15.31:0/1998175473 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2928085ee0 con 0x7f29340cc0d0 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.076+0000 7f29337fe700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f29200670e0 0x7f2920069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.076+0000 7f29337fe700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f29200670e0 0x7f2920069590 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2924005f00 tx=0x7f2924007c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:49.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.080+0000 7f291b7fe700 1 -- 172.21.15.31:0/1998175473 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2928052cd0 con 0x7f29340cc0d0 2023-10-06T05:18:49.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.282+0000 7f2939bff700 1 -- 172.21.15.31:0/1998175473 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f293406d9b0 con 0x7f29200670e0 2023-10-06T05:18:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.292+0000 7f291b7fe700 1 -- 172.21.15.31:0/1998175473 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311185 (secure 0 0 0) 0x7f293406d9b0 con 0x7f29200670e0 2023-10-06T05:18:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 -- 172.21.15.31:0/1998175473 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f29200670e0 msgr2=0x7f2920069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f29200670e0 0x7f2920069590 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2924005f00 tx=0x7f2924007c60 comp rx=0 tx=0).stop 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 -- 172.21.15.31:0/1998175473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29340cc0d0 msgr2=0x7f29340b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29340cc0d0 0x7f29340b1aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2928013000 tx=0x7f2928017000 comp rx=0 tx=0).stop 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 -- 172.21.15.31:0/1998175473 shutdown_connections 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29341595a0 0x7f293414a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2934068d80 0x7f29340ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f29200670e0 0x7f2920069590 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 --2- 172.21.15.31:0/1998175473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29340cc0d0 0x7f29340b1aa0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 -- 172.21.15.31:0/1998175473 >> 172.21.15.31:0/1998175473 conn(0x7f293406b620 msgr2=0x7f2934156260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 -- 172.21.15.31:0/1998175473 shutdown_connections 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.297+0000 7f2939bff700 1 -- 172.21.15.31:0/1998175473 wait complete. 2023-10-06T05:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:18:49.316 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-10-06T05:18:49.142036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2383560,"kb_used_data":5256,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591160,"statfs":{"total":773094113280,"available":770653347840,"internally_reserved":0,"allocated":5382144,"data_stored":2947764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.060789"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.468172+0000","last_change":"2023-10-06T05:18:39.462257+0000","last_active":"2023-10-06T05:18:39.468172+0000","last_peered":"2023-10-06T05:18:39.468172+0000","last_clean":"2023-10-06T05:18:39.468172+0000","last_became_active":"2023-10-06T05:18:39.461985+0000","last_became_peered":"2023-10-06T05:18:39.461985+0000","last_unstale":"2023-10-06T05:18:39.468172+0000","last_undegraded":"2023-10-06T05:18:39.468172+0000","last_fullsized":"2023-10-06T05:18:39.468172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:28.365507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.474486+0000","last_change":"2023-10-06T05:18:39.468872+0000","last_active":"2023-10-06T05:18:39.474486+0000","last_peered":"2023-10-06T05:18:39.474486+0000","last_clean":"2023-10-06T05:18:39.474486+0000","last_became_active":"2023-10-06T05:18:39.468492+0000","last_became_peered":"2023-10-06T05:18:39.468492+0000","last_unstale":"2023-10-06T05:18:39.474486+0000","last_undegraded":"2023-10-06T05:18:39.474486+0000","last_fullsized":"2023-10-06T05:18:39.474486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482204+0000","last_change":"2023-10-06T05:18:39.469486+0000","last_active":"2023-10-06T05:18:39.482204+0000","last_peered":"2023-10-06T05:18:39.482204+0000","last_clean":"2023-10-06T05:18:39.482204+0000","last_became_active":"2023-10-06T05:18:39.469030+0000","last_became_peered":"2023-10-06T05:18:39.469030+0000","last_unstale":"2023-10-06T05:18:39.482204+0000","last_undegraded":"2023-10-06T05:18:39.482204+0000","last_fullsized":"2023-10-06T05:18:39.482204+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:53.514320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.484334+0000","last_change":"2023-10-06T05:18:39.480637+0000","last_active":"2023-10-06T05:18:39.484334+0000","last_peered":"2023-10-06T05:18:39.484334+0000","last_clean":"2023-10-06T05:18:39.484334+0000","last_became_active":"2023-10-06T05:18:39.480368+0000","last_became_peered":"2023-10-06T05:18:39.480368+0000","last_unstale":"2023-10-06T05:18:39.484334+0000","last_undegraded":"2023-10-06T05:18:39.484334+0000","last_fullsized":"2023-10-06T05:18:39.484334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.474135+0000","last_change":"2023-10-06T05:18:39.462195+0000","last_active":"2023-10-06T05:18:39.474135+0000","last_peered":"2023-10-06T05:18:39.474135+0000","last_clean":"2023-10-06T05:18:39.474135+0000","last_became_active":"2023-10-06T05:18:39.461821+0000","last_became_peered":"2023-10-06T05:18:39.461821+0000","last_unstale":"2023-10-06T05:18:39.474135+0000","last_undegraded":"2023-10-06T05:18:39.474135+0000","last_fullsized":"2023-10-06T05:18:39.474135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:24.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.465902+0000","last_change":"2023-10-06T05:18:39.459673+0000","last_active":"2023-10-06T05:18:39.465902+0000","last_peered":"2023-10-06T05:18:39.465902+0000","last_clean":"2023-10-06T05:18:39.465902+0000","last_became_active":"2023-10-06T05:18:39.459356+0000","last_became_peered":"2023-10-06T05:18:39.459356+0000","last_unstale":"2023-10-06T05:18:39.465902+0000","last_undegraded":"2023-10-06T05:18:39.465902+0000","last_fullsized":"2023-10-06T05:18:39.465902+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:57.252713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482141+0000","last_change":"2023-10-06T05:18:39.469185+0000","last_active":"2023-10-06T05:18:39.482141+0000","last_peered":"2023-10-06T05:18:39.482141+0000","last_clean":"2023-10-06T05:18:39.482141+0000","last_became_active":"2023-10-06T05:18:39.468400+0000","last_became_peered":"2023-10-06T05:18:39.468400+0000","last_unstale":"2023-10-06T05:18:39.482141+0000","last_undegraded":"2023-10-06T05:18:39.482141+0000","last_fullsized":"2023-10-06T05:18:39.482141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:10.451499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.478426+0000","last_change":"2023-10-06T05:18:39.469087+0000","last_active":"2023-10-06T05:18:39.478426+0000","last_peered":"2023-10-06T05:18:39.478426+0000","last_clean":"2023-10-06T05:18:39.478426+0000","last_became_active":"2023-10-06T05:18:39.468729+0000","last_became_peered":"2023-10-06T05:18:39.468729+0000","last_unstale":"2023-10-06T05:18:39.478426+0000","last_undegraded":"2023-10-06T05:18:39.478426+0000","last_fullsized":"2023-10-06T05:18:39.478426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:40.412326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.486170+0000","last_change":"2023-10-06T05:18:39.484695+0000","last_active":"2023-10-06T05:18:39.486170+0000","last_peered":"2023-10-06T05:18:39.486170+0000","last_clean":"2023-10-06T05:18:39.486170+0000","last_became_active":"2023-10-06T05:18:39.484263+0000","last_became_peered":"2023-10-06T05:18:39.484263+0000","last_unstale":"2023-10-06T05:18:39.486170+0000","last_undegraded":"2023-10-06T05:18:39.486170+0000","last_fullsized":"2023-10-06T05:18:39.486170+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:20.672937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.458315+0000","last_change":"2023-10-06T05:18:39.455045+0000","last_active":"2023-10-06T05:18:39.458315+0000","last_peered":"2023-10-06T05:18:39.458315+0000","last_clean":"2023-10-06T05:18:39.458315+0000","last_became_active":"2023-10-06T05:18:39.454723+0000","last_became_peered":"2023-10-06T05:18:39.454723+0000","last_unstale":"2023-10-06T05:18:39.458315+0000","last_undegraded":"2023-10-06T05:18:39.458315+0000","last_fullsized":"2023-10-06T05:18:39.458315+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:02.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.477303+0000","last_change":"2023-10-06T05:18:39.475304+0000","last_active":"2023-10-06T05:18:39.477303+0000","last_peered":"2023-10-06T05:18:39.477303+0000","last_clean":"2023-10-06T05:18:39.477303+0000","last_became_active":"2023-10-06T05:18:39.475031+0000","last_became_peered":"2023-10-06T05:18:39.475031+0000","last_unstale":"2023-10-06T05:18:39.477303+0000","last_undegraded":"2023-10-06T05:18:39.477303+0000","last_fullsized":"2023-10-06T05:18:39.477303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:23.236759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.478255+0000","last_change":"2023-10-06T05:18:39.463976+0000","last_active":"2023-10-06T05:18:39.478255+0000","last_peered":"2023-10-06T05:18:39.478255+0000","last_clean":"2023-10-06T05:18:39.478255+0000","last_became_active":"2023-10-06T05:18:39.463682+0000","last_became_peered":"2023-10-06T05:18:39.463682+0000","last_unstale":"2023-10-06T05:18:39.478255+0000","last_undegraded":"2023-10-06T05:18:39.478255+0000","last_fullsized":"2023-10-06T05:18:39.478255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:53.514320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.461688+0000","last_change":"2023-10-06T05:18:39.456616+0000","last_active":"2023-10-06T05:18:39.461688+0000","last_peered":"2023-10-06T05:18:39.461688+0000","last_clean":"2023-10-06T05:18:39.461688+0000","last_became_active":"2023-10-06T05:18:39.456284+0000","last_became_peered":"2023-10-06T05:18:39.456284+0000","last_unstale":"2023-10-06T05:18:39.461688+0000","last_undegraded":"2023-10-06T05:18:39.461688+0000","last_fullsized":"2023-10-06T05:18:39.461688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:54.000999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482257+0000","last_change":"2023-10-06T05:18:39.470446+0000","last_active":"2023-10-06T05:18:39.482257+0000","last_peered":"2023-10-06T05:18:39.482257+0000","last_clean":"2023-10-06T05:18:39.482257+0000","last_became_active":"2023-10-06T05:18:39.469939+0000","last_became_peered":"2023-10-06T05:18:39.469939+0000","last_unstale":"2023-10-06T05:18:39.482257+0000","last_undegraded":"2023-10-06T05:18:39.482257+0000","last_fullsized":"2023-10-06T05:18:39.482257+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:16.099790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.475795+0000","last_change":"2023-10-06T05:18:39.472581+0000","last_active":"2023-10-06T05:18:39.475795+0000","last_peered":"2023-10-06T05:18:39.475795+0000","last_clean":"2023-10-06T05:18:39.475795+0000","last_became_active":"2023-10-06T05:18:39.472318+0000","last_became_peered":"2023-10-06T05:18:39.472318+0000","last_unstale":"2023-10-06T05:18:39.475795+0000","last_undegraded":"2023-10-06T05:18:39.475795+0000","last_fullsized":"2023-10-06T05:18:39.475795+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:27.931114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.458483+0000","last_change":"2023-10-06T05:18:39.456197+0000","last_active":"2023-10-06T05:18:39.458483+0000","last_peered":"2023-10-06T05:18:39.458483+0000","last_clean":"2023-10-06T05:18:39.458483+0000","last_became_active":"2023-10-06T05:18:39.455894+0000","last_became_peered":"2023-10-06T05:18:39.455894+0000","last_unstale":"2023-10-06T05:18:39.458483+0000","last_undegraded":"2023-10-06T05:18:39.458483+0000","last_fullsized":"2023-10-06T05:18:39.458483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:35.221967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.487941+0000","last_change":"2023-10-06T05:18:39.482892+0000","last_active":"2023-10-06T05:18:39.487941+0000","last_peered":"2023-10-06T05:18:39.487941+0000","last_clean":"2023-10-06T05:18:39.487941+0000","last_became_active":"2023-10-06T05:18:39.482544+0000","last_became_peered":"2023-10-06T05:18:39.482544+0000","last_unstale":"2023-10-06T05:18:39.487941+0000","last_undegraded":"2023-10-06T05:18:39.487941+0000","last_fullsized":"2023-10-06T05:18:39.487941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:51.293915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.475329+0000","last_change":"2023-10-06T05:18:39.472250+0000","last_active":"2023-10-06T05:18:39.475329+0000","last_peered":"2023-10-06T05:18:39.475329+0000","last_clean":"2023-10-06T05:18:39.475329+0000","last_became_active":"2023-10-06T05:18:39.471985+0000","last_became_peered":"2023-10-06T05:18:39.471985+0000","last_unstale":"2023-10-06T05:18:39.475329+0000","last_undegraded":"2023-10-06T05:18:39.475329+0000","last_fullsized":"2023-10-06T05:18:39.475329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482152+0000","last_change":"2023-10-06T05:18:39.468513+0000","last_active":"2023-10-06T05:18:39.482152+0000","last_peered":"2023-10-06T05:18:39.482152+0000","last_clean":"2023-10-06T05:18:39.482152+0000","last_became_active":"2023-10-06T05:18:39.468274+0000","last_became_peered":"2023-10-06T05:18:39.468274+0000","last_unstale":"2023-10-06T05:18:39.482152+0000","last_undegraded":"2023-10-06T05:18:39.482152+0000","last_fullsized":"2023-10-06T05:18:39.482152+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:13:41.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.467793+0000","last_change":"2023-10-06T05:18:39.459955+0000","last_active":"2023-10-06T05:18:39.467793+0000","last_peered":"2023-10-06T05:18:39.467793+0000","last_clean":"2023-10-06T05:18:39.467793+0000","last_became_active":"2023-10-06T05:18:39.459500+0000","last_became_peered":"2023-10-06T05:18:39.459500+0000","last_unstale":"2023-10-06T05:18:39.467793+0000","last_undegraded":"2023-10-06T05:18:39.467793+0000","last_fullsized":"2023-10-06T05:18:39.467793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:02.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.472810+0000","last_change":"2023-10-06T05:18:39.465971+0000","last_active":"2023-10-06T05:18:39.472810+0000","last_peered":"2023-10-06T05:18:39.472810+0000","last_clean":"2023-10-06T05:18:39.472810+0000","last_became_active":"2023-10-06T05:18:39.465698+0000","last_became_peered":"2023-10-06T05:18:39.465698+0000","last_unstale":"2023-10-06T05:18:39.472810+0000","last_undegraded":"2023-10-06T05:18:39.472810+0000","last_fullsized":"2023-10-06T05:18:39.472810+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:15.086481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482058+0000","last_change":"2023-10-06T05:18:39.475652+0000","last_active":"2023-10-06T05:18:39.482058+0000","last_peered":"2023-10-06T05:18:39.482058+0000","last_clean":"2023-10-06T05:18:39.482058+0000","last_became_active":"2023-10-06T05:18:39.475362+0000","last_became_peered":"2023-10-06T05:18:39.475362+0000","last_unstale":"2023-10-06T05:18:39.482058+0000","last_undegraded":"2023-10-06T05:18:39.482058+0000","last_fullsized":"2023-10-06T05:18:39.482058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:23.882938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.476321+0000","last_change":"2023-10-06T05:18:39.471209+0000","last_active":"2023-10-06T05:18:39.476321+0000","last_peered":"2023-10-06T05:18:39.476321+0000","last_clean":"2023-10-06T05:18:39.476321+0000","last_became_active":"2023-10-06T05:18:39.470883+0000","last_became_peered":"2023-10-06T05:18:39.470883+0000","last_unstale":"2023-10-06T05:18:39.476321+0000","last_undegraded":"2023-10-06T05:18:39.476321+0000","last_fullsized":"2023-10-06T05:18:39.476321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:09.672397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482155+0000","last_change":"2023-10-06T05:18:39.467789+0000","last_active":"2023-10-06T05:18:39.482155+0000","last_peered":"2023-10-06T05:18:39.482155+0000","last_clean":"2023-10-06T05:18:39.482155+0000","last_became_active":"2023-10-06T05:18:39.467514+0000","last_became_peered":"2023-10-06T05:18:39.467514+0000","last_unstale":"2023-10-06T05:18:39.482155+0000","last_undegraded":"2023-10-06T05:18:39.482155+0000","last_fullsized":"2023-10-06T05:18:39.482155+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:24.843295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.487986+0000","last_change":"2023-10-06T05:18:39.483483+0000","last_active":"2023-10-06T05:18:39.487986+0000","last_peered":"2023-10-06T05:18:39.487986+0000","last_clean":"2023-10-06T05:18:39.487986+0000","last_became_active":"2023-10-06T05:18:39.483163+0000","last_became_peered":"2023-10-06T05:18:39.483163+0000","last_unstale":"2023-10-06T05:18:39.487986+0000","last_undegraded":"2023-10-06T05:18:39.487986+0000","last_fullsized":"2023-10-06T05:18:39.487986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:40.412326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.478264+0000","last_change":"2023-10-06T05:18:39.467968+0000","last_active":"2023-10-06T05:18:39.478264+0000","last_peered":"2023-10-06T05:18:39.478264+0000","last_clean":"2023-10-06T05:18:39.478264+0000","last_became_active":"2023-10-06T05:18:39.467485+0000","last_became_peered":"2023-10-06T05:18:39.467485+0000","last_unstale":"2023-10-06T05:18:39.478264+0000","last_undegraded":"2023-10-06T05:18:39.478264+0000","last_fullsized":"2023-10-06T05:18:39.478264+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:39.628432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.487283+0000","last_change":"2023-10-06T05:18:39.486161+0000","last_active":"2023-10-06T05:18:39.487283+0000","last_peered":"2023-10-06T05:18:39.487283+0000","last_clean":"2023-10-06T05:18:39.487283+0000","last_became_active":"2023-10-06T05:18:39.485830+0000","last_became_peered":"2023-10-06T05:18:39.485830+0000","last_unstale":"2023-10-06T05:18:39.487283+0000","last_undegraded":"2023-10-06T05:18:39.487283+0000","last_fullsized":"2023-10-06T05:18:39.487283+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:42.744594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.475535+0000","last_change":"2023-10-06T05:18:39.467703+0000","last_active":"2023-10-06T05:18:39.475535+0000","last_peered":"2023-10-06T05:18:39.475535+0000","last_clean":"2023-10-06T05:18:39.475535+0000","last_became_active":"2023-10-06T05:18:39.467422+0000","last_became_peered":"2023-10-06T05:18:39.467422+0000","last_unstale":"2023-10-06T05:18:39.475535+0000","last_undegraded":"2023-10-06T05:18:39.475535+0000","last_fullsized":"2023-10-06T05:18:39.475535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:50.655754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482549+0000","last_change":"2023-10-06T05:18:39.473682+0000","last_active":"2023-10-06T05:18:39.482549+0000","last_peered":"2023-10-06T05:18:39.482549+0000","last_clean":"2023-10-06T05:18:39.482549+0000","last_became_active":"2023-10-06T05:18:39.473405+0000","last_became_peered":"2023-10-06T05:18:39.473405+0000","last_unstale":"2023-10-06T05:18:39.482549+0000","last_undegraded":"2023-10-06T05:18:39.482549+0000","last_fullsized":"2023-10-06T05:18:39.482549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:20.672937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.468769+0000","last_change":"2023-10-06T05:18:39.465661+0000","last_active":"2023-10-06T05:18:39.468769+0000","last_peered":"2023-10-06T05:18:39.468769+0000","last_clean":"2023-10-06T05:18:39.468769+0000","last_became_active":"2023-10-06T05:18:39.465383+0000","last_became_peered":"2023-10-06T05:18:39.465383+0000","last_unstale":"2023-10-06T05:18:39.468769+0000","last_undegraded":"2023-10-06T05:18:39.468769+0000","last_fullsized":"2023-10-06T05:18:39.468769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:09.084226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482169+0000","last_change":"2023-10-06T05:18:39.469394+0000","last_active":"2023-10-06T05:18:39.482169+0000","last_peered":"2023-10-06T05:18:39.482169+0000","last_clean":"2023-10-06T05:18:39.482169+0000","last_became_active":"2023-10-06T05:18:39.468774+0000","last_became_peered":"2023-10-06T05:18:39.468774+0000","last_unstale":"2023-10-06T05:18:39.482169+0000","last_undegraded":"2023-10-06T05:18:39.482169+0000","last_fullsized":"2023-10-06T05:18:39.482169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:44.049494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.479638+0000","last_change":"2023-10-06T05:18:39.478401+0000","last_active":"2023-10-06T05:18:39.479638+0000","last_peered":"2023-10-06T05:18:39.479638+0000","last_clean":"2023-10-06T05:18:39.479638+0000","last_became_active":"2023-10-06T05:18:39.478111+0000","last_became_peered":"2023-10-06T05:18:39.478111+0000","last_unstale":"2023-10-06T05:18:39.479638+0000","last_undegraded":"2023-10-06T05:18:39.479638+0000","last_fullsized":"2023-10-06T05:18:39.479638+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:42.146677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.468318+0000","last_change":"2023-10-06T05:18:39.459560+0000","last_active":"2023-10-06T05:18:39.468318+0000","last_peered":"2023-10-06T05:18:39.468318+0000","last_clean":"2023-10-06T05:18:39.468318+0000","last_became_active":"2023-10-06T05:18:39.459264+0000","last_became_peered":"2023-10-06T05:18:39.459264+0000","last_unstale":"2023-10-06T05:18:39.468318+0000","last_undegraded":"2023-10-06T05:18:39.468318+0000","last_fullsized":"2023-10-06T05:18:39.468318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:08.541104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"43'4","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.476869+0000","last_change":"2023-10-06T05:18:38.464327+0000","last_active":"2023-10-06T05:18:39.476869+0000","last_peered":"2023-10-06T05:18:39.476869+0000","last_clean":"2023-10-06T05:18:39.476869+0000","last_became_active":"2023-10-06T05:18:38.464063+0000","last_became_peered":"2023-10-06T05:18:38.464063+0000","last_unstale":"2023-10-06T05:18:39.476869+0000","last_undegraded":"2023-10-06T05:18:39.476869+0000","last_fullsized":"2023-10-06T05:18:39.476869+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:47.847969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.474491+0000","last_change":"2023-10-06T05:18:39.470956+0000","last_active":"2023-10-06T05:18:39.474491+0000","last_peered":"2023-10-06T05:18:39.474491+0000","last_clean":"2023-10-06T05:18:39.474491+0000","last_became_active":"2023-10-06T05:18:39.470602+0000","last_became_peered":"2023-10-06T05:18:39.470602+0000","last_unstale":"2023-10-06T05:18:39.474491+0000","last_undegraded":"2023-10-06T05:18:39.474491+0000","last_fullsized":"2023-10-06T05:18:39.474491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:36.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.479672+0000","last_change":"2023-10-06T05:18:38.467137+0000","last_active":"2023-10-06T05:18:39.479672+0000","last_peered":"2023-10-06T05:18:39.479672+0000","last_clean":"2023-10-06T05:18:39.479672+0000","last_became_active":"2023-10-06T05:18:38.466871+0000","last_became_peered":"2023-10-06T05:18:38.466871+0000","last_unstale":"2023-10-06T05:18:39.479672+0000","last_undegraded":"2023-10-06T05:18:39.479672+0000","last_fullsized":"2023-10-06T05:18:39.479672+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:26.910024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.467512+0000","last_change":"2023-10-06T05:18:39.460087+0000","last_active":"2023-10-06T05:18:39.467512+0000","last_peered":"2023-10-06T05:18:39.467512+0000","last_clean":"2023-10-06T05:18:39.467512+0000","last_became_active":"2023-10-06T05:18:39.459783+0000","last_became_peered":"2023-10-06T05:18:39.459783+0000","last_unstale":"2023-10-06T05:18:39.467512+0000","last_undegraded":"2023-10-06T05:18:39.467512+0000","last_fullsized":"2023-10-06T05:18:39.467512+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:09.410169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.473557+0000","last_change":"2023-10-06T05:18:38.465625+0000","last_active":"2023-10-06T05:18:39.473557+0000","last_peered":"2023-10-06T05:18:39.473557+0000","last_clean":"2023-10-06T05:18:39.473557+0000","last_became_active":"2023-10-06T05:18:38.465367+0000","last_became_peered":"2023-10-06T05:18:38.465367+0000","last_unstale":"2023-10-06T05:18:39.473557+0000","last_undegraded":"2023-10-06T05:18:39.473557+0000","last_fullsized":"2023-10-06T05:18:39.473557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:55.377855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.478166+0000","last_change":"2023-10-06T05:18:39.467390+0000","last_active":"2023-10-06T05:18:39.478166+0000","last_peered":"2023-10-06T05:18:39.478166+0000","last_clean":"2023-10-06T05:18:39.478166+0000","last_became_active":"2023-10-06T05:18:39.467112+0000","last_became_peered":"2023-10-06T05:18:39.467112+0000","last_unstale":"2023-10-06T05:18:39.478166+0000","last_undegraded":"2023-10-06T05:18:39.478166+0000","last_fullsized":"2023-10-06T05:18:39.478166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:06.138873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.484920+0000","last_change":"2023-10-06T05:18:38.466328+0000","last_active":"2023-10-06T05:18:39.484920+0000","last_peered":"2023-10-06T05:18:39.484920+0000","last_clean":"2023-10-06T05:18:39.484920+0000","last_became_active":"2023-10-06T05:18:38.466033+0000","last_became_peered":"2023-10-06T05:18:38.466033+0000","last_unstale":"2023-10-06T05:18:39.484920+0000","last_undegraded":"2023-10-06T05:18:39.484920+0000","last_fullsized":"2023-10-06T05:18:39.484920+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:23.634297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.481853+0000","last_change":"2023-10-06T05:18:39.481059+0000","last_active":"2023-10-06T05:18:39.481853+0000","last_peered":"2023-10-06T05:18:39.481853+0000","last_clean":"2023-10-06T05:18:39.481853+0000","last_became_active":"2023-10-06T05:18:39.480802+0000","last_became_peered":"2023-10-06T05:18:39.480802+0000","last_unstale":"2023-10-06T05:18:39.481853+0000","last_undegraded":"2023-10-06T05:18:39.481853+0000","last_fullsized":"2023-10-06T05:18:39.481853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482399+0000","last_change":"2023-10-06T05:18:38.464675+0000","last_active":"2023-10-06T05:18:39.482399+0000","last_peered":"2023-10-06T05:18:39.482399+0000","last_clean":"2023-10-06T05:18:39.482399+0000","last_became_active":"2023-10-06T05:18:38.464409+0000","last_became_peered":"2023-10-06T05:18:38.464409+0000","last_unstale":"2023-10-06T05:18:39.482399+0000","last_undegraded":"2023-10-06T05:18:39.482399+0000","last_fullsized":"2023-10-06T05:18:39.482399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:09.080286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.474550+0000","last_change":"2023-10-06T05:18:39.470480+0000","last_active":"2023-10-06T05:18:39.474550+0000","last_peered":"2023-10-06T05:18:39.474550+0000","last_clean":"2023-10-06T05:18:39.474550+0000","last_became_active":"2023-10-06T05:18:39.469630+0000","last_became_peered":"2023-10-06T05:18:39.469630+0000","last_unstale":"2023-10-06T05:18:39.474550+0000","last_undegraded":"2023-10-06T05:18:39.474550+0000","last_fullsized":"2023-10-06T05:18:39.474550+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:32.864143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.474136+0000","last_change":"2023-10-06T05:18:38.465029+0000","last_active":"2023-10-06T05:18:39.474136+0000","last_peered":"2023-10-06T05:18:39.474136+0000","last_clean":"2023-10-06T05:18:39.474136+0000","last_became_active":"2023-10-06T05:18:38.464703+0000","last_became_peered":"2023-10-06T05:18:38.464703+0000","last_unstale":"2023-10-06T05:18:39.474136+0000","last_undegraded":"2023-10-06T05:18:39.474136+0000","last_fullsized":"2023-10-06T05:18:39.474136+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.478240+0000","last_change":"2023-10-06T05:18:39.467752+0000","last_active":"2023-10-06T05:18:39.478240+0000","last_peered":"2023-10-06T05:18:39.478240+0000","last_clean":"2023-10-06T05:18:39.478240+0000","last_became_active":"2023-10-06T05:18:39.467102+0000","last_became_peered":"2023-10-06T05:18:39.467102+0000","last_unstale":"2023-10-06T05:18:39.478240+0000","last_undegraded":"2023-10-06T05:18:39.478240+0000","last_fullsized":"2023-10-06T05:18:39.478240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:19.930105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.451146+0000","last_change":"2023-10-06T05:18:38.460564+0000","last_active":"2023-10-06T05:18:39.451146+0000","last_peered":"2023-10-06T05:18:39.451146+0000","last_clean":"2023-10-06T05:18:39.451146+0000","last_became_active":"2023-10-06T05:18:38.460269+0000","last_became_peered":"2023-10-06T05:18:38.460269+0000","last_unstale":"2023-10-06T05:18:39.451146+0000","last_undegraded":"2023-10-06T05:18:39.451146+0000","last_fullsized":"2023-10-06T05:18:39.451146+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:33.245915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.486600+0000","last_change":"2023-10-06T05:18:39.483645+0000","last_active":"2023-10-06T05:18:39.486600+0000","last_peered":"2023-10-06T05:18:39.486600+0000","last_clean":"2023-10-06T05:18:39.486600+0000","last_became_active":"2023-10-06T05:18:39.483276+0000","last_became_peered":"2023-10-06T05:18:39.483276+0000","last_unstale":"2023-10-06T05:18:39.486600+0000","last_undegraded":"2023-10-06T05:18:39.486600+0000","last_fullsized":"2023-10-06T05:18:39.486600+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:14.319599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.455467+0000","last_change":"2023-10-06T05:18:38.461183+0000","last_active":"2023-10-06T05:18:39.455467+0000","last_peered":"2023-10-06T05:18:39.455467+0000","last_clean":"2023-10-06T05:18:39.455467+0000","last_became_active":"2023-10-06T05:18:38.460893+0000","last_became_peered":"2023-10-06T05:18:38.460893+0000","last_unstale":"2023-10-06T05:18:39.455467+0000","last_undegraded":"2023-10-06T05:18:39.455467+0000","last_fullsized":"2023-10-06T05:18:39.455467+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:12.619655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.459049+0000","last_change":"2023-10-06T05:18:39.455633+0000","last_active":"2023-10-06T05:18:39.459049+0000","last_peered":"2023-10-06T05:18:39.459049+0000","last_clean":"2023-10-06T05:18:39.459049+0000","last_became_active":"2023-10-06T05:18:39.455315+0000","last_became_peered":"2023-10-06T05:18:39.455315+0000","last_unstale":"2023-10-06T05:18:39.459049+0000","last_undegraded":"2023-10-06T05:18:39.459049+0000","last_fullsized":"2023-10-06T05:18:39.459049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:36.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.457412+0000","last_change":"2023-10-06T05:18:38.458370+0000","last_active":"2023-10-06T05:18:39.457412+0000","last_peered":"2023-10-06T05:18:39.457412+0000","last_clean":"2023-10-06T05:18:39.457412+0000","last_became_active":"2023-10-06T05:18:38.458104+0000","last_became_peered":"2023-10-06T05:18:38.458104+0000","last_unstale":"2023-10-06T05:18:39.457412+0000","last_undegraded":"2023-10-06T05:18:39.457412+0000","last_fullsized":"2023-10-06T05:18:39.457412+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:41.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.470782+0000","last_change":"2023-10-06T05:18:39.463471+0000","last_active":"2023-10-06T05:18:39.470782+0000","last_peered":"2023-10-06T05:18:39.470782+0000","last_clean":"2023-10-06T05:18:39.470782+0000","last_became_active":"2023-10-06T05:18:39.462983+0000","last_became_peered":"2023-10-06T05:18:39.462983+0000","last_unstale":"2023-10-06T05:18:39.470782+0000","last_undegraded":"2023-10-06T05:18:39.470782+0000","last_fullsized":"2023-10-06T05:18:39.470782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:13.771913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.447340+0000","last_change":"2023-10-06T05:18:38.463433+0000","last_active":"2023-10-06T05:18:39.447340+0000","last_peered":"2023-10-06T05:18:39.447340+0000","last_clean":"2023-10-06T05:18:39.447340+0000","last_became_active":"2023-10-06T05:18:38.463176+0000","last_became_peered":"2023-10-06T05:18:38.463176+0000","last_unstale":"2023-10-06T05:18:39.447340+0000","last_undegraded":"2023-10-06T05:18:39.447340+0000","last_fullsized":"2023-10-06T05:18:39.447340+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:41.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482236+0000","last_change":"2023-10-06T05:18:39.470253+0000","last_active":"2023-10-06T05:18:39.482236+0000","last_peered":"2023-10-06T05:18:39.482236+0000","last_clean":"2023-10-06T05:18:39.482236+0000","last_became_active":"2023-10-06T05:18:39.469887+0000","last_became_peered":"2023-10-06T05:18:39.469887+0000","last_unstale":"2023-10-06T05:18:39.482236+0000","last_undegraded":"2023-10-06T05:18:39.482236+0000","last_fullsized":"2023-10-06T05:18:39.482236+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:10.865806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"43'3","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.485007+0000","last_change":"2023-10-06T05:18:38.461448+0000","last_active":"2023-10-06T05:18:39.485007+0000","last_peered":"2023-10-06T05:18:39.485007+0000","last_clean":"2023-10-06T05:18:39.485007+0000","last_became_active":"2023-10-06T05:18:38.461184+0000","last_became_peered":"2023-10-06T05:18:38.461184+0000","last_unstale":"2023-10-06T05:18:39.485007+0000","last_undegraded":"2023-10-06T05:18:39.485007+0000","last_fullsized":"2023-10-06T05:18:39.485007+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:34.215548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.475712+0000","last_change":"2023-10-06T05:18:39.470309+0000","last_active":"2023-10-06T05:18:39.475712+0000","last_peered":"2023-10-06T05:18:39.475712+0000","last_clean":"2023-10-06T05:18:39.475712+0000","last_became_active":"2023-10-06T05:18:39.470041+0000","last_became_peered":"2023-10-06T05:18:39.470041+0000","last_unstale":"2023-10-06T05:18:39.475712+0000","last_undegraded":"2023-10-06T05:18:39.475712+0000","last_fullsized":"2023-10-06T05:18:39.475712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:13.626073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"43'6","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.488495+0000","last_change":"2023-10-06T05:18:38.460848+0000","last_active":"2023-10-06T05:18:39.488495+0000","last_peered":"2023-10-06T05:18:39.488495+0000","last_clean":"2023-10-06T05:18:39.488495+0000","last_became_active":"2023-10-06T05:18:38.460598+0000","last_became_peered":"2023-10-06T05:18:38.460598+0000","last_unstale":"2023-10-06T05:18:39.488495+0000","last_undegraded":"2023-10-06T05:18:39.488495+0000","last_fullsized":"2023-10-06T05:18:39.488495+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:12.765494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.471687+0000","last_change":"2023-10-06T05:18:39.469583+0000","last_active":"2023-10-06T05:18:39.471687+0000","last_peered":"2023-10-06T05:18:39.471687+0000","last_clean":"2023-10-06T05:18:39.471687+0000","last_became_active":"2023-10-06T05:18:39.469329+0000","last_became_peered":"2023-10-06T05:18:39.469329+0000","last_unstale":"2023-10-06T05:18:39.471687+0000","last_undegraded":"2023-10-06T05:18:39.471687+0000","last_fullsized":"2023-10-06T05:18:39.471687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:24.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482498+0000","last_change":"2023-10-06T05:18:38.463872+0000","last_active":"2023-10-06T05:18:39.482498+0000","last_peered":"2023-10-06T05:18:39.482498+0000","last_clean":"2023-10-06T05:18:39.482498+0000","last_became_active":"2023-10-06T05:18:38.463587+0000","last_became_peered":"2023-10-06T05:18:38.463587+0000","last_unstale":"2023-10-06T05:18:39.482498+0000","last_undegraded":"2023-10-06T05:18:39.482498+0000","last_fullsized":"2023-10-06T05:18:39.482498+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:27.960703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482181+0000","last_change":"2023-10-06T05:18:39.469269+0000","last_active":"2023-10-06T05:18:39.482181+0000","last_peered":"2023-10-06T05:18:39.482181+0000","last_clean":"2023-10-06T05:18:39.482181+0000","last_became_active":"2023-10-06T05:18:39.468731+0000","last_became_peered":"2023-10-06T05:18:39.468731+0000","last_unstale":"2023-10-06T05:18:39.482181+0000","last_undegraded":"2023-10-06T05:18:39.482181+0000","last_fullsized":"2023-10-06T05:18:39.482181+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:30.345865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.451652+0000","last_change":"2023-10-06T05:18:38.466701+0000","last_active":"2023-10-06T05:18:39.451652+0000","last_peered":"2023-10-06T05:18:39.451652+0000","last_clean":"2023-10-06T05:18:39.451652+0000","last_became_active":"2023-10-06T05:18:38.466409+0000","last_became_peered":"2023-10-06T05:18:38.466409+0000","last_unstale":"2023-10-06T05:18:39.451652+0000","last_undegraded":"2023-10-06T05:18:39.451652+0000","last_fullsized":"2023-10-06T05:18:39.451652+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:26.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.478076+0000","last_change":"2023-10-06T05:18:39.466764+0000","last_active":"2023-10-06T05:18:39.478076+0000","last_peered":"2023-10-06T05:18:39.478076+0000","last_clean":"2023-10-06T05:18:39.478076+0000","last_became_active":"2023-10-06T05:18:39.466483+0000","last_became_peered":"2023-10-06T05:18:39.466483+0000","last_unstale":"2023-10-06T05:18:39.478076+0000","last_undegraded":"2023-10-06T05:18:39.478076+0000","last_fullsized":"2023-10-06T05:18:39.478076+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:24.843295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.456781+0000","last_change":"2023-10-06T05:18:38.464573+0000","last_active":"2023-10-06T05:18:39.456781+0000","last_peered":"2023-10-06T05:18:39.456781+0000","last_clean":"2023-10-06T05:18:39.456781+0000","last_became_active":"2023-10-06T05:18:38.464303+0000","last_became_peered":"2023-10-06T05:18:38.464303+0000","last_unstale":"2023-10-06T05:18:39.456781+0000","last_undegraded":"2023-10-06T05:18:39.456781+0000","last_fullsized":"2023-10-06T05:18:39.456781+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.487966+0000","last_change":"2023-10-06T05:18:39.482861+0000","last_active":"2023-10-06T05:18:39.487966+0000","last_peered":"2023-10-06T05:18:39.487966+0000","last_clean":"2023-10-06T05:18:39.487966+0000","last_became_active":"2023-10-06T05:18:39.482492+0000","last_became_peered":"2023-10-06T05:18:39.482492+0000","last_unstale":"2023-10-06T05:18:39.487966+0000","last_undegraded":"2023-10-06T05:18:39.487966+0000","last_fullsized":"2023-10-06T05:18:39.487966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:19.389479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"43'8","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.484482+0000","last_change":"2023-10-06T05:18:38.458013+0000","last_active":"2023-10-06T05:18:39.484482+0000","last_peered":"2023-10-06T05:18:39.484482+0000","last_clean":"2023-10-06T05:18:39.484482+0000","last_became_active":"2023-10-06T05:18:38.457708+0000","last_became_peered":"2023-10-06T05:18:38.457708+0000","last_unstale":"2023-10-06T05:18:39.484482+0000","last_undegraded":"2023-10-06T05:18:39.484482+0000","last_fullsized":"2023-10-06T05:18:39.484482+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:35.907402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.475243+0000","last_change":"2023-10-06T05:18:39.461090+0000","last_active":"2023-10-06T05:18:39.475243+0000","last_peered":"2023-10-06T05:18:39.475243+0000","last_clean":"2023-10-06T05:18:39.475243+0000","last_became_active":"2023-10-06T05:18:39.460746+0000","last_became_peered":"2023-10-06T05:18:39.460746+0000","last_unstale":"2023-10-06T05:18:39.475243+0000","last_undegraded":"2023-10-06T05:18:39.475243+0000","last_fullsized":"2023-10-06T05:18:39.475243+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:10.451499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.486791+0000","last_change":"2023-10-06T05:18:38.466371+0000","last_active":"2023-10-06T05:18:39.486791+0000","last_peered":"2023-10-06T05:18:39.486791+0000","last_clean":"2023-10-06T05:18:39.486791+0000","last_became_active":"2023-10-06T05:18:38.466099+0000","last_became_peered":"2023-10-06T05:18:38.466099+0000","last_unstale":"2023-10-06T05:18:39.486791+0000","last_undegraded":"2023-10-06T05:18:39.486791+0000","last_fullsized":"2023-10-06T05:18:39.486791+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:12.765494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.477526+0000","last_change":"2023-10-06T05:18:39.472168+0000","last_active":"2023-10-06T05:18:39.477526+0000","last_peered":"2023-10-06T05:18:39.477526+0000","last_clean":"2023-10-06T05:18:39.477526+0000","last_became_active":"2023-10-06T05:18:39.471885+0000","last_became_peered":"2023-10-06T05:18:39.471885+0000","last_unstale":"2023-10-06T05:18:39.477526+0000","last_undegraded":"2023-10-06T05:18:39.477526+0000","last_fullsized":"2023-10-06T05:18:39.477526+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:11.273331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.472302+0000","last_change":"2023-10-06T05:18:38.456427+0000","last_active":"2023-10-06T05:18:39.472302+0000","last_peered":"2023-10-06T05:18:39.472302+0000","last_clean":"2023-10-06T05:18:39.472302+0000","last_became_active":"2023-10-06T05:18:38.456172+0000","last_became_peered":"2023-10-06T05:18:38.456172+0000","last_unstale":"2023-10-06T05:18:39.472302+0000","last_undegraded":"2023-10-06T05:18:39.472302+0000","last_fullsized":"2023-10-06T05:18:39.472302+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:13.476914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.475813+0000","last_change":"2023-10-06T05:18:39.471262+0000","last_active":"2023-10-06T05:18:39.475813+0000","last_peered":"2023-10-06T05:18:39.475813+0000","last_clean":"2023-10-06T05:18:39.475813+0000","last_became_active":"2023-10-06T05:18:39.470987+0000","last_became_peered":"2023-10-06T05:18:39.470987+0000","last_unstale":"2023-10-06T05:18:39.475813+0000","last_undegraded":"2023-10-06T05:18:39.475813+0000","last_fullsized":"2023-10-06T05:18:39.475813+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"43'1","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.483012+0000","last_change":"2023-10-06T05:18:38.466443+0000","last_active":"2023-10-06T05:18:39.483012+0000","last_peered":"2023-10-06T05:18:39.483012+0000","last_clean":"2023-10-06T05:18:39.483012+0000","last_became_active":"2023-10-06T05:18:38.465603+0000","last_became_peered":"2023-10-06T05:18:38.465603+0000","last_unstale":"2023-10-06T05:18:39.483012+0000","last_undegraded":"2023-10-06T05:18:39.483012+0000","last_fullsized":"2023-10-06T05:18:39.483012+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:08.665979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.465919+0000","last_change":"2023-10-06T05:18:39.461885+0000","last_active":"2023-10-06T05:18:39.465919+0000","last_peered":"2023-10-06T05:18:39.465919+0000","last_clean":"2023-10-06T05:18:39.465919+0000","last_became_active":"2023-10-06T05:18:39.461503+0000","last_became_peered":"2023-10-06T05:18:39.461503+0000","last_unstale":"2023-10-06T05:18:39.465919+0000","last_undegraded":"2023-10-06T05:18:39.465919+0000","last_fullsized":"2023-10-06T05:18:39.465919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:38.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.452361+0000","last_change":"2023-10-06T05:18:38.467188+0000","last_active":"2023-10-06T05:18:39.452361+0000","last_peered":"2023-10-06T05:18:39.452361+0000","last_clean":"2023-10-06T05:18:39.452361+0000","last_became_active":"2023-10-06T05:18:38.466475+0000","last_became_peered":"2023-10-06T05:18:38.466475+0000","last_unstale":"2023-10-06T05:18:39.452361+0000","last_undegraded":"2023-10-06T05:18:39.452361+0000","last_fullsized":"2023-10-06T05:18:39.452361+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:22.230340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.471655+0000","last_change":"2023-10-06T05:18:39.463773+0000","last_active":"2023-10-06T05:18:39.471655+0000","last_peered":"2023-10-06T05:18:39.471655+0000","last_clean":"2023-10-06T05:18:39.471655+0000","last_became_active":"2023-10-06T05:18:39.463147+0000","last_became_peered":"2023-10-06T05:18:39.463147+0000","last_unstale":"2023-10-06T05:18:39.471655+0000","last_undegraded":"2023-10-06T05:18:39.471655+0000","last_fullsized":"2023-10-06T05:18:39.471655+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:13.626073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.474545+0000","last_change":"2023-10-06T05:18:38.468654+0000","last_active":"2023-10-06T05:18:39.474545+0000","last_peered":"2023-10-06T05:18:39.474545+0000","last_clean":"2023-10-06T05:18:39.474545+0000","last_became_active":"2023-10-06T05:18:38.468394+0000","last_became_peered":"2023-10-06T05:18:38.468394+0000","last_unstale":"2023-10-06T05:18:39.474545+0000","last_undegraded":"2023-10-06T05:18:39.474545+0000","last_fullsized":"2023-10-06T05:18:39.474545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:56.721340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.469906+0000","last_change":"2023-10-06T05:18:39.460510+0000","last_active":"2023-10-06T05:18:39.469906+0000","last_peered":"2023-10-06T05:18:39.469906+0000","last_clean":"2023-10-06T05:18:39.469906+0000","last_became_active":"2023-10-06T05:18:39.460209+0000","last_became_peered":"2023-10-06T05:18:39.460209+0000","last_unstale":"2023-10-06T05:18:39.469906+0000","last_undegraded":"2023-10-06T05:18:39.469906+0000","last_fullsized":"2023-10-06T05:18:39.469906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:44.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.448563+0000","last_change":"2023-10-06T05:18:38.460519+0000","last_active":"2023-10-06T05:18:39.448563+0000","last_peered":"2023-10-06T05:18:39.448563+0000","last_clean":"2023-10-06T05:18:39.448563+0000","last_became_active":"2023-10-06T05:18:38.460215+0000","last_became_peered":"2023-10-06T05:18:38.460215+0000","last_unstale":"2023-10-06T05:18:39.448563+0000","last_undegraded":"2023-10-06T05:18:39.448563+0000","last_fullsized":"2023-10-06T05:18:39.448563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:12.619655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.487204+0000","last_change":"2023-10-06T05:18:39.479012+0000","last_active":"2023-10-06T05:18:39.487204+0000","last_peered":"2023-10-06T05:18:39.487204+0000","last_clean":"2023-10-06T05:18:39.487204+0000","last_became_active":"2023-10-06T05:18:39.478385+0000","last_became_peered":"2023-10-06T05:18:39.478385+0000","last_unstale":"2023-10-06T05:18:39.487204+0000","last_undegraded":"2023-10-06T05:18:39.487204+0000","last_fullsized":"2023-10-06T05:18:39.487204+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:06.138873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.455725+0000","last_change":"2023-10-06T05:18:38.462389+0000","last_active":"2023-10-06T05:18:39.455725+0000","last_peered":"2023-10-06T05:18:39.455725+0000","last_clean":"2023-10-06T05:18:39.455725+0000","last_became_active":"2023-10-06T05:18:38.462121+0000","last_became_peered":"2023-10-06T05:18:38.462121+0000","last_unstale":"2023-10-06T05:18:39.455725+0000","last_undegraded":"2023-10-06T05:18:39.455725+0000","last_fullsized":"2023-10-06T05:18:39.455725+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:23.179454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.458451+0000","last_change":"2023-10-06T05:18:39.455606+0000","last_active":"2023-10-06T05:18:39.458451+0000","last_peered":"2023-10-06T05:18:39.458451+0000","last_clean":"2023-10-06T05:18:39.458451+0000","last_became_active":"2023-10-06T05:18:39.455331+0000","last_became_peered":"2023-10-06T05:18:39.455331+0000","last_unstale":"2023-10-06T05:18:39.458451+0000","last_undegraded":"2023-10-06T05:18:39.458451+0000","last_fullsized":"2023-10-06T05:18:39.458451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:22.686454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.454379+0000","last_change":"2023-10-06T05:18:38.457211+0000","last_active":"2023-10-06T05:18:39.454379+0000","last_peered":"2023-10-06T05:18:39.454379+0000","last_clean":"2023-10-06T05:18:39.454379+0000","last_became_active":"2023-10-06T05:18:38.456921+0000","last_became_peered":"2023-10-06T05:18:38.456921+0000","last_unstale":"2023-10-06T05:18:39.454379+0000","last_undegraded":"2023-10-06T05:18:39.454379+0000","last_fullsized":"2023-10-06T05:18:39.454379+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:11.790017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.488011+0000","last_change":"2023-10-06T05:18:39.483778+0000","last_active":"2023-10-06T05:18:39.488011+0000","last_peered":"2023-10-06T05:18:39.488011+0000","last_clean":"2023-10-06T05:18:39.488011+0000","last_became_active":"2023-10-06T05:18:39.483456+0000","last_became_peered":"2023-10-06T05:18:39.483456+0000","last_unstale":"2023-10-06T05:18:39.488011+0000","last_undegraded":"2023-10-06T05:18:39.488011+0000","last_fullsized":"2023-10-06T05:18:39.488011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:23.882938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.446921+0000","last_change":"2023-10-06T05:18:38.460312+0000","last_active":"2023-10-06T05:18:39.446921+0000","last_peered":"2023-10-06T05:18:39.446921+0000","last_clean":"2023-10-06T05:18:39.446921+0000","last_became_active":"2023-10-06T05:18:38.460043+0000","last_became_peered":"2023-10-06T05:18:38.460043+0000","last_unstale":"2023-10-06T05:18:39.446921+0000","last_undegraded":"2023-10-06T05:18:39.446921+0000","last_fullsized":"2023-10-06T05:18:39.446921+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:09.080286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.447146+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:18:39.447146+0000","last_peered":"2023-10-06T05:18:39.447146+0000","last_clean":"2023-10-06T05:18:39.447146+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:18:39.447146+0000","last_undegraded":"2023-10-06T05:18:39.447146+0000","last_fullsized":"2023-10-06T05:18:39.447146+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.471438+0000","last_change":"2023-10-06T05:18:39.469053+0000","last_active":"2023-10-06T05:18:39.471438+0000","last_peered":"2023-10-06T05:18:39.471438+0000","last_clean":"2023-10-06T05:18:39.471438+0000","last_became_active":"2023-10-06T05:18:39.468748+0000","last_became_peered":"2023-10-06T05:18:39.468748+0000","last_unstale":"2023-10-06T05:18:39.471438+0000","last_undegraded":"2023-10-06T05:18:39.471438+0000","last_fullsized":"2023-10-06T05:18:39.471438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:03.110568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.448676+0000","last_change":"2023-10-06T05:18:38.458978+0000","last_active":"2023-10-06T05:18:39.448676+0000","last_peered":"2023-10-06T05:18:39.448676+0000","last_clean":"2023-10-06T05:18:39.448676+0000","last_became_active":"2023-10-06T05:18:38.458700+0000","last_became_peered":"2023-10-06T05:18:38.458700+0000","last_unstale":"2023-10-06T05:18:39.448676+0000","last_undegraded":"2023-10-06T05:18:39.448676+0000","last_fullsized":"2023-10-06T05:18:39.448676+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:23.179454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.448823+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:18:39.448823+0000","last_peered":"2023-10-06T05:18:39.448823+0000","last_clean":"2023-10-06T05:18:39.448823+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:18:39.448823+0000","last_undegraded":"2023-10-06T05:18:39.448823+0000","last_fullsized":"2023-10-06T05:18:39.448823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.482504+0000","last_change":"2023-10-06T05:18:39.474687+0000","last_active":"2023-10-06T05:18:39.482504+0000","last_peered":"2023-10-06T05:18:39.482504+0000","last_clean":"2023-10-06T05:18:39.482504+0000","last_became_active":"2023-10-06T05:18:39.474210+0000","last_became_peered":"2023-10-06T05:18:39.474210+0000","last_unstale":"2023-10-06T05:18:39.482504+0000","last_undegraded":"2023-10-06T05:18:39.482504+0000","last_fullsized":"2023-10-06T05:18:39.482504+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:24.640715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.458220+0000","last_change":"2023-10-06T05:18:38.460411+0000","last_active":"2023-10-06T05:18:39.458220+0000","last_peered":"2023-10-06T05:18:39.458220+0000","last_clean":"2023-10-06T05:18:39.458220+0000","last_became_active":"2023-10-06T05:18:38.460142+0000","last_became_peered":"2023-10-06T05:18:38.460142+0000","last_unstale":"2023-10-06T05:18:39.458220+0000","last_undegraded":"2023-10-06T05:18:39.458220+0000","last_fullsized":"2023-10-06T05:18:39.458220+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:10.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.446607+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:18:39.446607+0000","last_peered":"2023-10-06T05:18:39.446607+0000","last_clean":"2023-10-06T05:18:39.446607+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:18:39.446607+0000","last_undegraded":"2023-10-06T05:18:39.446607+0000","last_fullsized":"2023-10-06T05:18:39.446607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":182,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.455721+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:18:39.455721+0000","last_peered":"2023-10-06T05:18:39.455721+0000","last_clean":"2023-10-06T05:18:39.455721+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:18:39.455721+0000","last_undegraded":"2023-10-06T05:18:39.455721+0000","last_fullsized":"2023-10-06T05:18:39.455721+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.474710+0000","last_change":"2023-10-06T05:18:39.462821+0000","last_active":"2023-10-06T05:18:39.474710+0000","last_peered":"2023-10-06T05:18:39.474710+0000","last_clean":"2023-10-06T05:18:39.474710+0000","last_became_active":"2023-10-06T05:18:39.462540+0000","last_became_peered":"2023-10-06T05:18:39.462540+0000","last_unstale":"2023-10-06T05:18:39.474710+0000","last_undegraded":"2023-10-06T05:18:39.474710+0000","last_fullsized":"2023-10-06T05:18:39.474710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:41.261742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.455838+0000","last_change":"2023-10-06T05:18:38.463399+0000","last_active":"2023-10-06T05:18:39.455838+0000","last_peered":"2023-10-06T05:18:39.455838+0000","last_clean":"2023-10-06T05:18:39.455838+0000","last_became_active":"2023-10-06T05:18:38.463130+0000","last_became_peered":"2023-10-06T05:18:38.463130+0000","last_unstale":"2023-10-06T05:18:39.455838+0000","last_undegraded":"2023-10-06T05:18:39.455838+0000","last_fullsized":"2023-10-06T05:18:39.455838+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:35.907402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.457278+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:18:39.457278+0000","last_peered":"2023-10-06T05:18:39.457278+0000","last_clean":"2023-10-06T05:18:39.457278+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:18:39.457278+0000","last_undegraded":"2023-10-06T05:18:39.457278+0000","last_fullsized":"2023-10-06T05:18:39.457278+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.471117+0000","last_change":"2023-10-06T05:18:39.460842+0000","last_active":"2023-10-06T05:18:39.471117+0000","last_peered":"2023-10-06T05:18:39.471117+0000","last_clean":"2023-10-06T05:18:39.471117+0000","last_became_active":"2023-10-06T05:18:39.460568+0000","last_became_peered":"2023-10-06T05:18:39.460568+0000","last_unstale":"2023-10-06T05:18:39.471117+0000","last_undegraded":"2023-10-06T05:18:39.471117+0000","last_fullsized":"2023-10-06T05:18:39.471117+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:52.371640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T05:18:41.072900+0000","last_change":"2023-10-06T05:18:38.463382+0000","last_active":"2023-10-06T05:18:41.072900+0000","last_peered":"2023-10-06T05:18:41.072900+0000","last_clean":"2023-10-06T05:18:41.072900+0000","last_became_active":"2023-10-06T05:18:38.463112+0000","last_became_peered":"2023-10-06T05:18:38.463112+0000","last_unstale":"2023-10-06T05:18:41.072900+0000","last_undegraded":"2023-10-06T05:18:41.072900+0000","last_fullsized":"2023-10-06T05:18:41.072900+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:37.839836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.455643+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:18:39.455643+0000","last_peered":"2023-10-06T05:18:39.455643+0000","last_clean":"2023-10-06T05:18:39.455643+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:18:39.455643+0000","last_undegraded":"2023-10-06T05:18:39.455643+0000","last_fullsized":"2023-10-06T05:18:39.455643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.479657+0000","last_change":"2023-10-06T05:18:39.478438+0000","last_active":"2023-10-06T05:18:39.479657+0000","last_peered":"2023-10-06T05:18:39.479657+0000","last_clean":"2023-10-06T05:18:39.479657+0000","last_became_active":"2023-10-06T05:18:39.478139+0000","last_became_peered":"2023-10-06T05:18:39.478139+0000","last_unstale":"2023-10-06T05:18:39.479657+0000","last_undegraded":"2023-10-06T05:18:39.479657+0000","last_fullsized":"2023-10-06T05:18:39.479657+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:56.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"43'2","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.483732+0000","last_change":"2023-10-06T05:18:38.466799+0000","last_active":"2023-10-06T05:18:39.483732+0000","last_peered":"2023-10-06T05:18:39.483732+0000","last_clean":"2023-10-06T05:18:39.483732+0000","last_became_active":"2023-10-06T05:18:38.466486+0000","last_became_peered":"2023-10-06T05:18:38.466486+0000","last_unstale":"2023-10-06T05:18:39.483732+0000","last_undegraded":"2023-10-06T05:18:39.483732+0000","last_fullsized":"2023-10-06T05:18:39.483732+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:44:04.252646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.451863+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:18:39.451863+0000","last_peered":"2023-10-06T05:18:39.451863+0000","last_clean":"2023-10-06T05:18:39.451863+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:18:39.451863+0000","last_undegraded":"2023-10-06T05:18:39.451863+0000","last_fullsized":"2023-10-06T05:18:39.451863+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.478058+0000","last_change":"2023-10-06T05:18:39.466456+0000","last_active":"2023-10-06T05:18:39.478058+0000","last_peered":"2023-10-06T05:18:39.478058+0000","last_clean":"2023-10-06T05:18:39.478058+0000","last_became_active":"2023-10-06T05:18:39.466159+0000","last_became_peered":"2023-10-06T05:18:39.466159+0000","last_unstale":"2023-10-06T05:18:39.478058+0000","last_undegraded":"2023-10-06T05:18:39.478058+0000","last_fullsized":"2023-10-06T05:18:39.478058+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:06.022509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.451491+0000","last_change":"2023-10-06T05:18:38.467529+0000","last_active":"2023-10-06T05:18:39.451491+0000","last_peered":"2023-10-06T05:18:39.451491+0000","last_clean":"2023-10-06T05:18:39.451491+0000","last_became_active":"2023-10-06T05:18:38.467262+0000","last_became_peered":"2023-10-06T05:18:38.467262+0000","last_unstale":"2023-10-06T05:18:39.451491+0000","last_undegraded":"2023-10-06T05:18:39.451491+0000","last_fullsized":"2023-10-06T05:18:39.451491+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:34.215548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.453537+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:18:39.453537+0000","last_peered":"2023-10-06T05:18:39.453537+0000","last_clean":"2023-10-06T05:18:39.453537+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:18:39.453537+0000","last_undegraded":"2023-10-06T05:18:39.453537+0000","last_fullsized":"2023-10-06T05:18:39.453537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.487224+0000","last_change":"2023-10-06T05:18:39.479226+0000","last_active":"2023-10-06T05:18:39.487224+0000","last_peered":"2023-10-06T05:18:39.487224+0000","last_clean":"2023-10-06T05:18:39.487224+0000","last_became_active":"2023-10-06T05:18:39.478693+0000","last_became_peered":"2023-10-06T05:18:39.478693+0000","last_unstale":"2023-10-06T05:18:39.487224+0000","last_undegraded":"2023-10-06T05:18:39.487224+0000","last_fullsized":"2023-10-06T05:18:39.487224+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:38.431948+0000","last_clean_scrub_stamp":"2023-10-06T05:18:38.431948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:42.520562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.447028+0000","last_change":"2023-10-06T05:18:38.460634+0000","last_active":"2023-10-06T05:18:39.447028+0000","last_peered":"2023-10-06T05:18:39.447028+0000","last_clean":"2023-10-06T05:18:39.447028+0000","last_became_active":"2023-10-06T05:18:38.459850+0000","last_became_peered":"2023-10-06T05:18:38.459850+0000","last_unstale":"2023-10-06T05:18:39.447028+0000","last_undegraded":"2023-10-06T05:18:39.447028+0000","last_fullsized":"2023-10-06T05:18:39.447028+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:18:37.425529+0000","last_clean_scrub_stamp":"2023-10-06T05:18:37.425529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:10.266912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T05:18:39.450750+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:18:39.450750+0000","last_peered":"2023-10-06T05:18:39.450750+0000","last_clean":"2023-10-06T05:18:39.450750+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:18:39.450750+0000","last_undegraded":"2023-10-06T05:18:39.450750+0000","last_fullsized":"2023-10-06T05:18:39.450750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298380,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073460,"statfs":{"total":96636764160,"available":96331223040,"internally_reserved":0,"allocated":1118208,"data_stored":810794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.63300000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":512,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":524288,"data_stored":220926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.63}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.67700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298380,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073460,"statfs":{"total":96636764160,"available":96331223040,"internally_reserved":0,"allocated":1118208,"data_stored":810830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56599999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.97499999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.0820000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.0649999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297800,"kb_used_data":512,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074040,"statfs":{"total":96636764160,"available":96331816960,"internally_reserved":0,"allocated":524288,"data_stored":220358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.628},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.78800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.97199999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.01},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.95099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297804,"kb_used_data":516,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074036,"statfs":{"total":96636764160,"available":96331812864,"internally_reserved":0,"allocated":528384,"data_stored":221838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60799999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.64000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.68899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297808,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074032,"statfs":{"total":96636764160,"available":96331808768,"internally_reserved":0,"allocated":532480,"data_stored":221860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.28399999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.5},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.435}]},{"osd":2,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.621},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.54500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297796,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074044,"statfs":{"total":96636764160,"available":96331821056,"internally_reserved":0,"allocated":520192,"data_stored":220856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.59299999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.495},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.82399999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":7.742,"5min":7.742,"15min":7.742},"last":0.62}]},{"osd":5,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.755},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.73399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297792,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074048,"statfs":{"total":96636764160,"available":96331825152,"internally_reserved":0,"allocated":516096,"data_stored":220302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":9.2780000000000005,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":9.25,"5min":9.25,"15min":9.25},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.108000000000001,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.81499999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":10.384,"5min":10.384,"15min":10.384},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":9.1699999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.86899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.205,"5min":10.205,"15min":10.205},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.84099999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.95999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.0029999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.05}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:18:49.325 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:18:49.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:18:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.556+0000 7ff4183ff700 1 Processor -- start 2023-10-06T05:18:49.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.557+0000 7ff4183ff700 1 -- start start 2023-10-06T05:18:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.558+0000 7ff4183ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff410159930 0x7ff4100b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.558+0000 7ff4183ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff4100cd580 0x7ff4100b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.558+0000 7ff4183ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41015d2d0 con 0x7ff410159560 2023-10-06T05:18:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.558+0000 7ff4183ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41015d5b0 con 0x7ff410159930 2023-10-06T05:18:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.558+0000 7ff4183ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41015d890 con 0x7ff4100cd580 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.558+0000 7ff41599a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff410159930 0x7ff4100b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.558+0000 7ff41619b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff410159560 0x7ff4100cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59490/0 (socket says 172.21.15.31:59490) 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.558+0000 7ff41599a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff410159930 0x7ff4100b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33942/0 (socket says 172.21.15.31:33942) 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.558+0000 7ff41619b700 1 -- 172.21.15.31:0/3349276916 learned_addr learned my addr 172.21.15.31:0/3349276916 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.558+0000 7ff41699c700 1 --2- 172.21.15.31:0/3349276916 >> v2:172.21.15.62:3300/0 conn(0x7ff4100cd580 0x7ff4100b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.559+0000 7ff41599a700 1 -- 172.21.15.31:0/3349276916 >> v2:172.21.15.62:3300/0 conn(0x7ff4100cd580 msgr2=0x7ff4100b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.559+0000 7ff41599a700 1 --2- 172.21.15.31:0/3349276916 >> v2:172.21.15.62:3300/0 conn(0x7ff4100cd580 0x7ff4100b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.559+0000 7ff41599a700 1 -- 172.21.15.31:0/3349276916 >> v1:172.21.15.31:6789/0 conn(0x7ff410159560 legacy=0x7ff4100cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.559+0000 7ff41599a700 1 -- 172.21.15.31:0/3349276916 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41015db70 con 0x7ff410159930 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.559+0000 7ff415199700 1 -- 172.21.15.31:0/3349276916 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758126947 0 0) 0x7ff41015d2d0 con 0x7ff410159560 2023-10-06T05:18:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.559+0000 7ff41599a700 1 --2- 172.21.15.31:0/3349276916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 0x7ff4100b2470 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff4100b1740 tx=0x7ff4000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719c6eed63fd55ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.559+0000 7ff415199700 1 -- 172.21.15.31:0/3349276916 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40000b6b0 con 0x7ff410159930 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.559+0000 7ff415199700 1 -- 172.21.15.31:0/3349276916 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40000b810 con 0x7ff410159930 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.559+0000 7ff415199700 1 -- 172.21.15.31:0/3349276916 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40000bbb0 con 0x7ff410159930 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.560+0000 7ff4183ff700 1 -- 172.21.15.31:0/3349276916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 msgr2=0x7ff4100b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.560+0000 7ff4183ff700 1 --2- 172.21.15.31:0/3349276916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 0x7ff4100b2470 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff4100b1740 tx=0x7ff4000091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.560+0000 7ff4183ff700 1 -- 172.21.15.31:0/3349276916 shutdown_connections 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.560+0000 7ff4183ff700 1 --2- 172.21.15.31:0/3349276916 >> v2:172.21.15.62:3300/0 conn(0x7ff4100cd580 0x7ff4100b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.560+0000 7ff4183ff700 1 --2- 172.21.15.31:0/3349276916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 0x7ff4100b2470 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.560+0000 7ff4183ff700 1 -- 172.21.15.31:0/3349276916 >> 172.21.15.31:0/3349276916 conn(0x7ff41006ec90 msgr2=0x7ff41006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.560+0000 7ff4183ff700 1 -- 172.21.15.31:0/3349276916 shutdown_connections 2023-10-06T05:18:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.560+0000 7ff4183ff700 1 -- 172.21.15.31:0/3349276916 wait complete. 2023-10-06T05:18:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.561+0000 7ff4183ff700 1 Processor -- start 2023-10-06T05:18:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.561+0000 7ff4183ff700 1 -- start start 2023-10-06T05:18:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.562+0000 7ff4183ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100cd580 0x7ff4100c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.562+0000 7ff4183ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff410159560 0x7ff4100c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.562+0000 7ff4183ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 0x7ff41015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.562+0000 7ff4183ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41016af70 con 0x7ff410159930 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.562+0000 7ff4183ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100c5560 con 0x7ff4100cd580 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.562+0000 7ff4183ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41016a360 con 0x7ff410159560 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41699c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 0x7ff41015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41699c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 0x7ff41015bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33958/0 (socket says 172.21.15.31:33958) 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41699c700 1 -- 172.21.15.31:0/311131913 learned_addr learned my addr 172.21.15.31:0/311131913 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41619b700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100cd580 0x7ff4100c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41599a700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff410159560 0x7ff4100c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41699c700 1 -- 172.21.15.31:0/311131913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff410159560 msgr2=0x7ff4100c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41699c700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff410159560 0x7ff4100c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41699c700 1 -- 172.21.15.31:0/311131913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100cd580 msgr2=0x7ff4100c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41699c700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100cd580 0x7ff4100c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41699c700 1 -- 172.21.15.31:0/311131913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400009420 con 0x7ff410159930 2023-10-06T05:18:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.563+0000 7ff41699c700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 0x7ff41015bb30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff408007f30 tx=0x7ff4080098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.564+0000 7ff4067fc700 1 -- 172.21.15.31:0/311131913 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40800eb90 con 0x7ff410159930 2023-10-06T05:18:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.564+0000 7ff4067fc700 1 -- 172.21.15.31:0/311131913 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40800ecf0 con 0x7ff410159930 2023-10-06T05:18:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.564+0000 7ff4067fc700 1 -- 172.21.15.31:0/311131913 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408014670 con 0x7ff410159930 2023-10-06T05:18:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.564+0000 7ff4183ff700 1 -- 172.21.15.31:0/311131913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41016a640 con 0x7ff410159930 2023-10-06T05:18:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.564+0000 7ff4183ff700 1 -- 172.21.15.31:0/311131913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41016ac00 con 0x7ff410159930 2023-10-06T05:18:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.565+0000 7ff4183ff700 1 -- 172.21.15.31:0/311131913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4100c3ef0 con 0x7ff410159930 2023-10-06T05:18:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.566+0000 7ff4067fc700 1 -- 172.21.15.31:0/311131913 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7ff408014a60 con 0x7ff410159930 2023-10-06T05:18:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.567+0000 7ff4067fc700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff3f40670e0 0x7ff3f4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.567+0000 7ff4067fc700 1 -- 172.21.15.31:0/311131913 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6116+0+0 (secure 0 0 0) 0x7ff408084ff0 con 0x7ff410159930 2023-10-06T05:18:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.567+0000 7ff41619b700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff3f40670e0 0x7ff3f4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:49.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.568+0000 7ff41619b700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff3f40670e0 0x7ff3f4069590 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff41015d200 tx=0x7ff40c007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.571+0000 7ff4067fc700 1 -- 172.21.15.31:0/311131913 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff408055890 con 0x7ff410159930 2023-10-06T05:18:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.859+0000 7ff4183ff700 1 -- 172.21.15.31:0/311131913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff41004c380 con 0x7ff410159930 2023-10-06T05:18:49.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.863+0000 7ff4067fc700 1 -- 172.21.15.31:0/311131913 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff40800ee60 con 0x7ff410159930 2023-10-06T05:18:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 -- 172.21.15.31:0/311131913 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff3f40670e0 msgr2=0x7ff3f4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff3f40670e0 0x7ff3f4069590 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff41015d200 tx=0x7ff40c007d30 comp rx=0 tx=0).stop 2023-10-06T05:18:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 -- 172.21.15.31:0/311131913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 msgr2=0x7ff41015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 0x7ff41015bb30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff408007f30 tx=0x7ff4080098e0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 -- 172.21.15.31:0/311131913 shutdown_connections 2023-10-06T05:18:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff3f40670e0 0x7ff3f4069590 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff410159930 0x7ff41015bb30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100cd580 0x7ff4100c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 --2- 172.21.15.31:0/311131913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff410159560 0x7ff4100c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 -- 172.21.15.31:0/311131913 >> 172.21.15.31:0/311131913 conn(0x7ff41006ec90 msgr2=0x7ff4101577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 -- 172.21.15.31:0/311131913 shutdown_connections 2023-10-06T05:18:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:49.865+0000 7ff4183ff700 1 -- 172.21.15.31:0/311131913 wait complete. 2023-10-06T05:18:49.869 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:18:49.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:50.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.078+0000 7f6ee91ff700 1 Processor -- start 2023-10-06T05:18:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.079+0000 7f6ee91ff700 1 -- start start 2023-10-06T05:18:50.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee91ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ee4159500 0x7f6ee40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee91ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee415d2d0 0x7f6ee40cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee91ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee40c3f40 con 0x7f6ee41598d0 2023-10-06T05:18:50.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee91ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee40b2860 con 0x7f6ee4159500 2023-10-06T05:18:50.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee91ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee415d7d0 con 0x7f6ee415d2d0 2023-10-06T05:18:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ee4159500 0x7f6ee40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ee4159500 0x7f6ee40b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33972/0 (socket says 172.21.15.31:33972) 2023-10-06T05:18:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee2d9d700 1 -- 172.21.15.31:0/1456375749 learned_addr learned my addr 172.21.15.31:0/1456375749 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee259c700 1 --2- 172.21.15.31:0/1456375749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee415d2d0 0x7f6ee40cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee2d9d700 1 -- 172.21.15.31:0/1456375749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee415d2d0 msgr2=0x7f6ee40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee2d9d700 1 --2- 172.21.15.31:0/1456375749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee415d2d0 0x7f6ee40cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee2d9d700 1 -- 172.21.15.31:0/1456375749 >> v1:172.21.15.62:6789/0 conn(0x7f6ee41598d0 legacy=0x7f6ee40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.080+0000 7f6ee2d9d700 1 -- 172.21.15.31:0/1456375749 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee40cb000 con 0x7f6ee4159500 2023-10-06T05:18:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.081+0000 7f6ee2d9d700 1 --2- 172.21.15.31:0/1456375749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee4159500 0x7f6ee40b0ab0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40099b0 tx=0x7f6ed4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4423d3d287c13ed7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.081+0000 7f6ee1d9b700 1 -- 172.21.15.31:0/1456375749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400b860 con 0x7f6ee4159500 2023-10-06T05:18:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.081+0000 7f6ee1d9b700 1 -- 172.21.15.31:0/1456375749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed400b9c0 con 0x7f6ee4159500 2023-10-06T05:18:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.081+0000 7f6ee1d9b700 1 -- 172.21.15.31:0/1456375749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400bd60 con 0x7f6ee4159500 2023-10-06T05:18:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.081+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1456375749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee4159500 msgr2=0x7f6ee40b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.081+0000 7f6ee91ff700 1 --2- 172.21.15.31:0/1456375749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee4159500 0x7f6ee40b0ab0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40099b0 tx=0x7f6ed4009250 comp rx=0 tx=0).stop 2023-10-06T05:18:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.082+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1456375749 shutdown_connections 2023-10-06T05:18:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.082+0000 7f6ee91ff700 1 --2- 172.21.15.31:0/1456375749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee4159500 0x7f6ee40b0ab0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.082+0000 7f6ee91ff700 1 --2- 172.21.15.31:0/1456375749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee415d2d0 0x7f6ee40cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.082+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1456375749 >> 172.21.15.31:0/1456375749 conn(0x7f6ee406ec90 msgr2=0x7f6ee406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:50.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.082+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1456375749 shutdown_connections 2023-10-06T05:18:50.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.082+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1456375749 wait complete. 2023-10-06T05:18:50.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.084+0000 7f6ee91ff700 1 Processor -- start 2023-10-06T05:18:50.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.084+0000 7f6ee91ff700 1 -- start start 2023-10-06T05:18:50.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.085+0000 7f6ee91ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee4159500 0x7f6ee4156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.085+0000 7f6ee91ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee41598d0 0x7f6ee40cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.085+0000 7f6ee91ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee415d2d0 0x7f6ee40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.085+0000 7f6ee91ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40b4b90 con 0x7f6ee415d2d0 2023-10-06T05:18:50.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.085+0000 7f6ee91ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40b4e70 con 0x7f6ee41598d0 2023-10-06T05:18:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.085+0000 7f6ee91ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40b5150 con 0x7f6ee4159500 2023-10-06T05:18:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee415d2d0 0x7f6ee40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee415d2d0 0x7f6ee40cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33982/0 (socket says 172.21.15.31:33982) 2023-10-06T05:18:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee359e700 1 -- 172.21.15.31:0/1642766120 learned_addr learned my addr 172.21.15.31:0/1642766120 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee2d9d700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee4159500 0x7f6ee4156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee259c700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee41598d0 0x7f6ee40cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee359e700 1 -- 172.21.15.31:0/1642766120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee4159500 msgr2=0x7f6ee4156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee359e700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee4159500 0x7f6ee4156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee359e700 1 -- 172.21.15.31:0/1642766120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee41598d0 msgr2=0x7f6ee40cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee359e700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee41598d0 0x7f6ee40cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee359e700 1 -- 172.21.15.31:0/1642766120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed40094f0 con 0x7f6ee415d2d0 2023-10-06T05:18:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ee359e700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee415d2d0 0x7f6ee40cfb10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c8e0 tx=0x7f6edc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.086+0000 7f6ecaffd700 1 -- 172.21.15.31:0/1642766120 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00ce80 con 0x7f6ee415d2d0 2023-10-06T05:18:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.087+0000 7f6ecaffd700 1 -- 172.21.15.31:0/1642766120 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc00a7e0 con 0x7f6ee415d2d0 2023-10-06T05:18:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.087+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1642766120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee40b5430 con 0x7f6ee415d2d0 2023-10-06T05:18:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.087+0000 7f6ecaffd700 1 -- 172.21.15.31:0/1642766120 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00ab80 con 0x7f6ee415d2d0 2023-10-06T05:18:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.087+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1642766120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee4156f70 con 0x7f6ee415d2d0 2023-10-06T05:18:50.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.088+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1642766120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee404c380 con 0x7f6ee415d2d0 2023-10-06T05:18:50.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.089+0000 7f6ecaffd700 1 -- 172.21.15.31:0/1642766120 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6edc012420 con 0x7f6ee415d2d0 2023-10-06T05:18:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.089+0000 7f6ecaffd700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ed00670e0 0x7f6ed0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.090+0000 7f6ecaffd700 1 -- 172.21.15.31:0/1642766120 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f6edc084cd0 con 0x7f6ee415d2d0 2023-10-06T05:18:50.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.090+0000 7f6ee2d9d700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ed00670e0 0x7f6ed0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.090+0000 7f6ee2d9d700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ed00670e0 0x7f6ed0069590 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40099b0 tx=0x7f6ed4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.094+0000 7f6ecaffd700 1 -- 172.21.15.31:0/1642766120 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6edc051b20 con 0x7f6ee415d2d0 2023-10-06T05:18:50.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.340+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1642766120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ee40c7e10 con 0x7f6ee415d2d0 2023-10-06T05:18:50.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.341+0000 7f6ecaffd700 1 -- 172.21.15.31:0/1642766120 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f6edc055140 con 0x7f6ee415d2d0 2023-10-06T05:18:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:50.343 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:18:50.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.343+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1642766120 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ed00670e0 msgr2=0x7f6ed0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.343+0000 7f6ee91ff700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ed00670e0 0x7f6ed0069590 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40099b0 tx=0x7f6ed4017000 comp rx=0 tx=0).stop 2023-10-06T05:18:50.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.343+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1642766120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee415d2d0 msgr2=0x7f6ee40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.343+0000 7f6ee91ff700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee415d2d0 0x7f6ee40cfb10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c8e0 tx=0x7f6edc001710 comp rx=0 tx=0).stop 2023-10-06T05:18:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.344+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1642766120 shutdown_connections 2023-10-06T05:18:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.344+0000 7f6ee91ff700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6ed00670e0 0x7f6ed0069590 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.344+0000 7f6ee91ff700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ee415d2d0 0x7f6ee40cfb10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.344+0000 7f6ee91ff700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6ee4159500 0x7f6ee4156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.344+0000 7f6ee91ff700 1 --2- 172.21.15.31:0/1642766120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee41598d0 0x7f6ee40cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.344+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1642766120 >> 172.21.15.31:0/1642766120 conn(0x7f6ee406ec90 msgr2=0x7f6ee40c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:50.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.344+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1642766120 shutdown_connections 2023-10-06T05:18:50.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.344+0000 7f6ee91ff700 1 -- 172.21.15.31:0/1642766120 wait complete. 2023-10-06T05:18:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2023-10-06T05:18:50.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:18:38.946090+0000', 'modified': '2023-10-06T05:18:42.479138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5355}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5355': {'gid': 5355, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6837/1839995141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6835', 'nonce': 1839995141}, {'type': 'v1', 'addr': '172.21.15.31:6837', 'nonce': 1839995141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:18:50.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:18:50.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:18:50.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.967+0000 7fec3a3ff700 1 Processor -- start 2023-10-06T05:18:50.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.967+0000 7fec3a3ff700 1 -- start start 2023-10-06T05:18:50.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.968+0000 7fec3a3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec341595a0 0x7fec34159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.968+0000 7fec3a3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec340cc0d0 0x7fec34068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.968+0000 7fec3a3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec34068bb0 0x7fec340c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.968+0000 7fec3a3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec34069010 con 0x7fec34068bb0 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.968+0000 7fec3a3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec340c95b0 con 0x7fec341595a0 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.968+0000 7fec3a3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec340c9890 con 0x7fec340cc0d0 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec3899c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec34068bb0 0x7fec340c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec3899c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec34068bb0 0x7fec340c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33990/0 (socket says 172.21.15.31:33990) 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec3899c700 1 -- 172.21.15.31:0/2643509564 learned_addr learned my addr 172.21.15.31:0/2643509564 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec337fe700 1 --2- 172.21.15.31:0/2643509564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec340cc0d0 0x7fec34068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec33fff700 1 --2- 172.21.15.31:0/2643509564 >> v2:172.21.15.62:3300/0 conn(0x7fec341595a0 0x7fec34159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec3899c700 1 -- 172.21.15.31:0/2643509564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec340cc0d0 msgr2=0x7fec34068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec3899c700 1 --2- 172.21.15.31:0/2643509564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec340cc0d0 0x7fec34068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec3899c700 1 -- 172.21.15.31:0/2643509564 >> v2:172.21.15.62:3300/0 conn(0x7fec341595a0 msgr2=0x7fec34159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec3899c700 1 --2- 172.21.15.31:0/2643509564 >> v2:172.21.15.62:3300/0 conn(0x7fec341595a0 0x7fec34159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec3899c700 1 -- 172.21.15.31:0/2643509564 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec340c9b70 con 0x7fec34068bb0 2023-10-06T05:18:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.969+0000 7fec33fff700 1 --2- 172.21.15.31:0/2643509564 >> v2:172.21.15.62:3300/0 conn(0x7fec341595a0 0x7fec34159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.970+0000 7fec3899c700 1 --2- 172.21.15.31:0/2643509564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec34068bb0 0x7fec340c8930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fec3406c8d0 tx=0x7fec2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ffb2e37676d4a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.970+0000 7fec32ffd700 1 -- 172.21.15.31:0/2643509564 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00e070 con 0x7fec34068bb0 2023-10-06T05:18:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.970+0000 7fec32ffd700 1 -- 172.21.15.31:0/2643509564 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c00ba20 con 0x7fec34068bb0 2023-10-06T05:18:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.970+0000 7fec32ffd700 1 -- 172.21.15.31:0/2643509564 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0092e0 con 0x7fec34068bb0 2023-10-06T05:18:50.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.970+0000 7fec3a3ff700 1 -- 172.21.15.31:0/2643509564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec34068bb0 msgr2=0x7fec340c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.970+0000 7fec3a3ff700 1 --2- 172.21.15.31:0/2643509564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec34068bb0 0x7fec340c8930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fec3406c8d0 tx=0x7fec2c009840 comp rx=0 tx=0).stop 2023-10-06T05:18:50.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.970+0000 7fec3a3ff700 1 -- 172.21.15.31:0/2643509564 shutdown_connections 2023-10-06T05:18:50.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.970+0000 7fec3a3ff700 1 --2- 172.21.15.31:0/2643509564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec34068bb0 0x7fec340c8930 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.970+0000 7fec3a3ff700 1 --2- 172.21.15.31:0/2643509564 >> v2:172.21.15.62:3300/0 conn(0x7fec341595a0 0x7fec34159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.970+0000 7fec3a3ff700 1 --2- 172.21.15.31:0/2643509564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec340cc0d0 0x7fec34068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.971+0000 7fec3a3ff700 1 -- 172.21.15.31:0/2643509564 >> 172.21.15.31:0/2643509564 conn(0x7fec3406b620 msgr2=0x7fec3406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:50.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.971+0000 7fec3a3ff700 1 -- 172.21.15.31:0/2643509564 shutdown_connections 2023-10-06T05:18:50.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.971+0000 7fec3a3ff700 1 -- 172.21.15.31:0/2643509564 wait complete. 2023-10-06T05:18:50.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.973+0000 7fec3a3ff700 1 Processor -- start 2023-10-06T05:18:50.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.973+0000 7fec3a3ff700 1 -- start start 2023-10-06T05:18:50.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.974+0000 7fec3a3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec34068bb0 0x7fec34169340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.974+0000 7fec3a3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec340cc0d0 0x7fec34156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.974+0000 7fec3a3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec341595a0 0x7fec341532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.974+0000 7fec3a3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec34169880 con 0x7fec341595a0 2023-10-06T05:18:50.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.974+0000 7fec3a3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec34156eb0 con 0x7fec34068bb0 2023-10-06T05:18:50.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.974+0000 7fec3a3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3416a310 con 0x7fec340cc0d0 2023-10-06T05:18:50.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.974+0000 7fec3899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec341595a0 0x7fec341532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec3899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec341595a0 0x7fec341532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33998/0 (socket says 172.21.15.31:33998) 2023-10-06T05:18:50.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec3899c700 1 -- 172.21.15.31:0/3741317357 learned_addr learned my addr 172.21.15.31:0/3741317357 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:50.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec33fff700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec34068bb0 0x7fec34169340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec337fe700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec340cc0d0 0x7fec34156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec3899c700 1 -- 172.21.15.31:0/3741317357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec340cc0d0 msgr2=0x7fec34156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec3899c700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec340cc0d0 0x7fec34156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec3899c700 1 -- 172.21.15.31:0/3741317357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec34068bb0 msgr2=0x7fec34169340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec3899c700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec34068bb0 0x7fec34169340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec3899c700 1 -- 172.21.15.31:0/3741317357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c009ae0 con 0x7fec341595a0 2023-10-06T05:18:50.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec3899c700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec341595a0 0x7fec341532c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fec3406ee00 tx=0x7fec2c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.975+0000 7fec1bfff700 1 -- 172.21.15.31:0/3741317357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0032f0 con 0x7fec341595a0 2023-10-06T05:18:50.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.976+0000 7fec1bfff700 1 -- 172.21.15.31:0/3741317357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c0092e0 con 0x7fec341595a0 2023-10-06T05:18:50.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.976+0000 7fec1bfff700 1 -- 172.21.15.31:0/3741317357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c026460 con 0x7fec341595a0 2023-10-06T05:18:50.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.976+0000 7fec3a3ff700 1 -- 172.21.15.31:0/3741317357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec34169a70 con 0x7fec341595a0 2023-10-06T05:18:50.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.976+0000 7fec3a3ff700 1 -- 172.21.15.31:0/3741317357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3416a030 con 0x7fec341595a0 2023-10-06T05:18:50.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.978+0000 7fec1bfff700 1 -- 172.21.15.31:0/3741317357 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fec2c003040 con 0x7fec341595a0 2023-10-06T05:18:50.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.979+0000 7fec1bfff700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fec20067110 0x7fec200695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.979+0000 7fec1bfff700 1 -- 172.21.15.31:0/3741317357 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fec2c086ad0 con 0x7fec341595a0 2023-10-06T05:18:50.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.979+0000 7fec33fff700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fec20067110 0x7fec200695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.979+0000 7fec33fff700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fec20067110 0x7fec200695c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fec24006f00 tx=0x7fec24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.981+0000 7fec48b15b80 1 Processor -- start 2023-10-06T05:18:50.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.981+0000 7fec48b15b80 1 -- start start 2023-10-06T05:18:50.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.982+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 0x55eaa2ff3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.982+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2fd8f40 0x55eaa2ffbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.982+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ffc4d0 0x55eaa2ffc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.982+0000 7fec48b15b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2ec6310 con 0x55eaa2ffc4d0 2023-10-06T05:18:50.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.982+0000 7fec48b15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2f03130 con 0x55eaa2ff3450 2023-10-06T05:18:50.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.982+0000 7fec48b15b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2f00af0 con 0x55eaa2fd8f40 2023-10-06T05:18:50.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.982+0000 7fec3899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ffc4d0 0x55eaa2ffc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.982+0000 7fec3899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ffc4d0 0x55eaa2ffc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34008/0 (socket says 172.21.15.31:34008) 2023-10-06T05:18:50.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.982+0000 7fec3899c700 1 -- 172.21.15.31:0/386502150 learned_addr learned my addr 172.21.15.31:0/386502150 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:50.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec3899c700 1 -- 172.21.15.31:0/386502150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2fd8f40 msgr2=0x55eaa2ffbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:50.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec3899c700 1 --2- 172.21.15.31:0/386502150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2fd8f40 0x55eaa2ffbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec33fff700 1 --2- 172.21.15.31:0/386502150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 0x55eaa2ff3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec3899c700 1 -- 172.21.15.31:0/386502150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 msgr2=0x55eaa2ff3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:50.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec3899c700 1 --2- 172.21.15.31:0/386502150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 0x55eaa2ff3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec3899c700 1 -- 172.21.15.31:0/386502150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaa2d02790 con 0x55eaa2ffc4d0 2023-10-06T05:18:50.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec33fff700 1 --2- 172.21.15.31:0/386502150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 0x55eaa2ff3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:50.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec3899c700 1 --2- 172.21.15.31:0/386502150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ffc4d0 0x55eaa2ffc940 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fec2c084b80 tx=0x7fec2c009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2461d73c9595187 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec197fa700 1 -- 172.21.15.31:0/386502150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00e040 con 0x55eaa2ffc4d0 2023-10-06T05:18:50.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec197fa700 1 -- 172.21.15.31:0/386502150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c053820 con 0x55eaa2ffc4d0 2023-10-06T05:18:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.983+0000 7fec197fa700 1 -- 172.21.15.31:0/386502150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c05a680 con 0x55eaa2ffc4d0 2023-10-06T05:18:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.984+0000 7fec48b15b80 1 -- 172.21.15.31:0/386502150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ffc4d0 msgr2=0x55eaa2ffc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.984+0000 7fec48b15b80 1 --2- 172.21.15.31:0/386502150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ffc4d0 0x55eaa2ffc940 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fec2c084b80 tx=0x7fec2c009060 comp rx=0 tx=0).stop 2023-10-06T05:18:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.984+0000 7fec48b15b80 1 -- 172.21.15.31:0/386502150 shutdown_connections 2023-10-06T05:18:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.984+0000 7fec48b15b80 1 --2- 172.21.15.31:0/386502150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ffc4d0 0x55eaa2ffc940 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.984+0000 7fec48b15b80 1 --2- 172.21.15.31:0/386502150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 0x55eaa2ff3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.984+0000 7fec48b15b80 1 --2- 172.21.15.31:0/386502150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2fd8f40 0x55eaa2ffbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.984+0000 7fec48b15b80 1 -- 172.21.15.31:0/386502150 >> 172.21.15.31:0/386502150 conn(0x55eaa2ff6880 msgr2=0x55eaa2ff6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.984+0000 7fec48b15b80 1 -- 172.21.15.31:0/386502150 shutdown_connections 2023-10-06T05:18:50.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.984+0000 7fec48b15b80 1 -- 172.21.15.31:0/386502150 wait complete. 2023-10-06T05:18:50.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.985+0000 7fec48b15b80 1 -- start start 2023-10-06T05:18:50.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.985+0000 7fec48b15b80 1 Processor -- start 2023-10-06T05:18:50.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.986+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2fd0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.986+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ff3450 0x55eaa2fd16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.986+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ffc4d0 0x55eaa2fd1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.986+0000 7fec48b15b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2efc8e0 con 0x55eaa2fd8f40 2023-10-06T05:18:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.986+0000 7fec48b15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2ea8980 con 0x55eaa2ffc4d0 2023-10-06T05:18:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.986+0000 7fec48b15b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2ea8620 con 0x55eaa2ff3450 2023-10-06T05:18:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.986+0000 7fec48b15b80 10 client.0 _ll_register_callbacks cb 0x55eaa2d0ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.986+0000 7fec33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2fd0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.986+0000 7fec33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2fd0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34020/0 (socket says 172.21.15.31:34020) 2023-10-06T05:18:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.986+0000 7fec33fff700 1 -- 172.21.15.31:0/2294827697 learned_addr learned my addr 172.21.15.31:0/2294827697 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:50.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec3899c700 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ffc4d0 0x55eaa2fd1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec337fe700 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ff3450 0x55eaa2fd16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec33fff700 1 -- 172.21.15.31:0/2294827697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ff3450 msgr2=0x55eaa2fd16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec33fff700 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ff3450 0x55eaa2fd16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec33fff700 1 -- 172.21.15.31:0/2294827697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ffc4d0 msgr2=0x55eaa2fd1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec33fff700 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ffc4d0 0x55eaa2fd1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec33fff700 1 -- 172.21.15.31:0/2294827697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3416a030 con 0x55eaa2fd8f40 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec337fe700 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ff3450 0x55eaa2fd16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec33fff700 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2fd0b90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fec24016860 tx=0x7fec24019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856f4c11861b45f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec127fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec127fc700 1 -- 172.21.15.31:0/2294827697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec240148e0 con 0x55eaa2fd8f40 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec127fc700 1 -- 172.21.15.31:0/2294827697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fec24014a40 con 0x55eaa2fd8f40 2023-10-06T05:18:50.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.987+0000 7fec48b15b80 1 -- 172.21.15.31:0/2294827697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eaa2fa0d60 con 0x55eaa2fd8f40 2023-10-06T05:18:50.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec127fc700 1 -- 172.21.15.31:0/2294827697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec24014e50 con 0x55eaa2fd8f40 2023-10-06T05:18:50.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec48b15b80 10 client.5514 fetch_fsmap learned FSMap version 25 2023-10-06T05:18:50.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec48b15b80 1 -- 172.21.15.31:0/2294827697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55eaa2def0a0 con 0x55eaa2fd8f40 2023-10-06T05:18:50.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec127fc700 1 -- 172.21.15.31:0/2294827697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eaa2fa0d60 con 0x55eaa2fd8f40 2023-10-06T05:18:50.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec127fc700 1 -- 172.21.15.31:0/2294827697 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fec24005a00 con 0x55eaa2fd8f40 2023-10-06T05:18:50.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec48b15b80 10 client.5514 fetch_fsmap finished waiting for FSMap version 25 2023-10-06T05:18:50.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec48b15b80 10 client.5514 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:18:50.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec48b15b80 20 client.5514 populate_metadata read hostname 'smithi031' 2023-10-06T05:18:50.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec48b15b80 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.31:6835/1839995141,v1:172.21.15.31:6837/1839995141] conn(0x55eaa2fd3820 0x55eaa2fd3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:50.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec48b15b80 4 client.5514 mds_command: new command op to 5355 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:18:50.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec48b15b80 1 -- 172.21.15.31:0/2294827697 --> [v2:172.21.15.31:6835/1839995141,v1:172.21.15.31:6837/1839995141] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eaa2f7db30 con 0x55eaa2fd3820 2023-10-06T05:18:50.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.988+0000 7fec337fe700 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.31:6835/1839995141,v1:172.21.15.31:6837/1839995141] conn(0x55eaa2fd3820 0x55eaa2fd3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:50.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.989+0000 7fec337fe700 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.31:6835/1839995141,v1:172.21.15.31:6837/1839995141] conn(0x55eaa2fd3820 0x55eaa2fd3bf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4afd45f54835ab3 server_cookie=98f0c61a6afb3b95 in_seq=0 out_seq=0 2023-10-06T05:18:50.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.989+0000 7fec127fc700 10 client.5514 ms_handle_connect on v2:172.21.15.31:6835/1839995141 2023-10-06T05:18:50.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec127fc700 1 -- 172.21.15.31:0/2294827697 <== mds.0 v2:172.21.15.31:6835/1839995141 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55eaa2f7db30 con 0x55eaa2fd3820 2023-10-06T05:18:50.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec127fc700 10 client.5514 handle_command_reply: tid=0 2023-10-06T05:18:51.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 client.5514 shutdown 2023-10-06T05:18:51.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 -- 172.21.15.31:0/2294827697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 msgr2=0x55eaa2fd0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2fd0b90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fec24016860 tx=0x7fec24019040 comp rx=0 tx=0).stop 2023-10-06T05:18:51.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 -- 172.21.15.31:0/2294827697 shutdown_connections 2023-10-06T05:18:51.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.31:6835/1839995141,v1:172.21.15.31:6837/1839995141] conn(0x55eaa2fd3820 0x55eaa2fd3bf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ffc4d0 0x55eaa2fd1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2fd0b90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 --2- 172.21.15.31:0/2294827697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ff3450 0x55eaa2fd16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec127fc700 0 client.5514 ms_handle_reset on v2:172.21.15.31:6835/1839995141 2023-10-06T05:18:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 -- 172.21.15.31:0/2294827697 >> 172.21.15.31:0/2294827697 conn(0x55eaa2ff6880 msgr2=0x55eaa2fa8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 -- 172.21.15.31:0/2294827697 shutdown_connections 2023-10-06T05:18:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 1 -- 172.21.15.31:0/2294827697 wait complete. 2023-10-06T05:18:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:50.990+0000 7fec48b15b80 20 client.5514 trim_cache size 0 max 16384 2023-10-06T05:18:51.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.008+0000 7fec48b15b80 1 Processor -- start 2023-10-06T05:18:51.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.008+0000 7fec48b15b80 1 -- start start 2023-10-06T05:18:51.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.009+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2ff5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.009+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 0x55eaa2ff1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.009+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ffc4d0 0x55eaa2ff1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.009+0000 7fec48b15b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2d941e0 con 0x55eaa2fd8f40 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.009+0000 7fec48b15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2ff71c0 con 0x55eaa2ff3450 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.009+0000 7fec48b15b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2fd6b80 con 0x55eaa2ffc4d0 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.009+0000 7fec33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2ff5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.009+0000 7fec33fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2ff5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34024/0 (socket says 172.21.15.31:34024) 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.009+0000 7fec33fff700 1 -- 172.21.15.31:0/3951668753 learned_addr learned my addr 172.21.15.31:0/3951668753 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec337fe700 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 0x55eaa2ff1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec3899c700 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ffc4d0 0x55eaa2ff1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec33fff700 1 -- 172.21.15.31:0/3951668753 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ffc4d0 msgr2=0x55eaa2ff1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec33fff700 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ffc4d0 0x55eaa2ff1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec33fff700 1 -- 172.21.15.31:0/3951668753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 msgr2=0x55eaa2ff1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec33fff700 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 0x55eaa2ff1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec33fff700 1 -- 172.21.15.31:0/3951668753 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaa2ecb3c0 con 0x55eaa2fd8f40 2023-10-06T05:18:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec3899c700 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ffc4d0 0x55eaa2ff1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec337fe700 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 0x55eaa2ff1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec33fff700 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2ff5e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fec24016f80 tx=0x7fec2401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4578c47b61750a69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:51.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec19ffb700 1 -- 172.21.15.31:0/3951668753 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24014e50 con 0x55eaa2fd8f40 2023-10-06T05:18:51.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec19ffb700 1 -- 172.21.15.31:0/3951668753 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2401d050 con 0x55eaa2fd8f40 2023-10-06T05:18:51.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.010+0000 7fec19ffb700 1 -- 172.21.15.31:0/3951668753 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec240148e0 con 0x55eaa2fd8f40 2023-10-06T05:18:51.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.011+0000 7fec48b15b80 1 -- 172.21.15.31:0/3951668753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 msgr2=0x55eaa2ff5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.011+0000 7fec48b15b80 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2ff5e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fec24016f80 tx=0x7fec2401b000 comp rx=0 tx=0).stop 2023-10-06T05:18:51.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.011+0000 7fec48b15b80 1 -- 172.21.15.31:0/3951668753 shutdown_connections 2023-10-06T05:18:51.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.011+0000 7fec48b15b80 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ffc4d0 0x55eaa2ff1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.011+0000 7fec48b15b80 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2fd8f40 0x55eaa2ff5e50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.011+0000 7fec48b15b80 1 --2- 172.21.15.31:0/3951668753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2ff3450 0x55eaa2ff1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.011+0000 7fec48b15b80 1 -- 172.21.15.31:0/3951668753 >> 172.21.15.31:0/3951668753 conn(0x55eaa2ff6880 msgr2=0x55eaa2fcd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.011+0000 7fec48b15b80 1 -- 172.21.15.31:0/3951668753 shutdown_connections 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.011+0000 7fec48b15b80 1 -- 172.21.15.31:0/3951668753 wait complete. 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.012+0000 7fec48b15b80 1 -- start start 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.012+0000 7fec48b15b80 1 Processor -- start 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.013+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2fd8f40 0x55eaa3057ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.013+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ff3450 0x55eaa304e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.013+0000 7fec48b15b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ffc4d0 0x55eaa2fd3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.013+0000 7fec48b15b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa304eee0 con 0x55eaa2ff3450 2023-10-06T05:18:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.013+0000 7fec48b15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2fd43a0 con 0x55eaa2fd8f40 2023-10-06T05:18:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.013+0000 7fec48b15b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaa3059320 con 0x55eaa2ffc4d0 2023-10-06T05:18:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.013+0000 7fec48b15b80 10 client.0 _ll_register_callbacks cb 0x55eaa2d0ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:18:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.013+0000 7fec337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ff3450 0x55eaa304e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.013+0000 7fec337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ff3450 0x55eaa304e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34032/0 (socket says 172.21.15.31:34032) 2023-10-06T05:18:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.013+0000 7fec337fe700 1 -- 172.21.15.31:0/399416353 learned_addr learned my addr 172.21.15.31:0/399416353 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec33fff700 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2fd8f40 0x55eaa3057ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec337fe700 1 -- 172.21.15.31:0/399416353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ffc4d0 msgr2=0x55eaa2fd3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec337fe700 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ffc4d0 0x55eaa2fd3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec337fe700 1 -- 172.21.15.31:0/399416353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2fd8f40 msgr2=0x55eaa3057ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec337fe700 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2fd8f40 0x55eaa3057ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec337fe700 1 -- 172.21.15.31:0/399416353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaa2def0a0 con 0x55eaa2ff3450 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec33fff700 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2fd8f40 0x55eaa3057ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec337fe700 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ff3450 0x55eaa304e9a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x55eaa2fd59b0 tx=0x7fec2801c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a414fb202e9568 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec127fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec127fc700 1 -- 172.21.15.31:0/399416353 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2801b070 con 0x55eaa2ff3450 2023-10-06T05:18:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec48b15b80 1 -- 172.21.15.31:0/399416353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec2c053820 con 0x55eaa2ff3450 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.014+0000 7fec127fc700 1 -- 172.21.15.31:0/399416353 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fec280156a0 con 0x55eaa2ff3450 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec127fc700 1 -- 172.21.15.31:0/399416353 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec28015a70 con 0x55eaa2ff3450 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec48b15b80 10 client.5520 fetch_fsmap learned FSMap version 25 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec48b15b80 1 -- 172.21.15.31:0/399416353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55eaa2d631d0 con 0x55eaa2ff3450 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec127fc700 1 -- 172.21.15.31:0/399416353 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c053820 con 0x55eaa2ff3450 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec127fc700 1 -- 172.21.15.31:0/399416353 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fec28015bd0 con 0x55eaa2ff3450 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec48b15b80 10 client.5520 fetch_fsmap finished waiting for FSMap version 25 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec48b15b80 10 client.5520 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec48b15b80 20 client.5520 populate_metadata read hostname 'smithi031' 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec48b15b80 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.31:6835/1839995141,v1:172.21.15.31:6837/1839995141] conn(0x55eaa2fd15d0 0x55eaa2fd19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec48b15b80 4 client.5520 mds_command: new command op to 5355 tid=0[{"prefix": "status"}] 2023-10-06T05:18:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec48b15b80 1 -- 172.21.15.31:0/399416353 --> [v2:172.21.15.31:6835/1839995141,v1:172.21.15.31:6837/1839995141] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eaa2e9f8d0 con 0x55eaa2fd15d0 2023-10-06T05:18:51.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.015+0000 7fec33fff700 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.31:6835/1839995141,v1:172.21.15.31:6837/1839995141] conn(0x55eaa2fd15d0 0x55eaa2fd19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec127fc700 10 client.5520 ms_handle_connect on v2:172.21.15.31:6835/1839995141 2023-10-06T05:18:51.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec33fff700 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.31:6835/1839995141,v1:172.21.15.31:6837/1839995141] conn(0x55eaa2fd15d0 0x55eaa2fd19a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=913bcdc79598f2af server_cookie=7c3622c39b722050 in_seq=0 out_seq=0 2023-10-06T05:18:51.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec127fc700 1 -- 172.21.15.31:0/399416353 <== mds.0 v2:172.21.15.31:6835/1839995141 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55eaa2e9f8d0 con 0x55eaa2fd15d0 2023-10-06T05:18:51.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec127fc700 10 client.5520 handle_command_reply: tid=0 2023-10-06T05:18:51.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec48b15b80 1 client.5520 shutdown 2023-10-06T05:18:51.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec48b15b80 1 -- 172.21.15.31:0/399416353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ff3450 msgr2=0x55eaa304e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec48b15b80 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ff3450 0x55eaa304e9a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x55eaa2fd59b0 tx=0x7fec2801c250 comp rx=0 tx=0).stop 2023-10-06T05:18:51.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec48b15b80 1 -- 172.21.15.31:0/399416353 shutdown_connections 2023-10-06T05:18:51.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec48b15b80 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55eaa2ffc4d0 0x55eaa2fd3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec48b15b80 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.31:6835/1839995141,v1:172.21.15.31:6837/1839995141] conn(0x55eaa2fd15d0 0x55eaa2fd19a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec48b15b80 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eaa2fd8f40 0x55eaa3057ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec48b15b80 1 --2- 172.21.15.31:0/399416353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eaa2ff3450 0x55eaa304e9a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec127fc700 0 client.5520 ms_handle_reset on v2:172.21.15.31:6835/1839995141 2023-10-06T05:18:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec48b15b80 1 -- 172.21.15.31:0/399416353 >> 172.21.15.31:0/399416353 conn(0x55eaa2ff6880 msgr2=0x55eaa2fa2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.016+0000 7fec48b15b80 1 -- 172.21.15.31:0/399416353 shutdown_connections 2023-10-06T05:18:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec48b15b80 1 -- 172.21.15.31:0/399416353 wait complete. 2023-10-06T05:18:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec48b15b80 20 client.5520 trim_cache size 0 max 16384 2023-10-06T05:18:51.024 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5355, 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 11.575351018999999, 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 25, 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 46, 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 44, 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 18.227745495000001 2023-10-06T05:18:51.025 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:18:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 -- 172.21.15.31:0/3741317357 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fec20067110 msgr2=0x7fec200695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fec20067110 0x7fec200695c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fec24006f00 tx=0x7fec24008040 comp rx=0 tx=0).stop 2023-10-06T05:18:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 -- 172.21.15.31:0/3741317357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec341595a0 msgr2=0x7fec341532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec341595a0 0x7fec341532c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fec3406ee00 tx=0x7fec2c018000 comp rx=0 tx=0).stop 2023-10-06T05:18:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 -- 172.21.15.31:0/3741317357 shutdown_connections 2023-10-06T05:18:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fec20067110 0x7fec200695c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec341595a0 0x7fec341532c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec34068bb0 0x7fec34169340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 --2- 172.21.15.31:0/3741317357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec340cc0d0 0x7fec34156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 -- 172.21.15.31:0/3741317357 >> 172.21.15.31:0/3741317357 conn(0x7fec3406b620 msgr2=0x7fec340ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 -- 172.21.15.31:0/3741317357 shutdown_connections 2023-10-06T05:18:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.017+0000 7fec3a3ff700 1 -- 172.21.15.31:0/3741317357 wait complete. 2023-10-06T05:18:51.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:18:51.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.233+0000 7f1a49bff700 1 Processor -- start 2023-10-06T05:18:51.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.233+0000 7f1a49bff700 1 -- start start 2023-10-06T05:18:51.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.234+0000 7f1a49bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a440c7b60 0x7f1a44069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.234+0000 7f1a49bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a440aee70 0x7f1a4413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.234+0000 7f1a49bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a440ca5b0 con 0x7f1a440c7f30 2023-10-06T05:18:51.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.234+0000 7f1a49bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a440af2d0 con 0x7f1a440aee70 2023-10-06T05:18:51.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.234+0000 7f1a49bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4406f740 con 0x7f1a440c7b60 2023-10-06T05:18:51.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.235+0000 7f1a43fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a440aee70 0x7f1a4413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.235+0000 7f1a43fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a440aee70 0x7f1a4413f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34048/0 (socket says 172.21.15.31:34048) 2023-10-06T05:18:51.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.235+0000 7f1a43fff700 1 -- 172.21.15.31:0/1086880244 learned_addr learned my addr 172.21.15.31:0/1086880244 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:51.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.235+0000 7f1a437fe700 1 --2- 172.21.15.31:0/1086880244 >> v2:172.21.15.62:3300/0 conn(0x7f1a440c7b60 0x7f1a44069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.235+0000 7f1a43fff700 1 -- 172.21.15.31:0/1086880244 >> v2:172.21.15.62:3300/0 conn(0x7f1a440c7b60 msgr2=0x7f1a44069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.235+0000 7f1a43fff700 1 --2- 172.21.15.31:0/1086880244 >> v2:172.21.15.62:3300/0 conn(0x7f1a440c7b60 0x7f1a44069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.235+0000 7f1a43fff700 1 -- 172.21.15.31:0/1086880244 >> v1:172.21.15.31:6789/0 conn(0x7f1a440c7f30 legacy=0x7f1a4406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.235+0000 7f1a43fff700 1 -- 172.21.15.31:0/1086880244 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440c9c80 con 0x7f1a440aee70 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.235+0000 7f1a427fc700 1 -- 172.21.15.31:0/1086880244 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20021276 0 0) 0x7f1a440ca5b0 con 0x7f1a440c7f30 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a43fff700 1 --2- 172.21.15.31:0/1086880244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440aee70 0x7f1a4413f690 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1a4406fa80 tx=0x7f1a3c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256710ec8d749387 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a427fc700 1 -- 172.21.15.31:0/1086880244 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00e070 con 0x7f1a440aee70 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a427fc700 1 -- 172.21.15.31:0/1086880244 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c00ba50 con 0x7f1a440aee70 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a427fc700 1 -- 172.21.15.31:0/1086880244 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0092e0 con 0x7f1a440aee70 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a49bff700 1 -- 172.21.15.31:0/1086880244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440aee70 msgr2=0x7f1a4413f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a49bff700 1 --2- 172.21.15.31:0/1086880244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440aee70 0x7f1a4413f690 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1a4406fa80 tx=0x7f1a3c009840 comp rx=0 tx=0).stop 2023-10-06T05:18:51.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a49bff700 1 -- 172.21.15.31:0/1086880244 shutdown_connections 2023-10-06T05:18:51.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a49bff700 1 --2- 172.21.15.31:0/1086880244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440aee70 0x7f1a4413f690 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a49bff700 1 --2- 172.21.15.31:0/1086880244 >> v2:172.21.15.62:3300/0 conn(0x7f1a440c7b60 0x7f1a44069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a49bff700 1 -- 172.21.15.31:0/1086880244 >> 172.21.15.31:0/1086880244 conn(0x7f1a440cc0b0 msgr2=0x7f1a440cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:51.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.236+0000 7f1a49bff700 1 -- 172.21.15.31:0/1086880244 shutdown_connections 2023-10-06T05:18:51.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.237+0000 7f1a49bff700 1 -- 172.21.15.31:0/1086880244 wait complete. 2023-10-06T05:18:51.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.238+0000 7f1a49bff700 1 Processor -- start 2023-10-06T05:18:51.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.238+0000 7f1a49bff700 1 -- start start 2023-10-06T05:18:51.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.239+0000 7f1a49bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a440aee70 0x7f1a441552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.239+0000 7f1a49bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a440c7b60 0x7f1a440c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.239+0000 7f1a49bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440c7f30 0x7f1a4415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.239+0000 7f1a49bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44155800 con 0x7f1a440c7f30 2023-10-06T05:18:51.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.239+0000 7f1a49bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4413c910 con 0x7f1a440aee70 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.239+0000 7f1a49bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440c0ef0 con 0x7f1a440c7b60 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.239+0000 7f1a43fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440c7f30 0x7f1a4415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.239+0000 7f1a43fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440c7f30 0x7f1a4415ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34056/0 (socket says 172.21.15.31:34056) 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.239+0000 7f1a43fff700 1 -- 172.21.15.31:0/380125489 learned_addr learned my addr 172.21.15.31:0/380125489 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a437fe700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a440aee70 0x7f1a441552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a42ffd700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a440c7b60 0x7f1a440c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a43fff700 1 -- 172.21.15.31:0/380125489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a440c7b60 msgr2=0x7f1a440c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a43fff700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a440c7b60 0x7f1a440c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a43fff700 1 -- 172.21.15.31:0/380125489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a440aee70 msgr2=0x7f1a441552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a43fff700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a440aee70 0x7f1a441552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a43fff700 1 -- 172.21.15.31:0/380125489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c009ae0 con 0x7f1a440c7f30 2023-10-06T05:18:51.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a42ffd700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a440c7b60 0x7f1a440c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:51.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a43fff700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440c7f30 0x7f1a4415ae30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009880 tx=0x7f1a3c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:51.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a2b7fe700 1 -- 172.21.15.31:0/380125489 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00be60 con 0x7f1a440c7f30 2023-10-06T05:18:51.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a2b7fe700 1 -- 172.21.15.31:0/380125489 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c026460 con 0x7f1a440c7f30 2023-10-06T05:18:51.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a2b7fe700 1 -- 172.21.15.31:0/380125489 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c026820 con 0x7f1a440c7f30 2023-10-06T05:18:51.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a49bff700 1 -- 172.21.15.31:0/380125489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a440c1190 con 0x7f1a440c7f30 2023-10-06T05:18:51.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.240+0000 7f1a49bff700 1 -- 172.21.15.31:0/380125489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a440c1750 con 0x7f1a440c7f30 2023-10-06T05:18:51.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.241+0000 7f1a49bff700 1 -- 172.21.15.31:0/380125489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a440c9a20 con 0x7f1a440c7f30 2023-10-06T05:18:51.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.242+0000 7f1a2b7fe700 1 -- 172.21.15.31:0/380125489 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f1a3c003040 con 0x7f1a440c7f30 2023-10-06T05:18:51.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.243+0000 7f1a2b7fe700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1a300670e0 0x7f1a30069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:51.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.243+0000 7f1a2b7fe700 1 -- 172.21.15.31:0/380125489 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f1a3c086620 con 0x7f1a440c7f30 2023-10-06T05:18:51.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.243+0000 7f1a437fe700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1a300670e0 0x7f1a30069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:51.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.244+0000 7f1a437fe700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1a300670e0 0x7f1a30069590 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1a34006aa0 tx=0x7f1a34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:51.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.248+0000 7f1a2b7fe700 1 -- 172.21.15.31:0/380125489 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a3c053420 con 0x7f1a440c7f30 2023-10-06T05:18:51.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.493+0000 7f1a49bff700 1 -- 172.21.15.31:0/380125489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1a441567e0 con 0x7f1a440c7f30 2023-10-06T05:18:51.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.494+0000 7f1a2b7fe700 1 -- 172.21.15.31:0/380125489 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1a3c056a40 con 0x7f1a440c7f30 2023-10-06T05:18:51.495 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:18:51.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 -- 172.21.15.31:0/380125489 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1a300670e0 msgr2=0x7f1a30069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1a300670e0 0x7f1a30069590 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1a34006aa0 tx=0x7f1a34008040 comp rx=0 tx=0).stop 2023-10-06T05:18:51.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 -- 172.21.15.31:0/380125489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440c7f30 msgr2=0x7f1a4415ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:51.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440c7f30 0x7f1a4415ae30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009880 tx=0x7f1a3c018000 comp rx=0 tx=0).stop 2023-10-06T05:18:51.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 -- 172.21.15.31:0/380125489 shutdown_connections 2023-10-06T05:18:51.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1a300670e0 0x7f1a30069590 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a440c7f30 0x7f1a4415ae30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a440aee70 0x7f1a441552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 --2- 172.21.15.31:0/380125489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a440c7b60 0x7f1a440c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:51.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 -- 172.21.15.31:0/380125489 >> 172.21.15.31:0/380125489 conn(0x7f1a440cc0b0 msgr2=0x7f1a440aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:51.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 -- 172.21.15.31:0/380125489 shutdown_connections 2023-10-06T05:18:51.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:51.496+0000 7f1a49bff700 1 -- 172.21.15.31:0/380125489 wait complete. 2023-10-06T05:18:51.516 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:18:51.516 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:18:51.516 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:18:51.516 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:18:51.516 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:51.516 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:18:51.516 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:18:51.517 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:18:51.517 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:51.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:51.517 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:18:51.565 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:18:51.565 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:18:51.565 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:18:51.565 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:51.566 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:18:51.566 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:51.566 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:51.566 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:51.566 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:18:51.566 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:51.566 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:51.567 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:18:51.567 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:18:51.567 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:51.567 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:18:51.567 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:18:51.567 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42183sec preferred_lft 42183sec 2023-10-06T05:18:51.567 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:18:51.567 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:51.568 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:51.568 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:51.568 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:18:51.568 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:18:51.568 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:51.568 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:18:51.568 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:51.569 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:18:51.569 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:51.569 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:51.569 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:18:51.569 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:18:51.569 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:51.569 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:51.569 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:18:51.570 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:18:51.570 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:51.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:51.570 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:18:51.638 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:18:51.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:51.642 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:18:51.692 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:18:51.692 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:18:51.692 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:18:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:18:51.693 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:18:51.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:51.693 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:18:51.694 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:18:51.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:18:51.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:18:51.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:18:51.694 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:18:51.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:51.866 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:18:51.913 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:18:51.913 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:18:51.913 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:51.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:51.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:18:51.985 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:51.986 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:52.031 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:18:52.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:52.032 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:52.119 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:18:52.119 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:18:52.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:52.260 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:52.260 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:18:52.286 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:52.287 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:18:52.287 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:18:52.287 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:18:52.287 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:18:52.290 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:18:52.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:52.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:52.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:18:52.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.555+0000 7fdeb25ff700 1 Processor -- start 2023-10-06T05:18:52.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.555+0000 7fdeb25ff700 1 -- start start 2023-10-06T05:18:52.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.557+0000 7fdeb25ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac1596d0 0x7fdeac159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:52.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.557+0000 7fdeb25ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdeac0c16e0 0x7fdeac0c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:52.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.557+0000 7fdeb25ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac067f70 con 0x7fdeac0c6f80 2023-10-06T05:18:52.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.557+0000 7fdeb25ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac06b040 con 0x7fdeac0c16e0 2023-10-06T05:18:52.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.557+0000 7fdeb25ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0c20c0 con 0x7fdeac1596d0 2023-10-06T05:18:52.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.557+0000 7fdeab7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdeac0c6f80 0x7fdeac06a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59500/0 (socket says 172.21.15.31:59500) 2023-10-06T05:18:52.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.557+0000 7fdeab7fe700 1 -- 172.21.15.31:0/453000290 learned_addr learned my addr 172.21.15.31:0/453000290 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.557+0000 7fdeb0b9c700 1 --2- 172.21.15.31:0/453000290 >> v2:172.21.15.62:3300/0 conn(0x7fdeac0c16e0 0x7fdeac0c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.557+0000 7fdeabfff700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac1596d0 0x7fdeac159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.557+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318110625 0 0) 0x7fdeac067f70 con 0x7fdeac0c6f80 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde94003570 con 0x7fdeac0c6f80 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 826870244 0 0) 0x7fde94003570 con 0x7fdeac0c6f80 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac1596d0 msgr2=0x7fdeac159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac1596d0 0x7fdeac159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 >> v2:172.21.15.62:3300/0 conn(0x7fdeac0c16e0 msgr2=0x7fdeac0c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 --2- 172.21.15.31:0/453000290 >> v2:172.21.15.62:3300/0 conn(0x7fdeac0c16e0 0x7fdeac0c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac06e040 con 0x7fdeac0c6f80 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fdea0001470 con 0x7fdeac0c6f80 2023-10-06T05:18:52.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 >> v1:172.21.15.31:6789/0 conn(0x7fdeac0c6f80 legacy=0x7fdeac06a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde94003ae0 0x7fde94003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac1596d0 0x7fde94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde94004d50 0x7fde940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac06b040 con 0x7fde94003ae0 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0c20c0 con 0x7fde94004d50 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9400bb10 con 0x7fdeac1596d0 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeb0b9c700 1 --2- 172.21.15.31:0/453000290 >> v2:172.21.15.62:3300/0 conn(0x7fdeac0c16e0 0x7fdeac0c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeabfff700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde94003ae0 0x7fde94003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeab7fe700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac1596d0 0x7fde94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.558+0000 7fdeb0b9c700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde94004d50 0x7fde940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeabfff700 1 -- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac1596d0 msgr2=0x7fde94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeabfff700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac1596d0 0x7fde94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeabfff700 1 -- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde94004d50 msgr2=0x7fde940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeabfff700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde94004d50 0x7fde940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeabfff700 1 -- 172.21.15.31:0/453000290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde94003570 con 0x7fde94003ae0 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeab7fe700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac1596d0 0x7fde94004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeb0b9c700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde94004d50 0x7fde940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeabfff700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde94003ae0 0x7fde94003ef0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009c30 tx=0x7fde9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99409bfc5ccd9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c00e710 con 0x7fde94003ae0 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c00e870 con 0x7fde94003ae0 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.559+0000 7fdeaaffd700 1 -- 172.21.15.31:0/453000290 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c00ec30 con 0x7fde94003ae0 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.560+0000 7fdeb25ff700 1 -- 172.21.15.31:0/453000290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde94003ae0 msgr2=0x7fde94003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.560+0000 7fdeb25ff700 1 --2- 172.21.15.31:0/453000290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde94003ae0 0x7fde94003ef0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009c30 tx=0x7fde9c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.560+0000 7fdeb0b9c700 1 -- 172.21.15.31:0/453000290 reap_dead start 2023-10-06T05:18:52.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.560+0000 7fdeb25ff700 1 -- 172.21.15.31:0/453000290 shutdown_connections 2023-10-06T05:18:52.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.560+0000 7fdeb25ff700 1 -- 172.21.15.31:0/453000290 >> 172.21.15.31:0/453000290 conn(0x7fdeac0c9e80 msgr2=0x7fdeac06a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:52.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.560+0000 7fdeb25ff700 1 -- 172.21.15.31:0/453000290 shutdown_connections 2023-10-06T05:18:52.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.560+0000 7fdeb25ff700 1 -- 172.21.15.31:0/453000290 wait complete. 2023-10-06T05:18:52.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.561+0000 7fdeb25ff700 1 Processor -- start 2023-10-06T05:18:52.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.562+0000 7fdeb25ff700 1 -- start start 2023-10-06T05:18:52.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.562+0000 7fdeb25ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde94003ae0 0x7fdeac14fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:52.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.562+0000 7fdeb25ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac0c39f0 0x7fdeac0c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:52.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb25ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeac0c57f0 0x7fdeac0c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:52.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb25ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0b0480 con 0x7fdeac0c57f0 2023-10-06T05:18:52.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb25ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0b0760 con 0x7fde94003ae0 2023-10-06T05:18:52.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb25ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0c43a0 con 0x7fdeac0c39f0 2023-10-06T05:18:52.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb0b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeac0c57f0 0x7fdeac0c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:52.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb0b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeac0c57f0 0x7fdeac0c5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34076/0 (socket says 172.21.15.31:34076) 2023-10-06T05:18:52.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb0b9c700 1 -- 172.21.15.31:0/1649771993 learned_addr learned my addr 172.21.15.31:0/1649771993 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:52.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeabfff700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde94003ae0 0x7fdeac14fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:52.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeab7fe700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac0c39f0 0x7fdeac0c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:52.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb0b9c700 1 -- 172.21.15.31:0/1649771993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac0c39f0 msgr2=0x7fdeac0c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:52.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb0b9c700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac0c39f0 0x7fdeac0c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb0b9c700 1 -- 172.21.15.31:0/1649771993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde94003ae0 msgr2=0x7fdeac14fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:52.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb0b9c700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde94003ae0 0x7fdeac14fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.563+0000 7fdeb0b9c700 1 -- 172.21.15.31:0/1649771993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac1556b0 con 0x7fdeac0c57f0 2023-10-06T05:18:52.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.564+0000 7fdeb0b9c700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeac0c57f0 0x7fdeac0c5c60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0b2920 tx=0x7fdea400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:52.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.564+0000 7fde93fff700 1 -- 172.21.15.31:0/1649771993 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400e070 con 0x7fdeac0c57f0 2023-10-06T05:18:52.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.564+0000 7fde93fff700 1 -- 172.21.15.31:0/1649771993 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea4011de0 con 0x7fdeac0c57f0 2023-10-06T05:18:52.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.564+0000 7fde93fff700 1 -- 172.21.15.31:0/1649771993 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea40166c0 con 0x7fdeac0c57f0 2023-10-06T05:18:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.564+0000 7fdeb25ff700 1 -- 172.21.15.31:0/1649771993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac155990 con 0x7fdeac0c57f0 2023-10-06T05:18:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.564+0000 7fdeb25ff700 1 -- 172.21.15.31:0/1649771993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac0c61a0 con 0x7fdeac0c57f0 2023-10-06T05:18:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.565+0000 7fdeb25ff700 1 -- 172.21.15.31:0/1649771993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac0b08f0 con 0x7fdeac0c57f0 2023-10-06T05:18:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.566+0000 7fde93fff700 1 -- 172.21.15.31:0/1649771993 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fdea4016ab0 con 0x7fdeac0c57f0 2023-10-06T05:18:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.567+0000 7fde93fff700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fde980670e0 0x7fde98069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.567+0000 7fde93fff700 1 -- 172.21.15.31:0/1649771993 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fdea4085760 con 0x7fdeac0c57f0 2023-10-06T05:18:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.567+0000 7fdeabfff700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fde980670e0 0x7fde98069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.568+0000 7fdeabfff700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fde980670e0 0x7fde98069590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0091e0 tx=0x7fde9c01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:52.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.571+0000 7fde93fff700 1 -- 172.21.15.31:0/1649771993 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdea4052400 con 0x7fdeac0c57f0 2023-10-06T05:18:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.818+0000 7fdeb25ff700 1 -- 172.21.15.31:0/1649771993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdeac04c380 con 0x7fdeac0c57f0 2023-10-06T05:18:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.818+0000 7fde93fff700 1 -- 172.21.15.31:0/1649771993 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdea4083e60 con 0x7fdeac0c57f0 2023-10-06T05:18:52.820 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:18:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.820+0000 7fdeb25ff700 1 -- 172.21.15.31:0/1649771993 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fde980670e0 msgr2=0x7fde98069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.820+0000 7fdeb25ff700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fde980670e0 0x7fde98069590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0091e0 tx=0x7fde9c01a000 comp rx=0 tx=0).stop 2023-10-06T05:18:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.820+0000 7fdeb25ff700 1 -- 172.21.15.31:0/1649771993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeac0c57f0 msgr2=0x7fdeac0c5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.820+0000 7fdeb25ff700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeac0c57f0 0x7fdeac0c5c60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0b2920 tx=0x7fdea400b040 comp rx=0 tx=0).stop 2023-10-06T05:18:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.820+0000 7fdeb25ff700 1 -- 172.21.15.31:0/1649771993 shutdown_connections 2023-10-06T05:18:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.820+0000 7fdeb25ff700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fde980670e0 0x7fde98069590 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.820+0000 7fdeb25ff700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeac0c57f0 0x7fdeac0c5c60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.820+0000 7fdeb25ff700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde94003ae0 0x7fdeac14fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.820+0000 7fdeb25ff700 1 --2- 172.21.15.31:0/1649771993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeac0c39f0 0x7fdeac0c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.820+0000 7fdeb25ff700 1 -- 172.21.15.31:0/1649771993 >> 172.21.15.31:0/1649771993 conn(0x7fdeac0c9e80 msgr2=0x7fdeac06a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.821+0000 7fdeb25ff700 1 -- 172.21.15.31:0/1649771993 shutdown_connections 2023-10-06T05:18:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:52.821+0000 7fdeb25ff700 1 -- 172.21.15.31:0/1649771993 wait complete. 2023-10-06T05:18:52.840 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:18:52.840 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:18:52.840 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:18:52.840 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:18:52.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:18:52.841 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:18:52.841 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:18:52.841 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:18:52.841 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:18:52.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:52.842 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:18:52.895 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:18:52.895 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:18:52.895 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:18:52.895 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:52.895 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:18:52.895 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:52.895 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:52.895 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:52.895 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42182sec preferred_lft 42182sec 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:18:52.896 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:52.897 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:18:52.897 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:18:52.897 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:18:52.897 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:52.897 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:18:52.897 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:52.897 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:52.897 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:18:52.897 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:18:52.898 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:52.898 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:18:52.898 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:18:52.898 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:18:52.898 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:18:52.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:52.899 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:18:52.970 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:18:52.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:52.973 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:18:53.022 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:18:53.022 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:18:53.023 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:18:53.023 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:18:53.023 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:18:53.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:53.023 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:18:53.023 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:18:53.023 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:18:53.023 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:18:53.024 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:18:53.024 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:18:53.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:53.195 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:18:53.242 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:18:53.242 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:18:53.242 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:18:53.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:53.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:18:53.314 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:53.314 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:18:53.360 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:18:53.361 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:18:53.361 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:18:53.446 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:18:53.446 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:18:53.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:53.585 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:53.585 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:18:53.611 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:18:53.611 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:18:53.611 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:18:53.611 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:18:53.612 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:18:53.615 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:18:53.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:53.616 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:18:53.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:18:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.895+0000 7f5a81bff700 1 Processor -- start 2023-10-06T05:18:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.895+0000 7f5a81bff700 1 -- start start 2023-10-06T05:18:53.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a81bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a7c0ad670 0x7f5a7c06ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:53.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a81bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5a7c06a630 0x7f5a7c06aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:53.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a81bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7c06e600 con 0x7f5a7c156300 2023-10-06T05:18:53.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a81bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7c0adae0 con 0x7f5a7c0ad670 2023-10-06T05:18:53.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a81bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7c06aff0 con 0x7f5a7c06a630 2023-10-06T05:18:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a7affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a7c0ad670 0x7f5a7c06ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a7affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a7c0ad670 0x7f5a7c06ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34092/0 (socket says 172.21.15.31:34092) 2023-10-06T05:18:53.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a7affd700 1 -- 172.21.15.31:0/867952580 learned_addr learned my addr 172.21.15.31:0/867952580 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:53.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a7bfff700 1 --2- 172.21.15.31:0/867952580 >> v2:172.21.15.62:3300/0 conn(0x7f5a7c06a630 0x7f5a7c06aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:53.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a7affd700 1 -- 172.21.15.31:0/867952580 >> v2:172.21.15.62:3300/0 conn(0x7f5a7c06a630 msgr2=0x7f5a7c06aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:53.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a7affd700 1 --2- 172.21.15.31:0/867952580 >> v2:172.21.15.62:3300/0 conn(0x7f5a7c06a630 0x7f5a7c06aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:53.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a7affd700 1 -- 172.21.15.31:0/867952580 >> v1:172.21.15.62:6789/0 conn(0x7f5a7c156300 legacy=0x7f5a7c06d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:53.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.897+0000 7f5a7affd700 1 -- 172.21.15.31:0/867952580 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c06d9f0 con 0x7f5a7c0ad670 2023-10-06T05:18:53.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.898+0000 7f5a7affd700 1 --2- 172.21.15.31:0/867952580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 0x7f5a7c06ea90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c06d110 tx=0x7f5a70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74e25acd9e1e8c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:53.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.898+0000 7f5a7a7fc700 1 -- 172.21.15.31:0/867952580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000b860 con 0x7f5a7c0ad670 2023-10-06T05:18:53.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.898+0000 7f5a7a7fc700 1 -- 172.21.15.31:0/867952580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7000b9c0 con 0x7f5a7c0ad670 2023-10-06T05:18:53.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.898+0000 7f5a7a7fc700 1 -- 172.21.15.31:0/867952580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000bd60 con 0x7f5a7c0ad670 2023-10-06T05:18:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.899+0000 7f5a81bff700 1 -- 172.21.15.31:0/867952580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 msgr2=0x7f5a7c06ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.899+0000 7f5a81bff700 1 --2- 172.21.15.31:0/867952580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 0x7f5a7c06ea90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c06d110 tx=0x7f5a70009250 comp rx=0 tx=0).stop 2023-10-06T05:18:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.899+0000 7f5a81bff700 1 -- 172.21.15.31:0/867952580 shutdown_connections 2023-10-06T05:18:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.899+0000 7f5a81bff700 1 --2- 172.21.15.31:0/867952580 >> v2:172.21.15.62:3300/0 conn(0x7f5a7c06a630 0x7f5a7c06aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.899+0000 7f5a81bff700 1 --2- 172.21.15.31:0/867952580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 0x7f5a7c06ea90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.899+0000 7f5a81bff700 1 -- 172.21.15.31:0/867952580 >> 172.21.15.31:0/867952580 conn(0x7f5a7c068030 msgr2=0x7f5a7c0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.899+0000 7f5a81bff700 1 -- 172.21.15.31:0/867952580 shutdown_connections 2023-10-06T05:18:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.899+0000 7f5a81bff700 1 -- 172.21.15.31:0/867952580 wait complete. 2023-10-06T05:18:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.901+0000 7f5a81bff700 1 Processor -- start 2023-10-06T05:18:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.901+0000 7f5a81bff700 1 -- start start 2023-10-06T05:18:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.902+0000 7f5a81bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7c06a630 0x7f5a7c167590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.902+0000 7f5a81bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 0x7f5a7c154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.902+0000 7f5a81bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a7c156300 0x7f5a7c15c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.902+0000 7f5a81bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c167ad0 con 0x7f5a7c0ad670 2023-10-06T05:18:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.902+0000 7f5a81bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0c3470 con 0x7f5a7c06a630 2023-10-06T05:18:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.902+0000 7f5a81bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0c3750 con 0x7f5a7c156300 2023-10-06T05:18:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 0x7f5a7c154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 0x7f5a7c154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34096/0 (socket says 172.21.15.31:34096) 2023-10-06T05:18:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7affd700 1 -- 172.21.15.31:0/2277325791 learned_addr learned my addr 172.21.15.31:0/2277325791 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7bfff700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a7c156300 0x7f5a7c15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7affd700 1 -- 172.21.15.31:0/2277325791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a7c156300 msgr2=0x7f5a7c15c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7affd700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a7c156300 0x7f5a7c15c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7b7fe700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7c06a630 0x7f5a7c167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7affd700 1 -- 172.21.15.31:0/2277325791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7c06a630 msgr2=0x7f5a7c167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7affd700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7c06a630 0x7f5a7c167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7affd700 1 -- 172.21.15.31:0/2277325791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a700094f0 con 0x7f5a7c0ad670 2023-10-06T05:18:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7bfff700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a7c156300 0x7f5a7c15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7b7fe700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7c06a630 0x7f5a7c167590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a7affd700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 0x7f5a7c154510 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c06cf30 tx=0x7f5a70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.903+0000 7f5a637fe700 1 -- 172.21.15.31:0/2277325791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70011e60 con 0x7f5a7c0ad670 2023-10-06T05:18:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.904+0000 7f5a637fe700 1 -- 172.21.15.31:0/2277325791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7001a630 con 0x7f5a7c0ad670 2023-10-06T05:18:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.904+0000 7f5a81bff700 1 -- 172.21.15.31:0/2277325791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7c0c3a30 con 0x7f5a7c0ad670 2023-10-06T05:18:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.904+0000 7f5a637fe700 1 -- 172.21.15.31:0/2277325791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7001a9f0 con 0x7f5a7c0ad670 2023-10-06T05:18:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.904+0000 7f5a81bff700 1 -- 172.21.15.31:0/2277325791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7c0c3ff0 con 0x7f5a7c0ad670 2023-10-06T05:18:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.905+0000 7f5a81bff700 1 -- 172.21.15.31:0/2277325791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7c0c4210 con 0x7f5a7c0ad670 2023-10-06T05:18:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.906+0000 7f5a637fe700 1 -- 172.21.15.31:0/2277325791 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f5a7002d440 con 0x7f5a7c0ad670 2023-10-06T05:18:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.906+0000 7f5a637fe700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5a680670e0 0x7f5a68069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.907+0000 7f5a637fe700 1 -- 172.21.15.31:0/2277325791 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f5a700866b0 con 0x7f5a7c0ad670 2023-10-06T05:18:53.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.907+0000 7f5a7b7fe700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5a680670e0 0x7f5a68069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:53.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.907+0000 7f5a7b7fe700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5a680670e0 0x7f5a68069590 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0017d0 tx=0x7f5a6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:53.911+0000 7f5a637fe700 1 -- 172.21.15.31:0/2277325791 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a70053460 con 0x7f5a7c0ad670 2023-10-06T05:18:54.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:54.170+0000 7f5a81bff700 1 -- 172.21.15.31:0/2277325791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5a7c04c380 con 0x7f5a7c0ad670 2023-10-06T05:18:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.172+0000 7f5a637fe700 1 -- 172.21.15.31:0/2277325791 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 136+0+0 (secure 0 0 0) 0x7f5a70056a80 con 0x7f5a7c0ad670 2023-10-06T05:18:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 -- 172.21.15.31:0/2277325791 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5a680670e0 msgr2=0x7f5a68069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5a680670e0 0x7f5a68069590 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0017d0 tx=0x7f5a6c009040 comp rx=0 tx=0).stop 2023-10-06T05:18:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 -- 172.21.15.31:0/2277325791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 msgr2=0x7f5a7c154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 0x7f5a7c154510 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c06cf30 tx=0x7f5a70017000 comp rx=0 tx=0).stop 2023-10-06T05:18:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 -- 172.21.15.31:0/2277325791 shutdown_connections 2023-10-06T05:18:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a7c156300 0x7f5a7c15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7c06a630 0x7f5a7c167590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5a680670e0 0x7f5a68069590 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 --2- 172.21.15.31:0/2277325791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a7c0ad670 0x7f5a7c154510 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 -- 172.21.15.31:0/2277325791 >> 172.21.15.31:0/2277325791 conn(0x7f5a7c068030 msgr2=0x7f5a7c1586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 -- 172.21.15.31:0/2277325791 shutdown_connections 2023-10-06T05:18:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.175+0000 7f5a81bff700 1 -- 172.21.15.31:0/2277325791 wait complete. 2023-10-06T05:18:55.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:18:55.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.396+0000 7f2a331ff700 1 Processor -- start 2023-10-06T05:18:55.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.397+0000 7f2a331ff700 1 -- start start 2023-10-06T05:18:55.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.398+0000 7f2a331ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2a2c13a5b0 0x7f2a2c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.398+0000 7f2a331ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 0x7f2a2c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.398+0000 7f2a331ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c06ddb0 con 0x7f2a2c13a1e0 2023-10-06T05:18:55.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.398+0000 7f2a331ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c0c71b0 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.398+0000 7f2a331ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0c7490 con 0x7f2a2c06d980 2023-10-06T05:18:55.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.398+0000 7f2a30f9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2a2c13a1e0 0x7f2a2c0c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59516/0 (socket says 172.21.15.31:59516) 2023-10-06T05:18:55.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.398+0000 7f2a30f9b700 1 -- 172.21.15.31:0/2273444568 learned_addr learned my addr 172.21.15.31:0/2273444568 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:55.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.398+0000 7f2a2bfff700 1 --2- 172.21.15.31:0/2273444568 >> v2:172.21.15.62:3300/0 conn(0x7f2a2c13a5b0 0x7f2a2c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.398+0000 7f2a3179c700 1 --2- 172.21.15.31:0/2273444568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 0x7f2a2c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.399+0000 7f2a2b7fe700 1 -- 172.21.15.31:0/2273444568 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369333050 0 0) 0x7f2a2c06ddb0 con 0x7f2a2c13a1e0 2023-10-06T05:18:55.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.399+0000 7f2a2b7fe700 1 -- 172.21.15.31:0/2273444568 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a1c003570 con 0x7f2a2c13a1e0 2023-10-06T05:18:55.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.399+0000 7f2a3179c700 1 -- 172.21.15.31:0/2273444568 >> v2:172.21.15.62:3300/0 conn(0x7f2a2c13a5b0 msgr2=0x7f2a2c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:55.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.399+0000 7f2a3179c700 1 --2- 172.21.15.31:0/2273444568 >> v2:172.21.15.62:3300/0 conn(0x7f2a2c13a5b0 0x7f2a2c06d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.399+0000 7f2a2b7fe700 1 -- 172.21.15.31:0/2273444568 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1578906281 0 0) 0x7f2a1c003570 con 0x7f2a2c13a1e0 2023-10-06T05:18:55.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.399+0000 7f2a3179c700 1 -- 172.21.15.31:0/2273444568 >> v1:172.21.15.31:6789/0 conn(0x7f2a2c13a1e0 legacy=0x7f2a2c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.399+0000 7f2a3179c700 1 -- 172.21.15.31:0/2273444568 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c0c7770 con 0x7f2a2c06d980 2023-10-06T05:18:55.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.399+0000 7f2a3179c700 1 --2- 172.21.15.31:0/2273444568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 0x7f2a2c069560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009e60 tx=0x7f2a24009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0e1a1fa971ec489 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:55.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.400+0000 7f2a2b7fe700 1 -- 172.21.15.31:0/2273444568 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400e070 con 0x7f2a2c06d980 2023-10-06T05:18:55.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.400+0000 7f2a2b7fe700 1 -- 172.21.15.31:0/2273444568 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2400b7a0 con 0x7f2a2c06d980 2023-10-06T05:18:55.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.400+0000 7f2a2b7fe700 1 -- 172.21.15.31:0/2273444568 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a200015f0 con 0x7f2a2c06d980 2023-10-06T05:18:55.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.401+0000 7f2a331ff700 1 -- 172.21.15.31:0/2273444568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 msgr2=0x7f2a2c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.401+0000 7f2a331ff700 1 --2- 172.21.15.31:0/2273444568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 0x7f2a2c069560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009e60 tx=0x7f2a24009780 comp rx=0 tx=0).stop 2023-10-06T05:18:55.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.401+0000 7f2a331ff700 1 -- 172.21.15.31:0/2273444568 shutdown_connections 2023-10-06T05:18:55.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.401+0000 7f2a331ff700 1 --2- 172.21.15.31:0/2273444568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 0x7f2a2c069560 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.401+0000 7f2a331ff700 1 --2- 172.21.15.31:0/2273444568 >> v2:172.21.15.62:3300/0 conn(0x7f2a2c13a5b0 0x7f2a2c06d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.401+0000 7f2a331ff700 1 -- 172.21.15.31:0/2273444568 >> 172.21.15.31:0/2273444568 conn(0x7f2a2c0cc0b0 msgr2=0x7f2a2c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:55.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.401+0000 7f2a331ff700 1 -- 172.21.15.31:0/2273444568 shutdown_connections 2023-10-06T05:18:55.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.401+0000 7f2a331ff700 1 -- 172.21.15.31:0/2273444568 wait complete. 2023-10-06T05:18:55.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.402+0000 7f2a331ff700 1 Processor -- start 2023-10-06T05:18:55.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.402+0000 7f2a331ff700 1 -- start start 2023-10-06T05:18:55.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.403+0000 7f2a331ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 0x7f2a2c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.403+0000 7f2a331ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a2c13a1e0 0x7f2a2c16cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.403+0000 7f2a331ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a2c13a5b0 0x7f2a2c16d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.403+0000 7f2a331ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c151890 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.403+0000 7f2a331ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c1515b0 con 0x7f2a2c13a1e0 2023-10-06T05:18:55.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.403+0000 7f2a331ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0c9d20 con 0x7f2a2c06d980 2023-10-06T05:18:55.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a3179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a2c13a5b0 0x7f2a2c16d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a3179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a2c13a5b0 0x7f2a2c16d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34106/0 (socket says 172.21.15.31:34106) 2023-10-06T05:18:55.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a3179c700 1 -- 172.21.15.31:0/3688761909 learned_addr learned my addr 172.21.15.31:0/3688761909 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:55.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a30f9b700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 0x7f2a2c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a2bfff700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a2c13a1e0 0x7f2a2c16cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a3179c700 1 -- 172.21.15.31:0/3688761909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 msgr2=0x7f2a2c14ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a3179c700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 0x7f2a2c14ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a3179c700 1 -- 172.21.15.31:0/3688761909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a2c13a1e0 msgr2=0x7f2a2c16cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a3179c700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a2c13a1e0 0x7f2a2c16cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a3179c700 1 -- 172.21.15.31:0/3688761909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24009a20 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a2bfff700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a2c13a1e0 0x7f2a2c16cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:55.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.404+0000 7f2a3179c700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a2c13a5b0 0x7f2a2c16d0d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0c75d0 tx=0x7f2a24018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:55.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.405+0000 7f2a28ff9700 1 -- 172.21.15.31:0/3688761909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24012c70 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.405+0000 7f2a28ff9700 1 -- 172.21.15.31:0/3688761909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a24012e10 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.405+0000 7f2a331ff700 1 -- 172.21.15.31:0/3688761909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c0ca000 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.405+0000 7f2a28ff9700 1 -- 172.21.15.31:0/3688761909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2401d680 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.405+0000 7f2a331ff700 1 -- 172.21.15.31:0/3688761909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c0ae210 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.406+0000 7f2a331ff700 1 -- 172.21.15.31:0/3688761909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c0ca520 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.407+0000 7f2a28ff9700 1 -- 172.21.15.31:0/3688761909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2a24024070 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.408+0000 7f2a28ff9700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a100670e0 0x7f2a10069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.408+0000 7f2a28ff9700 1 -- 172.21.15.31:0/3688761909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f2a24086c70 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.408+0000 7f2a30f9b700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a100670e0 0x7f2a10069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.409+0000 7f2a30f9b700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a100670e0 0x7f2a10069590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2a20006d80 tx=0x7f2a200012c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:55.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.412+0000 7f2a28ff9700 1 -- 172.21.15.31:0/3688761909 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a24053a20 con 0x7f2a2c13a5b0 2023-10-06T05:18:55.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.650+0000 7f2a331ff700 1 -- 172.21.15.31:0/3688761909 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a2c04c380 con 0x7f2a100670e0 2023-10-06T05:18:55.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.651+0000 7f2a28ff9700 1 -- 172.21.15.31:0/3688761909 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2a2c04c380 con 0x7f2a100670e0 2023-10-06T05:18:55.653 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:18:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:18:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:18:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:18:55.654 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:18:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 -- 172.21.15.31:0/3688761909 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a100670e0 msgr2=0x7f2a10069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a100670e0 0x7f2a10069590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2a20006d80 tx=0x7f2a200012c0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 -- 172.21.15.31:0/3688761909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a2c13a5b0 msgr2=0x7f2a2c16d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a2c13a5b0 0x7f2a2c16d0d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0c75d0 tx=0x7f2a24018000 comp rx=0 tx=0).stop 2023-10-06T05:18:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 -- 172.21.15.31:0/3688761909 shutdown_connections 2023-10-06T05:18:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2a100670e0 0x7f2a10069590 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a2c13a5b0 0x7f2a2c16d0d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a2c06d980 0x7f2a2c14ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 --2- 172.21.15.31:0/3688761909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a2c13a1e0 0x7f2a2c16cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 -- 172.21.15.31:0/3688761909 >> 172.21.15.31:0/3688761909 conn(0x7f2a2c0cc0b0 msgr2=0x7f2a2c0af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 -- 172.21.15.31:0/3688761909 shutdown_connections 2023-10-06T05:18:55.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.654+0000 7f2a331ff700 1 -- 172.21.15.31:0/3688761909 wait complete. 2023-10-06T05:18:55.674 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:18:55.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.871+0000 7fe365fff700 1 Processor -- start 2023-10-06T05:18:55.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.871+0000 7fe365fff700 1 -- start start 2023-10-06T05:18:55.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.872+0000 7fe365fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe3600aef00 0x7fe3600c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.872+0000 7fe365fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe360068250 con 0x7fe3600c7190 2023-10-06T05:18:55.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.872+0000 7fe365fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36006ac80 con 0x7fe3600c7560 2023-10-06T05:18:55.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.872+0000 7fe365fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe360067f20 con 0x7fe3600aef00 2023-10-06T05:18:55.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.873+0000 7fe35effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe3600aef00 0x7fe3600c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.873+0000 7fe35effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe3600aef00 0x7fe3600c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34116/0 (socket says 172.21.15.31:34116) 2023-10-06T05:18:55.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.873+0000 7fe35effd700 1 -- 172.21.15.31:0/1901803952 learned_addr learned my addr 172.21.15.31:0/1901803952 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.873+0000 7fe35effd700 1 -- 172.21.15.31:0/1901803952 >> v1:172.21.15.62:6789/0 conn(0x7fe3600c7560 legacy=0x7fe3600686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.873+0000 7fe35effd700 1 -- 172.21.15.31:0/1901803952 >> v1:172.21.15.31:6789/0 conn(0x7fe3600c7190 legacy=0x7fe3600af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.873+0000 7fe35e7fc700 1 -- 172.21.15.31:0/1901803952 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697578457 0 0) 0x7fe360068250 con 0x7fe3600c7190 2023-10-06T05:18:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.873+0000 7fe35effd700 1 -- 172.21.15.31:0/1901803952 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36006aed0 con 0x7fe3600aef00 2023-10-06T05:18:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.873+0000 7fe35effd700 1 --2- 172.21.15.31:0/1901803952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600aef00 0x7fe3600c9d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe3540099b0 tx=0x7fe354009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8f3348b2a54934 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.874+0000 7fe35e7fc700 1 -- 172.21.15.31:0/1901803952 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35400b820 con 0x7fe3600aef00 2023-10-06T05:18:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.874+0000 7fe35e7fc700 1 -- 172.21.15.31:0/1901803952 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35400b980 con 0x7fe3600aef00 2023-10-06T05:18:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.874+0000 7fe35e7fc700 1 -- 172.21.15.31:0/1901803952 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35400bd20 con 0x7fe3600aef00 2023-10-06T05:18:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.874+0000 7fe365fff700 1 -- 172.21.15.31:0/1901803952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600aef00 msgr2=0x7fe3600c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.874+0000 7fe365fff700 1 --2- 172.21.15.31:0/1901803952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600aef00 0x7fe3600c9d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe3540099b0 tx=0x7fe354009250 comp rx=0 tx=0).stop 2023-10-06T05:18:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.874+0000 7fe365fff700 1 -- 172.21.15.31:0/1901803952 shutdown_connections 2023-10-06T05:18:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.874+0000 7fe365fff700 1 --2- 172.21.15.31:0/1901803952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600aef00 0x7fe3600c9d40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.874+0000 7fe365fff700 1 -- 172.21.15.31:0/1901803952 >> 172.21.15.31:0/1901803952 conn(0x7fe3600cd3d0 msgr2=0x7fe3600c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.874+0000 7fe365fff700 1 -- 172.21.15.31:0/1901803952 shutdown_connections 2023-10-06T05:18:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.874+0000 7fe365fff700 1 -- 172.21.15.31:0/1901803952 wait complete. 2023-10-06T05:18:55.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.876+0000 7fe365fff700 1 Processor -- start 2023-10-06T05:18:55.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.877+0000 7fe365fff700 1 -- start start 2023-10-06T05:18:55.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe365fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3600aef00 0x7fe36015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe365fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3600c7190 0x7fe36015ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe365fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600c7560 0x7fe360151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe365fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600c4a50 con 0x7fe3600c7560 2023-10-06T05:18:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe365fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3600c2e20 con 0x7fe3600c7190 2023-10-06T05:18:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe365fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36015fd20 con 0x7fe3600aef00 2023-10-06T05:18:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe35ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600c7560 0x7fe360151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe35ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600c7560 0x7fe360151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34130/0 (socket says 172.21.15.31:34130) 2023-10-06T05:18:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe35ffff700 1 -- 172.21.15.31:0/2892003955 learned_addr learned my addr 172.21.15.31:0/2892003955 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe35f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3600aef00 0x7fe36015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.878+0000 7fe35effd700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3600c7190 0x7fe36015ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe35ffff700 1 -- 172.21.15.31:0/2892003955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3600aef00 msgr2=0x7fe36015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe35ffff700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3600aef00 0x7fe36015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe35ffff700 1 -- 172.21.15.31:0/2892003955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3600c7190 msgr2=0x7fe36015ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe35ffff700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3600c7190 0x7fe36015ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe35ffff700 1 -- 172.21.15.31:0/2892003955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3540094f0 con 0x7fe3600c7560 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe35effd700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3600c7190 0x7fe36015ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe35ffff700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600c7560 0x7fe360151f70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe35800c990 tx=0x7fe3580026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe3477fe700 1 -- 172.21.15.31:0/2892003955 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800ce20 con 0x7fe3600c7560 2023-10-06T05:18:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe3477fe700 1 -- 172.21.15.31:0/2892003955 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35800a760 con 0x7fe3600c7560 2023-10-06T05:18:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe365fff700 1 -- 172.21.15.31:0/2892003955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3600c3e40 con 0x7fe3600c7560 2023-10-06T05:18:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe3477fe700 1 -- 172.21.15.31:0/2892003955 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800ab20 con 0x7fe3600c7560 2023-10-06T05:18:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.879+0000 7fe365fff700 1 -- 172.21.15.31:0/2892003955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3600c4400 con 0x7fe3600c7560 2023-10-06T05:18:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.880+0000 7fe365fff700 1 -- 172.21.15.31:0/2892003955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36004c380 con 0x7fe3600c7560 2023-10-06T05:18:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.881+0000 7fe3477fe700 1 -- 172.21.15.31:0/2892003955 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fe358012420 con 0x7fe3600c7560 2023-10-06T05:18:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.882+0000 7fe3477fe700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe34c0670e0 0x7fe34c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.883+0000 7fe3477fe700 1 -- 172.21.15.31:0/2892003955 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe358084e30 con 0x7fe3600c7560 2023-10-06T05:18:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.883+0000 7fe35f7fe700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe34c0670e0 0x7fe34c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.883+0000 7fe35f7fe700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe34c0670e0 0x7fe34c069590 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe350007fd0 tx=0x7fe350007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:55.887+0000 7fe3477fe700 1 -- 172.21.15.31:0/2892003955 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe358051be0 con 0x7fe3600c7560 2023-10-06T05:18:56.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.089+0000 7fe365fff700 1 -- 172.21.15.31:0/2892003955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe3600cc6e0 con 0x7fe3600c7560 2023-10-06T05:18:56.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.089+0000 7fe3477fe700 1 -- 172.21.15.31:0/2892003955 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe358055200 con 0x7fe3600c7560 2023-10-06T05:18:56.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 -- 172.21.15.31:0/2892003955 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe34c0670e0 msgr2=0x7fe34c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe34c0670e0 0x7fe34c069590 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe350007fd0 tx=0x7fe350007d30 comp rx=0 tx=0).stop 2023-10-06T05:18:56.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 -- 172.21.15.31:0/2892003955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600c7560 msgr2=0x7fe360151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600c7560 0x7fe360151f70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe35800c990 tx=0x7fe3580026e0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 -- 172.21.15.31:0/2892003955 shutdown_connections 2023-10-06T05:18:56.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe34c0670e0 0x7fe34c069590 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3600c7560 0x7fe360151f70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3600aef00 0x7fe36015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 --2- 172.21.15.31:0/2892003955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3600c7190 0x7fe36015ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 -- 172.21.15.31:0/2892003955 >> 172.21.15.31:0/2892003955 conn(0x7fe3600cd3d0 msgr2=0x7fe3601588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:56.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 -- 172.21.15.31:0/2892003955 shutdown_connections 2023-10-06T05:18:56.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.092+0000 7fe365fff700 1 -- 172.21.15.31:0/2892003955 wait complete. 2023-10-06T05:18:56.111 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:18:56.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:56.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:18:56.198 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:18:56.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:56.198 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:18:56.245 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:18:56.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:18:56.246 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:18:56.347 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:18:56.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:18:56.347 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:18:56.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:56.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.590+0000 7fe3ea1ff700 1 Processor -- start 2023-10-06T05:18:56.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.590+0000 7fe3ea1ff700 1 -- start start 2023-10-06T05:18:56.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.591+0000 7fe3ea1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe3e40c7b60 0x7fe3e4069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.591+0000 7fe3ea1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40aee70 0x7fe3e413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.591+0000 7fe3ea1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e40ca5b0 con 0x7fe3e40c7f30 2023-10-06T05:18:56.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.591+0000 7fe3ea1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e40af2d0 con 0x7fe3e40c7b60 2023-10-06T05:18:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.591+0000 7fe3ea1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e406f740 con 0x7fe3e40aee70 2023-10-06T05:18:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.592+0000 7fe3e3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40aee70 0x7fe3e413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.592+0000 7fe3e3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40aee70 0x7fe3e413f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56850/0 (socket says 172.21.15.31:56850) 2023-10-06T05:18:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.592+0000 7fe3e3fff700 1 -- 172.21.15.31:0/1407989916 learned_addr learned my addr 172.21.15.31:0/1407989916 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.592+0000 7fe3e2ffd700 1 --1- 172.21.15.31:0/1407989916 >> v1:172.21.15.62:6789/0 conn(0x7fe3e40c7f30 0x7fe3e406ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:35994/0 (socket says 172.21.15.31:35994) 2023-10-06T05:18:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.592+0000 7fe3e37fe700 1 --2- 172.21.15.31:0/1407989916 >> v2:172.21.15.62:3300/0 conn(0x7fe3e40c7b60 0x7fe3e4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.592+0000 7fe3e3fff700 1 -- 172.21.15.31:0/1407989916 >> v2:172.21.15.62:3300/0 conn(0x7fe3e40c7b60 msgr2=0x7fe3e4069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.592+0000 7fe3e3fff700 1 --2- 172.21.15.31:0/1407989916 >> v2:172.21.15.62:3300/0 conn(0x7fe3e40c7b60 0x7fe3e4069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.592+0000 7fe3e3fff700 1 -- 172.21.15.31:0/1407989916 >> v1:172.21.15.62:6789/0 conn(0x7fe3e40c7f30 legacy=0x7fe3e406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.592+0000 7fe3e3fff700 1 -- 172.21.15.31:0/1407989916 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40c9c80 con 0x7fe3e40aee70 2023-10-06T05:18:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.592+0000 7fe3e27fc700 1 -- 172.21.15.31:0/1407989916 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482966279 0 0) 0x7fe3e40ca5b0 con 0x7fe3e40c7f30 2023-10-06T05:18:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.593+0000 7fe3e3fff700 1 --2- 172.21.15.31:0/1407989916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40aee70 0x7fe3e413f690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009f20 tx=0x7fe3dc009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b4823410ce63a7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.593+0000 7fe3e27fc700 1 -- 172.21.15.31:0/1407989916 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc00e070 con 0x7fe3e40aee70 2023-10-06T05:18:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.593+0000 7fe3e27fc700 1 -- 172.21.15.31:0/1407989916 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc00b9d0 con 0x7fe3e40aee70 2023-10-06T05:18:56.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.593+0000 7fe3e27fc700 1 -- 172.21.15.31:0/1407989916 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc0092e0 con 0x7fe3e40aee70 2023-10-06T05:18:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.593+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/1407989916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40aee70 msgr2=0x7fe3e413f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.593+0000 7fe3ea1ff700 1 --2- 172.21.15.31:0/1407989916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40aee70 0x7fe3e413f690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009f20 tx=0x7fe3dc009840 comp rx=0 tx=0).stop 2023-10-06T05:18:56.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.593+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/1407989916 shutdown_connections 2023-10-06T05:18:56.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.594+0000 7fe3ea1ff700 1 --2- 172.21.15.31:0/1407989916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40aee70 0x7fe3e413f690 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.594+0000 7fe3ea1ff700 1 --2- 172.21.15.31:0/1407989916 >> v2:172.21.15.62:3300/0 conn(0x7fe3e40c7b60 0x7fe3e4069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.594+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/1407989916 >> 172.21.15.31:0/1407989916 conn(0x7fe3e40cc0b0 msgr2=0x7fe3e40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:56.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.594+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/1407989916 shutdown_connections 2023-10-06T05:18:56.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.594+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/1407989916 wait complete. 2023-10-06T05:18:56.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.595+0000 7fe3ea1ff700 1 Processor -- start 2023-10-06T05:18:56.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.595+0000 7fe3ea1ff700 1 -- start start 2023-10-06T05:18:56.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.596+0000 7fe3ea1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e40aee70 0x7fe3e413c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.596+0000 7fe3ea1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40c7b60 0x7fe3e4165af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.596+0000 7fe3ea1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40c7f30 0x7fe3e4155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.596+0000 7fe3ea1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4166030 con 0x7fe3e40c7b60 2023-10-06T05:18:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.596+0000 7fe3ea1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e41595a0 con 0x7fe3e40aee70 2023-10-06T05:18:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.596+0000 7fe3ea1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e41564e0 con 0x7fe3e40c7f30 2023-10-06T05:18:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40c7b60 0x7fe3e4165af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40c7f30 0x7fe3e4155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e40aee70 0x7fe3e413c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40c7f30 0x7fe3e4155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:56856/0 (socket says 172.21.15.31:56856) 2023-10-06T05:18:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e3fff700 1 -- 172.21.15.31:0/3420739245 learned_addr learned my addr 172.21.15.31:0/3420739245 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e2ffd700 1 -- 172.21.15.31:0/3420739245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40c7f30 msgr2=0x7fe3e4155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e2ffd700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40c7f30 0x7fe3e4155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e2ffd700 1 -- 172.21.15.31:0/3420739245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e40aee70 msgr2=0x7fe3e413c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e2ffd700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e40aee70 0x7fe3e413c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e2ffd700 1 -- 172.21.15.31:0/3420739245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc009ae0 con 0x7fe3e40c7b60 2023-10-06T05:18:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e37fe700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e40aee70 0x7fe3e413c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:18:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.597+0000 7fe3e2ffd700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40c7b60 0x7fe3e4165af0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e810 tx=0x7fe3d8001180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.598+0000 7fe3cb7fe700 1 -- 172.21.15.31:0/3420739245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8007c80 con 0x7fe3e40c7b60 2023-10-06T05:18:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.598+0000 7fe3cb7fe700 1 -- 172.21.15.31:0/3420739245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8007e00 con 0x7fe3e40c7b60 2023-10-06T05:18:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.598+0000 7fe3cb7fe700 1 -- 172.21.15.31:0/3420739245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8009320 con 0x7fe3e40c7b60 2023-10-06T05:18:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.598+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/3420739245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e4157040 con 0x7fe3e40c7b60 2023-10-06T05:18:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.598+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/3420739245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e4159230 con 0x7fe3e40c7b60 2023-10-06T05:18:56.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.599+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/3420739245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e4157540 con 0x7fe3e40c7b60 2023-10-06T05:18:56.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.600+0000 7fe3cb7fe700 1 -- 172.21.15.31:0/3420739245 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fe3d8009690 con 0x7fe3e40c7b60 2023-10-06T05:18:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.601+0000 7fe3cb7fe700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe3d00670e0 0x7fe3d0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.601+0000 7fe3cb7fe700 1 -- 172.21.15.31:0/3420739245 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe3d8085f40 con 0x7fe3e40c7b60 2023-10-06T05:18:56.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.601+0000 7fe3e37fe700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe3d00670e0 0x7fe3d0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:56.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.602+0000 7fe3e37fe700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe3d00670e0 0x7fe3d0069590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4011460 tx=0x7fe3d4009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:56.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.605+0000 7fe3cb7fe700 1 -- 172.21.15.31:0/3420739245 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3e4157540 con 0x7fe3e40c7b60 2023-10-06T05:18:56.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.852+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/3420739245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3e4157540 con 0x7fe3e40c7b60 2023-10-06T05:18:56.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.852+0000 7fe3cb7fe700 1 -- 172.21.15.31:0/3420739245 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fe3e4157540 con 0x7fe3e40c7b60 2023-10-06T05:18:56.854 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:56.854 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:18:56.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.854+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/3420739245 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe3d00670e0 msgr2=0x7fe3d0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.854+0000 7fe3ea1ff700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe3d00670e0 0x7fe3d0069590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4011460 tx=0x7fe3d4009120 comp rx=0 tx=0).stop 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.854+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/3420739245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40c7b60 msgr2=0x7fe3e4165af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.854+0000 7fe3ea1ff700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40c7b60 0x7fe3e4165af0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e810 tx=0x7fe3d8001180 comp rx=0 tx=0).stop 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.855+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/3420739245 shutdown_connections 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.855+0000 7fe3ea1ff700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3e40c7f30 0x7fe3e4155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.855+0000 7fe3ea1ff700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e40aee70 0x7fe3e413c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.855+0000 7fe3ea1ff700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe3d00670e0 0x7fe3d0069590 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.855+0000 7fe3ea1ff700 1 --2- 172.21.15.31:0/3420739245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40c7b60 0x7fe3e4165af0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.855+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/3420739245 >> 172.21.15.31:0/3420739245 conn(0x7fe3e40cc0b0 msgr2=0x7fe3e40c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.855+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/3420739245 shutdown_connections 2023-10-06T05:18:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:56.855+0000 7fe3ea1ff700 1 -- 172.21.15.31:0/3420739245 wait complete. 2023-10-06T05:18:56.858 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:18:56.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:18:57.092 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T05:18:57.093 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-10-06T05:18:57.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000336720 2023-10-06T05:18:57.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.290+0000 7f5424fff700 1 Processor -- start 2023-10-06T05:18:57.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.290+0000 7f5424fff700 1 -- start start 2023-10-06T05:18:57.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.291+0000 7f5424fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54201573a0 0x7f54200acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f5424fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5420157770 0x7f54200b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f5424fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5420157be0 0x7f54200b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f5424fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5420158cf0 con 0x7f5420157770 2023-10-06T05:18:57.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f5424fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54201580e0 con 0x7f5420157be0 2023-10-06T05:18:57.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f5424fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54201583c0 con 0x7f54201573a0 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f541dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5420157770 0x7f54200b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f541dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5420157770 0x7f54200b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51260/0 (socket says 172.21.15.31:51260) 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f541dd9b700 1 -- 172.21.15.31:0/2233190080 learned_addr learned my addr 172.21.15.31:0/2233190080 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f541ed9d700 1 --2- 172.21.15.31:0/2233190080 >> v2:172.21.15.62:3300/0 conn(0x7f5420157be0 0x7f54200b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f541e59c700 1 --2- 172.21.15.31:0/2233190080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54201573a0 0x7f54200acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f541dd9b700 1 -- 172.21.15.31:0/2233190080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54201573a0 msgr2=0x7f54200acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f541dd9b700 1 --2- 172.21.15.31:0/2233190080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54201573a0 0x7f54200acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f541dd9b700 1 -- 172.21.15.31:0/2233190080 >> v2:172.21.15.62:3300/0 conn(0x7f5420157be0 msgr2=0x7f54200b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f541dd9b700 1 --2- 172.21.15.31:0/2233190080 >> v2:172.21.15.62:3300/0 conn(0x7f5420157be0 0x7f54200b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.292+0000 7f541dd9b700 1 -- 172.21.15.31:0/2233190080 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54201586a0 con 0x7f5420157770 2023-10-06T05:18:57.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f541dd9b700 1 --2- 172.21.15.31:0/2233190080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157770 0x7f54200b1260 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f542006b3a0 tx=0x7f5414009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b8268cacf65940 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:57.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f541d59a700 1 -- 172.21.15.31:0/2233190080 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400b860 con 0x7f5420157770 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f541d59a700 1 -- 172.21.15.31:0/2233190080 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541400b9c0 con 0x7f5420157770 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f541d59a700 1 -- 172.21.15.31:0/2233190080 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400bd60 con 0x7f5420157770 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f5424fff700 1 -- 172.21.15.31:0/2233190080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157770 msgr2=0x7f54200b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f5424fff700 1 --2- 172.21.15.31:0/2233190080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157770 0x7f54200b1260 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f542006b3a0 tx=0x7f5414009250 comp rx=0 tx=0).stop 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f5424fff700 1 -- 172.21.15.31:0/2233190080 shutdown_connections 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f5424fff700 1 --2- 172.21.15.31:0/2233190080 >> v2:172.21.15.62:3300/0 conn(0x7f5420157be0 0x7f54200b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f5424fff700 1 --2- 172.21.15.31:0/2233190080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54201573a0 0x7f54200acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f5424fff700 1 --2- 172.21.15.31:0/2233190080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157770 0x7f54200b1260 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.293+0000 7f5424fff700 1 -- 172.21.15.31:0/2233190080 >> 172.21.15.31:0/2233190080 conn(0x7f5420067560 msgr2=0x7f5420067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.294+0000 7f5424fff700 1 -- 172.21.15.31:0/2233190080 shutdown_connections 2023-10-06T05:18:57.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.294+0000 7f5424fff700 1 -- 172.21.15.31:0/2233190080 wait complete. 2023-10-06T05:18:57.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.295+0000 7f5424fff700 1 Processor -- start 2023-10-06T05:18:57.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.295+0000 7f5424fff700 1 -- start start 2023-10-06T05:18:57.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.296+0000 7f5424fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54201573a0 0x7f542006f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.296+0000 7f5424fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5420157770 0x7f542016dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.296+0000 7f5424fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157be0 0x7f54200aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.296+0000 7f5424fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420151500 con 0x7f5420157be0 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.296+0000 7f5424fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200b01a0 con 0x7f54201573a0 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.296+0000 7f5424fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200ab210 con 0x7f5420157770 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.296+0000 7f541ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157be0 0x7f54200aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.296+0000 7f541ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157be0 0x7f54200aacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51270/0 (socket says 172.21.15.31:51270) 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.296+0000 7f541ed9d700 1 -- 172.21.15.31:0/3552659654 learned_addr learned my addr 172.21.15.31:0/3552659654 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f541e59c700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54201573a0 0x7f542006f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f541dd9b700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5420157770 0x7f542016dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f541ed9d700 1 -- 172.21.15.31:0/3552659654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5420157770 msgr2=0x7f542016dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f541ed9d700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5420157770 0x7f542016dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f541ed9d700 1 -- 172.21.15.31:0/3552659654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54201573a0 msgr2=0x7f542006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f541ed9d700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54201573a0 0x7f542006f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f541ed9d700 1 -- 172.21.15.31:0/3552659654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54140094f0 con 0x7f5420157be0 2023-10-06T05:18:57.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f541ed9d700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157be0 0x7f54200aacd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5418007f50 tx=0x7f5418009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:57.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f540e7fc700 1 -- 172.21.15.31:0/3552659654 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800eb70 con 0x7f5420157be0 2023-10-06T05:18:57.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f540e7fc700 1 -- 172.21.15.31:0/3552659654 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541800ecf0 con 0x7f5420157be0 2023-10-06T05:18:57.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f540e7fc700 1 -- 172.21.15.31:0/3552659654 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418014670 con 0x7f5420157be0 2023-10-06T05:18:57.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.297+0000 7f5424fff700 1 -- 172.21.15.31:0/3552659654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542006e5a0 con 0x7f5420157be0 2023-10-06T05:18:57.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.298+0000 7f5424fff700 1 -- 172.21.15.31:0/3552659654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542006eb60 con 0x7f5420157be0 2023-10-06T05:18:57.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.298+0000 7f5424fff700 1 -- 172.21.15.31:0/3552659654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54200ab820 con 0x7f5420157be0 2023-10-06T05:18:57.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.299+0000 7f540e7fc700 1 -- 172.21.15.31:0/3552659654 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f54180149e0 con 0x7f5420157be0 2023-10-06T05:18:57.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.300+0000 7f540e7fc700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f54000670e0 0x7f5400069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.301+0000 7f540e7fc700 1 -- 172.21.15.31:0/3552659654 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f5418085070 con 0x7f5420157be0 2023-10-06T05:18:57.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.301+0000 7f541e59c700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f54000670e0 0x7f5400069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.301+0000 7f541e59c700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f54000670e0 0x7f5400069590 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5410006fd0 tx=0x7f5410008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.304+0000 7f540e7fc700 1 -- 172.21.15.31:0/3552659654 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5418051e20 con 0x7f5420157be0 2023-10-06T05:18:57.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.540+0000 7f5424fff700 1 -- 172.21.15.31:0/3552659654 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000336720", "target": ["mon-mgr", ""]}) v1 -- 0x7f542004c380 con 0x7f54000670e0 2023-10-06T05:18:57.548 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:57.547+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:57.549 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:57.547+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:57.549 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:57.547+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:57.549 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:57.547+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:57.549 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:18:57.547+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:18:57.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.584+0000 7f540e7fc700 1 -- 172.21.15.31:0/3552659654 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f542004c380 con 0x7f54000670e0 2023-10-06T05:18:57.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 -- 172.21.15.31:0/3552659654 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f54000670e0 msgr2=0x7f5400069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f54000670e0 0x7f5400069590 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5410006fd0 tx=0x7f5410008040 comp rx=0 tx=0).stop 2023-10-06T05:18:57.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 -- 172.21.15.31:0/3552659654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157be0 msgr2=0x7f54200aacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157be0 0x7f54200aacd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5418007f50 tx=0x7f5418009860 comp rx=0 tx=0).stop 2023-10-06T05:18:57.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 -- 172.21.15.31:0/3552659654 shutdown_connections 2023-10-06T05:18:57.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f54000670e0 0x7f5400069590 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5420157be0 0x7f54200aacd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54201573a0 0x7f542006f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 --2- 172.21.15.31:0/3552659654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5420157770 0x7f542016dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 -- 172.21.15.31:0/3552659654 >> 172.21.15.31:0/3552659654 conn(0x7f5420067560 msgr2=0x7f54200ca1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:57.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 -- 172.21.15.31:0/3552659654 shutdown_connections 2023-10-06T05:18:57.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.586+0000 7f5424fff700 1 -- 172.21.15.31:0/3552659654 wait complete. 2023-10-06T05:18:57.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:57.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.799+0000 7f010abff700 1 Processor -- start 2023-10-06T05:18:57.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.800+0000 7f010abff700 1 -- start start 2023-10-06T05:18:57.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.801+0000 7f010abff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0104159580 0x7f01040cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.801+0000 7f010abff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040cb9f0 0x7f01040cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.801+0000 7f010abff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01040cf280 con 0x7f01040ccc80 2023-10-06T05:18:57.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.801+0000 7f010abff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01040ce670 con 0x7f0104159580 2023-10-06T05:18:57.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.801+0000 7f010abff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01040ce950 con 0x7f01040cb9f0 2023-10-06T05:18:57.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.801+0000 7f0103fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040cb9f0 0x7f01040cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.801+0000 7f010899b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0104159580 0x7f01040cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.801+0000 7f0103fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040cb9f0 0x7f01040cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:35850/0 (socket says 172.21.15.31:35850) 2023-10-06T05:18:57.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.801+0000 7f0103fff700 1 -- 172.21.15.31:0/3999251171 learned_addr learned my addr 172.21.15.31:0/3999251171 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:57.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.802+0000 7f0103fff700 1 -- 172.21.15.31:0/3999251171 >> v2:172.21.15.62:3300/0 conn(0x7f0104159580 msgr2=0x7f01040cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:57.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.802+0000 7f0103fff700 1 --2- 172.21.15.31:0/3999251171 >> v2:172.21.15.62:3300/0 conn(0x7f0104159580 0x7f01040cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.802+0000 7f0103fff700 1 -- 172.21.15.31:0/3999251171 >> v1:172.21.15.62:6789/0 conn(0x7f01040ccc80 legacy=0x7f01040cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.802+0000 7f0103fff700 1 -- 172.21.15.31:0/3999251171 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01040cec30 con 0x7f01040cb9f0 2023-10-06T05:18:57.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.802+0000 7f0103fff700 1 --2- 172.21.15.31:0/3999251171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040cb9f0 0x7f01040cf710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00f00099b0 tx=0x7f00f0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc3a552f82a12d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:57.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.802+0000 7f01037fe700 1 -- 172.21.15.31:0/3999251171 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0009ea0 con 0x7f01040cb9f0 2023-10-06T05:18:57.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.802+0000 7f01037fe700 1 -- 172.21.15.31:0/3999251171 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f000b860 con 0x7f01040cb9f0 2023-10-06T05:18:57.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.802+0000 7f01037fe700 1 -- 172.21.15.31:0/3999251171 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000bc00 con 0x7f01040cb9f0 2023-10-06T05:18:57.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.803+0000 7f010abff700 1 -- 172.21.15.31:0/3999251171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040cb9f0 msgr2=0x7f01040cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.803+0000 7f010abff700 1 --2- 172.21.15.31:0/3999251171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040cb9f0 0x7f01040cf710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00f00099b0 tx=0x7f00f0009250 comp rx=0 tx=0).stop 2023-10-06T05:18:57.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.803+0000 7f010abff700 1 -- 172.21.15.31:0/3999251171 shutdown_connections 2023-10-06T05:18:57.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.803+0000 7f010abff700 1 --2- 172.21.15.31:0/3999251171 >> v2:172.21.15.62:3300/0 conn(0x7f0104159580 0x7f01040cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.803+0000 7f010abff700 1 --2- 172.21.15.31:0/3999251171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040cb9f0 0x7f01040cf710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.803+0000 7f010abff700 1 -- 172.21.15.31:0/3999251171 >> 172.21.15.31:0/3999251171 conn(0x7f0104099710 msgr2=0x7f010406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:57.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.803+0000 7f010abff700 1 -- 172.21.15.31:0/3999251171 shutdown_connections 2023-10-06T05:18:57.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.803+0000 7f010abff700 1 -- 172.21.15.31:0/3999251171 wait complete. 2023-10-06T05:18:57.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.804+0000 7f010abff700 1 Processor -- start 2023-10-06T05:18:57.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.805+0000 7f010abff700 1 -- start start 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.805+0000 7f010abff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01040cb9f0 0x7f010416a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.805+0000 7f010abff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040ccc80 0x7f0104158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.805+0000 7f010abff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0104159580 0x7f01040c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.805+0000 7f010abff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010416aac0 con 0x7f01040cb9f0 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.805+0000 7f010abff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0104159000 con 0x7f0104159580 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.805+0000 7f010abff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01041697f0 con 0x7f01040ccc80 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.806+0000 7f010899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01040cb9f0 0x7f010416a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.806+0000 7f010899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01040cb9f0 0x7f010416a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51292/0 (socket says 172.21.15.31:51292) 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.806+0000 7f010899b700 1 -- 172.21.15.31:0/720487116 learned_addr learned my addr 172.21.15.31:0/720487116 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.806+0000 7f010919c700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0104159580 0x7f01040c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.806+0000 7f0103fff700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040ccc80 0x7f0104158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.806+0000 7f010899b700 1 -- 172.21.15.31:0/720487116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040ccc80 msgr2=0x7f0104158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.806+0000 7f010899b700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040ccc80 0x7f0104158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.806+0000 7f010899b700 1 -- 172.21.15.31:0/720487116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0104159580 msgr2=0x7f01040c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.806+0000 7f010899b700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0104159580 0x7f01040c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.806+0000 7f010899b700 1 -- 172.21.15.31:0/720487116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f00094f0 con 0x7f01040cb9f0 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.807+0000 7f010899b700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01040cb9f0 0x7f010416a580 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f00f800ea00 tx=0x7f00f8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.807+0000 7f0100ff9700 1 -- 172.21.15.31:0/720487116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800c6a0 con 0x7f01040cb9f0 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.807+0000 7f0100ff9700 1 -- 172.21.15.31:0/720487116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f800c800 con 0x7f01040cb9f0 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.807+0000 7f0100ff9700 1 -- 172.21.15.31:0/720487116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800cb80 con 0x7f01040cb9f0 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.807+0000 7f010abff700 1 -- 172.21.15.31:0/720487116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01040c6ab0 con 0x7f01040cb9f0 2023-10-06T05:18:57.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.807+0000 7f010abff700 1 -- 172.21.15.31:0/720487116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0104169d80 con 0x7f01040cb9f0 2023-10-06T05:18:57.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.808+0000 7f010abff700 1 -- 172.21.15.31:0/720487116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010404c380 con 0x7f01040cb9f0 2023-10-06T05:18:57.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.809+0000 7f0100ff9700 1 -- 172.21.15.31:0/720487116 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f00f8013470 con 0x7f01040cb9f0 2023-10-06T05:18:57.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.810+0000 7f0100ff9700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f00e80671b0 0x7f00e8069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:57.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.810+0000 7f0100ff9700 1 -- 172.21.15.31:0/720487116 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f00f8084ed0 con 0x7f01040cb9f0 2023-10-06T05:18:57.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.810+0000 7f0103fff700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f00e80671b0 0x7f00e8069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:57.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.811+0000 7f0103fff700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f00e80671b0 0x7f00e8069660 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f00f00099b0 tx=0x7f00f0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:57.814+0000 7f0100ff9700 1 -- 172.21.15.31:0/720487116 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00f8051d00 con 0x7f01040cb9f0 2023-10-06T05:18:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.063+0000 7f010abff700 1 -- 172.21.15.31:0/720487116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01040cef10 con 0x7f01040cb9f0 2023-10-06T05:18:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.064+0000 7f0100ff9700 1 -- 172.21.15.31:0/720487116 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f00f8021090 con 0x7f01040cb9f0 2023-10-06T05:18:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:58.066 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:18:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 -- 172.21.15.31:0/720487116 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f00e80671b0 msgr2=0x7f00e8069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f00e80671b0 0x7f00e8069660 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f00f00099b0 tx=0x7f00f0017000 comp rx=0 tx=0).stop 2023-10-06T05:18:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 -- 172.21.15.31:0/720487116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01040cb9f0 msgr2=0x7f010416a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01040cb9f0 0x7f010416a580 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f00f800ea00 tx=0x7f00f8009120 comp rx=0 tx=0).stop 2023-10-06T05:18:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 -- 172.21.15.31:0/720487116 shutdown_connections 2023-10-06T05:18:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0104159580 0x7f01040c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f00e80671b0 0x7f00e8069660 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01040cb9f0 0x7f010416a580 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 --2- 172.21.15.31:0/720487116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01040ccc80 0x7f0104158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 -- 172.21.15.31:0/720487116 >> 172.21.15.31:0/720487116 conn(0x7f0104099710 msgr2=0x7f01040af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 -- 172.21.15.31:0/720487116 shutdown_connections 2023-10-06T05:18:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.066+0000 7f010abff700 1 -- 172.21.15.31:0/720487116 wait complete. 2023-10-06T05:18:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:18:58.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:18:58.305 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.73898811900000005,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.966429595906181,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:18:58.306 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.96642959590618 }, "state": "open", "uptime": 0.738988119 } ] 2023-10-06T05:18:58.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:18:58.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.500+0000 7f8f5adff700 1 Processor -- start 2023-10-06T05:18:58.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.500+0000 7f8f5adff700 1 -- start start 2023-10-06T05:18:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.501+0000 7f8f5adff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f540c9040 0x7f8f54159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.501+0000 7f8f5adff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f540c95b0 con 0x7f8f54068d80 2023-10-06T05:18:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.501+0000 7f8f5adff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f540c9890 con 0x7f8f541595a0 2023-10-06T05:18:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.501+0000 7f8f5adff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f540c9b70 con 0x7f8f540c9040 2023-10-06T05:18:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f53fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8f54068d80 0x7f8f540c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41812/0 (socket says 172.21.15.31:41812) 2023-10-06T05:18:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f53fff700 1 -- 172.21.15.31:0/2839601335 learned_addr learned my addr 172.21.15.31:0/2839601335 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:18:58.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f5939c700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f540c9040 0x7f8f54159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832013711 0 0) 0x7f8f540c95b0 con 0x7f8f54068d80 2023-10-06T05:18:58.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f44003570 con 0x7f8f54068d80 2023-10-06T05:18:58.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226658722 0 0) 0x7f8f540c9890 con 0x7f8f541595a0 2023-10-06T05:18:58.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f540c95b0 con 0x7f8f541595a0 2023-10-06T05:18:58.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2842342654 0 0) 0x7f8f44003570 con 0x7f8f54068d80 2023-10-06T05:18:58.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f540c9040 msgr2=0x7f8f54159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:58.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f540c9040 0x7f8f54159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 >> v1:172.21.15.62:6789/0 conn(0x7f8f541595a0 legacy=0x7f8f54068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f540ca130 con 0x7f8f54068d80 2023-10-06T05:18:58.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f8f40001470 con 0x7f8f54068d80 2023-10-06T05:18:58.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.502+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 >> v1:172.21.15.31:6789/0 conn(0x7f8f54068d80 legacy=0x7f8f540c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f537fe700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f44003c20 0x7f8f44004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f537fe700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f540c9040 0x7f8f44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f537fe700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 0x7f8f440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f540c9b70 con 0x7f8f44004d50 2023-10-06T05:18:58.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4400bbf0 con 0x7f8f44003c20 2023-10-06T05:18:58.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4400bd60 con 0x7f8f540c9040 2023-10-06T05:18:58.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f5939c700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 0x7f8f440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f58b9b700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f44003c20 0x7f8f44004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f53fff700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f540c9040 0x7f8f44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f5939c700 1 -- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f540c9040 msgr2=0x7f8f44004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:58.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f5939c700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f540c9040 0x7f8f44004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f5939c700 1 -- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f44003c20 msgr2=0x7f8f44004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:18:58.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f5939c700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f44003c20 0x7f8f44004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.503+0000 7f8f5939c700 1 -- 172.21.15.31:0/2839601335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f44003570 con 0x7f8f44004d50 2023-10-06T05:18:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.504+0000 7f8f5939c700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 0x7f8f440051c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0094d0 tx=0x7f8f4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01dcff36a0adb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.504+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c00ea00 con 0x7f8f44004d50 2023-10-06T05:18:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.504+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4c00f040 con 0x7f8f44004d50 2023-10-06T05:18:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.504+0000 7f8f537fe700 1 -- 172.21.15.31:0/2839601335 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c00ed90 con 0x7f8f44004d50 2023-10-06T05:18:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.504+0000 7f8f5adff700 1 -- 172.21.15.31:0/2839601335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 msgr2=0x7f8f440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.504+0000 7f8f5adff700 1 --2- 172.21.15.31:0/2839601335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 0x7f8f440051c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0094d0 tx=0x7f8f4c00b040 comp rx=0 tx=0).stop 2023-10-06T05:18:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.504+0000 7f8f5939c700 1 -- 172.21.15.31:0/2839601335 reap_dead start 2023-10-06T05:18:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.504+0000 7f8f5adff700 1 -- 172.21.15.31:0/2839601335 shutdown_connections 2023-10-06T05:18:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.505+0000 7f8f5adff700 1 -- 172.21.15.31:0/2839601335 >> 172.21.15.31:0/2839601335 conn(0x7f8f5406b620 msgr2=0x7f8f5406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.505+0000 7f8f5adff700 1 -- 172.21.15.31:0/2839601335 shutdown_connections 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.505+0000 7f8f5adff700 1 -- 172.21.15.31:0/2839601335 wait complete. 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.506+0000 7f8f5adff700 1 Processor -- start 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.506+0000 7f8f5adff700 1 -- start start 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.507+0000 7f8f5adff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 0x7f8f54169880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.507+0000 7f8f5adff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f54158540 0x7f8f5415c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.507+0000 7f8f5adff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5416c6b0 0x7f8f5415d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.507+0000 7f8f5adff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5415cf40 con 0x7f8f44004d50 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.507+0000 7f8f5adff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5415ca20 con 0x7f8f5416c6b0 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.507+0000 7f8f5adff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f54155fa0 con 0x7f8f54158540 2023-10-06T05:18:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.507+0000 7f8f58b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 0x7f8f54169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f58b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 0x7f8f54169880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51302/0 (socket says 172.21.15.31:51302) 2023-10-06T05:18:58.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f58b9b700 1 -- 172.21.15.31:0/1177758047 learned_addr learned my addr 172.21.15.31:0/1177758047 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:18:58.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f5939c700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5416c6b0 0x7f8f5415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f53fff700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f54158540 0x7f8f5415c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f58b9b700 1 -- 172.21.15.31:0/1177758047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f54158540 msgr2=0x7f8f5415c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f58b9b700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f54158540 0x7f8f5415c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f58b9b700 1 -- 172.21.15.31:0/1177758047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5416c6b0 msgr2=0x7f8f5415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f58b9b700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5416c6b0 0x7f8f5415d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f58b9b700 1 -- 172.21.15.31:0/1177758047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f541589b0 con 0x7f8f44004d50 2023-10-06T05:18:58.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f5939c700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5416c6b0 0x7f8f5415d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:58.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f53fff700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f54158540 0x7f8f5415c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:18:58.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f58b9b700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 0x7f8f54169880 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8f48002bb0 tx=0x7f8f48007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:58.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.508+0000 7f8f50ff9700 1 -- 172.21.15.31:0/1177758047 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800f9f0 con 0x7f8f44004d50 2023-10-06T05:18:58.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.509+0000 7f8f50ff9700 1 -- 172.21.15.31:0/1177758047 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4800fb70 con 0x7f8f44004d50 2023-10-06T05:18:58.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.509+0000 7f8f5adff700 1 -- 172.21.15.31:0/1177758047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5415cc60 con 0x7f8f44004d50 2023-10-06T05:18:58.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.509+0000 7f8f50ff9700 1 -- 172.21.15.31:0/1177758047 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48006820 con 0x7f8f44004d50 2023-10-06T05:18:58.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.509+0000 7f8f5adff700 1 -- 172.21.15.31:0/1177758047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f54155c30 con 0x7f8f44004d50 2023-10-06T05:18:58.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.510+0000 7f8f5adff700 1 -- 172.21.15.31:0/1177758047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f541557b0 con 0x7f8f44004d50 2023-10-06T05:18:58.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.510+0000 7f8f50ff9700 1 -- 172.21.15.31:0/1177758047 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f8f48006bd0 con 0x7f8f44004d50 2023-10-06T05:18:58.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.511+0000 7f8f50ff9700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f38067170 0x7f8f38069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:18:58.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.511+0000 7f8f50ff9700 1 -- 172.21.15.31:0/1177758047 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f8f48084e80 con 0x7f8f44004d50 2023-10-06T05:18:58.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.512+0000 7f8f53fff700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f38067170 0x7f8f38069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:18:58.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.512+0000 7f8f53fff700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f38067170 0x7f8f38069620 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8f400011e0 tx=0x7f8f40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:18:58.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.516+0000 7f8f50ff9700 1 -- 172.21.15.31:0/1177758047 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f541557b0 con 0x7f8f44004d50 2023-10-06T05:18:58.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.763+0000 7f8f5adff700 1 -- 172.21.15.31:0/1177758047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f541557b0 con 0x7f8f44004d50 2023-10-06T05:18:58.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.764+0000 7f8f50ff9700 1 -- 172.21.15.31:0/1177758047 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f8f541557b0 con 0x7f8f44004d50 2023-10-06T05:18:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:18:58.766 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:18:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 -- 172.21.15.31:0/1177758047 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f38067170 msgr2=0x7f8f38069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f38067170 0x7f8f38069620 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8f400011e0 tx=0x7f8f40009040 comp rx=0 tx=0).stop 2023-10-06T05:18:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 -- 172.21.15.31:0/1177758047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 msgr2=0x7f8f54169880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:18:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 0x7f8f54169880 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8f48002bb0 tx=0x7f8f48007d30 comp rx=0 tx=0).stop 2023-10-06T05:18:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 -- 172.21.15.31:0/1177758047 shutdown_connections 2023-10-06T05:18:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5416c6b0 0x7f8f5415d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f38067170 0x7f8f38069620 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f44004d50 0x7f8f54169880 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 --2- 172.21.15.31:0/1177758047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f54158540 0x7f8f5415c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:18:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 -- 172.21.15.31:0/1177758047 >> 172.21.15.31:0/1177758047 conn(0x7f8f5406b620 msgr2=0x7f8f540cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:18:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 -- 172.21.15.31:0/1177758047 shutdown_connections 2023-10-06T05:18:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:18:58.766+0000 7f8f5adff700 1 -- 172.21.15.31:0/1177758047 wait complete. 2023-10-06T05:18:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:18:58.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:18:59.002 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.4359769120000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.934301812110881,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:18:59.002 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.93430181211088 }, "state": "open", "uptime": 1.435976912 } ] 2023-10-06T05:18:59.003 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:19:00.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:00.708 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:04.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:04.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.209+0000 7f6f5e9ff700 1 Processor -- start 2023-10-06T05:19:04.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.209+0000 7f6f5e9ff700 1 -- start start 2023-10-06T05:19:04.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.210+0000 7f6f5e9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 0x7f6f581596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:04.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.211+0000 7f6f5e9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5815e1c0 con 0x7f6f5815dd50 2023-10-06T05:19:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.211+0000 7f6f5e9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f580ccb30 con 0x7f6f58067c70 2023-10-06T05:19:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.211+0000 7f6f5e9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f580c8990 con 0x7f6f58159310 2023-10-06T05:19:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.211+0000 7f6f577fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6f5815dd50 0x7f6f58067560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41824/0 (socket says 172.21.15.31:41824) 2023-10-06T05:19:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.211+0000 7f6f577fe700 1 -- 172.21.15.31:0/1219182136 learned_addr learned my addr 172.21.15.31:0/1219182136 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:19:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.211+0000 7f6f57fff700 1 --2- 172.21.15.31:0/1219182136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 0x7f6f581596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.211+0000 7f6f56ffd700 1 -- 172.21.15.31:0/1219182136 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115650022 0 0) 0x7f6f5815e1c0 con 0x7f6f5815dd50 2023-10-06T05:19:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.211+0000 7f6f56ffd700 1 -- 172.21.15.31:0/1219182136 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f40003570 con 0x7f6f5815dd50 2023-10-06T05:19:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.212+0000 7f6f56ffd700 1 -- 172.21.15.31:0/1219182136 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876214404 0 0) 0x7f6f580ccb30 con 0x7f6f58067c70 2023-10-06T05:19:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.212+0000 7f6f56ffd700 1 -- 172.21.15.31:0/1219182136 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f5815e1c0 con 0x7f6f58067c70 2023-10-06T05:19:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.212+0000 7f6f56ffd700 1 -- 172.21.15.31:0/1219182136 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 358737922 0 0) 0x7f6f40003570 con 0x7f6f5815dd50 2023-10-06T05:19:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.212+0000 7f6f57fff700 1 -- 172.21.15.31:0/1219182136 >> v1:172.21.15.62:6789/0 conn(0x7f6f58067c70 legacy=0x7f6f5806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.212+0000 7f6f57fff700 1 -- 172.21.15.31:0/1219182136 >> v1:172.21.15.31:6789/0 conn(0x7f6f5815dd50 legacy=0x7f6f58067560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.212+0000 7f6f57fff700 1 -- 172.21.15.31:0/1219182136 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f580ccfc0 con 0x7f6f58159310 2023-10-06T05:19:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.212+0000 7f6f57fff700 1 --2- 172.21.15.31:0/1219182136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 0x7f6f581596e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009910 tx=0x7f6f480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1519154b6f26219e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.212+0000 7f6f56ffd700 1 -- 172.21.15.31:0/1219182136 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c001470 con 0x7f6f58159310 2023-10-06T05:19:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.212+0000 7f6f56ffd700 1 -- 172.21.15.31:0/1219182136 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f48011420 con 0x7f6f58159310 2023-10-06T05:19:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.212+0000 7f6f56ffd700 1 -- 172.21.15.31:0/1219182136 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f480117c0 con 0x7f6f58159310 2023-10-06T05:19:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.213+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/1219182136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 msgr2=0x7f6f581596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.213+0000 7f6f5e9ff700 1 --2- 172.21.15.31:0/1219182136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 0x7f6f581596e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009910 tx=0x7f6f480091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.213+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/1219182136 shutdown_connections 2023-10-06T05:19:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.213+0000 7f6f5e9ff700 1 --2- 172.21.15.31:0/1219182136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 0x7f6f581596e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.213+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/1219182136 >> 172.21.15.31:0/1219182136 conn(0x7f6f580cdc50 msgr2=0x7f6f580c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.213+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/1219182136 shutdown_connections 2023-10-06T05:19:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.213+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/1219182136 wait complete. 2023-10-06T05:19:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.214+0000 7f6f5e9ff700 1 Processor -- start 2023-10-06T05:19:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.214+0000 7f6f5e9ff700 1 -- start start 2023-10-06T05:19:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.215+0000 7f6f5e9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f58067c70 0x7f6f58153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.215+0000 7f6f5e9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 0x7f6f580c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.215+0000 7f6f5e9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f5815dd50 0x7f6f580c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.215+0000 7f6f5e9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f58156040 con 0x7f6f58067c70 2023-10-06T05:19:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.215+0000 7f6f5e9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f580c5dc0 con 0x7f6f5815dd50 2023-10-06T05:19:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.215+0000 7f6f5e9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f580c60a0 con 0x7f6f58159310 2023-10-06T05:19:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f57fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f58067c70 0x7f6f58153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:04.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f57fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f58067c70 0x7f6f58153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51306/0 (socket says 172.21.15.31:51306) 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f57fff700 1 -- 172.21.15.31:0/2428064419 learned_addr learned my addr 172.21.15.31:0/2428064419 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f577fe700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 0x7f6f580c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f5cf9c700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f5815dd50 0x7f6f580c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f57fff700 1 -- 172.21.15.31:0/2428064419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 msgr2=0x7f6f580c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f57fff700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 0x7f6f580c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f57fff700 1 -- 172.21.15.31:0/2428064419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f5815dd50 msgr2=0x7f6f580c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f57fff700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f5815dd50 0x7f6f580c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f57fff700 1 -- 172.21.15.31:0/2428064419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f48009420 con 0x7f6f58067c70 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.216+0000 7f6f57fff700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f58067c70 0x7f6f58153220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6f480091e0 tx=0x7f6f48016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.217+0000 7f6f3ffff700 1 -- 172.21.15.31:0/2428064419 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4801cd60 con 0x7f6f58067c70 2023-10-06T05:19:04.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.217+0000 7f6f3ffff700 1 -- 172.21.15.31:0/2428064419 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4801a450 con 0x7f6f58067c70 2023-10-06T05:19:04.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.217+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/2428064419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f580c6380 con 0x7f6f58067c70 2023-10-06T05:19:04.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.217+0000 7f6f3ffff700 1 -- 172.21.15.31:0/2428064419 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4801a810 con 0x7f6f58067c70 2023-10-06T05:19:04.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.217+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/2428064419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f580c6940 con 0x7f6f58067c70 2023-10-06T05:19:04.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.218+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/2428064419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5815a440 con 0x7f6f58067c70 2023-10-06T05:19:04.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.219+0000 7f6f3ffff700 1 -- 172.21.15.31:0/2428064419 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6f4801ac00 con 0x7f6f58067c70 2023-10-06T05:19:04.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.220+0000 7f6f3ffff700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6f44067170 0x7f6f44069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:04.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.220+0000 7f6f3ffff700 1 -- 172.21.15.31:0/2428064419 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f6f48086480 con 0x7f6f58067c70 2023-10-06T05:19:04.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.220+0000 7f6f577fe700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6f44067170 0x7f6f44069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:04.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.220+0000 7f6f577fe700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6f44067170 0x7f6f44069620 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6f580679f0 tx=0x7f6f4c0015c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:04.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.224+0000 7f6f3ffff700 1 -- 172.21.15.31:0/2428064419 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f48053220 con 0x7f6f58067c70 2023-10-06T05:19:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.473+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/2428064419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f580c6b30 con 0x7f6f58067c70 2023-10-06T05:19:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.474+0000 7f6f3ffff700 1 -- 172.21.15.31:0/2428064419 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f6f48056840 con 0x7f6f58067c70 2023-10-06T05:19:04.476 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:19:04.476 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:19:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.476+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/2428064419 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6f44067170 msgr2=0x7f6f44069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.476+0000 7f6f5e9ff700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6f44067170 0x7f6f44069620 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6f580679f0 tx=0x7f6f4c0015c0 comp rx=0 tx=0).stop 2023-10-06T05:19:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.476+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/2428064419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f58067c70 msgr2=0x7f6f58153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.476+0000 7f6f5e9ff700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f58067c70 0x7f6f58153220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6f480091e0 tx=0x7f6f48016000 comp rx=0 tx=0).stop 2023-10-06T05:19:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.476+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/2428064419 shutdown_connections 2023-10-06T05:19:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.476+0000 7f6f5e9ff700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f5815dd50 0x7f6f580c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.476+0000 7f6f5e9ff700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6f44067170 0x7f6f44069620 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.476+0000 7f6f5e9ff700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f58067c70 0x7f6f58153220 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.476+0000 7f6f5e9ff700 1 --2- 172.21.15.31:0/2428064419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f58159310 0x7f6f580c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.476+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/2428064419 >> 172.21.15.31:0/2428064419 conn(0x7f6f580cdc50 msgr2=0x7f6f580ca560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:04.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.477+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/2428064419 shutdown_connections 2023-10-06T05:19:04.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:04.477+0000 7f6f5e9ff700 1 -- 172.21.15.31:0/2428064419 wait complete. 2023-10-06T05:19:04.480 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:19:04.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:04.713 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":7.1468850899999996,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.672996459500681,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:19:04.713 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.67299645950068 }, "state": "open", "uptime": 7.14688509 } ] 2023-10-06T05:19:04.714 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:19:09.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:09.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.918+0000 7fcc5ddff700 1 Processor -- start 2023-10-06T05:19:09.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.919+0000 7fcc5ddff700 1 -- start start 2023-10-06T05:19:09.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.920+0000 7fcc5ddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc58068d80 0x7fcc58159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:09.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.920+0000 7fcc5ddff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcc580cc0d0 0x7fcc580c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:09.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.920+0000 7fcc5ddff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc580c95b0 con 0x7fcc581595a0 2023-10-06T05:19:09.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.920+0000 7fcc5ddff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc580c9890 con 0x7fcc580cc0d0 2023-10-06T05:19:09.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.920+0000 7fcc5ddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc580c9b70 con 0x7fcc58068d80 2023-10-06T05:19:09.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.920+0000 7fcc577fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcc581595a0 0x7fcc58068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60700/0 (socket says 172.21.15.31:60700) 2023-10-06T05:19:09.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.920+0000 7fcc577fe700 1 -- 172.21.15.31:0/915844506 learned_addr learned my addr 172.21.15.31:0/915844506 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:19:09.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.920+0000 7fcc57fff700 1 --2- 172.21.15.31:0/915844506 >> v2:172.21.15.31:3300/0 conn(0x7fcc580cc0d0 0x7fcc580c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:09.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc56ffd700 1 --2- 172.21.15.31:0/915844506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc58068d80 0x7fcc58159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:09.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc567fc700 1 -- 172.21.15.31:0/915844506 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386261498 0 0) 0x7fcc580c95b0 con 0x7fcc581595a0 2023-10-06T05:19:09.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc567fc700 1 -- 172.21.15.31:0/915844506 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc40003570 con 0x7fcc581595a0 2023-10-06T05:19:09.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc57fff700 1 -- 172.21.15.31:0/915844506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc58068d80 msgr2=0x7fcc58159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc57fff700 1 --2- 172.21.15.31:0/915844506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc58068d80 0x7fcc58159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc57fff700 1 -- 172.21.15.31:0/915844506 >> v1:172.21.15.31:6789/0 conn(0x7fcc581595a0 legacy=0x7fcc58068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc57fff700 1 -- 172.21.15.31:0/915844506 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc580ca130 con 0x7fcc580cc0d0 2023-10-06T05:19:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc57fff700 1 --2- 172.21.15.31:0/915844506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc580cc0d0 0x7fcc580c8930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009e60 tx=0x7fcc50009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee199573eaf2fa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc567fc700 1 -- 172.21.15.31:0/915844506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000e070 con 0x7fcc580cc0d0 2023-10-06T05:19:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc567fc700 1 -- 172.21.15.31:0/915844506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5000bb60 con 0x7fcc580cc0d0 2023-10-06T05:19:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.921+0000 7fcc567fc700 1 -- 172.21.15.31:0/915844506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50012430 con 0x7fcc580cc0d0 2023-10-06T05:19:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.922+0000 7fcc5ddff700 1 -- 172.21.15.31:0/915844506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc580cc0d0 msgr2=0x7fcc580c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.922+0000 7fcc5ddff700 1 --2- 172.21.15.31:0/915844506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc580cc0d0 0x7fcc580c8930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009e60 tx=0x7fcc50009780 comp rx=0 tx=0).stop 2023-10-06T05:19:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.922+0000 7fcc5ddff700 1 -- 172.21.15.31:0/915844506 shutdown_connections 2023-10-06T05:19:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.922+0000 7fcc5ddff700 1 --2- 172.21.15.31:0/915844506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc580cc0d0 0x7fcc580c8930 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.922+0000 7fcc5ddff700 1 --2- 172.21.15.31:0/915844506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc58068d80 0x7fcc58159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.922+0000 7fcc5ddff700 1 -- 172.21.15.31:0/915844506 >> 172.21.15.31:0/915844506 conn(0x7fcc5806b620 msgr2=0x7fcc5806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.922+0000 7fcc5ddff700 1 -- 172.21.15.31:0/915844506 shutdown_connections 2023-10-06T05:19:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.922+0000 7fcc5ddff700 1 -- 172.21.15.31:0/915844506 wait complete. 2023-10-06T05:19:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.924+0000 7fcc5ddff700 1 Processor -- start 2023-10-06T05:19:09.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.924+0000 7fcc5ddff700 1 -- start start 2023-10-06T05:19:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc5ddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc58068d80 0x7fcc580c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc5ddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc580cc0d0 0x7fcc58153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc5ddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc581595a0 0x7fcc58156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc5ddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5815b450 con 0x7fcc58068d80 2023-10-06T05:19:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc5ddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc580b3f30 con 0x7fcc580cc0d0 2023-10-06T05:19:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc5ddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc580c6ab0 con 0x7fcc581595a0 2023-10-06T05:19:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc58068d80 0x7fcc580c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc58068d80 0x7fcc580c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60254/0 (socket says 172.21.15.31:60254) 2023-10-06T05:19:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc577fe700 1 -- 172.21.15.31:0/1508824616 learned_addr learned my addr 172.21.15.31:0/1508824616 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc57fff700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc581595a0 0x7fcc58156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc56ffd700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc580cc0d0 0x7fcc58153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc577fe700 1 -- 172.21.15.31:0/1508824616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc581595a0 msgr2=0x7fcc58156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc577fe700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc581595a0 0x7fcc58156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc577fe700 1 -- 172.21.15.31:0/1508824616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc580cc0d0 msgr2=0x7fcc58153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc577fe700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc580cc0d0 0x7fcc58153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc577fe700 1 -- 172.21.15.31:0/1508824616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50009a20 con 0x7fcc58068d80 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.926+0000 7fcc56ffd700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc580cc0d0 0x7fcc58153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.927+0000 7fcc577fe700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc58068d80 0x7fcc580c64d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c7c0 tx=0x7fcc48007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.927+0000 7fcc3f7fe700 1 -- 172.21.15.31:0/1508824616 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48005b30 con 0x7fcc58068d80 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.927+0000 7fcc3f7fe700 1 -- 172.21.15.31:0/1508824616 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48005cb0 con 0x7fcc58068d80 2023-10-06T05:19:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.927+0000 7fcc5ddff700 1 -- 172.21.15.31:0/1508824616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc580c6d90 con 0x7fcc58068d80 2023-10-06T05:19:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.927+0000 7fcc3f7fe700 1 -- 172.21.15.31:0/1508824616 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48007320 con 0x7fcc58068d80 2023-10-06T05:19:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.927+0000 7fcc5ddff700 1 -- 172.21.15.31:0/1508824616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc580c7350 con 0x7fcc58068d80 2023-10-06T05:19:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.928+0000 7fcc5ddff700 1 -- 172.21.15.31:0/1508824616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5804c380 con 0x7fcc58068d80 2023-10-06T05:19:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.929+0000 7fcc3f7fe700 1 -- 172.21.15.31:0/1508824616 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fcc4801a360 con 0x7fcc58068d80 2023-10-06T05:19:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.930+0000 7fcc3f7fe700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc44067170 0x7fcc44069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.930+0000 7fcc3f7fe700 1 -- 172.21.15.31:0/1508824616 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fcc480841a0 con 0x7fcc58068d80 2023-10-06T05:19:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.930+0000 7fcc56ffd700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc44067170 0x7fcc44069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.931+0000 7fcc56ffd700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc44067170 0x7fcc44069620 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcc580cccf0 tx=0x7fcc4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:09.934+0000 7fcc3f7fe700 1 -- 172.21.15.31:0/1508824616 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc48050de0 con 0x7fcc58068d80 2023-10-06T05:19:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.180+0000 7fcc5ddff700 1 -- 172.21.15.31:0/1508824616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc580c9230 con 0x7fcc58068d80 2023-10-06T05:19:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.181+0000 7fcc3f7fe700 1 -- 172.21.15.31:0/1508824616 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fcc48054400 con 0x7fcc58068d80 2023-10-06T05:19:10.183 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:19:10.183 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:19:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 -- 172.21.15.31:0/1508824616 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc44067170 msgr2=0x7fcc44069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc44067170 0x7fcc44069620 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcc580cccf0 tx=0x7fcc4c008040 comp rx=0 tx=0).stop 2023-10-06T05:19:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 -- 172.21.15.31:0/1508824616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc58068d80 msgr2=0x7fcc580c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc58068d80 0x7fcc580c64d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c7c0 tx=0x7fcc48007e00 comp rx=0 tx=0).stop 2023-10-06T05:19:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 -- 172.21.15.31:0/1508824616 shutdown_connections 2023-10-06T05:19:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc581595a0 0x7fcc58156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc44067170 0x7fcc44069620 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc58068d80 0x7fcc580c64d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 --2- 172.21.15.31:0/1508824616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc580cc0d0 0x7fcc58153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 -- 172.21.15.31:0/1508824616 >> 172.21.15.31:0/1508824616 conn(0x7fcc5806b620 msgr2=0x7fcc5806cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 -- 172.21.15.31:0/1508824616 shutdown_connections 2023-10-06T05:19:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:10.183+0000 7fcc5ddff700 1 -- 172.21.15.31:0/1508824616 wait complete. 2023-10-06T05:19:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:19:10.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:10.421 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":12.85379333,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.415295814800878,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:19:10.421 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.415295814800878 }, "state": "open", "uptime": 12.85379333 } ] 2023-10-06T05:19:10.422 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:19:15.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.628+0000 7f154b3ff700 1 Processor -- start 2023-10-06T05:19:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.629+0000 7f154b3ff700 1 -- start start 2023-10-06T05:19:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.630+0000 7f154b3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15440c7f30 0x7f1544069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.630+0000 7f154b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 0x7f154413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.630+0000 7f154b3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15440ca5b0 con 0x7f15440c7b60 2023-10-06T05:19:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.630+0000 7f154b3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15440af2f0 con 0x7f15440c7f30 2023-10-06T05:19:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.630+0000 7f154b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154406f740 con 0x7f15440aee70 2023-10-06T05:19:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.630+0000 7f154899a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15440c7f30 0x7f1544069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.630+0000 7f154899a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15440c7f30 0x7f1544069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60264/0 (socket says 172.21.15.31:60264) 2023-10-06T05:19:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.630+0000 7f154899a700 1 -- 172.21.15.31:0/1997542128 learned_addr learned my addr 172.21.15.31:0/1997542128 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:15.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.630+0000 7f154999c700 1 --2- 172.21.15.31:0/1997542128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 0x7f154413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.631+0000 7f154899a700 1 -- 172.21.15.31:0/1997542128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 msgr2=0x7f154413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.631+0000 7f154899a700 1 --2- 172.21.15.31:0/1997542128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 0x7f154413f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.631+0000 7f154899a700 1 -- 172.21.15.31:0/1997542128 >> v1:172.21.15.31:6789/0 conn(0x7f15440c7b60 legacy=0x7f154406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.631+0000 7f154899a700 1 -- 172.21.15.31:0/1997542128 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15440c9c80 con 0x7f15440c7f30 2023-10-06T05:19:15.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.631+0000 7f154999c700 1 --2- 172.21.15.31:0/1997542128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 0x7f154413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.631+0000 7f153bfff700 1 -- 172.21.15.31:0/1997542128 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076111659 0 0) 0x7f15440ca5b0 con 0x7f15440c7b60 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.631+0000 7f154899a700 1 --2- 172.21.15.31:0/1997542128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7f30 0x7f1544069880 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f15340099b0 tx=0x7f1534009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42381c8ddad5f3ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.631+0000 7f153bfff700 1 -- 172.21.15.31:0/1997542128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400b860 con 0x7f15440c7f30 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.631+0000 7f153bfff700 1 -- 172.21.15.31:0/1997542128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153400b9c0 con 0x7f15440c7f30 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.631+0000 7f153bfff700 1 -- 172.21.15.31:0/1997542128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400bd60 con 0x7f15440c7f30 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.632+0000 7f154b3ff700 1 -- 172.21.15.31:0/1997542128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7f30 msgr2=0x7f1544069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.632+0000 7f154b3ff700 1 --2- 172.21.15.31:0/1997542128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7f30 0x7f1544069880 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f15340099b0 tx=0x7f1534009250 comp rx=0 tx=0).stop 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.632+0000 7f154b3ff700 1 -- 172.21.15.31:0/1997542128 shutdown_connections 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.632+0000 7f154b3ff700 1 --2- 172.21.15.31:0/1997542128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 0x7f154413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.632+0000 7f154b3ff700 1 --2- 172.21.15.31:0/1997542128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7f30 0x7f1544069880 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.632+0000 7f154b3ff700 1 -- 172.21.15.31:0/1997542128 >> 172.21.15.31:0/1997542128 conn(0x7f15440cc0b0 msgr2=0x7f15440cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:15.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.632+0000 7f154b3ff700 1 -- 172.21.15.31:0/1997542128 shutdown_connections 2023-10-06T05:19:15.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.632+0000 7f154b3ff700 1 -- 172.21.15.31:0/1997542128 wait complete. 2023-10-06T05:19:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.633+0000 7f154b3ff700 1 Processor -- start 2023-10-06T05:19:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.634+0000 7f154b3ff700 1 -- start start 2023-10-06T05:19:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 0x7f15440a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154b3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7b60 0x7f15440c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154b3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15440c7f30 0x7f15440c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154b3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15440a6980 con 0x7f15440c7b60 2023-10-06T05:19:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154b3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15440a5d70 con 0x7f15440c7f30 2023-10-06T05:19:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15440a6050 con 0x7f15440aee70 2023-10-06T05:19:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7b60 0x7f15440c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:15.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7b60 0x7f15440c3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60270/0 (socket says 172.21.15.31:60270) 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154899a700 1 -- 172.21.15.31:0/1389674892 learned_addr learned my addr 172.21.15.31:0/1389674892 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154919b700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 0x7f15440a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154999c700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15440c7f30 0x7f15440c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154899a700 1 -- 172.21.15.31:0/1389674892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 msgr2=0x7f15440a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154899a700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 0x7f15440a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154899a700 1 -- 172.21.15.31:0/1389674892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15440c7f30 msgr2=0x7f15440c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154899a700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15440c7f30 0x7f15440c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.635+0000 7f154899a700 1 -- 172.21.15.31:0/1389674892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15340094f0 con 0x7f15440c7b60 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.636+0000 7f154899a700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7b60 0x7f15440c3160 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1534009290 tx=0x7f1534017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.636+0000 7f15397fa700 1 -- 172.21.15.31:0/1389674892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534024440 con 0x7f15440c7b60 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.636+0000 7f15397fa700 1 -- 172.21.15.31:0/1389674892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15340245e0 con 0x7f15440c7b60 2023-10-06T05:19:15.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.636+0000 7f15397fa700 1 -- 172.21.15.31:0/1389674892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15340249a0 con 0x7f15440c7b60 2023-10-06T05:19:15.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.636+0000 7f154b3ff700 1 -- 172.21.15.31:0/1389674892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15440a6330 con 0x7f15440c7b60 2023-10-06T05:19:15.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.636+0000 7f154b3ff700 1 -- 172.21.15.31:0/1389674892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15440a59d0 con 0x7f15440c7b60 2023-10-06T05:19:15.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.637+0000 7f154b3ff700 1 -- 172.21.15.31:0/1389674892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15440a6af0 con 0x7f15440c7b60 2023-10-06T05:19:15.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.638+0000 7f15397fa700 1 -- 172.21.15.31:0/1389674892 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f153401a420 con 0x7f15440c7b60 2023-10-06T05:19:15.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.638+0000 7f15397fa700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1528067170 0x7f1528069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:15.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.639+0000 7f15397fa700 1 -- 172.21.15.31:0/1389674892 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f15340866d0 con 0x7f15440c7b60 2023-10-06T05:19:15.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.639+0000 7f154919b700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1528067170 0x7f1528069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:15.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.639+0000 7f154919b700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1528067170 0x7f1528069620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1540003ab0 tx=0x7f1540007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:15.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.643+0000 7f15397fa700 1 -- 172.21.15.31:0/1389674892 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1534053470 con 0x7f15440c7b60 2023-10-06T05:19:15.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.891+0000 7f154b3ff700 1 -- 172.21.15.31:0/1389674892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f154404c380 con 0x7f15440c7b60 2023-10-06T05:19:15.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.892+0000 7f15397fa700 1 -- 172.21.15.31:0/1389674892 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f1534056a90 con 0x7f15440c7b60 2023-10-06T05:19:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:19:15.894 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:19:15.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.894+0000 7f154b3ff700 1 -- 172.21.15.31:0/1389674892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1528067170 msgr2=0x7f1528069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.894+0000 7f154b3ff700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1528067170 0x7f1528069620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1540003ab0 tx=0x7f1540007ca0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.894+0000 7f154b3ff700 1 -- 172.21.15.31:0/1389674892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7b60 msgr2=0x7f15440c3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.894+0000 7f154b3ff700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7b60 0x7f15440c3160 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1534009290 tx=0x7f1534017000 comp rx=0 tx=0).stop 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.895+0000 7f154b3ff700 1 -- 172.21.15.31:0/1389674892 shutdown_connections 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.895+0000 7f154b3ff700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15440c7f30 0x7f15440c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.895+0000 7f154b3ff700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15440aee70 0x7f15440a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.895+0000 7f154b3ff700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1528067170 0x7f1528069620 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.895+0000 7f154b3ff700 1 --2- 172.21.15.31:0/1389674892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15440c7b60 0x7f15440c3160 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.895+0000 7f154b3ff700 1 -- 172.21.15.31:0/1389674892 >> 172.21.15.31:0/1389674892 conn(0x7f15440cc0b0 msgr2=0x7f15440b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.895+0000 7f154b3ff700 1 -- 172.21.15.31:0/1389674892 shutdown_connections 2023-10-06T05:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:15.895+0000 7f154b3ff700 1 -- 172.21.15.31:0/1389674892 wait complete. 2023-10-06T05:19:15.898 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:19:15.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:16.132 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":18.565701491999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.160749492598601,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:19:16.133 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.1607494925986 }, "state": "open", "uptime": 18.565701492 } ] 2023-10-06T05:19:16.133 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:19:21.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.339+0000 7f5ceb9ff700 1 Processor -- start 2023-10-06T05:19:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.339+0000 7f5ceb9ff700 1 -- start start 2023-10-06T05:19:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.340+0000 7f5ceb9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5ce40c7b60 0x7f5ce4069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.340+0000 7f5ceb9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce40c7f30 0x7f5ce40aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.340+0000 7f5ceb9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce40ca5b0 con 0x7f5ce406ef50 2023-10-06T05:19:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.340+0000 7f5ceb9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce406f740 con 0x7f5ce40c7b60 2023-10-06T05:19:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.341+0000 7f5ceb9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40c9c80 con 0x7f5ce40c7f30 2023-10-06T05:19:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.341+0000 7f5ce8f9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce40c7f30 0x7f5ce40aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.341+0000 7f5ce979b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5ce40c7b60 0x7f5ce4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.341+0000 7f5ce9f9c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5ce406ef50 0x7f5ce413f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:55574/0 (socket says 172.21.15.31:55574) 2023-10-06T05:19:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.341+0000 7f5ce9f9c700 1 -- 172.21.15.31:0/3885313097 learned_addr learned my addr 172.21.15.31:0/3885313097 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:19:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.341+0000 7f5ce979b700 1 -- 172.21.15.31:0/3885313097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce40c7f30 msgr2=0x7f5ce40aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.341+0000 7f5ce979b700 1 --2- 172.21.15.31:0/3885313097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce40c7f30 0x7f5ce40aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.341+0000 7f5ce979b700 1 -- 172.21.15.31:0/3885313097 >> v1:172.21.15.62:6789/0 conn(0x7f5ce406ef50 legacy=0x7f5ce413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.342+0000 7f5ce979b700 1 -- 172.21.15.31:0/3885313097 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40c9f60 con 0x7f5ce40c7b60 2023-10-06T05:19:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.341+0000 7f5cdbfff700 1 -- 172.21.15.31:0/3885313097 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166435822 0 0) 0x7f5ce40ca5b0 con 0x7f5ce406ef50 2023-10-06T05:19:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.342+0000 7f5ce8f9a700 1 --2- 172.21.15.31:0/3885313097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce40c7f30 0x7f5ce40aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.342+0000 7f5ce979b700 1 --2- 172.21.15.31:0/3885313097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce40c7b60 0x7f5ce4069880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00098d0 tx=0x7f5ce0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecc0873ef4e01a32 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.342+0000 7f5cdbfff700 1 -- 172.21.15.31:0/3885313097 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0005c10 con 0x7f5ce40c7b60 2023-10-06T05:19:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.342+0000 7f5cdbfff700 1 -- 172.21.15.31:0/3885313097 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce0005d70 con 0x7f5ce40c7b60 2023-10-06T05:19:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.342+0000 7f5cdbfff700 1 -- 172.21.15.31:0/3885313097 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0006280 con 0x7f5ce40c7b60 2023-10-06T05:19:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.343+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/3885313097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce40c7b60 msgr2=0x7f5ce4069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.343+0000 7f5ceb9ff700 1 --2- 172.21.15.31:0/3885313097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce40c7b60 0x7f5ce4069880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00098d0 tx=0x7f5ce0009120 comp rx=0 tx=0).stop 2023-10-06T05:19:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.343+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/3885313097 shutdown_connections 2023-10-06T05:19:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.343+0000 7f5ceb9ff700 1 --2- 172.21.15.31:0/3885313097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce40c7b60 0x7f5ce4069880 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.343+0000 7f5ceb9ff700 1 --2- 172.21.15.31:0/3885313097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce40c7f30 0x7f5ce40aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.343+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/3885313097 >> 172.21.15.31:0/3885313097 conn(0x7f5ce40cc0b0 msgr2=0x7f5ce40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:21.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.343+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/3885313097 shutdown_connections 2023-10-06T05:19:21.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.343+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/3885313097 wait complete. 2023-10-06T05:19:21.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.345+0000 7f5ceb9ff700 1 Processor -- start 2023-10-06T05:19:21.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.345+0000 7f5ceb9ff700 1 -- start start 2023-10-06T05:19:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.346+0000 7f5ceb9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce406ef50 0x7f5ce4139f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.346+0000 7f5ceb9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce40c7b60 0x7f5ce415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.346+0000 7f5ceb9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce40c7f30 0x7f5ce415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.346+0000 7f5ceb9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce41485f0 con 0x7f5ce40c7f30 2023-10-06T05:19:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.346+0000 7f5ceb9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce4148840 con 0x7f5ce40c7b60 2023-10-06T05:19:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.346+0000 7f5ceb9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce4148b20 con 0x7f5ce406ef50 2023-10-06T05:19:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce9f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce40c7f30 0x7f5ce415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce9f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce40c7f30 0x7f5ce415a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49710/0 (socket says 172.21.15.31:49710) 2023-10-06T05:19:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce9f9c700 1 -- 172.21.15.31:0/2156053676 learned_addr learned my addr 172.21.15.31:0/2156053676 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:21.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce979b700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce406ef50 0x7f5ce4139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce8f9a700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce40c7b60 0x7f5ce415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce9f9c700 1 -- 172.21.15.31:0/2156053676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce406ef50 msgr2=0x7f5ce4139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce9f9c700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce406ef50 0x7f5ce4139f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce9f9c700 1 -- 172.21.15.31:0/2156053676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce40c7b60 msgr2=0x7f5ce415a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce9f9c700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce40c7b60 0x7f5ce415a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce9f9c700 1 -- 172.21.15.31:0/2156053676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd4009420 con 0x7f5ce40c7f30 2023-10-06T05:19:21.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.347+0000 7f5ce9f9c700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce40c7f30 0x7f5ce415a7f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00cc10 tx=0x7f5cdc001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:21.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.348+0000 7f5cd97fa700 1 -- 172.21.15.31:0/2156053676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00a850 con 0x7f5ce40c7f30 2023-10-06T05:19:21.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.348+0000 7f5cd97fa700 1 -- 172.21.15.31:0/2156053676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc00a9d0 con 0x7f5ce40c7f30 2023-10-06T05:19:21.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.348+0000 7f5cd97fa700 1 -- 172.21.15.31:0/2156053676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00ad90 con 0x7f5ce40c7f30 2023-10-06T05:19:21.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.348+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/2156053676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce00093c0 con 0x7f5ce40c7f30 2023-10-06T05:19:21.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.348+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/2156053676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce41490e0 con 0x7f5ce40c7f30 2023-10-06T05:19:21.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.349+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/2156053676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce404c380 con 0x7f5ce40c7f30 2023-10-06T05:19:21.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.350+0000 7f5cd97fa700 1 -- 172.21.15.31:0/2156053676 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f5cdc007300 con 0x7f5ce40c7f30 2023-10-06T05:19:21.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.350+0000 7f5cd97fa700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5cc8067170 0x7f5cc8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.351+0000 7f5cd97fa700 1 -- 172.21.15.31:0/2156053676 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f5cdc084dc0 con 0x7f5ce40c7f30 2023-10-06T05:19:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.351+0000 7f5ce979b700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5cc8067170 0x7f5cc8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.351+0000 7f5ce979b700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5cc8067170 0x7f5cc8069620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009160 tx=0x7f5ce0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.355+0000 7f5cd97fa700 1 -- 172.21.15.31:0/2156053676 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5cdc051b50 con 0x7f5ce40c7f30 2023-10-06T05:19:21.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.601+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/2156053676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ce40ae8d0 con 0x7f5ce40c7f30 2023-10-06T05:19:21.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.602+0000 7f5cd97fa700 1 -- 172.21.15.31:0/2156053676 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f5cdc055170 con 0x7f5ce40c7f30 2023-10-06T05:19:21.604 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:19:21.604 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:19:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.604+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/2156053676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5cc8067170 msgr2=0x7f5cc8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.604+0000 7f5ceb9ff700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5cc8067170 0x7f5cc8069620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009160 tx=0x7f5ce0016000 comp rx=0 tx=0).stop 2023-10-06T05:19:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.604+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/2156053676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce40c7f30 msgr2=0x7f5ce415a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.604+0000 7f5ceb9ff700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce40c7f30 0x7f5ce415a7f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00cc10 tx=0x7f5cdc001970 comp rx=0 tx=0).stop 2023-10-06T05:19:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.604+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/2156053676 shutdown_connections 2023-10-06T05:19:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.604+0000 7f5ceb9ff700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5cc8067170 0x7f5cc8069620 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.604+0000 7f5ceb9ff700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce40c7f30 0x7f5ce415a7f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.605+0000 7f5ceb9ff700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ce406ef50 0x7f5ce4139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.605+0000 7f5ceb9ff700 1 --2- 172.21.15.31:0/2156053676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ce40c7b60 0x7f5ce415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.605+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/2156053676 >> 172.21.15.31:0/2156053676 conn(0x7f5ce40cc0b0 msgr2=0x7f5ce40b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:21.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.605+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/2156053676 shutdown_connections 2023-10-06T05:19:21.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:21.605+0000 7f5ceb9ff700 1 -- 172.21.15.31:0/2156053676 wait complete. 2023-10-06T05:19:21.608 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:19:21.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:21.842 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":24.274609700999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.909671496654948,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:19:21.843 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.909671496654948 }, "state": "open", "uptime": 24.274609701 } ] 2023-10-06T05:19:21.844 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:19:26.844 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:27.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.050+0000 7f52ca7ff700 1 Processor -- start 2023-10-06T05:19:27.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.051+0000 7f52ca7ff700 1 -- start start 2023-10-06T05:19:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.052+0000 7f52ca7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f52c4159500 0x7f52c40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.052+0000 7f52ca7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c40c3f40 con 0x7f52c40cd510 2023-10-06T05:19:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.052+0000 7f52ca7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c40cb000 con 0x7f52c415d2d0 2023-10-06T05:19:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.052+0000 7f52ca7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40b2860 con 0x7f52c4159500 2023-10-06T05:19:27.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.053+0000 7f52c8d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52c40cd510 0x7f52c40aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45160/0 (socket says 172.21.15.31:45160) 2023-10-06T05:19:27.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.053+0000 7f52c8d9c700 1 -- 172.21.15.31:0/2622916456 learned_addr learned my addr 172.21.15.31:0/2622916456 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:19:27.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.053+0000 7f52c3fff700 1 --2- 172.21.15.31:0/2622916456 >> v2:172.21.15.62:3300/0 conn(0x7f52c4159500 0x7f52c40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.053+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239024354 0 0) 0x7f52c40c3f40 con 0x7f52c40cd510 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.053+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52ac003570 con 0x7f52c40cd510 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.053+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285698892 0 0) 0x7f52c40cb000 con 0x7f52c415d2d0 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.053+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c40c3f40 con 0x7f52c415d2d0 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.053+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1230728124 0 0) 0x7f52ac003570 con 0x7f52c40cd510 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.053+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 >> v2:172.21.15.62:3300/0 conn(0x7f52c4159500 msgr2=0x7f52c40b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.053+0000 7f52c2ffd700 1 --2- 172.21.15.31:0/2622916456 >> v2:172.21.15.62:3300/0 conn(0x7f52c4159500 0x7f52c40b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 >> v1:172.21.15.62:6789/0 conn(0x7f52c415d2d0 legacy=0x7f52c40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c415d7d0 con 0x7f52c40cd510 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f52bc0017f0 con 0x7f52c40cd510 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 >> v1:172.21.15.31:6789/0 conn(0x7f52c40cd510 legacy=0x7f52c40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c2ffd700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ac003bf0 0x7f52ac003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c2ffd700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ac004810 0x7f52ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c2ffd700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0051c0 0x7f52ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40b2860 con 0x7f52ac0051c0 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac00c160 con 0x7f52ac004810 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac00c320 con 0x7f52ac003bf0 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c8d9c700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0051c0 0x7f52ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c3fff700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ac003bf0 0x7f52ac003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c37fe700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ac004810 0x7f52ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c8d9c700 1 -- 172.21.15.31:0/2622916456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ac003bf0 msgr2=0x7f52ac003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c8d9c700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52ac003bf0 0x7f52ac003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c8d9c700 1 -- 172.21.15.31:0/2622916456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ac004810 msgr2=0x7f52ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c8d9c700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ac004810 0x7f52ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.054+0000 7f52c8d9c700 1 -- 172.21.15.31:0/2622916456 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac003570 con 0x7f52ac0051c0 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.055+0000 7f52c8d9c700 1 -- 172.21.15.31:0/2622916456 reap_dead start 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.055+0000 7f52c8d9c700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0051c0 0x7f52ac00ba60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00a8c0 tx=0x7f52bc002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bbe949e8b6072 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.055+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0070d0 con 0x7f52ac0051c0 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.055+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc007230 con 0x7f52ac0051c0 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.055+0000 7f52c2ffd700 1 -- 172.21.15.31:0/2622916456 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0075d0 con 0x7f52ac0051c0 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.055+0000 7f52ca7ff700 1 -- 172.21.15.31:0/2622916456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0051c0 msgr2=0x7f52ac00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.055+0000 7f52ca7ff700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0051c0 0x7f52ac00ba60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00a8c0 tx=0x7f52bc002190 comp rx=0 tx=0).stop 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.055+0000 7f52ca7ff700 1 -- 172.21.15.31:0/2622916456 shutdown_connections 2023-10-06T05:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.056+0000 7f52ca7ff700 1 --2- 172.21.15.31:0/2622916456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0051c0 0x7f52ac00ba60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.056+0000 7f52ca7ff700 1 -- 172.21.15.31:0/2622916456 >> 172.21.15.31:0/2622916456 conn(0x7f52c406ec90 msgr2=0x7f52c406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:27.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.056+0000 7f52ca7ff700 1 -- 172.21.15.31:0/2622916456 shutdown_connections 2023-10-06T05:19:27.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.056+0000 7f52ca7ff700 1 -- 172.21.15.31:0/2622916456 wait complete. 2023-10-06T05:19:27.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.057+0000 7f52ca7ff700 1 Processor -- start 2023-10-06T05:19:27.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.057+0000 7f52ca7ff700 1 -- start start 2023-10-06T05:19:27.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.058+0000 7f52ca7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ac0051c0 0x7f52c4158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.058+0000 7f52ca7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c415b740 0x7f52c415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.058+0000 7f52ca7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52c416d1a0 0x7f52c416d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.058+0000 7f52ca7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c416db50 con 0x7f52c415b740 2023-10-06T05:19:27.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.058+0000 7f52ca7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c415ce60 con 0x7f52ac0051c0 2023-10-06T05:19:27.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.058+0000 7f52ca7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c41589b0 con 0x7f52c416d1a0 2023-10-06T05:19:27.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.058+0000 7f52c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c415b740 0x7f52c415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.058+0000 7f52c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c415b740 0x7f52c415bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47284/0 (socket says 172.21.15.31:47284) 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.058+0000 7f52c37fe700 1 -- 172.21.15.31:0/3301321118 learned_addr learned my addr 172.21.15.31:0/3301321118 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52c37fe700 1 -- 172.21.15.31:0/3301321118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52c416d1a0 msgr2=0x7f52c416d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52c37fe700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52c416d1a0 0x7f52c416d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52c37fe700 1 -- 172.21.15.31:0/3301321118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ac0051c0 msgr2=0x7f52c4158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52c3fff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ac0051c0 0x7f52c4158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52c37fe700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ac0051c0 0x7f52c4158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52c37fe700 1 -- 172.21.15.31:0/3301321118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c415c250 con 0x7f52c415b740 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52c3fff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ac0051c0 0x7f52c4158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52c37fe700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c415b740 0x7f52c415bbb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f52b800e780 tx=0x7f52b8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52abfff700 1 -- 172.21.15.31:0/3301321118 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8001560 con 0x7f52c415b740 2023-10-06T05:19:27.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52abfff700 1 -- 172.21.15.31:0/3301321118 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b8010040 con 0x7f52c415b740 2023-10-06T05:19:27.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52ca7ff700 1 -- 172.21.15.31:0/3301321118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c415c530 con 0x7f52c415b740 2023-10-06T05:19:27.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52abfff700 1 -- 172.21.15.31:0/3301321118 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8001910 con 0x7f52c415b740 2023-10-06T05:19:27.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.059+0000 7f52ca7ff700 1 -- 172.21.15.31:0/3301321118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c415caf0 con 0x7f52c415b740 2023-10-06T05:19:27.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.061+0000 7f52ca7ff700 1 -- 172.21.15.31:0/3301321118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c415cff0 con 0x7f52c415b740 2023-10-06T05:19:27.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.061+0000 7f52abfff700 1 -- 172.21.15.31:0/3301321118 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f52b8001cc0 con 0x7f52c415b740 2023-10-06T05:19:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.062+0000 7f52abfff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52b0067170 0x7f52b0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.062+0000 7f52abfff700 1 -- 172.21.15.31:0/3301321118 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f52b8084a50 con 0x7f52c415b740 2023-10-06T05:19:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.062+0000 7f52c3fff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52b0067170 0x7f52b0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.063+0000 7f52c3fff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52b0067170 0x7f52b0069620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f52b4006f00 tx=0x7f52b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.067+0000 7f52abfff700 1 -- 172.21.15.31:0/3301321118 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52b80516e0 con 0x7f52c415b740 2023-10-06T05:19:27.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.319+0000 7f52ca7ff700 1 -- 172.21.15.31:0/3301321118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52c404c380 con 0x7f52c415b740 2023-10-06T05:19:27.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.320+0000 7f52abfff700 1 -- 172.21.15.31:0/3301321118 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f52b8054d00 con 0x7f52c415b740 2023-10-06T05:19:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:19:27.322 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:19:27.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 -- 172.21.15.31:0/3301321118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52b0067170 msgr2=0x7f52b0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52b0067170 0x7f52b0069620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f52b4006f00 tx=0x7f52b4008040 comp rx=0 tx=0).stop 2023-10-06T05:19:27.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 -- 172.21.15.31:0/3301321118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c415b740 msgr2=0x7f52c415bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:27.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c415b740 0x7f52c415bbb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f52b800e780 tx=0x7f52b8009040 comp rx=0 tx=0).stop 2023-10-06T05:19:27.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 -- 172.21.15.31:0/3301321118 shutdown_connections 2023-10-06T05:19:27.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52c416d1a0 0x7f52c416d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52ac0051c0 0x7f52c4158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52b0067170 0x7f52b0069620 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 --2- 172.21.15.31:0/3301321118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52c415b740 0x7f52c415bbb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:27.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 -- 172.21.15.31:0/3301321118 >> 172.21.15.31:0/3301321118 conn(0x7f52c406ec90 msgr2=0x7f52c40c40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:27.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 -- 172.21.15.31:0/3301321118 shutdown_connections 2023-10-06T05:19:27.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:27.322+0000 7f52ca7ff700 1 -- 172.21.15.31:0/3301321118 wait complete. 2023-10-06T05:19:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:19:27.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:27.563 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":29.996517686000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.661323050947029,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:19:27.563 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.66132305094703 }, "state": "open", "uptime": 29.996517686 } ] 2023-10-06T05:19:27.564 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:19:30.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:30.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:19:32.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.770+0000 7ff825dff700 1 Processor -- start 2023-10-06T05:19:32.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.771+0000 7ff825dff700 1 -- start start 2023-10-06T05:19:32.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.772+0000 7ff825dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8200c7b60 0x7ff820069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.772+0000 7ff825dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8200c7f30 0x7ff8200aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.772+0000 7ff825dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8200ca5b0 con 0x7ff82006ef50 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.772+0000 7ff825dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82006f740 con 0x7ff8200c7f30 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.772+0000 7ff825dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200c9c80 con 0x7ff8200c7b60 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.772+0000 7ff81effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8200c7f30 0x7ff8200aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.772+0000 7ff81effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8200c7f30 0x7ff8200aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47288/0 (socket says 172.21.15.31:47288) 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.772+0000 7ff81effd700 1 -- 172.21.15.31:0/3061031077 learned_addr learned my addr 172.21.15.31:0/3061031077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.773+0000 7ff81f7fe700 1 --2- 172.21.15.31:0/3061031077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8200c7b60 0x7ff820069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.773+0000 7ff81effd700 1 -- 172.21.15.31:0/3061031077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8200c7b60 msgr2=0x7ff820069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.773+0000 7ff81effd700 1 --2- 172.21.15.31:0/3061031077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8200c7b60 0x7ff820069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.773+0000 7ff81effd700 1 -- 172.21.15.31:0/3061031077 >> v1:172.21.15.62:6789/0 conn(0x7ff82006ef50 legacy=0x7ff82013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:32.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.773+0000 7ff81effd700 1 -- 172.21.15.31:0/3061031077 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8200c9f60 con 0x7ff8200c7f30 2023-10-06T05:19:32.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.773+0000 7ff81effd700 1 --2- 172.21.15.31:0/3061031077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 0x7ff8200aee70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff8140099b0 tx=0x7ff814009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2935fd7a8c1b61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:32.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.773+0000 7ff81e7fc700 1 -- 172.21.15.31:0/3061031077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400b820 con 0x7ff8200c7f30 2023-10-06T05:19:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.773+0000 7ff81e7fc700 1 -- 172.21.15.31:0/3061031077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81400b980 con 0x7ff8200c7f30 2023-10-06T05:19:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.773+0000 7ff81e7fc700 1 -- 172.21.15.31:0/3061031077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400bd20 con 0x7ff8200c7f30 2023-10-06T05:19:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.774+0000 7ff825dff700 1 -- 172.21.15.31:0/3061031077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 msgr2=0x7ff8200aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.774+0000 7ff825dff700 1 --2- 172.21.15.31:0/3061031077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 0x7ff8200aee70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff8140099b0 tx=0x7ff814009250 comp rx=0 tx=0).stop 2023-10-06T05:19:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.774+0000 7ff825dff700 1 -- 172.21.15.31:0/3061031077 shutdown_connections 2023-10-06T05:19:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.774+0000 7ff825dff700 1 --2- 172.21.15.31:0/3061031077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8200c7b60 0x7ff820069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.774+0000 7ff825dff700 1 --2- 172.21.15.31:0/3061031077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 0x7ff8200aee70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.774+0000 7ff825dff700 1 -- 172.21.15.31:0/3061031077 >> 172.21.15.31:0/3061031077 conn(0x7ff8200cc0b0 msgr2=0x7ff8200cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.774+0000 7ff825dff700 1 -- 172.21.15.31:0/3061031077 shutdown_connections 2023-10-06T05:19:32.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.774+0000 7ff825dff700 1 -- 172.21.15.31:0/3061031077 wait complete. 2023-10-06T05:19:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.775+0000 7ff825dff700 1 Processor -- start 2023-10-06T05:19:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.776+0000 7ff825dff700 1 -- start start 2023-10-06T05:19:32.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.776+0000 7ff825dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff82006ef50 0x7ff82013ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.776+0000 7ff825dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8200c7b60 0x7ff8200afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.776+0000 7ff825dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 0x7ff8200b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.776+0000 7ff825dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff820159880 con 0x7ff8200c7f30 2023-10-06T05:19:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.776+0000 7ff825dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff820159b60 con 0x7ff82006ef50 2023-10-06T05:19:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.776+0000 7ff825dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200ae250 con 0x7ff8200c7b60 2023-10-06T05:19:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 0x7ff8200b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 0x7ff8200b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47298/0 (socket says 172.21.15.31:47298) 2023-10-06T05:19:32.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81ffff700 1 -- 172.21.15.31:0/1973062026 learned_addr learned my addr 172.21.15.31:0/1973062026 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:32.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81effd700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8200c7b60 0x7ff8200afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81f7fe700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff82006ef50 0x7ff82013ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81ffff700 1 -- 172.21.15.31:0/1973062026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8200c7b60 msgr2=0x7ff8200afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81ffff700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8200c7b60 0x7ff8200afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81ffff700 1 -- 172.21.15.31:0/1973062026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff82006ef50 msgr2=0x7ff82013ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81ffff700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff82006ef50 0x7ff82013ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81ffff700 1 -- 172.21.15.31:0/1973062026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8140094f0 con 0x7ff8200c7f30 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81f7fe700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff82006ef50 0x7ff82013ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.777+0000 7ff81ffff700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 0x7ff8200b0420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff81800bc60 tx=0x7ff818009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.778+0000 7ff8077fe700 1 -- 172.21.15.31:0/1973062026 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800c9c0 con 0x7ff8200c7f30 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.778+0000 7ff8077fe700 1 -- 172.21.15.31:0/1973062026 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81800cb80 con 0x7ff8200c7f30 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.778+0000 7ff825dff700 1 -- 172.21.15.31:0/1973062026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8200ae530 con 0x7ff8200c7f30 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.778+0000 7ff8077fe700 1 -- 172.21.15.31:0/1973062026 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818014410 con 0x7ff8200c7f30 2023-10-06T05:19:32.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.778+0000 7ff825dff700 1 -- 172.21.15.31:0/1973062026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82013efd0 con 0x7ff8200c7f30 2023-10-06T05:19:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.779+0000 7ff825dff700 1 -- 172.21.15.31:0/1973062026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff820159cf0 con 0x7ff8200c7f30 2023-10-06T05:19:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.780+0000 7ff8077fe700 1 -- 172.21.15.31:0/1973062026 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7ff818014800 con 0x7ff8200c7f30 2023-10-06T05:19:32.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.781+0000 7ff8077fe700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff80c067170 0x7ff80c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:32.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.781+0000 7ff8077fe700 1 -- 172.21.15.31:0/1973062026 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7ff818084ac0 con 0x7ff8200c7f30 2023-10-06T05:19:32.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.781+0000 7ff81f7fe700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff80c067170 0x7ff80c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:32.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.781+0000 7ff81f7fe700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff80c067170 0x7ff80c069620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff810006aa0 tx=0x7ff810008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:32.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:32.784+0000 7ff8077fe700 1 -- 172.21.15.31:0/1973062026 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff81804c5a0 con 0x7ff8200c7f30 2023-10-06T05:19:33.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.033+0000 7ff825dff700 1 -- 172.21.15.31:0/1973062026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff82004c380 con 0x7ff8200c7f30 2023-10-06T05:19:33.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.033+0000 7ff8077fe700 1 -- 172.21.15.31:0/1973062026 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7ff818051860 con 0x7ff8200c7f30 2023-10-06T05:19:33.036 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:19:33.036 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:19:33.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 -- 172.21.15.31:0/1973062026 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff80c067170 msgr2=0x7ff80c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff80c067170 0x7ff80c069620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff810006aa0 tx=0x7ff810008040 comp rx=0 tx=0).stop 2023-10-06T05:19:33.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 -- 172.21.15.31:0/1973062026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 msgr2=0x7ff8200b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:33.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 0x7ff8200b0420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff81800bc60 tx=0x7ff818009820 comp rx=0 tx=0).stop 2023-10-06T05:19:33.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 -- 172.21.15.31:0/1973062026 shutdown_connections 2023-10-06T05:19:33.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff80c067170 0x7ff80c069620 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8200c7f30 0x7ff8200b0420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff82006ef50 0x7ff82013ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 --2- 172.21.15.31:0/1973062026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8200c7b60 0x7ff8200afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:33.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 -- 172.21.15.31:0/1973062026 >> 172.21.15.31:0/1973062026 conn(0x7ff8200cc0b0 msgr2=0x7ff82006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:33.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 -- 172.21.15.31:0/1973062026 shutdown_connections 2023-10-06T05:19:33.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:33.036+0000 7ff825dff700 1 -- 172.21.15.31:0/1973062026 wait complete. 2023-10-06T05:19:33.040 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:19:33.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:33.276 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":35.708425484000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.416661785178054,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:19:33.277 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.416661785178054 }, "state": "open", "uptime": 35.708425484 } ] 2023-10-06T05:19:33.277 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:19:38.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.483+0000 7f781efff700 1 Processor -- start 2023-10-06T05:19:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.484+0000 7f781efff700 1 -- start start 2023-10-06T05:19:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.485+0000 7f781efff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f78180c7b60 0x7f7818069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.485+0000 7f781efff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78180c7f30 0x7f78180aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.485+0000 7f781efff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78180ca5b0 con 0x7f781806ef50 2023-10-06T05:19:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.485+0000 7f781efff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781806f740 con 0x7f78180c7b60 2023-10-06T05:19:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.485+0000 7f781efff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78180c9c80 con 0x7f78180c7f30 2023-10-06T05:19:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.485+0000 7f781cd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f78180c7b60 0x7f7818069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.485+0000 7f781d59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f781806ef50 0x7f781813f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58004/0 (socket says 172.21.15.31:58004) 2023-10-06T05:19:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.485+0000 7f781d59c700 1 -- 172.21.15.31:0/1221547143 learned_addr learned my addr 172.21.15.31:0/1221547143 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:19:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.485+0000 7f7817fff700 1 --2- 172.21.15.31:0/1221547143 >> v2:172.21.15.62:3300/0 conn(0x7f78180c7f30 0x7f78180aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.486+0000 7f78177fe700 1 -- 172.21.15.31:0/1221547143 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773494770 0 0) 0x7f78180ca5b0 con 0x7f781806ef50 2023-10-06T05:19:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.486+0000 7f781cd9b700 1 -- 172.21.15.31:0/1221547143 >> v2:172.21.15.62:3300/0 conn(0x7f78180c7f30 msgr2=0x7f78180aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.486+0000 7f781cd9b700 1 --2- 172.21.15.31:0/1221547143 >> v2:172.21.15.62:3300/0 conn(0x7f78180c7f30 0x7f78180aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.486+0000 7f781cd9b700 1 -- 172.21.15.31:0/1221547143 >> v1:172.21.15.31:6789/0 conn(0x7f781806ef50 legacy=0x7f781813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.486+0000 7f781cd9b700 1 -- 172.21.15.31:0/1221547143 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78180c9f60 con 0x7f78180c7b60 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.486+0000 7f781cd9b700 1 --2- 172.21.15.31:0/1221547143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78180c7b60 0x7f7818069880 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f780c0098d0 tx=0x7f780c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08955e1aea7e98b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.486+0000 7f78177fe700 1 -- 172.21.15.31:0/1221547143 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c009e70 con 0x7f78180c7b60 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.486+0000 7f78177fe700 1 -- 172.21.15.31:0/1221547143 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c00b820 con 0x7f78180c7b60 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.486+0000 7f78177fe700 1 -- 172.21.15.31:0/1221547143 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c00bbc0 con 0x7f78180c7b60 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.487+0000 7f781efff700 1 -- 172.21.15.31:0/1221547143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78180c7b60 msgr2=0x7f7818069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.487+0000 7f781efff700 1 --2- 172.21.15.31:0/1221547143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78180c7b60 0x7f7818069880 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f780c0098d0 tx=0x7f780c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.487+0000 7f781efff700 1 -- 172.21.15.31:0/1221547143 shutdown_connections 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.487+0000 7f781efff700 1 --2- 172.21.15.31:0/1221547143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78180c7b60 0x7f7818069880 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.487+0000 7f781efff700 1 --2- 172.21.15.31:0/1221547143 >> v2:172.21.15.62:3300/0 conn(0x7f78180c7f30 0x7f78180aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.487+0000 7f781efff700 1 -- 172.21.15.31:0/1221547143 >> 172.21.15.31:0/1221547143 conn(0x7f78180cc0b0 msgr2=0x7f78180cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.487+0000 7f781efff700 1 -- 172.21.15.31:0/1221547143 shutdown_connections 2023-10-06T05:19:38.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.487+0000 7f781efff700 1 -- 172.21.15.31:0/1221547143 wait complete. 2023-10-06T05:19:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.488+0000 7f781efff700 1 Processor -- start 2023-10-06T05:19:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.489+0000 7f781efff700 1 -- start start 2023-10-06T05:19:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.489+0000 7f781efff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781806ef50 0x7f7818158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.489+0000 7f781efff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78180c7b60 0x7f78180ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.489+0000 7f781efff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78180c7f30 0x7f781816cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.489+0000 7f781efff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781813ef90 con 0x7f781806ef50 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.489+0000 7f781efff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781816db90 con 0x7f78180c7b60 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.489+0000 7f781efff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78180ab830 con 0x7f78180c7f30 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781806ef50 0x7f7818158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781806ef50 0x7f7818158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58768/0 (socket says 172.21.15.31:58768) 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781cd9b700 1 -- 172.21.15.31:0/2162267158 learned_addr learned my addr 172.21.15.31:0/2162267158 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f7817fff700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78180c7b60 0x7f78180ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781d59c700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78180c7f30 0x7f781816cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781cd9b700 1 -- 172.21.15.31:0/2162267158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78180c7f30 msgr2=0x7f781816cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781cd9b700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78180c7f30 0x7f781816cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781cd9b700 1 -- 172.21.15.31:0/2162267158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78180c7b60 msgr2=0x7f78180ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781cd9b700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78180c7b60 0x7f78180ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781cd9b700 1 -- 172.21.15.31:0/2162267158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c0093c0 con 0x7f781806ef50 2023-10-06T05:19:38.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781d59c700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78180c7f30 0x7f781816cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:38.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.490+0000 7f781cd9b700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781806ef50 0x7f7818158b10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f781806cd80 tx=0x7f780c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:38.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.491+0000 7f7814ff9700 1 -- 172.21.15.31:0/2162267158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c00bba0 con 0x7f781806ef50 2023-10-06T05:19:38.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.491+0000 7f7814ff9700 1 -- 172.21.15.31:0/2162267158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c011420 con 0x7f781806ef50 2023-10-06T05:19:38.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.491+0000 7f781efff700 1 -- 172.21.15.31:0/2162267158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78180abb10 con 0x7f781806ef50 2023-10-06T05:19:38.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.491+0000 7f7814ff9700 1 -- 172.21.15.31:0/2162267158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c0117e0 con 0x7f781806ef50 2023-10-06T05:19:38.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.491+0000 7f781efff700 1 -- 172.21.15.31:0/2162267158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7818159050 con 0x7f781806ef50 2023-10-06T05:19:38.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.492+0000 7f781efff700 1 -- 172.21.15.31:0/2162267158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78180ac030 con 0x7f781806ef50 2023-10-06T05:19:38.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.493+0000 7f7814ff9700 1 -- 172.21.15.31:0/2162267158 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f780c011b50 con 0x7f781806ef50 2023-10-06T05:19:38.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.494+0000 7f7814ff9700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f77fc067170 0x7f77fc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:38.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.494+0000 7f7814ff9700 1 -- 172.21.15.31:0/2162267158 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f780c086430 con 0x7f781806ef50 2023-10-06T05:19:38.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.494+0000 7f7817fff700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f77fc067170 0x7f77fc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:38.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.494+0000 7f7817fff700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f77fc067170 0x7f77fc069620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f78180ab4c0 tx=0x7f7804008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:38.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.498+0000 7f7814ff9700 1 -- 172.21.15.31:0/2162267158 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78180ac030 con 0x7f781806ef50 2023-10-06T05:19:38.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.745+0000 7f781efff700 1 -- 172.21.15.31:0/2162267158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78180ac030 con 0x7f781806ef50 2023-10-06T05:19:38.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.746+0000 7f7814ff9700 1 -- 172.21.15.31:0/2162267158 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f780c0531d0 con 0x7f781806ef50 2023-10-06T05:19:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:19:38.748 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:19:38.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.748+0000 7f781efff700 1 -- 172.21.15.31:0/2162267158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f77fc067170 msgr2=0x7f77fc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.748+0000 7f781efff700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f77fc067170 0x7f77fc069620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f78180ab4c0 tx=0x7f7804008040 comp rx=0 tx=0).stop 2023-10-06T05:19:38.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.748+0000 7f781efff700 1 -- 172.21.15.31:0/2162267158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781806ef50 msgr2=0x7f7818158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.748+0000 7f781efff700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781806ef50 0x7f7818158b10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f781806cd80 tx=0x7f780c017000 comp rx=0 tx=0).stop 2023-10-06T05:19:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.748+0000 7f781efff700 1 -- 172.21.15.31:0/2162267158 shutdown_connections 2023-10-06T05:19:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.748+0000 7f781efff700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78180c7f30 0x7f781816cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.748+0000 7f781efff700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f77fc067170 0x7f77fc069620 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.748+0000 7f781efff700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781806ef50 0x7f7818158b10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.748+0000 7f781efff700 1 --2- 172.21.15.31:0/2162267158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78180c7b60 0x7f78180ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.748+0000 7f781efff700 1 -- 172.21.15.31:0/2162267158 >> 172.21.15.31:0/2162267158 conn(0x7f78180cc0b0 msgr2=0x7f781806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.749+0000 7f781efff700 1 -- 172.21.15.31:0/2162267158 shutdown_connections 2023-10-06T05:19:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:38.749+0000 7f781efff700 1 -- 172.21.15.31:0/2162267158 wait complete. 2023-10-06T05:19:38.752 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:19:38.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:38.985 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":41.419333301000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.175250169921174,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:19:38.985 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.175250169921174 }, "state": "open", "uptime": 41.419333301 } ] 2023-10-06T05:19:38.986 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:19:43.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.193+0000 7f63663ff700 1 Processor -- start 2023-10-06T05:19:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.194+0000 7f63663ff700 1 -- start start 2023-10-06T05:19:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f63663ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63601595a0 0x7f6360159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f63663ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f63600cc0d0 0x7f6360068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f63663ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6360068bb0 0x7f63600c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f63663ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6360069010 con 0x7f63601595a0 2023-10-06T05:19:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f63663ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63600c95b0 con 0x7f63600cc0d0 2023-10-06T05:19:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f63663ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63600c9890 con 0x7f6360068bb0 2023-10-06T05:19:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f635ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63601595a0 0x7f6360159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f635ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63601595a0 0x7f6360159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58772/0 (socket says 172.21.15.31:58772) 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f635ffff700 1 -- 172.21.15.31:0/1022475320 learned_addr learned my addr 172.21.15.31:0/1022475320 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f636499c700 1 --2- 172.21.15.31:0/1022475320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6360068bb0 0x7f63600c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.195+0000 7f635f7fe700 1 --2- 172.21.15.31:0/1022475320 >> v2:172.21.15.62:3300/0 conn(0x7f63600cc0d0 0x7f6360068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.196+0000 7f635ffff700 1 -- 172.21.15.31:0/1022475320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6360068bb0 msgr2=0x7f63600c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.196+0000 7f635ffff700 1 --2- 172.21.15.31:0/1022475320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6360068bb0 0x7f63600c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.196+0000 7f635ffff700 1 -- 172.21.15.31:0/1022475320 >> v2:172.21.15.62:3300/0 conn(0x7f63600cc0d0 msgr2=0x7f6360068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.196+0000 7f635ffff700 1 --2- 172.21.15.31:0/1022475320 >> v2:172.21.15.62:3300/0 conn(0x7f63600cc0d0 0x7f6360068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.196+0000 7f635ffff700 1 -- 172.21.15.31:0/1022475320 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600c9b70 con 0x7f63601595a0 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.196+0000 7f635f7fe700 1 --2- 172.21.15.31:0/1022475320 >> v2:172.21.15.62:3300/0 conn(0x7f63600cc0d0 0x7f6360068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:19:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.196+0000 7f635ffff700 1 --2- 172.21.15.31:0/1022475320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63601595a0 0x7f6360159970 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f636006a360 tx=0x7f6350009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967a99f43c3b0413 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.196+0000 7f635effd700 1 -- 172.21.15.31:0/1022475320 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350009e70 con 0x7f63601595a0 2023-10-06T05:19:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.196+0000 7f635effd700 1 -- 172.21.15.31:0/1022475320 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635000b860 con 0x7f63601595a0 2023-10-06T05:19:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.196+0000 7f635effd700 1 -- 172.21.15.31:0/1022475320 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000bc00 con 0x7f63601595a0 2023-10-06T05:19:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.197+0000 7f63663ff700 1 -- 172.21.15.31:0/1022475320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63601595a0 msgr2=0x7f6360159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.197+0000 7f63663ff700 1 --2- 172.21.15.31:0/1022475320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63601595a0 0x7f6360159970 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f636006a360 tx=0x7f6350009250 comp rx=0 tx=0).stop 2023-10-06T05:19:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.197+0000 7f63663ff700 1 -- 172.21.15.31:0/1022475320 shutdown_connections 2023-10-06T05:19:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.197+0000 7f63663ff700 1 --2- 172.21.15.31:0/1022475320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6360068bb0 0x7f63600c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.197+0000 7f63663ff700 1 --2- 172.21.15.31:0/1022475320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63601595a0 0x7f6360159970 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.197+0000 7f63663ff700 1 --2- 172.21.15.31:0/1022475320 >> v2:172.21.15.62:3300/0 conn(0x7f63600cc0d0 0x7f6360068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.197+0000 7f63663ff700 1 -- 172.21.15.31:0/1022475320 >> 172.21.15.31:0/1022475320 conn(0x7f636006b620 msgr2=0x7f636006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.197+0000 7f63663ff700 1 -- 172.21.15.31:0/1022475320 shutdown_connections 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.197+0000 7f63663ff700 1 -- 172.21.15.31:0/1022475320 wait complete. 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.198+0000 7f63663ff700 1 Processor -- start 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.199+0000 7f63663ff700 1 -- start start 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.199+0000 7f63663ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6360068bb0 0x7f6360152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.199+0000 7f63663ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63600cc0d0 0x7f63600b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.199+0000 7f63663ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63601595a0 0x7f6360157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f63663ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63601530e0 con 0x7f63600cc0d0 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f63663ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600b1090 con 0x7f6360068bb0 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f63663ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63600b18f0 con 0x7f63601595a0 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63600cc0d0 0x7f63600b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63600cc0d0 0x7f63600b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58784/0 (socket says 172.21.15.31:58784) 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f635f7fe700 1 -- 172.21.15.31:0/3875395823 learned_addr learned my addr 172.21.15.31:0/3875395823 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f635f7fe700 1 -- 172.21.15.31:0/3875395823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63601595a0 msgr2=0x7f6360157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f635f7fe700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63601595a0 0x7f6360157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f635f7fe700 1 -- 172.21.15.31:0/3875395823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6360068bb0 msgr2=0x7f6360152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f635f7fe700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6360068bb0 0x7f6360152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f635f7fe700 1 -- 172.21.15.31:0/3875395823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63500094f0 con 0x7f63600cc0d0 2023-10-06T05:19:44.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.200+0000 7f635f7fe700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63600cc0d0 0x7f63600b1d80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f635400e8d0 tx=0x7f6354008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.201+0000 7f6347fff700 1 -- 172.21.15.31:0/3875395823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635400ee70 con 0x7f63600cc0d0 2023-10-06T05:19:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.201+0000 7f6347fff700 1 -- 172.21.15.31:0/3875395823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635400c5e0 con 0x7f63600cc0d0 2023-10-06T05:19:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.201+0000 7f6347fff700 1 -- 172.21.15.31:0/3875395823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635400c980 con 0x7f63600cc0d0 2023-10-06T05:19:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.201+0000 7f63663ff700 1 -- 172.21.15.31:0/3875395823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6360158990 con 0x7f63600cc0d0 2023-10-06T05:19:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.201+0000 7f63663ff700 1 -- 172.21.15.31:0/3875395823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63600b1580 con 0x7f63600cc0d0 2023-10-06T05:19:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.202+0000 7f63663ff700 1 -- 172.21.15.31:0/3875395823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636004c380 con 0x7f63600cc0d0 2023-10-06T05:19:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.203+0000 7f6347fff700 1 -- 172.21.15.31:0/3875395823 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6354013410 con 0x7f63600cc0d0 2023-10-06T05:19:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.204+0000 7f6347fff700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f634c067170 0x7f634c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.204+0000 7f6347fff700 1 -- 172.21.15.31:0/3875395823 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f63540849a0 con 0x7f63600cc0d0 2023-10-06T05:19:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.204+0000 7f635ffff700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f634c067170 0x7f634c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.205+0000 7f635ffff700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f634c067170 0x7f634c069620 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f636006a360 tx=0x7f6350017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:44.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.208+0000 7f6347fff700 1 -- 172.21.15.31:0/3875395823 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6354051630 con 0x7f63600cc0d0 2023-10-06T05:19:44.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.458+0000 7f63663ff700 1 -- 172.21.15.31:0/3875395823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63600b1a30 con 0x7f63600cc0d0 2023-10-06T05:19:44.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.458+0000 7f6347fff700 1 -- 172.21.15.31:0/3875395823 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f6354054c50 con 0x7f63600cc0d0 2023-10-06T05:19:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:19:44.461 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:19:44.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 -- 172.21.15.31:0/3875395823 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f634c067170 msgr2=0x7f634c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f634c067170 0x7f634c069620 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f636006a360 tx=0x7f6350017000 comp rx=0 tx=0).stop 2023-10-06T05:19:44.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 -- 172.21.15.31:0/3875395823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63600cc0d0 msgr2=0x7f63600b1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:44.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63600cc0d0 0x7f63600b1d80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f635400e8d0 tx=0x7f6354008040 comp rx=0 tx=0).stop 2023-10-06T05:19:44.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 -- 172.21.15.31:0/3875395823 shutdown_connections 2023-10-06T05:19:44.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63601595a0 0x7f6360157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6360068bb0 0x7f6360152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f634c067170 0x7f634c069620 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 --2- 172.21.15.31:0/3875395823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63600cc0d0 0x7f63600b1d80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:44.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 -- 172.21.15.31:0/3875395823 >> 172.21.15.31:0/3875395823 conn(0x7f636006b620 msgr2=0x7f63601548e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:44.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 -- 172.21.15.31:0/3875395823 shutdown_connections 2023-10-06T05:19:44.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:44.461+0000 7f63663ff700 1 -- 172.21.15.31:0/3875395823 wait complete. 2023-10-06T05:19:44.465 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:19:44.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:44.698 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":47.132241084999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.93692017193705,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:19:44.699 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.93692017193705 }, "state": "open", "uptime": 47.132241085 } ] 2023-10-06T05:19:44.699 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:19:49.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:49.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.904+0000 7f45c19ff700 1 Processor -- start 2023-10-06T05:19:49.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.904+0000 7f45c19ff700 1 -- start start 2023-10-06T05:19:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.905+0000 7f45c19ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f45bc1595a0 0x7f45bc159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.905+0000 7f45c19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc068d80 0x7f45bc0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.905+0000 7f45c19ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45bc0c95b0 con 0x7f45bc0cc0d0 2023-10-06T05:19:49.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.905+0000 7f45c19ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45bc0c9890 con 0x7f45bc1595a0 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.905+0000 7f45c19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0c9b70 con 0x7f45bc068d80 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45ba7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f45bc0cc0d0 0x7f45bc068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60638/0 (socket says 172.21.15.31:60638) 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45ba7fc700 1 -- 172.21.15.31:0/2907650093 learned_addr learned my addr 172.21.15.31:0/2907650093 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45baffd700 1 --2- 172.21.15.31:0/2907650093 >> v2:172.21.15.62:3300/0 conn(0x7f45bc1595a0 0x7f45bc159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45bb7fe700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc068d80 0x7f45bc0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026143296 0 0) 0x7f45bc0c95b0 con 0x7f45bc0cc0d0 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45a4003570 con 0x7f45bc0cc0d0 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 475801602 0 0) 0x7f45a4003570 con 0x7f45bc0cc0d0 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc068d80 msgr2=0x7f45bc0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45b9ffb700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc068d80 0x7f45bc0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 >> v2:172.21.15.62:3300/0 conn(0x7f45bc1595a0 msgr2=0x7f45bc159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45b9ffb700 1 --2- 172.21.15.31:0/2907650093 >> v2:172.21.15.62:3300/0 conn(0x7f45bc1595a0 0x7f45bc159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0ca130 con 0x7f45bc0cc0d0 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.906+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f45b0001470 con 0x7f45bc0cc0d0 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 >> v1:172.21.15.31:6789/0 conn(0x7f45bc0cc0d0 legacy=0x7f45bc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45b9ffb700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc068d80 0x7f45a4003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45b9ffb700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45a40040b0 0x7f45a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45b9ffb700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4004d50 0x7f45a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0c9890 con 0x7f45a4004d50 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0c9b70 con 0x7f45a40040b0 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a400bb60 con 0x7f45bc068d80 2023-10-06T05:19:49.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45bb7fe700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4004d50 0x7f45a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45baffd700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc068d80 0x7f45a4003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45ba7fc700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45a40040b0 0x7f45a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45bb7fe700 1 -- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc068d80 msgr2=0x7f45a4003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45bb7fe700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc068d80 0x7f45a4003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45bb7fe700 1 -- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45a40040b0 msgr2=0x7f45a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45bb7fe700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45a40040b0 0x7f45a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45bb7fe700 1 -- 172.21.15.31:0/2907650093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a4003570 con 0x7f45a4004d50 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.907+0000 7f45baffd700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc068d80 0x7f45a4003ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.908+0000 7f45bb7fe700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4004d50 0x7f45a40051c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f45b40094d0 tx=0x7f45b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d911219283251afe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.908+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400ea60 con 0x7f45a4004d50 2023-10-06T05:19:49.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.908+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b400f040 con 0x7f45a4004d50 2023-10-06T05:19:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.908+0000 7f45b9ffb700 1 -- 172.21.15.31:0/2907650093 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400edf0 con 0x7f45a4004d50 2023-10-06T05:19:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.908+0000 7f45c19ff700 1 -- 172.21.15.31:0/2907650093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4004d50 msgr2=0x7f45a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.908+0000 7f45c19ff700 1 --2- 172.21.15.31:0/2907650093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4004d50 0x7f45a40051c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f45b40094d0 tx=0x7f45b400b040 comp rx=0 tx=0).stop 2023-10-06T05:19:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.908+0000 7f45bb7fe700 1 -- 172.21.15.31:0/2907650093 reap_dead start 2023-10-06T05:19:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.909+0000 7f45c19ff700 1 -- 172.21.15.31:0/2907650093 shutdown_connections 2023-10-06T05:19:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.909+0000 7f45c19ff700 1 -- 172.21.15.31:0/2907650093 >> 172.21.15.31:0/2907650093 conn(0x7f45bc06b620 msgr2=0x7f45bc06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.909+0000 7f45c19ff700 1 -- 172.21.15.31:0/2907650093 shutdown_connections 2023-10-06T05:19:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.909+0000 7f45c19ff700 1 -- 172.21.15.31:0/2907650093 wait complete. 2023-10-06T05:19:49.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.910+0000 7f45c19ff700 1 Processor -- start 2023-10-06T05:19:49.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.910+0000 7f45c19ff700 1 -- start start 2023-10-06T05:19:49.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.911+0000 7f45c19ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc14cf10 0x7f45bc157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.911+0000 7f45c19ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc14d2e0 0x7f45bc157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.911+0000 7f45c19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc0b2ff0 0x7f45bc0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.911+0000 7f45c19ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc16d200 con 0x7f45bc14cf10 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.911+0000 7f45c19ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc16d4e0 con 0x7f45bc14d2e0 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.911+0000 7f45c19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc16d7c0 con 0x7f45bc0b2ff0 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.911+0000 7f45baffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc14cf10 0x7f45bc157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.911+0000 7f45baffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc14cf10 0x7f45bc157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40570/0 (socket says 172.21.15.31:40570) 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.911+0000 7f45baffd700 1 -- 172.21.15.31:0/737257854 learned_addr learned my addr 172.21.15.31:0/737257854 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45baffd700 1 -- 172.21.15.31:0/737257854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc0b2ff0 msgr2=0x7f45bc0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45baffd700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc0b2ff0 0x7f45bc0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:49.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45baffd700 1 -- 172.21.15.31:0/737257854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc14d2e0 msgr2=0x7f45bc157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45baffd700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc14d2e0 0x7f45bc157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45baffd700 1 -- 172.21.15.31:0/737257854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc16daa0 con 0x7f45bc14cf10 2023-10-06T05:19:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45baffd700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc14cf10 0x7f45bc157a00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f45ac011d70 tx=0x7f45ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45a2ffd700 1 -- 172.21.15.31:0/737257854 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00fcd0 con 0x7f45bc14cf10 2023-10-06T05:19:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45a2ffd700 1 -- 172.21.15.31:0/737257854 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac00fe30 con 0x7f45bc14cf10 2023-10-06T05:19:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45a2ffd700 1 -- 172.21.15.31:0/737257854 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac014670 con 0x7f45bc14cf10 2023-10-06T05:19:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45c19ff700 1 -- 172.21.15.31:0/737257854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc16dd80 con 0x7f45bc14cf10 2023-10-06T05:19:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.912+0000 7f45c19ff700 1 -- 172.21.15.31:0/737257854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc150d60 con 0x7f45bc14cf10 2023-10-06T05:19:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.913+0000 7f45c19ff700 1 -- 172.21.15.31:0/737257854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc16dfc0 con 0x7f45bc14cf10 2023-10-06T05:19:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.914+0000 7f45a2ffd700 1 -- 172.21.15.31:0/737257854 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f45ac014a60 con 0x7f45bc14cf10 2023-10-06T05:19:49.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.915+0000 7f45a2ffd700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f45a80671a0 0x7f45a8069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:49.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.915+0000 7f45a2ffd700 1 -- 172.21.15.31:0/737257854 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f45ac018070 con 0x7f45bc14cf10 2023-10-06T05:19:49.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.916+0000 7f45ba7fc700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f45a80671a0 0x7f45a8069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:49.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.916+0000 7f45ba7fc700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f45a80671a0 0x7f45a8069650 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f45b0001ed0 tx=0x7f45b0001dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:49.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:49.920+0000 7f45a2ffd700 1 -- 172.21.15.31:0/737257854 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45bc16dfc0 con 0x7f45bc14cf10 2023-10-06T05:19:50.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.168+0000 7f45c19ff700 1 -- 172.21.15.31:0/737257854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45bc16df70 con 0x7f45bc14cf10 2023-10-06T05:19:50.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.168+0000 7f45a2ffd700 1 -- 172.21.15.31:0/737257854 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f45ac052440 con 0x7f45bc14cf10 2023-10-06T05:19:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:19:50.171 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:19:50.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 -- 172.21.15.31:0/737257854 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f45a80671a0 msgr2=0x7f45a8069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:50.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f45a80671a0 0x7f45a8069650 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f45b0001ed0 tx=0x7f45b0001dc0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 -- 172.21.15.31:0/737257854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc14cf10 msgr2=0x7f45bc157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:50.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc14cf10 0x7f45bc157a00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f45ac011d70 tx=0x7f45ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 -- 172.21.15.31:0/737257854 shutdown_connections 2023-10-06T05:19:50.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45bc0b2ff0 0x7f45bc0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f45a80671a0 0x7f45a8069650 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc14cf10 0x7f45bc157a00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 --2- 172.21.15.31:0/737257854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc14d2e0 0x7f45bc157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:50.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 -- 172.21.15.31:0/737257854 >> 172.21.15.31:0/737257854 conn(0x7f45bc06b620 msgr2=0x7f45bc15b340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:50.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 -- 172.21.15.31:0/737257854 shutdown_connections 2023-10-06T05:19:50.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:50.171+0000 7f45c19ff700 1 -- 172.21.15.31:0/737257854 wait complete. 2023-10-06T05:19:50.174 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:19:50.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:50.408 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":52.842148919000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.701838065558039,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:19:50.408 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.70183806555804 }, "state": "open", "uptime": 52.842148919 } ] 2023-10-06T05:19:50.409 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:19:55.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:19:55.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.614+0000 7ffb4cfff700 1 Processor -- start 2023-10-06T05:19:55.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.614+0000 7ffb4cfff700 1 -- start start 2023-10-06T05:19:55.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.616+0000 7ffb4cfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb481595a0 0x7ffb48159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:55.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.616+0000 7ffb4cfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb480cc0d0 0x7ffb48068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.616+0000 7ffb4cfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb48069010 con 0x7ffb48068bb0 2023-10-06T05:19:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.616+0000 7ffb4cfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb480c95b0 con 0x7ffb480cc0d0 2023-10-06T05:19:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.616+0000 7ffb4cfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480c9890 con 0x7ffb481595a0 2023-10-06T05:19:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.616+0000 7ffb45d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb480cc0d0 0x7ffb48068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.616+0000 7ffb45d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb480cc0d0 0x7ffb48068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40580/0 (socket says 172.21.15.31:40580) 2023-10-06T05:19:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.616+0000 7ffb45d9b700 1 -- 172.21.15.31:0/2675533575 learned_addr learned my addr 172.21.15.31:0/2675533575 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.616+0000 7ffb4659c700 1 --2- 172.21.15.31:0/2675533575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb481595a0 0x7ffb48159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:55.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.617+0000 7ffb45d9b700 1 -- 172.21.15.31:0/2675533575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb481595a0 msgr2=0x7ffb48159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:19:55.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.617+0000 7ffb45d9b700 1 --2- 172.21.15.31:0/2675533575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb481595a0 0x7ffb48159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.617+0000 7ffb45d9b700 1 -- 172.21.15.31:0/2675533575 >> v1:172.21.15.62:6789/0 conn(0x7ffb48068bb0 legacy=0x7ffb480c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.617+0000 7ffb45d9b700 1 -- 172.21.15.31:0/2675533575 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb480c9b70 con 0x7ffb480cc0d0 2023-10-06T05:19:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.617+0000 7ffb45d9b700 1 --2- 172.21.15.31:0/2675533575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 0x7ffb48068670 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0099b0 tx=0x7ffb3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8073aa6251a1b5a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.617+0000 7ffb4559a700 1 -- 172.21.15.31:0/2675533575 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00b860 con 0x7ffb480cc0d0 2023-10-06T05:19:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.617+0000 7ffb4559a700 1 -- 172.21.15.31:0/2675533575 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c00b9c0 con 0x7ffb480cc0d0 2023-10-06T05:19:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.617+0000 7ffb4559a700 1 -- 172.21.15.31:0/2675533575 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00bd60 con 0x7ffb480cc0d0 2023-10-06T05:19:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.618+0000 7ffb4cfff700 1 -- 172.21.15.31:0/2675533575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 msgr2=0x7ffb48068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.618+0000 7ffb4cfff700 1 --2- 172.21.15.31:0/2675533575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 0x7ffb48068670 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0099b0 tx=0x7ffb3c009250 comp rx=0 tx=0).stop 2023-10-06T05:19:55.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.618+0000 7ffb4cfff700 1 -- 172.21.15.31:0/2675533575 shutdown_connections 2023-10-06T05:19:55.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.618+0000 7ffb4cfff700 1 --2- 172.21.15.31:0/2675533575 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb481595a0 0x7ffb48159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.618+0000 7ffb4cfff700 1 --2- 172.21.15.31:0/2675533575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 0x7ffb48068670 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.618+0000 7ffb4cfff700 1 -- 172.21.15.31:0/2675533575 >> 172.21.15.31:0/2675533575 conn(0x7ffb4806b620 msgr2=0x7ffb4806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:55.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.618+0000 7ffb4cfff700 1 -- 172.21.15.31:0/2675533575 shutdown_connections 2023-10-06T05:19:55.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.618+0000 7ffb4cfff700 1 -- 172.21.15.31:0/2675533575 wait complete. 2023-10-06T05:19:55.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.619+0000 7ffb4cfff700 1 Processor -- start 2023-10-06T05:19:55.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.620+0000 7ffb4cfff700 1 -- start start 2023-10-06T05:19:55.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.620+0000 7ffb4cfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb48068bb0 0x7ffb480b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:55.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb4cfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 0x7ffb480c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:55.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb4cfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb481595a0 0x7ffb4813e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:55.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb4cfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480c74f0 con 0x7ffb480cc0d0 2023-10-06T05:19:55.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb4cfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480c77d0 con 0x7ffb481595a0 2023-10-06T05:19:55.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb4cfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480c7ab0 con 0x7ffb48068bb0 2023-10-06T05:19:55.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb45d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 0x7ffb480c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:55.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb45d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 0x7ffb480c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40592/0 (socket says 172.21.15.31:40592) 2023-10-06T05:19:55.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb45d9b700 1 -- 172.21.15.31:0/1534781080 learned_addr learned my addr 172.21.15.31:0/1534781080 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:19:55.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb4659c700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb48068bb0 0x7ffb480b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:55.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb45d9b700 1 -- 172.21.15.31:0/1534781080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb48068bb0 msgr2=0x7ffb480b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:55.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb45d9b700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb48068bb0 0x7ffb480b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb46d9d700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb481595a0 0x7ffb4813e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:55.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb45d9b700 1 -- 172.21.15.31:0/1534781080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb481595a0 msgr2=0x7ffb4813e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:55.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb45d9b700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb481595a0 0x7ffb4813e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb45d9b700 1 -- 172.21.15.31:0/1534781080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c0094f0 con 0x7ffb480cc0d0 2023-10-06T05:19:55.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.621+0000 7ffb46d9d700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb481595a0 0x7ffb4813e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:19:55.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.622+0000 7ffb45d9b700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 0x7ffb480c5560 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ffb4806ee00 tx=0x7ffb3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:55.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.622+0000 7ffb367fc700 1 -- 172.21.15.31:0/1534781080 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c024440 con 0x7ffb480cc0d0 2023-10-06T05:19:55.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.622+0000 7ffb367fc700 1 -- 172.21.15.31:0/1534781080 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c0245c0 con 0x7ffb480cc0d0 2023-10-06T05:19:55.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.622+0000 7ffb367fc700 1 -- 172.21.15.31:0/1534781080 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c024980 con 0x7ffb480cc0d0 2023-10-06T05:19:55.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.622+0000 7ffb4cfff700 1 -- 172.21.15.31:0/1534781080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb480c7d90 con 0x7ffb480cc0d0 2023-10-06T05:19:55.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.622+0000 7ffb4cfff700 1 -- 172.21.15.31:0/1534781080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb480c5aa0 con 0x7ffb480cc0d0 2023-10-06T05:19:55.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.623+0000 7ffb4cfff700 1 -- 172.21.15.31:0/1534781080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4804c380 con 0x7ffb480cc0d0 2023-10-06T05:19:55.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.624+0000 7ffb367fc700 1 -- 172.21.15.31:0/1534781080 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7ffb3c01a420 con 0x7ffb480cc0d0 2023-10-06T05:19:55.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.625+0000 7ffb367fc700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ffb28067170 0x7ffb28069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:19:55.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.625+0000 7ffb367fc700 1 -- 172.21.15.31:0/1534781080 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7ffb3c086850 con 0x7ffb480cc0d0 2023-10-06T05:19:55.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.625+0000 7ffb4659c700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ffb28067170 0x7ffb28069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:19:55.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.626+0000 7ffb4659c700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ffb28067170 0x7ffb28069620 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb380098e0 tx=0x7ffb38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:19:55.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.630+0000 7ffb367fc700 1 -- 172.21.15.31:0/1534781080 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb3c0535f0 con 0x7ffb480cc0d0 2023-10-06T05:19:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.874+0000 7ffb4cfff700 1 -- 172.21.15.31:0/1534781080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb480c5cc0 con 0x7ffb480cc0d0 2023-10-06T05:19:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.875+0000 7ffb367fc700 1 -- 172.21.15.31:0/1534781080 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7ffb3c056c10 con 0x7ffb480cc0d0 2023-10-06T05:19:55.877 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:19:55.877 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:19:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.877+0000 7ffb4cfff700 1 -- 172.21.15.31:0/1534781080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ffb28067170 msgr2=0x7ffb28069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.877+0000 7ffb4cfff700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ffb28067170 0x7ffb28069620 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb380098e0 tx=0x7ffb38008040 comp rx=0 tx=0).stop 2023-10-06T05:19:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.877+0000 7ffb4cfff700 1 -- 172.21.15.31:0/1534781080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 msgr2=0x7ffb480c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:19:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.877+0000 7ffb4cfff700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 0x7ffb480c5560 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ffb4806ee00 tx=0x7ffb3c017000 comp rx=0 tx=0).stop 2023-10-06T05:19:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.877+0000 7ffb4cfff700 1 -- 172.21.15.31:0/1534781080 shutdown_connections 2023-10-06T05:19:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.877+0000 7ffb4cfff700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb481595a0 0x7ffb4813e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.877+0000 7ffb4cfff700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb48068bb0 0x7ffb480b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.877+0000 7ffb4cfff700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ffb28067170 0x7ffb28069620 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.877+0000 7ffb4cfff700 1 --2- 172.21.15.31:0/1534781080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb480cc0d0 0x7ffb480c5560 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:19:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.877+0000 7ffb4cfff700 1 -- 172.21.15.31:0/1534781080 >> 172.21.15.31:0/1534781080 conn(0x7ffb4806b620 msgr2=0x7ffb481537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:19:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.878+0000 7ffb4cfff700 1 -- 172.21.15.31:0/1534781080 shutdown_connections 2023-10-06T05:19:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:19:55.878+0000 7ffb4cfff700 1 -- 172.21.15.31:0/1534781080 wait complete. 2023-10-06T05:19:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:19:55.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:19:56.115 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":58.549056798000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.469958046436787,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:19:56.116 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.469958046436787 }, "state": "open", "uptime": 58.549056798 } ] 2023-10-06T05:19:56.117 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:20:00.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:00.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:01.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:01.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.319+0000 7f9e4edff700 1 Processor -- start 2023-10-06T05:20:01.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.319+0000 7f9e4edff700 1 -- start start 2023-10-06T05:20:01.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.321+0000 7f9e4edff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9e480c7b60 0x7f9e48069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.321+0000 7f9e4edff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 0x7f9e4813f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.321+0000 7f9e4edff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e480ca5b0 con 0x7f9e480c7f30 2023-10-06T05:20:01.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.321+0000 7f9e4edff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e480af2d0 con 0x7f9e480c7b60 2023-10-06T05:20:01.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.321+0000 7f9e4edff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4806f740 con 0x7f9e480aee70 2023-10-06T05:20:01.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.321+0000 7f9e47fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9e480c7f30 0x7f9e4806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39624/0 (socket says 172.21.15.31:39624) 2023-10-06T05:20:01.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.321+0000 7f9e47fff700 1 -- 172.21.15.31:0/2879156182 learned_addr learned my addr 172.21.15.31:0/2879156182 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:01.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.321+0000 7f9e4d39c700 1 --2- 172.21.15.31:0/2879156182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 0x7f9e4813f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.321+0000 7f9e4cb9b700 1 --2- 172.21.15.31:0/2879156182 >> v2:172.21.15.62:3300/0 conn(0x7f9e480c7b60 0x7f9e48069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.322+0000 7f9e477fe700 1 -- 172.21.15.31:0/2879156182 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704759357 0 0) 0x7f9e480ca5b0 con 0x7f9e480c7f30 2023-10-06T05:20:01.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.322+0000 7f9e477fe700 1 -- 172.21.15.31:0/2879156182 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e38003570 con 0x7f9e480c7f30 2023-10-06T05:20:01.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.322+0000 7f9e4cb9b700 1 -- 172.21.15.31:0/2879156182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 msgr2=0x7f9e4813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:01.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.322+0000 7f9e4cb9b700 1 --2- 172.21.15.31:0/2879156182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 0x7f9e4813f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.322+0000 7f9e4cb9b700 1 -- 172.21.15.31:0/2879156182 >> v1:172.21.15.31:6789/0 conn(0x7f9e480c7f30 legacy=0x7f9e4806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.322+0000 7f9e477fe700 1 -- 172.21.15.31:0/2879156182 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2441833510 0 0) 0x7f9e38003570 con 0x7f9e480c7f30 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.322+0000 7f9e4cb9b700 1 -- 172.21.15.31:0/2879156182 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e480c9c80 con 0x7f9e480c7b60 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.322+0000 7f9e4d39c700 1 --2- 172.21.15.31:0/2879156182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 0x7f9e4813f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.322+0000 7f9e4cb9b700 1 --2- 172.21.15.31:0/2879156182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e480c7b60 0x7f9e48069880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0098d0 tx=0x7f9e3c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55b3ca49c67db5e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.322+0000 7f9e477fe700 1 -- 172.21.15.31:0/2879156182 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34001470 con 0x7f9e480c7b60 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.323+0000 7f9e477fe700 1 -- 172.21.15.31:0/2879156182 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c011420 con 0x7f9e480c7b60 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.323+0000 7f9e477fe700 1 -- 172.21.15.31:0/2879156182 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0117c0 con 0x7f9e480c7b60 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.323+0000 7f9e4edff700 1 -- 172.21.15.31:0/2879156182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e480c7b60 msgr2=0x7f9e48069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.323+0000 7f9e4edff700 1 --2- 172.21.15.31:0/2879156182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e480c7b60 0x7f9e48069880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0098d0 tx=0x7f9e3c009120 comp rx=0 tx=0).stop 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.323+0000 7f9e4edff700 1 -- 172.21.15.31:0/2879156182 shutdown_connections 2023-10-06T05:20:01.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.323+0000 7f9e4edff700 1 --2- 172.21.15.31:0/2879156182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e480c7b60 0x7f9e48069880 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.323+0000 7f9e4edff700 1 --2- 172.21.15.31:0/2879156182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 0x7f9e4813f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.323+0000 7f9e4edff700 1 -- 172.21.15.31:0/2879156182 >> 172.21.15.31:0/2879156182 conn(0x7f9e480cc0b0 msgr2=0x7f9e480cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:01.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.323+0000 7f9e4edff700 1 -- 172.21.15.31:0/2879156182 shutdown_connections 2023-10-06T05:20:01.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.323+0000 7f9e4edff700 1 -- 172.21.15.31:0/2879156182 wait complete. 2023-10-06T05:20:01.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.325+0000 7f9e4edff700 1 Processor -- start 2023-10-06T05:20:01.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.325+0000 7f9e4edff700 1 -- start start 2023-10-06T05:20:01.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4edff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 0x7f9e4813df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4edff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e480c7b60 0x7f9e4815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4edff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e480c7f30 0x7f9e4815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4edff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480aff60 con 0x7f9e480c7f30 2023-10-06T05:20:01.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4edff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e48155b10 con 0x7f9e480c7b60 2023-10-06T05:20:01.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4edff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4813e460 con 0x7f9e480aee70 2023-10-06T05:20:01.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4d39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e480c7f30 0x7f9e4815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4d39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e480c7f30 0x7f9e4815e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33722/0 (socket says 172.21.15.31:33722) 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4d39c700 1 -- 172.21.15.31:0/2755470121 learned_addr learned my addr 172.21.15.31:0/2755470121 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e47fff700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e480c7b60 0x7f9e4815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4cb9b700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 0x7f9e4813df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4d39c700 1 -- 172.21.15.31:0/2755470121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 msgr2=0x7f9e4813df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.326+0000 7f9e4d39c700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 0x7f9e4813df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.327+0000 7f9e4d39c700 1 -- 172.21.15.31:0/2755470121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e480c7b60 msgr2=0x7f9e4815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.327+0000 7f9e4d39c700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e480c7b60 0x7f9e4815dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.327+0000 7f9e4d39c700 1 -- 172.21.15.31:0/2755470121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e40009a20 con 0x7f9e480c7f30 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.327+0000 7f9e47fff700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e480c7b60 0x7f9e4815dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.327+0000 7f9e4d39c700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e480c7f30 0x7f9e4815e1a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009fc0 tx=0x7f9e4000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:01.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.327+0000 7f9e44ff9700 1 -- 172.21.15.31:0/2755470121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40009d20 con 0x7f9e480c7f30 2023-10-06T05:20:01.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.327+0000 7f9e44ff9700 1 -- 172.21.15.31:0/2755470121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4000f040 con 0x7f9e480c7f30 2023-10-06T05:20:01.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.327+0000 7f9e44ff9700 1 -- 172.21.15.31:0/2755470121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40014480 con 0x7f9e480c7f30 2023-10-06T05:20:01.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.327+0000 7f9e4edff700 1 -- 172.21.15.31:0/2755470121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c0093c0 con 0x7f9e480c7f30 2023-10-06T05:20:01.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.327+0000 7f9e4edff700 1 -- 172.21.15.31:0/2755470121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e480af910 con 0x7f9e480c7f30 2023-10-06T05:20:01.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.328+0000 7f9e4edff700 1 -- 172.21.15.31:0/2755470121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4804c380 con 0x7f9e480c7f30 2023-10-06T05:20:01.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.329+0000 7f9e44ff9700 1 -- 172.21.15.31:0/2755470121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f9e40003020 con 0x7f9e480c7f30 2023-10-06T05:20:01.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.330+0000 7f9e44ff9700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9e2c067170 0x7f9e2c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:01.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.330+0000 7f9e44ff9700 1 -- 172.21.15.31:0/2755470121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f9e400855d0 con 0x7f9e480c7f30 2023-10-06T05:20:01.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.330+0000 7f9e4cb9b700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9e2c067170 0x7f9e2c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:01.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.331+0000 7f9e4cb9b700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9e2c067170 0x7f9e2c069620 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0030c0 tx=0x7f9e3c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:01.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.334+0000 7f9e44ff9700 1 -- 172.21.15.31:0/2755470121 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e40052190 con 0x7f9e480c7f30 2023-10-06T05:20:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.579+0000 7f9e4edff700 1 -- 172.21.15.31:0/2755470121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e480c9f60 con 0x7f9e480c7f30 2023-10-06T05:20:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.580+0000 7f9e44ff9700 1 -- 172.21.15.31:0/2755470121 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f9e400557b0 con 0x7f9e480c7f30 2023-10-06T05:20:01.582 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:01.582 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:20:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.582+0000 7f9e4edff700 1 -- 172.21.15.31:0/2755470121 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9e2c067170 msgr2=0x7f9e2c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.582+0000 7f9e4edff700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9e2c067170 0x7f9e2c069620 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0030c0 tx=0x7f9e3c016000 comp rx=0 tx=0).stop 2023-10-06T05:20:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.582+0000 7f9e4edff700 1 -- 172.21.15.31:0/2755470121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e480c7f30 msgr2=0x7f9e4815e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.582+0000 7f9e4edff700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e480c7f30 0x7f9e4815e1a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009fc0 tx=0x7f9e4000b040 comp rx=0 tx=0).stop 2023-10-06T05:20:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.583+0000 7f9e4edff700 1 -- 172.21.15.31:0/2755470121 shutdown_connections 2023-10-06T05:20:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.583+0000 7f9e4edff700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9e2c067170 0x7f9e2c069620 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.583+0000 7f9e4edff700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e480c7f30 0x7f9e4815e1a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.583+0000 7f9e4edff700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e480aee70 0x7f9e4813df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.583+0000 7f9e4edff700 1 --2- 172.21.15.31:0/2755470121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e480c7b60 0x7f9e4815dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.583+0000 7f9e4edff700 1 -- 172.21.15.31:0/2755470121 >> 172.21.15.31:0/2755470121 conn(0x7f9e480cc0b0 msgr2=0x7f9e480c1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.583+0000 7f9e4edff700 1 -- 172.21.15.31:0/2755470121 shutdown_connections 2023-10-06T05:20:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:01.583+0000 7f9e4edff700 1 -- 172.21.15.31:0/2755470121 wait complete. 2023-10-06T05:20:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:20:01.602 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:20:01.819 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":64.252964727000005,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.241234976125419,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:20:01.910 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.24123497612542 }, "state": "open", "uptime": 64.252964727 } ] 2023-10-06T05:20:01.911 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:20:06.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:07.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.117+0000 7ff0427ff700 1 Processor -- start 2023-10-06T05:20:07.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.118+0000 7ff0427ff700 1 -- start start 2023-10-06T05:20:07.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.119+0000 7ff0427ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff03c0c7f30 0x7ff03c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.119+0000 7ff0427ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff03c0aee70 0x7ff03c13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.119+0000 7ff0427ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03c0ca5b0 con 0x7ff03c0c7b60 2023-10-06T05:20:07.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.119+0000 7ff0427ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03c0af2f0 con 0x7ff03c0aee70 2023-10-06T05:20:07.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.119+0000 7ff0427ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03c06f740 con 0x7ff03c0c7f30 2023-10-06T05:20:07.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.119+0000 7ff040d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff03c0aee70 0x7ff03c13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.119+0000 7ff03bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff03c0c7b60 0x7ff03c06ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49138/0 (socket says 172.21.15.31:49138) 2023-10-06T05:20:07.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.119+0000 7ff03bfff700 1 -- 172.21.15.31:0/3619531617 learned_addr learned my addr 172.21.15.31:0/3619531617 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:07.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.119+0000 7ff03b7fe700 1 --2- 172.21.15.31:0/3619531617 >> v2:172.21.15.62:3300/0 conn(0x7ff03c0c7f30 0x7ff03c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.120+0000 7ff040d9c700 1 -- 172.21.15.31:0/3619531617 >> v2:172.21.15.62:3300/0 conn(0x7ff03c0c7f30 msgr2=0x7ff03c069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:07.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.120+0000 7ff040d9c700 1 --2- 172.21.15.31:0/3619531617 >> v2:172.21.15.62:3300/0 conn(0x7ff03c0c7f30 0x7ff03c069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.120+0000 7ff040d9c700 1 -- 172.21.15.31:0/3619531617 >> v1:172.21.15.31:6789/0 conn(0x7ff03c0c7b60 legacy=0x7ff03c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.120+0000 7ff040d9c700 1 -- 172.21.15.31:0/3619531617 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c0c9c80 con 0x7ff03c0aee70 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.120+0000 7ff03affd700 1 -- 172.21.15.31:0/3619531617 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101468396 0 0) 0x7ff03c0ca5b0 con 0x7ff03c0c7b60 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.120+0000 7ff040d9c700 1 --2- 172.21.15.31:0/3619531617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 0x7ff03c13f690 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff03c06c990 tx=0x7ff034009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eff9bef22afb7b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.120+0000 7ff03affd700 1 -- 172.21.15.31:0/3619531617 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03400e070 con 0x7ff03c0aee70 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.120+0000 7ff03affd700 1 -- 172.21.15.31:0/3619531617 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03400bb60 con 0x7ff03c0aee70 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.120+0000 7ff03affd700 1 -- 172.21.15.31:0/3619531617 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034012430 con 0x7ff03c0aee70 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.121+0000 7ff0427ff700 1 -- 172.21.15.31:0/3619531617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 msgr2=0x7ff03c13f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.121+0000 7ff0427ff700 1 --2- 172.21.15.31:0/3619531617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 0x7ff03c13f690 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff03c06c990 tx=0x7ff034009780 comp rx=0 tx=0).stop 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.121+0000 7ff0427ff700 1 -- 172.21.15.31:0/3619531617 shutdown_connections 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.121+0000 7ff0427ff700 1 --2- 172.21.15.31:0/3619531617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 0x7ff03c13f690 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.121+0000 7ff0427ff700 1 --2- 172.21.15.31:0/3619531617 >> v2:172.21.15.62:3300/0 conn(0x7ff03c0c7f30 0x7ff03c069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.121+0000 7ff0427ff700 1 -- 172.21.15.31:0/3619531617 >> 172.21.15.31:0/3619531617 conn(0x7ff03c0cc0b0 msgr2=0x7ff03c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:07.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.121+0000 7ff0427ff700 1 -- 172.21.15.31:0/3619531617 shutdown_connections 2023-10-06T05:20:07.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.121+0000 7ff0427ff700 1 -- 172.21.15.31:0/3619531617 wait complete. 2023-10-06T05:20:07.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.123+0000 7ff0427ff700 1 Processor -- start 2023-10-06T05:20:07.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.123+0000 7ff0427ff700 1 -- start start 2023-10-06T05:20:07.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff0427ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 0x7ff03c0c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff0427ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff03c0c7b60 0x7ff03c157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff0427ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff03c0c7f30 0x7ff03c159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff0427ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c158580 con 0x7ff03c0aee70 2023-10-06T05:20:07.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff0427ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0acec0 con 0x7ff03c0c7f30 2023-10-06T05:20:07.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff0427ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0c6ff0 con 0x7ff03c0c7b60 2023-10-06T05:20:07.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff03bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 0x7ff03c0c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff03bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 0x7ff03c0c6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41540/0 (socket says 172.21.15.31:41540) 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff03bfff700 1 -- 172.21.15.31:0/2367260107 learned_addr learned my addr 172.21.15.31:0/2367260107 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff040d9c700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff03c0c7f30 0x7ff03c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff03b7fe700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff03c0c7b60 0x7ff03c157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff03bfff700 1 -- 172.21.15.31:0/2367260107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff03c0c7b60 msgr2=0x7ff03c157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff03bfff700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff03c0c7b60 0x7ff03c157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff03bfff700 1 -- 172.21.15.31:0/2367260107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff03c0c7f30 msgr2=0x7ff03c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff03bfff700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff03c0c7f30 0x7ff03c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.125+0000 7ff03bfff700 1 -- 172.21.15.31:0/2367260107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff034009a20 con 0x7ff03c0aee70 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.126+0000 7ff03bfff700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 0x7ff03c0c6ab0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00c7e0 tx=0x7ff02c007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.126+0000 7ff023fff700 1 -- 172.21.15.31:0/2367260107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c007300 con 0x7ff03c0aee70 2023-10-06T05:20:07.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.126+0000 7ff023fff700 1 -- 172.21.15.31:0/2367260107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c007460 con 0x7ff03c0aee70 2023-10-06T05:20:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.126+0000 7ff0427ff700 1 -- 172.21.15.31:0/2367260107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03c157ce0 con 0x7ff03c0aee70 2023-10-06T05:20:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.126+0000 7ff023fff700 1 -- 172.21.15.31:0/2367260107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0057f0 con 0x7ff03c0aee70 2023-10-06T05:20:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.126+0000 7ff0427ff700 1 -- 172.21.15.31:0/2367260107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c158320 con 0x7ff03c0aee70 2023-10-06T05:20:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.127+0000 7ff0427ff700 1 -- 172.21.15.31:0/2367260107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff03c04c380 con 0x7ff03c0aee70 2023-10-06T05:20:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.128+0000 7ff023fff700 1 -- 172.21.15.31:0/2367260107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7ff02c005c00 con 0x7ff03c0aee70 2023-10-06T05:20:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.129+0000 7ff023fff700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff028067170 0x7ff028069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.129+0000 7ff023fff700 1 -- 172.21.15.31:0/2367260107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7ff02c0845a0 con 0x7ff03c0aee70 2023-10-06T05:20:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.129+0000 7ff03b7fe700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff028067170 0x7ff028069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.130+0000 7ff03b7fe700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff028067170 0x7ff028069620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff03c16a3f0 tx=0x7ff030008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:07.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.133+0000 7ff023fff700 1 -- 172.21.15.31:0/2367260107 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff02c0513c0 con 0x7ff03c0aee70 2023-10-06T05:20:07.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.381+0000 7ff0427ff700 1 -- 172.21.15.31:0/2367260107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff03c0c9f60 con 0x7ff03c0aee70 2023-10-06T05:20:07.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.382+0000 7ff023fff700 1 -- 172.21.15.31:0/2367260107 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7ff02c0549e0 con 0x7ff03c0aee70 2023-10-06T05:20:07.384 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:07.384 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:20:07.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.384+0000 7ff0427ff700 1 -- 172.21.15.31:0/2367260107 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff028067170 msgr2=0x7ff028069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.384+0000 7ff0427ff700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff028067170 0x7ff028069620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff03c16a3f0 tx=0x7ff030008040 comp rx=0 tx=0).stop 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.384+0000 7ff0427ff700 1 -- 172.21.15.31:0/2367260107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 msgr2=0x7ff03c0c6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.384+0000 7ff0427ff700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 0x7ff03c0c6ab0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00c7e0 tx=0x7ff02c007c80 comp rx=0 tx=0).stop 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.384+0000 7ff0427ff700 1 -- 172.21.15.31:0/2367260107 shutdown_connections 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.384+0000 7ff0427ff700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff03c0c7f30 0x7ff03c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.384+0000 7ff0427ff700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff028067170 0x7ff028069620 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.384+0000 7ff0427ff700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03c0aee70 0x7ff03c0c6ab0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.384+0000 7ff0427ff700 1 --2- 172.21.15.31:0/2367260107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff03c0c7b60 0x7ff03c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.384+0000 7ff0427ff700 1 -- 172.21.15.31:0/2367260107 >> 172.21.15.31:0/2367260107 conn(0x7ff03c0cc0b0 msgr2=0x7ff03c0c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.385+0000 7ff0427ff700 1 -- 172.21.15.31:0/2367260107 shutdown_connections 2023-10-06T05:20:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:07.385+0000 7ff0427ff700 1 -- 172.21.15.31:0/2367260107 wait complete. 2023-10-06T05:20:07.388 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:20:07.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:20:07.626 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":70.058871009000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":17.011496923890945,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:20:07.626 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 17.011496923890945 }, "state": "open", "uptime": 70.058871009 } ] 2023-10-06T05:20:07.626 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:20:12.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:12.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.831+0000 7f8674bff700 1 Processor -- start 2023-10-06T05:20:12.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.831+0000 7f8674bff700 1 -- start start 2023-10-06T05:20:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.832+0000 7f8674bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 0x7f867006d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.833+0000 7f8674bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867006ddc0 con 0x7f867006d980 2023-10-06T05:20:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.833+0000 7f8674bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86700c71b0 con 0x7f867013a1e0 2023-10-06T05:20:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.833+0000 7f8674bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86700c7490 con 0x7f867013a5b0 2023-10-06T05:20:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.833+0000 7f866ed9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f867006d980 0x7f8670069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49152/0 (socket says 172.21.15.31:49152) 2023-10-06T05:20:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.833+0000 7f866ed9d700 1 -- 172.21.15.31:0/778758970 learned_addr learned my addr 172.21.15.31:0/778758970 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:12.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.833+0000 7f866dd9b700 1 --2- 172.21.15.31:0/778758970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 0x7f867006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.833+0000 7f866d59a700 1 -- 172.21.15.31:0/778758970 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288959231 0 0) 0x7f867006ddc0 con 0x7f867006d980 2023-10-06T05:20:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.833+0000 7f866d59a700 1 -- 172.21.15.31:0/778758970 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8658003570 con 0x7f867006d980 2023-10-06T05:20:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.833+0000 7f866d59a700 1 -- 172.21.15.31:0/778758970 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873196537 0 0) 0x7f86700c71b0 con 0x7f867013a1e0 2023-10-06T05:20:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.833+0000 7f866d59a700 1 -- 172.21.15.31:0/778758970 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f867006ddc0 con 0x7f867013a1e0 2023-10-06T05:20:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.834+0000 7f866dd9b700 1 -- 172.21.15.31:0/778758970 >> v1:172.21.15.62:6789/0 conn(0x7f867013a1e0 legacy=0x7f86700c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.834+0000 7f866dd9b700 1 -- 172.21.15.31:0/778758970 >> v1:172.21.15.31:6789/0 conn(0x7f867006d980 legacy=0x7f8670069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.834+0000 7f866dd9b700 1 -- 172.21.15.31:0/778758970 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86700c7770 con 0x7f867013a5b0 2023-10-06T05:20:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.834+0000 7f866dd9b700 1 --2- 172.21.15.31:0/778758970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 0x7f867006d440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8664009910 tx=0x7f86640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80847acef160f90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:12.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.834+0000 7f866d59a700 1 -- 172.21.15.31:0/778758970 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664011630 con 0x7f867013a5b0 2023-10-06T05:20:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.834+0000 7f866d59a700 1 -- 172.21.15.31:0/778758970 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8664011790 con 0x7f867013a5b0 2023-10-06T05:20:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.834+0000 7f866d59a700 1 -- 172.21.15.31:0/778758970 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664011b30 con 0x7f867013a5b0 2023-10-06T05:20:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.835+0000 7f8674bff700 1 -- 172.21.15.31:0/778758970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 msgr2=0x7f867006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.835+0000 7f8674bff700 1 --2- 172.21.15.31:0/778758970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 0x7f867006d440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8664009910 tx=0x7f86640091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.835+0000 7f8674bff700 1 -- 172.21.15.31:0/778758970 shutdown_connections 2023-10-06T05:20:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.835+0000 7f8674bff700 1 --2- 172.21.15.31:0/778758970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 0x7f867006d440 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.835+0000 7f8674bff700 1 -- 172.21.15.31:0/778758970 >> 172.21.15.31:0/778758970 conn(0x7f86700cc0b0 msgr2=0x7f86700cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.835+0000 7f8674bff700 1 -- 172.21.15.31:0/778758970 shutdown_connections 2023-10-06T05:20:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.835+0000 7f8674bff700 1 -- 172.21.15.31:0/778758970 wait complete. 2023-10-06T05:20:12.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.836+0000 7f8674bff700 1 Processor -- start 2023-10-06T05:20:12.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.837+0000 7f8674bff700 1 -- start start 2023-10-06T05:20:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f8674bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867006d980 0x7f8670152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f8674bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f867013a1e0 0x7f867016ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f8674bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 0x7f867016b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f8674bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8670153f80 con 0x7f867006d980 2023-10-06T05:20:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f8674bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8670154260 con 0x7f867013a1e0 2023-10-06T05:20:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f8674bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8670154540 con 0x7f867013a5b0 2023-10-06T05:20:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f866e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867006d980 0x7f8670152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f866e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867006d980 0x7f8670152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41556/0 (socket says 172.21.15.31:41556) 2023-10-06T05:20:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f866e59c700 1 -- 172.21.15.31:0/3567175650 learned_addr learned my addr 172.21.15.31:0/3567175650 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f866ed9d700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 0x7f867016b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f866dd9b700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f867013a1e0 0x7f867016ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f866e59c700 1 -- 172.21.15.31:0/3567175650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 msgr2=0x7f867016b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f866e59c700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 0x7f867016b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f866e59c700 1 -- 172.21.15.31:0/3567175650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f867013a1e0 msgr2=0x7f867016ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f866e59c700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f867013a1e0 0x7f867016ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.838+0000 7f866e59c700 1 -- 172.21.15.31:0/3567175650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8664009420 con 0x7f867006d980 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.839+0000 7f866e59c700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867006d980 0x7f8670152d20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f866000c7c0 tx=0x7f8660007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.839+0000 7f865e7fc700 1 -- 172.21.15.31:0/3567175650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660001530 con 0x7f867006d980 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.839+0000 7f865e7fc700 1 -- 172.21.15.31:0/3567175650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86600016b0 con 0x7f867006d980 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.839+0000 7f865e7fc700 1 -- 172.21.15.31:0/3567175650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660001a70 con 0x7f867006d980 2023-10-06T05:20:12.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.839+0000 7f8674bff700 1 -- 172.21.15.31:0/3567175650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8670154820 con 0x7f867006d980 2023-10-06T05:20:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.839+0000 7f8674bff700 1 -- 172.21.15.31:0/3567175650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8670153260 con 0x7f867006d980 2023-10-06T05:20:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.840+0000 7f8674bff700 1 -- 172.21.15.31:0/3567175650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8670154b50 con 0x7f867006d980 2023-10-06T05:20:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.841+0000 7f865e7fc700 1 -- 172.21.15.31:0/3567175650 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f866001b3b0 con 0x7f867006d980 2023-10-06T05:20:12.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.842+0000 7f865e7fc700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8650067170 0x7f8650069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:12.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.842+0000 7f865e7fc700 1 -- 172.21.15.31:0/3567175650 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f8660084260 con 0x7f867006d980 2023-10-06T05:20:12.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.842+0000 7f866dd9b700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8650067170 0x7f8650069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:12.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.843+0000 7f866dd9b700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8650067170 0x7f8650069620 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8664009860 tx=0x7f8664017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:12.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:12.846+0000 7f865e7fc700 1 -- 172.21.15.31:0/3567175650 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8660051000 con 0x7f867006d980 2023-10-06T05:20:13.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.092+0000 7f8674bff700 1 -- 172.21.15.31:0/3567175650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f867004c380 con 0x7f867006d980 2023-10-06T05:20:13.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.092+0000 7f865e7fc700 1 -- 172.21.15.31:0/3567175650 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f8660054620 con 0x7f867006d980 2023-10-06T05:20:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:13.095 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:20:13.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 -- 172.21.15.31:0/3567175650 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8650067170 msgr2=0x7f8650069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:13.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8650067170 0x7f8650069620 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8664009860 tx=0x7f8664017000 comp rx=0 tx=0).stop 2023-10-06T05:20:13.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 -- 172.21.15.31:0/3567175650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867006d980 msgr2=0x7f8670152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:13.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867006d980 0x7f8670152d20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f866000c7c0 tx=0x7f8660007e00 comp rx=0 tx=0).stop 2023-10-06T05:20:13.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 -- 172.21.15.31:0/3567175650 shutdown_connections 2023-10-06T05:20:13.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f867013a5b0 0x7f867016b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:13.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8650067170 0x7f8650069620 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:13.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867006d980 0x7f8670152d20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:13.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 --2- 172.21.15.31:0/3567175650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f867013a1e0 0x7f867016ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:13.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 -- 172.21.15.31:0/3567175650 >> 172.21.15.31:0/3567175650 conn(0x7f86700cc0b0 msgr2=0x7f86700c2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:13.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 -- 172.21.15.31:0/3567175650 shutdown_connections 2023-10-06T05:20:13.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:13.095+0000 7f8674bff700 1 -- 172.21.15.31:0/3567175650 wait complete. 2023-10-06T05:20:13.099 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:20:13.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:20:13.333 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5574,"entity":{"name":{"type":"client","num":5574},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":4152957218}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":75.766778872000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":16.788621036052525,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5574 172.21.15.31:0/4152957218","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:20:13.333 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 4152957218, "type": "any" }, "name": { "num": 5574, "type": "client" } }, "id": 5574, "inst": "client.5574 172.21.15.31:0/4152957218", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 16.788621036052525 }, "state": "open", "uptime": 75.766778872 } ] 2023-10-06T05:20:13.333 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-10-06T05:20:18.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:18.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.537+0000 7f51435ff700 1 Processor -- start 2023-10-06T05:20:18.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.537+0000 7f51435ff700 1 -- start start 2023-10-06T05:20:18.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.538+0000 7f51435ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f513c13a1e0 0x7f513c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.538+0000 7f51435ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c0c71b0 con 0x7f513c06db50 2023-10-06T05:20:18.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.538+0000 7f51435ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c0c7490 con 0x7f513c13a5b0 2023-10-06T05:20:18.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.538+0000 7f51435ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0c7770 con 0x7f513c13a1e0 2023-10-06T05:20:18.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.538+0000 7f5141b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f513c06db50 0x7f513c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46540/0 (socket says 172.21.15.31:46540) 2023-10-06T05:20:18.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.538+0000 7f5141b9c700 1 -- 172.21.15.31:0/315059320 learned_addr learned my addr 172.21.15.31:0/315059320 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f514139b700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f513c13a1e0 0x7f513c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382551958 0 0) 0x7f513c0c71b0 con 0x7f513c06db50 2023-10-06T05:20:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5128003570 con 0x7f513c06db50 2023-10-06T05:20:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144701362 0 0) 0x7f513c0c7490 con 0x7f513c13a5b0 2023-10-06T05:20:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f513c0c71b0 con 0x7f513c13a5b0 2023-10-06T05:20:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 492495701 0 0) 0x7f5128003570 con 0x7f513c06db50 2023-10-06T05:20:18.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f513c13a1e0 msgr2=0x7f513c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:18.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f513c13a1e0 0x7f513c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 >> v1:172.21.15.62:6789/0 conn(0x7f513c13a5b0 legacy=0x7f513c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c0c7a50 con 0x7f513c06db50 2023-10-06T05:20:18.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f51340017f0 con 0x7f513c06db50 2023-10-06T05:20:18.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 >> v1:172.21.15.31:6789/0 conn(0x7f513c06db50 legacy=0x7f513c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5128003c20 0x7f5128004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004810 0x7f5128004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f513c13a1e0 0x7f51280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0c7770 con 0x7f5128004810 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512800bb40 con 0x7f5128003c20 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.539+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512800bd00 con 0x7f513c13a1e0 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.540+0000 7f5140b9a700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004810 0x7f5128004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.540+0000 7f514139b700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5128003c20 0x7f5128004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.540+0000 7f5141b9c700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f513c13a1e0 0x7f51280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.540+0000 7f5140b9a700 1 -- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f513c13a1e0 msgr2=0x7f51280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.540+0000 7f5140b9a700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f513c13a1e0 0x7f51280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.540+0000 7f5140b9a700 1 -- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5128003c20 msgr2=0x7f5128004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.540+0000 7f5140b9a700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5128003c20 0x7f5128004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.540+0000 7f5140b9a700 1 -- 172.21.15.31:0/315059320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128003570 con 0x7f5128004810 2023-10-06T05:20:18.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.540+0000 7f5140b9a700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004810 0x7f5128004c80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f512c001a10 tx=0x7f512c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1f2c29c89cf4c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.540+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c00c5d0 con 0x7f5128004810 2023-10-06T05:20:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.541+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c00c730 con 0x7f5128004810 2023-10-06T05:20:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.541+0000 7f5133fff700 1 -- 172.21.15.31:0/315059320 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c00cad0 con 0x7f5128004810 2023-10-06T05:20:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.541+0000 7f51435ff700 1 -- 172.21.15.31:0/315059320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004810 msgr2=0x7f5128004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.541+0000 7f51435ff700 1 --2- 172.21.15.31:0/315059320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004810 0x7f5128004c80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f512c001a10 tx=0x7f512c007d30 comp rx=0 tx=0).stop 2023-10-06T05:20:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.541+0000 7f5141b9c700 1 -- 172.21.15.31:0/315059320 reap_dead start 2023-10-06T05:20:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.541+0000 7f51435ff700 1 -- 172.21.15.31:0/315059320 shutdown_connections 2023-10-06T05:20:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.541+0000 7f51435ff700 1 -- 172.21.15.31:0/315059320 >> 172.21.15.31:0/315059320 conn(0x7f513c0cc0b0 msgr2=0x7f513c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.541+0000 7f51435ff700 1 -- 172.21.15.31:0/315059320 shutdown_connections 2023-10-06T05:20:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.541+0000 7f51435ff700 1 -- 172.21.15.31:0/315059320 wait complete. 2023-10-06T05:20:18.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.543+0000 7f51435ff700 1 Processor -- start 2023-10-06T05:20:18.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.543+0000 7f51435ff700 1 -- start start 2023-10-06T05:20:18.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f51435ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5128004810 0x7f513c139070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f51435ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c16a910 0x7f513c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f51435ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f513c0caf10 0x7f513c0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f51435ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0cb8c0 con 0x7f513c16a910 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f51435ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b0ec0 con 0x7f513c0caf10 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f51435ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0affd0 con 0x7f5128004810 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f5140b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c16a910 0x7f513c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f514139b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5128004810 0x7f513c139070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f5140b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c16a910 0x7f513c0b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55596/0 (socket says 172.21.15.31:55596) 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f514139b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5128004810 0x7f513c139070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:58976/0 (socket says 172.21.15.31:58976) 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f5140b9a700 1 -- 172.21.15.31:0/1875870805 learned_addr learned my addr 172.21.15.31:0/1875870805 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:18.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.544+0000 7f5141b9c700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f513c0caf10 0x7f513c0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f5140b9a700 1 -- 172.21.15.31:0/1875870805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5128004810 msgr2=0x7f513c139070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f5140b9a700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5128004810 0x7f513c139070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f5140b9a700 1 -- 172.21.15.31:0/1875870805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f513c0caf10 msgr2=0x7f513c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f5140b9a700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f513c0caf10 0x7f513c0cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f5140b9a700 1 -- 172.21.15.31:0/1875870805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c0b02b0 con 0x7f513c16a910 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f5140b9a700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c16a910 0x7f513c0b1350 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f512800bb10 tx=0x7f512c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f51317fa700 1 -- 172.21.15.31:0/1875870805 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c00c3f0 con 0x7f513c16a910 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f51317fa700 1 -- 172.21.15.31:0/1875870805 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c00cab0 con 0x7f513c16a910 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f51317fa700 1 -- 172.21.15.31:0/1875870805 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c006a70 con 0x7f513c16a910 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f51435ff700 1 -- 172.21.15.31:0/1875870805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c0b0590 con 0x7f513c16a910 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.545+0000 7f51435ff700 1 -- 172.21.15.31:0/1875870805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c0b0b50 con 0x7f513c16a910 2023-10-06T05:20:18.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.546+0000 7f51435ff700 1 -- 172.21.15.31:0/1875870805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c04c380 con 0x7f513c16a910 2023-10-06T05:20:18.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.547+0000 7f51317fa700 1 -- 172.21.15.31:0/1875870805 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f512c019070 con 0x7f513c16a910 2023-10-06T05:20:18.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.548+0000 7f51317fa700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f51200671a0 0x7f5120069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:18.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.548+0000 7f51317fa700 1 -- 172.21.15.31:0/1875870805 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f512c0861e0 con 0x7f513c16a910 2023-10-06T05:20:18.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.548+0000 7f514139b700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f51200671a0 0x7f5120069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:18.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.549+0000 7f514139b700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f51200671a0 0x7f5120069650 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5138006ea0 tx=0x7f5138009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:18.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.553+0000 7f51317fa700 1 -- 172.21.15.31:0/1875870805 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f512c052f80 con 0x7f513c16a910 2023-10-06T05:20:18.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.800+0000 7f51435ff700 1 -- 172.21.15.31:0/1875870805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f513c0b1000 con 0x7f513c16a910 2023-10-06T05:20:18.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.801+0000 7f51317fa700 1 -- 172.21.15.31:0/1875870805 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f512c0565a0 con 0x7f513c16a910 2023-10-06T05:20:18.803 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:18.803 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:20:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 -- 172.21.15.31:0/1875870805 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f51200671a0 msgr2=0x7f5120069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f51200671a0 0x7f5120069650 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5138006ea0 tx=0x7f5138009120 comp rx=0 tx=0).stop 2023-10-06T05:20:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 -- 172.21.15.31:0/1875870805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c16a910 msgr2=0x7f513c0b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c16a910 0x7f513c0b1350 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f512800bb10 tx=0x7f512c015000 comp rx=0 tx=0).stop 2023-10-06T05:20:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 -- 172.21.15.31:0/1875870805 shutdown_connections 2023-10-06T05:20:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f513c0caf10 0x7f513c0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5128004810 0x7f513c139070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f51200671a0 0x7f5120069650 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 --2- 172.21.15.31:0/1875870805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c16a910 0x7f513c0b1350 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:18.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 -- 172.21.15.31:0/1875870805 >> 172.21.15.31:0/1875870805 conn(0x7f513c0cc0b0 msgr2=0x7f513c139840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:18.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 -- 172.21.15.31:0/1875870805 shutdown_connections 2023-10-06T05:20:18.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:18.803+0000 7f51435ff700 1 -- 172.21.15.31:0/1875870805 wait complete. 2023-10-06T05:20:18.807 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:20:18.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-06T05:20:19.043 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T05:20:19.043 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-10-06T05:20:19.043 DEBUG:tasks.ceph_test_case:wait_until_true: success in 70s and 0 retries 2023-10-06T05:20:19.044 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:20:19.044 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:19.149 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:19.255 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:20:19.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.449+0000 7fa07a5ff700 1 Processor -- start 2023-10-06T05:20:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.449+0000 7fa07a5ff700 1 -- start start 2023-10-06T05:20:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.450+0000 7fa07a5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa07413fb00 0x7fa07406c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.451+0000 7fa07a5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa07413fed0 0x7fa07406e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.451+0000 7fa07a5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07406be70 con 0x7fa07406ea90 2023-10-06T05:20:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.451+0000 7fa07a5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07406ef30 con 0x7fa07413fed0 2023-10-06T05:20:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.451+0000 7fa07a5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07406f130 con 0x7fa07413fb00 2023-10-06T05:20:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.451+0000 7fa078b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa07406ea90 0x7fa0740cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46552/0 (socket says 172.21.15.31:46552) 2023-10-06T05:20:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.451+0000 7fa078b9c700 1 -- 172.21.15.31:0/2164351491 learned_addr learned my addr 172.21.15.31:0/2164351491 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.451+0000 7fa0737fe700 1 --2- 172.21.15.31:0/2164351491 >> v2:172.21.15.62:3300/0 conn(0x7fa07413fed0 0x7fa07406e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.451+0000 7fa073fff700 1 --2- 172.21.15.31:0/2164351491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa07413fb00 0x7fa07406c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.451+0000 7fa072ffd700 1 -- 172.21.15.31:0/2164351491 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602887024 0 0) 0x7fa07406be70 con 0x7fa07406ea90 2023-10-06T05:20:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.451+0000 7fa072ffd700 1 -- 172.21.15.31:0/2164351491 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa05c003570 con 0x7fa07406ea90 2023-10-06T05:20:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.452+0000 7fa072ffd700 1 -- 172.21.15.31:0/2164351491 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2538347049 0 0) 0x7fa05c003570 con 0x7fa07406ea90 2023-10-06T05:20:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.452+0000 7fa0737fe700 1 -- 172.21.15.31:0/2164351491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa07413fb00 msgr2=0x7fa07406c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.452+0000 7fa0737fe700 1 --2- 172.21.15.31:0/2164351491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa07413fb00 0x7fa07406c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.452+0000 7fa0737fe700 1 -- 172.21.15.31:0/2164351491 >> v1:172.21.15.31:6789/0 conn(0x7fa07406ea90 legacy=0x7fa0740cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.452+0000 7fa0737fe700 1 -- 172.21.15.31:0/2164351491 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07406f410 con 0x7fa07413fed0 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.452+0000 7fa0737fe700 1 --2- 172.21.15.31:0/2164351491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa07413fed0 0x7fa07406e550 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa0680098d0 tx=0x7fa068009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55644cf302617f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.452+0000 7fa072ffd700 1 -- 172.21.15.31:0/2164351491 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c0017f0 con 0x7fa07413fed0 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.452+0000 7fa072ffd700 1 -- 172.21.15.31:0/2164351491 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa068011420 con 0x7fa07413fed0 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.452+0000 7fa072ffd700 1 -- 172.21.15.31:0/2164351491 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0680117c0 con 0x7fa07413fed0 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.453+0000 7fa07a5ff700 1 -- 172.21.15.31:0/2164351491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa07413fed0 msgr2=0x7fa07406e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.453+0000 7fa07a5ff700 1 --2- 172.21.15.31:0/2164351491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa07413fed0 0x7fa07406e550 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa0680098d0 tx=0x7fa068009120 comp rx=0 tx=0).stop 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.453+0000 7fa07a5ff700 1 -- 172.21.15.31:0/2164351491 shutdown_connections 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.453+0000 7fa07a5ff700 1 --2- 172.21.15.31:0/2164351491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa07413fb00 0x7fa07406c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.453+0000 7fa07a5ff700 1 --2- 172.21.15.31:0/2164351491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa07413fed0 0x7fa07406e550 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.453+0000 7fa07a5ff700 1 -- 172.21.15.31:0/2164351491 >> 172.21.15.31:0/2164351491 conn(0x7fa0740cdcb0 msgr2=0x7fa0740c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.453+0000 7fa07a5ff700 1 -- 172.21.15.31:0/2164351491 shutdown_connections 2023-10-06T05:20:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.453+0000 7fa07a5ff700 1 -- 172.21.15.31:0/2164351491 wait complete. 2023-10-06T05:20:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.454+0000 7fa07a5ff700 1 Processor -- start 2023-10-06T05:20:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.454+0000 7fa07a5ff700 1 -- start start 2023-10-06T05:20:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.455+0000 7fa07a5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa07406ea90 0x7fa074138f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.455+0000 7fa07a5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa07413fb00 0x7fa074140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.455+0000 7fa07a5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa07413fed0 0x7fa07416ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.455+0000 7fa07a5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741412a0 con 0x7fa07406ea90 2023-10-06T05:20:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.455+0000 7fa07a5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07416e300 con 0x7fa07413fed0 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.455+0000 7fa07a5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740b1490 con 0x7fa07413fb00 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.456+0000 7fa073fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa07406ea90 0x7fa074138f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.456+0000 7fa073fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa07406ea90 0x7fa074138f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55598/0 (socket says 172.21.15.31:55598) 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.456+0000 7fa073fff700 1 -- 172.21.15.31:0/931019186 learned_addr learned my addr 172.21.15.31:0/931019186 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.456+0000 7fa0737fe700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa07413fb00 0x7fa074140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.456+0000 7fa078b9c700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa07413fed0 0x7fa07416ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.456+0000 7fa073fff700 1 -- 172.21.15.31:0/931019186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa07413fb00 msgr2=0x7fa074140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.456+0000 7fa073fff700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa07413fb00 0x7fa074140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.456+0000 7fa073fff700 1 -- 172.21.15.31:0/931019186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa07413fed0 msgr2=0x7fa07416ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.456+0000 7fa073fff700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa07413fed0 0x7fa07416ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.456+0000 7fa073fff700 1 -- 172.21.15.31:0/931019186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0680093c0 con 0x7fa07406ea90 2023-10-06T05:20:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.457+0000 7fa073fff700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa07406ea90 0x7fa074138f40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa06400ec80 tx=0x7fa0640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:19.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.457+0000 7fa05bfff700 1 -- 172.21.15.31:0/931019186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06400c9d0 con 0x7fa07406ea90 2023-10-06T05:20:19.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.457+0000 7fa05bfff700 1 -- 172.21.15.31:0/931019186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06400cb30 con 0x7fa07406ea90 2023-10-06T05:20:19.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.457+0000 7fa07a5ff700 1 -- 172.21.15.31:0/931019186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa074139600 con 0x7fa07406ea90 2023-10-06T05:20:19.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.457+0000 7fa05bfff700 1 -- 172.21.15.31:0/931019186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064013470 con 0x7fa07406ea90 2023-10-06T05:20:19.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.457+0000 7fa07a5ff700 1 -- 172.21.15.31:0/931019186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa074139c40 con 0x7fa07406ea90 2023-10-06T05:20:19.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.458+0000 7fa07a5ff700 1 -- 172.21.15.31:0/931019186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa074139f60 con 0x7fa07406ea90 2023-10-06T05:20:19.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.459+0000 7fa05bfff700 1 -- 172.21.15.31:0/931019186 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fa064013820 con 0x7fa07406ea90 2023-10-06T05:20:19.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.460+0000 7fa05bfff700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa060067170 0x7fa060069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.460+0000 7fa05bfff700 1 -- 172.21.15.31:0/931019186 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fa064084db0 con 0x7fa07406ea90 2023-10-06T05:20:19.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.460+0000 7fa0737fe700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa060067170 0x7fa060069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.461+0000 7fa0737fe700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa060067170 0x7fa060069620 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa0680030c0 tx=0x7fa068016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:19.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.464+0000 7fa05bfff700 1 -- 172.21.15.31:0/931019186 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa064051bf0 con 0x7fa07406ea90 2023-10-06T05:20:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.710+0000 7fa07a5ff700 1 -- 172.21.15.31:0/931019186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa07404c380 con 0x7fa07406ea90 2023-10-06T05:20:19.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.710+0000 7fa05bfff700 1 -- 172.21.15.31:0/931019186 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fa064055210 con 0x7fa07406ea90 2023-10-06T05:20:19.712 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:19.712 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:20:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.712+0000 7fa07a5ff700 1 -- 172.21.15.31:0/931019186 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa060067170 msgr2=0x7fa060069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.712+0000 7fa07a5ff700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa060067170 0x7fa060069620 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa0680030c0 tx=0x7fa068016000 comp rx=0 tx=0).stop 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.712+0000 7fa07a5ff700 1 -- 172.21.15.31:0/931019186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa07406ea90 msgr2=0x7fa074138f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.712+0000 7fa07a5ff700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa07406ea90 0x7fa074138f40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa06400ec80 tx=0x7fa0640091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.713+0000 7fa07a5ff700 1 -- 172.21.15.31:0/931019186 shutdown_connections 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.713+0000 7fa07a5ff700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa07413fed0 0x7fa07416ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.713+0000 7fa07a5ff700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa060067170 0x7fa060069620 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.713+0000 7fa07a5ff700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa07406ea90 0x7fa074138f40 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.713+0000 7fa07a5ff700 1 --2- 172.21.15.31:0/931019186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa07413fb00 0x7fa074140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.713+0000 7fa07a5ff700 1 -- 172.21.15.31:0/931019186 >> 172.21.15.31:0/931019186 conn(0x7fa0740cdcb0 msgr2=0x7fa0740c2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.713+0000 7fa07a5ff700 1 -- 172.21.15.31:0/931019186 shutdown_connections 2023-10-06T05:20:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.713+0000 7fa07a5ff700 1 -- 172.21.15.31:0/931019186 wait complete. 2023-10-06T05:20:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:20:19.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:19.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.925+0000 7f1f323ff700 1 Processor -- start 2023-10-06T05:20:19.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.925+0000 7f1f323ff700 1 -- start start 2023-10-06T05:20:19.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.927+0000 7f1f323ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f2c0cd510 0x7f1f2c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.927+0000 7f1f323ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1f2c15d2d0 0x7f1f2c0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.927+0000 7f1f323ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2c0c3f40 con 0x7f1f2c159500 2023-10-06T05:20:19.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.927+0000 7f1f323ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2c0b2860 con 0x7f1f2c0cd510 2023-10-06T05:20:19.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.927+0000 7f1f323ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2c15d7d0 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.928+0000 7f1f2b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f2c0cd510 0x7f1f2c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.928+0000 7f1f2b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f2c0cd510 0x7f1f2c0b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55614/0 (socket says 172.21.15.31:55614) 2023-10-06T05:20:19.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.928+0000 7f1f2b7fe700 1 -- 172.21.15.31:0/2864308208 learned_addr learned my addr 172.21.15.31:0/2864308208 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.928+0000 7f1f3099c700 1 --2- 172.21.15.31:0/2864308208 >> v2:172.21.15.62:3300/0 conn(0x7f1f2c15d2d0 0x7f1f2c0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.928+0000 7f1f2b7fe700 1 -- 172.21.15.31:0/2864308208 >> v2:172.21.15.62:3300/0 conn(0x7f1f2c15d2d0 msgr2=0x7f1f2c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.928+0000 7f1f2b7fe700 1 --2- 172.21.15.31:0/2864308208 >> v2:172.21.15.62:3300/0 conn(0x7f1f2c15d2d0 0x7f1f2c0aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.928+0000 7f1f2b7fe700 1 -- 172.21.15.31:0/2864308208 >> v1:172.21.15.62:6789/0 conn(0x7f1f2c159500 legacy=0x7f1f2c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.928+0000 7f1f2b7fe700 1 -- 172.21.15.31:0/2864308208 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0cb000 con 0x7f1f2c0cd510 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.928+0000 7f1f3099c700 1 --2- 172.21.15.31:0/2864308208 >> v2:172.21.15.62:3300/0 conn(0x7f1f2c15d2d0 0x7f1f2c0aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.928+0000 7f1f2b7fe700 1 --2- 172.21.15.31:0/2864308208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c0cd510 0x7f1f2c0b0ab0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0ca9f0 tx=0x7f1f20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8362b5dd7dc1b42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f2affd700 1 -- 172.21.15.31:0/2864308208 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2000b860 con 0x7f1f2c0cd510 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f2affd700 1 -- 172.21.15.31:0/2864308208 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2000b9c0 con 0x7f1f2c0cd510 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f2affd700 1 -- 172.21.15.31:0/2864308208 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2000bd60 con 0x7f1f2c0cd510 2023-10-06T05:20:19.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f323ff700 1 -- 172.21.15.31:0/2864308208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c0cd510 msgr2=0x7f1f2c0b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f323ff700 1 --2- 172.21.15.31:0/2864308208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c0cd510 0x7f1f2c0b0ab0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0ca9f0 tx=0x7f1f20009250 comp rx=0 tx=0).stop 2023-10-06T05:20:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f323ff700 1 -- 172.21.15.31:0/2864308208 shutdown_connections 2023-10-06T05:20:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f323ff700 1 --2- 172.21.15.31:0/2864308208 >> v2:172.21.15.62:3300/0 conn(0x7f1f2c15d2d0 0x7f1f2c0aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f323ff700 1 --2- 172.21.15.31:0/2864308208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c0cd510 0x7f1f2c0b0ab0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f323ff700 1 -- 172.21.15.31:0/2864308208 >> 172.21.15.31:0/2864308208 conn(0x7f1f2c06ec90 msgr2=0x7f1f2c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f323ff700 1 -- 172.21.15.31:0/2864308208 shutdown_connections 2023-10-06T05:20:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.929+0000 7f1f323ff700 1 -- 172.21.15.31:0/2864308208 wait complete. 2023-10-06T05:20:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.931+0000 7f1f323ff700 1 Processor -- start 2023-10-06T05:20:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.932+0000 7f1f323ff700 1 -- start start 2023-10-06T05:20:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f323ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f2c0cd510 0x7f1f2c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f323ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f2c159500 0x7f1f2c14d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f323ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c15d2d0 0x7f1f2c0c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f323ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0c99a0 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f323ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0c8d90 con 0x7f1f2c159500 2023-10-06T05:20:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f323ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0c9070 con 0x7f1f2c0cd510 2023-10-06T05:20:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f3099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c15d2d0 0x7f1f2c0c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f3099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c15d2d0 0x7f1f2c0c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55628/0 (socket says 172.21.15.31:55628) 2023-10-06T05:20:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f3099c700 1 -- 172.21.15.31:0/3850961608 learned_addr learned my addr 172.21.15.31:0/3850961608 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f2b7fe700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f2c159500 0x7f1f2c14d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.933+0000 7f1f2bfff700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f2c0cd510 0x7f1f2c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f3099c700 1 -- 172.21.15.31:0/3850961608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f2c0cd510 msgr2=0x7f1f2c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f3099c700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f2c0cd510 0x7f1f2c157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f3099c700 1 -- 172.21.15.31:0/3850961608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f2c159500 msgr2=0x7f1f2c14d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f3099c700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f2c159500 0x7f1f2c14d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f3099c700 1 -- 172.21.15.31:0/3850961608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20009490 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f2b7fe700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f2c159500 0x7f1f2c14d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f3099c700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c15d2d0 0x7f1f2c0c9e30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009fc0 tx=0x7f1f240099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f13fff700 1 -- 172.21.15.31:0/3850961608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400dd40 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f13fff700 1 -- 172.21.15.31:0/3850961608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2400f040 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f323ff700 1 -- 172.21.15.31:0/3850961608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c0c9350 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f13fff700 1 -- 172.21.15.31:0/3850961608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24015690 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.934+0000 7f1f323ff700 1 -- 172.21.15.31:0/3850961608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c14dde0 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.935+0000 7f1f323ff700 1 -- 172.21.15.31:0/3850961608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2c04c380 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.936+0000 7f1f13fff700 1 -- 172.21.15.31:0/3850961608 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f1f2401e3b0 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.937+0000 7f1f13fff700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1f18067170 0x7f1f18069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.937+0000 7f1f13fff700 1 -- 172.21.15.31:0/3850961608 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f1f24085100 con 0x7f1f2c15d2d0 2023-10-06T05:20:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.937+0000 7f1f2bfff700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1f18067170 0x7f1f18069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.938+0000 7f1f2bfff700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1f18067170 0x7f1f18069620 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c003860 tx=0x7f1f1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:19.941+0000 7f1f13fff700 1 -- 172.21.15.31:0/3850961608 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f24051d90 con 0x7f1f2c15d2d0 2023-10-06T05:20:20.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.144+0000 7f1f323ff700 1 -- 172.21.15.31:0/3850961608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f2c0c66a0 con 0x7f1f2c15d2d0 2023-10-06T05:20:20.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.145+0000 7f1f13fff700 1 -- 172.21.15.31:0/3850961608 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+17009 (secure 0 0 0) 0x7f1f240553b0 con 0x7f1f2c15d2d0 2023-10-06T05:20:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:20.148 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:50.146155+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:36.639713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:37.929704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:20.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 -- 172.21.15.31:0/3850961608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1f18067170 msgr2=0x7f1f18069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1f18067170 0x7f1f18069620 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c003860 tx=0x7f1f1c009040 comp rx=0 tx=0).stop 2023-10-06T05:20:20.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 -- 172.21.15.31:0/3850961608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c15d2d0 msgr2=0x7f1f2c0c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c15d2d0 0x7f1f2c0c9e30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009fc0 tx=0x7f1f240099b0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 -- 172.21.15.31:0/3850961608 shutdown_connections 2023-10-06T05:20:20.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1f18067170 0x7f1f18069620 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f2c15d2d0 0x7f1f2c0c9e30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f2c0cd510 0x7f1f2c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 --2- 172.21.15.31:0/3850961608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f2c159500 0x7f1f2c14d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 -- 172.21.15.31:0/3850961608 >> 172.21.15.31:0/3850961608 conn(0x7f1f2c06ec90 msgr2=0x7f1f2c0ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:20.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 -- 172.21.15.31:0/3850961608 shutdown_connections 2023-10-06T05:20:20.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.148+0000 7f1f323ff700 1 -- 172.21.15.31:0/3850961608 wait complete. 2023-10-06T05:20:20.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:18:36.639713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:18:37.929704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:20:20.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:20:20.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.368+0000 7f6047fff700 1 Processor -- start 2023-10-06T05:20:20.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.368+0000 7f6047fff700 1 -- start start 2023-10-06T05:20:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.369+0000 7f6047fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6040153d40 0x7f60400b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.369+0000 7f6047fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60400af890 0x7f60400adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.369+0000 7f6047fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60400b0e40 con 0x7f60400b09c0 2023-10-06T05:20:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.369+0000 7f6047fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60400afc90 con 0x7f6040153d40 2023-10-06T05:20:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.369+0000 7f6047fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60400ad690 con 0x7f60400af890 2023-10-06T05:20:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.370+0000 7f6045d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6040153d40 0x7f60400b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.370+0000 7f604559a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f60400b09c0 0x7f60400af180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46556/0 (socket says 172.21.15.31:46556) 2023-10-06T05:20:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.370+0000 7f604559a700 1 -- 172.21.15.31:0/518836914 learned_addr learned my addr 172.21.15.31:0/518836914 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.370+0000 7f604659c700 1 --2- 172.21.15.31:0/518836914 >> v2:172.21.15.62:3300/0 conn(0x7f60400af890 0x7f60400adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.370+0000 7f6044d99700 1 -- 172.21.15.31:0/518836914 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671921658 0 0) 0x7f6030001110 con 0x7f60400b09c0 2023-10-06T05:20:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.370+0000 7f6044d99700 1 -- 172.21.15.31:0/518836914 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f602c003570 con 0x7f60400b09c0 2023-10-06T05:20:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.370+0000 7f6045d9b700 1 -- 172.21.15.31:0/518836914 >> v2:172.21.15.62:3300/0 conn(0x7f60400af890 msgr2=0x7f60400adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.370+0000 7f6045d9b700 1 --2- 172.21.15.31:0/518836914 >> v2:172.21.15.62:3300/0 conn(0x7f60400af890 0x7f60400adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.370+0000 7f6045d9b700 1 -- 172.21.15.31:0/518836914 >> v1:172.21.15.31:6789/0 conn(0x7f60400b09c0 legacy=0x7f60400af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.370+0000 7f6045d9b700 1 -- 172.21.15.31:0/518836914 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400acd60 con 0x7f6040153d40 2023-10-06T05:20:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.371+0000 7f6045d9b700 1 --2- 172.21.15.31:0/518836914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6040153d40 0x7f60400b0480 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f604006b310 tx=0x7f603c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426b0bc9f9fb111e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.371+0000 7f6044d99700 1 -- 172.21.15.31:0/518836914 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c011630 con 0x7f6040153d40 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.371+0000 7f6044d99700 1 -- 172.21.15.31:0/518836914 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c0117b0 con 0x7f6040153d40 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.371+0000 7f6044d99700 1 -- 172.21.15.31:0/518836914 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c011b70 con 0x7f6040153d40 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.371+0000 7f6047fff700 1 -- 172.21.15.31:0/518836914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6040153d40 msgr2=0x7f60400b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.371+0000 7f6047fff700 1 --2- 172.21.15.31:0/518836914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6040153d40 0x7f60400b0480 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f604006b310 tx=0x7f603c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.371+0000 7f6047fff700 1 -- 172.21.15.31:0/518836914 shutdown_connections 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.371+0000 7f6047fff700 1 --2- 172.21.15.31:0/518836914 >> v2:172.21.15.62:3300/0 conn(0x7f60400af890 0x7f60400adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.371+0000 7f6047fff700 1 --2- 172.21.15.31:0/518836914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6040153d40 0x7f60400b0480 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.371+0000 7f6047fff700 1 -- 172.21.15.31:0/518836914 >> 172.21.15.31:0/518836914 conn(0x7f6040067560 msgr2=0x7f6040067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.372+0000 7f6047fff700 1 -- 172.21.15.31:0/518836914 shutdown_connections 2023-10-06T05:20:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.372+0000 7f6047fff700 1 -- 172.21.15.31:0/518836914 wait complete. 2023-10-06T05:20:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.373+0000 7f6047fff700 1 Processor -- start 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.373+0000 7f6047fff700 1 -- start start 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.374+0000 7f6047fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60400af890 0x7f604014e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.374+0000 7f6047fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400b09c0 0x7f60400cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.374+0000 7f6047fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6040153d40 0x7f6040149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.374+0000 7f6047fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60400c9a10 con 0x7f60400b09c0 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.374+0000 7f6047fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60400ca520 con 0x7f6040153d40 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.374+0000 7f6047fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60400c9bf0 con 0x7f60400af890 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.374+0000 7f604559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400b09c0 0x7f60400cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.374+0000 7f604559a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400b09c0 0x7f60400cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55646/0 (socket says 172.21.15.31:55646) 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.374+0000 7f604559a700 1 -- 172.21.15.31:0/2176675531 learned_addr learned my addr 172.21.15.31:0/2176675531 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f604659c700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6040153d40 0x7f6040149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f6045d9b700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60400af890 0x7f604014e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f604559a700 1 -- 172.21.15.31:0/2176675531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60400af890 msgr2=0x7f604014e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f604559a700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60400af890 0x7f604014e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f604559a700 1 -- 172.21.15.31:0/2176675531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6040153d40 msgr2=0x7f6040149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f604559a700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6040153d40 0x7f6040149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f604559a700 1 -- 172.21.15.31:0/2176675531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c009420 con 0x7f60400b09c0 2023-10-06T05:20:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f6045d9b700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60400af890 0x7f604014e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f604559a700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400b09c0 0x7f60400cb320 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f603000c7c0 tx=0x7f6030007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f6035ffb700 1 -- 172.21.15.31:0/2176675531 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030005b30 con 0x7f60400b09c0 2023-10-06T05:20:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f6035ffb700 1 -- 172.21.15.31:0/2176675531 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6030005c90 con 0x7f60400b09c0 2023-10-06T05:20:20.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f6035ffb700 1 -- 172.21.15.31:0/2176675531 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030007340 con 0x7f60400b09c0 2023-10-06T05:20:20.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f6047fff700 1 -- 172.21.15.31:0/2176675531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60400c9ed0 con 0x7f60400b09c0 2023-10-06T05:20:20.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.375+0000 7f6047fff700 1 -- 172.21.15.31:0/2176675531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604014e860 con 0x7f60400b09c0 2023-10-06T05:20:20.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.376+0000 7f6047fff700 1 -- 172.21.15.31:0/2176675531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60400ca6b0 con 0x7f60400b09c0 2023-10-06T05:20:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.377+0000 7f6035ffb700 1 -- 172.21.15.31:0/2176675531 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f603001b400 con 0x7f60400b09c0 2023-10-06T05:20:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.378+0000 7f6035ffb700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f60240670e0 0x7f6024069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.378+0000 7f6035ffb700 1 -- 172.21.15.31:0/2176675531 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f6030084420 con 0x7f60400b09c0 2023-10-06T05:20:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.378+0000 7f6045d9b700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f60240670e0 0x7f6024069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.379+0000 7f6045d9b700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f60240670e0 0x7f6024069590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f604006b310 tx=0x7f603c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.382+0000 7f6035ffb700 1 -- 172.21.15.31:0/2176675531 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6030051300 con 0x7f60400b09c0 2023-10-06T05:20:20.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.584+0000 7f6047fff700 1 -- 172.21.15.31:0/2176675531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f60400b0e40 con 0x7f60400b09c0 2023-10-06T05:20:20.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.584+0000 7f6035ffb700 1 -- 172.21.15.31:0/2176675531 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v46) v1 ==== 92+0+10 (secure 0 0 0) 0x7f60400b0e40 con 0x7f60400b09c0 2023-10-06T05:20:20.587 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:20:20.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 -- 172.21.15.31:0/2176675531 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f60240670e0 msgr2=0x7f6024069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f60240670e0 0x7f6024069590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f604006b310 tx=0x7f603c017000 comp rx=0 tx=0).stop 2023-10-06T05:20:20.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 -- 172.21.15.31:0/2176675531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400b09c0 msgr2=0x7f60400cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400b09c0 0x7f60400cb320 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f603000c7c0 tx=0x7f6030007c70 comp rx=0 tx=0).stop 2023-10-06T05:20:20.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 -- 172.21.15.31:0/2176675531 shutdown_connections 2023-10-06T05:20:20.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6040153d40 0x7f6040149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60400af890 0x7f604014e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f60240670e0 0x7f6024069590 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 --2- 172.21.15.31:0/2176675531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400b09c0 0x7f60400cb320 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 -- 172.21.15.31:0/2176675531 >> 172.21.15.31:0/2176675531 conn(0x7f6040067560 msgr2=0x7f60400aae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:20.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 -- 172.21.15.31:0/2176675531 shutdown_connections 2023-10-06T05:20:20.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.587+0000 7f6047fff700 1 -- 172.21.15.31:0/2176675531 wait complete. 2023-10-06T05:20:20.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:20:20.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.802+0000 7f786f7ff700 1 Processor -- start 2023-10-06T05:20:20.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.802+0000 7f786f7ff700 1 -- start start 2023-10-06T05:20:20.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786f7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7868153d40 0x7f78680b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786f7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f78680b09c0 0x7f78680af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786f7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78680ad690 con 0x7f78680af6c0 2023-10-06T05:20:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786f7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78680b0e30 con 0x7f78680b09c0 2023-10-06T05:20:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786f7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78680acd60 con 0x7f7868153d40 2023-10-06T05:20:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786cd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f78680b09c0 0x7f78680af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786cd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f78680b09c0 0x7f78680af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55648/0 (socket says 172.21.15.31:55648) 2023-10-06T05:20:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786cd9a700 1 -- 172.21.15.31:0/3117277265 learned_addr learned my addr 172.21.15.31:0/3117277265 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786d59b700 1 --2- 172.21.15.31:0/3117277265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7868153d40 0x7f78680b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786cd9a700 1 -- 172.21.15.31:0/3117277265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7868153d40 msgr2=0x7f78680b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:20.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786cd9a700 1 --2- 172.21.15.31:0/3117277265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7868153d40 0x7f78680b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786cd9a700 1 -- 172.21.15.31:0/3117277265 >> v1:172.21.15.62:6789/0 conn(0x7f78680af6c0 legacy=0x7f7868159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786cd9a700 1 -- 172.21.15.31:0/3117277265 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680ad040 con 0x7f78680b09c0 2023-10-06T05:20:20.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f786d59b700 1 --2- 172.21.15.31:0/3117277265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7868153d40 0x7f78680b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:20.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.804+0000 7f785ffff700 1 -- 172.21.15.31:0/3117277265 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194976474 0 0) 0x7f78680ad690 con 0x7f78680af6c0 2023-10-06T05:20:20.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.805+0000 7f786cd9a700 1 --2- 172.21.15.31:0/3117277265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680b09c0 0x7f78680af180 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f78580099b0 tx=0x7f7858009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea52eb6daa74d8a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:20.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.805+0000 7f785ffff700 1 -- 172.21.15.31:0/3117277265 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800b860 con 0x7f78680b09c0 2023-10-06T05:20:20.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.805+0000 7f785ffff700 1 -- 172.21.15.31:0/3117277265 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785800b9c0 con 0x7f78680b09c0 2023-10-06T05:20:20.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.805+0000 7f785ffff700 1 -- 172.21.15.31:0/3117277265 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800bd60 con 0x7f78680b09c0 2023-10-06T05:20:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.805+0000 7f786f7ff700 1 -- 172.21.15.31:0/3117277265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680b09c0 msgr2=0x7f78680af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.805+0000 7f786f7ff700 1 --2- 172.21.15.31:0/3117277265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680b09c0 0x7f78680af180 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f78580099b0 tx=0x7f7858009250 comp rx=0 tx=0).stop 2023-10-06T05:20:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.806+0000 7f786f7ff700 1 -- 172.21.15.31:0/3117277265 shutdown_connections 2023-10-06T05:20:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.806+0000 7f786f7ff700 1 --2- 172.21.15.31:0/3117277265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7868153d40 0x7f78680b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.806+0000 7f786f7ff700 1 --2- 172.21.15.31:0/3117277265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680b09c0 0x7f78680af180 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.806+0000 7f786f7ff700 1 -- 172.21.15.31:0/3117277265 >> 172.21.15.31:0/3117277265 conn(0x7f7868067560 msgr2=0x7f7868067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:20.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.806+0000 7f786f7ff700 1 -- 172.21.15.31:0/3117277265 shutdown_connections 2023-10-06T05:20:20.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.806+0000 7f786f7ff700 1 -- 172.21.15.31:0/3117277265 wait complete. 2023-10-06T05:20:20.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.807+0000 7f786f7ff700 1 Processor -- start 2023-10-06T05:20:20.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.807+0000 7f786f7ff700 1 -- start start 2023-10-06T05:20:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.808+0000 7f786f7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680af6c0 0x7f786814fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.808+0000 7f786f7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78680b09c0 0x7f786814ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.808+0000 7f786f7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7868153d40 0x7f7868169560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.808+0000 7f786f7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78680c9820 con 0x7f78680af6c0 2023-10-06T05:20:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.808+0000 7f786f7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78681502d0 con 0x7f7868153d40 2023-10-06T05:20:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.808+0000 7f786f7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78681516b0 con 0x7f78680b09c0 2023-10-06T05:20:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.808+0000 7f786d59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680af6c0 0x7f786814fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.808+0000 7f786d59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680af6c0 0x7f786814fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55658/0 (socket says 172.21.15.31:55658) 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.808+0000 7f786d59b700 1 -- 172.21.15.31:0/2571690587 learned_addr learned my addr 172.21.15.31:0/2571690587 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f786d59b700 1 -- 172.21.15.31:0/2571690587 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78680b09c0 msgr2=0x7f786814ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f786d59b700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78680b09c0 0x7f786814ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f786d59b700 1 -- 172.21.15.31:0/2571690587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7868153d40 msgr2=0x7f7868169560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f786dd9c700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7868153d40 0x7f7868169560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f786d59b700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7868153d40 0x7f7868169560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f786d59b700 1 -- 172.21.15.31:0/2571690587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580094f0 con 0x7f78680af6c0 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f786dd9c700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7868153d40 0x7f7868169560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f786d59b700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680af6c0 0x7f786814fd90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f786400d860 tx=0x7f7864008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f785d7fa700 1 -- 172.21.15.31:0/2571690587 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640125f0 con 0x7f78680af6c0 2023-10-06T05:20:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f785d7fa700 1 -- 172.21.15.31:0/2571690587 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7864012750 con 0x7f78680af6c0 2023-10-06T05:20:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f785d7fa700 1 -- 172.21.15.31:0/2571690587 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864012ad0 con 0x7f78680af6c0 2023-10-06T05:20:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.809+0000 7f786f7ff700 1 -- 172.21.15.31:0/2571690587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786814f1c0 con 0x7f78680af6c0 2023-10-06T05:20:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.810+0000 7f786f7ff700 1 -- 172.21.15.31:0/2571690587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7868150530 con 0x7f78680af6c0 2023-10-06T05:20:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.810+0000 7f786f7ff700 1 -- 172.21.15.31:0/2571690587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786804c380 con 0x7f78680af6c0 2023-10-06T05:20:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.811+0000 7f785d7fa700 1 -- 172.21.15.31:0/2571690587 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7864014430 con 0x7f78680af6c0 2023-10-06T05:20:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.812+0000 7f785d7fa700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f784c0670e0 0x7f784c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.812+0000 7f785d7fa700 1 -- 172.21.15.31:0/2571690587 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f7864084e60 con 0x7f78680af6c0 2023-10-06T05:20:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.812+0000 7f786cd9a700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f784c0670e0 0x7f784c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.813+0000 7f786cd9a700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f784c0670e0 0x7f784c069590 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78580099b0 tx=0x7f7858017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:20.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:20.816+0000 7f785d7fa700 1 -- 172.21.15.31:0/2571690587 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7864055740 con 0x7f78680af6c0 2023-10-06T05:20:21.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.019+0000 7f786f7ff700 1 -- 172.21.15.31:0/2571690587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78680ad380 con 0x7f78680af6c0 2023-10-06T05:20:21.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.019+0000 7f785d7fa700 1 -- 172.21.15.31:0/2571690587 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78640552d0 con 0x7f78680af6c0 2023-10-06T05:20:21.021 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:20:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 -- 172.21.15.31:0/2571690587 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f784c0670e0 msgr2=0x7f784c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f784c0670e0 0x7f784c069590 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78580099b0 tx=0x7f7858017000 comp rx=0 tx=0).stop 2023-10-06T05:20:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 -- 172.21.15.31:0/2571690587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680af6c0 msgr2=0x7f786814fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680af6c0 0x7f786814fd90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f786400d860 tx=0x7f7864008040 comp rx=0 tx=0).stop 2023-10-06T05:20:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 -- 172.21.15.31:0/2571690587 shutdown_connections 2023-10-06T05:20:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7868153d40 0x7f7868169560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f784c0670e0 0x7f784c069590 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78680af6c0 0x7f786814fd90 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 --2- 172.21.15.31:0/2571690587 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78680b09c0 0x7f786814ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 -- 172.21.15.31:0/2571690587 >> 172.21.15.31:0/2571690587 conn(0x7f7868067560 msgr2=0x7f78680cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:21.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 -- 172.21.15.31:0/2571690587 shutdown_connections 2023-10-06T05:20:21.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.022+0000 7f786f7ff700 1 -- 172.21.15.31:0/2571690587 wait complete. 2023-10-06T05:20:21.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:20:21.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.239+0000 7f3ab39ff700 1 Processor -- start 2023-10-06T05:20:21.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.239+0000 7f3ab39ff700 1 -- start start 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.241+0000 7f3ab39ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3aac0cd580 0x7f3aac1645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.241+0000 7f3ab39ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac15d2d0 con 0x7f3aac159930 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.241+0000 7f3ab39ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac15d5b0 con 0x7f3aac159560 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.241+0000 7f3ab39ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aac15d890 con 0x7f3aac0cd580 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.241+0000 7f3ab0f9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3aac159930 0x7f3aac0b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46568/0 (socket says 172.21.15.31:46568) 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.241+0000 7f3ab0f9a700 1 -- 172.21.15.31:0/3764757780 learned_addr learned my addr 172.21.15.31:0/3764757780 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.241+0000 7f3ab1f9c700 1 --2- 172.21.15.31:0/3764757780 >> v2:172.21.15.62:3300/0 conn(0x7f3aac0cd580 0x7f3aac1645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.241+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838314952 0 0) 0x7f3aac15d2d0 con 0x7f3aac159930 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.241+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a98003570 con 0x7f3aac159930 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252705260 0 0) 0x7f3aac15d5b0 con 0x7f3aac159560 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aac15d2d0 con 0x7f3aac159560 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3159895184 0 0) 0x7f3a98003570 con 0x7f3aac159930 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 >> v2:172.21.15.62:3300/0 conn(0x7f3aac0cd580 msgr2=0x7f3aac1645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 --2- 172.21.15.31:0/3764757780 >> v2:172.21.15.62:3300/0 conn(0x7f3aac0cd580 0x7f3aac1645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 >> v1:172.21.15.62:6789/0 conn(0x7f3aac159560 legacy=0x7f3aac0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac15db70 con 0x7f3aac159930 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f3a9c001470 con 0x7f3aac159930 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 >> v1:172.21.15.31:6789/0 conn(0x7f3aac159930 legacy=0x7f3aac0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a98003bf0 0x7f3a98003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98004810 0x7f3a98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a980051c0 0x7f3a9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac15d890 con 0x7f3a98003bf0 2023-10-06T05:20:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9800c160 con 0x7f3a98004810 2023-10-06T05:20:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9800c320 con 0x7f3a980051c0 2023-10-06T05:20:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3ab179b700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a98003bf0 0x7f3a98003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3ab0f9a700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98004810 0x7f3a98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.242+0000 7f3ab1f9c700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a980051c0 0x7f3a9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3ab179b700 1 -- 172.21.15.31:0/3764757780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a980051c0 msgr2=0x7f3a9800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3ab179b700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a980051c0 0x7f3a9800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3ab179b700 1 -- 172.21.15.31:0/3764757780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98004810 msgr2=0x7f3a98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3ab179b700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98004810 0x7f3a98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3ab179b700 1 -- 172.21.15.31:0/3764757780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a98003570 con 0x7f3a98003bf0 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3ab1f9c700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a980051c0 0x7f3a9800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3ab1f9c700 1 -- 172.21.15.31:0/3764757780 reap_dead start 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3ab179b700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a98003bf0 0x7f3a98003fe0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80019e0 tx=0x7f3aa8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3df3d4dbae9683e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800c5d0 con 0x7f3a98003bf0 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa800c730 con 0x7f3a98003bf0 2023-10-06T05:20:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.243+0000 7f3aa3fff700 1 -- 172.21.15.31:0/3764757780 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800caf0 con 0x7f3a98003bf0 2023-10-06T05:20:21.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.244+0000 7f3ab39ff700 1 -- 172.21.15.31:0/3764757780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a98003bf0 msgr2=0x7f3a98003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.244+0000 7f3ab39ff700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a98003bf0 0x7f3a98003fe0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80019e0 tx=0x7f3aa8007d30 comp rx=0 tx=0).stop 2023-10-06T05:20:21.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.244+0000 7f3ab39ff700 1 -- 172.21.15.31:0/3764757780 shutdown_connections 2023-10-06T05:20:21.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.244+0000 7f3ab39ff700 1 --2- 172.21.15.31:0/3764757780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a98003bf0 0x7f3a98003fe0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.244+0000 7f3ab39ff700 1 -- 172.21.15.31:0/3764757780 >> 172.21.15.31:0/3764757780 conn(0x7f3aac06ec90 msgr2=0x7f3aac06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:21.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.244+0000 7f3ab39ff700 1 -- 172.21.15.31:0/3764757780 shutdown_connections 2023-10-06T05:20:21.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.244+0000 7f3ab39ff700 1 -- 172.21.15.31:0/3764757780 wait complete. 2023-10-06T05:20:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.245+0000 7f3ab39ff700 1 Processor -- start 2023-10-06T05:20:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.246+0000 7f3ab39ff700 1 -- start start 2023-10-06T05:20:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.246+0000 7f3ab39ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98003bf0 0x7f3aac1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.246+0000 7f3ab39ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3aac159d20 0x7f3aac0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.246+0000 7f3ab39ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3aac15a190 0x7f3aac0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.246+0000 7f3ab39ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0c4ff0 con 0x7f3aac15a190 2023-10-06T05:20:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.246+0000 7f3ab39ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac155f30 con 0x7f3a98003bf0 2023-10-06T05:20:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.246+0000 7f3ab39ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac15a700 con 0x7f3aac159d20 2023-10-06T05:20:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab1f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3aac15a190 0x7f3aac0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab1f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3aac15a190 0x7f3aac0cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55688/0 (socket says 172.21.15.31:55688) 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab1f9c700 1 -- 172.21.15.31:0/163943320 learned_addr learned my addr 172.21.15.31:0/163943320 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab179b700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98003bf0 0x7f3aac1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab0f9a700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3aac159d20 0x7f3aac0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab1f9c700 1 -- 172.21.15.31:0/163943320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3aac159d20 msgr2=0x7f3aac0cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab1f9c700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3aac159d20 0x7f3aac0cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab1f9c700 1 -- 172.21.15.31:0/163943320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98003bf0 msgr2=0x7f3aac1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab1f9c700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98003bf0 0x7f3aac1559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab1f9c700 1 -- 172.21.15.31:0/163943320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac15a9e0 con 0x7f3aac15a190 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab179b700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98003bf0 0x7f3aac1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.247+0000 7f3ab1f9c700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3aac15a190 0x7f3aac0cbce0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0b1c90 tx=0x7f3aa400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.248+0000 7f3aa17fa700 1 -- 172.21.15.31:0/163943320 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa401b070 con 0x7f3aac15a190 2023-10-06T05:20:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.248+0000 7f3aa17fa700 1 -- 172.21.15.31:0/163943320 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa400f040 con 0x7f3aac15a190 2023-10-06T05:20:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.248+0000 7f3aa17fa700 1 -- 172.21.15.31:0/163943320 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa400e050 con 0x7f3aac15a190 2023-10-06T05:20:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.248+0000 7f3ab39ff700 1 -- 172.21.15.31:0/163943320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac15acc0 con 0x7f3aac15a190 2023-10-06T05:20:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.248+0000 7f3ab39ff700 1 -- 172.21.15.31:0/163943320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac0c9f30 con 0x7f3aac15a190 2023-10-06T05:20:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.249+0000 7f3ab39ff700 1 -- 172.21.15.31:0/163943320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac0ca0f0 con 0x7f3aac15a190 2023-10-06T05:20:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.250+0000 7f3aa17fa700 1 -- 172.21.15.31:0/163943320 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f3aa4003020 con 0x7f3aac15a190 2023-10-06T05:20:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.251+0000 7f3aa17fa700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3a900670e0 0x7f3a90069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.251+0000 7f3aa17fa700 1 -- 172.21.15.31:0/163943320 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f3aa4085050 con 0x7f3aac15a190 2023-10-06T05:20:21.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.251+0000 7f3ab179b700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3a900670e0 0x7f3a90069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.252+0000 7f3ab179b700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3a900670e0 0x7f3a90069590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8000c00 tx=0x7f3aa8015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:21.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.256+0000 7f3aa17fa700 1 -- 172.21.15.31:0/163943320 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3aa4051d90 con 0x7f3aac15a190 2023-10-06T05:20:21.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.458+0000 7f3ab39ff700 1 -- 172.21.15.31:0/163943320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3aac04c380 con 0x7f3aac15a190 2023-10-06T05:20:21.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.458+0000 7f3aa17fa700 1 -- 172.21.15.31:0/163943320 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3aa4010ea0 con 0x7f3aac15a190 2023-10-06T05:20:21.460 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:20:21.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.460+0000 7f3ab39ff700 1 -- 172.21.15.31:0/163943320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3a900670e0 msgr2=0x7f3a90069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.460+0000 7f3ab39ff700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3a900670e0 0x7f3a90069590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8000c00 tx=0x7f3aa8015000 comp rx=0 tx=0).stop 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.460+0000 7f3ab39ff700 1 -- 172.21.15.31:0/163943320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3aac15a190 msgr2=0x7f3aac0cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.460+0000 7f3ab39ff700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3aac15a190 0x7f3aac0cbce0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0b1c90 tx=0x7f3aa400b040 comp rx=0 tx=0).stop 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.460+0000 7f3ab39ff700 1 -- 172.21.15.31:0/163943320 shutdown_connections 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.460+0000 7f3ab39ff700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3a900670e0 0x7f3a90069590 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.461+0000 7f3ab39ff700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3aac15a190 0x7f3aac0cbce0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.461+0000 7f3ab39ff700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98003bf0 0x7f3aac1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.461+0000 7f3ab39ff700 1 --2- 172.21.15.31:0/163943320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3aac159d20 0x7f3aac0cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.461+0000 7f3ab39ff700 1 -- 172.21.15.31:0/163943320 >> 172.21.15.31:0/163943320 conn(0x7f3aac06ec90 msgr2=0x7f3aac1562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.461+0000 7f3ab39ff700 1 -- 172.21.15.31:0/163943320 shutdown_connections 2023-10-06T05:20:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.461+0000 7f3ab39ff700 1 -- 172.21.15.31:0/163943320 wait complete. 2023-10-06T05:20:21.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:20:21.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.676+0000 7f3c2bdff700 1 Processor -- start 2023-10-06T05:20:21.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.676+0000 7f3c2bdff700 1 -- start start 2023-10-06T05:20:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.677+0000 7f3c2bdff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c241573a0 0x7f3c240acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.677+0000 7f3c2bdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c240ae860 0x7f3c241611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.677+0000 7f3c2bdff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c24158cf0 con 0x7f3c24157770 2023-10-06T05:20:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.677+0000 7f3c2bdff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c241580e0 con 0x7f3c241573a0 2023-10-06T05:20:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.677+0000 7f3c2bdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c241583c0 con 0x7f3c240ae860 2023-10-06T05:20:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.677+0000 7f3c29b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c241573a0 0x7f3c240acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.677+0000 7f3c29b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c241573a0 0x7f3c240acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55704/0 (socket says 172.21.15.31:55704) 2023-10-06T05:20:21.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.677+0000 7f3c29b9b700 1 -- 172.21.15.31:0/1178291511 learned_addr learned my addr 172.21.15.31:0/1178291511 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:21.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.678+0000 7f3c2a39c700 1 --2- 172.21.15.31:0/1178291511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c240ae860 0x7f3c241611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.678+0000 7f3c29b9b700 1 -- 172.21.15.31:0/1178291511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c240ae860 msgr2=0x7f3c241611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.678+0000 7f3c29b9b700 1 --2- 172.21.15.31:0/1178291511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c240ae860 0x7f3c241611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.678+0000 7f3c29b9b700 1 -- 172.21.15.31:0/1178291511 >> v1:172.21.15.62:6789/0 conn(0x7f3c24157770 legacy=0x7f3c240b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.678+0000 7f3c29b9b700 1 -- 172.21.15.31:0/1178291511 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c241586a0 con 0x7f3c241573a0 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.678+0000 7f3c2a39c700 1 --2- 172.21.15.31:0/1178291511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c240ae860 0x7f3c241611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.678+0000 7f3c29b9b700 1 --2- 172.21.15.31:0/1178291511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 0x7f3c240acb70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3c200099b0 tx=0x7f3c20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f3a802ff68bffe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.678+0000 7f3c28b99700 1 -- 172.21.15.31:0/1178291511 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000b820 con 0x7f3c241573a0 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.678+0000 7f3c28b99700 1 -- 172.21.15.31:0/1178291511 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2000b980 con 0x7f3c241573a0 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.678+0000 7f3c28b99700 1 -- 172.21.15.31:0/1178291511 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000bd20 con 0x7f3c241573a0 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.679+0000 7f3c2bdff700 1 -- 172.21.15.31:0/1178291511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 msgr2=0x7f3c240acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.679+0000 7f3c2bdff700 1 --2- 172.21.15.31:0/1178291511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 0x7f3c240acb70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3c200099b0 tx=0x7f3c20009250 comp rx=0 tx=0).stop 2023-10-06T05:20:21.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.679+0000 7f3c2bdff700 1 -- 172.21.15.31:0/1178291511 shutdown_connections 2023-10-06T05:20:21.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.679+0000 7f3c2bdff700 1 --2- 172.21.15.31:0/1178291511 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c240ae860 0x7f3c241611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.679+0000 7f3c2bdff700 1 --2- 172.21.15.31:0/1178291511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 0x7f3c240acb70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.679+0000 7f3c2bdff700 1 -- 172.21.15.31:0/1178291511 >> 172.21.15.31:0/1178291511 conn(0x7f3c24067560 msgr2=0x7f3c24067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:21.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.679+0000 7f3c2bdff700 1 -- 172.21.15.31:0/1178291511 shutdown_connections 2023-10-06T05:20:21.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.679+0000 7f3c2bdff700 1 -- 172.21.15.31:0/1178291511 wait complete. 2023-10-06T05:20:21.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.681+0000 7f3c2bdff700 1 Processor -- start 2023-10-06T05:20:21.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.681+0000 7f3c2bdff700 1 -- start start 2023-10-06T05:20:21.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c2bdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 0x7f3c2414eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c2bdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c24157770 0x7f3c2414bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c2bdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2414cfe0 0x7f3c2414d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c2bdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2414f060 con 0x7f3c241573a0 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c2bdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2414da80 con 0x7f3c2414cfe0 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c2bdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2414dd60 con 0x7f3c24157770 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c29b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 0x7f3c2414eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c29b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 0x7f3c2414eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55720/0 (socket says 172.21.15.31:55720) 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c29b9b700 1 -- 172.21.15.31:0/2738461718 learned_addr learned my addr 172.21.15.31:0/2738461718 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c2a39c700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2414cfe0 0x7f3c2414d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c2939a700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c24157770 0x7f3c2414bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c29b9b700 1 -- 172.21.15.31:0/2738461718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c24157770 msgr2=0x7f3c2414bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c29b9b700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c24157770 0x7f3c2414bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c29b9b700 1 -- 172.21.15.31:0/2738461718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2414cfe0 msgr2=0x7f3c2414d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c29b9b700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2414cfe0 0x7f3c2414d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.682+0000 7f3c29b9b700 1 -- 172.21.15.31:0/2738461718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200094f0 con 0x7f3c241573a0 2023-10-06T05:20:21.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.683+0000 7f3c29b9b700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 0x7f3c2414eb20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009290 tx=0x7f3c20016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:21.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.683+0000 7f3c19ffb700 1 -- 172.21.15.31:0/2738461718 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20011d80 con 0x7f3c241573a0 2023-10-06T05:20:21.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.683+0000 7f3c19ffb700 1 -- 172.21.15.31:0/2738461718 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c20020070 con 0x7f3c241573a0 2023-10-06T05:20:21.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.683+0000 7f3c19ffb700 1 -- 172.21.15.31:0/2738461718 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2001b6a0 con 0x7f3c241573a0 2023-10-06T05:20:21.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.683+0000 7f3c2bdff700 1 -- 172.21.15.31:0/2738461718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2414e040 con 0x7f3c241573a0 2023-10-06T05:20:21.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.683+0000 7f3c2bdff700 1 -- 172.21.15.31:0/2738461718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2414e600 con 0x7f3c241573a0 2023-10-06T05:20:21.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.684+0000 7f3c2bdff700 1 -- 172.21.15.31:0/2738461718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2414e7f0 con 0x7f3c241573a0 2023-10-06T05:20:21.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.685+0000 7f3c19ffb700 1 -- 172.21.15.31:0/2738461718 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f3c20022030 con 0x7f3c241573a0 2023-10-06T05:20:21.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.686+0000 7f3c19ffb700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3c08067110 0x7f3c080695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:21.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.686+0000 7f3c19ffb700 1 -- 172.21.15.31:0/2738461718 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f3c20086080 con 0x7f3c241573a0 2023-10-06T05:20:21.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.686+0000 7f3c2939a700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3c08067110 0x7f3c080695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:21.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.687+0000 7f3c2939a700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3c08067110 0x7f3c080695c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3c24152bd0 tx=0x7f3c14007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.691+0000 7f3c19ffb700 1 -- 172.21.15.31:0/2738461718 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c20052e30 con 0x7f3c241573a0 2023-10-06T05:20:21.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.890+0000 7f3c2bdff700 1 -- 172.21.15.31:0/2738461718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3c2404c380 con 0x7f3c241573a0 2023-10-06T05:20:21.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.891+0000 7f3c19ffb700 1 -- 172.21.15.31:0/2738461718 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3c2002b0a0 con 0x7f3c241573a0 2023-10-06T05:20:21.893 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:20:21.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 -- 172.21.15.31:0/2738461718 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3c08067110 msgr2=0x7f3c080695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3c08067110 0x7f3c080695c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3c24152bd0 tx=0x7f3c14007ea0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 -- 172.21.15.31:0/2738461718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 msgr2=0x7f3c2414eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:21.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 0x7f3c2414eb20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009290 tx=0x7f3c20016000 comp rx=0 tx=0).stop 2023-10-06T05:20:21.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 -- 172.21.15.31:0/2738461718 shutdown_connections 2023-10-06T05:20:21.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c2414cfe0 0x7f3c2414d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3c08067110 0x7f3c080695c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c241573a0 0x7f3c2414eb20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 --2- 172.21.15.31:0/2738461718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c24157770 0x7f3c2414bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:21.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 -- 172.21.15.31:0/2738461718 >> 172.21.15.31:0/2738461718 conn(0x7f3c24067560 msgr2=0x7f3c2414c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:21.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.893+0000 7f3c2bdff700 1 -- 172.21.15.31:0/2738461718 shutdown_connections 2023-10-06T05:20:21.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:21.894+0000 7f3c2bdff700 1 -- 172.21.15.31:0/2738461718 wait complete. 2023-10-06T05:20:21.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:22.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.105+0000 7f3f105ff700 1 Processor -- start 2023-10-06T05:20:22.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.105+0000 7f3f105ff700 1 -- start start 2023-10-06T05:20:22.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.107+0000 7f3f105ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f080c7b60 0x7f3f08069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.107+0000 7f3f105ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3f080c7f30 0x7f3f080aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.107+0000 7f3f105ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f080ca5b0 con 0x7f3f0806ef50 2023-10-06T05:20:22.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.107+0000 7f3f105ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0806f740 con 0x7f3f080c7b60 2023-10-06T05:20:22.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.107+0000 7f3f105ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f080c9c80 con 0x7f3f080c7f30 2023-10-06T05:20:22.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.107+0000 7f3f0e39b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f080c7b60 0x7f3f08069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.107+0000 7f3f0eb9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3f0806ef50 0x7f3f0813f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46570/0 (socket says 172.21.15.31:46570) 2023-10-06T05:20:22.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.107+0000 7f3f0eb9c700 1 -- 172.21.15.31:0/1328105775 learned_addr learned my addr 172.21.15.31:0/1328105775 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:22.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.107+0000 7f3f0db9a700 1 --2- 172.21.15.31:0/1328105775 >> v2:172.21.15.62:3300/0 conn(0x7f3f080c7f30 0x7f3f080aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.108+0000 7f3f0e39b700 1 -- 172.21.15.31:0/1328105775 >> v2:172.21.15.62:3300/0 conn(0x7f3f080c7f30 msgr2=0x7f3f080aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.108+0000 7f3f0e39b700 1 --2- 172.21.15.31:0/1328105775 >> v2:172.21.15.62:3300/0 conn(0x7f3f080c7f30 0x7f3f080aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.108+0000 7f3f0e39b700 1 -- 172.21.15.31:0/1328105775 >> v1:172.21.15.31:6789/0 conn(0x7f3f0806ef50 legacy=0x7f3f0813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.108+0000 7f3f0e39b700 1 -- 172.21.15.31:0/1328105775 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f080c9f60 con 0x7f3f080c7b60 2023-10-06T05:20:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.108+0000 7f3f0db9a700 1 --2- 172.21.15.31:0/1328105775 >> v2:172.21.15.62:3300/0 conn(0x7f3f080c7f30 0x7f3f080aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.108+0000 7f3f0d399700 1 -- 172.21.15.31:0/1328105775 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268069990 0 0) 0x7f3f080ca5b0 con 0x7f3f0806ef50 2023-10-06T05:20:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.108+0000 7f3f0e39b700 1 --2- 172.21.15.31:0/1328105775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f080c7b60 0x7f3f08069880 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f040098d0 tx=0x7f3f04009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613c01f9d05cea9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.108+0000 7f3f0d399700 1 -- 172.21.15.31:0/1328105775 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04005c10 con 0x7f3f080c7b60 2023-10-06T05:20:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.108+0000 7f3f0d399700 1 -- 172.21.15.31:0/1328105775 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04005d70 con 0x7f3f080c7b60 2023-10-06T05:20:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.108+0000 7f3f0d399700 1 -- 172.21.15.31:0/1328105775 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04006280 con 0x7f3f080c7b60 2023-10-06T05:20:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.109+0000 7f3f105ff700 1 -- 172.21.15.31:0/1328105775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f080c7b60 msgr2=0x7f3f08069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.109+0000 7f3f105ff700 1 --2- 172.21.15.31:0/1328105775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f080c7b60 0x7f3f08069880 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f040098d0 tx=0x7f3f04009120 comp rx=0 tx=0).stop 2023-10-06T05:20:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.109+0000 7f3f105ff700 1 -- 172.21.15.31:0/1328105775 shutdown_connections 2023-10-06T05:20:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.109+0000 7f3f105ff700 1 --2- 172.21.15.31:0/1328105775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f080c7b60 0x7f3f08069880 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.109+0000 7f3f105ff700 1 --2- 172.21.15.31:0/1328105775 >> v2:172.21.15.62:3300/0 conn(0x7f3f080c7f30 0x7f3f080aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.109+0000 7f3f105ff700 1 -- 172.21.15.31:0/1328105775 >> 172.21.15.31:0/1328105775 conn(0x7f3f080cc0b0 msgr2=0x7f3f080cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:22.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.109+0000 7f3f105ff700 1 -- 172.21.15.31:0/1328105775 shutdown_connections 2023-10-06T05:20:22.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.109+0000 7f3f105ff700 1 -- 172.21.15.31:0/1328105775 wait complete. 2023-10-06T05:20:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.110+0000 7f3f105ff700 1 Processor -- start 2023-10-06T05:20:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.111+0000 7f3f105ff700 1 -- start start 2023-10-06T05:20:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f105ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f0806ef50 0x7f3f08157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f105ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f080c7b60 0x7f3f080c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f105ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f080c7f30 0x7f3f080c1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f105ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f08157590 con 0x7f3f080c7f30 2023-10-06T05:20:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f105ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f08164800 con 0x7f3f0806ef50 2023-10-06T05:20:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f105ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f08138dd0 con 0x7f3f080c7b60 2023-10-06T05:20:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0eb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f080c7f30 0x7f3f080c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0e39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f0806ef50 0x7f3f08157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0e39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f0806ef50 0x7f3f08157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:57824/0 (socket says 172.21.15.31:57824) 2023-10-06T05:20:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0e39b700 1 -- 172.21.15.31:0/965261260 learned_addr learned my addr 172.21.15.31:0/965261260 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0db9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f080c7b60 0x7f3f080c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0eb9c700 1 -- 172.21.15.31:0/965261260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f080c7b60 msgr2=0x7f3f080c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0eb9c700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f080c7b60 0x7f3f080c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0eb9c700 1 -- 172.21.15.31:0/965261260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f0806ef50 msgr2=0x7f3f08157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0eb9c700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f0806ef50 0x7f3f08157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0eb9c700 1 -- 172.21.15.31:0/965261260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8009420 con 0x7f3f080c7f30 2023-10-06T05:20:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.112+0000 7f3f0e39b700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f0806ef50 0x7f3f08157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.113+0000 7f3f0eb9c700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f080c7f30 0x7f3f080c1390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000cb90 tx=0x7f3f000018f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.113+0000 7f3efe7fc700 1 -- 172.21.15.31:0/965261260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000a780 con 0x7f3f080c7f30 2023-10-06T05:20:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.113+0000 7f3efe7fc700 1 -- 172.21.15.31:0/965261260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0000a8e0 con 0x7f3f080c7f30 2023-10-06T05:20:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.113+0000 7f3efe7fc700 1 -- 172.21.15.31:0/965261260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000aca0 con 0x7f3f080c7f30 2023-10-06T05:20:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.113+0000 7f3f105ff700 1 -- 172.21.15.31:0/965261260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f040093c0 con 0x7f3f080c7f30 2023-10-06T05:20:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.113+0000 7f3f105ff700 1 -- 172.21.15.31:0/965261260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f081641b0 con 0x7f3f080c7f30 2023-10-06T05:20:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.114+0000 7f3f105ff700 1 -- 172.21.15.31:0/965261260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f081649b0 con 0x7f3f080c7f30 2023-10-06T05:20:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.115+0000 7f3efe7fc700 1 -- 172.21.15.31:0/965261260 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f3f00007300 con 0x7f3f080c7f30 2023-10-06T05:20:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.116+0000 7f3efe7fc700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3eec0670e0 0x7f3eec069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.116+0000 7f3efe7fc700 1 -- 172.21.15.31:0/965261260 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f3f00084c60 con 0x7f3f080c7f30 2023-10-06T05:20:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.116+0000 7f3f0e39b700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3eec0670e0 0x7f3eec069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.117+0000 7f3f0e39b700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3eec0670e0 0x7f3eec069590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006d60 tx=0x7f3f04016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.120+0000 7f3efe7fc700 1 -- 172.21.15.31:0/965261260 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f081649b0 con 0x7f3f080c7f30 2023-10-06T05:20:22.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.370+0000 7f3f105ff700 1 -- 172.21.15.31:0/965261260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f08164970 con 0x7f3f080c7f30 2023-10-06T05:20:22.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.370+0000 7f3efe7fc700 1 -- 172.21.15.31:0/965261260 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f3f00051a90 con 0x7f3f080c7f30 2023-10-06T05:20:22.372 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:22.372 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:20:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 -- 172.21.15.31:0/965261260 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3eec0670e0 msgr2=0x7f3eec069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3eec0670e0 0x7f3eec069590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006d60 tx=0x7f3f04016000 comp rx=0 tx=0).stop 2023-10-06T05:20:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 -- 172.21.15.31:0/965261260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f080c7f30 msgr2=0x7f3f080c1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f080c7f30 0x7f3f080c1390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000cb90 tx=0x7f3f000018f0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 -- 172.21.15.31:0/965261260 shutdown_connections 2023-10-06T05:20:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f3eec0670e0 0x7f3eec069590 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f080c7f30 0x7f3f080c1390 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f0806ef50 0x7f3f08157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 --2- 172.21.15.31:0/965261260 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f080c7b60 0x7f3f080c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 -- 172.21.15.31:0/965261260 >> 172.21.15.31:0/965261260 conn(0x7f3f080cc0b0 msgr2=0x7f3f080cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 -- 172.21.15.31:0/965261260 shutdown_connections 2023-10-06T05:20:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.373+0000 7f3f105ff700 1 -- 172.21.15.31:0/965261260 wait complete. 2023-10-06T05:20:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:20:22.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:22.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.589+0000 7f6db2dff700 1 Processor -- start 2023-10-06T05:20:22.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.589+0000 7f6db2dff700 1 -- start start 2023-10-06T05:20:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.590+0000 7f6db2dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6dac0c7b60 0x7f6dac069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.590+0000 7f6db2dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6dac0c7f30 0x7f6dac0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.590+0000 7f6db2dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dac0ca5b0 con 0x7f6dac06ef50 2023-10-06T05:20:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.590+0000 7f6db2dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dac06f740 con 0x7f6dac0c7b60 2023-10-06T05:20:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.590+0000 7f6db2dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dac0c9c80 con 0x7f6dac0c7f30 2023-10-06T05:20:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.591+0000 7f6db0b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6dac0c7b60 0x7f6dac069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.591+0000 7f6db139c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6dac06ef50 0x7f6dac13f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46572/0 (socket says 172.21.15.31:46572) 2023-10-06T05:20:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.591+0000 7f6db139c700 1 -- 172.21.15.31:0/4219535988 learned_addr learned my addr 172.21.15.31:0/4219535988 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.591+0000 7f6dabfff700 1 --2- 172.21.15.31:0/4219535988 >> v2:172.21.15.62:3300/0 conn(0x7f6dac0c7f30 0x7f6dac0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.591+0000 7f6db0b9b700 1 -- 172.21.15.31:0/4219535988 >> v2:172.21.15.62:3300/0 conn(0x7f6dac0c7f30 msgr2=0x7f6dac0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.591+0000 7f6db0b9b700 1 --2- 172.21.15.31:0/4219535988 >> v2:172.21.15.62:3300/0 conn(0x7f6dac0c7f30 0x7f6dac0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.591+0000 7f6db0b9b700 1 -- 172.21.15.31:0/4219535988 >> v1:172.21.15.31:6789/0 conn(0x7f6dac06ef50 legacy=0x7f6dac13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.591+0000 7f6db0b9b700 1 -- 172.21.15.31:0/4219535988 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac0c9f60 con 0x7f6dac0c7b60 2023-10-06T05:20:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.591+0000 7f6db0b9b700 1 --2- 172.21.15.31:0/4219535988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dac0c7b60 0x7f6dac069880 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6da00098d0 tx=0x7f6da0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bab629f331d6bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6dab7fe700 1 -- 172.21.15.31:0/4219535988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0005c10 con 0x7f6dac0c7b60 2023-10-06T05:20:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6dab7fe700 1 -- 172.21.15.31:0/4219535988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da0005d70 con 0x7f6dac0c7b60 2023-10-06T05:20:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6dab7fe700 1 -- 172.21.15.31:0/4219535988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0006280 con 0x7f6dac0c7b60 2023-10-06T05:20:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6db2dff700 1 -- 172.21.15.31:0/4219535988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dac0c7b60 msgr2=0x7f6dac069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6db2dff700 1 --2- 172.21.15.31:0/4219535988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dac0c7b60 0x7f6dac069880 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6da00098d0 tx=0x7f6da0009120 comp rx=0 tx=0).stop 2023-10-06T05:20:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6db2dff700 1 -- 172.21.15.31:0/4219535988 shutdown_connections 2023-10-06T05:20:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6db2dff700 1 --2- 172.21.15.31:0/4219535988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dac0c7b60 0x7f6dac069880 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6db2dff700 1 --2- 172.21.15.31:0/4219535988 >> v2:172.21.15.62:3300/0 conn(0x7f6dac0c7f30 0x7f6dac0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6db2dff700 1 -- 172.21.15.31:0/4219535988 >> 172.21.15.31:0/4219535988 conn(0x7f6dac0cc0b0 msgr2=0x7f6dac0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6db2dff700 1 -- 172.21.15.31:0/4219535988 shutdown_connections 2023-10-06T05:20:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.592+0000 7f6db2dff700 1 -- 172.21.15.31:0/4219535988 wait complete. 2023-10-06T05:20:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.594+0000 7f6db2dff700 1 Processor -- start 2023-10-06T05:20:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.594+0000 7f6db2dff700 1 -- start start 2023-10-06T05:20:22.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6db2dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6dac06ef50 0x7f6dac156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6db2dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dac0c7b60 0x7f6dac0ab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6db2dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dac0c7f30 0x7f6dac0abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6db2dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac0ad930 con 0x7f6dac0c7f30 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6db2dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac0ac270 con 0x7f6dac0c7b60 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6db2dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac0ac550 con 0x7f6dac06ef50 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6db139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dac0c7f30 0x7f6dac0abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6dabfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dac0c7b60 0x7f6dac0ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6db0b9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6dac06ef50 0x7f6dac156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6dabfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dac0c7b60 0x7f6dac0ab700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:57834/0 (socket says 172.21.15.31:57834) 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6dabfff700 1 -- 172.21.15.31:0/1511520479 learned_addr learned my addr 172.21.15.31:0/1511520479 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.595+0000 7f6db0b9b700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6dac06ef50 0x7f6dac156020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:59076/0 (socket says 172.21.15.31:59076) 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.596+0000 7f6db139c700 1 -- 172.21.15.31:0/1511520479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6dac06ef50 msgr2=0x7f6dac156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.596+0000 7f6db139c700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6dac06ef50 0x7f6dac156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.596+0000 7f6db139c700 1 -- 172.21.15.31:0/1511520479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dac0c7b60 msgr2=0x7f6dac0ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.596+0000 7f6db139c700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dac0c7b60 0x7f6dac0ab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.596+0000 7f6db139c700 1 -- 172.21.15.31:0/1511520479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00093c0 con 0x7f6dac0c7f30 2023-10-06T05:20:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.596+0000 7f6dabfff700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dac0c7b60 0x7f6dac0ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.596+0000 7f6db139c700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dac0c7f30 0x7f6dac0abc40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6da400cb40 tx=0x7f6da4001900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.596+0000 7f6da8ff9700 1 -- 172.21.15.31:0/1511520479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da400a7e0 con 0x7f6dac0c7f30 2023-10-06T05:20:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.596+0000 7f6da8ff9700 1 -- 172.21.15.31:0/1511520479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da400a940 con 0x7f6dac0c7f30 2023-10-06T05:20:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.596+0000 7f6db2dff700 1 -- 172.21.15.31:0/1511520479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dac0ac830 con 0x7f6dac0c7f30 2023-10-06T05:20:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.597+0000 7f6da8ff9700 1 -- 172.21.15.31:0/1511520479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da400ace0 con 0x7f6dac0c7f30 2023-10-06T05:20:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.597+0000 7f6db2dff700 1 -- 172.21.15.31:0/1511520479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dac156560 con 0x7f6dac0c7f30 2023-10-06T05:20:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.598+0000 7f6db2dff700 1 -- 172.21.15.31:0/1511520479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dac04c380 con 0x7f6dac0c7f30 2023-10-06T05:20:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.598+0000 7f6da8ff9700 1 -- 172.21.15.31:0/1511520479 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6da4007300 con 0x7f6dac0c7f30 2023-10-06T05:20:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.599+0000 7f6da8ff9700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d900670e0 0x7f6d90069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.599+0000 7f6da8ff9700 1 -- 172.21.15.31:0/1511520479 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f6da4084cb0 con 0x7f6dac0c7f30 2023-10-06T05:20:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.600+0000 7f6db0b9b700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d900670e0 0x7f6d90069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.600+0000 7f6db0b9b700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d900670e0 0x7f6d90069590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009160 tx=0x7f6da0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:22.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.604+0000 7f6da8ff9700 1 -- 172.21.15.31:0/1511520479 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6da4051a50 con 0x7f6dac0c7f30 2023-10-06T05:20:22.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.809+0000 7f6db2dff700 1 -- 172.21.15.31:0/1511520479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dac0cb5e0 con 0x7f6dac0c7f30 2023-10-06T05:20:22.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.810+0000 7f6da8ff9700 1 -- 172.21.15.31:0/1511520479 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+17009 (secure 0 0 0) 0x7f6da4055070 con 0x7f6dac0c7f30 2023-10-06T05:20:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:22.813 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:50.146155+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:36.639713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:37.929704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:22.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 -- 172.21.15.31:0/1511520479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d900670e0 msgr2=0x7f6d90069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d900670e0 0x7f6d90069590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009160 tx=0x7f6da0018000 comp rx=0 tx=0).stop 2023-10-06T05:20:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 -- 172.21.15.31:0/1511520479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dac0c7f30 msgr2=0x7f6dac0abc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dac0c7f30 0x7f6dac0abc40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6da400cb40 tx=0x7f6da4001900 comp rx=0 tx=0).stop 2023-10-06T05:20:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 -- 172.21.15.31:0/1511520479 shutdown_connections 2023-10-06T05:20:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d900670e0 0x7f6d90069590 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dac0c7f30 0x7f6dac0abc40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6dac06ef50 0x7f6dac156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 --2- 172.21.15.31:0/1511520479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dac0c7b60 0x7f6dac0ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 -- 172.21.15.31:0/1511520479 >> 172.21.15.31:0/1511520479 conn(0x7f6dac0cc0b0 msgr2=0x7f6dac0c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 -- 172.21.15.31:0/1511520479 shutdown_connections 2023-10-06T05:20:22.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:22.813+0000 7f6db2dff700 1 -- 172.21.15.31:0/1511520479 wait complete. 2023-10-06T05:20:22.832 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:20:22.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:23.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.025+0000 7fc013bff700 1 Processor -- start 2023-10-06T05:20:23.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.025+0000 7fc013bff700 1 -- start start 2023-10-06T05:20:23.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.026+0000 7fc013bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc00c1596d0 0x7fc00c159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.026+0000 7fc013bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc00c0c6f80 0x7fc00c06a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.026+0000 7fc013bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00c067f70 con 0x7fc00c0c16e0 2023-10-06T05:20:23.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.026+0000 7fc013bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00c06e040 con 0x7fc00c1596d0 2023-10-06T05:20:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.026+0000 7fc013bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00c0681c0 con 0x7fc00c0c6f80 2023-10-06T05:20:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.027+0000 7fc01199b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc00c1596d0 0x7fc00c159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.027+0000 7fc01199b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc00c1596d0 0x7fc00c159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55772/0 (socket says 172.21.15.31:55772) 2023-10-06T05:20:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.027+0000 7fc01199b700 1 -- 172.21.15.31:0/3395907276 learned_addr learned my addr 172.21.15.31:0/3395907276 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:23.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.027+0000 7fc01119a700 1 --2- 172.21.15.31:0/3395907276 >> v2:172.21.15.62:3300/0 conn(0x7fc00c0c6f80 0x7fc00c06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.027+0000 7fc01199b700 1 -- 172.21.15.31:0/3395907276 >> v2:172.21.15.62:3300/0 conn(0x7fc00c0c6f80 msgr2=0x7fc00c06a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:23.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.027+0000 7fc01199b700 1 --2- 172.21.15.31:0/3395907276 >> v2:172.21.15.62:3300/0 conn(0x7fc00c0c6f80 0x7fc00c06a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.027+0000 7fc01199b700 1 -- 172.21.15.31:0/3395907276 >> v1:172.21.15.62:6789/0 conn(0x7fc00c0c16e0 legacy=0x7fc00c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.027+0000 7fc01199b700 1 -- 172.21.15.31:0/3395907276 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c0684a0 con 0x7fc00c1596d0 2023-10-06T05:20:23.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.027+0000 7fc01199b700 1 --2- 172.21.15.31:0/3395907276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 0x7fc00c159aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc00c06dbe0 tx=0x7fc008009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7ee56f961b2d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:23.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc010999700 1 -- 172.21.15.31:0/3395907276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00800b860 con 0x7fc00c1596d0 2023-10-06T05:20:23.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc010999700 1 -- 172.21.15.31:0/3395907276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00800b9c0 con 0x7fc00c1596d0 2023-10-06T05:20:23.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc010999700 1 -- 172.21.15.31:0/3395907276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00800bd60 con 0x7fc00c1596d0 2023-10-06T05:20:23.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc013bff700 1 -- 172.21.15.31:0/3395907276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 msgr2=0x7fc00c159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc013bff700 1 --2- 172.21.15.31:0/3395907276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 0x7fc00c159aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc00c06dbe0 tx=0x7fc008009250 comp rx=0 tx=0).stop 2023-10-06T05:20:23.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc013bff700 1 -- 172.21.15.31:0/3395907276 shutdown_connections 2023-10-06T05:20:23.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc013bff700 1 --2- 172.21.15.31:0/3395907276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 0x7fc00c159aa0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc013bff700 1 --2- 172.21.15.31:0/3395907276 >> v2:172.21.15.62:3300/0 conn(0x7fc00c0c6f80 0x7fc00c06a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc013bff700 1 -- 172.21.15.31:0/3395907276 >> 172.21.15.31:0/3395907276 conn(0x7fc00c0c9e80 msgr2=0x7fc00c06a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:23.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc013bff700 1 -- 172.21.15.31:0/3395907276 shutdown_connections 2023-10-06T05:20:23.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.028+0000 7fc013bff700 1 -- 172.21.15.31:0/3395907276 wait complete. 2023-10-06T05:20:23.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.030+0000 7fc013bff700 1 Processor -- start 2023-10-06T05:20:23.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.030+0000 7fc013bff700 1 -- start start 2023-10-06T05:20:23.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc013bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc00c0c16e0 0x7fc00c0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc013bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00c0c6f80 0x7fc00c0c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc013bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 0x7fc00c0c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc013bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c0b08c0 con 0x7fc00c1596d0 2023-10-06T05:20:23.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc013bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c0afcb0 con 0x7fc00c0c16e0 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc013bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c0aff90 con 0x7fc00c0c6f80 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc01219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 0x7fc00c0c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc01219c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 0x7fc00c0c4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55784/0 (socket says 172.21.15.31:55784) 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc01219c700 1 -- 172.21.15.31:0/2490268564 learned_addr learned my addr 172.21.15.31:0/2490268564 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc01199b700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc00c0c16e0 0x7fc00c0b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.031+0000 7fc01119a700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00c0c6f80 0x7fc00c0c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc01219c700 1 -- 172.21.15.31:0/2490268564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00c0c6f80 msgr2=0x7fc00c0c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc01219c700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00c0c6f80 0x7fc00c0c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc01219c700 1 -- 172.21.15.31:0/2490268564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc00c0c16e0 msgr2=0x7fc00c0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc01219c700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc00c0c16e0 0x7fc00c0b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc01219c700 1 -- 172.21.15.31:0/2490268564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0080094f0 con 0x7fc00c1596d0 2023-10-06T05:20:23.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc01119a700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00c0c6f80 0x7fc00c0c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:23.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc01219c700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 0x7fc00c0c4350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc00400c8e0 tx=0x7fc004001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:23.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc001ffb700 1 -- 172.21.15.31:0/2490268564 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004012630 con 0x7fc00c1596d0 2023-10-06T05:20:23.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc001ffb700 1 -- 172.21.15.31:0/2490268564 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc004012790 con 0x7fc00c1596d0 2023-10-06T05:20:23.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc001ffb700 1 -- 172.21.15.31:0/2490268564 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004012b50 con 0x7fc00c1596d0 2023-10-06T05:20:23.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc013bff700 1 -- 172.21.15.31:0/2490268564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00c0b0270 con 0x7fc00c1596d0 2023-10-06T05:20:23.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.032+0000 7fc013bff700 1 -- 172.21.15.31:0/2490268564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c157940 con 0x7fc00c1596d0 2023-10-06T05:20:23.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.033+0000 7fc013bff700 1 -- 172.21.15.31:0/2490268564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00c0b0a70 con 0x7fc00c1596d0 2023-10-06T05:20:23.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.034+0000 7fc001ffb700 1 -- 172.21.15.31:0/2490268564 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fc00400a3b0 con 0x7fc00c1596d0 2023-10-06T05:20:23.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.035+0000 7fc001ffb700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbff00670e0 0x7fbff0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.035+0000 7fc001ffb700 1 -- 172.21.15.31:0/2490268564 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fc004084d00 con 0x7fc00c1596d0 2023-10-06T05:20:23.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.036+0000 7fc01199b700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbff00670e0 0x7fbff0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.036+0000 7fc01199b700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbff00670e0 0x7fbff0069590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc00c06dbe0 tx=0x7fc008017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:23.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.039+0000 7fc001ffb700 1 -- 172.21.15.31:0/2490268564 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc00c0b0a70 con 0x7fc00c1596d0 2023-10-06T05:20:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.289+0000 7fc013bff700 1 -- 172.21.15.31:0/2490268564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc00c04c380 con 0x7fc00c1596d0 2023-10-06T05:20:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.290+0000 7fc001ffb700 1 -- 172.21.15.31:0/2490268564 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fc004055540 con 0x7fc00c1596d0 2023-10-06T05:20:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:23.292 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:20:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 -- 172.21.15.31:0/2490268564 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbff00670e0 msgr2=0x7fbff0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbff00670e0 0x7fbff0069590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc00c06dbe0 tx=0x7fc008017000 comp rx=0 tx=0).stop 2023-10-06T05:20:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 -- 172.21.15.31:0/2490268564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 msgr2=0x7fc00c0c4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 0x7fc00c0c4350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc00400c8e0 tx=0x7fc004001710 comp rx=0 tx=0).stop 2023-10-06T05:20:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 -- 172.21.15.31:0/2490268564 shutdown_connections 2023-10-06T05:20:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fbff00670e0 0x7fbff0069590 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc00c1596d0 0x7fc00c0c4350 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc00c0c16e0 0x7fc00c0b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 --2- 172.21.15.31:0/2490268564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc00c0c6f80 0x7fc00c0c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 -- 172.21.15.31:0/2490268564 >> 172.21.15.31:0/2490268564 conn(0x7fc00c0c9e80 msgr2=0x7fc00c0c8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 -- 172.21.15.31:0/2490268564 shutdown_connections 2023-10-06T05:20:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.292+0000 7fc013bff700 1 -- 172.21.15.31:0/2490268564 wait complete. 2023-10-06T05:20:23.295 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:20:23.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:23.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.507+0000 7f9f18fff700 1 Processor -- start 2023-10-06T05:20:23.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.508+0000 7f9f18fff700 1 -- start start 2023-10-06T05:20:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.509+0000 7f9f18fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f1415d2d0 0x7f9f140b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.509+0000 7f9f18fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f140c3f40 con 0x7f9f14159500 2023-10-06T05:20:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.509+0000 7f9f18fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f140cb000 con 0x7f9f140cd510 2023-10-06T05:20:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.509+0000 7f9f18fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f140b2860 con 0x7f9f1415d2d0 2023-10-06T05:20:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.509+0000 7f9f12d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f1415d2d0 0x7f9f140b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.509+0000 7f9f1259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9f14159500 0x7f9f140cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46576/0 (socket says 172.21.15.31:46576) 2023-10-06T05:20:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.509+0000 7f9f1259c700 1 -- 172.21.15.31:0/2882056088 learned_addr learned my addr 172.21.15.31:0/2882056088 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.510+0000 7f9f1159a700 1 -- 172.21.15.31:0/2882056088 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728149416 0 0) 0x7f9f140c3f40 con 0x7f9f14159500 2023-10-06T05:20:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.510+0000 7f9f1159a700 1 -- 172.21.15.31:0/2882056088 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9efc003570 con 0x7f9f14159500 2023-10-06T05:20:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.510+0000 7f9f1159a700 1 -- 172.21.15.31:0/2882056088 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890604239 0 0) 0x7f9f140cb000 con 0x7f9f140cd510 2023-10-06T05:20:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.510+0000 7f9f12d9d700 1 -- 172.21.15.31:0/2882056088 >> v1:172.21.15.62:6789/0 conn(0x7f9f140cd510 legacy=0x7f9f140aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.510+0000 7f9f12d9d700 1 -- 172.21.15.31:0/2882056088 >> v1:172.21.15.31:6789/0 conn(0x7f9f14159500 legacy=0x7f9f140cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.510+0000 7f9f12d9d700 1 -- 172.21.15.31:0/2882056088 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1415d7d0 con 0x7f9f1415d2d0 2023-10-06T05:20:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.510+0000 7f9f12d9d700 1 --2- 172.21.15.31:0/2882056088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 0x7f9f140b0ab0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009e60 tx=0x7f9f0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1836ac06fa0d4f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.510+0000 7f9f1159a700 1 -- 172.21.15.31:0/2882056088 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c00e070 con 0x7f9f1415d2d0 2023-10-06T05:20:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.510+0000 7f9f1159a700 1 -- 172.21.15.31:0/2882056088 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c00bb60 con 0x7f9f1415d2d0 2023-10-06T05:20:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.510+0000 7f9f1159a700 1 -- 172.21.15.31:0/2882056088 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c012430 con 0x7f9f1415d2d0 2023-10-06T05:20:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.511+0000 7f9f18fff700 1 -- 172.21.15.31:0/2882056088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 msgr2=0x7f9f140b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.511+0000 7f9f18fff700 1 --2- 172.21.15.31:0/2882056088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 0x7f9f140b0ab0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009e60 tx=0x7f9f0c009780 comp rx=0 tx=0).stop 2023-10-06T05:20:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.511+0000 7f9f18fff700 1 -- 172.21.15.31:0/2882056088 shutdown_connections 2023-10-06T05:20:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.511+0000 7f9f18fff700 1 --2- 172.21.15.31:0/2882056088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 0x7f9f140b0ab0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.511+0000 7f9f18fff700 1 -- 172.21.15.31:0/2882056088 >> 172.21.15.31:0/2882056088 conn(0x7f9f1406ec90 msgr2=0x7f9f1406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.511+0000 7f9f18fff700 1 -- 172.21.15.31:0/2882056088 shutdown_connections 2023-10-06T05:20:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.511+0000 7f9f18fff700 1 -- 172.21.15.31:0/2882056088 wait complete. 2023-10-06T05:20:23.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.512+0000 7f9f18fff700 1 Processor -- start 2023-10-06T05:20:23.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.512+0000 7f9f18fff700 1 -- start start 2023-10-06T05:20:23.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.513+0000 7f9f18fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f140cd510 0x7f9f140b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.513+0000 7f9f18fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f14159500 0x7f9f1415b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.513+0000 7f9f18fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 0x7f9f1415be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.513+0000 7f9f18fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f140c5410 con 0x7f9f1415d2d0 2023-10-06T05:20:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.513+0000 7f9f18fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f140c4800 con 0x7f9f14159500 2023-10-06T05:20:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.513+0000 7f9f18fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f140c4ae0 con 0x7f9f140cd510 2023-10-06T05:20:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f12d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 0x7f9f1415be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f12d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 0x7f9f1415be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55794/0 (socket says 172.21.15.31:55794) 2023-10-06T05:20:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f12d9d700 1 -- 172.21.15.31:0/4101604919 learned_addr learned my addr 172.21.15.31:0/4101604919 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f1259c700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f140cd510 0x7f9f140b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f11d9b700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f14159500 0x7f9f1415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f12d9d700 1 -- 172.21.15.31:0/4101604919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f140cd510 msgr2=0x7f9f140b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f12d9d700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f140cd510 0x7f9f140b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f12d9d700 1 -- 172.21.15.31:0/4101604919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f14159500 msgr2=0x7f9f1415b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f12d9d700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f14159500 0x7f9f1415b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f12d9d700 1 -- 172.21.15.31:0/4101604919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c009a20 con 0x7f9f1415d2d0 2023-10-06T05:20:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f11d9b700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f14159500 0x7f9f1415b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.514+0000 7f9f12d9d700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 0x7f9f1415be80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9f14067c50 tx=0x7f9f0c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.515+0000 7f9f027fc700 1 -- 172.21.15.31:0/4101604919 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c01c610 con 0x7f9f1415d2d0 2023-10-06T05:20:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.515+0000 7f9f027fc700 1 -- 172.21.15.31:0/4101604919 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c01c790 con 0x7f9f1415d2d0 2023-10-06T05:20:23.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.515+0000 7f9f027fc700 1 -- 172.21.15.31:0/4101604919 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c01cb50 con 0x7f9f1415d2d0 2023-10-06T05:20:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.515+0000 7f9f18fff700 1 -- 172.21.15.31:0/4101604919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f140c4dc0 con 0x7f9f1415d2d0 2023-10-06T05:20:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.515+0000 7f9f18fff700 1 -- 172.21.15.31:0/4101604919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f140b6090 con 0x7f9f1415d2d0 2023-10-06T05:20:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.516+0000 7f9f18fff700 1 -- 172.21.15.31:0/4101604919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f140c55c0 con 0x7f9f1415d2d0 2023-10-06T05:20:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.517+0000 7f9f027fc700 1 -- 172.21.15.31:0/4101604919 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9f0c01b400 con 0x7f9f1415d2d0 2023-10-06T05:20:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.517+0000 7f9f027fc700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ef40670e0 0x7f9ef4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.518+0000 7f9f027fc700 1 -- 172.21.15.31:0/4101604919 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f9f0c086bc0 con 0x7f9f1415d2d0 2023-10-06T05:20:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.518+0000 7f9f1259c700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ef40670e0 0x7f9ef4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.518+0000 7f9f1259c700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ef40670e0 0x7f9ef4069590 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9f040058d0 tx=0x7f9f04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:23.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.522+0000 7f9f027fc700 1 -- 172.21.15.31:0/4101604919 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f140c55c0 con 0x7f9f1415d2d0 2023-10-06T05:20:23.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.769+0000 7f9f18fff700 1 -- 172.21.15.31:0/4101604919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f140c57a0 con 0x7f9f1415d2d0 2023-10-06T05:20:23.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.769+0000 7f9f027fc700 1 -- 172.21.15.31:0/4101604919 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f9f0c02b090 con 0x7f9f1415d2d0 2023-10-06T05:20:23.771 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:23.771 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:18:38.946090+0000","modified":"2023-10-06T05:18:42.479138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5355},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5355":{"gid":5355,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/1839995141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1839995141},{"type":"v1","addr":"172.21.15.31:6837","nonce":1839995141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T05:20:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 -- 172.21.15.31:0/4101604919 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ef40670e0 msgr2=0x7f9ef4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ef40670e0 0x7f9ef4069590 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9f040058d0 tx=0x7f9f04009040 comp rx=0 tx=0).stop 2023-10-06T05:20:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 -- 172.21.15.31:0/4101604919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 msgr2=0x7f9f1415be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 0x7f9f1415be80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9f14067c50 tx=0x7f9f0c019000 comp rx=0 tx=0).stop 2023-10-06T05:20:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 -- 172.21.15.31:0/4101604919 shutdown_connections 2023-10-06T05:20:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ef40670e0 0x7f9ef4069590 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1415d2d0 0x7f9f1415be80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f140cd510 0x7f9f140b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 --2- 172.21.15.31:0/4101604919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f14159500 0x7f9f1415b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 -- 172.21.15.31:0/4101604919 >> 172.21.15.31:0/4101604919 conn(0x7f9f1406ec90 msgr2=0x7f9f141582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:23.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 -- 172.21.15.31:0/4101604919 shutdown_connections 2023-10-06T05:20:23.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.772+0000 7f9f18fff700 1 -- 172.21.15.31:0/4101604919 wait complete. 2023-10-06T05:20:23.775 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2023-10-06T05:20:23.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.984+0000 7f1f129ff700 1 Processor -- start 2023-10-06T05:20:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.984+0000 7f1f129ff700 1 -- start start 2023-10-06T05:20:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.985+0000 7f1f129ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f0c13a1e0 0x7f1f0c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.985+0000 7f1f129ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1f0c13a5b0 0x7f1f0c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.985+0000 7f1f129ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c0c71b0 con 0x7f1f0c0c6c30 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f129ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c0c7490 con 0x7f1f0c13a1e0 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f129ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c0c7770 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f0bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f0c13a1e0 0x7f1f0c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f0bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f0c13a1e0 0x7f1f0c0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55800/0 (socket says 172.21.15.31:55800) 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f0bfff700 1 -- 172.21.15.31:0/3649629078 learned_addr learned my addr 172.21.15.31:0/3649629078 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f0b7fe700 1 --2- 172.21.15.31:0/3649629078 >> v2:172.21.15.62:3300/0 conn(0x7f1f0c13a5b0 0x7f1f0c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f0bfff700 1 -- 172.21.15.31:0/3649629078 >> v2:172.21.15.62:3300/0 conn(0x7f1f0c13a5b0 msgr2=0x7f1f0c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f0bfff700 1 --2- 172.21.15.31:0/3649629078 >> v2:172.21.15.62:3300/0 conn(0x7f1f0c13a5b0 0x7f1f0c06d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f0bfff700 1 -- 172.21.15.31:0/3649629078 >> v1:172.21.15.62:6789/0 conn(0x7f1f0c0c6c30 legacy=0x7f1f0c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:23.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f0bfff700 1 -- 172.21.15.31:0/3649629078 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0c7a50 con 0x7f1f0c13a1e0 2023-10-06T05:20:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f0b7fe700 1 --2- 172.21.15.31:0/3649629078 >> v2:172.21.15.62:3300/0 conn(0x7f1f0c13a5b0 0x7f1f0c06d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.986+0000 7f1f0bfff700 1 --2- 172.21.15.31:0/3649629078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a1e0 0x7f1f0c0c66f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c06d130 tx=0x7f1efc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5e25bf0c8a7291 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.987+0000 7f1f0affd700 1 -- 172.21.15.31:0/3649629078 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc005c10 con 0x7f1f0c13a1e0 2023-10-06T05:20:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.987+0000 7f1f0affd700 1 -- 172.21.15.31:0/3649629078 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc005d70 con 0x7f1f0c13a1e0 2023-10-06T05:20:23.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.987+0000 7f1f0affd700 1 -- 172.21.15.31:0/3649629078 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00b840 con 0x7f1f0c13a1e0 2023-10-06T05:20:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.987+0000 7f1f129ff700 1 -- 172.21.15.31:0/3649629078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a1e0 msgr2=0x7f1f0c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.987+0000 7f1f129ff700 1 --2- 172.21.15.31:0/3649629078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a1e0 0x7f1f0c0c66f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c06d130 tx=0x7f1efc009220 comp rx=0 tx=0).stop 2023-10-06T05:20:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.987+0000 7f1f129ff700 1 -- 172.21.15.31:0/3649629078 shutdown_connections 2023-10-06T05:20:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.987+0000 7f1f129ff700 1 --2- 172.21.15.31:0/3649629078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a1e0 0x7f1f0c0c66f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.987+0000 7f1f129ff700 1 --2- 172.21.15.31:0/3649629078 >> v2:172.21.15.62:3300/0 conn(0x7f1f0c13a5b0 0x7f1f0c06d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.987+0000 7f1f129ff700 1 -- 172.21.15.31:0/3649629078 >> 172.21.15.31:0/3649629078 conn(0x7f1f0c0cc0b0 msgr2=0x7f1f0c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.988+0000 7f1f129ff700 1 -- 172.21.15.31:0/3649629078 shutdown_connections 2023-10-06T05:20:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.988+0000 7f1f129ff700 1 -- 172.21.15.31:0/3649629078 wait complete. 2023-10-06T05:20:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.989+0000 7f1f129ff700 1 Processor -- start 2023-10-06T05:20:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.989+0000 7f1f129ff700 1 -- start start 2023-10-06T05:20:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.990+0000 7f1f129ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f0c0c6c30 0x7f1f0c0ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.990+0000 7f1f129ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f0c13a1e0 0x7f1f0c1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.990+0000 7f1f129ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a5b0 0x7f1f0c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.990+0000 7f1f129ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0cb6a0 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.990+0000 7f1f129ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c16e950 con 0x7f1f0c13a1e0 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.990+0000 7f1f129ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c16ec30 con 0x7f1f0c0c6c30 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.990+0000 7f1f10f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a5b0 0x7f1f0c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.990+0000 7f1f10f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a5b0 0x7f1f0c153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55806/0 (socket says 172.21.15.31:55806) 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.990+0000 7f1f10f9c700 1 -- 172.21.15.31:0/2282546431 learned_addr learned my addr 172.21.15.31:0/2282546431 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1f0b7fe700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f0c13a1e0 0x7f1f0c1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1f0bfff700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f0c0c6c30 0x7f1f0c0ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1f10f9c700 1 -- 172.21.15.31:0/2282546431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f0c0c6c30 msgr2=0x7f1f0c0ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1f10f9c700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f0c0c6c30 0x7f1f0c0ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1f10f9c700 1 -- 172.21.15.31:0/2282546431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f0c13a1e0 msgr2=0x7f1f0c1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1f10f9c700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f0c13a1e0 0x7f1f0c1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1f10f9c700 1 -- 172.21.15.31:0/2282546431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc009480 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1f0b7fe700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f0c13a1e0 0x7f1f0c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1f10f9c700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a5b0 0x7f1f0c153df0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400c930 tx=0x7f1f040026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1ef3fff700 1 -- 172.21.15.31:0/2282546431 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0400ce70 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1ef3fff700 1 -- 172.21.15.31:0/2282546431 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0400a7e0 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1ef3fff700 1 -- 172.21.15.31:0/2282546431 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0400aba0 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.991+0000 7f1f129ff700 1 -- 172.21.15.31:0/2282546431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c16ef10 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.992+0000 7f1f129ff700 1 -- 172.21.15.31:0/2282546431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c13bbe0 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.993+0000 7f1f129ff700 1 -- 172.21.15.31:0/2282546431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c16ea90 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.993+0000 7f1ef3fff700 1 -- 172.21.15.31:0/2282546431 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f1f04012420 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.994+0000 7f1ef3fff700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ef80670e0 0x7f1ef8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:23.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.994+0000 7f1ef3fff700 1 -- 172.21.15.31:0/2282546431 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f1f04084ec0 con 0x7f1f0c13a5b0 2023-10-06T05:20:23.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.995+0000 7f1f0bfff700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ef80670e0 0x7f1ef8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:23.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.995+0000 7f1f0bfff700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ef80670e0 0x7f1ef8069590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0030c0 tx=0x7f1efc015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:23.998+0000 7f1ef3fff700 1 -- 172.21.15.31:0/2282546431 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f0c16ea90 con 0x7f1f0c13a5b0 2023-10-06T05:20:24.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.201+0000 7f1f129ff700 1 -- 172.21.15.31:0/2282546431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f0c16ea90 con 0x7f1f0c13a5b0 2023-10-06T05:20:24.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.202+0000 7f1ef3fff700 1 -- 172.21.15.31:0/2282546431 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+17009 (secure 0 0 0) 0x7f1f0c16ea90 con 0x7f1f0c13a5b0 2023-10-06T05:20:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:24.204 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:18:50.146155+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:18:36.639713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T05:18:37.929704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:24.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 -- 172.21.15.31:0/2282546431 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ef80670e0 msgr2=0x7f1ef8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ef80670e0 0x7f1ef8069590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0030c0 tx=0x7f1efc015000 comp rx=0 tx=0).stop 2023-10-06T05:20:24.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 -- 172.21.15.31:0/2282546431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a5b0 msgr2=0x7f1f0c153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a5b0 0x7f1f0c153df0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400c930 tx=0x7f1f040026e0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 -- 172.21.15.31:0/2282546431 shutdown_connections 2023-10-06T05:20:24.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1ef80670e0 0x7f1ef8069590 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f0c13a5b0 0x7f1f0c153df0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f0c0c6c30 0x7f1f0c0ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 --2- 172.21.15.31:0/2282546431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f0c13a1e0 0x7f1f0c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 -- 172.21.15.31:0/2282546431 >> 172.21.15.31:0/2282546431 conn(0x7f1f0c0cc0b0 msgr2=0x7f1f0c156ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:24.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.204+0000 7f1f129ff700 1 -- 172.21.15.31:0/2282546431 shutdown_connections 2023-10-06T05:20:24.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.205+0000 7f1f129ff700 1 -- 172.21.15.31:0/2282546431 wait complete. 2023-10-06T05:20:24.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:20:24.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.419+0000 7ff2cbfff700 1 Processor -- start 2023-10-06T05:20:24.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.420+0000 7ff2cbfff700 1 -- start start 2023-10-06T05:20:24.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.421+0000 7ff2cbfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff2c413a1e0 0x7ff2c40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.421+0000 7ff2cbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2c406db50 0x7ff2c4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.421+0000 7ff2cbfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c40c71b0 con 0x7ff2c413a5b0 2023-10-06T05:20:24.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.421+0000 7ff2cbfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c40c7490 con 0x7ff2c406db50 2023-10-06T05:20:24.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.421+0000 7ff2cbfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c40c7770 con 0x7ff2c413a1e0 2023-10-06T05:20:24.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.421+0000 7ff2ca59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2c406db50 0x7ff2c4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.421+0000 7ff2ca59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2c406db50 0x7ff2c4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55816/0 (socket says 172.21.15.31:55816) 2023-10-06T05:20:24.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.421+0000 7ff2ca59c700 1 -- 172.21.15.31:0/2637909432 learned_addr learned my addr 172.21.15.31:0/2637909432 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:24.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.422+0000 7ff2c9d9b700 1 --2- 172.21.15.31:0/2637909432 >> v2:172.21.15.62:3300/0 conn(0x7ff2c413a1e0 0x7ff2c40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.422+0000 7ff2ca59c700 1 -- 172.21.15.31:0/2637909432 >> v2:172.21.15.62:3300/0 conn(0x7ff2c413a1e0 msgr2=0x7ff2c40c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:24.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.422+0000 7ff2ca59c700 1 --2- 172.21.15.31:0/2637909432 >> v2:172.21.15.62:3300/0 conn(0x7ff2c413a1e0 0x7ff2c40c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.422+0000 7ff2ca59c700 1 -- 172.21.15.31:0/2637909432 >> v1:172.21.15.62:6789/0 conn(0x7ff2c413a5b0 legacy=0x7ff2c406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.422+0000 7ff2ca59c700 1 -- 172.21.15.31:0/2637909432 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c40c7a50 con 0x7ff2c406db50 2023-10-06T05:20:24.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.422+0000 7ff2ca59c700 1 --2- 172.21.15.31:0/2637909432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c406db50 0x7ff2c4069560 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40cbf90 tx=0x7ff2bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964fd756ce4829e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:24.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.422+0000 7ff2c8d99700 1 -- 172.21.15.31:0/2637909432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc00e070 con 0x7ff2c406db50 2023-10-06T05:20:24.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.422+0000 7ff2c8d99700 1 -- 172.21.15.31:0/2637909432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc00ba50 con 0x7ff2c406db50 2023-10-06T05:20:24.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.422+0000 7ff2c8d99700 1 -- 172.21.15.31:0/2637909432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc0092e0 con 0x7ff2c406db50 2023-10-06T05:20:24.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.423+0000 7ff2cbfff700 1 -- 172.21.15.31:0/2637909432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c406db50 msgr2=0x7ff2c4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.423+0000 7ff2cbfff700 1 --2- 172.21.15.31:0/2637909432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c406db50 0x7ff2c4069560 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40cbf90 tx=0x7ff2bc009840 comp rx=0 tx=0).stop 2023-10-06T05:20:24.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.423+0000 7ff2cbfff700 1 -- 172.21.15.31:0/2637909432 shutdown_connections 2023-10-06T05:20:24.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.423+0000 7ff2cbfff700 1 --2- 172.21.15.31:0/2637909432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c406db50 0x7ff2c4069560 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.423+0000 7ff2cbfff700 1 --2- 172.21.15.31:0/2637909432 >> v2:172.21.15.62:3300/0 conn(0x7ff2c413a1e0 0x7ff2c40c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.423+0000 7ff2cbfff700 1 -- 172.21.15.31:0/2637909432 >> 172.21.15.31:0/2637909432 conn(0x7ff2c40cc0b0 msgr2=0x7ff2c40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:24.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.423+0000 7ff2cbfff700 1 -- 172.21.15.31:0/2637909432 shutdown_connections 2023-10-06T05:20:24.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.423+0000 7ff2cbfff700 1 -- 172.21.15.31:0/2637909432 wait complete. 2023-10-06T05:20:24.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.424+0000 7ff2cbfff700 1 Processor -- start 2023-10-06T05:20:24.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.425+0000 7ff2cbfff700 1 -- start start 2023-10-06T05:20:24.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.425+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2c406db50 0x7ff2c413d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c413a1e0 0x7ff2c4139a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c413a5b0 0x7ff2c41505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2cbfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c413d750 con 0x7ff2c413a5b0 2023-10-06T05:20:24.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2cbfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c413c470 con 0x7ff2c413a1e0 2023-10-06T05:20:24.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2cbfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c41513a0 con 0x7ff2c406db50 2023-10-06T05:20:24.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2ca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c413a5b0 0x7ff2c41505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2ca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c413a5b0 0x7ff2c41505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55826/0 (socket says 172.21.15.31:55826) 2023-10-06T05:20:24.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2ca59c700 1 -- 172.21.15.31:0/3766783301 learned_addr learned my addr 172.21.15.31:0/3766783301 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:24.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2c959a700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c413a1e0 0x7ff2c4139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2c9d9b700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2c406db50 0x7ff2c413d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2ca59c700 1 -- 172.21.15.31:0/3766783301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2c406db50 msgr2=0x7ff2c413d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2ca59c700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2c406db50 0x7ff2c413d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2ca59c700 1 -- 172.21.15.31:0/3766783301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c413a1e0 msgr2=0x7ff2c4139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2ca59c700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c413a1e0 0x7ff2c4139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.426+0000 7ff2ca59c700 1 -- 172.21.15.31:0/3766783301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc009ae0 con 0x7ff2c413a5b0 2023-10-06T05:20:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.427+0000 7ff2ca59c700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c413a5b0 0x7ff2c41505c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40cc010 tx=0x7ff2bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.427+0000 7ff2b9ffb700 1 -- 172.21.15.31:0/3766783301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc00be60 con 0x7ff2c413a5b0 2023-10-06T05:20:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.427+0000 7ff2b9ffb700 1 -- 172.21.15.31:0/3766783301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc008d20 con 0x7ff2c413a5b0 2023-10-06T05:20:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.427+0000 7ff2b9ffb700 1 -- 172.21.15.31:0/3766783301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc0266b0 con 0x7ff2c413a5b0 2023-10-06T05:20:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.427+0000 7ff2cbfff700 1 -- 172.21.15.31:0/3766783301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c413b860 con 0x7ff2c413a5b0 2023-10-06T05:20:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.427+0000 7ff2cbfff700 1 -- 172.21.15.31:0/3766783301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c413be20 con 0x7ff2c413a5b0 2023-10-06T05:20:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.428+0000 7ff2cbfff700 1 -- 172.21.15.31:0/3766783301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c404c380 con 0x7ff2c413a5b0 2023-10-06T05:20:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.429+0000 7ff2b9ffb700 1 -- 172.21.15.31:0/3766783301 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7ff2bc003040 con 0x7ff2c413a5b0 2023-10-06T05:20:24.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.430+0000 7ff2b9ffb700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff2a80670e0 0x7ff2a8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.430+0000 7ff2b9ffb700 1 -- 172.21.15.31:0/3766783301 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6116+0+0 (secure 0 0 0) 0x7ff2bc086a50 con 0x7ff2c413a5b0 2023-10-06T05:20:24.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.430+0000 7ff2c9d9b700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff2a80670e0 0x7ff2a8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.431+0000 7ff2c9d9b700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff2a80670e0 0x7ff2a8069590 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0006aa0 tx=0x7ff2c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:24.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.434+0000 7ff2b9ffb700 1 -- 172.21.15.31:0/3766783301 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2bc0536f0 con 0x7ff2c413a5b0 2023-10-06T05:20:24.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.684+0000 7ff2cbfff700 1 -- 172.21.15.31:0/3766783301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff2c40c7d30 con 0x7ff2c413a5b0 2023-10-06T05:20:24.687 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:20:24.685+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:20:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.694+0000 7ff2b9ffb700 1 -- 172.21.15.31:0/3766783301 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff2bc009940 con 0x7ff2c413a5b0 2023-10-06T05:20:24.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.696+0000 7ff2cbfff700 1 -- 172.21.15.31:0/3766783301 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff2a80670e0 msgr2=0x7ff2a8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.696+0000 7ff2cbfff700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff2a80670e0 0x7ff2a8069590 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0006aa0 tx=0x7ff2c0008040 comp rx=0 tx=0).stop 2023-10-06T05:20:24.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.696+0000 7ff2cbfff700 1 -- 172.21.15.31:0/3766783301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c413a5b0 msgr2=0x7ff2c41505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.697+0000 7ff2cbfff700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c413a5b0 0x7ff2c41505c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40cc010 tx=0x7ff2bc018000 comp rx=0 tx=0).stop 2023-10-06T05:20:24.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.697+0000 7ff2cbfff700 1 -- 172.21.15.31:0/3766783301 shutdown_connections 2023-10-06T05:20:24.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.697+0000 7ff2cbfff700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff2a80670e0 0x7ff2a8069590 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.697+0000 7ff2cbfff700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2c413a5b0 0x7ff2c41505c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.697+0000 7ff2cbfff700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2c406db50 0x7ff2c413d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.697+0000 7ff2cbfff700 1 --2- 172.21.15.31:0/3766783301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c413a1e0 0x7ff2c4139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.697+0000 7ff2cbfff700 1 -- 172.21.15.31:0/3766783301 >> 172.21.15.31:0/3766783301 conn(0x7ff2c40cc0b0 msgr2=0x7ff2c40c6720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:24.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.697+0000 7ff2cbfff700 1 -- 172.21.15.31:0/3766783301 shutdown_connections 2023-10-06T05:20:24.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.697+0000 7ff2cbfff700 1 -- 172.21.15.31:0/3766783301 wait complete. 2023-10-06T05:20:24.702 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:20:24.705 INFO:tasks.ceph.mds.c.smithi031.stderr: -8639> 2023-10-06T05:18:43.426+0000 7ff2e61f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:20:24.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:20:24.754 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:20:24.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.916+0000 7f7d37dff700 1 Processor -- start 2023-10-06T05:20:24.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.916+0000 7f7d37dff700 1 -- start start 2023-10-06T05:20:24.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.917+0000 7f7d37dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d30159560 0x7f7d300cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.917+0000 7f7d37dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d300b2b80 0x7f7d300cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.917+0000 7f7d37dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3015d2d0 con 0x7f7d30159930 2023-10-06T05:20:24.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.917+0000 7f7d37dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3015d5b0 con 0x7f7d30159560 2023-10-06T05:20:24.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.917+0000 7f7d37dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3015d890 con 0x7f7d300b2b80 2023-10-06T05:20:24.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.918+0000 7f7d3539a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7d30159930 0x7f7d300b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46584/0 (socket says 172.21.15.31:46584) 2023-10-06T05:20:24.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.918+0000 7f7d3539a700 1 -- 172.21.15.31:0/1051572047 learned_addr learned my addr 172.21.15.31:0/1051572047 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:24.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.918+0000 7f7d35b9b700 1 --2- 172.21.15.31:0/1051572047 >> v2:172.21.15.62:3300/0 conn(0x7f7d30159560 0x7f7d300cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.918+0000 7f7d3639c700 1 --2- 172.21.15.31:0/1051572047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d300b2b80 0x7f7d300cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.918+0000 7f7d34b99700 1 -- 172.21.15.31:0/1051572047 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433685353 0 0) 0x7f7d3015d2d0 con 0x7f7d30159930 2023-10-06T05:20:24.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.918+0000 7f7d34b99700 1 -- 172.21.15.31:0/1051572047 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d1c003570 con 0x7f7d30159930 2023-10-06T05:20:24.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.918+0000 7f7d35b9b700 1 -- 172.21.15.31:0/1051572047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d300b2b80 msgr2=0x7f7d300cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:24.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.918+0000 7f7d35b9b700 1 --2- 172.21.15.31:0/1051572047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d300b2b80 0x7f7d300cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.918+0000 7f7d35b9b700 1 -- 172.21.15.31:0/1051572047 >> v1:172.21.15.31:6789/0 conn(0x7f7d30159930 legacy=0x7f7d300b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.918+0000 7f7d35b9b700 1 -- 172.21.15.31:0/1051572047 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3015db70 con 0x7f7d30159560 2023-10-06T05:20:24.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.919+0000 7f7d35b9b700 1 --2- 172.21.15.31:0/1051572047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d30159560 0x7f7d300cce70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0098e0 tx=0x7f7d2c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95bc5f61ad451d77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:24.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.919+0000 7f7d34b99700 1 -- 172.21.15.31:0/1051572047 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c011630 con 0x7f7d30159560 2023-10-06T05:20:24.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.919+0000 7f7d34b99700 1 -- 172.21.15.31:0/1051572047 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c011790 con 0x7f7d30159560 2023-10-06T05:20:24.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.919+0000 7f7d34b99700 1 -- 172.21.15.31:0/1051572047 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c011b10 con 0x7f7d30159560 2023-10-06T05:20:24.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.919+0000 7f7d37dff700 1 -- 172.21.15.31:0/1051572047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d30159560 msgr2=0x7f7d300cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.919+0000 7f7d37dff700 1 --2- 172.21.15.31:0/1051572047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d30159560 0x7f7d300cce70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0098e0 tx=0x7f7d2c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.920+0000 7f7d37dff700 1 -- 172.21.15.31:0/1051572047 shutdown_connections 2023-10-06T05:20:24.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.920+0000 7f7d37dff700 1 --2- 172.21.15.31:0/1051572047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d30159560 0x7f7d300cce70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.920+0000 7f7d37dff700 1 --2- 172.21.15.31:0/1051572047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d300b2b80 0x7f7d300cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.920+0000 7f7d37dff700 1 -- 172.21.15.31:0/1051572047 >> 172.21.15.31:0/1051572047 conn(0x7f7d3006ec90 msgr2=0x7f7d3006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:24.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.920+0000 7f7d37dff700 1 -- 172.21.15.31:0/1051572047 shutdown_connections 2023-10-06T05:20:24.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.920+0000 7f7d37dff700 1 -- 172.21.15.31:0/1051572047 wait complete. 2023-10-06T05:20:24.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.921+0000 7f7d37dff700 1 Processor -- start 2023-10-06T05:20:24.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.921+0000 7f7d37dff700 1 -- start start 2023-10-06T05:20:24.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.922+0000 7f7d37dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d300b2b80 0x7f7d300cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.922+0000 7f7d37dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d30159560 0x7f7d3016a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.922+0000 7f7d37dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d30159930 0x7f7d300c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.922+0000 7f7d37dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300ca400 con 0x7f7d30159560 2023-10-06T05:20:24.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.922+0000 7f7d37dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3016ae30 con 0x7f7d30159930 2023-10-06T05:20:24.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.922+0000 7f7d37dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300cf1f0 con 0x7f7d300b2b80 2023-10-06T05:20:24.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d3539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d30159560 0x7f7d3016a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d3539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d30159560 0x7f7d3016a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55854/0 (socket says 172.21.15.31:55854) 2023-10-06T05:20:24.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d3539a700 1 -- 172.21.15.31:0/3878894743 learned_addr learned my addr 172.21.15.31:0/3878894743 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:24.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d3639c700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d30159930 0x7f7d300c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d35b9b700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d300b2b80 0x7f7d300cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d3539a700 1 -- 172.21.15.31:0/3878894743 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d300b2b80 msgr2=0x7f7d300cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d3539a700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d300b2b80 0x7f7d300cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d3539a700 1 -- 172.21.15.31:0/3878894743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d30159930 msgr2=0x7f7d300c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:24.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d3539a700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d30159930 0x7f7d300c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:24.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d3539a700 1 -- 172.21.15.31:0/3878894743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c009420 con 0x7f7d30159560 2023-10-06T05:20:24.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d3539a700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d30159560 0x7f7d3016a8f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7d200011e0 tx=0x7f7d20007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:24.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.923+0000 7f7d25ffb700 1 -- 172.21.15.31:0/3878894743 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20005b30 con 0x7f7d30159560 2023-10-06T05:20:24.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.924+0000 7f7d25ffb700 1 -- 172.21.15.31:0/3878894743 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20005c90 con 0x7f7d30159560 2023-10-06T05:20:24.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.924+0000 7f7d25ffb700 1 -- 172.21.15.31:0/3878894743 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20007320 con 0x7f7d30159560 2023-10-06T05:20:24.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.924+0000 7f7d37dff700 1 -- 172.21.15.31:0/3878894743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d300cf4d0 con 0x7f7d30159560 2023-10-06T05:20:24.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.924+0000 7f7d37dff700 1 -- 172.21.15.31:0/3878894743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d300cfa90 con 0x7f7d30159560 2023-10-06T05:20:24.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.925+0000 7f7d37dff700 1 -- 172.21.15.31:0/3878894743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d300cfcb0 con 0x7f7d30159560 2023-10-06T05:20:24.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.926+0000 7f7d25ffb700 1 -- 172.21.15.31:0/3878894743 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7d20007690 con 0x7f7d30159560 2023-10-06T05:20:24.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.926+0000 7f7d25ffb700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7d140670e0 0x7f7d14069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:24.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.927+0000 7f7d25ffb700 1 -- 172.21.15.31:0/3878894743 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f7d20083f20 con 0x7f7d30159560 2023-10-06T05:20:24.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.927+0000 7f7d35b9b700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7d140670e0 0x7f7d14069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:24.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.928+0000 7f7d35b9b700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7d140670e0 0x7f7d14069590 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0091e0 tx=0x7f7d2c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:24.931+0000 7f7d25ffb700 1 -- 172.21.15.31:0/3878894743 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d20050c80 con 0x7f7d30159560 2023-10-06T05:20:25.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.177+0000 7f7d37dff700 1 -- 172.21.15.31:0/3878894743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7d3004c380 con 0x7f7d30159560 2023-10-06T05:20:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.694+0000 7f7d25ffb700 1 -- 172.21.15.31:0/3878894743 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7d200170a0 con 0x7f7d30159560 2023-10-06T05:20:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 -- 172.21.15.31:0/3878894743 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7d140670e0 msgr2=0x7f7d14069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7d140670e0 0x7f7d14069590 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0091e0 tx=0x7f7d2c017000 comp rx=0 tx=0).stop 2023-10-06T05:20:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 -- 172.21.15.31:0/3878894743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d30159560 msgr2=0x7f7d3016a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d30159560 0x7f7d3016a8f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7d200011e0 tx=0x7f7d20007df0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 -- 172.21.15.31:0/3878894743 shutdown_connections 2023-10-06T05:20:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d30159930 0x7f7d300c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d300b2b80 0x7f7d300cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7d140670e0 0x7f7d14069590 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 --2- 172.21.15.31:0/3878894743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d30159560 0x7f7d3016a8f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 -- 172.21.15.31:0/3878894743 >> 172.21.15.31:0/3878894743 conn(0x7f7d3006ec90 msgr2=0x7f7d300affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.697+0000 7f7d37dff700 1 -- 172.21.15.31:0/3878894743 shutdown_connections 2023-10-06T05:20:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.698+0000 7f7d37dff700 1 -- 172.21.15.31:0/3878894743 wait complete. 2023-10-06T05:20:25.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:20:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.964+0000 7f019fdff700 1 Processor -- start 2023-10-06T05:20:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.964+0000 7f019fdff700 1 -- start start 2023-10-06T05:20:25.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.965+0000 7f019fdff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0198156300 0x7f01980ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.965+0000 7f019fdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f019806a630 0x7f019806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.965+0000 7f019fdff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019806e600 con 0x7f019806ea90 2023-10-06T05:20:25.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.965+0000 7f019fdff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019806b010 con 0x7f0198156300 2023-10-06T05:20:25.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.965+0000 7f019fdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019806ef00 con 0x7f019806a630 2023-10-06T05:20:25.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019d39a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f019806ea90 0x7f019806d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46596/0 (socket says 172.21.15.31:46596) 2023-10-06T05:20:25.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019d39a700 1 -- 172.21.15.31:0/1257930394 learned_addr learned my addr 172.21.15.31:0/1257930394 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:25.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019db9b700 1 --2- 172.21.15.31:0/1257930394 >> v2:172.21.15.62:3300/0 conn(0x7f0198156300 0x7f01980ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019e39c700 1 --2- 172.21.15.31:0/1257930394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f019806a630 0x7f019806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019cb99700 1 -- 172.21.15.31:0/1257930394 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468259401 0 0) 0x7f019806e600 con 0x7f019806ea90 2023-10-06T05:20:25.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019cb99700 1 -- 172.21.15.31:0/1257930394 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0184003570 con 0x7f019806ea90 2023-10-06T05:20:25.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019db9b700 1 -- 172.21.15.31:0/1257930394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f019806a630 msgr2=0x7f019806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:25.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019db9b700 1 --2- 172.21.15.31:0/1257930394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f019806a630 0x7f019806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019db9b700 1 -- 172.21.15.31:0/1257930394 >> v1:172.21.15.31:6789/0 conn(0x7f019806ea90 legacy=0x7f019806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019db9b700 1 -- 172.21.15.31:0/1257930394 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019806d9f0 con 0x7f0198156300 2023-10-06T05:20:25.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.966+0000 7f019e39c700 1 --2- 172.21.15.31:0/1257930394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f019806a630 0x7f019806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:25.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.967+0000 7f019db9b700 1 --2- 172.21.15.31:0/1257930394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0198156300 0x7f01980ad670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f01940098d0 tx=0x7f0194009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4af7d3abfcf58737 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:25.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.967+0000 7f019cb99700 1 -- 172.21.15.31:0/1257930394 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194005c10 con 0x7f0198156300 2023-10-06T05:20:25.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.967+0000 7f019cb99700 1 -- 172.21.15.31:0/1257930394 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0194005d70 con 0x7f0198156300 2023-10-06T05:20:25.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.967+0000 7f019cb99700 1 -- 172.21.15.31:0/1257930394 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194006260 con 0x7f0198156300 2023-10-06T05:20:25.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.968+0000 7f019fdff700 1 -- 172.21.15.31:0/1257930394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0198156300 msgr2=0x7f01980ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.968+0000 7f019fdff700 1 --2- 172.21.15.31:0/1257930394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0198156300 0x7f01980ad670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f01940098d0 tx=0x7f0194009120 comp rx=0 tx=0).stop 2023-10-06T05:20:25.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.968+0000 7f019fdff700 1 -- 172.21.15.31:0/1257930394 shutdown_connections 2023-10-06T05:20:25.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.968+0000 7f019fdff700 1 --2- 172.21.15.31:0/1257930394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0198156300 0x7f01980ad670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.968+0000 7f019fdff700 1 --2- 172.21.15.31:0/1257930394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f019806a630 0x7f019806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.968+0000 7f019fdff700 1 -- 172.21.15.31:0/1257930394 >> 172.21.15.31:0/1257930394 conn(0x7f0198068030 msgr2=0x7f01980cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:25.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.968+0000 7f019fdff700 1 -- 172.21.15.31:0/1257930394 shutdown_connections 2023-10-06T05:20:25.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.968+0000 7f019fdff700 1 -- 172.21.15.31:0/1257930394 wait complete. 2023-10-06T05:20:25.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.969+0000 7f019fdff700 1 Processor -- start 2023-10-06T05:20:25.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.969+0000 7f019fdff700 1 -- start start 2023-10-06T05:20:25.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.970+0000 7f019fdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019806a630 0x7f01980c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.970+0000 7f019fdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f019806ea90 0x7f019815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.970+0000 7f019fdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0198156300 0x7f0198154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.970+0000 7f019fdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01980c3c30 con 0x7f019806a630 2023-10-06T05:20:25.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.970+0000 7f019fdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0198153ca0 con 0x7f019806ea90 2023-10-06T05:20:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.970+0000 7f019fdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01980c5420 con 0x7f0198156300 2023-10-06T05:20:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019db9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019806a630 0x7f01980c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019db9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019806a630 0x7f01980c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55864/0 (socket says 172.21.15.31:55864) 2023-10-06T05:20:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019db9b700 1 -- 172.21.15.31:0/3308218313 learned_addr learned my addr 172.21.15.31:0/3308218313 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019d39a700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f019806ea90 0x7f019815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019e39c700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0198156300 0x7f0198154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019db9b700 1 -- 172.21.15.31:0/3308218313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0198156300 msgr2=0x7f0198154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019db9b700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0198156300 0x7f0198154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019db9b700 1 -- 172.21.15.31:0/3308218313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f019806ea90 msgr2=0x7f019815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019db9b700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f019806ea90 0x7f019815a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019db9b700 1 -- 172.21.15.31:0/3308218313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0190009a20 con 0x7f019806a630 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019e39c700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0198156300 0x7f0198154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.971+0000 7f019db9b700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019806a630 0x7f01980c24c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0194009ec0 tx=0x7f0194016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.972+0000 7f018dffb700 1 -- 172.21.15.31:0/3308218313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019401d910 con 0x7f019806a630 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.972+0000 7f018dffb700 1 -- 172.21.15.31:0/3308218313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019401da90 con 0x7f019806a630 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.972+0000 7f019fdff700 1 -- 172.21.15.31:0/3308218313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01940093c0 con 0x7f019806a630 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.972+0000 7f018dffb700 1 -- 172.21.15.31:0/3308218313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019401de50 con 0x7f019806a630 2023-10-06T05:20:25.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.972+0000 7f019fdff700 1 -- 172.21.15.31:0/3308218313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01980c5cc0 con 0x7f019806a630 2023-10-06T05:20:25.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.973+0000 7f019fdff700 1 -- 172.21.15.31:0/3308218313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01980c5840 con 0x7f019806a630 2023-10-06T05:20:25.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.974+0000 7f018dffb700 1 -- 172.21.15.31:0/3308218313 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f019401b690 con 0x7f019806a630 2023-10-06T05:20:25.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.974+0000 7f018dffb700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f017c0670e0 0x7f017c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:25.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.975+0000 7f018dffb700 1 -- 172.21.15.31:0/3308218313 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f01940866e0 con 0x7f019806a630 2023-10-06T05:20:25.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.975+0000 7f019d39a700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f017c0670e0 0x7f017c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:25.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.975+0000 7f019d39a700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f017c0670e0 0x7f017c069590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0188007fd0 tx=0x7f0188009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:25.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:25.979+0000 7f018dffb700 1 -- 172.21.15.31:0/3308218313 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01980c5840 con 0x7f019806a630 2023-10-06T05:20:26.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:26.186+0000 7f019fdff700 1 -- 172.21.15.31:0/3308218313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01980c5840 con 0x7f019806a630 2023-10-06T05:20:27.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.063+0000 7f018dffb700 1 -- 172.21.15.31:0/3308218313 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) v1 ==== 181+0+0 (secure 0 0 0) 0x7f01980c5840 con 0x7f019806a630 2023-10-06T05:20:27.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 -- 172.21.15.31:0/3308218313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f017c0670e0 msgr2=0x7f017c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:27.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f017c0670e0 0x7f017c069590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0188007fd0 tx=0x7f0188009040 comp rx=0 tx=0).stop 2023-10-06T05:20:27.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 -- 172.21.15.31:0/3308218313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019806a630 msgr2=0x7f01980c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:27.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019806a630 0x7f01980c24c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0194009ec0 tx=0x7f0194016000 comp rx=0 tx=0).stop 2023-10-06T05:20:27.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 -- 172.21.15.31:0/3308218313 shutdown_connections 2023-10-06T05:20:27.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0198156300 0x7f0198154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f017c0670e0 0x7f017c069590 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019806a630 0x7f01980c24c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 --2- 172.21.15.31:0/3308218313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f019806ea90 0x7f019815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 -- 172.21.15.31:0/3308218313 >> 172.21.15.31:0/3308218313 conn(0x7f0198068030 msgr2=0x7f0198155620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:27.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 -- 172.21.15.31:0/3308218313 shutdown_connections 2023-10-06T05:20:27.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.069+0000 7f019fdff700 1 -- 172.21.15.31:0/3308218313 wait complete. 2023-10-06T05:20:27.073 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:20:27.094 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:20:27.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.297+0000 7f2355dff700 1 Processor -- start 2023-10-06T05:20:27.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.297+0000 7f2355dff700 1 -- start start 2023-10-06T05:20:27.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.298+0000 7f2355dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 0x7f23500ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f2355dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f235006ea90 0x7f235006d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f2355dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f235006a630 0x7f235006aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f2355dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235006aff0 con 0x7f235006ea90 2023-10-06T05:20:27.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f2355dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235006e600 con 0x7f235006a630 2023-10-06T05:20:27.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f2355dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235006ef00 con 0x7f2350156300 2023-10-06T05:20:27.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f234effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f235006ea90 0x7f235006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f234effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f235006ea90 0x7f235006d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54282/0 (socket says 172.21.15.31:54282) 2023-10-06T05:20:27.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f234effd700 1 -- 172.21.15.31:0/3049334604 learned_addr learned my addr 172.21.15.31:0/3049334604 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:27.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f234f7fe700 1 --2- 172.21.15.31:0/3049334604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 0x7f23500ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f234ffff700 1 --2- 172.21.15.31:0/3049334604 >> v2:172.21.15.62:3300/0 conn(0x7f235006a630 0x7f235006aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f234effd700 1 -- 172.21.15.31:0/3049334604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 msgr2=0x7f23500ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:27.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f234effd700 1 --2- 172.21.15.31:0/3049334604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 0x7f23500ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f234effd700 1 -- 172.21.15.31:0/3049334604 >> v2:172.21.15.62:3300/0 conn(0x7f235006a630 msgr2=0x7f235006aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:27.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f234effd700 1 --2- 172.21.15.31:0/3049334604 >> v2:172.21.15.62:3300/0 conn(0x7f235006a630 0x7f235006aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.299+0000 7f234effd700 1 -- 172.21.15.31:0/3049334604 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235006d750 con 0x7f235006ea90 2023-10-06T05:20:27.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.300+0000 7f234effd700 1 --2- 172.21.15.31:0/3049334604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006ea90 0x7f235006d210 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f235006d0e0 tx=0x7f2344009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29722001615eb2bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:27.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.300+0000 7f234e7fc700 1 -- 172.21.15.31:0/3049334604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234400b860 con 0x7f235006ea90 2023-10-06T05:20:27.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.300+0000 7f234e7fc700 1 -- 172.21.15.31:0/3049334604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234400b9c0 con 0x7f235006ea90 2023-10-06T05:20:27.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.300+0000 7f234e7fc700 1 -- 172.21.15.31:0/3049334604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234400bd60 con 0x7f235006ea90 2023-10-06T05:20:27.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.300+0000 7f2355dff700 1 -- 172.21.15.31:0/3049334604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006ea90 msgr2=0x7f235006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:27.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.300+0000 7f2355dff700 1 --2- 172.21.15.31:0/3049334604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006ea90 0x7f235006d210 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f235006d0e0 tx=0x7f2344009250 comp rx=0 tx=0).stop 2023-10-06T05:20:27.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.300+0000 7f2355dff700 1 -- 172.21.15.31:0/3049334604 shutdown_connections 2023-10-06T05:20:27.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.301+0000 7f2355dff700 1 --2- 172.21.15.31:0/3049334604 >> v2:172.21.15.62:3300/0 conn(0x7f235006a630 0x7f235006aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.301+0000 7f2355dff700 1 --2- 172.21.15.31:0/3049334604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 0x7f23500ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.301+0000 7f2355dff700 1 --2- 172.21.15.31:0/3049334604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006ea90 0x7f235006d210 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.301+0000 7f2355dff700 1 -- 172.21.15.31:0/3049334604 >> 172.21.15.31:0/3049334604 conn(0x7f2350068030 msgr2=0x7f23500cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:27.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.301+0000 7f2355dff700 1 -- 172.21.15.31:0/3049334604 shutdown_connections 2023-10-06T05:20:27.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.301+0000 7f2355dff700 1 -- 172.21.15.31:0/3049334604 wait complete. 2023-10-06T05:20:27.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.302+0000 7f2355dff700 1 Processor -- start 2023-10-06T05:20:27.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.302+0000 7f2355dff700 1 -- start start 2023-10-06T05:20:27.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.303+0000 7f2355dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006a630 0x7f2350169680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.303+0000 7f2355dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f235006ea90 0x7f23501511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.303+0000 7f2355dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 0x7f235016d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.303+0000 7f2355dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235016a600 con 0x7f235006a630 2023-10-06T05:20:27.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.303+0000 7f2355dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23501516f0 con 0x7f235006ea90 2023-10-06T05:20:27.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.303+0000 7f2355dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23500af2b0 con 0x7f2350156300 2023-10-06T05:20:27.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.303+0000 7f234f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006a630 0x7f2350169680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.303+0000 7f234f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006a630 0x7f2350169680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54284/0 (socket says 172.21.15.31:54284) 2023-10-06T05:20:27.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.303+0000 7f234f7fe700 1 -- 172.21.15.31:0/1961646370 learned_addr learned my addr 172.21.15.31:0/1961646370 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:27.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f234effd700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f235006ea90 0x7f23501511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f234ffff700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 0x7f235016d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f234f7fe700 1 -- 172.21.15.31:0/1961646370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 msgr2=0x7f235016d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:27.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f234f7fe700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 0x7f235016d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f234f7fe700 1 -- 172.21.15.31:0/1961646370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f235006ea90 msgr2=0x7f23501511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:27.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f234f7fe700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f235006ea90 0x7f23501511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:27.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f234f7fe700 1 -- 172.21.15.31:0/1961646370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440094f0 con 0x7f235006a630 2023-10-06T05:20:27.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f234effd700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f235006ea90 0x7f23501511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:27.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f234ffff700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 0x7f235016d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:27.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f234f7fe700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006a630 0x7f2350169680 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f234000d840 tx=0x7f2340008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:27.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f23377fe700 1 -- 172.21.15.31:0/1961646370 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340014620 con 0x7f235006a630 2023-10-06T05:20:27.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f23377fe700 1 -- 172.21.15.31:0/1961646370 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2340014780 con 0x7f235006a630 2023-10-06T05:20:27.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f2355dff700 1 -- 172.21.15.31:0/1961646370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235016db70 con 0x7f235006a630 2023-10-06T05:20:27.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f23377fe700 1 -- 172.21.15.31:0/1961646370 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340014b40 con 0x7f235006a630 2023-10-06T05:20:27.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.304+0000 7f2355dff700 1 -- 172.21.15.31:0/1961646370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235016a040 con 0x7f235006a630 2023-10-06T05:20:27.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.305+0000 7f2355dff700 1 -- 172.21.15.31:0/1961646370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23500af590 con 0x7f235006a630 2023-10-06T05:20:27.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.306+0000 7f23377fe700 1 -- 172.21.15.31:0/1961646370 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f234001d400 con 0x7f235006a630 2023-10-06T05:20:27.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.307+0000 7f23377fe700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f233c0670e0 0x7f233c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:27.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.307+0000 7f23377fe700 1 -- 172.21.15.31:0/1961646370 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f23400851c0 con 0x7f235006a630 2023-10-06T05:20:27.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.308+0000 7f234effd700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f233c0670e0 0x7f233c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:27.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.308+0000 7f234effd700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f233c0670e0 0x7f233c069590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f235006d0e0 tx=0x7f2344017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:27.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.312+0000 7f23377fe700 1 -- 172.21.15.31:0/1961646370 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23400520d0 con 0x7f235006a630 2023-10-06T05:20:27.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:27.515+0000 7f2355dff700 1 -- 172.21.15.31:0/1961646370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f235006d9f0 con 0x7f235006a630 2023-10-06T05:20:28.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.202+0000 7f23377fe700 1 -- 172.21.15.31:0/1961646370 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) v1 ==== 169+0+0 (secure 0 0 0) 0x7f23400556f0 con 0x7f235006a630 2023-10-06T05:20:28.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.211+0000 7f2355dff700 1 -- 172.21.15.31:0/1961646370 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f233c0670e0 msgr2=0x7f233c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.211+0000 7f2355dff700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f233c0670e0 0x7f233c069590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f235006d0e0 tx=0x7f2344017000 comp rx=0 tx=0).stop 2023-10-06T05:20:28.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.211+0000 7f2355dff700 1 -- 172.21.15.31:0/1961646370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006a630 msgr2=0x7f2350169680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.211+0000 7f2355dff700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006a630 0x7f2350169680 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f234000d840 tx=0x7f2340008040 comp rx=0 tx=0).stop 2023-10-06T05:20:28.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.213+0000 7f2355dff700 1 -- 172.21.15.31:0/1961646370 shutdown_connections 2023-10-06T05:20:28.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.213+0000 7f2355dff700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2350156300 0x7f235016d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.213+0000 7f2355dff700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f233c0670e0 0x7f233c069590 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.213+0000 7f2355dff700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235006a630 0x7f2350169680 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.213+0000 7f2355dff700 1 --2- 172.21.15.31:0/1961646370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f235006ea90 0x7f23501511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.213+0000 7f2355dff700 1 -- 172.21.15.31:0/1961646370 >> 172.21.15.31:0/1961646370 conn(0x7f2350068030 msgr2=0x7f23500c6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:28.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.215+0000 7f2355dff700 1 -- 172.21.15.31:0/1961646370 shutdown_connections 2023-10-06T05:20:28.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.215+0000 7f2355dff700 1 -- 172.21.15.31:0/1961646370 wait complete. 2023-10-06T05:20:28.221 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:20:28.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:20:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.443+0000 7f04f6bff700 1 Processor -- start 2023-10-06T05:20:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.443+0000 7f04f6bff700 1 -- start start 2023-10-06T05:20:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04f6bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f04f0153d40 0x7f04f00b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04f6bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04f00af890 0x7f04f00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04f6bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f00b0e40 con 0x7f04f00b09c0 2023-10-06T05:20:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04f6bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f00afcd0 con 0x7f04f0153d40 2023-10-06T05:20:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04f6bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00ad690 con 0x7f04f00af890 2023-10-06T05:20:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04effff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f04f00b09c0 0x7f04f00af180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52652/0 (socket says 172.21.15.31:52652) 2023-10-06T05:20:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04effff700 1 -- 172.21.15.31:0/3029339950 learned_addr learned my addr 172.21.15.31:0/3029339950 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:28.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04f499b700 1 --2- 172.21.15.31:0/3029339950 >> v2:172.21.15.31:3300/0 conn(0x7f04f0153d40 0x7f04f00b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04f519c700 1 --2- 172.21.15.31:0/3029339950 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04f00af890 0x7f04f00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04ef7fe700 1 -- 172.21.15.31:0/3029339950 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689668013 0 0) 0x7f04dc001110 con 0x7f04f00b09c0 2023-10-06T05:20:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.445+0000 7f04ef7fe700 1 -- 172.21.15.31:0/3029339950 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04e0003570 con 0x7f04f00b09c0 2023-10-06T05:20:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.446+0000 7f04f499b700 1 -- 172.21.15.31:0/3029339950 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04f00af890 msgr2=0x7f04f00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.446+0000 7f04f499b700 1 --2- 172.21.15.31:0/3029339950 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04f00af890 0x7f04f00adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.446+0000 7f04f499b700 1 -- 172.21.15.31:0/3029339950 >> v1:172.21.15.31:6789/0 conn(0x7f04f00b09c0 legacy=0x7f04f00af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.446+0000 7f04f499b700 1 -- 172.21.15.31:0/3029339950 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f00acd60 con 0x7f04f0153d40 2023-10-06T05:20:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.446+0000 7f04f499b700 1 --2- 172.21.15.31:0/3029339950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f0153d40 0x7f04f00b0480 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f04e40098e0 tx=0x7f04e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67af5f78b6c7e98 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.446+0000 7f04ef7fe700 1 -- 172.21.15.31:0/3029339950 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e400b640 con 0x7f04f0153d40 2023-10-06T05:20:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.446+0000 7f04ef7fe700 1 -- 172.21.15.31:0/3029339950 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e400b7a0 con 0x7f04f0153d40 2023-10-06T05:20:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.446+0000 7f04ef7fe700 1 -- 172.21.15.31:0/3029339950 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e400bb20 con 0x7f04f0153d40 2023-10-06T05:20:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.447+0000 7f04f6bff700 1 -- 172.21.15.31:0/3029339950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f0153d40 msgr2=0x7f04f00b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.447+0000 7f04f6bff700 1 --2- 172.21.15.31:0/3029339950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f0153d40 0x7f04f00b0480 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f04e40098e0 tx=0x7f04e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.447+0000 7f04f6bff700 1 -- 172.21.15.31:0/3029339950 shutdown_connections 2023-10-06T05:20:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.447+0000 7f04f6bff700 1 --2- 172.21.15.31:0/3029339950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f0153d40 0x7f04f00b0480 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.447+0000 7f04f6bff700 1 --2- 172.21.15.31:0/3029339950 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04f00af890 0x7f04f00adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.447+0000 7f04f6bff700 1 -- 172.21.15.31:0/3029339950 >> 172.21.15.31:0/3029339950 conn(0x7f04f0067560 msgr2=0x7f04f0067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.447+0000 7f04f6bff700 1 -- 172.21.15.31:0/3029339950 shutdown_connections 2023-10-06T05:20:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.447+0000 7f04f6bff700 1 -- 172.21.15.31:0/3029339950 wait complete. 2023-10-06T05:20:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.448+0000 7f04f6bff700 1 Processor -- start 2023-10-06T05:20:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.448+0000 7f04f6bff700 1 -- start start 2023-10-06T05:20:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.449+0000 7f04f6bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f00af890 0x7f04f0156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.449+0000 7f04f6bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00b09c0 0x7f04f00c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.449+0000 7f04f6bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04f0153d40 0x7f04f01513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.449+0000 7f04f6bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00caa10 con 0x7f04f00b09c0 2023-10-06T05:20:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.449+0000 7f04f6bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00cacf0 con 0x7f04f00af890 2023-10-06T05:20:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.449+0000 7f04f6bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00cafd0 con 0x7f04f0153d40 2023-10-06T05:20:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.450+0000 7f04effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00b09c0 0x7f04f00c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.450+0000 7f04effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00b09c0 0x7f04f00c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54298/0 (socket says 172.21.15.31:54298) 2023-10-06T05:20:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.450+0000 7f04effff700 1 -- 172.21.15.31:0/1339935342 learned_addr learned my addr 172.21.15.31:0/1339935342 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.450+0000 7f04f499b700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f00af890 0x7f04f0156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.450+0000 7f04effff700 1 -- 172.21.15.31:0/1339935342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04f0153d40 msgr2=0x7f04f01513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.450+0000 7f04effff700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04f0153d40 0x7f04f01513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.450+0000 7f04effff700 1 -- 172.21.15.31:0/1339935342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f00af890 msgr2=0x7f04f0156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.450+0000 7f04effff700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f00af890 0x7f04f0156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.450+0000 7f04effff700 1 -- 172.21.15.31:0/1339935342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e4009420 con 0x7f04f00b09c0 2023-10-06T05:20:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.450+0000 7f04effff700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00b09c0 0x7f04f00c9fe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00c7c0 tx=0x7f04dc007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.451+0000 7f04ecff9700 1 -- 172.21.15.31:0/1339935342 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc005b30 con 0x7f04f00b09c0 2023-10-06T05:20:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.451+0000 7f04ecff9700 1 -- 172.21.15.31:0/1339935342 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04dc005c90 con 0x7f04f00b09c0 2023-10-06T05:20:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.451+0000 7f04f6bff700 1 -- 172.21.15.31:0/1339935342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f00cb2b0 con 0x7f04f00b09c0 2023-10-06T05:20:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.451+0000 7f04ecff9700 1 -- 172.21.15.31:0/1339935342 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc007320 con 0x7f04f00b09c0 2023-10-06T05:20:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.451+0000 7f04f6bff700 1 -- 172.21.15.31:0/1339935342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f0152150 con 0x7f04f00b09c0 2023-10-06T05:20:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.452+0000 7f04f6bff700 1 -- 172.21.15.31:0/1339935342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f00cb7d0 con 0x7f04f00b09c0 2023-10-06T05:20:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.453+0000 7f04ecff9700 1 -- 172.21.15.31:0/1339935342 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f04dc01b3b0 con 0x7f04f00b09c0 2023-10-06T05:20:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.454+0000 7f04ecff9700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f04d40670e0 0x7f04d4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.454+0000 7f04ecff9700 1 -- 172.21.15.31:0/1339935342 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f04dc083a70 con 0x7f04f00b09c0 2023-10-06T05:20:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.454+0000 7f04f499b700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f04d40670e0 0x7f04d4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.454+0000 7f04f499b700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f04d40670e0 0x7f04d4069590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f04f01529a0 tx=0x7f04e4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.458+0000 7f04ecff9700 1 -- 172.21.15.31:0/1339935342 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04f00cb7d0 con 0x7f04f00b09c0 2023-10-06T05:20:28.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.662+0000 7f04f6bff700 1 -- 172.21.15.31:0/1339935342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f04f00b0e40 con 0x7f04f00b09c0 2023-10-06T05:20:28.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.667+0000 7f04ecff9700 1 -- 172.21.15.31:0/1339935342 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f04f00b0e40 con 0x7f04f00b09c0 2023-10-06T05:20:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 -- 172.21.15.31:0/1339935342 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f04d40670e0 msgr2=0x7f04d4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f04d40670e0 0x7f04d4069590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f04f01529a0 tx=0x7f04e4017000 comp rx=0 tx=0).stop 2023-10-06T05:20:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 -- 172.21.15.31:0/1339935342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00b09c0 msgr2=0x7f04f00c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00b09c0 0x7f04f00c9fe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00c7c0 tx=0x7f04dc007c30 comp rx=0 tx=0).stop 2023-10-06T05:20:28.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 -- 172.21.15.31:0/1339935342 shutdown_connections 2023-10-06T05:20:28.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04f0153d40 0x7f04f01513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f00af890 0x7f04f0156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f04d40670e0 0x7f04d4069590 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 --2- 172.21.15.31:0/1339935342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f00b09c0 0x7f04f00c9fe0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 -- 172.21.15.31:0/1339935342 >> 172.21.15.31:0/1339935342 conn(0x7f04f0067560 msgr2=0x7f04f00cc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:28.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 -- 172.21.15.31:0/1339935342 shutdown_connections 2023-10-06T05:20:28.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.669+0000 7f04f6bff700 1 -- 172.21.15.31:0/1339935342 wait complete. 2023-10-06T05:20:28.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-10-06T05:20:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.883+0000 7f6c807ff700 1 Processor -- start 2023-10-06T05:20:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.883+0000 7f6c807ff700 1 -- start start 2023-10-06T05:20:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.884+0000 7f6c807ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c78068d80 0x7f6c78159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.884+0000 7f6c807ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6c780cc0d0 0x7f6c780c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.884+0000 7f6c807ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c780c95b0 con 0x7f6c781595a0 2023-10-06T05:20:28.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.884+0000 7f6c807ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780c9890 con 0x7f6c78068d80 2023-10-06T05:20:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.884+0000 7f6c807ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780c9b70 con 0x7f6c780cc0d0 2023-10-06T05:20:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.885+0000 7f6c7dd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c78068d80 0x7f6c78159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.885+0000 7f6c7e59b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6c781595a0 0x7f6c78068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52658/0 (socket says 172.21.15.31:52658) 2023-10-06T05:20:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.885+0000 7f6c7e59b700 1 -- 172.21.15.31:0/3586991618 learned_addr learned my addr 172.21.15.31:0/3586991618 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:28.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.885+0000 7f6c7ed9c700 1 --2- 172.21.15.31:0/3586991618 >> v2:172.21.15.62:3300/0 conn(0x7f6c780cc0d0 0x7f6c780c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.885+0000 7f6c7dd9a700 1 -- 172.21.15.31:0/3586991618 >> v2:172.21.15.62:3300/0 conn(0x7f6c780cc0d0 msgr2=0x7f6c780c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.885+0000 7f6c7dd9a700 1 --2- 172.21.15.31:0/3586991618 >> v2:172.21.15.62:3300/0 conn(0x7f6c780cc0d0 0x7f6c780c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.885+0000 7f6c7d599700 1 -- 172.21.15.31:0/3586991618 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542386083 0 0) 0x7f6c780c95b0 con 0x7f6c781595a0 2023-10-06T05:20:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.885+0000 7f6c7dd9a700 1 -- 172.21.15.31:0/3586991618 >> v1:172.21.15.31:6789/0 conn(0x7f6c781595a0 legacy=0x7f6c78068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.885+0000 7f6c7dd9a700 1 -- 172.21.15.31:0/3586991618 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780ca130 con 0x7f6c78068d80 2023-10-06T05:20:28.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.885+0000 7f6c7dd9a700 1 --2- 172.21.15.31:0/3586991618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c78068d80 0x7f6c78159970 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6c680098e0 tx=0x7f6c680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3316396f240882e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c7d599700 1 -- 172.21.15.31:0/3586991618 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68011630 con 0x7f6c78068d80 2023-10-06T05:20:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c7d599700 1 -- 172.21.15.31:0/3586991618 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c68011790 con 0x7f6c78068d80 2023-10-06T05:20:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c7d599700 1 -- 172.21.15.31:0/3586991618 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68011b30 con 0x7f6c78068d80 2023-10-06T05:20:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c807ff700 1 -- 172.21.15.31:0/3586991618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c78068d80 msgr2=0x7f6c78159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c807ff700 1 --2- 172.21.15.31:0/3586991618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c78068d80 0x7f6c78159970 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6c680098e0 tx=0x7f6c680091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c807ff700 1 -- 172.21.15.31:0/3586991618 shutdown_connections 2023-10-06T05:20:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c807ff700 1 --2- 172.21.15.31:0/3586991618 >> v2:172.21.15.62:3300/0 conn(0x7f6c780cc0d0 0x7f6c780c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c807ff700 1 --2- 172.21.15.31:0/3586991618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c78068d80 0x7f6c78159970 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c807ff700 1 -- 172.21.15.31:0/3586991618 >> 172.21.15.31:0/3586991618 conn(0x7f6c7806b620 msgr2=0x7f6c7806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:28.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c807ff700 1 -- 172.21.15.31:0/3586991618 shutdown_connections 2023-10-06T05:20:28.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.886+0000 7f6c807ff700 1 -- 172.21.15.31:0/3586991618 wait complete. 2023-10-06T05:20:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.888+0000 7f6c807ff700 1 Processor -- start 2023-10-06T05:20:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.888+0000 7f6c807ff700 1 -- start start 2023-10-06T05:20:28.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.889+0000 7f6c807ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c78068d80 0x7f6c780b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.889+0000 7f6c807ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c780cc0d0 0x7f6c7814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.889+0000 7f6c807ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c781595a0 0x7f6c7816d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.889+0000 7f6c807ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7816ddd0 con 0x7f6c781595a0 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.889+0000 7f6c807ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780b1130 con 0x7f6c78068d80 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.889+0000 7f6c807ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780b1410 con 0x7f6c780cc0d0 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.889+0000 7f6c7ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c781595a0 0x7f6c7816d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.889+0000 7f6c7ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c781595a0 0x7f6c7816d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54314/0 (socket says 172.21.15.31:54314) 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.889+0000 7f6c7ed9c700 1 -- 172.21.15.31:0/348986892 learned_addr learned my addr 172.21.15.31:0/348986892 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.889+0000 7f6c7e59b700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c78068d80 0x7f6c780b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c7dd9a700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c780cc0d0 0x7f6c7814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c7ed9c700 1 -- 172.21.15.31:0/348986892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c780cc0d0 msgr2=0x7f6c7814ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c7ed9c700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c780cc0d0 0x7f6c7814ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c7ed9c700 1 -- 172.21.15.31:0/348986892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c78068d80 msgr2=0x7f6c780b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c7ed9c700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c78068d80 0x7f6c780b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c7ed9c700 1 -- 172.21.15.31:0/348986892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68009420 con 0x7f6c781595a0 2023-10-06T05:20:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c7e59b700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c78068d80 0x7f6c780b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c7ed9c700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c781595a0 0x7f6c7816d260 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009e90 tx=0x7f6c7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c6e7fc700 1 -- 172.21.15.31:0/348986892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000cc20 con 0x7f6c781595a0 2023-10-06T05:20:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c6e7fc700 1 -- 172.21.15.31:0/348986892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7000f040 con 0x7f6c781595a0 2023-10-06T05:20:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c6e7fc700 1 -- 172.21.15.31:0/348986892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70014480 con 0x7f6c781595a0 2023-10-06T05:20:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c807ff700 1 -- 172.21.15.31:0/348986892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c780b16f0 con 0x7f6c781595a0 2023-10-06T05:20:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.890+0000 7f6c807ff700 1 -- 172.21.15.31:0/348986892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7814f1a0 con 0x7f6c781595a0 2023-10-06T05:20:28.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.891+0000 7f6c807ff700 1 -- 172.21.15.31:0/348986892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c780b18e0 con 0x7f6c781595a0 2023-10-06T05:20:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.892+0000 7f6c6e7fc700 1 -- 172.21.15.31:0/348986892 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6c70014830 con 0x7f6c781595a0 2023-10-06T05:20:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.893+0000 7f6c6e7fc700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c5c0670e0 0x7f6c5c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.893+0000 7f6c6e7fc700 1 -- 172.21.15.31:0/348986892 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f6c70085270 con 0x7f6c781595a0 2023-10-06T05:20:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.893+0000 7f6c7e59b700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c5c0670e0 0x7f6c5c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.894+0000 7f6c7e59b700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c5c0670e0 0x7f6c5c069590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c740011e0 tx=0x7f6c74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:28.897+0000 7f6c6e7fc700 1 -- 172.21.15.31:0/348986892 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c70055ce0 con 0x7f6c781595a0 2023-10-06T05:20:29.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.180+0000 7f6c807ff700 1 -- 172.21.15.31:0/348986892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f6c7804c380 con 0x7f6c781595a0 2023-10-06T05:20:29.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.206+0000 7f6c6e7fc700 1 -- 172.21.15.31:0/348986892 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v186) v1 ==== 140+0+0 (secure 0 0 0) 0x7f6c70055870 con 0x7f6c781595a0 2023-10-06T05:20:29.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 -- 172.21.15.31:0/348986892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c5c0670e0 msgr2=0x7f6c5c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:29.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c5c0670e0 0x7f6c5c069590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c740011e0 tx=0x7f6c74009040 comp rx=0 tx=0).stop 2023-10-06T05:20:29.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 -- 172.21.15.31:0/348986892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c781595a0 msgr2=0x7f6c7816d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:29.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c781595a0 0x7f6c7816d260 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009e90 tx=0x7f6c7000b040 comp rx=0 tx=0).stop 2023-10-06T05:20:29.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 -- 172.21.15.31:0/348986892 shutdown_connections 2023-10-06T05:20:29.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6c5c0670e0 0x7f6c5c069590 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:29.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c781595a0 0x7f6c7816d260 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:29.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c78068d80 0x7f6c780b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:29.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 --2- 172.21.15.31:0/348986892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c780cc0d0 0x7f6c7814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:29.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 -- 172.21.15.31:0/348986892 >> 172.21.15.31:0/348986892 conn(0x7f6c7806b620 msgr2=0x7f6c78153620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:29.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 -- 172.21.15.31:0/348986892 shutdown_connections 2023-10-06T05:20:29.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.208+0000 7f6c807ff700 1 -- 172.21.15.31:0/348986892 wait complete. 2023-10-06T05:20:29.228 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-10-06T05:20:29.228 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2023-10-06T05:20:29.229 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:20:29.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-10-06T05:20:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.423+0000 7f52babff700 1 Processor -- start 2023-10-06T05:20:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.423+0000 7f52babff700 1 -- start start 2023-10-06T05:20:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.425+0000 7f52babff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52b415d2d0 0x7f52b40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.425+0000 7f52babff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b40c3f40 con 0x7f52b4159500 2023-10-06T05:20:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.425+0000 7f52babff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b40cb000 con 0x7f52b40cd510 2023-10-06T05:20:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.425+0000 7f52babff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b40b2860 con 0x7f52b415d2d0 2023-10-06T05:20:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.425+0000 7f52b899b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52b4159500 0x7f52b40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52660/0 (socket says 172.21.15.31:52660) 2023-10-06T05:20:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.425+0000 7f52b899b700 1 -- 172.21.15.31:0/1677630765 learned_addr learned my addr 172.21.15.31:0/1677630765 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.425+0000 7f52b919c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52b415d2d0 0x7f52b40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.425+0000 7f52b37fe700 1 -- 172.21.15.31:0/1677630765 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710944005 0 0) 0x7f52b40c3f40 con 0x7f52b4159500 2023-10-06T05:20:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.426+0000 7f52b919c700 1 -- 172.21.15.31:0/1677630765 >> v1:172.21.15.62:6789/0 conn(0x7f52b40cd510 legacy=0x7f52b40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.426+0000 7f52b919c700 1 -- 172.21.15.31:0/1677630765 >> v1:172.21.15.31:6789/0 conn(0x7f52b4159500 legacy=0x7f52b40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.426+0000 7f52b919c700 1 -- 172.21.15.31:0/1677630765 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b415d7d0 con 0x7f52b415d2d0 2023-10-06T05:20:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.426+0000 7f52b919c700 1 --2- 172.21.15.31:0/1677630765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 0x7f52b40b0ab0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009e60 tx=0x7f52ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e138ef093cca6cc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.426+0000 7f52b37fe700 1 -- 172.21.15.31:0/1677630765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00e070 con 0x7f52b415d2d0 2023-10-06T05:20:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.426+0000 7f52b37fe700 1 -- 172.21.15.31:0/1677630765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac00bc30 con 0x7f52b415d2d0 2023-10-06T05:20:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.426+0000 7f52b37fe700 1 -- 172.21.15.31:0/1677630765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac012430 con 0x7f52b415d2d0 2023-10-06T05:20:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.427+0000 7f52babff700 1 -- 172.21.15.31:0/1677630765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 msgr2=0x7f52b40b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.427+0000 7f52babff700 1 --2- 172.21.15.31:0/1677630765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 0x7f52b40b0ab0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009e60 tx=0x7f52ac009780 comp rx=0 tx=0).stop 2023-10-06T05:20:29.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.427+0000 7f52babff700 1 -- 172.21.15.31:0/1677630765 shutdown_connections 2023-10-06T05:20:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.427+0000 7f52babff700 1 --2- 172.21.15.31:0/1677630765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 0x7f52b40b0ab0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.427+0000 7f52babff700 1 -- 172.21.15.31:0/1677630765 >> 172.21.15.31:0/1677630765 conn(0x7f52b406ec90 msgr2=0x7f52b406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.427+0000 7f52babff700 1 -- 172.21.15.31:0/1677630765 shutdown_connections 2023-10-06T05:20:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.427+0000 7f52babff700 1 -- 172.21.15.31:0/1677630765 wait complete. 2023-10-06T05:20:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.428+0000 7f52babff700 1 Processor -- start 2023-10-06T05:20:29.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.428+0000 7f52babff700 1 -- start start 2023-10-06T05:20:29.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.429+0000 7f52babff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b40cd510 0x7f52b41556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:29.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.429+0000 7f52babff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b4159500 0x7f52b40d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:29.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.429+0000 7f52babff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 0x7f52b40d0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:29.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.429+0000 7f52babff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40b47e0 con 0x7f52b415d2d0 2023-10-06T05:20:29.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.429+0000 7f52babff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40b5360 con 0x7f52b40cd510 2023-10-06T05:20:29.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.429+0000 7f52babff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40b4a30 con 0x7f52b4159500 2023-10-06T05:20:29.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b919c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 0x7f52b40d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:29.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b919c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 0x7f52b40d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54338/0 (socket says 172.21.15.31:54338) 2023-10-06T05:20:29.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b919c700 1 -- 172.21.15.31:0/287012409 learned_addr learned my addr 172.21.15.31:0/287012409 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:29.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b899b700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b40cd510 0x7f52b41556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:29.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b3fff700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b4159500 0x7f52b40d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:29.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b919c700 1 -- 172.21.15.31:0/287012409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b4159500 msgr2=0x7f52b40d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:29.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b919c700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b4159500 0x7f52b40d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:29.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b919c700 1 -- 172.21.15.31:0/287012409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b40cd510 msgr2=0x7f52b41556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:29.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b919c700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b40cd510 0x7f52b41556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:29.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b919c700 1 -- 172.21.15.31:0/287012409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac009a20 con 0x7f52b415d2d0 2023-10-06T05:20:29.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.430+0000 7f52b919c700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 0x7f52b40d0990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f52ac013000 tx=0x7f52ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:29.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.431+0000 7f52b0ff9700 1 -- 172.21.15.31:0/287012409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac027470 con 0x7f52b415d2d0 2023-10-06T05:20:29.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.431+0000 7f52b0ff9700 1 -- 172.21.15.31:0/287012409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac0275d0 con 0x7f52b415d2d0 2023-10-06T05:20:29.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.431+0000 7f52b0ff9700 1 -- 172.21.15.31:0/287012409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac027950 con 0x7f52b415d2d0 2023-10-06T05:20:29.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.431+0000 7f52babff700 1 -- 172.21.15.31:0/287012409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b40b4d10 con 0x7f52b415d2d0 2023-10-06T05:20:29.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.431+0000 7f52babff700 1 -- 172.21.15.31:0/287012409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b4155c30 con 0x7f52b415d2d0 2023-10-06T05:20:29.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.432+0000 7f52babff700 1 -- 172.21.15.31:0/287012409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b404c380 con 0x7f52b415d2d0 2023-10-06T05:20:29.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.433+0000 7f52b0ff9700 1 -- 172.21.15.31:0/287012409 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f52ac024070 con 0x7f52b415d2d0 2023-10-06T05:20:29.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.434+0000 7f52b0ff9700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52980670e0 0x7f5298069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:29.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.434+0000 7f52b0ff9700 1 -- 172.21.15.31:0/287012409 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f52ac086f20 con 0x7f52b415d2d0 2023-10-06T05:20:29.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.434+0000 7f52b899b700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52980670e0 0x7f5298069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:29.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.434+0000 7f52b899b700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52980670e0 0x7f5298069590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f52a8007fd0 tx=0x7f52a8007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:29.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.438+0000 7f52b0ff9700 1 -- 172.21.15.31:0/287012409 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52ac054010 con 0x7f52b415d2d0 2023-10-06T05:20:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:29.724+0000 7f52babff700 1 -- 172.21.15.31:0/287012409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f52b40cade0 con 0x7f52b415d2d0 2023-10-06T05:20:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.210+0000 7f52b0ff9700 1 -- 172.21.15.31:0/287012409 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v187) v1 ==== 164+0+0 (secure 0 0 0) 0x7f52ac057630 con 0x7f52b415d2d0 2023-10-06T05:20:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.212+0000 7f52babff700 1 -- 172.21.15.31:0/287012409 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52980670e0 msgr2=0x7f5298069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.212+0000 7f52babff700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52980670e0 0x7f5298069590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f52a8007fd0 tx=0x7f52a8007d30 comp rx=0 tx=0).stop 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.212+0000 7f52babff700 1 -- 172.21.15.31:0/287012409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 msgr2=0x7f52b40d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.212+0000 7f52babff700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 0x7f52b40d0990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f52ac013000 tx=0x7f52ac019000 comp rx=0 tx=0).stop 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.212+0000 7f52babff700 1 -- 172.21.15.31:0/287012409 shutdown_connections 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.212+0000 7f52babff700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f52980670e0 0x7f5298069590 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.212+0000 7f52babff700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b415d2d0 0x7f52b40d0990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.212+0000 7f52babff700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52b40cd510 0x7f52b41556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.212+0000 7f52babff700 1 --2- 172.21.15.31:0/287012409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52b4159500 0x7f52b40d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.212+0000 7f52babff700 1 -- 172.21.15.31:0/287012409 >> 172.21.15.31:0/287012409 conn(0x7f52b406ec90 msgr2=0x7f52b40ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.213+0000 7f52babff700 1 -- 172.21.15.31:0/287012409 shutdown_connections 2023-10-06T05:20:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.213+0000 7f52babff700 1 -- 172.21.15.31:0/287012409 wait complete. 2023-10-06T05:20:30.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:20:30.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.430+0000 7fb2f01ff700 1 Processor -- start 2023-10-06T05:20:30.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.431+0000 7fb2f01ff700 1 -- start start 2023-10-06T05:20:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.432+0000 7fb2f01ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2e80ae860 0x7fb2e8157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.432+0000 7fb2f01ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb2e80b1260 0x7fb2e80b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.432+0000 7fb2f01ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e8158cf0 con 0x7fb2e81573a0 2023-10-06T05:20:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.432+0000 7fb2f01ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e80b1c20 con 0x7fb2e80b1260 2023-10-06T05:20:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.432+0000 7fb2f01ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e81580e0 con 0x7fb2e80ae860 2023-10-06T05:20:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.432+0000 7fb2edf9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb2e81573a0 0x7fb2e80acb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52666/0 (socket says 172.21.15.31:52666) 2023-10-06T05:20:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.432+0000 7fb2edf9b700 1 -- 172.21.15.31:0/2491483544 learned_addr learned my addr 172.21.15.31:0/2491483544 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.432+0000 7fb2ee79c700 1 --2- 172.21.15.31:0/2491483544 >> v2:172.21.15.31:3300/0 conn(0x7fb2e80b1260 0x7fb2e80b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.432+0000 7fb2ed79a700 1 --2- 172.21.15.31:0/2491483544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2e80ae860 0x7fb2e8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.433+0000 7fb2ecf99700 1 -- 172.21.15.31:0/2491483544 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883118189 0 0) 0x7fb2e8158cf0 con 0x7fb2e81573a0 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.433+0000 7fb2ecf99700 1 -- 172.21.15.31:0/2491483544 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d4003570 con 0x7fb2e81573a0 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.433+0000 7fb2ee79c700 1 -- 172.21.15.31:0/2491483544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2e80ae860 msgr2=0x7fb2e8157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.433+0000 7fb2ee79c700 1 --2- 172.21.15.31:0/2491483544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2e80ae860 0x7fb2e8157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.433+0000 7fb2ee79c700 1 -- 172.21.15.31:0/2491483544 >> v1:172.21.15.31:6789/0 conn(0x7fb2e81573a0 legacy=0x7fb2e80acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.433+0000 7fb2ee79c700 1 -- 172.21.15.31:0/2491483544 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e81583c0 con 0x7fb2e80b1260 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.433+0000 7fb2ee79c700 1 --2- 172.21.15.31:0/2491483544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e80b1260 0x7fb2e80b16e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009e60 tx=0x7fb2e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948135bb8bdade92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.433+0000 7fb2ecf99700 1 -- 172.21.15.31:0/2491483544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e000e070 con 0x7fb2e80b1260 2023-10-06T05:20:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.433+0000 7fb2ecf99700 1 -- 172.21.15.31:0/2491483544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e000bab0 con 0x7fb2e80b1260 2023-10-06T05:20:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.433+0000 7fb2ecf99700 1 -- 172.21.15.31:0/2491483544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e000be70 con 0x7fb2e80b1260 2023-10-06T05:20:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.434+0000 7fb2f01ff700 1 -- 172.21.15.31:0/2491483544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e80b1260 msgr2=0x7fb2e80b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.434+0000 7fb2f01ff700 1 --2- 172.21.15.31:0/2491483544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e80b1260 0x7fb2e80b16e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009e60 tx=0x7fb2e0009780 comp rx=0 tx=0).stop 2023-10-06T05:20:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.434+0000 7fb2f01ff700 1 -- 172.21.15.31:0/2491483544 shutdown_connections 2023-10-06T05:20:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.434+0000 7fb2f01ff700 1 --2- 172.21.15.31:0/2491483544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e80b1260 0x7fb2e80b16e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.434+0000 7fb2f01ff700 1 --2- 172.21.15.31:0/2491483544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2e80ae860 0x7fb2e8157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.434+0000 7fb2f01ff700 1 -- 172.21.15.31:0/2491483544 >> 172.21.15.31:0/2491483544 conn(0x7fb2e8067560 msgr2=0x7fb2e8067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.434+0000 7fb2f01ff700 1 -- 172.21.15.31:0/2491483544 shutdown_connections 2023-10-06T05:20:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.434+0000 7fb2f01ff700 1 -- 172.21.15.31:0/2491483544 wait complete. 2023-10-06T05:20:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.435+0000 7fb2f01ff700 1 Processor -- start 2023-10-06T05:20:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.436+0000 7fb2f01ff700 1 -- start start 2023-10-06T05:20:30.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2f01ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2e80b1260 0x7fb2e8149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2f01ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e81573a0 0x7fb2e816d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2f01ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e815b7b0 0x7fb2e806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2f01ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e816dc80 con 0x7fb2e81573a0 2023-10-06T05:20:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2f01ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e815c930 con 0x7fb2e815b7b0 2023-10-06T05:20:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2f01ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e815bc40 con 0x7fb2e80b1260 2023-10-06T05:20:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2ed79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e81573a0 0x7fb2e816d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2ed79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e81573a0 0x7fb2e816d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54352/0 (socket says 172.21.15.31:54352) 2023-10-06T05:20:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2ed79a700 1 -- 172.21.15.31:0/3690463512 learned_addr learned my addr 172.21.15.31:0/3690463512 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2edf9b700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2e80b1260 0x7fb2e8149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2ee79c700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e815b7b0 0x7fb2e806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2ed79a700 1 -- 172.21.15.31:0/3690463512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2e80b1260 msgr2=0x7fb2e8149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2ed79a700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2e80b1260 0x7fb2e8149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2ed79a700 1 -- 172.21.15.31:0/3690463512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e815b7b0 msgr2=0x7fb2e806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2ed79a700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e815b7b0 0x7fb2e806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2ed79a700 1 -- 172.21.15.31:0/3690463512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e0009a20 con 0x7fb2e81573a0 2023-10-06T05:20:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.437+0000 7fb2ee79c700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e815b7b0 0x7fb2e806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.438+0000 7fb2ed79a700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e81573a0 0x7fb2e816d740 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800e900 tx=0x7fb2d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.438+0000 7fb2ddffb700 1 -- 172.21.15.31:0/3690463512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8013680 con 0x7fb2e81573a0 2023-10-06T05:20:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.438+0000 7fb2ddffb700 1 -- 172.21.15.31:0/3690463512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d80137e0 con 0x7fb2e81573a0 2023-10-06T05:20:30.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.438+0000 7fb2ddffb700 1 -- 172.21.15.31:0/3690463512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8013ba0 con 0x7fb2e81573a0 2023-10-06T05:20:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.438+0000 7fb2f01ff700 1 -- 172.21.15.31:0/3690463512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e815c2e0 con 0x7fb2e81573a0 2023-10-06T05:20:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.438+0000 7fb2f01ff700 1 -- 172.21.15.31:0/3690463512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e8155f40 con 0x7fb2e81573a0 2023-10-06T05:20:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.439+0000 7fb2f01ff700 1 -- 172.21.15.31:0/3690463512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e815cac0 con 0x7fb2e81573a0 2023-10-06T05:20:30.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.440+0000 7fb2ddffb700 1 -- 172.21.15.31:0/3690463512 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fb2d800c460 con 0x7fb2e81573a0 2023-10-06T05:20:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.441+0000 7fb2ddffb700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2cc0670e0 0x7fb2cc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.441+0000 7fb2ddffb700 1 -- 172.21.15.31:0/3690463512 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb2d80848c0 con 0x7fb2e81573a0 2023-10-06T05:20:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.441+0000 7fb2edf9b700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2cc0670e0 0x7fb2cc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.442+0000 7fb2edf9b700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2cc0670e0 0x7fb2cc069590 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4007fd0 tx=0x7fb2e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:30.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.445+0000 7fb2ddffb700 1 -- 172.21.15.31:0/3690463512 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2d80519b0 con 0x7fb2e81573a0 2023-10-06T05:20:30.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.646+0000 7fb2f01ff700 1 -- 172.21.15.31:0/3690463512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb2e804c380 con 0x7fb2e81573a0 2023-10-06T05:20:30.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.650+0000 7fb2ddffb700 1 -- 172.21.15.31:0/3690463512 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb2d8054fd0 con 0x7fb2e81573a0 2023-10-06T05:20:30.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.652+0000 7fb2f01ff700 1 -- 172.21.15.31:0/3690463512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2cc0670e0 msgr2=0x7fb2cc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.652+0000 7fb2f01ff700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2cc0670e0 0x7fb2cc069590 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4007fd0 tx=0x7fb2e4009040 comp rx=0 tx=0).stop 2023-10-06T05:20:30.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.652+0000 7fb2f01ff700 1 -- 172.21.15.31:0/3690463512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e81573a0 msgr2=0x7fb2e816d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:30.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.652+0000 7fb2f01ff700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e81573a0 0x7fb2e816d740 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800e900 tx=0x7fb2d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.653+0000 7fb2f01ff700 1 -- 172.21.15.31:0/3690463512 shutdown_connections 2023-10-06T05:20:30.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.653+0000 7fb2f01ff700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e815b7b0 0x7fb2e806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.653+0000 7fb2f01ff700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2e80b1260 0x7fb2e8149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.653+0000 7fb2f01ff700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb2cc0670e0 0x7fb2cc069590 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.653+0000 7fb2f01ff700 1 --2- 172.21.15.31:0/3690463512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2e81573a0 0x7fb2e816d740 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:30.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.653+0000 7fb2f01ff700 1 -- 172.21.15.31:0/3690463512 >> 172.21.15.31:0/3690463512 conn(0x7fb2e8067560 msgr2=0x7fb2e80a9930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:30.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.653+0000 7fb2f01ff700 1 -- 172.21.15.31:0/3690463512 shutdown_connections 2023-10-06T05:20:30.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:30.653+0000 7fb2f01ff700 1 -- 172.21.15.31:0/3690463512 wait complete. 2023-10-06T05:20:30.672 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:20:30.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:30.767 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:30.871 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:20:30.871 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:20:30.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:31.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:31.044 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:20:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.067+0000 7f6523fff700 1 Processor -- start 2023-10-06T05:20:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.067+0000 7f6523fff700 1 -- start start 2023-10-06T05:20:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.068+0000 7f6523fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f651c0c9040 0x7f651c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.068+0000 7f6523fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c0c95b0 con 0x7f651c068d80 2023-10-06T05:20:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.068+0000 7f6523fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c0c9890 con 0x7f651c1595a0 2023-10-06T05:20:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.068+0000 7f6523fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c0c9b70 con 0x7f651c0c9040 2023-10-06T05:20:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.069+0000 7f652159a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f651c068d80 0x7f651c0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52676/0 (socket says 172.21.15.31:52676) 2023-10-06T05:20:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.069+0000 7f652159a700 1 -- 172.21.15.31:0/2160752527 learned_addr learned my addr 172.21.15.31:0/2160752527 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.069+0000 7f652259c700 1 --2- 172.21.15.31:0/2160752527 >> v2:172.21.15.62:3300/0 conn(0x7f651c0c9040 0x7f651c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.069+0000 7f6520d99700 1 -- 172.21.15.31:0/2160752527 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766282918 0 0) 0x7f651c0c95b0 con 0x7f651c068d80 2023-10-06T05:20:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.069+0000 7f6520d99700 1 -- 172.21.15.31:0/2160752527 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6508003570 con 0x7f651c068d80 2023-10-06T05:20:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.069+0000 7f6520d99700 1 -- 172.21.15.31:0/2160752527 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630589035 0 0) 0x7f651c0c9890 con 0x7f651c1595a0 2023-10-06T05:20:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.069+0000 7f6520d99700 1 -- 172.21.15.31:0/2160752527 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f651c0c95b0 con 0x7f651c1595a0 2023-10-06T05:20:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.069+0000 7f652259c700 1 -- 172.21.15.31:0/2160752527 >> v1:172.21.15.62:6789/0 conn(0x7f651c1595a0 legacy=0x7f651c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.069+0000 7f652259c700 1 -- 172.21.15.31:0/2160752527 >> v1:172.21.15.31:6789/0 conn(0x7f651c068d80 legacy=0x7f651c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.069+0000 7f652259c700 1 -- 172.21.15.31:0/2160752527 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0ca130 con 0x7f651c0c9040 2023-10-06T05:20:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.070+0000 7f652259c700 1 --2- 172.21.15.31:0/2160752527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c0c9040 0x7f651c159970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6514009e90 tx=0x7f6514009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231942449236dc9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.070+0000 7f6520d99700 1 -- 172.21.15.31:0/2160752527 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400e070 con 0x7f651c0c9040 2023-10-06T05:20:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.070+0000 7f6520d99700 1 -- 172.21.15.31:0/2160752527 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651400bb00 con 0x7f651c0c9040 2023-10-06T05:20:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.070+0000 7f6520d99700 1 -- 172.21.15.31:0/2160752527 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514012430 con 0x7f651c0c9040 2023-10-06T05:20:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.070+0000 7f6523fff700 1 -- 172.21.15.31:0/2160752527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c0c9040 msgr2=0x7f651c159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.070+0000 7f6523fff700 1 --2- 172.21.15.31:0/2160752527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c0c9040 0x7f651c159970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6514009e90 tx=0x7f6514009780 comp rx=0 tx=0).stop 2023-10-06T05:20:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.071+0000 7f6523fff700 1 -- 172.21.15.31:0/2160752527 shutdown_connections 2023-10-06T05:20:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.071+0000 7f6523fff700 1 --2- 172.21.15.31:0/2160752527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c0c9040 0x7f651c159970 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.071+0000 7f6523fff700 1 -- 172.21.15.31:0/2160752527 >> 172.21.15.31:0/2160752527 conn(0x7f651c06b620 msgr2=0x7f651c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.071+0000 7f6523fff700 1 -- 172.21.15.31:0/2160752527 shutdown_connections 2023-10-06T05:20:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.071+0000 7f6523fff700 1 -- 172.21.15.31:0/2160752527 wait complete. 2023-10-06T05:20:31.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.072+0000 7f6523fff700 1 Processor -- start 2023-10-06T05:20:31.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.072+0000 7f6523fff700 1 -- start start 2023-10-06T05:20:31.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.073+0000 7f6523fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c068d80 0x7f651c1505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.073+0000 7f6523fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c0c9040 0x7f651c156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.073+0000 7f6523fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f651c1595a0 0x7f651c156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.073+0000 7f6523fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0b2b40 con 0x7f651c1595a0 2023-10-06T05:20:31.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.073+0000 7f6523fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0b1f30 con 0x7f651c068d80 2023-10-06T05:20:31.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.073+0000 7f6523fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0b2210 con 0x7f651c0c9040 2023-10-06T05:20:31.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f652259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f651c1595a0 0x7f651c156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f652159a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c0c9040 0x7f651c156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f6521d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c068d80 0x7f651c1505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f652159a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c0c9040 0x7f651c156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:42388/0 (socket says 172.21.15.31:42388) 2023-10-06T05:20:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f652159a700 1 -- 172.21.15.31:0/3799097696 learned_addr learned my addr 172.21.15.31:0/3799097696 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f652159a700 1 -- 172.21.15.31:0/3799097696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c068d80 msgr2=0x7f651c1505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f652159a700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c068d80 0x7f651c1505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f652159a700 1 -- 172.21.15.31:0/3799097696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f651c1595a0 msgr2=0x7f651c156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f652159a700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f651c1595a0 0x7f651c156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f652159a700 1 -- 172.21.15.31:0/3799097696 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6514009a20 con 0x7f651c0c9040 2023-10-06T05:20:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f652259c700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f651c1595a0 0x7f651c156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.074+0000 7f6521d9b700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c068d80 0x7f651c1505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:31.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.075+0000 7f652159a700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c0c9040 0x7f651c156800 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c7c0 tx=0x7f650c007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:31.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.075+0000 7f6511ffb700 1 -- 172.21.15.31:0/3799097696 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c005c00 con 0x7f651c0c9040 2023-10-06T05:20:31.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.075+0000 7f6511ffb700 1 -- 172.21.15.31:0/3799097696 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c005d60 con 0x7f651c0c9040 2023-10-06T05:20:31.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.075+0000 7f6523fff700 1 -- 172.21.15.31:0/3799097696 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c0b24f0 con 0x7f651c0c9040 2023-10-06T05:20:31.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.075+0000 7f6511ffb700 1 -- 172.21.15.31:0/3799097696 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c007320 con 0x7f651c0c9040 2023-10-06T05:20:31.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.075+0000 7f6523fff700 1 -- 172.21.15.31:0/3799097696 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c150b00 con 0x7f651c0c9040 2023-10-06T05:20:31.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.076+0000 7f6523fff700 1 -- 172.21.15.31:0/3799097696 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c0b2cb0 con 0x7f651c0c9040 2023-10-06T05:20:31.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.077+0000 7f6511ffb700 1 -- 172.21.15.31:0/3799097696 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f650c01b3b0 con 0x7f651c0c9040 2023-10-06T05:20:31.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.078+0000 7f6511ffb700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65000670e0 0x7f6500069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.078+0000 7f6511ffb700 1 -- 172.21.15.31:0/3799097696 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f650c083c20 con 0x7f651c0c9040 2023-10-06T05:20:31.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.078+0000 7f6521d9b700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65000670e0 0x7f6500069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.079+0000 7f6521d9b700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65000670e0 0x7f6500069590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6518001d30 tx=0x7f6518009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:31.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.083+0000 7f6511ffb700 1 -- 172.21.15.31:0/3799097696 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f650c050d10 con 0x7f651c0c9040 2023-10-06T05:20:31.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.332+0000 7f6523fff700 1 -- 172.21.15.31:0/3799097696 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f651c0b2350 con 0x7f651c0c9040 2023-10-06T05:20:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.333+0000 7f6511ffb700 1 -- 172.21.15.31:0/3799097696 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f651c0b2350 con 0x7f651c0c9040 2023-10-06T05:20:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:31.335 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3893202755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3893202755},{"type":"v1","addr":"172.21.15.62:6835","nonce":3893202755}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1993042142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1993042142},{"type":"v1","addr":"172.21.15.62:6834","nonce":1993042142}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/1936175202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1936175202},{"type":"v1","addr":"172.21.15.31:6836","nonce":1936175202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5766,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3858350040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3858350040},{"type":"v1","addr":"172.21.15.31:6837","nonce":3858350040}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[]} 2023-10-06T05:20:31.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 -- 172.21.15.31:0/3799097696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65000670e0 msgr2=0x7f6500069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65000670e0 0x7f6500069590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6518001d30 tx=0x7f6518009040 comp rx=0 tx=0).stop 2023-10-06T05:20:31.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 -- 172.21.15.31:0/3799097696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c0c9040 msgr2=0x7f651c156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c0c9040 0x7f651c156800 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c7c0 tx=0x7f650c007dd0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 -- 172.21.15.31:0/3799097696 shutdown_connections 2023-10-06T05:20:31.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65000670e0 0x7f6500069590 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f651c1595a0 0x7f651c156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c068d80 0x7f651c1505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 --2- 172.21.15.31:0/3799097696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c0c9040 0x7f651c156800 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 -- 172.21.15.31:0/3799097696 >> 172.21.15.31:0/3799097696 conn(0x7f651c06b620 msgr2=0x7f651c157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 -- 172.21.15.31:0/3799097696 shutdown_connections 2023-10-06T05:20:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.336+0000 7f6523fff700 1 -- 172.21.15.31:0/3799097696 wait complete. 2023-10-06T05:20:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-10-06T05:20:31.356 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:20:31.357 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:20:31.357 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:20:31.357 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:31.357 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:20:31.357 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:20:31.358 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:20:31.358 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:31.358 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:20:31.358 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:20:31.358 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:20:31.358 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:31.359 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:20:31.359 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:20:31.359 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:20:31.359 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:20:31.371 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:20:31.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:20:31.372 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:20:31.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:20:31.376 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:20:31.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:20:31.377 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:20:31.377 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:20:31.416 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:20:31.417 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:31.419 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:20:31.419 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:31.421 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:20:31.421 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:31.424 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:20:31.424 DEBUG:teuthology.parallel:result is None 2023-10-06T05:20:31.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:20:31.567 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:20:31.568 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:20:31.569 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:20:31.570 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.635+0000 7fc76c3ff700 1 Processor -- start 2023-10-06T05:20:31.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.635+0000 7fc76c3ff700 1 -- start start 2023-10-06T05:20:31.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.636+0000 7fc76c3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7640c7b60 0x7fc764069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.636+0000 7fc76c3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc7640c7f30 0x7fc7640aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.636+0000 7fc76c3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7640ca5b0 con 0x7fc76406ef50 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76c3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76406f740 con 0x7fc7640c7b60 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76c3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7640c9c80 con 0x7fc7640c7f30 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76a19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7640c7b60 0x7fc764069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76a19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7640c7b60 0x7fc764069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54412/0 (socket says 172.21.15.31:54412) 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76a19b700 1 -- 172.21.15.31:0/2175749563 learned_addr learned my addr 172.21.15.31:0/2175749563 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76a99c700 1 --1- 172.21.15.31:0/2175749563 >> v1:172.21.15.31:6789/0 conn(0x7fc76406ef50 0x7fc76413f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52714/0 (socket says 172.21.15.31:52714) 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76999a700 1 --2- 172.21.15.31:0/2175749563 >> v2:172.21.15.62:3300/0 conn(0x7fc7640c7f30 0x7fc7640aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76a19b700 1 -- 172.21.15.31:0/2175749563 >> v2:172.21.15.62:3300/0 conn(0x7fc7640c7f30 msgr2=0x7fc7640aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:31.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76a19b700 1 --2- 172.21.15.31:0/2175749563 >> v2:172.21.15.62:3300/0 conn(0x7fc7640c7f30 0x7fc7640aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76a19b700 1 -- 172.21.15.31:0/2175749563 >> v1:172.21.15.31:6789/0 conn(0x7fc76406ef50 legacy=0x7fc76413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76a19b700 1 -- 172.21.15.31:0/2175749563 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7640c9f60 con 0x7fc7640c7b60 2023-10-06T05:20:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.637+0000 7fc76a19b700 1 --2- 172.21.15.31:0/2175749563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7640c7b60 0x7fc764069880 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc760009990 tx=0x7fc760009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b61f4730a68e693 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.638+0000 7fc769199700 1 -- 172.21.15.31:0/2175749563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760009e80 con 0x7fc7640c7b60 2023-10-06T05:20:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.638+0000 7fc769199700 1 -- 172.21.15.31:0/2175749563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc760005c10 con 0x7fc7640c7b60 2023-10-06T05:20:31.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.638+0000 7fc769199700 1 -- 172.21.15.31:0/2175749563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76000b610 con 0x7fc7640c7b60 2023-10-06T05:20:31.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.638+0000 7fc76c3ff700 1 -- 172.21.15.31:0/2175749563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7640c7b60 msgr2=0x7fc764069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.638+0000 7fc76c3ff700 1 --2- 172.21.15.31:0/2175749563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7640c7b60 0x7fc764069880 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc760009990 tx=0x7fc760009220 comp rx=0 tx=0).stop 2023-10-06T05:20:31.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.638+0000 7fc76c3ff700 1 -- 172.21.15.31:0/2175749563 shutdown_connections 2023-10-06T05:20:31.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.638+0000 7fc76c3ff700 1 --2- 172.21.15.31:0/2175749563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7640c7b60 0x7fc764069880 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.638+0000 7fc76c3ff700 1 --2- 172.21.15.31:0/2175749563 >> v2:172.21.15.62:3300/0 conn(0x7fc7640c7f30 0x7fc7640aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.638+0000 7fc76c3ff700 1 -- 172.21.15.31:0/2175749563 >> 172.21.15.31:0/2175749563 conn(0x7fc7640cc0b0 msgr2=0x7fc7640cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:31.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.638+0000 7fc76c3ff700 1 -- 172.21.15.31:0/2175749563 shutdown_connections 2023-10-06T05:20:31.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.639+0000 7fc76c3ff700 1 -- 172.21.15.31:0/2175749563 wait complete. 2023-10-06T05:20:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.641+0000 7fc76c3ff700 1 Processor -- start 2023-10-06T05:20:31.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.641+0000 7fc76c3ff700 1 -- start start 2023-10-06T05:20:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.642+0000 7fc76c3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76406ef50 0x7fc764153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.642+0000 7fc76c3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7640c7b60 0x7fc7640aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.642+0000 7fc76c3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7640c7f30 0x7fc7640ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.642+0000 7fc76c3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640aaae0 con 0x7fc76406ef50 2023-10-06T05:20:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.642+0000 7fc76c3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640c2e60 con 0x7fc7640c7b60 2023-10-06T05:20:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.642+0000 7fc76c3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640a9a40 con 0x7fc7640c7f30 2023-10-06T05:20:31.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.642+0000 7fc76a19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76406ef50 0x7fc764153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.642+0000 7fc76a19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76406ef50 0x7fc764153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54416/0 (socket says 172.21.15.31:54416) 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.642+0000 7fc76a19b700 1 -- 172.21.15.31:0/643175182 learned_addr learned my addr 172.21.15.31:0/643175182 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76999a700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7640c7b60 0x7fc7640aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76a99c700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7640c7f30 0x7fc7640ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76a19b700 1 -- 172.21.15.31:0/643175182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7640c7f30 msgr2=0x7fc7640ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76a19b700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7640c7f30 0x7fc7640ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76a19b700 1 -- 172.21.15.31:0/643175182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7640c7b60 msgr2=0x7fc7640aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76a19b700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7640c7b60 0x7fc7640aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76a19b700 1 -- 172.21.15.31:0/643175182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760009480 con 0x7fc76406ef50 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76a99c700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7640c7f30 0x7fc7640ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:31.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76a19b700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76406ef50 0x7fc764153280 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc76406cd80 tx=0x7fc760016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc75a7fc700 1 -- 172.21.15.31:0/643175182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760011ca0 con 0x7fc76406ef50 2023-10-06T05:20:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc75a7fc700 1 -- 172.21.15.31:0/643175182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc760011e00 con 0x7fc76406ef50 2023-10-06T05:20:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc75a7fc700 1 -- 172.21.15.31:0/643175182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76001b680 con 0x7fc76406ef50 2023-10-06T05:20:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76c3ff700 1 -- 172.21.15.31:0/643175182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7640a9d20 con 0x7fc76406ef50 2023-10-06T05:20:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.643+0000 7fc76c3ff700 1 -- 172.21.15.31:0/643175182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7640aa2e0 con 0x7fc76406ef50 2023-10-06T05:20:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.644+0000 7fc76c3ff700 1 -- 172.21.15.31:0/643175182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76404c380 con 0x7fc76406ef50 2023-10-06T05:20:31.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.645+0000 7fc75a7fc700 1 -- 172.21.15.31:0/643175182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fc76001ba70 con 0x7fc76406ef50 2023-10-06T05:20:31.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.646+0000 7fc75a7fc700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc7480670e0 0x7fc748069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:31.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.646+0000 7fc75a7fc700 1 -- 172.21.15.31:0/643175182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fc7600861b0 con 0x7fc76406ef50 2023-10-06T05:20:31.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.646+0000 7fc76999a700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc7480670e0 0x7fc748069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:31.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.647+0000 7fc76999a700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc7480670e0 0x7fc748069590 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc7641538e0 tx=0x7fc754008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:31.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.651+0000 7fc75a7fc700 1 -- 172.21.15.31:0/643175182 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7600532a0 con 0x7fc76406ef50 2023-10-06T05:20:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:31.869+0000 7fc76c3ff700 1 -- 172.21.15.31:0/643175182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc7640c99c0 con 0x7fc76406ef50 2023-10-06T05:20:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.223+0000 7fc75a7fc700 1 -- 172.21.15.31:0/643175182 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc7600568c0 con 0x7fc76406ef50 2023-10-06T05:20:32.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.225+0000 7fc76c3ff700 1 -- 172.21.15.31:0/643175182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc7480670e0 msgr2=0x7fc748069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.225+0000 7fc76c3ff700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc7480670e0 0x7fc748069590 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc7641538e0 tx=0x7fc754008040 comp rx=0 tx=0).stop 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.225+0000 7fc76c3ff700 1 -- 172.21.15.31:0/643175182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76406ef50 msgr2=0x7fc764153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.225+0000 7fc76c3ff700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76406ef50 0x7fc764153280 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc76406cd80 tx=0x7fc760016000 comp rx=0 tx=0).stop 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.226+0000 7fc76c3ff700 1 -- 172.21.15.31:0/643175182 shutdown_connections 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.226+0000 7fc76c3ff700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7640c7f30 0x7fc7640ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.226+0000 7fc76c3ff700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc7480670e0 0x7fc748069590 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.226+0000 7fc76c3ff700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76406ef50 0x7fc764153280 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.226+0000 7fc76c3ff700 1 --2- 172.21.15.31:0/643175182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7640c7b60 0x7fc7640aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.226+0000 7fc76c3ff700 1 -- 172.21.15.31:0/643175182 >> 172.21.15.31:0/643175182 conn(0x7fc7640cc0b0 msgr2=0x7fc7640cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.226+0000 7fc76c3ff700 1 -- 172.21.15.31:0/643175182 shutdown_connections 2023-10-06T05:20:32.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.226+0000 7fc76c3ff700 1 -- 172.21.15.31:0/643175182 wait complete. 2023-10-06T05:20:32.229 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:20:32.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:20:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.444+0000 7f41eadff700 1 Processor -- start 2023-10-06T05:20:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.445+0000 7f41eadff700 1 -- start start 2023-10-06T05:20:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.446+0000 7f41eadff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f41e40cdfe0 0x7f41e40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.446+0000 7f41eadff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e40ccc80 0x7f41e40cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.446+0000 7f41eadff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e40cf280 con 0x7f41e4159580 2023-10-06T05:20:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.446+0000 7f41eadff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e40ce480 con 0x7f41e40cdfe0 2023-10-06T05:20:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.446+0000 7f41eadff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e40ce670 con 0x7f41e40ccc80 2023-10-06T05:20:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.446+0000 7f41e8b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f41e4159580 0x7f41e40cb9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52726/0 (socket says 172.21.15.31:52726) 2023-10-06T05:20:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.446+0000 7f41e8b9b700 1 -- 172.21.15.31:0/2315567477 learned_addr learned my addr 172.21.15.31:0/2315567477 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.446+0000 7f41e939c700 1 --2- 172.21.15.31:0/2315567477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e40ccc80 0x7f41e40cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.446+0000 7f41e3fff700 1 --2- 172.21.15.31:0/2315567477 >> v2:172.21.15.62:3300/0 conn(0x7f41e40cdfe0 0x7f41e40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.446+0000 7f41e37fe700 1 -- 172.21.15.31:0/2315567477 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369997993 0 0) 0x7f41e40cf280 con 0x7f41e4159580 2023-10-06T05:20:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.447+0000 7f41e37fe700 1 -- 172.21.15.31:0/2315567477 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d4003570 con 0x7f41e4159580 2023-10-06T05:20:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.447+0000 7f41e939c700 1 -- 172.21.15.31:0/2315567477 >> v2:172.21.15.62:3300/0 conn(0x7f41e40cdfe0 msgr2=0x7f41e40cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.447+0000 7f41e939c700 1 --2- 172.21.15.31:0/2315567477 >> v2:172.21.15.62:3300/0 conn(0x7f41e40cdfe0 0x7f41e40cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.447+0000 7f41e939c700 1 -- 172.21.15.31:0/2315567477 >> v1:172.21.15.31:6789/0 conn(0x7f41e4159580 legacy=0x7f41e40cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.447+0000 7f41e939c700 1 -- 172.21.15.31:0/2315567477 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e40ce950 con 0x7f41e40ccc80 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.447+0000 7f41e939c700 1 --2- 172.21.15.31:0/2315567477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e40ccc80 0x7f41e40cd0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009e60 tx=0x7f41dc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d57eea5babc701 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.447+0000 7f41e37fe700 1 -- 172.21.15.31:0/2315567477 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00e070 con 0x7f41e40ccc80 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.447+0000 7f41e37fe700 1 -- 172.21.15.31:0/2315567477 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc00bab0 con 0x7f41e40ccc80 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.447+0000 7f41e37fe700 1 -- 172.21.15.31:0/2315567477 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00be70 con 0x7f41e40ccc80 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.448+0000 7f41eadff700 1 -- 172.21.15.31:0/2315567477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e40ccc80 msgr2=0x7f41e40cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.448+0000 7f41eadff700 1 --2- 172.21.15.31:0/2315567477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e40ccc80 0x7f41e40cd0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009e60 tx=0x7f41dc009780 comp rx=0 tx=0).stop 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.448+0000 7f41eadff700 1 -- 172.21.15.31:0/2315567477 shutdown_connections 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.448+0000 7f41eadff700 1 --2- 172.21.15.31:0/2315567477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e40ccc80 0x7f41e40cd0a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.448+0000 7f41eadff700 1 --2- 172.21.15.31:0/2315567477 >> v2:172.21.15.62:3300/0 conn(0x7f41e40cdfe0 0x7f41e40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.448+0000 7f41eadff700 1 -- 172.21.15.31:0/2315567477 >> 172.21.15.31:0/2315567477 conn(0x7f41e4099710 msgr2=0x7f41e406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.448+0000 7f41eadff700 1 -- 172.21.15.31:0/2315567477 shutdown_connections 2023-10-06T05:20:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.448+0000 7f41eadff700 1 -- 172.21.15.31:0/2315567477 wait complete. 2023-10-06T05:20:32.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.449+0000 7f41eadff700 1 Processor -- start 2023-10-06T05:20:32.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.450+0000 7f41eadff700 1 -- start start 2023-10-06T05:20:32.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.450+0000 7f41eadff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41e40ccc80 0x7f41e416c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41eadff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e40cdfe0 0x7f41e41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41eadff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e4159580 0x7f41e40b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41eadff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e40c7a50 con 0x7f41e40cdfe0 2023-10-06T05:20:32.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41eadff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e40b00d0 con 0x7f41e40ccc80 2023-10-06T05:20:32.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41eadff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e416d040 con 0x7f41e4159580 2023-10-06T05:20:32.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e40cdfe0 0x7f41e41566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e40cdfe0 0x7f41e41566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54424/0 (socket says 172.21.15.31:54424) 2023-10-06T05:20:32.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41e3fff700 1 -- 172.21.15.31:0/1215804884 learned_addr learned my addr 172.21.15.31:0/1215804884 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:32.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41e8b9b700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41e40ccc80 0x7f41e416c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41e939c700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e4159580 0x7f41e40b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41e3fff700 1 -- 172.21.15.31:0/1215804884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e4159580 msgr2=0x7f41e40b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41e3fff700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e4159580 0x7f41e40b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41e3fff700 1 -- 172.21.15.31:0/1215804884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41e40ccc80 msgr2=0x7f41e416c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41e3fff700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41e40ccc80 0x7f41e416c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.451+0000 7f41e3fff700 1 -- 172.21.15.31:0/1215804884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc009a20 con 0x7f41e40cdfe0 2023-10-06T05:20:32.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.452+0000 7f41e3fff700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e40cdfe0 0x7f41e41566f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f41d000ec80 tx=0x7f41d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:32.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.452+0000 7f41e0ff9700 1 -- 172.21.15.31:0/1215804884 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000c850 con 0x7f41e40cdfe0 2023-10-06T05:20:32.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.452+0000 7f41e0ff9700 1 -- 172.21.15.31:0/1215804884 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d000c9b0 con 0x7f41e40cdfe0 2023-10-06T05:20:32.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.452+0000 7f41eadff700 1 -- 172.21.15.31:0/1215804884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e416d320 con 0x7f41e40cdfe0 2023-10-06T05:20:32.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.452+0000 7f41e0ff9700 1 -- 172.21.15.31:0/1215804884 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000cd50 con 0x7f41e40cdfe0 2023-10-06T05:20:32.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.452+0000 7f41eadff700 1 -- 172.21.15.31:0/1215804884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e416d6b0 con 0x7f41e40cdfe0 2023-10-06T05:20:32.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.453+0000 7f41eadff700 1 -- 172.21.15.31:0/1215804884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e40b0280 con 0x7f41e40cdfe0 2023-10-06T05:20:32.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.454+0000 7f41e0ff9700 1 -- 172.21.15.31:0/1215804884 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f41d0013680 con 0x7f41e40cdfe0 2023-10-06T05:20:32.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.455+0000 7f41e0ff9700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41c80670e0 0x7f41c8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.455+0000 7f41e0ff9700 1 -- 172.21.15.31:0/1215804884 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f41d0084920 con 0x7f41e40cdfe0 2023-10-06T05:20:32.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.455+0000 7f41e8b9b700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41c80670e0 0x7f41c8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.455+0000 7f41e8b9b700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41c80670e0 0x7f41c8069590 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f41d8001360 tx=0x7f41d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:32.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.459+0000 7f41e0ff9700 1 -- 172.21.15.31:0/1215804884 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41e40b0280 con 0x7f41e40cdfe0 2023-10-06T05:20:32.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.746+0000 7f41eadff700 1 -- 172.21.15.31:0/1215804884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f41e416cdf0 con 0x7f41e40cdfe0 2023-10-06T05:20:32.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.746+0000 7f41e0ff9700 1 -- 172.21.15.31:0/1215804884 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f41d0051a60 con 0x7f41e40cdfe0 2023-10-06T05:20:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:20:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:20:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:20:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:20:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:20:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:20:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:20:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:20:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:20:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:20:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:20:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:20:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:20:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:20:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:20:32.765 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:20:32.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 -- 172.21.15.31:0/1215804884 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41c80670e0 msgr2=0x7f41c8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41c80670e0 0x7f41c8069590 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f41d8001360 tx=0x7f41d8009040 comp rx=0 tx=0).stop 2023-10-06T05:20:32.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 -- 172.21.15.31:0/1215804884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e40cdfe0 msgr2=0x7f41e41566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e40cdfe0 0x7f41e41566f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f41d000ec80 tx=0x7f41d00091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 -- 172.21.15.31:0/1215804884 shutdown_connections 2023-10-06T05:20:32.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41e4159580 0x7f41e40b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41e40ccc80 0x7f41e416c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f41c80670e0 0x7f41c8069590 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 --2- 172.21.15.31:0/1215804884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e40cdfe0 0x7f41e41566f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 -- 172.21.15.31:0/1215804884 >> 172.21.15.31:0/1215804884 conn(0x7f41e4099710 msgr2=0x7f41e40b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:32.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 -- 172.21.15.31:0/1215804884 shutdown_connections 2023-10-06T05:20:32.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.749+0000 7f41eadff700 1 -- 172.21.15.31:0/1215804884 wait complete. 2023-10-06T05:20:32.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:32.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.962+0000 7fa4585ff700 1 Processor -- start 2023-10-06T05:20:32.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.962+0000 7fa4585ff700 1 -- start start 2023-10-06T05:20:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.963+0000 7fa4585ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4500ca770 0x7fa4500c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.963+0000 7fa4585ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4500cab40 0x7fa4501569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.963+0000 7fa4585ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa450156ef0 con 0x7fa45006ae00 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.963+0000 7fa4585ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4500c7a70 con 0x7fa4500ca770 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.963+0000 7fa4585ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500c6df0 con 0x7fa4500cab40 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.964+0000 7fa45639b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4500ca770 0x7fa4500c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.964+0000 7fa45639b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4500ca770 0x7fa4500c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54426/0 (socket says 172.21.15.31:54426) 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.964+0000 7fa45639b700 1 -- 172.21.15.31:0/2834996009 learned_addr learned my addr 172.21.15.31:0/2834996009 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.964+0000 7fa455b9a700 1 --2- 172.21.15.31:0/2834996009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4500cab40 0x7fa4501569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.964+0000 7fa45639b700 1 -- 172.21.15.31:0/2834996009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4500cab40 msgr2=0x7fa4501569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.964+0000 7fa45639b700 1 --2- 172.21.15.31:0/2834996009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4500cab40 0x7fa4501569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.964+0000 7fa45639b700 1 -- 172.21.15.31:0/2834996009 >> v1:172.21.15.31:6789/0 conn(0x7fa45006ae00 legacy=0x7fa45014c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.964+0000 7fa45639b700 1 -- 172.21.15.31:0/2834996009 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500c70d0 con 0x7fa4500ca770 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.964+0000 7fa45639b700 1 --2- 172.21.15.31:0/2834996009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500ca770 0x7fa4500c66f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00ab30 tx=0x7fa44c00a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbee5c848c77967 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa455399700 1 -- 172.21.15.31:0/2834996009 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c005c10 con 0x7fa4500ca770 2023-10-06T05:20:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa455399700 1 -- 172.21.15.31:0/2834996009 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c005d70 con 0x7fa4500ca770 2023-10-06T05:20:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa455399700 1 -- 172.21.15.31:0/2834996009 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00ca60 con 0x7fa4500ca770 2023-10-06T05:20:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa4585ff700 1 -- 172.21.15.31:0/2834996009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500ca770 msgr2=0x7fa4500c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa4585ff700 1 --2- 172.21.15.31:0/2834996009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500ca770 0x7fa4500c66f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00ab30 tx=0x7fa44c00a3c0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa4585ff700 1 -- 172.21.15.31:0/2834996009 shutdown_connections 2023-10-06T05:20:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa4585ff700 1 --2- 172.21.15.31:0/2834996009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500ca770 0x7fa4500c66f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa4585ff700 1 --2- 172.21.15.31:0/2834996009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4500cab40 0x7fa4501569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa4585ff700 1 -- 172.21.15.31:0/2834996009 >> 172.21.15.31:0/2834996009 conn(0x7fa4500cc0b0 msgr2=0x7fa4500cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa4585ff700 1 -- 172.21.15.31:0/2834996009 shutdown_connections 2023-10-06T05:20:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.965+0000 7fa4585ff700 1 -- 172.21.15.31:0/2834996009 wait complete. 2023-10-06T05:20:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.967+0000 7fa4585ff700 1 Processor -- start 2023-10-06T05:20:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.967+0000 7fa4585ff700 1 -- start start 2023-10-06T05:20:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa4585ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa45006ae00 0x7fa4500ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa4585ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4500ca770 0x7fa4500ab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa4585ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500cab40 0x7fa45013ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa4585ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500ad180 con 0x7fa4500cab40 2023-10-06T05:20:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa4585ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500adfc0 con 0x7fa4500ca770 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa4585ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500abc40 con 0x7fa45006ae00 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa456b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500cab40 0x7fa45013ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa456b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500cab40 0x7fa45013ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54440/0 (socket says 172.21.15.31:54440) 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa456b9c700 1 -- 172.21.15.31:0/3861061073 learned_addr learned my addr 172.21.15.31:0/3861061073 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa45639b700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa45006ae00 0x7fa4500ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.968+0000 7fa455b9a700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4500ca770 0x7fa4500ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa456b9c700 1 -- 172.21.15.31:0/3861061073 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa45006ae00 msgr2=0x7fa4500ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa456b9c700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa45006ae00 0x7fa4500ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa456b9c700 1 -- 172.21.15.31:0/3861061073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4500ca770 msgr2=0x7fa4500ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa456b9c700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4500ca770 0x7fa4500ab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa456b9c700 1 -- 172.21.15.31:0/3861061073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c00a620 con 0x7fa4500cab40 2023-10-06T05:20:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa456b9c700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500cab40 0x7fa45013ec20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa44800c840 tx=0x7fa448001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa4467fc700 1 -- 172.21.15.31:0/3861061073 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448012630 con 0x7fa4500cab40 2023-10-06T05:20:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa4467fc700 1 -- 172.21.15.31:0/3861061073 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4480127b0 con 0x7fa4500cab40 2023-10-06T05:20:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa4467fc700 1 -- 172.21.15.31:0/3861061073 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448012b70 con 0x7fa4500cab40 2023-10-06T05:20:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa4585ff700 1 -- 172.21.15.31:0/3861061073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4500ad3b0 con 0x7fa4500cab40 2023-10-06T05:20:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.969+0000 7fa4585ff700 1 -- 172.21.15.31:0/3861061073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4500ad970 con 0x7fa4500cab40 2023-10-06T05:20:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.971+0000 7fa4585ff700 1 -- 172.21.15.31:0/3861061073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4500ae130 con 0x7fa4500cab40 2023-10-06T05:20:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.971+0000 7fa4467fc700 1 -- 172.21.15.31:0/3861061073 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fa44801a3a0 con 0x7fa4500cab40 2023-10-06T05:20:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.973+0000 7fa4467fc700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4340670e0 0x7fa434069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.973+0000 7fa4467fc700 1 -- 172.21.15.31:0/3861061073 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa448084820 con 0x7fa4500cab40 2023-10-06T05:20:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.973+0000 7fa45639b700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4340670e0 0x7fa434069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.974+0000 7fa45639b700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4340670e0 0x7fa434069590 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00a400 tx=0x7fa44c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:32.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:32.977+0000 7fa4467fc700 1 -- 172.21.15.31:0/3861061073 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa448051960 con 0x7fa4500cab40 2023-10-06T05:20:33.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.178+0000 7fa4585ff700 1 -- 172.21.15.31:0/3861061073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa45004c380 con 0x7fa4500cab40 2023-10-06T05:20:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.179+0000 7fa4467fc700 1 -- 172.21.15.31:0/3861061073 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+13698 (secure 0 0 0) 0x7fa448054f80 con 0x7fa4500cab40 2023-10-06T05:20:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:33.182 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":53,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:20:32.229774+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.181+0000 7fa4585ff700 1 -- 172.21.15.31:0/3861061073 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4340670e0 msgr2=0x7fa434069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.181+0000 7fa4585ff700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4340670e0 0x7fa434069590 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00a400 tx=0x7fa44c019000 comp rx=0 tx=0).stop 2023-10-06T05:20:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.181+0000 7fa4585ff700 1 -- 172.21.15.31:0/3861061073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500cab40 msgr2=0x7fa45013ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.181+0000 7fa4585ff700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500cab40 0x7fa45013ec20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa44800c840 tx=0x7fa448001670 comp rx=0 tx=0).stop 2023-10-06T05:20:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.181+0000 7fa4585ff700 1 -- 172.21.15.31:0/3861061073 shutdown_connections 2023-10-06T05:20:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.181+0000 7fa4585ff700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4340670e0 0x7fa434069590 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.181+0000 7fa4585ff700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4500cab40 0x7fa45013ec20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.181+0000 7fa4585ff700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa45006ae00 0x7fa4500ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.181+0000 7fa4585ff700 1 --2- 172.21.15.31:0/3861061073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4500ca770 0x7fa4500ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.182+0000 7fa4585ff700 1 -- 172.21.15.31:0/3861061073 >> 172.21.15.31:0/3861061073 conn(0x7fa4500cc0b0 msgr2=0x7fa4500c19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:33.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.182+0000 7fa4585ff700 1 -- 172.21.15.31:0/3861061073 shutdown_connections 2023-10-06T05:20:33.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.182+0000 7fa4585ff700 1 -- 172.21.15.31:0/3861061073 wait complete. 2023-10-06T05:20:33.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:20:33.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:20:33.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.400+0000 7fb936bff700 1 Processor -- start 2023-10-06T05:20:33.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.400+0000 7fb936bff700 1 -- start start 2023-10-06T05:20:33.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.401+0000 7fb936bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159890 0x7fb9300c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.402+0000 7fb936bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb930158190 con 0x7fb930159c60 2023-10-06T05:20:33.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.402+0000 7fb936bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb930157580 con 0x7fb930155430 2023-10-06T05:20:33.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.402+0000 7fb936bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb930157860 con 0x7fb930159890 2023-10-06T05:20:33.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.402+0000 7fb92ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb930159c60 0x7fb930156a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52754/0 (socket says 172.21.15.31:52754) 2023-10-06T05:20:33.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.402+0000 7fb92ffff700 1 -- 172.21.15.31:0/43588117 learned_addr learned my addr 172.21.15.31:0/43588117 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.402+0000 7fb93499b700 1 --2- 172.21.15.31:0/43588117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159890 0x7fb9300c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.402+0000 7fb92f7fe700 1 -- 172.21.15.31:0/43588117 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248787136 0 0) 0x7fb930158190 con 0x7fb930159c60 2023-10-06T05:20:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.402+0000 7fb92f7fe700 1 -- 172.21.15.31:0/43588117 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb920003570 con 0x7fb930159c60 2023-10-06T05:20:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.402+0000 7fb92f7fe700 1 -- 172.21.15.31:0/43588117 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187596009 0 0) 0x7fb930157580 con 0x7fb930155430 2023-10-06T05:20:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.403+0000 7fb92f7fe700 1 -- 172.21.15.31:0/43588117 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb930158190 con 0x7fb930155430 2023-10-06T05:20:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.403+0000 7fb92f7fe700 1 -- 172.21.15.31:0/43588117 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1341746468 0 0) 0x7fb920003570 con 0x7fb930159c60 2023-10-06T05:20:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.403+0000 7fb93499b700 1 -- 172.21.15.31:0/43588117 >> v1:172.21.15.62:6789/0 conn(0x7fb930155430 legacy=0x7fb9301558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.403+0000 7fb93499b700 1 -- 172.21.15.31:0/43588117 >> v1:172.21.15.31:6789/0 conn(0x7fb930159c60 legacy=0x7fb930156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.403+0000 7fb93499b700 1 -- 172.21.15.31:0/43588117 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930157b40 con 0x7fb930159890 2023-10-06T05:20:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.403+0000 7fb93499b700 1 --2- 172.21.15.31:0/43588117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159890 0x7fb9300c89e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb92400aaf0 tx=0x7fb92400a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3f7e2c7bee0eb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.403+0000 7fb92f7fe700 1 -- 172.21.15.31:0/43588117 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c001470 con 0x7fb930159890 2023-10-06T05:20:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.403+0000 7fb92f7fe700 1 -- 172.21.15.31:0/43588117 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92400c660 con 0x7fb930159890 2023-10-06T05:20:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.403+0000 7fb92f7fe700 1 -- 172.21.15.31:0/43588117 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92400ca00 con 0x7fb930159890 2023-10-06T05:20:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.404+0000 7fb936bff700 1 -- 172.21.15.31:0/43588117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159890 msgr2=0x7fb9300c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.404+0000 7fb936bff700 1 --2- 172.21.15.31:0/43588117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159890 0x7fb9300c89e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb92400aaf0 tx=0x7fb92400a330 comp rx=0 tx=0).stop 2023-10-06T05:20:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.404+0000 7fb936bff700 1 -- 172.21.15.31:0/43588117 shutdown_connections 2023-10-06T05:20:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.404+0000 7fb936bff700 1 --2- 172.21.15.31:0/43588117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159890 0x7fb9300c89e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.404+0000 7fb936bff700 1 -- 172.21.15.31:0/43588117 >> 172.21.15.31:0/43588117 conn(0x7fb930067560 msgr2=0x7fb930067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.404+0000 7fb936bff700 1 -- 172.21.15.31:0/43588117 shutdown_connections 2023-10-06T05:20:33.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.404+0000 7fb936bff700 1 -- 172.21.15.31:0/43588117 wait complete. 2023-10-06T05:20:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.405+0000 7fb936bff700 1 Processor -- start 2023-10-06T05:20:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.406+0000 7fb936bff700 1 -- start start 2023-10-06T05:20:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.406+0000 7fb936bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb930155430 0x7fb93015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb936bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb930159890 0x7fb93016dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb936bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159c60 0x7fb9300c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb936bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb930150de0 con 0x7fb930155430 2023-10-06T05:20:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb936bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93016e200 con 0x7fb930159890 2023-10-06T05:20:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb936bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93015a8b0 con 0x7fb930159c60 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb93499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb930155430 0x7fb93015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb93499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb930155430 0x7fb93015b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54446/0 (socket says 172.21.15.31:54446) 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb93499b700 1 -- 172.21.15.31:0/1312510435 learned_addr learned my addr 172.21.15.31:0/1312510435 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb93519c700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159c60 0x7fb9300c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb92ffff700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb930159890 0x7fb93016dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb93499b700 1 -- 172.21.15.31:0/1312510435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159c60 msgr2=0x7fb9300c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb93499b700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159c60 0x7fb9300c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb93499b700 1 -- 172.21.15.31:0/1312510435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb930159890 msgr2=0x7fb93016dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb93499b700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb930159890 0x7fb93016dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.407+0000 7fb93499b700 1 -- 172.21.15.31:0/1312510435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92400a5d0 con 0x7fb930155430 2023-10-06T05:20:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.408+0000 7fb93499b700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb930155430 0x7fb93015b950 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb9240030c0 tx=0x7fb924019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.408+0000 7fb92cff9700 1 -- 172.21.15.31:0/1312510435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92400c7d0 con 0x7fb930155430 2023-10-06T05:20:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.408+0000 7fb92cff9700 1 -- 172.21.15.31:0/1312510435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92400c950 con 0x7fb930155430 2023-10-06T05:20:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.408+0000 7fb936bff700 1 -- 172.21.15.31:0/1312510435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93015ab90 con 0x7fb930155430 2023-10-06T05:20:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.408+0000 7fb92cff9700 1 -- 172.21.15.31:0/1312510435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240216a0 con 0x7fb930155430 2023-10-06T05:20:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.408+0000 7fb936bff700 1 -- 172.21.15.31:0/1312510435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93015b150 con 0x7fb930155430 2023-10-06T05:20:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.409+0000 7fb936bff700 1 -- 172.21.15.31:0/1312510435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93004c380 con 0x7fb930155430 2023-10-06T05:20:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.410+0000 7fb92cff9700 1 -- 172.21.15.31:0/1312510435 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fb9240103f0 con 0x7fb930155430 2023-10-06T05:20:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.411+0000 7fb92cff9700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb9140671b0 0x7fb914069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.411+0000 7fb92cff9700 1 -- 172.21.15.31:0/1312510435 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb924085b80 con 0x7fb930155430 2023-10-06T05:20:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.411+0000 7fb92ffff700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb9140671b0 0x7fb914069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.411+0000 7fb92ffff700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb9140671b0 0x7fb914069660 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0015a0 tx=0x7fb91c000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.415+0000 7fb92cff9700 1 -- 172.21.15.31:0/1312510435 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb924052d40 con 0x7fb930155430 2023-10-06T05:20:33.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.617+0000 7fb936bff700 1 -- 172.21.15.31:0/1312510435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb930157e20 con 0x7fb930155430 2023-10-06T05:20:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.618+0000 7fb92cff9700 1 -- 172.21.15.31:0/1312510435 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb9240260a0 con 0x7fb930155430 2023-10-06T05:20:33.620 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:20:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.620+0000 7fb936bff700 1 -- 172.21.15.31:0/1312510435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb9140671b0 msgr2=0x7fb914069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.620+0000 7fb936bff700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb9140671b0 0x7fb914069660 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0015a0 tx=0x7fb91c000bc0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.620+0000 7fb936bff700 1 -- 172.21.15.31:0/1312510435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb930155430 msgr2=0x7fb93015b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.620+0000 7fb936bff700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb930155430 0x7fb93015b950 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb9240030c0 tx=0x7fb924019000 comp rx=0 tx=0).stop 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.621+0000 7fb936bff700 1 -- 172.21.15.31:0/1312510435 shutdown_connections 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.621+0000 7fb936bff700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb930159c60 0x7fb9300c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.621+0000 7fb936bff700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb9140671b0 0x7fb914069660 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.621+0000 7fb936bff700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb930155430 0x7fb93015b950 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.621+0000 7fb936bff700 1 --2- 172.21.15.31:0/1312510435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb930159890 0x7fb93016dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.621+0000 7fb936bff700 1 -- 172.21.15.31:0/1312510435 >> 172.21.15.31:0/1312510435 conn(0x7fb930067560 msgr2=0x7fb93006e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.621+0000 7fb936bff700 1 -- 172.21.15.31:0/1312510435 shutdown_connections 2023-10-06T05:20:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.621+0000 7fb936bff700 1 -- 172.21.15.31:0/1312510435 wait complete. 2023-10-06T05:20:33.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:20:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.835+0000 7f8ba07ff700 1 Processor -- start 2023-10-06T05:20:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.835+0000 7f8ba07ff700 1 -- start start 2023-10-06T05:20:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.836+0000 7f8ba07ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b981573a0 0x7f8b980acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.836+0000 7f8ba07ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b980ae860 0x7f8b981611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8ba07ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b98158cf0 con 0x7f8b98157770 2023-10-06T05:20:33.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8ba07ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b981580e0 con 0x7f8b981573a0 2023-10-06T05:20:33.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8ba07ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b981583c0 con 0x7f8b980ae860 2023-10-06T05:20:33.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8b9e59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b981573a0 0x7f8b980acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8b9e59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b981573a0 0x7f8b980acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54450/0 (socket says 172.21.15.31:54450) 2023-10-06T05:20:33.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8b9e59b700 1 -- 172.21.15.31:0/1301311606 learned_addr learned my addr 172.21.15.31:0/1301311606 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:33.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8b9ed9c700 1 --2- 172.21.15.31:0/1301311606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b980ae860 0x7f8b981611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8b9e59b700 1 -- 172.21.15.31:0/1301311606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b980ae860 msgr2=0x7f8b981611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8b9e59b700 1 --2- 172.21.15.31:0/1301311606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b980ae860 0x7f8b981611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8b9e59b700 1 -- 172.21.15.31:0/1301311606 >> v1:172.21.15.31:6789/0 conn(0x7f8b98157770 legacy=0x7f8b980b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8b9e59b700 1 -- 172.21.15.31:0/1301311606 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b981586a0 con 0x7f8b981573a0 2023-10-06T05:20:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.837+0000 7f8b9d599700 1 -- 172.21.15.31:0/1301311606 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374173524 0 0) 0x7f8b98158cf0 con 0x7f8b98157770 2023-10-06T05:20:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.838+0000 7f8b9e59b700 1 --2- 172.21.15.31:0/1301311606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b981573a0 0x7f8b980acb70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8b940099b0 tx=0x7f8b94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c98fa1c0c3605ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.838+0000 7f8b9d599700 1 -- 172.21.15.31:0/1301311606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94009ea0 con 0x7f8b981573a0 2023-10-06T05:20:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.838+0000 7f8b9d599700 1 -- 172.21.15.31:0/1301311606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9400b860 con 0x7f8b981573a0 2023-10-06T05:20:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.838+0000 7f8b9d599700 1 -- 172.21.15.31:0/1301311606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9400bc00 con 0x7f8b981573a0 2023-10-06T05:20:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.838+0000 7f8ba07ff700 1 -- 172.21.15.31:0/1301311606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b981573a0 msgr2=0x7f8b980acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.838+0000 7f8ba07ff700 1 --2- 172.21.15.31:0/1301311606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b981573a0 0x7f8b980acb70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8b940099b0 tx=0x7f8b94009250 comp rx=0 tx=0).stop 2023-10-06T05:20:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.839+0000 7f8ba07ff700 1 -- 172.21.15.31:0/1301311606 shutdown_connections 2023-10-06T05:20:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.839+0000 7f8ba07ff700 1 --2- 172.21.15.31:0/1301311606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b981573a0 0x7f8b980acb70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.839+0000 7f8ba07ff700 1 --2- 172.21.15.31:0/1301311606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b980ae860 0x7f8b981611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.839+0000 7f8ba07ff700 1 -- 172.21.15.31:0/1301311606 >> 172.21.15.31:0/1301311606 conn(0x7f8b98067560 msgr2=0x7f8b98067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.839+0000 7f8ba07ff700 1 -- 172.21.15.31:0/1301311606 shutdown_connections 2023-10-06T05:20:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.839+0000 7f8ba07ff700 1 -- 172.21.15.31:0/1301311606 wait complete. 2023-10-06T05:20:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.840+0000 7f8ba07ff700 1 Processor -- start 2023-10-06T05:20:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.840+0000 7f8ba07ff700 1 -- start start 2023-10-06T05:20:33.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.841+0000 7f8ba07ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b981573a0 0x7f8b981565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.841+0000 7f8ba07ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98157770 0x7f8b98144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.841+0000 7f8ba07ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b9816aaf0 0x7f8b9816a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.841+0000 7f8ba07ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9816a890 con 0x7f8b98157770 2023-10-06T05:20:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.841+0000 7f8ba07ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980af0f0 con 0x7f8b981573a0 2023-10-06T05:20:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.841+0000 7f8ba07ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9815b180 con 0x7f8b9816aaf0 2023-10-06T05:20:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98157770 0x7f8b98144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98157770 0x7f8b98144cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54462/0 (socket says 172.21.15.31:54462) 2023-10-06T05:20:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9dd9a700 1 -- 172.21.15.31:0/2898039177 learned_addr learned my addr 172.21.15.31:0/2898039177 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9e59b700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b981573a0 0x7f8b981565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9ed9c700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b9816aaf0 0x7f8b9816a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9dd9a700 1 -- 172.21.15.31:0/2898039177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b9816aaf0 msgr2=0x7f8b9816a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9dd9a700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b9816aaf0 0x7f8b9816a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9dd9a700 1 -- 172.21.15.31:0/2898039177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b981573a0 msgr2=0x7f8b981565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9dd9a700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b981573a0 0x7f8b981565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9dd9a700 1 -- 172.21.15.31:0/2898039177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b940094f0 con 0x7f8b98157770 2023-10-06T05:20:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b9dd9a700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98157770 0x7f8b98144cf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800c530 tx=0x7f8b8800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.842+0000 7f8b8e7fc700 1 -- 172.21.15.31:0/2898039177 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88011590 con 0x7f8b98157770 2023-10-06T05:20:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.843+0000 7f8b8e7fc700 1 -- 172.21.15.31:0/2898039177 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b88011710 con 0x7f8b98157770 2023-10-06T05:20:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.843+0000 7f8ba07ff700 1 -- 172.21.15.31:0/2898039177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9815b460 con 0x7f8b98157770 2023-10-06T05:20:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.843+0000 7f8b8e7fc700 1 -- 172.21.15.31:0/2898039177 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88011ad0 con 0x7f8b98157770 2023-10-06T05:20:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.843+0000 7f8ba07ff700 1 -- 172.21.15.31:0/2898039177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9815b740 con 0x7f8b98157770 2023-10-06T05:20:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.844+0000 7f8ba07ff700 1 -- 172.21.15.31:0/2898039177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b980af5d0 con 0x7f8b98157770 2023-10-06T05:20:33.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.845+0000 7f8b8e7fc700 1 -- 172.21.15.31:0/2898039177 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8b8801b3b0 con 0x7f8b98157770 2023-10-06T05:20:33.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.846+0000 7f8b8e7fc700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b7c0670e0 0x7f8b7c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:33.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.846+0000 7f8b8e7fc700 1 -- 172.21.15.31:0/2898039177 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8b88083b80 con 0x7f8b98157770 2023-10-06T05:20:33.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.846+0000 7f8b9e59b700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b7c0670e0 0x7f8b7c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:33.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.847+0000 7f8b9e59b700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b7c0670e0 0x7f8b7c069590 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8b940099b0 tx=0x7f8b94017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:33.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:33.850+0000 7f8b8e7fc700 1 -- 172.21.15.31:0/2898039177 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b980af5d0 con 0x7f8b98157770 2023-10-06T05:20:34.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.053+0000 7f8ba07ff700 1 -- 172.21.15.31:0/2898039177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b980af5d0 con 0x7f8b98157770 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.053+0000 7f8b8e7fc700 1 -- 172.21.15.31:0/2898039177 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b980af5d0 con 0x7f8b98157770 2023-10-06T05:20:34.055 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:20:34.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 -- 172.21.15.31:0/2898039177 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b7c0670e0 msgr2=0x7f8b7c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b7c0670e0 0x7f8b7c069590 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8b940099b0 tx=0x7f8b94017000 comp rx=0 tx=0).stop 2023-10-06T05:20:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 -- 172.21.15.31:0/2898039177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98157770 msgr2=0x7f8b98144cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98157770 0x7f8b98144cf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800c530 tx=0x7f8b8800c840 comp rx=0 tx=0).stop 2023-10-06T05:20:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 -- 172.21.15.31:0/2898039177 shutdown_connections 2023-10-06T05:20:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b9816aaf0 0x7f8b9816a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b981573a0 0x7f8b981565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8b7c0670e0 0x7f8b7c069590 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 --2- 172.21.15.31:0/2898039177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98157770 0x7f8b98144cf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 -- 172.21.15.31:0/2898039177 >> 172.21.15.31:0/2898039177 conn(0x7f8b98067560 msgr2=0x7f8b980b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:34.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 -- 172.21.15.31:0/2898039177 shutdown_connections 2023-10-06T05:20:34.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.056+0000 7f8ba07ff700 1 -- 172.21.15.31:0/2898039177 wait complete. 2023-10-06T05:20:34.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:20:34.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.271+0000 7efee15ff700 1 Processor -- start 2023-10-06T05:20:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.271+0000 7efee15ff700 1 -- start start 2023-10-06T05:20:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.272+0000 7efee15ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efedc0c7f30 0x7efedc069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.272+0000 7efee15ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 0x7efedc13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.272+0000 7efee15ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efedc0ca5b0 con 0x7efedc0c7b60 2023-10-06T05:20:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.272+0000 7efee15ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc0af2f0 con 0x7efedc0c7f30 2023-10-06T05:20:34.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.272+0000 7efee15ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc06f740 con 0x7efedc0aee70 2023-10-06T05:20:34.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.273+0000 7efeda7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efedc0c7f30 0x7efedc069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.273+0000 7efeda7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efedc0c7f30 0x7efedc069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54472/0 (socket says 172.21.15.31:54472) 2023-10-06T05:20:34.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.273+0000 7efeda7fc700 1 -- 172.21.15.31:0/3919164317 learned_addr learned my addr 172.21.15.31:0/3919164317 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:34.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.273+0000 7efedaffd700 1 --1- 172.21.15.31:0/3919164317 >> v1:172.21.15.31:6789/0 conn(0x7efedc0c7b60 0x7efedc06ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52772/0 (socket says 172.21.15.31:52772) 2023-10-06T05:20:34.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.273+0000 7efedb7fe700 1 --2- 172.21.15.31:0/3919164317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 0x7efedc13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.273+0000 7efeda7fc700 1 -- 172.21.15.31:0/3919164317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 msgr2=0x7efedc13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:34.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.273+0000 7efeda7fc700 1 --2- 172.21.15.31:0/3919164317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 0x7efedc13f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.273+0000 7efeda7fc700 1 -- 172.21.15.31:0/3919164317 >> v1:172.21.15.31:6789/0 conn(0x7efedc0c7b60 legacy=0x7efedc06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.273+0000 7efeda7fc700 1 -- 172.21.15.31:0/3919164317 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc0c9c80 con 0x7efedc0c7f30 2023-10-06T05:20:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.273+0000 7efeda7fc700 1 --2- 172.21.15.31:0/3919164317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 0x7efedc069880 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efed00099b0 tx=0x7efed0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d3fdaa9418ffa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efed9ffb700 1 -- 172.21.15.31:0/3919164317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed000b860 con 0x7efedc0c7f30 2023-10-06T05:20:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efed9ffb700 1 -- 172.21.15.31:0/3919164317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed000b9c0 con 0x7efedc0c7f30 2023-10-06T05:20:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efed9ffb700 1 -- 172.21.15.31:0/3919164317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed000bd60 con 0x7efedc0c7f30 2023-10-06T05:20:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efee15ff700 1 -- 172.21.15.31:0/3919164317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 msgr2=0x7efedc069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efee15ff700 1 --2- 172.21.15.31:0/3919164317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 0x7efedc069880 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efed00099b0 tx=0x7efed0009250 comp rx=0 tx=0).stop 2023-10-06T05:20:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efee15ff700 1 -- 172.21.15.31:0/3919164317 shutdown_connections 2023-10-06T05:20:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efee15ff700 1 --2- 172.21.15.31:0/3919164317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 0x7efedc13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efee15ff700 1 --2- 172.21.15.31:0/3919164317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 0x7efedc069880 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efee15ff700 1 -- 172.21.15.31:0/3919164317 >> 172.21.15.31:0/3919164317 conn(0x7efedc0cc0b0 msgr2=0x7efedc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efee15ff700 1 -- 172.21.15.31:0/3919164317 shutdown_connections 2023-10-06T05:20:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.274+0000 7efee15ff700 1 -- 172.21.15.31:0/3919164317 wait complete. 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.276+0000 7efee15ff700 1 Processor -- start 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.276+0000 7efee15ff700 1 -- start start 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efee15ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 0x7efedc15b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efee15ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc0c7b60 0x7efedc169890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efee15ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 0x7efedc13e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efee15ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc13d190 con 0x7efedc0c7f30 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efee15ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc13d470 con 0x7efedc0c7b60 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efee15ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc13d750 con 0x7efedc0aee70 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efedb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 0x7efedc13e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efedb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 0x7efedc13e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54486/0 (socket says 172.21.15.31:54486) 2023-10-06T05:20:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efedb7fe700 1 -- 172.21.15.31:0/938922297 learned_addr learned my addr 172.21.15.31:0/938922297 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efedaffd700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 0x7efedc15b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efeda7fc700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc0c7b60 0x7efedc169890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efedb7fe700 1 -- 172.21.15.31:0/938922297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 msgr2=0x7efedc15b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efedb7fe700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 0x7efedc15b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efedb7fe700 1 -- 172.21.15.31:0/938922297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc0c7b60 msgr2=0x7efedc169890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.277+0000 7efedb7fe700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc0c7b60 0x7efedc169890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.278+0000 7efedb7fe700 1 -- 172.21.15.31:0/938922297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed00094f0 con 0x7efedc0c7f30 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.278+0000 7efedaffd700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 0x7efedc15b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.278+0000 7efedb7fe700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 0x7efedc13e210 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fa0 tx=0x7efed4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.278+0000 7efec2ffd700 1 -- 172.21.15.31:0/938922297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400ec70 con 0x7efedc0c7f30 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.278+0000 7efec2ffd700 1 -- 172.21.15.31:0/938922297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed400edd0 con 0x7efedc0c7f30 2023-10-06T05:20:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.278+0000 7efee15ff700 1 -- 172.21.15.31:0/938922297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc13da30 con 0x7efedc0c7f30 2023-10-06T05:20:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.278+0000 7efec2ffd700 1 -- 172.21.15.31:0/938922297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4014670 con 0x7efedc0c7f30 2023-10-06T05:20:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.278+0000 7efee15ff700 1 -- 172.21.15.31:0/938922297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc15b5a0 con 0x7efedc0c7f30 2023-10-06T05:20:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.279+0000 7efee15ff700 1 -- 172.21.15.31:0/938922297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc04c380 con 0x7efedc0c7f30 2023-10-06T05:20:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.280+0000 7efec2ffd700 1 -- 172.21.15.31:0/938922297 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7efed40149e0 con 0x7efedc0c7f30 2023-10-06T05:20:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.281+0000 7efec2ffd700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec80670e0 0x7efec8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.281+0000 7efec2ffd700 1 -- 172.21.15.31:0/938922297 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5206+0+0 (secure 0 0 0) 0x7efed4084e30 con 0x7efedc0c7f30 2023-10-06T05:20:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.281+0000 7efedaffd700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec80670e0 0x7efec8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.282+0000 7efedaffd700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec80670e0 0x7efec8069590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efecc0018c0 tx=0x7efecc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.285+0000 7efec2ffd700 1 -- 172.21.15.31:0/938922297 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efed4051f70 con 0x7efedc0c7f30 2023-10-06T05:20:34.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.494+0000 7efee15ff700 1 -- 172.21.15.31:0/938922297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efedc0c9f60 con 0x7efedc0c7f30 2023-10-06T05:20:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.494+0000 7efec2ffd700 1 -- 172.21.15.31:0/938922297 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) v1 ==== 84+0+126 (secure 0 0 0) 0x7efed4055590 con 0x7efedc0c7f30 2023-10-06T05:20:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:34.496 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:20:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:20:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:20:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:20:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:20:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:20:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:34.497 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:20:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 -- 172.21.15.31:0/938922297 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec80670e0 msgr2=0x7efec8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec80670e0 0x7efec8069590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efecc0018c0 tx=0x7efecc009040 comp rx=0 tx=0).stop 2023-10-06T05:20:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 -- 172.21.15.31:0/938922297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 msgr2=0x7efedc13e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 0x7efedc13e210 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fa0 tx=0x7efed4009900 comp rx=0 tx=0).stop 2023-10-06T05:20:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 -- 172.21.15.31:0/938922297 shutdown_connections 2023-10-06T05:20:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec80670e0 0x7efec8069590 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc0c7f30 0x7efedc13e210 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc0aee70 0x7efedc15b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 --2- 172.21.15.31:0/938922297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc0c7b60 0x7efedc169890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 -- 172.21.15.31:0/938922297 >> 172.21.15.31:0/938922297 conn(0x7efedc0cc0b0 msgr2=0x7efedc0b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 -- 172.21.15.31:0/938922297 shutdown_connections 2023-10-06T05:20:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.497+0000 7efee15ff700 1 -- 172.21.15.31:0/938922297 wait complete. 2023-10-06T05:20:34.516 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:20:34.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:20:34.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.714+0000 7f79901ff700 1 Processor -- start 2023-10-06T05:20:34.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.714+0000 7f79901ff700 1 -- start start 2023-10-06T05:20:34.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.715+0000 7f79901ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f79880cf7a0 0x7f798815a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.715+0000 7f79901ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79880cd050 con 0x7f7988159c70 2023-10-06T05:20:34.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.715+0000 7f79901ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79880ce020 con 0x7f798815a580 2023-10-06T05:20:34.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.715+0000 7f79901ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880cde90 con 0x7f79880cf7a0 2023-10-06T05:20:34.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.715+0000 7f798df9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7988159c70 0x7f79880caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52788/0 (socket says 172.21.15.31:52788) 2023-10-06T05:20:34.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.715+0000 7f798df9b700 1 -- 172.21.15.31:0/158475304 learned_addr learned my addr 172.21.15.31:0/158475304 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:34.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.715+0000 7f798d79a700 1 --2- 172.21.15.31:0/158475304 >> v2:172.21.15.62:3300/0 conn(0x7f79880cf7a0 0x7f798815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.716+0000 7f798cf99700 1 -- 172.21.15.31:0/158475304 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801806016 0 0) 0x7f79880cd050 con 0x7f7988159c70 2023-10-06T05:20:34.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.716+0000 7f798cf99700 1 -- 172.21.15.31:0/158475304 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7974003570 con 0x7f7988159c70 2023-10-06T05:20:34.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.716+0000 7f798cf99700 1 -- 172.21.15.31:0/158475304 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919577479 0 0) 0x7f79880ce020 con 0x7f798815a580 2023-10-06T05:20:34.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.716+0000 7f798cf99700 1 -- 172.21.15.31:0/158475304 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79880cd050 con 0x7f798815a580 2023-10-06T05:20:34.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.716+0000 7f798d79a700 1 -- 172.21.15.31:0/158475304 >> v1:172.21.15.62:6789/0 conn(0x7f798815a580 legacy=0x7f7988163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.716+0000 7f798d79a700 1 -- 172.21.15.31:0/158475304 >> v1:172.21.15.31:6789/0 conn(0x7f7988159c70 legacy=0x7f79880caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.716+0000 7f798d79a700 1 -- 172.21.15.31:0/158475304 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79880ce300 con 0x7f79880cf7a0 2023-10-06T05:20:34.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.716+0000 7f798d79a700 1 --2- 172.21.15.31:0/158475304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79880cf7a0 0x7f798815a040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f79780098e0 tx=0x7f79780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5117954602355203 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.716+0000 7f798cf99700 1 -- 172.21.15.31:0/158475304 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978011630 con 0x7f79880cf7a0 2023-10-06T05:20:34.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.717+0000 7f798cf99700 1 -- 172.21.15.31:0/158475304 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978011790 con 0x7f79880cf7a0 2023-10-06T05:20:34.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.717+0000 7f798cf99700 1 -- 172.21.15.31:0/158475304 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978011b30 con 0x7f79880cf7a0 2023-10-06T05:20:34.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.717+0000 7f79901ff700 1 -- 172.21.15.31:0/158475304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79880cf7a0 msgr2=0x7f798815a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.717+0000 7f79901ff700 1 --2- 172.21.15.31:0/158475304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79880cf7a0 0x7f798815a040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f79780098e0 tx=0x7f79780091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.717+0000 7f79901ff700 1 -- 172.21.15.31:0/158475304 shutdown_connections 2023-10-06T05:20:34.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.717+0000 7f79901ff700 1 --2- 172.21.15.31:0/158475304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79880cf7a0 0x7f798815a040 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.717+0000 7f79901ff700 1 -- 172.21.15.31:0/158475304 >> 172.21.15.31:0/158475304 conn(0x7f7988099710 msgr2=0x7f798806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:34.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.717+0000 7f79901ff700 1 -- 172.21.15.31:0/158475304 shutdown_connections 2023-10-06T05:20:34.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.717+0000 7f79901ff700 1 -- 172.21.15.31:0/158475304 wait complete. 2023-10-06T05:20:34.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.719+0000 7f79901ff700 1 Processor -- start 2023-10-06T05:20:34.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.719+0000 7f79901ff700 1 -- start start 2023-10-06T05:20:34.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f79901ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79880cf7a0 0x7f79881692f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f79901ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7988159c70 0x7f79881542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f79901ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f798815a580 0x7f79881547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f79901ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880b5d40 con 0x7f798815a580 2023-10-06T05:20:34.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f79901ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880b51c0 con 0x7f7988159c70 2023-10-06T05:20:34.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f79901ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79880b54a0 con 0x7f79880cf7a0 2023-10-06T05:20:34.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f798e79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f798815a580 0x7f79881547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f798df9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79880cf7a0 0x7f79881692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f798e79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f798815a580 0x7f79881547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54498/0 (socket says 172.21.15.31:54498) 2023-10-06T05:20:34.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f798e79c700 1 -- 172.21.15.31:0/2922401464 learned_addr learned my addr 172.21.15.31:0/2922401464 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:34.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f798d79a700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7988159c70 0x7f79881542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f798e79c700 1 -- 172.21.15.31:0/2922401464 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79880cf7a0 msgr2=0x7f79881692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f798e79c700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79880cf7a0 0x7f79881692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f798e79c700 1 -- 172.21.15.31:0/2922401464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7988159c70 msgr2=0x7f79881542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:34.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f798e79c700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7988159c70 0x7f79881542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:34.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.720+0000 7f798e79c700 1 -- 172.21.15.31:0/2922401464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7978009420 con 0x7f798815a580 2023-10-06T05:20:34.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.721+0000 7f798e79c700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f798815a580 0x7f79881547f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f798000c860 tx=0x7f7980002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.721+0000 7f797dffb700 1 -- 172.21.15.31:0/2922401464 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980002030 con 0x7f798815a580 2023-10-06T05:20:34.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.721+0000 7f797dffb700 1 -- 172.21.15.31:0/2922401464 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79800021b0 con 0x7f798815a580 2023-10-06T05:20:34.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.721+0000 7f797dffb700 1 -- 172.21.15.31:0/2922401464 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980012630 con 0x7f798815a580 2023-10-06T05:20:34.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.721+0000 7f79901ff700 1 -- 172.21.15.31:0/2922401464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79880b5780 con 0x7f798815a580 2023-10-06T05:20:34.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.721+0000 7f79901ff700 1 -- 172.21.15.31:0/2922401464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7988169830 con 0x7f798815a580 2023-10-06T05:20:34.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.722+0000 7f79901ff700 1 -- 172.21.15.31:0/2922401464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79880c7e90 con 0x7f798815a580 2023-10-06T05:20:34.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.723+0000 7f797dffb700 1 -- 172.21.15.31:0/2922401464 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7980012a20 con 0x7f798815a580 2023-10-06T05:20:34.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.724+0000 7f797dffb700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f796c0670e0 0x7f796c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:34.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.724+0000 7f797dffb700 1 -- 172.21.15.31:0/2922401464 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7980084600 con 0x7f798815a580 2023-10-06T05:20:34.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.724+0000 7f798df9b700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f796c0670e0 0x7f796c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:34.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.725+0000 7f798df9b700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f796c0670e0 0x7f796c069590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7984001230 tx=0x7f7984009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:34.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.728+0000 7f797dffb700 1 -- 172.21.15.31:0/2922401464 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79880c7e90 con 0x7f798815a580 2023-10-06T05:20:34.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:34.929+0000 7f79901ff700 1 -- 172.21.15.31:0/2922401464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f79880c7e90 con 0x7f798815a580 2023-10-06T05:20:35.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.237+0000 7f797dffb700 1 -- 172.21.15.31:0/2922401464 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v54) v1 ==== 138+0+0 (secure 0 0 0) 0x7f79880c7e90 con 0x7f798815a580 2023-10-06T05:20:35.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.241+0000 7f79901ff700 1 -- 172.21.15.31:0/2922401464 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f796c0670e0 msgr2=0x7f796c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:35.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.241+0000 7f79901ff700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f796c0670e0 0x7f796c069590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7984001230 tx=0x7f7984009040 comp rx=0 tx=0).stop 2023-10-06T05:20:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.241+0000 7f79901ff700 1 -- 172.21.15.31:0/2922401464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f798815a580 msgr2=0x7f79881547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.241+0000 7f79901ff700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f798815a580 0x7f79881547f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f798000c860 tx=0x7f7980002700 comp rx=0 tx=0).stop 2023-10-06T05:20:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.241+0000 7f79901ff700 1 -- 172.21.15.31:0/2922401464 shutdown_connections 2023-10-06T05:20:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.241+0000 7f79901ff700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f796c0670e0 0x7f796c069590 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.241+0000 7f79901ff700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f798815a580 0x7f79881547f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.241+0000 7f79901ff700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79880cf7a0 0x7f79881692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.241+0000 7f79901ff700 1 --2- 172.21.15.31:0/2922401464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7988159c70 0x7f79881542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.241+0000 7f79901ff700 1 -- 172.21.15.31:0/2922401464 >> 172.21.15.31:0/2922401464 conn(0x7f7988099710 msgr2=0x7f79880c5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.242+0000 7f79901ff700 1 -- 172.21.15.31:0/2922401464 shutdown_connections 2023-10-06T05:20:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.242+0000 7f79901ff700 1 -- 172.21.15.31:0/2922401464 wait complete. 2023-10-06T05:20:35.246 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:20:35.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:20:35.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.465+0000 7f8242bff700 1 Processor -- start 2023-10-06T05:20:35.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.466+0000 7f8242bff700 1 -- start start 2023-10-06T05:20:35.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.467+0000 7f8242bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f823c0c7190 0x7f823c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:35.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.467+0000 7f8242bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823c068250 con 0x7f823c0aef00 2023-10-06T05:20:35.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.467+0000 7f8242bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823c06ac80 con 0x7f823c0c7560 2023-10-06T05:20:35.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.467+0000 7f8242bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c067f20 con 0x7f823c0c7190 2023-10-06T05:20:35.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.467+0000 7f824099b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f823c0c7190 0x7f823c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:35.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.467+0000 7f824099b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f823c0c7190 0x7f823c0c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54508/0 (socket says 172.21.15.31:54508) 2023-10-06T05:20:35.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.467+0000 7f824099b700 1 -- 172.21.15.31:0/3073896007 learned_addr learned my addr 172.21.15.31:0/3073896007 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.468+0000 7f824099b700 1 -- 172.21.15.31:0/3073896007 >> v1:172.21.15.62:6789/0 conn(0x7f823c0c7560 legacy=0x7f823c0686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.468+0000 7f824099b700 1 -- 172.21.15.31:0/3073896007 >> v1:172.21.15.31:6789/0 conn(0x7f823c0aef00 legacy=0x7f823c0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.468+0000 7f824099b700 1 -- 172.21.15.31:0/3073896007 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c06aed0 con 0x7f823c0c7190 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.468+0000 7f823b7fe700 1 -- 172.21.15.31:0/3073896007 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901239244 0 0) 0x7f823c068250 con 0x7f823c0aef00 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.468+0000 7f824099b700 1 --2- 172.21.15.31:0/3073896007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7190 0x7f823c0c9d40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f82300099b0 tx=0x7f8230009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392832d817a78f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.468+0000 7f823b7fe700 1 -- 172.21.15.31:0/3073896007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000b860 con 0x7f823c0c7190 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.468+0000 7f823b7fe700 1 -- 172.21.15.31:0/3073896007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823000b9c0 con 0x7f823c0c7190 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.468+0000 7f823b7fe700 1 -- 172.21.15.31:0/3073896007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000bd60 con 0x7f823c0c7190 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.469+0000 7f8242bff700 1 -- 172.21.15.31:0/3073896007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7190 msgr2=0x7f823c0c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.469+0000 7f8242bff700 1 --2- 172.21.15.31:0/3073896007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7190 0x7f823c0c9d40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f82300099b0 tx=0x7f8230009250 comp rx=0 tx=0).stop 2023-10-06T05:20:35.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.469+0000 7f8242bff700 1 -- 172.21.15.31:0/3073896007 shutdown_connections 2023-10-06T05:20:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.469+0000 7f8242bff700 1 --2- 172.21.15.31:0/3073896007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7190 0x7f823c0c9d40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.469+0000 7f8242bff700 1 -- 172.21.15.31:0/3073896007 >> 172.21.15.31:0/3073896007 conn(0x7f823c0cd3d0 msgr2=0x7f823c0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.469+0000 7f8242bff700 1 -- 172.21.15.31:0/3073896007 shutdown_connections 2023-10-06T05:20:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.469+0000 7f8242bff700 1 -- 172.21.15.31:0/3073896007 wait complete. 2023-10-06T05:20:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.471+0000 7f8242bff700 1 Processor -- start 2023-10-06T05:20:35.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.471+0000 7f8242bff700 1 -- start start 2023-10-06T05:20:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.472+0000 7f8242bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c0aef00 0x7f823c157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f8242bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f823c0c7190 0x7f823c0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f8242bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7560 0x7f823c15efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f8242bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0c3dc0 con 0x7f823c0c7560 2023-10-06T05:20:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f8242bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0c5fc0 con 0x7f823c0aef00 2023-10-06T05:20:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f8242bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0c5a00 con 0x7f823c0c7190 2023-10-06T05:20:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f824119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7560 0x7f823c15efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f824099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c0aef00 0x7f823c157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f824119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7560 0x7f823c15efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54510/0 (socket says 172.21.15.31:54510) 2023-10-06T05:20:35.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f824119c700 1 -- 172.21.15.31:0/921659792 learned_addr learned my addr 172.21.15.31:0/921659792 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:35.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f823bfff700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f823c0c7190 0x7f823c0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:35.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f824119c700 1 -- 172.21.15.31:0/921659792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f823c0c7190 msgr2=0x7f823c0c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:35.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f824119c700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f823c0c7190 0x7f823c0c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:35.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f824119c700 1 -- 172.21.15.31:0/921659792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c0aef00 msgr2=0x7f823c157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:35.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f824119c700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c0aef00 0x7f823c157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:35.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.473+0000 7f824119c700 1 -- 172.21.15.31:0/921659792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82300094f0 con 0x7f823c0c7560 2023-10-06T05:20:35.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.474+0000 7f824119c700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7560 0x7f823c15efc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f823400c840 tx=0x7f8234001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:35.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.474+0000 7f8238ff9700 1 -- 172.21.15.31:0/921659792 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234012630 con 0x7f823c0c7560 2023-10-06T05:20:35.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.474+0000 7f8238ff9700 1 -- 172.21.15.31:0/921659792 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82340127b0 con 0x7f823c0c7560 2023-10-06T05:20:35.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.474+0000 7f8238ff9700 1 -- 172.21.15.31:0/921659792 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234012b70 con 0x7f823c0c7560 2023-10-06T05:20:35.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.474+0000 7f8242bff700 1 -- 172.21.15.31:0/921659792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c0c5ce0 con 0x7f823c0c7560 2023-10-06T05:20:35.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.474+0000 7f8242bff700 1 -- 172.21.15.31:0/921659792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c0c6830 con 0x7f823c0c7560 2023-10-06T05:20:35.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.475+0000 7f8242bff700 1 -- 172.21.15.31:0/921659792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823c0c3f70 con 0x7f823c0c7560 2023-10-06T05:20:35.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.476+0000 7f8238ff9700 1 -- 172.21.15.31:0/921659792 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f823401a3a0 con 0x7f823c0c7560 2023-10-06T05:20:35.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.478+0000 7f8238ff9700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f82200670e0 0x7f8220069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:35.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.478+0000 7f8238ff9700 1 -- 172.21.15.31:0/921659792 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f8234084a20 con 0x7f823c0c7560 2023-10-06T05:20:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.478+0000 7f824099b700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f82200670e0 0x7f8220069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:35.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.478+0000 7f824099b700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f82200670e0 0x7f8220069590 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f82300099b0 tx=0x7f8230017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:35.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.483+0000 7f8238ff9700 1 -- 172.21.15.31:0/921659792 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f823c0c3f70 con 0x7f823c0c7560 2023-10-06T05:20:35.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:35.689+0000 7f8242bff700 1 -- 172.21.15.31:0/921659792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f823c04c380 con 0x7f823c0c7560 2023-10-06T05:20:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.240+0000 7f8238ff9700 1 -- 172.21.15.31:0/921659792 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v55) v1 ==== 170+0+0 (secure 0 0 0) 0x7f82340519f0 con 0x7f823c0c7560 2023-10-06T05:20:36.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.249+0000 7f8242bff700 1 -- 172.21.15.31:0/921659792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f82200670e0 msgr2=0x7f8220069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.249+0000 7f8242bff700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f82200670e0 0x7f8220069590 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f82300099b0 tx=0x7f8230017000 comp rx=0 tx=0).stop 2023-10-06T05:20:36.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.249+0000 7f8242bff700 1 -- 172.21.15.31:0/921659792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7560 msgr2=0x7f823c15efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.249+0000 7f8242bff700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7560 0x7f823c15efc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f823400c840 tx=0x7f8234001670 comp rx=0 tx=0).stop 2023-10-06T05:20:36.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.250+0000 7f8242bff700 1 -- 172.21.15.31:0/921659792 shutdown_connections 2023-10-06T05:20:36.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.250+0000 7f8242bff700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f82200670e0 0x7f8220069590 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.250+0000 7f8242bff700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f823c0c7560 0x7f823c15efc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.250+0000 7f8242bff700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c0aef00 0x7f823c157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.250+0000 7f8242bff700 1 --2- 172.21.15.31:0/921659792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f823c0c7190 0x7f823c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.250+0000 7f8242bff700 1 -- 172.21.15.31:0/921659792 >> 172.21.15.31:0/921659792 conn(0x7f823c0cd3d0 msgr2=0x7f823c158850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.251+0000 7f8242bff700 1 -- 172.21.15.31:0/921659792 shutdown_connections 2023-10-06T05:20:36.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.251+0000 7f8242bff700 1 -- 172.21.15.31:0/921659792 wait complete. 2023-10-06T05:20:36.259 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:20:36.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:20:36.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.479+0000 7f9a5a1ff700 1 Processor -- start 2023-10-06T05:20:36.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.479+0000 7f9a5a1ff700 1 -- start start 2023-10-06T05:20:36.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.480+0000 7f9a5a1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159930 0x7f9a540b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.480+0000 7f9a5a1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5415d2d0 con 0x7f9a540cd580 2023-10-06T05:20:36.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.481+0000 7f9a5a1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5415d5b0 con 0x7f9a54159560 2023-10-06T05:20:36.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.481+0000 7f9a5a1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5415d890 con 0x7f9a54159930 2023-10-06T05:20:36.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.481+0000 7f9a53fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9a540cd580 0x7f9a540cb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52806/0 (socket says 172.21.15.31:52806) 2023-10-06T05:20:36.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.481+0000 7f9a53fff700 1 -- 172.21.15.31:0/445700670 learned_addr learned my addr 172.21.15.31:0/445700670 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:36.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.481+0000 7f9a52ffd700 1 --2- 172.21.15.31:0/445700670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159930 0x7f9a540b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.481+0000 7f9a527fc700 1 -- 172.21.15.31:0/445700670 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453355633 0 0) 0x7f9a5415d2d0 con 0x7f9a540cd580 2023-10-06T05:20:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.481+0000 7f9a527fc700 1 -- 172.21.15.31:0/445700670 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a3c003570 con 0x7f9a540cd580 2023-10-06T05:20:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.481+0000 7f9a527fc700 1 -- 172.21.15.31:0/445700670 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665732101 0 0) 0x7f9a5415d5b0 con 0x7f9a54159560 2023-10-06T05:20:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.481+0000 7f9a527fc700 1 -- 172.21.15.31:0/445700670 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a5415d2d0 con 0x7f9a54159560 2023-10-06T05:20:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.482+0000 7f9a52ffd700 1 -- 172.21.15.31:0/445700670 >> v1:172.21.15.62:6789/0 conn(0x7f9a54159560 legacy=0x7f9a540cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.482+0000 7f9a52ffd700 1 -- 172.21.15.31:0/445700670 >> v1:172.21.15.31:6789/0 conn(0x7f9a540cd580 legacy=0x7f9a540cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.482+0000 7f9a527fc700 1 -- 172.21.15.31:0/445700670 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 739123515 0 0) 0x7f9a3c003570 con 0x7f9a540cd580 2023-10-06T05:20:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.482+0000 7f9a52ffd700 1 -- 172.21.15.31:0/445700670 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5415db70 con 0x7f9a54159930 2023-10-06T05:20:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.482+0000 7f9a52ffd700 1 --2- 172.21.15.31:0/445700670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159930 0x7f9a540b2470 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009910 tx=0x7f9a480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c14146a8c409fced server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.482+0000 7f9a527fc700 1 -- 172.21.15.31:0/445700670 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0017f0 con 0x7f9a54159930 2023-10-06T05:20:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.482+0000 7f9a527fc700 1 -- 172.21.15.31:0/445700670 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a48011420 con 0x7f9a54159930 2023-10-06T05:20:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.482+0000 7f9a527fc700 1 -- 172.21.15.31:0/445700670 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a480117c0 con 0x7f9a54159930 2023-10-06T05:20:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.483+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/445700670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159930 msgr2=0x7f9a540b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.483+0000 7f9a5a1ff700 1 --2- 172.21.15.31:0/445700670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159930 0x7f9a540b2470 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009910 tx=0x7f9a480091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.483+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/445700670 shutdown_connections 2023-10-06T05:20:36.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.483+0000 7f9a5a1ff700 1 --2- 172.21.15.31:0/445700670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159930 0x7f9a540b2470 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.483+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/445700670 >> 172.21.15.31:0/445700670 conn(0x7f9a5406ec90 msgr2=0x7f9a5406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:36.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.483+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/445700670 shutdown_connections 2023-10-06T05:20:36.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.483+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/445700670 wait complete. 2023-10-06T05:20:36.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.484+0000 7f9a5a1ff700 1 Processor -- start 2023-10-06T05:20:36.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.485+0000 7f9a5a1ff700 1 -- start start 2023-10-06T05:20:36.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.485+0000 7f9a5a1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a540cd580 0x7f9a5414fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a5a1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159560 0x7f9a540cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a5a1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a54159930 0x7f9a540ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a5a1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a540c88c0 con 0x7f9a540cd580 2023-10-06T05:20:36.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a5a1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a540c5690 con 0x7f9a54159930 2023-10-06T05:20:36.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a5a1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a540c4df0 con 0x7f9a54159560 2023-10-06T05:20:36.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a540cd580 0x7f9a5414fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a540cd580 0x7f9a5414fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54512/0 (socket says 172.21.15.31:54512) 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a537fe700 1 -- 172.21.15.31:0/1721936064 learned_addr learned my addr 172.21.15.31:0/1721936064 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a52ffd700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159560 0x7f9a540cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a53fff700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a54159930 0x7f9a540ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a537fe700 1 -- 172.21.15.31:0/1721936064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159560 msgr2=0x7f9a540cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a537fe700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159560 0x7f9a540cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a537fe700 1 -- 172.21.15.31:0/1721936064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a54159930 msgr2=0x7f9a540ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a537fe700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a54159930 0x7f9a540ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.486+0000 7f9a537fe700 1 -- 172.21.15.31:0/1721936064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a48009420 con 0x7f9a540cd580 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.487+0000 7f9a537fe700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a540cd580 0x7f9a5414fcc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400c790 tx=0x7f9a44007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.487+0000 7f9a3b7fe700 1 -- 172.21.15.31:0/1721936064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44010510 con 0x7f9a540cd580 2023-10-06T05:20:36.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.487+0000 7f9a3b7fe700 1 -- 172.21.15.31:0/1721936064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a44010670 con 0x7f9a540cd580 2023-10-06T05:20:36.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.487+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/1721936064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a540c50d0 con 0x7f9a540cd580 2023-10-06T05:20:36.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.487+0000 7f9a3b7fe700 1 -- 172.21.15.31:0/1721936064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44010a10 con 0x7f9a540cd580 2023-10-06T05:20:36.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.487+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/1721936064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a540c8af0 con 0x7f9a540cd580 2023-10-06T05:20:36.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.488+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/1721936064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a540c8ce0 con 0x7f9a540cd580 2023-10-06T05:20:36.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.489+0000 7f9a3b7fe700 1 -- 172.21.15.31:0/1721936064 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9a4401b3b0 con 0x7f9a540cd580 2023-10-06T05:20:36.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.490+0000 7f9a3b7fe700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a400670e0 0x7f9a40069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:36.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.490+0000 7f9a3b7fe700 1 -- 172.21.15.31:0/1721936064 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f9a44084280 con 0x7f9a540cd580 2023-10-06T05:20:36.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.490+0000 7f9a52ffd700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a400670e0 0x7f9a40069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:36.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.491+0000 7f9a52ffd700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a400670e0 0x7f9a40069590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a480030c0 tx=0x7f9a48016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:36.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.495+0000 7f9a3b7fe700 1 -- 172.21.15.31:0/1721936064 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a440510d0 con 0x7f9a540cd580 2023-10-06T05:20:36.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:36.740+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/1721936064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9a5404c380 con 0x7f9a540cd580 2023-10-06T05:20:37.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.254+0000 7f9a3b7fe700 1 -- 172.21.15.31:0/1721936064 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) v1 ==== 173+0+0 (secure 0 0 0) 0x7f9a440546f0 con 0x7f9a540cd580 2023-10-06T05:20:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.257+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/1721936064 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a400670e0 msgr2=0x7f9a40069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.257+0000 7f9a5a1ff700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a400670e0 0x7f9a40069590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a480030c0 tx=0x7f9a48016000 comp rx=0 tx=0).stop 2023-10-06T05:20:37.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.257+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/1721936064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a540cd580 msgr2=0x7f9a5414fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.257+0000 7f9a5a1ff700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a540cd580 0x7f9a5414fcc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400c790 tx=0x7f9a44007e10 comp rx=0 tx=0).stop 2023-10-06T05:20:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.257+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/1721936064 shutdown_connections 2023-10-06T05:20:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.257+0000 7f9a5a1ff700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a54159930 0x7f9a540ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.257+0000 7f9a5a1ff700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a400670e0 0x7f9a40069590 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.257+0000 7f9a5a1ff700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a540cd580 0x7f9a5414fcc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.257+0000 7f9a5a1ff700 1 --2- 172.21.15.31:0/1721936064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a54159560 0x7f9a540cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.257+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/1721936064 >> 172.21.15.31:0/1721936064 conn(0x7f9a5406ec90 msgr2=0x7f9a54158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.258+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/1721936064 shutdown_connections 2023-10-06T05:20:37.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.258+0000 7f9a5a1ff700 1 -- 172.21.15.31:0/1721936064 wait complete. 2023-10-06T05:20:37.262 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 9 and data pool 10 2023-10-06T05:20:37.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.478+0000 7f9a81dff700 1 Processor -- start 2023-10-06T05:20:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.478+0000 7f9a81dff700 1 -- start start 2023-10-06T05:20:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.479+0000 7f9a81dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a7c159500 0x7f9a7c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.479+0000 7f9a81dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9a7c15d2d0 0x7f9a7c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.479+0000 7f9a81dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7c0c3f40 con 0x7f9a7c1598d0 2023-10-06T05:20:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.479+0000 7f9a81dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7c0b2860 con 0x7f9a7c159500 2023-10-06T05:20:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.479+0000 7f9a81dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7c15d7d0 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.479+0000 7f9a7b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a7c159500 0x7f9a7c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.479+0000 7f9a7b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a7c159500 0x7f9a7c0b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45348/0 (socket says 172.21.15.31:45348) 2023-10-06T05:20:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.479+0000 7f9a7b7fe700 1 -- 172.21.15.31:0/144301143 learned_addr learned my addr 172.21.15.31:0/144301143 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.480+0000 7f9a7affd700 1 --2- 172.21.15.31:0/144301143 >> v2:172.21.15.62:3300/0 conn(0x7f9a7c15d2d0 0x7f9a7c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.480+0000 7f9a7b7fe700 1 -- 172.21.15.31:0/144301143 >> v2:172.21.15.62:3300/0 conn(0x7f9a7c15d2d0 msgr2=0x7f9a7c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.480+0000 7f9a7b7fe700 1 --2- 172.21.15.31:0/144301143 >> v2:172.21.15.62:3300/0 conn(0x7f9a7c15d2d0 0x7f9a7c0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.480+0000 7f9a7b7fe700 1 -- 172.21.15.31:0/144301143 >> v1:172.21.15.62:6789/0 conn(0x7f9a7c1598d0 legacy=0x7f9a7c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.480+0000 7f9a7b7fe700 1 -- 172.21.15.31:0/144301143 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c0cb000 con 0x7f9a7c159500 2023-10-06T05:20:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.480+0000 7f9a7b7fe700 1 --2- 172.21.15.31:0/144301143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c159500 0x7f9a7c0b0ab0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c15d200 tx=0x7f9a6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108e0b99f8ff7b26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.480+0000 7f9a7a7fc700 1 -- 172.21.15.31:0/144301143 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00b820 con 0x7f9a7c159500 2023-10-06T05:20:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.480+0000 7f9a7a7fc700 1 -- 172.21.15.31:0/144301143 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c00b980 con 0x7f9a7c159500 2023-10-06T05:20:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.480+0000 7f9a7a7fc700 1 -- 172.21.15.31:0/144301143 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00bd20 con 0x7f9a7c159500 2023-10-06T05:20:37.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.481+0000 7f9a81dff700 1 -- 172.21.15.31:0/144301143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c159500 msgr2=0x7f9a7c0b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.481+0000 7f9a81dff700 1 --2- 172.21.15.31:0/144301143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c159500 0x7f9a7c0b0ab0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c15d200 tx=0x7f9a6c009250 comp rx=0 tx=0).stop 2023-10-06T05:20:37.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.481+0000 7f9a81dff700 1 -- 172.21.15.31:0/144301143 shutdown_connections 2023-10-06T05:20:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.481+0000 7f9a81dff700 1 --2- 172.21.15.31:0/144301143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c159500 0x7f9a7c0b0ab0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.481+0000 7f9a81dff700 1 --2- 172.21.15.31:0/144301143 >> v2:172.21.15.62:3300/0 conn(0x7f9a7c15d2d0 0x7f9a7c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.481+0000 7f9a81dff700 1 -- 172.21.15.31:0/144301143 >> 172.21.15.31:0/144301143 conn(0x7f9a7c06ec90 msgr2=0x7f9a7c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.481+0000 7f9a81dff700 1 -- 172.21.15.31:0/144301143 shutdown_connections 2023-10-06T05:20:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.481+0000 7f9a81dff700 1 -- 172.21.15.31:0/144301143 wait complete. 2023-10-06T05:20:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.483+0000 7f9a81dff700 1 Processor -- start 2023-10-06T05:20:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.483+0000 7f9a81dff700 1 -- start start 2023-10-06T05:20:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a81dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a7c159500 0x7f9a7c0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a81dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a7c1598d0 0x7f9a7c157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a81dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c15d2d0 0x7f9a7c0b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a81dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c15b900 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a81dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0c8a50 con 0x7f9a7c159500 2023-10-06T05:20:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a81dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0b4d20 con 0x7f9a7c1598d0 2023-10-06T05:20:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a7bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c15d2d0 0x7f9a7c0b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a7bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c15d2d0 0x7f9a7c0b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45350/0 (socket says 172.21.15.31:45350) 2023-10-06T05:20:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a7bfff700 1 -- 172.21.15.31:0/4019230296 learned_addr learned my addr 172.21.15.31:0/4019230296 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a7b7fe700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a7c159500 0x7f9a7c0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a7affd700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a7c1598d0 0x7f9a7c157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a7bfff700 1 -- 172.21.15.31:0/4019230296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a7c1598d0 msgr2=0x7f9a7c157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a7bfff700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a7c1598d0 0x7f9a7c157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a7bfff700 1 -- 172.21.15.31:0/4019230296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a7c159500 msgr2=0x7f9a7c0c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a7bfff700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a7c159500 0x7f9a7c0c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.484+0000 7f9a7bfff700 1 -- 172.21.15.31:0/4019230296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c0094f0 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.485+0000 7f9a7bfff700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c15d2d0 0x7f9a7c0b4260 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400c930 tx=0x7f9a740026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.485+0000 7f9a637fe700 1 -- 172.21.15.31:0/4019230296 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a74012630 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.485+0000 7f9a637fe700 1 -- 172.21.15.31:0/4019230296 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a740127d0 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.485+0000 7f9a81dff700 1 -- 172.21.15.31:0/4019230296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7c0b5000 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.485+0000 7f9a637fe700 1 -- 172.21.15.31:0/4019230296 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a74012b90 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.485+0000 7f9a81dff700 1 -- 172.21.15.31:0/4019230296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c15bb50 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.486+0000 7f9a81dff700 1 -- 172.21.15.31:0/4019230296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7c15bd70 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.487+0000 7f9a637fe700 1 -- 172.21.15.31:0/4019230296 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9a7401a3b0 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.488+0000 7f9a637fe700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a680670e0 0x7f9a68069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.488+0000 7f9a637fe700 1 -- 172.21.15.31:0/4019230296 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9a740846f0 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.489+0000 7f9a7b7fe700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a680670e0 0x7f9a68069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.489+0000 7f9a7b7fe700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a680670e0 0x7f9a68069590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c15d200 tx=0x7f9a6c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.493+0000 7f9a637fe700 1 -- 172.21.15.31:0/4019230296 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a74054f70 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.695+0000 7f9a81dff700 1 -- 172.21.15.31:0/4019230296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a7c04c380 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.696+0000 7f9a637fe700 1 -- 172.21.15.31:0/4019230296 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16929 (secure 0 0 0) 0x7f9a74054b00 con 0x7f9a7c15d2d0 2023-10-06T05:20:37.699 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:37.700 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":56,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:20:36.742391+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:34.970939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:35.729829+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 -- 172.21.15.31:0/4019230296 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a680670e0 msgr2=0x7f9a68069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a680670e0 0x7f9a68069590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c15d200 tx=0x7f9a6c016000 comp rx=0 tx=0).stop 2023-10-06T05:20:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 -- 172.21.15.31:0/4019230296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c15d2d0 msgr2=0x7f9a7c0b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c15d2d0 0x7f9a7c0b4260 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400c930 tx=0x7f9a740026e0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 -- 172.21.15.31:0/4019230296 shutdown_connections 2023-10-06T05:20:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9a680670e0 0x7f9a68069590 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a7c15d2d0 0x7f9a7c0b4260 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a7c159500 0x7f9a7c0c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 --2- 172.21.15.31:0/4019230296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a7c1598d0 0x7f9a7c157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 -- 172.21.15.31:0/4019230296 >> 172.21.15.31:0/4019230296 conn(0x7f9a7c06ec90 msgr2=0x7f9a7c0c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 -- 172.21.15.31:0/4019230296 shutdown_connections 2023-10-06T05:20:37.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.699+0000 7f9a81dff700 1 -- 172.21.15.31:0/4019230296 wait complete. 2023-10-06T05:20:37.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:37.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.911+0000 7f17cc3ff700 1 Processor -- start 2023-10-06T05:20:37.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.911+0000 7f17cc3ff700 1 -- start start 2023-10-06T05:20:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.913+0000 7f17cc3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f17c415a680 0x7f17c40cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.913+0000 7f17cc3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f17c413fec0 0x7f17c40cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.913+0000 7f17cc3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c40c7d20 con 0x7f17c413faf0 2023-10-06T05:20:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.913+0000 7f17cc3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c40cc3e0 con 0x7f17c415a680 2023-10-06T05:20:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.913+0000 7f17cc3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c40cc6c0 con 0x7f17c413fec0 2023-10-06T05:20:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.913+0000 7f17c999a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f17c415a680 0x7f17c40cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.913+0000 7f17c999a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f17c415a680 0x7f17c40cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45366/0 (socket says 172.21.15.31:45366) 2023-10-06T05:20:37.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.913+0000 7f17c999a700 1 -- 172.21.15.31:0/3597340553 learned_addr learned my addr 172.21.15.31:0/3597340553 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:37.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.913+0000 7f17ca19b700 1 --1- 172.21.15.31:0/3597340553 >> v1:172.21.15.31:6789/0 conn(0x7f17c413faf0 0x7f17c4159f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42030/0 (socket says 172.21.15.31:42030) 2023-10-06T05:20:37.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.913+0000 7f17ca99c700 1 --2- 172.21.15.31:0/3597340553 >> v2:172.21.15.62:3300/0 conn(0x7f17c413fec0 0x7f17c40cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.914+0000 7f17c999a700 1 -- 172.21.15.31:0/3597340553 >> v2:172.21.15.62:3300/0 conn(0x7f17c413fec0 msgr2=0x7f17c40cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:37.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.914+0000 7f17c999a700 1 --2- 172.21.15.31:0/3597340553 >> v2:172.21.15.62:3300/0 conn(0x7f17c413fec0 0x7f17c40cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.914+0000 7f17c999a700 1 -- 172.21.15.31:0/3597340553 >> v1:172.21.15.31:6789/0 conn(0x7f17c413faf0 legacy=0x7f17c4159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.914+0000 7f17c999a700 1 -- 172.21.15.31:0/3597340553 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c40c79b0 con 0x7f17c415a680 2023-10-06T05:20:37.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.914+0000 7f17c999a700 1 --2- 172.21.15.31:0/3597340553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c415a680 0x7f17c40cb7b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f17c40cac20 tx=0x7f17b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cbd2ec11da622f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.914+0000 7f17c9199700 1 -- 172.21.15.31:0/3597340553 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4009e70 con 0x7f17c415a680 2023-10-06T05:20:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.914+0000 7f17c9199700 1 -- 172.21.15.31:0/3597340553 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b400b860 con 0x7f17c415a680 2023-10-06T05:20:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.914+0000 7f17c9199700 1 -- 172.21.15.31:0/3597340553 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b400bc00 con 0x7f17c415a680 2023-10-06T05:20:37.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.915+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3597340553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c415a680 msgr2=0x7f17c40cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.915+0000 7f17cc3ff700 1 --2- 172.21.15.31:0/3597340553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c415a680 0x7f17c40cb7b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f17c40cac20 tx=0x7f17b4009250 comp rx=0 tx=0).stop 2023-10-06T05:20:37.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.915+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3597340553 shutdown_connections 2023-10-06T05:20:37.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.915+0000 7f17cc3ff700 1 --2- 172.21.15.31:0/3597340553 >> v2:172.21.15.62:3300/0 conn(0x7f17c413fec0 0x7f17c40cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.915+0000 7f17cc3ff700 1 --2- 172.21.15.31:0/3597340553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c415a680 0x7f17c40cb7b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.915+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3597340553 >> 172.21.15.31:0/3597340553 conn(0x7f17c40cdc50 msgr2=0x7f17c40c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.915+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3597340553 shutdown_connections 2023-10-06T05:20:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.915+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3597340553 wait complete. 2023-10-06T05:20:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.916+0000 7f17cc3ff700 1 Processor -- start 2023-10-06T05:20:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.917+0000 7f17cc3ff700 1 -- start start 2023-10-06T05:20:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17cc3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413faf0 0x7f17c40b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17cc3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fec0 0x7f17c40c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17cc3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17c415a680 0x7f17c40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17cc3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40b2d00 con 0x7f17c413fec0 2023-10-06T05:20:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17cc3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40b1e00 con 0x7f17c413faf0 2023-10-06T05:20:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17cc3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40b2080 con 0x7f17c415a680 2023-10-06T05:20:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17c999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fec0 0x7f17c40c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17c999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fec0 0x7f17c40c24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45380/0 (socket says 172.21.15.31:45380) 2023-10-06T05:20:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17c999a700 1 -- 172.21.15.31:0/3280410421 learned_addr learned my addr 172.21.15.31:0/3280410421 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17ca19b700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413faf0 0x7f17c40b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17ca99c700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17c415a680 0x7f17c40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17c999a700 1 -- 172.21.15.31:0/3280410421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17c415a680 msgr2=0x7f17c40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17c999a700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17c415a680 0x7f17c40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17c999a700 1 -- 172.21.15.31:0/3280410421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413faf0 msgr2=0x7f17c40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:37.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17c999a700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413faf0 0x7f17c40b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:37.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.918+0000 7f17c999a700 1 -- 172.21.15.31:0/3280410421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b40094f0 con 0x7f17c413fec0 2023-10-06T05:20:37.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.919+0000 7f17c999a700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fec0 0x7f17c40c24a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f17c406db10 tx=0x7f17b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.919+0000 7f17ba7fc700 1 -- 172.21.15.31:0/3280410421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b400bbe0 con 0x7f17c413fec0 2023-10-06T05:20:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.919+0000 7f17ba7fc700 1 -- 172.21.15.31:0/3280410421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b4011420 con 0x7f17c413fec0 2023-10-06T05:20:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.919+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3280410421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c40b2360 con 0x7f17c413fec0 2023-10-06T05:20:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.919+0000 7f17ba7fc700 1 -- 172.21.15.31:0/3280410421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b40117a0 con 0x7f17c413fec0 2023-10-06T05:20:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.919+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3280410421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c40b2920 con 0x7f17c413fec0 2023-10-06T05:20:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.920+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3280410421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c40b2e40 con 0x7f17c413fec0 2023-10-06T05:20:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.921+0000 7f17ba7fc700 1 -- 172.21.15.31:0/3280410421 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f17b4011b90 con 0x7f17c413fec0 2023-10-06T05:20:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.923+0000 7f17ba7fc700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f17a80670e0 0x7f17a8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.923+0000 7f17ba7fc700 1 -- 172.21.15.31:0/3280410421 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f17b40861f0 con 0x7f17c413fec0 2023-10-06T05:20:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.923+0000 7f17ca19b700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f17a80670e0 0x7f17a8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.923+0000 7f17ca19b700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f17a80670e0 0x7f17a8069590 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f17c0004fd0 tx=0x7f17c0007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:37.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:37.927+0000 7f17ba7fc700 1 -- 172.21.15.31:0/3280410421 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17c40b2e40 con 0x7f17c413fec0 2023-10-06T05:20:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.134+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3280410421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17c40b2e40 con 0x7f17c413fec0 2023-10-06T05:20:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.135+0000 7f17ba7fc700 1 -- 172.21.15.31:0/3280410421 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16916 (secure 0 0 0) 0x7f17c40b2e40 con 0x7f17c413fec0 2023-10-06T05:20:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:38.138 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:20:38.060326+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:34.970939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:35.729829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3280410421 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f17a80670e0 msgr2=0x7f17a8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f17a80670e0 0x7f17a8069590 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f17c0004fd0 tx=0x7f17c0007eb0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3280410421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fec0 msgr2=0x7f17c40c24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fec0 0x7f17c40c24a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f17c406db10 tx=0x7f17b4017000 comp rx=0 tx=0).stop 2023-10-06T05:20:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3280410421 shutdown_connections 2023-10-06T05:20:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17c415a680 0x7f17c40aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c413faf0 0x7f17c40b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f17a80670e0 0x7f17a8069590 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 --2- 172.21.15.31:0/3280410421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c413fec0 0x7f17c40c24a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3280410421 >> 172.21.15.31:0/3280410421 conn(0x7f17c40cdc50 msgr2=0x7f17c40c3980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3280410421 shutdown_connections 2023-10-06T05:20:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.138+0000 7f17cc3ff700 1 -- 172.21.15.31:0/3280410421 wait complete. 2023-10-06T05:20:38.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:20:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.356+0000 7f8d78fff700 1 Processor -- start 2023-10-06T05:20:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.356+0000 7f8d78fff700 1 -- start start 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.357+0000 7f8d78fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d740b2540 0x7f8d740cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.357+0000 7f8d78fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8d7415a4b0 0x7f8d740aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.357+0000 7f8d78fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7415a020 con 0x7f8d740cc340 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.358+0000 7f8d78fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d740ccc50 con 0x7f8d7415a4b0 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.358+0000 7f8d78fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740b07e0 con 0x7f8d740b2540 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.358+0000 7f8d72d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8d7415a4b0 0x7f8d740aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.358+0000 7f8d72d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8d7415a4b0 0x7f8d740aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45388/0 (socket says 172.21.15.31:45388) 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.358+0000 7f8d72d9d700 1 -- 172.21.15.31:0/668590756 learned_addr learned my addr 172.21.15.31:0/668590756 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:38.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.358+0000 7f8d71d9b700 1 --2- 172.21.15.31:0/668590756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d740b2540 0x7f8d740cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.358+0000 7f8d72d9d700 1 -- 172.21.15.31:0/668590756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d740b2540 msgr2=0x7f8d740cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:38.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.358+0000 7f8d72d9d700 1 --2- 172.21.15.31:0/668590756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d740b2540 0x7f8d740cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.358+0000 7f8d72d9d700 1 -- 172.21.15.31:0/668590756 >> v1:172.21.15.62:6789/0 conn(0x7f8d740cc340 legacy=0x7f8d740b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.358+0000 7f8d72d9d700 1 -- 172.21.15.31:0/668590756 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74159cf0 con 0x7f8d7415a4b0 2023-10-06T05:20:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.359+0000 7f8d72d9d700 1 --2- 172.21.15.31:0/668590756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d7415a4b0 0x7f8d740aecd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009f20 tx=0x7f8d6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a382759885e5bc7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.359+0000 7f8d7159a700 1 -- 172.21.15.31:0/668590756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00e070 con 0x7f8d7415a4b0 2023-10-06T05:20:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.359+0000 7f8d7159a700 1 -- 172.21.15.31:0/668590756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c00b9d0 con 0x7f8d7415a4b0 2023-10-06T05:20:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.359+0000 7f8d7159a700 1 -- 172.21.15.31:0/668590756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0092e0 con 0x7f8d7415a4b0 2023-10-06T05:20:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.359+0000 7f8d78fff700 1 -- 172.21.15.31:0/668590756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d7415a4b0 msgr2=0x7f8d740aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.359+0000 7f8d78fff700 1 --2- 172.21.15.31:0/668590756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d7415a4b0 0x7f8d740aecd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009f20 tx=0x7f8d6c009840 comp rx=0 tx=0).stop 2023-10-06T05:20:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.360+0000 7f8d78fff700 1 -- 172.21.15.31:0/668590756 shutdown_connections 2023-10-06T05:20:38.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.360+0000 7f8d78fff700 1 --2- 172.21.15.31:0/668590756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d7415a4b0 0x7f8d740aecd0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.360+0000 7f8d78fff700 1 --2- 172.21.15.31:0/668590756 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d740b2540 0x7f8d740cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.360+0000 7f8d78fff700 1 -- 172.21.15.31:0/668590756 >> 172.21.15.31:0/668590756 conn(0x7f8d74068670 msgr2=0x7f8d74068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:38.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.360+0000 7f8d78fff700 1 -- 172.21.15.31:0/668590756 shutdown_connections 2023-10-06T05:20:38.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.360+0000 7f8d78fff700 1 -- 172.21.15.31:0/668590756 wait complete. 2023-10-06T05:20:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.362+0000 7f8d78fff700 1 Processor -- start 2023-10-06T05:20:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.362+0000 7f8d78fff700 1 -- start start 2023-10-06T05:20:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.363+0000 7f8d78fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d740b2540 0x7f8d74169e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.363+0000 7f8d78fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d740cc340 0x7f8d74146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.363+0000 7f8d78fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d7415a4b0 0x7f8d740ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.363+0000 7f8d78fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74146ca0 con 0x7f8d740cc340 2023-10-06T05:20:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.363+0000 7f8d78fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740adcb0 con 0x7f8d740b2540 2023-10-06T05:20:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.363+0000 7f8d78fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740adf90 con 0x7f8d7415a4b0 2023-10-06T05:20:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.363+0000 7f8d71d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d740cc340 0x7f8d74146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d71d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d740cc340 0x7f8d74146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45396/0 (socket says 172.21.15.31:45396) 2023-10-06T05:20:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d71d9b700 1 -- 172.21.15.31:0/3873100463 learned_addr learned my addr 172.21.15.31:0/3873100463 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d7259c700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d740b2540 0x7f8d74169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d72d9d700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d7415a4b0 0x7f8d740ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d71d9b700 1 -- 172.21.15.31:0/3873100463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d7415a4b0 msgr2=0x7f8d740ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d71d9b700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d7415a4b0 0x7f8d740ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d71d9b700 1 -- 172.21.15.31:0/3873100463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d740b2540 msgr2=0x7f8d74169e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d71d9b700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d740b2540 0x7f8d74169e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d71d9b700 1 -- 172.21.15.31:0/3873100463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c009ae0 con 0x7f8d740cc340 2023-10-06T05:20:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d71d9b700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d740cc340 0x7f8d74146760 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800e930 tx=0x7f8d680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d627fc700 1 -- 172.21.15.31:0/3873100463 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6800eed0 con 0x7f8d740cc340 2023-10-06T05:20:38.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d627fc700 1 -- 172.21.15.31:0/3873100463 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6800c850 con 0x7f8d740cc340 2023-10-06T05:20:38.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d78fff700 1 -- 172.21.15.31:0/3873100463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d740ae270 con 0x7f8d740cc340 2023-10-06T05:20:38.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.364+0000 7f8d627fc700 1 -- 172.21.15.31:0/3873100463 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6800cbf0 con 0x7f8d740cc340 2023-10-06T05:20:38.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.365+0000 7f8d78fff700 1 -- 172.21.15.31:0/3873100463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d740ae830 con 0x7f8d740cc340 2023-10-06T05:20:38.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.366+0000 7f8d78fff700 1 -- 172.21.15.31:0/3873100463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7404c380 con 0x7f8d740cc340 2023-10-06T05:20:38.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.366+0000 7f8d627fc700 1 -- 172.21.15.31:0/3873100463 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8d68013470 con 0x7f8d740cc340 2023-10-06T05:20:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.368+0000 7f8d627fc700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8d540670e0 0x7f8d54069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.368+0000 7f8d627fc700 1 -- 172.21.15.31:0/3873100463 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8d68084c70 con 0x7f8d740cc340 2023-10-06T05:20:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.368+0000 7f8d7259c700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8d540670e0 0x7f8d54069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.369+0000 7f8d7259c700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8d540670e0 0x7f8d54069590 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8d64000c00 tx=0x7f8d64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:38.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.371+0000 7f8d627fc700 1 -- 172.21.15.31:0/3873100463 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d68051a60 con 0x7f8d740cc340 2023-10-06T05:20:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:38.619+0000 7f8d78fff700 1 -- 172.21.15.31:0/3873100463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8d740b2b40 con 0x7f8d740cc340 2023-10-06T05:20:39.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.275+0000 7f8d627fc700 1 -- 172.21.15.31:0/3873100463 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8d68055080 con 0x7f8d740cc340 2023-10-06T05:20:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.278+0000 7f8d78fff700 1 -- 172.21.15.31:0/3873100463 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8d540670e0 msgr2=0x7f8d54069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.278+0000 7f8d78fff700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8d540670e0 0x7f8d54069590 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8d64000c00 tx=0x7f8d64009040 comp rx=0 tx=0).stop 2023-10-06T05:20:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.278+0000 7f8d78fff700 1 -- 172.21.15.31:0/3873100463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d740cc340 msgr2=0x7f8d74146760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.278+0000 7f8d78fff700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d740cc340 0x7f8d74146760 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800e930 tx=0x7f8d680091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.280+0000 7f8d78fff700 1 -- 172.21.15.31:0/3873100463 shutdown_connections 2023-10-06T05:20:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.280+0000 7f8d78fff700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d7415a4b0 0x7f8d740ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.280+0000 7f8d78fff700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d740b2540 0x7f8d74169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.280+0000 7f8d78fff700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8d540670e0 0x7f8d54069590 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.280+0000 7f8d78fff700 1 --2- 172.21.15.31:0/3873100463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d740cc340 0x7f8d74146760 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.280+0000 7f8d78fff700 1 -- 172.21.15.31:0/3873100463 >> 172.21.15.31:0/3873100463 conn(0x7f8d74068670 msgr2=0x7f8d74156750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.280+0000 7f8d78fff700 1 -- 172.21.15.31:0/3873100463 shutdown_connections 2023-10-06T05:20:39.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.280+0000 7f8d78fff700 1 -- 172.21.15.31:0/3873100463 wait complete. 2023-10-06T05:20:39.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:20:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.504+0000 7f9cfdbff700 1 Processor -- start 2023-10-06T05:20:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.504+0000 7f9cfdbff700 1 -- start start 2023-10-06T05:20:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.505+0000 7f9cfdbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9cf80b0b90 0x7f9cf80af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.505+0000 7f9cfdbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cf80af6c0 0x7f9cf80adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.505+0000 7f9cfdbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf80ad690 con 0x7f9cf8153d40 2023-10-06T05:20:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.505+0000 7f9cfdbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80acd60 con 0x7f9cf80b0b90 2023-10-06T05:20:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.505+0000 7f9cfdbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80ad040 con 0x7f9cf80af6c0 2023-10-06T05:20:39.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.506+0000 7f9cf77fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9cf8153d40 0x7f9cf80b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42032/0 (socket says 172.21.15.31:42032) 2023-10-06T05:20:39.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.506+0000 7f9cf77fe700 1 -- 172.21.15.31:0/2007200116 learned_addr learned my addr 172.21.15.31:0/2007200116 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.506+0000 7f9cf6ffd700 1 --2- 172.21.15.31:0/2007200116 >> v2:172.21.15.62:3300/0 conn(0x7f9cf80b0b90 0x7f9cf80af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.506+0000 7f9cf7fff700 1 --2- 172.21.15.31:0/2007200116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cf80af6c0 0x7f9cf80adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.506+0000 7f9cf67fc700 1 -- 172.21.15.31:0/2007200116 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459456434 0 0) 0x7f9cf80ad690 con 0x7f9cf8153d40 2023-10-06T05:20:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.506+0000 7f9cf67fc700 1 -- 172.21.15.31:0/2007200116 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce0003570 con 0x7f9cf8153d40 2023-10-06T05:20:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.506+0000 7f9cf6ffd700 1 -- 172.21.15.31:0/2007200116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cf80af6c0 msgr2=0x7f9cf80adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.506+0000 7f9cf6ffd700 1 --2- 172.21.15.31:0/2007200116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cf80af6c0 0x7f9cf80adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.506+0000 7f9cf67fc700 1 -- 172.21.15.31:0/2007200116 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 122505516 0 0) 0x7f9ce0003570 con 0x7f9cf8153d40 2023-10-06T05:20:39.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.506+0000 7f9cf6ffd700 1 -- 172.21.15.31:0/2007200116 >> v1:172.21.15.31:6789/0 conn(0x7f9cf8153d40 legacy=0x7f9cf80b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.507+0000 7f9cf6ffd700 1 -- 172.21.15.31:0/2007200116 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80ad320 con 0x7f9cf80b0b90 2023-10-06T05:20:39.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.507+0000 7f9cf6ffd700 1 --2- 172.21.15.31:0/2007200116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cf80b0b90 0x7f9cf80af180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0098e0 tx=0x7f9cec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35466dfbf0a478ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.507+0000 7f9cf67fc700 1 -- 172.21.15.31:0/2007200116 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8001470 con 0x7f9cf80b0b90 2023-10-06T05:20:39.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.507+0000 7f9cf67fc700 1 -- 172.21.15.31:0/2007200116 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec011420 con 0x7f9cf80b0b90 2023-10-06T05:20:39.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.507+0000 7f9cf67fc700 1 -- 172.21.15.31:0/2007200116 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0117c0 con 0x7f9cf80b0b90 2023-10-06T05:20:39.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.508+0000 7f9cfdbff700 1 -- 172.21.15.31:0/2007200116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cf80b0b90 msgr2=0x7f9cf80af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.508+0000 7f9cfdbff700 1 --2- 172.21.15.31:0/2007200116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cf80b0b90 0x7f9cf80af180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0098e0 tx=0x7f9cec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.508+0000 7f9cfdbff700 1 -- 172.21.15.31:0/2007200116 shutdown_connections 2023-10-06T05:20:39.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.508+0000 7f9cfdbff700 1 --2- 172.21.15.31:0/2007200116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cf80af6c0 0x7f9cf80adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.508+0000 7f9cfdbff700 1 --2- 172.21.15.31:0/2007200116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cf80b0b90 0x7f9cf80af180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.508+0000 7f9cfdbff700 1 -- 172.21.15.31:0/2007200116 >> 172.21.15.31:0/2007200116 conn(0x7f9cf8067560 msgr2=0x7f9cf8067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:39.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.508+0000 7f9cfdbff700 1 -- 172.21.15.31:0/2007200116 shutdown_connections 2023-10-06T05:20:39.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.508+0000 7f9cfdbff700 1 -- 172.21.15.31:0/2007200116 wait complete. 2023-10-06T05:20:39.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.509+0000 7f9cfdbff700 1 Processor -- start 2023-10-06T05:20:39.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.509+0000 7f9cfdbff700 1 -- start start 2023-10-06T05:20:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.510+0000 7f9cfdbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cf80af6c0 0x7f9cf8151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.510+0000 7f9cfdbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cf80b0b90 0x7f9cf8157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.510+0000 7f9cfdbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cf8153d40 0x7f9cf816cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.510+0000 7f9cfdbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80c9c90 con 0x7f9cf80af6c0 2023-10-06T05:20:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.510+0000 7f9cfdbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81577c0 con 0x7f9cf80b0b90 2023-10-06T05:20:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.510+0000 7f9cfdbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80ca0b0 con 0x7f9cf8153d40 2023-10-06T05:20:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cf80af6c0 0x7f9cf8151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cf80af6c0 0x7f9cf8151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45400/0 (socket says 172.21.15.31:45400) 2023-10-06T05:20:39.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf77fe700 1 -- 172.21.15.31:0/1774801046 learned_addr learned my addr 172.21.15.31:0/1774801046 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf7fff700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cf8153d40 0x7f9cf816cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf6ffd700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cf80b0b90 0x7f9cf8157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf77fe700 1 -- 172.21.15.31:0/1774801046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cf8153d40 msgr2=0x7f9cf816cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf77fe700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cf8153d40 0x7f9cf816cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf77fe700 1 -- 172.21.15.31:0/1774801046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cf80b0b90 msgr2=0x7f9cf8157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf77fe700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cf80b0b90 0x7f9cf8157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf77fe700 1 -- 172.21.15.31:0/1774801046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec009420 con 0x7f9cf80af6c0 2023-10-06T05:20:39.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.511+0000 7f9cf77fe700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cf80af6c0 0x7f9cf8151be0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800cb90 tx=0x7f9ce8001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.512+0000 7f9cdf7fe700 1 -- 172.21.15.31:0/1774801046 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8001d10 con 0x7f9cf80af6c0 2023-10-06T05:20:39.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.512+0000 7f9cdf7fe700 1 -- 172.21.15.31:0/1774801046 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce8007360 con 0x7f9cf80af6c0 2023-10-06T05:20:39.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.512+0000 7f9cfdbff700 1 -- 172.21.15.31:0/1774801046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf815e300 con 0x7f9cf80af6c0 2023-10-06T05:20:39.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.512+0000 7f9cdf7fe700 1 -- 172.21.15.31:0/1774801046 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8007700 con 0x7f9cf80af6c0 2023-10-06T05:20:39.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.512+0000 7f9cfdbff700 1 -- 172.21.15.31:0/1774801046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf815e8c0 con 0x7f9cf80af6c0 2023-10-06T05:20:39.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.513+0000 7f9cfdbff700 1 -- 172.21.15.31:0/1774801046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf815eab0 con 0x7f9cf80af6c0 2023-10-06T05:20:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.514+0000 7f9cdf7fe700 1 -- 172.21.15.31:0/1774801046 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9ce8004a40 con 0x7f9cf80af6c0 2023-10-06T05:20:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.515+0000 7f9cdf7fe700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ce40670e0 0x7f9ce4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.515+0000 7f9cdf7fe700 1 -- 172.21.15.31:0/1774801046 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ce8084300 con 0x7f9cf80af6c0 2023-10-06T05:20:39.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.515+0000 7f9cf6ffd700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ce40670e0 0x7f9ce4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:39.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.515+0000 7f9cf6ffd700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ce40670e0 0x7f9ce4069590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80b0fb0 tx=0x7f9cec016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.519+0000 7f9cdf7fe700 1 -- 172.21.15.31:0/1774801046 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ce8051170 con 0x7f9cf80af6c0 2023-10-06T05:20:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:39.770+0000 7f9cfdbff700 1 -- 172.21.15.31:0/1774801046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9cf804c380 con 0x7f9cf80af6c0 2023-10-06T05:20:40.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.285+0000 7f9cdf7fe700 1 -- 172.21.15.31:0/1774801046 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9ce8054790 con 0x7f9cf80af6c0 2023-10-06T05:20:40.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 -- 172.21.15.31:0/1774801046 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ce40670e0 msgr2=0x7f9ce4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ce40670e0 0x7f9ce4069590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80b0fb0 tx=0x7f9cec016000 comp rx=0 tx=0).stop 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 -- 172.21.15.31:0/1774801046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cf80af6c0 msgr2=0x7f9cf8151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cf80af6c0 0x7f9cf8151be0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800cb90 tx=0x7f9ce8001a80 comp rx=0 tx=0).stop 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 -- 172.21.15.31:0/1774801046 shutdown_connections 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cf8153d40 0x7f9cf816cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f9ce40670e0 0x7f9ce4069590 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cf80af6c0 0x7f9cf8151be0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 --2- 172.21.15.31:0/1774801046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cf80b0b90 0x7f9cf8157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 -- 172.21.15.31:0/1774801046 >> 172.21.15.31:0/1774801046 conn(0x7f9cf8067560 msgr2=0x7f9cf80aad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 -- 172.21.15.31:0/1774801046 shutdown_connections 2023-10-06T05:20:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.288+0000 7f9cfdbff700 1 -- 172.21.15.31:0/1774801046 wait complete. 2023-10-06T05:20:40.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:40.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.505+0000 7f4fa4bff700 1 Processor -- start 2023-10-06T05:20:40.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.505+0000 7f4fa4bff700 1 -- start start 2023-10-06T05:20:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4fa4bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fa00ca770 0x7f4fa00c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4fa4bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fa00cab40 0x7f4fa01569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4fa4bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fa006ae00 0x7f4fa006b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4fa4bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa00c7a70 con 0x7f4fa006ae00 2023-10-06T05:20:40.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4fa4bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa0156ef0 con 0x7f4fa00ca770 2023-10-06T05:20:40.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4fa4bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa006b7c0 con 0x7f4fa00cab40 2023-10-06T05:20:40.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4f9ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fa006ae00 0x7f4fa006b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4f9e59c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fa00ca770 0x7f4fa00c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4f9ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fa006ae00 0x7f4fa006b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45410/0 (socket says 172.21.15.31:45410) 2023-10-06T05:20:40.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4f9ed9d700 1 -- 172.21.15.31:0/4221868032 learned_addr learned my addr 172.21.15.31:0/4221868032 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:40.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.507+0000 7f4f9dd9b700 1 --2- 172.21.15.31:0/4221868032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fa00cab40 0x7f4fa01569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.508+0000 7f4f9ed9d700 1 -- 172.21.15.31:0/4221868032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fa00cab40 msgr2=0x7f4fa01569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:40.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.508+0000 7f4f9ed9d700 1 --2- 172.21.15.31:0/4221868032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fa00cab40 0x7f4fa01569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.508+0000 7f4f9ed9d700 1 -- 172.21.15.31:0/4221868032 >> v2:172.21.15.62:3300/0 conn(0x7f4fa00ca770 msgr2=0x7f4fa00c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:40.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.508+0000 7f4f9ed9d700 1 --2- 172.21.15.31:0/4221868032 >> v2:172.21.15.62:3300/0 conn(0x7f4fa00ca770 0x7f4fa00c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.508+0000 7f4f9ed9d700 1 -- 172.21.15.31:0/4221868032 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa00c6df0 con 0x7f4fa006ae00 2023-10-06T05:20:40.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.508+0000 7f4f9e59c700 1 --2- 172.21.15.31:0/4221868032 >> v2:172.21.15.62:3300/0 conn(0x7f4fa00ca770 0x7f4fa00c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:40.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.508+0000 7f4f9ed9d700 1 --2- 172.21.15.31:0/4221868032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 0x7f4fa006b280 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00cce00 tx=0x7f4f98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af80586e685cdca7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.508+0000 7f4f9d59a700 1 -- 172.21.15.31:0/4221868032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800e070 con 0x7f4fa006ae00 2023-10-06T05:20:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.508+0000 7f4f9d59a700 1 -- 172.21.15.31:0/4221868032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9800b970 con 0x7f4fa006ae00 2023-10-06T05:20:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.508+0000 7f4f9d59a700 1 -- 172.21.15.31:0/4221868032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f980092e0 con 0x7f4fa006ae00 2023-10-06T05:20:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.509+0000 7f4fa4bff700 1 -- 172.21.15.31:0/4221868032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 msgr2=0x7f4fa006b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.509+0000 7f4fa4bff700 1 --2- 172.21.15.31:0/4221868032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 0x7f4fa006b280 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00cce00 tx=0x7f4f98009840 comp rx=0 tx=0).stop 2023-10-06T05:20:40.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.509+0000 7f4fa4bff700 1 -- 172.21.15.31:0/4221868032 shutdown_connections 2023-10-06T05:20:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.509+0000 7f4fa4bff700 1 --2- 172.21.15.31:0/4221868032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 0x7f4fa006b280 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.509+0000 7f4fa4bff700 1 --2- 172.21.15.31:0/4221868032 >> v2:172.21.15.62:3300/0 conn(0x7f4fa00ca770 0x7f4fa00c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.509+0000 7f4fa4bff700 1 --2- 172.21.15.31:0/4221868032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fa00cab40 0x7f4fa01569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.509+0000 7f4fa4bff700 1 -- 172.21.15.31:0/4221868032 >> 172.21.15.31:0/4221868032 conn(0x7f4fa00cc0b0 msgr2=0x7f4fa00cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.509+0000 7f4fa4bff700 1 -- 172.21.15.31:0/4221868032 shutdown_connections 2023-10-06T05:20:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.509+0000 7f4fa4bff700 1 -- 172.21.15.31:0/4221868032 wait complete. 2023-10-06T05:20:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.510+0000 7f4fa4bff700 1 Processor -- start 2023-10-06T05:20:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.511+0000 7f4fa4bff700 1 -- start start 2023-10-06T05:20:40.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.511+0000 7f4fa4bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 0x7f4fa00ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.511+0000 7f4fa4bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fa00ca770 0x7f4fa016a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.511+0000 7f4fa4bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00cab40 0x7f4fa0169ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.511+0000 7f4fa4bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa0152bf0 con 0x7f4fa006ae00 2023-10-06T05:20:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.511+0000 7f4fa4bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa016a430 con 0x7f4fa00cab40 2023-10-06T05:20:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.511+0000 7f4fa4bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa0166e60 con 0x7f4fa00ca770 2023-10-06T05:20:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.512+0000 7f4f9e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 0x7f4fa00ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.512+0000 7f4f9e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 0x7f4fa00ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45424/0 (socket says 172.21.15.31:45424) 2023-10-06T05:20:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.512+0000 7f4f9e59c700 1 -- 172.21.15.31:0/1110264059 learned_addr learned my addr 172.21.15.31:0/1110264059 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.512+0000 7f4f9e59c700 1 -- 172.21.15.31:0/1110264059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fa00ca770 msgr2=0x7f4fa016a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.512+0000 7f4f9e59c700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fa00ca770 0x7f4fa016a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.512+0000 7f4f9e59c700 1 -- 172.21.15.31:0/1110264059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00cab40 msgr2=0x7f4fa0169ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.512+0000 7f4f9e59c700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00cab40 0x7f4fa0169ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.512+0000 7f4f9e59c700 1 -- 172.21.15.31:0/1110264059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f900093c0 con 0x7f4fa006ae00 2023-10-06T05:20:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.512+0000 7f4f9e59c700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 0x7f4fa00ae200 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000ecb0 tx=0x7f4f90009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.513+0000 7f4f8e7fc700 1 -- 172.21.15.31:0/1110264059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9000ee60 con 0x7f4fa006ae00 2023-10-06T05:20:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.513+0000 7f4f8e7fc700 1 -- 172.21.15.31:0/1110264059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9000cbe0 con 0x7f4fa006ae00 2023-10-06T05:20:40.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.513+0000 7f4f8e7fc700 1 -- 172.21.15.31:0/1110264059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90005a00 con 0x7f4fa006ae00 2023-10-06T05:20:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.513+0000 7f4fa4bff700 1 -- 172.21.15.31:0/1110264059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f98009ae0 con 0x7f4fa006ae00 2023-10-06T05:20:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.513+0000 7f4fa4bff700 1 -- 172.21.15.31:0/1110264059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa0152e40 con 0x7f4fa006ae00 2023-10-06T05:20:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.514+0000 7f4fa4bff700 1 -- 172.21.15.31:0/1110264059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa004c380 con 0x7f4fa006ae00 2023-10-06T05:20:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.515+0000 7f4f8e7fc700 1 -- 172.21.15.31:0/1110264059 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f4f90006030 con 0x7f4fa006ae00 2023-10-06T05:20:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.515+0000 7f4f8e7fc700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4f800670e0 0x7f4f80069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.516+0000 7f4f8e7fc700 1 -- 172.21.15.31:0/1110264059 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4f90084fe0 con 0x7f4fa006ae00 2023-10-06T05:20:40.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.516+0000 7f4f9dd9b700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4f800670e0 0x7f4f80069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.516+0000 7f4f9dd9b700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4f800670e0 0x7f4f80069590 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4fa006f870 tx=0x7f4f940091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.520+0000 7f4f8e7fc700 1 -- 172.21.15.31:0/1110264059 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f90051e10 con 0x7f4fa006ae00 2023-10-06T05:20:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.767+0000 7f4fa4bff700 1 -- 172.21.15.31:0/1110264059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fa00cbe20 con 0x7f4fa006ae00 2023-10-06T05:20:40.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.767+0000 7f4f8e7fc700 1 -- 172.21.15.31:0/1110264059 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f4f90020020 con 0x7f4fa006ae00 2023-10-06T05:20:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:40.769 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5743,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2430199885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2430199885},{"type":"v1","addr":"172.21.15.31:6837","nonce":2430199885}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5746,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3762519665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3762519665},{"type":"v1","addr":"172.21.15.62:6834","nonce":3762519665}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5844,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2776536460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2776536460},{"type":"v1","addr":"172.21.15.31:6836","nonce":2776536460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:36.742745+0000","modified":"2023-10-06T05:20:40.276711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5853":{"gid":5853,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/982526111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":982526111},{"type":"v1","addr":"172.21.15.62:6835","nonce":982526111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:20:40.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.769+0000 7f4fa4bff700 1 -- 172.21.15.31:0/1110264059 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4f800670e0 msgr2=0x7f4f80069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.769+0000 7f4fa4bff700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4f800670e0 0x7f4f80069590 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4fa006f870 tx=0x7f4f940091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.770+0000 7f4fa4bff700 1 -- 172.21.15.31:0/1110264059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 msgr2=0x7f4fa00ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.770+0000 7f4fa4bff700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 0x7f4fa00ae200 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000ecb0 tx=0x7f4f90009120 comp rx=0 tx=0).stop 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.770+0000 7f4fa4bff700 1 -- 172.21.15.31:0/1110264059 shutdown_connections 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.770+0000 7f4fa4bff700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00cab40 0x7f4fa0169ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.770+0000 7f4fa4bff700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4f800670e0 0x7f4f80069590 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.770+0000 7f4fa4bff700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fa006ae00 0x7f4fa00ae200 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.770+0000 7f4fa4bff700 1 --2- 172.21.15.31:0/1110264059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4fa00ca770 0x7f4fa016a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.770+0000 7f4fa4bff700 1 -- 172.21.15.31:0/1110264059 >> 172.21.15.31:0/1110264059 conn(0x7f4fa00cc0b0 msgr2=0x7f4fa00cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.770+0000 7f4fa4bff700 1 -- 172.21.15.31:0/1110264059 shutdown_connections 2023-10-06T05:20:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.770+0000 7f4fa4bff700 1 -- 172.21.15.31:0/1110264059 wait complete. 2023-10-06T05:20:40.773 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 35 2023-10-06T05:20:40.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:40.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.984+0000 7f6cf55ff700 1 Processor -- start 2023-10-06T05:20:40.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.984+0000 7f6cf55ff700 1 -- start start 2023-10-06T05:20:40.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.985+0000 7f6cf55ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf015a930 0x7f6cf00cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.985+0000 7f6cf55ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6cf00cf5a0 0x7f6cf0163240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.985+0000 7f6cf55ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf00ce090 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.985+0000 7f6cf55ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf00cb6f0 con 0x7f6cf00cf5a0 2023-10-06T05:20:40.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6cf55ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00cb9d0 con 0x7f6cf015a930 2023-10-06T05:20:40.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6cee7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6cf00cb1b0 0x7f6cf00ccd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42046/0 (socket says 172.21.15.31:42046) 2023-10-06T05:20:40.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6cee7fc700 1 -- 172.21.15.31:0/499527496 learned_addr learned my addr 172.21.15.31:0/499527496 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:40.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6ceeffd700 1 --2- 172.21.15.31:0/499527496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf015a930 0x7f6cf00cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6cef7fe700 1 --2- 172.21.15.31:0/499527496 >> v2:172.21.15.62:3300/0 conn(0x7f6cf00cf5a0 0x7f6cf0163240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6cedffb700 1 -- 172.21.15.31:0/499527496 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323028321 0 0) 0x7f6cf00ce090 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6cedffb700 1 -- 172.21.15.31:0/499527496 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd8003570 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6cef7fe700 1 -- 172.21.15.31:0/499527496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf015a930 msgr2=0x7f6cf00cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6cef7fe700 1 --2- 172.21.15.31:0/499527496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf015a930 0x7f6cf00cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6cef7fe700 1 -- 172.21.15.31:0/499527496 >> v1:172.21.15.31:6789/0 conn(0x7f6cf00cb1b0 legacy=0x7f6cf00ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6cef7fe700 1 -- 172.21.15.31:0/499527496 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf00cbcb0 con 0x7f6cf00cf5a0 2023-10-06T05:20:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.986+0000 7f6ceeffd700 1 --2- 172.21.15.31:0/499527496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf015a930 0x7f6cf00cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.987+0000 7f6cef7fe700 1 --2- 172.21.15.31:0/499527496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cf00cf5a0 0x7f6cf0163240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009e60 tx=0x7f6ce8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a8b816c4fcc7b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.987+0000 7f6cedffb700 1 -- 172.21.15.31:0/499527496 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800e070 con 0x7f6cf00cf5a0 2023-10-06T05:20:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.987+0000 7f6cedffb700 1 -- 172.21.15.31:0/499527496 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce800bb40 con 0x7f6cf00cf5a0 2023-10-06T05:20:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.987+0000 7f6cedffb700 1 -- 172.21.15.31:0/499527496 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8012430 con 0x7f6cf00cf5a0 2023-10-06T05:20:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.988+0000 7f6cf55ff700 1 -- 172.21.15.31:0/499527496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cf00cf5a0 msgr2=0x7f6cf0163240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.988+0000 7f6cf55ff700 1 --2- 172.21.15.31:0/499527496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cf00cf5a0 0x7f6cf0163240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009e60 tx=0x7f6ce8009780 comp rx=0 tx=0).stop 2023-10-06T05:20:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.988+0000 7f6cf55ff700 1 -- 172.21.15.31:0/499527496 shutdown_connections 2023-10-06T05:20:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.988+0000 7f6cf55ff700 1 --2- 172.21.15.31:0/499527496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cf00cf5a0 0x7f6cf0163240 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.988+0000 7f6cf55ff700 1 --2- 172.21.15.31:0/499527496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf015a930 0x7f6cf00cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.988+0000 7f6cf55ff700 1 -- 172.21.15.31:0/499527496 >> 172.21.15.31:0/499527496 conn(0x7f6cf0099720 msgr2=0x7f6cf006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.988+0000 7f6cf55ff700 1 -- 172.21.15.31:0/499527496 shutdown_connections 2023-10-06T05:20:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.988+0000 7f6cf55ff700 1 -- 172.21.15.31:0/499527496 wait complete. 2023-10-06T05:20:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.989+0000 7f6cf55ff700 1 Processor -- start 2023-10-06T05:20:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.989+0000 7f6cf55ff700 1 -- start start 2023-10-06T05:20:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.990+0000 7f6cf55ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf00cb1b0 0x7f6cf00ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.990+0000 7f6cf55ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf00cf5a0 0x7f6cf009c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.990+0000 7f6cf55ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cf015a930 0x7f6cf0153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.990+0000 7f6cf55ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf009cd90 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.990+0000 7f6cf55ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf009c5d0 con 0x7f6cf015a930 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.990+0000 7f6cf55ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00c74d0 con 0x7f6cf00cf5a0 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6ceeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf00cb1b0 0x7f6cf00ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6ceeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf00cb1b0 0x7f6cf00ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45430/0 (socket says 172.21.15.31:45430) 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6ceeffd700 1 -- 172.21.15.31:0/1728193591 learned_addr learned my addr 172.21.15.31:0/1728193591 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6cef7fe700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cf015a930 0x7f6cf0153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6cee7fc700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf00cf5a0 0x7f6cf009c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6ceeffd700 1 -- 172.21.15.31:0/1728193591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf00cf5a0 msgr2=0x7f6cf009c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6ceeffd700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf00cf5a0 0x7f6cf009c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6ceeffd700 1 -- 172.21.15.31:0/1728193591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cf015a930 msgr2=0x7f6cf0153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6ceeffd700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cf015a930 0x7f6cf0153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6ceeffd700 1 -- 172.21.15.31:0/1728193591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce00093c0 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.991+0000 7f6ceeffd700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf00cb1b0 0x7f6cf00ca180 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000ec80 tx=0x7f6ce0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.992+0000 7f6cd6ffd700 1 -- 172.21.15.31:0/1728193591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0005a00 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.992+0000 7f6cd6ffd700 1 -- 172.21.15.31:0/1728193591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0005b60 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.992+0000 7f6cf55ff700 1 -- 172.21.15.31:0/1728193591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce8009a20 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.992+0000 7f6cd6ffd700 1 -- 172.21.15.31:0/1728193591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce000c850 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.992+0000 7f6cf55ff700 1 -- 172.21.15.31:0/1728193591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf00c7a50 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.993+0000 7f6cf55ff700 1 -- 172.21.15.31:0/1728193591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf00ca720 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.994+0000 7f6cd6ffd700 1 -- 172.21.15.31:0/1728193591 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6ce000cbc0 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.994+0000 7f6cd6ffd700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cdc0670e0 0x7f6cdc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:40.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.995+0000 7f6cd6ffd700 1 -- 172.21.15.31:0/1728193591 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ce0084ee0 con 0x7f6cf00cb1b0 2023-10-06T05:20:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.995+0000 7f6cee7fc700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cdc0670e0 0x7f6cdc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.995+0000 7f6cee7fc700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cdc0670e0 0x7f6cdc069590 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00c9950 tx=0x7f6ce4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:40.999+0000 7f6cd6ffd700 1 -- 172.21.15.31:0/1728193591 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ce0055760 con 0x7f6cf00cb1b0 2023-10-06T05:20:41.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.204+0000 7f6cf55ff700 1 -- 172.21.15.31:0/1728193591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cf004c380 con 0x7f6cf00cb1b0 2023-10-06T05:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.205+0000 7f6cd6ffd700 1 -- 172.21.15.31:0/1728193591 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16916 (secure 0 0 0) 0x7f6ce00552f0 con 0x7f6cf00cb1b0 2023-10-06T05:20:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:41.207 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:20:38.060326+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:34.970939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:35.729829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:41.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.207+0000 7f6cf55ff700 1 -- 172.21.15.31:0/1728193591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cdc0670e0 msgr2=0x7f6cdc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.207+0000 7f6cf55ff700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cdc0670e0 0x7f6cdc069590 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00c9950 tx=0x7f6ce4009040 comp rx=0 tx=0).stop 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.207+0000 7f6cf55ff700 1 -- 172.21.15.31:0/1728193591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf00cb1b0 msgr2=0x7f6cf00ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.207+0000 7f6cf55ff700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf00cb1b0 0x7f6cf00ca180 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000ec80 tx=0x7f6ce0009120 comp rx=0 tx=0).stop 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.208+0000 7f6cf55ff700 1 -- 172.21.15.31:0/1728193591 shutdown_connections 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.208+0000 7f6cf55ff700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cf015a930 0x7f6cf0153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.208+0000 7f6cf55ff700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6cdc0670e0 0x7f6cdc069590 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.208+0000 7f6cf55ff700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf00cb1b0 0x7f6cf00ca180 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.208+0000 7f6cf55ff700 1 --2- 172.21.15.31:0/1728193591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cf00cf5a0 0x7f6cf009c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.208+0000 7f6cf55ff700 1 -- 172.21.15.31:0/1728193591 >> 172.21.15.31:0/1728193591 conn(0x7f6cf0099720 msgr2=0x7f6cf0069910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.208+0000 7f6cf55ff700 1 -- 172.21.15.31:0/1728193591 shutdown_connections 2023-10-06T05:20:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.208+0000 7f6cf55ff700 1 -- 172.21.15.31:0/1728193591 wait complete. 2023-10-06T05:20:41.227 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:20:41.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:20:41.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.421+0000 7f05d13ff700 1 Processor -- start 2023-10-06T05:20:41.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.421+0000 7f05d13ff700 1 -- start start 2023-10-06T05:20:41.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.422+0000 7f05d13ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc159500 0x7f05cc0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.422+0000 7f05d13ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05cc0c3f40 con 0x7f05cc0cd510 2023-10-06T05:20:41.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.422+0000 7f05d13ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05cc0cb000 con 0x7f05cc15d2d0 2023-10-06T05:20:41.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.422+0000 7f05d13ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0b2860 con 0x7f05cc159500 2023-10-06T05:20:41.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05cb7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f05cc0cd510 0x7f05cc0aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42048/0 (socket says 172.21.15.31:42048) 2023-10-06T05:20:41.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05cb7fe700 1 -- 172.21.15.31:0/1956669996 learned_addr learned my addr 172.21.15.31:0/1956669996 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05caffd700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc159500 0x7f05cc0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868795783 0 0) 0x7f05cc0c3f40 con 0x7f05cc0cd510 2023-10-06T05:20:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05b4003570 con 0x7f05cc0cd510 2023-10-06T05:20:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403161412 0 0) 0x7f05cc0cb000 con 0x7f05cc15d2d0 2023-10-06T05:20:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05cc0c3f40 con 0x7f05cc15d2d0 2023-10-06T05:20:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2375347336 0 0) 0x7f05b4003570 con 0x7f05cc0cd510 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc159500 msgr2=0x7f05cc0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05c9ffb700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc159500 0x7f05cc0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 >> v1:172.21.15.62:6789/0 conn(0x7f05cc15d2d0 legacy=0x7f05cc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc15d7d0 con 0x7f05cc0cd510 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.423+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f05c40017f0 con 0x7f05cc0cd510 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 >> v1:172.21.15.31:6789/0 conn(0x7f05cc0cd510 legacy=0x7f05cc0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05c9ffb700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b4003c20 0x7f05b4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05c9ffb700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc159500 0x7f05b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05c9ffb700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b4004d50 0x7f05b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0b2860 con 0x7f05b4004d50 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b400bb40 con 0x7f05b4003c20 2023-10-06T05:20:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b400bd00 con 0x7f05cc159500 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05cb7fe700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b4004d50 0x7f05b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05caffd700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b4003c20 0x7f05b4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05ca7fc700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc159500 0x7f05b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05cb7fe700 1 -- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc159500 msgr2=0x7f05b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05cb7fe700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc159500 0x7f05b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05cb7fe700 1 -- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b4003c20 msgr2=0x7f05b4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05cb7fe700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b4003c20 0x7f05b4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.424+0000 7f05cb7fe700 1 -- 172.21.15.31:0/1956669996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b4003570 con 0x7f05b4004d50 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.425+0000 7f05cb7fe700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b4004d50 0x7f05b40051c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0cd910 tx=0x7f05c4002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd8aec9ae14a514 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.425+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c40070d0 con 0x7f05b4004d50 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.425+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c4007250 con 0x7f05b4004d50 2023-10-06T05:20:41.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.425+0000 7f05c9ffb700 1 -- 172.21.15.31:0/1956669996 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4007610 con 0x7f05b4004d50 2023-10-06T05:20:41.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.425+0000 7f05d13ff700 1 -- 172.21.15.31:0/1956669996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b4004d50 msgr2=0x7f05b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.425+0000 7f05d13ff700 1 --2- 172.21.15.31:0/1956669996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b4004d50 0x7f05b40051c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0cd910 tx=0x7f05c4002190 comp rx=0 tx=0).stop 2023-10-06T05:20:41.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.425+0000 7f05cb7fe700 1 -- 172.21.15.31:0/1956669996 reap_dead start 2023-10-06T05:20:41.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.425+0000 7f05d13ff700 1 -- 172.21.15.31:0/1956669996 shutdown_connections 2023-10-06T05:20:41.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.425+0000 7f05d13ff700 1 -- 172.21.15.31:0/1956669996 >> 172.21.15.31:0/1956669996 conn(0x7f05cc06ec90 msgr2=0x7f05cc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.426+0000 7f05d13ff700 1 -- 172.21.15.31:0/1956669996 shutdown_connections 2023-10-06T05:20:41.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.426+0000 7f05d13ff700 1 -- 172.21.15.31:0/1956669996 wait complete. 2023-10-06T05:20:41.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.427+0000 7f05d13ff700 1 Processor -- start 2023-10-06T05:20:41.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.427+0000 7f05d13ff700 1 -- start start 2023-10-06T05:20:41.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.428+0000 7f05d13ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b4004d50 0x7f05cc16cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.428+0000 7f05d13ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc0d0b80 0x7f05cc0c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.428+0000 7f05d13ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc0c47d0 0x7f05cc0cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.428+0000 7f05d13ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0d0ff0 con 0x7f05cc0c47d0 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.428+0000 7f05d13ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0ce170 con 0x7f05b4004d50 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.428+0000 7f05d13ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc16d610 con 0x7f05cc0d0b80 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.428+0000 7f05cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc0c47d0 0x7f05cc0cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.428+0000 7f05cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc0c47d0 0x7f05cc0cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45456/0 (socket says 172.21.15.31:45456) 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.428+0000 7f05cb7fe700 1 -- 172.21.15.31:0/3646209243 learned_addr learned my addr 172.21.15.31:0/3646209243 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05caffd700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b4004d50 0x7f05cc16cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05ca7fc700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc0d0b80 0x7f05cc0c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05cb7fe700 1 -- 172.21.15.31:0/3646209243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc0d0b80 msgr2=0x7f05cc0c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05cb7fe700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc0d0b80 0x7f05cc0c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05cb7fe700 1 -- 172.21.15.31:0/3646209243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b4004d50 msgr2=0x7f05cc16cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05cb7fe700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b4004d50 0x7f05cc16cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05cb7fe700 1 -- 172.21.15.31:0/3646209243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc16d8f0 con 0x7f05cc0c47d0 2023-10-06T05:20:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05ca7fc700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc0d0b80 0x7f05cc0c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05cb7fe700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc0c47d0 0x7f05cc0cf1c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f05c4012000 tx=0x7f05c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05b2ffd700 1 -- 172.21.15.31:0/3646209243 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4007590 con 0x7f05cc0c47d0 2023-10-06T05:20:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05b2ffd700 1 -- 172.21.15.31:0/3646209243 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c4001150 con 0x7f05cc0c47d0 2023-10-06T05:20:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05d13ff700 1 -- 172.21.15.31:0/3646209243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc16dbd0 con 0x7f05cc0c47d0 2023-10-06T05:20:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05b2ffd700 1 -- 172.21.15.31:0/3646209243 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c401e680 con 0x7f05cc0c47d0 2023-10-06T05:20:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.429+0000 7f05d13ff700 1 -- 172.21.15.31:0/3646209243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc0cf8c0 con 0x7f05cc0c47d0 2023-10-06T05:20:41.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.431+0000 7f05d13ff700 1 -- 172.21.15.31:0/3646209243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc0cdfd0 con 0x7f05cc0c47d0 2023-10-06T05:20:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.432+0000 7f05b2ffd700 1 -- 172.21.15.31:0/3646209243 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f05c401e9f0 con 0x7f05cc0c47d0 2023-10-06T05:20:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.432+0000 7f05b2ffd700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f05b80670e0 0x7f05b8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.433+0000 7f05b2ffd700 1 -- 172.21.15.31:0/3646209243 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f05c4086ec0 con 0x7f05cc0c47d0 2023-10-06T05:20:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.433+0000 7f05caffd700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f05b80670e0 0x7f05b8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.433+0000 7f05caffd700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f05b80670e0 0x7f05b8069590 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f05bc006f00 tx=0x7f05bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.437+0000 7f05b2ffd700 1 -- 172.21.15.31:0/3646209243 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05cc0cdfd0 con 0x7f05cc0c47d0 2023-10-06T05:20:41.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.634+0000 7f05d13ff700 1 -- 172.21.15.31:0/3646209243 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05cc0cdfd0 con 0x7f05b80670e0 2023-10-06T05:20:41.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.645+0000 7f05b2ffd700 1 -- 172.21.15.31:0/3646209243 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311126 (secure 0 0 0) 0x7f05cc0cdfd0 con 0x7f05b80670e0 2023-10-06T05:20:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.649+0000 7f05d13ff700 1 -- 172.21.15.31:0/3646209243 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f05b80670e0 msgr2=0x7f05b8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.649+0000 7f05d13ff700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f05b80670e0 0x7f05b8069590 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f05bc006f00 tx=0x7f05bc008040 comp rx=0 tx=0).stop 2023-10-06T05:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.649+0000 7f05d13ff700 1 -- 172.21.15.31:0/3646209243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc0c47d0 msgr2=0x7f05cc0cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.649+0000 7f05d13ff700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc0c47d0 0x7f05cc0cf1c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f05c4012000 tx=0x7f05c4016000 comp rx=0 tx=0).stop 2023-10-06T05:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.649+0000 7f05d13ff700 1 -- 172.21.15.31:0/3646209243 shutdown_connections 2023-10-06T05:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.649+0000 7f05d13ff700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f05b80670e0 0x7f05b8069590 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.649+0000 7f05d13ff700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05cc0c47d0 0x7f05cc0cf1c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.649+0000 7f05d13ff700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05b4004d50 0x7f05cc16cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.649+0000 7f05d13ff700 1 --2- 172.21.15.31:0/3646209243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05cc0d0b80 0x7f05cc0c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.649+0000 7f05d13ff700 1 -- 172.21.15.31:0/3646209243 >> 172.21.15.31:0/3646209243 conn(0x7f05cc06ec90 msgr2=0x7f05cc0c54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.650+0000 7f05d13ff700 1 -- 172.21.15.31:0/3646209243 shutdown_connections 2023-10-06T05:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.650+0000 7f05d13ff700 1 -- 172.21.15.31:0/3646209243 wait complete. 2023-10-06T05:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:20:41.671 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2023-10-06T05:20:40.110894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384848,"kb_used_data":6544,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589872,"statfs":{"total":773094113280,"available":770652028928,"internally_reserved":0,"allocated":6701056,"data_stored":3572746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.873223"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089707+0000","last_change":"2023-10-06T05:20:37.081501+0000","last_active":"2023-10-06T05:20:37.089707+0000","last_peered":"2023-10-06T05:20:37.089707+0000","last_clean":"2023-10-06T05:20:37.089707+0000","last_became_active":"2023-10-06T05:20:37.081169+0000","last_became_peered":"2023-10-06T05:20:37.081169+0000","last_unstale":"2023-10-06T05:20:37.089707+0000","last_undegraded":"2023-10-06T05:20:37.089707+0000","last_fullsized":"2023-10-06T05:20:37.089707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:57.659806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.081421+0000","last_change":"2023-10-06T05:20:37.077783+0000","last_active":"2023-10-06T05:20:37.081421+0000","last_peered":"2023-10-06T05:20:37.081421+0000","last_clean":"2023-10-06T05:20:37.081421+0000","last_became_active":"2023-10-06T05:20:37.077520+0000","last_became_peered":"2023-10-06T05:20:37.077520+0000","last_unstale":"2023-10-06T05:20:37.081421+0000","last_undegraded":"2023-10-06T05:20:37.081421+0000","last_fullsized":"2023-10-06T05:20:37.081421+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:53.413645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.087522+0000","last_change":"2023-10-06T05:20:37.085993+0000","last_active":"2023-10-06T05:20:37.087522+0000","last_peered":"2023-10-06T05:20:37.087522+0000","last_clean":"2023-10-06T05:20:37.087522+0000","last_became_active":"2023-10-06T05:20:37.085693+0000","last_became_peered":"2023-10-06T05:20:37.085693+0000","last_unstale":"2023-10-06T05:20:37.087522+0000","last_undegraded":"2023-10-06T05:20:37.087522+0000","last_fullsized":"2023-10-06T05:20:37.087522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:24.005054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.080473+0000","last_change":"2023-10-06T05:20:37.078536+0000","last_active":"2023-10-06T05:20:37.080473+0000","last_peered":"2023-10-06T05:20:37.080473+0000","last_clean":"2023-10-06T05:20:37.080473+0000","last_became_active":"2023-10-06T05:20:37.077817+0000","last_became_peered":"2023-10-06T05:20:37.077817+0000","last_unstale":"2023-10-06T05:20:37.080473+0000","last_undegraded":"2023-10-06T05:20:37.080473+0000","last_fullsized":"2023-10-06T05:20:37.080473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:12.980807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095161+0000","last_change":"2023-10-06T05:20:37.089595+0000","last_active":"2023-10-06T05:20:37.095161+0000","last_peered":"2023-10-06T05:20:37.095161+0000","last_clean":"2023-10-06T05:20:37.095161+0000","last_became_active":"2023-10-06T05:20:37.089331+0000","last_became_peered":"2023-10-06T05:20:37.089331+0000","last_unstale":"2023-10-06T05:20:37.095161+0000","last_undegraded":"2023-10-06T05:20:37.095161+0000","last_fullsized":"2023-10-06T05:20:37.095161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:16.321484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090585+0000","last_change":"2023-10-06T05:20:37.088048+0000","last_active":"2023-10-06T05:20:37.090585+0000","last_peered":"2023-10-06T05:20:37.090585+0000","last_clean":"2023-10-06T05:20:37.090585+0000","last_became_active":"2023-10-06T05:20:37.087837+0000","last_became_peered":"2023-10-06T05:20:37.087837+0000","last_unstale":"2023-10-06T05:20:37.090585+0000","last_undegraded":"2023-10-06T05:20:37.090585+0000","last_fullsized":"2023-10-06T05:20:37.090585+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:31.813149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089800+0000","last_change":"2023-10-06T05:20:37.084546+0000","last_active":"2023-10-06T05:20:37.089800+0000","last_peered":"2023-10-06T05:20:37.089800+0000","last_clean":"2023-10-06T05:20:37.089800+0000","last_became_active":"2023-10-06T05:20:37.084287+0000","last_became_peered":"2023-10-06T05:20:37.084287+0000","last_unstale":"2023-10-06T05:20:37.089800+0000","last_undegraded":"2023-10-06T05:20:37.089800+0000","last_fullsized":"2023-10-06T05:20:37.089800+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:55.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084722+0000","last_change":"2023-10-06T05:20:37.078343+0000","last_active":"2023-10-06T05:20:37.084722+0000","last_peered":"2023-10-06T05:20:37.084722+0000","last_clean":"2023-10-06T05:20:37.084722+0000","last_became_active":"2023-10-06T05:20:37.077990+0000","last_became_peered":"2023-10-06T05:20:37.077990+0000","last_unstale":"2023-10-06T05:20:37.084722+0000","last_undegraded":"2023-10-06T05:20:37.084722+0000","last_fullsized":"2023-10-06T05:20:37.084722+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:46.285631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.097412+0000","last_change":"2023-10-06T05:20:37.089003+0000","last_active":"2023-10-06T05:20:37.097412+0000","last_peered":"2023-10-06T05:20:37.097412+0000","last_clean":"2023-10-06T05:20:37.097412+0000","last_became_active":"2023-10-06T05:20:37.088727+0000","last_became_peered":"2023-10-06T05:20:37.088727+0000","last_unstale":"2023-10-06T05:20:37.097412+0000","last_undegraded":"2023-10-06T05:20:37.097412+0000","last_fullsized":"2023-10-06T05:20:37.097412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:43.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.096917+0000","last_change":"2023-10-06T05:20:37.087165+0000","last_active":"2023-10-06T05:20:37.096917+0000","last_peered":"2023-10-06T05:20:37.096917+0000","last_clean":"2023-10-06T05:20:37.096917+0000","last_became_active":"2023-10-06T05:20:37.086879+0000","last_became_peered":"2023-10-06T05:20:37.086879+0000","last_unstale":"2023-10-06T05:20:37.096917+0000","last_undegraded":"2023-10-06T05:20:37.096917+0000","last_fullsized":"2023-10-06T05:20:37.096917+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:43.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.080503+0000","last_change":"2023-10-06T05:20:37.077823+0000","last_active":"2023-10-06T05:20:37.080503+0000","last_peered":"2023-10-06T05:20:37.080503+0000","last_clean":"2023-10-06T05:20:37.080503+0000","last_became_active":"2023-10-06T05:20:37.077476+0000","last_became_peered":"2023-10-06T05:20:37.077476+0000","last_unstale":"2023-10-06T05:20:37.080503+0000","last_undegraded":"2023-10-06T05:20:37.080503+0000","last_fullsized":"2023-10-06T05:20:37.080503+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:45.587669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090505+0000","last_change":"2023-10-06T05:20:37.082144+0000","last_active":"2023-10-06T05:20:37.090505+0000","last_peered":"2023-10-06T05:20:37.090505+0000","last_clean":"2023-10-06T05:20:37.090505+0000","last_became_active":"2023-10-06T05:20:37.081819+0000","last_became_peered":"2023-10-06T05:20:37.081819+0000","last_unstale":"2023-10-06T05:20:37.090505+0000","last_undegraded":"2023-10-06T05:20:37.090505+0000","last_fullsized":"2023-10-06T05:20:37.090505+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:22.108891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.087577+0000","last_change":"2023-10-06T05:20:37.080639+0000","last_active":"2023-10-06T05:20:37.087577+0000","last_peered":"2023-10-06T05:20:37.087577+0000","last_clean":"2023-10-06T05:20:37.087577+0000","last_became_active":"2023-10-06T05:20:37.080314+0000","last_became_peered":"2023-10-06T05:20:37.080314+0000","last_unstale":"2023-10-06T05:20:37.087577+0000","last_undegraded":"2023-10-06T05:20:37.087577+0000","last_fullsized":"2023-10-06T05:20:37.087577+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:36.557041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.087835+0000","last_change":"2023-10-06T05:20:37.086145+0000","last_active":"2023-10-06T05:20:37.087835+0000","last_peered":"2023-10-06T05:20:37.087835+0000","last_clean":"2023-10-06T05:20:37.087835+0000","last_became_active":"2023-10-06T05:20:37.085783+0000","last_became_peered":"2023-10-06T05:20:37.085783+0000","last_unstale":"2023-10-06T05:20:37.087835+0000","last_undegraded":"2023-10-06T05:20:37.087835+0000","last_fullsized":"2023-10-06T05:20:37.087835+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:34.455912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.097302+0000","last_change":"2023-10-06T05:20:37.087958+0000","last_active":"2023-10-06T05:20:37.097302+0000","last_peered":"2023-10-06T05:20:37.097302+0000","last_clean":"2023-10-06T05:20:37.097302+0000","last_became_active":"2023-10-06T05:20:37.087368+0000","last_became_peered":"2023-10-06T05:20:37.087368+0000","last_unstale":"2023-10-06T05:20:37.097302+0000","last_undegraded":"2023-10-06T05:20:37.097302+0000","last_fullsized":"2023-10-06T05:20:37.097302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:06.140443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084792+0000","last_change":"2023-10-06T05:20:37.076365+0000","last_active":"2023-10-06T05:20:37.084792+0000","last_peered":"2023-10-06T05:20:37.084792+0000","last_clean":"2023-10-06T05:20:37.084792+0000","last_became_active":"2023-10-06T05:20:37.075914+0000","last_became_peered":"2023-10-06T05:20:37.075914+0000","last_unstale":"2023-10-06T05:20:37.084792+0000","last_undegraded":"2023-10-06T05:20:37.084792+0000","last_fullsized":"2023-10-06T05:20:37.084792+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:53.721927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089725+0000","last_change":"2023-10-06T05:20:37.082961+0000","last_active":"2023-10-06T05:20:37.089725+0000","last_peered":"2023-10-06T05:20:37.089725+0000","last_clean":"2023-10-06T05:20:37.089725+0000","last_became_active":"2023-10-06T05:20:37.082561+0000","last_became_peered":"2023-10-06T05:20:37.082561+0000","last_unstale":"2023-10-06T05:20:37.089725+0000","last_undegraded":"2023-10-06T05:20:37.089725+0000","last_fullsized":"2023-10-06T05:20:37.089725+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:32.245529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.088731+0000","last_change":"2023-10-06T05:20:37.086011+0000","last_active":"2023-10-06T05:20:37.088731+0000","last_peered":"2023-10-06T05:20:37.088731+0000","last_clean":"2023-10-06T05:20:37.088731+0000","last_became_active":"2023-10-06T05:20:37.085730+0000","last_became_peered":"2023-10-06T05:20:37.085730+0000","last_unstale":"2023-10-06T05:20:37.088731+0000","last_undegraded":"2023-10-06T05:20:37.088731+0000","last_fullsized":"2023-10-06T05:20:37.088731+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:01.873449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095549+0000","last_change":"2023-10-06T05:20:37.090042+0000","last_active":"2023-10-06T05:20:37.095549+0000","last_peered":"2023-10-06T05:20:37.095549+0000","last_clean":"2023-10-06T05:20:37.095549+0000","last_became_active":"2023-10-06T05:20:37.089638+0000","last_became_peered":"2023-10-06T05:20:37.089638+0000","last_unstale":"2023-10-06T05:20:37.095549+0000","last_undegraded":"2023-10-06T05:20:37.095549+0000","last_fullsized":"2023-10-06T05:20:37.095549+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:40.605095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095717+0000","last_change":"2023-10-06T05:20:37.091621+0000","last_active":"2023-10-06T05:20:37.095717+0000","last_peered":"2023-10-06T05:20:37.095717+0000","last_clean":"2023-10-06T05:20:37.095717+0000","last_became_active":"2023-10-06T05:20:37.091389+0000","last_became_peered":"2023-10-06T05:20:37.091389+0000","last_unstale":"2023-10-06T05:20:37.095717+0000","last_undegraded":"2023-10-06T05:20:37.095717+0000","last_fullsized":"2023-10-06T05:20:37.095717+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:24.005054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090066+0000","last_change":"2023-10-06T05:20:37.082625+0000","last_active":"2023-10-06T05:20:37.090066+0000","last_peered":"2023-10-06T05:20:37.090066+0000","last_clean":"2023-10-06T05:20:37.090066+0000","last_became_active":"2023-10-06T05:20:37.081398+0000","last_became_peered":"2023-10-06T05:20:37.081398+0000","last_unstale":"2023-10-06T05:20:37.090066+0000","last_undegraded":"2023-10-06T05:20:37.090066+0000","last_fullsized":"2023-10-06T05:20:37.090066+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:45.130913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090187+0000","last_change":"2023-10-06T05:20:37.089068+0000","last_active":"2023-10-06T05:20:37.090187+0000","last_peered":"2023-10-06T05:20:37.090187+0000","last_clean":"2023-10-06T05:20:37.090187+0000","last_became_active":"2023-10-06T05:20:37.088773+0000","last_became_peered":"2023-10-06T05:20:37.088773+0000","last_unstale":"2023-10-06T05:20:37.090187+0000","last_undegraded":"2023-10-06T05:20:37.090187+0000","last_fullsized":"2023-10-06T05:20:37.090187+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:34.455912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090138+0000","last_change":"2023-10-06T05:20:37.089105+0000","last_active":"2023-10-06T05:20:37.090138+0000","last_peered":"2023-10-06T05:20:37.090138+0000","last_clean":"2023-10-06T05:20:37.090138+0000","last_became_active":"2023-10-06T05:20:37.088785+0000","last_became_peered":"2023-10-06T05:20:37.088785+0000","last_unstale":"2023-10-06T05:20:37.090138+0000","last_undegraded":"2023-10-06T05:20:37.090138+0000","last_fullsized":"2023-10-06T05:20:37.090138+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:06.324052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.087336+0000","last_change":"2023-10-06T05:20:37.085079+0000","last_active":"2023-10-06T05:20:37.087336+0000","last_peered":"2023-10-06T05:20:37.087336+0000","last_clean":"2023-10-06T05:20:37.087336+0000","last_became_active":"2023-10-06T05:20:37.083960+0000","last_became_peered":"2023-10-06T05:20:37.083960+0000","last_unstale":"2023-10-06T05:20:37.087336+0000","last_undegraded":"2023-10-06T05:20:37.087336+0000","last_fullsized":"2023-10-06T05:20:37.087336+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:06.324052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.088058+0000","last_change":"2023-10-06T05:20:37.081143+0000","last_active":"2023-10-06T05:20:37.088058+0000","last_peered":"2023-10-06T05:20:37.088058+0000","last_clean":"2023-10-06T05:20:37.088058+0000","last_became_active":"2023-10-06T05:20:37.080806+0000","last_became_peered":"2023-10-06T05:20:37.080806+0000","last_unstale":"2023-10-06T05:20:37.088058+0000","last_undegraded":"2023-10-06T05:20:37.088058+0000","last_fullsized":"2023-10-06T05:20:37.088058+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:09.413283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090000+0000","last_change":"2023-10-06T05:20:37.082372+0000","last_active":"2023-10-06T05:20:37.090000+0000","last_peered":"2023-10-06T05:20:37.090000+0000","last_clean":"2023-10-06T05:20:37.090000+0000","last_became_active":"2023-10-06T05:20:37.081777+0000","last_became_peered":"2023-10-06T05:20:37.081777+0000","last_unstale":"2023-10-06T05:20:37.090000+0000","last_undegraded":"2023-10-06T05:20:37.090000+0000","last_fullsized":"2023-10-06T05:20:37.090000+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:19.678862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084312+0000","last_change":"2023-10-06T05:20:37.080029+0000","last_active":"2023-10-06T05:20:37.084312+0000","last_peered":"2023-10-06T05:20:37.084312+0000","last_clean":"2023-10-06T05:20:37.084312+0000","last_became_active":"2023-10-06T05:20:37.079637+0000","last_became_peered":"2023-10-06T05:20:37.079637+0000","last_unstale":"2023-10-06T05:20:37.084312+0000","last_undegraded":"2023-10-06T05:20:37.084312+0000","last_fullsized":"2023-10-06T05:20:37.084312+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:23.511829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.096161+0000","last_change":"2023-10-06T05:20:37.094029+0000","last_active":"2023-10-06T05:20:37.096161+0000","last_peered":"2023-10-06T05:20:37.096161+0000","last_clean":"2023-10-06T05:20:37.096161+0000","last_became_active":"2023-10-06T05:20:37.093799+0000","last_became_peered":"2023-10-06T05:20:37.093799+0000","last_unstale":"2023-10-06T05:20:37.096161+0000","last_undegraded":"2023-10-06T05:20:37.096161+0000","last_fullsized":"2023-10-06T05:20:37.096161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:57.053661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.096296+0000","last_change":"2023-10-06T05:20:37.087036+0000","last_active":"2023-10-06T05:20:37.096296+0000","last_peered":"2023-10-06T05:20:37.096296+0000","last_clean":"2023-10-06T05:20:37.096296+0000","last_became_active":"2023-10-06T05:20:37.086691+0000","last_became_peered":"2023-10-06T05:20:37.086691+0000","last_unstale":"2023-10-06T05:20:37.096296+0000","last_undegraded":"2023-10-06T05:20:37.096296+0000","last_fullsized":"2023-10-06T05:20:37.096296+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:07.490551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090655+0000","last_change":"2023-10-06T05:20:37.088904+0000","last_active":"2023-10-06T05:20:37.090655+0000","last_peered":"2023-10-06T05:20:37.090655+0000","last_clean":"2023-10-06T05:20:37.090655+0000","last_became_active":"2023-10-06T05:20:37.088624+0000","last_became_peered":"2023-10-06T05:20:37.088624+0000","last_unstale":"2023-10-06T05:20:37.090655+0000","last_undegraded":"2023-10-06T05:20:37.090655+0000","last_fullsized":"2023-10-06T05:20:37.090655+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:05.044908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.083131+0000","last_change":"2023-10-06T05:20:37.080032+0000","last_active":"2023-10-06T05:20:37.083131+0000","last_peered":"2023-10-06T05:20:37.083131+0000","last_clean":"2023-10-06T05:20:37.083131+0000","last_became_active":"2023-10-06T05:20:37.079743+0000","last_became_peered":"2023-10-06T05:20:37.079743+0000","last_unstale":"2023-10-06T05:20:37.083131+0000","last_undegraded":"2023-10-06T05:20:37.083131+0000","last_fullsized":"2023-10-06T05:20:37.083131+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:28.844309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.076936+0000","last_change":"2023-10-06T05:20:37.073826+0000","last_active":"2023-10-06T05:20:37.076936+0000","last_peered":"2023-10-06T05:20:37.076936+0000","last_clean":"2023-10-06T05:20:37.076936+0000","last_became_active":"2023-10-06T05:20:37.073380+0000","last_became_peered":"2023-10-06T05:20:37.073380+0000","last_unstale":"2023-10-06T05:20:37.076936+0000","last_undegraded":"2023-10-06T05:20:37.076936+0000","last_fullsized":"2023-10-06T05:20:37.076936+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:28.307492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095743+0000","last_change":"2023-10-06T05:20:37.093548+0000","last_active":"2023-10-06T05:20:37.095743+0000","last_peered":"2023-10-06T05:20:37.095743+0000","last_clean":"2023-10-06T05:20:37.095743+0000","last_became_active":"2023-10-06T05:20:37.092348+0000","last_became_peered":"2023-10-06T05:20:37.092348+0000","last_unstale":"2023-10-06T05:20:37.095743+0000","last_undegraded":"2023-10-06T05:20:37.095743+0000","last_fullsized":"2023-10-06T05:20:37.095743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:56.892725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"56'4","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.269664+0000","last_change":"2023-10-06T05:20:36.263537+0000","last_active":"2023-10-06T05:20:37.269664+0000","last_peered":"2023-10-06T05:20:37.269664+0000","last_clean":"2023-10-06T05:20:37.269664+0000","last_became_active":"2023-10-06T05:20:36.263078+0000","last_became_peered":"2023-10-06T05:20:36.263078+0000","last_unstale":"2023-10-06T05:20:37.269664+0000","last_undegraded":"2023-10-06T05:20:37.269664+0000","last_fullsized":"2023-10-06T05:20:37.269664+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:15.315585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.097883+0000","last_change":"2023-10-06T05:20:37.096408+0000","last_active":"2023-10-06T05:20:37.097883+0000","last_peered":"2023-10-06T05:20:37.097883+0000","last_clean":"2023-10-06T05:20:37.097883+0000","last_became_active":"2023-10-06T05:20:37.095989+0000","last_became_peered":"2023-10-06T05:20:37.095989+0000","last_unstale":"2023-10-06T05:20:37.097883+0000","last_undegraded":"2023-10-06T05:20:37.097883+0000","last_fullsized":"2023-10-06T05:20:37.097883+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:45.130913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"56'1","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.267701+0000","last_change":"2023-10-06T05:20:36.262702+0000","last_active":"2023-10-06T05:20:37.267701+0000","last_peered":"2023-10-06T05:20:37.267701+0000","last_clean":"2023-10-06T05:20:37.267701+0000","last_became_active":"2023-10-06T05:20:36.262219+0000","last_became_peered":"2023-10-06T05:20:36.262219+0000","last_unstale":"2023-10-06T05:20:37.267701+0000","last_undegraded":"2023-10-06T05:20:37.267701+0000","last_fullsized":"2023-10-06T05:20:37.267701+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:58.138911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.075135+0000","last_change":"2023-10-06T05:20:37.069595+0000","last_active":"2023-10-06T05:20:37.075135+0000","last_peered":"2023-10-06T05:20:37.075135+0000","last_clean":"2023-10-06T05:20:37.075135+0000","last_became_active":"2023-10-06T05:20:37.069350+0000","last_became_peered":"2023-10-06T05:20:37.069350+0000","last_unstale":"2023-10-06T05:20:37.075135+0000","last_undegraded":"2023-10-06T05:20:37.075135+0000","last_fullsized":"2023-10-06T05:20:37.075135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:35.259063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.269104+0000","last_change":"2023-10-06T05:20:36.267484+0000","last_active":"2023-10-06T05:20:37.269104+0000","last_peered":"2023-10-06T05:20:37.269104+0000","last_clean":"2023-10-06T05:20:37.269104+0000","last_became_active":"2023-10-06T05:20:36.267135+0000","last_became_peered":"2023-10-06T05:20:36.267135+0000","last_unstale":"2023-10-06T05:20:37.269104+0000","last_undegraded":"2023-10-06T05:20:37.269104+0000","last_fullsized":"2023-10-06T05:20:37.269104+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:08.681506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.097368+0000","last_change":"2023-10-06T05:20:37.088927+0000","last_active":"2023-10-06T05:20:37.097368+0000","last_peered":"2023-10-06T05:20:37.097368+0000","last_clean":"2023-10-06T05:20:37.097368+0000","last_became_active":"2023-10-06T05:20:37.088720+0000","last_became_peered":"2023-10-06T05:20:37.088720+0000","last_unstale":"2023-10-06T05:20:37.097368+0000","last_undegraded":"2023-10-06T05:20:37.097368+0000","last_fullsized":"2023-10-06T05:20:37.097368+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:59.144810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.266734+0000","last_change":"2023-10-06T05:20:36.258267+0000","last_active":"2023-10-06T05:20:37.266734+0000","last_peered":"2023-10-06T05:20:37.266734+0000","last_clean":"2023-10-06T05:20:37.266734+0000","last_became_active":"2023-10-06T05:20:36.258006+0000","last_became_peered":"2023-10-06T05:20:36.258006+0000","last_unstale":"2023-10-06T05:20:37.266734+0000","last_undegraded":"2023-10-06T05:20:37.266734+0000","last_fullsized":"2023-10-06T05:20:37.266734+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:08.407384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084820+0000","last_change":"2023-10-06T05:20:37.077780+0000","last_active":"2023-10-06T05:20:37.084820+0000","last_peered":"2023-10-06T05:20:37.084820+0000","last_clean":"2023-10-06T05:20:37.084820+0000","last_became_active":"2023-10-06T05:20:37.077526+0000","last_became_peered":"2023-10-06T05:20:37.077526+0000","last_unstale":"2023-10-06T05:20:37.084820+0000","last_undegraded":"2023-10-06T05:20:37.084820+0000","last_fullsized":"2023-10-06T05:20:37.084820+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:17.450471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"56'1","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.266066+0000","last_change":"2023-10-06T05:20:36.254716+0000","last_active":"2023-10-06T05:20:37.266066+0000","last_peered":"2023-10-06T05:20:37.266066+0000","last_clean":"2023-10-06T05:20:37.266066+0000","last_became_active":"2023-10-06T05:20:36.254166+0000","last_became_peered":"2023-10-06T05:20:36.254166+0000","last_unstale":"2023-10-06T05:20:37.266066+0000","last_undegraded":"2023-10-06T05:20:37.266066+0000","last_fullsized":"2023-10-06T05:20:37.266066+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:41.018227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089538+0000","last_change":"2023-10-06T05:20:37.082671+0000","last_active":"2023-10-06T05:20:37.089538+0000","last_peered":"2023-10-06T05:20:37.089538+0000","last_clean":"2023-10-06T05:20:37.089538+0000","last_became_active":"2023-10-06T05:20:37.082366+0000","last_became_peered":"2023-10-06T05:20:37.082366+0000","last_unstale":"2023-10-06T05:20:37.089538+0000","last_undegraded":"2023-10-06T05:20:37.089538+0000","last_fullsized":"2023-10-06T05:20:37.089538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:00.119383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"56'1","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.265918+0000","last_change":"2023-10-06T05:20:36.258045+0000","last_active":"2023-10-06T05:20:37.265918+0000","last_peered":"2023-10-06T05:20:37.265918+0000","last_clean":"2023-10-06T05:20:37.265918+0000","last_became_active":"2023-10-06T05:20:36.257730+0000","last_became_peered":"2023-10-06T05:20:36.257730+0000","last_unstale":"2023-10-06T05:20:37.265918+0000","last_undegraded":"2023-10-06T05:20:37.265918+0000","last_fullsized":"2023-10-06T05:20:37.265918+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:35.551141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095613+0000","last_change":"2023-10-06T05:20:37.090269+0000","last_active":"2023-10-06T05:20:37.095613+0000","last_peered":"2023-10-06T05:20:37.095613+0000","last_clean":"2023-10-06T05:20:37.095613+0000","last_became_active":"2023-10-06T05:20:37.089436+0000","last_became_peered":"2023-10-06T05:20:37.089436+0000","last_unstale":"2023-10-06T05:20:37.095613+0000","last_undegraded":"2023-10-06T05:20:37.095613+0000","last_fullsized":"2023-10-06T05:20:37.095613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:36.510841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.064669+0000","last_change":"2023-10-06T05:20:36.261897+0000","last_active":"2023-10-06T05:20:37.064669+0000","last_peered":"2023-10-06T05:20:37.064669+0000","last_clean":"2023-10-06T05:20:37.064669+0000","last_became_active":"2023-10-06T05:20:36.261603+0000","last_became_peered":"2023-10-06T05:20:36.261603+0000","last_unstale":"2023-10-06T05:20:37.064669+0000","last_undegraded":"2023-10-06T05:20:37.064669+0000","last_fullsized":"2023-10-06T05:20:37.064669+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:37.420899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.081722+0000","last_change":"2023-10-06T05:20:37.078582+0000","last_active":"2023-10-06T05:20:37.081722+0000","last_peered":"2023-10-06T05:20:37.081722+0000","last_clean":"2023-10-06T05:20:37.081722+0000","last_became_active":"2023-10-06T05:20:37.078114+0000","last_became_peered":"2023-10-06T05:20:37.078114+0000","last_unstale":"2023-10-06T05:20:37.081722+0000","last_undegraded":"2023-10-06T05:20:37.081722+0000","last_fullsized":"2023-10-06T05:20:37.081722+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:26.019845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.085729+0000","last_change":"2023-10-06T05:20:36.262979+0000","last_active":"2023-10-06T05:20:37.085729+0000","last_peered":"2023-10-06T05:20:37.085729+0000","last_clean":"2023-10-06T05:20:37.085729+0000","last_became_active":"2023-10-06T05:20:36.262645+0000","last_became_peered":"2023-10-06T05:20:36.262645+0000","last_unstale":"2023-10-06T05:20:37.085729+0000","last_undegraded":"2023-10-06T05:20:37.085729+0000","last_fullsized":"2023-10-06T05:20:37.085729+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:49.246636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.085046+0000","last_change":"2023-10-06T05:20:37.080406+0000","last_active":"2023-10-06T05:20:37.085046+0000","last_peered":"2023-10-06T05:20:37.085046+0000","last_clean":"2023-10-06T05:20:37.085046+0000","last_became_active":"2023-10-06T05:20:37.080067+0000","last_became_peered":"2023-10-06T05:20:37.080067+0000","last_unstale":"2023-10-06T05:20:37.085046+0000","last_undegraded":"2023-10-06T05:20:37.085046+0000","last_fullsized":"2023-10-06T05:20:37.085046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:53.917741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.073446+0000","last_change":"2023-10-06T05:20:36.261789+0000","last_active":"2023-10-06T05:20:37.073446+0000","last_peered":"2023-10-06T05:20:37.073446+0000","last_clean":"2023-10-06T05:20:37.073446+0000","last_became_active":"2023-10-06T05:20:36.261531+0000","last_became_peered":"2023-10-06T05:20:36.261531+0000","last_unstale":"2023-10-06T05:20:37.073446+0000","last_undegraded":"2023-10-06T05:20:37.073446+0000","last_fullsized":"2023-10-06T05:20:37.073446+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:38.304771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.077534+0000","last_change":"2023-10-06T05:20:37.076107+0000","last_active":"2023-10-06T05:20:37.077534+0000","last_peered":"2023-10-06T05:20:37.077534+0000","last_clean":"2023-10-06T05:20:37.077534+0000","last_became_active":"2023-10-06T05:20:37.075776+0000","last_became_peered":"2023-10-06T05:20:37.075776+0000","last_unstale":"2023-10-06T05:20:37.077534+0000","last_undegraded":"2023-10-06T05:20:37.077534+0000","last_fullsized":"2023-10-06T05:20:37.077534+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:43.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.076605+0000","last_change":"2023-10-06T05:20:36.265722+0000","last_active":"2023-10-06T05:20:37.076605+0000","last_peered":"2023-10-06T05:20:37.076605+0000","last_clean":"2023-10-06T05:20:37.076605+0000","last_became_active":"2023-10-06T05:20:36.265182+0000","last_became_peered":"2023-10-06T05:20:36.265182+0000","last_unstale":"2023-10-06T05:20:37.076605+0000","last_undegraded":"2023-10-06T05:20:37.076605+0000","last_fullsized":"2023-10-06T05:20:37.076605+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:48.090020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095714+0000","last_change":"2023-10-06T05:20:37.094118+0000","last_active":"2023-10-06T05:20:37.095714+0000","last_peered":"2023-10-06T05:20:37.095714+0000","last_clean":"2023-10-06T05:20:37.095714+0000","last_became_active":"2023-10-06T05:20:37.093803+0000","last_became_peered":"2023-10-06T05:20:37.093803+0000","last_unstale":"2023-10-06T05:20:37.095714+0000","last_undegraded":"2023-10-06T05:20:37.095714+0000","last_fullsized":"2023-10-06T05:20:37.095714+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:22.698300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"56'3","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.268071+0000","last_change":"2023-10-06T05:20:36.261993+0000","last_active":"2023-10-06T05:20:37.268071+0000","last_peered":"2023-10-06T05:20:37.268071+0000","last_clean":"2023-10-06T05:20:37.268071+0000","last_became_active":"2023-10-06T05:20:36.261433+0000","last_became_peered":"2023-10-06T05:20:36.261433+0000","last_unstale":"2023-10-06T05:20:37.268071+0000","last_undegraded":"2023-10-06T05:20:37.268071+0000","last_fullsized":"2023-10-06T05:20:37.268071+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:11.974908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084755+0000","last_change":"2023-10-06T05:20:37.081584+0000","last_active":"2023-10-06T05:20:37.084755+0000","last_peered":"2023-10-06T05:20:37.084755+0000","last_clean":"2023-10-06T05:20:37.084755+0000","last_became_active":"2023-10-06T05:20:37.081298+0000","last_became_peered":"2023-10-06T05:20:37.081298+0000","last_unstale":"2023-10-06T05:20:37.084755+0000","last_undegraded":"2023-10-06T05:20:37.084755+0000","last_fullsized":"2023-10-06T05:20:37.084755+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:39.310670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"56'6","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.271413+0000","last_change":"2023-10-06T05:20:36.260052+0000","last_active":"2023-10-06T05:20:37.271413+0000","last_peered":"2023-10-06T05:20:37.271413+0000","last_clean":"2023-10-06T05:20:37.271413+0000","last_became_active":"2023-10-06T05:20:36.259719+0000","last_became_peered":"2023-10-06T05:20:36.259719+0000","last_unstale":"2023-10-06T05:20:37.271413+0000","last_undegraded":"2023-10-06T05:20:37.271413+0000","last_fullsized":"2023-10-06T05:20:37.271413+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:54.688059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.094675+0000","last_change":"2023-10-06T05:20:37.091989+0000","last_active":"2023-10-06T05:20:37.094675+0000","last_peered":"2023-10-06T05:20:37.094675+0000","last_clean":"2023-10-06T05:20:37.094675+0000","last_became_active":"2023-10-06T05:20:37.091713+0000","last_became_peered":"2023-10-06T05:20:37.091713+0000","last_unstale":"2023-10-06T05:20:37.094675+0000","last_undegraded":"2023-10-06T05:20:37.094675+0000","last_fullsized":"2023-10-06T05:20:37.094675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:36.557041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.269794+0000","last_change":"2023-10-06T05:20:36.262413+0000","last_active":"2023-10-06T05:20:37.269794+0000","last_peered":"2023-10-06T05:20:37.269794+0000","last_clean":"2023-10-06T05:20:37.269794+0000","last_became_active":"2023-10-06T05:20:36.262111+0000","last_became_peered":"2023-10-06T05:20:36.262111+0000","last_unstale":"2023-10-06T05:20:37.269794+0000","last_undegraded":"2023-10-06T05:20:37.269794+0000","last_fullsized":"2023-10-06T05:20:37.269794+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:14.895156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.088417+0000","last_change":"2023-10-06T05:20:37.079786+0000","last_active":"2023-10-06T05:20:37.088417+0000","last_peered":"2023-10-06T05:20:37.088417+0000","last_clean":"2023-10-06T05:20:37.088417+0000","last_became_active":"2023-10-06T05:20:37.079160+0000","last_became_peered":"2023-10-06T05:20:37.079160+0000","last_unstale":"2023-10-06T05:20:37.088417+0000","last_undegraded":"2023-10-06T05:20:37.088417+0000","last_fullsized":"2023-10-06T05:20:37.088417+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:23.511829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.071864+0000","last_change":"2023-10-06T05:20:36.265000+0000","last_active":"2023-10-06T05:20:37.071864+0000","last_peered":"2023-10-06T05:20:37.071864+0000","last_clean":"2023-10-06T05:20:37.071864+0000","last_became_active":"2023-10-06T05:20:36.264733+0000","last_became_peered":"2023-10-06T05:20:36.264733+0000","last_unstale":"2023-10-06T05:20:37.071864+0000","last_undegraded":"2023-10-06T05:20:37.071864+0000","last_fullsized":"2023-10-06T05:20:37.071864+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:52.911842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095588+0000","last_change":"2023-10-06T05:20:37.090065+0000","last_active":"2023-10-06T05:20:37.095588+0000","last_peered":"2023-10-06T05:20:37.095588+0000","last_clean":"2023-10-06T05:20:37.095588+0000","last_became_active":"2023-10-06T05:20:37.089032+0000","last_became_peered":"2023-10-06T05:20:37.089032+0000","last_unstale":"2023-10-06T05:20:37.095588+0000","last_undegraded":"2023-10-06T05:20:37.095588+0000","last_fullsized":"2023-10-06T05:20:37.095588+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:36.693475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.072838+0000","last_change":"2023-10-06T05:20:36.262438+0000","last_active":"2023-10-06T05:20:37.072838+0000","last_peered":"2023-10-06T05:20:37.072838+0000","last_clean":"2023-10-06T05:20:37.072838+0000","last_became_active":"2023-10-06T05:20:36.261896+0000","last_became_peered":"2023-10-06T05:20:36.261896+0000","last_unstale":"2023-10-06T05:20:37.072838+0000","last_undegraded":"2023-10-06T05:20:37.072838+0000","last_fullsized":"2023-10-06T05:20:37.072838+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:03.522991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090317+0000","last_change":"2023-10-06T05:20:37.089025+0000","last_active":"2023-10-06T05:20:37.090317+0000","last_peered":"2023-10-06T05:20:37.090317+0000","last_clean":"2023-10-06T05:20:37.090317+0000","last_became_active":"2023-10-06T05:20:37.088747+0000","last_became_peered":"2023-10-06T05:20:37.088747+0000","last_unstale":"2023-10-06T05:20:37.090317+0000","last_undegraded":"2023-10-06T05:20:37.090317+0000","last_fullsized":"2023-10-06T05:20:37.090317+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:45.388735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"56'8","reported_seq":18,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.270992+0000","last_change":"2023-10-06T05:20:36.268415+0000","last_active":"2023-10-06T05:20:37.270992+0000","last_peered":"2023-10-06T05:20:37.270992+0000","last_clean":"2023-10-06T05:20:37.270992+0000","last_became_active":"2023-10-06T05:20:36.268002+0000","last_became_peered":"2023-10-06T05:20:36.268002+0000","last_unstale":"2023-10-06T05:20:37.270992+0000","last_undegraded":"2023-10-06T05:20:37.270992+0000","last_fullsized":"2023-10-06T05:20:37.270992+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:59.999307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.085147+0000","last_change":"2023-10-06T05:20:37.078999+0000","last_active":"2023-10-06T05:20:37.085147+0000","last_peered":"2023-10-06T05:20:37.085147+0000","last_clean":"2023-10-06T05:20:37.085147+0000","last_became_active":"2023-10-06T05:20:37.078672+0000","last_became_peered":"2023-10-06T05:20:37.078672+0000","last_unstale":"2023-10-06T05:20:37.085147+0000","last_undegraded":"2023-10-06T05:20:37.085147+0000","last_fullsized":"2023-10-06T05:20:37.085147+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:56.599683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.267857+0000","last_change":"2023-10-06T05:20:36.270612+0000","last_active":"2023-10-06T05:20:37.267857+0000","last_peered":"2023-10-06T05:20:37.267857+0000","last_clean":"2023-10-06T05:20:37.267857+0000","last_became_active":"2023-10-06T05:20:36.270240+0000","last_became_peered":"2023-10-06T05:20:36.270240+0000","last_unstale":"2023-10-06T05:20:37.267857+0000","last_undegraded":"2023-10-06T05:20:37.267857+0000","last_fullsized":"2023-10-06T05:20:37.267857+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:59.113484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.079467+0000","last_change":"2023-10-06T05:20:37.072546+0000","last_active":"2023-10-06T05:20:37.079467+0000","last_peered":"2023-10-06T05:20:37.079467+0000","last_clean":"2023-10-06T05:20:37.079467+0000","last_became_active":"2023-10-06T05:20:37.072095+0000","last_became_peered":"2023-10-06T05:20:37.072095+0000","last_unstale":"2023-10-06T05:20:37.079467+0000","last_undegraded":"2023-10-06T05:20:37.079467+0000","last_fullsized":"2023-10-06T05:20:37.079467+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:57.809186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.072151+0000","last_change":"2023-10-06T05:20:36.264173+0000","last_active":"2023-10-06T05:20:37.072151+0000","last_peered":"2023-10-06T05:20:37.072151+0000","last_clean":"2023-10-06T05:20:37.072151+0000","last_became_active":"2023-10-06T05:20:36.263863+0000","last_became_peered":"2023-10-06T05:20:36.263863+0000","last_unstale":"2023-10-06T05:20:37.072151+0000","last_undegraded":"2023-10-06T05:20:37.072151+0000","last_fullsized":"2023-10-06T05:20:37.072151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:52.407745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089970+0000","last_change":"2023-10-06T05:20:37.082151+0000","last_active":"2023-10-06T05:20:37.089970+0000","last_peered":"2023-10-06T05:20:37.089970+0000","last_clean":"2023-10-06T05:20:37.089970+0000","last_became_active":"2023-10-06T05:20:37.081358+0000","last_became_peered":"2023-10-06T05:20:37.081358+0000","last_unstale":"2023-10-06T05:20:37.089970+0000","last_undegraded":"2023-10-06T05:20:37.089970+0000","last_fullsized":"2023-10-06T05:20:37.089970+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:42.024126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.065894+0000","last_change":"2023-10-06T05:20:36.267479+0000","last_active":"2023-10-06T05:20:37.065894+0000","last_peered":"2023-10-06T05:20:37.065894+0000","last_clean":"2023-10-06T05:20:37.065894+0000","last_became_active":"2023-10-06T05:20:36.267147+0000","last_became_peered":"2023-10-06T05:20:36.267147+0000","last_unstale":"2023-10-06T05:20:37.065894+0000","last_undegraded":"2023-10-06T05:20:37.065894+0000","last_fullsized":"2023-10-06T05:20:37.065894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:45.279732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095680+0000","last_change":"2023-10-06T05:20:37.091301+0000","last_active":"2023-10-06T05:20:37.095680+0000","last_peered":"2023-10-06T05:20:37.095680+0000","last_clean":"2023-10-06T05:20:37.095680+0000","last_became_active":"2023-10-06T05:20:37.091039+0000","last_became_peered":"2023-10-06T05:20:37.091039+0000","last_unstale":"2023-10-06T05:20:37.095680+0000","last_undegraded":"2023-10-06T05:20:37.095680+0000","last_fullsized":"2023-10-06T05:20:37.095680+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:39.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.073450+0000","last_change":"2023-10-06T05:20:36.259657+0000","last_active":"2023-10-06T05:20:37.073450+0000","last_peered":"2023-10-06T05:20:37.073450+0000","last_clean":"2023-10-06T05:20:37.073450+0000","last_became_active":"2023-10-06T05:20:36.259336+0000","last_became_peered":"2023-10-06T05:20:36.259336+0000","last_unstale":"2023-10-06T05:20:37.073450+0000","last_undegraded":"2023-10-06T05:20:37.073450+0000","last_fullsized":"2023-10-06T05:20:37.073450+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:55.886826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.096706+0000","last_change":"2023-10-06T05:20:37.094362+0000","last_active":"2023-10-06T05:20:37.096706+0000","last_peered":"2023-10-06T05:20:37.096706+0000","last_clean":"2023-10-06T05:20:37.096706+0000","last_became_active":"2023-10-06T05:20:37.093995+0000","last_became_peered":"2023-10-06T05:20:37.093995+0000","last_unstale":"2023-10-06T05:20:37.096706+0000","last_undegraded":"2023-10-06T05:20:37.096706+0000","last_fullsized":"2023-10-06T05:20:37.096706+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:54.749017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.071130+0000","last_change":"2023-10-06T05:20:36.262413+0000","last_active":"2023-10-06T05:20:37.071130+0000","last_peered":"2023-10-06T05:20:37.071130+0000","last_clean":"2023-10-06T05:20:37.071130+0000","last_became_active":"2023-10-06T05:20:36.262136+0000","last_became_peered":"2023-10-06T05:20:36.262136+0000","last_unstale":"2023-10-06T05:20:37.071130+0000","last_undegraded":"2023-10-06T05:20:37.071130+0000","last_fullsized":"2023-10-06T05:20:37.071130+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:44.125014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.094636+0000","last_change":"2023-10-06T05:20:37.091611+0000","last_active":"2023-10-06T05:20:37.094636+0000","last_peered":"2023-10-06T05:20:37.094636+0000","last_clean":"2023-10-06T05:20:37.094636+0000","last_became_active":"2023-10-06T05:20:37.091269+0000","last_became_peered":"2023-10-06T05:20:37.091269+0000","last_unstale":"2023-10-06T05:20:37.094636+0000","last_undegraded":"2023-10-06T05:20:37.094636+0000","last_fullsized":"2023-10-06T05:20:37.094636+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:42.024126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.064262+0000","last_change":"2023-10-06T05:20:36.258074+0000","last_active":"2023-10-06T05:20:37.064262+0000","last_peered":"2023-10-06T05:20:37.064262+0000","last_clean":"2023-10-06T05:20:37.064262+0000","last_became_active":"2023-10-06T05:20:36.257783+0000","last_became_peered":"2023-10-06T05:20:36.257783+0000","last_unstale":"2023-10-06T05:20:37.064262+0000","last_undegraded":"2023-10-06T05:20:37.064262+0000","last_fullsized":"2023-10-06T05:20:37.064262+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:14.895156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089627+0000","last_change":"2023-10-06T05:20:37.083318+0000","last_active":"2023-10-06T05:20:37.089627+0000","last_peered":"2023-10-06T05:20:37.089627+0000","last_clean":"2023-10-06T05:20:37.089627+0000","last_became_active":"2023-10-06T05:20:37.083008+0000","last_became_peered":"2023-10-06T05:20:37.083008+0000","last_unstale":"2023-10-06T05:20:37.089627+0000","last_undegraded":"2023-10-06T05:20:37.089627+0000","last_fullsized":"2023-10-06T05:20:37.089627+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:26.067622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.064805+0000","last_change":"2023-10-06T05:20:36.261828+0000","last_active":"2023-10-06T05:20:37.064805+0000","last_peered":"2023-10-06T05:20:37.064805+0000","last_clean":"2023-10-06T05:20:37.064805+0000","last_became_active":"2023-10-06T05:20:36.261585+0000","last_became_peered":"2023-10-06T05:20:36.261585+0000","last_unstale":"2023-10-06T05:20:37.064805+0000","last_undegraded":"2023-10-06T05:20:37.064805+0000","last_fullsized":"2023-10-06T05:20:37.064805+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:21.102992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084672+0000","last_change":"2023-10-06T05:20:37.076025+0000","last_active":"2023-10-06T05:20:37.084672+0000","last_peered":"2023-10-06T05:20:37.084672+0000","last_clean":"2023-10-06T05:20:37.084672+0000","last_became_active":"2023-10-06T05:20:37.075681+0000","last_became_peered":"2023-10-06T05:20:37.075681+0000","last_unstale":"2023-10-06T05:20:37.084672+0000","last_undegraded":"2023-10-06T05:20:37.084672+0000","last_fullsized":"2023-10-06T05:20:37.084672+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:21.147856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.072852+0000","last_change":"2023-10-06T05:20:36.269404+0000","last_active":"2023-10-06T05:20:37.072852+0000","last_peered":"2023-10-06T05:20:37.072852+0000","last_clean":"2023-10-06T05:20:37.072852+0000","last_became_active":"2023-10-06T05:20:36.269137+0000","last_became_peered":"2023-10-06T05:20:36.269137+0000","last_unstale":"2023-10-06T05:20:37.072852+0000","last_undegraded":"2023-10-06T05:20:37.072852+0000","last_fullsized":"2023-10-06T05:20:37.072852+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:37.100558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.097892+0000","last_change":"2023-10-06T05:20:37.096590+0000","last_active":"2023-10-06T05:20:37.097892+0000","last_peered":"2023-10-06T05:20:37.097892+0000","last_clean":"2023-10-06T05:20:37.097892+0000","last_became_active":"2023-10-06T05:20:37.096094+0000","last_became_peered":"2023-10-06T05:20:37.096094+0000","last_unstale":"2023-10-06T05:20:37.097892+0000","last_undegraded":"2023-10-06T05:20:37.097892+0000","last_fullsized":"2023-10-06T05:20:37.097892+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:15.901055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"56'1","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.268004+0000","last_change":"2023-10-06T05:20:36.261789+0000","last_active":"2023-10-06T05:20:37.268004+0000","last_peered":"2023-10-06T05:20:37.268004+0000","last_clean":"2023-10-06T05:20:37.268004+0000","last_became_active":"2023-10-06T05:20:36.261466+0000","last_became_peered":"2023-10-06T05:20:36.261466+0000","last_unstale":"2023-10-06T05:20:37.268004+0000","last_undegraded":"2023-10-06T05:20:37.268004+0000","last_fullsized":"2023-10-06T05:20:37.268004+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:00.867550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":46,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.075424+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:20:37.075424+0000","last_peered":"2023-10-06T05:20:37.075424+0000","last_clean":"2023-10-06T05:20:37.075424+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:20:37.075424+0000","last_undegraded":"2023-10-06T05:20:37.075424+0000","last_fullsized":"2023-10-06T05:20:37.075424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095705+0000","last_change":"2023-10-06T05:20:37.091617+0000","last_active":"2023-10-06T05:20:37.095705+0000","last_peered":"2023-10-06T05:20:37.095705+0000","last_clean":"2023-10-06T05:20:37.095705+0000","last_became_active":"2023-10-06T05:20:37.091300+0000","last_became_peered":"2023-10-06T05:20:37.091300+0000","last_unstale":"2023-10-06T05:20:37.095705+0000","last_undegraded":"2023-10-06T05:20:37.095705+0000","last_fullsized":"2023-10-06T05:20:37.095705+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:33.806691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.064800+0000","last_change":"2023-10-06T05:20:36.259736+0000","last_active":"2023-10-06T05:20:37.064800+0000","last_peered":"2023-10-06T05:20:37.064800+0000","last_clean":"2023-10-06T05:20:37.064800+0000","last_became_active":"2023-10-06T05:20:36.259467+0000","last_became_peered":"2023-10-06T05:20:36.259467+0000","last_unstale":"2023-10-06T05:20:37.064800+0000","last_undegraded":"2023-10-06T05:20:37.064800+0000","last_fullsized":"2023-10-06T05:20:37.064800+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:04.039009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.064883+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:20:37.064883+0000","last_peered":"2023-10-06T05:20:37.064883+0000","last_clean":"2023-10-06T05:20:37.064883+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:20:37.064883+0000","last_undegraded":"2023-10-06T05:20:37.064883+0000","last_fullsized":"2023-10-06T05:20:37.064883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.076904+0000","last_change":"2023-10-06T05:20:37.073646+0000","last_active":"2023-10-06T05:20:37.076904+0000","last_peered":"2023-10-06T05:20:37.076904+0000","last_clean":"2023-10-06T05:20:37.076904+0000","last_became_active":"2023-10-06T05:20:37.073047+0000","last_became_peered":"2023-10-06T05:20:37.073047+0000","last_unstale":"2023-10-06T05:20:37.076904+0000","last_undegraded":"2023-10-06T05:20:37.076904+0000","last_fullsized":"2023-10-06T05:20:37.076904+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:39.205579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.267667+0000","last_change":"2023-10-06T05:20:36.267555+0000","last_active":"2023-10-06T05:20:37.267667+0000","last_peered":"2023-10-06T05:20:37.267667+0000","last_clean":"2023-10-06T05:20:37.267667+0000","last_became_active":"2023-10-06T05:20:36.267330+0000","last_became_peered":"2023-10-06T05:20:36.267330+0000","last_unstale":"2023-10-06T05:20:37.267667+0000","last_undegraded":"2023-10-06T05:20:37.267667+0000","last_fullsized":"2023-10-06T05:20:37.267667+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:33.450013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":45,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.073621+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:20:37.073621+0000","last_peered":"2023-10-06T05:20:37.073621+0000","last_clean":"2023-10-06T05:20:37.073621+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:20:37.073621+0000","last_undegraded":"2023-10-06T05:20:37.073621+0000","last_fullsized":"2023-10-06T05:20:37.073621+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":192,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.070315+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:20:37.070315+0000","last_peered":"2023-10-06T05:20:37.070315+0000","last_clean":"2023-10-06T05:20:37.070315+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:20:37.070315+0000","last_undegraded":"2023-10-06T05:20:37.070315+0000","last_fullsized":"2023-10-06T05:20:37.070315+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.086299+0000","last_change":"2023-10-06T05:20:37.079405+0000","last_active":"2023-10-06T05:20:37.086299+0000","last_peered":"2023-10-06T05:20:37.086299+0000","last_clean":"2023-10-06T05:20:37.086299+0000","last_became_active":"2023-10-06T05:20:37.078627+0000","last_became_peered":"2023-10-06T05:20:37.078627+0000","last_unstale":"2023-10-06T05:20:37.086299+0000","last_undegraded":"2023-10-06T05:20:37.086299+0000","last_fullsized":"2023-10-06T05:20:37.086299+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:38.426798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.071961+0000","last_change":"2023-10-06T05:20:36.264980+0000","last_active":"2023-10-06T05:20:37.071961+0000","last_peered":"2023-10-06T05:20:37.071961+0000","last_clean":"2023-10-06T05:20:37.071961+0000","last_became_active":"2023-10-06T05:20:36.264703+0000","last_became_peered":"2023-10-06T05:20:36.264703+0000","last_unstale":"2023-10-06T05:20:37.071961+0000","last_undegraded":"2023-10-06T05:20:37.071961+0000","last_fullsized":"2023-10-06T05:20:37.071961+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:44.382836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.072529+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:20:37.072529+0000","last_peered":"2023-10-06T05:20:37.072529+0000","last_clean":"2023-10-06T05:20:37.072529+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:20:37.072529+0000","last_undegraded":"2023-10-06T05:20:37.072529+0000","last_fullsized":"2023-10-06T05:20:37.072529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.096918+0000","last_change":"2023-10-06T05:20:37.094903+0000","last_active":"2023-10-06T05:20:37.096918+0000","last_peered":"2023-10-06T05:20:37.096918+0000","last_clean":"2023-10-06T05:20:37.096918+0000","last_became_active":"2023-10-06T05:20:37.093433+0000","last_became_peered":"2023-10-06T05:20:37.093433+0000","last_unstale":"2023-10-06T05:20:37.096918+0000","last_undegraded":"2023-10-06T05:20:37.096918+0000","last_fullsized":"2023-10-06T05:20:37.096918+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:55.693958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.066908+0000","last_change":"2023-10-06T05:20:36.254703+0000","last_active":"2023-10-06T05:20:37.066908+0000","last_peered":"2023-10-06T05:20:37.066908+0000","last_clean":"2023-10-06T05:20:37.066908+0000","last_became_active":"2023-10-06T05:20:36.254422+0000","last_became_peered":"2023-10-06T05:20:36.254422+0000","last_unstale":"2023-10-06T05:20:37.066908+0000","last_undegraded":"2023-10-06T05:20:37.066908+0000","last_fullsized":"2023-10-06T05:20:37.066908+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:44.581770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.070422+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:20:37.070422+0000","last_peered":"2023-10-06T05:20:37.070422+0000","last_clean":"2023-10-06T05:20:37.070422+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:20:37.070422+0000","last_undegraded":"2023-10-06T05:20:37.070422+0000","last_fullsized":"2023-10-06T05:20:37.070422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095316+0000","last_change":"2023-10-06T05:20:37.089844+0000","last_active":"2023-10-06T05:20:37.095316+0000","last_peered":"2023-10-06T05:20:37.095316+0000","last_clean":"2023-10-06T05:20:37.095316+0000","last_became_active":"2023-10-06T05:20:37.088699+0000","last_became_peered":"2023-10-06T05:20:37.088699+0000","last_unstale":"2023-10-06T05:20:37.095316+0000","last_undegraded":"2023-10-06T05:20:37.095316+0000","last_fullsized":"2023-10-06T05:20:37.095316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:44.095572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.065567+0000","last_change":"2023-10-06T05:20:36.263060+0000","last_active":"2023-10-06T05:20:37.065567+0000","last_peered":"2023-10-06T05:20:37.065567+0000","last_clean":"2023-10-06T05:20:37.065567+0000","last_became_active":"2023-10-06T05:20:36.262695+0000","last_became_peered":"2023-10-06T05:20:36.262695+0000","last_unstale":"2023-10-06T05:20:37.065567+0000","last_undegraded":"2023-10-06T05:20:37.065567+0000","last_fullsized":"2023-10-06T05:20:37.065567+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:05.134543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.070068+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:20:37.070068+0000","last_peered":"2023-10-06T05:20:37.070068+0000","last_clean":"2023-10-06T05:20:37.070068+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:20:37.070068+0000","last_undegraded":"2023-10-06T05:20:37.070068+0000","last_fullsized":"2023-10-06T05:20:37.070068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095031+0000","last_change":"2023-10-06T05:20:37.089858+0000","last_active":"2023-10-06T05:20:37.095031+0000","last_peered":"2023-10-06T05:20:37.095031+0000","last_clean":"2023-10-06T05:20:37.095031+0000","last_became_active":"2023-10-06T05:20:37.089278+0000","last_became_peered":"2023-10-06T05:20:37.089278+0000","last_unstale":"2023-10-06T05:20:37.095031+0000","last_undegraded":"2023-10-06T05:20:37.095031+0000","last_fullsized":"2023-10-06T05:20:37.095031+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:38.106457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.269510+0000","last_change":"2023-10-06T05:20:36.263009+0000","last_active":"2023-10-06T05:20:37.269510+0000","last_peered":"2023-10-06T05:20:37.269510+0000","last_clean":"2023-10-06T05:20:37.269510+0000","last_became_active":"2023-10-06T05:20:36.262604+0000","last_became_peered":"2023-10-06T05:20:36.262604+0000","last_unstale":"2023-10-06T05:20:37.269510+0000","last_undegraded":"2023-10-06T05:20:37.269510+0000","last_fullsized":"2023-10-06T05:20:37.269510+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:22.505929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":44,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.086129+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:20:37.086129+0000","last_peered":"2023-10-06T05:20:37.086129+0000","last_clean":"2023-10-06T05:20:37.086129+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:20:37.086129+0000","last_undegraded":"2023-10-06T05:20:37.086129+0000","last_fullsized":"2023-10-06T05:20:37.086129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089121+0000","last_change":"2023-10-06T05:20:37.086365+0000","last_active":"2023-10-06T05:20:37.089121+0000","last_peered":"2023-10-06T05:20:37.089121+0000","last_clean":"2023-10-06T05:20:37.089121+0000","last_became_active":"2023-10-06T05:20:37.085956+0000","last_became_peered":"2023-10-06T05:20:37.085956+0000","last_unstale":"2023-10-06T05:20:37.089121+0000","last_undegraded":"2023-10-06T05:20:37.089121+0000","last_fullsized":"2023-10-06T05:20:37.089121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:26.019845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"56'1","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.268366+0000","last_change":"2023-10-06T05:20:36.267263+0000","last_active":"2023-10-06T05:20:37.268366+0000","last_peered":"2023-10-06T05:20:37.268366+0000","last_clean":"2023-10-06T05:20:37.268366+0000","last_became_active":"2023-10-06T05:20:36.267001+0000","last_became_peered":"2023-10-06T05:20:36.267001+0000","last_unstale":"2023-10-06T05:20:37.268366+0000","last_undegraded":"2023-10-06T05:20:37.268366+0000","last_fullsized":"2023-10-06T05:20:37.268366+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:02.105855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":44,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.085960+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:20:37.085960+0000","last_peered":"2023-10-06T05:20:37.085960+0000","last_clean":"2023-10-06T05:20:37.085960+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:20:37.085960+0000","last_undegraded":"2023-10-06T05:20:37.085960+0000","last_fullsized":"2023-10-06T05:20:37.085960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298552,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073288,"statfs":{"total":96636764160,"available":96331046912,"internally_reserved":0,"allocated":1294336,"data_stored":891201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48699999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93500000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73899999999999999,"15min":0.73799999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.82899999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.51100000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.754,"15min":0.76400000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.749,"15min":0.755},"min":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.64700000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.92600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.76100000000000001,"15min":0.78900000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.73199999999999998,"15min":0.749},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90600000000000003,"5min":0.999,"15min":0.999},"last":0.879},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.81100000000000005,"15min":0.84699999999999998},"min":{"1min":0.53900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91700000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.86799999999999999,"15min":0.89900000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83999999999999997,"15min":0.85899999999999999},"min":{"1min":0.51400000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.0309999999999999,"5min":1.103,"15min":1.103},"last":0.94199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297960,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073880,"statfs":{"total":96636764160,"available":96331653120,"internally_reserved":0,"allocated":688128,"data_stored":300119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.498,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.006,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72299999999999998,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91900000000000004,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97599999999999998,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.626},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.046,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.875,"5min":1.369,"15min":1.369},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.58799999999999997,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92100000000000004,"5min":1.891,"15min":1.891},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.746,"15min":0.755},"min":{"1min":0.58099999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.96999999999999997,"5min":1.347,"15min":1.347},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.99299999999999999,"5min":1,"15min":1},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.75800000000000001,"15min":0.79300000000000004},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80500000000000005,"15min":0.83099999999999996},"min":{"1min":0.54800000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0089999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.90800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68300000000000005,"15min":0.69899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90100000000000002,"5min":1.415,"15min":1.415},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.80000000000000004,"15min":0.83899999999999997},"min":{"1min":0.501,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.997,"5min":2,"15min":2},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79200000000000004,"15min":0.81999999999999995},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.504},"max":{"1min":0.97899999999999998,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.52600000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96099999999999997,"5min":1.46,"15min":1.46},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298564,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073276,"statfs":{"total":96636764160,"available":96331034624,"internally_reserved":0,"allocated":1306624,"data_stored":892725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.441,"15min":0.433},"min":{"1min":0.34399999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.58999999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.55500000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72599999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.48599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.629}]},{"osd":4,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0700000000000001,"5min":1.121,"15min":1.121},"last":1.077},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80500000000000005,"15min":0.79000000000000004},"min":{"1min":0.64600000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0589999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":1.0860000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91900000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.59199999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":1.002}]},{"osd":6,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76400000000000001,"15min":0.748},"min":{"1min":0.60199999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.05,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81200000000000006,"15min":0.79700000000000004},"min":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0069999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.94399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.57799999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.98999999999999999,"5min":1.042,"15min":1.042},"last":1.05},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82799999999999996,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0369999999999999,"5min":1.103,"15min":1.103},"last":0.92500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297944,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073896,"statfs":{"total":96636764160,"available":96331669504,"internally_reserved":0,"allocated":671744,"data_stored":293444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.44400000000000001,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.504},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.499,"15min":0.495},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78900000000000003,"5min":1.042,"15min":1.042},"last":0.629}]},{"osd":2,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.42199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52300000000000002,"15min":0.51200000000000001},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.78700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.50700000000000001,"15min":0.48699999999999999},"min":{"1min":0.41099999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59099999999999997,"15min":0.57799999999999996},"min":{"1min":0.47899999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.76900000000000002,"15min":0.75600000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.79200000000000004,"15min":0.78800000000000003},"min":{"1min":0.67700000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.78400000000000003,"15min":0.77100000000000002},"min":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.79000000000000004,"15min":0.78300000000000003},"min":{"1min":0.63700000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.71799999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.81499999999999995,"15min":0.81000000000000005},"min":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.749},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.80800000000000005,"15min":0.79400000000000004},"min":{"1min":0.61099999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1799999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.97299999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.85199999999999998,"15min":0.84399999999999997},"min":{"1min":0.70999999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2090000000000001,"5min":1.264,"15min":1.264},"last":0.995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.67600000000000005,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1579999999999999,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.83399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297944,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073896,"statfs":{"total":96636764160,"available":96331669504,"internally_reserved":0,"allocated":671744,"data_stored":293444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.436,"15min":0.42999999999999999},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77400000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.39000000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66000000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42499999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.32800000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.504,"15min":0.504},"min":{"1min":0.41699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69799999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.42999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82799999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76100000000000001,"15min":0.76600000000000001},"min":{"1min":0.65100000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.95099999999999996,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0009999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78500000000000003,"15min":0.79800000000000004},"min":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96999999999999997,"5min":1.099,"15min":1.099},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0649999999999999,"5min":1.163,"15min":1.163},"last":0.624}]},{"osd":6,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.83699999999999997,"15min":0.85099999999999998},"min":{"1min":0.66700000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0960000000000001,"5min":1.21,"15min":1.21},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81699999999999995,"15min":0.82599999999999996},"min":{"1min":0.70099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.98499999999999999,"5min":1.042,"15min":1.042},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80700000000000005,"15min":0.81599999999999995},"min":{"1min":0.64700000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.85999999999999999,"15min":0.878},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0309999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.64800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297956,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073884,"statfs":{"total":96636764160,"available":96331657216,"internally_reserved":0,"allocated":684032,"data_stored":300085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65500000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.495,"15min":0.495},"min":{"1min":0.38400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.314}]},{"osd":1,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.46300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.504,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78500000000000003,"5min":1.014,"15min":1.014},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.161,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.103,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.57999999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.147,"5min":1.204,"15min":1.204},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.83599999999999997,"15min":0.84699999999999998},"min":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.284,"5min":1.329,"15min":1.329},"last":0.66000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.81399999999999995,"15min":0.82399999999999995},"min":{"1min":0.63900000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0840000000000001,"5min":1.484,"15min":1.484},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.66600000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.212,"5min":1.629,"15min":1.629},"last":0.52000000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85499999999999998,"15min":0.86599999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.236,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.63},{"interface":"front","average":{"1min":0.871,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.66000000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1259999999999999,"5min":1.325,"15min":1.325},"last":0.64300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297968,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073872,"statfs":{"total":96636764160,"available":96331644928,"internally_reserved":0,"allocated":696320,"data_stored":301621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76600000000000001,"15min":0.80600000000000005},"min":{"1min":0.53600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.91000000000000003,"5min":2.77,"15min":2.77},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72899999999999998,"15min":0.749},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93300000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.72999999999999998},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90300000000000002,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.52100000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86799999999999999,"5min":0.879,"15min":0.879},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.84099999999999997},"min":{"1min":0.54000000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94999999999999996,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.55500000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.89700000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.998,"15min":1.1220000000000001},"min":{"1min":0.54400000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.93700000000000006,"5min":7.742,"15min":7.742},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":1.079,"15min":1.079},"last":0.751},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74399999999999999,"15min":0.76100000000000001},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91100000000000003,"5min":1.181,"15min":1.181},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72199999999999998,"15min":0.749},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89500000000000002,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.80100000000000005,"15min":0.83299999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":1.171,"15min":1.171},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96199999999999997,"5min":1.228,"15min":1.228},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.83999999999999997,"15min":0.878},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.571,"15min":1.571},"last":0.82599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297960,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073880,"statfs":{"total":96636764160,"available":96331653120,"internally_reserved":0,"allocated":688128,"data_stored":300107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.91500000000000004,"15min":1.028},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0209999999999999,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.747},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.46400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.94199999999999995,"15min":1.05},"min":{"1min":0.49299999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0640000000000001,"5min":9.25,"15min":9.25},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.999,"15min":1.123},"min":{"1min":0.503,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.97499999999999998,"15min":1.095},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.97999999999999998,"5min":10.384,"15min":10.384},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.98999999999999999,"15min":1.1040000000000001},"min":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0660000000000001,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":1.028,"15min":1.157},"min":{"1min":0.54400000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.0740000000000001,"5min":10.205,"15min":10.205},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72499999999999998,"15min":0.72999999999999998},"min":{"1min":0.50600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.93700000000000006,"5min":1.292,"15min":1.292},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84099999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.78600000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85399999999999998,"5min":1.171,"15min":1.171},"last":0.625}]},{"osd":6,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90500000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.80400000000000005,"15min":0.82399999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":1.298,"15min":1.298},"last":0.64200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81699999999999995,"15min":0.83399999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0840000000000001,"5min":1.204,"15min":1.204},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.81399999999999995,"15min":0.82399999999999995},"min":{"1min":0.54300000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93999999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.71999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:20:41.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:20:41.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.905+0000 7f0d851ff700 1 Processor -- start 2023-10-06T05:20:41.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.906+0000 7f0d851ff700 1 -- start start 2023-10-06T05:20:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.907+0000 7f0d851ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d801595a0 0x7f0d80159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.907+0000 7f0d851ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0d80068d80 0x7f0d800c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.907+0000 7f0d851ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d800c95b0 con 0x7f0d800cc0d0 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.907+0000 7f0d851ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800c9890 con 0x7f0d801595a0 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.907+0000 7f0d851ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800c9b70 con 0x7f0d80068d80 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.907+0000 7f0d7ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d801595a0 0x7f0d80159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.907+0000 7f0d7ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d801595a0 0x7f0d80159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45472/0 (socket says 172.21.15.31:45472) 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.907+0000 7f0d7ed9d700 1 -- 172.21.15.31:0/159724596 learned_addr learned my addr 172.21.15.31:0/159724596 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.907+0000 7f0d7f59e700 1 --2- 172.21.15.31:0/159724596 >> v2:172.21.15.62:3300/0 conn(0x7f0d80068d80 0x7f0d800c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.908+0000 7f0d7ed9d700 1 -- 172.21.15.31:0/159724596 >> v2:172.21.15.62:3300/0 conn(0x7f0d80068d80 msgr2=0x7f0d800c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.908+0000 7f0d7ed9d700 1 --2- 172.21.15.31:0/159724596 >> v2:172.21.15.62:3300/0 conn(0x7f0d80068d80 0x7f0d800c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.908+0000 7f0d7ed9d700 1 -- 172.21.15.31:0/159724596 >> v1:172.21.15.62:6789/0 conn(0x7f0d800cc0d0 legacy=0x7f0d80068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.908+0000 7f0d7ed9d700 1 -- 172.21.15.31:0/159724596 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800ca130 con 0x7f0d801595a0 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.908+0000 7f0d7ed9d700 1 --2- 172.21.15.31:0/159724596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 0x7f0d80159970 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0d700099b0 tx=0x7f0d70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1a56eeda1c3c39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.908+0000 7f0d7dd9b700 1 -- 172.21.15.31:0/159724596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70009ea0 con 0x7f0d801595a0 2023-10-06T05:20:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.908+0000 7f0d7dd9b700 1 -- 172.21.15.31:0/159724596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7000b820 con 0x7f0d801595a0 2023-10-06T05:20:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.908+0000 7f0d7dd9b700 1 -- 172.21.15.31:0/159724596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7000bbc0 con 0x7f0d801595a0 2023-10-06T05:20:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.909+0000 7f0d851ff700 1 -- 172.21.15.31:0/159724596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 msgr2=0x7f0d80159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.909+0000 7f0d851ff700 1 --2- 172.21.15.31:0/159724596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 0x7f0d80159970 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0d700099b0 tx=0x7f0d70009250 comp rx=0 tx=0).stop 2023-10-06T05:20:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.909+0000 7f0d851ff700 1 -- 172.21.15.31:0/159724596 shutdown_connections 2023-10-06T05:20:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.909+0000 7f0d851ff700 1 --2- 172.21.15.31:0/159724596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 0x7f0d80159970 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.909+0000 7f0d851ff700 1 --2- 172.21.15.31:0/159724596 >> v2:172.21.15.62:3300/0 conn(0x7f0d80068d80 0x7f0d800c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.909+0000 7f0d851ff700 1 -- 172.21.15.31:0/159724596 >> 172.21.15.31:0/159724596 conn(0x7f0d8006b620 msgr2=0x7f0d8006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.909+0000 7f0d851ff700 1 -- 172.21.15.31:0/159724596 shutdown_connections 2023-10-06T05:20:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.909+0000 7f0d851ff700 1 -- 172.21.15.31:0/159724596 wait complete. 2023-10-06T05:20:41.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.910+0000 7f0d851ff700 1 Processor -- start 2023-10-06T05:20:41.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.911+0000 7f0d851ff700 1 -- start start 2023-10-06T05:20:41.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.911+0000 7f0d851ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d80068d80 0x7f0d80147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d851ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d800cc0d0 0x7f0d8015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d851ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 0x7f0d80154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d851ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8015e590 con 0x7f0d801595a0 2023-10-06T05:20:41.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d851ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d80148380 con 0x7f0d80068d80 2023-10-06T05:20:41.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d851ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8015d020 con 0x7f0d800cc0d0 2023-10-06T05:20:41.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 0x7f0d80154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 0x7f0d80154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45478/0 (socket says 172.21.15.31:45478) 2023-10-06T05:20:41.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7f59e700 1 -- 172.21.15.31:0/48540901 learned_addr learned my addr 172.21.15.31:0/48540901 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:41.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7ed9d700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d80068d80 0x7f0d80147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7e59c700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d800cc0d0 0x7f0d8015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7f59e700 1 -- 172.21.15.31:0/48540901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d800cc0d0 msgr2=0x7f0d8015e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7f59e700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d800cc0d0 0x7f0d8015e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7f59e700 1 -- 172.21.15.31:0/48540901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d80068d80 msgr2=0x7f0d80147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:41.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7f59e700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d80068d80 0x7f0d80147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:41.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7f59e700 1 -- 172.21.15.31:0/48540901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d700094f0 con 0x7f0d801595a0 2023-10-06T05:20:41.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.912+0000 7f0d7e59c700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d800cc0d0 0x7f0d8015e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:41.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.913+0000 7f0d7f59e700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 0x7f0d80154a00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009d70 tx=0x7f0d7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.913+0000 7f0d66ffd700 1 -- 172.21.15.31:0/48540901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800cb50 con 0x7f0d801595a0 2023-10-06T05:20:41.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.913+0000 7f0d66ffd700 1 -- 172.21.15.31:0/48540901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7800f040 con 0x7f0d801595a0 2023-10-06T05:20:41.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.913+0000 7f0d851ff700 1 -- 172.21.15.31:0/48540901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d80169b40 con 0x7f0d801595a0 2023-10-06T05:20:41.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.913+0000 7f0d66ffd700 1 -- 172.21.15.31:0/48540901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78014480 con 0x7f0d801595a0 2023-10-06T05:20:41.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.913+0000 7f0d851ff700 1 -- 172.21.15.31:0/48540901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8015ca60 con 0x7f0d801595a0 2023-10-06T05:20:41.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.914+0000 7f0d851ff700 1 -- 172.21.15.31:0/48540901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d80169cd0 con 0x7f0d801595a0 2023-10-06T05:20:41.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.915+0000 7f0d66ffd700 1 -- 172.21.15.31:0/48540901 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f0d78014830 con 0x7f0d801595a0 2023-10-06T05:20:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.916+0000 7f0d66ffd700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0d6c0670e0 0x7f0d6c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.916+0000 7f0d66ffd700 1 -- 172.21.15.31:0/48540901 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0d78085700 con 0x7f0d801595a0 2023-10-06T05:20:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.916+0000 7f0d7ed9d700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0d6c0670e0 0x7f0d6c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.917+0000 7f0d7ed9d700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0d6c0670e0 0x7f0d6c069590 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d700099b0 tx=0x7f0d70017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:41.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:41.920+0000 7f0d66ffd700 1 -- 172.21.15.31:0/48540901 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d780523e0 con 0x7f0d801595a0 2023-10-06T05:20:42.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.120+0000 7f0d851ff700 1 -- 172.21.15.31:0/48540901 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d8004c380 con 0x7f0d6c0670e0 2023-10-06T05:20:42.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.130+0000 7f0d66ffd700 1 -- 172.21.15.31:0/48540901 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311126 (secure 0 0 0) 0x7f0d8004c380 con 0x7f0d6c0670e0 2023-10-06T05:20:42.139 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.134+0000 7f0d851ff700 1 -- 172.21.15.31:0/48540901 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0d6c0670e0 msgr2=0x7f0d6c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.134+0000 7f0d851ff700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0d6c0670e0 0x7f0d6c069590 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d700099b0 tx=0x7f0d70017000 comp rx=0 tx=0).stop 2023-10-06T05:20:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.134+0000 7f0d851ff700 1 -- 172.21.15.31:0/48540901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 msgr2=0x7f0d80154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.134+0000 7f0d851ff700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 0x7f0d80154a00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009d70 tx=0x7f0d7800b040 comp rx=0 tx=0).stop 2023-10-06T05:20:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.134+0000 7f0d851ff700 1 -- 172.21.15.31:0/48540901 shutdown_connections 2023-10-06T05:20:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.134+0000 7f0d851ff700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0d6c0670e0 0x7f0d6c069590 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.134+0000 7f0d851ff700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d801595a0 0x7f0d80154a00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.134+0000 7f0d851ff700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d80068d80 0x7f0d80147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.134+0000 7f0d851ff700 1 --2- 172.21.15.31:0/48540901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d800cc0d0 0x7f0d8015e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.134+0000 7f0d851ff700 1 -- 172.21.15.31:0/48540901 >> 172.21.15.31:0/48540901 conn(0x7f0d8006b620 msgr2=0x7f0d80158790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.135+0000 7f0d851ff700 1 -- 172.21.15.31:0/48540901 shutdown_connections 2023-10-06T05:20:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.135+0000 7f0d851ff700 1 -- 172.21.15.31:0/48540901 wait complete. 2023-10-06T05:20:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:20:42.154 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-10-06T05:20:42.111363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384848,"kb_used_data":6544,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589872,"statfs":{"total":773094113280,"available":770652028928,"internally_reserved":0,"allocated":6701056,"data_stored":3572746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001251"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089707+0000","last_change":"2023-10-06T05:20:37.081501+0000","last_active":"2023-10-06T05:20:37.089707+0000","last_peered":"2023-10-06T05:20:37.089707+0000","last_clean":"2023-10-06T05:20:37.089707+0000","last_became_active":"2023-10-06T05:20:37.081169+0000","last_became_peered":"2023-10-06T05:20:37.081169+0000","last_unstale":"2023-10-06T05:20:37.089707+0000","last_undegraded":"2023-10-06T05:20:37.089707+0000","last_fullsized":"2023-10-06T05:20:37.089707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:57.659806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.081421+0000","last_change":"2023-10-06T05:20:37.077783+0000","last_active":"2023-10-06T05:20:37.081421+0000","last_peered":"2023-10-06T05:20:37.081421+0000","last_clean":"2023-10-06T05:20:37.081421+0000","last_became_active":"2023-10-06T05:20:37.077520+0000","last_became_peered":"2023-10-06T05:20:37.077520+0000","last_unstale":"2023-10-06T05:20:37.081421+0000","last_undegraded":"2023-10-06T05:20:37.081421+0000","last_fullsized":"2023-10-06T05:20:37.081421+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:53.413645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.087522+0000","last_change":"2023-10-06T05:20:37.085993+0000","last_active":"2023-10-06T05:20:37.087522+0000","last_peered":"2023-10-06T05:20:37.087522+0000","last_clean":"2023-10-06T05:20:37.087522+0000","last_became_active":"2023-10-06T05:20:37.085693+0000","last_became_peered":"2023-10-06T05:20:37.085693+0000","last_unstale":"2023-10-06T05:20:37.087522+0000","last_undegraded":"2023-10-06T05:20:37.087522+0000","last_fullsized":"2023-10-06T05:20:37.087522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:24.005054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.080473+0000","last_change":"2023-10-06T05:20:37.078536+0000","last_active":"2023-10-06T05:20:37.080473+0000","last_peered":"2023-10-06T05:20:37.080473+0000","last_clean":"2023-10-06T05:20:37.080473+0000","last_became_active":"2023-10-06T05:20:37.077817+0000","last_became_peered":"2023-10-06T05:20:37.077817+0000","last_unstale":"2023-10-06T05:20:37.080473+0000","last_undegraded":"2023-10-06T05:20:37.080473+0000","last_fullsized":"2023-10-06T05:20:37.080473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:12.980807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095161+0000","last_change":"2023-10-06T05:20:37.089595+0000","last_active":"2023-10-06T05:20:37.095161+0000","last_peered":"2023-10-06T05:20:37.095161+0000","last_clean":"2023-10-06T05:20:37.095161+0000","last_became_active":"2023-10-06T05:20:37.089331+0000","last_became_peered":"2023-10-06T05:20:37.089331+0000","last_unstale":"2023-10-06T05:20:37.095161+0000","last_undegraded":"2023-10-06T05:20:37.095161+0000","last_fullsized":"2023-10-06T05:20:37.095161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:16.321484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090585+0000","last_change":"2023-10-06T05:20:37.088048+0000","last_active":"2023-10-06T05:20:37.090585+0000","last_peered":"2023-10-06T05:20:37.090585+0000","last_clean":"2023-10-06T05:20:37.090585+0000","last_became_active":"2023-10-06T05:20:37.087837+0000","last_became_peered":"2023-10-06T05:20:37.087837+0000","last_unstale":"2023-10-06T05:20:37.090585+0000","last_undegraded":"2023-10-06T05:20:37.090585+0000","last_fullsized":"2023-10-06T05:20:37.090585+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:31.813149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089800+0000","last_change":"2023-10-06T05:20:37.084546+0000","last_active":"2023-10-06T05:20:37.089800+0000","last_peered":"2023-10-06T05:20:37.089800+0000","last_clean":"2023-10-06T05:20:37.089800+0000","last_became_active":"2023-10-06T05:20:37.084287+0000","last_became_peered":"2023-10-06T05:20:37.084287+0000","last_unstale":"2023-10-06T05:20:37.089800+0000","last_undegraded":"2023-10-06T05:20:37.089800+0000","last_fullsized":"2023-10-06T05:20:37.089800+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:55.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084722+0000","last_change":"2023-10-06T05:20:37.078343+0000","last_active":"2023-10-06T05:20:37.084722+0000","last_peered":"2023-10-06T05:20:37.084722+0000","last_clean":"2023-10-06T05:20:37.084722+0000","last_became_active":"2023-10-06T05:20:37.077990+0000","last_became_peered":"2023-10-06T05:20:37.077990+0000","last_unstale":"2023-10-06T05:20:37.084722+0000","last_undegraded":"2023-10-06T05:20:37.084722+0000","last_fullsized":"2023-10-06T05:20:37.084722+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:46.285631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.097412+0000","last_change":"2023-10-06T05:20:37.089003+0000","last_active":"2023-10-06T05:20:37.097412+0000","last_peered":"2023-10-06T05:20:37.097412+0000","last_clean":"2023-10-06T05:20:37.097412+0000","last_became_active":"2023-10-06T05:20:37.088727+0000","last_became_peered":"2023-10-06T05:20:37.088727+0000","last_unstale":"2023-10-06T05:20:37.097412+0000","last_undegraded":"2023-10-06T05:20:37.097412+0000","last_fullsized":"2023-10-06T05:20:37.097412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:43.196750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.096917+0000","last_change":"2023-10-06T05:20:37.087165+0000","last_active":"2023-10-06T05:20:37.096917+0000","last_peered":"2023-10-06T05:20:37.096917+0000","last_clean":"2023-10-06T05:20:37.096917+0000","last_became_active":"2023-10-06T05:20:37.086879+0000","last_became_peered":"2023-10-06T05:20:37.086879+0000","last_unstale":"2023-10-06T05:20:37.096917+0000","last_undegraded":"2023-10-06T05:20:37.096917+0000","last_fullsized":"2023-10-06T05:20:37.096917+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:43.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.080503+0000","last_change":"2023-10-06T05:20:37.077823+0000","last_active":"2023-10-06T05:20:37.080503+0000","last_peered":"2023-10-06T05:20:37.080503+0000","last_clean":"2023-10-06T05:20:37.080503+0000","last_became_active":"2023-10-06T05:20:37.077476+0000","last_became_peered":"2023-10-06T05:20:37.077476+0000","last_unstale":"2023-10-06T05:20:37.080503+0000","last_undegraded":"2023-10-06T05:20:37.080503+0000","last_fullsized":"2023-10-06T05:20:37.080503+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:45.587669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090505+0000","last_change":"2023-10-06T05:20:37.082144+0000","last_active":"2023-10-06T05:20:37.090505+0000","last_peered":"2023-10-06T05:20:37.090505+0000","last_clean":"2023-10-06T05:20:37.090505+0000","last_became_active":"2023-10-06T05:20:37.081819+0000","last_became_peered":"2023-10-06T05:20:37.081819+0000","last_unstale":"2023-10-06T05:20:37.090505+0000","last_undegraded":"2023-10-06T05:20:37.090505+0000","last_fullsized":"2023-10-06T05:20:37.090505+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:22.108891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.087577+0000","last_change":"2023-10-06T05:20:37.080639+0000","last_active":"2023-10-06T05:20:37.087577+0000","last_peered":"2023-10-06T05:20:37.087577+0000","last_clean":"2023-10-06T05:20:37.087577+0000","last_became_active":"2023-10-06T05:20:37.080314+0000","last_became_peered":"2023-10-06T05:20:37.080314+0000","last_unstale":"2023-10-06T05:20:37.087577+0000","last_undegraded":"2023-10-06T05:20:37.087577+0000","last_fullsized":"2023-10-06T05:20:37.087577+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:36.557041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.087835+0000","last_change":"2023-10-06T05:20:37.086145+0000","last_active":"2023-10-06T05:20:37.087835+0000","last_peered":"2023-10-06T05:20:37.087835+0000","last_clean":"2023-10-06T05:20:37.087835+0000","last_became_active":"2023-10-06T05:20:37.085783+0000","last_became_peered":"2023-10-06T05:20:37.085783+0000","last_unstale":"2023-10-06T05:20:37.087835+0000","last_undegraded":"2023-10-06T05:20:37.087835+0000","last_fullsized":"2023-10-06T05:20:37.087835+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:34.455912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.097302+0000","last_change":"2023-10-06T05:20:37.087958+0000","last_active":"2023-10-06T05:20:37.097302+0000","last_peered":"2023-10-06T05:20:37.097302+0000","last_clean":"2023-10-06T05:20:37.097302+0000","last_became_active":"2023-10-06T05:20:37.087368+0000","last_became_peered":"2023-10-06T05:20:37.087368+0000","last_unstale":"2023-10-06T05:20:37.097302+0000","last_undegraded":"2023-10-06T05:20:37.097302+0000","last_fullsized":"2023-10-06T05:20:37.097302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:06.140443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084792+0000","last_change":"2023-10-06T05:20:37.076365+0000","last_active":"2023-10-06T05:20:37.084792+0000","last_peered":"2023-10-06T05:20:37.084792+0000","last_clean":"2023-10-06T05:20:37.084792+0000","last_became_active":"2023-10-06T05:20:37.075914+0000","last_became_peered":"2023-10-06T05:20:37.075914+0000","last_unstale":"2023-10-06T05:20:37.084792+0000","last_undegraded":"2023-10-06T05:20:37.084792+0000","last_fullsized":"2023-10-06T05:20:37.084792+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:53.721927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089725+0000","last_change":"2023-10-06T05:20:37.082961+0000","last_active":"2023-10-06T05:20:37.089725+0000","last_peered":"2023-10-06T05:20:37.089725+0000","last_clean":"2023-10-06T05:20:37.089725+0000","last_became_active":"2023-10-06T05:20:37.082561+0000","last_became_peered":"2023-10-06T05:20:37.082561+0000","last_unstale":"2023-10-06T05:20:37.089725+0000","last_undegraded":"2023-10-06T05:20:37.089725+0000","last_fullsized":"2023-10-06T05:20:37.089725+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:32.245529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.088731+0000","last_change":"2023-10-06T05:20:37.086011+0000","last_active":"2023-10-06T05:20:37.088731+0000","last_peered":"2023-10-06T05:20:37.088731+0000","last_clean":"2023-10-06T05:20:37.088731+0000","last_became_active":"2023-10-06T05:20:37.085730+0000","last_became_peered":"2023-10-06T05:20:37.085730+0000","last_unstale":"2023-10-06T05:20:37.088731+0000","last_undegraded":"2023-10-06T05:20:37.088731+0000","last_fullsized":"2023-10-06T05:20:37.088731+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:01.873449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095549+0000","last_change":"2023-10-06T05:20:37.090042+0000","last_active":"2023-10-06T05:20:37.095549+0000","last_peered":"2023-10-06T05:20:37.095549+0000","last_clean":"2023-10-06T05:20:37.095549+0000","last_became_active":"2023-10-06T05:20:37.089638+0000","last_became_peered":"2023-10-06T05:20:37.089638+0000","last_unstale":"2023-10-06T05:20:37.095549+0000","last_undegraded":"2023-10-06T05:20:37.095549+0000","last_fullsized":"2023-10-06T05:20:37.095549+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:40.605095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095717+0000","last_change":"2023-10-06T05:20:37.091621+0000","last_active":"2023-10-06T05:20:37.095717+0000","last_peered":"2023-10-06T05:20:37.095717+0000","last_clean":"2023-10-06T05:20:37.095717+0000","last_became_active":"2023-10-06T05:20:37.091389+0000","last_became_peered":"2023-10-06T05:20:37.091389+0000","last_unstale":"2023-10-06T05:20:37.095717+0000","last_undegraded":"2023-10-06T05:20:37.095717+0000","last_fullsized":"2023-10-06T05:20:37.095717+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:24.005054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090066+0000","last_change":"2023-10-06T05:20:37.082625+0000","last_active":"2023-10-06T05:20:37.090066+0000","last_peered":"2023-10-06T05:20:37.090066+0000","last_clean":"2023-10-06T05:20:37.090066+0000","last_became_active":"2023-10-06T05:20:37.081398+0000","last_became_peered":"2023-10-06T05:20:37.081398+0000","last_unstale":"2023-10-06T05:20:37.090066+0000","last_undegraded":"2023-10-06T05:20:37.090066+0000","last_fullsized":"2023-10-06T05:20:37.090066+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:45.130913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090187+0000","last_change":"2023-10-06T05:20:37.089068+0000","last_active":"2023-10-06T05:20:37.090187+0000","last_peered":"2023-10-06T05:20:37.090187+0000","last_clean":"2023-10-06T05:20:37.090187+0000","last_became_active":"2023-10-06T05:20:37.088773+0000","last_became_peered":"2023-10-06T05:20:37.088773+0000","last_unstale":"2023-10-06T05:20:37.090187+0000","last_undegraded":"2023-10-06T05:20:37.090187+0000","last_fullsized":"2023-10-06T05:20:37.090187+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:34.455912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090138+0000","last_change":"2023-10-06T05:20:37.089105+0000","last_active":"2023-10-06T05:20:37.090138+0000","last_peered":"2023-10-06T05:20:37.090138+0000","last_clean":"2023-10-06T05:20:37.090138+0000","last_became_active":"2023-10-06T05:20:37.088785+0000","last_became_peered":"2023-10-06T05:20:37.088785+0000","last_unstale":"2023-10-06T05:20:37.090138+0000","last_undegraded":"2023-10-06T05:20:37.090138+0000","last_fullsized":"2023-10-06T05:20:37.090138+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:06.324052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.087336+0000","last_change":"2023-10-06T05:20:37.085079+0000","last_active":"2023-10-06T05:20:37.087336+0000","last_peered":"2023-10-06T05:20:37.087336+0000","last_clean":"2023-10-06T05:20:37.087336+0000","last_became_active":"2023-10-06T05:20:37.083960+0000","last_became_peered":"2023-10-06T05:20:37.083960+0000","last_unstale":"2023-10-06T05:20:37.087336+0000","last_undegraded":"2023-10-06T05:20:37.087336+0000","last_fullsized":"2023-10-06T05:20:37.087336+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:06.324052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.088058+0000","last_change":"2023-10-06T05:20:37.081143+0000","last_active":"2023-10-06T05:20:37.088058+0000","last_peered":"2023-10-06T05:20:37.088058+0000","last_clean":"2023-10-06T05:20:37.088058+0000","last_became_active":"2023-10-06T05:20:37.080806+0000","last_became_peered":"2023-10-06T05:20:37.080806+0000","last_unstale":"2023-10-06T05:20:37.088058+0000","last_undegraded":"2023-10-06T05:20:37.088058+0000","last_fullsized":"2023-10-06T05:20:37.088058+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:09.413283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090000+0000","last_change":"2023-10-06T05:20:37.082372+0000","last_active":"2023-10-06T05:20:37.090000+0000","last_peered":"2023-10-06T05:20:37.090000+0000","last_clean":"2023-10-06T05:20:37.090000+0000","last_became_active":"2023-10-06T05:20:37.081777+0000","last_became_peered":"2023-10-06T05:20:37.081777+0000","last_unstale":"2023-10-06T05:20:37.090000+0000","last_undegraded":"2023-10-06T05:20:37.090000+0000","last_fullsized":"2023-10-06T05:20:37.090000+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:19.678862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084312+0000","last_change":"2023-10-06T05:20:37.080029+0000","last_active":"2023-10-06T05:20:37.084312+0000","last_peered":"2023-10-06T05:20:37.084312+0000","last_clean":"2023-10-06T05:20:37.084312+0000","last_became_active":"2023-10-06T05:20:37.079637+0000","last_became_peered":"2023-10-06T05:20:37.079637+0000","last_unstale":"2023-10-06T05:20:37.084312+0000","last_undegraded":"2023-10-06T05:20:37.084312+0000","last_fullsized":"2023-10-06T05:20:37.084312+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:23.511829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.096161+0000","last_change":"2023-10-06T05:20:37.094029+0000","last_active":"2023-10-06T05:20:37.096161+0000","last_peered":"2023-10-06T05:20:37.096161+0000","last_clean":"2023-10-06T05:20:37.096161+0000","last_became_active":"2023-10-06T05:20:37.093799+0000","last_became_peered":"2023-10-06T05:20:37.093799+0000","last_unstale":"2023-10-06T05:20:37.096161+0000","last_undegraded":"2023-10-06T05:20:37.096161+0000","last_fullsized":"2023-10-06T05:20:37.096161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:57.053661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.096296+0000","last_change":"2023-10-06T05:20:37.087036+0000","last_active":"2023-10-06T05:20:37.096296+0000","last_peered":"2023-10-06T05:20:37.096296+0000","last_clean":"2023-10-06T05:20:37.096296+0000","last_became_active":"2023-10-06T05:20:37.086691+0000","last_became_peered":"2023-10-06T05:20:37.086691+0000","last_unstale":"2023-10-06T05:20:37.096296+0000","last_undegraded":"2023-10-06T05:20:37.096296+0000","last_fullsized":"2023-10-06T05:20:37.096296+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:07.490551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090655+0000","last_change":"2023-10-06T05:20:37.088904+0000","last_active":"2023-10-06T05:20:37.090655+0000","last_peered":"2023-10-06T05:20:37.090655+0000","last_clean":"2023-10-06T05:20:37.090655+0000","last_became_active":"2023-10-06T05:20:37.088624+0000","last_became_peered":"2023-10-06T05:20:37.088624+0000","last_unstale":"2023-10-06T05:20:37.090655+0000","last_undegraded":"2023-10-06T05:20:37.090655+0000","last_fullsized":"2023-10-06T05:20:37.090655+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:05.044908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.083131+0000","last_change":"2023-10-06T05:20:37.080032+0000","last_active":"2023-10-06T05:20:37.083131+0000","last_peered":"2023-10-06T05:20:37.083131+0000","last_clean":"2023-10-06T05:20:37.083131+0000","last_became_active":"2023-10-06T05:20:37.079743+0000","last_became_peered":"2023-10-06T05:20:37.079743+0000","last_unstale":"2023-10-06T05:20:37.083131+0000","last_undegraded":"2023-10-06T05:20:37.083131+0000","last_fullsized":"2023-10-06T05:20:37.083131+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:28.844309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.076936+0000","last_change":"2023-10-06T05:20:37.073826+0000","last_active":"2023-10-06T05:20:37.076936+0000","last_peered":"2023-10-06T05:20:37.076936+0000","last_clean":"2023-10-06T05:20:37.076936+0000","last_became_active":"2023-10-06T05:20:37.073380+0000","last_became_peered":"2023-10-06T05:20:37.073380+0000","last_unstale":"2023-10-06T05:20:37.076936+0000","last_undegraded":"2023-10-06T05:20:37.076936+0000","last_fullsized":"2023-10-06T05:20:37.076936+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:28.307492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095743+0000","last_change":"2023-10-06T05:20:37.093548+0000","last_active":"2023-10-06T05:20:37.095743+0000","last_peered":"2023-10-06T05:20:37.095743+0000","last_clean":"2023-10-06T05:20:37.095743+0000","last_became_active":"2023-10-06T05:20:37.092348+0000","last_became_peered":"2023-10-06T05:20:37.092348+0000","last_unstale":"2023-10-06T05:20:37.095743+0000","last_undegraded":"2023-10-06T05:20:37.095743+0000","last_fullsized":"2023-10-06T05:20:37.095743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:56.892725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"56'4","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.269664+0000","last_change":"2023-10-06T05:20:36.263537+0000","last_active":"2023-10-06T05:20:37.269664+0000","last_peered":"2023-10-06T05:20:37.269664+0000","last_clean":"2023-10-06T05:20:37.269664+0000","last_became_active":"2023-10-06T05:20:36.263078+0000","last_became_peered":"2023-10-06T05:20:36.263078+0000","last_unstale":"2023-10-06T05:20:37.269664+0000","last_undegraded":"2023-10-06T05:20:37.269664+0000","last_fullsized":"2023-10-06T05:20:37.269664+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:15.315585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.097883+0000","last_change":"2023-10-06T05:20:37.096408+0000","last_active":"2023-10-06T05:20:37.097883+0000","last_peered":"2023-10-06T05:20:37.097883+0000","last_clean":"2023-10-06T05:20:37.097883+0000","last_became_active":"2023-10-06T05:20:37.095989+0000","last_became_peered":"2023-10-06T05:20:37.095989+0000","last_unstale":"2023-10-06T05:20:37.097883+0000","last_undegraded":"2023-10-06T05:20:37.097883+0000","last_fullsized":"2023-10-06T05:20:37.097883+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:45.130913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"56'1","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.267701+0000","last_change":"2023-10-06T05:20:36.262702+0000","last_active":"2023-10-06T05:20:37.267701+0000","last_peered":"2023-10-06T05:20:37.267701+0000","last_clean":"2023-10-06T05:20:37.267701+0000","last_became_active":"2023-10-06T05:20:36.262219+0000","last_became_peered":"2023-10-06T05:20:36.262219+0000","last_unstale":"2023-10-06T05:20:37.267701+0000","last_undegraded":"2023-10-06T05:20:37.267701+0000","last_fullsized":"2023-10-06T05:20:37.267701+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:58.138911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.075135+0000","last_change":"2023-10-06T05:20:37.069595+0000","last_active":"2023-10-06T05:20:37.075135+0000","last_peered":"2023-10-06T05:20:37.075135+0000","last_clean":"2023-10-06T05:20:37.075135+0000","last_became_active":"2023-10-06T05:20:37.069350+0000","last_became_peered":"2023-10-06T05:20:37.069350+0000","last_unstale":"2023-10-06T05:20:37.075135+0000","last_undegraded":"2023-10-06T05:20:37.075135+0000","last_fullsized":"2023-10-06T05:20:37.075135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:35.259063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.269104+0000","last_change":"2023-10-06T05:20:36.267484+0000","last_active":"2023-10-06T05:20:37.269104+0000","last_peered":"2023-10-06T05:20:37.269104+0000","last_clean":"2023-10-06T05:20:37.269104+0000","last_became_active":"2023-10-06T05:20:36.267135+0000","last_became_peered":"2023-10-06T05:20:36.267135+0000","last_unstale":"2023-10-06T05:20:37.269104+0000","last_undegraded":"2023-10-06T05:20:37.269104+0000","last_fullsized":"2023-10-06T05:20:37.269104+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:08.681506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.097368+0000","last_change":"2023-10-06T05:20:37.088927+0000","last_active":"2023-10-06T05:20:37.097368+0000","last_peered":"2023-10-06T05:20:37.097368+0000","last_clean":"2023-10-06T05:20:37.097368+0000","last_became_active":"2023-10-06T05:20:37.088720+0000","last_became_peered":"2023-10-06T05:20:37.088720+0000","last_unstale":"2023-10-06T05:20:37.097368+0000","last_undegraded":"2023-10-06T05:20:37.097368+0000","last_fullsized":"2023-10-06T05:20:37.097368+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:59.144810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.266734+0000","last_change":"2023-10-06T05:20:36.258267+0000","last_active":"2023-10-06T05:20:37.266734+0000","last_peered":"2023-10-06T05:20:37.266734+0000","last_clean":"2023-10-06T05:20:37.266734+0000","last_became_active":"2023-10-06T05:20:36.258006+0000","last_became_peered":"2023-10-06T05:20:36.258006+0000","last_unstale":"2023-10-06T05:20:37.266734+0000","last_undegraded":"2023-10-06T05:20:37.266734+0000","last_fullsized":"2023-10-06T05:20:37.266734+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:08.407384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084820+0000","last_change":"2023-10-06T05:20:37.077780+0000","last_active":"2023-10-06T05:20:37.084820+0000","last_peered":"2023-10-06T05:20:37.084820+0000","last_clean":"2023-10-06T05:20:37.084820+0000","last_became_active":"2023-10-06T05:20:37.077526+0000","last_became_peered":"2023-10-06T05:20:37.077526+0000","last_unstale":"2023-10-06T05:20:37.084820+0000","last_undegraded":"2023-10-06T05:20:37.084820+0000","last_fullsized":"2023-10-06T05:20:37.084820+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:17.450471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"56'1","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.266066+0000","last_change":"2023-10-06T05:20:36.254716+0000","last_active":"2023-10-06T05:20:37.266066+0000","last_peered":"2023-10-06T05:20:37.266066+0000","last_clean":"2023-10-06T05:20:37.266066+0000","last_became_active":"2023-10-06T05:20:36.254166+0000","last_became_peered":"2023-10-06T05:20:36.254166+0000","last_unstale":"2023-10-06T05:20:37.266066+0000","last_undegraded":"2023-10-06T05:20:37.266066+0000","last_fullsized":"2023-10-06T05:20:37.266066+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:41.018227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089538+0000","last_change":"2023-10-06T05:20:37.082671+0000","last_active":"2023-10-06T05:20:37.089538+0000","last_peered":"2023-10-06T05:20:37.089538+0000","last_clean":"2023-10-06T05:20:37.089538+0000","last_became_active":"2023-10-06T05:20:37.082366+0000","last_became_peered":"2023-10-06T05:20:37.082366+0000","last_unstale":"2023-10-06T05:20:37.089538+0000","last_undegraded":"2023-10-06T05:20:37.089538+0000","last_fullsized":"2023-10-06T05:20:37.089538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:00.119383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"56'1","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.265918+0000","last_change":"2023-10-06T05:20:36.258045+0000","last_active":"2023-10-06T05:20:37.265918+0000","last_peered":"2023-10-06T05:20:37.265918+0000","last_clean":"2023-10-06T05:20:37.265918+0000","last_became_active":"2023-10-06T05:20:36.257730+0000","last_became_peered":"2023-10-06T05:20:36.257730+0000","last_unstale":"2023-10-06T05:20:37.265918+0000","last_undegraded":"2023-10-06T05:20:37.265918+0000","last_fullsized":"2023-10-06T05:20:37.265918+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:35.551141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095613+0000","last_change":"2023-10-06T05:20:37.090269+0000","last_active":"2023-10-06T05:20:37.095613+0000","last_peered":"2023-10-06T05:20:37.095613+0000","last_clean":"2023-10-06T05:20:37.095613+0000","last_became_active":"2023-10-06T05:20:37.089436+0000","last_became_peered":"2023-10-06T05:20:37.089436+0000","last_unstale":"2023-10-06T05:20:37.095613+0000","last_undegraded":"2023-10-06T05:20:37.095613+0000","last_fullsized":"2023-10-06T05:20:37.095613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:36.510841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.064669+0000","last_change":"2023-10-06T05:20:36.261897+0000","last_active":"2023-10-06T05:20:37.064669+0000","last_peered":"2023-10-06T05:20:37.064669+0000","last_clean":"2023-10-06T05:20:37.064669+0000","last_became_active":"2023-10-06T05:20:36.261603+0000","last_became_peered":"2023-10-06T05:20:36.261603+0000","last_unstale":"2023-10-06T05:20:37.064669+0000","last_undegraded":"2023-10-06T05:20:37.064669+0000","last_fullsized":"2023-10-06T05:20:37.064669+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:37.420899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.081722+0000","last_change":"2023-10-06T05:20:37.078582+0000","last_active":"2023-10-06T05:20:37.081722+0000","last_peered":"2023-10-06T05:20:37.081722+0000","last_clean":"2023-10-06T05:20:37.081722+0000","last_became_active":"2023-10-06T05:20:37.078114+0000","last_became_peered":"2023-10-06T05:20:37.078114+0000","last_unstale":"2023-10-06T05:20:37.081722+0000","last_undegraded":"2023-10-06T05:20:37.081722+0000","last_fullsized":"2023-10-06T05:20:37.081722+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:26.019845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.085729+0000","last_change":"2023-10-06T05:20:36.262979+0000","last_active":"2023-10-06T05:20:37.085729+0000","last_peered":"2023-10-06T05:20:37.085729+0000","last_clean":"2023-10-06T05:20:37.085729+0000","last_became_active":"2023-10-06T05:20:36.262645+0000","last_became_peered":"2023-10-06T05:20:36.262645+0000","last_unstale":"2023-10-06T05:20:37.085729+0000","last_undegraded":"2023-10-06T05:20:37.085729+0000","last_fullsized":"2023-10-06T05:20:37.085729+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:49.246636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.085046+0000","last_change":"2023-10-06T05:20:37.080406+0000","last_active":"2023-10-06T05:20:37.085046+0000","last_peered":"2023-10-06T05:20:37.085046+0000","last_clean":"2023-10-06T05:20:37.085046+0000","last_became_active":"2023-10-06T05:20:37.080067+0000","last_became_peered":"2023-10-06T05:20:37.080067+0000","last_unstale":"2023-10-06T05:20:37.085046+0000","last_undegraded":"2023-10-06T05:20:37.085046+0000","last_fullsized":"2023-10-06T05:20:37.085046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:53.917741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.073446+0000","last_change":"2023-10-06T05:20:36.261789+0000","last_active":"2023-10-06T05:20:37.073446+0000","last_peered":"2023-10-06T05:20:37.073446+0000","last_clean":"2023-10-06T05:20:37.073446+0000","last_became_active":"2023-10-06T05:20:36.261531+0000","last_became_peered":"2023-10-06T05:20:36.261531+0000","last_unstale":"2023-10-06T05:20:37.073446+0000","last_undegraded":"2023-10-06T05:20:37.073446+0000","last_fullsized":"2023-10-06T05:20:37.073446+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:38.304771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.077534+0000","last_change":"2023-10-06T05:20:37.076107+0000","last_active":"2023-10-06T05:20:37.077534+0000","last_peered":"2023-10-06T05:20:37.077534+0000","last_clean":"2023-10-06T05:20:37.077534+0000","last_became_active":"2023-10-06T05:20:37.075776+0000","last_became_peered":"2023-10-06T05:20:37.075776+0000","last_unstale":"2023-10-06T05:20:37.077534+0000","last_undegraded":"2023-10-06T05:20:37.077534+0000","last_fullsized":"2023-10-06T05:20:37.077534+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:43.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.076605+0000","last_change":"2023-10-06T05:20:36.265722+0000","last_active":"2023-10-06T05:20:37.076605+0000","last_peered":"2023-10-06T05:20:37.076605+0000","last_clean":"2023-10-06T05:20:37.076605+0000","last_became_active":"2023-10-06T05:20:36.265182+0000","last_became_peered":"2023-10-06T05:20:36.265182+0000","last_unstale":"2023-10-06T05:20:37.076605+0000","last_undegraded":"2023-10-06T05:20:37.076605+0000","last_fullsized":"2023-10-06T05:20:37.076605+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:48.090020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095714+0000","last_change":"2023-10-06T05:20:37.094118+0000","last_active":"2023-10-06T05:20:37.095714+0000","last_peered":"2023-10-06T05:20:37.095714+0000","last_clean":"2023-10-06T05:20:37.095714+0000","last_became_active":"2023-10-06T05:20:37.093803+0000","last_became_peered":"2023-10-06T05:20:37.093803+0000","last_unstale":"2023-10-06T05:20:37.095714+0000","last_undegraded":"2023-10-06T05:20:37.095714+0000","last_fullsized":"2023-10-06T05:20:37.095714+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:22.698300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"56'3","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.268071+0000","last_change":"2023-10-06T05:20:36.261993+0000","last_active":"2023-10-06T05:20:37.268071+0000","last_peered":"2023-10-06T05:20:37.268071+0000","last_clean":"2023-10-06T05:20:37.268071+0000","last_became_active":"2023-10-06T05:20:36.261433+0000","last_became_peered":"2023-10-06T05:20:36.261433+0000","last_unstale":"2023-10-06T05:20:37.268071+0000","last_undegraded":"2023-10-06T05:20:37.268071+0000","last_fullsized":"2023-10-06T05:20:37.268071+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:11.974908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084755+0000","last_change":"2023-10-06T05:20:37.081584+0000","last_active":"2023-10-06T05:20:37.084755+0000","last_peered":"2023-10-06T05:20:37.084755+0000","last_clean":"2023-10-06T05:20:37.084755+0000","last_became_active":"2023-10-06T05:20:37.081298+0000","last_became_peered":"2023-10-06T05:20:37.081298+0000","last_unstale":"2023-10-06T05:20:37.084755+0000","last_undegraded":"2023-10-06T05:20:37.084755+0000","last_fullsized":"2023-10-06T05:20:37.084755+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:39.310670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"56'6","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.271413+0000","last_change":"2023-10-06T05:20:36.260052+0000","last_active":"2023-10-06T05:20:37.271413+0000","last_peered":"2023-10-06T05:20:37.271413+0000","last_clean":"2023-10-06T05:20:37.271413+0000","last_became_active":"2023-10-06T05:20:36.259719+0000","last_became_peered":"2023-10-06T05:20:36.259719+0000","last_unstale":"2023-10-06T05:20:37.271413+0000","last_undegraded":"2023-10-06T05:20:37.271413+0000","last_fullsized":"2023-10-06T05:20:37.271413+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:54.688059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.094675+0000","last_change":"2023-10-06T05:20:37.091989+0000","last_active":"2023-10-06T05:20:37.094675+0000","last_peered":"2023-10-06T05:20:37.094675+0000","last_clean":"2023-10-06T05:20:37.094675+0000","last_became_active":"2023-10-06T05:20:37.091713+0000","last_became_peered":"2023-10-06T05:20:37.091713+0000","last_unstale":"2023-10-06T05:20:37.094675+0000","last_undegraded":"2023-10-06T05:20:37.094675+0000","last_fullsized":"2023-10-06T05:20:37.094675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:36.557041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.269794+0000","last_change":"2023-10-06T05:20:36.262413+0000","last_active":"2023-10-06T05:20:37.269794+0000","last_peered":"2023-10-06T05:20:37.269794+0000","last_clean":"2023-10-06T05:20:37.269794+0000","last_became_active":"2023-10-06T05:20:36.262111+0000","last_became_peered":"2023-10-06T05:20:36.262111+0000","last_unstale":"2023-10-06T05:20:37.269794+0000","last_undegraded":"2023-10-06T05:20:37.269794+0000","last_fullsized":"2023-10-06T05:20:37.269794+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:14.895156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.088417+0000","last_change":"2023-10-06T05:20:37.079786+0000","last_active":"2023-10-06T05:20:37.088417+0000","last_peered":"2023-10-06T05:20:37.088417+0000","last_clean":"2023-10-06T05:20:37.088417+0000","last_became_active":"2023-10-06T05:20:37.079160+0000","last_became_peered":"2023-10-06T05:20:37.079160+0000","last_unstale":"2023-10-06T05:20:37.088417+0000","last_undegraded":"2023-10-06T05:20:37.088417+0000","last_fullsized":"2023-10-06T05:20:37.088417+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:23.511829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.071864+0000","last_change":"2023-10-06T05:20:36.265000+0000","last_active":"2023-10-06T05:20:37.071864+0000","last_peered":"2023-10-06T05:20:37.071864+0000","last_clean":"2023-10-06T05:20:37.071864+0000","last_became_active":"2023-10-06T05:20:36.264733+0000","last_became_peered":"2023-10-06T05:20:36.264733+0000","last_unstale":"2023-10-06T05:20:37.071864+0000","last_undegraded":"2023-10-06T05:20:37.071864+0000","last_fullsized":"2023-10-06T05:20:37.071864+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:52.911842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095588+0000","last_change":"2023-10-06T05:20:37.090065+0000","last_active":"2023-10-06T05:20:37.095588+0000","last_peered":"2023-10-06T05:20:37.095588+0000","last_clean":"2023-10-06T05:20:37.095588+0000","last_became_active":"2023-10-06T05:20:37.089032+0000","last_became_peered":"2023-10-06T05:20:37.089032+0000","last_unstale":"2023-10-06T05:20:37.095588+0000","last_undegraded":"2023-10-06T05:20:37.095588+0000","last_fullsized":"2023-10-06T05:20:37.095588+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:36.693475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.072838+0000","last_change":"2023-10-06T05:20:36.262438+0000","last_active":"2023-10-06T05:20:37.072838+0000","last_peered":"2023-10-06T05:20:37.072838+0000","last_clean":"2023-10-06T05:20:37.072838+0000","last_became_active":"2023-10-06T05:20:36.261896+0000","last_became_peered":"2023-10-06T05:20:36.261896+0000","last_unstale":"2023-10-06T05:20:37.072838+0000","last_undegraded":"2023-10-06T05:20:37.072838+0000","last_fullsized":"2023-10-06T05:20:37.072838+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:03.522991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.090317+0000","last_change":"2023-10-06T05:20:37.089025+0000","last_active":"2023-10-06T05:20:37.090317+0000","last_peered":"2023-10-06T05:20:37.090317+0000","last_clean":"2023-10-06T05:20:37.090317+0000","last_became_active":"2023-10-06T05:20:37.088747+0000","last_became_peered":"2023-10-06T05:20:37.088747+0000","last_unstale":"2023-10-06T05:20:37.090317+0000","last_undegraded":"2023-10-06T05:20:37.090317+0000","last_fullsized":"2023-10-06T05:20:37.090317+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:45.388735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"56'8","reported_seq":18,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.270992+0000","last_change":"2023-10-06T05:20:36.268415+0000","last_active":"2023-10-06T05:20:37.270992+0000","last_peered":"2023-10-06T05:20:37.270992+0000","last_clean":"2023-10-06T05:20:37.270992+0000","last_became_active":"2023-10-06T05:20:36.268002+0000","last_became_peered":"2023-10-06T05:20:36.268002+0000","last_unstale":"2023-10-06T05:20:37.270992+0000","last_undegraded":"2023-10-06T05:20:37.270992+0000","last_fullsized":"2023-10-06T05:20:37.270992+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:59.999307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.085147+0000","last_change":"2023-10-06T05:20:37.078999+0000","last_active":"2023-10-06T05:20:37.085147+0000","last_peered":"2023-10-06T05:20:37.085147+0000","last_clean":"2023-10-06T05:20:37.085147+0000","last_became_active":"2023-10-06T05:20:37.078672+0000","last_became_peered":"2023-10-06T05:20:37.078672+0000","last_unstale":"2023-10-06T05:20:37.085147+0000","last_undegraded":"2023-10-06T05:20:37.085147+0000","last_fullsized":"2023-10-06T05:20:37.085147+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:56.599683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.267857+0000","last_change":"2023-10-06T05:20:36.270612+0000","last_active":"2023-10-06T05:20:37.267857+0000","last_peered":"2023-10-06T05:20:37.267857+0000","last_clean":"2023-10-06T05:20:37.267857+0000","last_became_active":"2023-10-06T05:20:36.270240+0000","last_became_peered":"2023-10-06T05:20:36.270240+0000","last_unstale":"2023-10-06T05:20:37.267857+0000","last_undegraded":"2023-10-06T05:20:37.267857+0000","last_fullsized":"2023-10-06T05:20:37.267857+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:59.113484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.079467+0000","last_change":"2023-10-06T05:20:37.072546+0000","last_active":"2023-10-06T05:20:37.079467+0000","last_peered":"2023-10-06T05:20:37.079467+0000","last_clean":"2023-10-06T05:20:37.079467+0000","last_became_active":"2023-10-06T05:20:37.072095+0000","last_became_peered":"2023-10-06T05:20:37.072095+0000","last_unstale":"2023-10-06T05:20:37.079467+0000","last_undegraded":"2023-10-06T05:20:37.079467+0000","last_fullsized":"2023-10-06T05:20:37.079467+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:57.809186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.072151+0000","last_change":"2023-10-06T05:20:36.264173+0000","last_active":"2023-10-06T05:20:37.072151+0000","last_peered":"2023-10-06T05:20:37.072151+0000","last_clean":"2023-10-06T05:20:37.072151+0000","last_became_active":"2023-10-06T05:20:36.263863+0000","last_became_peered":"2023-10-06T05:20:36.263863+0000","last_unstale":"2023-10-06T05:20:37.072151+0000","last_undegraded":"2023-10-06T05:20:37.072151+0000","last_fullsized":"2023-10-06T05:20:37.072151+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:52.407745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089970+0000","last_change":"2023-10-06T05:20:37.082151+0000","last_active":"2023-10-06T05:20:37.089970+0000","last_peered":"2023-10-06T05:20:37.089970+0000","last_clean":"2023-10-06T05:20:37.089970+0000","last_became_active":"2023-10-06T05:20:37.081358+0000","last_became_peered":"2023-10-06T05:20:37.081358+0000","last_unstale":"2023-10-06T05:20:37.089970+0000","last_undegraded":"2023-10-06T05:20:37.089970+0000","last_fullsized":"2023-10-06T05:20:37.089970+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:42.024126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.065894+0000","last_change":"2023-10-06T05:20:36.267479+0000","last_active":"2023-10-06T05:20:37.065894+0000","last_peered":"2023-10-06T05:20:37.065894+0000","last_clean":"2023-10-06T05:20:37.065894+0000","last_became_active":"2023-10-06T05:20:36.267147+0000","last_became_peered":"2023-10-06T05:20:36.267147+0000","last_unstale":"2023-10-06T05:20:37.065894+0000","last_undegraded":"2023-10-06T05:20:37.065894+0000","last_fullsized":"2023-10-06T05:20:37.065894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:45.279732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095680+0000","last_change":"2023-10-06T05:20:37.091301+0000","last_active":"2023-10-06T05:20:37.095680+0000","last_peered":"2023-10-06T05:20:37.095680+0000","last_clean":"2023-10-06T05:20:37.095680+0000","last_became_active":"2023-10-06T05:20:37.091039+0000","last_became_peered":"2023-10-06T05:20:37.091039+0000","last_unstale":"2023-10-06T05:20:37.095680+0000","last_undegraded":"2023-10-06T05:20:37.095680+0000","last_fullsized":"2023-10-06T05:20:37.095680+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:39.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.073450+0000","last_change":"2023-10-06T05:20:36.259657+0000","last_active":"2023-10-06T05:20:37.073450+0000","last_peered":"2023-10-06T05:20:37.073450+0000","last_clean":"2023-10-06T05:20:37.073450+0000","last_became_active":"2023-10-06T05:20:36.259336+0000","last_became_peered":"2023-10-06T05:20:36.259336+0000","last_unstale":"2023-10-06T05:20:37.073450+0000","last_undegraded":"2023-10-06T05:20:37.073450+0000","last_fullsized":"2023-10-06T05:20:37.073450+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:55.886826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.096706+0000","last_change":"2023-10-06T05:20:37.094362+0000","last_active":"2023-10-06T05:20:37.096706+0000","last_peered":"2023-10-06T05:20:37.096706+0000","last_clean":"2023-10-06T05:20:37.096706+0000","last_became_active":"2023-10-06T05:20:37.093995+0000","last_became_peered":"2023-10-06T05:20:37.093995+0000","last_unstale":"2023-10-06T05:20:37.096706+0000","last_undegraded":"2023-10-06T05:20:37.096706+0000","last_fullsized":"2023-10-06T05:20:37.096706+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:54.749017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.071130+0000","last_change":"2023-10-06T05:20:36.262413+0000","last_active":"2023-10-06T05:20:37.071130+0000","last_peered":"2023-10-06T05:20:37.071130+0000","last_clean":"2023-10-06T05:20:37.071130+0000","last_became_active":"2023-10-06T05:20:36.262136+0000","last_became_peered":"2023-10-06T05:20:36.262136+0000","last_unstale":"2023-10-06T05:20:37.071130+0000","last_undegraded":"2023-10-06T05:20:37.071130+0000","last_fullsized":"2023-10-06T05:20:37.071130+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:44.125014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.094636+0000","last_change":"2023-10-06T05:20:37.091611+0000","last_active":"2023-10-06T05:20:37.094636+0000","last_peered":"2023-10-06T05:20:37.094636+0000","last_clean":"2023-10-06T05:20:37.094636+0000","last_became_active":"2023-10-06T05:20:37.091269+0000","last_became_peered":"2023-10-06T05:20:37.091269+0000","last_unstale":"2023-10-06T05:20:37.094636+0000","last_undegraded":"2023-10-06T05:20:37.094636+0000","last_fullsized":"2023-10-06T05:20:37.094636+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:42.024126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.064262+0000","last_change":"2023-10-06T05:20:36.258074+0000","last_active":"2023-10-06T05:20:37.064262+0000","last_peered":"2023-10-06T05:20:37.064262+0000","last_clean":"2023-10-06T05:20:37.064262+0000","last_became_active":"2023-10-06T05:20:36.257783+0000","last_became_peered":"2023-10-06T05:20:36.257783+0000","last_unstale":"2023-10-06T05:20:37.064262+0000","last_undegraded":"2023-10-06T05:20:37.064262+0000","last_fullsized":"2023-10-06T05:20:37.064262+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:14.895156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089627+0000","last_change":"2023-10-06T05:20:37.083318+0000","last_active":"2023-10-06T05:20:37.089627+0000","last_peered":"2023-10-06T05:20:37.089627+0000","last_clean":"2023-10-06T05:20:37.089627+0000","last_became_active":"2023-10-06T05:20:37.083008+0000","last_became_peered":"2023-10-06T05:20:37.083008+0000","last_unstale":"2023-10-06T05:20:37.089627+0000","last_undegraded":"2023-10-06T05:20:37.089627+0000","last_fullsized":"2023-10-06T05:20:37.089627+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:26.067622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.064805+0000","last_change":"2023-10-06T05:20:36.261828+0000","last_active":"2023-10-06T05:20:37.064805+0000","last_peered":"2023-10-06T05:20:37.064805+0000","last_clean":"2023-10-06T05:20:37.064805+0000","last_became_active":"2023-10-06T05:20:36.261585+0000","last_became_peered":"2023-10-06T05:20:36.261585+0000","last_unstale":"2023-10-06T05:20:37.064805+0000","last_undegraded":"2023-10-06T05:20:37.064805+0000","last_fullsized":"2023-10-06T05:20:37.064805+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:21.102992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.084672+0000","last_change":"2023-10-06T05:20:37.076025+0000","last_active":"2023-10-06T05:20:37.084672+0000","last_peered":"2023-10-06T05:20:37.084672+0000","last_clean":"2023-10-06T05:20:37.084672+0000","last_became_active":"2023-10-06T05:20:37.075681+0000","last_became_peered":"2023-10-06T05:20:37.075681+0000","last_unstale":"2023-10-06T05:20:37.084672+0000","last_undegraded":"2023-10-06T05:20:37.084672+0000","last_fullsized":"2023-10-06T05:20:37.084672+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:21.147856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.072852+0000","last_change":"2023-10-06T05:20:36.269404+0000","last_active":"2023-10-06T05:20:37.072852+0000","last_peered":"2023-10-06T05:20:37.072852+0000","last_clean":"2023-10-06T05:20:37.072852+0000","last_became_active":"2023-10-06T05:20:36.269137+0000","last_became_peered":"2023-10-06T05:20:36.269137+0000","last_unstale":"2023-10-06T05:20:37.072852+0000","last_undegraded":"2023-10-06T05:20:37.072852+0000","last_fullsized":"2023-10-06T05:20:37.072852+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:37.100558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.097892+0000","last_change":"2023-10-06T05:20:37.096590+0000","last_active":"2023-10-06T05:20:37.097892+0000","last_peered":"2023-10-06T05:20:37.097892+0000","last_clean":"2023-10-06T05:20:37.097892+0000","last_became_active":"2023-10-06T05:20:37.096094+0000","last_became_peered":"2023-10-06T05:20:37.096094+0000","last_unstale":"2023-10-06T05:20:37.097892+0000","last_undegraded":"2023-10-06T05:20:37.097892+0000","last_fullsized":"2023-10-06T05:20:37.097892+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:15.901055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"56'1","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.268004+0000","last_change":"2023-10-06T05:20:36.261789+0000","last_active":"2023-10-06T05:20:37.268004+0000","last_peered":"2023-10-06T05:20:37.268004+0000","last_clean":"2023-10-06T05:20:37.268004+0000","last_became_active":"2023-10-06T05:20:36.261466+0000","last_became_peered":"2023-10-06T05:20:36.261466+0000","last_unstale":"2023-10-06T05:20:37.268004+0000","last_undegraded":"2023-10-06T05:20:37.268004+0000","last_fullsized":"2023-10-06T05:20:37.268004+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:03:00.867550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":46,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.075424+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:20:37.075424+0000","last_peered":"2023-10-06T05:20:37.075424+0000","last_clean":"2023-10-06T05:20:37.075424+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:20:37.075424+0000","last_undegraded":"2023-10-06T05:20:37.075424+0000","last_fullsized":"2023-10-06T05:20:37.075424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095705+0000","last_change":"2023-10-06T05:20:37.091617+0000","last_active":"2023-10-06T05:20:37.095705+0000","last_peered":"2023-10-06T05:20:37.095705+0000","last_clean":"2023-10-06T05:20:37.095705+0000","last_became_active":"2023-10-06T05:20:37.091300+0000","last_became_peered":"2023-10-06T05:20:37.091300+0000","last_unstale":"2023-10-06T05:20:37.095705+0000","last_undegraded":"2023-10-06T05:20:37.095705+0000","last_fullsized":"2023-10-06T05:20:37.095705+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:33.806691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.064800+0000","last_change":"2023-10-06T05:20:36.259736+0000","last_active":"2023-10-06T05:20:37.064800+0000","last_peered":"2023-10-06T05:20:37.064800+0000","last_clean":"2023-10-06T05:20:37.064800+0000","last_became_active":"2023-10-06T05:20:36.259467+0000","last_became_peered":"2023-10-06T05:20:36.259467+0000","last_unstale":"2023-10-06T05:20:37.064800+0000","last_undegraded":"2023-10-06T05:20:37.064800+0000","last_fullsized":"2023-10-06T05:20:37.064800+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:04.039009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.064883+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:20:37.064883+0000","last_peered":"2023-10-06T05:20:37.064883+0000","last_clean":"2023-10-06T05:20:37.064883+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:20:37.064883+0000","last_undegraded":"2023-10-06T05:20:37.064883+0000","last_fullsized":"2023-10-06T05:20:37.064883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.076904+0000","last_change":"2023-10-06T05:20:37.073646+0000","last_active":"2023-10-06T05:20:37.076904+0000","last_peered":"2023-10-06T05:20:37.076904+0000","last_clean":"2023-10-06T05:20:37.076904+0000","last_became_active":"2023-10-06T05:20:37.073047+0000","last_became_peered":"2023-10-06T05:20:37.073047+0000","last_unstale":"2023-10-06T05:20:37.076904+0000","last_undegraded":"2023-10-06T05:20:37.076904+0000","last_fullsized":"2023-10-06T05:20:37.076904+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:39.205579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.267667+0000","last_change":"2023-10-06T05:20:36.267555+0000","last_active":"2023-10-06T05:20:37.267667+0000","last_peered":"2023-10-06T05:20:37.267667+0000","last_clean":"2023-10-06T05:20:37.267667+0000","last_became_active":"2023-10-06T05:20:36.267330+0000","last_became_peered":"2023-10-06T05:20:36.267330+0000","last_unstale":"2023-10-06T05:20:37.267667+0000","last_undegraded":"2023-10-06T05:20:37.267667+0000","last_fullsized":"2023-10-06T05:20:37.267667+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:33.450013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":45,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.073621+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:20:37.073621+0000","last_peered":"2023-10-06T05:20:37.073621+0000","last_clean":"2023-10-06T05:20:37.073621+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:20:37.073621+0000","last_undegraded":"2023-10-06T05:20:37.073621+0000","last_fullsized":"2023-10-06T05:20:37.073621+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":192,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.070315+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:20:37.070315+0000","last_peered":"2023-10-06T05:20:37.070315+0000","last_clean":"2023-10-06T05:20:37.070315+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:20:37.070315+0000","last_undegraded":"2023-10-06T05:20:37.070315+0000","last_fullsized":"2023-10-06T05:20:37.070315+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.086299+0000","last_change":"2023-10-06T05:20:37.079405+0000","last_active":"2023-10-06T05:20:37.086299+0000","last_peered":"2023-10-06T05:20:37.086299+0000","last_clean":"2023-10-06T05:20:37.086299+0000","last_became_active":"2023-10-06T05:20:37.078627+0000","last_became_peered":"2023-10-06T05:20:37.078627+0000","last_unstale":"2023-10-06T05:20:37.086299+0000","last_undegraded":"2023-10-06T05:20:37.086299+0000","last_fullsized":"2023-10-06T05:20:37.086299+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:38.426798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.071961+0000","last_change":"2023-10-06T05:20:36.264980+0000","last_active":"2023-10-06T05:20:37.071961+0000","last_peered":"2023-10-06T05:20:37.071961+0000","last_clean":"2023-10-06T05:20:37.071961+0000","last_became_active":"2023-10-06T05:20:36.264703+0000","last_became_peered":"2023-10-06T05:20:36.264703+0000","last_unstale":"2023-10-06T05:20:37.071961+0000","last_undegraded":"2023-10-06T05:20:37.071961+0000","last_fullsized":"2023-10-06T05:20:37.071961+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:44.382836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.072529+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:20:37.072529+0000","last_peered":"2023-10-06T05:20:37.072529+0000","last_clean":"2023-10-06T05:20:37.072529+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:20:37.072529+0000","last_undegraded":"2023-10-06T05:20:37.072529+0000","last_fullsized":"2023-10-06T05:20:37.072529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.096918+0000","last_change":"2023-10-06T05:20:37.094903+0000","last_active":"2023-10-06T05:20:37.096918+0000","last_peered":"2023-10-06T05:20:37.096918+0000","last_clean":"2023-10-06T05:20:37.096918+0000","last_became_active":"2023-10-06T05:20:37.093433+0000","last_became_peered":"2023-10-06T05:20:37.093433+0000","last_unstale":"2023-10-06T05:20:37.096918+0000","last_undegraded":"2023-10-06T05:20:37.096918+0000","last_fullsized":"2023-10-06T05:20:37.096918+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:55.693958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.066908+0000","last_change":"2023-10-06T05:20:36.254703+0000","last_active":"2023-10-06T05:20:37.066908+0000","last_peered":"2023-10-06T05:20:37.066908+0000","last_clean":"2023-10-06T05:20:37.066908+0000","last_became_active":"2023-10-06T05:20:36.254422+0000","last_became_peered":"2023-10-06T05:20:36.254422+0000","last_unstale":"2023-10-06T05:20:37.066908+0000","last_undegraded":"2023-10-06T05:20:37.066908+0000","last_fullsized":"2023-10-06T05:20:37.066908+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:44.581770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.070422+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:20:37.070422+0000","last_peered":"2023-10-06T05:20:37.070422+0000","last_clean":"2023-10-06T05:20:37.070422+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:20:37.070422+0000","last_undegraded":"2023-10-06T05:20:37.070422+0000","last_fullsized":"2023-10-06T05:20:37.070422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095316+0000","last_change":"2023-10-06T05:20:37.089844+0000","last_active":"2023-10-06T05:20:37.095316+0000","last_peered":"2023-10-06T05:20:37.095316+0000","last_clean":"2023-10-06T05:20:37.095316+0000","last_became_active":"2023-10-06T05:20:37.088699+0000","last_became_peered":"2023-10-06T05:20:37.088699+0000","last_unstale":"2023-10-06T05:20:37.095316+0000","last_undegraded":"2023-10-06T05:20:37.095316+0000","last_fullsized":"2023-10-06T05:20:37.095316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:44.095572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.065567+0000","last_change":"2023-10-06T05:20:36.263060+0000","last_active":"2023-10-06T05:20:37.065567+0000","last_peered":"2023-10-06T05:20:37.065567+0000","last_clean":"2023-10-06T05:20:37.065567+0000","last_became_active":"2023-10-06T05:20:36.262695+0000","last_became_peered":"2023-10-06T05:20:36.262695+0000","last_unstale":"2023-10-06T05:20:37.065567+0000","last_undegraded":"2023-10-06T05:20:37.065567+0000","last_fullsized":"2023-10-06T05:20:37.065567+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:05.134543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.070068+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:20:37.070068+0000","last_peered":"2023-10-06T05:20:37.070068+0000","last_clean":"2023-10-06T05:20:37.070068+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:20:37.070068+0000","last_undegraded":"2023-10-06T05:20:37.070068+0000","last_fullsized":"2023-10-06T05:20:37.070068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.095031+0000","last_change":"2023-10-06T05:20:37.089858+0000","last_active":"2023-10-06T05:20:37.095031+0000","last_peered":"2023-10-06T05:20:37.095031+0000","last_clean":"2023-10-06T05:20:37.095031+0000","last_became_active":"2023-10-06T05:20:37.089278+0000","last_became_peered":"2023-10-06T05:20:37.089278+0000","last_unstale":"2023-10-06T05:20:37.095031+0000","last_undegraded":"2023-10-06T05:20:37.095031+0000","last_fullsized":"2023-10-06T05:20:37.095031+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:38.106457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"56'2","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.269510+0000","last_change":"2023-10-06T05:20:36.263009+0000","last_active":"2023-10-06T05:20:37.269510+0000","last_peered":"2023-10-06T05:20:37.269510+0000","last_clean":"2023-10-06T05:20:37.269510+0000","last_became_active":"2023-10-06T05:20:36.262604+0000","last_became_peered":"2023-10-06T05:20:36.262604+0000","last_unstale":"2023-10-06T05:20:37.269510+0000","last_undegraded":"2023-10-06T05:20:37.269510+0000","last_fullsized":"2023-10-06T05:20:37.269510+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:22.505929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":44,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.086129+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:20:37.086129+0000","last_peered":"2023-10-06T05:20:37.086129+0000","last_clean":"2023-10-06T05:20:37.086129+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:20:37.086129+0000","last_undegraded":"2023-10-06T05:20:37.086129+0000","last_fullsized":"2023-10-06T05:20:37.086129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.089121+0000","last_change":"2023-10-06T05:20:37.086365+0000","last_active":"2023-10-06T05:20:37.089121+0000","last_peered":"2023-10-06T05:20:37.089121+0000","last_clean":"2023-10-06T05:20:37.089121+0000","last_became_active":"2023-10-06T05:20:37.085956+0000","last_became_peered":"2023-10-06T05:20:37.085956+0000","last_unstale":"2023-10-06T05:20:37.089121+0000","last_undegraded":"2023-10-06T05:20:37.089121+0000","last_fullsized":"2023-10-06T05:20:37.089121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:36.236719+0000","last_clean_scrub_stamp":"2023-10-06T05:20:36.236719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:38:26.019845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"56'1","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.268366+0000","last_change":"2023-10-06T05:20:36.267263+0000","last_active":"2023-10-06T05:20:37.268366+0000","last_peered":"2023-10-06T05:20:37.268366+0000","last_clean":"2023-10-06T05:20:37.268366+0000","last_became_active":"2023-10-06T05:20:36.267001+0000","last_became_peered":"2023-10-06T05:20:36.267001+0000","last_unstale":"2023-10-06T05:20:37.268366+0000","last_undegraded":"2023-10-06T05:20:37.268366+0000","last_fullsized":"2023-10-06T05:20:37.268366+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:20:35.230820+0000","last_clean_scrub_stamp":"2023-10-06T05:20:35.230820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:02.105855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":44,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T05:20:37.085960+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:20:37.085960+0000","last_peered":"2023-10-06T05:20:37.085960+0000","last_clean":"2023-10-06T05:20:37.085960+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:20:37.085960+0000","last_undegraded":"2023-10-06T05:20:37.085960+0000","last_fullsized":"2023-10-06T05:20:37.085960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298552,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073288,"statfs":{"total":96636764160,"available":96331046912,"internally_reserved":0,"allocated":1294336,"data_stored":891201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48699999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93500000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73899999999999999,"15min":0.73799999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.82899999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.51100000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.754,"15min":0.76400000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.749,"15min":0.755},"min":{"1min":0.58399999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.64700000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.92600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.76100000000000001,"15min":0.78900000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.73199999999999998,"15min":0.749},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90600000000000003,"5min":0.999,"15min":0.999},"last":0.879},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.81100000000000005,"15min":0.84699999999999998},"min":{"1min":0.53900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91700000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.86799999999999999,"15min":0.89900000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83999999999999997,"15min":0.85899999999999999},"min":{"1min":0.51400000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.0309999999999999,"5min":1.103,"15min":1.103},"last":0.94199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297960,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073880,"statfs":{"total":96636764160,"available":96331653120,"internally_reserved":0,"allocated":688128,"data_stored":300119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.498,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.006,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72299999999999998,"15min":0.70899999999999996},"min":{"1min":0.59299999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91900000000000004,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.42199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97599999999999998,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.626},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.046,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.47899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.875,"5min":1.369,"15min":1.369},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.58799999999999997,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92100000000000004,"5min":1.891,"15min":1.891},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.746,"15min":0.755},"min":{"1min":0.58099999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.96999999999999997,"5min":1.347,"15min":1.347},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.99299999999999999,"5min":1,"15min":1},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.75800000000000001,"15min":0.79300000000000004},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80500000000000005,"15min":0.83099999999999996},"min":{"1min":0.54800000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0089999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.90800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68300000000000005,"15min":0.69899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90100000000000002,"5min":1.415,"15min":1.415},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.80000000000000004,"15min":0.83899999999999997},"min":{"1min":0.501,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.997,"5min":2,"15min":2},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79200000000000004,"15min":0.81999999999999995},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.504},"max":{"1min":0.97899999999999998,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.52600000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96099999999999997,"5min":1.46,"15min":1.46},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298564,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073276,"statfs":{"total":96636764160,"available":96331034624,"internally_reserved":0,"allocated":1306624,"data_stored":892725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.441,"15min":0.433},"min":{"1min":0.34399999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.58999999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69999999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.55500000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72599999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.48599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.38900000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.629}]},{"osd":4,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0700000000000001,"5min":1.121,"15min":1.121},"last":1.077},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80500000000000005,"15min":0.79000000000000004},"min":{"1min":0.64600000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0589999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":1.0860000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91900000000000004,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.59199999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":1.002}]},{"osd":6,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76400000000000001,"15min":0.748},"min":{"1min":0.60199999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.05,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81200000000000006,"15min":0.79700000000000004},"min":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0069999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.94399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.57799999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.98999999999999999,"5min":1.042,"15min":1.042},"last":1.05},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82799999999999996,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0369999999999999,"5min":1.103,"15min":1.103},"last":0.92500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297944,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073896,"statfs":{"total":96636764160,"available":96331669504,"internally_reserved":0,"allocated":671744,"data_stored":293444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.44400000000000001,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.504},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.499,"15min":0.495},"min":{"1min":0.35899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78900000000000003,"5min":1.042,"15min":1.042},"last":0.629}]},{"osd":2,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.42199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.52300000000000002,"15min":0.51200000000000001},"min":{"1min":0.38900000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.78700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.50700000000000001,"15min":0.48699999999999999},"min":{"1min":0.41099999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59099999999999997,"15min":0.57799999999999996},"min":{"1min":0.47899999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.76900000000000002,"15min":0.75600000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.79200000000000004,"15min":0.78800000000000003},"min":{"1min":0.67700000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.78400000000000003,"15min":0.77100000000000002},"min":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.79000000000000004,"15min":0.78300000000000003},"min":{"1min":0.63700000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.71799999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.81499999999999995,"15min":0.81000000000000005},"min":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.749},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.80800000000000005,"15min":0.79400000000000004},"min":{"1min":0.61099999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1799999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.97299999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.85199999999999998,"15min":0.84399999999999997},"min":{"1min":0.70999999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2090000000000001,"5min":1.264,"15min":1.264},"last":0.995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.67600000000000005,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1579999999999999,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.83399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297944,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073896,"statfs":{"total":96636764160,"available":96331669504,"internally_reserved":0,"allocated":671744,"data_stored":293444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.436,"15min":0.42999999999999999},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77400000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.39000000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66000000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42499999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.32800000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.504,"15min":0.504},"min":{"1min":0.41699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69799999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.42999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82799999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76100000000000001,"15min":0.76600000000000001},"min":{"1min":0.65100000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.95099999999999996,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0009999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78500000000000003,"15min":0.79800000000000004},"min":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96999999999999997,"5min":1.099,"15min":1.099},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0649999999999999,"5min":1.163,"15min":1.163},"last":0.624}]},{"osd":6,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.83699999999999997,"15min":0.85099999999999998},"min":{"1min":0.66700000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0960000000000001,"5min":1.21,"15min":1.21},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81699999999999995,"15min":0.82599999999999996},"min":{"1min":0.70099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.98499999999999999,"5min":1.042,"15min":1.042},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80700000000000005,"15min":0.81599999999999995},"min":{"1min":0.64700000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.85999999999999999,"15min":0.878},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0309999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.64800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297956,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073884,"statfs":{"total":96636764160,"available":96331657216,"internally_reserved":0,"allocated":684032,"data_stored":300085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65500000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.495,"15min":0.495},"min":{"1min":0.38400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.314}]},{"osd":1,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.46300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.504,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78500000000000003,"5min":1.014,"15min":1.014},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81999999999999995,"15min":0.81399999999999995},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.161,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.103,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.57999999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.147,"5min":1.204,"15min":1.204},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.83599999999999997,"15min":0.84699999999999998},"min":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.284,"5min":1.329,"15min":1.329},"last":0.66000000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.81399999999999995,"15min":0.82399999999999995},"min":{"1min":0.63900000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0840000000000001,"5min":1.484,"15min":1.484},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.66600000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.212,"5min":1.629,"15min":1.629},"last":0.52000000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85499999999999998,"15min":0.86599999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.236,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.63},{"interface":"front","average":{"1min":0.871,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.66000000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1259999999999999,"5min":1.325,"15min":1.325},"last":0.64300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297968,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073872,"statfs":{"total":96636764160,"available":96331644928,"internally_reserved":0,"allocated":696320,"data_stored":301621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76600000000000001,"15min":0.80600000000000005},"min":{"1min":0.53600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.91000000000000003,"5min":2.77,"15min":2.77},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72899999999999998,"15min":0.749},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93300000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.72999999999999998},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90300000000000002,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.52100000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86799999999999999,"5min":0.879,"15min":0.879},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.84099999999999997},"min":{"1min":0.54000000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94999999999999996,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.55500000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.89700000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.998,"15min":1.1220000000000001},"min":{"1min":0.54400000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.93700000000000006,"5min":7.742,"15min":7.742},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":1.079,"15min":1.079},"last":0.751},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74399999999999999,"15min":0.76100000000000001},"min":{"1min":0.48199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91100000000000003,"5min":1.181,"15min":1.181},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72199999999999998,"15min":0.749},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89500000000000002,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.80100000000000005,"15min":0.83299999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93000000000000005,"5min":1.171,"15min":1.171},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96199999999999997,"5min":1.228,"15min":1.228},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.83999999999999997,"15min":0.878},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.571,"15min":1.571},"last":0.82599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297960,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073880,"statfs":{"total":96636764160,"available":96331653120,"internally_reserved":0,"allocated":688128,"data_stored":300107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.91500000000000004,"15min":1.028},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0209999999999999,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.747},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.46400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.94199999999999995,"15min":1.05},"min":{"1min":0.49299999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0640000000000001,"5min":9.25,"15min":9.25},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.999,"15min":1.123},"min":{"1min":0.503,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.97499999999999998,"15min":1.095},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":0.97999999999999998,"5min":10.384,"15min":10.384},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.98999999999999999,"15min":1.1040000000000001},"min":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0660000000000001,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":1.028,"15min":1.157},"min":{"1min":0.54400000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.0740000000000001,"5min":10.205,"15min":10.205},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72499999999999998,"15min":0.72999999999999998},"min":{"1min":0.50600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.93700000000000006,"5min":1.292,"15min":1.292},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84099999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.78600000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85399999999999998,"5min":1.171,"15min":1.171},"last":0.625}]},{"osd":6,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90500000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.80400000000000005,"15min":0.82399999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92100000000000004,"5min":1.298,"15min":1.298},"last":0.64200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81699999999999995,"15min":0.83399999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0840000000000001,"5min":1.204,"15min":1.204},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.81399999999999995,"15min":0.82399999999999995},"min":{"1min":0.54300000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93999999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.71999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:20:42.163 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:20:42.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:20:42.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.394+0000 7fa4435ff700 1 Processor -- start 2023-10-06T05:20:42.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.395+0000 7fa4435ff700 1 -- start start 2023-10-06T05:20:42.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa4435ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa43c1573a0 0x7fa43c0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa4435ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa43c157770 0x7fa43c0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa4435ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa43c157be0 0x7fa43c0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa4435ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c158cf0 con 0x7fa43c1573a0 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa4435ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c1580e0 con 0x7fa43c157be0 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa4435ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c1583c0 con 0x7fa43c157770 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa44139b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa43c1573a0 0x7fa43c0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa44139b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa43c1573a0 0x7fa43c0acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45494/0 (socket says 172.21.15.31:45494) 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa44139b700 1 -- 172.21.15.31:0/512799150 learned_addr learned my addr 172.21.15.31:0/512799150 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa440b9a700 1 --2- 172.21.15.31:0/512799150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa43c157770 0x7fa43c0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.396+0000 7fa441b9c700 1 --2- 172.21.15.31:0/512799150 >> v2:172.21.15.62:3300/0 conn(0x7fa43c157be0 0x7fa43c0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.397+0000 7fa44139b700 1 -- 172.21.15.31:0/512799150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa43c157770 msgr2=0x7fa43c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.397+0000 7fa44139b700 1 --2- 172.21.15.31:0/512799150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa43c157770 0x7fa43c0b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.397+0000 7fa44139b700 1 -- 172.21.15.31:0/512799150 >> v2:172.21.15.62:3300/0 conn(0x7fa43c157be0 msgr2=0x7fa43c0b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.397+0000 7fa44139b700 1 --2- 172.21.15.31:0/512799150 >> v2:172.21.15.62:3300/0 conn(0x7fa43c157be0 0x7fa43c0b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.397+0000 7fa44139b700 1 -- 172.21.15.31:0/512799150 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c1586a0 con 0x7fa43c1573a0 2023-10-06T05:20:42.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.397+0000 7fa44139b700 1 --2- 172.21.15.31:0/512799150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 0x7fa43c0acb70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa43c06b3a0 tx=0x7fa438009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3b374bd46b4840 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.397+0000 7fa433fff700 1 -- 172.21.15.31:0/512799150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43800b640 con 0x7fa43c1573a0 2023-10-06T05:20:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.397+0000 7fa433fff700 1 -- 172.21.15.31:0/512799150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43800b7a0 con 0x7fa43c1573a0 2023-10-06T05:20:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.397+0000 7fa433fff700 1 -- 172.21.15.31:0/512799150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43800bb60 con 0x7fa43c1573a0 2023-10-06T05:20:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.398+0000 7fa4435ff700 1 -- 172.21.15.31:0/512799150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 msgr2=0x7fa43c0acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.398+0000 7fa4435ff700 1 --2- 172.21.15.31:0/512799150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 0x7fa43c0acb70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa43c06b3a0 tx=0x7fa438009250 comp rx=0 tx=0).stop 2023-10-06T05:20:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.398+0000 7fa4435ff700 1 -- 172.21.15.31:0/512799150 shutdown_connections 2023-10-06T05:20:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.398+0000 7fa4435ff700 1 --2- 172.21.15.31:0/512799150 >> v2:172.21.15.62:3300/0 conn(0x7fa43c157be0 0x7fa43c0b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.398+0000 7fa4435ff700 1 --2- 172.21.15.31:0/512799150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 0x7fa43c0acb70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.398+0000 7fa4435ff700 1 --2- 172.21.15.31:0/512799150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa43c157770 0x7fa43c0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.398+0000 7fa4435ff700 1 -- 172.21.15.31:0/512799150 >> 172.21.15.31:0/512799150 conn(0x7fa43c067560 msgr2=0x7fa43c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.398+0000 7fa4435ff700 1 -- 172.21.15.31:0/512799150 shutdown_connections 2023-10-06T05:20:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.398+0000 7fa4435ff700 1 -- 172.21.15.31:0/512799150 wait complete. 2023-10-06T05:20:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.400+0000 7fa4435ff700 1 Processor -- start 2023-10-06T05:20:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.400+0000 7fa4435ff700 1 -- start start 2023-10-06T05:20:42.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa4435ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 0x7fa43c16b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa4435ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa43c157770 0x7fa43c16a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa4435ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa43c157be0 0x7fa43c0cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa4435ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c16b070 con 0x7fa43c1573a0 2023-10-06T05:20:42.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa4435ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c15b580 con 0x7fa43c157770 2023-10-06T05:20:42.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa4435ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c16aab0 con 0x7fa43c157be0 2023-10-06T05:20:42.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa44139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 0x7fa43c16b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa44139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 0x7fa43c16b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45504/0 (socket says 172.21.15.31:45504) 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa44139b700 1 -- 172.21.15.31:0/957516859 learned_addr learned my addr 172.21.15.31:0/957516859 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa440b9a700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa43c157770 0x7fa43c16a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa441b9c700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa43c157be0 0x7fa43c0cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa44139b700 1 -- 172.21.15.31:0/957516859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa43c157be0 msgr2=0x7fa43c0cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa44139b700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa43c157be0 0x7fa43c0cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa44139b700 1 -- 172.21.15.31:0/957516859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa43c157770 msgr2=0x7fa43c16a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa44139b700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa43c157770 0x7fa43c16a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.401+0000 7fa44139b700 1 -- 172.21.15.31:0/957516859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380094f0 con 0x7fa43c1573a0 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.402+0000 7fa44139b700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 0x7fa43c16b250 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa43c064290 tx=0x7fa438017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.402+0000 7fa4317fa700 1 -- 172.21.15.31:0/957516859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43800b910 con 0x7fa43c1573a0 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.402+0000 7fa4435ff700 1 -- 172.21.15.31:0/957516859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c16ad90 con 0x7fa43c1573a0 2023-10-06T05:20:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.402+0000 7fa4317fa700 1 -- 172.21.15.31:0/957516859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43800ba70 con 0x7fa43c1573a0 2023-10-06T05:20:42.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.402+0000 7fa4317fa700 1 -- 172.21.15.31:0/957516859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438011630 con 0x7fa43c1573a0 2023-10-06T05:20:42.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.402+0000 7fa4435ff700 1 -- 172.21.15.31:0/957516859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c156130 con 0x7fa43c1573a0 2023-10-06T05:20:42.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.403+0000 7fa4435ff700 1 -- 172.21.15.31:0/957516859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c04c380 con 0x7fa43c1573a0 2023-10-06T05:20:42.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.404+0000 7fa4317fa700 1 -- 172.21.15.31:0/957516859 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fa4380119a0 con 0x7fa43c1573a0 2023-10-06T05:20:42.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.405+0000 7fa4317fa700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4200670e0 0x7fa420069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.405+0000 7fa4317fa700 1 -- 172.21.15.31:0/957516859 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa438085bd0 con 0x7fa43c1573a0 2023-10-06T05:20:42.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.405+0000 7fa440b9a700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4200670e0 0x7fa420069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.406+0000 7fa440b9a700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4200670e0 0x7fa420069590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009db0 tx=0x7fa42c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.409+0000 7fa4317fa700 1 -- 172.21.15.31:0/957516859 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa438052af0 con 0x7fa43c1573a0 2023-10-06T05:20:42.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.694+0000 7fa4435ff700 1 -- 172.21.15.31:0/957516859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa43c0c9110 con 0x7fa43c1573a0 2023-10-06T05:20:42.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.699+0000 7fa4317fa700 1 -- 172.21.15.31:0/957516859 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa438056110 con 0x7fa43c1573a0 2023-10-06T05:20:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 -- 172.21.15.31:0/957516859 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4200670e0 msgr2=0x7fa420069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4200670e0 0x7fa420069590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009db0 tx=0x7fa42c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 -- 172.21.15.31:0/957516859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 msgr2=0x7fa43c16b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 0x7fa43c16b250 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa43c064290 tx=0x7fa438017000 comp rx=0 tx=0).stop 2023-10-06T05:20:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 -- 172.21.15.31:0/957516859 shutdown_connections 2023-10-06T05:20:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa43c157be0 0x7fa43c0cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa4200670e0 0x7fa420069590 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa43c1573a0 0x7fa43c16b250 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 --2- 172.21.15.31:0/957516859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa43c157770 0x7fa43c16a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 -- 172.21.15.31:0/957516859 >> 172.21.15.31:0/957516859 conn(0x7fa43c067560 msgr2=0x7fa43c067930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 -- 172.21.15.31:0/957516859 shutdown_connections 2023-10-06T05:20:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.701+0000 7fa4435ff700 1 -- 172.21.15.31:0/957516859 wait complete. 2023-10-06T05:20:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:20:42.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:42.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.914+0000 7faf703ff700 1 Processor -- start 2023-10-06T05:20:42.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.914+0000 7faf703ff700 1 -- start start 2023-10-06T05:20:42.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf703ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 0x7faf681569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf703ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faf680b1cc0 0x7faf68156f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf703ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf68153b90 con 0x7faf6814cc80 2023-10-06T05:20:42.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf703ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf681549f0 con 0x7faf680b1cc0 2023-10-06T05:20:42.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf703ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf68153de0 con 0x7faf6814c8b0 2023-10-06T05:20:42.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf6d99a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faf6814cc80 0x7faf680af950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42064/0 (socket says 172.21.15.31:42064) 2023-10-06T05:20:42.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf6d99a700 1 -- 172.21.15.31:0/1777182767 learned_addr learned my addr 172.21.15.31:0/1777182767 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:42.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf6e19b700 1 --2- 172.21.15.31:0/1777182767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 0x7faf681569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf6e99c700 1 --2- 172.21.15.31:0/1777182767 >> v2:172.21.15.62:3300/0 conn(0x7faf680b1cc0 0x7faf68156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf6d199700 1 -- 172.21.15.31:0/1777182767 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029132581 0 0) 0x7faf68153b90 con 0x7faf6814cc80 2023-10-06T05:20:42.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.916+0000 7faf6d199700 1 -- 172.21.15.31:0/1777182767 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf54004680 con 0x7faf6814cc80 2023-10-06T05:20:42.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.917+0000 7faf6e99c700 1 -- 172.21.15.31:0/1777182767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 msgr2=0x7faf681569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:42.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.917+0000 7faf6e99c700 1 --2- 172.21.15.31:0/1777182767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 0x7faf681569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.917+0000 7faf6d199700 1 -- 172.21.15.31:0/1777182767 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1257839290 0 0) 0x7faf54004680 con 0x7faf6814cc80 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.917+0000 7faf6e99c700 1 -- 172.21.15.31:0/1777182767 >> v1:172.21.15.31:6789/0 conn(0x7faf6814cc80 legacy=0x7faf680af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.917+0000 7faf6e99c700 1 -- 172.21.15.31:0/1777182767 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf681540c0 con 0x7faf680b1cc0 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.917+0000 7faf6e99c700 1 --2- 172.21.15.31:0/1777182767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf680b1cc0 0x7faf68156f10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faf60009e60 tx=0x7faf60009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d365aeee2404e5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.917+0000 7faf6d199700 1 -- 172.21.15.31:0/1777182767 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000e070 con 0x7faf680b1cc0 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.917+0000 7faf6d199700 1 -- 172.21.15.31:0/1777182767 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6000b7a0 con 0x7faf680b1cc0 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.917+0000 7faf6d199700 1 -- 172.21.15.31:0/1777182767 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58001470 con 0x7faf680b1cc0 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.918+0000 7faf703ff700 1 -- 172.21.15.31:0/1777182767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf680b1cc0 msgr2=0x7faf68156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.918+0000 7faf703ff700 1 --2- 172.21.15.31:0/1777182767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf680b1cc0 0x7faf68156f10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faf60009e60 tx=0x7faf60009780 comp rx=0 tx=0).stop 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.918+0000 7faf703ff700 1 -- 172.21.15.31:0/1777182767 shutdown_connections 2023-10-06T05:20:42.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.918+0000 7faf703ff700 1 --2- 172.21.15.31:0/1777182767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf680b1cc0 0x7faf68156f10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.918+0000 7faf703ff700 1 --2- 172.21.15.31:0/1777182767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 0x7faf681569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.918+0000 7faf703ff700 1 -- 172.21.15.31:0/1777182767 >> 172.21.15.31:0/1777182767 conn(0x7faf680cc040 msgr2=0x7faf680cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:42.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.918+0000 7faf703ff700 1 -- 172.21.15.31:0/1777182767 shutdown_connections 2023-10-06T05:20:42.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.918+0000 7faf703ff700 1 -- 172.21.15.31:0/1777182767 wait complete. 2023-10-06T05:20:42.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.919+0000 7faf703ff700 1 Processor -- start 2023-10-06T05:20:42.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.919+0000 7faf703ff700 1 -- start start 2023-10-06T05:20:42.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.920+0000 7faf703ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf680b1cc0 0x7faf68153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.920+0000 7faf703ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 0x7faf6814a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.920+0000 7faf703ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6814cc80 0x7faf680a7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.920+0000 7faf703ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681538a0 con 0x7faf680b1cc0 2023-10-06T05:20:42.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.920+0000 7faf703ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf68151330 con 0x7faf6814cc80 2023-10-06T05:20:42.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.920+0000 7faf703ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf68151610 con 0x7faf6814c8b0 2023-10-06T05:20:42.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf680b1cc0 0x7faf68153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf680b1cc0 0x7faf68153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45520/0 (socket says 172.21.15.31:45520) 2023-10-06T05:20:42.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e19b700 1 -- 172.21.15.31:0/2129615936 learned_addr learned my addr 172.21.15.31:0/2129615936 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:42.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6d99a700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 0x7faf6814a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e19b700 1 -- 172.21.15.31:0/2129615936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 msgr2=0x7faf6814a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e99c700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6814cc80 0x7faf680a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e19b700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 0x7faf6814a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e19b700 1 -- 172.21.15.31:0/2129615936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6814cc80 msgr2=0x7faf680a7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:42.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e19b700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6814cc80 0x7faf680a7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:42.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e19b700 1 -- 172.21.15.31:0/2129615936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60009a20 con 0x7faf680b1cc0 2023-10-06T05:20:42.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e99c700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6814cc80 0x7faf680a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:42.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6d99a700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 0x7faf6814a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:42.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf6e19b700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf680b1cc0 0x7faf68153360 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7faf6400ec80 tx=0x7faf640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.921+0000 7faf5e7fc700 1 -- 172.21.15.31:0/2129615936 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400c9d0 con 0x7faf680b1cc0 2023-10-06T05:20:42.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.922+0000 7faf5e7fc700 1 -- 172.21.15.31:0/2129615936 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6400cb30 con 0x7faf680b1cc0 2023-10-06T05:20:42.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.922+0000 7faf5e7fc700 1 -- 172.21.15.31:0/2129615936 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64013470 con 0x7faf680b1cc0 2023-10-06T05:20:42.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.922+0000 7faf703ff700 1 -- 172.21.15.31:0/2129615936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf681518f0 con 0x7faf680b1cc0 2023-10-06T05:20:42.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.922+0000 7faf703ff700 1 -- 172.21.15.31:0/2129615936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6814a6e0 con 0x7faf680b1cc0 2023-10-06T05:20:42.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.923+0000 7faf703ff700 1 -- 172.21.15.31:0/2129615936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6804c380 con 0x7faf680b1cc0 2023-10-06T05:20:42.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.924+0000 7faf5e7fc700 1 -- 172.21.15.31:0/2129615936 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7faf64013820 con 0x7faf680b1cc0 2023-10-06T05:20:42.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.924+0000 7faf5e7fc700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf4c0670e0 0x7faf4c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:42.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.925+0000 7faf5e7fc700 1 -- 172.21.15.31:0/2129615936 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faf64084dd0 con 0x7faf680b1cc0 2023-10-06T05:20:42.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.925+0000 7faf6d99a700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf4c0670e0 0x7faf4c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:42.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.925+0000 7faf6d99a700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf4c0670e0 0x7faf4c069590 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf580011e0 tx=0x7faf58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:42.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:42.929+0000 7faf5e7fc700 1 -- 172.21.15.31:0/2129615936 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf64051bc0 con 0x7faf680b1cc0 2023-10-06T05:20:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.174+0000 7faf703ff700 1 -- 172.21.15.31:0/2129615936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf681543a0 con 0x7faf680b1cc0 2023-10-06T05:20:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.175+0000 7faf5e7fc700 1 -- 172.21.15.31:0/2129615936 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4595 (secure 0 0 0) 0x7faf640551e0 con 0x7faf680b1cc0 2023-10-06T05:20:43.176 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:43.177 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5743,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2430199885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2430199885},{"type":"v1","addr":"172.21.15.31:6837","nonce":2430199885}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5746,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3762519665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3762519665},{"type":"v1","addr":"172.21.15.62:6834","nonce":3762519665}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5844,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2776536460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2776536460},{"type":"v1","addr":"172.21.15.31:6836","nonce":2776536460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:36.742745+0000","modified":"2023-10-06T05:20:40.276711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5853":{"gid":5853,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/982526111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":982526111},{"type":"v1","addr":"172.21.15.62:6835","nonce":982526111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:20:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 -- 172.21.15.31:0/2129615936 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf4c0670e0 msgr2=0x7faf4c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf4c0670e0 0x7faf4c069590 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf580011e0 tx=0x7faf58009040 comp rx=0 tx=0).stop 2023-10-06T05:20:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 -- 172.21.15.31:0/2129615936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf680b1cc0 msgr2=0x7faf68153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf680b1cc0 0x7faf68153360 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7faf6400ec80 tx=0x7faf640091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 -- 172.21.15.31:0/2129615936 shutdown_connections 2023-10-06T05:20:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6814cc80 0x7faf680a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf4c0670e0 0x7faf4c069590 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf680b1cc0 0x7faf68153360 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 --2- 172.21.15.31:0/2129615936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6814c8b0 0x7faf6814a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 -- 172.21.15.31:0/2129615936 >> 172.21.15.31:0/2129615936 conn(0x7faf680cc040 msgr2=0x7faf6806dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 -- 172.21.15.31:0/2129615936 shutdown_connections 2023-10-06T05:20:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.177+0000 7faf703ff700 1 -- 172.21.15.31:0/2129615936 wait complete. 2023-10-06T05:20:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 35 2023-10-06T05:20:43.234 INFO:tasks.ceph.mds.b.smithi062.stderr:2023-10-06T05:20:43.232+0000 7fea461f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:20:43.599 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:20:36.742745+0000', 'modified': '2023-10-06T05:20:40.276711+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5853}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5853': {'gid': 5853, 'name': 'b', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/982526111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 982526111}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 982526111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:20:43.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:20:43.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:20:43.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.797+0000 7f75acfff700 1 Processor -- start 2023-10-06T05:20:43.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.797+0000 7f75acfff700 1 -- start start 2023-10-06T05:20:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.798+0000 7f75acfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f75a81595a0 0x7f75a8159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.798+0000 7f75acfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75a8068d80 0x7f75a80c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.798+0000 7f75acfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a80c95b0 con 0x7f75a80cc0d0 2023-10-06T05:20:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.798+0000 7f75acfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a80c9890 con 0x7f75a8068d80 2023-10-06T05:20:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.798+0000 7f75acfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a80c9b70 con 0x7f75a81595a0 2023-10-06T05:20:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.799+0000 7f75a6d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75a8068d80 0x7f75a80c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.799+0000 7f75a6d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75a8068d80 0x7f75a80c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45530/0 (socket says 172.21.15.31:45530) 2023-10-06T05:20:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.799+0000 7f75a6d9d700 1 -- 172.21.15.31:0/2230683541 learned_addr learned my addr 172.21.15.31:0/2230683541 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.799+0000 7f75a659c700 1 --2- 172.21.15.31:0/2230683541 >> v2:172.21.15.62:3300/0 conn(0x7f75a81595a0 0x7f75a8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.799+0000 7f75a6d9d700 1 -- 172.21.15.31:0/2230683541 >> v2:172.21.15.62:3300/0 conn(0x7f75a81595a0 msgr2=0x7f75a8159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.799+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/2230683541 >> v2:172.21.15.62:3300/0 conn(0x7f75a81595a0 0x7f75a8159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.799+0000 7f75a6d9d700 1 -- 172.21.15.31:0/2230683541 >> v1:172.21.15.62:6789/0 conn(0x7f75a80cc0d0 legacy=0x7f75a8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.799+0000 7f75a6d9d700 1 -- 172.21.15.31:0/2230683541 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80ca130 con 0x7f75a8068d80 2023-10-06T05:20:43.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.799+0000 7f75a659c700 1 --2- 172.21.15.31:0/2230683541 >> v2:172.21.15.62:3300/0 conn(0x7f75a81595a0 0x7f75a8159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:43.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/2230683541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a8068d80 0x7f75a80c8930 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009f20 tx=0x7f75a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8324b0ddaf1eca97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75a559a700 1 -- 172.21.15.31:0/2230683541 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a000e070 con 0x7f75a8068d80 2023-10-06T05:20:43.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75a559a700 1 -- 172.21.15.31:0/2230683541 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a000ba40 con 0x7f75a8068d80 2023-10-06T05:20:43.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75a559a700 1 -- 172.21.15.31:0/2230683541 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a00092e0 con 0x7f75a8068d80 2023-10-06T05:20:43.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75acfff700 1 -- 172.21.15.31:0/2230683541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a8068d80 msgr2=0x7f75a80c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75acfff700 1 --2- 172.21.15.31:0/2230683541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a8068d80 0x7f75a80c8930 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009f20 tx=0x7f75a0009840 comp rx=0 tx=0).stop 2023-10-06T05:20:43.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75acfff700 1 -- 172.21.15.31:0/2230683541 shutdown_connections 2023-10-06T05:20:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75acfff700 1 --2- 172.21.15.31:0/2230683541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a8068d80 0x7f75a80c8930 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75acfff700 1 --2- 172.21.15.31:0/2230683541 >> v2:172.21.15.62:3300/0 conn(0x7f75a81595a0 0x7f75a8159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75acfff700 1 -- 172.21.15.31:0/2230683541 >> 172.21.15.31:0/2230683541 conn(0x7f75a806b620 msgr2=0x7f75a806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.800+0000 7f75acfff700 1 -- 172.21.15.31:0/2230683541 shutdown_connections 2023-10-06T05:20:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.801+0000 7f75acfff700 1 -- 172.21.15.31:0/2230683541 wait complete. 2023-10-06T05:20:43.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.802+0000 7f75acfff700 1 Processor -- start 2023-10-06T05:20:43.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.802+0000 7f75acfff700 1 -- start start 2023-10-06T05:20:43.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75acfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a8068d80 0x7f75a8156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75acfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a80cc0d0 0x7f75a8152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75acfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a81595a0 0x7f75a80c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75acfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8156f40 con 0x7f75a81595a0 2023-10-06T05:20:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75acfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80c53a0 con 0x7f75a8068d80 2023-10-06T05:20:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75acfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80c5680 con 0x7f75a80cc0d0 2023-10-06T05:20:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75a6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a81595a0 0x7f75a80c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75a6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a81595a0 0x7f75a80c6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45546/0 (socket says 172.21.15.31:45546) 2023-10-06T05:20:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75a6d9d700 1 -- 172.21.15.31:0/3104502541 learned_addr learned my addr 172.21.15.31:0/3104502541 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75a659c700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a8068d80 0x7f75a8156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.803+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a80cc0d0 0x7f75a8152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75a6d9d700 1 -- 172.21.15.31:0/3104502541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a80cc0d0 msgr2=0x7f75a8152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a80cc0d0 0x7f75a8152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75a6d9d700 1 -- 172.21.15.31:0/3104502541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a8068d80 msgr2=0x7f75a8156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a8068d80 0x7f75a8156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75a6d9d700 1 -- 172.21.15.31:0/3104502541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a0009ae0 con 0x7f75a81595a0 2023-10-06T05:20:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a81595a0 0x7f75a80c6440 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75a0012000 tx=0x7f75a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75967fc700 1 -- 172.21.15.31:0/3104502541 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0008e40 con 0x7f75a81595a0 2023-10-06T05:20:43.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75967fc700 1 -- 172.21.15.31:0/3104502541 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a000be50 con 0x7f75a81595a0 2023-10-06T05:20:43.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75967fc700 1 -- 172.21.15.31:0/3104502541 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0025690 con 0x7f75a81595a0 2023-10-06T05:20:43.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75acfff700 1 -- 172.21.15.31:0/3104502541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a80c5960 con 0x7f75a81595a0 2023-10-06T05:20:43.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.804+0000 7f75acfff700 1 -- 172.21.15.31:0/3104502541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a80c5f20 con 0x7f75a81595a0 2023-10-06T05:20:43.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.807+0000 7f75967fc700 1 -- 172.21.15.31:0/3104502541 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f75a0003040 con 0x7f75a81595a0 2023-10-06T05:20:43.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.808+0000 7f75967fc700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f75880670e0 0x7f7588069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.808+0000 7f75967fc700 1 -- 172.21.15.31:0/3104502541 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f75a00861d0 con 0x7f75a81595a0 2023-10-06T05:20:43.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.808+0000 7f75a659c700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f75880670e0 0x7f7588069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.809+0000 7f75a659c700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f75880670e0 0x7f7588069590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7598006f00 tx=0x7f7598008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.810+0000 7f75bb7e7b80 1 Processor -- start 2023-10-06T05:20:43.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.810+0000 7f75bb7e7b80 1 -- start start 2023-10-06T05:20:43.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d33482450 0x563d33482820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33467f40 0x563d3348af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d3348b4d0 0x563d3348b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d33355310 con 0x563d33467f40 2023-10-06T05:20:43.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d33392130 con 0x563d3348b4d0 2023-10-06T05:20:43.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d3338faf0 con 0x563d33482450 2023-10-06T05:20:43.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33467f40 0x563d3348af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33467f40 0x563d3348af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45562/0 (socket says 172.21.15.31:45562) 2023-10-06T05:20:43.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75a5d9b700 1 -- 172.21.15.31:0/2074069969 learned_addr learned my addr 172.21.15.31:0/2074069969 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:43.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/2074069969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d3348b4d0 0x563d3348b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75a659c700 1 --2- 172.21.15.31:0/2074069969 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d33482450 0x563d33482820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75a5d9b700 1 -- 172.21.15.31:0/2074069969 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d33482450 msgr2=0x563d33482820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/2074069969 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d33482450 0x563d33482820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75a5d9b700 1 -- 172.21.15.31:0/2074069969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d3348b4d0 msgr2=0x563d3348b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.811+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/2074069969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d3348b4d0 0x563d3348b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.812+0000 7f75a5d9b700 1 -- 172.21.15.31:0/2074069969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d33191790 con 0x563d33467f40 2023-10-06T05:20:43.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.812+0000 7f75a659c700 1 --2- 172.21.15.31:0/2074069969 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d33482450 0x563d33482820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:43.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.812+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/2074069969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33467f40 0x563d3348af90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f759c0036d0 tx=0x7f759c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e860c58b3112fbad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.812+0000 7f75877fe700 1 -- 172.21.15.31:0/2074069969 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c010520 con 0x563d33467f40 2023-10-06T05:20:43.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.812+0000 7f75877fe700 1 -- 172.21.15.31:0/2074069969 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c010680 con 0x563d33467f40 2023-10-06T05:20:43.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.812+0000 7f75877fe700 1 -- 172.21.15.31:0/2074069969 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c010a20 con 0x563d33467f40 2023-10-06T05:20:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.813+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/2074069969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33467f40 msgr2=0x563d3348af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.813+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/2074069969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33467f40 0x563d3348af90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f759c0036d0 tx=0x7f759c009040 comp rx=0 tx=0).stop 2023-10-06T05:20:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.813+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/2074069969 shutdown_connections 2023-10-06T05:20:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.813+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/2074069969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d3348b4d0 0x563d3348b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.813+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/2074069969 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d33482450 0x563d33482820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.813+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/2074069969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33467f40 0x563d3348af90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.813+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/2074069969 >> 172.21.15.31:0/2074069969 conn(0x563d33485880 msgr2=0x563d33485c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.813+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/2074069969 shutdown_connections 2023-10-06T05:20:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.813+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/2074069969 wait complete. 2023-10-06T05:20:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.814+0000 7f75bb7e7b80 1 -- start start 2023-10-06T05:20:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.814+0000 7f75bb7e7b80 1 Processor -- start 2023-10-06T05:20:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d3345f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d3345b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3338b8e0 con 0x563d33482450 2023-10-06T05:20:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d33337980 con 0x563d33467f40 2023-10-06T05:20:43.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d33337620 con 0x563d3348b4d0 2023-10-06T05:20:43.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75bb7e7b80 10 client.0 _ll_register_callbacks cb 0x563d3319aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:20:43.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45566/0 (socket says 172.21.15.31:45566) 2023-10-06T05:20:43.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75a5d9b700 1 -- 172.21.15.31:0/4063147129 learned_addr learned my addr 172.21.15.31:0/4063147129 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:43.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75a659c700 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d3345f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.815+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d3345b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f75a5d9b700 1 -- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 msgr2=0x563d3345b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d3345b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f75a5d9b700 1 -- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 msgr2=0x563d3345f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d3345f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f75a5d9b700 1 -- 172.21.15.31:0/4063147129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c00f040 con 0x563d33482450 2023-10-06T05:20:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345b420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f759c013000 tx=0x7f759c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7769eedfbaae6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f7584ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:20:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f7584ff9700 1 -- 172.21.15.31:0/4063147129 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c0107f0 con 0x563d33482450 2023-10-06T05:20:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f7584ff9700 1 -- 172.21.15.31:0/4063147129 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f759c010950 con 0x563d33482450 2023-10-06T05:20:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/4063147129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d3342fd60 con 0x563d33482450 2023-10-06T05:20:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f7584ff9700 1 -- 172.21.15.31:0/4063147129 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f759c021e50 con 0x563d33482450 2023-10-06T05:20:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f75bb7e7b80 10 client.5994 fetch_fsmap learned FSMap version 35 2023-10-06T05:20:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.816+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/4063147129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x563d3327e0a0 con 0x563d33482450 2023-10-06T05:20:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.817+0000 7f7584ff9700 1 -- 172.21.15.31:0/4063147129 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563d3342fd60 con 0x563d33482450 2023-10-06T05:20:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.817+0000 7f7584ff9700 1 -- 172.21.15.31:0/4063147129 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 35) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f759c006820 con 0x563d33482450 2023-10-06T05:20:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.817+0000 7f75bb7e7b80 10 client.5994 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T05:20:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.817+0000 7f75bb7e7b80 10 client.5994 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:20:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.817+0000 7f75bb7e7b80 20 client.5994 populate_metadata read hostname 'smithi031' 2023-10-06T05:20:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.817+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:6833/982526111,v1:172.21.15.62:6835/982526111] conn(0x563d33461f70 0x563d33462340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.817+0000 7f75bb7e7b80 4 client.5994 mds_command: new command op to 5853 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:20:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.817+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/4063147129 --> [v2:172.21.15.62:6833/982526111,v1:172.21.15.62:6835/982526111] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d3340cb30 con 0x563d33461f70 2023-10-06T05:20:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.817+0000 7f75a659c700 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:6833/982526111,v1:172.21.15.62:6835/982526111] conn(0x563d33461f70 0x563d33462340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.818+0000 7f75a659c700 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:6833/982526111,v1:172.21.15.62:6835/982526111] conn(0x563d33461f70 0x563d33462340 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b26049fcd5b49e4b server_cookie=93477bd0a707de62 in_seq=0 out_seq=0 2023-10-06T05:20:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.818+0000 7f7584ff9700 10 client.5994 ms_handle_connect on v2:172.21.15.62:6833/982526111 2023-10-06T05:20:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.819+0000 7f7584ff9700 1 -- 172.21.15.31:0/4063147129 <== mds.0 v2:172.21.15.62:6833/982526111 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563d3340cb30 con 0x563d33461f70 2023-10-06T05:20:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.819+0000 7f7584ff9700 10 client.5994 handle_command_reply: tid=0 2023-10-06T05:20:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.819+0000 7f75bb7e7b80 1 client.5994 shutdown 2023-10-06T05:20:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/4063147129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 msgr2=0x563d3345b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345b420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f759c013000 tx=0x7f759c01a000 comp rx=0 tx=0).stop 2023-10-06T05:20:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/4063147129 shutdown_connections 2023-10-06T05:20:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d3345b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:6833/982526111,v1:172.21.15.62:6835/982526111] conn(0x563d33461f70 0x563d33462340 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d3345f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/4063147129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345b420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f7584ff9700 0 client.5994 ms_handle_reset on v2:172.21.15.62:6833/982526111 2023-10-06T05:20:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/4063147129 >> 172.21.15.31:0/4063147129 conn(0x563d33485880 msgr2=0x563d33437c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/4063147129 shutdown_connections 2023-10-06T05:20:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/4063147129 wait complete. 2023-10-06T05:20:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.820+0000 7f75bb7e7b80 20 client.5994 trim_cache size 0 max 16384 2023-10-06T05:20:43.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.837+0000 7f75bb7e7b80 1 Processor -- start 2023-10-06T05:20:43.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.838+0000 7f75bb7e7b80 1 -- start start 2023-10-06T05:20:43.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.838+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d33481c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.838+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.838+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d33431a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.838+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d33437a90 con 0x563d33482450 2023-10-06T05:20:43.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.838+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d332231e0 con 0x563d33467f40 2023-10-06T05:20:43.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.838+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d33438db0 con 0x563d3348b4d0 2023-10-06T05:20:43.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45568/0 (socket says 172.21.15.31:45568) 2023-10-06T05:20:43.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a5d9b700 1 -- 172.21.15.31:0/573492540 learned_addr learned my addr 172.21.15.31:0/573492540 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:43.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/573492540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d33431a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a659c700 1 --2- 172.21.15.31:0/573492540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d33481c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a5d9b700 1 -- 172.21.15.31:0/573492540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 msgr2=0x563d33431a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/573492540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d33431a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a5d9b700 1 -- 172.21.15.31:0/573492540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 msgr2=0x563d33481c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/573492540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d33481c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a5d9b700 1 -- 172.21.15.31:0/573492540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d3335a3c0 con 0x563d33482450 2023-10-06T05:20:43.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/573492540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d33431a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:43.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/573492540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345d170 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f759c0033e0 tx=0x7f759c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9ec3b9c40e89d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.839+0000 7f7587fff700 1 -- 172.21.15.31:0/573492540 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c006820 con 0x563d33482450 2023-10-06T05:20:43.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f7587fff700 1 -- 172.21.15.31:0/573492540 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c010310 con 0x563d33482450 2023-10-06T05:20:43.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f7587fff700 1 -- 172.21.15.31:0/573492540 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c0069a0 con 0x563d33482450 2023-10-06T05:20:43.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/573492540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 msgr2=0x563d3345d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/573492540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345d170 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f759c0033e0 tx=0x7f759c020000 comp rx=0 tx=0).stop 2023-10-06T05:20:43.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/573492540 shutdown_connections 2023-10-06T05:20:43.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/573492540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d33431a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/573492540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d33481c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/573492540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d3345d170 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/573492540 >> 172.21.15.31:0/573492540 conn(0x563d33485880 msgr2=0x563d3345c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/573492540 shutdown_connections 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.840+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/573492540 wait complete. 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.841+0000 7f75bb7e7b80 1 -- start start 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.841+0000 7f75bb7e7b80 1 Processor -- start 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.842+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d3346aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.842+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d334e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.842+0000 7f75bb7e7b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d33483450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.842+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d3346b420 con 0x563d33482450 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.842+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d334e6a80 con 0x563d33467f40 2023-10-06T05:20:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.842+0000 7f75bb7e7b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d3343d3c0 con 0x563d3348b4d0 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.842+0000 7f75bb7e7b80 10 client.0 _ll_register_callbacks cb 0x563d3319aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.842+0000 7f75a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d334e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.842+0000 7f75a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d334e6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45572/0 (socket says 172.21.15.31:45572) 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.842+0000 7f75a5d9b700 1 -- 172.21.15.31:0/614854348 learned_addr learned my addr 172.21.15.31:0/614854348 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75a5d9b700 1 -- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 msgr2=0x563d33483450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d33483450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75a659c700 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d3346aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d33483450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75a5d9b700 1 -- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 msgr2=0x563d3346aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d3346aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75a5d9b700 1 -- 172.21.15.31:0/614854348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d3327e0a0 con 0x563d33482450 2023-10-06T05:20:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75a659c700 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d3346aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75a6d9d700 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d33483450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75a5d9b700 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d334e6540 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f759c0034e0 tx=0x7f759c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6bd784fa76c2e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f7584ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f7584ff9700 1 -- 172.21.15.31:0/614854348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c010e10 con 0x563d33482450 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/614854348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f759c010680 con 0x563d33482450 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.843+0000 7f7584ff9700 1 -- 172.21.15.31:0/614854348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f759c006820 con 0x563d33482450 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f7584ff9700 1 -- 172.21.15.31:0/614854348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f759c007150 con 0x563d33482450 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f75bb7e7b80 10 client.6000 fetch_fsmap learned FSMap version 35 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/614854348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x563d331f21d0 con 0x563d33482450 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f7584ff9700 1 -- 172.21.15.31:0/614854348 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c010680 con 0x563d33482450 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f7584ff9700 1 -- 172.21.15.31:0/614854348 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 35) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f759c014480 con 0x563d33482450 2023-10-06T05:20:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f75bb7e7b80 10 client.6000 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T05:20:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f75bb7e7b80 10 client.6000 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:20:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f75bb7e7b80 20 client.6000 populate_metadata read hostname 'smithi031' 2023-10-06T05:20:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:6833/982526111,v1:172.21.15.62:6835/982526111] conn(0x563d33483a50 0x563d3343c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f75bb7e7b80 4 client.6000 mds_command: new command op to 5853 tid=0[{"prefix": "status"}] 2023-10-06T05:20:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/614854348 --> [v2:172.21.15.62:6833/982526111,v1:172.21.15.62:6835/982526111] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563d3332e8d0 con 0x563d33483a50 2023-10-06T05:20:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.844+0000 7f75a659c700 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:6833/982526111,v1:172.21.15.62:6835/982526111] conn(0x563d33483a50 0x563d3343c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.845+0000 7f75a659c700 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:6833/982526111,v1:172.21.15.62:6835/982526111] conn(0x563d33483a50 0x563d3343c430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d694e30037ea10 server_cookie=d29f5aa1ed74815d in_seq=0 out_seq=0 2023-10-06T05:20:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.845+0000 7f7584ff9700 10 client.6000 ms_handle_connect on v2:172.21.15.62:6833/982526111 2023-10-06T05:20:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.845+0000 7f7584ff9700 1 -- 172.21.15.31:0/614854348 <== mds.0 v2:172.21.15.62:6833/982526111 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x563d3332e8d0 con 0x563d33483a50 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.845+0000 7f7584ff9700 10 client.6000 handle_command_reply: tid=0 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.845+0000 7f75bb7e7b80 1 client.6000 shutdown 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/614854348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 msgr2=0x563d334e6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d334e6540 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f759c0034e0 tx=0x7f759c016040 comp rx=0 tx=0).stop 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/614854348 shutdown_connections 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563d3348b4d0 0x563d33483450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:6833/982526111,v1:172.21.15.62:6835/982526111] conn(0x563d33483a50 0x563d3343c430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d33467f40 0x563d3346aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 1 --2- 172.21.15.31:0/614854348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563d33482450 0x563d334e6540 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f7584ff9700 0 client.6000 ms_handle_reset on v2:172.21.15.62:6833/982526111 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/614854348 >> 172.21.15.31:0/614854348 conn(0x563d33485880 msgr2=0x563d3345d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/614854348 shutdown_connections 2023-10-06T05:20:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 1 -- 172.21.15.31:0/614854348 wait complete. 2023-10-06T05:20:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.846+0000 7f75bb7e7b80 20 client.6000 trim_cache size 0 max 16384 2023-10-06T05:20:43.853 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:20:43.853 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:20:43.853 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:20:43.853 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5853, 2023-10-06T05:20:43.853 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:20:43.853 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:20:43.854 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:20:43.854 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.591061431, 2023-10-06T05:20:43.854 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 35, 2023-10-06T05:20:43.854 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 57, 2023-10-06T05:20:43.854 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 57, 2023-10-06T05:20:43.854 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.272174297999999 2023-10-06T05:20:43.854 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:20:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 -- 172.21.15.31:0/3104502541 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f75880670e0 msgr2=0x7f7588069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f75880670e0 0x7f7588069590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7598006f00 tx=0x7f7598008040 comp rx=0 tx=0).stop 2023-10-06T05:20:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 -- 172.21.15.31:0/3104502541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a81595a0 msgr2=0x7f75a80c6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a81595a0 0x7f75a80c6440 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75a0012000 tx=0x7f75a0018000 comp rx=0 tx=0).stop 2023-10-06T05:20:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 -- 172.21.15.31:0/3104502541 shutdown_connections 2023-10-06T05:20:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f75880670e0 0x7f7588069590 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75a81595a0 0x7f75a80c6440 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a8068d80 0x7f75a8156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 --2- 172.21.15.31:0/3104502541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a80cc0d0 0x7f75a8152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 -- 172.21.15.31:0/3104502541 >> 172.21.15.31:0/3104502541 conn(0x7f75a806b620 msgr2=0x7f75a80ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 -- 172.21.15.31:0/3104502541 shutdown_connections 2023-10-06T05:20:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:43.847+0000 7f75acfff700 1 -- 172.21.15.31:0/3104502541 wait complete. 2023-10-06T05:20:43.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:20:44.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.060+0000 7f8688bff700 1 Processor -- start 2023-10-06T05:20:44.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.060+0000 7f8688bff700 1 -- start start 2023-10-06T05:20:44.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.061+0000 7f8688bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86840cd510 0x7f86840b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.061+0000 7f8688bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f868415d2d0 0x7f86840aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.061+0000 7f8688bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840c3f40 con 0x7f8684159500 2023-10-06T05:20:44.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.061+0000 7f8688bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840b2860 con 0x7f868415d2d0 2023-10-06T05:20:44.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.061+0000 7f8688bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868415d7d0 con 0x7f86840cd510 2023-10-06T05:20:44.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.061+0000 7f8682d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f868415d2d0 0x7f86840aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.061+0000 7f868259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8684159500 0x7f86840cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42068/0 (socket says 172.21.15.31:42068) 2023-10-06T05:20:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.061+0000 7f8682d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f868415d2d0 0x7f86840aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45578/0 (socket says 172.21.15.31:45578) 2023-10-06T05:20:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.061+0000 7f8682d9d700 1 -- 172.21.15.31:0/223123499 learned_addr learned my addr 172.21.15.31:0/223123499 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.062+0000 7f8681d9b700 1 --2- 172.21.15.31:0/223123499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86840cd510 0x7f86840b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.062+0000 7f8682d9d700 1 -- 172.21.15.31:0/223123499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86840cd510 msgr2=0x7f86840b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.062+0000 7f8682d9d700 1 --2- 172.21.15.31:0/223123499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86840cd510 0x7f86840b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.062+0000 7f8682d9d700 1 -- 172.21.15.31:0/223123499 >> v1:172.21.15.31:6789/0 conn(0x7f8684159500 legacy=0x7f86840cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.062+0000 7f8682d9d700 1 -- 172.21.15.31:0/223123499 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840cb000 con 0x7f868415d2d0 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.062+0000 7f8682d9d700 1 --2- 172.21.15.31:0/223123499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868415d2d0 0x7f86840aedd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f867c009f20 tx=0x7f867c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b6dc7ea7b33a85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.062+0000 7f868159a700 1 -- 172.21.15.31:0/223123499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00e070 con 0x7f868415d2d0 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.062+0000 7f868159a700 1 -- 172.21.15.31:0/223123499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c00b9d0 con 0x7f868415d2d0 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.062+0000 7f868159a700 1 -- 172.21.15.31:0/223123499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c0092e0 con 0x7f868415d2d0 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.063+0000 7f8688bff700 1 -- 172.21.15.31:0/223123499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868415d2d0 msgr2=0x7f86840aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.063+0000 7f8688bff700 1 --2- 172.21.15.31:0/223123499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868415d2d0 0x7f86840aedd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f867c009f20 tx=0x7f867c009840 comp rx=0 tx=0).stop 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.063+0000 7f8688bff700 1 -- 172.21.15.31:0/223123499 shutdown_connections 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.063+0000 7f8688bff700 1 --2- 172.21.15.31:0/223123499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868415d2d0 0x7f86840aedd0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.063+0000 7f8688bff700 1 --2- 172.21.15.31:0/223123499 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86840cd510 0x7f86840b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.063+0000 7f8688bff700 1 -- 172.21.15.31:0/223123499 >> 172.21.15.31:0/223123499 conn(0x7f868406ec90 msgr2=0x7f868406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.063+0000 7f8688bff700 1 -- 172.21.15.31:0/223123499 shutdown_connections 2023-10-06T05:20:44.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.063+0000 7f8688bff700 1 -- 172.21.15.31:0/223123499 wait complete. 2023-10-06T05:20:44.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.065+0000 7f8688bff700 1 Processor -- start 2023-10-06T05:20:44.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.065+0000 7f8688bff700 1 -- start start 2023-10-06T05:20:44.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f8688bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86840cd510 0x7f8684158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f8688bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8684159500 0x7f86840c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f8688bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f868415d2d0 0x7f86840c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f8688bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86841589d0 con 0x7f86840cd510 2023-10-06T05:20:44.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f8688bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868416d6b0 con 0x7f8684159500 2023-10-06T05:20:44.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f8688bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840c7e00 con 0x7f868415d2d0 2023-10-06T05:20:44.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f868259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86840cd510 0x7f8684158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f868259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86840cd510 0x7f8684158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45586/0 (socket says 172.21.15.31:45586) 2023-10-06T05:20:44.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f868259c700 1 -- 172.21.15.31:0/1230833610 learned_addr learned my addr 172.21.15.31:0/1230833610 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:44.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f8682d9d700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f868415d2d0 0x7f86840c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.066+0000 7f8681d9b700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8684159500 0x7f86840c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f868259c700 1 -- 172.21.15.31:0/1230833610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f868415d2d0 msgr2=0x7f86840c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f868259c700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f868415d2d0 0x7f86840c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f868259c700 1 -- 172.21.15.31:0/1230833610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8684159500 msgr2=0x7f86840c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f868259c700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8684159500 0x7f86840c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f868259c700 1 -- 172.21.15.31:0/1230833610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c009ae0 con 0x7f86840cd510 2023-10-06T05:20:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f868259c700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86840cd510 0x7f8684158490 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f867400c7f0 tx=0x7f8674007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f86727fc700 1 -- 172.21.15.31:0/1230833610 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674005a70 con 0x7f86840cd510 2023-10-06T05:20:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f86727fc700 1 -- 172.21.15.31:0/1230833610 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674005bf0 con 0x7f86840cd510 2023-10-06T05:20:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f8688bff700 1 -- 172.21.15.31:0/1230833610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86840c99d0 con 0x7f86840cd510 2023-10-06T05:20:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f86727fc700 1 -- 172.21.15.31:0/1230833610 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86740070d0 con 0x7f86840cd510 2023-10-06T05:20:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.067+0000 7f8688bff700 1 -- 172.21.15.31:0/1230833610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86840c77b0 con 0x7f86840cd510 2023-10-06T05:20:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.068+0000 7f8688bff700 1 -- 172.21.15.31:0/1230833610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868404c380 con 0x7f86840cd510 2023-10-06T05:20:44.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.069+0000 7f86727fc700 1 -- 172.21.15.31:0/1230833610 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8674007480 con 0x7f86840cd510 2023-10-06T05:20:44.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.070+0000 7f86727fc700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f86640670e0 0x7f8664069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:44.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.070+0000 7f86727fc700 1 -- 172.21.15.31:0/1230833610 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8674083e60 con 0x7f86840cd510 2023-10-06T05:20:44.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.071+0000 7f8681d9b700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f86640670e0 0x7f8664069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:44.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.071+0000 7f8681d9b700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f86640670e0 0x7f8664069590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f86840b6390 tx=0x7f8678009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:44.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.074+0000 7f86727fc700 1 -- 172.21.15.31:0/1230833610 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8674050c50 con 0x7f86840cd510 2023-10-06T05:20:44.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.327+0000 7f8688bff700 1 -- 172.21.15.31:0/1230833610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f86840d0d70 con 0x7f86840cd510 2023-10-06T05:20:44.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.327+0000 7f86727fc700 1 -- 172.21.15.31:0/1230833610 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8674017020 con 0x7f86840cd510 2023-10-06T05:20:44.329 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:20:44.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.329+0000 7f8688bff700 1 -- 172.21.15.31:0/1230833610 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f86640670e0 msgr2=0x7f8664069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.329+0000 7f8688bff700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f86640670e0 0x7f8664069590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f86840b6390 tx=0x7f8678009120 comp rx=0 tx=0).stop 2023-10-06T05:20:44.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.329+0000 7f8688bff700 1 -- 172.21.15.31:0/1230833610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86840cd510 msgr2=0x7f8684158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:44.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.329+0000 7f8688bff700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86840cd510 0x7f8684158490 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f867400c7f0 tx=0x7f8674007f70 comp rx=0 tx=0).stop 2023-10-06T05:20:44.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.330+0000 7f8688bff700 1 -- 172.21.15.31:0/1230833610 shutdown_connections 2023-10-06T05:20:44.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.330+0000 7f8688bff700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f868415d2d0 0x7f86840c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.330+0000 7f8688bff700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f86640670e0 0x7f8664069590 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.330+0000 7f8688bff700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86840cd510 0x7f8684158490 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.330+0000 7f8688bff700 1 --2- 172.21.15.31:0/1230833610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8684159500 0x7f86840c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:44.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.330+0000 7f8688bff700 1 -- 172.21.15.31:0/1230833610 >> 172.21.15.31:0/1230833610 conn(0x7f868406ec90 msgr2=0x7f86840ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:44.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.330+0000 7f8688bff700 1 -- 172.21.15.31:0/1230833610 shutdown_connections 2023-10-06T05:20:44.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:44.330+0000 7f8688bff700 1 -- 172.21.15.31:0/1230833610 wait complete. 2023-10-06T05:20:44.349 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:20:44.349 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:20:44.349 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:20:44.349 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:20:44.349 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:44.349 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:20:44.349 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:20:44.349 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:20:44.349 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:20:44.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:44.350 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:20:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:20:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42070sec preferred_lft 42070sec 2023-10-06T05:20:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:20:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:44.400 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:20:44.400 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:20:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:20:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:20:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:44.401 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:20:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:20:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:20:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:44.401 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:20:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:20:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:20:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:44.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:44.402 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:20:44.470 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:20:44.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:44.473 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:20:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:20:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:20:44.523 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:20:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:20:44.524 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:20:44.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:44.524 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:20:44.525 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:20:44.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:20:44.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:20:44.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:20:44.525 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:20:44.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:44.698 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:20:44.746 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:20:44.746 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:20:44.746 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:20:44.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:44.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:20:44.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:20:44.819 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:44.865 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:20:44.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:20:44.866 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:44.911 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:20:44.911 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:20:44.911 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:20:45.052 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:20:45.052 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:20:45.082 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:20:45.082 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:20:45.082 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:20:45.082 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:20:45.082 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:20:45.086 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:20:45.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:45.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:45.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:20:45.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.354+0000 7f055ffff700 1 Processor -- start 2023-10-06T05:20:45.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.354+0000 7f055ffff700 1 -- start start 2023-10-06T05:20:45.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.356+0000 7f055ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05580ca770 0x7f05580c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.356+0000 7f055ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f055806b510 0x7f05581569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.356+0000 7f055ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0558156ef0 con 0x7f05580cab40 2023-10-06T05:20:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.356+0000 7f055ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05580c7a70 con 0x7f05580ca770 2023-10-06T05:20:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.356+0000 7f055ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05580c6df0 con 0x7f055806b510 2023-10-06T05:20:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.356+0000 7f055dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05580ca770 0x7f05580c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.356+0000 7f055dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05580ca770 0x7f05580c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45600/0 (socket says 172.21.15.31:45600) 2023-10-06T05:20:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.356+0000 7f055dd9b700 1 -- 172.21.15.31:0/1141678192 learned_addr learned my addr 172.21.15.31:0/1141678192 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.356+0000 7f055e59c700 1 --2- 172.21.15.31:0/1141678192 >> v2:172.21.15.62:3300/0 conn(0x7f055806b510 0x7f05581569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.357+0000 7f055dd9b700 1 -- 172.21.15.31:0/1141678192 >> v2:172.21.15.62:3300/0 conn(0x7f055806b510 msgr2=0x7f05581569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.357+0000 7f055dd9b700 1 --2- 172.21.15.31:0/1141678192 >> v2:172.21.15.62:3300/0 conn(0x7f055806b510 0x7f05581569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.357+0000 7f055dd9b700 1 -- 172.21.15.31:0/1141678192 >> v1:172.21.15.62:6789/0 conn(0x7f05580cab40 legacy=0x7f055806ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.357+0000 7f055dd9b700 1 -- 172.21.15.31:0/1141678192 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05580c70d0 con 0x7f05580ca770 2023-10-06T05:20:45.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.357+0000 7f055dd9b700 1 --2- 172.21.15.31:0/1141678192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 0x7f05580c66f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0558150590 tx=0x7f0554009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb93cb7d5a01428 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.357+0000 7f055cd99700 1 -- 172.21.15.31:0/1141678192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554005c10 con 0x7f05580ca770 2023-10-06T05:20:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.357+0000 7f055cd99700 1 -- 172.21.15.31:0/1141678192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0554005d70 con 0x7f05580ca770 2023-10-06T05:20:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.357+0000 7f055cd99700 1 -- 172.21.15.31:0/1141678192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055400b840 con 0x7f05580ca770 2023-10-06T05:20:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.358+0000 7f055ffff700 1 -- 172.21.15.31:0/1141678192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 msgr2=0x7f05580c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.358+0000 7f055ffff700 1 --2- 172.21.15.31:0/1141678192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 0x7f05580c66f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0558150590 tx=0x7f0554009220 comp rx=0 tx=0).stop 2023-10-06T05:20:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.358+0000 7f055ffff700 1 -- 172.21.15.31:0/1141678192 shutdown_connections 2023-10-06T05:20:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.358+0000 7f055ffff700 1 --2- 172.21.15.31:0/1141678192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 0x7f05580c66f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.358+0000 7f055ffff700 1 --2- 172.21.15.31:0/1141678192 >> v2:172.21.15.62:3300/0 conn(0x7f055806b510 0x7f05581569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.358+0000 7f055ffff700 1 -- 172.21.15.31:0/1141678192 >> 172.21.15.31:0/1141678192 conn(0x7f05580cc0b0 msgr2=0x7f05580cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.358+0000 7f055ffff700 1 -- 172.21.15.31:0/1141678192 shutdown_connections 2023-10-06T05:20:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.358+0000 7f055ffff700 1 -- 172.21.15.31:0/1141678192 wait complete. 2023-10-06T05:20:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.359+0000 7f055ffff700 1 Processor -- start 2023-10-06T05:20:45.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.359+0000 7f055ffff700 1 -- start start 2023-10-06T05:20:45.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.360+0000 7f055ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f055806b510 0x7f05580ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.360+0000 7f055ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 0x7f05580b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.360+0000 7f055ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05580cab40 0x7f055816cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.360+0000 7f055ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05580aeae0 con 0x7f05580ca770 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.360+0000 7f055ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05580b1060 con 0x7f055806b510 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.360+0000 7f055ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05580b1340 con 0x7f05580cab40 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055d59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 0x7f05580b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055d59a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 0x7f05580b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45608/0 (socket says 172.21.15.31:45608) 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055d59a700 1 -- 172.21.15.31:0/1712713696 learned_addr learned my addr 172.21.15.31:0/1712713696 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055dd9b700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f055806b510 0x7f05580ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055e59c700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05580cab40 0x7f055816cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055d59a700 1 -- 172.21.15.31:0/1712713696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05580cab40 msgr2=0x7f055816cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055d59a700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05580cab40 0x7f055816cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055d59a700 1 -- 172.21.15.31:0/1712713696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f055806b510 msgr2=0x7f05580ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055d59a700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f055806b510 0x7f05580ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055d59a700 1 -- 172.21.15.31:0/1712713696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0554009480 con 0x7f05580ca770 2023-10-06T05:20:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055dd9b700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f055806b510 0x7f05580ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055e59c700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05580cab40 0x7f055816cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f055d59a700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 0x7f05580b07e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f054800b5e0 tx=0x7f0548009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.361+0000 7f054dffb700 1 -- 172.21.15.31:0/1712713696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054800f510 con 0x7f05580ca770 2023-10-06T05:20:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.362+0000 7f054dffb700 1 -- 172.21.15.31:0/1712713696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054800f670 con 0x7f05580ca770 2023-10-06T05:20:45.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.362+0000 7f054dffb700 1 -- 172.21.15.31:0/1712713696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054800fa30 con 0x7f05580ca770 2023-10-06T05:20:45.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.362+0000 7f055ffff700 1 -- 172.21.15.31:0/1712713696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05580b1620 con 0x7f05580ca770 2023-10-06T05:20:45.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.362+0000 7f055ffff700 1 -- 172.21.15.31:0/1712713696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05580c4f60 con 0x7f05580ca770 2023-10-06T05:20:45.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.363+0000 7f055ffff700 1 -- 172.21.15.31:0/1712713696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055804c380 con 0x7f05580ca770 2023-10-06T05:20:45.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.364+0000 7f054dffb700 1 -- 172.21.15.31:0/1712713696 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f054801a400 con 0x7f05580ca770 2023-10-06T05:20:45.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.365+0000 7f054dffb700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f053c0670e0 0x7f053c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:45.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.365+0000 7f054dffb700 1 -- 172.21.15.31:0/1712713696 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0548084570 con 0x7f05580ca770 2023-10-06T05:20:45.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.365+0000 7f055dd9b700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f053c0670e0 0x7f053c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:45.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.366+0000 7f055dd9b700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f053c0670e0 0x7f053c069590 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0554003080 tx=0x7f0554015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:45.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.369+0000 7f054dffb700 1 -- 172.21.15.31:0/1712713696 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05480513e0 con 0x7f05580ca770 2023-10-06T05:20:45.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.615+0000 7f055ffff700 1 -- 172.21.15.31:0/1712713696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f05580cbe20 con 0x7f05580ca770 2023-10-06T05:20:45.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.616+0000 7f054dffb700 1 -- 172.21.15.31:0/1712713696 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0548054a00 con 0x7f05580ca770 2023-10-06T05:20:45.618 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:20:45.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 -- 172.21.15.31:0/1712713696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f053c0670e0 msgr2=0x7f053c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f053c0670e0 0x7f053c069590 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0554003080 tx=0x7f0554015000 comp rx=0 tx=0).stop 2023-10-06T05:20:45.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 -- 172.21.15.31:0/1712713696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 msgr2=0x7f05580b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:45.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 0x7f05580b07e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f054800b5e0 tx=0x7f0548009120 comp rx=0 tx=0).stop 2023-10-06T05:20:45.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 -- 172.21.15.31:0/1712713696 shutdown_connections 2023-10-06T05:20:45.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05580cab40 0x7f055816cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f055806b510 0x7f05580ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f053c0670e0 0x7f053c069590 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 --2- 172.21.15.31:0/1712713696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05580ca770 0x7f05580b07e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:45.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 -- 172.21.15.31:0/1712713696 >> 172.21.15.31:0/1712713696 conn(0x7f05580cc0b0 msgr2=0x7f055814f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:45.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.618+0000 7f055ffff700 1 -- 172.21.15.31:0/1712713696 shutdown_connections 2023-10-06T05:20:45.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:45.619+0000 7f055ffff700 1 -- 172.21.15.31:0/1712713696 wait complete. 2023-10-06T05:20:45.637 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:20:45.638 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:20:45.638 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:20:45.638 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:20:45.638 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:20:45.638 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:20:45.638 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:20:45.638 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:20:45.638 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:20:45.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:45.638 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:20:45.691 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:20:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:20:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:20:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:20:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42069sec preferred_lft 42069sec 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:45.693 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:20:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:20:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:20:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:45.694 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:20:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:20:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:20:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:20:45.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:45.695 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:20:45.767 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:20:45.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:45.771 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:20:45.820 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:20:45.821 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:20:45.821 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:20:45.821 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:20:45.821 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:20:45.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:45.822 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:20:45.822 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:20:45.822 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:20:45.822 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:20:45.822 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:20:45.822 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:20:45.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:45.993 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:20:46.039 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:20:46.040 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:20:46.040 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:20:46.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:46.040 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:20:46.112 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:20:46.113 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:20:46.158 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:20:46.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:20:46.158 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:20:46.244 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:20:46.245 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:20:46.245 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:20:46.386 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:20:46.386 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:20:46.412 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:20:46.412 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:20:46.412 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:20:46.412 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:20:46.412 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:20:46.416 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:20:46.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:46.416 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:20:46.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:20:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.694+0000 7fc8355ff700 1 Processor -- start 2023-10-06T05:20:46.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.695+0000 7fc8355ff700 1 -- start start 2023-10-06T05:20:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.696+0000 7fc8355ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 0x7fc8300cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.696+0000 7fc8355ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83015d210 con 0x7fc8300cb3e0 2023-10-06T05:20:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.696+0000 7fc8355ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8300cc810 con 0x7fc83015a990 2023-10-06T05:20:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.696+0000 7fc8355ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83015d4b0 con 0x7fc830142c20 2023-10-06T05:20:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.696+0000 7fc82f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc8300cb3e0 0x7fc830068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42084/0 (socket says 172.21.15.31:42084) 2023-10-06T05:20:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.696+0000 7fc82f7fe700 1 -- 172.21.15.31:0/3829542958 learned_addr learned my addr 172.21.15.31:0/3829542958 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.696+0000 7fc82effd700 1 --2- 172.21.15.31:0/3829542958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 0x7fc8300cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82dffb700 1 -- 172.21.15.31:0/3829542958 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177956340 0 0) 0x7fc83015d210 con 0x7fc8300cb3e0 2023-10-06T05:20:46.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82dffb700 1 -- 172.21.15.31:0/3829542958 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc818003570 con 0x7fc8300cb3e0 2023-10-06T05:20:46.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82dffb700 1 -- 172.21.15.31:0/3829542958 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266580012 0 0) 0x7fc8300cc810 con 0x7fc83015a990 2023-10-06T05:20:46.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82dffb700 1 -- 172.21.15.31:0/3829542958 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc83015d210 con 0x7fc83015a990 2023-10-06T05:20:46.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82effd700 1 -- 172.21.15.31:0/3829542958 >> v1:172.21.15.62:6789/0 conn(0x7fc83015a990 legacy=0x7fc8300cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82effd700 1 -- 172.21.15.31:0/3829542958 >> v1:172.21.15.31:6789/0 conn(0x7fc8300cb3e0 legacy=0x7fc830068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82effd700 1 -- 172.21.15.31:0/3829542958 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83015d790 con 0x7fc830142c20 2023-10-06T05:20:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82effd700 1 --2- 172.21.15.31:0/3829542958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 0x7fc8300cbea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc820009910 tx=0x7fc8200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23078fe8fed47b95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82dffb700 1 -- 172.21.15.31:0/3829542958 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820011630 con 0x7fc830142c20 2023-10-06T05:20:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82dffb700 1 -- 172.21.15.31:0/3829542958 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc820011790 con 0x7fc830142c20 2023-10-06T05:20:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.697+0000 7fc82dffb700 1 -- 172.21.15.31:0/3829542958 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820011b30 con 0x7fc830142c20 2023-10-06T05:20:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.698+0000 7fc8355ff700 1 -- 172.21.15.31:0/3829542958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 msgr2=0x7fc8300cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.698+0000 7fc8355ff700 1 --2- 172.21.15.31:0/3829542958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 0x7fc8300cbea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc820009910 tx=0x7fc8200091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.698+0000 7fc8355ff700 1 -- 172.21.15.31:0/3829542958 shutdown_connections 2023-10-06T05:20:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.698+0000 7fc8355ff700 1 --2- 172.21.15.31:0/3829542958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 0x7fc8300cbea0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.698+0000 7fc8355ff700 1 -- 172.21.15.31:0/3829542958 >> 172.21.15.31:0/3829542958 conn(0x7fc8300ce2d0 msgr2=0x7fc8300ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.698+0000 7fc8355ff700 1 -- 172.21.15.31:0/3829542958 shutdown_connections 2023-10-06T05:20:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.698+0000 7fc8355ff700 1 -- 172.21.15.31:0/3829542958 wait complete. 2023-10-06T05:20:46.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.699+0000 7fc8355ff700 1 Processor -- start 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.700+0000 7fc8355ff700 1 -- start start 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.700+0000 7fc8355ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8300cb3e0 0x7fc8300acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.700+0000 7fc8355ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 0x7fc8300b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.700+0000 7fc8355ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc83015a990 0x7fc830152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.700+0000 7fc8355ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83016acb0 con 0x7fc83015a990 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.700+0000 7fc8355ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300b1070 con 0x7fc8300cb3e0 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc8355ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300b1910 con 0x7fc830142c20 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc83015a990 0x7fc830152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc83015a990 0x7fc830152b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45620/0 (socket says 172.21.15.31:45620) 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82f7fe700 1 -- 172.21.15.31:0/1276610359 learned_addr learned my addr 172.21.15.31:0/1276610359 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82effd700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8300cb3e0 0x7fc8300acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82e7fc700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 0x7fc8300b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82f7fe700 1 -- 172.21.15.31:0/1276610359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 msgr2=0x7fc8300b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82f7fe700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 0x7fc8300b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82f7fe700 1 -- 172.21.15.31:0/1276610359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8300cb3e0 msgr2=0x7fc8300acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82f7fe700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8300cb3e0 0x7fc8300acf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82f7fe700 1 -- 172.21.15.31:0/1276610359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820009420 con 0x7fc83015a990 2023-10-06T05:20:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82e7fc700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 0x7fc8300b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.701+0000 7fc82f7fe700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc83015a990 0x7fc830152b60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc82800cd00 tx=0x7fc828001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.702+0000 7fc816ffd700 1 -- 172.21.15.31:0/1276610359 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800ab70 con 0x7fc83015a990 2023-10-06T05:20:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.702+0000 7fc816ffd700 1 -- 172.21.15.31:0/1276610359 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800acd0 con 0x7fc83015a990 2023-10-06T05:20:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.702+0000 7fc8355ff700 1 -- 172.21.15.31:0/1276610359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8300ad560 con 0x7fc83015a990 2023-10-06T05:20:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.702+0000 7fc816ffd700 1 -- 172.21.15.31:0/1276610359 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828007340 con 0x7fc83015a990 2023-10-06T05:20:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.702+0000 7fc8355ff700 1 -- 172.21.15.31:0/1276610359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8300b15a0 con 0x7fc83015a990 2023-10-06T05:20:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.703+0000 7fc8355ff700 1 -- 172.21.15.31:0/1276610359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8300b1ac0 con 0x7fc83015a990 2023-10-06T05:20:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.704+0000 7fc816ffd700 1 -- 172.21.15.31:0/1276610359 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fc8280076f0 con 0x7fc83015a990 2023-10-06T05:20:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.705+0000 7fc816ffd700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc81c0670e0 0x7fc81c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.705+0000 7fc816ffd700 1 -- 172.21.15.31:0/1276610359 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc828084f80 con 0x7fc83015a990 2023-10-06T05:20:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.705+0000 7fc82effd700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc81c0670e0 0x7fc81c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:46.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.706+0000 7fc82effd700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc81c0670e0 0x7fc81c069590 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc820009860 tx=0x7fc820017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:46.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.709+0000 7fc816ffd700 1 -- 172.21.15.31:0/1276610359 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8300b1ac0 con 0x7fc83015a990 2023-10-06T05:20:46.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:46.955+0000 7fc8355ff700 1 -- 172.21.15.31:0/1276610359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc8300b1a80 con 0x7fc83015a990 2023-10-06T05:20:47.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.309+0000 7fc816ffd700 1 -- 172.21.15.31:0/1276610359 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 136+0+0 (secure 0 0 0) 0x7fc828051ce0 con 0x7fc83015a990 2023-10-06T05:20:47.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 -- 172.21.15.31:0/1276610359 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc81c0670e0 msgr2=0x7fc81c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc81c0670e0 0x7fc81c069590 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc820009860 tx=0x7fc820017000 comp rx=0 tx=0).stop 2023-10-06T05:20:47.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 -- 172.21.15.31:0/1276610359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc83015a990 msgr2=0x7fc830152b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc83015a990 0x7fc830152b60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc82800cd00 tx=0x7fc828001ac0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 -- 172.21.15.31:0/1276610359 shutdown_connections 2023-10-06T05:20:47.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc81c0670e0 0x7fc81c069590 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc83015a990 0x7fc830152b60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8300cb3e0 0x7fc8300acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 --2- 172.21.15.31:0/1276610359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc830142c20 0x7fc8300b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 -- 172.21.15.31:0/1276610359 >> 172.21.15.31:0/1276610359 conn(0x7fc8300ce2d0 msgr2=0x7fc83006da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:47.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 -- 172.21.15.31:0/1276610359 shutdown_connections 2023-10-06T05:20:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.312+0000 7fc8355ff700 1 -- 172.21.15.31:0/1276610359 wait complete. 2023-10-06T05:20:47.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:20:47.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.531+0000 7f680b7ff700 1 Processor -- start 2023-10-06T05:20:47.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.531+0000 7f680b7ff700 1 -- start start 2023-10-06T05:20:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.532+0000 7f680b7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68040cf5a0 0x7f68040ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.533+0000 7f680b7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68040ce090 con 0x7f68040cd270 2023-10-06T05:20:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.533+0000 7f680b7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68040cb6f0 con 0x7f680415a930 2023-10-06T05:20:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.533+0000 7f680b7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68040cb9d0 con 0x7f68040cf5a0 2023-10-06T05:20:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.533+0000 7f6809d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f68040cd270 0x7f6804163240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41160/0 (socket says 172.21.15.31:41160) 2023-10-06T05:20:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.533+0000 7f6809d9c700 1 -- 172.21.15.31:0/4100585739 learned_addr learned my addr 172.21.15.31:0/4100585739 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.533+0000 7f6808d9a700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68040cf5a0 0x7f68040ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.533+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265318070 0 0) 0x7f68040ce090 con 0x7f68040cd270 2023-10-06T05:20:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.533+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67f0003570 con 0x7f68040cd270 2023-10-06T05:20:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795214084 0 0) 0x7f68040cb6f0 con 0x7f680415a930 2023-10-06T05:20:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68040ce090 con 0x7f680415a930 2023-10-06T05:20:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3767753029 0 0) 0x7f67f0003570 con 0x7f68040cd270 2023-10-06T05:20:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68040cf5a0 msgr2=0x7f68040ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68040cf5a0 0x7f68040ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 >> v1:172.21.15.62:6789/0 conn(0x7f680415a930 legacy=0x7f68040cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68040cbcb0 con 0x7f68040cd270 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f67fc0017f0 con 0x7f68040cd270 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 >> v1:172.21.15.31:6789/0 conn(0x7f68040cd270 legacy=0x7f6804163240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67f0003c20 0x7f67f0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f0004810 0x7f67f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68040cf5a0 0x7f67f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68040cb9d0 con 0x7f67f0004810 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f000bb70 con 0x7f67f0003c20 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f000bd30 con 0x7f68040cf5a0 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f6808d9a700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f0004810 0x7f67f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f680959b700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67f0003c20 0x7f67f0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.534+0000 7f6809d9c700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68040cf5a0 0x7f67f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.535+0000 7f6808d9a700 1 -- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68040cf5a0 msgr2=0x7f67f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.535+0000 7f6808d9a700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68040cf5a0 0x7f67f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.535+0000 7f6808d9a700 1 -- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67f0003c20 msgr2=0x7f67f0004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.535+0000 7f6808d9a700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67f0003c20 0x7f67f0004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.535+0000 7f6808d9a700 1 -- 172.21.15.31:0/4100585739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0003570 con 0x7f67f0004810 2023-10-06T05:20:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.535+0000 7f6808d9a700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f0004810 0x7f67f0004c80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009c00 tx=0x7f67f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5483b915d5f73b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.535+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f400e710 con 0x7f67f0004810 2023-10-06T05:20:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.535+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f400e870 con 0x7f67f0004810 2023-10-06T05:20:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.535+0000 7f67fbfff700 1 -- 172.21.15.31:0/4100585739 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f400ec30 con 0x7f67f0004810 2023-10-06T05:20:47.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.536+0000 7f680b7ff700 1 -- 172.21.15.31:0/4100585739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f0004810 msgr2=0x7f67f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.536+0000 7f680b7ff700 1 --2- 172.21.15.31:0/4100585739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67f0004810 0x7f67f0004c80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009c00 tx=0x7f67f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.536+0000 7f6809d9c700 1 -- 172.21.15.31:0/4100585739 reap_dead start 2023-10-06T05:20:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.536+0000 7f680b7ff700 1 -- 172.21.15.31:0/4100585739 shutdown_connections 2023-10-06T05:20:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.536+0000 7f680b7ff700 1 -- 172.21.15.31:0/4100585739 >> 172.21.15.31:0/4100585739 conn(0x7f6804099720 msgr2=0x7f680406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.536+0000 7f680b7ff700 1 -- 172.21.15.31:0/4100585739 shutdown_connections 2023-10-06T05:20:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.536+0000 7f680b7ff700 1 -- 172.21.15.31:0/4100585739 wait complete. 2023-10-06T05:20:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.538+0000 7f680b7ff700 1 Processor -- start 2023-10-06T05:20:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.538+0000 7f680b7ff700 1 -- start start 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.539+0000 7f680b7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67f0004810 0x7f680409da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.539+0000 7f680b7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f680415bea0 0x7f6804169e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.539+0000 7f680b7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6804158ad0 0x7f680409b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f680b7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680409a190 con 0x7f680415bea0 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f680b7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680415e530 con 0x7f6804158ad0 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f680b7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680416a3b0 con 0x7f67f0004810 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f6808d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f680415bea0 0x7f6804169e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f6808d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f680415bea0 0x7f6804169e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49294/0 (socket says 172.21.15.31:49294) 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f6808d9a700 1 -- 172.21.15.31:0/368021999 learned_addr learned my addr 172.21.15.31:0/368021999 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f680959b700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67f0004810 0x7f680409da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f6809d9c700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6804158ad0 0x7f680409b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f6808d9a700 1 -- 172.21.15.31:0/368021999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67f0004810 msgr2=0x7f680409da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f6808d9a700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67f0004810 0x7f680409da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f6808d9a700 1 -- 172.21.15.31:0/368021999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6804158ad0 msgr2=0x7f680409b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f6808d9a700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6804158ad0 0x7f680409b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.540+0000 7f6808d9a700 1 -- 172.21.15.31:0/368021999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680416a5e0 con 0x7f680415bea0 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.541+0000 7f6808d9a700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f680415bea0 0x7f6804169e70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e4b0 tx=0x7f67f401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.541+0000 7f67f97fa700 1 -- 172.21.15.31:0/368021999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4023070 con 0x7f680415bea0 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.541+0000 7f67f97fa700 1 -- 172.21.15.31:0/368021999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f400e9e0 con 0x7f680415bea0 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.541+0000 7f680b7ff700 1 -- 172.21.15.31:0/368021999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680416a8c0 con 0x7f680415bea0 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.541+0000 7f67f97fa700 1 -- 172.21.15.31:0/368021999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f400e710 con 0x7f680415bea0 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.541+0000 7f680b7ff700 1 -- 172.21.15.31:0/368021999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680409a370 con 0x7f680415bea0 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.542+0000 7f680b7ff700 1 -- 172.21.15.31:0/368021999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680416ade0 con 0x7f680415bea0 2023-10-06T05:20:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.543+0000 7f67f97fa700 1 -- 172.21.15.31:0/368021999 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f67f4013470 con 0x7f680415bea0 2023-10-06T05:20:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.544+0000 7f67f97fa700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f67e80670e0 0x7f67e8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.544+0000 7f67f97fa700 1 -- 172.21.15.31:0/368021999 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f67f4085fe0 con 0x7f680415bea0 2023-10-06T05:20:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.544+0000 7f680959b700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f67e80670e0 0x7f67e8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.545+0000 7f680959b700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f67e80670e0 0x7f67e8069590 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6800001130 tx=0x7f6800007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:47.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.549+0000 7f67f97fa700 1 -- 172.21.15.31:0/368021999 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67f4052e30 con 0x7f680415bea0 2023-10-06T05:20:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.782+0000 7f680b7ff700 1 -- 172.21.15.31:0/368021999 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f680409a190 con 0x7f67e80670e0 2023-10-06T05:20:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.783+0000 7f67f97fa700 1 -- 172.21.15.31:0/368021999 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f680409a190 con 0x7f67e80670e0 2023-10-06T05:20:47.786 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:20:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:20:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:47.786 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:20:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.785+0000 7f680b7ff700 1 -- 172.21.15.31:0/368021999 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f67e80670e0 msgr2=0x7f67e8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.785+0000 7f680b7ff700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f67e80670e0 0x7f67e8069590 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6800001130 tx=0x7f6800007d80 comp rx=0 tx=0).stop 2023-10-06T05:20:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.786+0000 7f680b7ff700 1 -- 172.21.15.31:0/368021999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f680415bea0 msgr2=0x7f6804169e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.786+0000 7f680b7ff700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f680415bea0 0x7f6804169e70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e4b0 tx=0x7f67f401a000 comp rx=0 tx=0).stop 2023-10-06T05:20:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.786+0000 7f680b7ff700 1 -- 172.21.15.31:0/368021999 shutdown_connections 2023-10-06T05:20:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.786+0000 7f680b7ff700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6804158ad0 0x7f680409b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.786+0000 7f680b7ff700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67f0004810 0x7f680409da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.786+0000 7f680b7ff700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f67e80670e0 0x7f67e8069590 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.786+0000 7f680b7ff700 1 --2- 172.21.15.31:0/368021999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f680415bea0 0x7f6804169e70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.786+0000 7f680b7ff700 1 -- 172.21.15.31:0/368021999 >> 172.21.15.31:0/368021999 conn(0x7f6804099720 msgr2=0x7f68040b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.786+0000 7f680b7ff700 1 -- 172.21.15.31:0/368021999 shutdown_connections 2023-10-06T05:20:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:47.786+0000 7f680b7ff700 1 -- 172.21.15.31:0/368021999 wait complete. 2023-10-06T05:20:47.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:20:48.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.002+0000 7f2fb9fff700 1 Processor -- start 2023-10-06T05:20:48.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.002+0000 7f2fb9fff700 1 -- start start 2023-10-06T05:20:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.003+0000 7f2fb9fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fb40c8f40 0x7f2fb4068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.003+0000 7f2fb9fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb4068da0 con 0x7f2fb415ca40 2023-10-06T05:20:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.003+0000 7f2fb9fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb4069080 con 0x7f2fb406a300 2023-10-06T05:20:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.003+0000 7f2fb9fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb40cabc0 con 0x7f2fb40c8f40 2023-10-06T05:20:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.004+0000 7f2fb2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fb40c8f40 0x7f2fb4068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.004+0000 7f2fb2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fb40c8f40 0x7f2fb4068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49304/0 (socket says 172.21.15.31:49304) 2023-10-06T05:20:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.004+0000 7f2fb2ffd700 1 -- 172.21.15.31:0/2108509827 learned_addr learned my addr 172.21.15.31:0/2108509827 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:48.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.004+0000 7f2fb2ffd700 1 -- 172.21.15.31:0/2108509827 >> v1:172.21.15.62:6789/0 conn(0x7f2fb406a300 legacy=0x7f2fb40adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.004+0000 7f2fb2ffd700 1 -- 172.21.15.31:0/2108509827 >> v1:172.21.15.31:6789/0 conn(0x7f2fb415ca40 legacy=0x7f2fb40c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.004+0000 7f2fb2ffd700 1 -- 172.21.15.31:0/2108509827 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb4069360 con 0x7f2fb40c8f40 2023-10-06T05:20:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.004+0000 7f2fb2ffd700 1 --2- 172.21.15.31:0/2108509827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb40c8f40 0x7f2fb4068670 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80099b0 tx=0x7f2fa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356a731c69ad95d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.005+0000 7f2fb27fc700 1 -- 172.21.15.31:0/2108509827 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800b860 con 0x7f2fb40c8f40 2023-10-06T05:20:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.005+0000 7f2fb27fc700 1 -- 172.21.15.31:0/2108509827 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa800b9c0 con 0x7f2fb40c8f40 2023-10-06T05:20:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.005+0000 7f2fb27fc700 1 -- 172.21.15.31:0/2108509827 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800bd60 con 0x7f2fb40c8f40 2023-10-06T05:20:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.005+0000 7f2fb9fff700 1 -- 172.21.15.31:0/2108509827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb40c8f40 msgr2=0x7f2fb4068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.005+0000 7f2fb9fff700 1 --2- 172.21.15.31:0/2108509827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb40c8f40 0x7f2fb4068670 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80099b0 tx=0x7f2fa8009250 comp rx=0 tx=0).stop 2023-10-06T05:20:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.005+0000 7f2fb9fff700 1 -- 172.21.15.31:0/2108509827 shutdown_connections 2023-10-06T05:20:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.005+0000 7f2fb9fff700 1 --2- 172.21.15.31:0/2108509827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb40c8f40 0x7f2fb4068670 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.005+0000 7f2fb9fff700 1 -- 172.21.15.31:0/2108509827 >> 172.21.15.31:0/2108509827 conn(0x7f2fb40ce290 msgr2=0x7f2fb40ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.005+0000 7f2fb9fff700 1 -- 172.21.15.31:0/2108509827 shutdown_connections 2023-10-06T05:20:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.005+0000 7f2fb9fff700 1 -- 172.21.15.31:0/2108509827 wait complete. 2023-10-06T05:20:48.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.007+0000 7f2fb9fff700 1 Processor -- start 2023-10-06T05:20:48.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.007+0000 7f2fb9fff700 1 -- start start 2023-10-06T05:20:48.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb9fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fb406a300 0x7f2fb416a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb9fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb40c8f40 0x7f2fb40c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb9fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb415ca40 0x7f2fb416c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb9fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4159120 con 0x7f2fb415ca40 2023-10-06T05:20:48.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb9fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb40c8030 con 0x7f2fb40c8f40 2023-10-06T05:20:48.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb9fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb40c7170 con 0x7f2fb406a300 2023-10-06T05:20:48.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb415ca40 0x7f2fb416c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb37fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fb406a300 0x7f2fb416a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb415ca40 0x7f2fb416c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49316/0 (socket says 172.21.15.31:49316) 2023-10-06T05:20:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb3fff700 1 -- 172.21.15.31:0/4015945241 learned_addr learned my addr 172.21.15.31:0/4015945241 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.008+0000 7f2fb2ffd700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb40c8f40 0x7f2fb40c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2fb3fff700 1 -- 172.21.15.31:0/4015945241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fb406a300 msgr2=0x7f2fb416a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2fb3fff700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fb406a300 0x7f2fb416a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2fb3fff700 1 -- 172.21.15.31:0/4015945241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb40c8f40 msgr2=0x7f2fb40c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2fb3fff700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb40c8f40 0x7f2fb40c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2fb3fff700 1 -- 172.21.15.31:0/4015945241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa80094f0 con 0x7f2fb415ca40 2023-10-06T05:20:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2fb3fff700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb415ca40 0x7f2fb416c380 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00c8e0 tx=0x7f2fac001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2f9b7fe700 1 -- 172.21.15.31:0/4015945241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac012630 con 0x7f2fb415ca40 2023-10-06T05:20:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2f9b7fe700 1 -- 172.21.15.31:0/4015945241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac012790 con 0x7f2fb415ca40 2023-10-06T05:20:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2fb9fff700 1 -- 172.21.15.31:0/4015945241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb40c7420 con 0x7f2fb415ca40 2023-10-06T05:20:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2f9b7fe700 1 -- 172.21.15.31:0/4015945241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac012b50 con 0x7f2fb415ca40 2023-10-06T05:20:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.009+0000 7f2fb9fff700 1 -- 172.21.15.31:0/4015945241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb40c79e0 con 0x7f2fb415ca40 2023-10-06T05:20:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.010+0000 7f2fb9fff700 1 -- 172.21.15.31:0/4015945241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb404c380 con 0x7f2fb415ca40 2023-10-06T05:20:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.011+0000 7f2f9b7fe700 1 -- 172.21.15.31:0/4015945241 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2fac01a3a0 con 0x7f2fb415ca40 2023-10-06T05:20:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.012+0000 7f2f9b7fe700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2fa00670e0 0x7f2fa0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.012+0000 7f2f9b7fe700 1 -- 172.21.15.31:0/4015945241 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2fac084bb0 con 0x7f2fb415ca40 2023-10-06T05:20:48.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.012+0000 7f2fb37fe700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2fa00670e0 0x7f2fa0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.013+0000 7f2fb37fe700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2fa00670e0 0x7f2fa0069590 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40b4160 tx=0x7f2fa4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.016+0000 7f2f9b7fe700 1 -- 172.21.15.31:0/4015945241 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fac0519a0 con 0x7f2fb415ca40 2023-10-06T05:20:48.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.220+0000 7f2fb9fff700 1 -- 172.21.15.31:0/4015945241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2fb40ca1a0 con 0x7f2fb415ca40 2023-10-06T05:20:48.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.220+0000 7f2f9b7fe700 1 -- 172.21.15.31:0/4015945241 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2fac054fc0 con 0x7f2fb415ca40 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 -- 172.21.15.31:0/4015945241 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2fa00670e0 msgr2=0x7f2fa0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2fa00670e0 0x7f2fa0069590 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40b4160 tx=0x7f2fa4009040 comp rx=0 tx=0).stop 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 -- 172.21.15.31:0/4015945241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb415ca40 msgr2=0x7f2fb416c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb415ca40 0x7f2fb416c380 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00c8e0 tx=0x7f2fac001710 comp rx=0 tx=0).stop 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 -- 172.21.15.31:0/4015945241 shutdown_connections 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2fa00670e0 0x7f2fa0069590 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb415ca40 0x7f2fb416c380 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fb406a300 0x7f2fb416a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 --2- 172.21.15.31:0/4015945241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb40c8f40 0x7f2fb40c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 -- 172.21.15.31:0/4015945241 >> 172.21.15.31:0/4015945241 conn(0x7f2fb40ce290 msgr2=0x7f2fb415a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 -- 172.21.15.31:0/4015945241 shutdown_connections 2023-10-06T05:20:48.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.223+0000 7f2fb9fff700 1 -- 172.21.15.31:0/4015945241 wait complete. 2023-10-06T05:20:48.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:48.242 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvolume_0000000000234638) 2023-10-06T05:20:48.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:48.324 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T05:20:48.372 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T05:20:48.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:48.373 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvolume_0000000000234638) 2023-10-06T05:20:48.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000234639 2023-10-06T05:20:48.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.645+0000 7f4c4c3ff700 1 Processor -- start 2023-10-06T05:20:48.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.645+0000 7f4c4c3ff700 1 -- start start 2023-10-06T05:20:48.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4c3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c440b0b90 0x7f4c440af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4c3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4c440af6c0 0x7f4c440adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4c3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c440ad690 con 0x7f4c44153d40 2023-10-06T05:20:48.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4c3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c440acd60 con 0x7f4c440af6c0 2023-10-06T05:20:48.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4c3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440ad040 con 0x7f4c440b0b90 2023-10-06T05:20:48.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4a19b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4c44153d40 0x7f4c440b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41180/0 (socket says 172.21.15.31:41180) 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4a19b700 1 -- 172.21.15.31:0/854675404 learned_addr learned my addr 172.21.15.31:0/854675404 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4a99c700 1 --2- 172.21.15.31:0/854675404 >> v2:172.21.15.31:3300/0 conn(0x7f4c440af6c0 0x7f4c440adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c49199700 1 -- 172.21.15.31:0/854675404 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495939371 0 0) 0x7f4c440ad690 con 0x7f4c44153d40 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4999a700 1 --2- 172.21.15.31:0/854675404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c440b0b90 0x7f4c440af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c49199700 1 -- 172.21.15.31:0/854675404 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c30003570 con 0x7f4c44153d40 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4a99c700 1 -- 172.21.15.31:0/854675404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c440b0b90 msgr2=0x7f4c440af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4a99c700 1 --2- 172.21.15.31:0/854675404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c440b0b90 0x7f4c440af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4a99c700 1 -- 172.21.15.31:0/854675404 >> v1:172.21.15.31:6789/0 conn(0x7f4c44153d40 legacy=0x7f4c440b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.647+0000 7f4c4a99c700 1 -- 172.21.15.31:0/854675404 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440ad320 con 0x7f4c440af6c0 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.648+0000 7f4c4a99c700 1 --2- 172.21.15.31:0/854675404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 0x7f4c440adb20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009e60 tx=0x7f4c3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfc2cb815ca78f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.648+0000 7f4c49199700 1 -- 172.21.15.31:0/854675404 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00e070 con 0x7f4c440af6c0 2023-10-06T05:20:48.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.648+0000 7f4c49199700 1 -- 172.21.15.31:0/854675404 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00bb60 con 0x7f4c440af6c0 2023-10-06T05:20:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.648+0000 7f4c49199700 1 -- 172.21.15.31:0/854675404 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c012430 con 0x7f4c440af6c0 2023-10-06T05:20:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.648+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/854675404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 msgr2=0x7f4c440adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.648+0000 7f4c4c3ff700 1 --2- 172.21.15.31:0/854675404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 0x7f4c440adb20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009e60 tx=0x7f4c3c009780 comp rx=0 tx=0).stop 2023-10-06T05:20:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.649+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/854675404 shutdown_connections 2023-10-06T05:20:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.649+0000 7f4c4c3ff700 1 --2- 172.21.15.31:0/854675404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 0x7f4c440adb20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.649+0000 7f4c4c3ff700 1 --2- 172.21.15.31:0/854675404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c440b0b90 0x7f4c440af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.649+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/854675404 >> 172.21.15.31:0/854675404 conn(0x7f4c44067560 msgr2=0x7f4c44067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.649+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/854675404 shutdown_connections 2023-10-06T05:20:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.649+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/854675404 wait complete. 2023-10-06T05:20:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.650+0000 7f4c4c3ff700 1 Processor -- start 2023-10-06T05:20:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.650+0000 7f4c4c3ff700 1 -- start start 2023-10-06T05:20:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.651+0000 7f4c4c3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 0x7f4c440cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.651+0000 7f4c4c3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c440b0b90 0x7f4c440c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.651+0000 7f4c4c3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c44153d40 0x7f4c4413e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.651+0000 7f4c4c3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c1410 con 0x7f4c440af6c0 2023-10-06T05:20:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.651+0000 7f4c4c3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c9c00 con 0x7f4c440b0b90 2023-10-06T05:20:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.651+0000 7f4c4c3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c9ee0 con 0x7f4c44153d40 2023-10-06T05:20:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.651+0000 7f4c4a19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 0x7f4c440cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4a19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 0x7f4c440cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49332/0 (socket says 172.21.15.31:49332) 2023-10-06T05:20:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4a19b700 1 -- 172.21.15.31:0/3077592501 learned_addr learned my addr 172.21.15.31:0/3077592501 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4999a700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c440b0b90 0x7f4c440c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4a99c700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c44153d40 0x7f4c4413e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4a19b700 1 -- 172.21.15.31:0/3077592501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c44153d40 msgr2=0x7f4c4413e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4a19b700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c44153d40 0x7f4c4413e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4a19b700 1 -- 172.21.15.31:0/3077592501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c440b0b90 msgr2=0x7f4c440c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4a19b700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c440b0b90 0x7f4c440c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4a19b700 1 -- 172.21.15.31:0/3077592501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c009a20 con 0x7f4c440af6c0 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4a19b700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 0x7f4c440cc510 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000c7c0 tx=0x7f4c40007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c3a7fc700 1 -- 172.21.15.31:0/3077592501 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40005cd0 con 0x7f4c440af6c0 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c3a7fc700 1 -- 172.21.15.31:0/3077592501 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c40005e30 con 0x7f4c440af6c0 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c3a7fc700 1 -- 172.21.15.31:0/3077592501 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40007320 con 0x7f4c440af6c0 2023-10-06T05:20:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.652+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/3077592501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c440ca1c0 con 0x7f4c440af6c0 2023-10-06T05:20:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.653+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/3077592501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c440ca780 con 0x7f4c440af6c0 2023-10-06T05:20:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.654+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/3077592501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c440af180 con 0x7f4c440af6c0 2023-10-06T05:20:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.654+0000 7f4c3a7fc700 1 -- 172.21.15.31:0/3077592501 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f4c4001b3b0 con 0x7f4c440af6c0 2023-10-06T05:20:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.655+0000 7f4c3a7fc700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c280670e0 0x7f4c28069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.655+0000 7f4c3a7fc700 1 -- 172.21.15.31:0/3077592501 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4c400840e0 con 0x7f4c440af6c0 2023-10-06T05:20:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.656+0000 7f4c4999a700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c280670e0 0x7f4c28069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.656+0000 7f4c4999a700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c280670e0 0x7f4c28069590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4c4406f0b0 tx=0x7f4c34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.659+0000 7f4c3a7fc700 1 -- 172.21.15.31:0/3077592501 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c40050ed0 con 0x7f4c440af6c0 2023-10-06T05:20:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.890+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/3077592501 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000234639", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c4404c380 con 0x7f4c280670e0 2023-10-06T05:20:48.898 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:48.896+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:48.898 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:48.896+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:48.898 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:48.896+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:48.898 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:48.896+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:48.898 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:48.896+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.941+0000 7f4c3a7fc700 1 -- 172.21.15.31:0/3077592501 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4c4404c380 con 0x7f4c280670e0 2023-10-06T05:20:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/3077592501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c280670e0 msgr2=0x7f4c28069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c280670e0 0x7f4c28069590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4c4406f0b0 tx=0x7f4c34008040 comp rx=0 tx=0).stop 2023-10-06T05:20:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/3077592501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 msgr2=0x7f4c440cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 0x7f4c440cc510 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000c7c0 tx=0x7f4c40007e00 comp rx=0 tx=0).stop 2023-10-06T05:20:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/3077592501 shutdown_connections 2023-10-06T05:20:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c44153d40 0x7f4c4413e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c280670e0 0x7f4c28069590 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440af6c0 0x7f4c440cc510 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 --2- 172.21.15.31:0/3077592501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c440b0b90 0x7f4c440c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/3077592501 >> 172.21.15.31:0/3077592501 conn(0x7f4c44067560 msgr2=0x7f4c44151d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/3077592501 shutdown_connections 2023-10-06T05:20:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:48.944+0000 7f4c4c3ff700 1 -- 172.21.15.31:0/3077592501 wait complete. 2023-10-06T05:20:48.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:48.963 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvolume_0000000000234639/.meta ./volumes/_nogroup/subvolume_0000000000234638) 2023-10-06T05:20:51.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000234638 2023-10-06T05:20:51.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.798+0000 7f193b3ff700 1 Processor -- start 2023-10-06T05:20:51.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.799+0000 7f193b3ff700 1 -- start start 2023-10-06T05:20:51.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1934159560 0x7f19340cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193b3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19340b2b80 0x7f19340cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193b3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193415d2d0 con 0x7f1934159930 2023-10-06T05:20:51.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193b3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193415d5b0 con 0x7f19340b2b80 2023-10-06T05:20:51.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193415d890 con 0x7f1934159560 2023-10-06T05:20:51.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193919b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1934159560 0x7f19340cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193999c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19340b2b80 0x7f19340cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193999c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19340b2b80 0x7f19340cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:42400/0 (socket says 172.21.15.31:42400) 2023-10-06T05:20:51.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193919b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1934159560 0x7f19340cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57716/0 (socket says 172.21.15.31:57716) 2023-10-06T05:20:51.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193999c700 1 -- 172.21.15.31:0/1564314080 learned_addr learned my addr 172.21.15.31:0/1564314080 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:51.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.800+0000 7f193899a700 1 --1- 172.21.15.31:0/1564314080 >> v1:172.21.15.62:6789/0 conn(0x7f1934159930 0x7f19340b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:45766/0 (socket says 172.21.15.31:45766) 2023-10-06T05:20:51.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.801+0000 7f193999c700 1 -- 172.21.15.31:0/1564314080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1934159560 msgr2=0x7f19340cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.801+0000 7f193999c700 1 --2- 172.21.15.31:0/1564314080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1934159560 0x7f19340cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.801+0000 7f193999c700 1 -- 172.21.15.31:0/1564314080 >> v1:172.21.15.62:6789/0 conn(0x7f1934159930 legacy=0x7f19340b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.801+0000 7f193999c700 1 -- 172.21.15.31:0/1564314080 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193415db70 con 0x7f19340b2b80 2023-10-06T05:20:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.801+0000 7f193919b700 1 --2- 172.21.15.31:0/1564314080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1934159560 0x7f19340cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.801+0000 7f193999c700 1 --2- 172.21.15.31:0/1564314080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19340b2b80 0x7f19340cd3b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f192c009f20 tx=0x7f192c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a693c093c49e3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.801+0000 7f192bfff700 1 -- 172.21.15.31:0/1564314080 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00e070 con 0x7f19340b2b80 2023-10-06T05:20:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.802+0000 7f192bfff700 1 -- 172.21.15.31:0/1564314080 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c00b9d0 con 0x7f19340b2b80 2023-10-06T05:20:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.802+0000 7f192bfff700 1 -- 172.21.15.31:0/1564314080 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0092e0 con 0x7f19340b2b80 2023-10-06T05:20:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.802+0000 7f193b3ff700 1 -- 172.21.15.31:0/1564314080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19340b2b80 msgr2=0x7f19340cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.802+0000 7f193b3ff700 1 --2- 172.21.15.31:0/1564314080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19340b2b80 0x7f19340cd3b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f192c009f20 tx=0x7f192c009840 comp rx=0 tx=0).stop 2023-10-06T05:20:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.802+0000 7f193b3ff700 1 -- 172.21.15.31:0/1564314080 shutdown_connections 2023-10-06T05:20:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.802+0000 7f193b3ff700 1 --2- 172.21.15.31:0/1564314080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1934159560 0x7f19340cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.802+0000 7f193b3ff700 1 --2- 172.21.15.31:0/1564314080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19340b2b80 0x7f19340cd3b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.802+0000 7f193b3ff700 1 -- 172.21.15.31:0/1564314080 >> 172.21.15.31:0/1564314080 conn(0x7f193406ec90 msgr2=0x7f193406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.802+0000 7f193b3ff700 1 -- 172.21.15.31:0/1564314080 shutdown_connections 2023-10-06T05:20:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.802+0000 7f193b3ff700 1 -- 172.21.15.31:0/1564314080 wait complete. 2023-10-06T05:20:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.804+0000 7f193b3ff700 1 Processor -- start 2023-10-06T05:20:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.804+0000 7f193b3ff700 1 -- start start 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19340b2b80 0x7f19341504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193b3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1934159560 0x7f19340cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193b3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1934159930 0x7f193415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193b3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19340cebd0 con 0x7f1934159930 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193b3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19340cdfc0 con 0x7f1934159560 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19340ce2a0 con 0x7f19340b2b80 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1934159930 0x7f193415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1934159930 0x7f193415e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49358/0 (socket says 172.21.15.31:49358) 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193999c700 1 -- 172.21.15.31:0/4126210182 learned_addr learned my addr 172.21.15.31:0/4126210182 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193899a700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1934159560 0x7f19340cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193919b700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19340b2b80 0x7f19341504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193999c700 1 -- 172.21.15.31:0/4126210182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19340b2b80 msgr2=0x7f19341504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193999c700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19340b2b80 0x7f19341504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193999c700 1 -- 172.21.15.31:0/4126210182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1934159560 msgr2=0x7f19340cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193999c700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1934159560 0x7f19340cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193999c700 1 -- 172.21.15.31:0/4126210182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1930009420 con 0x7f1934159930 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.805+0000 7f193899a700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1934159560 0x7f19340cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.806+0000 7f193999c700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1934159930 0x7f193415e4f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f192c00b610 tx=0x7f192c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.806+0000 7f19297fa700 1 -- 172.21.15.31:0/4126210182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c008b80 con 0x7f1934159930 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.806+0000 7f19297fa700 1 -- 172.21.15.31:0/4126210182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c008ce0 con 0x7f1934159930 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.806+0000 7f19297fa700 1 -- 172.21.15.31:0/4126210182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c026690 con 0x7f1934159930 2023-10-06T05:20:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.806+0000 7f193b3ff700 1 -- 172.21.15.31:0/4126210182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192c009ae0 con 0x7f1934159930 2023-10-06T05:20:51.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.806+0000 7f193b3ff700 1 -- 172.21.15.31:0/4126210182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19340ce860 con 0x7f1934159930 2023-10-06T05:20:51.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.807+0000 7f193b3ff700 1 -- 172.21.15.31:0/4126210182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19340ced80 con 0x7f1934159930 2023-10-06T05:20:51.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.808+0000 7f19297fa700 1 -- 172.21.15.31:0/4126210182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f192c003040 con 0x7f1934159930 2023-10-06T05:20:51.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.809+0000 7f19297fa700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1918067170 0x7f1918069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:51.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.810+0000 7f19297fa700 1 -- 172.21.15.31:0/4126210182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f192c086ad0 con 0x7f1934159930 2023-10-06T05:20:51.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.810+0000 7f193919b700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1918067170 0x7f1918069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:51.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.810+0000 7f193919b700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1918067170 0x7f1918069620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1934159260 tx=0x7f19300091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:51.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:51.814+0000 7f19297fa700 1 -- 172.21.15.31:0/4126210182 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19340ced80 con 0x7f1934159930 2023-10-06T05:20:52.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.050+0000 7f193b3ff700 1 -- 172.21.15.31:0/4126210182 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000234638", "target": ["mon-mgr", ""]}) v1 -- 0x7f19340ced80 con 0x7f1918067170 2023-10-06T05:20:52.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.091+0000 7f19297fa700 1 -- 172.21.15.31:0/4126210182 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7f19340ced80 con 0x7f1918067170 2023-10-06T05:20:52.093 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000234638 2023-10-06T05:20:52.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 -- 172.21.15.31:0/4126210182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1918067170 msgr2=0x7f1918069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1918067170 0x7f1918069620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1934159260 tx=0x7f19300091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 -- 172.21.15.31:0/4126210182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1934159930 msgr2=0x7f193415e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1934159930 0x7f193415e4f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f192c00b610 tx=0x7f192c018000 comp rx=0 tx=0).stop 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 -- 172.21.15.31:0/4126210182 shutdown_connections 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1918067170 0x7f1918069620 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1934159930 0x7f193415e4f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19340b2b80 0x7f19341504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 --2- 172.21.15.31:0/4126210182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1934159560 0x7f19340cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 -- 172.21.15.31:0/4126210182 >> 172.21.15.31:0/4126210182 conn(0x7f193406ec90 msgr2=0x7f19340b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 -- 172.21.15.31:0/4126210182 shutdown_connections 2023-10-06T05:20:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.094+0000 7f193b3ff700 1 -- 172.21.15.31:0/4126210182 wait complete. 2023-10-06T05:20:52.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000234638 2023-10-06T05:20:52.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.311+0000 7faf27dff700 1 Processor -- start 2023-10-06T05:20:52.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.311+0000 7faf27dff700 1 -- start start 2023-10-06T05:20:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.312+0000 7faf27dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200ae860 0x7faf20157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.312+0000 7faf27dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faf200b1260 0x7faf200b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.313+0000 7faf27dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf20158cf0 con 0x7faf201573a0 2023-10-06T05:20:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.313+0000 7faf27dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf200b1c20 con 0x7faf200b1260 2023-10-06T05:20:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.313+0000 7faf27dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf201580e0 con 0x7faf200ae860 2023-10-06T05:20:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.313+0000 7faf2539a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200ae860 0x7faf20157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.313+0000 7faf25b9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7faf201573a0 0x7faf200acb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:45780/0 (socket says 172.21.15.31:45780) 2023-10-06T05:20:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.313+0000 7faf25b9b700 1 -- 172.21.15.31:0/3981113134 learned_addr learned my addr 172.21.15.31:0/3981113134 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.313+0000 7faf2639c700 1 --2- 172.21.15.31:0/3981113134 >> v2:172.21.15.62:3300/0 conn(0x7faf200b1260 0x7faf200b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.314+0000 7faf24b99700 1 -- 172.21.15.31:0/3981113134 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780941223 0 0) 0x7faf20158cf0 con 0x7faf201573a0 2023-10-06T05:20:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.314+0000 7faf24b99700 1 -- 172.21.15.31:0/3981113134 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf0c003570 con 0x7faf201573a0 2023-10-06T05:20:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.314+0000 7faf2639c700 1 -- 172.21.15.31:0/3981113134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200ae860 msgr2=0x7faf20157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.314+0000 7faf2639c700 1 --2- 172.21.15.31:0/3981113134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200ae860 0x7faf20157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.314+0000 7faf2639c700 1 -- 172.21.15.31:0/3981113134 >> v1:172.21.15.62:6789/0 conn(0x7faf201573a0 legacy=0x7faf200acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.314+0000 7faf2639c700 1 -- 172.21.15.31:0/3981113134 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf201583c0 con 0x7faf200b1260 2023-10-06T05:20:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.314+0000 7faf2539a700 1 --2- 172.21.15.31:0/3981113134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200ae860 0x7faf20157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.314+0000 7faf2639c700 1 --2- 172.21.15.31:0/3981113134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf200b1260 0x7faf200b16e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faf18009e60 tx=0x7faf18009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93e22edc7d5ec479 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.314+0000 7faf24b99700 1 -- 172.21.15.31:0/3981113134 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800e070 con 0x7faf200b1260 2023-10-06T05:20:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.315+0000 7faf24b99700 1 -- 172.21.15.31:0/3981113134 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1800bab0 con 0x7faf200b1260 2023-10-06T05:20:52.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.315+0000 7faf24b99700 1 -- 172.21.15.31:0/3981113134 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800be70 con 0x7faf200b1260 2023-10-06T05:20:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.315+0000 7faf27dff700 1 -- 172.21.15.31:0/3981113134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf200b1260 msgr2=0x7faf200b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.315+0000 7faf27dff700 1 --2- 172.21.15.31:0/3981113134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf200b1260 0x7faf200b16e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faf18009e60 tx=0x7faf18009780 comp rx=0 tx=0).stop 2023-10-06T05:20:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.315+0000 7faf27dff700 1 -- 172.21.15.31:0/3981113134 shutdown_connections 2023-10-06T05:20:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.315+0000 7faf27dff700 1 --2- 172.21.15.31:0/3981113134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf200b1260 0x7faf200b16e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.315+0000 7faf27dff700 1 --2- 172.21.15.31:0/3981113134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200ae860 0x7faf20157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.315+0000 7faf27dff700 1 -- 172.21.15.31:0/3981113134 >> 172.21.15.31:0/3981113134 conn(0x7faf20067560 msgr2=0x7faf20067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.315+0000 7faf27dff700 1 -- 172.21.15.31:0/3981113134 shutdown_connections 2023-10-06T05:20:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.315+0000 7faf27dff700 1 -- 172.21.15.31:0/3981113134 wait complete. 2023-10-06T05:20:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.317+0000 7faf27dff700 1 Processor -- start 2023-10-06T05:20:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.317+0000 7faf27dff700 1 -- start start 2023-10-06T05:20:52.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf27dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200b1260 0x7faf20149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf27dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf201573a0 0x7faf200af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf27dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf2015c120 0x7faf2015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf27dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf200b0b10 con 0x7faf201573a0 2023-10-06T05:20:52.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf27dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2014a0e0 con 0x7faf2015c120 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf27dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf200aff00 con 0x7faf200b1260 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf2539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf201573a0 0x7faf200af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf25b9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200b1260 0x7faf20149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf2539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf201573a0 0x7faf200af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49372/0 (socket says 172.21.15.31:49372) 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf2539a700 1 -- 172.21.15.31:0/2070915468 learned_addr learned my addr 172.21.15.31:0/2070915468 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.318+0000 7faf2639c700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf2015c120 0x7faf2015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf2539a700 1 -- 172.21.15.31:0/2070915468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200b1260 msgr2=0x7faf20149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf2539a700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200b1260 0x7faf20149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf2539a700 1 -- 172.21.15.31:0/2070915468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf2015c120 msgr2=0x7faf2015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf2539a700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf2015c120 0x7faf2015ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf2539a700 1 -- 172.21.15.31:0/2070915468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf10009420 con 0x7faf201573a0 2023-10-06T05:20:52.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf2539a700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf201573a0 0x7faf200af6b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faf1000ec80 tx=0x7faf100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf15ffb700 1 -- 172.21.15.31:0/2070915468 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1000c850 con 0x7faf201573a0 2023-10-06T05:20:52.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf15ffb700 1 -- 172.21.15.31:0/2070915468 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1000c9b0 con 0x7faf201573a0 2023-10-06T05:20:52.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf27dff700 1 -- 172.21.15.31:0/2070915468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf18009a20 con 0x7faf201573a0 2023-10-06T05:20:52.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf15ffb700 1 -- 172.21.15.31:0/2070915468 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1000cd50 con 0x7faf201573a0 2023-10-06T05:20:52.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.319+0000 7faf27dff700 1 -- 172.21.15.31:0/2070915468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf200b04c0 con 0x7faf201573a0 2023-10-06T05:20:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.320+0000 7faf27dff700 1 -- 172.21.15.31:0/2070915468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2004c380 con 0x7faf201573a0 2023-10-06T05:20:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.321+0000 7faf15ffb700 1 -- 172.21.15.31:0/2070915468 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7faf10013680 con 0x7faf201573a0 2023-10-06T05:20:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.322+0000 7faf15ffb700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf04067170 0x7faf04069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.322+0000 7faf15ffb700 1 -- 172.21.15.31:0/2070915468 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7faf10084a80 con 0x7faf201573a0 2023-10-06T05:20:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.322+0000 7faf25b9b700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf04067170 0x7faf04069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.323+0000 7faf25b9b700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf04067170 0x7faf04069620 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf1c007fd0 tx=0x7faf1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.326+0000 7faf15ffb700 1 -- 172.21.15.31:0/2070915468 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf100517d0 con 0x7faf201573a0 2023-10-06T05:20:52.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.559+0000 7faf27dff700 1 -- 172.21.15.31:0/2070915468 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000234638", "target": ["mon-mgr", ""]}) v1 -- 0x7faf200b0c50 con 0x7faf04067170 2023-10-06T05:20:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.564+0000 7faf15ffb700 1 -- 172.21.15.31:0/2070915468 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7faf200b0c50 con 0x7faf04067170 2023-10-06T05:20:52.566 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000234638 2023-10-06T05:20:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 -- 172.21.15.31:0/2070915468 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf04067170 msgr2=0x7faf04069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf04067170 0x7faf04069620 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf1c007fd0 tx=0x7faf1c009040 comp rx=0 tx=0).stop 2023-10-06T05:20:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 -- 172.21.15.31:0/2070915468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf201573a0 msgr2=0x7faf200af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf201573a0 0x7faf200af6b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faf1000ec80 tx=0x7faf100091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 -- 172.21.15.31:0/2070915468 shutdown_connections 2023-10-06T05:20:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf2015c120 0x7faf2015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf200b1260 0x7faf20149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf04067170 0x7faf04069620 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 --2- 172.21.15.31:0/2070915468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf201573a0 0x7faf200af6b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 -- 172.21.15.31:0/2070915468 >> 172.21.15.31:0/2070915468 conn(0x7faf20067560 msgr2=0x7faf20156630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 -- 172.21.15.31:0/2070915468 shutdown_connections 2023-10-06T05:20:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.566+0000 7faf27dff700 1 -- 172.21.15.31:0/2070915468 wait complete. 2023-10-06T05:20:52.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:52.586 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/dec92b720058e82fc5668c18fc9abf73.meta) 2023-10-06T05:20:52.661 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:20:52.661 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T05:20:52.661 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T05:20:52.662 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000234638 2023-10-06T05:20:52.662 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:20:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:52.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvolume_0000000000234638 alice 2023-10-06T05:20:52.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.862+0000 7faa247ff700 1 Processor -- start 2023-10-06T05:20:52.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.862+0000 7faa247ff700 1 -- start start 2023-10-06T05:20:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.863+0000 7faa247ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa1c154cd0 0x7faa1c1583e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.863+0000 7faa247ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faa1c1550a0 0x7faa1c0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.863+0000 7faa247ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c06ee10 0x7faa1c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.863+0000 7faa247ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c06e610 con 0x7faa1c154cd0 2023-10-06T05:20:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.863+0000 7faa247ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c06f640 con 0x7faa1c1550a0 2023-10-06T05:20:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.863+0000 7faa247ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0adf60 con 0x7faa1c06ee10 2023-10-06T05:20:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.863+0000 7faa2259b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa1c154cd0 0x7faa1c1583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa2259b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa1c154cd0 0x7faa1c1583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49376/0 (socket says 172.21.15.31:49376) 2023-10-06T05:20:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa2259b700 1 -- 172.21.15.31:0/3578960745 learned_addr learned my addr 172.21.15.31:0/3578960745 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa21d9a700 1 --2- 172.21.15.31:0/3578960745 >> v2:172.21.15.62:3300/0 conn(0x7faa1c1550a0 0x7faa1c0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa22d9c700 1 --2- 172.21.15.31:0/3578960745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c06ee10 0x7faa1c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa2259b700 1 -- 172.21.15.31:0/3578960745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c06ee10 msgr2=0x7faa1c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa2259b700 1 --2- 172.21.15.31:0/3578960745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c06ee10 0x7faa1c06d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa2259b700 1 -- 172.21.15.31:0/3578960745 >> v2:172.21.15.62:3300/0 conn(0x7faa1c1550a0 msgr2=0x7faa1c0ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa2259b700 1 --2- 172.21.15.31:0/3578960745 >> v2:172.21.15.62:3300/0 conn(0x7faa1c1550a0 0x7faa1c0ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa2259b700 1 -- 172.21.15.31:0/3578960745 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c158920 con 0x7faa1c154cd0 2023-10-06T05:20:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa21d9a700 1 --2- 172.21.15.31:0/3578960745 >> v2:172.21.15.62:3300/0 conn(0x7faa1c1550a0 0x7faa1c0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa22d9c700 1 --2- 172.21.15.31:0/3578960745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c06ee10 0x7faa1c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa2259b700 1 --2- 172.21.15.31:0/3578960745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 0x7faa1c1583e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7faa1c06ceb0 tx=0x7faa18009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723151f7bf355711 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.864+0000 7faa21599700 1 -- 172.21.15.31:0/3578960745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18005c10 con 0x7faa1c154cd0 2023-10-06T05:20:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa21599700 1 -- 172.21.15.31:0/3578960745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa18005d70 con 0x7faa1c154cd0 2023-10-06T05:20:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa21599700 1 -- 172.21.15.31:0/3578960745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1800b880 con 0x7faa1c154cd0 2023-10-06T05:20:52.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa247ff700 1 -- 172.21.15.31:0/3578960745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 msgr2=0x7faa1c1583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa247ff700 1 --2- 172.21.15.31:0/3578960745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 0x7faa1c1583e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7faa1c06ceb0 tx=0x7faa18009220 comp rx=0 tx=0).stop 2023-10-06T05:20:52.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa247ff700 1 -- 172.21.15.31:0/3578960745 shutdown_connections 2023-10-06T05:20:52.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa247ff700 1 --2- 172.21.15.31:0/3578960745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c06ee10 0x7faa1c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa247ff700 1 --2- 172.21.15.31:0/3578960745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 0x7faa1c1583e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa247ff700 1 --2- 172.21.15.31:0/3578960745 >> v2:172.21.15.62:3300/0 conn(0x7faa1c1550a0 0x7faa1c0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa247ff700 1 -- 172.21.15.31:0/3578960745 >> 172.21.15.31:0/3578960745 conn(0x7faa1c068030 msgr2=0x7faa1c0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:52.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa247ff700 1 -- 172.21.15.31:0/3578960745 shutdown_connections 2023-10-06T05:20:52.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.865+0000 7faa247ff700 1 -- 172.21.15.31:0/3578960745 wait complete. 2023-10-06T05:20:52.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.867+0000 7faa247ff700 1 Processor -- start 2023-10-06T05:20:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.867+0000 7faa247ff700 1 -- start start 2023-10-06T05:20:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa247ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa1c06ee10 0x7faa1c1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa247ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 0x7faa1c0c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa247ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c1550a0 0x7faa1c0c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa247ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0ab8f0 con 0x7faa1c154cd0 2023-10-06T05:20:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa247ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c16d510 con 0x7faa1c06ee10 2023-10-06T05:20:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa247ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c153bf0 con 0x7faa1c1550a0 2023-10-06T05:20:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa21d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 0x7faa1c0c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa21d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 0x7faa1c0c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49386/0 (socket says 172.21.15.31:49386) 2023-10-06T05:20:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa21d9a700 1 -- 172.21.15.31:0/1250447924 learned_addr learned my addr 172.21.15.31:0/1250447924 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa2259b700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa1c06ee10 0x7faa1c1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa22d9c700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c1550a0 0x7faa1c0c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa21d9a700 1 -- 172.21.15.31:0/1250447924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c1550a0 msgr2=0x7faa1c0c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa21d9a700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c1550a0 0x7faa1c0c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa21d9a700 1 -- 172.21.15.31:0/1250447924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa1c06ee10 msgr2=0x7faa1c1536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa21d9a700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa1c06ee10 0x7faa1c1536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa21d9a700 1 -- 172.21.15.31:0/1250447924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa18009480 con 0x7faa1c154cd0 2023-10-06T05:20:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.868+0000 7faa22d9c700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c1550a0 0x7faa1c0c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.869+0000 7faa21d9a700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 0x7faa1c0c5560 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c8b0 tx=0x7faa0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.869+0000 7faa127fc700 1 -- 172.21.15.31:0/1250447924 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011600 con 0x7faa1c154cd0 2023-10-06T05:20:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.869+0000 7faa127fc700 1 -- 172.21.15.31:0/1250447924 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c011760 con 0x7faa1c154cd0 2023-10-06T05:20:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.869+0000 7faa127fc700 1 -- 172.21.15.31:0/1250447924 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011b00 con 0x7faa1c154cd0 2023-10-06T05:20:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.869+0000 7faa247ff700 1 -- 172.21.15.31:0/1250447924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c0c6090 con 0x7faa1c154cd0 2023-10-06T05:20:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.869+0000 7faa247ff700 1 -- 172.21.15.31:0/1250447924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c16cec0 con 0x7faa1c154cd0 2023-10-06T05:20:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.870+0000 7faa247ff700 1 -- 172.21.15.31:0/1250447924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c16d6a0 con 0x7faa1c154cd0 2023-10-06T05:20:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.871+0000 7faa127fc700 1 -- 172.21.15.31:0/1250447924 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7faa0c018350 con 0x7faa1c154cd0 2023-10-06T05:20:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.872+0000 7faa127fc700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa00067170 0x7faa00069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.872+0000 7faa127fc700 1 -- 172.21.15.31:0/1250447924 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7faa0c084a80 con 0x7faa1c154cd0 2023-10-06T05:20:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.872+0000 7faa2259b700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa00067170 0x7faa00069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.873+0000 7faa2259b700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa00067170 0x7faa00069620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faa18009260 tx=0x7faa18016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:52.876+0000 7faa127fc700 1 -- 172.21.15.31:0/1250447924 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa0c0516c0 con 0x7faa1c154cd0 2023-10-06T05:20:53.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.116+0000 7faa247ff700 1 -- 172.21.15.31:0/1250447924 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000234638", "auth_id": "alice", "target": ["mon-mgr", ""]}) v1 -- 0x7faa1c04c380 con 0x7faa00067170 2023-10-06T05:20:53.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.169+0000 7faa127fc700 1 -- 172.21.15.31:0/1250447924 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7faa1c04c380 con 0x7faa00067170 2023-10-06T05:20:53.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 -- 172.21.15.31:0/1250447924 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa00067170 msgr2=0x7faa00069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa00067170 0x7faa00069620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faa18009260 tx=0x7faa18016000 comp rx=0 tx=0).stop 2023-10-06T05:20:53.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 -- 172.21.15.31:0/1250447924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 msgr2=0x7faa1c0c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 0x7faa1c0c5560 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c8b0 tx=0x7faa0c008040 comp rx=0 tx=0).stop 2023-10-06T05:20:53.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 -- 172.21.15.31:0/1250447924 shutdown_connections 2023-10-06T05:20:53.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa1c1550a0 0x7faa1c0c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa1c06ee10 0x7faa1c1536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa00067170 0x7faa00069620 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 --2- 172.21.15.31:0/1250447924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa1c154cd0 0x7faa1c0c5560 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 -- 172.21.15.31:0/1250447924 >> 172.21.15.31:0/1250447924 conn(0x7faa1c068030 msgr2=0x7faa1c154450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:53.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 -- 172.21.15.31:0/1250447924 shutdown_connections 2023-10-06T05:20:53.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.171+0000 7faa247ff700 1 -- 172.21.15.31:0/1250447924 wait complete. 2023-10-06T05:20:53.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2023-10-06T05:20:53.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.388+0000 7fd2ba9ff700 1 Processor -- start 2023-10-06T05:20:53.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.388+0000 7fd2ba9ff700 1 -- start start 2023-10-06T05:20:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.389+0000 7fd2ba9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b41596d0 0x7fd2b4159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.389+0000 7fd2ba9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd2b40c6f80 0x7fd2b406a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.389+0000 7fd2ba9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2b40c16e0 0x7fd2b40c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.389+0000 7fd2ba9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b40c20c0 con 0x7fd2b40c16e0 2023-10-06T05:20:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.389+0000 7fd2ba9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b4067f70 con 0x7fd2b40c6f80 2023-10-06T05:20:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.389+0000 7fd2ba9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b406e040 con 0x7fd2b41596d0 2023-10-06T05:20:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b8f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2b40c16e0 0x7fd2b40c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b8f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2b40c16e0 0x7fd2b40c1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49398/0 (socket says 172.21.15.31:49398) 2023-10-06T05:20:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b8f9c700 1 -- 172.21.15.31:0/2186070868 learned_addr learned my addr 172.21.15.31:0/2186070868 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b3fff700 1 --2- 172.21.15.31:0/2186070868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b41596d0 0x7fd2b4159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b37fe700 1 --2- 172.21.15.31:0/2186070868 >> v2:172.21.15.62:3300/0 conn(0x7fd2b40c6f80 0x7fd2b406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b8f9c700 1 -- 172.21.15.31:0/2186070868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b41596d0 msgr2=0x7fd2b4159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b8f9c700 1 --2- 172.21.15.31:0/2186070868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b41596d0 0x7fd2b4159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b8f9c700 1 -- 172.21.15.31:0/2186070868 >> v2:172.21.15.62:3300/0 conn(0x7fd2b40c6f80 msgr2=0x7fd2b406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b8f9c700 1 --2- 172.21.15.31:0/2186070868 >> v2:172.21.15.62:3300/0 conn(0x7fd2b40c6f80 0x7fd2b406a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b8f9c700 1 -- 172.21.15.31:0/2186070868 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b40681c0 con 0x7fd2b40c16e0 2023-10-06T05:20:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.390+0000 7fd2b8f9c700 1 --2- 172.21.15.31:0/2186070868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b40c16e0 0x7fd2b40c1b80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4069940 tx=0x7fd2ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faffbedf6ca616c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2b2ffd700 1 -- 172.21.15.31:0/2186070868 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac00e070 con 0x7fd2b40c16e0 2023-10-06T05:20:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2b2ffd700 1 -- 172.21.15.31:0/2186070868 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac00ba50 con 0x7fd2b40c16e0 2023-10-06T05:20:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2b2ffd700 1 -- 172.21.15.31:0/2186070868 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0092e0 con 0x7fd2b40c16e0 2023-10-06T05:20:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/2186070868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b40c16e0 msgr2=0x7fd2b40c1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2ba9ff700 1 --2- 172.21.15.31:0/2186070868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b40c16e0 0x7fd2b40c1b80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4069940 tx=0x7fd2ac009840 comp rx=0 tx=0).stop 2023-10-06T05:20:53.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/2186070868 shutdown_connections 2023-10-06T05:20:53.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2ba9ff700 1 --2- 172.21.15.31:0/2186070868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b40c16e0 0x7fd2b40c1b80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2ba9ff700 1 --2- 172.21.15.31:0/2186070868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b41596d0 0x7fd2b4159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2ba9ff700 1 --2- 172.21.15.31:0/2186070868 >> v2:172.21.15.62:3300/0 conn(0x7fd2b40c6f80 0x7fd2b406a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/2186070868 >> 172.21.15.31:0/2186070868 conn(0x7fd2b40c9e80 msgr2=0x7fd2b406a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:53.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/2186070868 shutdown_connections 2023-10-06T05:20:53.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.391+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/2186070868 wait complete. 2023-10-06T05:20:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.393+0000 7fd2ba9ff700 1 Processor -- start 2023-10-06T05:20:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.393+0000 7fd2ba9ff700 1 -- start start 2023-10-06T05:20:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2ba9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b40c16e0 0x7fd2b415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2ba9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2b40c6f80 0x7fd2b4169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2ba9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b41596d0 0x7fd2b414b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2ba9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b416a380 con 0x7fd2b41596d0 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2ba9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b415d020 con 0x7fd2b40c6f80 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2ba9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b416a730 con 0x7fd2b40c16e0 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2b8f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b41596d0 0x7fd2b414b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2b3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b40c16e0 0x7fd2b415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2b8f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b41596d0 0x7fd2b414b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49412/0 (socket says 172.21.15.31:49412) 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2b8f9c700 1 -- 172.21.15.31:0/525397792 learned_addr learned my addr 172.21.15.31:0/525397792 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2b3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b40c16e0 0x7fd2b415c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57756/0 (socket says 172.21.15.31:57756) 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.394+0000 7fd2b37fe700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2b40c6f80 0x7fd2b4169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd2b8f9c700 1 -- 172.21.15.31:0/525397792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b40c16e0 msgr2=0x7fd2b415c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd2b8f9c700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b40c16e0 0x7fd2b415c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd2b8f9c700 1 -- 172.21.15.31:0/525397792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2b40c6f80 msgr2=0x7fd2b4169e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd2b8f9c700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2b40c6f80 0x7fd2b4169e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd2b8f9c700 1 -- 172.21.15.31:0/525397792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac009ae0 con 0x7fd2b41596d0 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd2b8f9c700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b41596d0 0x7fd2b414b3e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4069a60 tx=0x7fd2ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd29bfff700 1 -- 172.21.15.31:0/525397792 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0032f0 con 0x7fd2b41596d0 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd29bfff700 1 -- 172.21.15.31:0/525397792 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac008d20 con 0x7fd2b41596d0 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd29bfff700 1 -- 172.21.15.31:0/525397792 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0266b0 con 0x7fd2b41596d0 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/525397792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b416aa10 con 0x7fd2b41596d0 2023-10-06T05:20:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.395+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/525397792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b414b920 con 0x7fd2b41596d0 2023-10-06T05:20:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.396+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/525397792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b416a590 con 0x7fd2b41596d0 2023-10-06T05:20:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.397+0000 7fd29bfff700 1 -- 172.21.15.31:0/525397792 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fd2ac003040 con 0x7fd2b41596d0 2023-10-06T05:20:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.398+0000 7fd29bfff700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd2a0067170 0x7fd2a0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.399+0000 7fd29bfff700 1 -- 172.21.15.31:0/525397792 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fd2ac086b80 con 0x7fd2b41596d0 2023-10-06T05:20:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.399+0000 7fd2b3fff700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd2a0067170 0x7fd2a0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.399+0000 7fd2b3fff700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd2a0067170 0x7fd2a0069620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4006fd0 tx=0x7fd2a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.403+0000 7fd29bfff700 1 -- 172.21.15.31:0/525397792 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2b416a590 con 0x7fd2b41596d0 2023-10-06T05:20:53.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.683+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/525397792 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) v1 -- 0x7fd2b416a590 con 0x7fd2b41596d0 2023-10-06T05:20:53.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.683+0000 7fd29bfff700 1 -- 172.21.15.31:0/525397792 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v14) v1 ==== 107+0+295 (secure 0 0 0) 0x7fd2b416a590 con 0x7fd2b41596d0 2023-10-06T05:20:53.685 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:53.685 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:20:53.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:20:53.686 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.alice", 2023-10-06T05:20:53.686 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQA1mR9lk52xCBAAn0EGSnWA66PUAhI58soddA==", 2023-10-06T05:20:53.686 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:20:53.686 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow rw path=/volumes/_nogroup/subvolume_0000000000234638", 2023-10-06T05:20:53.686 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:20:53.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw pool=cephfs_data" 2023-10-06T05:20:53.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:53.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:20:53.687 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:20:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.685+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/525397792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd2a0067170 msgr2=0x7fd2a0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.685+0000 7fd2ba9ff700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd2a0067170 0x7fd2a0069620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4006fd0 tx=0x7fd2a4008040 comp rx=0 tx=0).stop 2023-10-06T05:20:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.685+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/525397792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b41596d0 msgr2=0x7fd2b414b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.685+0000 7fd2ba9ff700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b41596d0 0x7fd2b414b3e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4069a60 tx=0x7fd2ac018000 comp rx=0 tx=0).stop 2023-10-06T05:20:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.686+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/525397792 shutdown_connections 2023-10-06T05:20:53.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.686+0000 7fd2ba9ff700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd2a0067170 0x7fd2a0069620 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.686+0000 7fd2ba9ff700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2b41596d0 0x7fd2b414b3e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.686+0000 7fd2ba9ff700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd2b40c16e0 0x7fd2b415c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.686+0000 7fd2ba9ff700 1 --2- 172.21.15.31:0/525397792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2b40c6f80 0x7fd2b4169e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.686+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/525397792 >> 172.21.15.31:0/525397792 conn(0x7fd2b40c9e80 msgr2=0x7fd2b4067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:53.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.686+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/525397792 shutdown_connections 2023-10-06T05:20:53.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.686+0000 7fd2ba9ff700 1 -- 172.21.15.31:0/525397792 wait complete. 2023-10-06T05:20:53.705 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000234638 2023-10-06T05:20:53.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.902+0000 7fecd23ff700 1 Processor -- start 2023-10-06T05:20:53.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.903+0000 7fecd23ff700 1 -- start start 2023-10-06T05:20:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.904+0000 7fecd23ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feccc153d40 0x7feccc0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.904+0000 7fecd23ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feccc0b0e40 con 0x7feccc0af890 2023-10-06T05:20:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.904+0000 7fecd23ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feccc0afc90 con 0x7feccc0b09c0 2023-10-06T05:20:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.904+0000 7fecd23ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feccc0ad690 con 0x7feccc153d40 2023-10-06T05:20:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.904+0000 7feccbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feccc153d40 0x7feccc0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.904+0000 7feccbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feccc153d40 0x7feccc0b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49418/0 (socket says 172.21.15.31:49418) 2023-10-06T05:20:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.904+0000 7feccbfff700 1 -- 172.21.15.31:0/1827702943 learned_addr learned my addr 172.21.15.31:0/1827702943 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.905+0000 7feccbfff700 1 -- 172.21.15.31:0/1827702943 >> v1:172.21.15.62:6789/0 conn(0x7feccc0b09c0 legacy=0x7feccc0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.905+0000 7feccbfff700 1 -- 172.21.15.31:0/1827702943 >> v1:172.21.15.31:6789/0 conn(0x7feccc0af890 legacy=0x7feccc0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.905+0000 7feccbfff700 1 -- 172.21.15.31:0/1827702943 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc0acd60 con 0x7feccc153d40 2023-10-06T05:20:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.905+0000 7feccbfff700 1 --2- 172.21.15.31:0/1827702943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc153d40 0x7feccc0b0480 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0099b0 tx=0x7fecbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ebf7a3c3f1f89c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.905+0000 7feccaffd700 1 -- 172.21.15.31:0/1827702943 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00b820 con 0x7feccc153d40 2023-10-06T05:20:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.905+0000 7feccaffd700 1 -- 172.21.15.31:0/1827702943 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc00b980 con 0x7feccc153d40 2023-10-06T05:20:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.905+0000 7feccaffd700 1 -- 172.21.15.31:0/1827702943 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00bd20 con 0x7feccc153d40 2023-10-06T05:20:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.906+0000 7fecd23ff700 1 -- 172.21.15.31:0/1827702943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc153d40 msgr2=0x7feccc0b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.906+0000 7fecd23ff700 1 --2- 172.21.15.31:0/1827702943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc153d40 0x7feccc0b0480 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0099b0 tx=0x7fecbc009250 comp rx=0 tx=0).stop 2023-10-06T05:20:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.906+0000 7fecd23ff700 1 -- 172.21.15.31:0/1827702943 shutdown_connections 2023-10-06T05:20:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.906+0000 7fecd23ff700 1 --2- 172.21.15.31:0/1827702943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc153d40 0x7feccc0b0480 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.906+0000 7fecd23ff700 1 -- 172.21.15.31:0/1827702943 >> 172.21.15.31:0/1827702943 conn(0x7feccc067560 msgr2=0x7feccc067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.906+0000 7fecd23ff700 1 -- 172.21.15.31:0/1827702943 shutdown_connections 2023-10-06T05:20:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.906+0000 7fecd23ff700 1 -- 172.21.15.31:0/1827702943 wait complete. 2023-10-06T05:20:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.907+0000 7fecd23ff700 1 Processor -- start 2023-10-06T05:20:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.908+0000 7fecd23ff700 1 -- start start 2023-10-06T05:20:53.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.908+0000 7fecd23ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc0af890 0x7feccc0c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.908+0000 7fecd23ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feccc0b09c0 0x7feccc14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.908+0000 7fecd23ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feccc153d40 0x7feccc0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7fecd23ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc0c7dd0 con 0x7feccc0af890 2023-10-06T05:20:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7fecd23ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc0c67f0 con 0x7feccc153d40 2023-10-06T05:20:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7fecd23ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc0c6ad0 con 0x7feccc0b09c0 2023-10-06T05:20:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc0af890 0x7feccc0c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc0af890 0x7feccc0c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49424/0 (socket says 172.21.15.31:49424) 2023-10-06T05:20:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccbfff700 1 -- 172.21.15.31:0/3984676283 learned_addr learned my addr 172.21.15.31:0/3984676283 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7fecd099c700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feccc153d40 0x7feccc0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccbfff700 1 -- 172.21.15.31:0/3984676283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feccc0b09c0 msgr2=0x7feccc14f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccb7fe700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feccc0b09c0 0x7feccc14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccbfff700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feccc0b09c0 0x7feccc14f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccbfff700 1 -- 172.21.15.31:0/3984676283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feccc153d40 msgr2=0x7feccc0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccbfff700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feccc153d40 0x7feccc0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccbfff700 1 -- 172.21.15.31:0/3984676283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc0094f0 con 0x7feccc0af890 2023-10-06T05:20:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7fecd099c700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feccc153d40 0x7feccc0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccb7fe700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feccc0b09c0 0x7feccc14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.909+0000 7feccbfff700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc0af890 0x7feccc0c7890 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c010 tx=0x7fecbc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.910+0000 7fecb3fff700 1 -- 172.21.15.31:0/3984676283 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc011e10 con 0x7feccc0af890 2023-10-06T05:20:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.910+0000 7fecb3fff700 1 -- 172.21.15.31:0/3984676283 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc020070 con 0x7feccc0af890 2023-10-06T05:20:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.910+0000 7fecd23ff700 1 -- 172.21.15.31:0/3984676283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc0c6db0 con 0x7feccc0af890 2023-10-06T05:20:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.910+0000 7fecb3fff700 1 -- 172.21.15.31:0/3984676283 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc01b680 con 0x7feccc0af890 2023-10-06T05:20:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.910+0000 7fecd23ff700 1 -- 172.21.15.31:0/3984676283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc0c7370 con 0x7feccc0af890 2023-10-06T05:20:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.911+0000 7fecd23ff700 1 -- 172.21.15.31:0/3984676283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feccc068210 con 0x7feccc0af890 2023-10-06T05:20:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.912+0000 7fecb3fff700 1 -- 172.21.15.31:0/3984676283 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fecbc022030 con 0x7feccc0af890 2023-10-06T05:20:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.913+0000 7fecb3fff700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb8067170 0x7fecb8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.913+0000 7fecb3fff700 1 -- 172.21.15.31:0/3984676283 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fecbc0868e0 con 0x7feccc0af890 2023-10-06T05:20:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.913+0000 7feccb7fe700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb8067170 0x7fecb8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.913+0000 7feccb7fe700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb8067170 0x7fecb8069620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feccc15f300 tx=0x7fecc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:53.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:53.917+0000 7fecb3fff700 1 -- 172.21.15.31:0/3984676283 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecbc053630 con 0x7feccc0af890 2023-10-06T05:20:54.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.153+0000 7fecd23ff700 1 -- 172.21.15.31:0/3984676283 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000234638", "target": ["mon-mgr", ""]}) v1 -- 0x7feccc0afc90 con 0x7fecb8067170 2023-10-06T05:20:54.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.168+0000 7fecb3fff700 1 -- 172.21.15.31:0/3984676283 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7feccc0afc90 con 0x7fecb8067170 2023-10-06T05:20:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.171+0000 7fecd23ff700 1 -- 172.21.15.31:0/3984676283 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb8067170 msgr2=0x7fecb8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.171+0000 7fecd23ff700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb8067170 0x7fecb8069620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feccc15f300 tx=0x7fecc0009040 comp rx=0 tx=0).stop 2023-10-06T05:20:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.171+0000 7fecd23ff700 1 -- 172.21.15.31:0/3984676283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc0af890 msgr2=0x7feccc0c7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.171+0000 7fecd23ff700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc0af890 0x7feccc0c7890 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c010 tx=0x7fecbc016000 comp rx=0 tx=0).stop 2023-10-06T05:20:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.171+0000 7fecd23ff700 1 -- 172.21.15.31:0/3984676283 shutdown_connections 2023-10-06T05:20:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.171+0000 7fecd23ff700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feccc153d40 0x7feccc0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.171+0000 7fecd23ff700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fecb8067170 0x7fecb8069620 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.171+0000 7fecd23ff700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feccc0af890 0x7feccc0c7890 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.171+0000 7fecd23ff700 1 --2- 172.21.15.31:0/3984676283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feccc0b09c0 0x7feccc14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.171+0000 7fecd23ff700 1 -- 172.21.15.31:0/3984676283 >> 172.21.15.31:0/3984676283 conn(0x7feccc067560 msgr2=0x7feccc06d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.172+0000 7fecd23ff700 1 -- 172.21.15.31:0/3984676283 shutdown_connections 2023-10-06T05:20:54.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.172+0000 7fecd23ff700 1 -- 172.21.15.31:0/3984676283 wait complete. 2023-10-06T05:20:54.176 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:54.174+0000 7efe4f603700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:54.176 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:54.174+0000 7efe4f603700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:54.176 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:54.174+0000 7efe4f603700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:54.176 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:54.174+0000 7efe4f603700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:54.176 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:54.174+0000 7efe4f603700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:54.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000234639 2023-10-06T05:20:54.195 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:54.194+0000 7efe4ee02700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:54.195 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:54.194+0000 7efe4ee02700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:54.195 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:54.194+0000 7efe4ee02700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:54.195 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:54.194+0000 7efe4ee02700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:54.196 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:20:54.194+0000 7efe4ee02700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:20:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.394+0000 7f4b521ff700 1 Processor -- start 2023-10-06T05:20:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.395+0000 7f4b521ff700 1 -- start start 2023-10-06T05:20:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.396+0000 7f4b521ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b4c1573a0 0x7f4b4c0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.396+0000 7f4b521ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b4c157770 0x7f4b4c0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.396+0000 7f4b521ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4b4c157be0 0x7f4b4c0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.396+0000 7f4b521ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c158cf0 con 0x7f4b4c1573a0 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.396+0000 7f4b521ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c1580e0 con 0x7f4b4c157be0 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.396+0000 7f4b521ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c1583c0 con 0x7f4b4c157770 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.396+0000 7f4b4b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b4c1573a0 0x7f4b4c0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.396+0000 7f4b4b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b4c1573a0 0x7f4b4c0acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49460/0 (socket says 172.21.15.31:49460) 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.396+0000 7f4b4b7fe700 1 -- 172.21.15.31:0/3743085243 learned_addr learned my addr 172.21.15.31:0/3743085243 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4bfff700 1 --2- 172.21.15.31:0/3743085243 >> v2:172.21.15.62:3300/0 conn(0x7f4b4c157be0 0x7f4b4c0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4affd700 1 --2- 172.21.15.31:0/3743085243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b4c157770 0x7f4b4c0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4b7fe700 1 -- 172.21.15.31:0/3743085243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b4c157770 msgr2=0x7f4b4c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4b7fe700 1 --2- 172.21.15.31:0/3743085243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b4c157770 0x7f4b4c0b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4b7fe700 1 -- 172.21.15.31:0/3743085243 >> v2:172.21.15.62:3300/0 conn(0x7f4b4c157be0 msgr2=0x7f4b4c0b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4b7fe700 1 --2- 172.21.15.31:0/3743085243 >> v2:172.21.15.62:3300/0 conn(0x7f4b4c157be0 0x7f4b4c0b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4b7fe700 1 -- 172.21.15.31:0/3743085243 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c1586a0 con 0x7f4b4c1573a0 2023-10-06T05:20:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4b7fe700 1 --2- 172.21.15.31:0/3743085243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 0x7f4b4c0acb70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c06b3a0 tx=0x7f4b3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64553d329ef63742 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/3743085243 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00b860 con 0x7f4b4c1573a0 2023-10-06T05:20:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/3743085243 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c00b9c0 con 0x7f4b4c1573a0 2023-10-06T05:20:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.397+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/3743085243 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00bd60 con 0x7f4b4c1573a0 2023-10-06T05:20:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.398+0000 7f4b521ff700 1 -- 172.21.15.31:0/3743085243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 msgr2=0x7f4b4c0acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.398+0000 7f4b521ff700 1 --2- 172.21.15.31:0/3743085243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 0x7f4b4c0acb70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c06b3a0 tx=0x7f4b3c009250 comp rx=0 tx=0).stop 2023-10-06T05:20:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.398+0000 7f4b521ff700 1 -- 172.21.15.31:0/3743085243 shutdown_connections 2023-10-06T05:20:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.398+0000 7f4b521ff700 1 --2- 172.21.15.31:0/3743085243 >> v2:172.21.15.62:3300/0 conn(0x7f4b4c157be0 0x7f4b4c0b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.398+0000 7f4b521ff700 1 --2- 172.21.15.31:0/3743085243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 0x7f4b4c0acb70 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.398+0000 7f4b521ff700 1 --2- 172.21.15.31:0/3743085243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b4c157770 0x7f4b4c0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.398+0000 7f4b521ff700 1 -- 172.21.15.31:0/3743085243 >> 172.21.15.31:0/3743085243 conn(0x7f4b4c067560 msgr2=0x7f4b4c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.398+0000 7f4b521ff700 1 -- 172.21.15.31:0/3743085243 shutdown_connections 2023-10-06T05:20:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.398+0000 7f4b521ff700 1 -- 172.21.15.31:0/3743085243 wait complete. 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.400+0000 7f4b521ff700 1 Processor -- start 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.400+0000 7f4b521ff700 1 -- start start 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b521ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 0x7f4b4c169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b521ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b4c157770 0x7f4b4c15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b521ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b4c157be0 0x7f4b4c146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b521ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c16aa50 con 0x7f4b4c1573a0 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b521ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c169fb0 con 0x7f4b4c157be0 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b521ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c15c1f0 con 0x7f4b4c157770 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b4b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 0x7f4b4c169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b4b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 0x7f4b4c169a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49464/0 (socket says 172.21.15.31:49464) 2023-10-06T05:20:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b4b7fe700 1 -- 172.21.15.31:0/3664486313 learned_addr learned my addr 172.21.15.31:0/3664486313 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b4affd700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b4c157770 0x7f4b4c15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b4bfff700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b4c157be0 0x7f4b4c146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b4b7fe700 1 -- 172.21.15.31:0/3664486313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b4c157770 msgr2=0x7f4b4c15bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.401+0000 7f4b4b7fe700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b4c157770 0x7f4b4c15bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.402+0000 7f4b4b7fe700 1 -- 172.21.15.31:0/3664486313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b4c157be0 msgr2=0x7f4b4c146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.402+0000 7f4b4b7fe700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b4c157be0 0x7f4b4c146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.402+0000 7f4b4b7fe700 1 -- 172.21.15.31:0/3664486313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c0094f0 con 0x7f4b4c1573a0 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.402+0000 7f4b4b7fe700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 0x7f4b4c169a70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c064290 tx=0x7f4b3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.402+0000 7f4b337fe700 1 -- 172.21.15.31:0/3664486313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c011dc0 con 0x7f4b4c1573a0 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.402+0000 7f4b337fe700 1 -- 172.21.15.31:0/3664486313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c024440 con 0x7f4b4c1573a0 2023-10-06T05:20:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.402+0000 7f4b521ff700 1 -- 172.21.15.31:0/3664486313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c16d2e0 con 0x7f4b4c1573a0 2023-10-06T05:20:54.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.402+0000 7f4b337fe700 1 -- 172.21.15.31:0/3664486313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c024800 con 0x7f4b4c1573a0 2023-10-06T05:20:54.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.402+0000 7f4b521ff700 1 -- 172.21.15.31:0/3664486313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4c16a490 con 0x7f4b4c1573a0 2023-10-06T05:20:54.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.403+0000 7f4b521ff700 1 -- 172.21.15.31:0/3664486313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4c16d0b0 con 0x7f4b4c1573a0 2023-10-06T05:20:54.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.404+0000 7f4b337fe700 1 -- 172.21.15.31:0/3664486313 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f4b3c01a420 con 0x7f4b4c1573a0 2023-10-06T05:20:54.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.405+0000 7f4b337fe700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4b38067170 0x7f4b38069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:54.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.405+0000 7f4b337fe700 1 -- 172.21.15.31:0/3664486313 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f4b3c086860 con 0x7f4b4c1573a0 2023-10-06T05:20:54.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.405+0000 7f4b4affd700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4b38067170 0x7f4b38069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:54.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.406+0000 7f4b4affd700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4b38067170 0x7f4b38069620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0acf30 tx=0x7f4b40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:54.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.410+0000 7f4b337fe700 1 -- 172.21.15.31:0/3664486313 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b3c053650 con 0x7f4b4c1573a0 2023-10-06T05:20:54.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.643+0000 7f4b521ff700 1 -- 172.21.15.31:0/3664486313 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000234639", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b4c04c380 con 0x7f4b38067170 2023-10-06T05:20:54.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.665+0000 7f4b337fe700 1 -- 172.21.15.31:0/3664486313 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4b4c04c380 con 0x7f4b38067170 2023-10-06T05:20:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.667+0000 7f4b521ff700 1 -- 172.21.15.31:0/3664486313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4b38067170 msgr2=0x7f4b38069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.667+0000 7f4b521ff700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4b38067170 0x7f4b38069620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0acf30 tx=0x7f4b40008040 comp rx=0 tx=0).stop 2023-10-06T05:20:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.668+0000 7f4b521ff700 1 -- 172.21.15.31:0/3664486313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 msgr2=0x7f4b4c169a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.668+0000 7f4b521ff700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 0x7f4b4c169a70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c064290 tx=0x7f4b3c017000 comp rx=0 tx=0).stop 2023-10-06T05:20:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.668+0000 7f4b521ff700 1 -- 172.21.15.31:0/3664486313 shutdown_connections 2023-10-06T05:20:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.668+0000 7f4b521ff700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b4c157be0 0x7f4b4c146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.668+0000 7f4b521ff700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4b38067170 0x7f4b38069620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.668+0000 7f4b521ff700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b4c1573a0 0x7f4b4c169a70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.668+0000 7f4b521ff700 1 --2- 172.21.15.31:0/3664486313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b4c157770 0x7f4b4c15bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.668+0000 7f4b521ff700 1 -- 172.21.15.31:0/3664486313 >> 172.21.15.31:0/3664486313 conn(0x7f4b4c067560 msgr2=0x7f4b4c0a9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.668+0000 7f4b521ff700 1 -- 172.21.15.31:0/3664486313 shutdown_connections 2023-10-06T05:20:54.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:54.668+0000 7f4b521ff700 1 -- 172.21.15.31:0/3664486313 wait complete. 2023-10-06T05:20:54.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:54.689 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T05:20:54.783 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T05:20:54.784 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:20:54.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:54.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:20:54.999 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:20:54.999 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:20:54.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:54.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:20:55.104 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:20:55.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:55.104 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:20:55.156 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:20:55.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:20:55.157 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:20:55.256 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:20:55.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:20:55.257 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:20:55.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.497+0000 7fef03bff700 1 Processor -- start 2023-10-06T05:20:55.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.497+0000 7fef03bff700 1 -- start start 2023-10-06T05:20:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef03bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feefc13a5b0 0x7feefc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef03bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feefc06d980 0x7feefc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef03bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc06ddc0 con 0x7feefc13a1e0 2023-10-06T05:20:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef03bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc0c71b0 con 0x7feefc06d980 2023-10-06T05:20:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef03bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0c7490 con 0x7feefc13a5b0 2023-10-06T05:20:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef0219c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feefc06d980 0x7feefc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef0199b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7feefc13a1e0 0x7feefc0c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41188/0 (socket says 172.21.15.31:41188) 2023-10-06T05:20:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef0199b700 1 -- 172.21.15.31:0/3120801059 learned_addr learned my addr 172.21.15.31:0/3120801059 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef0119a700 1 --2- 172.21.15.31:0/3120801059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feefc13a5b0 0x7feefc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef00999700 1 -- 172.21.15.31:0/3120801059 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630873562 0 0) 0x7feefc06ddc0 con 0x7feefc13a1e0 2023-10-06T05:20:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef0219c700 1 -- 172.21.15.31:0/3120801059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feefc13a5b0 msgr2=0x7feefc06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef0219c700 1 --2- 172.21.15.31:0/3120801059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feefc13a5b0 0x7feefc06d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef0219c700 1 -- 172.21.15.31:0/3120801059 >> v1:172.21.15.31:6789/0 conn(0x7feefc13a1e0 legacy=0x7feefc0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.499+0000 7fef0219c700 1 -- 172.21.15.31:0/3120801059 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0c7770 con 0x7feefc06d980 2023-10-06T05:20:55.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.500+0000 7fef0219c700 1 --2- 172.21.15.31:0/3120801059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc06d980 0x7feefc069560 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feef4009e60 tx=0x7feef4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaec1838b8a6a658 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.500+0000 7fef00999700 1 -- 172.21.15.31:0/3120801059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400e070 con 0x7feefc06d980 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.500+0000 7fef00999700 1 -- 172.21.15.31:0/3120801059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef400bb60 con 0x7feefc06d980 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.500+0000 7fef00999700 1 -- 172.21.15.31:0/3120801059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4012430 con 0x7feefc06d980 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.500+0000 7fef03bff700 1 -- 172.21.15.31:0/3120801059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc06d980 msgr2=0x7feefc069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.500+0000 7fef03bff700 1 --2- 172.21.15.31:0/3120801059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc06d980 0x7feefc069560 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feef4009e60 tx=0x7feef4009780 comp rx=0 tx=0).stop 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.501+0000 7fef03bff700 1 -- 172.21.15.31:0/3120801059 shutdown_connections 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.501+0000 7fef03bff700 1 --2- 172.21.15.31:0/3120801059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc06d980 0x7feefc069560 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.501+0000 7fef03bff700 1 --2- 172.21.15.31:0/3120801059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feefc13a5b0 0x7feefc06d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.501+0000 7fef03bff700 1 -- 172.21.15.31:0/3120801059 >> 172.21.15.31:0/3120801059 conn(0x7feefc0cc0b0 msgr2=0x7feefc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.501+0000 7fef03bff700 1 -- 172.21.15.31:0/3120801059 shutdown_connections 2023-10-06T05:20:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.501+0000 7fef03bff700 1 -- 172.21.15.31:0/3120801059 wait complete. 2023-10-06T05:20:55.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.502+0000 7fef03bff700 1 Processor -- start 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.502+0000 7fef03bff700 1 -- start start 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.503+0000 7fef03bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feefc06d980 0x7feefc16cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.503+0000 7fef03bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc13a1e0 0x7feefc14e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.503+0000 7fef03bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc13a5b0 0x7feefc0c1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.503+0000 7fef03bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0ca270 con 0x7feefc13a1e0 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.503+0000 7fef03bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc16d340 con 0x7feefc13a5b0 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.503+0000 7fef03bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc16d620 con 0x7feefc06d980 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc13a1e0 0x7feefc14e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc13a1e0 0x7feefc14e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49472/0 (socket says 172.21.15.31:49472) 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0119a700 1 -- 172.21.15.31:0/340728275 learned_addr learned my addr 172.21.15.31:0/340728275 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0199b700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feefc06d980 0x7feefc16cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0219c700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc13a5b0 0x7feefc0c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0119a700 1 -- 172.21.15.31:0/340728275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feefc06d980 msgr2=0x7feefc16cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0119a700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feefc06d980 0x7feefc16cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0119a700 1 -- 172.21.15.31:0/340728275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc13a5b0 msgr2=0x7feefc0c1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0119a700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc13a5b0 0x7feefc0c1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0119a700 1 -- 172.21.15.31:0/340728275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef4009a20 con 0x7feefc13a1e0 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.504+0000 7fef0119a700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc13a1e0 0x7feefc14e0c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7feeec00e930 tx=0x7feeec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.505+0000 7feef1ffb700 1 -- 172.21.15.31:0/340728275 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec013680 con 0x7feefc13a1e0 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.505+0000 7feef1ffb700 1 -- 172.21.15.31:0/340728275 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec0137e0 con 0x7feefc13a1e0 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.505+0000 7feef1ffb700 1 -- 172.21.15.31:0/340728275 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec013b80 con 0x7feefc13a1e0 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.505+0000 7fef03bff700 1 -- 172.21.15.31:0/340728275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc16d900 con 0x7feefc13a1e0 2023-10-06T05:20:55.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.505+0000 7fef03bff700 1 -- 172.21.15.31:0/340728275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc14e600 con 0x7feefc13a1e0 2023-10-06T05:20:55.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.506+0000 7fef03bff700 1 -- 172.21.15.31:0/340728275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc0ca3e0 con 0x7feefc13a1e0 2023-10-06T05:20:55.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.507+0000 7feef1ffb700 1 -- 172.21.15.31:0/340728275 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7feeec00c3e0 con 0x7feefc13a1e0 2023-10-06T05:20:55.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.508+0000 7feef1ffb700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7feee0067170 0x7feee0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.508+0000 7feef1ffb700 1 -- 172.21.15.31:0/340728275 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7feeec084bb0 con 0x7feefc13a1e0 2023-10-06T05:20:55.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.508+0000 7fef0199b700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7feee0067170 0x7feee0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.508+0000 7fef0199b700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7feee0067170 0x7feee0069620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feef8001360 tx=0x7feef8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.512+0000 7feef1ffb700 1 -- 172.21.15.31:0/340728275 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feeec051900 con 0x7feefc13a1e0 2023-10-06T05:20:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.756+0000 7fef03bff700 1 -- 172.21.15.31:0/340728275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feefc04c380 con 0x7feefc13a1e0 2023-10-06T05:20:55.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.756+0000 7feef1ffb700 1 -- 172.21.15.31:0/340728275 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4595 (secure 0 0 0) 0x7feeec054f20 con 0x7feefc13a1e0 2023-10-06T05:20:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:55.758 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5743,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2430199885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2430199885},{"type":"v1","addr":"172.21.15.31:6837","nonce":2430199885}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5746,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3762519665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3762519665},{"type":"v1","addr":"172.21.15.62:6834","nonce":3762519665}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5844,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2776536460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2776536460},{"type":"v1","addr":"172.21.15.31:6836","nonce":2776536460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:36.742745+0000","modified":"2023-10-06T05:20:40.276711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5853":{"gid":5853,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/982526111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":982526111},{"type":"v1","addr":"172.21.15.62:6835","nonce":982526111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:20:55.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 -- 172.21.15.31:0/340728275 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7feee0067170 msgr2=0x7feee0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7feee0067170 0x7feee0069620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feef8001360 tx=0x7feef8009040 comp rx=0 tx=0).stop 2023-10-06T05:20:55.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 -- 172.21.15.31:0/340728275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc13a1e0 msgr2=0x7feefc14e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc13a1e0 0x7feefc14e0c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7feeec00e930 tx=0x7feeec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 -- 172.21.15.31:0/340728275 shutdown_connections 2023-10-06T05:20:55.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc13a5b0 0x7feefc0c1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feefc06d980 0x7feefc16cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7feee0067170 0x7feee0069620 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 --2- 172.21.15.31:0/340728275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feefc13a1e0 0x7feefc14e0c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 -- 172.21.15.31:0/340728275 >> 172.21.15.31:0/340728275 conn(0x7feefc0cc0b0 msgr2=0x7feefc0c6920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 -- 172.21.15.31:0/340728275 shutdown_connections 2023-10-06T05:20:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.759+0000 7fef03bff700 1 -- 172.21.15.31:0/340728275 wait complete. 2023-10-06T05:20:55.762 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 36 2023-10-06T05:20:55.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.974+0000 7fb1b33ff700 1 Processor -- start 2023-10-06T05:20:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.974+0000 7fb1b33ff700 1 -- start start 2023-10-06T05:20:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.975+0000 7fb1b33ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1ac0c7b60 0x7fb1ac069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.975+0000 7fb1b33ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb1ac0aee70 0x7fb1ac13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.975+0000 7fb1b33ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ac0ca5b0 con 0x7fb1ac0c7f30 2023-10-06T05:20:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.975+0000 7fb1b33ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac0af2d0 con 0x7fb1ac0c7b60 2023-10-06T05:20:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.975+0000 7fb1b33ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac06f740 con 0x7fb1ac0aee70 2023-10-06T05:20:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.976+0000 7fb1b119b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1ac0c7b60 0x7fb1ac069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.976+0000 7fb1b119b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1ac0c7b60 0x7fb1ac069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49484/0 (socket says 172.21.15.31:49484) 2023-10-06T05:20:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.976+0000 7fb1b119b700 1 -- 172.21.15.31:0/2346087161 learned_addr learned my addr 172.21.15.31:0/2346087161 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.976+0000 7fb1b199c700 1 --2- 172.21.15.31:0/2346087161 >> v2:172.21.15.62:3300/0 conn(0x7fb1ac0aee70 0x7fb1ac13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.976+0000 7fb1b119b700 1 -- 172.21.15.31:0/2346087161 >> v2:172.21.15.62:3300/0 conn(0x7fb1ac0aee70 msgr2=0x7fb1ac13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.976+0000 7fb1b119b700 1 --2- 172.21.15.31:0/2346087161 >> v2:172.21.15.62:3300/0 conn(0x7fb1ac0aee70 0x7fb1ac13f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.976+0000 7fb1b119b700 1 -- 172.21.15.31:0/2346087161 >> v1:172.21.15.62:6789/0 conn(0x7fb1ac0c7f30 legacy=0x7fb1ac06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.976+0000 7fb1b119b700 1 -- 172.21.15.31:0/2346087161 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac0c9c80 con 0x7fb1ac0c7b60 2023-10-06T05:20:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.976+0000 7fb1b119b700 1 --2- 172.21.15.31:0/2346087161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0c7b60 0x7fb1ac069880 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac1569d0 tx=0x7fb1a8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91975c828a4c294 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.976+0000 7fb1a3fff700 1 -- 172.21.15.31:0/2346087161 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8005c10 con 0x7fb1ac0c7b60 2023-10-06T05:20:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.977+0000 7fb1a3fff700 1 -- 172.21.15.31:0/2346087161 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a8005d70 con 0x7fb1ac0c7b60 2023-10-06T05:20:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.977+0000 7fb1a3fff700 1 -- 172.21.15.31:0/2346087161 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8006260 con 0x7fb1ac0c7b60 2023-10-06T05:20:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.977+0000 7fb1b33ff700 1 -- 172.21.15.31:0/2346087161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0c7b60 msgr2=0x7fb1ac069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.977+0000 7fb1b33ff700 1 --2- 172.21.15.31:0/2346087161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0c7b60 0x7fb1ac069880 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac1569d0 tx=0x7fb1a8009220 comp rx=0 tx=0).stop 2023-10-06T05:20:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.977+0000 7fb1b33ff700 1 -- 172.21.15.31:0/2346087161 shutdown_connections 2023-10-06T05:20:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.977+0000 7fb1b33ff700 1 --2- 172.21.15.31:0/2346087161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0c7b60 0x7fb1ac069880 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.977+0000 7fb1b33ff700 1 --2- 172.21.15.31:0/2346087161 >> v2:172.21.15.62:3300/0 conn(0x7fb1ac0aee70 0x7fb1ac13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.977+0000 7fb1b33ff700 1 -- 172.21.15.31:0/2346087161 >> 172.21.15.31:0/2346087161 conn(0x7fb1ac0cc0b0 msgr2=0x7fb1ac0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.977+0000 7fb1b33ff700 1 -- 172.21.15.31:0/2346087161 shutdown_connections 2023-10-06T05:20:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.977+0000 7fb1b33ff700 1 -- 172.21.15.31:0/2346087161 wait complete. 2023-10-06T05:20:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.979+0000 7fb1b33ff700 1 Processor -- start 2023-10-06T05:20:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.979+0000 7fb1b33ff700 1 -- start start 2023-10-06T05:20:55.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b33ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0aee70 0x7fb1ac0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1ac0c7b60 0x7fb1ac13c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b33ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1ac0c7f30 0x7fb1ac13e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b33ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac13a5b0 con 0x7fb1ac0aee70 2023-10-06T05:20:55.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b33ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac0adb80 con 0x7fb1ac0c7f30 2023-10-06T05:20:55.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac13a7e0 con 0x7fb1ac0c7b60 2023-10-06T05:20:55.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0aee70 0x7fb1ac0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0aee70 0x7fb1ac0ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49488/0 (socket says 172.21.15.31:49488) 2023-10-06T05:20:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b119b700 1 -- 172.21.15.31:0/1971862066 learned_addr learned my addr 172.21.15.31:0/1971862066 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b199c700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1ac0c7f30 0x7fb1ac13e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.980+0000 7fb1b099a700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1ac0c7b60 0x7fb1ac13c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1b119b700 1 -- 172.21.15.31:0/1971862066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1ac0c7b60 msgr2=0x7fb1ac13c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1b119b700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1ac0c7b60 0x7fb1ac13c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1b119b700 1 -- 172.21.15.31:0/1971862066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1ac0c7f30 msgr2=0x7fb1ac13e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1b119b700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1ac0c7f30 0x7fb1ac13e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1b119b700 1 -- 172.21.15.31:0/1971862066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8009480 con 0x7fb1ac0aee70 2023-10-06T05:20:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1b119b700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0aee70 0x7fb1ac0ae170 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800b0e0 tx=0x7fb1a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1a17fa700 1 -- 172.21.15.31:0/1971862066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800b850 con 0x7fb1ac0aee70 2023-10-06T05:20:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1a17fa700 1 -- 172.21.15.31:0/1971862066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a8020070 con 0x7fb1ac0aee70 2023-10-06T05:20:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1b33ff700 1 -- 172.21.15.31:0/1971862066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac16d550 con 0x7fb1ac0aee70 2023-10-06T05:20:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1a17fa700 1 -- 172.21.15.31:0/1971862066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800bbe0 con 0x7fb1ac0aee70 2023-10-06T05:20:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.981+0000 7fb1b33ff700 1 -- 172.21.15.31:0/1971862066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac16db10 con 0x7fb1ac0aee70 2023-10-06T05:20:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.983+0000 7fb1b33ff700 1 -- 172.21.15.31:0/1971862066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac04c380 con 0x7fb1ac0aee70 2023-10-06T05:20:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.983+0000 7fb1a17fa700 1 -- 172.21.15.31:0/1971862066 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fb1a8022030 con 0x7fb1ac0aee70 2023-10-06T05:20:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.984+0000 7fb1a17fa700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb190067170 0x7fb190069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.984+0000 7fb1a17fa700 1 -- 172.21.15.31:0/1971862066 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fb1a8086860 con 0x7fb1ac0aee70 2023-10-06T05:20:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.985+0000 7fb1b099a700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb190067170 0x7fb190069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.985+0000 7fb1b099a700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb190067170 0x7fb190069620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0c24e0 tx=0x7fb19c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:55.989+0000 7fb1a17fa700 1 -- 172.21.15.31:0/1971862066 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1a8053580 con 0x7fb1ac0aee70 2023-10-06T05:20:56.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.194+0000 7fb1b33ff700 1 -- 172.21.15.31:0/1971862066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1ac0c9f60 con 0x7fb1ac0aee70 2023-10-06T05:20:56.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.195+0000 7fb1a17fa700 1 -- 172.21.15.31:0/1971862066 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+17156 (secure 0 0 0) 0x7fb1a8056ba0 con 0x7fb1ac0aee70 2023-10-06T05:20:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:56.197 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":59,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:20:50.325408+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:34.970939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:35.729829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":3}]},{"pgid":"10.23","mappings":[{"from":7,"to":3}]},{"pgid":"10.2f","mappings":[{"from":1,"to":3}]},{"pgid":"10.37","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:56.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 -- 172.21.15.31:0/1971862066 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb190067170 msgr2=0x7fb190069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb190067170 0x7fb190069620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0c24e0 tx=0x7fb19c009040 comp rx=0 tx=0).stop 2023-10-06T05:20:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 -- 172.21.15.31:0/1971862066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0aee70 msgr2=0x7fb1ac0ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0aee70 0x7fb1ac0ae170 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800b0e0 tx=0x7fb1a8016000 comp rx=0 tx=0).stop 2023-10-06T05:20:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 -- 172.21.15.31:0/1971862066 shutdown_connections 2023-10-06T05:20:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1ac0c7f30 0x7fb1ac13e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb190067170 0x7fb190069620 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0aee70 0x7fb1ac0ae170 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 --2- 172.21.15.31:0/1971862066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1ac0c7b60 0x7fb1ac13c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 -- 172.21.15.31:0/1971862066 >> 172.21.15.31:0/1971862066 conn(0x7fb1ac0cc0b0 msgr2=0x7fb1ac0c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 -- 172.21.15.31:0/1971862066 shutdown_connections 2023-10-06T05:20:56.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.197+0000 7fb1b33ff700 1 -- 172.21.15.31:0/1971862066 wait complete. 2023-10-06T05:20:56.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:20:34.970939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:20:35.729829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:20:56.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:20:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.435+0000 7fe08ddff700 1 Processor -- start 2023-10-06T05:20:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.435+0000 7fe08ddff700 1 -- start start 2023-10-06T05:20:56.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.437+0000 7fe08ddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0880aef00 0x7fe0880c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.437+0000 7fe08ddff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe088068250 con 0x7fe0880c7190 2023-10-06T05:20:56.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.437+0000 7fe08ddff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08806ac80 con 0x7fe0880c7560 2023-10-06T05:20:56.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.437+0000 7fe08ddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe088067f20 con 0x7fe0880aef00 2023-10-06T05:20:56.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.437+0000 7fe0877fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe0880c7190 0x7fe0880af7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41202/0 (socket says 172.21.15.31:41202) 2023-10-06T05:20:56.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.437+0000 7fe0877fe700 1 -- 172.21.15.31:0/3558709008 learned_addr learned my addr 172.21.15.31:0/3558709008 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:56.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.437+0000 7fe086ffd700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0880aef00 0x7fe0880c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.437+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337023199 0 0) 0x7fe088068250 con 0x7fe0880c7190 2023-10-06T05:20:56.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.437+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe070003570 con 0x7fe0880c7190 2023-10-06T05:20:56.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480650480 0 0) 0x7fe08806ac80 con 0x7fe0880c7560 2023-10-06T05:20:56.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe088068250 con 0x7fe0880c7560 2023-10-06T05:20:56.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 165007723 0 0) 0x7fe070003570 con 0x7fe0880c7190 2023-10-06T05:20:56.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0880aef00 msgr2=0x7fe0880c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:56.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0880aef00 0x7fe0880c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 >> v1:172.21.15.62:6789/0 conn(0x7fe0880c7560 legacy=0x7fe0880686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08806aed0 con 0x7fe0880c7190 2023-10-06T05:20:56.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fe078001470 con 0x7fe0880c7190 2023-10-06T05:20:56.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 >> v1:172.21.15.31:6789/0 conn(0x7fe0880c7190 legacy=0x7fe0880af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe070003c20 0x7fe070004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe070004810 0x7fe070004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0880aef00 0x7fe0700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe088067f20 con 0x7fe070003c20 2023-10-06T05:20:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07000bb40 con 0x7fe070004810 2023-10-06T05:20:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07000bd00 con 0x7fe0880aef00 2023-10-06T05:20:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0877fe700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe070003c20 0x7fe070004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe086ffd700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe070004810 0x7fe070004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe087fff700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0880aef00 0x7fe0700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0877fe700 1 -- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0880aef00 msgr2=0x7fe0700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0877fe700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0880aef00 0x7fe0700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0877fe700 1 -- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe070004810 msgr2=0x7fe070004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0877fe700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe070004810 0x7fe070004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe0877fe700 1 -- 172.21.15.31:0/3558709008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe070003570 con 0x7fe070003c20 2023-10-06T05:20:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.438+0000 7fe086ffd700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe070004810 0x7fe070004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.439+0000 7fe087fff700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0880aef00 0x7fe0700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.439+0000 7fe0877fe700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe070003c20 0x7fe070004010 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe07800c2d0 tx=0x7fe078001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6974606388f87bad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.439+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07800cc00 con 0x7fe070003c20 2023-10-06T05:20:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.439+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07800cd60 con 0x7fe070003c20 2023-10-06T05:20:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.439+0000 7fe0867fc700 1 -- 172.21.15.31:0/3558709008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe078005b70 con 0x7fe070003c20 2023-10-06T05:20:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.439+0000 7fe08ddff700 1 -- 172.21.15.31:0/3558709008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe070003c20 msgr2=0x7fe070004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.439+0000 7fe08ddff700 1 --2- 172.21.15.31:0/3558709008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe070003c20 0x7fe070004010 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe07800c2d0 tx=0x7fe078001e10 comp rx=0 tx=0).stop 2023-10-06T05:20:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.439+0000 7fe087fff700 1 -- 172.21.15.31:0/3558709008 reap_dead start 2023-10-06T05:20:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.439+0000 7fe08ddff700 1 -- 172.21.15.31:0/3558709008 shutdown_connections 2023-10-06T05:20:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.439+0000 7fe08ddff700 1 -- 172.21.15.31:0/3558709008 >> 172.21.15.31:0/3558709008 conn(0x7fe0880cd3d0 msgr2=0x7fe0880c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.440+0000 7fe08ddff700 1 -- 172.21.15.31:0/3558709008 shutdown_connections 2023-10-06T05:20:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.440+0000 7fe08ddff700 1 -- 172.21.15.31:0/3558709008 wait complete. 2023-10-06T05:20:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.441+0000 7fe08ddff700 1 Processor -- start 2023-10-06T05:20:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.441+0000 7fe08ddff700 1 -- start start 2023-10-06T05:20:56.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.442+0000 7fe08ddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe070003c20 0x7fe08815d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.442+0000 7fe08ddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0880c5360 0x7fe08816c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.442+0000 7fe08ddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe08816c9d0 0x7fe08814e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.442+0000 7fe08ddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880c57d0 con 0x7fe08816c9d0 2023-10-06T05:20:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.442+0000 7fe08ddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880c5a70 con 0x7fe0880c5360 2023-10-06T05:20:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.442+0000 7fe08ddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880c5d50 con 0x7fe070003c20 2023-10-06T05:20:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.442+0000 7fe087fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe08816c9d0 0x7fe08814e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe087fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe08816c9d0 0x7fe08814e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49508/0 (socket says 172.21.15.31:49508) 2023-10-06T05:20:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe087fff700 1 -- 172.21.15.31:0/3218051328 learned_addr learned my addr 172.21.15.31:0/3218051328 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe086ffd700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0880c5360 0x7fe08816c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe0877fe700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe070003c20 0x7fe08815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe087fff700 1 -- 172.21.15.31:0/3218051328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe070003c20 msgr2=0x7fe08815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe087fff700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe070003c20 0x7fe08815d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe087fff700 1 -- 172.21.15.31:0/3218051328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0880c5360 msgr2=0x7fe08816c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe087fff700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0880c5360 0x7fe08816c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe087fff700 1 -- 172.21.15.31:0/3218051328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0880c6030 con 0x7fe08816c9d0 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe087fff700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe08816c9d0 0x7fe08814e460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe0880c7930 tx=0x7fe080009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe06f7fe700 1 -- 172.21.15.31:0/3218051328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe080003300 con 0x7fe08816c9d0 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe06f7fe700 1 -- 172.21.15.31:0/3218051328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe080003460 con 0x7fe08816c9d0 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe06f7fe700 1 -- 172.21.15.31:0/3218051328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe080006df0 con 0x7fe08816c9d0 2023-10-06T05:20:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.443+0000 7fe08ddff700 1 -- 172.21.15.31:0/3218051328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0880c6310 con 0x7fe08816c9d0 2023-10-06T05:20:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.444+0000 7fe08ddff700 1 -- 172.21.15.31:0/3218051328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08814e9a0 con 0x7fe08816c9d0 2023-10-06T05:20:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.445+0000 7fe08ddff700 1 -- 172.21.15.31:0/3218051328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08804c380 con 0x7fe08816c9d0 2023-10-06T05:20:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.446+0000 7fe06f7fe700 1 -- 172.21.15.31:0/3218051328 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fe0800018f0 con 0x7fe08816c9d0 2023-10-06T05:20:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.446+0000 7fe06f7fe700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe074067170 0x7fe074069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.446+0000 7fe06f7fe700 1 -- 172.21.15.31:0/3218051328 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fe080084110 con 0x7fe08816c9d0 2023-10-06T05:20:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.447+0000 7fe0877fe700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe074067170 0x7fe074069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.447+0000 7fe0877fe700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe074067170 0x7fe074069620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe088155c20 tx=0x7fe078015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.451+0000 7fe06f7fe700 1 -- 172.21.15.31:0/3218051328 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe080050da0 con 0x7fe08816c9d0 2023-10-06T05:20:56.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.658+0000 7fe08ddff700 1 -- 172.21.15.31:0/3218051328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe08806b1b0 con 0x7fe08816c9d0 2023-10-06T05:20:56.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.658+0000 7fe06f7fe700 1 -- 172.21.15.31:0/3218051328 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe0800180a0 con 0x7fe08816c9d0 2023-10-06T05:20:56.660 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:20:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.660+0000 7fe08ddff700 1 -- 172.21.15.31:0/3218051328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe074067170 msgr2=0x7fe074069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.660+0000 7fe08ddff700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe074067170 0x7fe074069620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe088155c20 tx=0x7fe078015000 comp rx=0 tx=0).stop 2023-10-06T05:20:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.660+0000 7fe08ddff700 1 -- 172.21.15.31:0/3218051328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe08816c9d0 msgr2=0x7fe08814e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.660+0000 7fe08ddff700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe08816c9d0 0x7fe08814e460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe0880c7930 tx=0x7fe080009040 comp rx=0 tx=0).stop 2023-10-06T05:20:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.660+0000 7fe08ddff700 1 -- 172.21.15.31:0/3218051328 shutdown_connections 2023-10-06T05:20:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.660+0000 7fe08ddff700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe074067170 0x7fe074069620 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.660+0000 7fe08ddff700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe08816c9d0 0x7fe08814e460 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.660+0000 7fe08ddff700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe070003c20 0x7fe08815d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.660+0000 7fe08ddff700 1 --2- 172.21.15.31:0/3218051328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0880c5360 0x7fe08816c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.660+0000 7fe08ddff700 1 -- 172.21.15.31:0/3218051328 >> 172.21.15.31:0/3218051328 conn(0x7fe0880cd3d0 msgr2=0x7fe0881588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:56.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.661+0000 7fe08ddff700 1 -- 172.21.15.31:0/3218051328 shutdown_connections 2023-10-06T05:20:56.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.661+0000 7fe08ddff700 1 -- 172.21.15.31:0/3218051328 wait complete. 2023-10-06T05:20:56.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:20:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.876+0000 7fa39cfff700 1 Processor -- start 2023-10-06T05:20:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.876+0000 7fa39cfff700 1 -- start start 2023-10-06T05:20:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.877+0000 7fa39cfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa39815ca40 0x7fa398068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.877+0000 7fa39cfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa39806a300 0x7fa3980c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.877+0000 7fa39cfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3980c91b0 con 0x7fa3980c8d70 2023-10-06T05:20:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.877+0000 7fa39cfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3980cabc0 con 0x7fa39806a300 2023-10-06T05:20:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.877+0000 7fa39cfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa398068bb0 con 0x7fa39815ca40 2023-10-06T05:20:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.878+0000 7fa395d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa39806a300 0x7fa3980c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.878+0000 7fa395d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa39806a300 0x7fa3980c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49510/0 (socket says 172.21.15.31:49510) 2023-10-06T05:20:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.878+0000 7fa395d9b700 1 -- 172.21.15.31:0/750766707 learned_addr learned my addr 172.21.15.31:0/750766707 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.878+0000 7fa39659c700 1 --2- 172.21.15.31:0/750766707 >> v2:172.21.15.62:3300/0 conn(0x7fa39815ca40 0x7fa398068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.878+0000 7fa395d9b700 1 -- 172.21.15.31:0/750766707 >> v2:172.21.15.62:3300/0 conn(0x7fa39815ca40 msgr2=0x7fa398068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.878+0000 7fa395d9b700 1 --2- 172.21.15.31:0/750766707 >> v2:172.21.15.62:3300/0 conn(0x7fa39815ca40 0x7fa398068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.878+0000 7fa395d9b700 1 -- 172.21.15.31:0/750766707 >> v1:172.21.15.62:6789/0 conn(0x7fa3980c8d70 legacy=0x7fa39806cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.878+0000 7fa395d9b700 1 -- 172.21.15.31:0/750766707 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa398068da0 con 0x7fa39806a300 2023-10-06T05:20:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.879+0000 7fa395d9b700 1 --2- 172.21.15.31:0/750766707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa39806a300 0x7fa3980c8830 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa398067f80 tx=0x7fa38c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946caa9802b77bba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.879+0000 7fa39559a700 1 -- 172.21.15.31:0/750766707 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c009e70 con 0x7fa39806a300 2023-10-06T05:20:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.879+0000 7fa39559a700 1 -- 172.21.15.31:0/750766707 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c00b860 con 0x7fa39806a300 2023-10-06T05:20:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.879+0000 7fa39559a700 1 -- 172.21.15.31:0/750766707 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c00bc00 con 0x7fa39806a300 2023-10-06T05:20:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.879+0000 7fa39cfff700 1 -- 172.21.15.31:0/750766707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa39806a300 msgr2=0x7fa3980c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.879+0000 7fa39cfff700 1 --2- 172.21.15.31:0/750766707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa39806a300 0x7fa3980c8830 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa398067f80 tx=0x7fa38c009250 comp rx=0 tx=0).stop 2023-10-06T05:20:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.880+0000 7fa39cfff700 1 -- 172.21.15.31:0/750766707 shutdown_connections 2023-10-06T05:20:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.880+0000 7fa39cfff700 1 --2- 172.21.15.31:0/750766707 >> v2:172.21.15.62:3300/0 conn(0x7fa39815ca40 0x7fa398068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.880+0000 7fa39cfff700 1 --2- 172.21.15.31:0/750766707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa39806a300 0x7fa3980c8830 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.880+0000 7fa39cfff700 1 -- 172.21.15.31:0/750766707 >> 172.21.15.31:0/750766707 conn(0x7fa3980ce290 msgr2=0x7fa3980ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.880+0000 7fa39cfff700 1 -- 172.21.15.31:0/750766707 shutdown_connections 2023-10-06T05:20:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.880+0000 7fa39cfff700 1 -- 172.21.15.31:0/750766707 wait complete. 2023-10-06T05:20:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.881+0000 7fa39cfff700 1 Processor -- start 2023-10-06T05:20:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.881+0000 7fa39cfff700 1 -- start start 2023-10-06T05:20:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.882+0000 7fa39cfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39806a300 0x7fa3980b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.882+0000 7fa39cfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3980c8d70 0x7fa3980c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.882+0000 7fa39cfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa39815ca40 0x7fa3981577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.882+0000 7fa39cfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980c6960 con 0x7fa3980c8d70 2023-10-06T05:20:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.882+0000 7fa39cfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980c6350 con 0x7fa39815ca40 2023-10-06T05:20:56.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.882+0000 7fa39cfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980c4a40 con 0x7fa39806a300 2023-10-06T05:20:56.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.882+0000 7fa395d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3980c8d70 0x7fa3980c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.882+0000 7fa395d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3980c8d70 0x7fa3980c40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49516/0 (socket says 172.21.15.31:49516) 2023-10-06T05:20:56.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.882+0000 7fa395d9b700 1 -- 172.21.15.31:0/1362613501 learned_addr learned my addr 172.21.15.31:0/1362613501 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:56.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa396d9d700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa39815ca40 0x7fa3981577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa39659c700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39806a300 0x7fa3980b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa395d9b700 1 -- 172.21.15.31:0/1362613501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39806a300 msgr2=0x7fa3980b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa395d9b700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39806a300 0x7fa3980b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa395d9b700 1 -- 172.21.15.31:0/1362613501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa39815ca40 msgr2=0x7fa3981577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:56.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa395d9b700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa39815ca40 0x7fa3981577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:56.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa395d9b700 1 -- 172.21.15.31:0/1362613501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c0094f0 con 0x7fa3980c8d70 2023-10-06T05:20:56.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa39659c700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39806a300 0x7fa3980b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:56.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa395d9b700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3980c8d70 0x7fa3980c40f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa3980c9340 tx=0x7fa38c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa3867fc700 1 -- 172.21.15.31:0/1362613501 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c011420 con 0x7fa3980c8d70 2023-10-06T05:20:56.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa3867fc700 1 -- 172.21.15.31:0/1362613501 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c0115a0 con 0x7fa3980c8d70 2023-10-06T05:20:56.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa39cfff700 1 -- 172.21.15.31:0/1362613501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3980c65f0 con 0x7fa3980c8d70 2023-10-06T05:20:56.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.883+0000 7fa3867fc700 1 -- 172.21.15.31:0/1362613501 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c011960 con 0x7fa3980c8d70 2023-10-06T05:20:56.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.884+0000 7fa39cfff700 1 -- 172.21.15.31:0/1362613501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3980c4d20 con 0x7fa3980c8d70 2023-10-06T05:20:56.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.885+0000 7fa39cfff700 1 -- 172.21.15.31:0/1362613501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3980c6ad0 con 0x7fa3980c8d70 2023-10-06T05:20:56.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.886+0000 7fa3867fc700 1 -- 172.21.15.31:0/1362613501 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fa38c0253f0 con 0x7fa3980c8d70 2023-10-06T05:20:56.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.887+0000 7fa3867fc700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa378067170 0x7fa378069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:56.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.888+0000 7fa3867fc700 1 -- 172.21.15.31:0/1362613501 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fa38c0864d0 con 0x7fa3980c8d70 2023-10-06T05:20:56.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.888+0000 7fa39659c700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa378067170 0x7fa378069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:56.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.888+0000 7fa39659c700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa378067170 0x7fa378069620 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa388006f00 tx=0x7fa388008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:56.892+0000 7fa3867fc700 1 -- 172.21.15.31:0/1362613501 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa38c053220 con 0x7fa3980c8d70 2023-10-06T05:20:57.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.071+0000 7fa3867fc700 1 -- 172.21.15.31:0/1362613501 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa38c0535f0 con 0x7fa3980c8d70 2023-10-06T05:20:57.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.094+0000 7fa39cfff700 1 -- 172.21.15.31:0/1362613501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa39804c380 con 0x7fa3980c8d70 2023-10-06T05:20:57.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.095+0000 7fa3867fc700 1 -- 172.21.15.31:0/1362613501 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa38c069730 con 0x7fa3980c8d70 2023-10-06T05:20:57.097 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:20:57.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 -- 172.21.15.31:0/1362613501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa378067170 msgr2=0x7fa378069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa378067170 0x7fa378069620 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa388006f00 tx=0x7fa388008040 comp rx=0 tx=0).stop 2023-10-06T05:20:57.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 -- 172.21.15.31:0/1362613501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3980c8d70 msgr2=0x7fa3980c40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3980c8d70 0x7fa3980c40f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa3980c9340 tx=0x7fa38c017000 comp rx=0 tx=0).stop 2023-10-06T05:20:57.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 -- 172.21.15.31:0/1362613501 shutdown_connections 2023-10-06T05:20:57.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa39815ca40 0x7fa3981577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa39806a300 0x7fa3980b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa378067170 0x7fa378069620 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 --2- 172.21.15.31:0/1362613501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3980c8d70 0x7fa3980c40f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 -- 172.21.15.31:0/1362613501 >> 172.21.15.31:0/1362613501 conn(0x7fa3980ce290 msgr2=0x7fa398159790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 -- 172.21.15.31:0/1362613501 shutdown_connections 2023-10-06T05:20:57.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.097+0000 7fa39cfff700 1 -- 172.21.15.31:0/1362613501 wait complete. 2023-10-06T05:20:57.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:20:57.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.313+0000 7f2ecddff700 1 Processor -- start 2023-10-06T05:20:57.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.313+0000 7f2ecddff700 1 -- start start 2023-10-06T05:20:57.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.314+0000 7f2ecddff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ec815ca40 0x7f2ec8068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.314+0000 7f2ecddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 0x7f2ec80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.314+0000 7f2ecddff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec8068da0 con 0x7f2ec806a300 2023-10-06T05:20:57.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.314+0000 7f2ecddff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec80cabc0 con 0x7f2ec815ca40 2023-10-06T05:20:57.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.314+0000 7f2ecddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec8068bb0 con 0x7f2ec80c8f40 2023-10-06T05:20:57.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.314+0000 7f2ec77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ec815ca40 0x7f2ec8068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.315+0000 7f2ec77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ec815ca40 0x7f2ec8068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60392/0 (socket says 172.21.15.31:60392) 2023-10-06T05:20:57.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.315+0000 7f2ec77fe700 1 -- 172.21.15.31:0/720862 learned_addr learned my addr 172.21.15.31:0/720862 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:57.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.315+0000 7f2ec7fff700 1 --2- 172.21.15.31:0/720862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 0x7f2ec80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.315+0000 7f2ec77fe700 1 -- 172.21.15.31:0/720862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 msgr2=0x7f2ec80adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:57.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.315+0000 7f2ec77fe700 1 --2- 172.21.15.31:0/720862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 0x7f2ec80adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.315+0000 7f2ec77fe700 1 -- 172.21.15.31:0/720862 >> v1:172.21.15.62:6789/0 conn(0x7f2ec806a300 legacy=0x7f2ec80c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.315+0000 7f2ec77fe700 1 -- 172.21.15.31:0/720862 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec8069080 con 0x7f2ec815ca40 2023-10-06T05:20:57.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.315+0000 7f2ec7fff700 1 --2- 172.21.15.31:0/720862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 0x7f2ec80adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:57.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.315+0000 7f2ec77fe700 1 --2- 172.21.15.31:0/720862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 0x7f2ec8068670 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80099e0 tx=0x7f2eb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf84a0c0ba9c0c6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.315+0000 7f2ec67fc700 1 -- 172.21.15.31:0/720862 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800b820 con 0x7f2ec815ca40 2023-10-06T05:20:57.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.316+0000 7f2ec67fc700 1 -- 172.21.15.31:0/720862 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb800b980 con 0x7f2ec815ca40 2023-10-06T05:20:57.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.316+0000 7f2ec67fc700 1 -- 172.21.15.31:0/720862 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800bd20 con 0x7f2ec815ca40 2023-10-06T05:20:57.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.316+0000 7f2ecddff700 1 -- 172.21.15.31:0/720862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 msgr2=0x7f2ec8068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.316+0000 7f2ecddff700 1 --2- 172.21.15.31:0/720862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 0x7f2ec8068670 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80099e0 tx=0x7f2eb8009250 comp rx=0 tx=0).stop 2023-10-06T05:20:57.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.316+0000 7f2ecddff700 1 -- 172.21.15.31:0/720862 shutdown_connections 2023-10-06T05:20:57.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.316+0000 7f2ecddff700 1 --2- 172.21.15.31:0/720862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 0x7f2ec80adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.316+0000 7f2ecddff700 1 --2- 172.21.15.31:0/720862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 0x7f2ec8068670 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.316+0000 7f2ecddff700 1 -- 172.21.15.31:0/720862 >> 172.21.15.31:0/720862 conn(0x7f2ec80ce290 msgr2=0x7f2ec80ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.316+0000 7f2ecddff700 1 -- 172.21.15.31:0/720862 shutdown_connections 2023-10-06T05:20:57.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.316+0000 7f2ecddff700 1 -- 172.21.15.31:0/720862 wait complete. 2023-10-06T05:20:57.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.318+0000 7f2ecddff700 1 Processor -- start 2023-10-06T05:20:57.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.318+0000 7f2ecddff700 1 -- start start 2023-10-06T05:20:57.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ecddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ec806a300 0x7f2ec814d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ecddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 0x7f2ec815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ecddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 0x7f2ec815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ecddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80c6330 con 0x7f2ec815ca40 2023-10-06T05:20:57.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ecddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80c30d0 con 0x7f2ec806a300 2023-10-06T05:20:57.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ecddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80c6000 con 0x7f2ec80c8f40 2023-10-06T05:20:57.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ec7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 0x7f2ec815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ec7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 0x7f2ec815a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60402/0 (socket says 172.21.15.31:60402) 2023-10-06T05:20:57.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ec7fff700 1 -- 172.21.15.31:0/1401356600 learned_addr learned my addr 172.21.15.31:0/1401356600 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:57.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ec77fe700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ec806a300 0x7f2ec814d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ec6ffd700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 0x7f2ec815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ec7fff700 1 -- 172.21.15.31:0/1401356600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 msgr2=0x7f2ec815a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ec7fff700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 0x7f2ec815a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ec7fff700 1 -- 172.21.15.31:0/1401356600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ec806a300 msgr2=0x7f2ec814d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ec7fff700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ec806a300 0x7f2ec814d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.319+0000 7f2ec7fff700 1 -- 172.21.15.31:0/1401356600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80094f0 con 0x7f2ec815ca40 2023-10-06T05:20:57.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.320+0000 7f2ec77fe700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ec806a300 0x7f2ec814d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:57.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.320+0000 7f2ec7fff700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 0x7f2ec815a7f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000d0d0 tx=0x7f2ec00099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.320+0000 7f2eaf7fe700 1 -- 172.21.15.31:0/1401356600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000dd70 con 0x7f2ec815ca40 2023-10-06T05:20:57.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.320+0000 7f2eaf7fe700 1 -- 172.21.15.31:0/1401356600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec000f040 con 0x7f2ec815ca40 2023-10-06T05:20:57.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.320+0000 7f2eaf7fe700 1 -- 172.21.15.31:0/1401356600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0015650 con 0x7f2ec815ca40 2023-10-06T05:20:57.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.320+0000 7f2ecddff700 1 -- 172.21.15.31:0/1401356600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec80c2830 con 0x7f2ec815ca40 2023-10-06T05:20:57.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.320+0000 7f2ecddff700 1 -- 172.21.15.31:0/1401356600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec80c2df0 con 0x7f2ec815ca40 2023-10-06T05:20:57.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.321+0000 7f2ecddff700 1 -- 172.21.15.31:0/1401356600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec804c380 con 0x7f2ec815ca40 2023-10-06T05:20:57.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.322+0000 7f2eaf7fe700 1 -- 172.21.15.31:0/1401356600 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2ec0015a00 con 0x7f2ec815ca40 2023-10-06T05:20:57.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.323+0000 7f2eaf7fe700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eb4067290 0x7f2eb4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.323+0000 7f2eaf7fe700 1 -- 172.21.15.31:0/1401356600 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f2ec0085550 con 0x7f2ec815ca40 2023-10-06T05:20:57.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.323+0000 7f2ec77fe700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eb4067290 0x7f2eb4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.324+0000 7f2ec77fe700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eb4067290 0x7f2eb4069740 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80099e0 tx=0x7f2eb8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.327+0000 7f2eaf7fe700 1 -- 172.21.15.31:0/1401356600 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ec0052070 con 0x7f2ec815ca40 2023-10-06T05:20:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.530+0000 7f2ecddff700 1 -- 172.21.15.31:0/1401356600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ec80ca1a0 con 0x7f2ec815ca40 2023-10-06T05:20:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.530+0000 7f2eaf7fe700 1 -- 172.21.15.31:0/1401356600 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ec0015cb0 con 0x7f2ec815ca40 2023-10-06T05:20:57.532 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:20:57.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 -- 172.21.15.31:0/1401356600 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eb4067290 msgr2=0x7f2eb4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eb4067290 0x7f2eb4069740 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80099e0 tx=0x7f2eb8017000 comp rx=0 tx=0).stop 2023-10-06T05:20:57.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 -- 172.21.15.31:0/1401356600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 msgr2=0x7f2ec815a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 0x7f2ec815a7f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000d0d0 tx=0x7f2ec00099f0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 -- 172.21.15.31:0/1401356600 shutdown_connections 2023-10-06T05:20:57.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2eb4067290 0x7f2eb4069740 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec815ca40 0x7f2ec815a7f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ec806a300 0x7f2ec814d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 --2- 172.21.15.31:0/1401356600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ec80c8f40 0x7f2ec815a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 -- 172.21.15.31:0/1401356600 >> 172.21.15.31:0/1401356600 conn(0x7f2ec80ce290 msgr2=0x7f2ec80b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 -- 172.21.15.31:0/1401356600 shutdown_connections 2023-10-06T05:20:57.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.533+0000 7f2ecddff700 1 -- 172.21.15.31:0/1401356600 wait complete. 2023-10-06T05:20:57.552 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:20:57.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.748+0000 7faec5fff700 1 Processor -- start 2023-10-06T05:20:57.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.748+0000 7faec5fff700 1 -- start start 2023-10-06T05:20:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faec5fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec0159560 0x7faec00cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faec5fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faec0159930 0x7faec00cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faec5fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faec00b0d60 0x7faec00b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faec5fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec015d2d0 con 0x7faec00b0d60 2023-10-06T05:20:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faec5fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec015d5b0 con 0x7faec0159930 2023-10-06T05:20:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faec5fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec015d890 con 0x7faec0159560 2023-10-06T05:20:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faebffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faec00b0d60 0x7faec00b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faebffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faec00b0d60 0x7faec00b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60404/0 (socket says 172.21.15.31:60404) 2023-10-06T05:20:57.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faebffff700 1 -- 172.21.15.31:0/3333364679 learned_addr learned my addr 172.21.15.31:0/3333364679 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faebeffd700 1 --2- 172.21.15.31:0/3333364679 >> v2:172.21.15.62:3300/0 conn(0x7faec0159930 0x7faec00cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.749+0000 7faebf7fe700 1 --2- 172.21.15.31:0/3333364679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec0159560 0x7faec00cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.750+0000 7faebffff700 1 -- 172.21.15.31:0/3333364679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec0159560 msgr2=0x7faec00cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.750+0000 7faebffff700 1 --2- 172.21.15.31:0/3333364679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec0159560 0x7faec00cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.750+0000 7faebffff700 1 -- 172.21.15.31:0/3333364679 >> v2:172.21.15.62:3300/0 conn(0x7faec0159930 msgr2=0x7faec00cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.750+0000 7faebffff700 1 --2- 172.21.15.31:0/3333364679 >> v2:172.21.15.62:3300/0 conn(0x7faec0159930 0x7faec00cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.750+0000 7faebffff700 1 -- 172.21.15.31:0/3333364679 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec015db70 con 0x7faec00b0d60 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.750+0000 7faebeffd700 1 --2- 172.21.15.31:0/3333364679 >> v2:172.21.15.62:3300/0 conn(0x7faec0159930 0x7faec00cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.750+0000 7faebffff700 1 --2- 172.21.15.31:0/3333364679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00b0d60 0x7faec00b2470 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faec00d03f0 tx=0x7faeb8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2670f3a8695652f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.750+0000 7faebe7fc700 1 -- 172.21.15.31:0/3333364679 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb800e070 con 0x7faec00b0d60 2023-10-06T05:20:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.750+0000 7faebe7fc700 1 -- 172.21.15.31:0/3333364679 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb800ba50 con 0x7faec00b0d60 2023-10-06T05:20:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.750+0000 7faebe7fc700 1 -- 172.21.15.31:0/3333364679 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80092e0 con 0x7faec00b0d60 2023-10-06T05:20:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.751+0000 7faec5fff700 1 -- 172.21.15.31:0/3333364679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00b0d60 msgr2=0x7faec00b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.751+0000 7faec5fff700 1 --2- 172.21.15.31:0/3333364679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00b0d60 0x7faec00b2470 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faec00d03f0 tx=0x7faeb8009840 comp rx=0 tx=0).stop 2023-10-06T05:20:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.751+0000 7faec5fff700 1 -- 172.21.15.31:0/3333364679 shutdown_connections 2023-10-06T05:20:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.751+0000 7faec5fff700 1 --2- 172.21.15.31:0/3333364679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00b0d60 0x7faec00b2470 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.751+0000 7faec5fff700 1 --2- 172.21.15.31:0/3333364679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec0159560 0x7faec00cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.751+0000 7faec5fff700 1 --2- 172.21.15.31:0/3333364679 >> v2:172.21.15.62:3300/0 conn(0x7faec0159930 0x7faec00cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.751+0000 7faec5fff700 1 -- 172.21.15.31:0/3333364679 >> 172.21.15.31:0/3333364679 conn(0x7faec006ec90 msgr2=0x7faec006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.751+0000 7faec5fff700 1 -- 172.21.15.31:0/3333364679 shutdown_connections 2023-10-06T05:20:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.751+0000 7faec5fff700 1 -- 172.21.15.31:0/3333364679 wait complete. 2023-10-06T05:20:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.752+0000 7faec5fff700 1 Processor -- start 2023-10-06T05:20:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.753+0000 7faec5fff700 1 -- start start 2023-10-06T05:20:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.753+0000 7faec5fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec00b0d60 0x7faec0150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faec5fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0159560 0x7faec00b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faec5fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec0159930 0x7faec00b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faec5fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00c3ef0 con 0x7faec0159930 2023-10-06T05:20:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faec5fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00c41d0 con 0x7faec0159560 2023-10-06T05:20:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faec5fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec00c44b0 con 0x7faec00b0d60 2023-10-06T05:20:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faebffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec0159930 0x7faec00b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faebffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec0159930 0x7faec00b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60416/0 (socket says 172.21.15.31:60416) 2023-10-06T05:20:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faebffff700 1 -- 172.21.15.31:0/2848902677 learned_addr learned my addr 172.21.15.31:0/2848902677 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faebeffd700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0159560 0x7faec00b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faebf7fe700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec00b0d60 0x7faec0150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faebffff700 1 -- 172.21.15.31:0/2848902677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec00b0d60 msgr2=0x7faec0150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faebffff700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec00b0d60 0x7faec0150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faebffff700 1 -- 172.21.15.31:0/2848902677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0159560 msgr2=0x7faec00b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faebffff700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0159560 0x7faec00b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.754+0000 7faebffff700 1 -- 172.21.15.31:0/2848902677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8009ae0 con 0x7faec0159930 2023-10-06T05:20:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.755+0000 7faebffff700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec0159930 0x7faec00b4010 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7faec00893d0 tx=0x7faeb8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.755+0000 7faea77fe700 1 -- 172.21.15.31:0/2848902677 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80032f0 con 0x7faec0159930 2023-10-06T05:20:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.755+0000 7faea77fe700 1 -- 172.21.15.31:0/2848902677 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb8008d90 con 0x7faec0159930 2023-10-06T05:20:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.755+0000 7faec5fff700 1 -- 172.21.15.31:0/2848902677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec00c4790 con 0x7faec0159930 2023-10-06T05:20:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.755+0000 7faea77fe700 1 -- 172.21.15.31:0/2848902677 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80266b0 con 0x7faec0159930 2023-10-06T05:20:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.755+0000 7faec5fff700 1 -- 172.21.15.31:0/2848902677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec0150e70 con 0x7faec0159930 2023-10-06T05:20:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.756+0000 7faec5fff700 1 -- 172.21.15.31:0/2848902677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec004c380 con 0x7faec0159930 2023-10-06T05:20:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.757+0000 7faea77fe700 1 -- 172.21.15.31:0/2848902677 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7faeb8003040 con 0x7faec0159930 2023-10-06T05:20:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.758+0000 7faea77fe700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faeac067290 0x7faeac069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.758+0000 7faea77fe700 1 -- 172.21.15.31:0/2848902677 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7faeb80867d0 con 0x7faec0159930 2023-10-06T05:20:57.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.758+0000 7faebf7fe700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faeac067290 0x7faeac069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:57.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.759+0000 7faebf7fe700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faeac067290 0x7faeac069740 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faeb0006fd0 tx=0x7faeb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:57.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.763+0000 7faea77fe700 1 -- 172.21.15.31:0/2848902677 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faeb80533a0 con 0x7faec0159930 2023-10-06T05:20:57.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.965+0000 7faec5fff700 1 -- 172.21.15.31:0/2848902677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faec015de50 con 0x7faec0159930 2023-10-06T05:20:57.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.965+0000 7faea77fe700 1 -- 172.21.15.31:0/2848902677 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7faeb80569c0 con 0x7faec0159930 2023-10-06T05:20:57.967 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:20:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 -- 172.21.15.31:0/2848902677 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faeac067290 msgr2=0x7faeac069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faeac067290 0x7faeac069740 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faeb0006fd0 tx=0x7faeb0008040 comp rx=0 tx=0).stop 2023-10-06T05:20:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 -- 172.21.15.31:0/2848902677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec0159930 msgr2=0x7faec00b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec0159930 0x7faec00b4010 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7faec00893d0 tx=0x7faeb8018000 comp rx=0 tx=0).stop 2023-10-06T05:20:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 -- 172.21.15.31:0/2848902677 shutdown_connections 2023-10-06T05:20:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faeac067290 0x7faeac069740 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec0159930 0x7faec00b4010 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec00b0d60 0x7faec0150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 --2- 172.21.15.31:0/2848902677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec0159560 0x7faec00b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:57.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 -- 172.21.15.31:0/2848902677 >> 172.21.15.31:0/2848902677 conn(0x7faec006ec90 msgr2=0x7faec00cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:57.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 -- 172.21.15.31:0/2848902677 shutdown_connections 2023-10-06T05:20:57.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:57.968+0000 7faec5fff700 1 -- 172.21.15.31:0/2848902677 wait complete. 2023-10-06T05:20:57.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.181+0000 7f111519d700 1 Processor -- start 2023-10-06T05:20:58.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.181+0000 7f111519d700 1 -- start start 2023-10-06T05:20:58.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.182+0000 7f111519d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f11101597b0 0x7f11100b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.182+0000 7f111519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11100ceeb0 0x7f11100cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.182+0000 7f111519d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11100cbf40 con 0x7f11100ceae0 2023-10-06T05:20:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.182+0000 7f111519d700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11100cc220 con 0x7f11101597b0 2023-10-06T05:20:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.182+0000 7f111519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11100cc500 con 0x7f11100ceeb0 2023-10-06T05:20:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.182+0000 7f110ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f11100ceae0 0x7f11101590a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39574/0 (socket says 172.21.15.31:39574) 2023-10-06T05:20:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.182+0000 7f110ffff700 1 -- 172.21.15.31:0/463109445 learned_addr learned my addr 172.21.15.31:0/463109445 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:20:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110f7fe700 1 --2- 172.21.15.31:0/463109445 >> v2:172.21.15.62:3300/0 conn(0x7f11101597b0 0x7f11100b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f111499c700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11100ceeb0 0x7f11100cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716220707 0 0) 0x7f11100cbf40 con 0x7f11100ceae0 2023-10-06T05:20:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10f8003570 con 0x7f11100ceae0 2023-10-06T05:20:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 587074672 0 0) 0x7f10f8003570 con 0x7f11100ceae0 2023-10-06T05:20:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11100ceeb0 msgr2=0x7f11100cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11100ceeb0 0x7f11100cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 >> v2:172.21.15.62:3300/0 conn(0x7f11101597b0 msgr2=0x7f11100b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 --2- 172.21.15.31:0/463109445 >> v2:172.21.15.62:3300/0 conn(0x7f11101597b0 0x7f11100b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11100cc7e0 con 0x7f11100ceae0 2023-10-06T05:20:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f11000015f0 con 0x7f11100ceae0 2023-10-06T05:20:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 >> v1:172.21.15.31:6789/0 conn(0x7f11100ceae0 legacy=0x7f11101590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11100ceeb0 0x7f10f8003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80040b0 0x7f10f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10f8004d50 0x7f10f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11100cc220 con 0x7f10f80040b0 2023-10-06T05:20:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11100cc500 con 0x7f10f8004d50 2023-10-06T05:20:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.183+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f800bb60 con 0x7f11100ceeb0 2023-10-06T05:20:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f110f7fe700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80040b0 0x7f10f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f110ffff700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11100ceeb0 0x7f10f8003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f111499c700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10f8004d50 0x7f10f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f110f7fe700 1 -- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11100ceeb0 msgr2=0x7f10f8003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f110f7fe700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11100ceeb0 0x7f10f8003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f110f7fe700 1 -- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10f8004d50 msgr2=0x7f10f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f110f7fe700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10f8004d50 0x7f10f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f110f7fe700 1 -- 172.21.15.31:0/463109445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8003570 con 0x7f10f80040b0 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f111499c700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10f8004d50 0x7f10f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f110f7fe700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80040b0 0x7f10f8004810 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1104006f00 tx=0x7f11040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e48b3d515173e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.184+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110400e710 con 0x7f10f80040b0 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.185+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110400e870 con 0x7f10f80040b0 2023-10-06T05:20:58.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.185+0000 7f110effd700 1 -- 172.21.15.31:0/463109445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110400ec30 con 0x7f10f80040b0 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.185+0000 7f111519d700 1 -- 172.21.15.31:0/463109445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80040b0 msgr2=0x7f10f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.185+0000 7f111519d700 1 --2- 172.21.15.31:0/463109445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80040b0 0x7f10f8004810 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1104006f00 tx=0x7f11040091a0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.185+0000 7f111499c700 1 -- 172.21.15.31:0/463109445 reap_dead start 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.185+0000 7f111519d700 1 -- 172.21.15.31:0/463109445 shutdown_connections 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.185+0000 7f111519d700 1 -- 172.21.15.31:0/463109445 >> 172.21.15.31:0/463109445 conn(0x7f1110099560 msgr2=0x7f111006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.185+0000 7f111519d700 1 -- 172.21.15.31:0/463109445 shutdown_connections 2023-10-06T05:20:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.185+0000 7f111519d700 1 -- 172.21.15.31:0/463109445 wait complete. 2023-10-06T05:20:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.187+0000 7f111519d700 1 Processor -- start 2023-10-06T05:20:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.187+0000 7f111519d700 1 -- start start 2023-10-06T05:20:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f111519d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f80040b0 0x7f1110154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f111519d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11100c6090 0x7f11100c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f111519d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11100c7de0 0x7f11100c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f111519d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11101554b0 con 0x7f11100c6090 2023-10-06T05:20:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f111519d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1110159d50 con 0x7f11100c7de0 2023-10-06T05:20:58.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f111519d700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11100c6a40 con 0x7f10f80040b0 2023-10-06T05:20:58.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f110f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11100c6090 0x7f11100c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f110f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11100c6090 0x7f11100c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60434/0 (socket says 172.21.15.31:60434) 2023-10-06T05:20:58.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f110f7fe700 1 -- 172.21.15.31:0/1814054167 learned_addr learned my addr 172.21.15.31:0/1814054167 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:58.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f111499c700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11100c7de0 0x7f11100c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f110ffff700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f80040b0 0x7f1110154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f110f7fe700 1 -- 172.21.15.31:0/1814054167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f80040b0 msgr2=0x7f1110154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f110f7fe700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f80040b0 0x7f1110154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f110f7fe700 1 -- 172.21.15.31:0/1814054167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11100c7de0 msgr2=0x7f11100c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f110f7fe700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11100c7de0 0x7f11100c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f110f7fe700 1 -- 172.21.15.31:0/1814054167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1110159f80 con 0x7f11100c6090 2023-10-06T05:20:58.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.188+0000 7f110ffff700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f80040b0 0x7f1110154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:58.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.189+0000 7f110f7fe700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11100c6090 0x7f11100c6500 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f10f8004760 tx=0x7f110401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.189+0000 7f10f7fff700 1 -- 172.21.15.31:0/1814054167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104022070 con 0x7f11100c6090 2023-10-06T05:20:58.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.189+0000 7f10f7fff700 1 -- 172.21.15.31:0/1814054167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110400e9e0 con 0x7f11100c6090 2023-10-06T05:20:58.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.189+0000 7f10f7fff700 1 -- 172.21.15.31:0/1814054167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104027470 con 0x7f11100c6090 2023-10-06T05:20:58.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.189+0000 7f111519d700 1 -- 172.21.15.31:0/1814054167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11100c8ad0 con 0x7f11100c6090 2023-10-06T05:20:58.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.189+0000 7f111519d700 1 -- 172.21.15.31:0/1814054167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111015a220 con 0x7f11100c6090 2023-10-06T05:20:58.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.190+0000 7f111519d700 1 -- 172.21.15.31:0/1814054167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111004c380 con 0x7f11100c6090 2023-10-06T05:20:58.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.191+0000 7f10f7fff700 1 -- 172.21.15.31:0/1814054167 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f110401c450 con 0x7f11100c6090 2023-10-06T05:20:58.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.192+0000 7f10f7fff700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10fc067280 0x7f10fc069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.192+0000 7f10f7fff700 1 -- 172.21.15.31:0/1814054167 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f11040868f0 con 0x7f11100c6090 2023-10-06T05:20:58.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.192+0000 7f110ffff700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10fc067280 0x7f10fc069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.193+0000 7f110ffff700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10fc067280 0x7f10fc069730 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11000011d0 tx=0x7f1100009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.196+0000 7f10f7fff700 1 -- 172.21.15.31:0/1814054167 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1104053580 con 0x7f11100c6090 2023-10-06T05:20:58.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.445+0000 7f111519d700 1 -- 172.21.15.31:0/1814054167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11100c87c0 con 0x7f11100c6090 2023-10-06T05:20:58.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.445+0000 7f10f7fff700 1 -- 172.21.15.31:0/1814054167 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f1104056b60 con 0x7f11100c6090 2023-10-06T05:20:58.447 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:58.448 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5743,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2430199885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2430199885},{"type":"v1","addr":"172.21.15.31:6837","nonce":2430199885}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5746,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3762519665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3762519665},{"type":"v1","addr":"172.21.15.62:6834","nonce":3762519665}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5844,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2776536460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2776536460},{"type":"v1","addr":"172.21.15.31:6836","nonce":2776536460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:36.742745+0000","modified":"2023-10-06T05:20:40.276711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5853":{"gid":5853,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/982526111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":982526111},{"type":"v1","addr":"172.21.15.62:6835","nonce":982526111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:20:58.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 -- 172.21.15.31:0/1814054167 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10fc067280 msgr2=0x7f10fc069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10fc067280 0x7f10fc069730 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11000011d0 tx=0x7f1100009040 comp rx=0 tx=0).stop 2023-10-06T05:20:58.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 -- 172.21.15.31:0/1814054167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11100c6090 msgr2=0x7f11100c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11100c6090 0x7f11100c6500 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f10f8004760 tx=0x7f110401a000 comp rx=0 tx=0).stop 2023-10-06T05:20:58.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 -- 172.21.15.31:0/1814054167 shutdown_connections 2023-10-06T05:20:58.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11100c7de0 0x7f11100c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f80040b0 0x7f1110154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f10fc067280 0x7f10fc069730 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 --2- 172.21.15.31:0/1814054167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11100c6090 0x7f11100c6500 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 -- 172.21.15.31:0/1814054167 >> 172.21.15.31:0/1814054167 conn(0x7f1110099560 msgr2=0x7f1110157ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:58.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 -- 172.21.15.31:0/1814054167 shutdown_connections 2023-10-06T05:20:58.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.448+0000 7f111519d700 1 -- 172.21.15.31:0/1814054167 wait complete. 2023-10-06T05:20:58.451 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 36 2023-10-06T05:20:58.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:20:58.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.660+0000 7f03055ff700 1 Processor -- start 2023-10-06T05:20:58.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.660+0000 7f03055ff700 1 -- start start 2023-10-06T05:20:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f03055ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f030013a5b0 0x7f030006d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f03055ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030006d980 0x7f0300069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f03055ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030006ddb0 con 0x7f030013a1e0 2023-10-06T05:20:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f03055ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03000c71b0 con 0x7f030013a5b0 2023-10-06T05:20:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f03055ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000c7490 con 0x7f030006d980 2023-10-06T05:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f02fe7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f030013a5b0 0x7f030006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f02fe7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f030013a5b0 0x7f030006d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60442/0 (socket says 172.21.15.31:60442) 2023-10-06T05:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f02fe7fc700 1 -- 172.21.15.31:0/2545388839 learned_addr learned my addr 172.21.15.31:0/2545388839 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f02ff7fe700 1 --2- 172.21.15.31:0/2545388839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030006d980 0x7f0300069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f02fe7fc700 1 -- 172.21.15.31:0/2545388839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030006d980 msgr2=0x7f0300069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.662+0000 7f02fe7fc700 1 --2- 172.21.15.31:0/2545388839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030006d980 0x7f0300069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.663+0000 7f02fe7fc700 1 -- 172.21.15.31:0/2545388839 >> v1:172.21.15.62:6789/0 conn(0x7f030013a1e0 legacy=0x7f03000c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.663+0000 7f02fe7fc700 1 -- 172.21.15.31:0/2545388839 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03000c7770 con 0x7f030013a5b0 2023-10-06T05:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.663+0000 7f02ff7fe700 1 --2- 172.21.15.31:0/2545388839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030006d980 0x7f0300069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.663+0000 7f02fe7fc700 1 --2- 172.21.15.31:0/2545388839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030013a5b0 0x7f030006d440 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f02f40099b0 tx=0x7f02f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d98dd9f079a3609 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.663+0000 7f02fdffb700 1 -- 172.21.15.31:0/2545388839 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f400b860 con 0x7f030013a5b0 2023-10-06T05:20:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.663+0000 7f02fdffb700 1 -- 172.21.15.31:0/2545388839 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f400b9c0 con 0x7f030013a5b0 2023-10-06T05:20:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.663+0000 7f02fdffb700 1 -- 172.21.15.31:0/2545388839 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f400bd60 con 0x7f030013a5b0 2023-10-06T05:20:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.664+0000 7f03055ff700 1 -- 172.21.15.31:0/2545388839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030013a5b0 msgr2=0x7f030006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.664+0000 7f03055ff700 1 --2- 172.21.15.31:0/2545388839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030013a5b0 0x7f030006d440 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f02f40099b0 tx=0x7f02f4009250 comp rx=0 tx=0).stop 2023-10-06T05:20:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.664+0000 7f03055ff700 1 -- 172.21.15.31:0/2545388839 shutdown_connections 2023-10-06T05:20:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.664+0000 7f03055ff700 1 --2- 172.21.15.31:0/2545388839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030006d980 0x7f0300069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.664+0000 7f03055ff700 1 --2- 172.21.15.31:0/2545388839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030013a5b0 0x7f030006d440 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.664+0000 7f03055ff700 1 -- 172.21.15.31:0/2545388839 >> 172.21.15.31:0/2545388839 conn(0x7f03000cc0b0 msgr2=0x7f03000cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.664+0000 7f03055ff700 1 -- 172.21.15.31:0/2545388839 shutdown_connections 2023-10-06T05:20:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.664+0000 7f03055ff700 1 -- 172.21.15.31:0/2545388839 wait complete. 2023-10-06T05:20:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.665+0000 7f03055ff700 1 Processor -- start 2023-10-06T05:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.665+0000 7f03055ff700 1 -- start start 2023-10-06T05:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.666+0000 7f03055ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030006d980 0x7f03001693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.666+0000 7f03055ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f030013a1e0 0x7f03000c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.666+0000 7f03055ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030013a5b0 0x7f03000c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.666+0000 7f03055ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03000c3da0 con 0x7f030006d980 2023-10-06T05:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.666+0000 7f03055ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03000c3200 con 0x7f030013a1e0 2023-10-06T05:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.666+0000 7f03055ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000c3470 con 0x7f030013a5b0 2023-10-06T05:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030006d980 0x7f03001693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030006d980 0x7f03001693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60448/0 (socket says 172.21.15.31:60448) 2023-10-06T05:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02feffd700 1 -- 172.21.15.31:0/1105952591 learned_addr learned my addr 172.21.15.31:0/1105952591 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02fe7fc700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f030013a1e0 0x7f03000c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02ff7fe700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030013a5b0 0x7f03000c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02feffd700 1 -- 172.21.15.31:0/1105952591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030013a5b0 msgr2=0x7f03000c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02feffd700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030013a5b0 0x7f03000c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02feffd700 1 -- 172.21.15.31:0/1105952591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f030013a1e0 msgr2=0x7f03000c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02feffd700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f030013a1e0 0x7f03000c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02feffd700 1 -- 172.21.15.31:0/1105952591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f40094f0 con 0x7f030006d980 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.667+0000 7f02feffd700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030006d980 0x7f03001693c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f02f000c7c0 tx=0x7f02f0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.668+0000 7f02e6ffd700 1 -- 172.21.15.31:0/1105952591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011590 con 0x7f030006d980 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.668+0000 7f02e6ffd700 1 -- 172.21.15.31:0/1105952591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f00116f0 con 0x7f030006d980 2023-10-06T05:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.668+0000 7f03055ff700 1 -- 172.21.15.31:0/1105952591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03000c3750 con 0x7f030006d980 2023-10-06T05:20:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.668+0000 7f02e6ffd700 1 -- 172.21.15.31:0/1105952591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011ab0 con 0x7f030006d980 2023-10-06T05:20:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.668+0000 7f03055ff700 1 -- 172.21.15.31:0/1105952591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0300169900 con 0x7f030006d980 2023-10-06T05:20:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.669+0000 7f03055ff700 1 -- 172.21.15.31:0/1105952591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030004c380 con 0x7f030006d980 2023-10-06T05:20:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.670+0000 7f02e6ffd700 1 -- 172.21.15.31:0/1105952591 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f02f001b3b0 con 0x7f030006d980 2023-10-06T05:20:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.671+0000 7f02e6ffd700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f02ec067290 0x7f02ec069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.671+0000 7f02e6ffd700 1 -- 172.21.15.31:0/1105952591 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f02f0084020 con 0x7f030006d980 2023-10-06T05:20:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.671+0000 7f02fe7fc700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f02ec067290 0x7f02ec069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.671+0000 7f02fe7fc700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f02ec067290 0x7f02ec069740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f030013dba0 tx=0x7f02f4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.675+0000 7f02e6ffd700 1 -- 172.21.15.31:0/1105952591 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02f0050d40 con 0x7f030006d980 2023-10-06T05:20:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.878+0000 7f03055ff700 1 -- 172.21.15.31:0/1105952591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03000c7a50 con 0x7f030006d980 2023-10-06T05:20:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.879+0000 7f02e6ffd700 1 -- 172.21.15.31:0/1105952591 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+17156 (secure 0 0 0) 0x7f02f0017070 con 0x7f030006d980 2023-10-06T05:20:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:58.882 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":59,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:20:50.325408+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:34.970939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:35.729829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":3}]},{"pgid":"10.23","mappings":[{"from":7,"to":3}]},{"pgid":"10.2f","mappings":[{"from":1,"to":3}]},{"pgid":"10.37","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:20:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 -- 172.21.15.31:0/1105952591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f02ec067290 msgr2=0x7f02ec069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f02ec067290 0x7f02ec069740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f030013dba0 tx=0x7f02f4017000 comp rx=0 tx=0).stop 2023-10-06T05:20:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 -- 172.21.15.31:0/1105952591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030006d980 msgr2=0x7f03001693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030006d980 0x7f03001693c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f02f000c7c0 tx=0x7f02f0007f70 comp rx=0 tx=0).stop 2023-10-06T05:20:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 -- 172.21.15.31:0/1105952591 shutdown_connections 2023-10-06T05:20:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f030013a5b0 0x7f03000c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f02ec067290 0x7f02ec069740 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f030006d980 0x7f03001693c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 --2- 172.21.15.31:0/1105952591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f030013a1e0 0x7f03000c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 -- 172.21.15.31:0/1105952591 >> 172.21.15.31:0/1105952591 conn(0x7f03000cc0b0 msgr2=0x7f03000c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 -- 172.21.15.31:0/1105952591 shutdown_connections 2023-10-06T05:20:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:58.882+0000 7f03055ff700 1 -- 172.21.15.31:0/1105952591 wait complete. 2023-10-06T05:20:58.902 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:20:58.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:59.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.096+0000 7f564e3ff700 1 Processor -- start 2023-10-06T05:20:59.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.097+0000 7f564e3ff700 1 -- start start 2023-10-06T05:20:59.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.098+0000 7f564e3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56481595a0 0x7f5648159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.098+0000 7f564e3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5648068d80 0x7f56480c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.098+0000 7f564e3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56480c95b0 con 0x7f56480cc0d0 2023-10-06T05:20:59.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.098+0000 7f564e3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56480c9890 con 0x7f56481595a0 2023-10-06T05:20:59.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.098+0000 7f564e3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56480c9b70 con 0x7f5648068d80 2023-10-06T05:20:59.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.098+0000 7f5647fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56481595a0 0x7f5648159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.098+0000 7f5647fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56481595a0 0x7f5648159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:55036/0 (socket says 172.21.15.31:55036) 2023-10-06T05:20:59.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.098+0000 7f5647fff700 1 -- 172.21.15.31:0/4158222264 learned_addr learned my addr 172.21.15.31:0/4158222264 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:59.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.098+0000 7f564c99c700 1 --2- 172.21.15.31:0/4158222264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5648068d80 0x7f56480c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.099+0000 7f5647fff700 1 -- 172.21.15.31:0/4158222264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5648068d80 msgr2=0x7f56480c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:59.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.099+0000 7f5647fff700 1 --2- 172.21.15.31:0/4158222264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5648068d80 0x7f56480c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.099+0000 7f5647fff700 1 -- 172.21.15.31:0/4158222264 >> v1:172.21.15.62:6789/0 conn(0x7f56480cc0d0 legacy=0x7f5648068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.099+0000 7f5647fff700 1 -- 172.21.15.31:0/4158222264 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480ca130 con 0x7f56481595a0 2023-10-06T05:20:59.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.099+0000 7f5647fff700 1 --2- 172.21.15.31:0/4158222264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56481595a0 0x7f5648159970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f56380099b0 tx=0x7f5638009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f3855af5a37c11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.099+0000 7f5646ffd700 1 -- 172.21.15.31:0/4158222264 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638009ea0 con 0x7f56481595a0 2023-10-06T05:20:59.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.099+0000 7f5646ffd700 1 -- 172.21.15.31:0/4158222264 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563800b860 con 0x7f56481595a0 2023-10-06T05:20:59.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.099+0000 7f5646ffd700 1 -- 172.21.15.31:0/4158222264 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800bc00 con 0x7f56481595a0 2023-10-06T05:20:59.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.100+0000 7f564e3ff700 1 -- 172.21.15.31:0/4158222264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56481595a0 msgr2=0x7f5648159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.100+0000 7f564e3ff700 1 --2- 172.21.15.31:0/4158222264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56481595a0 0x7f5648159970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f56380099b0 tx=0x7f5638009250 comp rx=0 tx=0).stop 2023-10-06T05:20:59.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.100+0000 7f564e3ff700 1 -- 172.21.15.31:0/4158222264 shutdown_connections 2023-10-06T05:20:59.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.100+0000 7f564e3ff700 1 --2- 172.21.15.31:0/4158222264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5648068d80 0x7f56480c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.100+0000 7f564e3ff700 1 --2- 172.21.15.31:0/4158222264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56481595a0 0x7f5648159970 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.100+0000 7f564e3ff700 1 -- 172.21.15.31:0/4158222264 >> 172.21.15.31:0/4158222264 conn(0x7f564806b620 msgr2=0x7f564806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:59.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.100+0000 7f564e3ff700 1 -- 172.21.15.31:0/4158222264 shutdown_connections 2023-10-06T05:20:59.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.100+0000 7f564e3ff700 1 -- 172.21.15.31:0/4158222264 wait complete. 2023-10-06T05:20:59.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.101+0000 7f564e3ff700 1 Processor -- start 2023-10-06T05:20:59.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.102+0000 7f564e3ff700 1 -- start start 2023-10-06T05:20:59.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.102+0000 7f564e3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5648068d80 0x7f56480c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.102+0000 7f564e3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56480cc0d0 0x7f56481566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f564e3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56481595a0 0x7f56480b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f564e3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56480b1090 con 0x7f5648068d80 2023-10-06T05:20:59.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f564e3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56480b1370 con 0x7f56480cc0d0 2023-10-06T05:20:59.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f564e3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56480b4070 con 0x7f56481595a0 2023-10-06T05:20:59.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f5647fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5648068d80 0x7f56480c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f56477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56480cc0d0 0x7f56481566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f5647fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5648068d80 0x7f56480c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60454/0 (socket says 172.21.15.31:60454) 2023-10-06T05:20:59.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f5647fff700 1 -- 172.21.15.31:0/3228608394 learned_addr learned my addr 172.21.15.31:0/3228608394 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f564c99c700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56481595a0 0x7f56480b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f5647fff700 1 -- 172.21.15.31:0/3228608394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56481595a0 msgr2=0x7f56480b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f5647fff700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56481595a0 0x7f56480b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f5647fff700 1 -- 172.21.15.31:0/3228608394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56480cc0d0 msgr2=0x7f56481566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f5647fff700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56480cc0d0 0x7f56481566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f5647fff700 1 -- 172.21.15.31:0/3228608394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56380094f0 con 0x7f5648068d80 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.103+0000 7f56477fe700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56480cc0d0 0x7f56481566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.104+0000 7f5647fff700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5648068d80 0x7f56480c83d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5638006fd0 tx=0x7f5638017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.104+0000 7f562ffff700 1 -- 172.21.15.31:0/3228608394 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800bbe0 con 0x7f5648068d80 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.104+0000 7f562ffff700 1 -- 172.21.15.31:0/3228608394 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5638000c90 con 0x7f5648068d80 2023-10-06T05:20:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.104+0000 7f564e3ff700 1 -- 172.21.15.31:0/3228608394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56480c8910 con 0x7f5648068d80 2023-10-06T05:20:59.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.104+0000 7f562ffff700 1 -- 172.21.15.31:0/3228608394 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638011650 con 0x7f5648068d80 2023-10-06T05:20:59.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.104+0000 7f564e3ff700 1 -- 172.21.15.31:0/3228608394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56480c8e90 con 0x7f5648068d80 2023-10-06T05:20:59.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.105+0000 7f564e3ff700 1 -- 172.21.15.31:0/3228608394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56480b1650 con 0x7f5648068d80 2023-10-06T05:20:59.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.106+0000 7f562ffff700 1 -- 172.21.15.31:0/3228608394 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f5638011a40 con 0x7f5648068d80 2023-10-06T05:20:59.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.107+0000 7f562ffff700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5634067290 0x7f5634069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.107+0000 7f562ffff700 1 -- 172.21.15.31:0/3228608394 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f5638086430 con 0x7f5648068d80 2023-10-06T05:20:59.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.107+0000 7f56477fe700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5634067290 0x7f5634069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.108+0000 7f56477fe700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5634067290 0x7f5634069740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f564806b230 tx=0x7f563c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.111+0000 7f562ffff700 1 -- 172.21.15.31:0/3228608394 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5638053060 con 0x7f5648068d80 2023-10-06T05:20:59.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.359+0000 7f564e3ff700 1 -- 172.21.15.31:0/3228608394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f564804c380 con 0x7f5648068d80 2023-10-06T05:20:59.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.362+0000 7f562ffff700 1 -- 172.21.15.31:0/3228608394 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f5638056680 con 0x7f5648068d80 2023-10-06T05:20:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:59.364 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5743,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2430199885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2430199885},{"type":"v1","addr":"172.21.15.31:6837","nonce":2430199885}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5746,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3762519665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3762519665},{"type":"v1","addr":"172.21.15.62:6834","nonce":3762519665}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5844,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2776536460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2776536460},{"type":"v1","addr":"172.21.15.31:6836","nonce":2776536460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:36.742745+0000","modified":"2023-10-06T05:20:40.276711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5853":{"gid":5853,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/982526111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":982526111},{"type":"v1","addr":"172.21.15.62:6835","nonce":982526111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:20:59.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 -- 172.21.15.31:0/3228608394 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5634067290 msgr2=0x7f5634069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5634067290 0x7f5634069740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f564806b230 tx=0x7f563c009040 comp rx=0 tx=0).stop 2023-10-06T05:20:59.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 -- 172.21.15.31:0/3228608394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5648068d80 msgr2=0x7f56480c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5648068d80 0x7f56480c83d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5638006fd0 tx=0x7f5638017000 comp rx=0 tx=0).stop 2023-10-06T05:20:59.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 -- 172.21.15.31:0/3228608394 shutdown_connections 2023-10-06T05:20:59.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56481595a0 0x7f56480b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f5634067290 0x7f5634069740 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5648068d80 0x7f56480c83d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 --2- 172.21.15.31:0/3228608394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56480cc0d0 0x7f56481566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 -- 172.21.15.31:0/3228608394 >> 172.21.15.31:0/3228608394 conn(0x7f564806b620 msgr2=0x7f5648158580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:59.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 -- 172.21.15.31:0/3228608394 shutdown_connections 2023-10-06T05:20:59.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.364+0000 7f564e3ff700 1 -- 172.21.15.31:0/3228608394 wait complete. 2023-10-06T05:20:59.367 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 36 2023-10-06T05:20:59.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:20:59.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.577+0000 7f6b24dff700 1 Processor -- start 2023-10-06T05:20:59.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.577+0000 7f6b24dff700 1 -- start start 2023-10-06T05:20:59.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.578+0000 7f6b24dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b201596d0 0x7f6b20159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.578+0000 7f6b24dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6b200c6f80 0x7f6b2006a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.578+0000 7f6b24dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b20067f70 con 0x7f6b200c16e0 2023-10-06T05:20:59.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.578+0000 7f6b24dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2006e040 con 0x7f6b201596d0 2023-10-06T05:20:59.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.578+0000 7f6b24dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b200681c0 con 0x7f6b200c6f80 2023-10-06T05:20:59.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.579+0000 7f6b1e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b201596d0 0x7f6b20159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.579+0000 7f6b1e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b201596d0 0x7f6b20159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60464/0 (socket says 172.21.15.31:60464) 2023-10-06T05:20:59.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.579+0000 7f6b1e59c700 1 -- 172.21.15.31:0/3470098266 learned_addr learned my addr 172.21.15.31:0/3470098266 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:59.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.579+0000 7f6b1dd9b700 1 --2- 172.21.15.31:0/3470098266 >> v2:172.21.15.62:3300/0 conn(0x7f6b200c6f80 0x7f6b2006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.579+0000 7f6b1e59c700 1 -- 172.21.15.31:0/3470098266 >> v2:172.21.15.62:3300/0 conn(0x7f6b200c6f80 msgr2=0x7f6b2006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:59.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.579+0000 7f6b1e59c700 1 --2- 172.21.15.31:0/3470098266 >> v2:172.21.15.62:3300/0 conn(0x7f6b200c6f80 0x7f6b2006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.579+0000 7f6b1e59c700 1 -- 172.21.15.31:0/3470098266 >> v1:172.21.15.62:6789/0 conn(0x7f6b200c16e0 legacy=0x7f6b2015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:20:59.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.579+0000 7f6b1e59c700 1 -- 172.21.15.31:0/3470098266 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200684a0 con 0x7f6b201596d0 2023-10-06T05:20:59.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.579+0000 7f6b1dd9b700 1 --2- 172.21.15.31:0/3470098266 >> v2:172.21.15.62:3300/0 conn(0x7f6b200c6f80 0x7f6b2006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:20:59.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.579+0000 7f6b1e59c700 1 --2- 172.21.15.31:0/3470098266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 0x7f6b20159aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6b2006dbe0 tx=0x7f6b10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83f1acfecf9f07d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b1d59a700 1 -- 172.21.15.31:0/3470098266 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000b820 con 0x7f6b201596d0 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b1d59a700 1 -- 172.21.15.31:0/3470098266 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1000b980 con 0x7f6b201596d0 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b1d59a700 1 -- 172.21.15.31:0/3470098266 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000bd20 con 0x7f6b201596d0 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b24dff700 1 -- 172.21.15.31:0/3470098266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 msgr2=0x7f6b20159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b24dff700 1 --2- 172.21.15.31:0/3470098266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 0x7f6b20159aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6b2006dbe0 tx=0x7f6b10009250 comp rx=0 tx=0).stop 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b24dff700 1 -- 172.21.15.31:0/3470098266 shutdown_connections 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b24dff700 1 --2- 172.21.15.31:0/3470098266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 0x7f6b20159aa0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b24dff700 1 --2- 172.21.15.31:0/3470098266 >> v2:172.21.15.62:3300/0 conn(0x7f6b200c6f80 0x7f6b2006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b24dff700 1 -- 172.21.15.31:0/3470098266 >> 172.21.15.31:0/3470098266 conn(0x7f6b200c9e80 msgr2=0x7f6b2006a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b24dff700 1 -- 172.21.15.31:0/3470098266 shutdown_connections 2023-10-06T05:20:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.580+0000 7f6b24dff700 1 -- 172.21.15.31:0/3470098266 wait complete. 2023-10-06T05:20:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.582+0000 7f6b24dff700 1 Processor -- start 2023-10-06T05:20:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.582+0000 7f6b24dff700 1 -- start start 2023-10-06T05:20:59.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b24dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b200c16e0 0x7f6b200c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b24dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b200c6f80 0x7f6b20158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b24dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 0x7f6b200b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b24dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2015cbc0 con 0x7f6b201596d0 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b24dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2015aad0 con 0x7f6b200c6f80 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b24dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2015cdd0 con 0x7f6b200c16e0 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b1ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 0x7f6b200b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b1e59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b200c16e0 0x7f6b200c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b1ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 0x7f6b200b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60476/0 (socket says 172.21.15.31:60476) 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b1ed9d700 1 -- 172.21.15.31:0/2185488894 learned_addr learned my addr 172.21.15.31:0/2185488894 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.583+0000 7f6b1dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b200c6f80 0x7f6b20158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b1ed9d700 1 -- 172.21.15.31:0/2185488894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b200c16e0 msgr2=0x7f6b200c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b1ed9d700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b200c16e0 0x7f6b200c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b1ed9d700 1 -- 172.21.15.31:0/2185488894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b200c6f80 msgr2=0x7f6b20158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b1ed9d700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b200c6f80 0x7f6b20158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b1ed9d700 1 -- 172.21.15.31:0/2185488894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100094f0 con 0x7f6b201596d0 2023-10-06T05:20:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b1ed9d700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 0x7f6b200b1710 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800c930 tx=0x7f6b180026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b0e7fc700 1 -- 172.21.15.31:0/2185488894 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18012630 con 0x7f6b201596d0 2023-10-06T05:20:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b0e7fc700 1 -- 172.21.15.31:0/2185488894 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b180127d0 con 0x7f6b201596d0 2023-10-06T05:20:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b0e7fc700 1 -- 172.21.15.31:0/2185488894 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18012b90 con 0x7f6b201596d0 2023-10-06T05:20:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b24dff700 1 -- 172.21.15.31:0/2185488894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2015d0b0 con 0x7f6b201596d0 2023-10-06T05:20:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.584+0000 7f6b24dff700 1 -- 172.21.15.31:0/2185488894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2015a760 con 0x7f6b201596d0 2023-10-06T05:20:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.586+0000 7f6b24dff700 1 -- 172.21.15.31:0/2185488894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2004c380 con 0x7f6b201596d0 2023-10-06T05:20:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.586+0000 7f6b0e7fc700 1 -- 172.21.15.31:0/2185488894 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6b1801b400 con 0x7f6b201596d0 2023-10-06T05:20:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.587+0000 7f6b0e7fc700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b00067290 0x7f6b00069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:20:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.587+0000 7f6b0e7fc700 1 -- 172.21.15.31:0/2185488894 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f6b18084f10 con 0x7f6b201596d0 2023-10-06T05:20:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.588+0000 7f6b1e59c700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b00067290 0x7f6b00069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:20:59.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.588+0000 7f6b1e59c700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b00067290 0x7f6b00069740 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b2006dbe0 tx=0x7f6b10016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:20:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.591+0000 7f6b0e7fc700 1 -- 172.21.15.31:0/2185488894 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b180555d0 con 0x7f6b201596d0 2023-10-06T05:20:59.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.837+0000 7f6b24dff700 1 -- 172.21.15.31:0/2185488894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b20063760 con 0x7f6b201596d0 2023-10-06T05:20:59.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.838+0000 7f6b0e7fc700 1 -- 172.21.15.31:0/2185488894 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f6b18055160 con 0x7f6b201596d0 2023-10-06T05:20:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:20:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5743,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2430199885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2430199885},{"type":"v1","addr":"172.21.15.31:6837","nonce":2430199885}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5746,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3762519665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3762519665},{"type":"v1","addr":"172.21.15.62:6834","nonce":3762519665}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5844,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2776536460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2776536460},{"type":"v1","addr":"172.21.15.31:6836","nonce":2776536460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:20:36.742745+0000","modified":"2023-10-06T05:20:40.276711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5853":{"gid":5853,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/982526111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":982526111},{"type":"v1","addr":"172.21.15.62:6835","nonce":982526111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T05:20:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 -- 172.21.15.31:0/2185488894 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b00067290 msgr2=0x7f6b00069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b00067290 0x7f6b00069740 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b2006dbe0 tx=0x7f6b10016000 comp rx=0 tx=0).stop 2023-10-06T05:20:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 -- 172.21.15.31:0/2185488894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 msgr2=0x7f6b200b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:20:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 0x7f6b200b1710 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800c930 tx=0x7f6b180026e0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 -- 172.21.15.31:0/2185488894 shutdown_connections 2023-10-06T05:20:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6b00067290 0x7f6b00069740 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b201596d0 0x7f6b200b1710 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b200c16e0 0x7f6b200c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 --2- 172.21.15.31:0/2185488894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b200c6f80 0x7f6b20158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:20:59.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 -- 172.21.15.31:0/2185488894 >> 172.21.15.31:0/2185488894 conn(0x7f6b200c9e80 msgr2=0x7f6b2015d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:20:59.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 -- 172.21.15.31:0/2185488894 shutdown_connections 2023-10-06T05:20:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:20:59.840+0000 7f6b24dff700 1 -- 172.21.15.31:0/2185488894 wait complete. 2023-10-06T05:20:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 36 2023-10-06T05:20:59.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:00.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.053+0000 7fdf5c3ff700 1 Processor -- start 2023-10-06T05:21:00.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.053+0000 7fdf5c3ff700 1 -- start start 2023-10-06T05:21:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5c3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdf540c7f30 0x7fdf54069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5c3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 0x7fdf5413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5c3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf540ca5b0 con 0x7fdf540c7b60 2023-10-06T05:21:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5c3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf540af2f0 con 0x7fdf540c7f30 2023-10-06T05:21:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5c3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5406f740 con 0x7fdf540aee70 2023-10-06T05:21:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5999a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdf540c7f30 0x7fdf54069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5999a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdf540c7f30 0x7fdf54069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60488/0 (socket says 172.21.15.31:60488) 2023-10-06T05:21:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5999a700 1 -- 172.21.15.31:0/586338793 learned_addr learned my addr 172.21.15.31:0/586338793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5a19b700 1 --1- 172.21.15.31:0/586338793 >> v1:172.21.15.31:6789/0 conn(0x7fdf540c7b60 0x7fdf5406ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39576/0 (socket says 172.21.15.31:39576) 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5a99c700 1 --2- 172.21.15.31:0/586338793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 0x7fdf5413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5999a700 1 -- 172.21.15.31:0/586338793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 msgr2=0x7fdf5413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5999a700 1 --2- 172.21.15.31:0/586338793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 0x7fdf5413f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5999a700 1 -- 172.21.15.31:0/586338793 >> v1:172.21.15.31:6789/0 conn(0x7fdf540c7b60 legacy=0x7fdf5406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5999a700 1 -- 172.21.15.31:0/586338793 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540c9c80 con 0x7fdf540c7f30 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.055+0000 7fdf5a99c700 1 --2- 172.21.15.31:0/586338793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 0x7fdf5413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.056+0000 7fdf5999a700 1 --2- 172.21.15.31:0/586338793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7f30 0x7fdf54069880 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fdf440099b0 tx=0x7fdf44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c082c16a8ee9d2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.056+0000 7fdf59199700 1 -- 172.21.15.31:0/586338793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400b860 con 0x7fdf540c7f30 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.056+0000 7fdf59199700 1 -- 172.21.15.31:0/586338793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4400b9c0 con 0x7fdf540c7f30 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.056+0000 7fdf59199700 1 -- 172.21.15.31:0/586338793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400bd60 con 0x7fdf540c7f30 2023-10-06T05:21:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.056+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/586338793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7f30 msgr2=0x7fdf54069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.056+0000 7fdf5c3ff700 1 --2- 172.21.15.31:0/586338793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7f30 0x7fdf54069880 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fdf440099b0 tx=0x7fdf44009250 comp rx=0 tx=0).stop 2023-10-06T05:21:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.057+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/586338793 shutdown_connections 2023-10-06T05:21:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.057+0000 7fdf5c3ff700 1 --2- 172.21.15.31:0/586338793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 0x7fdf5413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.057+0000 7fdf5c3ff700 1 --2- 172.21.15.31:0/586338793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7f30 0x7fdf54069880 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.057+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/586338793 >> 172.21.15.31:0/586338793 conn(0x7fdf540cc0b0 msgr2=0x7fdf540cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.057+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/586338793 shutdown_connections 2023-10-06T05:21:00.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.057+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/586338793 wait complete. 2023-10-06T05:21:00.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.058+0000 7fdf5c3ff700 1 Processor -- start 2023-10-06T05:21:00.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.058+0000 7fdf5c3ff700 1 -- start start 2023-10-06T05:21:00.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.059+0000 7fdf5c3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 0x7fdf5413b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.059+0000 7fdf5c3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7b60 0x7fdf540c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.059+0000 7fdf5c3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf540c7f30 0x7fdf540ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.059+0000 7fdf5c3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540ad410 con 0x7fdf540c7b60 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.059+0000 7fdf5c3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540ac800 con 0x7fdf540c7f30 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.059+0000 7fdf5c3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540acae0 con 0x7fdf540aee70 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.059+0000 7fdf5999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7b60 0x7fdf540c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.059+0000 7fdf5999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7b60 0x7fdf540c33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60500/0 (socket says 172.21.15.31:60500) 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.059+0000 7fdf5999a700 1 -- 172.21.15.31:0/846668608 learned_addr learned my addr 172.21.15.31:0/846668608 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf5a19b700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 0x7fdf5413b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf5a99c700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf540c7f30 0x7fdf540ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf5999a700 1 -- 172.21.15.31:0/846668608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 msgr2=0x7fdf5413b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf5999a700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 0x7fdf5413b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf5999a700 1 -- 172.21.15.31:0/846668608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf540c7f30 msgr2=0x7fdf540ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf5999a700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf540c7f30 0x7fdf540ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf5999a700 1 -- 172.21.15.31:0/846668608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf440094f0 con 0x7fdf540c7b60 2023-10-06T05:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf5999a700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7b60 0x7fdf540c33b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdf440090b0 tx=0x7fdf44019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf4a7fc700 1 -- 172.21.15.31:0/846668608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4401f070 con 0x7fdf540c7b60 2023-10-06T05:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf4a7fc700 1 -- 172.21.15.31:0/846668608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4401ade0 con 0x7fdf540c7b60 2023-10-06T05:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/846668608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf540acdc0 con 0x7fdf540c7b60 2023-10-06T05:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf4a7fc700 1 -- 172.21.15.31:0/846668608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf440106c0 con 0x7fdf540c7b60 2023-10-06T05:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.060+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/846668608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5413b670 con 0x7fdf540c7b60 2023-10-06T05:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.061+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/846668608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf540ad580 con 0x7fdf540c7b60 2023-10-06T05:21:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.062+0000 7fdf4a7fc700 1 -- 172.21.15.31:0/846668608 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fdf44023070 con 0x7fdf540c7b60 2023-10-06T05:21:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.063+0000 7fdf4a7fc700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf38067290 0x7fdf38069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.063+0000 7fdf4a7fc700 1 -- 172.21.15.31:0/846668608 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7fdf44014070 con 0x7fdf540c7b60 2023-10-06T05:21:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.063+0000 7fdf5a19b700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf38067290 0x7fdf38069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.064+0000 7fdf5a19b700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf38067290 0x7fdf38069740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdf50002830 tx=0x7fdf50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.067+0000 7fdf4a7fc700 1 -- 172.21.15.31:0/846668608 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf44054a60 con 0x7fdf540c7b60 2023-10-06T05:21:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.277+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/846668608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf5404c380 con 0x7fdf540c7b60 2023-10-06T05:21:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.278+0000 7fdf4a7fc700 1 -- 172.21.15.31:0/846668608 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+17156 (secure 0 0 0) 0x7fdf44058080 con 0x7fdf540c7b60 2023-10-06T05:21:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:00.280 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":59,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:20:50.325408+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:20:34.970939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T05:20:35.729829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":3}]},{"pgid":"10.23","mappings":[{"from":7,"to":3}]},{"pgid":"10.2f","mappings":[{"from":1,"to":3}]},{"pgid":"10.37","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:00.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/846668608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf38067290 msgr2=0x7fdf38069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf38067290 0x7fdf38069740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdf50002830 tx=0x7fdf50009040 comp rx=0 tx=0).stop 2023-10-06T05:21:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/846668608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7b60 msgr2=0x7fdf540c33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7b60 0x7fdf540c33b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdf440090b0 tx=0x7fdf44019000 comp rx=0 tx=0).stop 2023-10-06T05:21:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/846668608 shutdown_connections 2023-10-06T05:21:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf540c7f30 0x7fdf540ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf540aee70 0x7fdf5413b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fdf38067290 0x7fdf38069740 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 --2- 172.21.15.31:0/846668608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf540c7b60 0x7fdf540c33b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/846668608 >> 172.21.15.31:0/846668608 conn(0x7fdf540cc0b0 msgr2=0x7fdf540c24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.280+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/846668608 shutdown_connections 2023-10-06T05:21:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.281+0000 7fdf5c3ff700 1 -- 172.21.15.31:0/846668608 wait complete. 2023-10-06T05:21:00.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:21:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.496+0000 7f65cd7ff700 1 Processor -- start 2023-10-06T05:21:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.496+0000 7f65cd7ff700 1 -- start start 2023-10-06T05:21:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.497+0000 7f65cd7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 0x7f65c806e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.497+0000 7f65cd7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65c806ea90 0x7f65c80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.497+0000 7f65cd7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c806be70 con 0x7f65c813fb00 2023-10-06T05:21:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.497+0000 7f65cd7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c806ef30 con 0x7f65c806ea90 2023-10-06T05:21:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.497+0000 7f65cd7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c806ca10 con 0x7f65c813fed0 2023-10-06T05:21:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.498+0000 7f65c77fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65c806ea90 0x7f65c80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.498+0000 7f65c67fc700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 0x7f65c806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.498+0000 7f65c77fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65c806ea90 0x7f65c80cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:55094/0 (socket says 172.21.15.31:55094) 2023-10-06T05:21:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.498+0000 7f65c77fe700 1 -- 172.21.15.31:0/2511473964 learned_addr learned my addr 172.21.15.31:0/2511473964 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:00.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.498+0000 7f65c77fe700 1 -- 172.21.15.31:0/2511473964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 msgr2=0x7f65c806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.498+0000 7f65c77fe700 1 --2- 172.21.15.31:0/2511473964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 0x7f65c806e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.498+0000 7f65c77fe700 1 -- 172.21.15.31:0/2511473964 >> v1:172.21.15.62:6789/0 conn(0x7f65c813fb00 legacy=0x7f65c806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.498+0000 7f65c77fe700 1 -- 172.21.15.31:0/2511473964 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c806f130 con 0x7f65c806ea90 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.499+0000 7f65c77fe700 1 --2- 172.21.15.31:0/2511473964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c806ea90 0x7f65c80cadb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009f20 tx=0x7f65c0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b342305aaffa64c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.499+0000 7f65c5ffb700 1 -- 172.21.15.31:0/2511473964 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000e070 con 0x7f65c806ea90 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.499+0000 7f65c5ffb700 1 -- 172.21.15.31:0/2511473964 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c000b9d0 con 0x7f65c806ea90 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.499+0000 7f65c5ffb700 1 -- 172.21.15.31:0/2511473964 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c00092e0 con 0x7f65c806ea90 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.500+0000 7f65cd7ff700 1 -- 172.21.15.31:0/2511473964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c806ea90 msgr2=0x7f65c80cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.500+0000 7f65cd7ff700 1 --2- 172.21.15.31:0/2511473964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c806ea90 0x7f65c80cadb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009f20 tx=0x7f65c0009840 comp rx=0 tx=0).stop 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.500+0000 7f65cd7ff700 1 -- 172.21.15.31:0/2511473964 shutdown_connections 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.500+0000 7f65cd7ff700 1 --2- 172.21.15.31:0/2511473964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c806ea90 0x7f65c80cadb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.500+0000 7f65cd7ff700 1 --2- 172.21.15.31:0/2511473964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 0x7f65c806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.500+0000 7f65cd7ff700 1 -- 172.21.15.31:0/2511473964 >> 172.21.15.31:0/2511473964 conn(0x7f65c80cdcb0 msgr2=0x7f65c80c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.500+0000 7f65cd7ff700 1 -- 172.21.15.31:0/2511473964 shutdown_connections 2023-10-06T05:21:00.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.500+0000 7f65cd7ff700 1 -- 172.21.15.31:0/2511473964 wait complete. 2023-10-06T05:21:00.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.501+0000 7f65cd7ff700 1 Processor -- start 2023-10-06T05:21:00.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.501+0000 7f65cd7ff700 1 -- start start 2023-10-06T05:21:00.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.502+0000 7f65cd7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c806ea90 0x7f65c8156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.502+0000 7f65cd7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c813fb00 0x7f65c813b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.502+0000 7f65cd7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 0x7f65c813be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.502+0000 7f65cd7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c81419a0 con 0x7f65c813fb00 2023-10-06T05:21:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.502+0000 7f65cd7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8140dd0 con 0x7f65c806ea90 2023-10-06T05:21:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.502+0000 7f65cd7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8141070 con 0x7f65c813fed0 2023-10-06T05:21:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c813fb00 0x7f65c813b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c813fb00 0x7f65c813b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60506/0 (socket says 172.21.15.31:60506) 2023-10-06T05:21:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c67fc700 1 -- 172.21.15.31:0/995186930 learned_addr learned my addr 172.21.15.31:0/995186930 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c6ffd700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c806ea90 0x7f65c8156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c77fe700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 0x7f65c813be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c67fc700 1 -- 172.21.15.31:0/995186930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 msgr2=0x7f65c813be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c67fc700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 0x7f65c813be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c67fc700 1 -- 172.21.15.31:0/995186930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c806ea90 msgr2=0x7f65c8156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c67fc700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c806ea90 0x7f65c8156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c67fc700 1 -- 172.21.15.31:0/995186930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0009ae0 con 0x7f65c813fb00 2023-10-06T05:21:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c77fe700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 0x7f65c813be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c6ffd700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c806ea90 0x7f65c8156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.503+0000 7f65c67fc700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c813fb00 0x7f65c813b930 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00ec80 tx=0x7f65bc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.504+0000 7f65aeffd700 1 -- 172.21.15.31:0/995186930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00c9d0 con 0x7f65c813fb00 2023-10-06T05:21:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.504+0000 7f65aeffd700 1 -- 172.21.15.31:0/995186930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc00cb30 con 0x7f65c813fb00 2023-10-06T05:21:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.504+0000 7f65cd7ff700 1 -- 172.21.15.31:0/995186930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c8141350 con 0x7f65c813fb00 2023-10-06T05:21:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.504+0000 7f65aeffd700 1 -- 172.21.15.31:0/995186930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc013470 con 0x7f65c813fb00 2023-10-06T05:21:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.504+0000 7f65cd7ff700 1 -- 172.21.15.31:0/995186930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c81574c0 con 0x7f65c813fb00 2023-10-06T05:21:00.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.505+0000 7f65cd7ff700 1 -- 172.21.15.31:0/995186930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c804c380 con 0x7f65c813fb00 2023-10-06T05:21:00.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.506+0000 7f65aeffd700 1 -- 172.21.15.31:0/995186930 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f65bc013860 con 0x7f65c813fb00 2023-10-06T05:21:00.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.507+0000 7f65aeffd700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65b4067290 0x7f65b4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.507+0000 7f65aeffd700 1 -- 172.21.15.31:0/995186930 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6203+0+0 (secure 0 0 0) 0x7f65bc085390 con 0x7f65c813fb00 2023-10-06T05:21:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.507+0000 7f65c6ffd700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65b4067290 0x7f65b4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.507+0000 7f65c6ffd700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65b4067290 0x7f65b4069740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f65b8007fd0 tx=0x7f65b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.511+0000 7f65aeffd700 1 -- 172.21.15.31:0/995186930 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65bc051fc0 con 0x7f65c813fb00 2023-10-06T05:21:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.763+0000 7f65cd7ff700 1 -- 172.21.15.31:0/995186930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f65c806c310 con 0x7f65c813fb00 2023-10-06T05:21:00.765 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:21:00.764+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:21:00.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.769+0000 7f65aeffd700 1 -- 172.21.15.31:0/995186930 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7f65bc0555e0 con 0x7f65c813fb00 2023-10-06T05:21:00.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 -- 172.21.15.31:0/995186930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65b4067290 msgr2=0x7f65b4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65b4067290 0x7f65b4069740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f65b8007fd0 tx=0x7f65b8009040 comp rx=0 tx=0).stop 2023-10-06T05:21:00.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 -- 172.21.15.31:0/995186930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c813fb00 msgr2=0x7f65c813b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c813fb00 0x7f65c813b930 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00ec80 tx=0x7f65bc009120 comp rx=0 tx=0).stop 2023-10-06T05:21:00.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 -- 172.21.15.31:0/995186930 shutdown_connections 2023-10-06T05:21:00.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65c813fed0 0x7f65c813be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c806ea90 0x7f65c8156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f65b4067290 0x7f65b4069740 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 --2- 172.21.15.31:0/995186930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65c813fb00 0x7f65c813b930 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 -- 172.21.15.31:0/995186930 >> 172.21.15.31:0/995186930 conn(0x7f65c80cdcb0 msgr2=0x7f65c8158e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.771+0000 7f65cd7ff700 1 -- 172.21.15.31:0/995186930 shutdown_connections 2023-10-06T05:21:00.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.772+0000 7f65cd7ff700 1 -- 172.21.15.31:0/995186930 wait complete. 2023-10-06T05:21:00.775 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:21:00.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:21:00.837 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:21:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.987+0000 7fb79bbff700 1 Processor -- start 2023-10-06T05:21:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.987+0000 7fb79bbff700 1 -- start start 2023-10-06T05:21:00.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.988+0000 7fb79bbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb794159560 0x7fb7940cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.988+0000 7fb79bbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7940b2b80 0x7fb7940cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.988+0000 7fb79bbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79415d2d0 con 0x7fb794159930 2023-10-06T05:21:00.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.988+0000 7fb79bbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79415d5b0 con 0x7fb7940b2b80 2023-10-06T05:21:00.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.988+0000 7fb79bbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79415d890 con 0x7fb794159560 2023-10-06T05:21:00.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.988+0000 7fb79a19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7940b2b80 0x7fb7940cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.988+0000 7fb79a19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7940b2b80 0x7fb7940cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60518/0 (socket says 172.21.15.31:60518) 2023-10-06T05:21:00.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.988+0000 7fb79a19c700 1 -- 172.21.15.31:0/4280879006 learned_addr learned my addr 172.21.15.31:0/4280879006 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:00.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.988+0000 7fb79999b700 1 --2- 172.21.15.31:0/4280879006 >> v2:172.21.15.62:3300/0 conn(0x7fb794159560 0x7fb7940cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.989+0000 7fb79a19c700 1 -- 172.21.15.31:0/4280879006 >> v2:172.21.15.62:3300/0 conn(0x7fb794159560 msgr2=0x7fb7940cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:00.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.989+0000 7fb79a19c700 1 --2- 172.21.15.31:0/4280879006 >> v2:172.21.15.62:3300/0 conn(0x7fb794159560 0x7fb7940cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.989+0000 7fb79a19c700 1 -- 172.21.15.31:0/4280879006 >> v1:172.21.15.62:6789/0 conn(0x7fb794159930 legacy=0x7fb7940b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.989+0000 7fb79a19c700 1 -- 172.21.15.31:0/4280879006 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79415db70 con 0x7fb7940b2b80 2023-10-06T05:21:00.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.989+0000 7fb798999700 1 -- 172.21.15.31:0/4280879006 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766940764 0 0) 0x7fb79415d2d0 con 0x7fb794159930 2023-10-06T05:21:00.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.989+0000 7fb79a19c700 1 --2- 172.21.15.31:0/4280879006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 0x7fb7940cd3b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb794063d80 tx=0x7fb78c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d05e1f5c665b0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:00.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.989+0000 7fb798999700 1 -- 172.21.15.31:0/4280879006 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00e070 con 0x7fb7940b2b80 2023-10-06T05:21:00.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.989+0000 7fb798999700 1 -- 172.21.15.31:0/4280879006 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c00ba20 con 0x7fb7940b2b80 2023-10-06T05:21:00.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.989+0000 7fb798999700 1 -- 172.21.15.31:0/4280879006 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0092e0 con 0x7fb7940b2b80 2023-10-06T05:21:00.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.990+0000 7fb79bbff700 1 -- 172.21.15.31:0/4280879006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 msgr2=0x7fb7940cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:00.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.990+0000 7fb79bbff700 1 --2- 172.21.15.31:0/4280879006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 0x7fb7940cd3b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb794063d80 tx=0x7fb78c009840 comp rx=0 tx=0).stop 2023-10-06T05:21:00.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.990+0000 7fb79bbff700 1 -- 172.21.15.31:0/4280879006 shutdown_connections 2023-10-06T05:21:00.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.990+0000 7fb79bbff700 1 --2- 172.21.15.31:0/4280879006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 0x7fb7940cd3b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.990+0000 7fb79bbff700 1 --2- 172.21.15.31:0/4280879006 >> v2:172.21.15.62:3300/0 conn(0x7fb794159560 0x7fb7940cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:00.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.990+0000 7fb79bbff700 1 -- 172.21.15.31:0/4280879006 >> 172.21.15.31:0/4280879006 conn(0x7fb79406ec90 msgr2=0x7fb79406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:00.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.990+0000 7fb79bbff700 1 -- 172.21.15.31:0/4280879006 shutdown_connections 2023-10-06T05:21:00.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.990+0000 7fb79bbff700 1 -- 172.21.15.31:0/4280879006 wait complete. 2023-10-06T05:21:00.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.992+0000 7fb79bbff700 1 Processor -- start 2023-10-06T05:21:00.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.992+0000 7fb79bbff700 1 -- start start 2023-10-06T05:21:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79bbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 0x7fb7940d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79bbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb794159560 0x7fb79416c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79bbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb794159930 0x7fb79415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79bbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940d0bc0 con 0x7fb7940b2b80 2023-10-06T05:21:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79bbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940c4090 con 0x7fb794159930 2023-10-06T05:21:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79bbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940c4370 con 0x7fb794159560 2023-10-06T05:21:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79999b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 0x7fb7940d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:00.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79999b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 0x7fb7940d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60528/0 (socket says 172.21.15.31:60528) 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79999b700 1 -- 172.21.15.31:0/1110030920 learned_addr learned my addr 172.21.15.31:0/1110030920 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79a19c700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb794159930 0x7fb79415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79919a700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb794159560 0x7fb79416c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79999b700 1 -- 172.21.15.31:0/1110030920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb794159560 msgr2=0x7fb79416c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79999b700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb794159560 0x7fb79416c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79999b700 1 -- 172.21.15.31:0/1110030920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb794159930 msgr2=0x7fb79415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79999b700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb794159930 0x7fb79415be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79999b700 1 -- 172.21.15.31:0/1110030920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c009ae0 con 0x7fb7940b2b80 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.993+0000 7fb79919a700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb794159560 0x7fb79416c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.994+0000 7fb79999b700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 0x7fb7940d0360 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb79000ec80 tx=0x7fb7900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:01.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.994+0000 7fb789ffb700 1 -- 172.21.15.31:0/1110030920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000c9d0 con 0x7fb7940b2b80 2023-10-06T05:21:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.994+0000 7fb789ffb700 1 -- 172.21.15.31:0/1110030920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79000cb30 con 0x7fb7940b2b80 2023-10-06T05:21:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.994+0000 7fb79bbff700 1 -- 172.21.15.31:0/1110030920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7940c4650 con 0x7fb7940b2b80 2023-10-06T05:21:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.994+0000 7fb789ffb700 1 -- 172.21.15.31:0/1110030920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790013470 con 0x7fb7940b2b80 2023-10-06T05:21:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.994+0000 7fb79bbff700 1 -- 172.21.15.31:0/1110030920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7940d08a0 con 0x7fb7940b2b80 2023-10-06T05:21:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.995+0000 7fb79bbff700 1 -- 172.21.15.31:0/1110030920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79404c380 con 0x7fb7940b2b80 2023-10-06T05:21:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.996+0000 7fb789ffb700 1 -- 172.21.15.31:0/1110030920 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb790013820 con 0x7fb7940b2b80 2023-10-06T05:21:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.997+0000 7fb789ffb700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb778067290 0x7fb778069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.997+0000 7fb789ffb700 1 -- 172.21.15.31:0/1110030920 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6289+0+0 (secure 0 0 0) 0x7fb7900850f0 con 0x7fb7940b2b80 2023-10-06T05:21:01.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.997+0000 7fb79919a700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb778067290 0x7fb778069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:01.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:00.998+0000 7fb79919a700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb778067290 0x7fb778069740 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb7940cf2c0 tx=0x7fb784009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:01.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.001+0000 7fb789ffb700 1 -- 172.21.15.31:0/1110030920 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb790051cc0 con 0x7fb7940b2b80 2023-10-06T05:21:01.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:01.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.255+0000 7fb79bbff700 1 -- 172.21.15.31:0/1110030920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb79415de50 con 0x7fb7940b2b80 2023-10-06T05:21:01.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.770+0000 7fb789ffb700 1 -- 172.21.15.31:0/1110030920 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb7900552e0 con 0x7fb7940b2b80 2023-10-06T05:21:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 -- 172.21.15.31:0/1110030920 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb778067290 msgr2=0x7fb778069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb778067290 0x7fb778069740 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb7940cf2c0 tx=0x7fb784009040 comp rx=0 tx=0).stop 2023-10-06T05:21:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 -- 172.21.15.31:0/1110030920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 msgr2=0x7fb7940d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 0x7fb7940d0360 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb79000ec80 tx=0x7fb7900091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 -- 172.21.15.31:0/1110030920 shutdown_connections 2023-10-06T05:21:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb794159930 0x7fb79415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb778067290 0x7fb778069740 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7940b2b80 0x7fb7940d0360 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 --2- 172.21.15.31:0/1110030920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb794159560 0x7fb79416c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 -- 172.21.15.31:0/1110030920 >> 172.21.15.31:0/1110030920 conn(0x7fb79406ec90 msgr2=0x7fb794068220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 -- 172.21.15.31:0/1110030920 shutdown_connections 2023-10-06T05:21:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.773+0000 7fb79bbff700 1 -- 172.21.15.31:0/1110030920 wait complete. 2023-10-06T05:21:01.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:21:01.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.996+0000 7fd5a8fff700 1 Processor -- start 2023-10-06T05:21:01.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.996+0000 7fd5a8fff700 1 -- start start 2023-10-06T05:21:01.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.998+0000 7fd5a8fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd5a415a990 0x7fd5a40cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.998+0000 7fd5a8fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a40cc5b0 con 0x7fd5a4142c20 2023-10-06T05:21:02.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.998+0000 7fd5a8fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a415d210 con 0x7fd5a4142ff0 2023-10-06T05:21:02.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.998+0000 7fd5a8fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a40cc810 con 0x7fd5a415a990 2023-10-06T05:21:02.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.998+0000 7fd5a259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd5a4142c20 0x7fd5a40cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39590/0 (socket says 172.21.15.31:39590) 2023-10-06T05:21:02.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.998+0000 7fd5a259c700 1 -- 172.21.15.31:0/965736038 learned_addr learned my addr 172.21.15.31:0/965736038 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:02.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.998+0000 7fd5a1d9b700 1 --2- 172.21.15.31:0/965736038 >> v2:172.21.15.62:3300/0 conn(0x7fd5a415a990 0x7fd5a40cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.998+0000 7fd5a159a700 1 -- 172.21.15.31:0/965736038 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812038392 0 0) 0x7fd5a40cc5b0 con 0x7fd5a4142c20 2023-10-06T05:21:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.998+0000 7fd5a159a700 1 -- 172.21.15.31:0/965736038 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd58c003570 con 0x7fd5a4142c20 2023-10-06T05:21:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.999+0000 7fd5a159a700 1 -- 172.21.15.31:0/965736038 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419725861 0 0) 0x7fd5a415d210 con 0x7fd5a4142ff0 2023-10-06T05:21:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.999+0000 7fd5a159a700 1 -- 172.21.15.31:0/965736038 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a40cc5b0 con 0x7fd5a4142ff0 2023-10-06T05:21:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.999+0000 7fd5a1d9b700 1 -- 172.21.15.31:0/965736038 >> v1:172.21.15.62:6789/0 conn(0x7fd5a4142ff0 legacy=0x7fd5a4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.999+0000 7fd5a1d9b700 1 -- 172.21.15.31:0/965736038 >> v1:172.21.15.31:6789/0 conn(0x7fd5a4142c20 legacy=0x7fd5a40cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.999+0000 7fd5a1d9b700 1 -- 172.21.15.31:0/965736038 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a415d4b0 con 0x7fd5a415a990 2023-10-06T05:21:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.999+0000 7fd5a1d9b700 1 --2- 172.21.15.31:0/965736038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a415a990 0x7fd5a40cacd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5980098e0 tx=0x7fd5980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9473bb5ca256e20e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.999+0000 7fd5a159a700 1 -- 172.21.15.31:0/965736038 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598011630 con 0x7fd5a415a990 2023-10-06T05:21:02.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.999+0000 7fd5a159a700 1 -- 172.21.15.31:0/965736038 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd598011790 con 0x7fd5a415a990 2023-10-06T05:21:02.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:01.999+0000 7fd5a159a700 1 -- 172.21.15.31:0/965736038 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598011b30 con 0x7fd5a415a990 2023-10-06T05:21:02.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.000+0000 7fd5a8fff700 1 -- 172.21.15.31:0/965736038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a415a990 msgr2=0x7fd5a40cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.000+0000 7fd5a8fff700 1 --2- 172.21.15.31:0/965736038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a415a990 0x7fd5a40cacd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5980098e0 tx=0x7fd5980091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.000+0000 7fd5a8fff700 1 -- 172.21.15.31:0/965736038 shutdown_connections 2023-10-06T05:21:02.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.000+0000 7fd5a8fff700 1 --2- 172.21.15.31:0/965736038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a415a990 0x7fd5a40cacd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.000+0000 7fd5a8fff700 1 -- 172.21.15.31:0/965736038 >> 172.21.15.31:0/965736038 conn(0x7fd5a40ce2d0 msgr2=0x7fd5a40ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:02.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.000+0000 7fd5a8fff700 1 -- 172.21.15.31:0/965736038 shutdown_connections 2023-10-06T05:21:02.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.000+0000 7fd5a8fff700 1 -- 172.21.15.31:0/965736038 wait complete. 2023-10-06T05:21:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.001+0000 7fd5a8fff700 1 Processor -- start 2023-10-06T05:21:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.002+0000 7fd5a8fff700 1 -- start start 2023-10-06T05:21:02.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a8fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5a4142c20 0x7fd5a40ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a8fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a4142ff0 0x7fd5a40b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a8fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a415a990 0x7fd5a40ab290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a8fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40ac9f0 con 0x7fd5a4142ff0 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a8fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40abde0 con 0x7fd5a415a990 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a8fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40ac0c0 con 0x7fd5a4142c20 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a4142ff0 0x7fd5a40b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a4142ff0 0x7fd5a40b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60544/0 (socket says 172.21.15.31:60544) 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a1d9b700 1 -- 172.21.15.31:0/843401728 learned_addr learned my addr 172.21.15.31:0/843401728 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a259c700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5a4142c20 0x7fd5a40ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a2d9d700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a415a990 0x7fd5a40ab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a1d9b700 1 -- 172.21.15.31:0/843401728 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5a4142c20 msgr2=0x7fd5a40ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a1d9b700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5a4142c20 0x7fd5a40ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a1d9b700 1 -- 172.21.15.31:0/843401728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a415a990 msgr2=0x7fd5a40ab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a1d9b700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a415a990 0x7fd5a40ab290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a1d9b700 1 -- 172.21.15.31:0/843401728 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd598009420 con 0x7fd5a4142ff0 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.003+0000 7fd5a2d9d700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a415a990 0x7fd5a40ab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.004+0000 7fd5a1d9b700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a4142ff0 0x7fd5a40b1120 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40cc740 tx=0x7fd598017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.004+0000 7fd5927fc700 1 -- 172.21.15.31:0/843401728 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59801f070 con 0x7fd5a4142ff0 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.004+0000 7fd5927fc700 1 -- 172.21.15.31:0/843401728 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd598000c90 con 0x7fd5a4142ff0 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.004+0000 7fd5a8fff700 1 -- 172.21.15.31:0/843401728 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a40ac3a0 con 0x7fd5a4142ff0 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.004+0000 7fd5927fc700 1 -- 172.21.15.31:0/843401728 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598025420 con 0x7fd5a4142ff0 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.004+0000 7fd5a8fff700 1 -- 172.21.15.31:0/843401728 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a4153540 con 0x7fd5a4142ff0 2023-10-06T05:21:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.005+0000 7fd5a8fff700 1 -- 172.21.15.31:0/843401728 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a40acb60 con 0x7fd5a4142ff0 2023-10-06T05:21:02.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.006+0000 7fd5927fc700 1 -- 172.21.15.31:0/843401728 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd598023070 con 0x7fd5a4142ff0 2023-10-06T05:21:02.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.008+0000 7fd5927fc700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd584067290 0x7fd584069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:02.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.008+0000 7fd5927fc700 1 -- 172.21.15.31:0/843401728 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6289+0+0 (secure 0 0 0) 0x7fd598085fb0 con 0x7fd5a4142ff0 2023-10-06T05:21:02.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.008+0000 7fd5a259c700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd584067290 0x7fd584069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:02.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.009+0000 7fd5a259c700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd584067290 0x7fd584069740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd5940058d0 tx=0x7fd594009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:02.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.012+0000 7fd5927fc700 1 -- 172.21.15.31:0/843401728 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd598052b80 con 0x7fd5a4142ff0 2023-10-06T05:21:02.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.218+0000 7fd5a8fff700 1 -- 172.21.15.31:0/843401728 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd5a404c380 con 0x7fd5a4142ff0 2023-10-06T05:21:02.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.780+0000 7fd5927fc700 1 -- 172.21.15.31:0/843401728 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd59802a020 con 0x7fd5a4142ff0 2023-10-06T05:21:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.785+0000 7fd5a8fff700 1 -- 172.21.15.31:0/843401728 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd584067290 msgr2=0x7fd584069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.785+0000 7fd5a8fff700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd584067290 0x7fd584069740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd5940058d0 tx=0x7fd594009040 comp rx=0 tx=0).stop 2023-10-06T05:21:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.785+0000 7fd5a8fff700 1 -- 172.21.15.31:0/843401728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a4142ff0 msgr2=0x7fd5a40b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.785+0000 7fd5a8fff700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a4142ff0 0x7fd5a40b1120 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40cc740 tx=0x7fd598017000 comp rx=0 tx=0).stop 2023-10-06T05:21:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.786+0000 7fd5a8fff700 1 -- 172.21.15.31:0/843401728 shutdown_connections 2023-10-06T05:21:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.786+0000 7fd5a8fff700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a415a990 0x7fd5a40ab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.786+0000 7fd5a8fff700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5a4142c20 0x7fd5a40ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.786+0000 7fd5a8fff700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fd584067290 0x7fd584069740 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.786+0000 7fd5a8fff700 1 --2- 172.21.15.31:0/843401728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a4142ff0 0x7fd5a40b1120 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.786+0000 7fd5a8fff700 1 -- 172.21.15.31:0/843401728 >> 172.21.15.31:0/843401728 conn(0x7fd5a40ce2d0 msgr2=0x7fd5a415cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.786+0000 7fd5a8fff700 1 -- 172.21.15.31:0/843401728 shutdown_connections 2023-10-06T05:21:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:02.786+0000 7fd5a8fff700 1 -- 172.21.15.31:0/843401728 wait complete. 2023-10-06T05:21:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:21:02.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:21:03.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.015+0000 7f8f4efff700 1 Processor -- start 2023-10-06T05:21:03.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.015+0000 7f8f4efff700 1 -- start start 2023-10-06T05:21:03.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.017+0000 7f8f4efff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8f4815d1d0 0x7f8f480cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.017+0000 7f8f4efff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8f480cb960 0x7f8f480ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.017+0000 7f8f4efff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480cdf70 con 0x7f8f48159510 2023-10-06T05:21:03.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.017+0000 7f8f4efff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480cbde0 con 0x7f8f4815d1d0 2023-10-06T05:21:03.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.017+0000 7f8f4efff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480ce250 con 0x7f8f480cb960 2023-10-06T05:21:03.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.017+0000 7f8f4cd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8f48159510 0x7f8f4815a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39602/0 (socket says 172.21.15.31:39602) 2023-10-06T05:21:03.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.017+0000 7f8f4cd9b700 1 -- 172.21.15.31:0/1018592999 learned_addr learned my addr 172.21.15.31:0/1018592999 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:03.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.018+0000 7f8f47fff700 1 --2- 172.21.15.31:0/1018592999 >> v2:172.21.15.31:3300/0 conn(0x7f8f4815d1d0 0x7f8f480cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.018+0000 7f8f4d59c700 1 --2- 172.21.15.31:0/1018592999 >> v2:172.21.15.62:3300/0 conn(0x7f8f480cb960 0x7f8f480ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.018+0000 7f8f477fe700 1 -- 172.21.15.31:0/1018592999 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392718822 0 0) 0x7f8f480cdf70 con 0x7f8f48159510 2023-10-06T05:21:03.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.018+0000 7f8f477fe700 1 -- 172.21.15.31:0/1018592999 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f38003570 con 0x7f8f48159510 2023-10-06T05:21:03.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.018+0000 7f8f47fff700 1 -- 172.21.15.31:0/1018592999 >> v2:172.21.15.62:3300/0 conn(0x7f8f480cb960 msgr2=0x7f8f480ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:03.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.018+0000 7f8f47fff700 1 --2- 172.21.15.31:0/1018592999 >> v2:172.21.15.62:3300/0 conn(0x7f8f480cb960 0x7f8f480ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.018+0000 7f8f47fff700 1 -- 172.21.15.31:0/1018592999 >> v1:172.21.15.31:6789/0 conn(0x7f8f48159510 legacy=0x7f8f4815a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.018+0000 7f8f47fff700 1 -- 172.21.15.31:0/1018592999 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4815d970 con 0x7f8f4815d1d0 2023-10-06T05:21:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.018+0000 7f8f47fff700 1 --2- 172.21.15.31:0/1018592999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4815d1d0 0x7f8f480cb420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8f340098e0 tx=0x7f8f340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff8ba5ac5443ad1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f477fe700 1 -- 172.21.15.31:0/1018592999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34011630 con 0x7f8f4815d1d0 2023-10-06T05:21:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f477fe700 1 -- 172.21.15.31:0/1018592999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f34011790 con 0x7f8f4815d1d0 2023-10-06T05:21:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f477fe700 1 -- 172.21.15.31:0/1018592999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34011b30 con 0x7f8f4815d1d0 2023-10-06T05:21:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f4efff700 1 -- 172.21.15.31:0/1018592999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4815d1d0 msgr2=0x7f8f480cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f4efff700 1 --2- 172.21.15.31:0/1018592999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4815d1d0 0x7f8f480cb420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8f340098e0 tx=0x7f8f340091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f4efff700 1 -- 172.21.15.31:0/1018592999 shutdown_connections 2023-10-06T05:21:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f4efff700 1 --2- 172.21.15.31:0/1018592999 >> v2:172.21.15.62:3300/0 conn(0x7f8f480cb960 0x7f8f480ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f4efff700 1 --2- 172.21.15.31:0/1018592999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4815d1d0 0x7f8f480cb420 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f4efff700 1 -- 172.21.15.31:0/1018592999 >> 172.21.15.31:0/1018592999 conn(0x7f8f480996a0 msgr2=0x7f8f4806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f4efff700 1 -- 172.21.15.31:0/1018592999 shutdown_connections 2023-10-06T05:21:03.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.019+0000 7f8f4efff700 1 -- 172.21.15.31:0/1018592999 wait complete. 2023-10-06T05:21:03.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.021+0000 7f8f4efff700 1 Processor -- start 2023-10-06T05:21:03.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.021+0000 7f8f4efff700 1 -- start start 2023-10-06T05:21:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.022+0000 7f8f4efff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f480cb960 0x7f8f480c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.022+0000 7f8f4efff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f48159510 0x7f8f480c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.022+0000 7f8f4efff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4815d1d0 0x7f8f480c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.022+0000 7f8f4efff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c5190 con 0x7f8f480cb960 2023-10-06T05:21:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.022+0000 7f8f4efff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c4580 con 0x7f8f4815d1d0 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.022+0000 7f8f4efff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c4860 con 0x7f8f48159510 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.022+0000 7f8f4cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f480cb960 0x7f8f480c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.022+0000 7f8f4cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f480cb960 0x7f8f480c5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60564/0 (socket says 172.21.15.31:60564) 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.022+0000 7f8f4cd9b700 1 -- 172.21.15.31:0/3042927523 learned_addr learned my addr 172.21.15.31:0/3042927523 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.022+0000 7f8f47fff700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f48159510 0x7f8f480c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f4cd9b700 1 -- 172.21.15.31:0/3042927523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f48159510 msgr2=0x7f8f480c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f4d59c700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4815d1d0 0x7f8f480c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f4cd9b700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f48159510 0x7f8f480c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f4cd9b700 1 -- 172.21.15.31:0/3042927523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4815d1d0 msgr2=0x7f8f480c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f4cd9b700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4815d1d0 0x7f8f480c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f4cd9b700 1 -- 172.21.15.31:0/3042927523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009420 con 0x7f8f480cb960 2023-10-06T05:21:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f47fff700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f48159510 0x7f8f480c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f4d59c700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4815d1d0 0x7f8f480c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f4cd9b700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f480cb960 0x7f8f480c5620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c790 tx=0x7f8f3c007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f44ff9700 1 -- 172.21.15.31:0/3042927523 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c007300 con 0x7f8f480cb960 2023-10-06T05:21:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f44ff9700 1 -- 172.21.15.31:0/3042927523 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c007460 con 0x7f8f480cb960 2023-10-06T05:21:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f44ff9700 1 -- 172.21.15.31:0/3042927523 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0057f0 con 0x7f8f480cb960 2023-10-06T05:21:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f4efff700 1 -- 172.21.15.31:0/3042927523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f480c4b40 con 0x7f8f480cb960 2023-10-06T05:21:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.023+0000 7f8f4efff700 1 -- 172.21.15.31:0/3042927523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f480c4230 con 0x7f8f480cb960 2023-10-06T05:21:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.024+0000 7f8f4efff700 1 -- 172.21.15.31:0/3042927523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f480c5470 con 0x7f8f480cb960 2023-10-06T05:21:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.025+0000 7f8f44ff9700 1 -- 172.21.15.31:0/3042927523 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8f3c005c20 con 0x7f8f480cb960 2023-10-06T05:21:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.026+0000 7f8f44ff9700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f2c067280 0x7f8f2c069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.026+0000 7f8f44ff9700 1 -- 172.21.15.31:0/3042927523 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5822+0+0 (secure 0 0 0) 0x7f8f3c084790 con 0x7f8f480cb960 2023-10-06T05:21:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.026+0000 7f8f47fff700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f2c067280 0x7f8f2c069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.027+0000 7f8f47fff700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f2c067280 0x7f8f2c069730 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f480c3d60 tx=0x7f8f34017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:03.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.031+0000 7f8f44ff9700 1 -- 172.21.15.31:0/3042927523 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f3c0514e0 con 0x7f8f480cb960 2023-10-06T05:21:03.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.233+0000 7f8f4efff700 1 -- 172.21.15.31:0/3042927523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8f4808a770 con 0x7f8f480cb960 2023-10-06T05:21:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.782+0000 7f8f44ff9700 1 -- 172.21.15.31:0/3042927523 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8f3c054b00 con 0x7f8f480cb960 2023-10-06T05:21:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.789+0000 7f8f4efff700 1 -- 172.21.15.31:0/3042927523 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f2c067280 msgr2=0x7f8f2c069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.789+0000 7f8f4efff700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f2c067280 0x7f8f2c069730 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f480c3d60 tx=0x7f8f34017000 comp rx=0 tx=0).stop 2023-10-06T05:21:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.789+0000 7f8f4efff700 1 -- 172.21.15.31:0/3042927523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f480cb960 msgr2=0x7f8f480c5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:03.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.789+0000 7f8f4efff700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f480cb960 0x7f8f480c5620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c790 tx=0x7f8f3c007c80 comp rx=0 tx=0).stop 2023-10-06T05:21:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.790+0000 7f8f4efff700 1 -- 172.21.15.31:0/3042927523 shutdown_connections 2023-10-06T05:21:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.790+0000 7f8f4efff700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f4815d1d0 0x7f8f480c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.790+0000 7f8f4efff700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f8f2c067280 0x7f8f2c069730 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.790+0000 7f8f4efff700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f480cb960 0x7f8f480c5620 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.790+0000 7f8f4efff700 1 --2- 172.21.15.31:0/3042927523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f48159510 0x7f8f480c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:03.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.790+0000 7f8f4efff700 1 -- 172.21.15.31:0/3042927523 >> 172.21.15.31:0/3042927523 conn(0x7f8f480996a0 msgr2=0x7f8f4815a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.790+0000 7f8f4efff700 1 -- 172.21.15.31:0/3042927523 shutdown_connections 2023-10-06T05:21:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:03.790+0000 7f8f4efff700 1 -- 172.21.15.31:0/3042927523 wait complete. 2023-10-06T05:21:03.793 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:21:03.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:21:04.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.013+0000 7f05713ff700 1 Processor -- start 2023-10-06T05:21:04.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.013+0000 7f05713ff700 1 -- start start 2023-10-06T05:21:04.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.014+0000 7f05713ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c0c7190 0x7f056c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.014+0000 7f05713ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c068250 con 0x7f056c0c7560 2023-10-06T05:21:04.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.014+0000 7f05713ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c06ac80 con 0x7f056c0aef00 2023-10-06T05:21:04.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.014+0000 7f05713ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c067f20 con 0x7f056c0c7190 2023-10-06T05:21:04.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.014+0000 7f056b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f056c0c7560 0x7f056c0686e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39618/0 (socket says 172.21.15.31:39618) 2023-10-06T05:21:04.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.014+0000 7f056b7fe700 1 -- 172.21.15.31:0/132565234 learned_addr learned my addr 172.21.15.31:0/132565234 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:04.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.014+0000 7f056affd700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c0c7190 0x7f056c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202961119 0 0) 0x7f056c068250 con 0x7f056c0c7560 2023-10-06T05:21:04.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0554003570 con 0x7f056c0c7560 2023-10-06T05:21:04.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039631530 0 0) 0x7f056c06ac80 con 0x7f056c0aef00 2023-10-06T05:21:04.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f056c068250 con 0x7f056c0aef00 2023-10-06T05:21:04.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2583951020 0 0) 0x7f0554003570 con 0x7f056c0c7560 2023-10-06T05:21:04.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c0c7190 msgr2=0x7f056c0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:04.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c0c7190 0x7f056c0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 >> v1:172.21.15.62:6789/0 conn(0x7f056c0aef00 legacy=0x7f056c0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c06aed0 con 0x7f056c0c7560 2023-10-06T05:21:04.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f05640017f0 con 0x7f056c0c7560 2023-10-06T05:21:04.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 >> v1:172.21.15.31:6789/0 conn(0x7f056c0c7560 legacy=0x7f056c0686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c0c7190 0x7f0554003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 0x7f0554004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0554004d50 0x7f05540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c067f20 con 0x7f05540041d0 2023-10-06T05:21:04.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055400bbf0 con 0x7f0554004d50 2023-10-06T05:21:04.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055400bdb0 con 0x7f056c0c7190 2023-10-06T05:21:04.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.015+0000 7f056a7fc700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 0x7f0554004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.016+0000 7f056affd700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c0c7190 0x7f0554003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.016+0000 7f056b7fe700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0554004d50 0x7f05540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.016+0000 7f056a7fc700 1 -- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c0c7190 msgr2=0x7f0554003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:04.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.016+0000 7f056a7fc700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c0c7190 0x7f0554003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.016+0000 7f056a7fc700 1 -- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0554004d50 msgr2=0x7f05540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:04.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.016+0000 7f056a7fc700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0554004d50 0x7f05540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.016+0000 7f056a7fc700 1 -- 172.21.15.31:0/132565234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0554003570 con 0x7f05540041d0 2023-10-06T05:21:04.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.016+0000 7f056a7fc700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 0x7f0554004810 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0560001a10 tx=0x7f0560007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f931eca7148baf05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:04.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.016+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000c5d0 con 0x7f05540041d0 2023-10-06T05:21:04.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.017+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056000c730 con 0x7f05540041d0 2023-10-06T05:21:04.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.017+0000 7f0569ffb700 1 -- 172.21.15.31:0/132565234 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000cad0 con 0x7f05540041d0 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.017+0000 7f05713ff700 1 -- 172.21.15.31:0/132565234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 msgr2=0x7f0554004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.017+0000 7f05713ff700 1 --2- 172.21.15.31:0/132565234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 0x7f0554004810 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0560001a10 tx=0x7f0560007d30 comp rx=0 tx=0).stop 2023-10-06T05:21:04.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.017+0000 7f056b7fe700 1 -- 172.21.15.31:0/132565234 reap_dead start 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.017+0000 7f05713ff700 1 -- 172.21.15.31:0/132565234 shutdown_connections 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.017+0000 7f05713ff700 1 -- 172.21.15.31:0/132565234 >> 172.21.15.31:0/132565234 conn(0x7f056c0cd3d0 msgr2=0x7f056c0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.017+0000 7f05713ff700 1 -- 172.21.15.31:0/132565234 shutdown_connections 2023-10-06T05:21:04.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.017+0000 7f05713ff700 1 -- 172.21.15.31:0/132565234 wait complete. 2023-10-06T05:21:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.019+0000 7f05713ff700 1 Processor -- start 2023-10-06T05:21:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.019+0000 7f05713ff700 1 -- start start 2023-10-06T05:21:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f05713ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 0x7f056c0c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f05713ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c15c3a0 0x7f056c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f05713ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f056c15c810 0x7f056c15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f05713ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0c1960 con 0x7f05540041d0 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f05713ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0c1c00 con 0x7f056c15c810 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f05713ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0c1ee0 con 0x7f056c15c3a0 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 0x7f056c0c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 0x7f056c0c29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60586/0 (socket says 172.21.15.31:60586) 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056affd700 1 -- 172.21.15.31:0/1797914825 learned_addr learned my addr 172.21.15.31:0/1797914825 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056a7fc700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c15c3a0 0x7f056c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056b7fe700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f056c15c810 0x7f056c15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056affd700 1 -- 172.21.15.31:0/1797914825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c15c3a0 msgr2=0x7f056c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056affd700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c15c3a0 0x7f056c15b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056affd700 1 -- 172.21.15.31:0/1797914825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f056c15c810 msgr2=0x7f056c15bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056affd700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f056c15c810 0x7f056c15bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056affd700 1 -- 172.21.15.31:0/1797914825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c0c21c0 con 0x7f05540041d0 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056b7fe700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f056c15c810 0x7f056c15bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.021+0000 7f056a7fc700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c15c3a0 0x7f056c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.022+0000 7f056affd700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 0x7f056c0c29c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f055c011d40 tx=0x7f055c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.022+0000 7f0552ffd700 1 -- 172.21.15.31:0/1797914825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c00fac0 con 0x7f05540041d0 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.022+0000 7f0552ffd700 1 -- 172.21.15.31:0/1797914825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c00fc20 con 0x7f05540041d0 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.022+0000 7f0552ffd700 1 -- 172.21.15.31:0/1797914825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c014440 con 0x7f05540041d0 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.022+0000 7f05713ff700 1 -- 172.21.15.31:0/1797914825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c0c24a0 con 0x7f05540041d0 2023-10-06T05:21:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.022+0000 7f05713ff700 1 -- 172.21.15.31:0/1797914825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c0c4e60 con 0x7f05540041d0 2023-10-06T05:21:04.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.023+0000 7f05713ff700 1 -- 172.21.15.31:0/1797914825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c0c2660 con 0x7f05540041d0 2023-10-06T05:21:04.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.024+0000 7f0552ffd700 1 -- 172.21.15.31:0/1797914825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f055c014a80 con 0x7f05540041d0 2023-10-06T05:21:04.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.025+0000 7f0552ffd700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0558067290 0x7f0558069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.025+0000 7f0552ffd700 1 -- 172.21.15.31:0/1797914825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f055c018030 con 0x7f05540041d0 2023-10-06T05:21:04.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.025+0000 7f056a7fc700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0558067290 0x7f0558069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.026+0000 7f056a7fc700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0558067290 0x7f0558069740 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0560008fd0 tx=0x7f0560015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:04.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.029+0000 7f0552ffd700 1 -- 172.21.15.31:0/1797914825 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f055c052590 con 0x7f05540041d0 2023-10-06T05:21:04.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.238+0000 7f05713ff700 1 -- 172.21.15.31:0/1797914825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f056c0c2810 con 0x7f05540041d0 2023-10-06T05:21:04.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.242+0000 7f0552ffd700 1 -- 172.21.15.31:0/1797914825 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f055c055bb0 con 0x7f05540041d0 2023-10-06T05:21:04.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.244+0000 7f05713ff700 1 -- 172.21.15.31:0/1797914825 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0558067290 msgr2=0x7f0558069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.244+0000 7f05713ff700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0558067290 0x7f0558069740 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0560008fd0 tx=0x7f0560015000 comp rx=0 tx=0).stop 2023-10-06T05:21:04.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.244+0000 7f05713ff700 1 -- 172.21.15.31:0/1797914825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 msgr2=0x7f056c0c29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.244+0000 7f05713ff700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 0x7f056c0c29c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f055c011d40 tx=0x7f055c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.244+0000 7f05713ff700 1 -- 172.21.15.31:0/1797914825 shutdown_connections 2023-10-06T05:21:04.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.244+0000 7f05713ff700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f056c15c810 0x7f056c15bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.244+0000 7f05713ff700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0558067290 0x7f0558069740 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.244+0000 7f05713ff700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05540041d0 0x7f056c0c29c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.244+0000 7f05713ff700 1 --2- 172.21.15.31:0/1797914825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f056c15c3a0 0x7f056c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.244+0000 7f05713ff700 1 -- 172.21.15.31:0/1797914825 >> 172.21.15.31:0/1797914825 conn(0x7f056c0cd3d0 msgr2=0x7f056c06e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:04.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.245+0000 7f05713ff700 1 -- 172.21.15.31:0/1797914825 shutdown_connections 2023-10-06T05:21:04.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.245+0000 7f05713ff700 1 -- 172.21.15.31:0/1797914825 wait complete. 2023-10-06T05:21:04.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-10-06T05:21:04.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.457+0000 7f20f69ff700 1 Processor -- start 2023-10-06T05:21:04.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.458+0000 7f20f69ff700 1 -- start start 2023-10-06T05:21:04.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.459+0000 7f20f69ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f20f0159500 0x7f20f00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.459+0000 7f20f69ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f00c3f40 con 0x7f20f015d2d0 2023-10-06T05:21:04.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.459+0000 7f20f69ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f00cb000 con 0x7f20f00cd510 2023-10-06T05:21:04.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.459+0000 7f20f69ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00b2860 con 0x7f20f0159500 2023-10-06T05:21:04.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.459+0000 7f20ef7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f20f015d2d0 0x7f20f00cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39628/0 (socket says 172.21.15.31:39628) 2023-10-06T05:21:04.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.459+0000 7f20ef7fe700 1 -- 172.21.15.31:0/2685554700 learned_addr learned my addr 172.21.15.31:0/2685554700 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.459+0000 7f20effff700 1 --2- 172.21.15.31:0/2685554700 >> v2:172.21.15.62:3300/0 conn(0x7f20f0159500 0x7f20f00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.460+0000 7f20eeffd700 1 -- 172.21.15.31:0/2685554700 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029098836 0 0) 0x7f20f00c3f40 con 0x7f20f015d2d0 2023-10-06T05:21:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.460+0000 7f20eeffd700 1 -- 172.21.15.31:0/2685554700 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20d8003570 con 0x7f20f015d2d0 2023-10-06T05:21:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.460+0000 7f20eeffd700 1 -- 172.21.15.31:0/2685554700 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726841023 0 0) 0x7f20d8003570 con 0x7f20f015d2d0 2023-10-06T05:21:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.460+0000 7f20effff700 1 -- 172.21.15.31:0/2685554700 >> v1:172.21.15.62:6789/0 conn(0x7f20f00cd510 legacy=0x7f20f00aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.460+0000 7f20effff700 1 -- 172.21.15.31:0/2685554700 >> v1:172.21.15.31:6789/0 conn(0x7f20f015d2d0 legacy=0x7f20f00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.460+0000 7f20effff700 1 -- 172.21.15.31:0/2685554700 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f015d7d0 con 0x7f20f0159500 2023-10-06T05:21:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.460+0000 7f20effff700 1 --2- 172.21.15.31:0/2685554700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0159500 0x7f20f00b0ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f20e00098e0 tx=0x7f20e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5807203917c7d0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.461+0000 7f20eeffd700 1 -- 172.21.15.31:0/2685554700 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e40015f0 con 0x7f20f0159500 2023-10-06T05:21:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.461+0000 7f20eeffd700 1 -- 172.21.15.31:0/2685554700 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e0011420 con 0x7f20f0159500 2023-10-06T05:21:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.461+0000 7f20eeffd700 1 -- 172.21.15.31:0/2685554700 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e00117a0 con 0x7f20f0159500 2023-10-06T05:21:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.461+0000 7f20f69ff700 1 -- 172.21.15.31:0/2685554700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0159500 msgr2=0x7f20f00b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.461+0000 7f20f69ff700 1 --2- 172.21.15.31:0/2685554700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0159500 0x7f20f00b0ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f20e00098e0 tx=0x7f20e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.461+0000 7f20f69ff700 1 -- 172.21.15.31:0/2685554700 shutdown_connections 2023-10-06T05:21:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.461+0000 7f20f69ff700 1 --2- 172.21.15.31:0/2685554700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0159500 0x7f20f00b0ab0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.461+0000 7f20f69ff700 1 -- 172.21.15.31:0/2685554700 >> 172.21.15.31:0/2685554700 conn(0x7f20f006ec90 msgr2=0x7f20f006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:04.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.461+0000 7f20f69ff700 1 -- 172.21.15.31:0/2685554700 shutdown_connections 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.461+0000 7f20f69ff700 1 -- 172.21.15.31:0/2685554700 wait complete. 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.463+0000 7f20f69ff700 1 Processor -- start 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.463+0000 7f20f69ff700 1 -- start start 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20f69ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20f00cd510 0x7f20f015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20f69ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f0159500 0x7f20f0158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20f69ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015d2d0 0x7f20f00b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20f69ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f016a900 con 0x7f20f0159500 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20f69ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f016abe0 con 0x7f20f015d2d0 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20f69ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f016aec0 con 0x7f20f00cd510 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f0159500 0x7f20f0158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f0159500 0x7f20f0158820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60590/0 (socket says 172.21.15.31:60590) 2023-10-06T05:21:04.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20ef7fe700 1 -- 172.21.15.31:0/1787251574 learned_addr learned my addr 172.21.15.31:0/1787251574 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20effff700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20f00cd510 0x7f20f015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.464+0000 7f20f4f9c700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015d2d0 0x7f20f00b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20ef7fe700 1 -- 172.21.15.31:0/1787251574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20f00cd510 msgr2=0x7f20f015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20ef7fe700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20f00cd510 0x7f20f015a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20ef7fe700 1 -- 172.21.15.31:0/1787251574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015d2d0 msgr2=0x7f20f00b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20ef7fe700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015d2d0 0x7f20f00b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20ef7fe700 1 -- 172.21.15.31:0/1787251574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0009420 con 0x7f20f0159500 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20f4f9c700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015d2d0 0x7f20f00b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20ef7fe700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f0159500 0x7f20f0158820 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f20e400c810 tx=0x7f20e40016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20d7fff700 1 -- 172.21.15.31:0/1787251574 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4007300 con 0x7f20f0159500 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20f69ff700 1 -- 172.21.15.31:0/1787251574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f016b1a0 con 0x7f20f0159500 2023-10-06T05:21:04.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20d7fff700 1 -- 172.21.15.31:0/1787251574 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e4007480 con 0x7f20f0159500 2023-10-06T05:21:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20d7fff700 1 -- 172.21.15.31:0/1787251574 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e40057f0 con 0x7f20f0159500 2023-10-06T05:21:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.465+0000 7f20f69ff700 1 -- 172.21.15.31:0/1787251574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f0158d60 con 0x7f20f0159500 2023-10-06T05:21:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.466+0000 7f20f69ff700 1 -- 172.21.15.31:0/1787251574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f00c6b60 con 0x7f20f0159500 2023-10-06T05:21:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.467+0000 7f20d7fff700 1 -- 172.21.15.31:0/1787251574 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f20e4005ba0 con 0x7f20f0159500 2023-10-06T05:21:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.468+0000 7f20d7fff700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f20dc067290 0x7f20dc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.468+0000 7f20d7fff700 1 -- 172.21.15.31:0/1787251574 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f20e40846f0 con 0x7f20f0159500 2023-10-06T05:21:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.468+0000 7f20effff700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f20dc067290 0x7f20dc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.469+0000 7f20effff700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f20dc067290 0x7f20dc069740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009fd0 tx=0x7f20e0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.472+0000 7f20d7fff700 1 -- 172.21.15.31:0/1787251574 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20e40515b0 con 0x7f20f0159500 2023-10-06T05:21:04.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.756+0000 7f20f69ff700 1 -- 172.21.15.31:0/1787251574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f20f0158fb0 con 0x7f20f0159500 2023-10-06T05:21:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.794+0000 7f20d7fff700 1 -- 172.21.15.31:0/1787251574 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v221) v1 ==== 161+0+0 (secure 0 0 0) 0x7f20e4054bd0 con 0x7f20f0159500 2023-10-06T05:21:04.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.796+0000 7f20f69ff700 1 -- 172.21.15.31:0/1787251574 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f20dc067290 msgr2=0x7f20dc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.796+0000 7f20f69ff700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f20dc067290 0x7f20dc069740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009fd0 tx=0x7f20e0016000 comp rx=0 tx=0).stop 2023-10-06T05:21:04.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.796+0000 7f20f69ff700 1 -- 172.21.15.31:0/1787251574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f0159500 msgr2=0x7f20f0158820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:04.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.796+0000 7f20f69ff700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f0159500 0x7f20f0158820 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f20e400c810 tx=0x7f20e40016c0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.796+0000 7f20f69ff700 1 -- 172.21.15.31:0/1787251574 shutdown_connections 2023-10-06T05:21:04.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.796+0000 7f20f69ff700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015d2d0 0x7f20f00b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.796+0000 7f20f69ff700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20f00cd510 0x7f20f015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.796+0000 7f20f69ff700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f20dc067290 0x7f20dc069740 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.796+0000 7f20f69ff700 1 --2- 172.21.15.31:0/1787251574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f0159500 0x7f20f0158820 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:04.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.796+0000 7f20f69ff700 1 -- 172.21.15.31:0/1787251574 >> 172.21.15.31:0/1787251574 conn(0x7f20f006ec90 msgr2=0x7f20f00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.797+0000 7f20f69ff700 1 -- 172.21.15.31:0/1787251574 shutdown_connections 2023-10-06T05:21:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:04.797+0000 7f20f69ff700 1 -- 172.21.15.31:0/1787251574 wait complete. 2023-10-06T05:21:04.816 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2023-10-06T05:21:04.816 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:21:04.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2023-10-06T05:21:05.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.011+0000 7f1038bff700 1 Processor -- start 2023-10-06T05:21:05.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.011+0000 7f1038bff700 1 -- start start 2023-10-06T05:21:05.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f1038bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1034068d80 0x7f1034159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:05.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f1038bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10340cc0d0 0x7f10340c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:05.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f1038bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10340c95b0 con 0x7f10341595a0 2023-10-06T05:21:05.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f1038bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10340c9890 con 0x7f1034068d80 2023-10-06T05:21:05.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f1038bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10340c9b70 con 0x7f10340cc0d0 2023-10-06T05:21:05.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f103259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f10341595a0 0x7f1034068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39634/0 (socket says 172.21.15.31:39634) 2023-10-06T05:21:05.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f103259c700 1 -- 172.21.15.31:0/1599729006 learned_addr learned my addr 172.21.15.31:0/1599729006 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:05.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f1032d9d700 1 --2- 172.21.15.31:0/1599729006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10340cc0d0 0x7f10340c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:05.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f1031d9b700 1 --2- 172.21.15.31:0/1599729006 >> v2:172.21.15.62:3300/0 conn(0x7f1034068d80 0x7f1034159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:05.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f103159a700 1 -- 172.21.15.31:0/1599729006 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274949749 0 0) 0x7f10340c95b0 con 0x7f10341595a0 2023-10-06T05:21:05.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.013+0000 7f103159a700 1 -- 172.21.15.31:0/1599729006 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c003570 con 0x7f10341595a0 2023-10-06T05:21:05.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.014+0000 7f1032d9d700 1 -- 172.21.15.31:0/1599729006 >> v2:172.21.15.62:3300/0 conn(0x7f1034068d80 msgr2=0x7f1034159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:05.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.014+0000 7f1032d9d700 1 --2- 172.21.15.31:0/1599729006 >> v2:172.21.15.62:3300/0 conn(0x7f1034068d80 0x7f1034159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:05.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.014+0000 7f1032d9d700 1 -- 172.21.15.31:0/1599729006 >> v1:172.21.15.31:6789/0 conn(0x7f10341595a0 legacy=0x7f1034068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:05.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.014+0000 7f1032d9d700 1 -- 172.21.15.31:0/1599729006 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10340ca130 con 0x7f10340cc0d0 2023-10-06T05:21:05.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.014+0000 7f1032d9d700 1 --2- 172.21.15.31:0/1599729006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10340cc0d0 0x7f10340c8930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f102c009e90 tx=0x7f102c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468cacbb610eee04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:05.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.014+0000 7f103159a700 1 -- 172.21.15.31:0/1599729006 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c00e070 con 0x7f10340cc0d0 2023-10-06T05:21:05.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.014+0000 7f103159a700 1 -- 172.21.15.31:0/1599729006 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c00bb40 con 0x7f10340cc0d0 2023-10-06T05:21:05.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.014+0000 7f103159a700 1 -- 172.21.15.31:0/1599729006 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c012430 con 0x7f10340cc0d0 2023-10-06T05:21:05.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.015+0000 7f1038bff700 1 -- 172.21.15.31:0/1599729006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10340cc0d0 msgr2=0x7f10340c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:05.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.015+0000 7f1038bff700 1 --2- 172.21.15.31:0/1599729006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10340cc0d0 0x7f10340c8930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f102c009e90 tx=0x7f102c009780 comp rx=0 tx=0).stop 2023-10-06T05:21:05.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.015+0000 7f1038bff700 1 -- 172.21.15.31:0/1599729006 shutdown_connections 2023-10-06T05:21:05.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.015+0000 7f1038bff700 1 --2- 172.21.15.31:0/1599729006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10340cc0d0 0x7f10340c8930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:05.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.015+0000 7f1038bff700 1 --2- 172.21.15.31:0/1599729006 >> v2:172.21.15.62:3300/0 conn(0x7f1034068d80 0x7f1034159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:05.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.015+0000 7f1038bff700 1 -- 172.21.15.31:0/1599729006 >> 172.21.15.31:0/1599729006 conn(0x7f103406b620 msgr2=0x7f103406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:05.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.015+0000 7f1038bff700 1 -- 172.21.15.31:0/1599729006 shutdown_connections 2023-10-06T05:21:05.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.015+0000 7f1038bff700 1 -- 172.21.15.31:0/1599729006 wait complete. 2023-10-06T05:21:05.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.016+0000 7f1038bff700 1 Processor -- start 2023-10-06T05:21:05.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.017+0000 7f1038bff700 1 -- start start 2023-10-06T05:21:05.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.017+0000 7f1038bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1034068d80 0x7f10340b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:05.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.017+0000 7f1038bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10340cc0d0 0x7f1034153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:05.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1038bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10341595a0 0x7f10340c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:05.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1038bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1034153aa0 con 0x7f10341595a0 2023-10-06T05:21:05.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1038bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10340c58a0 con 0x7f10340cc0d0 2023-10-06T05:21:05.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1038bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10340c5b80 con 0x7f1034068d80 2023-10-06T05:21:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1032d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10341595a0 0x7f10340c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1031d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10340cc0d0 0x7f1034153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f103259c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1034068d80 0x7f10340b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1032d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10341595a0 0x7f10340c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60594/0 (socket says 172.21.15.31:60594) 2023-10-06T05:21:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1032d9d700 1 -- 172.21.15.31:0/912456637 learned_addr learned my addr 172.21.15.31:0/912456637 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1032d9d700 1 -- 172.21.15.31:0/912456637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1034068d80 msgr2=0x7f10340b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1032d9d700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1034068d80 0x7f10340b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1032d9d700 1 -- 172.21.15.31:0/912456637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10340cc0d0 msgr2=0x7f1034153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:05.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1032d9d700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10340cc0d0 0x7f1034153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.018+0000 7f1032d9d700 1 -- 172.21.15.31:0/912456637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c009a20 con 0x7f10341595a0 2023-10-06T05:21:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.019+0000 7f1032d9d700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10341595a0 0x7f10340c6940 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f102c013000 tx=0x7f102c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.019+0000 7f10227fc700 1 -- 172.21.15.31:0/912456637 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c012ce0 con 0x7f10341595a0 2023-10-06T05:21:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.019+0000 7f10227fc700 1 -- 172.21.15.31:0/912456637 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c021070 con 0x7f10341595a0 2023-10-06T05:21:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.019+0000 7f10227fc700 1 -- 172.21.15.31:0/912456637 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c026680 con 0x7f10341595a0 2023-10-06T05:21:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.019+0000 7f1038bff700 1 -- 172.21.15.31:0/912456637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10340c5e60 con 0x7f10341595a0 2023-10-06T05:21:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.019+0000 7f1038bff700 1 -- 172.21.15.31:0/912456637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10340c6420 con 0x7f10341595a0 2023-10-06T05:21:05.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.020+0000 7f1038bff700 1 -- 172.21.15.31:0/912456637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103404c380 con 0x7f10341595a0 2023-10-06T05:21:05.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.021+0000 7f10227fc700 1 -- 172.21.15.31:0/912456637 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f102c023030 con 0x7f10341595a0 2023-10-06T05:21:05.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.022+0000 7f10227fc700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1014067290 0x7f1014069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:05.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.022+0000 7f10227fc700 1 -- 172.21.15.31:0/912456637 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f102c086980 con 0x7f10341595a0 2023-10-06T05:21:05.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.022+0000 7f103259c700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1014067290 0x7f1014069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:05.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.023+0000 7f103259c700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1014067290 0x7f1014069740 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1024007fd0 tx=0x7f1024009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.026+0000 7f10227fc700 1 -- 172.21.15.31:0/912456637 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f102c053940 con 0x7f10341595a0 2023-10-06T05:21:05.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.312+0000 7f1038bff700 1 -- 172.21.15.31:0/912456637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7f1034156870 con 0x7f10341595a0 2023-10-06T05:21:05.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.797+0000 7f10227fc700 1 -- 172.21.15.31:0/912456637 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v222) v1 ==== 134+0+0 (secure 0 0 0) 0x7f102c012e60 con 0x7f10341595a0 2023-10-06T05:21:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 -- 172.21.15.31:0/912456637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1014067290 msgr2=0x7f1014069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1014067290 0x7f1014069740 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1024007fd0 tx=0x7f1024009040 comp rx=0 tx=0).stop 2023-10-06T05:21:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 -- 172.21.15.31:0/912456637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10341595a0 msgr2=0x7f10340c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:05.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10341595a0 0x7f10340c6940 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f102c013000 tx=0x7f102c019000 comp rx=0 tx=0).stop 2023-10-06T05:21:05.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 -- 172.21.15.31:0/912456637 shutdown_connections 2023-10-06T05:21:05.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1014067290 0x7f1014069740 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:05.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10341595a0 0x7f10340c6940 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:05.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1034068d80 0x7f10340b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:05.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 --2- 172.21.15.31:0/912456637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10340cc0d0 0x7f1034153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:05.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 -- 172.21.15.31:0/912456637 >> 172.21.15.31:0/912456637 conn(0x7f103406b620 msgr2=0x7f1034156d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:05.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.799+0000 7f1038bff700 1 -- 172.21.15.31:0/912456637 shutdown_connections 2023-10-06T05:21:05.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:05.800+0000 7f1038bff700 1 -- 172.21.15.31:0/912456637 wait complete. 2023-10-06T05:21:05.821 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:21:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.022+0000 7f25eb9ff700 1 Processor -- start 2023-10-06T05:21:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.022+0000 7f25eb9ff700 1 -- start start 2023-10-06T05:21:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.023+0000 7f25eb9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25e40b1260 0x7f25e40b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.023+0000 7f25eb9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e4158cf0 con 0x7f25e41573a0 2023-10-06T05:21:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.023+0000 7f25eb9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e4157e80 con 0x7f25e40ae860 2023-10-06T05:21:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.023+0000 7f25eb9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e40b1c40 con 0x7f25e40b1260 2023-10-06T05:21:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.023+0000 7f25e979b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f25e41573a0 0x7f25e40acb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39648/0 (socket says 172.21.15.31:39648) 2023-10-06T05:21:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.023+0000 7f25e979b700 1 -- 172.21.15.31:0/2703105984 learned_addr learned my addr 172.21.15.31:0/2703105984 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.024+0000 7f25e9f9c700 1 --2- 172.21.15.31:0/2703105984 >> v2:172.21.15.31:3300/0 conn(0x7f25e40b1260 0x7f25e40b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.024+0000 7f25dbfff700 1 -- 172.21.15.31:0/2703105984 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247208905 0 0) 0x7f25e4158cf0 con 0x7f25e41573a0 2023-10-06T05:21:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.024+0000 7f25dbfff700 1 -- 172.21.15.31:0/2703105984 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25d0003570 con 0x7f25e41573a0 2023-10-06T05:21:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.024+0000 7f25e9f9c700 1 -- 172.21.15.31:0/2703105984 >> v1:172.21.15.62:6789/0 conn(0x7f25e40ae860 legacy=0x7f25e4157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.024+0000 7f25e9f9c700 1 -- 172.21.15.31:0/2703105984 >> v1:172.21.15.31:6789/0 conn(0x7f25e41573a0 legacy=0x7f25e40acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.024+0000 7f25e9f9c700 1 -- 172.21.15.31:0/2703105984 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e41580e0 con 0x7f25e40b1260 2023-10-06T05:21:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.024+0000 7f25e9f9c700 1 --2- 172.21.15.31:0/2703105984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 0x7f25e40b1700 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009e60 tx=0x7f25dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edaaebb42a0903f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.025+0000 7f25dbfff700 1 -- 172.21.15.31:0/2703105984 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc00e070 con 0x7f25e40b1260 2023-10-06T05:21:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.025+0000 7f25dbfff700 1 -- 172.21.15.31:0/2703105984 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc00bb60 con 0x7f25e40b1260 2023-10-06T05:21:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.025+0000 7f25dbfff700 1 -- 172.21.15.31:0/2703105984 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc012430 con 0x7f25e40b1260 2023-10-06T05:21:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.025+0000 7f25eb9ff700 1 -- 172.21.15.31:0/2703105984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 msgr2=0x7f25e40b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.025+0000 7f25eb9ff700 1 --2- 172.21.15.31:0/2703105984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 0x7f25e40b1700 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009e60 tx=0x7f25dc009780 comp rx=0 tx=0).stop 2023-10-06T05:21:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.025+0000 7f25eb9ff700 1 -- 172.21.15.31:0/2703105984 shutdown_connections 2023-10-06T05:21:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.025+0000 7f25eb9ff700 1 --2- 172.21.15.31:0/2703105984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 0x7f25e40b1700 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.025+0000 7f25eb9ff700 1 -- 172.21.15.31:0/2703105984 >> 172.21.15.31:0/2703105984 conn(0x7f25e4067560 msgr2=0x7f25e4067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.025+0000 7f25eb9ff700 1 -- 172.21.15.31:0/2703105984 shutdown_connections 2023-10-06T05:21:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.025+0000 7f25eb9ff700 1 -- 172.21.15.31:0/2703105984 wait complete. 2023-10-06T05:21:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.027+0000 7f25eb9ff700 1 Processor -- start 2023-10-06T05:21:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.027+0000 7f25eb9ff700 1 -- start start 2023-10-06T05:21:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25eb9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 0x7f25e416ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25eb9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25e41573a0 0x7f25e406dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25eb9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25e40cb3d0 0x7f25e4169710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25eb9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e415ea20 con 0x7f25e40b1260 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25eb9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e416d340 con 0x7f25e40cb3d0 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25eb9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e416d620 con 0x7f25e41573a0 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25e979b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 0x7f25e416ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25e979b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 0x7f25e416ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60612/0 (socket says 172.21.15.31:60612) 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25e979b700 1 -- 172.21.15.31:0/1408714462 learned_addr learned my addr 172.21.15.31:0/1408714462 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25e9f9c700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25e40cb3d0 0x7f25e4169710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.028+0000 7f25e8f9a700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25e41573a0 0x7f25e406dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25e979b700 1 -- 172.21.15.31:0/1408714462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25e41573a0 msgr2=0x7f25e406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25e979b700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25e41573a0 0x7f25e406dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25e979b700 1 -- 172.21.15.31:0/1408714462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25e40cb3d0 msgr2=0x7f25e4169710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25e979b700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25e40cb3d0 0x7f25e4169710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25e979b700 1 -- 172.21.15.31:0/1408714462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc009a20 con 0x7f25e40b1260 2023-10-06T05:21:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25e8f9a700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25e41573a0 0x7f25e406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25e979b700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 0x7f25e416ccf0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f25e000c590 tx=0x7f25e0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25d97fa700 1 -- 172.21.15.31:0/1408714462 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0005cd0 con 0x7f25e40b1260 2023-10-06T05:21:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25d97fa700 1 -- 172.21.15.31:0/1408714462 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e0005e50 con 0x7f25e40b1260 2023-10-06T05:21:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25d97fa700 1 -- 172.21.15.31:0/1408714462 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0007320 con 0x7f25e40b1260 2023-10-06T05:21:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25eb9ff700 1 -- 172.21.15.31:0/1408714462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e416d900 con 0x7f25e40b1260 2023-10-06T05:21:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.029+0000 7f25eb9ff700 1 -- 172.21.15.31:0/1408714462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e416dbe0 con 0x7f25e40b1260 2023-10-06T05:21:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.030+0000 7f25eb9ff700 1 -- 172.21.15.31:0/1408714462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e40ab940 con 0x7f25e40b1260 2023-10-06T05:21:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.031+0000 7f25d97fa700 1 -- 172.21.15.31:0/1408714462 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f25e001b3b0 con 0x7f25e40b1260 2023-10-06T05:21:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.032+0000 7f25d97fa700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25c8067290 0x7f25c8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.032+0000 7f25d97fa700 1 -- 172.21.15.31:0/1408714462 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f25e0083f70 con 0x7f25e40b1260 2023-10-06T05:21:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.032+0000 7f25e8f9a700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25c8067290 0x7f25c8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.033+0000 7f25e8f9a700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25c8067290 0x7f25c8069740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f25d4007fd0 tx=0x7f25d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.036+0000 7f25d97fa700 1 -- 172.21.15.31:0/1408714462 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25e0050f30 con 0x7f25e40b1260 2023-10-06T05:21:06.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.242+0000 7f25eb9ff700 1 -- 172.21.15.31:0/1408714462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f25e404c380 con 0x7f25e40b1260 2023-10-06T05:21:06.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.246+0000 7f25d97fa700 1 -- 172.21.15.31:0/1408714462 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f25e00170c0 con 0x7f25e40b1260 2023-10-06T05:21:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 -- 172.21.15.31:0/1408714462 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25c8067290 msgr2=0x7f25c8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25c8067290 0x7f25c8069740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f25d4007fd0 tx=0x7f25d4009040 comp rx=0 tx=0).stop 2023-10-06T05:21:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 -- 172.21.15.31:0/1408714462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 msgr2=0x7f25e416ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 0x7f25e416ccf0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f25e000c590 tx=0x7f25e0007e00 comp rx=0 tx=0).stop 2023-10-06T05:21:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 -- 172.21.15.31:0/1408714462 shutdown_connections 2023-10-06T05:21:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25e40cb3d0 0x7f25e4169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25c8067290 0x7f25c8069740 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25e40b1260 0x7f25e416ccf0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 --2- 172.21.15.31:0/1408714462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25e41573a0 0x7f25e406dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 -- 172.21.15.31:0/1408714462 >> 172.21.15.31:0/1408714462 conn(0x7f25e4067560 msgr2=0x7f25e4159320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:06.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 -- 172.21.15.31:0/1408714462 shutdown_connections 2023-10-06T05:21:06.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.249+0000 7f25eb9ff700 1 -- 172.21.15.31:0/1408714462 wait complete. 2023-10-06T05:21:06.268 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:21:06.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:21:06.370 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:21:06.473 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:21:06.473 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:21:06.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.667+0000 7f22c87ff700 1 Processor -- start 2023-10-06T05:21:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.667+0000 7f22c87ff700 1 -- start start 2023-10-06T05:21:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c87ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22c00b1610 0x7f22c00b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c87ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22c00b19e0 0x7f22c015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c87ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f22c00cd690 0x7f22c0159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c87ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c015e200 con 0x7f22c00b1610 2023-10-06T05:21:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c87ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c0068f50 con 0x7f22c00cd690 2023-10-06T05:21:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c87ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00b3ab0 con 0x7f22c00b19e0 2023-10-06T05:21:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c659b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22c00b1610 0x7f22c00b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c659b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22c00b1610 0x7f22c00b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60624/0 (socket says 172.21.15.31:60624) 2023-10-06T05:21:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c659b700 1 -- 172.21.15.31:0/3138783560 learned_addr learned my addr 172.21.15.31:0/3138783560 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c5d9a700 1 --2- 172.21.15.31:0/3138783560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22c00b19e0 0x7f22c015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c6d9c700 1 --2- 172.21.15.31:0/3138783560 >> v2:172.21.15.62:3300/0 conn(0x7f22c00cd690 0x7f22c0159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c659b700 1 -- 172.21.15.31:0/3138783560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22c00b19e0 msgr2=0x7f22c015dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c659b700 1 --2- 172.21.15.31:0/3138783560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22c00b19e0 0x7f22c015dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.669+0000 7f22c659b700 1 -- 172.21.15.31:0/3138783560 >> v2:172.21.15.62:3300/0 conn(0x7f22c00cd690 msgr2=0x7f22c0159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.670+0000 7f22c659b700 1 --2- 172.21.15.31:0/3138783560 >> v2:172.21.15.62:3300/0 conn(0x7f22c00cd690 0x7f22c0159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.670+0000 7f22c659b700 1 -- 172.21.15.31:0/3138783560 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c00b3d90 con 0x7f22c00b1610 2023-10-06T05:21:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.670+0000 7f22c659b700 1 --2- 172.21.15.31:0/3138783560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b1610 0x7f22c00b33d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f22c00cf1c0 tx=0x7f22bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c242220cfe8dd682 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.670+0000 7f22c5599700 1 -- 172.21.15.31:0/3138783560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00b860 con 0x7f22c00b1610 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.670+0000 7f22c5599700 1 -- 172.21.15.31:0/3138783560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc00b9c0 con 0x7f22c00b1610 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.670+0000 7f22c5599700 1 -- 172.21.15.31:0/3138783560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00bd60 con 0x7f22c00b1610 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.671+0000 7f22c87ff700 1 -- 172.21.15.31:0/3138783560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b1610 msgr2=0x7f22c00b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.671+0000 7f22c87ff700 1 --2- 172.21.15.31:0/3138783560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b1610 0x7f22c00b33d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f22c00cf1c0 tx=0x7f22bc009250 comp rx=0 tx=0).stop 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.671+0000 7f22c87ff700 1 -- 172.21.15.31:0/3138783560 shutdown_connections 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.671+0000 7f22c87ff700 1 --2- 172.21.15.31:0/3138783560 >> v2:172.21.15.62:3300/0 conn(0x7f22c00cd690 0x7f22c0159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.671+0000 7f22c87ff700 1 --2- 172.21.15.31:0/3138783560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b1610 0x7f22c00b33d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.671+0000 7f22c87ff700 1 --2- 172.21.15.31:0/3138783560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22c00b19e0 0x7f22c015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.671+0000 7f22c87ff700 1 -- 172.21.15.31:0/3138783560 >> 172.21.15.31:0/3138783560 conn(0x7f22c006b5b0 msgr2=0x7f22c006b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.671+0000 7f22c87ff700 1 -- 172.21.15.31:0/3138783560 shutdown_connections 2023-10-06T05:21:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.671+0000 7f22c87ff700 1 -- 172.21.15.31:0/3138783560 wait complete. 2023-10-06T05:21:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.672+0000 7f22c87ff700 1 Processor -- start 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.673+0000 7f22c87ff700 1 -- start start 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.673+0000 7f22c87ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22c00b1610 0x7f22c00c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.673+0000 7f22c87ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b19e0 0x7f22c015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.673+0000 7f22c87ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22c00cd690 0x7f22c015b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c87ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c00c8140 con 0x7f22c00b19e0 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c87ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c015c520 con 0x7f22c00b1610 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c87ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c015cef0 con 0x7f22c00cd690 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c5d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b19e0 0x7f22c015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c5d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b19e0 0x7f22c015c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60638/0 (socket says 172.21.15.31:60638) 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c5d9a700 1 -- 172.21.15.31:0/1857650997 learned_addr learned my addr 172.21.15.31:0/1857650997 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c6d9c700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22c00cd690 0x7f22c015b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c659b700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22c00b1610 0x7f22c00c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c5d9a700 1 -- 172.21.15.31:0/1857650997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22c00cd690 msgr2=0x7f22c015b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c5d9a700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22c00cd690 0x7f22c015b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c5d9a700 1 -- 172.21.15.31:0/1857650997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22c00b1610 msgr2=0x7f22c00c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c5d9a700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22c00b1610 0x7f22c00c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.674+0000 7f22c5d9a700 1 -- 172.21.15.31:0/1857650997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0094f0 con 0x7f22c00b19e0 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.675+0000 7f22c5d9a700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b19e0 0x7f22c015c9b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f22b000d890 tx=0x7f22b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.675+0000 7f22b67fc700 1 -- 172.21.15.31:0/1857650997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b00125f0 con 0x7f22c00b19e0 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.675+0000 7f22b67fc700 1 -- 172.21.15.31:0/1857650997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b0012750 con 0x7f22c00b19e0 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.675+0000 7f22b67fc700 1 -- 172.21.15.31:0/1857650997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0012af0 con 0x7f22c00b19e0 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.675+0000 7f22c87ff700 1 -- 172.21.15.31:0/1857650997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c015b660 con 0x7f22c00b19e0 2023-10-06T05:21:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.675+0000 7f22c87ff700 1 -- 172.21.15.31:0/1857650997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c015bbf0 con 0x7f22c00b19e0 2023-10-06T05:21:06.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.676+0000 7f22c87ff700 1 -- 172.21.15.31:0/1857650997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c015c690 con 0x7f22c00b19e0 2023-10-06T05:21:06.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.677+0000 7f22b67fc700 1 -- 172.21.15.31:0/1857650997 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f22b001e460 con 0x7f22c00b19e0 2023-10-06T05:21:06.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.677+0000 7f22b67fc700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f22a4067290 0x7f22a4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.678+0000 7f22b67fc700 1 -- 172.21.15.31:0/1857650997 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f22b0084e00 con 0x7f22c00b19e0 2023-10-06T05:21:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.678+0000 7f22c659b700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f22a4067290 0x7f22a4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.678+0000 7f22c659b700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f22a4067290 0x7f22a4069740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f22c00cf1c0 tx=0x7f22bc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:06.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.682+0000 7f22b67fc700 1 -- 172.21.15.31:0/1857650997 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22b0051e00 con 0x7f22c00b19e0 2023-10-06T05:21:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.929+0000 7f22c87ff700 1 -- 172.21.15.31:0/1857650997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22c0068be0 con 0x7f22c00b19e0 2023-10-06T05:21:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.930+0000 7f22b67fc700 1 -- 172.21.15.31:0/1857650997 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f22b0055420 con 0x7f22c00b19e0 2023-10-06T05:21:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:06.932 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5582,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/583850064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":583850064},{"type":"v1","addr":"172.21.15.62:6835","nonce":583850064}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5743,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/2430199885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":2430199885},{"type":"v1","addr":"172.21.15.31:6837","nonce":2430199885}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5746,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/3762519665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3762519665},{"type":"v1","addr":"172.21.15.62:6834","nonce":3762519665}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5844,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2776536460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2776536460},{"type":"v1","addr":"172.21.15.31:6836","nonce":2776536460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[]} 2023-10-06T05:21:06.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.932+0000 7f22c87ff700 1 -- 172.21.15.31:0/1857650997 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f22a4067290 msgr2=0x7f22a4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.932+0000 7f22c87ff700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f22a4067290 0x7f22a4069740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f22c00cf1c0 tx=0x7f22bc017000 comp rx=0 tx=0).stop 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.932+0000 7f22c87ff700 1 -- 172.21.15.31:0/1857650997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b19e0 msgr2=0x7f22c015c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.932+0000 7f22c87ff700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b19e0 0x7f22c015c9b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f22b000d890 tx=0x7f22b0008040 comp rx=0 tx=0).stop 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.932+0000 7f22c87ff700 1 -- 172.21.15.31:0/1857650997 shutdown_connections 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.932+0000 7f22c87ff700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f22c00cd690 0x7f22c015b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.932+0000 7f22c87ff700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22c00b1610 0x7f22c00c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.932+0000 7f22c87ff700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f22a4067290 0x7f22a4069740 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.932+0000 7f22c87ff700 1 --2- 172.21.15.31:0/1857650997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c00b19e0 0x7f22c015c9b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.932+0000 7f22c87ff700 1 -- 172.21.15.31:0/1857650997 >> 172.21.15.31:0/1857650997 conn(0x7f22c006b5b0 msgr2=0x7f22c0152950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.933+0000 7f22c87ff700 1 -- 172.21.15.31:0/1857650997 shutdown_connections 2023-10-06T05:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:06.933+0000 7f22c87ff700 1 -- 172.21.15.31:0/1857650997 wait complete. 2023-10-06T05:21:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2023-10-06T05:21:06.952 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:21:06.952 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:21:06.952 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:21:06.952 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:06.952 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:21:06.952 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:21:06.953 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:21:06.953 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:06.953 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:21:06.953 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:21:06.953 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:21:06.953 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:06.953 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:21:06.954 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:21:06.954 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:21:06.954 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:06.966 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:21:06.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:21:06.967 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:21:06.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:21:06.970 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:21:06.970 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:21:06.971 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:21:06.971 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:21:07.011 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:21:07.012 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:07.014 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:21:07.014 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:07.016 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:21:07.016 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:07.018 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:21:07.019 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:07.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:21:07.159 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:21:07.160 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:21:07.169 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:21:07.169 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:21:07.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.227+0000 7fe9589ff700 1 Processor -- start 2023-10-06T05:21:07.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.227+0000 7fe9589ff700 1 -- start start 2023-10-06T05:21:07.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.228+0000 7fe9589ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9540c7b60 0x7fe954069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.228+0000 7fe9589ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe9540aee70 0x7fe95413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.228+0000 7fe9589ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9540ca5b0 con 0x7fe9540c7f30 2023-10-06T05:21:07.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.228+0000 7fe9589ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9540af2d0 con 0x7fe9540c7b60 2023-10-06T05:21:07.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.228+0000 7fe9589ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95406f740 con 0x7fe9540aee70 2023-10-06T05:21:07.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.229+0000 7fe95259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9540c7b60 0x7fe954069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.229+0000 7fe951d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe9540c7f30 0x7fe95406ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41214/0 (socket says 172.21.15.31:41214) 2023-10-06T05:21:07.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.229+0000 7fe951d9b700 1 -- 172.21.15.31:0/2189941727 learned_addr learned my addr 172.21.15.31:0/2189941727 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:07.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.229+0000 7fe952d9d700 1 --2- 172.21.15.31:0/2189941727 >> v2:172.21.15.62:3300/0 conn(0x7fe9540aee70 0x7fe95413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.229+0000 7fe95259c700 1 -- 172.21.15.31:0/2189941727 >> v2:172.21.15.62:3300/0 conn(0x7fe9540aee70 msgr2=0x7fe95413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:07.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.229+0000 7fe95259c700 1 --2- 172.21.15.31:0/2189941727 >> v2:172.21.15.62:3300/0 conn(0x7fe9540aee70 0x7fe95413f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.229+0000 7fe95259c700 1 -- 172.21.15.31:0/2189941727 >> v1:172.21.15.31:6789/0 conn(0x7fe9540c7f30 legacy=0x7fe95406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.229+0000 7fe95259c700 1 -- 172.21.15.31:0/2189941727 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9540c9c80 con 0x7fe9540c7b60 2023-10-06T05:21:07.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.229+0000 7fe95159a700 1 -- 172.21.15.31:0/2189941727 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283696852 0 0) 0x7fe9540ca5b0 con 0x7fe9540c7f30 2023-10-06T05:21:07.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.229+0000 7fe95259c700 1 --2- 172.21.15.31:0/2189941727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540c7b60 0x7fe954069880 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe9541569d0 tx=0x7fe94400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a96d2881d5ef16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:07.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe95159a700 1 -- 172.21.15.31:0/2189941727 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944005c10 con 0x7fe9540c7b60 2023-10-06T05:21:07.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe95159a700 1 -- 172.21.15.31:0/2189941727 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe944005d70 con 0x7fe9540c7b60 2023-10-06T05:21:07.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe95159a700 1 -- 172.21.15.31:0/2189941727 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94400db60 con 0x7fe9540c7b60 2023-10-06T05:21:07.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe9589ff700 1 -- 172.21.15.31:0/2189941727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540c7b60 msgr2=0x7fe954069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe9589ff700 1 --2- 172.21.15.31:0/2189941727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540c7b60 0x7fe954069880 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe9541569d0 tx=0x7fe94400b3e0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe9589ff700 1 -- 172.21.15.31:0/2189941727 shutdown_connections 2023-10-06T05:21:07.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe9589ff700 1 --2- 172.21.15.31:0/2189941727 >> v2:172.21.15.62:3300/0 conn(0x7fe9540aee70 0x7fe95413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe9589ff700 1 --2- 172.21.15.31:0/2189941727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540c7b60 0x7fe954069880 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe9589ff700 1 -- 172.21.15.31:0/2189941727 >> 172.21.15.31:0/2189941727 conn(0x7fe9540cc0b0 msgr2=0x7fe9540cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:07.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe9589ff700 1 -- 172.21.15.31:0/2189941727 shutdown_connections 2023-10-06T05:21:07.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.230+0000 7fe9589ff700 1 -- 172.21.15.31:0/2189941727 wait complete. 2023-10-06T05:21:07.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.232+0000 7fe9589ff700 1 Processor -- start 2023-10-06T05:21:07.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.232+0000 7fe9589ff700 1 -- start start 2023-10-06T05:21:07.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe9589ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540aee70 0x7fe954156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe9589ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9540c7b60 0x7fe954155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe9589ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9540c7f30 0x7fe95416c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe9589ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe954156f40 con 0x7fe9540aee70 2023-10-06T05:21:07.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe9589ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9541594e0 con 0x7fe9540c7b60 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe9589ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9541588d0 con 0x7fe9540c7f30 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe95259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540aee70 0x7fe954156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe95259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540aee70 0x7fe954156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60242/0 (socket says 172.21.15.31:60242) 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe95259c700 1 -- 172.21.15.31:0/1898596013 learned_addr learned my addr 172.21.15.31:0/1898596013 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe951d9b700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9540c7b60 0x7fe954155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.233+0000 7fe952d9d700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9540c7f30 0x7fe95416c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe95259c700 1 -- 172.21.15.31:0/1898596013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9540c7f30 msgr2=0x7fe95416c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe95259c700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9540c7f30 0x7fe95416c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe95259c700 1 -- 172.21.15.31:0/1898596013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9540c7b60 msgr2=0x7fe954155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe95259c700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9540c7b60 0x7fe954155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:07.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe95259c700 1 -- 172.21.15.31:0/1898596013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94400b660 con 0x7fe9540aee70 2023-10-06T05:21:07.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe95259c700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540aee70 0x7fe954156a00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe95406cd80 tx=0x7fe94401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:07.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe9427fc700 1 -- 172.21.15.31:0/1898596013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94400d930 con 0x7fe9540aee70 2023-10-06T05:21:07.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe9427fc700 1 -- 172.21.15.31:0/1898596013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94400da90 con 0x7fe9540aee70 2023-10-06T05:21:07.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe9427fc700 1 -- 172.21.15.31:0/1898596013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944022680 con 0x7fe9540aee70 2023-10-06T05:21:07.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe9589ff700 1 -- 172.21.15.31:0/1898596013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe954158bb0 con 0x7fe9540aee70 2023-10-06T05:21:07.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.234+0000 7fe9589ff700 1 -- 172.21.15.31:0/1898596013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe954159170 con 0x7fe9540aee70 2023-10-06T05:21:07.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.235+0000 7fe9589ff700 1 -- 172.21.15.31:0/1898596013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe954159300 con 0x7fe9540aee70 2023-10-06T05:21:07.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.236+0000 7fe9427fc700 1 -- 172.21.15.31:0/1898596013 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe944019430 con 0x7fe9540aee70 2023-10-06T05:21:07.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.237+0000 7fe9427fc700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe934067290 0x7fe934069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:07.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.237+0000 7fe9427fc700 1 -- 172.21.15.31:0/1898596013 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fe944014030 con 0x7fe9540aee70 2023-10-06T05:21:07.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.237+0000 7fe951d9b700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe934067290 0x7fe934069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:07.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.238+0000 7fe951d9b700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe934067290 0x7fe934069740 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe95406b020 tx=0x7fe948007f50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.242+0000 7fe9427fc700 1 -- 172.21.15.31:0/1898596013 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe944053440 con 0x7fe9540aee70 2023-10-06T05:21:07.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:07.456+0000 7fe9589ff700 1 -- 172.21.15.31:0/1898596013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe95404c380 con 0x7fe9540aee70 2023-10-06T05:21:08.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.817+0000 7fe9427fc700 1 -- 172.21.15.31:0/1898596013 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v67) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe944027080 con 0x7fe9540aee70 2023-10-06T05:21:08.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.820+0000 7fe9589ff700 1 -- 172.21.15.31:0/1898596013 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe934067290 msgr2=0x7fe934069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.820+0000 7fe9589ff700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe934067290 0x7fe934069740 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe95406b020 tx=0x7fe948007f50 comp rx=0 tx=0).stop 2023-10-06T05:21:08.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.820+0000 7fe9589ff700 1 -- 172.21.15.31:0/1898596013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540aee70 msgr2=0x7fe954156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:08.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.820+0000 7fe9589ff700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540aee70 0x7fe954156a00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe95406cd80 tx=0x7fe94401a000 comp rx=0 tx=0).stop 2023-10-06T05:21:08.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.821+0000 7fe9589ff700 1 -- 172.21.15.31:0/1898596013 shutdown_connections 2023-10-06T05:21:08.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.821+0000 7fe9589ff700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9540c7f30 0x7fe95416c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.821+0000 7fe9589ff700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe934067290 0x7fe934069740 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.821+0000 7fe9589ff700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9540aee70 0x7fe954156a00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.821+0000 7fe9589ff700 1 --2- 172.21.15.31:0/1898596013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9540c7b60 0x7fe954155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:08.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.821+0000 7fe9589ff700 1 -- 172.21.15.31:0/1898596013 >> 172.21.15.31:0/1898596013 conn(0x7fe9540cc0b0 msgr2=0x7fe9540c21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:08.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.821+0000 7fe9589ff700 1 -- 172.21.15.31:0/1898596013 shutdown_connections 2023-10-06T05:21:08.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:08.821+0000 7fe9589ff700 1 -- 172.21.15.31:0/1898596013 wait complete. 2023-10-06T05:21:08.824 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:21:08.844 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:21:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.041+0000 7fe85e9ff700 1 Processor -- start 2023-10-06T05:21:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.041+0000 7fe85e9ff700 1 -- start start 2023-10-06T05:21:09.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.043+0000 7fe85e9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7b60 0x7fe858069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.043+0000 7fe85e9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8580ca5b0 con 0x7fe8580c7f30 2023-10-06T05:21:09.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.043+0000 7fe85e9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8580af2d0 con 0x7fe8580aee70 2023-10-06T05:21:09.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.043+0000 7fe85e9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85806f740 con 0x7fe8580c7b60 2023-10-06T05:21:09.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.043+0000 7fe8577fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe8580c7f30 0x7fe85806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41216/0 (socket says 172.21.15.31:41216) 2023-10-06T05:21:09.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.043+0000 7fe8577fe700 1 -- 172.21.15.31:0/1613279771 learned_addr learned my addr 172.21.15.31:0/1613279771 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:09.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.043+0000 7fe857fff700 1 --2- 172.21.15.31:0/1613279771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7b60 0x7fe858069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.044+0000 7fe856ffd700 1 -- 172.21.15.31:0/1613279771 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691121518 0 0) 0x7fe8580ca5b0 con 0x7fe8580c7f30 2023-10-06T05:21:09.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.044+0000 7fe856ffd700 1 -- 172.21.15.31:0/1613279771 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe840003570 con 0x7fe8580c7f30 2023-10-06T05:21:09.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.044+0000 7fe856ffd700 1 -- 172.21.15.31:0/1613279771 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653919988 0 0) 0x7fe8580af2d0 con 0x7fe8580aee70 2023-10-06T05:21:09.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.044+0000 7fe856ffd700 1 -- 172.21.15.31:0/1613279771 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8580ca5b0 con 0x7fe8580aee70 2023-10-06T05:21:09.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.044+0000 7fe857fff700 1 -- 172.21.15.31:0/1613279771 >> v1:172.21.15.62:6789/0 conn(0x7fe8580aee70 legacy=0x7fe85813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.044+0000 7fe857fff700 1 -- 172.21.15.31:0/1613279771 >> v1:172.21.15.31:6789/0 conn(0x7fe8580c7f30 legacy=0x7fe85806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.044+0000 7fe857fff700 1 -- 172.21.15.31:0/1613279771 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580c9c80 con 0x7fe8580c7b60 2023-10-06T05:21:09.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.044+0000 7fe857fff700 1 --2- 172.21.15.31:0/1613279771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7b60 0x7fe858069880 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe848009900 tx=0x7fe848009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e99993891cee6ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:09.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.044+0000 7fe856ffd700 1 -- 172.21.15.31:0/1613279771 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848005c10 con 0x7fe8580c7b60 2023-10-06T05:21:09.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.045+0000 7fe856ffd700 1 -- 172.21.15.31:0/1613279771 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe848005d70 con 0x7fe8580c7b60 2023-10-06T05:21:09.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.045+0000 7fe856ffd700 1 -- 172.21.15.31:0/1613279771 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848006260 con 0x7fe8580c7b60 2023-10-06T05:21:09.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.045+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1613279771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7b60 msgr2=0x7fe858069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.045+0000 7fe85e9ff700 1 --2- 172.21.15.31:0/1613279771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7b60 0x7fe858069880 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe848009900 tx=0x7fe848009120 comp rx=0 tx=0).stop 2023-10-06T05:21:09.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.045+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1613279771 shutdown_connections 2023-10-06T05:21:09.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.045+0000 7fe85e9ff700 1 --2- 172.21.15.31:0/1613279771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7b60 0x7fe858069880 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.045+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1613279771 >> 172.21.15.31:0/1613279771 conn(0x7fe8580cc0b0 msgr2=0x7fe8580cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.045+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1613279771 shutdown_connections 2023-10-06T05:21:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.045+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1613279771 wait complete. 2023-10-06T05:21:09.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.047+0000 7fe85e9ff700 1 Processor -- start 2023-10-06T05:21:09.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.047+0000 7fe85e9ff700 1 -- start start 2023-10-06T05:21:09.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe85e9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8580aee70 0x7fe8580adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe85e9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580c7b60 0x7fe85816ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe85e9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7f30 0x7fe85816b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe85e9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858158270 con 0x7fe8580c7b60 2023-10-06T05:21:09.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe85e9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858157660 con 0x7fe8580aee70 2023-10-06T05:21:09.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe85e9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe858157940 con 0x7fe8580c7f30 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe8577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580c7b60 0x7fe85816ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe857fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8580aee70 0x7fe8580adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe8577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580c7b60 0x7fe85816ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60254/0 (socket says 172.21.15.31:60254) 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe8577fe700 1 -- 172.21.15.31:0/1912051700 learned_addr learned my addr 172.21.15.31:0/1912051700 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe85cf9c700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7f30 0x7fe85816b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe8577fe700 1 -- 172.21.15.31:0/1912051700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7f30 msgr2=0x7fe85816b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe8577fe700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7f30 0x7fe85816b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe8577fe700 1 -- 172.21.15.31:0/1912051700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8580aee70 msgr2=0x7fe8580adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe8577fe700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8580aee70 0x7fe8580adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.048+0000 7fe8577fe700 1 -- 172.21.15.31:0/1912051700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480093c0 con 0x7fe8580c7b60 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.049+0000 7fe8577fe700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580c7b60 0x7fe85816ae70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00c790 tx=0x7fe84c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:09.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.049+0000 7fe83ffff700 1 -- 172.21.15.31:0/1912051700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c005b30 con 0x7fe8580c7b60 2023-10-06T05:21:09.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.049+0000 7fe83ffff700 1 -- 172.21.15.31:0/1912051700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c005cd0 con 0x7fe8580c7b60 2023-10-06T05:21:09.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.049+0000 7fe83ffff700 1 -- 172.21.15.31:0/1912051700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c007320 con 0x7fe8580c7b60 2023-10-06T05:21:09.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.049+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1912051700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe858157c20 con 0x7fe8580c7b60 2023-10-06T05:21:09.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.049+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1912051700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe858157360 con 0x7fe8580c7b60 2023-10-06T05:21:09.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.050+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1912051700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8581583b0 con 0x7fe8580c7b60 2023-10-06T05:21:09.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.051+0000 7fe83ffff700 1 -- 172.21.15.31:0/1912051700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe84c012400 con 0x7fe8580c7b60 2023-10-06T05:21:09.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.052+0000 7fe83ffff700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe844067290 0x7fe844069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.052+0000 7fe83ffff700 1 -- 172.21.15.31:0/1912051700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe84c083d00 con 0x7fe8580c7b60 2023-10-06T05:21:09.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.052+0000 7fe857fff700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe844067290 0x7fe844069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.053+0000 7fe857fff700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe844067290 0x7fe844069740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe848009160 tx=0x7fe848016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:09.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.056+0000 7fe83ffff700 1 -- 172.21.15.31:0/1912051700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe84c04b5a0 con 0x7fe8580c7b60 2023-10-06T05:21:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.342+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1912051700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe85804c380 con 0x7fe8580c7b60 2023-10-06T05:21:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.343+0000 7fe83ffff700 1 -- 172.21.15.31:0/1912051700 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5632 (secure 0 0 0) 0x7fe84c017020 con 0x7fe8580c7b60 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:21:09.345 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:21:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:21:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:21:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:21:09.346 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:21:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:21:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:21:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:21:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.350 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.351 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:09.352 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:21:09.354 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:09.355 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:21:09.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:21:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:21:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:09.357 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:21:09.358 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:21:09.359 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.alice", 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQA1mR9lk52xCBAAn0EGSnWA66PUAhI58soddA==", 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow rw path=/volumes/_nogroup/subvolume_0000000000234638", 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw pool=cephfs_data" 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:21:09.361 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:21:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:21:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:09.363 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:21:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1912051700 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe844067290 msgr2=0x7fe844069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe844067290 0x7fe844069740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe848009160 tx=0x7fe848016000 comp rx=0 tx=0).stop 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1912051700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580c7b60 msgr2=0x7fe85816ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580c7b60 0x7fe85816ae70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00c790 tx=0x7fe84c007e80 comp rx=0 tx=0).stop 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1912051700 shutdown_connections 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8580c7f30 0x7fe85816b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8580aee70 0x7fe8580adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe844067290 0x7fe844069740 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 --2- 172.21.15.31:0/1912051700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8580c7b60 0x7fe85816ae70 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1912051700 >> 172.21.15.31:0/1912051700 conn(0x7fe8580cc0b0 msgr2=0x7fe8580b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1912051700 shutdown_connections 2023-10-06T05:21:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.345+0000 7fe85e9ff700 1 -- 172.21.15.31:0/1912051700 wait complete. 2023-10-06T05:21:09.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2023-10-06T05:21:09.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.561+0000 7f7ad55ff700 1 Processor -- start 2023-10-06T05:21:09.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.561+0000 7f7ad55ff700 1 -- start start 2023-10-06T05:21:09.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.562+0000 7f7ad55ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ad013a1e0 0x7f7ad00c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.562+0000 7f7ad55ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7ad006db50 0x7f7ad0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.562+0000 7f7ad55ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad00c71b0 con 0x7f7ad013a5b0 2023-10-06T05:21:09.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.562+0000 7f7ad55ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad00c7490 con 0x7f7ad006db50 2023-10-06T05:21:09.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.562+0000 7f7ad55ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00c7770 con 0x7f7ad013a1e0 2023-10-06T05:21:09.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.563+0000 7f7acf7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7ad006db50 0x7f7ad0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.563+0000 7f7aceffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ad013a1e0 0x7f7ad00c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.563+0000 7f7ace7fc700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7ad013a5b0 0x7f7ad006d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:52202/0 (socket says 172.21.15.31:52202) 2023-10-06T05:21:09.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.563+0000 7f7ace7fc700 1 -- 172.21.15.31:0/737501906 learned_addr learned my addr 172.21.15.31:0/737501906 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:09.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.563+0000 7f7acf7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7ad006db50 0x7f7ad0069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:55460/0 (socket says 172.21.15.31:55460) 2023-10-06T05:21:09.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.563+0000 7f7acf7fe700 1 -- 172.21.15.31:0/737501906 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ad013a1e0 msgr2=0x7f7ad00c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:09.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.563+0000 7f7acf7fe700 1 --2- 172.21.15.31:0/737501906 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ad013a1e0 0x7f7ad00c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.563+0000 7f7acf7fe700 1 -- 172.21.15.31:0/737501906 >> v1:172.21.15.62:6789/0 conn(0x7f7ad013a5b0 legacy=0x7f7ad006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.563+0000 7f7acf7fe700 1 -- 172.21.15.31:0/737501906 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad00c7a50 con 0x7f7ad006db50 2023-10-06T05:21:09.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.563+0000 7f7acdffb700 1 -- 172.21.15.31:0/737501906 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83289457 0 0) 0x7f7ad00c71b0 con 0x7f7ad013a5b0 2023-10-06T05:21:09.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.564+0000 7f7acf7fe700 1 --2- 172.21.15.31:0/737501906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad006db50 0x7f7ad0069560 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009e60 tx=0x7f7ac8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b24c42008ff6b85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:09.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.564+0000 7f7acdffb700 1 -- 172.21.15.31:0/737501906 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac800e070 con 0x7f7ad006db50 2023-10-06T05:21:09.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.564+0000 7f7acdffb700 1 -- 172.21.15.31:0/737501906 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac800bb60 con 0x7f7ad006db50 2023-10-06T05:21:09.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.564+0000 7f7acdffb700 1 -- 172.21.15.31:0/737501906 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8012430 con 0x7f7ad006db50 2023-10-06T05:21:09.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.565+0000 7f7ad55ff700 1 -- 172.21.15.31:0/737501906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad006db50 msgr2=0x7f7ad0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.565+0000 7f7ad55ff700 1 --2- 172.21.15.31:0/737501906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad006db50 0x7f7ad0069560 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009e60 tx=0x7f7ac8009780 comp rx=0 tx=0).stop 2023-10-06T05:21:09.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.565+0000 7f7ad55ff700 1 -- 172.21.15.31:0/737501906 shutdown_connections 2023-10-06T05:21:09.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.565+0000 7f7ad55ff700 1 --2- 172.21.15.31:0/737501906 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ad013a1e0 0x7f7ad00c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.565+0000 7f7ad55ff700 1 --2- 172.21.15.31:0/737501906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad006db50 0x7f7ad0069560 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.565+0000 7f7ad55ff700 1 -- 172.21.15.31:0/737501906 >> 172.21.15.31:0/737501906 conn(0x7f7ad00cc0b0 msgr2=0x7f7ad00cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:09.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.565+0000 7f7ad55ff700 1 -- 172.21.15.31:0/737501906 shutdown_connections 2023-10-06T05:21:09.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.565+0000 7f7ad55ff700 1 -- 172.21.15.31:0/737501906 wait complete. 2023-10-06T05:21:09.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.567+0000 7f7ad55ff700 1 Processor -- start 2023-10-06T05:21:09.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.567+0000 7f7ad55ff700 1 -- start start 2023-10-06T05:21:09.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7ad55ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad006db50 0x7f7ad0157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7ad55ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad013a1e0 0x7f7ad00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7ad55ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ad013a5b0 0x7f7ad0137a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7ad55ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00c9c80 con 0x7f7ad013a1e0 2023-10-06T05:21:09.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7ad55ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00c5940 con 0x7f7ad006db50 2023-10-06T05:21:09.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7ad55ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00c5c20 con 0x7f7ad013a5b0 2023-10-06T05:21:09.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7ace7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad013a1e0 0x7f7ad00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7ace7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad013a1e0 0x7f7ad00c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60268/0 (socket says 172.21.15.31:60268) 2023-10-06T05:21:09.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7ace7fc700 1 -- 172.21.15.31:0/1078322869 learned_addr learned my addr 172.21.15.31:0/1078322869 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:09.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7aceffd700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad006db50 0x7f7ad0157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.568+0000 7f7acf7fe700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ad013a5b0 0x7f7ad0137a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.569+0000 7f7ace7fc700 1 -- 172.21.15.31:0/1078322869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ad013a5b0 msgr2=0x7f7ad0137a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.569+0000 7f7ace7fc700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ad013a5b0 0x7f7ad0137a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.569+0000 7f7ace7fc700 1 -- 172.21.15.31:0/1078322869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad006db50 msgr2=0x7f7ad0157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.569+0000 7f7ace7fc700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad006db50 0x7f7ad0157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.569+0000 7f7ace7fc700 1 -- 172.21.15.31:0/1078322869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac8009a20 con 0x7f7ad013a1e0 2023-10-06T05:21:09.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.569+0000 7f7ace7fc700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad013a1e0 0x7f7ad00c50a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c5b0 tx=0x7f7ac4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:09.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.569+0000 7f7ab6ffd700 1 -- 172.21.15.31:0/1078322869 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4005b30 con 0x7f7ad013a1e0 2023-10-06T05:21:09.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.569+0000 7f7ab6ffd700 1 -- 172.21.15.31:0/1078322869 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac4005c90 con 0x7f7ad013a1e0 2023-10-06T05:21:09.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.569+0000 7f7ab6ffd700 1 -- 172.21.15.31:0/1078322869 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4007300 con 0x7f7ad013a1e0 2023-10-06T05:21:09.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.569+0000 7f7ad55ff700 1 -- 172.21.15.31:0/1078322869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad00c5f00 con 0x7f7ad013a1e0 2023-10-06T05:21:09.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.570+0000 7f7ad55ff700 1 -- 172.21.15.31:0/1078322869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad00c55e0 con 0x7f7ad013a1e0 2023-10-06T05:21:09.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.571+0000 7f7ad55ff700 1 -- 172.21.15.31:0/1078322869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad00c9e30 con 0x7f7ad013a1e0 2023-10-06T05:21:09.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.572+0000 7f7ab6ffd700 1 -- 172.21.15.31:0/1078322869 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7ac401b3b0 con 0x7f7ad013a1e0 2023-10-06T05:21:09.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.573+0000 7f7ab6ffd700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7abc067290 0x7f7abc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:09.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.574+0000 7f7ab6ffd700 1 -- 172.21.15.31:0/1078322869 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7ac4083cf0 con 0x7f7ad013a1e0 2023-10-06T05:21:09.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.574+0000 7f7aceffd700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7abc067290 0x7f7abc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:09.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.574+0000 7f7aceffd700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7abc067290 0x7f7abc069740 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009da0 tx=0x7f7ac0009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:09.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.577+0000 7f7ab6ffd700 1 -- 172.21.15.31:0/1078322869 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ad00c9e30 con 0x7f7ad013a1e0 2023-10-06T05:21:09.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.860+0000 7f7ad55ff700 1 -- 172.21.15.31:0/1078322869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) v1 -- 0x7f7ad00c9e30 con 0x7f7ad013a1e0 2023-10-06T05:21:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.864+0000 7f7ab6ffd700 1 -- 172.21.15.31:0/1078322869 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v15) v1 ==== 82+0+0 (secure 0 0 0) 0x7f7ad00c9e30 con 0x7f7ad013a1e0 2023-10-06T05:21:09.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.866+0000 7f7ad55ff700 1 -- 172.21.15.31:0/1078322869 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7abc067290 msgr2=0x7f7abc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.866+0000 7f7ad55ff700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7abc067290 0x7f7abc069740 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009da0 tx=0x7f7ac0009120 comp rx=0 tx=0).stop 2023-10-06T05:21:09.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.866+0000 7f7ad55ff700 1 -- 172.21.15.31:0/1078322869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad013a1e0 msgr2=0x7f7ad00c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.866+0000 7f7ad55ff700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad013a1e0 0x7f7ad00c50a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c5b0 tx=0x7f7ac4001070 comp rx=0 tx=0).stop 2023-10-06T05:21:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.866+0000 7f7ad55ff700 1 -- 172.21.15.31:0/1078322869 shutdown_connections 2023-10-06T05:21:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.866+0000 7f7ad55ff700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ad013a5b0 0x7f7ad0137a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.866+0000 7f7ad55ff700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad006db50 0x7f7ad0157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.866+0000 7f7ad55ff700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f7abc067290 0x7f7abc069740 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.866+0000 7f7ad55ff700 1 --2- 172.21.15.31:0/1078322869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad013a1e0 0x7f7ad00c50a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.866+0000 7f7ad55ff700 1 -- 172.21.15.31:0/1078322869 >> 172.21.15.31:0/1078322869 conn(0x7f7ad00cc0b0 msgr2=0x7f7ad00b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.867+0000 7f7ad55ff700 1 -- 172.21.15.31:0/1078322869 shutdown_connections 2023-10-06T05:21:09.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:09.867+0000 7f7ad55ff700 1 -- 172.21.15.31:0/1078322869 wait complete. 2023-10-06T05:21:09.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:10.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.079+0000 7f236a9ff700 1 Processor -- start 2023-10-06T05:21:10.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.079+0000 7f236a9ff700 1 -- start start 2023-10-06T05:21:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f236a9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f236413a1e0 0x7f23640c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f236a9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f236413a5b0 0x7f236406d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f236a9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23640c71b0 con 0x7f23640c6c30 2023-10-06T05:21:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f236a9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23640c7490 con 0x7f236413a5b0 2023-10-06T05:21:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f236a9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23640c7770 con 0x7f236413a1e0 2023-10-06T05:21:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f23637fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f236413a5b0 0x7f236406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f2368f9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f23640c6c30 0x7f236415afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41226/0 (socket says 172.21.15.31:41226) 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f23637fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f236413a5b0 0x7f236406d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60274/0 (socket says 172.21.15.31:60274) 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f2368f9c700 1 -- 172.21.15.31:0/358732208 learned_addr learned my addr 172.21.15.31:0/358732208 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f2363fff700 1 --2- 172.21.15.31:0/358732208 >> v2:172.21.15.62:3300/0 conn(0x7f236413a1e0 0x7f23640c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f23637fe700 1 -- 172.21.15.31:0/358732208 >> v2:172.21.15.62:3300/0 conn(0x7f236413a1e0 msgr2=0x7f23640c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f23637fe700 1 --2- 172.21.15.31:0/358732208 >> v2:172.21.15.62:3300/0 conn(0x7f236413a1e0 0x7f23640c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.081+0000 7f23637fe700 1 -- 172.21.15.31:0/358732208 >> v1:172.21.15.31:6789/0 conn(0x7f23640c6c30 legacy=0x7f236415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.082+0000 7f23637fe700 1 -- 172.21.15.31:0/358732208 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23640c7a50 con 0x7f236413a5b0 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.082+0000 7f2362ffd700 1 -- 172.21.15.31:0/358732208 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391623200 0 0) 0x7f23640c71b0 con 0x7f23640c6c30 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.082+0000 7f23637fe700 1 --2- 172.21.15.31:0/358732208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 0x7f236406d440 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f23640cbf90 tx=0x7f23580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7e437f590d3404 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.082+0000 7f2362ffd700 1 -- 172.21.15.31:0/358732208 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235800b6b0 con 0x7f236413a5b0 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.082+0000 7f2362ffd700 1 -- 172.21.15.31:0/358732208 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235800b810 con 0x7f236413a5b0 2023-10-06T05:21:10.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.082+0000 7f2362ffd700 1 -- 172.21.15.31:0/358732208 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235800bbb0 con 0x7f236413a5b0 2023-10-06T05:21:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.083+0000 7f236a9ff700 1 -- 172.21.15.31:0/358732208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 msgr2=0x7f236406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.083+0000 7f236a9ff700 1 --2- 172.21.15.31:0/358732208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 0x7f236406d440 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f23640cbf90 tx=0x7f23580091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.083+0000 7f236a9ff700 1 -- 172.21.15.31:0/358732208 shutdown_connections 2023-10-06T05:21:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.083+0000 7f236a9ff700 1 --2- 172.21.15.31:0/358732208 >> v2:172.21.15.62:3300/0 conn(0x7f236413a1e0 0x7f23640c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.083+0000 7f236a9ff700 1 --2- 172.21.15.31:0/358732208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 0x7f236406d440 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.083+0000 7f236a9ff700 1 -- 172.21.15.31:0/358732208 >> 172.21.15.31:0/358732208 conn(0x7f23640cc0b0 msgr2=0x7f23640cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.083+0000 7f236a9ff700 1 -- 172.21.15.31:0/358732208 shutdown_connections 2023-10-06T05:21:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.083+0000 7f236a9ff700 1 -- 172.21.15.31:0/358732208 wait complete. 2023-10-06T05:21:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.084+0000 7f236a9ff700 1 Processor -- start 2023-10-06T05:21:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.084+0000 7f236a9ff700 1 -- start start 2023-10-06T05:21:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.085+0000 7f236a9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23640c6c30 0x7f23640c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.085+0000 7f236a9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236413a1e0 0x7f2364158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.085+0000 7f236a9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 0x7f23641593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.085+0000 7f236a9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2364159be0 con 0x7f236413a5b0 2023-10-06T05:21:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.085+0000 7f236a9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2364159ec0 con 0x7f236413a1e0 2023-10-06T05:21:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.085+0000 7f236a9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236415a1a0 con 0x7f23640c6c30 2023-10-06T05:21:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f2368f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 0x7f23641593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f2368f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 0x7f23641593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60276/0 (socket says 172.21.15.31:60276) 2023-10-06T05:21:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f2368f9c700 1 -- 172.21.15.31:0/3977076058 learned_addr learned my addr 172.21.15.31:0/3977076058 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f2363fff700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23640c6c30 0x7f23640c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f23637fe700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236413a1e0 0x7f2364158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f2368f9c700 1 -- 172.21.15.31:0/3977076058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23640c6c30 msgr2=0x7f23640c3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f2368f9c700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23640c6c30 0x7f23640c3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f2368f9c700 1 -- 172.21.15.31:0/3977076058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236413a1e0 msgr2=0x7f2364158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f2368f9c700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236413a1e0 0x7f2364158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f2368f9c700 1 -- 172.21.15.31:0/3977076058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358009420 con 0x7f236413a5b0 2023-10-06T05:21:10.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.086+0000 7f2368f9c700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 0x7f23641593b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f235c00cc90 tx=0x7f235c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.087+0000 7f234bfff700 1 -- 172.21.15.31:0/3977076058 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c00cea0 con 0x7f236413a5b0 2023-10-06T05:21:10.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.087+0000 7f234bfff700 1 -- 172.21.15.31:0/3977076058 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c00ab70 con 0x7f236413a5b0 2023-10-06T05:21:10.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.087+0000 7f236a9ff700 1 -- 172.21.15.31:0/3977076058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236415a480 con 0x7f236413a5b0 2023-10-06T05:21:10.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.087+0000 7f234bfff700 1 -- 172.21.15.31:0/3977076058 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c0070f0 con 0x7f236413a5b0 2023-10-06T05:21:10.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.087+0000 7f236a9ff700 1 -- 172.21.15.31:0/3977076058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23641598f0 con 0x7f236413a5b0 2023-10-06T05:21:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.088+0000 7f236a9ff700 1 -- 172.21.15.31:0/3977076058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236415a9a0 con 0x7f236413a5b0 2023-10-06T05:21:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.089+0000 7f234bfff700 1 -- 172.21.15.31:0/3977076058 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f235c0074a0 con 0x7f236413a5b0 2023-10-06T05:21:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.090+0000 7f234bfff700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2350067290 0x7f2350069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.090+0000 7f234bfff700 1 -- 172.21.15.31:0/3977076058 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f235c084e40 con 0x7f236413a5b0 2023-10-06T05:21:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.090+0000 7f2363fff700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2350067290 0x7f2350069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.090+0000 7f2363fff700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2350067290 0x7f2350069740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2354006aa0 tx=0x7f2354008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.094+0000 7f234bfff700 1 -- 172.21.15.31:0/3977076058 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f235c051dd0 con 0x7f236413a5b0 2023-10-06T05:21:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.302+0000 7f236a9ff700 1 -- 172.21.15.31:0/3977076058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f236404c380 con 0x7f236413a5b0 2023-10-06T05:21:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.303+0000 7f234bfff700 1 -- 172.21.15.31:0/3977076058 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+13699 (secure 0 0 0) 0x7f235c0553f0 con 0x7f236413a5b0 2023-10-06T05:21:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":67,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:08.812586+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:10.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.305+0000 7f236a9ff700 1 -- 172.21.15.31:0/3977076058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2350067290 msgr2=0x7f2350069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.305+0000 7f236a9ff700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2350067290 0x7f2350069740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2354006aa0 tx=0x7f2354008040 comp rx=0 tx=0).stop 2023-10-06T05:21:10.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.305+0000 7f236a9ff700 1 -- 172.21.15.31:0/3977076058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 msgr2=0x7f23641593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.305+0000 7f236a9ff700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 0x7f23641593b0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f235c00cc90 tx=0x7f235c001a80 comp rx=0 tx=0).stop 2023-10-06T05:21:10.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.306+0000 7f236a9ff700 1 -- 172.21.15.31:0/3977076058 shutdown_connections 2023-10-06T05:21:10.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.306+0000 7f236a9ff700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2350067290 0x7f2350069740 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.306+0000 7f236a9ff700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f236413a5b0 0x7f23641593b0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.306+0000 7f236a9ff700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23640c6c30 0x7f23640c3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.306+0000 7f236a9ff700 1 --2- 172.21.15.31:0/3977076058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236413a1e0 0x7f2364158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.306+0000 7f236a9ff700 1 -- 172.21.15.31:0/3977076058 >> 172.21.15.31:0/3977076058 conn(0x7f23640cc0b0 msgr2=0x7f236406d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.306+0000 7f236a9ff700 1 -- 172.21.15.31:0/3977076058 shutdown_connections 2023-10-06T05:21:10.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.306+0000 7f236a9ff700 1 -- 172.21.15.31:0/3977076058 wait complete. 2023-10-06T05:21:10.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:21:10.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:21:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.522+0000 7f4c463ff700 1 Processor -- start 2023-10-06T05:21:10.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.523+0000 7f4c463ff700 1 -- start start 2023-10-06T05:21:10.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.524+0000 7f4c463ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4c401573a0 0x7f4c400acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.524+0000 7f4c463ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c40158cf0 con 0x7f4c400ae860 2023-10-06T05:21:10.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.524+0000 7f4c463ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c401580e0 con 0x7f4c40157770 2023-10-06T05:21:10.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.524+0000 7f4c463ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c401583c0 con 0x7f4c401573a0 2023-10-06T05:21:10.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.524+0000 7f4c3ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4c401573a0 0x7f4c400acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.524+0000 7f4c4499c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4c400ae860 0x7f4c401611e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41230/0 (socket says 172.21.15.31:41230) 2023-10-06T05:21:10.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.524+0000 7f4c4499c700 1 -- 172.21.15.31:0/2368454943 learned_addr learned my addr 172.21.15.31:0/2368454943 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:10.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.525+0000 7f4c3ffff700 1 -- 172.21.15.31:0/2368454943 >> v1:172.21.15.62:6789/0 conn(0x7f4c40157770 legacy=0x7f4c400b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:10.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.525+0000 7f4c3ffff700 1 -- 172.21.15.31:0/2368454943 >> v1:172.21.15.31:6789/0 conn(0x7f4c400ae860 legacy=0x7f4c401611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.525+0000 7f4c3ffff700 1 -- 172.21.15.31:0/2368454943 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c401586a0 con 0x7f4c401573a0 2023-10-06T05:21:10.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.525+0000 7f4c3ffff700 1 --2- 172.21.15.31:0/2368454943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 0x7f4c400acb70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4c300098e0 tx=0x7f4c300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ce7007bb00d35c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.525+0000 7f4c3effd700 1 -- 172.21.15.31:0/2368454943 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30009e80 con 0x7f4c401573a0 2023-10-06T05:21:10.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.525+0000 7f4c3effd700 1 -- 172.21.15.31:0/2368454943 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3000b820 con 0x7f4c401573a0 2023-10-06T05:21:10.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.525+0000 7f4c3effd700 1 -- 172.21.15.31:0/2368454943 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3000bba0 con 0x7f4c401573a0 2023-10-06T05:21:10.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.526+0000 7f4c463ff700 1 -- 172.21.15.31:0/2368454943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 msgr2=0x7f4c400acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.526+0000 7f4c463ff700 1 --2- 172.21.15.31:0/2368454943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 0x7f4c400acb70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4c300098e0 tx=0x7f4c300091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.526+0000 7f4c463ff700 1 -- 172.21.15.31:0/2368454943 shutdown_connections 2023-10-06T05:21:10.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.526+0000 7f4c463ff700 1 --2- 172.21.15.31:0/2368454943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 0x7f4c400acb70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.526+0000 7f4c463ff700 1 -- 172.21.15.31:0/2368454943 >> 172.21.15.31:0/2368454943 conn(0x7f4c40067560 msgr2=0x7f4c40067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:10.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.526+0000 7f4c463ff700 1 -- 172.21.15.31:0/2368454943 shutdown_connections 2023-10-06T05:21:10.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.526+0000 7f4c463ff700 1 -- 172.21.15.31:0/2368454943 wait complete. 2023-10-06T05:21:10.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.527+0000 7f4c463ff700 1 Processor -- start 2023-10-06T05:21:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.528+0000 7f4c463ff700 1 -- start start 2023-10-06T05:21:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.528+0000 7f4c463ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 0x7f4c400aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c463ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c40157770 0x7f4c4014bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c463ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c40155710 0x7f4c400aa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c463ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c401548f0 con 0x7f4c401573a0 2023-10-06T05:21:10.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c463ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c40153a00 con 0x7f4c40157770 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c463ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c40153ce0 con 0x7f4c40155710 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c3ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 0x7f4c400aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c3ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 0x7f4c400aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60286/0 (socket says 172.21.15.31:60286) 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c3ffff700 1 -- 172.21.15.31:0/3766430423 learned_addr learned my addr 172.21.15.31:0/3766430423 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c4499c700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c40155710 0x7f4c400aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c3f7fe700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c40157770 0x7f4c4014bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c3ffff700 1 -- 172.21.15.31:0/3766430423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c40155710 msgr2=0x7f4c400aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c3ffff700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c40155710 0x7f4c400aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c3ffff700 1 -- 172.21.15.31:0/3766430423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c40157770 msgr2=0x7f4c4014bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c3ffff700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c40157770 0x7f4c4014bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.529+0000 7f4c3ffff700 1 -- 172.21.15.31:0/3766430423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c30009420 con 0x7f4c401573a0 2023-10-06T05:21:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.530+0000 7f4c3ffff700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 0x7f4c400aff00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4c300030c0 tx=0x7f4c30016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.530+0000 7f4c27fff700 1 -- 172.21.15.31:0/3766430423 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3001b470 con 0x7f4c401573a0 2023-10-06T05:21:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.530+0000 7f4c27fff700 1 -- 172.21.15.31:0/3766430423 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3001b5f0 con 0x7f4c401573a0 2023-10-06T05:21:10.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.530+0000 7f4c27fff700 1 -- 172.21.15.31:0/3766430423 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3001b9b0 con 0x7f4c401573a0 2023-10-06T05:21:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.530+0000 7f4c463ff700 1 -- 172.21.15.31:0/3766430423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c40153fc0 con 0x7f4c401573a0 2023-10-06T05:21:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.530+0000 7f4c463ff700 1 -- 172.21.15.31:0/3766430423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c40154580 con 0x7f4c401573a0 2023-10-06T05:21:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.531+0000 7f4c463ff700 1 -- 172.21.15.31:0/3766430423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c40154740 con 0x7f4c401573a0 2023-10-06T05:21:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.532+0000 7f4c27fff700 1 -- 172.21.15.31:0/3766430423 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f4c3001c450 con 0x7f4c401573a0 2023-10-06T05:21:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.533+0000 7f4c27fff700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c2c067290 0x7f4c2c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.533+0000 7f4c27fff700 1 -- 172.21.15.31:0/3766430423 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4c300865a0 con 0x7f4c401573a0 2023-10-06T05:21:10.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.533+0000 7f4c3f7fe700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c2c067290 0x7f4c2c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.534+0000 7f4c3f7fe700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c2c067290 0x7f4c2c069740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4c400c8200 tx=0x7f4c34007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.537+0000 7f4c27fff700 1 -- 172.21.15.31:0/3766430423 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c300535d0 con 0x7f4c401573a0 2023-10-06T05:21:10.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.738+0000 7f4c463ff700 1 -- 172.21.15.31:0/3766430423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4c4004c380 con 0x7f4c401573a0 2023-10-06T05:21:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.738+0000 7f4c27fff700 1 -- 172.21.15.31:0/3766430423 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v67) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4c3002a0a0 con 0x7f4c401573a0 2023-10-06T05:21:10.740 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:21:10.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.740+0000 7f4c463ff700 1 -- 172.21.15.31:0/3766430423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c2c067290 msgr2=0x7f4c2c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.740+0000 7f4c463ff700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c2c067290 0x7f4c2c069740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4c400c8200 tx=0x7f4c34007ea0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.741+0000 7f4c463ff700 1 -- 172.21.15.31:0/3766430423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 msgr2=0x7f4c400aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.741+0000 7f4c463ff700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 0x7f4c400aff00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4c300030c0 tx=0x7f4c30016000 comp rx=0 tx=0).stop 2023-10-06T05:21:10.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.741+0000 7f4c463ff700 1 -- 172.21.15.31:0/3766430423 shutdown_connections 2023-10-06T05:21:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.741+0000 7f4c463ff700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c40155710 0x7f4c400aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.741+0000 7f4c463ff700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4c2c067290 0x7f4c2c069740 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.741+0000 7f4c463ff700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c401573a0 0x7f4c400aff00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.741+0000 7f4c463ff700 1 --2- 172.21.15.31:0/3766430423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c40157770 0x7f4c4014bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.741+0000 7f4c463ff700 1 -- 172.21.15.31:0/3766430423 >> 172.21.15.31:0/3766430423 conn(0x7f4c40067560 msgr2=0x7f4c400695d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:10.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.741+0000 7f4c463ff700 1 -- 172.21.15.31:0/3766430423 shutdown_connections 2023-10-06T05:21:10.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.741+0000 7f4c463ff700 1 -- 172.21.15.31:0/3766430423 wait complete. 2023-10-06T05:21:10.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:21:10.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.957+0000 7fa349fff700 1 Processor -- start 2023-10-06T05:21:10.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.957+0000 7fa349fff700 1 -- start start 2023-10-06T05:21:10.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.959+0000 7fa349fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa3440b2540 0x7fa3440cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.959+0000 7fa349fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa34415a4b0 0x7fa3440aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.959+0000 7fa349fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa34415a020 con 0x7fa3440cc340 2023-10-06T05:21:10.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.959+0000 7fa349fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3440ccc50 con 0x7fa3440b2540 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.959+0000 7fa349fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3440b07e0 con 0x7fa34415a4b0 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.959+0000 7fa342ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa3440b2540 0x7fa3440cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.959+0000 7fa342ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa3440b2540 0x7fa3440cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60294/0 (socket says 172.21.15.31:60294) 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.959+0000 7fa342ffd700 1 -- 172.21.15.31:0/1723371821 learned_addr learned my addr 172.21.15.31:0/1723371821 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.959+0000 7fa343fff700 1 --2- 172.21.15.31:0/1723371821 >> v2:172.21.15.62:3300/0 conn(0x7fa34415a4b0 0x7fa3440aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.960+0000 7fa342ffd700 1 -- 172.21.15.31:0/1723371821 >> v2:172.21.15.62:3300/0 conn(0x7fa34415a4b0 msgr2=0x7fa3440aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.960+0000 7fa342ffd700 1 --2- 172.21.15.31:0/1723371821 >> v2:172.21.15.62:3300/0 conn(0x7fa34415a4b0 0x7fa3440aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.960+0000 7fa342ffd700 1 -- 172.21.15.31:0/1723371821 >> v1:172.21.15.31:6789/0 conn(0x7fa3440cc340 legacy=0x7fa3440b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.960+0000 7fa342ffd700 1 -- 172.21.15.31:0/1723371821 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa344159cf0 con 0x7fa3440b2540 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.960+0000 7fa343fff700 1 --2- 172.21.15.31:0/1723371821 >> v2:172.21.15.62:3300/0 conn(0x7fa34415a4b0 0x7fa3440aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.960+0000 7fa3427fc700 1 -- 172.21.15.31:0/1723371821 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345127086 0 0) 0x7fa34415a020 con 0x7fa3440cc340 2023-10-06T05:21:10.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.960+0000 7fa342ffd700 1 --2- 172.21.15.31:0/1723371821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440b2540 0x7fa3440cc710 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa3380099b0 tx=0x7fa338009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbab91dd0b40152b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.960+0000 7fa3427fc700 1 -- 172.21.15.31:0/1723371821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33800b820 con 0x7fa3440b2540 2023-10-06T05:21:10.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.960+0000 7fa3427fc700 1 -- 172.21.15.31:0/1723371821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33800b980 con 0x7fa3440b2540 2023-10-06T05:21:10.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.960+0000 7fa3427fc700 1 -- 172.21.15.31:0/1723371821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33800bd20 con 0x7fa3440b2540 2023-10-06T05:21:10.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.961+0000 7fa349fff700 1 -- 172.21.15.31:0/1723371821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440b2540 msgr2=0x7fa3440cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.961+0000 7fa349fff700 1 --2- 172.21.15.31:0/1723371821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440b2540 0x7fa3440cc710 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa3380099b0 tx=0x7fa338009250 comp rx=0 tx=0).stop 2023-10-06T05:21:10.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.961+0000 7fa349fff700 1 -- 172.21.15.31:0/1723371821 shutdown_connections 2023-10-06T05:21:10.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.961+0000 7fa349fff700 1 --2- 172.21.15.31:0/1723371821 >> v2:172.21.15.62:3300/0 conn(0x7fa34415a4b0 0x7fa3440aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.961+0000 7fa349fff700 1 --2- 172.21.15.31:0/1723371821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440b2540 0x7fa3440cc710 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.961+0000 7fa349fff700 1 -- 172.21.15.31:0/1723371821 >> 172.21.15.31:0/1723371821 conn(0x7fa344068670 msgr2=0x7fa344068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:10.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.961+0000 7fa349fff700 1 -- 172.21.15.31:0/1723371821 shutdown_connections 2023-10-06T05:21:10.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.961+0000 7fa349fff700 1 -- 172.21.15.31:0/1723371821 wait complete. 2023-10-06T05:21:10.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.962+0000 7fa349fff700 1 Processor -- start 2023-10-06T05:21:10.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.962+0000 7fa349fff700 1 -- start start 2023-10-06T05:21:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.963+0000 7fa349fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3440b2540 0x7fa3441569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.963+0000 7fa349fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440cc340 0x7fa3440af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.963+0000 7fa349fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa34415a4b0 0x7fa3440af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.963+0000 7fa349fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34416e250 con 0x7fa3440cc340 2023-10-06T05:21:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.963+0000 7fa349fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34416d3a0 con 0x7fa3440b2540 2023-10-06T05:21:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.963+0000 7fa349fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34416d640 con 0x7fa34415a4b0 2023-10-06T05:21:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa342ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440cc340 0x7fa3440af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa342ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440cc340 0x7fa3440af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60302/0 (socket says 172.21.15.31:60302) 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa342ffd700 1 -- 172.21.15.31:0/2723220227 learned_addr learned my addr 172.21.15.31:0/2723220227 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa343fff700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa34415a4b0 0x7fa3440af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa3437fe700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3440b2540 0x7fa3441569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa342ffd700 1 -- 172.21.15.31:0/2723220227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa34415a4b0 msgr2=0x7fa3440af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa342ffd700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa34415a4b0 0x7fa3440af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa342ffd700 1 -- 172.21.15.31:0/2723220227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3440b2540 msgr2=0x7fa3441569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa342ffd700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3440b2540 0x7fa3441569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa342ffd700 1 -- 172.21.15.31:0/2723220227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3380094f0 con 0x7fa3440cc340 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa3437fe700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3440b2540 0x7fa3441569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.964+0000 7fa342ffd700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440cc340 0x7fa3440af1e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa3380090b0 tx=0x7fa338017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.965+0000 7fa32b7fe700 1 -- 172.21.15.31:0/2723220227 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33800bd00 con 0x7fa3440cc340 2023-10-06T05:21:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.965+0000 7fa32b7fe700 1 -- 172.21.15.31:0/2723220227 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33801a450 con 0x7fa3440cc340 2023-10-06T05:21:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.965+0000 7fa32b7fe700 1 -- 172.21.15.31:0/2723220227 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33801a7f0 con 0x7fa3440cc340 2023-10-06T05:21:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.965+0000 7fa349fff700 1 -- 172.21.15.31:0/2723220227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34416d920 con 0x7fa3440cc340 2023-10-06T05:21:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.965+0000 7fa349fff700 1 -- 172.21.15.31:0/2723220227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34416dee0 con 0x7fa3440cc340 2023-10-06T05:21:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.966+0000 7fa349fff700 1 -- 172.21.15.31:0/2723220227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34404c380 con 0x7fa3440cc340 2023-10-06T05:21:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.967+0000 7fa32b7fe700 1 -- 172.21.15.31:0/2723220227 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa33801ab60 con 0x7fa3440cc340 2023-10-06T05:21:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.968+0000 7fa32b7fe700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa330067290 0x7fa330069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.968+0000 7fa32b7fe700 1 -- 172.21.15.31:0/2723220227 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa338085e60 con 0x7fa3440cc340 2023-10-06T05:21:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.968+0000 7fa3437fe700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa330067290 0x7fa330069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.968+0000 7fa3437fe700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa330067290 0x7fa330069740 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa344152e10 tx=0x7fa334009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:10.972+0000 7fa32b7fe700 1 -- 172.21.15.31:0/2723220227 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa338052e70 con 0x7fa3440cc340 2023-10-06T05:21:11.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.173+0000 7fa349fff700 1 -- 172.21.15.31:0/2723220227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3440b2b40 con 0x7fa3440cc340 2023-10-06T05:21:11.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.174+0000 7fa32b7fe700 1 -- 172.21.15.31:0/2723220227 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3380200a0 con 0x7fa3440cc340 2023-10-06T05:21:11.176 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:21:11.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 -- 172.21.15.31:0/2723220227 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa330067290 msgr2=0x7fa330069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa330067290 0x7fa330069740 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa344152e10 tx=0x7fa334009040 comp rx=0 tx=0).stop 2023-10-06T05:21:11.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 -- 172.21.15.31:0/2723220227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440cc340 msgr2=0x7fa3440af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440cc340 0x7fa3440af1e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa3380090b0 tx=0x7fa338017000 comp rx=0 tx=0).stop 2023-10-06T05:21:11.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 -- 172.21.15.31:0/2723220227 shutdown_connections 2023-10-06T05:21:11.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa34415a4b0 0x7fa3440af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3440b2540 0x7fa3441569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fa330067290 0x7fa330069740 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 --2- 172.21.15.31:0/2723220227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3440cc340 0x7fa3440af1e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 -- 172.21.15.31:0/2723220227 >> 172.21.15.31:0/2723220227 conn(0x7fa344068670 msgr2=0x7fa344154870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:11.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 -- 172.21.15.31:0/2723220227 shutdown_connections 2023-10-06T05:21:11.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.176+0000 7fa349fff700 1 -- 172.21.15.31:0/2723220227 wait complete. 2023-10-06T05:21:11.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:21:11.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.389+0000 7fe512fff700 1 Processor -- start 2023-10-06T05:21:11.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.389+0000 7fe512fff700 1 -- start start 2023-10-06T05:21:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.391+0000 7fe512fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe50c06b510 0x7fe50c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.391+0000 7fe512fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe50c0c6df0 con 0x7fe50c0cab40 2023-10-06T05:21:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.391+0000 7fe512fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe50c0c70d0 con 0x7fe50c0ca770 2023-10-06T05:21:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.391+0000 7fe512fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50c0c7a70 con 0x7fe50c06b510 2023-10-06T05:21:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe51159c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe50c0cab40 0x7fe50c14c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41250/0 (socket says 172.21.15.31:41250) 2023-10-06T05:21:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe51159c700 1 -- 172.21.15.31:0/3808814837 learned_addr learned my addr 172.21.15.31:0/3808814837 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50bfff700 1 --2- 172.21.15.31:0/3808814837 >> v2:172.21.15.62:3300/0 conn(0x7fe50c06b510 0x7fe50c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818139344 0 0) 0x7fe50c0c6df0 con 0x7fe50c0cab40 2023-10-06T05:21:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4fc003570 con 0x7fe50c0cab40 2023-10-06T05:21:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611193228 0 0) 0x7fe50c0c70d0 con 0x7fe50c0ca770 2023-10-06T05:21:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe50c0c6df0 con 0x7fe50c0ca770 2023-10-06T05:21:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2004067305 0 0) 0x7fe4fc003570 con 0x7fe50c0cab40 2023-10-06T05:21:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 >> v2:172.21.15.62:3300/0 conn(0x7fe50c06b510 msgr2=0x7fe50c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 --2- 172.21.15.31:0/3808814837 >> v2:172.21.15.62:3300/0 conn(0x7fe50c06b510 0x7fe50c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 >> v1:172.21.15.62:6789/0 conn(0x7fe50c0ca770 legacy=0x7fe50c06ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0c73b0 con 0x7fe50c0cab40 2023-10-06T05:21:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fe5040017f0 con 0x7fe50c0cab40 2023-10-06T05:21:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 >> v1:172.21.15.31:6789/0 conn(0x7fe50c0cab40 legacy=0x7fe50c14c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4fc003bf0 0x7fe4fc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.392+0000 7fe50b7fe700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4fc004810 0x7fe4fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe50b7fe700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe4fc00dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c0c7a70 con 0x7fe4fc0051c0 2023-10-06T05:21:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc00e2c0 con 0x7fe4fc003bf0 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc00e480 con 0x7fe4fc004810 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe51159c700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe4fc00dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe50bfff700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4fc004810 0x7fe4fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe510d9b700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4fc003bf0 0x7fe4fc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe51159c700 1 -- 172.21.15.31:0/3808814837 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4fc004810 msgr2=0x7fe4fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe51159c700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4fc004810 0x7fe4fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe51159c700 1 -- 172.21.15.31:0/3808814837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4fc003bf0 msgr2=0x7fe4fc003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe51159c700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4fc003bf0 0x7fe4fc003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe51159c700 1 -- 172.21.15.31:0/3808814837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc003570 con 0x7fe4fc0051c0 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe51159c700 1 -- 172.21.15.31:0/3808814837 reap_dead start 2023-10-06T05:21:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.393+0000 7fe51159c700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe4fc00dc70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe50400a8f0 tx=0x7fe504002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a5aa4d200b64ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.394+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040070d0 con 0x7fe4fc0051c0 2023-10-06T05:21:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.394+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe504007250 con 0x7fe4fc0051c0 2023-10-06T05:21:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.394+0000 7fe50b7fe700 1 -- 172.21.15.31:0/3808814837 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504007610 con 0x7fe4fc0051c0 2023-10-06T05:21:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.394+0000 7fe512fff700 1 -- 172.21.15.31:0/3808814837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 msgr2=0x7fe4fc00dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.394+0000 7fe512fff700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe4fc00dc70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe50400a8f0 tx=0x7fe504002190 comp rx=0 tx=0).stop 2023-10-06T05:21:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.394+0000 7fe512fff700 1 -- 172.21.15.31:0/3808814837 shutdown_connections 2023-10-06T05:21:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.394+0000 7fe512fff700 1 --2- 172.21.15.31:0/3808814837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe4fc00dc70 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.394+0000 7fe512fff700 1 -- 172.21.15.31:0/3808814837 >> 172.21.15.31:0/3808814837 conn(0x7fe50c0cc0b0 msgr2=0x7fe50c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.394+0000 7fe512fff700 1 -- 172.21.15.31:0/3808814837 shutdown_connections 2023-10-06T05:21:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.394+0000 7fe512fff700 1 -- 172.21.15.31:0/3808814837 wait complete. 2023-10-06T05:21:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.396+0000 7fe512fff700 1 Processor -- start 2023-10-06T05:21:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.397+0000 7fe512fff700 1 -- start start 2023-10-06T05:21:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe512fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe50c0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe512fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe50c0aff90 0x7fe50c0ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe512fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe50c0b03e0 0x7fe50c0ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe512fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c0b1790 con 0x7fe4fc0051c0 2023-10-06T05:21:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe512fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c0c6b50 con 0x7fe50c0b03e0 2023-10-06T05:21:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe512fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c14e360 con 0x7fe50c0aff90 2023-10-06T05:21:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe510d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe50c0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe510d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe50c0c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60324/0 (socket says 172.21.15.31:60324) 2023-10-06T05:21:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe510d9b700 1 -- 172.21.15.31:0/3299726184 learned_addr learned my addr 172.21.15.31:0/3299726184 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe51159c700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe50c0b03e0 0x7fe50c0ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.398+0000 7fe50bfff700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe50c0aff90 0x7fe50c0ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe510d9b700 1 -- 172.21.15.31:0/3299726184 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe50c0aff90 msgr2=0x7fe50c0ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe510d9b700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe50c0aff90 0x7fe50c0ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe510d9b700 1 -- 172.21.15.31:0/3299726184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe50c0b03e0 msgr2=0x7fe50c0ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe510d9b700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe50c0b03e0 0x7fe50c0ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe510d9b700 1 -- 172.21.15.31:0/3299726184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0b08c0 con 0x7fe4fc0051c0 2023-10-06T05:21:11.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe51159c700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe50c0b03e0 0x7fe50c0ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:11.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe510d9b700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe50c0c6610 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe50000e7b0 tx=0x7fe500009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:11.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe508ff9700 1 -- 172.21.15.31:0/3299726184 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5000135a0 con 0x7fe4fc0051c0 2023-10-06T05:21:11.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe508ff9700 1 -- 172.21.15.31:0/3299726184 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe500010040 con 0x7fe4fc0051c0 2023-10-06T05:21:11.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe512fff700 1 -- 172.21.15.31:0/3299726184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50c0b0b80 con 0x7fe4fc0051c0 2023-10-06T05:21:11.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe508ff9700 1 -- 172.21.15.31:0/3299726184 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe500013950 con 0x7fe4fc0051c0 2023-10-06T05:21:11.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.399+0000 7fe512fff700 1 -- 172.21.15.31:0/3299726184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50c0b1140 con 0x7fe4fc0051c0 2023-10-06T05:21:11.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.400+0000 7fe512fff700 1 -- 172.21.15.31:0/3299726184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50c04c380 con 0x7fe4fc0051c0 2023-10-06T05:21:11.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.401+0000 7fe508ff9700 1 -- 172.21.15.31:0/3299726184 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe500013d40 con 0x7fe4fc0051c0 2023-10-06T05:21:11.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.402+0000 7fe508ff9700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4f0067290 0x7fe4f0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.402+0000 7fe508ff9700 1 -- 172.21.15.31:0/3299726184 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe500084a20 con 0x7fe4fc0051c0 2023-10-06T05:21:11.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.402+0000 7fe50bfff700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4f0067290 0x7fe4f0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.403+0000 7fe50bfff700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4f0067290 0x7fe4f0069740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800ad80 tx=0x7fe4f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:11.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.407+0000 7fe508ff9700 1 -- 172.21.15.31:0/3299726184 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe500051920 con 0x7fe4fc0051c0 2023-10-06T05:21:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.614+0000 7fe512fff700 1 -- 172.21.15.31:0/3299726184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe50c0c7690 con 0x7fe4fc0051c0 2023-10-06T05:21:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.615+0000 7fe508ff9700 1 -- 172.21.15.31:0/3299726184 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe500054f40 con 0x7fe4fc0051c0 2023-10-06T05:21:11.616 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:11.617 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:21:11.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:11.617 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:21:11.617 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:21:11.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:11.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:11.617 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:21:11.617 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:21:11.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:11.617 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:21:11.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 -- 172.21.15.31:0/3299726184 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4f0067290 msgr2=0x7fe4f0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4f0067290 0x7fe4f0069740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800ad80 tx=0x7fe4f8008040 comp rx=0 tx=0).stop 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 -- 172.21.15.31:0/3299726184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 msgr2=0x7fe50c0c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe50c0c6610 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe50000e7b0 tx=0x7fe500009040 comp rx=0 tx=0).stop 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 -- 172.21.15.31:0/3299726184 shutdown_connections 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe50c0b03e0 0x7fe50c0ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fe4f0067290 0x7fe4f0069740 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4fc0051c0 0x7fe50c0c6610 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 --2- 172.21.15.31:0/3299726184 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe50c0aff90 0x7fe50c0ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 -- 172.21.15.31:0/3299726184 >> 172.21.15.31:0/3299726184 conn(0x7fe50c0cc0b0 msgr2=0x7fe50c0c1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 -- 172.21.15.31:0/3299726184 shutdown_connections 2023-10-06T05:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.617+0000 7fe512fff700 1 -- 172.21.15.31:0/3299726184 wait complete. 2023-10-06T05:21:11.636 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:21:11.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:21:11.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.834+0000 7f0bb49ff700 1 Processor -- start 2023-10-06T05:21:11.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.834+0000 7f0bb49ff700 1 -- start start 2023-10-06T05:21:11.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.835+0000 7f0bb49ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb01550a0 0x7f0bb00ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.835+0000 7f0bb49ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0bb006ee10 0x7f0bb006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.835+0000 7f0bb49ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb006f640 con 0x7f0bb0154cd0 2023-10-06T05:21:11.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.835+0000 7f0bb49ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb006e610 con 0x7f0bb006ee10 2023-10-06T05:21:11.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.835+0000 7f0bb49ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00adf60 con 0x7f0bb01550a0 2023-10-06T05:21:11.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.835+0000 7f0bae59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0bb0154cd0 0x7f0bb01583e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41256/0 (socket says 172.21.15.31:41256) 2023-10-06T05:21:11.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.835+0000 7f0bae59c700 1 -- 172.21.15.31:0/2978950117 learned_addr learned my addr 172.21.15.31:0/2978950117 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0baed9d700 1 --2- 172.21.15.31:0/2978950117 >> v2:172.21.15.62:3300/0 conn(0x7f0bb006ee10 0x7f0bb006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0badd9b700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb01550a0 0x7f0bb00ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834725297 0 0) 0x7f0bb006f640 con 0x7f0bb0154cd0 2023-10-06T05:21:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b98003570 con 0x7f0bb0154cd0 2023-10-06T05:21:11.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2312036980 0 0) 0x7f0b98003570 con 0x7f0bb0154cd0 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb01550a0 msgr2=0x7f0bb00ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb01550a0 0x7f0bb00ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 >> v2:172.21.15.62:3300/0 conn(0x7f0bb006ee10 msgr2=0x7f0bb006d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 --2- 172.21.15.31:0/2978950117 >> v2:172.21.15.62:3300/0 conn(0x7f0bb006ee10 0x7f0bb006d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb006f360 con 0x7f0bb0154cd0 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f0ba00015f0 con 0x7f0bb0154cd0 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 >> v1:172.21.15.31:6789/0 conn(0x7f0bb0154cd0 legacy=0x7f0bb01583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b98003a60 0x7f0b98003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0b98004810 0x7f0b98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 0x7f0b9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb006e610 con 0x7f0b980051c0 2023-10-06T05:21:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00adf60 con 0x7f0b98003a60 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9800bfa0 con 0x7f0b98004810 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.836+0000 7f0badd9b700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb01550a0 0x7f0bb00ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0baed9d700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 0x7f0b9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0bae59c700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b98003a60 0x7f0b98003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0badd9b700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0b98004810 0x7f0b98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0baed9d700 1 -- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0b98004810 msgr2=0x7f0b98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0baed9d700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0b98004810 0x7f0b98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0baed9d700 1 -- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b98003a60 msgr2=0x7f0b98003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0baed9d700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b98003a60 0x7f0b98003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0baed9d700 1 -- 172.21.15.31:0/2978950117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b98003570 con 0x7f0b980051c0 2023-10-06T05:21:11.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0badd9b700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0b98004810 0x7f0b98004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0bae59c700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b98003a60 0x7f0b98003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0baed9d700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 0x7f0b9800ba60 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800e210 tx=0x7f0ba800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6529022d3da73f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800ea60 con 0x7f0b980051c0 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.837+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba800f040 con 0x7f0b980051c0 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.838+0000 7f0bad59a700 1 -- 172.21.15.31:0/2978950117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800edf0 con 0x7f0b980051c0 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.838+0000 7f0bb49ff700 1 -- 172.21.15.31:0/2978950117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 msgr2=0x7f0b9800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.838+0000 7f0bb49ff700 1 --2- 172.21.15.31:0/2978950117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 0x7f0b9800ba60 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800e210 tx=0x7f0ba800b040 comp rx=0 tx=0).stop 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.838+0000 7f0baed9d700 1 -- 172.21.15.31:0/2978950117 reap_dead start 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.838+0000 7f0bb49ff700 1 -- 172.21.15.31:0/2978950117 shutdown_connections 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.838+0000 7f0bb49ff700 1 -- 172.21.15.31:0/2978950117 >> 172.21.15.31:0/2978950117 conn(0x7f0bb0068030 msgr2=0x7f0bb00cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.838+0000 7f0bb49ff700 1 -- 172.21.15.31:0/2978950117 shutdown_connections 2023-10-06T05:21:11.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.838+0000 7f0bb49ff700 1 -- 172.21.15.31:0/2978950117 wait complete. 2023-10-06T05:21:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.840+0000 7f0bb49ff700 1 Processor -- start 2023-10-06T05:21:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.840+0000 7f0bb49ff700 1 -- start start 2023-10-06T05:21:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bb49ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 0x7f0bb01521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bb49ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb015add0 0x7f0bb015b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bb49ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb00c71e0 0x7f0bb00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bb49ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb015b780 con 0x7f0b980051c0 2023-10-06T05:21:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bb49ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00c3090 con 0x7f0bb00c71e0 2023-10-06T05:21:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bb49ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00c3650 con 0x7f0bb015add0 2023-10-06T05:21:11.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 0x7f0bb01521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 0x7f0bb01521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60340/0 (socket says 172.21.15.31:60340) 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bae59c700 1 -- 172.21.15.31:0/3488881599 learned_addr learned my addr 172.21.15.31:0/3488881599 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0badd9b700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb015add0 0x7f0bb015b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0baed9d700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb00c71e0 0x7f0bb00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bae59c700 1 -- 172.21.15.31:0/3488881599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb015add0 msgr2=0x7f0bb015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bae59c700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb015add0 0x7f0bb015b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bae59c700 1 -- 172.21.15.31:0/3488881599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb00c71e0 msgr2=0x7f0bb00c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bae59c700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb00c71e0 0x7f0bb00c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0bae59c700 1 -- 172.21.15.31:0/3488881599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00c7fb0 con 0x7f0b980051c0 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.841+0000 7f0baed9d700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb00c71e0 0x7f0bb00c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.842+0000 7f0bae59c700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 0x7f0bb01521d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0001380 tx=0x7f0ba0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:11.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.842+0000 7f0b9e7fc700 1 -- 172.21.15.31:0/3488881599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0001000 con 0x7f0b980051c0 2023-10-06T05:21:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.842+0000 7f0b9e7fc700 1 -- 172.21.15.31:0/3488881599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba0010040 con 0x7f0b980051c0 2023-10-06T05:21:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.842+0000 7f0bb49ff700 1 -- 172.21.15.31:0/3488881599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb00c8290 con 0x7f0b980051c0 2023-10-06T05:21:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.842+0000 7f0b9e7fc700 1 -- 172.21.15.31:0/3488881599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00074f0 con 0x7f0b980051c0 2023-10-06T05:21:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.842+0000 7f0bb49ff700 1 -- 172.21.15.31:0/3488881599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb00c7b90 con 0x7f0b980051c0 2023-10-06T05:21:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.843+0000 7f0bb49ff700 1 -- 172.21.15.31:0/3488881599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb00c7d50 con 0x7f0b980051c0 2023-10-06T05:21:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.844+0000 7f0b9e7fc700 1 -- 172.21.15.31:0/3488881599 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f0ba0005a00 con 0x7f0b980051c0 2023-10-06T05:21:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.845+0000 7f0b9e7fc700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0b90067280 0x7f0b90069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.845+0000 7f0b9e7fc700 1 -- 172.21.15.31:0/3488881599 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0ba0084540 con 0x7f0b980051c0 2023-10-06T05:21:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.845+0000 7f0badd9b700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0b90067280 0x7f0b90069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.846+0000 7f0badd9b700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0b90067280 0x7f0b90069730 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0bb016cd40 tx=0x7f0ba4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:11.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:11.849+0000 7f0b9e7fc700 1 -- 172.21.15.31:0/3488881599 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ba00515d0 con 0x7f0b980051c0 2023-10-06T05:21:12.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.052+0000 7f0bb49ff700 1 -- 172.21.15.31:0/3488881599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0bb00c8480 con 0x7f0b980051c0 2023-10-06T05:21:12.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.146+0000 7f0b9e7fc700 1 -- 172.21.15.31:0/3488881599 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0ba005a030 con 0x7f0b980051c0 2023-10-06T05:21:12.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.151+0000 7f0bb49ff700 1 -- 172.21.15.31:0/3488881599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0b90067280 msgr2=0x7f0b90069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:12.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.151+0000 7f0bb49ff700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0b90067280 0x7f0b90069730 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0bb016cd40 tx=0x7f0ba4008040 comp rx=0 tx=0).stop 2023-10-06T05:21:12.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.151+0000 7f0bb49ff700 1 -- 172.21.15.31:0/3488881599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 msgr2=0x7f0bb01521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:12.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.151+0000 7f0bb49ff700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 0x7f0bb01521d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0001380 tx=0x7f0ba0009040 comp rx=0 tx=0).stop 2023-10-06T05:21:12.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.152+0000 7f0bb49ff700 1 -- 172.21.15.31:0/3488881599 shutdown_connections 2023-10-06T05:21:12.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.152+0000 7f0bb49ff700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb00c71e0 0x7f0bb00c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.152+0000 7f0bb49ff700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0b90067280 0x7f0b90069730 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.152+0000 7f0bb49ff700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b980051c0 0x7f0bb01521d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.152+0000 7f0bb49ff700 1 --2- 172.21.15.31:0/3488881599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb015add0 0x7f0bb015b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.152+0000 7f0bb49ff700 1 -- 172.21.15.31:0/3488881599 >> 172.21.15.31:0/3488881599 conn(0x7f0bb0068030 msgr2=0x7f0bb00b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:12.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.152+0000 7f0bb49ff700 1 -- 172.21.15.31:0/3488881599 shutdown_connections 2023-10-06T05:21:12.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.152+0000 7f0bb49ff700 1 -- 172.21.15.31:0/3488881599 wait complete. 2023-10-06T05:21:12.155 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:21:12.173 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:21:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.375+0000 7f479adff700 1 Processor -- start 2023-10-06T05:21:12.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.375+0000 7f479adff700 1 -- start start 2023-10-06T05:21:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.376+0000 7f479adff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4794159560 0x7f47940cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.376+0000 7f479adff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47940b2b80 0x7f47940cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.376+0000 7f479adff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479415d2d0 con 0x7f4794159930 2023-10-06T05:21:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.376+0000 7f479adff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479415d5b0 con 0x7f4794159560 2023-10-06T05:21:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.376+0000 7f479adff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479415d890 con 0x7f47940b2b80 2023-10-06T05:21:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.376+0000 7f4798b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4794159560 0x7f47940cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.377+0000 7f4793fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4794159930 0x7f47940b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41264/0 (socket says 172.21.15.31:41264) 2023-10-06T05:21:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.377+0000 7f4793fff700 1 -- 172.21.15.31:0/3663407812 learned_addr learned my addr 172.21.15.31:0/3663407812 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.377+0000 7f479939c700 1 --2- 172.21.15.31:0/3663407812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47940b2b80 0x7f47940cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.377+0000 7f4798b9b700 1 -- 172.21.15.31:0/3663407812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47940b2b80 msgr2=0x7f47940cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:12.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.377+0000 7f4798b9b700 1 --2- 172.21.15.31:0/3663407812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47940b2b80 0x7f47940cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.377+0000 7f4798b9b700 1 -- 172.21.15.31:0/3663407812 >> v1:172.21.15.31:6789/0 conn(0x7f4794159930 legacy=0x7f47940b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.377+0000 7f4798b9b700 1 -- 172.21.15.31:0/3663407812 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479415db70 con 0x7f4794159560 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.377+0000 7f47937fe700 1 -- 172.21.15.31:0/3663407812 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208451799 0 0) 0x7f479415d2d0 con 0x7f4794159930 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.377+0000 7f479939c700 1 --2- 172.21.15.31:0/3663407812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47940b2b80 0x7f47940cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.377+0000 7f4798b9b700 1 --2- 172.21.15.31:0/3663407812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 0x7f47940cce70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f47880098e0 tx=0x7f47880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d56925507f9fdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f47937fe700 1 -- 172.21.15.31:0/3663407812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788009e80 con 0x7f4794159560 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f47937fe700 1 -- 172.21.15.31:0/3663407812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478800b860 con 0x7f4794159560 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f47937fe700 1 -- 172.21.15.31:0/3663407812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478800bbe0 con 0x7f4794159560 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f479adff700 1 -- 172.21.15.31:0/3663407812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 msgr2=0x7f47940cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f479adff700 1 --2- 172.21.15.31:0/3663407812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 0x7f47940cce70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f47880098e0 tx=0x7f47880091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f479adff700 1 -- 172.21.15.31:0/3663407812 shutdown_connections 2023-10-06T05:21:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f479adff700 1 --2- 172.21.15.31:0/3663407812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 0x7f47940cce70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f479adff700 1 --2- 172.21.15.31:0/3663407812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47940b2b80 0x7f47940cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f479adff700 1 -- 172.21.15.31:0/3663407812 >> 172.21.15.31:0/3663407812 conn(0x7f479406ec90 msgr2=0x7f479406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f479adff700 1 -- 172.21.15.31:0/3663407812 shutdown_connections 2023-10-06T05:21:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.378+0000 7f479adff700 1 -- 172.21.15.31:0/3663407812 wait complete. 2023-10-06T05:21:12.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.380+0000 7f479adff700 1 Processor -- start 2023-10-06T05:21:12.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.380+0000 7f479adff700 1 -- start start 2023-10-06T05:21:12.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.381+0000 7f479adff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47940b2b80 0x7f47941554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.381+0000 7f479adff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 0x7f47940cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.381+0000 7f479adff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4794159930 0x7f47940b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.381+0000 7f479adff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47941559e0 con 0x7f4794159560 2023-10-06T05:21:12.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.381+0000 7f479adff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4794169710 con 0x7f4794159930 2023-10-06T05:21:12.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.381+0000 7f479adff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479416a510 con 0x7f47940b2b80 2023-10-06T05:21:12.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.381+0000 7f4793fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 0x7f47940cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.381+0000 7f4793fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 0x7f47940cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60364/0 (socket says 172.21.15.31:60364) 2023-10-06T05:21:12.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.381+0000 7f4793fff700 1 -- 172.21.15.31:0/3549704102 learned_addr learned my addr 172.21.15.31:0/3549704102 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:12.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f4793fff700 1 -- 172.21.15.31:0/3549704102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47940b2b80 msgr2=0x7f47941554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:12.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f4793fff700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47940b2b80 0x7f47941554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f4793fff700 1 -- 172.21.15.31:0/3549704102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4794159930 msgr2=0x7f47940b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f479939c700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4794159930 0x7f47940b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f4793fff700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4794159930 0x7f47940b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f4793fff700 1 -- 172.21.15.31:0/3549704102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4788009420 con 0x7f4794159560 2023-10-06T05:21:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f479939c700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4794159930 0x7f47940b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f4793fff700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 0x7f47940cb9f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f478000c5e0 tx=0x7f4780001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f4790ff9700 1 -- 172.21.15.31:0/3549704102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780005b30 con 0x7f4794159560 2023-10-06T05:21:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f4790ff9700 1 -- 172.21.15.31:0/3549704102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4780005c90 con 0x7f4794159560 2023-10-06T05:21:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f4790ff9700 1 -- 172.21.15.31:0/3549704102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47800072e0 con 0x7f4794159560 2023-10-06T05:21:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.382+0000 7f479adff700 1 -- 172.21.15.31:0/3549704102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4794169900 con 0x7f4794159560 2023-10-06T05:21:12.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.383+0000 7f479adff700 1 -- 172.21.15.31:0/3549704102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4794169ec0 con 0x7f4794159560 2023-10-06T05:21:12.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.383+0000 7f479adff700 1 -- 172.21.15.31:0/3549704102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479404c380 con 0x7f4794159560 2023-10-06T05:21:12.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.385+0000 7f4790ff9700 1 -- 172.21.15.31:0/3549704102 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f478001b3b0 con 0x7f4794159560 2023-10-06T05:21:12.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.385+0000 7f4790ff9700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4778067290 0x7f4778069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:12.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.385+0000 7f4790ff9700 1 -- 172.21.15.31:0/3549704102 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f4780083f60 con 0x7f4794159560 2023-10-06T05:21:12.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.386+0000 7f4798b9b700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4778067290 0x7f4778069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:12.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.386+0000 7f4798b9b700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4778067290 0x7f4778069740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f47880091e0 tx=0x7f4788017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:12.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.390+0000 7f4790ff9700 1 -- 172.21.15.31:0/3549704102 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f478004b5a0 con 0x7f4794159560 2023-10-06T05:21:12.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:12.598+0000 7f479adff700 1 -- 172.21.15.31:0/3549704102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f479416a650 con 0x7f4794159560 2023-10-06T05:21:13.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.152+0000 7f4790ff9700 1 -- 172.21.15.31:0/3549704102 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4780017020 con 0x7f4794159560 2023-10-06T05:21:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.159+0000 7f479adff700 1 -- 172.21.15.31:0/3549704102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4778067290 msgr2=0x7f4778069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.159+0000 7f479adff700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4778067290 0x7f4778069740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f47880091e0 tx=0x7f4788017000 comp rx=0 tx=0).stop 2023-10-06T05:21:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.159+0000 7f479adff700 1 -- 172.21.15.31:0/3549704102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 msgr2=0x7f47940cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.159+0000 7f479adff700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 0x7f47940cb9f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f478000c5e0 tx=0x7f4780001070 comp rx=0 tx=0).stop 2023-10-06T05:21:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.161+0000 7f479adff700 1 -- 172.21.15.31:0/3549704102 shutdown_connections 2023-10-06T05:21:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.161+0000 7f479adff700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4794159930 0x7f47940b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.161+0000 7f479adff700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47940b2b80 0x7f47941554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.161+0000 7f479adff700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4778067290 0x7f4778069740 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.161+0000 7f479adff700 1 --2- 172.21.15.31:0/3549704102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4794159560 0x7f47940cb9f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.161+0000 7f479adff700 1 -- 172.21.15.31:0/3549704102 >> 172.21.15.31:0/3549704102 conn(0x7f479406ec90 msgr2=0x7f47941562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.162+0000 7f479adff700 1 -- 172.21.15.31:0/3549704102 shutdown_connections 2023-10-06T05:21:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.162+0000 7f479adff700 1 -- 172.21.15.31:0/3549704102 wait complete. 2023-10-06T05:21:13.168 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:21:13.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:21:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.391+0000 7f14977ff700 1 Processor -- start 2023-10-06T05:21:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.391+0000 7f14977ff700 1 -- start start 2023-10-06T05:21:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.392+0000 7f14977ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1490153d40 0x7f14900b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.392+0000 7f14977ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900b09c0 0x7f14900af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.392+0000 7f14977ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900ad690 con 0x7f14900af6c0 2023-10-06T05:21:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.392+0000 7f14977ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14900b0e10 con 0x7f1490153d40 2023-10-06T05:21:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.392+0000 7f14977ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14900acd60 con 0x7f14900b09c0 2023-10-06T05:21:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.393+0000 7f149559b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1490153d40 0x7f14900b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.393+0000 7f149559b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1490153d40 0x7f14900b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60378/0 (socket says 172.21.15.31:60378) 2023-10-06T05:21:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.393+0000 7f149559b700 1 -- 172.21.15.31:0/1954906701 learned_addr learned my addr 172.21.15.31:0/1954906701 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.393+0000 7f1494d9a700 1 --2- 172.21.15.31:0/1954906701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900b09c0 0x7f14900af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.393+0000 7f149559b700 1 -- 172.21.15.31:0/1954906701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900b09c0 msgr2=0x7f14900af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:13.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.393+0000 7f149559b700 1 --2- 172.21.15.31:0/1954906701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900b09c0 0x7f14900af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.393+0000 7f149559b700 1 -- 172.21.15.31:0/1954906701 >> v1:172.21.15.62:6789/0 conn(0x7f14900af6c0 legacy=0x7f1490159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.393+0000 7f149559b700 1 -- 172.21.15.31:0/1954906701 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14900ad040 con 0x7f1490153d40 2023-10-06T05:21:13.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.393+0000 7f149559b700 1 --2- 172.21.15.31:0/1954906701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 0x7f14900b0480 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f148c0099b0 tx=0x7f148c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617dcb890d859cd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:13.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.394+0000 7f1487fff700 1 -- 172.21.15.31:0/1954906701 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c009ea0 con 0x7f1490153d40 2023-10-06T05:21:13.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.394+0000 7f1487fff700 1 -- 172.21.15.31:0/1954906701 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c00b860 con 0x7f1490153d40 2023-10-06T05:21:13.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.394+0000 7f1487fff700 1 -- 172.21.15.31:0/1954906701 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c00bc00 con 0x7f1490153d40 2023-10-06T05:21:13.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.394+0000 7f14977ff700 1 -- 172.21.15.31:0/1954906701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 msgr2=0x7f14900b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.394+0000 7f14977ff700 1 --2- 172.21.15.31:0/1954906701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 0x7f14900b0480 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f148c0099b0 tx=0x7f148c009250 comp rx=0 tx=0).stop 2023-10-06T05:21:13.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.394+0000 7f14977ff700 1 -- 172.21.15.31:0/1954906701 shutdown_connections 2023-10-06T05:21:13.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.394+0000 7f14977ff700 1 --2- 172.21.15.31:0/1954906701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 0x7f14900b0480 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.394+0000 7f14977ff700 1 --2- 172.21.15.31:0/1954906701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900b09c0 0x7f14900af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.394+0000 7f14977ff700 1 -- 172.21.15.31:0/1954906701 >> 172.21.15.31:0/1954906701 conn(0x7f1490067560 msgr2=0x7f1490067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:13.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.394+0000 7f14977ff700 1 -- 172.21.15.31:0/1954906701 shutdown_connections 2023-10-06T05:21:13.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.395+0000 7f14977ff700 1 -- 172.21.15.31:0/1954906701 wait complete. 2023-10-06T05:21:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.397+0000 7f14977ff700 1 Processor -- start 2023-10-06T05:21:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.397+0000 7f14977ff700 1 -- start start 2023-10-06T05:21:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.398+0000 7f14977ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900af6c0 0x7f1490146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.398+0000 7f14977ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14900b09c0 0x7f1490147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.398+0000 7f14977ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 0x7f14901454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.398+0000 7f14977ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1490145050 con 0x7f1490153d40 2023-10-06T05:21:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.398+0000 7f14977ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14901444b0 con 0x7f14900b09c0 2023-10-06T05:21:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.398+0000 7f14977ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1490142bc0 con 0x7f14900af6c0 2023-10-06T05:21:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.398+0000 7f1495d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 0x7f14901454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.398+0000 7f1495d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 0x7f14901454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60390/0 (socket says 172.21.15.31:60390) 2023-10-06T05:21:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.398+0000 7f1495d9c700 1 -- 172.21.15.31:0/197267801 learned_addr learned my addr 172.21.15.31:0/197267801 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f1494d9a700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14900b09c0 0x7f1490147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f149559b700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900af6c0 0x7f1490146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f1495d9c700 1 -- 172.21.15.31:0/197267801 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900af6c0 msgr2=0x7f1490146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f1495d9c700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900af6c0 0x7f1490146fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f1495d9c700 1 -- 172.21.15.31:0/197267801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14900b09c0 msgr2=0x7f1490147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f1495d9c700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14900b09c0 0x7f1490147fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f1495d9c700 1 -- 172.21.15.31:0/197267801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c0094f0 con 0x7f1490153d40 2023-10-06T05:21:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f1494d9a700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14900b09c0 0x7f1490147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f149559b700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900af6c0 0x7f1490146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f1495d9c700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 0x7f14901454e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f148800c8e0 tx=0x7f1488001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f14857fa700 1 -- 172.21.15.31:0/197267801 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148800ce80 con 0x7f1490153d40 2023-10-06T05:21:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f14857fa700 1 -- 172.21.15.31:0/197267801 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148800a7e0 con 0x7f1490153d40 2023-10-06T05:21:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f14857fa700 1 -- 172.21.15.31:0/197267801 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148800ab80 con 0x7f1490153d40 2023-10-06T05:21:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f14977ff700 1 -- 172.21.15.31:0/197267801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1490144720 con 0x7f1490153d40 2023-10-06T05:21:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.399+0000 7f14977ff700 1 -- 172.21.15.31:0/197267801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1490144ce0 con 0x7f1490153d40 2023-10-06T05:21:13.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.400+0000 7f14977ff700 1 -- 172.21.15.31:0/197267801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149004c380 con 0x7f1490153d40 2023-10-06T05:21:13.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.402+0000 7f14857fa700 1 -- 172.21.15.31:0/197267801 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f1488012420 con 0x7f1490153d40 2023-10-06T05:21:13.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.403+0000 7f14857fa700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1474067290 0x7f1474069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:13.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.403+0000 7f14857fa700 1 -- 172.21.15.31:0/197267801 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f1488084f60 con 0x7f1490153d40 2023-10-06T05:21:13.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.403+0000 7f149559b700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1474067290 0x7f1474069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:13.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.403+0000 7f149559b700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1474067290 0x7f1474069740 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f148c0099b0 tx=0x7f148c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:13.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.407+0000 7f14857fa700 1 -- 172.21.15.31:0/197267801 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1488051b80 con 0x7f1490153d40 2023-10-06T05:21:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:13.659+0000 7f14977ff700 1 -- 172.21.15.31:0/197267801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1490145190 con 0x7f1490153d40 2023-10-06T05:21:14.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.165+0000 7f14857fa700 1 -- 172.21.15.31:0/197267801 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7f14880551a0 con 0x7f1490153d40 2023-10-06T05:21:14.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.174+0000 7f14977ff700 1 -- 172.21.15.31:0/197267801 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1474067290 msgr2=0x7f1474069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.174+0000 7f14977ff700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1474067290 0x7f1474069740 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f148c0099b0 tx=0x7f148c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:14.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.174+0000 7f14977ff700 1 -- 172.21.15.31:0/197267801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 msgr2=0x7f14901454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.174+0000 7f14977ff700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 0x7f14901454e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f148800c8e0 tx=0x7f1488001710 comp rx=0 tx=0).stop 2023-10-06T05:21:14.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.180+0000 7f14977ff700 1 -- 172.21.15.31:0/197267801 shutdown_connections 2023-10-06T05:21:14.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.180+0000 7f14977ff700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1474067290 0x7f1474069740 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.180+0000 7f14977ff700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1490153d40 0x7f14901454e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.180+0000 7f14977ff700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14900af6c0 0x7f1490146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.180+0000 7f14977ff700 1 --2- 172.21.15.31:0/197267801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14900b09c0 0x7f1490147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.180+0000 7f14977ff700 1 -- 172.21.15.31:0/197267801 >> 172.21.15.31:0/197267801 conn(0x7f1490067560 msgr2=0x7f149014cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:14.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.182+0000 7f14977ff700 1 -- 172.21.15.31:0/197267801 shutdown_connections 2023-10-06T05:21:14.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.182+0000 7f14977ff700 1 -- 172.21.15.31:0/197267801 wait complete. 2023-10-06T05:21:14.189 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 11 and data pool 12 2023-10-06T05:21:14.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:14.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.404+0000 7f64d95ff700 1 Processor -- start 2023-10-06T05:21:14.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.404+0000 7f64d95ff700 1 -- start start 2023-10-06T05:21:14.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.405+0000 7f64d95ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f64d40cd510 0x7f64d40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.405+0000 7f64d95ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d415d2d0 0x7f64d40aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.405+0000 7f64d95ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d40c3f40 con 0x7f64d4159500 2023-10-06T05:21:14.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.405+0000 7f64d95ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d40b2860 con 0x7f64d40cd510 2023-10-06T05:21:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.405+0000 7f64d95ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d415d7d0 con 0x7f64d415d2d0 2023-10-06T05:21:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d27fc700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f64d40cd510 0x7f64d40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d37fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d415d2d0 0x7f64d40aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d2ffd700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f64d4159500 0x7f64d40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:52228/0 (socket says 172.21.15.31:52228) 2023-10-06T05:21:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d2ffd700 1 -- 172.21.15.31:0/3615774074 learned_addr learned my addr 172.21.15.31:0/3615774074 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d1ffb700 1 -- 172.21.15.31:0/3615774074 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799923791 0 0) 0x7f64d40c3f40 con 0x7f64d4159500 2023-10-06T05:21:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d1ffb700 1 -- 172.21.15.31:0/3615774074 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64bc003570 con 0x7f64d4159500 2023-10-06T05:21:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d27fc700 1 -- 172.21.15.31:0/3615774074 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d415d2d0 msgr2=0x7f64d40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d27fc700 1 --2- 172.21.15.31:0/3615774074 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d415d2d0 0x7f64d40aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d27fc700 1 -- 172.21.15.31:0/3615774074 >> v1:172.21.15.62:6789/0 conn(0x7f64d4159500 legacy=0x7f64d40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d27fc700 1 -- 172.21.15.31:0/3615774074 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d40cb000 con 0x7f64d40cd510 2023-10-06T05:21:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.406+0000 7f64d37fe700 1 --2- 172.21.15.31:0/3615774074 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d415d2d0 0x7f64d40aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.407+0000 7f64d27fc700 1 --2- 172.21.15.31:0/3615774074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d40cd510 0x7f64d40b0ab0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098d0 tx=0x7f64c8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71c370b8e52efd6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.407+0000 7f64d1ffb700 1 -- 172.21.15.31:0/3615774074 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8005c10 con 0x7f64d40cd510 2023-10-06T05:21:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.407+0000 7f64d1ffb700 1 -- 172.21.15.31:0/3615774074 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8005d70 con 0x7f64d40cd510 2023-10-06T05:21:14.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.407+0000 7f64d1ffb700 1 -- 172.21.15.31:0/3615774074 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8006280 con 0x7f64d40cd510 2023-10-06T05:21:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.408+0000 7f64d95ff700 1 -- 172.21.15.31:0/3615774074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d40cd510 msgr2=0x7f64d40b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.408+0000 7f64d95ff700 1 --2- 172.21.15.31:0/3615774074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d40cd510 0x7f64d40b0ab0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098d0 tx=0x7f64c8009120 comp rx=0 tx=0).stop 2023-10-06T05:21:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.408+0000 7f64d95ff700 1 -- 172.21.15.31:0/3615774074 shutdown_connections 2023-10-06T05:21:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.408+0000 7f64d95ff700 1 --2- 172.21.15.31:0/3615774074 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d415d2d0 0x7f64d40aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.408+0000 7f64d95ff700 1 --2- 172.21.15.31:0/3615774074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d40cd510 0x7f64d40b0ab0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.408+0000 7f64d95ff700 1 -- 172.21.15.31:0/3615774074 >> 172.21.15.31:0/3615774074 conn(0x7f64d406ec90 msgr2=0x7f64d406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.408+0000 7f64d95ff700 1 -- 172.21.15.31:0/3615774074 shutdown_connections 2023-10-06T05:21:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.408+0000 7f64d95ff700 1 -- 172.21.15.31:0/3615774074 wait complete. 2023-10-06T05:21:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.410+0000 7f64d95ff700 1 Processor -- start 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.410+0000 7f64d95ff700 1 -- start start 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.411+0000 7f64d95ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d40cd510 0x7f64d40c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d95ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d4159500 0x7f64d41598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d95ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d415d2d0 0x7f64d40c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d95ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4149de0 con 0x7f64d415d2d0 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d95ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40b3290 con 0x7f64d40cd510 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d95ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40b3570 con 0x7f64d4159500 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d40cd510 0x7f64d40c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d415d2d0 0x7f64d40c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d40cd510 0x7f64d40c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:55640/0 (socket says 172.21.15.31:55640) 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d2ffd700 1 -- 172.21.15.31:0/1841594734 learned_addr learned my addr 172.21.15.31:0/1841594734 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d27fc700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d4159500 0x7f64d41598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d37fe700 1 -- 172.21.15.31:0/1841594734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d4159500 msgr2=0x7f64d41598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d37fe700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d4159500 0x7f64d41598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d37fe700 1 -- 172.21.15.31:0/1841594734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d40cd510 msgr2=0x7f64d40c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d37fe700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d40cd510 0x7f64d40c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.412+0000 7f64d37fe700 1 -- 172.21.15.31:0/1841594734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc009a20 con 0x7f64d415d2d0 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.413+0000 7f64d37fe700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d415d2d0 0x7f64d40c74d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009e70 tx=0x7f64cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.413+0000 7f64baffd700 1 -- 172.21.15.31:0/1841594734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00caa0 con 0x7f64d415d2d0 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.413+0000 7f64baffd700 1 -- 172.21.15.31:0/1841594734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc00f040 con 0x7f64d415d2d0 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.413+0000 7f64d95ff700 1 -- 172.21.15.31:0/1841594734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c80093c0 con 0x7f64d415d2d0 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.413+0000 7f64baffd700 1 -- 172.21.15.31:0/1841594734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00ce90 con 0x7f64d415d2d0 2023-10-06T05:21:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.413+0000 7f64d95ff700 1 -- 172.21.15.31:0/1841594734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d40b3b30 con 0x7f64d415d2d0 2023-10-06T05:21:14.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.414+0000 7f64d95ff700 1 -- 172.21.15.31:0/1841594734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d404c380 con 0x7f64d415d2d0 2023-10-06T05:21:14.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.415+0000 7f64baffd700 1 -- 172.21.15.31:0/1841594734 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f64cc003020 con 0x7f64d415d2d0 2023-10-06T05:21:14.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.416+0000 7f64baffd700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f64c0067290 0x7f64c0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.416+0000 7f64baffd700 1 -- 172.21.15.31:0/1841594734 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f64cc0854f0 con 0x7f64d415d2d0 2023-10-06T05:21:14.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.416+0000 7f64d2ffd700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f64c0067290 0x7f64c0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.417+0000 7f64d2ffd700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f64c0067290 0x7f64c0069740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f64c4001ad0 tx=0x7f64c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:14.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.420+0000 7f64baffd700 1 -- 172.21.15.31:0/1841594734 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64cc0520a0 con 0x7f64d415d2d0 2023-10-06T05:21:14.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.624+0000 7f64d95ff700 1 -- 172.21.15.31:0/1841594734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64d415e170 con 0x7f64d415d2d0 2023-10-06T05:21:14.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.625+0000 7f64baffd700 1 -- 172.21.15.31:0/1841594734 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16930 (secure 0 0 0) 0x7f64cc0556c0 con 0x7f64d415d2d0 2023-10-06T05:21:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:14.628 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":70,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:13.661002+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:12.092711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:12.637981+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 -- 172.21.15.31:0/1841594734 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f64c0067290 msgr2=0x7f64c0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f64c0067290 0x7f64c0069740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f64c4001ad0 tx=0x7f64c4009040 comp rx=0 tx=0).stop 2023-10-06T05:21:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 -- 172.21.15.31:0/1841594734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d415d2d0 msgr2=0x7f64d40c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d415d2d0 0x7f64d40c74d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009e70 tx=0x7f64cc00b040 comp rx=0 tx=0).stop 2023-10-06T05:21:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 -- 172.21.15.31:0/1841594734 shutdown_connections 2023-10-06T05:21:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f64c0067290 0x7f64c0069740 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d415d2d0 0x7f64d40c74d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d40cd510 0x7f64d40c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 --2- 172.21.15.31:0/1841594734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d4159500 0x7f64d41598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 -- 172.21.15.31:0/1841594734 >> 172.21.15.31:0/1841594734 conn(0x7f64d406ec90 msgr2=0x7f64d4156180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:14.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 -- 172.21.15.31:0/1841594734 shutdown_connections 2023-10-06T05:21:14.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.628+0000 7f64d95ff700 1 -- 172.21.15.31:0/1841594734 wait complete. 2023-10-06T05:21:14.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:14.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.841+0000 7f6d21fff700 1 Processor -- start 2023-10-06T05:21:14.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.841+0000 7f6d21fff700 1 -- start start 2023-10-06T05:21:14.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.842+0000 7f6d21fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d1c159500 0x7f6d1c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.842+0000 7f6d21fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d1c15d2d0 0x7f6d1c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.842+0000 7f6d21fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6d1c1598d0 0x7f6d1c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.842+0000 7f6d21fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c3f40 con 0x7f6d1c15d2d0 2023-10-06T05:21:14.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.842+0000 7f6d21fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c0cb000 con 0x7f6d1c1598d0 2023-10-06T05:21:14.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.842+0000 7f6d21fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b2860 con 0x7f6d1c159500 2023-10-06T05:21:14.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d1c15d2d0 0x7f6d1c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d1c15d2d0 0x7f6d1c0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60408/0 (socket says 172.21.15.31:60408) 2023-10-06T05:21:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1affd700 1 -- 172.21.15.31:0/74455392 learned_addr learned my addr 172.21.15.31:0/74455392 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1b7fe700 1 --2- 172.21.15.31:0/74455392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d1c159500 0x7f6d1c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1bfff700 1 --2- 172.21.15.31:0/74455392 >> v2:172.21.15.62:3300/0 conn(0x7f6d1c1598d0 0x7f6d1c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1affd700 1 -- 172.21.15.31:0/74455392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d1c159500 msgr2=0x7f6d1c0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:14.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1affd700 1 --2- 172.21.15.31:0/74455392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d1c159500 0x7f6d1c0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1affd700 1 -- 172.21.15.31:0/74455392 >> v2:172.21.15.62:3300/0 conn(0x7f6d1c1598d0 msgr2=0x7f6d1c0cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1affd700 1 --2- 172.21.15.31:0/74455392 >> v2:172.21.15.62:3300/0 conn(0x7f6d1c1598d0 0x7f6d1c0cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1affd700 1 -- 172.21.15.31:0/74455392 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c15d7d0 con 0x7f6d1c15d2d0 2023-10-06T05:21:14.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.843+0000 7f6d1affd700 1 --2- 172.21.15.31:0/74455392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c15d2d0 0x7f6d1c0cce00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0c7440 tx=0x7f6d10009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf8b7941c0dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d1a7fc700 1 -- 172.21.15.31:0/74455392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10005c10 con 0x7f6d1c15d2d0 2023-10-06T05:21:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d1a7fc700 1 -- 172.21.15.31:0/74455392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d10005d70 con 0x7f6d1c15d2d0 2023-10-06T05:21:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d1a7fc700 1 -- 172.21.15.31:0/74455392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10006260 con 0x7f6d1c15d2d0 2023-10-06T05:21:14.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d21fff700 1 -- 172.21.15.31:0/74455392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c15d2d0 msgr2=0x7f6d1c0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d21fff700 1 --2- 172.21.15.31:0/74455392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c15d2d0 0x7f6d1c0cce00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0c7440 tx=0x7f6d10009220 comp rx=0 tx=0).stop 2023-10-06T05:21:14.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d21fff700 1 -- 172.21.15.31:0/74455392 shutdown_connections 2023-10-06T05:21:14.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d21fff700 1 --2- 172.21.15.31:0/74455392 >> v2:172.21.15.62:3300/0 conn(0x7f6d1c1598d0 0x7f6d1c0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d21fff700 1 --2- 172.21.15.31:0/74455392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d1c159500 0x7f6d1c0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d21fff700 1 --2- 172.21.15.31:0/74455392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c15d2d0 0x7f6d1c0cce00 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d21fff700 1 -- 172.21.15.31:0/74455392 >> 172.21.15.31:0/74455392 conn(0x7f6d1c06ec90 msgr2=0x7f6d1c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.844+0000 7f6d21fff700 1 -- 172.21.15.31:0/74455392 shutdown_connections 2023-10-06T05:21:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.845+0000 7f6d21fff700 1 -- 172.21.15.31:0/74455392 wait complete. 2023-10-06T05:21:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.846+0000 7f6d21fff700 1 Processor -- start 2023-10-06T05:21:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.846+0000 7f6d21fff700 1 -- start start 2023-10-06T05:21:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.847+0000 7f6d21fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d1c159500 0x7f6d1c0b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.847+0000 7f6d21fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c1598d0 0x7f6d1c0b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.847+0000 7f6d21fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c15d2d0 0x7f6d1c0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.847+0000 7f6d21fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b3290 con 0x7f6d1c1598d0 2023-10-06T05:21:14.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.847+0000 7f6d21fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c15cc00 con 0x7f6d1c15d2d0 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.847+0000 7f6d21fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c15ac20 con 0x7f6d1c159500 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.847+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c1598d0 0x7f6d1c0b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.847+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c1598d0 0x7f6d1c0b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60424/0 (socket says 172.21.15.31:60424) 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.847+0000 7f6d1affd700 1 -- 172.21.15.31:0/1944040234 learned_addr learned my addr 172.21.15.31:0/1944040234 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.848+0000 7f6d1b7fe700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d1c159500 0x7f6d1c0b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.848+0000 7f6d1bfff700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c15d2d0 0x7f6d1c0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.848+0000 7f6d1affd700 1 -- 172.21.15.31:0/1944040234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d1c159500 msgr2=0x7f6d1c0b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.848+0000 7f6d1affd700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d1c159500 0x7f6d1c0b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.848+0000 7f6d1affd700 1 -- 172.21.15.31:0/1944040234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c15d2d0 msgr2=0x7f6d1c0b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.848+0000 7f6d1affd700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c15d2d0 0x7f6d1c0b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:14.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.848+0000 7f6d1affd700 1 -- 172.21.15.31:0/1944040234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d10009480 con 0x7f6d1c1598d0 2023-10-06T05:21:14.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.848+0000 7f6d1affd700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c1598d0 0x7f6d1c0b57a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009f50 tx=0x7f6d10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:14.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.848+0000 7f6d037fe700 1 -- 172.21.15.31:0/1944040234 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10011850 con 0x7f6d1c1598d0 2023-10-06T05:21:14.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.849+0000 7f6d037fe700 1 -- 172.21.15.31:0/1944040234 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d100119b0 con 0x7f6d1c1598d0 2023-10-06T05:21:14.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.849+0000 7f6d037fe700 1 -- 172.21.15.31:0/1944040234 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10011d50 con 0x7f6d1c1598d0 2023-10-06T05:21:14.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.849+0000 7f6d21fff700 1 -- 172.21.15.31:0/1944040234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c15aec0 con 0x7f6d1c1598d0 2023-10-06T05:21:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.849+0000 7f6d21fff700 1 -- 172.21.15.31:0/1944040234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c15b480 con 0x7f6d1c1598d0 2023-10-06T05:21:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.850+0000 7f6d21fff700 1 -- 172.21.15.31:0/1944040234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c0b3440 con 0x7f6d1c1598d0 2023-10-06T05:21:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.850+0000 7f6d037fe700 1 -- 172.21.15.31:0/1944040234 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6d1001b640 con 0x7f6d1c1598d0 2023-10-06T05:21:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.852+0000 7f6d037fe700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d08067290 0x7f6d08069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.852+0000 7f6d037fe700 1 -- 172.21.15.31:0/1944040234 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6d100862d0 con 0x7f6d1c1598d0 2023-10-06T05:21:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.852+0000 7f6d1b7fe700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d08067290 0x7f6d08069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.853+0000 7f6d1b7fe700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d08067290 0x7f6d08069740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009db0 tx=0x7f6d0c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:14.856+0000 7f6d037fe700 1 -- 172.21.15.31:0/1944040234 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d1c0b3440 con 0x7f6d1c1598d0 2023-10-06T05:21:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.060+0000 7f6d21fff700 1 -- 172.21.15.31:0/1944040234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d1c04c380 con 0x7f6d1c1598d0 2023-10-06T05:21:15.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.061+0000 7f6d037fe700 1 -- 172.21.15.31:0/1944040234 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16930 (secure 0 0 0) 0x7f6d10052f90 con 0x7f6d1c1598d0 2023-10-06T05:21:15.063 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:15.063 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":70,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:13.661002+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:12.092711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:12.637981+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 -- 172.21.15.31:0/1944040234 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d08067290 msgr2=0x7f6d08069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d08067290 0x7f6d08069740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009db0 tx=0x7f6d0c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 -- 172.21.15.31:0/1944040234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c1598d0 msgr2=0x7f6d1c0b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c1598d0 0x7f6d1c0b57a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009f50 tx=0x7f6d10016000 comp rx=0 tx=0).stop 2023-10-06T05:21:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 -- 172.21.15.31:0/1944040234 shutdown_connections 2023-10-06T05:21:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d1c15d2d0 0x7f6d1c0b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d1c159500 0x7f6d1c0b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f6d08067290 0x7f6d08069740 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 --2- 172.21.15.31:0/1944040234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c1598d0 0x7f6d1c0b57a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 -- 172.21.15.31:0/1944040234 >> 172.21.15.31:0/1944040234 conn(0x7f6d1c06ec90 msgr2=0x7f6d1c0cb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:15.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 -- 172.21.15.31:0/1944040234 shutdown_connections 2023-10-06T05:21:15.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.063+0000 7f6d21fff700 1 -- 172.21.15.31:0/1944040234 wait complete. 2023-10-06T05:21:15.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:21:15.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.283+0000 7f28d2bff700 1 Processor -- start 2023-10-06T05:21:15.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.284+0000 7f28d2bff700 1 -- start start 2023-10-06T05:21:15.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.285+0000 7f28d2bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f28cc0c7190 0x7f28cc0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.285+0000 7f28d2bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0c7560 0x7f28cc0686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.285+0000 7f28d2bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28cc068250 con 0x7f28cc0aef00 2023-10-06T05:21:15.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.285+0000 7f28d2bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28cc06ac80 con 0x7f28cc0c7190 2023-10-06T05:21:15.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.285+0000 7f28d2bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc067f20 con 0x7f28cc0c7560 2023-10-06T05:21:15.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.285+0000 7f28d099b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f28cc0c7190 0x7f28cc0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.285+0000 7f28d119c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0c7560 0x7f28cc0686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.285+0000 7f28cbfff700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f28cc0aef00 0x7f28cc0af7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:52232/0 (socket says 172.21.15.31:52232) 2023-10-06T05:21:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.285+0000 7f28cbfff700 1 -- 172.21.15.31:0/2601710720 learned_addr learned my addr 172.21.15.31:0/2601710720 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.286+0000 7f28cb7fe700 1 -- 172.21.15.31:0/2601710720 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111434222 0 0) 0x7f28cc068250 con 0x7f28cc0aef00 2023-10-06T05:21:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.286+0000 7f28cb7fe700 1 -- 172.21.15.31:0/2601710720 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28bc003570 con 0x7f28cc0aef00 2023-10-06T05:21:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.286+0000 7f28d119c700 1 -- 172.21.15.31:0/2601710720 >> v2:172.21.15.62:3300/0 conn(0x7f28cc0c7190 msgr2=0x7f28cc0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.286+0000 7f28d119c700 1 --2- 172.21.15.31:0/2601710720 >> v2:172.21.15.62:3300/0 conn(0x7f28cc0c7190 0x7f28cc0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.286+0000 7f28d119c700 1 -- 172.21.15.31:0/2601710720 >> v1:172.21.15.62:6789/0 conn(0x7f28cc0aef00 legacy=0x7f28cc0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.286+0000 7f28d119c700 1 -- 172.21.15.31:0/2601710720 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc06aed0 con 0x7f28cc0c7560 2023-10-06T05:21:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.286+0000 7f28d099b700 1 --2- 172.21.15.31:0/2601710720 >> v2:172.21.15.62:3300/0 conn(0x7f28cc0c7190 0x7f28cc0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:15.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.286+0000 7f28d119c700 1 --2- 172.21.15.31:0/2601710720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0c7560 0x7f28cc0686e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009e60 tx=0x7f28c4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9c3305b1465801 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28cb7fe700 1 -- 172.21.15.31:0/2601710720 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400e070 con 0x7f28cc0c7560 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28cb7fe700 1 -- 172.21.15.31:0/2601710720 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c400bab0 con 0x7f28cc0c7560 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28cb7fe700 1 -- 172.21.15.31:0/2601710720 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400be70 con 0x7f28cc0c7560 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28d2bff700 1 -- 172.21.15.31:0/2601710720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0c7560 msgr2=0x7f28cc0686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28d2bff700 1 --2- 172.21.15.31:0/2601710720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0c7560 0x7f28cc0686e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009e60 tx=0x7f28c4009780 comp rx=0 tx=0).stop 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28d2bff700 1 -- 172.21.15.31:0/2601710720 shutdown_connections 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28d2bff700 1 --2- 172.21.15.31:0/2601710720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0c7560 0x7f28cc0686e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28d2bff700 1 --2- 172.21.15.31:0/2601710720 >> v2:172.21.15.62:3300/0 conn(0x7f28cc0c7190 0x7f28cc0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28d2bff700 1 -- 172.21.15.31:0/2601710720 >> 172.21.15.31:0/2601710720 conn(0x7f28cc0cd3d0 msgr2=0x7f28cc0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28d2bff700 1 -- 172.21.15.31:0/2601710720 shutdown_connections 2023-10-06T05:21:15.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.287+0000 7f28d2bff700 1 -- 172.21.15.31:0/2601710720 wait complete. 2023-10-06T05:21:15.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.289+0000 7f28d2bff700 1 Processor -- start 2023-10-06T05:21:15.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.289+0000 7f28d2bff700 1 -- start start 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28d2bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0aef00 0x7f28cc15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28d2bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28cc0c7190 0x7f28cc14a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28d2bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28cc0c7560 0x7f28cc0c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28d2bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0c4a10 con 0x7f28cc0c7560 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28d2bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0c0920 con 0x7f28cc0c7190 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28d2bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc15bfd0 con 0x7f28cc0aef00 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28d099b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0aef00 0x7f28cc15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28d119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28cc0c7560 0x7f28cc0c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28cbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28cc0c7190 0x7f28cc14a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28d099b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0aef00 0x7f28cc15b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:38228/0 (socket says 172.21.15.31:38228) 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.290+0000 7f28d099b700 1 -- 172.21.15.31:0/1663207285 learned_addr learned my addr 172.21.15.31:0/1663207285 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:15.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28d099b700 1 -- 172.21.15.31:0/1663207285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28cc0c7190 msgr2=0x7f28cc14a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28d099b700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28cc0c7190 0x7f28cc14a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28d099b700 1 -- 172.21.15.31:0/1663207285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28cc0c7560 msgr2=0x7f28cc0c03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28d099b700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28cc0c7560 0x7f28cc0c03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28d099b700 1 -- 172.21.15.31:0/1663207285 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c0009420 con 0x7f28cc0aef00 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28d119c700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28cc0c7560 0x7f28cc0c03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28d099b700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0aef00 0x7f28cc15b990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f28c000ec80 tx=0x7f28c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28c8ff9700 1 -- 172.21.15.31:0/1663207285 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c000c920 con 0x7f28cc0aef00 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28c8ff9700 1 -- 172.21.15.31:0/1663207285 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c000ca80 con 0x7f28cc0aef00 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28c8ff9700 1 -- 172.21.15.31:0/1663207285 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c000ce00 con 0x7f28cc0aef00 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28d2bff700 1 -- 172.21.15.31:0/1663207285 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c4009a20 con 0x7f28cc0aef00 2023-10-06T05:21:15.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.291+0000 7f28d2bff700 1 -- 172.21.15.31:0/1663207285 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc15c590 con 0x7f28cc0aef00 2023-10-06T05:21:15.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.292+0000 7f28d2bff700 1 -- 172.21.15.31:0/1663207285 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc04c380 con 0x7f28cc0aef00 2023-10-06T05:21:15.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.293+0000 7f28c8ff9700 1 -- 172.21.15.31:0/1663207285 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f28c0013680 con 0x7f28cc0aef00 2023-10-06T05:21:15.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.294+0000 7f28c8ff9700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f28b0067290 0x7f28b0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:15.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.294+0000 7f28c8ff9700 1 -- 172.21.15.31:0/1663207285 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f28c0085380 con 0x7f28cc0aef00 2023-10-06T05:21:15.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.294+0000 7f28cbfff700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f28b0067290 0x7f28b0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:15.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.295+0000 7f28cbfff700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f28b0067290 0x7f28b0069740 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f28cc16ce40 tx=0x7f28b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:15.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.299+0000 7f28c8ff9700 1 -- 172.21.15.31:0/1663207285 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28c0052040 con 0x7f28cc0aef00 2023-10-06T05:21:15.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:15.550+0000 7f28d2bff700 1 -- 172.21.15.31:0/1663207285 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f28cc15c110 con 0x7f28cc0aef00 2023-10-06T05:21:16.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.174+0000 7f28c8ff9700 1 -- 172.21.15.31:0/1663207285 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7f28cc15c110 con 0x7f28cc0aef00 2023-10-06T05:21:16.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.177+0000 7f28d2bff700 1 -- 172.21.15.31:0/1663207285 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f28b0067290 msgr2=0x7f28b0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.177+0000 7f28d2bff700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f28b0067290 0x7f28b0069740 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f28cc16ce40 tx=0x7f28b8009040 comp rx=0 tx=0).stop 2023-10-06T05:21:16.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.177+0000 7f28d2bff700 1 -- 172.21.15.31:0/1663207285 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0aef00 msgr2=0x7f28cc15b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.177+0000 7f28d2bff700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0aef00 0x7f28cc15b990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f28c000ec80 tx=0x7f28c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.177+0000 7f28d2bff700 1 -- 172.21.15.31:0/1663207285 shutdown_connections 2023-10-06T05:21:16.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.177+0000 7f28d2bff700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f28b0067290 0x7f28b0069740 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.177+0000 7f28d2bff700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28cc0c7560 0x7f28cc0c03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.177+0000 7f28d2bff700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28cc0aef00 0x7f28cc15b990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.177+0000 7f28d2bff700 1 --2- 172.21.15.31:0/1663207285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28cc0c7190 0x7f28cc14a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.177+0000 7f28d2bff700 1 -- 172.21.15.31:0/1663207285 >> 172.21.15.31:0/1663207285 conn(0x7f28cc0cd3d0 msgr2=0x7f28cc06dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:16.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.178+0000 7f28d2bff700 1 -- 172.21.15.31:0/1663207285 shutdown_connections 2023-10-06T05:21:16.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.178+0000 7f28d2bff700 1 -- 172.21.15.31:0/1663207285 wait complete. 2023-10-06T05:21:16.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:21:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.402+0000 7efee6bff700 1 Processor -- start 2023-10-06T05:21:16.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.402+0000 7efee6bff700 1 -- start start 2023-10-06T05:21:16.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.404+0000 7efee6bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee0153d40 0x7efee00b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.404+0000 7efee6bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efee00af890 0x7efee00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.404+0000 7efee6bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee00ad690 con 0x7efee00b09c0 2023-10-06T05:21:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.404+0000 7efee6bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00afcd0 con 0x7efee00af890 2023-10-06T05:21:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.404+0000 7efee6bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee00b0e30 con 0x7efee0153d40 2023-10-06T05:21:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.404+0000 7efedffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efee00b09c0 0x7efee00af180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41272/0 (socket says 172.21.15.31:41272) 2023-10-06T05:21:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.404+0000 7efedffff700 1 -- 172.21.15.31:0/549547466 learned_addr learned my addr 172.21.15.31:0/549547466 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.404+0000 7efee499b700 1 --2- 172.21.15.31:0/549547466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee0153d40 0x7efee00b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.404+0000 7efee519c700 1 --2- 172.21.15.31:0/549547466 >> v2:172.21.15.62:3300/0 conn(0x7efee00af890 0x7efee00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.404+0000 7efedf7fe700 1 -- 172.21.15.31:0/549547466 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691166476 0 0) 0x7efee00ad690 con 0x7efee00b09c0 2023-10-06T05:21:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.405+0000 7efedf7fe700 1 -- 172.21.15.31:0/549547466 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed0003570 con 0x7efee00b09c0 2023-10-06T05:21:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.405+0000 7efee519c700 1 -- 172.21.15.31:0/549547466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee0153d40 msgr2=0x7efee00b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.405+0000 7efee519c700 1 --2- 172.21.15.31:0/549547466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee0153d40 0x7efee00b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.405+0000 7efee519c700 1 -- 172.21.15.31:0/549547466 >> v1:172.21.15.31:6789/0 conn(0x7efee00b09c0 legacy=0x7efee00af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.405+0000 7efedf7fe700 1 -- 172.21.15.31:0/549547466 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483664998 0 0) 0x7efed0003570 con 0x7efee00b09c0 2023-10-06T05:21:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.405+0000 7efee519c700 1 -- 172.21.15.31:0/549547466 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00acd60 con 0x7efee00af890 2023-10-06T05:21:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.405+0000 7efee499b700 1 --2- 172.21.15.31:0/549547466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee0153d40 0x7efee00b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.405+0000 7efee519c700 1 --2- 172.21.15.31:0/549547466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee00af890 0x7efee00adb20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efed8009e60 tx=0x7efed8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff53a66e9c5cc120 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.405+0000 7efedf7fe700 1 -- 172.21.15.31:0/549547466 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800e070 con 0x7efee00af890 2023-10-06T05:21:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.406+0000 7efedf7fe700 1 -- 172.21.15.31:0/549547466 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed800b7a0 con 0x7efee00af890 2023-10-06T05:21:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.406+0000 7efedf7fe700 1 -- 172.21.15.31:0/549547466 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc001470 con 0x7efee00af890 2023-10-06T05:21:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.406+0000 7efee6bff700 1 -- 172.21.15.31:0/549547466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee00af890 msgr2=0x7efee00adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.406+0000 7efee6bff700 1 --2- 172.21.15.31:0/549547466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee00af890 0x7efee00adb20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efed8009e60 tx=0x7efed8009780 comp rx=0 tx=0).stop 2023-10-06T05:21:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.406+0000 7efee6bff700 1 -- 172.21.15.31:0/549547466 shutdown_connections 2023-10-06T05:21:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.406+0000 7efee6bff700 1 --2- 172.21.15.31:0/549547466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee00af890 0x7efee00adb20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.406+0000 7efee6bff700 1 --2- 172.21.15.31:0/549547466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee0153d40 0x7efee00b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.406+0000 7efee6bff700 1 -- 172.21.15.31:0/549547466 >> 172.21.15.31:0/549547466 conn(0x7efee0067560 msgr2=0x7efee0067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.406+0000 7efee6bff700 1 -- 172.21.15.31:0/549547466 shutdown_connections 2023-10-06T05:21:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.406+0000 7efee6bff700 1 -- 172.21.15.31:0/549547466 wait complete. 2023-10-06T05:21:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.408+0000 7efee6bff700 1 Processor -- start 2023-10-06T05:21:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.408+0000 7efee6bff700 1 -- start start 2023-10-06T05:21:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee6bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee00af890 0x7efee01502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee6bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee00b09c0 0x7efee01592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee6bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee0153d40 0x7efee0159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee6bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee015c080 con 0x7efee00af890 2023-10-06T05:21:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee6bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee0156e20 con 0x7efee0153d40 2023-10-06T05:21:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee6bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee015c2d0 con 0x7efee00b09c0 2023-10-06T05:21:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee00af890 0x7efee01502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee00af890 0x7efee01502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60456/0 (socket says 172.21.15.31:60456) 2023-10-06T05:21:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee499b700 1 -- 172.21.15.31:0/2856942369 learned_addr learned my addr 172.21.15.31:0/2856942369 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee499b700 1 -- 172.21.15.31:0/2856942369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee00b09c0 msgr2=0x7efee01592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efedffff700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee00b09c0 0x7efee01592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee519c700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee0153d40 0x7efee0159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee499b700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee00b09c0 0x7efee01592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee499b700 1 -- 172.21.15.31:0/2856942369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee0153d40 msgr2=0x7efee0159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee499b700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee0153d40 0x7efee0159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee499b700 1 -- 172.21.15.31:0/2856942369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4009420 con 0x7efee00af890 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efedffff700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee00b09c0 0x7efee01592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.410+0000 7efee519c700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee0153d40 0x7efee0159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.411+0000 7efee499b700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee00af890 0x7efee01502f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efed400ecb0 tx=0x7efed40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.411+0000 7efedcff9700 1 -- 172.21.15.31:0/2856942369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400c9d0 con 0x7efee00af890 2023-10-06T05:21:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.411+0000 7efedcff9700 1 -- 172.21.15.31:0/2856942369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed400cb30 con 0x7efee00af890 2023-10-06T05:21:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.411+0000 7efee6bff700 1 -- 172.21.15.31:0/2856942369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed8009a20 con 0x7efee00af890 2023-10-06T05:21:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.411+0000 7efedcff9700 1 -- 172.21.15.31:0/2856942369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400ceb0 con 0x7efee00af890 2023-10-06T05:21:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.411+0000 7efee6bff700 1 -- 172.21.15.31:0/2856942369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee0157070 con 0x7efee00af890 2023-10-06T05:21:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.412+0000 7efee6bff700 1 -- 172.21.15.31:0/2856942369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee004c380 con 0x7efee00af890 2023-10-06T05:21:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.413+0000 7efedcff9700 1 -- 172.21.15.31:0/2856942369 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7efed4013680 con 0x7efee00af890 2023-10-06T05:21:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.414+0000 7efedcff9700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec4067290 0x7efec4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.414+0000 7efedcff9700 1 -- 172.21.15.31:0/2856942369 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efed4084fb0 con 0x7efee00af890 2023-10-06T05:21:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.414+0000 7efedffff700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec4067290 0x7efec4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.415+0000 7efedffff700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec4067290 0x7efec4069740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efecc001ed0 tx=0x7efecc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:16.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.419+0000 7efedcff9700 1 -- 172.21.15.31:0/2856942369 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efed4055700 con 0x7efee00af890 2023-10-06T05:21:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:16.663+0000 7efee6bff700 1 -- 172.21.15.31:0/2856942369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efee00ad040 con 0x7efee00af890 2023-10-06T05:21:17.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.175+0000 7efedcff9700 1 -- 172.21.15.31:0/2856942369 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7efed4055290 con 0x7efee00af890 2023-10-06T05:21:17.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.178+0000 7efee6bff700 1 -- 172.21.15.31:0/2856942369 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec4067290 msgr2=0x7efec4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.178+0000 7efee6bff700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec4067290 0x7efec4069740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efecc001ed0 tx=0x7efecc009040 comp rx=0 tx=0).stop 2023-10-06T05:21:17.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.178+0000 7efee6bff700 1 -- 172.21.15.31:0/2856942369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee00af890 msgr2=0x7efee01502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.178+0000 7efee6bff700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee00af890 0x7efee01502f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efed400ecb0 tx=0x7efed40091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.178+0000 7efee6bff700 1 -- 172.21.15.31:0/2856942369 shutdown_connections 2023-10-06T05:21:17.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.178+0000 7efee6bff700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee0153d40 0x7efee0159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.178+0000 7efee6bff700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7efec4067290 0x7efec4069740 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.178+0000 7efee6bff700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee00af890 0x7efee01502f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.178+0000 7efee6bff700 1 --2- 172.21.15.31:0/2856942369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee00b09c0 0x7efee01592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.178+0000 7efee6bff700 1 -- 172.21.15.31:0/2856942369 >> 172.21.15.31:0/2856942369 conn(0x7efee0067560 msgr2=0x7efee00cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:17.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.179+0000 7efee6bff700 1 -- 172.21.15.31:0/2856942369 shutdown_connections 2023-10-06T05:21:17.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.179+0000 7efee6bff700 1 -- 172.21.15.31:0/2856942369 wait complete. 2023-10-06T05:21:17.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.400+0000 7f27d2fff700 1 Processor -- start 2023-10-06T05:21:17.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.400+0000 7f27d2fff700 1 -- start start 2023-10-06T05:21:17.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.401+0000 7f27d2fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f27cc159500 0x7f27cc0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.401+0000 7f27d2fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f27cc15d2d0 0x7f27cc0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.401+0000 7f27d2fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc0c3f40 con 0x7f27cc1598d0 2023-10-06T05:21:17.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27d2fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc0b2860 con 0x7f27cc159500 2023-10-06T05:21:17.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27d2fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc15d7d0 con 0x7f27cc15d2d0 2023-10-06T05:21:17.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27d0d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f27cc159500 0x7f27cc0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27d0d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f27cc159500 0x7f27cc0b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60748/0 (socket says 172.21.15.31:60748) 2023-10-06T05:21:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27d0d9b700 1 -- 172.21.15.31:0/1213788991 learned_addr learned my addr 172.21.15.31:0/1213788991 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27cbfff700 1 --2- 172.21.15.31:0/1213788991 >> v2:172.21.15.62:3300/0 conn(0x7f27cc15d2d0 0x7f27cc0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27d0d9b700 1 -- 172.21.15.31:0/1213788991 >> v2:172.21.15.62:3300/0 conn(0x7f27cc15d2d0 msgr2=0x7f27cc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27d0d9b700 1 --2- 172.21.15.31:0/1213788991 >> v2:172.21.15.62:3300/0 conn(0x7f27cc15d2d0 0x7f27cc0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27d0d9b700 1 -- 172.21.15.31:0/1213788991 >> v1:172.21.15.62:6789/0 conn(0x7f27cc1598d0 legacy=0x7f27cc0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27d0d9b700 1 -- 172.21.15.31:0/1213788991 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0cb000 con 0x7f27cc159500 2023-10-06T05:21:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.402+0000 7f27cbfff700 1 --2- 172.21.15.31:0/1213788991 >> v2:172.21.15.62:3300/0 conn(0x7f27cc15d2d0 0x7f27cc0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:17.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.403+0000 7f27d0d9b700 1 --2- 172.21.15.31:0/1213788991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 0x7f27cc0b0ab0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f27cc15d200 tx=0x7f27c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd10c25f5a5270f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.403+0000 7f27cb7fe700 1 -- 172.21.15.31:0/1213788991 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000b820 con 0x7f27cc159500 2023-10-06T05:21:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.403+0000 7f27cb7fe700 1 -- 172.21.15.31:0/1213788991 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c000b980 con 0x7f27cc159500 2023-10-06T05:21:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.403+0000 7f27cb7fe700 1 -- 172.21.15.31:0/1213788991 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000bd20 con 0x7f27cc159500 2023-10-06T05:21:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.403+0000 7f27d2fff700 1 -- 172.21.15.31:0/1213788991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 msgr2=0x7f27cc0b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.403+0000 7f27d2fff700 1 --2- 172.21.15.31:0/1213788991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 0x7f27cc0b0ab0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f27cc15d200 tx=0x7f27c0009250 comp rx=0 tx=0).stop 2023-10-06T05:21:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.403+0000 7f27d2fff700 1 -- 172.21.15.31:0/1213788991 shutdown_connections 2023-10-06T05:21:17.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.403+0000 7f27d2fff700 1 --2- 172.21.15.31:0/1213788991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 0x7f27cc0b0ab0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.403+0000 7f27d2fff700 1 --2- 172.21.15.31:0/1213788991 >> v2:172.21.15.62:3300/0 conn(0x7f27cc15d2d0 0x7f27cc0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.403+0000 7f27d2fff700 1 -- 172.21.15.31:0/1213788991 >> 172.21.15.31:0/1213788991 conn(0x7f27cc06ec90 msgr2=0x7f27cc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:17.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.404+0000 7f27d2fff700 1 -- 172.21.15.31:0/1213788991 shutdown_connections 2023-10-06T05:21:17.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.404+0000 7f27d2fff700 1 -- 172.21.15.31:0/1213788991 wait complete. 2023-10-06T05:21:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.405+0000 7f27d2fff700 1 Processor -- start 2023-10-06T05:21:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.405+0000 7f27d2fff700 1 -- start start 2023-10-06T05:21:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27d2fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 0x7f27cc0b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27d2fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27cc1598d0 0x7f27cc0b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27d2fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27cc15d2d0 0x7f27cc0b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27d2fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0c76e0 con 0x7f27cc159500 2023-10-06T05:21:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27d2fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc16e150 con 0x7f27cc15d2d0 2023-10-06T05:21:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27d2fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc169c80 con 0x7f27cc1598d0 2023-10-06T05:21:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27d0d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 0x7f27cc0b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27d0d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 0x7f27cc0b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60758/0 (socket says 172.21.15.31:60758) 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27d0d9b700 1 -- 172.21.15.31:0/2287684416 learned_addr learned my addr 172.21.15.31:0/2287684416 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27cbfff700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27cc1598d0 0x7f27cc0b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.406+0000 7f27d159c700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27cc15d2d0 0x7f27cc0b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27d0d9b700 1 -- 172.21.15.31:0/2287684416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27cc1598d0 msgr2=0x7f27cc0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27d0d9b700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27cc1598d0 0x7f27cc0b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27d0d9b700 1 -- 172.21.15.31:0/2287684416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27cc15d2d0 msgr2=0x7f27cc0b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27d0d9b700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27cc15d2d0 0x7f27cc0b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27d0d9b700 1 -- 172.21.15.31:0/2287684416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c00094f0 con 0x7f27cc159500 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27d0d9b700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 0x7f27cc0b01f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f27cc067c50 tx=0x7f27c0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27c8ff9700 1 -- 172.21.15.31:0/2287684416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0011e20 con 0x7f27cc159500 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27c8ff9700 1 -- 172.21.15.31:0/2287684416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c0020070 con 0x7f27cc159500 2023-10-06T05:21:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27d2fff700 1 -- 172.21.15.31:0/2287684416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc169f20 con 0x7f27cc159500 2023-10-06T05:21:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27c8ff9700 1 -- 172.21.15.31:0/2287684416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c001b6a0 con 0x7f27cc159500 2023-10-06T05:21:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.407+0000 7f27d2fff700 1 -- 172.21.15.31:0/2287684416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc16a560 con 0x7f27cc159500 2023-10-06T05:21:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.408+0000 7f27d2fff700 1 -- 172.21.15.31:0/2287684416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27cc04c380 con 0x7f27cc159500 2023-10-06T05:21:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.409+0000 7f27c8ff9700 1 -- 172.21.15.31:0/2287684416 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f27c0022030 con 0x7f27cc159500 2023-10-06T05:21:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.410+0000 7f27c8ff9700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f27b0067290 0x7f27b0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.410+0000 7f27c8ff9700 1 -- 172.21.15.31:0/2287684416 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f27c0086bb0 con 0x7f27cc159500 2023-10-06T05:21:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.410+0000 7f27cbfff700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f27b0067290 0x7f27b0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.411+0000 7f27cbfff700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f27b0067290 0x7f27b0069740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0ca8c0 tx=0x7f27b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:17.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.415+0000 7f27c8ff9700 1 -- 172.21.15.31:0/2287684416 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27c0053870 con 0x7f27cc159500 2023-10-06T05:21:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.661+0000 7f27d2fff700 1 -- 172.21.15.31:0/2287684416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27cc04c380 con 0x7f27cc159500 2023-10-06T05:21:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.662+0000 7f27c8ff9700 1 -- 172.21.15.31:0/2287684416 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f27c0056e90 con 0x7f27cc159500 2023-10-06T05:21:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:17.664 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2151449662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2151449662},{"type":"v1","addr":"172.21.15.62:6834","nonce":2151449662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/810564890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":810564890},{"type":"v1","addr":"172.21.15.31:6836","nonce":810564890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/802676715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":802676715},{"type":"v1","addr":"172.21.15.31:6837","nonce":802676715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:13.661482+0000","modified":"2023-10-06T05:21:17.173141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6249":{"gid":6249,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/934773709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":934773709},{"type":"v1","addr":"172.21.15.62:6835","nonce":934773709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.664+0000 7f27d2fff700 1 -- 172.21.15.31:0/2287684416 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f27b0067290 msgr2=0x7f27b0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.664+0000 7f27d2fff700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f27b0067290 0x7f27b0069740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0ca8c0 tx=0x7f27b8008040 comp rx=0 tx=0).stop 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.664+0000 7f27d2fff700 1 -- 172.21.15.31:0/2287684416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 msgr2=0x7f27cc0b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.664+0000 7f27d2fff700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 0x7f27cc0b01f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f27cc067c50 tx=0x7f27c0016000 comp rx=0 tx=0).stop 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.665+0000 7f27d2fff700 1 -- 172.21.15.31:0/2287684416 shutdown_connections 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.665+0000 7f27d2fff700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27cc15d2d0 0x7f27cc0b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.665+0000 7f27d2fff700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f27b0067290 0x7f27b0069740 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.665+0000 7f27d2fff700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27cc159500 0x7f27cc0b01f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.665+0000 7f27d2fff700 1 --2- 172.21.15.31:0/2287684416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27cc1598d0 0x7f27cc0b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.665+0000 7f27d2fff700 1 -- 172.21.15.31:0/2287684416 >> 172.21.15.31:0/2287684416 conn(0x7f27cc06ec90 msgr2=0x7f27cc158990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.665+0000 7f27d2fff700 1 -- 172.21.15.31:0/2287684416 shutdown_connections 2023-10-06T05:21:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.665+0000 7f27d2fff700 1 -- 172.21.15.31:0/2287684416 wait complete. 2023-10-06T05:21:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 44 2023-10-06T05:21:17.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.877+0000 7fc455fff700 1 Processor -- start 2023-10-06T05:21:17.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.877+0000 7fc455fff700 1 -- start start 2023-10-06T05:21:17.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.878+0000 7fc455fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc450159580 0x7fc4500cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.878+0000 7fc455fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc4500cb9f0 0x7fc4500cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.878+0000 7fc455fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500ccc80 0x7fc4500cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.878+0000 7fc455fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4500cf280 con 0x7fc450159580 2023-10-06T05:21:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.878+0000 7fc455fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4500ce670 con 0x7fc4500cb9f0 2023-10-06T05:21:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.878+0000 7fc455fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4500ce950 con 0x7fc4500ccc80 2023-10-06T05:21:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.878+0000 7fc44f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc450159580 0x7fc4500cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.878+0000 7fc44f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc450159580 0x7fc4500cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60764/0 (socket says 172.21.15.31:60764) 2023-10-06T05:21:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.878+0000 7fc44f7fe700 1 -- 172.21.15.31:0/1660109333 learned_addr learned my addr 172.21.15.31:0/1660109333 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44ffff700 1 --2- 172.21.15.31:0/1660109333 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500ccc80 0x7fc4500cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44effd700 1 --2- 172.21.15.31:0/1660109333 >> v2:172.21.15.62:3300/0 conn(0x7fc4500cb9f0 0x7fc4500cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44f7fe700 1 -- 172.21.15.31:0/1660109333 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500ccc80 msgr2=0x7fc4500cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44f7fe700 1 --2- 172.21.15.31:0/1660109333 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500ccc80 0x7fc4500cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44f7fe700 1 -- 172.21.15.31:0/1660109333 >> v2:172.21.15.62:3300/0 conn(0x7fc4500cb9f0 msgr2=0x7fc4500cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44f7fe700 1 --2- 172.21.15.31:0/1660109333 >> v2:172.21.15.62:3300/0 conn(0x7fc4500cb9f0 0x7fc4500cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44f7fe700 1 -- 172.21.15.31:0/1660109333 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4500cec30 con 0x7fc450159580 2023-10-06T05:21:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44ffff700 1 --2- 172.21.15.31:0/1660109333 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500ccc80 0x7fc4500cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44f7fe700 1 --2- 172.21.15.31:0/1660109333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 0x7fc4500cdfe0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc45006fc50 tx=0x7fc440009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834485fb8260bf64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:17.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44e7fc700 1 -- 172.21.15.31:0/1660109333 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440005c10 con 0x7fc450159580 2023-10-06T05:21:17.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44e7fc700 1 -- 172.21.15.31:0/1660109333 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc440005d70 con 0x7fc450159580 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.879+0000 7fc44e7fc700 1 -- 172.21.15.31:0/1660109333 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44000b880 con 0x7fc450159580 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.880+0000 7fc455fff700 1 -- 172.21.15.31:0/1660109333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 msgr2=0x7fc4500cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.880+0000 7fc455fff700 1 --2- 172.21.15.31:0/1660109333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 0x7fc4500cdfe0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc45006fc50 tx=0x7fc440009220 comp rx=0 tx=0).stop 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.880+0000 7fc455fff700 1 -- 172.21.15.31:0/1660109333 shutdown_connections 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.880+0000 7fc455fff700 1 --2- 172.21.15.31:0/1660109333 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500ccc80 0x7fc4500cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.880+0000 7fc455fff700 1 --2- 172.21.15.31:0/1660109333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 0x7fc4500cdfe0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.880+0000 7fc455fff700 1 --2- 172.21.15.31:0/1660109333 >> v2:172.21.15.62:3300/0 conn(0x7fc4500cb9f0 0x7fc4500cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.880+0000 7fc455fff700 1 -- 172.21.15.31:0/1660109333 >> 172.21.15.31:0/1660109333 conn(0x7fc450099710 msgr2=0x7fc45006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.880+0000 7fc455fff700 1 -- 172.21.15.31:0/1660109333 shutdown_connections 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.880+0000 7fc455fff700 1 -- 172.21.15.31:0/1660109333 wait complete. 2023-10-06T05:21:17.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.882+0000 7fc455fff700 1 Processor -- start 2023-10-06T05:21:17.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.882+0000 7fc455fff700 1 -- start start 2023-10-06T05:21:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc455fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500cb9f0 0x7fc45016cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc455fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4500ccc80 0x7fc45016d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc455fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 0x7fc4500c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc455fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45016a7d0 con 0x7fc450159580 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc455fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45016aab0 con 0x7fc4500ccc80 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc455fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4500c5010 con 0x7fc4500cb9f0 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 0x7fc4500c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 0x7fc4500c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60770/0 (socket says 172.21.15.31:60770) 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44ffff700 1 -- 172.21.15.31:0/1899220552 learned_addr learned my addr 172.21.15.31:0/1899220552 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44f7fe700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500cb9f0 0x7fc45016cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44effd700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4500ccc80 0x7fc45016d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44ffff700 1 -- 172.21.15.31:0/1899220552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500cb9f0 msgr2=0x7fc45016cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44ffff700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500cb9f0 0x7fc45016cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44ffff700 1 -- 172.21.15.31:0/1899220552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4500ccc80 msgr2=0x7fc45016d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44ffff700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4500ccc80 0x7fc45016d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44ffff700 1 -- 172.21.15.31:0/1899220552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc440009480 con 0x7fc450159580 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.883+0000 7fc44effd700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4500ccc80 0x7fc45016d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.884+0000 7fc44ffff700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 0x7fc4500c4250 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc44800d1b0 tx=0x7fc448009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.884+0000 7fc4377fe700 1 -- 172.21.15.31:0/1899220552 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44800dda0 con 0x7fc450159580 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.884+0000 7fc4377fe700 1 -- 172.21.15.31:0/1899220552 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44800f040 con 0x7fc450159580 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.884+0000 7fc455fff700 1 -- 172.21.15.31:0/1899220552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4500c52f0 con 0x7fc450159580 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.884+0000 7fc4377fe700 1 -- 172.21.15.31:0/1899220552 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448015690 con 0x7fc450159580 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.884+0000 7fc455fff700 1 -- 172.21.15.31:0/1899220552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4500c6160 con 0x7fc450159580 2023-10-06T05:21:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.885+0000 7fc455fff700 1 -- 172.21.15.31:0/1899220552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4500c6440 con 0x7fc450159580 2023-10-06T05:21:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.886+0000 7fc4377fe700 1 -- 172.21.15.31:0/1899220552 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fc44801e3b0 con 0x7fc450159580 2023-10-06T05:21:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.887+0000 7fc4377fe700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc43c067290 0x7fc43c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.887+0000 7fc4377fe700 1 -- 172.21.15.31:0/1899220552 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc448085260 con 0x7fc450159580 2023-10-06T05:21:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.887+0000 7fc44f7fe700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc43c067290 0x7fc43c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.888+0000 7fc44f7fe700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc43c067290 0x7fc43c069740 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc4500c3d20 tx=0x7fc440016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:17.891+0000 7fc4377fe700 1 -- 172.21.15.31:0/1899220552 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc448051e10 con 0x7fc450159580 2023-10-06T05:21:18.078 INFO:tasks.ceph.mds.d.smithi062.stderr:2023-10-06T05:21:18.077+0000 7fc007df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:21:18.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.094+0000 7fc455fff700 1 -- 172.21.15.31:0/1899220552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc45004c380 con 0x7fc450159580 2023-10-06T05:21:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.095+0000 7fc4377fe700 1 -- 172.21.15.31:0/1899220552 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16917 (secure 0 0 0) 0x7fc448055430 con 0x7fc450159580 2023-10-06T05:21:18.097 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:18.097 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":71,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:15.160084+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:12.092711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:12.637981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 -- 172.21.15.31:0/1899220552 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc43c067290 msgr2=0x7fc43c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc43c067290 0x7fc43c069740 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc4500c3d20 tx=0x7fc440016000 comp rx=0 tx=0).stop 2023-10-06T05:21:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 -- 172.21.15.31:0/1899220552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 msgr2=0x7fc4500c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 0x7fc4500c4250 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc44800d1b0 tx=0x7fc448009a50 comp rx=0 tx=0).stop 2023-10-06T05:21:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 -- 172.21.15.31:0/1899220552 shutdown_connections 2023-10-06T05:21:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc43c067290 0x7fc43c069740 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc450159580 0x7fc4500c4250 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4500cb9f0 0x7fc45016cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 --2- 172.21.15.31:0/1899220552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4500ccc80 0x7fc45016d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 -- 172.21.15.31:0/1899220552 >> 172.21.15.31:0/1899220552 conn(0x7fc450099710 msgr2=0x7fc4500b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 -- 172.21.15.31:0/1899220552 shutdown_connections 2023-10-06T05:21:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.097+0000 7fc455fff700 1 -- 172.21.15.31:0/1899220552 wait complete. 2023-10-06T05:21:18.117 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:21:18.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.313+0000 7fccaa1ff700 1 Processor -- start 2023-10-06T05:21:18.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.313+0000 7fccaa1ff700 1 -- start start 2023-10-06T05:21:18.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.314+0000 7fccaa1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcca41596d0 0x7fcca4159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.314+0000 7fccaa1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcca40c16e0 0x7fcca40c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.314+0000 7fccaa1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca4067f70 con 0x7fcca40c6f80 2023-10-06T05:21:18.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.314+0000 7fccaa1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca406b040 con 0x7fcca41596d0 2023-10-06T05:21:18.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.314+0000 7fccaa1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca40c20c0 con 0x7fcca40c16e0 2023-10-06T05:21:18.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.314+0000 7fcc9bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcca40c6f80 0x7fcca406a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35578/0 (socket says 172.21.15.31:35578) 2023-10-06T05:21:18.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.314+0000 7fcc9bfff700 1 -- 172.21.15.31:0/1222809296 learned_addr learned my addr 172.21.15.31:0/1222809296 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:18.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.315+0000 7fcca3fff700 1 --2- 172.21.15.31:0/1222809296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcca40c16e0 0x7fcca40c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:18.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.315+0000 7fcca37fe700 1 --2- 172.21.15.31:0/1222809296 >> v2:172.21.15.62:3300/0 conn(0x7fcca41596d0 0x7fcca4159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:18.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.315+0000 7fcca2ffd700 1 -- 172.21.15.31:0/1222809296 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162312846 0 0) 0x7fcca4067f70 con 0x7fcca40c6f80 2023-10-06T05:21:18.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.315+0000 7fcca2ffd700 1 -- 172.21.15.31:0/1222809296 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc90003570 con 0x7fcca40c6f80 2023-10-06T05:21:18.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.315+0000 7fcca3fff700 1 -- 172.21.15.31:0/1222809296 >> v2:172.21.15.62:3300/0 conn(0x7fcca41596d0 msgr2=0x7fcca4159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:18.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.315+0000 7fcca3fff700 1 --2- 172.21.15.31:0/1222809296 >> v2:172.21.15.62:3300/0 conn(0x7fcca41596d0 0x7fcca4159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.315+0000 7fcca2ffd700 1 -- 172.21.15.31:0/1222809296 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2674721836 0 0) 0x7fcc90003570 con 0x7fcca40c6f80 2023-10-06T05:21:18.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.315+0000 7fcca3fff700 1 -- 172.21.15.31:0/1222809296 >> v1:172.21.15.31:6789/0 conn(0x7fcca40c6f80 legacy=0x7fcca406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.315+0000 7fcca3fff700 1 -- 172.21.15.31:0/1222809296 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca406e040 con 0x7fcca40c16e0 2023-10-06T05:21:18.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.315+0000 7fcca37fe700 1 --2- 172.21.15.31:0/1222809296 >> v2:172.21.15.62:3300/0 conn(0x7fcca41596d0 0x7fcca4159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:18.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.316+0000 7fcca3fff700 1 --2- 172.21.15.31:0/1222809296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcca40c16e0 0x7fcca40c1b80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009e60 tx=0x7fcc9c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207038034d1a5e70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.316+0000 7fcca2ffd700 1 -- 172.21.15.31:0/1222809296 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00e070 con 0x7fcca40c16e0 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.316+0000 7fcca2ffd700 1 -- 172.21.15.31:0/1222809296 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c00b7a0 con 0x7fcca40c16e0 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.316+0000 7fcca2ffd700 1 -- 172.21.15.31:0/1222809296 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c001470 con 0x7fcca40c16e0 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.316+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1222809296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcca40c16e0 msgr2=0x7fcca40c1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.316+0000 7fccaa1ff700 1 --2- 172.21.15.31:0/1222809296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcca40c16e0 0x7fcca40c1b80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009e60 tx=0x7fcc9c009780 comp rx=0 tx=0).stop 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.316+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1222809296 shutdown_connections 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.317+0000 7fccaa1ff700 1 --2- 172.21.15.31:0/1222809296 >> v2:172.21.15.62:3300/0 conn(0x7fcca41596d0 0x7fcca4159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.317+0000 7fccaa1ff700 1 --2- 172.21.15.31:0/1222809296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcca40c16e0 0x7fcca40c1b80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.317+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1222809296 >> 172.21.15.31:0/1222809296 conn(0x7fcca40c9e80 msgr2=0x7fcca406a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.317+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1222809296 shutdown_connections 2023-10-06T05:21:18.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.317+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1222809296 wait complete. 2023-10-06T05:21:18.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.319+0000 7fccaa1ff700 1 Processor -- start 2023-10-06T05:21:18.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.319+0000 7fccaa1ff700 1 -- start start 2023-10-06T05:21:18.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.320+0000 7fccaa1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcca40c16e0 0x7fcca40cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.320+0000 7fccaa1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcca40c6f80 0x7fcca41500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.320+0000 7fccaa1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcca41596d0 0x7fcca41505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.320+0000 7fccaa1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca40b22b0 con 0x7fcca40c16e0 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.320+0000 7fccaa1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca40b16a0 con 0x7fcca40c6f80 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.320+0000 7fccaa1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca40b1980 con 0x7fcca41596d0 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.320+0000 7fcca37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcca40c16e0 0x7fcca40cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.320+0000 7fcca37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcca40c16e0 0x7fcca40cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60786/0 (socket says 172.21.15.31:60786) 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcca37fe700 1 -- 172.21.15.31:0/1092397554 learned_addr learned my addr 172.21.15.31:0/1092397554 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcca37fe700 1 -- 172.21.15.31:0/1092397554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcca41596d0 msgr2=0x7fcca41505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcca37fe700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcca41596d0 0x7fcca41505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcca37fe700 1 -- 172.21.15.31:0/1092397554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcca40c6f80 msgr2=0x7fcca41500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcc9bfff700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcca40c6f80 0x7fcca41500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcca37fe700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcca40c6f80 0x7fcca41500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcca37fe700 1 -- 172.21.15.31:0/1092397554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc94009420 con 0x7fcca40c16e0 2023-10-06T05:21:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcc9bfff700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcca40c6f80 0x7fcca41500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcca37fe700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcca40c16e0 0x7fcca40cb1f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400ec80 tx=0x7fcc940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcc9b7fe700 1 -- 172.21.15.31:0/1092397554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400c9d0 con 0x7fcca40c16e0 2023-10-06T05:21:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcc9b7fe700 1 -- 172.21.15.31:0/1092397554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9400cb30 con 0x7fcca40c16e0 2023-10-06T05:21:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fcc9b7fe700 1 -- 172.21.15.31:0/1092397554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400ceb0 con 0x7fcca40c16e0 2023-10-06T05:21:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.321+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1092397554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9c009a20 con 0x7fcca40c16e0 2023-10-06T05:21:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.322+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1092397554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca40b1f40 con 0x7fcca40c16e0 2023-10-06T05:21:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.323+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1092397554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca404c380 con 0x7fcca40c16e0 2023-10-06T05:21:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.323+0000 7fcc9b7fe700 1 -- 172.21.15.31:0/1092397554 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fcc94013680 con 0x7fcca40c16e0 2023-10-06T05:21:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.324+0000 7fcc9b7fe700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc84067290 0x7fcc84069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.324+0000 7fcc9b7fe700 1 -- 172.21.15.31:0/1092397554 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcc94085060 con 0x7fcca40c16e0 2023-10-06T05:21:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.324+0000 7fcc9bfff700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc84067290 0x7fcc84069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:18.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.325+0000 7fcc9bfff700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc84067290 0x7fcc84069740 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0011e0 tx=0x7fcc8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:18.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.328+0000 7fcc9b7fe700 1 -- 172.21.15.31:0/1092397554 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc94051d20 con 0x7fcca40c16e0 2023-10-06T05:21:18.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.527+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1092397554 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcca4063760 con 0x7fcc84067290 2023-10-06T05:21:18.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.538+0000 7fcc9b7fe700 1 -- 172.21.15.31:0/1092397554 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308529 (secure 0 0 0) 0x7fcca4063760 con 0x7fcc84067290 2023-10-06T05:21:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1092397554 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc84067290 msgr2=0x7fcc84069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc84067290 0x7fcc84069740 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0011e0 tx=0x7fcc8c009040 comp rx=0 tx=0).stop 2023-10-06T05:21:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1092397554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcca40c16e0 msgr2=0x7fcca40cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcca40c16e0 0x7fcca40cb1f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400ec80 tx=0x7fcc940091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1092397554 shutdown_connections 2023-10-06T05:21:18.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcca41596d0 0x7fcca41505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fcc84067290 0x7fcc84069740 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcca40c16e0 0x7fcca40cb1f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 --2- 172.21.15.31:0/1092397554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcca40c6f80 0x7fcca41500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1092397554 >> 172.21.15.31:0/1092397554 conn(0x7fcca40c9e80 msgr2=0x7fcca40c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:18.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1092397554 shutdown_connections 2023-10-06T05:21:18.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.542+0000 7fccaa1ff700 1 -- 172.21.15.31:0/1092397554 wait complete. 2023-10-06T05:21:18.547 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:21:18.562 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2023-10-06T05:21:18.118385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386024,"kb_used_data":7720,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752588696,"statfs":{"total":773094113280,"available":770650824704,"internally_reserved":0,"allocated":7905280,"data_stored":4158720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.969865"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157298+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:40.622847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162888+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:39.527935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156908+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:37.837314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.160141+0000","last_change":"2023-10-06T05:21:14.160278+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.160141+0000","last_undegraded":"2023-10-06T05:21:14.160141+0000","last_fullsized":"2023-10-06T05:21:14.160141+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:57.160327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.159070+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:16.114354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155121+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:43.798148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158115+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:07.141522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157896+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:00.913829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154871+0000","last_change":"2023-10-06T05:21:14.155317+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154871+0000","last_undegraded":"2023-10-06T05:21:14.154871+0000","last_fullsized":"2023-10-06T05:21:14.154871+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:04.649668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159817+0000","last_change":"2023-10-06T05:21:14.159956+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159817+0000","last_undegraded":"2023-10-06T05:21:14.159817+0000","last_fullsized":"2023-10-06T05:21:14.159817+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:55.793132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.164442+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:21.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159761+0000","last_change":"2023-10-06T05:21:14.159896+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159761+0000","last_undegraded":"2023-10-06T05:21:14.159761+0000","last_fullsized":"2023-10-06T05:21:14.159761+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:16.811458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156981+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:43.232828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.154665+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:20.503103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162708+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:33.730974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156775+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:04.855790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158524+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:50.744732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157660+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:20.672647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.152314+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:25.760034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157280+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.928620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154926+0000","last_change":"2023-10-06T05:21:14.155091+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154926+0000","last_undegraded":"2023-10-06T05:21:14.154926+0000","last_fullsized":"2023-10-06T05:21:14.154926+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:43.694140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158340+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:44.399326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154629+0000","last_change":"2023-10-06T05:21:14.155278+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154629+0000","last_undegraded":"2023-10-06T05:21:14.154629+0000","last_fullsized":"2023-10-06T05:21:14.154629+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:48.387423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.160885+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:29.781374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.163961+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:49.243354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159370+0000","last_change":"2023-10-06T05:21:14.159548+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159370+0000","last_undegraded":"2023-10-06T05:21:14.159370+0000","last_fullsized":"2023-10-06T05:21:14.159370+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:50.021472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158766+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.976026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156773+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:11.364687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.165301+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:40.622847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157061+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:26.471873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155647+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:22.094091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156138+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:21.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155381+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:45.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.159241+0000","last_change":"2023-10-06T05:21:13.159241+0000","last_active":"2023-10-06T05:21:13.159241+0000","last_peered":"2023-10-06T05:21:13.159241+0000","last_clean":"2023-10-06T05:21:13.159241+0000","last_became_active":"2023-10-06T05:21:13.158913+0000","last_became_peered":"2023-10-06T05:21:13.158913+0000","last_unstale":"2023-10-06T05:21:13.159241+0000","last_undegraded":"2023-10-06T05:21:13.159241+0000","last_fullsized":"2023-10-06T05:21:13.159241+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:58.015087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.161893+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:12.975931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171676+0000","last_change":"2023-10-06T05:21:13.171676+0000","last_active":"2023-10-06T05:21:13.171676+0000","last_peered":"2023-10-06T05:21:13.171676+0000","last_clean":"2023-10-06T05:21:13.171676+0000","last_became_active":"2023-10-06T05:21:13.171360+0000","last_became_peered":"2023-10-06T05:21:13.171360+0000","last_unstale":"2023-10-06T05:21:13.171676+0000","last_undegraded":"2023-10-06T05:21:13.171676+0000","last_fullsized":"2023-10-06T05:21:13.171676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:24.757454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154702+0000","last_change":"2023-10-06T05:21:14.155168+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154702+0000","last_undegraded":"2023-10-06T05:21:14.154702+0000","last_fullsized":"2023-10-06T05:21:14.154702+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:48.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.176256+0000","last_change":"2023-10-06T05:21:13.176256+0000","last_active":"2023-10-06T05:21:13.176256+0000","last_peered":"2023-10-06T05:21:13.176256+0000","last_clean":"2023-10-06T05:21:13.176256+0000","last_became_active":"2023-10-06T05:21:13.175487+0000","last_became_peered":"2023-10-06T05:21:13.175487+0000","last_unstale":"2023-10-06T05:21:13.176256+0000","last_undegraded":"2023-10-06T05:21:13.176256+0000","last_fullsized":"2023-10-06T05:21:13.176256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:19.500524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154776+0000","last_change":"2023-10-06T05:21:14.154975+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154776+0000","last_undegraded":"2023-10-06T05:21:14.154776+0000","last_fullsized":"2023-10-06T05:21:14.154776+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:09.170341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.169221+0000","last_change":"2023-10-06T05:21:13.169221+0000","last_active":"2023-10-06T05:21:13.169221+0000","last_peered":"2023-10-06T05:21:13.169221+0000","last_clean":"2023-10-06T05:21:13.169221+0000","last_became_active":"2023-10-06T05:21:13.168853+0000","last_became_peered":"2023-10-06T05:21:13.168853+0000","last_unstale":"2023-10-06T05:21:13.169221+0000","last_undegraded":"2023-10-06T05:21:13.169221+0000","last_fullsized":"2023-10-06T05:21:13.169221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:12.290741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.163333+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:43.798148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.167639+0000","last_change":"2023-10-06T05:21:13.162613+0000","last_active":"2023-10-06T05:21:13.167639+0000","last_peered":"2023-10-06T05:21:13.167639+0000","last_clean":"2023-10-06T05:21:13.167639+0000","last_became_active":"2023-10-06T05:21:13.162167+0000","last_became_peered":"2023-10-06T05:21:13.162167+0000","last_unstale":"2023-10-06T05:21:13.167639+0000","last_undegraded":"2023-10-06T05:21:13.167639+0000","last_fullsized":"2023-10-06T05:21:13.167639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:36.834735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.164889+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:26.471873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.175784+0000","last_change":"2023-10-06T05:21:13.170589+0000","last_active":"2023-10-06T05:21:13.175784+0000","last_peered":"2023-10-06T05:21:13.175784+0000","last_clean":"2023-10-06T05:21:13.175784+0000","last_became_active":"2023-10-06T05:21:13.170282+0000","last_became_peered":"2023-10-06T05:21:13.170282+0000","last_unstale":"2023-10-06T05:21:13.175784+0000","last_undegraded":"2023-10-06T05:21:13.175784+0000","last_fullsized":"2023-10-06T05:21:13.175784+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:53.541975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.155752+0000","last_change":"2023-10-06T05:21:14.155916+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.155752+0000","last_undegraded":"2023-10-06T05:21:14.155752+0000","last_fullsized":"2023-10-06T05:21:14.155752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:54.544554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.175791+0000","last_change":"2023-10-06T05:21:13.175791+0000","last_active":"2023-10-06T05:21:13.175791+0000","last_peered":"2023-10-06T05:21:13.175791+0000","last_clean":"2023-10-06T05:21:13.175791+0000","last_became_active":"2023-10-06T05:21:13.175326+0000","last_became_peered":"2023-10-06T05:21:13.175326+0000","last_unstale":"2023-10-06T05:21:13.175791+0000","last_undegraded":"2023-10-06T05:21:13.175791+0000","last_fullsized":"2023-10-06T05:21:13.175791+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.040745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158575+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:56.726499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.170180+0000","last_change":"2023-10-06T05:21:13.170180+0000","last_active":"2023-10-06T05:21:13.170180+0000","last_peered":"2023-10-06T05:21:13.170180+0000","last_clean":"2023-10-06T05:21:13.170180+0000","last_became_active":"2023-10-06T05:21:13.169857+0000","last_became_peered":"2023-10-06T05:21:13.169857+0000","last_unstale":"2023-10-06T05:21:13.170180+0000","last_undegraded":"2023-10-06T05:21:13.170180+0000","last_fullsized":"2023-10-06T05:21:13.170180+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:45.593601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.161768+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:25.760034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.172453+0000","last_change":"2023-10-06T05:21:13.172453+0000","last_active":"2023-10-06T05:21:13.172453+0000","last_peered":"2023-10-06T05:21:13.172453+0000","last_clean":"2023-10-06T05:21:13.172453+0000","last_became_active":"2023-10-06T05:21:13.171973+0000","last_became_peered":"2023-10-06T05:21:13.171973+0000","last_unstale":"2023-10-06T05:21:13.172453+0000","last_undegraded":"2023-10-06T05:21:13.172453+0000","last_fullsized":"2023-10-06T05:21:13.172453+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:53.470594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156446+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:48.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.170853+0000","last_change":"2023-10-06T05:21:13.168940+0000","last_active":"2023-10-06T05:21:13.170853+0000","last_peered":"2023-10-06T05:21:13.170853+0000","last_clean":"2023-10-06T05:21:13.170853+0000","last_became_active":"2023-10-06T05:21:13.168641+0000","last_became_peered":"2023-10-06T05:21:13.168641+0000","last_unstale":"2023-10-06T05:21:13.170853+0000","last_undegraded":"2023-10-06T05:21:13.170853+0000","last_fullsized":"2023-10-06T05:21:13.170853+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:42.691561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.154172+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:54.473173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.163900+0000","last_change":"2023-10-06T05:21:13.163900+0000","last_active":"2023-10-06T05:21:13.163900+0000","last_peered":"2023-10-06T05:21:13.163900+0000","last_clean":"2023-10-06T05:21:13.163900+0000","last_became_active":"2023-10-06T05:21:13.163604+0000","last_became_peered":"2023-10-06T05:21:13.163604+0000","last_unstale":"2023-10-06T05:21:13.163900+0000","last_undegraded":"2023-10-06T05:21:13.163900+0000","last_fullsized":"2023-10-06T05:21:13.163900+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:55.585059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155983+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:25.096218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.170791+0000","last_change":"2023-10-06T05:21:13.170791+0000","last_active":"2023-10-06T05:21:13.170791+0000","last_peered":"2023-10-06T05:21:13.170791+0000","last_clean":"2023-10-06T05:21:13.170791+0000","last_became_active":"2023-10-06T05:21:13.170470+0000","last_became_peered":"2023-10-06T05:21:13.170470+0000","last_unstale":"2023-10-06T05:21:13.170791+0000","last_undegraded":"2023-10-06T05:21:13.170791+0000","last_fullsized":"2023-10-06T05:21:13.170791+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:26.418570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159317+0000","last_change":"2023-10-06T05:21:14.159488+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159317+0000","last_undegraded":"2023-10-06T05:21:14.159317+0000","last_fullsized":"2023-10-06T05:21:14.159317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:45.462119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.175866+0000","last_change":"2023-10-06T05:21:13.171211+0000","last_active":"2023-10-06T05:21:13.175866+0000","last_peered":"2023-10-06T05:21:13.175866+0000","last_clean":"2023-10-06T05:21:13.175866+0000","last_became_active":"2023-10-06T05:21:13.170917+0000","last_became_peered":"2023-10-06T05:21:13.170917+0000","last_unstale":"2023-10-06T05:21:13.175866+0000","last_undegraded":"2023-10-06T05:21:13.175866+0000","last_fullsized":"2023-10-06T05:21:13.175866+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:08.167762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156475+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:56.735464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171789+0000","last_change":"2023-10-06T05:21:13.171789+0000","last_active":"2023-10-06T05:21:13.171789+0000","last_peered":"2023-10-06T05:21:13.171789+0000","last_clean":"2023-10-06T05:21:13.171789+0000","last_became_active":"2023-10-06T05:21:13.171451+0000","last_became_peered":"2023-10-06T05:21:13.171451+0000","last_unstale":"2023-10-06T05:21:13.171789+0000","last_undegraded":"2023-10-06T05:21:13.171789+0000","last_fullsized":"2023-10-06T05:21:13.171789+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:11.973352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157028+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:02.476152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171171+0000","last_change":"2023-10-06T05:21:13.171171+0000","last_active":"2023-10-06T05:21:13.171171+0000","last_peered":"2023-10-06T05:21:13.171171+0000","last_clean":"2023-10-06T05:21:13.171171+0000","last_became_active":"2023-10-06T05:21:13.170856+0000","last_became_peered":"2023-10-06T05:21:13.170856+0000","last_unstale":"2023-10-06T05:21:13.171171+0000","last_undegraded":"2023-10-06T05:21:13.171171+0000","last_fullsized":"2023-10-06T05:21:13.171171+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:45.319480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157405+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:08.922350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.176681+0000","last_change":"2023-10-06T05:21:13.176681+0000","last_active":"2023-10-06T05:21:13.176681+0000","last_peered":"2023-10-06T05:21:13.176681+0000","last_clean":"2023-10-06T05:21:13.176681+0000","last_became_active":"2023-10-06T05:21:13.176432+0000","last_became_peered":"2023-10-06T05:21:13.176432+0000","last_unstale":"2023-10-06T05:21:13.176681+0000","last_undegraded":"2023-10-06T05:21:13.176681+0000","last_fullsized":"2023-10-06T05:21:13.176681+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:15.808879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154629+0000","last_change":"2023-10-06T05:21:14.154830+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154629+0000","last_undegraded":"2023-10-06T05:21:14.154629+0000","last_fullsized":"2023-10-06T05:21:14.154629+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:06.020289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.169145+0000","last_change":"2023-10-06T05:21:13.165877+0000","last_active":"2023-10-06T05:21:13.169145+0000","last_peered":"2023-10-06T05:21:13.169145+0000","last_clean":"2023-10-06T05:21:13.169145+0000","last_became_active":"2023-10-06T05:21:13.165605+0000","last_became_peered":"2023-10-06T05:21:13.165605+0000","last_unstale":"2023-10-06T05:21:13.169145+0000","last_undegraded":"2023-10-06T05:21:13.169145+0000","last_fullsized":"2023-10-06T05:21:13.169145+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157597+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:26.639167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171761+0000","last_change":"2023-10-06T05:21:13.171761+0000","last_active":"2023-10-06T05:21:13.171761+0000","last_peered":"2023-10-06T05:21:13.171761+0000","last_clean":"2023-10-06T05:21:13.171761+0000","last_became_active":"2023-10-06T05:21:13.171491+0000","last_became_peered":"2023-10-06T05:21:13.171491+0000","last_unstale":"2023-10-06T05:21:13.171761+0000","last_undegraded":"2023-10-06T05:21:13.171761+0000","last_fullsized":"2023-10-06T05:21:13.171761+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:56.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.155136+0000","last_change":"2023-10-06T05:21:14.155680+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.155136+0000","last_undegraded":"2023-10-06T05:21:14.155136+0000","last_fullsized":"2023-10-06T05:21:14.155136+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:04.797474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.167952+0000","last_change":"2023-10-06T05:21:13.167952+0000","last_active":"2023-10-06T05:21:13.167952+0000","last_peered":"2023-10-06T05:21:13.167952+0000","last_clean":"2023-10-06T05:21:13.167952+0000","last_became_active":"2023-10-06T05:21:13.166565+0000","last_became_peered":"2023-10-06T05:21:13.166565+0000","last_unstale":"2023-10-06T05:21:13.167952+0000","last_undegraded":"2023-10-06T05:21:13.167952+0000","last_fullsized":"2023-10-06T05:21:13.167952+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:38.525356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.164809+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:04.855790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171867+0000","last_change":"2023-10-06T05:21:13.171867+0000","last_active":"2023-10-06T05:21:13.171867+0000","last_peered":"2023-10-06T05:21:13.171867+0000","last_clean":"2023-10-06T05:21:13.171867+0000","last_became_active":"2023-10-06T05:21:13.171530+0000","last_became_peered":"2023-10-06T05:21:13.171530+0000","last_unstale":"2023-10-06T05:21:13.171867+0000","last_undegraded":"2023-10-06T05:21:13.171867+0000","last_fullsized":"2023-10-06T05:21:13.171867+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:28.778795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.159646+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:15.167368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.172743+0000","last_change":"2023-10-06T05:21:13.172743+0000","last_active":"2023-10-06T05:21:13.172743+0000","last_peered":"2023-10-06T05:21:13.172743+0000","last_clean":"2023-10-06T05:21:13.172743+0000","last_became_active":"2023-10-06T05:21:13.172453+0000","last_became_peered":"2023-10-06T05:21:13.172453+0000","last_unstale":"2023-10-06T05:21:13.172743+0000","last_undegraded":"2023-10-06T05:21:13.172743+0000","last_fullsized":"2023-10-06T05:21:13.172743+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:01.292501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155880+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:35.471644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171851+0000","last_change":"2023-10-06T05:21:13.171851+0000","last_active":"2023-10-06T05:21:13.171851+0000","last_peered":"2023-10-06T05:21:13.171851+0000","last_clean":"2023-10-06T05:21:13.171851+0000","last_became_active":"2023-10-06T05:21:13.171552+0000","last_became_peered":"2023-10-06T05:21:13.171552+0000","last_unstale":"2023-10-06T05:21:13.171851+0000","last_undegraded":"2023-10-06T05:21:13.171851+0000","last_fullsized":"2023-10-06T05:21:13.171851+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:14:14.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157183+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:21.116415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.172335+0000","last_change":"2023-10-06T05:21:13.172335+0000","last_active":"2023-10-06T05:21:13.172335+0000","last_peered":"2023-10-06T05:21:13.172335+0000","last_clean":"2023-10-06T05:21:13.172335+0000","last_became_active":"2023-10-06T05:21:13.172046+0000","last_became_peered":"2023-10-06T05:21:13.172046+0000","last_unstale":"2023-10-06T05:21:13.172335+0000","last_undegraded":"2023-10-06T05:21:13.172335+0000","last_fullsized":"2023-10-06T05:21:13.172335+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:49.234603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.154688+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:22.127378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.174008+0000","last_change":"2023-10-06T05:21:13.174008+0000","last_active":"2023-10-06T05:21:13.174008+0000","last_peered":"2023-10-06T05:21:13.174008+0000","last_clean":"2023-10-06T05:21:13.174008+0000","last_became_active":"2023-10-06T05:21:13.173503+0000","last_became_peered":"2023-10-06T05:21:13.173503+0000","last_unstale":"2023-10-06T05:21:13.174008+0000","last_undegraded":"2023-10-06T05:21:13.174008+0000","last_fullsized":"2023-10-06T05:21:13.174008+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:14.164789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.152598+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:12.975931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.165946+0000","last_change":"2023-10-06T05:21:13.165946+0000","last_active":"2023-10-06T05:21:13.165946+0000","last_peered":"2023-10-06T05:21:13.165946+0000","last_clean":"2023-10-06T05:21:13.165946+0000","last_became_active":"2023-10-06T05:21:13.165677+0000","last_became_peered":"2023-10-06T05:21:13.165677+0000","last_unstale":"2023-10-06T05:21:13.165946+0000","last_undegraded":"2023-10-06T05:21:13.165946+0000","last_fullsized":"2023-10-06T05:21:13.165946+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:32.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.160560+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:21:13.160560+0000","last_peered":"2023-10-06T05:21:13.160560+0000","last_clean":"2023-10-06T05:21:13.160560+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:21:13.160560+0000","last_undegraded":"2023-10-06T05:21:13.160560+0000","last_fullsized":"2023-10-06T05:21:13.160560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156102+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:36.053585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.169590+0000","last_change":"2023-10-06T05:21:13.165590+0000","last_active":"2023-10-06T05:21:13.169590+0000","last_peered":"2023-10-06T05:21:13.169590+0000","last_clean":"2023-10-06T05:21:13.169590+0000","last_became_active":"2023-10-06T05:21:13.165314+0000","last_became_peered":"2023-10-06T05:21:13.165314+0000","last_unstale":"2023-10-06T05:21:13.169590+0000","last_undegraded":"2023-10-06T05:21:13.169590+0000","last_fullsized":"2023-10-06T05:21:13.169590+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:24.093639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.151269+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:21:12.151269+0000","last_peered":"2023-10-06T05:21:12.151269+0000","last_clean":"2023-10-06T05:21:12.151269+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:21:12.151269+0000","last_undegraded":"2023-10-06T05:21:12.151269+0000","last_fullsized":"2023-10-06T05:21:12.151269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162394+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:54.473173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.166267+0000","last_change":"2023-10-06T05:21:13.166267+0000","last_active":"2023-10-06T05:21:13.166267+0000","last_peered":"2023-10-06T05:21:13.166267+0000","last_clean":"2023-10-06T05:21:13.166267+0000","last_became_active":"2023-10-06T05:21:13.165887+0000","last_became_peered":"2023-10-06T05:21:13.165887+0000","last_unstale":"2023-10-06T05:21:13.166267+0000","last_undegraded":"2023-10-06T05:21:13.166267+0000","last_fullsized":"2023-10-06T05:21:13.166267+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:56.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.151973+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:21:12.151973+0000","last_peered":"2023-10-06T05:21:12.151973+0000","last_clean":"2023-10-06T05:21:12.151973+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:21:12.151973+0000","last_undegraded":"2023-10-06T05:21:12.151973+0000","last_fullsized":"2023-10-06T05:21:12.151973+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":202,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.156005+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:21:12.156005+0000","last_peered":"2023-10-06T05:21:12.156005+0000","last_clean":"2023-10-06T05:21:12.156005+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:21:12.156005+0000","last_undegraded":"2023-10-06T05:21:12.156005+0000","last_fullsized":"2023-10-06T05:21:12.156005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157962+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:25.384243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.168485+0000","last_change":"2023-10-06T05:21:13.168485+0000","last_active":"2023-10-06T05:21:13.168485+0000","last_peered":"2023-10-06T05:21:13.168485+0000","last_clean":"2023-10-06T05:21:13.168485+0000","last_became_active":"2023-10-06T05:21:13.168162+0000","last_became_peered":"2023-10-06T05:21:13.168162+0000","last_unstale":"2023-10-06T05:21:13.168485+0000","last_undegraded":"2023-10-06T05:21:13.168485+0000","last_fullsized":"2023-10-06T05:21:13.168485+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.040745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.156129+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:21:12.156129+0000","last_peered":"2023-10-06T05:21:12.156129+0000","last_clean":"2023-10-06T05:21:12.156129+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:21:12.156129+0000","last_undegraded":"2023-10-06T05:21:12.156129+0000","last_fullsized":"2023-10-06T05:21:12.156129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158198+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:46.254829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171949+0000","last_change":"2023-10-06T05:21:13.171949+0000","last_active":"2023-10-06T05:21:13.171949+0000","last_peered":"2023-10-06T05:21:13.171949+0000","last_clean":"2023-10-06T05:21:13.171949+0000","last_became_active":"2023-10-06T05:21:13.171646+0000","last_became_peered":"2023-10-06T05:21:13.171646+0000","last_unstale":"2023-10-06T05:21:13.171949+0000","last_undegraded":"2023-10-06T05:21:13.171949+0000","last_fullsized":"2023-10-06T05:21:13.171949+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:00.022649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.156123+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:21:12.156123+0000","last_peered":"2023-10-06T05:21:12.156123+0000","last_clean":"2023-10-06T05:21:12.156123+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:21:12.156123+0000","last_undegraded":"2023-10-06T05:21:12.156123+0000","last_fullsized":"2023-10-06T05:21:12.156123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162802+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:20.503103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.168910+0000","last_change":"2023-10-06T05:21:13.168910+0000","last_active":"2023-10-06T05:21:13.168910+0000","last_peered":"2023-10-06T05:21:13.168910+0000","last_clean":"2023-10-06T05:21:13.168910+0000","last_became_active":"2023-10-06T05:21:13.168616+0000","last_became_peered":"2023-10-06T05:21:13.168616+0000","last_unstale":"2023-10-06T05:21:13.168910+0000","last_undegraded":"2023-10-06T05:21:13.168910+0000","last_fullsized":"2023-10-06T05:21:13.168910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:15.808879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.162010+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:21:12.162010+0000","last_peered":"2023-10-06T05:21:12.162010+0000","last_clean":"2023-10-06T05:21:12.162010+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:21:12.162010+0000","last_undegraded":"2023-10-06T05:21:12.162010+0000","last_fullsized":"2023-10-06T05:21:12.162010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156041+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:49.243354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.166892+0000","last_change":"2023-10-06T05:21:13.166892+0000","last_active":"2023-10-06T05:21:13.166892+0000","last_peered":"2023-10-06T05:21:13.166892+0000","last_clean":"2023-10-06T05:21:13.166892+0000","last_became_active":"2023-10-06T05:21:13.166496+0000","last_became_peered":"2023-10-06T05:21:13.166496+0000","last_unstale":"2023-10-06T05:21:13.166892+0000","last_undegraded":"2023-10-06T05:21:13.166892+0000","last_fullsized":"2023-10-06T05:21:13.166892+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:13.510131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.152394+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:21:12.152394+0000","last_peered":"2023-10-06T05:21:12.152394+0000","last_clean":"2023-10-06T05:21:12.152394+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:21:12.152394+0000","last_undegraded":"2023-10-06T05:21:12.152394+0000","last_fullsized":"2023-10-06T05:21:12.152394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157810+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:20.516682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.159342+0000","last_change":"2023-10-06T05:21:13.159342+0000","last_active":"2023-10-06T05:21:13.159342+0000","last_peered":"2023-10-06T05:21:13.159342+0000","last_clean":"2023-10-06T05:21:13.159342+0000","last_became_active":"2023-10-06T05:21:13.158741+0000","last_became_peered":"2023-10-06T05:21:13.158741+0000","last_unstale":"2023-10-06T05:21:13.159342+0000","last_undegraded":"2023-10-06T05:21:13.159342+0000","last_fullsized":"2023-10-06T05:21:13.159342+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:22.191827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":53,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.152293+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:21:12.152293+0000","last_peered":"2023-10-06T05:21:12.152293+0000","last_clean":"2023-10-06T05:21:12.152293+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:21:12.152293+0000","last_undegraded":"2023-10-06T05:21:12.152293+0000","last_fullsized":"2023-10-06T05:21:12.152293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298688,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073152,"statfs":{"total":96636764160,"available":96330907648,"internally_reserved":0,"allocated":1433600,"data_stored":962616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.88200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.68200000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93600000000000005,"5min":1.018,"15min":1.018},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95999999999999996,"5min":1.056,"15min":1.056},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73399999999999999,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67700000000000005,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84499999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93300000000000005,"5min":0.999,"15min":0.999},"last":0.755},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.83799999999999997},"min":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0389999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.626,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.85399999999999998},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0029999999999999,"5min":1.103,"15min":1.103},"last":0.90100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.47099999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89800000000000002,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.627},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90800000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78500000000000003,"5min":1.369,"15min":1.369},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.891,"15min":1.891},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89800000000000002,"5min":1.347,"15min":1.347},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":1,"15min":1},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73499999999999999,"15min":0.78500000000000003},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.95499999999999996,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.746},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78200000000000003,"15min":0.82399999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.82199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85899999999999999,"5min":1.415,"15min":1.415},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77500000000000002,"15min":0.83099999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98199999999999998,"5min":2,"15min":2},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.78000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95999999999999996,"5min":1.46,"15min":1.46},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298688,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073152,"statfs":{"total":96636764160,"available":96330907648,"internally_reserved":0,"allocated":1433600,"data_stored":962616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.378},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.75},{"interface":"front","average":{"1min":0.877,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.59699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82499999999999996},"min":{"1min":0.63600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.753},"min":{"1min":0.621,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.052,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.82999999999999996,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.83499999999999996,"15min":0.82499999999999996},"min":{"1min":0.67100000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.69299999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.087,"5min":1.103,"15min":1.103},"last":0.73099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68300000000000005,"5min":1.042,"15min":1.042},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.436},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1519999999999999,"5min":1.242,"15min":1.242},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":1.198,"15min":1.198},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.77300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1659999999999999,"5min":1.26,"15min":1.26},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.073,"5min":1.323,"15min":1.323},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.123,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.264,"15min":1.264},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.04,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.374}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46500000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.624,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.503},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68799999999999994,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93500000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90600000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96299999999999997,"5min":1.099,"15min":1.099},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78500000000000003,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.90000000000000002,"5min":1.163,"15min":1.163},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98799999999999999,"5min":1.21,"15min":1.21},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.022,"5min":1.042,"15min":1.042},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84399999999999997,"15min":0.872},"min":{"1min":0.64600000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0469999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.82399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.44600000000000001,"15min":0.46200000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57199999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.624,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.441},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.497},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72299999999999998,"5min":1.014,"15min":1.014},"last":0.503}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.145,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1830000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.03,"5min":1.204,"15min":1.204},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.329},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.82099999999999995},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.048,"5min":1.484,"15min":1.484},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83499999999999996,"15min":0.85399999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1930000000000001,"5min":1.629,"15min":1.629},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.83999999999999997,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1950000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.085,"5min":1.325,"15min":1.325},"last":0.61499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74399999999999999,"15min":0.79800000000000004},"min":{"1min":0.52100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":2.77,"15min":2.77},"last":0.92400000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.78700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.76600000000000001,"15min":0.82999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89600000000000002,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.1120000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.58599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.90500000000000003,"15min":1.091},"min":{"1min":0.50800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":7.742,"15min":7.742},"last":0.93999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.48999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96299999999999997,"5min":1.079,"15min":1.079},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90900000000000003,"5min":1.181,"15min":1.181},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.745},"min":{"1min":0.56499999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.60599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.92200000000000004,"5min":1.171,"15min":1.171},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.624,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96499999999999997,"5min":1.228,"15min":1.228},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81499999999999995,"15min":0.86899999999999999},"min":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97399999999999998,"5min":1.571,"15min":1.571},"last":0.73499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.83099999999999996,"15min":1},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.85499999999999998,"15min":1.0209999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.079,"5min":9.25,"15min":9.25},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.875,"15min":1.0620000000000001},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":10.384,"15min":10.384},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":1.0760000000000001},"min":{"1min":0.49299999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0349999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.78400000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":1.1240000000000001},"min":{"1min":0.54800000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.95199999999999996,"5min":10.205,"15min":10.205},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96399999999999997,"5min":1.292,"15min":1.292},"last":0.625}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.048,"5min":1.171,"15min":1.171},"last":0.83299999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78400000000000003,"15min":0.81699999999999995},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.298,"15min":1.298},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80300000000000005,"15min":0.82899999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98999999999999999,"5min":1.204,"15min":1.204},"last":0.877},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90200000000000002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.67300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:18.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:18.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.795+0000 7fc2ddbff700 1 Processor -- start 2023-10-06T05:21:18.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.795+0000 7fc2ddbff700 1 -- start start 2023-10-06T05:21:18.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.796+0000 7fc2ddbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c6f80 0x7fc2d8159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.796+0000 7fc2ddbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d8067f70 con 0x7fc2d81596d0 2023-10-06T05:21:18.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.796+0000 7fc2ddbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d806e040 con 0x7fc2d80c16e0 2023-10-06T05:21:18.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.796+0000 7fc2ddbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d806b040 con 0x7fc2d80c6f80 2023-10-06T05:21:18.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.796+0000 7fc2d77fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc2d81596d0 0x7fc2d806a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35590/0 (socket says 172.21.15.31:35590) 2023-10-06T05:21:18.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.796+0000 7fc2d77fe700 1 -- 172.21.15.31:0/608029230 learned_addr learned my addr 172.21.15.31:0/608029230 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:18.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.797+0000 7fc2d6ffd700 1 --2- 172.21.15.31:0/608029230 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c6f80 0x7fc2d8159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:18.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.797+0000 7fc2d67fc700 1 -- 172.21.15.31:0/608029230 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460603952 0 0) 0x7fc2d8067f70 con 0x7fc2d81596d0 2023-10-06T05:21:18.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.797+0000 7fc2d67fc700 1 -- 172.21.15.31:0/608029230 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2c0003570 con 0x7fc2d81596d0 2023-10-06T05:21:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.797+0000 7fc2d67fc700 1 -- 172.21.15.31:0/608029230 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222910334 0 0) 0x7fc2d806e040 con 0x7fc2d80c16e0 2023-10-06T05:21:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.797+0000 7fc2d67fc700 1 -- 172.21.15.31:0/608029230 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2d8067f70 con 0x7fc2d80c16e0 2023-10-06T05:21:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.797+0000 7fc2d6ffd700 1 -- 172.21.15.31:0/608029230 >> v1:172.21.15.62:6789/0 conn(0x7fc2d80c16e0 legacy=0x7fc2d806e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.797+0000 7fc2d6ffd700 1 -- 172.21.15.31:0/608029230 >> v1:172.21.15.31:6789/0 conn(0x7fc2d81596d0 legacy=0x7fc2d806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.797+0000 7fc2d6ffd700 1 -- 172.21.15.31:0/608029230 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d80681c0 con 0x7fc2d80c6f80 2023-10-06T05:21:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.797+0000 7fc2d6ffd700 1 --2- 172.21.15.31:0/608029230 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c6f80 0x7fc2d8159aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009910 tx=0x7fc2cc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ced2820381a730 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.798+0000 7fc2d67fc700 1 -- 172.21.15.31:0/608029230 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc011630 con 0x7fc2d80c6f80 2023-10-06T05:21:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.798+0000 7fc2d67fc700 1 -- 172.21.15.31:0/608029230 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2cc011790 con 0x7fc2d80c6f80 2023-10-06T05:21:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.798+0000 7fc2d67fc700 1 -- 172.21.15.31:0/608029230 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc011b30 con 0x7fc2d80c6f80 2023-10-06T05:21:18.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.798+0000 7fc2ddbff700 1 -- 172.21.15.31:0/608029230 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c6f80 msgr2=0x7fc2d8159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.798+0000 7fc2ddbff700 1 --2- 172.21.15.31:0/608029230 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c6f80 0x7fc2d8159aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009910 tx=0x7fc2cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.798+0000 7fc2ddbff700 1 -- 172.21.15.31:0/608029230 shutdown_connections 2023-10-06T05:21:18.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.798+0000 7fc2ddbff700 1 --2- 172.21.15.31:0/608029230 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c6f80 0x7fc2d8159aa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.798+0000 7fc2ddbff700 1 -- 172.21.15.31:0/608029230 >> 172.21.15.31:0/608029230 conn(0x7fc2d80c9e80 msgr2=0x7fc2d806a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:18.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.798+0000 7fc2ddbff700 1 -- 172.21.15.31:0/608029230 shutdown_connections 2023-10-06T05:21:18.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.798+0000 7fc2ddbff700 1 -- 172.21.15.31:0/608029230 wait complete. 2023-10-06T05:21:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.800+0000 7fc2ddbff700 1 Processor -- start 2023-10-06T05:21:18.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.800+0000 7fc2ddbff700 1 -- start start 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2ddbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c16e0 0x7fc2d8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2ddbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2d80c6f80 0x7fc2d8146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2ddbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2d81596d0 0x7fc2d8147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2ddbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d816ce80 con 0x7fc2d80c6f80 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2ddbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d81479a0 con 0x7fc2d81596d0 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2ddbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80b0740 con 0x7fc2d80c16e0 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2d80c6f80 0x7fc2d8146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2d80c6f80 0x7fc2d8146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60800/0 (socket says 172.21.15.31:60800) 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2d6ffd700 1 -- 172.21.15.31:0/939953731 learned_addr learned my addr 172.21.15.31:0/939953731 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2d77fe700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c16e0 0x7fc2d8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:18.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.801+0000 7fc2d7fff700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2d81596d0 0x7fc2d8147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2d6ffd700 1 -- 172.21.15.31:0/939953731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c16e0 msgr2=0x7fc2d8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2d6ffd700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c16e0 0x7fc2d8159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2d6ffd700 1 -- 172.21.15.31:0/939953731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2d81596d0 msgr2=0x7fc2d8147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2d6ffd700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2d81596d0 0x7fc2d8147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2d6ffd700 1 -- 172.21.15.31:0/939953731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc009420 con 0x7fc2d80c6f80 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2d7fff700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2d81596d0 0x7fc2d8147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2d6ffd700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2d80c6f80 0x7fc2d8146f20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0091e0 tx=0x7fc2cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/939953731 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc01f070 con 0x7fc2d80c6f80 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/939953731 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2cc000bc0 con 0x7fc2d80c6f80 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/939953731 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc025420 con 0x7fc2d80c6f80 2023-10-06T05:21:18.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2ddbff700 1 -- 172.21.15.31:0/939953731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d815ca80 con 0x7fc2d80c6f80 2023-10-06T05:21:18.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.802+0000 7fc2ddbff700 1 -- 172.21.15.31:0/939953731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d80b0990 con 0x7fc2d80c6f80 2023-10-06T05:21:18.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.803+0000 7fc2ddbff700 1 -- 172.21.15.31:0/939953731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d815ce70 con 0x7fc2d80c6f80 2023-10-06T05:21:18.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.804+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/939953731 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fc2cc023070 con 0x7fc2d80c6f80 2023-10-06T05:21:18.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.805+0000 7fc2bf7fe700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc2c4067290 0x7fc2c4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:18.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.805+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/939953731 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc2cc086010 con 0x7fc2d80c6f80 2023-10-06T05:21:18.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.805+0000 7fc2d77fe700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc2c4067290 0x7fc2c4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:18.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.806+0000 7fc2d77fe700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc2c4067290 0x7fc2c4069740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8007fd0 tx=0x7fc2c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:18.809+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/939953731 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2cc052cd0 con 0x7fc2d80c6f80 2023-10-06T05:21:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.007+0000 7fc2ddbff700 1 -- 172.21.15.31:0/939953731 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2d80b0b80 con 0x7fc2c4067290 2023-10-06T05:21:19.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.016+0000 7fc2bf7fe700 1 -- 172.21.15.31:0/939953731 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308529 (secure 0 0 0) 0x7fc2d80b0b80 con 0x7fc2c4067290 2023-10-06T05:21:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 -- 172.21.15.31:0/939953731 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc2c4067290 msgr2=0x7fc2c4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc2c4067290 0x7fc2c4069740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8007fd0 tx=0x7fc2c8009040 comp rx=0 tx=0).stop 2023-10-06T05:21:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 -- 172.21.15.31:0/939953731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2d80c6f80 msgr2=0x7fc2d8146f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2d80c6f80 0x7fc2d8146f20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0091e0 tx=0x7fc2cc017000 comp rx=0 tx=0).stop 2023-10-06T05:21:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 -- 172.21.15.31:0/939953731 shutdown_connections 2023-10-06T05:21:19.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2d81596d0 0x7fc2d8147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2d80c16e0 0x7fc2d8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc2c4067290 0x7fc2c4069740 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 --2- 172.21.15.31:0/939953731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2d80c6f80 0x7fc2d8146f20 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 -- 172.21.15.31:0/939953731 >> 172.21.15.31:0/939953731 conn(0x7fc2d80c9e80 msgr2=0x7fc2d80c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:19.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 -- 172.21.15.31:0/939953731 shutdown_connections 2023-10-06T05:21:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.021+0000 7fc2ddbff700 1 -- 172.21.15.31:0/939953731 wait complete. 2023-10-06T05:21:19.030 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:21:19.040 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2023-10-06T05:21:18.118385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386024,"kb_used_data":7720,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752588696,"statfs":{"total":773094113280,"available":770650824704,"internally_reserved":0,"allocated":7905280,"data_stored":4158720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.969865"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157298+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:40.622847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162888+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:39.527935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156908+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:37.837314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.160141+0000","last_change":"2023-10-06T05:21:14.160278+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.160141+0000","last_undegraded":"2023-10-06T05:21:14.160141+0000","last_fullsized":"2023-10-06T05:21:14.160141+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:57.160327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.159070+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:16.114354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155121+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:43.798148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158115+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:07.141522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157896+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:00.913829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154871+0000","last_change":"2023-10-06T05:21:14.155317+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154871+0000","last_undegraded":"2023-10-06T05:21:14.154871+0000","last_fullsized":"2023-10-06T05:21:14.154871+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:04.649668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159817+0000","last_change":"2023-10-06T05:21:14.159956+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159817+0000","last_undegraded":"2023-10-06T05:21:14.159817+0000","last_fullsized":"2023-10-06T05:21:14.159817+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:55.793132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.164442+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:21.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159761+0000","last_change":"2023-10-06T05:21:14.159896+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159761+0000","last_undegraded":"2023-10-06T05:21:14.159761+0000","last_fullsized":"2023-10-06T05:21:14.159761+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:16.811458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156981+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:43.232828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.154665+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:20.503103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162708+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:33.730974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156775+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:04.855790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158524+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:50.744732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157660+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:20.672647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.152314+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:25.760034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157280+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.928620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154926+0000","last_change":"2023-10-06T05:21:14.155091+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154926+0000","last_undegraded":"2023-10-06T05:21:14.154926+0000","last_fullsized":"2023-10-06T05:21:14.154926+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:43.694140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158340+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:44.399326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154629+0000","last_change":"2023-10-06T05:21:14.155278+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154629+0000","last_undegraded":"2023-10-06T05:21:14.154629+0000","last_fullsized":"2023-10-06T05:21:14.154629+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:48.387423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.160885+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:29.781374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.163961+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:49.243354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159370+0000","last_change":"2023-10-06T05:21:14.159548+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159370+0000","last_undegraded":"2023-10-06T05:21:14.159370+0000","last_fullsized":"2023-10-06T05:21:14.159370+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:50.021472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158766+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.976026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156773+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:11.364687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.165301+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:40.622847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157061+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:26.471873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155647+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:22.094091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156138+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:21.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155381+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:45.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.159241+0000","last_change":"2023-10-06T05:21:13.159241+0000","last_active":"2023-10-06T05:21:13.159241+0000","last_peered":"2023-10-06T05:21:13.159241+0000","last_clean":"2023-10-06T05:21:13.159241+0000","last_became_active":"2023-10-06T05:21:13.158913+0000","last_became_peered":"2023-10-06T05:21:13.158913+0000","last_unstale":"2023-10-06T05:21:13.159241+0000","last_undegraded":"2023-10-06T05:21:13.159241+0000","last_fullsized":"2023-10-06T05:21:13.159241+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:58.015087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.161893+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:12.975931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171676+0000","last_change":"2023-10-06T05:21:13.171676+0000","last_active":"2023-10-06T05:21:13.171676+0000","last_peered":"2023-10-06T05:21:13.171676+0000","last_clean":"2023-10-06T05:21:13.171676+0000","last_became_active":"2023-10-06T05:21:13.171360+0000","last_became_peered":"2023-10-06T05:21:13.171360+0000","last_unstale":"2023-10-06T05:21:13.171676+0000","last_undegraded":"2023-10-06T05:21:13.171676+0000","last_fullsized":"2023-10-06T05:21:13.171676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:24.757454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154702+0000","last_change":"2023-10-06T05:21:14.155168+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154702+0000","last_undegraded":"2023-10-06T05:21:14.154702+0000","last_fullsized":"2023-10-06T05:21:14.154702+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:48.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.176256+0000","last_change":"2023-10-06T05:21:13.176256+0000","last_active":"2023-10-06T05:21:13.176256+0000","last_peered":"2023-10-06T05:21:13.176256+0000","last_clean":"2023-10-06T05:21:13.176256+0000","last_became_active":"2023-10-06T05:21:13.175487+0000","last_became_peered":"2023-10-06T05:21:13.175487+0000","last_unstale":"2023-10-06T05:21:13.176256+0000","last_undegraded":"2023-10-06T05:21:13.176256+0000","last_fullsized":"2023-10-06T05:21:13.176256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:19.500524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154776+0000","last_change":"2023-10-06T05:21:14.154975+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154776+0000","last_undegraded":"2023-10-06T05:21:14.154776+0000","last_fullsized":"2023-10-06T05:21:14.154776+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:09.170341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.169221+0000","last_change":"2023-10-06T05:21:13.169221+0000","last_active":"2023-10-06T05:21:13.169221+0000","last_peered":"2023-10-06T05:21:13.169221+0000","last_clean":"2023-10-06T05:21:13.169221+0000","last_became_active":"2023-10-06T05:21:13.168853+0000","last_became_peered":"2023-10-06T05:21:13.168853+0000","last_unstale":"2023-10-06T05:21:13.169221+0000","last_undegraded":"2023-10-06T05:21:13.169221+0000","last_fullsized":"2023-10-06T05:21:13.169221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:12.290741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.163333+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:43.798148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.167639+0000","last_change":"2023-10-06T05:21:13.162613+0000","last_active":"2023-10-06T05:21:13.167639+0000","last_peered":"2023-10-06T05:21:13.167639+0000","last_clean":"2023-10-06T05:21:13.167639+0000","last_became_active":"2023-10-06T05:21:13.162167+0000","last_became_peered":"2023-10-06T05:21:13.162167+0000","last_unstale":"2023-10-06T05:21:13.167639+0000","last_undegraded":"2023-10-06T05:21:13.167639+0000","last_fullsized":"2023-10-06T05:21:13.167639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:36.834735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.164889+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:26.471873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.175784+0000","last_change":"2023-10-06T05:21:13.170589+0000","last_active":"2023-10-06T05:21:13.175784+0000","last_peered":"2023-10-06T05:21:13.175784+0000","last_clean":"2023-10-06T05:21:13.175784+0000","last_became_active":"2023-10-06T05:21:13.170282+0000","last_became_peered":"2023-10-06T05:21:13.170282+0000","last_unstale":"2023-10-06T05:21:13.175784+0000","last_undegraded":"2023-10-06T05:21:13.175784+0000","last_fullsized":"2023-10-06T05:21:13.175784+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:53.541975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.155752+0000","last_change":"2023-10-06T05:21:14.155916+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.155752+0000","last_undegraded":"2023-10-06T05:21:14.155752+0000","last_fullsized":"2023-10-06T05:21:14.155752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:54.544554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.175791+0000","last_change":"2023-10-06T05:21:13.175791+0000","last_active":"2023-10-06T05:21:13.175791+0000","last_peered":"2023-10-06T05:21:13.175791+0000","last_clean":"2023-10-06T05:21:13.175791+0000","last_became_active":"2023-10-06T05:21:13.175326+0000","last_became_peered":"2023-10-06T05:21:13.175326+0000","last_unstale":"2023-10-06T05:21:13.175791+0000","last_undegraded":"2023-10-06T05:21:13.175791+0000","last_fullsized":"2023-10-06T05:21:13.175791+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.040745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158575+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:56.726499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.170180+0000","last_change":"2023-10-06T05:21:13.170180+0000","last_active":"2023-10-06T05:21:13.170180+0000","last_peered":"2023-10-06T05:21:13.170180+0000","last_clean":"2023-10-06T05:21:13.170180+0000","last_became_active":"2023-10-06T05:21:13.169857+0000","last_became_peered":"2023-10-06T05:21:13.169857+0000","last_unstale":"2023-10-06T05:21:13.170180+0000","last_undegraded":"2023-10-06T05:21:13.170180+0000","last_fullsized":"2023-10-06T05:21:13.170180+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:45.593601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.161768+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:25.760034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.172453+0000","last_change":"2023-10-06T05:21:13.172453+0000","last_active":"2023-10-06T05:21:13.172453+0000","last_peered":"2023-10-06T05:21:13.172453+0000","last_clean":"2023-10-06T05:21:13.172453+0000","last_became_active":"2023-10-06T05:21:13.171973+0000","last_became_peered":"2023-10-06T05:21:13.171973+0000","last_unstale":"2023-10-06T05:21:13.172453+0000","last_undegraded":"2023-10-06T05:21:13.172453+0000","last_fullsized":"2023-10-06T05:21:13.172453+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:53.470594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156446+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:48.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.170853+0000","last_change":"2023-10-06T05:21:13.168940+0000","last_active":"2023-10-06T05:21:13.170853+0000","last_peered":"2023-10-06T05:21:13.170853+0000","last_clean":"2023-10-06T05:21:13.170853+0000","last_became_active":"2023-10-06T05:21:13.168641+0000","last_became_peered":"2023-10-06T05:21:13.168641+0000","last_unstale":"2023-10-06T05:21:13.170853+0000","last_undegraded":"2023-10-06T05:21:13.170853+0000","last_fullsized":"2023-10-06T05:21:13.170853+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:42.691561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.154172+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:54.473173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.163900+0000","last_change":"2023-10-06T05:21:13.163900+0000","last_active":"2023-10-06T05:21:13.163900+0000","last_peered":"2023-10-06T05:21:13.163900+0000","last_clean":"2023-10-06T05:21:13.163900+0000","last_became_active":"2023-10-06T05:21:13.163604+0000","last_became_peered":"2023-10-06T05:21:13.163604+0000","last_unstale":"2023-10-06T05:21:13.163900+0000","last_undegraded":"2023-10-06T05:21:13.163900+0000","last_fullsized":"2023-10-06T05:21:13.163900+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:55.585059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155983+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:25.096218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.170791+0000","last_change":"2023-10-06T05:21:13.170791+0000","last_active":"2023-10-06T05:21:13.170791+0000","last_peered":"2023-10-06T05:21:13.170791+0000","last_clean":"2023-10-06T05:21:13.170791+0000","last_became_active":"2023-10-06T05:21:13.170470+0000","last_became_peered":"2023-10-06T05:21:13.170470+0000","last_unstale":"2023-10-06T05:21:13.170791+0000","last_undegraded":"2023-10-06T05:21:13.170791+0000","last_fullsized":"2023-10-06T05:21:13.170791+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:26.418570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159317+0000","last_change":"2023-10-06T05:21:14.159488+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159317+0000","last_undegraded":"2023-10-06T05:21:14.159317+0000","last_fullsized":"2023-10-06T05:21:14.159317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:45.462119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.175866+0000","last_change":"2023-10-06T05:21:13.171211+0000","last_active":"2023-10-06T05:21:13.175866+0000","last_peered":"2023-10-06T05:21:13.175866+0000","last_clean":"2023-10-06T05:21:13.175866+0000","last_became_active":"2023-10-06T05:21:13.170917+0000","last_became_peered":"2023-10-06T05:21:13.170917+0000","last_unstale":"2023-10-06T05:21:13.175866+0000","last_undegraded":"2023-10-06T05:21:13.175866+0000","last_fullsized":"2023-10-06T05:21:13.175866+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:08.167762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156475+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:56.735464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171789+0000","last_change":"2023-10-06T05:21:13.171789+0000","last_active":"2023-10-06T05:21:13.171789+0000","last_peered":"2023-10-06T05:21:13.171789+0000","last_clean":"2023-10-06T05:21:13.171789+0000","last_became_active":"2023-10-06T05:21:13.171451+0000","last_became_peered":"2023-10-06T05:21:13.171451+0000","last_unstale":"2023-10-06T05:21:13.171789+0000","last_undegraded":"2023-10-06T05:21:13.171789+0000","last_fullsized":"2023-10-06T05:21:13.171789+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:11.973352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157028+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:02.476152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171171+0000","last_change":"2023-10-06T05:21:13.171171+0000","last_active":"2023-10-06T05:21:13.171171+0000","last_peered":"2023-10-06T05:21:13.171171+0000","last_clean":"2023-10-06T05:21:13.171171+0000","last_became_active":"2023-10-06T05:21:13.170856+0000","last_became_peered":"2023-10-06T05:21:13.170856+0000","last_unstale":"2023-10-06T05:21:13.171171+0000","last_undegraded":"2023-10-06T05:21:13.171171+0000","last_fullsized":"2023-10-06T05:21:13.171171+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:45.319480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157405+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:08.922350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.176681+0000","last_change":"2023-10-06T05:21:13.176681+0000","last_active":"2023-10-06T05:21:13.176681+0000","last_peered":"2023-10-06T05:21:13.176681+0000","last_clean":"2023-10-06T05:21:13.176681+0000","last_became_active":"2023-10-06T05:21:13.176432+0000","last_became_peered":"2023-10-06T05:21:13.176432+0000","last_unstale":"2023-10-06T05:21:13.176681+0000","last_undegraded":"2023-10-06T05:21:13.176681+0000","last_fullsized":"2023-10-06T05:21:13.176681+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:15.808879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154629+0000","last_change":"2023-10-06T05:21:14.154830+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154629+0000","last_undegraded":"2023-10-06T05:21:14.154629+0000","last_fullsized":"2023-10-06T05:21:14.154629+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:06.020289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.169145+0000","last_change":"2023-10-06T05:21:13.165877+0000","last_active":"2023-10-06T05:21:13.169145+0000","last_peered":"2023-10-06T05:21:13.169145+0000","last_clean":"2023-10-06T05:21:13.169145+0000","last_became_active":"2023-10-06T05:21:13.165605+0000","last_became_peered":"2023-10-06T05:21:13.165605+0000","last_unstale":"2023-10-06T05:21:13.169145+0000","last_undegraded":"2023-10-06T05:21:13.169145+0000","last_fullsized":"2023-10-06T05:21:13.169145+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157597+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:26.639167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171761+0000","last_change":"2023-10-06T05:21:13.171761+0000","last_active":"2023-10-06T05:21:13.171761+0000","last_peered":"2023-10-06T05:21:13.171761+0000","last_clean":"2023-10-06T05:21:13.171761+0000","last_became_active":"2023-10-06T05:21:13.171491+0000","last_became_peered":"2023-10-06T05:21:13.171491+0000","last_unstale":"2023-10-06T05:21:13.171761+0000","last_undegraded":"2023-10-06T05:21:13.171761+0000","last_fullsized":"2023-10-06T05:21:13.171761+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:56.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.155136+0000","last_change":"2023-10-06T05:21:14.155680+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.155136+0000","last_undegraded":"2023-10-06T05:21:14.155136+0000","last_fullsized":"2023-10-06T05:21:14.155136+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:04.797474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.167952+0000","last_change":"2023-10-06T05:21:13.167952+0000","last_active":"2023-10-06T05:21:13.167952+0000","last_peered":"2023-10-06T05:21:13.167952+0000","last_clean":"2023-10-06T05:21:13.167952+0000","last_became_active":"2023-10-06T05:21:13.166565+0000","last_became_peered":"2023-10-06T05:21:13.166565+0000","last_unstale":"2023-10-06T05:21:13.167952+0000","last_undegraded":"2023-10-06T05:21:13.167952+0000","last_fullsized":"2023-10-06T05:21:13.167952+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:38.525356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.164809+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:04.855790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171867+0000","last_change":"2023-10-06T05:21:13.171867+0000","last_active":"2023-10-06T05:21:13.171867+0000","last_peered":"2023-10-06T05:21:13.171867+0000","last_clean":"2023-10-06T05:21:13.171867+0000","last_became_active":"2023-10-06T05:21:13.171530+0000","last_became_peered":"2023-10-06T05:21:13.171530+0000","last_unstale":"2023-10-06T05:21:13.171867+0000","last_undegraded":"2023-10-06T05:21:13.171867+0000","last_fullsized":"2023-10-06T05:21:13.171867+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:28.778795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.159646+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:15.167368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.172743+0000","last_change":"2023-10-06T05:21:13.172743+0000","last_active":"2023-10-06T05:21:13.172743+0000","last_peered":"2023-10-06T05:21:13.172743+0000","last_clean":"2023-10-06T05:21:13.172743+0000","last_became_active":"2023-10-06T05:21:13.172453+0000","last_became_peered":"2023-10-06T05:21:13.172453+0000","last_unstale":"2023-10-06T05:21:13.172743+0000","last_undegraded":"2023-10-06T05:21:13.172743+0000","last_fullsized":"2023-10-06T05:21:13.172743+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:01.292501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155880+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:35.471644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171851+0000","last_change":"2023-10-06T05:21:13.171851+0000","last_active":"2023-10-06T05:21:13.171851+0000","last_peered":"2023-10-06T05:21:13.171851+0000","last_clean":"2023-10-06T05:21:13.171851+0000","last_became_active":"2023-10-06T05:21:13.171552+0000","last_became_peered":"2023-10-06T05:21:13.171552+0000","last_unstale":"2023-10-06T05:21:13.171851+0000","last_undegraded":"2023-10-06T05:21:13.171851+0000","last_fullsized":"2023-10-06T05:21:13.171851+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:14:14.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157183+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:21.116415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.172335+0000","last_change":"2023-10-06T05:21:13.172335+0000","last_active":"2023-10-06T05:21:13.172335+0000","last_peered":"2023-10-06T05:21:13.172335+0000","last_clean":"2023-10-06T05:21:13.172335+0000","last_became_active":"2023-10-06T05:21:13.172046+0000","last_became_peered":"2023-10-06T05:21:13.172046+0000","last_unstale":"2023-10-06T05:21:13.172335+0000","last_undegraded":"2023-10-06T05:21:13.172335+0000","last_fullsized":"2023-10-06T05:21:13.172335+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:49.234603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.154688+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:22.127378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.174008+0000","last_change":"2023-10-06T05:21:13.174008+0000","last_active":"2023-10-06T05:21:13.174008+0000","last_peered":"2023-10-06T05:21:13.174008+0000","last_clean":"2023-10-06T05:21:13.174008+0000","last_became_active":"2023-10-06T05:21:13.173503+0000","last_became_peered":"2023-10-06T05:21:13.173503+0000","last_unstale":"2023-10-06T05:21:13.174008+0000","last_undegraded":"2023-10-06T05:21:13.174008+0000","last_fullsized":"2023-10-06T05:21:13.174008+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:14.164789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.152598+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:12.975931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.165946+0000","last_change":"2023-10-06T05:21:13.165946+0000","last_active":"2023-10-06T05:21:13.165946+0000","last_peered":"2023-10-06T05:21:13.165946+0000","last_clean":"2023-10-06T05:21:13.165946+0000","last_became_active":"2023-10-06T05:21:13.165677+0000","last_became_peered":"2023-10-06T05:21:13.165677+0000","last_unstale":"2023-10-06T05:21:13.165946+0000","last_undegraded":"2023-10-06T05:21:13.165946+0000","last_fullsized":"2023-10-06T05:21:13.165946+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:32.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.160560+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:21:13.160560+0000","last_peered":"2023-10-06T05:21:13.160560+0000","last_clean":"2023-10-06T05:21:13.160560+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:21:13.160560+0000","last_undegraded":"2023-10-06T05:21:13.160560+0000","last_fullsized":"2023-10-06T05:21:13.160560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156102+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:36.053585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.169590+0000","last_change":"2023-10-06T05:21:13.165590+0000","last_active":"2023-10-06T05:21:13.169590+0000","last_peered":"2023-10-06T05:21:13.169590+0000","last_clean":"2023-10-06T05:21:13.169590+0000","last_became_active":"2023-10-06T05:21:13.165314+0000","last_became_peered":"2023-10-06T05:21:13.165314+0000","last_unstale":"2023-10-06T05:21:13.169590+0000","last_undegraded":"2023-10-06T05:21:13.169590+0000","last_fullsized":"2023-10-06T05:21:13.169590+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:24.093639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.151269+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:21:12.151269+0000","last_peered":"2023-10-06T05:21:12.151269+0000","last_clean":"2023-10-06T05:21:12.151269+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:21:12.151269+0000","last_undegraded":"2023-10-06T05:21:12.151269+0000","last_fullsized":"2023-10-06T05:21:12.151269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162394+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:54.473173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.166267+0000","last_change":"2023-10-06T05:21:13.166267+0000","last_active":"2023-10-06T05:21:13.166267+0000","last_peered":"2023-10-06T05:21:13.166267+0000","last_clean":"2023-10-06T05:21:13.166267+0000","last_became_active":"2023-10-06T05:21:13.165887+0000","last_became_peered":"2023-10-06T05:21:13.165887+0000","last_unstale":"2023-10-06T05:21:13.166267+0000","last_undegraded":"2023-10-06T05:21:13.166267+0000","last_fullsized":"2023-10-06T05:21:13.166267+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:56.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.151973+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:21:12.151973+0000","last_peered":"2023-10-06T05:21:12.151973+0000","last_clean":"2023-10-06T05:21:12.151973+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:21:12.151973+0000","last_undegraded":"2023-10-06T05:21:12.151973+0000","last_fullsized":"2023-10-06T05:21:12.151973+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":202,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.156005+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:21:12.156005+0000","last_peered":"2023-10-06T05:21:12.156005+0000","last_clean":"2023-10-06T05:21:12.156005+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:21:12.156005+0000","last_undegraded":"2023-10-06T05:21:12.156005+0000","last_fullsized":"2023-10-06T05:21:12.156005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157962+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:25.384243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.168485+0000","last_change":"2023-10-06T05:21:13.168485+0000","last_active":"2023-10-06T05:21:13.168485+0000","last_peered":"2023-10-06T05:21:13.168485+0000","last_clean":"2023-10-06T05:21:13.168485+0000","last_became_active":"2023-10-06T05:21:13.168162+0000","last_became_peered":"2023-10-06T05:21:13.168162+0000","last_unstale":"2023-10-06T05:21:13.168485+0000","last_undegraded":"2023-10-06T05:21:13.168485+0000","last_fullsized":"2023-10-06T05:21:13.168485+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.040745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.156129+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:21:12.156129+0000","last_peered":"2023-10-06T05:21:12.156129+0000","last_clean":"2023-10-06T05:21:12.156129+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:21:12.156129+0000","last_undegraded":"2023-10-06T05:21:12.156129+0000","last_fullsized":"2023-10-06T05:21:12.156129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158198+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:46.254829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171949+0000","last_change":"2023-10-06T05:21:13.171949+0000","last_active":"2023-10-06T05:21:13.171949+0000","last_peered":"2023-10-06T05:21:13.171949+0000","last_clean":"2023-10-06T05:21:13.171949+0000","last_became_active":"2023-10-06T05:21:13.171646+0000","last_became_peered":"2023-10-06T05:21:13.171646+0000","last_unstale":"2023-10-06T05:21:13.171949+0000","last_undegraded":"2023-10-06T05:21:13.171949+0000","last_fullsized":"2023-10-06T05:21:13.171949+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:00.022649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.156123+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:21:12.156123+0000","last_peered":"2023-10-06T05:21:12.156123+0000","last_clean":"2023-10-06T05:21:12.156123+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:21:12.156123+0000","last_undegraded":"2023-10-06T05:21:12.156123+0000","last_fullsized":"2023-10-06T05:21:12.156123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162802+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:20.503103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.168910+0000","last_change":"2023-10-06T05:21:13.168910+0000","last_active":"2023-10-06T05:21:13.168910+0000","last_peered":"2023-10-06T05:21:13.168910+0000","last_clean":"2023-10-06T05:21:13.168910+0000","last_became_active":"2023-10-06T05:21:13.168616+0000","last_became_peered":"2023-10-06T05:21:13.168616+0000","last_unstale":"2023-10-06T05:21:13.168910+0000","last_undegraded":"2023-10-06T05:21:13.168910+0000","last_fullsized":"2023-10-06T05:21:13.168910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:15.808879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.162010+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:21:12.162010+0000","last_peered":"2023-10-06T05:21:12.162010+0000","last_clean":"2023-10-06T05:21:12.162010+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:21:12.162010+0000","last_undegraded":"2023-10-06T05:21:12.162010+0000","last_fullsized":"2023-10-06T05:21:12.162010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156041+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:49.243354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.166892+0000","last_change":"2023-10-06T05:21:13.166892+0000","last_active":"2023-10-06T05:21:13.166892+0000","last_peered":"2023-10-06T05:21:13.166892+0000","last_clean":"2023-10-06T05:21:13.166892+0000","last_became_active":"2023-10-06T05:21:13.166496+0000","last_became_peered":"2023-10-06T05:21:13.166496+0000","last_unstale":"2023-10-06T05:21:13.166892+0000","last_undegraded":"2023-10-06T05:21:13.166892+0000","last_fullsized":"2023-10-06T05:21:13.166892+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:13.510131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.152394+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:21:12.152394+0000","last_peered":"2023-10-06T05:21:12.152394+0000","last_clean":"2023-10-06T05:21:12.152394+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:21:12.152394+0000","last_undegraded":"2023-10-06T05:21:12.152394+0000","last_fullsized":"2023-10-06T05:21:12.152394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157810+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:20.516682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.159342+0000","last_change":"2023-10-06T05:21:13.159342+0000","last_active":"2023-10-06T05:21:13.159342+0000","last_peered":"2023-10-06T05:21:13.159342+0000","last_clean":"2023-10-06T05:21:13.159342+0000","last_became_active":"2023-10-06T05:21:13.158741+0000","last_became_peered":"2023-10-06T05:21:13.158741+0000","last_unstale":"2023-10-06T05:21:13.159342+0000","last_undegraded":"2023-10-06T05:21:13.159342+0000","last_fullsized":"2023-10-06T05:21:13.159342+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:22.191827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":53,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.152293+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:21:12.152293+0000","last_peered":"2023-10-06T05:21:12.152293+0000","last_clean":"2023-10-06T05:21:12.152293+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:21:12.152293+0000","last_undegraded":"2023-10-06T05:21:12.152293+0000","last_fullsized":"2023-10-06T05:21:12.152293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298688,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073152,"statfs":{"total":96636764160,"available":96330907648,"internally_reserved":0,"allocated":1433600,"data_stored":962616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.88200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.68200000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93600000000000005,"5min":1.018,"15min":1.018},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95999999999999996,"5min":1.056,"15min":1.056},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73399999999999999,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67700000000000005,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84499999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93300000000000005,"5min":0.999,"15min":0.999},"last":0.755},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.83799999999999997},"min":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0389999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.626,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.85399999999999998},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0029999999999999,"5min":1.103,"15min":1.103},"last":0.90100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.47099999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89800000000000002,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.627},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90800000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78500000000000003,"5min":1.369,"15min":1.369},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.891,"15min":1.891},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89800000000000002,"5min":1.347,"15min":1.347},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":1,"15min":1},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73499999999999999,"15min":0.78500000000000003},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.95499999999999996,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.746},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78200000000000003,"15min":0.82399999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.82199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85899999999999999,"5min":1.415,"15min":1.415},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77500000000000002,"15min":0.83099999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98199999999999998,"5min":2,"15min":2},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.78000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95999999999999996,"5min":1.46,"15min":1.46},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298688,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073152,"statfs":{"total":96636764160,"available":96330907648,"internally_reserved":0,"allocated":1433600,"data_stored":962616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.378},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.75},{"interface":"front","average":{"1min":0.877,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.59699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82499999999999996},"min":{"1min":0.63600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.753},"min":{"1min":0.621,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.052,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.82999999999999996,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.83499999999999996,"15min":0.82499999999999996},"min":{"1min":0.67100000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.69299999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.087,"5min":1.103,"15min":1.103},"last":0.73099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68300000000000005,"5min":1.042,"15min":1.042},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.436},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1519999999999999,"5min":1.242,"15min":1.242},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":1.198,"15min":1.198},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.77300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1659999999999999,"5min":1.26,"15min":1.26},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.073,"5min":1.323,"15min":1.323},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.123,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.264,"15min":1.264},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.04,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.374}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46500000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.624,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.503},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68799999999999994,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93500000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90600000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96299999999999997,"5min":1.099,"15min":1.099},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78500000000000003,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.90000000000000002,"5min":1.163,"15min":1.163},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98799999999999999,"5min":1.21,"15min":1.21},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.022,"5min":1.042,"15min":1.042},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84399999999999997,"15min":0.872},"min":{"1min":0.64600000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0469999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.82399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.44600000000000001,"15min":0.46200000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57199999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.624,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.441},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.497},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72299999999999998,"5min":1.014,"15min":1.014},"last":0.503}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.145,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1830000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.03,"5min":1.204,"15min":1.204},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.329},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.82099999999999995},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.048,"5min":1.484,"15min":1.484},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83499999999999996,"15min":0.85399999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1930000000000001,"5min":1.629,"15min":1.629},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.83999999999999997,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1950000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.085,"5min":1.325,"15min":1.325},"last":0.61499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74399999999999999,"15min":0.79800000000000004},"min":{"1min":0.52100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":2.77,"15min":2.77},"last":0.92400000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.78700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.76600000000000001,"15min":0.82999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89600000000000002,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.1120000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.58599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.90500000000000003,"15min":1.091},"min":{"1min":0.50800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":7.742,"15min":7.742},"last":0.93999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.48999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96299999999999997,"5min":1.079,"15min":1.079},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90900000000000003,"5min":1.181,"15min":1.181},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.745},"min":{"1min":0.56499999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.60599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.92200000000000004,"5min":1.171,"15min":1.171},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.624,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96499999999999997,"5min":1.228,"15min":1.228},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81499999999999995,"15min":0.86899999999999999},"min":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97399999999999998,"5min":1.571,"15min":1.571},"last":0.73499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.83099999999999996,"15min":1},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.85499999999999998,"15min":1.0209999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.079,"5min":9.25,"15min":9.25},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.875,"15min":1.0620000000000001},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":10.384,"15min":10.384},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":1.0760000000000001},"min":{"1min":0.49299999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0349999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.78400000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":1.1240000000000001},"min":{"1min":0.54800000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.95199999999999996,"5min":10.205,"15min":10.205},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96399999999999997,"5min":1.292,"15min":1.292},"last":0.625}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.048,"5min":1.171,"15min":1.171},"last":0.83299999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78400000000000003,"15min":0.81699999999999995},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.298,"15min":1.298},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80300000000000005,"15min":0.82899999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98999999999999999,"5min":1.204,"15min":1.204},"last":0.877},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90200000000000002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.67300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:19.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.278+0000 7fc770fff700 1 Processor -- start 2023-10-06T05:21:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.278+0000 7fc770fff700 1 -- start start 2023-10-06T05:21:19.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.279+0000 7fc770fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc76c1595a0 0x7fc76c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.279+0000 7fc770fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc76c068d80 0x7fc76c0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.279+0000 7fc770fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76c0c95b0 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.279+0000 7fc770fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76c0c9890 con 0x7fc76c068d80 2023-10-06T05:21:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.279+0000 7fc770fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76c0c9b70 con 0x7fc76c1595a0 2023-10-06T05:21:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.280+0000 7fc76ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc76c068d80 0x7fc76c0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.280+0000 7fc76ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc76c068d80 0x7fc76c0c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60810/0 (socket says 172.21.15.31:60810) 2023-10-06T05:21:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.280+0000 7fc76ad9d700 1 -- 172.21.15.31:0/213775445 learned_addr learned my addr 172.21.15.31:0/213775445 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.280+0000 7fc76a59c700 1 --2- 172.21.15.31:0/213775445 >> v2:172.21.15.62:3300/0 conn(0x7fc76c1595a0 0x7fc76c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.280+0000 7fc76ad9d700 1 -- 172.21.15.31:0/213775445 >> v2:172.21.15.62:3300/0 conn(0x7fc76c1595a0 msgr2=0x7fc76c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.280+0000 7fc76ad9d700 1 --2- 172.21.15.31:0/213775445 >> v2:172.21.15.62:3300/0 conn(0x7fc76c1595a0 0x7fc76c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.280+0000 7fc76ad9d700 1 -- 172.21.15.31:0/213775445 >> v1:172.21.15.62:6789/0 conn(0x7fc76c0cc0d0 legacy=0x7fc76c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.280+0000 7fc76ad9d700 1 -- 172.21.15.31:0/213775445 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c0ca130 con 0x7fc76c068d80 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.280+0000 7fc76ad9d700 1 --2- 172.21.15.31:0/213775445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c068d80 0x7fc76c0c8930 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc764009f20 tx=0x7fc764009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59284d6fe548146d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc76959a700 1 -- 172.21.15.31:0/213775445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76400e070 con 0x7fc76c068d80 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc76959a700 1 -- 172.21.15.31:0/213775445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76400b990 con 0x7fc76c068d80 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc76959a700 1 -- 172.21.15.31:0/213775445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640092e0 con 0x7fc76c068d80 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc770fff700 1 -- 172.21.15.31:0/213775445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c068d80 msgr2=0x7fc76c0c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc770fff700 1 --2- 172.21.15.31:0/213775445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c068d80 0x7fc76c0c8930 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc764009f20 tx=0x7fc764009840 comp rx=0 tx=0).stop 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc770fff700 1 -- 172.21.15.31:0/213775445 shutdown_connections 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc770fff700 1 --2- 172.21.15.31:0/213775445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c068d80 0x7fc76c0c8930 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc770fff700 1 --2- 172.21.15.31:0/213775445 >> v2:172.21.15.62:3300/0 conn(0x7fc76c1595a0 0x7fc76c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc770fff700 1 -- 172.21.15.31:0/213775445 >> 172.21.15.31:0/213775445 conn(0x7fc76c06b620 msgr2=0x7fc76c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:19.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc770fff700 1 -- 172.21.15.31:0/213775445 shutdown_connections 2023-10-06T05:21:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.281+0000 7fc770fff700 1 -- 172.21.15.31:0/213775445 wait complete. 2023-10-06T05:21:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.283+0000 7fc770fff700 1 Processor -- start 2023-10-06T05:21:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.284+0000 7fc770fff700 1 -- start start 2023-10-06T05:21:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.285+0000 7fc770fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc76c068d80 0x7fc76c152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.285+0000 7fc770fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c0cc0d0 0x7fc76c14f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.285+0000 7fc770fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc76c1595a0 0x7fc76c14f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.285+0000 7fc770fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c169a50 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.285+0000 7fc770fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0b1090 con 0x7fc76c068d80 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.285+0000 7fc770fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0c7590 con 0x7fc76c1595a0 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.285+0000 7fc769d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c0cc0d0 0x7fc76c14f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.285+0000 7fc769d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c0cc0d0 0x7fc76c14f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60818/0 (socket says 172.21.15.31:60818) 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.285+0000 7fc769d9b700 1 -- 172.21.15.31:0/946091937 learned_addr learned my addr 172.21.15.31:0/946091937 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc76a59c700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc76c068d80 0x7fc76c152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc76ad9d700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc76c1595a0 0x7fc76c14f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc769d9b700 1 -- 172.21.15.31:0/946091937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc76c1595a0 msgr2=0x7fc76c14f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc769d9b700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc76c1595a0 0x7fc76c14f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc769d9b700 1 -- 172.21.15.31:0/946091937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc76c068d80 msgr2=0x7fc76c152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc769d9b700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc76c068d80 0x7fc76c152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc769d9b700 1 -- 172.21.15.31:0/946091937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764009ae0 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc769d9b700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c0cc0d0 0x7fc76c14f2d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc76000c590 tx=0x7fc76000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc75a7fc700 1 -- 172.21.15.31:0/946091937 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760011590 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc75a7fc700 1 -- 172.21.15.31:0/946091937 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7600116f0 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc75a7fc700 1 -- 172.21.15.31:0/946091937 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760011ab0 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc770fff700 1 -- 172.21.15.31:0/946091937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c0b1330 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.286+0000 7fc770fff700 1 -- 172.21.15.31:0/946091937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c0b18f0 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.287+0000 7fc770fff700 1 -- 172.21.15.31:0/946091937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76c06f190 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.288+0000 7fc75a7fc700 1 -- 172.21.15.31:0/946091937 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fc76001b3b0 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.289+0000 7fc75a7fc700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc74c0672c0 0x7fc74c069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.289+0000 7fc75a7fc700 1 -- 172.21.15.31:0/946091937 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc7600841c0 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.290+0000 7fc76a59c700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc74c0672c0 0x7fc74c069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.290+0000 7fc76a59c700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc74c0672c0 0x7fc74c069770 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009d40 tx=0x7fc75c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.293+0000 7fc75a7fc700 1 -- 172.21.15.31:0/946091937 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc760050e80 con 0x7fc76c0cc0d0 2023-10-06T05:21:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.492+0000 7fc770fff700 1 -- 172.21.15.31:0/946091937 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc76c0b1b10 con 0x7fc74c0672c0 2023-10-06T05:21:19.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.501+0000 7fc75a7fc700 1 -- 172.21.15.31:0/946091937 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308529 (secure 0 0 0) 0x7fc76c0b1b10 con 0x7fc74c0672c0 2023-10-06T05:21:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.505+0000 7fc770fff700 1 -- 172.21.15.31:0/946091937 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc74c0672c0 msgr2=0x7fc74c069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.505+0000 7fc770fff700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc74c0672c0 0x7fc74c069770 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009d40 tx=0x7fc75c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.505+0000 7fc770fff700 1 -- 172.21.15.31:0/946091937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c0cc0d0 msgr2=0x7fc76c14f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.505+0000 7fc770fff700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c0cc0d0 0x7fc76c14f2d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc76000c590 tx=0x7fc76000c950 comp rx=0 tx=0).stop 2023-10-06T05:21:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.505+0000 7fc770fff700 1 -- 172.21.15.31:0/946091937 shutdown_connections 2023-10-06T05:21:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.505+0000 7fc770fff700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc76c1595a0 0x7fc76c14f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.505+0000 7fc770fff700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc76c068d80 0x7fc76c152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.505+0000 7fc770fff700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc74c0672c0 0x7fc74c069770 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.505+0000 7fc770fff700 1 --2- 172.21.15.31:0/946091937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76c0cc0d0 0x7fc76c14f2d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.505+0000 7fc770fff700 1 -- 172.21.15.31:0/946091937 >> 172.21.15.31:0/946091937 conn(0x7fc76c06b620 msgr2=0x7fc76c068670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.506+0000 7fc770fff700 1 -- 172.21.15.31:0/946091937 shutdown_connections 2023-10-06T05:21:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.506+0000 7fc770fff700 1 -- 172.21.15.31:0/946091937 wait complete. 2023-10-06T05:21:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:21:19.525 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2023-10-06T05:21:18.118385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386024,"kb_used_data":7720,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752588696,"statfs":{"total":773094113280,"available":770650824704,"internally_reserved":0,"allocated":7905280,"data_stored":4158720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.969865"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157298+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:40.622847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162888+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:39.527935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156908+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:37.837314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.160141+0000","last_change":"2023-10-06T05:21:14.160278+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.160141+0000","last_undegraded":"2023-10-06T05:21:14.160141+0000","last_fullsized":"2023-10-06T05:21:14.160141+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:57.160327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.159070+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:16.114354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155121+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:43.798148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158115+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:07.141522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157896+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:00.913829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154871+0000","last_change":"2023-10-06T05:21:14.155317+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154871+0000","last_undegraded":"2023-10-06T05:21:14.154871+0000","last_fullsized":"2023-10-06T05:21:14.154871+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:04.649668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159817+0000","last_change":"2023-10-06T05:21:14.159956+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159817+0000","last_undegraded":"2023-10-06T05:21:14.159817+0000","last_fullsized":"2023-10-06T05:21:14.159817+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:55.793132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.164442+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:21.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159761+0000","last_change":"2023-10-06T05:21:14.159896+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159761+0000","last_undegraded":"2023-10-06T05:21:14.159761+0000","last_fullsized":"2023-10-06T05:21:14.159761+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:16.811458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156981+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:43.232828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.154665+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:20.503103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162708+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:33.730974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156775+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:04.855790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158524+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:50.744732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157660+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:20.672647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.152314+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:25.760034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157280+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:02.928620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154926+0000","last_change":"2023-10-06T05:21:14.155091+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154926+0000","last_undegraded":"2023-10-06T05:21:14.154926+0000","last_fullsized":"2023-10-06T05:21:14.154926+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:43.694140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158340+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:44.399326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154629+0000","last_change":"2023-10-06T05:21:14.155278+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154629+0000","last_undegraded":"2023-10-06T05:21:14.154629+0000","last_fullsized":"2023-10-06T05:21:14.154629+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:48.387423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.160885+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:29.781374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.163961+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:49.243354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159370+0000","last_change":"2023-10-06T05:21:14.159548+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159370+0000","last_undegraded":"2023-10-06T05:21:14.159370+0000","last_fullsized":"2023-10-06T05:21:14.159370+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:50.021472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158766+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.976026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156773+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:11.364687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.165301+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:40.622847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157061+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:26.471873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155647+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:22.094091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156138+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:21.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155381+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:45.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.159241+0000","last_change":"2023-10-06T05:21:13.159241+0000","last_active":"2023-10-06T05:21:13.159241+0000","last_peered":"2023-10-06T05:21:13.159241+0000","last_clean":"2023-10-06T05:21:13.159241+0000","last_became_active":"2023-10-06T05:21:13.158913+0000","last_became_peered":"2023-10-06T05:21:13.158913+0000","last_unstale":"2023-10-06T05:21:13.159241+0000","last_undegraded":"2023-10-06T05:21:13.159241+0000","last_fullsized":"2023-10-06T05:21:13.159241+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:58.015087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.161893+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:12.975931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171676+0000","last_change":"2023-10-06T05:21:13.171676+0000","last_active":"2023-10-06T05:21:13.171676+0000","last_peered":"2023-10-06T05:21:13.171676+0000","last_clean":"2023-10-06T05:21:13.171676+0000","last_became_active":"2023-10-06T05:21:13.171360+0000","last_became_peered":"2023-10-06T05:21:13.171360+0000","last_unstale":"2023-10-06T05:21:13.171676+0000","last_undegraded":"2023-10-06T05:21:13.171676+0000","last_fullsized":"2023-10-06T05:21:13.171676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:24.757454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154702+0000","last_change":"2023-10-06T05:21:14.155168+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154702+0000","last_undegraded":"2023-10-06T05:21:14.154702+0000","last_fullsized":"2023-10-06T05:21:14.154702+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:48.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.176256+0000","last_change":"2023-10-06T05:21:13.176256+0000","last_active":"2023-10-06T05:21:13.176256+0000","last_peered":"2023-10-06T05:21:13.176256+0000","last_clean":"2023-10-06T05:21:13.176256+0000","last_became_active":"2023-10-06T05:21:13.175487+0000","last_became_peered":"2023-10-06T05:21:13.175487+0000","last_unstale":"2023-10-06T05:21:13.176256+0000","last_undegraded":"2023-10-06T05:21:13.176256+0000","last_fullsized":"2023-10-06T05:21:13.176256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:19.500524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154776+0000","last_change":"2023-10-06T05:21:14.154975+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154776+0000","last_undegraded":"2023-10-06T05:21:14.154776+0000","last_fullsized":"2023-10-06T05:21:14.154776+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:09.170341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.169221+0000","last_change":"2023-10-06T05:21:13.169221+0000","last_active":"2023-10-06T05:21:13.169221+0000","last_peered":"2023-10-06T05:21:13.169221+0000","last_clean":"2023-10-06T05:21:13.169221+0000","last_became_active":"2023-10-06T05:21:13.168853+0000","last_became_peered":"2023-10-06T05:21:13.168853+0000","last_unstale":"2023-10-06T05:21:13.169221+0000","last_undegraded":"2023-10-06T05:21:13.169221+0000","last_fullsized":"2023-10-06T05:21:13.169221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:12.290741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.163333+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:43.798148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.167639+0000","last_change":"2023-10-06T05:21:13.162613+0000","last_active":"2023-10-06T05:21:13.167639+0000","last_peered":"2023-10-06T05:21:13.167639+0000","last_clean":"2023-10-06T05:21:13.167639+0000","last_became_active":"2023-10-06T05:21:13.162167+0000","last_became_peered":"2023-10-06T05:21:13.162167+0000","last_unstale":"2023-10-06T05:21:13.167639+0000","last_undegraded":"2023-10-06T05:21:13.167639+0000","last_fullsized":"2023-10-06T05:21:13.167639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:36.834735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.164889+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:26.471873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.175784+0000","last_change":"2023-10-06T05:21:13.170589+0000","last_active":"2023-10-06T05:21:13.175784+0000","last_peered":"2023-10-06T05:21:13.175784+0000","last_clean":"2023-10-06T05:21:13.175784+0000","last_became_active":"2023-10-06T05:21:13.170282+0000","last_became_peered":"2023-10-06T05:21:13.170282+0000","last_unstale":"2023-10-06T05:21:13.175784+0000","last_undegraded":"2023-10-06T05:21:13.175784+0000","last_fullsized":"2023-10-06T05:21:13.175784+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:53.541975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.155752+0000","last_change":"2023-10-06T05:21:14.155916+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.155752+0000","last_undegraded":"2023-10-06T05:21:14.155752+0000","last_fullsized":"2023-10-06T05:21:14.155752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:54.544554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.175791+0000","last_change":"2023-10-06T05:21:13.175791+0000","last_active":"2023-10-06T05:21:13.175791+0000","last_peered":"2023-10-06T05:21:13.175791+0000","last_clean":"2023-10-06T05:21:13.175791+0000","last_became_active":"2023-10-06T05:21:13.175326+0000","last_became_peered":"2023-10-06T05:21:13.175326+0000","last_unstale":"2023-10-06T05:21:13.175791+0000","last_undegraded":"2023-10-06T05:21:13.175791+0000","last_fullsized":"2023-10-06T05:21:13.175791+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.040745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158575+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:56.726499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.170180+0000","last_change":"2023-10-06T05:21:13.170180+0000","last_active":"2023-10-06T05:21:13.170180+0000","last_peered":"2023-10-06T05:21:13.170180+0000","last_clean":"2023-10-06T05:21:13.170180+0000","last_became_active":"2023-10-06T05:21:13.169857+0000","last_became_peered":"2023-10-06T05:21:13.169857+0000","last_unstale":"2023-10-06T05:21:13.170180+0000","last_undegraded":"2023-10-06T05:21:13.170180+0000","last_fullsized":"2023-10-06T05:21:13.170180+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:45.593601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.161768+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:25.760034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.172453+0000","last_change":"2023-10-06T05:21:13.172453+0000","last_active":"2023-10-06T05:21:13.172453+0000","last_peered":"2023-10-06T05:21:13.172453+0000","last_clean":"2023-10-06T05:21:13.172453+0000","last_became_active":"2023-10-06T05:21:13.171973+0000","last_became_peered":"2023-10-06T05:21:13.171973+0000","last_unstale":"2023-10-06T05:21:13.172453+0000","last_undegraded":"2023-10-06T05:21:13.172453+0000","last_fullsized":"2023-10-06T05:21:13.172453+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:53.470594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156446+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:48.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.170853+0000","last_change":"2023-10-06T05:21:13.168940+0000","last_active":"2023-10-06T05:21:13.170853+0000","last_peered":"2023-10-06T05:21:13.170853+0000","last_clean":"2023-10-06T05:21:13.170853+0000","last_became_active":"2023-10-06T05:21:13.168641+0000","last_became_peered":"2023-10-06T05:21:13.168641+0000","last_unstale":"2023-10-06T05:21:13.170853+0000","last_undegraded":"2023-10-06T05:21:13.170853+0000","last_fullsized":"2023-10-06T05:21:13.170853+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:42.691561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.154172+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:54.473173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.163900+0000","last_change":"2023-10-06T05:21:13.163900+0000","last_active":"2023-10-06T05:21:13.163900+0000","last_peered":"2023-10-06T05:21:13.163900+0000","last_clean":"2023-10-06T05:21:13.163900+0000","last_became_active":"2023-10-06T05:21:13.163604+0000","last_became_peered":"2023-10-06T05:21:13.163604+0000","last_unstale":"2023-10-06T05:21:13.163900+0000","last_undegraded":"2023-10-06T05:21:13.163900+0000","last_fullsized":"2023-10-06T05:21:13.163900+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:55.585059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155983+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:25.096218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.170791+0000","last_change":"2023-10-06T05:21:13.170791+0000","last_active":"2023-10-06T05:21:13.170791+0000","last_peered":"2023-10-06T05:21:13.170791+0000","last_clean":"2023-10-06T05:21:13.170791+0000","last_became_active":"2023-10-06T05:21:13.170470+0000","last_became_peered":"2023-10-06T05:21:13.170470+0000","last_unstale":"2023-10-06T05:21:13.170791+0000","last_undegraded":"2023-10-06T05:21:13.170791+0000","last_fullsized":"2023-10-06T05:21:13.170791+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:26.418570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.159317+0000","last_change":"2023-10-06T05:21:14.159488+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.159317+0000","last_undegraded":"2023-10-06T05:21:14.159317+0000","last_fullsized":"2023-10-06T05:21:14.159317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:45.462119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.175866+0000","last_change":"2023-10-06T05:21:13.171211+0000","last_active":"2023-10-06T05:21:13.175866+0000","last_peered":"2023-10-06T05:21:13.175866+0000","last_clean":"2023-10-06T05:21:13.175866+0000","last_became_active":"2023-10-06T05:21:13.170917+0000","last_became_peered":"2023-10-06T05:21:13.170917+0000","last_unstale":"2023-10-06T05:21:13.175866+0000","last_undegraded":"2023-10-06T05:21:13.175866+0000","last_fullsized":"2023-10-06T05:21:13.175866+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:08.167762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156475+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:56.735464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171789+0000","last_change":"2023-10-06T05:21:13.171789+0000","last_active":"2023-10-06T05:21:13.171789+0000","last_peered":"2023-10-06T05:21:13.171789+0000","last_clean":"2023-10-06T05:21:13.171789+0000","last_became_active":"2023-10-06T05:21:13.171451+0000","last_became_peered":"2023-10-06T05:21:13.171451+0000","last_unstale":"2023-10-06T05:21:13.171789+0000","last_undegraded":"2023-10-06T05:21:13.171789+0000","last_fullsized":"2023-10-06T05:21:13.171789+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:11.973352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157028+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:02.476152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171171+0000","last_change":"2023-10-06T05:21:13.171171+0000","last_active":"2023-10-06T05:21:13.171171+0000","last_peered":"2023-10-06T05:21:13.171171+0000","last_clean":"2023-10-06T05:21:13.171171+0000","last_became_active":"2023-10-06T05:21:13.170856+0000","last_became_peered":"2023-10-06T05:21:13.170856+0000","last_unstale":"2023-10-06T05:21:13.171171+0000","last_undegraded":"2023-10-06T05:21:13.171171+0000","last_fullsized":"2023-10-06T05:21:13.171171+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:45.319480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157405+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:08.922350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.176681+0000","last_change":"2023-10-06T05:21:13.176681+0000","last_active":"2023-10-06T05:21:13.176681+0000","last_peered":"2023-10-06T05:21:13.176681+0000","last_clean":"2023-10-06T05:21:13.176681+0000","last_became_active":"2023-10-06T05:21:13.176432+0000","last_became_peered":"2023-10-06T05:21:13.176432+0000","last_unstale":"2023-10-06T05:21:13.176681+0000","last_undegraded":"2023-10-06T05:21:13.176681+0000","last_fullsized":"2023-10-06T05:21:13.176681+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:15.808879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.154629+0000","last_change":"2023-10-06T05:21:14.154830+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.154629+0000","last_undegraded":"2023-10-06T05:21:14.154629+0000","last_fullsized":"2023-10-06T05:21:14.154629+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:06.020289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.169145+0000","last_change":"2023-10-06T05:21:13.165877+0000","last_active":"2023-10-06T05:21:13.169145+0000","last_peered":"2023-10-06T05:21:13.169145+0000","last_clean":"2023-10-06T05:21:13.169145+0000","last_became_active":"2023-10-06T05:21:13.165605+0000","last_became_peered":"2023-10-06T05:21:13.165605+0000","last_unstale":"2023-10-06T05:21:13.169145+0000","last_undegraded":"2023-10-06T05:21:13.169145+0000","last_fullsized":"2023-10-06T05:21:13.169145+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157597+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:26.639167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171761+0000","last_change":"2023-10-06T05:21:13.171761+0000","last_active":"2023-10-06T05:21:13.171761+0000","last_peered":"2023-10-06T05:21:13.171761+0000","last_clean":"2023-10-06T05:21:13.171761+0000","last_became_active":"2023-10-06T05:21:13.171491+0000","last_became_peered":"2023-10-06T05:21:13.171491+0000","last_unstale":"2023-10-06T05:21:13.171761+0000","last_undegraded":"2023-10-06T05:21:13.171761+0000","last_fullsized":"2023-10-06T05:21:13.171761+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:56.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":2,"reported_epoch":70,"state":"creating+activating","last_fresh":"2023-10-06T05:21:14.155136+0000","last_change":"2023-10-06T05:21:14.155680+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:14.155136+0000","last_undegraded":"2023-10-06T05:21:14.155136+0000","last_fullsized":"2023-10-06T05:21:14.155136+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:04.797474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.167952+0000","last_change":"2023-10-06T05:21:13.167952+0000","last_active":"2023-10-06T05:21:13.167952+0000","last_peered":"2023-10-06T05:21:13.167952+0000","last_clean":"2023-10-06T05:21:13.167952+0000","last_became_active":"2023-10-06T05:21:13.166565+0000","last_became_peered":"2023-10-06T05:21:13.166565+0000","last_unstale":"2023-10-06T05:21:13.167952+0000","last_undegraded":"2023-10-06T05:21:13.167952+0000","last_fullsized":"2023-10-06T05:21:13.167952+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:38.525356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.164809+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:04.855790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171867+0000","last_change":"2023-10-06T05:21:13.171867+0000","last_active":"2023-10-06T05:21:13.171867+0000","last_peered":"2023-10-06T05:21:13.171867+0000","last_clean":"2023-10-06T05:21:13.171867+0000","last_became_active":"2023-10-06T05:21:13.171530+0000","last_became_peered":"2023-10-06T05:21:13.171530+0000","last_unstale":"2023-10-06T05:21:13.171867+0000","last_undegraded":"2023-10-06T05:21:13.171867+0000","last_fullsized":"2023-10-06T05:21:13.171867+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:28.778795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.159646+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:15.167368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.172743+0000","last_change":"2023-10-06T05:21:13.172743+0000","last_active":"2023-10-06T05:21:13.172743+0000","last_peered":"2023-10-06T05:21:13.172743+0000","last_clean":"2023-10-06T05:21:13.172743+0000","last_became_active":"2023-10-06T05:21:13.172453+0000","last_became_peered":"2023-10-06T05:21:13.172453+0000","last_unstale":"2023-10-06T05:21:13.172743+0000","last_undegraded":"2023-10-06T05:21:13.172743+0000","last_fullsized":"2023-10-06T05:21:13.172743+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:01.292501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.155880+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:35.471644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171851+0000","last_change":"2023-10-06T05:21:13.171851+0000","last_active":"2023-10-06T05:21:13.171851+0000","last_peered":"2023-10-06T05:21:13.171851+0000","last_clean":"2023-10-06T05:21:13.171851+0000","last_became_active":"2023-10-06T05:21:13.171552+0000","last_became_peered":"2023-10-06T05:21:13.171552+0000","last_unstale":"2023-10-06T05:21:13.171851+0000","last_undegraded":"2023-10-06T05:21:13.171851+0000","last_fullsized":"2023-10-06T05:21:13.171851+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:14:14.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157183+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:21.116415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.172335+0000","last_change":"2023-10-06T05:21:13.172335+0000","last_active":"2023-10-06T05:21:13.172335+0000","last_peered":"2023-10-06T05:21:13.172335+0000","last_clean":"2023-10-06T05:21:13.172335+0000","last_became_active":"2023-10-06T05:21:13.172046+0000","last_became_peered":"2023-10-06T05:21:13.172046+0000","last_unstale":"2023-10-06T05:21:13.172335+0000","last_undegraded":"2023-10-06T05:21:13.172335+0000","last_fullsized":"2023-10-06T05:21:13.172335+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:49.234603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.154688+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:22.127378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.174008+0000","last_change":"2023-10-06T05:21:13.174008+0000","last_active":"2023-10-06T05:21:13.174008+0000","last_peered":"2023-10-06T05:21:13.174008+0000","last_clean":"2023-10-06T05:21:13.174008+0000","last_became_active":"2023-10-06T05:21:13.173503+0000","last_became_peered":"2023-10-06T05:21:13.173503+0000","last_unstale":"2023-10-06T05:21:13.174008+0000","last_undegraded":"2023-10-06T05:21:13.174008+0000","last_fullsized":"2023-10-06T05:21:13.174008+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:14.164789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.152598+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:12.975931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.165946+0000","last_change":"2023-10-06T05:21:13.165946+0000","last_active":"2023-10-06T05:21:13.165946+0000","last_peered":"2023-10-06T05:21:13.165946+0000","last_clean":"2023-10-06T05:21:13.165946+0000","last_became_active":"2023-10-06T05:21:13.165677+0000","last_became_peered":"2023-10-06T05:21:13.165677+0000","last_unstale":"2023-10-06T05:21:13.165946+0000","last_undegraded":"2023-10-06T05:21:13.165946+0000","last_fullsized":"2023-10-06T05:21:13.165946+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:32.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.160560+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:21:13.160560+0000","last_peered":"2023-10-06T05:21:13.160560+0000","last_clean":"2023-10-06T05:21:13.160560+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:21:13.160560+0000","last_undegraded":"2023-10-06T05:21:13.160560+0000","last_fullsized":"2023-10-06T05:21:13.160560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156102+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:36.053585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.169590+0000","last_change":"2023-10-06T05:21:13.165590+0000","last_active":"2023-10-06T05:21:13.169590+0000","last_peered":"2023-10-06T05:21:13.169590+0000","last_clean":"2023-10-06T05:21:13.169590+0000","last_became_active":"2023-10-06T05:21:13.165314+0000","last_became_peered":"2023-10-06T05:21:13.165314+0000","last_unstale":"2023-10-06T05:21:13.169590+0000","last_undegraded":"2023-10-06T05:21:13.169590+0000","last_fullsized":"2023-10-06T05:21:13.169590+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:24.093639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.151269+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:21:12.151269+0000","last_peered":"2023-10-06T05:21:12.151269+0000","last_clean":"2023-10-06T05:21:12.151269+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:21:12.151269+0000","last_undegraded":"2023-10-06T05:21:12.151269+0000","last_fullsized":"2023-10-06T05:21:12.151269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162394+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:54.473173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.166267+0000","last_change":"2023-10-06T05:21:13.166267+0000","last_active":"2023-10-06T05:21:13.166267+0000","last_peered":"2023-10-06T05:21:13.166267+0000","last_clean":"2023-10-06T05:21:13.166267+0000","last_became_active":"2023-10-06T05:21:13.165887+0000","last_became_peered":"2023-10-06T05:21:13.165887+0000","last_unstale":"2023-10-06T05:21:13.166267+0000","last_undegraded":"2023-10-06T05:21:13.166267+0000","last_fullsized":"2023-10-06T05:21:13.166267+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:56.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.151973+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:21:12.151973+0000","last_peered":"2023-10-06T05:21:12.151973+0000","last_clean":"2023-10-06T05:21:12.151973+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:21:12.151973+0000","last_undegraded":"2023-10-06T05:21:12.151973+0000","last_fullsized":"2023-10-06T05:21:12.151973+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":202,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.156005+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:21:12.156005+0000","last_peered":"2023-10-06T05:21:12.156005+0000","last_clean":"2023-10-06T05:21:12.156005+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:21:12.156005+0000","last_undegraded":"2023-10-06T05:21:12.156005+0000","last_fullsized":"2023-10-06T05:21:12.156005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157962+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:25.384243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.168485+0000","last_change":"2023-10-06T05:21:13.168485+0000","last_active":"2023-10-06T05:21:13.168485+0000","last_peered":"2023-10-06T05:21:13.168485+0000","last_clean":"2023-10-06T05:21:13.168485+0000","last_became_active":"2023-10-06T05:21:13.168162+0000","last_became_peered":"2023-10-06T05:21:13.168162+0000","last_unstale":"2023-10-06T05:21:13.168485+0000","last_undegraded":"2023-10-06T05:21:13.168485+0000","last_fullsized":"2023-10-06T05:21:13.168485+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.040745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.156129+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:21:12.156129+0000","last_peered":"2023-10-06T05:21:12.156129+0000","last_clean":"2023-10-06T05:21:12.156129+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:21:12.156129+0000","last_undegraded":"2023-10-06T05:21:12.156129+0000","last_fullsized":"2023-10-06T05:21:12.156129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.158198+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:46.254829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.171949+0000","last_change":"2023-10-06T05:21:13.171949+0000","last_active":"2023-10-06T05:21:13.171949+0000","last_peered":"2023-10-06T05:21:13.171949+0000","last_clean":"2023-10-06T05:21:13.171949+0000","last_became_active":"2023-10-06T05:21:13.171646+0000","last_became_peered":"2023-10-06T05:21:13.171646+0000","last_unstale":"2023-10-06T05:21:13.171949+0000","last_undegraded":"2023-10-06T05:21:13.171949+0000","last_fullsized":"2023-10-06T05:21:13.171949+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:00.022649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.156123+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:21:12.156123+0000","last_peered":"2023-10-06T05:21:12.156123+0000","last_clean":"2023-10-06T05:21:12.156123+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:21:12.156123+0000","last_undegraded":"2023-10-06T05:21:12.156123+0000","last_fullsized":"2023-10-06T05:21:12.156123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.162802+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:20.503103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.168910+0000","last_change":"2023-10-06T05:21:13.168910+0000","last_active":"2023-10-06T05:21:13.168910+0000","last_peered":"2023-10-06T05:21:13.168910+0000","last_clean":"2023-10-06T05:21:13.168910+0000","last_became_active":"2023-10-06T05:21:13.168616+0000","last_became_peered":"2023-10-06T05:21:13.168616+0000","last_unstale":"2023-10-06T05:21:13.168910+0000","last_undegraded":"2023-10-06T05:21:13.168910+0000","last_fullsized":"2023-10-06T05:21:13.168910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:15.808879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.162010+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:21:12.162010+0000","last_peered":"2023-10-06T05:21:12.162010+0000","last_clean":"2023-10-06T05:21:12.162010+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:21:12.162010+0000","last_undegraded":"2023-10-06T05:21:12.162010+0000","last_fullsized":"2023-10-06T05:21:12.162010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.156041+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:49.243354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.166892+0000","last_change":"2023-10-06T05:21:13.166892+0000","last_active":"2023-10-06T05:21:13.166892+0000","last_peered":"2023-10-06T05:21:13.166892+0000","last_clean":"2023-10-06T05:21:13.166892+0000","last_became_active":"2023-10-06T05:21:13.166496+0000","last_became_peered":"2023-10-06T05:21:13.166496+0000","last_unstale":"2023-10-06T05:21:13.166892+0000","last_undegraded":"2023-10-06T05:21:13.166892+0000","last_fullsized":"2023-10-06T05:21:13.166892+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:13.510131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.152394+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:21:12.152394+0000","last_peered":"2023-10-06T05:21:12.152394+0000","last_clean":"2023-10-06T05:21:12.152394+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:21:12.152394+0000","last_undegraded":"2023-10-06T05:21:12.152394+0000","last_fullsized":"2023-10-06T05:21:12.152394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:21:13.145495+0000","last_change":"2023-10-06T05:21:13.157810+0000","last_active":"2023-10-06T05:21:13.145495+0000","last_peered":"2023-10-06T05:21:13.145495+0000","last_clean":"2023-10-06T05:21:13.145495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:21:13.145495+0000","last_undegraded":"2023-10-06T05:21:13.145495+0000","last_fullsized":"2023-10-06T05:21:13.145495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:20.516682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-10-06T05:21:13.159342+0000","last_change":"2023-10-06T05:21:13.159342+0000","last_active":"2023-10-06T05:21:13.159342+0000","last_peered":"2023-10-06T05:21:13.159342+0000","last_clean":"2023-10-06T05:21:13.159342+0000","last_became_active":"2023-10-06T05:21:13.158741+0000","last_became_peered":"2023-10-06T05:21:13.158741+0000","last_unstale":"2023-10-06T05:21:13.159342+0000","last_undegraded":"2023-10-06T05:21:13.159342+0000","last_fullsized":"2023-10-06T05:21:13.159342+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:22.191827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":53,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-06T05:21:12.152293+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:21:12.152293+0000","last_peered":"2023-10-06T05:21:12.152293+0000","last_clean":"2023-10-06T05:21:12.152293+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:21:12.152293+0000","last_undegraded":"2023-10-06T05:21:12.152293+0000","last_fullsized":"2023-10-06T05:21:12.152293+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298688,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073152,"statfs":{"total":96636764160,"available":96330907648,"internally_reserved":0,"allocated":1433600,"data_stored":962616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.88200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.68200000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93600000000000005,"5min":1.018,"15min":1.018},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95999999999999996,"5min":1.056,"15min":1.056},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73399999999999999,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67700000000000005,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84499999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93300000000000005,"5min":0.999,"15min":0.999},"last":0.755},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.83799999999999997},"min":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0389999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.626,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.85399999999999998},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0029999999999999,"5min":1.103,"15min":1.103},"last":0.90100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.47099999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89800000000000002,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.627},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90800000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78500000000000003,"5min":1.369,"15min":1.369},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.891,"15min":1.891},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89800000000000002,"5min":1.347,"15min":1.347},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":1,"15min":1},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73499999999999999,"15min":0.78500000000000003},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.95499999999999996,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.746},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78200000000000003,"15min":0.82399999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.82199999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85899999999999999,"5min":1.415,"15min":1.415},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77500000000000002,"15min":0.83099999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98199999999999998,"5min":2,"15min":2},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.78000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95999999999999996,"5min":1.46,"15min":1.46},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298688,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073152,"statfs":{"total":96636764160,"available":96330907648,"internally_reserved":0,"allocated":1433600,"data_stored":962616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.378},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.75},{"interface":"front","average":{"1min":0.877,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.59699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82499999999999996},"min":{"1min":0.63600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.753},"min":{"1min":0.621,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.052,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.82999999999999996,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.83499999999999996,"15min":0.82499999999999996},"min":{"1min":0.67100000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.69299999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.087,"5min":1.103,"15min":1.103},"last":0.73099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68300000000000005,"5min":1.042,"15min":1.042},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.436},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1519999999999999,"5min":1.242,"15min":1.242},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":1.198,"15min":1.198},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.77300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1659999999999999,"5min":1.26,"15min":1.26},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.073,"5min":1.323,"15min":1.323},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.123,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.264,"15min":1.264},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.04,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.374}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46500000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.624,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.503},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68799999999999994,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93500000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90600000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96299999999999997,"5min":1.099,"15min":1.099},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78500000000000003,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.90000000000000002,"5min":1.163,"15min":1.163},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98799999999999999,"5min":1.21,"15min":1.21},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.022,"5min":1.042,"15min":1.042},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84399999999999997,"15min":0.872},"min":{"1min":0.64600000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0469999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.82399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.44600000000000001,"15min":0.46200000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57199999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.624,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.441},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.497},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72299999999999998,"5min":1.014,"15min":1.014},"last":0.503}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.145,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1830000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.03,"5min":1.204,"15min":1.204},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.329},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.82099999999999995},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.048,"5min":1.484,"15min":1.484},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83499999999999996,"15min":0.85399999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1930000000000001,"5min":1.629,"15min":1.629},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.83999999999999997,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1950000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.085,"5min":1.325,"15min":1.325},"last":0.61499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74399999999999999,"15min":0.79800000000000004},"min":{"1min":0.52100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":2.77,"15min":2.77},"last":0.92400000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.78700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.76600000000000001,"15min":0.82999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89600000000000002,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.1120000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.58599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.90500000000000003,"15min":1.091},"min":{"1min":0.50800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":7.742,"15min":7.742},"last":0.93999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.48999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96299999999999997,"5min":1.079,"15min":1.079},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90900000000000003,"5min":1.181,"15min":1.181},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.745},"min":{"1min":0.56499999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.60599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.92200000000000004,"5min":1.171,"15min":1.171},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.624,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96499999999999997,"5min":1.228,"15min":1.228},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81499999999999995,"15min":0.86899999999999999},"min":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97399999999999998,"5min":1.571,"15min":1.571},"last":0.73499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":372248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.83099999999999996,"15min":1},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.85499999999999998,"15min":1.0209999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.079,"5min":9.25,"15min":9.25},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.875,"15min":1.0620000000000001},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":10.384,"15min":10.384},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":1.0760000000000001},"min":{"1min":0.49299999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0349999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.78400000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":1.1240000000000001},"min":{"1min":0.54800000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.95199999999999996,"5min":10.205,"15min":10.205},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96399999999999997,"5min":1.292,"15min":1.292},"last":0.625}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.048,"5min":1.171,"15min":1.171},"last":0.83299999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78400000000000003,"15min":0.81699999999999995},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.298,"15min":1.298},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80300000000000005,"15min":0.82899999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98999999999999999,"5min":1.204,"15min":1.204},"last":0.877},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90200000000000002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.67300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:19.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2023-10-06T05:21:19.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157298+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:29:40.622847+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:19.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2023-10-06T05:21:19.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.162888+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:06:39.527935+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3d is not active+clean 2023-10-06T05:21:19.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.156908+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:48:37.837314+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:19.534 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3c is not active+clean 2023-10-06T05:21:19.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.160141+0000', 'last_change': '2023-10-06T05:21:14.160278+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.160141+0000', 'last_undegraded': '2023-10-06T05:21:14.160141+0000', 'last_fullsized': '2023-10-06T05:21:14.160141+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:27:57.160327+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': ['5', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:19.534 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3b is not active+clean 2023-10-06T05:21:19.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.159070+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:32:16.114354+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:19.534 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3a is not active+clean 2023-10-06T05:21:19.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.155121+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:15:43.798148+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:19.534 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2023-10-06T05:21:19.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.158115+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:47:07.141522+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:19.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.38 is not active+clean 2023-10-06T05:21:19.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157896+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:11:00.913829+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:19.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2023-10-06T05:21:19.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.154871+0000', 'last_change': '2023-10-06T05:21:14.155317+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.154871+0000', 'last_undegraded': '2023-10-06T05:21:14.154871+0000', 'last_fullsized': '2023-10-06T05:21:14.154871+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:38:04.649668+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': ['4', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:19.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.36 is not active+clean 2023-10-06T05:21:19.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.159817+0000', 'last_change': '2023-10-06T05:21:14.159956+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.159817+0000', 'last_undegraded': '2023-10-06T05:21:14.159817+0000', 'last_fullsized': '2023-10-06T05:21:14.159817+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:33:55.793132+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': ['5', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:19.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2023-10-06T05:21:19.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.164442+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:14:21.689708+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.34 is not active+clean 2023-10-06T05:21:19.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.159761+0000', 'last_change': '2023-10-06T05:21:14.159896+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.159761+0000', 'last_undegraded': '2023-10-06T05:21:14.159761+0000', 'last_fullsized': '2023-10-06T05:21:14.159761+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:28:16.811458+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': ['5', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:19.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.33 is not active+clean 2023-10-06T05:21:19.536 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.156981+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:20:43.232828+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:19.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.32 is not active+clean 2023-10-06T05:21:19.536 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.154665+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:09:20.503103+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:19.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2023-10-06T05:21:19.536 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.162708+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T17:00:33.730974+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2023-10-06T05:21:19.536 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.156775+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:09:04.855790+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:19.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2f is not active+clean 2023-10-06T05:21:19.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.158524+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:34:50.744732+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:19.537 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2e is not active+clean 2023-10-06T05:21:19.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157660+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:29:20.672647+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:19.537 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2d is not active+clean 2023-10-06T05:21:19.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.152314+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:59:25.760034+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:19.537 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2c is not active+clean 2023-10-06T05:21:19.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157280+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:39:02.928620+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:19.537 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2023-10-06T05:21:19.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.154926+0000', 'last_change': '2023-10-06T05:21:14.155091+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.154926+0000', 'last_undegraded': '2023-10-06T05:21:14.154926+0000', 'last_fullsized': '2023-10-06T05:21:14.154926+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:25:43.694140+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': ['4', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:19.538 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2a is not active+clean 2023-10-06T05:21:19.538 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.158340+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:00:44.399326+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:19.538 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2023-10-06T05:21:19.538 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.154629+0000', 'last_change': '2023-10-06T05:21:14.155278+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.154629+0000', 'last_undegraded': '2023-10-06T05:21:14.154629+0000', 'last_fullsized': '2023-10-06T05:21:14.154629+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:14:48.387423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': ['4', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:19.538 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2023-10-06T05:21:19.538 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.160885+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:01:29.781374+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.538 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2023-10-06T05:21:19.538 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.163961+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:32:49.243354+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.538 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.26 is not active+clean 2023-10-06T05:21:19.538 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.159370+0000', 'last_change': '2023-10-06T05:21:14.159548+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.159370+0000', 'last_undegraded': '2023-10-06T05:21:14.159370+0000', 'last_fullsized': '2023-10-06T05:21:14.159370+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:47:50.021472+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': ['5', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:19.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2023-10-06T05:21:19.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.158766+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:29:17.976026+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:19.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.24 is not active+clean 2023-10-06T05:21:19.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.156773+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:57:11.364687+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:19.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2023-10-06T05:21:19.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.165301+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:29:40.622847+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2023-10-06T05:21:19.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157061+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:12:26.471873+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:19.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.21 is not active+clean 2023-10-06T05:21:19.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.155647+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:52:22.094091+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:19.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2023-10-06T05:21:19.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.156138+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:14:21.689708+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:19.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2023-10-06T05:21:19.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.155381+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:54:45.766521+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:19.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2023-10-06T05:21:19.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.161893+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:01:12.975931+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2023-10-06T05:21:19.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.154702+0000', 'last_change': '2023-10-06T05:21:14.155168+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.154702+0000', 'last_undegraded': '2023-10-06T05:21:14.154702+0000', 'last_fullsized': '2023-10-06T05:21:14.154702+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:52:48.452087+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': ['4', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:19.541 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2023-10-06T05:21:19.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.154776+0000', 'last_change': '2023-10-06T05:21:14.154975+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.154776+0000', 'last_undegraded': '2023-10-06T05:21:14.154776+0000', 'last_fullsized': '2023-10-06T05:21:14.154776+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:18:09.170341+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': ['4', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:19.541 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2023-10-06T05:21:19.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.163333+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:15:43.798148+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.541 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2023-10-06T05:21:19.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.164889+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:12:26.471873+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.541 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2023-10-06T05:21:19.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.155752+0000', 'last_change': '2023-10-06T05:21:14.155916+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.155752+0000', 'last_undegraded': '2023-10-06T05:21:14.155752+0000', 'last_fullsized': '2023-10-06T05:21:14.155752+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:39:54.544554+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': ['4', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:19.541 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2023-10-06T05:21:19.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.158575+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:48:56.726499+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:19.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2023-10-06T05:21:19.542 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.161768+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:59:25.760034+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2023-10-06T05:21:19.542 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.156446+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:52:48.452087+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:19.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2023-10-06T05:21:19.542 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.154172+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:28:54.473173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:19.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2023-10-06T05:21:19.542 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.155983+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:56:25.096218+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:19.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2023-10-06T05:21:19.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.159317+0000', 'last_change': '2023-10-06T05:21:14.159488+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.159317+0000', 'last_undegraded': '2023-10-06T05:21:14.159317+0000', 'last_fullsized': '2023-10-06T05:21:14.159317+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T15:10:45.462119+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T05:21:19.543 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2023-10-06T05:21:19.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.156475+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:19:56.735464+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:19.543 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2023-10-06T05:21:19.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157028+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:20:02.476152+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:19.543 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2023-10-06T05:21:19.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157405+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:11:08.922350+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:19.543 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2023-10-06T05:21:19.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.154629+0000', 'last_change': '2023-10-06T05:21:14.154830+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.154629+0000', 'last_undegraded': '2023-10-06T05:21:14.154629+0000', 'last_fullsized': '2023-10-06T05:21:14.154629+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:18:06.020289+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': ['4', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:19.544 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2023-10-06T05:21:19.544 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157597+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:49:26.639167+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:19.544 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2023-10-06T05:21:19.544 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 70, 'state': 'creating+activating', 'last_fresh': '2023-10-06T05:21:14.155136+0000', 'last_change': '2023-10-06T05:21:14.155680+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:14.155136+0000', 'last_undegraded': '2023-10-06T05:21:14.155136+0000', 'last_fullsized': '2023-10-06T05:21:14.155136+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:16:04.797474+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': ['4', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T05:21:19.544 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2023-10-06T05:21:19.544 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.164809+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:09:04.855790+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.544 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2023-10-06T05:21:19.544 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.159646+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:35:15.167368+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:19.544 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2023-10-06T05:21:19.544 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.155880+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:13:35.471644+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:19.545 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2023-10-06T05:21:19.545 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157183+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:22:21.116415+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:19.545 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2023-10-06T05:21:19.545 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.154688+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:45:22.127378+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T05:21:19.545 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2023-10-06T05:21:19.545 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.152598+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T06:01:12.975931+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:19.545 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2023-10-06T05:21:19.545 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.156102+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:18:36.053585+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:19.545 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2023-10-06T05:21:19.546 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.162394+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:28:54.473173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.546 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2023-10-06T05:21:19.546 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157962+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:17:25.384243+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T05:21:19.546 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2023-10-06T05:21:19.546 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.158198+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:27:46.254829+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T05:21:19.546 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2023-10-06T05:21:19.546 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.162802+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T16:09:20.503103+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T05:21:19.547 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2023-10-06T05:21:19.547 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.156041+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:32:49.243354+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T05:21:19.547 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2023-10-06T05:21:19.547 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T05:21:13.145495+0000', 'last_change': '2023-10-06T05:21:13.157810+0000', 'last_active': '2023-10-06T05:21:13.145495+0000', 'last_peered': '2023-10-06T05:21:13.145495+0000', 'last_clean': '2023-10-06T05:21:13.145495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T05:21:13.145495+0000', 'last_undegraded': '2023-10-06T05:21:13.145495+0000', 'last_fullsized': '2023-10-06T05:21:13.145495+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'last_clean_scrub_stamp': '2023-10-06T05:21:13.145495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:49:20.516682+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T05:21:19.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T05:21:19.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.758+0000 7f02945ff700 1 Processor -- start 2023-10-06T05:21:19.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.759+0000 7f02945ff700 1 -- start start 2023-10-06T05:21:19.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.760+0000 7f02945ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 0x7f028c06e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.760+0000 7f02945ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f028c06ea90 0x7f028c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.760+0000 7f02945ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c06be70 con 0x7f028c13fb00 2023-10-06T05:21:19.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.760+0000 7f02945ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c06ef30 con 0x7f028c06ea90 2023-10-06T05:21:19.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.760+0000 7f02945ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c06ca10 con 0x7f028c13fed0 2023-10-06T05:21:19.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.760+0000 7f029239b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f028c13fb00 0x7f028c06c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35604/0 (socket says 172.21.15.31:35604) 2023-10-06T05:21:19.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.760+0000 7f029239b700 1 -- 172.21.15.31:0/2356283774 learned_addr learned my addr 172.21.15.31:0/2356283774 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:19.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.760+0000 7f0292b9c700 1 --2- 172.21.15.31:0/2356283774 >> v2:172.21.15.31:3300/0 conn(0x7f028c06ea90 0x7f028c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.760+0000 7f0291b9a700 1 --2- 172.21.15.31:0/2356283774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 0x7f028c06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.761+0000 7f0292b9c700 1 -- 172.21.15.31:0/2356283774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 msgr2=0x7f028c06e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:19.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.761+0000 7f0292b9c700 1 --2- 172.21.15.31:0/2356283774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 0x7f028c06e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.761+0000 7f0292b9c700 1 -- 172.21.15.31:0/2356283774 >> v1:172.21.15.31:6789/0 conn(0x7f028c13fb00 legacy=0x7f028c06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.761+0000 7f0292b9c700 1 -- 172.21.15.31:0/2356283774 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c06f130 con 0x7f028c06ea90 2023-10-06T05:21:19.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.761+0000 7f0291b9a700 1 --2- 172.21.15.31:0/2356283774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 0x7f028c06e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:19.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.761+0000 7f0291399700 1 -- 172.21.15.31:0/2356283774 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823157491 0 0) 0x7f028c06be70 con 0x7f028c13fb00 2023-10-06T05:21:19.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.761+0000 7f0292b9c700 1 --2- 172.21.15.31:0/2356283774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c06ea90 0x7f028c0cadb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0284009e60 tx=0x7f0284009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35eb3ebcf7591fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.761+0000 7f0291399700 1 -- 172.21.15.31:0/2356283774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028400e070 con 0x7f028c06ea90 2023-10-06T05:21:19.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.761+0000 7f0291399700 1 -- 172.21.15.31:0/2356283774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028400bc30 con 0x7f028c06ea90 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.761+0000 7f0291399700 1 -- 172.21.15.31:0/2356283774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284012430 con 0x7f028c06ea90 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.762+0000 7f02945ff700 1 -- 172.21.15.31:0/2356283774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c06ea90 msgr2=0x7f028c0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.762+0000 7f02945ff700 1 --2- 172.21.15.31:0/2356283774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c06ea90 0x7f028c0cadb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0284009e60 tx=0x7f0284009780 comp rx=0 tx=0).stop 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.762+0000 7f02945ff700 1 -- 172.21.15.31:0/2356283774 shutdown_connections 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.762+0000 7f02945ff700 1 --2- 172.21.15.31:0/2356283774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c06ea90 0x7f028c0cadb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.762+0000 7f02945ff700 1 --2- 172.21.15.31:0/2356283774 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 0x7f028c06e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.762+0000 7f02945ff700 1 -- 172.21.15.31:0/2356283774 >> 172.21.15.31:0/2356283774 conn(0x7f028c0cdcb0 msgr2=0x7f028c0c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.762+0000 7f02945ff700 1 -- 172.21.15.31:0/2356283774 shutdown_connections 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.762+0000 7f02945ff700 1 -- 172.21.15.31:0/2356283774 wait complete. 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.764+0000 7f02945ff700 1 Processor -- start 2023-10-06T05:21:19.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.764+0000 7f02945ff700 1 -- start start 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f02945ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c06ea90 0x7f028c0b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f02945ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c13fb00 0x7f028c15df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f02945ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 0x7f028c15e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f02945ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0aff80 con 0x7f028c13fb00 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f02945ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c15e980 con 0x7f028c06ea90 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f02945ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c169e40 con 0x7f028c13fed0 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f0291b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c13fb00 0x7f028c15df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f0291b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c13fb00 0x7f028c15df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60838/0 (socket says 172.21.15.31:60838) 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f0291b9a700 1 -- 172.21.15.31:0/3559935939 learned_addr learned my addr 172.21.15.31:0/3559935939 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f0292b9c700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 0x7f028c15e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f029239b700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c06ea90 0x7f028c0b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f0291b9a700 1 -- 172.21.15.31:0/3559935939 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 msgr2=0x7f028c15e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f0291b9a700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 0x7f028c15e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f0291b9a700 1 -- 172.21.15.31:0/3559935939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c06ea90 msgr2=0x7f028c0b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f0291b9a700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c06ea90 0x7f028c0b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f0291b9a700 1 -- 172.21.15.31:0/3559935939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0284009a20 con 0x7f028c13fb00 2023-10-06T05:21:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f0292b9c700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 0x7f028c15e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.765+0000 7f029239b700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c06ea90 0x7f028c0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.766+0000 7f0291b9a700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c13fb00 0x7f028c15df00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e920 tx=0x7f027c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.766+0000 7f02827fc700 1 -- 172.21.15.31:0/3559935939 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00ee60 con 0x7f028c13fb00 2023-10-06T05:21:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.766+0000 7f02827fc700 1 -- 172.21.15.31:0/3559935939 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c005a00 con 0x7f028c13fb00 2023-10-06T05:21:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.766+0000 7f02827fc700 1 -- 172.21.15.31:0/3559935939 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c005da0 con 0x7f028c13fb00 2023-10-06T05:21:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.766+0000 7f02945ff700 1 -- 172.21.15.31:0/3559935939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c16a380 con 0x7f028c13fb00 2023-10-06T05:21:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.766+0000 7f02945ff700 1 -- 172.21.15.31:0/3559935939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c0b2970 con 0x7f028c13fb00 2023-10-06T05:21:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.767+0000 7f02945ff700 1 -- 172.21.15.31:0/3559935939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c0b27d0 con 0x7f028c13fb00 2023-10-06T05:21:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.768+0000 7f02827fc700 1 -- 172.21.15.31:0/3559935939 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f027c006230 con 0x7f028c13fb00 2023-10-06T05:21:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.769+0000 7f02827fc700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0270067290 0x7f0270069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.769+0000 7f02827fc700 1 -- 172.21.15.31:0/3559935939 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f027c084e90 con 0x7f028c13fb00 2023-10-06T05:21:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.769+0000 7f029239b700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0270067290 0x7f0270069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.769+0000 7f029239b700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0270067290 0x7f0270069740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f02880018d0 tx=0x7f0288009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:19.773+0000 7f02827fc700 1 -- 172.21.15.31:0/3559935939 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f028c0b27d0 con 0x7f028c13fb00 2023-10-06T05:21:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.056+0000 7f02945ff700 1 -- 172.21.15.31:0/3559935939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f028c0b27d0 con 0x7f028c13fb00 2023-10-06T05:21:20.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.057+0000 7f02827fc700 1 -- 172.21.15.31:0/3559935939 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1085 (secure 0 0 0) 0x7f028c0b27d0 con 0x7f028c13fb00 2023-10-06T05:21:20.059 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:20.059 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":278,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":71,"num_osds":8,"num_up_osds":8,"osd_up_since":1696569413,"num_in_osds":8,"osd_in_since":1696569404,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":51},{"state_name":"active+clean","count":41},{"state_name":"creating+activating","count":13}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":2443288576,"bytes_avail":770650824704,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":44,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"d","status":"up:active","gid":6249}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-10-06T05:19:44.097775+0000","services":{}},"progress_events":{}} 2023-10-06T05:21:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 -- 172.21.15.31:0/3559935939 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0270067290 msgr2=0x7f0270069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0270067290 0x7f0270069740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f02880018d0 tx=0x7f0288009040 comp rx=0 tx=0).stop 2023-10-06T05:21:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 -- 172.21.15.31:0/3559935939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c13fb00 msgr2=0x7f028c15df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c13fb00 0x7f028c15df00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e920 tx=0x7f027c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 -- 172.21.15.31:0/3559935939 shutdown_connections 2023-10-06T05:21:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f028c13fed0 0x7f028c15e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c06ea90 0x7f028c0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f0270067290 0x7f0270069740 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 --2- 172.21.15.31:0/3559935939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f028c13fb00 0x7f028c15df00 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 -- 172.21.15.31:0/3559935939 >> 172.21.15.31:0/3559935939 conn(0x7f028c0cdcb0 msgr2=0x7f028c0c2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:20.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 -- 172.21.15.31:0/3559935939 shutdown_connections 2023-10-06T05:21:20.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.059+0000 7f02945ff700 1 -- 172.21.15.31:0/3559935939 wait complete. 2023-10-06T05:21:20.078 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T05:21:20.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.273+0000 7f2ef75ff700 1 Processor -- start 2023-10-06T05:21:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.273+0000 7f2ef75ff700 1 -- start start 2023-10-06T05:21:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.274+0000 7f2ef75ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef0068d80 0x7f2ef0159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.274+0000 7f2ef75ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ef00cc0d0 0x7f2ef00c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.274+0000 7f2ef75ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef00c95b0 con 0x7f2ef01595a0 2023-10-06T05:21:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.274+0000 7f2ef75ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef00c9890 con 0x7f2ef00cc0d0 2023-10-06T05:21:20.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.274+0000 7f2ef75ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00c9b70 con 0x7f2ef0068d80 2023-10-06T05:21:20.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.275+0000 7f2ef539b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2ef01595a0 0x7f2ef0068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35608/0 (socket says 172.21.15.31:35608) 2023-10-06T05:21:20.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.275+0000 7f2ef539b700 1 -- 172.21.15.31:0/2535657423 learned_addr learned my addr 172.21.15.31:0/2535657423 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:20.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.275+0000 7f2ef5b9c700 1 --2- 172.21.15.31:0/2535657423 >> v2:172.21.15.31:3300/0 conn(0x7f2ef00cc0d0 0x7f2ef00c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.275+0000 7f2ef4b9a700 1 --2- 172.21.15.31:0/2535657423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef0068d80 0x7f2ef0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.275+0000 7f2ee7fff700 1 -- 172.21.15.31:0/2535657423 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771320591 0 0) 0x7f2ef00c95b0 con 0x7f2ef01595a0 2023-10-06T05:21:20.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.275+0000 7f2ee7fff700 1 -- 172.21.15.31:0/2535657423 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2edc003570 con 0x7f2ef01595a0 2023-10-06T05:21:20.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.275+0000 7f2ef5b9c700 1 -- 172.21.15.31:0/2535657423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef0068d80 msgr2=0x7f2ef0159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:20.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.275+0000 7f2ef5b9c700 1 --2- 172.21.15.31:0/2535657423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef0068d80 0x7f2ef0159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.275+0000 7f2ef5b9c700 1 -- 172.21.15.31:0/2535657423 >> v1:172.21.15.31:6789/0 conn(0x7f2ef01595a0 legacy=0x7f2ef0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.275+0000 7f2ef5b9c700 1 -- 172.21.15.31:0/2535657423 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00ca130 con 0x7f2ef00cc0d0 2023-10-06T05:21:20.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ef5b9c700 1 --2- 172.21.15.31:0/2535657423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef00cc0d0 0x7f2ef00c8930 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009e60 tx=0x7f2ee8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d499234d51584777 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:20.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ee7fff700 1 -- 172.21.15.31:0/2535657423 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800e070 con 0x7f2ef00cc0d0 2023-10-06T05:21:20.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ee7fff700 1 -- 172.21.15.31:0/2535657423 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee800bb60 con 0x7f2ef00cc0d0 2023-10-06T05:21:20.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ee7fff700 1 -- 172.21.15.31:0/2535657423 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8012430 con 0x7f2ef00cc0d0 2023-10-06T05:21:20.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ef75ff700 1 -- 172.21.15.31:0/2535657423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef00cc0d0 msgr2=0x7f2ef00c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ef75ff700 1 --2- 172.21.15.31:0/2535657423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef00cc0d0 0x7f2ef00c8930 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009e60 tx=0x7f2ee8009780 comp rx=0 tx=0).stop 2023-10-06T05:21:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ef75ff700 1 -- 172.21.15.31:0/2535657423 shutdown_connections 2023-10-06T05:21:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ef75ff700 1 --2- 172.21.15.31:0/2535657423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef00cc0d0 0x7f2ef00c8930 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ef75ff700 1 --2- 172.21.15.31:0/2535657423 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef0068d80 0x7f2ef0159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ef75ff700 1 -- 172.21.15.31:0/2535657423 >> 172.21.15.31:0/2535657423 conn(0x7f2ef006b620 msgr2=0x7f2ef006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.276+0000 7f2ef75ff700 1 -- 172.21.15.31:0/2535657423 shutdown_connections 2023-10-06T05:21:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.277+0000 7f2ef75ff700 1 -- 172.21.15.31:0/2535657423 wait complete. 2023-10-06T05:21:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.278+0000 7f2ef75ff700 1 Processor -- start 2023-10-06T05:21:20.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.278+0000 7f2ef75ff700 1 -- start start 2023-10-06T05:21:20.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.279+0000 7f2ef75ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef0068d80 0x7f2ef00c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.279+0000 7f2ef75ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef00cc0d0 0x7f2ef0153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.279+0000 7f2ef75ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ef01595a0 0x7f2ef0156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.279+0000 7f2ef75ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef015b450 con 0x7f2ef0068d80 2023-10-06T05:21:20.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.279+0000 7f2ef75ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00b3f30 con 0x7f2ef01595a0 2023-10-06T05:21:20.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.279+0000 7f2ef75ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00c6ab0 con 0x7f2ef00cc0d0 2023-10-06T05:21:20.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.279+0000 7f2ef539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef0068d80 0x7f2ef00c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.279+0000 7f2ef539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef0068d80 0x7f2ef00c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60868/0 (socket says 172.21.15.31:60868) 2023-10-06T05:21:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.279+0000 7f2ef539b700 1 -- 172.21.15.31:0/1654602034 learned_addr learned my addr 172.21.15.31:0/1654602034 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef4b9a700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef00cc0d0 0x7f2ef0153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef539b700 1 -- 172.21.15.31:0/1654602034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef00cc0d0 msgr2=0x7f2ef0153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef539b700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef00cc0d0 0x7f2ef0153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef539b700 1 -- 172.21.15.31:0/1654602034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ef01595a0 msgr2=0x7f2ef0156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef5b9c700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ef01595a0 0x7f2ef0156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef539b700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ef01595a0 0x7f2ef0156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef539b700 1 -- 172.21.15.31:0/1654602034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee8009a20 con 0x7f2ef0068d80 2023-10-06T05:21:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef4b9a700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef00cc0d0 0x7f2ef0153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef5b9c700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ef01595a0 0x7f2ef0156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef539b700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef0068d80 0x7f2ef00c64d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c7c0 tx=0x7f2eec007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ee57fa700 1 -- 172.21.15.31:0/1654602034 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec005b30 con 0x7f2ef0068d80 2023-10-06T05:21:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ee57fa700 1 -- 172.21.15.31:0/1654602034 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec005c90 con 0x7f2ef0068d80 2023-10-06T05:21:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ee57fa700 1 -- 172.21.15.31:0/1654602034 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec007340 con 0x7f2ef0068d80 2023-10-06T05:21:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.280+0000 7f2ef75ff700 1 -- 172.21.15.31:0/1654602034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef00c6d90 con 0x7f2ef0068d80 2023-10-06T05:21:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.281+0000 7f2ef75ff700 1 -- 172.21.15.31:0/1654602034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef00c7350 con 0x7f2ef0068d80 2023-10-06T05:21:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.281+0000 7f2ef75ff700 1 -- 172.21.15.31:0/1654602034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef004c380 con 0x7f2ef0068d80 2023-10-06T05:21:20.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.282+0000 7f2ee57fa700 1 -- 172.21.15.31:0/1654602034 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2eec01b400 con 0x7f2ef0068d80 2023-10-06T05:21:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.283+0000 7f2ee57fa700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed4067290 0x7f2ed4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.283+0000 7f2ee57fa700 1 -- 172.21.15.31:0/1654602034 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2eec0848f0 con 0x7f2ef0068d80 2023-10-06T05:21:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.283+0000 7f2ef4b9a700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed4067290 0x7f2ed4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:20.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.284+0000 7f2ef4b9a700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed4067290 0x7f2ed4069740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00cccf0 tx=0x7f2ee0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.287+0000 7f2ee57fa700 1 -- 172.21.15.31:0/1654602034 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2eec051550 con 0x7f2ef0068d80 2023-10-06T05:21:20.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.485+0000 7f2ef75ff700 1 -- 172.21.15.31:0/1654602034 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ef00c9230 con 0x7f2ed4067290 2023-10-06T05:21:20.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.495+0000 7f2ee57fa700 1 -- 172.21.15.31:0/1654602034 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311340 (secure 0 0 0) 0x7f2ef00c9230 con 0x7f2ed4067290 2023-10-06T05:21:20.500 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:20.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.499+0000 7f2ef75ff700 1 -- 172.21.15.31:0/1654602034 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed4067290 msgr2=0x7f2ed4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.499+0000 7f2ef75ff700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed4067290 0x7f2ed4069740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00cccf0 tx=0x7f2ee0008040 comp rx=0 tx=0).stop 2023-10-06T05:21:20.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.499+0000 7f2ef75ff700 1 -- 172.21.15.31:0/1654602034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef0068d80 msgr2=0x7f2ef00c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:20.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.499+0000 7f2ef75ff700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef0068d80 0x7f2ef00c64d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c7c0 tx=0x7f2eec007e00 comp rx=0 tx=0).stop 2023-10-06T05:21:20.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.499+0000 7f2ef75ff700 1 -- 172.21.15.31:0/1654602034 shutdown_connections 2023-10-06T05:21:20.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.499+0000 7f2ef75ff700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ef01595a0 0x7f2ef0156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.499+0000 7f2ef75ff700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2ed4067290 0x7f2ed4069740 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.499+0000 7f2ef75ff700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef0068d80 0x7f2ef00c64d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.499+0000 7f2ef75ff700 1 --2- 172.21.15.31:0/1654602034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ef00cc0d0 0x7f2ef0153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:20.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.499+0000 7f2ef75ff700 1 -- 172.21.15.31:0/1654602034 >> 172.21.15.31:0/1654602034 conn(0x7f2ef006b620 msgr2=0x7f2ef006cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:20.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.500+0000 7f2ef75ff700 1 -- 172.21.15.31:0/1654602034 shutdown_connections 2023-10-06T05:21:20.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:20.500+0000 7f2ef75ff700 1 -- 172.21.15.31:0/1654602034 wait complete. 2023-10-06T05:21:20.504 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:21:20.519 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2023-10-06T05:21:20.118687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386280,"kb_used_data":7976,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752588440,"statfs":{"total":773094113280,"available":770650562560,"internally_reserved":0,"allocated":8167424,"data_stored":4273388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.969054"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.184768+0000","last_change":"2023-10-06T05:21:14.182605+0000","last_active":"2023-10-06T05:21:14.184768+0000","last_peered":"2023-10-06T05:21:14.184768+0000","last_clean":"2023-10-06T05:21:14.184768+0000","last_became_active":"2023-10-06T05:21:14.182308+0000","last_became_peered":"2023-10-06T05:21:14.182308+0000","last_unstale":"2023-10-06T05:21:14.184768+0000","last_undegraded":"2023-10-06T05:21:14.184768+0000","last_fullsized":"2023-10-06T05:21:14.184768+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:36.647693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.180147+0000","last_change":"2023-10-06T05:21:14.174918+0000","last_active":"2023-10-06T05:21:14.180147+0000","last_peered":"2023-10-06T05:21:14.180147+0000","last_clean":"2023-10-06T05:21:14.180147+0000","last_became_active":"2023-10-06T05:21:14.174566+0000","last_became_peered":"2023-10-06T05:21:14.174566+0000","last_unstale":"2023-10-06T05:21:14.180147+0000","last_undegraded":"2023-10-06T05:21:14.180147+0000","last_fullsized":"2023-10-06T05:21:14.180147+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:09.628005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178625+0000","last_change":"2023-10-06T05:21:14.174684+0000","last_active":"2023-10-06T05:21:14.178625+0000","last_peered":"2023-10-06T05:21:14.178625+0000","last_clean":"2023-10-06T05:21:14.178625+0000","last_became_active":"2023-10-06T05:21:14.174370+0000","last_became_peered":"2023-10-06T05:21:14.174370+0000","last_unstale":"2023-10-06T05:21:14.178625+0000","last_undegraded":"2023-10-06T05:21:14.178625+0000","last_fullsized":"2023-10-06T05:21:14.178625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:32.078654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.185470+0000","last_change":"2023-10-06T05:21:14.177280+0000","last_active":"2023-10-06T05:21:14.185470+0000","last_peered":"2023-10-06T05:21:14.185470+0000","last_clean":"2023-10-06T05:21:14.185470+0000","last_became_active":"2023-10-06T05:21:14.176617+0000","last_became_peered":"2023-10-06T05:21:14.176617+0000","last_unstale":"2023-10-06T05:21:14.185470+0000","last_undegraded":"2023-10-06T05:21:14.185470+0000","last_fullsized":"2023-10-06T05:21:14.185470+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:57.160327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178291+0000","last_change":"2023-10-06T05:21:14.171897+0000","last_active":"2023-10-06T05:21:14.178291+0000","last_peered":"2023-10-06T05:21:14.178291+0000","last_clean":"2023-10-06T05:21:14.178291+0000","last_became_active":"2023-10-06T05:21:14.171542+0000","last_became_peered":"2023-10-06T05:21:14.171542+0000","last_unstale":"2023-10-06T05:21:14.178291+0000","last_undegraded":"2023-10-06T05:21:14.178291+0000","last_fullsized":"2023-10-06T05:21:14.178291+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:08.721924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188396+0000","last_change":"2023-10-06T05:21:14.178877+0000","last_active":"2023-10-06T05:21:14.188396+0000","last_peered":"2023-10-06T05:21:14.188396+0000","last_clean":"2023-10-06T05:21:14.188396+0000","last_became_active":"2023-10-06T05:21:14.178036+0000","last_became_peered":"2023-10-06T05:21:14.178036+0000","last_unstale":"2023-10-06T05:21:14.188396+0000","last_undegraded":"2023-10-06T05:21:14.188396+0000","last_fullsized":"2023-10-06T05:21:14.188396+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:40.622847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.179117+0000","last_change":"2023-10-06T05:21:14.176248+0000","last_active":"2023-10-06T05:21:14.179117+0000","last_peered":"2023-10-06T05:21:14.179117+0000","last_clean":"2023-10-06T05:21:14.179117+0000","last_became_active":"2023-10-06T05:21:14.175935+0000","last_became_peered":"2023-10-06T05:21:14.175935+0000","last_unstale":"2023-10-06T05:21:14.179117+0000","last_undegraded":"2023-10-06T05:21:14.179117+0000","last_fullsized":"2023-10-06T05:21:14.179117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:21.116415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.170189+0000","last_change":"2023-10-06T05:21:14.167168+0000","last_active":"2023-10-06T05:21:14.170189+0000","last_peered":"2023-10-06T05:21:14.170189+0000","last_clean":"2023-10-06T05:21:14.170189+0000","last_became_active":"2023-10-06T05:21:14.166894+0000","last_became_peered":"2023-10-06T05:21:14.166894+0000","last_unstale":"2023-10-06T05:21:14.170189+0000","last_undegraded":"2023-10-06T05:21:14.170189+0000","last_fullsized":"2023-10-06T05:21:14.170189+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:33.508458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.172744+0000","last_change":"2023-10-06T05:21:14.165341+0000","last_active":"2023-10-06T05:21:14.172744+0000","last_peered":"2023-10-06T05:21:14.172744+0000","last_clean":"2023-10-06T05:21:14.172744+0000","last_became_active":"2023-10-06T05:21:14.165037+0000","last_became_peered":"2023-10-06T05:21:14.165037+0000","last_unstale":"2023-10-06T05:21:14.172744+0000","last_undegraded":"2023-10-06T05:21:14.172744+0000","last_fullsized":"2023-10-06T05:21:14.172744+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:04.649668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.185976+0000","last_change":"2023-10-06T05:21:14.184650+0000","last_active":"2023-10-06T05:21:14.185976+0000","last_peered":"2023-10-06T05:21:14.185976+0000","last_clean":"2023-10-06T05:21:14.185976+0000","last_became_active":"2023-10-06T05:21:14.184407+0000","last_became_peered":"2023-10-06T05:21:14.184407+0000","last_unstale":"2023-10-06T05:21:14.185976+0000","last_undegraded":"2023-10-06T05:21:14.185976+0000","last_fullsized":"2023-10-06T05:21:14.185976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:55.793132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.187819+0000","last_change":"2023-10-06T05:21:14.176765+0000","last_active":"2023-10-06T05:21:14.187819+0000","last_peered":"2023-10-06T05:21:14.187819+0000","last_clean":"2023-10-06T05:21:14.187819+0000","last_became_active":"2023-10-06T05:21:14.175854+0000","last_became_peered":"2023-10-06T05:21:14.175854+0000","last_unstale":"2023-10-06T05:21:14.187819+0000","last_undegraded":"2023-10-06T05:21:14.187819+0000","last_fullsized":"2023-10-06T05:21:14.187819+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:33.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.185450+0000","last_change":"2023-10-06T05:21:14.177074+0000","last_active":"2023-10-06T05:21:14.185450+0000","last_peered":"2023-10-06T05:21:14.185450+0000","last_clean":"2023-10-06T05:21:14.185450+0000","last_became_active":"2023-10-06T05:21:14.176305+0000","last_became_peered":"2023-10-06T05:21:14.176305+0000","last_unstale":"2023-10-06T05:21:14.185450+0000","last_undegraded":"2023-10-06T05:21:14.185450+0000","last_fullsized":"2023-10-06T05:21:14.185450+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:16.811458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.174113+0000","last_change":"2023-10-06T05:21:14.169942+0000","last_active":"2023-10-06T05:21:14.174113+0000","last_peered":"2023-10-06T05:21:14.174113+0000","last_clean":"2023-10-06T05:21:14.174113+0000","last_became_active":"2023-10-06T05:21:14.169627+0000","last_became_peered":"2023-10-06T05:21:14.169627+0000","last_unstale":"2023-10-06T05:21:14.174113+0000","last_undegraded":"2023-10-06T05:21:14.174113+0000","last_fullsized":"2023-10-06T05:21:14.174113+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:20.105525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188375+0000","last_change":"2023-10-06T05:21:14.178776+0000","last_active":"2023-10-06T05:21:14.188375+0000","last_peered":"2023-10-06T05:21:14.188375+0000","last_clean":"2023-10-06T05:21:14.188375+0000","last_became_active":"2023-10-06T05:21:14.177691+0000","last_became_peered":"2023-10-06T05:21:14.177691+0000","last_unstale":"2023-10-06T05:21:14.188375+0000","last_undegraded":"2023-10-06T05:21:14.188375+0000","last_fullsized":"2023-10-06T05:21:14.188375+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:26.471873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.176705+0000","last_change":"2023-10-06T05:21:14.172116+0000","last_active":"2023-10-06T05:21:14.176705+0000","last_peered":"2023-10-06T05:21:14.176705+0000","last_clean":"2023-10-06T05:21:14.176705+0000","last_became_active":"2023-10-06T05:21:14.171841+0000","last_became_peered":"2023-10-06T05:21:14.171841+0000","last_unstale":"2023-10-06T05:21:14.176705+0000","last_undegraded":"2023-10-06T05:21:14.176705+0000","last_fullsized":"2023-10-06T05:21:14.176705+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:44.869933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.189318+0000","last_change":"2023-10-06T05:21:14.186797+0000","last_active":"2023-10-06T05:21:14.189318+0000","last_peered":"2023-10-06T05:21:14.189318+0000","last_clean":"2023-10-06T05:21:14.189318+0000","last_became_active":"2023-10-06T05:21:14.186495+0000","last_became_peered":"2023-10-06T05:21:14.186495+0000","last_unstale":"2023-10-06T05:21:14.189318+0000","last_undegraded":"2023-10-06T05:21:14.189318+0000","last_fullsized":"2023-10-06T05:21:14.189318+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:11.954285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.187062+0000","last_change":"2023-10-06T05:21:14.178718+0000","last_active":"2023-10-06T05:21:14.187062+0000","last_peered":"2023-10-06T05:21:14.187062+0000","last_clean":"2023-10-06T05:21:14.187062+0000","last_became_active":"2023-10-06T05:21:14.178204+0000","last_became_peered":"2023-10-06T05:21:14.178204+0000","last_unstale":"2023-10-06T05:21:14.187062+0000","last_undegraded":"2023-10-06T05:21:14.187062+0000","last_fullsized":"2023-10-06T05:21:14.187062+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:34.358634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177158+0000","last_change":"2023-10-06T05:21:14.173162+0000","last_active":"2023-10-06T05:21:14.177158+0000","last_peered":"2023-10-06T05:21:14.177158+0000","last_clean":"2023-10-06T05:21:14.177158+0000","last_became_active":"2023-10-06T05:21:14.172561+0000","last_became_peered":"2023-10-06T05:21:14.172561+0000","last_unstale":"2023-10-06T05:21:14.177158+0000","last_undegraded":"2023-10-06T05:21:14.177158+0000","last_fullsized":"2023-10-06T05:21:14.177158+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:58.200257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178576+0000","last_change":"2023-10-06T05:21:14.172747+0000","last_active":"2023-10-06T05:21:14.178576+0000","last_peered":"2023-10-06T05:21:14.178576+0000","last_clean":"2023-10-06T05:21:14.178576+0000","last_became_active":"2023-10-06T05:21:14.172482+0000","last_became_peered":"2023-10-06T05:21:14.172482+0000","last_unstale":"2023-10-06T05:21:14.178576+0000","last_undegraded":"2023-10-06T05:21:14.178576+0000","last_fullsized":"2023-10-06T05:21:14.178576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:33.730974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178577+0000","last_change":"2023-10-06T05:21:14.170904+0000","last_active":"2023-10-06T05:21:14.178577+0000","last_peered":"2023-10-06T05:21:14.178577+0000","last_clean":"2023-10-06T05:21:14.178577+0000","last_became_active":"2023-10-06T05:21:14.170585+0000","last_became_peered":"2023-10-06T05:21:14.170585+0000","last_unstale":"2023-10-06T05:21:14.178577+0000","last_undegraded":"2023-10-06T05:21:14.178577+0000","last_fullsized":"2023-10-06T05:21:14.178577+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:05.216268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.181042+0000","last_change":"2023-10-06T05:21:14.173743+0000","last_active":"2023-10-06T05:21:14.181042+0000","last_peered":"2023-10-06T05:21:14.181042+0000","last_clean":"2023-10-06T05:21:14.181042+0000","last_became_active":"2023-10-06T05:21:14.173451+0000","last_became_peered":"2023-10-06T05:21:14.173451+0000","last_unstale":"2023-10-06T05:21:14.181042+0000","last_undegraded":"2023-10-06T05:21:14.181042+0000","last_fullsized":"2023-10-06T05:21:14.181042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:43.694140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178096+0000","last_change":"2023-10-06T05:21:14.170792+0000","last_active":"2023-10-06T05:21:14.178096+0000","last_peered":"2023-10-06T05:21:14.178096+0000","last_clean":"2023-10-06T05:21:14.178096+0000","last_became_active":"2023-10-06T05:21:14.170472+0000","last_became_peered":"2023-10-06T05:21:14.170472+0000","last_unstale":"2023-10-06T05:21:14.178096+0000","last_undegraded":"2023-10-06T05:21:14.178096+0000","last_fullsized":"2023-10-06T05:21:14.178096+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:12.167839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.190939+0000","last_change":"2023-10-06T05:21:14.187686+0000","last_active":"2023-10-06T05:21:14.190939+0000","last_peered":"2023-10-06T05:21:14.190939+0000","last_clean":"2023-10-06T05:21:14.190939+0000","last_became_active":"2023-10-06T05:21:14.187152+0000","last_became_peered":"2023-10-06T05:21:14.187152+0000","last_unstale":"2023-10-06T05:21:14.190939+0000","last_undegraded":"2023-10-06T05:21:14.190939+0000","last_fullsized":"2023-10-06T05:21:14.190939+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:48.387423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.176142+0000","last_change":"2023-10-06T05:21:14.170515+0000","last_active":"2023-10-06T05:21:14.176142+0000","last_peered":"2023-10-06T05:21:14.176142+0000","last_clean":"2023-10-06T05:21:14.176142+0000","last_became_active":"2023-10-06T05:21:14.170160+0000","last_became_peered":"2023-10-06T05:21:14.170160+0000","last_unstale":"2023-10-06T05:21:14.176142+0000","last_undegraded":"2023-10-06T05:21:14.176142+0000","last_fullsized":"2023-10-06T05:21:14.176142+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:40.977536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.187988+0000","last_change":"2023-10-06T05:21:14.182544+0000","last_active":"2023-10-06T05:21:14.187988+0000","last_peered":"2023-10-06T05:21:14.187988+0000","last_clean":"2023-10-06T05:21:14.187988+0000","last_became_active":"2023-10-06T05:21:14.182130+0000","last_became_peered":"2023-10-06T05:21:14.182130+0000","last_unstale":"2023-10-06T05:21:14.187988+0000","last_undegraded":"2023-10-06T05:21:14.187988+0000","last_fullsized":"2023-10-06T05:21:14.187988+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:11.954285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.184779+0000","last_change":"2023-10-06T05:21:14.177784+0000","last_active":"2023-10-06T05:21:14.184779+0000","last_peered":"2023-10-06T05:21:14.184779+0000","last_clean":"2023-10-06T05:21:14.184779+0000","last_became_active":"2023-10-06T05:21:14.177508+0000","last_became_peered":"2023-10-06T05:21:14.177508+0000","last_unstale":"2023-10-06T05:21:14.184779+0000","last_undegraded":"2023-10-06T05:21:14.184779+0000","last_fullsized":"2023-10-06T05:21:14.184779+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:50.021472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.189466+0000","last_change":"2023-10-06T05:21:14.187124+0000","last_active":"2023-10-06T05:21:14.189466+0000","last_peered":"2023-10-06T05:21:14.189466+0000","last_clean":"2023-10-06T05:21:14.189466+0000","last_became_active":"2023-10-06T05:21:14.186793+0000","last_became_peered":"2023-10-06T05:21:14.186793+0000","last_unstale":"2023-10-06T05:21:14.189466+0000","last_undegraded":"2023-10-06T05:21:14.189466+0000","last_fullsized":"2023-10-06T05:21:14.189466+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:44.869933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188474+0000","last_change":"2023-10-06T05:21:14.179486+0000","last_active":"2023-10-06T05:21:14.188474+0000","last_peered":"2023-10-06T05:21:14.188474+0000","last_clean":"2023-10-06T05:21:14.188474+0000","last_became_active":"2023-10-06T05:21:14.179157+0000","last_became_peered":"2023-10-06T05:21:14.179157+0000","last_unstale":"2023-10-06T05:21:14.188474+0000","last_undegraded":"2023-10-06T05:21:14.188474+0000","last_fullsized":"2023-10-06T05:21:14.188474+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:24.135963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.187799+0000","last_change":"2023-10-06T05:21:14.183092+0000","last_active":"2023-10-06T05:21:14.187799+0000","last_peered":"2023-10-06T05:21:14.187799+0000","last_clean":"2023-10-06T05:21:14.187799+0000","last_became_active":"2023-10-06T05:21:14.182773+0000","last_became_peered":"2023-10-06T05:21:14.182773+0000","last_unstale":"2023-10-06T05:21:14.187799+0000","last_undegraded":"2023-10-06T05:21:14.187799+0000","last_fullsized":"2023-10-06T05:21:14.187799+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:36.591514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.189695+0000","last_change":"2023-10-06T05:21:14.183911+0000","last_active":"2023-10-06T05:21:14.189695+0000","last_peered":"2023-10-06T05:21:14.189695+0000","last_clean":"2023-10-06T05:21:14.189695+0000","last_became_active":"2023-10-06T05:21:14.183581+0000","last_became_peered":"2023-10-06T05:21:14.183581+0000","last_unstale":"2023-10-06T05:21:14.189695+0000","last_undegraded":"2023-10-06T05:21:14.189695+0000","last_fullsized":"2023-10-06T05:21:14.189695+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:09.678269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178282+0000","last_change":"2023-10-06T05:21:14.176614+0000","last_active":"2023-10-06T05:21:14.178282+0000","last_peered":"2023-10-06T05:21:14.178282+0000","last_clean":"2023-10-06T05:21:14.178282+0000","last_became_active":"2023-10-06T05:21:14.176306+0000","last_became_peered":"2023-10-06T05:21:14.176306+0000","last_unstale":"2023-10-06T05:21:14.178282+0000","last_undegraded":"2023-10-06T05:21:14.178282+0000","last_fullsized":"2023-10-06T05:21:14.178282+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:55.307288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.182824+0000","last_change":"2023-10-06T05:21:14.180752+0000","last_active":"2023-10-06T05:21:14.182824+0000","last_peered":"2023-10-06T05:21:14.182824+0000","last_clean":"2023-10-06T05:21:14.182824+0000","last_became_active":"2023-10-06T05:21:14.180370+0000","last_became_peered":"2023-10-06T05:21:14.180370+0000","last_unstale":"2023-10-06T05:21:14.182824+0000","last_undegraded":"2023-10-06T05:21:14.182824+0000","last_fullsized":"2023-10-06T05:21:14.182824+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:09.628005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177051+0000","last_change":"2023-10-06T05:21:14.172625+0000","last_active":"2023-10-06T05:21:14.177051+0000","last_peered":"2023-10-06T05:21:14.177051+0000","last_clean":"2023-10-06T05:21:14.177051+0000","last_became_active":"2023-10-06T05:21:14.172292+0000","last_became_peered":"2023-10-06T05:21:14.172292+0000","last_unstale":"2023-10-06T05:21:14.177051+0000","last_undegraded":"2023-10-06T05:21:14.177051+0000","last_fullsized":"2023-10-06T05:21:14.177051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:41.885766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"70'4","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191330+0000","last_change":"2023-10-06T05:21:13.159241+0000","last_active":"2023-10-06T05:21:14.191330+0000","last_peered":"2023-10-06T05:21:14.191330+0000","last_clean":"2023-10-06T05:21:14.191330+0000","last_became_active":"2023-10-06T05:21:13.158913+0000","last_became_peered":"2023-10-06T05:21:13.158913+0000","last_unstale":"2023-10-06T05:21:14.191330+0000","last_undegraded":"2023-10-06T05:21:14.191330+0000","last_fullsized":"2023-10-06T05:21:14.191330+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:58.015087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.190713+0000","last_change":"2023-10-06T05:21:14.186435+0000","last_active":"2023-10-06T05:21:14.190713+0000","last_peered":"2023-10-06T05:21:14.190713+0000","last_clean":"2023-10-06T05:21:14.190713+0000","last_became_active":"2023-10-06T05:21:14.186117+0000","last_became_peered":"2023-10-06T05:21:14.186117+0000","last_unstale":"2023-10-06T05:21:14.190713+0000","last_undegraded":"2023-10-06T05:21:14.190713+0000","last_fullsized":"2023-10-06T05:21:14.190713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:09.678269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188865+0000","last_change":"2023-10-06T05:21:13.171676+0000","last_active":"2023-10-06T05:21:14.188865+0000","last_peered":"2023-10-06T05:21:14.188865+0000","last_clean":"2023-10-06T05:21:14.188865+0000","last_became_active":"2023-10-06T05:21:13.171360+0000","last_became_peered":"2023-10-06T05:21:13.171360+0000","last_unstale":"2023-10-06T05:21:14.188865+0000","last_undegraded":"2023-10-06T05:21:14.188865+0000","last_fullsized":"2023-10-06T05:21:14.188865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:24.757454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.172558+0000","last_change":"2023-10-06T05:21:14.165541+0000","last_active":"2023-10-06T05:21:14.172558+0000","last_peered":"2023-10-06T05:21:14.172558+0000","last_clean":"2023-10-06T05:21:14.172558+0000","last_became_active":"2023-10-06T05:21:14.164858+0000","last_became_peered":"2023-10-06T05:21:14.164858+0000","last_unstale":"2023-10-06T05:21:14.172558+0000","last_undegraded":"2023-10-06T05:21:14.172558+0000","last_fullsized":"2023-10-06T05:21:14.172558+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:48.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188676+0000","last_change":"2023-10-06T05:21:13.176256+0000","last_active":"2023-10-06T05:21:14.188676+0000","last_peered":"2023-10-06T05:21:14.188676+0000","last_clean":"2023-10-06T05:21:14.188676+0000","last_became_active":"2023-10-06T05:21:13.175487+0000","last_became_peered":"2023-10-06T05:21:13.175487+0000","last_unstale":"2023-10-06T05:21:14.188676+0000","last_undegraded":"2023-10-06T05:21:14.188676+0000","last_fullsized":"2023-10-06T05:21:14.188676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:19.500524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177411+0000","last_change":"2023-10-06T05:21:14.172859+0000","last_active":"2023-10-06T05:21:14.177411+0000","last_peered":"2023-10-06T05:21:14.177411+0000","last_clean":"2023-10-06T05:21:14.177411+0000","last_became_active":"2023-10-06T05:21:14.172537+0000","last_became_peered":"2023-10-06T05:21:14.172537+0000","last_unstale":"2023-10-06T05:21:14.177411+0000","last_undegraded":"2023-10-06T05:21:14.177411+0000","last_fullsized":"2023-10-06T05:21:14.177411+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:09.170341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.189543+0000","last_change":"2023-10-06T05:21:13.169221+0000","last_active":"2023-10-06T05:21:14.189543+0000","last_peered":"2023-10-06T05:21:14.189543+0000","last_clean":"2023-10-06T05:21:14.189543+0000","last_became_active":"2023-10-06T05:21:13.168853+0000","last_became_peered":"2023-10-06T05:21:13.168853+0000","last_unstale":"2023-10-06T05:21:14.189543+0000","last_undegraded":"2023-10-06T05:21:14.189543+0000","last_fullsized":"2023-10-06T05:21:14.189543+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:12.290741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.186550+0000","last_change":"2023-10-06T05:21:14.175452+0000","last_active":"2023-10-06T05:21:14.186550+0000","last_peered":"2023-10-06T05:21:14.186550+0000","last_clean":"2023-10-06T05:21:14.186550+0000","last_became_active":"2023-10-06T05:21:14.175187+0000","last_became_peered":"2023-10-06T05:21:14.175187+0000","last_unstale":"2023-10-06T05:21:14.186550+0000","last_undegraded":"2023-10-06T05:21:14.186550+0000","last_fullsized":"2023-10-06T05:21:14.186550+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:58.329878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.185249+0000","last_change":"2023-10-06T05:21:13.162613+0000","last_active":"2023-10-06T05:21:14.185249+0000","last_peered":"2023-10-06T05:21:14.185249+0000","last_clean":"2023-10-06T05:21:14.185249+0000","last_became_active":"2023-10-06T05:21:13.162167+0000","last_became_peered":"2023-10-06T05:21:13.162167+0000","last_unstale":"2023-10-06T05:21:14.185249+0000","last_undegraded":"2023-10-06T05:21:14.185249+0000","last_fullsized":"2023-10-06T05:21:14.185249+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:36.834735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.186354+0000","last_change":"2023-10-06T05:21:14.176595+0000","last_active":"2023-10-06T05:21:14.186354+0000","last_peered":"2023-10-06T05:21:14.186354+0000","last_clean":"2023-10-06T05:21:14.186354+0000","last_became_active":"2023-10-06T05:21:14.176255+0000","last_became_peered":"2023-10-06T05:21:14.176255+0000","last_unstale":"2023-10-06T05:21:14.186354+0000","last_undegraded":"2023-10-06T05:21:14.186354+0000","last_fullsized":"2023-10-06T05:21:14.186354+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:13.467491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191089+0000","last_change":"2023-10-06T05:21:13.170589+0000","last_active":"2023-10-06T05:21:14.191089+0000","last_peered":"2023-10-06T05:21:14.191089+0000","last_clean":"2023-10-06T05:21:14.191089+0000","last_became_active":"2023-10-06T05:21:13.170282+0000","last_became_peered":"2023-10-06T05:21:13.170282+0000","last_unstale":"2023-10-06T05:21:14.191089+0000","last_undegraded":"2023-10-06T05:21:14.191089+0000","last_fullsized":"2023-10-06T05:21:14.191089+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:53.541975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.173404+0000","last_change":"2023-10-06T05:21:14.166677+0000","last_active":"2023-10-06T05:21:14.173404+0000","last_peered":"2023-10-06T05:21:14.173404+0000","last_clean":"2023-10-06T05:21:14.173404+0000","last_became_active":"2023-10-06T05:21:14.165665+0000","last_became_peered":"2023-10-06T05:21:14.165665+0000","last_unstale":"2023-10-06T05:21:14.173404+0000","last_undegraded":"2023-10-06T05:21:14.173404+0000","last_fullsized":"2023-10-06T05:21:14.173404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:54.544554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.164878+0000","last_change":"2023-10-06T05:21:13.175791+0000","last_active":"2023-10-06T05:21:14.164878+0000","last_peered":"2023-10-06T05:21:14.164878+0000","last_clean":"2023-10-06T05:21:14.164878+0000","last_became_active":"2023-10-06T05:21:13.175326+0000","last_became_peered":"2023-10-06T05:21:13.175326+0000","last_unstale":"2023-10-06T05:21:14.164878+0000","last_undegraded":"2023-10-06T05:21:14.164878+0000","last_fullsized":"2023-10-06T05:21:14.164878+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.040745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178777+0000","last_change":"2023-10-06T05:21:14.175687+0000","last_active":"2023-10-06T05:21:14.178777+0000","last_peered":"2023-10-06T05:21:14.178777+0000","last_clean":"2023-10-06T05:21:14.178777+0000","last_became_active":"2023-10-06T05:21:14.175356+0000","last_became_peered":"2023-10-06T05:21:14.175356+0000","last_unstale":"2023-10-06T05:21:14.178777+0000","last_undegraded":"2023-10-06T05:21:14.178777+0000","last_fullsized":"2023-10-06T05:21:14.178777+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:37.837314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.160846+0000","last_change":"2023-10-06T05:21:13.170180+0000","last_active":"2023-10-06T05:21:14.160846+0000","last_peered":"2023-10-06T05:21:14.160846+0000","last_clean":"2023-10-06T05:21:14.160846+0000","last_became_active":"2023-10-06T05:21:13.169857+0000","last_became_peered":"2023-10-06T05:21:13.169857+0000","last_unstale":"2023-10-06T05:21:14.160846+0000","last_undegraded":"2023-10-06T05:21:14.160846+0000","last_fullsized":"2023-10-06T05:21:14.160846+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:45.593601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.179005+0000","last_change":"2023-10-06T05:21:14.173085+0000","last_active":"2023-10-06T05:21:14.179005+0000","last_peered":"2023-10-06T05:21:14.179005+0000","last_clean":"2023-10-06T05:21:14.179005+0000","last_became_active":"2023-10-06T05:21:14.172809+0000","last_became_peered":"2023-10-06T05:21:14.172809+0000","last_unstale":"2023-10-06T05:21:14.179005+0000","last_undegraded":"2023-10-06T05:21:14.179005+0000","last_fullsized":"2023-10-06T05:21:14.179005+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.976026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.165824+0000","last_change":"2023-10-06T05:21:13.172453+0000","last_active":"2023-10-06T05:21:14.165824+0000","last_peered":"2023-10-06T05:21:14.165824+0000","last_clean":"2023-10-06T05:21:14.165824+0000","last_became_active":"2023-10-06T05:21:13.171973+0000","last_became_peered":"2023-10-06T05:21:13.171973+0000","last_unstale":"2023-10-06T05:21:14.165824+0000","last_undegraded":"2023-10-06T05:21:14.165824+0000","last_fullsized":"2023-10-06T05:21:14.165824+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:53.470594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.186146+0000","last_change":"2023-10-06T05:21:14.184231+0000","last_active":"2023-10-06T05:21:14.186146+0000","last_peered":"2023-10-06T05:21:14.186146+0000","last_clean":"2023-10-06T05:21:14.186146+0000","last_became_active":"2023-10-06T05:21:14.183847+0000","last_became_peered":"2023-10-06T05:21:14.183847+0000","last_unstale":"2023-10-06T05:21:14.186146+0000","last_undegraded":"2023-10-06T05:21:14.186146+0000","last_fullsized":"2023-10-06T05:21:14.186146+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:52.043325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.160235+0000","last_change":"2023-10-06T05:21:13.168940+0000","last_active":"2023-10-06T05:21:14.160235+0000","last_peered":"2023-10-06T05:21:14.160235+0000","last_clean":"2023-10-06T05:21:14.160235+0000","last_became_active":"2023-10-06T05:21:13.168641+0000","last_became_peered":"2023-10-06T05:21:13.168641+0000","last_unstale":"2023-10-06T05:21:14.160235+0000","last_undegraded":"2023-10-06T05:21:14.160235+0000","last_fullsized":"2023-10-06T05:21:14.160235+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:42.691561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177501+0000","last_change":"2023-10-06T05:21:14.169447+0000","last_active":"2023-10-06T05:21:14.177501+0000","last_peered":"2023-10-06T05:21:14.177501+0000","last_clean":"2023-10-06T05:21:14.177501+0000","last_became_active":"2023-10-06T05:21:14.169082+0000","last_became_peered":"2023-10-06T05:21:14.169082+0000","last_unstale":"2023-10-06T05:21:14.177501+0000","last_undegraded":"2023-10-06T05:21:14.177501+0000","last_fullsized":"2023-10-06T05:21:14.177501+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:04.855790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"70'3","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191663+0000","last_change":"2023-10-06T05:21:13.163900+0000","last_active":"2023-10-06T05:21:14.191663+0000","last_peered":"2023-10-06T05:21:14.191663+0000","last_clean":"2023-10-06T05:21:14.191663+0000","last_became_active":"2023-10-06T05:21:13.163604+0000","last_became_peered":"2023-10-06T05:21:13.163604+0000","last_unstale":"2023-10-06T05:21:14.191663+0000","last_undegraded":"2023-10-06T05:21:14.191663+0000","last_fullsized":"2023-10-06T05:21:14.191663+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:55.585059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178253+0000","last_change":"2023-10-06T05:21:14.175350+0000","last_active":"2023-10-06T05:21:14.178253+0000","last_peered":"2023-10-06T05:21:14.178253+0000","last_clean":"2023-10-06T05:21:14.178253+0000","last_became_active":"2023-10-06T05:21:14.174791+0000","last_became_peered":"2023-10-06T05:21:14.174791+0000","last_unstale":"2023-10-06T05:21:14.178253+0000","last_undegraded":"2023-10-06T05:21:14.178253+0000","last_fullsized":"2023-10-06T05:21:14.178253+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:50.374014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"70'6","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.194885+0000","last_change":"2023-10-06T05:21:13.170791+0000","last_active":"2023-10-06T05:21:14.194885+0000","last_peered":"2023-10-06T05:21:14.194885+0000","last_clean":"2023-10-06T05:21:14.194885+0000","last_became_active":"2023-10-06T05:21:13.170470+0000","last_became_peered":"2023-10-06T05:21:13.170470+0000","last_unstale":"2023-10-06T05:21:14.194885+0000","last_undegraded":"2023-10-06T05:21:14.194885+0000","last_fullsized":"2023-10-06T05:21:14.194885+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:26.418570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.186174+0000","last_change":"2023-10-06T05:21:14.183193+0000","last_active":"2023-10-06T05:21:14.186174+0000","last_peered":"2023-10-06T05:21:14.186174+0000","last_clean":"2023-10-06T05:21:14.186174+0000","last_became_active":"2023-10-06T05:21:14.178379+0000","last_became_peered":"2023-10-06T05:21:14.178379+0000","last_unstale":"2023-10-06T05:21:14.186174+0000","last_undegraded":"2023-10-06T05:21:14.186174+0000","last_fullsized":"2023-10-06T05:21:14.186174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:45.462119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.193439+0000","last_change":"2023-10-06T05:21:13.171211+0000","last_active":"2023-10-06T05:21:14.193439+0000","last_peered":"2023-10-06T05:21:14.193439+0000","last_clean":"2023-10-06T05:21:14.193439+0000","last_became_active":"2023-10-06T05:21:13.170917+0000","last_became_peered":"2023-10-06T05:21:13.170917+0000","last_unstale":"2023-10-06T05:21:14.193439+0000","last_undegraded":"2023-10-06T05:21:14.193439+0000","last_fullsized":"2023-10-06T05:21:14.193439+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:08.167762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177573+0000","last_change":"2023-10-06T05:21:14.174706+0000","last_active":"2023-10-06T05:21:14.177573+0000","last_peered":"2023-10-06T05:21:14.177573+0000","last_clean":"2023-10-06T05:21:14.177573+0000","last_became_active":"2023-10-06T05:21:14.174424+0000","last_became_peered":"2023-10-06T05:21:14.174424+0000","last_unstale":"2023-10-06T05:21:14.177573+0000","last_undegraded":"2023-10-06T05:21:14.177573+0000","last_fullsized":"2023-10-06T05:21:14.177573+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:14.512710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.163365+0000","last_change":"2023-10-06T05:21:13.171789+0000","last_active":"2023-10-06T05:21:14.163365+0000","last_peered":"2023-10-06T05:21:14.163365+0000","last_clean":"2023-10-06T05:21:14.163365+0000","last_became_active":"2023-10-06T05:21:13.171451+0000","last_became_peered":"2023-10-06T05:21:13.171451+0000","last_unstale":"2023-10-06T05:21:14.163365+0000","last_undegraded":"2023-10-06T05:21:14.163365+0000","last_fullsized":"2023-10-06T05:21:14.163365+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:11.973352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.182329+0000","last_change":"2023-10-06T05:21:14.177560+0000","last_active":"2023-10-06T05:21:14.182329+0000","last_peered":"2023-10-06T05:21:14.182329+0000","last_clean":"2023-10-06T05:21:14.182329+0000","last_became_active":"2023-10-06T05:21:14.177108+0000","last_became_peered":"2023-10-06T05:21:14.177108+0000","last_unstale":"2023-10-06T05:21:14.182329+0000","last_undegraded":"2023-10-06T05:21:14.182329+0000","last_fullsized":"2023-10-06T05:21:14.182329+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:25.096218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.160903+0000","last_change":"2023-10-06T05:21:13.171171+0000","last_active":"2023-10-06T05:21:14.160903+0000","last_peered":"2023-10-06T05:21:14.160903+0000","last_clean":"2023-10-06T05:21:14.160903+0000","last_became_active":"2023-10-06T05:21:13.170856+0000","last_became_peered":"2023-10-06T05:21:13.170856+0000","last_unstale":"2023-10-06T05:21:14.160903+0000","last_undegraded":"2023-10-06T05:21:14.160903+0000","last_fullsized":"2023-10-06T05:21:14.160903+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:45.319480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.183798+0000","last_change":"2023-10-06T05:21:14.178892+0000","last_active":"2023-10-06T05:21:14.183798+0000","last_peered":"2023-10-06T05:21:14.183798+0000","last_clean":"2023-10-06T05:21:14.183798+0000","last_became_active":"2023-10-06T05:21:14.178604+0000","last_became_peered":"2023-10-06T05:21:14.178604+0000","last_unstale":"2023-10-06T05:21:14.183798+0000","last_undegraded":"2023-10-06T05:21:14.183798+0000","last_fullsized":"2023-10-06T05:21:14.183798+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:44.494854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"70'8","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.192861+0000","last_change":"2023-10-06T05:21:13.176681+0000","last_active":"2023-10-06T05:21:14.192861+0000","last_peered":"2023-10-06T05:21:14.192861+0000","last_clean":"2023-10-06T05:21:14.192861+0000","last_became_active":"2023-10-06T05:21:13.176432+0000","last_became_peered":"2023-10-06T05:21:13.176432+0000","last_unstale":"2023-10-06T05:21:14.192861+0000","last_undegraded":"2023-10-06T05:21:14.192861+0000","last_fullsized":"2023-10-06T05:21:14.192861+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:15.808879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.184727+0000","last_change":"2023-10-06T05:21:14.179416+0000","last_active":"2023-10-06T05:21:14.184727+0000","last_peered":"2023-10-06T05:21:14.184727+0000","last_clean":"2023-10-06T05:21:14.184727+0000","last_became_active":"2023-10-06T05:21:14.179112+0000","last_became_peered":"2023-10-06T05:21:14.179112+0000","last_unstale":"2023-10-06T05:21:14.184727+0000","last_undegraded":"2023-10-06T05:21:14.184727+0000","last_fullsized":"2023-10-06T05:21:14.184727+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:06.020289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191736+0000","last_change":"2023-10-06T05:21:13.165877+0000","last_active":"2023-10-06T05:21:14.191736+0000","last_peered":"2023-10-06T05:21:14.191736+0000","last_clean":"2023-10-06T05:21:14.191736+0000","last_became_active":"2023-10-06T05:21:13.165605+0000","last_became_peered":"2023-10-06T05:21:13.165605+0000","last_unstale":"2023-10-06T05:21:14.191736+0000","last_undegraded":"2023-10-06T05:21:14.191736+0000","last_fullsized":"2023-10-06T05:21:14.191736+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.182367+0000","last_change":"2023-10-06T05:21:14.177458+0000","last_active":"2023-10-06T05:21:14.182367+0000","last_peered":"2023-10-06T05:21:14.182367+0000","last_clean":"2023-10-06T05:21:14.182367+0000","last_became_active":"2023-10-06T05:21:14.177186+0000","last_became_peered":"2023-10-06T05:21:14.177186+0000","last_unstale":"2023-10-06T05:21:14.182367+0000","last_undegraded":"2023-10-06T05:21:14.182367+0000","last_fullsized":"2023-10-06T05:21:14.182367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:22.094091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T05:21:16.417743+0000","last_change":"2023-10-06T05:21:13.171761+0000","last_active":"2023-10-06T05:21:16.417743+0000","last_peered":"2023-10-06T05:21:16.417743+0000","last_clean":"2023-10-06T05:21:16.417743+0000","last_became_active":"2023-10-06T05:21:13.171491+0000","last_became_peered":"2023-10-06T05:21:13.171491+0000","last_unstale":"2023-10-06T05:21:16.417743+0000","last_undegraded":"2023-10-06T05:21:16.417743+0000","last_fullsized":"2023-10-06T05:21:16.417743+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:56.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.172586+0000","last_change":"2023-10-06T05:21:14.165763+0000","last_active":"2023-10-06T05:21:14.172586+0000","last_peered":"2023-10-06T05:21:14.172586+0000","last_clean":"2023-10-06T05:21:14.172586+0000","last_became_active":"2023-10-06T05:21:14.165281+0000","last_became_peered":"2023-10-06T05:21:14.165281+0000","last_unstale":"2023-10-06T05:21:14.172586+0000","last_undegraded":"2023-10-06T05:21:14.172586+0000","last_fullsized":"2023-10-06T05:21:14.172586+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:04.797474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.165699+0000","last_change":"2023-10-06T05:21:13.167952+0000","last_active":"2023-10-06T05:21:14.165699+0000","last_peered":"2023-10-06T05:21:14.165699+0000","last_clean":"2023-10-06T05:21:14.165699+0000","last_became_active":"2023-10-06T05:21:13.166565+0000","last_became_peered":"2023-10-06T05:21:13.166565+0000","last_unstale":"2023-10-06T05:21:14.165699+0000","last_undegraded":"2023-10-06T05:21:14.165699+0000","last_fullsized":"2023-10-06T05:21:14.165699+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:38.525356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.179036+0000","last_change":"2023-10-06T05:21:14.173602+0000","last_active":"2023-10-06T05:21:14.179036+0000","last_peered":"2023-10-06T05:21:14.179036+0000","last_clean":"2023-10-06T05:21:14.179036+0000","last_became_active":"2023-10-06T05:21:14.173343+0000","last_became_peered":"2023-10-06T05:21:14.173343+0000","last_unstale":"2023-10-06T05:21:14.179036+0000","last_undegraded":"2023-10-06T05:21:14.179036+0000","last_fullsized":"2023-10-06T05:21:14.179036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:46.254829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.164715+0000","last_change":"2023-10-06T05:21:13.171867+0000","last_active":"2023-10-06T05:21:14.164715+0000","last_peered":"2023-10-06T05:21:14.164715+0000","last_clean":"2023-10-06T05:21:14.164715+0000","last_became_active":"2023-10-06T05:21:13.171530+0000","last_became_peered":"2023-10-06T05:21:13.171530+0000","last_unstale":"2023-10-06T05:21:14.164715+0000","last_undegraded":"2023-10-06T05:21:14.164715+0000","last_fullsized":"2023-10-06T05:21:14.164715+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:28.778795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.186269+0000","last_change":"2023-10-06T05:21:14.180578+0000","last_active":"2023-10-06T05:21:14.186269+0000","last_peered":"2023-10-06T05:21:14.186269+0000","last_clean":"2023-10-06T05:21:14.186269+0000","last_became_active":"2023-10-06T05:21:14.180270+0000","last_became_peered":"2023-10-06T05:21:14.180270+0000","last_unstale":"2023-10-06T05:21:14.186269+0000","last_undegraded":"2023-10-06T05:21:14.186269+0000","last_fullsized":"2023-10-06T05:21:14.186269+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:22.127378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.165873+0000","last_change":"2023-10-06T05:21:13.172743+0000","last_active":"2023-10-06T05:21:14.165873+0000","last_peered":"2023-10-06T05:21:14.165873+0000","last_clean":"2023-10-06T05:21:14.165873+0000","last_became_active":"2023-10-06T05:21:13.172453+0000","last_became_peered":"2023-10-06T05:21:13.172453+0000","last_unstale":"2023-10-06T05:21:14.165873+0000","last_undegraded":"2023-10-06T05:21:14.165873+0000","last_fullsized":"2023-10-06T05:21:14.165873+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:01.292501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.182295+0000","last_change":"2023-10-06T05:21:14.176866+0000","last_active":"2023-10-06T05:21:14.182295+0000","last_peered":"2023-10-06T05:21:14.182295+0000","last_clean":"2023-10-06T05:21:14.182295+0000","last_became_active":"2023-10-06T05:21:14.176561+0000","last_became_peered":"2023-10-06T05:21:14.176561+0000","last_unstale":"2023-10-06T05:21:14.182295+0000","last_undegraded":"2023-10-06T05:21:14.182295+0000","last_fullsized":"2023-10-06T05:21:14.182295+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:20.672647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.161010+0000","last_change":"2023-10-06T05:21:13.171851+0000","last_active":"2023-10-06T05:21:14.161010+0000","last_peered":"2023-10-06T05:21:14.161010+0000","last_clean":"2023-10-06T05:21:14.161010+0000","last_became_active":"2023-10-06T05:21:13.171552+0000","last_became_peered":"2023-10-06T05:21:13.171552+0000","last_unstale":"2023-10-06T05:21:14.161010+0000","last_undegraded":"2023-10-06T05:21:14.161010+0000","last_fullsized":"2023-10-06T05:21:14.161010+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:14:14.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177138+0000","last_change":"2023-10-06T05:21:14.172782+0000","last_active":"2023-10-06T05:21:14.177138+0000","last_peered":"2023-10-06T05:21:14.177138+0000","last_clean":"2023-10-06T05:21:14.177138+0000","last_became_active":"2023-10-06T05:21:14.172172+0000","last_became_peered":"2023-10-06T05:21:14.172172+0000","last_unstale":"2023-10-06T05:21:14.177138+0000","last_undegraded":"2023-10-06T05:21:14.177138+0000","last_fullsized":"2023-10-06T05:21:14.177138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:34.203137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.167547+0000","last_change":"2023-10-06T05:21:13.172335+0000","last_active":"2023-10-06T05:21:14.167547+0000","last_peered":"2023-10-06T05:21:14.167547+0000","last_clean":"2023-10-06T05:21:14.167547+0000","last_became_active":"2023-10-06T05:21:13.172046+0000","last_became_peered":"2023-10-06T05:21:13.172046+0000","last_unstale":"2023-10-06T05:21:14.167547+0000","last_undegraded":"2023-10-06T05:21:14.167547+0000","last_fullsized":"2023-10-06T05:21:14.167547+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:49.234603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.168079+0000","last_change":"2023-10-06T05:21:14.164995+0000","last_active":"2023-10-06T05:21:14.168079+0000","last_peered":"2023-10-06T05:21:14.168079+0000","last_clean":"2023-10-06T05:21:14.168079+0000","last_became_active":"2023-10-06T05:21:14.164682+0000","last_became_peered":"2023-10-06T05:21:14.164682+0000","last_unstale":"2023-10-06T05:21:14.168079+0000","last_undegraded":"2023-10-06T05:21:14.168079+0000","last_fullsized":"2023-10-06T05:21:14.168079+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:23.212689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.153382+0000","last_change":"2023-10-06T05:21:13.174008+0000","last_active":"2023-10-06T05:21:14.153382+0000","last_peered":"2023-10-06T05:21:14.153382+0000","last_clean":"2023-10-06T05:21:14.153382+0000","last_became_active":"2023-10-06T05:21:13.173503+0000","last_became_peered":"2023-10-06T05:21:13.173503+0000","last_unstale":"2023-10-06T05:21:14.153382+0000","last_undegraded":"2023-10-06T05:21:14.153382+0000","last_fullsized":"2023-10-06T05:21:14.153382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:14.164789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178686+0000","last_change":"2023-10-06T05:21:14.174626+0000","last_active":"2023-10-06T05:21:14.178686+0000","last_peered":"2023-10-06T05:21:14.178686+0000","last_clean":"2023-10-06T05:21:14.178686+0000","last_became_active":"2023-10-06T05:21:14.174013+0000","last_became_peered":"2023-10-06T05:21:14.174013+0000","last_unstale":"2023-10-06T05:21:14.178686+0000","last_undegraded":"2023-10-06T05:21:14.178686+0000","last_fullsized":"2023-10-06T05:21:14.178686+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:39.527935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.183455+0000","last_change":"2023-10-06T05:21:13.165946+0000","last_active":"2023-10-06T05:21:14.183455+0000","last_peered":"2023-10-06T05:21:14.183455+0000","last_clean":"2023-10-06T05:21:14.183455+0000","last_became_active":"2023-10-06T05:21:13.165677+0000","last_became_peered":"2023-10-06T05:21:13.165677+0000","last_unstale":"2023-10-06T05:21:14.183455+0000","last_undegraded":"2023-10-06T05:21:14.183455+0000","last_fullsized":"2023-10-06T05:21:14.183455+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:32.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":59,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.155638+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:21:14.155638+0000","last_peered":"2023-10-06T05:21:14.155638+0000","last_clean":"2023-10-06T05:21:14.155638+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:21:14.155638+0000","last_undegraded":"2023-10-06T05:21:14.155638+0000","last_fullsized":"2023-10-06T05:21:14.155638+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.181376+0000","last_change":"2023-10-06T05:21:14.178020+0000","last_active":"2023-10-06T05:21:14.181376+0000","last_peered":"2023-10-06T05:21:14.181376+0000","last_clean":"2023-10-06T05:21:14.181376+0000","last_became_active":"2023-10-06T05:21:14.177661+0000","last_became_peered":"2023-10-06T05:21:14.177661+0000","last_unstale":"2023-10-06T05:21:14.181376+0000","last_undegraded":"2023-10-06T05:21:14.181376+0000","last_fullsized":"2023-10-06T05:21:14.181376+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:10.252525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188058+0000","last_change":"2023-10-06T05:21:13.165590+0000","last_active":"2023-10-06T05:21:14.188058+0000","last_peered":"2023-10-06T05:21:14.188058+0000","last_clean":"2023-10-06T05:21:14.188058+0000","last_became_active":"2023-10-06T05:21:13.165314+0000","last_became_peered":"2023-10-06T05:21:13.165314+0000","last_unstale":"2023-10-06T05:21:14.188058+0000","last_undegraded":"2023-10-06T05:21:14.188058+0000","last_fullsized":"2023-10-06T05:21:14.188058+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:24.093639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.156112+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:21:14.156112+0000","last_peered":"2023-10-06T05:21:14.156112+0000","last_clean":"2023-10-06T05:21:14.156112+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:21:14.156112+0000","last_undegraded":"2023-10-06T05:21:14.156112+0000","last_fullsized":"2023-10-06T05:21:14.156112+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.187934+0000","last_change":"2023-10-06T05:21:14.180321+0000","last_active":"2023-10-06T05:21:14.187934+0000","last_peered":"2023-10-06T05:21:14.187934+0000","last_clean":"2023-10-06T05:21:14.187934+0000","last_became_active":"2023-10-06T05:21:14.179982+0000","last_became_peered":"2023-10-06T05:21:14.179982+0000","last_unstale":"2023-10-06T05:21:14.187934+0000","last_undegraded":"2023-10-06T05:21:14.187934+0000","last_fullsized":"2023-10-06T05:21:14.187934+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:36.647693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.156177+0000","last_change":"2023-10-06T05:21:13.166267+0000","last_active":"2023-10-06T05:21:14.156177+0000","last_peered":"2023-10-06T05:21:14.156177+0000","last_clean":"2023-10-06T05:21:14.156177+0000","last_became_active":"2023-10-06T05:21:13.165887+0000","last_became_peered":"2023-10-06T05:21:13.165887+0000","last_unstale":"2023-10-06T05:21:14.156177+0000","last_undegraded":"2023-10-06T05:21:14.156177+0000","last_fullsized":"2023-10-06T05:21:14.156177+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:56.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.154477+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:21:14.154477+0000","last_peered":"2023-10-06T05:21:14.154477+0000","last_clean":"2023-10-06T05:21:14.154477+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:21:14.154477+0000","last_undegraded":"2023-10-06T05:21:14.154477+0000","last_fullsized":"2023-10-06T05:21:14.154477+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":204,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.164043+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:21:14.164043+0000","last_peered":"2023-10-06T05:21:14.164043+0000","last_clean":"2023-10-06T05:21:14.164043+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:21:14.164043+0000","last_undegraded":"2023-10-06T05:21:14.164043+0000","last_fullsized":"2023-10-06T05:21:14.164043+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.189282+0000","last_change":"2023-10-06T05:21:14.187192+0000","last_active":"2023-10-06T05:21:14.189282+0000","last_peered":"2023-10-06T05:21:14.189282+0000","last_clean":"2023-10-06T05:21:14.189282+0000","last_became_active":"2023-10-06T05:21:14.186922+0000","last_became_peered":"2023-10-06T05:21:14.186922+0000","last_unstale":"2023-10-06T05:21:14.189282+0000","last_undegraded":"2023-10-06T05:21:14.189282+0000","last_fullsized":"2023-10-06T05:21:14.189282+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:45.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.156653+0000","last_change":"2023-10-06T05:21:13.168485+0000","last_active":"2023-10-06T05:21:14.156653+0000","last_peered":"2023-10-06T05:21:14.156653+0000","last_clean":"2023-10-06T05:21:14.156653+0000","last_became_active":"2023-10-06T05:21:13.168162+0000","last_became_peered":"2023-10-06T05:21:13.168162+0000","last_unstale":"2023-10-06T05:21:14.156653+0000","last_undegraded":"2023-10-06T05:21:14.156653+0000","last_fullsized":"2023-10-06T05:21:14.156653+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.040745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.163449+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:21:14.163449+0000","last_peered":"2023-10-06T05:21:14.163449+0000","last_clean":"2023-10-06T05:21:14.163449+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:21:14.163449+0000","last_undegraded":"2023-10-06T05:21:14.163449+0000","last_fullsized":"2023-10-06T05:21:14.163449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178513+0000","last_change":"2023-10-06T05:21:14.171617+0000","last_active":"2023-10-06T05:21:14.178513+0000","last_peered":"2023-10-06T05:21:14.178513+0000","last_clean":"2023-10-06T05:21:14.178513+0000","last_became_active":"2023-10-06T05:21:14.171336+0000","last_became_peered":"2023-10-06T05:21:14.171336+0000","last_unstale":"2023-10-06T05:21:14.178513+0000","last_undegraded":"2023-10-06T05:21:14.178513+0000","last_fullsized":"2023-10-06T05:21:14.178513+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:16.803474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.156598+0000","last_change":"2023-10-06T05:21:13.171949+0000","last_active":"2023-10-06T05:21:14.156598+0000","last_peered":"2023-10-06T05:21:14.156598+0000","last_clean":"2023-10-06T05:21:14.156598+0000","last_became_active":"2023-10-06T05:21:13.171646+0000","last_became_peered":"2023-10-06T05:21:13.171646+0000","last_unstale":"2023-10-06T05:21:14.156598+0000","last_undegraded":"2023-10-06T05:21:14.156598+0000","last_fullsized":"2023-10-06T05:21:14.156598+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:00.022649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.164135+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:21:14.164135+0000","last_peered":"2023-10-06T05:21:14.164135+0000","last_clean":"2023-10-06T05:21:14.164135+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:21:14.164135+0000","last_undegraded":"2023-10-06T05:21:14.164135+0000","last_fullsized":"2023-10-06T05:21:14.164135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191317+0000","last_change":"2023-10-06T05:21:14.187053+0000","last_active":"2023-10-06T05:21:14.191317+0000","last_peered":"2023-10-06T05:21:14.191317+0000","last_clean":"2023-10-06T05:21:14.191317+0000","last_became_active":"2023-10-06T05:21:14.186478+0000","last_became_peered":"2023-10-06T05:21:14.186478+0000","last_unstale":"2023-10-06T05:21:14.191317+0000","last_undegraded":"2023-10-06T05:21:14.191317+0000","last_fullsized":"2023-10-06T05:21:14.191317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:16.803474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191563+0000","last_change":"2023-10-06T05:21:13.168910+0000","last_active":"2023-10-06T05:21:14.191563+0000","last_peered":"2023-10-06T05:21:14.191563+0000","last_clean":"2023-10-06T05:21:14.191563+0000","last_became_active":"2023-10-06T05:21:13.168616+0000","last_became_peered":"2023-10-06T05:21:13.168616+0000","last_unstale":"2023-10-06T05:21:14.191563+0000","last_undegraded":"2023-10-06T05:21:14.191563+0000","last_fullsized":"2023-10-06T05:21:14.191563+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:15.808879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.162156+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:21:14.162156+0000","last_peered":"2023-10-06T05:21:14.162156+0000","last_clean":"2023-10-06T05:21:14.162156+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:21:14.162156+0000","last_undegraded":"2023-10-06T05:21:14.162156+0000","last_fullsized":"2023-10-06T05:21:14.162156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178767+0000","last_change":"2023-10-06T05:21:14.173537+0000","last_active":"2023-10-06T05:21:14.178767+0000","last_peered":"2023-10-06T05:21:14.178767+0000","last_clean":"2023-10-06T05:21:14.178767+0000","last_became_active":"2023-10-06T05:21:14.172896+0000","last_became_peered":"2023-10-06T05:21:14.172896+0000","last_unstale":"2023-10-06T05:21:14.178767+0000","last_undegraded":"2023-10-06T05:21:14.178767+0000","last_fullsized":"2023-10-06T05:21:14.178767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:21.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.153679+0000","last_change":"2023-10-06T05:21:13.166892+0000","last_active":"2023-10-06T05:21:14.153679+0000","last_peered":"2023-10-06T05:21:14.153679+0000","last_clean":"2023-10-06T05:21:14.153679+0000","last_became_active":"2023-10-06T05:21:13.166496+0000","last_became_peered":"2023-10-06T05:21:13.166496+0000","last_unstale":"2023-10-06T05:21:14.153679+0000","last_undegraded":"2023-10-06T05:21:14.153679+0000","last_fullsized":"2023-10-06T05:21:14.153679+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:13.510131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.167144+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:21:14.167144+0000","last_peered":"2023-10-06T05:21:14.167144+0000","last_clean":"2023-10-06T05:21:14.167144+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:21:14.167144+0000","last_undegraded":"2023-10-06T05:21:14.167144+0000","last_fullsized":"2023-10-06T05:21:14.167144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188569+0000","last_change":"2023-10-06T05:21:14.180138+0000","last_active":"2023-10-06T05:21:14.188569+0000","last_peered":"2023-10-06T05:21:14.188569+0000","last_clean":"2023-10-06T05:21:14.188569+0000","last_became_active":"2023-10-06T05:21:14.179803+0000","last_became_peered":"2023-10-06T05:21:14.179803+0000","last_unstale":"2023-10-06T05:21:14.188569+0000","last_undegraded":"2023-10-06T05:21:14.188569+0000","last_fullsized":"2023-10-06T05:21:14.188569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:15.241769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.192315+0000","last_change":"2023-10-06T05:21:13.159342+0000","last_active":"2023-10-06T05:21:14.192315+0000","last_peered":"2023-10-06T05:21:14.192315+0000","last_clean":"2023-10-06T05:21:14.192315+0000","last_became_active":"2023-10-06T05:21:13.158741+0000","last_became_peered":"2023-10-06T05:21:13.158741+0000","last_unstale":"2023-10-06T05:21:14.192315+0000","last_undegraded":"2023-10-06T05:21:14.192315+0000","last_fullsized":"2023-10-06T05:21:14.192315+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:22.191827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.166711+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:21:14.166711+0000","last_peered":"2023-10-06T05:21:14.166711+0000","last_clean":"2023-10-06T05:21:14.166711+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:21:14.166711+0000","last_undegraded":"2023-10-06T05:21:14.166711+0000","last_fullsized":"2023-10-06T05:21:14.166711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298716,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073124,"statfs":{"total":96636764160,"available":96330878976,"internally_reserved":0,"allocated":1462272,"data_stored":976395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.68200000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93600000000000005,"5min":1.018,"15min":1.018},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95999999999999996,"5min":1.056,"15min":1.056},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73399999999999999,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.83899999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67700000000000005,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.748},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93300000000000005,"5min":0.999,"15min":0.999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.83799999999999997},"min":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0389999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.0569999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.626,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.85399999999999998},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0029999999999999,"5min":1.103,"15min":1.103},"last":1.012}]}]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":387541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.47099999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89800000000000002,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.77100000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90800000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78500000000000003,"5min":1.369,"15min":1.369},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.891,"15min":1.891},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89800000000000002,"5min":1.347,"15min":1.347},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":1,"15min":1},"last":0.79800000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73499999999999999,"15min":0.78500000000000003},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.95499999999999996,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78200000000000003,"15min":0.82399999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85899999999999999,"5min":1.415,"15min":1.415},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77500000000000002,"15min":0.83099999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98199999999999998,"5min":2,"15min":2},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.78000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95999999999999996,"5min":1.46,"15min":1.46},"last":0.85599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298720,"kb_used_data":1432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073120,"statfs":{"total":96636764160,"available":96330874880,"internally_reserved":0,"allocated":1466368,"data_stored":976485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.443}]},{"osd":3,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.59699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82499999999999996},"min":{"1min":0.63600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.753},"min":{"1min":0.621,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.052,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.82999999999999996,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.83499999999999996,"15min":0.82499999999999996},"min":{"1min":0.67100000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.69299999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.087,"5min":1.103,"15min":1.103},"last":0.80900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":386005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68300000000000005,"5min":1.042,"15min":1.042},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.441},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1519999999999999,"5min":1.242,"15min":1.242},"last":0.627},{"interface":"front","average":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":1.198,"15min":1.198},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.77300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1659999999999999,"5min":1.26,"15min":1.26},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.073,"5min":1.323,"15min":1.323},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.123,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.89900000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.264,"15min":1.264},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.04,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.65400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":386719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46500000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.624,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.503},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68799999999999994,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93500000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90600000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96299999999999997,"5min":1.099,"15min":1.099},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78500000000000003,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.90000000000000002,"5min":1.163,"15min":1.163},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98799999999999999,"5min":1.21,"15min":1.21},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.022,"5min":1.042,"15min":1.042},"last":0.91100000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84399999999999997,"15min":0.872},"min":{"1min":0.64600000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0469999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.82299999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298136,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073704,"statfs":{"total":96636764160,"available":96331472896,"internally_reserved":0,"allocated":868352,"data_stored":386559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.44600000000000001,"15min":0.46200000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57199999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.624,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72299999999999998,"5min":1.014,"15min":1.014},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.145,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1830000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.78700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.03,"5min":1.204,"15min":1.204},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.329},"last":0.54800000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.82099999999999995},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.048,"5min":1.484,"15min":1.484},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83499999999999996,"15min":0.85399999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1930000000000001,"5min":1.629,"15min":1.629},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.83999999999999997,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1950000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.085,"5min":1.325,"15min":1.325},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298152,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073688,"statfs":{"total":96636764160,"available":96331456512,"internally_reserved":0,"allocated":884736,"data_stored":387645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74399999999999999,"15min":0.79800000000000004},"min":{"1min":0.52100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":2.77,"15min":2.77},"last":0.73499999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.76600000000000001,"15min":0.82999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89600000000000002,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.58599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.90500000000000003,"15min":1.091},"min":{"1min":0.50800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":7.742,"15min":7.742},"last":0.88600000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.48999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96299999999999997,"5min":1.079,"15min":1.079},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90900000000000003,"5min":1.181,"15min":1.181},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.745},"min":{"1min":0.56499999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.751},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.60599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.92200000000000004,"5min":1.171,"15min":1.171},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.624,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96499999999999997,"5min":1.228,"15min":1.228},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81499999999999995,"15min":0.86899999999999999},"min":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97399999999999998,"5min":1.571,"15min":1.571},"last":0.82499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298136,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073704,"statfs":{"total":96636764160,"available":96331472896,"internally_reserved":0,"allocated":868352,"data_stored":386039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.83099999999999996,"15min":1},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.63200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.85499999999999998,"15min":1.0209999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.079,"5min":9.25,"15min":9.25},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.875,"15min":1.0620000000000001},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":10.384,"15min":10.384},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":1.0760000000000001},"min":{"1min":0.49299999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0349999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.621}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":1.1240000000000001},"min":{"1min":0.54800000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.95199999999999996,"5min":10.205,"15min":10.205},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96399999999999997,"5min":1.292,"15min":1.292},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.048,"5min":1.171,"15min":1.171},"last":0.49299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78400000000000003,"15min":0.81699999999999995},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.298,"15min":1.298},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80300000000000005,"15min":0.82899999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98999999999999999,"5min":1.204,"15min":1.204},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90200000000000002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:23.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:23.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.851+0000 7fc21d5ff700 1 Processor -- start 2023-10-06T05:21:23.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.851+0000 7fc21d5ff700 1 -- start start 2023-10-06T05:21:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.852+0000 7fc21d5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2180c7b60 0x7fc218069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.852+0000 7fc21d5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc2180c7f30 0x7fc2180aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.852+0000 7fc21d5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180ca5b0 con 0x7fc21806ef50 2023-10-06T05:21:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.852+0000 7fc21d5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21806f740 con 0x7fc2180c7b60 2023-10-06T05:21:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.852+0000 7fc21d5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180c9c80 con 0x7fc2180c7f30 2023-10-06T05:21:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.852+0000 7fc216ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2180c7b60 0x7fc218069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.853+0000 7fc216ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2180c7b60 0x7fc218069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60878/0 (socket says 172.21.15.31:60878) 2023-10-06T05:21:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.853+0000 7fc216ffd700 1 -- 172.21.15.31:0/4107646045 learned_addr learned my addr 172.21.15.31:0/4107646045 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.853+0000 7fc2177fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc21806ef50 0x7fc21813f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35620/0 (socket says 172.21.15.31:35620) 2023-10-06T05:21:23.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.853+0000 7fc2167fc700 1 --2- 172.21.15.31:0/4107646045 >> v2:172.21.15.62:3300/0 conn(0x7fc2180c7f30 0x7fc2180aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.853+0000 7fc216ffd700 1 -- 172.21.15.31:0/4107646045 >> v2:172.21.15.62:3300/0 conn(0x7fc2180c7f30 msgr2=0x7fc2180aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:23.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.853+0000 7fc216ffd700 1 --2- 172.21.15.31:0/4107646045 >> v2:172.21.15.62:3300/0 conn(0x7fc2180c7f30 0x7fc2180aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.853+0000 7fc216ffd700 1 -- 172.21.15.31:0/4107646045 >> v1:172.21.15.31:6789/0 conn(0x7fc21806ef50 legacy=0x7fc21813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.853+0000 7fc216ffd700 1 -- 172.21.15.31:0/4107646045 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180c9f60 con 0x7fc2180c7b60 2023-10-06T05:21:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.853+0000 7fc216ffd700 1 --2- 172.21.15.31:0/4107646045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 0x7fc218069880 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc20800bc30 tx=0x7fc20800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73221f0ebdc365ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.853+0000 7fc215ffb700 1 -- 172.21.15.31:0/4107646045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208005c10 con 0x7fc2180c7b60 2023-10-06T05:21:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.854+0000 7fc215ffb700 1 -- 172.21.15.31:0/4107646045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208005d70 con 0x7fc2180c7b60 2023-10-06T05:21:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.854+0000 7fc215ffb700 1 -- 172.21.15.31:0/4107646045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800dc30 con 0x7fc2180c7b60 2023-10-06T05:21:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.854+0000 7fc21d5ff700 1 -- 172.21.15.31:0/4107646045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 msgr2=0x7fc218069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.854+0000 7fc21d5ff700 1 --2- 172.21.15.31:0/4107646045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 0x7fc218069880 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc20800bc30 tx=0x7fc20800b4c0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.854+0000 7fc21d5ff700 1 -- 172.21.15.31:0/4107646045 shutdown_connections 2023-10-06T05:21:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.854+0000 7fc21d5ff700 1 --2- 172.21.15.31:0/4107646045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 0x7fc218069880 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.854+0000 7fc21d5ff700 1 --2- 172.21.15.31:0/4107646045 >> v2:172.21.15.62:3300/0 conn(0x7fc2180c7f30 0x7fc2180aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.854+0000 7fc21d5ff700 1 -- 172.21.15.31:0/4107646045 >> 172.21.15.31:0/4107646045 conn(0x7fc2180cc0b0 msgr2=0x7fc2180cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.854+0000 7fc21d5ff700 1 -- 172.21.15.31:0/4107646045 shutdown_connections 2023-10-06T05:21:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.854+0000 7fc21d5ff700 1 -- 172.21.15.31:0/4107646045 wait complete. 2023-10-06T05:21:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.856+0000 7fc21d5ff700 1 Processor -- start 2023-10-06T05:21:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.856+0000 7fc21d5ff700 1 -- start start 2023-10-06T05:21:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc21d5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc21806ef50 0x7fc218156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc21d5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 0x7fc21815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc21d5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2180c7f30 0x7fc2180ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc21d5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc218157df0 con 0x7fc2180c7b60 2023-10-06T05:21:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc21d5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc218156780 con 0x7fc2180c7f30 2023-10-06T05:21:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc21d5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21815d8a0 con 0x7fc21806ef50 2023-10-06T05:21:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc2167fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 0x7fc21815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc2167fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 0x7fc21815c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60882/0 (socket says 172.21.15.31:60882) 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc2167fc700 1 -- 172.21.15.31:0/2733655776 learned_addr learned my addr 172.21.15.31:0/2733655776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc216ffd700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc21806ef50 0x7fc218156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc2177fe700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2180c7f30 0x7fc2180ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc2167fc700 1 -- 172.21.15.31:0/2733655776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc21806ef50 msgr2=0x7fc218156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc2167fc700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc21806ef50 0x7fc218156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc2167fc700 1 -- 172.21.15.31:0/2733655776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2180c7f30 msgr2=0x7fc2180ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc2167fc700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2180c7f30 0x7fc2180ad3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc2167fc700 1 -- 172.21.15.31:0/2733655776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20800b720 con 0x7fc2180c7b60 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.857+0000 7fc216ffd700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc21806ef50 0x7fc218156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.858+0000 7fc2167fc700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 0x7fc21815c630 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00d890 tx=0x7fc20c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.858+0000 7fc1feffd700 1 -- 172.21.15.31:0/2733655776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00b600 con 0x7fc2180c7b60 2023-10-06T05:21:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.858+0000 7fc1feffd700 1 -- 172.21.15.31:0/2733655776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00b760 con 0x7fc2180c7b60 2023-10-06T05:21:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.858+0000 7fc1feffd700 1 -- 172.21.15.31:0/2733655776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00bae0 con 0x7fc2180c7b60 2023-10-06T05:21:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.858+0000 7fc21d5ff700 1 -- 172.21.15.31:0/2733655776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21815db80 con 0x7fc2180c7b60 2023-10-06T05:21:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.858+0000 7fc21d5ff700 1 -- 172.21.15.31:0/2733655776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21815e140 con 0x7fc2180c7b60 2023-10-06T05:21:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.859+0000 7fc21d5ff700 1 -- 172.21.15.31:0/2733655776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21804c380 con 0x7fc2180c7b60 2023-10-06T05:21:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.860+0000 7fc1feffd700 1 -- 172.21.15.31:0/2733655776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fc20c01e460 con 0x7fc2180c7b60 2023-10-06T05:21:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.861+0000 7fc1feffd700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc204067290 0x7fc204069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.861+0000 7fc1feffd700 1 -- 172.21.15.31:0/2733655776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc20c0852f0 con 0x7fc2180c7b60 2023-10-06T05:21:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.861+0000 7fc216ffd700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc204067290 0x7fc204069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.862+0000 7fc216ffd700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc204067290 0x7fc204069740 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc20800b500 tx=0x7fc20801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:23.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:23.865+0000 7fc1feffd700 1 -- 172.21.15.31:0/2733655776 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc20c051fb0 con 0x7fc2180c7b60 2023-10-06T05:21:24.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.062+0000 7fc21d5ff700 1 -- 172.21.15.31:0/2733655776 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2180b1a50 con 0x7fc204067290 2023-10-06T05:21:24.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.071+0000 7fc1feffd700 1 -- 172.21.15.31:0/2733655776 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311340 (secure 0 0 0) 0x7fc2180b1a50 con 0x7fc204067290 2023-10-06T05:21:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.075+0000 7fc21d5ff700 1 -- 172.21.15.31:0/2733655776 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc204067290 msgr2=0x7fc204069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.075+0000 7fc21d5ff700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc204067290 0x7fc204069740 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc20800b500 tx=0x7fc20801a000 comp rx=0 tx=0).stop 2023-10-06T05:21:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.075+0000 7fc21d5ff700 1 -- 172.21.15.31:0/2733655776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 msgr2=0x7fc21815c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.075+0000 7fc21d5ff700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 0x7fc21815c630 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00d890 tx=0x7fc20c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.076+0000 7fc21d5ff700 1 -- 172.21.15.31:0/2733655776 shutdown_connections 2023-10-06T05:21:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.076+0000 7fc21d5ff700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2180c7f30 0x7fc2180ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.076+0000 7fc21d5ff700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc21806ef50 0x7fc218156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.076+0000 7fc21d5ff700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fc204067290 0x7fc204069740 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.076+0000 7fc21d5ff700 1 --2- 172.21.15.31:0/2733655776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2180c7b60 0x7fc21815c630 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.076+0000 7fc21d5ff700 1 -- 172.21.15.31:0/2733655776 >> 172.21.15.31:0/2733655776 conn(0x7fc2180cc0b0 msgr2=0x7fc21813e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:24.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.076+0000 7fc21d5ff700 1 -- 172.21.15.31:0/2733655776 shutdown_connections 2023-10-06T05:21:24.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.076+0000 7fc21d5ff700 1 -- 172.21.15.31:0/2733655776 wait complete. 2023-10-06T05:21:24.081 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:21:24.095 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2023-10-06T05:21:22.119449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386280,"kb_used_data":7976,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752588440,"statfs":{"total":773094113280,"available":770650562560,"internally_reserved":0,"allocated":8167424,"data_stored":4273388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001947"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.184768+0000","last_change":"2023-10-06T05:21:14.182605+0000","last_active":"2023-10-06T05:21:14.184768+0000","last_peered":"2023-10-06T05:21:14.184768+0000","last_clean":"2023-10-06T05:21:14.184768+0000","last_became_active":"2023-10-06T05:21:14.182308+0000","last_became_peered":"2023-10-06T05:21:14.182308+0000","last_unstale":"2023-10-06T05:21:14.184768+0000","last_undegraded":"2023-10-06T05:21:14.184768+0000","last_fullsized":"2023-10-06T05:21:14.184768+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:36.647693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.180147+0000","last_change":"2023-10-06T05:21:14.174918+0000","last_active":"2023-10-06T05:21:14.180147+0000","last_peered":"2023-10-06T05:21:14.180147+0000","last_clean":"2023-10-06T05:21:14.180147+0000","last_became_active":"2023-10-06T05:21:14.174566+0000","last_became_peered":"2023-10-06T05:21:14.174566+0000","last_unstale":"2023-10-06T05:21:14.180147+0000","last_undegraded":"2023-10-06T05:21:14.180147+0000","last_fullsized":"2023-10-06T05:21:14.180147+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:09.628005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178625+0000","last_change":"2023-10-06T05:21:14.174684+0000","last_active":"2023-10-06T05:21:14.178625+0000","last_peered":"2023-10-06T05:21:14.178625+0000","last_clean":"2023-10-06T05:21:14.178625+0000","last_became_active":"2023-10-06T05:21:14.174370+0000","last_became_peered":"2023-10-06T05:21:14.174370+0000","last_unstale":"2023-10-06T05:21:14.178625+0000","last_undegraded":"2023-10-06T05:21:14.178625+0000","last_fullsized":"2023-10-06T05:21:14.178625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:32.078654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.185470+0000","last_change":"2023-10-06T05:21:14.177280+0000","last_active":"2023-10-06T05:21:14.185470+0000","last_peered":"2023-10-06T05:21:14.185470+0000","last_clean":"2023-10-06T05:21:14.185470+0000","last_became_active":"2023-10-06T05:21:14.176617+0000","last_became_peered":"2023-10-06T05:21:14.176617+0000","last_unstale":"2023-10-06T05:21:14.185470+0000","last_undegraded":"2023-10-06T05:21:14.185470+0000","last_fullsized":"2023-10-06T05:21:14.185470+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:57.160327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178291+0000","last_change":"2023-10-06T05:21:14.171897+0000","last_active":"2023-10-06T05:21:14.178291+0000","last_peered":"2023-10-06T05:21:14.178291+0000","last_clean":"2023-10-06T05:21:14.178291+0000","last_became_active":"2023-10-06T05:21:14.171542+0000","last_became_peered":"2023-10-06T05:21:14.171542+0000","last_unstale":"2023-10-06T05:21:14.178291+0000","last_undegraded":"2023-10-06T05:21:14.178291+0000","last_fullsized":"2023-10-06T05:21:14.178291+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:08.721924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188396+0000","last_change":"2023-10-06T05:21:14.178877+0000","last_active":"2023-10-06T05:21:14.188396+0000","last_peered":"2023-10-06T05:21:14.188396+0000","last_clean":"2023-10-06T05:21:14.188396+0000","last_became_active":"2023-10-06T05:21:14.178036+0000","last_became_peered":"2023-10-06T05:21:14.178036+0000","last_unstale":"2023-10-06T05:21:14.188396+0000","last_undegraded":"2023-10-06T05:21:14.188396+0000","last_fullsized":"2023-10-06T05:21:14.188396+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:40.622847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.179117+0000","last_change":"2023-10-06T05:21:14.176248+0000","last_active":"2023-10-06T05:21:14.179117+0000","last_peered":"2023-10-06T05:21:14.179117+0000","last_clean":"2023-10-06T05:21:14.179117+0000","last_became_active":"2023-10-06T05:21:14.175935+0000","last_became_peered":"2023-10-06T05:21:14.175935+0000","last_unstale":"2023-10-06T05:21:14.179117+0000","last_undegraded":"2023-10-06T05:21:14.179117+0000","last_fullsized":"2023-10-06T05:21:14.179117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:21.116415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.170189+0000","last_change":"2023-10-06T05:21:14.167168+0000","last_active":"2023-10-06T05:21:14.170189+0000","last_peered":"2023-10-06T05:21:14.170189+0000","last_clean":"2023-10-06T05:21:14.170189+0000","last_became_active":"2023-10-06T05:21:14.166894+0000","last_became_peered":"2023-10-06T05:21:14.166894+0000","last_unstale":"2023-10-06T05:21:14.170189+0000","last_undegraded":"2023-10-06T05:21:14.170189+0000","last_fullsized":"2023-10-06T05:21:14.170189+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:33.508458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.172744+0000","last_change":"2023-10-06T05:21:14.165341+0000","last_active":"2023-10-06T05:21:14.172744+0000","last_peered":"2023-10-06T05:21:14.172744+0000","last_clean":"2023-10-06T05:21:14.172744+0000","last_became_active":"2023-10-06T05:21:14.165037+0000","last_became_peered":"2023-10-06T05:21:14.165037+0000","last_unstale":"2023-10-06T05:21:14.172744+0000","last_undegraded":"2023-10-06T05:21:14.172744+0000","last_fullsized":"2023-10-06T05:21:14.172744+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:04.649668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.185976+0000","last_change":"2023-10-06T05:21:14.184650+0000","last_active":"2023-10-06T05:21:14.185976+0000","last_peered":"2023-10-06T05:21:14.185976+0000","last_clean":"2023-10-06T05:21:14.185976+0000","last_became_active":"2023-10-06T05:21:14.184407+0000","last_became_peered":"2023-10-06T05:21:14.184407+0000","last_unstale":"2023-10-06T05:21:14.185976+0000","last_undegraded":"2023-10-06T05:21:14.185976+0000","last_fullsized":"2023-10-06T05:21:14.185976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:55.793132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.187819+0000","last_change":"2023-10-06T05:21:14.176765+0000","last_active":"2023-10-06T05:21:14.187819+0000","last_peered":"2023-10-06T05:21:14.187819+0000","last_clean":"2023-10-06T05:21:14.187819+0000","last_became_active":"2023-10-06T05:21:14.175854+0000","last_became_peered":"2023-10-06T05:21:14.175854+0000","last_unstale":"2023-10-06T05:21:14.187819+0000","last_undegraded":"2023-10-06T05:21:14.187819+0000","last_fullsized":"2023-10-06T05:21:14.187819+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:33.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.185450+0000","last_change":"2023-10-06T05:21:14.177074+0000","last_active":"2023-10-06T05:21:14.185450+0000","last_peered":"2023-10-06T05:21:14.185450+0000","last_clean":"2023-10-06T05:21:14.185450+0000","last_became_active":"2023-10-06T05:21:14.176305+0000","last_became_peered":"2023-10-06T05:21:14.176305+0000","last_unstale":"2023-10-06T05:21:14.185450+0000","last_undegraded":"2023-10-06T05:21:14.185450+0000","last_fullsized":"2023-10-06T05:21:14.185450+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:16.811458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.174113+0000","last_change":"2023-10-06T05:21:14.169942+0000","last_active":"2023-10-06T05:21:14.174113+0000","last_peered":"2023-10-06T05:21:14.174113+0000","last_clean":"2023-10-06T05:21:14.174113+0000","last_became_active":"2023-10-06T05:21:14.169627+0000","last_became_peered":"2023-10-06T05:21:14.169627+0000","last_unstale":"2023-10-06T05:21:14.174113+0000","last_undegraded":"2023-10-06T05:21:14.174113+0000","last_fullsized":"2023-10-06T05:21:14.174113+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:20.105525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188375+0000","last_change":"2023-10-06T05:21:14.178776+0000","last_active":"2023-10-06T05:21:14.188375+0000","last_peered":"2023-10-06T05:21:14.188375+0000","last_clean":"2023-10-06T05:21:14.188375+0000","last_became_active":"2023-10-06T05:21:14.177691+0000","last_became_peered":"2023-10-06T05:21:14.177691+0000","last_unstale":"2023-10-06T05:21:14.188375+0000","last_undegraded":"2023-10-06T05:21:14.188375+0000","last_fullsized":"2023-10-06T05:21:14.188375+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:26.471873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.176705+0000","last_change":"2023-10-06T05:21:14.172116+0000","last_active":"2023-10-06T05:21:14.176705+0000","last_peered":"2023-10-06T05:21:14.176705+0000","last_clean":"2023-10-06T05:21:14.176705+0000","last_became_active":"2023-10-06T05:21:14.171841+0000","last_became_peered":"2023-10-06T05:21:14.171841+0000","last_unstale":"2023-10-06T05:21:14.176705+0000","last_undegraded":"2023-10-06T05:21:14.176705+0000","last_fullsized":"2023-10-06T05:21:14.176705+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:44.869933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.189318+0000","last_change":"2023-10-06T05:21:14.186797+0000","last_active":"2023-10-06T05:21:14.189318+0000","last_peered":"2023-10-06T05:21:14.189318+0000","last_clean":"2023-10-06T05:21:14.189318+0000","last_became_active":"2023-10-06T05:21:14.186495+0000","last_became_peered":"2023-10-06T05:21:14.186495+0000","last_unstale":"2023-10-06T05:21:14.189318+0000","last_undegraded":"2023-10-06T05:21:14.189318+0000","last_fullsized":"2023-10-06T05:21:14.189318+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:11.954285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.187062+0000","last_change":"2023-10-06T05:21:14.178718+0000","last_active":"2023-10-06T05:21:14.187062+0000","last_peered":"2023-10-06T05:21:14.187062+0000","last_clean":"2023-10-06T05:21:14.187062+0000","last_became_active":"2023-10-06T05:21:14.178204+0000","last_became_peered":"2023-10-06T05:21:14.178204+0000","last_unstale":"2023-10-06T05:21:14.187062+0000","last_undegraded":"2023-10-06T05:21:14.187062+0000","last_fullsized":"2023-10-06T05:21:14.187062+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:34.358634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177158+0000","last_change":"2023-10-06T05:21:14.173162+0000","last_active":"2023-10-06T05:21:14.177158+0000","last_peered":"2023-10-06T05:21:14.177158+0000","last_clean":"2023-10-06T05:21:14.177158+0000","last_became_active":"2023-10-06T05:21:14.172561+0000","last_became_peered":"2023-10-06T05:21:14.172561+0000","last_unstale":"2023-10-06T05:21:14.177158+0000","last_undegraded":"2023-10-06T05:21:14.177158+0000","last_fullsized":"2023-10-06T05:21:14.177158+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:58.200257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178576+0000","last_change":"2023-10-06T05:21:14.172747+0000","last_active":"2023-10-06T05:21:14.178576+0000","last_peered":"2023-10-06T05:21:14.178576+0000","last_clean":"2023-10-06T05:21:14.178576+0000","last_became_active":"2023-10-06T05:21:14.172482+0000","last_became_peered":"2023-10-06T05:21:14.172482+0000","last_unstale":"2023-10-06T05:21:14.178576+0000","last_undegraded":"2023-10-06T05:21:14.178576+0000","last_fullsized":"2023-10-06T05:21:14.178576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:33.730974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178577+0000","last_change":"2023-10-06T05:21:14.170904+0000","last_active":"2023-10-06T05:21:14.178577+0000","last_peered":"2023-10-06T05:21:14.178577+0000","last_clean":"2023-10-06T05:21:14.178577+0000","last_became_active":"2023-10-06T05:21:14.170585+0000","last_became_peered":"2023-10-06T05:21:14.170585+0000","last_unstale":"2023-10-06T05:21:14.178577+0000","last_undegraded":"2023-10-06T05:21:14.178577+0000","last_fullsized":"2023-10-06T05:21:14.178577+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:05.216268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.181042+0000","last_change":"2023-10-06T05:21:14.173743+0000","last_active":"2023-10-06T05:21:14.181042+0000","last_peered":"2023-10-06T05:21:14.181042+0000","last_clean":"2023-10-06T05:21:14.181042+0000","last_became_active":"2023-10-06T05:21:14.173451+0000","last_became_peered":"2023-10-06T05:21:14.173451+0000","last_unstale":"2023-10-06T05:21:14.181042+0000","last_undegraded":"2023-10-06T05:21:14.181042+0000","last_fullsized":"2023-10-06T05:21:14.181042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:43.694140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178096+0000","last_change":"2023-10-06T05:21:14.170792+0000","last_active":"2023-10-06T05:21:14.178096+0000","last_peered":"2023-10-06T05:21:14.178096+0000","last_clean":"2023-10-06T05:21:14.178096+0000","last_became_active":"2023-10-06T05:21:14.170472+0000","last_became_peered":"2023-10-06T05:21:14.170472+0000","last_unstale":"2023-10-06T05:21:14.178096+0000","last_undegraded":"2023-10-06T05:21:14.178096+0000","last_fullsized":"2023-10-06T05:21:14.178096+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:12.167839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.190939+0000","last_change":"2023-10-06T05:21:14.187686+0000","last_active":"2023-10-06T05:21:14.190939+0000","last_peered":"2023-10-06T05:21:14.190939+0000","last_clean":"2023-10-06T05:21:14.190939+0000","last_became_active":"2023-10-06T05:21:14.187152+0000","last_became_peered":"2023-10-06T05:21:14.187152+0000","last_unstale":"2023-10-06T05:21:14.190939+0000","last_undegraded":"2023-10-06T05:21:14.190939+0000","last_fullsized":"2023-10-06T05:21:14.190939+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:48.387423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.176142+0000","last_change":"2023-10-06T05:21:14.170515+0000","last_active":"2023-10-06T05:21:14.176142+0000","last_peered":"2023-10-06T05:21:14.176142+0000","last_clean":"2023-10-06T05:21:14.176142+0000","last_became_active":"2023-10-06T05:21:14.170160+0000","last_became_peered":"2023-10-06T05:21:14.170160+0000","last_unstale":"2023-10-06T05:21:14.176142+0000","last_undegraded":"2023-10-06T05:21:14.176142+0000","last_fullsized":"2023-10-06T05:21:14.176142+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:40.977536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.187988+0000","last_change":"2023-10-06T05:21:14.182544+0000","last_active":"2023-10-06T05:21:14.187988+0000","last_peered":"2023-10-06T05:21:14.187988+0000","last_clean":"2023-10-06T05:21:14.187988+0000","last_became_active":"2023-10-06T05:21:14.182130+0000","last_became_peered":"2023-10-06T05:21:14.182130+0000","last_unstale":"2023-10-06T05:21:14.187988+0000","last_undegraded":"2023-10-06T05:21:14.187988+0000","last_fullsized":"2023-10-06T05:21:14.187988+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:11.954285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.184779+0000","last_change":"2023-10-06T05:21:14.177784+0000","last_active":"2023-10-06T05:21:14.184779+0000","last_peered":"2023-10-06T05:21:14.184779+0000","last_clean":"2023-10-06T05:21:14.184779+0000","last_became_active":"2023-10-06T05:21:14.177508+0000","last_became_peered":"2023-10-06T05:21:14.177508+0000","last_unstale":"2023-10-06T05:21:14.184779+0000","last_undegraded":"2023-10-06T05:21:14.184779+0000","last_fullsized":"2023-10-06T05:21:14.184779+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:50.021472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.189466+0000","last_change":"2023-10-06T05:21:14.187124+0000","last_active":"2023-10-06T05:21:14.189466+0000","last_peered":"2023-10-06T05:21:14.189466+0000","last_clean":"2023-10-06T05:21:14.189466+0000","last_became_active":"2023-10-06T05:21:14.186793+0000","last_became_peered":"2023-10-06T05:21:14.186793+0000","last_unstale":"2023-10-06T05:21:14.189466+0000","last_undegraded":"2023-10-06T05:21:14.189466+0000","last_fullsized":"2023-10-06T05:21:14.189466+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:44.869933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188474+0000","last_change":"2023-10-06T05:21:14.179486+0000","last_active":"2023-10-06T05:21:14.188474+0000","last_peered":"2023-10-06T05:21:14.188474+0000","last_clean":"2023-10-06T05:21:14.188474+0000","last_became_active":"2023-10-06T05:21:14.179157+0000","last_became_peered":"2023-10-06T05:21:14.179157+0000","last_unstale":"2023-10-06T05:21:14.188474+0000","last_undegraded":"2023-10-06T05:21:14.188474+0000","last_fullsized":"2023-10-06T05:21:14.188474+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:24.135963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.187799+0000","last_change":"2023-10-06T05:21:14.183092+0000","last_active":"2023-10-06T05:21:14.187799+0000","last_peered":"2023-10-06T05:21:14.187799+0000","last_clean":"2023-10-06T05:21:14.187799+0000","last_became_active":"2023-10-06T05:21:14.182773+0000","last_became_peered":"2023-10-06T05:21:14.182773+0000","last_unstale":"2023-10-06T05:21:14.187799+0000","last_undegraded":"2023-10-06T05:21:14.187799+0000","last_fullsized":"2023-10-06T05:21:14.187799+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:36.591514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.189695+0000","last_change":"2023-10-06T05:21:14.183911+0000","last_active":"2023-10-06T05:21:14.189695+0000","last_peered":"2023-10-06T05:21:14.189695+0000","last_clean":"2023-10-06T05:21:14.189695+0000","last_became_active":"2023-10-06T05:21:14.183581+0000","last_became_peered":"2023-10-06T05:21:14.183581+0000","last_unstale":"2023-10-06T05:21:14.189695+0000","last_undegraded":"2023-10-06T05:21:14.189695+0000","last_fullsized":"2023-10-06T05:21:14.189695+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:09.678269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178282+0000","last_change":"2023-10-06T05:21:14.176614+0000","last_active":"2023-10-06T05:21:14.178282+0000","last_peered":"2023-10-06T05:21:14.178282+0000","last_clean":"2023-10-06T05:21:14.178282+0000","last_became_active":"2023-10-06T05:21:14.176306+0000","last_became_peered":"2023-10-06T05:21:14.176306+0000","last_unstale":"2023-10-06T05:21:14.178282+0000","last_undegraded":"2023-10-06T05:21:14.178282+0000","last_fullsized":"2023-10-06T05:21:14.178282+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:55.307288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.182824+0000","last_change":"2023-10-06T05:21:14.180752+0000","last_active":"2023-10-06T05:21:14.182824+0000","last_peered":"2023-10-06T05:21:14.182824+0000","last_clean":"2023-10-06T05:21:14.182824+0000","last_became_active":"2023-10-06T05:21:14.180370+0000","last_became_peered":"2023-10-06T05:21:14.180370+0000","last_unstale":"2023-10-06T05:21:14.182824+0000","last_undegraded":"2023-10-06T05:21:14.182824+0000","last_fullsized":"2023-10-06T05:21:14.182824+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:09.628005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177051+0000","last_change":"2023-10-06T05:21:14.172625+0000","last_active":"2023-10-06T05:21:14.177051+0000","last_peered":"2023-10-06T05:21:14.177051+0000","last_clean":"2023-10-06T05:21:14.177051+0000","last_became_active":"2023-10-06T05:21:14.172292+0000","last_became_peered":"2023-10-06T05:21:14.172292+0000","last_unstale":"2023-10-06T05:21:14.177051+0000","last_undegraded":"2023-10-06T05:21:14.177051+0000","last_fullsized":"2023-10-06T05:21:14.177051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:41.885766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"70'4","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191330+0000","last_change":"2023-10-06T05:21:13.159241+0000","last_active":"2023-10-06T05:21:14.191330+0000","last_peered":"2023-10-06T05:21:14.191330+0000","last_clean":"2023-10-06T05:21:14.191330+0000","last_became_active":"2023-10-06T05:21:13.158913+0000","last_became_peered":"2023-10-06T05:21:13.158913+0000","last_unstale":"2023-10-06T05:21:14.191330+0000","last_undegraded":"2023-10-06T05:21:14.191330+0000","last_fullsized":"2023-10-06T05:21:14.191330+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:58.015087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.190713+0000","last_change":"2023-10-06T05:21:14.186435+0000","last_active":"2023-10-06T05:21:14.190713+0000","last_peered":"2023-10-06T05:21:14.190713+0000","last_clean":"2023-10-06T05:21:14.190713+0000","last_became_active":"2023-10-06T05:21:14.186117+0000","last_became_peered":"2023-10-06T05:21:14.186117+0000","last_unstale":"2023-10-06T05:21:14.190713+0000","last_undegraded":"2023-10-06T05:21:14.190713+0000","last_fullsized":"2023-10-06T05:21:14.190713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:09.678269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188865+0000","last_change":"2023-10-06T05:21:13.171676+0000","last_active":"2023-10-06T05:21:14.188865+0000","last_peered":"2023-10-06T05:21:14.188865+0000","last_clean":"2023-10-06T05:21:14.188865+0000","last_became_active":"2023-10-06T05:21:13.171360+0000","last_became_peered":"2023-10-06T05:21:13.171360+0000","last_unstale":"2023-10-06T05:21:14.188865+0000","last_undegraded":"2023-10-06T05:21:14.188865+0000","last_fullsized":"2023-10-06T05:21:14.188865+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:24.757454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.172558+0000","last_change":"2023-10-06T05:21:14.165541+0000","last_active":"2023-10-06T05:21:14.172558+0000","last_peered":"2023-10-06T05:21:14.172558+0000","last_clean":"2023-10-06T05:21:14.172558+0000","last_became_active":"2023-10-06T05:21:14.164858+0000","last_became_peered":"2023-10-06T05:21:14.164858+0000","last_unstale":"2023-10-06T05:21:14.172558+0000","last_undegraded":"2023-10-06T05:21:14.172558+0000","last_fullsized":"2023-10-06T05:21:14.172558+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:48.452087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188676+0000","last_change":"2023-10-06T05:21:13.176256+0000","last_active":"2023-10-06T05:21:14.188676+0000","last_peered":"2023-10-06T05:21:14.188676+0000","last_clean":"2023-10-06T05:21:14.188676+0000","last_became_active":"2023-10-06T05:21:13.175487+0000","last_became_peered":"2023-10-06T05:21:13.175487+0000","last_unstale":"2023-10-06T05:21:14.188676+0000","last_undegraded":"2023-10-06T05:21:14.188676+0000","last_fullsized":"2023-10-06T05:21:14.188676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:19.500524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177411+0000","last_change":"2023-10-06T05:21:14.172859+0000","last_active":"2023-10-06T05:21:14.177411+0000","last_peered":"2023-10-06T05:21:14.177411+0000","last_clean":"2023-10-06T05:21:14.177411+0000","last_became_active":"2023-10-06T05:21:14.172537+0000","last_became_peered":"2023-10-06T05:21:14.172537+0000","last_unstale":"2023-10-06T05:21:14.177411+0000","last_undegraded":"2023-10-06T05:21:14.177411+0000","last_fullsized":"2023-10-06T05:21:14.177411+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:09.170341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.189543+0000","last_change":"2023-10-06T05:21:13.169221+0000","last_active":"2023-10-06T05:21:14.189543+0000","last_peered":"2023-10-06T05:21:14.189543+0000","last_clean":"2023-10-06T05:21:14.189543+0000","last_became_active":"2023-10-06T05:21:13.168853+0000","last_became_peered":"2023-10-06T05:21:13.168853+0000","last_unstale":"2023-10-06T05:21:14.189543+0000","last_undegraded":"2023-10-06T05:21:14.189543+0000","last_fullsized":"2023-10-06T05:21:14.189543+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:12.290741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.186550+0000","last_change":"2023-10-06T05:21:14.175452+0000","last_active":"2023-10-06T05:21:14.186550+0000","last_peered":"2023-10-06T05:21:14.186550+0000","last_clean":"2023-10-06T05:21:14.186550+0000","last_became_active":"2023-10-06T05:21:14.175187+0000","last_became_peered":"2023-10-06T05:21:14.175187+0000","last_unstale":"2023-10-06T05:21:14.186550+0000","last_undegraded":"2023-10-06T05:21:14.186550+0000","last_fullsized":"2023-10-06T05:21:14.186550+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:58.329878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.185249+0000","last_change":"2023-10-06T05:21:13.162613+0000","last_active":"2023-10-06T05:21:14.185249+0000","last_peered":"2023-10-06T05:21:14.185249+0000","last_clean":"2023-10-06T05:21:14.185249+0000","last_became_active":"2023-10-06T05:21:13.162167+0000","last_became_peered":"2023-10-06T05:21:13.162167+0000","last_unstale":"2023-10-06T05:21:14.185249+0000","last_undegraded":"2023-10-06T05:21:14.185249+0000","last_fullsized":"2023-10-06T05:21:14.185249+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:36.834735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.186354+0000","last_change":"2023-10-06T05:21:14.176595+0000","last_active":"2023-10-06T05:21:14.186354+0000","last_peered":"2023-10-06T05:21:14.186354+0000","last_clean":"2023-10-06T05:21:14.186354+0000","last_became_active":"2023-10-06T05:21:14.176255+0000","last_became_peered":"2023-10-06T05:21:14.176255+0000","last_unstale":"2023-10-06T05:21:14.186354+0000","last_undegraded":"2023-10-06T05:21:14.186354+0000","last_fullsized":"2023-10-06T05:21:14.186354+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:13.467491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191089+0000","last_change":"2023-10-06T05:21:13.170589+0000","last_active":"2023-10-06T05:21:14.191089+0000","last_peered":"2023-10-06T05:21:14.191089+0000","last_clean":"2023-10-06T05:21:14.191089+0000","last_became_active":"2023-10-06T05:21:13.170282+0000","last_became_peered":"2023-10-06T05:21:13.170282+0000","last_unstale":"2023-10-06T05:21:14.191089+0000","last_undegraded":"2023-10-06T05:21:14.191089+0000","last_fullsized":"2023-10-06T05:21:14.191089+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:53.541975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.173404+0000","last_change":"2023-10-06T05:21:14.166677+0000","last_active":"2023-10-06T05:21:14.173404+0000","last_peered":"2023-10-06T05:21:14.173404+0000","last_clean":"2023-10-06T05:21:14.173404+0000","last_became_active":"2023-10-06T05:21:14.165665+0000","last_became_peered":"2023-10-06T05:21:14.165665+0000","last_unstale":"2023-10-06T05:21:14.173404+0000","last_undegraded":"2023-10-06T05:21:14.173404+0000","last_fullsized":"2023-10-06T05:21:14.173404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:54.544554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.164878+0000","last_change":"2023-10-06T05:21:13.175791+0000","last_active":"2023-10-06T05:21:14.164878+0000","last_peered":"2023-10-06T05:21:14.164878+0000","last_clean":"2023-10-06T05:21:14.164878+0000","last_became_active":"2023-10-06T05:21:13.175326+0000","last_became_peered":"2023-10-06T05:21:13.175326+0000","last_unstale":"2023-10-06T05:21:14.164878+0000","last_undegraded":"2023-10-06T05:21:14.164878+0000","last_fullsized":"2023-10-06T05:21:14.164878+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.040745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178777+0000","last_change":"2023-10-06T05:21:14.175687+0000","last_active":"2023-10-06T05:21:14.178777+0000","last_peered":"2023-10-06T05:21:14.178777+0000","last_clean":"2023-10-06T05:21:14.178777+0000","last_became_active":"2023-10-06T05:21:14.175356+0000","last_became_peered":"2023-10-06T05:21:14.175356+0000","last_unstale":"2023-10-06T05:21:14.178777+0000","last_undegraded":"2023-10-06T05:21:14.178777+0000","last_fullsized":"2023-10-06T05:21:14.178777+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:37.837314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.160846+0000","last_change":"2023-10-06T05:21:13.170180+0000","last_active":"2023-10-06T05:21:14.160846+0000","last_peered":"2023-10-06T05:21:14.160846+0000","last_clean":"2023-10-06T05:21:14.160846+0000","last_became_active":"2023-10-06T05:21:13.169857+0000","last_became_peered":"2023-10-06T05:21:13.169857+0000","last_unstale":"2023-10-06T05:21:14.160846+0000","last_undegraded":"2023-10-06T05:21:14.160846+0000","last_fullsized":"2023-10-06T05:21:14.160846+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:45.593601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.179005+0000","last_change":"2023-10-06T05:21:14.173085+0000","last_active":"2023-10-06T05:21:14.179005+0000","last_peered":"2023-10-06T05:21:14.179005+0000","last_clean":"2023-10-06T05:21:14.179005+0000","last_became_active":"2023-10-06T05:21:14.172809+0000","last_became_peered":"2023-10-06T05:21:14.172809+0000","last_unstale":"2023-10-06T05:21:14.179005+0000","last_undegraded":"2023-10-06T05:21:14.179005+0000","last_fullsized":"2023-10-06T05:21:14.179005+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.976026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.165824+0000","last_change":"2023-10-06T05:21:13.172453+0000","last_active":"2023-10-06T05:21:14.165824+0000","last_peered":"2023-10-06T05:21:14.165824+0000","last_clean":"2023-10-06T05:21:14.165824+0000","last_became_active":"2023-10-06T05:21:13.171973+0000","last_became_peered":"2023-10-06T05:21:13.171973+0000","last_unstale":"2023-10-06T05:21:14.165824+0000","last_undegraded":"2023-10-06T05:21:14.165824+0000","last_fullsized":"2023-10-06T05:21:14.165824+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:53.470594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.186146+0000","last_change":"2023-10-06T05:21:14.184231+0000","last_active":"2023-10-06T05:21:14.186146+0000","last_peered":"2023-10-06T05:21:14.186146+0000","last_clean":"2023-10-06T05:21:14.186146+0000","last_became_active":"2023-10-06T05:21:14.183847+0000","last_became_peered":"2023-10-06T05:21:14.183847+0000","last_unstale":"2023-10-06T05:21:14.186146+0000","last_undegraded":"2023-10-06T05:21:14.186146+0000","last_fullsized":"2023-10-06T05:21:14.186146+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:52.043325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.160235+0000","last_change":"2023-10-06T05:21:13.168940+0000","last_active":"2023-10-06T05:21:14.160235+0000","last_peered":"2023-10-06T05:21:14.160235+0000","last_clean":"2023-10-06T05:21:14.160235+0000","last_became_active":"2023-10-06T05:21:13.168641+0000","last_became_peered":"2023-10-06T05:21:13.168641+0000","last_unstale":"2023-10-06T05:21:14.160235+0000","last_undegraded":"2023-10-06T05:21:14.160235+0000","last_fullsized":"2023-10-06T05:21:14.160235+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:42.691561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177501+0000","last_change":"2023-10-06T05:21:14.169447+0000","last_active":"2023-10-06T05:21:14.177501+0000","last_peered":"2023-10-06T05:21:14.177501+0000","last_clean":"2023-10-06T05:21:14.177501+0000","last_became_active":"2023-10-06T05:21:14.169082+0000","last_became_peered":"2023-10-06T05:21:14.169082+0000","last_unstale":"2023-10-06T05:21:14.177501+0000","last_undegraded":"2023-10-06T05:21:14.177501+0000","last_fullsized":"2023-10-06T05:21:14.177501+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:04.855790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"70'3","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191663+0000","last_change":"2023-10-06T05:21:13.163900+0000","last_active":"2023-10-06T05:21:14.191663+0000","last_peered":"2023-10-06T05:21:14.191663+0000","last_clean":"2023-10-06T05:21:14.191663+0000","last_became_active":"2023-10-06T05:21:13.163604+0000","last_became_peered":"2023-10-06T05:21:13.163604+0000","last_unstale":"2023-10-06T05:21:14.191663+0000","last_undegraded":"2023-10-06T05:21:14.191663+0000","last_fullsized":"2023-10-06T05:21:14.191663+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:55.585059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178253+0000","last_change":"2023-10-06T05:21:14.175350+0000","last_active":"2023-10-06T05:21:14.178253+0000","last_peered":"2023-10-06T05:21:14.178253+0000","last_clean":"2023-10-06T05:21:14.178253+0000","last_became_active":"2023-10-06T05:21:14.174791+0000","last_became_peered":"2023-10-06T05:21:14.174791+0000","last_unstale":"2023-10-06T05:21:14.178253+0000","last_undegraded":"2023-10-06T05:21:14.178253+0000","last_fullsized":"2023-10-06T05:21:14.178253+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:50.374014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"70'6","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.194885+0000","last_change":"2023-10-06T05:21:13.170791+0000","last_active":"2023-10-06T05:21:14.194885+0000","last_peered":"2023-10-06T05:21:14.194885+0000","last_clean":"2023-10-06T05:21:14.194885+0000","last_became_active":"2023-10-06T05:21:13.170470+0000","last_became_peered":"2023-10-06T05:21:13.170470+0000","last_unstale":"2023-10-06T05:21:14.194885+0000","last_undegraded":"2023-10-06T05:21:14.194885+0000","last_fullsized":"2023-10-06T05:21:14.194885+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:26.418570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.186174+0000","last_change":"2023-10-06T05:21:14.183193+0000","last_active":"2023-10-06T05:21:14.186174+0000","last_peered":"2023-10-06T05:21:14.186174+0000","last_clean":"2023-10-06T05:21:14.186174+0000","last_became_active":"2023-10-06T05:21:14.178379+0000","last_became_peered":"2023-10-06T05:21:14.178379+0000","last_unstale":"2023-10-06T05:21:14.186174+0000","last_undegraded":"2023-10-06T05:21:14.186174+0000","last_fullsized":"2023-10-06T05:21:14.186174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:45.462119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.193439+0000","last_change":"2023-10-06T05:21:13.171211+0000","last_active":"2023-10-06T05:21:14.193439+0000","last_peered":"2023-10-06T05:21:14.193439+0000","last_clean":"2023-10-06T05:21:14.193439+0000","last_became_active":"2023-10-06T05:21:13.170917+0000","last_became_peered":"2023-10-06T05:21:13.170917+0000","last_unstale":"2023-10-06T05:21:14.193439+0000","last_undegraded":"2023-10-06T05:21:14.193439+0000","last_fullsized":"2023-10-06T05:21:14.193439+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:08.167762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177573+0000","last_change":"2023-10-06T05:21:14.174706+0000","last_active":"2023-10-06T05:21:14.177573+0000","last_peered":"2023-10-06T05:21:14.177573+0000","last_clean":"2023-10-06T05:21:14.177573+0000","last_became_active":"2023-10-06T05:21:14.174424+0000","last_became_peered":"2023-10-06T05:21:14.174424+0000","last_unstale":"2023-10-06T05:21:14.177573+0000","last_undegraded":"2023-10-06T05:21:14.177573+0000","last_fullsized":"2023-10-06T05:21:14.177573+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:14.512710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.163365+0000","last_change":"2023-10-06T05:21:13.171789+0000","last_active":"2023-10-06T05:21:14.163365+0000","last_peered":"2023-10-06T05:21:14.163365+0000","last_clean":"2023-10-06T05:21:14.163365+0000","last_became_active":"2023-10-06T05:21:13.171451+0000","last_became_peered":"2023-10-06T05:21:13.171451+0000","last_unstale":"2023-10-06T05:21:14.163365+0000","last_undegraded":"2023-10-06T05:21:14.163365+0000","last_fullsized":"2023-10-06T05:21:14.163365+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:11.973352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.182329+0000","last_change":"2023-10-06T05:21:14.177560+0000","last_active":"2023-10-06T05:21:14.182329+0000","last_peered":"2023-10-06T05:21:14.182329+0000","last_clean":"2023-10-06T05:21:14.182329+0000","last_became_active":"2023-10-06T05:21:14.177108+0000","last_became_peered":"2023-10-06T05:21:14.177108+0000","last_unstale":"2023-10-06T05:21:14.182329+0000","last_undegraded":"2023-10-06T05:21:14.182329+0000","last_fullsized":"2023-10-06T05:21:14.182329+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:25.096218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.160903+0000","last_change":"2023-10-06T05:21:13.171171+0000","last_active":"2023-10-06T05:21:14.160903+0000","last_peered":"2023-10-06T05:21:14.160903+0000","last_clean":"2023-10-06T05:21:14.160903+0000","last_became_active":"2023-10-06T05:21:13.170856+0000","last_became_peered":"2023-10-06T05:21:13.170856+0000","last_unstale":"2023-10-06T05:21:14.160903+0000","last_undegraded":"2023-10-06T05:21:14.160903+0000","last_fullsized":"2023-10-06T05:21:14.160903+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:45.319480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.183798+0000","last_change":"2023-10-06T05:21:14.178892+0000","last_active":"2023-10-06T05:21:14.183798+0000","last_peered":"2023-10-06T05:21:14.183798+0000","last_clean":"2023-10-06T05:21:14.183798+0000","last_became_active":"2023-10-06T05:21:14.178604+0000","last_became_peered":"2023-10-06T05:21:14.178604+0000","last_unstale":"2023-10-06T05:21:14.183798+0000","last_undegraded":"2023-10-06T05:21:14.183798+0000","last_fullsized":"2023-10-06T05:21:14.183798+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:44.494854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"70'8","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.192861+0000","last_change":"2023-10-06T05:21:13.176681+0000","last_active":"2023-10-06T05:21:14.192861+0000","last_peered":"2023-10-06T05:21:14.192861+0000","last_clean":"2023-10-06T05:21:14.192861+0000","last_became_active":"2023-10-06T05:21:13.176432+0000","last_became_peered":"2023-10-06T05:21:13.176432+0000","last_unstale":"2023-10-06T05:21:14.192861+0000","last_undegraded":"2023-10-06T05:21:14.192861+0000","last_fullsized":"2023-10-06T05:21:14.192861+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:15.808879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.184727+0000","last_change":"2023-10-06T05:21:14.179416+0000","last_active":"2023-10-06T05:21:14.184727+0000","last_peered":"2023-10-06T05:21:14.184727+0000","last_clean":"2023-10-06T05:21:14.184727+0000","last_became_active":"2023-10-06T05:21:14.179112+0000","last_became_peered":"2023-10-06T05:21:14.179112+0000","last_unstale":"2023-10-06T05:21:14.184727+0000","last_undegraded":"2023-10-06T05:21:14.184727+0000","last_fullsized":"2023-10-06T05:21:14.184727+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:06.020289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191736+0000","last_change":"2023-10-06T05:21:13.165877+0000","last_active":"2023-10-06T05:21:14.191736+0000","last_peered":"2023-10-06T05:21:14.191736+0000","last_clean":"2023-10-06T05:21:14.191736+0000","last_became_active":"2023-10-06T05:21:13.165605+0000","last_became_peered":"2023-10-06T05:21:13.165605+0000","last_unstale":"2023-10-06T05:21:14.191736+0000","last_undegraded":"2023-10-06T05:21:14.191736+0000","last_fullsized":"2023-10-06T05:21:14.191736+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:03.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.182367+0000","last_change":"2023-10-06T05:21:14.177458+0000","last_active":"2023-10-06T05:21:14.182367+0000","last_peered":"2023-10-06T05:21:14.182367+0000","last_clean":"2023-10-06T05:21:14.182367+0000","last_became_active":"2023-10-06T05:21:14.177186+0000","last_became_peered":"2023-10-06T05:21:14.177186+0000","last_unstale":"2023-10-06T05:21:14.182367+0000","last_undegraded":"2023-10-06T05:21:14.182367+0000","last_fullsized":"2023-10-06T05:21:14.182367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:22.094091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"71'1","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-06T05:21:16.417743+0000","last_change":"2023-10-06T05:21:13.171761+0000","last_active":"2023-10-06T05:21:16.417743+0000","last_peered":"2023-10-06T05:21:16.417743+0000","last_clean":"2023-10-06T05:21:16.417743+0000","last_became_active":"2023-10-06T05:21:13.171491+0000","last_became_peered":"2023-10-06T05:21:13.171491+0000","last_unstale":"2023-10-06T05:21:16.417743+0000","last_undegraded":"2023-10-06T05:21:16.417743+0000","last_fullsized":"2023-10-06T05:21:16.417743+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:56.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.172586+0000","last_change":"2023-10-06T05:21:14.165763+0000","last_active":"2023-10-06T05:21:14.172586+0000","last_peered":"2023-10-06T05:21:14.172586+0000","last_clean":"2023-10-06T05:21:14.172586+0000","last_became_active":"2023-10-06T05:21:14.165281+0000","last_became_peered":"2023-10-06T05:21:14.165281+0000","last_unstale":"2023-10-06T05:21:14.172586+0000","last_undegraded":"2023-10-06T05:21:14.172586+0000","last_fullsized":"2023-10-06T05:21:14.172586+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:04.797474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.165699+0000","last_change":"2023-10-06T05:21:13.167952+0000","last_active":"2023-10-06T05:21:14.165699+0000","last_peered":"2023-10-06T05:21:14.165699+0000","last_clean":"2023-10-06T05:21:14.165699+0000","last_became_active":"2023-10-06T05:21:13.166565+0000","last_became_peered":"2023-10-06T05:21:13.166565+0000","last_unstale":"2023-10-06T05:21:14.165699+0000","last_undegraded":"2023-10-06T05:21:14.165699+0000","last_fullsized":"2023-10-06T05:21:14.165699+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:38.525356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.179036+0000","last_change":"2023-10-06T05:21:14.173602+0000","last_active":"2023-10-06T05:21:14.179036+0000","last_peered":"2023-10-06T05:21:14.179036+0000","last_clean":"2023-10-06T05:21:14.179036+0000","last_became_active":"2023-10-06T05:21:14.173343+0000","last_became_peered":"2023-10-06T05:21:14.173343+0000","last_unstale":"2023-10-06T05:21:14.179036+0000","last_undegraded":"2023-10-06T05:21:14.179036+0000","last_fullsized":"2023-10-06T05:21:14.179036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:46.254829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.164715+0000","last_change":"2023-10-06T05:21:13.171867+0000","last_active":"2023-10-06T05:21:14.164715+0000","last_peered":"2023-10-06T05:21:14.164715+0000","last_clean":"2023-10-06T05:21:14.164715+0000","last_became_active":"2023-10-06T05:21:13.171530+0000","last_became_peered":"2023-10-06T05:21:13.171530+0000","last_unstale":"2023-10-06T05:21:14.164715+0000","last_undegraded":"2023-10-06T05:21:14.164715+0000","last_fullsized":"2023-10-06T05:21:14.164715+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:28.778795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.186269+0000","last_change":"2023-10-06T05:21:14.180578+0000","last_active":"2023-10-06T05:21:14.186269+0000","last_peered":"2023-10-06T05:21:14.186269+0000","last_clean":"2023-10-06T05:21:14.186269+0000","last_became_active":"2023-10-06T05:21:14.180270+0000","last_became_peered":"2023-10-06T05:21:14.180270+0000","last_unstale":"2023-10-06T05:21:14.186269+0000","last_undegraded":"2023-10-06T05:21:14.186269+0000","last_fullsized":"2023-10-06T05:21:14.186269+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:22.127378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.165873+0000","last_change":"2023-10-06T05:21:13.172743+0000","last_active":"2023-10-06T05:21:14.165873+0000","last_peered":"2023-10-06T05:21:14.165873+0000","last_clean":"2023-10-06T05:21:14.165873+0000","last_became_active":"2023-10-06T05:21:13.172453+0000","last_became_peered":"2023-10-06T05:21:13.172453+0000","last_unstale":"2023-10-06T05:21:14.165873+0000","last_undegraded":"2023-10-06T05:21:14.165873+0000","last_fullsized":"2023-10-06T05:21:14.165873+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:01.292501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.182295+0000","last_change":"2023-10-06T05:21:14.176866+0000","last_active":"2023-10-06T05:21:14.182295+0000","last_peered":"2023-10-06T05:21:14.182295+0000","last_clean":"2023-10-06T05:21:14.182295+0000","last_became_active":"2023-10-06T05:21:14.176561+0000","last_became_peered":"2023-10-06T05:21:14.176561+0000","last_unstale":"2023-10-06T05:21:14.182295+0000","last_undegraded":"2023-10-06T05:21:14.182295+0000","last_fullsized":"2023-10-06T05:21:14.182295+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:20.672647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.161010+0000","last_change":"2023-10-06T05:21:13.171851+0000","last_active":"2023-10-06T05:21:14.161010+0000","last_peered":"2023-10-06T05:21:14.161010+0000","last_clean":"2023-10-06T05:21:14.161010+0000","last_became_active":"2023-10-06T05:21:13.171552+0000","last_became_peered":"2023-10-06T05:21:13.171552+0000","last_unstale":"2023-10-06T05:21:14.161010+0000","last_undegraded":"2023-10-06T05:21:14.161010+0000","last_fullsized":"2023-10-06T05:21:14.161010+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:14:14.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.177138+0000","last_change":"2023-10-06T05:21:14.172782+0000","last_active":"2023-10-06T05:21:14.177138+0000","last_peered":"2023-10-06T05:21:14.177138+0000","last_clean":"2023-10-06T05:21:14.177138+0000","last_became_active":"2023-10-06T05:21:14.172172+0000","last_became_peered":"2023-10-06T05:21:14.172172+0000","last_unstale":"2023-10-06T05:21:14.177138+0000","last_undegraded":"2023-10-06T05:21:14.177138+0000","last_fullsized":"2023-10-06T05:21:14.177138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:34.203137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.167547+0000","last_change":"2023-10-06T05:21:13.172335+0000","last_active":"2023-10-06T05:21:14.167547+0000","last_peered":"2023-10-06T05:21:14.167547+0000","last_clean":"2023-10-06T05:21:14.167547+0000","last_became_active":"2023-10-06T05:21:13.172046+0000","last_became_peered":"2023-10-06T05:21:13.172046+0000","last_unstale":"2023-10-06T05:21:14.167547+0000","last_undegraded":"2023-10-06T05:21:14.167547+0000","last_fullsized":"2023-10-06T05:21:14.167547+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:49.234603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.168079+0000","last_change":"2023-10-06T05:21:14.164995+0000","last_active":"2023-10-06T05:21:14.168079+0000","last_peered":"2023-10-06T05:21:14.168079+0000","last_clean":"2023-10-06T05:21:14.168079+0000","last_became_active":"2023-10-06T05:21:14.164682+0000","last_became_peered":"2023-10-06T05:21:14.164682+0000","last_unstale":"2023-10-06T05:21:14.168079+0000","last_undegraded":"2023-10-06T05:21:14.168079+0000","last_fullsized":"2023-10-06T05:21:14.168079+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:23.212689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.153382+0000","last_change":"2023-10-06T05:21:13.174008+0000","last_active":"2023-10-06T05:21:14.153382+0000","last_peered":"2023-10-06T05:21:14.153382+0000","last_clean":"2023-10-06T05:21:14.153382+0000","last_became_active":"2023-10-06T05:21:13.173503+0000","last_became_peered":"2023-10-06T05:21:13.173503+0000","last_unstale":"2023-10-06T05:21:14.153382+0000","last_undegraded":"2023-10-06T05:21:14.153382+0000","last_fullsized":"2023-10-06T05:21:14.153382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:14.164789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178686+0000","last_change":"2023-10-06T05:21:14.174626+0000","last_active":"2023-10-06T05:21:14.178686+0000","last_peered":"2023-10-06T05:21:14.178686+0000","last_clean":"2023-10-06T05:21:14.178686+0000","last_became_active":"2023-10-06T05:21:14.174013+0000","last_became_peered":"2023-10-06T05:21:14.174013+0000","last_unstale":"2023-10-06T05:21:14.178686+0000","last_undegraded":"2023-10-06T05:21:14.178686+0000","last_fullsized":"2023-10-06T05:21:14.178686+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:39.527935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.183455+0000","last_change":"2023-10-06T05:21:13.165946+0000","last_active":"2023-10-06T05:21:14.183455+0000","last_peered":"2023-10-06T05:21:14.183455+0000","last_clean":"2023-10-06T05:21:14.183455+0000","last_became_active":"2023-10-06T05:21:13.165677+0000","last_became_peered":"2023-10-06T05:21:13.165677+0000","last_unstale":"2023-10-06T05:21:14.183455+0000","last_undegraded":"2023-10-06T05:21:14.183455+0000","last_fullsized":"2023-10-06T05:21:14.183455+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:00:32.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":59,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.155638+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:21:14.155638+0000","last_peered":"2023-10-06T05:21:14.155638+0000","last_clean":"2023-10-06T05:21:14.155638+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:21:14.155638+0000","last_undegraded":"2023-10-06T05:21:14.155638+0000","last_fullsized":"2023-10-06T05:21:14.155638+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.181376+0000","last_change":"2023-10-06T05:21:14.178020+0000","last_active":"2023-10-06T05:21:14.181376+0000","last_peered":"2023-10-06T05:21:14.181376+0000","last_clean":"2023-10-06T05:21:14.181376+0000","last_became_active":"2023-10-06T05:21:14.177661+0000","last_became_peered":"2023-10-06T05:21:14.177661+0000","last_unstale":"2023-10-06T05:21:14.181376+0000","last_undegraded":"2023-10-06T05:21:14.181376+0000","last_fullsized":"2023-10-06T05:21:14.181376+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:10.252525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188058+0000","last_change":"2023-10-06T05:21:13.165590+0000","last_active":"2023-10-06T05:21:14.188058+0000","last_peered":"2023-10-06T05:21:14.188058+0000","last_clean":"2023-10-06T05:21:14.188058+0000","last_became_active":"2023-10-06T05:21:13.165314+0000","last_became_peered":"2023-10-06T05:21:13.165314+0000","last_unstale":"2023-10-06T05:21:14.188058+0000","last_undegraded":"2023-10-06T05:21:14.188058+0000","last_fullsized":"2023-10-06T05:21:14.188058+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:24.093639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.156112+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:21:14.156112+0000","last_peered":"2023-10-06T05:21:14.156112+0000","last_clean":"2023-10-06T05:21:14.156112+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:21:14.156112+0000","last_undegraded":"2023-10-06T05:21:14.156112+0000","last_fullsized":"2023-10-06T05:21:14.156112+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.187934+0000","last_change":"2023-10-06T05:21:14.180321+0000","last_active":"2023-10-06T05:21:14.187934+0000","last_peered":"2023-10-06T05:21:14.187934+0000","last_clean":"2023-10-06T05:21:14.187934+0000","last_became_active":"2023-10-06T05:21:14.179982+0000","last_became_peered":"2023-10-06T05:21:14.179982+0000","last_unstale":"2023-10-06T05:21:14.187934+0000","last_undegraded":"2023-10-06T05:21:14.187934+0000","last_fullsized":"2023-10-06T05:21:14.187934+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:36.647693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.156177+0000","last_change":"2023-10-06T05:21:13.166267+0000","last_active":"2023-10-06T05:21:14.156177+0000","last_peered":"2023-10-06T05:21:14.156177+0000","last_clean":"2023-10-06T05:21:14.156177+0000","last_became_active":"2023-10-06T05:21:13.165887+0000","last_became_peered":"2023-10-06T05:21:13.165887+0000","last_unstale":"2023-10-06T05:21:14.156177+0000","last_undegraded":"2023-10-06T05:21:14.156177+0000","last_fullsized":"2023-10-06T05:21:14.156177+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:56.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.154477+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:21:14.154477+0000","last_peered":"2023-10-06T05:21:14.154477+0000","last_clean":"2023-10-06T05:21:14.154477+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:21:14.154477+0000","last_undegraded":"2023-10-06T05:21:14.154477+0000","last_fullsized":"2023-10-06T05:21:14.154477+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":204,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.164043+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:21:14.164043+0000","last_peered":"2023-10-06T05:21:14.164043+0000","last_clean":"2023-10-06T05:21:14.164043+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:21:14.164043+0000","last_undegraded":"2023-10-06T05:21:14.164043+0000","last_fullsized":"2023-10-06T05:21:14.164043+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.189282+0000","last_change":"2023-10-06T05:21:14.187192+0000","last_active":"2023-10-06T05:21:14.189282+0000","last_peered":"2023-10-06T05:21:14.189282+0000","last_clean":"2023-10-06T05:21:14.189282+0000","last_became_active":"2023-10-06T05:21:14.186922+0000","last_became_peered":"2023-10-06T05:21:14.186922+0000","last_unstale":"2023-10-06T05:21:14.189282+0000","last_undegraded":"2023-10-06T05:21:14.189282+0000","last_fullsized":"2023-10-06T05:21:14.189282+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:45.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.156653+0000","last_change":"2023-10-06T05:21:13.168485+0000","last_active":"2023-10-06T05:21:14.156653+0000","last_peered":"2023-10-06T05:21:14.156653+0000","last_clean":"2023-10-06T05:21:14.156653+0000","last_became_active":"2023-10-06T05:21:13.168162+0000","last_became_peered":"2023-10-06T05:21:13.168162+0000","last_unstale":"2023-10-06T05:21:14.156653+0000","last_undegraded":"2023-10-06T05:21:14.156653+0000","last_fullsized":"2023-10-06T05:21:14.156653+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:51.040745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.163449+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:21:14.163449+0000","last_peered":"2023-10-06T05:21:14.163449+0000","last_clean":"2023-10-06T05:21:14.163449+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:21:14.163449+0000","last_undegraded":"2023-10-06T05:21:14.163449+0000","last_fullsized":"2023-10-06T05:21:14.163449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178513+0000","last_change":"2023-10-06T05:21:14.171617+0000","last_active":"2023-10-06T05:21:14.178513+0000","last_peered":"2023-10-06T05:21:14.178513+0000","last_clean":"2023-10-06T05:21:14.178513+0000","last_became_active":"2023-10-06T05:21:14.171336+0000","last_became_peered":"2023-10-06T05:21:14.171336+0000","last_unstale":"2023-10-06T05:21:14.178513+0000","last_undegraded":"2023-10-06T05:21:14.178513+0000","last_fullsized":"2023-10-06T05:21:14.178513+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:16.803474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.156598+0000","last_change":"2023-10-06T05:21:13.171949+0000","last_active":"2023-10-06T05:21:14.156598+0000","last_peered":"2023-10-06T05:21:14.156598+0000","last_clean":"2023-10-06T05:21:14.156598+0000","last_became_active":"2023-10-06T05:21:13.171646+0000","last_became_peered":"2023-10-06T05:21:13.171646+0000","last_unstale":"2023-10-06T05:21:14.156598+0000","last_undegraded":"2023-10-06T05:21:14.156598+0000","last_fullsized":"2023-10-06T05:21:14.156598+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:00.022649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.164135+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:21:14.164135+0000","last_peered":"2023-10-06T05:21:14.164135+0000","last_clean":"2023-10-06T05:21:14.164135+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:21:14.164135+0000","last_undegraded":"2023-10-06T05:21:14.164135+0000","last_fullsized":"2023-10-06T05:21:14.164135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191317+0000","last_change":"2023-10-06T05:21:14.187053+0000","last_active":"2023-10-06T05:21:14.191317+0000","last_peered":"2023-10-06T05:21:14.191317+0000","last_clean":"2023-10-06T05:21:14.191317+0000","last_became_active":"2023-10-06T05:21:14.186478+0000","last_became_peered":"2023-10-06T05:21:14.186478+0000","last_unstale":"2023-10-06T05:21:14.191317+0000","last_undegraded":"2023-10-06T05:21:14.191317+0000","last_fullsized":"2023-10-06T05:21:14.191317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:16.803474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"70'2","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.191563+0000","last_change":"2023-10-06T05:21:13.168910+0000","last_active":"2023-10-06T05:21:14.191563+0000","last_peered":"2023-10-06T05:21:14.191563+0000","last_clean":"2023-10-06T05:21:14.191563+0000","last_became_active":"2023-10-06T05:21:13.168616+0000","last_became_peered":"2023-10-06T05:21:13.168616+0000","last_unstale":"2023-10-06T05:21:14.191563+0000","last_undegraded":"2023-10-06T05:21:14.191563+0000","last_fullsized":"2023-10-06T05:21:14.191563+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:15.808879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.162156+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:21:14.162156+0000","last_peered":"2023-10-06T05:21:14.162156+0000","last_clean":"2023-10-06T05:21:14.162156+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:21:14.162156+0000","last_undegraded":"2023-10-06T05:21:14.162156+0000","last_fullsized":"2023-10-06T05:21:14.162156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.178767+0000","last_change":"2023-10-06T05:21:14.173537+0000","last_active":"2023-10-06T05:21:14.178767+0000","last_peered":"2023-10-06T05:21:14.178767+0000","last_clean":"2023-10-06T05:21:14.178767+0000","last_became_active":"2023-10-06T05:21:14.172896+0000","last_became_peered":"2023-10-06T05:21:14.172896+0000","last_unstale":"2023-10-06T05:21:14.178767+0000","last_undegraded":"2023-10-06T05:21:14.178767+0000","last_fullsized":"2023-10-06T05:21:14.178767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:21.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.153679+0000","last_change":"2023-10-06T05:21:13.166892+0000","last_active":"2023-10-06T05:21:14.153679+0000","last_peered":"2023-10-06T05:21:14.153679+0000","last_clean":"2023-10-06T05:21:14.153679+0000","last_became_active":"2023-10-06T05:21:13.166496+0000","last_became_peered":"2023-10-06T05:21:13.166496+0000","last_unstale":"2023-10-06T05:21:14.153679+0000","last_undegraded":"2023-10-06T05:21:14.153679+0000","last_fullsized":"2023-10-06T05:21:14.153679+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:13.510131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.167144+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:21:14.167144+0000","last_peered":"2023-10-06T05:21:14.167144+0000","last_clean":"2023-10-06T05:21:14.167144+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:21:14.167144+0000","last_undegraded":"2023-10-06T05:21:14.167144+0000","last_fullsized":"2023-10-06T05:21:14.167144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.188569+0000","last_change":"2023-10-06T05:21:14.180138+0000","last_active":"2023-10-06T05:21:14.188569+0000","last_peered":"2023-10-06T05:21:14.188569+0000","last_clean":"2023-10-06T05:21:14.188569+0000","last_became_active":"2023-10-06T05:21:14.179803+0000","last_became_peered":"2023-10-06T05:21:14.179803+0000","last_unstale":"2023-10-06T05:21:14.188569+0000","last_undegraded":"2023-10-06T05:21:14.188569+0000","last_fullsized":"2023-10-06T05:21:14.188569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:13.145495+0000","last_clean_scrub_stamp":"2023-10-06T05:21:13.145495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:15.241769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"70'1","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.192315+0000","last_change":"2023-10-06T05:21:13.159342+0000","last_active":"2023-10-06T05:21:14.192315+0000","last_peered":"2023-10-06T05:21:14.192315+0000","last_clean":"2023-10-06T05:21:14.192315+0000","last_became_active":"2023-10-06T05:21:13.158741+0000","last_became_peered":"2023-10-06T05:21:13.158741+0000","last_unstale":"2023-10-06T05:21:14.192315+0000","last_undegraded":"2023-10-06T05:21:14.192315+0000","last_fullsized":"2023-10-06T05:21:14.192315+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:12.142916+0000","last_clean_scrub_stamp":"2023-10-06T05:21:12.142916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:22.191827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-10-06T05:21:14.166711+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:21:14.166711+0000","last_peered":"2023-10-06T05:21:14.166711+0000","last_clean":"2023-10-06T05:21:14.166711+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:21:14.166711+0000","last_undegraded":"2023-10-06T05:21:14.166711+0000","last_fullsized":"2023-10-06T05:21:14.166711+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298716,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073124,"statfs":{"total":96636764160,"available":96330878976,"internally_reserved":0,"allocated":1462272,"data_stored":976395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.68200000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93600000000000005,"5min":1.018,"15min":1.018},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95999999999999996,"5min":1.056,"15min":1.056},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73399999999999999,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.83899999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67700000000000005,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.748},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93300000000000005,"5min":0.999,"15min":0.999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.83799999999999997},"min":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0389999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.0569999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.626,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.85399999999999998},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0029999999999999,"5min":1.103,"15min":1.103},"last":1.012}]}]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":387541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.47099999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89800000000000002,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.77100000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90800000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78500000000000003,"5min":1.369,"15min":1.369},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.891,"15min":1.891},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89800000000000002,"5min":1.347,"15min":1.347},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":1,"15min":1},"last":0.79800000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73499999999999999,"15min":0.78500000000000003},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.95499999999999996,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78200000000000003,"15min":0.82399999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85899999999999999,"5min":1.415,"15min":1.415},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77500000000000002,"15min":0.83099999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98199999999999998,"5min":2,"15min":2},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.78000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95999999999999996,"5min":1.46,"15min":1.46},"last":0.85599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298720,"kb_used_data":1432,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073120,"statfs":{"total":96636764160,"available":96330874880,"internally_reserved":0,"allocated":1466368,"data_stored":976485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.443}]},{"osd":3,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.59699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82499999999999996},"min":{"1min":0.63600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.753},"min":{"1min":0.621,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.052,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.82999999999999996,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.83499999999999996,"15min":0.82499999999999996},"min":{"1min":0.67100000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.69299999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.087,"5min":1.103,"15min":1.103},"last":0.80900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":386005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68300000000000005,"5min":1.042,"15min":1.042},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.441},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1519999999999999,"5min":1.242,"15min":1.242},"last":0.627},{"interface":"front","average":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":1.198,"15min":1.198},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.77300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1659999999999999,"5min":1.26,"15min":1.26},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.073,"5min":1.323,"15min":1.323},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.123,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.89900000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.264,"15min":1.264},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.04,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.65400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298144,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073696,"statfs":{"total":96636764160,"available":96331464704,"internally_reserved":0,"allocated":876544,"data_stored":386719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46500000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.624,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.503},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68799999999999994,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93500000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90600000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96299999999999997,"5min":1.099,"15min":1.099},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78500000000000003,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.90000000000000002,"5min":1.163,"15min":1.163},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98799999999999999,"5min":1.21,"15min":1.21},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.022,"5min":1.042,"15min":1.042},"last":0.91100000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84399999999999997,"15min":0.872},"min":{"1min":0.64600000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0469999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.82299999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298136,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073704,"statfs":{"total":96636764160,"available":96331472896,"internally_reserved":0,"allocated":868352,"data_stored":386559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.44600000000000001,"15min":0.46200000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57199999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.624,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72299999999999998,"5min":1.014,"15min":1.014},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.145,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1830000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.78700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.03,"5min":1.204,"15min":1.204},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.329},"last":0.54800000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.82099999999999995},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.048,"5min":1.484,"15min":1.484},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83499999999999996,"15min":0.85399999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1930000000000001,"5min":1.629,"15min":1.629},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.83999999999999997,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1950000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.085,"5min":1.325,"15min":1.325},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298152,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073688,"statfs":{"total":96636764160,"available":96331456512,"internally_reserved":0,"allocated":884736,"data_stored":387645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74399999999999999,"15min":0.79800000000000004},"min":{"1min":0.52100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":2.77,"15min":2.77},"last":0.73499999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.76600000000000001,"15min":0.82999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89600000000000002,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.58599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.90500000000000003,"15min":1.091},"min":{"1min":0.50800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":7.742,"15min":7.742},"last":0.88600000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.48999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96299999999999997,"5min":1.079,"15min":1.079},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90900000000000003,"5min":1.181,"15min":1.181},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.745},"min":{"1min":0.56499999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.751},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.60599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.92200000000000004,"5min":1.171,"15min":1.171},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.624,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96499999999999997,"5min":1.228,"15min":1.228},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81499999999999995,"15min":0.86899999999999999},"min":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97399999999999998,"5min":1.571,"15min":1.571},"last":0.82499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298136,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073704,"statfs":{"total":96636764160,"available":96331472896,"internally_reserved":0,"allocated":868352,"data_stored":386039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.83099999999999996,"15min":1},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.63200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.85499999999999998,"15min":1.0209999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.079,"5min":9.25,"15min":9.25},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.875,"15min":1.0620000000000001},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":10.384,"15min":10.384},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":1.0760000000000001},"min":{"1min":0.49299999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0349999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.621}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":1.1240000000000001},"min":{"1min":0.54800000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.95199999999999996,"5min":10.205,"15min":10.205},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96399999999999997,"5min":1.292,"15min":1.292},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.048,"5min":1.171,"15min":1.171},"last":0.49299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78400000000000003,"15min":0.81699999999999995},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.298,"15min":1.298},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80300000000000005,"15min":0.82899999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98999999999999999,"5min":1.204,"15min":1.204},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90200000000000002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:21:24.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:21:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.338+0000 7f03c6bff700 1 Processor -- start 2023-10-06T05:21:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.338+0000 7f03c6bff700 1 -- start start 2023-10-06T05:21:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.339+0000 7f03c6bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03c015ca40 0x7f03c0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.339+0000 7f03c6bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03c00c8f40 0x7f03c00adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.339+0000 7f03c6bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c0068da0 con 0x7f03c006a300 2023-10-06T05:21:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.339+0000 7f03c6bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c00cabc0 con 0x7f03c015ca40 2023-10-06T05:21:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.339+0000 7f03c6bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c0068bb0 con 0x7f03c00c8f40 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.339+0000 7f03c499b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03c015ca40 0x7f03c0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03c499b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03c015ca40 0x7f03c0068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60886/0 (socket says 172.21.15.31:60886) 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03c499b700 1 -- 172.21.15.31:0/744620056 learned_addr learned my addr 172.21.15.31:0/744620056 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03bffff700 1 --1- 172.21.15.31:0/744620056 >> v1:172.21.15.31:6789/0 conn(0x7f03c006a300 0x7f03c00c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35630/0 (socket says 172.21.15.31:35630) 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03c519c700 1 --2- 172.21.15.31:0/744620056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03c00c8f40 0x7f03c00adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03c499b700 1 -- 172.21.15.31:0/744620056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03c00c8f40 msgr2=0x7f03c00adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03c499b700 1 --2- 172.21.15.31:0/744620056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03c00c8f40 0x7f03c00adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03c499b700 1 -- 172.21.15.31:0/744620056 >> v1:172.21.15.31:6789/0 conn(0x7f03c006a300 legacy=0x7f03c00c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03c499b700 1 -- 172.21.15.31:0/744620056 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0069080 con 0x7f03c015ca40 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03bf7fe700 1 -- 172.21.15.31:0/744620056 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314327748 0 0) 0x7f03c0068da0 con 0x7f03c006a300 2023-10-06T05:21:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03c499b700 1 --2- 172.21.15.31:0/744620056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c015ca40 0x7f03c0068670 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f03b40099e0 tx=0x7f03b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc38b45f6ce75bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.340+0000 7f03bf7fe700 1 -- 172.21.15.31:0/744620056 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b400b820 con 0x7f03c015ca40 2023-10-06T05:21:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.341+0000 7f03bf7fe700 1 -- 172.21.15.31:0/744620056 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b400b980 con 0x7f03c015ca40 2023-10-06T05:21:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.341+0000 7f03bf7fe700 1 -- 172.21.15.31:0/744620056 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b400bd20 con 0x7f03c015ca40 2023-10-06T05:21:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.341+0000 7f03c6bff700 1 -- 172.21.15.31:0/744620056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c015ca40 msgr2=0x7f03c0068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.341+0000 7f03c6bff700 1 --2- 172.21.15.31:0/744620056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c015ca40 0x7f03c0068670 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f03b40099e0 tx=0x7f03b4009250 comp rx=0 tx=0).stop 2023-10-06T05:21:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.341+0000 7f03c6bff700 1 -- 172.21.15.31:0/744620056 shutdown_connections 2023-10-06T05:21:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.341+0000 7f03c6bff700 1 --2- 172.21.15.31:0/744620056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c015ca40 0x7f03c0068670 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.341+0000 7f03c6bff700 1 --2- 172.21.15.31:0/744620056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03c00c8f40 0x7f03c00adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.341+0000 7f03c6bff700 1 -- 172.21.15.31:0/744620056 >> 172.21.15.31:0/744620056 conn(0x7f03c00ce290 msgr2=0x7f03c00ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.341+0000 7f03c6bff700 1 -- 172.21.15.31:0/744620056 shutdown_connections 2023-10-06T05:21:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.341+0000 7f03c6bff700 1 -- 172.21.15.31:0/744620056 wait complete. 2023-10-06T05:21:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.343+0000 7f03c6bff700 1 Processor -- start 2023-10-06T05:21:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.343+0000 7f03c6bff700 1 -- start start 2023-10-06T05:21:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c6bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c006a300 0x7f03c0159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c6bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03c00c8f40 0x7f03c00c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c6bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c015ca40 0x7f03c00c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c6bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c015a150 con 0x7f03c006a300 2023-10-06T05:21:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c6bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00c1a00 con 0x7f03c015ca40 2023-10-06T05:21:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c6bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00c5d60 con 0x7f03c00c8f40 2023-10-06T05:21:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c006a300 0x7f03c0159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c499b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c006a300 0x7f03c0159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60894/0 (socket says 172.21.15.31:60894) 2023-10-06T05:21:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c499b700 1 -- 172.21.15.31:0/3802571777 learned_addr learned my addr 172.21.15.31:0/3802571777 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03bffff700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03c00c8f40 0x7f03c00c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c519c700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c015ca40 0x7f03c00c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c499b700 1 -- 172.21.15.31:0/3802571777 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03c00c8f40 msgr2=0x7f03c00c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c499b700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03c00c8f40 0x7f03c00c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c499b700 1 -- 172.21.15.31:0/3802571777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c015ca40 msgr2=0x7f03c00c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c499b700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c015ca40 0x7f03c00c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.344+0000 7f03c499b700 1 -- 172.21.15.31:0/3802571777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b40094f0 con 0x7f03c006a300 2023-10-06T05:21:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.345+0000 7f03c499b700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c006a300 0x7f03c0159c10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f03c006e040 tx=0x7f03b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.345+0000 7f03bcff9700 1 -- 172.21.15.31:0/3802571777 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4026450 con 0x7f03c006a300 2023-10-06T05:21:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.345+0000 7f03bcff9700 1 -- 172.21.15.31:0/3802571777 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b40265d0 con 0x7f03c006a300 2023-10-06T05:21:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.345+0000 7f03bcff9700 1 -- 172.21.15.31:0/3802571777 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4026990 con 0x7f03c006a300 2023-10-06T05:21:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.345+0000 7f03c6bff700 1 -- 172.21.15.31:0/3802571777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c00c6040 con 0x7f03c006a300 2023-10-06T05:21:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.345+0000 7f03c6bff700 1 -- 172.21.15.31:0/3802571777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c00c6600 con 0x7f03c006a300 2023-10-06T05:21:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.346+0000 7f03c6bff700 1 -- 172.21.15.31:0/3802571777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c004c380 con 0x7f03c006a300 2023-10-06T05:21:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.347+0000 7f03bcff9700 1 -- 172.21.15.31:0/3802571777 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f03b401a420 con 0x7f03c006a300 2023-10-06T05:21:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.348+0000 7f03bcff9700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f03a4067290 0x7f03a4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.348+0000 7f03bcff9700 1 -- 172.21.15.31:0/3802571777 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f03b4086aa0 con 0x7f03c006a300 2023-10-06T05:21:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.349+0000 7f03bffff700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f03a4067290 0x7f03a4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.349+0000 7f03bffff700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f03a4067290 0x7f03a4069740 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f03c00c4e30 tx=0x7f03ac007bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:24.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.353+0000 7f03bcff9700 1 -- 172.21.15.31:0/3802571777 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03b40537e0 con 0x7f03c006a300 2023-10-06T05:21:24.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.636+0000 7f03c6bff700 1 -- 172.21.15.31:0/3802571777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f03c00ca1a0 con 0x7f03c006a300 2023-10-06T05:21:24.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.641+0000 7f03bcff9700 1 -- 172.21.15.31:0/3802571777 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f03b40150e0 con 0x7f03c006a300 2023-10-06T05:21:24.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 -- 172.21.15.31:0/3802571777 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f03a4067290 msgr2=0x7f03a4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f03a4067290 0x7f03a4069740 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f03c00c4e30 tx=0x7f03ac007bf0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 -- 172.21.15.31:0/3802571777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c006a300 msgr2=0x7f03c0159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c006a300 0x7f03c0159c10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f03c006e040 tx=0x7f03b4017000 comp rx=0 tx=0).stop 2023-10-06T05:21:24.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 -- 172.21.15.31:0/3802571777 shutdown_connections 2023-10-06T05:21:24.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c015ca40 0x7f03c00c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f03a4067290 0x7f03a4069740 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c006a300 0x7f03c0159c10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 --2- 172.21.15.31:0/3802571777 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03c00c8f40 0x7f03c00c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 -- 172.21.15.31:0/3802571777 >> 172.21.15.31:0/3802571777 conn(0x7f03c00ce290 msgr2=0x7f03c015a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:24.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 -- 172.21.15.31:0/3802571777 shutdown_connections 2023-10-06T05:21:24.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.643+0000 7f03c6bff700 1 -- 172.21.15.31:0/3802571777 wait complete. 2023-10-06T05:21:24.646 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:21:24.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:24.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.855+0000 7f46ea1ff700 1 Processor -- start 2023-10-06T05:21:24.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.855+0000 7f46ea1ff700 1 -- start start 2023-10-06T05:21:24.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.857+0000 7f46ea1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f46e40caa10 0x7f46e40c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.857+0000 7f46ea1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e40c6760 con 0x7f46e40ca640 2023-10-06T05:21:24.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.857+0000 7f46ea1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e40ab0f0 con 0x7f46e40c8510 2023-10-06T05:21:24.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.857+0000 7f46ea1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e4159a10 con 0x7f46e40caa10 2023-10-06T05:21:24.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.857+0000 7f46dbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f46e40caa10 0x7f46e40c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.857+0000 7f46dbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f46e40caa10 0x7f46e40c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60908/0 (socket says 172.21.15.31:60908) 2023-10-06T05:21:24.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.857+0000 7f46dbfff700 1 -- 172.21.15.31:0/1389780330 learned_addr learned my addr 172.21.15.31:0/1389780330 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:24.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.857+0000 7f46e37fe700 1 --1- 172.21.15.31:0/1389780330 >> v1:172.21.15.31:6789/0 conn(0x7f46e40ca640 0x7f46e40c7340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35644/0 (socket says 172.21.15.31:35644) 2023-10-06T05:21:24.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.858+0000 7f46dbfff700 1 -- 172.21.15.31:0/1389780330 >> v1:172.21.15.62:6789/0 conn(0x7f46e40c8510 legacy=0x7f46e4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.858+0000 7f46dbfff700 1 -- 172.21.15.31:0/1389780330 >> v1:172.21.15.31:6789/0 conn(0x7f46e40ca640 legacy=0x7f46e40c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.858+0000 7f46dbfff700 1 -- 172.21.15.31:0/1389780330 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e40c8960 con 0x7f46e40caa10 2023-10-06T05:21:24.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.858+0000 7f46e2ffd700 1 -- 172.21.15.31:0/1389780330 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335781002 0 0) 0x7f46e40c6760 con 0x7f46e40ca640 2023-10-06T05:21:24.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.858+0000 7f46dbfff700 1 --2- 172.21.15.31:0/1389780330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 0x7f46e40c7fd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0099b0 tx=0x7f46cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7d6b06fe05e5f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:24.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.858+0000 7f46e2ffd700 1 -- 172.21.15.31:0/1389780330 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00b860 con 0x7f46e40caa10 2023-10-06T05:21:24.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.858+0000 7f46e2ffd700 1 -- 172.21.15.31:0/1389780330 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc00b9c0 con 0x7f46e40caa10 2023-10-06T05:21:24.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.858+0000 7f46e2ffd700 1 -- 172.21.15.31:0/1389780330 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00bd60 con 0x7f46e40caa10 2023-10-06T05:21:24.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.859+0000 7f46ea1ff700 1 -- 172.21.15.31:0/1389780330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 msgr2=0x7f46e40c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.859+0000 7f46ea1ff700 1 --2- 172.21.15.31:0/1389780330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 0x7f46e40c7fd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0099b0 tx=0x7f46cc009250 comp rx=0 tx=0).stop 2023-10-06T05:21:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.859+0000 7f46ea1ff700 1 -- 172.21.15.31:0/1389780330 shutdown_connections 2023-10-06T05:21:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.859+0000 7f46ea1ff700 1 --2- 172.21.15.31:0/1389780330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 0x7f46e40c7fd0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.859+0000 7f46ea1ff700 1 -- 172.21.15.31:0/1389780330 >> 172.21.15.31:0/1389780330 conn(0x7f46e40cc0b0 msgr2=0x7f46e40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.859+0000 7f46ea1ff700 1 -- 172.21.15.31:0/1389780330 shutdown_connections 2023-10-06T05:21:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.859+0000 7f46ea1ff700 1 -- 172.21.15.31:0/1389780330 wait complete. 2023-10-06T05:21:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.860+0000 7f46ea1ff700 1 Processor -- start 2023-10-06T05:21:24.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.861+0000 7f46ea1ff700 1 -- start start 2023-10-06T05:21:24.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.861+0000 7f46ea1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e40c8510 0x7f46e41578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.861+0000 7f46ea1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46e40ca640 0x7f46e415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.861+0000 7f46ea1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 0x7f46e40c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.861+0000 7f46ea1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4157e20 con 0x7f46e40caa10 2023-10-06T05:21:24.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.861+0000 7f46ea1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e415a2a0 con 0x7f46e40c8510 2023-10-06T05:21:24.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.861+0000 7f46ea1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e416a380 con 0x7f46e40ca640 2023-10-06T05:21:24.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 0x7f46e40c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 0x7f46e40c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60910/0 (socket says 172.21.15.31:60910) 2023-10-06T05:21:24.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46e3fff700 1 -- 172.21.15.31:0/3261510035 learned_addr learned my addr 172.21.15.31:0/3261510035 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:24.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46dbfff700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46e40ca640 0x7f46e415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46e37fe700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e40c8510 0x7f46e41578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46e3fff700 1 -- 172.21.15.31:0/3261510035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46e40ca640 msgr2=0x7f46e415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46e3fff700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46e40ca640 0x7f46e415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46e3fff700 1 -- 172.21.15.31:0/3261510035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e40c8510 msgr2=0x7f46e41578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46e3fff700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e40c8510 0x7f46e41578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46e3fff700 1 -- 172.21.15.31:0/3261510035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0094f0 con 0x7f46e40caa10 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.862+0000 7f46e3fff700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 0x7f46e40c54a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00c8e0 tx=0x7f46dc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.863+0000 7f46db7fe700 1 -- 172.21.15.31:0/3261510035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc012630 con 0x7f46e40caa10 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.863+0000 7f46db7fe700 1 -- 172.21.15.31:0/3261510035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc012790 con 0x7f46e40caa10 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.863+0000 7f46db7fe700 1 -- 172.21.15.31:0/3261510035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc012b50 con 0x7f46e40caa10 2023-10-06T05:21:24.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.863+0000 7f46ea1ff700 1 -- 172.21.15.31:0/3261510035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e416a660 con 0x7f46e40caa10 2023-10-06T05:21:24.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.863+0000 7f46ea1ff700 1 -- 172.21.15.31:0/3261510035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e4159f30 con 0x7f46e40caa10 2023-10-06T05:21:24.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.864+0000 7f46ea1ff700 1 -- 172.21.15.31:0/3261510035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e415a430 con 0x7f46e40caa10 2023-10-06T05:21:24.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.865+0000 7f46db7fe700 1 -- 172.21.15.31:0/3261510035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f46dc01a3a0 con 0x7f46e40caa10 2023-10-06T05:21:24.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.866+0000 7f46db7fe700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f46c4067290 0x7f46c4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.866+0000 7f46db7fe700 1 -- 172.21.15.31:0/3261510035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f46dc084e80 con 0x7f46e40caa10 2023-10-06T05:21:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.866+0000 7f46e37fe700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f46c4067290 0x7f46c4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.867+0000 7f46e37fe700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f46c4067290 0x7f46c4069740 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f46e40c2ad0 tx=0x7f46d4007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:24.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:24.870+0000 7f46db7fe700 1 -- 172.21.15.31:0/3261510035 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46dc0554d0 con 0x7f46e40caa10 2023-10-06T05:21:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.118+0000 7f46ea1ff700 1 -- 172.21.15.31:0/3261510035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46e404c380 con 0x7f46e40caa10 2023-10-06T05:21:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.118+0000 7f46db7fe700 1 -- 172.21.15.31:0/3261510035 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f46dc055060 con 0x7f46e40caa10 2023-10-06T05:21:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:25.120 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2151449662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2151449662},{"type":"v1","addr":"172.21.15.62:6834","nonce":2151449662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/810564890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":810564890},{"type":"v1","addr":"172.21.15.31:6836","nonce":810564890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/802676715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":802676715},{"type":"v1","addr":"172.21.15.31:6837","nonce":802676715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:13.661482+0000","modified":"2023-10-06T05:21:17.173141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6249":{"gid":6249,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/934773709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":934773709},{"type":"v1","addr":"172.21.15.62:6835","nonce":934773709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 -- 172.21.15.31:0/3261510035 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f46c4067290 msgr2=0x7f46c4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f46c4067290 0x7f46c4069740 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f46e40c2ad0 tx=0x7f46d4007d30 comp rx=0 tx=0).stop 2023-10-06T05:21:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 -- 172.21.15.31:0/3261510035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 msgr2=0x7f46e40c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 0x7f46e40c54a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00c8e0 tx=0x7f46dc001710 comp rx=0 tx=0).stop 2023-10-06T05:21:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 -- 172.21.15.31:0/3261510035 shutdown_connections 2023-10-06T05:21:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f46c4067290 0x7f46c4069740 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46e40caa10 0x7f46e40c54a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e40c8510 0x7f46e41578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 --2- 172.21.15.31:0/3261510035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46e40ca640 0x7f46e415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 -- 172.21.15.31:0/3261510035 >> 172.21.15.31:0/3261510035 conn(0x7f46e40cc0b0 msgr2=0x7f46e4158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:25.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 -- 172.21.15.31:0/3261510035 shutdown_connections 2023-10-06T05:21:25.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.121+0000 7f46ea1ff700 1 -- 172.21.15.31:0/3261510035 wait complete. 2023-10-06T05:21:25.124 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 44 2023-10-06T05:21:25.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:13.661482+0000', 'modified': '2023-10-06T05:21:17.173141+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6249}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6249': {'gid': 6249, 'name': 'd', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/934773709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 934773709}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 934773709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:21:25.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:21:25.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:21:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.745+0000 7f8517dff700 1 Processor -- start 2023-10-06T05:21:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.745+0000 7f8517dff700 1 -- start start 2023-10-06T05:21:25.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.746+0000 7f8517dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8510159580 0x7f85100cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.746+0000 7f8517dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 0x7f85100cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f8517dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f85100ccc80 0x7f85100cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f8517dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85100cf280 con 0x7f85100ccc80 2023-10-06T05:21:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f8517dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85100ce670 con 0x7f8510159580 2023-10-06T05:21:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f8517dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85100ce950 con 0x7f85100cb9f0 2023-10-06T05:21:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f851639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f85100ccc80 0x7f85100cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f851639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f85100ccc80 0x7f85100cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60924/0 (socket says 172.21.15.31:60924) 2023-10-06T05:21:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f851639c700 1 -- 172.21.15.31:0/2596587843 learned_addr learned my addr 172.21.15.31:0/2596587843 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f851539a700 1 --2- 172.21.15.31:0/2596587843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 0x7f85100cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f8515b9b700 1 --2- 172.21.15.31:0/2596587843 >> v2:172.21.15.62:3300/0 conn(0x7f8510159580 0x7f85100cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f851639c700 1 -- 172.21.15.31:0/2596587843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 msgr2=0x7f85100cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f851639c700 1 --2- 172.21.15.31:0/2596587843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 0x7f85100cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f851639c700 1 -- 172.21.15.31:0/2596587843 >> v2:172.21.15.62:3300/0 conn(0x7f8510159580 msgr2=0x7f85100cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f851639c700 1 --2- 172.21.15.31:0/2596587843 >> v2:172.21.15.62:3300/0 conn(0x7f8510159580 0x7f85100cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f851639c700 1 -- 172.21.15.31:0/2596587843 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85100cec30 con 0x7f85100ccc80 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f851539a700 1 --2- 172.21.15.31:0/2596587843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 0x7f85100cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.747+0000 7f8515b9b700 1 --2- 172.21.15.31:0/2596587843 >> v2:172.21.15.62:3300/0 conn(0x7f8510159580 0x7f85100cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.748+0000 7f851639c700 1 --2- 172.21.15.31:0/2596587843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 0x7f85100cd0c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f85100889e0 tx=0x7f8508009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426cfe1df98c1968 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.748+0000 7f8514b99700 1 -- 172.21.15.31:0/2596587843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850800e070 con 0x7f85100ccc80 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.748+0000 7f8514b99700 1 -- 172.21.15.31:0/2596587843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850800ba50 con 0x7f85100ccc80 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.748+0000 7f8514b99700 1 -- 172.21.15.31:0/2596587843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85080092e0 con 0x7f85100ccc80 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.749+0000 7f8517dff700 1 -- 172.21.15.31:0/2596587843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 msgr2=0x7f85100cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.749+0000 7f8517dff700 1 --2- 172.21.15.31:0/2596587843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 0x7f85100cd0c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f85100889e0 tx=0x7f8508009840 comp rx=0 tx=0).stop 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.749+0000 7f8517dff700 1 -- 172.21.15.31:0/2596587843 shutdown_connections 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.749+0000 7f8517dff700 1 --2- 172.21.15.31:0/2596587843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 0x7f85100cd0c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.749+0000 7f8517dff700 1 --2- 172.21.15.31:0/2596587843 >> v2:172.21.15.62:3300/0 conn(0x7f8510159580 0x7f85100cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.749+0000 7f8517dff700 1 --2- 172.21.15.31:0/2596587843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 0x7f85100cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.749+0000 7f8517dff700 1 -- 172.21.15.31:0/2596587843 >> 172.21.15.31:0/2596587843 conn(0x7f8510099710 msgr2=0x7f851006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:25.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.749+0000 7f8517dff700 1 -- 172.21.15.31:0/2596587843 shutdown_connections 2023-10-06T05:21:25.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.749+0000 7f8517dff700 1 -- 172.21.15.31:0/2596587843 wait complete. 2023-10-06T05:21:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.750+0000 7f8517dff700 1 Processor -- start 2023-10-06T05:21:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.750+0000 7f8517dff700 1 -- start start 2023-10-06T05:21:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.751+0000 7f8517dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 0x7f8510157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.751+0000 7f8517dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 0x7f85100c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.751+0000 7f8517dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8510159580 0x7f85100c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.751+0000 7f8517dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510157e70 con 0x7f85100ccc80 2023-10-06T05:21:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.751+0000 7f8517dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851009a180 con 0x7f8510159580 2023-10-06T05:21:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.751+0000 7f8517dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851009a460 con 0x7f85100cb9f0 2023-10-06T05:21:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.751+0000 7f851539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 0x7f85100c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f851539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 0x7f85100c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60926/0 (socket says 172.21.15.31:60926) 2023-10-06T05:21:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f851539a700 1 -- 172.21.15.31:0/1396846816 learned_addr learned my addr 172.21.15.31:0/1396846816 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f8515b9b700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 0x7f8510157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f851639c700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8510159580 0x7f85100c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f851539a700 1 -- 172.21.15.31:0/1396846816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 msgr2=0x7f8510157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f851539a700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 0x7f8510157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f851539a700 1 -- 172.21.15.31:0/1396846816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8510159580 msgr2=0x7f85100c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f851539a700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8510159580 0x7f85100c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f851539a700 1 -- 172.21.15.31:0/1396846816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8508009ae0 con 0x7f85100ccc80 2023-10-06T05:21:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f851639c700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8510159580 0x7f85100c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f851539a700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 0x7f85100c8490 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f850000e8d0 tx=0x7f8500008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f8505ffb700 1 -- 172.21.15.31:0/1396846816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850000ee70 con 0x7f85100ccc80 2023-10-06T05:21:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f8505ffb700 1 -- 172.21.15.31:0/1396846816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850000c5e0 con 0x7f85100ccc80 2023-10-06T05:21:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.752+0000 7f8517dff700 1 -- 172.21.15.31:0/1396846816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85100c8e00 con 0x7f85100ccc80 2023-10-06T05:21:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.753+0000 7f8505ffb700 1 -- 172.21.15.31:0/1396846816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850000c980 con 0x7f85100ccc80 2023-10-06T05:21:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.753+0000 7f8517dff700 1 -- 172.21.15.31:0/1396846816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85100c93c0 con 0x7f85100ccc80 2023-10-06T05:21:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.755+0000 7f8505ffb700 1 -- 172.21.15.31:0/1396846816 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8500013410 con 0x7f85100ccc80 2023-10-06T05:21:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.756+0000 7f8505ffb700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f84f4067290 0x7f84f4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.756+0000 7f8505ffb700 1 -- 172.21.15.31:0/1396846816 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8500084b30 con 0x7f85100ccc80 2023-10-06T05:21:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.756+0000 7f8515b9b700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f84f4067290 0x7f84f4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.756+0000 7f8515b9b700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f84f4067290 0x7f84f4069740 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f851015e4d0 tx=0x7f850c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.757+0000 7f85265f0b80 1 Processor -- start 2023-10-06T05:21:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.758+0000 7f85265f0b80 1 -- start start 2023-10-06T05:21:25.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.758+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed9268820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.758+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 0x55aed9270f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.758+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9271940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.758+0000 7f85265f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed913b310 con 0x55aed92714d0 2023-10-06T05:21:25.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.758+0000 7f85265f0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed9178130 con 0x55aed924df40 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.758+0000 7f85265f0b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aed9175af0 con 0x55aed9268450 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9271940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9271940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60928/0 (socket says 172.21.15.31:60928) 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851639c700 1 -- 172.21.15.31:0/1659585114 learned_addr learned my addr 172.21.15.31:0/1659585114 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f8515b9b700 1 --2- 172.21.15.31:0/1659585114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed9268820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851539a700 1 --2- 172.21.15.31:0/1659585114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 0x55aed9270f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851639c700 1 -- 172.21.15.31:0/1659585114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 msgr2=0x55aed9268820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851639c700 1 --2- 172.21.15.31:0/1659585114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed9268820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851639c700 1 -- 172.21.15.31:0/1659585114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 msgr2=0x55aed9270f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851639c700 1 --2- 172.21.15.31:0/1659585114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 0x55aed9270f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851639c700 1 -- 172.21.15.31:0/1659585114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aed8f77790 con 0x55aed92714d0 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851539a700 1 --2- 172.21.15.31:0/1659585114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 0x55aed9270f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.759+0000 7f851639c700 1 --2- 172.21.15.31:0/1659585114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9271940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f85100888e0 tx=0x7f8508018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d92f3511f395b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f84faffd700 1 -- 172.21.15.31:0/1659585114 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508009940 con 0x55aed92714d0 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f84faffd700 1 -- 172.21.15.31:0/1659585114 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8508007200 con 0x55aed92714d0 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f84faffd700 1 -- 172.21.15.31:0/1659585114 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850800bdd0 con 0x55aed92714d0 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f85265f0b80 1 -- 172.21.15.31:0/1659585114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 msgr2=0x55aed9271940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f85265f0b80 1 --2- 172.21.15.31:0/1659585114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9271940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f85100888e0 tx=0x7f8508018000 comp rx=0 tx=0).stop 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f85265f0b80 1 -- 172.21.15.31:0/1659585114 shutdown_connections 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f85265f0b80 1 --2- 172.21.15.31:0/1659585114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9271940 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f85265f0b80 1 --2- 172.21.15.31:0/1659585114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed9268820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f85265f0b80 1 --2- 172.21.15.31:0/1659585114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 0x55aed9270f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f85265f0b80 1 -- 172.21.15.31:0/1659585114 >> 172.21.15.31:0/1659585114 conn(0x55aed926b880 msgr2=0x55aed926bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.760+0000 7f85265f0b80 1 -- 172.21.15.31:0/1659585114 shutdown_connections 2023-10-06T05:21:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.761+0000 7f85265f0b80 1 -- 172.21.15.31:0/1659585114 wait complete. 2023-10-06T05:21:25.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.761+0000 7f85265f0b80 1 -- start start 2023-10-06T05:21:25.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.762+0000 7f85265f0b80 1 Processor -- start 2023-10-06T05:21:25.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.763+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed924df40 0x55aed9241810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.763+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed9243600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.763+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed92714d0 0x55aed92428e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.763+0000 7f85265f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed91718e0 con 0x55aed924df40 2023-10-06T05:21:25.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.763+0000 7f85265f0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed911d980 con 0x55aed92714d0 2023-10-06T05:21:25.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.763+0000 7f85265f0b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aed911d620 con 0x55aed9268450 2023-10-06T05:21:25.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.763+0000 7f85265f0b80 10 client.0 _ll_register_callbacks cb 0x55aed8f80a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f8515b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed924df40 0x55aed9241810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f8515b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed924df40 0x55aed9241810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60944/0 (socket says 172.21.15.31:60944) 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f8515b9b700 1 -- 172.21.15.31:0/468146559 learned_addr learned my addr 172.21.15.31:0/468146559 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f851539a700 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed9243600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f851639c700 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed92714d0 0x55aed92428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f8515b9b700 1 -- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 msgr2=0x55aed9243600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f8515b9b700 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed9243600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f8515b9b700 1 -- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed92714d0 msgr2=0x55aed92428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f8515b9b700 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed92714d0 0x55aed92428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f8515b9b700 1 -- 172.21.15.31:0/468146559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85100cec30 con 0x55aed924df40 2023-10-06T05:21:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f851639c700 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed92714d0 0x55aed92428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:25.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f8515b9b700 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed924df40 0x55aed9241810 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f850c014870 tx=0x7f850c018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53593b13c107abe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.764+0000 7f84effff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:21:25.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f84effff700 1 -- 172.21.15.31:0/468146559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c021630 con 0x55aed924df40 2023-10-06T05:21:25.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f84effff700 1 -- 172.21.15.31:0/468146559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f850c021790 con 0x55aed924df40 2023-10-06T05:21:25.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f85265f0b80 1 -- 172.21.15.31:0/468146559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aed9215d60 con 0x55aed924df40 2023-10-06T05:21:25.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f84effff700 1 -- 172.21.15.31:0/468146559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f850c021b60 con 0x55aed924df40 2023-10-06T05:21:25.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f85265f0b80 10 client.6426 fetch_fsmap learned FSMap version 44 2023-10-06T05:21:25.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f85265f0b80 1 -- 172.21.15.31:0/468146559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55aed90640a0 con 0x55aed924df40 2023-10-06T05:21:25.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f84effff700 1 -- 172.21.15.31:0/468146559 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aed9215d60 con 0x55aed924df40 2023-10-06T05:21:25.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f84effff700 1 -- 172.21.15.31:0/468146559 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 44) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f850c021cc0 con 0x55aed924df40 2023-10-06T05:21:25.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f85265f0b80 10 client.6426 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T05:21:25.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f85265f0b80 10 client.6426 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:21:25.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f85265f0b80 20 client.6426 populate_metadata read hostname 'smithi031' 2023-10-06T05:21:25.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f85265f0b80 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:6833/934773709,v1:172.21.15.62:6835/934773709] conn(0x55aed9247930 0x55aed9247d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f85265f0b80 4 client.6426 mds_command: new command op to 6249 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:21:25.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.765+0000 7f85265f0b80 1 -- 172.21.15.31:0/468146559 --> [v2:172.21.15.62:6833/934773709,v1:172.21.15.62:6835/934773709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aed91f2b30 con 0x55aed9247930 2023-10-06T05:21:25.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.766+0000 7f851539a700 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:6833/934773709,v1:172.21.15.62:6835/934773709] conn(0x55aed9247930 0x55aed9247d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.766+0000 7f851539a700 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:6833/934773709,v1:172.21.15.62:6835/934773709] conn(0x55aed9247930 0x55aed9247d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ffccd46bc23bc2 server_cookie=5233bcecbca33e6d in_seq=0 out_seq=0 2023-10-06T05:21:25.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.766+0000 7f84effff700 10 client.6426 ms_handle_connect on v2:172.21.15.62:6833/934773709 2023-10-06T05:21:25.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.767+0000 7f84effff700 1 -- 172.21.15.31:0/468146559 <== mds.0 v2:172.21.15.62:6833/934773709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55aed91f2b30 con 0x55aed9247930 2023-10-06T05:21:25.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.767+0000 7f84effff700 10 client.6426 handle_command_reply: tid=0 2023-10-06T05:21:25.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.767+0000 7f85265f0b80 1 client.6426 shutdown 2023-10-06T05:21:25.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 1 -- 172.21.15.31:0/468146559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed924df40 msgr2=0x55aed9241810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed924df40 0x55aed9241810 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f850c014870 tx=0x7f850c018040 comp rx=0 tx=0).stop 2023-10-06T05:21:25.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 1 -- 172.21.15.31:0/468146559 shutdown_connections 2023-10-06T05:21:25.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:6833/934773709,v1:172.21.15.62:6835/934773709] conn(0x55aed9247930 0x55aed9247d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed92714d0 0x55aed92428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed924df40 0x55aed9241810 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 1 --2- 172.21.15.31:0/468146559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed9243600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f84effff700 0 client.6426 ms_handle_reset on v2:172.21.15.62:6833/934773709 2023-10-06T05:21:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 1 -- 172.21.15.31:0/468146559 >> 172.21.15.31:0/468146559 conn(0x55aed926b880 msgr2=0x55aed921dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 1 -- 172.21.15.31:0/468146559 shutdown_connections 2023-10-06T05:21:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 1 -- 172.21.15.31:0/468146559 wait complete. 2023-10-06T05:21:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.768+0000 7f85265f0b80 20 client.6426 trim_cache size 0 max 16384 2023-10-06T05:21:25.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.786+0000 7f85265f0b80 1 Processor -- start 2023-10-06T05:21:25.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.786+0000 7f85265f0b80 1 -- start start 2023-10-06T05:21:25.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed924df40 0x55aed9246180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed9268450 0x55aed92466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9249170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f85265f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed90091e0 con 0x55aed92714d0 2023-10-06T05:21:25.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f85265f0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed92517f0 con 0x55aed9268450 2023-10-06T05:21:25.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f85265f0b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aed92198d0 con 0x55aed924df40 2023-10-06T05:21:25.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f851639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9249170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f851639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9249170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60946/0 (socket says 172.21.15.31:60946) 2023-10-06T05:21:25.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f851639c700 1 -- 172.21.15.31:0/1971432056 learned_addr learned my addr 172.21.15.31:0/1971432056 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:25.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f8515b9b700 1 --2- 172.21.15.31:0/1971432056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed924df40 0x55aed9246180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.787+0000 7f851539a700 1 --2- 172.21.15.31:0/1971432056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed9268450 0x55aed92466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.788+0000 7f851639c700 1 -- 172.21.15.31:0/1971432056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed924df40 msgr2=0x55aed9246180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.788+0000 7f851639c700 1 --2- 172.21.15.31:0/1971432056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed924df40 0x55aed9246180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.788+0000 7f851639c700 1 -- 172.21.15.31:0/1971432056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed9268450 msgr2=0x55aed92466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.788+0000 7f851639c700 1 --2- 172.21.15.31:0/1971432056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed9268450 0x55aed92466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.788+0000 7f851639c700 1 -- 172.21.15.31:0/1971432056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aed9243fc0 con 0x55aed92714d0 2023-10-06T05:21:25.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.788+0000 7f8515b9b700 1 --2- 172.21.15.31:0/1971432056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed924df40 0x55aed9246180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:25.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.788+0000 7f851639c700 1 --2- 172.21.15.31:0/1971432056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9249170 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8508012000 tx=0x7f850801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19699e4264842271 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.788+0000 7f84fb7fe700 1 -- 172.21.15.31:0/1971432056 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508007370 con 0x55aed92714d0 2023-10-06T05:21:25.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.788+0000 7f84fb7fe700 1 -- 172.21.15.31:0/1971432056 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850802d040 con 0x55aed92714d0 2023-10-06T05:21:25.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.788+0000 7f84fb7fe700 1 -- 172.21.15.31:0/1971432056 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850800be10 con 0x55aed92714d0 2023-10-06T05:21:25.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.789+0000 7f85265f0b80 1 -- 172.21.15.31:0/1971432056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 msgr2=0x55aed9249170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.789+0000 7f85265f0b80 1 --2- 172.21.15.31:0/1971432056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9249170 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8508012000 tx=0x7f850801f000 comp rx=0 tx=0).stop 2023-10-06T05:21:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.789+0000 7f85265f0b80 1 -- 172.21.15.31:0/1971432056 shutdown_connections 2023-10-06T05:21:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.789+0000 7f85265f0b80 1 --2- 172.21.15.31:0/1971432056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9249170 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.789+0000 7f85265f0b80 1 --2- 172.21.15.31:0/1971432056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed924df40 0x55aed9246180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.789+0000 7f85265f0b80 1 --2- 172.21.15.31:0/1971432056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed9268450 0x55aed92466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.789+0000 7f85265f0b80 1 -- 172.21.15.31:0/1971432056 >> 172.21.15.31:0/1971432056 conn(0x55aed926b880 msgr2=0x55aed92230d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.789+0000 7f85265f0b80 1 -- 172.21.15.31:0/1971432056 shutdown_connections 2023-10-06T05:21:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.789+0000 7f85265f0b80 1 -- 172.21.15.31:0/1971432056 wait complete. 2023-10-06T05:21:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.790+0000 7f85265f0b80 1 -- start start 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.790+0000 7f85265f0b80 1 Processor -- start 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 0x55aed92c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed92c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f85265f0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9246180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f85265f0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed9223e60 con 0x55aed92714d0 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f85265f0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed926bd80 con 0x55aed924df40 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f85265f0b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aed923e4a0 con 0x55aed9268450 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f85265f0b80 10 client.0 _ll_register_callbacks cb 0x55aed8f80a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f851639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9246180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f851639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9246180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60952/0 (socket says 172.21.15.31:60952) 2023-10-06T05:21:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f851639c700 1 -- 172.21.15.31:0/2895422457 learned_addr learned my addr 172.21.15.31:0/2895422457 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f851539a700 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed92c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f8515b9b700 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 0x55aed92c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f851639c700 1 -- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 msgr2=0x55aed92c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f851639c700 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed92c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f851639c700 1 -- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 msgr2=0x55aed92c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f851639c700 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 0x55aed92c3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f851639c700 1 -- 172.21.15.31:0/2895422457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aed8f77790 con 0x55aed92714d0 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.791+0000 7f851539a700 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed92c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f8515b9b700 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 0x55aed92c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f851639c700 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9246180 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8508009450 tx=0x7f850802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6b2958f922db7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f84effff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:21:25.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f84effff700 1 -- 172.21.15.31:0/2895422457 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508008cd0 con 0x55aed92714d0 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f84effff700 1 -- 172.21.15.31:0/2895422457 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8508008e30 con 0x55aed92714d0 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f85265f0b80 1 -- 172.21.15.31:0/2895422457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8508007200 con 0x55aed92714d0 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f84effff700 1 -- 172.21.15.31:0/2895422457 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f850803de40 con 0x55aed92714d0 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f85265f0b80 10 client.6432 fetch_fsmap learned FSMap version 44 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f85265f0b80 1 -- 172.21.15.31:0/2895422457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55aed91403c0 con 0x55aed92714d0 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f84effff700 1 -- 172.21.15.31:0/2895422457 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8508007200 con 0x55aed92714d0 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.792+0000 7f84effff700 1 -- 172.21.15.31:0/2895422457 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 44) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f850803a030 con 0x55aed92714d0 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.793+0000 7f85265f0b80 10 client.6432 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.793+0000 7f85265f0b80 10 client.6432 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.793+0000 7f85265f0b80 20 client.6432 populate_metadata read hostname 'smithi031' 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.793+0000 7f85265f0b80 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:6833/934773709,v1:172.21.15.62:6835/934773709] conn(0x55aed92d0350 0x55aed92d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:25.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.793+0000 7f85265f0b80 4 client.6432 mds_command: new command op to 6249 tid=0[{"prefix": "status"}] 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.793+0000 7f85265f0b80 1 -- 172.21.15.31:0/2895422457 --> [v2:172.21.15.62:6833/934773709,v1:172.21.15.62:6835/934773709] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aed91148d0 con 0x55aed92d0350 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.793+0000 7f8515b9b700 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:6833/934773709,v1:172.21.15.62:6835/934773709] conn(0x55aed92d0350 0x55aed92d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.793+0000 7f84effff700 10 client.6432 ms_handle_connect on v2:172.21.15.62:6833/934773709 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.793+0000 7f8515b9b700 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:6833/934773709,v1:172.21.15.62:6835/934773709] conn(0x55aed92d0350 0x55aed92d0720 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302a8235acebb8ab server_cookie=1dc17d429d07de8b in_seq=0 out_seq=0 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f84effff700 1 -- 172.21.15.31:0/2895422457 <== mds.0 v2:172.21.15.62:6833/934773709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55aed91148d0 con 0x55aed92d0350 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f84effff700 10 client.6432 handle_command_reply: tid=0 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 client.6432 shutdown 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 -- 172.21.15.31:0/2895422457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 msgr2=0x55aed9246180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9246180 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8508009450 tx=0x7f850802e000 comp rx=0 tx=0).stop 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 -- 172.21.15.31:0/2895422457 shutdown_connections 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:6833/934773709,v1:172.21.15.62:6835/934773709] conn(0x55aed92d0350 0x55aed92d0720 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aed92714d0 0x55aed9246180 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aed924df40 0x55aed92c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 --2- 172.21.15.31:0/2895422457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55aed9268450 0x55aed92c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f84effff700 0 client.6432 ms_handle_reset on v2:172.21.15.62:6833/934773709 2023-10-06T05:21:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 -- 172.21.15.31:0/2895422457 >> 172.21.15.31:0/2895422457 conn(0x55aed926b880 msgr2=0x55aed92453d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 -- 172.21.15.31:0/2895422457 shutdown_connections 2023-10-06T05:21:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 1 -- 172.21.15.31:0/2895422457 wait complete. 2023-10-06T05:21:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.794+0000 7f85265f0b80 20 client.6432 trim_cache size 0 max 16384 2023-10-06T05:21:25.809 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:21:25.809 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:21:25.809 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:21:25.809 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 6249, 2023-10-06T05:21:25.809 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:21:25.809 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:21:25.810 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:21:25.810 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 11.632612773, 2023-10-06T05:21:25.810 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 44, 2023-10-06T05:21:25.810 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 71, 2023-10-06T05:21:25.810 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 71, 2023-10-06T05:21:25.810 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 18.622577578000001 2023-10-06T05:21:25.810 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:21:25.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 -- 172.21.15.31:0/1396846816 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f84f4067290 msgr2=0x7f84f4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f84f4067290 0x7f84f4069740 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f851015e4d0 tx=0x7f850c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:25.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 -- 172.21.15.31:0/1396846816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 msgr2=0x7f85100c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:25.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 0x7f85100c8490 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f850000e8d0 tx=0x7f8500008040 comp rx=0 tx=0).stop 2023-10-06T05:21:25.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 -- 172.21.15.31:0/1396846816 shutdown_connections 2023-10-06T05:21:25.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8510159580 0x7f85100c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85100cb9f0 0x7f8510157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f84f4067290 0x7f84f4069740 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 --2- 172.21.15.31:0/1396846816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85100ccc80 0x7f85100c8490 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:25.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 -- 172.21.15.31:0/1396846816 >> 172.21.15.31:0/1396846816 conn(0x7f8510099710 msgr2=0x7f85100cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:25.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 -- 172.21.15.31:0/1396846816 shutdown_connections 2023-10-06T05:21:25.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:25.795+0000 7f8517dff700 1 -- 172.21.15.31:0/1396846816 wait complete. 2023-10-06T05:21:25.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:21:26.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.007+0000 7f10a7bff700 1 Processor -- start 2023-10-06T05:21:26.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.007+0000 7f10a7bff700 1 -- start start 2023-10-06T05:21:26.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.009+0000 7f10a7bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a013fda0 0x7f10a0069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.009+0000 7f10a7bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a00ca5b0 con 0x7f10a00c7b60 2023-10-06T05:21:26.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.009+0000 7f10a7bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a006f740 con 0x7f10a00c7f30 2023-10-06T05:21:26.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.009+0000 7f10a7bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00c9c80 con 0x7f10a013fda0 2023-10-06T05:21:26.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.009+0000 7f10a599b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f10a00c7b60 0x7f10a006ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35658/0 (socket says 172.21.15.31:35658) 2023-10-06T05:21:26.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.009+0000 7f10a599b700 1 -- 172.21.15.31:0/171720724 learned_addr learned my addr 172.21.15.31:0/171720724 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.009+0000 7f10a619c700 1 --2- 172.21.15.31:0/171720724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a013fda0 0x7f10a0069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.010+0000 7f10a4999700 1 -- 172.21.15.31:0/171720724 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019909123 0 0) 0x7f10a00ca5b0 con 0x7f10a00c7b60 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.010+0000 7f10a4999700 1 -- 172.21.15.31:0/171720724 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f108c003570 con 0x7f10a00c7b60 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.010+0000 7f10a4999700 1 -- 172.21.15.31:0/171720724 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398098972 0 0) 0x7f10a006f740 con 0x7f10a00c7f30 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.010+0000 7f10a4999700 1 -- 172.21.15.31:0/171720724 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a00ca5b0 con 0x7f10a00c7f30 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.010+0000 7f10a619c700 1 -- 172.21.15.31:0/171720724 >> v1:172.21.15.62:6789/0 conn(0x7f10a00c7f30 legacy=0x7f10a013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.010+0000 7f10a619c700 1 -- 172.21.15.31:0/171720724 >> v1:172.21.15.31:6789/0 conn(0x7f10a00c7b60 legacy=0x7f10a006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.010+0000 7f10a4999700 1 -- 172.21.15.31:0/171720724 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2051082942 0 0) 0x7f108c003570 con 0x7f10a00c7b60 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.010+0000 7f10a619c700 1 -- 172.21.15.31:0/171720724 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00c9f60 con 0x7f10a013fda0 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.010+0000 7f10a619c700 1 --2- 172.21.15.31:0/171720724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a013fda0 0x7f10a0069880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1098009e90 tx=0x7f1098009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=397c75364df58385 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.010+0000 7f10a4999700 1 -- 172.21.15.31:0/171720724 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109800e070 con 0x7f10a013fda0 2023-10-06T05:21:26.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.011+0000 7f10a4999700 1 -- 172.21.15.31:0/171720724 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109800b870 con 0x7f10a013fda0 2023-10-06T05:21:26.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.011+0000 7f10a4999700 1 -- 172.21.15.31:0/171720724 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c0015f0 con 0x7f10a013fda0 2023-10-06T05:21:26.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.011+0000 7f10a7bff700 1 -- 172.21.15.31:0/171720724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a013fda0 msgr2=0x7f10a0069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.011+0000 7f10a7bff700 1 --2- 172.21.15.31:0/171720724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a013fda0 0x7f10a0069880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1098009e90 tx=0x7f1098009780 comp rx=0 tx=0).stop 2023-10-06T05:21:26.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.011+0000 7f10a7bff700 1 -- 172.21.15.31:0/171720724 shutdown_connections 2023-10-06T05:21:26.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.011+0000 7f10a7bff700 1 --2- 172.21.15.31:0/171720724 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a013fda0 0x7f10a0069880 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.011+0000 7f10a7bff700 1 -- 172.21.15.31:0/171720724 >> 172.21.15.31:0/171720724 conn(0x7f10a00cc0b0 msgr2=0x7f10a00cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:26.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.011+0000 7f10a7bff700 1 -- 172.21.15.31:0/171720724 shutdown_connections 2023-10-06T05:21:26.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.011+0000 7f10a7bff700 1 -- 172.21.15.31:0/171720724 wait complete. 2023-10-06T05:21:26.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.013+0000 7f10a7bff700 1 Processor -- start 2023-10-06T05:21:26.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.013+0000 7f10a7bff700 1 -- start start 2023-10-06T05:21:26.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a7bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a00c7b60 0x7f10a00c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a7bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a00c7f30 0x7f10a00c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a7bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a013fda0 0x7f10a00c1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a7bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00c2e70 con 0x7f10a00c7f30 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a7bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00ae880 con 0x7f10a013fda0 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a7bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00a9a60 con 0x7f10a00c7b60 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a00c7f30 0x7f10a00c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a00c7f30 0x7f10a00c0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60966/0 (socket says 172.21.15.31:60966) 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a519a700 1 -- 172.21.15.31:0/3166193491 learned_addr learned my addr 172.21.15.31:0/3166193491 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a599b700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a00c7b60 0x7f10a00c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a619c700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a013fda0 0x7f10a00c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a519a700 1 -- 172.21.15.31:0/3166193491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a00c7b60 msgr2=0x7f10a00c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a519a700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a00c7b60 0x7f10a00c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a519a700 1 -- 172.21.15.31:0/3166193491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a013fda0 msgr2=0x7f10a00c1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.014+0000 7f10a519a700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a013fda0 0x7f10a00c1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.015+0000 7f10a519a700 1 -- 172.21.15.31:0/3166193491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1098009a20 con 0x7f10a00c7f30 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.015+0000 7f10a519a700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a00c7f30 0x7f10a00c0e50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f109000c7c0 tx=0x7f1090007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.015+0000 7f1095ffb700 1 -- 172.21.15.31:0/3166193491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10900015f0 con 0x7f10a00c7f30 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.015+0000 7f1095ffb700 1 -- 172.21.15.31:0/3166193491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1090001790 con 0x7f10a00c7f30 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.015+0000 7f1095ffb700 1 -- 172.21.15.31:0/3166193491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090001b50 con 0x7f10a00c7f30 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.015+0000 7f10a7bff700 1 -- 172.21.15.31:0/3166193491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a00ab9a0 con 0x7f10a00c7f30 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.015+0000 7f10a7bff700 1 -- 172.21.15.31:0/3166193491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a00ae2c0 con 0x7f10a00c7f30 2023-10-06T05:21:26.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.016+0000 7f10a7bff700 1 -- 172.21.15.31:0/3166193491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a00a9bd0 con 0x7f10a00c7f30 2023-10-06T05:21:26.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.017+0000 7f1095ffb700 1 -- 172.21.15.31:0/3166193491 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f109001b400 con 0x7f10a00c7f30 2023-10-06T05:21:26.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.018+0000 7f1095ffb700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1084067290 0x7f1084069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:26.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.018+0000 7f1095ffb700 1 -- 172.21.15.31:0/3166193491 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1090084b10 con 0x7f10a00c7f30 2023-10-06T05:21:26.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.018+0000 7f10a599b700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1084067290 0x7f1084069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:26.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.019+0000 7f10a599b700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1084067290 0x7f1084069740 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f109c000c00 tx=0x7f109c0016c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:26.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.022+0000 7f1095ffb700 1 -- 172.21.15.31:0/3166193491 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10900516d0 con 0x7f10a00c7f30 2023-10-06T05:21:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.273+0000 7f10a7bff700 1 -- 172.21.15.31:0/3166193491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f10a004c380 con 0x7f10a00c7f30 2023-10-06T05:21:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.273+0000 7f1095ffb700 1 -- 172.21.15.31:0/3166193491 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1090054cf0 con 0x7f10a00c7f30 2023-10-06T05:21:26.275 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:21:26.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.275+0000 7f10a7bff700 1 -- 172.21.15.31:0/3166193491 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1084067290 msgr2=0x7f1084069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.275+0000 7f10a7bff700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1084067290 0x7f1084069740 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f109c000c00 tx=0x7f109c0016c0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.275+0000 7f10a7bff700 1 -- 172.21.15.31:0/3166193491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a00c7f30 msgr2=0x7f10a00c0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.275+0000 7f10a7bff700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a00c7f30 0x7f10a00c0e50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f109000c7c0 tx=0x7f1090007dd0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.276+0000 7f10a7bff700 1 -- 172.21.15.31:0/3166193491 shutdown_connections 2023-10-06T05:21:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.276+0000 7f10a7bff700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a013fda0 0x7f10a00c1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.276+0000 7f10a7bff700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10a00c7b60 0x7f10a00c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.276+0000 7f10a7bff700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f1084067290 0x7f1084069740 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.276+0000 7f10a7bff700 1 --2- 172.21.15.31:0/3166193491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a00c7f30 0x7f10a00c0e50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.276+0000 7f10a7bff700 1 -- 172.21.15.31:0/3166193491 >> 172.21.15.31:0/3166193491 conn(0x7f10a00cc0b0 msgr2=0x7f10a00cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.276+0000 7f10a7bff700 1 -- 172.21.15.31:0/3166193491 shutdown_connections 2023-10-06T05:21:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:26.276+0000 7f10a7bff700 1 -- 172.21.15.31:0/3166193491 wait complete. 2023-10-06T05:21:26.295 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:21:26.295 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:21:26.295 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:21:26.295 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:21:26.295 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:26.295 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:21:26.295 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:21:26.295 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:21:26.295 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:26.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:26.296 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:21:26.343 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:21:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:21:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:21:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:21:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:26.344 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:21:26.344 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:21:26.344 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:21:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:21:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:21:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42028sec preferred_lft 42028sec 2023-10-06T05:21:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:21:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:26.345 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:21:26.346 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:21:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:21:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:21:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:26.346 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:21:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:21:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:26.347 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:21:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:21:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:26.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:26.347 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:21:26.415 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:21:26.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:26.418 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:21:26.467 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:21:26.467 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:21:26.467 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:21:26.468 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:21:26.468 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:21:26.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:26.468 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:21:26.468 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:21:26.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:21:26.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:21:26.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:21:26.468 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:21:26.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:26.641 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:21:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:21:26.688 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:21:26.688 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:26.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:26.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:21:26.759 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:26.759 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:26.805 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:21:26.806 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:26.806 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:26.891 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:21:26.891 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:21:26.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:21:27.032 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:21:27.032 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:21:27.059 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:21:27.059 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:21:27.059 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:21:27.059 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:21:27.059 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:21:27.063 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:21:27.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:27.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:27.136 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:21:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.330+0000 7f2d43dff700 1 Processor -- start 2023-10-06T05:21:27.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.330+0000 7f2d43dff700 1 -- start start 2023-10-06T05:21:27.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.331+0000 7f2d43dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 0x7f2d3c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:27.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.331+0000 7f2d43dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3c0c95b0 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.331+0000 7f2d43dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3c0c9890 con 0x7f2d3c1595a0 2023-10-06T05:21:27.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.331+0000 7f2d43dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0c9b70 con 0x7f2d3c068d80 2023-10-06T05:21:27.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d4239c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2d3c0cc0d0 0x7f2d3c0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47552/0 (socket says 172.21.15.31:47552) 2023-10-06T05:21:27.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d4239c700 1 -- 172.21.15.31:0/1555033281 learned_addr learned my addr 172.21.15.31:0/1555033281 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:27.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d4139a700 1 --2- 172.21.15.31:0/1555033281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 0x7f2d3c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d40b99700 1 -- 172.21.15.31:0/1555033281 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311398179 0 0) 0x7f2d3c0c95b0 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d40b99700 1 -- 172.21.15.31:0/1555033281 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d28003570 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d40b99700 1 -- 172.21.15.31:0/1555033281 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131793205 0 0) 0x7f2d3c0c9890 con 0x7f2d3c1595a0 2023-10-06T05:21:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d40b99700 1 -- 172.21.15.31:0/1555033281 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d3c0c95b0 con 0x7f2d3c1595a0 2023-10-06T05:21:27.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d4139a700 1 -- 172.21.15.31:0/1555033281 >> v1:172.21.15.62:6789/0 conn(0x7f2d3c1595a0 legacy=0x7f2d3c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d4139a700 1 -- 172.21.15.31:0/1555033281 >> v1:172.21.15.31:6789/0 conn(0x7f2d3c0cc0d0 legacy=0x7f2d3c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d40b99700 1 -- 172.21.15.31:0/1555033281 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 179706306 0 0) 0x7f2d28003570 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.332+0000 7f2d4139a700 1 -- 172.21.15.31:0/1555033281 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c0ca130 con 0x7f2d3c068d80 2023-10-06T05:21:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.333+0000 7f2d4139a700 1 --2- 172.21.15.31:0/1555033281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 0x7f2d3c159970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009910 tx=0x7f2d2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9ab89b993131b79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:27.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.333+0000 7f2d40b99700 1 -- 172.21.15.31:0/1555033281 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d340017f0 con 0x7f2d3c068d80 2023-10-06T05:21:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.333+0000 7f2d40b99700 1 -- 172.21.15.31:0/1555033281 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c011420 con 0x7f2d3c068d80 2023-10-06T05:21:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.333+0000 7f2d40b99700 1 -- 172.21.15.31:0/1555033281 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0117c0 con 0x7f2d3c068d80 2023-10-06T05:21:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.334+0000 7f2d43dff700 1 -- 172.21.15.31:0/1555033281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 msgr2=0x7f2d3c159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.334+0000 7f2d43dff700 1 --2- 172.21.15.31:0/1555033281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 0x7f2d3c159970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009910 tx=0x7f2d2c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.334+0000 7f2d43dff700 1 -- 172.21.15.31:0/1555033281 shutdown_connections 2023-10-06T05:21:27.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.334+0000 7f2d43dff700 1 --2- 172.21.15.31:0/1555033281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 0x7f2d3c159970 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.334+0000 7f2d43dff700 1 -- 172.21.15.31:0/1555033281 >> 172.21.15.31:0/1555033281 conn(0x7f2d3c06b620 msgr2=0x7f2d3c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.334+0000 7f2d43dff700 1 -- 172.21.15.31:0/1555033281 shutdown_connections 2023-10-06T05:21:27.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.334+0000 7f2d43dff700 1 -- 172.21.15.31:0/1555033281 wait complete. 2023-10-06T05:21:27.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.335+0000 7f2d43dff700 1 Processor -- start 2023-10-06T05:21:27.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.335+0000 7f2d43dff700 1 -- start start 2023-10-06T05:21:27.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.336+0000 7f2d43dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 0x7f2d3c155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:27.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.336+0000 7f2d43dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d3c0cc0d0 0x7f2d3c0c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:27.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.336+0000 7f2d43dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c1595a0 0x7f2d3c142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.336+0000 7f2d43dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c156030 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.336+0000 7f2d43dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c153ae0 con 0x7f2d3c1595a0 2023-10-06T05:21:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.336+0000 7f2d43dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c155660 con 0x7f2d3c068d80 2023-10-06T05:21:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d3c0cc0d0 0x7f2d3c0c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d3c0cc0d0 0x7f2d3c0c7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47344/0 (socket says 172.21.15.31:47344) 2023-10-06T05:21:27.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4139a700 1 -- 172.21.15.31:0/2200671209 learned_addr learned my addr 172.21.15.31:0/2200671209 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4239c700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c1595a0 0x7f2d3c142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d41b9b700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 0x7f2d3c155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4139a700 1 -- 172.21.15.31:0/2200671209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 msgr2=0x7f2d3c155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4139a700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 0x7f2d3c155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4139a700 1 -- 172.21.15.31:0/2200671209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c1595a0 msgr2=0x7f2d3c142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4139a700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c1595a0 0x7f2d3c142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4139a700 1 -- 172.21.15.31:0/2200671209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c009420 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4239c700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c1595a0 0x7f2d3c142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.337+0000 7f2d4139a700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d3c0cc0d0 0x7f2d3c0c7370 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0030c0 tx=0x7f2d2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.338+0000 7f2d31ffb700 1 -- 172.21.15.31:0/2200671209 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c01cc50 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.338+0000 7f2d31ffb700 1 -- 172.21.15.31:0/2200671209 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c01cdb0 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.338+0000 7f2d43dff700 1 -- 172.21.15.31:0/2200671209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3c154a50 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.338+0000 7f2d31ffb700 1 -- 172.21.15.31:0/2200671209 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c01a6a0 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.338+0000 7f2d43dff700 1 -- 172.21.15.31:0/2200671209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3c155010 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.339+0000 7f2d43dff700 1 -- 172.21.15.31:0/2200671209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3c1557f0 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.340+0000 7f2d31ffb700 1 -- 172.21.15.31:0/2200671209 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2d2c01aa90 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.341+0000 7f2d31ffb700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2d20067290 0x7f2d20069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:27.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.341+0000 7f2d31ffb700 1 -- 172.21.15.31:0/2200671209 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2d2c086a70 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.341+0000 7f2d41b9b700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2d20067290 0x7f2d20069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:27.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.342+0000 7f2d41b9b700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2d20067290 0x7f2d20069740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c153db0 tx=0x7f2d38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:27.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.346+0000 7f2d31ffb700 1 -- 172.21.15.31:0/2200671209 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d2c053730 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.597+0000 7f2d43dff700 1 -- 172.21.15.31:0/2200671209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2d3c04c380 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.597+0000 7f2d31ffb700 1 -- 172.21.15.31:0/2200671209 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2d2c056d50 con 0x7f2d3c0cc0d0 2023-10-06T05:21:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:21:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.599+0000 7f2d43dff700 1 -- 172.21.15.31:0/2200671209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2d20067290 msgr2=0x7f2d20069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:27.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.599+0000 7f2d43dff700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2d20067290 0x7f2d20069740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c153db0 tx=0x7f2d38009040 comp rx=0 tx=0).stop 2023-10-06T05:21:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.599+0000 7f2d43dff700 1 -- 172.21.15.31:0/2200671209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d3c0cc0d0 msgr2=0x7f2d3c0c7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.599+0000 7f2d43dff700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d3c0cc0d0 0x7f2d3c0c7370 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0030c0 tx=0x7f2d2c016000 comp rx=0 tx=0).stop 2023-10-06T05:21:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.599+0000 7f2d43dff700 1 -- 172.21.15.31:0/2200671209 shutdown_connections 2023-10-06T05:21:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.599+0000 7f2d43dff700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d3c1595a0 0x7f2d3c142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.599+0000 7f2d43dff700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d3c068d80 0x7f2d3c155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.599+0000 7f2d43dff700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f2d20067290 0x7f2d20069740 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.599+0000 7f2d43dff700 1 --2- 172.21.15.31:0/2200671209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d3c0cc0d0 0x7f2d3c0c7370 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.599+0000 7f2d43dff700 1 -- 172.21.15.31:0/2200671209 >> 172.21.15.31:0/2200671209 conn(0x7f2d3c06b620 msgr2=0x7f2d3c0ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.600+0000 7f2d43dff700 1 -- 172.21.15.31:0/2200671209 shutdown_connections 2023-10-06T05:21:27.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:27.600+0000 7f2d43dff700 1 -- 172.21.15.31:0/2200671209 wait complete. 2023-10-06T05:21:27.619 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:21:27.619 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:21:27.619 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:21:27.619 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:21:27.619 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:21:27.619 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:21:27.620 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:21:27.620 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:21:27.620 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:21:27.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:27.620 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:21:27.672 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:27.673 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42027sec preferred_lft 42027sec 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:21:27.674 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:21:27.675 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:21:27.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:27.677 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:21:27.748 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:21:27.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:27.751 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:21:27.801 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:21:27.802 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:21:27.802 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:21:27.802 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:21:27.802 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:21:27.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:27.803 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:21:27.803 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:21:27.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:21:27.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:21:27.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:21:27.803 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:21:27.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:27.975 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:21:28.021 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:21:28.021 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:21:28.022 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:21:28.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:28.022 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:21:28.135 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:28.136 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:21:28.183 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:21:28.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:21:28.183 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:21:28.268 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:21:28.268 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:21:28.268 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:21:28.409 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:21:28.409 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:21:28.434 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:21:28.435 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:21:28.435 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:21:28.435 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:21:28.436 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:21:28.438 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:21:28.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:28.439 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:21:28.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:21:28.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.718+0000 7f58b93ff700 1 Processor -- start 2023-10-06T05:21:28.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.719+0000 7f58b93ff700 1 -- start start 2023-10-06T05:21:28.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.720+0000 7f58b93ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58b4159410 0x7f58b40c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.720+0000 7f58b93ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b41597e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.720+0000 7f58b93ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b4069750 con 0x7f58b40c73c0 2023-10-06T05:21:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.720+0000 7f58b93ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b40c7860 con 0x7f58b4159410 2023-10-06T05:21:28.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.720+0000 7f58b93ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b4069a30 con 0x7f58b406b4c0 2023-10-06T05:21:28.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.720+0000 7f58b37fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b41597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.720+0000 7f58b2ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58b4159410 0x7f58b40c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.721+0000 7f58b37fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b41597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:37064/0 (socket says 172.21.15.31:37064) 2023-10-06T05:21:28.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.721+0000 7f58b37fe700 1 -- 172.21.15.31:0/945298299 learned_addr learned my addr 172.21.15.31:0/945298299 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:28.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.721+0000 7f58b37fe700 1 -- 172.21.15.31:0/945298299 >> v2:172.21.15.62:3300/0 conn(0x7f58b4159410 msgr2=0x7f58b40c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:28.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.721+0000 7f58b37fe700 1 --2- 172.21.15.31:0/945298299 >> v2:172.21.15.62:3300/0 conn(0x7f58b4159410 0x7f58b40c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.721+0000 7f58b37fe700 1 -- 172.21.15.31:0/945298299 >> v1:172.21.15.62:6789/0 conn(0x7f58b40c73c0 legacy=0x7f58b406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.721+0000 7f58b37fe700 1 -- 172.21.15.31:0/945298299 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4069d10 con 0x7f58b406b4c0 2023-10-06T05:21:28.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.721+0000 7f58b37fe700 1 --2- 172.21.15.31:0/945298299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b41597e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009f20 tx=0x7f58ac009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3371e8eb291e4cff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.721+0000 7f58b1ffb700 1 -- 172.21.15.31:0/945298299 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00e070 con 0x7f58b406b4c0 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.721+0000 7f58b1ffb700 1 -- 172.21.15.31:0/945298299 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00b9d0 con 0x7f58b406b4c0 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.722+0000 7f58b1ffb700 1 -- 172.21.15.31:0/945298299 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0092e0 con 0x7f58b406b4c0 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.722+0000 7f58b93ff700 1 -- 172.21.15.31:0/945298299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 msgr2=0x7f58b41597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.722+0000 7f58b93ff700 1 --2- 172.21.15.31:0/945298299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b41597e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009f20 tx=0x7f58ac009840 comp rx=0 tx=0).stop 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.722+0000 7f58b93ff700 1 -- 172.21.15.31:0/945298299 shutdown_connections 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.722+0000 7f58b93ff700 1 --2- 172.21.15.31:0/945298299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b41597e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.722+0000 7f58b93ff700 1 --2- 172.21.15.31:0/945298299 >> v2:172.21.15.62:3300/0 conn(0x7f58b4159410 0x7f58b40c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.722+0000 7f58b93ff700 1 -- 172.21.15.31:0/945298299 >> 172.21.15.31:0/945298299 conn(0x7f58b40c9350 msgr2=0x7f58b40c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.722+0000 7f58b93ff700 1 -- 172.21.15.31:0/945298299 shutdown_connections 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.722+0000 7f58b93ff700 1 -- 172.21.15.31:0/945298299 wait complete. 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.724+0000 7f58b93ff700 1 Processor -- start 2023-10-06T05:21:28.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.724+0000 7f58b93ff700 1 -- start start 2023-10-06T05:21:28.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b93ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b415c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b93ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58b40c73c0 0x7f58b40b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b93ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b4159410 0x7f58b4153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b93ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40b1b00 con 0x7f58b4159410 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b93ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40b0a60 con 0x7f58b40c73c0 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b93ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40b0d40 con 0x7f58b406b4c0 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b4159410 0x7f58b4153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b2ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b2ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b415c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:37070/0 (socket says 172.21.15.31:37070) 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b2ffd700 1 -- 172.21.15.31:0/1201049089 learned_addr learned my addr 172.21.15.31:0/1201049089 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b27fc700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58b40c73c0 0x7f58b40b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b2ffd700 1 -- 172.21.15.31:0/1201049089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58b40c73c0 msgr2=0x7f58b40b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b2ffd700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58b40c73c0 0x7f58b40b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b2ffd700 1 -- 172.21.15.31:0/1201049089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b4159410 msgr2=0x7f58b4153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b2ffd700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b4159410 0x7f58b4153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.725+0000 7f58b2ffd700 1 -- 172.21.15.31:0/1201049089 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac009ae0 con 0x7f58b406b4c0 2023-10-06T05:21:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.726+0000 7f58b2ffd700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b415c420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f58a400ec80 tx=0x7f58a40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.726+0000 7f589affd700 1 -- 172.21.15.31:0/1201049089 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a400c850 con 0x7f58b406b4c0 2023-10-06T05:21:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.726+0000 7f589affd700 1 -- 172.21.15.31:0/1201049089 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a400c9b0 con 0x7f58b406b4c0 2023-10-06T05:21:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.726+0000 7f589affd700 1 -- 172.21.15.31:0/1201049089 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a400cd50 con 0x7f58b406b4c0 2023-10-06T05:21:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.726+0000 7f58b93ff700 1 -- 172.21.15.31:0/1201049089 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b40b1020 con 0x7f58b406b4c0 2023-10-06T05:21:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.726+0000 7f58b93ff700 1 -- 172.21.15.31:0/1201049089 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b40b1660 con 0x7f58b406b4c0 2023-10-06T05:21:28.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.727+0000 7f58b93ff700 1 -- 172.21.15.31:0/1201049089 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b40b1950 con 0x7f58b406b4c0 2023-10-06T05:21:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.728+0000 7f589affd700 1 -- 172.21.15.31:0/1201049089 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f58a4013680 con 0x7f58b406b4c0 2023-10-06T05:21:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.729+0000 7f589affd700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f58a0067290 0x7f58a0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.729+0000 7f589affd700 1 -- 172.21.15.31:0/1201049089 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f58a4084e90 con 0x7f58b406b4c0 2023-10-06T05:21:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.730+0000 7f58b27fc700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f58a0067290 0x7f58a0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:28.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.730+0000 7f58b27fc700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f58a0067290 0x7f58a0069740 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f58b40adcd0 tx=0x7f58a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:28.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.734+0000 7f589affd700 1 -- 172.21.15.31:0/1201049089 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58a4051b50 con 0x7f58b406b4c0 2023-10-06T05:21:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:28.982+0000 7f58b93ff700 1 -- 172.21.15.31:0/1201049089 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f58b404c380 con 0x7f58b406b4c0 2023-10-06T05:21:29.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.087+0000 7f589affd700 1 -- 172.21.15.31:0/1201049089 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 136+0+0 (secure 0 0 0) 0x7f58a4055170 con 0x7f58b406b4c0 2023-10-06T05:21:30.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 -- 172.21.15.31:0/1201049089 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f58a0067290 msgr2=0x7f58a0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f58a0067290 0x7f58a0069740 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f58b40adcd0 tx=0x7f58a8009040 comp rx=0 tx=0).stop 2023-10-06T05:21:30.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 -- 172.21.15.31:0/1201049089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 msgr2=0x7f58b415c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b415c420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f58a400ec80 tx=0x7f58a40091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 -- 172.21.15.31:0/1201049089 shutdown_connections 2023-10-06T05:21:30.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f58a0067290 0x7f58a0069740 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58b4159410 0x7f58b4153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58b406b4c0 0x7f58b415c420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 --2- 172.21.15.31:0/1201049089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58b40c73c0 0x7f58b40b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 -- 172.21.15.31:0/1201049089 >> 172.21.15.31:0/1201049089 conn(0x7f58b40c9350 msgr2=0x7f58b40ca970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 -- 172.21.15.31:0/1201049089 shutdown_connections 2023-10-06T05:21:30.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:29.089+0000 7f58b93ff700 1 -- 172.21.15.31:0/1201049089 wait complete. 2023-10-06T05:21:30.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:21:30.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.301+0000 7ff4773ff700 1 Processor -- start 2023-10-06T05:21:30.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.301+0000 7ff4773ff700 1 -- start start 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.302+0000 7ff4773ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff470159500 0x7ff4700b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.302+0000 7ff4773ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4700c3f40 con 0x7ff47015d2d0 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.302+0000 7ff4773ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4700cb000 con 0x7ff4700cd510 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.302+0000 7ff4773ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4700b2860 con 0x7ff470159500 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.302+0000 7ff47519b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff470159500 0x7ff4700b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.302+0000 7ff47499a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff47015d2d0 0x7ff4700cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47568/0 (socket says 172.21.15.31:47568) 2023-10-06T05:21:30.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.302+0000 7ff47499a700 1 -- 172.21.15.31:0/756366140 learned_addr learned my addr 172.21.15.31:0/756366140 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:30.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.303+0000 7ff47519b700 1 -- 172.21.15.31:0/756366140 >> v1:172.21.15.62:6789/0 conn(0x7ff4700cd510 legacy=0x7ff4700aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.303+0000 7ff47519b700 1 -- 172.21.15.31:0/756366140 >> v1:172.21.15.31:6789/0 conn(0x7ff47015d2d0 legacy=0x7ff4700cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.303+0000 7ff47519b700 1 -- 172.21.15.31:0/756366140 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47015d7d0 con 0x7ff470159500 2023-10-06T05:21:30.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.303+0000 7ff47519b700 1 --2- 172.21.15.31:0/756366140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 0x7ff4700b0ab0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0098e0 tx=0x7ff46c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bedb9e078dafae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.303+0000 7ff467fff700 1 -- 172.21.15.31:0/756366140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c009e80 con 0x7ff470159500 2023-10-06T05:21:30.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.303+0000 7ff467fff700 1 -- 172.21.15.31:0/756366140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c00b860 con 0x7ff470159500 2023-10-06T05:21:30.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.303+0000 7ff467fff700 1 -- 172.21.15.31:0/756366140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c00bbe0 con 0x7ff470159500 2023-10-06T05:21:30.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.304+0000 7ff4773ff700 1 -- 172.21.15.31:0/756366140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 msgr2=0x7ff4700b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.304+0000 7ff4773ff700 1 --2- 172.21.15.31:0/756366140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 0x7ff4700b0ab0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0098e0 tx=0x7ff46c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.304+0000 7ff4773ff700 1 -- 172.21.15.31:0/756366140 shutdown_connections 2023-10-06T05:21:30.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.304+0000 7ff4773ff700 1 --2- 172.21.15.31:0/756366140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 0x7ff4700b0ab0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.304+0000 7ff4773ff700 1 -- 172.21.15.31:0/756366140 >> 172.21.15.31:0/756366140 conn(0x7ff47006ec90 msgr2=0x7ff47006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:30.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.304+0000 7ff4773ff700 1 -- 172.21.15.31:0/756366140 shutdown_connections 2023-10-06T05:21:30.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.304+0000 7ff4773ff700 1 -- 172.21.15.31:0/756366140 wait complete. 2023-10-06T05:21:30.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.306+0000 7ff4773ff700 1 Processor -- start 2023-10-06T05:21:30.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.306+0000 7ff4773ff700 1 -- start start 2023-10-06T05:21:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff4773ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4700cd510 0x7ff4700b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff4773ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 0x7ff4700c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff4773ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff47015d2d0 0x7ff4700cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff4773ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700b5190 con 0x7ff470159500 2023-10-06T05:21:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff4773ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700c97f0 con 0x7ff4700cd510 2023-10-06T05:21:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff4773ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700c9ad0 con 0x7ff47015d2d0 2023-10-06T05:21:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47499a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 0x7ff4700c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47499a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 0x7ff4700c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47376/0 (socket says 172.21.15.31:47376) 2023-10-06T05:21:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47499a700 1 -- 172.21.15.31:0/716484522 learned_addr learned my addr 172.21.15.31:0/716484522 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47519b700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4700cd510 0x7ff4700b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47599c700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff47015d2d0 0x7ff4700cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47499a700 1 -- 172.21.15.31:0/716484522 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff47015d2d0 msgr2=0x7ff4700cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47499a700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff47015d2d0 0x7ff4700cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47499a700 1 -- 172.21.15.31:0/716484522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4700cd510 msgr2=0x7ff4700b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47499a700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4700cd510 0x7ff4700b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47499a700 1 -- 172.21.15.31:0/716484522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c009420 con 0x7ff470159500 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.307+0000 7ff47519b700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4700cd510 0x7ff4700b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.308+0000 7ff47499a700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 0x7ff4700c8e80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff46000c590 tx=0x7ff460007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.308+0000 7ff4657fa700 1 -- 172.21.15.31:0/716484522 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460001160 con 0x7ff470159500 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.308+0000 7ff4657fa700 1 -- 172.21.15.31:0/716484522 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4600072e0 con 0x7ff470159500 2023-10-06T05:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.308+0000 7ff4657fa700 1 -- 172.21.15.31:0/716484522 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4600057f0 con 0x7ff470159500 2023-10-06T05:21:30.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.308+0000 7ff4773ff700 1 -- 172.21.15.31:0/716484522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4700c9db0 con 0x7ff470159500 2023-10-06T05:21:30.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.308+0000 7ff4773ff700 1 -- 172.21.15.31:0/716484522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47015a7e0 con 0x7ff470159500 2023-10-06T05:21:30.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.309+0000 7ff4773ff700 1 -- 172.21.15.31:0/716484522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47004c380 con 0x7ff470159500 2023-10-06T05:21:30.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.310+0000 7ff4657fa700 1 -- 172.21.15.31:0/716484522 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff460005c20 con 0x7ff470159500 2023-10-06T05:21:30.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.311+0000 7ff4657fa700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff454067290 0x7ff454069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.312+0000 7ff4657fa700 1 -- 172.21.15.31:0/716484522 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff460084060 con 0x7ff470159500 2023-10-06T05:21:30.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.312+0000 7ff47519b700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff454067290 0x7ff454069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.312+0000 7ff47519b700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff454067290 0x7ff454069740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0093f0 tx=0x7ff46c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.316+0000 7ff4657fa700 1 -- 172.21.15.31:0/716484522 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff460050d20 con 0x7ff470159500 2023-10-06T05:21:30.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.546+0000 7ff4773ff700 1 -- 172.21.15.31:0/716484522 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4700cade0 con 0x7ff454067290 2023-10-06T05:21:30.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.548+0000 7ff4657fa700 1 -- 172.21.15.31:0/716484522 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff4700cade0 con 0x7ff454067290 2023-10-06T05:21:30.549 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:21:30.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:30.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:21:30.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:30.550 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:21:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 -- 172.21.15.31:0/716484522 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff454067290 msgr2=0x7ff454069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff454067290 0x7ff454069740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0093f0 tx=0x7ff46c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 -- 172.21.15.31:0/716484522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 msgr2=0x7ff4700c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 0x7ff4700c8e80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff46000c590 tx=0x7ff460007e40 comp rx=0 tx=0).stop 2023-10-06T05:21:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 -- 172.21.15.31:0/716484522 shutdown_connections 2023-10-06T05:21:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff47015d2d0 0x7ff4700cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4700cd510 0x7ff4700b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7ff454067290 0x7ff454069740 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 --2- 172.21.15.31:0/716484522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470159500 0x7ff4700c8e80 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 -- 172.21.15.31:0/716484522 >> 172.21.15.31:0/716484522 conn(0x7ff47006ec90 msgr2=0x7ff470158570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 -- 172.21.15.31:0/716484522 shutdown_connections 2023-10-06T05:21:30.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.550+0000 7ff4773ff700 1 -- 172.21.15.31:0/716484522 wait complete. 2023-10-06T05:21:30.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:21:30.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.764+0000 7fb1565ff700 1 Processor -- start 2023-10-06T05:21:30.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.765+0000 7fb1565ff700 1 -- start start 2023-10-06T05:21:30.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.766+0000 7fb1565ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1500b0b90 0x7fb1500af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.766+0000 7fb1565ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb1500af6c0 0x7fb1500adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.766+0000 7fb1565ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1500ad690 con 0x7fb150153d40 2023-10-06T05:21:30.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.766+0000 7fb1565ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1500acd60 con 0x7fb1500b0b90 2023-10-06T05:21:30.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.766+0000 7fb1565ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1500ad040 con 0x7fb1500af6c0 2023-10-06T05:21:30.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.766+0000 7fb14f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1500b0b90 0x7fb1500af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.766+0000 7fb14ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb150153d40 0x7fb1500b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47578/0 (socket says 172.21.15.31:47578) 2023-10-06T05:21:30.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.766+0000 7fb14f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1500b0b90 0x7fb1500af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47382/0 (socket says 172.21.15.31:47382) 2023-10-06T05:21:30.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.766+0000 7fb14ffff700 1 -- 172.21.15.31:0/2748446623 learned_addr learned my addr 172.21.15.31:0/2748446623 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:30.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.766+0000 7fb154b9c700 1 --2- 172.21.15.31:0/2748446623 >> v2:172.21.15.62:3300/0 conn(0x7fb1500af6c0 0x7fb1500adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.767+0000 7fb14f7fe700 1 -- 172.21.15.31:0/2748446623 >> v2:172.21.15.62:3300/0 conn(0x7fb1500af6c0 msgr2=0x7fb1500adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.767+0000 7fb14f7fe700 1 --2- 172.21.15.31:0/2748446623 >> v2:172.21.15.62:3300/0 conn(0x7fb1500af6c0 0x7fb1500adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.767+0000 7fb14f7fe700 1 -- 172.21.15.31:0/2748446623 >> v1:172.21.15.31:6789/0 conn(0x7fb150153d40 legacy=0x7fb1500b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.767+0000 7fb14f7fe700 1 -- 172.21.15.31:0/2748446623 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1500ad320 con 0x7fb1500b0b90 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.767+0000 7fb14f7fe700 1 --2- 172.21.15.31:0/2748446623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 0x7fb1500af180 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb150156dd0 tx=0x7fb1440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971100bf5e4e9c5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.767+0000 7fb14effd700 1 -- 172.21.15.31:0/2748446623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144011630 con 0x7fb1500b0b90 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.767+0000 7fb14effd700 1 -- 172.21.15.31:0/2748446623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb144011790 con 0x7fb1500b0b90 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.767+0000 7fb14effd700 1 -- 172.21.15.31:0/2748446623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144011b30 con 0x7fb1500b0b90 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.768+0000 7fb1565ff700 1 -- 172.21.15.31:0/2748446623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 msgr2=0x7fb1500af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.768+0000 7fb1565ff700 1 --2- 172.21.15.31:0/2748446623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 0x7fb1500af180 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb150156dd0 tx=0x7fb1440091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.768+0000 7fb1565ff700 1 -- 172.21.15.31:0/2748446623 shutdown_connections 2023-10-06T05:21:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.768+0000 7fb1565ff700 1 --2- 172.21.15.31:0/2748446623 >> v2:172.21.15.62:3300/0 conn(0x7fb1500af6c0 0x7fb1500adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.768+0000 7fb1565ff700 1 --2- 172.21.15.31:0/2748446623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 0x7fb1500af180 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.768+0000 7fb1565ff700 1 -- 172.21.15.31:0/2748446623 >> 172.21.15.31:0/2748446623 conn(0x7fb150067560 msgr2=0x7fb150067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:30.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.768+0000 7fb1565ff700 1 -- 172.21.15.31:0/2748446623 shutdown_connections 2023-10-06T05:21:30.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.768+0000 7fb1565ff700 1 -- 172.21.15.31:0/2748446623 wait complete. 2023-10-06T05:21:30.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.770+0000 7fb1565ff700 1 Processor -- start 2023-10-06T05:21:30.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.770+0000 7fb1565ff700 1 -- start start 2023-10-06T05:21:30.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.771+0000 7fb1565ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1500af6c0 0x7fb150151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.771+0000 7fb1565ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 0x7fb1500c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb1565ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb150153d40 0x7fb15016a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb1565ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15016b1b0 con 0x7fb1500b0b90 2023-10-06T05:21:30.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb1565ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15016abf0 con 0x7fb1500af6c0 2023-10-06T05:21:30.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb1565ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15016aed0 con 0x7fb150153d40 2023-10-06T05:21:30.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 0x7fb1500c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 0x7fb1500c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47398/0 (socket says 172.21.15.31:47398) 2023-10-06T05:21:30.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14f7fe700 1 -- 172.21.15.31:0/1136644863 learned_addr learned my addr 172.21.15.31:0/1136644863 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:30.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14f7fe700 1 -- 172.21.15.31:0/1136644863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb150153d40 msgr2=0x7fb15016a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14f7fe700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb150153d40 0x7fb15016a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14f7fe700 1 -- 172.21.15.31:0/1136644863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1500af6c0 msgr2=0x7fb150151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14ffff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1500af6c0 0x7fb150151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14f7fe700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1500af6c0 0x7fb150151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14f7fe700 1 -- 172.21.15.31:0/1136644863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb144009420 con 0x7fb1500b0b90 2023-10-06T05:21:30.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14ffff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1500af6c0 0x7fb150151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:30.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.772+0000 7fb14f7fe700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 0x7fb1500c9760 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb144013000 tx=0x7fb144017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.773+0000 7fb137fff700 1 -- 172.21.15.31:0/1136644863 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144011b10 con 0x7fb1500b0b90 2023-10-06T05:21:30.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.773+0000 7fb137fff700 1 -- 172.21.15.31:0/1136644863 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1440233f0 con 0x7fb1500b0b90 2023-10-06T05:21:30.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.773+0000 7fb137fff700 1 -- 172.21.15.31:0/1136644863 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1440237b0 con 0x7fb1500b0b90 2023-10-06T05:21:30.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.773+0000 7fb1565ff700 1 -- 172.21.15.31:0/1136644863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1500c8f80 con 0x7fb1500b0b90 2023-10-06T05:21:30.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.773+0000 7fb1565ff700 1 -- 172.21.15.31:0/1136644863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb150152480 con 0x7fb1500b0b90 2023-10-06T05:21:30.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.774+0000 7fb1565ff700 1 -- 172.21.15.31:0/1136644863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1500afbc0 con 0x7fb1500b0b90 2023-10-06T05:21:30.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.775+0000 7fb137fff700 1 -- 172.21.15.31:0/1136644863 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb1440193b0 con 0x7fb1500b0b90 2023-10-06T05:21:30.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.776+0000 7fb137fff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb13c067290 0x7fb13c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:30.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.776+0000 7fb137fff700 1 -- 172.21.15.31:0/1136644863 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb144086170 con 0x7fb1500b0b90 2023-10-06T05:21:30.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.776+0000 7fb14ffff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb13c067290 0x7fb13c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:30.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.777+0000 7fb14ffff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb13c067290 0x7fb13c069740 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb140001700 tx=0x7fb140009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:30.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.780+0000 7fb137fff700 1 -- 172.21.15.31:0/1136644863 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb144052eb0 con 0x7fb1500b0b90 2023-10-06T05:21:30.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.983+0000 7fb1565ff700 1 -- 172.21.15.31:0/1136644863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb15004c380 con 0x7fb1500b0b90 2023-10-06T05:21:30.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.983+0000 7fb137fff700 1 -- 172.21.15.31:0/1136644863 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb1440564d0 con 0x7fb1500b0b90 2023-10-06T05:21:30.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 -- 172.21.15.31:0/1136644863 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb13c067290 msgr2=0x7fb13c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb13c067290 0x7fb13c069740 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb140001700 tx=0x7fb140009040 comp rx=0 tx=0).stop 2023-10-06T05:21:30.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 -- 172.21.15.31:0/1136644863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 msgr2=0x7fb1500c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:30.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 0x7fb1500c9760 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb144013000 tx=0x7fb144017000 comp rx=0 tx=0).stop 2023-10-06T05:21:30.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 -- 172.21.15.31:0/1136644863 shutdown_connections 2023-10-06T05:21:30.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb150153d40 0x7fb15016a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1500af6c0 0x7fb150151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7fb13c067290 0x7fb13c069740 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 --2- 172.21.15.31:0/1136644863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1500b0b90 0x7fb1500c9760 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:30.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 -- 172.21.15.31:0/1136644863 >> 172.21.15.31:0/1136644863 conn(0x7fb150067560 msgr2=0x7fb15015fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:30.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 -- 172.21.15.31:0/1136644863 shutdown_connections 2023-10-06T05:21:30.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:30.986+0000 7fb1565ff700 1 -- 172.21.15.31:0/1136644863 wait complete. 2023-10-06T05:21:31.005 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:21:31.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:31.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:21:31.096 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:21:31.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:31.097 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:21:31.143 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:21:31.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:21:31.144 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:21:31.237 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:21:31.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:21:31.238 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:21:31.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:31.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:31.340 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:21:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.483+0000 7faa50bff700 1 Processor -- start 2023-10-06T05:21:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.483+0000 7faa50bff700 1 -- start start 2023-10-06T05:21:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa50bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa4c13a5b0 0x7faa4c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa50bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faa4c06d980 0x7faa4c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa50bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa4c06ddb0 con 0x7faa4c13a1e0 2023-10-06T05:21:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa50bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4c0c71b0 con 0x7faa4c13a5b0 2023-10-06T05:21:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa50bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4c0c7490 con 0x7faa4c06d980 2023-10-06T05:21:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa49d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa4c13a5b0 0x7faa4c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa49d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faa4c13a5b0 0x7faa4c06d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47412/0 (socket says 172.21.15.31:47412) 2023-10-06T05:21:31.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa49d9b700 1 -- 172.21.15.31:0/1150696151 learned_addr learned my addr 172.21.15.31:0/1150696151 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:31.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa4ad9d700 1 --2- 172.21.15.31:0/1150696151 >> v2:172.21.15.62:3300/0 conn(0x7faa4c06d980 0x7faa4c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa49d9b700 1 -- 172.21.15.31:0/1150696151 >> v2:172.21.15.62:3300/0 conn(0x7faa4c06d980 msgr2=0x7faa4c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:31.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.485+0000 7faa49d9b700 1 --2- 172.21.15.31:0/1150696151 >> v2:172.21.15.62:3300/0 conn(0x7faa4c06d980 0x7faa4c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.486+0000 7faa49d9b700 1 -- 172.21.15.31:0/1150696151 >> v1:172.21.15.62:6789/0 conn(0x7faa4c13a1e0 legacy=0x7faa4c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:31.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.486+0000 7faa49d9b700 1 -- 172.21.15.31:0/1150696151 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c0c7770 con 0x7faa4c13a5b0 2023-10-06T05:21:31.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.486+0000 7faa49d9b700 1 --2- 172.21.15.31:0/1150696151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 0x7faa4c06d440 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faa4c06b920 tx=0x7faa40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c318a4ae018b607 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:31.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.486+0000 7faa4959a700 1 -- 172.21.15.31:0/1150696151 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4000b860 con 0x7faa4c13a5b0 2023-10-06T05:21:31.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.486+0000 7faa4959a700 1 -- 172.21.15.31:0/1150696151 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4000b9c0 con 0x7faa4c13a5b0 2023-10-06T05:21:31.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.486+0000 7faa4959a700 1 -- 172.21.15.31:0/1150696151 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4000bd60 con 0x7faa4c13a5b0 2023-10-06T05:21:31.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.487+0000 7faa50bff700 1 -- 172.21.15.31:0/1150696151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 msgr2=0x7faa4c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.487+0000 7faa50bff700 1 --2- 172.21.15.31:0/1150696151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 0x7faa4c06d440 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faa4c06b920 tx=0x7faa40009250 comp rx=0 tx=0).stop 2023-10-06T05:21:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.487+0000 7faa50bff700 1 -- 172.21.15.31:0/1150696151 shutdown_connections 2023-10-06T05:21:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.487+0000 7faa50bff700 1 --2- 172.21.15.31:0/1150696151 >> v2:172.21.15.62:3300/0 conn(0x7faa4c06d980 0x7faa4c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.487+0000 7faa50bff700 1 --2- 172.21.15.31:0/1150696151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 0x7faa4c06d440 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.487+0000 7faa50bff700 1 -- 172.21.15.31:0/1150696151 >> 172.21.15.31:0/1150696151 conn(0x7faa4c0cc0b0 msgr2=0x7faa4c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.487+0000 7faa50bff700 1 -- 172.21.15.31:0/1150696151 shutdown_connections 2023-10-06T05:21:31.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.487+0000 7faa50bff700 1 -- 172.21.15.31:0/1150696151 wait complete. 2023-10-06T05:21:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.488+0000 7faa50bff700 1 Processor -- start 2023-10-06T05:21:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.488+0000 7faa50bff700 1 -- start start 2023-10-06T05:21:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.489+0000 7faa50bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa4c06d980 0x7faa4c139980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.489+0000 7faa50bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4c13a1e0 0x7faa4c0b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.489+0000 7faa50bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 0x7faa4c069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.489+0000 7faa50bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c1528b0 con 0x7faa4c13a5b0 2023-10-06T05:21:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.489+0000 7faa50bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c0aff10 con 0x7faa4c13a1e0 2023-10-06T05:21:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.489+0000 7faa50bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c1522a0 con 0x7faa4c06d980 2023-10-06T05:21:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa4ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 0x7faa4c069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa4ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 0x7faa4c069970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47422/0 (socket says 172.21.15.31:47422) 2023-10-06T05:21:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa4ad9d700 1 -- 172.21.15.31:0/744028136 learned_addr learned my addr 172.21.15.31:0/744028136 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa49d9b700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4c13a1e0 0x7faa4c0b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa4a59c700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa4c06d980 0x7faa4c139980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa4ad9d700 1 -- 172.21.15.31:0/744028136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa4c06d980 msgr2=0x7faa4c139980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa4ad9d700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa4c06d980 0x7faa4c139980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa4ad9d700 1 -- 172.21.15.31:0/744028136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4c13a1e0 msgr2=0x7faa4c0b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa4ad9d700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4c13a1e0 0x7faa4c0b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa4ad9d700 1 -- 172.21.15.31:0/744028136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400094f0 con 0x7faa4c13a5b0 2023-10-06T05:21:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa49d9b700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4c13a1e0 0x7faa4c0b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa4ad9d700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 0x7faa4c069970 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7faa44009fa0 tx=0x7faa44009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa3a7fc700 1 -- 172.21.15.31:0/744028136 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4400ed20 con 0x7faa4c13a5b0 2023-10-06T05:21:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.490+0000 7faa3a7fc700 1 -- 172.21.15.31:0/744028136 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4400ee80 con 0x7faa4c13a5b0 2023-10-06T05:21:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.491+0000 7faa3a7fc700 1 -- 172.21.15.31:0/744028136 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44014670 con 0x7faa4c13a5b0 2023-10-06T05:21:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.491+0000 7faa50bff700 1 -- 172.21.15.31:0/744028136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4c0afc30 con 0x7faa4c13a5b0 2023-10-06T05:21:31.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.491+0000 7faa50bff700 1 -- 172.21.15.31:0/744028136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c139ec0 con 0x7faa4c13a5b0 2023-10-06T05:21:31.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.492+0000 7faa50bff700 1 -- 172.21.15.31:0/744028136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c152100 con 0x7faa4c13a5b0 2023-10-06T05:21:31.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.492+0000 7faa3a7fc700 1 -- 172.21.15.31:0/744028136 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7faa440149e0 con 0x7faa4c13a5b0 2023-10-06T05:21:31.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.493+0000 7faa3a7fc700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa2c067290 0x7faa2c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:31.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.494+0000 7faa3a7fc700 1 -- 172.21.15.31:0/744028136 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faa44085400 con 0x7faa4c13a5b0 2023-10-06T05:21:31.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.494+0000 7faa4a59c700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa2c067290 0x7faa2c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.494+0000 7faa4a59c700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa2c067290 0x7faa2c069740 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faa3c003890 tx=0x7faa3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.498+0000 7faa3a7fc700 1 -- 172.21.15.31:0/744028136 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa4c152100 con 0x7faa4c13a5b0 2023-10-06T05:21:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.745+0000 7faa50bff700 1 -- 172.21.15.31:0/744028136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa4c152100 con 0x7faa4c13a5b0 2023-10-06T05:21:31.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.746+0000 7faa3a7fc700 1 -- 172.21.15.31:0/744028136 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4590 (secure 0 0 0) 0x7faa4c152100 con 0x7faa4c13a5b0 2023-10-06T05:21:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:31.748 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2151449662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2151449662},{"type":"v1","addr":"172.21.15.62:6834","nonce":2151449662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/810564890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":810564890},{"type":"v1","addr":"172.21.15.31:6836","nonce":810564890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/802676715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":802676715},{"type":"v1","addr":"172.21.15.31:6837","nonce":802676715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:13.661482+0000","modified":"2023-10-06T05:21:17.173141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6249":{"gid":6249,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/934773709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":934773709},{"type":"v1","addr":"172.21.15.62:6835","nonce":934773709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.748+0000 7faa50bff700 1 -- 172.21.15.31:0/744028136 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa2c067290 msgr2=0x7faa2c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.748+0000 7faa50bff700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa2c067290 0x7faa2c069740 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faa3c003890 tx=0x7faa3c009040 comp rx=0 tx=0).stop 2023-10-06T05:21:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.748+0000 7faa50bff700 1 -- 172.21.15.31:0/744028136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 msgr2=0x7faa4c069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.748+0000 7faa50bff700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 0x7faa4c069970 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7faa44009fa0 tx=0x7faa44009900 comp rx=0 tx=0).stop 2023-10-06T05:21:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.748+0000 7faa50bff700 1 -- 172.21.15.31:0/744028136 shutdown_connections 2023-10-06T05:21:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.748+0000 7faa50bff700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faa2c067290 0x7faa2c069740 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.748+0000 7faa50bff700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa4c13a5b0 0x7faa4c069970 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.748+0000 7faa50bff700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa4c06d980 0x7faa4c139980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.748+0000 7faa50bff700 1 --2- 172.21.15.31:0/744028136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa4c13a1e0 0x7faa4c0b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.748+0000 7faa50bff700 1 -- 172.21.15.31:0/744028136 >> 172.21.15.31:0/744028136 conn(0x7faa4c0cc0b0 msgr2=0x7faa4c1555b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.749+0000 7faa50bff700 1 -- 172.21.15.31:0/744028136 shutdown_connections 2023-10-06T05:21:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:31.749+0000 7faa50bff700 1 -- 172.21.15.31:0/744028136 wait complete. 2023-10-06T05:21:31.752 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 45 2023-10-06T05:21:31.768 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-10-06T05:21:31.985 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-10-06T05:21:31.985 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-10-06T05:21:31.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000529202 2023-10-06T05:21:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.185+0000 7f4d99fff700 1 Processor -- start 2023-10-06T05:21:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.185+0000 7f4d99fff700 1 -- start start 2023-10-06T05:21:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.186+0000 7f4d99fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4d941573a0 0x7f4d940acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.186+0000 7f4d99fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d94157770 0x7f4d940b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.186+0000 7f4d99fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d94157c10 0x7f4d940b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.186+0000 7f4d99fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d94158cf0 con 0x7f4d94157770 2023-10-06T05:21:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.186+0000 7f4d99fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d941580e0 con 0x7f4d941573a0 2023-10-06T05:21:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.186+0000 7f4d99fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d941583c0 con 0x7f4d94157c10 2023-10-06T05:21:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.186+0000 7f4d92ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d94157770 0x7f4d940b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d92ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d94157770 0x7f4d940b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47428/0 (socket says 172.21.15.31:47428) 2023-10-06T05:21:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d92ffd700 1 -- 172.21.15.31:0/1175100475 learned_addr learned my addr 172.21.15.31:0/1175100475 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d937fe700 1 --2- 172.21.15.31:0/1175100475 >> v2:172.21.15.62:3300/0 conn(0x7f4d941573a0 0x7f4d940acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d93fff700 1 --2- 172.21.15.31:0/1175100475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d94157c10 0x7f4d940b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d92ffd700 1 -- 172.21.15.31:0/1175100475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d94157c10 msgr2=0x7f4d940b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d92ffd700 1 --2- 172.21.15.31:0/1175100475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d94157c10 0x7f4d940b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d92ffd700 1 -- 172.21.15.31:0/1175100475 >> v2:172.21.15.62:3300/0 conn(0x7f4d941573a0 msgr2=0x7f4d940acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d92ffd700 1 --2- 172.21.15.31:0/1175100475 >> v2:172.21.15.62:3300/0 conn(0x7f4d941573a0 0x7f4d940acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d92ffd700 1 -- 172.21.15.31:0/1175100475 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d941586a0 con 0x7f4d94157770 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d937fe700 1 --2- 172.21.15.31:0/1175100475 >> v2:172.21.15.62:3300/0 conn(0x7f4d941573a0 0x7f4d940acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.187+0000 7f4d92ffd700 1 --2- 172.21.15.31:0/1175100475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 0x7f4d940b1260 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4d940cd110 tx=0x7f4d88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf00ba19f5ab3e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d927fc700 1 -- 172.21.15.31:0/1175100475 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800b860 con 0x7f4d94157770 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d927fc700 1 -- 172.21.15.31:0/1175100475 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8800b9c0 con 0x7f4d94157770 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d927fc700 1 -- 172.21.15.31:0/1175100475 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800bd60 con 0x7f4d94157770 2023-10-06T05:21:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d99fff700 1 -- 172.21.15.31:0/1175100475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 msgr2=0x7f4d940b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d99fff700 1 --2- 172.21.15.31:0/1175100475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 0x7f4d940b1260 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4d940cd110 tx=0x7f4d88009250 comp rx=0 tx=0).stop 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d99fff700 1 -- 172.21.15.31:0/1175100475 shutdown_connections 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d99fff700 1 --2- 172.21.15.31:0/1175100475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d94157c10 0x7f4d940b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d99fff700 1 --2- 172.21.15.31:0/1175100475 >> v2:172.21.15.62:3300/0 conn(0x7f4d941573a0 0x7f4d940acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d99fff700 1 --2- 172.21.15.31:0/1175100475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 0x7f4d940b1260 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d99fff700 1 -- 172.21.15.31:0/1175100475 >> 172.21.15.31:0/1175100475 conn(0x7f4d94067560 msgr2=0x7f4d94067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d99fff700 1 -- 172.21.15.31:0/1175100475 shutdown_connections 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.188+0000 7f4d99fff700 1 -- 172.21.15.31:0/1175100475 wait complete. 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.190+0000 7f4d99fff700 1 Processor -- start 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.190+0000 7f4d99fff700 1 -- start start 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.191+0000 7f4d99fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d941573a0 0x7f4d9414f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.191+0000 7f4d99fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 0x7f4d94151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.191+0000 7f4d99fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d94157c10 0x7f4d9414d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.191+0000 7f4d99fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d94153210 con 0x7f4d94157770 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.191+0000 7f4d99fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940c83a0 con 0x7f4d94157c10 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.191+0000 7f4d99fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d94152590 con 0x7f4d941573a0 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.191+0000 7f4d92ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 0x7f4d94151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.191+0000 7f4d92ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 0x7f4d94151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47444/0 (socket says 172.21.15.31:47444) 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.191+0000 7f4d92ffd700 1 -- 172.21.15.31:0/1101016898 learned_addr learned my addr 172.21.15.31:0/1101016898 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.191+0000 7f4d937fe700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d941573a0 0x7f4d9414f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.192+0000 7f4d93fff700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d94157c10 0x7f4d9414d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.192+0000 7f4d92ffd700 1 -- 172.21.15.31:0/1101016898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d941573a0 msgr2=0x7f4d9414f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.192+0000 7f4d92ffd700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d941573a0 0x7f4d9414f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.192+0000 7f4d92ffd700 1 -- 172.21.15.31:0/1101016898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d94157c10 msgr2=0x7f4d9414d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.192+0000 7f4d92ffd700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d94157c10 0x7f4d9414d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.192+0000 7f4d92ffd700 1 -- 172.21.15.31:0/1101016898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d880094f0 con 0x7f4d94157770 2023-10-06T05:21:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.192+0000 7f4d937fe700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d941573a0 0x7f4d9414f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.192+0000 7f4d92ffd700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 0x7f4d94151ad0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009290 tx=0x7f4d88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.192+0000 7f4d7b7fe700 1 -- 172.21.15.31:0/1101016898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d88011e40 con 0x7f4d94157770 2023-10-06T05:21:32.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.193+0000 7f4d7b7fe700 1 -- 172.21.15.31:0/1101016898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8801f070 con 0x7f4d94157770 2023-10-06T05:21:32.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.193+0000 7f4d7b7fe700 1 -- 172.21.15.31:0/1101016898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d880266d0 con 0x7f4d94157770 2023-10-06T05:21:32.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.193+0000 7f4d99fff700 1 -- 172.21.15.31:0/1101016898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d94152870 con 0x7f4d94157770 2023-10-06T05:21:32.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.193+0000 7f4d99fff700 1 -- 172.21.15.31:0/1101016898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d94152e30 con 0x7f4d94157770 2023-10-06T05:21:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.194+0000 7f4d99fff700 1 -- 172.21.15.31:0/1101016898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d94153350 con 0x7f4d94157770 2023-10-06T05:21:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.195+0000 7f4d7b7fe700 1 -- 172.21.15.31:0/1101016898 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f4d88026a40 con 0x7f4d94157770 2023-10-06T05:21:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.196+0000 7f4d7b7fe700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d80067290 0x7f4d80069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.196+0000 7f4d7b7fe700 1 -- 172.21.15.31:0/1101016898 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4d88021070 con 0x7f4d94157770 2023-10-06T05:21:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.196+0000 7f4d937fe700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d80067290 0x7f4d80069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.196+0000 7f4d937fe700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d80067290 0x7f4d80069740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4d84006f00 tx=0x7f4d84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.200+0000 7f4d7b7fe700 1 -- 172.21.15.31:0/1101016898 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d94153350 con 0x7f4d94157770 2023-10-06T05:21:32.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.437+0000 7f4d99fff700 1 -- 172.21.15.31:0/1101016898 --> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000529202", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d94153350 con 0x7f4d80067290 2023-10-06T05:21:32.445 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:32.443+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:32.445 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:32.443+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:32.445 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:32.443+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:32.445 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:32.443+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:32.446 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:32.443+0000 7efe4cdfe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:32.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.478+0000 7f4d7b7fe700 1 -- 172.21.15.31:0/1101016898 <== mgr.4104 v2:172.21.15.31:6832/114556 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4d94153350 con 0x7f4d80067290 2023-10-06T05:21:32.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.480+0000 7f4d99fff700 1 -- 172.21.15.31:0/1101016898 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d80067290 msgr2=0x7f4d80069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.480+0000 7f4d99fff700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d80067290 0x7f4d80069740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4d84006f00 tx=0x7f4d84008040 comp rx=0 tx=0).stop 2023-10-06T05:21:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.480+0000 7f4d99fff700 1 -- 172.21.15.31:0/1101016898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 msgr2=0x7f4d94151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.480+0000 7f4d99fff700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 0x7f4d94151ad0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009290 tx=0x7f4d88017000 comp rx=0 tx=0).stop 2023-10-06T05:21:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.480+0000 7f4d99fff700 1 -- 172.21.15.31:0/1101016898 shutdown_connections 2023-10-06T05:21:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.481+0000 7f4d99fff700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d94157c10 0x7f4d9414d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.481+0000 7f4d99fff700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4d941573a0 0x7f4d9414f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.481+0000 7f4d99fff700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f4d80067290 0x7f4d80069740 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.481+0000 7f4d99fff700 1 --2- 172.21.15.31:0/1101016898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d94157770 0x7f4d94151ad0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.481+0000 7f4d99fff700 1 -- 172.21.15.31:0/1101016898 >> 172.21.15.31:0/1101016898 conn(0x7f4d94067560 msgr2=0x7f4d941555c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.481+0000 7f4d99fff700 1 -- 172.21.15.31:0/1101016898 shutdown_connections 2023-10-06T05:21:32.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.481+0000 7f4d99fff700 1 -- 172.21.15.31:0/1101016898 wait complete. 2023-10-06T05:21:32.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:32.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.693+0000 7f25415ff700 1 Processor -- start 2023-10-06T05:21:32.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.693+0000 7f25415ff700 1 -- start start 2023-10-06T05:21:32.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.694+0000 7f25415ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f253c0c7b60 0x7f253c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.694+0000 7f25415ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f253c0aee70 0x7f253c13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.694+0000 7f25415ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c0ca5b0 con 0x7f253c0c7f30 2023-10-06T05:21:32.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.695+0000 7f25415ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c0af2d0 con 0x7f253c0aee70 2023-10-06T05:21:32.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.695+0000 7f25415ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c06f740 con 0x7f253c0c7b60 2023-10-06T05:21:32.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.695+0000 7f253b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f253c0aee70 0x7f253c13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.695+0000 7f253b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f253c0aee70 0x7f253c13f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47484/0 (socket says 172.21.15.31:47484) 2023-10-06T05:21:32.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.695+0000 7f253b7fe700 1 -- 172.21.15.31:0/4210898789 learned_addr learned my addr 172.21.15.31:0/4210898789 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:32.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.695+0000 7f253affd700 1 --2- 172.21.15.31:0/4210898789 >> v2:172.21.15.62:3300/0 conn(0x7f253c0c7b60 0x7f253c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.695+0000 7f253b7fe700 1 -- 172.21.15.31:0/4210898789 >> v2:172.21.15.62:3300/0 conn(0x7f253c0c7b60 msgr2=0x7f253c069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:32.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.695+0000 7f253b7fe700 1 --2- 172.21.15.31:0/4210898789 >> v2:172.21.15.62:3300/0 conn(0x7f253c0c7b60 0x7f253c069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.695+0000 7f253b7fe700 1 -- 172.21.15.31:0/4210898789 >> v1:172.21.15.31:6789/0 conn(0x7f253c0c7f30 legacy=0x7f253c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.695+0000 7f253b7fe700 1 -- 172.21.15.31:0/4210898789 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0c9c80 con 0x7f253c0aee70 2023-10-06T05:21:32.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f253b7fe700 1 --2- 172.21.15.31:0/4210898789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0aee70 0x7f253c13f690 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f253c06fa80 tx=0x7f2534009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0967e1c8e21569 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f2539ffb700 1 -- 172.21.15.31:0/4210898789 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400e070 con 0x7f253c0aee70 2023-10-06T05:21:32.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f2539ffb700 1 -- 172.21.15.31:0/4210898789 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253400ba20 con 0x7f253c0aee70 2023-10-06T05:21:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f2539ffb700 1 -- 172.21.15.31:0/4210898789 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340092e0 con 0x7f253c0aee70 2023-10-06T05:21:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f25415ff700 1 -- 172.21.15.31:0/4210898789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0aee70 msgr2=0x7f253c13f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f25415ff700 1 --2- 172.21.15.31:0/4210898789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0aee70 0x7f253c13f690 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f253c06fa80 tx=0x7f2534009840 comp rx=0 tx=0).stop 2023-10-06T05:21:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f25415ff700 1 -- 172.21.15.31:0/4210898789 shutdown_connections 2023-10-06T05:21:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f25415ff700 1 --2- 172.21.15.31:0/4210898789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0aee70 0x7f253c13f690 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f25415ff700 1 --2- 172.21.15.31:0/4210898789 >> v2:172.21.15.62:3300/0 conn(0x7f253c0c7b60 0x7f253c069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f25415ff700 1 -- 172.21.15.31:0/4210898789 >> 172.21.15.31:0/4210898789 conn(0x7f253c0cc0b0 msgr2=0x7f253c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f25415ff700 1 -- 172.21.15.31:0/4210898789 shutdown_connections 2023-10-06T05:21:32.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.696+0000 7f25415ff700 1 -- 172.21.15.31:0/4210898789 wait complete. 2023-10-06T05:21:32.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.698+0000 7f25415ff700 1 Processor -- start 2023-10-06T05:21:32.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.698+0000 7f25415ff700 1 -- start start 2023-10-06T05:21:32.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.699+0000 7f25415ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0aee70 0x7f253c0c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.699+0000 7f25415ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0c7b60 0x7f253c169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.699+0000 7f25415ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0c7f30 0x7f253c0ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.699+0000 7f25415ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c16aac0 con 0x7f253c0c7f30 2023-10-06T05:21:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.699+0000 7f25415ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c16d630 con 0x7f253c0aee70 2023-10-06T05:21:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.699+0000 7f25415ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c16a500 con 0x7f253c0c7b60 2023-10-06T05:21:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.699+0000 7f253b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0c7f30 0x7f253c0ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.699+0000 7f253b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0c7f30 0x7f253c0ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47494/0 (socket says 172.21.15.31:47494) 2023-10-06T05:21:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.699+0000 7f253b7fe700 1 -- 172.21.15.31:0/1940859182 learned_addr learned my addr 172.21.15.31:0/1940859182 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f253affd700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0aee70 0x7f253c0c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f253a7fc700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0c7b60 0x7f253c169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f253b7fe700 1 -- 172.21.15.31:0/1940859182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0c7b60 msgr2=0x7f253c169b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f253b7fe700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0c7b60 0x7f253c169b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f253b7fe700 1 -- 172.21.15.31:0/1940859182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0aee70 msgr2=0x7f253c0c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f253b7fe700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0aee70 0x7f253c0c2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f253b7fe700 1 -- 172.21.15.31:0/1940859182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2534009ae0 con 0x7f253c0c7f30 2023-10-06T05:21:32.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f253affd700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0aee70 0x7f253c0c2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:32.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f253a7fc700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0c7b60 0x7f253c169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:32.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f253b7fe700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0c7f30 0x7f253c0ade70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f253c06cd80 tx=0x7f2534018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f2522ffd700 1 -- 172.21.15.31:0/1940859182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534008be0 con 0x7f253c0c7f30 2023-10-06T05:21:32.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f2522ffd700 1 -- 172.21.15.31:0/1940859182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2534008d40 con 0x7f253c0c7f30 2023-10-06T05:21:32.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f2522ffd700 1 -- 172.21.15.31:0/1940859182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340266b0 con 0x7f253c0c7f30 2023-10-06T05:21:32.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.700+0000 7f25415ff700 1 -- 172.21.15.31:0/1940859182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c16a7e0 con 0x7f253c0c7f30 2023-10-06T05:21:32.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.701+0000 7f25415ff700 1 -- 172.21.15.31:0/1940859182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c0c2d50 con 0x7f253c0c7f30 2023-10-06T05:21:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.701+0000 7f25415ff700 1 -- 172.21.15.31:0/1940859182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c16d7c0 con 0x7f253c0c7f30 2023-10-06T05:21:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.702+0000 7f2522ffd700 1 -- 172.21.15.31:0/1940859182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2534003040 con 0x7f253c0c7f30 2023-10-06T05:21:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.703+0000 7f2522ffd700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25280672c0 0x7f2528069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.704+0000 7f2522ffd700 1 -- 172.21.15.31:0/1940859182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2534086d30 con 0x7f253c0c7f30 2023-10-06T05:21:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.704+0000 7f253affd700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25280672c0 0x7f2528069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.704+0000 7f253affd700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25280672c0 0x7f2528069770 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f252c006ea0 tx=0x7f252c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.707+0000 7f2522ffd700 1 -- 172.21.15.31:0/1940859182 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2534053990 con 0x7f253c0c7f30 2023-10-06T05:21:32.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.956+0000 7f25415ff700 1 -- 172.21.15.31:0/1940859182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f253c04c380 con 0x7f253c0c7f30 2023-10-06T05:21:32.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.956+0000 7f2522ffd700 1 -- 172.21.15.31:0/1940859182 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f2534056fb0 con 0x7f253c0c7f30 2023-10-06T05:21:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:32.958 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2151449662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2151449662},{"type":"v1","addr":"172.21.15.62:6834","nonce":2151449662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/810564890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":810564890},{"type":"v1","addr":"172.21.15.31:6836","nonce":810564890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/802676715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":802676715},{"type":"v1","addr":"172.21.15.31:6837","nonce":802676715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:13.661482+0000","modified":"2023-10-06T05:21:17.173141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6249":{"gid":6249,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/934773709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":934773709},{"type":"v1","addr":"172.21.15.62:6835","nonce":934773709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.958+0000 7f25415ff700 1 -- 172.21.15.31:0/1940859182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25280672c0 msgr2=0x7f2528069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.958+0000 7f25415ff700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25280672c0 0x7f2528069770 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f252c006ea0 tx=0x7f252c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.959+0000 7f25415ff700 1 -- 172.21.15.31:0/1940859182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0c7f30 msgr2=0x7f253c0ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.959+0000 7f25415ff700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0c7f30 0x7f253c0ade70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f253c06cd80 tx=0x7f2534018000 comp rx=0 tx=0).stop 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.959+0000 7f25415ff700 1 -- 172.21.15.31:0/1940859182 shutdown_connections 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.959+0000 7f25415ff700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f25280672c0 0x7f2528069770 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.959+0000 7f25415ff700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c0c7f30 0x7f253c0ade70 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.959+0000 7f25415ff700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0aee70 0x7f253c0c2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.959+0000 7f25415ff700 1 --2- 172.21.15.31:0/1940859182 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0c7b60 0x7f253c169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.959+0000 7f25415ff700 1 -- 172.21.15.31:0/1940859182 >> 172.21.15.31:0/1940859182 conn(0x7f253c0cc0b0 msgr2=0x7f253c0b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.959+0000 7f25415ff700 1 -- 172.21.15.31:0/1940859182 shutdown_connections 2023-10-06T05:21:32.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:32.959+0000 7f25415ff700 1 -- 172.21.15.31:0/1940859182 wait complete. 2023-10-06T05:21:32.962 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 45 2023-10-06T05:21:32.978 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-10-06T05:21:33.207 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5786,"entity":{"name":{"type":"client","num":5786},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2612034927}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.74498883000000005,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.966148158506027,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5786 172.21.15.31:0/2612034927","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"y","hostname":"smithi031","pid":"114556","root":"/"}}] 2023-10-06T05:21:33.208 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114556", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2612034927, "type": "any" }, "name": { "num": 5786, "type": "client" } }, "id": 5786, "inst": "client.5786 172.21.15.31:0/2612034927", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 19.966148158506027 }, "state": "open", "uptime": 0.74498883 } ] 2023-10-06T05:21:33.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-06T05:21:33.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.404+0000 7faf731ff700 1 Processor -- start 2023-10-06T05:21:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.404+0000 7faf731ff700 1 -- start start 2023-10-06T05:21:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.405+0000 7faf731ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf6c159500 0x7faf6c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.405+0000 7faf731ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faf6c15d2d0 0x7faf6c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.405+0000 7faf731ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6c0c3f40 con 0x7faf6c1598d0 2023-10-06T05:21:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.405+0000 7faf731ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf6c0b2860 con 0x7faf6c159500 2023-10-06T05:21:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.405+0000 7faf731ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf6c15d7d0 con 0x7faf6c15d2d0 2023-10-06T05:21:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.405+0000 7faf70f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf6c159500 0x7faf6c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.405+0000 7faf70f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf6c159500 0x7faf6c0b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47506/0 (socket says 172.21.15.31:47506) 2023-10-06T05:21:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.405+0000 7faf70f9b700 1 -- 172.21.15.31:0/2832094154 learned_addr learned my addr 172.21.15.31:0/2832094154 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.406+0000 7faf6bfff700 1 --2- 172.21.15.31:0/2832094154 >> v2:172.21.15.62:3300/0 conn(0x7faf6c15d2d0 0x7faf6c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.406+0000 7faf70f9b700 1 -- 172.21.15.31:0/2832094154 >> v2:172.21.15.62:3300/0 conn(0x7faf6c15d2d0 msgr2=0x7faf6c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.406+0000 7faf70f9b700 1 --2- 172.21.15.31:0/2832094154 >> v2:172.21.15.62:3300/0 conn(0x7faf6c15d2d0 0x7faf6c0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.406+0000 7faf70f9b700 1 -- 172.21.15.31:0/2832094154 >> v1:172.21.15.62:6789/0 conn(0x7faf6c1598d0 legacy=0x7faf6c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.406+0000 7faf70f9b700 1 -- 172.21.15.31:0/2832094154 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c0cb000 con 0x7faf6c159500 2023-10-06T05:21:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.406+0000 7faf70f9b700 1 --2- 172.21.15.31:0/2832094154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c159500 0x7faf6c0b0ab0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faf6c15d200 tx=0x7faf60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa30a1b334f7e27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.406+0000 7faf6b7fe700 1 -- 172.21.15.31:0/2832094154 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000b860 con 0x7faf6c159500 2023-10-06T05:21:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.406+0000 7faf6b7fe700 1 -- 172.21.15.31:0/2832094154 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6000b9c0 con 0x7faf6c159500 2023-10-06T05:21:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.406+0000 7faf6b7fe700 1 -- 172.21.15.31:0/2832094154 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000bd60 con 0x7faf6c159500 2023-10-06T05:21:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.407+0000 7faf731ff700 1 -- 172.21.15.31:0/2832094154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c159500 msgr2=0x7faf6c0b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.407+0000 7faf731ff700 1 --2- 172.21.15.31:0/2832094154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c159500 0x7faf6c0b0ab0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faf6c15d200 tx=0x7faf60009250 comp rx=0 tx=0).stop 2023-10-06T05:21:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.407+0000 7faf731ff700 1 -- 172.21.15.31:0/2832094154 shutdown_connections 2023-10-06T05:21:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.407+0000 7faf731ff700 1 --2- 172.21.15.31:0/2832094154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c159500 0x7faf6c0b0ab0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.407+0000 7faf731ff700 1 --2- 172.21.15.31:0/2832094154 >> v2:172.21.15.62:3300/0 conn(0x7faf6c15d2d0 0x7faf6c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.407+0000 7faf731ff700 1 -- 172.21.15.31:0/2832094154 >> 172.21.15.31:0/2832094154 conn(0x7faf6c06ec90 msgr2=0x7faf6c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.407+0000 7faf731ff700 1 -- 172.21.15.31:0/2832094154 shutdown_connections 2023-10-06T05:21:33.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.407+0000 7faf731ff700 1 -- 172.21.15.31:0/2832094154 wait complete. 2023-10-06T05:21:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.408+0000 7faf731ff700 1 Processor -- start 2023-10-06T05:21:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.409+0000 7faf731ff700 1 -- start start 2023-10-06T05:21:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.409+0000 7faf731ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6c159500 0x7faf6c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.409+0000 7faf731ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c1598d0 0x7faf6c0b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf731ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6c15d2d0 0x7faf6c156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf731ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c15beb0 con 0x7faf6c1598d0 2023-10-06T05:21:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf731ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c15ba20 con 0x7faf6c159500 2023-10-06T05:21:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf731ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0b6190 con 0x7faf6c15d2d0 2023-10-06T05:21:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c1598d0 0x7faf6c0b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c1598d0 0x7faf6c0b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47516/0 (socket says 172.21.15.31:47516) 2023-10-06T05:21:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf6bfff700 1 -- 172.21.15.31:0/2547362549 learned_addr learned my addr 172.21.15.31:0/2547362549 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf70f9b700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6c159500 0x7faf6c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf7179c700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6c15d2d0 0x7faf6c156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf6bfff700 1 -- 172.21.15.31:0/2547362549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6c15d2d0 msgr2=0x7faf6c156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf6bfff700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6c15d2d0 0x7faf6c156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf6bfff700 1 -- 172.21.15.31:0/2547362549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6c159500 msgr2=0x7faf6c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf6bfff700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6c159500 0x7faf6c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:33.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf6bfff700 1 -- 172.21.15.31:0/2547362549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600094f0 con 0x7faf6c1598d0 2023-10-06T05:21:33.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.410+0000 7faf6bfff700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c1598d0 0x7faf6c0b5c50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faf5800d870 tx=0x7faf58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.411+0000 7faf68ff9700 1 -- 172.21.15.31:0/2547362549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58014620 con 0x7faf6c1598d0 2023-10-06T05:21:33.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.411+0000 7faf68ff9700 1 -- 172.21.15.31:0/2547362549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf58014780 con 0x7faf6c1598d0 2023-10-06T05:21:33.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.411+0000 7faf68ff9700 1 -- 172.21.15.31:0/2547362549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58014b40 con 0x7faf6c1598d0 2023-10-06T05:21:33.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.411+0000 7faf731ff700 1 -- 172.21.15.31:0/2547362549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c15ae10 con 0x7faf6c1598d0 2023-10-06T05:21:33.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.411+0000 7faf731ff700 1 -- 172.21.15.31:0/2547362549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c15b3d0 con 0x7faf6c1598d0 2023-10-06T05:21:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.412+0000 7faf731ff700 1 -- 172.21.15.31:0/2547362549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c15bbb0 con 0x7faf6c1598d0 2023-10-06T05:21:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.413+0000 7faf68ff9700 1 -- 172.21.15.31:0/2547362549 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 85046+0+0 (secure 0 0 0) 0x7faf5801d400 con 0x7faf6c1598d0 2023-10-06T05:21:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.413+0000 7faf68ff9700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf50067290 0x7faf50069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.414+0000 7faf68ff9700 1 -- 172.21.15.31:0/2547362549 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faf580853b0 con 0x7faf6c1598d0 2023-10-06T05:21:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.414+0000 7faf70f9b700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf50067290 0x7faf50069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.414+0000 7faf70f9b700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf50067290 0x7faf50069740 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faf6c15d200 tx=0x7faf60017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:33.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.418+0000 7faf68ff9700 1 -- 172.21.15.31:0/2547362549 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf58052070 con 0x7faf6c1598d0 2023-10-06T05:21:33.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.675+0000 7faf731ff700 1 -- 172.21.15.31:0/2547362549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7faf6c04c380 con 0x7faf6c1598d0 2023-10-06T05:21:33.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.682+0000 7faf68ff9700 1 -- 172.21.15.31:0/2547362549 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v10) v1 ==== 81+0+450695 (secure 0 0 0) 0x7faf58055690 con 0x7faf6c1598d0 2023-10-06T05:21:33.690 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:33.690 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:21:33.690 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 10, 2023-10-06T05:21:33.690 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 4104, 2023-10-06T05:21:33.690 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "y", 2023-10-06T05:21:33.690 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2023-10-06T05:21:33.690 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:33.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6832", 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 114556 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6833", 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 114556 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6833/114556", 2023-10-06T05:21:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2023-10-06T05:16:48.048502+0000", 2023-10-06T05:21:33.692 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540138320759226367, 2023-10-06T05:21:33.692 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2023-10-06T05:21:33.692 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2023-10-06T05:21:33.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.692 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 4101, 2023-10-06T05:21:33.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "x", 2023-10-06T05:21:33.692 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540138320759226367, 2023-10-06T05:21:33.692 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-10-06T05:21:33.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-10-06T05:21:33.701 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.702 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-10-06T05:21:33.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-10-06T05:21:33.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.712 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T05:21:33.712 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T05:21:33.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.712 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-10-06T05:21:33.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "7", 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T05:21:33.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-10-06T05:21:33.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2400", 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T05:21:33.718 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "7", 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.720 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.721 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.723 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.723 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.723 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-10-06T05:21:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-10-06T05:21:33.727 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.729 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-10-06T05:21:33.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.731 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-10-06T05:21:33.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.734 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.737 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.738 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.738 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T05:21:33.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T05:21:33.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.738 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-10-06T05:21:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T05:21:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-10-06T05:21:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.740 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-10-06T05:21:33.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.743 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T05:21:33.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T05:21:33.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.749 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.1.5", 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.750 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-10-06T05:21:33.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-10-06T05:21:33.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-10-06T05:21:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-10-06T05:21:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T05:21:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-10-06T05:21:33.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-10-06T05:21:33.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.768 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.770 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.772 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T05:21:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T05:21:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-10-06T05:21:33.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.782 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.786 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-10-06T05:21:33.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-10-06T05:21:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-10-06T05:21:33.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-10-06T05:21:33.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T05:21:33.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.796 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.799 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.800 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.801 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-10-06T05:21:33.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:33.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T05:21:33.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-10-06T05:21:33.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T05:21:33.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T05:21:33.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.818 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.819 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.823 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T05:21:33.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.825 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.831 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T05:21:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T05:21:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T05:21:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T05:21:33.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.839 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T05:21:33.841 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T05:21:33.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.841 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.841 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.842 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.842 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.842 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.842 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.842 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.842 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.842 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T05:21:33.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T05:21:33.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.851 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.852 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.853 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T05:21:33.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:33.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.856 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.857 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-10-06T05:21:33.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-10-06T05:21:33.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.857 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.857 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.859 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.859 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.859 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-10-06T05:21:33.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-10-06T05:21:33.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.859 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-10-06T05:21:33.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.866 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.867 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T05:21:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-10-06T05:21:33.877 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T05:21:33.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-10-06T05:21:33.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T05:21:33.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.896 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.897 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.900 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.901 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-10-06T05:21:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.906 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-10-06T05:21:33.906 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-10-06T05:21:33.906 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-10-06T05:21:33.906 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.906 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.912 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.915 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.917 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.918 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.920 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T05:21:33.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T05:21:33.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.920 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.920 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-10-06T05:21:33.920 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T05:21:33.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.921 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-10-06T05:21:33.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-10-06T05:21:33.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-10-06T05:21:33.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T05:21:33.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.926 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.929 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.930 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.930 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.930 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.930 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.930 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.930 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.930 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.931 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-10-06T05:21:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.932 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.933 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.933 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.934 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.934 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.934 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.934 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.934 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.934 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.934 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.934 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.935 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.935 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.937 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.937 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.937 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.937 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.937 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.937 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.937 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.941 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.942 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.943 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.947 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:33.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.950 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.952 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.953 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.954 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.957 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.964 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-10-06T05:21:33.966 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.967 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.968 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.970 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.971 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T05:21:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-10-06T05:21:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-10-06T05:21:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-10-06T05:21:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T05:21:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T05:21:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T05:21:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-10-06T05:21:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T05:21:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-10-06T05:21:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-10-06T05:21:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-10-06T05:21:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-10-06T05:21:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-10-06T05:21:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-10-06T05:21:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T05:21:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "drive_group_interval": { 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "drive_group_interval", 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300.0", 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.162 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-10-06T05:21:34.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-10-06T05:21:34.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.164 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-10-06T05:21:34.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-10-06T05:21:34.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-10-06T05:21:34.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T05:21:34.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-10-06T05:21:34.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.176 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-10-06T05:21:34.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-10-06T05:21:34.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.177 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.177 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.178 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-10-06T05:21:34.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-10-06T05:21:34.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.180 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-10-06T05:21:34.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-10-06T05:21:34.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-10-06T05:21:34.182 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.183 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.183 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.183 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-10-06T05:21:34.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-10-06T05:21:34.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.186 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.189 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.191 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.191 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.191 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.191 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.191 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.191 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.192 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.192 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-10-06T05:21:34.195 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.196 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.196 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.196 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.197 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.197 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.198 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.198 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.198 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.198 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.198 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.199 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.201 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.202 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.202 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.202 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.202 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.202 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.202 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.203 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.204 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-10-06T05:21:34.206 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.207 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.207 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.207 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.210 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.211 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.211 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.211 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.211 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.211 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.211 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.212 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.214 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.215 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T05:21:34.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-10-06T05:21:34.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.222 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.223 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.223 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.223 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.223 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.223 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.225 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-10-06T05:21:34.229 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.230 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.230 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.230 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-10-06T05:21:34.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-10-06T05:21:34.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T05:21:34.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.232 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-10-06T05:21:34.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-10-06T05:21:34.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T05:21:34.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.233 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-10-06T05:21:34.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-10-06T05:21:34.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T05:21:34.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.236 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-10-06T05:21:34.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-10-06T05:21:34.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T05:21:34.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.239 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-10-06T05:21:34.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-10-06T05:21:34.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.241 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-10-06T05:21:34.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-10-06T05:21:34.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.242 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-10-06T05:21:34.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-10-06T05:21:34.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T05:21:34.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T05:21:34.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T05:21:34.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.244 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.246 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-10-06T05:21:34.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-10-06T05:21:34.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-10-06T05:21:34.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.251 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.251 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.251 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.251 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.251 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.251 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.251 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.254 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.256 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.257 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.258 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.258 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.258 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.258 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.258 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.258 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.259 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.262 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-10-06T05:21:34.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-10-06T05:21:34.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-10-06T05:21:34.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-10-06T05:21:34.266 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.266 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.266 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.267 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.270 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.272 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.272 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.274 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.277 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-10-06T05:21:34.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-10-06T05:21:34.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-10-06T05:21:34.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T05:21:34.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T05:21:34.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-10-06T05:21:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-10-06T05:21:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-10-06T05:21:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T05:21:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-10-06T05:21:34.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-10-06T05:21:34.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.286 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T05:21:34.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T05:21:34.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T05:21:34.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.288 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.289 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.290 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.290 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.290 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.290 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.290 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.290 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.291 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.293 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.295 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.295 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.295 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.295 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.296 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.296 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.296 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-10-06T05:21:34.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T05:21:34.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.302 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.302 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2023-10-06T05:21:34.302 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2023-10-06T05:21:34.302 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2023-10-06T05:21:34.302 INFO:teuthology.orchestra.run.smithi031.stdout: "restful" 2023-10-06T05:21:34.303 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.303 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-10-06T05:21:34.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-10-06T05:21:34.303 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.303 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.303 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.303 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T05:21:34.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T05:21:34.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.305 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.306 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.306 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.306 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.306 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.306 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.306 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.306 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.310 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-10-06T05:21:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.312 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-10-06T05:21:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-10-06T05:21:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-10-06T05:21:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-10-06T05:21:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.314 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-10-06T05:21:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-10-06T05:21:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-10-06T05:21:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-10-06T05:21:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-10-06T05:21:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-10-06T05:21:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-10-06T05:21:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-10-06T05:21:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-10-06T05:21:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-10-06T05:21:34.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-10-06T05:21:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-10-06T05:21:34.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-10-06T05:21:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.323 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-10-06T05:21:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-10-06T05:21:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T05:21:34.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.325 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-10-06T05:21:34.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-10-06T05:21:34.327 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-10-06T05:21:34.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T05:21:34.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T05:21:34.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.330 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-10-06T05:21:34.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-10-06T05:21:34.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T05:21:34.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T05:21:34.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T05:21:34.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "7", 2023-10-06T05:21:34.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-10-06T05:21:34.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-10-06T05:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-10-06T05:21:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-10-06T05:21:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2400", 2023-10-06T05:21:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T05:21:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T05:21:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-10-06T05:21:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-10-06T05:21:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T05:21:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2023-10-06T05:21:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-10-06T05:21:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "7", 2023-10-06T05:21:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T05:21:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T05:21:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.341 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.341 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.341 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.341 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.341 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.342 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.342 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.343 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.345 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.346 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.346 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.346 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.346 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.346 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.347 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.347 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-10-06T05:21:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-10-06T05:21:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T05:21:34.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T05:21:34.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.350 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T05:21:34.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T05:21:34.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-10-06T05:21:34.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.352 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-10-06T05:21:34.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-10-06T05:21:34.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.353 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T05:21:34.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T05:21:34.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-10-06T05:21:34.357 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.358 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.358 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.358 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-10-06T05:21:34.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-10-06T05:21:34.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T05:21:34.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-10-06T05:21:34.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-10-06T05:21:34.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T05:21:34.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-10-06T05:21:34.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T05:21:34.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.367 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-10-06T05:21:34.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-10-06T05:21:34.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T05:21:34.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.370 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-10-06T05:21:34.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-10-06T05:21:34.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T05:21:34.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.372 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-10-06T05:21:34.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-10-06T05:21:34.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T05:21:34.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T05:21:34.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.373 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-10-06T05:21:34.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-10-06T05:21:34.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T05:21:34.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-10-06T05:21:34.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.375 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-10-06T05:21:34.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-10-06T05:21:34.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T05:21:34.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T05:21:34.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.377 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-10-06T05:21:34.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-10-06T05:21:34.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-10-06T05:21:34.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.1.5", 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.379 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-10-06T05:21:34.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-10-06T05:21:34.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.396 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:34.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-10-06T05:21:34.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-10-06T05:21:34.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.414 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T05:21:34.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-10-06T05:21:34.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-10-06T05:21:34.432 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T05:21:34.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T05:21:34.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T05:21:34.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T05:21:34.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T05:21:34.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T05:21:34.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T05:21:34.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.472 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-10-06T05:21:34.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T05:21:34.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T05:21:34.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.480 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.480 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.482 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T05:21:34.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T05:21:34.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-10-06T05:21:34.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.490 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.491 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.492 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.494 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.502 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.504 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-10-06T05:21:34.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.510 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-10-06T05:21:34.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.517 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.520 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.522 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.524 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.531 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-10-06T05:21:34.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-10-06T05:21:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-10-06T05:21:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-10-06T05:21:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T05:21:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-10-06T05:21:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-10-06T05:21:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T05:21:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-10-06T05:21:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-10-06T05:21:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-10-06T05:21:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-10-06T05:21:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.630 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.636 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.638 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.641 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.643 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-10-06T05:21:34.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.654 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T05:21:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-10-06T05:21:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "drive_group_interval": { 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "drive_group_interval", 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300.0", 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-10-06T05:21:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-10-06T05:21:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-10-06T05:21:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-10-06T05:21:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-10-06T05:21:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-10-06T05:21:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-10-06T05:21:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T05:21:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-10-06T05:21:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-10-06T05:21:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-10-06T05:21:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-10-06T05:21:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-10-06T05:21:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-10-06T05:21:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-10-06T05:21:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T05:21:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T05:21:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T05:21:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-10-06T05:21:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-10-06T05:21:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-10-06T05:21:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-10-06T05:21:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T05:21:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-10-06T05:21:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-10-06T05:21:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T05:21:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-10-06T05:21:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-10-06T05:21:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-10-06T05:21:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T05:21:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-10-06T05:21:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-10-06T05:21:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T05:21:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-10-06T05:21:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-10-06T05:21:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.852 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.852 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.852 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T05:21:34.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.853 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-10-06T05:21:34.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-10-06T05:21:34.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.854 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.856 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.856 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-10-06T05:21:34.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-10-06T05:21:34.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.857 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.859 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.859 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.860 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-10-06T05:21:34.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-10-06T05:21:34.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.860 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.860 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T05:21:34.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.861 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-10-06T05:21:34.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-10-06T05:21:34.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T05:21:34.863 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T05:21:34.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-10-06T05:21:34.864 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-10-06T05:21:34.864 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.864 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.865 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-10-06T05:21:34.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-10-06T05:21:34.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-10-06T05:21:34.867 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.867 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.867 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.867 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.868 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-10-06T05:21:34.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-10-06T05:21:34.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.870 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.873 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.873 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.874 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.874 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.874 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.874 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.874 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.875 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.877 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.880 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-10-06T05:21:34.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-10-06T05:21:34.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.881 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-10-06T05:21:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-10-06T05:21:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.883 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-10-06T05:21:34.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-10-06T05:21:34.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T05:21:34.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-10-06T05:21:34.885 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.885 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.886 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.886 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.886 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.886 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.887 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.888 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.888 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.888 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.888 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.888 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.888 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.891 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.892 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.893 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.896 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.897 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.897 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.898 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.898 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.898 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.898 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.898 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.898 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.899 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.901 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.902 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.902 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.902 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.903 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.903 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.903 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.903 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.903 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.903 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.903 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.904 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-10-06T05:21:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-10-06T05:21:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-10-06T05:21:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T05:21:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-10-06T05:21:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-10-06T05:21:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-10-06T05:21:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T05:21:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-10-06T05:21:34.911 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-10-06T05:21:34.911 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-10-06T05:21:34.911 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-10-06T05:21:34.911 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-10-06T05:21:34.912 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-10-06T05:21:34.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-10-06T05:21:34.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-10-06T05:21:34.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.913 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.913 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.913 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.914 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-10-06T05:21:34.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-10-06T05:21:34.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.916 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-10-06T05:21:34.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-10-06T05:21:34.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-10-06T05:21:34.916 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-10-06T05:21:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.918 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.918 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.918 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-10-06T05:21:34.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-10-06T05:21:34.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.919 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.919 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.920 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.920 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.920 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.920 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.920 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.920 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.920 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.921 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-10-06T05:21:34.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-10-06T05:21:34.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.922 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-10-06T05:21:34.923 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-10-06T05:21:34.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-10-06T05:21:34.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-10-06T05:21:34.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: "services": {}, 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-10-06T05:21:34.929 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2023-10-06T05:21:34.930 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2023-10-06T05:21:34.931 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-10-06T05:21:34.932 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-10-06T05:21:34.932 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-10-06T05:21:34.932 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-10-06T05:21:34.932 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-10-06T05:21:34.932 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-10-06T05:21:34.932 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-10-06T05:21:34.932 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-10-06T05:21:34.932 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-10-06T05:21:34.932 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-10-06T05:21:34.933 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.933 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 0, 2023-10-06T05:21:34.933 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2023-10-06T05:21:34.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-10-06T05:21:34.933 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:34.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1652493878 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3647032433 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T05:21:34.935 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1563541111 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2247400373 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2357011998 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 514347988 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 990299348 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1729374487 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:34.940 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1049444580 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-10-06T05:21:34.941 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3570608959 2023-10-06T05:21:34.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.942 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:34.942 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:34.942 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:21:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 -- 172.21.15.31:0/2547362549 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf50067290 msgr2=0x7faf50069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf50067290 0x7faf50069740 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faf6c15d200 tx=0x7faf60017000 comp rx=0 tx=0).stop 2023-10-06T05:21:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 -- 172.21.15.31:0/2547362549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c1598d0 msgr2=0x7faf6c0b5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c1598d0 0x7faf6c0b5c50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faf5800d870 tx=0x7faf58008040 comp rx=0 tx=0).stop 2023-10-06T05:21:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 -- 172.21.15.31:0/2547362549 shutdown_connections 2023-10-06T05:21:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf6c15d2d0 0x7faf6c156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf6c159500 0x7faf6c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7faf50067290 0x7faf50069740 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 --2- 172.21.15.31:0/2547362549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf6c1598d0 0x7faf6c0b5c50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 -- 172.21.15.31:0/2547362549 >> 172.21.15.31:0/2547362549 conn(0x7faf6c06ec90 msgr2=0x7faf6c0c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 -- 172.21.15.31:0/2547362549 shutdown_connections 2023-10-06T05:21:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:33.687+0000 7faf731ff700 1 -- 172.21.15.31:0/2547362549 wait complete. 2023-10-06T05:21:34.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail y 2023-10-06T05:21:35.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.157+0000 7f81061ff700 1 Processor -- start 2023-10-06T05:21:35.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.157+0000 7f81061ff700 1 -- start start 2023-10-06T05:21:35.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.158+0000 7f81061ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f81000c7b60 0x7f8100069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.158+0000 7f81061ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 0x7f810013f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.159+0000 7f81061ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81000ca5b0 con 0x7f81000c7f30 2023-10-06T05:21:35.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.159+0000 7f81061ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81000af2d0 con 0x7f81000c7b60 2023-10-06T05:21:35.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.159+0000 7f81061ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810006f740 con 0x7f81000aee70 2023-10-06T05:21:35.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.159+0000 7f80feffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f81000c7f30 0x7f810006ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47600/0 (socket says 172.21.15.31:47600) 2023-10-06T05:21:35.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.159+0000 7f80feffd700 1 -- 172.21.15.31:0/500451603 learned_addr learned my addr 172.21.15.31:0/500451603 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:35.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.159+0000 7f80fffff700 1 --2- 172.21.15.31:0/500451603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 0x7f810013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.159+0000 7f80ff7fe700 1 --2- 172.21.15.31:0/500451603 >> v2:172.21.15.62:3300/0 conn(0x7f81000c7b60 0x7f8100069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.159+0000 7f80fe7fc700 1 -- 172.21.15.31:0/500451603 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117784394 0 0) 0x7f81000ca5b0 con 0x7f81000c7f30 2023-10-06T05:21:35.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.159+0000 7f80fe7fc700 1 -- 172.21.15.31:0/500451603 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80e8003570 con 0x7f81000c7f30 2023-10-06T05:21:35.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.160+0000 7f80fffff700 1 -- 172.21.15.31:0/500451603 >> v2:172.21.15.62:3300/0 conn(0x7f81000c7b60 msgr2=0x7f8100069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:35.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.160+0000 7f80fffff700 1 --2- 172.21.15.31:0/500451603 >> v2:172.21.15.62:3300/0 conn(0x7f81000c7b60 0x7f8100069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.160+0000 7f80fffff700 1 -- 172.21.15.31:0/500451603 >> v1:172.21.15.31:6789/0 conn(0x7f81000c7f30 legacy=0x7f810006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.160+0000 7f80fffff700 1 -- 172.21.15.31:0/500451603 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81000c9c80 con 0x7f81000aee70 2023-10-06T05:21:35.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.160+0000 7f80ff7fe700 1 --2- 172.21.15.31:0/500451603 >> v2:172.21.15.62:3300/0 conn(0x7f81000c7b60 0x7f8100069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:35.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.160+0000 7f80fffff700 1 --2- 172.21.15.31:0/500451603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 0x7f810013f690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009e60 tx=0x7f80f8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79b13660dc42ab8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.160+0000 7f80fe7fc700 1 -- 172.21.15.31:0/500451603 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f800e070 con 0x7f81000aee70 2023-10-06T05:21:35.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.160+0000 7f80fe7fc700 1 -- 172.21.15.31:0/500451603 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f800bab0 con 0x7f81000aee70 2023-10-06T05:21:35.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.160+0000 7f80fe7fc700 1 -- 172.21.15.31:0/500451603 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f800be70 con 0x7f81000aee70 2023-10-06T05:21:35.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.161+0000 7f81061ff700 1 -- 172.21.15.31:0/500451603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 msgr2=0x7f810013f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.161+0000 7f81061ff700 1 --2- 172.21.15.31:0/500451603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 0x7f810013f690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009e60 tx=0x7f80f8009780 comp rx=0 tx=0).stop 2023-10-06T05:21:35.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.161+0000 7f81061ff700 1 -- 172.21.15.31:0/500451603 shutdown_connections 2023-10-06T05:21:35.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.161+0000 7f81061ff700 1 --2- 172.21.15.31:0/500451603 >> v2:172.21.15.62:3300/0 conn(0x7f81000c7b60 0x7f8100069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.161+0000 7f81061ff700 1 --2- 172.21.15.31:0/500451603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 0x7f810013f690 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.161+0000 7f81061ff700 1 -- 172.21.15.31:0/500451603 >> 172.21.15.31:0/500451603 conn(0x7f81000cc0b0 msgr2=0x7f81000cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.161+0000 7f81061ff700 1 -- 172.21.15.31:0/500451603 shutdown_connections 2023-10-06T05:21:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.161+0000 7f81061ff700 1 -- 172.21.15.31:0/500451603 wait complete. 2023-10-06T05:21:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.162+0000 7f81061ff700 1 Processor -- start 2023-10-06T05:21:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.162+0000 7f81061ff700 1 -- start start 2023-10-06T05:21:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.163+0000 7f81061ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 0x7f81001698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.163+0000 7f81061ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81000c7b60 0x7f810016c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.163+0000 7f81061ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81000c7f30 0x7f810016d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.163+0000 7f81061ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810016cde0 con 0x7f81000c7b60 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.163+0000 7f81061ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810016dec0 con 0x7f81000c7f30 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.163+0000 7f81061ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81000b0f50 con 0x7f81000aee70 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81000c7b60 0x7f810016c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81000c7b60 0x7f810016c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47518/0 (socket says 172.21.15.31:47518) 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80feffd700 1 -- 172.21.15.31:0/2516384058 learned_addr learned my addr 172.21.15.31:0/2516384058 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80ff7fe700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 0x7f81001698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80fffff700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81000c7f30 0x7f810016d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80feffd700 1 -- 172.21.15.31:0/2516384058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 msgr2=0x7f81001698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80feffd700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 0x7f81001698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80feffd700 1 -- 172.21.15.31:0/2516384058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81000c7f30 msgr2=0x7f810016d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80feffd700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81000c7f30 0x7f810016d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80feffd700 1 -- 172.21.15.31:0/2516384058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f00093c0 con 0x7f81000c7b60 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80ff7fe700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 0x7f81001698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.164+0000 7f80feffd700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81000c7b60 0x7f810016c8a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f80f400c5c0 tx=0x7f80f4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.165+0000 7f80e77fe700 1 -- 172.21.15.31:0/2516384058 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f4005b30 con 0x7f81000c7b60 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.165+0000 7f80e77fe700 1 -- 172.21.15.31:0/2516384058 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f4005cb0 con 0x7f81000c7b60 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.165+0000 7f81061ff700 1 -- 172.21.15.31:0/2516384058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f8009a20 con 0x7f81000c7b60 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.165+0000 7f80e77fe700 1 -- 172.21.15.31:0/2516384058 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f4007320 con 0x7f81000c7b60 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.165+0000 7f81061ff700 1 -- 172.21.15.31:0/2516384058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810016a1a0 con 0x7f81000c7b60 2023-10-06T05:21:35.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.166+0000 7f81061ff700 1 -- 172.21.15.31:0/2516384058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810004c380 con 0x7f81000c7b60 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.167+0000 7f80e77fe700 1 -- 172.21.15.31:0/2516384058 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f80f401b3b0 con 0x7f81000c7b60 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.168+0000 7f80e77fe700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80ec067320 0x7f80ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.168+0000 7f80e77fe700 1 -- 172.21.15.31:0/2516384058 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f80f4084380 con 0x7f81000c7b60 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.168+0000 7f80ff7fe700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80ec067320 0x7f80ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.168+0000 7f80ff7fe700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80ec067320 0x7f80ec0697d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009390 tx=0x7f80f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.172+0000 7f80e77fe700 1 -- 172.21.15.31:0/2516384058 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80f4050e70 con 0x7f81000c7b60 2023-10-06T05:21:35.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.433+0000 7f81061ff700 1 -- 172.21.15.31:0/2516384058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr fail", "who": "y"} v 0) v1 -- 0x7f81000c9f60 con 0x7f81000c7b60 2023-10-06T05:21:35.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.439+0000 7f80e77fe700 1 -- 172.21.15.31:0/2516384058 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 12) v1 ==== 42252+0+0 (secure 0 0 0) 0x7f80f40840d0 con 0x7f81000c7b60 2023-10-06T05:21:35.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.439+0000 7f80e77fe700 1 -- 172.21.15.31:0/2516384058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80ec067320 msgr2=0x7f80ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.439+0000 7f80e77fe700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80ec067320 0x7f80ec0697d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009390 tx=0x7f80f0008040 comp rx=0 tx=0).stop 2023-10-06T05:21:35.442 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:35.440+0000 7efec79f5700 -1 mgr handle_mgr_map I was active but no longer am 2023-10-06T05:21:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.442+0000 7f80e77fe700 1 -- 172.21.15.31:0/2516384058 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "mgr fail", "who": "y"}]=0 v12) v1 ==== 68+0+0 (secure 0 0 0) 0x7f80f4052ed0 con 0x7f81000c7b60 2023-10-06T05:21:35.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.445+0000 7f81061ff700 1 -- 172.21.15.31:0/2516384058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81000c7b60 msgr2=0x7f810016c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.445+0000 7f81061ff700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81000c7b60 0x7f810016c8a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f80f400c5c0 tx=0x7f80f4007e00 comp rx=0 tx=0).stop 2023-10-06T05:21:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.446+0000 7f81061ff700 1 -- 172.21.15.31:0/2516384058 shutdown_connections 2023-10-06T05:21:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.446+0000 7f81061ff700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81000c7f30 0x7f810016d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.446+0000 7f81061ff700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f81000aee70 0x7f81001698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.446+0000 7f81061ff700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.31:6832/114556,v1:172.21.15.31:6833/114556] conn(0x7f80ec067320 0x7f80ec0697d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.446+0000 7f81061ff700 1 --2- 172.21.15.31:0/2516384058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81000c7b60 0x7f810016c8a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.446+0000 7f81061ff700 1 -- 172.21.15.31:0/2516384058 >> 172.21.15.31:0/2516384058 conn(0x7f81000cc0b0 msgr2=0x7f81000c1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.446+0000 7f81061ff700 1 -- 172.21.15.31:0/2516384058 shutdown_connections 2023-10-06T05:21:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.446+0000 7f81061ff700 1 -- 172.21.15.31:0/2516384058 wait complete. 2023-10-06T05:21:35.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:35.552 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:21:35.550+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:35.552 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:21:35.550+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:35.552 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:21:35.550+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:35.552 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:21:35.550+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:35.552 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:21:35.550+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:35.553 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:21:35.551+0000 7f2e4a189700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:35.553 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:21:35.551+0000 7f2e4a189700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:35.553 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:21:35.551+0000 7f2e4a189700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:35.553 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:21:35.551+0000 7f2e4a189700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:35.553 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:21:35.551+0000 7f2e4a189700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:21:35.694 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:35.692+0000 7ff5a84761c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:21:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.700+0000 7f3d401ff700 1 Processor -- start 2023-10-06T05:21:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.700+0000 7f3d401ff700 1 -- start start 2023-10-06T05:21:35.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.701+0000 7f3d401ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d38159040 0x7f3d38159480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d401ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d380c77f0 con 0x7f3d3815d3f0 2023-10-06T05:21:35.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d401ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d380a9ab0 con 0x7f3d3815d7c0 2023-10-06T05:21:35.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d401ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380ab260 con 0x7f3d38159040 2023-10-06T05:21:35.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d3df9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3d3815d3f0 0x7f3d380aab50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47602/0 (socket says 172.21.15.31:47602) 2023-10-06T05:21:35.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d3df9b700 1 -- 172.21.15.31:0/435911072 learned_addr learned my addr 172.21.15.31:0/435911072 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:35.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d3e79c700 1 --2- 172.21.15.31:0/435911072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d38159040 0x7f3d38159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d3cf99700 1 -- 172.21.15.31:0/435911072 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312237082 0 0) 0x7f3d380c77f0 con 0x7f3d3815d3f0 2023-10-06T05:21:35.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d3cf99700 1 -- 172.21.15.31:0/435911072 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d24003570 con 0x7f3d3815d3f0 2023-10-06T05:21:35.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d3cf99700 1 -- 172.21.15.31:0/435911072 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68924876 0 0) 0x7f3d380a9ab0 con 0x7f3d3815d7c0 2023-10-06T05:21:35.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d3cf99700 1 -- 172.21.15.31:0/435911072 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d380c77f0 con 0x7f3d3815d7c0 2023-10-06T05:21:35.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d3e79c700 1 -- 172.21.15.31:0/435911072 >> v1:172.21.15.62:6789/0 conn(0x7f3d3815d7c0 legacy=0x7f3d380c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d3e79c700 1 -- 172.21.15.31:0/435911072 >> v1:172.21.15.31:6789/0 conn(0x7f3d3815d3f0 legacy=0x7f3d380aab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.702+0000 7f3d3e79c700 1 -- 172.21.15.31:0/435911072 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d380a9d90 con 0x7f3d38159040 2023-10-06T05:21:35.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.703+0000 7f3d3e79c700 1 --2- 172.21.15.31:0/435911072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d38159040 0x7f3d38159480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d380c79e0 tx=0x7f3d30009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9e8eb202da908e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.703+0000 7f3d3cf99700 1 -- 172.21.15.31:0/435911072 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30016070 con 0x7f3d38159040 2023-10-06T05:21:35.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.703+0000 7f3d3cf99700 1 -- 172.21.15.31:0/435911072 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3000bab0 con 0x7f3d38159040 2023-10-06T05:21:35.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.703+0000 7f3d3cf99700 1 -- 172.21.15.31:0/435911072 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3000e070 con 0x7f3d38159040 2023-10-06T05:21:35.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.704+0000 7f3d401ff700 1 -- 172.21.15.31:0/435911072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d38159040 msgr2=0x7f3d38159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.704+0000 7f3d401ff700 1 --2- 172.21.15.31:0/435911072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d38159040 0x7f3d38159480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d380c79e0 tx=0x7f3d30009780 comp rx=0 tx=0).stop 2023-10-06T05:21:35.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.704+0000 7f3d401ff700 1 -- 172.21.15.31:0/435911072 shutdown_connections 2023-10-06T05:21:35.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.704+0000 7f3d401ff700 1 --2- 172.21.15.31:0/435911072 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d38159040 0x7f3d38159480 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.704+0000 7f3d401ff700 1 -- 172.21.15.31:0/435911072 >> 172.21.15.31:0/435911072 conn(0x7f3d380cc0b0 msgr2=0x7f3d380cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:35.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.704+0000 7f3d401ff700 1 -- 172.21.15.31:0/435911072 shutdown_connections 2023-10-06T05:21:35.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.704+0000 7f3d401ff700 1 -- 172.21.15.31:0/435911072 wait complete. 2023-10-06T05:21:35.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.706+0000 7f3d401ff700 1 Processor -- start 2023-10-06T05:21:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.706+0000 7f3d401ff700 1 -- start start 2023-10-06T05:21:35.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.707+0000 7f3d401ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d38159040 0x7f3d38153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d401ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d3815d3f0 0x7f3d380afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d401ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d3815d7c0 0x7f3d380c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:35.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d401ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380c5d10 con 0x7f3d3815d3f0 2023-10-06T05:21:35.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d401ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380c3200 con 0x7f3d38159040 2023-10-06T05:21:35.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d401ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380b1a60 con 0x7f3d3815d7c0 2023-10-06T05:21:35.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d38159040 0x7f3d38153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d3815d3f0 0x7f3d380afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3e79c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d3815d7c0 0x7f3d380c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:35.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d38159040 0x7f3d38153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:57816/0 (socket says 172.21.15.31:57816) 2023-10-06T05:21:35.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d3815d3f0 0x7f3d380afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47542/0 (socket says 172.21.15.31:47542) 2023-10-06T05:21:35.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3df9b700 1 -- 172.21.15.31:0/2985461344 learned_addr learned my addr 172.21.15.31:0/2985461344 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:35.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3df9b700 1 -- 172.21.15.31:0/2985461344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d3815d7c0 msgr2=0x7f3d380c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3df9b700 1 --2- 172.21.15.31:0/2985461344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d3815d7c0 0x7f3d380c3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3df9b700 1 -- 172.21.15.31:0/2985461344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d3815d3f0 msgr2=0x7f3d380afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:35.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3df9b700 1 --2- 172.21.15.31:0/2985461344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d3815d3f0 0x7f3d380afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:35.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3df9b700 1 -- 172.21.15.31:0/2985461344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d30009a20 con 0x7f3d38159040 2023-10-06T05:21:35.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3d79a700 1 --2- 172.21.15.31:0/2985461344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d3815d3f0 0x7f3d380afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:35.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.708+0000 7f3d3e79c700 1 --2- 172.21.15.31:0/2985461344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d3815d7c0 0x7f3d380c3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:35.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.709+0000 7f3d3df9b700 1 --2- 172.21.15.31:0/2985461344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d38159040 0x7f3d38153ce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c790 tx=0x7f3d34007ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:35.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.709+0000 7f3d2dffb700 1 -- 172.21.15.31:0/2985461344 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340072e0 con 0x7f3d38159040 2023-10-06T05:21:35.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.709+0000 7f3d2dffb700 1 -- 172.21.15.31:0/2985461344 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d34007440 con 0x7f3d38159040 2023-10-06T05:21:35.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.709+0000 7f3d401ff700 1 -- 172.21.15.31:0/2985461344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d380c34c0 con 0x7f3d38159040 2023-10-06T05:21:35.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.709+0000 7f3d2dffb700 1 -- 172.21.15.31:0/2985461344 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340057f0 con 0x7f3d38159040 2023-10-06T05:21:35.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.709+0000 7f3d401ff700 1 -- 172.21.15.31:0/2985461344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d380b1d40 con 0x7f3d38159040 2023-10-06T05:21:35.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.710+0000 7f3d401ff700 1 -- 172.21.15.31:0/2985461344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d3804c380 con 0x7f3d38159040 2023-10-06T05:21:35.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.710+0000 7f3d2dffb700 1 -- 172.21.15.31:0/2985461344 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 42252+0+0 (secure 0 0 0) 0x7f3d34005be0 con 0x7f3d38159040 2023-10-06T05:21:35.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.711+0000 7f3d2dffb700 1 -- 172.21.15.31:0/2985461344 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f3d340473b0 con 0x7f3d38159040 2023-10-06T05:21:35.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:35.716+0000 7f3d2dffb700 1 -- 172.21.15.31:0/2985461344 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d3400ea40 con 0x7f3d38159040 2023-10-06T05:21:35.896 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:35.894+0000 7ff5a84761c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:21:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.004+0000 7f3d401ff700 1 -- 172.21.15.31:0/2985461344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d3806b360 con 0x7f3d38159040 2023-10-06T05:21:36.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.004+0000 7f3d2dffb700 1 -- 172.21.15.31:0/2985461344 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f3d34017070 con 0x7f3d38159040 2023-10-06T05:21:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2151449662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2151449662},{"type":"v1","addr":"172.21.15.62:6834","nonce":2151449662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/810564890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":810564890},{"type":"v1","addr":"172.21.15.31:6836","nonce":810564890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/802676715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":802676715},{"type":"v1","addr":"172.21.15.31:6837","nonce":802676715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:13.661482+0000","modified":"2023-10-06T05:21:17.173141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6249":{"gid":6249,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/934773709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":934773709},{"type":"v1","addr":"172.21.15.62:6835","nonce":934773709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:36.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.007+0000 7f3d401ff700 1 -- 172.21.15.31:0/2985461344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d38159040 msgr2=0x7f3d38153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.007+0000 7f3d401ff700 1 --2- 172.21.15.31:0/2985461344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d38159040 0x7f3d38153ce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c790 tx=0x7f3d34007ce0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.007+0000 7f3d401ff700 1 -- 172.21.15.31:0/2985461344 shutdown_connections 2023-10-06T05:21:36.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.007+0000 7f3d401ff700 1 --2- 172.21.15.31:0/2985461344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d3815d7c0 0x7f3d380c3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.007+0000 7f3d401ff700 1 --2- 172.21.15.31:0/2985461344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d38159040 0x7f3d38153ce0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.007+0000 7f3d401ff700 1 --2- 172.21.15.31:0/2985461344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d3815d3f0 0x7f3d380afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.007+0000 7f3d401ff700 1 -- 172.21.15.31:0/2985461344 >> 172.21.15.31:0/2985461344 conn(0x7f3d380cc0b0 msgr2=0x7f3d38154b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:36.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.007+0000 7f3d401ff700 1 -- 172.21.15.31:0/2985461344 shutdown_connections 2023-10-06T05:21:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.007+0000 7f3d401ff700 1 -- 172.21.15.31:0/2985461344 wait complete. 2023-10-06T05:21:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 45 2023-10-06T05:21:36.027 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2023-10-06T05:21:36.227 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:36.225+0000 7ff5a84761c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:21:36.249 INFO:teuthology.orchestra.run.smithi062.stdout:[{"id":5825,"entity":{"name":{"type":"client","num":5825},"addr":{"type":"any","addr":"172.21.15.62:0","nonce":1638150644}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.67698983199999996,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.996897156679792,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5825 172.21.15.62:0/1638150644","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"x","hostname":"smithi062","pid":"114414","root":"/"}},{"id":6531,"entity":{"name":{"type":"client","num":6531},"addr":{"type":"any","addr":"172.21.15.62:0","nonce":255392389}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.67798981700000005,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.996899463550887,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6531 172.21.15.62:0/255392389","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564","ceph_version":"ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)","entity_id":"x","hostname":"smithi062","pid":"114414","root":"/"}}] 2023-10-06T05:21:36.249 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "x", "hostname": "smithi062", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114414", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.62:0", "nonce": 1638150644, "type": "any" }, "name": { "num": 5825, "type": "client" } }, "id": 5825, "inst": "client.5825 172.21.15.62:0/1638150644", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.996897156679792 }, "state": "open", "uptime": 0.676989832 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "x", "hostname": "smithi062", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "114414", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.62:0", "nonce": 255392389, "type": "any" }, "name": { "num": 6531, "type": "client" } }, "id": 6531, "inst": "client.6531 172.21.15.62:0/255392389", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.996899463550887 }, "state": "open", "uptime": 0.677989817 } ] 2023-10-06T05:21:36.250 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-06T05:21:36.250 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:21:36.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:21:36.358 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:21:36.453 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:21:36.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:36.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.651+0000 7f336dbff700 1 Processor -- start 2023-10-06T05:21:36.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.652+0000 7f336dbff700 1 -- start start 2023-10-06T05:21:36.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.653+0000 7f336dbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f336813a5b0 0x7f336806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.653+0000 7f336dbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336806d980 0x7f3368069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.653+0000 7f336dbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336806ddb0 con 0x7f336813a1e0 2023-10-06T05:21:36.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.653+0000 7f336dbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33680c71b0 con 0x7f336813a5b0 2023-10-06T05:21:36.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.653+0000 7f336dbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33680c7490 con 0x7f336806d980 2023-10-06T05:21:36.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.653+0000 7f33677fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f336813a1e0 0x7f33680c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47606/0 (socket says 172.21.15.31:47606) 2023-10-06T05:21:36.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.653+0000 7f33677fe700 1 -- 172.21.15.31:0/643144909 learned_addr learned my addr 172.21.15.31:0/643144909 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:36.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.653+0000 7f3366ffd700 1 --2- 172.21.15.31:0/643144909 >> v2:172.21.15.62:3300/0 conn(0x7f336813a5b0 0x7f336806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.653+0000 7f3367fff700 1 --2- 172.21.15.31:0/643144909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336806d980 0x7f3368069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.654+0000 7f33667fc700 1 -- 172.21.15.31:0/643144909 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177768922 0 0) 0x7f336806ddb0 con 0x7f336813a1e0 2023-10-06T05:21:36.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.654+0000 7f33667fc700 1 -- 172.21.15.31:0/643144909 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3350003570 con 0x7f336813a1e0 2023-10-06T05:21:36.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.654+0000 7f3367fff700 1 -- 172.21.15.31:0/643144909 >> v2:172.21.15.62:3300/0 conn(0x7f336813a5b0 msgr2=0x7f336806d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.654+0000 7f3367fff700 1 --2- 172.21.15.31:0/643144909 >> v2:172.21.15.62:3300/0 conn(0x7f336813a5b0 0x7f336806d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.654+0000 7f3367fff700 1 -- 172.21.15.31:0/643144909 >> v1:172.21.15.31:6789/0 conn(0x7f336813a1e0 legacy=0x7f33680c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.654+0000 7f3367fff700 1 -- 172.21.15.31:0/643144909 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33680c7770 con 0x7f336806d980 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.654+0000 7f3366ffd700 1 --2- 172.21.15.31:0/643144909 >> v2:172.21.15.62:3300/0 conn(0x7f336813a5b0 0x7f336806d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.654+0000 7f3367fff700 1 --2- 172.21.15.31:0/643144909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336806d980 0x7f3368069560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3360009e60 tx=0x7f3360009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e269048f2fb70ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.654+0000 7f33667fc700 1 -- 172.21.15.31:0/643144909 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336000e070 con 0x7f336806d980 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.655+0000 7f33667fc700 1 -- 172.21.15.31:0/643144909 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336000bab0 con 0x7f336806d980 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.655+0000 7f33667fc700 1 -- 172.21.15.31:0/643144909 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336000be70 con 0x7f336806d980 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.655+0000 7f336dbff700 1 -- 172.21.15.31:0/643144909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336806d980 msgr2=0x7f3368069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.655+0000 7f336dbff700 1 --2- 172.21.15.31:0/643144909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336806d980 0x7f3368069560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3360009e60 tx=0x7f3360009780 comp rx=0 tx=0).stop 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.655+0000 7f336dbff700 1 -- 172.21.15.31:0/643144909 shutdown_connections 2023-10-06T05:21:36.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.655+0000 7f336dbff700 1 --2- 172.21.15.31:0/643144909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336806d980 0x7f3368069560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.655+0000 7f336dbff700 1 --2- 172.21.15.31:0/643144909 >> v2:172.21.15.62:3300/0 conn(0x7f336813a5b0 0x7f336806d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.655+0000 7f336dbff700 1 -- 172.21.15.31:0/643144909 >> 172.21.15.31:0/643144909 conn(0x7f33680cc0b0 msgr2=0x7f33680cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.655+0000 7f336dbff700 1 -- 172.21.15.31:0/643144909 shutdown_connections 2023-10-06T05:21:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.655+0000 7f336dbff700 1 -- 172.21.15.31:0/643144909 wait complete. 2023-10-06T05:21:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.657+0000 7f336dbff700 1 Processor -- start 2023-10-06T05:21:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.657+0000 7f336dbff700 1 -- start start 2023-10-06T05:21:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f336dbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f336806d980 0x7f3368157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f336dbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336813a1e0 0x7f3368131150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f336dbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336813a5b0 0x7f3368131690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f336dbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33680c5a10 con 0x7f336813a1e0 2023-10-06T05:21:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f336dbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3368156080 con 0x7f336806d980 2023-10-06T05:21:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f336dbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3368155750 con 0x7f336813a5b0 2023-10-06T05:21:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f3366ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336813a1e0 0x7f3368131150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f3366ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336813a1e0 0x7f3368131150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47558/0 (socket says 172.21.15.31:47558) 2023-10-06T05:21:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f3366ffd700 1 -- 172.21.15.31:0/1394593913 learned_addr learned my addr 172.21.15.31:0/1394593913 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f33677fe700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f336806d980 0x7f3368157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f3367fff700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336813a5b0 0x7f3368131690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.658+0000 7f3366ffd700 1 -- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336813a5b0 msgr2=0x7f3368131690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.659+0000 7f3366ffd700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336813a5b0 0x7f3368131690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.659+0000 7f3366ffd700 1 -- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f336806d980 msgr2=0x7f3368157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.659+0000 7f3366ffd700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f336806d980 0x7f3368157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.659+0000 7f3366ffd700 1 -- 172.21.15.31:0/1394593913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c009420 con 0x7f336813a1e0 2023-10-06T05:21:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.659+0000 7f3366ffd700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336813a1e0 0x7f3368131150 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f335c00ec50 tx=0x7f335c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.659+0000 7f334f7fe700 1 -- 172.21.15.31:0/1394593913 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c00c850 con 0x7f336813a1e0 2023-10-06T05:21:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.659+0000 7f334f7fe700 1 -- 172.21.15.31:0/1394593913 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c00c9b0 con 0x7f336813a1e0 2023-10-06T05:21:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.659+0000 7f334f7fe700 1 -- 172.21.15.31:0/1394593913 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c00cd50 con 0x7f336813a1e0 2023-10-06T05:21:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.659+0000 7f336dbff700 1 -- 172.21.15.31:0/1394593913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3360009a20 con 0x7f336813a1e0 2023-10-06T05:21:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.659+0000 7f336dbff700 1 -- 172.21.15.31:0/1394593913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3368155d10 con 0x7f336813a1e0 2023-10-06T05:21:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.660+0000 7f334f7fe700 1 -- 172.21.15.31:0/1394593913 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f335c013680 con 0x7f336813a1e0 2023-10-06T05:21:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.660+0000 7f336dbff700 1 -- 172.21.15.31:0/1394593913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3368156210 con 0x7f336813a1e0 2023-10-06T05:21:36.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.661+0000 7f334f7fe700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3354035900 0x7f3354037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:36.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.661+0000 7f334f7fe700 1 -- 172.21.15.31:0/1394593913 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f335c0488f0 con 0x7f336813a1e0 2023-10-06T05:21:36.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.661+0000 7f33677fe700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3354035900 0x7f3354037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:36.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.662+0000 7f33677fe700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3354035900 0x7f3354037db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3358001360 tx=0x7f3358009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.666+0000 7f334f7fe700 1 -- 172.21.15.31:0/1394593913 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f335c0123d0 con 0x7f336813a1e0 2023-10-06T05:21:36.790 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:36.787+0000 7ff5a84761c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:21:36.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.912+0000 7f336dbff700 1 -- 172.21.15.31:0/1394593913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f336804c380 con 0x7f336813a1e0 2023-10-06T05:21:36.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.913+0000 7f334f7fe700 1 -- 172.21.15.31:0/1394593913 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f335c011350 con 0x7f336813a1e0 2023-10-06T05:21:36.916 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2151449662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2151449662},{"type":"v1","addr":"172.21.15.62:6834","nonce":2151449662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/810564890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":810564890},{"type":"v1","addr":"172.21.15.31:6836","nonce":810564890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/802676715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":802676715},{"type":"v1","addr":"172.21.15.31:6837","nonce":802676715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:13.661482+0000","modified":"2023-10-06T05:21:17.173141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6249":{"gid":6249,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/934773709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":934773709},{"type":"v1","addr":"172.21.15.62:6835","nonce":934773709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 -- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3354035900 msgr2=0x7f3354037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3354035900 0x7f3354037db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3358001360 tx=0x7f3358009040 comp rx=0 tx=0).stop 2023-10-06T05:21:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 -- 172.21.15.31:0/1394593913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336813a1e0 msgr2=0x7f3368131150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336813a1e0 0x7f3368131150 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f335c00ec50 tx=0x7f335c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 -- 172.21.15.31:0/1394593913 shutdown_connections 2023-10-06T05:21:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3354035900 0x7f3354037db0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f336813a5b0 0x7f3368131690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f336806d980 0x7f3368157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 --2- 172.21.15.31:0/1394593913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336813a1e0 0x7f3368131150 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 -- 172.21.15.31:0/1394593913 >> 172.21.15.31:0/1394593913 conn(0x7f33680cc0b0 msgr2=0x7f33680c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 -- 172.21.15.31:0/1394593913 shutdown_connections 2023-10-06T05:21:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:36.915+0000 7f336dbff700 1 -- 172.21.15.31:0/1394593913 wait complete. 2023-10-06T05:21:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 45 2023-10-06T05:21:36.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:36.968 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:36.965+0000 7ff5a84761c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:21:37.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.133+0000 7f1e2ffff700 1 Processor -- start 2023-10-06T05:21:37.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.133+0000 7f1e2ffff700 1 -- start start 2023-10-06T05:21:37.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.135+0000 7f1e2ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1e2813a1e0 0x7f1e280c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.135+0000 7f1e2ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e2813a5b0 0x7f1e2806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.135+0000 7f1e2ffff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e280c71b0 con 0x7f1e280c6c30 2023-10-06T05:21:37.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.135+0000 7f1e2ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e280c7490 con 0x7f1e2813a5b0 2023-10-06T05:21:37.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.135+0000 7f1e2ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e280c7770 con 0x7f1e2813a1e0 2023-10-06T05:21:37.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.135+0000 7f1e2d59a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e2813a5b0 0x7f1e2806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.135+0000 7f1e2d59a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e2813a5b0 0x7f1e2806d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58832/0 (socket says 172.21.15.31:58832) 2023-10-06T05:21:37.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.135+0000 7f1e2d59a700 1 -- 172.21.15.31:0/2627397108 learned_addr learned my addr 172.21.15.31:0/2627397108 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:37.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.135+0000 7f1e2dd9b700 1 --2- 172.21.15.31:0/2627397108 >> v2:172.21.15.62:3300/0 conn(0x7f1e2813a1e0 0x7f1e280c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.136+0000 7f1e2d59a700 1 -- 172.21.15.31:0/2627397108 >> v2:172.21.15.62:3300/0 conn(0x7f1e2813a1e0 msgr2=0x7f1e280c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.136+0000 7f1e2d59a700 1 --2- 172.21.15.31:0/2627397108 >> v2:172.21.15.62:3300/0 conn(0x7f1e2813a1e0 0x7f1e280c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.136+0000 7f1e2d59a700 1 -- 172.21.15.31:0/2627397108 >> v1:172.21.15.62:6789/0 conn(0x7f1e280c6c30 legacy=0x7f1e2815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.136+0000 7f1e2d59a700 1 -- 172.21.15.31:0/2627397108 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e280c7a50 con 0x7f1e2813a5b0 2023-10-06T05:21:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.136+0000 7f1e2d59a700 1 --2- 172.21.15.31:0/2627397108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 0x7f1e2806d440 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1e280cbf90 tx=0x7f1e18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1640d4b61b96f716 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.136+0000 7f1e2cd99700 1 -- 172.21.15.31:0/2627397108 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18009e70 con 0x7f1e2813a5b0 2023-10-06T05:21:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.136+0000 7f1e2cd99700 1 -- 172.21.15.31:0/2627397108 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1800b860 con 0x7f1e2813a5b0 2023-10-06T05:21:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.136+0000 7f1e2cd99700 1 -- 172.21.15.31:0/2627397108 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1800bc00 con 0x7f1e2813a5b0 2023-10-06T05:21:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.137+0000 7f1e2ffff700 1 -- 172.21.15.31:0/2627397108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 msgr2=0x7f1e2806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.137+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/2627397108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 0x7f1e2806d440 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1e280cbf90 tx=0x7f1e18009250 comp rx=0 tx=0).stop 2023-10-06T05:21:37.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.137+0000 7f1e2ffff700 1 -- 172.21.15.31:0/2627397108 shutdown_connections 2023-10-06T05:21:37.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.137+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/2627397108 >> v2:172.21.15.62:3300/0 conn(0x7f1e2813a1e0 0x7f1e280c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.137+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/2627397108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 0x7f1e2806d440 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.137+0000 7f1e2ffff700 1 -- 172.21.15.31:0/2627397108 >> 172.21.15.31:0/2627397108 conn(0x7f1e280cc0b0 msgr2=0x7f1e280cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.137+0000 7f1e2ffff700 1 -- 172.21.15.31:0/2627397108 shutdown_connections 2023-10-06T05:21:37.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.137+0000 7f1e2ffff700 1 -- 172.21.15.31:0/2627397108 wait complete. 2023-10-06T05:21:37.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.138+0000 7f1e2ffff700 1 Processor -- start 2023-10-06T05:21:37.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.139+0000 7f1e2ffff700 1 -- start start 2023-10-06T05:21:37.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.139+0000 7f1e2ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e280c6c30 0x7f1e2813c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.139+0000 7f1e2ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2813a1e0 0x7f1e280cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 0x7f1e280b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e280af600 con 0x7f1e2813a5b0 2023-10-06T05:21:37.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e280aefd0 con 0x7f1e2813a1e0 2023-10-06T05:21:37.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2813cbb0 con 0x7f1e280c6c30 2023-10-06T05:21:37.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 0x7f1e280b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 0x7f1e280b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58848/0 (socket says 172.21.15.31:58848) 2023-10-06T05:21:37.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2e59c700 1 -- 172.21.15.31:0/1602375008 learned_addr learned my addr 172.21.15.31:0/1602375008 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:37.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2d59a700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2813a1e0 0x7f1e280cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2dd9b700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e280c6c30 0x7f1e2813c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2e59c700 1 -- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e280c6c30 msgr2=0x7f1e2813c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2e59c700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e280c6c30 0x7f1e2813c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2e59c700 1 -- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2813a1e0 msgr2=0x7f1e280cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2e59c700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2813a1e0 0x7f1e280cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2e59c700 1 -- 172.21.15.31:0/1602375008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e180094f0 con 0x7f1e2813a5b0 2023-10-06T05:21:37.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.140+0000 7f1e2dd9b700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e280c6c30 0x7f1e2813c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:37.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.141+0000 7f1e2e59c700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 0x7f1e280b13e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000c8e0 tx=0x7f1e20001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.141+0000 7f1e1dffb700 1 -- 172.21.15.31:0/1602375008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20012630 con 0x7f1e2813a5b0 2023-10-06T05:21:37.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.141+0000 7f1e1dffb700 1 -- 172.21.15.31:0/1602375008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20012790 con 0x7f1e2813a5b0 2023-10-06T05:21:37.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.141+0000 7f1e1dffb700 1 -- 172.21.15.31:0/1602375008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20012b50 con 0x7f1e2813a5b0 2023-10-06T05:21:37.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.141+0000 7f1e2ffff700 1 -- 172.21.15.31:0/1602375008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e280af290 con 0x7f1e2813a5b0 2023-10-06T05:21:37.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.141+0000 7f1e2ffff700 1 -- 172.21.15.31:0/1602375008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e280c1e20 con 0x7f1e2813a5b0 2023-10-06T05:21:37.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.142+0000 7f1e2ffff700 1 -- 172.21.15.31:0/1602375008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2804c380 con 0x7f1e2813a5b0 2023-10-06T05:21:37.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.142+0000 7f1e1dffb700 1 -- 172.21.15.31:0/1602375008 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f1e2000a3b0 con 0x7f1e2813a5b0 2023-10-06T05:21:37.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.143+0000 7f1e1dffb700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e0c035900 0x7f1e0c037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.143+0000 7f1e1dffb700 1 -- 172.21.15.31:0/1602375008 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f1e200488b0 con 0x7f1e2813a5b0 2023-10-06T05:21:37.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.143+0000 7f1e2dd9b700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e0c035900 0x7f1e0c037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.144+0000 7f1e2dd9b700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e0c035900 0x7f1e0c037db0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009d70 tx=0x7f1e24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.148+0000 7f1e1dffb700 1 -- 172.21.15.31:0/1602375008 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e2000a660 con 0x7f1e2813a5b0 2023-10-06T05:21:37.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.355+0000 7f1e2ffff700 1 -- 172.21.15.31:0/1602375008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e280c7d30 con 0x7f1e2813a5b0 2023-10-06T05:21:37.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.356+0000 7f1e1dffb700 1 -- 172.21.15.31:0/1602375008 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+17718 (secure 0 0 0) 0x7f1e200212a0 con 0x7f1e2813a5b0 2023-10-06T05:21:37.359 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:37.359 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:35.434576+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:12.092711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:12.637981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1049444580":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/990299348":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/514347988":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/2247400373":"2023-10-07T05:21:35.434529+0000","172.21.15.31:6833/114556":"2023-10-07T05:21:35.434529+0000","172.21.15.31:6832/114556":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/2357011998":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/1563541111":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/1729374487":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/3570608959":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/1652493878":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/2612034927":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/3647032433":"2023-10-07T05:21:35.434529+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:37.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 -- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e0c035900 msgr2=0x7f1e0c037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e0c035900 0x7f1e0c037db0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009d70 tx=0x7f1e24008040 comp rx=0 tx=0).stop 2023-10-06T05:21:37.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 -- 172.21.15.31:0/1602375008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 msgr2=0x7f1e280b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 0x7f1e280b13e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000c8e0 tx=0x7f1e20001710 comp rx=0 tx=0).stop 2023-10-06T05:21:37.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 -- 172.21.15.31:0/1602375008 shutdown_connections 2023-10-06T05:21:37.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e0c035900 0x7f1e0c037db0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e2813a5b0 0x7f1e280b13e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e280c6c30 0x7f1e2813c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/1602375008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2813a1e0 0x7f1e280cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 -- 172.21.15.31:0/1602375008 >> 172.21.15.31:0/1602375008 conn(0x7f1e280cc0b0 msgr2=0x7f1e28156410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 -- 172.21.15.31:0/1602375008 shutdown_connections 2023-10-06T05:21:37.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.359+0000 7f1e2ffff700 1 -- 172.21.15.31:0/1602375008 wait complete. 2023-10-06T05:21:37.387 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:21:12.092711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:21:12.637981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:21:37.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:21:37.409 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:37.406+0000 7ff5a84761c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:21:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.608+0000 7f0acbdff700 1 Processor -- start 2023-10-06T05:21:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.609+0000 7f0acbdff700 1 -- start start 2023-10-06T05:21:37.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.610+0000 7f0acbdff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ac40b2bf0 0x7f0ac40c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.610+0000 7f0acbdff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac40cbae0 con 0x7f0ac40cce60 2023-10-06T05:21:37.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.610+0000 7f0acbdff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac40cf120 con 0x7f0ac40b2820 2023-10-06T05:21:37.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.610+0000 7f0acbdff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac40cb280 con 0x7f0ac40b2bf0 2023-10-06T05:21:37.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.610+0000 7f0aca39c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0ac40cce60 0x7f0ac40cbf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54296/0 (socket says 172.21.15.31:54296) 2023-10-06T05:21:37.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.610+0000 7f0aca39c700 1 -- 172.21.15.31:0/1273636019 learned_addr learned my addr 172.21.15.31:0/1273636019 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.610+0000 7f0ac939a700 1 --2- 172.21.15.31:0/1273636019 >> v2:172.21.15.62:3300/0 conn(0x7f0ac40b2bf0 0x7f0ac40c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.611+0000 7f0ac8b99700 1 -- 172.21.15.31:0/1273636019 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380629441 0 0) 0x7f0ac40cbae0 con 0x7f0ac40cce60 2023-10-06T05:21:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.611+0000 7f0ac8b99700 1 -- 172.21.15.31:0/1273636019 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab0003570 con 0x7f0ac40cce60 2023-10-06T05:21:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.611+0000 7f0ac8b99700 1 -- 172.21.15.31:0/1273636019 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239216597 0 0) 0x7f0ac40cf120 con 0x7f0ac40b2820 2023-10-06T05:21:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.611+0000 7f0ac8b99700 1 -- 172.21.15.31:0/1273636019 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ac40cbae0 con 0x7f0ac40b2820 2023-10-06T05:21:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.611+0000 7f0ac8b99700 1 -- 172.21.15.31:0/1273636019 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1742182020 0 0) 0x7f0ab0003570 con 0x7f0ac40cce60 2023-10-06T05:21:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.611+0000 7f0ac939a700 1 -- 172.21.15.31:0/1273636019 >> v1:172.21.15.62:6789/0 conn(0x7f0ac40b2820 legacy=0x7f0ac40cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.611+0000 7f0ac939a700 1 -- 172.21.15.31:0/1273636019 >> v1:172.21.15.31:6789/0 conn(0x7f0ac40cce60 legacy=0x7f0ac40cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.611+0000 7f0ac939a700 1 -- 172.21.15.31:0/1273636019 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac40cf3f0 con 0x7f0ac40b2bf0 2023-10-06T05:21:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.611+0000 7f0ac939a700 1 --2- 172.21.15.31:0/1273636019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac40b2bf0 0x7f0ac40c5240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40098e0 tx=0x7f0ab40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d65d0f4195cea94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.611+0000 7f0ac8b99700 1 -- 172.21.15.31:0/1273636019 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0017f0 con 0x7f0ac40b2bf0 2023-10-06T05:21:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.612+0000 7f0ac8b99700 1 -- 172.21.15.31:0/1273636019 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab4011420 con 0x7f0ac40b2bf0 2023-10-06T05:21:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.612+0000 7f0ac8b99700 1 -- 172.21.15.31:0/1273636019 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40117c0 con 0x7f0ac40b2bf0 2023-10-06T05:21:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.612+0000 7f0acbdff700 1 -- 172.21.15.31:0/1273636019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac40b2bf0 msgr2=0x7f0ac40c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.612+0000 7f0acbdff700 1 --2- 172.21.15.31:0/1273636019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac40b2bf0 0x7f0ac40c5240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40098e0 tx=0x7f0ab40091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.612+0000 7f0acbdff700 1 -- 172.21.15.31:0/1273636019 shutdown_connections 2023-10-06T05:21:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.612+0000 7f0acbdff700 1 --2- 172.21.15.31:0/1273636019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac40b2bf0 0x7f0ac40c5240 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.612+0000 7f0acbdff700 1 -- 172.21.15.31:0/1273636019 >> 172.21.15.31:0/1273636019 conn(0x7f0ac40996f0 msgr2=0x7f0ac406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.612+0000 7f0acbdff700 1 -- 172.21.15.31:0/1273636019 shutdown_connections 2023-10-06T05:21:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.612+0000 7f0acbdff700 1 -- 172.21.15.31:0/1273636019 wait complete. 2023-10-06T05:21:37.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.614+0000 7f0acbdff700 1 Processor -- start 2023-10-06T05:21:37.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.614+0000 7f0acbdff700 1 -- start start 2023-10-06T05:21:37.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0acbdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac40b2820 0x7f0ac4155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0acbdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac40b2bf0 0x7f0ac4152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0acbdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac40cce60 0x7f0ac40c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0acbdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac4152e60 con 0x7f0ac40cce60 2023-10-06T05:21:37.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0acbdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac4169790 con 0x7f0ac40b2820 2023-10-06T05:21:37.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0acbdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41699d0 con 0x7f0ac40b2bf0 2023-10-06T05:21:37.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0aca39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac40cce60 0x7f0ac40c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0aca39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac40cce60 0x7f0ac40c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58864/0 (socket says 172.21.15.31:58864) 2023-10-06T05:21:37.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0aca39c700 1 -- 172.21.15.31:0/3256353876 learned_addr learned my addr 172.21.15.31:0/3256353876 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:37.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0ac9b9b700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac40b2820 0x7f0ac4155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.615+0000 7f0ac939a700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac40b2bf0 0x7f0ac4152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0aca39c700 1 -- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac40b2bf0 msgr2=0x7f0ac4152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0aca39c700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac40b2bf0 0x7f0ac4152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0aca39c700 1 -- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac40b2820 msgr2=0x7f0ac4155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0aca39c700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac40b2820 0x7f0ac4155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0aca39c700 1 -- 172.21.15.31:0/3256353876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4009420 con 0x7f0ac40cce60 2023-10-06T05:21:37.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0aca39c700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac40cce60 0x7f0ac40c9160 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0abc001560 tx=0x7f0abc007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0ab9ffb700 1 -- 172.21.15.31:0/3256353876 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc004dc0 con 0x7f0ac40cce60 2023-10-06T05:21:37.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0ab9ffb700 1 -- 172.21.15.31:0/3256353876 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc007150 con 0x7f0ac40cce60 2023-10-06T05:21:37.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0acbdff700 1 -- 172.21.15.31:0/3256353876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac4169cb0 con 0x7f0ac40cce60 2023-10-06T05:21:37.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0ab9ffb700 1 -- 172.21.15.31:0/3256353876 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0074f0 con 0x7f0ac40cce60 2023-10-06T05:21:37.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.616+0000 7f0acbdff700 1 -- 172.21.15.31:0/3256353876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac416a270 con 0x7f0ac40cce60 2023-10-06T05:21:37.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.617+0000 7f0ab9ffb700 1 -- 172.21.15.31:0/3256353876 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f0abc0078a0 con 0x7f0ac40cce60 2023-10-06T05:21:37.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.617+0000 7f0acbdff700 1 -- 172.21.15.31:0/3256353876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac404c380 con 0x7f0ac40cce60 2023-10-06T05:21:37.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.618+0000 7f0ab9ffb700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0aa8035900 0x7f0aa8037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:37.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.618+0000 7f0ab9ffb700 1 -- 172.21.15.31:0/3256353876 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f0abc047b40 con 0x7f0ac40cce60 2023-10-06T05:21:37.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.619+0000 7f0ac9b9b700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0aa8035900 0x7f0aa8037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:37.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.619+0000 7f0ac9b9b700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0aa8035900 0x7f0aa8037db0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0003ae0 tx=0x7f0ac0007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:37.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.623+0000 7f0ab9ffb700 1 -- 172.21.15.31:0/3256353876 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0abc01b360 con 0x7f0ac40cce60 2023-10-06T05:21:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.824+0000 7f0acbdff700 1 -- 172.21.15.31:0/3256353876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0ac40cb490 con 0x7f0ac40cce60 2023-10-06T05:21:37.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.825+0000 7f0ab9ffb700 1 -- 172.21.15.31:0/3256353876 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v72) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0abc018020 con 0x7f0ac40cce60 2023-10-06T05:21:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:21:37.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 -- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0aa8035900 msgr2=0x7f0aa8037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0aa8035900 0x7f0aa8037db0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0003ae0 tx=0x7f0ac0007d40 comp rx=0 tx=0).stop 2023-10-06T05:21:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 -- 172.21.15.31:0/3256353876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac40cce60 msgr2=0x7f0ac40c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac40cce60 0x7f0ac40c9160 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0abc001560 tx=0x7f0abc007d20 comp rx=0 tx=0).stop 2023-10-06T05:21:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 -- 172.21.15.31:0/3256353876 shutdown_connections 2023-10-06T05:21:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0aa8035900 0x7f0aa8037db0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ac40cce60 0x7f0ac40c9160 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac40b2820 0x7f0ac4155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 --2- 172.21.15.31:0/3256353876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac40b2bf0 0x7f0ac4152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 -- 172.21.15.31:0/3256353876 >> 172.21.15.31:0/3256353876 conn(0x7f0ac40996f0 msgr2=0x7f0ac4068320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 -- 172.21.15.31:0/3256353876 shutdown_connections 2023-10-06T05:21:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:37.827+0000 7f0acbdff700 1 -- 172.21.15.31:0/3256353876 wait complete. 2023-10-06T05:21:37.849 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:21:38.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.047+0000 7f64de5ff700 1 Processor -- start 2023-10-06T05:21:38.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.048+0000 7f64de5ff700 1 -- start start 2023-10-06T05:21:38.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.049+0000 7f64de5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f64d80cc340 0x7f64d80b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.049+0000 7f64de5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f64d80cc710 0x7f64d80b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.049+0000 7f64de5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d815a020 con 0x7f64d815a4b0 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.049+0000 7f64de5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d8159cf0 con 0x7f64d80cc340 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.049+0000 7f64de5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d80ccb90 con 0x7f64d80cc710 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.049+0000 7f64d7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f64d80cc340 0x7f64d80b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.049+0000 7f64dcb9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f64d815a4b0 0x7f64d80aecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54304/0 (socket says 172.21.15.31:54304) 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.049+0000 7f64dcb9c700 1 -- 172.21.15.31:0/94437198 learned_addr learned my addr 172.21.15.31:0/94437198 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.049+0000 7f64d7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f64d80cc340 0x7f64d80b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58876/0 (socket says 172.21.15.31:58876) 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.049+0000 7f64d77fe700 1 --2- 172.21.15.31:0/94437198 >> v2:172.21.15.62:3300/0 conn(0x7f64d80cc710 0x7f64d80b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.050+0000 7f64d7fff700 1 -- 172.21.15.31:0/94437198 >> v2:172.21.15.62:3300/0 conn(0x7f64d80cc710 msgr2=0x7f64d80b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.050+0000 7f64d7fff700 1 --2- 172.21.15.31:0/94437198 >> v2:172.21.15.62:3300/0 conn(0x7f64d80cc710 0x7f64d80b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.050+0000 7f64d7fff700 1 -- 172.21.15.31:0/94437198 >> v1:172.21.15.31:6789/0 conn(0x7f64d815a4b0 legacy=0x7f64d80aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.050+0000 7f64d7fff700 1 -- 172.21.15.31:0/94437198 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80b2b40 con 0x7f64d80cc340 2023-10-06T05:21:38.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.050+0000 7f64d6ffd700 1 -- 172.21.15.31:0/94437198 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136940566 0 0) 0x7f64d815a020 con 0x7f64d815a4b0 2023-10-06T05:21:38.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.050+0000 7f64d7fff700 1 --2- 172.21.15.31:0/94437198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc340 0x7f64d80b2540 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098d0 tx=0x7f64c8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dff958535665294 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.050+0000 7f64d6ffd700 1 -- 172.21.15.31:0/94437198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8005c10 con 0x7f64d80cc340 2023-10-06T05:21:38.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.050+0000 7f64d6ffd700 1 -- 172.21.15.31:0/94437198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8005d70 con 0x7f64d80cc340 2023-10-06T05:21:38.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.050+0000 7f64d6ffd700 1 -- 172.21.15.31:0/94437198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8006280 con 0x7f64d80cc340 2023-10-06T05:21:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.051+0000 7f64de5ff700 1 -- 172.21.15.31:0/94437198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc340 msgr2=0x7f64d80b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.051+0000 7f64de5ff700 1 --2- 172.21.15.31:0/94437198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc340 0x7f64d80b2540 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098d0 tx=0x7f64c8009120 comp rx=0 tx=0).stop 2023-10-06T05:21:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.051+0000 7f64de5ff700 1 -- 172.21.15.31:0/94437198 shutdown_connections 2023-10-06T05:21:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.051+0000 7f64de5ff700 1 --2- 172.21.15.31:0/94437198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc340 0x7f64d80b2540 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.051+0000 7f64de5ff700 1 --2- 172.21.15.31:0/94437198 >> v2:172.21.15.62:3300/0 conn(0x7f64d80cc710 0x7f64d80b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.051+0000 7f64de5ff700 1 -- 172.21.15.31:0/94437198 >> 172.21.15.31:0/94437198 conn(0x7f64d8068670 msgr2=0x7f64d8068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.051+0000 7f64de5ff700 1 -- 172.21.15.31:0/94437198 shutdown_connections 2023-10-06T05:21:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.051+0000 7f64de5ff700 1 -- 172.21.15.31:0/94437198 wait complete. 2023-10-06T05:21:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.053+0000 7f64de5ff700 1 Processor -- start 2023-10-06T05:21:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.053+0000 7f64de5ff700 1 -- start start 2023-10-06T05:21:38.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64de5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80cc340 0x7f64d81540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64de5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc710 0x7f64d81503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64de5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d815a4b0 0x7f64d8151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64de5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8150c80 con 0x7f64d80cc710 2023-10-06T05:21:38.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64de5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8150f60 con 0x7f64d815a4b0 2023-10-06T05:21:38.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64de5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8151240 con 0x7f64d80cc340 2023-10-06T05:21:38.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64d77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc710 0x7f64d81503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64d77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc710 0x7f64d81503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58886/0 (socket says 172.21.15.31:58886) 2023-10-06T05:21:38.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64d77fe700 1 -- 172.21.15.31:0/909609317 learned_addr learned my addr 172.21.15.31:0/909609317 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:38.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64d7fff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80cc340 0x7f64d81540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64dcb9c700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d815a4b0 0x7f64d8151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64d77fe700 1 -- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80cc340 msgr2=0x7f64d81540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64d77fe700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80cc340 0x7f64d81540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64d77fe700 1 -- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d815a4b0 msgr2=0x7f64d8151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64d77fe700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d815a4b0 0x7f64d8151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64d77fe700 1 -- 172.21.15.31:0/909609317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c80093c0 con 0x7f64d80cc710 2023-10-06T05:21:38.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.054+0000 7f64d7fff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80cc340 0x7f64d81540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:38.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.055+0000 7f64d77fe700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc710 0x7f64d81503a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00ec80 tx=0x7f64cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.055+0000 7f64bffff700 1 -- 172.21.15.31:0/909609317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00c9d0 con 0x7f64d80cc710 2023-10-06T05:21:38.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.055+0000 7f64bffff700 1 -- 172.21.15.31:0/909609317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc00cb30 con 0x7f64d80cc710 2023-10-06T05:21:38.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.055+0000 7f64bffff700 1 -- 172.21.15.31:0/909609317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc013470 con 0x7f64d80cc710 2023-10-06T05:21:38.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.055+0000 7f64de5ff700 1 -- 172.21.15.31:0/909609317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d8151520 con 0x7f64d80cc710 2023-10-06T05:21:38.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.055+0000 7f64de5ff700 1 -- 172.21.15.31:0/909609317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d8152310 con 0x7f64d80cc710 2023-10-06T05:21:38.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.057+0000 7f64bffff700 1 -- 172.21.15.31:0/909609317 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f64cc013820 con 0x7f64d80cc710 2023-10-06T05:21:38.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.057+0000 7f64de5ff700 1 -- 172.21.15.31:0/909609317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d804c380 con 0x7f64d80cc710 2023-10-06T05:21:38.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.057+0000 7f64bffff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f64c4035900 0x7f64c4037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.057+0000 7f64bffff700 1 -- 172.21.15.31:0/909609317 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f64cc048850 con 0x7f64d80cc710 2023-10-06T05:21:38.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.057+0000 7f64d7fff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f64c4035900 0x7f64c4037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.058+0000 7f64d7fff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f64c4035900 0x7f64c4037db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009390 tx=0x7f64c8016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.062+0000 7f64bffff700 1 -- 172.21.15.31:0/909609317 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64cc0236b0 con 0x7f64d80cc710 2023-10-06T05:21:38.233 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:38.230+0000 7ff5a84761c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:21:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.270+0000 7f64de5ff700 1 -- 172.21.15.31:0/909609317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64d80b2e20 con 0x7f64d80cc710 2023-10-06T05:21:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.270+0000 7f64bffff700 1 -- 172.21.15.31:0/909609317 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64cc0232f0 con 0x7f64d80cc710 2023-10-06T05:21:38.273 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.272+0000 7f64de5ff700 1 -- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f64c4035900 msgr2=0x7f64c4037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.272+0000 7f64de5ff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f64c4035900 0x7f64c4037db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009390 tx=0x7f64c8016000 comp rx=0 tx=0).stop 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.272+0000 7f64de5ff700 1 -- 172.21.15.31:0/909609317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc710 msgr2=0x7f64d81503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.272+0000 7f64de5ff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc710 0x7f64d81503a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00ec80 tx=0x7f64cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.273+0000 7f64de5ff700 1 -- 172.21.15.31:0/909609317 shutdown_connections 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.273+0000 7f64de5ff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f64c4035900 0x7f64c4037db0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.273+0000 7f64de5ff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d815a4b0 0x7f64d8151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.273+0000 7f64de5ff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80cc340 0x7f64d81540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.273+0000 7f64de5ff700 1 --2- 172.21.15.31:0/909609317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d80cc710 0x7f64d81503a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.273+0000 7f64de5ff700 1 -- 172.21.15.31:0/909609317 >> 172.21.15.31:0/909609317 conn(0x7f64d8068670 msgr2=0x7f64d8067ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.273+0000 7f64de5ff700 1 -- 172.21.15.31:0/909609317 shutdown_connections 2023-10-06T05:21:38.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.273+0000 7f64de5ff700 1 -- 172.21.15.31:0/909609317 wait complete. 2023-10-06T05:21:38.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:21:38.326 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:38.323+0000 7ff5a84761c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:21:38.492 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:38.490+0000 7ff5a84761c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:21:38.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.493+0000 7f7ebcdff700 1 Processor -- start 2023-10-06T05:21:38.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.493+0000 7f7ebcdff700 1 -- start start 2023-10-06T05:21:38.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.494+0000 7f7ebcdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80af590 0x7f7eb80c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.494+0000 7f7ebcdff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7eb80c52d0 0x7f7eb80cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.494+0000 7f7ebcdff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb80cb800 con 0x7f7eb80cf110 2023-10-06T05:21:38.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.494+0000 7f7ebcdff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb80cf5b0 con 0x7f7eb80c52d0 2023-10-06T05:21:38.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.494+0000 7f7ebcdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80cb280 con 0x7f7eb80af590 2023-10-06T05:21:38.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.495+0000 7f7eb659c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80af590 0x7f7eb80c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.495+0000 7f7eaffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7eb80c52d0 0x7f7eb80cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.495+0000 7f7eb6d9d700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7eb80cf110 0x7f7eb815d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:40918/0 (socket says 172.21.15.31:40918) 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.495+0000 7f7eb6d9d700 1 -- 172.21.15.31:0/1403548237 learned_addr learned my addr 172.21.15.31:0/1403548237 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.495+0000 7f7eb659c700 1 -- 172.21.15.31:0/1403548237 >> v2:172.21.15.62:3300/0 conn(0x7f7eb80c52d0 msgr2=0x7f7eb80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.495+0000 7f7eb659c700 1 --2- 172.21.15.31:0/1403548237 >> v2:172.21.15.62:3300/0 conn(0x7f7eb80c52d0 0x7f7eb80cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.495+0000 7f7eb659c700 1 -- 172.21.15.31:0/1403548237 >> v1:172.21.15.62:6789/0 conn(0x7f7eb80cf110 legacy=0x7f7eb815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.495+0000 7f7eb659c700 1 -- 172.21.15.31:0/1403548237 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb80c7090 con 0x7f7eb80af590 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.495+0000 7f7eaffff700 1 --2- 172.21.15.31:0/1403548237 >> v2:172.21.15.62:3300/0 conn(0x7f7eb80c52d0 0x7f7eb80cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.496+0000 7f7eb659c700 1 --2- 172.21.15.31:0/1403548237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80af590 0x7f7eb80c6160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80098d0 tx=0x7f7ea8009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e43ec42e18c3a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.496+0000 7f7eb5d9b700 1 -- 172.21.15.31:0/1403548237 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8005c10 con 0x7f7eb80af590 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.496+0000 7f7eb5d9b700 1 -- 172.21.15.31:0/1403548237 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea8005d70 con 0x7f7eb80af590 2023-10-06T05:21:38.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.496+0000 7f7eb5d9b700 1 -- 172.21.15.31:0/1403548237 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8006280 con 0x7f7eb80af590 2023-10-06T05:21:38.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.496+0000 7f7ebcdff700 1 -- 172.21.15.31:0/1403548237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80af590 msgr2=0x7f7eb80c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.496+0000 7f7ebcdff700 1 --2- 172.21.15.31:0/1403548237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80af590 0x7f7eb80c6160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80098d0 tx=0x7f7ea8009120 comp rx=0 tx=0).stop 2023-10-06T05:21:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.497+0000 7f7ebcdff700 1 -- 172.21.15.31:0/1403548237 shutdown_connections 2023-10-06T05:21:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.497+0000 7f7ebcdff700 1 --2- 172.21.15.31:0/1403548237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80af590 0x7f7eb80c6160 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.497+0000 7f7ebcdff700 1 --2- 172.21.15.31:0/1403548237 >> v2:172.21.15.62:3300/0 conn(0x7f7eb80c52d0 0x7f7eb80cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.497+0000 7f7ebcdff700 1 -- 172.21.15.31:0/1403548237 >> 172.21.15.31:0/1403548237 conn(0x7f7eb80996f0 msgr2=0x7f7eb806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.497+0000 7f7ebcdff700 1 -- 172.21.15.31:0/1403548237 shutdown_connections 2023-10-06T05:21:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.497+0000 7f7ebcdff700 1 -- 172.21.15.31:0/1403548237 wait complete. 2023-10-06T05:21:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.498+0000 7f7ebcdff700 1 Processor -- start 2023-10-06T05:21:38.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.498+0000 7f7ebcdff700 1 -- start start 2023-10-06T05:21:38.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.499+0000 7f7ebcdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7eb80af590 0x7f7eb80c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.499+0000 7f7ebcdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb80c52d0 0x7f7eb815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.499+0000 7f7ebcdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80cf110 0x7f7eb8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.499+0000 7f7ebcdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80c9610 con 0x7f7eb80c52d0 2023-10-06T05:21:38.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.499+0000 7f7ebcdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80c8340 con 0x7f7eb80af590 2023-10-06T05:21:38.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.499+0000 7f7ebcdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80c9180 con 0x7f7eb80cf110 2023-10-06T05:21:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eaffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb80c52d0 0x7f7eb815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eb659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7eb80af590 0x7f7eb80c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eaffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb80c52d0 0x7f7eb815c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58892/0 (socket says 172.21.15.31:58892) 2023-10-06T05:21:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eaffff700 1 -- 172.21.15.31:0/2793927613 learned_addr learned my addr 172.21.15.31:0/2793927613 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eb6d9d700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80cf110 0x7f7eb8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eaffff700 1 -- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80cf110 msgr2=0x7f7eb8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eaffff700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80cf110 0x7f7eb8159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eaffff700 1 -- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7eb80af590 msgr2=0x7f7eb80c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eaffff700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7eb80af590 0x7f7eb80c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eaffff700 1 -- 172.21.15.31:0/2793927613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0009420 con 0x7f7eb80c52d0 2023-10-06T05:21:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.500+0000 7f7eaffff700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb80c52d0 0x7f7eb815c260 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000ec80 tx=0x7f7ea00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.501+0000 7f7eae7fc700 1 -- 172.21.15.31:0/2793927613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000c850 con 0x7f7eb80c52d0 2023-10-06T05:21:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.501+0000 7f7eae7fc700 1 -- 172.21.15.31:0/2793927613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea000c9b0 con 0x7f7eb80c52d0 2023-10-06T05:21:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.501+0000 7f7eae7fc700 1 -- 172.21.15.31:0/2793927613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000cd30 con 0x7f7eb80c52d0 2023-10-06T05:21:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.501+0000 7f7ebcdff700 1 -- 172.21.15.31:0/2793927613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea80093c0 con 0x7f7eb80c52d0 2023-10-06T05:21:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.501+0000 7f7ebcdff700 1 -- 172.21.15.31:0/2793927613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb80c8b30 con 0x7f7eb80c52d0 2023-10-06T05:21:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.502+0000 7f7ebcdff700 1 -- 172.21.15.31:0/2793927613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb80c9310 con 0x7f7eb80c52d0 2023-10-06T05:21:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.502+0000 7f7eae7fc700 1 -- 172.21.15.31:0/2793927613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f7ea0013680 con 0x7f7eb80c52d0 2023-10-06T05:21:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.503+0000 7f7eae7fc700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7e98035900 0x7f7e98037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.503+0000 7f7eae7fc700 1 -- 172.21.15.31:0/2793927613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f7ea0048a60 con 0x7f7eb80c52d0 2023-10-06T05:21:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.504+0000 7f7eb659c700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7e98035900 0x7f7e98037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.504+0000 7f7eb659c700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7e98035900 0x7f7e98037db0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009160 tx=0x7f7ea8016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.508+0000 7f7eae7fc700 1 -- 172.21.15.31:0/2793927613 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ea0013930 con 0x7f7eb80c52d0 2023-10-06T05:21:38.586 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:38.583+0000 7ff5a84761c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:21:38.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.711+0000 7f7ebcdff700 1 -- 172.21.15.31:0/2793927613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7eb815cff0 con 0x7f7eb80c52d0 2023-10-06T05:21:38.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.711+0000 7f7eae7fc700 1 -- 172.21.15.31:0/2793927613 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ea001ab10 con 0x7f7eb80c52d0 2023-10-06T05:21:38.714 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:21:38.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 -- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7e98035900 msgr2=0x7f7e98037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7e98035900 0x7f7e98037db0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009160 tx=0x7f7ea8016000 comp rx=0 tx=0).stop 2023-10-06T05:21:38.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 -- 172.21.15.31:0/2793927613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb80c52d0 msgr2=0x7f7eb815c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb80c52d0 0x7f7eb815c260 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000ec80 tx=0x7f7ea00091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 -- 172.21.15.31:0/2793927613 shutdown_connections 2023-10-06T05:21:38.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7e98035900 0x7f7e98037db0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7eb80cf110 0x7f7eb8159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7eb80af590 0x7f7eb80c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 --2- 172.21.15.31:0/2793927613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb80c52d0 0x7f7eb815c260 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 -- 172.21.15.31:0/2793927613 >> 172.21.15.31:0/2793927613 conn(0x7f7eb80996f0 msgr2=0x7f7eb815d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:38.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 -- 172.21.15.31:0/2793927613 shutdown_connections 2023-10-06T05:21:38.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.714+0000 7f7ebcdff700 1 -- 172.21.15.31:0/2793927613 wait complete. 2023-10-06T05:21:38.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:21:38.806 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:38.803+0000 7ff5a84761c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:21:38.907 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:38.904+0000 7ff5a84761c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:21:38.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.976+0000 7f981e9ff700 1 Processor -- start 2023-10-06T05:21:38.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.976+0000 7f981e9ff700 1 -- start start 2023-10-06T05:21:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.979+0000 7f981e9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.979+0000 7f981e9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9818157770 0x7f98180b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.979+0000 7f981e9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9818158cf0 con 0x7f9818157be0 2023-10-06T05:21:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.979+0000 7f981e9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98181580e0 con 0x7f9818157770 2023-10-06T05:21:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.979+0000 7f981e9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98181583c0 con 0x7f98181573a0 2023-10-06T05:21:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.979+0000 7f9817fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.979+0000 7f98177fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9818157770 0x7f98180b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.979+0000 7f9817fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:55574/0 (socket says 172.21.15.31:55574) 2023-10-06T05:21:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.979+0000 7f9817fff700 1 -- 172.21.15.31:0/274467924 learned_addr learned my addr 172.21.15.31:0/274467924 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.980+0000 7f9816ffd700 1 -- 172.21.15.31:0/274467924 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343419327 0 0) 0x7f9818158cf0 con 0x7f9818157be0 2023-10-06T05:21:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.980+0000 7f9816ffd700 1 -- 172.21.15.31:0/274467924 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9800003570 con 0x7f9818157be0 2023-10-06T05:21:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.980+0000 7f98177fe700 1 -- 172.21.15.31:0/274467924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 msgr2=0x7f98180acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.980+0000 7f98177fe700 1 --2- 172.21.15.31:0/274467924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.980+0000 7f98177fe700 1 -- 172.21.15.31:0/274467924 >> v1:172.21.15.62:6789/0 conn(0x7f9818157be0 legacy=0x7f98180ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.980+0000 7f98177fe700 1 -- 172.21.15.31:0/274467924 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98181586a0 con 0x7f9818157770 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.980+0000 7f9817fff700 1 --2- 172.21.15.31:0/274467924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.980+0000 7f98177fe700 1 --2- 172.21.15.31:0/274467924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9818157770 0x7f98180b1260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f980c0098e0 tx=0x7f980c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8bb4270992486e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.980+0000 7f9816ffd700 1 -- 172.21.15.31:0/274467924 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c011630 con 0x7f9818157770 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.981+0000 7f9816ffd700 1 -- 172.21.15.31:0/274467924 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c011790 con 0x7f9818157770 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.981+0000 7f9816ffd700 1 -- 172.21.15.31:0/274467924 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c011b30 con 0x7f9818157770 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.981+0000 7f981e9ff700 1 -- 172.21.15.31:0/274467924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9818157770 msgr2=0x7f98180b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.981+0000 7f981e9ff700 1 --2- 172.21.15.31:0/274467924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9818157770 0x7f98180b1260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f980c0098e0 tx=0x7f980c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.981+0000 7f981e9ff700 1 -- 172.21.15.31:0/274467924 shutdown_connections 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.981+0000 7f981e9ff700 1 --2- 172.21.15.31:0/274467924 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.981+0000 7f981e9ff700 1 --2- 172.21.15.31:0/274467924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9818157770 0x7f98180b1260 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.981+0000 7f981e9ff700 1 -- 172.21.15.31:0/274467924 >> 172.21.15.31:0/274467924 conn(0x7f9818067560 msgr2=0x7f9818067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:38.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.981+0000 7f981e9ff700 1 -- 172.21.15.31:0/274467924 shutdown_connections 2023-10-06T05:21:38.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.981+0000 7f981e9ff700 1 -- 172.21.15.31:0/274467924 wait complete. 2023-10-06T05:21:38.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.983+0000 7f981e9ff700 1 Processor -- start 2023-10-06T05:21:38.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.983+0000 7f981e9ff700 1 -- start start 2023-10-06T05:21:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.984+0000 7f981e9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.984+0000 7f981e9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9818157770 0x7f981806ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.984+0000 7f981e9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9818157be0 0x7f981806e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.984+0000 7f981e9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180b1020 con 0x7f9818157770 2023-10-06T05:21:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.984+0000 7f981e9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180a9e60 con 0x7f9818157be0 2023-10-06T05:21:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.984+0000 7f981e9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98180aa140 con 0x7f98181573a0 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.984+0000 7f98177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9818157770 0x7f981806ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.984+0000 7f98177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9818157770 0x7f981806ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58902/0 (socket says 172.21.15.31:58902) 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.984+0000 7f98177fe700 1 -- 172.21.15.31:0/3463010582 learned_addr learned my addr 172.21.15.31:0/3463010582 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.984+0000 7f9817fff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f981cf9c700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9818157be0 0x7f981806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f98177fe700 1 -- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 msgr2=0x7f98180b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f98177fe700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f98177fe700 1 -- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9818157be0 msgr2=0x7f981806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f98177fe700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9818157be0 0x7f981806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f98177fe700 1 -- 172.21.15.31:0/3463010582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98080094f0 con 0x7f9818157770 2023-10-06T05:21:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f9817fff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f98177fe700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9818157770 0x7f981806ef70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9818064290 tx=0x7f980c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f97fffff700 1 -- 172.21.15.31:0/3463010582 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c011b10 con 0x7f9818157770 2023-10-06T05:21:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f97fffff700 1 -- 172.21.15.31:0/3463010582 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c0253f0 con 0x7f9818157770 2023-10-06T05:21:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f97fffff700 1 -- 172.21.15.31:0/3463010582 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0257b0 con 0x7f9818157770 2023-10-06T05:21:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.985+0000 7f981e9ff700 1 -- 172.21.15.31:0/3463010582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c009420 con 0x7f9818157770 2023-10-06T05:21:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.986+0000 7f981e9ff700 1 -- 172.21.15.31:0/3463010582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98180aa700 con 0x7f9818157770 2023-10-06T05:21:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.987+0000 7f97fffff700 1 -- 172.21.15.31:0/3463010582 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f980c025ba0 con 0x7f9818157770 2023-10-06T05:21:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.987+0000 7f981e9ff700 1 -- 172.21.15.31:0/3463010582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981804c380 con 0x7f9818157770 2023-10-06T05:21:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.987+0000 7f97fffff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9804035900 0x7f9804037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.987+0000 7f97fffff700 1 -- 172.21.15.31:0/3463010582 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f980c049dd0 con 0x7f9818157770 2023-10-06T05:21:38.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.988+0000 7f9817fff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9804035900 0x7f9804037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:38.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.988+0000 7f9817fff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9804035900 0x7f9804037db0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9808006fd0 tx=0x7f9808008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:38.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:38.992+0000 7f97fffff700 1 -- 172.21.15.31:0/3463010582 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f980c00b960 con 0x7f9818157770 2023-10-06T05:21:39.009 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:39.006+0000 7ff5a84761c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:21:39.108 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:39.105+0000 7ff5a84761c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:21:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.196+0000 7f981e9ff700 1 -- 172.21.15.31:0/3463010582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98180b1fb0 con 0x7f9818157770 2023-10-06T05:21:39.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.197+0000 7f97fffff700 1 -- 172.21.15.31:0/3463010582 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7f980c015070 con 0x7f9818157770 2023-10-06T05:21:39.199 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:21:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 -- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9804035900 msgr2=0x7f9804037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9804035900 0x7f9804037db0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9808006fd0 tx=0x7f9808008040 comp rx=0 tx=0).stop 2023-10-06T05:21:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 -- 172.21.15.31:0/3463010582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9818157770 msgr2=0x7f981806ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9818157770 0x7f981806ef70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9818064290 tx=0x7f980c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 -- 172.21.15.31:0/3463010582 shutdown_connections 2023-10-06T05:21:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9804035900 0x7f9804037db0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9818157be0 0x7f981806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98181573a0 0x7f98180b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 --2- 172.21.15.31:0/3463010582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9818157770 0x7f981806ef70 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 -- 172.21.15.31:0/3463010582 >> 172.21.15.31:0/3463010582 conn(0x7f9818067560 msgr2=0x7f98180ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 -- 172.21.15.31:0/3463010582 shutdown_connections 2023-10-06T05:21:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.199+0000 7f981e9ff700 1 -- 172.21.15.31:0/3463010582 wait complete. 2023-10-06T05:21:39.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:39.308 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:39.306+0000 7ff5a84761c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:21:39.412 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:39.409+0000 7ff5a84761c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:21:39.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.457+0000 7fa3b7bff700 1 Processor -- start 2023-10-06T05:21:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.457+0000 7fa3b7bff700 1 -- start start 2023-10-06T05:21:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.458+0000 7fa3b7bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b00cd510 0x7fa3b00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.458+0000 7fa3b7bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b00c3f40 con 0x7fa3b0159500 2023-10-06T05:21:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.458+0000 7fa3b7bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b00cb000 con 0x7fa3b015d2d0 2023-10-06T05:21:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.458+0000 7fa3b7bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00b2860 con 0x7fa3b00cd510 2023-10-06T05:21:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b599b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa3b0159500 0x7fa3b00cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54314/0 (socket says 172.21.15.31:54314) 2023-10-06T05:21:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b599b700 1 -- 172.21.15.31:0/881359862 learned_addr learned my addr 172.21.15.31:0/881359862 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b519a700 1 --2- 172.21.15.31:0/881359862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b00cd510 0x7fa3b00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b4999700 1 -- 172.21.15.31:0/881359862 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795848770 0 0) 0x7fa3b00c3f40 con 0x7fa3b0159500 2023-10-06T05:21:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b4999700 1 -- 172.21.15.31:0/881359862 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa39c003570 con 0x7fa3b0159500 2023-10-06T05:21:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b4999700 1 -- 172.21.15.31:0/881359862 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352579271 0 0) 0x7fa3b00cb000 con 0x7fa3b015d2d0 2023-10-06T05:21:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b4999700 1 -- 172.21.15.31:0/881359862 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3b00c3f40 con 0x7fa3b015d2d0 2023-10-06T05:21:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b4999700 1 -- 172.21.15.31:0/881359862 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642043881 0 0) 0x7fa39c003570 con 0x7fa3b0159500 2023-10-06T05:21:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b519a700 1 -- 172.21.15.31:0/881359862 >> v1:172.21.15.62:6789/0 conn(0x7fa3b015d2d0 legacy=0x7fa3b00aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b519a700 1 -- 172.21.15.31:0/881359862 >> v1:172.21.15.31:6789/0 conn(0x7fa3b0159500 legacy=0x7fa3b00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.459+0000 7fa3b519a700 1 -- 172.21.15.31:0/881359862 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b015d7d0 con 0x7fa3b00cd510 2023-10-06T05:21:39.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.460+0000 7fa3b519a700 1 --2- 172.21.15.31:0/881359862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b00cd510 0x7fa3b00b0ab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009900 tx=0x7fa3a0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc28334d75e351b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.460+0000 7fa3b4999700 1 -- 172.21.15.31:0/881359862 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac001470 con 0x7fa3b00cd510 2023-10-06T05:21:39.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.460+0000 7fa3b4999700 1 -- 172.21.15.31:0/881359862 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a0011420 con 0x7fa3b00cd510 2023-10-06T05:21:39.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.460+0000 7fa3b4999700 1 -- 172.21.15.31:0/881359862 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00117c0 con 0x7fa3b00cd510 2023-10-06T05:21:39.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.461+0000 7fa3b7bff700 1 -- 172.21.15.31:0/881359862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b00cd510 msgr2=0x7fa3b00b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.461+0000 7fa3b7bff700 1 --2- 172.21.15.31:0/881359862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b00cd510 0x7fa3b00b0ab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009900 tx=0x7fa3a0009120 comp rx=0 tx=0).stop 2023-10-06T05:21:39.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.461+0000 7fa3b7bff700 1 -- 172.21.15.31:0/881359862 shutdown_connections 2023-10-06T05:21:39.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.461+0000 7fa3b7bff700 1 --2- 172.21.15.31:0/881359862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b00cd510 0x7fa3b00b0ab0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.461+0000 7fa3b7bff700 1 -- 172.21.15.31:0/881359862 >> 172.21.15.31:0/881359862 conn(0x7fa3b006ec90 msgr2=0x7fa3b006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:39.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.461+0000 7fa3b7bff700 1 -- 172.21.15.31:0/881359862 shutdown_connections 2023-10-06T05:21:39.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.461+0000 7fa3b7bff700 1 -- 172.21.15.31:0/881359862 wait complete. 2023-10-06T05:21:39.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.462+0000 7fa3b7bff700 1 Processor -- start 2023-10-06T05:21:39.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.463+0000 7fa3b7bff700 1 -- start start 2023-10-06T05:21:39.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.463+0000 7fa3b7bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b00cd510 0x7fa3b015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.463+0000 7fa3b7bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3b0159500 0x7fa3b015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b7bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b015d2d0 0x7fa3b01698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b7bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00b4cc0 con 0x7fa3b0159500 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b7bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00b43d0 con 0x7fa3b00cd510 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b7bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00b4670 con 0x7fa3b015d2d0 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3b0159500 0x7fa3b015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b519a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3b0159500 0x7fa3b015c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58916/0 (socket says 172.21.15.31:58916) 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b519a700 1 -- 172.21.15.31:0/1970972337 learned_addr learned my addr 172.21.15.31:0/1970972337 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b599b700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b00cd510 0x7fa3b015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b619c700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b015d2d0 0x7fa3b01698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b519a700 1 -- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b015d2d0 msgr2=0x7fa3b01698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b519a700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b015d2d0 0x7fa3b01698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b519a700 1 -- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b00cd510 msgr2=0x7fa3b015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b519a700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b00cd510 0x7fa3b015a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.464+0000 7fa3b519a700 1 -- 172.21.15.31:0/1970972337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a00093c0 con 0x7fa3b0159500 2023-10-06T05:21:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.465+0000 7fa3b519a700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3b0159500 0x7fa3b015c2c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009160 tx=0x7fa3a0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.465+0000 7fa3a5ffb700 1 -- 172.21.15.31:0/1970972337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a001cb30 con 0x7fa3b0159500 2023-10-06T05:21:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.465+0000 7fa3a5ffb700 1 -- 172.21.15.31:0/1970972337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a001cc90 con 0x7fa3b0159500 2023-10-06T05:21:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.465+0000 7fa3b7bff700 1 -- 172.21.15.31:0/1970972337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b00b4950 con 0x7fa3b0159500 2023-10-06T05:21:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.465+0000 7fa3a5ffb700 1 -- 172.21.15.31:0/1970972337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a001a6a0 con 0x7fa3b0159500 2023-10-06T05:21:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.465+0000 7fa3b7bff700 1 -- 172.21.15.31:0/1970972337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b015a830 con 0x7fa3b0159500 2023-10-06T05:21:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.466+0000 7fa3b7bff700 1 -- 172.21.15.31:0/1970972337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b004c380 con 0x7fa3b0159500 2023-10-06T05:21:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.466+0000 7fa3a5ffb700 1 -- 172.21.15.31:0/1970972337 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7fa3a001aa90 con 0x7fa3b0159500 2023-10-06T05:21:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.467+0000 7fa3a5ffb700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa394035900 0x7fa394037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.467+0000 7fa3a5ffb700 1 -- 172.21.15.31:0/1970972337 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fa3a0049cd0 con 0x7fa3b0159500 2023-10-06T05:21:39.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.468+0000 7fa3b599b700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa394035900 0x7fa394037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.468+0000 7fa3b599b700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa394035900 0x7fa394037db0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0011e0 tx=0x7fa3ac001620 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.472+0000 7fa3a5ffb700 1 -- 172.21.15.31:0/1970972337 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3a001ce00 con 0x7fa3b0159500 2023-10-06T05:21:39.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.719+0000 7fa3b7bff700 1 -- 172.21.15.31:0/1970972337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3b00b4e00 con 0x7fa3b0159500 2023-10-06T05:21:39.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.719+0000 7fa3a5ffb700 1 -- 172.21.15.31:0/1970972337 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4590 (secure 0 0 0) 0x7fa3a0019730 con 0x7fa3b0159500 2023-10-06T05:21:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:39.722 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2151449662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2151449662},{"type":"v1","addr":"172.21.15.62:6834","nonce":2151449662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/810564890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":810564890},{"type":"v1","addr":"172.21.15.31:6836","nonce":810564890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/802676715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":802676715},{"type":"v1","addr":"172.21.15.31:6837","nonce":802676715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:13.661482+0000","modified":"2023-10-06T05:21:17.173141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6249":{"gid":6249,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/934773709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":934773709},{"type":"v1","addr":"172.21.15.62:6835","nonce":934773709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:39.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 -- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa394035900 msgr2=0x7fa394037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa394035900 0x7fa394037db0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0011e0 tx=0x7fa3ac001620 comp rx=0 tx=0).stop 2023-10-06T05:21:39.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 -- 172.21.15.31:0/1970972337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3b0159500 msgr2=0x7fa3b015c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3b0159500 0x7fa3b015c2c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009160 tx=0x7fa3a0016000 comp rx=0 tx=0).stop 2023-10-06T05:21:39.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 -- 172.21.15.31:0/1970972337 shutdown_connections 2023-10-06T05:21:39.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa394035900 0x7fa394037db0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3b015d2d0 0x7fa3b01698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b00cd510 0x7fa3b015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 --2- 172.21.15.31:0/1970972337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3b0159500 0x7fa3b015c2c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 -- 172.21.15.31:0/1970972337 >> 172.21.15.31:0/1970972337 conn(0x7fa3b006ec90 msgr2=0x7fa3b00ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:39.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 -- 172.21.15.31:0/1970972337 shutdown_connections 2023-10-06T05:21:39.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.722+0000 7fa3b7bff700 1 -- 172.21.15.31:0/1970972337 wait complete. 2023-10-06T05:21:39.727 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 45 2023-10-06T05:21:39.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:39.764 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:39.762+0000 7ff5a84761c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:21:39.857 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:39.854+0000 7ff5a84761c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:21:39.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.938+0000 7fc09f9ff700 1 Processor -- start 2023-10-06T05:21:39.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.938+0000 7fc09f9ff700 1 -- start start 2023-10-06T05:21:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.939+0000 7fc09f9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc09813a5b0 0x7fc09806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.939+0000 7fc09f9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09806ddb0 con 0x7fc09813a1e0 2023-10-06T05:21:39.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.940+0000 7fc09f9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0980c71b0 con 0x7fc09806d980 2023-10-06T05:21:39.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.940+0000 7fc09f9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0980c7490 con 0x7fc09813a5b0 2023-10-06T05:21:39.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.940+0000 7fc09d79b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc09813a1e0 0x7fc0980c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54324/0 (socket says 172.21.15.31:54324) 2023-10-06T05:21:39.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.940+0000 7fc09d79b700 1 -- 172.21.15.31:0/1138986494 learned_addr learned my addr 172.21.15.31:0/1138986494 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:39.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.940+0000 7fc09cf9a700 1 --2- 172.21.15.31:0/1138986494 >> v2:172.21.15.62:3300/0 conn(0x7fc09813a5b0 0x7fc09806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.940+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857482922 0 0) 0x7fc09806ddb0 con 0x7fc09813a1e0 2023-10-06T05:21:39.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.940+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc084003570 con 0x7fc09813a1e0 2023-10-06T05:21:39.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.940+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847032009 0 0) 0x7fc0980c71b0 con 0x7fc09806d980 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.940+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc09806ddb0 con 0x7fc09806d980 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.940+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 801093855 0 0) 0x7fc084003570 con 0x7fc09813a1e0 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 >> v2:172.21.15.62:3300/0 conn(0x7fc09813a5b0 msgr2=0x7fc09806d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 --2- 172.21.15.31:0/1138986494 >> v2:172.21.15.62:3300/0 conn(0x7fc09813a5b0 0x7fc09806d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 >> v1:172.21.15.62:6789/0 conn(0x7fc09806d980 legacy=0x7fc098069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980c7770 con 0x7fc09813a1e0 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fc0940015f0 con 0x7fc09813a1e0 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 >> v1:172.21.15.31:6789/0 conn(0x7fc09813a1e0 legacy=0x7fc0980c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc084003c40 0x7fc084004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc084004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc084005210 0x7fc08400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980c7490 con 0x7fc084004810 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc084010540 con 0x7fc084003c40 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840106d0 con 0x7fc084005210 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc09cf9a700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc084004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc09df9c700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc084005210 0x7fc08400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc09d79b700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc084003c40 0x7fc084004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc09cf9a700 1 -- 172.21.15.31:0/1138986494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc084005210 msgr2=0x7fc08400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc09cf9a700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc084005210 0x7fc08400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc09cf9a700 1 -- 172.21.15.31:0/1138986494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc084003c40 msgr2=0x7fc084004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc09cf9a700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc084003c40 0x7fc084004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.942+0000 7fc09df9c700 1 -- 172.21.15.31:0/1138986494 reap_dead start 2023-10-06T05:21:39.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.941+0000 7fc09cf9a700 1 -- 172.21.15.31:0/1138986494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc084003570 con 0x7fc084004810 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.942+0000 7fc09d79b700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc084003c40 0x7fc084004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.942+0000 7fc09cf9a700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc084004c80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc088006f00 tx=0x7fc0880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6dcf71dee962b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.942+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08800e710 con 0x7fc084004810 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.942+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08800e870 con 0x7fc084004810 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.942+0000 7fc08ffff700 1 -- 172.21.15.31:0/1138986494 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08800ec30 con 0x7fc084004810 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.943+0000 7fc09f9ff700 1 -- 172.21.15.31:0/1138986494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 msgr2=0x7fc084004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.943+0000 7fc09f9ff700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc084004c80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc088006f00 tx=0x7fc0880091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.943+0000 7fc09f9ff700 1 -- 172.21.15.31:0/1138986494 shutdown_connections 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.943+0000 7fc09f9ff700 1 --2- 172.21.15.31:0/1138986494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc084004c80 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.943+0000 7fc09f9ff700 1 -- 172.21.15.31:0/1138986494 >> 172.21.15.31:0/1138986494 conn(0x7fc0980cc0b0 msgr2=0x7fc0980cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.943+0000 7fc09f9ff700 1 -- 172.21.15.31:0/1138986494 shutdown_connections 2023-10-06T05:21:39.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.943+0000 7fc09f9ff700 1 -- 172.21.15.31:0/1138986494 wait complete. 2023-10-06T05:21:39.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.944+0000 7fc09f9ff700 1 Processor -- start 2023-10-06T05:21:39.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.945+0000 7fc09f9ff700 1 -- start start 2023-10-06T05:21:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.945+0000 7fc09f9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc09813d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.945+0000 7fc09f9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc09813b1a0 0x7fc09813b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.945+0000 7fc09f9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc098168c00 0x7fc098169070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09f9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0981695b0 con 0x7fc084004810 2023-10-06T05:21:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09f9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980afae0 con 0x7fc09813b1a0 2023-10-06T05:21:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09f9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980afdc0 con 0x7fc098168c00 2023-10-06T05:21:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc09813d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09d79b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc09813d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58932/0 (socket says 172.21.15.31:58932) 2023-10-06T05:21:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09d79b700 1 -- 172.21.15.31:0/946141462 learned_addr learned my addr 172.21.15.31:0/946141462 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09df9c700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc098168c00 0x7fc098169070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09cf9a700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc09813b1a0 0x7fc09813b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09d79b700 1 -- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc098168c00 msgr2=0x7fc098169070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09d79b700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc098168c00 0x7fc098169070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09d79b700 1 -- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc09813b1a0 msgr2=0x7fc09813b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09d79b700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc09813b1a0 0x7fc09813b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.946+0000 7fc09d79b700 1 -- 172.21.15.31:0/946141462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098150ad0 con 0x7fc084004810 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.947+0000 7fc09d79b700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc09813d3c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc09400e610 tx=0x7fc094009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.947+0000 7fc08d7fa700 1 -- 172.21.15.31:0/946141462 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0940072e0 con 0x7fc084004810 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.947+0000 7fc08d7fa700 1 -- 172.21.15.31:0/946141462 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc094007440 con 0x7fc084004810 2023-10-06T05:21:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.947+0000 7fc08d7fa700 1 -- 172.21.15.31:0/946141462 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0940057f0 con 0x7fc084004810 2023-10-06T05:21:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.947+0000 7fc09f9ff700 1 -- 172.21.15.31:0/946141462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc098150db0 con 0x7fc084004810 2023-10-06T05:21:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.947+0000 7fc09f9ff700 1 -- 172.21.15.31:0/946141462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09813d900 con 0x7fc084004810 2023-10-06T05:21:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.948+0000 7fc08d7fa700 1 -- 172.21.15.31:0/946141462 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7fc094005b60 con 0x7fc084004810 2023-10-06T05:21:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.948+0000 7fc09f9ff700 1 -- 172.21.15.31:0/946141462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09804c380 con 0x7fc084004810 2023-10-06T05:21:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.948+0000 7fc08d7fa700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc07c035900 0x7fc07c037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.949+0000 7fc08d7fa700 1 -- 172.21.15.31:0/946141462 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fc094047dd0 con 0x7fc084004810 2023-10-06T05:21:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.949+0000 7fc09cf9a700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc07c035900 0x7fc07c037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.950+0000 7fc09cf9a700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc07c035900 0x7fc07c037db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc0880091e0 tx=0x7fc08801a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:39.954+0000 7fc08d7fa700 1 -- 172.21.15.31:0/946141462 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc094005e10 con 0x7fc084004810 2023-10-06T05:21:40.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.161+0000 7fc09f9ff700 1 -- 172.21.15.31:0/946141462 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0980c7a50 con 0x7fc084004810 2023-10-06T05:21:40.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.162+0000 7fc08d7fa700 1 -- 172.21.15.31:0/946141462 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+17718 (secure 0 0 0) 0x7fc09401c7d0 con 0x7fc084004810 2023-10-06T05:21:40.165 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:40.165 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:35.434576+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:12.092711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:12.637981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1049444580":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/990299348":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/514347988":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/2247400373":"2023-10-07T05:21:35.434529+0000","172.21.15.31:6833/114556":"2023-10-07T05:21:35.434529+0000","172.21.15.31:6832/114556":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/2357011998":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/1563541111":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/1729374487":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/3570608959":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/1652493878":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/2612034927":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/3647032433":"2023-10-07T05:21:35.434529+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:40.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 -- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc07c035900 msgr2=0x7fc07c037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc07c035900 0x7fc07c037db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc0880091e0 tx=0x7fc08801a000 comp rx=0 tx=0).stop 2023-10-06T05:21:40.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 -- 172.21.15.31:0/946141462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 msgr2=0x7fc09813d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc09813d3c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc09400e610 tx=0x7fc094009040 comp rx=0 tx=0).stop 2023-10-06T05:21:40.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 -- 172.21.15.31:0/946141462 shutdown_connections 2023-10-06T05:21:40.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc07c035900 0x7fc07c037db0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc098168c00 0x7fc098169070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc084004810 0x7fc09813d3c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 --2- 172.21.15.31:0/946141462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc09813b1a0 0x7fc09813b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 -- 172.21.15.31:0/946141462 >> 172.21.15.31:0/946141462 conn(0x7fc0980cc0b0 msgr2=0x7fc0980c2680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:40.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 -- 172.21.15.31:0/946141462 shutdown_connections 2023-10-06T05:21:40.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.164+0000 7fc09f9ff700 1 -- 172.21.15.31:0/946141462 wait complete. 2023-10-06T05:21:40.185 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:21:40.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:40.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.419+0000 7fd721fff700 1 Processor -- start 2023-10-06T05:21:40.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.419+0000 7fd721fff700 1 -- start start 2023-10-06T05:21:40.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd721fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd71c0c7b60 0x7fd71c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd721fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd71c0aee70 0x7fd71c13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd721fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71c0ca5b0 con 0x7fd71c0c7f30 2023-10-06T05:21:40.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd721fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c0af2d0 con 0x7fd71c0aee70 2023-10-06T05:21:40.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd721fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c06f740 con 0x7fd71c0c7b60 2023-10-06T05:21:40.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd71bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd71c0aee70 0x7fd71c13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd71bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd71c0aee70 0x7fd71c13f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58940/0 (socket says 172.21.15.31:58940) 2023-10-06T05:21:40.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd71bfff700 1 -- 172.21.15.31:0/611430150 learned_addr learned my addr 172.21.15.31:0/611430150 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:40.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd71affd700 1 --1- 172.21.15.31:0/611430150 >> v1:172.21.15.31:6789/0 conn(0x7fd71c0c7f30 0x7fd71c06ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54330/0 (socket says 172.21.15.31:54330) 2023-10-06T05:21:40.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd71b7fe700 1 --2- 172.21.15.31:0/611430150 >> v2:172.21.15.62:3300/0 conn(0x7fd71c0c7b60 0x7fd71c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd71bfff700 1 -- 172.21.15.31:0/611430150 >> v2:172.21.15.62:3300/0 conn(0x7fd71c0c7b60 msgr2=0x7fd71c069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:40.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd71bfff700 1 --2- 172.21.15.31:0/611430150 >> v2:172.21.15.62:3300/0 conn(0x7fd71c0c7b60 0x7fd71c069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.422+0000 7fd71bfff700 1 -- 172.21.15.31:0/611430150 >> v1:172.21.15.31:6789/0 conn(0x7fd71c0c7f30 legacy=0x7fd71c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.423+0000 7fd71bfff700 1 -- 172.21.15.31:0/611430150 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c0c9c80 con 0x7fd71c0aee70 2023-10-06T05:21:40.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.423+0000 7fd71a7fc700 1 -- 172.21.15.31:0/611430150 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797112105 0 0) 0x7fd71c0ca5b0 con 0x7fd71c0c7f30 2023-10-06T05:21:40.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.423+0000 7fd71bfff700 1 --2- 172.21.15.31:0/611430150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0aee70 0x7fd71c13f690 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd71c06fa80 tx=0x7fd714009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92c62d87d6ccb56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.423+0000 7fd71a7fc700 1 -- 172.21.15.31:0/611430150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400e070 con 0x7fd71c0aee70 2023-10-06T05:21:40.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.423+0000 7fd71a7fc700 1 -- 172.21.15.31:0/611430150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71400ba20 con 0x7fd71c0aee70 2023-10-06T05:21:40.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.423+0000 7fd71a7fc700 1 -- 172.21.15.31:0/611430150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140092e0 con 0x7fd71c0aee70 2023-10-06T05:21:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.424+0000 7fd721fff700 1 -- 172.21.15.31:0/611430150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0aee70 msgr2=0x7fd71c13f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.424+0000 7fd721fff700 1 --2- 172.21.15.31:0/611430150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0aee70 0x7fd71c13f690 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd71c06fa80 tx=0x7fd714009840 comp rx=0 tx=0).stop 2023-10-06T05:21:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.424+0000 7fd721fff700 1 -- 172.21.15.31:0/611430150 shutdown_connections 2023-10-06T05:21:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.424+0000 7fd721fff700 1 --2- 172.21.15.31:0/611430150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0aee70 0x7fd71c13f690 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.424+0000 7fd721fff700 1 --2- 172.21.15.31:0/611430150 >> v2:172.21.15.62:3300/0 conn(0x7fd71c0c7b60 0x7fd71c069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.424+0000 7fd721fff700 1 -- 172.21.15.31:0/611430150 >> 172.21.15.31:0/611430150 conn(0x7fd71c0cc0b0 msgr2=0x7fd71c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.424+0000 7fd721fff700 1 -- 172.21.15.31:0/611430150 shutdown_connections 2023-10-06T05:21:40.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.424+0000 7fd721fff700 1 -- 172.21.15.31:0/611430150 wait complete. 2023-10-06T05:21:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.425+0000 7fd721fff700 1 Processor -- start 2023-10-06T05:21:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.426+0000 7fd721fff700 1 -- start start 2023-10-06T05:21:40.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.426+0000 7fd721fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd71c0aee70 0x7fd71c0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.426+0000 7fd721fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0c7b60 0x7fd71c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd721fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd71c0c7f30 0x7fd71c156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd721fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0c6c30 con 0x7fd71c0c7b60 2023-10-06T05:21:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd721fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0b00c0 con 0x7fd71c0c7f30 2023-10-06T05:21:40.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd721fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0b03a0 con 0x7fd71c0aee70 2023-10-06T05:21:40.430 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:40.427+0000 7ff5a84761c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:21:40.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0c7b60 0x7fd71c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0c7b60 0x7fd71c0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58948/0 (socket says 172.21.15.31:58948) 2023-10-06T05:21:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71affd700 1 -- 172.21.15.31:0/1530041054 learned_addr learned my addr 172.21.15.31:0/1530041054 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71bfff700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd71c0c7f30 0x7fd71c156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71b7fe700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd71c0aee70 0x7fd71c0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71affd700 1 -- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd71c0aee70 msgr2=0x7fd71c0afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71affd700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd71c0aee70 0x7fd71c0afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71affd700 1 -- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd71c0c7f30 msgr2=0x7fd71c156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71affd700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd71c0c7f30 0x7fd71c156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71affd700 1 -- 172.21.15.31:0/1530041054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd714009ae0 con 0x7fd71c0c7b60 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.427+0000 7fd71b7fe700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd71c0aee70 0x7fd71c0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.428+0000 7fd71affd700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0c7b60 0x7fd71c0c66f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd71000c5b0 tx=0x7fd710001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.428+0000 7fd7037fe700 1 -- 172.21.15.31:0/1530041054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710005a70 con 0x7fd71c0c7b60 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.428+0000 7fd7037fe700 1 -- 172.21.15.31:0/1530041054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd710005bd0 con 0x7fd71c0c7b60 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.428+0000 7fd7037fe700 1 -- 172.21.15.31:0/1530041054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7100070d0 con 0x7fd71c0c7b60 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.428+0000 7fd721fff700 1 -- 172.21.15.31:0/1530041054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c0b0680 con 0x7fd71c0c7b60 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.428+0000 7fd721fff700 1 -- 172.21.15.31:0/1530041054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c0b0c40 con 0x7fd71c0c7b60 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.429+0000 7fd7037fe700 1 -- 172.21.15.31:0/1530041054 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7fd7100074c0 con 0x7fd71c0c7b60 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.429+0000 7fd721fff700 1 -- 172.21.15.31:0/1530041054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71c0b0e60 con 0x7fd71c0c7b60 2023-10-06T05:21:40.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.430+0000 7fd7037fe700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd708035900 0x7fd708037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.430+0000 7fd7037fe700 1 -- 172.21.15.31:0/1530041054 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fd710047910 con 0x7fd71c0c7b60 2023-10-06T05:21:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.430+0000 7fd71b7fe700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd708035900 0x7fd708037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.431+0000 7fd71b7fe700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd708035900 0x7fd708037db0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009d70 tx=0x7fd70c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.434+0000 7fd7037fe700 1 -- 172.21.15.31:0/1530041054 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd71000ee40 con 0x7fd71c0c7b60 2023-10-06T05:21:40.623 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:40.621+0000 7ff5a84761c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:21:40.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.683+0000 7fd721fff700 1 -- 172.21.15.31:0/1530041054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd71c04c380 con 0x7fd71c0c7b60 2023-10-06T05:21:40.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.684+0000 7fd7037fe700 1 -- 172.21.15.31:0/1530041054 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4590 (secure 0 0 0) 0x7fd710017090 con 0x7fd71c0c7b60 2023-10-06T05:21:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:40.687 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2151449662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2151449662},{"type":"v1","addr":"172.21.15.62:6834","nonce":2151449662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/810564890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":810564890},{"type":"v1","addr":"172.21.15.31:6836","nonce":810564890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/802676715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":802676715},{"type":"v1","addr":"172.21.15.31:6837","nonce":802676715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:13.661482+0000","modified":"2023-10-06T05:21:17.173141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6249":{"gid":6249,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/934773709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":934773709},{"type":"v1","addr":"172.21.15.62:6835","nonce":934773709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:40.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 -- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd708035900 msgr2=0x7fd708037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd708035900 0x7fd708037db0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009d70 tx=0x7fd70c008040 comp rx=0 tx=0).stop 2023-10-06T05:21:40.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 -- 172.21.15.31:0/1530041054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0c7b60 msgr2=0x7fd71c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0c7b60 0x7fd71c0c66f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd71000c5b0 tx=0x7fd710001040 comp rx=0 tx=0).stop 2023-10-06T05:21:40.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 -- 172.21.15.31:0/1530041054 shutdown_connections 2023-10-06T05:21:40.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd708035900 0x7fd708037db0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd71c0c7f30 0x7fd71c156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd71c0aee70 0x7fd71c0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 --2- 172.21.15.31:0/1530041054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd71c0c7b60 0x7fd71c0c66f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 -- 172.21.15.31:0/1530041054 >> 172.21.15.31:0/1530041054 conn(0x7fd71c0cc0b0 msgr2=0x7fd71c0c2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:40.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 -- 172.21.15.31:0/1530041054 shutdown_connections 2023-10-06T05:21:40.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.687+0000 7fd721fff700 1 -- 172.21.15.31:0/1530041054 wait complete. 2023-10-06T05:21:40.691 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 45 2023-10-06T05:21:40.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:40.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.912+0000 7f537bbff700 1 Processor -- start 2023-10-06T05:21:40.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.912+0000 7f537bbff700 1 -- start start 2023-10-06T05:21:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.914+0000 7f537bbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f53740ca770 0x7f53740c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.914+0000 7f537bbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53740c6df0 con 0x7f53740cab40 2023-10-06T05:21:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.914+0000 7f537bbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53740c70d0 con 0x7f537406b510 2023-10-06T05:21:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.914+0000 7f537bbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53740c7a70 con 0x7f53740ca770 2023-10-06T05:21:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.914+0000 7f537919a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f53740cab40 0x7f537406ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54346/0 (socket says 172.21.15.31:54346) 2023-10-06T05:21:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.914+0000 7f537919a700 1 -- 172.21.15.31:0/2396781110 learned_addr learned my addr 172.21.15.31:0/2396781110 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.914+0000 7f537999b700 1 --2- 172.21.15.31:0/2396781110 >> v2:172.21.15.62:3300/0 conn(0x7f53740ca770 0x7f53740c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.915+0000 7f5378999700 1 -- 172.21.15.31:0/2396781110 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305970998 0 0) 0x7f53740c6df0 con 0x7f53740cab40 2023-10-06T05:21:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.915+0000 7f5378999700 1 -- 172.21.15.31:0/2396781110 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5360004680 con 0x7f53740cab40 2023-10-06T05:21:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.915+0000 7f5378999700 1 -- 172.21.15.31:0/2396781110 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146208992 0 0) 0x7f53740c70d0 con 0x7f537406b510 2023-10-06T05:21:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.915+0000 7f5378999700 1 -- 172.21.15.31:0/2396781110 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53740c6df0 con 0x7f537406b510 2023-10-06T05:21:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.915+0000 7f5378999700 1 -- 172.21.15.31:0/2396781110 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3659338502 0 0) 0x7f5360004680 con 0x7f53740cab40 2023-10-06T05:21:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.915+0000 7f537999b700 1 -- 172.21.15.31:0/2396781110 >> v1:172.21.15.62:6789/0 conn(0x7f537406b510 legacy=0x7f537414c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.915+0000 7f537999b700 1 -- 172.21.15.31:0/2396781110 >> v1:172.21.15.31:6789/0 conn(0x7f53740cab40 legacy=0x7f537406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.915+0000 7f537999b700 1 -- 172.21.15.31:0/2396781110 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53740c73b0 con 0x7f53740ca770 2023-10-06T05:21:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.915+0000 7f537999b700 1 --2- 172.21.15.31:0/2396781110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53740ca770 0x7f53740c66f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f53700098d0 tx=0x7f5370009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d4d01435fea54e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.916+0000 7f5378999700 1 -- 172.21.15.31:0/2396781110 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364001470 con 0x7f53740ca770 2023-10-06T05:21:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.916+0000 7f5378999700 1 -- 172.21.15.31:0/2396781110 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5370011420 con 0x7f53740ca770 2023-10-06T05:21:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.916+0000 7f5378999700 1 -- 172.21.15.31:0/2396781110 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53700117c0 con 0x7f53740ca770 2023-10-06T05:21:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.916+0000 7f537bbff700 1 -- 172.21.15.31:0/2396781110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53740ca770 msgr2=0x7f53740c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.916+0000 7f537bbff700 1 --2- 172.21.15.31:0/2396781110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53740ca770 0x7f53740c66f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f53700098d0 tx=0x7f5370009120 comp rx=0 tx=0).stop 2023-10-06T05:21:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.916+0000 7f537bbff700 1 -- 172.21.15.31:0/2396781110 shutdown_connections 2023-10-06T05:21:40.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.916+0000 7f537bbff700 1 --2- 172.21.15.31:0/2396781110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53740ca770 0x7f53740c66f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.916+0000 7f537bbff700 1 -- 172.21.15.31:0/2396781110 >> 172.21.15.31:0/2396781110 conn(0x7f53740cc0b0 msgr2=0x7f53740cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:40.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.916+0000 7f537bbff700 1 -- 172.21.15.31:0/2396781110 shutdown_connections 2023-10-06T05:21:40.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.916+0000 7f537bbff700 1 -- 172.21.15.31:0/2396781110 wait complete. 2023-10-06T05:21:40.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.918+0000 7f537bbff700 1 Processor -- start 2023-10-06T05:21:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.919+0000 7f537bbff700 1 -- start start 2023-10-06T05:21:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537bbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537406b510 0x7f537413ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537bbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53740ca770 0x7f53741549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537bbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53740cab40 0x7f5374154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537bbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5374140420 con 0x7f53740ca770 2023-10-06T05:21:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537bbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5374140700 con 0x7f53740cab40 2023-10-06T05:21:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537bbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53741409e0 con 0x7f537406b510 2023-10-06T05:21:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53740ca770 0x7f53741549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53740ca770 0x7f53741549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58958/0 (socket says 172.21.15.31:58958) 2023-10-06T05:21:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537919a700 1 -- 172.21.15.31:0/705878098 learned_addr learned my addr 172.21.15.31:0/705878098 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:40.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537999b700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537406b510 0x7f537413ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.920+0000 7f537a19c700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53740cab40 0x7f5374154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f537919a700 1 -- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537406b510 msgr2=0x7f537413ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f537919a700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537406b510 0x7f537413ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f537919a700 1 -- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53740cab40 msgr2=0x7f5374154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:40.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f537919a700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53740cab40 0x7f5374154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:40.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f537919a700 1 -- 172.21.15.31:0/705878098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53700093c0 con 0x7f53740ca770 2023-10-06T05:21:40.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f537919a700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53740ca770 0x7f53741549f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f536400cb90 tx=0x7f5364001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f5369ffb700 1 -- 172.21.15.31:0/705878098 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364007150 con 0x7f53740ca770 2023-10-06T05:21:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f5369ffb700 1 -- 172.21.15.31:0/705878098 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53640072f0 con 0x7f53740ca770 2023-10-06T05:21:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f537bbff700 1 -- 172.21.15.31:0/705878098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5374140cc0 con 0x7f53740ca770 2023-10-06T05:21:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f5369ffb700 1 -- 172.21.15.31:0/705878098 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53640076b0 con 0x7f53740ca770 2023-10-06T05:21:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.921+0000 7f537bbff700 1 -- 172.21.15.31:0/705878098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53740aee50 con 0x7f53740ca770 2023-10-06T05:21:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.922+0000 7f537bbff700 1 -- 172.21.15.31:0/705878098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537404c380 con 0x7f53740ca770 2023-10-06T05:21:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.923+0000 7f5369ffb700 1 -- 172.21.15.31:0/705878098 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f5364004a40 con 0x7f53740ca770 2023-10-06T05:21:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.923+0000 7f5369ffb700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5358035900 0x7f5358037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.923+0000 7f5369ffb700 1 -- 172.21.15.31:0/705878098 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f5364047cd0 con 0x7f53740ca770 2023-10-06T05:21:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.924+0000 7f537999b700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5358035900 0x7f5358037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:40.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.924+0000 7f537999b700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5358035900 0x7f5358037db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5370009390 tx=0x7f5370016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:40.928+0000 7f5369ffb700 1 -- 172.21.15.31:0/705878098 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f536400f260 con 0x7f53740ca770 2023-10-06T05:21:41.025 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:41.022+0000 7ff5a84761c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:21:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.188+0000 7f537bbff700 1 -- 172.21.15.31:0/705878098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53740cbe20 con 0x7f53740ca770 2023-10-06T05:21:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.189+0000 7f5369ffb700 1 -- 172.21.15.31:0/705878098 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f5364018070 con 0x7f53740ca770 2023-10-06T05:21:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:41.192 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2151449662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2151449662},{"type":"v1","addr":"172.21.15.62:6834","nonce":2151449662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/810564890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":810564890},{"type":"v1","addr":"172.21.15.31:6836","nonce":810564890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/802676715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":802676715},{"type":"v1","addr":"172.21.15.31:6837","nonce":802676715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:13.661482+0000","modified":"2023-10-06T05:21:17.173141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6249":{"gid":6249,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/934773709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":934773709},{"type":"v1","addr":"172.21.15.62:6835","nonce":934773709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T05:21:41.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 -- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5358035900 msgr2=0x7f5358037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5358035900 0x7f5358037db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5370009390 tx=0x7f5370016000 comp rx=0 tx=0).stop 2023-10-06T05:21:41.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 -- 172.21.15.31:0/705878098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53740ca770 msgr2=0x7f53741549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53740ca770 0x7f53741549f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f536400cb90 tx=0x7f5364001ab0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 -- 172.21.15.31:0/705878098 shutdown_connections 2023-10-06T05:21:41.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5358035900 0x7f5358037db0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53740cab40 0x7f5374154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537406b510 0x7f537413ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 --2- 172.21.15.31:0/705878098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53740ca770 0x7f53741549f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 -- 172.21.15.31:0/705878098 >> 172.21.15.31:0/705878098 conn(0x7f53740cc0b0 msgr2=0x7f53740c1470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:41.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.191+0000 7f537bbff700 1 -- 172.21.15.31:0/705878098 shutdown_connections 2023-10-06T05:21:41.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.192+0000 7f537bbff700 1 -- 172.21.15.31:0/705878098 wait complete. 2023-10-06T05:21:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 45 2023-10-06T05:21:41.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.409+0000 7f8b285ff700 1 Processor -- start 2023-10-06T05:21:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.410+0000 7f8b285ff700 1 -- start start 2023-10-06T05:21:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.411+0000 7f8b285ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b2015d3f0 0x7f8b200aab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.411+0000 7f8b285ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b2015d7c0 0x7f8b200c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.411+0000 7f8b285ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b200a9ab0 con 0x7f8b200c7620 2023-10-06T05:21:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.411+0000 7f8b285ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b200a98a0 con 0x7f8b2015d3f0 2023-10-06T05:21:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.411+0000 7f8b285ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200a9d90 con 0x7f8b2015d7c0 2023-10-06T05:21:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.411+0000 7f8b2639b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b2015d3f0 0x7f8b200aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.411+0000 7f8b2639b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b2015d3f0 0x7f8b200aab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58964/0 (socket says 172.21.15.31:58964) 2023-10-06T05:21:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.411+0000 7f8b2639b700 1 -- 172.21.15.31:0/2252417600 learned_addr learned my addr 172.21.15.31:0/2252417600 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.412+0000 7f8b25b9a700 1 --2- 172.21.15.31:0/2252417600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b2015d7c0 0x7f8b200c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.412+0000 7f8b2639b700 1 -- 172.21.15.31:0/2252417600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b2015d7c0 msgr2=0x7f8b200c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.412+0000 7f8b2639b700 1 --2- 172.21.15.31:0/2252417600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b2015d7c0 0x7f8b200c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.412+0000 7f8b2639b700 1 -- 172.21.15.31:0/2252417600 >> v1:172.21.15.62:6789/0 conn(0x7f8b200c7620 legacy=0x7f8b20159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.412+0000 7f8b2639b700 1 -- 172.21.15.31:0/2252417600 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b200aa070 con 0x7f8b2015d3f0 2023-10-06T05:21:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.412+0000 7f8b2639b700 1 --2- 172.21.15.31:0/2252417600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 0x7f8b200aab50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009990 tx=0x7f8b1c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4994ccb26a4c01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.412+0000 7f8b25399700 1 -- 172.21.15.31:0/2252417600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c005c10 con 0x7f8b2015d3f0 2023-10-06T05:21:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.412+0000 7f8b25399700 1 -- 172.21.15.31:0/2252417600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c005d70 con 0x7f8b2015d3f0 2023-10-06T05:21:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.412+0000 7f8b25399700 1 -- 172.21.15.31:0/2252417600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c00b880 con 0x7f8b2015d3f0 2023-10-06T05:21:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.413+0000 7f8b285ff700 1 -- 172.21.15.31:0/2252417600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 msgr2=0x7f8b200aab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.413+0000 7f8b285ff700 1 --2- 172.21.15.31:0/2252417600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 0x7f8b200aab50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009990 tx=0x7f8b1c009220 comp rx=0 tx=0).stop 2023-10-06T05:21:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.413+0000 7f8b285ff700 1 -- 172.21.15.31:0/2252417600 shutdown_connections 2023-10-06T05:21:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.413+0000 7f8b285ff700 1 --2- 172.21.15.31:0/2252417600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 0x7f8b200aab50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.413+0000 7f8b285ff700 1 --2- 172.21.15.31:0/2252417600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b2015d7c0 0x7f8b200c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.413+0000 7f8b285ff700 1 -- 172.21.15.31:0/2252417600 >> 172.21.15.31:0/2252417600 conn(0x7f8b200cc0b0 msgr2=0x7f8b200cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.413+0000 7f8b285ff700 1 -- 172.21.15.31:0/2252417600 shutdown_connections 2023-10-06T05:21:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.413+0000 7f8b285ff700 1 -- 172.21.15.31:0/2252417600 wait complete. 2023-10-06T05:21:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.415+0000 7f8b285ff700 1 Processor -- start 2023-10-06T05:21:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.415+0000 7f8b285ff700 1 -- start start 2023-10-06T05:21:41.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b285ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b200c7620 0x7f8b20153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b285ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 0x7f8b20169570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b285ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2015d7c0 0x7f8b2016a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b285ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200b1cf0 con 0x7f8b2015d3f0 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b285ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b20169ab0 con 0x7f8b2015d7c0 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b285ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b20169d80 con 0x7f8b200c7620 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b25b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 0x7f8b20169570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b25b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 0x7f8b20169570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58976/0 (socket says 172.21.15.31:58976) 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b25b9a700 1 -- 172.21.15.31:0/3886323183 learned_addr learned my addr 172.21.15.31:0/3886323183 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b2639b700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b200c7620 0x7f8b20153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.416+0000 7f8b26b9c700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2015d7c0 0x7f8b2016a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b25b9a700 1 -- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b200c7620 msgr2=0x7f8b20153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b25b9a700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b200c7620 0x7f8b20153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b25b9a700 1 -- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2015d7c0 msgr2=0x7f8b2016a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b25b9a700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2015d7c0 0x7f8b2016a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b25b9a700 1 -- 172.21.15.31:0/3886323183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c009480 con 0x7f8b2015d3f0 2023-10-06T05:21:41.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b25b9a700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 0x7f8b20169570 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d860 tx=0x7f8b10008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b167fc700 1 -- 172.21.15.31:0/3886323183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b100125f0 con 0x7f8b2015d3f0 2023-10-06T05:21:41.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b167fc700 1 -- 172.21.15.31:0/3886323183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b10012770 con 0x7f8b2015d3f0 2023-10-06T05:21:41.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b167fc700 1 -- 172.21.15.31:0/3886323183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10012b30 con 0x7f8b2015d3f0 2023-10-06T05:21:41.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b285ff700 1 -- 172.21.15.31:0/3886323183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2016a060 con 0x7f8b2015d3f0 2023-10-06T05:21:41.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.417+0000 7f8b285ff700 1 -- 172.21.15.31:0/3886323183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b20154bb0 con 0x7f8b2015d3f0 2023-10-06T05:21:41.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.418+0000 7f8b167fc700 1 -- 172.21.15.31:0/3886323183 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f8b10013480 con 0x7f8b2015d3f0 2023-10-06T05:21:41.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.418+0000 7f8b285ff700 1 -- 172.21.15.31:0/3886323183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b20154a10 con 0x7f8b2015d3f0 2023-10-06T05:21:41.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.419+0000 7f8b167fc700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b04035900 0x7f8b04037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.419+0000 7f8b167fc700 1 -- 172.21.15.31:0/3886323183 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f8b10048850 con 0x7f8b2015d3f0 2023-10-06T05:21:41.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.419+0000 7f8b2639b700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b04035900 0x7f8b04037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.420+0000 7f8b2639b700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b04035900 0x7f8b04037db0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009260 tx=0x7f8b1c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.424+0000 7f8b167fc700 1 -- 172.21.15.31:0/3886323183 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b20154a10 con 0x7f8b2015d3f0 2023-10-06T05:21:41.471 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:41.469+0000 7ff5a84761c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:21:41.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.630+0000 7f8b285ff700 1 -- 172.21.15.31:0/3886323183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b20154a10 con 0x7f8b2015d3f0 2023-10-06T05:21:41.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.631+0000 7f8b167fc700 1 -- 172.21.15.31:0/3886323183 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+17718 (secure 0 0 0) 0x7f8b20154a10 con 0x7f8b2015d3f0 2023-10-06T05:21:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:41.633 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:35.434576+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:12.092711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:12.637981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1049444580":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/990299348":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/514347988":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/2247400373":"2023-10-07T05:21:35.434529+0000","172.21.15.31:6833/114556":"2023-10-07T05:21:35.434529+0000","172.21.15.31:6832/114556":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/2357011998":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/1563541111":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/1729374487":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/3570608959":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/1652493878":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/2612034927":"2023-10-07T05:21:35.434529+0000","172.21.15.31:0/3647032433":"2023-10-07T05:21:35.434529+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.634+0000 7f8b285ff700 1 -- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b04035900 msgr2=0x7f8b04037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.634+0000 7f8b285ff700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b04035900 0x7f8b04037db0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009260 tx=0x7f8b1c016000 comp rx=0 tx=0).stop 2023-10-06T05:21:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.635+0000 7f8b285ff700 1 -- 172.21.15.31:0/3886323183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 msgr2=0x7f8b20169570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.635+0000 7f8b285ff700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 0x7f8b20169570 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d860 tx=0x7f8b10008040 comp rx=0 tx=0).stop 2023-10-06T05:21:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.635+0000 7f8b285ff700 1 -- 172.21.15.31:0/3886323183 shutdown_connections 2023-10-06T05:21:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.635+0000 7f8b285ff700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b04035900 0x7f8b04037db0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.635+0000 7f8b285ff700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2015d7c0 0x7f8b2016a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.635+0000 7f8b285ff700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b200c7620 0x7f8b20153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.635+0000 7f8b285ff700 1 --2- 172.21.15.31:0/3886323183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b2015d3f0 0x7f8b20169570 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.635+0000 7f8b285ff700 1 -- 172.21.15.31:0/3886323183 >> 172.21.15.31:0/3886323183 conn(0x7f8b200cc0b0 msgr2=0x7f8b20158270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:41.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.635+0000 7f8b285ff700 1 -- 172.21.15.31:0/3886323183 shutdown_connections 2023-10-06T05:21:41.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.635+0000 7f8b285ff700 1 -- 172.21.15.31:0/3886323183 wait complete. 2023-10-06T05:21:41.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:21:41.668 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:41.666+0000 7ff5a84761c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:21:41.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.860+0000 7f10f69ff700 1 Processor -- start 2023-10-06T05:21:41.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.860+0000 7f10f69ff700 1 -- start start 2023-10-06T05:21:41.866 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:41.861+0000 7ff5a84761c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:21:41.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.861+0000 7f10f69ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 0x7f10f00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10f69ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10f015d2d0 0x7f10f00aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10f69ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f00c3f40 con 0x7f10f0159500 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10f69ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00b2860 con 0x7f10f015d2d0 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10f69ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f015d7d0 con 0x7f10f00cd510 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10effff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f10f0159500 0x7f10f00cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54356/0 (socket says 172.21.15.31:54356) 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10effff700 1 -- 172.21.15.31:0/2135731766 learned_addr learned my addr 172.21.15.31:0/2135731766 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10f4f9c700 1 --2- 172.21.15.31:0/2135731766 >> v2:172.21.15.31:3300/0 conn(0x7f10f015d2d0 0x7f10f00aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10ef7fe700 1 --2- 172.21.15.31:0/2135731766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 0x7f10f00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10eeffd700 1 -- 172.21.15.31:0/2135731766 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158080043 0 0) 0x7f10f00c3f40 con 0x7f10f0159500 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10f4f9c700 1 -- 172.21.15.31:0/2135731766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 msgr2=0x7f10f00b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10f4f9c700 1 --2- 172.21.15.31:0/2135731766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 0x7f10f00b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10f4f9c700 1 -- 172.21.15.31:0/2135731766 >> v1:172.21.15.31:6789/0 conn(0x7f10f0159500 legacy=0x7f10f00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.862+0000 7f10f4f9c700 1 -- 172.21.15.31:0/2135731766 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00cb000 con 0x7f10f015d2d0 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.863+0000 7f10f4f9c700 1 --2- 172.21.15.31:0/2135731766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 0x7f10f00aedd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009e60 tx=0x7f10e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c2659f6233c2b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.863+0000 7f10eeffd700 1 -- 172.21.15.31:0/2135731766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e800e070 con 0x7f10f015d2d0 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.863+0000 7f10eeffd700 1 -- 172.21.15.31:0/2135731766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e800bb60 con 0x7f10f015d2d0 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.863+0000 7f10eeffd700 1 -- 172.21.15.31:0/2135731766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8012430 con 0x7f10f015d2d0 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.864+0000 7f10f69ff700 1 -- 172.21.15.31:0/2135731766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 msgr2=0x7f10f00aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.864+0000 7f10f69ff700 1 --2- 172.21.15.31:0/2135731766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 0x7f10f00aedd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009e60 tx=0x7f10e8009780 comp rx=0 tx=0).stop 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.864+0000 7f10f69ff700 1 -- 172.21.15.31:0/2135731766 shutdown_connections 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.864+0000 7f10f69ff700 1 --2- 172.21.15.31:0/2135731766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 0x7f10f00aedd0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.864+0000 7f10f69ff700 1 --2- 172.21.15.31:0/2135731766 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 0x7f10f00b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.864+0000 7f10f69ff700 1 -- 172.21.15.31:0/2135731766 >> 172.21.15.31:0/2135731766 conn(0x7f10f006ec90 msgr2=0x7f10f006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.864+0000 7f10f69ff700 1 -- 172.21.15.31:0/2135731766 shutdown_connections 2023-10-06T05:21:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.864+0000 7f10f69ff700 1 -- 172.21.15.31:0/2135731766 wait complete. 2023-10-06T05:21:41.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.866+0000 7f10f69ff700 1 Processor -- start 2023-10-06T05:21:41.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.866+0000 7f10f69ff700 1 -- start start 2023-10-06T05:21:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.867+0000 7f10f69ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 0x7f10f00b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.867+0000 7f10f69ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10f0159500 0x7f10f01537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.867+0000 7f10f69ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 0x7f10f00cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.867+0000 7f10f69ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00b3f00 con 0x7f10f015d2d0 2023-10-06T05:21:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.867+0000 7f10f69ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0157810 con 0x7f10f0159500 2023-10-06T05:21:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.867+0000 7f10f69ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00b3330 con 0x7f10f00cd510 2023-10-06T05:21:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10f4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 0x7f10f00cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10effff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 0x7f10f00b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10f4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 0x7f10f00cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58986/0 (socket says 172.21.15.31:58986) 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10f4f9c700 1 -- 172.21.15.31:0/3861412957 learned_addr learned my addr 172.21.15.31:0/3861412957 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10ef7fe700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10f0159500 0x7f10f01537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10effff700 1 -- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10f0159500 msgr2=0x7f10f01537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10effff700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10f0159500 0x7f10f01537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10effff700 1 -- 172.21.15.31:0/3861412957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 msgr2=0x7f10f00cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10effff700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 0x7f10f00cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10effff700 1 -- 172.21.15.31:0/3861412957 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e8009a20 con 0x7f10f00cd510 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.868+0000 7f10f4f9c700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 0x7f10f00cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.869+0000 7f10effff700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 0x7f10f00b4390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f10e00011e0 tx=0x7f10e0007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.869+0000 7f10d7fff700 1 -- 172.21.15.31:0/3861412957 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0005b30 con 0x7f10f00cd510 2023-10-06T05:21:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.869+0000 7f10d7fff700 1 -- 172.21.15.31:0/3861412957 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e0005c90 con 0x7f10f00cd510 2023-10-06T05:21:41.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.869+0000 7f10d7fff700 1 -- 172.21.15.31:0/3861412957 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0007300 con 0x7f10f00cd510 2023-10-06T05:21:41.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.869+0000 7f10f69ff700 1 -- 172.21.15.31:0/3861412957 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f00b35d0 con 0x7f10f00cd510 2023-10-06T05:21:41.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.869+0000 7f10f69ff700 1 -- 172.21.15.31:0/3861412957 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f00b3b90 con 0x7f10f00cd510 2023-10-06T05:21:41.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.870+0000 7f10f69ff700 1 -- 172.21.15.31:0/3861412957 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f004c380 con 0x7f10f00cd510 2023-10-06T05:21:41.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.870+0000 7f10d7fff700 1 -- 172.21.15.31:0/3861412957 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7f10e001b3b0 con 0x7f10f00cd510 2023-10-06T05:21:41.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.871+0000 7f10d7fff700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10dc035900 0x7f10dc037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:41.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.871+0000 7f10d7fff700 1 -- 172.21.15.31:0/3861412957 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f10e00479c0 con 0x7f10f00cd510 2023-10-06T05:21:41.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.871+0000 7f10ef7fe700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10dc035900 0x7f10dc037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:41.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.872+0000 7f10ef7fe700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10dc035900 0x7f10dc037db0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f10f016ce60 tx=0x7f10e4009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:41.877+0000 7f10d7fff700 1 -- 172.21.15.31:0/3861412957 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10e001b660 con 0x7f10f00cd510 2023-10-06T05:21:41.978 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-10-06T05:21:41.976+0000 7ff5a84761c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:21:42.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.123+0000 7f10f69ff700 1 -- 172.21.15.31:0/3861412957 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f10f00d0200 con 0x7f10f00cd510 2023-10-06T05:21:42.125 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:21:42.124+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:21:42.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.130+0000 7f10d7fff700 1 -- 172.21.15.31:0/3861412957 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7f10e0017020 con 0x7f10f00cd510 2023-10-06T05:21:42.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 -- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10dc035900 msgr2=0x7f10dc037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10dc035900 0x7f10dc037db0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f10f016ce60 tx=0x7f10e4009120 comp rx=0 tx=0).stop 2023-10-06T05:21:42.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 -- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 msgr2=0x7f10f00b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 0x7f10f00b4390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f10e00011e0 tx=0x7f10e0007d30 comp rx=0 tx=0).stop 2023-10-06T05:21:42.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 -- 172.21.15.31:0/3861412957 shutdown_connections 2023-10-06T05:21:42.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10dc035900 0x7f10dc037db0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f015d2d0 0x7f10f00cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10f00cd510 0x7f10f00b4390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 --2- 172.21.15.31:0/3861412957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10f0159500 0x7f10f01537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 -- 172.21.15.31:0/3861412957 >> 172.21.15.31:0/3861412957 conn(0x7f10f006ec90 msgr2=0x7f10f00ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:42.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 -- 172.21.15.31:0/3861412957 shutdown_connections 2023-10-06T05:21:42.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.132+0000 7f10f69ff700 1 -- 172.21.15.31:0/3861412957 wait complete. 2023-10-06T05:21:42.136 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:21:42.141 INFO:tasks.ceph.mds.d.smithi062.stderr: -5983> 2023-10-06T05:21:18.077+0000 7fc007df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:21:42.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:21:42.190 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:21:42.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.388+0000 7fba17fff700 1 Processor -- start 2023-10-06T05:21:42.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.388+0000 7fba17fff700 1 -- start start 2023-10-06T05:21:42.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.389+0000 7fba17fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fba10159560 0x7fba100cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.389+0000 7fba17fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1015d2d0 con 0x7fba100b2b80 2023-10-06T05:21:42.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.389+0000 7fba17fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1015d5b0 con 0x7fba10159930 2023-10-06T05:21:42.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.389+0000 7fba17fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1015d890 con 0x7fba10159560 2023-10-06T05:21:42.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.390+0000 7fba1659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fba100b2b80 0x7fba100cb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54360/0 (socket says 172.21.15.31:54360) 2023-10-06T05:21:42.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.390+0000 7fba1659c700 1 -- 172.21.15.31:0/139542673 learned_addr learned my addr 172.21.15.31:0/139542673 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:42.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.390+0000 7fba15d9b700 1 --2- 172.21.15.31:0/139542673 >> v2:172.21.15.62:3300/0 conn(0x7fba10159560 0x7fba100cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.390+0000 7fba14d99700 1 -- 172.21.15.31:0/139542673 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966419343 0 0) 0x7fba1015d2d0 con 0x7fba100b2b80 2023-10-06T05:21:42.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.390+0000 7fba14d99700 1 -- 172.21.15.31:0/139542673 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9fc003570 con 0x7fba100b2b80 2023-10-06T05:21:42.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.390+0000 7fba14d99700 1 -- 172.21.15.31:0/139542673 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438164536 0 0) 0x7fba1015d5b0 con 0x7fba10159930 2023-10-06T05:21:42.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.390+0000 7fba14d99700 1 -- 172.21.15.31:0/139542673 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba1015d2d0 con 0x7fba10159930 2023-10-06T05:21:42.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.390+0000 7fba15d9b700 1 -- 172.21.15.31:0/139542673 >> v1:172.21.15.62:6789/0 conn(0x7fba10159930 legacy=0x7fba100b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.390+0000 7fba15d9b700 1 -- 172.21.15.31:0/139542673 >> v1:172.21.15.31:6789/0 conn(0x7fba100b2b80 legacy=0x7fba100cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.390+0000 7fba15d9b700 1 -- 172.21.15.31:0/139542673 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1015db70 con 0x7fba10159560 2023-10-06T05:21:42.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.391+0000 7fba15d9b700 1 --2- 172.21.15.31:0/139542673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba10159560 0x7fba100cce70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0098e0 tx=0x7fba0c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1971dbc49e5fb1b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:42.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.391+0000 7fba14d99700 1 -- 172.21.15.31:0/139542673 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c011630 con 0x7fba10159560 2023-10-06T05:21:42.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.391+0000 7fba14d99700 1 -- 172.21.15.31:0/139542673 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c011790 con 0x7fba10159560 2023-10-06T05:21:42.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.391+0000 7fba14d99700 1 -- 172.21.15.31:0/139542673 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c011b30 con 0x7fba10159560 2023-10-06T05:21:42.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.391+0000 7fba17fff700 1 -- 172.21.15.31:0/139542673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba10159560 msgr2=0x7fba100cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.391+0000 7fba17fff700 1 --2- 172.21.15.31:0/139542673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba10159560 0x7fba100cce70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0098e0 tx=0x7fba0c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.391+0000 7fba17fff700 1 -- 172.21.15.31:0/139542673 shutdown_connections 2023-10-06T05:21:42.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.391+0000 7fba17fff700 1 --2- 172.21.15.31:0/139542673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba10159560 0x7fba100cce70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.391+0000 7fba17fff700 1 -- 172.21.15.31:0/139542673 >> 172.21.15.31:0/139542673 conn(0x7fba1006ec90 msgr2=0x7fba1006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:42.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.392+0000 7fba17fff700 1 -- 172.21.15.31:0/139542673 shutdown_connections 2023-10-06T05:21:42.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.392+0000 7fba17fff700 1 -- 172.21.15.31:0/139542673 wait complete. 2023-10-06T05:21:42.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.393+0000 7fba17fff700 1 Processor -- start 2023-10-06T05:21:42.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.393+0000 7fba17fff700 1 -- start start 2023-10-06T05:21:42.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba17fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba100b2b80 0x7fba100c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba17fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba10159560 0x7fba100cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba17fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba10159930 0x7fba100cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba17fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba100b4b20 con 0x7fba10159930 2023-10-06T05:21:42.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba17fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba10155890 con 0x7fba10159560 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba17fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba100b3c00 con 0x7fba100b2b80 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba1659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba10159930 0x7fba100cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba1659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba10159930 0x7fba100cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58988/0 (socket says 172.21.15.31:58988) 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba1659c700 1 -- 172.21.15.31:0/3991075234 learned_addr learned my addr 172.21.15.31:0/3991075234 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba1559a700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba10159560 0x7fba100cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.394+0000 7fba15d9b700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba100b2b80 0x7fba100c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba1659c700 1 -- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba100b2b80 msgr2=0x7fba100c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba1659c700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba100b2b80 0x7fba100c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba1659c700 1 -- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba10159560 msgr2=0x7fba100cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba1659c700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba10159560 0x7fba100cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:42.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba1659c700 1 -- 172.21.15.31:0/3991075234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c009420 con 0x7fba10159930 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba1659c700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba10159930 0x7fba100cfd80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fba0800cd00 tx=0x7fba08001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba05ffb700 1 -- 172.21.15.31:0/3991075234 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0800ce00 con 0x7fba10159930 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba05ffb700 1 -- 172.21.15.31:0/3991075234 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0800aac0 con 0x7fba10159930 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba17fff700 1 -- 172.21.15.31:0/3991075234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba100b3ea0 con 0x7fba10159930 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba05ffb700 1 -- 172.21.15.31:0/3991075234 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0800ae80 con 0x7fba10159930 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.395+0000 7fba17fff700 1 -- 172.21.15.31:0/3991075234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba100b4460 con 0x7fba10159930 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.397+0000 7fba17fff700 1 -- 172.21.15.31:0/3991075234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba100b4c60 con 0x7fba10159930 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.397+0000 7fba05ffb700 1 -- 172.21.15.31:0/3991075234 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 42643+0+0 (secure 0 0 0) 0x7fba08007300 con 0x7fba10159930 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.397+0000 7fba05ffb700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9f4035900 0x7fb9f4037db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.397+0000 7fba05ffb700 1 -- 172.21.15.31:0/3991075234 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6703+0+0 (secure 0 0 0) 0x7fba08048a90 con 0x7fba10159930 2023-10-06T05:21:42.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.398+0000 7fba15d9b700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9f4035900 0x7fb9f4037db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:42.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.398+0000 7fba15d9b700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9f4035900 0x7fb9f4037db0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0093f0 tx=0x7fba0c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:42.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.402+0000 7fba05ffb700 1 -- 172.21.15.31:0/3991075234 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba100b4c60 con 0x7fba10159930 2023-10-06T05:21:42.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.471+0000 7fba05ffb700 1 -- 172.21.15.31:0/3991075234 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fba0801b780 con 0x7fba10159930 2023-10-06T05:21:42.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:42.652+0000 7fba17fff700 1 -- 172.21.15.31:0/3991075234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fba100b4c60 con 0x7fba10159930 2023-10-06T05:21:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.129+0000 7fba05ffb700 1 -- 172.21.15.31:0/3991075234 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7fba100b4c60 con 0x7fba10159930 2023-10-06T05:21:43.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 -- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9f4035900 msgr2=0x7fb9f4037db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9f4035900 0x7fb9f4037db0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0093f0 tx=0x7fba0c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:43.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 -- 172.21.15.31:0/3991075234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba10159930 msgr2=0x7fba100cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba10159930 0x7fba100cfd80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fba0800cd00 tx=0x7fba08001ac0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 -- 172.21.15.31:0/3991075234 shutdown_connections 2023-10-06T05:21:43.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9f4035900 0x7fb9f4037db0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba10159930 0x7fba100cfd80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba100b2b80 0x7fba100c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 --2- 172.21.15.31:0/3991075234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba10159560 0x7fba100cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 -- 172.21.15.31:0/3991075234 >> 172.21.15.31:0/3991075234 conn(0x7fba1006ec90 msgr2=0x7fba10159d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:43.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 -- 172.21.15.31:0/3991075234 shutdown_connections 2023-10-06T05:21:43.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.132+0000 7fba17fff700 1 -- 172.21.15.31:0/3991075234 wait complete. 2023-10-06T05:21:43.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:21:43.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.351+0000 7fa579bff700 1 Processor -- start 2023-10-06T05:21:43.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.351+0000 7fa579bff700 1 -- start start 2023-10-06T05:21:43.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.352+0000 7fa579bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa574159410 0x7fa5740c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.352+0000 7fa579bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5740c73c0 0x7fa57406adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.352+0000 7fa579bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5740c7830 con 0x7fa57406b2f0 2023-10-06T05:21:43.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.352+0000 7fa579bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa574069750 con 0x7fa5740c73c0 2023-10-06T05:21:43.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.352+0000 7fa579bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa574069a30 con 0x7fa574159410 2023-10-06T05:21:43.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.352+0000 7fa572ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5740c73c0 0x7fa57406adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.352+0000 7fa572ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5740c73c0 0x7fa57406adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58996/0 (socket says 172.21.15.31:58996) 2023-10-06T05:21:43.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.352+0000 7fa572ffd700 1 -- 172.21.15.31:0/234144662 learned_addr learned my addr 172.21.15.31:0/234144662 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:43.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.353+0000 7fa5737fe700 1 --2- 172.21.15.31:0/234144662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa574159410 0x7fa5740c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.353+0000 7fa572ffd700 1 -- 172.21.15.31:0/234144662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa574159410 msgr2=0x7fa5740c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:43.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.353+0000 7fa572ffd700 1 --2- 172.21.15.31:0/234144662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa574159410 0x7fa5740c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.353+0000 7fa572ffd700 1 -- 172.21.15.31:0/234144662 >> v1:172.21.15.62:6789/0 conn(0x7fa57406b2f0 legacy=0x7fa574067560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:43.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.353+0000 7fa572ffd700 1 -- 172.21.15.31:0/234144662 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa574069d10 con 0x7fa5740c73c0 2023-10-06T05:21:43.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.353+0000 7fa572ffd700 1 --2- 172.21.15.31:0/234144662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5740c73c0 0x7fa57406adb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa5680099b0 tx=0x7fa568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f303230ad9efd67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:43.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.353+0000 7fa5727fc700 1 -- 172.21.15.31:0/234144662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800b860 con 0x7fa5740c73c0 2023-10-06T05:21:43.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.354+0000 7fa5727fc700 1 -- 172.21.15.31:0/234144662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56800b9c0 con 0x7fa5740c73c0 2023-10-06T05:21:43.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.354+0000 7fa5727fc700 1 -- 172.21.15.31:0/234144662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800bd60 con 0x7fa5740c73c0 2023-10-06T05:21:43.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.354+0000 7fa579bff700 1 -- 172.21.15.31:0/234144662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5740c73c0 msgr2=0x7fa57406adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.354+0000 7fa579bff700 1 --2- 172.21.15.31:0/234144662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5740c73c0 0x7fa57406adb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa5680099b0 tx=0x7fa568009250 comp rx=0 tx=0).stop 2023-10-06T05:21:43.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.354+0000 7fa579bff700 1 -- 172.21.15.31:0/234144662 shutdown_connections 2023-10-06T05:21:43.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.354+0000 7fa579bff700 1 --2- 172.21.15.31:0/234144662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa574159410 0x7fa5740c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.354+0000 7fa579bff700 1 --2- 172.21.15.31:0/234144662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5740c73c0 0x7fa57406adb0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.354+0000 7fa579bff700 1 -- 172.21.15.31:0/234144662 >> 172.21.15.31:0/234144662 conn(0x7fa5740c9350 msgr2=0x7fa5740c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:43.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.354+0000 7fa579bff700 1 -- 172.21.15.31:0/234144662 shutdown_connections 2023-10-06T05:21:43.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.354+0000 7fa579bff700 1 -- 172.21.15.31:0/234144662 wait complete. 2023-10-06T05:21:43.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.356+0000 7fa579bff700 1 Processor -- start 2023-10-06T05:21:43.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.356+0000 7fa579bff700 1 -- start start 2023-10-06T05:21:43.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa579bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57406b2f0 0x7fa5740af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa579bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5740c73c0 0x7fa5740c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa579bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa574159410 0x7fa574153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa579bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740b10a0 con 0x7fa57406b2f0 2023-10-06T05:21:43.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa579bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740b0c10 con 0x7fa5740c73c0 2023-10-06T05:21:43.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa579bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740b0000 con 0x7fa574159410 2023-10-06T05:21:43.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa5737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57406b2f0 0x7fa5740af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa5737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57406b2f0 0x7fa5740af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59012/0 (socket says 172.21.15.31:59012) 2023-10-06T05:21:43.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa5737fe700 1 -- 172.21.15.31:0/3181539763 learned_addr learned my addr 172.21.15.31:0/3181539763 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:43.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa572ffd700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5740c73c0 0x7fa5740c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa573fff700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa574159410 0x7fa574153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa5737fe700 1 -- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa574159410 msgr2=0x7fa574153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa5737fe700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa574159410 0x7fa574153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa5737fe700 1 -- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5740c73c0 msgr2=0x7fa5740c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:43.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa5737fe700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5740c73c0 0x7fa5740c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:43.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.357+0000 7fa5737fe700 1 -- 172.21.15.31:0/3181539763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680094f0 con 0x7fa57406b2f0 2023-10-06T05:21:43.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.358+0000 7fa5737fe700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57406b2f0 0x7fa5740af690 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa56400d890 tx=0x7fa564008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:43.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.358+0000 7fa55b7fe700 1 -- 172.21.15.31:0/3181539763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640125f0 con 0x7fa57406b2f0 2023-10-06T05:21:43.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.358+0000 7fa55b7fe700 1 -- 172.21.15.31:0/3181539763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564012750 con 0x7fa57406b2f0 2023-10-06T05:21:43.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.358+0000 7fa579bff700 1 -- 172.21.15.31:0/3181539763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5740b02e0 con 0x7fa57406b2f0 2023-10-06T05:21:43.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.358+0000 7fa55b7fe700 1 -- 172.21.15.31:0/3181539763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564012af0 con 0x7fa57406b2f0 2023-10-06T05:21:43.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.358+0000 7fa579bff700 1 -- 172.21.15.31:0/3181539763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5740b08a0 con 0x7fa57406b2f0 2023-10-06T05:21:43.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.359+0000 7fa579bff700 1 -- 172.21.15.31:0/3181539763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5740b0ef0 con 0x7fa57406b2f0 2023-10-06T05:21:43.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.360+0000 7fa55b7fe700 1 -- 172.21.15.31:0/3181539763 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fa56401e460 con 0x7fa57406b2f0 2023-10-06T05:21:43.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.361+0000 7fa55b7fe700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa560067050 0x7fa560069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:43.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.361+0000 7fa55b7fe700 1 -- 172.21.15.31:0/3181539763 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6703+0+0 (secure 0 0 0) 0x7fa564085050 con 0x7fa57406b2f0 2023-10-06T05:21:43.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.362+0000 7fa572ffd700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa560067050 0x7fa560069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:43.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.362+0000 7fa572ffd700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa560067050 0x7fa560069500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa5680099b0 tx=0x7fa568017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:43.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.365+0000 7fa55b7fe700 1 -- 172.21.15.31:0/3181539763 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa564051d30 con 0x7fa57406b2f0 2023-10-06T05:21:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:43.570+0000 7fa579bff700 1 -- 172.21.15.31:0/3181539763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa57404c380 con 0x7fa57406b2f0 2023-10-06T05:21:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.475+0000 7fa55b7fe700 1 -- 172.21.15.31:0/3181539763 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa564055350 con 0x7fa57406b2f0 2023-10-06T05:21:44.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.482+0000 7fa579bff700 1 -- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa560067050 msgr2=0x7fa560069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.482+0000 7fa579bff700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa560067050 0x7fa560069500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa5680099b0 tx=0x7fa568017000 comp rx=0 tx=0).stop 2023-10-06T05:21:44.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.482+0000 7fa579bff700 1 -- 172.21.15.31:0/3181539763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57406b2f0 msgr2=0x7fa5740af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.482+0000 7fa579bff700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57406b2f0 0x7fa5740af690 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa56400d890 tx=0x7fa564008040 comp rx=0 tx=0).stop 2023-10-06T05:21:44.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.482+0000 7fa579bff700 1 -- 172.21.15.31:0/3181539763 shutdown_connections 2023-10-06T05:21:44.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.482+0000 7fa579bff700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa560067050 0x7fa560069500 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.482+0000 7fa579bff700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa574159410 0x7fa574153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.482+0000 7fa579bff700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa57406b2f0 0x7fa5740af690 secure :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa56400d890 tx=0x7fa564008040 comp rx=0 tx=0).stop 2023-10-06T05:21:44.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.482+0000 7fa579bff700 1 --2- 172.21.15.31:0/3181539763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5740c73c0 0x7fa5740c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.482+0000 7fa579bff700 1 -- 172.21.15.31:0/3181539763 >> 172.21.15.31:0/3181539763 conn(0x7fa5740c9350 msgr2=0x7fa5740cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:44.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.483+0000 7fa579bff700 1 -- 172.21.15.31:0/3181539763 shutdown_connections 2023-10-06T05:21:44.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.483+0000 7fa579bff700 1 -- 172.21.15.31:0/3181539763 wait complete. 2023-10-06T05:21:44.487 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:21:44.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:21:44.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.711+0000 7f72c9fff700 1 Processor -- start 2023-10-06T05:21:44.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.712+0000 7f72c9fff700 1 -- start start 2023-10-06T05:21:44.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.713+0000 7f72c9fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f72c40c8510 0x7f72c40a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.713+0000 7f72c9fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c6a60 0x7f72c40c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.713+0000 7f72c9fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c4157950 con 0x7f72c4157580 2023-10-06T05:21:44.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.713+0000 7f72c9fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c40c0de0 con 0x7f72c40c8510 2023-10-06T05:21:44.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.713+0000 7f72c9fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40c1680 con 0x7f72c40c6a60 2023-10-06T05:21:44.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.713+0000 7f72c3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c6a60 0x7f72c40c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.713+0000 7f72c2ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f72c40c8510 0x7f72c40a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.713+0000 7f72c3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c6a60 0x7f72c40c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:55696/0 (socket says 172.21.15.31:55696) 2023-10-06T05:21:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.713+0000 7f72c3fff700 1 -- 172.21.15.31:0/852774429 learned_addr learned my addr 172.21.15.31:0/852774429 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.713+0000 7f72c37fe700 1 --1- 172.21.15.31:0/852774429 >> v1:172.21.15.62:6789/0 conn(0x7f72c4157580 0x7f72c40c7e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:40978/0 (socket says 172.21.15.31:40978) 2023-10-06T05:21:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.714+0000 7f72c2ffd700 1 -- 172.21.15.31:0/852774429 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c6a60 msgr2=0x7f72c40c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.714+0000 7f72c2ffd700 1 --2- 172.21.15.31:0/852774429 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c6a60 0x7f72c40c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.714+0000 7f72c2ffd700 1 -- 172.21.15.31:0/852774429 >> v1:172.21.15.62:6789/0 conn(0x7f72c4157580 legacy=0x7f72c40c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.714+0000 7f72c2ffd700 1 -- 172.21.15.31:0/852774429 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c40c77d0 con 0x7f72c40c8510 2023-10-06T05:21:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.714+0000 7f72c3fff700 1 --2- 172.21.15.31:0/852774429 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c6a60 0x7f72c40c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.714+0000 7f72c2ffd700 1 --2- 172.21.15.31:0/852774429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72c40c8510 0x7f72c40a8d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f72b80098e0 tx=0x7f72b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12586bf28ccca011 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:44.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.714+0000 7f72c27fc700 1 -- 172.21.15.31:0/852774429 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800b640 con 0x7f72c40c8510 2023-10-06T05:21:44.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.715+0000 7f72c27fc700 1 -- 172.21.15.31:0/852774429 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800b7a0 con 0x7f72c40c8510 2023-10-06T05:21:44.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.715+0000 7f72c27fc700 1 -- 172.21.15.31:0/852774429 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800bb40 con 0x7f72c40c8510 2023-10-06T05:21:44.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.715+0000 7f72c9fff700 1 -- 172.21.15.31:0/852774429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72c40c8510 msgr2=0x7f72c40a8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.715+0000 7f72c9fff700 1 --2- 172.21.15.31:0/852774429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72c40c8510 0x7f72c40a8d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f72b80098e0 tx=0x7f72b80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.715+0000 7f72c9fff700 1 -- 172.21.15.31:0/852774429 shutdown_connections 2023-10-06T05:21:44.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.715+0000 7f72c9fff700 1 --2- 172.21.15.31:0/852774429 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c6a60 0x7f72c40c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.715+0000 7f72c9fff700 1 --2- 172.21.15.31:0/852774429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72c40c8510 0x7f72c40a8d60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.715+0000 7f72c9fff700 1 -- 172.21.15.31:0/852774429 >> 172.21.15.31:0/852774429 conn(0x7f72c40c8ab0 msgr2=0x7f72c40c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:44.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.715+0000 7f72c9fff700 1 -- 172.21.15.31:0/852774429 shutdown_connections 2023-10-06T05:21:44.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.715+0000 7f72c9fff700 1 -- 172.21.15.31:0/852774429 wait complete. 2023-10-06T05:21:44.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.717+0000 7f72c9fff700 1 Processor -- start 2023-10-06T05:21:44.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.717+0000 7f72c9fff700 1 -- start start 2023-10-06T05:21:44.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c9fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72c40c6a60 0x7f72c40c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c9fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c8510 0x7f72c4155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c9fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72c4157580 0x7f72c4155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c9fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40aede0 con 0x7f72c4157580 2023-10-06T05:21:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c9fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40af060 con 0x7f72c40c6a60 2023-10-06T05:21:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c9fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40af340 con 0x7f72c40c8510 2023-10-06T05:21:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c2ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c8510 0x7f72c4155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72c4157580 0x7f72c4155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72c40c6a60 0x7f72c40c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c2ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c8510 0x7f72c4155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:55704/0 (socket says 172.21.15.31:55704) 2023-10-06T05:21:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.718+0000 7f72c2ffd700 1 -- 172.21.15.31:0/1445299117 learned_addr learned my addr 172.21.15.31:0/1445299117 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72c2ffd700 1 -- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72c40c6a60 msgr2=0x7f72c40c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72c2ffd700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72c40c6a60 0x7f72c40c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72c2ffd700 1 -- 172.21.15.31:0/1445299117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72c4157580 msgr2=0x7f72c4155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72c2ffd700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72c4157580 0x7f72c4155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72c2ffd700 1 -- 172.21.15.31:0/1445299117 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc009ae0 con 0x7f72c40c8510 2023-10-06T05:21:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72c3fff700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72c4157580 0x7f72c4155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72c2ffd700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c8510 0x7f72c4155230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f72c406c9b0 tx=0x7f72b8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72ab7fe700 1 -- 172.21.15.31:0/1445299117 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b801bc40 con 0x7f72c40c8510 2023-10-06T05:21:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72ab7fe700 1 -- 172.21.15.31:0/1445299117 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b801bde0 con 0x7f72c40c8510 2023-10-06T05:21:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72ab7fe700 1 -- 172.21.15.31:0/1445299117 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8011670 con 0x7f72c40c8510 2023-10-06T05:21:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.719+0000 7f72c9fff700 1 -- 172.21.15.31:0/1445299117 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b8009420 con 0x7f72c40c8510 2023-10-06T05:21:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.720+0000 7f72c9fff700 1 -- 172.21.15.31:0/1445299117 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c40af900 con 0x7f72c40c8510 2023-10-06T05:21:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.721+0000 7f72c9fff700 1 -- 172.21.15.31:0/1445299117 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c404c380 con 0x7f72c40c8510 2023-10-06T05:21:44.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.722+0000 7f72ab7fe700 1 -- 172.21.15.31:0/1445299117 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f72b80119e0 con 0x7f72c40c8510 2023-10-06T05:21:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.722+0000 7f72ab7fe700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72b0067120 0x7f72b00695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.722+0000 7f72ab7fe700 1 -- 172.21.15.31:0/1445299117 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f72b8085f10 con 0x7f72c40c8510 2023-10-06T05:21:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.723+0000 7f72c37fe700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72b0067120 0x7f72b00695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.723+0000 7f72c37fe700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72b0067120 0x7f72b00695d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f72b4004fd0 tx=0x7f72b4007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.728+0000 7f72ab7fe700 1 -- 172.21.15.31:0/1445299117 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72b8052e20 con 0x7f72c40c8510 2023-10-06T05:21:44.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:44.930+0000 7f72c9fff700 1 -- 172.21.15.31:0/1445299117 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f72c40af480 con 0x7f72c40c8510 2023-10-06T05:21:45.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.487+0000 7f72ab7fe700 1 -- 172.21.15.31:0/1445299117 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) v1 ==== 169+0+0 (secure 0 0 0) 0x7f72c40af480 con 0x7f72c40c8510 2023-10-06T05:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.492+0000 7f72c9fff700 1 -- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72b0067120 msgr2=0x7f72b00695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.492+0000 7f72c9fff700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72b0067120 0x7f72b00695d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f72b4004fd0 tx=0x7f72b4007eb0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.492+0000 7f72c9fff700 1 -- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c8510 msgr2=0x7f72c4155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.492+0000 7f72c9fff700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c8510 0x7f72c4155230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f72c406c9b0 tx=0x7f72b8017000 comp rx=0 tx=0).stop 2023-10-06T05:21:45.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.493+0000 7f72c9fff700 1 -- 172.21.15.31:0/1445299117 shutdown_connections 2023-10-06T05:21:45.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.493+0000 7f72c9fff700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72b0067120 0x7f72b00695d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.493+0000 7f72c9fff700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72c4157580 0x7f72c4155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.493+0000 7f72c9fff700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72c40c6a60 0x7f72c40c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.493+0000 7f72c9fff700 1 --2- 172.21.15.31:0/1445299117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72c40c8510 0x7f72c4155230 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.493+0000 7f72c9fff700 1 -- 172.21.15.31:0/1445299117 >> 172.21.15.31:0/1445299117 conn(0x7f72c40c8ab0 msgr2=0x7f72c40692c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:45.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.494+0000 7f72c9fff700 1 -- 172.21.15.31:0/1445299117 shutdown_connections 2023-10-06T05:21:45.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.494+0000 7f72c9fff700 1 -- 172.21.15.31:0/1445299117 wait complete. 2023-10-06T05:21:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:21:45.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:21:45.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.719+0000 7fd9fbbff700 1 Processor -- start 2023-10-06T05:21:45.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.720+0000 7fd9fbbff700 1 -- start start 2023-10-06T05:21:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.721+0000 7fd9fbbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd9f4159890 0x7fd9f40c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.721+0000 7fd9fbbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9f4155430 0x7fd9f4155890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9fbbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f4158190 con 0x7fd9f4159c60 2023-10-06T05:21:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9fbbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f4157580 con 0x7fd9f4155430 2023-10-06T05:21:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9fbbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f4157860 con 0x7fd9f4159890 2023-10-06T05:21:45.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9fa19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9f4155430 0x7fd9f4155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9fa19c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9f4155430 0x7fd9f4155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59038/0 (socket says 172.21.15.31:59038) 2023-10-06T05:21:45.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9fa19c700 1 -- 172.21.15.31:0/1882775993 learned_addr learned my addr 172.21.15.31:0/1882775993 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:45.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9f999b700 1 --2- 172.21.15.31:0/1882775993 >> v2:172.21.15.62:3300/0 conn(0x7fd9f4159890 0x7fd9f40c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9fa19c700 1 -- 172.21.15.31:0/1882775993 >> v2:172.21.15.62:3300/0 conn(0x7fd9f4159890 msgr2=0x7fd9f40c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:45.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9fa19c700 1 --2- 172.21.15.31:0/1882775993 >> v2:172.21.15.62:3300/0 conn(0x7fd9f4159890 0x7fd9f40c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9fa19c700 1 -- 172.21.15.31:0/1882775993 >> v1:172.21.15.62:6789/0 conn(0x7fd9f4159c60 legacy=0x7fd9f4156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.722+0000 7fd9fa19c700 1 -- 172.21.15.31:0/1882775993 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f4157b40 con 0x7fd9f4155430 2023-10-06T05:21:45.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.723+0000 7fd9fa19c700 1 --2- 172.21.15.31:0/1882775993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 0x7fd9f4155890 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4064120 tx=0x7fd9ec00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5983c24a4f0aab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:45.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.723+0000 7fd9f8999700 1 -- 172.21.15.31:0/1882775993 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec010070 con 0x7fd9f4155430 2023-10-06T05:21:45.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.723+0000 7fd9f8999700 1 -- 172.21.15.31:0/1882775993 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec00dc80 con 0x7fd9f4155430 2023-10-06T05:21:45.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.723+0000 7fd9f8999700 1 -- 172.21.15.31:0/1882775993 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec016400 con 0x7fd9f4155430 2023-10-06T05:21:45.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.724+0000 7fd9fbbff700 1 -- 172.21.15.31:0/1882775993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 msgr2=0x7fd9f4155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.724+0000 7fd9fbbff700 1 --2- 172.21.15.31:0/1882775993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 0x7fd9f4155890 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4064120 tx=0x7fd9ec00ba50 comp rx=0 tx=0).stop 2023-10-06T05:21:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.724+0000 7fd9fbbff700 1 -- 172.21.15.31:0/1882775993 shutdown_connections 2023-10-06T05:21:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.724+0000 7fd9fbbff700 1 --2- 172.21.15.31:0/1882775993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 0x7fd9f4155890 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.724+0000 7fd9fbbff700 1 --2- 172.21.15.31:0/1882775993 >> v2:172.21.15.62:3300/0 conn(0x7fd9f4159890 0x7fd9f40c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.724+0000 7fd9fbbff700 1 -- 172.21.15.31:0/1882775993 >> 172.21.15.31:0/1882775993 conn(0x7fd9f4067560 msgr2=0x7fd9f4067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.724+0000 7fd9fbbff700 1 -- 172.21.15.31:0/1882775993 shutdown_connections 2023-10-06T05:21:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.724+0000 7fd9fbbff700 1 -- 172.21.15.31:0/1882775993 wait complete. 2023-10-06T05:21:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.725+0000 7fd9fbbff700 1 Processor -- start 2023-10-06T05:21:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.725+0000 7fd9fbbff700 1 -- start start 2023-10-06T05:21:45.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.726+0000 7fd9fbbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 0x7fd9f415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.726+0000 7fd9fbbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f4159890 0x7fd9f41502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.726+0000 7fd9fbbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9f4159c60 0x7fd9f4150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.726+0000 7fd9fbbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f4150d50 con 0x7fd9f4155430 2023-10-06T05:21:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.726+0000 7fd9fbbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f40c66a0 con 0x7fd9f4159890 2023-10-06T05:21:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.726+0000 7fd9fbbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f40cb7c0 con 0x7fd9f4159c60 2023-10-06T05:21:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.726+0000 7fd9f999b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 0x7fd9f415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9f999b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 0x7fd9f415aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59046/0 (socket says 172.21.15.31:59046) 2023-10-06T05:21:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9f999b700 1 -- 172.21.15.31:0/2696778622 learned_addr learned my addr 172.21.15.31:0/2696778622 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9fa19c700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9f4159c60 0x7fd9f4150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9f919a700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f4159890 0x7fd9f41502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9f999b700 1 -- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9f4159c60 msgr2=0x7fd9f4150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9f999b700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9f4159c60 0x7fd9f4150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9f999b700 1 -- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f4159890 msgr2=0x7fd9f41502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9f999b700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f4159890 0x7fd9f41502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9f999b700 1 -- 172.21.15.31:0/2696778622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec00bcf0 con 0x7fd9f4155430 2023-10-06T05:21:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9f999b700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 0x7fd9f415aa20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000d890 tx=0x7fd9f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9e9ffb700 1 -- 172.21.15.31:0/2696778622 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00125f0 con 0x7fd9f4155430 2023-10-06T05:21:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9e9ffb700 1 -- 172.21.15.31:0/2696778622 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f0012770 con 0x7fd9f4155430 2023-10-06T05:21:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9e9ffb700 1 -- 172.21.15.31:0/2696778622 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0012b30 con 0x7fd9f4155430 2023-10-06T05:21:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.727+0000 7fd9fbbff700 1 -- 172.21.15.31:0/2696778622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f40c68f0 con 0x7fd9f4155430 2023-10-06T05:21:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.728+0000 7fd9fbbff700 1 -- 172.21.15.31:0/2696778622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f40c6eb0 con 0x7fd9f4155430 2023-10-06T05:21:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.729+0000 7fd9fbbff700 1 -- 172.21.15.31:0/2696778622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f404c380 con 0x7fd9f4155430 2023-10-06T05:21:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.730+0000 7fd9e9ffb700 1 -- 172.21.15.31:0/2696778622 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd9f001e460 con 0x7fd9f4155430 2023-10-06T05:21:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.730+0000 7fd9e9ffb700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9d8067050 0x7fd9d8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.731+0000 7fd9e9ffb700 1 -- 172.21.15.31:0/2696778622 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fd9f0084e30 con 0x7fd9f4155430 2023-10-06T05:21:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.731+0000 7fd9f919a700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9d8067050 0x7fd9d8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.732+0000 7fd9f919a700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9d8067050 0x7fd9d8069500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40af800 tx=0x7fd9e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.735+0000 7fd9e9ffb700 1 -- 172.21.15.31:0/2696778622 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9f0051f00 con 0x7fd9f4155430 2023-10-06T05:21:45.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.937+0000 7fd9fbbff700 1 -- 172.21.15.31:0/2696778622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd9f4157e20 con 0x7fd9f4155430 2023-10-06T05:21:45.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.941+0000 7fd9e9ffb700 1 -- 172.21.15.31:0/2696778622 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd9f0055520 con 0x7fd9f4155430 2023-10-06T05:21:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 -- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9d8067050 msgr2=0x7fd9d8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9d8067050 0x7fd9d8069500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40af800 tx=0x7fd9e4009040 comp rx=0 tx=0).stop 2023-10-06T05:21:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 -- 172.21.15.31:0/2696778622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 msgr2=0x7fd9f415aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 0x7fd9f415aa20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000d890 tx=0x7fd9f0008040 comp rx=0 tx=0).stop 2023-10-06T05:21:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 -- 172.21.15.31:0/2696778622 shutdown_connections 2023-10-06T05:21:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9d8067050 0x7fd9d8069500 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9f4159c60 0x7fd9f4150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9f4155430 0x7fd9f415aa20 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 --2- 172.21.15.31:0/2696778622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f4159890 0x7fd9f41502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 -- 172.21.15.31:0/2696778622 >> 172.21.15.31:0/2696778622 conn(0x7fd9f4067560 msgr2=0x7fd9f4158620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 -- 172.21.15.31:0/2696778622 shutdown_connections 2023-10-06T05:21:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:45.943+0000 7fd9fbbff700 1 -- 172.21.15.31:0/2696778622 wait complete. 2023-10-06T05:21:45.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2023-10-06T05:21:46.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.161+0000 7f84285ff700 1 Processor -- start 2023-10-06T05:21:46.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.162+0000 7f84285ff700 1 -- start start 2023-10-06T05:21:46.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.163+0000 7f84285ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84200cdfe0 0x7f84200cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.163+0000 7f84285ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84200cf280 con 0x7f84200ccc80 2023-10-06T05:21:46.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.163+0000 7f84285ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84200ce670 con 0x7f8420159580 2023-10-06T05:21:46.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.163+0000 7f84285ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84200ce450 con 0x7f84200cdfe0 2023-10-06T05:21:46.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.163+0000 7f8426b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f84200ccc80 0x7f84200cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54372/0 (socket says 172.21.15.31:54372) 2023-10-06T05:21:46.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.163+0000 7f8426b9c700 1 -- 172.21.15.31:0/3231585968 learned_addr learned my addr 172.21.15.31:0/3231585968 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.163+0000 7f8425b9a700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84200cdfe0 0x7f84200cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.163+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692322272 0 0) 0x7f84200cf280 con 0x7f84200ccc80 2023-10-06T05:21:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.163+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f840c003570 con 0x7f84200ccc80 2023-10-06T05:21:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694326263 0 0) 0x7f840c003570 con 0x7f84200ccc80 2023-10-06T05:21:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84200cdfe0 msgr2=0x7f84200cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84200cdfe0 0x7f84200cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 >> v1:172.21.15.62:6789/0 conn(0x7f8420159580 legacy=0x7f84200cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200ce950 con 0x7f84200ccc80 2023-10-06T05:21:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f84180017f0 con 0x7f84200ccc80 2023-10-06T05:21:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 >> v1:172.21.15.31:6789/0 conn(0x7f84200ccc80 legacy=0x7f84200cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84200cdfe0 0x7f840c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f840c003fd0 0x7f840c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c004d50 0x7f840c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84200ce450 con 0x7f840c003fd0 2023-10-06T05:21:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c00bba0 con 0x7f840c004d50 2023-10-06T05:21:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c00bd60 con 0x7f84200cdfe0 2023-10-06T05:21:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8425b9a700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f840c003fd0 0x7f840c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f842639b700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84200cdfe0 0x7f840c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.164+0000 7f8426b9c700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c004d50 0x7f840c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.165+0000 7f8425b9a700 1 -- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84200cdfe0 msgr2=0x7f840c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.165+0000 7f8425b9a700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84200cdfe0 0x7f840c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.165+0000 7f8425b9a700 1 -- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c004d50 msgr2=0x7f840c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.165+0000 7f8425b9a700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c004d50 0x7f840c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.165+0000 7f8425b9a700 1 -- 172.21.15.31:0/3231585968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c003570 con 0x7f840c003fd0 2023-10-06T05:21:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.165+0000 7f8425b9a700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f840c003fd0 0x7f840c004810 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8410009c00 tx=0x7f84100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e1e0cd53548d70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.165+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841000e710 con 0x7f840c003fd0 2023-10-06T05:21:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.165+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841000e870 con 0x7f840c003fd0 2023-10-06T05:21:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.165+0000 7f8425399700 1 -- 172.21.15.31:0/3231585968 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841000ec30 con 0x7f840c003fd0 2023-10-06T05:21:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.166+0000 7f84285ff700 1 -- 172.21.15.31:0/3231585968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f840c003fd0 msgr2=0x7f840c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.166+0000 7f84285ff700 1 --2- 172.21.15.31:0/3231585968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f840c003fd0 0x7f840c004810 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8410009c00 tx=0x7f84100091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.166+0000 7f8426b9c700 1 -- 172.21.15.31:0/3231585968 reap_dead start 2023-10-06T05:21:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.166+0000 7f84285ff700 1 -- 172.21.15.31:0/3231585968 shutdown_connections 2023-10-06T05:21:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.166+0000 7f84285ff700 1 -- 172.21.15.31:0/3231585968 >> 172.21.15.31:0/3231585968 conn(0x7f8420099710 msgr2=0x7f842006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.166+0000 7f84285ff700 1 -- 172.21.15.31:0/3231585968 shutdown_connections 2023-10-06T05:21:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.166+0000 7f84285ff700 1 -- 172.21.15.31:0/3231585968 wait complete. 2023-10-06T05:21:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.167+0000 7f84285ff700 1 Processor -- start 2023-10-06T05:21:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.168+0000 7f84285ff700 1 -- start start 2023-10-06T05:21:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.168+0000 7f84285ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c003fd0 0x7f84200c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.168+0000 7f84285ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8420158350 0x7f84201587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.168+0000 7f84285ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8420159d00 0x7f842015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.168+0000 7f84285ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842015e2a0 con 0x7f8420159d00 2023-10-06T05:21:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.168+0000 7f84285ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842015a6b0 con 0x7f840c003fd0 2023-10-06T05:21:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.168+0000 7f84285ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8420158ce0 con 0x7f8420158350 2023-10-06T05:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8426b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8420159d00 0x7f842015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8426b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8420159d00 0x7f842015a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59070/0 (socket says 172.21.15.31:59070) 2023-10-06T05:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8426b9c700 1 -- 172.21.15.31:0/2759027661 learned_addr learned my addr 172.21.15.31:0/2759027661 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8425b9a700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8420158350 0x7f84201587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f842639b700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c003fd0 0x7f84200c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8426b9c700 1 -- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8420158350 msgr2=0x7f84201587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8426b9c700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8420158350 0x7f84201587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8426b9c700 1 -- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c003fd0 msgr2=0x7f84200c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8426b9c700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c003fd0 0x7f84200c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8426b9c700 1 -- 172.21.15.31:0/2759027661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84201590c0 con 0x7f8420159d00 2023-10-06T05:21:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8425b9a700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8420158350 0x7f84201587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f842639b700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c003fd0 0x7f84200c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.169+0000 7f8426b9c700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8420159d00 0x7f842015a170 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f84200889b0 tx=0x7f8418002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.170+0000 7f84167fc700 1 -- 172.21.15.31:0/2759027661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800ac30 con 0x7f8420159d00 2023-10-06T05:21:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.170+0000 7f84167fc700 1 -- 172.21.15.31:0/2759027661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841800ad90 con 0x7f8420159d00 2023-10-06T05:21:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.170+0000 7f84167fc700 1 -- 172.21.15.31:0/2759027661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418013600 con 0x7f8420159d00 2023-10-06T05:21:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.170+0000 7f84285ff700 1 -- 172.21.15.31:0/2759027661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84200c9b50 con 0x7f8420159d00 2023-10-06T05:21:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.170+0000 7f84285ff700 1 -- 172.21.15.31:0/2759027661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84200ca110 con 0x7f8420159d00 2023-10-06T05:21:46.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.171+0000 7f84285ff700 1 -- 172.21.15.31:0/2759027661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842015e410 con 0x7f8420159d00 2023-10-06T05:21:46.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.172+0000 7f84167fc700 1 -- 172.21.15.31:0/2759027661 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f84180139f0 con 0x7f8420159d00 2023-10-06T05:21:46.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.173+0000 7f84167fc700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8404067050 0x7f8404069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.173+0000 7f84167fc700 1 -- 172.21.15.31:0/2759027661 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f8418085320 con 0x7f8420159d00 2023-10-06T05:21:46.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.174+0000 7f842639b700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8404067050 0x7f8404069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.174+0000 7f842639b700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8404067050 0x7f8404069500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:46.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.178+0000 7f84167fc700 1 -- 172.21.15.31:0/2759027661 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f842015e410 con 0x7f8420159d00 2023-10-06T05:21:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.461+0000 7f84285ff700 1 -- 172.21.15.31:0/2759027661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7f842015e410 con 0x7f8420159d00 2023-10-06T05:21:46.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.483+0000 7f84167fc700 1 -- 172.21.15.31:0/2759027661 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v260) v1 ==== 131+0+0 (secure 0 0 0) 0x7f842015e410 con 0x7f8420159d00 2023-10-06T05:21:46.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.485+0000 7f84285ff700 1 -- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8404067050 msgr2=0x7f8404069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.485+0000 7f84285ff700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8404067050 0x7f8404069500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c009040 comp rx=0 tx=0).stop 2023-10-06T05:21:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.485+0000 7f84285ff700 1 -- 172.21.15.31:0/2759027661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8420159d00 msgr2=0x7f842015a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.485+0000 7f84285ff700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8420159d00 0x7f842015a170 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f84200889b0 tx=0x7f8418002120 comp rx=0 tx=0).stop 2023-10-06T05:21:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.485+0000 7f84285ff700 1 -- 172.21.15.31:0/2759027661 shutdown_connections 2023-10-06T05:21:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.485+0000 7f84285ff700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8404067050 0x7f8404069500 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.485+0000 7f84285ff700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8420159d00 0x7f842015a170 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.485+0000 7f84285ff700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c003fd0 0x7f84200c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.485+0000 7f84285ff700 1 --2- 172.21.15.31:0/2759027661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8420158350 0x7f84201587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.485+0000 7f84285ff700 1 -- 172.21.15.31:0/2759027661 >> 172.21.15.31:0/2759027661 conn(0x7f8420099710 msgr2=0x7f84200cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.486+0000 7f84285ff700 1 -- 172.21.15.31:0/2759027661 shutdown_connections 2023-10-06T05:21:46.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.486+0000 7f84285ff700 1 -- 172.21.15.31:0/2759027661 wait complete. 2023-10-06T05:21:46.505 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-10-06T05:21:46.505 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2023-10-06T05:21:46.505 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:21:46.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2023-10-06T05:21:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.699+0000 7ff05a9ff700 1 Processor -- start 2023-10-06T05:21:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.699+0000 7ff05a9ff700 1 -- start start 2023-10-06T05:21:46.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.701+0000 7ff05a9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff054068d80 0x7ff054159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.701+0000 7ff05a9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff0540cc0d0 0x7ff0540c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.701+0000 7ff05a9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0540c95b0 con 0x7ff0541595a0 2023-10-06T05:21:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.701+0000 7ff05a9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0540c9890 con 0x7ff0540cc0d0 2023-10-06T05:21:46.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.701+0000 7ff05a9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540c9b70 con 0x7ff054068d80 2023-10-06T05:21:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.701+0000 7ff058f9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff0540cc0d0 0x7ff0540c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.701+0000 7ff0537fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff054068d80 0x7ff054159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.701+0000 7ff058f9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff0540cc0d0 0x7ff0540c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:46850/0 (socket says 172.21.15.31:46850) 2023-10-06T05:21:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.701+0000 7ff058f9c700 1 -- 172.21.15.31:0/3678524357 learned_addr learned my addr 172.21.15.31:0/3678524357 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.701+0000 7ff053fff700 1 --1- 172.21.15.31:0/3678524357 >> v1:172.21.15.62:6789/0 conn(0x7ff0541595a0 0x7ff054068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:41002/0 (socket says 172.21.15.31:41002) 2023-10-06T05:21:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.702+0000 7ff058f9c700 1 -- 172.21.15.31:0/3678524357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff054068d80 msgr2=0x7ff054159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.702+0000 7ff058f9c700 1 --2- 172.21.15.31:0/3678524357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff054068d80 0x7ff054159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.702+0000 7ff058f9c700 1 -- 172.21.15.31:0/3678524357 >> v1:172.21.15.62:6789/0 conn(0x7ff0541595a0 legacy=0x7ff054068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.702+0000 7ff058f9c700 1 -- 172.21.15.31:0/3678524357 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540ca130 con 0x7ff0540cc0d0 2023-10-06T05:21:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.702+0000 7ff058f9c700 1 --2- 172.21.15.31:0/3678524357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0540cc0d0 0x7ff0540c8930 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009f20 tx=0x7ff04c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed1c1fdd6e7353e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.702+0000 7ff052ffd700 1 -- 172.21.15.31:0/3678524357 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00e070 con 0x7ff0540cc0d0 2023-10-06T05:21:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.702+0000 7ff052ffd700 1 -- 172.21.15.31:0/3678524357 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c00b9d0 con 0x7ff0540cc0d0 2023-10-06T05:21:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.702+0000 7ff052ffd700 1 -- 172.21.15.31:0/3678524357 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c0092e0 con 0x7ff0540cc0d0 2023-10-06T05:21:46.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.703+0000 7ff05a9ff700 1 -- 172.21.15.31:0/3678524357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0540cc0d0 msgr2=0x7ff0540c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.703+0000 7ff05a9ff700 1 --2- 172.21.15.31:0/3678524357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0540cc0d0 0x7ff0540c8930 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009f20 tx=0x7ff04c009840 comp rx=0 tx=0).stop 2023-10-06T05:21:46.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.703+0000 7ff05a9ff700 1 -- 172.21.15.31:0/3678524357 shutdown_connections 2023-10-06T05:21:46.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.703+0000 7ff05a9ff700 1 --2- 172.21.15.31:0/3678524357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0540cc0d0 0x7ff0540c8930 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.703+0000 7ff05a9ff700 1 --2- 172.21.15.31:0/3678524357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff054068d80 0x7ff054159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.703+0000 7ff05a9ff700 1 -- 172.21.15.31:0/3678524357 >> 172.21.15.31:0/3678524357 conn(0x7ff05406b620 msgr2=0x7ff05406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:46.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.703+0000 7ff05a9ff700 1 -- 172.21.15.31:0/3678524357 shutdown_connections 2023-10-06T05:21:46.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.703+0000 7ff05a9ff700 1 -- 172.21.15.31:0/3678524357 wait complete. 2023-10-06T05:21:46.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.705+0000 7ff05a9ff700 1 Processor -- start 2023-10-06T05:21:46.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.705+0000 7ff05a9ff700 1 -- start start 2023-10-06T05:21:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff05a9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff054068d80 0x7ff05414b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff05a9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0540cc0d0 0x7ff0540c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff05a9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0541595a0 0x7ff0540c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff05a9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05414b880 con 0x7ff054068d80 2023-10-06T05:21:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff05a9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540c6f20 con 0x7ff0540cc0d0 2023-10-06T05:21:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff05a9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540c6bf0 con 0x7ff0541595a0 2023-10-06T05:21:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff053fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff054068d80 0x7ff05414b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff053fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff054068d80 0x7ff05414b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59074/0 (socket says 172.21.15.31:59074) 2023-10-06T05:21:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff053fff700 1 -- 172.21.15.31:0/1517082236 learned_addr learned my addr 172.21.15.31:0/1517082236 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff058f9c700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0541595a0 0x7ff0540c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff0537fe700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0540cc0d0 0x7ff0540c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff053fff700 1 -- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0541595a0 msgr2=0x7ff0540c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff053fff700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0541595a0 0x7ff0540c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff053fff700 1 -- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0540cc0d0 msgr2=0x7ff0540c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff053fff700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0540cc0d0 0x7ff0540c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:46.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.707+0000 7ff053fff700 1 -- 172.21.15.31:0/1517082236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c009ae0 con 0x7ff054068d80 2023-10-06T05:21:46.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.708+0000 7ff053fff700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff054068d80 0x7ff05414b340 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff04400c7c0 tx=0x7ff044007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:46.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.708+0000 7ff03bfff700 1 -- 172.21.15.31:0/1517082236 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044005a70 con 0x7ff054068d80 2023-10-06T05:21:46.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.708+0000 7ff03bfff700 1 -- 172.21.15.31:0/1517082236 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff044005bf0 con 0x7ff054068d80 2023-10-06T05:21:46.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.708+0000 7ff03bfff700 1 -- 172.21.15.31:0/1517082236 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0440070d0 con 0x7ff054068d80 2023-10-06T05:21:46.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.708+0000 7ff05a9ff700 1 -- 172.21.15.31:0/1517082236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05415c400 con 0x7ff054068d80 2023-10-06T05:21:46.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.708+0000 7ff05a9ff700 1 -- 172.21.15.31:0/1517082236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05415c9c0 con 0x7ff054068d80 2023-10-06T05:21:46.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.709+0000 7ff05a9ff700 1 -- 172.21.15.31:0/1517082236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05404c380 con 0x7ff054068d80 2023-10-06T05:21:46.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.710+0000 7ff03bfff700 1 -- 172.21.15.31:0/1517082236 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7ff044007480 con 0x7ff054068d80 2023-10-06T05:21:46.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.711+0000 7ff03bfff700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff040067050 0x7ff040069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.711+0000 7ff03bfff700 1 -- 172.21.15.31:0/1517082236 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5851+0+0 (secure 0 0 0) 0x7ff044083b30 con 0x7ff054068d80 2023-10-06T05:21:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.711+0000 7ff0537fe700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff040067050 0x7ff040069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.712+0000 7ff0537fe700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff040067050 0x7ff040069500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff05415d4e0 tx=0x7ff0480091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.715+0000 7ff03bfff700 1 -- 172.21.15.31:0/1517082236 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff044050b60 con 0x7ff054068d80 2023-10-06T05:21:47.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:46.999+0000 7ff05a9ff700 1 -- 172.21.15.31:0/1517082236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7ff054158c40 con 0x7ff054068d80 2023-10-06T05:21:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.489+0000 7ff03bfff700 1 -- 172.21.15.31:0/1517082236 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v261) v1 ==== 149+0+0 (secure 0 0 0) 0x7ff044017020 con 0x7ff054068d80 2023-10-06T05:21:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.491+0000 7ff05a9ff700 1 -- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff040067050 msgr2=0x7ff040069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.491+0000 7ff05a9ff700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff040067050 0x7ff040069500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff05415d4e0 tx=0x7ff0480091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.491+0000 7ff05a9ff700 1 -- 172.21.15.31:0/1517082236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff054068d80 msgr2=0x7ff05414b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.491+0000 7ff05a9ff700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff054068d80 0x7ff05414b340 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff04400c7c0 tx=0x7ff044007f70 comp rx=0 tx=0).stop 2023-10-06T05:21:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.492+0000 7ff05a9ff700 1 -- 172.21.15.31:0/1517082236 shutdown_connections 2023-10-06T05:21:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.492+0000 7ff05a9ff700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff040067050 0x7ff040069500 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.492+0000 7ff05a9ff700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0541595a0 0x7ff0540c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.492+0000 7ff05a9ff700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff054068d80 0x7ff05414b340 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.492+0000 7ff05a9ff700 1 --2- 172.21.15.31:0/1517082236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0540cc0d0 0x7ff0540c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.492+0000 7ff05a9ff700 1 -- 172.21.15.31:0/1517082236 >> 172.21.15.31:0/1517082236 conn(0x7ff05406b620 msgr2=0x7ff0540ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:47.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.492+0000 7ff05a9ff700 1 -- 172.21.15.31:0/1517082236 shutdown_connections 2023-10-06T05:21:47.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.492+0000 7ff05a9ff700 1 -- 172.21.15.31:0/1517082236 wait complete. 2023-10-06T05:21:47.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:21:47.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.711+0000 7fb9195ff700 1 Processor -- start 2023-10-06T05:21:47.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.711+0000 7fb9195ff700 1 -- start start 2023-10-06T05:21:47.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.712+0000 7fb9195ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb914159560 0x7fb9140cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.712+0000 7fb9195ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb9140b2b80 0x7fb9140cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.712+0000 7fb9195ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91415d2d0 con 0x7fb914159930 2023-10-06T05:21:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.712+0000 7fb9195ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91415d5b0 con 0x7fb914159560 2023-10-06T05:21:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.712+0000 7fb9195ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91415d890 con 0x7fb9140b2b80 2023-10-06T05:21:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.712+0000 7fb912ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb914159560 0x7fb9140cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.712+0000 7fb912ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb914159560 0x7fb9140cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45754/0 (socket says 172.21.15.31:45754) 2023-10-06T05:21:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.712+0000 7fb912ffd700 1 -- 172.21.15.31:0/871246899 learned_addr learned my addr 172.21.15.31:0/871246899 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.712+0000 7fb9127fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb914159930 0x7fb9140b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35730/0 (socket says 172.21.15.31:35730) 2023-10-06T05:21:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.713+0000 7fb912ffd700 1 -- 172.21.15.31:0/871246899 >> v2:172.21.15.62:3300/0 conn(0x7fb9140b2b80 msgr2=0x7fb9140cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.713+0000 7fb912ffd700 1 --2- 172.21.15.31:0/871246899 >> v2:172.21.15.62:3300/0 conn(0x7fb9140b2b80 0x7fb9140cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.713+0000 7fb912ffd700 1 -- 172.21.15.31:0/871246899 >> v1:172.21.15.31:6789/0 conn(0x7fb914159930 legacy=0x7fb9140b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.713+0000 7fb912ffd700 1 -- 172.21.15.31:0/871246899 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91415db70 con 0x7fb914159560 2023-10-06T05:21:47.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.713+0000 7fb912ffd700 1 --2- 172.21.15.31:0/871246899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 0x7fb9140cce70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb9140b1740 tx=0x7fb904009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956325d96b09a226 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.713+0000 7fb911ffb700 1 -- 172.21.15.31:0/871246899 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90400b860 con 0x7fb914159560 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.713+0000 7fb911ffb700 1 -- 172.21.15.31:0/871246899 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90400b9c0 con 0x7fb914159560 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.713+0000 7fb911ffb700 1 -- 172.21.15.31:0/871246899 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90400bd60 con 0x7fb914159560 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.714+0000 7fb9195ff700 1 -- 172.21.15.31:0/871246899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 msgr2=0x7fb9140cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.714+0000 7fb9195ff700 1 --2- 172.21.15.31:0/871246899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 0x7fb9140cce70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb9140b1740 tx=0x7fb904009250 comp rx=0 tx=0).stop 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.714+0000 7fb9195ff700 1 -- 172.21.15.31:0/871246899 shutdown_connections 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.714+0000 7fb9195ff700 1 --2- 172.21.15.31:0/871246899 >> v2:172.21.15.62:3300/0 conn(0x7fb9140b2b80 0x7fb9140cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.714+0000 7fb9195ff700 1 --2- 172.21.15.31:0/871246899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 0x7fb9140cce70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.714+0000 7fb9195ff700 1 -- 172.21.15.31:0/871246899 >> 172.21.15.31:0/871246899 conn(0x7fb91406ec90 msgr2=0x7fb91406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.714+0000 7fb9195ff700 1 -- 172.21.15.31:0/871246899 shutdown_connections 2023-10-06T05:21:47.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.714+0000 7fb9195ff700 1 -- 172.21.15.31:0/871246899 wait complete. 2023-10-06T05:21:47.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.716+0000 7fb9195ff700 1 Processor -- start 2023-10-06T05:21:47.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.716+0000 7fb9195ff700 1 -- start start 2023-10-06T05:21:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9195ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9140b2b80 0x7fb914145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9195ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 0x7fb9140c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9195ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb914159930 0x7fb9140c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9195ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140c8740 con 0x7fb914159560 2023-10-06T05:21:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9195ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140c67c0 con 0x7fb9140b2b80 2023-10-06T05:21:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9195ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140c7b30 con 0x7fb914159930 2023-10-06T05:21:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9127fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 0x7fb9140c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb912ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9140b2b80 0x7fb914145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9127fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 0x7fb9140c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45766/0 (socket says 172.21.15.31:45766) 2023-10-06T05:21:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9127fc700 1 -- 172.21.15.31:0/540831018 learned_addr learned my addr 172.21.15.31:0/540831018 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9137fe700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb914159930 0x7fb9140c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9127fc700 1 -- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb914159930 msgr2=0x7fb9140c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9127fc700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb914159930 0x7fb9140c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9127fc700 1 -- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9140b2b80 msgr2=0x7fb914145780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9127fc700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9140b2b80 0x7fb914145780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.718+0000 7fb9127fc700 1 -- 172.21.15.31:0/540831018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9040094f0 con 0x7fb914159560 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.719+0000 7fb9127fc700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 0x7fb9140c5d40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c560 tx=0x7fb908007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.719+0000 7fb8faffd700 1 -- 172.21.15.31:0/540831018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb908005a70 con 0x7fb914159560 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.719+0000 7fb8faffd700 1 -- 172.21.15.31:0/540831018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb908005bd0 con 0x7fb914159560 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.719+0000 7fb8faffd700 1 -- 172.21.15.31:0/540831018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9080070d0 con 0x7fb914159560 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.719+0000 7fb9195ff700 1 -- 172.21.15.31:0/540831018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9140c7e10 con 0x7fb914159560 2023-10-06T05:21:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.719+0000 7fb9195ff700 1 -- 172.21.15.31:0/540831018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9140c83d0 con 0x7fb914159560 2023-10-06T05:21:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.720+0000 7fb9195ff700 1 -- 172.21.15.31:0/540831018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9140c88f0 con 0x7fb914159560 2023-10-06T05:21:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.721+0000 7fb8faffd700 1 -- 172.21.15.31:0/540831018 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fb908007480 con 0x7fb914159560 2023-10-06T05:21:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.722+0000 7fb8faffd700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb900067050 0x7fb900069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.722+0000 7fb8faffd700 1 -- 172.21.15.31:0/540831018 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fb908083b30 con 0x7fb914159560 2023-10-06T05:21:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.722+0000 7fb912ffd700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb900067050 0x7fb900069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.723+0000 7fb912ffd700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb900067050 0x7fb900069500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb9140b1740 tx=0x7fb904017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:47.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.726+0000 7fb8faffd700 1 -- 172.21.15.31:0/540831018 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9140c88f0 con 0x7fb914159560 2023-10-06T05:21:47.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.928+0000 7fb9195ff700 1 -- 172.21.15.31:0/540831018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb91404c380 con 0x7fb914159560 2023-10-06T05:21:47.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.932+0000 7fb8faffd700 1 -- 172.21.15.31:0/540831018 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb908017020 con 0x7fb914159560 2023-10-06T05:21:47.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.934+0000 7fb9195ff700 1 -- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb900067050 msgr2=0x7fb900069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.934+0000 7fb9195ff700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb900067050 0x7fb900069500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb9140b1740 tx=0x7fb904017000 comp rx=0 tx=0).stop 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.934+0000 7fb9195ff700 1 -- 172.21.15.31:0/540831018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 msgr2=0x7fb9140c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.934+0000 7fb9195ff700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 0x7fb9140c5d40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c560 tx=0x7fb908007f70 comp rx=0 tx=0).stop 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.935+0000 7fb9195ff700 1 -- 172.21.15.31:0/540831018 shutdown_connections 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.935+0000 7fb9195ff700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb900067050 0x7fb900069500 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.935+0000 7fb9195ff700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb914159930 0x7fb9140c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.935+0000 7fb9195ff700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9140b2b80 0x7fb914145780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.935+0000 7fb9195ff700 1 --2- 172.21.15.31:0/540831018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb914159560 0x7fb9140c5d40 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.935+0000 7fb9195ff700 1 -- 172.21.15.31:0/540831018 >> 172.21.15.31:0/540831018 conn(0x7fb91406ec90 msgr2=0x7fb9140cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.935+0000 7fb9195ff700 1 -- 172.21.15.31:0/540831018 shutdown_connections 2023-10-06T05:21:47.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:47.935+0000 7fb9195ff700 1 -- 172.21.15.31:0/540831018 wait complete. 2023-10-06T05:21:47.953 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:21:47.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:21:48.060 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:21:48.170 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:21:48.170 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:21:48.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:48.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.365+0000 7fd3ed9ff700 1 Processor -- start 2023-10-06T05:21:48.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.365+0000 7fd3ed9ff700 1 -- start start 2023-10-06T05:21:48.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.366+0000 7fd3ed9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80c7b60 0x7fd3e8069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.366+0000 7fd3ed9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3e80aee70 0x7fd3e813f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.366+0000 7fd3ed9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e80ca5b0 con 0x7fd3e80c7f30 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.366+0000 7fd3ed9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e80af2d0 con 0x7fd3e80aee70 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.366+0000 7fd3ed9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e806f740 con 0x7fd3e80c7b60 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3e80aee70 0x7fd3e813f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3e80aee70 0x7fd3e813f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45768/0 (socket says 172.21.15.31:45768) 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e77fe700 1 -- 172.21.15.31:0/3517429563 learned_addr learned my addr 172.21.15.31:0/3517429563 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e67fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd3e80c7f30 0x7fd3e806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35732/0 (socket says 172.21.15.31:35732) 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e6ffd700 1 --2- 172.21.15.31:0/3517429563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80c7b60 0x7fd3e8069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e77fe700 1 -- 172.21.15.31:0/3517429563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80c7b60 msgr2=0x7fd3e8069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e77fe700 1 --2- 172.21.15.31:0/3517429563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80c7b60 0x7fd3e8069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e77fe700 1 -- 172.21.15.31:0/3517429563 >> v1:172.21.15.31:6789/0 conn(0x7fd3e80c7f30 legacy=0x7fd3e806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e77fe700 1 -- 172.21.15.31:0/3517429563 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e80c9c80 con 0x7fd3e80aee70 2023-10-06T05:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e5ffb700 1 -- 172.21.15.31:0/3517429563 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731215650 0 0) 0x7fd3e80ca5b0 con 0x7fd3e80c7f30 2023-10-06T05:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e6ffd700 1 --2- 172.21.15.31:0/3517429563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80c7b60 0x7fd3e8069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.367+0000 7fd3e77fe700 1 --2- 172.21.15.31:0/3517429563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80aee70 0x7fd3e813f690 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0009f20 tx=0x7fd3e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6273526978ac840 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3e5ffb700 1 -- 172.21.15.31:0/3517429563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000e070 con 0x7fd3e80aee70 2023-10-06T05:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3e5ffb700 1 -- 172.21.15.31:0/3517429563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e000ba80 con 0x7fd3e80aee70 2023-10-06T05:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3e5ffb700 1 -- 172.21.15.31:0/3517429563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e00092e0 con 0x7fd3e80aee70 2023-10-06T05:21:48.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/3517429563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80aee70 msgr2=0x7fd3e813f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3ed9ff700 1 --2- 172.21.15.31:0/3517429563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80aee70 0x7fd3e813f690 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0009f20 tx=0x7fd3e0009840 comp rx=0 tx=0).stop 2023-10-06T05:21:48.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/3517429563 shutdown_connections 2023-10-06T05:21:48.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3ed9ff700 1 --2- 172.21.15.31:0/3517429563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80aee70 0x7fd3e813f690 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3ed9ff700 1 --2- 172.21.15.31:0/3517429563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80c7b60 0x7fd3e8069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/3517429563 >> 172.21.15.31:0/3517429563 conn(0x7fd3e80cc0b0 msgr2=0x7fd3e80cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/3517429563 shutdown_connections 2023-10-06T05:21:48.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.368+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/3517429563 wait complete. 2023-10-06T05:21:48.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.370+0000 7fd3ed9ff700 1 Processor -- start 2023-10-06T05:21:48.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.370+0000 7fd3ed9ff700 1 -- start start 2023-10-06T05:21:48.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.371+0000 7fd3ed9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80aee70 0x7fd3e813cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.371+0000 7fd3ed9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80c7b60 0x7fd3e81693a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.371+0000 7fd3ed9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3e80c7f30 0x7fd3e81698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.371+0000 7fd3ed9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e80abd30 con 0x7fd3e80c7b60 2023-10-06T05:21:48.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.371+0000 7fd3ed9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e80ab700 con 0x7fd3e80c7f30 2023-10-06T05:21:48.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.371+0000 7fd3ed9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e8169f00 con 0x7fd3e80aee70 2023-10-06T05:21:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.371+0000 7fd3e67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80c7b60 0x7fd3e81693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.371+0000 7fd3e67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80c7b60 0x7fd3e81693a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45782/0 (socket says 172.21.15.31:45782) 2023-10-06T05:21:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.371+0000 7fd3e67fc700 1 -- 172.21.15.31:0/486466556 learned_addr learned my addr 172.21.15.31:0/486466556 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.371+0000 7fd3e6ffd700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80aee70 0x7fd3e813cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3e67fc700 1 -- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80aee70 msgr2=0x7fd3e813cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3e77fe700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3e80c7f30 0x7fd3e81698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3e67fc700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80aee70 0x7fd3e813cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3e67fc700 1 -- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3e80c7f30 msgr2=0x7fd3e81698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3e67fc700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3e80c7f30 0x7fd3e81698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3e67fc700 1 -- 172.21.15.31:0/486466556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e0009ae0 con 0x7fd3e80c7b60 2023-10-06T05:21:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3e6ffd700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80aee70 0x7fd3e813cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3e77fe700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3e80c7f30 0x7fd3e81698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3e67fc700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80c7b60 0x7fd3e81693a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c5b0 tx=0x7fd3dc001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3ceffd700 1 -- 172.21.15.31:0/486466556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc005a70 con 0x7fd3e80c7b60 2023-10-06T05:21:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3ceffd700 1 -- 172.21.15.31:0/486466556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc005bd0 con 0x7fd3e80c7b60 2023-10-06T05:21:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/486466556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e816a1e0 con 0x7fd3e80c7b60 2023-10-06T05:21:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3ceffd700 1 -- 172.21.15.31:0/486466556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0070d0 con 0x7fd3e80c7b60 2023-10-06T05:21:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.372+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/486466556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e8156300 con 0x7fd3e80c7b60 2023-10-06T05:21:48.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.373+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/486466556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e80abee0 con 0x7fd3e80c7b60 2023-10-06T05:21:48.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.374+0000 7fd3ceffd700 1 -- 172.21.15.31:0/486466556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd3dc007480 con 0x7fd3e80c7b60 2023-10-06T05:21:48.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.375+0000 7fd3ceffd700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3d4067050 0x7fd3d4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.375+0000 7fd3ceffd700 1 -- 172.21.15.31:0/486466556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fd3dc083cf0 con 0x7fd3e80c7b60 2023-10-06T05:21:48.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.376+0000 7fd3e6ffd700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3d4067050 0x7fd3d4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.376+0000 7fd3e6ffd700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3d4067050 0x7fd3d4069500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8006ea0 tx=0x7fd3d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.379+0000 7fd3ceffd700 1 -- 172.21.15.31:0/486466556 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3e80abee0 con 0x7fd3e80c7b60 2023-10-06T05:21:48.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.625+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/486466556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3e80aea50 con 0x7fd3e80c7b60 2023-10-06T05:21:48.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.625+0000 7fd3ceffd700 1 -- 172.21.15.31:0/486466556 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+3523 (secure 0 0 0) 0x7fd3dc017020 con 0x7fd3e80c7b60 2023-10-06T05:21:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:48.627 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5627,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2151449662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2151449662},{"type":"v1","addr":"172.21.15.62:6834","nonce":2151449662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/810564890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":810564890},{"type":"v1","addr":"172.21.15.31:6836","nonce":810564890}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6237,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/802676715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":802676715},{"type":"v1","addr":"172.21.15.31:6837","nonce":802676715}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1748300369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1748300369},{"type":"v1","addr":"172.21.15.62:6835","nonce":1748300369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[]} 2023-10-06T05:21:48.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.627+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3d4067050 msgr2=0x7fd3d4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.627+0000 7fd3ed9ff700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3d4067050 0x7fd3d4069500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8006ea0 tx=0x7fd3d8008040 comp rx=0 tx=0).stop 2023-10-06T05:21:48.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.627+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/486466556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80c7b60 msgr2=0x7fd3e81693a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.627+0000 7fd3ed9ff700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80c7b60 0x7fd3e81693a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c5b0 tx=0x7fd3dc001040 comp rx=0 tx=0).stop 2023-10-06T05:21:48.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.627+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/486466556 shutdown_connections 2023-10-06T05:21:48.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.627+0000 7fd3ed9ff700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3d4067050 0x7fd3d4069500 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.627+0000 7fd3ed9ff700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3e80c7f30 0x7fd3e81698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.627+0000 7fd3ed9ff700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3e80aee70 0x7fd3e813cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.627+0000 7fd3ed9ff700 1 --2- 172.21.15.31:0/486466556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e80c7b60 0x7fd3e81693a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.627+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/486466556 >> 172.21.15.31:0/486466556 conn(0x7fd3e80cc0b0 msgr2=0x7fd3e80a9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.628+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/486466556 shutdown_connections 2023-10-06T05:21:48.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.628+0000 7fd3ed9ff700 1 -- 172.21.15.31:0/486466556 wait complete. 2023-10-06T05:21:48.631 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 47 2023-10-06T05:21:48.647 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:21:48.647 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:21:48.647 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:21:48.647 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:48.647 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:21:48.647 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:21:48.648 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:21:48.648 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:48.648 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:21:48.648 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:21:48.648 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:21:48.648 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:48.649 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:21:48.649 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:21:48.649 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:21:48.649 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:21:48.660 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:21:48.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:21:48.661 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:21:48.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:21:48.666 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:21:48.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:21:48.667 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:21:48.667 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:21:48.705 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:21:48.705 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:48.708 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:21:48.708 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:48.711 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:21:48.711 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:48.714 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:21:48.714 DEBUG:teuthology.parallel:result is None 2023-10-06T05:21:48.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:21:48.852 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:21:48.853 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:21:48.861 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:21:48.861 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:21:48.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.917+0000 7f69223ff700 1 Processor -- start 2023-10-06T05:21:48.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.917+0000 7f69223ff700 1 -- start start 2023-10-06T05:21:48.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.918+0000 7f69223ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f691c159580 0x7f691c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.918+0000 7f69223ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0cb9f0 0x7f691c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.918+0000 7f69223ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c0cf280 con 0x7f691c0ccc80 2023-10-06T05:21:48.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.918+0000 7f69223ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691c0ce670 con 0x7f691c159580 2023-10-06T05:21:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.918+0000 7f69223ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0ce950 con 0x7f691c0cb9f0 2023-10-06T05:21:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.919+0000 7f692099c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f691c0ccc80 0x7f691c0cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35744/0 (socket says 172.21.15.31:35744) 2023-10-06T05:21:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.919+0000 7f691b7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0cb9f0 0x7f691c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.919+0000 7f692099c700 1 -- 172.21.15.31:0/2804150083 learned_addr learned my addr 172.21.15.31:0/2804150083 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.919+0000 7f691bfff700 1 --2- 172.21.15.31:0/2804150083 >> v2:172.21.15.62:3300/0 conn(0x7f691c159580 0x7f691c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.919+0000 7f691b7fe700 1 -- 172.21.15.31:0/2804150083 >> v2:172.21.15.62:3300/0 conn(0x7f691c159580 msgr2=0x7f691c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.919+0000 7f691b7fe700 1 --2- 172.21.15.31:0/2804150083 >> v2:172.21.15.62:3300/0 conn(0x7f691c159580 0x7f691c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.919+0000 7f691b7fe700 1 -- 172.21.15.31:0/2804150083 >> v1:172.21.15.31:6789/0 conn(0x7f691c0ccc80 legacy=0x7f691c0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.919+0000 7f691b7fe700 1 -- 172.21.15.31:0/2804150083 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c0cec30 con 0x7f691c0cb9f0 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.919+0000 7f691affd700 1 -- 172.21.15.31:0/2804150083 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185055874 0 0) 0x7f691c0cf280 con 0x7f691c0ccc80 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.919+0000 7f691b7fe700 1 --2- 172.21.15.31:0/2804150083 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0cb9f0 0x7f691c0cf710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f69100098e0 tx=0x7f69100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19eaa14b8e5c9e8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.920+0000 7f691affd700 1 -- 172.21.15.31:0/2804150083 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000b640 con 0x7f691c0cb9f0 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.920+0000 7f691affd700 1 -- 172.21.15.31:0/2804150083 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691000b7a0 con 0x7f691c0cb9f0 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.920+0000 7f691affd700 1 -- 172.21.15.31:0/2804150083 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000bb40 con 0x7f691c0cb9f0 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.920+0000 7f69223ff700 1 -- 172.21.15.31:0/2804150083 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0cb9f0 msgr2=0x7f691c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.920+0000 7f69223ff700 1 --2- 172.21.15.31:0/2804150083 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0cb9f0 0x7f691c0cf710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f69100098e0 tx=0x7f69100091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.920+0000 7f69223ff700 1 -- 172.21.15.31:0/2804150083 shutdown_connections 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.920+0000 7f69223ff700 1 --2- 172.21.15.31:0/2804150083 >> v2:172.21.15.62:3300/0 conn(0x7f691c159580 0x7f691c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.920+0000 7f69223ff700 1 --2- 172.21.15.31:0/2804150083 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0cb9f0 0x7f691c0cf710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.920+0000 7f69223ff700 1 -- 172.21.15.31:0/2804150083 >> 172.21.15.31:0/2804150083 conn(0x7f691c099710 msgr2=0x7f691c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:48.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.921+0000 7f69223ff700 1 -- 172.21.15.31:0/2804150083 shutdown_connections 2023-10-06T05:21:48.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.921+0000 7f69223ff700 1 -- 172.21.15.31:0/2804150083 wait complete. 2023-10-06T05:21:48.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.922+0000 7f69223ff700 1 Processor -- start 2023-10-06T05:21:48.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.922+0000 7f69223ff700 1 -- start start 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f69223ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f691c0cb9f0 0x7f691c068dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f69223ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0ccc80 0x7f691c1557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f69223ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f691c159580 0x7f691c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f69223ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0b3e70 con 0x7f691c0cb9f0 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f69223ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0b3280 con 0x7f691c159580 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f69223ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691c0b3540 con 0x7f691c0ccc80 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f691bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f691c0cb9f0 0x7f691c068dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f691bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f691c0cb9f0 0x7f691c068dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45830/0 (socket says 172.21.15.31:45830) 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f691b7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0ccc80 0x7f691c1557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f691bfff700 1 -- 172.21.15.31:0/1844170938 learned_addr learned my addr 172.21.15.31:0/1844170938 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.923+0000 7f692099c700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f691c159580 0x7f691c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.924+0000 7f691b7fe700 1 -- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f691c159580 msgr2=0x7f691c155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.924+0000 7f691b7fe700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f691c159580 0x7f691c155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.924+0000 7f691b7fe700 1 -- 172.21.15.31:0/1844170938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f691c0cb9f0 msgr2=0x7f691c068dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.924+0000 7f691b7fe700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f691c0cb9f0 0x7f691c068dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.924+0000 7f691b7fe700 1 -- 172.21.15.31:0/1844170938 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910009420 con 0x7f691c0ccc80 2023-10-06T05:21:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.924+0000 7f691b7fe700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0ccc80 0x7f691c1557c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6910016000 tx=0x7f6910017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.924+0000 7f6903fff700 1 -- 172.21.15.31:0/1844170938 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000b910 con 0x7f691c0ccc80 2023-10-06T05:21:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.924+0000 7f6903fff700 1 -- 172.21.15.31:0/1844170938 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691000ba70 con 0x7f691c0ccc80 2023-10-06T05:21:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.924+0000 7f69223ff700 1 -- 172.21.15.31:0/1844170938 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691c0b3820 con 0x7f691c0ccc80 2023-10-06T05:21:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.924+0000 7f6903fff700 1 -- 172.21.15.31:0/1844170938 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910024670 con 0x7f691c0ccc80 2023-10-06T05:21:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.925+0000 7f69223ff700 1 -- 172.21.15.31:0/1844170938 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691c069310 con 0x7f691c0ccc80 2023-10-06T05:21:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.926+0000 7f69223ff700 1 -- 172.21.15.31:0/1844170938 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691c0b3fe0 con 0x7f691c0ccc80 2023-10-06T05:21:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.926+0000 7f6903fff700 1 -- 172.21.15.31:0/1844170938 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f6910024aa0 con 0x7f691c0ccc80 2023-10-06T05:21:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.927+0000 7f6903fff700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6908067050 0x7f6908069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.927+0000 7f6903fff700 1 -- 172.21.15.31:0/1844170938 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f6910086150 con 0x7f691c0ccc80 2023-10-06T05:21:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.928+0000 7f691bfff700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6908067050 0x7f6908069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.928+0000 7f691bfff700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6908067050 0x7f6908069500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f690c009dc0 tx=0x7f690c009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:48.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:48.932+0000 7f6903fff700 1 -- 172.21.15.31:0/1844170938 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6910053180 con 0x7f691c0ccc80 2023-10-06T05:21:49.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.150+0000 7f69223ff700 1 -- 172.21.15.31:0/1844170938 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f691c04c380 con 0x7f691c0ccc80 2023-10-06T05:21:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.505+0000 7f6903fff700 1 -- 172.21.15.31:0/1844170938 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) v1 ==== 97+0+0 (secure 0 0 0) 0x7f69100567a0 con 0x7f691c0ccc80 2023-10-06T05:21:49.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.507+0000 7f69223ff700 1 -- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6908067050 msgr2=0x7f6908069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.507+0000 7f69223ff700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6908067050 0x7f6908069500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f690c009dc0 tx=0x7f690c009120 comp rx=0 tx=0).stop 2023-10-06T05:21:49.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.507+0000 7f69223ff700 1 -- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0ccc80 msgr2=0x7f691c1557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.507+0000 7f69223ff700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0ccc80 0x7f691c1557c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6910016000 tx=0x7f6910017000 comp rx=0 tx=0).stop 2023-10-06T05:21:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.508+0000 7f69223ff700 1 -- 172.21.15.31:0/1844170938 shutdown_connections 2023-10-06T05:21:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.508+0000 7f69223ff700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6908067050 0x7f6908069500 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.508+0000 7f69223ff700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f691c159580 0x7f691c155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.508+0000 7f69223ff700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f691c0cb9f0 0x7f691c068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.508+0000 7f69223ff700 1 --2- 172.21.15.31:0/1844170938 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f691c0ccc80 0x7f691c1557c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.508+0000 7f69223ff700 1 -- 172.21.15.31:0/1844170938 >> 172.21.15.31:0/1844170938 conn(0x7f691c099710 msgr2=0x7f691c0cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:49.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.508+0000 7f69223ff700 1 -- 172.21.15.31:0/1844170938 shutdown_connections 2023-10-06T05:21:49.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.508+0000 7f69223ff700 1 -- 172.21.15.31:0/1844170938 wait complete. 2023-10-06T05:21:49.513 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:21:49.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:21:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.724+0000 7f0afcfff700 1 Processor -- start 2023-10-06T05:21:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.724+0000 7f0afcfff700 1 -- start start 2023-10-06T05:21:49.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.726+0000 7f0afcfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0af8068d80 0x7f0af8159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.726+0000 7f0afcfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0af80cc0d0 0x7f0af80c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.726+0000 7f0afcfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80c95b0 con 0x7f0af81595a0 2023-10-06T05:21:49.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.726+0000 7f0afcfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af80c9890 con 0x7f0af80cc0d0 2023-10-06T05:21:49.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.726+0000 7f0afcfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80c9b70 con 0x7f0af8068d80 2023-10-06T05:21:49.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.726+0000 7f0af659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0af81595a0 0x7f0af8068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35760/0 (socket says 172.21.15.31:35760) 2023-10-06T05:21:49.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.726+0000 7f0af659c700 1 -- 172.21.15.31:0/1875610247 learned_addr learned my addr 172.21.15.31:0/1875610247 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:49.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.726+0000 7f0af6d9d700 1 --2- 172.21.15.31:0/1875610247 >> v2:172.21.15.62:3300/0 conn(0x7f0af80cc0d0 0x7f0af80c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.726+0000 7f0aeffff700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0af8068d80 0x7f0af8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673949189 0 0) 0x7f0af80c95b0 con 0x7f0af81595a0 2023-10-06T05:21:49.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae4003570 con 0x7f0af81595a0 2023-10-06T05:21:49.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4100307245 0 0) 0x7f0ae4003570 con 0x7f0af81595a0 2023-10-06T05:21:49.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0af8068d80 msgr2=0x7f0af8159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:49.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0af8068d80 0x7f0af8159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 >> v2:172.21.15.62:3300/0 conn(0x7f0af80cc0d0 msgr2=0x7f0af80c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 --2- 172.21.15.31:0/1875610247 >> v2:172.21.15.62:3300/0 conn(0x7f0af80cc0d0 0x7f0af80c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80ca130 con 0x7f0af81595a0 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f0ae8001470 con 0x7f0af81595a0 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 >> v1:172.21.15.31:6789/0 conn(0x7f0af81595a0 legacy=0x7f0af8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4003ae0 0x7f0ae4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0af8068d80 0x7f0ae4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4004d50 0x7f0ae40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80c9890 con 0x7f0ae4003ae0 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80c9b70 con 0x7f0ae4004d50 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae400bb40 con 0x7f0af8068d80 2023-10-06T05:21:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af6d9d700 1 --2- 172.21.15.31:0/1875610247 >> v2:172.21.15.62:3300/0 conn(0x7f0af80cc0d0 0x7f0af80c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.727+0000 7f0af659c700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4003ae0 0x7f0ae4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0aeffff700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0af8068d80 0x7f0ae4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af6d9d700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4004d50 0x7f0ae40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af659c700 1 -- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0af8068d80 msgr2=0x7f0ae4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af659c700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0af8068d80 0x7f0ae4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af659c700 1 -- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4004d50 msgr2=0x7f0ae40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af659c700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4004d50 0x7f0ae40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af659c700 1 -- 172.21.15.31:0/1875610247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae4003570 con 0x7f0ae4003ae0 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af6d9d700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4004d50 0x7f0ae40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0aeffff700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0af8068d80 0x7f0ae4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af659c700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4003ae0 0x7f0ae4003ef0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c2d0 tx=0x7f0ae8001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91a28a5a460212b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800cb40 con 0x7f0ae4003ae0 2023-10-06T05:21:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae800cca0 con 0x7f0ae4003ae0 2023-10-06T05:21:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.728+0000 7f0af5d9b700 1 -- 172.21.15.31:0/1875610247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8005b70 con 0x7f0ae4003ae0 2023-10-06T05:21:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.729+0000 7f0afcfff700 1 -- 172.21.15.31:0/1875610247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4003ae0 msgr2=0x7f0ae4003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.729+0000 7f0afcfff700 1 --2- 172.21.15.31:0/1875610247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4003ae0 0x7f0ae4003ef0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c2d0 tx=0x7f0ae8001dc0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.729+0000 7f0af6d9d700 1 -- 172.21.15.31:0/1875610247 reap_dead start 2023-10-06T05:21:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.729+0000 7f0afcfff700 1 -- 172.21.15.31:0/1875610247 shutdown_connections 2023-10-06T05:21:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.729+0000 7f0afcfff700 1 -- 172.21.15.31:0/1875610247 >> 172.21.15.31:0/1875610247 conn(0x7f0af806b620 msgr2=0x7f0af806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.729+0000 7f0afcfff700 1 -- 172.21.15.31:0/1875610247 shutdown_connections 2023-10-06T05:21:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.729+0000 7f0afcfff700 1 -- 172.21.15.31:0/1875610247 wait complete. 2023-10-06T05:21:49.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.730+0000 7f0afcfff700 1 Processor -- start 2023-10-06T05:21:49.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.731+0000 7f0afcfff700 1 -- start start 2023-10-06T05:21:49.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0afcfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae4003ae0 0x7f0af8153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0afcfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af81566e0 0x7f0af8156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0afcfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0af80c3b50 0x7f0af80c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0afcfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af81699b0 con 0x7f0af81566e0 2023-10-06T05:21:49.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0afcfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af8157090 con 0x7f0af80c3b50 2023-10-06T05:21:49.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0afcfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80c47f0 con 0x7f0ae4003ae0 2023-10-06T05:21:49.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0aeffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af81566e0 0x7f0af8156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0aeffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af81566e0 0x7f0af8156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45836/0 (socket says 172.21.15.31:45836) 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0aeffff700 1 -- 172.21.15.31:0/898220133 learned_addr learned my addr 172.21.15.31:0/898220133 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0af6d9d700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0af80c3b50 0x7f0af80c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0af659c700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae4003ae0 0x7f0af8153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0aeffff700 1 -- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae4003ae0 msgr2=0x7f0af8153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0aeffff700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae4003ae0 0x7f0af8153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0aeffff700 1 -- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0af80c3b50 msgr2=0x7f0af80c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0aeffff700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0af80c3b50 0x7f0af80c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0aeffff700 1 -- 172.21.15.31:0/898220133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80c4ad0 con 0x7f0af81566e0 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0af659c700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae4003ae0 0x7f0af8153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.732+0000 7f0aeffff700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af81566e0 0x7f0af8156b50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00118e0 tx=0x7f0ae0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.733+0000 7f0aee7fc700 1 -- 172.21.15.31:0/898220133 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0011e80 con 0x7f0af81566e0 2023-10-06T05:21:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.733+0000 7f0aee7fc700 1 -- 172.21.15.31:0/898220133 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae000f860 con 0x7f0af81566e0 2023-10-06T05:21:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.733+0000 7f0aee7fc700 1 -- 172.21.15.31:0/898220133 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000fc00 con 0x7f0af81566e0 2023-10-06T05:21:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.733+0000 7f0afcfff700 1 -- 172.21.15.31:0/898220133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af80c4db0 con 0x7f0af81566e0 2023-10-06T05:21:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.733+0000 7f0afcfff700 1 -- 172.21.15.31:0/898220133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af80c4510 con 0x7f0af81566e0 2023-10-06T05:21:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.734+0000 7f0afcfff700 1 -- 172.21.15.31:0/898220133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8169b60 con 0x7f0af81566e0 2023-10-06T05:21:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.735+0000 7f0aee7fc700 1 -- 172.21.15.31:0/898220133 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f0ae0015460 con 0x7f0af81566e0 2023-10-06T05:21:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.736+0000 7f0aee7fc700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ad8067050 0x7f0ad8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.736+0000 7f0aee7fc700 1 -- 172.21.15.31:0/898220133 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0ae0084d60 con 0x7f0af81566e0 2023-10-06T05:21:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.736+0000 7f0af659c700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ad8067050 0x7f0ad8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.737+0000 7f0af659c700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ad8067050 0x7f0ad8069500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8015040 tx=0x7f0ae8012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:49.740+0000 7f0aee7fc700 1 -- 172.21.15.31:0/898220133 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0af8169b60 con 0x7f0af81566e0 2023-10-06T05:21:50.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.019+0000 7f0afcfff700 1 -- 172.21.15.31:0/898220133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0af804c380 con 0x7f0af81566e0 2023-10-06T05:21:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.020+0000 7f0aee7fc700 1 -- 172.21.15.31:0/898220133 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0ae0055aa0 con 0x7f0af81566e0 2023-10-06T05:21:50.022 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:50.022 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:21:50.022 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:21:50.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.023 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:21:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:21:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:21:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:21:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:21:50.025 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.026 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:50.026 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:21:50.026 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:21:50.026 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.028 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:21:50.029 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:21:50.029 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.029 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:50.029 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:50.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.029 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:21:50.029 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:21:50.030 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.030 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:50.030 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:50.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.030 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:21:50.030 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:21:50.030 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:50.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.033 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:21:50.034 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:50.035 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:21:50.036 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:21:50.036 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:21:50.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.036 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:21:50.037 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:21:50.037 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.037 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:50.037 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:21:50.037 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:21:50.037 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:21:50.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.038 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:21:50.038 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:21:50.038 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.038 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:21:50.038 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:21:50.038 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:21:50.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:21:50.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:21:50.040 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:21:50.041 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:21:50.041 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:21:50.041 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:21:50.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:50.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:21:50.041 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:21:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.022+0000 7f0afcfff700 1 -- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ad8067050 msgr2=0x7f0ad8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.022+0000 7f0afcfff700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ad8067050 0x7f0ad8069500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8015040 tx=0x7f0ae8012000 comp rx=0 tx=0).stop 2023-10-06T05:21:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.022+0000 7f0afcfff700 1 -- 172.21.15.31:0/898220133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af81566e0 msgr2=0x7f0af8156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.022+0000 7f0afcfff700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af81566e0 0x7f0af8156b50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00118e0 tx=0x7f0ae0008040 comp rx=0 tx=0).stop 2023-10-06T05:21:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.023+0000 7f0afcfff700 1 -- 172.21.15.31:0/898220133 shutdown_connections 2023-10-06T05:21:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.023+0000 7f0afcfff700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ad8067050 0x7f0ad8069500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.023+0000 7f0afcfff700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0af80c3b50 0x7f0af80c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.023+0000 7f0afcfff700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae4003ae0 0x7f0af8153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.023+0000 7f0afcfff700 1 --2- 172.21.15.31:0/898220133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af81566e0 0x7f0af8156b50 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.023+0000 7f0afcfff700 1 -- 172.21.15.31:0/898220133 >> 172.21.15.31:0/898220133 conn(0x7f0af806b620 msgr2=0x7f0af806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:50.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.023+0000 7f0afcfff700 1 -- 172.21.15.31:0/898220133 shutdown_connections 2023-10-06T05:21:50.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.023+0000 7f0afcfff700 1 -- 172.21.15.31:0/898220133 wait complete. 2023-10-06T05:21:50.044 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:50.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.239+0000 7f1a12fff700 1 Processor -- start 2023-10-06T05:21:50.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.239+0000 7f1a12fff700 1 -- start start 2023-10-06T05:21:50.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.241+0000 7f1a12fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a0c068d80 0x7f1a0c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.241+0000 7f1a12fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c0c95b0 con 0x7f1a0c1595a0 2023-10-06T05:21:50.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.241+0000 7f1a12fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c0c9890 con 0x7f1a0c0cc0d0 2023-10-06T05:21:50.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.241+0000 7f1a12fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c0c9b70 con 0x7f1a0c068d80 2023-10-06T05:21:50.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.241+0000 7f1a10d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1a0c1595a0 0x7f1a0c068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35770/0 (socket says 172.21.15.31:35770) 2023-10-06T05:21:50.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.241+0000 7f1a10d9b700 1 -- 172.21.15.31:0/3277818878 learned_addr learned my addr 172.21.15.31:0/3277818878 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:50.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.241+0000 7f1a0bfff700 1 --2- 172.21.15.31:0/3277818878 >> v2:172.21.15.62:3300/0 conn(0x7f1a0c068d80 0x7f1a0c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.241+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3734485585 0 0) 0x7f1a0c0c95b0 con 0x7f1a0c1595a0 2023-10-06T05:21:50.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.241+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19fc003570 con 0x7f1a0c1595a0 2023-10-06T05:21:50.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2598886108 0 0) 0x7f19fc003570 con 0x7f1a0c1595a0 2023-10-06T05:21:50.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 >> v2:172.21.15.62:3300/0 conn(0x7f1a0c068d80 msgr2=0x7f1a0c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:50.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 --2- 172.21.15.31:0/3277818878 >> v2:172.21.15.62:3300/0 conn(0x7f1a0c068d80 0x7f1a0c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 >> v1:172.21.15.62:6789/0 conn(0x7f1a0c0cc0d0 legacy=0x7f1a0c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c0ca130 con 0x7f1a0c1595a0 2023-10-06T05:21:50.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f1a00001470 con 0x7f1a0c1595a0 2023-10-06T05:21:50.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 >> v1:172.21.15.31:6789/0 conn(0x7f1a0c1595a0 legacy=0x7f1a0c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19fc003ae0 0x7f19fc003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19fc004810 0x7f19fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19fc005230 0x7f19fc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c0c9b70 con 0x7f19fc005230 2023-10-06T05:21:50.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc00c160 con 0x7f19fc004810 2023-10-06T05:21:50.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc00c320 con 0x7f19fc003ae0 2023-10-06T05:21:50.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a1159c700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19fc005230 0x7f19fc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a0bfff700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19fc004810 0x7f19fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.242+0000 7f1a10d9b700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19fc003ae0 0x7f19fc003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.243+0000 7f1a1159c700 1 -- 172.21.15.31:0/3277818878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19fc003ae0 msgr2=0x7f19fc003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:50.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.243+0000 7f1a1159c700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19fc003ae0 0x7f19fc003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.243+0000 7f1a1159c700 1 -- 172.21.15.31:0/3277818878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19fc004810 msgr2=0x7f19fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:50.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.243+0000 7f1a1159c700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19fc004810 0x7f19fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.243+0000 7f1a1159c700 1 -- 172.21.15.31:0/3277818878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc003570 con 0x7f19fc005230 2023-10-06T05:21:50.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.243+0000 7f1a1159c700 1 -- 172.21.15.31:0/3277818878 reap_dead start 2023-10-06T05:21:50.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.243+0000 7f1a1159c700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19fc005230 0x7f19fc00ba60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1a04001fc0 tx=0x7f1a04001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a454f11677eee9c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:50.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.243+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a04001950 con 0x7f19fc005230 2023-10-06T05:21:50.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.243+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0400c8b0 con 0x7f19fc005230 2023-10-06T05:21:50.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.243+0000 7f1a0b7fe700 1 -- 172.21.15.31:0/3277818878 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400cc90 con 0x7f19fc005230 2023-10-06T05:21:50.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.244+0000 7f1a12fff700 1 -- 172.21.15.31:0/3277818878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19fc005230 msgr2=0x7f19fc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.244+0000 7f1a12fff700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19fc005230 0x7f19fc00ba60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1a04001fc0 tx=0x7f1a04001640 comp rx=0 tx=0).stop 2023-10-06T05:21:50.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.244+0000 7f1a12fff700 1 -- 172.21.15.31:0/3277818878 shutdown_connections 2023-10-06T05:21:50.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.244+0000 7f1a12fff700 1 --2- 172.21.15.31:0/3277818878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19fc005230 0x7f19fc00ba60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.244+0000 7f1a12fff700 1 -- 172.21.15.31:0/3277818878 >> 172.21.15.31:0/3277818878 conn(0x7f1a0c06b620 msgr2=0x7f1a0c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:50.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.244+0000 7f1a12fff700 1 -- 172.21.15.31:0/3277818878 shutdown_connections 2023-10-06T05:21:50.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.244+0000 7f1a12fff700 1 -- 172.21.15.31:0/3277818878 wait complete. 2023-10-06T05:21:50.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.245+0000 7f1a12fff700 1 Processor -- start 2023-10-06T05:21:50.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.246+0000 7f1a12fff700 1 -- start start 2023-10-06T05:21:50.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a12fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19fc005230 0x7f1a0c139e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a12fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c1538a0 0x7f1a0c153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a12fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a0c154250 0x7f1a0c14f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a12fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c1546c0 con 0x7f1a0c1538a0 2023-10-06T05:21:50.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a12fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c1549a0 con 0x7f19fc005230 2023-10-06T05:21:50.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a12fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c154c80 con 0x7f1a0c154250 2023-10-06T05:21:50.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c1538a0 0x7f1a0c153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c1538a0 0x7f1a0c153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45868/0 (socket says 172.21.15.31:45868) 2023-10-06T05:21:50.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a0bfff700 1 -- 172.21.15.31:0/828351065 learned_addr learned my addr 172.21.15.31:0/828351065 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:50.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a10d9b700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19fc005230 0x7f1a0c139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a1159c700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a0c154250 0x7f1a0c14f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a0bfff700 1 -- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a0c154250 msgr2=0x7f1a0c14f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a0bfff700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a0c154250 0x7f1a0c14f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a0bfff700 1 -- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19fc005230 msgr2=0x7f1a0c139e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a0bfff700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19fc005230 0x7f1a0c139e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.247+0000 7f1a0bfff700 1 -- 172.21.15.31:0/828351065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c14efd0 con 0x7f1a0c1538a0 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.248+0000 7f1a0bfff700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c1538a0 0x7f1a0c153d10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009fd0 tx=0x7f19f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.248+0000 7f1a08ff9700 1 -- 172.21.15.31:0/828351065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800f790 con 0x7f1a0c1538a0 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.248+0000 7f1a08ff9700 1 -- 172.21.15.31:0/828351065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f800f8f0 con 0x7f1a0c1538a0 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.248+0000 7f1a08ff9700 1 -- 172.21.15.31:0/828351065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800fc70 con 0x7f1a0c1538a0 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.248+0000 7f1a12fff700 1 -- 172.21.15.31:0/828351065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c13a350 con 0x7f1a0c1538a0 2023-10-06T05:21:50.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.248+0000 7f1a12fff700 1 -- 172.21.15.31:0/828351065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c14fbd0 con 0x7f1a0c1538a0 2023-10-06T05:21:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.249+0000 7f1a12fff700 1 -- 172.21.15.31:0/828351065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c14f340 con 0x7f1a0c1538a0 2023-10-06T05:21:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.250+0000 7f1a08ff9700 1 -- 172.21.15.31:0/828351065 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f19f8014650 con 0x7f1a0c1538a0 2023-10-06T05:21:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.251+0000 7f1a08ff9700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067050 0x7f19f0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.251+0000 7f1a08ff9700 1 -- 172.21.15.31:0/828351065 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f19f8018030 con 0x7f1a0c1538a0 2023-10-06T05:21:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.251+0000 7f1a10d9b700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067050 0x7f19f0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.252+0000 7f1a10d9b700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067050 0x7f19f0069500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1a000015a0 tx=0x7f1a00001d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:50.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.255+0000 7f1a08ff9700 1 -- 172.21.15.31:0/828351065 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19f8052120 con 0x7f1a0c1538a0 2023-10-06T05:21:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.460+0000 7f1a12fff700 1 -- 172.21.15.31:0/828351065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a0c13a540 con 0x7f1a0c1538a0 2023-10-06T05:21:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.461+0000 7f1a08ff9700 1 -- 172.21.15.31:0/828351065 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13699 (secure 0 0 0) 0x7f19f8022090 con 0x7f1a0c1538a0 2023-10-06T05:21:50.463 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:50.463 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":79,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:49.511288+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 -- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067050 msgr2=0x7f19f0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067050 0x7f19f0069500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1a000015a0 tx=0x7f1a00001d80 comp rx=0 tx=0).stop 2023-10-06T05:21:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 -- 172.21.15.31:0/828351065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c1538a0 msgr2=0x7f1a0c153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c1538a0 0x7f1a0c153d10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009fd0 tx=0x7f19f80091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 -- 172.21.15.31:0/828351065 shutdown_connections 2023-10-06T05:21:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067050 0x7f19f0069500 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a0c154250 0x7f1a0c14f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19fc005230 0x7f1a0c139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 --2- 172.21.15.31:0/828351065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c1538a0 0x7f1a0c153d10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 -- 172.21.15.31:0/828351065 >> 172.21.15.31:0/828351065 conn(0x7f1a0c06b620 msgr2=0x7f1a0c06d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.463+0000 7f1a12fff700 1 -- 172.21.15.31:0/828351065 shutdown_connections 2023-10-06T05:21:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.464+0000 7f1a12fff700 1 -- 172.21.15.31:0/828351065 wait complete. 2023-10-06T05:21:50.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:21:50.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:21:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.682+0000 7f6036fff700 1 Processor -- start 2023-10-06T05:21:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.682+0000 7f6036fff700 1 -- start start 2023-10-06T05:21:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.683+0000 7f6036fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f603015ca40 0x7f6030068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.683+0000 7f6036fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 0x7f60300c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.684+0000 7f6036fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6030068da0 con 0x7f60300c8d70 2023-10-06T05:21:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.684+0000 7f6036fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60300cabc0 con 0x7f603015ca40 2023-10-06T05:21:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.684+0000 7f6036fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6030068bb0 con 0x7f603006a300 2023-10-06T05:21:50.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.684+0000 7f602ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 0x7f60300c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.684+0000 7f6034d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f603015ca40 0x7f6030068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.684+0000 7f602ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 0x7f60300c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:59208/0 (socket says 172.21.15.31:59208) 2023-10-06T05:21:50.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.684+0000 7f602ffff700 1 -- 172.21.15.31:0/38964133 learned_addr learned my addr 172.21.15.31:0/38964133 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:50.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.685+0000 7f602ffff700 1 -- 172.21.15.31:0/38964133 >> v2:172.21.15.62:3300/0 conn(0x7f603015ca40 msgr2=0x7f6030068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:50.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.685+0000 7f602ffff700 1 --2- 172.21.15.31:0/38964133 >> v2:172.21.15.62:3300/0 conn(0x7f603015ca40 0x7f6030068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.685+0000 7f602ffff700 1 -- 172.21.15.31:0/38964133 >> v1:172.21.15.62:6789/0 conn(0x7f60300c8d70 legacy=0x7f603006cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.685+0000 7f602ffff700 1 -- 172.21.15.31:0/38964133 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6030069080 con 0x7f603006a300 2023-10-06T05:21:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.685+0000 7f602ffff700 1 --2- 172.21.15.31:0/38964133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 0x7f60300c8830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f601c0099b0 tx=0x7f601c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba0f22fc0749dce2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.685+0000 7f602f7fe700 1 -- 172.21.15.31:0/38964133 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c009ea0 con 0x7f603006a300 2023-10-06T05:21:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.685+0000 7f602f7fe700 1 -- 172.21.15.31:0/38964133 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c00b860 con 0x7f603006a300 2023-10-06T05:21:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.685+0000 7f602f7fe700 1 -- 172.21.15.31:0/38964133 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00bc00 con 0x7f603006a300 2023-10-06T05:21:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.686+0000 7f6036fff700 1 -- 172.21.15.31:0/38964133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 msgr2=0x7f60300c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.686+0000 7f6036fff700 1 --2- 172.21.15.31:0/38964133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 0x7f60300c8830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f601c0099b0 tx=0x7f601c009250 comp rx=0 tx=0).stop 2023-10-06T05:21:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.686+0000 7f6036fff700 1 -- 172.21.15.31:0/38964133 shutdown_connections 2023-10-06T05:21:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.686+0000 7f6036fff700 1 --2- 172.21.15.31:0/38964133 >> v2:172.21.15.62:3300/0 conn(0x7f603015ca40 0x7f6030068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.686+0000 7f6036fff700 1 --2- 172.21.15.31:0/38964133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 0x7f60300c8830 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.686+0000 7f6036fff700 1 -- 172.21.15.31:0/38964133 >> 172.21.15.31:0/38964133 conn(0x7f60300ce290 msgr2=0x7f60300ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.686+0000 7f6036fff700 1 -- 172.21.15.31:0/38964133 shutdown_connections 2023-10-06T05:21:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.686+0000 7f6036fff700 1 -- 172.21.15.31:0/38964133 wait complete. 2023-10-06T05:21:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.687+0000 7f6036fff700 1 Processor -- start 2023-10-06T05:21:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.687+0000 7f6036fff700 1 -- start start 2023-10-06T05:21:50.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.688+0000 7f6036fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 0x7f60300b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.688+0000 7f6036fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60300c8d70 0x7f60300c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.688+0000 7f6036fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f603015ca40 0x7f60300c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.688+0000 7f6036fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300c1fb0 con 0x7f60300c8d70 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.688+0000 7f6036fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300b3ee0 con 0x7f603015ca40 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.688+0000 7f6036fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60300c2250 con 0x7f603006a300 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.689+0000 7f602ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60300c8d70 0x7f60300c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.689+0000 7f602ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60300c8d70 0x7f60300c27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45870/0 (socket says 172.21.15.31:45870) 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.689+0000 7f602ffff700 1 -- 172.21.15.31:0/925642479 learned_addr learned my addr 172.21.15.31:0/925642479 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.689+0000 7f6034d9b700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 0x7f60300b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.689+0000 7f603559c700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f603015ca40 0x7f60300c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.689+0000 7f602ffff700 1 -- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 msgr2=0x7f60300b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.689+0000 7f602ffff700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 0x7f60300b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.689+0000 7f602ffff700 1 -- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f603015ca40 msgr2=0x7f60300c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.689+0000 7f602ffff700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f603015ca40 0x7f60300c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.689+0000 7f602ffff700 1 -- 172.21.15.31:0/925642479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c0094f0 con 0x7f60300c8d70 2023-10-06T05:21:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.690+0000 7f602ffff700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60300c8d70 0x7f60300c27a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f60300b4370 tx=0x7f601c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.690+0000 7f602cff9700 1 -- 172.21.15.31:0/925642479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c000d10 con 0x7f60300c8d70 2023-10-06T05:21:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.690+0000 7f602cff9700 1 -- 172.21.15.31:0/925642479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c00bbe0 con 0x7f60300c8d70 2023-10-06T05:21:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.690+0000 7f602cff9700 1 -- 172.21.15.31:0/925642479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c011670 con 0x7f60300c8d70 2023-10-06T05:21:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.690+0000 7f6036fff700 1 -- 172.21.15.31:0/925642479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60300b36e0 con 0x7f60300c8d70 2023-10-06T05:21:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.690+0000 7f6036fff700 1 -- 172.21.15.31:0/925642479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60300b1580 con 0x7f60300c8d70 2023-10-06T05:21:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.691+0000 7f6036fff700 1 -- 172.21.15.31:0/925642479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60300b3510 con 0x7f60300c8d70 2023-10-06T05:21:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.692+0000 7f602cff9700 1 -- 172.21.15.31:0/925642479 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f601c011aa0 con 0x7f60300c8d70 2023-10-06T05:21:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.693+0000 7f602cff9700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6014067050 0x7f6014069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.693+0000 7f602cff9700 1 -- 172.21.15.31:0/925642479 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f601c085d10 con 0x7f60300c8d70 2023-10-06T05:21:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.694+0000 7f6034d9b700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6014067050 0x7f6014069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:50.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.694+0000 7f6034d9b700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6014067050 0x7f6014069500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f60300c8a80 tx=0x7f60240091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:50.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.698+0000 7f602cff9700 1 -- 172.21.15.31:0/925642479 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f601c08b020 con 0x7f60300c8d70 2023-10-06T05:21:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.897+0000 7f6036fff700 1 -- 172.21.15.31:0/925642479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f603004c380 con 0x7f60300c8d70 2023-10-06T05:21:50.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.897+0000 7f602cff9700 1 -- 172.21.15.31:0/925642479 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7f601c052fc0 con 0x7f60300c8d70 2023-10-06T05:21:50.899 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:21:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.899+0000 7f6036fff700 1 -- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6014067050 msgr2=0x7f6014069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.899+0000 7f6036fff700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6014067050 0x7f6014069500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f60300c8a80 tx=0x7f60240091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.899+0000 7f6036fff700 1 -- 172.21.15.31:0/925642479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60300c8d70 msgr2=0x7f60300c27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.899+0000 7f6036fff700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60300c8d70 0x7f60300c27a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f60300b4370 tx=0x7f601c017000 comp rx=0 tx=0).stop 2023-10-06T05:21:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.900+0000 7f6036fff700 1 -- 172.21.15.31:0/925642479 shutdown_connections 2023-10-06T05:21:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.900+0000 7f6036fff700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6014067050 0x7f6014069500 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.900+0000 7f6036fff700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f603015ca40 0x7f60300c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.900+0000 7f6036fff700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f603006a300 0x7f60300b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.900+0000 7f6036fff700 1 --2- 172.21.15.31:0/925642479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60300c8d70 0x7f60300c27a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:50.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.900+0000 7f6036fff700 1 -- 172.21.15.31:0/925642479 >> 172.21.15.31:0/925642479 conn(0x7f60300ce290 msgr2=0x7f603015b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:50.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.900+0000 7f6036fff700 1 -- 172.21.15.31:0/925642479 shutdown_connections 2023-10-06T05:21:50.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:50.900+0000 7f6036fff700 1 -- 172.21.15.31:0/925642479 wait complete. 2023-10-06T05:21:50.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:21:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.115+0000 7f73e89ff700 1 Processor -- start 2023-10-06T05:21:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.115+0000 7f73e89ff700 1 -- start start 2023-10-06T05:21:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.116+0000 7f73e89ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f73e4159930 0x7f73e40b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.116+0000 7f73e89ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40cd580 0x7f73e40b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.116+0000 7f73e89ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e415d2d0 con 0x7f73e4159560 2023-10-06T05:21:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.116+0000 7f73e89ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e415d5b0 con 0x7f73e4159930 2023-10-06T05:21:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.116+0000 7f73e89ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e415d890 con 0x7f73e40cd580 2023-10-06T05:21:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f73e4159560 0x7f73e40cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35776/0 (socket says 172.21.15.31:35776) 2023-10-06T05:21:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e259c700 1 -- 172.21.15.31:0/226932855 learned_addr learned my addr 172.21.15.31:0/226932855 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e1d9b700 1 --2- 172.21.15.31:0/226932855 >> v2:172.21.15.62:3300/0 conn(0x7f73e4159930 0x7f73e40b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e2d9d700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40cd580 0x7f73e40b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558977844 0 0) 0x7f73e415d2d0 con 0x7f73e4159560 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73cc003570 con 0x7f73e4159560 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3612604831 0 0) 0x7f73cc003570 con 0x7f73e4159560 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40cd580 msgr2=0x7f73e40b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40cd580 0x7f73e40b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 >> v2:172.21.15.62:3300/0 conn(0x7f73e4159930 msgr2=0x7f73e40b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 --2- 172.21.15.31:0/226932855 >> v2:172.21.15.62:3300/0 conn(0x7f73e4159930 0x7f73e40b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e415db70 con 0x7f73e4159560 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f73d4001470 con 0x7f73e4159560 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 >> v1:172.21.15.31:6789/0 conn(0x7f73e4159560 legacy=0x7f73e40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73cc003a60 0x7f73cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40cd580 0x7f73cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.117+0000 7f73e159a700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73cc004d50 0x7f73cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e415d5b0 con 0x7f73cc003a60 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e415d890 con 0x7f73cc004d50 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc00ba60 con 0x7f73e40cd580 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e1d9b700 1 --2- 172.21.15.31:0/226932855 >> v2:172.21.15.62:3300/0 conn(0x7f73e4159930 0x7f73e40b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e259c700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73cc003a60 0x7f73cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e2d9d700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73cc004d50 0x7f73cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e1d9b700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40cd580 0x7f73cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e259c700 1 -- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40cd580 msgr2=0x7f73cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e259c700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40cd580 0x7f73cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e259c700 1 -- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73cc004d50 msgr2=0x7f73cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e259c700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73cc004d50 0x7f73cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e259c700 1 -- 172.21.15.31:0/226932855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc003570 con 0x7f73cc003a60 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.118+0000 7f73e259c700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73cc003a60 0x7f73cc003e70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f73d400c3c0 tx=0x7f73d4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd662065423103b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.119+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d400cc30 con 0x7f73cc003a60 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.119+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d400cdd0 con 0x7f73cc003a60 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.119+0000 7f73e159a700 1 -- 172.21.15.31:0/226932855 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d4005b70 con 0x7f73cc003a60 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.119+0000 7f73e89ff700 1 -- 172.21.15.31:0/226932855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73cc003a60 msgr2=0x7f73cc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.119+0000 7f73e89ff700 1 --2- 172.21.15.31:0/226932855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73cc003a60 0x7f73cc003e70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f73d400c3c0 tx=0x7f73d4001e90 comp rx=0 tx=0).stop 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.119+0000 7f73e2d9d700 1 -- 172.21.15.31:0/226932855 reap_dead start 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.119+0000 7f73e89ff700 1 -- 172.21.15.31:0/226932855 shutdown_connections 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.119+0000 7f73e89ff700 1 -- 172.21.15.31:0/226932855 >> 172.21.15.31:0/226932855 conn(0x7f73e406ec90 msgr2=0x7f73e406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.119+0000 7f73e89ff700 1 -- 172.21.15.31:0/226932855 shutdown_connections 2023-10-06T05:21:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.119+0000 7f73e89ff700 1 -- 172.21.15.31:0/226932855 wait complete. 2023-10-06T05:21:51.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.121+0000 7f73e89ff700 1 Processor -- start 2023-10-06T05:21:51.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.121+0000 7f73e89ff700 1 -- start start 2023-10-06T05:21:51.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e89ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73cc003a60 0x7f73e4155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e89ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40c91d0 0x7f73e415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e89ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e415b1a0 0x7f73e41584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e89ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e415c270 con 0x7f73e415b1a0 2023-10-06T05:21:51.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e89ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e415b660 con 0x7f73cc003a60 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e89ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e415b940 con 0x7f73e40c91d0 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e415b1a0 0x7f73e41584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e415b1a0 0x7f73e41584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45890/0 (socket says 172.21.15.31:45890) 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e2d9d700 1 -- 172.21.15.31:0/2712934211 learned_addr learned my addr 172.21.15.31:0/2712934211 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e259c700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73cc003a60 0x7f73e4155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.122+0000 7f73e1d9b700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40c91d0 0x7f73e415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73e2d9d700 1 -- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40c91d0 msgr2=0x7f73e415ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73e2d9d700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40c91d0 0x7f73e415ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73e2d9d700 1 -- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73cc003a60 msgr2=0x7f73e4155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73e2d9d700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73cc003a60 0x7f73e4155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73e2d9d700 1 -- 172.21.15.31:0/2712934211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e415bc20 con 0x7f73e415b1a0 2023-10-06T05:21:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73e2d9d700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e415b1a0 0x7f73e41584e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0094d0 tx=0x7f73dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73d27fc700 1 -- 172.21.15.31:0/2712934211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc00e040 con 0x7f73e415b1a0 2023-10-06T05:21:51.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73d27fc700 1 -- 172.21.15.31:0/2712934211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc00f040 con 0x7f73e415b1a0 2023-10-06T05:21:51.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73d27fc700 1 -- 172.21.15.31:0/2712934211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc00fe20 con 0x7f73e415b1a0 2023-10-06T05:21:51.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73e89ff700 1 -- 172.21.15.31:0/2712934211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e415bf00 con 0x7f73e415b1a0 2023-10-06T05:21:51.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.123+0000 7f73e89ff700 1 -- 172.21.15.31:0/2712934211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e4158be0 con 0x7f73e415b1a0 2023-10-06T05:21:51.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.124+0000 7f73e89ff700 1 -- 172.21.15.31:0/2712934211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e415c090 con 0x7f73e415b1a0 2023-10-06T05:21:51.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.125+0000 7f73d27fc700 1 -- 172.21.15.31:0/2712934211 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f73dc015640 con 0x7f73e415b1a0 2023-10-06T05:21:51.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.126+0000 7f73d27fc700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73c4067050 0x7f73c4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.126+0000 7f73d27fc700 1 -- 172.21.15.31:0/2712934211 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f73dc084df0 con 0x7f73e415b1a0 2023-10-06T05:21:51.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.127+0000 7f73e259c700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73c4067050 0x7f73c4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.127+0000 7f73e259c700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73c4067050 0x7f73c4069500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f73d400c3c0 tx=0x7f73d4015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.131+0000 7f73d27fc700 1 -- 172.21.15.31:0/2712934211 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73e415c090 con 0x7f73e415b1a0 2023-10-06T05:21:51.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.332+0000 7f73e89ff700 1 -- 172.21.15.31:0/2712934211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73e415c090 con 0x7f73e415b1a0 2023-10-06T05:21:51.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.332+0000 7f73d27fc700 1 -- 172.21.15.31:0/2712934211 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73e415c090 con 0x7f73e415b1a0 2023-10-06T05:21:51.334 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:21:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 -- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73c4067050 msgr2=0x7f73c4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73c4067050 0x7f73c4069500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f73d400c3c0 tx=0x7f73d4015000 comp rx=0 tx=0).stop 2023-10-06T05:21:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 -- 172.21.15.31:0/2712934211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e415b1a0 msgr2=0x7f73e41584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e415b1a0 0x7f73e41584e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0094d0 tx=0x7f73dc00b040 comp rx=0 tx=0).stop 2023-10-06T05:21:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 -- 172.21.15.31:0/2712934211 shutdown_connections 2023-10-06T05:21:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73c4067050 0x7f73c4069500 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e415b1a0 0x7f73e41584e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73cc003a60 0x7f73e4155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 --2- 172.21.15.31:0/2712934211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e40c91d0 0x7f73e415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 -- 172.21.15.31:0/2712934211 >> 172.21.15.31:0/2712934211 conn(0x7f73e406ec90 msgr2=0x7f73e40b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 -- 172.21.15.31:0/2712934211 shutdown_connections 2023-10-06T05:21:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.335+0000 7f73e89ff700 1 -- 172.21.15.31:0/2712934211 wait complete. 2023-10-06T05:21:51.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:21:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.548+0000 7f429f3ff700 1 Processor -- start 2023-10-06T05:21:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.549+0000 7f429f3ff700 1 -- start start 2023-10-06T05:21:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.550+0000 7f429f3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f429813a1e0 0x7f42980c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.550+0000 7f429f3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a5b0 0x7f429806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.550+0000 7f429f3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42980c71b0 con 0x7f42980c6c30 2023-10-06T05:21:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.550+0000 7f429f3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42980c7490 con 0x7f429813a1e0 2023-10-06T05:21:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.550+0000 7f429f3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42980c7770 con 0x7f429813a5b0 2023-10-06T05:21:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.550+0000 7f429d99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f42980c6c30 0x7f429815afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35780/0 (socket says 172.21.15.31:35780) 2023-10-06T05:21:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.550+0000 7f429d19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f429813a1e0 0x7f42980c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.550+0000 7f429d99c700 1 -- 172.21.15.31:0/1055493339 learned_addr learned my addr 172.21.15.31:0/1055493339 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.550+0000 7f429c99a700 1 --2- 172.21.15.31:0/1055493339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a5b0 0x7f429806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.551+0000 7f429d19b700 1 -- 172.21.15.31:0/1055493339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a5b0 msgr2=0x7f429806d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.551+0000 7f428ffff700 1 -- 172.21.15.31:0/1055493339 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450364979 0 0) 0x7f42980c71b0 con 0x7f42980c6c30 2023-10-06T05:21:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.551+0000 7f429d19b700 1 --2- 172.21.15.31:0/1055493339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a5b0 0x7f429806d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.551+0000 7f429d19b700 1 -- 172.21.15.31:0/1055493339 >> v1:172.21.15.31:6789/0 conn(0x7f42980c6c30 legacy=0x7f429815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.551+0000 7f429d19b700 1 -- 172.21.15.31:0/1055493339 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42980c7a50 con 0x7f429813a1e0 2023-10-06T05:21:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.551+0000 7f429c99a700 1 --2- 172.21.15.31:0/1055493339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a5b0 0x7f429806d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.551+0000 7f429d19b700 1 --2- 172.21.15.31:0/1055493339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a1e0 0x7f42980c66f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f42940098d0 tx=0x7f4294009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602bcbd9b5fae443 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.551+0000 7f428ffff700 1 -- 172.21.15.31:0/1055493339 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294009e70 con 0x7f429813a1e0 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.551+0000 7f428ffff700 1 -- 172.21.15.31:0/1055493339 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4294005c10 con 0x7f429813a1e0 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.551+0000 7f428ffff700 1 -- 172.21.15.31:0/1055493339 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400b650 con 0x7f429813a1e0 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.552+0000 7f429f3ff700 1 -- 172.21.15.31:0/1055493339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a1e0 msgr2=0x7f42980c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.552+0000 7f429f3ff700 1 --2- 172.21.15.31:0/1055493339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a1e0 0x7f42980c66f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f42940098d0 tx=0x7f4294009120 comp rx=0 tx=0).stop 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.552+0000 7f429f3ff700 1 -- 172.21.15.31:0/1055493339 shutdown_connections 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.552+0000 7f429f3ff700 1 --2- 172.21.15.31:0/1055493339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a1e0 0x7f42980c66f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.552+0000 7f429f3ff700 1 --2- 172.21.15.31:0/1055493339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a5b0 0x7f429806d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.552+0000 7f429f3ff700 1 -- 172.21.15.31:0/1055493339 >> 172.21.15.31:0/1055493339 conn(0x7f42980cc0b0 msgr2=0x7f42980cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.552+0000 7f429f3ff700 1 -- 172.21.15.31:0/1055493339 shutdown_connections 2023-10-06T05:21:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.552+0000 7f429f3ff700 1 -- 172.21.15.31:0/1055493339 wait complete. 2023-10-06T05:21:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.553+0000 7f429f3ff700 1 Processor -- start 2023-10-06T05:21:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.554+0000 7f429f3ff700 1 -- start start 2023-10-06T05:21:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.554+0000 7f429f3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42980c6c30 0x7f42980cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.554+0000 7f429f3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a1e0 0x7f429814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.554+0000 7f429f3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a5b0 0x7f4298139900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.554+0000 7f429f3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42980c4130 con 0x7f429813a5b0 2023-10-06T05:21:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.554+0000 7f429f3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42980c4cb0 con 0x7f42980c6c30 2023-10-06T05:21:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429f3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42980c4380 con 0x7f429813a1e0 2023-10-06T05:21:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a5b0 0x7f4298139900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a5b0 0x7f4298139900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45918/0 (socket says 172.21.15.31:45918) 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429d99c700 1 -- 172.21.15.31:0/4291324816 learned_addr learned my addr 172.21.15.31:0/4291324816 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429d19b700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42980c6c30 0x7f42980cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429c99a700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a1e0 0x7f429814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429d99c700 1 -- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a1e0 msgr2=0x7f429814f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429d99c700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a1e0 0x7f429814f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429d99c700 1 -- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42980c6c30 msgr2=0x7f42980cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429d99c700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42980c6c30 0x7f42980cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.555+0000 7f429d99c700 1 -- 172.21.15.31:0/4291324816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42940093c0 con 0x7f429813a5b0 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.556+0000 7f429d99c700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a5b0 0x7f4298139900 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f429000ccc0 tx=0x7f4290001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.556+0000 7f428d7fa700 1 -- 172.21.15.31:0/4291324816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429000a8a0 con 0x7f429813a5b0 2023-10-06T05:21:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.556+0000 7f428d7fa700 1 -- 172.21.15.31:0/4291324816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429000aa00 con 0x7f429813a5b0 2023-10-06T05:21:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.556+0000 7f428d7fa700 1 -- 172.21.15.31:0/4291324816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429000adc0 con 0x7f429813a5b0 2023-10-06T05:21:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.556+0000 7f429f3ff700 1 -- 172.21.15.31:0/4291324816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42980c4660 con 0x7f429813a5b0 2023-10-06T05:21:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.556+0000 7f429f3ff700 1 -- 172.21.15.31:0/4291324816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42980cbb30 con 0x7f429813a5b0 2023-10-06T05:21:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.557+0000 7f429f3ff700 1 -- 172.21.15.31:0/4291324816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42980c4e40 con 0x7f429813a5b0 2023-10-06T05:21:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.558+0000 7f428d7fa700 1 -- 172.21.15.31:0/4291324816 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f4290007300 con 0x7f429813a5b0 2023-10-06T05:21:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.559+0000 7f428d7fa700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067050 0x7f427c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.559+0000 7f428d7fa700 1 -- 172.21.15.31:0/4291324816 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4290084870 con 0x7f429813a5b0 2023-10-06T05:21:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.559+0000 7f429d19b700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067050 0x7f427c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.560+0000 7f429d19b700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067050 0x7f427c069500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4294009160 tx=0x7f4294016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.563+0000 7f428d7fa700 1 -- 172.21.15.31:0/4291324816 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4290051ba0 con 0x7f429813a5b0 2023-10-06T05:21:51.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.787+0000 7f429f3ff700 1 -- 172.21.15.31:0/4291324816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f429804c380 con 0x7f429813a5b0 2023-10-06T05:21:51.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.787+0000 7f428d7fa700 1 -- 172.21.15.31:0/4291324816 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+126 (secure 0 0 0) 0x7f42900551c0 con 0x7f429813a5b0 2023-10-06T05:21:51.789 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:51.790 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:21:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:21:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:21:51.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:21:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:21:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:21:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:21:51.791 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:21:51.791 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:21:51.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 -- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067050 msgr2=0x7f427c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067050 0x7f427c069500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4294009160 tx=0x7f4294016000 comp rx=0 tx=0).stop 2023-10-06T05:21:51.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 -- 172.21.15.31:0/4291324816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a5b0 msgr2=0x7f4298139900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:51.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a5b0 0x7f4298139900 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f429000ccc0 tx=0x7f4290001a80 comp rx=0 tx=0).stop 2023-10-06T05:21:51.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 -- 172.21.15.31:0/4291324816 shutdown_connections 2023-10-06T05:21:51.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067050 0x7f427c069500 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429813a5b0 0x7f4298139900 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42980c6c30 0x7f42980cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 --2- 172.21.15.31:0/4291324816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429813a1e0 0x7f429814f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 -- 172.21.15.31:0/4291324816 >> 172.21.15.31:0/4291324816 conn(0x7f42980cc0b0 msgr2=0x7f42980afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 -- 172.21.15.31:0/4291324816 shutdown_connections 2023-10-06T05:21:51.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:51.790+0000 7f429f3ff700 1 -- 172.21.15.31:0/4291324816 wait complete. 2023-10-06T05:21:51.809 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:21:51.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:21:52.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.007+0000 7f3a29bff700 1 Processor -- start 2023-10-06T05:21:52.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.007+0000 7f3a29bff700 1 -- start start 2023-10-06T05:21:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a29bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a24142c20 0x7f3a240cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a29bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a2415a990 0x7f3a240cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a29bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a24142ff0 0x7f3a240cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a29bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2415d210 con 0x7f3a24142c20 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a29bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a240cc810 con 0x7f3a24142ff0 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a29bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2415d4b0 con 0x7f3a2415a990 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a237fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a24142c20 0x7f3a240cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a237fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a24142c20 0x7f3a240cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45920/0 (socket says 172.21.15.31:45920) 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a237fe700 1 -- 172.21.15.31:0/3534304219 learned_addr learned my addr 172.21.15.31:0/3534304219 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a22ffd700 1 --2- 172.21.15.31:0/3534304219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a2415a990 0x7f3a240cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.009+0000 7f3a23fff700 1 --2- 172.21.15.31:0/3534304219 >> v2:172.21.15.62:3300/0 conn(0x7f3a24142ff0 0x7f3a240cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.010+0000 7f3a237fe700 1 -- 172.21.15.31:0/3534304219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a2415a990 msgr2=0x7f3a240cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.010+0000 7f3a237fe700 1 --2- 172.21.15.31:0/3534304219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a2415a990 0x7f3a240cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.010+0000 7f3a237fe700 1 -- 172.21.15.31:0/3534304219 >> v2:172.21.15.62:3300/0 conn(0x7f3a24142ff0 msgr2=0x7f3a240cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.010+0000 7f3a237fe700 1 --2- 172.21.15.31:0/3534304219 >> v2:172.21.15.62:3300/0 conn(0x7f3a24142ff0 0x7f3a240cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.010+0000 7f3a237fe700 1 -- 172.21.15.31:0/3534304219 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2415d790 con 0x7f3a24142c20 2023-10-06T05:21:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.010+0000 7f3a23fff700 1 --2- 172.21.15.31:0/3534304219 >> v2:172.21.15.62:3300/0 conn(0x7f3a24142ff0 0x7f3a240cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.010+0000 7f3a237fe700 1 --2- 172.21.15.31:0/3534304219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a24142c20 0x7f3a240cbea0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3a240c51c0 tx=0x7f3a14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90509f5ef8de0e09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:52.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.010+0000 7f3a227fc700 1 -- 172.21.15.31:0/3534304219 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1400b860 con 0x7f3a24142c20 2023-10-06T05:21:52.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.010+0000 7f3a227fc700 1 -- 172.21.15.31:0/3534304219 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1400b9c0 con 0x7f3a24142c20 2023-10-06T05:21:52.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.010+0000 7f3a227fc700 1 -- 172.21.15.31:0/3534304219 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1400bd60 con 0x7f3a24142c20 2023-10-06T05:21:52.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.011+0000 7f3a29bff700 1 -- 172.21.15.31:0/3534304219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a24142c20 msgr2=0x7f3a240cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.011+0000 7f3a29bff700 1 --2- 172.21.15.31:0/3534304219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a24142c20 0x7f3a240cbea0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3a240c51c0 tx=0x7f3a14009250 comp rx=0 tx=0).stop 2023-10-06T05:21:52.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.011+0000 7f3a29bff700 1 -- 172.21.15.31:0/3534304219 shutdown_connections 2023-10-06T05:21:52.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.011+0000 7f3a29bff700 1 --2- 172.21.15.31:0/3534304219 >> v2:172.21.15.62:3300/0 conn(0x7f3a24142ff0 0x7f3a240cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.011+0000 7f3a29bff700 1 --2- 172.21.15.31:0/3534304219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a24142c20 0x7f3a240cbea0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.011+0000 7f3a29bff700 1 --2- 172.21.15.31:0/3534304219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a2415a990 0x7f3a240cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.011+0000 7f3a29bff700 1 -- 172.21.15.31:0/3534304219 >> 172.21.15.31:0/3534304219 conn(0x7f3a240ce2d0 msgr2=0x7f3a240ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:52.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.011+0000 7f3a29bff700 1 -- 172.21.15.31:0/3534304219 shutdown_connections 2023-10-06T05:21:52.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.011+0000 7f3a29bff700 1 -- 172.21.15.31:0/3534304219 wait complete. 2023-10-06T05:21:52.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.012+0000 7f3a29bff700 1 Processor -- start 2023-10-06T05:21:52.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.013+0000 7f3a29bff700 1 -- start start 2023-10-06T05:21:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.013+0000 7f3a29bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24142c20 0x7f3a2413eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.013+0000 7f3a29bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a24142ff0 0x7f3a240b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a29bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2415a990 0x7f3a2416cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a29bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24169860 con 0x7f3a2415a990 2023-10-06T05:21:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a29bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2416de60 con 0x7f3a24142c20 2023-10-06T05:21:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a29bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a240b1bb0 con 0x7f3a24142ff0 2023-10-06T05:21:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a23fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2415a990 0x7f3a2416cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a23fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2415a990 0x7f3a2416cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45932/0 (socket says 172.21.15.31:45932) 2023-10-06T05:21:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a23fff700 1 -- 172.21.15.31:0/1763302543 learned_addr learned my addr 172.21.15.31:0/1763302543 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a237fe700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24142c20 0x7f3a2413eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a22ffd700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a24142ff0 0x7f3a240b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a23fff700 1 -- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a24142ff0 msgr2=0x7f3a240b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a23fff700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a24142ff0 0x7f3a240b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a23fff700 1 -- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24142c20 msgr2=0x7f3a2413eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a23fff700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24142c20 0x7f3a2413eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a23fff700 1 -- 172.21.15.31:0/1763302543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a140094f0 con 0x7f3a2415a990 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.014+0000 7f3a23fff700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2415a990 0x7f3a2416cfd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00d180 tx=0x7f3a1c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.015+0000 7f3a0b7fe700 1 -- 172.21.15.31:0/1763302543 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c00dcc0 con 0x7f3a2415a990 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.015+0000 7f3a0b7fe700 1 -- 172.21.15.31:0/1763302543 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c00f040 con 0x7f3a2415a990 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.015+0000 7f3a0b7fe700 1 -- 172.21.15.31:0/1763302543 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c015690 con 0x7f3a2415a990 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.015+0000 7f3a29bff700 1 -- 172.21.15.31:0/1763302543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a240b1e90 con 0x7f3a2415a990 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.015+0000 7f3a29bff700 1 -- 172.21.15.31:0/1763302543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a240b2450 con 0x7f3a2415a990 2023-10-06T05:21:52.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.016+0000 7f3a29bff700 1 -- 172.21.15.31:0/1763302543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2404c380 con 0x7f3a2415a990 2023-10-06T05:21:52.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.017+0000 7f3a0b7fe700 1 -- 172.21.15.31:0/1763302543 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f3a1c01e3b0 con 0x7f3a2415a990 2023-10-06T05:21:52.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.018+0000 7f3a0b7fe700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a10067050 0x7f3a10069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:52.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.018+0000 7f3a0b7fe700 1 -- 172.21.15.31:0/1763302543 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3a1c084b00 con 0x7f3a2415a990 2023-10-06T05:21:52.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.018+0000 7f3a237fe700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a10067050 0x7f3a10069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:52.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.019+0000 7f3a237fe700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a10067050 0x7f3a10069500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a240c51c0 tx=0x7f3a14017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:52.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.021+0000 7f3a0b7fe700 1 -- 172.21.15.31:0/1763302543 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a1c086e60 con 0x7f3a2415a990 2023-10-06T05:21:52.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:52.227+0000 7f3a29bff700 1 -- 172.21.15.31:0/1763302543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3a2415da70 con 0x7f3a2415a990 2023-10-06T05:21:53.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.081+0000 7f3a0b7fe700 1 -- 172.21.15.31:0/1763302543 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3a1c01a0b0 con 0x7f3a2415a990 2023-10-06T05:21:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.086+0000 7f3a29bff700 1 -- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a10067050 msgr2=0x7f3a10069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.086+0000 7f3a29bff700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a10067050 0x7f3a10069500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a240c51c0 tx=0x7f3a14017000 comp rx=0 tx=0).stop 2023-10-06T05:21:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.087+0000 7f3a29bff700 1 -- 172.21.15.31:0/1763302543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2415a990 msgr2=0x7f3a2416cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.087+0000 7f3a29bff700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2415a990 0x7f3a2416cfd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00d180 tx=0x7f3a1c009a50 comp rx=0 tx=0).stop 2023-10-06T05:21:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.088+0000 7f3a29bff700 1 -- 172.21.15.31:0/1763302543 shutdown_connections 2023-10-06T05:21:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.088+0000 7f3a29bff700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a10067050 0x7f3a10069500 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.088+0000 7f3a29bff700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a2415a990 0x7f3a2416cfd0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.088+0000 7f3a29bff700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24142c20 0x7f3a2413eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.088+0000 7f3a29bff700 1 --2- 172.21.15.31:0/1763302543 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a24142ff0 0x7f3a240b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.088+0000 7f3a29bff700 1 -- 172.21.15.31:0/1763302543 >> 172.21.15.31:0/1763302543 conn(0x7f3a240ce2d0 msgr2=0x7f3a2406d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:53.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.088+0000 7f3a29bff700 1 -- 172.21.15.31:0/1763302543 shutdown_connections 2023-10-06T05:21:53.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.088+0000 7f3a29bff700 1 -- 172.21.15.31:0/1763302543 wait complete. 2023-10-06T05:21:53.091 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:21:53.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:21:53.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.317+0000 7f50519ff700 1 Processor -- start 2023-10-06T05:21:53.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.317+0000 7f50519ff700 1 -- start start 2023-10-06T05:21:53.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f50519ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f504c159930 0x7f504c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f50519ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f504c0cd580 0x7f504c0b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f50519ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c15d2d0 con 0x7f504c159560 2023-10-06T05:21:53.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f50519ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c15d5b0 con 0x7f504c0cd580 2023-10-06T05:21:53.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f50519ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c15d890 con 0x7f504c159930 2023-10-06T05:21:53.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f504b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f504c0cd580 0x7f504c0b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f504b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f504c0cd580 0x7f504c0b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45934/0 (socket says 172.21.15.31:45934) 2023-10-06T05:21:53.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f504b7fe700 1 -- 172.21.15.31:0/1232579516 learned_addr learned my addr 172.21.15.31:0/1232579516 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:53.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f504a7fc700 1 --2- 172.21.15.31:0/1232579516 >> v2:172.21.15.62:3300/0 conn(0x7f504c159930 0x7f504c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f504b7fe700 1 -- 172.21.15.31:0/1232579516 >> v2:172.21.15.62:3300/0 conn(0x7f504c159930 msgr2=0x7f504c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:53.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f504b7fe700 1 --2- 172.21.15.31:0/1232579516 >> v2:172.21.15.62:3300/0 conn(0x7f504c159930 0x7f504c0b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f504b7fe700 1 -- 172.21.15.31:0/1232579516 >> v1:172.21.15.62:6789/0 conn(0x7f504c159560 legacy=0x7f504c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.319+0000 7f504b7fe700 1 -- 172.21.15.31:0/1232579516 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c15db70 con 0x7f504c0cd580 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.320+0000 7f504b7fe700 1 --2- 172.21.15.31:0/1232579516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 0x7f504c0b29b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f504c089160 tx=0x7f5044009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8816e20d0c92dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.320+0000 7f5049ffb700 1 -- 172.21.15.31:0/1232579516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504400e070 con 0x7f504c0cd580 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.320+0000 7f5049ffb700 1 -- 172.21.15.31:0/1232579516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504400ba20 con 0x7f504c0cd580 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.320+0000 7f5049ffb700 1 -- 172.21.15.31:0/1232579516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50440092e0 con 0x7f504c0cd580 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.320+0000 7f50519ff700 1 -- 172.21.15.31:0/1232579516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 msgr2=0x7f504c0b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.320+0000 7f50519ff700 1 --2- 172.21.15.31:0/1232579516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 0x7f504c0b29b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f504c089160 tx=0x7f5044009840 comp rx=0 tx=0).stop 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.321+0000 7f50519ff700 1 -- 172.21.15.31:0/1232579516 shutdown_connections 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.321+0000 7f50519ff700 1 --2- 172.21.15.31:0/1232579516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 0x7f504c0b29b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.321+0000 7f50519ff700 1 --2- 172.21.15.31:0/1232579516 >> v2:172.21.15.62:3300/0 conn(0x7f504c159930 0x7f504c0b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.321+0000 7f50519ff700 1 -- 172.21.15.31:0/1232579516 >> 172.21.15.31:0/1232579516 conn(0x7f504c06ec90 msgr2=0x7f504c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.321+0000 7f50519ff700 1 -- 172.21.15.31:0/1232579516 shutdown_connections 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.321+0000 7f50519ff700 1 -- 172.21.15.31:0/1232579516 wait complete. 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.322+0000 7f50519ff700 1 Processor -- start 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.322+0000 7f50519ff700 1 -- start start 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.323+0000 7f50519ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 0x7f504c0cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.323+0000 7f50519ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f504c159560 0x7f504c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.323+0000 7f50519ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f504c159930 0x7f504c16a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.323+0000 7f50519ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0d0b70 con 0x7f504c0cd580 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.323+0000 7f50519ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0cf0d0 con 0x7f504c159560 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.323+0000 7f50519ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0c59e0 con 0x7f504c159930 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.323+0000 7f504affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 0x7f504c0cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.323+0000 7f504affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 0x7f504c0cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45938/0 (socket says 172.21.15.31:45938) 2023-10-06T05:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.323+0000 7f504affd700 1 -- 172.21.15.31:0/3259300816 learned_addr learned my addr 172.21.15.31:0/3259300816 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:53.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f504a7fc700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f504c159560 0x7f504c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f504b7fe700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f504c159930 0x7f504c16a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f504affd700 1 -- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f504c159930 msgr2=0x7f504c16a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f504affd700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f504c159930 0x7f504c16a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f504affd700 1 -- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f504c159560 msgr2=0x7f504c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:53.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f504affd700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f504c159560 0x7f504c0c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:53.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f504affd700 1 -- 172.21.15.31:0/3259300816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5044009ae0 con 0x7f504c0cd580 2023-10-06T05:21:53.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f504affd700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 0x7f504c0cf560 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c590 tx=0x7f503c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:53.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f5032ffd700 1 -- 172.21.15.31:0/3259300816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c011590 con 0x7f504c0cd580 2023-10-06T05:21:53.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f5032ffd700 1 -- 172.21.15.31:0/3259300816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c0116f0 con 0x7f504c0cd580 2023-10-06T05:21:53.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f50519ff700 1 -- 172.21.15.31:0/3259300816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c16a9b0 con 0x7f504c0cd580 2023-10-06T05:21:53.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.324+0000 7f5032ffd700 1 -- 172.21.15.31:0/3259300816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c011ab0 con 0x7f504c0cd580 2023-10-06T05:21:53.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.325+0000 7f50519ff700 1 -- 172.21.15.31:0/3259300816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504c0ce7a0 con 0x7f504c0cd580 2023-10-06T05:21:53.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.326+0000 7f50519ff700 1 -- 172.21.15.31:0/3259300816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504c04c380 con 0x7f504c0cd580 2023-10-06T05:21:53.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.327+0000 7f5032ffd700 1 -- 172.21.15.31:0/3259300816 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f503c01b3b0 con 0x7f504c0cd580 2023-10-06T05:21:53.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.328+0000 7f5032ffd700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5038067050 0x7f5038069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:53.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.328+0000 7f5032ffd700 1 -- 172.21.15.31:0/3259300816 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f503c083bd0 con 0x7f504c0cd580 2023-10-06T05:21:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.328+0000 7f504a7fc700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5038067050 0x7f5038069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.329+0000 7f504a7fc700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5038067050 0x7f5038069500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5040009ca0 tx=0x7f5040008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:53.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.332+0000 7f5032ffd700 1 -- 172.21.15.31:0/3259300816 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f503c050db0 con 0x7f504c0cd580 2023-10-06T05:21:53.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:53.539+0000 7f50519ff700 1 -- 172.21.15.31:0/3259300816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f504c0cf3b0 con 0x7f504c0cd580 2023-10-06T05:21:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.087+0000 7f5032ffd700 1 -- 172.21.15.31:0/3259300816 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f503c017070 con 0x7f504c0cd580 2023-10-06T05:21:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.095+0000 7f50519ff700 1 -- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5038067050 msgr2=0x7f5038069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.095+0000 7f50519ff700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5038067050 0x7f5038069500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5040009ca0 tx=0x7f5040008040 comp rx=0 tx=0).stop 2023-10-06T05:21:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.095+0000 7f50519ff700 1 -- 172.21.15.31:0/3259300816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 msgr2=0x7f504c0cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.095+0000 7f50519ff700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 0x7f504c0cf560 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c590 tx=0x7f503c00c950 comp rx=0 tx=0).stop 2023-10-06T05:21:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.095+0000 7f50519ff700 1 -- 172.21.15.31:0/3259300816 shutdown_connections 2023-10-06T05:21:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.095+0000 7f50519ff700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5038067050 0x7f5038069500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.095+0000 7f50519ff700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f504c159930 0x7f504c16a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.095+0000 7f50519ff700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f504c0cd580 0x7f504c0cf560 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.095+0000 7f50519ff700 1 --2- 172.21.15.31:0/3259300816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f504c159560 0x7f504c0c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.095+0000 7f50519ff700 1 -- 172.21.15.31:0/3259300816 >> 172.21.15.31:0/3259300816 conn(0x7f504c06ec90 msgr2=0x7f504c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:54.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.096+0000 7f50519ff700 1 -- 172.21.15.31:0/3259300816 shutdown_connections 2023-10-06T05:21:54.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.096+0000 7f50519ff700 1 -- 172.21.15.31:0/3259300816 wait complete. 2023-10-06T05:21:54.112 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:21:54.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:21:54.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.326+0000 7fab689ff700 1 Processor -- start 2023-10-06T05:21:54.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.326+0000 7fab689ff700 1 -- start start 2023-10-06T05:21:54.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.327+0000 7fab689ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fab640c8f40 0x7fab64068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.327+0000 7fab689ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab64068da0 con 0x7fab6415ca40 2023-10-06T05:21:54.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.327+0000 7fab689ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab64069080 con 0x7fab6406a300 2023-10-06T05:21:54.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.327+0000 7fab689ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab640cabc0 con 0x7fab640c8f40 2023-10-06T05:21:54.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.327+0000 7fab6259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fab6415ca40 0x7fab640c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35796/0 (socket says 172.21.15.31:35796) 2023-10-06T05:21:54.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.327+0000 7fab6259c700 1 -- 172.21.15.31:0/1271284805 learned_addr learned my addr 172.21.15.31:0/1271284805 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.327+0000 7fab61d9b700 1 --2- 172.21.15.31:0/1271284805 >> v2:172.21.15.62:3300/0 conn(0x7fab640c8f40 0x7fab64068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.328+0000 7fab6159a700 1 -- 172.21.15.31:0/1271284805 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023059426 0 0) 0x7fab64068da0 con 0x7fab6415ca40 2023-10-06T05:21:54.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.328+0000 7fab6159a700 1 -- 172.21.15.31:0/1271284805 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab4c003570 con 0x7fab6415ca40 2023-10-06T05:21:54.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.328+0000 7fab61d9b700 1 -- 172.21.15.31:0/1271284805 >> v1:172.21.15.62:6789/0 conn(0x7fab6406a300 legacy=0x7fab640adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.328+0000 7fab61d9b700 1 -- 172.21.15.31:0/1271284805 >> v1:172.21.15.31:6789/0 conn(0x7fab6415ca40 legacy=0x7fab640c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.328+0000 7fab61d9b700 1 -- 172.21.15.31:0/1271284805 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab64069360 con 0x7fab640c8f40 2023-10-06T05:21:54.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.328+0000 7fab61d9b700 1 --2- 172.21.15.31:0/1271284805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab640c8f40 0x7fab64068670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fab580098e0 tx=0x7fab580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63de2c840f0218d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.329+0000 7fab6159a700 1 -- 172.21.15.31:0/1271284805 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5800b640 con 0x7fab640c8f40 2023-10-06T05:21:54.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.329+0000 7fab6159a700 1 -- 172.21.15.31:0/1271284805 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5800b7a0 con 0x7fab640c8f40 2023-10-06T05:21:54.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.329+0000 7fab6159a700 1 -- 172.21.15.31:0/1271284805 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5800bb40 con 0x7fab640c8f40 2023-10-06T05:21:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.329+0000 7fab689ff700 1 -- 172.21.15.31:0/1271284805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab640c8f40 msgr2=0x7fab64068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.329+0000 7fab689ff700 1 --2- 172.21.15.31:0/1271284805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab640c8f40 0x7fab64068670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fab580098e0 tx=0x7fab580091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.329+0000 7fab689ff700 1 -- 172.21.15.31:0/1271284805 shutdown_connections 2023-10-06T05:21:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.329+0000 7fab689ff700 1 --2- 172.21.15.31:0/1271284805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab640c8f40 0x7fab64068670 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.329+0000 7fab689ff700 1 -- 172.21.15.31:0/1271284805 >> 172.21.15.31:0/1271284805 conn(0x7fab640ce290 msgr2=0x7fab640ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.329+0000 7fab689ff700 1 -- 172.21.15.31:0/1271284805 shutdown_connections 2023-10-06T05:21:54.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.329+0000 7fab689ff700 1 -- 172.21.15.31:0/1271284805 wait complete. 2023-10-06T05:21:54.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.331+0000 7fab689ff700 1 Processor -- start 2023-10-06T05:21:54.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.331+0000 7fab689ff700 1 -- start start 2023-10-06T05:21:54.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab689ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab6406a300 0x7fab6414d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab689ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab640c8f40 0x7fab640c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab689ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab6415ca40 0x7fab6415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab689ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6416d910 con 0x7fab640c8f40 2023-10-06T05:21:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab689ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6416cd00 con 0x7fab6415ca40 2023-10-06T05:21:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab689ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6416cfe0 con 0x7fab6406a300 2023-10-06T05:21:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab61d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab640c8f40 0x7fab640c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab61d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab640c8f40 0x7fab640c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45944/0 (socket says 172.21.15.31:45944) 2023-10-06T05:21:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab61d9b700 1 -- 172.21.15.31:0/3994495833 learned_addr learned my addr 172.21.15.31:0/3994495833 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab6259c700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab6406a300 0x7fab6414d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.332+0000 7fab62d9d700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab6415ca40 0x7fab6415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab61d9b700 1 -- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab6406a300 msgr2=0x7fab6414d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab61d9b700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab6406a300 0x7fab6414d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab61d9b700 1 -- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab6415ca40 msgr2=0x7fab6415a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab61d9b700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab6415ca40 0x7fab6415a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab61d9b700 1 -- 172.21.15.31:0/3994495833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab58009420 con 0x7fab640c8f40 2023-10-06T05:21:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab61d9b700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab640c8f40 0x7fab640c4510 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fab6406e040 tx=0x7fab58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab527fc700 1 -- 172.21.15.31:0/3994495833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5800bb20 con 0x7fab640c8f40 2023-10-06T05:21:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab527fc700 1 -- 172.21.15.31:0/3994495833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab58011420 con 0x7fab640c8f40 2023-10-06T05:21:54.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab527fc700 1 -- 172.21.15.31:0/3994495833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab580117c0 con 0x7fab640c8f40 2023-10-06T05:21:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab689ff700 1 -- 172.21.15.31:0/3994495833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6416d2c0 con 0x7fab640c8f40 2023-10-06T05:21:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.333+0000 7fab689ff700 1 -- 172.21.15.31:0/3994495833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab640c4a50 con 0x7fab640c8f40 2023-10-06T05:21:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.334+0000 7fab689ff700 1 -- 172.21.15.31:0/3994495833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6404c380 con 0x7fab640c8f40 2023-10-06T05:21:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.335+0000 7fab527fc700 1 -- 172.21.15.31:0/3994495833 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fab58011b70 con 0x7fab640c8f40 2023-10-06T05:21:54.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.337+0000 7fab527fc700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab44067050 0x7fab44069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:54.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.337+0000 7fab527fc700 1 -- 172.21.15.31:0/3994495833 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fab58085f40 con 0x7fab640c8f40 2023-10-06T05:21:54.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.338+0000 7fab6259c700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab44067050 0x7fab44069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:54.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.338+0000 7fab6259c700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab44067050 0x7fab44069500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab540058d0 tx=0x7fab54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:54.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.342+0000 7fab527fc700 1 -- 172.21.15.31:0/3994495833 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab58052f00 con 0x7fab640c8f40 2023-10-06T05:21:54.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:54.589+0000 7fab689ff700 1 -- 172.21.15.31:0/3994495833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fab6416da50 con 0x7fab640c8f40 2023-10-06T05:21:55.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.090+0000 7fab527fc700 1 -- 172.21.15.31:0/3994495833 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v50) v1 ==== 174+0+0 (secure 0 0 0) 0x7fab58056520 con 0x7fab640c8f40 2023-10-06T05:21:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.096+0000 7fab689ff700 1 -- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab44067050 msgr2=0x7fab44069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.096+0000 7fab689ff700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab44067050 0x7fab44069500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab540058d0 tx=0x7fab54009040 comp rx=0 tx=0).stop 2023-10-06T05:21:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.096+0000 7fab689ff700 1 -- 172.21.15.31:0/3994495833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab640c8f40 msgr2=0x7fab640c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.096+0000 7fab689ff700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab640c8f40 0x7fab640c4510 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fab6406e040 tx=0x7fab58017000 comp rx=0 tx=0).stop 2023-10-06T05:21:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.099+0000 7fab689ff700 1 -- 172.21.15.31:0/3994495833 shutdown_connections 2023-10-06T05:21:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.099+0000 7fab689ff700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab44067050 0x7fab44069500 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.099+0000 7fab689ff700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab6415ca40 0x7fab6415a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.099+0000 7fab689ff700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab6406a300 0x7fab6414d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.099+0000 7fab689ff700 1 --2- 172.21.15.31:0/3994495833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab640c8f40 0x7fab640c4510 secure :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fab6406e040 tx=0x7fab58017000 comp rx=0 tx=0).stop 2023-10-06T05:21:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.099+0000 7fab689ff700 1 -- 172.21.15.31:0/3994495833 >> 172.21.15.31:0/3994495833 conn(0x7fab640ce290 msgr2=0x7fab640b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.102+0000 7fab689ff700 1 -- 172.21.15.31:0/3994495833 shutdown_connections 2023-10-06T05:21:55.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.102+0000 7fab689ff700 1 -- 172.21.15.31:0/3994495833 wait complete. 2023-10-06T05:21:55.122 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 13 and data pool 14 2023-10-06T05:21:55.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:55.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.337+0000 7f48cc7ff700 1 Processor -- start 2023-10-06T05:21:55.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.337+0000 7f48cc7ff700 1 -- start start 2023-10-06T05:21:55.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.338+0000 7f48cc7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f48c413a1e0 0x7f48c40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.338+0000 7f48cc7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c413a5b0 0x7f48c406d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48cc7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f48c40c6c30 0x7f48c406d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48cc7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c40c71b0 con 0x7f48c413a1e0 2023-10-06T05:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48cc7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c40c7490 con 0x7f48c40c6c30 2023-10-06T05:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48cc7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c40c7770 con 0x7f48c413a5b0 2023-10-06T05:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48ca59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f48c413a1e0 0x7f48c40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48ca59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f48c413a1e0 0x7f48c40c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45952/0 (socket says 172.21.15.31:45952) 2023-10-06T05:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48ca59b700 1 -- 172.21.15.31:0/4071087137 learned_addr learned my addr 172.21.15.31:0/4071087137 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48ca59b700 1 -- 172.21.15.31:0/4071087137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c413a5b0 msgr2=0x7f48c406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48ca59b700 1 --2- 172.21.15.31:0/4071087137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c413a5b0 0x7f48c406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48ca59b700 1 -- 172.21.15.31:0/4071087137 >> v2:172.21.15.62:3300/0 conn(0x7f48c40c6c30 msgr2=0x7f48c406d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48cad9c700 1 --2- 172.21.15.31:0/4071087137 >> v2:172.21.15.62:3300/0 conn(0x7f48c40c6c30 0x7f48c406d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48ca59b700 1 --2- 172.21.15.31:0/4071087137 >> v2:172.21.15.62:3300/0 conn(0x7f48c40c6c30 0x7f48c406d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48ca59b700 1 -- 172.21.15.31:0/4071087137 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c40c7a50 con 0x7f48c413a1e0 2023-10-06T05:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.339+0000 7f48cad9c700 1 --2- 172.21.15.31:0/4071087137 >> v2:172.21.15.62:3300/0 conn(0x7f48c40c6c30 0x7f48c406d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.340+0000 7f48ca59b700 1 --2- 172.21.15.31:0/4071087137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a1e0 0x7f48c40c66f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f48c406b530 tx=0x7f48c0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fd52cf1e9d45bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.340+0000 7f48c9599700 1 -- 172.21.15.31:0/4071087137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0005c10 con 0x7f48c413a1e0 2023-10-06T05:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.340+0000 7f48c9599700 1 -- 172.21.15.31:0/4071087137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c0005d70 con 0x7f48c413a1e0 2023-10-06T05:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.340+0000 7f48c9599700 1 -- 172.21.15.31:0/4071087137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c000b880 con 0x7f48c413a1e0 2023-10-06T05:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.340+0000 7f48cc7ff700 1 -- 172.21.15.31:0/4071087137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a1e0 msgr2=0x7f48c40c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.340+0000 7f48cc7ff700 1 --2- 172.21.15.31:0/4071087137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a1e0 0x7f48c40c66f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f48c406b530 tx=0x7f48c0009220 comp rx=0 tx=0).stop 2023-10-06T05:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.341+0000 7f48cc7ff700 1 -- 172.21.15.31:0/4071087137 shutdown_connections 2023-10-06T05:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.341+0000 7f48cc7ff700 1 --2- 172.21.15.31:0/4071087137 >> v2:172.21.15.62:3300/0 conn(0x7f48c40c6c30 0x7f48c406d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.341+0000 7f48cc7ff700 1 --2- 172.21.15.31:0/4071087137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a1e0 0x7f48c40c66f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.341+0000 7f48cc7ff700 1 --2- 172.21.15.31:0/4071087137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c413a5b0 0x7f48c406d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.341+0000 7f48cc7ff700 1 -- 172.21.15.31:0/4071087137 >> 172.21.15.31:0/4071087137 conn(0x7f48c40cc0b0 msgr2=0x7f48c40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.341+0000 7f48cc7ff700 1 -- 172.21.15.31:0/4071087137 shutdown_connections 2023-10-06T05:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.341+0000 7f48cc7ff700 1 -- 172.21.15.31:0/4071087137 wait complete. 2023-10-06T05:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.343+0000 7f48cc7ff700 1 Processor -- start 2023-10-06T05:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.343+0000 7f48cc7ff700 1 -- start start 2023-10-06T05:21:55.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.344+0000 7f48cc7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c40c6c30 0x7f48c416d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.344+0000 7f48cc7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c413a1e0 0x7f48c40c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.344+0000 7f48cc7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a5b0 0x7f48c40cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.344+0000 7f48cc7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c4156f20 con 0x7f48c413a5b0 2023-10-06T05:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.344+0000 7f48cc7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c41571c0 con 0x7f48c40c6c30 2023-10-06T05:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.344+0000 7f48cc7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c41574a0 con 0x7f48c413a1e0 2023-10-06T05:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.344+0000 7f48cad9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a5b0 0x7f48c40cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48cad9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a5b0 0x7f48c40cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45968/0 (socket says 172.21.15.31:45968) 2023-10-06T05:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48cad9c700 1 -- 172.21.15.31:0/1079884804 learned_addr learned my addr 172.21.15.31:0/1079884804 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48ca59b700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c40c6c30 0x7f48c416d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48c9d9a700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c413a1e0 0x7f48c40c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48cad9c700 1 -- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c413a1e0 msgr2=0x7f48c40c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48cad9c700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c413a1e0 0x7f48c40c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48cad9c700 1 -- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c40c6c30 msgr2=0x7f48c416d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48cad9c700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c40c6c30 0x7f48c416d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48cad9c700 1 -- 172.21.15.31:0/1079884804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0009480 con 0x7f48c413a5b0 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48ca59b700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c40c6c30 0x7f48c416d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48c9d9a700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c413a1e0 0x7f48c40c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48cad9c700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a5b0 0x7f48c40cac90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00a380 tx=0x7f48bc007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48ba7fc700 1 -- 172.21.15.31:0/1079884804 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00ae10 con 0x7f48c413a5b0 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48ba7fc700 1 -- 172.21.15.31:0/1079884804 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48bc00d650 con 0x7f48c413a5b0 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48cc7ff700 1 -- 172.21.15.31:0/1079884804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c4157780 con 0x7f48c413a5b0 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.345+0000 7f48ba7fc700 1 -- 172.21.15.31:0/1079884804 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00da10 con 0x7f48c413a5b0 2023-10-06T05:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.346+0000 7f48cc7ff700 1 -- 172.21.15.31:0/1079884804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c416e210 con 0x7f48c413a5b0 2023-10-06T05:21:55.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.347+0000 7f48cc7ff700 1 -- 172.21.15.31:0/1079884804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c404c380 con 0x7f48c413a5b0 2023-10-06T05:21:55.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.348+0000 7f48ba7fc700 1 -- 172.21.15.31:0/1079884804 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f48bc01e3b0 con 0x7f48c413a5b0 2023-10-06T05:21:55.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.348+0000 7f48ba7fc700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f48a8067050 0x7f48a8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.348+0000 7f48ba7fc700 1 -- 172.21.15.31:0/1079884804 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f48bc085250 con 0x7f48c413a5b0 2023-10-06T05:21:55.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.349+0000 7f48ca59b700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f48a8067050 0x7f48a8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.350+0000 7f48ca59b700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f48a8067050 0x7f48a8069500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48c40af6a0 tx=0x7f48c0016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.352+0000 7f48ba7fc700 1 -- 172.21.15.31:0/1079884804 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48bc052250 con 0x7f48c413a5b0 2023-10-06T05:21:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.553+0000 7f48cc7ff700 1 -- 172.21.15.31:0/1079884804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48c40c7d30 con 0x7f48c413a5b0 2023-10-06T05:21:55.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.554+0000 7f48ba7fc700 1 -- 172.21.15.31:0/1079884804 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16930 (secure 0 0 0) 0x7f48bc055870 con 0x7f48c413a5b0 2023-10-06T05:21:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:55.556 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":82,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:54.590340+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:52.267847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:53.579487+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.556+0000 7f48cc7ff700 1 -- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f48a8067050 msgr2=0x7f48a8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.556+0000 7f48cc7ff700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f48a8067050 0x7f48a8069500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48c40af6a0 tx=0x7f48c0016000 comp rx=0 tx=0).stop 2023-10-06T05:21:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.556+0000 7f48cc7ff700 1 -- 172.21.15.31:0/1079884804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a5b0 msgr2=0x7f48c40cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.556+0000 7f48cc7ff700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a5b0 0x7f48c40cac90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00a380 tx=0x7f48bc007900 comp rx=0 tx=0).stop 2023-10-06T05:21:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.556+0000 7f48cc7ff700 1 -- 172.21.15.31:0/1079884804 shutdown_connections 2023-10-06T05:21:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.556+0000 7f48cc7ff700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f48a8067050 0x7f48a8069500 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.556+0000 7f48cc7ff700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48c413a5b0 0x7f48c40cac90 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.556+0000 7f48cc7ff700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c40c6c30 0x7f48c416d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.556+0000 7f48cc7ff700 1 --2- 172.21.15.31:0/1079884804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48c413a1e0 0x7f48c40c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.556+0000 7f48cc7ff700 1 -- 172.21.15.31:0/1079884804 >> 172.21.15.31:0/1079884804 conn(0x7f48c40cc0b0 msgr2=0x7f48c40b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.557+0000 7f48cc7ff700 1 -- 172.21.15.31:0/1079884804 shutdown_connections 2023-10-06T05:21:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.557+0000 7f48cc7ff700 1 -- 172.21.15.31:0/1079884804 wait complete. 2023-10-06T05:21:55.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:55.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.771+0000 7fcfe33ff700 1 Processor -- start 2023-10-06T05:21:55.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.771+0000 7fcfe33ff700 1 -- start start 2023-10-06T05:21:55.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.772+0000 7fcfe33ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcfdc13a5b0 0x7fcfdc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.772+0000 7fcfe33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc06d980 0x7fcfdc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.772+0000 7fcfe33ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc06ddb0 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.772+0000 7fcfe33ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfdc0c71b0 con 0x7fcfdc13a5b0 2023-10-06T05:21:55.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.773+0000 7fcfe33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0c7490 con 0x7fcfdc06d980 2023-10-06T05:21:55.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.773+0000 7fcfe119b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcfdc13a1e0 0x7fcfdc0c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35798/0 (socket says 172.21.15.31:35798) 2023-10-06T05:21:55.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.773+0000 7fcfe119b700 1 -- 172.21.15.31:0/2877038778 learned_addr learned my addr 172.21.15.31:0/2877038778 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:55.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.773+0000 7fcfe099a700 1 --2- 172.21.15.31:0/2877038778 >> v2:172.21.15.62:3300/0 conn(0x7fcfdc13a5b0 0x7fcfdc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.773+0000 7fcfe199c700 1 --2- 172.21.15.31:0/2877038778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc06d980 0x7fcfdc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.773+0000 7fcfd3fff700 1 -- 172.21.15.31:0/2877038778 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896353132 0 0) 0x7fcfdc06ddb0 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.773+0000 7fcfd3fff700 1 -- 172.21.15.31:0/2877038778 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc8003570 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.774+0000 7fcfe099a700 1 -- 172.21.15.31:0/2877038778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc06d980 msgr2=0x7fcfdc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:55.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.774+0000 7fcfe099a700 1 --2- 172.21.15.31:0/2877038778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc06d980 0x7fcfdc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.774+0000 7fcfe099a700 1 -- 172.21.15.31:0/2877038778 >> v1:172.21.15.31:6789/0 conn(0x7fcfdc13a1e0 legacy=0x7fcfdc0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.774+0000 7fcfd3fff700 1 -- 172.21.15.31:0/2877038778 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3367061578 0 0) 0x7fcfc8003570 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.774+0000 7fcfe099a700 1 -- 172.21.15.31:0/2877038778 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc0c7770 con 0x7fcfdc13a5b0 2023-10-06T05:21:55.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.774+0000 7fcfe199c700 1 --2- 172.21.15.31:0/2877038778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc06d980 0x7fcfdc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:55.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.774+0000 7fcfe099a700 1 --2- 172.21.15.31:0/2877038778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc13a5b0 0x7fcfdc06d440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0098e0 tx=0x7fcfcc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a8a25bf3d40f944 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.774+0000 7fcfd3fff700 1 -- 172.21.15.31:0/2877038778 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc011630 con 0x7fcfdc13a5b0 2023-10-06T05:21:55.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.774+0000 7fcfd3fff700 1 -- 172.21.15.31:0/2877038778 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc011790 con 0x7fcfdc13a5b0 2023-10-06T05:21:55.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.774+0000 7fcfd3fff700 1 -- 172.21.15.31:0/2877038778 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc011b10 con 0x7fcfdc13a5b0 2023-10-06T05:21:55.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.775+0000 7fcfe33ff700 1 -- 172.21.15.31:0/2877038778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc13a5b0 msgr2=0x7fcfdc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.775+0000 7fcfe33ff700 1 --2- 172.21.15.31:0/2877038778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc13a5b0 0x7fcfdc06d440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0098e0 tx=0x7fcfcc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.775+0000 7fcfe33ff700 1 -- 172.21.15.31:0/2877038778 shutdown_connections 2023-10-06T05:21:55.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.775+0000 7fcfe33ff700 1 --2- 172.21.15.31:0/2877038778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc06d980 0x7fcfdc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.775+0000 7fcfe33ff700 1 --2- 172.21.15.31:0/2877038778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc13a5b0 0x7fcfdc06d440 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.775+0000 7fcfe33ff700 1 -- 172.21.15.31:0/2877038778 >> 172.21.15.31:0/2877038778 conn(0x7fcfdc0cc0b0 msgr2=0x7fcfdc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.775+0000 7fcfe33ff700 1 -- 172.21.15.31:0/2877038778 shutdown_connections 2023-10-06T05:21:55.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.775+0000 7fcfe33ff700 1 -- 172.21.15.31:0/2877038778 wait complete. 2023-10-06T05:21:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.776+0000 7fcfe33ff700 1 Processor -- start 2023-10-06T05:21:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.777+0000 7fcfe33ff700 1 -- start start 2023-10-06T05:21:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.777+0000 7fcfe33ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc06d980 0x7fcfdc125aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.777+0000 7fcfe33ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfdc13a1e0 0x7fcfdc167080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc13a5b0 0x7fcfdc13c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe33ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0c4e40 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe33ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0c3920 con 0x7fcfdc06d980 2023-10-06T05:21:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0c56c0 con 0x7fcfdc13a5b0 2023-10-06T05:21:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfdc13a1e0 0x7fcfdc167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfdc13a1e0 0x7fcfdc167080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45984/0 (socket says 172.21.15.31:45984) 2023-10-06T05:21:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe099a700 1 -- 172.21.15.31:0/563159238 learned_addr learned my addr 172.21.15.31:0/563159238 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe199c700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc13a5b0 0x7fcfdc13c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe119b700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc06d980 0x7fcfdc125aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe099a700 1 -- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc13a5b0 msgr2=0x7fcfdc13c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe099a700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc13a5b0 0x7fcfdc13c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe099a700 1 -- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc06d980 msgr2=0x7fcfdc125aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe099a700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc06d980 0x7fcfdc125aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.778+0000 7fcfe099a700 1 -- 172.21.15.31:0/563159238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd4009a20 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.779+0000 7fcfe099a700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfdc13a1e0 0x7fcfdc167080 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0cc010 tx=0x7fcfcc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.779+0000 7fcfd17fa700 1 -- 172.21.15.31:0/563159238 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc011b10 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.779+0000 7fcfd17fa700 1 -- 172.21.15.31:0/563159238 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc000c90 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.779+0000 7fcfd17fa700 1 -- 172.21.15.31:0/563159238 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc01a660 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.779+0000 7fcfe33ff700 1 -- 172.21.15.31:0/563159238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc009420 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.779+0000 7fcfe33ff700 1 -- 172.21.15.31:0/563159238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc0c5350 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.780+0000 7fcfe33ff700 1 -- 172.21.15.31:0/563159238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfdc0c5870 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.781+0000 7fcfd17fa700 1 -- 172.21.15.31:0/563159238 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fcfcc01aa90 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.782+0000 7fcfd17fa700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcfc0067050 0x7fcfc0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:55.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.782+0000 7fcfd17fa700 1 -- 172.21.15.31:0/563159238 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcfcc086330 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.783+0000 7fcfe119b700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcfc0067050 0x7fcfc0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:55.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.783+0000 7fcfe119b700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcfc0067050 0x7fcfc0069500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80015c0 tx=0x7fcfd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:55.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.786+0000 7fcfd17fa700 1 -- 172.21.15.31:0/563159238 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfdc0c5870 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.990+0000 7fcfe33ff700 1 -- 172.21.15.31:0/563159238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfdc0c5830 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.991+0000 7fcfd17fa700 1 -- 172.21.15.31:0/563159238 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16930 (secure 0 0 0) 0x7fcfcc0533f0 con 0x7fcfdc13a1e0 2023-10-06T05:21:55.994 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:55.994 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":82,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:54.590340+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:52.267847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:53.579487+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:55.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.993+0000 7fcfe33ff700 1 -- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcfc0067050 msgr2=0x7fcfc0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.993+0000 7fcfe33ff700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcfc0067050 0x7fcfc0069500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80015c0 tx=0x7fcfd8009040 comp rx=0 tx=0).stop 2023-10-06T05:21:55.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.993+0000 7fcfe33ff700 1 -- 172.21.15.31:0/563159238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfdc13a1e0 msgr2=0x7fcfdc167080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:55.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.993+0000 7fcfe33ff700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfdc13a1e0 0x7fcfdc167080 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0cc010 tx=0x7fcfcc017000 comp rx=0 tx=0).stop 2023-10-06T05:21:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.993+0000 7fcfe33ff700 1 -- 172.21.15.31:0/563159238 shutdown_connections 2023-10-06T05:21:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.993+0000 7fcfe33ff700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcfc0067050 0x7fcfc0069500 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.993+0000 7fcfe33ff700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc13a5b0 0x7fcfdc13c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.993+0000 7fcfe33ff700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc06d980 0x7fcfdc125aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.993+0000 7fcfe33ff700 1 --2- 172.21.15.31:0/563159238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfdc13a1e0 0x7fcfdc167080 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.993+0000 7fcfe33ff700 1 -- 172.21.15.31:0/563159238 >> 172.21.15.31:0/563159238 conn(0x7fcfdc0cc0b0 msgr2=0x7fcfdc0af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.994+0000 7fcfe33ff700 1 -- 172.21.15.31:0/563159238 shutdown_connections 2023-10-06T05:21:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:55.994+0000 7fcfe33ff700 1 -- 172.21.15.31:0/563159238 wait complete. 2023-10-06T05:21:56.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:21:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.215+0000 7f328fbff700 1 Processor -- start 2023-10-06T05:21:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.215+0000 7f328fbff700 1 -- start start 2023-10-06T05:21:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328fbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32880c7190 0x7f32880c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328fbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880c7560 0x7f32880686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328fbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3288068250 con 0x7f32880aef00 2023-10-06T05:21:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328fbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328806ac80 con 0x7f32880c7190 2023-10-06T05:21:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328fbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3288067f20 con 0x7f32880c7560 2023-10-06T05:21:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328d99b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32880c7190 0x7f32880c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328d99b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32880c7190 0x7f32880c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45994/0 (socket says 172.21.15.31:45994) 2023-10-06T05:21:56.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328d99b700 1 -- 172.21.15.31:0/1789496905 learned_addr learned my addr 172.21.15.31:0/1789496905 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328e19c700 1 --2- 172.21.15.31:0/1789496905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880c7560 0x7f32880686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328d99b700 1 -- 172.21.15.31:0/1789496905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880c7560 msgr2=0x7f32880686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328d99b700 1 --2- 172.21.15.31:0/1789496905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880c7560 0x7f32880686e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328d99b700 1 -- 172.21.15.31:0/1789496905 >> v1:172.21.15.31:6789/0 conn(0x7f32880aef00 legacy=0x7f32880af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328d99b700 1 -- 172.21.15.31:0/1789496905 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328806aed0 con 0x7f32880c7190 2023-10-06T05:21:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.217+0000 7f328c999700 1 -- 172.21.15.31:0/1789496905 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833213085 0 0) 0x7f3288068250 con 0x7f32880aef00 2023-10-06T05:21:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.218+0000 7f328d99b700 1 --2- 172.21.15.31:0/1789496905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7190 0x7f32880c9d40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f32840099b0 tx=0x7f3284009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4143cb5c8f077f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.218+0000 7f328c999700 1 -- 172.21.15.31:0/1789496905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328400b860 con 0x7f32880c7190 2023-10-06T05:21:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.218+0000 7f328c999700 1 -- 172.21.15.31:0/1789496905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328400b9c0 con 0x7f32880c7190 2023-10-06T05:21:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.218+0000 7f328c999700 1 -- 172.21.15.31:0/1789496905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328400bd60 con 0x7f32880c7190 2023-10-06T05:21:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.218+0000 7f328fbff700 1 -- 172.21.15.31:0/1789496905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7190 msgr2=0x7f32880c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.218+0000 7f328fbff700 1 --2- 172.21.15.31:0/1789496905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7190 0x7f32880c9d40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f32840099b0 tx=0x7f3284009250 comp rx=0 tx=0).stop 2023-10-06T05:21:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.219+0000 7f328fbff700 1 -- 172.21.15.31:0/1789496905 shutdown_connections 2023-10-06T05:21:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.219+0000 7f328fbff700 1 --2- 172.21.15.31:0/1789496905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7190 0x7f32880c9d40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.219+0000 7f328fbff700 1 --2- 172.21.15.31:0/1789496905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880c7560 0x7f32880686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.219+0000 7f328fbff700 1 -- 172.21.15.31:0/1789496905 >> 172.21.15.31:0/1789496905 conn(0x7f32880cd3d0 msgr2=0x7f32880c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:56.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.219+0000 7f328fbff700 1 -- 172.21.15.31:0/1789496905 shutdown_connections 2023-10-06T05:21:56.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.219+0000 7f328fbff700 1 -- 172.21.15.31:0/1789496905 wait complete. 2023-10-06T05:21:56.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.221+0000 7f328fbff700 1 Processor -- start 2023-10-06T05:21:56.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.221+0000 7f328fbff700 1 -- start start 2023-10-06T05:21:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.222+0000 7f328fbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880aef00 0x7f32880c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.222+0000 7f328fbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32880c7190 0x7f328814ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.222+0000 7f328fbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7560 0x7f328816d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.222+0000 7f328fbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32880c2cf0 con 0x7f32880c7560 2023-10-06T05:21:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.222+0000 7f328fbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328815c9a0 con 0x7f32880c7190 2023-10-06T05:21:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.222+0000 7f328fbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32880c2450 con 0x7f32880aef00 2023-10-06T05:21:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328e19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7560 0x7f328816d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328e19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7560 0x7f328816d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46010/0 (socket says 172.21.15.31:46010) 2023-10-06T05:21:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328e19c700 1 -- 172.21.15.31:0/2057996121 learned_addr learned my addr 172.21.15.31:0/2057996121 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328d99b700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880aef00 0x7f32880c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328e19c700 1 -- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880aef00 msgr2=0x7f32880c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328d19a700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32880c7190 0x7f328814ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328e19c700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880aef00 0x7f32880c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328e19c700 1 -- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32880c7190 msgr2=0x7f328814ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328e19c700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32880c7190 0x7f328814ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328e19c700 1 -- 172.21.15.31:0/2057996121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840094f0 con 0x7f32880c7560 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328d99b700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880aef00 0x7f32880c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328d19a700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32880c7190 0x7f328814ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f328e19c700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7560 0x7f328816d0d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3280009fa0 tx=0x7f3280009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.223+0000 7f327dffb700 1 -- 172.21.15.31:0/2057996121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328000ed20 con 0x7f32880c7560 2023-10-06T05:21:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.224+0000 7f327dffb700 1 -- 172.21.15.31:0/2057996121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328000ee80 con 0x7f32880c7560 2023-10-06T05:21:56.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.224+0000 7f327dffb700 1 -- 172.21.15.31:0/2057996121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280014670 con 0x7f32880c7560 2023-10-06T05:21:56.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.224+0000 7f328fbff700 1 -- 172.21.15.31:0/2057996121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32880c2730 con 0x7f32880c7560 2023-10-06T05:21:56.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.224+0000 7f328fbff700 1 -- 172.21.15.31:0/2057996121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32880c3ae0 con 0x7f32880c7560 2023-10-06T05:21:56.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.225+0000 7f328fbff700 1 -- 172.21.15.31:0/2057996121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328804c380 con 0x7f32880c7560 2023-10-06T05:21:56.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.226+0000 7f327dffb700 1 -- 172.21.15.31:0/2057996121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f32800149e0 con 0x7f32880c7560 2023-10-06T05:21:56.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.226+0000 7f327dffb700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f326c067050 0x7f326c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:56.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.227+0000 7f327dffb700 1 -- 172.21.15.31:0/2057996121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3280085090 con 0x7f32880c7560 2023-10-06T05:21:56.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.227+0000 7f328d99b700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f326c067050 0x7f326c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:56.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.228+0000 7f328d99b700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f326c067050 0x7f326c069500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f32840099b0 tx=0x7f3284017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:56.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.231+0000 7f327dffb700 1 -- 172.21.15.31:0/2057996121 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3280051ff0 con 0x7f32880c7560 2023-10-06T05:21:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:56.477+0000 7f328fbff700 1 -- 172.21.15.31:0/2057996121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f32880cc6e0 con 0x7f32880c7560 2023-10-06T05:21:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.110+0000 7f327dffb700 1 -- 172.21.15.31:0/2057996121 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3280055610 con 0x7f32880c7560 2023-10-06T05:21:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 -- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f326c067050 msgr2=0x7f326c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f326c067050 0x7f326c069500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f32840099b0 tx=0x7f3284017000 comp rx=0 tx=0).stop 2023-10-06T05:21:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 -- 172.21.15.31:0/2057996121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7560 msgr2=0x7f328816d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7560 0x7f328816d0d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3280009fa0 tx=0x7f3280009900 comp rx=0 tx=0).stop 2023-10-06T05:21:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 -- 172.21.15.31:0/2057996121 shutdown_connections 2023-10-06T05:21:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f326c067050 0x7f326c069500 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32880c7560 0x7f328816d0d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32880aef00 0x7f32880c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 --2- 172.21.15.31:0/2057996121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32880c7190 0x7f328814ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 -- 172.21.15.31:0/2057996121 >> 172.21.15.31:0/2057996121 conn(0x7f32880cd3d0 msgr2=0x7f32880cba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.113+0000 7f328fbff700 1 -- 172.21.15.31:0/2057996121 shutdown_connections 2023-10-06T05:21:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.114+0000 7f328fbff700 1 -- 172.21.15.31:0/2057996121 wait complete. 2023-10-06T05:21:57.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:21:57.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.337+0000 7f2f4b9ff700 1 Processor -- start 2023-10-06T05:21:57.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.337+0000 7f2f4b9ff700 1 -- start start 2023-10-06T05:21:57.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.338+0000 7f2f4b9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f44159930 0x7f2f440b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.338+0000 7f2f4b9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2f440cd580 0x7f2f440b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.338+0000 7f2f4b9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4415d2d0 con 0x7f2f44159560 2023-10-06T05:21:57.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.338+0000 7f2f4b9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4415d5b0 con 0x7f2f440cd580 2023-10-06T05:21:57.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.338+0000 7f2f4b9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4415d890 con 0x7f2f44159930 2023-10-06T05:21:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.339+0000 7f2f48f9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f44159930 0x7f2f440b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.339+0000 7f2f4979b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2f44159560 0x7f2f440cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:39478/0 (socket says 172.21.15.31:39478) 2023-10-06T05:21:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.339+0000 7f2f49f9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2f440cd580 0x7f2f440b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.339+0000 7f2f4979b700 1 -- 172.21.15.31:0/1630180403 learned_addr learned my addr 172.21.15.31:0/1630180403 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f3bfff700 1 -- 172.21.15.31:0/1630180403 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142730677 0 0) 0x7f2f4415d2d0 con 0x7f2f44159560 2023-10-06T05:21:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f3bfff700 1 -- 172.21.15.31:0/1630180403 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f30003570 con 0x7f2f44159560 2023-10-06T05:21:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f49f9c700 1 -- 172.21.15.31:0/1630180403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f44159930 msgr2=0x7f2f440b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f49f9c700 1 --2- 172.21.15.31:0/1630180403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f44159930 0x7f2f440b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f49f9c700 1 -- 172.21.15.31:0/1630180403 >> v1:172.21.15.62:6789/0 conn(0x7f2f44159560 legacy=0x7f2f440cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f49f9c700 1 -- 172.21.15.31:0/1630180403 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4415db70 con 0x7f2f440cd580 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f48f9a700 1 --2- 172.21.15.31:0/1630180403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f44159930 0x7f2f440b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f49f9c700 1 --2- 172.21.15.31:0/1630180403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f440cd580 0x7f2f440b29b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009e60 tx=0x7f2f3c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea84f04fd169e82c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f3bfff700 1 -- 172.21.15.31:0/1630180403 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00e070 con 0x7f2f440cd580 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f3bfff700 1 -- 172.21.15.31:0/1630180403 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c00bab0 con 0x7f2f440cd580 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.340+0000 7f2f3bfff700 1 -- 172.21.15.31:0/1630180403 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00be70 con 0x7f2f440cd580 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.341+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/1630180403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f440cd580 msgr2=0x7f2f440b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.341+0000 7f2f4b9ff700 1 --2- 172.21.15.31:0/1630180403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f440cd580 0x7f2f440b29b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009e60 tx=0x7f2f3c009780 comp rx=0 tx=0).stop 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.341+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/1630180403 shutdown_connections 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.341+0000 7f2f4b9ff700 1 --2- 172.21.15.31:0/1630180403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f440cd580 0x7f2f440b29b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.341+0000 7f2f4b9ff700 1 --2- 172.21.15.31:0/1630180403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f44159930 0x7f2f440b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.341+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/1630180403 >> 172.21.15.31:0/1630180403 conn(0x7f2f4406ec90 msgr2=0x7f2f4406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:57.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.341+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/1630180403 shutdown_connections 2023-10-06T05:21:57.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.341+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/1630180403 wait complete. 2023-10-06T05:21:57.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.342+0000 7f2f4b9ff700 1 Processor -- start 2023-10-06T05:21:57.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.343+0000 7f2f4b9ff700 1 -- start start 2023-10-06T05:21:57.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.343+0000 7f2f4b9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f440cd580 0x7f2f44155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f4b9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f44159560 0x7f2f440ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f4b9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f44159930 0x7f2f440cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f4b9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440c6850 con 0x7f2f44159560 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f4b9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440c7740 con 0x7f2f44159930 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f4b9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440c5c40 con 0x7f2f440cd580 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f48f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f44159560 0x7f2f440ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f48f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f44159560 0x7f2f440ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56306/0 (socket says 172.21.15.31:56306) 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f48f9a700 1 -- 172.21.15.31:0/763659920 learned_addr learned my addr 172.21.15.31:0/763659920 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f4979b700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f440cd580 0x7f2f44155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f49f9c700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f44159930 0x7f2f440cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f48f9a700 1 -- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f440cd580 msgr2=0x7f2f44155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f48f9a700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f440cd580 0x7f2f44155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f48f9a700 1 -- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f44159930 msgr2=0x7f2f440cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f48f9a700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f44159930 0x7f2f440cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:57.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.344+0000 7f2f48f9a700 1 -- 172.21.15.31:0/763659920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f34009420 con 0x7f2f44159560 2023-10-06T05:21:57.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.345+0000 7f2f48f9a700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f44159560 0x7f2f440ce4a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400ec80 tx=0x7f2f340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.345+0000 7f2f397fa700 1 -- 172.21.15.31:0/763659920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400c850 con 0x7f2f44159560 2023-10-06T05:21:57.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.345+0000 7f2f397fa700 1 -- 172.21.15.31:0/763659920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3400c9b0 con 0x7f2f44159560 2023-10-06T05:21:57.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.345+0000 7f2f397fa700 1 -- 172.21.15.31:0/763659920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400cd30 con 0x7f2f44159560 2023-10-06T05:21:57.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.345+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/763659920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c009a20 con 0x7f2f44159560 2023-10-06T05:21:57.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.345+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/763659920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f440c6200 con 0x7f2f44159560 2023-10-06T05:21:57.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.346+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/763659920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4404c380 con 0x7f2f44159560 2023-10-06T05:21:57.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.347+0000 7f2f397fa700 1 -- 172.21.15.31:0/763659920 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f2f34013680 con 0x7f2f44159560 2023-10-06T05:21:57.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.348+0000 7f2f397fa700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2f28067050 0x7f2f28069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:57.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.348+0000 7f2f397fa700 1 -- 172.21.15.31:0/763659920 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2f34084dc0 con 0x7f2f44159560 2023-10-06T05:21:57.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.348+0000 7f2f4979b700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2f28067050 0x7f2f28069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:57.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.349+0000 7f2f4979b700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2f28067050 0x7f2f28069500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2f40007fd0 tx=0x7f2f40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.352+0000 7f2f397fa700 1 -- 172.21.15.31:0/763659920 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f340557b0 con 0x7f2f44159560 2023-10-06T05:21:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:57.605+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/763659920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2f440c6990 con 0x7f2f44159560 2023-10-06T05:21:58.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.117+0000 7f2f397fa700 1 -- 172.21.15.31:0/763659920 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2f340170c0 con 0x7f2f44159560 2023-10-06T05:21:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.120+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2f28067050 msgr2=0x7f2f28069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.120+0000 7f2f4b9ff700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2f28067050 0x7f2f28069500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2f40007fd0 tx=0x7f2f40009040 comp rx=0 tx=0).stop 2023-10-06T05:21:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.120+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/763659920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f44159560 msgr2=0x7f2f440ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.120+0000 7f2f4b9ff700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f44159560 0x7f2f440ce4a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400ec80 tx=0x7f2f340091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.120+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/763659920 shutdown_connections 2023-10-06T05:21:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.120+0000 7f2f4b9ff700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2f28067050 0x7f2f28069500 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2f40007fd0 tx=0x7f2f40009040 comp rx=0 tx=0).stop 2023-10-06T05:21:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.120+0000 7f2f4b9ff700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f44159930 0x7f2f440cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.120+0000 7f2f4b9ff700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f440cd580 0x7f2f44155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.120+0000 7f2f4b9ff700 1 --2- 172.21.15.31:0/763659920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f44159560 0x7f2f440ce4a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.120+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/763659920 >> 172.21.15.31:0/763659920 conn(0x7f2f4406ec90 msgr2=0x7f2f440b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.121+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/763659920 shutdown_connections 2023-10-06T05:21:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.121+0000 7f2f4b9ff700 1 -- 172.21.15.31:0/763659920 wait complete. 2023-10-06T05:21:58.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:21:58.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.340+0000 7f94757ff700 1 Processor -- start 2023-10-06T05:21:58.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.341+0000 7f94757ff700 1 -- start start 2023-10-06T05:21:58.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.342+0000 7f94757ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a5b0 0x7f947006d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.342+0000 7f94757ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f947006d980 0x7f9470069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.342+0000 7f94757ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947006ddc0 con 0x7f947013a1e0 2023-10-06T05:21:58.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.342+0000 7f94757ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94700c71b0 con 0x7f947006d980 2023-10-06T05:21:58.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.342+0000 7f94757ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94700c7490 con 0x7f947013a5b0 2023-10-06T05:21:58.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.342+0000 7f946effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f947013a1e0 0x7f94700c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59548/0 (socket says 172.21.15.31:59548) 2023-10-06T05:21:58.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.342+0000 7f946effd700 1 -- 172.21.15.31:0/3494934804 learned_addr learned my addr 172.21.15.31:0/3494934804 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:58.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.342+0000 7f946f7fe700 1 --2- 172.21.15.31:0/3494934804 >> v2:172.21.15.31:3300/0 conn(0x7f947006d980 0x7f9470069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.342+0000 7f946e7fc700 1 --2- 172.21.15.31:0/3494934804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a5b0 0x7f947006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.343+0000 7f946dffb700 1 -- 172.21.15.31:0/3494934804 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544982291 0 0) 0x7f947006ddc0 con 0x7f947013a1e0 2023-10-06T05:21:58.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.343+0000 7f946f7fe700 1 -- 172.21.15.31:0/3494934804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a5b0 msgr2=0x7f947006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:58.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.343+0000 7f946f7fe700 1 --2- 172.21.15.31:0/3494934804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a5b0 0x7f947006d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.343+0000 7f946f7fe700 1 -- 172.21.15.31:0/3494934804 >> v1:172.21.15.31:6789/0 conn(0x7f947013a1e0 legacy=0x7f94700c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.343+0000 7f946f7fe700 1 -- 172.21.15.31:0/3494934804 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700c7770 con 0x7f947006d980 2023-10-06T05:21:58.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.343+0000 7f946f7fe700 1 --2- 172.21.15.31:0/3494934804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 0x7f9470069560 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9468009e60 tx=0x7f9468009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963e62a8da876582 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:58.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.343+0000 7f946dffb700 1 -- 172.21.15.31:0/3494934804 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946800e070 con 0x7f947006d980 2023-10-06T05:21:58.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.343+0000 7f946dffb700 1 -- 172.21.15.31:0/3494934804 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946800bc30 con 0x7f947006d980 2023-10-06T05:21:58.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.343+0000 7f946dffb700 1 -- 172.21.15.31:0/3494934804 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468012430 con 0x7f947006d980 2023-10-06T05:21:58.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.344+0000 7f94757ff700 1 -- 172.21.15.31:0/3494934804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 msgr2=0x7f9470069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.344+0000 7f94757ff700 1 --2- 172.21.15.31:0/3494934804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 0x7f9470069560 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9468009e60 tx=0x7f9468009780 comp rx=0 tx=0).stop 2023-10-06T05:21:58.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.344+0000 7f94757ff700 1 -- 172.21.15.31:0/3494934804 shutdown_connections 2023-10-06T05:21:58.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.344+0000 7f94757ff700 1 --2- 172.21.15.31:0/3494934804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 0x7f9470069560 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.344+0000 7f94757ff700 1 --2- 172.21.15.31:0/3494934804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a5b0 0x7f947006d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.344+0000 7f94757ff700 1 -- 172.21.15.31:0/3494934804 >> 172.21.15.31:0/3494934804 conn(0x7f94700cc0b0 msgr2=0x7f94700cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:58.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.344+0000 7f94757ff700 1 -- 172.21.15.31:0/3494934804 shutdown_connections 2023-10-06T05:21:58.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.344+0000 7f94757ff700 1 -- 172.21.15.31:0/3494934804 wait complete. 2023-10-06T05:21:58.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.346+0000 7f94757ff700 1 Processor -- start 2023-10-06T05:21:58.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.346+0000 7f94757ff700 1 -- start start 2023-10-06T05:21:58.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.347+0000 7f94757ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 0x7f94700c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.347+0000 7f94757ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a1e0 0x7f9470151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.347+0000 7f94757ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947013a5b0 0x7f94700c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.347+0000 7f94757ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947013b8b0 con 0x7f947006d980 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.347+0000 7f94757ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947013a9c0 con 0x7f947013a5b0 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f94757ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947013aca0 con 0x7f947013a1e0 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 0x7f94700c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 0x7f94700c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56322/0 (socket says 172.21.15.31:56322) 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946effd700 1 -- 172.21.15.31:0/773608083 learned_addr learned my addr 172.21.15.31:0/773608083 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946e7fc700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a1e0 0x7f9470151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946f7fe700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947013a5b0 0x7f94700c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946effd700 1 -- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a1e0 msgr2=0x7f9470151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946effd700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a1e0 0x7f9470151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946effd700 1 -- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947013a5b0 msgr2=0x7f94700c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946effd700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947013a5b0 0x7f94700c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946effd700 1 -- 172.21.15.31:0/773608083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9468009a20 con 0x7f947006d980 2023-10-06T05:21:58.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946e7fc700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a1e0 0x7f9470151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:58.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946f7fe700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947013a5b0 0x7f94700c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:58.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.348+0000 7f946effd700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 0x7f94700c5d40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f946000c7e0 tx=0x7f9460007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:58.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.349+0000 7f9456ffd700 1 -- 172.21.15.31:0/773608083 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460007300 con 0x7f947006d980 2023-10-06T05:21:58.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.349+0000 7f9456ffd700 1 -- 172.21.15.31:0/773608083 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9460007460 con 0x7f947006d980 2023-10-06T05:21:58.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.349+0000 7f94757ff700 1 -- 172.21.15.31:0/773608083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947013af80 con 0x7f947006d980 2023-10-06T05:21:58.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.349+0000 7f9456ffd700 1 -- 172.21.15.31:0/773608083 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600057f0 con 0x7f947006d980 2023-10-06T05:21:58.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.349+0000 7f94757ff700 1 -- 172.21.15.31:0/773608083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947013b540 con 0x7f947006d980 2023-10-06T05:21:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.350+0000 7f94757ff700 1 -- 172.21.15.31:0/773608083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9470158980 con 0x7f947006d980 2023-10-06T05:21:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.351+0000 7f9456ffd700 1 -- 172.21.15.31:0/773608083 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f9460005c00 con 0x7f947006d980 2023-10-06T05:21:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.352+0000 7f9456ffd700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f945c067080 0x7f945c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.352+0000 7f9456ffd700 1 -- 172.21.15.31:0/773608083 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9460084400 con 0x7f947006d980 2023-10-06T05:21:58.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.352+0000 7f946e7fc700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f945c067080 0x7f945c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.353+0000 7f946e7fc700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f945c067080 0x7f945c069530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f947015af70 tx=0x7f9464008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:58.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.356+0000 7f9456ffd700 1 -- 172.21.15.31:0/773608083 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94600513e0 con 0x7f947006d980 2023-10-06T05:21:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.603+0000 7f94757ff700 1 -- 172.21.15.31:0/773608083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f947004c380 con 0x7f947006d980 2023-10-06T05:21:58.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.603+0000 7f9456ffd700 1 -- 172.21.15.31:0/773608083 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f9460054a00 con 0x7f947006d980 2023-10-06T05:21:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:58.605 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1878570982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1878570982},{"type":"v1","addr":"172.21.15.62:6834","nonce":1878570982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6687,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/427382574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":427382574},{"type":"v1","addr":"172.21.15.31:6834","nonce":427382574}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4141055233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4141055233},{"type":"v1","addr":"172.21.15.31:6835","nonce":4141055233}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:54.590687+0000","modified":"2023-10-06T05:21:58.111036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"d","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3654179987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3654179987},{"type":"v1","addr":"172.21.15.62:6835","nonce":3654179987}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:21:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 -- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f945c067080 msgr2=0x7f945c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f945c067080 0x7f945c069530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f947015af70 tx=0x7f9464008040 comp rx=0 tx=0).stop 2023-10-06T05:21:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 -- 172.21.15.31:0/773608083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 msgr2=0x7f94700c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 0x7f94700c5d40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f946000c7e0 tx=0x7f9460007c80 comp rx=0 tx=0).stop 2023-10-06T05:21:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 -- 172.21.15.31:0/773608083 shutdown_connections 2023-10-06T05:21:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f945c067080 0x7f945c069530 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947013a5b0 0x7f94700c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947006d980 0x7f94700c5d40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 --2- 172.21.15.31:0/773608083 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f947013a1e0 0x7f9470151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 -- 172.21.15.31:0/773608083 >> 172.21.15.31:0/773608083 conn(0x7f94700cc0b0 msgr2=0x7f94700afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 -- 172.21.15.31:0/773608083 shutdown_connections 2023-10-06T05:21:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.606+0000 7f94757ff700 1 -- 172.21.15.31:0/773608083 wait complete. 2023-10-06T05:21:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 53 2023-10-06T05:21:58.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:21:58.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.818+0000 7f15e8bff700 1 Processor -- start 2023-10-06T05:21:58.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.818+0000 7f15e8bff700 1 -- start start 2023-10-06T05:21:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.820+0000 7f15e8bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15e41595a0 0x7f15e4159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.820+0000 7f15e8bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15e4068d80 0x7f15e40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.820+0000 7f15e8bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e40c95b0 con 0x7f15e40cc0d0 2023-10-06T05:21:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.820+0000 7f15e8bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e40c9890 con 0x7f15e4068d80 2023-10-06T05:21:58.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.820+0000 7f15e8bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40c9b70 con 0x7f15e41595a0 2023-10-06T05:21:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.820+0000 7f15e2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15e4068d80 0x7f15e40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.820+0000 7f15e2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15e4068d80 0x7f15e40c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56334/0 (socket says 172.21.15.31:56334) 2023-10-06T05:21:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.820+0000 7f15e2d9d700 1 -- 172.21.15.31:0/1186277364 learned_addr learned my addr 172.21.15.31:0/1186277364 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.820+0000 7f15e259c700 1 --2- 172.21.15.31:0/1186277364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15e41595a0 0x7f15e4159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.821+0000 7f15e2d9d700 1 -- 172.21.15.31:0/1186277364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15e41595a0 msgr2=0x7f15e4159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.821+0000 7f15e2d9d700 1 --2- 172.21.15.31:0/1186277364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15e41595a0 0x7f15e4159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.821+0000 7f15e2d9d700 1 -- 172.21.15.31:0/1186277364 >> v1:172.21.15.62:6789/0 conn(0x7f15e40cc0d0 legacy=0x7f15e4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.821+0000 7f15e2d9d700 1 -- 172.21.15.31:0/1186277364 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40ca130 con 0x7f15e4068d80 2023-10-06T05:21:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.821+0000 7f15e159a700 1 -- 172.21.15.31:0/1186277364 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295941841 0 0) 0x7f15e40c95b0 con 0x7f15e40cc0d0 2023-10-06T05:21:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.821+0000 7f15e2d9d700 1 --2- 172.21.15.31:0/1186277364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e4068d80 0x7f15e40c8930 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009f20 tx=0x7f15dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e602247395a21d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.821+0000 7f15e159a700 1 -- 172.21.15.31:0/1186277364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc00e070 con 0x7f15e4068d80 2023-10-06T05:21:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.821+0000 7f15e159a700 1 -- 172.21.15.31:0/1186277364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15dc00b9d0 con 0x7f15e4068d80 2023-10-06T05:21:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.821+0000 7f15e159a700 1 -- 172.21.15.31:0/1186277364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0092e0 con 0x7f15e4068d80 2023-10-06T05:21:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.822+0000 7f15e8bff700 1 -- 172.21.15.31:0/1186277364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e4068d80 msgr2=0x7f15e40c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.822+0000 7f15e8bff700 1 --2- 172.21.15.31:0/1186277364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e4068d80 0x7f15e40c8930 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009f20 tx=0x7f15dc009840 comp rx=0 tx=0).stop 2023-10-06T05:21:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.822+0000 7f15e8bff700 1 -- 172.21.15.31:0/1186277364 shutdown_connections 2023-10-06T05:21:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.822+0000 7f15e8bff700 1 --2- 172.21.15.31:0/1186277364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e4068d80 0x7f15e40c8930 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.822+0000 7f15e8bff700 1 --2- 172.21.15.31:0/1186277364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15e41595a0 0x7f15e4159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.822+0000 7f15e8bff700 1 -- 172.21.15.31:0/1186277364 >> 172.21.15.31:0/1186277364 conn(0x7f15e406b620 msgr2=0x7f15e406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.822+0000 7f15e8bff700 1 -- 172.21.15.31:0/1186277364 shutdown_connections 2023-10-06T05:21:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.822+0000 7f15e8bff700 1 -- 172.21.15.31:0/1186277364 wait complete. 2023-10-06T05:21:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.823+0000 7f15e8bff700 1 Processor -- start 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.823+0000 7f15e8bff700 1 -- start start 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.824+0000 7f15e8bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e4068d80 0x7f15e4156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.824+0000 7f15e8bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15e40cc0d0 0x7f15e41546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.824+0000 7f15e8bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e41595a0 0x7f15e4154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.824+0000 7f15e8bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40c6760 con 0x7f15e41595a0 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.824+0000 7f15e8bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e41552c0 con 0x7f15e4068d80 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.824+0000 7f15e8bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e41555a0 con 0x7f15e40cc0d0 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e41595a0 0x7f15e4154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e41595a0 0x7f15e4154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56340/0 (socket says 172.21.15.31:56340) 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e2d9d700 1 -- 172.21.15.31:0/2800882988 learned_addr learned my addr 172.21.15.31:0/2800882988 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e259c700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e4068d80 0x7f15e4156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e1d9b700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15e40cc0d0 0x7f15e41546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e2d9d700 1 -- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15e40cc0d0 msgr2=0x7f15e41546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e2d9d700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15e40cc0d0 0x7f15e41546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e2d9d700 1 -- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e4068d80 msgr2=0x7f15e4156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e2d9d700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e4068d80 0x7f15e4156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e2d9d700 1 -- 172.21.15.31:0/2800882988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc009ae0 con 0x7f15e41595a0 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.825+0000 7f15e2d9d700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e41595a0 0x7f15e4154bf0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f15dc012000 tx=0x7f15dc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.826+0000 7f15d27fc700 1 -- 172.21.15.31:0/2800882988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc008b10 con 0x7f15e41595a0 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.826+0000 7f15d27fc700 1 -- 172.21.15.31:0/2800882988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15dc008c90 con 0x7f15e41595a0 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.826+0000 7f15d27fc700 1 -- 172.21.15.31:0/2800882988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0256b0 con 0x7f15e41595a0 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.826+0000 7f15e8bff700 1 -- 172.21.15.31:0/2800882988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e4155880 con 0x7f15e41595a0 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.826+0000 7f15e8bff700 1 -- 172.21.15.31:0/2800882988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e4155e40 con 0x7f15e41595a0 2023-10-06T05:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.827+0000 7f15e8bff700 1 -- 172.21.15.31:0/2800882988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e404c380 con 0x7f15e41595a0 2023-10-06T05:21:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.828+0000 7f15d27fc700 1 -- 172.21.15.31:0/2800882988 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f15dc003040 con 0x7f15e41595a0 2023-10-06T05:21:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.829+0000 7f15d27fc700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f15c4067050 0x7f15c4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.829+0000 7f15d27fc700 1 -- 172.21.15.31:0/2800882988 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f15dc085e90 con 0x7f15e41595a0 2023-10-06T05:21:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.829+0000 7f15e259c700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f15c4067050 0x7f15c4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.830+0000 7f15e259c700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f15c4067050 0x7f15c4069500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009db0 tx=0x7f15d40091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:58.833+0000 7f15d27fc700 1 -- 172.21.15.31:0/2800882988 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15dc052d90 con 0x7f15e41595a0 2023-10-06T05:21:59.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.036+0000 7f15e8bff700 1 -- 172.21.15.31:0/2800882988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15e40ce4d0 con 0x7f15e41595a0 2023-10-06T05:21:59.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.037+0000 7f15d27fc700 1 -- 172.21.15.31:0/2800882988 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16917 (secure 0 0 0) 0x7f15dc0563b0 con 0x7f15e41595a0 2023-10-06T05:21:59.039 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:59.039 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:21:56.096339+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:52.267847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:53.579487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:21:59.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.039+0000 7f15e8bff700 1 -- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f15c4067050 msgr2=0x7f15c4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.039+0000 7f15e8bff700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f15c4067050 0x7f15c4069500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009db0 tx=0x7f15d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.040+0000 7f15e8bff700 1 -- 172.21.15.31:0/2800882988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e41595a0 msgr2=0x7f15e4154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.040+0000 7f15e8bff700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e41595a0 0x7f15e4154bf0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f15dc012000 tx=0x7f15dc018000 comp rx=0 tx=0).stop 2023-10-06T05:21:59.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.040+0000 7f15e8bff700 1 -- 172.21.15.31:0/2800882988 shutdown_connections 2023-10-06T05:21:59.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.040+0000 7f15e8bff700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f15c4067050 0x7f15c4069500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.040+0000 7f15e8bff700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15e41595a0 0x7f15e4154bf0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.040+0000 7f15e8bff700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15e4068d80 0x7f15e4156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.040+0000 7f15e8bff700 1 --2- 172.21.15.31:0/2800882988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15e40cc0d0 0x7f15e41546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.040+0000 7f15e8bff700 1 -- 172.21.15.31:0/2800882988 >> 172.21.15.31:0/2800882988 conn(0x7f15e406b620 msgr2=0x7f15e406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:59.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.040+0000 7f15e8bff700 1 -- 172.21.15.31:0/2800882988 shutdown_connections 2023-10-06T05:21:59.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.040+0000 7f15e8bff700 1 -- 172.21.15.31:0/2800882988 wait complete. 2023-10-06T05:21:59.059 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:21:59.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.254+0000 7f017adff700 1 Processor -- start 2023-10-06T05:21:59.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.254+0000 7f017adff700 1 -- start start 2023-10-06T05:21:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.255+0000 7f017adff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01740c7b60 0x7f0174069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.255+0000 7f017adff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01740c7f30 0x7f01740aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.255+0000 7f017adff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f017406ef50 0x7f017413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.255+0000 7f017adff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01740ca5b0 con 0x7f01740c7f30 2023-10-06T05:21:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.255+0000 7f017adff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f017406f740 con 0x7f017406ef50 2023-10-06T05:21:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.255+0000 7f017adff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01740c9c80 con 0x7f01740c7b60 2023-10-06T05:21:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.255+0000 7f0173fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01740c7f30 0x7f01740aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f0173fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01740c7f30 0x7f01740aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56352/0 (socket says 172.21.15.31:56352) 2023-10-06T05:21:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f0173fff700 1 -- 172.21.15.31:0/2116148466 learned_addr learned my addr 172.21.15.31:0/2116148466 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f0178b9b700 1 --2- 172.21.15.31:0/2116148466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01740c7b60 0x7f0174069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f017939c700 1 --2- 172.21.15.31:0/2116148466 >> v2:172.21.15.62:3300/0 conn(0x7f017406ef50 0x7f017413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f0173fff700 1 -- 172.21.15.31:0/2116148466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01740c7b60 msgr2=0x7f0174069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f0173fff700 1 --2- 172.21.15.31:0/2116148466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01740c7b60 0x7f0174069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f0173fff700 1 -- 172.21.15.31:0/2116148466 >> v2:172.21.15.62:3300/0 conn(0x7f017406ef50 msgr2=0x7f017413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:21:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f0173fff700 1 --2- 172.21.15.31:0/2116148466 >> v2:172.21.15.62:3300/0 conn(0x7f017406ef50 0x7f017413f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f0173fff700 1 -- 172.21.15.31:0/2116148466 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01740c9f60 con 0x7f01740c7f30 2023-10-06T05:21:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f0173fff700 1 --2- 172.21.15.31:0/2116148466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01740c7f30 0x7f01740aee70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f01740cbca0 tx=0x7f0160009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dcacf74c4cf8fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f01737fe700 1 -- 172.21.15.31:0/2116148466 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016000b820 con 0x7f01740c7f30 2023-10-06T05:21:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f01737fe700 1 -- 172.21.15.31:0/2116148466 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016000b980 con 0x7f01740c7f30 2023-10-06T05:21:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.256+0000 7f01737fe700 1 -- 172.21.15.31:0/2116148466 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016000bd20 con 0x7f01740c7f30 2023-10-06T05:21:59.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.257+0000 7f017adff700 1 -- 172.21.15.31:0/2116148466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01740c7f30 msgr2=0x7f01740aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.257+0000 7f017adff700 1 --2- 172.21.15.31:0/2116148466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01740c7f30 0x7f01740aee70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f01740cbca0 tx=0x7f0160009250 comp rx=0 tx=0).stop 2023-10-06T05:21:59.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.257+0000 7f017adff700 1 -- 172.21.15.31:0/2116148466 shutdown_connections 2023-10-06T05:21:59.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.257+0000 7f017adff700 1 --2- 172.21.15.31:0/2116148466 >> v2:172.21.15.62:3300/0 conn(0x7f017406ef50 0x7f017413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.257+0000 7f017adff700 1 --2- 172.21.15.31:0/2116148466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01740c7b60 0x7f0174069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.257+0000 7f017adff700 1 --2- 172.21.15.31:0/2116148466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01740c7f30 0x7f01740aee70 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.257+0000 7f017adff700 1 -- 172.21.15.31:0/2116148466 >> 172.21.15.31:0/2116148466 conn(0x7f01740cc0b0 msgr2=0x7f01740cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.257+0000 7f017adff700 1 -- 172.21.15.31:0/2116148466 shutdown_connections 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.257+0000 7f017adff700 1 -- 172.21.15.31:0/2116148466 wait complete. 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.259+0000 7f017adff700 1 Processor -- start 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.259+0000 7f017adff700 1 -- start start 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f017adff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017406ef50 0x7f017415add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f017adff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01740c7b60 0x7f017415df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f017adff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01740c7f30 0x7f017415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f017adff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01740b1660 con 0x7f017406ef50 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f017adff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0174169a50 con 0x7f01740c7f30 2023-10-06T05:21:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f017adff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017415b820 con 0x7f01740c7b60 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f0178b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017406ef50 0x7f017415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f0178b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017406ef50 0x7f017415add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56362/0 (socket says 172.21.15.31:56362) 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f0178b9b700 1 -- 172.21.15.31:0/4146411653 learned_addr learned my addr 172.21.15.31:0/4146411653 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f0173fff700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01740c7b60 0x7f017415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f017939c700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01740c7f30 0x7f017415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f0178b9b700 1 -- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01740c7b60 msgr2=0x7f017415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f0178b9b700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01740c7b60 0x7f017415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f0178b9b700 1 -- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01740c7f30 msgr2=0x7f017415e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f0178b9b700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01740c7f30 0x7f017415e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f0178b9b700 1 -- 172.21.15.31:0/4146411653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01600094f0 con 0x7f017406ef50 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.260+0000 7f017939c700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01740c7f30 0x7f017415e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:21:59.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.261+0000 7f0178b9b700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017406ef50 0x7f017415add0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f016800d810 tx=0x7f0168008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.261+0000 7f0170ff9700 1 -- 172.21.15.31:0/4146411653 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01680125f0 con 0x7f017406ef50 2023-10-06T05:21:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.261+0000 7f0170ff9700 1 -- 172.21.15.31:0/4146411653 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0168012750 con 0x7f017406ef50 2023-10-06T05:21:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.261+0000 7f017adff700 1 -- 172.21.15.31:0/4146411653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0174169d10 con 0x7f017406ef50 2023-10-06T05:21:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.261+0000 7f0170ff9700 1 -- 172.21.15.31:0/4146411653 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168012af0 con 0x7f017406ef50 2023-10-06T05:21:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.261+0000 7f017adff700 1 -- 172.21.15.31:0/4146411653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017416a2d0 con 0x7f017406ef50 2023-10-06T05:21:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.262+0000 7f017adff700 1 -- 172.21.15.31:0/4146411653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017416a640 con 0x7f017406ef50 2023-10-06T05:21:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.263+0000 7f0170ff9700 1 -- 172.21.15.31:0/4146411653 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f016801e460 con 0x7f017406ef50 2023-10-06T05:21:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.264+0000 7f0170ff9700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0158067050 0x7f0158069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.264+0000 7f0170ff9700 1 -- 172.21.15.31:0/4146411653 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0168084e90 con 0x7f017406ef50 2023-10-06T05:21:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.264+0000 7f0173fff700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0158067050 0x7f0158069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.265+0000 7f0173fff700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0158067050 0x7f0158069500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f01740cbca0 tx=0x7f0160017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:59.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.268+0000 7f0170ff9700 1 -- 172.21.15.31:0/4146411653 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0168055880 con 0x7f017406ef50 2023-10-06T05:21:59.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.465+0000 7f017adff700 1 -- 172.21.15.31:0/4146411653 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f017404c380 con 0x7f0158067050 2023-10-06T05:21:59.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.478+0000 7f0170ff9700 1 -- 172.21.15.31:0/4146411653 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311310 (secure 0 0 0) 0x7f017404c380 con 0x7f0158067050 2023-10-06T05:21:59.483 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 -- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0158067050 msgr2=0x7f0158069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0158067050 0x7f0158069500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f01740cbca0 tx=0x7f0160017000 comp rx=0 tx=0).stop 2023-10-06T05:21:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 -- 172.21.15.31:0/4146411653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017406ef50 msgr2=0x7f017415add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017406ef50 0x7f017415add0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f016800d810 tx=0x7f0168008040 comp rx=0 tx=0).stop 2023-10-06T05:21:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 -- 172.21.15.31:0/4146411653 shutdown_connections 2023-10-06T05:21:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0158067050 0x7f0158069500 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01740c7f30 0x7f017415e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017406ef50 0x7f017415add0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 --2- 172.21.15.31:0/4146411653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01740c7b60 0x7f017415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 -- 172.21.15.31:0/4146411653 >> 172.21.15.31:0/4146411653 conn(0x7f01740cc0b0 msgr2=0x7f01740ca940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 -- 172.21.15.31:0/4146411653 shutdown_connections 2023-10-06T05:21:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.482+0000 7f017adff700 1 -- 172.21.15.31:0/4146411653 wait complete. 2023-10-06T05:21:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:21:59.501 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T05:21:59.463623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2391508,"kb_used_data":9108,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752583212,"statfs":{"total":773094113280,"available":770645209088,"internally_reserved":0,"allocated":9326592,"data_stored":4842981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001787"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.128835+0000","last_change":"2023-10-06T05:21:55.125060+0000","last_active":"2023-10-06T05:21:55.128835+0000","last_peered":"2023-10-06T05:21:55.128835+0000","last_clean":"2023-10-06T05:21:55.128835+0000","last_became_active":"2023-10-06T05:21:55.124721+0000","last_became_peered":"2023-10-06T05:21:55.124721+0000","last_unstale":"2023-10-06T05:21:55.128835+0000","last_undegraded":"2023-10-06T05:21:55.128835+0000","last_fullsized":"2023-10-06T05:21:55.128835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:24.323427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.110904+0000","last_change":"2023-10-06T05:21:55.110904+0000","last_active":"2023-10-06T05:21:55.110904+0000","last_peered":"2023-10-06T05:21:55.110904+0000","last_clean":"2023-10-06T05:21:55.110904+0000","last_became_active":"2023-10-06T05:21:55.110634+0000","last_became_peered":"2023-10-06T05:21:55.110634+0000","last_unstale":"2023-10-06T05:21:55.110904+0000","last_undegraded":"2023-10-06T05:21:55.110904+0000","last_fullsized":"2023-10-06T05:21:55.110904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:39.207736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.120718+0000","last_change":"2023-10-06T05:21:55.112039+0000","last_active":"2023-10-06T05:21:55.120718+0000","last_peered":"2023-10-06T05:21:55.120718+0000","last_clean":"2023-10-06T05:21:55.120718+0000","last_became_active":"2023-10-06T05:21:55.111712+0000","last_became_peered":"2023-10-06T05:21:55.111712+0000","last_unstale":"2023-10-06T05:21:55.120718+0000","last_undegraded":"2023-10-06T05:21:55.120718+0000","last_fullsized":"2023-10-06T05:21:55.120718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:25.396272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117633+0000","last_change":"2023-10-06T05:21:55.113456+0000","last_active":"2023-10-06T05:21:55.117633+0000","last_peered":"2023-10-06T05:21:55.117633+0000","last_clean":"2023-10-06T05:21:55.117633+0000","last_became_active":"2023-10-06T05:21:55.112984+0000","last_became_peered":"2023-10-06T05:21:55.112984+0000","last_unstale":"2023-10-06T05:21:55.117633+0000","last_undegraded":"2023-10-06T05:21:55.117633+0000","last_fullsized":"2023-10-06T05:21:55.117633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:04.065404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.111553+0000","last_change":"2023-10-06T05:21:55.111553+0000","last_active":"2023-10-06T05:21:55.111553+0000","last_peered":"2023-10-06T05:21:55.111553+0000","last_clean":"2023-10-06T05:21:55.111553+0000","last_became_active":"2023-10-06T05:21:55.111278+0000","last_became_peered":"2023-10-06T05:21:55.111278+0000","last_unstale":"2023-10-06T05:21:55.111553+0000","last_undegraded":"2023-10-06T05:21:55.111553+0000","last_fullsized":"2023-10-06T05:21:55.111553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.623407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109277+0000","last_change":"2023-10-06T05:21:55.109277+0000","last_active":"2023-10-06T05:21:55.109277+0000","last_peered":"2023-10-06T05:21:55.109277+0000","last_clean":"2023-10-06T05:21:55.109277+0000","last_became_active":"2023-10-06T05:21:55.108625+0000","last_became_peered":"2023-10-06T05:21:55.108625+0000","last_unstale":"2023-10-06T05:21:55.109277+0000","last_undegraded":"2023-10-06T05:21:55.109277+0000","last_fullsized":"2023-10-06T05:21:55.109277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:12.449837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.128468+0000","last_change":"2023-10-06T05:21:55.124389+0000","last_active":"2023-10-06T05:21:55.128468+0000","last_peered":"2023-10-06T05:21:55.128468+0000","last_clean":"2023-10-06T05:21:55.128468+0000","last_became_active":"2023-10-06T05:21:55.124108+0000","last_became_peered":"2023-10-06T05:21:55.124108+0000","last_unstale":"2023-10-06T05:21:55.128468+0000","last_undegraded":"2023-10-06T05:21:55.128468+0000","last_fullsized":"2023-10-06T05:21:55.128468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:48.334561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.126279+0000","last_change":"2023-10-06T05:21:55.121070+0000","last_active":"2023-10-06T05:21:55.126279+0000","last_peered":"2023-10-06T05:21:55.126279+0000","last_clean":"2023-10-06T05:21:55.126279+0000","last_became_active":"2023-10-06T05:21:55.120668+0000","last_became_peered":"2023-10-06T05:21:55.120668+0000","last_unstale":"2023-10-06T05:21:55.126279+0000","last_undegraded":"2023-10-06T05:21:55.126279+0000","last_fullsized":"2023-10-06T05:21:55.126279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:44.248381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.108025+0000","last_change":"2023-10-06T05:21:55.108025+0000","last_active":"2023-10-06T05:21:55.108025+0000","last_peered":"2023-10-06T05:21:55.108025+0000","last_clean":"2023-10-06T05:21:55.108025+0000","last_became_active":"2023-10-06T05:21:55.107747+0000","last_became_peered":"2023-10-06T05:21:55.107747+0000","last_unstale":"2023-10-06T05:21:55.108025+0000","last_undegraded":"2023-10-06T05:21:55.108025+0000","last_fullsized":"2023-10-06T05:21:55.108025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:31.508522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.108653+0000","last_change":"2023-10-06T05:21:55.108653+0000","last_active":"2023-10-06T05:21:55.108653+0000","last_peered":"2023-10-06T05:21:55.108653+0000","last_clean":"2023-10-06T05:21:55.108653+0000","last_became_active":"2023-10-06T05:21:55.108400+0000","last_became_peered":"2023-10-06T05:21:55.108400+0000","last_unstale":"2023-10-06T05:21:55.108653+0000","last_undegraded":"2023-10-06T05:21:55.108653+0000","last_fullsized":"2023-10-06T05:21:55.108653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:57.737443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.125579+0000","last_change":"2023-10-06T05:21:55.125579+0000","last_active":"2023-10-06T05:21:55.125579+0000","last_peered":"2023-10-06T05:21:55.125579+0000","last_clean":"2023-10-06T05:21:55.125579+0000","last_became_active":"2023-10-06T05:21:55.125072+0000","last_became_peered":"2023-10-06T05:21:55.125072+0000","last_unstale":"2023-10-06T05:21:55.125579+0000","last_undegraded":"2023-10-06T05:21:55.125579+0000","last_fullsized":"2023-10-06T05:21:55.125579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:31.402430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.106136+0000","last_change":"2023-10-06T05:21:55.106136+0000","last_active":"2023-10-06T05:21:55.106136+0000","last_peered":"2023-10-06T05:21:55.106136+0000","last_clean":"2023-10-06T05:21:55.106136+0000","last_became_active":"2023-10-06T05:21:55.105864+0000","last_became_peered":"2023-10-06T05:21:55.105864+0000","last_unstale":"2023-10-06T05:21:55.106136+0000","last_undegraded":"2023-10-06T05:21:55.106136+0000","last_fullsized":"2023-10-06T05:21:55.106136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:21.556816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.126219+0000","last_change":"2023-10-06T05:21:55.120795+0000","last_active":"2023-10-06T05:21:55.126219+0000","last_peered":"2023-10-06T05:21:55.126219+0000","last_clean":"2023-10-06T05:21:55.126219+0000","last_became_active":"2023-10-06T05:21:55.120506+0000","last_became_peered":"2023-10-06T05:21:55.120506+0000","last_unstale":"2023-10-06T05:21:55.126219+0000","last_undegraded":"2023-10-06T05:21:55.126219+0000","last_fullsized":"2023-10-06T05:21:55.126219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:31.592670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.108986+0000","last_change":"2023-10-06T05:21:55.108986+0000","last_active":"2023-10-06T05:21:55.108986+0000","last_peered":"2023-10-06T05:21:55.108986+0000","last_clean":"2023-10-06T05:21:55.108986+0000","last_became_active":"2023-10-06T05:21:55.108683+0000","last_became_peered":"2023-10-06T05:21:55.108683+0000","last_unstale":"2023-10-06T05:21:55.108986+0000","last_undegraded":"2023-10-06T05:21:55.108986+0000","last_fullsized":"2023-10-06T05:21:55.108986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:39.502991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107796+0000","last_change":"2023-10-06T05:21:55.104581+0000","last_active":"2023-10-06T05:21:55.107796+0000","last_peered":"2023-10-06T05:21:55.107796+0000","last_clean":"2023-10-06T05:21:55.107796+0000","last_became_active":"2023-10-06T05:21:55.104226+0000","last_became_peered":"2023-10-06T05:21:55.104226+0000","last_unstale":"2023-10-06T05:21:55.107796+0000","last_undegraded":"2023-10-06T05:21:55.107796+0000","last_fullsized":"2023-10-06T05:21:55.107796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:32.401527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.114721+0000","last_change":"2023-10-06T05:21:55.114721+0000","last_active":"2023-10-06T05:21:55.114721+0000","last_peered":"2023-10-06T05:21:55.114721+0000","last_clean":"2023-10-06T05:21:55.114721+0000","last_became_active":"2023-10-06T05:21:55.114455+0000","last_became_peered":"2023-10-06T05:21:55.114455+0000","last_unstale":"2023-10-06T05:21:55.114721+0000","last_undegraded":"2023-10-06T05:21:55.114721+0000","last_fullsized":"2023-10-06T05:21:55.114721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:01.621113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.124179+0000","last_change":"2023-10-06T05:21:55.124179+0000","last_active":"2023-10-06T05:21:55.124179+0000","last_peered":"2023-10-06T05:21:55.124179+0000","last_clean":"2023-10-06T05:21:55.124179+0000","last_became_active":"2023-10-06T05:21:55.119479+0000","last_became_peered":"2023-10-06T05:21:55.119479+0000","last_unstale":"2023-10-06T05:21:55.124179+0000","last_undegraded":"2023-10-06T05:21:55.124179+0000","last_fullsized":"2023-10-06T05:21:55.124179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:49.772782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.113011+0000","last_change":"2023-10-06T05:21:55.104047+0000","last_active":"2023-10-06T05:21:55.113011+0000","last_peered":"2023-10-06T05:21:55.113011+0000","last_clean":"2023-10-06T05:21:55.113011+0000","last_became_active":"2023-10-06T05:21:55.103693+0000","last_became_peered":"2023-10-06T05:21:55.103693+0000","last_unstale":"2023-10-06T05:21:55.113011+0000","last_undegraded":"2023-10-06T05:21:55.113011+0000","last_fullsized":"2023-10-06T05:21:55.113011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:05.927815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109914+0000","last_change":"2023-10-06T05:21:55.109914+0000","last_active":"2023-10-06T05:21:55.109914+0000","last_peered":"2023-10-06T05:21:55.109914+0000","last_clean":"2023-10-06T05:21:55.109914+0000","last_became_active":"2023-10-06T05:21:55.109525+0000","last_became_peered":"2023-10-06T05:21:55.109525+0000","last_unstale":"2023-10-06T05:21:55.109914+0000","last_undegraded":"2023-10-06T05:21:55.109914+0000","last_fullsized":"2023-10-06T05:21:55.109914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:10.295171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107235+0000","last_change":"2023-10-06T05:21:55.097872+0000","last_active":"2023-10-06T05:21:55.107235+0000","last_peered":"2023-10-06T05:21:55.107235+0000","last_clean":"2023-10-06T05:21:55.107235+0000","last_became_active":"2023-10-06T05:21:55.096116+0000","last_became_peered":"2023-10-06T05:21:55.096116+0000","last_unstale":"2023-10-06T05:21:55.107235+0000","last_undegraded":"2023-10-06T05:21:55.107235+0000","last_fullsized":"2023-10-06T05:21:55.107235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:58.692209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117744+0000","last_change":"2023-10-06T05:21:55.117744+0000","last_active":"2023-10-06T05:21:55.117744+0000","last_peered":"2023-10-06T05:21:55.117744+0000","last_clean":"2023-10-06T05:21:55.117744+0000","last_became_active":"2023-10-06T05:21:55.117472+0000","last_became_peered":"2023-10-06T05:21:55.117472+0000","last_unstale":"2023-10-06T05:21:55.117744+0000","last_undegraded":"2023-10-06T05:21:55.117744+0000","last_fullsized":"2023-10-06T05:21:55.117744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:43.471419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109532+0000","last_change":"2023-10-06T05:21:55.109532+0000","last_active":"2023-10-06T05:21:55.109532+0000","last_peered":"2023-10-06T05:21:55.109532+0000","last_clean":"2023-10-06T05:21:55.109532+0000","last_became_active":"2023-10-06T05:21:55.109265+0000","last_became_peered":"2023-10-06T05:21:55.109265+0000","last_unstale":"2023-10-06T05:21:55.109532+0000","last_undegraded":"2023-10-06T05:21:55.109532+0000","last_fullsized":"2023-10-06T05:21:55.109532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:58.734374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.114632+0000","last_change":"2023-10-06T05:21:55.114632+0000","last_active":"2023-10-06T05:21:55.114632+0000","last_peered":"2023-10-06T05:21:55.114632+0000","last_clean":"2023-10-06T05:21:55.114632+0000","last_became_active":"2023-10-06T05:21:55.114334+0000","last_became_peered":"2023-10-06T05:21:55.114334+0000","last_unstale":"2023-10-06T05:21:55.114632+0000","last_undegraded":"2023-10-06T05:21:55.114632+0000","last_fullsized":"2023-10-06T05:21:55.114632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:06.293881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.121057+0000","last_change":"2023-10-06T05:21:55.119478+0000","last_active":"2023-10-06T05:21:55.121057+0000","last_peered":"2023-10-06T05:21:55.121057+0000","last_clean":"2023-10-06T05:21:55.121057+0000","last_became_active":"2023-10-06T05:21:55.119230+0000","last_became_peered":"2023-10-06T05:21:55.119230+0000","last_unstale":"2023-10-06T05:21:55.121057+0000","last_undegraded":"2023-10-06T05:21:55.121057+0000","last_fullsized":"2023-10-06T05:21:55.121057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:31.402430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.115458+0000","last_change":"2023-10-06T05:21:55.115458+0000","last_active":"2023-10-06T05:21:55.115458+0000","last_peered":"2023-10-06T05:21:55.115458+0000","last_clean":"2023-10-06T05:21:55.115458+0000","last_became_active":"2023-10-06T05:21:55.115147+0000","last_became_peered":"2023-10-06T05:21:55.115147+0000","last_unstale":"2023-10-06T05:21:55.115458+0000","last_undegraded":"2023-10-06T05:21:55.115458+0000","last_fullsized":"2023-10-06T05:21:55.115458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:59.909748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109318+0000","last_change":"2023-10-06T05:21:55.109318+0000","last_active":"2023-10-06T05:21:55.109318+0000","last_peered":"2023-10-06T05:21:55.109318+0000","last_clean":"2023-10-06T05:21:55.109318+0000","last_became_active":"2023-10-06T05:21:55.108924+0000","last_became_peered":"2023-10-06T05:21:55.108924+0000","last_unstale":"2023-10-06T05:21:55.109318+0000","last_undegraded":"2023-10-06T05:21:55.109318+0000","last_fullsized":"2023-10-06T05:21:55.109318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:55.685815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.112316+0000","last_change":"2023-10-06T05:21:55.112316+0000","last_active":"2023-10-06T05:21:55.112316+0000","last_peered":"2023-10-06T05:21:55.112316+0000","last_clean":"2023-10-06T05:21:55.112316+0000","last_became_active":"2023-10-06T05:21:55.112037+0000","last_became_peered":"2023-10-06T05:21:55.112037+0000","last_unstale":"2023-10-06T05:21:55.112316+0000","last_undegraded":"2023-10-06T05:21:55.112316+0000","last_fullsized":"2023-10-06T05:21:55.112316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:27:17.581662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.111516+0000","last_change":"2023-10-06T05:21:55.107108+0000","last_active":"2023-10-06T05:21:55.111516+0000","last_peered":"2023-10-06T05:21:55.111516+0000","last_clean":"2023-10-06T05:21:55.111516+0000","last_became_active":"2023-10-06T05:21:55.105082+0000","last_became_peered":"2023-10-06T05:21:55.105082+0000","last_unstale":"2023-10-06T05:21:55.111516+0000","last_undegraded":"2023-10-06T05:21:55.111516+0000","last_fullsized":"2023-10-06T05:21:55.111516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:12.731133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.129103+0000","last_change":"2023-10-06T05:21:55.126806+0000","last_active":"2023-10-06T05:21:55.129103+0000","last_peered":"2023-10-06T05:21:55.129103+0000","last_clean":"2023-10-06T05:21:55.129103+0000","last_became_active":"2023-10-06T05:21:55.126550+0000","last_became_peered":"2023-10-06T05:21:55.126550+0000","last_unstale":"2023-10-06T05:21:55.129103+0000","last_undegraded":"2023-10-06T05:21:55.129103+0000","last_fullsized":"2023-10-06T05:21:55.129103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:57.947210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107377+0000","last_change":"2023-10-06T05:21:55.107377+0000","last_active":"2023-10-06T05:21:55.107377+0000","last_peered":"2023-10-06T05:21:55.107377+0000","last_clean":"2023-10-06T05:21:55.107377+0000","last_became_active":"2023-10-06T05:21:55.107085+0000","last_became_peered":"2023-10-06T05:21:55.107085+0000","last_unstale":"2023-10-06T05:21:55.107377+0000","last_undegraded":"2023-10-06T05:21:55.107377+0000","last_fullsized":"2023-10-06T05:21:55.107377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:34.006992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.108435+0000","last_change":"2023-10-06T05:21:55.108435+0000","last_active":"2023-10-06T05:21:55.108435+0000","last_peered":"2023-10-06T05:21:55.108435+0000","last_clean":"2023-10-06T05:21:55.108435+0000","last_became_active":"2023-10-06T05:21:55.108053+0000","last_became_peered":"2023-10-06T05:21:55.108053+0000","last_unstale":"2023-10-06T05:21:55.108435+0000","last_undegraded":"2023-10-06T05:21:55.108435+0000","last_fullsized":"2023-10-06T05:21:55.108435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:39.502991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.115251+0000","last_change":"2023-10-06T05:21:55.115251+0000","last_active":"2023-10-06T05:21:55.115251+0000","last_peered":"2023-10-06T05:21:55.115251+0000","last_clean":"2023-10-06T05:21:55.115251+0000","last_became_active":"2023-10-06T05:21:55.114971+0000","last_became_peered":"2023-10-06T05:21:55.114971+0000","last_unstale":"2023-10-06T05:21:55.115251+0000","last_undegraded":"2023-10-06T05:21:55.115251+0000","last_fullsized":"2023-10-06T05:21:55.115251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:59.799333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117166+0000","last_change":"2023-10-06T05:21:55.117166+0000","last_active":"2023-10-06T05:21:55.117166+0000","last_peered":"2023-10-06T05:21:55.117166+0000","last_clean":"2023-10-06T05:21:55.117166+0000","last_became_active":"2023-10-06T05:21:55.116801+0000","last_became_peered":"2023-10-06T05:21:55.116801+0000","last_unstale":"2023-10-06T05:21:55.117166+0000","last_undegraded":"2023-10-06T05:21:55.117166+0000","last_fullsized":"2023-10-06T05:21:55.117166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:21.451511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"82'4","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.127949+0000","last_change":"2023-10-06T05:21:54.110171+0000","last_active":"2023-10-06T05:21:55.127949+0000","last_peered":"2023-10-06T05:21:55.127949+0000","last_clean":"2023-10-06T05:21:55.127949+0000","last_became_active":"2023-10-06T05:21:54.109861+0000","last_became_peered":"2023-10-06T05:21:54.109861+0000","last_unstale":"2023-10-06T05:21:55.127949+0000","last_undegraded":"2023-10-06T05:21:55.127949+0000","last_fullsized":"2023-10-06T05:21:55.127949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:20.006465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.124820+0000","last_change":"2023-10-06T05:21:55.124820+0000","last_active":"2023-10-06T05:21:55.124820+0000","last_peered":"2023-10-06T05:21:55.124820+0000","last_clean":"2023-10-06T05:21:55.124820+0000","last_became_active":"2023-10-06T05:21:55.124228+0000","last_became_peered":"2023-10-06T05:21:55.124228+0000","last_unstale":"2023-10-06T05:21:55.124820+0000","last_undegraded":"2023-10-06T05:21:55.124820+0000","last_fullsized":"2023-10-06T05:21:55.124820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:23.957678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"82'1","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.094363+0000","last_change":"2023-10-06T05:21:54.106329+0000","last_active":"2023-10-06T05:21:55.094363+0000","last_peered":"2023-10-06T05:21:55.094363+0000","last_clean":"2023-10-06T05:21:55.094363+0000","last_became_active":"2023-10-06T05:21:54.105964+0000","last_became_peered":"2023-10-06T05:21:54.105964+0000","last_unstale":"2023-10-06T05:21:55.094363+0000","last_undegraded":"2023-10-06T05:21:55.094363+0000","last_fullsized":"2023-10-06T05:21:55.094363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:51.047874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.129050+0000","last_change":"2023-10-06T05:21:55.126284+0000","last_active":"2023-10-06T05:21:55.129050+0000","last_peered":"2023-10-06T05:21:55.129050+0000","last_clean":"2023-10-06T05:21:55.129050+0000","last_became_active":"2023-10-06T05:21:55.125958+0000","last_became_peered":"2023-10-06T05:21:55.125958+0000","last_unstale":"2023-10-06T05:21:55.129050+0000","last_undegraded":"2023-10-06T05:21:55.129050+0000","last_fullsized":"2023-10-06T05:21:55.129050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:44.318530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.126892+0000","last_change":"2023-10-06T05:21:54.108473+0000","last_active":"2023-10-06T05:21:55.126892+0000","last_peered":"2023-10-06T05:21:55.126892+0000","last_clean":"2023-10-06T05:21:55.126892+0000","last_became_active":"2023-10-06T05:21:54.108221+0000","last_became_peered":"2023-10-06T05:21:54.108221+0000","last_unstale":"2023-10-06T05:21:55.126892+0000","last_undegraded":"2023-10-06T05:21:55.126892+0000","last_fullsized":"2023-10-06T05:21:55.126892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:24.393277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.124167+0000","last_change":"2023-10-06T05:21:55.124167+0000","last_active":"2023-10-06T05:21:55.124167+0000","last_peered":"2023-10-06T05:21:55.124167+0000","last_clean":"2023-10-06T05:21:55.124167+0000","last_became_active":"2023-10-06T05:21:55.119767+0000","last_became_peered":"2023-10-06T05:21:55.119767+0000","last_unstale":"2023-10-06T05:21:55.124167+0000","last_undegraded":"2023-10-06T05:21:55.124167+0000","last_fullsized":"2023-10-06T05:21:55.124167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:37.907262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"82'2","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.119283+0000","last_change":"2023-10-06T05:21:54.103971+0000","last_active":"2023-10-06T05:21:55.119283+0000","last_peered":"2023-10-06T05:21:55.119283+0000","last_clean":"2023-10-06T05:21:55.119283+0000","last_became_active":"2023-10-06T05:21:54.103703+0000","last_became_peered":"2023-10-06T05:21:54.103703+0000","last_unstale":"2023-10-06T05:21:55.119283+0000","last_undegraded":"2023-10-06T05:21:55.119283+0000","last_fullsized":"2023-10-06T05:21:55.119283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:02.133614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107858+0000","last_change":"2023-10-06T05:21:55.105294+0000","last_active":"2023-10-06T05:21:55.107858+0000","last_peered":"2023-10-06T05:21:55.107858+0000","last_clean":"2023-10-06T05:21:55.107858+0000","last_became_active":"2023-10-06T05:21:55.104082+0000","last_became_peered":"2023-10-06T05:21:55.104082+0000","last_unstale":"2023-10-06T05:21:55.107858+0000","last_undegraded":"2023-10-06T05:21:55.107858+0000","last_fullsized":"2023-10-06T05:21:55.107858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:19.385103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"82'1","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.096355+0000","last_change":"2023-10-06T05:21:54.106503+0000","last_active":"2023-10-06T05:21:55.096355+0000","last_peered":"2023-10-06T05:21:55.096355+0000","last_clean":"2023-10-06T05:21:55.096355+0000","last_became_active":"2023-10-06T05:21:54.106199+0000","last_became_peered":"2023-10-06T05:21:54.106199+0000","last_unstale":"2023-10-06T05:21:55.096355+0000","last_undegraded":"2023-10-06T05:21:55.096355+0000","last_fullsized":"2023-10-06T05:21:55.096355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:27.601855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.125243+0000","last_change":"2023-10-06T05:21:55.125243+0000","last_active":"2023-10-06T05:21:55.125243+0000","last_peered":"2023-10-06T05:21:55.125243+0000","last_clean":"2023-10-06T05:21:55.125243+0000","last_became_active":"2023-10-06T05:21:55.124906+0000","last_became_peered":"2023-10-06T05:21:55.124906+0000","last_unstale":"2023-10-06T05:21:55.125243+0000","last_undegraded":"2023-10-06T05:21:55.125243+0000","last_fullsized":"2023-10-06T05:21:55.125243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:29.796079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.121321+0000","last_change":"2023-10-06T05:21:54.105162+0000","last_active":"2023-10-06T05:21:55.121321+0000","last_peered":"2023-10-06T05:21:55.121321+0000","last_clean":"2023-10-06T05:21:55.121321+0000","last_became_active":"2023-10-06T05:21:54.104813+0000","last_became_peered":"2023-10-06T05:21:54.104813+0000","last_unstale":"2023-10-06T05:21:55.121321+0000","last_undegraded":"2023-10-06T05:21:55.121321+0000","last_fullsized":"2023-10-06T05:21:55.121321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:37.813534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.112316+0000","last_change":"2023-10-06T05:21:55.112316+0000","last_active":"2023-10-06T05:21:55.112316+0000","last_peered":"2023-10-06T05:21:55.112316+0000","last_clean":"2023-10-06T05:21:55.112316+0000","last_became_active":"2023-10-06T05:21:55.109005+0000","last_became_peered":"2023-10-06T05:21:55.109005+0000","last_unstale":"2023-10-06T05:21:55.112316+0000","last_undegraded":"2023-10-06T05:21:55.112316+0000","last_fullsized":"2023-10-06T05:21:55.112316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:26.304630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.094581+0000","last_change":"2023-10-06T05:21:54.103935+0000","last_active":"2023-10-06T05:21:55.094581+0000","last_peered":"2023-10-06T05:21:55.094581+0000","last_clean":"2023-10-06T05:21:55.094581+0000","last_became_active":"2023-10-06T05:21:54.103634+0000","last_became_peered":"2023-10-06T05:21:54.103634+0000","last_unstale":"2023-10-06T05:21:55.094581+0000","last_undegraded":"2023-10-06T05:21:55.094581+0000","last_fullsized":"2023-10-06T05:21:55.094581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:57.731380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117513+0000","last_change":"2023-10-06T05:21:55.112906+0000","last_active":"2023-10-06T05:21:55.117513+0000","last_peered":"2023-10-06T05:21:55.117513+0000","last_clean":"2023-10-06T05:21:55.117513+0000","last_became_active":"2023-10-06T05:21:55.112632+0000","last_became_peered":"2023-10-06T05:21:55.112632+0000","last_unstale":"2023-10-06T05:21:55.117513+0000","last_undegraded":"2023-10-06T05:21:55.117513+0000","last_fullsized":"2023-10-06T05:21:55.117513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:52.372864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117562+0000","last_change":"2023-10-06T05:21:54.112755+0000","last_active":"2023-10-06T05:21:55.117562+0000","last_peered":"2023-10-06T05:21:55.117562+0000","last_clean":"2023-10-06T05:21:55.117562+0000","last_became_active":"2023-10-06T05:21:54.112226+0000","last_became_peered":"2023-10-06T05:21:54.112226+0000","last_unstale":"2023-10-06T05:21:55.117562+0000","last_undegraded":"2023-10-06T05:21:55.117562+0000","last_fullsized":"2023-10-06T05:21:55.117562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:49.698916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107526+0000","last_change":"2023-10-06T05:21:55.107526+0000","last_active":"2023-10-06T05:21:55.107526+0000","last_peered":"2023-10-06T05:21:55.107526+0000","last_clean":"2023-10-06T05:21:55.107526+0000","last_became_active":"2023-10-06T05:21:55.107051+0000","last_became_peered":"2023-10-06T05:21:55.107051+0000","last_unstale":"2023-10-06T05:21:55.107526+0000","last_undegraded":"2023-10-06T05:21:55.107526+0000","last_fullsized":"2023-10-06T05:21:55.107526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:59.909748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.111377+0000","last_change":"2023-10-06T05:21:54.108181+0000","last_active":"2023-10-06T05:21:54.111377+0000","last_peered":"2023-10-06T05:21:54.111377+0000","last_clean":"2023-10-06T05:21:54.111377+0000","last_became_active":"2023-10-06T05:21:54.107890+0000","last_became_peered":"2023-10-06T05:21:54.107890+0000","last_unstale":"2023-10-06T05:21:54.111377+0000","last_undegraded":"2023-10-06T05:21:54.111377+0000","last_fullsized":"2023-10-06T05:21:54.111377+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:03.566492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.128415+0000","last_change":"2023-10-06T05:21:55.124073+0000","last_active":"2023-10-06T05:21:55.128415+0000","last_peered":"2023-10-06T05:21:55.128415+0000","last_clean":"2023-10-06T05:21:55.128415+0000","last_became_active":"2023-10-06T05:21:55.123701+0000","last_became_peered":"2023-10-06T05:21:55.123701+0000","last_unstale":"2023-10-06T05:21:55.128415+0000","last_undegraded":"2023-10-06T05:21:55.128415+0000","last_fullsized":"2023-10-06T05:21:55.128415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:15.861250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.094761+0000","last_change":"2023-10-06T05:21:54.102840+0000","last_active":"2023-10-06T05:21:55.094761+0000","last_peered":"2023-10-06T05:21:55.094761+0000","last_clean":"2023-10-06T05:21:55.094761+0000","last_became_active":"2023-10-06T05:21:54.102563+0000","last_became_peered":"2023-10-06T05:21:54.102563+0000","last_unstale":"2023-10-06T05:21:55.094761+0000","last_undegraded":"2023-10-06T05:21:55.094761+0000","last_fullsized":"2023-10-06T05:21:55.094761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:58.906754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.120868+0000","last_change":"2023-10-06T05:21:55.118589+0000","last_active":"2023-10-06T05:21:55.120868+0000","last_peered":"2023-10-06T05:21:55.120868+0000","last_clean":"2023-10-06T05:21:55.120868+0000","last_became_active":"2023-10-06T05:21:55.118312+0000","last_became_peered":"2023-10-06T05:21:55.118312+0000","last_unstale":"2023-10-06T05:21:55.120868+0000","last_undegraded":"2023-10-06T05:21:55.120868+0000","last_fullsized":"2023-10-06T05:21:55.120868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:50.701910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"82'3","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.124737+0000","last_change":"2023-10-06T05:21:54.112396+0000","last_active":"2023-10-06T05:21:55.124737+0000","last_peered":"2023-10-06T05:21:55.124737+0000","last_clean":"2023-10-06T05:21:55.124737+0000","last_became_active":"2023-10-06T05:21:54.112063+0000","last_became_peered":"2023-10-06T05:21:54.112063+0000","last_unstale":"2023-10-06T05:21:55.124737+0000","last_undegraded":"2023-10-06T05:21:55.124737+0000","last_fullsized":"2023-10-06T05:21:55.124737+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:49.825750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.116513+0000","last_change":"2023-10-06T05:21:55.107434+0000","last_active":"2023-10-06T05:21:55.116513+0000","last_peered":"2023-10-06T05:21:55.116513+0000","last_clean":"2023-10-06T05:21:55.116513+0000","last_became_active":"2023-10-06T05:21:55.107087+0000","last_became_peered":"2023-10-06T05:21:55.107087+0000","last_unstale":"2023-10-06T05:21:55.116513+0000","last_undegraded":"2023-10-06T05:21:55.116513+0000","last_fullsized":"2023-10-06T05:21:55.116513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:00.508009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"82'6","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.124656+0000","last_change":"2023-10-06T05:21:54.111052+0000","last_active":"2023-10-06T05:21:55.124656+0000","last_peered":"2023-10-06T05:21:55.124656+0000","last_clean":"2023-10-06T05:21:55.124656+0000","last_became_active":"2023-10-06T05:21:54.110744+0000","last_became_peered":"2023-10-06T05:21:54.110744+0000","last_unstale":"2023-10-06T05:21:55.124656+0000","last_undegraded":"2023-10-06T05:21:55.124656+0000","last_fullsized":"2023-10-06T05:21:55.124656+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:49.698916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.126260+0000","last_change":"2023-10-06T05:21:55.121058+0000","last_active":"2023-10-06T05:21:55.126260+0000","last_peered":"2023-10-06T05:21:55.126260+0000","last_clean":"2023-10-06T05:21:55.126260+0000","last_became_active":"2023-10-06T05:21:55.120247+0000","last_became_peered":"2023-10-06T05:21:55.120247+0000","last_unstale":"2023-10-06T05:21:55.126260+0000","last_undegraded":"2023-10-06T05:21:55.126260+0000","last_fullsized":"2023-10-06T05:21:55.126260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:11.288762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.127469+0000","last_change":"2023-10-06T05:21:54.112529+0000","last_active":"2023-10-06T05:21:55.127469+0000","last_peered":"2023-10-06T05:21:55.127469+0000","last_clean":"2023-10-06T05:21:55.127469+0000","last_became_active":"2023-10-06T05:21:54.112200+0000","last_became_peered":"2023-10-06T05:21:54.112200+0000","last_unstale":"2023-10-06T05:21:55.127469+0000","last_undegraded":"2023-10-06T05:21:55.127469+0000","last_fullsized":"2023-10-06T05:21:55.127469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:04.924821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.111916+0000","last_change":"2023-10-06T05:21:55.111916+0000","last_active":"2023-10-06T05:21:55.111916+0000","last_peered":"2023-10-06T05:21:55.111916+0000","last_clean":"2023-10-06T05:21:55.111916+0000","last_became_active":"2023-10-06T05:21:55.111635+0000","last_became_peered":"2023-10-06T05:21:55.111635+0000","last_unstale":"2023-10-06T05:21:55.111916+0000","last_undegraded":"2023-10-06T05:21:55.111916+0000","last_fullsized":"2023-10-06T05:21:55.111916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:02.549324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.112700+0000","last_change":"2023-10-06T05:21:54.111339+0000","last_active":"2023-10-06T05:21:54.112700+0000","last_peered":"2023-10-06T05:21:54.112700+0000","last_clean":"2023-10-06T05:21:54.112700+0000","last_became_active":"2023-10-06T05:21:54.111050+0000","last_became_peered":"2023-10-06T05:21:54.111050+0000","last_unstale":"2023-10-06T05:21:54.112700+0000","last_undegraded":"2023-10-06T05:21:54.112700+0000","last_fullsized":"2023-10-06T05:21:54.112700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:04.924821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109186+0000","last_change":"2023-10-06T05:21:55.109186+0000","last_active":"2023-10-06T05:21:55.109186+0000","last_peered":"2023-10-06T05:21:55.109186+0000","last_clean":"2023-10-06T05:21:55.109186+0000","last_became_active":"2023-10-06T05:21:55.108920+0000","last_became_peered":"2023-10-06T05:21:55.108920+0000","last_unstale":"2023-10-06T05:21:55.109186+0000","last_undegraded":"2023-10-06T05:21:55.109186+0000","last_fullsized":"2023-10-06T05:21:55.109186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:58.909995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.090009+0000","last_change":"2023-10-06T05:21:54.105966+0000","last_active":"2023-10-06T05:21:55.090009+0000","last_peered":"2023-10-06T05:21:55.090009+0000","last_clean":"2023-10-06T05:21:55.090009+0000","last_became_active":"2023-10-06T05:21:54.105674+0000","last_became_peered":"2023-10-06T05:21:54.105674+0000","last_unstale":"2023-10-06T05:21:55.090009+0000","last_undegraded":"2023-10-06T05:21:55.090009+0000","last_fullsized":"2023-10-06T05:21:55.090009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:45.082000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.118164+0000","last_change":"2023-10-06T05:21:55.118164+0000","last_active":"2023-10-06T05:21:55.118164+0000","last_peered":"2023-10-06T05:21:55.118164+0000","last_clean":"2023-10-06T05:21:55.118164+0000","last_became_active":"2023-10-06T05:21:55.117845+0000","last_became_peered":"2023-10-06T05:21:55.117845+0000","last_unstale":"2023-10-06T05:21:55.118164+0000","last_undegraded":"2023-10-06T05:21:55.118164+0000","last_fullsized":"2023-10-06T05:21:55.118164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:27.188799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"82'8","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.122777+0000","last_change":"2023-10-06T05:21:54.100274+0000","last_active":"2023-10-06T05:21:55.122777+0000","last_peered":"2023-10-06T05:21:55.122777+0000","last_clean":"2023-10-06T05:21:55.122777+0000","last_became_active":"2023-10-06T05:21:54.099993+0000","last_became_peered":"2023-10-06T05:21:54.099993+0000","last_unstale":"2023-10-06T05:21:55.122777+0000","last_undegraded":"2023-10-06T05:21:55.122777+0000","last_fullsized":"2023-10-06T05:21:55.122777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:21:03.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.119102+0000","last_change":"2023-10-06T05:21:55.119102+0000","last_active":"2023-10-06T05:21:55.119102+0000","last_peered":"2023-10-06T05:21:55.119102+0000","last_clean":"2023-10-06T05:21:55.119102+0000","last_became_active":"2023-10-06T05:21:55.115908+0000","last_became_peered":"2023-10-06T05:21:55.115908+0000","last_unstale":"2023-10-06T05:21:55.119102+0000","last_undegraded":"2023-10-06T05:21:55.119102+0000","last_fullsized":"2023-10-06T05:21:55.119102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:22.621796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.123844+0000","last_change":"2023-10-06T05:21:54.110852+0000","last_active":"2023-10-06T05:21:55.123844+0000","last_peered":"2023-10-06T05:21:55.123844+0000","last_clean":"2023-10-06T05:21:55.123844+0000","last_became_active":"2023-10-06T05:21:54.110477+0000","last_became_peered":"2023-10-06T05:21:54.110477+0000","last_unstale":"2023-10-06T05:21:55.123844+0000","last_undegraded":"2023-10-06T05:21:55.123844+0000","last_fullsized":"2023-10-06T05:21:55.123844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:02.896225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.125003+0000","last_change":"2023-10-06T05:21:55.125003+0000","last_active":"2023-10-06T05:21:55.125003+0000","last_peered":"2023-10-06T05:21:55.125003+0000","last_clean":"2023-10-06T05:21:55.125003+0000","last_became_active":"2023-10-06T05:21:55.124575+0000","last_became_peered":"2023-10-06T05:21:55.124575+0000","last_unstale":"2023-10-06T05:21:55.125003+0000","last_undegraded":"2023-10-06T05:21:55.125003+0000","last_fullsized":"2023-10-06T05:21:55.125003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:10.843465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.108334+0000","last_change":"2023-10-06T05:21:54.104130+0000","last_active":"2023-10-06T05:21:54.108334+0000","last_peered":"2023-10-06T05:21:54.108334+0000","last_clean":"2023-10-06T05:21:54.108334+0000","last_became_active":"2023-10-06T05:21:54.103814+0000","last_became_peered":"2023-10-06T05:21:54.103814+0000","last_unstale":"2023-10-06T05:21:54.108334+0000","last_undegraded":"2023-10-06T05:21:54.108334+0000","last_fullsized":"2023-10-06T05:21:54.108334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:18.101312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117668+0000","last_change":"2023-10-06T05:21:55.117668+0000","last_active":"2023-10-06T05:21:55.117668+0000","last_peered":"2023-10-06T05:21:55.117668+0000","last_clean":"2023-10-06T05:21:55.117668+0000","last_became_active":"2023-10-06T05:21:55.117387+0000","last_became_peered":"2023-10-06T05:21:55.117387+0000","last_unstale":"2023-10-06T05:21:55.117668+0000","last_undegraded":"2023-10-06T05:21:55.117668+0000","last_fullsized":"2023-10-06T05:21:55.117668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:21.413755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.110448+0000","last_change":"2023-10-06T05:21:54.107990+0000","last_active":"2023-10-06T05:21:54.110448+0000","last_peered":"2023-10-06T05:21:54.110448+0000","last_clean":"2023-10-06T05:21:54.110448+0000","last_became_active":"2023-10-06T05:21:54.107583+0000","last_became_peered":"2023-10-06T05:21:54.107583+0000","last_unstale":"2023-10-06T05:21:54.110448+0000","last_undegraded":"2023-10-06T05:21:54.110448+0000","last_fullsized":"2023-10-06T05:21:54.110448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:18.382108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.110138+0000","last_change":"2023-10-06T05:21:55.110138+0000","last_active":"2023-10-06T05:21:55.110138+0000","last_peered":"2023-10-06T05:21:55.110138+0000","last_clean":"2023-10-06T05:21:55.110138+0000","last_became_active":"2023-10-06T05:21:55.109834+0000","last_became_peered":"2023-10-06T05:21:55.109834+0000","last_unstale":"2023-10-06T05:21:55.110138+0000","last_undegraded":"2023-10-06T05:21:55.110138+0000","last_fullsized":"2023-10-06T05:21:55.110138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:22.541524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.109701+0000","last_change":"2023-10-06T05:21:54.104572+0000","last_active":"2023-10-06T05:21:54.109701+0000","last_peered":"2023-10-06T05:21:54.109701+0000","last_clean":"2023-10-06T05:21:54.109701+0000","last_became_active":"2023-10-06T05:21:54.104255+0000","last_became_peered":"2023-10-06T05:21:54.104255+0000","last_unstale":"2023-10-06T05:21:54.109701+0000","last_undegraded":"2023-10-06T05:21:54.109701+0000","last_fullsized":"2023-10-06T05:21:54.109701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:21.559202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.108087+0000","last_change":"2023-10-06T05:21:55.108087+0000","last_active":"2023-10-06T05:21:55.108087+0000","last_peered":"2023-10-06T05:21:55.108087+0000","last_clean":"2023-10-06T05:21:55.108087+0000","last_became_active":"2023-10-06T05:21:55.107783+0000","last_became_peered":"2023-10-06T05:21:55.107783+0000","last_unstale":"2023-10-06T05:21:55.108087+0000","last_undegraded":"2023-10-06T05:21:55.108087+0000","last_fullsized":"2023-10-06T05:21:55.108087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:50.561974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.112878+0000","last_change":"2023-10-06T05:21:54.111755+0000","last_active":"2023-10-06T05:21:54.112878+0000","last_peered":"2023-10-06T05:21:54.112878+0000","last_clean":"2023-10-06T05:21:54.112878+0000","last_became_active":"2023-10-06T05:21:54.111518+0000","last_became_peered":"2023-10-06T05:21:54.111518+0000","last_unstale":"2023-10-06T05:21:54.112878+0000","last_undegraded":"2023-10-06T05:21:54.112878+0000","last_fullsized":"2023-10-06T05:21:54.112878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:26.571172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.123535+0000","last_change":"2023-10-06T05:21:55.109079+0000","last_active":"2023-10-06T05:21:55.123535+0000","last_peered":"2023-10-06T05:21:55.123535+0000","last_clean":"2023-10-06T05:21:55.123535+0000","last_became_active":"2023-10-06T05:21:55.108562+0000","last_became_peered":"2023-10-06T05:21:55.108562+0000","last_unstale":"2023-10-06T05:21:55.123535+0000","last_undegraded":"2023-10-06T05:21:55.123535+0000","last_fullsized":"2023-10-06T05:21:55.123535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:27.574167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.110347+0000","last_change":"2023-10-06T05:21:54.105835+0000","last_active":"2023-10-06T05:21:54.110347+0000","last_peered":"2023-10-06T05:21:54.110347+0000","last_clean":"2023-10-06T05:21:54.110347+0000","last_became_active":"2023-10-06T05:21:54.105572+0000","last_became_peered":"2023-10-06T05:21:54.105572+0000","last_unstale":"2023-10-06T05:21:54.110347+0000","last_undegraded":"2023-10-06T05:21:54.110347+0000","last_fullsized":"2023-10-06T05:21:54.110347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:51.369870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.113956+0000","last_change":"2023-10-06T05:21:55.113956+0000","last_active":"2023-10-06T05:21:55.113956+0000","last_peered":"2023-10-06T05:21:55.113956+0000","last_clean":"2023-10-06T05:21:55.113956+0000","last_became_active":"2023-10-06T05:21:55.113590+0000","last_became_peered":"2023-10-06T05:21:55.113590+0000","last_unstale":"2023-10-06T05:21:55.113956+0000","last_undegraded":"2023-10-06T05:21:55.113956+0000","last_fullsized":"2023-10-06T05:21:55.113956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:19.194349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.109821+0000","last_change":"2023-10-06T05:21:54.104657+0000","last_active":"2023-10-06T05:21:54.109821+0000","last_peered":"2023-10-06T05:21:54.109821+0000","last_clean":"2023-10-06T05:21:54.109821+0000","last_became_active":"2023-10-06T05:21:54.104347+0000","last_became_peered":"2023-10-06T05:21:54.104347+0000","last_unstale":"2023-10-06T05:21:54.109821+0000","last_undegraded":"2023-10-06T05:21:54.109821+0000","last_fullsized":"2023-10-06T05:21:54.109821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:20.176009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.113774+0000","last_change":"2023-10-06T05:21:55.113774+0000","last_active":"2023-10-06T05:21:55.113774+0000","last_peered":"2023-10-06T05:21:55.113774+0000","last_clean":"2023-10-06T05:21:55.113774+0000","last_became_active":"2023-10-06T05:21:55.113408+0000","last_became_peered":"2023-10-06T05:21:55.113408+0000","last_unstale":"2023-10-06T05:21:55.113774+0000","last_undegraded":"2023-10-06T05:21:55.113774+0000","last_fullsized":"2023-10-06T05:21:55.113774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:48.334561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117700+0000","last_change":"2023-10-06T05:21:54.104809+0000","last_active":"2023-10-06T05:21:55.117700+0000","last_peered":"2023-10-06T05:21:55.117700+0000","last_clean":"2023-10-06T05:21:55.117700+0000","last_became_active":"2023-10-06T05:21:54.104495+0000","last_became_peered":"2023-10-06T05:21:54.104495+0000","last_unstale":"2023-10-06T05:21:55.117700+0000","last_undegraded":"2023-10-06T05:21:55.117700+0000","last_fullsized":"2023-10-06T05:21:55.117700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:23.320432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.118653+0000","last_change":"2023-10-06T05:21:55.118653+0000","last_active":"2023-10-06T05:21:55.118653+0000","last_peered":"2023-10-06T05:21:55.118653+0000","last_clean":"2023-10-06T05:21:55.118653+0000","last_became_active":"2023-10-06T05:21:55.118238+0000","last_became_peered":"2023-10-06T05:21:55.118238+0000","last_unstale":"2023-10-06T05:21:55.118653+0000","last_undegraded":"2023-10-06T05:21:55.118653+0000","last_fullsized":"2023-10-06T05:21:55.118653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:58.734374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.111316+0000","last_change":"2023-10-06T05:21:54.109392+0000","last_active":"2023-10-06T05:21:54.111316+0000","last_peered":"2023-10-06T05:21:54.111316+0000","last_clean":"2023-10-06T05:21:54.111316+0000","last_became_active":"2023-10-06T05:21:54.109090+0000","last_became_peered":"2023-10-06T05:21:54.109090+0000","last_unstale":"2023-10-06T05:21:54.111316+0000","last_undegraded":"2023-10-06T05:21:54.111316+0000","last_fullsized":"2023-10-06T05:21:54.111316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:10.285768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.093043+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:21:54.093043+0000","last_peered":"2023-10-06T05:21:54.093043+0000","last_clean":"2023-10-06T05:21:54.093043+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:21:54.093043+0000","last_undegraded":"2023-10-06T05:21:54.093043+0000","last_fullsized":"2023-10-06T05:21:54.093043+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.106438+0000","last_change":"2023-10-06T05:21:55.106438+0000","last_active":"2023-10-06T05:21:55.106438+0000","last_peered":"2023-10-06T05:21:55.106438+0000","last_clean":"2023-10-06T05:21:55.106438+0000","last_became_active":"2023-10-06T05:21:55.106172+0000","last_became_peered":"2023-10-06T05:21:55.106172+0000","last_unstale":"2023-10-06T05:21:55.106438+0000","last_undegraded":"2023-10-06T05:21:55.106438+0000","last_fullsized":"2023-10-06T05:21:55.106438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:17.834770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.102789+0000","last_change":"2023-10-06T05:21:54.100165+0000","last_active":"2023-10-06T05:21:54.102789+0000","last_peered":"2023-10-06T05:21:54.102789+0000","last_clean":"2023-10-06T05:21:54.102789+0000","last_became_active":"2023-10-06T05:21:54.099888+0000","last_became_peered":"2023-10-06T05:21:54.099888+0000","last_unstale":"2023-10-06T05:21:54.102789+0000","last_undegraded":"2023-10-06T05:21:54.102789+0000","last_fullsized":"2023-10-06T05:21:54.102789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:52.906906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.094375+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:21:55.094375+0000","last_peered":"2023-10-06T05:21:55.094375+0000","last_clean":"2023-10-06T05:21:55.094375+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:21:55.094375+0000","last_undegraded":"2023-10-06T05:21:55.094375+0000","last_fullsized":"2023-10-06T05:21:55.094375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117460+0000","last_change":"2023-10-06T05:21:55.117460+0000","last_active":"2023-10-06T05:21:55.117460+0000","last_peered":"2023-10-06T05:21:55.117460+0000","last_clean":"2023-10-06T05:21:55.117460+0000","last_became_active":"2023-10-06T05:21:55.117182+0000","last_became_peered":"2023-10-06T05:21:55.117182+0000","last_unstale":"2023-10-06T05:21:55.117460+0000","last_undegraded":"2023-10-06T05:21:55.117460+0000","last_fullsized":"2023-10-06T05:21:55.117460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:43.984677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"82'1","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.104867+0000","last_change":"2023-10-06T05:21:54.109419+0000","last_active":"2023-10-06T05:21:55.104867+0000","last_peered":"2023-10-06T05:21:55.104867+0000","last_clean":"2023-10-06T05:21:55.104867+0000","last_became_active":"2023-10-06T05:21:54.109036+0000","last_became_peered":"2023-10-06T05:21:54.109036+0000","last_unstale":"2023-10-06T05:21:55.104867+0000","last_undegraded":"2023-10-06T05:21:55.104867+0000","last_fullsized":"2023-10-06T05:21:55.104867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:02.886880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.090842+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:21:55.090842+0000","last_peered":"2023-10-06T05:21:55.090842+0000","last_clean":"2023-10-06T05:21:55.090842+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:21:55.090842+0000","last_undegraded":"2023-10-06T05:21:55.090842+0000","last_fullsized":"2023-10-06T05:21:55.090842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":252,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117791+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:21:55.117791+0000","last_peered":"2023-10-06T05:21:55.117791+0000","last_clean":"2023-10-06T05:21:55.117791+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:21:55.117791+0000","last_undegraded":"2023-10-06T05:21:55.117791+0000","last_fullsized":"2023-10-06T05:21:55.117791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.118520+0000","last_change":"2023-10-06T05:21:55.118520+0000","last_active":"2023-10-06T05:21:55.118520+0000","last_peered":"2023-10-06T05:21:55.118520+0000","last_clean":"2023-10-06T05:21:55.118520+0000","last_became_active":"2023-10-06T05:21:55.118204+0000","last_became_peered":"2023-10-06T05:21:55.118204+0000","last_unstale":"2023-10-06T05:21:55.118520+0000","last_undegraded":"2023-10-06T05:21:55.118520+0000","last_fullsized":"2023-10-06T05:21:55.118520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:17.791265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.120682+0000","last_change":"2023-10-06T05:21:54.105846+0000","last_active":"2023-10-06T05:21:55.120682+0000","last_peered":"2023-10-06T05:21:55.120682+0000","last_clean":"2023-10-06T05:21:55.120682+0000","last_became_active":"2023-10-06T05:21:54.105572+0000","last_became_peered":"2023-10-06T05:21:54.105572+0000","last_unstale":"2023-10-06T05:21:55.120682+0000","last_undegraded":"2023-10-06T05:21:55.120682+0000","last_fullsized":"2023-10-06T05:21:55.120682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:26.571172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109371+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:21:55.109371+0000","last_peered":"2023-10-06T05:21:55.109371+0000","last_clean":"2023-10-06T05:21:55.109371+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:21:55.109371+0000","last_undegraded":"2023-10-06T05:21:55.109371+0000","last_fullsized":"2023-10-06T05:21:55.109371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.132069+0000","last_change":"2023-10-06T05:21:55.129506+0000","last_active":"2023-10-06T05:21:55.132069+0000","last_peered":"2023-10-06T05:21:55.132069+0000","last_clean":"2023-10-06T05:21:55.132069+0000","last_became_active":"2023-10-06T05:21:55.129219+0000","last_became_peered":"2023-10-06T05:21:55.129219+0000","last_unstale":"2023-10-06T05:21:55.132069+0000","last_undegraded":"2023-10-06T05:21:55.132069+0000","last_fullsized":"2023-10-06T05:21:55.132069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:13.432899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.091306+0000","last_change":"2023-10-06T05:21:54.107130+0000","last_active":"2023-10-06T05:21:55.091306+0000","last_peered":"2023-10-06T05:21:55.091306+0000","last_clean":"2023-10-06T05:21:55.091306+0000","last_became_active":"2023-10-06T05:21:54.106816+0000","last_became_peered":"2023-10-06T05:21:54.106816+0000","last_unstale":"2023-10-06T05:21:55.091306+0000","last_undegraded":"2023-10-06T05:21:55.091306+0000","last_fullsized":"2023-10-06T05:21:55.091306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:53.371705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117874+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:21:55.117874+0000","last_peered":"2023-10-06T05:21:55.117874+0000","last_clean":"2023-10-06T05:21:55.117874+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:21:55.117874+0000","last_undegraded":"2023-10-06T05:21:55.117874+0000","last_fullsized":"2023-10-06T05:21:55.117874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.105219+0000","last_change":"2023-10-06T05:21:55.105219+0000","last_active":"2023-10-06T05:21:55.105219+0000","last_peered":"2023-10-06T05:21:55.105219+0000","last_clean":"2023-10-06T05:21:55.105219+0000","last_became_active":"2023-10-06T05:21:55.103917+0000","last_became_peered":"2023-10-06T05:21:55.103917+0000","last_unstale":"2023-10-06T05:21:55.105219+0000","last_undegraded":"2023-10-06T05:21:55.105219+0000","last_fullsized":"2023-10-06T05:21:55.105219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:39.207736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"82'1","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.094742+0000","last_change":"2023-10-06T05:21:54.105918+0000","last_active":"2023-10-06T05:21:55.094742+0000","last_peered":"2023-10-06T05:21:55.094742+0000","last_clean":"2023-10-06T05:21:55.094742+0000","last_became_active":"2023-10-06T05:21:54.105521+0000","last_became_peered":"2023-10-06T05:21:54.105521+0000","last_unstale":"2023-10-06T05:21:55.094742+0000","last_undegraded":"2023-10-06T05:21:55.094742+0000","last_fullsized":"2023-10-06T05:21:55.094742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:21.849334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.096244+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:21:54.096244+0000","last_peered":"2023-10-06T05:21:54.096244+0000","last_clean":"2023-10-06T05:21:54.096244+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:21:54.096244+0000","last_undegraded":"2023-10-06T05:21:54.096244+0000","last_fullsized":"2023-10-06T05:21:54.096244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107671+0000","last_change":"2023-10-06T05:21:55.107671+0000","last_active":"2023-10-06T05:21:55.107671+0000","last_peered":"2023-10-06T05:21:55.107671+0000","last_clean":"2023-10-06T05:21:55.107671+0000","last_became_active":"2023-10-06T05:21:55.106299+0000","last_became_peered":"2023-10-06T05:21:55.106299+0000","last_unstale":"2023-10-06T05:21:55.107671+0000","last_undegraded":"2023-10-06T05:21:55.107671+0000","last_fullsized":"2023-10-06T05:21:55.107671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:25.622184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.103833+0000","last_change":"2023-10-06T05:21:54.101103+0000","last_active":"2023-10-06T05:21:54.103833+0000","last_peered":"2023-10-06T05:21:54.103833+0000","last_clean":"2023-10-06T05:21:54.103833+0000","last_became_active":"2023-10-06T05:21:54.100815+0000","last_became_peered":"2023-10-06T05:21:54.100815+0000","last_unstale":"2023-10-06T05:21:54.103833+0000","last_undegraded":"2023-10-06T05:21:54.103833+0000","last_fullsized":"2023-10-06T05:21:54.103833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:34.404148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.097874+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:21:54.097874+0000","last_peered":"2023-10-06T05:21:54.097874+0000","last_clean":"2023-10-06T05:21:54.097874+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:21:54.097874+0000","last_undegraded":"2023-10-06T05:21:54.097874+0000","last_fullsized":"2023-10-06T05:21:54.097874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.113708+0000","last_change":"2023-10-06T05:21:55.113708+0000","last_active":"2023-10-06T05:21:55.113708+0000","last_peered":"2023-10-06T05:21:55.113708+0000","last_clean":"2023-10-06T05:21:55.113708+0000","last_became_active":"2023-10-06T05:21:55.113346+0000","last_became_peered":"2023-10-06T05:21:55.113346+0000","last_unstale":"2023-10-06T05:21:55.113708+0000","last_undegraded":"2023-10-06T05:21:55.113708+0000","last_fullsized":"2023-10-06T05:21:55.113708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:55.274486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"82'2","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.114192+0000","last_change":"2023-10-06T05:21:54.105131+0000","last_active":"2023-10-06T05:21:55.114192+0000","last_peered":"2023-10-06T05:21:55.114192+0000","last_clean":"2023-10-06T05:21:55.114192+0000","last_became_active":"2023-10-06T05:21:54.104823+0000","last_became_peered":"2023-10-06T05:21:54.104823+0000","last_unstale":"2023-10-06T05:21:55.114192+0000","last_undegraded":"2023-10-06T05:21:55.114192+0000","last_fullsized":"2023-10-06T05:21:55.114192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:31.398533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.098007+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:21:54.098007+0000","last_peered":"2023-10-06T05:21:54.098007+0000","last_clean":"2023-10-06T05:21:54.098007+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:21:54.098007+0000","last_undegraded":"2023-10-06T05:21:54.098007+0000","last_fullsized":"2023-10-06T05:21:54.098007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302956,"kb_used_data":1572,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068884,"statfs":{"total":96636764160,"available":96326537216,"internally_reserved":0,"allocated":1609728,"data_stored":1046798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.748}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.68200000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93600000000000005,"5min":1.018,"15min":1.018},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95999999999999996,"5min":1.056,"15min":1.056},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73399999999999999,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67700000000000005,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93300000000000005,"5min":0.999,"15min":0.999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.83799999999999997},"min":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0389999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.46600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.626,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.85399999999999998},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0029999999999999,"5min":1.103,"15min":1.103},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298292,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073548,"statfs":{"total":96636764160,"available":96331313152,"internally_reserved":0,"allocated":1028096,"data_stored":458486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.47099999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89800000000000002,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90800000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78500000000000003,"5min":1.369,"15min":1.369},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.891,"15min":1.891},"last":0.624}]},{"osd":3,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89800000000000002,"5min":1.347,"15min":1.347},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":1,"15min":1},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73499999999999999,"15min":0.78500000000000003},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.95499999999999996,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78200000000000003,"15min":0.82399999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.74399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85899999999999999,"5min":1.415,"15min":1.415},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77500000000000002,"15min":0.83099999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98199999999999998,"5min":2,"15min":2},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.78000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95999999999999996,"5min":1.46,"15min":1.46},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298868,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072972,"statfs":{"total":96636764160,"available":96330723328,"internally_reserved":0,"allocated":1617920,"data_stored":1053453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.379},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.59699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82499999999999996},"min":{"1min":0.63600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.753},"min":{"1min":0.621,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.052,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.82999999999999996,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.83499999999999996,"15min":0.82499999999999996},"min":{"1min":0.67100000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.69299999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.087,"5min":1.103,"15min":1.103},"last":0.72499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073544,"statfs":{"total":96636764160,"available":96331309056,"internally_reserved":0,"allocated":1032192,"data_stored":458588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68300000000000005,"5min":1.042,"15min":1.042},"last":0.33100000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.438},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1519999999999999,"5min":1.242,"15min":1.242},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":1.198,"15min":1.198},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.77300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1659999999999999,"5min":1.26,"15min":1.26},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.073,"5min":1.323,"15min":1.323},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.123,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.264,"15min":1.264},"last":0.745},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.04,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298272,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073568,"statfs":{"total":96636764160,"available":96331333632,"internally_reserved":0,"allocated":1007616,"data_stored":456374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.437},{"interface":"front","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.46000000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46500000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.624,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.503},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68799999999999994,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93500000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90600000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96299999999999997,"5min":1.099,"15min":1.099},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78500000000000003,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.90000000000000002,"5min":1.163,"15min":1.163},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98799999999999999,"5min":1.21,"15min":1.21},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.022,"5min":1.042,"15min":1.042},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84399999999999997,"15min":0.872},"min":{"1min":0.64600000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0469999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.56499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298272,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073568,"statfs":{"total":96636764160,"available":96331333632,"internally_reserved":0,"allocated":1007616,"data_stored":456374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.44600000000000001,"15min":0.46200000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57199999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.624,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.496},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72299999999999998,"5min":1.014,"15min":1.014},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.145,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1830000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.03,"5min":1.204,"15min":1.204},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.329},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.82099999999999995},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.048,"5min":1.484,"15min":1.484},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83499999999999996,"15min":0.85399999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1930000000000001,"5min":1.629,"15min":1.629},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.83999999999999997,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1950000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.085,"5min":1.325,"15min":1.325},"last":0.82299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298280,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073560,"statfs":{"total":96636764160,"available":96331325440,"internally_reserved":0,"allocated":1015808,"data_stored":456534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74399999999999999,"15min":0.79800000000000004},"min":{"1min":0.52100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":2.77,"15min":2.77},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.625}]},{"osd":2,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.76600000000000001,"15min":0.82999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89600000000000002,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.58599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.90500000000000003,"15min":1.091},"min":{"1min":0.50800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":7.742,"15min":7.742},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.48999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96299999999999997,"5min":1.079,"15min":1.079},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90900000000000003,"5min":1.181,"15min":1.181},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.745},"min":{"1min":0.56499999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.60599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.92200000000000004,"5min":1.171,"15min":1.171},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.624,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96499999999999997,"5min":1.228,"15min":1.228},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81499999999999995,"15min":0.86899999999999999},"min":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97399999999999998,"5min":1.571,"15min":1.571},"last":0.91700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298272,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073568,"statfs":{"total":96636764160,"available":96331333632,"internally_reserved":0,"allocated":1007616,"data_stored":456374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.83099999999999996,"15min":1},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.85499999999999998,"15min":1.0209999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.079,"5min":9.25,"15min":9.25},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.76000000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.875,"15min":1.0620000000000001},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":10.384,"15min":10.384},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":1.0760000000000001},"min":{"1min":0.49299999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0349999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":1.1240000000000001},"min":{"1min":0.54800000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.95199999999999996,"5min":10.205,"15min":10.205},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96399999999999997,"5min":1.292,"15min":1.292},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.048,"5min":1.171,"15min":1.171},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78400000000000003,"15min":0.81699999999999995},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.298,"15min":1.298},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80300000000000005,"15min":0.82899999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98999999999999999,"5min":1.204,"15min":1.204},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90200000000000002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:59.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:21:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.735+0000 7f8ac7dff700 1 Processor -- start 2023-10-06T05:21:59.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.735+0000 7f8ac7dff700 1 -- start start 2023-10-06T05:21:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.736+0000 7f8ac7dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8ac006ec60 0x7f8ac00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.736+0000 7f8ac7dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac006be70 con 0x7f8ac013fb00 2023-10-06T05:21:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.736+0000 7f8ac7dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac006ca10 con 0x7f8ac013fed0 2023-10-06T05:21:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.736+0000 7f8ac7dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac006f130 con 0x7f8ac006ec60 2023-10-06T05:21:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.736+0000 7f8ac639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8ac006ec60 0x7f8ac00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.737+0000 7f8ac5b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8ac013fb00 0x7f8ac006c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59564/0 (socket says 172.21.15.31:59564) 2023-10-06T05:21:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.737+0000 7f8ac5b9b700 1 -- 172.21.15.31:0/1478507600 learned_addr learned my addr 172.21.15.31:0/1478507600 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:21:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.737+0000 7f8ac639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8ac006ec60 0x7f8ac00cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56374/0 (socket says 172.21.15.31:56374) 2023-10-06T05:21:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.737+0000 7f8ac639c700 1 -- 172.21.15.31:0/1478507600 >> v1:172.21.15.62:6789/0 conn(0x7f8ac013fed0 legacy=0x7f8ac006e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.737+0000 7f8ac639c700 1 -- 172.21.15.31:0/1478507600 >> v1:172.21.15.31:6789/0 conn(0x7f8ac013fb00 legacy=0x7f8ac006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.737+0000 7f8ac639c700 1 -- 172.21.15.31:0/1478507600 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac006f410 con 0x7f8ac006ec60 2023-10-06T05:21:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.737+0000 7f8ac4b99700 1 -- 172.21.15.31:0/1478507600 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089499487 0 0) 0x7f8ac006be70 con 0x7f8ac013fb00 2023-10-06T05:21:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.737+0000 7f8ac639c700 1 --2- 172.21.15.31:0/1478507600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac006ec60 0x7f8ac00cadb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009e60 tx=0x7f8ab8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5505e28a16a177b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.737+0000 7f8ac4b99700 1 -- 172.21.15.31:0/1478507600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800e070 con 0x7f8ac006ec60 2023-10-06T05:21:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.737+0000 7f8ac4b99700 1 -- 172.21.15.31:0/1478507600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab800bc30 con 0x7f8ac006ec60 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.738+0000 7f8ac4b99700 1 -- 172.21.15.31:0/1478507600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8012430 con 0x7f8ac006ec60 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.738+0000 7f8ac7dff700 1 -- 172.21.15.31:0/1478507600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac006ec60 msgr2=0x7f8ac00cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.738+0000 7f8ac7dff700 1 --2- 172.21.15.31:0/1478507600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac006ec60 0x7f8ac00cadb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009e60 tx=0x7f8ab8009780 comp rx=0 tx=0).stop 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.738+0000 7f8ac7dff700 1 -- 172.21.15.31:0/1478507600 shutdown_connections 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.738+0000 7f8ac7dff700 1 --2- 172.21.15.31:0/1478507600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac006ec60 0x7f8ac00cadb0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.738+0000 7f8ac7dff700 1 -- 172.21.15.31:0/1478507600 >> 172.21.15.31:0/1478507600 conn(0x7f8ac00cdcb0 msgr2=0x7f8ac00c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.738+0000 7f8ac7dff700 1 -- 172.21.15.31:0/1478507600 shutdown_connections 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.738+0000 7f8ac7dff700 1 -- 172.21.15.31:0/1478507600 wait complete. 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.740+0000 7f8ac7dff700 1 Processor -- start 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.740+0000 7f8ac7dff700 1 -- start start 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac7dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac006ec60 0x7f8ac016c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac7dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ac013fb00 0x7f8ac00aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac7dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac013fed0 0x7f8ac00b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac7dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00b13e0 con 0x7f8ac013fed0 2023-10-06T05:21:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac7dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00b07d0 con 0x7f8ac006ec60 2023-10-06T05:21:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac7dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00b0ab0 con 0x7f8ac013fb00 2023-10-06T05:21:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac013fed0 0x7f8ac00b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac013fed0 0x7f8ac00b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56386/0 (socket says 172.21.15.31:56386) 2023-10-06T05:21:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac639c700 1 -- 172.21.15.31:0/427326364 learned_addr learned my addr 172.21.15.31:0/427326364 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:21:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac5b9b700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac006ec60 0x7f8ac016c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac539a700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ac013fb00 0x7f8ac00aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac639c700 1 -- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ac013fb00 msgr2=0x7f8ac00aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.741+0000 7f8ac639c700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ac013fb00 0x7f8ac00aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.742+0000 7f8ac639c700 1 -- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac006ec60 msgr2=0x7f8ac016c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.742+0000 7f8ac639c700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac006ec60 0x7f8ac016c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.742+0000 7f8ac639c700 1 -- 172.21.15.31:0/427326364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab8009a20 con 0x7f8ac013fed0 2023-10-06T05:21:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.742+0000 7f8ac639c700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac013fed0 0x7f8ac00b20c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8ac006da70 tx=0x7f8ab8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.742+0000 7f8ab5ffb700 1 -- 172.21.15.31:0/427326364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab801b420 con 0x7f8ac013fed0 2023-10-06T05:21:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.742+0000 7f8ab5ffb700 1 -- 172.21.15.31:0/427326364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab801b580 con 0x7f8ac013fed0 2023-10-06T05:21:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.742+0000 7f8ab5ffb700 1 -- 172.21.15.31:0/427326364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab801b940 con 0x7f8ac013fed0 2023-10-06T05:21:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.742+0000 7f8ac7dff700 1 -- 172.21.15.31:0/427326364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac00b0d90 con 0x7f8ac013fed0 2023-10-06T05:21:59.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.742+0000 7f8ac7dff700 1 -- 172.21.15.31:0/427326364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac00b04c0 con 0x7f8ac013fed0 2023-10-06T05:21:59.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.743+0000 7f8ac7dff700 1 -- 172.21.15.31:0/427326364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac004c380 con 0x7f8ac013fed0 2023-10-06T05:21:59.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.744+0000 7f8ab5ffb700 1 -- 172.21.15.31:0/427326364 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f8ab801bcb0 con 0x7f8ac013fed0 2023-10-06T05:21:59.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.745+0000 7f8ab5ffb700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8aa4067050 0x7f8aa4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:21:59.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.745+0000 7f8ab5ffb700 1 -- 172.21.15.31:0/427326364 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8ab80864b0 con 0x7f8ac013fed0 2023-10-06T05:21:59.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.746+0000 7f8ac5b9b700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8aa4067050 0x7f8aa4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:21:59.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.746+0000 7f8ac5b9b700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8aa4067050 0x7f8aa4069500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8abc007fd0 tx=0x7f8abc007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:21:59.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.749+0000 7f8ab5ffb700 1 -- 172.21.15.31:0/427326364 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ab8053300 con 0x7f8ac013fed0 2023-10-06T05:21:59.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.948+0000 7f8ac7dff700 1 -- 172.21.15.31:0/427326364 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ac00b1520 con 0x7f8aa4067050 2023-10-06T05:21:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.958+0000 7f8ab5ffb700 1 -- 172.21.15.31:0/427326364 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311310 (secure 0 0 0) 0x7f8ac00b1520 con 0x7f8aa4067050 2023-10-06T05:21:59.968 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:21:59.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.962+0000 7f8ac7dff700 1 -- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8aa4067050 msgr2=0x7f8aa4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8aa4067050 0x7f8aa4069500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8abc007fd0 tx=0x7f8abc007d30 comp rx=0 tx=0).stop 2023-10-06T05:21:59.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 -- 172.21.15.31:0/427326364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac013fed0 msgr2=0x7f8ac00b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:21:59.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac013fed0 0x7f8ac00b20c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8ac006da70 tx=0x7f8ab8019000 comp rx=0 tx=0).stop 2023-10-06T05:21:59.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 -- 172.21.15.31:0/427326364 shutdown_connections 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8aa4067050 0x7f8aa4069500 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac013fed0 0x7f8ac00b20c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac006ec60 0x7f8ac016c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 --2- 172.21.15.31:0/427326364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ac013fb00 0x7f8ac00aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 -- 172.21.15.31:0/427326364 >> 172.21.15.31:0/427326364 conn(0x7f8ac00cdcb0 msgr2=0x7f8ac0067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:21:59.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 -- 172.21.15.31:0/427326364 shutdown_connections 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:21:59.963+0000 7f8ac7dff700 1 -- 172.21.15.31:0/427326364 wait complete. 2023-10-06T05:21:59.974 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:21:59.982 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T05:21:59.463623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2391508,"kb_used_data":9108,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752583212,"statfs":{"total":773094113280,"available":770645209088,"internally_reserved":0,"allocated":9326592,"data_stored":4842981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001787"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.128835+0000","last_change":"2023-10-06T05:21:55.125060+0000","last_active":"2023-10-06T05:21:55.128835+0000","last_peered":"2023-10-06T05:21:55.128835+0000","last_clean":"2023-10-06T05:21:55.128835+0000","last_became_active":"2023-10-06T05:21:55.124721+0000","last_became_peered":"2023-10-06T05:21:55.124721+0000","last_unstale":"2023-10-06T05:21:55.128835+0000","last_undegraded":"2023-10-06T05:21:55.128835+0000","last_fullsized":"2023-10-06T05:21:55.128835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:24.323427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.110904+0000","last_change":"2023-10-06T05:21:55.110904+0000","last_active":"2023-10-06T05:21:55.110904+0000","last_peered":"2023-10-06T05:21:55.110904+0000","last_clean":"2023-10-06T05:21:55.110904+0000","last_became_active":"2023-10-06T05:21:55.110634+0000","last_became_peered":"2023-10-06T05:21:55.110634+0000","last_unstale":"2023-10-06T05:21:55.110904+0000","last_undegraded":"2023-10-06T05:21:55.110904+0000","last_fullsized":"2023-10-06T05:21:55.110904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:39.207736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.120718+0000","last_change":"2023-10-06T05:21:55.112039+0000","last_active":"2023-10-06T05:21:55.120718+0000","last_peered":"2023-10-06T05:21:55.120718+0000","last_clean":"2023-10-06T05:21:55.120718+0000","last_became_active":"2023-10-06T05:21:55.111712+0000","last_became_peered":"2023-10-06T05:21:55.111712+0000","last_unstale":"2023-10-06T05:21:55.120718+0000","last_undegraded":"2023-10-06T05:21:55.120718+0000","last_fullsized":"2023-10-06T05:21:55.120718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:25.396272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117633+0000","last_change":"2023-10-06T05:21:55.113456+0000","last_active":"2023-10-06T05:21:55.117633+0000","last_peered":"2023-10-06T05:21:55.117633+0000","last_clean":"2023-10-06T05:21:55.117633+0000","last_became_active":"2023-10-06T05:21:55.112984+0000","last_became_peered":"2023-10-06T05:21:55.112984+0000","last_unstale":"2023-10-06T05:21:55.117633+0000","last_undegraded":"2023-10-06T05:21:55.117633+0000","last_fullsized":"2023-10-06T05:21:55.117633+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:04.065404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.111553+0000","last_change":"2023-10-06T05:21:55.111553+0000","last_active":"2023-10-06T05:21:55.111553+0000","last_peered":"2023-10-06T05:21:55.111553+0000","last_clean":"2023-10-06T05:21:55.111553+0000","last_became_active":"2023-10-06T05:21:55.111278+0000","last_became_peered":"2023-10-06T05:21:55.111278+0000","last_unstale":"2023-10-06T05:21:55.111553+0000","last_undegraded":"2023-10-06T05:21:55.111553+0000","last_fullsized":"2023-10-06T05:21:55.111553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.623407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109277+0000","last_change":"2023-10-06T05:21:55.109277+0000","last_active":"2023-10-06T05:21:55.109277+0000","last_peered":"2023-10-06T05:21:55.109277+0000","last_clean":"2023-10-06T05:21:55.109277+0000","last_became_active":"2023-10-06T05:21:55.108625+0000","last_became_peered":"2023-10-06T05:21:55.108625+0000","last_unstale":"2023-10-06T05:21:55.109277+0000","last_undegraded":"2023-10-06T05:21:55.109277+0000","last_fullsized":"2023-10-06T05:21:55.109277+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:12.449837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.128468+0000","last_change":"2023-10-06T05:21:55.124389+0000","last_active":"2023-10-06T05:21:55.128468+0000","last_peered":"2023-10-06T05:21:55.128468+0000","last_clean":"2023-10-06T05:21:55.128468+0000","last_became_active":"2023-10-06T05:21:55.124108+0000","last_became_peered":"2023-10-06T05:21:55.124108+0000","last_unstale":"2023-10-06T05:21:55.128468+0000","last_undegraded":"2023-10-06T05:21:55.128468+0000","last_fullsized":"2023-10-06T05:21:55.128468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:48.334561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.126279+0000","last_change":"2023-10-06T05:21:55.121070+0000","last_active":"2023-10-06T05:21:55.126279+0000","last_peered":"2023-10-06T05:21:55.126279+0000","last_clean":"2023-10-06T05:21:55.126279+0000","last_became_active":"2023-10-06T05:21:55.120668+0000","last_became_peered":"2023-10-06T05:21:55.120668+0000","last_unstale":"2023-10-06T05:21:55.126279+0000","last_undegraded":"2023-10-06T05:21:55.126279+0000","last_fullsized":"2023-10-06T05:21:55.126279+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:44.248381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.108025+0000","last_change":"2023-10-06T05:21:55.108025+0000","last_active":"2023-10-06T05:21:55.108025+0000","last_peered":"2023-10-06T05:21:55.108025+0000","last_clean":"2023-10-06T05:21:55.108025+0000","last_became_active":"2023-10-06T05:21:55.107747+0000","last_became_peered":"2023-10-06T05:21:55.107747+0000","last_unstale":"2023-10-06T05:21:55.108025+0000","last_undegraded":"2023-10-06T05:21:55.108025+0000","last_fullsized":"2023-10-06T05:21:55.108025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:31.508522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.108653+0000","last_change":"2023-10-06T05:21:55.108653+0000","last_active":"2023-10-06T05:21:55.108653+0000","last_peered":"2023-10-06T05:21:55.108653+0000","last_clean":"2023-10-06T05:21:55.108653+0000","last_became_active":"2023-10-06T05:21:55.108400+0000","last_became_peered":"2023-10-06T05:21:55.108400+0000","last_unstale":"2023-10-06T05:21:55.108653+0000","last_undegraded":"2023-10-06T05:21:55.108653+0000","last_fullsized":"2023-10-06T05:21:55.108653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:57.737443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.125579+0000","last_change":"2023-10-06T05:21:55.125579+0000","last_active":"2023-10-06T05:21:55.125579+0000","last_peered":"2023-10-06T05:21:55.125579+0000","last_clean":"2023-10-06T05:21:55.125579+0000","last_became_active":"2023-10-06T05:21:55.125072+0000","last_became_peered":"2023-10-06T05:21:55.125072+0000","last_unstale":"2023-10-06T05:21:55.125579+0000","last_undegraded":"2023-10-06T05:21:55.125579+0000","last_fullsized":"2023-10-06T05:21:55.125579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:31.402430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.106136+0000","last_change":"2023-10-06T05:21:55.106136+0000","last_active":"2023-10-06T05:21:55.106136+0000","last_peered":"2023-10-06T05:21:55.106136+0000","last_clean":"2023-10-06T05:21:55.106136+0000","last_became_active":"2023-10-06T05:21:55.105864+0000","last_became_peered":"2023-10-06T05:21:55.105864+0000","last_unstale":"2023-10-06T05:21:55.106136+0000","last_undegraded":"2023-10-06T05:21:55.106136+0000","last_fullsized":"2023-10-06T05:21:55.106136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:21.556816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.126219+0000","last_change":"2023-10-06T05:21:55.120795+0000","last_active":"2023-10-06T05:21:55.126219+0000","last_peered":"2023-10-06T05:21:55.126219+0000","last_clean":"2023-10-06T05:21:55.126219+0000","last_became_active":"2023-10-06T05:21:55.120506+0000","last_became_peered":"2023-10-06T05:21:55.120506+0000","last_unstale":"2023-10-06T05:21:55.126219+0000","last_undegraded":"2023-10-06T05:21:55.126219+0000","last_fullsized":"2023-10-06T05:21:55.126219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:31.592670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.108986+0000","last_change":"2023-10-06T05:21:55.108986+0000","last_active":"2023-10-06T05:21:55.108986+0000","last_peered":"2023-10-06T05:21:55.108986+0000","last_clean":"2023-10-06T05:21:55.108986+0000","last_became_active":"2023-10-06T05:21:55.108683+0000","last_became_peered":"2023-10-06T05:21:55.108683+0000","last_unstale":"2023-10-06T05:21:55.108986+0000","last_undegraded":"2023-10-06T05:21:55.108986+0000","last_fullsized":"2023-10-06T05:21:55.108986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:39.502991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107796+0000","last_change":"2023-10-06T05:21:55.104581+0000","last_active":"2023-10-06T05:21:55.107796+0000","last_peered":"2023-10-06T05:21:55.107796+0000","last_clean":"2023-10-06T05:21:55.107796+0000","last_became_active":"2023-10-06T05:21:55.104226+0000","last_became_peered":"2023-10-06T05:21:55.104226+0000","last_unstale":"2023-10-06T05:21:55.107796+0000","last_undegraded":"2023-10-06T05:21:55.107796+0000","last_fullsized":"2023-10-06T05:21:55.107796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:32.401527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.114721+0000","last_change":"2023-10-06T05:21:55.114721+0000","last_active":"2023-10-06T05:21:55.114721+0000","last_peered":"2023-10-06T05:21:55.114721+0000","last_clean":"2023-10-06T05:21:55.114721+0000","last_became_active":"2023-10-06T05:21:55.114455+0000","last_became_peered":"2023-10-06T05:21:55.114455+0000","last_unstale":"2023-10-06T05:21:55.114721+0000","last_undegraded":"2023-10-06T05:21:55.114721+0000","last_fullsized":"2023-10-06T05:21:55.114721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:01.621113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.124179+0000","last_change":"2023-10-06T05:21:55.124179+0000","last_active":"2023-10-06T05:21:55.124179+0000","last_peered":"2023-10-06T05:21:55.124179+0000","last_clean":"2023-10-06T05:21:55.124179+0000","last_became_active":"2023-10-06T05:21:55.119479+0000","last_became_peered":"2023-10-06T05:21:55.119479+0000","last_unstale":"2023-10-06T05:21:55.124179+0000","last_undegraded":"2023-10-06T05:21:55.124179+0000","last_fullsized":"2023-10-06T05:21:55.124179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:49.772782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.113011+0000","last_change":"2023-10-06T05:21:55.104047+0000","last_active":"2023-10-06T05:21:55.113011+0000","last_peered":"2023-10-06T05:21:55.113011+0000","last_clean":"2023-10-06T05:21:55.113011+0000","last_became_active":"2023-10-06T05:21:55.103693+0000","last_became_peered":"2023-10-06T05:21:55.103693+0000","last_unstale":"2023-10-06T05:21:55.113011+0000","last_undegraded":"2023-10-06T05:21:55.113011+0000","last_fullsized":"2023-10-06T05:21:55.113011+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:05.927815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109914+0000","last_change":"2023-10-06T05:21:55.109914+0000","last_active":"2023-10-06T05:21:55.109914+0000","last_peered":"2023-10-06T05:21:55.109914+0000","last_clean":"2023-10-06T05:21:55.109914+0000","last_became_active":"2023-10-06T05:21:55.109525+0000","last_became_peered":"2023-10-06T05:21:55.109525+0000","last_unstale":"2023-10-06T05:21:55.109914+0000","last_undegraded":"2023-10-06T05:21:55.109914+0000","last_fullsized":"2023-10-06T05:21:55.109914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:10.295171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107235+0000","last_change":"2023-10-06T05:21:55.097872+0000","last_active":"2023-10-06T05:21:55.107235+0000","last_peered":"2023-10-06T05:21:55.107235+0000","last_clean":"2023-10-06T05:21:55.107235+0000","last_became_active":"2023-10-06T05:21:55.096116+0000","last_became_peered":"2023-10-06T05:21:55.096116+0000","last_unstale":"2023-10-06T05:21:55.107235+0000","last_undegraded":"2023-10-06T05:21:55.107235+0000","last_fullsized":"2023-10-06T05:21:55.107235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:58.692209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117744+0000","last_change":"2023-10-06T05:21:55.117744+0000","last_active":"2023-10-06T05:21:55.117744+0000","last_peered":"2023-10-06T05:21:55.117744+0000","last_clean":"2023-10-06T05:21:55.117744+0000","last_became_active":"2023-10-06T05:21:55.117472+0000","last_became_peered":"2023-10-06T05:21:55.117472+0000","last_unstale":"2023-10-06T05:21:55.117744+0000","last_undegraded":"2023-10-06T05:21:55.117744+0000","last_fullsized":"2023-10-06T05:21:55.117744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:43.471419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109532+0000","last_change":"2023-10-06T05:21:55.109532+0000","last_active":"2023-10-06T05:21:55.109532+0000","last_peered":"2023-10-06T05:21:55.109532+0000","last_clean":"2023-10-06T05:21:55.109532+0000","last_became_active":"2023-10-06T05:21:55.109265+0000","last_became_peered":"2023-10-06T05:21:55.109265+0000","last_unstale":"2023-10-06T05:21:55.109532+0000","last_undegraded":"2023-10-06T05:21:55.109532+0000","last_fullsized":"2023-10-06T05:21:55.109532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:58.734374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.114632+0000","last_change":"2023-10-06T05:21:55.114632+0000","last_active":"2023-10-06T05:21:55.114632+0000","last_peered":"2023-10-06T05:21:55.114632+0000","last_clean":"2023-10-06T05:21:55.114632+0000","last_became_active":"2023-10-06T05:21:55.114334+0000","last_became_peered":"2023-10-06T05:21:55.114334+0000","last_unstale":"2023-10-06T05:21:55.114632+0000","last_undegraded":"2023-10-06T05:21:55.114632+0000","last_fullsized":"2023-10-06T05:21:55.114632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:06.293881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.121057+0000","last_change":"2023-10-06T05:21:55.119478+0000","last_active":"2023-10-06T05:21:55.121057+0000","last_peered":"2023-10-06T05:21:55.121057+0000","last_clean":"2023-10-06T05:21:55.121057+0000","last_became_active":"2023-10-06T05:21:55.119230+0000","last_became_peered":"2023-10-06T05:21:55.119230+0000","last_unstale":"2023-10-06T05:21:55.121057+0000","last_undegraded":"2023-10-06T05:21:55.121057+0000","last_fullsized":"2023-10-06T05:21:55.121057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:27:31.402430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.115458+0000","last_change":"2023-10-06T05:21:55.115458+0000","last_active":"2023-10-06T05:21:55.115458+0000","last_peered":"2023-10-06T05:21:55.115458+0000","last_clean":"2023-10-06T05:21:55.115458+0000","last_became_active":"2023-10-06T05:21:55.115147+0000","last_became_peered":"2023-10-06T05:21:55.115147+0000","last_unstale":"2023-10-06T05:21:55.115458+0000","last_undegraded":"2023-10-06T05:21:55.115458+0000","last_fullsized":"2023-10-06T05:21:55.115458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:59.909748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109318+0000","last_change":"2023-10-06T05:21:55.109318+0000","last_active":"2023-10-06T05:21:55.109318+0000","last_peered":"2023-10-06T05:21:55.109318+0000","last_clean":"2023-10-06T05:21:55.109318+0000","last_became_active":"2023-10-06T05:21:55.108924+0000","last_became_peered":"2023-10-06T05:21:55.108924+0000","last_unstale":"2023-10-06T05:21:55.109318+0000","last_undegraded":"2023-10-06T05:21:55.109318+0000","last_fullsized":"2023-10-06T05:21:55.109318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:32:55.685815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.112316+0000","last_change":"2023-10-06T05:21:55.112316+0000","last_active":"2023-10-06T05:21:55.112316+0000","last_peered":"2023-10-06T05:21:55.112316+0000","last_clean":"2023-10-06T05:21:55.112316+0000","last_became_active":"2023-10-06T05:21:55.112037+0000","last_became_peered":"2023-10-06T05:21:55.112037+0000","last_unstale":"2023-10-06T05:21:55.112316+0000","last_undegraded":"2023-10-06T05:21:55.112316+0000","last_fullsized":"2023-10-06T05:21:55.112316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:27:17.581662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.111516+0000","last_change":"2023-10-06T05:21:55.107108+0000","last_active":"2023-10-06T05:21:55.111516+0000","last_peered":"2023-10-06T05:21:55.111516+0000","last_clean":"2023-10-06T05:21:55.111516+0000","last_became_active":"2023-10-06T05:21:55.105082+0000","last_became_peered":"2023-10-06T05:21:55.105082+0000","last_unstale":"2023-10-06T05:21:55.111516+0000","last_undegraded":"2023-10-06T05:21:55.111516+0000","last_fullsized":"2023-10-06T05:21:55.111516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:12.731133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.129103+0000","last_change":"2023-10-06T05:21:55.126806+0000","last_active":"2023-10-06T05:21:55.129103+0000","last_peered":"2023-10-06T05:21:55.129103+0000","last_clean":"2023-10-06T05:21:55.129103+0000","last_became_active":"2023-10-06T05:21:55.126550+0000","last_became_peered":"2023-10-06T05:21:55.126550+0000","last_unstale":"2023-10-06T05:21:55.129103+0000","last_undegraded":"2023-10-06T05:21:55.129103+0000","last_fullsized":"2023-10-06T05:21:55.129103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:57.947210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107377+0000","last_change":"2023-10-06T05:21:55.107377+0000","last_active":"2023-10-06T05:21:55.107377+0000","last_peered":"2023-10-06T05:21:55.107377+0000","last_clean":"2023-10-06T05:21:55.107377+0000","last_became_active":"2023-10-06T05:21:55.107085+0000","last_became_peered":"2023-10-06T05:21:55.107085+0000","last_unstale":"2023-10-06T05:21:55.107377+0000","last_undegraded":"2023-10-06T05:21:55.107377+0000","last_fullsized":"2023-10-06T05:21:55.107377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:34.006992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.108435+0000","last_change":"2023-10-06T05:21:55.108435+0000","last_active":"2023-10-06T05:21:55.108435+0000","last_peered":"2023-10-06T05:21:55.108435+0000","last_clean":"2023-10-06T05:21:55.108435+0000","last_became_active":"2023-10-06T05:21:55.108053+0000","last_became_peered":"2023-10-06T05:21:55.108053+0000","last_unstale":"2023-10-06T05:21:55.108435+0000","last_undegraded":"2023-10-06T05:21:55.108435+0000","last_fullsized":"2023-10-06T05:21:55.108435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:39.502991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.115251+0000","last_change":"2023-10-06T05:21:55.115251+0000","last_active":"2023-10-06T05:21:55.115251+0000","last_peered":"2023-10-06T05:21:55.115251+0000","last_clean":"2023-10-06T05:21:55.115251+0000","last_became_active":"2023-10-06T05:21:55.114971+0000","last_became_peered":"2023-10-06T05:21:55.114971+0000","last_unstale":"2023-10-06T05:21:55.115251+0000","last_undegraded":"2023-10-06T05:21:55.115251+0000","last_fullsized":"2023-10-06T05:21:55.115251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:59.799333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117166+0000","last_change":"2023-10-06T05:21:55.117166+0000","last_active":"2023-10-06T05:21:55.117166+0000","last_peered":"2023-10-06T05:21:55.117166+0000","last_clean":"2023-10-06T05:21:55.117166+0000","last_became_active":"2023-10-06T05:21:55.116801+0000","last_became_peered":"2023-10-06T05:21:55.116801+0000","last_unstale":"2023-10-06T05:21:55.117166+0000","last_undegraded":"2023-10-06T05:21:55.117166+0000","last_fullsized":"2023-10-06T05:21:55.117166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:21.451511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"82'4","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.127949+0000","last_change":"2023-10-06T05:21:54.110171+0000","last_active":"2023-10-06T05:21:55.127949+0000","last_peered":"2023-10-06T05:21:55.127949+0000","last_clean":"2023-10-06T05:21:55.127949+0000","last_became_active":"2023-10-06T05:21:54.109861+0000","last_became_peered":"2023-10-06T05:21:54.109861+0000","last_unstale":"2023-10-06T05:21:55.127949+0000","last_undegraded":"2023-10-06T05:21:55.127949+0000","last_fullsized":"2023-10-06T05:21:55.127949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:20.006465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.124820+0000","last_change":"2023-10-06T05:21:55.124820+0000","last_active":"2023-10-06T05:21:55.124820+0000","last_peered":"2023-10-06T05:21:55.124820+0000","last_clean":"2023-10-06T05:21:55.124820+0000","last_became_active":"2023-10-06T05:21:55.124228+0000","last_became_peered":"2023-10-06T05:21:55.124228+0000","last_unstale":"2023-10-06T05:21:55.124820+0000","last_undegraded":"2023-10-06T05:21:55.124820+0000","last_fullsized":"2023-10-06T05:21:55.124820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:23.957678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"82'1","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.094363+0000","last_change":"2023-10-06T05:21:54.106329+0000","last_active":"2023-10-06T05:21:55.094363+0000","last_peered":"2023-10-06T05:21:55.094363+0000","last_clean":"2023-10-06T05:21:55.094363+0000","last_became_active":"2023-10-06T05:21:54.105964+0000","last_became_peered":"2023-10-06T05:21:54.105964+0000","last_unstale":"2023-10-06T05:21:55.094363+0000","last_undegraded":"2023-10-06T05:21:55.094363+0000","last_fullsized":"2023-10-06T05:21:55.094363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:51.047874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.129050+0000","last_change":"2023-10-06T05:21:55.126284+0000","last_active":"2023-10-06T05:21:55.129050+0000","last_peered":"2023-10-06T05:21:55.129050+0000","last_clean":"2023-10-06T05:21:55.129050+0000","last_became_active":"2023-10-06T05:21:55.125958+0000","last_became_peered":"2023-10-06T05:21:55.125958+0000","last_unstale":"2023-10-06T05:21:55.129050+0000","last_undegraded":"2023-10-06T05:21:55.129050+0000","last_fullsized":"2023-10-06T05:21:55.129050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:44.318530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.126892+0000","last_change":"2023-10-06T05:21:54.108473+0000","last_active":"2023-10-06T05:21:55.126892+0000","last_peered":"2023-10-06T05:21:55.126892+0000","last_clean":"2023-10-06T05:21:55.126892+0000","last_became_active":"2023-10-06T05:21:54.108221+0000","last_became_peered":"2023-10-06T05:21:54.108221+0000","last_unstale":"2023-10-06T05:21:55.126892+0000","last_undegraded":"2023-10-06T05:21:55.126892+0000","last_fullsized":"2023-10-06T05:21:55.126892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:24.393277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.124167+0000","last_change":"2023-10-06T05:21:55.124167+0000","last_active":"2023-10-06T05:21:55.124167+0000","last_peered":"2023-10-06T05:21:55.124167+0000","last_clean":"2023-10-06T05:21:55.124167+0000","last_became_active":"2023-10-06T05:21:55.119767+0000","last_became_peered":"2023-10-06T05:21:55.119767+0000","last_unstale":"2023-10-06T05:21:55.124167+0000","last_undegraded":"2023-10-06T05:21:55.124167+0000","last_fullsized":"2023-10-06T05:21:55.124167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:37.907262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"82'2","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.119283+0000","last_change":"2023-10-06T05:21:54.103971+0000","last_active":"2023-10-06T05:21:55.119283+0000","last_peered":"2023-10-06T05:21:55.119283+0000","last_clean":"2023-10-06T05:21:55.119283+0000","last_became_active":"2023-10-06T05:21:54.103703+0000","last_became_peered":"2023-10-06T05:21:54.103703+0000","last_unstale":"2023-10-06T05:21:55.119283+0000","last_undegraded":"2023-10-06T05:21:55.119283+0000","last_fullsized":"2023-10-06T05:21:55.119283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:02.133614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107858+0000","last_change":"2023-10-06T05:21:55.105294+0000","last_active":"2023-10-06T05:21:55.107858+0000","last_peered":"2023-10-06T05:21:55.107858+0000","last_clean":"2023-10-06T05:21:55.107858+0000","last_became_active":"2023-10-06T05:21:55.104082+0000","last_became_peered":"2023-10-06T05:21:55.104082+0000","last_unstale":"2023-10-06T05:21:55.107858+0000","last_undegraded":"2023-10-06T05:21:55.107858+0000","last_fullsized":"2023-10-06T05:21:55.107858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:19.385103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"82'1","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.096355+0000","last_change":"2023-10-06T05:21:54.106503+0000","last_active":"2023-10-06T05:21:55.096355+0000","last_peered":"2023-10-06T05:21:55.096355+0000","last_clean":"2023-10-06T05:21:55.096355+0000","last_became_active":"2023-10-06T05:21:54.106199+0000","last_became_peered":"2023-10-06T05:21:54.106199+0000","last_unstale":"2023-10-06T05:21:55.096355+0000","last_undegraded":"2023-10-06T05:21:55.096355+0000","last_fullsized":"2023-10-06T05:21:55.096355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:27.601855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.125243+0000","last_change":"2023-10-06T05:21:55.125243+0000","last_active":"2023-10-06T05:21:55.125243+0000","last_peered":"2023-10-06T05:21:55.125243+0000","last_clean":"2023-10-06T05:21:55.125243+0000","last_became_active":"2023-10-06T05:21:55.124906+0000","last_became_peered":"2023-10-06T05:21:55.124906+0000","last_unstale":"2023-10-06T05:21:55.125243+0000","last_undegraded":"2023-10-06T05:21:55.125243+0000","last_fullsized":"2023-10-06T05:21:55.125243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:29.796079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"82'1","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.121321+0000","last_change":"2023-10-06T05:21:54.105162+0000","last_active":"2023-10-06T05:21:55.121321+0000","last_peered":"2023-10-06T05:21:55.121321+0000","last_clean":"2023-10-06T05:21:55.121321+0000","last_became_active":"2023-10-06T05:21:54.104813+0000","last_became_peered":"2023-10-06T05:21:54.104813+0000","last_unstale":"2023-10-06T05:21:55.121321+0000","last_undegraded":"2023-10-06T05:21:55.121321+0000","last_fullsized":"2023-10-06T05:21:55.121321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:37.813534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.112316+0000","last_change":"2023-10-06T05:21:55.112316+0000","last_active":"2023-10-06T05:21:55.112316+0000","last_peered":"2023-10-06T05:21:55.112316+0000","last_clean":"2023-10-06T05:21:55.112316+0000","last_became_active":"2023-10-06T05:21:55.109005+0000","last_became_peered":"2023-10-06T05:21:55.109005+0000","last_unstale":"2023-10-06T05:21:55.112316+0000","last_undegraded":"2023-10-06T05:21:55.112316+0000","last_fullsized":"2023-10-06T05:21:55.112316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:26.304630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.094581+0000","last_change":"2023-10-06T05:21:54.103935+0000","last_active":"2023-10-06T05:21:55.094581+0000","last_peered":"2023-10-06T05:21:55.094581+0000","last_clean":"2023-10-06T05:21:55.094581+0000","last_became_active":"2023-10-06T05:21:54.103634+0000","last_became_peered":"2023-10-06T05:21:54.103634+0000","last_unstale":"2023-10-06T05:21:55.094581+0000","last_undegraded":"2023-10-06T05:21:55.094581+0000","last_fullsized":"2023-10-06T05:21:55.094581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:57.731380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117513+0000","last_change":"2023-10-06T05:21:55.112906+0000","last_active":"2023-10-06T05:21:55.117513+0000","last_peered":"2023-10-06T05:21:55.117513+0000","last_clean":"2023-10-06T05:21:55.117513+0000","last_became_active":"2023-10-06T05:21:55.112632+0000","last_became_peered":"2023-10-06T05:21:55.112632+0000","last_unstale":"2023-10-06T05:21:55.117513+0000","last_undegraded":"2023-10-06T05:21:55.117513+0000","last_fullsized":"2023-10-06T05:21:55.117513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:52.372864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117562+0000","last_change":"2023-10-06T05:21:54.112755+0000","last_active":"2023-10-06T05:21:55.117562+0000","last_peered":"2023-10-06T05:21:55.117562+0000","last_clean":"2023-10-06T05:21:55.117562+0000","last_became_active":"2023-10-06T05:21:54.112226+0000","last_became_peered":"2023-10-06T05:21:54.112226+0000","last_unstale":"2023-10-06T05:21:55.117562+0000","last_undegraded":"2023-10-06T05:21:55.117562+0000","last_fullsized":"2023-10-06T05:21:55.117562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:49.698916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107526+0000","last_change":"2023-10-06T05:21:55.107526+0000","last_active":"2023-10-06T05:21:55.107526+0000","last_peered":"2023-10-06T05:21:55.107526+0000","last_clean":"2023-10-06T05:21:55.107526+0000","last_became_active":"2023-10-06T05:21:55.107051+0000","last_became_peered":"2023-10-06T05:21:55.107051+0000","last_unstale":"2023-10-06T05:21:55.107526+0000","last_undegraded":"2023-10-06T05:21:55.107526+0000","last_fullsized":"2023-10-06T05:21:55.107526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:59.909748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.111377+0000","last_change":"2023-10-06T05:21:54.108181+0000","last_active":"2023-10-06T05:21:54.111377+0000","last_peered":"2023-10-06T05:21:54.111377+0000","last_clean":"2023-10-06T05:21:54.111377+0000","last_became_active":"2023-10-06T05:21:54.107890+0000","last_became_peered":"2023-10-06T05:21:54.107890+0000","last_unstale":"2023-10-06T05:21:54.111377+0000","last_undegraded":"2023-10-06T05:21:54.111377+0000","last_fullsized":"2023-10-06T05:21:54.111377+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:03.566492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.128415+0000","last_change":"2023-10-06T05:21:55.124073+0000","last_active":"2023-10-06T05:21:55.128415+0000","last_peered":"2023-10-06T05:21:55.128415+0000","last_clean":"2023-10-06T05:21:55.128415+0000","last_became_active":"2023-10-06T05:21:55.123701+0000","last_became_peered":"2023-10-06T05:21:55.123701+0000","last_unstale":"2023-10-06T05:21:55.128415+0000","last_undegraded":"2023-10-06T05:21:55.128415+0000","last_fullsized":"2023-10-06T05:21:55.128415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:15.861250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.094761+0000","last_change":"2023-10-06T05:21:54.102840+0000","last_active":"2023-10-06T05:21:55.094761+0000","last_peered":"2023-10-06T05:21:55.094761+0000","last_clean":"2023-10-06T05:21:55.094761+0000","last_became_active":"2023-10-06T05:21:54.102563+0000","last_became_peered":"2023-10-06T05:21:54.102563+0000","last_unstale":"2023-10-06T05:21:55.094761+0000","last_undegraded":"2023-10-06T05:21:55.094761+0000","last_fullsized":"2023-10-06T05:21:55.094761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:58.906754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.120868+0000","last_change":"2023-10-06T05:21:55.118589+0000","last_active":"2023-10-06T05:21:55.120868+0000","last_peered":"2023-10-06T05:21:55.120868+0000","last_clean":"2023-10-06T05:21:55.120868+0000","last_became_active":"2023-10-06T05:21:55.118312+0000","last_became_peered":"2023-10-06T05:21:55.118312+0000","last_unstale":"2023-10-06T05:21:55.120868+0000","last_undegraded":"2023-10-06T05:21:55.120868+0000","last_fullsized":"2023-10-06T05:21:55.120868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:50.701910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"82'3","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.124737+0000","last_change":"2023-10-06T05:21:54.112396+0000","last_active":"2023-10-06T05:21:55.124737+0000","last_peered":"2023-10-06T05:21:55.124737+0000","last_clean":"2023-10-06T05:21:55.124737+0000","last_became_active":"2023-10-06T05:21:54.112063+0000","last_became_peered":"2023-10-06T05:21:54.112063+0000","last_unstale":"2023-10-06T05:21:55.124737+0000","last_undegraded":"2023-10-06T05:21:55.124737+0000","last_fullsized":"2023-10-06T05:21:55.124737+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:49.825750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.116513+0000","last_change":"2023-10-06T05:21:55.107434+0000","last_active":"2023-10-06T05:21:55.116513+0000","last_peered":"2023-10-06T05:21:55.116513+0000","last_clean":"2023-10-06T05:21:55.116513+0000","last_became_active":"2023-10-06T05:21:55.107087+0000","last_became_peered":"2023-10-06T05:21:55.107087+0000","last_unstale":"2023-10-06T05:21:55.116513+0000","last_undegraded":"2023-10-06T05:21:55.116513+0000","last_fullsized":"2023-10-06T05:21:55.116513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:00.508009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"82'6","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.124656+0000","last_change":"2023-10-06T05:21:54.111052+0000","last_active":"2023-10-06T05:21:55.124656+0000","last_peered":"2023-10-06T05:21:55.124656+0000","last_clean":"2023-10-06T05:21:55.124656+0000","last_became_active":"2023-10-06T05:21:54.110744+0000","last_became_peered":"2023-10-06T05:21:54.110744+0000","last_unstale":"2023-10-06T05:21:55.124656+0000","last_undegraded":"2023-10-06T05:21:55.124656+0000","last_fullsized":"2023-10-06T05:21:55.124656+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:49.698916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.126260+0000","last_change":"2023-10-06T05:21:55.121058+0000","last_active":"2023-10-06T05:21:55.126260+0000","last_peered":"2023-10-06T05:21:55.126260+0000","last_clean":"2023-10-06T05:21:55.126260+0000","last_became_active":"2023-10-06T05:21:55.120247+0000","last_became_peered":"2023-10-06T05:21:55.120247+0000","last_unstale":"2023-10-06T05:21:55.126260+0000","last_undegraded":"2023-10-06T05:21:55.126260+0000","last_fullsized":"2023-10-06T05:21:55.126260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:11.288762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.127469+0000","last_change":"2023-10-06T05:21:54.112529+0000","last_active":"2023-10-06T05:21:55.127469+0000","last_peered":"2023-10-06T05:21:55.127469+0000","last_clean":"2023-10-06T05:21:55.127469+0000","last_became_active":"2023-10-06T05:21:54.112200+0000","last_became_peered":"2023-10-06T05:21:54.112200+0000","last_unstale":"2023-10-06T05:21:55.127469+0000","last_undegraded":"2023-10-06T05:21:55.127469+0000","last_fullsized":"2023-10-06T05:21:55.127469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:04.924821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.111916+0000","last_change":"2023-10-06T05:21:55.111916+0000","last_active":"2023-10-06T05:21:55.111916+0000","last_peered":"2023-10-06T05:21:55.111916+0000","last_clean":"2023-10-06T05:21:55.111916+0000","last_became_active":"2023-10-06T05:21:55.111635+0000","last_became_peered":"2023-10-06T05:21:55.111635+0000","last_unstale":"2023-10-06T05:21:55.111916+0000","last_undegraded":"2023-10-06T05:21:55.111916+0000","last_fullsized":"2023-10-06T05:21:55.111916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:02.549324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.112700+0000","last_change":"2023-10-06T05:21:54.111339+0000","last_active":"2023-10-06T05:21:54.112700+0000","last_peered":"2023-10-06T05:21:54.112700+0000","last_clean":"2023-10-06T05:21:54.112700+0000","last_became_active":"2023-10-06T05:21:54.111050+0000","last_became_peered":"2023-10-06T05:21:54.111050+0000","last_unstale":"2023-10-06T05:21:54.112700+0000","last_undegraded":"2023-10-06T05:21:54.112700+0000","last_fullsized":"2023-10-06T05:21:54.112700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:04.924821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109186+0000","last_change":"2023-10-06T05:21:55.109186+0000","last_active":"2023-10-06T05:21:55.109186+0000","last_peered":"2023-10-06T05:21:55.109186+0000","last_clean":"2023-10-06T05:21:55.109186+0000","last_became_active":"2023-10-06T05:21:55.108920+0000","last_became_peered":"2023-10-06T05:21:55.108920+0000","last_unstale":"2023-10-06T05:21:55.109186+0000","last_undegraded":"2023-10-06T05:21:55.109186+0000","last_fullsized":"2023-10-06T05:21:55.109186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:58.909995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.090009+0000","last_change":"2023-10-06T05:21:54.105966+0000","last_active":"2023-10-06T05:21:55.090009+0000","last_peered":"2023-10-06T05:21:55.090009+0000","last_clean":"2023-10-06T05:21:55.090009+0000","last_became_active":"2023-10-06T05:21:54.105674+0000","last_became_peered":"2023-10-06T05:21:54.105674+0000","last_unstale":"2023-10-06T05:21:55.090009+0000","last_undegraded":"2023-10-06T05:21:55.090009+0000","last_fullsized":"2023-10-06T05:21:55.090009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:45.082000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.118164+0000","last_change":"2023-10-06T05:21:55.118164+0000","last_active":"2023-10-06T05:21:55.118164+0000","last_peered":"2023-10-06T05:21:55.118164+0000","last_clean":"2023-10-06T05:21:55.118164+0000","last_became_active":"2023-10-06T05:21:55.117845+0000","last_became_peered":"2023-10-06T05:21:55.117845+0000","last_unstale":"2023-10-06T05:21:55.118164+0000","last_undegraded":"2023-10-06T05:21:55.118164+0000","last_fullsized":"2023-10-06T05:21:55.118164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:27.188799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"82'8","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.122777+0000","last_change":"2023-10-06T05:21:54.100274+0000","last_active":"2023-10-06T05:21:55.122777+0000","last_peered":"2023-10-06T05:21:55.122777+0000","last_clean":"2023-10-06T05:21:55.122777+0000","last_became_active":"2023-10-06T05:21:54.099993+0000","last_became_peered":"2023-10-06T05:21:54.099993+0000","last_unstale":"2023-10-06T05:21:55.122777+0000","last_undegraded":"2023-10-06T05:21:55.122777+0000","last_fullsized":"2023-10-06T05:21:55.122777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:21:03.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.119102+0000","last_change":"2023-10-06T05:21:55.119102+0000","last_active":"2023-10-06T05:21:55.119102+0000","last_peered":"2023-10-06T05:21:55.119102+0000","last_clean":"2023-10-06T05:21:55.119102+0000","last_became_active":"2023-10-06T05:21:55.115908+0000","last_became_peered":"2023-10-06T05:21:55.115908+0000","last_unstale":"2023-10-06T05:21:55.119102+0000","last_undegraded":"2023-10-06T05:21:55.119102+0000","last_fullsized":"2023-10-06T05:21:55.119102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:22.621796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.123844+0000","last_change":"2023-10-06T05:21:54.110852+0000","last_active":"2023-10-06T05:21:55.123844+0000","last_peered":"2023-10-06T05:21:55.123844+0000","last_clean":"2023-10-06T05:21:55.123844+0000","last_became_active":"2023-10-06T05:21:54.110477+0000","last_became_peered":"2023-10-06T05:21:54.110477+0000","last_unstale":"2023-10-06T05:21:55.123844+0000","last_undegraded":"2023-10-06T05:21:55.123844+0000","last_fullsized":"2023-10-06T05:21:55.123844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:02.896225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.125003+0000","last_change":"2023-10-06T05:21:55.125003+0000","last_active":"2023-10-06T05:21:55.125003+0000","last_peered":"2023-10-06T05:21:55.125003+0000","last_clean":"2023-10-06T05:21:55.125003+0000","last_became_active":"2023-10-06T05:21:55.124575+0000","last_became_peered":"2023-10-06T05:21:55.124575+0000","last_unstale":"2023-10-06T05:21:55.125003+0000","last_undegraded":"2023-10-06T05:21:55.125003+0000","last_fullsized":"2023-10-06T05:21:55.125003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:45:10.843465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.108334+0000","last_change":"2023-10-06T05:21:54.104130+0000","last_active":"2023-10-06T05:21:54.108334+0000","last_peered":"2023-10-06T05:21:54.108334+0000","last_clean":"2023-10-06T05:21:54.108334+0000","last_became_active":"2023-10-06T05:21:54.103814+0000","last_became_peered":"2023-10-06T05:21:54.103814+0000","last_unstale":"2023-10-06T05:21:54.108334+0000","last_undegraded":"2023-10-06T05:21:54.108334+0000","last_fullsized":"2023-10-06T05:21:54.108334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:18.101312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117668+0000","last_change":"2023-10-06T05:21:55.117668+0000","last_active":"2023-10-06T05:21:55.117668+0000","last_peered":"2023-10-06T05:21:55.117668+0000","last_clean":"2023-10-06T05:21:55.117668+0000","last_became_active":"2023-10-06T05:21:55.117387+0000","last_became_peered":"2023-10-06T05:21:55.117387+0000","last_unstale":"2023-10-06T05:21:55.117668+0000","last_undegraded":"2023-10-06T05:21:55.117668+0000","last_fullsized":"2023-10-06T05:21:55.117668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:21.413755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.110448+0000","last_change":"2023-10-06T05:21:54.107990+0000","last_active":"2023-10-06T05:21:54.110448+0000","last_peered":"2023-10-06T05:21:54.110448+0000","last_clean":"2023-10-06T05:21:54.110448+0000","last_became_active":"2023-10-06T05:21:54.107583+0000","last_became_peered":"2023-10-06T05:21:54.107583+0000","last_unstale":"2023-10-06T05:21:54.110448+0000","last_undegraded":"2023-10-06T05:21:54.110448+0000","last_fullsized":"2023-10-06T05:21:54.110448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:18.382108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.110138+0000","last_change":"2023-10-06T05:21:55.110138+0000","last_active":"2023-10-06T05:21:55.110138+0000","last_peered":"2023-10-06T05:21:55.110138+0000","last_clean":"2023-10-06T05:21:55.110138+0000","last_became_active":"2023-10-06T05:21:55.109834+0000","last_became_peered":"2023-10-06T05:21:55.109834+0000","last_unstale":"2023-10-06T05:21:55.110138+0000","last_undegraded":"2023-10-06T05:21:55.110138+0000","last_fullsized":"2023-10-06T05:21:55.110138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:22.541524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.109701+0000","last_change":"2023-10-06T05:21:54.104572+0000","last_active":"2023-10-06T05:21:54.109701+0000","last_peered":"2023-10-06T05:21:54.109701+0000","last_clean":"2023-10-06T05:21:54.109701+0000","last_became_active":"2023-10-06T05:21:54.104255+0000","last_became_peered":"2023-10-06T05:21:54.104255+0000","last_unstale":"2023-10-06T05:21:54.109701+0000","last_undegraded":"2023-10-06T05:21:54.109701+0000","last_fullsized":"2023-10-06T05:21:54.109701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:21.559202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.108087+0000","last_change":"2023-10-06T05:21:55.108087+0000","last_active":"2023-10-06T05:21:55.108087+0000","last_peered":"2023-10-06T05:21:55.108087+0000","last_clean":"2023-10-06T05:21:55.108087+0000","last_became_active":"2023-10-06T05:21:55.107783+0000","last_became_peered":"2023-10-06T05:21:55.107783+0000","last_unstale":"2023-10-06T05:21:55.108087+0000","last_undegraded":"2023-10-06T05:21:55.108087+0000","last_fullsized":"2023-10-06T05:21:55.108087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:50.561974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.112878+0000","last_change":"2023-10-06T05:21:54.111755+0000","last_active":"2023-10-06T05:21:54.112878+0000","last_peered":"2023-10-06T05:21:54.112878+0000","last_clean":"2023-10-06T05:21:54.112878+0000","last_became_active":"2023-10-06T05:21:54.111518+0000","last_became_peered":"2023-10-06T05:21:54.111518+0000","last_unstale":"2023-10-06T05:21:54.112878+0000","last_undegraded":"2023-10-06T05:21:54.112878+0000","last_fullsized":"2023-10-06T05:21:54.112878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:26.571172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.123535+0000","last_change":"2023-10-06T05:21:55.109079+0000","last_active":"2023-10-06T05:21:55.123535+0000","last_peered":"2023-10-06T05:21:55.123535+0000","last_clean":"2023-10-06T05:21:55.123535+0000","last_became_active":"2023-10-06T05:21:55.108562+0000","last_became_peered":"2023-10-06T05:21:55.108562+0000","last_unstale":"2023-10-06T05:21:55.123535+0000","last_undegraded":"2023-10-06T05:21:55.123535+0000","last_fullsized":"2023-10-06T05:21:55.123535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:27.574167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.110347+0000","last_change":"2023-10-06T05:21:54.105835+0000","last_active":"2023-10-06T05:21:54.110347+0000","last_peered":"2023-10-06T05:21:54.110347+0000","last_clean":"2023-10-06T05:21:54.110347+0000","last_became_active":"2023-10-06T05:21:54.105572+0000","last_became_peered":"2023-10-06T05:21:54.105572+0000","last_unstale":"2023-10-06T05:21:54.110347+0000","last_undegraded":"2023-10-06T05:21:54.110347+0000","last_fullsized":"2023-10-06T05:21:54.110347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:51.369870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.113956+0000","last_change":"2023-10-06T05:21:55.113956+0000","last_active":"2023-10-06T05:21:55.113956+0000","last_peered":"2023-10-06T05:21:55.113956+0000","last_clean":"2023-10-06T05:21:55.113956+0000","last_became_active":"2023-10-06T05:21:55.113590+0000","last_became_peered":"2023-10-06T05:21:55.113590+0000","last_unstale":"2023-10-06T05:21:55.113956+0000","last_undegraded":"2023-10-06T05:21:55.113956+0000","last_fullsized":"2023-10-06T05:21:55.113956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:19.194349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.109821+0000","last_change":"2023-10-06T05:21:54.104657+0000","last_active":"2023-10-06T05:21:54.109821+0000","last_peered":"2023-10-06T05:21:54.109821+0000","last_clean":"2023-10-06T05:21:54.109821+0000","last_became_active":"2023-10-06T05:21:54.104347+0000","last_became_peered":"2023-10-06T05:21:54.104347+0000","last_unstale":"2023-10-06T05:21:54.109821+0000","last_undegraded":"2023-10-06T05:21:54.109821+0000","last_fullsized":"2023-10-06T05:21:54.109821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:20.176009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.113774+0000","last_change":"2023-10-06T05:21:55.113774+0000","last_active":"2023-10-06T05:21:55.113774+0000","last_peered":"2023-10-06T05:21:55.113774+0000","last_clean":"2023-10-06T05:21:55.113774+0000","last_became_active":"2023-10-06T05:21:55.113408+0000","last_became_peered":"2023-10-06T05:21:55.113408+0000","last_unstale":"2023-10-06T05:21:55.113774+0000","last_undegraded":"2023-10-06T05:21:55.113774+0000","last_fullsized":"2023-10-06T05:21:55.113774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:48.334561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117700+0000","last_change":"2023-10-06T05:21:54.104809+0000","last_active":"2023-10-06T05:21:55.117700+0000","last_peered":"2023-10-06T05:21:55.117700+0000","last_clean":"2023-10-06T05:21:55.117700+0000","last_became_active":"2023-10-06T05:21:54.104495+0000","last_became_peered":"2023-10-06T05:21:54.104495+0000","last_unstale":"2023-10-06T05:21:55.117700+0000","last_undegraded":"2023-10-06T05:21:55.117700+0000","last_fullsized":"2023-10-06T05:21:55.117700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:23.320432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.118653+0000","last_change":"2023-10-06T05:21:55.118653+0000","last_active":"2023-10-06T05:21:55.118653+0000","last_peered":"2023-10-06T05:21:55.118653+0000","last_clean":"2023-10-06T05:21:55.118653+0000","last_became_active":"2023-10-06T05:21:55.118238+0000","last_became_peered":"2023-10-06T05:21:55.118238+0000","last_unstale":"2023-10-06T05:21:55.118653+0000","last_undegraded":"2023-10-06T05:21:55.118653+0000","last_fullsized":"2023-10-06T05:21:55.118653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:58.734374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.111316+0000","last_change":"2023-10-06T05:21:54.109392+0000","last_active":"2023-10-06T05:21:54.111316+0000","last_peered":"2023-10-06T05:21:54.111316+0000","last_clean":"2023-10-06T05:21:54.111316+0000","last_became_active":"2023-10-06T05:21:54.109090+0000","last_became_peered":"2023-10-06T05:21:54.109090+0000","last_unstale":"2023-10-06T05:21:54.111316+0000","last_undegraded":"2023-10-06T05:21:54.111316+0000","last_fullsized":"2023-10-06T05:21:54.111316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:10.285768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.093043+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:21:54.093043+0000","last_peered":"2023-10-06T05:21:54.093043+0000","last_clean":"2023-10-06T05:21:54.093043+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:21:54.093043+0000","last_undegraded":"2023-10-06T05:21:54.093043+0000","last_fullsized":"2023-10-06T05:21:54.093043+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.106438+0000","last_change":"2023-10-06T05:21:55.106438+0000","last_active":"2023-10-06T05:21:55.106438+0000","last_peered":"2023-10-06T05:21:55.106438+0000","last_clean":"2023-10-06T05:21:55.106438+0000","last_became_active":"2023-10-06T05:21:55.106172+0000","last_became_peered":"2023-10-06T05:21:55.106172+0000","last_unstale":"2023-10-06T05:21:55.106438+0000","last_undegraded":"2023-10-06T05:21:55.106438+0000","last_fullsized":"2023-10-06T05:21:55.106438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:17.834770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.102789+0000","last_change":"2023-10-06T05:21:54.100165+0000","last_active":"2023-10-06T05:21:54.102789+0000","last_peered":"2023-10-06T05:21:54.102789+0000","last_clean":"2023-10-06T05:21:54.102789+0000","last_became_active":"2023-10-06T05:21:54.099888+0000","last_became_peered":"2023-10-06T05:21:54.099888+0000","last_unstale":"2023-10-06T05:21:54.102789+0000","last_undegraded":"2023-10-06T05:21:54.102789+0000","last_fullsized":"2023-10-06T05:21:54.102789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:52.906906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.094375+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:21:55.094375+0000","last_peered":"2023-10-06T05:21:55.094375+0000","last_clean":"2023-10-06T05:21:55.094375+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:21:55.094375+0000","last_undegraded":"2023-10-06T05:21:55.094375+0000","last_fullsized":"2023-10-06T05:21:55.094375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117460+0000","last_change":"2023-10-06T05:21:55.117460+0000","last_active":"2023-10-06T05:21:55.117460+0000","last_peered":"2023-10-06T05:21:55.117460+0000","last_clean":"2023-10-06T05:21:55.117460+0000","last_became_active":"2023-10-06T05:21:55.117182+0000","last_became_peered":"2023-10-06T05:21:55.117182+0000","last_unstale":"2023-10-06T05:21:55.117460+0000","last_undegraded":"2023-10-06T05:21:55.117460+0000","last_fullsized":"2023-10-06T05:21:55.117460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:43.984677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"82'1","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.104867+0000","last_change":"2023-10-06T05:21:54.109419+0000","last_active":"2023-10-06T05:21:55.104867+0000","last_peered":"2023-10-06T05:21:55.104867+0000","last_clean":"2023-10-06T05:21:55.104867+0000","last_became_active":"2023-10-06T05:21:54.109036+0000","last_became_peered":"2023-10-06T05:21:54.109036+0000","last_unstale":"2023-10-06T05:21:55.104867+0000","last_undegraded":"2023-10-06T05:21:55.104867+0000","last_fullsized":"2023-10-06T05:21:55.104867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:02.886880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.090842+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:21:55.090842+0000","last_peered":"2023-10-06T05:21:55.090842+0000","last_clean":"2023-10-06T05:21:55.090842+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:21:55.090842+0000","last_undegraded":"2023-10-06T05:21:55.090842+0000","last_fullsized":"2023-10-06T05:21:55.090842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":252,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117791+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:21:55.117791+0000","last_peered":"2023-10-06T05:21:55.117791+0000","last_clean":"2023-10-06T05:21:55.117791+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:21:55.117791+0000","last_undegraded":"2023-10-06T05:21:55.117791+0000","last_fullsized":"2023-10-06T05:21:55.117791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.118520+0000","last_change":"2023-10-06T05:21:55.118520+0000","last_active":"2023-10-06T05:21:55.118520+0000","last_peered":"2023-10-06T05:21:55.118520+0000","last_clean":"2023-10-06T05:21:55.118520+0000","last_became_active":"2023-10-06T05:21:55.118204+0000","last_became_peered":"2023-10-06T05:21:55.118204+0000","last_unstale":"2023-10-06T05:21:55.118520+0000","last_undegraded":"2023-10-06T05:21:55.118520+0000","last_fullsized":"2023-10-06T05:21:55.118520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:17.791265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"82'2","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.120682+0000","last_change":"2023-10-06T05:21:54.105846+0000","last_active":"2023-10-06T05:21:55.120682+0000","last_peered":"2023-10-06T05:21:55.120682+0000","last_clean":"2023-10-06T05:21:55.120682+0000","last_became_active":"2023-10-06T05:21:54.105572+0000","last_became_peered":"2023-10-06T05:21:54.105572+0000","last_unstale":"2023-10-06T05:21:55.120682+0000","last_undegraded":"2023-10-06T05:21:55.120682+0000","last_fullsized":"2023-10-06T05:21:55.120682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:26.571172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.109371+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:21:55.109371+0000","last_peered":"2023-10-06T05:21:55.109371+0000","last_clean":"2023-10-06T05:21:55.109371+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:21:55.109371+0000","last_undegraded":"2023-10-06T05:21:55.109371+0000","last_fullsized":"2023-10-06T05:21:55.109371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.132069+0000","last_change":"2023-10-06T05:21:55.129506+0000","last_active":"2023-10-06T05:21:55.132069+0000","last_peered":"2023-10-06T05:21:55.132069+0000","last_clean":"2023-10-06T05:21:55.132069+0000","last_became_active":"2023-10-06T05:21:55.129219+0000","last_became_peered":"2023-10-06T05:21:55.129219+0000","last_unstale":"2023-10-06T05:21:55.132069+0000","last_undegraded":"2023-10-06T05:21:55.132069+0000","last_fullsized":"2023-10-06T05:21:55.132069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:13.432899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.091306+0000","last_change":"2023-10-06T05:21:54.107130+0000","last_active":"2023-10-06T05:21:55.091306+0000","last_peered":"2023-10-06T05:21:55.091306+0000","last_clean":"2023-10-06T05:21:55.091306+0000","last_became_active":"2023-10-06T05:21:54.106816+0000","last_became_peered":"2023-10-06T05:21:54.106816+0000","last_unstale":"2023-10-06T05:21:55.091306+0000","last_undegraded":"2023-10-06T05:21:55.091306+0000","last_fullsized":"2023-10-06T05:21:55.091306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:53.371705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.117874+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:21:55.117874+0000","last_peered":"2023-10-06T05:21:55.117874+0000","last_clean":"2023-10-06T05:21:55.117874+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:21:55.117874+0000","last_undegraded":"2023-10-06T05:21:55.117874+0000","last_fullsized":"2023-10-06T05:21:55.117874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.105219+0000","last_change":"2023-10-06T05:21:55.105219+0000","last_active":"2023-10-06T05:21:55.105219+0000","last_peered":"2023-10-06T05:21:55.105219+0000","last_clean":"2023-10-06T05:21:55.105219+0000","last_became_active":"2023-10-06T05:21:55.103917+0000","last_became_peered":"2023-10-06T05:21:55.103917+0000","last_unstale":"2023-10-06T05:21:55.105219+0000","last_undegraded":"2023-10-06T05:21:55.105219+0000","last_fullsized":"2023-10-06T05:21:55.105219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:39.207736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"82'1","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.094742+0000","last_change":"2023-10-06T05:21:54.105918+0000","last_active":"2023-10-06T05:21:55.094742+0000","last_peered":"2023-10-06T05:21:55.094742+0000","last_clean":"2023-10-06T05:21:55.094742+0000","last_became_active":"2023-10-06T05:21:54.105521+0000","last_became_peered":"2023-10-06T05:21:54.105521+0000","last_unstale":"2023-10-06T05:21:55.094742+0000","last_undegraded":"2023-10-06T05:21:55.094742+0000","last_fullsized":"2023-10-06T05:21:55.094742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:21.849334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.096244+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:21:54.096244+0000","last_peered":"2023-10-06T05:21:54.096244+0000","last_clean":"2023-10-06T05:21:54.096244+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:21:54.096244+0000","last_undegraded":"2023-10-06T05:21:54.096244+0000","last_fullsized":"2023-10-06T05:21:54.096244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.107671+0000","last_change":"2023-10-06T05:21:55.107671+0000","last_active":"2023-10-06T05:21:55.107671+0000","last_peered":"2023-10-06T05:21:55.107671+0000","last_clean":"2023-10-06T05:21:55.107671+0000","last_became_active":"2023-10-06T05:21:55.106299+0000","last_became_peered":"2023-10-06T05:21:55.106299+0000","last_unstale":"2023-10-06T05:21:55.107671+0000","last_undegraded":"2023-10-06T05:21:55.107671+0000","last_fullsized":"2023-10-06T05:21:55.107671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:25.622184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.103833+0000","last_change":"2023-10-06T05:21:54.101103+0000","last_active":"2023-10-06T05:21:54.103833+0000","last_peered":"2023-10-06T05:21:54.103833+0000","last_clean":"2023-10-06T05:21:54.103833+0000","last_became_active":"2023-10-06T05:21:54.100815+0000","last_became_peered":"2023-10-06T05:21:54.100815+0000","last_unstale":"2023-10-06T05:21:54.103833+0000","last_undegraded":"2023-10-06T05:21:54.103833+0000","last_fullsized":"2023-10-06T05:21:54.103833+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:34.404148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.097874+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:21:54.097874+0000","last_peered":"2023-10-06T05:21:54.097874+0000","last_clean":"2023-10-06T05:21:54.097874+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:21:54.097874+0000","last_undegraded":"2023-10-06T05:21:54.097874+0000","last_fullsized":"2023-10-06T05:21:54.097874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.113708+0000","last_change":"2023-10-06T05:21:55.113708+0000","last_active":"2023-10-06T05:21:55.113708+0000","last_peered":"2023-10-06T05:21:55.113708+0000","last_clean":"2023-10-06T05:21:55.113708+0000","last_became_active":"2023-10-06T05:21:55.113346+0000","last_became_peered":"2023-10-06T05:21:55.113346+0000","last_unstale":"2023-10-06T05:21:55.113708+0000","last_undegraded":"2023-10-06T05:21:55.113708+0000","last_fullsized":"2023-10-06T05:21:55.113708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:54.079464+0000","last_clean_scrub_stamp":"2023-10-06T05:21:54.079464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:55.274486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"82'2","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-06T05:21:55.114192+0000","last_change":"2023-10-06T05:21:54.105131+0000","last_active":"2023-10-06T05:21:55.114192+0000","last_peered":"2023-10-06T05:21:55.114192+0000","last_clean":"2023-10-06T05:21:55.114192+0000","last_became_active":"2023-10-06T05:21:54.104823+0000","last_became_peered":"2023-10-06T05:21:54.104823+0000","last_unstale":"2023-10-06T05:21:55.114192+0000","last_undegraded":"2023-10-06T05:21:55.114192+0000","last_fullsized":"2023-10-06T05:21:55.114192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:21:53.076470+0000","last_clean_scrub_stamp":"2023-10-06T05:21:53.076470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:31.398533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-06T05:21:54.098007+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:21:54.098007+0000","last_peered":"2023-10-06T05:21:54.098007+0000","last_clean":"2023-10-06T05:21:54.098007+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:21:54.098007+0000","last_undegraded":"2023-10-06T05:21:54.098007+0000","last_fullsized":"2023-10-06T05:21:54.098007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302956,"kb_used_data":1572,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068884,"statfs":{"total":96636764160,"available":96326537216,"internally_reserved":0,"allocated":1609728,"data_stored":1046798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.748}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.68200000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93600000000000005,"5min":1.018,"15min":1.018},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95999999999999996,"5min":1.056,"15min":1.056},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73399999999999999,"15min":0.72899999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67700000000000005,"15min":0.72099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0189999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.45400000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93300000000000005,"5min":0.999,"15min":0.999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78400000000000003,"15min":0.83799999999999997},"min":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0389999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.46600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84599999999999997,"15min":0.89200000000000002},"min":{"1min":0.626,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.85399999999999998},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0029999999999999,"5min":1.103,"15min":1.103},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298292,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073548,"statfs":{"total":96636764160,"available":96331313152,"internally_reserved":0,"allocated":1028096,"data_stored":458486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.47099999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89800000000000002,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86899999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90800000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78500000000000003,"5min":1.369,"15min":1.369},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.891,"15min":1.891},"last":0.624}]},{"osd":3,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.89800000000000002,"5min":1.347,"15min":1.347},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":1,"15min":1},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73499999999999999,"15min":0.78500000000000003},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.95499999999999996,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78200000000000003,"15min":0.82399999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.74399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.69599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85899999999999999,"5min":1.415,"15min":1.415},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77500000000000002,"15min":0.83099999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98199999999999998,"5min":2,"15min":2},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97999999999999998,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.78000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95999999999999996,"5min":1.46,"15min":1.46},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298868,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072972,"statfs":{"total":96636764160,"available":96330723328,"internally_reserved":0,"allocated":1617920,"data_stored":1053453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.379},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.59699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82499999999999996},"min":{"1min":0.63600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77800000000000002,"15min":0.753},"min":{"1min":0.621,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.052,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.82999999999999996,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.83499999999999996,"15min":0.82499999999999996},"min":{"1min":0.67100000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84199999999999997,"15min":0.82999999999999996},"min":{"1min":0.69299999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.087,"5min":1.103,"15min":1.103},"last":0.72499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298296,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073544,"statfs":{"total":96636764160,"available":96331309056,"internally_reserved":0,"allocated":1032192,"data_stored":458588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68300000000000005,"5min":1.042,"15min":1.042},"last":0.33100000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79700000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.438},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1519999999999999,"5min":1.242,"15min":1.242},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":1.198,"15min":1.198},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.77300000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1659999999999999,"5min":1.26,"15min":1.26},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":1.163,"15min":1.163},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.073,"5min":1.323,"15min":1.323},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.61699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.123,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.264,"15min":1.264},"last":0.745},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.04,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298272,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073568,"statfs":{"total":96636764160,"available":96331333632,"internally_reserved":0,"allocated":1007616,"data_stored":456374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.42999999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.437},{"interface":"front","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.31,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.46000000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46500000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.624,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.499,"15min":0.503},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72699999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68799999999999994,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.76300000000000001},"min":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93500000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90600000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77300000000000002,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96299999999999997,"5min":1.099,"15min":1.099},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78500000000000003,"15min":0.81100000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.90000000000000002,"5min":1.163,"15min":1.163},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.98799999999999999,"5min":1.21,"15min":1.21},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.022,"5min":1.042,"15min":1.042},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 05:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84399999999999997,"15min":0.872},"min":{"1min":0.64600000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0469999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.56499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298272,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073568,"statfs":{"total":96636764160,"available":96331333632,"internally_reserved":0,"allocated":1007616,"data_stored":456374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.44600000000000001,"15min":0.46200000000000002},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57199999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67500000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.624,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.496},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.748,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72299999999999998,"5min":1.014,"15min":1.014},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.145,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83299999999999996,"15min":0.83399999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1830000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.03,"5min":1.204,"15min":1.204},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.61199999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.329},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.82099999999999995},"min":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.048,"5min":1.484,"15min":1.484},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83499999999999996,"15min":0.85399999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1930000000000001,"5min":1.629,"15min":1.629},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.83999999999999997,"15min":0.86099999999999999},"min":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1950000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.59499999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.085,"5min":1.325,"15min":1.325},"last":0.82299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298280,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073560,"statfs":{"total":96636764160,"available":96331325440,"internally_reserved":0,"allocated":1015808,"data_stored":456534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67300000000000004,"15min":0.68000000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74399999999999999,"15min":0.79800000000000004},"min":{"1min":0.52100000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":2.77,"15min":2.77},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92600000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.625}]},{"osd":2,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.76600000000000001,"15min":0.82999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89600000000000002,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.58599999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.90200000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.90500000000000003,"15min":1.091},"min":{"1min":0.50800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.93999999999999995,"5min":7.742,"15min":7.742},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.74199999999999999},"min":{"1min":0.48999999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96299999999999997,"5min":1.079,"15min":1.079},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90900000000000003,"5min":1.181,"15min":1.181},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71099999999999997,"15min":0.745},"min":{"1min":0.56499999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.60599999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.92200000000000004,"5min":1.171,"15min":1.171},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.624,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.96499999999999997,"5min":1.228,"15min":1.228},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81499999999999995,"15min":0.86899999999999999},"min":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97399999999999998,"5min":1.571,"15min":1.571},"last":0.91700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298272,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073568,"statfs":{"total":96636764160,"available":96331333632,"internally_reserved":0,"allocated":1007616,"data_stored":456374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.83099999999999996,"15min":1},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.438,"5min":0.434,"15min":0.434},"max":{"1min":0.998,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.85499999999999998,"15min":1.0209999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.079,"5min":9.25,"15min":9.25},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.90200000000000002,"15min":1.091},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.76000000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.875,"15min":1.0620000000000001},"min":{"1min":0.49199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0189999999999999,"5min":10.384,"15min":10.384},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.90400000000000003,"15min":1.0760000000000001},"min":{"1min":0.49299999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0349999999999999,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":1.1240000000000001},"min":{"1min":0.54800000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.95199999999999996,"5min":10.205,"15min":10.205},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96399999999999997,"5min":1.292,"15min":1.292},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.048,"5min":1.171,"15min":1.171},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.78400000000000003,"15min":0.81699999999999995},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.298,"15min":1.298},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80300000000000005,"15min":0.82899999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98999999999999999,"5min":1.204,"15min":1.204},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.81999999999999995},"min":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90200000000000002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:21:59.990 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:21:59.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:22:00.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.225+0000 7fad407ff700 1 Processor -- start 2023-10-06T05:22:00.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.225+0000 7fad407ff700 1 -- start start 2023-10-06T05:22:00.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.227+0000 7fad407ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fad38153d40 0x7fad380b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.227+0000 7fad407ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad380b0e40 con 0x7fad380b09c0 2023-10-06T05:22:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.227+0000 7fad407ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad380afc90 con 0x7fad380af890 2023-10-06T05:22:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.227+0000 7fad407ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad380ad690 con 0x7fad38153d40 2023-10-06T05:22:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.227+0000 7fad3dd9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fad380b09c0 0x7fad380af180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59566/0 (socket says 172.21.15.31:59566) 2023-10-06T05:22:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.227+0000 7fad3dd9a700 1 -- 172.21.15.31:0/1228207944 learned_addr learned my addr 172.21.15.31:0/1228207944 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.227+0000 7fad3e59b700 1 --2- 172.21.15.31:0/1228207944 >> v2:172.21.15.62:3300/0 conn(0x7fad38153d40 0x7fad380b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.227+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70948613 0 0) 0x7fad28001110 con 0x7fad380b09c0 2023-10-06T05:22:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.227+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad24003570 con 0x7fad380b09c0 2023-10-06T05:22:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223321720 0 0) 0x7fad30001490 con 0x7fad380af890 2023-10-06T05:22:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad28001110 con 0x7fad380af890 2023-10-06T05:22:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 530222494 0 0) 0x7fad24003570 con 0x7fad380b09c0 2023-10-06T05:22:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 >> v2:172.21.15.62:3300/0 conn(0x7fad38153d40 msgr2=0x7fad380b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 --2- 172.21.15.31:0/1228207944 >> v2:172.21.15.62:3300/0 conn(0x7fad38153d40 0x7fad380b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 >> v1:172.21.15.62:6789/0 conn(0x7fad380af890 legacy=0x7fad380adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad380acd60 con 0x7fad380b09c0 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fad28001600 con 0x7fad380b09c0 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 >> v1:172.21.15.31:6789/0 conn(0x7fad380b09c0 legacy=0x7fad380af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad24003c40 0x7fad24004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad24004810 0x7fad24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad24005210 0x7fad2400dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380ad690 con 0x7fad24003c40 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2400e340 con 0x7fad24005210 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2400e4d0 con 0x7fad24004810 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3e59b700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad24003c40 0x7fad24004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3dd9a700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad24004810 0x7fad24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.228+0000 7fad3ed9c700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad24005210 0x7fad2400dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.229+0000 7fad3e59b700 1 -- 172.21.15.31:0/1228207944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad24004810 msgr2=0x7fad24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.229+0000 7fad3e59b700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad24004810 0x7fad24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.229+0000 7fad3e59b700 1 -- 172.21.15.31:0/1228207944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad24005210 msgr2=0x7fad2400dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.229+0000 7fad3e59b700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad24005210 0x7fad2400dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.229+0000 7fad3e59b700 1 -- 172.21.15.31:0/1228207944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad24003570 con 0x7fad24003c40 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.229+0000 7fad3ed9c700 1 -- 172.21.15.31:0/1228207944 reap_dead start 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.229+0000 7fad3e59b700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad24003c40 0x7fad24004030 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fad34009c30 tx=0x7fad340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b1f4561d7950ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.229+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3400e710 con 0x7fad24003c40 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.229+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3400e870 con 0x7fad24003c40 2023-10-06T05:22:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.229+0000 7fad3d599700 1 -- 172.21.15.31:0/1228207944 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3400ec30 con 0x7fad24003c40 2023-10-06T05:22:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.230+0000 7fad407ff700 1 -- 172.21.15.31:0/1228207944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad24003c40 msgr2=0x7fad24004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.230+0000 7fad407ff700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad24003c40 0x7fad24004030 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fad34009c30 tx=0x7fad340091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.230+0000 7fad407ff700 1 -- 172.21.15.31:0/1228207944 shutdown_connections 2023-10-06T05:22:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.230+0000 7fad407ff700 1 --2- 172.21.15.31:0/1228207944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad24003c40 0x7fad24004030 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.230+0000 7fad407ff700 1 -- 172.21.15.31:0/1228207944 >> 172.21.15.31:0/1228207944 conn(0x7fad38067560 msgr2=0x7fad38067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.230+0000 7fad407ff700 1 -- 172.21.15.31:0/1228207944 shutdown_connections 2023-10-06T05:22:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.230+0000 7fad407ff700 1 -- 172.21.15.31:0/1228207944 wait complete. 2023-10-06T05:22:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.231+0000 7fad407ff700 1 Processor -- start 2023-10-06T05:22:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.231+0000 7fad407ff700 1 -- start start 2023-10-06T05:22:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.232+0000 7fad407ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad24003c40 0x7fad3814f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.232+0000 7fad407ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad380cbb70 0x7fad380c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.232+0000 7fad407ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3816a690 0x7fad381514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.232+0000 7fad407ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380ca8e0 con 0x7fad3816a690 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.232+0000 7fad407ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3814fcd0 con 0x7fad380cbb70 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.232+0000 7fad407ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad380cb6e0 con 0x7fad24003c40 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3816a690 0x7fad381514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3e59b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad24003c40 0x7fad3814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3dd9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad380cbb70 0x7fad380c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3816a690 0x7fad381514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56400/0 (socket says 172.21.15.31:56400) 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3ed9c700 1 -- 172.21.15.31:0/1631643053 learned_addr learned my addr 172.21.15.31:0/1631643053 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3ed9c700 1 -- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad24003c40 msgr2=0x7fad3814f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3ed9c700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad24003c40 0x7fad3814f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3ed9c700 1 -- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad380cbb70 msgr2=0x7fad380c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3ed9c700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad380cbb70 0x7fad380c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3ed9c700 1 -- 172.21.15.31:0/1631643053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad380cab10 con 0x7fad3816a690 2023-10-06T05:22:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3dd9a700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad380cbb70 0x7fad380c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.233+0000 7fad3ed9c700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3816a690 0x7fad381514e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fad300012e0 tx=0x7fad30009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.234+0000 7fad2e7fc700 1 -- 172.21.15.31:0/1631643053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30006db0 con 0x7fad3816a690 2023-10-06T05:22:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.234+0000 7fad2e7fc700 1 -- 172.21.15.31:0/1631643053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad30001870 con 0x7fad3816a690 2023-10-06T05:22:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.234+0000 7fad407ff700 1 -- 172.21.15.31:0/1631643053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad380cadb0 con 0x7fad3816a690 2023-10-06T05:22:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.234+0000 7fad2e7fc700 1 -- 172.21.15.31:0/1631643053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30001c30 con 0x7fad3816a690 2023-10-06T05:22:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.234+0000 7fad407ff700 1 -- 172.21.15.31:0/1631643053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad380cb370 con 0x7fad3816a690 2023-10-06T05:22:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.235+0000 7fad407ff700 1 -- 172.21.15.31:0/1631643053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad380cb870 con 0x7fad3816a690 2023-10-06T05:22:00.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.236+0000 7fad2e7fc700 1 -- 172.21.15.31:0/1631643053 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fad30003070 con 0x7fad3816a690 2023-10-06T05:22:00.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.237+0000 7fad2e7fc700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad1c067050 0x7fad1c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.237+0000 7fad2e7fc700 1 -- 172.21.15.31:0/1631643053 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fad30083e70 con 0x7fad3816a690 2023-10-06T05:22:00.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.238+0000 7fad3e59b700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad1c067050 0x7fad1c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.238+0000 7fad3e59b700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad1c067050 0x7fad1c069500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad3813f8e0 tx=0x7fad3401a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.241+0000 7fad2e7fc700 1 -- 172.21.15.31:0/1631643053 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad3004c5a0 con 0x7fad3816a690 2023-10-06T05:22:00.514 INFO:tasks.ceph.mds.d.smithi062.stderr:2023-10-06T05:22:00.512+0000 7f8042ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:22:00.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.527+0000 7fad407ff700 1 -- 172.21.15.31:0/1631643053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fad380afc90 con 0x7fad3816a690 2023-10-06T05:22:00.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.531+0000 7fad2e7fc700 1 -- 172.21.15.31:0/1631643053 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fad380afc90 con 0x7fad3816a690 2023-10-06T05:22:00.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 -- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad1c067050 msgr2=0x7fad1c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad1c067050 0x7fad1c069500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad3813f8e0 tx=0x7fad3401a000 comp rx=0 tx=0).stop 2023-10-06T05:22:00.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 -- 172.21.15.31:0/1631643053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3816a690 msgr2=0x7fad381514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3816a690 0x7fad381514e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fad300012e0 tx=0x7fad30009040 comp rx=0 tx=0).stop 2023-10-06T05:22:00.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 -- 172.21.15.31:0/1631643053 shutdown_connections 2023-10-06T05:22:00.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad1c067050 0x7fad1c069500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3816a690 0x7fad381514e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad24003c40 0x7fad3814f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 --2- 172.21.15.31:0/1631643053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad380cbb70 0x7fad380c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 -- 172.21.15.31:0/1631643053 >> 172.21.15.31:0/1631643053 conn(0x7fad38067560 msgr2=0x7fad38152a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:00.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 -- 172.21.15.31:0/1631643053 shutdown_connections 2023-10-06T05:22:00.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.534+0000 7fad407ff700 1 -- 172.21.15.31:0/1631643053 wait complete. 2023-10-06T05:22:00.538 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:22:00.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.746+0000 7f9e13dff700 1 Processor -- start 2023-10-06T05:22:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.746+0000 7f9e13dff700 1 -- start start 2023-10-06T05:22:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.747+0000 7f9e13dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9e0c159500 0x7f9e0c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.747+0000 7f9e13dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0c0c3f40 con 0x7f9e0c15d2d0 2023-10-06T05:22:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.747+0000 7f9e13dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0c0cb000 con 0x7f9e0c0cd510 2023-10-06T05:22:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.747+0000 7f9e13dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0c0b2860 con 0x7f9e0c159500 2023-10-06T05:22:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e1139a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9e0c15d2d0 0x7f9e0c0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59568/0 (socket says 172.21.15.31:59568) 2023-10-06T05:22:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e1139a700 1 -- 172.21.15.31:0/28455473 learned_addr learned my addr 172.21.15.31:0/28455473 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e11b9b700 1 --2- 172.21.15.31:0/28455473 >> v2:172.21.15.62:3300/0 conn(0x7f9e0c159500 0x7f9e0c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431896244 0 0) 0x7f9e0c0c3f40 con 0x7f9e0c15d2d0 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df8003570 con 0x7f9e0c15d2d0 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198681678 0 0) 0x7f9e0c0cb000 con 0x7f9e0c0cd510 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e0c0c3f40 con 0x7f9e0c0cd510 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034076377 0 0) 0x7f9df8003570 con 0x7f9e0c15d2d0 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 >> v2:172.21.15.62:3300/0 conn(0x7f9e0c159500 msgr2=0x7f9e0c0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 --2- 172.21.15.31:0/28455473 >> v2:172.21.15.62:3300/0 conn(0x7f9e0c159500 0x7f9e0c0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 >> v1:172.21.15.62:6789/0 conn(0x7f9e0c0cd510 legacy=0x7f9e0c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c15d7d0 con 0x7f9e0c15d2d0 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f9dfc0015f0 con 0x7f9e0c15d2d0 2023-10-06T05:22:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 >> v1:172.21.15.31:6789/0 conn(0x7f9e0c15d2d0 legacy=0x7f9e0c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.748+0000 7f9e10b99700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9df8003c40 0x7f9df8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e10b99700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df8004810 0x7f9df8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e10b99700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9df8005210 0x7f9df800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0b2860 con 0x7f9df8003c40 2023-10-06T05:22:00.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df800c1b0 con 0x7f9df8004810 2023-10-06T05:22:00.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df800c370 con 0x7f9df8005210 2023-10-06T05:22:00.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e11b9b700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9df8003c40 0x7f9df8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e1139a700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df8004810 0x7f9df8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e1239c700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9df8005210 0x7f9df800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e11b9b700 1 -- 172.21.15.31:0/28455473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9df8005210 msgr2=0x7f9df800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e11b9b700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9df8005210 0x7f9df800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e11b9b700 1 -- 172.21.15.31:0/28455473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df8004810 msgr2=0x7f9df8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e11b9b700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df8004810 0x7f9df8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e11b9b700 1 -- 172.21.15.31:0/28455473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8003570 con 0x7f9df8003c40 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.749+0000 7f9e1239c700 1 -- 172.21.15.31:0/28455473 reap_dead start 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.750+0000 7f9e11b9b700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9df8003c40 0x7f9df8004030 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f9e08006f00 tx=0x7f9e080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a82842073593ced server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.750+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800e710 con 0x7f9df8003c40 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.750+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0800e870 con 0x7f9df8003c40 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.750+0000 7f9e10b99700 1 -- 172.21.15.31:0/28455473 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800ec30 con 0x7f9df8003c40 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.750+0000 7f9e13dff700 1 -- 172.21.15.31:0/28455473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9df8003c40 msgr2=0x7f9df8004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.750+0000 7f9e13dff700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9df8003c40 0x7f9df8004030 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f9e08006f00 tx=0x7f9e080091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.750+0000 7f9e13dff700 1 -- 172.21.15.31:0/28455473 shutdown_connections 2023-10-06T05:22:00.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.750+0000 7f9e13dff700 1 --2- 172.21.15.31:0/28455473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9df8003c40 0x7f9df8004030 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.750+0000 7f9e13dff700 1 -- 172.21.15.31:0/28455473 >> 172.21.15.31:0/28455473 conn(0x7f9e0c06ec90 msgr2=0x7f9e0c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:00.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.750+0000 7f9e13dff700 1 -- 172.21.15.31:0/28455473 shutdown_connections 2023-10-06T05:22:00.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.751+0000 7f9e13dff700 1 -- 172.21.15.31:0/28455473 wait complete. 2023-10-06T05:22:00.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.752+0000 7f9e13dff700 1 Processor -- start 2023-10-06T05:22:00.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.752+0000 7f9e13dff700 1 -- start start 2023-10-06T05:22:00.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.753+0000 7f9e13dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df8003c40 0x7f9e0c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.753+0000 7f9e13dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e0c0c9670 0x7f9e0c0c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.753+0000 7f9e13dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e0c0c8040 0x7f9e0c0d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.753+0000 7f9e13dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0ca6f0 con 0x7f9e0c0c9670 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.753+0000 7f9e13dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0c85d0 con 0x7f9df8003c40 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.753+0000 7f9e13dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0c88b0 con 0x7f9e0c0c8040 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.753+0000 7f9e1139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e0c0c9670 0x7f9e0c0c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.753+0000 7f9e1139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e0c0c9670 0x7f9e0c0c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56422/0 (socket says 172.21.15.31:56422) 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.753+0000 7f9e1139a700 1 -- 172.21.15.31:0/1165789740 learned_addr learned my addr 172.21.15.31:0/1165789740 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e1239c700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e0c0c8040 0x7f9e0c0d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e11b9b700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df8003c40 0x7f9e0c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e1139a700 1 -- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e0c0c8040 msgr2=0x7f9e0c0d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e1139a700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e0c0c8040 0x7f9e0c0d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e1139a700 1 -- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df8003c40 msgr2=0x7f9e0c0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:00.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e1139a700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df8003c40 0x7f9e0c0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:00.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e1139a700 1 -- 172.21.15.31:0/1165789740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c0c8b90 con 0x7f9e0c0c9670 2023-10-06T05:22:00.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e1139a700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e0c0c9670 0x7f9e0c0c7b00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0012a0 tx=0x7f9dfc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:00.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e01ffb700 1 -- 172.21.15.31:0/1165789740 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0072e0 con 0x7f9e0c0c9670 2023-10-06T05:22:00.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e01ffb700 1 -- 172.21.15.31:0/1165789740 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc007460 con 0x7f9e0c0c9670 2023-10-06T05:22:00.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e01ffb700 1 -- 172.21.15.31:0/1165789740 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0057f0 con 0x7f9e0c0c9670 2023-10-06T05:22:00.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e13dff700 1 -- 172.21.15.31:0/1165789740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0c0c8e70 con 0x7f9e0c0c9670 2023-10-06T05:22:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.754+0000 7f9e13dff700 1 -- 172.21.15.31:0/1165789740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c0c7780 con 0x7f9e0c0c9670 2023-10-06T05:22:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.755+0000 7f9e13dff700 1 -- 172.21.15.31:0/1165789740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0c04c380 con 0x7f9e0c0c9670 2023-10-06T05:22:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.756+0000 7f9e01ffb700 1 -- 172.21.15.31:0/1165789740 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f9dfc005b60 con 0x7f9e0c0c9670 2023-10-06T05:22:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.757+0000 7f9e01ffb700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9df0067050 0x7f9df0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.757+0000 7f9e01ffb700 1 -- 172.21.15.31:0/1165789740 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9dfc084930 con 0x7f9e0c0c9670 2023-10-06T05:22:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.758+0000 7f9e11b9b700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9df0067050 0x7f9df0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.758+0000 7f9e11b9b700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9df0067050 0x7f9df0069500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9e080091e0 tx=0x7f9e0801a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:00.761+0000 7f9e01ffb700 1 -- 172.21.15.31:0/1165789740 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dfc051890 con 0x7f9e0c0c9670 2023-10-06T05:22:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.006+0000 7f9e13dff700 1 -- 172.21.15.31:0/1165789740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e0c0cade0 con 0x7f9e0c0c9670 2023-10-06T05:22:01.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.006+0000 7f9e01ffb700 1 -- 172.21.15.31:0/1165789740 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f9dfc054eb0 con 0x7f9e0c0c9670 2023-10-06T05:22:01.009 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:01.010 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1878570982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1878570982},{"type":"v1","addr":"172.21.15.62:6834","nonce":1878570982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6687,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/427382574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":427382574},{"type":"v1","addr":"172.21.15.31:6834","nonce":427382574}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6690,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4141055233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4141055233},{"type":"v1","addr":"172.21.15.31:6835","nonce":4141055233}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:54.590687+0000","modified":"2023-10-06T05:21:58.111036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"d","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3654179987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3654179987},{"type":"v1","addr":"172.21.15.62:6835","nonce":3654179987}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:01.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 -- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9df0067050 msgr2=0x7f9df0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9df0067050 0x7f9df0069500 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9e080091e0 tx=0x7f9e0801a000 comp rx=0 tx=0).stop 2023-10-06T05:22:01.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 -- 172.21.15.31:0/1165789740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e0c0c9670 msgr2=0x7f9e0c0c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e0c0c9670 0x7f9e0c0c7b00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0012a0 tx=0x7f9dfc009040 comp rx=0 tx=0).stop 2023-10-06T05:22:01.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 -- 172.21.15.31:0/1165789740 shutdown_connections 2023-10-06T05:22:01.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9df0067050 0x7f9df0069500 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e0c0c8040 0x7f9e0c0d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df8003c40 0x7f9e0c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 --2- 172.21.15.31:0/1165789740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e0c0c9670 0x7f9e0c0c7b00 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 -- 172.21.15.31:0/1165789740 >> 172.21.15.31:0/1165789740 conn(0x7f9e0c06ec90 msgr2=0x7f9e0c1565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:01.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 -- 172.21.15.31:0/1165789740 shutdown_connections 2023-10-06T05:22:01.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.009+0000 7f9e13dff700 1 -- 172.21.15.31:0/1165789740 wait complete. 2023-10-06T05:22:01.013 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 53 2023-10-06T05:22:01.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:21:54.590687+0000', 'modified': '2023-10-06T05:21:58.111036+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6702}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6702': {'gid': 6702, 'name': 'd', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/3654179987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 3654179987}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 3654179987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:22:01.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:22:01.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:22:01.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:01.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:01.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.642+0000 7fd2383ff700 1 Processor -- start 2023-10-06T05:22:01.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.642+0000 7fd2383ff700 1 -- start start 2023-10-06T05:22:01.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.644+0000 7fd2383ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd23013a1e0 0x7fd2300c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.644+0000 7fd2383ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd23013a5b0 0x7fd23006d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.644+0000 7fd2383ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2300c71b0 con 0x7fd2300c6c30 2023-10-06T05:22:01.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.644+0000 7fd2383ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2300c7490 con 0x7fd23013a1e0 2023-10-06T05:22:01.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.644+0000 7fd2383ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2300c7770 con 0x7fd23013a5b0 2023-10-06T05:22:01.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.644+0000 7fd23619b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd23013a1e0 0x7fd2300c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.644+0000 7fd23619b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd23013a1e0 0x7fd2300c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56426/0 (socket says 172.21.15.31:56426) 2023-10-06T05:22:01.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.644+0000 7fd23619b700 1 -- 172.21.15.31:0/2216663541 learned_addr learned my addr 172.21.15.31:0/2216663541 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:01.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.644+0000 7fd23599a700 1 --2- 172.21.15.31:0/2216663541 >> v2:172.21.15.62:3300/0 conn(0x7fd23013a5b0 0x7fd23006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.644+0000 7fd23619b700 1 -- 172.21.15.31:0/2216663541 >> v2:172.21.15.62:3300/0 conn(0x7fd23013a5b0 msgr2=0x7fd23006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.645+0000 7fd23619b700 1 --2- 172.21.15.31:0/2216663541 >> v2:172.21.15.62:3300/0 conn(0x7fd23013a5b0 0x7fd23006d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.645+0000 7fd23619b700 1 -- 172.21.15.31:0/2216663541 >> v1:172.21.15.62:6789/0 conn(0x7fd2300c6c30 legacy=0x7fd23015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.645+0000 7fd23619b700 1 -- 172.21.15.31:0/2216663541 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2300c7a50 con 0x7fd23013a1e0 2023-10-06T05:22:01.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.645+0000 7fd23619b700 1 --2- 172.21.15.31:0/2216663541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 0x7fd2300c66f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd23006d130 tx=0x7fd22c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f7e2047a505126 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.645+0000 7fd235199700 1 -- 172.21.15.31:0/2216663541 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c005c10 con 0x7fd23013a1e0 2023-10-06T05:22:01.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.645+0000 7fd235199700 1 -- 172.21.15.31:0/2216663541 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c005d70 con 0x7fd23013a1e0 2023-10-06T05:22:01.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.645+0000 7fd235199700 1 -- 172.21.15.31:0/2216663541 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c00b880 con 0x7fd23013a1e0 2023-10-06T05:22:01.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.646+0000 7fd2383ff700 1 -- 172.21.15.31:0/2216663541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 msgr2=0x7fd2300c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.646+0000 7fd2383ff700 1 --2- 172.21.15.31:0/2216663541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 0x7fd2300c66f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd23006d130 tx=0x7fd22c009220 comp rx=0 tx=0).stop 2023-10-06T05:22:01.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.646+0000 7fd2383ff700 1 -- 172.21.15.31:0/2216663541 shutdown_connections 2023-10-06T05:22:01.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.646+0000 7fd2383ff700 1 --2- 172.21.15.31:0/2216663541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 0x7fd2300c66f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.646+0000 7fd2383ff700 1 --2- 172.21.15.31:0/2216663541 >> v2:172.21.15.62:3300/0 conn(0x7fd23013a5b0 0x7fd23006d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.646+0000 7fd2383ff700 1 -- 172.21.15.31:0/2216663541 >> 172.21.15.31:0/2216663541 conn(0x7fd2300cc0b0 msgr2=0x7fd2300cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:01.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.646+0000 7fd2383ff700 1 -- 172.21.15.31:0/2216663541 shutdown_connections 2023-10-06T05:22:01.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.646+0000 7fd2383ff700 1 -- 172.21.15.31:0/2216663541 wait complete. 2023-10-06T05:22:01.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.647+0000 7fd2383ff700 1 Processor -- start 2023-10-06T05:22:01.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.648+0000 7fd2383ff700 1 -- start start 2023-10-06T05:22:01.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.648+0000 7fd2383ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2300c6c30 0x7fd23013c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd2383ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 0x7fd2300ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd2383ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd23013a5b0 0x7fd230169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd2383ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd230152f00 con 0x7fd23013a1e0 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd2383ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2300c69c0 con 0x7fd2300c6c30 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd2383ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2300caf30 con 0x7fd23013a5b0 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd23599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 0x7fd2300ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd23599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 0x7fd2300ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56432/0 (socket says 172.21.15.31:56432) 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd23599a700 1 -- 172.21.15.31:0/3782166282 learned_addr learned my addr 172.21.15.31:0/3782166282 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd23619b700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2300c6c30 0x7fd23013c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd23699c700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd23013a5b0 0x7fd230169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd23599a700 1 -- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd23013a5b0 msgr2=0x7fd230169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd23599a700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd23013a5b0 0x7fd230169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd23599a700 1 -- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2300c6c30 msgr2=0x7fd23013c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd23599a700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2300c6c30 0x7fd23013c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.649+0000 7fd23599a700 1 -- 172.21.15.31:0/3782166282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c009480 con 0x7fd23013a1e0 2023-10-06T05:22:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.650+0000 7fd23599a700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 0x7fd2300ca670 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd220006b00 tx=0x7fd220008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.650+0000 7fd2267fc700 1 -- 172.21.15.31:0/3782166282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2200125f0 con 0x7fd23013a1e0 2023-10-06T05:22:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.650+0000 7fd2267fc700 1 -- 172.21.15.31:0/3782166282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd220012770 con 0x7fd23013a1e0 2023-10-06T05:22:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.650+0000 7fd2267fc700 1 -- 172.21.15.31:0/3782166282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220012b30 con 0x7fd23013a1e0 2023-10-06T05:22:01.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.650+0000 7fd2383ff700 1 -- 172.21.15.31:0/3782166282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2300cb210 con 0x7fd23013a1e0 2023-10-06T05:22:01.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.650+0000 7fd2383ff700 1 -- 172.21.15.31:0/3782166282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2300cabb0 con 0x7fd23013a1e0 2023-10-06T05:22:01.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.652+0000 7fd2267fc700 1 -- 172.21.15.31:0/3782166282 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd220013480 con 0x7fd23013a1e0 2023-10-06T05:22:01.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.653+0000 7fd2267fc700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd214067050 0x7fd214069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.653+0000 7fd2267fc700 1 -- 172.21.15.31:0/3782166282 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd220084ba0 con 0x7fd23013a1e0 2023-10-06T05:22:01.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.653+0000 7fd23619b700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd214067050 0x7fd214069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.654+0000 7fd23619b700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd214067050 0x7fd214069500 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd230155290 tx=0x7fd22c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.655+0000 7fd246b82b80 1 Processor -- start 2023-10-06T05:22:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.655+0000 7fd246b82b80 1 -- start start 2023-10-06T05:22:01.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e706450 0x56394e706820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e70ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e70f4d0 0x56394e70f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd246b82b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56394e5d9310 con 0x56394e6ebf40 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd246b82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56394e616130 con 0x56394e70f4d0 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd246b82b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56394e613af0 con 0x56394e706450 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd23599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e70ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd23599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e70ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56436/0 (socket says 172.21.15.31:56436) 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd23599a700 1 -- 172.21.15.31:0/2321409274 learned_addr learned my addr 172.21.15.31:0/2321409274 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd23619b700 1 --2- 172.21.15.31:0/2321409274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e706450 0x56394e706820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.656+0000 7fd23699c700 1 --2- 172.21.15.31:0/2321409274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e70f4d0 0x56394e70f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.657+0000 7fd23599a700 1 -- 172.21.15.31:0/2321409274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e706450 msgr2=0x56394e706820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.657+0000 7fd23599a700 1 --2- 172.21.15.31:0/2321409274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e706450 0x56394e706820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.657+0000 7fd23599a700 1 -- 172.21.15.31:0/2321409274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e70f4d0 msgr2=0x56394e70f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.657+0000 7fd23599a700 1 --2- 172.21.15.31:0/2321409274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e70f4d0 0x56394e70f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.657+0000 7fd23599a700 1 -- 172.21.15.31:0/2321409274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56394e415790 con 0x56394e6ebf40 2023-10-06T05:22:01.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.657+0000 7fd23599a700 1 --2- 172.21.15.31:0/2321409274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e70ef90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd220051cc0 tx=0x7fd2200553b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd159ee15890b48a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.657+0000 7fd2137fe700 1 -- 172.21.15.31:0/2321409274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220058a10 con 0x56394e6ebf40 2023-10-06T05:22:01.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.657+0000 7fd2137fe700 1 -- 172.21.15.31:0/2321409274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22005c760 con 0x56394e6ebf40 2023-10-06T05:22:01.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.657+0000 7fd2137fe700 1 -- 172.21.15.31:0/2321409274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22001a310 con 0x56394e6ebf40 2023-10-06T05:22:01.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.658+0000 7fd246b82b80 1 -- 172.21.15.31:0/2321409274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 msgr2=0x56394e70ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.658+0000 7fd246b82b80 1 --2- 172.21.15.31:0/2321409274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e70ef90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd220051cc0 tx=0x7fd2200553b0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.658+0000 7fd246b82b80 1 -- 172.21.15.31:0/2321409274 shutdown_connections 2023-10-06T05:22:01.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.658+0000 7fd246b82b80 1 --2- 172.21.15.31:0/2321409274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e70f4d0 0x56394e70f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.658+0000 7fd246b82b80 1 --2- 172.21.15.31:0/2321409274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e706450 0x56394e706820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.658+0000 7fd246b82b80 1 --2- 172.21.15.31:0/2321409274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e70ef90 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.658+0000 7fd246b82b80 1 -- 172.21.15.31:0/2321409274 >> 172.21.15.31:0/2321409274 conn(0x56394e709880 msgr2=0x56394e709c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:01.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.658+0000 7fd246b82b80 1 -- 172.21.15.31:0/2321409274 shutdown_connections 2023-10-06T05:22:01.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.658+0000 7fd246b82b80 1 -- 172.21.15.31:0/2321409274 wait complete. 2023-10-06T05:22:01.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.659+0000 7fd246b82b80 1 -- start start 2023-10-06T05:22:01.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.659+0000 7fd246b82b80 1 Processor -- start 2023-10-06T05:22:01.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e7058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e706450 0x56394e6e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e70f4d0 0x56394e6c0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd246b82b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56394e60f8e0 con 0x56394e6ebf40 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd246b82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56394e5bb980 con 0x56394e70f4d0 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd246b82b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56394e5bb620 con 0x56394e706450 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd246b82b80 10 client.0 _ll_register_callbacks cb 0x56394e41ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd23619b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e7058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd23619b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e7058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56438/0 (socket says 172.21.15.31:56438) 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd23619b700 1 -- 172.21.15.31:0/869487962 learned_addr learned my addr 172.21.15.31:0/869487962 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd23599a700 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e706450 0x56394e6e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.660+0000 7fd23699c700 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e70f4d0 0x56394e6c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd23619b700 1 -- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e706450 msgr2=0x56394e6e6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd23619b700 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e706450 0x56394e6e6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd23619b700 1 -- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e70f4d0 msgr2=0x56394e6c0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd23619b700 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e70f4d0 0x56394e6c0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd23619b700 1 -- 172.21.15.31:0/869487962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2300cabb0 con 0x56394e6ebf40 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd23619b700 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e7058a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd2300cc010 tx=0x7fd22c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a4e073a2864d70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd210ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd210ff9700 1 -- 172.21.15.31:0/869487962 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c030070 con 0x56394e6ebf40 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd210ff9700 1 -- 172.21.15.31:0/869487962 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd22c029b60 con 0x56394e6ebf40 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd246b82b80 1 -- 172.21.15.31:0/869487962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56394e6b3d60 con 0x56394e6ebf40 2023-10-06T05:22:01.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd210ff9700 1 -- 172.21.15.31:0/869487962 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd22c021070 con 0x56394e6ebf40 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.661+0000 7fd246b82b80 10 client.6834 fetch_fsmap learned FSMap version 53 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.662+0000 7fd246b82b80 1 -- 172.21.15.31:0/869487962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x56394e5020a0 con 0x56394e6ebf40 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.662+0000 7fd210ff9700 1 -- 172.21.15.31:0/869487962 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56394e6b3d60 con 0x56394e6ebf40 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.662+0000 7fd210ff9700 1 -- 172.21.15.31:0/869487962 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 53) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd22c035450 con 0x56394e6ebf40 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.662+0000 7fd246b82b80 10 client.6834 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.662+0000 7fd246b82b80 10 client.6834 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.662+0000 7fd246b82b80 20 client.6834 populate_metadata read hostname 'smithi031' 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.662+0000 7fd246b82b80 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:6833/3654179987,v1:172.21.15.62:6835/3654179987] conn(0x56394e6e58f0 0x56394e6e5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.662+0000 7fd246b82b80 4 client.6834 mds_command: new command op to 6702 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.662+0000 7fd246b82b80 1 -- 172.21.15.31:0/869487962 --> [v2:172.21.15.62:6833/3654179987,v1:172.21.15.62:6835/3654179987] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56394e690b30 con 0x56394e6e58f0 2023-10-06T05:22:01.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.662+0000 7fd23599a700 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:6833/3654179987,v1:172.21.15.62:6835/3654179987] conn(0x56394e6e58f0 0x56394e6e5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.663+0000 7fd23599a700 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:6833/3654179987,v1:172.21.15.62:6835/3654179987] conn(0x56394e6e58f0 0x56394e6e5cc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe83235bad0423cc server_cookie=52917ad84fee799 in_seq=0 out_seq=0 2023-10-06T05:22:01.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.663+0000 7fd210ff9700 10 client.6834 ms_handle_connect on v2:172.21.15.62:6833/3654179987 2023-10-06T05:22:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd210ff9700 1 -- 172.21.15.31:0/869487962 <== mds.0 v2:172.21.15.62:6833/3654179987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56394e690b30 con 0x56394e6e58f0 2023-10-06T05:22:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd210ff9700 10 client.6834 handle_command_reply: tid=0 2023-10-06T05:22:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd246b82b80 1 client.6834 shutdown 2023-10-06T05:22:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd246b82b80 1 -- 172.21.15.31:0/869487962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 msgr2=0x56394e7058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd246b82b80 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e7058a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd2300cc010 tx=0x7fd22c021000 comp rx=0 tx=0).stop 2023-10-06T05:22:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd246b82b80 1 -- 172.21.15.31:0/869487962 shutdown_connections 2023-10-06T05:22:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd246b82b80 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e70f4d0 0x56394e6c0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd246b82b80 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:6833/3654179987,v1:172.21.15.62:6835/3654179987] conn(0x56394e6e58f0 0x56394e6e5cc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd246b82b80 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e6ebf40 0x56394e7058a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd246b82b80 1 --2- 172.21.15.31:0/869487962 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e706450 0x56394e6e6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd210ff9700 0 client.6834 ms_handle_reset on v2:172.21.15.62:6833/3654179987 2023-10-06T05:22:01.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.664+0000 7fd246b82b80 1 -- 172.21.15.31:0/869487962 >> 172.21.15.31:0/869487962 conn(0x56394e709880 msgr2=0x56394e6c2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:01.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.665+0000 7fd246b82b80 1 -- 172.21.15.31:0/869487962 shutdown_connections 2023-10-06T05:22:01.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.665+0000 7fd246b82b80 1 -- 172.21.15.31:0/869487962 wait complete. 2023-10-06T05:22:01.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.665+0000 7fd246b82b80 20 client.6834 trim_cache size 0 max 16384 2023-10-06T05:22:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.682+0000 7fd246b82b80 1 Processor -- start 2023-10-06T05:22:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.682+0000 7fd246b82b80 1 -- start start 2023-10-06T05:22:01.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.683+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e6ebf40 0x56394e6bc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.683+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 0x56394e6e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.683+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.683+0000 7fd246b82b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56394e4a71e0 con 0x56394e70f4d0 2023-10-06T05:22:01.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.683+0000 7fd246b82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56394e6bdba0 con 0x56394e706450 2023-10-06T05:22:01.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.683+0000 7fd246b82b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56394e6e9530 con 0x56394e6ebf40 2023-10-06T05:22:01.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.683+0000 7fd23699c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.683+0000 7fd23699c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6e73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56446/0 (socket says 172.21.15.31:56446) 2023-10-06T05:22:01.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.683+0000 7fd23699c700 1 -- 172.21.15.31:0/3895623360 learned_addr learned my addr 172.21.15.31:0/3895623360 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:01.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd23599a700 1 --2- 172.21.15.31:0/3895623360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 0x56394e6e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd23619b700 1 --2- 172.21.15.31:0/3895623360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e6ebf40 0x56394e6bc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd23699c700 1 -- 172.21.15.31:0/3895623360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e6ebf40 msgr2=0x56394e6bc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd23699c700 1 --2- 172.21.15.31:0/3895623360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e6ebf40 0x56394e6bc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd23699c700 1 -- 172.21.15.31:0/3895623360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 msgr2=0x56394e6e1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd23699c700 1 --2- 172.21.15.31:0/3895623360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 0x56394e6e1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd23699c700 1 -- 172.21.15.31:0/3895623360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56394e5de3c0 con 0x56394e70f4d0 2023-10-06T05:22:01.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd23599a700 1 --2- 172.21.15.31:0/3895623360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 0x56394e6e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:01.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd23699c700 1 --2- 172.21.15.31:0/3895623360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6e73f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fd228002da0 tx=0x7fd228001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d4dcef7e9712c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd213fff700 1 -- 172.21.15.31:0/3895623360 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228014dc0 con 0x56394e70f4d0 2023-10-06T05:22:01.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd213fff700 1 -- 172.21.15.31:0/3895623360 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22801f460 con 0x56394e70f4d0 2023-10-06T05:22:01.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.684+0000 7fd213fff700 1 -- 172.21.15.31:0/3895623360 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22801f820 con 0x56394e70f4d0 2023-10-06T05:22:01.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.685+0000 7fd246b82b80 1 -- 172.21.15.31:0/3895623360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 msgr2=0x56394e6e73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.685+0000 7fd246b82b80 1 --2- 172.21.15.31:0/3895623360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6e73f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fd228002da0 tx=0x7fd228001690 comp rx=0 tx=0).stop 2023-10-06T05:22:01.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.685+0000 7fd246b82b80 1 -- 172.21.15.31:0/3895623360 shutdown_connections 2023-10-06T05:22:01.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.685+0000 7fd246b82b80 1 --2- 172.21.15.31:0/3895623360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6e73f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.685+0000 7fd246b82b80 1 --2- 172.21.15.31:0/3895623360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e6ebf40 0x56394e6bc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.685+0000 7fd246b82b80 1 --2- 172.21.15.31:0/3895623360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 0x56394e6e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.685+0000 7fd246b82b80 1 -- 172.21.15.31:0/3895623360 >> 172.21.15.31:0/3895623360 conn(0x56394e709880 msgr2=0x56394e6bcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:01.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.685+0000 7fd246b82b80 1 -- 172.21.15.31:0/3895623360 shutdown_connections 2023-10-06T05:22:01.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.685+0000 7fd246b82b80 1 -- 172.21.15.31:0/3895623360 wait complete. 2023-10-06T05:22:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.686+0000 7fd246b82b80 1 -- start start 2023-10-06T05:22:01.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.686+0000 7fd246b82b80 1 Processor -- start 2023-10-06T05:22:01.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e6ebf40 0x56394e709c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 0x56394e70dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd246b82b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6bee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd246b82b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56394e765190 con 0x56394e70f4d0 2023-10-06T05:22:01.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd246b82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56394e70a660 con 0x56394e706450 2023-10-06T05:22:01.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd246b82b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56394e6e24d0 con 0x56394e6ebf40 2023-10-06T05:22:01.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd246b82b80 10 client.0 _ll_register_callbacks cb 0x56394e41ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:22:01.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd23699c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd23699c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6bee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56452/0 (socket says 172.21.15.31:56452) 2023-10-06T05:22:01.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd23699c700 1 -- 172.21.15.31:0/3756417231 learned_addr learned my addr 172.21.15.31:0/3756417231 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:01.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd23619b700 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e6ebf40 0x56394e709c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.687+0000 7fd23599a700 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 0x56394e70dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd23699c700 1 -- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e6ebf40 msgr2=0x56394e709c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd23699c700 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e6ebf40 0x56394e709c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd23699c700 1 -- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 msgr2=0x56394e70dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd23699c700 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 0x56394e70dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd23699c700 1 -- 172.21.15.31:0/3756417231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22800f040 con 0x56394e70f4d0 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd23599a700 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 0x56394e70dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd23699c700 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6bee20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd228004020 tx=0x7fd22801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db4d6b860284405 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd210ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd210ff9700 1 -- 172.21.15.31:0/3756417231 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228024070 con 0x56394e70f4d0 2023-10-06T05:22:01.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd246b82b80 1 -- 172.21.15.31:0/3756417231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd22005c760 con 0x56394e70f4d0 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.688+0000 7fd210ff9700 1 -- 172.21.15.31:0/3756417231 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd228001010 con 0x56394e70f4d0 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd210ff9700 1 -- 172.21.15.31:0/3756417231 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd228016420 con 0x56394e70f4d0 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd246b82b80 10 client.6840 fetch_fsmap learned FSMap version 53 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd246b82b80 1 -- 172.21.15.31:0/3756417231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x56394e4761d0 con 0x56394e70f4d0 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd210ff9700 1 -- 172.21.15.31:0/3756417231 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22005c760 con 0x56394e70f4d0 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd210ff9700 1 -- 172.21.15.31:0/3756417231 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 53) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd228016600 con 0x56394e70f4d0 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd246b82b80 10 client.6840 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd246b82b80 10 client.6840 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd246b82b80 20 client.6840 populate_metadata read hostname 'smithi031' 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd246b82b80 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:6833/3654179987,v1:172.21.15.62:6835/3654179987] conn(0x56394e761f90 0x56394e6bf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd246b82b80 4 client.6840 mds_command: new command op to 6702 tid=0[{"prefix": "status"}] 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd246b82b80 1 -- 172.21.15.31:0/3756417231 --> [v2:172.21.15.62:6833/3654179987,v1:172.21.15.62:6835/3654179987] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56394e5b28d0 con 0x56394e761f90 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.689+0000 7fd23619b700 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:6833/3654179987,v1:172.21.15.62:6835/3654179987] conn(0x56394e761f90 0x56394e6bf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd23619b700 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:6833/3654179987,v1:172.21.15.62:6835/3654179987] conn(0x56394e761f90 0x56394e6bf360 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=889e710a91a76437 server_cookie=6d0633f530631be5 in_seq=0 out_seq=0 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd210ff9700 10 client.6840 ms_handle_connect on v2:172.21.15.62:6833/3654179987 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd210ff9700 1 -- 172.21.15.31:0/3756417231 <== mds.0 v2:172.21.15.62:6833/3654179987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x56394e5b28d0 con 0x56394e761f90 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd210ff9700 10 client.6840 handle_command_reply: tid=0 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd246b82b80 1 client.6840 shutdown 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd246b82b80 1 -- 172.21.15.31:0/3756417231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 msgr2=0x56394e6bee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd246b82b80 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6bee20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd228004020 tx=0x7fd22801f000 comp rx=0 tx=0).stop 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd246b82b80 1 -- 172.21.15.31:0/3756417231 shutdown_connections 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd246b82b80 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:6833/3654179987,v1:172.21.15.62:6835/3654179987] conn(0x56394e761f90 0x56394e6bf360 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd246b82b80 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56394e70f4d0 0x56394e6bee20 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd246b82b80 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394e6ebf40 0x56394e709c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd246b82b80 1 --2- 172.21.15.31:0/3756417231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56394e706450 0x56394e70dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd210ff9700 0 client.6840 ms_handle_reset on v2:172.21.15.62:6833/3654179987 2023-10-06T05:22:01.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.690+0000 7fd246b82b80 1 -- 172.21.15.31:0/3756417231 >> 172.21.15.31:0/3756417231 conn(0x56394e709880 msgr2=0x56394e6e6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:01.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd246b82b80 1 -- 172.21.15.31:0/3756417231 shutdown_connections 2023-10-06T05:22:01.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd246b82b80 1 -- 172.21.15.31:0/3756417231 wait complete. 2023-10-06T05:22:01.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd246b82b80 20 client.6840 trim_cache size 0 max 16384 2023-10-06T05:22:01.701 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 6702, 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.6031482690000001, 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 53, 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 83, 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 83, 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.82595343 2023-10-06T05:22:01.702 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:22:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 -- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd214067050 msgr2=0x7fd214069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd214067050 0x7fd214069500 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd230155290 tx=0x7fd22c016000 comp rx=0 tx=0).stop 2023-10-06T05:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 -- 172.21.15.31:0/3782166282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 msgr2=0x7fd2300ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 0x7fd2300ca670 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd220006b00 tx=0x7fd220008040 comp rx=0 tx=0).stop 2023-10-06T05:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 -- 172.21.15.31:0/3782166282 shutdown_connections 2023-10-06T05:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd214067050 0x7fd214069500 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd23013a5b0 0x7fd230169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2300c6c30 0x7fd23013c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 --2- 172.21.15.31:0/3782166282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd23013a1e0 0x7fd2300ca670 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 -- 172.21.15.31:0/3782166282 >> 172.21.15.31:0/3782166282 conn(0x7fd2300cc0b0 msgr2=0x7fd23006d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.691+0000 7fd2383ff700 1 -- 172.21.15.31:0/3782166282 shutdown_connections 2023-10-06T05:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.692+0000 7fd2383ff700 1 -- 172.21.15.31:0/3782166282 wait complete. 2023-10-06T05:22:01.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:22:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.903+0000 7fdcfb3ff700 1 Processor -- start 2023-10-06T05:22:01.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.903+0000 7fdcfb3ff700 1 -- start start 2023-10-06T05:22:01.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.904+0000 7fdcfb3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf40cdfe0 0x7fdcf40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.904+0000 7fdcfb3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf40cf280 con 0x7fdcf4159580 2023-10-06T05:22:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.904+0000 7fdcfb3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf40ce670 con 0x7fdcf40ccc80 2023-10-06T05:22:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.904+0000 7fdcfb3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40ce450 con 0x7fdcf40cdfe0 2023-10-06T05:22:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcf919b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdcf4159580 0x7fdcf40cb9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59578/0 (socket says 172.21.15.31:59578) 2023-10-06T05:22:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcf919b700 1 -- 172.21.15.31:0/1920271343 learned_addr learned my addr 172.21.15.31:0/1920271343 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcf899a700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf40cdfe0 0x7fdcf40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187229744 0 0) 0x7fdcf40cf280 con 0x7fdcf4159580 2023-10-06T05:22:01.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce0003570 con 0x7fdcf4159580 2023-10-06T05:22:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926947709 0 0) 0x7fdcf40ce670 con 0x7fdcf40ccc80 2023-10-06T05:22:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf40cf280 con 0x7fdcf40ccc80 2023-10-06T05:22:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1225205137 0 0) 0x7fdce0003570 con 0x7fdcf4159580 2023-10-06T05:22:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf40cdfe0 msgr2=0x7fdcf40cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf40cdfe0 0x7fdcf40cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 >> v1:172.21.15.62:6789/0 conn(0x7fdcf40ccc80 legacy=0x7fdcf40cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf40ce950 con 0x7fdcf4159580 2023-10-06T05:22:01.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fdcf00015f0 con 0x7fdcf4159580 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 >> v1:172.21.15.31:6789/0 conn(0x7fdcf4159580 legacy=0x7fdcf40cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf40cdfe0 0x7fdce0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce00041d0 0x7fdce0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 0x7fdce00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40ce450 con 0x7fdce0004d50 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce000bbf0 con 0x7fdce00041d0 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.905+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce000bdb0 con 0x7fdcf40cdfe0 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.906+0000 7fdcf919b700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf40cdfe0 0x7fdce0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.906+0000 7fdcf999c700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 0x7fdce00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.906+0000 7fdcf899a700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce00041d0 0x7fdce0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.906+0000 7fdcf999c700 1 -- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf40cdfe0 msgr2=0x7fdce0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.906+0000 7fdcf999c700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf40cdfe0 0x7fdce0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.906+0000 7fdcf999c700 1 -- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce00041d0 msgr2=0x7fdce0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.906+0000 7fdcf999c700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdce00041d0 0x7fdce0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.906+0000 7fdcf999c700 1 -- 172.21.15.31:0/1920271343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce0003570 con 0x7fdce0004d50 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.906+0000 7fdcf919b700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf40cdfe0 0x7fdce0003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.906+0000 7fdcf999c700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 0x7fdce00051c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fdcec002160 tx=0x7fdcec0017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a58c367a9a72127 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.907+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec001af0 con 0x7fdce0004d50 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.907+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec00c980 con 0x7fdce0004d50 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.907+0000 7fdcebfff700 1 -- 172.21.15.31:0/1920271343 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00cd20 con 0x7fdce0004d50 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.907+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/1920271343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 msgr2=0x7fdce00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.907+0000 7fdcfb3ff700 1 --2- 172.21.15.31:0/1920271343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 0x7fdce00051c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fdcec002160 tx=0x7fdcec0017e0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.907+0000 7fdcf999c700 1 -- 172.21.15.31:0/1920271343 reap_dead start 2023-10-06T05:22:01.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.907+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/1920271343 shutdown_connections 2023-10-06T05:22:01.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.907+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/1920271343 >> 172.21.15.31:0/1920271343 conn(0x7fdcf4099710 msgr2=0x7fdcf406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:01.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.907+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/1920271343 shutdown_connections 2023-10-06T05:22:01.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.907+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/1920271343 wait complete. 2023-10-06T05:22:01.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.909+0000 7fdcfb3ff700 1 Processor -- start 2023-10-06T05:22:01.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.909+0000 7fdcfb3ff700 1 -- start start 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.910+0000 7fdcfb3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 0x7fdcf4157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.910+0000 7fdcfb3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf4158760 0x7fdcf4169570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.910+0000 7fdcfb3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf4169ab0 0x7fdcf4169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.910+0000 7fdcfb3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40c6a70 con 0x7fdce0004d50 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.910+0000 7fdcfb3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf416a470 con 0x7fdcf4169ab0 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.910+0000 7fdcfb3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40b0ed0 con 0x7fdcf4158760 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.910+0000 7fdcf919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 0x7fdcf4157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.910+0000 7fdcf919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 0x7fdcf4157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56474/0 (socket says 172.21.15.31:56474) 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.910+0000 7fdcf919b700 1 -- 172.21.15.31:0/3355632606 learned_addr learned my addr 172.21.15.31:0/3355632606 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdcf999c700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf4169ab0 0x7fdcf4169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdcf899a700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf4158760 0x7fdcf4169570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdcf919b700 1 -- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf4158760 msgr2=0x7fdcf4169570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdcf919b700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf4158760 0x7fdcf4169570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdcf919b700 1 -- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf4169ab0 msgr2=0x7fdcf4169f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdcf919b700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf4169ab0 0x7fdcf4169f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdcf919b700 1 -- 172.21.15.31:0/3355632606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf40c61d0 con 0x7fdce0004d50 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdcf919b700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 0x7fdcf4157eb0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0007b20 tx=0x7fdcf0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdce97fa700 1 -- 172.21.15.31:0/3355632606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000d9d0 con 0x7fdce0004d50 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdce97fa700 1 -- 172.21.15.31:0/3355632606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf000db30 con 0x7fdce0004d50 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/3355632606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf40c64b0 con 0x7fdce0004d50 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdce97fa700 1 -- 172.21.15.31:0/3355632606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000ded0 con 0x7fdce0004d50 2023-10-06T05:22:01.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.911+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/3355632606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf41583f0 con 0x7fdce0004d50 2023-10-06T05:22:01.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.912+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/3355632606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf40b0d30 con 0x7fdce0004d50 2023-10-06T05:22:01.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.913+0000 7fdce97fa700 1 -- 172.21.15.31:0/3355632606 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fdcf00072e0 con 0x7fdce0004d50 2023-10-06T05:22:01.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.914+0000 7fdce97fa700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcd8067050 0x7fdcd8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:01.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.915+0000 7fdce97fa700 1 -- 172.21.15.31:0/3355632606 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdcf0084b20 con 0x7fdce0004d50 2023-10-06T05:22:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.915+0000 7fdcf899a700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcd8067050 0x7fdcd8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:01.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.916+0000 7fdcf899a700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcd8067050 0x7fdcd8069500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40b2e80 tx=0x7fdce4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:01.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:01.919+0000 7fdce97fa700 1 -- 172.21.15.31:0/3355632606 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcf40b0d30 con 0x7fdce0004d50 2023-10-06T05:22:02.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.172+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/3355632606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdcf40b0d30 con 0x7fdce0004d50 2023-10-06T05:22:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.172+0000 7fdce97fa700 1 -- 172.21.15.31:0/3355632606 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdcf40b0d30 con 0x7fdce0004d50 2023-10-06T05:22:02.175 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:22:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcd8067050 msgr2=0x7fdcd8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcd8067050 0x7fdcd8069500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40b2e80 tx=0x7fdce4008040 comp rx=0 tx=0).stop 2023-10-06T05:22:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/3355632606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 msgr2=0x7fdcf4157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 0x7fdcf4157eb0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0007b20 tx=0x7fdcf0009040 comp rx=0 tx=0).stop 2023-10-06T05:22:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/3355632606 shutdown_connections 2023-10-06T05:22:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcd8067050 0x7fdcd8069500 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf4169ab0 0x7fdcf4169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce0004d50 0x7fdcf4157eb0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:02.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 --2- 172.21.15.31:0/3355632606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdcf4158760 0x7fdcf4169570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/3355632606 >> 172.21.15.31:0/3355632606 conn(0x7fdcf4099710 msgr2=0x7fdcf40af270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/3355632606 shutdown_connections 2023-10-06T05:22:02.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:02.175+0000 7fdcfb3ff700 1 -- 172.21.15.31:0/3355632606 wait complete. 2023-10-06T05:22:02.194 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:22:02.194 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:22:02.194 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:22:02.194 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:22:02.194 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:02.194 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:22:02.194 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:22:02.194 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:22:02.194 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:02.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:02.195 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:22:02.242 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:22:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:22:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:22:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:22:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41992sec preferred_lft 41992sec 2023-10-06T05:22:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:22:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:22:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:02.245 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:22:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:22:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:02.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:02.245 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:22:02.314 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:22:02.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:02.317 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:22:02.366 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:22:02.367 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:22:02.367 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:22:02.367 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:22:02.367 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:22:02.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:02.368 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:22:02.368 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:22:02.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:22:02.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:22:02.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:22:02.368 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:22:02.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:02.540 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:22:02.587 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:22:02.587 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:22:02.587 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:02.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:02.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:22:02.700 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:02.700 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:02.746 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:02.746 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:02.746 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:02.832 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:22:02.832 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:22:02.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:02.973 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:02.973 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:22:03.001 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:03.001 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:22:03.001 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:22:03.001 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:22:03.002 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:22:03.005 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:22:03.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:03.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:03.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:22:03.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.274+0000 7fc44d3ff700 1 Processor -- start 2023-10-06T05:22:03.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.275+0000 7fc44d3ff700 1 -- start start 2023-10-06T05:22:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc44d3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4481595f0 0x7fc4481599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc44d3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc448064580 0x7fc4480af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:03.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc44d3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4480ada10 con 0x7fc4480af800 2023-10-06T05:22:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc44d3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4480aeeb0 con 0x7fc4481595f0 2023-10-06T05:22:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc44d3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480ace70 con 0x7fc448064580 2023-10-06T05:22:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc446ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4481595f0 0x7fc4481599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc446ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4481595f0 0x7fc4481599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56480/0 (socket says 172.21.15.31:56480) 2023-10-06T05:22:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc446ffd700 1 -- 172.21.15.31:0/1360778238 learned_addr learned my addr 172.21.15.31:0/1360778238 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc4467fc700 1 --2- 172.21.15.31:0/1360778238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc448064580 0x7fc4480af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc446ffd700 1 -- 172.21.15.31:0/1360778238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc448064580 msgr2=0x7fc4480af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:03.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc446ffd700 1 --2- 172.21.15.31:0/1360778238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc448064580 0x7fc4480af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.276+0000 7fc446ffd700 1 -- 172.21.15.31:0/1360778238 >> v1:172.21.15.62:6789/0 conn(0x7fc4480af800 legacy=0x7fc4480b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.277+0000 7fc446ffd700 1 -- 172.21.15.31:0/1360778238 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480ad0e0 con 0x7fc4481595f0 2023-10-06T05:22:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.277+0000 7fc446ffd700 1 --2- 172.21.15.31:0/1360778238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 0x7fc4481599c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc438009990 tx=0x7fc438009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54d052911eae422 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.277+0000 7fc445ffb700 1 -- 172.21.15.31:0/1360778238 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438005c10 con 0x7fc4481595f0 2023-10-06T05:22:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.277+0000 7fc445ffb700 1 -- 172.21.15.31:0/1360778238 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc438005d70 con 0x7fc4481595f0 2023-10-06T05:22:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.277+0000 7fc445ffb700 1 -- 172.21.15.31:0/1360778238 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43800b910 con 0x7fc4481595f0 2023-10-06T05:22:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.278+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1360778238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 msgr2=0x7fc4481599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.278+0000 7fc44d3ff700 1 --2- 172.21.15.31:0/1360778238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 0x7fc4481599c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc438009990 tx=0x7fc438009220 comp rx=0 tx=0).stop 2023-10-06T05:22:03.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.278+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1360778238 shutdown_connections 2023-10-06T05:22:03.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.278+0000 7fc44d3ff700 1 --2- 172.21.15.31:0/1360778238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 0x7fc4481599c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.278+0000 7fc44d3ff700 1 --2- 172.21.15.31:0/1360778238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc448064580 0x7fc4480af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.278+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1360778238 >> 172.21.15.31:0/1360778238 conn(0x7fc44806ebf0 msgr2=0x7fc44806eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:03.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.278+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1360778238 shutdown_connections 2023-10-06T05:22:03.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.278+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1360778238 wait complete. 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.279+0000 7fc44d3ff700 1 Processor -- start 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.279+0000 7fc44d3ff700 1 -- start start 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.280+0000 7fc44d3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4480af800 0x7fc448151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.280+0000 7fc44d3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 0x7fc4481541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.280+0000 7fc44d3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44815eb40 0x7fc4480698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.280+0000 7fc44d3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4481519a0 con 0x7fc4481595f0 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.280+0000 7fc44d3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44815bcd0 con 0x7fc4480af800 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.280+0000 7fc44d3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44815bfb0 con 0x7fc44815eb40 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4467fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 0x7fc4481541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4467fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 0x7fc4481541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56482/0 (socket says 172.21.15.31:56482) 2023-10-06T05:22:03.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4467fc700 1 -- 172.21.15.31:0/1440408235 learned_addr learned my addr 172.21.15.31:0/1440408235 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:03.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc446ffd700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4480af800 0x7fc448151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:03.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4477fe700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44815eb40 0x7fc4480698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:03.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4467fc700 1 -- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44815eb40 msgr2=0x7fc4480698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:03.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4467fc700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44815eb40 0x7fc4480698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4467fc700 1 -- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4480af800 msgr2=0x7fc448151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:03.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4467fc700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4480af800 0x7fc448151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4467fc700 1 -- 172.21.15.31:0/1440408235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc438009480 con 0x7fc4481595f0 2023-10-06T05:22:03.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc446ffd700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4480af800 0x7fc448151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:03.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4477fe700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44815eb40 0x7fc4480698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:03.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.281+0000 7fc4467fc700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 0x7fc4481541d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00d890 tx=0x7fc43c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:03.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.282+0000 7fc42effd700 1 -- 172.21.15.31:0/1440408235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0125f0 con 0x7fc4481595f0 2023-10-06T05:22:03.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.282+0000 7fc42effd700 1 -- 172.21.15.31:0/1440408235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c012770 con 0x7fc4481595f0 2023-10-06T05:22:03.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.282+0000 7fc42effd700 1 -- 172.21.15.31:0/1440408235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c012b30 con 0x7fc4481595f0 2023-10-06T05:22:03.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.282+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1440408235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc448154710 con 0x7fc4481595f0 2023-10-06T05:22:03.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.282+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1440408235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc448154cc0 con 0x7fc4481595f0 2023-10-06T05:22:03.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.283+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1440408235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44815c120 con 0x7fc4481595f0 2023-10-06T05:22:03.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.284+0000 7fc42effd700 1 -- 172.21.15.31:0/1440408235 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fc43c01e460 con 0x7fc4481595f0 2023-10-06T05:22:03.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.285+0000 7fc42effd700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc434067050 0x7fc434069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:03.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.285+0000 7fc42effd700 1 -- 172.21.15.31:0/1440408235 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc43c084f30 con 0x7fc4481595f0 2023-10-06T05:22:03.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.285+0000 7fc446ffd700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc434067050 0x7fc434069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.286+0000 7fc446ffd700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc434067050 0x7fc434069500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc44815e3d0 tx=0x7fc438016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.289+0000 7fc42effd700 1 -- 172.21.15.31:0/1440408235 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc43c051e90 con 0x7fc4481595f0 2023-10-06T05:22:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.535+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1440408235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc44804c380 con 0x7fc4481595f0 2023-10-06T05:22:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.535+0000 7fc42effd700 1 -- 172.21.15.31:0/1440408235 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc43c0554b0 con 0x7fc4481595f0 2023-10-06T05:22:03.912 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:22:03.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.537+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc434067050 msgr2=0x7fc434069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:03.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.537+0000 7fc44d3ff700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc434067050 0x7fc434069500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc44815e3d0 tx=0x7fc438016000 comp rx=0 tx=0).stop 2023-10-06T05:22:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.537+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1440408235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 msgr2=0x7fc4481541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.537+0000 7fc44d3ff700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 0x7fc4481541d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00d890 tx=0x7fc43c008040 comp rx=0 tx=0).stop 2023-10-06T05:22:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.538+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1440408235 shutdown_connections 2023-10-06T05:22:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.538+0000 7fc44d3ff700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc434067050 0x7fc434069500 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.538+0000 7fc44d3ff700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44815eb40 0x7fc4480698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.538+0000 7fc44d3ff700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4480af800 0x7fc448151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.538+0000 7fc44d3ff700 1 --2- 172.21.15.31:0/1440408235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4481595f0 0x7fc4481541d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.538+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1440408235 >> 172.21.15.31:0/1440408235 conn(0x7fc44806ebf0 msgr2=0x7fc44815b340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.538+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1440408235 shutdown_connections 2023-10-06T05:22:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:03.538+0000 7fc44d3ff700 1 -- 172.21.15.31:0/1440408235 wait complete. 2023-10-06T05:22:03.915 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:22:03.915 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:22:03.915 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:22:03.915 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:22:03.915 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:03.915 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:22:03.915 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:22:03.915 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:22:03.915 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:03.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:03.916 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:22:03.967 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:22:04.060 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:22:04.060 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:22:04.060 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:04.060 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:22:04.060 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:04.060 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:04.061 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:04.061 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:22:04.061 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:04.061 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:04.061 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:22:04.061 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:04.061 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:04.061 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:22:04.061 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:04.061 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:22:04.062 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41990sec preferred_lft 41990sec 2023-10-06T05:22:04.062 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:22:04.062 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:04.062 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:22:04.062 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:04.062 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:22:04.062 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:04.062 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:22:04.062 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:04.063 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:04.063 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:22:04.063 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:22:04.063 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:04.063 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:04.063 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:22:04.063 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:22:04.063 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:04.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:04.065 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:22:04.140 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:22:04.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:04.143 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:22:04.193 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:22:04.194 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:22:04.194 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:22:04.194 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:22:04.194 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:22:04.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:04.195 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:22:04.195 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:22:04.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:22:04.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:22:04.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:22:04.195 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:22:04.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:04.368 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:22:04.415 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:22:04.415 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:22:04.415 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:04.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:04.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:22:04.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:04.487 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:04.533 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:04.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:04.534 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:04.619 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:22:04.619 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:22:04.619 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:04.759 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:04.759 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:22:04.785 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:04.786 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:22:04.786 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:22:04.786 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:22:04.786 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:22:04.789 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:22:04.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:04.789 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:04.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:22:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.069+0000 7f663e7ff700 1 Processor -- start 2023-10-06T05:22:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.069+0000 7f663e7ff700 1 -- start start 2023-10-06T05:22:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.070+0000 7f663e7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6638161140 0x7f6638069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:05.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.070+0000 7f663e7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 0x7f663815f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:05.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.070+0000 7f663e7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663806aad0 con 0x7f663815ed70 2023-10-06T05:22:05.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.070+0000 7f663e7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6638069ec0 con 0x7f6638161140 2023-10-06T05:22:05.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.071+0000 7f663e7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663806a1a0 con 0x7f663806d920 2023-10-06T05:22:05.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.071+0000 7f663cd9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 0x7f663815f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:05.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.071+0000 7f66377fe700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f663815ed70 0x7f663806d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:39552/0 (socket says 172.21.15.31:39552) 2023-10-06T05:22:05.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.071+0000 7f66377fe700 1 -- 172.21.15.31:0/349945481 learned_addr learned my addr 172.21.15.31:0/349945481 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:05.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.071+0000 7f6637fff700 1 --2- 172.21.15.31:0/349945481 >> v2:172.21.15.62:3300/0 conn(0x7f6638161140 0x7f6638069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:05.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.071+0000 7f663cd9c700 1 -- 172.21.15.31:0/349945481 >> v2:172.21.15.62:3300/0 conn(0x7f6638161140 msgr2=0x7f6638069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:05.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.071+0000 7f663cd9c700 1 --2- 172.21.15.31:0/349945481 >> v2:172.21.15.62:3300/0 conn(0x7f6638161140 0x7f6638069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:05.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.071+0000 7f663cd9c700 1 -- 172.21.15.31:0/349945481 >> v1:172.21.15.62:6789/0 conn(0x7f663815ed70 legacy=0x7f663806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:05.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.071+0000 7f6636ffd700 1 -- 172.21.15.31:0/349945481 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432245551 0 0) 0x7f663806aad0 con 0x7f663815ed70 2023-10-06T05:22:05.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.071+0000 7f663cd9c700 1 -- 172.21.15.31:0/349945481 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663806a480 con 0x7f663806d920 2023-10-06T05:22:05.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.072+0000 7f663cd9c700 1 --2- 172.21.15.31:0/349945481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 0x7f663815f380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6630009e60 tx=0x7f6630009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75cb6adda47dfd99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:05.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.072+0000 7f6636ffd700 1 -- 172.21.15.31:0/349945481 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663000e070 con 0x7f663806d920 2023-10-06T05:22:05.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.072+0000 7f6636ffd700 1 -- 172.21.15.31:0/349945481 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663000bb60 con 0x7f663806d920 2023-10-06T05:22:05.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.072+0000 7f6636ffd700 1 -- 172.21.15.31:0/349945481 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630012430 con 0x7f663806d920 2023-10-06T05:22:05.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.072+0000 7f663e7ff700 1 -- 172.21.15.31:0/349945481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 msgr2=0x7f663815f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:05.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.072+0000 7f663e7ff700 1 --2- 172.21.15.31:0/349945481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 0x7f663815f380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6630009e60 tx=0x7f6630009780 comp rx=0 tx=0).stop 2023-10-06T05:22:05.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.073+0000 7f663e7ff700 1 -- 172.21.15.31:0/349945481 shutdown_connections 2023-10-06T05:22:05.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.073+0000 7f663e7ff700 1 --2- 172.21.15.31:0/349945481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 0x7f663815f380 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:05.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.073+0000 7f663e7ff700 1 --2- 172.21.15.31:0/349945481 >> v2:172.21.15.62:3300/0 conn(0x7f6638161140 0x7f6638069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:05.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.073+0000 7f663e7ff700 1 -- 172.21.15.31:0/349945481 >> 172.21.15.31:0/349945481 conn(0x7f6638068030 msgr2=0x7f66380cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:05.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.073+0000 7f663e7ff700 1 -- 172.21.15.31:0/349945481 shutdown_connections 2023-10-06T05:22:05.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.073+0000 7f663e7ff700 1 -- 172.21.15.31:0/349945481 wait complete. 2023-10-06T05:22:05.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.074+0000 7f663e7ff700 1 Processor -- start 2023-10-06T05:22:05.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.074+0000 7f663e7ff700 1 -- start start 2023-10-06T05:22:05.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.075+0000 7f663e7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 0x7f66380bc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:05.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.075+0000 7f663e7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f663815ed70 0x7f66380c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:05.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.075+0000 7f663e7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6638161140 0x7f66380c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:05.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.075+0000 7f663e7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66380c0b40 con 0x7f6638161140 2023-10-06T05:22:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.075+0000 7f663e7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66380c1dc0 con 0x7f663815ed70 2023-10-06T05:22:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.075+0000 7f663e7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66380c06b0 con 0x7f663806d920 2023-10-06T05:22:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.075+0000 7f663cd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6638161140 0x7f66380c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.075+0000 7f663cd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6638161140 0x7f66380c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56496/0 (socket says 172.21.15.31:56496) 2023-10-06T05:22:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.076+0000 7f663cd9c700 1 -- 172.21.15.31:0/785222369 learned_addr learned my addr 172.21.15.31:0/785222369 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.076+0000 7f66377fe700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f663815ed70 0x7f66380c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.076+0000 7f6637fff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 0x7f66380bc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.076+0000 7f663cd9c700 1 -- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 msgr2=0x7f66380bc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:05.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.076+0000 7f663cd9c700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 0x7f66380bc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:05.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.076+0000 7f663cd9c700 1 -- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f663815ed70 msgr2=0x7f66380c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:05.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.076+0000 7f663cd9c700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f663815ed70 0x7f66380c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:05.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.076+0000 7f663cd9c700 1 -- 172.21.15.31:0/785222369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6630009a20 con 0x7f6638161140 2023-10-06T05:22:05.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.076+0000 7f6637fff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 0x7f66380bc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:05.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.076+0000 7f663cd9c700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6638161140 0x7f66380c8170 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f663806a2e0 tx=0x7f6630019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:05.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.077+0000 7f661ffff700 1 -- 172.21.15.31:0/785222369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630012e60 con 0x7f6638161140 2023-10-06T05:22:05.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.077+0000 7f661ffff700 1 -- 172.21.15.31:0/785222369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6630027470 con 0x7f6638161140 2023-10-06T05:22:05.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.077+0000 7f661ffff700 1 -- 172.21.15.31:0/785222369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630027830 con 0x7f6638161140 2023-10-06T05:22:05.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.077+0000 7f663e7ff700 1 -- 172.21.15.31:0/785222369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66380bfaa0 con 0x7f6638161140 2023-10-06T05:22:05.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.077+0000 7f663e7ff700 1 -- 172.21.15.31:0/785222369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66380c0060 con 0x7f6638161140 2023-10-06T05:22:05.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.078+0000 7f663e7ff700 1 -- 172.21.15.31:0/785222369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66380c0860 con 0x7f6638161140 2023-10-06T05:22:05.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.079+0000 7f661ffff700 1 -- 172.21.15.31:0/785222369 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f6630024070 con 0x7f6638161140 2023-10-06T05:22:05.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.079+0000 7f661ffff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6624067050 0x7f6624069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:05.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.080+0000 7f661ffff700 1 -- 172.21.15.31:0/785222369 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6630086ec0 con 0x7f6638161140 2023-10-06T05:22:05.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.080+0000 7f6637fff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6624067050 0x7f6624069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:05.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.081+0000 7f6637fff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6624067050 0x7f6624069500 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6628011460 tx=0x7f6628009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:05.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.084+0000 7f661ffff700 1 -- 172.21.15.31:0/785222369 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66380c0860 con 0x7f6638161140 2023-10-06T05:22:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:05.331+0000 7f663e7ff700 1 -- 172.21.15.31:0/785222369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f66380c0820 con 0x7f6638161140 2023-10-06T05:22:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.137+0000 7f661ffff700 1 -- 172.21.15.31:0/785222369 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6630053e20 con 0x7f6638161140 2023-10-06T05:22:06.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.140+0000 7f663e7ff700 1 -- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6624067050 msgr2=0x7f6624069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.140+0000 7f663e7ff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6624067050 0x7f6624069500 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6628011460 tx=0x7f6628009120 comp rx=0 tx=0).stop 2023-10-06T05:22:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.140+0000 7f663e7ff700 1 -- 172.21.15.31:0/785222369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6638161140 msgr2=0x7f66380c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.140+0000 7f663e7ff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6638161140 0x7f66380c8170 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f663806a2e0 tx=0x7f6630019000 comp rx=0 tx=0).stop 2023-10-06T05:22:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.140+0000 7f663e7ff700 1 -- 172.21.15.31:0/785222369 shutdown_connections 2023-10-06T05:22:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.140+0000 7f663e7ff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6624067050 0x7f6624069500 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.140+0000 7f663e7ff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6638161140 0x7f66380c8170 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.140+0000 7f663e7ff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f663806d920 0x7f66380bc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.140+0000 7f663e7ff700 1 --2- 172.21.15.31:0/785222369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f663815ed70 0x7f66380c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.140+0000 7f663e7ff700 1 -- 172.21.15.31:0/785222369 >> 172.21.15.31:0/785222369 conn(0x7f6638068030 msgr2=0x7f663815f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.141+0000 7f663e7ff700 1 -- 172.21.15.31:0/785222369 shutdown_connections 2023-10-06T05:22:06.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.141+0000 7f663e7ff700 1 -- 172.21.15.31:0/785222369 wait complete. 2023-10-06T05:22:06.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:22:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.362+0000 7f39b8fff700 1 Processor -- start 2023-10-06T05:22:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.363+0000 7f39b8fff700 1 -- start start 2023-10-06T05:22:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b8fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39b4068d80 0x7f39b4159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b8fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39b40cc0d0 0x7f39b40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b8fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b40c95b0 con 0x7f39b41595a0 2023-10-06T05:22:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b8fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b40c9890 con 0x7f39b4068d80 2023-10-06T05:22:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b8fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b40c9b70 con 0x7f39b40cc0d0 2023-10-06T05:22:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b1d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39b4068d80 0x7f39b4159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b1d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39b4068d80 0x7f39b4159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56504/0 (socket says 172.21.15.31:56504) 2023-10-06T05:22:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b1d9b700 1 -- 172.21.15.31:0/1619763874 learned_addr learned my addr 172.21.15.31:0/1619763874 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b2d9d700 1 --2- 172.21.15.31:0/1619763874 >> v2:172.21.15.62:3300/0 conn(0x7f39b40cc0d0 0x7f39b40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b1d9b700 1 -- 172.21.15.31:0/1619763874 >> v2:172.21.15.62:3300/0 conn(0x7f39b40cc0d0 msgr2=0x7f39b40c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.364+0000 7f39b1d9b700 1 --2- 172.21.15.31:0/1619763874 >> v2:172.21.15.62:3300/0 conn(0x7f39b40cc0d0 0x7f39b40c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.365+0000 7f39b1d9b700 1 -- 172.21.15.31:0/1619763874 >> v1:172.21.15.62:6789/0 conn(0x7f39b41595a0 legacy=0x7f39b4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.365+0000 7f39b1d9b700 1 -- 172.21.15.31:0/1619763874 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b40ca130 con 0x7f39b4068d80 2023-10-06T05:22:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.365+0000 7f39b1d9b700 1 --2- 172.21.15.31:0/1619763874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b4068d80 0x7f39b4159970 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f39a80099b0 tx=0x7f39a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77e6c513bfbcd78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.365+0000 7f39b159a700 1 -- 172.21.15.31:0/1619763874 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a800b820 con 0x7f39b4068d80 2023-10-06T05:22:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.365+0000 7f39b159a700 1 -- 172.21.15.31:0/1619763874 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a800b980 con 0x7f39b4068d80 2023-10-06T05:22:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.365+0000 7f39b159a700 1 -- 172.21.15.31:0/1619763874 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a800bd20 con 0x7f39b4068d80 2023-10-06T05:22:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.366+0000 7f39b8fff700 1 -- 172.21.15.31:0/1619763874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b4068d80 msgr2=0x7f39b4159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.366+0000 7f39b8fff700 1 --2- 172.21.15.31:0/1619763874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b4068d80 0x7f39b4159970 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f39a80099b0 tx=0x7f39a8009250 comp rx=0 tx=0).stop 2023-10-06T05:22:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.366+0000 7f39b8fff700 1 -- 172.21.15.31:0/1619763874 shutdown_connections 2023-10-06T05:22:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.366+0000 7f39b8fff700 1 --2- 172.21.15.31:0/1619763874 >> v2:172.21.15.62:3300/0 conn(0x7f39b40cc0d0 0x7f39b40c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.366+0000 7f39b8fff700 1 --2- 172.21.15.31:0/1619763874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b4068d80 0x7f39b4159970 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.366+0000 7f39b8fff700 1 -- 172.21.15.31:0/1619763874 >> 172.21.15.31:0/1619763874 conn(0x7f39b406b620 msgr2=0x7f39b406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.366+0000 7f39b8fff700 1 -- 172.21.15.31:0/1619763874 shutdown_connections 2023-10-06T05:22:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.366+0000 7f39b8fff700 1 -- 172.21.15.31:0/1619763874 wait complete. 2023-10-06T05:22:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.367+0000 7f39b8fff700 1 Processor -- start 2023-10-06T05:22:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.367+0000 7f39b8fff700 1 -- start start 2023-10-06T05:22:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.368+0000 7f39b8fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4068d80 0x7f39b415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.368+0000 7f39b8fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b40cc0d0 0x7f39b415b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.368+0000 7f39b8fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b41595a0 0x7f39b416d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.368+0000 7f39b8fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b415bb60 con 0x7f39b40cc0d0 2023-10-06T05:22:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.368+0000 7f39b8fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b415e1c0 con 0x7f39b4068d80 2023-10-06T05:22:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.368+0000 7f39b8fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b415e4a0 con 0x7f39b41595a0 2023-10-06T05:22:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b40cc0d0 0x7f39b415b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b40cc0d0 0x7f39b415b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56518/0 (socket says 172.21.15.31:56518) 2023-10-06T05:22:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b1d9b700 1 -- 172.21.15.31:0/2920606269 learned_addr learned my addr 172.21.15.31:0/2920606269 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b259c700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4068d80 0x7f39b415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b2d9d700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b41595a0 0x7f39b416d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b1d9b700 1 -- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b41595a0 msgr2=0x7f39b416d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b1d9b700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b41595a0 0x7f39b416d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b1d9b700 1 -- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4068d80 msgr2=0x7f39b415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b1d9b700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4068d80 0x7f39b415dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b1d9b700 1 -- 172.21.15.31:0/2920606269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80094f0 con 0x7f39b40cc0d0 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b2d9d700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b41595a0 0x7f39b416d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.369+0000 7f39b1d9b700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b40cc0d0 0x7f39b415b620 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f39a80094c0 tx=0x7f39a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.370+0000 7f39a27fc700 1 -- 172.21.15.31:0/2920606269 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8024440 con 0x7f39b40cc0d0 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.370+0000 7f39a27fc700 1 -- 172.21.15.31:0/2920606269 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a80245c0 con 0x7f39b40cc0d0 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.370+0000 7f39a27fc700 1 -- 172.21.15.31:0/2920606269 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8024980 con 0x7f39b40cc0d0 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.370+0000 7f39b8fff700 1 -- 172.21.15.31:0/2920606269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b415e780 con 0x7f39b40cc0d0 2023-10-06T05:22:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.370+0000 7f39b8fff700 1 -- 172.21.15.31:0/2920606269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b415ed40 con 0x7f39b40cc0d0 2023-10-06T05:22:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.371+0000 7f39b8fff700 1 -- 172.21.15.31:0/2920606269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b415ef30 con 0x7f39b40cc0d0 2023-10-06T05:22:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.371+0000 7f39a27fc700 1 -- 172.21.15.31:0/2920606269 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f39a801a420 con 0x7f39b40cc0d0 2023-10-06T05:22:06.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.372+0000 7f39a27fc700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3994067050 0x7f3994069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.372+0000 7f39a27fc700 1 -- 172.21.15.31:0/2920606269 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f39a8086770 con 0x7f39b40cc0d0 2023-10-06T05:22:06.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.373+0000 7f39b259c700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3994067050 0x7f3994069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.374+0000 7f39b259c700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3994067050 0x7f3994069500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f39a40037c0 tx=0x7f39a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.377+0000 7f39a27fc700 1 -- 172.21.15.31:0/2920606269 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39a80536d0 con 0x7f39b40cc0d0 2023-10-06T05:22:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.606+0000 7f39b8fff700 1 -- 172.21.15.31:0/2920606269 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f39b404c380 con 0x7f3994067050 2023-10-06T05:22:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.608+0000 7f39a27fc700 1 -- 172.21.15.31:0/2920606269 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f39b404c380 con 0x7f3994067050 2023-10-06T05:22:06.610 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:22:06.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:06.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:22:06.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:06.610 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:22:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.610+0000 7f39b8fff700 1 -- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3994067050 msgr2=0x7f3994069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.610+0000 7f39b8fff700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3994067050 0x7f3994069500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f39a40037c0 tx=0x7f39a4009040 comp rx=0 tx=0).stop 2023-10-06T05:22:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.610+0000 7f39b8fff700 1 -- 172.21.15.31:0/2920606269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b40cc0d0 msgr2=0x7f39b415b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.610+0000 7f39b8fff700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b40cc0d0 0x7f39b415b620 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f39a80094c0 tx=0x7f39a8017000 comp rx=0 tx=0).stop 2023-10-06T05:22:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.611+0000 7f39b8fff700 1 -- 172.21.15.31:0/2920606269 shutdown_connections 2023-10-06T05:22:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.611+0000 7f39b8fff700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3994067050 0x7f3994069500 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.611+0000 7f39b8fff700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39b41595a0 0x7f39b416d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.611+0000 7f39b8fff700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b4068d80 0x7f39b415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.611+0000 7f39b8fff700 1 --2- 172.21.15.31:0/2920606269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b40cc0d0 0x7f39b415b620 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.611+0000 7f39b8fff700 1 -- 172.21.15.31:0/2920606269 >> 172.21.15.31:0/2920606269 conn(0x7f39b406b620 msgr2=0x7f39b4153740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.611+0000 7f39b8fff700 1 -- 172.21.15.31:0/2920606269 shutdown_connections 2023-10-06T05:22:06.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.611+0000 7f39b8fff700 1 -- 172.21.15.31:0/2920606269 wait complete. 2023-10-06T05:22:06.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:22:06.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.828+0000 7f997afff700 1 Processor -- start 2023-10-06T05:22:06.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.828+0000 7f997afff700 1 -- start start 2023-10-06T05:22:06.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.830+0000 7f997afff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9974159890 0x7f99740c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.830+0000 7f997afff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159c60 0x7f9974156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.830+0000 7f997afff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9974158190 con 0x7f9974156fa0 2023-10-06T05:22:06.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.830+0000 7f997afff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9974157580 con 0x7f9974159890 2023-10-06T05:22:06.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.830+0000 7f997afff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9974157860 con 0x7f9974159c60 2023-10-06T05:22:06.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.830+0000 7f9973fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159c60 0x7f9974156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.830+0000 7f9978d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9974159890 0x7f99740c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f9973fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159c60 0x7f9974156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:53458/0 (socket says 172.21.15.31:53458) 2023-10-06T05:22:06.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f9978d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9974159890 0x7f99740c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:37986/0 (socket says 172.21.15.31:37986) 2023-10-06T05:22:06.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f9973fff700 1 -- 172.21.15.31:0/1237093106 learned_addr learned my addr 172.21.15.31:0/1237093106 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:06.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f997959c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9974156fa0 0x7f9974155430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:39566/0 (socket says 172.21.15.31:39566) 2023-10-06T05:22:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f99737fe700 1 -- 172.21.15.31:0/1237093106 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827036242 0 0) 0x7f9974158190 con 0x7f9974156fa0 2023-10-06T05:22:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f99737fe700 1 -- 172.21.15.31:0/1237093106 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9964003570 con 0x7f9974156fa0 2023-10-06T05:22:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f9973fff700 1 -- 172.21.15.31:0/1237093106 >> v2:172.21.15.62:3300/0 conn(0x7f9974159890 msgr2=0x7f99740c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f9973fff700 1 --2- 172.21.15.31:0/1237093106 >> v2:172.21.15.62:3300/0 conn(0x7f9974159890 0x7f99740c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f9973fff700 1 -- 172.21.15.31:0/1237093106 >> v1:172.21.15.62:6789/0 conn(0x7f9974156fa0 legacy=0x7f9974155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f9973fff700 1 -- 172.21.15.31:0/1237093106 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9974157b40 con 0x7f9974159c60 2023-10-06T05:22:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f9978d9b700 1 --2- 172.21.15.31:0/1237093106 >> v2:172.21.15.62:3300/0 conn(0x7f9974159890 0x7f99740c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:06.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.831+0000 7f9973fff700 1 --2- 172.21.15.31:0/1237093106 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159c60 0x7f9974156a60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f996000bc80 tx=0x7f996000b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d9df094e1c6374a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.832+0000 7f99737fe700 1 -- 172.21.15.31:0/1237093106 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996000db00 con 0x7f9974159c60 2023-10-06T05:22:06.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.832+0000 7f99737fe700 1 -- 172.21.15.31:0/1237093106 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996000dc60 con 0x7f9974159c60 2023-10-06T05:22:06.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.832+0000 7f99737fe700 1 -- 172.21.15.31:0/1237093106 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960019480 con 0x7f9974159c60 2023-10-06T05:22:06.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.833+0000 7f997afff700 1 -- 172.21.15.31:0/1237093106 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159c60 msgr2=0x7f9974156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.833+0000 7f997afff700 1 --2- 172.21.15.31:0/1237093106 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159c60 0x7f9974156a60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f996000bc80 tx=0x7f996000b4f0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.833+0000 7f997afff700 1 -- 172.21.15.31:0/1237093106 shutdown_connections 2023-10-06T05:22:06.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.833+0000 7f997afff700 1 --2- 172.21.15.31:0/1237093106 >> v2:172.21.15.62:3300/0 conn(0x7f9974159890 0x7f99740c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.833+0000 7f997afff700 1 --2- 172.21.15.31:0/1237093106 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159c60 0x7f9974156a60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.833+0000 7f997afff700 1 -- 172.21.15.31:0/1237093106 >> 172.21.15.31:0/1237093106 conn(0x7f9974067560 msgr2=0x7f9974067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:06.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.833+0000 7f997afff700 1 -- 172.21.15.31:0/1237093106 shutdown_connections 2023-10-06T05:22:06.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.833+0000 7f997afff700 1 -- 172.21.15.31:0/1237093106 wait complete. 2023-10-06T05:22:06.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.835+0000 7f997afff700 1 Processor -- start 2023-10-06T05:22:06.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.835+0000 7f997afff700 1 -- start start 2023-10-06T05:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.836+0000 7f997afff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9974156fa0 0x7f99740ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.836+0000 7f997afff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159890 0x7f9974153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.836+0000 7f997afff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9974159c60 0x7f997414ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.836+0000 7f997afff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974154960 con 0x7f9974159c60 2023-10-06T05:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.836+0000 7f997afff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974153ed0 con 0x7f9974156fa0 2023-10-06T05:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.836+0000 7f997afff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9974165550 con 0x7f9974159890 2023-10-06T05:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f997959c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9974159c60 0x7f997414ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f997959c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9974159c60 0x7f997414ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56524/0 (socket says 172.21.15.31:56524) 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f997959c700 1 -- 172.21.15.31:0/882084196 learned_addr learned my addr 172.21.15.31:0/882084196 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f9978d9b700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9974156fa0 0x7f99740ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f9973fff700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159890 0x7f9974153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f997959c700 1 -- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159890 msgr2=0x7f9974153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f997959c700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159890 0x7f9974153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f997959c700 1 -- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9974156fa0 msgr2=0x7f99740ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f997959c700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9974156fa0 0x7f99740ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f997959c700 1 -- 172.21.15.31:0/882084196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968009420 con 0x7f9974159c60 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.837+0000 7f997959c700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9974159c60 0x7f997414ce70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f996c00cc70 tx=0x7f996c001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.838+0000 7f9970ff9700 1 -- 172.21.15.31:0/882084196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00a810 con 0x7f9974159c60 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.838+0000 7f9970ff9700 1 -- 172.21.15.31:0/882084196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00a970 con 0x7f9974159c60 2023-10-06T05:22:06.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.838+0000 7f9970ff9700 1 -- 172.21.15.31:0/882084196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00ad30 con 0x7f9974159c60 2023-10-06T05:22:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.838+0000 7f997afff700 1 -- 172.21.15.31:0/882084196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996000b790 con 0x7f9974159c60 2023-10-06T05:22:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.838+0000 7f997afff700 1 -- 172.21.15.31:0/882084196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99741543a0 con 0x7f9974159c60 2023-10-06T05:22:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.839+0000 7f997afff700 1 -- 172.21.15.31:0/882084196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997404c380 con 0x7f9974159c60 2023-10-06T05:22:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.840+0000 7f9970ff9700 1 -- 172.21.15.31:0/882084196 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f996c012630 con 0x7f9974159c60 2023-10-06T05:22:06.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.841+0000 7f9970ff9700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9958067050 0x7f9958069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:06.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.841+0000 7f9970ff9700 1 -- 172.21.15.31:0/882084196 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f996c084b90 con 0x7f9974159c60 2023-10-06T05:22:06.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.841+0000 7f9978d9b700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9958067050 0x7f9958069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:06.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.842+0000 7f9978d9b700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9958067050 0x7f9958069500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9968009f40 tx=0x7f99680091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:06.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:06.845+0000 7f9970ff9700 1 -- 172.21.15.31:0/882084196 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f996c051b90 con 0x7f9974159c60 2023-10-06T05:22:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.048+0000 7f997afff700 1 -- 172.21.15.31:0/882084196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f99740ccc40 con 0x7f9974159c60 2023-10-06T05:22:07.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.048+0000 7f9970ff9700 1 -- 172.21.15.31:0/882084196 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f996c0551b0 con 0x7f9974159c60 2023-10-06T05:22:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 -- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9958067050 msgr2=0x7f9958069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9958067050 0x7f9958069500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9968009f40 tx=0x7f99680091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 -- 172.21.15.31:0/882084196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9974159c60 msgr2=0x7f997414ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9974159c60 0x7f997414ce70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f996c00cc70 tx=0x7f996c001a90 comp rx=0 tx=0).stop 2023-10-06T05:22:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 -- 172.21.15.31:0/882084196 shutdown_connections 2023-10-06T05:22:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9958067050 0x7f9958069500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9974159c60 0x7f997414ce70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9974156fa0 0x7f99740ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 --2- 172.21.15.31:0/882084196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9974159890 0x7f9974153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 -- 172.21.15.31:0/882084196 >> 172.21.15.31:0/882084196 conn(0x7f9974067560 msgr2=0x7f99741514f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 -- 172.21.15.31:0/882084196 shutdown_connections 2023-10-06T05:22:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.051+0000 7f997afff700 1 -- 172.21.15.31:0/882084196 wait complete. 2023-10-06T05:22:07.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2023-10-06T05:22:07.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.266+0000 7f69581ff700 1 Processor -- start 2023-10-06T05:22:07.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.266+0000 7f69581ff700 1 -- start start 2023-10-06T05:22:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f69581ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f695015ca40 0x7f6950068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f69581ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695006a300 0x7f69500c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f69581ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69500c8d70 0x7f695006cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f69581ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69500cabc0 con 0x7f695015ca40 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f69581ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6950068bb0 con 0x7f69500c8d70 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f69581ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69500c91f0 con 0x7f695006a300 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f6955f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f695015ca40 0x7f6950068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f6955f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f695015ca40 0x7f6950068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48128/0 (socket says 172.21.15.31:48128) 2023-10-06T05:22:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f6955f9b700 1 -- 172.21.15.31:0/2818722223 learned_addr learned my addr 172.21.15.31:0/2818722223 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f695679c700 1 --2- 172.21.15.31:0/2818722223 >> v2:172.21.15.62:3300/0 conn(0x7f69500c8d70 0x7f695006cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f695579a700 1 --2- 172.21.15.31:0/2818722223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695006a300 0x7f69500c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f6955f9b700 1 -- 172.21.15.31:0/2818722223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695006a300 msgr2=0x7f69500c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f6955f9b700 1 --2- 172.21.15.31:0/2818722223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695006a300 0x7f69500c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f6955f9b700 1 -- 172.21.15.31:0/2818722223 >> v2:172.21.15.62:3300/0 conn(0x7f69500c8d70 msgr2=0x7f695006cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f6955f9b700 1 --2- 172.21.15.31:0/2818722223 >> v2:172.21.15.62:3300/0 conn(0x7f69500c8d70 0x7f695006cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f6955f9b700 1 -- 172.21.15.31:0/2818722223 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6950068da0 con 0x7f695015ca40 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.268+0000 7f695679c700 1 --2- 172.21.15.31:0/2818722223 >> v2:172.21.15.62:3300/0 conn(0x7f69500c8d70 0x7f695006cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.269+0000 7f6955f9b700 1 --2- 172.21.15.31:0/2818722223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695015ca40 0x7f6950068670 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f695006c620 tx=0x7f694c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5408ff81655a7cbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.269+0000 7f6954f99700 1 -- 172.21.15.31:0/2818722223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c00b820 con 0x7f695015ca40 2023-10-06T05:22:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.269+0000 7f6954f99700 1 -- 172.21.15.31:0/2818722223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c00b980 con 0x7f695015ca40 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.269+0000 7f6954f99700 1 -- 172.21.15.31:0/2818722223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c00bd20 con 0x7f695015ca40 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.269+0000 7f69581ff700 1 -- 172.21.15.31:0/2818722223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695015ca40 msgr2=0x7f6950068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.269+0000 7f69581ff700 1 --2- 172.21.15.31:0/2818722223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695015ca40 0x7f6950068670 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f695006c620 tx=0x7f694c009250 comp rx=0 tx=0).stop 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.270+0000 7f69581ff700 1 -- 172.21.15.31:0/2818722223 shutdown_connections 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.270+0000 7f69581ff700 1 --2- 172.21.15.31:0/2818722223 >> v2:172.21.15.62:3300/0 conn(0x7f69500c8d70 0x7f695006cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.270+0000 7f69581ff700 1 --2- 172.21.15.31:0/2818722223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695015ca40 0x7f6950068670 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.270+0000 7f69581ff700 1 --2- 172.21.15.31:0/2818722223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695006a300 0x7f69500c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.270+0000 7f69581ff700 1 -- 172.21.15.31:0/2818722223 >> 172.21.15.31:0/2818722223 conn(0x7f69500ce290 msgr2=0x7f69500ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.270+0000 7f69581ff700 1 -- 172.21.15.31:0/2818722223 shutdown_connections 2023-10-06T05:22:07.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.270+0000 7f69581ff700 1 -- 172.21.15.31:0/2818722223 wait complete. 2023-10-06T05:22:07.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.271+0000 7f69581ff700 1 Processor -- start 2023-10-06T05:22:07.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.271+0000 7f69581ff700 1 -- start start 2023-10-06T05:22:07.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.272+0000 7f69581ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695006a300 0x7f69500b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.272+0000 7f69581ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69500c8d70 0x7f69500b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.272+0000 7f69581ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695015ca40 0x7f695015e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.272+0000 7f69581ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69500b24b0 con 0x7f695015ca40 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.272+0000 7f69581ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695015e600 con 0x7f69500c8d70 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.272+0000 7f69581ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69500b18a0 con 0x7f695006a300 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.272+0000 7f695679c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695015ca40 0x7f695015e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.272+0000 7f695679c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695015ca40 0x7f695015e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48134/0 (socket says 172.21.15.31:48134) 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f695679c700 1 -- 172.21.15.31:0/2526636395 learned_addr learned my addr 172.21.15.31:0/2526636395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f695579a700 1 --2- 172.21.15.31:0/2526636395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69500c8d70 0x7f69500b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f6955f9b700 1 --2- 172.21.15.31:0/2526636395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695006a300 0x7f69500b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f695679c700 1 -- 172.21.15.31:0/2526636395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695006a300 msgr2=0x7f69500b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f695679c700 1 --2- 172.21.15.31:0/2526636395 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f695006a300 0x7f69500b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f695679c700 1 -- 172.21.15.31:0/2526636395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69500c8d70 msgr2=0x7f69500b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f695679c700 1 --2- 172.21.15.31:0/2526636395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69500c8d70 0x7f69500b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f695679c700 1 -- 172.21.15.31:0/2526636395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0094f0 con 0x7f695015ca40 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f695579a700 1 --2- 172.21.15.31:0/2526636395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69500c8d70 0x7f69500b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f695679c700 1 --2- 172.21.15.31:0/2526636395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695015ca40 0x7f695015e0c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f694800d180 tx=0x7f6948009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f6945ffb700 1 -- 172.21.15.31:0/2526636395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694800de20 con 0x7f695015ca40 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f6945ffb700 1 -- 172.21.15.31:0/2526636395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694800f040 con 0x7f695015ca40 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f69581ff700 1 -- 172.21.15.31:0/2526636395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69500b1b80 con 0x7f695015ca40 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.273+0000 7f6945ffb700 1 -- 172.21.15.31:0/2526636395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6948015690 con 0x7f695015ca40 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.274+0000 7f69581ff700 1 -- 172.21.15.31:0/2526636395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69500b2140 con 0x7f695015ca40 2023-10-06T05:22:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.275+0000 7f69581ff700 1 -- 172.21.15.31:0/2526636395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695004c380 con 0x7f695015ca40 2023-10-06T05:22:07.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.276+0000 7f6945ffb700 1 -- 172.21.15.31:0/2526636395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f694801e3b0 con 0x7f695015ca40 2023-10-06T05:22:07.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.276+0000 7f6945ffb700 1 --2- 172.21.15.31:0/2526636395 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6934067050 0x7f6934069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.276+0000 7f6945ffb700 1 -- 172.21.15.31:0/2526636395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6948084fe0 con 0x7f695015ca40 2023-10-06T05:22:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.277+0000 7f6955f9b700 1 --2- 172.21.15.31:0/2526636395 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6934067050 0x7f6934069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.278+0000 7f6955f9b700 1 --2- 172.21.15.31:0/2526636395 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6934067050 0x7f6934069500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f695006c620 tx=0x7f694c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.280+0000 7f6945ffb700 1 -- 172.21.15.31:0/2526636395 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6948051e50 con 0x7f695015ca40 2023-10-06T05:22:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:Invalid command: invalid chars #@ in abcd@# 2023-10-06T05:22:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:fs volume create [] : Create a CephFS volume 2023-10-06T05:22:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2023-10-06T05:22:07.519 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-06T05:22:07.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2023-10-06T05:22:07.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.715+0000 7f7c87dff700 1 Processor -- start 2023-10-06T05:22:07.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.715+0000 7f7c87dff700 1 -- start start 2023-10-06T05:22:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.716+0000 7f7c87dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c80159560 0x7f7c800cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.716+0000 7f7c87dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8015d2d0 con 0x7f7c80159930 2023-10-06T05:22:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.716+0000 7f7c87dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8015d5b0 con 0x7f7c800b2b80 2023-10-06T05:22:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.716+0000 7f7c87dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8015d890 con 0x7f7c80159560 2023-10-06T05:22:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.716+0000 7f7c85b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c80159560 0x7f7c800cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.717+0000 7f7c85b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c80159560 0x7f7c800cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48142/0 (socket says 172.21.15.31:48142) 2023-10-06T05:22:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.717+0000 7f7c85b9b700 1 -- 172.21.15.31:0/3494483949 learned_addr learned my addr 172.21.15.31:0/3494483949 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.717+0000 7f7c8539a700 1 --1- 172.21.15.31:0/3494483949 >> v1:172.21.15.31:6789/0 conn(0x7f7c80159930 0x7f7c800b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46118/0 (socket says 172.21.15.31:46118) 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.717+0000 7f7c85b9b700 1 -- 172.21.15.31:0/3494483949 >> v1:172.21.15.62:6789/0 conn(0x7f7c800b2b80 legacy=0x7f7c800cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.717+0000 7f7c85b9b700 1 -- 172.21.15.31:0/3494483949 >> v1:172.21.15.31:6789/0 conn(0x7f7c80159930 legacy=0x7f7c800b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.717+0000 7f7c85b9b700 1 -- 172.21.15.31:0/3494483949 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8015db70 con 0x7f7c80159560 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.717+0000 7f7c85b9b700 1 --2- 172.21.15.31:0/3494483949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 0x7f7c800cce70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0099b0 tx=0x7f7c7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c62932345cb2889 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.717+0000 7f7c84b99700 1 -- 172.21.15.31:0/3494483949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c00b860 con 0x7f7c80159560 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.717+0000 7f7c84b99700 1 -- 172.21.15.31:0/3494483949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c00b9c0 con 0x7f7c80159560 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.717+0000 7f7c84b99700 1 -- 172.21.15.31:0/3494483949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c00bd60 con 0x7f7c80159560 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.718+0000 7f7c87dff700 1 -- 172.21.15.31:0/3494483949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 msgr2=0x7f7c800cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.718+0000 7f7c87dff700 1 --2- 172.21.15.31:0/3494483949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 0x7f7c800cce70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0099b0 tx=0x7f7c7c009250 comp rx=0 tx=0).stop 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.718+0000 7f7c87dff700 1 -- 172.21.15.31:0/3494483949 shutdown_connections 2023-10-06T05:22:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.718+0000 7f7c87dff700 1 --2- 172.21.15.31:0/3494483949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 0x7f7c800cce70 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.718+0000 7f7c87dff700 1 -- 172.21.15.31:0/3494483949 >> 172.21.15.31:0/3494483949 conn(0x7f7c8006ec90 msgr2=0x7f7c8006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:07.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.718+0000 7f7c87dff700 1 -- 172.21.15.31:0/3494483949 shutdown_connections 2023-10-06T05:22:07.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.718+0000 7f7c87dff700 1 -- 172.21.15.31:0/3494483949 wait complete. 2023-10-06T05:22:07.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.720+0000 7f7c87dff700 1 Processor -- start 2023-10-06T05:22:07.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.720+0000 7f7c87dff700 1 -- start start 2023-10-06T05:22:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c87dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c800b2b80 0x7f7c8016d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c87dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 0x7f7c800b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c87dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c80159930 0x7f7c800c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c87dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8016e970 con 0x7f7c80159560 2023-10-06T05:22:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c87dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800c65e0 con 0x7f7c800b2b80 2023-10-06T05:22:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c87dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8016e0d0 con 0x7f7c80159930 2023-10-06T05:22:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c8539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 0x7f7c800b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c85b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c800b2b80 0x7f7c8016d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c8539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 0x7f7c800b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48146/0 (socket says 172.21.15.31:48146) 2023-10-06T05:22:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c8539a700 1 -- 172.21.15.31:0/1499779133 learned_addr learned my addr 172.21.15.31:0/1499779133 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c8639c700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c80159930 0x7f7c800c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c8539a700 1 -- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c80159930 msgr2=0x7f7c800c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c8539a700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c80159930 0x7f7c800c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c8539a700 1 -- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c800b2b80 msgr2=0x7f7c8016d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:07.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c8539a700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c800b2b80 0x7f7c8016d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:07.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.722+0000 7f7c8539a700 1 -- 172.21.15.31:0/1499779133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c0094f0 con 0x7f7c80159560 2023-10-06T05:22:07.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.723+0000 7f7c85b9b700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c800b2b80 0x7f7c8016d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:07.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.723+0000 7f7c8539a700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 0x7f7c800b3ee0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c560 tx=0x7f7c70007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.723+0000 7f7c75ffb700 1 -- 172.21.15.31:0/1499779133 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70005a70 con 0x7f7c80159560 2023-10-06T05:22:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.723+0000 7f7c75ffb700 1 -- 172.21.15.31:0/1499779133 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c70005bd0 con 0x7f7c80159560 2023-10-06T05:22:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.723+0000 7f7c75ffb700 1 -- 172.21.15.31:0/1499779133 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700070d0 con 0x7f7c80159560 2023-10-06T05:22:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.723+0000 7f7c87dff700 1 -- 172.21.15.31:0/1499779133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8016e3b0 con 0x7f7c80159560 2023-10-06T05:22:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.723+0000 7f7c87dff700 1 -- 172.21.15.31:0/1499779133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c800c5060 con 0x7f7c80159560 2023-10-06T05:22:07.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.724+0000 7f7c87dff700 1 -- 172.21.15.31:0/1499779133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c800c5280 con 0x7f7c80159560 2023-10-06T05:22:07.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.725+0000 7f7c75ffb700 1 -- 172.21.15.31:0/1499779133 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f7c70007480 con 0x7f7c80159560 2023-10-06T05:22:07.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.726+0000 7f7c75ffb700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7c64067050 0x7f7c64069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:07.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.726+0000 7f7c75ffb700 1 -- 172.21.15.31:0/1499779133 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7c70083b70 con 0x7f7c80159560 2023-10-06T05:22:07.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.727+0000 7f7c85b9b700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7c64067050 0x7f7c64069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:07.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.727+0000 7f7c85b9b700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7c64067050 0x7f7c64069500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0099b0 tx=0x7f7c7c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:07.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.730+0000 7f7c75ffb700 1 -- 172.21.15.31:0/1499779133 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c70050b70 con 0x7f7c80159560 2023-10-06T05:22:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:07.958+0000 7f7c87dff700 1 -- 172.21.15.31:0/1499779133 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c8004c380 con 0x7f7c64067050 2023-10-06T05:22:10.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.171+0000 7f7c75ffb700 1 -- 172.21.15.31:0/1499779133 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f7c8004c380 con 0x7f7c64067050 2023-10-06T05:22:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.178+0000 7f7c87dff700 1 -- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7c64067050 msgr2=0x7f7c64069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.178+0000 7f7c87dff700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7c64067050 0x7f7c64069500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0099b0 tx=0x7f7c7c017000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.178+0000 7f7c87dff700 1 -- 172.21.15.31:0/1499779133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 msgr2=0x7f7c800b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.178+0000 7f7c87dff700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 0x7f7c800b3ee0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c560 tx=0x7f7c70007f70 comp rx=0 tx=0).stop 2023-10-06T05:22:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.179+0000 7f7c87dff700 1 -- 172.21.15.31:0/1499779133 shutdown_connections 2023-10-06T05:22:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.179+0000 7f7c87dff700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7c64067050 0x7f7c64069500 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0099b0 tx=0x7f7c7c017000 comp rx=0 tx=0).stop 2023-10-06T05:22:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.179+0000 7f7c87dff700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c80159930 0x7f7c800c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.179+0000 7f7c87dff700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c800b2b80 0x7f7c8016d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.179+0000 7f7c87dff700 1 --2- 172.21.15.31:0/1499779133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c80159560 0x7f7c800b3ee0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.179+0000 7f7c87dff700 1 -- 172.21.15.31:0/1499779133 >> 172.21.15.31:0/1499779133 conn(0x7f7c8006ec90 msgr2=0x7f7c80159d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.181+0000 7f7c87dff700 1 -- 172.21.15.31:0/1499779133 shutdown_connections 2023-10-06T05:22:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.181+0000 7f7c87dff700 1 -- 172.21.15.31:0/1499779133 wait complete. 2023-10-06T05:22:10.192 INFO:teuthology.orchestra.run.smithi031.stderr:Volume created successfully (no MDS daemons created) 2023-10-06T05:22:10.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2023-10-06T05:22:10.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.414+0000 7f8ce43ff700 1 Processor -- start 2023-10-06T05:22:10.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.414+0000 7f8ce43ff700 1 -- start start 2023-10-06T05:22:10.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.415+0000 7f8ce43ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8cdc06a300 0x7f8cdc068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.415+0000 7f8ce43ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cdc068da0 con 0x7f8cdc0c8f40 2023-10-06T05:22:10.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.415+0000 7f8ce43ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cdc069080 con 0x7f8cdc15ca40 2023-10-06T05:22:10.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.415+0000 7f8ce43ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cdc0cabc0 con 0x7f8cdc06a300 2023-10-06T05:22:10.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.415+0000 7f8ce199a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8cdc0c8f40 0x7f8cdc0adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46134/0 (socket says 172.21.15.31:46134) 2023-10-06T05:22:10.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.416+0000 7f8ce199a700 1 -- 172.21.15.31:0/2075583832 learned_addr learned my addr 172.21.15.31:0/2075583832 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:10.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.416+0000 7f8ce299c700 1 --2- 172.21.15.31:0/2075583832 >> v2:172.21.15.31:3300/0 conn(0x7f8cdc06a300 0x7f8cdc068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.416+0000 7f8ce1199700 1 -- 172.21.15.31:0/2075583832 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809350433 0 0) 0x7f8cdc068da0 con 0x7f8cdc0c8f40 2023-10-06T05:22:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.416+0000 7f8ce1199700 1 -- 172.21.15.31:0/2075583832 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cc8003570 con 0x7f8cdc0c8f40 2023-10-06T05:22:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.416+0000 7f8ce299c700 1 -- 172.21.15.31:0/2075583832 >> v1:172.21.15.62:6789/0 conn(0x7f8cdc15ca40 legacy=0x7f8cdc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.416+0000 7f8ce299c700 1 -- 172.21.15.31:0/2075583832 >> v1:172.21.15.31:6789/0 conn(0x7f8cdc0c8f40 legacy=0x7f8cdc0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.416+0000 7f8ce299c700 1 -- 172.21.15.31:0/2075583832 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc069360 con 0x7f8cdc06a300 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.416+0000 7f8ce299c700 1 --2- 172.21.15.31:0/2075583832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cdc06a300 0x7f8cdc068670 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009e60 tx=0x7f8cd4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398bd53030842fd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.417+0000 7f8ce1199700 1 -- 172.21.15.31:0/2075583832 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd400e070 con 0x7f8cdc06a300 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.417+0000 7f8ce1199700 1 -- 172.21.15.31:0/2075583832 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd400bb60 con 0x7f8cdc06a300 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.417+0000 7f8ce1199700 1 -- 172.21.15.31:0/2075583832 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4012430 con 0x7f8cdc06a300 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.417+0000 7f8ce43ff700 1 -- 172.21.15.31:0/2075583832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cdc06a300 msgr2=0x7f8cdc068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.417+0000 7f8ce43ff700 1 --2- 172.21.15.31:0/2075583832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cdc06a300 0x7f8cdc068670 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009e60 tx=0x7f8cd4009780 comp rx=0 tx=0).stop 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.417+0000 7f8ce43ff700 1 -- 172.21.15.31:0/2075583832 shutdown_connections 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.417+0000 7f8ce43ff700 1 --2- 172.21.15.31:0/2075583832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cdc06a300 0x7f8cdc068670 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.417+0000 7f8ce43ff700 1 -- 172.21.15.31:0/2075583832 >> 172.21.15.31:0/2075583832 conn(0x7f8cdc0ce290 msgr2=0x7f8cdc0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.417+0000 7f8ce43ff700 1 -- 172.21.15.31:0/2075583832 shutdown_connections 2023-10-06T05:22:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.417+0000 7f8ce43ff700 1 -- 172.21.15.31:0/2075583832 wait complete. 2023-10-06T05:22:10.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.419+0000 7f8ce43ff700 1 Processor -- start 2023-10-06T05:22:10.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.419+0000 7f8ce43ff700 1 -- start start 2023-10-06T05:22:10.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce43ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cdc06a300 0x7f8cdc0c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce43ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cdc0c8f40 0x7f8cdc0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce43ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cdc15ca40 0x7f8cdc0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce43ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0b10e0 con 0x7f8cdc15ca40 2023-10-06T05:22:10.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce43ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0c4120 con 0x7f8cdc0c8f40 2023-10-06T05:22:10.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce43ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0c4400 con 0x7f8cdc06a300 2023-10-06T05:22:10.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce299c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cdc15ca40 0x7f8cdc0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce299c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cdc15ca40 0x7f8cdc0c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48174/0 (socket says 172.21.15.31:48174) 2023-10-06T05:22:10.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce299c700 1 -- 172.21.15.31:0/3722153157 learned_addr learned my addr 172.21.15.31:0/3722153157 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:10.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce199a700 1 --2- 172.21.15.31:0/3722153157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cdc0c8f40 0x7f8cdc0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce219b700 1 --2- 172.21.15.31:0/3722153157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cdc06a300 0x7f8cdc0c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.420+0000 7f8ce299c700 1 -- 172.21.15.31:0/3722153157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cdc06a300 msgr2=0x7f8cdc0c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.421+0000 7f8ce299c700 1 --2- 172.21.15.31:0/3722153157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cdc06a300 0x7f8cdc0c36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.421+0000 7f8ce299c700 1 -- 172.21.15.31:0/3722153157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cdc0c8f40 msgr2=0x7f8cdc0c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.421+0000 7f8ce299c700 1 --2- 172.21.15.31:0/3722153157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cdc0c8f40 0x7f8cdc0c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.421+0000 7f8ce299c700 1 -- 172.21.15.31:0/3722153157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4009a20 con 0x7f8cdc15ca40 2023-10-06T05:22:10.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.421+0000 7f8ce299c700 1 --2- 172.21.15.31:0/3722153157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cdc15ca40 0x7f8cdc0c6da0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc06e040 tx=0x7f8cd4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.421+0000 7f8cd27fc700 1 -- 172.21.15.31:0/3722153157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4027470 con 0x7f8cdc15ca40 2023-10-06T05:22:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.421+0000 7f8cd27fc700 1 -- 172.21.15.31:0/3722153157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd40275f0 con 0x7f8cdc15ca40 2023-10-06T05:22:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.421+0000 7f8ce43ff700 1 -- 172.21.15.31:0/3722153157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cdc0b1380 con 0x7f8cdc15ca40 2023-10-06T05:22:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.421+0000 7f8cd27fc700 1 -- 172.21.15.31:0/3722153157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40279b0 con 0x7f8cdc15ca40 2023-10-06T05:22:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.421+0000 7f8ce43ff700 1 -- 172.21.15.31:0/3722153157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cdc0c7b30 con 0x7f8cdc15ca40 2023-10-06T05:22:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.422+0000 7f8ce43ff700 1 -- 172.21.15.31:0/3722153157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cdc0b15c0 con 0x7f8cdc15ca40 2023-10-06T05:22:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.423+0000 7f8cd27fc700 1 -- 172.21.15.31:0/3722153157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f8cd4024070 con 0x7f8cdc15ca40 2023-10-06T05:22:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.424+0000 7f8cd27fc700 1 --2- 172.21.15.31:0/3722153157 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8cc0067050 0x7f8cc0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.424+0000 7f8cd27fc700 1 -- 172.21.15.31:0/3722153157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f8cd40874d0 con 0x7f8cdc15ca40 2023-10-06T05:22:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.425+0000 7f8ce219b700 1 --2- 172.21.15.31:0/3722153157 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8cc0067050 0x7f8cc0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.425+0000 7f8ce219b700 1 --2- 172.21.15.31:0/3722153157 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8cc0067050 0x7f8cc0069500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc06bf30 tx=0x7f8cd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.429+0000 7f8cd27fc700 1 -- 172.21.15.31:0/3722153157 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cdc0b15c0 con 0x7f8cdc15ca40 2023-10-06T05:22:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:Invalid command: invalid chars #@ in abcd@# 2023-10-06T05:22:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2023-10-06T05:22:10.667 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2023-10-06T05:22:10.673 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-06T05:22:10.674 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2023-10-06T05:22:10.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.870+0000 7f32a33ff700 1 Processor -- start 2023-10-06T05:22:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.870+0000 7f32a33ff700 1 -- start start 2023-10-06T05:22:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a33ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f329c0ae860 0x7f329c157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a33ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f329c0b1260 0x7f329c0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a33ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329c158cf0 con 0x7f329c1573a0 2023-10-06T05:22:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a33ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329c0b1c40 con 0x7f329c0b1260 2023-10-06T05:22:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a33ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329c1580e0 con 0x7f329c0ae860 2023-10-06T05:22:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a199c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f329c0b1260 0x7f329c0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a199c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f329c0b1260 0x7f329c0b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48184/0 (socket says 172.21.15.31:48184) 2023-10-06T05:22:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a199c700 1 -- 172.21.15.31:0/4133082861 learned_addr learned my addr 172.21.15.31:0/4133082861 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a199c700 1 -- 172.21.15.31:0/4133082861 >> v2:172.21.15.62:3300/0 conn(0x7f329c0ae860 msgr2=0x7f329c157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a099a700 1 --2- 172.21.15.31:0/4133082861 >> v2:172.21.15.62:3300/0 conn(0x7f329c0ae860 0x7f329c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a199c700 1 --2- 172.21.15.31:0/4133082861 >> v2:172.21.15.62:3300/0 conn(0x7f329c0ae860 0x7f329c157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a199c700 1 -- 172.21.15.31:0/4133082861 >> v1:172.21.15.62:6789/0 conn(0x7f329c1573a0 legacy=0x7f329c0acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a199c700 1 -- 172.21.15.31:0/4133082861 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c1583c0 con 0x7f329c0b1260 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.872+0000 7f32a099a700 1 --2- 172.21.15.31:0/4133082861 >> v2:172.21.15.62:3300/0 conn(0x7f329c0ae860 0x7f329c157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.873+0000 7f32a199c700 1 --2- 172.21.15.31:0/4133082861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329c0b1260 0x7f329c0b1700 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f329c0c9c30 tx=0x7f3294009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55d7b10a302c1ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.873+0000 7f3293fff700 1 -- 172.21.15.31:0/4133082861 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400e070 con 0x7f329c0b1260 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.873+0000 7f3293fff700 1 -- 172.21.15.31:0/4133082861 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329400ba10 con 0x7f329c0b1260 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.873+0000 7f3293fff700 1 -- 172.21.15.31:0/4133082861 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32940092e0 con 0x7f329c0b1260 2023-10-06T05:22:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.873+0000 7f32a33ff700 1 -- 172.21.15.31:0/4133082861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329c0b1260 msgr2=0x7f329c0b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.873+0000 7f32a33ff700 1 --2- 172.21.15.31:0/4133082861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329c0b1260 0x7f329c0b1700 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f329c0c9c30 tx=0x7f3294009840 comp rx=0 tx=0).stop 2023-10-06T05:22:10.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.873+0000 7f32a33ff700 1 -- 172.21.15.31:0/4133082861 shutdown_connections 2023-10-06T05:22:10.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.873+0000 7f32a33ff700 1 --2- 172.21.15.31:0/4133082861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329c0b1260 0x7f329c0b1700 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.873+0000 7f32a33ff700 1 --2- 172.21.15.31:0/4133082861 >> v2:172.21.15.62:3300/0 conn(0x7f329c0ae860 0x7f329c157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.873+0000 7f32a33ff700 1 -- 172.21.15.31:0/4133082861 >> 172.21.15.31:0/4133082861 conn(0x7f329c067560 msgr2=0x7f329c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:10.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.874+0000 7f32a33ff700 1 -- 172.21.15.31:0/4133082861 shutdown_connections 2023-10-06T05:22:10.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.874+0000 7f32a33ff700 1 -- 172.21.15.31:0/4133082861 wait complete. 2023-10-06T05:22:10.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.876+0000 7f32a33ff700 1 Processor -- start 2023-10-06T05:22:10.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.876+0000 7f32a33ff700 1 -- start start 2023-10-06T05:22:10.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.877+0000 7f32a33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f329c0b1260 0x7f329c0aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.877+0000 7f32a33ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c1573a0 0x7f329c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.877+0000 7f32a33ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329c155290 0x7f329c0a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.877+0000 7f32a33ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0a7370 con 0x7f329c155290 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.877+0000 7f32a33ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c155730 con 0x7f329c1573a0 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.877+0000 7f32a33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0a67f0 con 0x7f329c0b1260 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.877+0000 7f32a199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329c155290 0x7f329c0a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.877+0000 7f32a199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329c155290 0x7f329c0a7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48194/0 (socket says 172.21.15.31:48194) 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.877+0000 7f32a199c700 1 -- 172.21.15.31:0/1717626479 learned_addr learned my addr 172.21.15.31:0/1717626479 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.877+0000 7f32a119b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f329c0b1260 0x7f329c0aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32a099a700 1 --2- 172.21.15.31:0/1717626479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c1573a0 0x7f329c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32a199c700 1 -- 172.21.15.31:0/1717626479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f329c0b1260 msgr2=0x7f329c0aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32a199c700 1 --2- 172.21.15.31:0/1717626479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f329c0b1260 0x7f329c0aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32a199c700 1 -- 172.21.15.31:0/1717626479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c1573a0 msgr2=0x7f329c0a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:10.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32a199c700 1 --2- 172.21.15.31:0/1717626479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c1573a0 0x7f329c0a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:10.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32a199c700 1 -- 172.21.15.31:0/1717626479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294009ae0 con 0x7f329c155290 2023-10-06T05:22:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32a199c700 1 --2- 172.21.15.31:0/1717626479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329c155290 0x7f329c0a7f40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f329c064290 tx=0x7f3294017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32917fa700 1 -- 172.21.15.31:0/1717626479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32940032f0 con 0x7f329c155290 2023-10-06T05:22:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32917fa700 1 -- 172.21.15.31:0/1717626479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3294012d00 con 0x7f329c155290 2023-10-06T05:22:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32917fa700 1 -- 172.21.15.31:0/1717626479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329401d670 con 0x7f329c155290 2023-10-06T05:22:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.878+0000 7f32a33ff700 1 -- 172.21.15.31:0/1717626479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c0a6ad0 con 0x7f329c155290 2023-10-06T05:22:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.879+0000 7f32a33ff700 1 -- 172.21.15.31:0/1717626479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c0a7090 con 0x7f329c155290 2023-10-06T05:22:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.880+0000 7f32a33ff700 1 -- 172.21.15.31:0/1717626479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c15c140 con 0x7f329c155290 2023-10-06T05:22:10.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.881+0000 7f32917fa700 1 -- 172.21.15.31:0/1717626479 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f329401daa0 con 0x7f329c155290 2023-10-06T05:22:10.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.882+0000 7f32917fa700 1 --2- 172.21.15.31:0/1717626479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3280067050 0x7f3280069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:10.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.882+0000 7f32917fa700 1 -- 172.21.15.31:0/1717626479 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f32940863b0 con 0x7f329c155290 2023-10-06T05:22:10.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.882+0000 7f32a119b700 1 --2- 172.21.15.31:0/1717626479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3280067050 0x7f3280069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:10.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.883+0000 7f32a119b700 1 --2- 172.21.15.31:0/1717626479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3280067050 0x7f3280069500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3298009af0 tx=0x7f3298009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:10.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:10.886+0000 7f32917fa700 1 -- 172.21.15.31:0/1717626479 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3294052f60 con 0x7f329c155290 2023-10-06T05:22:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:Invalid command: invalid chars @# in abcd@# 2023-10-06T05:22:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2023-10-06T05:22:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2023-10-06T05:22:11.134 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-06T05:22:11.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2023-10-06T05:22:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.337+0000 7fd674dff700 1 Processor -- start 2023-10-06T05:22:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.337+0000 7fd674dff700 1 -- start start 2023-10-06T05:22:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.338+0000 7fd674dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd670159560 0x7fd6700cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.338+0000 7fd674dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6700b2b80 0x7fd6700cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.338+0000 7fd674dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67015d2d0 con 0x7fd670159930 2023-10-06T05:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.338+0000 7fd674dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67015d5b0 con 0x7fd670159560 2023-10-06T05:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.338+0000 7fd674dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67015d890 con 0x7fd6700b2b80 2023-10-06T05:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.338+0000 7fd66e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd670159560 0x7fd6700cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.338+0000 7fd66e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd670159560 0x7fd6700cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48196/0 (socket says 172.21.15.31:48196) 2023-10-06T05:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.338+0000 7fd66e59c700 1 -- 172.21.15.31:0/2605430251 learned_addr learned my addr 172.21.15.31:0/2605430251 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.338+0000 7fd66dd9b700 1 --1- 172.21.15.31:0/2605430251 >> v1:172.21.15.31:6789/0 conn(0x7fd670159930 0x7fd6700b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46140/0 (socket says 172.21.15.31:46140) 2023-10-06T05:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.339+0000 7fd66ed9d700 1 --2- 172.21.15.31:0/2605430251 >> v2:172.21.15.62:3300/0 conn(0x7fd6700b2b80 0x7fd6700cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.339+0000 7fd66e59c700 1 -- 172.21.15.31:0/2605430251 >> v2:172.21.15.62:3300/0 conn(0x7fd6700b2b80 msgr2=0x7fd6700cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:11.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.339+0000 7fd66e59c700 1 --2- 172.21.15.31:0/2605430251 >> v2:172.21.15.62:3300/0 conn(0x7fd6700b2b80 0x7fd6700cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.339+0000 7fd66e59c700 1 -- 172.21.15.31:0/2605430251 >> v1:172.21.15.31:6789/0 conn(0x7fd670159930 legacy=0x7fd6700b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.339+0000 7fd66d59a700 1 -- 172.21.15.31:0/2605430251 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201043605 0 0) 0x7fd67015d2d0 con 0x7fd670159930 2023-10-06T05:22:11.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.339+0000 7fd66e59c700 1 -- 172.21.15.31:0/2605430251 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67015db70 con 0x7fd670159560 2023-10-06T05:22:11.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.339+0000 7fd66e59c700 1 --2- 172.21.15.31:0/2605430251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670159560 0x7fd6700cce70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd6700b1740 tx=0x7fd660009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a72b25016a2773 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:11.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.339+0000 7fd66d59a700 1 -- 172.21.15.31:0/2605430251 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66000b860 con 0x7fd670159560 2023-10-06T05:22:11.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.339+0000 7fd66d59a700 1 -- 172.21.15.31:0/2605430251 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66000b9c0 con 0x7fd670159560 2023-10-06T05:22:11.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.339+0000 7fd66d59a700 1 -- 172.21.15.31:0/2605430251 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66000bd60 con 0x7fd670159560 2023-10-06T05:22:11.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.340+0000 7fd674dff700 1 -- 172.21.15.31:0/2605430251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670159560 msgr2=0x7fd6700cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.340+0000 7fd674dff700 1 --2- 172.21.15.31:0/2605430251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670159560 0x7fd6700cce70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd6700b1740 tx=0x7fd660009250 comp rx=0 tx=0).stop 2023-10-06T05:22:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.340+0000 7fd674dff700 1 -- 172.21.15.31:0/2605430251 shutdown_connections 2023-10-06T05:22:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.340+0000 7fd674dff700 1 --2- 172.21.15.31:0/2605430251 >> v2:172.21.15.62:3300/0 conn(0x7fd6700b2b80 0x7fd6700cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.340+0000 7fd674dff700 1 --2- 172.21.15.31:0/2605430251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670159560 0x7fd6700cce70 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.340+0000 7fd674dff700 1 -- 172.21.15.31:0/2605430251 >> 172.21.15.31:0/2605430251 conn(0x7fd67006ec90 msgr2=0x7fd67006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.340+0000 7fd674dff700 1 -- 172.21.15.31:0/2605430251 shutdown_connections 2023-10-06T05:22:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.340+0000 7fd674dff700 1 -- 172.21.15.31:0/2605430251 wait complete. 2023-10-06T05:22:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.342+0000 7fd674dff700 1 Processor -- start 2023-10-06T05:22:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.342+0000 7fd674dff700 1 -- start start 2023-10-06T05:22:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd674dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6700b2b80 0x7fd6700d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd674dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd670159560 0x7fd6700c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd674dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd670159930 0x7fd6700c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd674dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700cff20 con 0x7fd6700b2b80 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd674dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700cf940 con 0x7fd670159560 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd674dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700c6d00 con 0x7fd670159930 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd66e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6700b2b80 0x7fd6700d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd66e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6700b2b80 0x7fd6700d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48198/0 (socket says 172.21.15.31:48198) 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd66e59c700 1 -- 172.21.15.31:0/1996003299 learned_addr learned my addr 172.21.15.31:0/1996003299 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd66dd9b700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd670159560 0x7fd6700c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd66ed9d700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd670159930 0x7fd6700c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd66e59c700 1 -- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd670159930 msgr2=0x7fd6700c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd66e59c700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd670159930 0x7fd6700c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd66e59c700 1 -- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd670159560 msgr2=0x7fd6700c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd66e59c700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd670159560 0x7fd6700c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:11.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.343+0000 7fd66e59c700 1 -- 172.21.15.31:0/1996003299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6600094f0 con 0x7fd6700b2b80 2023-10-06T05:22:11.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.344+0000 7fd66e59c700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6700b2b80 0x7fd6700d0340 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd660009290 tx=0x7fd660017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:11.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.344+0000 7fd65e7fc700 1 -- 172.21.15.31:0/1996003299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660024430 con 0x7fd6700b2b80 2023-10-06T05:22:11.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.344+0000 7fd65e7fc700 1 -- 172.21.15.31:0/1996003299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd660024590 con 0x7fd6700b2b80 2023-10-06T05:22:11.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.344+0000 7fd65e7fc700 1 -- 172.21.15.31:0/1996003299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660024950 con 0x7fd6700b2b80 2023-10-06T05:22:11.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.344+0000 7fd674dff700 1 -- 172.21.15.31:0/1996003299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6700c6fe0 con 0x7fd6700b2b80 2023-10-06T05:22:11.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.344+0000 7fd674dff700 1 -- 172.21.15.31:0/1996003299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6700d0880 con 0x7fd6700b2b80 2023-10-06T05:22:11.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.345+0000 7fd674dff700 1 -- 172.21.15.31:0/1996003299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6700d0a70 con 0x7fd6700b2b80 2023-10-06T05:22:11.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.346+0000 7fd65e7fc700 1 -- 172.21.15.31:0/1996003299 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd66001a420 con 0x7fd6700b2b80 2023-10-06T05:22:11.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.348+0000 7fd65e7fc700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd650067050 0x7fd650069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:11.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.348+0000 7fd65e7fc700 1 -- 172.21.15.31:0/1996003299 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6892+0+0 (secure 0 0 0) 0x7fd660086710 con 0x7fd6700b2b80 2023-10-06T05:22:11.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.348+0000 7fd66dd9b700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd650067050 0x7fd650069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:11.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.349+0000 7fd66dd9b700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd650067050 0x7fd650069500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd6700d0f20 tx=0x7fd664007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.352+0000 7fd65e7fc700 1 -- 172.21.15.31:0/1996003299 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd660053450 con 0x7fd6700b2b80 2023-10-06T05:22:11.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:11.583+0000 7fd674dff700 1 -- 172.21.15.31:0/1996003299 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6700cfd00 con 0x7fd650067050 2023-10-06T05:22:11.590 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:22:11.588+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:11.633 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:22:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.610+0000 7fd65e7fc700 1 -- 172.21.15.31:0/1996003299 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+78 (secure 0 0 0) 0x7fd6700cfd00 con 0x7fd650067050 2023-10-06T05:22:14.612 INFO:teuthology.orchestra.run.smithi031.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2023-10-06T05:22:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 -- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd650067050 msgr2=0x7fd650069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd650067050 0x7fd650069500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd6700d0f20 tx=0x7fd664007ef0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 -- 172.21.15.31:0/1996003299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6700b2b80 msgr2=0x7fd6700d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6700b2b80 0x7fd6700d0340 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd660009290 tx=0x7fd660017000 comp rx=0 tx=0).stop 2023-10-06T05:22:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 -- 172.21.15.31:0/1996003299 shutdown_connections 2023-10-06T05:22:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd650067050 0x7fd650069500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd670159930 0x7fd6700c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6700b2b80 0x7fd6700d0340 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 --2- 172.21.15.31:0/1996003299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd670159560 0x7fd6700c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 -- 172.21.15.31:0/1996003299 >> 172.21.15.31:0/1996003299 conn(0x7fd67006ec90 msgr2=0x7fd6700ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.615+0000 7fd674dff700 1 -- 172.21.15.31:0/1996003299 shutdown_connections 2023-10-06T05:22:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:14.616+0000 7fd674dff700 1 -- 172.21.15.31:0/1996003299 wait complete. 2023-10-06T05:22:14.638 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:22:14.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:14.739 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:14.845 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:22:14.845 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:22:14.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:14.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:14.937 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:22:14.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:14.938 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:22:14.984 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:22:14.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:14.985 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:15.086 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:22:15.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:15.169 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:22:15.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.411+0000 7f2542fff700 1 Processor -- start 2023-10-06T05:22:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.411+0000 7f2542fff700 1 -- start start 2023-10-06T05:22:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.413+0000 7f2542fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f253c06b510 0x7f253c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.413+0000 7f2542fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 0x7f253c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.413+0000 7f2542fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c156ef0 con 0x7f253c0ca770 2023-10-06T05:22:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.413+0000 7f2542fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c0c7a70 con 0x7f253c06b510 2023-10-06T05:22:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.413+0000 7f2542fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0c6df0 con 0x7f253c0cab40 2023-10-06T05:22:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.413+0000 7f253bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f253c06b510 0x7f253c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.413+0000 7f254159c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 0x7f253c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.413+0000 7f2540d9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f253c0ca770 0x7f253c06ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:49482/0 (socket says 172.21.15.31:49482) 2023-10-06T05:22:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.413+0000 7f2540d9b700 1 -- 172.21.15.31:0/2993062987 learned_addr learned my addr 172.21.15.31:0/2993062987 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.414+0000 7f254159c700 1 -- 172.21.15.31:0/2993062987 >> v2:172.21.15.62:3300/0 conn(0x7f253c06b510 msgr2=0x7f253c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.414+0000 7f254159c700 1 --2- 172.21.15.31:0/2993062987 >> v2:172.21.15.62:3300/0 conn(0x7f253c06b510 0x7f253c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.414+0000 7f253b7fe700 1 -- 172.21.15.31:0/2993062987 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004844061 0 0) 0x7f253c156ef0 con 0x7f253c0ca770 2023-10-06T05:22:15.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.414+0000 7f254159c700 1 -- 172.21.15.31:0/2993062987 >> v1:172.21.15.62:6789/0 conn(0x7f253c0ca770 legacy=0x7f253c06ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.414+0000 7f254159c700 1 -- 172.21.15.31:0/2993062987 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0c70d0 con 0x7f253c0cab40 2023-10-06T05:22:15.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.414+0000 7f254159c700 1 --2- 172.21.15.31:0/2993062987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 0x7f253c1569b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f253400af70 tx=0x7f253400a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5bbcd0022341032 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:15.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.414+0000 7f253b7fe700 1 -- 172.21.15.31:0/2993062987 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400f070 con 0x7f253c0cab40 2023-10-06T05:22:15.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.414+0000 7f253b7fe700 1 -- 172.21.15.31:0/2993062987 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253400a2e0 con 0x7f253c0cab40 2023-10-06T05:22:15.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.414+0000 7f253b7fe700 1 -- 172.21.15.31:0/2993062987 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400cc70 con 0x7f253c0cab40 2023-10-06T05:22:15.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.415+0000 7f2542fff700 1 -- 172.21.15.31:0/2993062987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 msgr2=0x7f253c1569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.415+0000 7f2542fff700 1 --2- 172.21.15.31:0/2993062987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 0x7f253c1569b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f253400af70 tx=0x7f253400a890 comp rx=0 tx=0).stop 2023-10-06T05:22:15.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.415+0000 7f2542fff700 1 -- 172.21.15.31:0/2993062987 shutdown_connections 2023-10-06T05:22:15.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.415+0000 7f2542fff700 1 --2- 172.21.15.31:0/2993062987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 0x7f253c1569b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.415+0000 7f2542fff700 1 --2- 172.21.15.31:0/2993062987 >> v2:172.21.15.62:3300/0 conn(0x7f253c06b510 0x7f253c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.415+0000 7f2542fff700 1 -- 172.21.15.31:0/2993062987 >> 172.21.15.31:0/2993062987 conn(0x7f253c0cc0b0 msgr2=0x7f253c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:15.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.415+0000 7f2542fff700 1 -- 172.21.15.31:0/2993062987 shutdown_connections 2023-10-06T05:22:15.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.415+0000 7f2542fff700 1 -- 172.21.15.31:0/2993062987 wait complete. 2023-10-06T05:22:15.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.416+0000 7f2542fff700 1 Processor -- start 2023-10-06T05:22:15.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.417+0000 7f2542fff700 1 -- start start 2023-10-06T05:22:15.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.417+0000 7f2542fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c06b510 0x7f253c14ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2542fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0ca770 0x7f253c0ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2542fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 0x7f253c06ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2542fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0ad620 con 0x7f253c06b510 2023-10-06T05:22:15.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2542fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0aca10 con 0x7f253c0ca770 2023-10-06T05:22:15.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2542fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0accf0 con 0x7f253c0cab40 2023-10-06T05:22:15.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2540d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c06b510 0x7f253c14ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2540d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c06b510 0x7f253c14ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48210/0 (socket says 172.21.15.31:48210) 2023-10-06T05:22:15.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2540d9b700 1 -- 172.21.15.31:0/2284254827 learned_addr learned my addr 172.21.15.31:0/2284254827 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:15.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f253bfff700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0ca770 0x7f253c0ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f254159c700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 0x7f253c06ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2540d9b700 1 -- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 msgr2=0x7f253c06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2540d9b700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 0x7f253c06ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2540d9b700 1 -- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0ca770 msgr2=0x7f253c0ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2540d9b700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0ca770 0x7f253c0ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.418+0000 7f2540d9b700 1 -- 172.21.15.31:0/2284254827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253400ab30 con 0x7f253c06b510 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.419+0000 7f2540d9b700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c06b510 0x7f253c14ef40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f253000c600 tx=0x7f2530007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.419+0000 7f2538ff9700 1 -- 172.21.15.31:0/2284254827 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530005c50 con 0x7f253c06b510 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.419+0000 7f2538ff9700 1 -- 172.21.15.31:0/2284254827 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2530005db0 con 0x7f253c06b510 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.419+0000 7f2542fff700 1 -- 172.21.15.31:0/2284254827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c0acfd0 con 0x7f253c06b510 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.419+0000 7f2538ff9700 1 -- 172.21.15.31:0/2284254827 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530007320 con 0x7f253c06b510 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.419+0000 7f2542fff700 1 -- 172.21.15.31:0/2284254827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c0ac600 con 0x7f253c06b510 2023-10-06T05:22:15.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.420+0000 7f2542fff700 1 -- 172.21.15.31:0/2284254827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c04c380 con 0x7f253c06b510 2023-10-06T05:22:15.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.421+0000 7f2538ff9700 1 -- 172.21.15.31:0/2284254827 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f25300076d0 con 0x7f253c06b510 2023-10-06T05:22:15.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.422+0000 7f2538ff9700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2520067050 0x7f2520069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.422+0000 7f2538ff9700 1 -- 172.21.15.31:0/2284254827 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f2530083d70 con 0x7f253c06b510 2023-10-06T05:22:15.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.422+0000 7f253bfff700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2520067050 0x7f2520069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.423+0000 7f253bfff700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2520067050 0x7f2520069500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f253c0b00a0 tx=0x7f25280091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:15.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.426+0000 7f2538ff9700 1 -- 172.21.15.31:0/2284254827 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2530050ca0 con 0x7f253c06b510 2023-10-06T05:22:15.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.674+0000 7f2542fff700 1 -- 172.21.15.31:0/2284254827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f253c0cbe20 con 0x7f253c06b510 2023-10-06T05:22:15.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.675+0000 7f2538ff9700 1 -- 172.21.15.31:0/2284254827 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f2530017070 con 0x7f253c06b510 2023-10-06T05:22:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:15.677 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1878570982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1878570982},{"type":"v1","addr":"172.21.15.62:6834","nonce":1878570982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6687,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/427382574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":427382574},{"type":"v1","addr":"172.21.15.31:6834","nonce":427382574}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1760549774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1760549774},{"type":"v1","addr":"172.21.15.31:6835","nonce":1760549774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:54.590687+0000","modified":"2023-10-06T05:21:58.111036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"d","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3654179987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3654179987},{"type":"v1","addr":"172.21.15.62:6835","nonce":3654179987}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:15.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.677+0000 7f2542fff700 1 -- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2520067050 msgr2=0x7f2520069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.677+0000 7f2542fff700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2520067050 0x7f2520069500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f253c0b00a0 tx=0x7f25280091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.677+0000 7f2542fff700 1 -- 172.21.15.31:0/2284254827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c06b510 msgr2=0x7f253c14ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.677+0000 7f2542fff700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c06b510 0x7f253c14ef40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f253000c600 tx=0x7f2530007d80 comp rx=0 tx=0).stop 2023-10-06T05:22:15.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.677+0000 7f2542fff700 1 -- 172.21.15.31:0/2284254827 shutdown_connections 2023-10-06T05:22:15.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.677+0000 7f2542fff700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2520067050 0x7f2520069500 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.677+0000 7f2542fff700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f253c0cab40 0x7f253c06ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.677+0000 7f2542fff700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f253c06b510 0x7f253c14ef40 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.677+0000 7f2542fff700 1 --2- 172.21.15.31:0/2284254827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f253c0ca770 0x7f253c0ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.677+0000 7f2542fff700 1 -- 172.21.15.31:0/2284254827 >> 172.21.15.31:0/2284254827 conn(0x7f253c0cc0b0 msgr2=0x7f253c1572e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:15.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.678+0000 7f2542fff700 1 -- 172.21.15.31:0/2284254827 shutdown_connections 2023-10-06T05:22:15.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.678+0000 7f2542fff700 1 -- 172.21.15.31:0/2284254827 wait complete. 2023-10-06T05:22:15.681 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 58 2023-10-06T05:22:15.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:15.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.892+0000 7faa0a7ff700 1 Processor -- start 2023-10-06T05:22:15.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.892+0000 7faa0a7ff700 1 -- start start 2023-10-06T05:22:15.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.894+0000 7faa0a7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faa0406a570 0x7faa04162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.894+0000 7faa0a7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa040c71b0 con 0x7faa0413a5b0 2023-10-06T05:22:15.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.894+0000 7faa0a7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa040c7490 con 0x7faa0413a1e0 2023-10-06T05:22:15.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.894+0000 7faa0a7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa040c7770 con 0x7faa0406a570 2023-10-06T05:22:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.894+0000 7faa037fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faa0413a5b0 0x7faa0406d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46148/0 (socket says 172.21.15.31:46148) 2023-10-06T05:22:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.894+0000 7faa037fe700 1 -- 172.21.15.31:0/2866467219 learned_addr learned my addr 172.21.15.31:0/2866467219 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.894+0000 7faa08d9c700 1 --2- 172.21.15.31:0/2866467219 >> v2:172.21.15.62:3300/0 conn(0x7faa0406a570 0x7faa04162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.894+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392322580 0 0) 0x7faa040c71b0 con 0x7faa0413a5b0 2023-10-06T05:22:15.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.894+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9ec003570 con 0x7faa0413a5b0 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 280245263 0 0) 0x7fa9ec003570 con 0x7faa0413a5b0 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 >> v2:172.21.15.62:3300/0 conn(0x7faa0406a570 msgr2=0x7faa04162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 --2- 172.21.15.31:0/2866467219 >> v2:172.21.15.62:3300/0 conn(0x7faa0406a570 0x7faa04162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 >> v1:172.21.15.62:6789/0 conn(0x7faa0413a1e0 legacy=0x7faa040c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa040c7a50 con 0x7faa0413a5b0 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fa9f8001470 con 0x7faa0413a5b0 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 >> v1:172.21.15.31:6789/0 conn(0x7faa0413a5b0 legacy=0x7faa0406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7fa9ec003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9ec004810 0x7fa9ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9ec005210 0x7fa9ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa040c7770 con 0x7fa9ec003ab0 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec00c130 con 0x7fa9ec005210 2023-10-06T05:22:15.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec00c2f0 con 0x7fa9ec004810 2023-10-06T05:22:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa03fff700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7fa9ec003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa08d9c700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9ec005210 0x7fa9ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.895+0000 7faa037fe700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9ec004810 0x7fa9ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.896+0000 7faa03fff700 1 -- 172.21.15.31:0/2866467219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9ec004810 msgr2=0x7fa9ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.896+0000 7faa03fff700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9ec004810 0x7fa9ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.896+0000 7faa03fff700 1 -- 172.21.15.31:0/2866467219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9ec005210 msgr2=0x7fa9ec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.896+0000 7faa03fff700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9ec005210 0x7fa9ec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.896+0000 7faa03fff700 1 -- 172.21.15.31:0/2866467219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec003570 con 0x7fa9ec003ab0 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.896+0000 7faa08d9c700 1 -- 172.21.15.31:0/2866467219 reap_dead start 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.896+0000 7faa03fff700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7fa9ec003ec0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40018d0 tx=0x7fa9f4007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ad2ed47b0a10f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.896+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4001290 con 0x7fa9ec003ab0 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.896+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f4012420 con 0x7fa9ec003ab0 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.896+0000 7faa02ffd700 1 -- 172.21.15.31:0/2866467219 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40127e0 con 0x7fa9ec003ab0 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.897+0000 7faa0a7ff700 1 -- 172.21.15.31:0/2866467219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 msgr2=0x7fa9ec003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.897+0000 7faa0a7ff700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7fa9ec003ec0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40018d0 tx=0x7fa9f4007d80 comp rx=0 tx=0).stop 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.897+0000 7faa0a7ff700 1 -- 172.21.15.31:0/2866467219 shutdown_connections 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.897+0000 7faa0a7ff700 1 --2- 172.21.15.31:0/2866467219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7fa9ec003ec0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.897+0000 7faa0a7ff700 1 -- 172.21.15.31:0/2866467219 >> 172.21.15.31:0/2866467219 conn(0x7faa040cc0b0 msgr2=0x7faa040cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:15.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.897+0000 7faa0a7ff700 1 -- 172.21.15.31:0/2866467219 shutdown_connections 2023-10-06T05:22:15.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.897+0000 7faa0a7ff700 1 -- 172.21.15.31:0/2866467219 wait complete. 2023-10-06T05:22:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.898+0000 7faa0a7ff700 1 Processor -- start 2023-10-06T05:22:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.899+0000 7faa0a7ff700 1 -- start start 2023-10-06T05:22:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.899+0000 7faa0a7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7faa04138b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.899+0000 7faa0a7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa0413aa70 0x7faa0413ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.899+0000 7faa0a7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa0413aee0 0x7faa0413d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.899+0000 7faa0a7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa04167990 con 0x7fa9ec003ab0 2023-10-06T05:22:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.899+0000 7faa0a7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041687f0 con 0x7faa0413aa70 2023-10-06T05:22:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.899+0000 7faa0a7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa04139050 con 0x7faa0413aee0 2023-10-06T05:22:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7faa04138b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7faa04138b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48242/0 (socket says 172.21.15.31:48242) 2023-10-06T05:22:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa03fff700 1 -- 172.21.15.31:0/38531557 learned_addr learned my addr 172.21.15.31:0/38531557 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa08d9c700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa0413aee0 0x7faa0413d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa037fe700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa0413aa70 0x7faa0413ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa03fff700 1 -- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa0413aee0 msgr2=0x7faa0413d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa03fff700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa0413aee0 0x7faa0413d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa03fff700 1 -- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa0413aa70 msgr2=0x7faa0413ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa03fff700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa0413aa70 0x7faa0413ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa03fff700 1 -- 172.21.15.31:0/38531557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa04167be0 con 0x7fa9ec003ab0 2023-10-06T05:22:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.900+0000 7faa03fff700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7faa04138b10 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4001060 tx=0x7fa9f4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.901+0000 7fa9ebfff700 1 -- 172.21.15.31:0/38531557 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f401e610 con 0x7fa9ec003ab0 2023-10-06T05:22:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.901+0000 7fa9ebfff700 1 -- 172.21.15.31:0/38531557 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f401e770 con 0x7fa9ec003ab0 2023-10-06T05:22:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.901+0000 7fa9ebfff700 1 -- 172.21.15.31:0/38531557 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f401eb30 con 0x7fa9ec003ab0 2023-10-06T05:22:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.901+0000 7faa0a7ff700 1 -- 172.21.15.31:0/38531557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa04167ec0 con 0x7fa9ec003ab0 2023-10-06T05:22:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.901+0000 7faa0a7ff700 1 -- 172.21.15.31:0/38531557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa04168480 con 0x7fa9ec003ab0 2023-10-06T05:22:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.902+0000 7faa0a7ff700 1 -- 172.21.15.31:0/38531557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa040c4a50 con 0x7fa9ec003ab0 2023-10-06T05:22:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.903+0000 7fa9ebfff700 1 -- 172.21.15.31:0/38531557 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fa9f4019030 con 0x7fa9ec003ab0 2023-10-06T05:22:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.904+0000 7fa9ebfff700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9f0067050 0x7fa9f0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.904+0000 7fa9ebfff700 1 -- 172.21.15.31:0/38531557 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fa9f4085f40 con 0x7fa9ec003ab0 2023-10-06T05:22:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.904+0000 7faa037fe700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9f0067050 0x7fa9f0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.905+0000 7faa037fe700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9f0067050 0x7fa9f0069500 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8000e10 tx=0x7fa9f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:15.907+0000 7fa9ebfff700 1 -- 172.21.15.31:0/38531557 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9f4052ef0 con 0x7fa9ec003ab0 2023-10-06T05:22:16.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.117+0000 7faa0a7ff700 1 -- 172.21.15.31:0/38531557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa0404c380 con 0x7fa9ec003ab0 2023-10-06T05:22:16.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.118+0000 7fa9ebfff700 1 -- 172.21.15.31:0/38531557 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+17046 (secure 0 0 0) 0x7fa9f4017090 con 0x7fa9ec003ab0 2023-10-06T05:22:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:16.120 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:22:14.597965+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:52.267847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:53.579487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6835/4141055233":"2023-10-07T05:22:11.589575+0000","172.21.15.31:6833/4141055233":"2023-10-07T05:22:11.589575+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:16.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 -- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9f0067050 msgr2=0x7fa9f0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9f0067050 0x7fa9f0069500 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8000e10 tx=0x7fa9f8009040 comp rx=0 tx=0).stop 2023-10-06T05:22:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 -- 172.21.15.31:0/38531557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 msgr2=0x7faa04138b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7faa04138b10 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4001060 tx=0x7fa9f4015000 comp rx=0 tx=0).stop 2023-10-06T05:22:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 -- 172.21.15.31:0/38531557 shutdown_connections 2023-10-06T05:22:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9f0067050 0x7fa9f0069500 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa0413aee0 0x7faa0413d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ec003ab0 0x7faa04138b10 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 --2- 172.21.15.31:0/38531557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa0413aa70 0x7faa0413ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 -- 172.21.15.31:0/38531557 >> 172.21.15.31:0/38531557 conn(0x7faa040cc0b0 msgr2=0x7faa040b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.120+0000 7faa0a7ff700 1 -- 172.21.15.31:0/38531557 shutdown_connections 2023-10-06T05:22:16.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.121+0000 7faa0a7ff700 1 -- 172.21.15.31:0/38531557 wait complete. 2023-10-06T05:22:16.140 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:21:52.267847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:21:53.579487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:22:16.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:22:16.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.338+0000 7f5e68bff700 1 Processor -- start 2023-10-06T05:22:16.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.339+0000 7f5e68bff700 1 -- start start 2023-10-06T05:22:16.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e68bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e640c8f40 0x7f5e64068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e68bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5e6406a300 0x7f5e640adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e68bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e64068da0 con 0x7f5e6415ca40 2023-10-06T05:22:16.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e68bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e640cabc0 con 0x7f5e640c8f40 2023-10-06T05:22:16.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e68bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e64068bb0 con 0x7f5e6406a300 2023-10-06T05:22:16.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e61d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e640c8f40 0x7f5e64068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e61d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e640c8f40 0x7f5e64068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48246/0 (socket says 172.21.15.31:48246) 2023-10-06T05:22:16.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e61d9b700 1 -- 172.21.15.31:0/1275399038 learned_addr learned my addr 172.21.15.31:0/1275399038 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:16.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e62d9d700 1 --2- 172.21.15.31:0/1275399038 >> v2:172.21.15.62:3300/0 conn(0x7f5e6406a300 0x7f5e640adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e61d9b700 1 -- 172.21.15.31:0/1275399038 >> v2:172.21.15.62:3300/0 conn(0x7f5e6406a300 msgr2=0x7f5e640adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:16.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e61d9b700 1 --2- 172.21.15.31:0/1275399038 >> v2:172.21.15.62:3300/0 conn(0x7f5e6406a300 0x7f5e640adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e61d9b700 1 -- 172.21.15.31:0/1275399038 >> v1:172.21.15.62:6789/0 conn(0x7f5e6415ca40 legacy=0x7f5e640c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e61d9b700 1 -- 172.21.15.31:0/1275399038 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e64069080 con 0x7f5e640c8f40 2023-10-06T05:22:16.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.340+0000 7f5e62d9d700 1 --2- 172.21.15.31:0/1275399038 >> v2:172.21.15.62:3300/0 conn(0x7f5e6406a300 0x7f5e640adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:16.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.341+0000 7f5e61d9b700 1 --2- 172.21.15.31:0/1275399038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e640c8f40 0x7f5e64068670 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5e6406c5f0 tx=0x7f5e58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585dd9ef91a7eef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:16.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.341+0000 7f5e6159a700 1 -- 172.21.15.31:0/1275399038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800b860 con 0x7f5e640c8f40 2023-10-06T05:22:16.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.341+0000 7f5e6159a700 1 -- 172.21.15.31:0/1275399038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5800b9c0 con 0x7f5e640c8f40 2023-10-06T05:22:16.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.341+0000 7f5e6159a700 1 -- 172.21.15.31:0/1275399038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800bd60 con 0x7f5e640c8f40 2023-10-06T05:22:16.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.341+0000 7f5e68bff700 1 -- 172.21.15.31:0/1275399038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e640c8f40 msgr2=0x7f5e64068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.341+0000 7f5e68bff700 1 --2- 172.21.15.31:0/1275399038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e640c8f40 0x7f5e64068670 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5e6406c5f0 tx=0x7f5e58009250 comp rx=0 tx=0).stop 2023-10-06T05:22:16.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.342+0000 7f5e68bff700 1 -- 172.21.15.31:0/1275399038 shutdown_connections 2023-10-06T05:22:16.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.342+0000 7f5e68bff700 1 --2- 172.21.15.31:0/1275399038 >> v2:172.21.15.62:3300/0 conn(0x7f5e6406a300 0x7f5e640adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.342+0000 7f5e68bff700 1 --2- 172.21.15.31:0/1275399038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e640c8f40 0x7f5e64068670 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.342+0000 7f5e68bff700 1 -- 172.21.15.31:0/1275399038 >> 172.21.15.31:0/1275399038 conn(0x7f5e640ce290 msgr2=0x7f5e640ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:16.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.342+0000 7f5e68bff700 1 -- 172.21.15.31:0/1275399038 shutdown_connections 2023-10-06T05:22:16.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.342+0000 7f5e68bff700 1 -- 172.21.15.31:0/1275399038 wait complete. 2023-10-06T05:22:16.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.343+0000 7f5e68bff700 1 Processor -- start 2023-10-06T05:22:16.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.343+0000 7f5e68bff700 1 -- start start 2023-10-06T05:22:16.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.344+0000 7f5e68bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e6406a300 0x7f5e6416a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.344+0000 7f5e68bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e640c8f40 0x7f5e64159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.344+0000 7f5e68bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6415ca40 0x7f5e640b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.344+0000 7f5e68bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6416d7f0 con 0x7f5e6415ca40 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.344+0000 7f5e68bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e640b2b50 con 0x7f5e6406a300 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.344+0000 7f5e68bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e640b2da0 con 0x7f5e640c8f40 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e62d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6415ca40 0x7f5e640b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e62d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6415ca40 0x7f5e640b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48254/0 (socket says 172.21.15.31:48254) 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e62d9d700 1 -- 172.21.15.31:0/456031558 learned_addr learned my addr 172.21.15.31:0/456031558 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e61d9b700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e640c8f40 0x7f5e64159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e6259c700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e6406a300 0x7f5e6416a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e62d9d700 1 -- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e640c8f40 msgr2=0x7f5e64159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e62d9d700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e640c8f40 0x7f5e64159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e62d9d700 1 -- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e6406a300 msgr2=0x7f5e6416a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e62d9d700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e6406a300 0x7f5e6416a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e62d9d700 1 -- 172.21.15.31:0/456031558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580094f0 con 0x7f5e6415ca40 2023-10-06T05:22:16.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e61d9b700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e640c8f40 0x7f5e64159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:16.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e62d9d700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6415ca40 0x7f5e640b2610 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009fc0 tx=0x7f5e5c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:16.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e527fc700 1 -- 172.21.15.31:0/456031558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c00dd40 con 0x7f5e6415ca40 2023-10-06T05:22:16.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e527fc700 1 -- 172.21.15.31:0/456031558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c00f040 con 0x7f5e6415ca40 2023-10-06T05:22:16.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e527fc700 1 -- 172.21.15.31:0/456031558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c015690 con 0x7f5e6415ca40 2023-10-06T05:22:16.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.345+0000 7f5e68bff700 1 -- 172.21.15.31:0/456031558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e640b3080 con 0x7f5e6415ca40 2023-10-06T05:22:16.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.346+0000 7f5e68bff700 1 -- 172.21.15.31:0/456031558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6416d480 con 0x7f5e6415ca40 2023-10-06T05:22:16.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.347+0000 7f5e68bff700 1 -- 172.21.15.31:0/456031558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6404c380 con 0x7f5e6415ca40 2023-10-06T05:22:16.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.348+0000 7f5e527fc700 1 -- 172.21.15.31:0/456031558 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f5e5c01e3b0 con 0x7f5e6415ca40 2023-10-06T05:22:16.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.349+0000 7f5e527fc700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e44067050 0x7f5e44069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.349+0000 7f5e527fc700 1 -- 172.21.15.31:0/456031558 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5e5c084ec0 con 0x7f5e6415ca40 2023-10-06T05:22:16.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.349+0000 7f5e6259c700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e44067050 0x7f5e44069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.350+0000 7f5e6259c700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e44067050 0x7f5e44069500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5e6406bd60 tx=0x7f5e54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:16.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.353+0000 7f5e527fc700 1 -- 172.21.15.31:0/456031558 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e5c051d60 con 0x7f5e6415ca40 2023-10-06T05:22:16.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.555+0000 7f5e68bff700 1 -- 172.21.15.31:0/456031558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5e6416d930 con 0x7f5e6415ca40 2023-10-06T05:22:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.556+0000 7f5e527fc700 1 -- 172.21.15.31:0/456031558 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v90) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5e5c01a0a0 con 0x7f5e6415ca40 2023-10-06T05:22:16.558 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:22:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 -- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e44067050 msgr2=0x7f5e44069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e44067050 0x7f5e44069500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5e6406bd60 tx=0x7f5e54009040 comp rx=0 tx=0).stop 2023-10-06T05:22:16.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 -- 172.21.15.31:0/456031558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6415ca40 msgr2=0x7f5e640b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6415ca40 0x7f5e640b2610 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009fc0 tx=0x7f5e5c0099b0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 -- 172.21.15.31:0/456031558 shutdown_connections 2023-10-06T05:22:16.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e44067050 0x7f5e44069500 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e6415ca40 0x7f5e640b2610 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e6406a300 0x7f5e6416a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 --2- 172.21.15.31:0/456031558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e640c8f40 0x7f5e64159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 -- 172.21.15.31:0/456031558 >> 172.21.15.31:0/456031558 conn(0x7f5e640ce290 msgr2=0x7f5e640cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:16.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.558+0000 7f5e68bff700 1 -- 172.21.15.31:0/456031558 shutdown_connections 2023-10-06T05:22:16.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.559+0000 7f5e68bff700 1 -- 172.21.15.31:0/456031558 wait complete. 2023-10-06T05:22:16.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:22:16.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.774+0000 7fa841dff700 1 Processor -- start 2023-10-06T05:22:16.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.774+0000 7fa841dff700 1 -- start start 2023-10-06T05:22:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.776+0000 7fa841dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 0x7fa83c0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.776+0000 7fa841dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa83c157770 0x7fa83c0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.776+0000 7fa841dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa83c157be0 0x7fa83c0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.776+0000 7fa841dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83c158cf0 con 0x7fa83c157be0 2023-10-06T05:22:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.776+0000 7fa841dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83c1580e0 con 0x7fa83c157770 2023-10-06T05:22:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.776+0000 7fa841dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c1583c0 con 0x7fa83c1573a0 2023-10-06T05:22:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa83c157be0 0x7fa83c0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa83c157be0 0x7fa83c0b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48266/0 (socket says 172.21.15.31:48266) 2023-10-06T05:22:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83bfff700 1 -- 172.21.15.31:0/2666505185 learned_addr learned my addr 172.21.15.31:0/2666505185 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83affd700 1 --2- 172.21.15.31:0/2666505185 >> v2:172.21.15.62:3300/0 conn(0x7fa83c157770 0x7fa83c0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83b7fe700 1 --2- 172.21.15.31:0/2666505185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 0x7fa83c0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83bfff700 1 -- 172.21.15.31:0/2666505185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 msgr2=0x7fa83c0acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83bfff700 1 --2- 172.21.15.31:0/2666505185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 0x7fa83c0acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83bfff700 1 -- 172.21.15.31:0/2666505185 >> v2:172.21.15.62:3300/0 conn(0x7fa83c157770 msgr2=0x7fa83c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83bfff700 1 --2- 172.21.15.31:0/2666505185 >> v2:172.21.15.62:3300/0 conn(0x7fa83c157770 0x7fa83c0b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83bfff700 1 -- 172.21.15.31:0/2666505185 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c1586a0 con 0x7fa83c157be0 2023-10-06T05:22:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.777+0000 7fa83bfff700 1 --2- 172.21.15.31:0/2666505185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157be0 0x7fa83c0b17a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0cd170 tx=0x7fa834009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fa0efd39f47769 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa83a7fc700 1 -- 172.21.15.31:0/2666505185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400e070 con 0x7fa83c157be0 2023-10-06T05:22:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa83a7fc700 1 -- 172.21.15.31:0/2666505185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83400b970 con 0x7fa83c157be0 2023-10-06T05:22:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa83a7fc700 1 -- 172.21.15.31:0/2666505185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340092e0 con 0x7fa83c157be0 2023-10-06T05:22:16.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa841dff700 1 -- 172.21.15.31:0/2666505185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157be0 msgr2=0x7fa83c0b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa841dff700 1 --2- 172.21.15.31:0/2666505185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157be0 0x7fa83c0b17a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0cd170 tx=0x7fa834009840 comp rx=0 tx=0).stop 2023-10-06T05:22:16.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa841dff700 1 -- 172.21.15.31:0/2666505185 shutdown_connections 2023-10-06T05:22:16.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa841dff700 1 --2- 172.21.15.31:0/2666505185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157be0 0x7fa83c0b17a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa841dff700 1 --2- 172.21.15.31:0/2666505185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 0x7fa83c0acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa841dff700 1 --2- 172.21.15.31:0/2666505185 >> v2:172.21.15.62:3300/0 conn(0x7fa83c157770 0x7fa83c0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa841dff700 1 -- 172.21.15.31:0/2666505185 >> 172.21.15.31:0/2666505185 conn(0x7fa83c067560 msgr2=0x7fa83c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:16.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa841dff700 1 -- 172.21.15.31:0/2666505185 shutdown_connections 2023-10-06T05:22:16.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.778+0000 7fa841dff700 1 -- 172.21.15.31:0/2666505185 wait complete. 2023-10-06T05:22:16.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.780+0000 7fa841dff700 1 Processor -- start 2023-10-06T05:22:16.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.781+0000 7fa841dff700 1 -- start start 2023-10-06T05:22:16.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.782+0000 7fa841dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 0x7fa83c15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.782+0000 7fa841dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157770 0x7fa83c067930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.782+0000 7fa841dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa83c157be0 0x7fa83c067e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.782+0000 7fa841dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c16a6b0 con 0x7fa83c157770 2023-10-06T05:22:16.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.782+0000 7fa841dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c16a860 con 0x7fa83c157be0 2023-10-06T05:22:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.782+0000 7fa841dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c15b620 con 0x7fa83c1573a0 2023-10-06T05:22:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.782+0000 7fa83affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157770 0x7fa83c067930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.782+0000 7fa83affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157770 0x7fa83c067930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48280/0 (socket says 172.21.15.31:48280) 2023-10-06T05:22:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.782+0000 7fa83affd700 1 -- 172.21.15.31:0/2008844344 learned_addr learned my addr 172.21.15.31:0/2008844344 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.782+0000 7fa83b7fe700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 0x7fa83c15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa83bfff700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa83c157be0 0x7fa83c067e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa83affd700 1 -- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 msgr2=0x7fa83c15b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa83affd700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 0x7fa83c15b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa83affd700 1 -- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa83c157be0 msgr2=0x7fa83c067e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa83affd700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa83c157be0 0x7fa83c067e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa83affd700 1 -- 172.21.15.31:0/2008844344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834009ae0 con 0x7fa83c157770 2023-10-06T05:22:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa83b7fe700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 0x7fa83c15b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa83affd700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157770 0x7fa83c067930 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fa83000ecb0 tx=0x7fa8300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa8237fe700 1 -- 172.21.15.31:0/2008844344 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000c970 con 0x7fa83c157770 2023-10-06T05:22:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa8237fe700 1 -- 172.21.15.31:0/2008844344 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83000cad0 con 0x7fa83c157770 2023-10-06T05:22:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa841dff700 1 -- 172.21.15.31:0/2008844344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c152cf0 con 0x7fa83c157770 2023-10-06T05:22:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa8237fe700 1 -- 172.21.15.31:0/2008844344 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000ce70 con 0x7fa83c157770 2023-10-06T05:22:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.783+0000 7fa841dff700 1 -- 172.21.15.31:0/2008844344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c0b0ef0 con 0x7fa83c157770 2023-10-06T05:22:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.784+0000 7fa841dff700 1 -- 172.21.15.31:0/2008844344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83c04c380 con 0x7fa83c157770 2023-10-06T05:22:16.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.785+0000 7fa8237fe700 1 -- 172.21.15.31:0/2008844344 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fa830013680 con 0x7fa83c157770 2023-10-06T05:22:16.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.786+0000 7fa8237fe700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa828067050 0x7fa828069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:16.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.786+0000 7fa8237fe700 1 -- 172.21.15.31:0/2008844344 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fa830084b50 con 0x7fa83c157770 2023-10-06T05:22:16.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.787+0000 7fa83b7fe700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa828067050 0x7fa828069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:16.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.788+0000 7fa83b7fe700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa828067050 0x7fa828069500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0b1230 tx=0x7fa82c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.790+0000 7fa8237fe700 1 -- 172.21.15.31:0/2008844344 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa830051b00 con 0x7fa83c157770 2023-10-06T05:22:16.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.994+0000 7fa841dff700 1 -- 172.21.15.31:0/2008844344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa83c0ab3f0 con 0x7fa83c157770 2023-10-06T05:22:16.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.995+0000 7fa8237fe700 1 -- 172.21.15.31:0/2008844344 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa830055120 con 0x7fa83c157770 2023-10-06T05:22:16.997 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:22:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 -- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa828067050 msgr2=0x7fa828069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa828067050 0x7fa828069500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0b1230 tx=0x7fa82c008040 comp rx=0 tx=0).stop 2023-10-06T05:22:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 -- 172.21.15.31:0/2008844344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157770 msgr2=0x7fa83c067930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157770 0x7fa83c067930 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fa83000ecb0 tx=0x7fa8300091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 -- 172.21.15.31:0/2008844344 shutdown_connections 2023-10-06T05:22:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa828067050 0x7fa828069500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa83c157be0 0x7fa83c067e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa83c1573a0 0x7fa83c15b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 --2- 172.21.15.31:0/2008844344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c157770 0x7fa83c067930 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 -- 172.21.15.31:0/2008844344 >> 172.21.15.31:0/2008844344 conn(0x7fa83c067560 msgr2=0x7fa83c0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 -- 172.21.15.31:0/2008844344 shutdown_connections 2023-10-06T05:22:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:16.997+0000 7fa841dff700 1 -- 172.21.15.31:0/2008844344 wait complete. 2023-10-06T05:22:17.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:22:17.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.213+0000 7f8fabbff700 1 Processor -- start 2023-10-06T05:22:17.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.213+0000 7f8fabbff700 1 -- start start 2023-10-06T05:22:17.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.214+0000 7f8fabbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8fa40c8f40 0x7f8fa4068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.214+0000 7f8fabbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa406a300 0x7f8fa40adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.214+0000 7f8fabbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa4068da0 con 0x7f8fa415ca40 2023-10-06T05:22:17.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.214+0000 7f8fabbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa40cabc0 con 0x7f8fa40c8f40 2023-10-06T05:22:17.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.214+0000 7f8fabbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa4068bb0 con 0x7f8fa406a300 2023-10-06T05:22:17.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.215+0000 7f8faa19c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa406a300 0x7f8fa40adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.215+0000 7f8fa919a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8fa40c8f40 0x7f8fa4068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.215+0000 7f8faa19c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa406a300 0x7f8fa40adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:34384/0 (socket says 172.21.15.31:34384) 2023-10-06T05:22:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.215+0000 7f8faa19c700 1 -- 172.21.15.31:0/3199010210 learned_addr learned my addr 172.21.15.31:0/3199010210 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.215+0000 7f8faa19c700 1 -- 172.21.15.31:0/3199010210 >> v2:172.21.15.62:3300/0 conn(0x7f8fa40c8f40 msgr2=0x7f8fa4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.215+0000 7f8faa19c700 1 --2- 172.21.15.31:0/3199010210 >> v2:172.21.15.62:3300/0 conn(0x7f8fa40c8f40 0x7f8fa4068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.215+0000 7f8faa19c700 1 -- 172.21.15.31:0/3199010210 >> v1:172.21.15.62:6789/0 conn(0x7f8fa415ca40 legacy=0x7f8fa40c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.215+0000 7f8faa19c700 1 -- 172.21.15.31:0/3199010210 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa4069080 con 0x7f8fa406a300 2023-10-06T05:22:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.216+0000 7f8faa19c700 1 --2- 172.21.15.31:0/3199010210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa406a300 0x7f8fa40adde0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009f20 tx=0x7f8f9c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd627f20a5c7c29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.216+0000 7f8fa8999700 1 -- 172.21.15.31:0/3199010210 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00e070 con 0x7f8fa406a300 2023-10-06T05:22:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.216+0000 7f8fa8999700 1 -- 172.21.15.31:0/3199010210 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c00b9d0 con 0x7f8fa406a300 2023-10-06T05:22:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.216+0000 7f8fa8999700 1 -- 172.21.15.31:0/3199010210 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0092e0 con 0x7f8fa406a300 2023-10-06T05:22:17.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.216+0000 7f8fabbff700 1 -- 172.21.15.31:0/3199010210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa406a300 msgr2=0x7f8fa40adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.216+0000 7f8fabbff700 1 --2- 172.21.15.31:0/3199010210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa406a300 0x7f8fa40adde0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009f20 tx=0x7f8f9c009840 comp rx=0 tx=0).stop 2023-10-06T05:22:17.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.216+0000 7f8fabbff700 1 -- 172.21.15.31:0/3199010210 shutdown_connections 2023-10-06T05:22:17.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.216+0000 7f8fabbff700 1 --2- 172.21.15.31:0/3199010210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa406a300 0x7f8fa40adde0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.216+0000 7f8fabbff700 1 --2- 172.21.15.31:0/3199010210 >> v2:172.21.15.62:3300/0 conn(0x7f8fa40c8f40 0x7f8fa4068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.216+0000 7f8fabbff700 1 -- 172.21.15.31:0/3199010210 >> 172.21.15.31:0/3199010210 conn(0x7f8fa40ce290 msgr2=0x7f8fa40ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:17.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.217+0000 7f8fabbff700 1 -- 172.21.15.31:0/3199010210 shutdown_connections 2023-10-06T05:22:17.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.217+0000 7f8fabbff700 1 -- 172.21.15.31:0/3199010210 wait complete. 2023-10-06T05:22:17.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.218+0000 7f8fabbff700 1 Processor -- start 2023-10-06T05:22:17.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.218+0000 7f8fabbff700 1 -- start start 2023-10-06T05:22:17.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.219+0000 7f8fabbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa406a300 0x7f8fa40c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.219+0000 7f8fabbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa40c8f40 0x7f8fa4169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.219+0000 7f8fabbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa415ca40 0x7f8fa414c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.219+0000 7f8fabbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40b1270 con 0x7f8fa40c8f40 2023-10-06T05:22:17.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.219+0000 7f8fabbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa4169e80 con 0x7f8fa406a300 2023-10-06T05:22:17.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.219+0000 7f8fabbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa414cd50 con 0x7f8fa415ca40 2023-10-06T05:22:17.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.219+0000 7f8fa919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa40c8f40 0x7f8fa4169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.219+0000 7f8fa919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa40c8f40 0x7f8fa4169940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45962/0 (socket says 172.21.15.31:45962) 2023-10-06T05:22:17.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.219+0000 7f8fa919a700 1 -- 172.21.15.31:0/2406453781 learned_addr learned my addr 172.21.15.31:0/2406453781 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8faa19c700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa415ca40 0x7f8fa414c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8fa999b700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa406a300 0x7f8fa40c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8fa919a700 1 -- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa415ca40 msgr2=0x7f8fa414c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8fa919a700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa415ca40 0x7f8fa414c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8fa919a700 1 -- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa406a300 msgr2=0x7f8fa40c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8fa919a700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa406a300 0x7f8fa40c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8fa919a700 1 -- 172.21.15.31:0/2406453781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c009ae0 con 0x7f8fa40c8f40 2023-10-06T05:22:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8fa999b700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa406a300 0x7f8fa40c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:17.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8fa919a700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa40c8f40 0x7f8fa4169940 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400ec80 tx=0x7f8f940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8f99ffb700 1 -- 172.21.15.31:0/2406453781 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400c9d0 con 0x7f8fa40c8f40 2023-10-06T05:22:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8f99ffb700 1 -- 172.21.15.31:0/2406453781 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9400cb30 con 0x7f8fa40c8f40 2023-10-06T05:22:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8fabbff700 1 -- 172.21.15.31:0/2406453781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa416a110 con 0x7f8fa40c8f40 2023-10-06T05:22:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.220+0000 7f8f99ffb700 1 -- 172.21.15.31:0/2406453781 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94013470 con 0x7f8fa40c8f40 2023-10-06T05:22:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.221+0000 7f8fabbff700 1 -- 172.21.15.31:0/2406453781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa416a6d0 con 0x7f8fa40c8f40 2023-10-06T05:22:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.221+0000 7f8fabbff700 1 -- 172.21.15.31:0/2406453781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa40b1040 con 0x7f8fa40c8f40 2023-10-06T05:22:17.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.222+0000 7f8f99ffb700 1 -- 172.21.15.31:0/2406453781 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f8f94013820 con 0x7f8fa40c8f40 2023-10-06T05:22:17.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.223+0000 7f8f99ffb700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f88067050 0x7f8f88069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.223+0000 7f8f99ffb700 1 -- 172.21.15.31:0/2406453781 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f8f94084c60 con 0x7f8fa40c8f40 2023-10-06T05:22:17.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.224+0000 7f8fa999b700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f88067050 0x7f8f88069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.225+0000 7f8fa999b700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f88067050 0x7f8f88069500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8fa41674f0 tx=0x7f8fa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.228+0000 7f8f99ffb700 1 -- 172.21.15.31:0/2406453781 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f94051b90 con 0x7f8fa40c8f40 2023-10-06T05:22:17.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.428+0000 7f8fabbff700 1 -- 172.21.15.31:0/2406453781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fa416a8c0 con 0x7f8fa40c8f40 2023-10-06T05:22:17.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.428+0000 7f8f99ffb700 1 -- 172.21.15.31:0/2406453781 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f940551b0 con 0x7f8fa40c8f40 2023-10-06T05:22:17.430 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:22:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 -- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f88067050 msgr2=0x7f8f88069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f88067050 0x7f8f88069500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8fa41674f0 tx=0x7f8fa0009040 comp rx=0 tx=0).stop 2023-10-06T05:22:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 -- 172.21.15.31:0/2406453781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa40c8f40 msgr2=0x7f8fa4169940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa40c8f40 0x7f8fa4169940 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400ec80 tx=0x7f8f940091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 -- 172.21.15.31:0/2406453781 shutdown_connections 2023-10-06T05:22:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f88067050 0x7f8f88069500 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa415ca40 0x7f8fa414c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa406a300 0x7f8fa40c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 --2- 172.21.15.31:0/2406453781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa40c8f40 0x7f8fa4169940 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 -- 172.21.15.31:0/2406453781 >> 172.21.15.31:0/2406453781 conn(0x7f8fa40ce290 msgr2=0x7f8fa40cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:17.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 -- 172.21.15.31:0/2406453781 shutdown_connections 2023-10-06T05:22:17.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.431+0000 7f8fabbff700 1 -- 172.21.15.31:0/2406453781 wait complete. 2023-10-06T05:22:17.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:22:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.650+0000 7f8c535ff700 1 Processor -- start 2023-10-06T05:22:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.650+0000 7f8c535ff700 1 -- start start 2023-10-06T05:22:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.651+0000 7f8c535ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0b0b90 0x7f8c4c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.651+0000 7f8c535ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4c0ad690 con 0x7f8c4c153d40 2023-10-06T05:22:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.651+0000 7f8c535ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4c0acd60 con 0x7f8c4c0af6c0 2023-10-06T05:22:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.651+0000 7f8c535ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0ad040 con 0x7f8c4c0b0b90 2023-10-06T05:22:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.651+0000 7f8c5139b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8c4c153d40 0x7f8c4c0b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53780/0 (socket says 172.21.15.31:53780) 2023-10-06T05:22:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.651+0000 7f8c5139b700 1 -- 172.21.15.31:0/4255287673 learned_addr learned my addr 172.21.15.31:0/4255287673 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c50b9a700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0b0b90 0x7f8c4c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099127539 0 0) 0x7f8c4c0ad690 con 0x7f8c4c153d40 2023-10-06T05:22:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c38003570 con 0x7f8c4c153d40 2023-10-06T05:22:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1024175015 0 0) 0x7f8c38003570 con 0x7f8c4c153d40 2023-10-06T05:22:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0b0b90 msgr2=0x7f8c4c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0b0b90 0x7f8c4c0af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 >> v1:172.21.15.62:6789/0 conn(0x7f8c4c0af6c0 legacy=0x7f8c4c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c0ad320 con 0x7f8c4c153d40 2023-10-06T05:22:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f8c48001470 con 0x7f8c4c153d40 2023-10-06T05:22:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 >> v1:172.21.15.31:6789/0 conn(0x7f8c4c153d40 legacy=0x7f8c4c0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0b0b90 0x7f8c38003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c38003fd0 0x7f8c38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c38004d50 0x7f8c380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0ad040 con 0x7f8c38004d50 2023-10-06T05:22:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3800bb40 con 0x7f8c38003fd0 2023-10-06T05:22:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.652+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3800bd00 con 0x7f8c4c0b0b90 2023-10-06T05:22:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c51b9c700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c38004d50 0x7f8c380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c50b9a700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c38003fd0 0x7f8c38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c5139b700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0b0b90 0x7f8c38003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c51b9c700 1 -- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0b0b90 msgr2=0x7f8c38003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c51b9c700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0b0b90 0x7f8c38003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c51b9c700 1 -- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c38003fd0 msgr2=0x7f8c38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c51b9c700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c38003fd0 0x7f8c38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c51b9c700 1 -- 172.21.15.31:0/4255287673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c38003570 con 0x7f8c38004d50 2023-10-06T05:22:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c51b9c700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c38004d50 0x7f8c380051c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8c440020e0 tx=0x7f8c44001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8eb8ee6fe3b553d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44001aa0 con 0x7f8c38004d50 2023-10-06T05:22:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4400c8b0 con 0x7f8c38004d50 2023-10-06T05:22:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.653+0000 7f8c43fff700 1 -- 172.21.15.31:0/4255287673 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400cc70 con 0x7f8c38004d50 2023-10-06T05:22:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.654+0000 7f8c535ff700 1 -- 172.21.15.31:0/4255287673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c38004d50 msgr2=0x7f8c380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.654+0000 7f8c535ff700 1 --2- 172.21.15.31:0/4255287673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c38004d50 0x7f8c380051c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8c440020e0 tx=0x7f8c44001790 comp rx=0 tx=0).stop 2023-10-06T05:22:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.654+0000 7f8c51b9c700 1 -- 172.21.15.31:0/4255287673 reap_dead start 2023-10-06T05:22:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.654+0000 7f8c535ff700 1 -- 172.21.15.31:0/4255287673 shutdown_connections 2023-10-06T05:22:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.654+0000 7f8c535ff700 1 -- 172.21.15.31:0/4255287673 >> 172.21.15.31:0/4255287673 conn(0x7f8c4c067560 msgr2=0x7f8c4c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.654+0000 7f8c535ff700 1 -- 172.21.15.31:0/4255287673 shutdown_connections 2023-10-06T05:22:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.654+0000 7f8c535ff700 1 -- 172.21.15.31:0/4255287673 wait complete. 2023-10-06T05:22:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.656+0000 7f8c535ff700 1 Processor -- start 2023-10-06T05:22:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.656+0000 7f8c535ff700 1 -- start start 2023-10-06T05:22:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c535ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c38004d50 0x7f8c4c0aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c535ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c163670 0x7f8c4c1696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c535ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c0ada20 0x7f8c4c0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c535ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0ae3e0 con 0x7f8c4c163670 2023-10-06T05:22:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c535ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c14b1a0 con 0x7f8c4c0ada20 2023-10-06T05:22:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c535ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c169f40 con 0x7f8c38004d50 2023-10-06T05:22:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c50b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c163670 0x7f8c4c1696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c50b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c163670 0x7f8c4c1696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45976/0 (socket says 172.21.15.31:45976) 2023-10-06T05:22:17.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c50b9a700 1 -- 172.21.15.31:0/2728034974 learned_addr learned my addr 172.21.15.31:0/2728034974 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:17.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c51b9c700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c0ada20 0x7f8c4c0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c5139b700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c38004d50 0x7f8c4c0aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c50b9a700 1 -- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c38004d50 msgr2=0x7f8c4c0aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c50b9a700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c38004d50 0x7f8c4c0aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c50b9a700 1 -- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c0ada20 msgr2=0x7f8c4c0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c50b9a700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c0ada20 0x7f8c4c0adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c50b9a700 1 -- 172.21.15.31:0/2728034974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c16a220 con 0x7f8c4c163670 2023-10-06T05:22:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c51b9c700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c0ada20 0x7f8c4c0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:17.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.657+0000 7f8c5139b700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c38004d50 0x7f8c4c0aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:17.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.658+0000 7f8c50b9a700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c163670 0x7f8c4c1696a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c011bd0 tx=0x7f8c3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.658+0000 7f8c417fa700 1 -- 172.21.15.31:0/2728034974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00fce0 con 0x7f8c4c163670 2023-10-06T05:22:17.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.658+0000 7f8c417fa700 1 -- 172.21.15.31:0/2728034974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00fe60 con 0x7f8c4c163670 2023-10-06T05:22:17.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.658+0000 7f8c417fa700 1 -- 172.21.15.31:0/2728034974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c019660 con 0x7f8c4c163670 2023-10-06T05:22:17.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.658+0000 7f8c535ff700 1 -- 172.21.15.31:0/2728034974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c16a500 con 0x7f8c4c163670 2023-10-06T05:22:17.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.658+0000 7f8c535ff700 1 -- 172.21.15.31:0/2728034974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c163ae0 con 0x7f8c4c163670 2023-10-06T05:22:17.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.659+0000 7f8c535ff700 1 -- 172.21.15.31:0/2728034974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4c04c380 con 0x7f8c4c163670 2023-10-06T05:22:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.660+0000 7f8c417fa700 1 -- 172.21.15.31:0/2728034974 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f8c3c019a50 con 0x7f8c4c163670 2023-10-06T05:22:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.661+0000 7f8c417fa700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c30067050 0x7f8c30069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.661+0000 7f8c417fa700 1 -- 172.21.15.31:0/2728034974 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f8c3c084d00 con 0x7f8c4c163670 2023-10-06T05:22:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.661+0000 7f8c5139b700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c30067050 0x7f8c30069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.662+0000 7f8c5139b700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c30067050 0x7f8c30069500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8c480010e0 tx=0x7f8c48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:17.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.665+0000 7f8c417fa700 1 -- 172.21.15.31:0/2728034974 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c3c051c30 con 0x7f8c4c163670 2023-10-06T05:22:17.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.870+0000 7f8c535ff700 1 -- 172.21.15.31:0/2728034974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c4c14b2e0 con 0x7f8c4c163670 2023-10-06T05:22:17.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.870+0000 7f8c417fa700 1 -- 172.21.15.31:0/2728034974 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c3c01e0a0 con 0x7f8c4c163670 2023-10-06T05:22:17.872 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:22:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 -- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c30067050 msgr2=0x7f8c30069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c30067050 0x7f8c30069500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8c480010e0 tx=0x7f8c48009040 comp rx=0 tx=0).stop 2023-10-06T05:22:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 -- 172.21.15.31:0/2728034974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c163670 msgr2=0x7f8c4c1696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c163670 0x7f8c4c1696a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c011bd0 tx=0x7f8c3c008040 comp rx=0 tx=0).stop 2023-10-06T05:22:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 -- 172.21.15.31:0/2728034974 shutdown_connections 2023-10-06T05:22:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c30067050 0x7f8c30069500 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c0ada20 0x7f8c4c0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c38004d50 0x7f8c4c0aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 --2- 172.21.15.31:0/2728034974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c163670 0x7f8c4c1696a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:17.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 -- 172.21.15.31:0/2728034974 >> 172.21.15.31:0/2728034974 conn(0x7f8c4c067560 msgr2=0x7f8c4c0cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:17.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 -- 172.21.15.31:0/2728034974 shutdown_connections 2023-10-06T05:22:17.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:17.873+0000 7f8c535ff700 1 -- 172.21.15.31:0/2728034974 wait complete. 2023-10-06T05:22:17.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:18.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.085+0000 7f5e635ff700 1 Processor -- start 2023-10-06T05:22:18.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.086+0000 7f5e635ff700 1 -- start start 2023-10-06T05:22:18.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.087+0000 7f5e635ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e5c068d80 0x7f5e5c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.087+0000 7f5e635ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 0x7f5e5c0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.087+0000 7f5e635ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5c0c95b0 con 0x7f5e5c1595a0 2023-10-06T05:22:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.087+0000 7f5e635ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c0c9890 con 0x7f5e5c068d80 2023-10-06T05:22:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.087+0000 7f5e635ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0c9b70 con 0x7f5e5c0cc0d0 2023-10-06T05:22:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.087+0000 7f5e60b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e5c068d80 0x7f5e5c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.087+0000 7f5e6139b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5e5c1595a0 0x7f5e5c068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53782/0 (socket says 172.21.15.31:53782) 2023-10-06T05:22:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.087+0000 7f5e6139b700 1 -- 172.21.15.31:0/2664458042 learned_addr learned my addr 172.21.15.31:0/2664458042 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:18.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e60b9a700 1 -- 172.21.15.31:0/2664458042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 msgr2=0x7f5e5c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:18.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e61b9c700 1 --2- 172.21.15.31:0/2664458042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 0x7f5e5c0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e60b9a700 1 --2- 172.21.15.31:0/2664458042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 0x7f5e5c0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e60b9a700 1 -- 172.21.15.31:0/2664458042 >> v1:172.21.15.31:6789/0 conn(0x7f5e5c1595a0 legacy=0x7f5e5c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e53fff700 1 -- 172.21.15.31:0/2664458042 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619370539 0 0) 0x7f5e5c0c95b0 con 0x7f5e5c1595a0 2023-10-06T05:22:18.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e60b9a700 1 -- 172.21.15.31:0/2664458042 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0ca130 con 0x7f5e5c068d80 2023-10-06T05:22:18.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e61b9c700 1 --2- 172.21.15.31:0/2664458042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 0x7f5e5c0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:18.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e60b9a700 1 --2- 172.21.15.31:0/2664458042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 0x7f5e5c159970 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009910 tx=0x7f5e4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f261daa825d42c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e53fff700 1 -- 172.21.15.31:0/2664458042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00b820 con 0x7f5e5c068d80 2023-10-06T05:22:18.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e53fff700 1 -- 172.21.15.31:0/2664458042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c00b980 con 0x7f5e5c068d80 2023-10-06T05:22:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.088+0000 7f5e53fff700 1 -- 172.21.15.31:0/2664458042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00bd20 con 0x7f5e5c068d80 2023-10-06T05:22:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.089+0000 7f5e635ff700 1 -- 172.21.15.31:0/2664458042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 msgr2=0x7f5e5c159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.089+0000 7f5e635ff700 1 --2- 172.21.15.31:0/2664458042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 0x7f5e5c159970 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009910 tx=0x7f5e4c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.089+0000 7f5e635ff700 1 -- 172.21.15.31:0/2664458042 shutdown_connections 2023-10-06T05:22:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.089+0000 7f5e635ff700 1 --2- 172.21.15.31:0/2664458042 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 0x7f5e5c0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.089+0000 7f5e635ff700 1 --2- 172.21.15.31:0/2664458042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 0x7f5e5c159970 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.089+0000 7f5e635ff700 1 -- 172.21.15.31:0/2664458042 >> 172.21.15.31:0/2664458042 conn(0x7f5e5c06b620 msgr2=0x7f5e5c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.089+0000 7f5e635ff700 1 -- 172.21.15.31:0/2664458042 shutdown_connections 2023-10-06T05:22:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.089+0000 7f5e635ff700 1 -- 172.21.15.31:0/2664458042 wait complete. 2023-10-06T05:22:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.091+0000 7f5e635ff700 1 Processor -- start 2023-10-06T05:22:18.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.091+0000 7f5e635ff700 1 -- start start 2023-10-06T05:22:18.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e635ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 0x7f5e5c154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e635ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 0x7f5e5c0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e635ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c1595a0 0x7f5e5c150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e635ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c1526b0 con 0x7f5e5c068d80 2023-10-06T05:22:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e635ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c151d80 con 0x7f5e5c1595a0 2023-10-06T05:22:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e635ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c152060 con 0x7f5e5c0cc0d0 2023-10-06T05:22:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e6139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 0x7f5e5c154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e6139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 0x7f5e5c154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46002/0 (socket says 172.21.15.31:46002) 2023-10-06T05:22:18.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e6139b700 1 -- 172.21.15.31:0/3746294155 learned_addr learned my addr 172.21.15.31:0/3746294155 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e60b9a700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 0x7f5e5c0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e61b9c700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c1595a0 0x7f5e5c150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e6139b700 1 -- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 msgr2=0x7f5e5c0c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e6139b700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 0x7f5e5c0c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e6139b700 1 -- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c1595a0 msgr2=0x7f5e5c150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e6139b700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c1595a0 0x7f5e5c150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.092+0000 7f5e6139b700 1 -- 172.21.15.31:0/3746294155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c009420 con 0x7f5e5c068d80 2023-10-06T05:22:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.093+0000 7f5e6139b700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 0x7f5e5c154210 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800c7e0 tx=0x7f5e58007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.093+0000 7f5e517fa700 1 -- 172.21.15.31:0/3746294155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58005b30 con 0x7f5e5c068d80 2023-10-06T05:22:18.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.093+0000 7f5e517fa700 1 -- 172.21.15.31:0/3746294155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e58005c90 con 0x7f5e5c068d80 2023-10-06T05:22:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.093+0000 7f5e635ff700 1 -- 172.21.15.31:0/3746294155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5c152340 con 0x7f5e5c068d80 2023-10-06T05:22:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.093+0000 7f5e517fa700 1 -- 172.21.15.31:0/3746294155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58007300 con 0x7f5e5c068d80 2023-10-06T05:22:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.093+0000 7f5e635ff700 1 -- 172.21.15.31:0/3746294155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c0c6db0 con 0x7f5e5c068d80 2023-10-06T05:22:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.094+0000 7f5e635ff700 1 -- 172.21.15.31:0/3746294155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5c0c7000 con 0x7f5e5c068d80 2023-10-06T05:22:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.095+0000 7f5e517fa700 1 -- 172.21.15.31:0/3746294155 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f5e5801b400 con 0x7f5e5c068d80 2023-10-06T05:22:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.096+0000 7f5e517fa700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e40067050 0x7f5e40069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.096+0000 7f5e517fa700 1 -- 172.21.15.31:0/3746294155 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5e580843d0 con 0x7f5e5c068d80 2023-10-06T05:22:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.097+0000 7f5e60b9a700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e40067050 0x7f5e40069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.098+0000 7f5e60b9a700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e40067050 0x7f5e40069500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c069180 tx=0x7f5e4c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.101+0000 7f5e517fa700 1 -- 172.21.15.31:0/3746294155 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e58051380 con 0x7f5e5c068d80 2023-10-06T05:22:18.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.347+0000 7f5e635ff700 1 -- 172.21.15.31:0/3746294155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e5c04c380 con 0x7f5e5c068d80 2023-10-06T05:22:18.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.348+0000 7f5e517fa700 1 -- 172.21.15.31:0/3746294155 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f5e580549a0 con 0x7f5e5c068d80 2023-10-06T05:22:18.350 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:18.350 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1878570982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1878570982},{"type":"v1","addr":"172.21.15.62:6834","nonce":1878570982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6687,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/427382574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":427382574},{"type":"v1","addr":"172.21.15.31:6834","nonce":427382574}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1760549774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1760549774},{"type":"v1","addr":"172.21.15.31:6835","nonce":1760549774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:54.590687+0000","modified":"2023-10-06T05:21:58.111036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"d","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3654179987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3654179987},{"type":"v1","addr":"172.21.15.62:6835","nonce":3654179987}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 -- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e40067050 msgr2=0x7f5e40069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e40067050 0x7f5e40069500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c069180 tx=0x7f5e4c017000 comp rx=0 tx=0).stop 2023-10-06T05:22:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 -- 172.21.15.31:0/3746294155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 msgr2=0x7f5e5c154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 0x7f5e5c154210 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800c7e0 tx=0x7f5e58007df0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 -- 172.21.15.31:0/3746294155 shutdown_connections 2023-10-06T05:22:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5e40067050 0x7f5e40069500 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e5c1595a0 0x7f5e5c150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c068d80 0x7f5e5c154210 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 --2- 172.21.15.31:0/3746294155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e5c0cc0d0 0x7f5e5c0c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 -- 172.21.15.31:0/3746294155 >> 172.21.15.31:0/3746294155 conn(0x7f5e5c06b620 msgr2=0x7f5e5c0cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:18.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 -- 172.21.15.31:0/3746294155 shutdown_connections 2023-10-06T05:22:18.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.350+0000 7f5e635ff700 1 -- 172.21.15.31:0/3746294155 wait complete. 2023-10-06T05:22:18.355 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 58 2023-10-06T05:22:18.369 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:18.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.564+0000 7f95021ff700 1 Processor -- start 2023-10-06T05:22:18.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.564+0000 7f95021ff700 1 -- start start 2023-10-06T05:22:18.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.565+0000 7f95021ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94fc13fda0 0x7f94fc069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.565+0000 7f95021ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94fc0ca5b0 con 0x7f94fc0c7f30 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.565+0000 7f95021ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94fc06f740 con 0x7f94fc0c7b60 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.565+0000 7f95021ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc0c9c80 con 0x7f94fc13fda0 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.565+0000 7f94fbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94fc13fda0 0x7f94fc069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.565+0000 7f94fbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94fc13fda0 0x7f94fc069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46008/0 (socket says 172.21.15.31:46008) 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.565+0000 7f94faffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f94fc0c7f30 0x7f94fc13f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53786/0 (socket says 172.21.15.31:53786) 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.565+0000 7f94fbfff700 1 -- 172.21.15.31:0/3848138280 learned_addr learned my addr 172.21.15.31:0/3848138280 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.566+0000 7f94fbfff700 1 -- 172.21.15.31:0/3848138280 >> v1:172.21.15.62:6789/0 conn(0x7f94fc0c7b60 legacy=0x7f94fc06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.566+0000 7f94fbfff700 1 -- 172.21.15.31:0/3848138280 >> v1:172.21.15.31:6789/0 conn(0x7f94fc0c7f30 legacy=0x7f94fc13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.566+0000 7f94fbfff700 1 -- 172.21.15.31:0/3848138280 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc0c9f60 con 0x7f94fc13fda0 2023-10-06T05:22:18.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.566+0000 7f94fa7fc700 1 -- 172.21.15.31:0/3848138280 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575603287 0 0) 0x7f94fc0ca5b0 con 0x7f94fc0c7f30 2023-10-06T05:22:18.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.566+0000 7f94fbfff700 1 --2- 172.21.15.31:0/3848138280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc13fda0 0x7f94fc069880 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009f20 tx=0x7f94f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437ecc11e5153d63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:18.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.566+0000 7f94fa7fc700 1 -- 172.21.15.31:0/3848138280 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f400e070 con 0x7f94fc13fda0 2023-10-06T05:22:18.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.566+0000 7f94fa7fc700 1 -- 172.21.15.31:0/3848138280 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f400ba80 con 0x7f94fc13fda0 2023-10-06T05:22:18.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.566+0000 7f94fa7fc700 1 -- 172.21.15.31:0/3848138280 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40092e0 con 0x7f94fc13fda0 2023-10-06T05:22:18.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.567+0000 7f95021ff700 1 -- 172.21.15.31:0/3848138280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc13fda0 msgr2=0x7f94fc069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.567+0000 7f95021ff700 1 --2- 172.21.15.31:0/3848138280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc13fda0 0x7f94fc069880 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009f20 tx=0x7f94f4009840 comp rx=0 tx=0).stop 2023-10-06T05:22:18.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.567+0000 7f95021ff700 1 -- 172.21.15.31:0/3848138280 shutdown_connections 2023-10-06T05:22:18.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.567+0000 7f95021ff700 1 --2- 172.21.15.31:0/3848138280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc13fda0 0x7f94fc069880 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.567+0000 7f95021ff700 1 -- 172.21.15.31:0/3848138280 >> 172.21.15.31:0/3848138280 conn(0x7f94fc0cc0b0 msgr2=0x7f94fc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:18.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.567+0000 7f95021ff700 1 -- 172.21.15.31:0/3848138280 shutdown_connections 2023-10-06T05:22:18.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.567+0000 7f95021ff700 1 -- 172.21.15.31:0/3848138280 wait complete. 2023-10-06T05:22:18.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.568+0000 7f95021ff700 1 Processor -- start 2023-10-06T05:22:18.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.569+0000 7f95021ff700 1 -- start start 2023-10-06T05:22:18.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.569+0000 7f95021ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94fc0c7b60 0x7f94fc142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.569+0000 7f95021ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc0c7f30 0x7f94fc0c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.569+0000 7f95021ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94fc13fda0 0x7f94fc06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f95021ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0c22c0 con 0x7f94fc0c7f30 2023-10-06T05:22:18.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f95021ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0c25a0 con 0x7f94fc13fda0 2023-10-06T05:22:18.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f95021ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0c2880 con 0x7f94fc0c7b60 2023-10-06T05:22:18.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94faffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc0c7f30 0x7f94fc0c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94faffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc0c7f30 0x7f94fc0c18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46018/0 (socket says 172.21.15.31:46018) 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94faffd700 1 -- 172.21.15.31:0/1340261627 learned_addr learned my addr 172.21.15.31:0/1340261627 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94fbfff700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94fc13fda0 0x7f94fc06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94fb7fe700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94fc0c7b60 0x7f94fc142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94faffd700 1 -- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94fc0c7b60 msgr2=0x7f94fc142940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94faffd700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94fc0c7b60 0x7f94fc142940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94faffd700 1 -- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94fc13fda0 msgr2=0x7f94fc06e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94faffd700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94fc13fda0 0x7f94fc06e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94faffd700 1 -- 172.21.15.31:0/1340261627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f4009ae0 con 0x7f94fc0c7f30 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94fbfff700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94fc13fda0 0x7f94fc06e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.570+0000 7f94faffd700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc0c7f30 0x7f94fc0c18e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f94f000c810 tx=0x7f94f0007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:18.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.571+0000 7f94e37fe700 1 -- 172.21.15.31:0/1340261627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f00073d0 con 0x7f94fc0c7f30 2023-10-06T05:22:18.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.571+0000 7f94e37fe700 1 -- 172.21.15.31:0/1340261627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f0007550 con 0x7f94fc0c7f30 2023-10-06T05:22:18.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.571+0000 7f94e37fe700 1 -- 172.21.15.31:0/1340261627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0004a80 con 0x7f94fc0c7f30 2023-10-06T05:22:18.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.571+0000 7f95021ff700 1 -- 172.21.15.31:0/1340261627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94fc0c2b60 con 0x7f94fc0c7f30 2023-10-06T05:22:18.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.571+0000 7f95021ff700 1 -- 172.21.15.31:0/1340261627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc142e80 con 0x7f94fc0c7f30 2023-10-06T05:22:18.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.572+0000 7f95021ff700 1 -- 172.21.15.31:0/1340261627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc140660 con 0x7f94fc0c7f30 2023-10-06T05:22:18.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.573+0000 7f94e37fe700 1 -- 172.21.15.31:0/1340261627 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f94f001c3b0 con 0x7f94fc0c7f30 2023-10-06T05:22:18.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.574+0000 7f94e37fe700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94e8067050 0x7f94e8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:18.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.574+0000 7f94e37fe700 1 -- 172.21.15.31:0/1340261627 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f94f0083dd0 con 0x7f94fc0c7f30 2023-10-06T05:22:18.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.574+0000 7f94fb7fe700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94e8067050 0x7f94e8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:18.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.575+0000 7f94fb7fe700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94e8067050 0x7f94e8069500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f94ec003890 tx=0x7f94ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:18.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.578+0000 7f94e37fe700 1 -- 172.21.15.31:0/1340261627 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94fc140660 con 0x7f94fc0c7f30 2023-10-06T05:22:18.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.782+0000 7f95021ff700 1 -- 172.21.15.31:0/1340261627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94fc140a40 con 0x7f94fc0c7f30 2023-10-06T05:22:18.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.783+0000 7f94e37fe700 1 -- 172.21.15.31:0/1340261627 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+17046 (secure 0 0 0) 0x7f94fc140a40 con 0x7f94fc0c7f30 2023-10-06T05:22:18.785 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:18.785 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:22:14.597965+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:52.267847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:53.579487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6835/4141055233":"2023-10-07T05:22:11.589575+0000","172.21.15.31:6833/4141055233":"2023-10-07T05:22:11.589575+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:18.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 -- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94e8067050 msgr2=0x7f94e8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94e8067050 0x7f94e8069500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f94ec003890 tx=0x7f94ec009040 comp rx=0 tx=0).stop 2023-10-06T05:22:18.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 -- 172.21.15.31:0/1340261627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc0c7f30 msgr2=0x7f94fc0c18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:18.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc0c7f30 0x7f94fc0c18e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f94f000c810 tx=0x7f94f0007d90 comp rx=0 tx=0).stop 2023-10-06T05:22:18.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 -- 172.21.15.31:0/1340261627 shutdown_connections 2023-10-06T05:22:18.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94e8067050 0x7f94e8069500 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94fc13fda0 0x7f94fc06e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94fc0c7b60 0x7f94fc142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 --2- 172.21.15.31:0/1340261627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94fc0c7f30 0x7f94fc0c18e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:18.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 -- 172.21.15.31:0/1340261627 >> 172.21.15.31:0/1340261627 conn(0x7f94fc0cc0b0 msgr2=0x7f94fc0b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:18.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 -- 172.21.15.31:0/1340261627 shutdown_connections 2023-10-06T05:22:18.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.786+0000 7f95021ff700 1 -- 172.21.15.31:0/1340261627 wait complete. 2023-10-06T05:22:18.805 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:22:18.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:19.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.999+0000 7fd381bff700 1 Processor -- start 2023-10-06T05:22:19.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:18.999+0000 7fd381bff700 1 -- start start 2023-10-06T05:22:19.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.000+0000 7fd381bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 0x7fd37c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.000+0000 7fd381bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd37c0aee70 0x7fd37c13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.000+0000 7fd381bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c0ca5b0 con 0x7fd37c0c7f30 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.000+0000 7fd381bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37c0af2d0 con 0x7fd37c0aee70 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.000+0000 7fd381bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c06f740 con 0x7fd37c0c7b60 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.001+0000 7fd37bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd37c0aee70 0x7fd37c13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.001+0000 7fd37affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd37c0c7f30 0x7fd37c06ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53794/0 (socket says 172.21.15.31:53794) 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.001+0000 7fd37affd700 1 -- 172.21.15.31:0/1817461966 learned_addr learned my addr 172.21.15.31:0/1817461966 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.001+0000 7fd37b7fe700 1 --2- 172.21.15.31:0/1817461966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 0x7fd37c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.001+0000 7fd37bfff700 1 -- 172.21.15.31:0/1817461966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 msgr2=0x7fd37c069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.001+0000 7fd37bfff700 1 --2- 172.21.15.31:0/1817461966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 0x7fd37c069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.001+0000 7fd37bfff700 1 -- 172.21.15.31:0/1817461966 >> v1:172.21.15.31:6789/0 conn(0x7fd37c0c7f30 legacy=0x7fd37c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.001+0000 7fd37bfff700 1 -- 172.21.15.31:0/1817461966 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0c9c80 con 0x7fd37c0aee70 2023-10-06T05:22:19.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.001+0000 7fd37a7fc700 1 -- 172.21.15.31:0/1817461966 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493047965 0 0) 0x7fd37c0ca5b0 con 0x7fd37c0c7f30 2023-10-06T05:22:19.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.002+0000 7fd37bfff700 1 --2- 172.21.15.31:0/1817461966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 0x7fd37c13f690 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd37400d0d0 tx=0x7fd37400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bed509342d585e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.002+0000 7fd37a7fc700 1 -- 172.21.15.31:0/1817461966 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374010070 con 0x7fd37c0aee70 2023-10-06T05:22:19.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.002+0000 7fd37a7fc700 1 -- 172.21.15.31:0/1817461966 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37400b2e0 con 0x7fd37c0aee70 2023-10-06T05:22:19.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.002+0000 7fd37a7fc700 1 -- 172.21.15.31:0/1817461966 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400dd80 con 0x7fd37c0aee70 2023-10-06T05:22:19.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.002+0000 7fd381bff700 1 -- 172.21.15.31:0/1817461966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 msgr2=0x7fd37c13f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.002+0000 7fd381bff700 1 --2- 172.21.15.31:0/1817461966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 0x7fd37c13f690 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd37400d0d0 tx=0x7fd37400b990 comp rx=0 tx=0).stop 2023-10-06T05:22:19.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.002+0000 7fd381bff700 1 -- 172.21.15.31:0/1817461966 shutdown_connections 2023-10-06T05:22:19.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.002+0000 7fd381bff700 1 --2- 172.21.15.31:0/1817461966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 0x7fd37c13f690 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.002+0000 7fd381bff700 1 --2- 172.21.15.31:0/1817461966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 0x7fd37c069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.002+0000 7fd381bff700 1 -- 172.21.15.31:0/1817461966 >> 172.21.15.31:0/1817461966 conn(0x7fd37c0cc0b0 msgr2=0x7fd37c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:19.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.003+0000 7fd381bff700 1 -- 172.21.15.31:0/1817461966 shutdown_connections 2023-10-06T05:22:19.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.003+0000 7fd381bff700 1 -- 172.21.15.31:0/1817461966 wait complete. 2023-10-06T05:22:19.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.004+0000 7fd381bff700 1 Processor -- start 2023-10-06T05:22:19.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.004+0000 7fd381bff700 1 -- start start 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.005+0000 7fd381bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 0x7fd37c0ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.005+0000 7fd381bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 0x7fd37c0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.005+0000 7fd381bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0c7f30 0x7fd37c1692f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.005+0000 7fd381bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0c3c10 con 0x7fd37c0aee70 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.005+0000 7fd381bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0ae770 con 0x7fd37c0c7f30 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.005+0000 7fd381bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c156470 con 0x7fd37c0c7b60 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.005+0000 7fd37b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 0x7fd37c0ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.005+0000 7fd37b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 0x7fd37c0ab9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46046/0 (socket says 172.21.15.31:46046) 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.005+0000 7fd37b7fe700 1 -- 172.21.15.31:0/134365255 learned_addr learned my addr 172.21.15.31:0/134365255 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd37affd700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 0x7fd37c0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd37bfff700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0c7f30 0x7fd37c1692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd37b7fe700 1 -- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 msgr2=0x7fd37c0c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd37b7fe700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 0x7fd37c0c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd37b7fe700 1 -- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0c7f30 msgr2=0x7fd37c1692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd37b7fe700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0c7f30 0x7fd37c1692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd37b7fe700 1 -- 172.21.15.31:0/134365255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37400bc30 con 0x7fd37c0aee70 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd37bfff700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0c7f30 0x7fd37c1692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd37affd700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 0x7fd37c0c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd37b7fe700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 0x7fd37c0ab9a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00ecb0 tx=0x7fd36c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd3637fe700 1 -- 172.21.15.31:0/134365255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00c9d0 con 0x7fd37c0aee70 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd3637fe700 1 -- 172.21.15.31:0/134365255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c00cb30 con 0x7fd37c0aee70 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd381bff700 1 -- 172.21.15.31:0/134365255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c0c3940 con 0x7fd37c0aee70 2023-10-06T05:22:19.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd3637fe700 1 -- 172.21.15.31:0/134365255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00ceb0 con 0x7fd37c0aee70 2023-10-06T05:22:19.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.006+0000 7fd381bff700 1 -- 172.21.15.31:0/134365255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c0ae1b0 con 0x7fd37c0aee70 2023-10-06T05:22:19.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.007+0000 7fd381bff700 1 -- 172.21.15.31:0/134365255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c04c380 con 0x7fd37c0aee70 2023-10-06T05:22:19.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.008+0000 7fd3637fe700 1 -- 172.21.15.31:0/134365255 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd36c013680 con 0x7fd37c0aee70 2023-10-06T05:22:19.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.009+0000 7fd3637fe700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd368067050 0x7fd368069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.009+0000 7fd3637fe700 1 -- 172.21.15.31:0/134365255 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd36c084fd0 con 0x7fd37c0aee70 2023-10-06T05:22:19.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.010+0000 7fd37affd700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd368067050 0x7fd368069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.010+0000 7fd37affd700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd368067050 0x7fd368069500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0c6250 tx=0x7fd370009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.013+0000 7fd3637fe700 1 -- 172.21.15.31:0/134365255 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd36c051f00 con 0x7fd37c0aee70 2023-10-06T05:22:19.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.265+0000 7fd381bff700 1 -- 172.21.15.31:0/134365255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd37c0c3d70 con 0x7fd37c0aee70 2023-10-06T05:22:19.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.265+0000 7fd3637fe700 1 -- 172.21.15.31:0/134365255 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fd36c017090 con 0x7fd37c0aee70 2023-10-06T05:22:19.267 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:19.267 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1878570982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1878570982},{"type":"v1","addr":"172.21.15.62:6834","nonce":1878570982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6687,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/427382574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":427382574},{"type":"v1","addr":"172.21.15.31:6834","nonce":427382574}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1760549774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1760549774},{"type":"v1","addr":"172.21.15.31:6835","nonce":1760549774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:54.590687+0000","modified":"2023-10-06T05:21:58.111036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"d","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3654179987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3654179987},{"type":"v1","addr":"172.21.15.62:6835","nonce":3654179987}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:19.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.267+0000 7fd381bff700 1 -- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd368067050 msgr2=0x7fd368069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.267+0000 7fd381bff700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd368067050 0x7fd368069500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0c6250 tx=0x7fd370009040 comp rx=0 tx=0).stop 2023-10-06T05:22:19.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.267+0000 7fd381bff700 1 -- 172.21.15.31:0/134365255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 msgr2=0x7fd37c0ab9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.267+0000 7fd381bff700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 0x7fd37c0ab9a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00ecb0 tx=0x7fd36c009120 comp rx=0 tx=0).stop 2023-10-06T05:22:19.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.268+0000 7fd381bff700 1 -- 172.21.15.31:0/134365255 shutdown_connections 2023-10-06T05:22:19.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.268+0000 7fd381bff700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd368067050 0x7fd368069500 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.268+0000 7fd381bff700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c0c7f30 0x7fd37c1692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.268+0000 7fd381bff700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd37c0aee70 0x7fd37c0ab9a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.268+0000 7fd381bff700 1 --2- 172.21.15.31:0/134365255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd37c0c7b60 0x7fd37c0c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.268+0000 7fd381bff700 1 -- 172.21.15.31:0/134365255 >> 172.21.15.31:0/134365255 conn(0x7fd37c0cc0b0 msgr2=0x7fd37c0c6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:19.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.268+0000 7fd381bff700 1 -- 172.21.15.31:0/134365255 shutdown_connections 2023-10-06T05:22:19.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.268+0000 7fd381bff700 1 -- 172.21.15.31:0/134365255 wait complete. 2023-10-06T05:22:19.271 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 58 2023-10-06T05:22:19.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.480+0000 7fcfb25ff700 1 Processor -- start 2023-10-06T05:22:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.481+0000 7fcfb25ff700 1 -- start start 2023-10-06T05:22:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.482+0000 7fcfb25ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfac13a1e0 0x7fcfac0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.482+0000 7fcfb25ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcfac13a5b0 0x7fcfac06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.482+0000 7fcfb25ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac0c71b0 con 0x7fcfac0c6c30 2023-10-06T05:22:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.482+0000 7fcfb25ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfac0c7490 con 0x7fcfac13a5b0 2023-10-06T05:22:19.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.482+0000 7fcfb25ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac0c7770 con 0x7fcfac13a1e0 2023-10-06T05:22:19.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.482+0000 7fcfab7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcfac13a5b0 0x7fcfac06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.482+0000 7fcfab7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcfac13a5b0 0x7fcfac06d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46060/0 (socket says 172.21.15.31:46060) 2023-10-06T05:22:19.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.482+0000 7fcfab7fe700 1 -- 172.21.15.31:0/558210432 learned_addr learned my addr 172.21.15.31:0/558210432 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:19.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.482+0000 7fcfabfff700 1 --2- 172.21.15.31:0/558210432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfac13a1e0 0x7fcfac0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.483+0000 7fcfab7fe700 1 -- 172.21.15.31:0/558210432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfac13a1e0 msgr2=0x7fcfac0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:19.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.483+0000 7fcfab7fe700 1 --2- 172.21.15.31:0/558210432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfac13a1e0 0x7fcfac0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.483+0000 7fcfab7fe700 1 -- 172.21.15.31:0/558210432 >> v1:172.21.15.62:6789/0 conn(0x7fcfac0c6c30 legacy=0x7fcfac15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:19.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.483+0000 7fcfab7fe700 1 -- 172.21.15.31:0/558210432 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac0c7a50 con 0x7fcfac13a5b0 2023-10-06T05:22:19.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.483+0000 7fcfab7fe700 1 --2- 172.21.15.31:0/558210432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a5b0 0x7fcfac06d440 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00099b0 tx=0x7fcfa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ec3ac6f660311c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.483+0000 7fcfaaffd700 1 -- 172.21.15.31:0/558210432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000b820 con 0x7fcfac13a5b0 2023-10-06T05:22:19.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.483+0000 7fcfaaffd700 1 -- 172.21.15.31:0/558210432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa000b980 con 0x7fcfac13a5b0 2023-10-06T05:22:19.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.483+0000 7fcfaaffd700 1 -- 172.21.15.31:0/558210432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000bd20 con 0x7fcfac13a5b0 2023-10-06T05:22:19.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.484+0000 7fcfb25ff700 1 -- 172.21.15.31:0/558210432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a5b0 msgr2=0x7fcfac06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.484+0000 7fcfb25ff700 1 --2- 172.21.15.31:0/558210432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a5b0 0x7fcfac06d440 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00099b0 tx=0x7fcfa0009250 comp rx=0 tx=0).stop 2023-10-06T05:22:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.484+0000 7fcfb25ff700 1 -- 172.21.15.31:0/558210432 shutdown_connections 2023-10-06T05:22:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.484+0000 7fcfb25ff700 1 --2- 172.21.15.31:0/558210432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfac13a1e0 0x7fcfac0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.484+0000 7fcfb25ff700 1 --2- 172.21.15.31:0/558210432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a5b0 0x7fcfac06d440 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.484+0000 7fcfb25ff700 1 -- 172.21.15.31:0/558210432 >> 172.21.15.31:0/558210432 conn(0x7fcfac0cc0b0 msgr2=0x7fcfac0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.484+0000 7fcfb25ff700 1 -- 172.21.15.31:0/558210432 shutdown_connections 2023-10-06T05:22:19.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.484+0000 7fcfb25ff700 1 -- 172.21.15.31:0/558210432 wait complete. 2023-10-06T05:22:19.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.485+0000 7fcfb25ff700 1 Processor -- start 2023-10-06T05:22:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.486+0000 7fcfb25ff700 1 -- start start 2023-10-06T05:22:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.486+0000 7fcfb25ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfac0c6c30 0x7fcfac152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.486+0000 7fcfb25ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a1e0 0x7fcfac0c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.486+0000 7fcfb25ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfac13a5b0 0x7fcfac153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.486+0000 7fcfb25ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac0c5550 con 0x7fcfac13a1e0 2023-10-06T05:22:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.486+0000 7fcfb25ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac0c4670 con 0x7fcfac0c6c30 2023-10-06T05:22:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.486+0000 7fcfb25ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac0c4940 con 0x7fcfac13a5b0 2023-10-06T05:22:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a1e0 0x7fcfac0c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a1e0 0x7fcfac0c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46064/0 (socket says 172.21.15.31:46064) 2023-10-06T05:22:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfab7fe700 1 -- 172.21.15.31:0/1124295287 learned_addr learned my addr 172.21.15.31:0/1124295287 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfabfff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfac0c6c30 0x7fcfac152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfb0b9c700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfac13a5b0 0x7fcfac153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfab7fe700 1 -- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfac13a5b0 msgr2=0x7fcfac153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfab7fe700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfac13a5b0 0x7fcfac153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfab7fe700 1 -- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfac0c6c30 msgr2=0x7fcfac152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfab7fe700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfac0c6c30 0x7fcfac152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfab7fe700 1 -- 172.21.15.31:0/1124295287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa00094f0 con 0x7fcfac13a1e0 2023-10-06T05:22:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfabfff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfac0c6c30 0x7fcfac152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.487+0000 7fcfab7fe700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a1e0 0x7fcfac0c4130 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009290 tx=0x7fcfa0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.488+0000 7fcf93fff700 1 -- 172.21.15.31:0/1124295287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0011d90 con 0x7fcfac13a1e0 2023-10-06T05:22:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.488+0000 7fcf93fff700 1 -- 172.21.15.31:0/1124295287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa0020070 con 0x7fcfac13a1e0 2023-10-06T05:22:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.488+0000 7fcf93fff700 1 -- 172.21.15.31:0/1124295287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa001b6a0 con 0x7fcfac13a1e0 2023-10-06T05:22:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.488+0000 7fcfb25ff700 1 -- 172.21.15.31:0/1124295287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac0c4c20 con 0x7fcfac13a1e0 2023-10-06T05:22:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.488+0000 7fcfb25ff700 1 -- 172.21.15.31:0/1124295287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac0c51e0 con 0x7fcfac13a1e0 2023-10-06T05:22:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.489+0000 7fcfb25ff700 1 -- 172.21.15.31:0/1124295287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac0c5370 con 0x7fcfac13a1e0 2023-10-06T05:22:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.490+0000 7fcf93fff700 1 -- 172.21.15.31:0/1124295287 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fcfa0022030 con 0x7fcfac13a1e0 2023-10-06T05:22:19.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.490+0000 7fcf93fff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcf98067050 0x7fcf98069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.490+0000 7fcf93fff700 1 -- 172.21.15.31:0/1124295287 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fcfa0086790 con 0x7fcfac13a1e0 2023-10-06T05:22:19.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.491+0000 7fcfabfff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcf98067050 0x7fcf98069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.492+0000 7fcfabfff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcf98067050 0x7fcf98069500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c006aa0 tx=0x7fcf9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.495+0000 7fcf93fff700 1 -- 172.21.15.31:0/1124295287 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfa00536c0 con 0x7fcfac13a1e0 2023-10-06T05:22:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.747+0000 7fcfb25ff700 1 -- 172.21.15.31:0/1124295287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfac0c5370 con 0x7fcfac13a1e0 2023-10-06T05:22:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.747+0000 7fcf93fff700 1 -- 172.21.15.31:0/1124295287 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fcfa0056ce0 con 0x7fcfac13a1e0 2023-10-06T05:22:19.749 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:19.750 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1878570982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1878570982},{"type":"v1","addr":"172.21.15.62:6834","nonce":1878570982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6687,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/427382574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":427382574},{"type":"v1","addr":"172.21.15.31:6834","nonce":427382574}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1760549774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1760549774},{"type":"v1","addr":"172.21.15.31:6835","nonce":1760549774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:21:54.590687+0000","modified":"2023-10-06T05:21:58.111036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6702":{"gid":6702,"name":"d","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3654179987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3654179987},{"type":"v1","addr":"172.21.15.62:6835","nonce":3654179987}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T05:22:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 -- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcf98067050 msgr2=0x7fcf98069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcf98067050 0x7fcf98069500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c006aa0 tx=0x7fcf9c008040 comp rx=0 tx=0).stop 2023-10-06T05:22:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 -- 172.21.15.31:0/1124295287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a1e0 msgr2=0x7fcfac0c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a1e0 0x7fcfac0c4130 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009290 tx=0x7fcfa0016000 comp rx=0 tx=0).stop 2023-10-06T05:22:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 -- 172.21.15.31:0/1124295287 shutdown_connections 2023-10-06T05:22:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcf98067050 0x7fcf98069500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfac13a5b0 0x7fcfac153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfac0c6c30 0x7fcfac152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 --2- 172.21.15.31:0/1124295287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac13a1e0 0x7fcfac0c4130 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 -- 172.21.15.31:0/1124295287 >> 172.21.15.31:0/1124295287 conn(0x7fcfac0cc0b0 msgr2=0x7fcfac13c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:19.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 -- 172.21.15.31:0/1124295287 shutdown_connections 2023-10-06T05:22:19.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.750+0000 7fcfb25ff700 1 -- 172.21.15.31:0/1124295287 wait complete. 2023-10-06T05:22:19.754 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 58 2023-10-06T05:22:19.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:19.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.963+0000 7f15713ff700 1 Processor -- start 2023-10-06T05:22:19.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.964+0000 7f15713ff700 1 -- start start 2023-10-06T05:22:19.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.965+0000 7f15713ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 0x7f156c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.965+0000 7f15713ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f156c0cc0d0 0x7f156c0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.965+0000 7f15713ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0c95b0 con 0x7f156c1595a0 2023-10-06T05:22:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.965+0000 7f15713ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0c9890 con 0x7f156c0cc0d0 2023-10-06T05:22:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.965+0000 7f15713ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0c9b70 con 0x7f156c068d80 2023-10-06T05:22:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.965+0000 7f156affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f156c1595a0 0x7f156c068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53802/0 (socket says 172.21.15.31:53802) 2023-10-06T05:22:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.965+0000 7f156affd700 1 -- 172.21.15.31:0/1796651842 learned_addr learned my addr 172.21.15.31:0/1796651842 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.965+0000 7f156b7fe700 1 --2- 172.21.15.31:0/1796651842 >> v2:172.21.15.31:3300/0 conn(0x7f156c0cc0d0 0x7f156c0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.965+0000 7f156a7fc700 1 --2- 172.21.15.31:0/1796651842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 0x7f156c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.966+0000 7f156b7fe700 1 -- 172.21.15.31:0/1796651842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 msgr2=0x7f156c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.966+0000 7f156b7fe700 1 --2- 172.21.15.31:0/1796651842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 0x7f156c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.966+0000 7f156b7fe700 1 -- 172.21.15.31:0/1796651842 >> v1:172.21.15.31:6789/0 conn(0x7f156c1595a0 legacy=0x7f156c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.966+0000 7f1569ffb700 1 -- 172.21.15.31:0/1796651842 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341161164 0 0) 0x7f156c0c95b0 con 0x7f156c1595a0 2023-10-06T05:22:19.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.966+0000 7f156b7fe700 1 -- 172.21.15.31:0/1796651842 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0ca130 con 0x7f156c0cc0d0 2023-10-06T05:22:19.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.966+0000 7f156a7fc700 1 --2- 172.21.15.31:0/1796651842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 0x7f156c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:19.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.966+0000 7f156b7fe700 1 --2- 172.21.15.31:0/1796651842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 0x7f156c0c8930 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f1564009e60 tx=0x7f1564009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95a980f3003526c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.966+0000 7f1569ffb700 1 -- 172.21.15.31:0/1796651842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156400e070 con 0x7f156c0cc0d0 2023-10-06T05:22:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.966+0000 7f1569ffb700 1 -- 172.21.15.31:0/1796651842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156400bc30 con 0x7f156c0cc0d0 2023-10-06T05:22:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.966+0000 7f1569ffb700 1 -- 172.21.15.31:0/1796651842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564012430 con 0x7f156c0cc0d0 2023-10-06T05:22:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.967+0000 7f15713ff700 1 -- 172.21.15.31:0/1796651842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 msgr2=0x7f156c0c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.967+0000 7f15713ff700 1 --2- 172.21.15.31:0/1796651842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 0x7f156c0c8930 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f1564009e60 tx=0x7f1564009780 comp rx=0 tx=0).stop 2023-10-06T05:22:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.967+0000 7f15713ff700 1 -- 172.21.15.31:0/1796651842 shutdown_connections 2023-10-06T05:22:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.967+0000 7f15713ff700 1 --2- 172.21.15.31:0/1796651842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 0x7f156c0c8930 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.967+0000 7f15713ff700 1 --2- 172.21.15.31:0/1796651842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 0x7f156c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.967+0000 7f15713ff700 1 -- 172.21.15.31:0/1796651842 >> 172.21.15.31:0/1796651842 conn(0x7f156c06b620 msgr2=0x7f156c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.967+0000 7f15713ff700 1 -- 172.21.15.31:0/1796651842 shutdown_connections 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.967+0000 7f15713ff700 1 -- 172.21.15.31:0/1796651842 wait complete. 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.968+0000 7f15713ff700 1 Processor -- start 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.969+0000 7f15713ff700 1 -- start start 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.969+0000 7f15713ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 0x7f156c15be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.969+0000 7f15713ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 0x7f156c0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f15713ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f156c1595a0 0x7f156c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f15713ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0c8d20 con 0x7f156c0cc0d0 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f15713ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0c7a90 con 0x7f156c1595a0 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f15713ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0c8890 con 0x7f156c068d80 2023-10-06T05:22:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 0x7f156c0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 0x7f156c0c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46090/0 (socket says 172.21.15.31:46090) 2023-10-06T05:22:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156a7fc700 1 -- 172.21.15.31:0/3511246122 learned_addr learned my addr 172.21.15.31:0/3511246122 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156affd700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 0x7f156c15be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156b7fe700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f156c1595a0 0x7f156c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156a7fc700 1 -- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 msgr2=0x7f156c15be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156a7fc700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 0x7f156c15be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156a7fc700 1 -- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f156c1595a0 msgr2=0x7f156c155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156a7fc700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f156c1595a0 0x7f156c155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156a7fc700 1 -- 172.21.15.31:0/3511246122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1564009a20 con 0x7f156c0cc0d0 2023-10-06T05:22:19.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.970+0000 7f156b7fe700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f156c1595a0 0x7f156c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:19.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.971+0000 7f156a7fc700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 0x7f156c0c7550 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f156000d890 tx=0x7f1560008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.971+0000 7f1552ffd700 1 -- 172.21.15.31:0/3511246122 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15600125f0 con 0x7f156c0cc0d0 2023-10-06T05:22:19.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.971+0000 7f1552ffd700 1 -- 172.21.15.31:0/3511246122 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1560012750 con 0x7f156c0cc0d0 2023-10-06T05:22:19.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.971+0000 7f15713ff700 1 -- 172.21.15.31:0/3511246122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c155f80 con 0x7f156c0cc0d0 2023-10-06T05:22:19.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.971+0000 7f1552ffd700 1 -- 172.21.15.31:0/3511246122 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560012af0 con 0x7f156c0cc0d0 2023-10-06T05:22:19.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.971+0000 7f15713ff700 1 -- 172.21.15.31:0/3511246122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c0c7f60 con 0x7f156c0cc0d0 2023-10-06T05:22:19.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.972+0000 7f15713ff700 1 -- 172.21.15.31:0/3511246122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c0c8a20 con 0x7f156c0cc0d0 2023-10-06T05:22:19.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.973+0000 7f1552ffd700 1 -- 172.21.15.31:0/3511246122 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f1560014430 con 0x7f156c0cc0d0 2023-10-06T05:22:19.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.974+0000 7f1552ffd700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067050 0x7f1558069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:19.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.974+0000 7f1552ffd700 1 -- 172.21.15.31:0/3511246122 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1560084c40 con 0x7f156c0cc0d0 2023-10-06T05:22:19.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.974+0000 7f156affd700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067050 0x7f1558069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:19.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.975+0000 7f156affd700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067050 0x7f1558069500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f156c0c4130 tx=0x7f155c007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:19.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:19.978+0000 7f1552ffd700 1 -- 172.21.15.31:0/3511246122 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1560051b70 con 0x7f156c0cc0d0 2023-10-06T05:22:20.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.182+0000 7f15713ff700 1 -- 172.21.15.31:0/3511246122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f156c04c380 con 0x7f156c0cc0d0 2023-10-06T05:22:20.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.183+0000 7f1552ffd700 1 -- 172.21.15.31:0/3511246122 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+17046 (secure 0 0 0) 0x7f1560055190 con 0x7f156c0cc0d0 2023-10-06T05:22:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:20.185 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:22:14.597965+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:21:52.267847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T05:21:53.579487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6835/4141055233":"2023-10-07T05:22:11.589575+0000","172.21.15.31:6833/4141055233":"2023-10-07T05:22:11.589575+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.185+0000 7f15713ff700 1 -- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067050 msgr2=0x7f1558069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.185+0000 7f15713ff700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067050 0x7f1558069500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f156c0c4130 tx=0x7f155c007d30 comp rx=0 tx=0).stop 2023-10-06T05:22:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.185+0000 7f15713ff700 1 -- 172.21.15.31:0/3511246122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 msgr2=0x7f156c0c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.185+0000 7f15713ff700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 0x7f156c0c7550 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f156000d890 tx=0x7f1560008040 comp rx=0 tx=0).stop 2023-10-06T05:22:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.186+0000 7f15713ff700 1 -- 172.21.15.31:0/3511246122 shutdown_connections 2023-10-06T05:22:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.186+0000 7f15713ff700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067050 0x7f1558069500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.186+0000 7f15713ff700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f156c1595a0 0x7f156c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.186+0000 7f15713ff700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156c068d80 0x7f156c15be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.186+0000 7f15713ff700 1 --2- 172.21.15.31:0/3511246122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0cc0d0 0x7f156c0c7550 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.186+0000 7f15713ff700 1 -- 172.21.15.31:0/3511246122 >> 172.21.15.31:0/3511246122 conn(0x7f156c06b620 msgr2=0x7f156c06cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.186+0000 7f15713ff700 1 -- 172.21.15.31:0/3511246122 shutdown_connections 2023-10-06T05:22:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.186+0000 7f15713ff700 1 -- 172.21.15.31:0/3511246122 wait complete. 2023-10-06T05:22:20.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:22:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.401+0000 7f21e5bff700 1 Processor -- start 2023-10-06T05:22:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.401+0000 7f21e5bff700 1 -- start start 2023-10-06T05:22:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.402+0000 7f21e5bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21e00c7b60 0x7f21e0069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.402+0000 7f21e5bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21e00aee70 0x7f21e013f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.402+0000 7f21e5bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e00ca5b0 con 0x7f21e00c7f30 2023-10-06T05:22:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.402+0000 7f21e5bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e00af2d0 con 0x7f21e00aee70 2023-10-06T05:22:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.402+0000 7f21e5bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e006f740 con 0x7f21e00c7b60 2023-10-06T05:22:20.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.403+0000 7f21dffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21e00aee70 0x7f21e013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.403+0000 7f21dffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21e00aee70 0x7f21e013f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46094/0 (socket says 172.21.15.31:46094) 2023-10-06T05:22:20.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.403+0000 7f21dffff700 1 -- 172.21.15.31:0/322231416 learned_addr learned my addr 172.21.15.31:0/322231416 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:20.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.403+0000 7f21df7fe700 1 --2- 172.21.15.31:0/322231416 >> v2:172.21.15.62:3300/0 conn(0x7f21e00c7b60 0x7f21e0069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.403+0000 7f21dffff700 1 -- 172.21.15.31:0/322231416 >> v2:172.21.15.62:3300/0 conn(0x7f21e00c7b60 msgr2=0x7f21e0069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.403+0000 7f21dffff700 1 --2- 172.21.15.31:0/322231416 >> v2:172.21.15.62:3300/0 conn(0x7f21e00c7b60 0x7f21e0069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.403+0000 7f21dffff700 1 -- 172.21.15.31:0/322231416 >> v1:172.21.15.62:6789/0 conn(0x7f21e00c7f30 legacy=0x7f21e006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.403+0000 7f21dffff700 1 -- 172.21.15.31:0/322231416 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e00c9c80 con 0x7f21e00aee70 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21dffff700 1 --2- 172.21.15.31:0/322231416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00aee70 0x7f21e013f690 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f21e006fa80 tx=0x7f21d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1667fdb5b0e3d13f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21de7fc700 1 -- 172.21.15.31:0/322231416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d800e070 con 0x7f21e00aee70 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21de7fc700 1 -- 172.21.15.31:0/322231416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d800ba20 con 0x7f21e00aee70 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21de7fc700 1 -- 172.21.15.31:0/322231416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d80092e0 con 0x7f21e00aee70 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21e5bff700 1 -- 172.21.15.31:0/322231416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00aee70 msgr2=0x7f21e013f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21e5bff700 1 --2- 172.21.15.31:0/322231416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00aee70 0x7f21e013f690 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f21e006fa80 tx=0x7f21d8009840 comp rx=0 tx=0).stop 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21e5bff700 1 -- 172.21.15.31:0/322231416 shutdown_connections 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21e5bff700 1 --2- 172.21.15.31:0/322231416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00aee70 0x7f21e013f690 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21e5bff700 1 --2- 172.21.15.31:0/322231416 >> v2:172.21.15.62:3300/0 conn(0x7f21e00c7b60 0x7f21e0069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21e5bff700 1 -- 172.21.15.31:0/322231416 >> 172.21.15.31:0/322231416 conn(0x7f21e00cc0b0 msgr2=0x7f21e00cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.404+0000 7f21e5bff700 1 -- 172.21.15.31:0/322231416 shutdown_connections 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.405+0000 7f21e5bff700 1 -- 172.21.15.31:0/322231416 wait complete. 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.406+0000 7f21e5bff700 1 Processor -- start 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.406+0000 7f21e5bff700 1 -- start start 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.407+0000 7f21e5bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21e00aee70 0x7f21e016b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.407+0000 7f21e5bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e00c7b60 0x7f21e016bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.407+0000 7f21e5bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00c7f30 0x7f21e0169be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.407+0000 7f21e5bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00a5ec0 con 0x7f21e00c7f30 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.407+0000 7f21e5bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e013dc10 con 0x7f21e00c7b60 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.407+0000 7f21e5bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e016c510 con 0x7f21e00aee70 2023-10-06T05:22:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.407+0000 7f21dffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00c7f30 0x7f21e0169be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.407+0000 7f21dffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00c7f30 0x7f21e0169be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46104/0 (socket says 172.21.15.31:46104) 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.407+0000 7f21dffff700 1 -- 172.21.15.31:0/1377021961 learned_addr learned my addr 172.21.15.31:0/1377021961 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21df7fe700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21e00aee70 0x7f21e016b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21deffd700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e00c7b60 0x7f21e016bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21dffff700 1 -- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21e00aee70 msgr2=0x7f21e016b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21dffff700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21e00aee70 0x7f21e016b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21dffff700 1 -- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e00c7b60 msgr2=0x7f21e016bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21dffff700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e00c7b60 0x7f21e016bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21dffff700 1 -- 172.21.15.31:0/1377021961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8009ae0 con 0x7f21e00c7f30 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21deffd700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e00c7b60 0x7f21e016bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21dffff700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00c7f30 0x7f21e0169be0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009880 tx=0x7f21d8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21c77fe700 1 -- 172.21.15.31:0/1377021961 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8008be0 con 0x7f21e00c7f30 2023-10-06T05:22:20.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21c77fe700 1 -- 172.21.15.31:0/1377021961 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d8008d40 con 0x7f21e00c7f30 2023-10-06T05:22:20.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21e5bff700 1 -- 172.21.15.31:0/1377021961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e016c7f0 con 0x7f21e00c7f30 2023-10-06T05:22:20.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.408+0000 7f21c77fe700 1 -- 172.21.15.31:0/1377021961 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d80266b0 con 0x7f21e00c7f30 2023-10-06T05:22:20.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.409+0000 7f21e5bff700 1 -- 172.21.15.31:0/1377021961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e016cdb0 con 0x7f21e00c7f30 2023-10-06T05:22:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.410+0000 7f21c57fa700 1 -- 172.21.15.31:0/1377021961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b80052b0 con 0x7f21e00c7f30 2023-10-06T05:22:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.410+0000 7f21c77fe700 1 -- 172.21.15.31:0/1377021961 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f21d8003040 con 0x7f21e00c7f30 2023-10-06T05:22:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.411+0000 7f21c77fe700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21cc067050 0x7f21cc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.411+0000 7f21c77fe700 1 -- 172.21.15.31:0/1377021961 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f21d8086a10 con 0x7f21e00c7f30 2023-10-06T05:22:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.412+0000 7f21df7fe700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21cc067050 0x7f21cc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.412+0000 7f21df7fe700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21cc067050 0x7f21cc069500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009d30 tx=0x7f21d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.416+0000 7f21c77fe700 1 -- 172.21.15.31:0/1377021961 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21d8053960 con 0x7f21e00c7f30 2023-10-06T05:22:20.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.665+0000 7f21c57fa700 1 -- 172.21.15.31:0/1377021961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f21b8005830 con 0x7f21e00c7f30 2023-10-06T05:22:20.667 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:22:20.666+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:22:20.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.673+0000 7f21c77fe700 1 -- 172.21.15.31:0/1377021961 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7f21d8056f80 con 0x7f21e00c7f30 2023-10-06T05:22:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 -- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21cc067050 msgr2=0x7f21cc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21cc067050 0x7f21cc069500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009d30 tx=0x7f21d0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 -- 172.21.15.31:0/1377021961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00c7f30 msgr2=0x7f21e0169be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00c7f30 0x7f21e0169be0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009880 tx=0x7f21d8018000 comp rx=0 tx=0).stop 2023-10-06T05:22:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 -- 172.21.15.31:0/1377021961 shutdown_connections 2023-10-06T05:22:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21cc067050 0x7f21cc069500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e00c7f30 0x7f21e0169be0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21e00aee70 0x7f21e016b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 --2- 172.21.15.31:0/1377021961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e00c7b60 0x7f21e016bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 -- 172.21.15.31:0/1377021961 >> 172.21.15.31:0/1377021961 conn(0x7f21e00cc0b0 msgr2=0x7f21e013e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 -- 172.21.15.31:0/1377021961 shutdown_connections 2023-10-06T05:22:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.676+0000 7f21c57fa700 1 -- 172.21.15.31:0/1377021961 wait complete. 2023-10-06T05:22:20.680 INFO:tasks.ceph.mds.d.smithi062.stderr: -1179> 2023-10-06T05:22:00.512+0000 7f8042ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:22:20.680 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:22:20.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:22:20.722 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:22:20.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.892+0000 7fec52dff700 1 Processor -- start 2023-10-06T05:22:20.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.892+0000 7fec52dff700 1 -- start start 2023-10-06T05:22:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.893+0000 7fec52dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec4c159930 0x7fec4c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.893+0000 7fec52dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec4c0cd580 0x7fec4c0b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.893+0000 7fec52dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4c15d2d0 con 0x7fec4c159560 2023-10-06T05:22:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.893+0000 7fec52dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4c15d5b0 con 0x7fec4c0cd580 2023-10-06T05:22:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.893+0000 7fec52dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c15d890 con 0x7fec4c159930 2023-10-06T05:22:20.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.894+0000 7fec50b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fec4c159560 0x7fec4c0cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53814/0 (socket says 172.21.15.31:53814) 2023-10-06T05:22:20.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.894+0000 7fec50b9b700 1 -- 172.21.15.31:0/634398483 learned_addr learned my addr 172.21.15.31:0/634398483 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:20.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.894+0000 7fec5139c700 1 --2- 172.21.15.31:0/634398483 >> v2:172.21.15.62:3300/0 conn(0x7fec4c0cd580 0x7fec4c0b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.894+0000 7fec4bfff700 1 --2- 172.21.15.31:0/634398483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec4c159930 0x7fec4c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.894+0000 7fec4b7fe700 1 -- 172.21.15.31:0/634398483 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064465589 0 0) 0x7fec4c15d2d0 con 0x7fec4c159560 2023-10-06T05:22:20.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.894+0000 7fec4b7fe700 1 -- 172.21.15.31:0/634398483 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec3c003570 con 0x7fec4c159560 2023-10-06T05:22:20.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.894+0000 7fec5139c700 1 -- 172.21.15.31:0/634398483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec4c159930 msgr2=0x7fec4c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:20.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.894+0000 7fec5139c700 1 --2- 172.21.15.31:0/634398483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec4c159930 0x7fec4c0b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.894+0000 7fec5139c700 1 -- 172.21.15.31:0/634398483 >> v1:172.21.15.31:6789/0 conn(0x7fec4c159560 legacy=0x7fec4c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.894+0000 7fec5139c700 1 -- 172.21.15.31:0/634398483 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c15db70 con 0x7fec4c0cd580 2023-10-06T05:22:20.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.895+0000 7fec5139c700 1 --2- 172.21.15.31:0/634398483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec4c0cd580 0x7fec4c0b29b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fec44009e60 tx=0x7fec44009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f4b7c3bcc8c0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.895+0000 7fec4b7fe700 1 -- 172.21.15.31:0/634398483 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4400e070 con 0x7fec4c0cd580 2023-10-06T05:22:20.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.895+0000 7fec4b7fe700 1 -- 172.21.15.31:0/634398483 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4400bab0 con 0x7fec4c0cd580 2023-10-06T05:22:20.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.895+0000 7fec4b7fe700 1 -- 172.21.15.31:0/634398483 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4400be70 con 0x7fec4c0cd580 2023-10-06T05:22:20.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.895+0000 7fec52dff700 1 -- 172.21.15.31:0/634398483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec4c0cd580 msgr2=0x7fec4c0b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.895+0000 7fec52dff700 1 --2- 172.21.15.31:0/634398483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec4c0cd580 0x7fec4c0b29b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fec44009e60 tx=0x7fec44009780 comp rx=0 tx=0).stop 2023-10-06T05:22:20.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.896+0000 7fec52dff700 1 -- 172.21.15.31:0/634398483 shutdown_connections 2023-10-06T05:22:20.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.896+0000 7fec52dff700 1 --2- 172.21.15.31:0/634398483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec4c0cd580 0x7fec4c0b29b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.896+0000 7fec52dff700 1 --2- 172.21.15.31:0/634398483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec4c159930 0x7fec4c0b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.896+0000 7fec52dff700 1 -- 172.21.15.31:0/634398483 >> 172.21.15.31:0/634398483 conn(0x7fec4c06ec90 msgr2=0x7fec4c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:20.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.896+0000 7fec52dff700 1 -- 172.21.15.31:0/634398483 shutdown_connections 2023-10-06T05:22:20.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.896+0000 7fec52dff700 1 -- 172.21.15.31:0/634398483 wait complete. 2023-10-06T05:22:20.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.897+0000 7fec52dff700 1 Processor -- start 2023-10-06T05:22:20.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.897+0000 7fec52dff700 1 -- start start 2023-10-06T05:22:20.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.898+0000 7fec52dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0cd580 0x7fec4c155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.898+0000 7fec52dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec4c159560 0x7fec4c0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.898+0000 7fec52dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec4c159930 0x7fec4c0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.898+0000 7fec52dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c155d80 con 0x7fec4c0cd580 2023-10-06T05:22:20.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.898+0000 7fec52dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c0c9820 con 0x7fec4c159560 2023-10-06T05:22:20.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.898+0000 7fec52dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c0c9ac0 con 0x7fec4c159930 2023-10-06T05:22:20.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.898+0000 7fec50b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0cd580 0x7fec4c155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec50b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0cd580 0x7fec4c155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46106/0 (socket says 172.21.15.31:46106) 2023-10-06T05:22:20.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec50b9b700 1 -- 172.21.15.31:0/1365895526 learned_addr learned my addr 172.21.15.31:0/1365895526 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:20.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec50b9b700 1 -- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec4c159930 msgr2=0x7fec4c0c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec50b9b700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec4c159930 0x7fec4c0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec50b9b700 1 -- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec4c159560 msgr2=0x7fec4c0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:20.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec50b9b700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec4c159560 0x7fec4c0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:20.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec50b9b700 1 -- 172.21.15.31:0/1365895526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec44009a20 con 0x7fec4c0cd580 2023-10-06T05:22:20.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec50b9b700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0cd580 0x7fec4c155840 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fec4000c7c0 tx=0x7fec40007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec48ff9700 1 -- 172.21.15.31:0/1365895526 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40005b30 con 0x7fec4c0cd580 2023-10-06T05:22:20.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec48ff9700 1 -- 172.21.15.31:0/1365895526 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec40005c90 con 0x7fec4c0cd580 2023-10-06T05:22:20.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec52dff700 1 -- 172.21.15.31:0/1365895526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c0c9da0 con 0x7fec4c0cd580 2023-10-06T05:22:20.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.899+0000 7fec48ff9700 1 -- 172.21.15.31:0/1365895526 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40007340 con 0x7fec4c0cd580 2023-10-06T05:22:20.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.900+0000 7fec52dff700 1 -- 172.21.15.31:0/1365895526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c0c4390 con 0x7fec4c0cd580 2023-10-06T05:22:20.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.901+0000 7fec52dff700 1 -- 172.21.15.31:0/1365895526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c04c380 con 0x7fec4c0cd580 2023-10-06T05:22:20.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.901+0000 7fec48ff9700 1 -- 172.21.15.31:0/1365895526 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fec4001b400 con 0x7fec4c0cd580 2023-10-06T05:22:20.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.903+0000 7fec48ff9700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec30067050 0x7fec30069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:20.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.903+0000 7fec48ff9700 1 -- 172.21.15.31:0/1365895526 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fec40084350 con 0x7fec4c0cd580 2023-10-06T05:22:20.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.903+0000 7fec4bfff700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec30067050 0x7fec30069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:20.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.904+0000 7fec4bfff700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec30067050 0x7fec30069500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0d1240 tx=0x7fec380091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:20.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:20.907+0000 7fec48ff9700 1 -- 172.21.15.31:0/1365895526 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec40051320 con 0x7fec4c0cd580 2023-10-06T05:22:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.160+0000 7fec52dff700 1 -- 172.21.15.31:0/1365895526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fec4c0c4ae0 con 0x7fec4c0cd580 2023-10-06T05:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.674+0000 7fec48ff9700 1 -- 172.21.15.31:0/1365895526 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7fec40054940 con 0x7fec4c0cd580 2023-10-06T05:22:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.676+0000 7fec52dff700 1 -- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec30067050 msgr2=0x7fec30069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.676+0000 7fec52dff700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec30067050 0x7fec30069500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0d1240 tx=0x7fec380091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.676+0000 7fec52dff700 1 -- 172.21.15.31:0/1365895526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0cd580 msgr2=0x7fec4c155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.676+0000 7fec52dff700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0cd580 0x7fec4c155840 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fec4000c7c0 tx=0x7fec40007e00 comp rx=0 tx=0).stop 2023-10-06T05:22:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.677+0000 7fec52dff700 1 -- 172.21.15.31:0/1365895526 shutdown_connections 2023-10-06T05:22:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.677+0000 7fec52dff700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec30067050 0x7fec30069500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.677+0000 7fec52dff700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec4c159930 0x7fec4c0c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.677+0000 7fec52dff700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0cd580 0x7fec4c155840 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.677+0000 7fec52dff700 1 --2- 172.21.15.31:0/1365895526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec4c159560 0x7fec4c0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.677+0000 7fec52dff700 1 -- 172.21.15.31:0/1365895526 >> 172.21.15.31:0/1365895526 conn(0x7fec4c06ec90 msgr2=0x7fec4c15ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:21.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.677+0000 7fec52dff700 1 -- 172.21.15.31:0/1365895526 shutdown_connections 2023-10-06T05:22:21.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.677+0000 7fec52dff700 1 -- 172.21.15.31:0/1365895526 wait complete. 2023-10-06T05:22:21.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:22:21.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.898+0000 7fe672fff700 1 Processor -- start 2023-10-06T05:22:21.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.899+0000 7fe672fff700 1 -- start start 2023-10-06T05:22:21.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.900+0000 7fe672fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe66c15d920 0x7fe66c1645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:21.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.900+0000 7fe672fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66c15cfc0 con 0x7fe66c153950 2023-10-06T05:22:21.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.900+0000 7fe672fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66c15bb20 con 0x7fe66c153d20 2023-10-06T05:22:21.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.900+0000 7fe672fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c154170 con 0x7fe66c15d920 2023-10-06T05:22:21.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.900+0000 7fe67159c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe66c15d920 0x7fe66c1645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:21.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.900+0000 7fe670d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe66c153950 0x7fe66c0cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53824/0 (socket says 172.21.15.31:53824) 2023-10-06T05:22:21.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.900+0000 7fe670d9b700 1 -- 172.21.15.31:0/1915053561 learned_addr learned my addr 172.21.15.31:0/1915053561 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:21.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.901+0000 7fe67159c700 1 -- 172.21.15.31:0/1915053561 >> v1:172.21.15.62:6789/0 conn(0x7fe66c153d20 legacy=0x7fe66c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:21.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.901+0000 7fe67159c700 1 -- 172.21.15.31:0/1915053561 >> v1:172.21.15.31:6789/0 conn(0x7fe66c153950 legacy=0x7fe66c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:21.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.901+0000 7fe67159c700 1 -- 172.21.15.31:0/1915053561 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c15af10 con 0x7fe66c15d920 2023-10-06T05:22:21.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.901+0000 7fe66b7fe700 1 -- 172.21.15.31:0/1915053561 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120131326 0 0) 0x7fe66c15cfc0 con 0x7fe66c153950 2023-10-06T05:22:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.901+0000 7fe67159c700 1 --2- 172.21.15.31:0/1915053561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 0x7fe66c1645b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe664009e60 tx=0x7fe664009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759687b984f8e8fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.901+0000 7fe66b7fe700 1 -- 172.21.15.31:0/1915053561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66400e070 con 0x7fe66c15d920 2023-10-06T05:22:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.901+0000 7fe66b7fe700 1 -- 172.21.15.31:0/1915053561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66400bc30 con 0x7fe66c15d920 2023-10-06T05:22:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.901+0000 7fe66b7fe700 1 -- 172.21.15.31:0/1915053561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664012430 con 0x7fe66c15d920 2023-10-06T05:22:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.902+0000 7fe672fff700 1 -- 172.21.15.31:0/1915053561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 msgr2=0x7fe66c1645b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.902+0000 7fe672fff700 1 --2- 172.21.15.31:0/1915053561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 0x7fe66c1645b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe664009e60 tx=0x7fe664009780 comp rx=0 tx=0).stop 2023-10-06T05:22:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.902+0000 7fe672fff700 1 -- 172.21.15.31:0/1915053561 shutdown_connections 2023-10-06T05:22:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.902+0000 7fe672fff700 1 --2- 172.21.15.31:0/1915053561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 0x7fe66c1645b0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.902+0000 7fe672fff700 1 -- 172.21.15.31:0/1915053561 >> 172.21.15.31:0/1915053561 conn(0x7fe66c0681b0 msgr2=0x7fe66c0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.902+0000 7fe672fff700 1 -- 172.21.15.31:0/1915053561 shutdown_connections 2023-10-06T05:22:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.902+0000 7fe672fff700 1 -- 172.21.15.31:0/1915053561 wait complete. 2023-10-06T05:22:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.904+0000 7fe672fff700 1 Processor -- start 2023-10-06T05:22:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.904+0000 7fe672fff700 1 -- start start 2023-10-06T05:22:21.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe672fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe66c153950 0x7fe66c0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe672fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe66c153d20 0x7fe66c0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe672fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 0x7fe66c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe672fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0b5720 con 0x7fe66c15d920 2023-10-06T05:22:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe672fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c169f80 con 0x7fe66c153d20 2023-10-06T05:22:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe672fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c16a200 con 0x7fe66c153950 2023-10-06T05:22:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe67159c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 0x7fe66c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe67159c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 0x7fe66c0c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46132/0 (socket says 172.21.15.31:46132) 2023-10-06T05:22:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe67159c700 1 -- 172.21.15.31:0/958891795 learned_addr learned my addr 172.21.15.31:0/958891795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe670d9b700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe66c153950 0x7fe66c0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe66bfff700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe66c153d20 0x7fe66c0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe67159c700 1 -- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe66c153950 msgr2=0x7fe66c0c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe67159c700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe66c153950 0x7fe66c0c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe67159c700 1 -- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe66c153d20 msgr2=0x7fe66c0c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe67159c700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe66c153d20 0x7fe66c0c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.906+0000 7fe67159c700 1 -- 172.21.15.31:0/958891795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664009a20 con 0x7fe66c15d920 2023-10-06T05:22:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.907+0000 7fe67159c700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 0x7fe66c0c66e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe664013000 tx=0x7fe664019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:21.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.907+0000 7fe668ff9700 1 -- 172.21.15.31:0/958891795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664012e80 con 0x7fe66c15d920 2023-10-06T05:22:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.907+0000 7fe668ff9700 1 -- 172.21.15.31:0/958891795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66401b600 con 0x7fe66c15d920 2023-10-06T05:22:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.907+0000 7fe668ff9700 1 -- 172.21.15.31:0/958891795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66401b9c0 con 0x7fe66c15d920 2023-10-06T05:22:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.907+0000 7fe672fff700 1 -- 172.21.15.31:0/958891795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66c0c9b70 con 0x7fe66c15d920 2023-10-06T05:22:21.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.907+0000 7fe672fff700 1 -- 172.21.15.31:0/958891795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c0c7b30 con 0x7fe66c15d920 2023-10-06T05:22:21.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.908+0000 7fe672fff700 1 -- 172.21.15.31:0/958891795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c04c380 con 0x7fe66c15d920 2023-10-06T05:22:21.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.909+0000 7fe668ff9700 1 -- 172.21.15.31:0/958891795 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe66401bd30 con 0x7fe66c15d920 2023-10-06T05:22:21.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.910+0000 7fe668ff9700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe650067050 0x7fe650069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:21.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.910+0000 7fe668ff9700 1 -- 172.21.15.31:0/958891795 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fe664086c60 con 0x7fe66c15d920 2023-10-06T05:22:21.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.910+0000 7fe670d9b700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe650067050 0x7fe650069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:21.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.911+0000 7fe670d9b700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe650067050 0x7fe650069500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe6600027b0 tx=0x7fe660009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:21.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:21.914+0000 7fe668ff9700 1 -- 172.21.15.31:0/958891795 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe664053a00 con 0x7fe66c15d920 2023-10-06T05:22:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:22.118+0000 7fe672fff700 1 -- 172.21.15.31:0/958891795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe66c15b1f0 con 0x7fe66c15d920 2023-10-06T05:22:23.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.088+0000 7fe668ff9700 1 -- 172.21.15.31:0/958891795 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe664057020 con 0x7fe66c15d920 2023-10-06T05:22:23.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.092+0000 7fe672fff700 1 -- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe650067050 msgr2=0x7fe650069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.092+0000 7fe672fff700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe650067050 0x7fe650069500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe6600027b0 tx=0x7fe660009040 comp rx=0 tx=0).stop 2023-10-06T05:22:23.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.092+0000 7fe672fff700 1 -- 172.21.15.31:0/958891795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 msgr2=0x7fe66c0c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.092+0000 7fe672fff700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 0x7fe66c0c66e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe664013000 tx=0x7fe664019000 comp rx=0 tx=0).stop 2023-10-06T05:22:23.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.093+0000 7fe672fff700 1 -- 172.21.15.31:0/958891795 shutdown_connections 2023-10-06T05:22:23.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.093+0000 7fe672fff700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe650067050 0x7fe650069500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.093+0000 7fe672fff700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe66c15d920 0x7fe66c0c66e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.093+0000 7fe672fff700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe66c153950 0x7fe66c0c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.093+0000 7fe672fff700 1 --2- 172.21.15.31:0/958891795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe66c153d20 0x7fe66c0c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.093+0000 7fe672fff700 1 -- 172.21.15.31:0/958891795 >> 172.21.15.31:0/958891795 conn(0x7fe66c0681b0 msgr2=0x7fe66c0d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:23.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.093+0000 7fe672fff700 1 -- 172.21.15.31:0/958891795 shutdown_connections 2023-10-06T05:22:23.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.094+0000 7fe672fff700 1 -- 172.21.15.31:0/958891795 wait complete. 2023-10-06T05:22:23.098 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:22:23.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:22:23.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.320+0000 7f3c09fff700 1 Processor -- start 2023-10-06T05:22:23.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.320+0000 7f3c09fff700 1 -- start start 2023-10-06T05:22:23.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.321+0000 7f3c09fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3c040ad670 0x7f3c0406ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c09fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c0406a630 0x7f3c0406aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c09fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0406e600 con 0x7f3c04156300 2023-10-06T05:22:23.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c09fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c040adae0 con 0x7f3c0406a630 2023-10-06T05:22:23.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c09fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c0406aff0 con 0x7f3c040ad670 2023-10-06T05:22:23.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c03fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c0406a630 0x7f3c0406aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c03fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c0406a630 0x7f3c0406aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46138/0 (socket says 172.21.15.31:46138) 2023-10-06T05:22:23.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c03fff700 1 -- 172.21.15.31:0/223065530 learned_addr learned my addr 172.21.15.31:0/223065530 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:23.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c02ffd700 1 --2- 172.21.15.31:0/223065530 >> v2:172.21.15.62:3300/0 conn(0x7f3c040ad670 0x7f3c0406ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c03fff700 1 -- 172.21.15.31:0/223065530 >> v2:172.21.15.62:3300/0 conn(0x7f3c040ad670 msgr2=0x7f3c0406ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:23.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c03fff700 1 --2- 172.21.15.31:0/223065530 >> v2:172.21.15.62:3300/0 conn(0x7f3c040ad670 0x7f3c0406ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c03fff700 1 -- 172.21.15.31:0/223065530 >> v1:172.21.15.62:6789/0 conn(0x7f3c04156300 legacy=0x7f3c0406d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.322+0000 7f3c03fff700 1 -- 172.21.15.31:0/223065530 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0406d9f0 con 0x7f3c0406a630 2023-10-06T05:22:23.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.323+0000 7f3c03fff700 1 --2- 172.21.15.31:0/223065530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c0406a630 0x7f3c0406aab0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3c040b2160 tx=0x7f3bfc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a085fe42fa5f2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:23.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.323+0000 7f3c027fc700 1 -- 172.21.15.31:0/223065530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00e070 con 0x7f3c0406a630 2023-10-06T05:22:23.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.323+0000 7f3c027fc700 1 -- 172.21.15.31:0/223065530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bfc00ba20 con 0x7f3c0406a630 2023-10-06T05:22:23.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.323+0000 7f3c027fc700 1 -- 172.21.15.31:0/223065530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc0092e0 con 0x7f3c0406a630 2023-10-06T05:22:23.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.323+0000 7f3c09fff700 1 -- 172.21.15.31:0/223065530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c0406a630 msgr2=0x7f3c0406aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.323+0000 7f3c09fff700 1 --2- 172.21.15.31:0/223065530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c0406a630 0x7f3c0406aab0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3c040b2160 tx=0x7f3bfc009840 comp rx=0 tx=0).stop 2023-10-06T05:22:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.323+0000 7f3c09fff700 1 -- 172.21.15.31:0/223065530 shutdown_connections 2023-10-06T05:22:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.323+0000 7f3c09fff700 1 --2- 172.21.15.31:0/223065530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c0406a630 0x7f3c0406aab0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.323+0000 7f3c09fff700 1 --2- 172.21.15.31:0/223065530 >> v2:172.21.15.62:3300/0 conn(0x7f3c040ad670 0x7f3c0406ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.323+0000 7f3c09fff700 1 -- 172.21.15.31:0/223065530 >> 172.21.15.31:0/223065530 conn(0x7f3c04068030 msgr2=0x7f3c040cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.324+0000 7f3c09fff700 1 -- 172.21.15.31:0/223065530 shutdown_connections 2023-10-06T05:22:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.324+0000 7f3c09fff700 1 -- 172.21.15.31:0/223065530 wait complete. 2023-10-06T05:22:23.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.325+0000 7f3c09fff700 1 Processor -- start 2023-10-06T05:22:23.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.325+0000 7f3c09fff700 1 -- start start 2023-10-06T05:22:23.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.326+0000 7f3c09fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c0406a630 0x7f3c040c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.326+0000 7f3c09fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c040ad670 0x7f3c040c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.326+0000 7f3c09fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c04156300 0x7f3c041549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.326+0000 7f3c09fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c040c76b0 con 0x7f3c04156300 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.326+0000 7f3c09fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c040c6aa0 con 0x7f3c040ad670 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.326+0000 7f3c09fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c040c6d80 con 0x7f3c0406a630 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c04156300 0x7f3c041549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c04156300 0x7f3c041549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46150/0 (socket says 172.21.15.31:46150) 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c03fff700 1 -- 172.21.15.31:0/332957050 learned_addr learned my addr 172.21.15.31:0/332957050 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c02ffd700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c040ad670 0x7f3c040c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c037fe700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c0406a630 0x7f3c040c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c03fff700 1 -- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c0406a630 msgr2=0x7f3c040c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c03fff700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c0406a630 0x7f3c040c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c03fff700 1 -- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c040ad670 msgr2=0x7f3c040c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:23.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c03fff700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c040ad670 0x7f3c040c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:23.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c03fff700 1 -- 172.21.15.31:0/332957050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc009ae0 con 0x7f3c04156300 2023-10-06T05:22:23.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.327+0000 7f3c03fff700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c04156300 0x7f3c041549d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009880 tx=0x7f3bfc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:23.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.328+0000 7f3beb7fe700 1 -- 172.21.15.31:0/332957050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc008b30 con 0x7f3c04156300 2023-10-06T05:22:23.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.328+0000 7f3beb7fe700 1 -- 172.21.15.31:0/332957050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bfc008cb0 con 0x7f3c04156300 2023-10-06T05:22:23.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.328+0000 7f3beb7fe700 1 -- 172.21.15.31:0/332957050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc0266b0 con 0x7f3c04156300 2023-10-06T05:22:23.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.328+0000 7f3c09fff700 1 -- 172.21.15.31:0/332957050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c040c7060 con 0x7f3c04156300 2023-10-06T05:22:23.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.328+0000 7f3c09fff700 1 -- 172.21.15.31:0/332957050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c040c5a30 con 0x7f3c04156300 2023-10-06T05:22:23.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.329+0000 7f3c09fff700 1 -- 172.21.15.31:0/332957050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0404c380 con 0x7f3c04156300 2023-10-06T05:22:23.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.330+0000 7f3beb7fe700 1 -- 172.21.15.31:0/332957050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f3bfc003040 con 0x7f3c04156300 2023-10-06T05:22:23.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.331+0000 7f3beb7fe700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bf0067050 0x7f3bf0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:23.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.331+0000 7f3beb7fe700 1 -- 172.21.15.31:0/332957050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f3bfc086190 con 0x7f3c04156300 2023-10-06T05:22:23.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.331+0000 7f3c037fe700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bf0067050 0x7f3bf0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:23.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.332+0000 7f3c037fe700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bf0067050 0x7f3bf0069500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40037f0 tx=0x7f3bf4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:23.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.335+0000 7f3beb7fe700 1 -- 172.21.15.31:0/332957050 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bfc053110 con 0x7f3c04156300 2023-10-06T05:22:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:23.539+0000 7f3c09fff700 1 -- 172.21.15.31:0/332957050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3c040cab30 con 0x7f3c04156300 2023-10-06T05:22:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.093+0000 7f3beb7fe700 1 -- 172.21.15.31:0/332957050 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3bfc056730 con 0x7f3c04156300 2023-10-06T05:22:24.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.100+0000 7f3c09fff700 1 -- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bf0067050 msgr2=0x7f3bf0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.100+0000 7f3c09fff700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bf0067050 0x7f3bf0069500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40037f0 tx=0x7f3bf4009040 comp rx=0 tx=0).stop 2023-10-06T05:22:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.100+0000 7f3c09fff700 1 -- 172.21.15.31:0/332957050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c04156300 msgr2=0x7f3c041549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.100+0000 7f3c09fff700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c04156300 0x7f3c041549d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009880 tx=0x7f3bfc018000 comp rx=0 tx=0).stop 2023-10-06T05:22:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.100+0000 7f3c09fff700 1 -- 172.21.15.31:0/332957050 shutdown_connections 2023-10-06T05:22:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.100+0000 7f3c09fff700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bf0067050 0x7f3bf0069500 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.100+0000 7f3c09fff700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c04156300 0x7f3c041549d0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.100+0000 7f3c09fff700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c0406a630 0x7f3c040c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.100+0000 7f3c09fff700 1 --2- 172.21.15.31:0/332957050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c040ad670 0x7f3c040c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.100+0000 7f3c09fff700 1 -- 172.21.15.31:0/332957050 >> 172.21.15.31:0/332957050 conn(0x7f3c04068030 msgr2=0x7f3c041554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.103+0000 7f3c09fff700 1 -- 172.21.15.31:0/332957050 shutdown_connections 2023-10-06T05:22:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.103+0000 7f3c09fff700 1 -- 172.21.15.31:0/332957050 wait complete. 2023-10-06T05:22:24.109 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:22:24.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:22:24.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.328+0000 7f9867dff700 1 Processor -- start 2023-10-06T05:22:24.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.329+0000 7f9867dff700 1 -- start start 2023-10-06T05:22:24.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.330+0000 7f9867dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600aef00 0x7f98600c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.330+0000 7f9867dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f98600c7560 0x7f98600686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.330+0000 7f9867dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9860068250 con 0x7f98600c7190 2023-10-06T05:22:24.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.330+0000 7f9867dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986006ac80 con 0x7f98600c7560 2023-10-06T05:22:24.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.330+0000 7f9867dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9860067f20 con 0x7f98600aef00 2023-10-06T05:22:24.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.330+0000 7f986539a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600aef00 0x7f98600c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.330+0000 7f986639c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f98600c7560 0x7f98600686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.330+0000 7f986539a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600aef00 0x7f98600c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:34556/0 (socket says 172.21.15.31:34556) 2023-10-06T05:22:24.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.330+0000 7f986539a700 1 -- 172.21.15.31:0/145048136 learned_addr learned my addr 172.21.15.31:0/145048136 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:24.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.330+0000 7f9865b9b700 1 --1- 172.21.15.31:0/145048136 >> v1:172.21.15.62:6789/0 conn(0x7f98600c7190 0x7f98600af7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:55724/0 (socket says 172.21.15.31:55724) 2023-10-06T05:22:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.331+0000 7f9864b99700 1 -- 172.21.15.31:0/145048136 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217833538 0 0) 0x7f9860068250 con 0x7f98600c7190 2023-10-06T05:22:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.331+0000 7f986539a700 1 -- 172.21.15.31:0/145048136 >> v2:172.21.15.62:3300/0 conn(0x7f98600c7560 msgr2=0x7f98600686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.331+0000 7f986539a700 1 --2- 172.21.15.31:0/145048136 >> v2:172.21.15.62:3300/0 conn(0x7f98600c7560 0x7f98600686e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.331+0000 7f986539a700 1 -- 172.21.15.31:0/145048136 >> v1:172.21.15.62:6789/0 conn(0x7f98600c7190 legacy=0x7f98600af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.331+0000 7f986539a700 1 -- 172.21.15.31:0/145048136 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986006aed0 con 0x7f98600aef00 2023-10-06T05:22:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.331+0000 7f986539a700 1 --2- 172.21.15.31:0/145048136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600aef00 0x7f98600c9d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f98500099b0 tx=0x7f9850009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=219bbc5ae867676e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.331+0000 7f9864b99700 1 -- 172.21.15.31:0/145048136 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850009ea0 con 0x7f98600aef00 2023-10-06T05:22:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.331+0000 7f9864b99700 1 -- 172.21.15.31:0/145048136 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985000b860 con 0x7f98600aef00 2023-10-06T05:22:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.331+0000 7f9864b99700 1 -- 172.21.15.31:0/145048136 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000bc00 con 0x7f98600aef00 2023-10-06T05:22:24.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.332+0000 7f9867dff700 1 -- 172.21.15.31:0/145048136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600aef00 msgr2=0x7f98600c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.332+0000 7f9867dff700 1 --2- 172.21.15.31:0/145048136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600aef00 0x7f98600c9d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f98500099b0 tx=0x7f9850009250 comp rx=0 tx=0).stop 2023-10-06T05:22:24.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.332+0000 7f9867dff700 1 -- 172.21.15.31:0/145048136 shutdown_connections 2023-10-06T05:22:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.332+0000 7f9867dff700 1 --2- 172.21.15.31:0/145048136 >> v2:172.21.15.62:3300/0 conn(0x7f98600c7560 0x7f98600686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.332+0000 7f9867dff700 1 --2- 172.21.15.31:0/145048136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600aef00 0x7f98600c9d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.332+0000 7f9867dff700 1 -- 172.21.15.31:0/145048136 >> 172.21.15.31:0/145048136 conn(0x7f98600cd3d0 msgr2=0x7f98600c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.332+0000 7f9867dff700 1 -- 172.21.15.31:0/145048136 shutdown_connections 2023-10-06T05:22:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.332+0000 7f9867dff700 1 -- 172.21.15.31:0/145048136 wait complete. 2023-10-06T05:22:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.334+0000 7f9867dff700 1 Processor -- start 2023-10-06T05:22:24.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.334+0000 7f9867dff700 1 -- start start 2023-10-06T05:22:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f9867dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98600aef00 0x7f98600c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f9867dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600c7190 0x7f98600c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f9867dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98600c7560 0x7f9860149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f9867dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98600c4910 con 0x7f98600c7560 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f9867dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986015a210 con 0x7f98600aef00 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f9867dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98600c4070 con 0x7f98600c7190 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f986639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98600c7560 0x7f9860149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f986639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98600c7560 0x7f9860149ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46164/0 (socket says 172.21.15.31:46164) 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f9865b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98600aef00 0x7f98600c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f986639c700 1 -- 172.21.15.31:0/2056418512 learned_addr learned my addr 172.21.15.31:0/2056418512 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f986539a700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600c7190 0x7f98600c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f986639c700 1 -- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600c7190 msgr2=0x7f98600c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f986639c700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600c7190 0x7f98600c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f986639c700 1 -- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98600aef00 msgr2=0x7f98600c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f986639c700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98600aef00 0x7f98600c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.335+0000 7f986639c700 1 -- 172.21.15.31:0/2056418512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98500094f0 con 0x7f98600c7560 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.336+0000 7f986639c700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98600c7560 0x7f9860149ce0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9858009e90 tx=0x7f985800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.336+0000 7f9855ffb700 1 -- 172.21.15.31:0/2056418512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985800cb70 con 0x7f98600c7560 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.336+0000 7f9855ffb700 1 -- 172.21.15.31:0/2056418512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985800f040 con 0x7f98600c7560 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.336+0000 7f9867dff700 1 -- 172.21.15.31:0/2056418512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98600c4350 con 0x7f98600c7560 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.336+0000 7f9855ffb700 1 -- 172.21.15.31:0/2056418512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858014480 con 0x7f98600c7560 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.336+0000 7f9867dff700 1 -- 172.21.15.31:0/2056418512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986014a220 con 0x7f98600c7560 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.337+0000 7f9867dff700 1 -- 172.21.15.31:0/2056418512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986015a380 con 0x7f98600c7560 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.338+0000 7f9855ffb700 1 -- 172.21.15.31:0/2056418512 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f9858014830 con 0x7f98600c7560 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.339+0000 7f9855ffb700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9844067050 0x7f9844069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.339+0000 7f9855ffb700 1 -- 172.21.15.31:0/2056418512 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f98580850e0 con 0x7f98600c7560 2023-10-06T05:22:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.339+0000 7f9865b9b700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9844067050 0x7f9844069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.340+0000 7f9865b9b700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9844067050 0x7f9844069500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f985c007fd0 tx=0x7f985c007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.343+0000 7f9855ffb700 1 -- 172.21.15.31:0/2056418512 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98580521d0 con 0x7f98600c7560 2023-10-06T05:22:24.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.543+0000 7f9867dff700 1 -- 172.21.15.31:0/2056418512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f986004c380 con 0x7f98600c7560 2023-10-06T05:22:24.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.547+0000 7f9855ffb700 1 -- 172.21.15.31:0/2056418512 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f98580557f0 con 0x7f98600c7560 2023-10-06T05:22:24.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.549+0000 7f9867dff700 1 -- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9844067050 msgr2=0x7f9844069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.549+0000 7f9867dff700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9844067050 0x7f9844069500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f985c007fd0 tx=0x7f985c007d50 comp rx=0 tx=0).stop 2023-10-06T05:22:24.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.549+0000 7f9867dff700 1 -- 172.21.15.31:0/2056418512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98600c7560 msgr2=0x7f9860149ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.549+0000 7f9867dff700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98600c7560 0x7f9860149ce0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9858009e90 tx=0x7f985800b040 comp rx=0 tx=0).stop 2023-10-06T05:22:24.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.550+0000 7f9867dff700 1 -- 172.21.15.31:0/2056418512 shutdown_connections 2023-10-06T05:22:24.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.550+0000 7f9867dff700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9844067050 0x7f9844069500 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.550+0000 7f9867dff700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98600c7560 0x7f9860149ce0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.550+0000 7f9867dff700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98600aef00 0x7f98600c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.550+0000 7f9867dff700 1 --2- 172.21.15.31:0/2056418512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98600c7190 0x7f98600c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.550+0000 7f9867dff700 1 -- 172.21.15.31:0/2056418512 >> 172.21.15.31:0/2056418512 conn(0x7f98600cd3d0 msgr2=0x7f9860157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:24.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.550+0000 7f9867dff700 1 -- 172.21.15.31:0/2056418512 shutdown_connections 2023-10-06T05:22:24.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.550+0000 7f9867dff700 1 -- 172.21.15.31:0/2056418512 wait complete. 2023-10-06T05:22:24.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2023-10-06T05:22:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.763+0000 7f0bd99ff700 1 Processor -- start 2023-10-06T05:22:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.763+0000 7f0bd99ff700 1 -- start start 2023-10-06T05:22:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.764+0000 7f0bd99ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd40c9040 0x7f0bd4159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.764+0000 7f0bd99ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd40c95b0 con 0x7f0bd41595a0 2023-10-06T05:22:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.764+0000 7f0bd99ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd40c9890 con 0x7f0bd4068d80 2023-10-06T05:22:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.764+0000 7f0bd99ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40c9b70 con 0x7f0bd40c9040 2023-10-06T05:22:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.764+0000 7f0bd2ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0bd41595a0 0x7f0bd4068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53836/0 (socket says 172.21.15.31:53836) 2023-10-06T05:22:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.764+0000 7f0bd2ffd700 1 -- 172.21.15.31:0/201997309 learned_addr learned my addr 172.21.15.31:0/201997309 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.765+0000 7f0bd37fe700 1 --2- 172.21.15.31:0/201997309 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd40c9040 0x7f0bd4159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.765+0000 7f0bd1ffb700 1 -- 172.21.15.31:0/201997309 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551845459 0 0) 0x7f0bd40c95b0 con 0x7f0bd41595a0 2023-10-06T05:22:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.765+0000 7f0bd1ffb700 1 -- 172.21.15.31:0/201997309 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bbc003570 con 0x7f0bd41595a0 2023-10-06T05:22:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.765+0000 7f0bd1ffb700 1 -- 172.21.15.31:0/201997309 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128813613 0 0) 0x7f0bd40c9890 con 0x7f0bd4068d80 2023-10-06T05:22:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.765+0000 7f0bd1ffb700 1 -- 172.21.15.31:0/201997309 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bd40c95b0 con 0x7f0bd4068d80 2023-10-06T05:22:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.765+0000 7f0bd37fe700 1 -- 172.21.15.31:0/201997309 >> v1:172.21.15.62:6789/0 conn(0x7f0bd4068d80 legacy=0x7f0bd40c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.765+0000 7f0bd37fe700 1 -- 172.21.15.31:0/201997309 >> v1:172.21.15.31:6789/0 conn(0x7f0bd41595a0 legacy=0x7f0bd4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.765+0000 7f0bd37fe700 1 -- 172.21.15.31:0/201997309 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd40ca130 con 0x7f0bd40c9040 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.765+0000 7f0bd37fe700 1 --2- 172.21.15.31:0/201997309 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd40c9040 0x7f0bd4159970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009ec0 tx=0x7f0bcc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8014885a07b7c6f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.766+0000 7f0bd1ffb700 1 -- 172.21.15.31:0/201997309 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00e070 con 0x7f0bd40c9040 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.766+0000 7f0bd1ffb700 1 -- 172.21.15.31:0/201997309 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc00bb40 con 0x7f0bd40c9040 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.766+0000 7f0bd1ffb700 1 -- 172.21.15.31:0/201997309 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc012430 con 0x7f0bd40c9040 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.766+0000 7f0bd99ff700 1 -- 172.21.15.31:0/201997309 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd40c9040 msgr2=0x7f0bd4159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.766+0000 7f0bd99ff700 1 --2- 172.21.15.31:0/201997309 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd40c9040 0x7f0bd4159970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009ec0 tx=0x7f0bcc009780 comp rx=0 tx=0).stop 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.766+0000 7f0bd99ff700 1 -- 172.21.15.31:0/201997309 shutdown_connections 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.766+0000 7f0bd99ff700 1 --2- 172.21.15.31:0/201997309 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd40c9040 0x7f0bd4159970 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.766+0000 7f0bd99ff700 1 -- 172.21.15.31:0/201997309 >> 172.21.15.31:0/201997309 conn(0x7f0bd406b620 msgr2=0x7f0bd406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.766+0000 7f0bd99ff700 1 -- 172.21.15.31:0/201997309 shutdown_connections 2023-10-06T05:22:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.766+0000 7f0bd99ff700 1 -- 172.21.15.31:0/201997309 wait complete. 2023-10-06T05:22:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.768+0000 7f0bd99ff700 1 Processor -- start 2023-10-06T05:22:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.768+0000 7f0bd99ff700 1 -- start start 2023-10-06T05:22:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd99ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd4068d80 0x7f0bd40c3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd99ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd40c9040 0x7f0bd40b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd99ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd41595a0 0x7f0bd40b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd99ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40b1320 con 0x7f0bd41595a0 2023-10-06T05:22:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd99ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40c3b50 con 0x7f0bd40c9040 2023-10-06T05:22:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd99ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40c3e30 con 0x7f0bd4068d80 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd41595a0 0x7f0bd40b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd41595a0 0x7f0bd40b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46166/0 (socket says 172.21.15.31:46166) 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd37fe700 1 -- 172.21.15.31:0/2562853087 learned_addr learned my addr 172.21.15.31:0/2562853087 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd2ffd700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd4068d80 0x7f0bd40c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.769+0000 7f0bd27fc700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd40c9040 0x7f0bd40b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bd37fe700 1 -- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd4068d80 msgr2=0x7f0bd40c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bd37fe700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd4068d80 0x7f0bd40c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bd37fe700 1 -- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd40c9040 msgr2=0x7f0bd40b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bd37fe700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd40c9040 0x7f0bd40b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bd37fe700 1 -- 172.21.15.31:0/2562853087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc009a20 con 0x7f0bd41595a0 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bd37fe700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd41595a0 0x7f0bd40b3340 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc013000 tx=0x7f0bcc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bbaffd700 1 -- 172.21.15.31:0/2562853087 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc01b420 con 0x7f0bd41595a0 2023-10-06T05:22:24.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bbaffd700 1 -- 172.21.15.31:0/2562853087 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc01b580 con 0x7f0bd41595a0 2023-10-06T05:22:24.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bbaffd700 1 -- 172.21.15.31:0/2562853087 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc01b940 con 0x7f0bd41595a0 2023-10-06T05:22:24.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bd99ff700 1 -- 172.21.15.31:0/2562853087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd40c4110 con 0x7f0bd41595a0 2023-10-06T05:22:24.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.770+0000 7f0bd99ff700 1 -- 172.21.15.31:0/2562853087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd40c46d0 con 0x7f0bd41595a0 2023-10-06T05:22:24.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.771+0000 7f0bd99ff700 1 -- 172.21.15.31:0/2562853087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd40c4910 con 0x7f0bd41595a0 2023-10-06T05:22:24.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.772+0000 7f0bbaffd700 1 -- 172.21.15.31:0/2562853087 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f0bcc01bd30 con 0x7f0bd41595a0 2023-10-06T05:22:24.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.773+0000 7f0bbaffd700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0bc0067050 0x7f0bc0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:24.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.773+0000 7f0bbaffd700 1 -- 172.21.15.31:0/2562853087 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f0bcc085fe0 con 0x7f0bd41595a0 2023-10-06T05:22:24.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.774+0000 7f0bd2ffd700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0bc0067050 0x7f0bc0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:24.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.774+0000 7f0bd2ffd700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0bc0067050 0x7f0bc0069500 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4007fd0 tx=0x7f0bc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:24.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:24.777+0000 7f0bbaffd700 1 -- 172.21.15.31:0/2562853087 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bd40c4910 con 0x7f0bd41595a0 2023-10-06T05:22:25.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.065+0000 7f0bd99ff700 1 -- 172.21.15.31:0/2562853087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7f0bd40c4910 con 0x7f0bd41595a0 2023-10-06T05:22:25.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.094+0000 7f0bbaffd700 1 -- 172.21.15.31:0/2562853087 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v295) v1 ==== 146+0+0 (secure 0 0 0) 0x7f0bd40c4910 con 0x7f0bd41595a0 2023-10-06T05:22:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 -- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0bc0067050 msgr2=0x7f0bc0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0bc0067050 0x7f0bc0069500 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4007fd0 tx=0x7f0bc4009040 comp rx=0 tx=0).stop 2023-10-06T05:22:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 -- 172.21.15.31:0/2562853087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd41595a0 msgr2=0x7f0bd40b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd41595a0 0x7f0bd40b3340 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc013000 tx=0x7f0bcc019000 comp rx=0 tx=0).stop 2023-10-06T05:22:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 -- 172.21.15.31:0/2562853087 shutdown_connections 2023-10-06T05:22:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0bc0067050 0x7f0bc0069500 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd41595a0 0x7f0bd40b3340 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bd4068d80 0x7f0bd40c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 --2- 172.21.15.31:0/2562853087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd40c9040 0x7f0bd40b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 -- 172.21.15.31:0/2562853087 >> 172.21.15.31:0/2562853087 conn(0x7f0bd406b620 msgr2=0x7f0bd40ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:25.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 -- 172.21.15.31:0/2562853087 shutdown_connections 2023-10-06T05:22:25.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.097+0000 7f0bd99ff700 1 -- 172.21.15.31:0/2562853087 wait complete. 2023-10-06T05:22:25.116 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2023-10-06T05:22:25.116 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2023-10-06T05:22:25.116 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:22:25.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2023-10-06T05:22:25.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.312+0000 7f3a613ff700 1 Processor -- start 2023-10-06T05:22:25.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.312+0000 7f3a613ff700 1 -- start start 2023-10-06T05:22:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.313+0000 7f3a613ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a5c13a1e0 0x7f3a5c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.313+0000 7f3a613ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a5c13a5b0 0x7f3a5c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.313+0000 7f3a613ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5c0c71b0 con 0x7f3a5c0c6c30 2023-10-06T05:22:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.313+0000 7f3a613ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5c0c7490 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.313+0000 7f3a613ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5c0c7770 con 0x7f3a5c13a5b0 2023-10-06T05:22:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.313+0000 7f3a5affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a5c13a1e0 0x7f3a5c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.313+0000 7f3a5affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a5c13a1e0 0x7f3a5c0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46180/0 (socket says 172.21.15.31:46180) 2023-10-06T05:22:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.313+0000 7f3a5affd700 1 -- 172.21.15.31:0/2523981361 learned_addr learned my addr 172.21.15.31:0/2523981361 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.314+0000 7f3a5a7fc700 1 --2- 172.21.15.31:0/2523981361 >> v2:172.21.15.62:3300/0 conn(0x7f3a5c13a5b0 0x7f3a5c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.314+0000 7f3a5affd700 1 -- 172.21.15.31:0/2523981361 >> v2:172.21.15.62:3300/0 conn(0x7f3a5c13a5b0 msgr2=0x7f3a5c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:25.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.314+0000 7f3a5affd700 1 --2- 172.21.15.31:0/2523981361 >> v2:172.21.15.62:3300/0 conn(0x7f3a5c13a5b0 0x7f3a5c06d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.314+0000 7f3a5affd700 1 -- 172.21.15.31:0/2523981361 >> v1:172.21.15.62:6789/0 conn(0x7f3a5c0c6c30 legacy=0x7f3a5c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:25.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.314+0000 7f3a5affd700 1 -- 172.21.15.31:0/2523981361 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0c7a50 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.314+0000 7f3a5affd700 1 --2- 172.21.15.31:0/2523981361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 0x7f3a5c0c66f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c06d130 tx=0x7f3a4c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6316ea8cdea75fff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:25.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.314+0000 7f3a59ffb700 1 -- 172.21.15.31:0/2523981361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c005c10 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.314+0000 7f3a59ffb700 1 -- 172.21.15.31:0/2523981361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4c005d70 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.315+0000 7f3a59ffb700 1 -- 172.21.15.31:0/2523981361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00b840 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.315+0000 7f3a613ff700 1 -- 172.21.15.31:0/2523981361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 msgr2=0x7f3a5c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.315+0000 7f3a613ff700 1 --2- 172.21.15.31:0/2523981361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 0x7f3a5c0c66f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c06d130 tx=0x7f3a4c009220 comp rx=0 tx=0).stop 2023-10-06T05:22:25.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.315+0000 7f3a613ff700 1 -- 172.21.15.31:0/2523981361 shutdown_connections 2023-10-06T05:22:25.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.315+0000 7f3a613ff700 1 --2- 172.21.15.31:0/2523981361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 0x7f3a5c0c66f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.315+0000 7f3a613ff700 1 --2- 172.21.15.31:0/2523981361 >> v2:172.21.15.62:3300/0 conn(0x7f3a5c13a5b0 0x7f3a5c06d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.315+0000 7f3a613ff700 1 -- 172.21.15.31:0/2523981361 >> 172.21.15.31:0/2523981361 conn(0x7f3a5c0cc0b0 msgr2=0x7f3a5c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.315+0000 7f3a613ff700 1 -- 172.21.15.31:0/2523981361 shutdown_connections 2023-10-06T05:22:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.315+0000 7f3a613ff700 1 -- 172.21.15.31:0/2523981361 wait complete. 2023-10-06T05:22:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.317+0000 7f3a613ff700 1 Processor -- start 2023-10-06T05:22:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.318+0000 7f3a613ff700 1 -- start start 2023-10-06T05:22:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a613ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a5c0c6c30 0x7f3a5c13c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a613ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 0x7f3a5c16daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a613ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c13a5b0 0x7f3a5c139980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a613ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c14e8a0 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a613ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c14f140 con 0x7f3a5c13a5b0 2023-10-06T05:22:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a613ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c16dfe0 con 0x7f3a5c0c6c30 2023-10-06T05:22:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a5a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 0x7f3a5c16daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a5a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 0x7f3a5c16daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46196/0 (socket says 172.21.15.31:46196) 2023-10-06T05:22:25.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a5a7fc700 1 -- 172.21.15.31:0/2966695573 learned_addr learned my addr 172.21.15.31:0/2966695573 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:25.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a5affd700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a5c0c6c30 0x7f3a5c13c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.319+0000 7f3a5b7fe700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c13a5b0 0x7f3a5c139980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a5a7fc700 1 -- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a5c0c6c30 msgr2=0x7f3a5c13c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a5a7fc700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a5c0c6c30 0x7f3a5c13c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a5a7fc700 1 -- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c13a5b0 msgr2=0x7f3a5c139980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:25.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a5a7fc700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c13a5b0 0x7f3a5c139980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:25.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a5a7fc700 1 -- 172.21.15.31:0/2966695573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c009480 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a5a7fc700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 0x7f3a5c16daa0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000d890 tx=0x7f3a50008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:25.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a42ffd700 1 -- 172.21.15.31:0/2966695573 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50012630 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a42ffd700 1 -- 172.21.15.31:0/2966695573 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50012790 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a613ff700 1 -- 172.21.15.31:0/2966695573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c13c980 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a42ffd700 1 -- 172.21.15.31:0/2966695573 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50012b30 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.320+0000 7f3a613ff700 1 -- 172.21.15.31:0/2966695573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c14edd0 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.321+0000 7f3a613ff700 1 -- 172.21.15.31:0/2966695573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c04c380 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.322+0000 7f3a42ffd700 1 -- 172.21.15.31:0/2966695573 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f3a5001d400 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.323+0000 7f3a42ffd700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a48067050 0x7f3a48069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.323+0000 7f3a42ffd700 1 -- 172.21.15.31:0/2966695573 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f3a50084890 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.324+0000 7f3a5affd700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a48067050 0x7f3a48069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.324+0000 7f3a5affd700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a48067050 0x7f3a48069500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0030c0 tx=0x7f3a4c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:25.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.327+0000 7f3a42ffd700 1 -- 172.21.15.31:0/2966695573 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a50051a90 con 0x7f3a5c13a1e0 2023-10-06T05:22:25.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:25.615+0000 7f3a613ff700 1 -- 172.21.15.31:0/2966695573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7f3a5c0c7d30 con 0x7f3a5c13a1e0 2023-10-06T05:22:26.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.095+0000 7f3a42ffd700 1 -- 172.21.15.31:0/2966695573 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v296) v1 ==== 152+0+0 (secure 0 0 0) 0x7f3a500550b0 con 0x7f3a5c13a1e0 2023-10-06T05:22:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 -- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a48067050 msgr2=0x7f3a48069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a48067050 0x7f3a48069500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0030c0 tx=0x7f3a4c015000 comp rx=0 tx=0).stop 2023-10-06T05:22:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 -- 172.21.15.31:0/2966695573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 msgr2=0x7f3a5c16daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 0x7f3a5c16daa0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000d890 tx=0x7f3a50008040 comp rx=0 tx=0).stop 2023-10-06T05:22:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 -- 172.21.15.31:0/2966695573 shutdown_connections 2023-10-06T05:22:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3a48067050 0x7f3a48069500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c13a5b0 0x7f3a5c139980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a5c0c6c30 0x7f3a5c13c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 --2- 172.21.15.31:0/2966695573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a5c13a1e0 0x7f3a5c16daa0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 -- 172.21.15.31:0/2966695573 >> 172.21.15.31:0/2966695573 conn(0x7f3a5c0cc0b0 msgr2=0x7f3a5c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 -- 172.21.15.31:0/2966695573 shutdown_connections 2023-10-06T05:22:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.098+0000 7f3a613ff700 1 -- 172.21.15.31:0/2966695573 wait complete. 2023-10-06T05:22:26.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:22:26.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.316+0000 7f55ec5ff700 1 Processor -- start 2023-10-06T05:22:26.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.316+0000 7f55ec5ff700 1 -- start start 2023-10-06T05:22:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ec5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55e41573a0 0x7f55e40acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ec5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f55e4157770 0x7f55e40b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ec5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e4157c10 0x7f55e40b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ec5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e4158cf0 con 0x7f55e41573a0 2023-10-06T05:22:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ec5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e41580e0 con 0x7f55e4157770 2023-10-06T05:22:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ec5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e41583c0 con 0x7f55e4157c10 2023-10-06T05:22:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ea39b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55e41573a0 0x7f55e40acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ea39b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55e41573a0 0x7f55e40acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46210/0 (socket says 172.21.15.31:46210) 2023-10-06T05:22:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ea39b700 1 -- 172.21.15.31:0/4141692408 learned_addr learned my addr 172.21.15.31:0/4141692408 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55e9b9a700 1 --2- 172.21.15.31:0/4141692408 >> v2:172.21.15.62:3300/0 conn(0x7f55e4157770 0x7f55e40b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55eab9c700 1 --2- 172.21.15.31:0/4141692408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e4157c10 0x7f55e40b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ea39b700 1 -- 172.21.15.31:0/4141692408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e4157c10 msgr2=0x7f55e40b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ea39b700 1 --2- 172.21.15.31:0/4141692408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e4157c10 0x7f55e40b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ea39b700 1 -- 172.21.15.31:0/4141692408 >> v2:172.21.15.62:3300/0 conn(0x7f55e4157770 msgr2=0x7f55e40b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ea39b700 1 --2- 172.21.15.31:0/4141692408 >> v2:172.21.15.62:3300/0 conn(0x7f55e4157770 0x7f55e40b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55ea39b700 1 -- 172.21.15.31:0/4141692408 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e41586a0 con 0x7f55e41573a0 2023-10-06T05:22:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.318+0000 7f55e9b9a700 1 --2- 172.21.15.31:0/4141692408 >> v2:172.21.15.62:3300/0 conn(0x7f55e4157770 0x7f55e40b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.319+0000 7f55ea39b700 1 --2- 172.21.15.31:0/4141692408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e41573a0 0x7f55e40acb70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f55e406b3a0 tx=0x7f55e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dea5f0ed465f0bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.319+0000 7f55e9399700 1 -- 172.21.15.31:0/4141692408 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e000b860 con 0x7f55e41573a0 2023-10-06T05:22:26.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.319+0000 7f55e9399700 1 -- 172.21.15.31:0/4141692408 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e000b9c0 con 0x7f55e41573a0 2023-10-06T05:22:26.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.319+0000 7f55e9399700 1 -- 172.21.15.31:0/4141692408 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e000bd60 con 0x7f55e41573a0 2023-10-06T05:22:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.319+0000 7f55ec5ff700 1 -- 172.21.15.31:0/4141692408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e41573a0 msgr2=0x7f55e40acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.319+0000 7f55ec5ff700 1 --2- 172.21.15.31:0/4141692408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e41573a0 0x7f55e40acb70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f55e406b3a0 tx=0x7f55e0009250 comp rx=0 tx=0).stop 2023-10-06T05:22:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.320+0000 7f55ec5ff700 1 -- 172.21.15.31:0/4141692408 shutdown_connections 2023-10-06T05:22:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.320+0000 7f55ec5ff700 1 --2- 172.21.15.31:0/4141692408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e4157c10 0x7f55e40b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.320+0000 7f55ec5ff700 1 --2- 172.21.15.31:0/4141692408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e41573a0 0x7f55e40acb70 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.320+0000 7f55ec5ff700 1 --2- 172.21.15.31:0/4141692408 >> v2:172.21.15.62:3300/0 conn(0x7f55e4157770 0x7f55e40b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.320+0000 7f55ec5ff700 1 -- 172.21.15.31:0/4141692408 >> 172.21.15.31:0/4141692408 conn(0x7f55e4067560 msgr2=0x7f55e4067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:26.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.320+0000 7f55ec5ff700 1 -- 172.21.15.31:0/4141692408 shutdown_connections 2023-10-06T05:22:26.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.320+0000 7f55ec5ff700 1 -- 172.21.15.31:0/4141692408 wait complete. 2023-10-06T05:22:26.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.322+0000 7f55ec5ff700 1 Processor -- start 2023-10-06T05:22:26.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.322+0000 7f55ec5ff700 1 -- start start 2023-10-06T05:22:26.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.323+0000 7f55ec5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e41573a0 0x7f55e414c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.323+0000 7f55ec5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e4157770 0x7f55e4154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.323+0000 7f55ec5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e4157c10 0x7f55e4154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.323+0000 7f55ec5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e414cd10 con 0x7f55e4157c10 2023-10-06T05:22:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.323+0000 7f55ec5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e40a9d10 con 0x7f55e4157770 2023-10-06T05:22:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.323+0000 7f55ec5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e40a9ff0 con 0x7f55e41573a0 2023-10-06T05:22:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55eab9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e4157c10 0x7f55e4154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55eab9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e4157c10 0x7f55e4154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46212/0 (socket says 172.21.15.31:46212) 2023-10-06T05:22:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55eab9c700 1 -- 172.21.15.31:0/2914439670 learned_addr learned my addr 172.21.15.31:0/2914439670 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55ea39b700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e41573a0 0x7f55e414c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55e9b9a700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e4157770 0x7f55e4154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55eab9c700 1 -- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e41573a0 msgr2=0x7f55e414c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55eab9c700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e41573a0 0x7f55e414c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55eab9c700 1 -- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e4157770 msgr2=0x7f55e4154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55eab9c700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e4157770 0x7f55e4154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55eab9c700 1 -- 172.21.15.31:0/2914439670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00094f0 con 0x7f55e4157c10 2023-10-06T05:22:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.324+0000 7f55eab9c700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e4157c10 0x7f55e4154d40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f55dc007f20 tx=0x7f55dc009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.325+0000 7f55da7fc700 1 -- 172.21.15.31:0/2914439670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc00eb90 con 0x7f55e4157c10 2023-10-06T05:22:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.325+0000 7f55da7fc700 1 -- 172.21.15.31:0/2914439670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc00ed10 con 0x7f55e4157c10 2023-10-06T05:22:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.325+0000 7f55da7fc700 1 -- 172.21.15.31:0/2914439670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc014670 con 0x7f55e4157c10 2023-10-06T05:22:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.325+0000 7f55ec5ff700 1 -- 172.21.15.31:0/2914439670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e40aa2d0 con 0x7f55e4157c10 2023-10-06T05:22:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.325+0000 7f55ec5ff700 1 -- 172.21.15.31:0/2914439670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e414ca60 con 0x7f55e4157c10 2023-10-06T05:22:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.326+0000 7f55ec5ff700 1 -- 172.21.15.31:0/2914439670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e404c380 con 0x7f55e4157c10 2023-10-06T05:22:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.327+0000 7f55da7fc700 1 -- 172.21.15.31:0/2914439670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f55dc014a20 con 0x7f55e4157c10 2023-10-06T05:22:26.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.328+0000 7f55da7fc700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55c8067050 0x7f55c8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.328+0000 7f55da7fc700 1 -- 172.21.15.31:0/2914439670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f55dc084b60 con 0x7f55e4157c10 2023-10-06T05:22:26.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.328+0000 7f55ea39b700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55c8067050 0x7f55c8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.329+0000 7f55ea39b700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55c8067050 0x7f55c8069500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f55e406b3a0 tx=0x7f55e0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:26.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.332+0000 7f55da7fc700 1 -- 172.21.15.31:0/2914439670 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55dc051d60 con 0x7f55e4157c10 2023-10-06T05:22:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.537+0000 7f55ec5ff700 1 -- 172.21.15.31:0/2914439670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f55e40b1fb0 con 0x7f55e4157c10 2023-10-06T05:22:26.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.541+0000 7f55da7fc700 1 -- 172.21.15.31:0/2914439670 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f55dc055380 con 0x7f55e4157c10 2023-10-06T05:22:26.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 -- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55c8067050 msgr2=0x7f55c8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55c8067050 0x7f55c8069500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f55e406b3a0 tx=0x7f55e0017000 comp rx=0 tx=0).stop 2023-10-06T05:22:26.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 -- 172.21.15.31:0/2914439670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e4157c10 msgr2=0x7f55e4154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e4157c10 0x7f55e4154d40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f55dc007f20 tx=0x7f55dc009800 comp rx=0 tx=0).stop 2023-10-06T05:22:26.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 -- 172.21.15.31:0/2914439670 shutdown_connections 2023-10-06T05:22:26.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55c8067050 0x7f55c8069500 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e4157c10 0x7f55e4154d40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e41573a0 0x7f55e414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 --2- 172.21.15.31:0/2914439670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e4157770 0x7f55e4154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 -- 172.21.15.31:0/2914439670 >> 172.21.15.31:0/2914439670 conn(0x7f55e4067560 msgr2=0x7f55e406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:26.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 -- 172.21.15.31:0/2914439670 shutdown_connections 2023-10-06T05:22:26.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.544+0000 7f55ec5ff700 1 -- 172.21.15.31:0/2914439670 wait complete. 2023-10-06T05:22:26.563 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:22:26.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:26.661 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:22:26.772 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:22:26.772 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:22:26.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:26.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.966+0000 7fe7e9fff700 1 Processor -- start 2023-10-06T05:22:26.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.966+0000 7fe7e9fff700 1 -- start start 2023-10-06T05:22:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.967+0000 7fe7e9fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e40cd510 0x7fe7e40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.967+0000 7fe7e9fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e40c3f40 con 0x7fe7e4159500 2023-10-06T05:22:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.967+0000 7fe7e9fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e40cb000 con 0x7fe7e415d2d0 2023-10-06T05:22:26.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.967+0000 7fe7e9fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e40b2860 con 0x7fe7e40cd510 2023-10-06T05:22:26.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.967+0000 7fe7e37fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe7e4159500 0x7fe7e40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41918/0 (socket says 172.21.15.31:41918) 2023-10-06T05:22:26.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.967+0000 7fe7e37fe700 1 -- 172.21.15.31:0/1388904988 learned_addr learned my addr 172.21.15.31:0/1388904988 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:26.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899185913 0 0) 0x7fe7e40c3f40 con 0x7fe7e4159500 2023-10-06T05:22:26.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7cc003570 con 0x7fe7e4159500 2023-10-06T05:22:26.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e2ffd700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e40cd510 0x7fe7e40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 416257999 0 0) 0x7fe7cc003570 con 0x7fe7e4159500 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e40cd510 msgr2=0x7fe7e40b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e40cd510 0x7fe7e40b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 >> v1:172.21.15.62:6789/0 conn(0x7fe7e415d2d0 legacy=0x7fe7e40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e415d7d0 con 0x7fe7e4159500 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fe7d4001470 con 0x7fe7e4159500 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 >> v1:172.21.15.31:6789/0 conn(0x7fe7e4159500 legacy=0x7fe7e40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e40cd510 0x7fe7cc003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7cc004060 0x7fe7cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7cc004d50 0x7fe7cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e40b2860 con 0x7fe7cc004d50 2023-10-06T05:22:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc00bc20 con 0x7fe7cc004060 2023-10-06T05:22:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.968+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc00bde0 con 0x7fe7e40cd510 2023-10-06T05:22:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.969+0000 7fe7e3fff700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7cc004d50 0x7fe7cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.969+0000 7fe7e37fe700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e40cd510 0x7fe7cc003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.969+0000 7fe7e2ffd700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7cc004060 0x7fe7cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.969+0000 7fe7e3fff700 1 -- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e40cd510 msgr2=0x7fe7cc003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.969+0000 7fe7e3fff700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7e40cd510 0x7fe7cc003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.969+0000 7fe7e3fff700 1 -- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7cc004060 msgr2=0x7fe7cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.969+0000 7fe7e3fff700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7cc004060 0x7fe7cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.969+0000 7fe7e3fff700 1 -- 172.21.15.31:0/1388904988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc003570 con 0x7fe7cc004d50 2023-10-06T05:22:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.969+0000 7fe7e3fff700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7cc004d50 0x7fe7cc0051c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc001fc0 tx=0x7fe7dc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeae21b9999b7315 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.970+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc001950 con 0x7fe7cc004d50 2023-10-06T05:22:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.970+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc00c8b0 con 0x7fe7cc004d50 2023-10-06T05:22:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.970+0000 7fe7e27fc700 1 -- 172.21.15.31:0/1388904988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc00cc50 con 0x7fe7cc004d50 2023-10-06T05:22:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.970+0000 7fe7e9fff700 1 -- 172.21.15.31:0/1388904988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7cc004d50 msgr2=0x7fe7cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.970+0000 7fe7e9fff700 1 --2- 172.21.15.31:0/1388904988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7cc004d50 0x7fe7cc0051c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc001fc0 tx=0x7fe7dc001640 comp rx=0 tx=0).stop 2023-10-06T05:22:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.970+0000 7fe7e3fff700 1 -- 172.21.15.31:0/1388904988 reap_dead start 2023-10-06T05:22:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.970+0000 7fe7e9fff700 1 -- 172.21.15.31:0/1388904988 shutdown_connections 2023-10-06T05:22:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.970+0000 7fe7e9fff700 1 -- 172.21.15.31:0/1388904988 >> 172.21.15.31:0/1388904988 conn(0x7fe7e406ec90 msgr2=0x7fe7e406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.970+0000 7fe7e9fff700 1 -- 172.21.15.31:0/1388904988 shutdown_connections 2023-10-06T05:22:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.970+0000 7fe7e9fff700 1 -- 172.21.15.31:0/1388904988 wait complete. 2023-10-06T05:22:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.972+0000 7fe7e9fff700 1 Processor -- start 2023-10-06T05:22:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.972+0000 7fe7e9fff700 1 -- start start 2023-10-06T05:22:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e9fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7cc004d50 0x7fe7e41530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e9fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e40c8ce0 0x7fe7e40c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e9fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40afd90 0x7fe7e40b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e9fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e40b0700 con 0x7fe7e40afd90 2023-10-06T05:22:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e9fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e41535f0 con 0x7fe7e40c8ce0 2023-10-06T05:22:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e9fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e415a640 con 0x7fe7cc004d50 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40afd90 0x7fe7e40b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e37fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7cc004d50 0x7fe7e41530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40afd90 0x7fe7e40b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42888/0 (socket says 172.21.15.31:42888) 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e3fff700 1 -- 172.21.15.31:0/810791574 learned_addr learned my addr 172.21.15.31:0/810791574 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.973+0000 7fe7e2ffd700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e40c8ce0 0x7fe7e40c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7e3fff700 1 -- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7cc004d50 msgr2=0x7fe7e41530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7e3fff700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7cc004d50 0x7fe7e41530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7e3fff700 1 -- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e40c8ce0 msgr2=0x7fe7e40c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7e3fff700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e40c8ce0 0x7fe7e40c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7e3fff700 1 -- 172.21.15.31:0/810791574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e40c9850 con 0x7fe7e40afd90 2023-10-06T05:22:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7e3fff700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40afd90 0x7fe7e40b01c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00bbf0 tx=0x7fe7dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7cb7fe700 1 -- 172.21.15.31:0/810791574 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc00c6d0 con 0x7fe7e40afd90 2023-10-06T05:22:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7cb7fe700 1 -- 172.21.15.31:0/810791574 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc00cc30 con 0x7fe7e40afd90 2023-10-06T05:22:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7e9fff700 1 -- 172.21.15.31:0/810791574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e40c9b30 con 0x7fe7e40afd90 2023-10-06T05:22:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7cb7fe700 1 -- 172.21.15.31:0/810791574 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc006dd0 con 0x7fe7e40afd90 2023-10-06T05:22:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.974+0000 7fe7e9fff700 1 -- 172.21.15.31:0/810791574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e40ca0f0 con 0x7fe7e40afd90 2023-10-06T05:22:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.975+0000 7fe7e9fff700 1 -- 172.21.15.31:0/810791574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e40ca280 con 0x7fe7e40afd90 2023-10-06T05:22:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.976+0000 7fe7cb7fe700 1 -- 172.21.15.31:0/810791574 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe7dc01f620 con 0x7fe7e40afd90 2023-10-06T05:22:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.977+0000 7fe7cb7fe700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe7d0067050 0x7fe7d0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.977+0000 7fe7cb7fe700 1 -- 172.21.15.31:0/810791574 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fe7dc085890 con 0x7fe7e40afd90 2023-10-06T05:22:26.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.978+0000 7fe7e37fe700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe7d0067050 0x7fe7d0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:26.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.978+0000 7fe7e37fe700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe7d0067050 0x7fe7d0069500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4001e90 tx=0x7fe7d4001d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:26.981+0000 7fe7cb7fe700 1 -- 172.21.15.31:0/810791574 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7e40ca280 con 0x7fe7e40afd90 2023-10-06T05:22:27.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.233+0000 7fe7e9fff700 1 -- 172.21.15.31:0/810791574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7e40ca280 con 0x7fe7e40afd90 2023-10-06T05:22:27.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.233+0000 7fe7cb7fe700 1 -- 172.21.15.31:0/810791574 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3526 (secure 0 0 0) 0x7fe7e40ca280 con 0x7fe7e40afd90 2023-10-06T05:22:27.235 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:27.235 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1878570982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1878570982},{"type":"v1","addr":"172.21.15.62:6834","nonce":1878570982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6687,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/427382574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":427382574},{"type":"v1","addr":"172.21.15.31:6834","nonce":427382574}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1760549774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1760549774},{"type":"v1","addr":"172.21.15.31:6835","nonce":1760549774}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":6990,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3705087189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3705087189},{"type":"v1","addr":"172.21.15.62:6835","nonce":3705087189}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[]} 2023-10-06T05:22:27.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 -- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe7d0067050 msgr2=0x7fe7d0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:27.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe7d0067050 0x7fe7d0069500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4001e90 tx=0x7fe7d4001d80 comp rx=0 tx=0).stop 2023-10-06T05:22:27.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 -- 172.21.15.31:0/810791574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40afd90 msgr2=0x7fe7e40b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:27.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40afd90 0x7fe7e40b01c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00bbf0 tx=0x7fe7dc015000 comp rx=0 tx=0).stop 2023-10-06T05:22:27.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 -- 172.21.15.31:0/810791574 shutdown_connections 2023-10-06T05:22:27.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe7d0067050 0x7fe7d0069500 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:27.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e40afd90 0x7fe7e40b01c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:27.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe7cc004d50 0x7fe7e41530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:27.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 --2- 172.21.15.31:0/810791574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e40c8ce0 0x7fe7e40c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:27.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 -- 172.21.15.31:0/810791574 >> 172.21.15.31:0/810791574 conn(0x7fe7e406ec90 msgr2=0x7fe7e4153ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:27.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 -- 172.21.15.31:0/810791574 shutdown_connections 2023-10-06T05:22:27.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.235+0000 7fe7e9fff700 1 -- 172.21.15.31:0/810791574 wait complete. 2023-10-06T05:22:27.238 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 60 2023-10-06T05:22:27.255 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:22:27.255 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:22:27.255 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:22:27.255 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:22:27.255 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:22:27.256 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:22:27.256 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:22:27.256 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:22:27.256 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:22:27.256 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:22:27.256 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:22:27.256 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:22:27.257 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:22:27.257 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:22:27.257 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:22:27.257 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:22:27.269 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:22:27.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:22:27.270 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:22:27.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:22:27.272 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:22:27.272 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:22:27.273 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:22:27.273 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:22:27.315 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:22:27.316 DEBUG:teuthology.parallel:result is None 2023-10-06T05:22:27.318 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:22:27.318 DEBUG:teuthology.parallel:result is None 2023-10-06T05:22:27.319 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:22:27.319 DEBUG:teuthology.parallel:result is None 2023-10-06T05:22:27.322 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:22:27.322 DEBUG:teuthology.parallel:result is None 2023-10-06T05:22:27.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:22:27.465 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:22:27.466 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:22:27.470 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:22:27.470 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:22:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.530+0000 7f84ee7ff700 1 Processor -- start 2023-10-06T05:22:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.530+0000 7f84ee7ff700 1 -- start start 2023-10-06T05:22:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.531+0000 7f84ee7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84e80c7f30 0x7f84e8069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.531+0000 7f84ee7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f84e80aee70 0x7f84e813f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.531+0000 7f84ee7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e80ca5b0 con 0x7f84e80c7b60 2023-10-06T05:22:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.531+0000 7f84ee7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e80af2f0 con 0x7f84e80c7f30 2023-10-06T05:22:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.531+0000 7f84ee7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e806f740 con 0x7f84e80aee70 2023-10-06T05:22:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84e77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f84e80c7f30 0x7f84e8069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84e7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f84e80c7b60 0x7f84e806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41926/0 (socket says 172.21.15.31:41926) 2023-10-06T05:22:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84e7fff700 1 -- 172.21.15.31:0/1723361457 learned_addr learned my addr 172.21.15.31:0/1723361457 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84ecd9c700 1 --2- 172.21.15.31:0/1723361457 >> v2:172.21.15.62:3300/0 conn(0x7f84e80aee70 0x7f84e813f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84e6ffd700 1 -- 172.21.15.31:0/1723361457 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272162256 0 0) 0x7f84e80ca5b0 con 0x7f84e80c7b60 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84e6ffd700 1 -- 172.21.15.31:0/1723361457 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84d0003570 con 0x7f84e80c7b60 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84e77fe700 1 -- 172.21.15.31:0/1723361457 >> v2:172.21.15.62:3300/0 conn(0x7f84e80aee70 msgr2=0x7f84e813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84e77fe700 1 --2- 172.21.15.31:0/1723361457 >> v2:172.21.15.62:3300/0 conn(0x7f84e80aee70 0x7f84e813f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84e77fe700 1 -- 172.21.15.31:0/1723361457 >> v1:172.21.15.31:6789/0 conn(0x7f84e80c7b60 legacy=0x7f84e806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84e77fe700 1 -- 172.21.15.31:0/1723361457 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e80c9c80 con 0x7f84e80c7f30 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.532+0000 7f84ecd9c700 1 --2- 172.21.15.31:0/1723361457 >> v2:172.21.15.62:3300/0 conn(0x7f84e80aee70 0x7f84e813f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84e77fe700 1 --2- 172.21.15.31:0/1723361457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 0x7f84e8069880 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f84e806d6e0 tx=0x7f84dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d52e5a6457ba9e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84e6ffd700 1 -- 172.21.15.31:0/1723361457 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc011630 con 0x7f84e80c7f30 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84e6ffd700 1 -- 172.21.15.31:0/1723361457 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc011790 con 0x7f84e80c7f30 2023-10-06T05:22:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84e6ffd700 1 -- 172.21.15.31:0/1723361457 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc011b30 con 0x7f84e80c7f30 2023-10-06T05:22:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84ee7ff700 1 -- 172.21.15.31:0/1723361457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 msgr2=0x7f84e8069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84ee7ff700 1 --2- 172.21.15.31:0/1723361457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 0x7f84e8069880 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f84e806d6e0 tx=0x7f84dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84ee7ff700 1 -- 172.21.15.31:0/1723361457 shutdown_connections 2023-10-06T05:22:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84ee7ff700 1 --2- 172.21.15.31:0/1723361457 >> v2:172.21.15.62:3300/0 conn(0x7f84e80aee70 0x7f84e813f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84ee7ff700 1 --2- 172.21.15.31:0/1723361457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 0x7f84e8069880 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84ee7ff700 1 -- 172.21.15.31:0/1723361457 >> 172.21.15.31:0/1723361457 conn(0x7f84e80cc0b0 msgr2=0x7f84e80cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.533+0000 7f84ee7ff700 1 -- 172.21.15.31:0/1723361457 shutdown_connections 2023-10-06T05:22:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.534+0000 7f84ee7ff700 1 -- 172.21.15.31:0/1723361457 wait complete. 2023-10-06T05:22:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.535+0000 7f84ee7ff700 1 Processor -- start 2023-10-06T05:22:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.535+0000 7f84ee7ff700 1 -- start start 2023-10-06T05:22:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.536+0000 7f84ee7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e80aee70 0x7f84e816a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.536+0000 7f84ee7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84e80c7b60 0x7f84e813c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.536+0000 7f84ee7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 0x7f84e80c1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.536+0000 7f84ee7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e80c5140 con 0x7f84e80c7f30 2023-10-06T05:22:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.536+0000 7f84ee7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e80c4b60 con 0x7f84e80aee70 2023-10-06T05:22:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.536+0000 7f84ee7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e816a540 con 0x7f84e80c7b60 2023-10-06T05:22:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.536+0000 7f84ecd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 0x7f84e80c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.536+0000 7f84ecd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 0x7f84e80c1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42940/0 (socket says 172.21.15.31:42940) 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.536+0000 7f84ecd9c700 1 -- 172.21.15.31:0/326670304 learned_addr learned my addr 172.21.15.31:0/326670304 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84e7fff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e80aee70 0x7f84e816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84e77fe700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84e80c7b60 0x7f84e813c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84ecd9c700 1 -- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84e80c7b60 msgr2=0x7f84e813c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84ecd9c700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84e80c7b60 0x7f84e813c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84ecd9c700 1 -- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e80aee70 msgr2=0x7f84e816a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84ecd9c700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e80aee70 0x7f84e816a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84ecd9c700 1 -- 172.21.15.31:0/326670304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc009420 con 0x7f84e80c7f30 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84e7fff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e80aee70 0x7f84e816a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84e77fe700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84e80c7b60 0x7f84e813c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84ecd9c700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 0x7f84e80c1230 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f84e000d1b0 tx=0x7f84e0009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84cffff700 1 -- 172.21.15.31:0/326670304 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e000de50 con 0x7f84e80c7f30 2023-10-06T05:22:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84cffff700 1 -- 172.21.15.31:0/326670304 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e000f040 con 0x7f84e80c7f30 2023-10-06T05:22:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84cffff700 1 -- 172.21.15.31:0/326670304 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0015690 con 0x7f84e80c7f30 2023-10-06T05:22:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84ee7ff700 1 -- 172.21.15.31:0/326670304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e80c44b0 con 0x7f84e80c7f30 2023-10-06T05:22:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.537+0000 7f84ee7ff700 1 -- 172.21.15.31:0/326670304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e80ae390 con 0x7f84e80c7f30 2023-10-06T05:22:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.538+0000 7f84ee7ff700 1 -- 172.21.15.31:0/326670304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e804c380 con 0x7f84e80c7f30 2023-10-06T05:22:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.539+0000 7f84cffff700 1 -- 172.21.15.31:0/326670304 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f84e001e3b0 con 0x7f84e80c7f30 2023-10-06T05:22:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.541+0000 7f84cffff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f84d4067050 0x7f84d4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.541+0000 7f84cffff700 1 -- 172.21.15.31:0/326670304 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f84e0084d10 con 0x7f84e80c7f30 2023-10-06T05:22:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.541+0000 7f84e7fff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f84d4067050 0x7f84d4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.542+0000 7f84e7fff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f84d4067050 0x7f84d4069500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84d8001360 tx=0x7f84d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.545+0000 7f84cffff700 1 -- 172.21.15.31:0/326670304 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84e008a020 con 0x7f84e80c7f30 2023-10-06T05:22:27.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:27.760+0000 7f84ee7ff700 1 -- 172.21.15.31:0/326670304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f84e80c9f60 con 0x7f84e80c7f30 2023-10-06T05:22:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.109+0000 7f84cffff700 1 -- 172.21.15.31:0/326670304 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v95) v1 ==== 97+0+0 (secure 0 0 0) 0x7f84e001a080 con 0x7f84e80c7f30 2023-10-06T05:22:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 -- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f84d4067050 msgr2=0x7f84d4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f84d4067050 0x7f84d4069500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84d8001360 tx=0x7f84d8009040 comp rx=0 tx=0).stop 2023-10-06T05:22:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 -- 172.21.15.31:0/326670304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 msgr2=0x7f84e80c1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 0x7f84e80c1230 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f84e000d1b0 tx=0x7f84e0009a50 comp rx=0 tx=0).stop 2023-10-06T05:22:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 -- 172.21.15.31:0/326670304 shutdown_connections 2023-10-06T05:22:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f84d4067050 0x7f84d4069500 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e80c7f30 0x7f84e80c1230 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e80aee70 0x7f84e816a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 --2- 172.21.15.31:0/326670304 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f84e80c7b60 0x7f84e813c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 -- 172.21.15.31:0/326670304 >> 172.21.15.31:0/326670304 conn(0x7f84e80cc0b0 msgr2=0x7f84e813e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 -- 172.21.15.31:0/326670304 shutdown_connections 2023-10-06T05:22:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.112+0000 7f84ee7ff700 1 -- 172.21.15.31:0/326670304 wait complete. 2023-10-06T05:22:28.115 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:22:28.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:22:28.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.327+0000 7f8bf1bff700 1 Processor -- start 2023-10-06T05:22:28.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.327+0000 7f8bf1bff700 1 -- start start 2023-10-06T05:22:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8bf1bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8bec1595a0 0x7f8bec159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8bf1bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8bec0cc0d0 0x7f8bec068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8bf1bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec068bb0 0x7f8bec0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8bf1bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bec069010 con 0x7f8bec1595a0 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8bf1bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bec0c95b0 con 0x7f8bec0cc0d0 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8bf1bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec0c9890 con 0x7f8bec068bb0 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8beb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8bec1595a0 0x7f8bec159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8beb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8bec1595a0 0x7f8bec159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42952/0 (socket says 172.21.15.31:42952) 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8beb7fe700 1 -- 172.21.15.31:0/1963501544 learned_addr learned my addr 172.21.15.31:0/1963501544 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8bebfff700 1 --2- 172.21.15.31:0/1963501544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec068bb0 0x7f8bec0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8beaffd700 1 --2- 172.21.15.31:0/1963501544 >> v2:172.21.15.62:3300/0 conn(0x7f8bec0cc0d0 0x7f8bec068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8beb7fe700 1 -- 172.21.15.31:0/1963501544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec068bb0 msgr2=0x7f8bec0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.329+0000 7f8beb7fe700 1 --2- 172.21.15.31:0/1963501544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec068bb0 0x7f8bec0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.330+0000 7f8beb7fe700 1 -- 172.21.15.31:0/1963501544 >> v2:172.21.15.62:3300/0 conn(0x7f8bec0cc0d0 msgr2=0x7f8bec068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:28.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.330+0000 7f8beb7fe700 1 --2- 172.21.15.31:0/1963501544 >> v2:172.21.15.62:3300/0 conn(0x7f8bec0cc0d0 0x7f8bec068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.330+0000 7f8beb7fe700 1 -- 172.21.15.31:0/1963501544 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec0c9b70 con 0x7f8bec1595a0 2023-10-06T05:22:28.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.330+0000 7f8beb7fe700 1 --2- 172.21.15.31:0/1963501544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec1595a0 0x7f8bec159970 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8bec06a360 tx=0x7f8bdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d81b46a46d1babd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.330+0000 7f8bea7fc700 1 -- 172.21.15.31:0/1963501544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc00b640 con 0x7f8bec1595a0 2023-10-06T05:22:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.330+0000 7f8bea7fc700 1 -- 172.21.15.31:0/1963501544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bdc00b7a0 con 0x7f8bec1595a0 2023-10-06T05:22:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.330+0000 7f8bea7fc700 1 -- 172.21.15.31:0/1963501544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc00bb60 con 0x7f8bec1595a0 2023-10-06T05:22:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.330+0000 7f8bf1bff700 1 -- 172.21.15.31:0/1963501544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec1595a0 msgr2=0x7f8bec159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.330+0000 7f8bf1bff700 1 --2- 172.21.15.31:0/1963501544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec1595a0 0x7f8bec159970 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8bec06a360 tx=0x7f8bdc009250 comp rx=0 tx=0).stop 2023-10-06T05:22:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.331+0000 7f8bf1bff700 1 -- 172.21.15.31:0/1963501544 shutdown_connections 2023-10-06T05:22:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.331+0000 7f8bf1bff700 1 --2- 172.21.15.31:0/1963501544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec068bb0 0x7f8bec0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.331+0000 7f8bf1bff700 1 --2- 172.21.15.31:0/1963501544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec1595a0 0x7f8bec159970 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.331+0000 7f8bf1bff700 1 --2- 172.21.15.31:0/1963501544 >> v2:172.21.15.62:3300/0 conn(0x7f8bec0cc0d0 0x7f8bec068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.331+0000 7f8bf1bff700 1 -- 172.21.15.31:0/1963501544 >> 172.21.15.31:0/1963501544 conn(0x7f8bec06b620 msgr2=0x7f8bec06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.331+0000 7f8bf1bff700 1 -- 172.21.15.31:0/1963501544 shutdown_connections 2023-10-06T05:22:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.331+0000 7f8bf1bff700 1 -- 172.21.15.31:0/1963501544 wait complete. 2023-10-06T05:22:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.332+0000 7f8bf1bff700 1 Processor -- start 2023-10-06T05:22:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.332+0000 7f8bf1bff700 1 -- start start 2023-10-06T05:22:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.333+0000 7f8bf1bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec068bb0 0x7f8bec153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.333+0000 7f8bf1bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec0cc0d0 0x7f8bec158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.333+0000 7f8bf1bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bec1595a0 0x7f8bec158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.333+0000 7f8bf1bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec15ee80 con 0x7f8bec068bb0 2023-10-06T05:22:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.333+0000 7f8bf1bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec158ab0 con 0x7f8bec1595a0 2023-10-06T05:22:28.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.333+0000 7f8bf1bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec15e270 con 0x7f8bec0cc0d0 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.333+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec068bb0 0x7f8bec153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.333+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec068bb0 0x7f8bec153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42968/0 (socket says 172.21.15.31:42968) 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.333+0000 7f8beb7fe700 1 -- 172.21.15.31:0/538875124 learned_addr learned my addr 172.21.15.31:0/538875124 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8beaffd700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec0cc0d0 0x7f8bec158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8bebfff700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bec1595a0 0x7f8bec158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8beb7fe700 1 -- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec0cc0d0 msgr2=0x7f8bec158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8beb7fe700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec0cc0d0 0x7f8bec158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8beb7fe700 1 -- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bec1595a0 msgr2=0x7f8bec158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8beb7fe700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bec1595a0 0x7f8bec158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8beb7fe700 1 -- 172.21.15.31:0/538875124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc0094f0 con 0x7f8bec068bb0 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8beaffd700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec0cc0d0 0x7f8bec158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8beb7fe700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec068bb0 0x7f8bec153ac0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8bec06ee00 tx=0x7f8bdc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8bd37fe700 1 -- 172.21.15.31:0/538875124 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc000d10 con 0x7f8bec068bb0 2023-10-06T05:22:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8bd37fe700 1 -- 172.21.15.31:0/538875124 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bdc00bb20 con 0x7f8bec068bb0 2023-10-06T05:22:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8bd37fe700 1 -- 172.21.15.31:0/538875124 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc011670 con 0x7f8bec068bb0 2023-10-06T05:22:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.334+0000 7f8bf1bff700 1 -- 172.21.15.31:0/538875124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec15e550 con 0x7f8bec068bb0 2023-10-06T05:22:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.335+0000 7f8bf1bff700 1 -- 172.21.15.31:0/538875124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec15eb10 con 0x7f8bec068bb0 2023-10-06T05:22:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.336+0000 7f8bf1bff700 1 -- 172.21.15.31:0/538875124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bec04c380 con 0x7f8bec068bb0 2023-10-06T05:22:28.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.336+0000 7f8bd37fe700 1 -- 172.21.15.31:0/538875124 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f8bdc0119e0 con 0x7f8bec068bb0 2023-10-06T05:22:28.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.337+0000 7f8bd37fe700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8bd8067050 0x7f8bd8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.337+0000 7f8bd37fe700 1 -- 172.21.15.31:0/538875124 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8bdc085cb0 con 0x7f8bec068bb0 2023-10-06T05:22:28.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.338+0000 7f8beaffd700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8bd8067050 0x7f8bd8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.338+0000 7f8beaffd700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8bd8067050 0x7f8bd8069500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0c94c0 tx=0x7f8be0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.342+0000 7f8bd37fe700 1 -- 172.21.15.31:0/538875124 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bdc052f60 con 0x7f8bec068bb0 2023-10-06T05:22:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.620+0000 7f8bf1bff700 1 -- 172.21.15.31:0/538875124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8bec0cda30 con 0x7f8bec068bb0 2023-10-06T05:22:28.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.621+0000 7f8bd37fe700 1 -- 172.21.15.31:0/538875124 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8bdc056580 con 0x7f8bec068bb0 2023-10-06T05:22:28.623 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:28.623 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:22:28.624 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:22:28.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.624 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:22:28.624 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:22:28.624 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.624 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:22:28.624 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:22:28.624 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:22:28.624 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.627 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.628 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.629 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.630 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:22:28.630 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:22:28.630 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.630 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.630 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.630 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.632 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:22:28.632 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:22:28.632 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.632 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.632 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.632 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.632 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:22:28.633 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:22:28.633 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.633 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.633 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.633 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.634 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:22:28.634 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:22:28.634 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.634 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:22:28.634 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:22:28.635 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.635 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:22:28.635 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:22:28.636 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.636 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:22:28.636 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:22:28.636 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:22:28.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.637 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:22:28.637 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:22:28.637 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.637 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:22:28.637 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:22:28.637 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:22:28.638 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:22:28.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.638 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:22:28.638 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:22:28.639 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.639 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:22:28.639 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:22:28.639 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:22:28.639 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:22:28.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.640 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:22:28.640 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:22:28.640 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.640 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:22:28.641 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:22:28.641 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:22:28.641 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:22:28.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.642 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:22:28.642 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:22:28.642 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.642 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:22:28.642 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:22:28.642 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:22:28.642 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.643 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:22:28.643 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:22:28.643 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.643 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:22:28.644 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:22:28.644 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:28.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:28.644 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:22:28.644 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:22:28.644 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:22:28.645 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:22:28.645 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:22:28.645 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:22:28.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:28.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:22:28.645 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:22:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 -- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8bd8067050 msgr2=0x7f8bd8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8bd8067050 0x7f8bd8069500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0c94c0 tx=0x7f8be0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:28.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 -- 172.21.15.31:0/538875124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec068bb0 msgr2=0x7f8bec153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec068bb0 0x7f8bec153ac0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8bec06ee00 tx=0x7f8bdc017000 comp rx=0 tx=0).stop 2023-10-06T05:22:28.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 -- 172.21.15.31:0/538875124 shutdown_connections 2023-10-06T05:22:28.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8bd8067050 0x7f8bd8069500 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bec1595a0 0x7f8bec158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bec068bb0 0x7f8bec153ac0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 --2- 172.21.15.31:0/538875124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bec0cc0d0 0x7f8bec158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 -- 172.21.15.31:0/538875124 >> 172.21.15.31:0/538875124 conn(0x7f8bec06b620 msgr2=0x7f8bec15b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 -- 172.21.15.31:0/538875124 shutdown_connections 2023-10-06T05:22:28.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.623+0000 7f8bf1bff700 1 -- 172.21.15.31:0/538875124 wait complete. 2023-10-06T05:22:28.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:28.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.845+0000 7f034ebff700 1 Processor -- start 2023-10-06T05:22:28.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.845+0000 7f034ebff700 1 -- start start 2023-10-06T05:22:28.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.846+0000 7f034ebff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034813a5b0 0x7f034806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.846+0000 7f034ebff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034806ddc0 con 0x7f034806d980 2023-10-06T05:22:28.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.846+0000 7f034ebff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03480c71b0 con 0x7f034813a1e0 2023-10-06T05:22:28.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.846+0000 7f034ebff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03480c7490 con 0x7f034813a5b0 2023-10-06T05:22:28.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.847+0000 7f034d19c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f034806d980 0x7f0348069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41938/0 (socket says 172.21.15.31:41938) 2023-10-06T05:22:28.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.847+0000 7f034d19c700 1 -- 172.21.15.31:0/2877453349 learned_addr learned my addr 172.21.15.31:0/2877453349 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:28.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.847+0000 7f0347fff700 1 --2- 172.21.15.31:0/2877453349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034813a5b0 0x7f034806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.847+0000 7f03477fe700 1 -- 172.21.15.31:0/2877453349 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206697701 0 0) 0x7f034806ddc0 con 0x7f034806d980 2023-10-06T05:22:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.847+0000 7f03477fe700 1 -- 172.21.15.31:0/2877453349 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0338003570 con 0x7f034806d980 2023-10-06T05:22:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.847+0000 7f03477fe700 1 -- 172.21.15.31:0/2877453349 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032124494 0 0) 0x7f03480c71b0 con 0x7f034813a1e0 2023-10-06T05:22:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.847+0000 7f03477fe700 1 -- 172.21.15.31:0/2877453349 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f034806ddc0 con 0x7f034813a1e0 2023-10-06T05:22:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.847+0000 7f0347fff700 1 -- 172.21.15.31:0/2877453349 >> v1:172.21.15.62:6789/0 conn(0x7f034813a1e0 legacy=0x7f03480c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.847+0000 7f0347fff700 1 -- 172.21.15.31:0/2877453349 >> v1:172.21.15.31:6789/0 conn(0x7f034806d980 legacy=0x7f0348069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.848+0000 7f03477fe700 1 -- 172.21.15.31:0/2877453349 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 741181337 0 0) 0x7f0338003570 con 0x7f034806d980 2023-10-06T05:22:28.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.848+0000 7f0347fff700 1 -- 172.21.15.31:0/2877453349 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03480c7770 con 0x7f034813a5b0 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.848+0000 7f0347fff700 1 --2- 172.21.15.31:0/2877453349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034813a5b0 0x7f034806d440 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0334009910 tx=0x7f03340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8db0764041874b61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.848+0000 7f03477fe700 1 -- 172.21.15.31:0/2877453349 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400017f0 con 0x7f034813a5b0 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.848+0000 7f03477fe700 1 -- 172.21.15.31:0/2877453349 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0334011420 con 0x7f034813a5b0 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.848+0000 7f03477fe700 1 -- 172.21.15.31:0/2877453349 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03340117c0 con 0x7f034813a5b0 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.849+0000 7f034ebff700 1 -- 172.21.15.31:0/2877453349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034813a5b0 msgr2=0x7f034806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.849+0000 7f034ebff700 1 --2- 172.21.15.31:0/2877453349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034813a5b0 0x7f034806d440 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0334009910 tx=0x7f03340091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.849+0000 7f034ebff700 1 -- 172.21.15.31:0/2877453349 shutdown_connections 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.849+0000 7f034ebff700 1 --2- 172.21.15.31:0/2877453349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034813a5b0 0x7f034806d440 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.849+0000 7f034ebff700 1 -- 172.21.15.31:0/2877453349 >> 172.21.15.31:0/2877453349 conn(0x7f03480cc0b0 msgr2=0x7f03480cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.849+0000 7f034ebff700 1 -- 172.21.15.31:0/2877453349 shutdown_connections 2023-10-06T05:22:28.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.849+0000 7f034ebff700 1 -- 172.21.15.31:0/2877453349 wait complete. 2023-10-06T05:22:28.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.850+0000 7f034ebff700 1 Processor -- start 2023-10-06T05:22:28.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.851+0000 7f034ebff700 1 -- start start 2023-10-06T05:22:28.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.851+0000 7f034ebff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034806d980 0x7f03480c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.851+0000 7f034ebff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034813a1e0 0x7f03481520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f034ebff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f034813a5b0 0x7f03481525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f034ebff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03480c53a0 con 0x7f034813a1e0 2023-10-06T05:22:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f034ebff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034813ad30 con 0x7f034813a5b0 2023-10-06T05:22:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f034ebff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03480c4ce0 con 0x7f034806d980 2023-10-06T05:22:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f0347fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034813a1e0 0x7f03481520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f034c99b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034806d980 0x7f03480c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f0347fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034813a1e0 0x7f03481520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42970/0 (socket says 172.21.15.31:42970) 2023-10-06T05:22:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f0347fff700 1 -- 172.21.15.31:0/3447798795 learned_addr learned my addr 172.21.15.31:0/3447798795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f034d19c700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f034813a5b0 0x7f03481525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f0347fff700 1 -- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034806d980 msgr2=0x7f03480c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f0347fff700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034806d980 0x7f03480c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f0347fff700 1 -- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f034813a5b0 msgr2=0x7f03481525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f0347fff700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f034813a5b0 0x7f03481525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.852+0000 7f0347fff700 1 -- 172.21.15.31:0/3447798795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0334009420 con 0x7f034813a1e0 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.853+0000 7f0347fff700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034813a1e0 0x7f03481520a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f03340091e0 tx=0x7f0334016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.853+0000 7f0344ff9700 1 -- 172.21.15.31:0/3447798795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033401cbe0 con 0x7f034813a1e0 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.853+0000 7f0344ff9700 1 -- 172.21.15.31:0/3447798795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033401cd40 con 0x7f034813a1e0 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.853+0000 7f0344ff9700 1 -- 172.21.15.31:0/3447798795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033401a6a0 con 0x7f034813a1e0 2023-10-06T05:22:28.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.853+0000 7f034ebff700 1 -- 172.21.15.31:0/3447798795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03480c4fc0 con 0x7f034813a1e0 2023-10-06T05:22:28.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.853+0000 7f034ebff700 1 -- 172.21.15.31:0/3447798795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03480c48c0 con 0x7f034813a1e0 2023-10-06T05:22:28.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.854+0000 7f034ebff700 1 -- 172.21.15.31:0/3447798795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03480c54e0 con 0x7f034813a1e0 2023-10-06T05:22:28.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.855+0000 7f0344ff9700 1 -- 172.21.15.31:0/3447798795 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f033401aad0 con 0x7f034813a1e0 2023-10-06T05:22:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.856+0000 7f0344ff9700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f032c067050 0x7f032c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.857+0000 7f0344ff9700 1 -- 172.21.15.31:0/3447798795 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f03340862e0 con 0x7f034813a1e0 2023-10-06T05:22:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.857+0000 7f034c99b700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f032c067050 0x7f032c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.857+0000 7f034c99b700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f032c067050 0x7f032c069500 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f033c003ae0 tx=0x7f033c007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:28.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:28.861+0000 7f0344ff9700 1 -- 172.21.15.31:0/3447798795 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03480c54e0 con 0x7f034813a1e0 2023-10-06T05:22:29.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.064+0000 7f034ebff700 1 -- 172.21.15.31:0/3447798795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03480c54e0 con 0x7f034813a1e0 2023-10-06T05:22:29.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.065+0000 7f0344ff9700 1 -- 172.21.15.31:0/3447798795 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+13699 (secure 0 0 0) 0x7f03480c54e0 con 0x7f034813a1e0 2023-10-06T05:22:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:29.067 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":97,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:22:28.115182+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.067+0000 7f034ebff700 1 -- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f032c067050 msgr2=0x7f032c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.067+0000 7f034ebff700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f032c067050 0x7f032c069500 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f033c003ae0 tx=0x7f033c007d40 comp rx=0 tx=0).stop 2023-10-06T05:22:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.067+0000 7f034ebff700 1 -- 172.21.15.31:0/3447798795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034813a1e0 msgr2=0x7f03481520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.067+0000 7f034ebff700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034813a1e0 0x7f03481520a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f03340091e0 tx=0x7f0334016000 comp rx=0 tx=0).stop 2023-10-06T05:22:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.067+0000 7f034ebff700 1 -- 172.21.15.31:0/3447798795 shutdown_connections 2023-10-06T05:22:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.067+0000 7f034ebff700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f032c067050 0x7f032c069500 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.068+0000 7f034ebff700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f034813a5b0 0x7f03481525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.068+0000 7f034ebff700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f034806d980 0x7f03480c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.068+0000 7f034ebff700 1 --2- 172.21.15.31:0/3447798795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f034813a1e0 0x7f03481520a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.068+0000 7f034ebff700 1 -- 172.21.15.31:0/3447798795 >> 172.21.15.31:0/3447798795 conn(0x7f03480cc0b0 msgr2=0x7f03481571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.068+0000 7f034ebff700 1 -- 172.21.15.31:0/3447798795 shutdown_connections 2023-10-06T05:22:29.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.068+0000 7f034ebff700 1 -- 172.21.15.31:0/3447798795 wait complete. 2023-10-06T05:22:29.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:22:29.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:22:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.286+0000 7f5c123ff700 1 Processor -- start 2023-10-06T05:22:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.287+0000 7f5c123ff700 1 -- start start 2023-10-06T05:22:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.288+0000 7f5c123ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c0c153d40 0x7f5c0c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.288+0000 7f5c123ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c0c0b09c0 0x7f5c0c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.288+0000 7f5c123ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c0ad690 con 0x7f5c0c0af6c0 2023-10-06T05:22:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.288+0000 7f5c123ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c0b0e30 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.288+0000 7f5c123ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0acd60 con 0x7f5c0c153d40 2023-10-06T05:22:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.288+0000 7f5c0b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c0c0b09c0 0x7f5c0c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.288+0000 7f5c0b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c0c0b09c0 0x7f5c0c0af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42972/0 (socket says 172.21.15.31:42972) 2023-10-06T05:22:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.288+0000 7f5c0b7fe700 1 -- 172.21.15.31:0/1841079017 learned_addr learned my addr 172.21.15.31:0/1841079017 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.288+0000 7f5c1099c700 1 --1- 172.21.15.31:0/1841079017 >> v1:172.21.15.31:6789/0 conn(0x7f5c0c0af6c0 0x7f5c0c159ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41952/0 (socket says 172.21.15.31:41952) 2023-10-06T05:22:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.288+0000 7f5c0bfff700 1 --2- 172.21.15.31:0/1841079017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c0c153d40 0x7f5c0c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.289+0000 7f5c0b7fe700 1 -- 172.21.15.31:0/1841079017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c0c153d40 msgr2=0x7f5c0c0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.289+0000 7f5c0affd700 1 -- 172.21.15.31:0/1841079017 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631548846 0 0) 0x7f5c0c0ad690 con 0x7f5c0c0af6c0 2023-10-06T05:22:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.289+0000 7f5c0b7fe700 1 --2- 172.21.15.31:0/1841079017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c0c153d40 0x7f5c0c0b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.289+0000 7f5c0b7fe700 1 -- 172.21.15.31:0/1841079017 >> v1:172.21.15.31:6789/0 conn(0x7f5c0c0af6c0 legacy=0x7f5c0c159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.289+0000 7f5c0b7fe700 1 -- 172.21.15.31:0/1841079017 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c0ad040 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.289+0000 7f5c0b7fe700 1 --2- 172.21.15.31:0/1841079017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 0x7f5c0c0af180 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5c000099b0 tx=0x7f5c00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ba0b75b9a5c1a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.289+0000 7f5c0affd700 1 -- 172.21.15.31:0/1841079017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00009ea0 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.289+0000 7f5c0affd700 1 -- 172.21.15.31:0/1841079017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0000b860 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.289+0000 7f5c0affd700 1 -- 172.21.15.31:0/1841079017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000bc00 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.290+0000 7f5c123ff700 1 -- 172.21.15.31:0/1841079017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 msgr2=0x7f5c0c0af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.290+0000 7f5c123ff700 1 --2- 172.21.15.31:0/1841079017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 0x7f5c0c0af180 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5c000099b0 tx=0x7f5c00009250 comp rx=0 tx=0).stop 2023-10-06T05:22:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.290+0000 7f5c123ff700 1 -- 172.21.15.31:0/1841079017 shutdown_connections 2023-10-06T05:22:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.290+0000 7f5c123ff700 1 --2- 172.21.15.31:0/1841079017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c0c153d40 0x7f5c0c0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.290+0000 7f5c123ff700 1 --2- 172.21.15.31:0/1841079017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 0x7f5c0c0af180 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.290+0000 7f5c123ff700 1 -- 172.21.15.31:0/1841079017 >> 172.21.15.31:0/1841079017 conn(0x7f5c0c067560 msgr2=0x7f5c0c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.290+0000 7f5c123ff700 1 -- 172.21.15.31:0/1841079017 shutdown_connections 2023-10-06T05:22:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.290+0000 7f5c123ff700 1 -- 172.21.15.31:0/1841079017 wait complete. 2023-10-06T05:22:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.291+0000 7f5c123ff700 1 Processor -- start 2023-10-06T05:22:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.292+0000 7f5c123ff700 1 -- start start 2023-10-06T05:22:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.292+0000 7f5c123ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c0c0af6c0 0x7f5c0c144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.292+0000 7f5c123ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 0x7f5c0c0c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.292+0000 7f5c123ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c0c153d40 0x7f5c0c0abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.292+0000 7f5c123ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0c2130 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.292+0000 7f5c123ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c169730 con 0x7f5c0c153d40 2023-10-06T05:22:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.292+0000 7f5c123ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0c1890 con 0x7f5c0c0af6c0 2023-10-06T05:22:29.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c0b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 0x7f5c0c0c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c0b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 0x7f5c0c0c0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42978/0 (socket says 172.21.15.31:42978) 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c0b7fe700 1 -- 172.21.15.31:0/677723103 learned_addr learned my addr 172.21.15.31:0/677723103 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c0bfff700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c0c0af6c0 0x7f5c0c144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c1099c700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c0c153d40 0x7f5c0c0abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c0b7fe700 1 -- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c0c0af6c0 msgr2=0x7f5c0c144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c0b7fe700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c0c0af6c0 0x7f5c0c144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c0b7fe700 1 -- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c0c153d40 msgr2=0x7f5c0c0abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c0b7fe700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c0c153d40 0x7f5c0c0abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c0b7fe700 1 -- 172.21.15.31:0/677723103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c000094f0 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c1099c700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c0c153d40 0x7f5c0c0abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.293+0000 7f5c0b7fe700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 0x7f5c0c0c0ed0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c06f430 tx=0x7f5c00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.294+0000 7f5bf3fff700 1 -- 172.21.15.31:0/677723103 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000bbe0 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.294+0000 7f5bf3fff700 1 -- 172.21.15.31:0/677723103 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c00011420 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.294+0000 7f5bf3fff700 1 -- 172.21.15.31:0/677723103 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c000117e0 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.294+0000 7f5c123ff700 1 -- 172.21.15.31:0/677723103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0c0c1b70 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.294+0000 7f5c123ff700 1 -- 172.21.15.31:0/677723103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c145020 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.295+0000 7f5c123ff700 1 -- 172.21.15.31:0/677723103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c1698a0 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.296+0000 7f5bf3fff700 1 -- 172.21.15.31:0/677723103 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f5c00011bd0 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.296+0000 7f5bf3fff700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5bf8067080 0x7f5bf8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.297+0000 7f5bf3fff700 1 -- 172.21.15.31:0/677723103 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5c00085d60 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.297+0000 7f5c0bfff700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5bf8067080 0x7f5bf8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.297+0000 7f5c0bfff700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5bf8067080 0x7f5bf8069530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009d10 tx=0x7f5bfc0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.301+0000 7f5bf3fff700 1 -- 172.21.15.31:0/677723103 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c000530b0 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.506+0000 7f5c123ff700 1 -- 172.21.15.31:0/677723103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c0c04c380 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.506+0000 7f5bf3fff700 1 -- 172.21.15.31:0/677723103 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v97) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5c000566d0 con 0x7f5c0c0b09c0 2023-10-06T05:22:29.508 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:22:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 -- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5bf8067080 msgr2=0x7f5bf8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5bf8067080 0x7f5bf8069530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009d10 tx=0x7f5bfc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 -- 172.21.15.31:0/677723103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 msgr2=0x7f5c0c0c0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 0x7f5c0c0c0ed0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c06f430 tx=0x7f5c00017000 comp rx=0 tx=0).stop 2023-10-06T05:22:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 -- 172.21.15.31:0/677723103 shutdown_connections 2023-10-06T05:22:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5bf8067080 0x7f5bf8069530 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c0c153d40 0x7f5c0c0abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c0c0af6c0 0x7f5c0c144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 --2- 172.21.15.31:0/677723103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c0c0b09c0 0x7f5c0c0c0ed0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 -- 172.21.15.31:0/677723103 >> 172.21.15.31:0/677723103 conn(0x7f5c0c067560 msgr2=0x7f5c0c06ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 -- 172.21.15.31:0/677723103 shutdown_connections 2023-10-06T05:22:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.508+0000 7f5c123ff700 1 -- 172.21.15.31:0/677723103 wait complete. 2023-10-06T05:22:29.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:22:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.722+0000 7fa3679ff700 1 Processor -- start 2023-10-06T05:22:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.723+0000 7fa3679ff700 1 -- start start 2023-10-06T05:22:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.724+0000 7fa3679ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa3600cc710 0x7fa3600aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.724+0000 7fa3679ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36015a020 con 0x7fa3600cc340 2023-10-06T05:22:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.724+0000 7fa3679ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36015abc0 con 0x7fa3600b2540 2023-10-06T05:22:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.724+0000 7fa3679ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3600b07e0 con 0x7fa3600cc710 2023-10-06T05:22:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.724+0000 7fa36579b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa3600cc340 0x7fa3600b00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41968/0 (socket says 172.21.15.31:41968) 2023-10-06T05:22:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.724+0000 7fa36579b700 1 -- 172.21.15.31:0/2960168242 learned_addr learned my addr 172.21.15.31:0/2960168242 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.724+0000 7fa365f9c700 1 --2- 172.21.15.31:0/2960168242 >> v2:172.21.15.62:3300/0 conn(0x7fa3600cc710 0x7fa3600aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.725+0000 7fa357fff700 1 -- 172.21.15.31:0/2960168242 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557478096 0 0) 0x7fa36015a020 con 0x7fa3600cc340 2023-10-06T05:22:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.725+0000 7fa357fff700 1 -- 172.21.15.31:0/2960168242 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa34c003570 con 0x7fa3600cc340 2023-10-06T05:22:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.725+0000 7fa365f9c700 1 -- 172.21.15.31:0/2960168242 >> v1:172.21.15.62:6789/0 conn(0x7fa3600b2540 legacy=0x7fa36015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.725+0000 7fa365f9c700 1 -- 172.21.15.31:0/2960168242 >> v1:172.21.15.31:6789/0 conn(0x7fa3600cc340 legacy=0x7fa3600b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.725+0000 7fa365f9c700 1 -- 172.21.15.31:0/2960168242 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa360159cf0 con 0x7fa3600cc710 2023-10-06T05:22:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.725+0000 7fa365f9c700 1 --2- 172.21.15.31:0/2960168242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3600cc710 0x7fa3600aecd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa358009e90 tx=0x7fa358009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ef13f878575c4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.725+0000 7fa357fff700 1 -- 172.21.15.31:0/2960168242 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35800e070 con 0x7fa3600cc710 2023-10-06T05:22:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.725+0000 7fa357fff700 1 -- 172.21.15.31:0/2960168242 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35800bc40 con 0x7fa3600cc710 2023-10-06T05:22:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.726+0000 7fa357fff700 1 -- 172.21.15.31:0/2960168242 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358012430 con 0x7fa3600cc710 2023-10-06T05:22:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.726+0000 7fa3679ff700 1 -- 172.21.15.31:0/2960168242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3600cc710 msgr2=0x7fa3600aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.726+0000 7fa3679ff700 1 --2- 172.21.15.31:0/2960168242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3600cc710 0x7fa3600aecd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa358009e90 tx=0x7fa358009780 comp rx=0 tx=0).stop 2023-10-06T05:22:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.726+0000 7fa3679ff700 1 -- 172.21.15.31:0/2960168242 shutdown_connections 2023-10-06T05:22:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.726+0000 7fa3679ff700 1 --2- 172.21.15.31:0/2960168242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3600cc710 0x7fa3600aecd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.726+0000 7fa3679ff700 1 -- 172.21.15.31:0/2960168242 >> 172.21.15.31:0/2960168242 conn(0x7fa360068670 msgr2=0x7fa360068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.726+0000 7fa3679ff700 1 -- 172.21.15.31:0/2960168242 shutdown_connections 2023-10-06T05:22:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.726+0000 7fa3679ff700 1 -- 172.21.15.31:0/2960168242 wait complete. 2023-10-06T05:22:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.728+0000 7fa3679ff700 1 Processor -- start 2023-10-06T05:22:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.728+0000 7fa3679ff700 1 -- start start 2023-10-06T05:22:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa3679ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3600b2540 0x7fa3600aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa3679ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3600cc340 0x7fa36016a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa3679ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cc710 0x7fa36014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa3679ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3600afb10 con 0x7fa3600cc340 2023-10-06T05:22:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa3679ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3600af550 con 0x7fa3600b2540 2023-10-06T05:22:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa3679ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3600af830 con 0x7fa3600cc710 2023-10-06T05:22:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa364f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3600cc340 0x7fa36016a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa364f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3600cc340 0x7fa36016a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42982/0 (socket says 172.21.15.31:42982) 2023-10-06T05:22:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa364f9a700 1 -- 172.21.15.31:0/1962446271 learned_addr learned my addr 172.21.15.31:0/1962446271 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa36579b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3600b2540 0x7fa3600aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa365f9c700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cc710 0x7fa36014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa364f9a700 1 -- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cc710 msgr2=0x7fa36014bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa364f9a700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cc710 0x7fa36014bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa364f9a700 1 -- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3600b2540 msgr2=0x7fa3600aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa364f9a700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3600b2540 0x7fa3600aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa364f9a700 1 -- 172.21.15.31:0/1962446271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358009a20 con 0x7fa3600cc340 2023-10-06T05:22:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.729+0000 7fa365f9c700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cc710 0x7fa36014bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.730+0000 7fa364f9a700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3600cc340 0x7fa36016a4a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa35000c4e0 tx=0x7fa35000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.730+0000 7fa3557fa700 1 -- 172.21.15.31:0/1962446271 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350011590 con 0x7fa3600cc340 2023-10-06T05:22:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.730+0000 7fa3557fa700 1 -- 172.21.15.31:0/1962446271 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa350011710 con 0x7fa3600cc340 2023-10-06T05:22:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.730+0000 7fa3679ff700 1 -- 172.21.15.31:0/1962446271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa360169ca0 con 0x7fa3600cc340 2023-10-06T05:22:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.730+0000 7fa3557fa700 1 -- 172.21.15.31:0/1962446271 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350011ad0 con 0x7fa3600cc340 2023-10-06T05:22:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.730+0000 7fa3679ff700 1 -- 172.21.15.31:0/1962446271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36016a9e0 con 0x7fa3600cc340 2023-10-06T05:22:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.731+0000 7fa3679ff700 1 -- 172.21.15.31:0/1962446271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa360169b00 con 0x7fa3600cc340 2023-10-06T05:22:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.732+0000 7fa3557fa700 1 -- 172.21.15.31:0/1962446271 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fa35001b3b0 con 0x7fa3600cc340 2023-10-06T05:22:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.733+0000 7fa3557fa700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa344067050 0x7fa344069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.733+0000 7fa3557fa700 1 -- 172.21.15.31:0/1962446271 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa3500839e0 con 0x7fa3600cc340 2023-10-06T05:22:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.733+0000 7fa36579b700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa344067050 0x7fa344069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.734+0000 7fa36579b700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa344067050 0x7fa344069500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa360154760 tx=0x7fa35c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.737+0000 7fa3557fa700 1 -- 172.21.15.31:0/1962446271 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa360169b00 con 0x7fa3600cc340 2023-10-06T05:22:29.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.939+0000 7fa3679ff700 1 -- 172.21.15.31:0/1962446271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa360169b00 con 0x7fa3600cc340 2023-10-06T05:22:29.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.940+0000 7fa3557fa700 1 -- 172.21.15.31:0/1962446271 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa360169b00 con 0x7fa3600cc340 2023-10-06T05:22:29.941 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:22:29.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 -- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa344067050 msgr2=0x7fa344069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa344067050 0x7fa344069500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa360154760 tx=0x7fa35c009040 comp rx=0 tx=0).stop 2023-10-06T05:22:29.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 -- 172.21.15.31:0/1962446271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3600cc340 msgr2=0x7fa36016a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:29.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3600cc340 0x7fa36016a4a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa35000c4e0 tx=0x7fa35000c7f0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 -- 172.21.15.31:0/1962446271 shutdown_connections 2023-10-06T05:22:29.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa344067050 0x7fa344069500 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cc710 0x7fa36014bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3600b2540 0x7fa3600aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 --2- 172.21.15.31:0/1962446271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3600cc340 0x7fa36016a4a0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:29.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 -- 172.21.15.31:0/1962446271 >> 172.21.15.31:0/1962446271 conn(0x7fa360068670 msgr2=0x7fa360068a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:29.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 -- 172.21.15.31:0/1962446271 shutdown_connections 2023-10-06T05:22:29.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:29.942+0000 7fa3679ff700 1 -- 172.21.15.31:0/1962446271 wait complete. 2023-10-06T05:22:29.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:22:30.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.155+0000 7f288bfff700 1 Processor -- start 2023-10-06T05:22:30.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.156+0000 7f288bfff700 1 -- start start 2023-10-06T05:22:30.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.157+0000 7f288bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2884159500 0x7f28840b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.157+0000 7f288bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 0x7f28840cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.157+0000 7f288bfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28840c3f40 con 0x7f28841598d0 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.157+0000 7f288bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28840b2860 con 0x7f2884159500 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.157+0000 7f288bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288415d7d0 con 0x7f288415d2d0 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.157+0000 7f288a59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f28841598d0 0x7f28840aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41974/0 (socket says 172.21.15.31:41974) 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.157+0000 7f288a59c700 1 -- 172.21.15.31:0/1232283250 learned_addr learned my addr 172.21.15.31:0/1232283250 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.157+0000 7f2889d9b700 1 --2- 172.21.15.31:0/1232283250 >> v2:172.21.15.62:3300/0 conn(0x7f2884159500 0x7f28840b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.157+0000 7f288959a700 1 --2- 172.21.15.31:0/1232283250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 0x7f28840cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.158+0000 7f2888d99700 1 -- 172.21.15.31:0/1232283250 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117016069 0 0) 0x7f28840c3f40 con 0x7f28841598d0 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.158+0000 7f2888d99700 1 -- 172.21.15.31:0/1232283250 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2870003570 con 0x7f28841598d0 2023-10-06T05:22:30.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.158+0000 7f2889d9b700 1 -- 172.21.15.31:0/1232283250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 msgr2=0x7f28840cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:30.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.158+0000 7f2889d9b700 1 --2- 172.21.15.31:0/1232283250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 0x7f28840cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.158+0000 7f2889d9b700 1 -- 172.21.15.31:0/1232283250 >> v1:172.21.15.31:6789/0 conn(0x7f28841598d0 legacy=0x7f28840aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.158+0000 7f2889d9b700 1 -- 172.21.15.31:0/1232283250 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840cb000 con 0x7f2884159500 2023-10-06T05:22:30.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.158+0000 7f288959a700 1 --2- 172.21.15.31:0/1232283250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 0x7f28840cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:30.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.158+0000 7f2889d9b700 1 --2- 172.21.15.31:0/1232283250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2884159500 0x7f28840b0ab0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f28800098e0 tx=0x7f28800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2815989b23561a38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.158+0000 7f2888d99700 1 -- 172.21.15.31:0/1232283250 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880011630 con 0x7f2884159500 2023-10-06T05:22:30.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.158+0000 7f2888d99700 1 -- 172.21.15.31:0/1232283250 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2880011790 con 0x7f2884159500 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.159+0000 7f2888d99700 1 -- 172.21.15.31:0/1232283250 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880011b30 con 0x7f2884159500 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.159+0000 7f288bfff700 1 -- 172.21.15.31:0/1232283250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2884159500 msgr2=0x7f28840b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.159+0000 7f288bfff700 1 --2- 172.21.15.31:0/1232283250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2884159500 0x7f28840b0ab0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f28800098e0 tx=0x7f28800091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.159+0000 7f288bfff700 1 -- 172.21.15.31:0/1232283250 shutdown_connections 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.159+0000 7f288bfff700 1 --2- 172.21.15.31:0/1232283250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2884159500 0x7f28840b0ab0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.159+0000 7f288bfff700 1 --2- 172.21.15.31:0/1232283250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 0x7f28840cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.159+0000 7f288bfff700 1 -- 172.21.15.31:0/1232283250 >> 172.21.15.31:0/1232283250 conn(0x7f288406ec90 msgr2=0x7f288406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.159+0000 7f288bfff700 1 -- 172.21.15.31:0/1232283250 shutdown_connections 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.159+0000 7f288bfff700 1 -- 172.21.15.31:0/1232283250 wait complete. 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.161+0000 7f288bfff700 1 Processor -- start 2023-10-06T05:22:30.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.161+0000 7f288bfff700 1 -- start start 2023-10-06T05:22:30.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f288bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2884159500 0x7f288415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f288bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28841598d0 0x7f28840b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f288bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 0x7f28840b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f288bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288415aed0 con 0x7f2884159500 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f288bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288415a2c0 con 0x7f28841598d0 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f288bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288415a5a0 con 0x7f288415d2d0 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f2889d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2884159500 0x7f288415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f2889d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2884159500 0x7f288415b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42998/0 (socket says 172.21.15.31:42998) 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f2889d9b700 1 -- 172.21.15.31:0/3523123098 learned_addr learned my addr 172.21.15.31:0/3523123098 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f288959a700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28841598d0 0x7f28840b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.162+0000 7f288a59c700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 0x7f28840b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f2889d9b700 1 -- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 msgr2=0x7f28840b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f2889d9b700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 0x7f28840b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f2889d9b700 1 -- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28841598d0 msgr2=0x7f28840b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f2889d9b700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28841598d0 0x7f28840b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f2889d9b700 1 -- 172.21.15.31:0/3523123098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28740093c0 con 0x7f2884159500 2023-10-06T05:22:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f288959a700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28841598d0 0x7f28840b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f2889d9b700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2884159500 0x7f288415b360 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f28840ca9f0 tx=0x7f2880017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f2879ffb700 1 -- 172.21.15.31:0/3523123098 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880011b10 con 0x7f2884159500 2023-10-06T05:22:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f2879ffb700 1 -- 172.21.15.31:0/3523123098 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28800253f0 con 0x7f2884159500 2023-10-06T05:22:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f288bfff700 1 -- 172.21.15.31:0/3523123098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2880009420 con 0x7f2884159500 2023-10-06T05:22:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f2879ffb700 1 -- 172.21.15.31:0/3523123098 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880025790 con 0x7f2884159500 2023-10-06T05:22:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.163+0000 7f288bfff700 1 -- 172.21.15.31:0/3523123098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288415ab60 con 0x7f2884159500 2023-10-06T05:22:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.164+0000 7f288bfff700 1 -- 172.21.15.31:0/3523123098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288415b1b0 con 0x7f2884159500 2023-10-06T05:22:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.165+0000 7f2879ffb700 1 -- 172.21.15.31:0/3523123098 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f2880025b80 con 0x7f2884159500 2023-10-06T05:22:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.167+0000 7f2879ffb700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2868067050 0x7f2868069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.167+0000 7f2879ffb700 1 -- 172.21.15.31:0/3523123098 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2880085da0 con 0x7f2884159500 2023-10-06T05:22:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.167+0000 7f288959a700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2868067050 0x7f2868069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.168+0000 7f288959a700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2868067050 0x7f2868069500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2874009390 tx=0x7f2874008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.171+0000 7f2879ffb700 1 -- 172.21.15.31:0/3523123098 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2880053050 con 0x7f2884159500 2023-10-06T05:22:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.373+0000 7f288bfff700 1 -- 172.21.15.31:0/3523123098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f288404c380 con 0x7f2884159500 2023-10-06T05:22:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.373+0000 7f2879ffb700 1 -- 172.21.15.31:0/3523123098 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2880015090 con 0x7f2884159500 2023-10-06T05:22:30.375 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:30.376 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:22:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 -- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2868067050 msgr2=0x7f2868069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2868067050 0x7f2868069500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2874009390 tx=0x7f2874008040 comp rx=0 tx=0).stop 2023-10-06T05:22:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 -- 172.21.15.31:0/3523123098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2884159500 msgr2=0x7f288415b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2884159500 0x7f288415b360 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f28840ca9f0 tx=0x7f2880017000 comp rx=0 tx=0).stop 2023-10-06T05:22:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 -- 172.21.15.31:0/3523123098 shutdown_connections 2023-10-06T05:22:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2868067050 0x7f2868069500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f288415d2d0 0x7f28840b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2884159500 0x7f288415b360 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 --2- 172.21.15.31:0/3523123098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28841598d0 0x7f28840b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 -- 172.21.15.31:0/3523123098 >> 172.21.15.31:0/3523123098 conn(0x7f288406ec90 msgr2=0x7f28840cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 -- 172.21.15.31:0/3523123098 shutdown_connections 2023-10-06T05:22:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.376+0000 7f288bfff700 1 -- 172.21.15.31:0/3523123098 wait complete. 2023-10-06T05:22:30.395 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:22:30.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:22:30.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.593+0000 7f35451ff700 1 Processor -- start 2023-10-06T05:22:30.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.593+0000 7f35451ff700 1 -- start start 2023-10-06T05:22:30.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.594+0000 7f35451ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f354015a7f0 0x7f35401645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.594+0000 7f35451ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3540069750 con 0x7f35400c7590 2023-10-06T05:22:30.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.594+0000 7f35451ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3540069a30 con 0x7f3540159410 2023-10-06T05:22:30.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.594+0000 7f35451ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3540069d10 con 0x7f354015a7f0 2023-10-06T05:22:30.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.594+0000 7f353f59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f354015a7f0 0x7f35401645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.594+0000 7f353e59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f35400c7590 0x7f354006adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41976/0 (socket says 172.21.15.31:41976) 2023-10-06T05:22:30.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.594+0000 7f353e59c700 1 -- 172.21.15.31:0/3357995771 learned_addr learned my addr 172.21.15.31:0/3357995771 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:30.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.595+0000 7f353f59e700 1 -- 172.21.15.31:0/3357995771 >> v1:172.21.15.62:6789/0 conn(0x7f3540159410 legacy=0x7f35400c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.595+0000 7f353f59e700 1 -- 172.21.15.31:0/3357995771 >> v1:172.21.15.31:6789/0 conn(0x7f35400c7590 legacy=0x7f354006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.595+0000 7f353f59e700 1 -- 172.21.15.31:0/3357995771 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3540069ff0 con 0x7f354015a7f0 2023-10-06T05:22:30.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.595+0000 7f353f59e700 1 --2- 172.21.15.31:0/3357995771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f354015a7f0 0x7f35401645e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3538009e60 tx=0x7f3538009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f5ef564a91290d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.595+0000 7f353dd9b700 1 -- 172.21.15.31:0/3357995771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800e070 con 0x7f354015a7f0 2023-10-06T05:22:30.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.595+0000 7f353dd9b700 1 -- 172.21.15.31:0/3357995771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353800bc30 con 0x7f354015a7f0 2023-10-06T05:22:30.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.595+0000 7f353dd9b700 1 -- 172.21.15.31:0/3357995771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538012430 con 0x7f354015a7f0 2023-10-06T05:22:30.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.596+0000 7f35451ff700 1 -- 172.21.15.31:0/3357995771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f354015a7f0 msgr2=0x7f35401645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.596+0000 7f35451ff700 1 --2- 172.21.15.31:0/3357995771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f354015a7f0 0x7f35401645e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3538009e60 tx=0x7f3538009780 comp rx=0 tx=0).stop 2023-10-06T05:22:30.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.596+0000 7f35451ff700 1 -- 172.21.15.31:0/3357995771 shutdown_connections 2023-10-06T05:22:30.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.596+0000 7f35451ff700 1 --2- 172.21.15.31:0/3357995771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f354015a7f0 0x7f35401645e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.596+0000 7f35451ff700 1 -- 172.21.15.31:0/3357995771 >> 172.21.15.31:0/3357995771 conn(0x7f35400c9350 msgr2=0x7f35400c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:30.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.596+0000 7f35451ff700 1 -- 172.21.15.31:0/3357995771 shutdown_connections 2023-10-06T05:22:30.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.596+0000 7f35451ff700 1 -- 172.21.15.31:0/3357995771 wait complete. 2023-10-06T05:22:30.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.598+0000 7f35451ff700 1 Processor -- start 2023-10-06T05:22:30.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.598+0000 7f35451ff700 1 -- start start 2023-10-06T05:22:30.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f35451ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35400c7590 0x7f354014d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f35451ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3540159410 0x7f3540156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f35451ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f354015a7f0 0x7f354015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f35451ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354014d7a0 con 0x7f35400c7590 2023-10-06T05:22:30.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f35451ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3540152a60 con 0x7f354015a7f0 2023-10-06T05:22:30.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f35451ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3540155680 con 0x7f3540159410 2023-10-06T05:22:30.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35400c7590 0x7f354014d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35400c7590 0x7f354014d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43018/0 (socket says 172.21.15.31:43018) 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353ed9d700 1 -- 172.21.15.31:0/352950685 learned_addr learned my addr 172.21.15.31:0/352950685 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353e59c700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3540159410 0x7f3540156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353f59e700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f354015a7f0 0x7f354015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353ed9d700 1 -- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3540159410 msgr2=0x7f3540156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353ed9d700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3540159410 0x7f3540156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353ed9d700 1 -- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f354015a7f0 msgr2=0x7f354015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353ed9d700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f354015a7f0 0x7f354015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353ed9d700 1 -- 172.21.15.31:0/352950685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3538009a20 con 0x7f35400c7590 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.599+0000 7f353f59e700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f354015a7f0 0x7f354015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.600+0000 7f353ed9d700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35400c7590 0x7f354014d260 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f353000c7c0 tx=0x7f3530007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.600+0000 7f3526ffd700 1 -- 172.21.15.31:0/352950685 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530011590 con 0x7f35400c7590 2023-10-06T05:22:30.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.600+0000 7f3526ffd700 1 -- 172.21.15.31:0/352950685 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3530011710 con 0x7f35400c7590 2023-10-06T05:22:30.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.600+0000 7f3526ffd700 1 -- 172.21.15.31:0/352950685 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530011ad0 con 0x7f35400c7590 2023-10-06T05:22:30.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.600+0000 7f35451ff700 1 -- 172.21.15.31:0/352950685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3540157160 con 0x7f35400c7590 2023-10-06T05:22:30.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.600+0000 7f35451ff700 1 -- 172.21.15.31:0/352950685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3540155030 con 0x7f35400c7590 2023-10-06T05:22:30.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.601+0000 7f35451ff700 1 -- 172.21.15.31:0/352950685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354004c380 con 0x7f35400c7590 2023-10-06T05:22:30.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.602+0000 7f3526ffd700 1 -- 172.21.15.31:0/352950685 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f353001b3b0 con 0x7f35400c7590 2023-10-06T05:22:30.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.603+0000 7f3526ffd700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f352c067050 0x7f352c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:30.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.603+0000 7f3526ffd700 1 -- 172.21.15.31:0/352950685 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f35300839d0 con 0x7f35400c7590 2023-10-06T05:22:30.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.604+0000 7f353e59c700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f352c067050 0x7f352c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:30.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.604+0000 7f353e59c700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f352c067050 0x7f352c069500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35400c6d30 tx=0x7f3534007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:30.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.607+0000 7f3526ffd700 1 -- 172.21.15.31:0/352950685 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3530050c80 con 0x7f35400c7590 2023-10-06T05:22:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:30.810+0000 7f35451ff700 1 -- 172.21.15.31:0/352950685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3540063620 con 0x7f35400c7590 2023-10-06T05:22:31.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.122+0000 7f3526ffd700 1 -- 172.21.15.31:0/352950685 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v98) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3530017070 con 0x7f35400c7590 2023-10-06T05:22:31.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.126+0000 7f35451ff700 1 -- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f352c067050 msgr2=0x7f352c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.126+0000 7f35451ff700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f352c067050 0x7f352c069500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35400c6d30 tx=0x7f3534007eb0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.126+0000 7f35451ff700 1 -- 172.21.15.31:0/352950685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35400c7590 msgr2=0x7f354014d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.127+0000 7f35451ff700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35400c7590 0x7f354014d260 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f353000c7c0 tx=0x7f3530007f40 comp rx=0 tx=0).stop 2023-10-06T05:22:31.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.128+0000 7f35451ff700 1 -- 172.21.15.31:0/352950685 shutdown_connections 2023-10-06T05:22:31.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.128+0000 7f35451ff700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f352c067050 0x7f352c069500 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.128+0000 7f35451ff700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f354015a7f0 0x7f354015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.128+0000 7f35451ff700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35400c7590 0x7f354014d260 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.128+0000 7f35451ff700 1 --2- 172.21.15.31:0/352950685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3540159410 0x7f3540156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.128+0000 7f35451ff700 1 -- 172.21.15.31:0/352950685 >> 172.21.15.31:0/352950685 conn(0x7f35400c9350 msgr2=0x7f354006b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:31.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.129+0000 7f35451ff700 1 -- 172.21.15.31:0/352950685 shutdown_connections 2023-10-06T05:22:31.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.129+0000 7f35451ff700 1 -- 172.21.15.31:0/352950685 wait complete. 2023-10-06T05:22:31.132 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:22:31.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:22:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.350+0000 7f69f0fff700 1 Processor -- start 2023-10-06T05:22:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.350+0000 7f69f0fff700 1 -- start start 2023-10-06T05:22:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.351+0000 7f69f0fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69ec15ca40 0x7f69ec068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.351+0000 7f69f0fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69ec0c8f40 0x7f69ec0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.351+0000 7f69f0fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec068da0 con 0x7f69ec06a300 2023-10-06T05:22:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.351+0000 7f69f0fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec0cabc0 con 0x7f69ec0c8f40 2023-10-06T05:22:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.351+0000 7f69f0fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec068bb0 con 0x7f69ec15ca40 2023-10-06T05:22:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.352+0000 7f69e9d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f69ec06a300 0x7f69ec0c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41988/0 (socket says 172.21.15.31:41988) 2023-10-06T05:22:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.352+0000 7f69e9d9b700 1 -- 172.21.15.31:0/3520779225 learned_addr learned my addr 172.21.15.31:0/3520779225 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.352+0000 7f69ead9d700 1 --2- 172.21.15.31:0/3520779225 >> v2:172.21.15.62:3300/0 conn(0x7f69ec0c8f40 0x7f69ec0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.352+0000 7f69ea59c700 1 --2- 172.21.15.31:0/3520779225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69ec15ca40 0x7f69ec068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.352+0000 7f69e959a700 1 -- 172.21.15.31:0/3520779225 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166333494 0 0) 0x7f69ec068da0 con 0x7f69ec06a300 2023-10-06T05:22:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.352+0000 7f69e959a700 1 -- 172.21.15.31:0/3520779225 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d4003570 con 0x7f69ec06a300 2023-10-06T05:22:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.352+0000 7f69ead9d700 1 -- 172.21.15.31:0/3520779225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69ec15ca40 msgr2=0x7f69ec068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.352+0000 7f69ead9d700 1 --2- 172.21.15.31:0/3520779225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69ec15ca40 0x7f69ec068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.352+0000 7f69ead9d700 1 -- 172.21.15.31:0/3520779225 >> v1:172.21.15.31:6789/0 conn(0x7f69ec06a300 legacy=0x7f69ec0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.352+0000 7f69ead9d700 1 -- 172.21.15.31:0/3520779225 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec069080 con 0x7f69ec0c8f40 2023-10-06T05:22:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.353+0000 7f69ead9d700 1 --2- 172.21.15.31:0/3520779225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec0c8f40 0x7f69ec0adde0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009e60 tx=0x7f69e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=445a1ffd606d02d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.353+0000 7f69e959a700 1 -- 172.21.15.31:0/3520779225 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e400e070 con 0x7f69ec0c8f40 2023-10-06T05:22:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.353+0000 7f69e959a700 1 -- 172.21.15.31:0/3520779225 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e400bab0 con 0x7f69ec0c8f40 2023-10-06T05:22:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.353+0000 7f69e959a700 1 -- 172.21.15.31:0/3520779225 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e400be70 con 0x7f69ec0c8f40 2023-10-06T05:22:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.354+0000 7f69f0fff700 1 -- 172.21.15.31:0/3520779225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec0c8f40 msgr2=0x7f69ec0adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.354+0000 7f69f0fff700 1 --2- 172.21.15.31:0/3520779225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec0c8f40 0x7f69ec0adde0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009e60 tx=0x7f69e4009780 comp rx=0 tx=0).stop 2023-10-06T05:22:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.354+0000 7f69f0fff700 1 -- 172.21.15.31:0/3520779225 shutdown_connections 2023-10-06T05:22:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.354+0000 7f69f0fff700 1 --2- 172.21.15.31:0/3520779225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec0c8f40 0x7f69ec0adde0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.354+0000 7f69f0fff700 1 --2- 172.21.15.31:0/3520779225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69ec15ca40 0x7f69ec068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.354+0000 7f69f0fff700 1 -- 172.21.15.31:0/3520779225 >> 172.21.15.31:0/3520779225 conn(0x7f69ec0ce290 msgr2=0x7f69ec0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.354+0000 7f69f0fff700 1 -- 172.21.15.31:0/3520779225 shutdown_connections 2023-10-06T05:22:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.354+0000 7f69f0fff700 1 -- 172.21.15.31:0/3520779225 wait complete. 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.355+0000 7f69f0fff700 1 Processor -- start 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.355+0000 7f69f0fff700 1 -- start start 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.356+0000 7f69f0fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69ec06a300 0x7f69ec146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.356+0000 7f69f0fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec0c8f40 0x7f69ec15e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.356+0000 7f69f0fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69ec15ca40 0x7f69ec0c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.356+0000 7f69f0fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0c6450 con 0x7f69ec15ca40 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.356+0000 7f69f0fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0c5840 con 0x7f69ec0c8f40 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.356+0000 7f69f0fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0c5b20 con 0x7f69ec06a300 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69ead9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69ec15ca40 0x7f69ec0c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69ead9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69ec15ca40 0x7f69ec0c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43024/0 (socket says 172.21.15.31:43024) 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69ead9d700 1 -- 172.21.15.31:0/1420094636 learned_addr learned my addr 172.21.15.31:0/1420094636 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69e9d9b700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec0c8f40 0x7f69ec15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69ea59c700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69ec06a300 0x7f69ec146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69ead9d700 1 -- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69ec06a300 msgr2=0x7f69ec146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69ead9d700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69ec06a300 0x7f69ec146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69ead9d700 1 -- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec0c8f40 msgr2=0x7f69ec15e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69ead9d700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec0c8f40 0x7f69ec15e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69ead9d700 1 -- 172.21.15.31:0/1420094636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e4009a20 con 0x7f69ec15ca40 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.357+0000 7f69ead9d700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69ec15ca40 0x7f69ec0c68e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f69e4013000 tx=0x7f69e4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.358+0000 7f69da7fc700 1 -- 172.21.15.31:0/1420094636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4012e10 con 0x7f69ec15ca40 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.358+0000 7f69da7fc700 1 -- 172.21.15.31:0/1420094636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e4027470 con 0x7f69ec15ca40 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.358+0000 7f69da7fc700 1 -- 172.21.15.31:0/1420094636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4027830 con 0x7f69ec15ca40 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.358+0000 7f69f0fff700 1 -- 172.21.15.31:0/1420094636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec0c5e00 con 0x7f69ec15ca40 2023-10-06T05:22:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.358+0000 7f69f0fff700 1 -- 172.21.15.31:0/1420094636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec146b80 con 0x7f69ec15ca40 2023-10-06T05:22:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.359+0000 7f69cbfff700 1 -- 172.21.15.31:0/1420094636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c40052b0 con 0x7f69ec15ca40 2023-10-06T05:22:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.361+0000 7f69da7fc700 1 -- 172.21.15.31:0/1420094636 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f69e4024070 con 0x7f69ec15ca40 2023-10-06T05:22:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.362+0000 7f69da7fc700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f69cc067050 0x7f69cc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.362+0000 7f69da7fc700 1 -- 172.21.15.31:0/1420094636 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f69e40873f0 con 0x7f69ec15ca40 2023-10-06T05:22:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.362+0000 7f69ea59c700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f69cc067050 0x7f69cc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.363+0000 7f69ea59c700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f69cc067050 0x7f69cc069500 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009db0 tx=0x7f69dc0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.366+0000 7f69da7fc700 1 -- 172.21.15.31:0/1420094636 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69e4054650 con 0x7f69ec15ca40 2023-10-06T05:22:31.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:31.567+0000 7f69cbfff700 1 -- 172.21.15.31:0/1420094636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f69c4005560 con 0x7f69ec15ca40 2023-10-06T05:22:31.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:31.604 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:22:32.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.131+0000 7f69da7fc700 1 -- 172.21.15.31:0/1420094636 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v99) v1 ==== 170+0+0 (secure 0 0 0) 0x7f69e4057c70 con 0x7f69ec15ca40 2023-10-06T05:22:32.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.137+0000 7f69cbfff700 1 -- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f69cc067050 msgr2=0x7f69cc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.137+0000 7f69cbfff700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f69cc067050 0x7f69cc069500 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009db0 tx=0x7f69dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.137+0000 7f69cbfff700 1 -- 172.21.15.31:0/1420094636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69ec15ca40 msgr2=0x7f69ec0c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.137+0000 7f69cbfff700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69ec15ca40 0x7f69ec0c68e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f69e4013000 tx=0x7f69e4019000 comp rx=0 tx=0).stop 2023-10-06T05:22:32.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.138+0000 7f69cbfff700 1 -- 172.21.15.31:0/1420094636 shutdown_connections 2023-10-06T05:22:32.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.138+0000 7f69cbfff700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f69cc067050 0x7f69cc069500 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.138+0000 7f69cbfff700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69ec15ca40 0x7f69ec0c68e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.138+0000 7f69cbfff700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69ec06a300 0x7f69ec146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.138+0000 7f69cbfff700 1 --2- 172.21.15.31:0/1420094636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec0c8f40 0x7f69ec15e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.138+0000 7f69cbfff700 1 -- 172.21.15.31:0/1420094636 >> 172.21.15.31:0/1420094636 conn(0x7f69ec0ce290 msgr2=0x7f69ec0cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:32.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.141+0000 7f69cbfff700 1 -- 172.21.15.31:0/1420094636 shutdown_connections 2023-10-06T05:22:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.142+0000 7f69cbfff700 1 -- 172.21.15.31:0/1420094636 wait complete. 2023-10-06T05:22:32.156 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:22:32.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:22:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.370+0000 7fe5f67ff700 1 Processor -- start 2023-10-06T05:22:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.370+0000 7fe5f67ff700 1 -- start start 2023-10-06T05:22:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.371+0000 7fe5f67ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00b10c0 0x7fe5f00cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.371+0000 7fe5f67ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f0069380 con 0x7fe5f00cc1d0 2023-10-06T05:22:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.372+0000 7fe5f67ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f0068770 con 0x7fe5f00caeb0 2023-10-06T05:22:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.372+0000 7fe5f67ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0068a50 con 0x7fe5f00b10c0 2023-10-06T05:22:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.372+0000 7fe5f4d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe5f00cc1d0 0x7fe5f00afea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42004/0 (socket says 172.21.15.31:42004) 2023-10-06T05:22:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.372+0000 7fe5f4d9c700 1 -- 172.21.15.31:0/823452627 learned_addr learned my addr 172.21.15.31:0/823452627 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.372+0000 7fe5ef7fe700 1 --2- 172.21.15.31:0/823452627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00b10c0 0x7fe5f00cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.372+0000 7fe5eeffd700 1 -- 172.21.15.31:0/823452627 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448803688 0 0) 0x7fe5f0069380 con 0x7fe5f00cc1d0 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.372+0000 7fe5eeffd700 1 -- 172.21.15.31:0/823452627 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5d8003570 con 0x7fe5f00cc1d0 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.372+0000 7fe5eeffd700 1 -- 172.21.15.31:0/823452627 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358105074 0 0) 0x7fe5f0068770 con 0x7fe5f00caeb0 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.372+0000 7fe5eeffd700 1 -- 172.21.15.31:0/823452627 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f0069380 con 0x7fe5f00caeb0 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.373+0000 7fe5ef7fe700 1 -- 172.21.15.31:0/823452627 >> v1:172.21.15.62:6789/0 conn(0x7fe5f00caeb0 legacy=0x7fe5f0159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.373+0000 7fe5ef7fe700 1 -- 172.21.15.31:0/823452627 >> v1:172.21.15.31:6789/0 conn(0x7fe5f00cc1d0 legacy=0x7fe5f00afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.373+0000 7fe5ef7fe700 1 -- 172.21.15.31:0/823452627 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f0068d30 con 0x7fe5f00b10c0 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.373+0000 7fe5ef7fe700 1 --2- 172.21.15.31:0/823452627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00b10c0 0x7fe5f00cd0f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009910 tx=0x7fe5e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b729e7af63d1b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.373+0000 7fe5eeffd700 1 -- 172.21.15.31:0/823452627 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4011630 con 0x7fe5f00b10c0 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.373+0000 7fe5eeffd700 1 -- 172.21.15.31:0/823452627 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4011790 con 0x7fe5f00b10c0 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.373+0000 7fe5eeffd700 1 -- 172.21.15.31:0/823452627 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4011b30 con 0x7fe5f00b10c0 2023-10-06T05:22:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.374+0000 7fe5f67ff700 1 -- 172.21.15.31:0/823452627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00b10c0 msgr2=0x7fe5f00cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.374+0000 7fe5f67ff700 1 --2- 172.21.15.31:0/823452627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00b10c0 0x7fe5f00cd0f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009910 tx=0x7fe5e40091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.374+0000 7fe5f67ff700 1 -- 172.21.15.31:0/823452627 shutdown_connections 2023-10-06T05:22:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.374+0000 7fe5f67ff700 1 --2- 172.21.15.31:0/823452627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00b10c0 0x7fe5f00cd0f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.374+0000 7fe5f67ff700 1 -- 172.21.15.31:0/823452627 >> 172.21.15.31:0/823452627 conn(0x7fe5f0097550 msgr2=0x7fe5f00639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.374+0000 7fe5f67ff700 1 -- 172.21.15.31:0/823452627 shutdown_connections 2023-10-06T05:22:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.374+0000 7fe5f67ff700 1 -- 172.21.15.31:0/823452627 wait complete. 2023-10-06T05:22:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.375+0000 7fe5f67ff700 1 Processor -- start 2023-10-06T05:22:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.375+0000 7fe5f67ff700 1 -- start start 2023-10-06T05:22:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.376+0000 7fe5f67ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f00b10c0 0x7fe5f0158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.376+0000 7fe5f67ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00caeb0 0x7fe5f00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.376+0000 7fe5f67ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f00cc1d0 0x7fe5f00b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.376+0000 7fe5f67ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00c7870 con 0x7fe5f00b10c0 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.376+0000 7fe5f67ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f016a390 con 0x7fe5f00cc1d0 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.376+0000 7fe5f67ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0169a60 con 0x7fe5f00caeb0 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.376+0000 7fe5effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f00b10c0 0x7fe5f0158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f00b10c0 0x7fe5f0158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43040/0 (socket says 172.21.15.31:43040) 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5effff700 1 -- 172.21.15.31:0/1532900306 learned_addr learned my addr 172.21.15.31:0/1532900306 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5ef7fe700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00caeb0 0x7fe5f00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5f4d9c700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f00cc1d0 0x7fe5f00b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5effff700 1 -- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00caeb0 msgr2=0x7fe5f00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5effff700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00caeb0 0x7fe5f00b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5effff700 1 -- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f00cc1d0 msgr2=0x7fe5f00b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5effff700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f00cc1d0 0x7fe5f00b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:32.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5effff700 1 -- 172.21.15.31:0/1532900306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e4009420 con 0x7fe5f00b10c0 2023-10-06T05:22:32.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5f4d9c700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f00cc1d0 0x7fe5f00b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:32.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5ef7fe700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00caeb0 0x7fe5f00b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:32.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5effff700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f00b10c0 0x7fe5f0158730 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000c7f0 tx=0x7fe5e0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:32.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.377+0000 7fe5d7fff700 1 -- 172.21.15.31:0/1532900306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0011680 con 0x7fe5f00b10c0 2023-10-06T05:22:32.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.378+0000 7fe5d7fff700 1 -- 172.21.15.31:0/1532900306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e00117e0 con 0x7fe5f00b10c0 2023-10-06T05:22:32.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.378+0000 7fe5d7fff700 1 -- 172.21.15.31:0/1532900306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0011ba0 con 0x7fe5f00b10c0 2023-10-06T05:22:32.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.378+0000 7fe5f67ff700 1 -- 172.21.15.31:0/1532900306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f0169d40 con 0x7fe5f00b10c0 2023-10-06T05:22:32.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.378+0000 7fe5f67ff700 1 -- 172.21.15.31:0/1532900306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f0158c70 con 0x7fe5f00b10c0 2023-10-06T05:22:32.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.379+0000 7fe5f67ff700 1 -- 172.21.15.31:0/1532900306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f00b3c00 con 0x7fe5f00b10c0 2023-10-06T05:22:32.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.380+0000 7fe5d7fff700 1 -- 172.21.15.31:0/1532900306 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe5e0012440 con 0x7fe5f00b10c0 2023-10-06T05:22:32.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.381+0000 7fe5d7fff700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe5dc067050 0x7fe5dc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:32.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.381+0000 7fe5d7fff700 1 -- 172.21.15.31:0/1532900306 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fe5e00851b0 con 0x7fe5f00b10c0 2023-10-06T05:22:32.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.382+0000 7fe5ef7fe700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe5dc067050 0x7fe5dc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:32.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.383+0000 7fe5ef7fe700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe5dc067050 0x7fe5dc069500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40091e0 tx=0x7fe5e4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.385+0000 7fe5d7fff700 1 -- 172.21.15.31:0/1532900306 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5e00521f0 con 0x7fe5f00b10c0 2023-10-06T05:22:32.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:32.636+0000 7fe5f67ff700 1 -- 172.21.15.31:0/1532900306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe5f004c380 con 0x7fe5f00b10c0 2023-10-06T05:22:33.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.134+0000 7fe5d7fff700 1 -- 172.21.15.31:0/1532900306 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe5e0055810 con 0x7fe5f00b10c0 2023-10-06T05:22:33.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.138+0000 7fe5f67ff700 1 -- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe5dc067050 msgr2=0x7fe5dc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.138+0000 7fe5f67ff700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe5dc067050 0x7fe5dc069500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40091e0 tx=0x7fe5e4017000 comp rx=0 tx=0).stop 2023-10-06T05:22:33.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.138+0000 7fe5f67ff700 1 -- 172.21.15.31:0/1532900306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f00b10c0 msgr2=0x7fe5f0158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.138+0000 7fe5f67ff700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f00b10c0 0x7fe5f0158730 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000c7f0 tx=0x7fe5e0007e00 comp rx=0 tx=0).stop 2023-10-06T05:22:33.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.139+0000 7fe5f67ff700 1 -- 172.21.15.31:0/1532900306 shutdown_connections 2023-10-06T05:22:33.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.139+0000 7fe5f67ff700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe5dc067050 0x7fe5dc069500 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.139+0000 7fe5f67ff700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f00cc1d0 0x7fe5f00b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.139+0000 7fe5f67ff700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5f00b10c0 0x7fe5f0158730 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.139+0000 7fe5f67ff700 1 --2- 172.21.15.31:0/1532900306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5f00caeb0 0x7fe5f00b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.139+0000 7fe5f67ff700 1 -- 172.21.15.31:0/1532900306 >> 172.21.15.31:0/1532900306 conn(0x7fe5f0097550 msgr2=0x7fe5f00cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.140+0000 7fe5f67ff700 1 -- 172.21.15.31:0/1532900306 shutdown_connections 2023-10-06T05:22:33.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.140+0000 7fe5f67ff700 1 -- 172.21.15.31:0/1532900306 wait complete. 2023-10-06T05:22:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 17 and data pool 18 2023-10-06T05:22:33.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.373+0000 7fce54fff700 1 Processor -- start 2023-10-06T05:22:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.373+0000 7fce54fff700 1 -- start start 2023-10-06T05:22:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.374+0000 7fce54fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5013a5b0 0x7fce5006d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.374+0000 7fce54fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5006ddc0 con 0x7fce5013a1e0 2023-10-06T05:22:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.374+0000 7fce54fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce500c71b0 con 0x7fce5006d980 2023-10-06T05:22:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.374+0000 7fce54fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500c7490 con 0x7fce5013a5b0 2023-10-06T05:22:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4e59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fce5013a1e0 0x7fce500c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42012/0 (socket says 172.21.15.31:42012) 2023-10-06T05:22:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4e59c700 1 -- 172.21.15.31:0/33111843 learned_addr learned my addr 172.21.15.31:0/33111843 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:33.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4dd9b700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5013a5b0 0x7fce5006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148705098 0 0) 0x7fce5006ddc0 con 0x7fce5013a1e0 2023-10-06T05:22:33.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce38003570 con 0x7fce5013a1e0 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858639005 0 0) 0x7fce38003570 con 0x7fce5013a1e0 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5013a5b0 msgr2=0x7fce5006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4d59a700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5013a5b0 0x7fce5006d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 >> v1:172.21.15.62:6789/0 conn(0x7fce5006d980 legacy=0x7fce50069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce500c7770 con 0x7fce5013a1e0 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.375+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fce400015f0 con 0x7fce5013a1e0 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 >> v1:172.21.15.31:6789/0 conn(0x7fce5013a1e0 legacy=0x7fce500c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4d59a700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce38003a90 0x7fce38003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4d59a700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce38004810 0x7fce38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4d59a700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5013a5b0 0x7fce380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500c7490 con 0x7fce38004810 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3800bb10 con 0x7fce38003a90 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3800bcd0 con 0x7fce5013a5b0 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4dd9b700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce38004810 0x7fce38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4e59c700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce38003a90 0x7fce38003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4ed9d700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5013a5b0 0x7fce380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4dd9b700 1 -- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5013a5b0 msgr2=0x7fce380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4dd9b700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5013a5b0 0x7fce380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4dd9b700 1 -- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce38003a90 msgr2=0x7fce38003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4dd9b700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce38003a90 0x7fce38003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4dd9b700 1 -- 172.21.15.31:0/33111843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38003570 con 0x7fce38004810 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.376+0000 7fce4ed9d700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce5013a5b0 0x7fce380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.377+0000 7fce4dd9b700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce38004810 0x7fce38004c80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fce44009c00 tx=0x7fce440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78c95095ee968a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.377+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400e710 con 0x7fce38004810 2023-10-06T05:22:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.377+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4400e870 con 0x7fce38004810 2023-10-06T05:22:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.377+0000 7fce4d59a700 1 -- 172.21.15.31:0/33111843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400ec30 con 0x7fce38004810 2023-10-06T05:22:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.377+0000 7fce54fff700 1 -- 172.21.15.31:0/33111843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce38004810 msgr2=0x7fce38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.377+0000 7fce54fff700 1 --2- 172.21.15.31:0/33111843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce38004810 0x7fce38004c80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fce44009c00 tx=0x7fce440091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.377+0000 7fce4ed9d700 1 -- 172.21.15.31:0/33111843 reap_dead start 2023-10-06T05:22:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.377+0000 7fce54fff700 1 -- 172.21.15.31:0/33111843 shutdown_connections 2023-10-06T05:22:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.377+0000 7fce54fff700 1 -- 172.21.15.31:0/33111843 >> 172.21.15.31:0/33111843 conn(0x7fce500cc0b0 msgr2=0x7fce500cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.377+0000 7fce54fff700 1 -- 172.21.15.31:0/33111843 shutdown_connections 2023-10-06T05:22:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.378+0000 7fce54fff700 1 -- 172.21.15.31:0/33111843 wait complete. 2023-10-06T05:22:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.379+0000 7fce54fff700 1 Processor -- start 2023-10-06T05:22:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.379+0000 7fce54fff700 1 -- start start 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.380+0000 7fce54fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce38004810 0x7fce50168c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.380+0000 7fce54fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5013bfb0 0x7fce500c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.380+0000 7fce54fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce500b0820 0x7fce500b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.380+0000 7fce54fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce50169140 con 0x7fce500b0820 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.380+0000 7fce54fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b11d0 con 0x7fce5013bfb0 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.380+0000 7fce54fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce50167ea0 con 0x7fce38004810 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.380+0000 7fce4ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce500b0820 0x7fce500b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.380+0000 7fce4ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce500b0820 0x7fce500b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43058/0 (socket says 172.21.15.31:43058) 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.380+0000 7fce4ed9d700 1 -- 172.21.15.31:0/2764403542 learned_addr learned my addr 172.21.15.31:0/2764403542 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce4dd9b700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5013bfb0 0x7fce500c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce4e59c700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce38004810 0x7fce50168c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce4ed9d700 1 -- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce38004810 msgr2=0x7fce50168c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce4ed9d700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce38004810 0x7fce50168c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce4ed9d700 1 -- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5013bfb0 msgr2=0x7fce500c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce4ed9d700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5013bfb0 0x7fce500c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce4ed9d700 1 -- 172.21.15.31:0/2764403542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50168180 con 0x7fce500b0820 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce4dd9b700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5013bfb0 0x7fce500c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce4e59c700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce38004810 0x7fce50168c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce4ed9d700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce500b0820 0x7fce500b0c90 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fce5006b950 tx=0x7fce48009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce3e7fc700 1 -- 172.21.15.31:0/2764403542 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48006ba0 con 0x7fce500b0820 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce3e7fc700 1 -- 172.21.15.31:0/2764403542 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48006d20 con 0x7fce500b0820 2023-10-06T05:22:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce3e7fc700 1 -- 172.21.15.31:0/2764403542 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48013880 con 0x7fce500b0820 2023-10-06T05:22:33.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.381+0000 7fce54fff700 1 -- 172.21.15.31:0/2764403542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce50168460 con 0x7fce500b0820 2023-10-06T05:22:33.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.382+0000 7fce54fff700 1 -- 172.21.15.31:0/2764403542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5013c420 con 0x7fce500b0820 2023-10-06T05:22:33.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.382+0000 7fce54fff700 1 -- 172.21.15.31:0/2764403542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce500c31f0 con 0x7fce500b0820 2023-10-06T05:22:33.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.383+0000 7fce3e7fc700 1 -- 172.21.15.31:0/2764403542 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fce48003030 con 0x7fce500b0820 2023-10-06T05:22:33.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.384+0000 7fce3e7fc700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce30067050 0x7fce30069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.385+0000 7fce3e7fc700 1 -- 172.21.15.31:0/2764403542 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce48084710 con 0x7fce500b0820 2023-10-06T05:22:33.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.385+0000 7fce4e59c700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce30067050 0x7fce30069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.386+0000 7fce4e59c700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce30067050 0x7fce30069500 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce40001720 tx=0x7fce40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.388+0000 7fce3e7fc700 1 -- 172.21.15.31:0/2764403542 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce500c31f0 con 0x7fce500b0820 2023-10-06T05:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.589+0000 7fce54fff700 1 -- 172.21.15.31:0/2764403542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce500c31f0 con 0x7fce500b0820 2023-10-06T05:22:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.590+0000 7fce3e7fc700 1 -- 172.21.15.31:0/2764403542 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16933 (secure 0 0 0) 0x7fce500c31f0 con 0x7fce500b0820 2023-10-06T05:22:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:33.592 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":100,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:22:32.637684+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.850575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.607953+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.592+0000 7fce54fff700 1 -- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce30067050 msgr2=0x7fce30069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.592+0000 7fce54fff700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce30067050 0x7fce30069500 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce40001720 tx=0x7fce40009040 comp rx=0 tx=0).stop 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.592+0000 7fce54fff700 1 -- 172.21.15.31:0/2764403542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce500b0820 msgr2=0x7fce500b0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.592+0000 7fce54fff700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce500b0820 0x7fce500b0c90 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fce5006b950 tx=0x7fce48009040 comp rx=0 tx=0).stop 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.592+0000 7fce54fff700 1 -- 172.21.15.31:0/2764403542 shutdown_connections 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.592+0000 7fce54fff700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce30067050 0x7fce30069500 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.592+0000 7fce54fff700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce500b0820 0x7fce500b0c90 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.592+0000 7fce54fff700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce38004810 0x7fce50168c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.592+0000 7fce54fff700 1 --2- 172.21.15.31:0/2764403542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce5013bfb0 0x7fce500c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.592+0000 7fce54fff700 1 -- 172.21.15.31:0/2764403542 >> 172.21.15.31:0/2764403542 conn(0x7fce500cc0b0 msgr2=0x7fce500c25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.593+0000 7fce54fff700 1 -- 172.21.15.31:0/2764403542 shutdown_connections 2023-10-06T05:22:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.593+0000 7fce54fff700 1 -- 172.21.15.31:0/2764403542 wait complete. 2023-10-06T05:22:33.612 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:33.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.806+0000 7f73e9bff700 1 Processor -- start 2023-10-06T05:22:33.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.806+0000 7f73e9bff700 1 -- start start 2023-10-06T05:22:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.807+0000 7f73e9bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f73e406ec30 0x7f73e414ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.807+0000 7f73e9bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e40c6df0 con 0x7f73e406b510 2023-10-06T05:22:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.807+0000 7f73e9bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e40c70d0 con 0x7f73e40ca770 2023-10-06T05:22:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.807+0000 7f73e9bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e40c7a70 con 0x7f73e406ec30 2023-10-06T05:22:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.808+0000 7f73e2ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f73e406b510 0x7f73e414c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42016/0 (socket says 172.21.15.31:42016) 2023-10-06T05:22:33.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.808+0000 7f73e2ffd700 1 -- 172.21.15.31:0/2379466395 learned_addr learned my addr 172.21.15.31:0/2379466395 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:33.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.808+0000 7f73e3fff700 1 --2- 172.21.15.31:0/2379466395 >> v2:172.21.15.62:3300/0 conn(0x7f73e406ec30 0x7f73e414ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.808+0000 7f73e27fc700 1 -- 172.21.15.31:0/2379466395 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881077296 0 0) 0x7f73e40c6df0 con 0x7f73e406b510 2023-10-06T05:22:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.808+0000 7f73e27fc700 1 -- 172.21.15.31:0/2379466395 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73cc004680 con 0x7f73e406b510 2023-10-06T05:22:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.808+0000 7f73e27fc700 1 -- 172.21.15.31:0/2379466395 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377047405 0 0) 0x7f73e40c70d0 con 0x7f73e40ca770 2023-10-06T05:22:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.808+0000 7f73e27fc700 1 -- 172.21.15.31:0/2379466395 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e40c6df0 con 0x7f73e40ca770 2023-10-06T05:22:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.808+0000 7f73e3fff700 1 -- 172.21.15.31:0/2379466395 >> v1:172.21.15.62:6789/0 conn(0x7f73e40ca770 legacy=0x7f73e406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.808+0000 7f73e27fc700 1 -- 172.21.15.31:0/2379466395 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 397356398 0 0) 0x7f73cc004680 con 0x7f73e406b510 2023-10-06T05:22:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.808+0000 7f73e3fff700 1 -- 172.21.15.31:0/2379466395 >> v1:172.21.15.31:6789/0 conn(0x7f73e406b510 legacy=0x7f73e414c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.809+0000 7f73e3fff700 1 -- 172.21.15.31:0/2379466395 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e40c73b0 con 0x7f73e406ec30 2023-10-06T05:22:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.809+0000 7f73e3fff700 1 --2- 172.21.15.31:0/2379466395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73e406ec30 0x7f73e414ca10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009e60 tx=0x7f73dc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552e8e2372329ebc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.809+0000 7f73e27fc700 1 -- 172.21.15.31:0/2379466395 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc00e070 con 0x7f73e406ec30 2023-10-06T05:22:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.809+0000 7f73e27fc700 1 -- 172.21.15.31:0/2379466395 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc00b7a0 con 0x7f73e406ec30 2023-10-06T05:22:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.809+0000 7f73e27fc700 1 -- 172.21.15.31:0/2379466395 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8001470 con 0x7f73e406ec30 2023-10-06T05:22:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.810+0000 7f73e9bff700 1 -- 172.21.15.31:0/2379466395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73e406ec30 msgr2=0x7f73e414ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.810+0000 7f73e9bff700 1 --2- 172.21.15.31:0/2379466395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73e406ec30 0x7f73e414ca10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009e60 tx=0x7f73dc009780 comp rx=0 tx=0).stop 2023-10-06T05:22:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.810+0000 7f73e9bff700 1 -- 172.21.15.31:0/2379466395 shutdown_connections 2023-10-06T05:22:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.810+0000 7f73e9bff700 1 --2- 172.21.15.31:0/2379466395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73e406ec30 0x7f73e414ca10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.810+0000 7f73e9bff700 1 -- 172.21.15.31:0/2379466395 >> 172.21.15.31:0/2379466395 conn(0x7f73e40cc0b0 msgr2=0x7f73e40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.810+0000 7f73e9bff700 1 -- 172.21.15.31:0/2379466395 shutdown_connections 2023-10-06T05:22:33.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.810+0000 7f73e9bff700 1 -- 172.21.15.31:0/2379466395 wait complete. 2023-10-06T05:22:33.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.811+0000 7f73e9bff700 1 Processor -- start 2023-10-06T05:22:33.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.811+0000 7f73e9bff700 1 -- start start 2023-10-06T05:22:33.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.812+0000 7f73e9bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e406b510 0x7f73e40ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.812+0000 7f73e9bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e406ec30 0x7f73e40ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.812+0000 7f73e9bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73e40ca770 0x7f73e4146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.812+0000 7f73e9bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40c3160 con 0x7f73e406b510 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.812+0000 7f73e9bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e4148090 con 0x7f73e40ca770 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.812+0000 7f73e9bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e4148650 con 0x7f73e406ec30 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e406b510 0x7f73e40ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e406b510 0x7f73e40ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43064/0 (socket says 172.21.15.31:43064) 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e37fe700 1 -- 172.21.15.31:0/1004495552 learned_addr learned my addr 172.21.15.31:0/1004495552 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e2ffd700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e406ec30 0x7f73e40ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e3fff700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73e40ca770 0x7f73e4146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e37fe700 1 -- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e406ec30 msgr2=0x7f73e40ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e37fe700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e406ec30 0x7f73e40ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e37fe700 1 -- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73e40ca770 msgr2=0x7f73e4146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e37fe700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73e40ca770 0x7f73e4146060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e37fe700 1 -- 172.21.15.31:0/1004495552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc009a20 con 0x7f73e406b510 2023-10-06T05:22:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e2ffd700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e406ec30 0x7f73e40ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.813+0000 7f73e37fe700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e406b510 0x7f73e40ae370 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f73d400c560 tx=0x7f73d4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.814+0000 7f73cb7fe700 1 -- 172.21.15.31:0/1004495552 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d4010510 con 0x7f73e406b510 2023-10-06T05:22:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.814+0000 7f73cb7fe700 1 -- 172.21.15.31:0/1004495552 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d4010670 con 0x7f73e406b510 2023-10-06T05:22:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.814+0000 7f73cb7fe700 1 -- 172.21.15.31:0/1004495552 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d4010a30 con 0x7f73e406b510 2023-10-06T05:22:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.814+0000 7f73e9bff700 1 -- 172.21.15.31:0/1004495552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e40aca00 con 0x7f73e406b510 2023-10-06T05:22:33.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.814+0000 7f73e9bff700 1 -- 172.21.15.31:0/1004495552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e41482e0 con 0x7f73e406b510 2023-10-06T05:22:33.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.815+0000 7f73e9bff700 1 -- 172.21.15.31:0/1004495552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e40acf00 con 0x7f73e406b510 2023-10-06T05:22:33.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.816+0000 7f73cb7fe700 1 -- 172.21.15.31:0/1004495552 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f73d4002050 con 0x7f73e406b510 2023-10-06T05:22:33.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.817+0000 7f73cb7fe700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73d0067050 0x7f73d0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:33.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.817+0000 7f73cb7fe700 1 -- 172.21.15.31:0/1004495552 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f73d4084230 con 0x7f73e406b510 2023-10-06T05:22:33.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.817+0000 7f73e2ffd700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73d0067050 0x7f73d0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:33.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.818+0000 7f73e2ffd700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73d0067050 0x7f73d0069500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f73e4147fa0 tx=0x7f73d8000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:33.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:33.821+0000 7f73cb7fe700 1 -- 172.21.15.31:0/1004495552 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73d4051190 con 0x7f73e406b510 2023-10-06T05:22:34.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.022+0000 7f73e9bff700 1 -- 172.21.15.31:0/1004495552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73e404c380 con 0x7f73e406b510 2023-10-06T05:22:34.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.023+0000 7f73cb7fe700 1 -- 172.21.15.31:0/1004495552 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f73d40547b0 con 0x7f73e406b510 2023-10-06T05:22:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:34.025 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":100,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:22:32.637684+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.850575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.607953+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 -- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73d0067050 msgr2=0x7f73d0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73d0067050 0x7f73d0069500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f73e4147fa0 tx=0x7f73d8000bc0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 -- 172.21.15.31:0/1004495552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e406b510 msgr2=0x7f73e40ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e406b510 0x7f73e40ae370 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f73d400c560 tx=0x7f73d4007e00 comp rx=0 tx=0).stop 2023-10-06T05:22:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 -- 172.21.15.31:0/1004495552 shutdown_connections 2023-10-06T05:22:34.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73d0067050 0x7f73d0069500 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73e40ca770 0x7f73e4146060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e406b510 0x7f73e40ae370 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 --2- 172.21.15.31:0/1004495552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73e406ec30 0x7f73e40ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 -- 172.21.15.31:0/1004495552 >> 172.21.15.31:0/1004495552 conn(0x7f73e40cc0b0 msgr2=0x7f73e4158950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:34.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 -- 172.21.15.31:0/1004495552 shutdown_connections 2023-10-06T05:22:34.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.025+0000 7f73e9bff700 1 -- 172.21.15.31:0/1004495552 wait complete. 2023-10-06T05:22:34.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:22:34.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.244+0000 7f9840bff700 1 Processor -- start 2023-10-06T05:22:34.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.245+0000 7f9840bff700 1 -- start start 2023-10-06T05:22:34.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.246+0000 7f9840bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f983c159560 0x7f983c0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.246+0000 7f9840bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 0x7f983c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.246+0000 7f9840bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c15d2d0 con 0x7f983c159930 2023-10-06T05:22:34.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.246+0000 7f9840bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983c15d5b0 con 0x7f983c159560 2023-10-06T05:22:34.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.246+0000 7f9840bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c15d890 con 0x7f983c0b2b80 2023-10-06T05:22:34.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.246+0000 7f983a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f983c159560 0x7f983c0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.246+0000 7f983a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f983c159560 0x7f983c0cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43078/0 (socket says 172.21.15.31:43078) 2023-10-06T05:22:34.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.246+0000 7f983a59c700 1 -- 172.21.15.31:0/3169336545 learned_addr learned my addr 172.21.15.31:0/3169336545 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:34.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.246+0000 7f983ad9d700 1 --2- 172.21.15.31:0/3169336545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 0x7f983c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.246+0000 7f983a59c700 1 -- 172.21.15.31:0/3169336545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 msgr2=0x7f983c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:34.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.247+0000 7f983a59c700 1 --2- 172.21.15.31:0/3169336545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 0x7f983c0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.247+0000 7f983a59c700 1 -- 172.21.15.31:0/3169336545 >> v1:172.21.15.62:6789/0 conn(0x7f983c159930 legacy=0x7f983c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.247+0000 7f983a59c700 1 -- 172.21.15.31:0/3169336545 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c15db70 con 0x7f983c159560 2023-10-06T05:22:34.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.247+0000 7f983ad9d700 1 --2- 172.21.15.31:0/3169336545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 0x7f983c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:34.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.247+0000 7f983a59c700 1 --2- 172.21.15.31:0/3169336545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159560 0x7f983c0cce70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f982c0099b0 tx=0x7f982c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453870f3e41e8e7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.247+0000 7f983959a700 1 -- 172.21.15.31:0/3169336545 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c00b820 con 0x7f983c159560 2023-10-06T05:22:34.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.247+0000 7f983959a700 1 -- 172.21.15.31:0/3169336545 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c00b980 con 0x7f983c159560 2023-10-06T05:22:34.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.247+0000 7f983959a700 1 -- 172.21.15.31:0/3169336545 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c00bd20 con 0x7f983c159560 2023-10-06T05:22:34.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.248+0000 7f9840bff700 1 -- 172.21.15.31:0/3169336545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159560 msgr2=0x7f983c0cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.248+0000 7f9840bff700 1 --2- 172.21.15.31:0/3169336545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159560 0x7f983c0cce70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f982c0099b0 tx=0x7f982c009250 comp rx=0 tx=0).stop 2023-10-06T05:22:34.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.248+0000 7f9840bff700 1 -- 172.21.15.31:0/3169336545 shutdown_connections 2023-10-06T05:22:34.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.248+0000 7f9840bff700 1 --2- 172.21.15.31:0/3169336545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 0x7f983c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.248+0000 7f9840bff700 1 --2- 172.21.15.31:0/3169336545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159560 0x7f983c0cce70 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.248+0000 7f9840bff700 1 -- 172.21.15.31:0/3169336545 >> 172.21.15.31:0/3169336545 conn(0x7f983c06ec90 msgr2=0x7f983c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:34.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.248+0000 7f9840bff700 1 -- 172.21.15.31:0/3169336545 shutdown_connections 2023-10-06T05:22:34.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.248+0000 7f9840bff700 1 -- 172.21.15.31:0/3169336545 wait complete. 2023-10-06T05:22:34.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.249+0000 7f9840bff700 1 Processor -- start 2023-10-06T05:22:34.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.249+0000 7f9840bff700 1 -- start start 2023-10-06T05:22:34.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.250+0000 7f9840bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 0x7f983c147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.250+0000 7f9840bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f983c159560 0x7f983c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.250+0000 7f9840bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159930 0x7f983c0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.250+0000 7f9840bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0d1130 con 0x7f983c159930 2023-10-06T05:22:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.250+0000 7f9840bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0d0090 con 0x7f983c159560 2023-10-06T05:22:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.250+0000 7f9840bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0d0370 con 0x7f983c0b2b80 2023-10-06T05:22:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f983ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159930 0x7f983c0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f983ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159930 0x7f983c0b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43092/0 (socket says 172.21.15.31:43092) 2023-10-06T05:22:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f983ad9d700 1 -- 172.21.15.31:0/3544945568 learned_addr learned my addr 172.21.15.31:0/3544945568 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f983a59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 0x7f983c147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f9839d9b700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f983c159560 0x7f983c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f983ad9d700 1 -- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 msgr2=0x7f983c147820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f983ad9d700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 0x7f983c147820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f983ad9d700 1 -- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f983c159560 msgr2=0x7f983c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:34.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f983ad9d700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f983c159560 0x7f983c0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:34.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f983ad9d700 1 -- 172.21.15.31:0/3544945568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c0094f0 con 0x7f983c159930 2023-10-06T05:22:34.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.251+0000 7f983ad9d700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159930 0x7f983c0b3ba0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f983400a380 tx=0x7f9834007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.252+0000 7f982a7fc700 1 -- 172.21.15.31:0/3544945568 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834015440 con 0x7f983c159930 2023-10-06T05:22:34.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.252+0000 7f982a7fc700 1 -- 172.21.15.31:0/3544945568 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98340155a0 con 0x7f983c159930 2023-10-06T05:22:34.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.252+0000 7f9840bff700 1 -- 172.21.15.31:0/3544945568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c0d0650 con 0x7f983c159930 2023-10-06T05:22:34.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.252+0000 7f982a7fc700 1 -- 172.21.15.31:0/3544945568 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834015960 con 0x7f983c159930 2023-10-06T05:22:34.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.252+0000 7f9840bff700 1 -- 172.21.15.31:0/3544945568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c0d0c10 con 0x7f983c159930 2023-10-06T05:22:34.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.253+0000 7f9840bff700 1 -- 172.21.15.31:0/3544945568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983c0d0e30 con 0x7f983c159930 2023-10-06T05:22:34.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.254+0000 7f982a7fc700 1 -- 172.21.15.31:0/3544945568 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f983400d440 con 0x7f983c159930 2023-10-06T05:22:34.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.254+0000 7f982a7fc700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f981c067050 0x7f981c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:34.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.255+0000 7f982a7fc700 1 -- 172.21.15.31:0/3544945568 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98340853a0 con 0x7f983c159930 2023-10-06T05:22:34.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.255+0000 7f983a59c700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f981c067050 0x7f981c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:34.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.256+0000 7f983a59c700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f981c067050 0x7f981c069500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f982c0099b0 tx=0x7f982c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:34.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.259+0000 7f982a7fc700 1 -- 172.21.15.31:0/3544945568 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9834052300 con 0x7f983c159930 2023-10-06T05:22:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:34.508+0000 7f9840bff700 1 -- 172.21.15.31:0/3544945568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f983c04c380 con 0x7f983c159930 2023-10-06T05:22:35.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.158+0000 7f982a7fc700 1 -- 172.21.15.31:0/3544945568 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9834055920 con 0x7f983c159930 2023-10-06T05:22:35.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 -- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f981c067050 msgr2=0x7f981c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f981c067050 0x7f981c069500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f982c0099b0 tx=0x7f982c016000 comp rx=0 tx=0).stop 2023-10-06T05:22:35.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 -- 172.21.15.31:0/3544945568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159930 msgr2=0x7f983c0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159930 0x7f983c0b3ba0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f983400a380 tx=0x7f9834007900 comp rx=0 tx=0).stop 2023-10-06T05:22:35.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 -- 172.21.15.31:0/3544945568 shutdown_connections 2023-10-06T05:22:35.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f981c067050 0x7f981c069500 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f983c159930 0x7f983c0b3ba0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f983c0b2b80 0x7f983c147820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 --2- 172.21.15.31:0/3544945568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f983c159560 0x7f983c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 -- 172.21.15.31:0/3544945568 >> 172.21.15.31:0/3544945568 conn(0x7f983c06ec90 msgr2=0x7f983c15a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:35.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 -- 172.21.15.31:0/3544945568 shutdown_connections 2023-10-06T05:22:35.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.161+0000 7f9840bff700 1 -- 172.21.15.31:0/3544945568 wait complete. 2023-10-06T05:22:35.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:22:35.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.384+0000 7f91f77ff700 1 Processor -- start 2023-10-06T05:22:35.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.384+0000 7f91f77ff700 1 -- start start 2023-10-06T05:22:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.386+0000 7f91f77ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91f015ca40 0x7f91f0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.386+0000 7f91f77ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91f006a300 0x7f91f00c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.386+0000 7f91f77ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91f00c8d70 0x7f91f006cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.386+0000 7f91f77ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00cabc0 con 0x7f91f00c8d70 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.386+0000 7f91f77ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f0068bb0 con 0x7f91f006a300 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.386+0000 7f91f77ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f00c91f0 con 0x7f91f015ca40 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f5d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91f00c8d70 0x7f91f006cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f5d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f91f00c8d70 0x7f91f006cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43102/0 (socket says 172.21.15.31:43102) 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f5d9c700 1 -- 172.21.15.31:0/263002005 learned_addr learned my addr 172.21.15.31:0/263002005 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f559b700 1 --2- 172.21.15.31:0/263002005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91f015ca40 0x7f91f0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f4d9a700 1 --2- 172.21.15.31:0/263002005 >> v2:172.21.15.62:3300/0 conn(0x7f91f006a300 0x7f91f00c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f5d9c700 1 -- 172.21.15.31:0/263002005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91f015ca40 msgr2=0x7f91f0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f5d9c700 1 --2- 172.21.15.31:0/263002005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91f015ca40 0x7f91f0068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f5d9c700 1 -- 172.21.15.31:0/263002005 >> v2:172.21.15.62:3300/0 conn(0x7f91f006a300 msgr2=0x7f91f00c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f5d9c700 1 --2- 172.21.15.31:0/263002005 >> v2:172.21.15.62:3300/0 conn(0x7f91f006a300 0x7f91f00c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f5d9c700 1 -- 172.21.15.31:0/263002005 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f0068da0 con 0x7f91f00c8d70 2023-10-06T05:22:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91f5d9c700 1 --2- 172.21.15.31:0/263002005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f00c8d70 0x7f91f006cf40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f91f006f220 tx=0x7f91e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8806cc24cf09bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.387+0000 7f91e7fff700 1 -- 172.21.15.31:0/263002005 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e800e070 con 0x7f91f00c8d70 2023-10-06T05:22:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91e7fff700 1 -- 172.21.15.31:0/263002005 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e800ba40 con 0x7f91f00c8d70 2023-10-06T05:22:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91e7fff700 1 -- 172.21.15.31:0/263002005 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e80092e0 con 0x7f91f00c8d70 2023-10-06T05:22:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91f77ff700 1 -- 172.21.15.31:0/263002005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f00c8d70 msgr2=0x7f91f006cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91f77ff700 1 --2- 172.21.15.31:0/263002005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f00c8d70 0x7f91f006cf40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f91f006f220 tx=0x7f91e8009840 comp rx=0 tx=0).stop 2023-10-06T05:22:35.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91f77ff700 1 -- 172.21.15.31:0/263002005 shutdown_connections 2023-10-06T05:22:35.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91f77ff700 1 --2- 172.21.15.31:0/263002005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f00c8d70 0x7f91f006cf40 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91f77ff700 1 --2- 172.21.15.31:0/263002005 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91f015ca40 0x7f91f0068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91f77ff700 1 --2- 172.21.15.31:0/263002005 >> v2:172.21.15.62:3300/0 conn(0x7f91f006a300 0x7f91f00c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91f77ff700 1 -- 172.21.15.31:0/263002005 >> 172.21.15.31:0/263002005 conn(0x7f91f00ce290 msgr2=0x7f91f00ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91f77ff700 1 -- 172.21.15.31:0/263002005 shutdown_connections 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.388+0000 7f91f77ff700 1 -- 172.21.15.31:0/263002005 wait complete. 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.390+0000 7f91f77ff700 1 Processor -- start 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.390+0000 7f91f77ff700 1 -- start start 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f77ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91f006a300 0x7f91f016a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f77ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91f00c8d70 0x7f91f016d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f77ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f015ca40 0x7f91f0159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f77ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f016a8e0 con 0x7f91f015ca40 2023-10-06T05:22:35.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f77ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f016dde0 con 0x7f91f00c8d70 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f77ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01698e0 con 0x7f91f006a300 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f5d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f015ca40 0x7f91f0159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f5d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f015ca40 0x7f91f0159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43106/0 (socket says 172.21.15.31:43106) 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f5d9c700 1 -- 172.21.15.31:0/4135015930 learned_addr learned my addr 172.21.15.31:0/4135015930 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f559b700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91f006a300 0x7f91f016a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.391+0000 7f91f4d9a700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91f00c8d70 0x7f91f016d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91f5d9c700 1 -- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91f006a300 msgr2=0x7f91f016a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91f5d9c700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91f006a300 0x7f91f016a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91f5d9c700 1 -- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91f00c8d70 msgr2=0x7f91f016d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91f5d9c700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91f00c8d70 0x7f91f016d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91f5d9c700 1 -- 172.21.15.31:0/4135015930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e8009ae0 con 0x7f91f015ca40 2023-10-06T05:22:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91f5d9c700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f015ca40 0x7f91f0159520 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009880 tx=0x7f91e8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91e57fa700 1 -- 172.21.15.31:0/4135015930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e80032f0 con 0x7f91f015ca40 2023-10-06T05:22:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91e57fa700 1 -- 172.21.15.31:0/4135015930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8008ce0 con 0x7f91f015ca40 2023-10-06T05:22:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91e57fa700 1 -- 172.21.15.31:0/4135015930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8026690 con 0x7f91f015ca40 2023-10-06T05:22:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91f77ff700 1 -- 172.21.15.31:0/4135015930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f00c4140 con 0x7f91f015ca40 2023-10-06T05:22:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.392+0000 7f91f77ff700 1 -- 172.21.15.31:0/4135015930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f0169ba0 con 0x7f91f015ca40 2023-10-06T05:22:35.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.393+0000 7f91f77ff700 1 -- 172.21.15.31:0/4135015930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f00adce0 con 0x7f91f015ca40 2023-10-06T05:22:35.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.394+0000 7f91e57fa700 1 -- 172.21.15.31:0/4135015930 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f91e8003040 con 0x7f91f015ca40 2023-10-06T05:22:35.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.395+0000 7f91e57fa700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f91d4067050 0x7f91d4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:35.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.395+0000 7f91e57fa700 1 -- 172.21.15.31:0/4135015930 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91e80862f0 con 0x7f91f015ca40 2023-10-06T05:22:35.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.396+0000 7f91f559b700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f91d4067050 0x7f91d4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:35.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.396+0000 7f91f559b700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f91d4067050 0x7f91d4069500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f91ec006fd0 tx=0x7f91ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.399+0000 7f91e57fa700 1 -- 172.21.15.31:0/4135015930 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91e80531f0 con 0x7f91f015ca40 2023-10-06T05:22:35.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:35.650+0000 7f91f77ff700 1 -- 172.21.15.31:0/4135015930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f91f006fb10 con 0x7f91f015ca40 2023-10-06T05:22:36.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.163+0000 7f91e57fa700 1 -- 172.21.15.31:0/4135015930 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91e8056810 con 0x7f91f015ca40 2023-10-06T05:22:36.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 -- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f91d4067050 msgr2=0x7f91d4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f91d4067050 0x7f91d4069500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f91ec006fd0 tx=0x7f91ec008040 comp rx=0 tx=0).stop 2023-10-06T05:22:36.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 -- 172.21.15.31:0/4135015930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f015ca40 msgr2=0x7f91f0159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f015ca40 0x7f91f0159520 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009880 tx=0x7f91e8018000 comp rx=0 tx=0).stop 2023-10-06T05:22:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 -- 172.21.15.31:0/4135015930 shutdown_connections 2023-10-06T05:22:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f91d4067050 0x7f91d4069500 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f015ca40 0x7f91f0159520 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91f006a300 0x7f91f016a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 --2- 172.21.15.31:0/4135015930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91f00c8d70 0x7f91f016d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 -- 172.21.15.31:0/4135015930 >> 172.21.15.31:0/4135015930 conn(0x7f91f00ce290 msgr2=0x7f91f015a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 -- 172.21.15.31:0/4135015930 shutdown_connections 2023-10-06T05:22:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.166+0000 7f91f77ff700 1 -- 172.21.15.31:0/4135015930 wait complete. 2023-10-06T05:22:36.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:36.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.386+0000 7effa1dff700 1 Processor -- start 2023-10-06T05:22:36.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.386+0000 7effa1dff700 1 -- start start 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.387+0000 7effa1dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff9c13a1e0 0x7eff9c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.387+0000 7effa1dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7eff9c13a5b0 0x7eff9c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.387+0000 7effa1dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff9c0c71b0 con 0x7eff9c0c6c30 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.387+0000 7effa1dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c0c7490 con 0x7eff9c13a1e0 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.387+0000 7effa1dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c0c7770 con 0x7eff9c13a5b0 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.388+0000 7eff9b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff9c13a1e0 0x7eff9c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.388+0000 7eff9b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff9c13a1e0 0x7eff9c0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43118/0 (socket says 172.21.15.31:43118) 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.388+0000 7eff9b7fe700 1 -- 172.21.15.31:0/3779499044 learned_addr learned my addr 172.21.15.31:0/3779499044 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.388+0000 7eff9affd700 1 --2- 172.21.15.31:0/3779499044 >> v2:172.21.15.62:3300/0 conn(0x7eff9c13a5b0 0x7eff9c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.388+0000 7eff9b7fe700 1 -- 172.21.15.31:0/3779499044 >> v2:172.21.15.62:3300/0 conn(0x7eff9c13a5b0 msgr2=0x7eff9c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:36.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.388+0000 7eff9b7fe700 1 --2- 172.21.15.31:0/3779499044 >> v2:172.21.15.62:3300/0 conn(0x7eff9c13a5b0 0x7eff9c06d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.388+0000 7eff9b7fe700 1 -- 172.21.15.31:0/3779499044 >> v1:172.21.15.62:6789/0 conn(0x7eff9c0c6c30 legacy=0x7eff9c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.388+0000 7eff9b7fe700 1 -- 172.21.15.31:0/3779499044 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c0c7a50 con 0x7eff9c13a1e0 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.388+0000 7eff9b7fe700 1 --2- 172.21.15.31:0/3779499044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a1e0 0x7eff9c0c66f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7eff9c06d130 tx=0x7eff8c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9db7179a483592a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.388+0000 7eff9a7fc700 1 -- 172.21.15.31:0/3779499044 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c005c10 con 0x7eff9c13a1e0 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.389+0000 7eff9a7fc700 1 -- 172.21.15.31:0/3779499044 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c005d70 con 0x7eff9c13a1e0 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.389+0000 7eff9a7fc700 1 -- 172.21.15.31:0/3779499044 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00b840 con 0x7eff9c13a1e0 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.389+0000 7effa1dff700 1 -- 172.21.15.31:0/3779499044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a1e0 msgr2=0x7eff9c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.389+0000 7effa1dff700 1 --2- 172.21.15.31:0/3779499044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a1e0 0x7eff9c0c66f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7eff9c06d130 tx=0x7eff8c009220 comp rx=0 tx=0).stop 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.389+0000 7effa1dff700 1 -- 172.21.15.31:0/3779499044 shutdown_connections 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.389+0000 7effa1dff700 1 --2- 172.21.15.31:0/3779499044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a1e0 0x7eff9c0c66f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.389+0000 7effa1dff700 1 --2- 172.21.15.31:0/3779499044 >> v2:172.21.15.62:3300/0 conn(0x7eff9c13a5b0 0x7eff9c06d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.389+0000 7effa1dff700 1 -- 172.21.15.31:0/3779499044 >> 172.21.15.31:0/3779499044 conn(0x7eff9c0cc0b0 msgr2=0x7eff9c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.389+0000 7effa1dff700 1 -- 172.21.15.31:0/3779499044 shutdown_connections 2023-10-06T05:22:36.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.389+0000 7effa1dff700 1 -- 172.21.15.31:0/3779499044 wait complete. 2023-10-06T05:22:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.391+0000 7effa1dff700 1 Processor -- start 2023-10-06T05:22:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.392+0000 7effa1dff700 1 -- start start 2023-10-06T05:22:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7effa1dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c0c6c30 0x7eff9c0add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7effa1dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff9c13a1e0 0x7eff9c139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7effa1dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a5b0 0x7eff9c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7effa1dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c152f30 con 0x7eff9c13a5b0 2023-10-06T05:22:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7effa1dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0cbab0 con 0x7eff9c0c6c30 2023-10-06T05:22:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7effa1dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c152060 con 0x7eff9c13a1e0 2023-10-06T05:22:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7eff9bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a5b0 0x7eff9c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7eff9bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a5b0 0x7eff9c1533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43132/0 (socket says 172.21.15.31:43132) 2023-10-06T05:22:36.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7eff9bfff700 1 -- 172.21.15.31:0/1836138839 learned_addr learned my addr 172.21.15.31:0/1836138839 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:36.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7eff9b7fe700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c0c6c30 0x7eff9c0add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7eff9affd700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff9c13a1e0 0x7eff9c139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.393+0000 7eff9bfff700 1 -- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff9c13a1e0 msgr2=0x7eff9c139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7eff9bfff700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff9c13a1e0 0x7eff9c139910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7eff9bfff700 1 -- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c0c6c30 msgr2=0x7eff9c0add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7eff9bfff700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c0c6c30 0x7eff9c0add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7eff9bfff700 1 -- 172.21.15.31:0/1836138839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c009480 con 0x7eff9c13a5b0 2023-10-06T05:22:36.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7eff9affd700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff9c13a1e0 0x7eff9c139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:36.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7eff9bfff700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a5b0 0x7eff9c1533c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7eff9400c930 tx=0x7eff940026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7eff837fe700 1 -- 172.21.15.31:0/1836138839 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94012630 con 0x7eff9c13a5b0 2023-10-06T05:22:36.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7eff837fe700 1 -- 172.21.15.31:0/1836138839 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff940127d0 con 0x7eff9c13a5b0 2023-10-06T05:22:36.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7effa1dff700 1 -- 172.21.15.31:0/1836138839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c152320 con 0x7eff9c13a5b0 2023-10-06T05:22:36.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7eff837fe700 1 -- 172.21.15.31:0/1836138839 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94012b90 con 0x7eff9c13a5b0 2023-10-06T05:22:36.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.394+0000 7effa1dff700 1 -- 172.21.15.31:0/1836138839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c1528e0 con 0x7eff9c13a5b0 2023-10-06T05:22:36.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.395+0000 7effa1dff700 1 -- 172.21.15.31:0/1836138839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c1530e0 con 0x7eff9c13a5b0 2023-10-06T05:22:36.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.396+0000 7eff837fe700 1 -- 172.21.15.31:0/1836138839 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7eff9401b400 con 0x7eff9c13a5b0 2023-10-06T05:22:36.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.397+0000 7eff837fe700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7eff88067050 0x7eff88069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.397+0000 7eff837fe700 1 -- 172.21.15.31:0/1836138839 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7eff94084bd0 con 0x7eff9c13a5b0 2023-10-06T05:22:36.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.397+0000 7eff9b7fe700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7eff88067050 0x7eff88069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.398+0000 7eff9b7fe700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7eff88067050 0x7eff88069500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff8c003080 tx=0x7eff8c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.401+0000 7eff837fe700 1 -- 172.21.15.31:0/1836138839 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff9c1530e0 con 0x7eff9c13a5b0 2023-10-06T05:22:36.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.654+0000 7effa1dff700 1 -- 172.21.15.31:0/1836138839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff9c04c380 con 0x7eff9c13a5b0 2023-10-06T05:22:36.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.655+0000 7eff837fe700 1 -- 172.21.15.31:0/1836138839 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4596 (secure 0 0 0) 0x7eff94051b30 con 0x7eff9c13a5b0 2023-10-06T05:22:36.656 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:36.657 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1885821393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1885821393},{"type":"v1","addr":"172.21.15.31:6834","nonce":1885821393}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6889,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/258742412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":258742412},{"type":"v1","addr":"172.21.15.31:6835","nonce":258742412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4070877337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4070877337},{"type":"v1","addr":"172.21.15.62:6834","nonce":4070877337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.638176+0000","modified":"2023-10-06T05:22:36.157996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7059":{"gid":7059,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2094863015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2094863015},{"type":"v1","addr":"172.21.15.62:6835","nonce":2094863015}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:22:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 -- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7eff88067050 msgr2=0x7eff88069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7eff88067050 0x7eff88069500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff8c003080 tx=0x7eff8c015000 comp rx=0 tx=0).stop 2023-10-06T05:22:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 -- 172.21.15.31:0/1836138839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a5b0 msgr2=0x7eff9c1533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a5b0 0x7eff9c1533c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7eff9400c930 tx=0x7eff940026e0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 -- 172.21.15.31:0/1836138839 shutdown_connections 2023-10-06T05:22:36.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7eff88067050 0x7eff88069500 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c13a5b0 0x7eff9c1533c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff9c0c6c30 0x7eff9c0add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 --2- 172.21.15.31:0/1836138839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff9c13a1e0 0x7eff9c139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 -- 172.21.15.31:0/1836138839 >> 172.21.15.31:0/1836138839 conn(0x7eff9c0cc0b0 msgr2=0x7eff9c0c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 -- 172.21.15.31:0/1836138839 shutdown_connections 2023-10-06T05:22:36.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.657+0000 7effa1dff700 1 -- 172.21.15.31:0/1836138839 wait complete. 2023-10-06T05:22:36.660 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 67 2023-10-06T05:22:36.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:22:36.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.870+0000 7ffa631ff700 1 Processor -- start 2023-10-06T05:22:36.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.870+0000 7ffa631ff700 1 -- start start 2023-10-06T05:22:36.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.871+0000 7ffa631ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffa5c15a930 0x7ffa5c0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.871+0000 7ffa631ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cb1b0 0x7ffa5c0ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.871+0000 7ffa631ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5c0ce090 con 0x7ffa5c0cd270 2023-10-06T05:22:36.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.871+0000 7ffa631ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5c0cb6f0 con 0x7ffa5c15a930 2023-10-06T05:22:36.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.871+0000 7ffa631ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0cb9d0 con 0x7ffa5c0cb1b0 2023-10-06T05:22:36.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.871+0000 7ffa60f9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffa5c15a930 0x7ffa5c0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.871+0000 7ffa5bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cb1b0 0x7ffa5c0ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.871+0000 7ffa60f9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffa5c15a930 0x7ffa5c0cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:34422/0 (socket says 172.21.15.31:34422) 2023-10-06T05:22:36.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.871+0000 7ffa60f9b700 1 -- 172.21.15.31:0/3503302622 learned_addr learned my addr 172.21.15.31:0/3503302622 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:36.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.872+0000 7ffa60f9b700 1 -- 172.21.15.31:0/3503302622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cb1b0 msgr2=0x7ffa5c0ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:36.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.872+0000 7ffa60f9b700 1 --2- 172.21.15.31:0/3503302622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cb1b0 0x7ffa5c0ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.872+0000 7ffa60f9b700 1 -- 172.21.15.31:0/3503302622 >> v1:172.21.15.62:6789/0 conn(0x7ffa5c0cd270 legacy=0x7ffa5c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.872+0000 7ffa60f9b700 1 -- 172.21.15.31:0/3503302622 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c0cbcb0 con 0x7ffa5c15a930 2023-10-06T05:22:36.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.872+0000 7ffa60f9b700 1 --2- 172.21.15.31:0/3503302622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c15a930 0x7ffa5c0cac70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009990 tx=0x7ffa50009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a5ded488f9509ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.872+0000 7ffa5b7fe700 1 -- 172.21.15.31:0/3503302622 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50009e80 con 0x7ffa5c15a930 2023-10-06T05:22:36.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.873+0000 7ffa5b7fe700 1 -- 172.21.15.31:0/3503302622 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa50005c10 con 0x7ffa5c15a930 2023-10-06T05:22:36.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.873+0000 7ffa5b7fe700 1 -- 172.21.15.31:0/3503302622 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5000b6e0 con 0x7ffa5c15a930 2023-10-06T05:22:36.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.873+0000 7ffa631ff700 1 -- 172.21.15.31:0/3503302622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c15a930 msgr2=0x7ffa5c0cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.873+0000 7ffa631ff700 1 --2- 172.21.15.31:0/3503302622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c15a930 0x7ffa5c0cac70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009990 tx=0x7ffa50009220 comp rx=0 tx=0).stop 2023-10-06T05:22:36.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.873+0000 7ffa631ff700 1 -- 172.21.15.31:0/3503302622 shutdown_connections 2023-10-06T05:22:36.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.873+0000 7ffa631ff700 1 --2- 172.21.15.31:0/3503302622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c15a930 0x7ffa5c0cac70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.873+0000 7ffa631ff700 1 --2- 172.21.15.31:0/3503302622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cb1b0 0x7ffa5c0ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.873+0000 7ffa631ff700 1 -- 172.21.15.31:0/3503302622 >> 172.21.15.31:0/3503302622 conn(0x7ffa5c099720 msgr2=0x7ffa5c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:36.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.873+0000 7ffa631ff700 1 -- 172.21.15.31:0/3503302622 shutdown_connections 2023-10-06T05:22:36.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.873+0000 7ffa631ff700 1 -- 172.21.15.31:0/3503302622 wait complete. 2023-10-06T05:22:36.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.875+0000 7ffa631ff700 1 Processor -- start 2023-10-06T05:22:36.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.875+0000 7ffa631ff700 1 -- start start 2023-10-06T05:22:36.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa631ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c0cb1b0 0x7ffa5c09a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa631ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cd270 0x7ffa5c16d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa631ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c15a930 0x7ffa5c0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa631ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c153480 con 0x7ffa5c15a930 2023-10-06T05:22:36.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa631ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c16abd0 con 0x7ffa5c0cb1b0 2023-10-06T05:22:36.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa631ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c09a820 con 0x7ffa5c0cd270 2023-10-06T05:22:36.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa6179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c15a930 0x7ffa5c0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa60f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c0cb1b0 0x7ffa5c09a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa5bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cd270 0x7ffa5c16d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa60f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c0cb1b0 0x7ffa5c09a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:34426/0 (socket says 172.21.15.31:34426) 2023-10-06T05:22:36.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa6179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c15a930 0x7ffa5c0c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43140/0 (socket says 172.21.15.31:43140) 2023-10-06T05:22:36.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.876+0000 7ffa60f9b700 1 -- 172.21.15.31:0/4004069577 learned_addr learned my addr 172.21.15.31:0/4004069577 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:36.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa6179c700 1 -- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cd270 msgr2=0x7ffa5c16d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa6179c700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cd270 0x7ffa5c16d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa6179c700 1 -- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c0cb1b0 msgr2=0x7ffa5c09a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:36.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa6179c700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c0cb1b0 0x7ffa5c09a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:36.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa6179c700 1 -- 172.21.15.31:0/4004069577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa50009480 con 0x7ffa5c15a930 2023-10-06T05:22:36.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa5bfff700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cd270 0x7ffa5c16d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:36.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa60f9b700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c0cb1b0 0x7ffa5c09a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:36.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa6179c700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c15a930 0x7ffa5c0c9690 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c830 tx=0x7ffa54001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa58ff9700 1 -- 172.21.15.31:0/4004069577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54012630 con 0x7ffa5c15a930 2023-10-06T05:22:36.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa58ff9700 1 -- 172.21.15.31:0/4004069577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa54012790 con 0x7ffa5c15a930 2023-10-06T05:22:36.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa631ff700 1 -- 172.21.15.31:0/4004069577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c09ab00 con 0x7ffa5c15a930 2023-10-06T05:22:36.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa58ff9700 1 -- 172.21.15.31:0/4004069577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54012b50 con 0x7ffa5c15a930 2023-10-06T05:22:36.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.877+0000 7ffa631ff700 1 -- 172.21.15.31:0/4004069577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c09b0c0 con 0x7ffa5c15a930 2023-10-06T05:22:36.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.878+0000 7ffa631ff700 1 -- 172.21.15.31:0/4004069577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c04c380 con 0x7ffa5c15a930 2023-10-06T05:22:36.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.879+0000 7ffa58ff9700 1 -- 172.21.15.31:0/4004069577 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7ffa5401a3a0 con 0x7ffa5c15a930 2023-10-06T05:22:36.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.880+0000 7ffa58ff9700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa40067050 0x7ffa40069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:36.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.880+0000 7ffa58ff9700 1 -- 172.21.15.31:0/4004069577 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffa54084a20 con 0x7ffa5c15a930 2023-10-06T05:22:36.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.881+0000 7ffa60f9b700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa40067050 0x7ffa40069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:36.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.881+0000 7ffa60f9b700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa40067050 0x7ffa40069500 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffa50003080 tx=0x7ffa50016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:36.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:36.885+0000 7ffa58ff9700 1 -- 172.21.15.31:0/4004069577 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa54051980 con 0x7ffa5c15a930 2023-10-06T05:22:37.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.086+0000 7ffa631ff700 1 -- 172.21.15.31:0/4004069577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa5c16ad10 con 0x7ffa5c15a930 2023-10-06T05:22:37.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.087+0000 7ffa58ff9700 1 -- 172.21.15.31:0/4004069577 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16920 (secure 0 0 0) 0x7ffa54054fa0 con 0x7ffa5c15a930 2023-10-06T05:22:37.089 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:37.089 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":101,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:22:34.142984+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.850575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.607953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:22:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 -- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa40067050 msgr2=0x7ffa40069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa40067050 0x7ffa40069500 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffa50003080 tx=0x7ffa50016000 comp rx=0 tx=0).stop 2023-10-06T05:22:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 -- 172.21.15.31:0/4004069577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c15a930 msgr2=0x7ffa5c0c9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c15a930 0x7ffa5c0c9690 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c830 tx=0x7ffa54001690 comp rx=0 tx=0).stop 2023-10-06T05:22:37.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 -- 172.21.15.31:0/4004069577 shutdown_connections 2023-10-06T05:22:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa40067050 0x7ffa40069500 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c15a930 0x7ffa5c0c9690 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa5c0cb1b0 0x7ffa5c09a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 --2- 172.21.15.31:0/4004069577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa5c0cd270 0x7ffa5c16d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 -- 172.21.15.31:0/4004069577 >> 172.21.15.31:0/4004069577 conn(0x7ffa5c099720 msgr2=0x7ffa5c0b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.089+0000 7ffa631ff700 1 -- 172.21.15.31:0/4004069577 shutdown_connections 2023-10-06T05:22:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.090+0000 7ffa631ff700 1 -- 172.21.15.31:0/4004069577 wait complete. 2023-10-06T05:22:37.109 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:22:37.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:22:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.304+0000 7f43513ff700 1 Processor -- start 2023-10-06T05:22:37.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.304+0000 7f43513ff700 1 -- start start 2023-10-06T05:22:37.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.305+0000 7f43513ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7b60 0x7f434c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.305+0000 7f43513ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434c0ca5b0 con 0x7f434c0aee70 2023-10-06T05:22:37.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.305+0000 7f43513ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434c0af2d0 con 0x7f434c0c7f30 2023-10-06T05:22:37.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.305+0000 7f43513ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c06f740 con 0x7f434c0c7b60 2023-10-06T05:22:37.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.305+0000 7f434b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f434c0aee70 0x7f434c13f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44482/0 (socket says 172.21.15.31:44482) 2023-10-06T05:22:37.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.305+0000 7f434b7fe700 1 -- 172.21.15.31:0/543449727 learned_addr learned my addr 172.21.15.31:0/543449727 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:37.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.306+0000 7f434affd700 1 --2- 172.21.15.31:0/543449727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7b60 0x7f434c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.306+0000 7f4349ffb700 1 -- 172.21.15.31:0/543449727 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966126221 0 0) 0x7f434c0ca5b0 con 0x7f434c0aee70 2023-10-06T05:22:37.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.306+0000 7f4349ffb700 1 -- 172.21.15.31:0/543449727 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4334003570 con 0x7f434c0aee70 2023-10-06T05:22:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.306+0000 7f4349ffb700 1 -- 172.21.15.31:0/543449727 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216216523 0 0) 0x7f434c0af2d0 con 0x7f434c0c7f30 2023-10-06T05:22:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.306+0000 7f434affd700 1 -- 172.21.15.31:0/543449727 >> v1:172.21.15.62:6789/0 conn(0x7f434c0c7f30 legacy=0x7f434c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.306+0000 7f434affd700 1 -- 172.21.15.31:0/543449727 >> v1:172.21.15.31:6789/0 conn(0x7f434c0aee70 legacy=0x7f434c13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.306+0000 7f434affd700 1 -- 172.21.15.31:0/543449727 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c0c9c80 con 0x7f434c0c7b60 2023-10-06T05:22:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.306+0000 7f434affd700 1 --2- 172.21.15.31:0/543449727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7b60 0x7f434c069880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f433c009900 tx=0x7f433c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df16dc6f2f6c3c0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.307+0000 7f4349ffb700 1 -- 172.21.15.31:0/543449727 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c011630 con 0x7f434c0c7b60 2023-10-06T05:22:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.307+0000 7f4349ffb700 1 -- 172.21.15.31:0/543449727 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c011790 con 0x7f434c0c7b60 2023-10-06T05:22:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.307+0000 7f4349ffb700 1 -- 172.21.15.31:0/543449727 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c011b30 con 0x7f434c0c7b60 2023-10-06T05:22:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.307+0000 7f43513ff700 1 -- 172.21.15.31:0/543449727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7b60 msgr2=0x7f434c069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.307+0000 7f43513ff700 1 --2- 172.21.15.31:0/543449727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7b60 0x7f434c069880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f433c009900 tx=0x7f433c009120 comp rx=0 tx=0).stop 2023-10-06T05:22:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.307+0000 7f43513ff700 1 -- 172.21.15.31:0/543449727 shutdown_connections 2023-10-06T05:22:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.307+0000 7f43513ff700 1 --2- 172.21.15.31:0/543449727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7b60 0x7f434c069880 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.307+0000 7f43513ff700 1 -- 172.21.15.31:0/543449727 >> 172.21.15.31:0/543449727 conn(0x7f434c0cc0b0 msgr2=0x7f434c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.307+0000 7f43513ff700 1 -- 172.21.15.31:0/543449727 shutdown_connections 2023-10-06T05:22:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.308+0000 7f43513ff700 1 -- 172.21.15.31:0/543449727 wait complete. 2023-10-06T05:22:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.310+0000 7f43513ff700 1 Processor -- start 2023-10-06T05:22:37.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.310+0000 7f43513ff700 1 -- start start 2023-10-06T05:22:37.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.311+0000 7f43513ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434c0aee70 0x7f434c0aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.311+0000 7f43513ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c0c7b60 0x7f434c0aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.311+0000 7f43513ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7f30 0x7f434c0c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.311+0000 7f43513ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0aa4e0 con 0x7f434c0c7b60 2023-10-06T05:22:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.311+0000 7f43513ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0a9940 con 0x7f434c0aee70 2023-10-06T05:22:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.311+0000 7f43513ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0a9bb0 con 0x7f434c0c7f30 2023-10-06T05:22:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.311+0000 7f434a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c0c7b60 0x7f434c0aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.311+0000 7f434a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c0c7b60 0x7f434c0aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42410/0 (socket says 172.21.15.31:42410) 2023-10-06T05:22:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.311+0000 7f434a7fc700 1 -- 172.21.15.31:0/3243955789 learned_addr learned my addr 172.21.15.31:0/3243955789 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.311+0000 7f434affd700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434c0aee70 0x7f434c0aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f434b7fe700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7f30 0x7f434c0c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f434a7fc700 1 -- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7f30 msgr2=0x7f434c0c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f434a7fc700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7f30 0x7f434c0c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f434a7fc700 1 -- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434c0aee70 msgr2=0x7f434c0aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f434a7fc700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434c0aee70 0x7f434c0aa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f434a7fc700 1 -- 172.21.15.31:0/3243955789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0093c0 con 0x7f434c0c7b60 2023-10-06T05:22:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f434a7fc700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c0c7b60 0x7f434c0aaeb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f434000c560 tx=0x7f434000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f4332ffd700 1 -- 172.21.15.31:0/3243955789 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340011590 con 0x7f434c0c7b60 2023-10-06T05:22:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f4332ffd700 1 -- 172.21.15.31:0/3243955789 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4340011710 con 0x7f434c0c7b60 2023-10-06T05:22:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f43513ff700 1 -- 172.21.15.31:0/3243955789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434c0a9e90 con 0x7f434c0c7b60 2023-10-06T05:22:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.312+0000 7f4332ffd700 1 -- 172.21.15.31:0/3243955789 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340011ad0 con 0x7f434c0c7b60 2023-10-06T05:22:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.313+0000 7f43513ff700 1 -- 172.21.15.31:0/3243955789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434c0ae8e0 con 0x7f434c0c7b60 2023-10-06T05:22:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.313+0000 7f43513ff700 1 -- 172.21.15.31:0/3243955789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434c04c380 con 0x7f434c0c7b60 2023-10-06T05:22:37.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.314+0000 7f4332ffd700 1 -- 172.21.15.31:0/3243955789 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f434001b3b0 con 0x7f434c0c7b60 2023-10-06T05:22:37.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.315+0000 7f4332ffd700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4338067050 0x7f4338069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.315+0000 7f4332ffd700 1 -- 172.21.15.31:0/3243955789 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4340083d30 con 0x7f434c0c7b60 2023-10-06T05:22:37.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.316+0000 7f434affd700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4338067050 0x7f4338069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.316+0000 7f434affd700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4338067050 0x7f4338069500 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f433c009850 tx=0x7f433c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.319+0000 7f4332ffd700 1 -- 172.21.15.31:0/3243955789 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4340050c90 con 0x7f434c0c7b60 2023-10-06T05:22:37.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.517+0000 7f43513ff700 1 -- 172.21.15.31:0/3243955789 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f434c0aead0 con 0x7f4338067050 2023-10-06T05:22:37.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.529+0000 7f4332ffd700 1 -- 172.21.15.31:0/3243955789 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311581 (secure 0 0 0) 0x7f434c0aead0 con 0x7f4338067050 2023-10-06T05:22:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 -- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4338067050 msgr2=0x7f4338069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4338067050 0x7f4338069500 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f433c009850 tx=0x7f433c016000 comp rx=0 tx=0).stop 2023-10-06T05:22:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 -- 172.21.15.31:0/3243955789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c0c7b60 msgr2=0x7f434c0aaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c0c7b60 0x7f434c0aaeb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f434000c560 tx=0x7f434000c870 comp rx=0 tx=0).stop 2023-10-06T05:22:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 -- 172.21.15.31:0/3243955789 shutdown_connections 2023-10-06T05:22:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4338067050 0x7f4338069500 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f434c0c7f30 0x7f434c0c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f434c0aee70 0x7f434c0aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 --2- 172.21.15.31:0/3243955789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c0c7b60 0x7f434c0aaeb0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 -- 172.21.15.31:0/3243955789 >> 172.21.15.31:0/3243955789 conn(0x7f434c0cc0b0 msgr2=0x7f434c0c8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 -- 172.21.15.31:0/3243955789 shutdown_connections 2023-10-06T05:22:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.533+0000 7f43513ff700 1 -- 172.21.15.31:0/3243955789 wait complete. 2023-10-06T05:22:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:22:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-10-06T05:22:37.473304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397544,"kb_used_data":11048,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752577176,"statfs":{"total":773094113280,"available":770639028224,"internally_reserved":0,"allocated":11313152,"data_stored":5839100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001595"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.161574+0000","last_change":"2023-10-06T05:22:33.157614+0000","last_active":"2023-10-06T05:22:33.161574+0000","last_peered":"2023-10-06T05:22:33.161574+0000","last_clean":"2023-10-06T05:22:33.161574+0000","last_became_active":"2023-10-06T05:22:33.157350+0000","last_became_peered":"2023-10-06T05:22:33.157350+0000","last_unstale":"2023-10-06T05:22:33.161574+0000","last_undegraded":"2023-10-06T05:22:33.161574+0000","last_fullsized":"2023-10-06T05:22:33.161574+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:25.981504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.173234+0000","last_change":"2023-10-06T05:22:33.168369+0000","last_active":"2023-10-06T05:22:33.173234+0000","last_peered":"2023-10-06T05:22:33.173234+0000","last_clean":"2023-10-06T05:22:33.173234+0000","last_became_active":"2023-10-06T05:22:33.168164+0000","last_became_peered":"2023-10-06T05:22:33.168164+0000","last_unstale":"2023-10-06T05:22:33.173234+0000","last_undegraded":"2023-10-06T05:22:33.173234+0000","last_fullsized":"2023-10-06T05:22:33.173234+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:44.071118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163389+0000","last_change":"2023-10-06T05:22:33.155575+0000","last_active":"2023-10-06T05:22:33.163389+0000","last_peered":"2023-10-06T05:22:33.163389+0000","last_clean":"2023-10-06T05:22:33.163389+0000","last_became_active":"2023-10-06T05:22:33.155319+0000","last_became_peered":"2023-10-06T05:22:33.155319+0000","last_unstale":"2023-10-06T05:22:33.163389+0000","last_undegraded":"2023-10-06T05:22:33.163389+0000","last_fullsized":"2023-10-06T05:22:33.163389+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:57.486209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.167992+0000","last_change":"2023-10-06T05:22:33.161735+0000","last_active":"2023-10-06T05:22:33.167992+0000","last_peered":"2023-10-06T05:22:33.167992+0000","last_clean":"2023-10-06T05:22:33.167992+0000","last_became_active":"2023-10-06T05:22:33.160656+0000","last_became_peered":"2023-10-06T05:22:33.160656+0000","last_unstale":"2023-10-06T05:22:33.167992+0000","last_undegraded":"2023-10-06T05:22:33.167992+0000","last_fullsized":"2023-10-06T05:22:33.167992+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:57.482645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.161590+0000","last_change":"2023-10-06T05:22:33.155788+0000","last_active":"2023-10-06T05:22:33.161590+0000","last_peered":"2023-10-06T05:22:33.161590+0000","last_clean":"2023-10-06T05:22:33.161590+0000","last_became_active":"2023-10-06T05:22:33.155515+0000","last_became_peered":"2023-10-06T05:22:33.155515+0000","last_unstale":"2023-10-06T05:22:33.161590+0000","last_undegraded":"2023-10-06T05:22:33.161590+0000","last_fullsized":"2023-10-06T05:22:33.161590+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:16.079961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164520+0000","last_change":"2023-10-06T05:22:33.158729+0000","last_active":"2023-10-06T05:22:33.164520+0000","last_peered":"2023-10-06T05:22:33.164520+0000","last_clean":"2023-10-06T05:22:33.164520+0000","last_became_active":"2023-10-06T05:22:33.158460+0000","last_became_peered":"2023-10-06T05:22:33.158460+0000","last_unstale":"2023-10-06T05:22:33.164520+0000","last_undegraded":"2023-10-06T05:22:33.164520+0000","last_fullsized":"2023-10-06T05:22:33.164520+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:48.210445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.155993+0000","last_change":"2023-10-06T05:22:33.153360+0000","last_active":"2023-10-06T05:22:33.155993+0000","last_peered":"2023-10-06T05:22:33.155993+0000","last_clean":"2023-10-06T05:22:33.155993+0000","last_became_active":"2023-10-06T05:22:33.153130+0000","last_became_peered":"2023-10-06T05:22:33.153130+0000","last_unstale":"2023-10-06T05:22:33.155993+0000","last_undegraded":"2023-10-06T05:22:33.155993+0000","last_fullsized":"2023-10-06T05:22:33.155993+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:18.879342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163609+0000","last_change":"2023-10-06T05:22:33.154892+0000","last_active":"2023-10-06T05:22:33.163609+0000","last_peered":"2023-10-06T05:22:33.163609+0000","last_clean":"2023-10-06T05:22:33.163609+0000","last_became_active":"2023-10-06T05:22:33.154625+0000","last_became_peered":"2023-10-06T05:22:33.154625+0000","last_unstale":"2023-10-06T05:22:33.163609+0000","last_undegraded":"2023-10-06T05:22:33.163609+0000","last_fullsized":"2023-10-06T05:22:33.163609+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:21.843425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176799+0000","last_change":"2023-10-06T05:22:33.167185+0000","last_active":"2023-10-06T05:22:33.176799+0000","last_peered":"2023-10-06T05:22:33.176799+0000","last_clean":"2023-10-06T05:22:33.176799+0000","last_became_active":"2023-10-06T05:22:33.166628+0000","last_became_peered":"2023-10-06T05:22:33.166628+0000","last_unstale":"2023-10-06T05:22:33.176799+0000","last_undegraded":"2023-10-06T05:22:33.176799+0000","last_fullsized":"2023-10-06T05:22:33.176799+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:53.985285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163425+0000","last_change":"2023-10-06T05:22:33.159341+0000","last_active":"2023-10-06T05:22:33.163425+0000","last_peered":"2023-10-06T05:22:33.163425+0000","last_clean":"2023-10-06T05:22:33.163425+0000","last_became_active":"2023-10-06T05:22:33.159078+0000","last_became_peered":"2023-10-06T05:22:33.159078+0000","last_unstale":"2023-10-06T05:22:33.163425+0000","last_undegraded":"2023-10-06T05:22:33.163425+0000","last_fullsized":"2023-10-06T05:22:33.163425+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:11.741017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176310+0000","last_change":"2023-10-06T05:22:33.171738+0000","last_active":"2023-10-06T05:22:33.176310+0000","last_peered":"2023-10-06T05:22:33.176310+0000","last_clean":"2023-10-06T05:22:33.176310+0000","last_became_active":"2023-10-06T05:22:33.171398+0000","last_became_peered":"2023-10-06T05:22:33.171398+0000","last_unstale":"2023-10-06T05:22:33.176310+0000","last_undegraded":"2023-10-06T05:22:33.176310+0000","last_fullsized":"2023-10-06T05:22:33.176310+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:37.422324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.156563+0000","last_change":"2023-10-06T05:22:33.151503+0000","last_active":"2023-10-06T05:22:33.156563+0000","last_peered":"2023-10-06T05:22:33.156563+0000","last_clean":"2023-10-06T05:22:33.156563+0000","last_became_active":"2023-10-06T05:22:33.151219+0000","last_became_peered":"2023-10-06T05:22:33.151219+0000","last_unstale":"2023-10-06T05:22:33.156563+0000","last_undegraded":"2023-10-06T05:22:33.156563+0000","last_fullsized":"2023-10-06T05:22:33.156563+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:04.442056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.175359+0000","last_change":"2023-10-06T05:22:33.171406+0000","last_active":"2023-10-06T05:22:33.175359+0000","last_peered":"2023-10-06T05:22:33.175359+0000","last_clean":"2023-10-06T05:22:33.175359+0000","last_became_active":"2023-10-06T05:22:33.171112+0000","last_became_peered":"2023-10-06T05:22:33.171112+0000","last_unstale":"2023-10-06T05:22:33.175359+0000","last_undegraded":"2023-10-06T05:22:33.175359+0000","last_fullsized":"2023-10-06T05:22:33.175359+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:08.383615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162567+0000","last_change":"2023-10-06T05:22:33.155804+0000","last_active":"2023-10-06T05:22:33.162567+0000","last_peered":"2023-10-06T05:22:33.162567+0000","last_clean":"2023-10-06T05:22:33.162567+0000","last_became_active":"2023-10-06T05:22:33.155527+0000","last_became_peered":"2023-10-06T05:22:33.155527+0000","last_unstale":"2023-10-06T05:22:33.162567+0000","last_undegraded":"2023-10-06T05:22:33.162567+0000","last_fullsized":"2023-10-06T05:22:33.162567+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:07.085636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176324+0000","last_change":"2023-10-06T05:22:33.172283+0000","last_active":"2023-10-06T05:22:33.176324+0000","last_peered":"2023-10-06T05:22:33.176324+0000","last_clean":"2023-10-06T05:22:33.176324+0000","last_became_active":"2023-10-06T05:22:33.171980+0000","last_became_peered":"2023-10-06T05:22:33.171980+0000","last_unstale":"2023-10-06T05:22:33.176324+0000","last_undegraded":"2023-10-06T05:22:33.176324+0000","last_fullsized":"2023-10-06T05:22:33.176324+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:31.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.152525+0000","last_change":"2023-10-06T05:22:33.149888+0000","last_active":"2023-10-06T05:22:33.152525+0000","last_peered":"2023-10-06T05:22:33.152525+0000","last_clean":"2023-10-06T05:22:33.152525+0000","last_became_active":"2023-10-06T05:22:33.149631+0000","last_became_peered":"2023-10-06T05:22:33.149631+0000","last_unstale":"2023-10-06T05:22:33.152525+0000","last_undegraded":"2023-10-06T05:22:33.152525+0000","last_fullsized":"2023-10-06T05:22:33.152525+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:21.065658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.175383+0000","last_change":"2023-10-06T05:22:33.171727+0000","last_active":"2023-10-06T05:22:33.175383+0000","last_peered":"2023-10-06T05:22:33.175383+0000","last_clean":"2023-10-06T05:22:33.175383+0000","last_became_active":"2023-10-06T05:22:33.171496+0000","last_became_peered":"2023-10-06T05:22:33.171496+0000","last_unstale":"2023-10-06T05:22:33.175383+0000","last_undegraded":"2023-10-06T05:22:33.175383+0000","last_fullsized":"2023-10-06T05:22:33.175383+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:44.347426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.173215+0000","last_change":"2023-10-06T05:22:33.168255+0000","last_active":"2023-10-06T05:22:33.173215+0000","last_peered":"2023-10-06T05:22:33.173215+0000","last_clean":"2023-10-06T05:22:33.173215+0000","last_became_active":"2023-10-06T05:22:33.168055+0000","last_became_peered":"2023-10-06T05:22:33.168055+0000","last_unstale":"2023-10-06T05:22:33.173215+0000","last_undegraded":"2023-10-06T05:22:33.173215+0000","last_fullsized":"2023-10-06T05:22:33.173215+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:53.985285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.173158+0000","last_change":"2023-10-06T05:22:33.164745+0000","last_active":"2023-10-06T05:22:33.173158+0000","last_peered":"2023-10-06T05:22:33.173158+0000","last_clean":"2023-10-06T05:22:33.173158+0000","last_became_active":"2023-10-06T05:22:33.164346+0000","last_became_peered":"2023-10-06T05:22:33.164346+0000","last_unstale":"2023-10-06T05:22:33.173158+0000","last_undegraded":"2023-10-06T05:22:33.173158+0000","last_fullsized":"2023-10-06T05:22:33.173158+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:15.061842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.156949+0000","last_change":"2023-10-06T05:22:33.151785+0000","last_active":"2023-10-06T05:22:33.156949+0000","last_peered":"2023-10-06T05:22:33.156949+0000","last_clean":"2023-10-06T05:22:33.156949+0000","last_became_active":"2023-10-06T05:22:33.151522+0000","last_became_peered":"2023-10-06T05:22:33.151522+0000","last_unstale":"2023-10-06T05:22:33.156949+0000","last_undegraded":"2023-10-06T05:22:33.156949+0000","last_fullsized":"2023-10-06T05:22:33.156949+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:33.316756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174605+0000","last_change":"2023-10-06T05:22:33.161428+0000","last_active":"2023-10-06T05:22:33.174605+0000","last_peered":"2023-10-06T05:22:33.174605+0000","last_clean":"2023-10-06T05:22:33.174605+0000","last_became_active":"2023-10-06T05:22:33.161039+0000","last_became_peered":"2023-10-06T05:22:33.161039+0000","last_unstale":"2023-10-06T05:22:33.174605+0000","last_undegraded":"2023-10-06T05:22:33.174605+0000","last_fullsized":"2023-10-06T05:22:33.174605+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:01.452838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.158252+0000","last_change":"2023-10-06T05:22:33.154065+0000","last_active":"2023-10-06T05:22:33.158252+0000","last_peered":"2023-10-06T05:22:33.158252+0000","last_clean":"2023-10-06T05:22:33.158252+0000","last_became_active":"2023-10-06T05:22:33.153771+0000","last_became_peered":"2023-10-06T05:22:33.153771+0000","last_unstale":"2023-10-06T05:22:33.158252+0000","last_undegraded":"2023-10-06T05:22:33.158252+0000","last_fullsized":"2023-10-06T05:22:33.158252+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:29.935435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176685+0000","last_change":"2023-10-06T05:22:33.163482+0000","last_active":"2023-10-06T05:22:33.176685+0000","last_peered":"2023-10-06T05:22:33.176685+0000","last_clean":"2023-10-06T05:22:33.176685+0000","last_became_active":"2023-10-06T05:22:33.163128+0000","last_became_peered":"2023-10-06T05:22:33.163128+0000","last_unstale":"2023-10-06T05:22:33.176685+0000","last_undegraded":"2023-10-06T05:22:33.176685+0000","last_fullsized":"2023-10-06T05:22:33.176685+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:21.843425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176522+0000","last_change":"2023-10-06T05:22:33.171232+0000","last_active":"2023-10-06T05:22:33.176522+0000","last_peered":"2023-10-06T05:22:33.176522+0000","last_clean":"2023-10-06T05:22:33.176522+0000","last_became_active":"2023-10-06T05:22:33.170892+0000","last_became_peered":"2023-10-06T05:22:33.170892+0000","last_unstale":"2023-10-06T05:22:33.176522+0000","last_undegraded":"2023-10-06T05:22:33.176522+0000","last_fullsized":"2023-10-06T05:22:33.176522+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:31.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174708+0000","last_change":"2023-10-06T05:22:33.170644+0000","last_active":"2023-10-06T05:22:33.174708+0000","last_peered":"2023-10-06T05:22:33.174708+0000","last_clean":"2023-10-06T05:22:33.174708+0000","last_became_active":"2023-10-06T05:22:33.170385+0000","last_became_peered":"2023-10-06T05:22:33.170385+0000","last_unstale":"2023-10-06T05:22:33.174708+0000","last_undegraded":"2023-10-06T05:22:33.174708+0000","last_fullsized":"2023-10-06T05:22:33.174708+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:29.935435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.157214+0000","last_change":"2023-10-06T05:22:33.152671+0000","last_active":"2023-10-06T05:22:33.157214+0000","last_peered":"2023-10-06T05:22:33.157214+0000","last_clean":"2023-10-06T05:22:33.157214+0000","last_became_active":"2023-10-06T05:22:33.152355+0000","last_became_peered":"2023-10-06T05:22:33.152355+0000","last_unstale":"2023-10-06T05:22:33.157214+0000","last_undegraded":"2023-10-06T05:22:33.157214+0000","last_fullsized":"2023-10-06T05:22:33.157214+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:01.999949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.173179+0000","last_change":"2023-10-06T05:22:33.165156+0000","last_active":"2023-10-06T05:22:33.173179+0000","last_peered":"2023-10-06T05:22:33.173179+0000","last_clean":"2023-10-06T05:22:33.173179+0000","last_became_active":"2023-10-06T05:22:33.164795+0000","last_became_peered":"2023-10-06T05:22:33.164795+0000","last_unstale":"2023-10-06T05:22:33.173179+0000","last_undegraded":"2023-10-06T05:22:33.173179+0000","last_fullsized":"2023-10-06T05:22:33.173179+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:20.680894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.175622+0000","last_change":"2023-10-06T05:22:33.171400+0000","last_active":"2023-10-06T05:22:33.175622+0000","last_peered":"2023-10-06T05:22:33.175622+0000","last_clean":"2023-10-06T05:22:33.175622+0000","last_became_active":"2023-10-06T05:22:33.171143+0000","last_became_peered":"2023-10-06T05:22:33.171143+0000","last_unstale":"2023-10-06T05:22:33.175622+0000","last_undegraded":"2023-10-06T05:22:33.175622+0000","last_fullsized":"2023-10-06T05:22:33.175622+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:29.286874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.158912+0000","last_change":"2023-10-06T05:22:33.153997+0000","last_active":"2023-10-06T05:22:33.158912+0000","last_peered":"2023-10-06T05:22:33.158912+0000","last_clean":"2023-10-06T05:22:33.158912+0000","last_became_active":"2023-10-06T05:22:33.153729+0000","last_became_peered":"2023-10-06T05:22:33.153729+0000","last_unstale":"2023-10-06T05:22:33.158912+0000","last_undegraded":"2023-10-06T05:22:33.158912+0000","last_fullsized":"2023-10-06T05:22:33.158912+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:50.492107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.167116+0000","last_change":"2023-10-06T05:22:33.160582+0000","last_active":"2023-10-06T05:22:33.167116+0000","last_peered":"2023-10-06T05:22:33.167116+0000","last_clean":"2023-10-06T05:22:33.167116+0000","last_became_active":"2023-10-06T05:22:33.160070+0000","last_became_peered":"2023-10-06T05:22:33.160070+0000","last_unstale":"2023-10-06T05:22:33.167116+0000","last_undegraded":"2023-10-06T05:22:33.167116+0000","last_fullsized":"2023-10-06T05:22:33.167116+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:03.037220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174807+0000","last_change":"2023-10-06T05:22:33.169931+0000","last_active":"2023-10-06T05:22:33.174807+0000","last_peered":"2023-10-06T05:22:33.174807+0000","last_clean":"2023-10-06T05:22:33.174807+0000","last_became_active":"2023-10-06T05:22:33.169681+0000","last_became_peered":"2023-10-06T05:22:33.169681+0000","last_unstale":"2023-10-06T05:22:33.174807+0000","last_undegraded":"2023-10-06T05:22:33.174807+0000","last_fullsized":"2023-10-06T05:22:33.174807+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:37.422324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176544+0000","last_change":"2023-10-06T05:22:33.171532+0000","last_active":"2023-10-06T05:22:33.176544+0000","last_peered":"2023-10-06T05:22:33.176544+0000","last_clean":"2023-10-06T05:22:33.176544+0000","last_became_active":"2023-10-06T05:22:33.170899+0000","last_became_peered":"2023-10-06T05:22:33.170899+0000","last_unstale":"2023-10-06T05:22:33.176544+0000","last_undegraded":"2023-10-06T05:22:33.176544+0000","last_fullsized":"2023-10-06T05:22:33.176544+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:39.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164248+0000","last_change":"2023-10-06T05:22:33.159089+0000","last_active":"2023-10-06T05:22:33.164248+0000","last_peered":"2023-10-06T05:22:33.164248+0000","last_clean":"2023-10-06T05:22:33.164248+0000","last_became_active":"2023-10-06T05:22:33.158807+0000","last_became_peered":"2023-10-06T05:22:33.158807+0000","last_unstale":"2023-10-06T05:22:33.164248+0000","last_undegraded":"2023-10-06T05:22:33.164248+0000","last_fullsized":"2023-10-06T05:22:33.164248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:07.764539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"100'4","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.177420+0000","last_change":"2023-10-06T05:22:32.148571+0000","last_active":"2023-10-06T05:22:33.177420+0000","last_peered":"2023-10-06T05:22:33.177420+0000","last_clean":"2023-10-06T05:22:33.177420+0000","last_became_active":"2023-10-06T05:22:32.148269+0000","last_became_peered":"2023-10-06T05:22:32.148269+0000","last_unstale":"2023-10-06T05:22:33.177420+0000","last_undegraded":"2023-10-06T05:22:33.177420+0000","last_fullsized":"2023-10-06T05:22:33.177420+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:51.971958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.161294+0000","last_change":"2023-10-06T05:22:33.158395+0000","last_active":"2023-10-06T05:22:33.161294+0000","last_peered":"2023-10-06T05:22:33.161294+0000","last_clean":"2023-10-06T05:22:33.161294+0000","last_became_active":"2023-10-06T05:22:33.158085+0000","last_became_peered":"2023-10-06T05:22:33.158085+0000","last_unstale":"2023-10-06T05:22:33.161294+0000","last_undegraded":"2023-10-06T05:22:33.161294+0000","last_fullsized":"2023-10-06T05:22:33.161294+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:29.806530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.157811+0000","last_change":"2023-10-06T05:22:32.144559+0000","last_active":"2023-10-06T05:22:33.157811+0000","last_peered":"2023-10-06T05:22:33.157811+0000","last_clean":"2023-10-06T05:22:33.157811+0000","last_became_active":"2023-10-06T05:22:32.144305+0000","last_became_peered":"2023-10-06T05:22:32.144305+0000","last_unstale":"2023-10-06T05:22:33.157811+0000","last_undegraded":"2023-10-06T05:22:33.157811+0000","last_fullsized":"2023-10-06T05:22:33.157811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:21.849427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174686+0000","last_change":"2023-10-06T05:22:33.169270+0000","last_active":"2023-10-06T05:22:33.174686+0000","last_peered":"2023-10-06T05:22:33.174686+0000","last_clean":"2023-10-06T05:22:33.174686+0000","last_became_active":"2023-10-06T05:22:33.168910+0000","last_became_peered":"2023-10-06T05:22:33.168910+0000","last_unstale":"2023-10-06T05:22:33.174686+0000","last_undegraded":"2023-10-06T05:22:33.174686+0000","last_fullsized":"2023-10-06T05:22:33.174686+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:41.548531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.160875+0000","last_change":"2023-10-06T05:22:32.150422+0000","last_active":"2023-10-06T05:22:33.160875+0000","last_peered":"2023-10-06T05:22:33.160875+0000","last_clean":"2023-10-06T05:22:33.160875+0000","last_became_active":"2023-10-06T05:22:32.150114+0000","last_became_peered":"2023-10-06T05:22:32.150114+0000","last_unstale":"2023-10-06T05:22:33.160875+0000","last_undegraded":"2023-10-06T05:22:33.160875+0000","last_fullsized":"2023-10-06T05:22:33.160875+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:03.439807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176821+0000","last_change":"2023-10-06T05:22:33.167630+0000","last_active":"2023-10-06T05:22:33.176821+0000","last_peered":"2023-10-06T05:22:33.176821+0000","last_clean":"2023-10-06T05:22:33.176821+0000","last_became_active":"2023-10-06T05:22:33.167338+0000","last_became_peered":"2023-10-06T05:22:33.167338+0000","last_unstale":"2023-10-06T05:22:33.176821+0000","last_undegraded":"2023-10-06T05:22:33.176821+0000","last_fullsized":"2023-10-06T05:22:33.176821+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:37.101927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.161507+0000","last_change":"2023-10-06T05:22:32.146346+0000","last_active":"2023-10-06T05:22:33.161507+0000","last_peered":"2023-10-06T05:22:33.161507+0000","last_clean":"2023-10-06T05:22:33.161507+0000","last_became_active":"2023-10-06T05:22:32.146089+0000","last_became_peered":"2023-10-06T05:22:32.146089+0000","last_unstale":"2023-10-06T05:22:33.161507+0000","last_undegraded":"2023-10-06T05:22:33.161507+0000","last_fullsized":"2023-10-06T05:22:33.161507+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:14.055773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.170240+0000","last_change":"2023-10-06T05:22:33.166279+0000","last_active":"2023-10-06T05:22:33.170240+0000","last_peered":"2023-10-06T05:22:33.170240+0000","last_clean":"2023-10-06T05:22:33.170240+0000","last_became_active":"2023-10-06T05:22:33.166045+0000","last_became_peered":"2023-10-06T05:22:33.166045+0000","last_unstale":"2023-10-06T05:22:33.170240+0000","last_undegraded":"2023-10-06T05:22:33.170240+0000","last_fullsized":"2023-10-06T05:22:33.170240+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:08.993634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163931+0000","last_change":"2023-10-06T05:22:32.151688+0000","last_active":"2023-10-06T05:22:33.163931+0000","last_peered":"2023-10-06T05:22:33.163931+0000","last_clean":"2023-10-06T05:22:33.163931+0000","last_became_active":"2023-10-06T05:22:32.151325+0000","last_became_peered":"2023-10-06T05:22:32.151325+0000","last_unstale":"2023-10-06T05:22:33.163931+0000","last_undegraded":"2023-10-06T05:22:33.163931+0000","last_fullsized":"2023-10-06T05:22:33.163931+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:05.805032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.169932+0000","last_change":"2023-10-06T05:22:33.162619+0000","last_active":"2023-10-06T05:22:33.169932+0000","last_peered":"2023-10-06T05:22:33.169932+0000","last_clean":"2023-10-06T05:22:33.169932+0000","last_became_active":"2023-10-06T05:22:33.162339+0000","last_became_peered":"2023-10-06T05:22:33.162339+0000","last_unstale":"2023-10-06T05:22:33.169932+0000","last_undegraded":"2023-10-06T05:22:33.169932+0000","last_fullsized":"2023-10-06T05:22:33.169932+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:39.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.158097+0000","last_change":"2023-10-06T05:22:32.154100+0000","last_active":"2023-10-06T05:22:33.158097+0000","last_peered":"2023-10-06T05:22:33.158097+0000","last_clean":"2023-10-06T05:22:33.158097+0000","last_became_active":"2023-10-06T05:22:32.153791+0000","last_became_peered":"2023-10-06T05:22:32.153791+0000","last_unstale":"2023-10-06T05:22:33.158097+0000","last_undegraded":"2023-10-06T05:22:33.158097+0000","last_fullsized":"2023-10-06T05:22:33.158097+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:19.502941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162583+0000","last_change":"2023-10-06T05:22:33.157803+0000","last_active":"2023-10-06T05:22:33.162583+0000","last_peered":"2023-10-06T05:22:33.162583+0000","last_clean":"2023-10-06T05:22:33.162583+0000","last_became_active":"2023-10-06T05:22:33.157518+0000","last_became_peered":"2023-10-06T05:22:33.157518+0000","last_unstale":"2023-10-06T05:22:33.162583+0000","last_undegraded":"2023-10-06T05:22:33.162583+0000","last_fullsized":"2023-10-06T05:22:33.162583+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:57.482645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.156189+0000","last_change":"2023-10-06T05:22:32.148843+0000","last_active":"2023-10-06T05:22:33.156189+0000","last_peered":"2023-10-06T05:22:33.156189+0000","last_clean":"2023-10-06T05:22:33.156189+0000","last_became_active":"2023-10-06T05:22:32.147791+0000","last_became_peered":"2023-10-06T05:22:32.147791+0000","last_unstale":"2023-10-06T05:22:33.156189+0000","last_undegraded":"2023-10-06T05:22:33.156189+0000","last_fullsized":"2023-10-06T05:22:33.156189+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:07.377546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164160+0000","last_change":"2023-10-06T05:22:33.161428+0000","last_active":"2023-10-06T05:22:33.164160+0000","last_peered":"2023-10-06T05:22:33.164160+0000","last_clean":"2023-10-06T05:22:33.164160+0000","last_became_active":"2023-10-06T05:22:33.161119+0000","last_became_peered":"2023-10-06T05:22:33.161119+0000","last_unstale":"2023-10-06T05:22:33.164160+0000","last_undegraded":"2023-10-06T05:22:33.164160+0000","last_fullsized":"2023-10-06T05:22:33.164160+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:20.803230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.139636+0000","last_change":"2023-10-06T05:22:32.153974+0000","last_active":"2023-10-06T05:22:33.139636+0000","last_peered":"2023-10-06T05:22:33.139636+0000","last_clean":"2023-10-06T05:22:33.139636+0000","last_became_active":"2023-10-06T05:22:32.153624+0000","last_became_peered":"2023-10-06T05:22:32.153624+0000","last_unstale":"2023-10-06T05:22:33.139636+0000","last_undegraded":"2023-10-06T05:22:33.139636+0000","last_fullsized":"2023-10-06T05:22:33.139636+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:27.023255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174461+0000","last_change":"2023-10-06T05:22:33.169639+0000","last_active":"2023-10-06T05:22:33.174461+0000","last_peered":"2023-10-06T05:22:33.174461+0000","last_clean":"2023-10-06T05:22:33.174461+0000","last_became_active":"2023-10-06T05:22:33.169345+0000","last_became_peered":"2023-10-06T05:22:33.169345+0000","last_unstale":"2023-10-06T05:22:33.174461+0000","last_undegraded":"2023-10-06T05:22:33.174461+0000","last_fullsized":"2023-10-06T05:22:33.174461+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:04.445876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.160611+0000","last_change":"2023-10-06T05:22:32.154420+0000","last_active":"2023-10-06T05:22:33.160611+0000","last_peered":"2023-10-06T05:22:33.160611+0000","last_clean":"2023-10-06T05:22:33.160611+0000","last_became_active":"2023-10-06T05:22:32.154155+0000","last_became_peered":"2023-10-06T05:22:32.154155+0000","last_unstale":"2023-10-06T05:22:33.160611+0000","last_undegraded":"2023-10-06T05:22:33.160611+0000","last_fullsized":"2023-10-06T05:22:33.160611+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:19.674825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.158367+0000","last_change":"2023-10-06T05:22:33.154681+0000","last_active":"2023-10-06T05:22:33.158367+0000","last_peered":"2023-10-06T05:22:33.158367+0000","last_clean":"2023-10-06T05:22:33.158367+0000","last_became_active":"2023-10-06T05:22:33.154394+0000","last_became_peered":"2023-10-06T05:22:33.154394+0000","last_unstale":"2023-10-06T05:22:33.158367+0000","last_undegraded":"2023-10-06T05:22:33.158367+0000","last_fullsized":"2023-10-06T05:22:33.158367+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:41.548531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.157144+0000","last_change":"2023-10-06T05:22:32.149516+0000","last_active":"2023-10-06T05:22:33.157144+0000","last_peered":"2023-10-06T05:22:33.157144+0000","last_clean":"2023-10-06T05:22:33.157144+0000","last_became_active":"2023-10-06T05:22:32.149219+0000","last_became_peered":"2023-10-06T05:22:32.149219+0000","last_unstale":"2023-10-06T05:22:33.157144+0000","last_undegraded":"2023-10-06T05:22:33.157144+0000","last_fullsized":"2023-10-06T05:22:33.157144+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:58.727736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163104+0000","last_change":"2023-10-06T05:22:33.159501+0000","last_active":"2023-10-06T05:22:33.163104+0000","last_peered":"2023-10-06T05:22:33.163104+0000","last_clean":"2023-10-06T05:22:33.163104+0000","last_became_active":"2023-10-06T05:22:33.157861+0000","last_became_peered":"2023-10-06T05:22:33.157861+0000","last_unstale":"2023-10-06T05:22:33.163104+0000","last_undegraded":"2023-10-06T05:22:33.163104+0000","last_fullsized":"2023-10-06T05:22:33.163104+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:14.720386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.173338+0000","last_change":"2023-10-06T05:22:32.156667+0000","last_active":"2023-10-06T05:22:33.173338+0000","last_peered":"2023-10-06T05:22:33.173338+0000","last_clean":"2023-10-06T05:22:33.173338+0000","last_became_active":"2023-10-06T05:22:32.156220+0000","last_became_peered":"2023-10-06T05:22:32.156220+0000","last_unstale":"2023-10-06T05:22:33.173338+0000","last_undegraded":"2023-10-06T05:22:33.173338+0000","last_fullsized":"2023-10-06T05:22:33.173338+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:28.929366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.177598+0000","last_change":"2023-10-06T05:22:33.175414+0000","last_active":"2023-10-06T05:22:33.177598+0000","last_peered":"2023-10-06T05:22:33.177598+0000","last_clean":"2023-10-06T05:22:33.177598+0000","last_became_active":"2023-10-06T05:22:33.175128+0000","last_became_peered":"2023-10-06T05:22:33.175128+0000","last_unstale":"2023-10-06T05:22:33.177598+0000","last_undegraded":"2023-10-06T05:22:33.177598+0000","last_fullsized":"2023-10-06T05:22:33.177598+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:04.496995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.144462+0000","last_change":"2023-10-06T05:22:32.154414+0000","last_active":"2023-10-06T05:22:33.144462+0000","last_peered":"2023-10-06T05:22:33.144462+0000","last_clean":"2023-10-06T05:22:33.144462+0000","last_became_active":"2023-10-06T05:22:32.154112+0000","last_became_peered":"2023-10-06T05:22:32.154112+0000","last_unstale":"2023-10-06T05:22:33.144462+0000","last_undegraded":"2023-10-06T05:22:33.144462+0000","last_fullsized":"2023-10-06T05:22:33.144462+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:24.975434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162917+0000","last_change":"2023-10-06T05:22:33.158930+0000","last_active":"2023-10-06T05:22:33.162917+0000","last_peered":"2023-10-06T05:22:33.162917+0000","last_clean":"2023-10-06T05:22:33.162917+0000","last_became_active":"2023-10-06T05:22:33.158679+0000","last_became_peered":"2023-10-06T05:22:33.158679+0000","last_unstale":"2023-10-06T05:22:33.162917+0000","last_undegraded":"2023-10-06T05:22:33.162917+0000","last_fullsized":"2023-10-06T05:22:33.162917+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:29.294874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164993+0000","last_change":"2023-10-06T05:22:32.149992+0000","last_active":"2023-10-06T05:22:33.164993+0000","last_peered":"2023-10-06T05:22:33.164993+0000","last_clean":"2023-10-06T05:22:33.164993+0000","last_became_active":"2023-10-06T05:22:32.149651+0000","last_became_peered":"2023-10-06T05:22:32.149651+0000","last_unstale":"2023-10-06T05:22:33.164993+0000","last_undegraded":"2023-10-06T05:22:33.164993+0000","last_fullsized":"2023-10-06T05:22:33.164993+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:34.103173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.175311+0000","last_change":"2023-10-06T05:22:33.172314+0000","last_active":"2023-10-06T05:22:33.175311+0000","last_peered":"2023-10-06T05:22:33.175311+0000","last_clean":"2023-10-06T05:22:33.175311+0000","last_became_active":"2023-10-06T05:22:33.172055+0000","last_became_peered":"2023-10-06T05:22:33.172055+0000","last_unstale":"2023-10-06T05:22:33.175311+0000","last_undegraded":"2023-10-06T05:22:33.175311+0000","last_fullsized":"2023-10-06T05:22:33.175311+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:37.101927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.144394+0000","last_change":"2023-10-06T05:22:32.156462+0000","last_active":"2023-10-06T05:22:33.144394+0000","last_peered":"2023-10-06T05:22:33.144394+0000","last_clean":"2023-10-06T05:22:33.144394+0000","last_became_active":"2023-10-06T05:22:32.156125+0000","last_became_peered":"2023-10-06T05:22:32.156125+0000","last_unstale":"2023-10-06T05:22:33.144394+0000","last_undegraded":"2023-10-06T05:22:33.144394+0000","last_fullsized":"2023-10-06T05:22:33.144394+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:36.416254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162548+0000","last_change":"2023-10-06T05:22:33.155802+0000","last_active":"2023-10-06T05:22:33.162548+0000","last_peered":"2023-10-06T05:22:33.162548+0000","last_clean":"2023-10-06T05:22:33.162548+0000","last_became_active":"2023-10-06T05:22:33.155191+0000","last_became_peered":"2023-10-06T05:22:33.155191+0000","last_unstale":"2023-10-06T05:22:33.162548+0000","last_undegraded":"2023-10-06T05:22:33.162548+0000","last_fullsized":"2023-10-06T05:22:33.162548+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:01.452838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.141607+0000","last_change":"2023-10-06T05:22:32.151440+0000","last_active":"2023-10-06T05:22:33.141607+0000","last_peered":"2023-10-06T05:22:33.141607+0000","last_clean":"2023-10-06T05:22:33.141607+0000","last_became_active":"2023-10-06T05:22:32.151139+0000","last_became_peered":"2023-10-06T05:22:32.151139+0000","last_unstale":"2023-10-06T05:22:33.141607+0000","last_undegraded":"2023-10-06T05:22:33.141607+0000","last_fullsized":"2023-10-06T05:22:33.141607+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:38.195603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.171971+0000","last_change":"2023-10-06T05:22:33.167483+0000","last_active":"2023-10-06T05:22:33.171971+0000","last_peered":"2023-10-06T05:22:33.171971+0000","last_clean":"2023-10-06T05:22:33.171971+0000","last_became_active":"2023-10-06T05:22:33.167133+0000","last_became_peered":"2023-10-06T05:22:33.167133+0000","last_unstale":"2023-10-06T05:22:33.171971+0000","last_undegraded":"2023-10-06T05:22:33.171971+0000","last_fullsized":"2023-10-06T05:22:33.171971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:48.296249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.145461+0000","last_change":"2023-10-06T05:22:32.150342+0000","last_active":"2023-10-06T05:22:33.145461+0000","last_peered":"2023-10-06T05:22:33.145461+0000","last_clean":"2023-10-06T05:22:33.145461+0000","last_became_active":"2023-10-06T05:22:32.150102+0000","last_became_peered":"2023-10-06T05:22:32.150102+0000","last_unstale":"2023-10-06T05:22:33.145461+0000","last_undegraded":"2023-10-06T05:22:33.145461+0000","last_fullsized":"2023-10-06T05:22:33.145461+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:40.542462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162156+0000","last_change":"2023-10-06T05:22:33.158390+0000","last_active":"2023-10-06T05:22:33.162156+0000","last_peered":"2023-10-06T05:22:33.162156+0000","last_clean":"2023-10-06T05:22:33.162156+0000","last_became_active":"2023-10-06T05:22:33.158109+0000","last_became_peered":"2023-10-06T05:22:33.158109+0000","last_unstale":"2023-10-06T05:22:33.162156+0000","last_undegraded":"2023-10-06T05:22:33.162156+0000","last_fullsized":"2023-10-06T05:22:33.162156+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:59.627977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.157221+0000","last_change":"2023-10-06T05:22:32.156374+0000","last_active":"2023-10-06T05:22:33.157221+0000","last_peered":"2023-10-06T05:22:33.157221+0000","last_clean":"2023-10-06T05:22:33.157221+0000","last_became_active":"2023-10-06T05:22:32.156019+0000","last_became_peered":"2023-10-06T05:22:32.156019+0000","last_unstale":"2023-10-06T05:22:33.157221+0000","last_undegraded":"2023-10-06T05:22:33.157221+0000","last_fullsized":"2023-10-06T05:22:33.157221+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:18.205665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.170171+0000","last_change":"2023-10-06T05:22:33.165787+0000","last_active":"2023-10-06T05:22:33.170171+0000","last_peered":"2023-10-06T05:22:33.170171+0000","last_clean":"2023-10-06T05:22:33.170171+0000","last_became_active":"2023-10-06T05:22:33.165596+0000","last_became_peered":"2023-10-06T05:22:33.165596+0000","last_unstale":"2023-10-06T05:22:33.170171+0000","last_undegraded":"2023-10-06T05:22:33.170171+0000","last_fullsized":"2023-10-06T05:22:33.170171+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:06.811102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.142393+0000","last_change":"2023-10-06T05:22:32.154455+0000","last_active":"2023-10-06T05:22:33.142393+0000","last_peered":"2023-10-06T05:22:33.142393+0000","last_clean":"2023-10-06T05:22:33.142393+0000","last_became_active":"2023-10-06T05:22:32.154097+0000","last_became_peered":"2023-10-06T05:22:32.154097+0000","last_unstale":"2023-10-06T05:22:33.142393+0000","last_undegraded":"2023-10-06T05:22:33.142393+0000","last_fullsized":"2023-10-06T05:22:33.142393+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:21.480112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176695+0000","last_change":"2023-10-06T05:22:33.167095+0000","last_active":"2023-10-06T05:22:33.176695+0000","last_peered":"2023-10-06T05:22:33.176695+0000","last_clean":"2023-10-06T05:22:33.176695+0000","last_became_active":"2023-10-06T05:22:33.166702+0000","last_became_peered":"2023-10-06T05:22:33.166702+0000","last_unstale":"2023-10-06T05:22:33.176695+0000","last_undegraded":"2023-10-06T05:22:33.176695+0000","last_fullsized":"2023-10-06T05:22:33.176695+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:29.935418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.142814+0000","last_change":"2023-10-06T05:22:32.154673+0000","last_active":"2023-10-06T05:22:33.142814+0000","last_peered":"2023-10-06T05:22:33.142814+0000","last_clean":"2023-10-06T05:22:33.142814+0000","last_became_active":"2023-10-06T05:22:32.153966+0000","last_became_peered":"2023-10-06T05:22:32.153966+0000","last_unstale":"2023-10-06T05:22:33.142814+0000","last_undegraded":"2023-10-06T05:22:33.142814+0000","last_fullsized":"2023-10-06T05:22:33.142814+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:54.532757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.175362+0000","last_change":"2023-10-06T05:22:33.172615+0000","last_active":"2023-10-06T05:22:33.175362+0000","last_peered":"2023-10-06T05:22:33.175362+0000","last_clean":"2023-10-06T05:22:33.175362+0000","last_became_active":"2023-10-06T05:22:33.172142+0000","last_became_peered":"2023-10-06T05:22:33.172142+0000","last_unstale":"2023-10-06T05:22:33.175362+0000","last_undegraded":"2023-10-06T05:22:33.175362+0000","last_fullsized":"2023-10-06T05:22:33.175362+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:04.346901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.140004+0000","last_change":"2023-10-06T05:22:32.150065+0000","last_active":"2023-10-06T05:22:33.140004+0000","last_peered":"2023-10-06T05:22:33.140004+0000","last_clean":"2023-10-06T05:22:33.140004+0000","last_became_active":"2023-10-06T05:22:32.149757+0000","last_became_peered":"2023-10-06T05:22:32.149757+0000","last_unstale":"2023-10-06T05:22:33.140004+0000","last_undegraded":"2023-10-06T05:22:33.140004+0000","last_fullsized":"2023-10-06T05:22:33.140004+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:14.202372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.156534+0000","last_change":"2023-10-06T05:22:33.151471+0000","last_active":"2023-10-06T05:22:33.156534+0000","last_peered":"2023-10-06T05:22:33.156534+0000","last_clean":"2023-10-06T05:22:33.156534+0000","last_became_active":"2023-10-06T05:22:33.151193+0000","last_became_peered":"2023-10-06T05:22:33.151193+0000","last_unstale":"2023-10-06T05:22:33.156534+0000","last_undegraded":"2023-10-06T05:22:33.156534+0000","last_fullsized":"2023-10-06T05:22:33.156534+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:54.647785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.139476+0000","last_change":"2023-10-06T05:22:32.151554+0000","last_active":"2023-10-06T05:22:33.139476+0000","last_peered":"2023-10-06T05:22:33.139476+0000","last_clean":"2023-10-06T05:22:33.139476+0000","last_became_active":"2023-10-06T05:22:32.151238+0000","last_became_peered":"2023-10-06T05:22:32.151238+0000","last_unstale":"2023-10-06T05:22:33.139476+0000","last_undegraded":"2023-10-06T05:22:33.139476+0000","last_fullsized":"2023-10-06T05:22:33.139476+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:43.341357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162421+0000","last_change":"2023-10-06T05:22:33.155077+0000","last_active":"2023-10-06T05:22:33.162421+0000","last_peered":"2023-10-06T05:22:33.162421+0000","last_clean":"2023-10-06T05:22:33.162421+0000","last_became_active":"2023-10-06T05:22:33.154816+0000","last_became_peered":"2023-10-06T05:22:33.154816+0000","last_unstale":"2023-10-06T05:22:33.162421+0000","last_undegraded":"2023-10-06T05:22:33.162421+0000","last_fullsized":"2023-10-06T05:22:33.162421+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:23.892518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.137887+0000","last_change":"2023-10-06T05:22:32.152394+0000","last_active":"2023-10-06T05:22:33.137887+0000","last_peered":"2023-10-06T05:22:33.137887+0000","last_clean":"2023-10-06T05:22:33.137887+0000","last_became_active":"2023-10-06T05:22:32.152058+0000","last_became_peered":"2023-10-06T05:22:32.152058+0000","last_unstale":"2023-10-06T05:22:33.137887+0000","last_undegraded":"2023-10-06T05:22:33.137887+0000","last_fullsized":"2023-10-06T05:22:33.137887+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:37.158566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.167639+0000","last_change":"2023-10-06T05:22:33.160995+0000","last_active":"2023-10-06T05:22:33.167639+0000","last_peered":"2023-10-06T05:22:33.167639+0000","last_clean":"2023-10-06T05:22:33.167639+0000","last_became_active":"2023-10-06T05:22:33.160472+0000","last_became_peered":"2023-10-06T05:22:33.160472+0000","last_unstale":"2023-10-06T05:22:33.167639+0000","last_undegraded":"2023-10-06T05:22:33.167639+0000","last_fullsized":"2023-10-06T05:22:33.167639+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:07.085636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.139264+0000","last_change":"2023-10-06T05:22:32.150623+0000","last_active":"2023-10-06T05:22:33.139264+0000","last_peered":"2023-10-06T05:22:33.139264+0000","last_clean":"2023-10-06T05:22:33.139264+0000","last_became_active":"2023-10-06T05:22:32.150334+0000","last_became_peered":"2023-10-06T05:22:32.150334+0000","last_unstale":"2023-10-06T05:22:33.139264+0000","last_undegraded":"2023-10-06T05:22:33.139264+0000","last_fullsized":"2023-10-06T05:22:33.139264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:02.031150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164746+0000","last_change":"2023-10-06T05:22:33.163172+0000","last_active":"2023-10-06T05:22:33.164746+0000","last_peered":"2023-10-06T05:22:33.164746+0000","last_clean":"2023-10-06T05:22:33.164746+0000","last_became_active":"2023-10-06T05:22:33.162820+0000","last_became_peered":"2023-10-06T05:22:33.162820+0000","last_unstale":"2023-10-06T05:22:33.164746+0000","last_undegraded":"2023-10-06T05:22:33.164746+0000","last_fullsized":"2023-10-06T05:22:33.164746+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:03.639154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.144947+0000","last_change":"2023-10-06T05:22:32.155107+0000","last_active":"2023-10-06T05:22:33.144947+0000","last_peered":"2023-10-06T05:22:33.144947+0000","last_clean":"2023-10-06T05:22:33.144947+0000","last_became_active":"2023-10-06T05:22:32.154771+0000","last_became_peered":"2023-10-06T05:22:32.154771+0000","last_unstale":"2023-10-06T05:22:33.144947+0000","last_undegraded":"2023-10-06T05:22:33.144947+0000","last_fullsized":"2023-10-06T05:22:33.144947+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:03.490926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.169475+0000","last_change":"2023-10-06T05:22:33.164494+0000","last_active":"2023-10-06T05:22:33.169475+0000","last_peered":"2023-10-06T05:22:33.169475+0000","last_clean":"2023-10-06T05:22:33.169475+0000","last_became_active":"2023-10-06T05:22:33.164190+0000","last_became_peered":"2023-10-06T05:22:33.164190+0000","last_unstale":"2023-10-06T05:22:33.169475+0000","last_undegraded":"2023-10-06T05:22:33.169475+0000","last_fullsized":"2023-10-06T05:22:33.169475+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:55.243447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.142424+0000","last_change":"2023-10-06T05:22:32.154271+0000","last_active":"2023-10-06T05:22:33.142424+0000","last_peered":"2023-10-06T05:22:33.142424+0000","last_clean":"2023-10-06T05:22:33.142424+0000","last_became_active":"2023-10-06T05:22:32.154002+0000","last_became_peered":"2023-10-06T05:22:32.154002+0000","last_unstale":"2023-10-06T05:22:33.142424+0000","last_undegraded":"2023-10-06T05:22:33.142424+0000","last_fullsized":"2023-10-06T05:22:33.142424+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:56.590205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.142525+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:22:33.142525+0000","last_peered":"2023-10-06T05:22:33.142525+0000","last_clean":"2023-10-06T05:22:33.142525+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:22:33.142525+0000","last_undegraded":"2023-10-06T05:22:33.142525+0000","last_fullsized":"2023-10-06T05:22:33.142525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174517+0000","last_change":"2023-10-06T05:22:33.168666+0000","last_active":"2023-10-06T05:22:33.174517+0000","last_peered":"2023-10-06T05:22:33.174517+0000","last_clean":"2023-10-06T05:22:33.174517+0000","last_became_active":"2023-10-06T05:22:33.168309+0000","last_became_peered":"2023-10-06T05:22:33.168309+0000","last_unstale":"2023-10-06T05:22:33.174517+0000","last_undegraded":"2023-10-06T05:22:33.174517+0000","last_fullsized":"2023-10-06T05:22:33.174517+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:16.079961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.157292+0000","last_change":"2023-10-06T05:22:32.145009+0000","last_active":"2023-10-06T05:22:33.157292+0000","last_peered":"2023-10-06T05:22:33.157292+0000","last_clean":"2023-10-06T05:22:33.157292+0000","last_became_active":"2023-10-06T05:22:32.144659+0000","last_became_peered":"2023-10-06T05:22:32.144659+0000","last_unstale":"2023-10-06T05:22:33.157292+0000","last_undegraded":"2023-10-06T05:22:33.157292+0000","last_fullsized":"2023-10-06T05:22:33.157292+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:07.987565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.142507+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:22:33.142507+0000","last_peered":"2023-10-06T05:22:33.142507+0000","last_clean":"2023-10-06T05:22:33.142507+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:22:33.142507+0000","last_undegraded":"2023-10-06T05:22:33.142507+0000","last_fullsized":"2023-10-06T05:22:33.142507+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.153371+0000","last_change":"2023-10-06T05:22:33.144576+0000","last_active":"2023-10-06T05:22:33.153371+0000","last_peered":"2023-10-06T05:22:33.153371+0000","last_clean":"2023-10-06T05:22:33.153371+0000","last_became_active":"2023-10-06T05:22:33.144237+0000","last_became_peered":"2023-10-06T05:22:33.144237+0000","last_unstale":"2023-10-06T05:22:33.153371+0000","last_undegraded":"2023-10-06T05:22:33.153371+0000","last_fullsized":"2023-10-06T05:22:33.153371+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:27.815053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.141473+0000","last_change":"2023-10-06T05:22:32.152796+0000","last_active":"2023-10-06T05:22:33.141473+0000","last_peered":"2023-10-06T05:22:33.141473+0000","last_clean":"2023-10-06T05:22:33.141473+0000","last_became_active":"2023-10-06T05:22:32.152557+0000","last_became_peered":"2023-10-06T05:22:32.152557+0000","last_unstale":"2023-10-06T05:22:33.141473+0000","last_undegraded":"2023-10-06T05:22:33.141473+0000","last_fullsized":"2023-10-06T05:22:33.141473+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:36.416254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.153186+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:22:33.153186+0000","last_peered":"2023-10-06T05:22:33.153186+0000","last_clean":"2023-10-06T05:22:33.153186+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:22:33.153186+0000","last_undegraded":"2023-10-06T05:22:33.153186+0000","last_fullsized":"2023-10-06T05:22:33.153186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174632+0000","last_change":"2023-10-06T05:22:33.162318+0000","last_active":"2023-10-06T05:22:33.174632+0000","last_peered":"2023-10-06T05:22:33.174632+0000","last_clean":"2023-10-06T05:22:33.174632+0000","last_became_active":"2023-10-06T05:22:33.162005+0000","last_became_peered":"2023-10-06T05:22:33.162005+0000","last_unstale":"2023-10-06T05:22:33.174632+0000","last_undegraded":"2023-10-06T05:22:33.174632+0000","last_fullsized":"2023-10-06T05:22:33.174632+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:57.486209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"100'8","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174880+0000","last_change":"2023-10-06T05:22:32.148896+0000","last_active":"2023-10-06T05:22:33.174880+0000","last_peered":"2023-10-06T05:22:33.174880+0000","last_clean":"2023-10-06T05:22:33.174880+0000","last_became_active":"2023-10-06T05:22:32.148625+0000","last_became_peered":"2023-10-06T05:22:32.148625+0000","last_unstale":"2023-10-06T05:22:33.174880+0000","last_undegraded":"2023-10-06T05:22:33.174880+0000","last_fullsized":"2023-10-06T05:22:33.174880+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:24.975434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.145280+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:22:33.145280+0000","last_peered":"2023-10-06T05:22:33.145280+0000","last_clean":"2023-10-06T05:22:33.145280+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:22:33.145280+0000","last_undegraded":"2023-10-06T05:22:33.145280+0000","last_fullsized":"2023-10-06T05:22:33.145280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164760+0000","last_change":"2023-10-06T05:22:33.163160+0000","last_active":"2023-10-06T05:22:33.164760+0000","last_peered":"2023-10-06T05:22:33.164760+0000","last_clean":"2023-10-06T05:22:33.164760+0000","last_became_active":"2023-10-06T05:22:33.162870+0000","last_became_peered":"2023-10-06T05:22:33.162870+0000","last_unstale":"2023-10-06T05:22:33.164760+0000","last_undegraded":"2023-10-06T05:22:33.164760+0000","last_fullsized":"2023-10-06T05:22:33.164760+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:38.164635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.143578+0000","last_change":"2023-10-06T05:22:32.146142+0000","last_active":"2023-10-06T05:22:33.143578+0000","last_peered":"2023-10-06T05:22:33.143578+0000","last_clean":"2023-10-06T05:22:33.143578+0000","last_became_active":"2023-10-06T05:22:32.145839+0000","last_became_peered":"2023-10-06T05:22:32.145839+0000","last_unstale":"2023-10-06T05:22:33.143578+0000","last_undegraded":"2023-10-06T05:22:33.143578+0000","last_fullsized":"2023-10-06T05:22:33.143578+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:28.929366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":78,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.144484+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:22:33.144484+0000","last_peered":"2023-10-06T05:22:33.144484+0000","last_clean":"2023-10-06T05:22:33.144484+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:22:33.144484+0000","last_undegraded":"2023-10-06T05:22:33.144484+0000","last_fullsized":"2023-10-06T05:22:33.144484+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.165216+0000","last_change":"2023-10-06T05:22:33.155778+0000","last_active":"2023-10-06T05:22:33.165216+0000","last_peered":"2023-10-06T05:22:33.165216+0000","last_clean":"2023-10-06T05:22:33.165216+0000","last_became_active":"2023-10-06T05:22:33.155425+0000","last_became_peered":"2023-10-06T05:22:33.155425+0000","last_unstale":"2023-10-06T05:22:33.165216+0000","last_undegraded":"2023-10-06T05:22:33.165216+0000","last_fullsized":"2023-10-06T05:22:33.165216+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:16.199208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164878+0000","last_change":"2023-10-06T05:22:32.148615+0000","last_active":"2023-10-06T05:22:33.164878+0000","last_peered":"2023-10-06T05:22:33.164878+0000","last_clean":"2023-10-06T05:22:33.164878+0000","last_became_active":"2023-10-06T05:22:32.148319+0000","last_became_peered":"2023-10-06T05:22:32.148319+0000","last_unstale":"2023-10-06T05:22:33.164878+0000","last_undegraded":"2023-10-06T05:22:33.164878+0000","last_fullsized":"2023-10-06T05:22:33.164878+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:30.230308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":267,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.145095+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:22:33.145095+0000","last_peered":"2023-10-06T05:22:33.145095+0000","last_clean":"2023-10-06T05:22:33.145095+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:22:33.145095+0000","last_undegraded":"2023-10-06T05:22:33.145095+0000","last_fullsized":"2023-10-06T05:22:33.145095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.146037+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:22:33.146037+0000","last_peered":"2023-10-06T05:22:33.146037+0000","last_clean":"2023-10-06T05:22:33.146037+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:22:33.146037+0000","last_undegraded":"2023-10-06T05:22:33.146037+0000","last_fullsized":"2023-10-06T05:22:33.146037+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174693+0000","last_change":"2023-10-06T05:22:33.170634+0000","last_active":"2023-10-06T05:22:33.174693+0000","last_peered":"2023-10-06T05:22:33.174693+0000","last_clean":"2023-10-06T05:22:33.174693+0000","last_became_active":"2023-10-06T05:22:33.170242+0000","last_became_peered":"2023-10-06T05:22:33.170242+0000","last_unstale":"2023-10-06T05:22:33.174693+0000","last_undegraded":"2023-10-06T05:22:33.174693+0000","last_fullsized":"2023-10-06T05:22:33.174693+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:25.981504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"100'6","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.177195+0000","last_change":"2023-10-06T05:22:32.152283+0000","last_active":"2023-10-06T05:22:33.177195+0000","last_peered":"2023-10-06T05:22:33.177195+0000","last_clean":"2023-10-06T05:22:33.177195+0000","last_became_active":"2023-10-06T05:22:32.151990+0000","last_became_peered":"2023-10-06T05:22:32.151990+0000","last_unstale":"2023-10-06T05:22:33.177195+0000","last_undegraded":"2023-10-06T05:22:33.177195+0000","last_fullsized":"2023-10-06T05:22:33.177195+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:47.290180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":94,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.141999+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:22:33.141999+0000","last_peered":"2023-10-06T05:22:33.141999+0000","last_clean":"2023-10-06T05:22:33.141999+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:22:33.141999+0000","last_undegraded":"2023-10-06T05:22:33.141999+0000","last_fullsized":"2023-10-06T05:22:33.141999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163343+0000","last_change":"2023-10-06T05:22:33.156007+0000","last_active":"2023-10-06T05:22:33.163343+0000","last_peered":"2023-10-06T05:22:33.163343+0000","last_clean":"2023-10-06T05:22:33.163343+0000","last_became_active":"2023-10-06T05:22:33.155723+0000","last_became_peered":"2023-10-06T05:22:33.155723+0000","last_unstale":"2023-10-06T05:22:33.163343+0000","last_undegraded":"2023-10-06T05:22:33.163343+0000","last_fullsized":"2023-10-06T05:22:33.163343+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:29.935418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"100'3","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.169043+0000","last_change":"2023-10-06T05:22:32.149317+0000","last_active":"2023-10-06T05:22:33.169043+0000","last_peered":"2023-10-06T05:22:33.169043+0000","last_clean":"2023-10-06T05:22:33.169043+0000","last_became_active":"2023-10-06T05:22:32.148874+0000","last_became_peered":"2023-10-06T05:22:32.148874+0000","last_unstale":"2023-10-06T05:22:33.169043+0000","last_undegraded":"2023-10-06T05:22:33.169043+0000","last_fullsized":"2023-10-06T05:22:33.169043+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:19.797161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.138062+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:22:33.138062+0000","last_peered":"2023-10-06T05:22:33.138062+0000","last_clean":"2023-10-06T05:22:33.138062+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:22:33.138062+0000","last_undegraded":"2023-10-06T05:22:33.138062+0000","last_fullsized":"2023-10-06T05:22:33.138062+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303208,"kb_used_data":1824,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068632,"statfs":{"total":96636764160,"available":96326279168,"internally_reserved":0,"allocated":1867776,"data_stored":1172835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.375,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93200000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.504},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93300000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.97599999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72999999999999998,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.51600000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82699999999999996,"5min":1.018,"15min":1.018},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71799999999999997,"15min":0.752},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88100000000000001,"5min":1.056,"15min":1.056},"last":0.68000000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.47699999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65300000000000002,"15min":0.71299999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81899999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72899999999999998,"15min":0.77900000000000003},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70499999999999996,"15min":0.73999999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97899999999999998,"5min":0.999,"15min":0.999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76500000000000001,"15min":0.83199999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.82399999999999995,"15min":0.88400000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.012,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81499999999999995,"15min":0.85099999999999998},"min":{"1min":0.59799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.86299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298520,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073320,"statfs":{"total":96636764160,"available":96331079680,"internally_reserved":0,"allocated":1261568,"data_stored":582273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.66900000000000004,"15min":0.68300000000000005},"min":{"1min":0.44400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80400000000000005,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.49299999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2170000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.504,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95899999999999996,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.48799999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.073,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.093,"5min":1.369,"15min":1.369},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73199999999999998,"15min":0.76300000000000001},"min":{"1min":0.58099999999999996,"5min":0.438,"15min":0.438},"max":{"1min":0.94199999999999995,"5min":1.891,"15min":1.891},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73299999999999998,"15min":0.751},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.018,"5min":1.347,"15min":1.347},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.58599999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98699999999999999,"5min":1,"15min":1},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72699999999999998,"15min":0.78300000000000003},"min":{"1min":0.47199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.1140000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.77600000000000002,"15min":0.82199999999999995},"min":{"1min":0.62,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.278,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.82999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88900000000000001,"5min":1.415,"15min":1.415},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.75800000000000001,"15min":0.82499999999999996},"min":{"1min":0.53400000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.008,"5min":2,"15min":2},"last":0.90200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76200000000000001,"15min":0.81000000000000005},"min":{"1min":0.53100000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0609999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.752,"15min":0.77800000000000002},"min":{"1min":0.56299999999999994,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99099999999999999,"5min":1.46,"15min":1.46},"last":1.0429999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303196,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068644,"statfs":{"total":96636764160,"available":96326291456,"internally_reserved":0,"allocated":1855488,"data_stored":1172109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58399999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.496},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.498,"15min":0.496},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60099999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46100000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69999999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.46899999999999997},"min":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.64200000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56200000000000006,"15min":0.52500000000000002},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78400000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.499,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68999999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0209999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83199999999999996,"15min":0.79900000000000004},"min":{"1min":0.65700000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.054,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77100000000000002,"15min":0.751},"min":{"1min":0.59799999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98499999999999999,"5min":1.099,"15min":1.099},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.624,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.997,"5min":1.181,"15min":1.181},"last":0.625}]},{"osd":6,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78200000000000003,"15min":0.754},"min":{"1min":0.57199999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98799999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.82499999999999996,"15min":0.80100000000000005},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.95299999999999996,"5min":1.123,"15min":1.123},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83099999999999996,"15min":0.82299999999999995},"min":{"1min":0.63600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83999999999999997,"15min":0.82899999999999996},"min":{"1min":0.58199999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98199999999999998,"5min":1.103,"15min":1.103},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298520,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073320,"statfs":{"total":96636764160,"available":96331079680,"internally_reserved":0,"allocated":1261568,"data_stored":581741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.29899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55200000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":1.042,"15min":1.042},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.43099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52000000000000002,"15min":0.49099999999999999},"min":{"1min":0.33200000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68200000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.46800000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70999999999999996,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45800000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77000000000000002,"15min":0.75600000000000001},"min":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95699999999999996,"5min":1.242,"15min":1.242},"last":0.498},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78200000000000003,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.499,"15min":0.499},"max":{"1min":0.92300000000000004,"5min":1.198,"15min":1.198},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78600000000000003,"15min":0.77200000000000002},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.90700000000000003,"5min":1.26,"15min":1.26},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86499999999999999,"5min":1.163,"15min":1.163},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94299999999999995,"5min":1.323,"15min":1.323},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.58099999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0049999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.006,"5min":1.264,"15min":1.264},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81799999999999995,"15min":0.83199999999999996},"min":{"1min":0.64700000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.004,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.57099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298516,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073324,"statfs":{"total":96636764160,"available":96331083776,"internally_reserved":0,"allocated":1257472,"data_stored":581719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.433,"15min":0.42899999999999999},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64600000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.34499999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.33700000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71699999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.498,"15min":0.502},"min":{"1min":0.34999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.54900000000000004},"min":{"1min":0.42199999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77300000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.5}]},{"osd":4,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.73999999999999999,"15min":0.75900000000000001},"min":{"1min":0.54700000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.022,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.752,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.085,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.82999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.75,"15min":0.78600000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.042,"5min":1.099,"15min":1.099},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.80600000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0049999999999999,"5min":1.163,"15min":1.163},"last":0.93700000000000006}]},{"osd":6,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79900000000000004,"15min":0.83899999999999997},"min":{"1min":0.58499999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.056,"5min":1.21,"15min":1.21},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79000000000000004,"15min":0.81699999999999995},"min":{"1min":0.58699999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.91800000000000004,"5min":1.042,"15min":1.042},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.78000000000000003,"15min":0.80700000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0609999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.88},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.82399999999999995,"15min":0.86599999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1080000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.84999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298532,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073308,"statfs":{"total":96636764160,"available":96331067392,"internally_reserved":0,"allocated":1273856,"data_stored":583289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.443,"15min":0.46100000000000002},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.495},"min":{"1min":0.316,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64500000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47999999999999998,"15min":0.47299999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.52100000000000002,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71499999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98999999999999999,"5min":1.014,"15min":1.014},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81399999999999995,"15min":0.81200000000000006},"min":{"1min":0.62,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1220000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.82899999999999996,"15min":0.83299999999999996},"min":{"1min":0.70499999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98399999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.58399999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.006,"5min":1.204,"15min":1.204},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.80600000000000005,"15min":0.83699999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0209999999999999,"5min":1.329,"15min":1.329},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79800000000000004,"15min":0.81899999999999995},"min":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0840000000000001,"5min":1.484,"15min":1.484},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.81799999999999995,"15min":0.84799999999999998},"min":{"1min":0.56999999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0529999999999999,"5min":1.629,"15min":1.629},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82499999999999996,"15min":0.85599999999999998},"min":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.137,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.81699999999999995,"15min":0.82999999999999996},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.194,"5min":1.325,"15min":1.325},"last":0.78600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298528,"kb_used_data":1240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073312,"statfs":{"total":96636764160,"available":96331071488,"internally_reserved":0,"allocated":1269760,"data_stored":583255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.47199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90600000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70899999999999996,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":0.92400000000000004,"5min":2.77,"15min":2.77},"last":0.82699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.69599999999999995,"15min":0.73799999999999999},"min":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92200000000000004,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69499999999999995,"15min":0.72199999999999998},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90100000000000002,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.495,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93999999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73899999999999999,"15min":0.82099999999999995},"min":{"1min":0.56000000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1120000000000001,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.83799999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.81599999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.80300000000000005,"15min":1.0569999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93999999999999995,"5min":7.742,"15min":7.742},"last":0.89500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69199999999999995,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96499999999999997,"5min":1.079,"15min":1.079},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.71099999999999997,"15min":0.75},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85799999999999998,"5min":1.181,"15min":1.181},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85499999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76800000000000002,"15min":0.82199999999999995},"min":{"1min":0.59599999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.95299999999999996,"5min":1.171,"15min":1.171},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.78800000000000003,"15min":0.83099999999999996},"min":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.024,"5min":1.228,"15min":1.228},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.78500000000000003,"15min":0.85899999999999999},"min":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.93500000000000005,"5min":1.571,"15min":1.571},"last":0.72599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298524,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073316,"statfs":{"total":96636764160,"available":96331075584,"internally_reserved":0,"allocated":1265664,"data_stored":581879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.73599999999999999,"15min":0.96799999999999997},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.45700000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.94699999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79800000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.77100000000000002,"15min":0.99299999999999999},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84699999999999998,"5min":9.25,"15min":9.25},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.79700000000000004,"15min":1.056},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85199999999999998,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.76700000000000002,"15min":1.026},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82299999999999995,"5min":10.384,"15min":10.384},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81499999999999995,"15min":1.046},"min":{"1min":0.54500000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88900000000000001,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.82399999999999995,"15min":1.089},"min":{"1min":0.53300000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.87,"5min":10.205,"15min":10.205},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82499999999999996,"5min":1.292,"15min":1.292},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.46200000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88700000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.72599999999999998,"15min":0.77200000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.036,"5min":1.171,"15min":1.171},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.73599999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88400000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.52000000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93999999999999995,"5min":1.298,"15min":1.298},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78600000000000003,"15min":0.82399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.107,"5min":1.204,"15min":1.204},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.54400000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93799999999999994,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.72299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:22:37.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:22:37.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.787+0000 7ff489dff700 1 Processor -- start 2023-10-06T05:22:37.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.787+0000 7ff489dff700 1 -- start start 2023-10-06T05:22:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.788+0000 7ff489dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff4841595a0 0x7ff484159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.788+0000 7ff489dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 0x7ff484068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.788+0000 7ff489dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff484068bb0 0x7ff4840c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.788+0000 7ff489dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff484069010 con 0x7ff484068bb0 2023-10-06T05:22:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.788+0000 7ff489dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4840c95b0 con 0x7ff4841595a0 2023-10-06T05:22:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.788+0000 7ff489dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c9890 con 0x7ff4840cc0d0 2023-10-06T05:22:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff483fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff484068bb0 0x7ff4840c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff483fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff484068bb0 0x7ff4840c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42422/0 (socket says 172.21.15.31:42422) 2023-10-06T05:22:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff483fff700 1 -- 172.21.15.31:0/98137265 learned_addr learned my addr 172.21.15.31:0/98137265 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff4837fe700 1 --2- 172.21.15.31:0/98137265 >> v2:172.21.15.62:3300/0 conn(0x7ff4841595a0 0x7ff484159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff482ffd700 1 --2- 172.21.15.31:0/98137265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 0x7ff484068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff483fff700 1 -- 172.21.15.31:0/98137265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 msgr2=0x7ff484068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff483fff700 1 --2- 172.21.15.31:0/98137265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 0x7ff484068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff483fff700 1 -- 172.21.15.31:0/98137265 >> v2:172.21.15.62:3300/0 conn(0x7ff4841595a0 msgr2=0x7ff484159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff483fff700 1 --2- 172.21.15.31:0/98137265 >> v2:172.21.15.62:3300/0 conn(0x7ff4841595a0 0x7ff484159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff483fff700 1 -- 172.21.15.31:0/98137265 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4840c9b70 con 0x7ff484068bb0 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff482ffd700 1 --2- 172.21.15.31:0/98137265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 0x7ff484068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.789+0000 7ff483fff700 1 --2- 172.21.15.31:0/98137265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 0x7ff4840c8930 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff48406c8d0 tx=0x7ff47c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc542ea8246261b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff4827fc700 1 -- 172.21.15.31:0/98137265 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00e070 con 0x7ff484068bb0 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff4827fc700 1 -- 172.21.15.31:0/98137265 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c00ba20 con 0x7ff484068bb0 2023-10-06T05:22:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff4827fc700 1 -- 172.21.15.31:0/98137265 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0092e0 con 0x7ff484068bb0 2023-10-06T05:22:37.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff489dff700 1 -- 172.21.15.31:0/98137265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 msgr2=0x7ff4840c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff489dff700 1 --2- 172.21.15.31:0/98137265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 0x7ff4840c8930 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff48406c8d0 tx=0x7ff47c009840 comp rx=0 tx=0).stop 2023-10-06T05:22:37.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff489dff700 1 -- 172.21.15.31:0/98137265 shutdown_connections 2023-10-06T05:22:37.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff489dff700 1 --2- 172.21.15.31:0/98137265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 0x7ff4840c8930 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff489dff700 1 --2- 172.21.15.31:0/98137265 >> v2:172.21.15.62:3300/0 conn(0x7ff4841595a0 0x7ff484159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff489dff700 1 --2- 172.21.15.31:0/98137265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 0x7ff484068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff489dff700 1 -- 172.21.15.31:0/98137265 >> 172.21.15.31:0/98137265 conn(0x7ff48406b620 msgr2=0x7ff48406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:37.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.790+0000 7ff489dff700 1 -- 172.21.15.31:0/98137265 shutdown_connections 2023-10-06T05:22:37.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.791+0000 7ff489dff700 1 -- 172.21.15.31:0/98137265 wait complete. 2023-10-06T05:22:37.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.792+0000 7ff489dff700 1 Processor -- start 2023-10-06T05:22:37.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.792+0000 7ff489dff700 1 -- start start 2023-10-06T05:22:37.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.793+0000 7ff489dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 0x7ff4840c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.793+0000 7ff489dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 0x7ff484147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.793+0000 7ff489dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4841595a0 0x7ff4840b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.793+0000 7ff489dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c6050 con 0x7ff484068bb0 2023-10-06T05:22:37.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.793+0000 7ff489dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c5440 con 0x7ff4841595a0 2023-10-06T05:22:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.793+0000 7ff489dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c5720 con 0x7ff4840cc0d0 2023-10-06T05:22:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.793+0000 7ff4837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 0x7ff4840c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.793+0000 7ff4837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 0x7ff4840c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42434/0 (socket says 172.21.15.31:42434) 2023-10-06T05:22:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.793+0000 7ff4837fe700 1 -- 172.21.15.31:0/2493636209 learned_addr learned my addr 172.21.15.31:0/2493636209 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff483fff700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4841595a0 0x7ff4840b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff482ffd700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 0x7ff484147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff4837fe700 1 -- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 msgr2=0x7ff484147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff4837fe700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 0x7ff484147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff4837fe700 1 -- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4841595a0 msgr2=0x7ff4840b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:37.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff4837fe700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4841595a0 0x7ff4840b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:37.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff4837fe700 1 -- 172.21.15.31:0/2493636209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c009ae0 con 0x7ff484068bb0 2023-10-06T05:22:37.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff483fff700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4841595a0 0x7ff4840b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:37.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff4837fe700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 0x7ff4840c64e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff47400e930 tx=0x7ff4740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff46b7fe700 1 -- 172.21.15.31:0/2493636209 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47400eed0 con 0x7ff484068bb0 2023-10-06T05:22:37.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff46b7fe700 1 -- 172.21.15.31:0/2493636209 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47400c850 con 0x7ff484068bb0 2023-10-06T05:22:37.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff46b7fe700 1 -- 172.21.15.31:0/2493636209 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47400cbf0 con 0x7ff484068bb0 2023-10-06T05:22:37.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff489dff700 1 -- 172.21.15.31:0/2493636209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4840c5a00 con 0x7ff484068bb0 2023-10-06T05:22:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.794+0000 7ff489dff700 1 -- 172.21.15.31:0/2493636209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4840c6a20 con 0x7ff484068bb0 2023-10-06T05:22:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.796+0000 7ff489dff700 1 -- 172.21.15.31:0/2493636209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4840ce0c0 con 0x7ff484068bb0 2023-10-06T05:22:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.796+0000 7ff46b7fe700 1 -- 172.21.15.31:0/2493636209 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7ff474013470 con 0x7ff484068bb0 2023-10-06T05:22:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.797+0000 7ff46b7fe700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff470067050 0x7ff470069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.797+0000 7ff46b7fe700 1 -- 172.21.15.31:0/2493636209 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff474084ad0 con 0x7ff484068bb0 2023-10-06T05:22:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.798+0000 7ff482ffd700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff470067050 0x7ff470069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.798+0000 7ff482ffd700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff470067050 0x7ff470069500 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff478006f00 tx=0x7ff478008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.801+0000 7ff46b7fe700 1 -- 172.21.15.31:0/2493636209 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff474051a30 con 0x7ff484068bb0 2023-10-06T05:22:37.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:37.995+0000 7ff489dff700 1 -- 172.21.15.31:0/2493636209 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4840c6190 con 0x7ff470067050 2023-10-06T05:22:38.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.005+0000 7ff46b7fe700 1 -- 172.21.15.31:0/2493636209 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311581 (secure 0 0 0) 0x7ff478010640 con 0x7ff470067050 2023-10-06T05:22:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:38.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.009+0000 7ff489dff700 1 -- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff470067050 msgr2=0x7ff470069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.009+0000 7ff489dff700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff470067050 0x7ff470069500 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff478006f00 tx=0x7ff478008040 comp rx=0 tx=0).stop 2023-10-06T05:22:38.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.009+0000 7ff489dff700 1 -- 172.21.15.31:0/2493636209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 msgr2=0x7ff4840c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.009+0000 7ff489dff700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 0x7ff4840c64e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff47400e930 tx=0x7ff4740091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.009+0000 7ff489dff700 1 -- 172.21.15.31:0/2493636209 shutdown_connections 2023-10-06T05:22:38.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.009+0000 7ff489dff700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff470067050 0x7ff470069500 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.009+0000 7ff489dff700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4841595a0 0x7ff4840b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.009+0000 7ff489dff700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff484068bb0 0x7ff4840c64e0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.009+0000 7ff489dff700 1 --2- 172.21.15.31:0/2493636209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff4840cc0d0 0x7ff484147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.009+0000 7ff489dff700 1 -- 172.21.15.31:0/2493636209 >> 172.21.15.31:0/2493636209 conn(0x7ff48406b620 msgr2=0x7ff48406cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.010+0000 7ff489dff700 1 -- 172.21.15.31:0/2493636209 shutdown_connections 2023-10-06T05:22:38.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.010+0000 7ff489dff700 1 -- 172.21.15.31:0/2493636209 wait complete. 2023-10-06T05:22:38.015 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:22:38.029 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-10-06T05:22:37.473304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397544,"kb_used_data":11048,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752577176,"statfs":{"total":773094113280,"available":770639028224,"internally_reserved":0,"allocated":11313152,"data_stored":5839100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001595"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.161574+0000","last_change":"2023-10-06T05:22:33.157614+0000","last_active":"2023-10-06T05:22:33.161574+0000","last_peered":"2023-10-06T05:22:33.161574+0000","last_clean":"2023-10-06T05:22:33.161574+0000","last_became_active":"2023-10-06T05:22:33.157350+0000","last_became_peered":"2023-10-06T05:22:33.157350+0000","last_unstale":"2023-10-06T05:22:33.161574+0000","last_undegraded":"2023-10-06T05:22:33.161574+0000","last_fullsized":"2023-10-06T05:22:33.161574+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:25.981504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.173234+0000","last_change":"2023-10-06T05:22:33.168369+0000","last_active":"2023-10-06T05:22:33.173234+0000","last_peered":"2023-10-06T05:22:33.173234+0000","last_clean":"2023-10-06T05:22:33.173234+0000","last_became_active":"2023-10-06T05:22:33.168164+0000","last_became_peered":"2023-10-06T05:22:33.168164+0000","last_unstale":"2023-10-06T05:22:33.173234+0000","last_undegraded":"2023-10-06T05:22:33.173234+0000","last_fullsized":"2023-10-06T05:22:33.173234+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:44.071118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163389+0000","last_change":"2023-10-06T05:22:33.155575+0000","last_active":"2023-10-06T05:22:33.163389+0000","last_peered":"2023-10-06T05:22:33.163389+0000","last_clean":"2023-10-06T05:22:33.163389+0000","last_became_active":"2023-10-06T05:22:33.155319+0000","last_became_peered":"2023-10-06T05:22:33.155319+0000","last_unstale":"2023-10-06T05:22:33.163389+0000","last_undegraded":"2023-10-06T05:22:33.163389+0000","last_fullsized":"2023-10-06T05:22:33.163389+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:57.486209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.167992+0000","last_change":"2023-10-06T05:22:33.161735+0000","last_active":"2023-10-06T05:22:33.167992+0000","last_peered":"2023-10-06T05:22:33.167992+0000","last_clean":"2023-10-06T05:22:33.167992+0000","last_became_active":"2023-10-06T05:22:33.160656+0000","last_became_peered":"2023-10-06T05:22:33.160656+0000","last_unstale":"2023-10-06T05:22:33.167992+0000","last_undegraded":"2023-10-06T05:22:33.167992+0000","last_fullsized":"2023-10-06T05:22:33.167992+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:57.482645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.161590+0000","last_change":"2023-10-06T05:22:33.155788+0000","last_active":"2023-10-06T05:22:33.161590+0000","last_peered":"2023-10-06T05:22:33.161590+0000","last_clean":"2023-10-06T05:22:33.161590+0000","last_became_active":"2023-10-06T05:22:33.155515+0000","last_became_peered":"2023-10-06T05:22:33.155515+0000","last_unstale":"2023-10-06T05:22:33.161590+0000","last_undegraded":"2023-10-06T05:22:33.161590+0000","last_fullsized":"2023-10-06T05:22:33.161590+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:16.079961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164520+0000","last_change":"2023-10-06T05:22:33.158729+0000","last_active":"2023-10-06T05:22:33.164520+0000","last_peered":"2023-10-06T05:22:33.164520+0000","last_clean":"2023-10-06T05:22:33.164520+0000","last_became_active":"2023-10-06T05:22:33.158460+0000","last_became_peered":"2023-10-06T05:22:33.158460+0000","last_unstale":"2023-10-06T05:22:33.164520+0000","last_undegraded":"2023-10-06T05:22:33.164520+0000","last_fullsized":"2023-10-06T05:22:33.164520+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:48.210445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.155993+0000","last_change":"2023-10-06T05:22:33.153360+0000","last_active":"2023-10-06T05:22:33.155993+0000","last_peered":"2023-10-06T05:22:33.155993+0000","last_clean":"2023-10-06T05:22:33.155993+0000","last_became_active":"2023-10-06T05:22:33.153130+0000","last_became_peered":"2023-10-06T05:22:33.153130+0000","last_unstale":"2023-10-06T05:22:33.155993+0000","last_undegraded":"2023-10-06T05:22:33.155993+0000","last_fullsized":"2023-10-06T05:22:33.155993+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:18.879342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163609+0000","last_change":"2023-10-06T05:22:33.154892+0000","last_active":"2023-10-06T05:22:33.163609+0000","last_peered":"2023-10-06T05:22:33.163609+0000","last_clean":"2023-10-06T05:22:33.163609+0000","last_became_active":"2023-10-06T05:22:33.154625+0000","last_became_peered":"2023-10-06T05:22:33.154625+0000","last_unstale":"2023-10-06T05:22:33.163609+0000","last_undegraded":"2023-10-06T05:22:33.163609+0000","last_fullsized":"2023-10-06T05:22:33.163609+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:21.843425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176799+0000","last_change":"2023-10-06T05:22:33.167185+0000","last_active":"2023-10-06T05:22:33.176799+0000","last_peered":"2023-10-06T05:22:33.176799+0000","last_clean":"2023-10-06T05:22:33.176799+0000","last_became_active":"2023-10-06T05:22:33.166628+0000","last_became_peered":"2023-10-06T05:22:33.166628+0000","last_unstale":"2023-10-06T05:22:33.176799+0000","last_undegraded":"2023-10-06T05:22:33.176799+0000","last_fullsized":"2023-10-06T05:22:33.176799+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:53.985285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163425+0000","last_change":"2023-10-06T05:22:33.159341+0000","last_active":"2023-10-06T05:22:33.163425+0000","last_peered":"2023-10-06T05:22:33.163425+0000","last_clean":"2023-10-06T05:22:33.163425+0000","last_became_active":"2023-10-06T05:22:33.159078+0000","last_became_peered":"2023-10-06T05:22:33.159078+0000","last_unstale":"2023-10-06T05:22:33.163425+0000","last_undegraded":"2023-10-06T05:22:33.163425+0000","last_fullsized":"2023-10-06T05:22:33.163425+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:27:11.741017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176310+0000","last_change":"2023-10-06T05:22:33.171738+0000","last_active":"2023-10-06T05:22:33.176310+0000","last_peered":"2023-10-06T05:22:33.176310+0000","last_clean":"2023-10-06T05:22:33.176310+0000","last_became_active":"2023-10-06T05:22:33.171398+0000","last_became_peered":"2023-10-06T05:22:33.171398+0000","last_unstale":"2023-10-06T05:22:33.176310+0000","last_undegraded":"2023-10-06T05:22:33.176310+0000","last_fullsized":"2023-10-06T05:22:33.176310+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:37.422324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.156563+0000","last_change":"2023-10-06T05:22:33.151503+0000","last_active":"2023-10-06T05:22:33.156563+0000","last_peered":"2023-10-06T05:22:33.156563+0000","last_clean":"2023-10-06T05:22:33.156563+0000","last_became_active":"2023-10-06T05:22:33.151219+0000","last_became_peered":"2023-10-06T05:22:33.151219+0000","last_unstale":"2023-10-06T05:22:33.156563+0000","last_undegraded":"2023-10-06T05:22:33.156563+0000","last_fullsized":"2023-10-06T05:22:33.156563+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:15:04.442056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.175359+0000","last_change":"2023-10-06T05:22:33.171406+0000","last_active":"2023-10-06T05:22:33.175359+0000","last_peered":"2023-10-06T05:22:33.175359+0000","last_clean":"2023-10-06T05:22:33.175359+0000","last_became_active":"2023-10-06T05:22:33.171112+0000","last_became_peered":"2023-10-06T05:22:33.171112+0000","last_unstale":"2023-10-06T05:22:33.175359+0000","last_undegraded":"2023-10-06T05:22:33.175359+0000","last_fullsized":"2023-10-06T05:22:33.175359+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:08.383615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162567+0000","last_change":"2023-10-06T05:22:33.155804+0000","last_active":"2023-10-06T05:22:33.162567+0000","last_peered":"2023-10-06T05:22:33.162567+0000","last_clean":"2023-10-06T05:22:33.162567+0000","last_became_active":"2023-10-06T05:22:33.155527+0000","last_became_peered":"2023-10-06T05:22:33.155527+0000","last_unstale":"2023-10-06T05:22:33.162567+0000","last_undegraded":"2023-10-06T05:22:33.162567+0000","last_fullsized":"2023-10-06T05:22:33.162567+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:07.085636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176324+0000","last_change":"2023-10-06T05:22:33.172283+0000","last_active":"2023-10-06T05:22:33.176324+0000","last_peered":"2023-10-06T05:22:33.176324+0000","last_clean":"2023-10-06T05:22:33.176324+0000","last_became_active":"2023-10-06T05:22:33.171980+0000","last_became_peered":"2023-10-06T05:22:33.171980+0000","last_unstale":"2023-10-06T05:22:33.176324+0000","last_undegraded":"2023-10-06T05:22:33.176324+0000","last_fullsized":"2023-10-06T05:22:33.176324+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:31.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.152525+0000","last_change":"2023-10-06T05:22:33.149888+0000","last_active":"2023-10-06T05:22:33.152525+0000","last_peered":"2023-10-06T05:22:33.152525+0000","last_clean":"2023-10-06T05:22:33.152525+0000","last_became_active":"2023-10-06T05:22:33.149631+0000","last_became_peered":"2023-10-06T05:22:33.149631+0000","last_unstale":"2023-10-06T05:22:33.152525+0000","last_undegraded":"2023-10-06T05:22:33.152525+0000","last_fullsized":"2023-10-06T05:22:33.152525+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:21.065658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.175383+0000","last_change":"2023-10-06T05:22:33.171727+0000","last_active":"2023-10-06T05:22:33.175383+0000","last_peered":"2023-10-06T05:22:33.175383+0000","last_clean":"2023-10-06T05:22:33.175383+0000","last_became_active":"2023-10-06T05:22:33.171496+0000","last_became_peered":"2023-10-06T05:22:33.171496+0000","last_unstale":"2023-10-06T05:22:33.175383+0000","last_undegraded":"2023-10-06T05:22:33.175383+0000","last_fullsized":"2023-10-06T05:22:33.175383+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:44.347426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.173215+0000","last_change":"2023-10-06T05:22:33.168255+0000","last_active":"2023-10-06T05:22:33.173215+0000","last_peered":"2023-10-06T05:22:33.173215+0000","last_clean":"2023-10-06T05:22:33.173215+0000","last_became_active":"2023-10-06T05:22:33.168055+0000","last_became_peered":"2023-10-06T05:22:33.168055+0000","last_unstale":"2023-10-06T05:22:33.173215+0000","last_undegraded":"2023-10-06T05:22:33.173215+0000","last_fullsized":"2023-10-06T05:22:33.173215+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:53.985285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.173158+0000","last_change":"2023-10-06T05:22:33.164745+0000","last_active":"2023-10-06T05:22:33.173158+0000","last_peered":"2023-10-06T05:22:33.173158+0000","last_clean":"2023-10-06T05:22:33.173158+0000","last_became_active":"2023-10-06T05:22:33.164346+0000","last_became_peered":"2023-10-06T05:22:33.164346+0000","last_unstale":"2023-10-06T05:22:33.173158+0000","last_undegraded":"2023-10-06T05:22:33.173158+0000","last_fullsized":"2023-10-06T05:22:33.173158+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:15.061842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.156949+0000","last_change":"2023-10-06T05:22:33.151785+0000","last_active":"2023-10-06T05:22:33.156949+0000","last_peered":"2023-10-06T05:22:33.156949+0000","last_clean":"2023-10-06T05:22:33.156949+0000","last_became_active":"2023-10-06T05:22:33.151522+0000","last_became_peered":"2023-10-06T05:22:33.151522+0000","last_unstale":"2023-10-06T05:22:33.156949+0000","last_undegraded":"2023-10-06T05:22:33.156949+0000","last_fullsized":"2023-10-06T05:22:33.156949+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:33.316756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174605+0000","last_change":"2023-10-06T05:22:33.161428+0000","last_active":"2023-10-06T05:22:33.174605+0000","last_peered":"2023-10-06T05:22:33.174605+0000","last_clean":"2023-10-06T05:22:33.174605+0000","last_became_active":"2023-10-06T05:22:33.161039+0000","last_became_peered":"2023-10-06T05:22:33.161039+0000","last_unstale":"2023-10-06T05:22:33.174605+0000","last_undegraded":"2023-10-06T05:22:33.174605+0000","last_fullsized":"2023-10-06T05:22:33.174605+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:01.452838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.158252+0000","last_change":"2023-10-06T05:22:33.154065+0000","last_active":"2023-10-06T05:22:33.158252+0000","last_peered":"2023-10-06T05:22:33.158252+0000","last_clean":"2023-10-06T05:22:33.158252+0000","last_became_active":"2023-10-06T05:22:33.153771+0000","last_became_peered":"2023-10-06T05:22:33.153771+0000","last_unstale":"2023-10-06T05:22:33.158252+0000","last_undegraded":"2023-10-06T05:22:33.158252+0000","last_fullsized":"2023-10-06T05:22:33.158252+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:29.935435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176685+0000","last_change":"2023-10-06T05:22:33.163482+0000","last_active":"2023-10-06T05:22:33.176685+0000","last_peered":"2023-10-06T05:22:33.176685+0000","last_clean":"2023-10-06T05:22:33.176685+0000","last_became_active":"2023-10-06T05:22:33.163128+0000","last_became_peered":"2023-10-06T05:22:33.163128+0000","last_unstale":"2023-10-06T05:22:33.176685+0000","last_undegraded":"2023-10-06T05:22:33.176685+0000","last_fullsized":"2023-10-06T05:22:33.176685+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:21.843425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176522+0000","last_change":"2023-10-06T05:22:33.171232+0000","last_active":"2023-10-06T05:22:33.176522+0000","last_peered":"2023-10-06T05:22:33.176522+0000","last_clean":"2023-10-06T05:22:33.176522+0000","last_became_active":"2023-10-06T05:22:33.170892+0000","last_became_peered":"2023-10-06T05:22:33.170892+0000","last_unstale":"2023-10-06T05:22:33.176522+0000","last_undegraded":"2023-10-06T05:22:33.176522+0000","last_fullsized":"2023-10-06T05:22:33.176522+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:31.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174708+0000","last_change":"2023-10-06T05:22:33.170644+0000","last_active":"2023-10-06T05:22:33.174708+0000","last_peered":"2023-10-06T05:22:33.174708+0000","last_clean":"2023-10-06T05:22:33.174708+0000","last_became_active":"2023-10-06T05:22:33.170385+0000","last_became_peered":"2023-10-06T05:22:33.170385+0000","last_unstale":"2023-10-06T05:22:33.174708+0000","last_undegraded":"2023-10-06T05:22:33.174708+0000","last_fullsized":"2023-10-06T05:22:33.174708+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:29.935435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.157214+0000","last_change":"2023-10-06T05:22:33.152671+0000","last_active":"2023-10-06T05:22:33.157214+0000","last_peered":"2023-10-06T05:22:33.157214+0000","last_clean":"2023-10-06T05:22:33.157214+0000","last_became_active":"2023-10-06T05:22:33.152355+0000","last_became_peered":"2023-10-06T05:22:33.152355+0000","last_unstale":"2023-10-06T05:22:33.157214+0000","last_undegraded":"2023-10-06T05:22:33.157214+0000","last_fullsized":"2023-10-06T05:22:33.157214+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:01.999949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.173179+0000","last_change":"2023-10-06T05:22:33.165156+0000","last_active":"2023-10-06T05:22:33.173179+0000","last_peered":"2023-10-06T05:22:33.173179+0000","last_clean":"2023-10-06T05:22:33.173179+0000","last_became_active":"2023-10-06T05:22:33.164795+0000","last_became_peered":"2023-10-06T05:22:33.164795+0000","last_unstale":"2023-10-06T05:22:33.173179+0000","last_undegraded":"2023-10-06T05:22:33.173179+0000","last_fullsized":"2023-10-06T05:22:33.173179+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:20.680894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.175622+0000","last_change":"2023-10-06T05:22:33.171400+0000","last_active":"2023-10-06T05:22:33.175622+0000","last_peered":"2023-10-06T05:22:33.175622+0000","last_clean":"2023-10-06T05:22:33.175622+0000","last_became_active":"2023-10-06T05:22:33.171143+0000","last_became_peered":"2023-10-06T05:22:33.171143+0000","last_unstale":"2023-10-06T05:22:33.175622+0000","last_undegraded":"2023-10-06T05:22:33.175622+0000","last_fullsized":"2023-10-06T05:22:33.175622+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:29.286874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.158912+0000","last_change":"2023-10-06T05:22:33.153997+0000","last_active":"2023-10-06T05:22:33.158912+0000","last_peered":"2023-10-06T05:22:33.158912+0000","last_clean":"2023-10-06T05:22:33.158912+0000","last_became_active":"2023-10-06T05:22:33.153729+0000","last_became_peered":"2023-10-06T05:22:33.153729+0000","last_unstale":"2023-10-06T05:22:33.158912+0000","last_undegraded":"2023-10-06T05:22:33.158912+0000","last_fullsized":"2023-10-06T05:22:33.158912+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:50.492107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.167116+0000","last_change":"2023-10-06T05:22:33.160582+0000","last_active":"2023-10-06T05:22:33.167116+0000","last_peered":"2023-10-06T05:22:33.167116+0000","last_clean":"2023-10-06T05:22:33.167116+0000","last_became_active":"2023-10-06T05:22:33.160070+0000","last_became_peered":"2023-10-06T05:22:33.160070+0000","last_unstale":"2023-10-06T05:22:33.167116+0000","last_undegraded":"2023-10-06T05:22:33.167116+0000","last_fullsized":"2023-10-06T05:22:33.167116+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:03.037220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174807+0000","last_change":"2023-10-06T05:22:33.169931+0000","last_active":"2023-10-06T05:22:33.174807+0000","last_peered":"2023-10-06T05:22:33.174807+0000","last_clean":"2023-10-06T05:22:33.174807+0000","last_became_active":"2023-10-06T05:22:33.169681+0000","last_became_peered":"2023-10-06T05:22:33.169681+0000","last_unstale":"2023-10-06T05:22:33.174807+0000","last_undegraded":"2023-10-06T05:22:33.174807+0000","last_fullsized":"2023-10-06T05:22:33.174807+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:37.422324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176544+0000","last_change":"2023-10-06T05:22:33.171532+0000","last_active":"2023-10-06T05:22:33.176544+0000","last_peered":"2023-10-06T05:22:33.176544+0000","last_clean":"2023-10-06T05:22:33.176544+0000","last_became_active":"2023-10-06T05:22:33.170899+0000","last_became_peered":"2023-10-06T05:22:33.170899+0000","last_unstale":"2023-10-06T05:22:33.176544+0000","last_undegraded":"2023-10-06T05:22:33.176544+0000","last_fullsized":"2023-10-06T05:22:33.176544+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:39.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164248+0000","last_change":"2023-10-06T05:22:33.159089+0000","last_active":"2023-10-06T05:22:33.164248+0000","last_peered":"2023-10-06T05:22:33.164248+0000","last_clean":"2023-10-06T05:22:33.164248+0000","last_became_active":"2023-10-06T05:22:33.158807+0000","last_became_peered":"2023-10-06T05:22:33.158807+0000","last_unstale":"2023-10-06T05:22:33.164248+0000","last_undegraded":"2023-10-06T05:22:33.164248+0000","last_fullsized":"2023-10-06T05:22:33.164248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:07.764539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"100'4","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.177420+0000","last_change":"2023-10-06T05:22:32.148571+0000","last_active":"2023-10-06T05:22:33.177420+0000","last_peered":"2023-10-06T05:22:33.177420+0000","last_clean":"2023-10-06T05:22:33.177420+0000","last_became_active":"2023-10-06T05:22:32.148269+0000","last_became_peered":"2023-10-06T05:22:32.148269+0000","last_unstale":"2023-10-06T05:22:33.177420+0000","last_undegraded":"2023-10-06T05:22:33.177420+0000","last_fullsized":"2023-10-06T05:22:33.177420+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:51.971958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.161294+0000","last_change":"2023-10-06T05:22:33.158395+0000","last_active":"2023-10-06T05:22:33.161294+0000","last_peered":"2023-10-06T05:22:33.161294+0000","last_clean":"2023-10-06T05:22:33.161294+0000","last_became_active":"2023-10-06T05:22:33.158085+0000","last_became_peered":"2023-10-06T05:22:33.158085+0000","last_unstale":"2023-10-06T05:22:33.161294+0000","last_undegraded":"2023-10-06T05:22:33.161294+0000","last_fullsized":"2023-10-06T05:22:33.161294+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:29.806530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.157811+0000","last_change":"2023-10-06T05:22:32.144559+0000","last_active":"2023-10-06T05:22:33.157811+0000","last_peered":"2023-10-06T05:22:33.157811+0000","last_clean":"2023-10-06T05:22:33.157811+0000","last_became_active":"2023-10-06T05:22:32.144305+0000","last_became_peered":"2023-10-06T05:22:32.144305+0000","last_unstale":"2023-10-06T05:22:33.157811+0000","last_undegraded":"2023-10-06T05:22:33.157811+0000","last_fullsized":"2023-10-06T05:22:33.157811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:21.849427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174686+0000","last_change":"2023-10-06T05:22:33.169270+0000","last_active":"2023-10-06T05:22:33.174686+0000","last_peered":"2023-10-06T05:22:33.174686+0000","last_clean":"2023-10-06T05:22:33.174686+0000","last_became_active":"2023-10-06T05:22:33.168910+0000","last_became_peered":"2023-10-06T05:22:33.168910+0000","last_unstale":"2023-10-06T05:22:33.174686+0000","last_undegraded":"2023-10-06T05:22:33.174686+0000","last_fullsized":"2023-10-06T05:22:33.174686+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:41.548531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.160875+0000","last_change":"2023-10-06T05:22:32.150422+0000","last_active":"2023-10-06T05:22:33.160875+0000","last_peered":"2023-10-06T05:22:33.160875+0000","last_clean":"2023-10-06T05:22:33.160875+0000","last_became_active":"2023-10-06T05:22:32.150114+0000","last_became_peered":"2023-10-06T05:22:32.150114+0000","last_unstale":"2023-10-06T05:22:33.160875+0000","last_undegraded":"2023-10-06T05:22:33.160875+0000","last_fullsized":"2023-10-06T05:22:33.160875+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:03.439807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176821+0000","last_change":"2023-10-06T05:22:33.167630+0000","last_active":"2023-10-06T05:22:33.176821+0000","last_peered":"2023-10-06T05:22:33.176821+0000","last_clean":"2023-10-06T05:22:33.176821+0000","last_became_active":"2023-10-06T05:22:33.167338+0000","last_became_peered":"2023-10-06T05:22:33.167338+0000","last_unstale":"2023-10-06T05:22:33.176821+0000","last_undegraded":"2023-10-06T05:22:33.176821+0000","last_fullsized":"2023-10-06T05:22:33.176821+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:37.101927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.161507+0000","last_change":"2023-10-06T05:22:32.146346+0000","last_active":"2023-10-06T05:22:33.161507+0000","last_peered":"2023-10-06T05:22:33.161507+0000","last_clean":"2023-10-06T05:22:33.161507+0000","last_became_active":"2023-10-06T05:22:32.146089+0000","last_became_peered":"2023-10-06T05:22:32.146089+0000","last_unstale":"2023-10-06T05:22:33.161507+0000","last_undegraded":"2023-10-06T05:22:33.161507+0000","last_fullsized":"2023-10-06T05:22:33.161507+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:14.055773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.170240+0000","last_change":"2023-10-06T05:22:33.166279+0000","last_active":"2023-10-06T05:22:33.170240+0000","last_peered":"2023-10-06T05:22:33.170240+0000","last_clean":"2023-10-06T05:22:33.170240+0000","last_became_active":"2023-10-06T05:22:33.166045+0000","last_became_peered":"2023-10-06T05:22:33.166045+0000","last_unstale":"2023-10-06T05:22:33.170240+0000","last_undegraded":"2023-10-06T05:22:33.170240+0000","last_fullsized":"2023-10-06T05:22:33.170240+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:08.993634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163931+0000","last_change":"2023-10-06T05:22:32.151688+0000","last_active":"2023-10-06T05:22:33.163931+0000","last_peered":"2023-10-06T05:22:33.163931+0000","last_clean":"2023-10-06T05:22:33.163931+0000","last_became_active":"2023-10-06T05:22:32.151325+0000","last_became_peered":"2023-10-06T05:22:32.151325+0000","last_unstale":"2023-10-06T05:22:33.163931+0000","last_undegraded":"2023-10-06T05:22:33.163931+0000","last_fullsized":"2023-10-06T05:22:33.163931+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:05.805032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.169932+0000","last_change":"2023-10-06T05:22:33.162619+0000","last_active":"2023-10-06T05:22:33.169932+0000","last_peered":"2023-10-06T05:22:33.169932+0000","last_clean":"2023-10-06T05:22:33.169932+0000","last_became_active":"2023-10-06T05:22:33.162339+0000","last_became_peered":"2023-10-06T05:22:33.162339+0000","last_unstale":"2023-10-06T05:22:33.169932+0000","last_undegraded":"2023-10-06T05:22:33.169932+0000","last_fullsized":"2023-10-06T05:22:33.169932+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:39.201673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.158097+0000","last_change":"2023-10-06T05:22:32.154100+0000","last_active":"2023-10-06T05:22:33.158097+0000","last_peered":"2023-10-06T05:22:33.158097+0000","last_clean":"2023-10-06T05:22:33.158097+0000","last_became_active":"2023-10-06T05:22:32.153791+0000","last_became_peered":"2023-10-06T05:22:32.153791+0000","last_unstale":"2023-10-06T05:22:33.158097+0000","last_undegraded":"2023-10-06T05:22:33.158097+0000","last_fullsized":"2023-10-06T05:22:33.158097+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:19.502941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162583+0000","last_change":"2023-10-06T05:22:33.157803+0000","last_active":"2023-10-06T05:22:33.162583+0000","last_peered":"2023-10-06T05:22:33.162583+0000","last_clean":"2023-10-06T05:22:33.162583+0000","last_became_active":"2023-10-06T05:22:33.157518+0000","last_became_peered":"2023-10-06T05:22:33.157518+0000","last_unstale":"2023-10-06T05:22:33.162583+0000","last_undegraded":"2023-10-06T05:22:33.162583+0000","last_fullsized":"2023-10-06T05:22:33.162583+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:57.482645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.156189+0000","last_change":"2023-10-06T05:22:32.148843+0000","last_active":"2023-10-06T05:22:33.156189+0000","last_peered":"2023-10-06T05:22:33.156189+0000","last_clean":"2023-10-06T05:22:33.156189+0000","last_became_active":"2023-10-06T05:22:32.147791+0000","last_became_peered":"2023-10-06T05:22:32.147791+0000","last_unstale":"2023-10-06T05:22:33.156189+0000","last_undegraded":"2023-10-06T05:22:33.156189+0000","last_fullsized":"2023-10-06T05:22:33.156189+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:07.377546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164160+0000","last_change":"2023-10-06T05:22:33.161428+0000","last_active":"2023-10-06T05:22:33.164160+0000","last_peered":"2023-10-06T05:22:33.164160+0000","last_clean":"2023-10-06T05:22:33.164160+0000","last_became_active":"2023-10-06T05:22:33.161119+0000","last_became_peered":"2023-10-06T05:22:33.161119+0000","last_unstale":"2023-10-06T05:22:33.164160+0000","last_undegraded":"2023-10-06T05:22:33.164160+0000","last_fullsized":"2023-10-06T05:22:33.164160+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:20.803230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.139636+0000","last_change":"2023-10-06T05:22:32.153974+0000","last_active":"2023-10-06T05:22:33.139636+0000","last_peered":"2023-10-06T05:22:33.139636+0000","last_clean":"2023-10-06T05:22:33.139636+0000","last_became_active":"2023-10-06T05:22:32.153624+0000","last_became_peered":"2023-10-06T05:22:32.153624+0000","last_unstale":"2023-10-06T05:22:33.139636+0000","last_undegraded":"2023-10-06T05:22:33.139636+0000","last_fullsized":"2023-10-06T05:22:33.139636+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:27.023255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174461+0000","last_change":"2023-10-06T05:22:33.169639+0000","last_active":"2023-10-06T05:22:33.174461+0000","last_peered":"2023-10-06T05:22:33.174461+0000","last_clean":"2023-10-06T05:22:33.174461+0000","last_became_active":"2023-10-06T05:22:33.169345+0000","last_became_peered":"2023-10-06T05:22:33.169345+0000","last_unstale":"2023-10-06T05:22:33.174461+0000","last_undegraded":"2023-10-06T05:22:33.174461+0000","last_fullsized":"2023-10-06T05:22:33.174461+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:04.445876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.160611+0000","last_change":"2023-10-06T05:22:32.154420+0000","last_active":"2023-10-06T05:22:33.160611+0000","last_peered":"2023-10-06T05:22:33.160611+0000","last_clean":"2023-10-06T05:22:33.160611+0000","last_became_active":"2023-10-06T05:22:32.154155+0000","last_became_peered":"2023-10-06T05:22:32.154155+0000","last_unstale":"2023-10-06T05:22:33.160611+0000","last_undegraded":"2023-10-06T05:22:33.160611+0000","last_fullsized":"2023-10-06T05:22:33.160611+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:19.674825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.158367+0000","last_change":"2023-10-06T05:22:33.154681+0000","last_active":"2023-10-06T05:22:33.158367+0000","last_peered":"2023-10-06T05:22:33.158367+0000","last_clean":"2023-10-06T05:22:33.158367+0000","last_became_active":"2023-10-06T05:22:33.154394+0000","last_became_peered":"2023-10-06T05:22:33.154394+0000","last_unstale":"2023-10-06T05:22:33.158367+0000","last_undegraded":"2023-10-06T05:22:33.158367+0000","last_fullsized":"2023-10-06T05:22:33.158367+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:41.548531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.157144+0000","last_change":"2023-10-06T05:22:32.149516+0000","last_active":"2023-10-06T05:22:33.157144+0000","last_peered":"2023-10-06T05:22:33.157144+0000","last_clean":"2023-10-06T05:22:33.157144+0000","last_became_active":"2023-10-06T05:22:32.149219+0000","last_became_peered":"2023-10-06T05:22:32.149219+0000","last_unstale":"2023-10-06T05:22:33.157144+0000","last_undegraded":"2023-10-06T05:22:33.157144+0000","last_fullsized":"2023-10-06T05:22:33.157144+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:58.727736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163104+0000","last_change":"2023-10-06T05:22:33.159501+0000","last_active":"2023-10-06T05:22:33.163104+0000","last_peered":"2023-10-06T05:22:33.163104+0000","last_clean":"2023-10-06T05:22:33.163104+0000","last_became_active":"2023-10-06T05:22:33.157861+0000","last_became_peered":"2023-10-06T05:22:33.157861+0000","last_unstale":"2023-10-06T05:22:33.163104+0000","last_undegraded":"2023-10-06T05:22:33.163104+0000","last_fullsized":"2023-10-06T05:22:33.163104+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:09:14.720386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.173338+0000","last_change":"2023-10-06T05:22:32.156667+0000","last_active":"2023-10-06T05:22:33.173338+0000","last_peered":"2023-10-06T05:22:33.173338+0000","last_clean":"2023-10-06T05:22:33.173338+0000","last_became_active":"2023-10-06T05:22:32.156220+0000","last_became_peered":"2023-10-06T05:22:32.156220+0000","last_unstale":"2023-10-06T05:22:33.173338+0000","last_undegraded":"2023-10-06T05:22:33.173338+0000","last_fullsized":"2023-10-06T05:22:33.173338+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:28.929366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.177598+0000","last_change":"2023-10-06T05:22:33.175414+0000","last_active":"2023-10-06T05:22:33.177598+0000","last_peered":"2023-10-06T05:22:33.177598+0000","last_clean":"2023-10-06T05:22:33.177598+0000","last_became_active":"2023-10-06T05:22:33.175128+0000","last_became_peered":"2023-10-06T05:22:33.175128+0000","last_unstale":"2023-10-06T05:22:33.177598+0000","last_undegraded":"2023-10-06T05:22:33.177598+0000","last_fullsized":"2023-10-06T05:22:33.177598+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:04.496995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.144462+0000","last_change":"2023-10-06T05:22:32.154414+0000","last_active":"2023-10-06T05:22:33.144462+0000","last_peered":"2023-10-06T05:22:33.144462+0000","last_clean":"2023-10-06T05:22:33.144462+0000","last_became_active":"2023-10-06T05:22:32.154112+0000","last_became_peered":"2023-10-06T05:22:32.154112+0000","last_unstale":"2023-10-06T05:22:33.144462+0000","last_undegraded":"2023-10-06T05:22:33.144462+0000","last_fullsized":"2023-10-06T05:22:33.144462+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:24.975434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162917+0000","last_change":"2023-10-06T05:22:33.158930+0000","last_active":"2023-10-06T05:22:33.162917+0000","last_peered":"2023-10-06T05:22:33.162917+0000","last_clean":"2023-10-06T05:22:33.162917+0000","last_became_active":"2023-10-06T05:22:33.158679+0000","last_became_peered":"2023-10-06T05:22:33.158679+0000","last_unstale":"2023-10-06T05:22:33.162917+0000","last_undegraded":"2023-10-06T05:22:33.162917+0000","last_fullsized":"2023-10-06T05:22:33.162917+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:29.294874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164993+0000","last_change":"2023-10-06T05:22:32.149992+0000","last_active":"2023-10-06T05:22:33.164993+0000","last_peered":"2023-10-06T05:22:33.164993+0000","last_clean":"2023-10-06T05:22:33.164993+0000","last_became_active":"2023-10-06T05:22:32.149651+0000","last_became_peered":"2023-10-06T05:22:32.149651+0000","last_unstale":"2023-10-06T05:22:33.164993+0000","last_undegraded":"2023-10-06T05:22:33.164993+0000","last_fullsized":"2023-10-06T05:22:33.164993+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:34.103173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.175311+0000","last_change":"2023-10-06T05:22:33.172314+0000","last_active":"2023-10-06T05:22:33.175311+0000","last_peered":"2023-10-06T05:22:33.175311+0000","last_clean":"2023-10-06T05:22:33.175311+0000","last_became_active":"2023-10-06T05:22:33.172055+0000","last_became_peered":"2023-10-06T05:22:33.172055+0000","last_unstale":"2023-10-06T05:22:33.175311+0000","last_undegraded":"2023-10-06T05:22:33.175311+0000","last_fullsized":"2023-10-06T05:22:33.175311+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:37.101927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.144394+0000","last_change":"2023-10-06T05:22:32.156462+0000","last_active":"2023-10-06T05:22:33.144394+0000","last_peered":"2023-10-06T05:22:33.144394+0000","last_clean":"2023-10-06T05:22:33.144394+0000","last_became_active":"2023-10-06T05:22:32.156125+0000","last_became_peered":"2023-10-06T05:22:32.156125+0000","last_unstale":"2023-10-06T05:22:33.144394+0000","last_undegraded":"2023-10-06T05:22:33.144394+0000","last_fullsized":"2023-10-06T05:22:33.144394+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:36.416254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162548+0000","last_change":"2023-10-06T05:22:33.155802+0000","last_active":"2023-10-06T05:22:33.162548+0000","last_peered":"2023-10-06T05:22:33.162548+0000","last_clean":"2023-10-06T05:22:33.162548+0000","last_became_active":"2023-10-06T05:22:33.155191+0000","last_became_peered":"2023-10-06T05:22:33.155191+0000","last_unstale":"2023-10-06T05:22:33.162548+0000","last_undegraded":"2023-10-06T05:22:33.162548+0000","last_fullsized":"2023-10-06T05:22:33.162548+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:01.452838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.141607+0000","last_change":"2023-10-06T05:22:32.151440+0000","last_active":"2023-10-06T05:22:33.141607+0000","last_peered":"2023-10-06T05:22:33.141607+0000","last_clean":"2023-10-06T05:22:33.141607+0000","last_became_active":"2023-10-06T05:22:32.151139+0000","last_became_peered":"2023-10-06T05:22:32.151139+0000","last_unstale":"2023-10-06T05:22:33.141607+0000","last_undegraded":"2023-10-06T05:22:33.141607+0000","last_fullsized":"2023-10-06T05:22:33.141607+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:38.195603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.171971+0000","last_change":"2023-10-06T05:22:33.167483+0000","last_active":"2023-10-06T05:22:33.171971+0000","last_peered":"2023-10-06T05:22:33.171971+0000","last_clean":"2023-10-06T05:22:33.171971+0000","last_became_active":"2023-10-06T05:22:33.167133+0000","last_became_peered":"2023-10-06T05:22:33.167133+0000","last_unstale":"2023-10-06T05:22:33.171971+0000","last_undegraded":"2023-10-06T05:22:33.171971+0000","last_fullsized":"2023-10-06T05:22:33.171971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:48.296249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.145461+0000","last_change":"2023-10-06T05:22:32.150342+0000","last_active":"2023-10-06T05:22:33.145461+0000","last_peered":"2023-10-06T05:22:33.145461+0000","last_clean":"2023-10-06T05:22:33.145461+0000","last_became_active":"2023-10-06T05:22:32.150102+0000","last_became_peered":"2023-10-06T05:22:32.150102+0000","last_unstale":"2023-10-06T05:22:33.145461+0000","last_undegraded":"2023-10-06T05:22:33.145461+0000","last_fullsized":"2023-10-06T05:22:33.145461+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:40.542462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162156+0000","last_change":"2023-10-06T05:22:33.158390+0000","last_active":"2023-10-06T05:22:33.162156+0000","last_peered":"2023-10-06T05:22:33.162156+0000","last_clean":"2023-10-06T05:22:33.162156+0000","last_became_active":"2023-10-06T05:22:33.158109+0000","last_became_peered":"2023-10-06T05:22:33.158109+0000","last_unstale":"2023-10-06T05:22:33.162156+0000","last_undegraded":"2023-10-06T05:22:33.162156+0000","last_fullsized":"2023-10-06T05:22:33.162156+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:59.627977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.157221+0000","last_change":"2023-10-06T05:22:32.156374+0000","last_active":"2023-10-06T05:22:33.157221+0000","last_peered":"2023-10-06T05:22:33.157221+0000","last_clean":"2023-10-06T05:22:33.157221+0000","last_became_active":"2023-10-06T05:22:32.156019+0000","last_became_peered":"2023-10-06T05:22:32.156019+0000","last_unstale":"2023-10-06T05:22:33.157221+0000","last_undegraded":"2023-10-06T05:22:33.157221+0000","last_fullsized":"2023-10-06T05:22:33.157221+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:18.205665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.170171+0000","last_change":"2023-10-06T05:22:33.165787+0000","last_active":"2023-10-06T05:22:33.170171+0000","last_peered":"2023-10-06T05:22:33.170171+0000","last_clean":"2023-10-06T05:22:33.170171+0000","last_became_active":"2023-10-06T05:22:33.165596+0000","last_became_peered":"2023-10-06T05:22:33.165596+0000","last_unstale":"2023-10-06T05:22:33.170171+0000","last_undegraded":"2023-10-06T05:22:33.170171+0000","last_fullsized":"2023-10-06T05:22:33.170171+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:06.811102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.142393+0000","last_change":"2023-10-06T05:22:32.154455+0000","last_active":"2023-10-06T05:22:33.142393+0000","last_peered":"2023-10-06T05:22:33.142393+0000","last_clean":"2023-10-06T05:22:33.142393+0000","last_became_active":"2023-10-06T05:22:32.154097+0000","last_became_peered":"2023-10-06T05:22:32.154097+0000","last_unstale":"2023-10-06T05:22:33.142393+0000","last_undegraded":"2023-10-06T05:22:33.142393+0000","last_fullsized":"2023-10-06T05:22:33.142393+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:21.480112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.176695+0000","last_change":"2023-10-06T05:22:33.167095+0000","last_active":"2023-10-06T05:22:33.176695+0000","last_peered":"2023-10-06T05:22:33.176695+0000","last_clean":"2023-10-06T05:22:33.176695+0000","last_became_active":"2023-10-06T05:22:33.166702+0000","last_became_peered":"2023-10-06T05:22:33.166702+0000","last_unstale":"2023-10-06T05:22:33.176695+0000","last_undegraded":"2023-10-06T05:22:33.176695+0000","last_fullsized":"2023-10-06T05:22:33.176695+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:29.935418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.142814+0000","last_change":"2023-10-06T05:22:32.154673+0000","last_active":"2023-10-06T05:22:33.142814+0000","last_peered":"2023-10-06T05:22:33.142814+0000","last_clean":"2023-10-06T05:22:33.142814+0000","last_became_active":"2023-10-06T05:22:32.153966+0000","last_became_peered":"2023-10-06T05:22:32.153966+0000","last_unstale":"2023-10-06T05:22:33.142814+0000","last_undegraded":"2023-10-06T05:22:33.142814+0000","last_fullsized":"2023-10-06T05:22:33.142814+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:54.532757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.175362+0000","last_change":"2023-10-06T05:22:33.172615+0000","last_active":"2023-10-06T05:22:33.175362+0000","last_peered":"2023-10-06T05:22:33.175362+0000","last_clean":"2023-10-06T05:22:33.175362+0000","last_became_active":"2023-10-06T05:22:33.172142+0000","last_became_peered":"2023-10-06T05:22:33.172142+0000","last_unstale":"2023-10-06T05:22:33.175362+0000","last_undegraded":"2023-10-06T05:22:33.175362+0000","last_fullsized":"2023-10-06T05:22:33.175362+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:04.346901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.140004+0000","last_change":"2023-10-06T05:22:32.150065+0000","last_active":"2023-10-06T05:22:33.140004+0000","last_peered":"2023-10-06T05:22:33.140004+0000","last_clean":"2023-10-06T05:22:33.140004+0000","last_became_active":"2023-10-06T05:22:32.149757+0000","last_became_peered":"2023-10-06T05:22:32.149757+0000","last_unstale":"2023-10-06T05:22:33.140004+0000","last_undegraded":"2023-10-06T05:22:33.140004+0000","last_fullsized":"2023-10-06T05:22:33.140004+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:14.202372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.156534+0000","last_change":"2023-10-06T05:22:33.151471+0000","last_active":"2023-10-06T05:22:33.156534+0000","last_peered":"2023-10-06T05:22:33.156534+0000","last_clean":"2023-10-06T05:22:33.156534+0000","last_became_active":"2023-10-06T05:22:33.151193+0000","last_became_peered":"2023-10-06T05:22:33.151193+0000","last_unstale":"2023-10-06T05:22:33.156534+0000","last_undegraded":"2023-10-06T05:22:33.156534+0000","last_fullsized":"2023-10-06T05:22:33.156534+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:54.647785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.139476+0000","last_change":"2023-10-06T05:22:32.151554+0000","last_active":"2023-10-06T05:22:33.139476+0000","last_peered":"2023-10-06T05:22:33.139476+0000","last_clean":"2023-10-06T05:22:33.139476+0000","last_became_active":"2023-10-06T05:22:32.151238+0000","last_became_peered":"2023-10-06T05:22:32.151238+0000","last_unstale":"2023-10-06T05:22:33.139476+0000","last_undegraded":"2023-10-06T05:22:33.139476+0000","last_fullsized":"2023-10-06T05:22:33.139476+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:43.341357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.162421+0000","last_change":"2023-10-06T05:22:33.155077+0000","last_active":"2023-10-06T05:22:33.162421+0000","last_peered":"2023-10-06T05:22:33.162421+0000","last_clean":"2023-10-06T05:22:33.162421+0000","last_became_active":"2023-10-06T05:22:33.154816+0000","last_became_peered":"2023-10-06T05:22:33.154816+0000","last_unstale":"2023-10-06T05:22:33.162421+0000","last_undegraded":"2023-10-06T05:22:33.162421+0000","last_fullsized":"2023-10-06T05:22:33.162421+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:23.892518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.137887+0000","last_change":"2023-10-06T05:22:32.152394+0000","last_active":"2023-10-06T05:22:33.137887+0000","last_peered":"2023-10-06T05:22:33.137887+0000","last_clean":"2023-10-06T05:22:33.137887+0000","last_became_active":"2023-10-06T05:22:32.152058+0000","last_became_peered":"2023-10-06T05:22:32.152058+0000","last_unstale":"2023-10-06T05:22:33.137887+0000","last_undegraded":"2023-10-06T05:22:33.137887+0000","last_fullsized":"2023-10-06T05:22:33.137887+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:37.158566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.167639+0000","last_change":"2023-10-06T05:22:33.160995+0000","last_active":"2023-10-06T05:22:33.167639+0000","last_peered":"2023-10-06T05:22:33.167639+0000","last_clean":"2023-10-06T05:22:33.167639+0000","last_became_active":"2023-10-06T05:22:33.160472+0000","last_became_peered":"2023-10-06T05:22:33.160472+0000","last_unstale":"2023-10-06T05:22:33.167639+0000","last_undegraded":"2023-10-06T05:22:33.167639+0000","last_fullsized":"2023-10-06T05:22:33.167639+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:07.085636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.139264+0000","last_change":"2023-10-06T05:22:32.150623+0000","last_active":"2023-10-06T05:22:33.139264+0000","last_peered":"2023-10-06T05:22:33.139264+0000","last_clean":"2023-10-06T05:22:33.139264+0000","last_became_active":"2023-10-06T05:22:32.150334+0000","last_became_peered":"2023-10-06T05:22:32.150334+0000","last_unstale":"2023-10-06T05:22:33.139264+0000","last_undegraded":"2023-10-06T05:22:33.139264+0000","last_fullsized":"2023-10-06T05:22:33.139264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:02.031150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164746+0000","last_change":"2023-10-06T05:22:33.163172+0000","last_active":"2023-10-06T05:22:33.164746+0000","last_peered":"2023-10-06T05:22:33.164746+0000","last_clean":"2023-10-06T05:22:33.164746+0000","last_became_active":"2023-10-06T05:22:33.162820+0000","last_became_peered":"2023-10-06T05:22:33.162820+0000","last_unstale":"2023-10-06T05:22:33.164746+0000","last_undegraded":"2023-10-06T05:22:33.164746+0000","last_fullsized":"2023-10-06T05:22:33.164746+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:03.639154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.144947+0000","last_change":"2023-10-06T05:22:32.155107+0000","last_active":"2023-10-06T05:22:33.144947+0000","last_peered":"2023-10-06T05:22:33.144947+0000","last_clean":"2023-10-06T05:22:33.144947+0000","last_became_active":"2023-10-06T05:22:32.154771+0000","last_became_peered":"2023-10-06T05:22:32.154771+0000","last_unstale":"2023-10-06T05:22:33.144947+0000","last_undegraded":"2023-10-06T05:22:33.144947+0000","last_fullsized":"2023-10-06T05:22:33.144947+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:03.490926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.169475+0000","last_change":"2023-10-06T05:22:33.164494+0000","last_active":"2023-10-06T05:22:33.169475+0000","last_peered":"2023-10-06T05:22:33.169475+0000","last_clean":"2023-10-06T05:22:33.169475+0000","last_became_active":"2023-10-06T05:22:33.164190+0000","last_became_peered":"2023-10-06T05:22:33.164190+0000","last_unstale":"2023-10-06T05:22:33.169475+0000","last_undegraded":"2023-10-06T05:22:33.169475+0000","last_fullsized":"2023-10-06T05:22:33.169475+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:55.243447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.142424+0000","last_change":"2023-10-06T05:22:32.154271+0000","last_active":"2023-10-06T05:22:33.142424+0000","last_peered":"2023-10-06T05:22:33.142424+0000","last_clean":"2023-10-06T05:22:33.142424+0000","last_became_active":"2023-10-06T05:22:32.154002+0000","last_became_peered":"2023-10-06T05:22:32.154002+0000","last_unstale":"2023-10-06T05:22:33.142424+0000","last_undegraded":"2023-10-06T05:22:33.142424+0000","last_fullsized":"2023-10-06T05:22:33.142424+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:56.590205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.142525+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:22:33.142525+0000","last_peered":"2023-10-06T05:22:33.142525+0000","last_clean":"2023-10-06T05:22:33.142525+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:22:33.142525+0000","last_undegraded":"2023-10-06T05:22:33.142525+0000","last_fullsized":"2023-10-06T05:22:33.142525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174517+0000","last_change":"2023-10-06T05:22:33.168666+0000","last_active":"2023-10-06T05:22:33.174517+0000","last_peered":"2023-10-06T05:22:33.174517+0000","last_clean":"2023-10-06T05:22:33.174517+0000","last_became_active":"2023-10-06T05:22:33.168309+0000","last_became_peered":"2023-10-06T05:22:33.168309+0000","last_unstale":"2023-10-06T05:22:33.174517+0000","last_undegraded":"2023-10-06T05:22:33.174517+0000","last_fullsized":"2023-10-06T05:22:33.174517+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:16.079961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.157292+0000","last_change":"2023-10-06T05:22:32.145009+0000","last_active":"2023-10-06T05:22:33.157292+0000","last_peered":"2023-10-06T05:22:33.157292+0000","last_clean":"2023-10-06T05:22:33.157292+0000","last_became_active":"2023-10-06T05:22:32.144659+0000","last_became_peered":"2023-10-06T05:22:32.144659+0000","last_unstale":"2023-10-06T05:22:33.157292+0000","last_undegraded":"2023-10-06T05:22:33.157292+0000","last_fullsized":"2023-10-06T05:22:33.157292+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:07.987565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.142507+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:22:33.142507+0000","last_peered":"2023-10-06T05:22:33.142507+0000","last_clean":"2023-10-06T05:22:33.142507+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:22:33.142507+0000","last_undegraded":"2023-10-06T05:22:33.142507+0000","last_fullsized":"2023-10-06T05:22:33.142507+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.153371+0000","last_change":"2023-10-06T05:22:33.144576+0000","last_active":"2023-10-06T05:22:33.153371+0000","last_peered":"2023-10-06T05:22:33.153371+0000","last_clean":"2023-10-06T05:22:33.153371+0000","last_became_active":"2023-10-06T05:22:33.144237+0000","last_became_peered":"2023-10-06T05:22:33.144237+0000","last_unstale":"2023-10-06T05:22:33.153371+0000","last_undegraded":"2023-10-06T05:22:33.153371+0000","last_fullsized":"2023-10-06T05:22:33.153371+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:27.815053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.141473+0000","last_change":"2023-10-06T05:22:32.152796+0000","last_active":"2023-10-06T05:22:33.141473+0000","last_peered":"2023-10-06T05:22:33.141473+0000","last_clean":"2023-10-06T05:22:33.141473+0000","last_became_active":"2023-10-06T05:22:32.152557+0000","last_became_peered":"2023-10-06T05:22:32.152557+0000","last_unstale":"2023-10-06T05:22:33.141473+0000","last_undegraded":"2023-10-06T05:22:33.141473+0000","last_fullsized":"2023-10-06T05:22:33.141473+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:36.416254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.153186+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:22:33.153186+0000","last_peered":"2023-10-06T05:22:33.153186+0000","last_clean":"2023-10-06T05:22:33.153186+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:22:33.153186+0000","last_undegraded":"2023-10-06T05:22:33.153186+0000","last_fullsized":"2023-10-06T05:22:33.153186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174632+0000","last_change":"2023-10-06T05:22:33.162318+0000","last_active":"2023-10-06T05:22:33.174632+0000","last_peered":"2023-10-06T05:22:33.174632+0000","last_clean":"2023-10-06T05:22:33.174632+0000","last_became_active":"2023-10-06T05:22:33.162005+0000","last_became_peered":"2023-10-06T05:22:33.162005+0000","last_unstale":"2023-10-06T05:22:33.174632+0000","last_undegraded":"2023-10-06T05:22:33.174632+0000","last_fullsized":"2023-10-06T05:22:33.174632+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:57.486209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"100'8","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174880+0000","last_change":"2023-10-06T05:22:32.148896+0000","last_active":"2023-10-06T05:22:33.174880+0000","last_peered":"2023-10-06T05:22:33.174880+0000","last_clean":"2023-10-06T05:22:33.174880+0000","last_became_active":"2023-10-06T05:22:32.148625+0000","last_became_peered":"2023-10-06T05:22:32.148625+0000","last_unstale":"2023-10-06T05:22:33.174880+0000","last_undegraded":"2023-10-06T05:22:33.174880+0000","last_fullsized":"2023-10-06T05:22:33.174880+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:24.975434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.145280+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:22:33.145280+0000","last_peered":"2023-10-06T05:22:33.145280+0000","last_clean":"2023-10-06T05:22:33.145280+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:22:33.145280+0000","last_undegraded":"2023-10-06T05:22:33.145280+0000","last_fullsized":"2023-10-06T05:22:33.145280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164760+0000","last_change":"2023-10-06T05:22:33.163160+0000","last_active":"2023-10-06T05:22:33.164760+0000","last_peered":"2023-10-06T05:22:33.164760+0000","last_clean":"2023-10-06T05:22:33.164760+0000","last_became_active":"2023-10-06T05:22:33.162870+0000","last_became_peered":"2023-10-06T05:22:33.162870+0000","last_unstale":"2023-10-06T05:22:33.164760+0000","last_undegraded":"2023-10-06T05:22:33.164760+0000","last_fullsized":"2023-10-06T05:22:33.164760+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:38.164635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.143578+0000","last_change":"2023-10-06T05:22:32.146142+0000","last_active":"2023-10-06T05:22:33.143578+0000","last_peered":"2023-10-06T05:22:33.143578+0000","last_clean":"2023-10-06T05:22:33.143578+0000","last_became_active":"2023-10-06T05:22:32.145839+0000","last_became_peered":"2023-10-06T05:22:32.145839+0000","last_unstale":"2023-10-06T05:22:33.143578+0000","last_undegraded":"2023-10-06T05:22:33.143578+0000","last_fullsized":"2023-10-06T05:22:33.143578+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:28.929366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":78,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.144484+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:22:33.144484+0000","last_peered":"2023-10-06T05:22:33.144484+0000","last_clean":"2023-10-06T05:22:33.144484+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:22:33.144484+0000","last_undegraded":"2023-10-06T05:22:33.144484+0000","last_fullsized":"2023-10-06T05:22:33.144484+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.165216+0000","last_change":"2023-10-06T05:22:33.155778+0000","last_active":"2023-10-06T05:22:33.165216+0000","last_peered":"2023-10-06T05:22:33.165216+0000","last_clean":"2023-10-06T05:22:33.165216+0000","last_became_active":"2023-10-06T05:22:33.155425+0000","last_became_peered":"2023-10-06T05:22:33.155425+0000","last_unstale":"2023-10-06T05:22:33.165216+0000","last_undegraded":"2023-10-06T05:22:33.165216+0000","last_fullsized":"2023-10-06T05:22:33.165216+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:16.199208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.164878+0000","last_change":"2023-10-06T05:22:32.148615+0000","last_active":"2023-10-06T05:22:33.164878+0000","last_peered":"2023-10-06T05:22:33.164878+0000","last_clean":"2023-10-06T05:22:33.164878+0000","last_became_active":"2023-10-06T05:22:32.148319+0000","last_became_peered":"2023-10-06T05:22:32.148319+0000","last_unstale":"2023-10-06T05:22:33.164878+0000","last_undegraded":"2023-10-06T05:22:33.164878+0000","last_fullsized":"2023-10-06T05:22:33.164878+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:30.230308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":267,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.145095+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:22:33.145095+0000","last_peered":"2023-10-06T05:22:33.145095+0000","last_clean":"2023-10-06T05:22:33.145095+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:22:33.145095+0000","last_undegraded":"2023-10-06T05:22:33.145095+0000","last_fullsized":"2023-10-06T05:22:33.145095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.146037+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:22:33.146037+0000","last_peered":"2023-10-06T05:22:33.146037+0000","last_clean":"2023-10-06T05:22:33.146037+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:22:33.146037+0000","last_undegraded":"2023-10-06T05:22:33.146037+0000","last_fullsized":"2023-10-06T05:22:33.146037+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.174693+0000","last_change":"2023-10-06T05:22:33.170634+0000","last_active":"2023-10-06T05:22:33.174693+0000","last_peered":"2023-10-06T05:22:33.174693+0000","last_clean":"2023-10-06T05:22:33.174693+0000","last_became_active":"2023-10-06T05:22:33.170242+0000","last_became_peered":"2023-10-06T05:22:33.170242+0000","last_unstale":"2023-10-06T05:22:33.174693+0000","last_undegraded":"2023-10-06T05:22:33.174693+0000","last_fullsized":"2023-10-06T05:22:33.174693+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:25.981504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"100'6","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.177195+0000","last_change":"2023-10-06T05:22:32.152283+0000","last_active":"2023-10-06T05:22:33.177195+0000","last_peered":"2023-10-06T05:22:33.177195+0000","last_clean":"2023-10-06T05:22:33.177195+0000","last_became_active":"2023-10-06T05:22:32.151990+0000","last_became_peered":"2023-10-06T05:22:32.151990+0000","last_unstale":"2023-10-06T05:22:33.177195+0000","last_undegraded":"2023-10-06T05:22:33.177195+0000","last_fullsized":"2023-10-06T05:22:33.177195+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:47.290180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":94,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.141999+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:22:33.141999+0000","last_peered":"2023-10-06T05:22:33.141999+0000","last_clean":"2023-10-06T05:22:33.141999+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:22:33.141999+0000","last_undegraded":"2023-10-06T05:22:33.141999+0000","last_fullsized":"2023-10-06T05:22:33.141999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.163343+0000","last_change":"2023-10-06T05:22:33.156007+0000","last_active":"2023-10-06T05:22:33.163343+0000","last_peered":"2023-10-06T05:22:33.163343+0000","last_clean":"2023-10-06T05:22:33.163343+0000","last_became_active":"2023-10-06T05:22:33.155723+0000","last_became_peered":"2023-10-06T05:22:33.155723+0000","last_unstale":"2023-10-06T05:22:33.163343+0000","last_undegraded":"2023-10-06T05:22:33.163343+0000","last_fullsized":"2023-10-06T05:22:33.163343+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:32.121735+0000","last_clean_scrub_stamp":"2023-10-06T05:22:32.121735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:29.935418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"100'3","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.169043+0000","last_change":"2023-10-06T05:22:32.149317+0000","last_active":"2023-10-06T05:22:33.169043+0000","last_peered":"2023-10-06T05:22:33.169043+0000","last_clean":"2023-10-06T05:22:33.169043+0000","last_became_active":"2023-10-06T05:22:32.148874+0000","last_became_peered":"2023-10-06T05:22:32.148874+0000","last_unstale":"2023-10-06T05:22:33.169043+0000","last_undegraded":"2023-10-06T05:22:33.169043+0000","last_fullsized":"2023-10-06T05:22:33.169043+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:22:31.115665+0000","last_clean_scrub_stamp":"2023-10-06T05:22:31.115665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:19.797161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-06T05:22:33.138062+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:22:33.138062+0000","last_peered":"2023-10-06T05:22:33.138062+0000","last_clean":"2023-10-06T05:22:33.138062+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:22:33.138062+0000","last_undegraded":"2023-10-06T05:22:33.138062+0000","last_fullsized":"2023-10-06T05:22:33.138062+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303208,"kb_used_data":1824,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068632,"statfs":{"total":96636764160,"available":96326279168,"internally_reserved":0,"allocated":1867776,"data_stored":1172835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.375,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93200000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.504},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93300000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.97599999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72999999999999998,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90000000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.51600000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82699999999999996,"5min":1.018,"15min":1.018},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71799999999999997,"15min":0.752},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88100000000000001,"5min":1.056,"15min":1.056},"last":0.68000000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72599999999999998,"15min":0.748},"min":{"1min":0.47699999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65300000000000002,"15min":0.71299999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81899999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72899999999999998,"15min":0.77900000000000003},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70499999999999996,"15min":0.73999999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97899999999999998,"5min":0.999,"15min":0.999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76500000000000001,"15min":0.83199999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.82399999999999995,"15min":0.88400000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.012,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81499999999999995,"15min":0.85099999999999998},"min":{"1min":0.59799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.86299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298520,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073320,"statfs":{"total":96636764160,"available":96331079680,"internally_reserved":0,"allocated":1261568,"data_stored":582273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.66900000000000004,"15min":0.68300000000000005},"min":{"1min":0.44400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80400000000000005,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.49299999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2170000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.504,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95899999999999996,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.48799999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.073,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":1.093,"5min":1.369,"15min":1.369},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73199999999999998,"15min":0.76300000000000001},"min":{"1min":0.58099999999999996,"5min":0.438,"15min":0.438},"max":{"1min":0.94199999999999995,"5min":1.891,"15min":1.891},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73299999999999998,"15min":0.751},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.018,"5min":1.347,"15min":1.347},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.58599999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98699999999999999,"5min":1,"15min":1},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72699999999999998,"15min":0.78300000000000003},"min":{"1min":0.47199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.1140000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.77600000000000002,"15min":0.82199999999999995},"min":{"1min":0.62,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.278,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.82999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88900000000000001,"5min":1.415,"15min":1.415},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.75800000000000001,"15min":0.82499999999999996},"min":{"1min":0.53400000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.008,"5min":2,"15min":2},"last":0.90200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.76200000000000001,"15min":0.81000000000000005},"min":{"1min":0.53100000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0609999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.752,"15min":0.77800000000000002},"min":{"1min":0.56299999999999994,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99099999999999999,"5min":1.46,"15min":1.46},"last":1.0429999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303196,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068644,"statfs":{"total":96636764160,"available":96326291456,"internally_reserved":0,"allocated":1855488,"data_stored":1172109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58399999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.496},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.498,"15min":0.496},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60099999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46100000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69999999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.46899999999999997},"min":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.64200000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56200000000000006,"15min":0.52500000000000002},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78400000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.499,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.35499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68999999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0209999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83199999999999996,"15min":0.79900000000000004},"min":{"1min":0.65700000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.054,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.77100000000000002,"15min":0.751},"min":{"1min":0.59799999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98499999999999999,"5min":1.099,"15min":1.099},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.624,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.997,"5min":1.181,"15min":1.181},"last":0.625}]},{"osd":6,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78200000000000003,"15min":0.754},"min":{"1min":0.57199999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98799999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.82499999999999996,"15min":0.80100000000000005},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.95299999999999996,"5min":1.123,"15min":1.123},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83099999999999996,"15min":0.82299999999999995},"min":{"1min":0.63600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83999999999999997,"15min":0.82899999999999996},"min":{"1min":0.58199999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98199999999999998,"5min":1.103,"15min":1.103},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298520,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073320,"statfs":{"total":96636764160,"available":96331079680,"internally_reserved":0,"allocated":1261568,"data_stored":581741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.29899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55200000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.33100000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":1.042,"15min":1.042},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.504,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53400000000000003,"15min":0.51600000000000001},"min":{"1min":0.43099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52000000000000002,"15min":0.49099999999999999},"min":{"1min":0.33200000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68200000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.46800000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70999999999999996,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45800000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77000000000000002,"15min":0.75600000000000001},"min":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95699999999999996,"5min":1.242,"15min":1.242},"last":0.498},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78200000000000003,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.499,"15min":0.499},"max":{"1min":0.92300000000000004,"5min":1.198,"15min":1.198},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78600000000000003,"15min":0.77200000000000002},"min":{"1min":0.61199999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.90700000000000003,"5min":1.26,"15min":1.26},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86499999999999999,"5min":1.163,"15min":1.163},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94299999999999995,"5min":1.323,"15min":1.323},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81899999999999995,"15min":0.79800000000000004},"min":{"1min":0.58099999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0049999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.006,"5min":1.264,"15min":1.264},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81799999999999995,"15min":0.83199999999999996},"min":{"1min":0.64700000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.004,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.57099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298516,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073324,"statfs":{"total":96636764160,"available":96331083776,"internally_reserved":0,"allocated":1257472,"data_stored":581719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.433,"15min":0.42899999999999999},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64600000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.34499999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.33700000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71699999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.498,"15min":0.502},"min":{"1min":0.34999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.54900000000000004},"min":{"1min":0.42199999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77300000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.5}]},{"osd":4,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.73999999999999999,"15min":0.75900000000000001},"min":{"1min":0.54700000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.022,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.752,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.085,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.82999999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.75,"15min":0.78600000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.042,"5min":1.099,"15min":1.099},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76800000000000002,"15min":0.80600000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0049999999999999,"5min":1.163,"15min":1.163},"last":0.93700000000000006}]},{"osd":6,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79900000000000004,"15min":0.83899999999999997},"min":{"1min":0.58499999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.056,"5min":1.21,"15min":1.21},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79000000000000004,"15min":0.81699999999999995},"min":{"1min":0.58699999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.91800000000000004,"5min":1.042,"15min":1.042},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.78000000000000003,"15min":0.80700000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0609999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.88},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.82399999999999995,"15min":0.86599999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1080000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.84999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298532,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073308,"statfs":{"total":96636764160,"available":96331067392,"internally_reserved":0,"allocated":1273856,"data_stored":583289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.443,"15min":0.46100000000000002},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.495},"min":{"1min":0.316,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64500000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47999999999999998,"15min":0.47299999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.52100000000000002,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71499999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98999999999999999,"5min":1.014,"15min":1.014},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81399999999999995,"15min":0.81200000000000006},"min":{"1min":0.62,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1220000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.82899999999999996,"15min":0.83299999999999996},"min":{"1min":0.70499999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98399999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.58399999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.006,"5min":1.204,"15min":1.204},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.80600000000000005,"15min":0.83699999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0209999999999999,"5min":1.329,"15min":1.329},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79800000000000004,"15min":0.81899999999999995},"min":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0840000000000001,"5min":1.484,"15min":1.484},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.81799999999999995,"15min":0.84799999999999998},"min":{"1min":0.56999999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0529999999999999,"5min":1.629,"15min":1.629},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82499999999999996,"15min":0.85599999999999998},"min":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.137,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.81699999999999995,"15min":0.82999999999999996},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.194,"5min":1.325,"15min":1.325},"last":0.78600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298528,"kb_used_data":1240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073312,"statfs":{"total":96636764160,"available":96331071488,"internally_reserved":0,"allocated":1269760,"data_stored":583255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.47199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90600000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70899999999999996,"15min":0.78700000000000003},"min":{"1min":0.442,"5min":0.438,"15min":0.438},"max":{"1min":0.92400000000000004,"5min":2.77,"15min":2.77},"last":0.82699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.69599999999999995,"15min":0.73799999999999999},"min":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92200000000000004,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69499999999999995,"15min":0.72199999999999998},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90100000000000002,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.495,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93999999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73899999999999999,"15min":0.82099999999999995},"min":{"1min":0.56000000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1120000000000001,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.83799999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.81599999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.80300000000000005,"15min":1.0569999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93999999999999995,"5min":7.742,"15min":7.742},"last":0.89500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69199999999999995,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96499999999999997,"5min":1.079,"15min":1.079},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.71099999999999997,"15min":0.75},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85799999999999998,"5min":1.181,"15min":1.181},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.73999999999999999},"min":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.85499999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76800000000000002,"15min":0.82199999999999995},"min":{"1min":0.59599999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.95299999999999996,"5min":1.171,"15min":1.171},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:22:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.78800000000000003,"15min":0.83099999999999996},"min":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.024,"5min":1.228,"15min":1.228},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.78500000000000003,"15min":0.85899999999999999},"min":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.93500000000000005,"5min":1.571,"15min":1.571},"last":0.72599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298524,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073316,"statfs":{"total":96636764160,"available":96331075584,"internally_reserved":0,"allocated":1265664,"data_stored":581879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.73599999999999999,"15min":0.96799999999999997},"min":{"1min":0.433,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79700000000000004,"5min":9.2780000000000005,"15min":9.2780000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.45700000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.94699999999999995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79800000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.77100000000000002,"15min":0.99299999999999999},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84699999999999998,"5min":9.25,"15min":9.25},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.79700000000000004,"15min":1.056},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85199999999999998,"5min":10.108000000000001,"15min":10.108000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.76700000000000002,"15min":1.026},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82299999999999995,"5min":10.384,"15min":10.384},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81499999999999995,"15min":1.046},"min":{"1min":0.54500000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88900000000000001,"5min":9.1699999999999999,"15min":9.1699999999999999},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.82399999999999995,"15min":1.089},"min":{"1min":0.53300000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.87,"5min":10.205,"15min":10.205},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.82499999999999996,"5min":1.292,"15min":1.292},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.46200000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88700000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.72599999999999998,"15min":0.77200000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.036,"5min":1.171,"15min":1.171},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70399999999999996,"15min":0.73599999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88400000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.52000000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93999999999999995,"5min":1.298,"15min":1.298},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78600000000000003,"15min":0.82399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.107,"5min":1.204,"15min":1.204},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.54400000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93799999999999994,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.72299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:22:38.075 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:22:38.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:22:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.276+0000 7f06bb3ff700 1 Processor -- start 2023-10-06T05:22:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.277+0000 7f06bb3ff700 1 -- start start 2023-10-06T05:22:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06bb3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f06b4159560 0x7f06b40cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06bb3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06b4159930 0x7f06b40cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06bb3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b415d2d0 con 0x7f06b40b0d60 2023-10-06T05:22:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06bb3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b415d5b0 con 0x7f06b4159930 2023-10-06T05:22:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06bb3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b415d890 con 0x7f06b4159560 2023-10-06T05:22:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06b899a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06b4159930 0x7f06b40cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06b899a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06b4159930 0x7f06b40cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42440/0 (socket says 172.21.15.31:42440) 2023-10-06T05:22:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06b899a700 1 -- 172.21.15.31:0/708920856 learned_addr learned my addr 172.21.15.31:0/708920856 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:38.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06b919b700 1 --2- 172.21.15.31:0/708920856 >> v2:172.21.15.62:3300/0 conn(0x7f06b4159560 0x7f06b40cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06b899a700 1 -- 172.21.15.31:0/708920856 >> v2:172.21.15.62:3300/0 conn(0x7f06b4159560 msgr2=0x7f06b40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06b899a700 1 --2- 172.21.15.31:0/708920856 >> v2:172.21.15.62:3300/0 conn(0x7f06b4159560 0x7f06b40cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06b899a700 1 -- 172.21.15.31:0/708920856 >> v1:172.21.15.62:6789/0 conn(0x7f06b40b0d60 legacy=0x7f06b40b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.278+0000 7f06b899a700 1 -- 172.21.15.31:0/708920856 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b415db70 con 0x7f06b4159930 2023-10-06T05:22:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.279+0000 7f06b899a700 1 --2- 172.21.15.31:0/708920856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159930 0x7f06b40cd3b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f06b4063d80 tx=0x7f06a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9acbd8e10f24749 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.279+0000 7f06abfff700 1 -- 172.21.15.31:0/708920856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4009e70 con 0x7f06b4159930 2023-10-06T05:22:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.279+0000 7f06abfff700 1 -- 172.21.15.31:0/708920856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a400b860 con 0x7f06b4159930 2023-10-06T05:22:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.279+0000 7f06abfff700 1 -- 172.21.15.31:0/708920856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a400bc00 con 0x7f06b4159930 2023-10-06T05:22:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.279+0000 7f06bb3ff700 1 -- 172.21.15.31:0/708920856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159930 msgr2=0x7f06b40cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.280+0000 7f06bb3ff700 1 --2- 172.21.15.31:0/708920856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159930 0x7f06b40cd3b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f06b4063d80 tx=0x7f06a4009250 comp rx=0 tx=0).stop 2023-10-06T05:22:38.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.280+0000 7f06bb3ff700 1 -- 172.21.15.31:0/708920856 shutdown_connections 2023-10-06T05:22:38.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.280+0000 7f06bb3ff700 1 --2- 172.21.15.31:0/708920856 >> v2:172.21.15.62:3300/0 conn(0x7f06b4159560 0x7f06b40cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.280+0000 7f06bb3ff700 1 --2- 172.21.15.31:0/708920856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159930 0x7f06b40cd3b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.280+0000 7f06bb3ff700 1 -- 172.21.15.31:0/708920856 >> 172.21.15.31:0/708920856 conn(0x7f06b406ec90 msgr2=0x7f06b406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.280+0000 7f06bb3ff700 1 -- 172.21.15.31:0/708920856 shutdown_connections 2023-10-06T05:22:38.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.280+0000 7f06bb3ff700 1 -- 172.21.15.31:0/708920856 wait complete. 2023-10-06T05:22:38.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.282+0000 7f06bb3ff700 1 Processor -- start 2023-10-06T05:22:38.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.282+0000 7f06bb3ff700 1 -- start start 2023-10-06T05:22:38.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.283+0000 7f06bb3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b40b0d60 0x7f06b416ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.283+0000 7f06bb3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159560 0x7f06b40cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.283+0000 7f06bb3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06b4159930 0x7f06b416ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.283+0000 7f06bb3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40cfde0 con 0x7f06b4159560 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.283+0000 7f06bb3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b4151f90 con 0x7f06b40b0d60 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.283+0000 7f06bb3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b4151660 con 0x7f06b4159930 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159560 0x7f06b40cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159560 0x7f06b40cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42454/0 (socket says 172.21.15.31:42454) 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b899a700 1 -- 172.21.15.31:0/542466256 learned_addr learned my addr 172.21.15.31:0/542466256 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b919b700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b40b0d60 0x7f06b416ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b999c700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06b4159930 0x7f06b416ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b899a700 1 -- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06b4159930 msgr2=0x7f06b416ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b899a700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06b4159930 0x7f06b416ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b899a700 1 -- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b40b0d60 msgr2=0x7f06b416ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b899a700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b40b0d60 0x7f06b416ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b899a700 1 -- 172.21.15.31:0/542466256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a40094f0 con 0x7f06b4159560 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b999c700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06b4159930 0x7f06b416ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06b899a700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159560 0x7f06b40cf8a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f06b40cd980 tx=0x7f06a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06a97fa700 1 -- 172.21.15.31:0/542466256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4011420 con 0x7f06b4159560 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.284+0000 7f06a97fa700 1 -- 172.21.15.31:0/542466256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a40115a0 con 0x7f06b4159560 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.285+0000 7f06bb3ff700 1 -- 172.21.15.31:0/542466256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b4151940 con 0x7f06b4159560 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.285+0000 7f06a97fa700 1 -- 172.21.15.31:0/542466256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4011960 con 0x7f06b4159560 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.285+0000 7f06bb3ff700 1 -- 172.21.15.31:0/542466256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b40cfff0 con 0x7f06b4159560 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.286+0000 7f06bb3ff700 1 -- 172.21.15.31:0/542466256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b40d1300 con 0x7f06b4159560 2023-10-06T05:22:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.286+0000 7f06a97fa700 1 -- 172.21.15.31:0/542466256 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f06a4011d50 con 0x7f06b4159560 2023-10-06T05:22:38.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.287+0000 7f06a97fa700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0698067050 0x7f0698069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.287+0000 7f06a97fa700 1 -- 172.21.15.31:0/542466256 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f06a4085e50 con 0x7f06b4159560 2023-10-06T05:22:38.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.288+0000 7f06b919b700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0698067050 0x7f0698069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.288+0000 7f06b919b700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0698067050 0x7f0698069500 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009d40 tx=0x7f06b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.292+0000 7f06a97fa700 1 -- 172.21.15.31:0/542466256 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06a4052df0 con 0x7f06b4159560 2023-10-06T05:22:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.578+0000 7f06bb3ff700 1 -- 172.21.15.31:0/542466256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f06b40b1260 con 0x7f06b4159560 2023-10-06T05:22:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.583+0000 7f06a97fa700 1 -- 172.21.15.31:0/542466256 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f06a40150e0 con 0x7f06b4159560 2023-10-06T05:22:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 -- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0698067050 msgr2=0x7f0698069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0698067050 0x7f0698069500 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009d40 tx=0x7f06b0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 -- 172.21.15.31:0/542466256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159560 msgr2=0x7f06b40cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159560 0x7f06b40cf8a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f06b40cd980 tx=0x7f06a4017000 comp rx=0 tx=0).stop 2023-10-06T05:22:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 -- 172.21.15.31:0/542466256 shutdown_connections 2023-10-06T05:22:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0698067050 0x7f0698069500 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06b4159930 0x7f06b416ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b40b0d60 0x7f06b416ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 --2- 172.21.15.31:0/542466256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b4159560 0x7f06b40cf8a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 -- 172.21.15.31:0/542466256 >> 172.21.15.31:0/542466256 conn(0x7f06b406ec90 msgr2=0x7f06b40cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 -- 172.21.15.31:0/542466256 shutdown_connections 2023-10-06T05:22:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.585+0000 7f06bb3ff700 1 -- 172.21.15.31:0/542466256 wait complete. 2023-10-06T05:22:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:22:38.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:22:38.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.796+0000 7f506a5ff700 1 Processor -- start 2023-10-06T05:22:38.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.796+0000 7f506a5ff700 1 -- start start 2023-10-06T05:22:38.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.797+0000 7f506a5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f50640cd510 0x7f50640b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f506a5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f506415d2d0 0x7f50640aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f506a5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50640c3f40 con 0x7f5064159500 2023-10-06T05:22:38.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f506a5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50640b2860 con 0x7f506415d2d0 2023-10-06T05:22:38.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f506a5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506415d7d0 con 0x7f50640cd510 2023-10-06T05:22:38.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f5063fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5064159500 0x7f50640cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44496/0 (socket says 172.21.15.31:44496) 2023-10-06T05:22:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f5063fff700 1 -- 172.21.15.31:0/279676358 learned_addr learned my addr 172.21.15.31:0/279676358 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f5068b9c700 1 --2- 172.21.15.31:0/279676358 >> v2:172.21.15.31:3300/0 conn(0x7f506415d2d0 0x7f50640aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f50637fe700 1 --2- 172.21.15.31:0/279676358 >> v2:172.21.15.62:3300/0 conn(0x7f50640cd510 0x7f50640b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f5062ffd700 1 -- 172.21.15.31:0/279676358 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223209668 0 0) 0x7f50640c3f40 con 0x7f5064159500 2023-10-06T05:22:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f5062ffd700 1 -- 172.21.15.31:0/279676358 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f504c003570 con 0x7f5064159500 2023-10-06T05:22:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f5068b9c700 1 -- 172.21.15.31:0/279676358 >> v2:172.21.15.62:3300/0 conn(0x7f50640cd510 msgr2=0x7f50640b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f5068b9c700 1 --2- 172.21.15.31:0/279676358 >> v2:172.21.15.62:3300/0 conn(0x7f50640cd510 0x7f50640b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f5068b9c700 1 -- 172.21.15.31:0/279676358 >> v1:172.21.15.31:6789/0 conn(0x7f5064159500 legacy=0x7f50640cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.798+0000 7f5068b9c700 1 -- 172.21.15.31:0/279676358 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50640cb000 con 0x7f506415d2d0 2023-10-06T05:22:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.799+0000 7f5068b9c700 1 --2- 172.21.15.31:0/279676358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 0x7f50640aedd0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f506415d230 tx=0x7f505c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6902f8fa6045dbdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.799+0000 7f5062ffd700 1 -- 172.21.15.31:0/279676358 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00e070 con 0x7f506415d2d0 2023-10-06T05:22:38.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.799+0000 7f5062ffd700 1 -- 172.21.15.31:0/279676358 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c00bab0 con 0x7f506415d2d0 2023-10-06T05:22:38.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.799+0000 7f5062ffd700 1 -- 172.21.15.31:0/279676358 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00be70 con 0x7f506415d2d0 2023-10-06T05:22:38.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.799+0000 7f506a5ff700 1 -- 172.21.15.31:0/279676358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 msgr2=0x7f50640aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.799+0000 7f506a5ff700 1 --2- 172.21.15.31:0/279676358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 0x7f50640aedd0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f506415d230 tx=0x7f505c009780 comp rx=0 tx=0).stop 2023-10-06T05:22:38.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.800+0000 7f506a5ff700 1 -- 172.21.15.31:0/279676358 shutdown_connections 2023-10-06T05:22:38.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.800+0000 7f506a5ff700 1 --2- 172.21.15.31:0/279676358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 0x7f50640aedd0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.800+0000 7f506a5ff700 1 --2- 172.21.15.31:0/279676358 >> v2:172.21.15.62:3300/0 conn(0x7f50640cd510 0x7f50640b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.800+0000 7f506a5ff700 1 -- 172.21.15.31:0/279676358 >> 172.21.15.31:0/279676358 conn(0x7f506406ec90 msgr2=0x7f506406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.800+0000 7f506a5ff700 1 -- 172.21.15.31:0/279676358 shutdown_connections 2023-10-06T05:22:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.800+0000 7f506a5ff700 1 -- 172.21.15.31:0/279676358 wait complete. 2023-10-06T05:22:38.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.801+0000 7f506a5ff700 1 Processor -- start 2023-10-06T05:22:38.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.801+0000 7f506a5ff700 1 -- start start 2023-10-06T05:22:38.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.802+0000 7f506a5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50640cd510 0x7f50640c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.802+0000 7f506a5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5064159500 0x7f50640b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.802+0000 7f506a5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 0x7f5064155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.802+0000 7f506a5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50640b4e10 con 0x7f506415d2d0 2023-10-06T05:22:38.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.802+0000 7f506a5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50640b50f0 con 0x7f5064159500 2023-10-06T05:22:38.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.802+0000 7f506a5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50640b53d0 con 0x7f50640cd510 2023-10-06T05:22:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.802+0000 7f5068b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 0x7f5064155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f5068b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 0x7f5064155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42478/0 (socket says 172.21.15.31:42478) 2023-10-06T05:22:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f5068b9c700 1 -- 172.21.15.31:0/135917521 learned_addr learned my addr 172.21.15.31:0/135917521 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f5063fff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50640cd510 0x7f50640c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f50637fe700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5064159500 0x7f50640b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f5068b9c700 1 -- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50640cd510 msgr2=0x7f50640c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f5068b9c700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50640cd510 0x7f50640c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f5068b9c700 1 -- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5064159500 msgr2=0x7f50640b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f5068b9c700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5064159500 0x7f50640b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f5068b9c700 1 -- 172.21.15.31:0/135917521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c009a20 con 0x7f506415d2d0 2023-10-06T05:22:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f5063fff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50640cd510 0x7f50640c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f50637fe700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5064159500 0x7f50640b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.803+0000 7f5068b9c700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 0x7f5064155110 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5064067c50 tx=0x7f505c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.804+0000 7f504bfff700 1 -- 172.21.15.31:0/135917521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c012e60 con 0x7f506415d2d0 2023-10-06T05:22:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.804+0000 7f504bfff700 1 -- 172.21.15.31:0/135917521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c027470 con 0x7f506415d2d0 2023-10-06T05:22:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.804+0000 7f506a5ff700 1 -- 172.21.15.31:0/135917521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50640b56b0 con 0x7f506415d2d0 2023-10-06T05:22:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.804+0000 7f504bfff700 1 -- 172.21.15.31:0/135917521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c027810 con 0x7f506415d2d0 2023-10-06T05:22:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.804+0000 7f506a5ff700 1 -- 172.21.15.31:0/135917521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506415c5d0 con 0x7f506415d2d0 2023-10-06T05:22:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.805+0000 7f506a5ff700 1 -- 172.21.15.31:0/135917521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506404c380 con 0x7f506415d2d0 2023-10-06T05:22:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.806+0000 7f504bfff700 1 -- 172.21.15.31:0/135917521 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f505c024070 con 0x7f506415d2d0 2023-10-06T05:22:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.806+0000 7f504bfff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5050067050 0x7f5050069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.807+0000 7f504bfff700 1 -- 172.21.15.31:0/135917521 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f505c086f40 con 0x7f506415d2d0 2023-10-06T05:22:38.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.807+0000 7f5063fff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5050067050 0x7f5050069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:38.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.807+0000 7f5063fff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5050067050 0x7f5050069500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f50540058d0 tx=0x7f5054009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:38.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:38.811+0000 7f504bfff700 1 -- 172.21.15.31:0/135917521 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f505c053ea0 con 0x7f506415d2d0 2023-10-06T05:22:39.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.056+0000 7f506a5ff700 1 -- 172.21.15.31:0/135917521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5064159120 con 0x7f506415d2d0 2023-10-06T05:22:39.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.056+0000 7f504bfff700 1 -- 172.21.15.31:0/135917521 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f505c02c090 con 0x7f506415d2d0 2023-10-06T05:22:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:39.059 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1885821393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1885821393},{"type":"v1","addr":"172.21.15.31:6834","nonce":1885821393}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6889,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/258742412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":258742412},{"type":"v1","addr":"172.21.15.31:6835","nonce":258742412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4070877337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4070877337},{"type":"v1","addr":"172.21.15.62:6834","nonce":4070877337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.638176+0000","modified":"2023-10-06T05:22:36.157996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7059":{"gid":7059,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2094863015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2094863015},{"type":"v1","addr":"172.21.15.62:6835","nonce":2094863015}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:22:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 -- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5050067050 msgr2=0x7f5050069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5050067050 0x7f5050069500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f50540058d0 tx=0x7f5054009040 comp rx=0 tx=0).stop 2023-10-06T05:22:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 -- 172.21.15.31:0/135917521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 msgr2=0x7f5064155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 0x7f5064155110 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5064067c50 tx=0x7f505c019000 comp rx=0 tx=0).stop 2023-10-06T05:22:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 -- 172.21.15.31:0/135917521 shutdown_connections 2023-10-06T05:22:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5050067050 0x7f5050069500 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506415d2d0 0x7f5064155110 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50640cd510 0x7f50640c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 --2- 172.21.15.31:0/135917521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5064159500 0x7f50640b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 -- 172.21.15.31:0/135917521 >> 172.21.15.31:0/135917521 conn(0x7f506406ec90 msgr2=0x7f50640cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 -- 172.21.15.31:0/135917521 shutdown_connections 2023-10-06T05:22:39.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.059+0000 7f506a5ff700 1 -- 172.21.15.31:0/135917521 wait complete. 2023-10-06T05:22:39.063 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 67 2023-10-06T05:22:39.119 INFO:tasks.ceph.mds.d.smithi062.stderr:2023-10-06T05:22:39.117+0000 7f4fbcdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:22:39.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:22:32.638176+0000', 'modified': '2023-10-06T05:22:36.157996+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7059}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7059': {'gid': 7059, 'name': 'd', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/2094863015', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 2094863015}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2094863015}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:22:39.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:22:39.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:22:39.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.686+0000 7f31c2fff700 1 Processor -- start 2023-10-06T05:22:39.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.686+0000 7f31c2fff700 1 -- start start 2023-10-06T05:22:39.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.687+0000 7f31c2fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f31bc13a1e0 0x7f31bc0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.688+0000 7f31c2fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31bc0c71b0 con 0x7f31bc06db50 2023-10-06T05:22:39.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.688+0000 7f31c2fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31bc0c7490 con 0x7f31bc13a5b0 2023-10-06T05:22:39.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.688+0000 7f31c2fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31bc0c7770 con 0x7f31bc13a1e0 2023-10-06T05:22:39.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.688+0000 7f31c159c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f31bc06db50 0x7f31bc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44500/0 (socket says 172.21.15.31:44500) 2023-10-06T05:22:39.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.688+0000 7f31c159c700 1 -- 172.21.15.31:0/3743967674 learned_addr learned my addr 172.21.15.31:0/3743967674 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:39.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.688+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/3743967674 >> v2:172.21.15.62:3300/0 conn(0x7f31bc13a1e0 0x7f31bc0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.688+0000 7f31bb7fe700 1 -- 172.21.15.31:0/3743967674 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161246195 0 0) 0x7f31bc0c71b0 con 0x7f31bc06db50 2023-10-06T05:22:39.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.688+0000 7f31bb7fe700 1 -- 172.21.15.31:0/3743967674 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31ac003570 con 0x7f31bc06db50 2023-10-06T05:22:39.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.689+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3743967674 >> v1:172.21.15.62:6789/0 conn(0x7f31bc13a5b0 legacy=0x7f31bc06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.689+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3743967674 >> v1:172.21.15.31:6789/0 conn(0x7f31bc06db50 legacy=0x7f31bc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.689+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3743967674 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc0c7a50 con 0x7f31bc13a1e0 2023-10-06T05:22:39.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.689+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/3743967674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31bc13a1e0 0x7f31bc0c66f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f31b00098d0 tx=0x7f31b0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cff3797c338ef7a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.689+0000 7f31bb7fe700 1 -- 172.21.15.31:0/3743967674 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b0005c10 con 0x7f31bc13a1e0 2023-10-06T05:22:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.689+0000 7f31bb7fe700 1 -- 172.21.15.31:0/3743967674 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b0005d70 con 0x7f31bc13a1e0 2023-10-06T05:22:39.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.689+0000 7f31bb7fe700 1 -- 172.21.15.31:0/3743967674 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b0006240 con 0x7f31bc13a1e0 2023-10-06T05:22:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.690+0000 7f31c2fff700 1 -- 172.21.15.31:0/3743967674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31bc13a1e0 msgr2=0x7f31bc0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.690+0000 7f31c2fff700 1 --2- 172.21.15.31:0/3743967674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31bc13a1e0 0x7f31bc0c66f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f31b00098d0 tx=0x7f31b0009120 comp rx=0 tx=0).stop 2023-10-06T05:22:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.690+0000 7f31c2fff700 1 -- 172.21.15.31:0/3743967674 shutdown_connections 2023-10-06T05:22:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.690+0000 7f31c2fff700 1 --2- 172.21.15.31:0/3743967674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31bc13a1e0 0x7f31bc0c66f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.690+0000 7f31c2fff700 1 -- 172.21.15.31:0/3743967674 >> 172.21.15.31:0/3743967674 conn(0x7f31bc0cc0b0 msgr2=0x7f31bc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.690+0000 7f31c2fff700 1 -- 172.21.15.31:0/3743967674 shutdown_connections 2023-10-06T05:22:39.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.690+0000 7f31c2fff700 1 -- 172.21.15.31:0/3743967674 wait complete. 2023-10-06T05:22:39.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.691+0000 7f31c2fff700 1 Processor -- start 2023-10-06T05:22:39.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.692+0000 7f31c2fff700 1 -- start start 2023-10-06T05:22:39.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.692+0000 7f31c2fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31bc06db50 0x7f31bc155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.692+0000 7f31c2fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31bc13a1e0 0x7f31bc16a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.692+0000 7f31c2fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31bc13a5b0 0x7f31bc16a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.692+0000 7f31c2fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc16ae00 con 0x7f31bc06db50 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.692+0000 7f31c2fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc152950 con 0x7f31bc13a5b0 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.692+0000 7f31c2fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc152060 con 0x7f31bc13a1e0 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31c0d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31bc06db50 0x7f31bc155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31c0d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31bc06db50 0x7f31bc155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42484/0 (socket says 172.21.15.31:42484) 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3244577156 learned_addr learned my addr 172.21.15.31:0/3244577156 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31c159c700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31bc13a5b0 0x7f31bc16a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31bbfff700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31bc13a1e0 0x7f31bc16a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31bc13a1e0 msgr2=0x7f31bc16a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31bc13a1e0 0x7f31bc16a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31bc13a5b0 msgr2=0x7f31bc16a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31bc13a5b0 0x7f31bc16a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3244577156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b00093c0 con 0x7f31bc06db50 2023-10-06T05:22:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.693+0000 7f31bbfff700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31bc13a1e0 0x7f31bc16a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.694+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31bc06db50 0x7f31bc155800 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009f70 tx=0x7f31b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.694+0000 7f31b8ff9700 1 -- 172.21.15.31:0/3244577156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b001d730 con 0x7f31bc06db50 2023-10-06T05:22:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.694+0000 7f31b8ff9700 1 -- 172.21.15.31:0/3244577156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b001d8b0 con 0x7f31bc06db50 2023-10-06T05:22:39.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.694+0000 7f31c2fff700 1 -- 172.21.15.31:0/3244577156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc0c4790 con 0x7f31bc06db50 2023-10-06T05:22:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.694+0000 7f31b8ff9700 1 -- 172.21.15.31:0/3244577156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b001dc70 con 0x7f31bc06db50 2023-10-06T05:22:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.694+0000 7f31c2fff700 1 -- 172.21.15.31:0/3244577156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc1525e0 con 0x7f31bc06db50 2023-10-06T05:22:39.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.696+0000 7f31b8ff9700 1 -- 172.21.15.31:0/3244577156 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f31b001b690 con 0x7f31bc06db50 2023-10-06T05:22:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.697+0000 7f31b8ff9700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31a0067050 0x7f31a0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.697+0000 7f31b8ff9700 1 -- 172.21.15.31:0/3244577156 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f31b0086520 con 0x7f31bc06db50 2023-10-06T05:22:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.697+0000 7f31bbfff700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31a0067050 0x7f31a0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.698+0000 7f31bbfff700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31a0067050 0x7f31a0069500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f31bc16d3a0 tx=0x7f31a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.699+0000 7f31d173db80 1 Processor -- start 2023-10-06T05:22:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.699+0000 7f31d173db80 1 -- start start 2023-10-06T05:22:39.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31d173db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b537261820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31d173db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537269f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31d173db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 0x55b53726a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31d173db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b537134310 con 0x55b537246f40 2023-10-06T05:22:39.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31d173db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b537171130 con 0x55b53726a4d0 2023-10-06T05:22:39.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31d173db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b53716eaf0 con 0x55b537261450 2023-10-06T05:22:39.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537269f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31c0d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b537261820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537269f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42500/0 (socket says 172.21.15.31:42500) 2023-10-06T05:22:39.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31bbfff700 1 -- 172.21.15.31:0/994529390 learned_addr learned my addr 172.21.15.31:0/994529390 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:39.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31c159c700 1 --2- 172.21.15.31:0/994529390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 0x55b53726a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31bbfff700 1 -- 172.21.15.31:0/994529390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 msgr2=0x55b537261820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31bbfff700 1 --2- 172.21.15.31:0/994529390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b537261820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31bbfff700 1 -- 172.21.15.31:0/994529390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 msgr2=0x55b53726a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31bbfff700 1 --2- 172.21.15.31:0/994529390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 0x55b53726a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.700+0000 7f31bbfff700 1 -- 172.21.15.31:0/994529390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b536f70790 con 0x55b537246f40 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.701+0000 7f31bbfff700 1 --2- 172.21.15.31:0/994529390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537269f90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f31a800e7c0 tx=0x7f31a8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f20f15d7b300a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.701+0000 7f31a5ffb700 1 -- 172.21.15.31:0/994529390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8013700 con 0x55b537246f40 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.701+0000 7f31a5ffb700 1 -- 172.21.15.31:0/994529390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a8013860 con 0x55b537246f40 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.701+0000 7f31a5ffb700 1 -- 172.21.15.31:0/994529390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8013be0 con 0x55b537246f40 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.701+0000 7f31d173db80 1 -- 172.21.15.31:0/994529390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 msgr2=0x55b537269f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.701+0000 7f31d173db80 1 --2- 172.21.15.31:0/994529390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537269f90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f31a800e7c0 tx=0x7f31a8013000 comp rx=0 tx=0).stop 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.702+0000 7f31d173db80 1 -- 172.21.15.31:0/994529390 shutdown_connections 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.702+0000 7f31d173db80 1 --2- 172.21.15.31:0/994529390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 0x55b53726a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.702+0000 7f31d173db80 1 --2- 172.21.15.31:0/994529390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b537261820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.702+0000 7f31d173db80 1 --2- 172.21.15.31:0/994529390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537269f90 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.702+0000 7f31d173db80 1 -- 172.21.15.31:0/994529390 >> 172.21.15.31:0/994529390 conn(0x55b537264880 msgr2=0x55b537264c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.702+0000 7f31d173db80 1 -- 172.21.15.31:0/994529390 shutdown_connections 2023-10-06T05:22:39.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.702+0000 7f31d173db80 1 -- 172.21.15.31:0/994529390 wait complete. 2023-10-06T05:22:39.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.702+0000 7f31d173db80 1 -- start start 2023-10-06T05:22:39.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.703+0000 7f31d173db80 1 Processor -- start 2023-10-06T05:22:39.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31d173db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 0x55b53723e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31d173db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537261450 0x55b53723eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31d173db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b53726a4d0 0x55b537262a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31d173db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b53716a8e0 con 0x55b537261450 2023-10-06T05:22:39.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31d173db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b537116980 con 0x55b537246f40 2023-10-06T05:22:39.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31d173db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b537116620 con 0x55b53726a4d0 2023-10-06T05:22:39.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31d173db80 10 client.0 _ll_register_callbacks cb 0x55b536f79a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:22:39.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537261450 0x55b53723eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537261450 0x55b53723eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42510/0 (socket says 172.21.15.31:42510) 2023-10-06T05:22:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31bbfff700 1 -- 172.21.15.31:0/1400522720 learned_addr learned my addr 172.21.15.31:0/1400522720 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 0x55b53723e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31c159c700 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b53726a4d0 0x55b537262a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31bbfff700 1 -- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b53726a4d0 msgr2=0x55b537262a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31bbfff700 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b53726a4d0 0x55b537262a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31bbfff700 1 -- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 msgr2=0x55b53723e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31bbfff700 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 0x55b53723e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31bbfff700 1 -- 172.21.15.31:0/1400522720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a800f040 con 0x55b537261450 2023-10-06T05:22:39.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.704+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 0x55b53723e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.705+0000 7f31bbfff700 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537261450 0x55b53723eb10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f31a8014000 tx=0x7f31a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c543f3744b3c165d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.705+0000 7f319affd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.705+0000 7f319affd700 1 -- 172.21.15.31:0/1400522720 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8028070 con 0x55b537261450 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.705+0000 7f319affd700 1 -- 172.21.15.31:0/1400522720 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f31a8015850 con 0x55b537261450 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.705+0000 7f31d173db80 1 -- 172.21.15.31:0/1400522720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b53720ed60 con 0x55b537261450 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.705+0000 7f319affd700 1 -- 172.21.15.31:0/1400522720 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31a8015c20 con 0x55b537261450 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.705+0000 7f31d173db80 10 client.7194 fetch_fsmap learned FSMap version 67 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.705+0000 7f31d173db80 1 -- 172.21.15.31:0/1400522720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55b53705d0a0 con 0x55b537261450 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.705+0000 7f319affd700 1 -- 172.21.15.31:0/1400522720 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b53720ed60 con 0x55b537261450 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.706+0000 7f319affd700 1 -- 172.21.15.31:0/1400522720 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 67) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f31a8015080 con 0x55b537261450 2023-10-06T05:22:39.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.706+0000 7f31d173db80 10 client.7194 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.706+0000 7f31d173db80 10 client.7194 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.706+0000 7f31d173db80 20 client.7194 populate_metadata read hostname 'smithi031' 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.706+0000 7f31d173db80 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:6833/2094863015,v1:172.21.15.62:6835/2094863015] conn(0x55b537262f60 0x55b537273dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.706+0000 7f31d173db80 4 client.7194 mds_command: new command op to 7059 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.706+0000 7f31d173db80 1 -- 172.21.15.31:0/1400522720 --> [v2:172.21.15.62:6833/2094863015,v1:172.21.15.62:6835/2094863015] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5371ebb30 con 0x55b537262f60 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.706+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:6833/2094863015,v1:172.21.15.62:6835/2094863015] conn(0x55b537262f60 0x55b537273dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.707+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:6833/2094863015,v1:172.21.15.62:6835/2094863015] conn(0x55b537262f60 0x55b537273dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b7f15531aa75980 server_cookie=96c39411a951b1b in_seq=0 out_seq=0 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.707+0000 7f319affd700 10 client.7194 ms_handle_connect on v2:172.21.15.62:6833/2094863015 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f319affd700 1 -- 172.21.15.31:0/1400522720 <== mds.0 v2:172.21.15.62:6833/2094863015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55b5371ebb30 con 0x55b537262f60 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f319affd700 10 client.7194 handle_command_reply: tid=0 2023-10-06T05:22:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 client.7194 shutdown 2023-10-06T05:22:39.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 -- 172.21.15.31:0/1400522720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537261450 msgr2=0x55b53723eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537261450 0x55b53723eb10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f31a8014000 tx=0x7f31a8017000 comp rx=0 tx=0).stop 2023-10-06T05:22:39.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 -- 172.21.15.31:0/1400522720 shutdown_connections 2023-10-06T05:22:39.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:6833/2094863015,v1:172.21.15.62:6835/2094863015] conn(0x55b537262f60 0x55b537273dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b53726a4d0 0x55b537262a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 0x55b53723e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 --2- 172.21.15.31:0/1400522720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537261450 0x55b53723eb10 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 -- 172.21.15.31:0/1400522720 >> 172.21.15.31:0/1400522720 conn(0x55b537264880 msgr2=0x55b53721a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f319affd700 0 client.7194 ms_handle_reset on v2:172.21.15.62:6833/2094863015 2023-10-06T05:22:39.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 -- 172.21.15.31:0/1400522720 shutdown_connections 2023-10-06T05:22:39.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 1 -- 172.21.15.31:0/1400522720 wait complete. 2023-10-06T05:22:39.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.708+0000 7f31d173db80 20 client.7194 trim_cache size 0 max 16384 2023-10-06T05:22:39.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.726+0000 7f31d173db80 1 Processor -- start 2023-10-06T05:22:39.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.726+0000 7f31d173db80 1 -- start start 2023-10-06T05:22:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31d173db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 0x55b5372728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31d173db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b53721a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31d173db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b53726a4d0 0x55b53721a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31d173db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5370021e0 con 0x55b53726a4d0 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31d173db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5372167c0 con 0x55b537246f40 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31d173db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b537234ae0 con 0x55b537261450 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31c159c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b53726a4d0 0x55b53721a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31c159c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b53726a4d0 0x55b53721a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42514/0 (socket says 172.21.15.31:42514) 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31c159c700 1 -- 172.21.15.31:0/3384119979 learned_addr learned my addr 172.21.15.31:0/3384119979 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31bbfff700 1 --2- 172.21.15.31:0/3384119979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b53721a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/3384119979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 0x55b5372728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31c159c700 1 -- 172.21.15.31:0/3384119979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 msgr2=0x55b53721a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31c159c700 1 --2- 172.21.15.31:0/3384119979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b53721a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31c159c700 1 -- 172.21.15.31:0/3384119979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 msgr2=0x55b5372728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31c159c700 1 --2- 172.21.15.31:0/3384119979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 0x55b5372728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.727+0000 7f31c159c700 1 -- 172.21.15.31:0/3384119979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5371393c0 con 0x55b53726a4d0 2023-10-06T05:22:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31c159c700 1 --2- 172.21.15.31:0/3384119979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b53726a4d0 0x55b53721a600 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f31b4015470 tx=0x7f31b4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8fc01dc6dfb041 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31a67fc700 1 -- 172.21.15.31:0/3384119979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4007110 con 0x55b53726a4d0 2023-10-06T05:22:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31a67fc700 1 -- 172.21.15.31:0/3384119979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b401a070 con 0x55b53726a4d0 2023-10-06T05:22:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31a67fc700 1 -- 172.21.15.31:0/3384119979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4007290 con 0x55b53726a4d0 2023-10-06T05:22:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31d173db80 1 -- 172.21.15.31:0/3384119979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b53726a4d0 msgr2=0x55b53721a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31d173db80 1 --2- 172.21.15.31:0/3384119979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b53726a4d0 0x55b53721a600 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f31b4015470 tx=0x7f31b4009040 comp rx=0 tx=0).stop 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31d173db80 1 -- 172.21.15.31:0/3384119979 shutdown_connections 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31d173db80 1 --2- 172.21.15.31:0/3384119979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b53726a4d0 0x55b53721a600 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31d173db80 1 --2- 172.21.15.31:0/3384119979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b537246f40 0x55b5372728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31d173db80 1 --2- 172.21.15.31:0/3384119979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b53721a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31d173db80 1 -- 172.21.15.31:0/3384119979 >> 172.21.15.31:0/3384119979 conn(0x55b537264880 msgr2=0x55b537244600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.728+0000 7f31d173db80 1 -- 172.21.15.31:0/3384119979 shutdown_connections 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.729+0000 7f31d173db80 1 -- 172.21.15.31:0/3384119979 wait complete. 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.729+0000 7f31d173db80 1 -- start start 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.729+0000 7f31d173db80 1 Processor -- start 2023-10-06T05:22:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31d173db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537240520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31d173db80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b5372630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31d173db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 0x55b53723b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31d173db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b537240a60 con 0x55b537246f40 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31d173db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b53723ecf0 con 0x55b53726a4d0 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31d173db80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b537262990 con 0x55b537261450 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31d173db80 10 client.0 _ll_register_callbacks cb 0x55b536f79a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c0d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537240520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c0d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537240520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42518/0 (socket says 172.21.15.31:42518) 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3795216791 learned_addr learned my addr 172.21.15.31:0/3795216791 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c159c700 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 0x55b53723b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31bbfff700 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b5372630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 msgr2=0x55b5372630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b5372630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 msgr2=0x55b53723b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 0x55b53723b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c0d9b700 1 -- 172.21.15.31:0/3795216791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ac003570 con 0x55b537246f40 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c159c700 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 0x55b53723b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f31c0d9b700 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537240520 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f31b00069e0 tx=0x7f31b0080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9271fc2728fd3a08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.731+0000 7f319affd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f319affd700 1 -- 172.21.15.31:0/3795216791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b005e160 con 0x55b537246f40 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f319affd700 1 -- 172.21.15.31:0/3795216791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f31b00554e0 con 0x55b537246f40 2023-10-06T05:22:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f31d173db80 1 -- 172.21.15.31:0/3795216791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b537260f30 con 0x55b537246f40 2023-10-06T05:22:39.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f319affd700 1 -- 172.21.15.31:0/3795216791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31b0084a40 con 0x55b537246f40 2023-10-06T05:22:39.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f31d173db80 10 client.7200 fetch_fsmap learned FSMap version 67 2023-10-06T05:22:39.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f31d173db80 1 -- 172.21.15.31:0/3795216791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55b536fd11d0 con 0x55b537246f40 2023-10-06T05:22:39.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f319affd700 1 -- 172.21.15.31:0/3795216791 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b537260f30 con 0x55b537246f40 2023-10-06T05:22:39.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f319affd700 1 -- 172.21.15.31:0/3795216791 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 67) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f31b001a3e0 con 0x55b537246f40 2023-10-06T05:22:39.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f31d173db80 10 client.7200 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T05:22:39.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f31d173db80 10 client.7200 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:22:39.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f31d173db80 20 client.7200 populate_metadata read hostname 'smithi031' 2023-10-06T05:22:39.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f31d173db80 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:6833/2094863015,v1:172.21.15.62:6835/2094863015] conn(0x55b53723ba70 0x55b537249360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f31d173db80 4 client.7200 mds_command: new command op to 7059 tid=0[{"prefix": "status"}] 2023-10-06T05:22:39.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.732+0000 7f31d173db80 1 -- 172.21.15.31:0/3795216791 --> [v2:172.21.15.62:6833/2094863015,v1:172.21.15.62:6835/2094863015] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f31a8013860 con 0x55b53723ba70 2023-10-06T05:22:39.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.733+0000 7f31bbfff700 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:6833/2094863015,v1:172.21.15.62:6835/2094863015] conn(0x55b53723ba70 0x55b537249360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.733+0000 7f31bbfff700 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:6833/2094863015,v1:172.21.15.62:6835/2094863015] conn(0x55b53723ba70 0x55b537249360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ebbb5b3f03e0335 server_cookie=b825bae4898dbda1 in_seq=0 out_seq=0 2023-10-06T05:22:39.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.733+0000 7f319affd700 10 client.7200 ms_handle_connect on v2:172.21.15.62:6833/2094863015 2023-10-06T05:22:39.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f319affd700 1 -- 172.21.15.31:0/3795216791 <== mds.0 v2:172.21.15.62:6833/2094863015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f31a8013860 con 0x55b53723ba70 2023-10-06T05:22:39.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f319affd700 10 client.7200 handle_command_reply: tid=0 2023-10-06T05:22:39.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 client.7200 shutdown 2023-10-06T05:22:39.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 -- 172.21.15.31:0/3795216791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 msgr2=0x55b537240520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537240520 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f31b00069e0 tx=0x7f31b0080040 comp rx=0 tx=0).stop 2023-10-06T05:22:39.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 -- 172.21.15.31:0/3795216791 shutdown_connections 2023-10-06T05:22:39.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:6833/2094863015,v1:172.21.15.62:6835/2094863015] conn(0x55b53723ba70 0x55b537249360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b53726a4d0 0x55b53723b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b537246f40 0x55b537240520 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 --2- 172.21.15.31:0/3795216791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b537261450 0x55b5372630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f319affd700 0 client.7200 ms_handle_reset on v2:172.21.15.62:6833/2094863015 2023-10-06T05:22:39.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 -- 172.21.15.31:0/3795216791 >> 172.21.15.31:0/3795216791 conn(0x55b537264880 msgr2=0x55b537239fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 -- 172.21.15.31:0/3795216791 shutdown_connections 2023-10-06T05:22:39.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 1 -- 172.21.15.31:0/3795216791 wait complete. 2023-10-06T05:22:39.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.734+0000 7f31d173db80 20 client.7200 trim_cache size 0 max 16384 2023-10-06T05:22:39.747 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 7059, 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.6053939399999999, 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 67, 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 101, 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 101, 2023-10-06T05:22:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.260314162 2023-10-06T05:22:39.749 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:22:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 -- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31a0067050 msgr2=0x7f31a0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31a0067050 0x7f31a0069500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f31bc16d3a0 tx=0x7f31a8009040 comp rx=0 tx=0).stop 2023-10-06T05:22:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 -- 172.21.15.31:0/3244577156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31bc06db50 msgr2=0x7f31bc155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31bc06db50 0x7f31bc155800 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009f70 tx=0x7f31b0016000 comp rx=0 tx=0).stop 2023-10-06T05:22:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 -- 172.21.15.31:0/3244577156 shutdown_connections 2023-10-06T05:22:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31a0067050 0x7f31a0069500 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31bc13a5b0 0x7f31bc16a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31bc06db50 0x7f31bc155800 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 --2- 172.21.15.31:0/3244577156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31bc13a1e0 0x7f31bc16a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 -- 172.21.15.31:0/3244577156 >> 172.21.15.31:0/3244577156 conn(0x7f31bc0cc0b0 msgr2=0x7f31bc0c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 -- 172.21.15.31:0/3244577156 shutdown_connections 2023-10-06T05:22:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.735+0000 7f31c2fff700 1 -- 172.21.15.31:0/3244577156 wait complete. 2023-10-06T05:22:39.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:22:39.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.949+0000 7fd02ddff700 1 Processor -- start 2023-10-06T05:22:39.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.949+0000 7fd02ddff700 1 -- start start 2023-10-06T05:22:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.950+0000 7fd02ddff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd028159500 0x7fd0280b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.950+0000 7fd02ddff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd02815d2d0 0x7fd0280cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.950+0000 7fd02ddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0281598d0 0x7fd0280cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.950+0000 7fd02ddff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0280c3f40 con 0x7fd028159500 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.950+0000 7fd02ddff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0280cb000 con 0x7fd02815d2d0 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.950+0000 7fd02ddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0280b2860 con 0x7fd0281598d0 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.951+0000 7fd0277fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd028159500 0x7fd0280b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.951+0000 7fd0277fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd028159500 0x7fd0280b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42530/0 (socket says 172.21.15.31:42530) 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.951+0000 7fd0277fe700 1 -- 172.21.15.31:0/893535705 learned_addr learned my addr 172.21.15.31:0/893535705 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.951+0000 7fd027fff700 1 --2- 172.21.15.31:0/893535705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0281598d0 0x7fd0280cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.951+0000 7fd026ffd700 1 --2- 172.21.15.31:0/893535705 >> v2:172.21.15.62:3300/0 conn(0x7fd02815d2d0 0x7fd0280cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.951+0000 7fd0277fe700 1 -- 172.21.15.31:0/893535705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0281598d0 msgr2=0x7fd0280cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.951+0000 7fd0277fe700 1 --2- 172.21.15.31:0/893535705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0281598d0 0x7fd0280cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.951+0000 7fd0277fe700 1 -- 172.21.15.31:0/893535705 >> v2:172.21.15.62:3300/0 conn(0x7fd02815d2d0 msgr2=0x7fd0280cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.951+0000 7fd0277fe700 1 --2- 172.21.15.31:0/893535705 >> v2:172.21.15.62:3300/0 conn(0x7fd02815d2d0 0x7fd0280cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.951+0000 7fd0277fe700 1 -- 172.21.15.31:0/893535705 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02815d7d0 con 0x7fd028159500 2023-10-06T05:22:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.952+0000 7fd0277fe700 1 --2- 172.21.15.31:0/893535705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 0x7fd0280b0ab0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd02815d200 tx=0x7fd018009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a5f7511b8a5368 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.952+0000 7fd0267fc700 1 -- 172.21.15.31:0/893535705 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01800b640 con 0x7fd028159500 2023-10-06T05:22:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.952+0000 7fd0267fc700 1 -- 172.21.15.31:0/893535705 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01800b7a0 con 0x7fd028159500 2023-10-06T05:22:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.952+0000 7fd0267fc700 1 -- 172.21.15.31:0/893535705 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01800bb60 con 0x7fd028159500 2023-10-06T05:22:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.952+0000 7fd02ddff700 1 -- 172.21.15.31:0/893535705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 msgr2=0x7fd0280b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.952+0000 7fd02ddff700 1 --2- 172.21.15.31:0/893535705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 0x7fd0280b0ab0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd02815d200 tx=0x7fd018009250 comp rx=0 tx=0).stop 2023-10-06T05:22:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.953+0000 7fd02ddff700 1 -- 172.21.15.31:0/893535705 shutdown_connections 2023-10-06T05:22:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.953+0000 7fd02ddff700 1 --2- 172.21.15.31:0/893535705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0281598d0 0x7fd0280cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.953+0000 7fd02ddff700 1 --2- 172.21.15.31:0/893535705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 0x7fd0280b0ab0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.953+0000 7fd02ddff700 1 --2- 172.21.15.31:0/893535705 >> v2:172.21.15.62:3300/0 conn(0x7fd02815d2d0 0x7fd0280cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.953+0000 7fd02ddff700 1 -- 172.21.15.31:0/893535705 >> 172.21.15.31:0/893535705 conn(0x7fd02806ec90 msgr2=0x7fd02806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.953+0000 7fd02ddff700 1 -- 172.21.15.31:0/893535705 shutdown_connections 2023-10-06T05:22:39.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.953+0000 7fd02ddff700 1 -- 172.21.15.31:0/893535705 wait complete. 2023-10-06T05:22:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.955+0000 7fd02ddff700 1 Processor -- start 2023-10-06T05:22:39.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.955+0000 7fd02ddff700 1 -- start start 2023-10-06T05:22:39.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.956+0000 7fd02ddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 0x7fd0280c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.956+0000 7fd02ddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0281598d0 0x7fd0280c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.956+0000 7fd02ddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd02815d2d0 0x7fd0280c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.956+0000 7fd02ddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02815b8f0 con 0x7fd028159500 2023-10-06T05:22:39.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.956+0000 7fd02ddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0280c7570 con 0x7fd02815d2d0 2023-10-06T05:22:39.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.956+0000 7fd02ddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0280c7850 con 0x7fd0281598d0 2023-10-06T05:22:39.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.956+0000 7fd0277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 0x7fd0280c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.956+0000 7fd0277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 0x7fd0280c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42542/0 (socket says 172.21.15.31:42542) 2023-10-06T05:22:39.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.956+0000 7fd0277fe700 1 -- 172.21.15.31:0/1531666815 learned_addr learned my addr 172.21.15.31:0/1531666815 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.957+0000 7fd0277fe700 1 -- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0281598d0 msgr2=0x7fd0280c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.957+0000 7fd0277fe700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0281598d0 0x7fd0280c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.957+0000 7fd0277fe700 1 -- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd02815d2d0 msgr2=0x7fd0280c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.957+0000 7fd0277fe700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd02815d2d0 0x7fd0280c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.957+0000 7fd0277fe700 1 -- 172.21.15.31:0/1531666815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0180094f0 con 0x7fd028159500 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.957+0000 7fd0277fe700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 0x7fd0280c8330 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd028067c50 tx=0x7fd018017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.957+0000 7fd00f7fe700 1 -- 172.21.15.31:0/1531666815 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018000d10 con 0x7fd028159500 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.957+0000 7fd00f7fe700 1 -- 172.21.15.31:0/1531666815 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01800bb20 con 0x7fd028159500 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.957+0000 7fd02ddff700 1 -- 172.21.15.31:0/1531666815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0280c7b30 con 0x7fd028159500 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.957+0000 7fd00f7fe700 1 -- 172.21.15.31:0/1531666815 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018011670 con 0x7fd028159500 2023-10-06T05:22:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.958+0000 7fd02ddff700 1 -- 172.21.15.31:0/1531666815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0280c92f0 con 0x7fd028159500 2023-10-06T05:22:39.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.958+0000 7fd02ddff700 1 -- 172.21.15.31:0/1531666815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0280c8050 con 0x7fd028159500 2023-10-06T05:22:39.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.959+0000 7fd00f7fe700 1 -- 172.21.15.31:0/1531666815 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd0180119e0 con 0x7fd028159500 2023-10-06T05:22:39.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.960+0000 7fd00f7fe700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd014067050 0x7fd014069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:39.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.960+0000 7fd00f7fe700 1 -- 172.21.15.31:0/1531666815 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd018086150 con 0x7fd028159500 2023-10-06T05:22:39.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.961+0000 7fd026ffd700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd014067050 0x7fd014069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:39.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.961+0000 7fd026ffd700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd014067050 0x7fd014069500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd0280b6200 tx=0x7fd01c009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:39.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:39.964+0000 7fd00f7fe700 1 -- 172.21.15.31:0/1531666815 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0280c8050 con 0x7fd028159500 2023-10-06T05:22:40.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.214+0000 7fd02ddff700 1 -- 172.21.15.31:0/1531666815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd0280c8050 con 0x7fd028159500 2023-10-06T05:22:40.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.215+0000 7fd00f7fe700 1 -- 172.21.15.31:0/1531666815 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd0280c8050 con 0x7fd028159500 2023-10-06T05:22:40.218 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:22:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 -- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd014067050 msgr2=0x7fd014069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd014067050 0x7fd014069500 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd0280b6200 tx=0x7fd01c009120 comp rx=0 tx=0).stop 2023-10-06T05:22:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 -- 172.21.15.31:0/1531666815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 msgr2=0x7fd0280c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 0x7fd0280c8330 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd028067c50 tx=0x7fd018017000 comp rx=0 tx=0).stop 2023-10-06T05:22:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 -- 172.21.15.31:0/1531666815 shutdown_connections 2023-10-06T05:22:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd014067050 0x7fd014069500 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd02815d2d0 0x7fd0280c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd028159500 0x7fd0280c8330 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 --2- 172.21.15.31:0/1531666815 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0281598d0 0x7fd0280c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 -- 172.21.15.31:0/1531666815 >> 172.21.15.31:0/1531666815 conn(0x7fd02806ec90 msgr2=0x7fd0280cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 -- 172.21.15.31:0/1531666815 shutdown_connections 2023-10-06T05:22:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:40.217+0000 7fd02ddff700 1 -- 172.21.15.31:0/1531666815 wait complete. 2023-10-06T05:22:40.237 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:22:40.237 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:22:40.237 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:22:40.237 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:22:40.237 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:40.238 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:22:40.238 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:22:40.238 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:22:40.238 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:40.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:40.238 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:22:40.286 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:22:40.286 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:22:40.287 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41954sec preferred_lft 41954sec 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:40.288 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:22:40.289 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:40.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:40.290 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:22:40.358 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:22:40.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:40.361 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:22:40.410 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:22:40.411 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:22:40.411 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:22:40.411 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:22:40.411 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:22:40.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:40.411 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:22:40.411 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:22:40.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:22:40.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:22:40.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:22:40.412 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:22:40.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:40.584 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:22:40.630 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:22:40.631 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:22:40.632 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:40.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:40.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:22:40.705 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:40.742 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:40.789 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:22:40.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:40.789 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:40.875 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:22:40.876 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:22:40.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:41.015 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:41.015 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:22:41.043 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:41.043 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:22:41.043 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:22:41.043 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:22:41.044 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:22:41.048 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:22:41.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:41.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:22:41.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:22:41.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.317+0000 7f13b27ff700 1 Processor -- start 2023-10-06T05:22:41.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.317+0000 7f13b27ff700 1 -- start start 2023-10-06T05:22:41.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.318+0000 7f13b27ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13ac159500 0x7f13ac0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.318+0000 7f13b27ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f13ac15d2d0 0x7f13ac0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.318+0000 7f13b27ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13ac1598d0 0x7f13ac0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.318+0000 7f13b27ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac0c3f40 con 0x7f13ac1598d0 2023-10-06T05:22:41.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.318+0000 7f13b27ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac0cb000 con 0x7f13ac15d2d0 2023-10-06T05:22:41.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.318+0000 7f13b27ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0b2860 con 0x7f13ac159500 2023-10-06T05:22:41.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13b0d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13ac1598d0 0x7f13ac0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13b0d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13ac1598d0 0x7f13ac0cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42546/0 (socket says 172.21.15.31:42546) 2023-10-06T05:22:41.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13b0d9c700 1 -- 172.21.15.31:0/3978183478 learned_addr learned my addr 172.21.15.31:0/3978183478 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:41.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13ab7fe700 1 --2- 172.21.15.31:0/3978183478 >> v2:172.21.15.62:3300/0 conn(0x7f13ac15d2d0 0x7f13ac0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13abfff700 1 --2- 172.21.15.31:0/3978183478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13ac159500 0x7f13ac0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13b0d9c700 1 -- 172.21.15.31:0/3978183478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13ac159500 msgr2=0x7f13ac0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13b0d9c700 1 --2- 172.21.15.31:0/3978183478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13ac159500 0x7f13ac0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13b0d9c700 1 -- 172.21.15.31:0/3978183478 >> v2:172.21.15.62:3300/0 conn(0x7f13ac15d2d0 msgr2=0x7f13ac0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13b0d9c700 1 --2- 172.21.15.31:0/3978183478 >> v2:172.21.15.62:3300/0 conn(0x7f13ac15d2d0 0x7f13ac0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13b0d9c700 1 -- 172.21.15.31:0/3978183478 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac15d7d0 con 0x7f13ac1598d0 2023-10-06T05:22:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.319+0000 7f13b0d9c700 1 --2- 172.21.15.31:0/3978183478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 0x7f13ac0cd340 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f13ac069250 tx=0x7f13a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d86dc512bbce84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13aaffd700 1 -- 172.21.15.31:0/3978183478 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400e070 con 0x7f13ac1598d0 2023-10-06T05:22:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13aaffd700 1 -- 172.21.15.31:0/3978183478 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a400ba50 con 0x7f13ac1598d0 2023-10-06T05:22:41.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13aaffd700 1 -- 172.21.15.31:0/3978183478 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40092e0 con 0x7f13ac1598d0 2023-10-06T05:22:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13b27ff700 1 -- 172.21.15.31:0/3978183478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 msgr2=0x7f13ac0cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13b27ff700 1 --2- 172.21.15.31:0/3978183478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 0x7f13ac0cd340 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f13ac069250 tx=0x7f13a4009840 comp rx=0 tx=0).stop 2023-10-06T05:22:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13b27ff700 1 -- 172.21.15.31:0/3978183478 shutdown_connections 2023-10-06T05:22:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13b27ff700 1 --2- 172.21.15.31:0/3978183478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 0x7f13ac0cd340 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13b27ff700 1 --2- 172.21.15.31:0/3978183478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13ac159500 0x7f13ac0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13b27ff700 1 --2- 172.21.15.31:0/3978183478 >> v2:172.21.15.62:3300/0 conn(0x7f13ac15d2d0 0x7f13ac0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13b27ff700 1 -- 172.21.15.31:0/3978183478 >> 172.21.15.31:0/3978183478 conn(0x7f13ac06ec90 msgr2=0x7f13ac06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13b27ff700 1 -- 172.21.15.31:0/3978183478 shutdown_connections 2023-10-06T05:22:41.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.320+0000 7f13b27ff700 1 -- 172.21.15.31:0/3978183478 wait complete. 2023-10-06T05:22:41.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.322+0000 7f13b27ff700 1 Processor -- start 2023-10-06T05:22:41.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.323+0000 7f13b27ff700 1 -- start start 2023-10-06T05:22:41.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13b27ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13ac159500 0x7f13ac16a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13b27ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 0x7f13ac15a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13b27ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13ac15d2d0 0x7f13ac15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13b27ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0b58b0 con 0x7f13ac1598d0 2023-10-06T05:22:41.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13b27ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac15a1b0 con 0x7f13ac159500 2023-10-06T05:22:41.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13b27ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac16a930 con 0x7f13ac15d2d0 2023-10-06T05:22:41.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 0x7f13ac15a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 0x7f13ac15a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42552/0 (socket says 172.21.15.31:42552) 2023-10-06T05:22:41.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13ab7fe700 1 -- 172.21.15.31:0/668013542 learned_addr learned my addr 172.21.15.31:0/668013542 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13abfff700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13ac159500 0x7f13ac16a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.324+0000 7f13b0d9c700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13ac15d2d0 0x7f13ac15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f13ab7fe700 1 -- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13ac15d2d0 msgr2=0x7f13ac15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f13ab7fe700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13ac15d2d0 0x7f13ac15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f13ab7fe700 1 -- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13ac159500 msgr2=0x7f13ac16a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f13ab7fe700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13ac159500 0x7f13ac16a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f13ab7fe700 1 -- 172.21.15.31:0/668013542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a4009ae0 con 0x7f13ac1598d0 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f13ab7fe700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 0x7f13ac15a640 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f13a000d8c0 tx=0x7f13a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f1393fff700 1 -- 172.21.15.31:0/668013542 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0018070 con 0x7f13ac1598d0 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f1393fff700 1 -- 172.21.15.31:0/668013542 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a0005a00 con 0x7f13ac1598d0 2023-10-06T05:22:41.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f13b27ff700 1 -- 172.21.15.31:0/668013542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac0b5b00 con 0x7f13ac1598d0 2023-10-06T05:22:41.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f1393fff700 1 -- 172.21.15.31:0/668013542 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0005dc0 con 0x7f13ac1598d0 2023-10-06T05:22:41.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.325+0000 7f13b27ff700 1 -- 172.21.15.31:0/668013542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac159e40 con 0x7f13ac1598d0 2023-10-06T05:22:41.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.326+0000 7f13b27ff700 1 -- 172.21.15.31:0/668013542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac0b6020 con 0x7f13ac1598d0 2023-10-06T05:22:41.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.327+0000 7f1393fff700 1 -- 172.21.15.31:0/668013542 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f13a0012800 con 0x7f13ac1598d0 2023-10-06T05:22:41.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.328+0000 7f1393fff700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1398067050 0x7f1398069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:41.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.328+0000 7f1393fff700 1 -- 172.21.15.31:0/668013542 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f13a0084870 con 0x7f13ac1598d0 2023-10-06T05:22:41.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.329+0000 7f13abfff700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1398067050 0x7f1398069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:41.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.330+0000 7f13abfff700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1398067050 0x7f1398069500 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f139c006fd0 tx=0x7f139c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:41.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.333+0000 7f1393fff700 1 -- 172.21.15.31:0/668013542 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13ac0b6020 con 0x7f13ac1598d0 2023-10-06T05:22:41.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.582+0000 7f13b27ff700 1 -- 172.21.15.31:0/668013542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f13ac159ca0 con 0x7f13ac1598d0 2023-10-06T05:22:41.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.582+0000 7f1393fff700 1 -- 172.21.15.31:0/668013542 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7f13ac159ca0 con 0x7f13ac1598d0 2023-10-06T05:22:41.585 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:22:41.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.584+0000 7f13b27ff700 1 -- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1398067050 msgr2=0x7f1398069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.584+0000 7f13b27ff700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1398067050 0x7f1398069500 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f139c006fd0 tx=0x7f139c008040 comp rx=0 tx=0).stop 2023-10-06T05:22:41.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.584+0000 7f13b27ff700 1 -- 172.21.15.31:0/668013542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 msgr2=0x7f13ac15a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:41.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.584+0000 7f13b27ff700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 0x7f13ac15a640 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f13a000d8c0 tx=0x7f13a0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:41.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.584+0000 7f13b27ff700 1 -- 172.21.15.31:0/668013542 shutdown_connections 2023-10-06T05:22:41.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.584+0000 7f13b27ff700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1398067050 0x7f1398069500 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.584+0000 7f13b27ff700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13ac15d2d0 0x7f13ac15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.584+0000 7f13b27ff700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13ac159500 0x7f13ac16a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.584+0000 7f13b27ff700 1 --2- 172.21.15.31:0/668013542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ac1598d0 0x7f13ac15a640 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:41.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.584+0000 7f13b27ff700 1 -- 172.21.15.31:0/668013542 >> 172.21.15.31:0/668013542 conn(0x7f13ac06ec90 msgr2=0x7f13ac0cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:41.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.585+0000 7f13b27ff700 1 -- 172.21.15.31:0/668013542 shutdown_connections 2023-10-06T05:22:41.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:41.585+0000 7f13b27ff700 1 -- 172.21.15.31:0/668013542 wait complete. 2023-10-06T05:22:41.604 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:22:41.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:22:41.605 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:22:41.605 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:22:41.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:41.605 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:22:41.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:22:41.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:22:41.605 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:41.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:41.606 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:22:41.658 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:22:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:22:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:22:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:22:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:41.659 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:22:41.659 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41953sec preferred_lft 41953sec 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:22:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:22:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:22:41.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:41.662 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:22:41.736 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:22:41.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:41.740 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:22:41.789 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:22:41.789 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:22:41.789 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:22:41.789 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:22:41.789 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:22:41.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:41.790 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:22:41.790 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:22:41.790 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:22:41.791 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:22:41.791 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:22:41.791 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:22:41.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:41.963 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:22:42.009 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:22:42.009 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:22:42.010 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:42.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:42.010 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:22:42.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:42.082 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:42.127 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:22:42.128 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:22:42.128 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:42.213 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:22:42.214 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:22:42.214 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:42.354 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:42.355 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:22:42.381 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:22:42.382 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:22:42.382 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:22:42.382 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:22:42.382 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:22:42.385 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:22:42.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:22:42.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:22:42.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:22:42.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.665+0000 7f1ff67ff700 1 Processor -- start 2023-10-06T05:22:42.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.665+0000 7f1ff67ff700 1 -- start start 2023-10-06T05:22:42.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.666+0000 7f1ff67ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 0x7f1ff00ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.666+0000 7f1ff67ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ff006a630 0x7f1ff006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.666+0000 7f1ff67ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff006e600 con 0x7f1ff006ea90 2023-10-06T05:22:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.666+0000 7f1ff67ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff006b010 con 0x7f1ff006a630 2023-10-06T05:22:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.666+0000 7f1ff67ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff006ef00 con 0x7f1ff0156300 2023-10-06T05:22:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.666+0000 7f1feffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 0x7f1ff00ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.666+0000 7f1ff4d9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ff006a630 0x7f1ff006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.666+0000 7f1fef7fe700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1ff006ea90 0x7f1ff006d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:60278/0 (socket says 172.21.15.31:60278) 2023-10-06T05:22:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.666+0000 7f1fef7fe700 1 -- 172.21.15.31:0/2091938436 learned_addr learned my addr 172.21.15.31:0/2091938436 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.667+0000 7f1feffff700 1 -- 172.21.15.31:0/2091938436 >> v2:172.21.15.62:3300/0 conn(0x7f1ff006a630 msgr2=0x7f1ff006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.667+0000 7f1feffff700 1 --2- 172.21.15.31:0/2091938436 >> v2:172.21.15.62:3300/0 conn(0x7f1ff006a630 0x7f1ff006aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.667+0000 7f1feffff700 1 -- 172.21.15.31:0/2091938436 >> v1:172.21.15.62:6789/0 conn(0x7f1ff006ea90 legacy=0x7f1ff006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.667+0000 7f1feeffd700 1 -- 172.21.15.31:0/2091938436 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393521726 0 0) 0x7f1ff006e600 con 0x7f1ff006ea90 2023-10-06T05:22:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.667+0000 7f1feffff700 1 -- 172.21.15.31:0/2091938436 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff006d9f0 con 0x7f1ff0156300 2023-10-06T05:22:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.667+0000 7f1ff4d9c700 1 --2- 172.21.15.31:0/2091938436 >> v2:172.21.15.62:3300/0 conn(0x7f1ff006a630 0x7f1ff006aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.667+0000 7f1feffff700 1 --2- 172.21.15.31:0/2091938436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 0x7f1ff00ad670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00098d0 tx=0x7f1fe0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fcfbf6cc102e143 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1feeffd700 1 -- 172.21.15.31:0/2091938436 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0005c10 con 0x7f1ff0156300 2023-10-06T05:22:42.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1feeffd700 1 -- 172.21.15.31:0/2091938436 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe0005d70 con 0x7f1ff0156300 2023-10-06T05:22:42.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1feeffd700 1 -- 172.21.15.31:0/2091938436 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0006280 con 0x7f1ff0156300 2023-10-06T05:22:42.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2091938436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 msgr2=0x7f1ff00ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1ff67ff700 1 --2- 172.21.15.31:0/2091938436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 0x7f1ff00ad670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00098d0 tx=0x7f1fe0009120 comp rx=0 tx=0).stop 2023-10-06T05:22:42.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2091938436 shutdown_connections 2023-10-06T05:22:42.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1ff67ff700 1 --2- 172.21.15.31:0/2091938436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 0x7f1ff00ad670 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1ff67ff700 1 --2- 172.21.15.31:0/2091938436 >> v2:172.21.15.62:3300/0 conn(0x7f1ff006a630 0x7f1ff006aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2091938436 >> 172.21.15.31:0/2091938436 conn(0x7f1ff0068030 msgr2=0x7f1ff00cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:42.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2091938436 shutdown_connections 2023-10-06T05:22:42.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.668+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2091938436 wait complete. 2023-10-06T05:22:42.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.670+0000 7f1ff67ff700 1 Processor -- start 2023-10-06T05:22:42.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.670+0000 7f1ff67ff700 1 -- start start 2023-10-06T05:22:42.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1ff67ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff006a630 0x7f1ff00b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1ff67ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff006ea90 0x7f1ff0152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1ff67ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 0x7f1ff0151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1ff67ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff00bfc00 con 0x7f1ff006a630 2023-10-06T05:22:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1ff67ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff0152f90 con 0x7f1ff006ea90 2023-10-06T05:22:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1ff67ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff0153270 con 0x7f1ff0156300 2023-10-06T05:22:42.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1feffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff006a630 0x7f1ff00b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1feffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff006a630 0x7f1ff00b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42556/0 (socket says 172.21.15.31:42556) 2023-10-06T05:22:42.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1feffff700 1 -- 172.21.15.31:0/2990468500 learned_addr learned my addr 172.21.15.31:0/2990468500 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:42.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1fef7fe700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff006ea90 0x7f1ff0152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.671+0000 7f1ff4d9c700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 0x7f1ff0151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.672+0000 7f1fef7fe700 1 -- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 msgr2=0x7f1ff0151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.672+0000 7f1fef7fe700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 0x7f1ff0151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.672+0000 7f1fef7fe700 1 -- 172.21.15.31:0/2990468500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff006a630 msgr2=0x7f1ff00b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.672+0000 7f1fef7fe700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff006a630 0x7f1ff00b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.672+0000 7f1fef7fe700 1 -- 172.21.15.31:0/2990468500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe8009a20 con 0x7f1ff006ea90 2023-10-06T05:22:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.672+0000 7f1fef7fe700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff006ea90 0x7f1ff0152880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40011e0 tx=0x7f1fe4007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.672+0000 7f1fd7fff700 1 -- 172.21.15.31:0/2990468500 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4005b30 con 0x7f1ff006ea90 2023-10-06T05:22:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.672+0000 7f1fd7fff700 1 -- 172.21.15.31:0/2990468500 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe4005c90 con 0x7f1ff006ea90 2023-10-06T05:22:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.672+0000 7f1fd7fff700 1 -- 172.21.15.31:0/2990468500 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4007300 con 0x7f1ff006ea90 2023-10-06T05:22:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.672+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2990468500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe00093c0 con 0x7f1ff006ea90 2023-10-06T05:22:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.673+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2990468500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff0151a70 con 0x7f1ff006ea90 2023-10-06T05:22:42.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.673+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2990468500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff00bfa60 con 0x7f1ff006ea90 2023-10-06T05:22:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.675+0000 7f1fd7fff700 1 -- 172.21.15.31:0/2990468500 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f1fe401b3b0 con 0x7f1ff006ea90 2023-10-06T05:22:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.675+0000 7f1fd7fff700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1fdc067050 0x7f1fdc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.676+0000 7f1fd7fff700 1 -- 172.21.15.31:0/2990468500 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1fe4083c70 con 0x7f1ff006ea90 2023-10-06T05:22:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.676+0000 7f1feffff700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1fdc067050 0x7f1fdc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.676+0000 7f1feffff700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1fdc067050 0x7f1fdc069500 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000e330 tx=0x7f1fe0018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:42.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.682+0000 7f1fd7fff700 1 -- 172.21.15.31:0/2990468500 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ff00bfa60 con 0x7f1ff006ea90 2023-10-06T05:22:42.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:42.926+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2990468500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1ff00bfa60 con 0x7f1ff006ea90 2023-10-06T05:22:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.179+0000 7f1fd7fff700 1 -- 172.21.15.31:0/2990468500 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1ff00bfa60 con 0x7f1ff006ea90 2023-10-06T05:22:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1fdc067050 msgr2=0x7f1fdc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1fdc067050 0x7f1fdc069500 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000e330 tx=0x7f1fe0018000 comp rx=0 tx=0).stop 2023-10-06T05:22:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff006ea90 msgr2=0x7f1ff0152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff006ea90 0x7f1ff0152880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40011e0 tx=0x7f1fe4007d30 comp rx=0 tx=0).stop 2023-10-06T05:22:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2990468500 shutdown_connections 2023-10-06T05:22:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1fdc067050 0x7f1fdc069500 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1ff0156300 0x7f1ff0151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ff006a630 0x7f1ff00b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 --2- 172.21.15.31:0/2990468500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ff006ea90 0x7f1ff0152880 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2990468500 >> 172.21.15.31:0/2990468500 conn(0x7f1ff0068030 msgr2=0x7f1ff00c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2990468500 shutdown_connections 2023-10-06T05:22:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.182+0000 7f1ff67ff700 1 -- 172.21.15.31:0/2990468500 wait complete. 2023-10-06T05:22:43.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:22:43.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.398+0000 7f3bd7dff700 1 Processor -- start 2023-10-06T05:22:43.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.399+0000 7f3bd7dff700 1 -- start start 2023-10-06T05:22:43.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.400+0000 7f3bd7dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3bd00cae30 0x7f3bd00c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.400+0000 7f3bd7dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3bd00b01b0 0x7f3bd00b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.400+0000 7f3bd7dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd0069170 con 0x7f3bd00cb200 2023-10-06T05:22:43.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.400+0000 7f3bd7dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00685f0 con 0x7f3bd00b01b0 2023-10-06T05:22:43.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.400+0000 7f3bd7dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00b0b70 con 0x7f3bd00cae30 2023-10-06T05:22:43.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.400+0000 7f3bd539a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3bd00cb200 0x7f3bd00ca090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44514/0 (socket says 172.21.15.31:44514) 2023-10-06T05:22:43.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.400+0000 7f3bd539a700 1 -- 172.21.15.31:0/3473432441 learned_addr learned my addr 172.21.15.31:0/3473432441 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:43.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.400+0000 7f3bd639c700 1 --2- 172.21.15.31:0/3473432441 >> v2:172.21.15.31:3300/0 conn(0x7f3bd00b01b0 0x7f3bd00b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.400+0000 7f3bd5b9b700 1 --2- 172.21.15.31:0/3473432441 >> v2:172.21.15.62:3300/0 conn(0x7f3bd00cae30 0x7f3bd00c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.401+0000 7f3bd4b99700 1 -- 172.21.15.31:0/3473432441 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940476057 0 0) 0x7f3bd0069170 con 0x7f3bd00cb200 2023-10-06T05:22:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.401+0000 7f3bd4b99700 1 -- 172.21.15.31:0/3473432441 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bbc003570 con 0x7f3bd00cb200 2023-10-06T05:22:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.401+0000 7f3bd639c700 1 -- 172.21.15.31:0/3473432441 >> v2:172.21.15.62:3300/0 conn(0x7f3bd00cae30 msgr2=0x7f3bd00c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.401+0000 7f3bd639c700 1 --2- 172.21.15.31:0/3473432441 >> v2:172.21.15.62:3300/0 conn(0x7f3bd00cae30 0x7f3bd00c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.401+0000 7f3bd639c700 1 -- 172.21.15.31:0/3473432441 >> v1:172.21.15.31:6789/0 conn(0x7f3bd00cb200 legacy=0x7f3bd00ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.401+0000 7f3bd639c700 1 -- 172.21.15.31:0/3473432441 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd0068840 con 0x7f3bd00b01b0 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.401+0000 7f3bd639c700 1 --2- 172.21.15.31:0/3473432441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00b01b0 0x7f3bd00b0630 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006e640 tx=0x7f3bc8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe75d6cd5aff170d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.401+0000 7f3bd4b99700 1 -- 172.21.15.31:0/3473432441 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800e070 con 0x7f3bd00b01b0 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.401+0000 7f3bd4b99700 1 -- 172.21.15.31:0/3473432441 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc800bab0 con 0x7f3bd00b01b0 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.401+0000 7f3bd4b99700 1 -- 172.21.15.31:0/3473432441 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800be70 con 0x7f3bd00b01b0 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.402+0000 7f3bd7dff700 1 -- 172.21.15.31:0/3473432441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00b01b0 msgr2=0x7f3bd00b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.402+0000 7f3bd7dff700 1 --2- 172.21.15.31:0/3473432441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00b01b0 0x7f3bd00b0630 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006e640 tx=0x7f3bc8009780 comp rx=0 tx=0).stop 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.402+0000 7f3bd7dff700 1 -- 172.21.15.31:0/3473432441 shutdown_connections 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.402+0000 7f3bd7dff700 1 --2- 172.21.15.31:0/3473432441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00b01b0 0x7f3bd00b0630 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.402+0000 7f3bd7dff700 1 --2- 172.21.15.31:0/3473432441 >> v2:172.21.15.62:3300/0 conn(0x7f3bd00cae30 0x7f3bd00c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.402+0000 7f3bd7dff700 1 -- 172.21.15.31:0/3473432441 >> 172.21.15.31:0/3473432441 conn(0x7f3bd0087fc0 msgr2=0x7f3bd00639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.402+0000 7f3bd7dff700 1 -- 172.21.15.31:0/3473432441 shutdown_connections 2023-10-06T05:22:43.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.402+0000 7f3bd7dff700 1 -- 172.21.15.31:0/3473432441 wait complete. 2023-10-06T05:22:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.403+0000 7f3bd7dff700 1 Processor -- start 2023-10-06T05:22:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.404+0000 7f3bd7dff700 1 -- start start 2023-10-06T05:22:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd7dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd00b01b0 0x7f3bd0146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd7dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd00cae30 0x7f3bd0158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd7dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00cb200 0x7f3bd00c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd7dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00c5c20 con 0x7f3bd00cb200 2023-10-06T05:22:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd7dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00c5e30 con 0x7f3bd00cae30 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd7dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00c6110 con 0x7f3bd00b01b0 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00cb200 0x7f3bd00c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd639c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00cb200 0x7f3bd00c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42570/0 (socket says 172.21.15.31:42570) 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd639c700 1 -- 172.21.15.31:0/1320274213 learned_addr learned my addr 172.21.15.31:0/1320274213 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd539a700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd00cae30 0x7f3bd0158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd5b9b700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd00b01b0 0x7f3bd0146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd639c700 1 -- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd00b01b0 msgr2=0x7f3bd0146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd639c700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd00b01b0 0x7f3bd0146fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd639c700 1 -- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd00cae30 msgr2=0x7f3bd0158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd639c700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd00cae30 0x7f3bd0158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.405+0000 7f3bd639c700 1 -- 172.21.15.31:0/1320274213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8009a20 con 0x7f3bd00cb200 2023-10-06T05:22:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.406+0000 7f3bd639c700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00cb200 0x7f3bd00c3cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006e880 tx=0x7f3bc8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.406+0000 7f3bc5ffb700 1 -- 172.21.15.31:0/1320274213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8012db0 con 0x7f3bd00cb200 2023-10-06T05:22:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.406+0000 7f3bc5ffb700 1 -- 172.21.15.31:0/1320274213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8027470 con 0x7f3bd00cb200 2023-10-06T05:22:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.406+0000 7f3bc5ffb700 1 -- 172.21.15.31:0/1320274213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8027830 con 0x7f3bd00cb200 2023-10-06T05:22:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.406+0000 7f3bd7dff700 1 -- 172.21.15.31:0/1320274213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd00c63f0 con 0x7f3bd00cb200 2023-10-06T05:22:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.406+0000 7f3bd7dff700 1 -- 172.21.15.31:0/1320274213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd0155fb0 con 0x7f3bd00cb200 2023-10-06T05:22:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.407+0000 7f3bd7dff700 1 -- 172.21.15.31:0/1320274213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd004c380 con 0x7f3bd00cb200 2023-10-06T05:22:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.408+0000 7f3bc5ffb700 1 -- 172.21.15.31:0/1320274213 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f3bc8024070 con 0x7f3bd00cb200 2023-10-06T05:22:43.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.409+0000 7f3bc5ffb700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bb4067050 0x7f3bb4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.409+0000 7f3bc5ffb700 1 -- 172.21.15.31:0/1320274213 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3bc8086e60 con 0x7f3bd00cb200 2023-10-06T05:22:43.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.409+0000 7f3bd5b9b700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bb4067050 0x7f3bb4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.410+0000 7f3bd5b9b700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bb4067050 0x7f3bb4069500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009d00 tx=0x7f3bcc009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.413+0000 7f3bc5ffb700 1 -- 172.21.15.31:0/1320274213 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bc8053dc0 con 0x7f3bd00cb200 2023-10-06T05:22:43.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.645+0000 7f3bd7dff700 1 -- 172.21.15.31:0/1320274213 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bd00c5fa0 con 0x7f3bb4067050 2023-10-06T05:22:43.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.648+0000 7f3bc5ffb700 1 -- 172.21.15.31:0/1320274213 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3bd00c5fa0 con 0x7f3bb4067050 2023-10-06T05:22:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:22:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:22:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:22:43.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 -- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bb4067050 msgr2=0x7f3bb4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bb4067050 0x7f3bb4069500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009d00 tx=0x7f3bcc009120 comp rx=0 tx=0).stop 2023-10-06T05:22:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 -- 172.21.15.31:0/1320274213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00cb200 msgr2=0x7f3bd00c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00cb200 0x7f3bd00c3cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006e880 tx=0x7f3bc8019000 comp rx=0 tx=0).stop 2023-10-06T05:22:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 -- 172.21.15.31:0/1320274213 shutdown_connections 2023-10-06T05:22:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3bb4067050 0x7f3bb4069500 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bd00cb200 0x7f3bd00c3cd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd00b01b0 0x7f3bd0146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 --2- 172.21.15.31:0/1320274213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd00cae30 0x7f3bd0158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 -- 172.21.15.31:0/1320274213 >> 172.21.15.31:0/1320274213 conn(0x7f3bd0087fc0 msgr2=0x7f3bd0155490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 -- 172.21.15.31:0/1320274213 shutdown_connections 2023-10-06T05:22:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.650+0000 7f3bd7dff700 1 -- 172.21.15.31:0/1320274213 wait complete. 2023-10-06T05:22:43.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:22:43.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.867+0000 7f2e391ff700 1 Processor -- start 2023-10-06T05:22:43.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.867+0000 7f2e391ff700 1 -- start start 2023-10-06T05:22:43.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.868+0000 7f2e391ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e340b0b90 0x7f2e340af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.868+0000 7f2e391ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e340ad690 con 0x7f2e34153d40 2023-10-06T05:22:43.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.868+0000 7f2e391ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e340acd60 con 0x7f2e340af6c0 2023-10-06T05:22:43.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.868+0000 7f2e391ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e340ad040 con 0x7f2e340b0b90 2023-10-06T05:22:43.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.869+0000 7f2e32d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2e34153d40 0x7f2e340b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44530/0 (socket says 172.21.15.31:44530) 2023-10-06T05:22:43.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.869+0000 7f2e32d9d700 1 -- 172.21.15.31:0/1211767681 learned_addr learned my addr 172.21.15.31:0/1211767681 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:43.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.869+0000 7f2e3259c700 1 --2- 172.21.15.31:0/1211767681 >> v2:172.21.15.31:3300/0 conn(0x7f2e340b0b90 0x7f2e340af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.869+0000 7f2e31d9b700 1 -- 172.21.15.31:0/1211767681 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053547292 0 0) 0x7f2e340ad690 con 0x7f2e34153d40 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.869+0000 7f2e31d9b700 1 -- 172.21.15.31:0/1211767681 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e1c003570 con 0x7f2e34153d40 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.869+0000 7f2e3259c700 1 -- 172.21.15.31:0/1211767681 >> v1:172.21.15.62:6789/0 conn(0x7f2e340af6c0 legacy=0x7f2e340adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.869+0000 7f2e3259c700 1 -- 172.21.15.31:0/1211767681 >> v1:172.21.15.31:6789/0 conn(0x7f2e34153d40 legacy=0x7f2e340b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.869+0000 7f2e3259c700 1 -- 172.21.15.31:0/1211767681 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340ad320 con 0x7f2e340b0b90 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.869+0000 7f2e3259c700 1 --2- 172.21.15.31:0/1211767681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340b0b90 0x7f2e340af180 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2e280098e0 tx=0x7f2e280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c7186af50e8b45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.870+0000 7f2e31d9b700 1 -- 172.21.15.31:0/1211767681 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2800b640 con 0x7f2e340b0b90 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.870+0000 7f2e31d9b700 1 -- 172.21.15.31:0/1211767681 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2800b7a0 con 0x7f2e340b0b90 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.870+0000 7f2e31d9b700 1 -- 172.21.15.31:0/1211767681 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2800bb40 con 0x7f2e340b0b90 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.870+0000 7f2e391ff700 1 -- 172.21.15.31:0/1211767681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340b0b90 msgr2=0x7f2e340af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.870+0000 7f2e391ff700 1 --2- 172.21.15.31:0/1211767681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340b0b90 0x7f2e340af180 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2e280098e0 tx=0x7f2e280091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.870+0000 7f2e391ff700 1 -- 172.21.15.31:0/1211767681 shutdown_connections 2023-10-06T05:22:43.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.870+0000 7f2e391ff700 1 --2- 172.21.15.31:0/1211767681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340b0b90 0x7f2e340af180 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.870+0000 7f2e391ff700 1 -- 172.21.15.31:0/1211767681 >> 172.21.15.31:0/1211767681 conn(0x7f2e34067560 msgr2=0x7f2e34067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:43.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.870+0000 7f2e391ff700 1 -- 172.21.15.31:0/1211767681 shutdown_connections 2023-10-06T05:22:43.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.870+0000 7f2e391ff700 1 -- 172.21.15.31:0/1211767681 wait complete. 2023-10-06T05:22:43.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.872+0000 7f2e391ff700 1 Processor -- start 2023-10-06T05:22:43.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.872+0000 7f2e391ff700 1 -- start start 2023-10-06T05:22:43.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e391ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340af6c0 0x7f2e34144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e391ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e340b0b90 0x7f2e340cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e391ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e34153d40 0x7f2e340c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e391ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e340c90d0 con 0x7f2e340af6c0 2023-10-06T05:22:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e391ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e340c93b0 con 0x7f2e34153d40 2023-10-06T05:22:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e391ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e340c1a60 con 0x7f2e340b0b90 2023-10-06T05:22:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e32d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340af6c0 0x7f2e34144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e32d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340af6c0 0x7f2e34144be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42582/0 (socket says 172.21.15.31:42582) 2023-10-06T05:22:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e32d9d700 1 -- 172.21.15.31:0/1537329211 learned_addr learned my addr 172.21.15.31:0/1537329211 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e3259c700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e340b0b90 0x7f2e340cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e3359e700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e34153d40 0x7f2e340c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e32d9d700 1 -- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e340b0b90 msgr2=0x7f2e340cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e32d9d700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e340b0b90 0x7f2e340cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e32d9d700 1 -- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e34153d40 msgr2=0x7f2e340c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e32d9d700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e34153d40 0x7f2e340c29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e32d9d700 1 -- 172.21.15.31:0/1537329211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e28009420 con 0x7f2e340af6c0 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.873+0000 7f2e3359e700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e34153d40 0x7f2e340c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.874+0000 7f2e32d9d700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340af6c0 0x7f2e34144be0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400c5c0 tx=0x7f2e24007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.874+0000 7f2e1affd700 1 -- 172.21.15.31:0/1537329211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24005b30 con 0x7f2e340af6c0 2023-10-06T05:22:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.874+0000 7f2e1affd700 1 -- 172.21.15.31:0/1537329211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e24005cb0 con 0x7f2e340af6c0 2023-10-06T05:22:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.874+0000 7f2e1affd700 1 -- 172.21.15.31:0/1537329211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24007320 con 0x7f2e340af6c0 2023-10-06T05:22:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.874+0000 7f2e391ff700 1 -- 172.21.15.31:0/1537329211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e340c1d40 con 0x7f2e340af6c0 2023-10-06T05:22:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.874+0000 7f2e391ff700 1 -- 172.21.15.31:0/1537329211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e34145120 con 0x7f2e340af6c0 2023-10-06T05:22:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.875+0000 7f2e391ff700 1 -- 172.21.15.31:0/1537329211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e340c2260 con 0x7f2e340af6c0 2023-10-06T05:22:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.876+0000 7f2e1affd700 1 -- 172.21.15.31:0/1537329211 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f2e2401b3b0 con 0x7f2e340af6c0 2023-10-06T05:22:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.877+0000 7f2e1affd700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2e20067050 0x7f2e20069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.877+0000 7f2e1affd700 1 -- 172.21.15.31:0/1537329211 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2e24083d90 con 0x7f2e340af6c0 2023-10-06T05:22:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.878+0000 7f2e3259c700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2e20067050 0x7f2e20069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.878+0000 7f2e3259c700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2e20067050 0x7f2e20069500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e340b0fb0 tx=0x7f2e28017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:43.881+0000 7f2e1affd700 1 -- 172.21.15.31:0/1537329211 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e24050cf0 con 0x7f2e340af6c0 2023-10-06T05:22:44.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.085+0000 7f2e391ff700 1 -- 172.21.15.31:0/1537329211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2e3404c380 con 0x7f2e340af6c0 2023-10-06T05:22:44.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.085+0000 7f2e1affd700 1 -- 172.21.15.31:0/1537329211 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2e24017020 con 0x7f2e340af6c0 2023-10-06T05:22:44.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 -- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2e20067050 msgr2=0x7f2e20069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2e20067050 0x7f2e20069500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e340b0fb0 tx=0x7f2e28017000 comp rx=0 tx=0).stop 2023-10-06T05:22:44.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 -- 172.21.15.31:0/1537329211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340af6c0 msgr2=0x7f2e34144be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340af6c0 0x7f2e34144be0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400c5c0 tx=0x7f2e24007e00 comp rx=0 tx=0).stop 2023-10-06T05:22:44.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 -- 172.21.15.31:0/1537329211 shutdown_connections 2023-10-06T05:22:44.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2e20067050 0x7f2e20069500 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e34153d40 0x7f2e340c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e340af6c0 0x7f2e34144be0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 --2- 172.21.15.31:0/1537329211 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e340b0b90 0x7f2e340cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 -- 172.21.15.31:0/1537329211 >> 172.21.15.31:0/1537329211 conn(0x7f2e34067560 msgr2=0x7f2e3406d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 -- 172.21.15.31:0/1537329211 shutdown_connections 2023-10-06T05:22:44.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.088+0000 7f2e391ff700 1 -- 172.21.15.31:0/1537329211 wait complete. 2023-10-06T05:22:44.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:44.108 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1) 2023-10-06T05:22:44.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:44.190 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/fake) 2023-10-06T05:22:44.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:44.273 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T05:22:44.320 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T05:22:44.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:44.321 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1) 2023-10-06T05:22:44.435 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:22:44.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvolume_0000000000063418/.meta 2023-10-06T05:22:44.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000063418 2023-10-06T05:22:44.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.745+0000 7fec9a7ff700 1 Processor -- start 2023-10-06T05:22:44.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.746+0000 7fec9a7ff700 1 -- start start 2023-10-06T05:22:44.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.747+0000 7fec9a7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec94153d40 0x7fec940b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.747+0000 7fec9a7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940af890 0x7fec940adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.747+0000 7fec9a7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec940b0e40 con 0x7fec940b09c0 2023-10-06T05:22:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.747+0000 7fec9a7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec940afcd0 con 0x7fec94153d40 2023-10-06T05:22:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.747+0000 7fec9a7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec940ad690 con 0x7fec940af890 2023-10-06T05:22:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.747+0000 7fec93fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec94153d40 0x7fec940b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.747+0000 7fec93fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec94153d40 0x7fec940b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42592/0 (socket says 172.21.15.31:42592) 2023-10-06T05:22:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.747+0000 7fec93fff700 1 -- 172.21.15.31:0/2501065812 learned_addr learned my addr 172.21.15.31:0/2501065812 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:44.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.748+0000 7fec98d9c700 1 --2- 172.21.15.31:0/2501065812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940af890 0x7fec940adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.748+0000 7fec93fff700 1 -- 172.21.15.31:0/2501065812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940af890 msgr2=0x7fec940adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.748+0000 7fec93fff700 1 --2- 172.21.15.31:0/2501065812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940af890 0x7fec940adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.748+0000 7fec93fff700 1 -- 172.21.15.31:0/2501065812 >> v1:172.21.15.62:6789/0 conn(0x7fec940b09c0 legacy=0x7fec940af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.748+0000 7fec93fff700 1 -- 172.21.15.31:0/2501065812 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec940acd60 con 0x7fec94153d40 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.748+0000 7fec98d9c700 1 --2- 172.21.15.31:0/2501065812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940af890 0x7fec940adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.748+0000 7fec93fff700 1 --2- 172.21.15.31:0/2501065812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec94153d40 0x7fec940b0480 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fec8400bc50 tx=0x7fec8400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c4a5b839a6cb1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.748+0000 7fec92ffd700 1 -- 172.21.15.31:0/2501065812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400be60 con 0x7fec94153d40 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.748+0000 7fec92ffd700 1 -- 172.21.15.31:0/2501065812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8400de00 con 0x7fec94153d40 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.748+0000 7fec92ffd700 1 -- 172.21.15.31:0/2501065812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec840196b0 con 0x7fec94153d40 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.749+0000 7fec9a7ff700 1 -- 172.21.15.31:0/2501065812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec94153d40 msgr2=0x7fec940b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.749+0000 7fec9a7ff700 1 --2- 172.21.15.31:0/2501065812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec94153d40 0x7fec940b0480 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fec8400bc50 tx=0x7fec8400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.749+0000 7fec9a7ff700 1 -- 172.21.15.31:0/2501065812 shutdown_connections 2023-10-06T05:22:44.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.749+0000 7fec9a7ff700 1 --2- 172.21.15.31:0/2501065812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940af890 0x7fec940adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.749+0000 7fec9a7ff700 1 --2- 172.21.15.31:0/2501065812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec94153d40 0x7fec940b0480 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.749+0000 7fec9a7ff700 1 -- 172.21.15.31:0/2501065812 >> 172.21.15.31:0/2501065812 conn(0x7fec94067560 msgr2=0x7fec94067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:44.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.749+0000 7fec9a7ff700 1 -- 172.21.15.31:0/2501065812 shutdown_connections 2023-10-06T05:22:44.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.749+0000 7fec9a7ff700 1 -- 172.21.15.31:0/2501065812 wait complete. 2023-10-06T05:22:44.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.751+0000 7fec9a7ff700 1 Processor -- start 2023-10-06T05:22:44.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.751+0000 7fec9a7ff700 1 -- start start 2023-10-06T05:22:44.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec9a7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec940af890 0x7fec9414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec9a7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940b09c0 0x7fec94154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec9a7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec94153d40 0x7fec94142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec9a7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9414c500 con 0x7fec940af890 2023-10-06T05:22:44.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec9a7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9414c7e0 con 0x7fec94153d40 2023-10-06T05:22:44.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec9a7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9414cac0 con 0x7fec940b09c0 2023-10-06T05:22:44.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec93fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec940af890 0x7fec9414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec93fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec940af890 0x7fec9414e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42596/0 (socket says 172.21.15.31:42596) 2023-10-06T05:22:44.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec93fff700 1 -- 172.21.15.31:0/340640034 learned_addr learned my addr 172.21.15.31:0/340640034 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:44.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec937fe700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940b09c0 0x7fec94154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec98d9c700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec94153d40 0x7fec94142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec93fff700 1 -- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940b09c0 msgr2=0x7fec94154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec93fff700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940b09c0 0x7fec94154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec93fff700 1 -- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec94153d40 msgr2=0x7fec94142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec93fff700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec94153d40 0x7fec94142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec93fff700 1 -- 172.21.15.31:0/340640034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8400b790 con 0x7fec940af890 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.752+0000 7fec937fe700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940b09c0 0x7fec94154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.753+0000 7fec93fff700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec940af890 0x7fec9414e150 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fec9406f430 tx=0x7fec8401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.753+0000 7fec7bfff700 1 -- 172.21.15.31:0/340640034 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec840108d0 con 0x7fec940af890 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.753+0000 7fec7bfff700 1 -- 172.21.15.31:0/340640034 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec84010a30 con 0x7fec940af890 2023-10-06T05:22:44.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.753+0000 7fec7bfff700 1 -- 172.21.15.31:0/340640034 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84010df0 con 0x7fec940af890 2023-10-06T05:22:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.753+0000 7fec9a7ff700 1 -- 172.21.15.31:0/340640034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9414cda0 con 0x7fec940af890 2023-10-06T05:22:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.753+0000 7fec9a7ff700 1 -- 172.21.15.31:0/340640034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9414e690 con 0x7fec940af890 2023-10-06T05:22:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.754+0000 7fec9a7ff700 1 -- 172.21.15.31:0/340640034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9404c380 con 0x7fec940af890 2023-10-06T05:22:44.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.755+0000 7fec7bfff700 1 -- 172.21.15.31:0/340640034 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fec8402c430 con 0x7fec940af890 2023-10-06T05:22:44.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.756+0000 7fec7bfff700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec80067050 0x7fec80069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:44.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.756+0000 7fec7bfff700 1 -- 172.21.15.31:0/340640034 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fec84014070 con 0x7fec940af890 2023-10-06T05:22:44.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.757+0000 7fec937fe700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec80067050 0x7fec80069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:44.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.757+0000 7fec937fe700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec80067050 0x7fec80069500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fec88007fd0 tx=0x7fec88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:44.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:44.761+0000 7fec7bfff700 1 -- 172.21.15.31:0/340640034 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec84053710 con 0x7fec940af890 2023-10-06T05:22:45.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.001+0000 7fec9a7ff700 1 -- 172.21.15.31:0/340640034 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "target": ["mon-mgr", ""]}) v1 -- 0x7fec940ad040 con 0x7fec80067050 2023-10-06T05:22:45.010 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:45.008+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:45.010 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:45.008+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:45.011 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:45.008+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:45.011 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:45.008+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:45.011 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:45.008+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:45.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.060+0000 7fec7bfff700 1 -- 172.21.15.31:0/340640034 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7fec940ad040 con 0x7fec80067050 2023-10-06T05:22:45.062 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1 2023-10-06T05:22:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 -- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec80067050 msgr2=0x7fec80069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec80067050 0x7fec80069500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fec88007fd0 tx=0x7fec88009040 comp rx=0 tx=0).stop 2023-10-06T05:22:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 -- 172.21.15.31:0/340640034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec940af890 msgr2=0x7fec9414e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec940af890 0x7fec9414e150 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fec9406f430 tx=0x7fec8401b000 comp rx=0 tx=0).stop 2023-10-06T05:22:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 -- 172.21.15.31:0/340640034 shutdown_connections 2023-10-06T05:22:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fec80067050 0x7fec80069500 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec94153d40 0x7fec94142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec940af890 0x7fec9414e150 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 --2- 172.21.15.31:0/340640034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fec940b09c0 0x7fec94154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 -- 172.21.15.31:0/340640034 >> 172.21.15.31:0/340640034 conn(0x7fec94067560 msgr2=0x7fec940adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:45.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 -- 172.21.15.31:0/340640034 shutdown_connections 2023-10-06T05:22:45.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.063+0000 7fec9a7ff700 1 -- 172.21.15.31:0/340640034 wait complete. 2023-10-06T05:22:45.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2023-10-06T05:22:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.278+0000 7ff5a5bff700 1 Processor -- start 2023-10-06T05:22:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.279+0000 7ff5a5bff700 1 -- start start 2023-10-06T05:22:45.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.280+0000 7ff5a5bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff5a00c7190 0x7ff5a00c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.280+0000 7ff5a5bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff5a00aef00 0x7ff5a00af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.280+0000 7ff5a5bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a00afcf0 0x7ff5a00686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.280+0000 7ff5a5bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a0068250 con 0x7ff5a00aef00 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.280+0000 7ff5a5bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a0067f20 con 0x7ff5a00c7190 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.280+0000 7ff5a5bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a006aed0 con 0x7ff5a00afcf0 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff5a00aef00 0x7ff5a00af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff5a00aef00 0x7ff5a00af7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42598/0 (socket says 172.21.15.31:42598) 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59effd700 1 -- 172.21.15.31:0/2349356855 learned_addr learned my addr 172.21.15.31:0/2349356855 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59f7fe700 1 --2- 172.21.15.31:0/2349356855 >> v2:172.21.15.62:3300/0 conn(0x7ff5a00c7190 0x7ff5a00c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59ffff700 1 --2- 172.21.15.31:0/2349356855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a00afcf0 0x7ff5a00686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59effd700 1 -- 172.21.15.31:0/2349356855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a00afcf0 msgr2=0x7ff5a00686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59effd700 1 --2- 172.21.15.31:0/2349356855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a00afcf0 0x7ff5a00686e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59effd700 1 -- 172.21.15.31:0/2349356855 >> v2:172.21.15.62:3300/0 conn(0x7ff5a00c7190 msgr2=0x7ff5a00c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59effd700 1 --2- 172.21.15.31:0/2349356855 >> v2:172.21.15.62:3300/0 conn(0x7ff5a00c7190 0x7ff5a00c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59effd700 1 -- 172.21.15.31:0/2349356855 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a006b1b0 con 0x7ff5a00aef00 2023-10-06T05:22:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59f7fe700 1 --2- 172.21.15.31:0/2349356855 >> v2:172.21.15.62:3300/0 conn(0x7ff5a00c7190 0x7ff5a00c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.281+0000 7ff59effd700 1 --2- 172.21.15.31:0/2349356855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00aef00 0x7ff5a00af7b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff5a006f4e0 tx=0x7ff594009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbdaaa09d257dbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff59e7fc700 1 -- 172.21.15.31:0/2349356855 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59400b860 con 0x7ff5a00aef00 2023-10-06T05:22:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff59e7fc700 1 -- 172.21.15.31:0/2349356855 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59400b9c0 con 0x7ff5a00aef00 2023-10-06T05:22:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff59e7fc700 1 -- 172.21.15.31:0/2349356855 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59400bd60 con 0x7ff5a00aef00 2023-10-06T05:22:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff5a5bff700 1 -- 172.21.15.31:0/2349356855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00aef00 msgr2=0x7ff5a00af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff5a5bff700 1 --2- 172.21.15.31:0/2349356855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00aef00 0x7ff5a00af7b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff5a006f4e0 tx=0x7ff594009250 comp rx=0 tx=0).stop 2023-10-06T05:22:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff5a5bff700 1 -- 172.21.15.31:0/2349356855 shutdown_connections 2023-10-06T05:22:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff5a5bff700 1 --2- 172.21.15.31:0/2349356855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a00afcf0 0x7ff5a00686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff5a5bff700 1 --2- 172.21.15.31:0/2349356855 >> v2:172.21.15.62:3300/0 conn(0x7ff5a00c7190 0x7ff5a00c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff5a5bff700 1 --2- 172.21.15.31:0/2349356855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00aef00 0x7ff5a00af7b0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff5a5bff700 1 -- 172.21.15.31:0/2349356855 >> 172.21.15.31:0/2349356855 conn(0x7ff5a00cd3d0 msgr2=0x7ff5a00c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:45.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff5a5bff700 1 -- 172.21.15.31:0/2349356855 shutdown_connections 2023-10-06T05:22:45.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.282+0000 7ff5a5bff700 1 -- 172.21.15.31:0/2349356855 wait complete. 2023-10-06T05:22:45.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.284+0000 7ff5a5bff700 1 Processor -- start 2023-10-06T05:22:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.285+0000 7ff5a5bff700 1 -- start start 2023-10-06T05:22:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.286+0000 7ff5a5bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a00aef00 0x7ff5a0157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.286+0000 7ff5a5bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a00afcf0 0x7ff5a00c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.286+0000 7ff5a5bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00c7190 0x7ff5a00c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.286+0000 7ff5a5bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a01581d0 con 0x7ff5a00c7190 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.286+0000 7ff5a5bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a015cfc0 con 0x7ff5a00aef00 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.286+0000 7ff5a5bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a00c25c0 con 0x7ff5a00afcf0 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.286+0000 7ff59ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00c7190 0x7ff5a00c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.286+0000 7ff59ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00c7190 0x7ff5a00c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42604/0 (socket says 172.21.15.31:42604) 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.286+0000 7ff59ffff700 1 -- 172.21.15.31:0/3854540852 learned_addr learned my addr 172.21.15.31:0/3854540852 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff59effd700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a00afcf0 0x7ff5a00c2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff59f7fe700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a00aef00 0x7ff5a0157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff59ffff700 1 -- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a00afcf0 msgr2=0x7ff5a00c2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff59ffff700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a00afcf0 0x7ff5a00c2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff59ffff700 1 -- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a00aef00 msgr2=0x7ff5a0157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff59ffff700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a00aef00 0x7ff5a0157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff59ffff700 1 -- 172.21.15.31:0/3854540852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940094f0 con 0x7ff5a00c7190 2023-10-06T05:22:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff59f7fe700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a00aef00 0x7ff5a0157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff59ffff700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00c7190 0x7ff5a00c5b00 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff598009fa0 tx=0x7ff598009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff5877fe700 1 -- 172.21.15.31:0/3854540852 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59800ed20 con 0x7ff5a00c7190 2023-10-06T05:22:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff5877fe700 1 -- 172.21.15.31:0/3854540852 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59800ee80 con 0x7ff5a00c7190 2023-10-06T05:22:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff5a5bff700 1 -- 172.21.15.31:0/3854540852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a00c28a0 con 0x7ff5a00c7190 2023-10-06T05:22:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff5877fe700 1 -- 172.21.15.31:0/3854540852 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598014670 con 0x7ff5a00c7190 2023-10-06T05:22:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.287+0000 7ff5a5bff700 1 -- 172.21.15.31:0/3854540852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a00c06c0 con 0x7ff5a00c7190 2023-10-06T05:22:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.288+0000 7ff5a5bff700 1 -- 172.21.15.31:0/3854540852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a004c380 con 0x7ff5a00c7190 2023-10-06T05:22:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.289+0000 7ff5877fe700 1 -- 172.21.15.31:0/3854540852 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7ff5980149e0 con 0x7ff5a00c7190 2023-10-06T05:22:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.290+0000 7ff5877fe700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff58c067050 0x7ff58c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.290+0000 7ff5877fe700 1 -- 172.21.15.31:0/3854540852 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff598084fe0 con 0x7ff5a00c7190 2023-10-06T05:22:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.291+0000 7ff59f7fe700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff58c067050 0x7ff58c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.292+0000 7ff59f7fe700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff58c067050 0x7ff58c069500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff590006f00 tx=0x7ff590008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.295+0000 7ff5877fe700 1 -- 172.21.15.31:0/3854540852 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff598051f80 con 0x7ff5a00c7190 2023-10-06T05:22:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.525+0000 7ff5a5bff700 1 -- 172.21.15.31:0/3854540852 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5a00cc6e0 con 0x7ff58c067050 2023-10-06T05:22:45.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.529+0000 7ff5877fe700 1 -- 172.21.15.31:0/3854540852 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+61 (secure 0 0 0) 0x7ff5a00cc6e0 con 0x7ff58c067050 2023-10-06T05:22:45.531 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:22:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:45.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subvolume_0000000000063418" 2023-10-06T05:22:45.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:45.532 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:22:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 -- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff58c067050 msgr2=0x7ff58c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff58c067050 0x7ff58c069500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff590006f00 tx=0x7ff590008040 comp rx=0 tx=0).stop 2023-10-06T05:22:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 -- 172.21.15.31:0/3854540852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00c7190 msgr2=0x7ff5a00c5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00c7190 0x7ff5a00c5b00 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff598009fa0 tx=0x7ff598009900 comp rx=0 tx=0).stop 2023-10-06T05:22:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 -- 172.21.15.31:0/3854540852 shutdown_connections 2023-10-06T05:22:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff58c067050 0x7ff58c069500 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a00c7190 0x7ff5a00c5b00 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a00aef00 0x7ff5a0157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 --2- 172.21.15.31:0/3854540852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a00afcf0 0x7ff5a00c2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 -- 172.21.15.31:0/3854540852 >> 172.21.15.31:0/3854540852 conn(0x7ff5a00cd3d0 msgr2=0x7ff5a0158930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:45.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 -- 172.21.15.31:0/3854540852 shutdown_connections 2023-10-06T05:22:45.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.532+0000 7ff5a5bff700 1 -- 172.21.15.31:0/3854540852 wait complete. 2023-10-06T05:22:45.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvolume_0000000000063418 2023-10-06T05:22:45.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.747+0000 7fe8fc9ff700 1 Processor -- start 2023-10-06T05:22:45.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.747+0000 7fe8fc9ff700 1 -- start start 2023-10-06T05:22:45.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.748+0000 7fe8fc9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe8f8159c60 0x7fe8f80c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.748+0000 7fe8fc9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f8158190 con 0x7fe8f8155430 2023-10-06T05:22:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.748+0000 7fe8fc9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f8157580 con 0x7fe8f8159890 2023-10-06T05:22:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.748+0000 7fe8fc9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f8157860 con 0x7fe8f8159c60 2023-10-06T05:22:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.748+0000 7fe8f6d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe8f8155430 0x7fe8f8155890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44538/0 (socket says 172.21.15.31:44538) 2023-10-06T05:22:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.748+0000 7fe8f6d9d700 1 -- 172.21.15.31:0/3266488671 learned_addr learned my addr 172.21.15.31:0/3266488671 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.749+0000 7fe8f5d9b700 1 --2- 172.21.15.31:0/3266488671 >> v2:172.21.15.62:3300/0 conn(0x7fe8f8159c60 0x7fe8f80c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.749+0000 7fe8f559a700 1 -- 172.21.15.31:0/3266488671 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694151351 0 0) 0x7fe8f8158190 con 0x7fe8f8155430 2023-10-06T05:22:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.749+0000 7fe8f559a700 1 -- 172.21.15.31:0/3266488671 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e0004680 con 0x7fe8f8155430 2023-10-06T05:22:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.749+0000 7fe8f5d9b700 1 -- 172.21.15.31:0/3266488671 >> v1:172.21.15.62:6789/0 conn(0x7fe8f8159890 legacy=0x7fe8f8156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.749+0000 7fe8f5d9b700 1 -- 172.21.15.31:0/3266488671 >> v1:172.21.15.31:6789/0 conn(0x7fe8f8155430 legacy=0x7fe8f8155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.749+0000 7fe8f5d9b700 1 -- 172.21.15.31:0/3266488671 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8157b40 con 0x7fe8f8159c60 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.750+0000 7fe8f5d9b700 1 --2- 172.21.15.31:0/3266488671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8f8159c60 0x7fe8f80c89e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0098e0 tx=0x7fe8ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bfb16df934aa1b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.750+0000 7fe8f559a700 1 -- 172.21.15.31:0/3266488671 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00b640 con 0x7fe8f8159c60 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.750+0000 7fe8f559a700 1 -- 172.21.15.31:0/3266488671 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec00b7a0 con 0x7fe8f8159c60 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.750+0000 7fe8f559a700 1 -- 172.21.15.31:0/3266488671 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00bb40 con 0x7fe8f8159c60 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.750+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/3266488671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8f8159c60 msgr2=0x7fe8f80c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.750+0000 7fe8fc9ff700 1 --2- 172.21.15.31:0/3266488671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8f8159c60 0x7fe8f80c89e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0098e0 tx=0x7fe8ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.750+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/3266488671 shutdown_connections 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.750+0000 7fe8fc9ff700 1 --2- 172.21.15.31:0/3266488671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8f8159c60 0x7fe8f80c89e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.750+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/3266488671 >> 172.21.15.31:0/3266488671 conn(0x7fe8f8067560 msgr2=0x7fe8f8067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.751+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/3266488671 shutdown_connections 2023-10-06T05:22:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.751+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/3266488671 wait complete. 2023-10-06T05:22:45.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.752+0000 7fe8fc9ff700 1 Processor -- start 2023-10-06T05:22:45.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.752+0000 7fe8fc9ff700 1 -- start start 2023-10-06T05:22:45.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.753+0000 7fe8fc9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8f8155430 0x7fe8f81699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.753+0000 7fe8fc9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8f8159890 0x7fe8f816d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.753+0000 7fe8fc9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8f8159c60 0x7fe8f81500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.753+0000 7fe8fc9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f816d2b0 con 0x7fe8f8159890 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.753+0000 7fe8fc9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f816a240 con 0x7fe8f8155430 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.753+0000 7fe8fc9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f816a520 con 0x7fe8f8159c60 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.753+0000 7fe8f5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8f8159890 0x7fe8f816d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.753+0000 7fe8f5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8f8159890 0x7fe8f816d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42612/0 (socket says 172.21.15.31:42612) 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.753+0000 7fe8f5d9b700 1 -- 172.21.15.31:0/1478577734 learned_addr learned my addr 172.21.15.31:0/1478577734 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8f659c700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8f8155430 0x7fe8f81699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8f6d9d700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8f8159c60 0x7fe8f81500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8f5d9b700 1 -- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8f8159c60 msgr2=0x7fe8f81500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8f5d9b700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8f8159c60 0x7fe8f81500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8f5d9b700 1 -- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8f8155430 msgr2=0x7fe8f81699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8f5d9b700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8f8155430 0x7fe8f81699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8f5d9b700 1 -- 172.21.15.31:0/1478577734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec009420 con 0x7fe8f8159890 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8f6d9d700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8f8159c60 0x7fe8f81500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8f5d9b700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8f8159890 0x7fe8f816d740 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe8f806f430 tx=0x7fe8ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8e67fc700 1 -- 172.21.15.31:0/1478577734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00bb20 con 0x7fe8f8159890 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8e67fc700 1 -- 172.21.15.31:0/1478577734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec011420 con 0x7fe8f8159890 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8e67fc700 1 -- 172.21.15.31:0/1478577734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0117a0 con 0x7fe8f8159890 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.754+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/1478577734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f816a800 con 0x7fe8f8159890 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.755+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/1478577734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f8169f30 con 0x7fe8f8159890 2023-10-06T05:22:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.756+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/1478577734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f816d420 con 0x7fe8f8159890 2023-10-06T05:22:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.756+0000 7fe8e67fc700 1 -- 172.21.15.31:0/1478577734 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe8ec011b50 con 0x7fe8f8159890 2023-10-06T05:22:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.758+0000 7fe8e67fc700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe8d8067050 0x7fe8d8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.758+0000 7fe8e67fc700 1 -- 172.21.15.31:0/1478577734 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe8ec085f60 con 0x7fe8f8159890 2023-10-06T05:22:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.758+0000 7fe8f659c700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe8d8067050 0x7fe8d8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.759+0000 7fe8f659c700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe8d8067050 0x7fe8d8069500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe8f806f280 tx=0x7fe8e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:45.762+0000 7fe8e67fc700 1 -- 172.21.15.31:0/1478577734 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8ec052f60 con 0x7fe8f8159890 2023-10-06T05:22:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.001+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/1478577734 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8f804c380 con 0x7fe8d8067050 2023-10-06T05:22:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.031+0000 7fe8e67fc700 1 -- 172.21.15.31:0/1478577734 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+682 (secure 0 0 0) 0x7fe8f804c380 con 0x7fe8d8067050 2023-10-06T05:22:46.033 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:22:46.033 INFO:teuthology.orchestra.run.smithi031.stdout: "atime": "2023-10-06 05:22:44", 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "undefined", 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": "infinite", 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 0, 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-10-06 05:22:44", 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "ctime": "2023-10-06 05:22:45", 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "features": [ 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-clone", 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-autoprotect" 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 0, 2023-10-06T05:22:46.034 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": 16877, 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_addrs": [ 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.31:6789", 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6789", 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6790" 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "mtime": "2023-10-06 05:22:44", 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1", 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_namespace": "", 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete", 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "subvolume", 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uid": 0 2023-10-06T05:22:46.035 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:22:46.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe8d8067050 msgr2=0x7fe8d8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe8d8067050 0x7fe8d8069500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe8f806f280 tx=0x7fe8e8009040 comp rx=0 tx=0).stop 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/1478577734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8f8159890 msgr2=0x7fe8f816d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8f8159890 0x7fe8f816d740 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe8f806f430 tx=0x7fe8ec017000 comp rx=0 tx=0).stop 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/1478577734 shutdown_connections 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe8d8067050 0x7fe8d8069500 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8f8159c60 0x7fe8f81500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8f8155430 0x7fe8f81699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 --2- 172.21.15.31:0/1478577734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8f8159890 0x7fe8f816d740 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/1478577734 >> 172.21.15.31:0/1478577734 conn(0x7fe8f8067560 msgr2=0x7fe8f81560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/1478577734 shutdown_connections 2023-10-06T05:22:46.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.034+0000 7fe8fc9ff700 1 -- 172.21.15.31:0/1478577734 wait complete. 2023-10-06T05:22:46.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvolume_0000000000063418 10485760 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.256+0000 7f66f09ff700 1 Processor -- start 2023-10-06T05:22:46.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.257+0000 7f66f09ff700 1 -- start start 2023-10-06T05:22:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.258+0000 7f66f09ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 0x7f66ec0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.258+0000 7f66f09ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f66ec15a990 0x7f66ec0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.258+0000 7f66f09ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f66ec142ff0 0x7f66ec0cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.258+0000 7f66f09ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ec15d210 con 0x7f66ec142ff0 2023-10-06T05:22:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.258+0000 7f66f09ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ec0cc810 con 0x7f66ec15a990 2023-10-06T05:22:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.258+0000 7f66f09ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec15d4b0 con 0x7f66ec142c20 2023-10-06T05:22:46.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.258+0000 7f66ead9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f66ec142ff0 0x7f66ec0cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.258+0000 7f66ead9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f66ec142ff0 0x7f66ec0cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42616/0 (socket says 172.21.15.31:42616) 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.258+0000 7f66ead9d700 1 -- 172.21.15.31:0/4239172194 learned_addr learned my addr 172.21.15.31:0/4239172194 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.258+0000 7f66e9d9b700 1 --2- 172.21.15.31:0/4239172194 >> v2:172.21.15.62:3300/0 conn(0x7f66ec15a990 0x7f66ec0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66ea59c700 1 --2- 172.21.15.31:0/4239172194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 0x7f66ec0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66ead9d700 1 -- 172.21.15.31:0/4239172194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 msgr2=0x7f66ec0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66ead9d700 1 --2- 172.21.15.31:0/4239172194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 0x7f66ec0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66ead9d700 1 -- 172.21.15.31:0/4239172194 >> v2:172.21.15.62:3300/0 conn(0x7f66ec15a990 msgr2=0x7f66ec0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66ead9d700 1 --2- 172.21.15.31:0/4239172194 >> v2:172.21.15.62:3300/0 conn(0x7f66ec15a990 0x7f66ec0cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66ead9d700 1 -- 172.21.15.31:0/4239172194 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec15d790 con 0x7f66ec142ff0 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66ea59c700 1 --2- 172.21.15.31:0/4239172194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 0x7f66ec0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:46.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66e9d9b700 1 --2- 172.21.15.31:0/4239172194 >> v2:172.21.15.62:3300/0 conn(0x7f66ec15a990 0x7f66ec0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:46.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66ead9d700 1 --2- 172.21.15.31:0/4239172194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 0x7f66ec0cb210 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f66ec06f290 tx=0x7f66e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9c3bea55d52305 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66e959a700 1 -- 172.21.15.31:0/4239172194 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400e070 con 0x7f66ec142ff0 2023-10-06T05:22:46.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66e959a700 1 -- 172.21.15.31:0/4239172194 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e400ba50 con 0x7f66ec142ff0 2023-10-06T05:22:46.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.259+0000 7f66e959a700 1 -- 172.21.15.31:0/4239172194 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e40092e0 con 0x7f66ec142ff0 2023-10-06T05:22:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.260+0000 7f66f09ff700 1 -- 172.21.15.31:0/4239172194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 msgr2=0x7f66ec0cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.260+0000 7f66f09ff700 1 --2- 172.21.15.31:0/4239172194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 0x7f66ec0cb210 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f66ec06f290 tx=0x7f66e4009840 comp rx=0 tx=0).stop 2023-10-06T05:22:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.260+0000 7f66f09ff700 1 -- 172.21.15.31:0/4239172194 shutdown_connections 2023-10-06T05:22:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.260+0000 7f66f09ff700 1 --2- 172.21.15.31:0/4239172194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 0x7f66ec0cb210 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.260+0000 7f66f09ff700 1 --2- 172.21.15.31:0/4239172194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 0x7f66ec0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.260+0000 7f66f09ff700 1 --2- 172.21.15.31:0/4239172194 >> v2:172.21.15.62:3300/0 conn(0x7f66ec15a990 0x7f66ec0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.260+0000 7f66f09ff700 1 -- 172.21.15.31:0/4239172194 >> 172.21.15.31:0/4239172194 conn(0x7f66ec0ce2d0 msgr2=0x7f66ec0ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.260+0000 7f66f09ff700 1 -- 172.21.15.31:0/4239172194 shutdown_connections 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.260+0000 7f66f09ff700 1 -- 172.21.15.31:0/4239172194 wait complete. 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.261+0000 7f66f09ff700 1 Processor -- start 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.262+0000 7f66f09ff700 1 -- start start 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.262+0000 7f66f09ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 0x7f66ec13fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.262+0000 7f66f09ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 0x7f66ec0b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.262+0000 7f66f09ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66ec15a990 0x7f66ec0ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.262+0000 7f66f09ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec0b32c0 con 0x7f66ec142ff0 2023-10-06T05:22:46.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.262+0000 7f66f09ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec140310 con 0x7f66ec15a990 2023-10-06T05:22:46.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.262+0000 7f66f09ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec0b26b0 con 0x7f66ec142c20 2023-10-06T05:22:46.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66e9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 0x7f66ec0b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66e9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 0x7f66ec0b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42622/0 (socket says 172.21.15.31:42622) 2023-10-06T05:22:46.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66e9d9b700 1 -- 172.21.15.31:0/902656187 learned_addr learned my addr 172.21.15.31:0/902656187 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66ea59c700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 0x7f66ec13fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66ead9d700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66ec15a990 0x7f66ec0ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66e9d9b700 1 -- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 msgr2=0x7f66ec13fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66e9d9b700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 0x7f66ec13fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66e9d9b700 1 -- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66ec15a990 msgr2=0x7f66ec0ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66e9d9b700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66ec15a990 0x7f66ec0ab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66e9d9b700 1 -- 172.21.15.31:0/902656187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e4009ae0 con 0x7f66ec142ff0 2023-10-06T05:22:46.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66ead9d700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66ec15a990 0x7f66ec0ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:46.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.263+0000 7f66e9d9b700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 0x7f66ec0b3a50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f66e000c800 tx=0x7f66e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.264+0000 7f66da7fc700 1 -- 172.21.15.31:0/902656187 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0011600 con 0x7f66ec142ff0 2023-10-06T05:22:46.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.264+0000 7f66da7fc700 1 -- 172.21.15.31:0/902656187 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e0011760 con 0x7f66ec142ff0 2023-10-06T05:22:46.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.264+0000 7f66f09ff700 1 -- 172.21.15.31:0/902656187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ec0b2990 con 0x7f66ec142ff0 2023-10-06T05:22:46.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.264+0000 7f66da7fc700 1 -- 172.21.15.31:0/902656187 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0011b20 con 0x7f66ec142ff0 2023-10-06T05:22:46.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.264+0000 7f66f09ff700 1 -- 172.21.15.31:0/902656187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ec0b2f50 con 0x7f66ec142ff0 2023-10-06T05:22:46.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.265+0000 7f66f09ff700 1 -- 172.21.15.31:0/902656187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ec04c380 con 0x7f66ec142ff0 2023-10-06T05:22:46.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.266+0000 7f66da7fc700 1 -- 172.21.15.31:0/902656187 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f66e0010350 con 0x7f66ec142ff0 2023-10-06T05:22:46.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.267+0000 7f66da7fc700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f66cc067050 0x7f66cc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.267+0000 7f66da7fc700 1 -- 172.21.15.31:0/902656187 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f66e00848f0 con 0x7f66ec142ff0 2023-10-06T05:22:46.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.267+0000 7f66ea59c700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f66cc067050 0x7f66cc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.268+0000 7f66ea59c700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f66cc067050 0x7f66cc069500 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f66ec153840 tx=0x7f66dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.271+0000 7f66da7fc700 1 -- 172.21.15.31:0/902656187 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66e0051740 con 0x7f66ec142ff0 2023-10-06T05:22:46.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.516+0000 7f66f09ff700 1 -- 172.21.15.31:0/902656187 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "new_size": "10485760", "target": ["mon-mgr", ""]}) v1 -- 0x7f66ec0b3400 con 0x7f66cc067050 2023-10-06T05:22:46.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.549+0000 7f66da7fc700 1 -- 172.21.15.31:0/902656187 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+128 (secure 0 0 0) 0x7f66ec0b3400 con 0x7f66cc067050 2023-10-06T05:22:46.551 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:22:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 0 2023-10-06T05:22:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": 10485760 2023-10-06T05:22:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:22:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:22:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "0.00" 2023-10-06T05:22:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:46.552 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:22:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.551+0000 7f66f09ff700 1 -- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f66cc067050 msgr2=0x7f66cc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.551+0000 7f66f09ff700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f66cc067050 0x7f66cc069500 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f66ec153840 tx=0x7f66dc008040 comp rx=0 tx=0).stop 2023-10-06T05:22:46.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.551+0000 7f66f09ff700 1 -- 172.21.15.31:0/902656187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 msgr2=0x7f66ec0b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.551+0000 7f66f09ff700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 0x7f66ec0b3a50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f66e000c800 tx=0x7f66e0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:46.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.551+0000 7f66f09ff700 1 -- 172.21.15.31:0/902656187 shutdown_connections 2023-10-06T05:22:46.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.551+0000 7f66f09ff700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f66cc067050 0x7f66cc069500 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.551+0000 7f66f09ff700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66ec15a990 0x7f66ec0ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.551+0000 7f66f09ff700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66ec142c20 0x7f66ec13fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.551+0000 7f66f09ff700 1 --2- 172.21.15.31:0/902656187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ec142ff0 0x7f66ec0b3a50 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.551+0000 7f66f09ff700 1 -- 172.21.15.31:0/902656187 >> 172.21.15.31:0/902656187 conn(0x7f66ec0ce2d0 msgr2=0x7f66ec0ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.552+0000 7f66f09ff700 1 -- 172.21.15.31:0/902656187 shutdown_connections 2023-10-06T05:22:46.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.552+0000 7f66f09ff700 1 -- 172.21.15.31:0/902656187 wait complete. 2023-10-06T05:22:46.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvolume_0000000000063418 2023-10-06T05:22:46.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.769+0000 7fe5b2bff700 1 Processor -- start 2023-10-06T05:22:46.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.770+0000 7fe5b2bff700 1 -- start start 2023-10-06T05:22:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.771+0000 7fe5b2bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe5ac153d40 0x7fe5ac0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.771+0000 7fe5b2bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe5ac0af890 0x7fe5ac0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.771+0000 7fe5b2bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ac0b0e40 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.771+0000 7fe5b2bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac0afc90 con 0x7fe5ac0af890 2023-10-06T05:22:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.771+0000 7fe5b2bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac0ad690 con 0x7fe5ac153d40 2023-10-06T05:22:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.771+0000 7fe5b119c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe5ac0af890 0x7fe5ac0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.771+0000 7fe5b119c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe5ac0af890 0x7fe5ac0adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42628/0 (socket says 172.21.15.31:42628) 2023-10-06T05:22:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.771+0000 7fe5b119c700 1 -- 172.21.15.31:0/608803365 learned_addr learned my addr 172.21.15.31:0/608803365 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:46.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.771+0000 7fe5b099b700 1 --2- 172.21.15.31:0/608803365 >> v2:172.21.15.62:3300/0 conn(0x7fe5ac153d40 0x7fe5ac0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.772+0000 7fe5b119c700 1 -- 172.21.15.31:0/608803365 >> v2:172.21.15.62:3300/0 conn(0x7fe5ac153d40 msgr2=0x7fe5ac0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:46.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.772+0000 7fe5b119c700 1 --2- 172.21.15.31:0/608803365 >> v2:172.21.15.62:3300/0 conn(0x7fe5ac153d40 0x7fe5ac0b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.772+0000 7fe5b119c700 1 -- 172.21.15.31:0/608803365 >> v1:172.21.15.62:6789/0 conn(0x7fe5ac0b09c0 legacy=0x7fe5ac0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.772+0000 7fe5b119c700 1 -- 172.21.15.31:0/608803365 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac0acd60 con 0x7fe5ac0af890 2023-10-06T05:22:46.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.772+0000 7fe5b119c700 1 --2- 172.21.15.31:0/608803365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0af890 0x7fe5ac0adb20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac06d1d0 tx=0x7fe5a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0465ab634ffbbe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.772+0000 7fe5ab7fe700 1 -- 172.21.15.31:0/608803365 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40092e0 con 0x7fe5ac0af890 2023-10-06T05:22:46.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.772+0000 7fe5ab7fe700 1 -- 172.21.15.31:0/608803365 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a400ba20 con 0x7fe5ac0af890 2023-10-06T05:22:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.772+0000 7fe5ab7fe700 1 -- 172.21.15.31:0/608803365 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a400bde0 con 0x7fe5ac0af890 2023-10-06T05:22:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.773+0000 7fe5b2bff700 1 -- 172.21.15.31:0/608803365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0af890 msgr2=0x7fe5ac0adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.773+0000 7fe5b2bff700 1 --2- 172.21.15.31:0/608803365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0af890 0x7fe5ac0adb20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac06d1d0 tx=0x7fe5a4009840 comp rx=0 tx=0).stop 2023-10-06T05:22:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.773+0000 7fe5b2bff700 1 -- 172.21.15.31:0/608803365 shutdown_connections 2023-10-06T05:22:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.773+0000 7fe5b2bff700 1 --2- 172.21.15.31:0/608803365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0af890 0x7fe5ac0adb20 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.773+0000 7fe5b2bff700 1 --2- 172.21.15.31:0/608803365 >> v2:172.21.15.62:3300/0 conn(0x7fe5ac153d40 0x7fe5ac0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.773+0000 7fe5b2bff700 1 -- 172.21.15.31:0/608803365 >> 172.21.15.31:0/608803365 conn(0x7fe5ac067560 msgr2=0x7fe5ac067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.773+0000 7fe5b2bff700 1 -- 172.21.15.31:0/608803365 shutdown_connections 2023-10-06T05:22:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.773+0000 7fe5b2bff700 1 -- 172.21.15.31:0/608803365 wait complete. 2023-10-06T05:22:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.774+0000 7fe5b2bff700 1 Processor -- start 2023-10-06T05:22:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.775+0000 7fe5b2bff700 1 -- start start 2023-10-06T05:22:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.775+0000 7fe5b2bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ac0af890 0x7fe5ac152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.775+0000 7fe5b2bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0b09c0 0x7fe5ac158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5b2bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ac153d40 0x7fe5ac159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5b2bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac1526a0 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5b2bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0cab30 con 0x7fe5ac153d40 2023-10-06T05:22:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5b2bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0c9f20 con 0x7fe5ac0af890 2023-10-06T05:22:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0b09c0 0x7fe5ac158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0b09c0 0x7fe5ac158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42638/0 (socket says 172.21.15.31:42638) 2023-10-06T05:22:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5abfff700 1 -- 172.21.15.31:0/971922906 learned_addr learned my addr 172.21.15.31:0/971922906 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5b119c700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ac153d40 0x7fe5ac159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5b099b700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ac0af890 0x7fe5ac152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5abfff700 1 -- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ac0af890 msgr2=0x7fe5ac152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5abfff700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ac0af890 0x7fe5ac152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5abfff700 1 -- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ac153d40 msgr2=0x7fe5ac159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5abfff700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ac153d40 0x7fe5ac159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5abfff700 1 -- 172.21.15.31:0/971922906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a4009ae0 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.776+0000 7fe5abfff700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0b09c0 0x7fe5ac158f40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe59800c590 tx=0x7fe59800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.777+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/971922906 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598011590 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.777+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/971922906 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5980116f0 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.777+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/971922906 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598011ab0 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.777+0000 7fe5b2bff700 1 -- 172.21.15.31:0/971922906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac0ca200 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.777+0000 7fe5b2bff700 1 -- 172.21.15.31:0/971922906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac0ca7c0 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.778+0000 7fe5b2bff700 1 -- 172.21.15.31:0/971922906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac04c380 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.779+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/971922906 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe59801b3b0 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.780+0000 7fe5a8ff9700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe590067050 0x7fe590069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.780+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/971922906 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe598083c30 con 0x7fe5ac0b09c0 2023-10-06T05:22:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.780+0000 7fe5b099b700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe590067050 0x7fe590069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.781+0000 7fe5b099b700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe590067050 0x7fe590069500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009ca0 tx=0x7fe5a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:46.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:46.784+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/971922906 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe598050b90 con 0x7fe5ac0b09c0 2023-10-06T05:22:47.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.020+0000 7fe5b2bff700 1 -- 172.21.15.31:0/971922906 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5ac0caca0 con 0x7fe590067050 2023-10-06T05:22:47.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.050+0000 7fe5a8ff9700 1 -- 172.21.15.31:0/971922906 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+675 (secure 0 0 0) 0x7fe5ac0caca0 con 0x7fe590067050 2023-10-06T05:22:47.054 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:22:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "atime": "2023-10-06 05:22:44", 2023-10-06T05:22:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "0.00", 2023-10-06T05:22:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": 10485760, 2023-10-06T05:22:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 0, 2023-10-06T05:22:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-10-06 05:22:44", 2023-10-06T05:22:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "ctime": "2023-10-06 05:22:46", 2023-10-06T05:22:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-10-06T05:22:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "features": [ 2023-10-06T05:22:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-clone", 2023-10-06T05:22:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-autoprotect" 2023-10-06T05:22:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:22:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 0, 2023-10-06T05:22:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": 16877, 2023-10-06T05:22:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_addrs": [ 2023-10-06T05:22:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.31:6789", 2023-10-06T05:22:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6789", 2023-10-06T05:22:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6790" 2023-10-06T05:22:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:22:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "mtime": "2023-10-06 05:22:44", 2023-10-06T05:22:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1", 2023-10-06T05:22:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_namespace": "", 2023-10-06T05:22:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete", 2023-10-06T05:22:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "subvolume", 2023-10-06T05:22:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "uid": 0 2023-10-06T05:22:47.058 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:22:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.052+0000 7fe5b2bff700 1 -- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe590067050 msgr2=0x7fe590069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.052+0000 7fe5b2bff700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe590067050 0x7fe590069500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009ca0 tx=0x7fe5a0008040 comp rx=0 tx=0).stop 2023-10-06T05:22:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.052+0000 7fe5b2bff700 1 -- 172.21.15.31:0/971922906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0b09c0 msgr2=0x7fe5ac158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.052+0000 7fe5b2bff700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0b09c0 0x7fe5ac158f40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe59800c590 tx=0x7fe59800c950 comp rx=0 tx=0).stop 2023-10-06T05:22:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.052+0000 7fe5b2bff700 1 -- 172.21.15.31:0/971922906 shutdown_connections 2023-10-06T05:22:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.052+0000 7fe5b2bff700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe590067050 0x7fe590069500 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.052+0000 7fe5b2bff700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5ac153d40 0x7fe5ac159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.052+0000 7fe5b2bff700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5ac0af890 0x7fe5ac152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.052+0000 7fe5b2bff700 1 --2- 172.21.15.31:0/971922906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5ac0b09c0 0x7fe5ac158f40 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.052+0000 7fe5b2bff700 1 -- 172.21.15.31:0/971922906 >> 172.21.15.31:0/971922906 conn(0x7fe5ac067560 msgr2=0x7fe5ac06e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.053+0000 7fe5b2bff700 1 -- 172.21.15.31:0/971922906 shutdown_connections 2023-10-06T05:22:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.053+0000 7fe5b2bff700 1 -- 172.21.15.31:0/971922906 wait complete. 2023-10-06T05:22:47.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000063418 --mode 777 --uid 1000 --gid 1000 2023-10-06T05:22:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.276+0000 7f40475ff700 1 Processor -- start 2023-10-06T05:22:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.276+0000 7f40475ff700 1 -- start start 2023-10-06T05:22:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f40475ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4040153d40 0x7f40400b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f40475ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40400b09c0 0x7f40400af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f40475ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f40400af6c0 0x7f4040159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f40475ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40400ad690 con 0x7f4040153d40 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f40475ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40400b0e10 con 0x7f40400af6c0 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f40475ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40400acd60 con 0x7f40400b09c0 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f404539b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4040153d40 0x7f40400b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f404539b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4040153d40 0x7f40400b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59222/0 (socket says 172.21.15.31:59222) 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f404539b700 1 -- 172.21.15.31:0/3254159196 learned_addr learned my addr 172.21.15.31:0/3254159196 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f4045b9c700 1 --2- 172.21.15.31:0/3254159196 >> v2:172.21.15.62:3300/0 conn(0x7f40400af6c0 0x7f4040159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f4044b9a700 1 --2- 172.21.15.31:0/3254159196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40400b09c0 0x7f40400af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f404539b700 1 -- 172.21.15.31:0/3254159196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40400b09c0 msgr2=0x7f40400af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f404539b700 1 --2- 172.21.15.31:0/3254159196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40400b09c0 0x7f40400af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f404539b700 1 -- 172.21.15.31:0/3254159196 >> v2:172.21.15.62:3300/0 conn(0x7f40400af6c0 msgr2=0x7f4040159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f404539b700 1 --2- 172.21.15.31:0/3254159196 >> v2:172.21.15.62:3300/0 conn(0x7f40400af6c0 0x7f4040159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.278+0000 7f404539b700 1 -- 172.21.15.31:0/3254159196 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40400ad040 con 0x7f4040153d40 2023-10-06T05:22:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.279+0000 7f404539b700 1 --2- 172.21.15.31:0/3254159196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 0x7f40400b0480 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f404006d180 tx=0x7f403c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cfd0a191d85cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.279+0000 7f4037fff700 1 -- 172.21.15.31:0/3254159196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c00b640 con 0x7f4040153d40 2023-10-06T05:22:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.279+0000 7f4037fff700 1 -- 172.21.15.31:0/3254159196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c00b7a0 con 0x7f4040153d40 2023-10-06T05:22:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.279+0000 7f4037fff700 1 -- 172.21.15.31:0/3254159196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c00bb60 con 0x7f4040153d40 2023-10-06T05:22:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.280+0000 7f40475ff700 1 -- 172.21.15.31:0/3254159196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 msgr2=0x7f40400b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.280+0000 7f40475ff700 1 --2- 172.21.15.31:0/3254159196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 0x7f40400b0480 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f404006d180 tx=0x7f403c009250 comp rx=0 tx=0).stop 2023-10-06T05:22:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.280+0000 7f40475ff700 1 -- 172.21.15.31:0/3254159196 shutdown_connections 2023-10-06T05:22:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.280+0000 7f40475ff700 1 --2- 172.21.15.31:0/3254159196 >> v2:172.21.15.62:3300/0 conn(0x7f40400af6c0 0x7f4040159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.280+0000 7f40475ff700 1 --2- 172.21.15.31:0/3254159196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 0x7f40400b0480 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.280+0000 7f40475ff700 1 --2- 172.21.15.31:0/3254159196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40400b09c0 0x7f40400af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.280+0000 7f40475ff700 1 -- 172.21.15.31:0/3254159196 >> 172.21.15.31:0/3254159196 conn(0x7f4040067560 msgr2=0x7f4040067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.280+0000 7f40475ff700 1 -- 172.21.15.31:0/3254159196 shutdown_connections 2023-10-06T05:22:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.280+0000 7f40475ff700 1 -- 172.21.15.31:0/3254159196 wait complete. 2023-10-06T05:22:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.281+0000 7f40475ff700 1 Processor -- start 2023-10-06T05:22:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.281+0000 7f40475ff700 1 -- start start 2023-10-06T05:22:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.282+0000 7f40475ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40400af6c0 0x7f40400c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.282+0000 7f40475ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40400b09c0 0x7f40400cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.282+0000 7f40475ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 0x7f404014edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.282+0000 7f40475ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40400c9cd0 con 0x7f4040153d40 2023-10-06T05:22:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.282+0000 7f40475ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40401501b0 con 0x7f40400af6c0 2023-10-06T05:22:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.282+0000 7f40475ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40400c9ee0 con 0x7f40400b09c0 2023-10-06T05:22:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f4045b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 0x7f404014edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f4045b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 0x7f404014edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59228/0 (socket says 172.21.15.31:59228) 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f4045b9c700 1 -- 172.21.15.31:0/3699400739 learned_addr learned my addr 172.21.15.31:0/3699400739 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f4044b9a700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40400b09c0 0x7f40400cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f404539b700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40400af6c0 0x7f40400c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f4045b9c700 1 -- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40400b09c0 msgr2=0x7f40400cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f4045b9c700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40400b09c0 0x7f40400cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f4045b9c700 1 -- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40400af6c0 msgr2=0x7f40400c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f4045b9c700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40400af6c0 0x7f40400c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f4045b9c700 1 -- 172.21.15.31:0/3699400739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0094f0 con 0x7f4040153d40 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f4045b9c700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 0x7f404014edd0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4038009e90 tx=0x7f403800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.283+0000 7f40357fa700 1 -- 172.21.15.31:0/3699400739 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403800cb70 con 0x7f4040153d40 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.284+0000 7f40357fa700 1 -- 172.21.15.31:0/3699400739 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403800f040 con 0x7f4040153d40 2023-10-06T05:22:47.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.284+0000 7f40357fa700 1 -- 172.21.15.31:0/3699400739 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038014480 con 0x7f4040153d40 2023-10-06T05:22:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.284+0000 7f40475ff700 1 -- 172.21.15.31:0/3699400739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40400ca1c0 con 0x7f4040153d40 2023-10-06T05:22:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.284+0000 7f40475ff700 1 -- 172.21.15.31:0/3699400739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4040154110 con 0x7f4040153d40 2023-10-06T05:22:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.285+0000 7f40475ff700 1 -- 172.21.15.31:0/3699400739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404004c380 con 0x7f4040153d40 2023-10-06T05:22:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.285+0000 7f40357fa700 1 -- 172.21.15.31:0/3699400739 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f4038014830 con 0x7f4040153d40 2023-10-06T05:22:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.287+0000 7f40357fa700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40240670e0 0x7f4024069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.287+0000 7f40357fa700 1 -- 172.21.15.31:0/3699400739 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4038085520 con 0x7f4040153d40 2023-10-06T05:22:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.287+0000 7f404539b700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40240670e0 0x7f4024069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.288+0000 7f404539b700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40240670e0 0x7f4024069590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f404006d180 tx=0x7f403c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.291+0000 7f40357fa700 1 -- 172.21.15.31:0/3699400739 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4038052200 con 0x7f4040153d40 2023-10-06T05:22:47.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.523+0000 7f40475ff700 1 -- 172.21.15.31:0/3699400739 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) v1 -- 0x7f40400ad320 con 0x7f40240670e0 2023-10-06T05:22:47.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.559+0000 7f40357fa700 1 -- 172.21.15.31:0/3699400739 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f40400ad320 con 0x7f40240670e0 2023-10-06T05:22:47.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 -- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40240670e0 msgr2=0x7f4024069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40240670e0 0x7f4024069590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f404006d180 tx=0x7f403c017000 comp rx=0 tx=0).stop 2023-10-06T05:22:47.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 -- 172.21.15.31:0/3699400739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 msgr2=0x7f404014edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 0x7f404014edd0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4038009e90 tx=0x7f403800b040 comp rx=0 tx=0).stop 2023-10-06T05:22:47.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 -- 172.21.15.31:0/3699400739 shutdown_connections 2023-10-06T05:22:47.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40240670e0 0x7f4024069590 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4040153d40 0x7f404014edd0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40400af6c0 0x7f40400c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 --2- 172.21.15.31:0/3699400739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40400b09c0 0x7f40400cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 -- 172.21.15.31:0/3699400739 >> 172.21.15.31:0/3699400739 conn(0x7f4040067560 msgr2=0x7f404006daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 -- 172.21.15.31:0/3699400739 shutdown_connections 2023-10-06T05:22:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.562+0000 7f40475ff700 1 -- 172.21.15.31:0/3699400739 wait complete. 2023-10-06T05:22:47.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000063418 2023-10-06T05:22:47.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.778+0000 7fbb9bbff700 1 Processor -- start 2023-10-06T05:22:47.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.778+0000 7fbb9bbff700 1 -- start start 2023-10-06T05:22:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.779+0000 7fbb9bbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9415ca40 0x7fbb94068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.779+0000 7fbb9bbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbb940c8f40 0x7fbb940adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.779+0000 7fbb9bbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb94068da0 con 0x7fbb9406a300 2023-10-06T05:22:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.779+0000 7fbb9bbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb940cabc0 con 0x7fbb940c8f40 2023-10-06T05:22:47.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.779+0000 7fbb9bbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94068bb0 con 0x7fbb9415ca40 2023-10-06T05:22:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.780+0000 7fbb9999b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9415ca40 0x7fbb94068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.780+0000 7fbb9a19c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbb940c8f40 0x7fbb940adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.780+0000 7fbb9a19c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbb940c8f40 0x7fbb940adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:46562/0 (socket says 172.21.15.31:46562) 2023-10-06T05:22:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.780+0000 7fbb9a19c700 1 -- 172.21.15.31:0/2821553713 learned_addr learned my addr 172.21.15.31:0/2821553713 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.780+0000 7fbb9919a700 1 --1- 172.21.15.31:0/2821553713 >> v1:172.21.15.62:6789/0 conn(0x7fbb9406a300 0x7fbb940c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:44650/0 (socket says 172.21.15.31:44650) 2023-10-06T05:22:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.780+0000 7fbb9999b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9415ca40 0x7fbb94068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:43576/0 (socket says 172.21.15.31:43576) 2023-10-06T05:22:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.780+0000 7fbb98999700 1 -- 172.21.15.31:0/2821553713 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212593252 0 0) 0x7fbb94068da0 con 0x7fbb9406a300 2023-10-06T05:22:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.780+0000 7fbb98999700 1 -- 172.21.15.31:0/2821553713 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb80003570 con 0x7fbb9406a300 2023-10-06T05:22:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.781+0000 7fbb9999b700 1 -- 172.21.15.31:0/2821553713 >> v2:172.21.15.62:3300/0 conn(0x7fbb940c8f40 msgr2=0x7fbb940adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.781+0000 7fbb9999b700 1 --2- 172.21.15.31:0/2821553713 >> v2:172.21.15.62:3300/0 conn(0x7fbb940c8f40 0x7fbb940adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.781+0000 7fbb9999b700 1 -- 172.21.15.31:0/2821553713 >> v1:172.21.15.62:6789/0 conn(0x7fbb9406a300 legacy=0x7fbb940c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.781+0000 7fbb9999b700 1 -- 172.21.15.31:0/2821553713 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94069080 con 0x7fbb9415ca40 2023-10-06T05:22:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.781+0000 7fbb9a19c700 1 --2- 172.21.15.31:0/2821553713 >> v2:172.21.15.62:3300/0 conn(0x7fbb940c8f40 0x7fbb940adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.781+0000 7fbb9999b700 1 --2- 172.21.15.31:0/2821553713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9415ca40 0x7fbb94068670 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb900098e0 tx=0x7fbb900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df90139f59b3c06a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.781+0000 7fbb98999700 1 -- 172.21.15.31:0/2821553713 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90011630 con 0x7fbb9415ca40 2023-10-06T05:22:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.781+0000 7fbb98999700 1 -- 172.21.15.31:0/2821553713 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb90011790 con 0x7fbb9415ca40 2023-10-06T05:22:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.781+0000 7fbb98999700 1 -- 172.21.15.31:0/2821553713 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90011b30 con 0x7fbb9415ca40 2023-10-06T05:22:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.782+0000 7fbb9bbff700 1 -- 172.21.15.31:0/2821553713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9415ca40 msgr2=0x7fbb94068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.782+0000 7fbb9bbff700 1 --2- 172.21.15.31:0/2821553713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9415ca40 0x7fbb94068670 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb900098e0 tx=0x7fbb900091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.782+0000 7fbb9bbff700 1 -- 172.21.15.31:0/2821553713 shutdown_connections 2023-10-06T05:22:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.782+0000 7fbb9bbff700 1 --2- 172.21.15.31:0/2821553713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9415ca40 0x7fbb94068670 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.782+0000 7fbb9bbff700 1 --2- 172.21.15.31:0/2821553713 >> v2:172.21.15.62:3300/0 conn(0x7fbb940c8f40 0x7fbb940adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.782+0000 7fbb9bbff700 1 -- 172.21.15.31:0/2821553713 >> 172.21.15.31:0/2821553713 conn(0x7fbb940ce290 msgr2=0x7fbb940ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.782+0000 7fbb9bbff700 1 -- 172.21.15.31:0/2821553713 shutdown_connections 2023-10-06T05:22:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.782+0000 7fbb9bbff700 1 -- 172.21.15.31:0/2821553713 wait complete. 2023-10-06T05:22:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.783+0000 7fbb9bbff700 1 Processor -- start 2023-10-06T05:22:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.783+0000 7fbb9bbff700 1 -- start start 2023-10-06T05:22:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.784+0000 7fbb9bbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9406a300 0x7fbb9414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.784+0000 7fbb9bbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb940c8f40 0x7fbb940c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.784+0000 7fbb9bbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb9415ca40 0x7fbb9415b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.784+0000 7fbb9bbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9416a7c0 con 0x7fbb940c8f40 2023-10-06T05:22:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.784+0000 7fbb9bbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94169630 con 0x7fbb9415ca40 2023-10-06T05:22:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.784+0000 7fbb9bbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9414fd80 con 0x7fbb9406a300 2023-10-06T05:22:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb940c8f40 0x7fbb940c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb940c8f40 0x7fbb940c3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59238/0 (socket says 172.21.15.31:59238) 2023-10-06T05:22:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9919a700 1 -- 172.21.15.31:0/3964048859 learned_addr learned my addr 172.21.15.31:0/3964048859 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9999b700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9406a300 0x7fbb9414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9a19c700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb9415ca40 0x7fbb9415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9919a700 1 -- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9406a300 msgr2=0x7fbb9414f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9919a700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9406a300 0x7fbb9414f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9919a700 1 -- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb9415ca40 msgr2=0x7fbb9415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9919a700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb9415ca40 0x7fbb9415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9919a700 1 -- 172.21.15.31:0/3964048859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c009ae0 con 0x7fbb940c8f40 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.785+0000 7fbb9919a700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb940c8f40 0x7fbb940c3010 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400c590 tx=0x7fbb84007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.786+0000 7fbb89ffb700 1 -- 172.21.15.31:0/3964048859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84005a70 con 0x7fbb940c8f40 2023-10-06T05:22:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.786+0000 7fbb89ffb700 1 -- 172.21.15.31:0/3964048859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb84005bf0 con 0x7fbb940c8f40 2023-10-06T05:22:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.786+0000 7fbb89ffb700 1 -- 172.21.15.31:0/3964048859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb840070d0 con 0x7fbb940c8f40 2023-10-06T05:22:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.786+0000 7fbb9bbff700 1 -- 172.21.15.31:0/3964048859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb90009420 con 0x7fbb940c8f40 2023-10-06T05:22:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.786+0000 7fbb9bbff700 1 -- 172.21.15.31:0/3964048859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb94169dc0 con 0x7fbb940c8f40 2023-10-06T05:22:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.787+0000 7fbb9bbff700 1 -- 172.21.15.31:0/3964048859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9404c380 con 0x7fbb940c8f40 2023-10-06T05:22:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.788+0000 7fbb89ffb700 1 -- 172.21.15.31:0/3964048859 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fbb840074c0 con 0x7fbb940c8f40 2023-10-06T05:22:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.789+0000 7fbb89ffb700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbb780670e0 0x7fbb78069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.789+0000 7fbb89ffb700 1 -- 172.21.15.31:0/3964048859 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbb84083ea0 con 0x7fbb940c8f40 2023-10-06T05:22:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.789+0000 7fbb9999b700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbb780670e0 0x7fbb78069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.790+0000 7fbb9999b700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbb780670e0 0x7fbb78069590 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbb900091e0 tx=0x7fbb90017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:47.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:47.793+0000 7fbb89ffb700 1 -- 172.21.15.31:0/3964048859 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb84050c90 con 0x7fbb940c8f40 2023-10-06T05:22:48.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.031+0000 7fbb9bbff700 1 -- 172.21.15.31:0/3964048859 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb940ca1a0 con 0x7fbb780670e0 2023-10-06T05:22:48.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.067+0000 7fbb89ffb700 1 -- 172.21.15.31:0/3964048859 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7fbb940ca1a0 con 0x7fbb780670e0 2023-10-06T05:22:48.069 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1 2023-10-06T05:22:48.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 -- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbb780670e0 msgr2=0x7fbb78069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:48.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbb780670e0 0x7fbb78069590 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbb900091e0 tx=0x7fbb90017000 comp rx=0 tx=0).stop 2023-10-06T05:22:48.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 -- 172.21.15.31:0/3964048859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb940c8f40 msgr2=0x7fbb940c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:48.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb940c8f40 0x7fbb940c3010 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400c590 tx=0x7fbb84007e00 comp rx=0 tx=0).stop 2023-10-06T05:22:48.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 -- 172.21.15.31:0/3964048859 shutdown_connections 2023-10-06T05:22:48.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbb780670e0 0x7fbb78069590 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb9415ca40 0x7fbb9415b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb9406a300 0x7fbb9414f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 --2- 172.21.15.31:0/3964048859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb940c8f40 0x7fbb940c3010 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 -- 172.21.15.31:0/3964048859 >> 172.21.15.31:0/3964048859 conn(0x7fbb940ce290 msgr2=0x7fbb9415a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:48.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 -- 172.21.15.31:0/3964048859 shutdown_connections 2023-10-06T05:22:48.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.070+0000 7fbb9bbff700 1 -- 172.21.15.31:0/3964048859 wait complete. 2023-10-06T05:22:48.089 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000063418 with 8 files each 1MB size under directory volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1 2023-10-06T05:22:48.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:48.090 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T05:22:48.159 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:22:48.159 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:22:48.159 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173551 s, 60.4 MB/s 2023-10-06T05:22:48.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:48.160 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T05:22:48.223 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:22:48.223 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:22:48.223 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01565 s, 67.0 MB/s 2023-10-06T05:22:48.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:48.224 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T05:22:48.291 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:22:48.292 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:22:48.292 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173507 s, 60.4 MB/s 2023-10-06T05:22:48.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:48.293 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T05:22:48.355 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:22:48.355 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:22:48.355 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154348 s, 67.9 MB/s 2023-10-06T05:22:48.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:48.357 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T05:22:48.423 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:22:48.423 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:22:48.423 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188695 s, 55.6 MB/s 2023-10-06T05:22:48.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:48.427 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T05:22:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:22:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:22:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184782 s, 56.7 MB/s 2023-10-06T05:22:48.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:48.494 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T05:22:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:22:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:22:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154513 s, 67.9 MB/s 2023-10-06T05:22:48.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:48.559 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2023-10-06T05:22:48.623 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-10-06T05:22:48.623 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-10-06T05:22:48.623 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165489 s, 63.4 MB/s 2023-10-06T05:22:48.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvolume_0000000000063418 snapshot_0000000000055671 2023-10-06T05:22:48.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.823+0000 7f8aad9ff700 1 Processor -- start 2023-10-06T05:22:48.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.823+0000 7f8aad9ff700 1 -- start start 2023-10-06T05:22:48.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.824+0000 7f8aad9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8aa8156300 0x7f8aa80ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.824+0000 7f8aad9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8aa806ea90 0x7f8aa806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.824+0000 7f8aad9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa806a630 0x7f8aa806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.824+0000 7f8aad9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa806aff0 con 0x7f8aa8156300 2023-10-06T05:22:48.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.824+0000 7f8aad9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa806e600 con 0x7f8aa806ea90 2023-10-06T05:22:48.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.824+0000 7f8aad9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa806ef00 con 0x7f8aa806a630 2023-10-06T05:22:48.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.824+0000 7f8aa6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8aa8156300 0x7f8aa80ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.824+0000 7f8aa6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8aa8156300 0x7f8aa80ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59252/0 (socket says 172.21.15.31:59252) 2023-10-06T05:22:48.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.824+0000 7f8aa6ffd700 1 -- 172.21.15.31:0/1832686255 learned_addr learned my addr 172.21.15.31:0/1832686255 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:48.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.824+0000 7f8aa67fc700 1 --2- 172.21.15.31:0/1832686255 >> v2:172.21.15.62:3300/0 conn(0x7f8aa806ea90 0x7f8aa806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa77fe700 1 --2- 172.21.15.31:0/1832686255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa806a630 0x7f8aa806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa6ffd700 1 -- 172.21.15.31:0/1832686255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa806a630 msgr2=0x7f8aa806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:48.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa6ffd700 1 --2- 172.21.15.31:0/1832686255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa806a630 0x7f8aa806aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa6ffd700 1 -- 172.21.15.31:0/1832686255 >> v2:172.21.15.62:3300/0 conn(0x7f8aa806ea90 msgr2=0x7f8aa806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:48.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa6ffd700 1 --2- 172.21.15.31:0/1832686255 >> v2:172.21.15.62:3300/0 conn(0x7f8aa806ea90 0x7f8aa806d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa6ffd700 1 -- 172.21.15.31:0/1832686255 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa806d750 con 0x7f8aa8156300 2023-10-06T05:22:48.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa77fe700 1 --2- 172.21.15.31:0/1832686255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa806a630 0x7f8aa806aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:48.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa6ffd700 1 --2- 172.21.15.31:0/1832686255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa8156300 0x7f8aa80ad670 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80c3020 tx=0x7f8a98009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b84a7d77678be2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa5ffb700 1 -- 172.21.15.31:0/1832686255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98005c10 con 0x7f8aa8156300 2023-10-06T05:22:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa5ffb700 1 -- 172.21.15.31:0/1832686255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a98005d70 con 0x7f8aa8156300 2023-10-06T05:22:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.825+0000 7f8aa5ffb700 1 -- 172.21.15.31:0/1832686255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800b840 con 0x7f8aa8156300 2023-10-06T05:22:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.826+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1832686255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa8156300 msgr2=0x7f8aa80ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.826+0000 7f8aad9ff700 1 --2- 172.21.15.31:0/1832686255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa8156300 0x7f8aa80ad670 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80c3020 tx=0x7f8a98009220 comp rx=0 tx=0).stop 2023-10-06T05:22:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.826+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1832686255 shutdown_connections 2023-10-06T05:22:48.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.826+0000 7f8aad9ff700 1 --2- 172.21.15.31:0/1832686255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa806a630 0x7f8aa806aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.826+0000 7f8aad9ff700 1 --2- 172.21.15.31:0/1832686255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa8156300 0x7f8aa80ad670 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.826+0000 7f8aad9ff700 1 --2- 172.21.15.31:0/1832686255 >> v2:172.21.15.62:3300/0 conn(0x7f8aa806ea90 0x7f8aa806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.826+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1832686255 >> 172.21.15.31:0/1832686255 conn(0x7f8aa8068030 msgr2=0x7f8aa80cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:48.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.826+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1832686255 shutdown_connections 2023-10-06T05:22:48.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.826+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1832686255 wait complete. 2023-10-06T05:22:48.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.827+0000 7f8aad9ff700 1 Processor -- start 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.828+0000 7f8aad9ff700 1 -- start start 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.828+0000 7f8aad9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aa806a630 0x7f8aa8151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aad9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa806ea90 0x7f8aa8154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aad9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa8156300 0x7f8aa8154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aad9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa8158390 con 0x7f8aa806ea90 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aad9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa8158ed0 con 0x7f8aa806a630 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aad9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa81585a0 con 0x7f8aa8156300 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa806ea90 0x7f8aa8154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa806ea90 0x7f8aa8154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59262/0 (socket says 172.21.15.31:59262) 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa67fc700 1 -- 172.21.15.31:0/1316856981 learned_addr learned my addr 172.21.15.31:0/1316856981 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa6ffd700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aa806a630 0x7f8aa8151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa77fe700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa8156300 0x7f8aa8154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa67fc700 1 -- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa8156300 msgr2=0x7f8aa8154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:48.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa67fc700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa8156300 0x7f8aa8154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa67fc700 1 -- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aa806a630 msgr2=0x7f8aa8151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:48.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa67fc700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aa806a630 0x7f8aa8151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:48.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa67fc700 1 -- 172.21.15.31:0/1316856981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a98009480 con 0x7f8aa806ea90 2023-10-06T05:22:48.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa77fe700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa8156300 0x7f8aa8154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:48.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.829+0000 7f8aa67fc700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa806ea90 0x7f8aa8154630 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00d860 tx=0x7f8a9c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:48.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.830+0000 7f8a8effd700 1 -- 172.21.15.31:0/1316856981 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0125f0 con 0x7f8aa806ea90 2023-10-06T05:22:48.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.830+0000 7f8a8effd700 1 -- 172.21.15.31:0/1316856981 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c012750 con 0x7f8aa806ea90 2023-10-06T05:22:48.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.830+0000 7f8a8effd700 1 -- 172.21.15.31:0/1316856981 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c012b10 con 0x7f8aa806ea90 2023-10-06T05:22:48.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.830+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1316856981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa8158880 con 0x7f8aa806ea90 2023-10-06T05:22:48.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.830+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1316856981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa8159360 con 0x7f8aa806ea90 2023-10-06T05:22:48.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.831+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1316856981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa8159580 con 0x7f8aa806ea90 2023-10-06T05:22:48.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.832+0000 7f8a8effd700 1 -- 172.21.15.31:0/1316856981 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8a9c01e460 con 0x7f8aa806ea90 2023-10-06T05:22:48.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.833+0000 7f8a8effd700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8a940670e0 0x7f8a94069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:48.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.833+0000 7f8a8effd700 1 -- 172.21.15.31:0/1316856981 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8a9c084fe0 con 0x7f8aa806ea90 2023-10-06T05:22:48.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.833+0000 7f8aa6ffd700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8a940670e0 0x7f8a94069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:48.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.834+0000 7f8aa6ffd700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8a940670e0 0x7f8a94069590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009260 tx=0x7f8a98016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:48.837+0000 7f8a8effd700 1 -- 172.21.15.31:0/1316856981 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a9c051dd0 con 0x7f8aa806ea90 2023-10-06T05:22:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.074+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1316856981 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "snap_name": "snapshot_0000000000055671", "target": ["mon-mgr", ""]}) v1 -- 0x7f8aa804c380 con 0x7f8a940670e0 2023-10-06T05:22:49.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.118+0000 7f8a8effd700 1 -- 172.21.15.31:0/1316856981 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8aa804c380 con 0x7f8a940670e0 2023-10-06T05:22:49.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.120+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8a940670e0 msgr2=0x7f8a94069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.120+0000 7f8aad9ff700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8a940670e0 0x7f8a94069590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009260 tx=0x7f8a98016000 comp rx=0 tx=0).stop 2023-10-06T05:22:49.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.120+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1316856981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa806ea90 msgr2=0x7f8aa8154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.120+0000 7f8aad9ff700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa806ea90 0x7f8aa8154630 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00d860 tx=0x7f8a9c008040 comp rx=0 tx=0).stop 2023-10-06T05:22:49.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.121+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1316856981 shutdown_connections 2023-10-06T05:22:49.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.121+0000 7f8aad9ff700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8a940670e0 0x7f8a94069590 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.121+0000 7f8aad9ff700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8aa8156300 0x7f8aa8154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.121+0000 7f8aad9ff700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aa806a630 0x7f8aa8151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.121+0000 7f8aad9ff700 1 --2- 172.21.15.31:0/1316856981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aa806ea90 0x7f8aa8154630 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.121+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1316856981 >> 172.21.15.31:0/1316856981 conn(0x7f8aa8068030 msgr2=0x7f8aa80c9170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:49.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.121+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1316856981 shutdown_connections 2023-10-06T05:22:49.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.121+0000 7f8aad9ff700 1 -- 172.21.15.31:0/1316856981 wait complete. 2023-10-06T05:22:49.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvolume_0000000000063418 snapshot_0000000000055671 clone_0000000000030936 2023-10-06T05:22:49.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.343+0000 7f17acdff700 1 Processor -- start 2023-10-06T05:22:49.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.343+0000 7f17acdff700 1 -- start start 2023-10-06T05:22:49.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.344+0000 7f17acdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a8159930 0x7f17a80b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.344+0000 7f17acdff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a815d2d0 con 0x7f17a80cd580 2023-10-06T05:22:49.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.344+0000 7f17acdff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a815d5b0 con 0x7f17a8159560 2023-10-06T05:22:49.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.344+0000 7f17acdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a815d890 con 0x7f17a8159930 2023-10-06T05:22:49.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.344+0000 7f17a6d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f17a80cd580 0x7f17a80cb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51896/0 (socket says 172.21.15.31:51896) 2023-10-06T05:22:49.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.344+0000 7f17a6d9d700 1 -- 172.21.15.31:0/3526706365 learned_addr learned my addr 172.21.15.31:0/3526706365 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:49.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.344+0000 7f17a5d9b700 1 --2- 172.21.15.31:0/3526706365 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a8159930 0x7f17a80b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.345+0000 7f17a559a700 1 -- 172.21.15.31:0/3526706365 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166613860 0 0) 0x7f17a815d2d0 con 0x7f17a80cd580 2023-10-06T05:22:49.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.345+0000 7f17a559a700 1 -- 172.21.15.31:0/3526706365 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1790003570 con 0x7f17a80cd580 2023-10-06T05:22:49.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.345+0000 7f17a559a700 1 -- 172.21.15.31:0/3526706365 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478516443 0 0) 0x7f17a815d5b0 con 0x7f17a8159560 2023-10-06T05:22:49.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.345+0000 7f17a5d9b700 1 -- 172.21.15.31:0/3526706365 >> v1:172.21.15.62:6789/0 conn(0x7f17a8159560 legacy=0x7f17a80cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.345+0000 7f17a5d9b700 1 -- 172.21.15.31:0/3526706365 >> v1:172.21.15.31:6789/0 conn(0x7f17a80cd580 legacy=0x7f17a80cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.345+0000 7f17a5d9b700 1 -- 172.21.15.31:0/3526706365 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a815db70 con 0x7f17a8159930 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.345+0000 7f17a5d9b700 1 --2- 172.21.15.31:0/3526706365 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a8159930 0x7f17a80b2470 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f179c009910 tx=0x7f179c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d94a5d29dd5552d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.346+0000 7f17a559a700 1 -- 172.21.15.31:0/3526706365 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a00017f0 con 0x7f17a8159930 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.346+0000 7f17a559a700 1 -- 172.21.15.31:0/3526706365 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c011420 con 0x7f17a8159930 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.346+0000 7f17a559a700 1 -- 172.21.15.31:0/3526706365 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c0117c0 con 0x7f17a8159930 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.346+0000 7f17acdff700 1 -- 172.21.15.31:0/3526706365 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a8159930 msgr2=0x7f17a80b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.346+0000 7f17acdff700 1 --2- 172.21.15.31:0/3526706365 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a8159930 0x7f17a80b2470 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f179c009910 tx=0x7f179c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.346+0000 7f17acdff700 1 -- 172.21.15.31:0/3526706365 shutdown_connections 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.346+0000 7f17acdff700 1 --2- 172.21.15.31:0/3526706365 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a8159930 0x7f17a80b2470 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.346+0000 7f17acdff700 1 -- 172.21.15.31:0/3526706365 >> 172.21.15.31:0/3526706365 conn(0x7f17a806ec90 msgr2=0x7f17a806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.346+0000 7f17acdff700 1 -- 172.21.15.31:0/3526706365 shutdown_connections 2023-10-06T05:22:49.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.346+0000 7f17acdff700 1 -- 172.21.15.31:0/3526706365 wait complete. 2023-10-06T05:22:49.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.348+0000 7f17acdff700 1 Processor -- start 2023-10-06T05:22:49.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.348+0000 7f17acdff700 1 -- start start 2023-10-06T05:22:49.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17acdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a80cd580 0x7f17a81499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17acdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17a8159560 0x7f17a80d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17acdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17a8159930 0x7f17a80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17acdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a8149f10 con 0x7f17a8159560 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17acdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80c9ce0 con 0x7f17a8159930 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17acdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80c9fc0 con 0x7f17a80cd580 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17a8159560 0x7f17a80d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17a5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17a8159560 0x7f17a80d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59278/0 (socket says 172.21.15.31:59278) 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17a5d9b700 1 -- 172.21.15.31:0/4179611972 learned_addr learned my addr 172.21.15.31:0/4179611972 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17a659c700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a80cd580 0x7f17a81499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.349+0000 7f17a6d9d700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17a8159930 0x7f17a80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17a5d9b700 1 -- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a80cd580 msgr2=0x7f17a81499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17a5d9b700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a80cd580 0x7f17a81499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17a5d9b700 1 -- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17a8159930 msgr2=0x7f17a80d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17a5d9b700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17a8159930 0x7f17a80d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17a5d9b700 1 -- 172.21.15.31:0/4179611972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c009420 con 0x7f17a8159560 2023-10-06T05:22:49.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17a5d9b700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17a8159560 0x7f17a80d01e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f17a80893d0 tx=0x7f179c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17967fc700 1 -- 172.21.15.31:0/4179611972 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c010ac0 con 0x7f17a8159560 2023-10-06T05:22:49.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17967fc700 1 -- 172.21.15.31:0/4179611972 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c010c20 con 0x7f17a8159560 2023-10-06T05:22:49.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17967fc700 1 -- 172.21.15.31:0/4179611972 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c01d430 con 0x7f17a8159560 2023-10-06T05:22:49.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17acdff700 1 -- 172.21.15.31:0/4179611972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a80ca2a0 con 0x7f17a8159560 2023-10-06T05:22:49.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.350+0000 7f17acdff700 1 -- 172.21.15.31:0/4179611972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a80c4270 con 0x7f17a8159560 2023-10-06T05:22:49.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.351+0000 7f17acdff700 1 -- 172.21.15.31:0/4179611972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a80c4460 con 0x7f17a8159560 2023-10-06T05:22:49.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.352+0000 7f17967fc700 1 -- 172.21.15.31:0/4179611972 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f179c01d820 con 0x7f17a8159560 2023-10-06T05:22:49.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.353+0000 7f17967fc700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f17880670e0 0x7f1788069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.353+0000 7f17967fc700 1 -- 172.21.15.31:0/4179611972 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f179c0866a0 con 0x7f17a8159560 2023-10-06T05:22:49.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.353+0000 7f17a659c700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f17880670e0 0x7f1788069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.354+0000 7f17a659c700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f17880670e0 0x7f1788069590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1798007fd0 tx=0x7f1798007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.357+0000 7f17967fc700 1 -- 172.21.15.31:0/4179611972 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f179c053490 con 0x7f17a8159560 2023-10-06T05:22:49.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.601+0000 7f17acdff700 1 -- 172.21.15.31:0/4179611972 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "snap_name": "snapshot_0000000000055671", "target_sub_name": "clone_0000000000030936", "target": ["mon-mgr", ""]}) v1 -- 0x7f17a804c380 con 0x7f17880670e0 2023-10-06T05:22:49.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.686+0000 7f17967fc700 1 -- 172.21.15.31:0/4179611972 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f17a804c380 con 0x7f17880670e0 2023-10-06T05:22:49.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.688+0000 7f17acdff700 1 -- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f17880670e0 msgr2=0x7f1788069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.688+0000 7f17acdff700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f17880670e0 0x7f1788069590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1798007fd0 tx=0x7f1798007d30 comp rx=0 tx=0).stop 2023-10-06T05:22:49.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.688+0000 7f17acdff700 1 -- 172.21.15.31:0/4179611972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17a8159560 msgr2=0x7f17a80d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.688+0000 7f17acdff700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17a8159560 0x7f17a80d01e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f17a80893d0 tx=0x7f179c017000 comp rx=0 tx=0).stop 2023-10-06T05:22:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.689+0000 7f17acdff700 1 -- 172.21.15.31:0/4179611972 shutdown_connections 2023-10-06T05:22:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.689+0000 7f17acdff700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f17880670e0 0x7f1788069590 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.689+0000 7f17acdff700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17a8159930 0x7f17a80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.689+0000 7f17acdff700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f17a80cd580 0x7f17a81499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.689+0000 7f17acdff700 1 --2- 172.21.15.31:0/4179611972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17a8159560 0x7f17a80d01e0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.689+0000 7f17acdff700 1 -- 172.21.15.31:0/4179611972 >> 172.21.15.31:0/4179611972 conn(0x7f17a806ec90 msgr2=0x7f17a80ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.689+0000 7f17acdff700 1 -- 172.21.15.31:0/4179611972 shutdown_connections 2023-10-06T05:22:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.689+0000 7f17acdff700 1 -- 172.21.15.31:0/4179611972 wait complete. 2023-10-06T05:22:49.696 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:49.691+0000 7f2e48986700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:49.696 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:49.691+0000 7f2e48986700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:49.696 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:49.691+0000 7f2e48986700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:49.696 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:49.691+0000 7f2e48986700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:49.696 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:49.691+0000 7f2e48986700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:49.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs clone_0000000000030936 2023-10-06T05:22:49.712 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:49.710+0000 7f2e49988700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:49.712 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:49.710+0000 7f2e49988700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:49.712 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:49.710+0000 7f2e49988700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:49.712 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:49.710+0000 7f2e49988700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:49.712 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:22:49.710+0000 7f2e49988700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:22:49.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.918+0000 7f8c431ff700 1 Processor -- start 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.919+0000 7f8c431ff700 1 -- start start 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c431ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8c3c159890 0x7f8c3c0c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c431ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c3c159c60 0x7f8c3c156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c431ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8c3c156fa0 0x7f8c3c155430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c431ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c158190 con 0x7f8c3c156fa0 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c431ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c157580 con 0x7f8c3c159890 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c431ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c157860 con 0x7f8c3c159c60 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c4179c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8c3c156fa0 0x7f8c3c155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c4179c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8c3c156fa0 0x7f8c3c155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59284/0 (socket says 172.21.15.31:59284) 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c3bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c3c159c60 0x7f8c3c156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c4179c700 1 -- 172.21.15.31:0/3946682186 learned_addr learned my addr 172.21.15.31:0/3946682186 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:49.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.920+0000 7f8c40f9b700 1 --2- 172.21.15.31:0/3946682186 >> v2:172.21.15.62:3300/0 conn(0x7f8c3c159890 0x7f8c3c0c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.921+0000 7f8c4179c700 1 -- 172.21.15.31:0/3946682186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c3c159c60 msgr2=0x7f8c3c156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:49.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.921+0000 7f8c4179c700 1 --2- 172.21.15.31:0/3946682186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c3c159c60 0x7f8c3c156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.921+0000 7f8c4179c700 1 -- 172.21.15.31:0/3946682186 >> v2:172.21.15.62:3300/0 conn(0x7f8c3c159890 msgr2=0x7f8c3c0c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:49.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.921+0000 7f8c4179c700 1 --2- 172.21.15.31:0/3946682186 >> v2:172.21.15.62:3300/0 conn(0x7f8c3c159890 0x7f8c3c0c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.921+0000 7f8c4179c700 1 -- 172.21.15.31:0/3946682186 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c157b40 con 0x7f8c3c156fa0 2023-10-06T05:22:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.921+0000 7f8c4179c700 1 --2- 172.21.15.31:0/3946682186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c156fa0 0x7f8c3c155430 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009f20 tx=0x7f8c34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b84c6c2a6190450 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.921+0000 7f8c3b7fe700 1 -- 172.21.15.31:0/3946682186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400e070 con 0x7f8c3c156fa0 2023-10-06T05:22:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c3b7fe700 1 -- 172.21.15.31:0/3946682186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3400b930 con 0x7f8c3c156fa0 2023-10-06T05:22:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c3b7fe700 1 -- 172.21.15.31:0/3946682186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c340092e0 con 0x7f8c3c156fa0 2023-10-06T05:22:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c431ff700 1 -- 172.21.15.31:0/3946682186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c156fa0 msgr2=0x7f8c3c155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c431ff700 1 --2- 172.21.15.31:0/3946682186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c156fa0 0x7f8c3c155430 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009f20 tx=0x7f8c34009840 comp rx=0 tx=0).stop 2023-10-06T05:22:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c431ff700 1 -- 172.21.15.31:0/3946682186 shutdown_connections 2023-10-06T05:22:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c431ff700 1 --2- 172.21.15.31:0/3946682186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c156fa0 0x7f8c3c155430 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c431ff700 1 --2- 172.21.15.31:0/3946682186 >> v2:172.21.15.62:3300/0 conn(0x7f8c3c159890 0x7f8c3c0c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c431ff700 1 --2- 172.21.15.31:0/3946682186 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c3c159c60 0x7f8c3c156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c431ff700 1 -- 172.21.15.31:0/3946682186 >> 172.21.15.31:0/3946682186 conn(0x7f8c3c067560 msgr2=0x7f8c3c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c431ff700 1 -- 172.21.15.31:0/3946682186 shutdown_connections 2023-10-06T05:22:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.922+0000 7f8c431ff700 1 -- 172.21.15.31:0/3946682186 wait complete. 2023-10-06T05:22:49.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.924+0000 7f8c431ff700 1 Processor -- start 2023-10-06T05:22:49.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.925+0000 7f8c431ff700 1 -- start start 2023-10-06T05:22:49.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c431ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c3c156fa0 0x7f8c3c15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c431ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c159890 0x7f8c3c152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c431ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c3c159c60 0x7f8c3c146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c431ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c152560 con 0x7f8c3c159890 2023-10-06T05:22:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c431ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c151c00 con 0x7f8c3c156fa0 2023-10-06T05:22:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c431ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c15b1f0 con 0x7f8c3c159c60 2023-10-06T05:22:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c3bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c159890 0x7f8c3c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c3bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c159890 0x7f8c3c152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59286/0 (socket says 172.21.15.31:59286) 2023-10-06T05:22:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c3bfff700 1 -- 172.21.15.31:0/3718382473 learned_addr learned my addr 172.21.15.31:0/3718382473 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c40f9b700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c3c156fa0 0x7f8c3c15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.926+0000 7f8c4179c700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c3c159c60 0x7f8c3c146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c3bfff700 1 -- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c3c159c60 msgr2=0x7f8c3c146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c3bfff700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c3c159c60 0x7f8c3c146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c3bfff700 1 -- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c3c156fa0 msgr2=0x7f8c3c15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c3bfff700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c3c156fa0 0x7f8c3c15c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c3bfff700 1 -- 172.21.15.31:0/3718382473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c34009ae0 con 0x7f8c3c159890 2023-10-06T05:22:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c3bfff700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c159890 0x7f8c3c152020 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800e950 tx=0x7f8c280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c38ff9700 1 -- 172.21.15.31:0/3718382473 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28013680 con 0x7f8c3c159890 2023-10-06T05:22:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c38ff9700 1 -- 172.21.15.31:0/3718382473 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c280137e0 con 0x7f8c3c159890 2023-10-06T05:22:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c431ff700 1 -- 172.21.15.31:0/3718382473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c15b4d0 con 0x7f8c3c159890 2023-10-06T05:22:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c38ff9700 1 -- 172.21.15.31:0/3718382473 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28013ba0 con 0x7f8c3c159890 2023-10-06T05:22:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.927+0000 7f8c431ff700 1 -- 172.21.15.31:0/3718382473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c15ba90 con 0x7f8c3c159890 2023-10-06T05:22:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.928+0000 7f8c431ff700 1 -- 172.21.15.31:0/3718382473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c04c380 con 0x7f8c3c159890 2023-10-06T05:22:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.929+0000 7f8c38ff9700 1 -- 172.21.15.31:0/3718382473 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8c280123d0 con 0x7f8c3c159890 2023-10-06T05:22:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.930+0000 7f8c38ff9700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c200670e0 0x7f8c20069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.930+0000 7f8c38ff9700 1 -- 172.21.15.31:0/3718382473 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8c28084b40 con 0x7f8c3c159890 2023-10-06T05:22:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.930+0000 7f8c40f9b700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c200670e0 0x7f8c20069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:49.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.931+0000 7f8c40f9b700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c200670e0 0x7f8c20069590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009db0 tx=0x7f8c300091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:49.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:49.934+0000 7f8c38ff9700 1 -- 172.21.15.31:0/3718382473 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c28051930 con 0x7f8c3c159890 2023-10-06T05:22:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.181+0000 7f8c431ff700 1 -- 172.21.15.31:0/3718382473 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "clone_0000000000030936", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c3c157e20 con 0x7f8c200670e0 2023-10-06T05:22:50.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.996+0000 7f8c38ff9700 1 -- 172.21.15.31:0/3718382473 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f8c3c157e20 con 0x7f8c200670e0 2023-10-06T05:22:50.998 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:22:50.998 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-10-06T05:22:50.998 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete" 2023-10-06T05:22:50.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:22:50.999 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:22:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 -- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c200670e0 msgr2=0x7f8c20069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c200670e0 0x7f8c20069590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009db0 tx=0x7f8c300091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 -- 172.21.15.31:0/3718382473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c159890 msgr2=0x7f8c3c152020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c159890 0x7f8c3c152020 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800e950 tx=0x7f8c280091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 -- 172.21.15.31:0/3718382473 shutdown_connections 2023-10-06T05:22:51.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c200670e0 0x7f8c20069590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c3c159c60 0x7f8c3c146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c3c156fa0 0x7f8c3c15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 --2- 172.21.15.31:0/3718382473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c159890 0x7f8c3c152020 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 -- 172.21.15.31:0/3718382473 >> 172.21.15.31:0/3718382473 conn(0x7f8c3c067560 msgr2=0x7f8c3c067c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:51.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 -- 172.21.15.31:0/3718382473 shutdown_connections 2023-10-06T05:22:51.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:50.999+0000 7f8c431ff700 1 -- 172.21.15.31:0/3718382473 wait complete. 2023-10-06T05:22:51.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:51.022 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/clone_0000000000030936/.meta) 2023-10-06T05:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout:version = 2 2023-10-06T05:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout:type = clone 2023-10-06T05:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b 2023-10-06T05:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:22:51.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000063418 2023-10-06T05:22:51.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.304+0000 7f55fddff700 1 Processor -- start 2023-10-06T05:22:51.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.304+0000 7f55fddff700 1 -- start start 2023-10-06T05:22:51.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55fddff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55f80c8f40 0x7f55f8068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55fddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f806a300 0x7f55f80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55fddff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f8068da0 con 0x7f55f815ca40 2023-10-06T05:22:51.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55fddff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f80cabc0 con 0x7f55f80c8f40 2023-10-06T05:22:51.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55fddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8068bb0 con 0x7f55f806a300 2023-10-06T05:22:51.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55f80c8f40 0x7f55f8068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55f80c8f40 0x7f55f8068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59302/0 (socket says 172.21.15.31:59302) 2023-10-06T05:22:51.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f6ffd700 1 -- 172.21.15.31:0/4263803649 learned_addr learned my addr 172.21.15.31:0/4263803649 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:51.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f77fe700 1 --1- 172.21.15.31:0/4263803649 >> v1:172.21.15.31:6789/0 conn(0x7f55f815ca40 0x7f55f80c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51908/0 (socket says 172.21.15.31:51908) 2023-10-06T05:22:51.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f7fff700 1 --2- 172.21.15.31:0/4263803649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f806a300 0x7f55f80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f67fc700 1 -- 172.21.15.31:0/4263803649 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564701369 0 0) 0x7f55f8068da0 con 0x7f55f815ca40 2023-10-06T05:22:51.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f67fc700 1 -- 172.21.15.31:0/4263803649 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55e0003570 con 0x7f55f815ca40 2023-10-06T05:22:51.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f6ffd700 1 -- 172.21.15.31:0/4263803649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f806a300 msgr2=0x7f55f80adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:51.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f6ffd700 1 --2- 172.21.15.31:0/4263803649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f806a300 0x7f55f80adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f6ffd700 1 -- 172.21.15.31:0/4263803649 >> v1:172.21.15.31:6789/0 conn(0x7f55f815ca40 legacy=0x7f55f80c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.306+0000 7f55f6ffd700 1 -- 172.21.15.31:0/4263803649 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f8069080 con 0x7f55f80c8f40 2023-10-06T05:22:51.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.307+0000 7f55f6ffd700 1 --2- 172.21.15.31:0/4263803649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f80c8f40 0x7f55f8068670 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0099b0 tx=0x7f55ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb7157f7ceee1a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.307+0000 7f55f67fc700 1 -- 172.21.15.31:0/4263803649 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00b680 con 0x7f55f80c8f40 2023-10-06T05:22:51.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.307+0000 7f55f67fc700 1 -- 172.21.15.31:0/4263803649 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec00b7e0 con 0x7f55f80c8f40 2023-10-06T05:22:51.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.307+0000 7f55f67fc700 1 -- 172.21.15.31:0/4263803649 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00bb80 con 0x7f55f80c8f40 2023-10-06T05:22:51.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.307+0000 7f55fddff700 1 -- 172.21.15.31:0/4263803649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f80c8f40 msgr2=0x7f55f8068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.307+0000 7f55fddff700 1 --2- 172.21.15.31:0/4263803649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f80c8f40 0x7f55f8068670 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0099b0 tx=0x7f55ec009250 comp rx=0 tx=0).stop 2023-10-06T05:22:51.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.308+0000 7f55fddff700 1 -- 172.21.15.31:0/4263803649 shutdown_connections 2023-10-06T05:22:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.308+0000 7f55fddff700 1 --2- 172.21.15.31:0/4263803649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f806a300 0x7f55f80adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.308+0000 7f55fddff700 1 --2- 172.21.15.31:0/4263803649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f80c8f40 0x7f55f8068670 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.308+0000 7f55fddff700 1 -- 172.21.15.31:0/4263803649 >> 172.21.15.31:0/4263803649 conn(0x7f55f80ce290 msgr2=0x7f55f80ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.308+0000 7f55fddff700 1 -- 172.21.15.31:0/4263803649 shutdown_connections 2023-10-06T05:22:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.308+0000 7f55fddff700 1 -- 172.21.15.31:0/4263803649 wait complete. 2023-10-06T05:22:51.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.309+0000 7f55fddff700 1 Processor -- start 2023-10-06T05:22:51.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.309+0000 7f55fddff700 1 -- start start 2023-10-06T05:22:51.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.310+0000 7f55fddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f806a300 0x7f55f814c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.310+0000 7f55fddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f80c8f40 0x7f55f80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.310+0000 7f55fddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f815ca40 0x7f55f80c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.310+0000 7f55fddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80b34c0 con 0x7f55f815ca40 2023-10-06T05:22:51.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.310+0000 7f55fddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80b27e0 con 0x7f55f806a300 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.310+0000 7f55fddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80b2ac0 con 0x7f55f80c8f40 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.310+0000 7f55f7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f815ca40 0x7f55f80c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.310+0000 7f55f7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f815ca40 0x7f55f80c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59306/0 (socket says 172.21.15.31:59306) 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.310+0000 7f55f7fff700 1 -- 172.21.15.31:0/2054244373 learned_addr learned my addr 172.21.15.31:0/2054244373 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55f6ffd700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f80c8f40 0x7f55f80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55f77fe700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f806a300 0x7f55f814c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55f7fff700 1 -- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f80c8f40 msgr2=0x7f55f80b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55f7fff700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f80c8f40 0x7f55f80b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55f7fff700 1 -- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f806a300 msgr2=0x7f55f814c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55f7fff700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f806a300 0x7f55f814c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55f7fff700 1 -- 172.21.15.31:0/2054244373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec0094f0 con 0x7f55f815ca40 2023-10-06T05:22:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55f6ffd700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f80c8f40 0x7f55f80b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55f77fe700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f806a300 0x7f55f814c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55f7fff700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f815ca40 0x7f55f80c52a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009e90 tx=0x7f55f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55df7fe700 1 -- 172.21.15.31:0/2054244373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f000cc20 con 0x7f55f815ca40 2023-10-06T05:22:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55df7fe700 1 -- 172.21.15.31:0/2054244373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f000f040 con 0x7f55f815ca40 2023-10-06T05:22:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55df7fe700 1 -- 172.21.15.31:0/2054244373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0014480 con 0x7f55f815ca40 2023-10-06T05:22:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.311+0000 7f55fddff700 1 -- 172.21.15.31:0/2054244373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f80b3150 con 0x7f55f815ca40 2023-10-06T05:22:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.312+0000 7f55fddff700 1 -- 172.21.15.31:0/2054244373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f80b3e90 con 0x7f55f815ca40 2023-10-06T05:22:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.313+0000 7f55fddff700 1 -- 172.21.15.31:0/2054244373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f80b3630 con 0x7f55f815ca40 2023-10-06T05:22:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.313+0000 7f55df7fe700 1 -- 172.21.15.31:0/2054244373 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f55f0014830 con 0x7f55f815ca40 2023-10-06T05:22:51.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.314+0000 7f55df7fe700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55e4067230 0x7f55e40696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.314+0000 7f55df7fe700 1 -- 172.21.15.31:0/2054244373 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f55f00855f0 con 0x7f55f815ca40 2023-10-06T05:22:51.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.315+0000 7f55f77fe700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55e4067230 0x7f55e40696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.315+0000 7f55f77fe700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55e4067230 0x7f55e40696e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005f00 tx=0x7f55e8007c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.319+0000 7f55df7fe700 1 -- 172.21.15.31:0/2054244373 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55f0055d30 con 0x7f55f815ca40 2023-10-06T05:22:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.557+0000 7f55fddff700 1 -- 172.21.15.31:0/2054244373 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "target": ["mon-mgr", ""]}) v1 -- 0x7f55f804c380 con 0x7f55e4067230 2023-10-06T05:22:51.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.575+0000 7f55df7fe700 1 -- 172.21.15.31:0/2054244373 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f55f804c380 con 0x7f55e4067230 2023-10-06T05:22:51.577 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1 2023-10-06T05:22:51.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.577+0000 7f55fddff700 1 -- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55e4067230 msgr2=0x7f55e40696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.577+0000 7f55fddff700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55e4067230 0x7f55e40696e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005f00 tx=0x7f55e8007c60 comp rx=0 tx=0).stop 2023-10-06T05:22:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.577+0000 7f55fddff700 1 -- 172.21.15.31:0/2054244373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f815ca40 msgr2=0x7f55f80c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.577+0000 7f55fddff700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f815ca40 0x7f55f80c52a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009e90 tx=0x7f55f000b040 comp rx=0 tx=0).stop 2023-10-06T05:22:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.578+0000 7f55fddff700 1 -- 172.21.15.31:0/2054244373 shutdown_connections 2023-10-06T05:22:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.578+0000 7f55fddff700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55e4067230 0x7f55e40696e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.578+0000 7f55fddff700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55f815ca40 0x7f55f80c52a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.578+0000 7f55fddff700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f806a300 0x7f55f814c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.578+0000 7f55fddff700 1 --2- 172.21.15.31:0/2054244373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55f80c8f40 0x7f55f80b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.578+0000 7f55fddff700 1 -- 172.21.15.31:0/2054244373 >> 172.21.15.31:0/2054244373 conn(0x7f55f80ce290 msgr2=0x7f55f80cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.578+0000 7f55fddff700 1 -- 172.21.15.31:0/2054244373 shutdown_connections 2023-10-06T05:22:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.578+0000 7f55fddff700 1 -- 172.21.15.31:0/2054244373 wait complete. 2023-10-06T05:22:51.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs clone_0000000000030936 2023-10-06T05:22:51.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.792+0000 7f3126bff700 1 Processor -- start 2023-10-06T05:22:51.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.792+0000 7f3126bff700 1 -- start start 2023-10-06T05:22:51.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.794+0000 7f3126bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3120159890 0x7f31200c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.794+0000 7f3126bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3120159c60 0x7f3120156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.794+0000 7f3126bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3120156fa0 0x7f3120155430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.794+0000 7f3126bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3120158190 con 0x7f3120156fa0 2023-10-06T05:22:51.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.794+0000 7f3126bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3120157580 con 0x7f3120159890 2023-10-06T05:22:51.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.794+0000 7f3126bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3120157860 con 0x7f3120159c60 2023-10-06T05:22:51.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.794+0000 7f312519c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3120156fa0 0x7f3120155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.794+0000 7f312499b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3120159890 0x7f31200c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f312519c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3120156fa0 0x7f3120155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59316/0 (socket says 172.21.15.31:59316) 2023-10-06T05:22:51.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f312519c700 1 -- 172.21.15.31:0/44878915 learned_addr learned my addr 172.21.15.31:0/44878915 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:51.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f311ffff700 1 --2- 172.21.15.31:0/44878915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3120159c60 0x7f3120156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f312519c700 1 -- 172.21.15.31:0/44878915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3120159c60 msgr2=0x7f3120156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:51.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f312519c700 1 --2- 172.21.15.31:0/44878915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3120159c60 0x7f3120156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f312519c700 1 -- 172.21.15.31:0/44878915 >> v2:172.21.15.62:3300/0 conn(0x7f3120159890 msgr2=0x7f31200c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:51.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f312519c700 1 --2- 172.21.15.31:0/44878915 >> v2:172.21.15.62:3300/0 conn(0x7f3120159890 0x7f31200c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f312519c700 1 -- 172.21.15.31:0/44878915 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120157b40 con 0x7f3120156fa0 2023-10-06T05:22:51.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f312499b700 1 --2- 172.21.15.31:0/44878915 >> v2:172.21.15.62:3300/0 conn(0x7f3120159890 0x7f31200c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:22:51.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f312519c700 1 --2- 172.21.15.31:0/44878915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120156fa0 0x7f3120155430 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3118009f20 tx=0x7f3118009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f0f4bfc6e65b2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.795+0000 7f311f7fe700 1 -- 172.21.15.31:0/44878915 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180092e0 con 0x7f3120156fa0 2023-10-06T05:22:51.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f311f7fe700 1 -- 172.21.15.31:0/44878915 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311800b930 con 0x7f3120156fa0 2023-10-06T05:22:51.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f311f7fe700 1 -- 172.21.15.31:0/44878915 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311800bcf0 con 0x7f3120156fa0 2023-10-06T05:22:51.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f3126bff700 1 -- 172.21.15.31:0/44878915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120156fa0 msgr2=0x7f3120155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f3126bff700 1 --2- 172.21.15.31:0/44878915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120156fa0 0x7f3120155430 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3118009f20 tx=0x7f3118009840 comp rx=0 tx=0).stop 2023-10-06T05:22:51.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f3126bff700 1 -- 172.21.15.31:0/44878915 shutdown_connections 2023-10-06T05:22:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f3126bff700 1 --2- 172.21.15.31:0/44878915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120156fa0 0x7f3120155430 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f3126bff700 1 --2- 172.21.15.31:0/44878915 >> v2:172.21.15.62:3300/0 conn(0x7f3120159890 0x7f31200c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f3126bff700 1 --2- 172.21.15.31:0/44878915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3120159c60 0x7f3120156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f3126bff700 1 -- 172.21.15.31:0/44878915 >> 172.21.15.31:0/44878915 conn(0x7f3120067560 msgr2=0x7f3120067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f3126bff700 1 -- 172.21.15.31:0/44878915 shutdown_connections 2023-10-06T05:22:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.796+0000 7f3126bff700 1 -- 172.21.15.31:0/44878915 wait complete. 2023-10-06T05:22:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.798+0000 7f3126bff700 1 Processor -- start 2023-10-06T05:22:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.798+0000 7f3126bff700 1 -- start start 2023-10-06T05:22:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f3126bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3120156fa0 0x7f31200ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f3126bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3120159890 0x7f31200a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f3126bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120159c60 0x7f31201691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f3126bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31200aac90 con 0x7f3120159c60 2023-10-06T05:22:51.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f3126bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31200a9de0 con 0x7f3120156fa0 2023-10-06T05:22:51.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f3126bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31200aa080 con 0x7f3120159890 2023-10-06T05:22:51.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f312519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120159c60 0x7f31201691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f312519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120159c60 0x7f31201691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59330/0 (socket says 172.21.15.31:59330) 2023-10-06T05:22:51.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f312519c700 1 -- 172.21.15.31:0/2733173717 learned_addr learned my addr 172.21.15.31:0/2733173717 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:51.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f312499b700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3120156fa0 0x7f31200ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f311ffff700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3120159890 0x7f31200a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f312519c700 1 -- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3120159890 msgr2=0x7f31200a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f312519c700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3120159890 0x7f31200a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f312519c700 1 -- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3120156fa0 msgr2=0x7f31200ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f312519c700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3120156fa0 0x7f31200ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.799+0000 7f312519c700 1 -- 172.21.15.31:0/2733173717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3114009420 con 0x7f3120159c60 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.800+0000 7f312519c700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120159c60 0x7f31201691d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3118009ab0 tx=0x7f311801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.800+0000 7f311cff9700 1 -- 172.21.15.31:0/2733173717 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118011ea0 con 0x7f3120159c60 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.800+0000 7f311cff9700 1 -- 172.21.15.31:0/2733173717 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31180032f0 con 0x7f3120159c60 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.800+0000 7f311cff9700 1 -- 172.21.15.31:0/2733173717 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311801a8b0 con 0x7f3120159c60 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.800+0000 7f3126bff700 1 -- 172.21.15.31:0/2733173717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3118009ae0 con 0x7f3120159c60 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.800+0000 7f3126bff700 1 -- 172.21.15.31:0/2733173717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31200aa640 con 0x7f3120159c60 2023-10-06T05:22:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.801+0000 7f3126bff700 1 -- 172.21.15.31:0/2733173717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31200aae00 con 0x7f3120159c60 2023-10-06T05:22:51.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.802+0000 7f311cff9700 1 -- 172.21.15.31:0/2733173717 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f3118003040 con 0x7f3120159c60 2023-10-06T05:22:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.803+0000 7f311cff9700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3104067200 0x7f31040696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.803+0000 7f311cff9700 1 -- 172.21.15.31:0/2733173717 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f31180867e0 con 0x7f3120159c60 2023-10-06T05:22:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.804+0000 7f312499b700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3104067200 0x7f31040696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:51.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.804+0000 7f312499b700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3104067200 0x7f31040696b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3114009f40 tx=0x7f31140091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:51.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:51.807+0000 7f311cff9700 1 -- 172.21.15.31:0/2733173717 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3118053450 con 0x7f3120159c60 2023-10-06T05:22:52.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.045+0000 7f3126bff700 1 -- 172.21.15.31:0/2733173717 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "clone_0000000000030936", "target": ["mon-mgr", ""]}) v1 -- 0x7f312004c380 con 0x7f3104067200 2023-10-06T05:22:52.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.053+0000 7f311cff9700 1 -- 172.21.15.31:0/2733173717 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+78 (secure 0 0 0) 0x7f312004c380 con 0x7f3104067200 2023-10-06T05:22:52.055 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b 2023-10-06T05:22:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 -- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3104067200 msgr2=0x7f31040696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3104067200 0x7f31040696b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3114009f40 tx=0x7f31140091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 -- 172.21.15.31:0/2733173717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120159c60 msgr2=0x7f31201691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120159c60 0x7f31201691d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3118009ab0 tx=0x7f311801a000 comp rx=0 tx=0).stop 2023-10-06T05:22:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 -- 172.21.15.31:0/2733173717 shutdown_connections 2023-10-06T05:22:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3104067200 0x7f31040696b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3120159c60 0x7f31201691d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3120156fa0 0x7f31200ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 --2- 172.21.15.31:0/2733173717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3120159890 0x7f31200a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 -- 172.21.15.31:0/2733173717 >> 172.21.15.31:0/2733173717 conn(0x7f3120067560 msgr2=0x7f31201555d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 -- 172.21.15.31:0/2733173717 shutdown_connections 2023-10-06T05:22:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.056+0000 7f3126bff700 1 -- 172.21.15.31:0/2733173717 wait complete. 2023-10-06T05:22:52.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:52.075 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671) 2023-10-06T05:22:52.125 INFO:teuthology.orchestra.run.smithi031.stdout:9 2023-10-06T05:22:52.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:52.125 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b) 2023-10-06T05:22:52.177 INFO:teuthology.orchestra.run.smithi031.stdout:9 2023-10-06T05:22:52.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs clone_0000000000030936 2023-10-06T05:22:52.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.374+0000 7f502ffff700 1 Processor -- start 2023-10-06T05:22:52.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.374+0000 7f502ffff700 1 -- start start 2023-10-06T05:22:52.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 0x7f50280cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5028159930 0x7f50280cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f50280b0d60 0x7f50280b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502ffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502815d2d0 con 0x7f5028159930 2023-10-06T05:22:52.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502ffff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502815d5b0 con 0x7f50280b0d60 2023-10-06T05:22:52.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502815d890 con 0x7f5028159560 2023-10-06T05:22:52.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502d59a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5028159930 0x7f50280cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502d59a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5028159930 0x7f50280cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59342/0 (socket says 172.21.15.31:59342) 2023-10-06T05:22:52.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502d59a700 1 -- 172.21.15.31:0/2365664132 learned_addr learned my addr 172.21.15.31:0/2365664132 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:52.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502dd9b700 1 --2- 172.21.15.31:0/2365664132 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 0x7f50280cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502e59c700 1 --2- 172.21.15.31:0/2365664132 >> v2:172.21.15.62:3300/0 conn(0x7f50280b0d60 0x7f50280b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502d59a700 1 -- 172.21.15.31:0/2365664132 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 msgr2=0x7f50280cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:52.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.376+0000 7f502d59a700 1 --2- 172.21.15.31:0/2365664132 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 0x7f50280cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.377+0000 7f502d59a700 1 -- 172.21.15.31:0/2365664132 >> v2:172.21.15.62:3300/0 conn(0x7f50280b0d60 msgr2=0x7f50280b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:22:52.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.377+0000 7f502d59a700 1 --2- 172.21.15.31:0/2365664132 >> v2:172.21.15.62:3300/0 conn(0x7f50280b0d60 0x7f50280b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.377+0000 7f502d59a700 1 -- 172.21.15.31:0/2365664132 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502815db70 con 0x7f5028159930 2023-10-06T05:22:52.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.377+0000 7f502e59c700 1 --2- 172.21.15.31:0/2365664132 >> v2:172.21.15.62:3300/0 conn(0x7f50280b0d60 0x7f50280b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:52.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.377+0000 7f502d59a700 1 --2- 172.21.15.31:0/2365664132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5028159930 0x7f50280cd3b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f502806fbc0 tx=0x7f5018009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596a4346b92629d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.377+0000 7f502cd99700 1 -- 172.21.15.31:0/2365664132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018009e70 con 0x7f5028159930 2023-10-06T05:22:52.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.377+0000 7f502cd99700 1 -- 172.21.15.31:0/2365664132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501800b860 con 0x7f5028159930 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.377+0000 7f502cd99700 1 -- 172.21.15.31:0/2365664132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501800bc00 con 0x7f5028159930 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.378+0000 7f502ffff700 1 -- 172.21.15.31:0/2365664132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5028159930 msgr2=0x7f50280cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.378+0000 7f502ffff700 1 --2- 172.21.15.31:0/2365664132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5028159930 0x7f50280cd3b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f502806fbc0 tx=0x7f5018009250 comp rx=0 tx=0).stop 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.378+0000 7f502ffff700 1 -- 172.21.15.31:0/2365664132 shutdown_connections 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.378+0000 7f502ffff700 1 --2- 172.21.15.31:0/2365664132 >> v2:172.21.15.62:3300/0 conn(0x7f50280b0d60 0x7f50280b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.378+0000 7f502ffff700 1 --2- 172.21.15.31:0/2365664132 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 0x7f50280cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.378+0000 7f502ffff700 1 --2- 172.21.15.31:0/2365664132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5028159930 0x7f50280cd3b0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.378+0000 7f502ffff700 1 -- 172.21.15.31:0/2365664132 >> 172.21.15.31:0/2365664132 conn(0x7f502806ec90 msgr2=0x7f502806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.378+0000 7f502ffff700 1 -- 172.21.15.31:0/2365664132 shutdown_connections 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.378+0000 7f502ffff700 1 -- 172.21.15.31:0/2365664132 wait complete. 2023-10-06T05:22:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.379+0000 7f502ffff700 1 Processor -- start 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.379+0000 7f502ffff700 1 -- start start 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.380+0000 7f502ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50280b0d60 0x7f502814c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.380+0000 7f502ffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 0x7f50280c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.380+0000 7f502ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5028159930 0x7f502815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.380+0000 7f502ffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280c48a0 con 0x7f50280b0d60 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.380+0000 7f502ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280c56b0 con 0x7f5028159930 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.380+0000 7f502ffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502814ce30 con 0x7f5028159560 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50280b0d60 0x7f502814c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50280b0d60 0x7f502814c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59350/0 (socket says 172.21.15.31:59350) 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502dd9b700 1 -- 172.21.15.31:0/2477944818 learned_addr learned my addr 172.21.15.31:0/2477944818 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502e59c700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5028159930 0x7f502815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502dd9b700 1 -- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 msgr2=0x7f50280c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502d59a700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 0x7f50280c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502dd9b700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 0x7f50280c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502dd9b700 1 -- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5028159930 msgr2=0x7f502815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502dd9b700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5028159930 0x7f502815b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502dd9b700 1 -- 172.21.15.31:0/2477944818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50180094f0 con 0x7f50280b0d60 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502e59c700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5028159930 0x7f502815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502d59a700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 0x7f50280c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.381+0000 7f502dd9b700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50280b0d60 0x7f502814c8f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f502400ec80 tx=0x7f50240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.382+0000 7f501dffb700 1 -- 172.21.15.31:0/2477944818 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502400c9d0 con 0x7f50280b0d60 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.382+0000 7f501dffb700 1 -- 172.21.15.31:0/2477944818 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502400cb50 con 0x7f50280b0d60 2023-10-06T05:22:52.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.382+0000 7f501dffb700 1 -- 172.21.15.31:0/2477944818 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024013470 con 0x7f50280b0d60 2023-10-06T05:22:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.382+0000 7f502ffff700 1 -- 172.21.15.31:0/2477944818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50280c5b40 con 0x7f50280b0d60 2023-10-06T05:22:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.382+0000 7f502ffff700 1 -- 172.21.15.31:0/2477944818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50280c4aa0 con 0x7f50280b0d60 2023-10-06T05:22:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.383+0000 7f502ffff700 1 -- 172.21.15.31:0/2477944818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502804c380 con 0x7f50280b0d60 2023-10-06T05:22:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.384+0000 7f501dffb700 1 -- 172.21.15.31:0/2477944818 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5024013860 con 0x7f50280b0d60 2023-10-06T05:22:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.384+0000 7f501dffb700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f500c067200 0x7f500c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.384+0000 7f501dffb700 1 -- 172.21.15.31:0/2477944818 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5024084ef0 con 0x7f50280b0d60 2023-10-06T05:22:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.385+0000 7f502d59a700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f500c067200 0x7f500c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.386+0000 7f502d59a700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f500c067200 0x7f500c0696b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f502806fbc0 tx=0x7f5018017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.388+0000 7f501dffb700 1 -- 172.21.15.31:0/2477944818 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5024051c70 con 0x7f50280b0d60 2023-10-06T05:22:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.628+0000 7f502ffff700 1 -- 172.21.15.31:0/2477944818 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "clone_0000000000030936", "target": ["mon-mgr", ""]}) v1 -- 0x7f502815de50 con 0x7f500c067200 2023-10-06T05:22:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.640+0000 7f501dffb700 1 -- 172.21.15.31:0/2477944818 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+710 (secure 0 0 0) 0x7f502815de50 con 0x7f500c067200 2023-10-06T05:22:52.642 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "atime": "2023-10-06 05:22:44", 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "80.00", 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": 10485760, 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 8388608, 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-10-06 05:22:49", 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "ctime": "2023-10-06 05:22:49", 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "features": [ 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-clone", 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-autoprotect", 2023-10-06T05:22:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-retention" 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 1000, 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": 16895, 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_addrs": [ 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.31:6789", 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6789", 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.62:6790" 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "mtime": "2023-10-06 05:22:48", 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b", 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_namespace": "", 2023-10-06T05:22:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete", 2023-10-06T05:22:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "clone", 2023-10-06T05:22:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: "uid": 1000 2023-10-06T05:22:52.645 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.642+0000 7f502ffff700 1 -- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f500c067200 msgr2=0x7f500c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.642+0000 7f502ffff700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f500c067200 0x7f500c0696b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f502806fbc0 tx=0x7f5018017000 comp rx=0 tx=0).stop 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.642+0000 7f502ffff700 1 -- 172.21.15.31:0/2477944818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50280b0d60 msgr2=0x7f502814c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.642+0000 7f502ffff700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50280b0d60 0x7f502814c8f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f502400ec80 tx=0x7f50240091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.643+0000 7f502ffff700 1 -- 172.21.15.31:0/2477944818 shutdown_connections 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.643+0000 7f502ffff700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f500c067200 0x7f500c0696b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.643+0000 7f502ffff700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5028159930 0x7f502815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.643+0000 7f502ffff700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50280b0d60 0x7f502814c8f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.643+0000 7f502ffff700 1 --2- 172.21.15.31:0/2477944818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5028159560 0x7f50280c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.643+0000 7f502ffff700 1 -- 172.21.15.31:0/2477944818 >> 172.21.15.31:0/2477944818 conn(0x7f502806ec90 msgr2=0x7f5028158b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:52.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.643+0000 7f502ffff700 1 -- 172.21.15.31:0/2477944818 shutdown_connections 2023-10-06T05:22:52.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:52.643+0000 7f502ffff700 1 -- 172.21.15.31:0/2477944818 wait complete. 2023-10-06T05:22:52.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:52.662 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671) 2023-10-06T05:22:52.708 INFO:teuthology.orchestra.run.smithi031.stderr:volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671: ceph.quota.max_bytes: No such attribute 2023-10-06T05:22:52.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:22:52.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:52.710 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671) 2023-10-06T05:22:52.757 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T05:22:52.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:52.757 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671) 2023-10-06T05:22:52.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:52.806 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671) 2023-10-06T05:22:52.854 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671 2023-10-06T05:22:52.854 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.2 2023-10-06T05:22:52.854 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.5 2023-10-06T05:22:52.854 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.4 2023-10-06T05:22:52.854 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.7 2023-10-06T05:22:52.854 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.6 2023-10-06T05:22:52.855 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.0 2023-10-06T05:22:52.855 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.3 2023-10-06T05:22:52.855 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.1 2023-10-06T05:22:52.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:52.856 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671) 2023-10-06T05:22:52.901 INFO:teuthology.orchestra.run.smithi031.stdout:41ff 2023-10-06T05:22:52.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:52.903 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/) 2023-10-06T05:22:52.948 INFO:teuthology.orchestra.run.smithi031.stdout:41ff 2023-10-06T05:22:52.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:52.949 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671) 2023-10-06T05:22:52.998 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:52.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:52.999 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/) 2023-10-06T05:22:53.044 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:53.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.046 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671) 2023-10-06T05:22:53.132 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:53.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.133 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/) 2023-10-06T05:22:53.219 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:53.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.220 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671) 2023-10-06T05:22:53.266 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:53.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.267 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/) 2023-10-06T05:22:53.313 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:53.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.314 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.2) 2023-10-06T05:22:53.363 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:53.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.365 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.2) 2023-10-06T05:22:53.416 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:53.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.417 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.2) 2023-10-06T05:22:53.463 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:53.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.464 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.2) 2023-10-06T05:22:53.510 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:53.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.511 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.2) 2023-10-06T05:22:53.598 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:53.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.599 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.2) 2023-10-06T05:22:53.645 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:53.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.646 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.2) 2023-10-06T05:22:53.732 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:53.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.733 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.2) 2023-10-06T05:22:53.781 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:53.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.782 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.5) 2023-10-06T05:22:53.830 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:53.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.832 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.5) 2023-10-06T05:22:53.883 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:53.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.884 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.5) 2023-10-06T05:22:53.930 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:53.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.932 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.5) 2023-10-06T05:22:53.977 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:53.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:53.978 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.5) 2023-10-06T05:22:54.024 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:54.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.026 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.5) 2023-10-06T05:22:54.071 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:54.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.073 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.5) 2023-10-06T05:22:54.160 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:54.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.161 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.5) 2023-10-06T05:22:54.208 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:54.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.209 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.4) 2023-10-06T05:22:54.257 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:54.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.258 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.4) 2023-10-06T05:22:54.309 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:54.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.311 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.4) 2023-10-06T05:22:54.357 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:54.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.358 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.4) 2023-10-06T05:22:54.443 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:54.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.444 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.4) 2023-10-06T05:22:54.490 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:54.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.491 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.4) 2023-10-06T05:22:54.537 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:54.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.538 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.4) 2023-10-06T05:22:54.625 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:54.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.626 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.4) 2023-10-06T05:22:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:54.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.674 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.7) 2023-10-06T05:22:54.723 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:54.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.724 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.7) 2023-10-06T05:22:54.775 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:54.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.776 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.7) 2023-10-06T05:22:54.823 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:54.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.824 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.7) 2023-10-06T05:22:54.911 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:54.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:54.913 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.7) 2023-10-06T05:22:54.998 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:54.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.000 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.7) 2023-10-06T05:22:55.045 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:55.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.047 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.7) 2023-10-06T05:22:55.093 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:55.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.094 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.7) 2023-10-06T05:22:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:55.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.142 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.6) 2023-10-06T05:22:55.189 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:55.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.190 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.6) 2023-10-06T05:22:55.242 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:55.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.246 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.6) 2023-10-06T05:22:55.292 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:55.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.293 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.6) 2023-10-06T05:22:55.379 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:55.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.381 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.6) 2023-10-06T05:22:55.426 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:55.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.428 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.6) 2023-10-06T05:22:55.514 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:55.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.515 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.6) 2023-10-06T05:22:55.601 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:55.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.602 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.6) 2023-10-06T05:22:55.649 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:55.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.650 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.0) 2023-10-06T05:22:55.738 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:55.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.740 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.0) 2023-10-06T05:22:55.790 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:55.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.791 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.0) 2023-10-06T05:22:55.838 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:55.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.839 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.0) 2023-10-06T05:22:55.885 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:55.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.886 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.0) 2023-10-06T05:22:55.973 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:55.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:55.974 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.0) 2023-10-06T05:22:56.021 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:56.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.022 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.0) 2023-10-06T05:22:56.069 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:56.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.073 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.0) 2023-10-06T05:22:56.121 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:56.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.122 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.3) 2023-10-06T05:22:56.171 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:56.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.173 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.3) 2023-10-06T05:22:56.219 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:56.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.221 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.3) 2023-10-06T05:22:56.267 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:56.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.268 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.3) 2023-10-06T05:22:56.355 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:56.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.356 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.3) 2023-10-06T05:22:56.402 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:56.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.403 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.3) 2023-10-06T05:22:56.489 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:56.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.490 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.3) 2023-10-06T05:22:56.539 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:56.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.540 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.3) 2023-10-06T05:22:56.587 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:56.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.588 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.1) 2023-10-06T05:22:56.635 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:56.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.637 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.1) 2023-10-06T05:22:56.683 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-10-06T05:22:56.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.685 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.1) 2023-10-06T05:22:56.731 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:56.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.733 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.1) 2023-10-06T05:22:56.779 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-10-06T05:22:56.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.780 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.1) 2023-10-06T05:22:56.827 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:56.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.828 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.1) 2023-10-06T05:22:56.874 INFO:teuthology.orchestra.run.smithi031.stdout:1285 2023-10-06T05:22:56.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.875 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1/.snap/snapshot_0000000000055671/subvolume_file.1) 2023-10-06T05:22:56.924 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:56.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:22:56.925 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/clone_0000000000030936/0d20c17d-83dd-40b0-9c35-99facbe9936b/subvolume_file.1) 2023-10-06T05:22:56.971 INFO:teuthology.orchestra.run.smithi031.stdout:1696569768 2023-10-06T05:22:56.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvolume_0000000000063418 fake clone_0000000000030937 2023-10-06T05:22:57.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.174+0000 7f157cbff700 1 Processor -- start 2023-10-06T05:22:57.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.174+0000 7f157cbff700 1 -- start start 2023-10-06T05:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.176+0000 7f157cbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780b2bf0 0x7f15780c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.176+0000 7f157cbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780cbae0 con 0x7f15780b2820 2023-10-06T05:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.176+0000 7f157cbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780cf120 con 0x7f15780cce60 2023-10-06T05:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.176+0000 7f157cbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15780cb280 con 0x7f15780b2bf0 2023-10-06T05:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.176+0000 7f157659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f15780b2820 0x7f15780cea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47816/0 (socket says 172.21.15.31:47816) 2023-10-06T05:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.176+0000 7f157659c700 1 -- 172.21.15.31:0/1546097278 learned_addr learned my addr 172.21.15.31:0/1546097278 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.176+0000 7f1575d9b700 1 --2- 172.21.15.31:0/1546097278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780b2bf0 0x7f15780c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.177+0000 7f157559a700 1 -- 172.21.15.31:0/1546097278 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864189628 0 0) 0x7f15780cbae0 con 0x7f15780b2820 2023-10-06T05:22:57.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.177+0000 7f157559a700 1 -- 172.21.15.31:0/1546097278 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1560003570 con 0x7f15780b2820 2023-10-06T05:22:57.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.177+0000 7f157559a700 1 -- 172.21.15.31:0/1546097278 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098724840 0 0) 0x7f15780cf120 con 0x7f15780cce60 2023-10-06T05:22:57.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.177+0000 7f157559a700 1 -- 172.21.15.31:0/1546097278 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15780cbae0 con 0x7f15780cce60 2023-10-06T05:22:57.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.177+0000 7f1575d9b700 1 -- 172.21.15.31:0/1546097278 >> v1:172.21.15.62:6789/0 conn(0x7f15780cce60 legacy=0x7f15780cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.177+0000 7f1575d9b700 1 -- 172.21.15.31:0/1546097278 >> v1:172.21.15.31:6789/0 conn(0x7f15780b2820 legacy=0x7f15780cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.177+0000 7f1575d9b700 1 -- 172.21.15.31:0/1546097278 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780cf3f0 con 0x7f15780b2bf0 2023-10-06T05:22:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.177+0000 7f1575d9b700 1 --2- 172.21.15.31:0/1546097278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780b2bf0 0x7f15780c5240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f156c009910 tx=0x7f156c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65dbfcfb040d4248 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.177+0000 7f157559a700 1 -- 172.21.15.31:0/1546097278 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c011630 con 0x7f15780b2bf0 2023-10-06T05:22:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.178+0000 7f157559a700 1 -- 172.21.15.31:0/1546097278 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c011790 con 0x7f15780b2bf0 2023-10-06T05:22:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.178+0000 7f157559a700 1 -- 172.21.15.31:0/1546097278 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c011b30 con 0x7f15780b2bf0 2023-10-06T05:22:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.178+0000 7f157cbff700 1 -- 172.21.15.31:0/1546097278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780b2bf0 msgr2=0x7f15780c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.178+0000 7f157cbff700 1 --2- 172.21.15.31:0/1546097278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780b2bf0 0x7f15780c5240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f156c009910 tx=0x7f156c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.178+0000 7f157cbff700 1 -- 172.21.15.31:0/1546097278 shutdown_connections 2023-10-06T05:22:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.178+0000 7f157cbff700 1 --2- 172.21.15.31:0/1546097278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780b2bf0 0x7f15780c5240 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.178+0000 7f157cbff700 1 -- 172.21.15.31:0/1546097278 >> 172.21.15.31:0/1546097278 conn(0x7f15780996f0 msgr2=0x7f157806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.178+0000 7f157cbff700 1 -- 172.21.15.31:0/1546097278 shutdown_connections 2023-10-06T05:22:57.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.178+0000 7f157cbff700 1 -- 172.21.15.31:0/1546097278 wait complete. 2023-10-06T05:22:57.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.180+0000 7f157cbff700 1 Processor -- start 2023-10-06T05:22:57.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.181+0000 7f157cbff700 1 -- start start 2023-10-06T05:22:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f157cbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15780b2820 0x7f157816a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f157cbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15780b2bf0 0x7f157814ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f157cbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780cce60 0x7f15781532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f157cbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157816a730 con 0x7f15780b2bf0 2023-10-06T05:22:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f157cbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1578150ab0 con 0x7f15780b2820 2023-10-06T05:22:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f157cbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157814fec0 con 0x7f15780cce60 2023-10-06T05:22:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f1575d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15780b2bf0 0x7f157814ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f1575d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15780b2bf0 0x7f157814ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57150/0 (socket says 172.21.15.31:57150) 2023-10-06T05:22:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f1575d9b700 1 -- 172.21.15.31:0/2956748300 learned_addr learned my addr 172.21.15.31:0/2956748300 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f157659c700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15780b2820 0x7f157816a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f1576d9d700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780cce60 0x7f15781532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f1575d9b700 1 -- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780cce60 msgr2=0x7f15781532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.182+0000 7f1575d9b700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780cce60 0x7f15781532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.183+0000 7f1575d9b700 1 -- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15780b2820 msgr2=0x7f157816a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.183+0000 7f1575d9b700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15780b2820 0x7f157816a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.183+0000 7f1575d9b700 1 -- 172.21.15.31:0/2956748300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c009420 con 0x7f15780b2bf0 2023-10-06T05:22:57.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.183+0000 7f1575d9b700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15780b2bf0 0x7f157814ed00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f156c009f00 tx=0x7f156c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.183+0000 7f15667fc700 1 -- 172.21.15.31:0/2956748300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c01f070 con 0x7f15780b2bf0 2023-10-06T05:22:57.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.183+0000 7f15667fc700 1 -- 172.21.15.31:0/2956748300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c002f70 con 0x7f15780b2bf0 2023-10-06T05:22:57.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.183+0000 7f157cbff700 1 -- 172.21.15.31:0/2956748300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1578150180 con 0x7f15780b2bf0 2023-10-06T05:22:57.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.183+0000 7f15667fc700 1 -- 172.21.15.31:0/2956748300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c011900 con 0x7f15780b2bf0 2023-10-06T05:22:57.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.183+0000 7f157cbff700 1 -- 172.21.15.31:0/2956748300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1578150740 con 0x7f15780b2bf0 2023-10-06T05:22:57.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.184+0000 7f157cbff700 1 -- 172.21.15.31:0/2956748300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157804c380 con 0x7f15780b2bf0 2023-10-06T05:22:57.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.185+0000 7f15667fc700 1 -- 172.21.15.31:0/2956748300 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f156c023070 con 0x7f15780b2bf0 2023-10-06T05:22:57.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.186+0000 7f15667fc700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067200 0x7f15580696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.186+0000 7f15667fc700 1 -- 172.21.15.31:0/2956748300 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f156c085dd0 con 0x7f15780b2bf0 2023-10-06T05:22:57.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.187+0000 7f157659c700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067200 0x7f15580696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.187+0000 7f157659c700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067200 0x7f15580696b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1568001360 tx=0x7f1568009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.191+0000 7f15667fc700 1 -- 172.21.15.31:0/2956748300 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f156c052b50 con 0x7f15780b2bf0 2023-10-06T05:22:57.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.436+0000 7f157cbff700 1 -- 172.21.15.31:0/2956748300 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "snap_name": "fake", "target_sub_name": "clone_0000000000030937", "target": ["mon-mgr", ""]}) v1 -- 0x7f1578150bf0 con 0x7f1558067200 2023-10-06T05:22:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.510+0000 7f15667fc700 1 -- 172.21.15.31:0/2956748300 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1578150bf0 con 0x7f1558067200 2023-10-06T05:22:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 -- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067200 msgr2=0x7f15580696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067200 0x7f15580696b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1568001360 tx=0x7f1568009040 comp rx=0 tx=0).stop 2023-10-06T05:22:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 -- 172.21.15.31:0/2956748300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15780b2bf0 msgr2=0x7f157814ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15780b2bf0 0x7f157814ed00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f156c009f00 tx=0x7f156c017000 comp rx=0 tx=0).stop 2023-10-06T05:22:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 -- 172.21.15.31:0/2956748300 shutdown_connections 2023-10-06T05:22:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1558067200 0x7f15580696b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15780cce60 0x7f15781532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15780b2820 0x7f157816a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 --2- 172.21.15.31:0/2956748300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15780b2bf0 0x7f157814ed00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 -- 172.21.15.31:0/2956748300 >> 172.21.15.31:0/2956748300 conn(0x7f15780996f0 msgr2=0x7f15780cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 -- 172.21.15.31:0/2956748300 shutdown_connections 2023-10-06T05:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.513+0000 7f157cbff700 1 -- 172.21.15.31:0/2956748300 wait complete. 2023-10-06T05:22:57.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs clone_0000000000030937 2023-10-06T05:22:57.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.729+0000 7fa89f7ff700 1 Processor -- start 2023-10-06T05:22:57.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.730+0000 7fa89f7ff700 1 -- start start 2023-10-06T05:22:57.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.731+0000 7fa89f7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa898159c60 0x7fa8980c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.732+0000 7fa89f7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa898158190 con 0x7fa898155430 2023-10-06T05:22:57.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.732+0000 7fa89f7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa898157580 con 0x7fa898159890 2023-10-06T05:22:57.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.732+0000 7fa89f7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa898157860 con 0x7fa898159c60 2023-10-06T05:22:57.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.732+0000 7fa89cd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa898159c60 0x7fa8980c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.732+0000 7fa89dd9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa898155430 0x7fa898155890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47832/0 (socket says 172.21.15.31:47832) 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.732+0000 7fa89dd9c700 1 -- 172.21.15.31:0/924583441 learned_addr learned my addr 172.21.15.31:0/924583441 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.732+0000 7fa89cd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa898159c60 0x7fa8980c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57160/0 (socket says 172.21.15.31:57160) 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.732+0000 7fa89cd9a700 1 -- 172.21.15.31:0/924583441 >> v1:172.21.15.62:6789/0 conn(0x7fa898159890 legacy=0x7fa898156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.732+0000 7fa89cd9a700 1 -- 172.21.15.31:0/924583441 >> v1:172.21.15.31:6789/0 conn(0x7fa898155430 legacy=0x7fa898155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.732+0000 7fa89cd9a700 1 -- 172.21.15.31:0/924583441 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa898157b40 con 0x7fa898159c60 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.733+0000 7fa89cd9a700 1 --2- 172.21.15.31:0/924583441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159c60 0x7fa8980c89e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa88800bb90 tx=0x7fa88800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b635305fe32d1f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.733+0000 7fa88ffff700 1 -- 172.21.15.31:0/924583441 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800da90 con 0x7fa898159c60 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.733+0000 7fa88ffff700 1 -- 172.21.15.31:0/924583441 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88800dbf0 con 0x7fa898159c60 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.733+0000 7fa88ffff700 1 -- 172.21.15.31:0/924583441 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888017410 con 0x7fa898159c60 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.733+0000 7fa89f7ff700 1 -- 172.21.15.31:0/924583441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159c60 msgr2=0x7fa8980c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.733+0000 7fa89f7ff700 1 --2- 172.21.15.31:0/924583441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159c60 0x7fa8980c89e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa88800bb90 tx=0x7fa88800b430 comp rx=0 tx=0).stop 2023-10-06T05:22:57.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.733+0000 7fa89f7ff700 1 -- 172.21.15.31:0/924583441 shutdown_connections 2023-10-06T05:22:57.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.733+0000 7fa89f7ff700 1 --2- 172.21.15.31:0/924583441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159c60 0x7fa8980c89e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.733+0000 7fa89f7ff700 1 -- 172.21.15.31:0/924583441 >> 172.21.15.31:0/924583441 conn(0x7fa898067560 msgr2=0x7fa898067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:22:57.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.734+0000 7fa89f7ff700 1 -- 172.21.15.31:0/924583441 shutdown_connections 2023-10-06T05:22:57.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.734+0000 7fa89f7ff700 1 -- 172.21.15.31:0/924583441 wait complete. 2023-10-06T05:22:57.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.736+0000 7fa89f7ff700 1 Processor -- start 2023-10-06T05:22:57.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.736+0000 7fa89f7ff700 1 -- start start 2023-10-06T05:22:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.737+0000 7fa89f7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa898155430 0x7fa89806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.737+0000 7fa89f7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159890 0x7fa8980c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.737+0000 7fa89f7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898159c60 0x7fa89814e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.737+0000 7fa89f7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa898169950 con 0x7fa898159890 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.737+0000 7fa89f7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980c9e30 con 0x7fa898159c60 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.737+0000 7fa89f7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980c8f60 con 0x7fa898155430 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.737+0000 7fa89cd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159890 0x7fa8980c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.737+0000 7fa89cd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159890 0x7fa8980c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57164/0 (socket says 172.21.15.31:57164) 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.737+0000 7fa89cd9a700 1 -- 172.21.15.31:0/21750372 learned_addr learned my addr 172.21.15.31:0/21750372 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89dd9c700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898159c60 0x7fa89814e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89d59b700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa898155430 0x7fa89806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89cd9a700 1 -- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa898155430 msgr2=0x7fa89806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89cd9a700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa898155430 0x7fa89806e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89cd9a700 1 -- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898159c60 msgr2=0x7fa89814e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:22:57.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89cd9a700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898159c60 0x7fa89814e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89cd9a700 1 -- 172.21.15.31:0/21750372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88800b6d0 con 0x7fa898159890 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89dd9c700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898159c60 0x7fa89814e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89d59b700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa898155430 0x7fa89806e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89cd9a700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159890 0x7fa8980c71a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa89806f430 tx=0x7fa888019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa88d7fa700 1 -- 172.21.15.31:0/21750372 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888009050 con 0x7fa898159890 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa88d7fa700 1 -- 172.21.15.31:0/21750372 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88800a040 con 0x7fa898159890 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa88d7fa700 1 -- 172.21.15.31:0/21750372 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888021830 con 0x7fa898159890 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.738+0000 7fa89f7ff700 1 -- 172.21.15.31:0/21750372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8980c6da0 con 0x7fa898159890 2023-10-06T05:22:57.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.739+0000 7fa89f7ff700 1 -- 172.21.15.31:0/21750372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8980c9500 con 0x7fa898159890 2023-10-06T05:22:57.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.740+0000 7fa89f7ff700 1 -- 172.21.15.31:0/21750372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89804c380 con 0x7fa898159890 2023-10-06T05:22:57.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.740+0000 7fa88d7fa700 1 -- 172.21.15.31:0/21750372 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa888015070 con 0x7fa898159890 2023-10-06T05:22:57.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.741+0000 7fa88d7fa700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa87c067230 0x7fa87c0696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:22:57.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.741+0000 7fa88d7fa700 1 -- 172.21.15.31:0/21750372 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa888086700 con 0x7fa898159890 2023-10-06T05:22:57.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.742+0000 7fa89d59b700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa87c067230 0x7fa87c0696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:22:57.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.742+0000 7fa89d59b700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa87c067230 0x7fa87c0696e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa894001820 tx=0x7fa894009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:22:57.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.745+0000 7fa88d7fa700 1 -- 172.21.15.31:0/21750372 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa888053370 con 0x7fa898159890 2023-10-06T05:22:57.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:22:57.990+0000 7fa89f7ff700 1 -- 172.21.15.31:0/21750372 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "clone_0000000000030937", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8980ccc40 con 0x7fa87c067230 2023-10-06T05:23:01.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:01.738 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.500+0000 7fa88d7fa700 1 -- 172.21.15.31:0/21750372 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+168 (secure 0 0 0) 0x7fa8980ccc40 con 0x7fa87c067230 2023-10-06T05:23:02.502 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:23:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-10-06T05:23:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "pending", 2023-10-06T05:23:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "source": { 2023-10-06T05:23:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "volume": "cephfs", 2023-10-06T05:23:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "subvolume": "subvolume_0000000000063418", 2023-10-06T05:23:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot": "fake" 2023-10-06T05:23:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:02.503 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:23:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.502+0000 7fa89f7ff700 1 -- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa87c067230 msgr2=0x7fa87c0696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.502+0000 7fa89f7ff700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa87c067230 0x7fa87c0696e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa894001820 tx=0x7fa894009040 comp rx=0 tx=0).stop 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.502+0000 7fa89f7ff700 1 -- 172.21.15.31:0/21750372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159890 msgr2=0x7fa8980c71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.502+0000 7fa89f7ff700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159890 0x7fa8980c71a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa89806f430 tx=0x7fa888019000 comp rx=0 tx=0).stop 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.503+0000 7fa89f7ff700 1 -- 172.21.15.31:0/21750372 shutdown_connections 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.503+0000 7fa89f7ff700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa87c067230 0x7fa87c0696e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.503+0000 7fa89f7ff700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898159c60 0x7fa89814e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.503+0000 7fa89f7ff700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa898155430 0x7fa89806e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.503+0000 7fa89f7ff700 1 --2- 172.21.15.31:0/21750372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa898159890 0x7fa8980c71a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.503+0000 7fa89f7ff700 1 -- 172.21.15.31:0/21750372 >> 172.21.15.31:0/21750372 conn(0x7fa898067560 msgr2=0x7fa8980c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.503+0000 7fa89f7ff700 1 -- 172.21.15.31:0/21750372 shutdown_connections 2023-10-06T05:23:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:02.503+0000 7fa89f7ff700 1 -- 172.21.15.31:0/21750372 wait complete. 2023-10-06T05:23:03.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs clone_0000000000030937 2023-10-06T05:23:03.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.734+0000 7efde25ff700 1 Processor -- start 2023-10-06T05:23:03.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.735+0000 7efde25ff700 1 -- start start 2023-10-06T05:23:03.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde25ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc15ca40 0x7efddc068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde25ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efddc06a300 0x7efddc0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde25ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efddc0c8d70 0x7efddc06cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde25ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc0cabc0 con 0x7efddc0c8d70 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde25ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc068bb0 con 0x7efddc06a300 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde25ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0c91f0 con 0x7efddc15ca40 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde0b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efddc0c8d70 0x7efddc06cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde0b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efddc0c8d70 0x7efddc06cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57180/0 (socket says 172.21.15.31:57180) 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde0b9c700 1 -- 172.21.15.31:0/4286953263 learned_addr learned my addr 172.21.15.31:0/4286953263 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efddbfff700 1 --2- 172.21.15.31:0/4286953263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc15ca40 0x7efddc068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efddb7fe700 1 --2- 172.21.15.31:0/4286953263 >> v2:172.21.15.62:3300/0 conn(0x7efddc06a300 0x7efddc0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde0b9c700 1 -- 172.21.15.31:0/4286953263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc15ca40 msgr2=0x7efddc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde0b9c700 1 --2- 172.21.15.31:0/4286953263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc15ca40 0x7efddc068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde0b9c700 1 -- 172.21.15.31:0/4286953263 >> v2:172.21.15.62:3300/0 conn(0x7efddc06a300 msgr2=0x7efddc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde0b9c700 1 --2- 172.21.15.31:0/4286953263 >> v2:172.21.15.62:3300/0 conn(0x7efddc06a300 0x7efddc0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.736+0000 7efde0b9c700 1 -- 172.21.15.31:0/4286953263 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc068da0 con 0x7efddc0c8d70 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.737+0000 7efde0b9c700 1 --2- 172.21.15.31:0/4286953263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc0c8d70 0x7efddc06cf40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efddc06f220 tx=0x7efdd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751c3602c9cce63d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.737+0000 7efddaffd700 1 -- 172.21.15.31:0/4286953263 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd400e070 con 0x7efddc0c8d70 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.737+0000 7efddaffd700 1 -- 172.21.15.31:0/4286953263 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd400ba40 con 0x7efddc0c8d70 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.737+0000 7efddaffd700 1 -- 172.21.15.31:0/4286953263 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd40092e0 con 0x7efddc0c8d70 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.737+0000 7efde25ff700 1 -- 172.21.15.31:0/4286953263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc0c8d70 msgr2=0x7efddc06cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.737+0000 7efde25ff700 1 --2- 172.21.15.31:0/4286953263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc0c8d70 0x7efddc06cf40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efddc06f220 tx=0x7efdd4009840 comp rx=0 tx=0).stop 2023-10-06T05:23:03.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.738+0000 7efde25ff700 1 -- 172.21.15.31:0/4286953263 shutdown_connections 2023-10-06T05:23:03.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.738+0000 7efde25ff700 1 --2- 172.21.15.31:0/4286953263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc0c8d70 0x7efddc06cf40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.738+0000 7efde25ff700 1 --2- 172.21.15.31:0/4286953263 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc15ca40 0x7efddc068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.738+0000 7efde25ff700 1 --2- 172.21.15.31:0/4286953263 >> v2:172.21.15.62:3300/0 conn(0x7efddc06a300 0x7efddc0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.738+0000 7efde25ff700 1 -- 172.21.15.31:0/4286953263 >> 172.21.15.31:0/4286953263 conn(0x7efddc0ce290 msgr2=0x7efddc0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:03.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.738+0000 7efde25ff700 1 -- 172.21.15.31:0/4286953263 shutdown_connections 2023-10-06T05:23:03.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.738+0000 7efde25ff700 1 -- 172.21.15.31:0/4286953263 wait complete. 2023-10-06T05:23:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.739+0000 7efde25ff700 1 Processor -- start 2023-10-06T05:23:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.739+0000 7efde25ff700 1 -- start start 2023-10-06T05:23:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.740+0000 7efde25ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc06a300 0x7efddc16a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.740+0000 7efde25ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc0c8d70 0x7efddc16d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.740+0000 7efde25ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efddc15ca40 0x7efddc159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.740+0000 7efde25ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc16a8e0 con 0x7efddc06a300 2023-10-06T05:23:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.740+0000 7efde25ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc16dde0 con 0x7efddc15ca40 2023-10-06T05:23:03.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.740+0000 7efde25ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc1698e0 con 0x7efddc0c8d70 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc06a300 0x7efddc16a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc06a300 0x7efddc16a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57184/0 (socket says 172.21.15.31:57184) 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddbfff700 1 -- 172.21.15.31:0/3224967962 learned_addr learned my addr 172.21.15.31:0/3224967962 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddb7fe700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc0c8d70 0x7efddc16d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efde0b9c700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efddc15ca40 0x7efddc159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddbfff700 1 -- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc0c8d70 msgr2=0x7efddc16d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddbfff700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc0c8d70 0x7efddc16d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddbfff700 1 -- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efddc15ca40 msgr2=0x7efddc159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddbfff700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efddc15ca40 0x7efddc159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddbfff700 1 -- 172.21.15.31:0/3224967962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd4009ae0 con 0x7efddc06a300 2023-10-06T05:23:03.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efde0b9c700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efddc15ca40 0x7efddc159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddb7fe700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc0c8d70 0x7efddc16d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.741+0000 7efddbfff700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc06a300 0x7efddc16a3a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00d890 tx=0x7efdcc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.742+0000 7efdc3fff700 1 -- 172.21.15.31:0/3224967962 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc00b600 con 0x7efddc06a300 2023-10-06T05:23:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.742+0000 7efdc3fff700 1 -- 172.21.15.31:0/3224967962 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc00b760 con 0x7efddc06a300 2023-10-06T05:23:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.742+0000 7efde25ff700 1 -- 172.21.15.31:0/3224967962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc0c4140 con 0x7efddc06a300 2023-10-06T05:23:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.742+0000 7efdc3fff700 1 -- 172.21.15.31:0/3224967962 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc00bae0 con 0x7efddc06a300 2023-10-06T05:23:03.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.742+0000 7efde25ff700 1 -- 172.21.15.31:0/3224967962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc169ba0 con 0x7efddc06a300 2023-10-06T05:23:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.743+0000 7efde25ff700 1 -- 172.21.15.31:0/3224967962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc0adce0 con 0x7efddc06a300 2023-10-06T05:23:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.744+0000 7efdc3fff700 1 -- 172.21.15.31:0/3224967962 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7efdcc01e460 con 0x7efddc06a300 2023-10-06T05:23:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.745+0000 7efdc3fff700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efdc8067200 0x7efdc80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.745+0000 7efdc3fff700 1 -- 172.21.15.31:0/3224967962 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efdcc0851a0 con 0x7efddc06a300 2023-10-06T05:23:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.745+0000 7efddb7fe700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efdc8067200 0x7efdc80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:03.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.746+0000 7efddb7fe700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efdc8067200 0x7efdc80696b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efddc0c35b0 tx=0x7efdd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:03.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.749+0000 7efdc3fff700 1 -- 172.21.15.31:0/3224967962 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efdcc051f60 con 0x7efddc06a300 2023-10-06T05:23:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:03.994+0000 7efde25ff700 1 -- 172.21.15.31:0/3224967962 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "clone_0000000000030937", "target": ["mon-mgr", ""]}) v1 -- 0x7efddc06fb10 con 0x7efdc8067200 2023-10-06T05:23:07.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.489+0000 7efdc3fff700 1 -- 172.21.15.31:0/3224967962 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7efddc06fb10 con 0x7efdc8067200 2023-10-06T05:23:07.755 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:23:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-10-06T05:23:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete" 2023-10-06T05:23:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:07.756 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:23:07.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 -- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efdc8067200 msgr2=0x7efdc80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:07.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efdc8067200 0x7efdc80696b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efddc0c35b0 tx=0x7efdd0008040 comp rx=0 tx=0).stop 2023-10-06T05:23:07.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 -- 172.21.15.31:0/3224967962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc06a300 msgr2=0x7efddc16a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:07.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc06a300 0x7efddc16a3a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00d890 tx=0x7efdcc008040 comp rx=0 tx=0).stop 2023-10-06T05:23:07.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 -- 172.21.15.31:0/3224967962 shutdown_connections 2023-10-06T05:23:07.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efdc8067200 0x7efdc80696b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:07.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efddc15ca40 0x7efddc159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:07.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efddc06a300 0x7efddc16a3a0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 --2- 172.21.15.31:0/3224967962 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efddc0c8d70 0x7efddc16d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:07.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 -- 172.21.15.31:0/3224967962 >> 172.21.15.31:0/3224967962 conn(0x7efddc0ce290 msgr2=0x7efddc15a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:07.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 -- 172.21.15.31:0/3224967962 shutdown_connections 2023-10-06T05:23:07.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:07.492+0000 7efde25ff700 1 -- 172.21.15.31:0/3224967962 wait complete. 2023-10-06T05:23:07.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:07.900 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/clone_0000000000030937/.meta) 2023-10-06T05:23:07.978 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:23:08.052 INFO:teuthology.orchestra.run.smithi031.stdout:version = 2 2023-10-06T05:23:08.052 INFO:teuthology.orchestra.run.smithi031.stdout:type = clone 2023-10-06T05:23:08.052 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/clone_0000000000030937/ae91984d-3db4-4e7b-a41e-2bb71b899568 2023-10-06T05:23:08.053 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:23:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:08.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvolume_0000000000063418 snapshot_0000000000055671 2023-10-06T05:23:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.258+0000 7efe6d9ff700 1 Processor -- start 2023-10-06T05:23:08.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.258+0000 7efe6d9ff700 1 -- start start 2023-10-06T05:23:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.259+0000 7efe6d9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efe680ad670 0x7efe6806ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.259+0000 7efe6d9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe6806a630 0x7efe6806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.259+0000 7efe6d9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6806e600 con 0x7efe68156300 2023-10-06T05:23:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.259+0000 7efe6d9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe680adae0 con 0x7efe6806a630 2023-10-06T05:23:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.259+0000 7efe6d9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6806aff0 con 0x7efe680ad670 2023-10-06T05:23:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.260+0000 7efe677fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe6806a630 0x7efe6806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.260+0000 7efe677fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe6806a630 0x7efe6806aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49958/0 (socket says 172.21.15.31:49958) 2023-10-06T05:23:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.260+0000 7efe677fe700 1 -- 172.21.15.31:0/1593621604 learned_addr learned my addr 172.21.15.31:0/1593621604 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.260+0000 7efe667fc700 1 --2- 172.21.15.31:0/1593621604 >> v2:172.21.15.62:3300/0 conn(0x7efe680ad670 0x7efe6806ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.260+0000 7efe677fe700 1 -- 172.21.15.31:0/1593621604 >> v2:172.21.15.62:3300/0 conn(0x7efe680ad670 msgr2=0x7efe6806ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.260+0000 7efe677fe700 1 --2- 172.21.15.31:0/1593621604 >> v2:172.21.15.62:3300/0 conn(0x7efe680ad670 0x7efe6806ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.260+0000 7efe677fe700 1 -- 172.21.15.31:0/1593621604 >> v1:172.21.15.62:6789/0 conn(0x7efe68156300 legacy=0x7efe6806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.260+0000 7efe677fe700 1 -- 172.21.15.31:0/1593621604 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6806d9f0 con 0x7efe6806a630 2023-10-06T05:23:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.260+0000 7efe677fe700 1 --2- 172.21.15.31:0/1593621604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6806a630 0x7efe6806aab0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7efe680b2160 tx=0x7efe60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd5ad156b596e64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe65ffb700 1 -- 172.21.15.31:0/1593621604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6000e070 con 0x7efe6806a630 2023-10-06T05:23:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe65ffb700 1 -- 172.21.15.31:0/1593621604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6000ba20 con 0x7efe6806a630 2023-10-06T05:23:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe65ffb700 1 -- 172.21.15.31:0/1593621604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe600092e0 con 0x7efe6806a630 2023-10-06T05:23:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe6d9ff700 1 -- 172.21.15.31:0/1593621604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6806a630 msgr2=0x7efe6806aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe6d9ff700 1 --2- 172.21.15.31:0/1593621604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6806a630 0x7efe6806aab0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7efe680b2160 tx=0x7efe60009840 comp rx=0 tx=0).stop 2023-10-06T05:23:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe6d9ff700 1 -- 172.21.15.31:0/1593621604 shutdown_connections 2023-10-06T05:23:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe6d9ff700 1 --2- 172.21.15.31:0/1593621604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe6806a630 0x7efe6806aab0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe6d9ff700 1 --2- 172.21.15.31:0/1593621604 >> v2:172.21.15.62:3300/0 conn(0x7efe680ad670 0x7efe6806ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe6d9ff700 1 -- 172.21.15.31:0/1593621604 >> 172.21.15.31:0/1593621604 conn(0x7efe68068030 msgr2=0x7efe680cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe6d9ff700 1 -- 172.21.15.31:0/1593621604 shutdown_connections 2023-10-06T05:23:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.261+0000 7efe6d9ff700 1 -- 172.21.15.31:0/1593621604 wait complete. 2023-10-06T05:23:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.263+0000 7efe6d9ff700 1 Processor -- start 2023-10-06T05:23:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.263+0000 7efe6d9ff700 1 -- start start 2023-10-06T05:23:08.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe6d9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6806a630 0x7efe681552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe6d9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe680ad670 0x7efe6815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe6d9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe68156300 0x7efe68158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe6d9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6815c790 con 0x7efe680ad670 2023-10-06T05:23:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe6d9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe68155820 con 0x7efe6806a630 2023-10-06T05:23:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe6d9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6815bb80 con 0x7efe68156300 2023-10-06T05:23:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe667fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe680ad670 0x7efe6815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe667fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe680ad670 0x7efe6815a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49970/0 (socket says 172.21.15.31:49970) 2023-10-06T05:23:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe667fc700 1 -- 172.21.15.31:0/2750002465 learned_addr learned my addr 172.21.15.31:0/2750002465 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe66ffd700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6806a630 0x7efe681552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe677fe700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe68156300 0x7efe68158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe667fc700 1 -- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe68156300 msgr2=0x7efe68158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe667fc700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe68156300 0x7efe68158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe667fc700 1 -- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6806a630 msgr2=0x7efe681552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe667fc700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6806a630 0x7efe681552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe667fc700 1 -- 172.21.15.31:0/2750002465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe60009ae0 con 0x7efe680ad670 2023-10-06T05:23:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.264+0000 7efe66ffd700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6806a630 0x7efe681552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.265+0000 7efe667fc700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe680ad670 0x7efe6815a650 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00ec80 tx=0x7efe5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.265+0000 7efe4effd700 1 -- 172.21.15.31:0/2750002465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00c9d0 con 0x7efe680ad670 2023-10-06T05:23:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.265+0000 7efe4effd700 1 -- 172.21.15.31:0/2750002465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c00cb30 con 0x7efe680ad670 2023-10-06T05:23:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.265+0000 7efe6d9ff700 1 -- 172.21.15.31:0/2750002465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6815be60 con 0x7efe680ad670 2023-10-06T05:23:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.265+0000 7efe4effd700 1 -- 172.21.15.31:0/2750002465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c013470 con 0x7efe680ad670 2023-10-06T05:23:08.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.265+0000 7efe6d9ff700 1 -- 172.21.15.31:0/2750002465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6815c420 con 0x7efe680ad670 2023-10-06T05:23:08.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.266+0000 7efe6d9ff700 1 -- 172.21.15.31:0/2750002465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6804c380 con 0x7efe680ad670 2023-10-06T05:23:08.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.267+0000 7efe4effd700 1 -- 172.21.15.31:0/2750002465 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7efe5c013820 con 0x7efe680ad670 2023-10-06T05:23:08.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.268+0000 7efe4effd700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efe54067200 0x7efe540696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.268+0000 7efe4effd700 1 -- 172.21.15.31:0/2750002465 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efe5c084e90 con 0x7efe680ad670 2023-10-06T05:23:08.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.268+0000 7efe66ffd700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efe54067200 0x7efe540696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.269+0000 7efe66ffd700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efe54067200 0x7efe540696b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efe580018f0 tx=0x7efe58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.272+0000 7efe4effd700 1 -- 172.21.15.31:0/2750002465 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe5c051c10 con 0x7efe680ad670 2023-10-06T05:23:08.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.513+0000 7efe6d9ff700 1 -- 172.21.15.31:0/2750002465 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "snap_name": "snapshot_0000000000055671", "target": ["mon-mgr", ""]}) v1 -- 0x7efe68159700 con 0x7efe54067200 2023-10-06T05:23:08.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.534+0000 7efe4effd700 1 -- 172.21.15.31:0/2750002465 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+115 (secure 0 0 0) 0x7efe68159700 con 0x7efe54067200 2023-10-06T05:23:08.536 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:23:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-10-06 05:22:49.113464", 2023-10-06T05:23:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-10-06T05:23:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "has_pending_clones": "no" 2023-10-06T05:23:08.537 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:23:08.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 -- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efe54067200 msgr2=0x7efe540696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efe54067200 0x7efe540696b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efe580018f0 tx=0x7efe58009040 comp rx=0 tx=0).stop 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 -- 172.21.15.31:0/2750002465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe680ad670 msgr2=0x7efe6815a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe680ad670 0x7efe6815a650 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00ec80 tx=0x7efe5c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 -- 172.21.15.31:0/2750002465 shutdown_connections 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efe54067200 0x7efe540696b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe68156300 0x7efe68158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe6806a630 0x7efe681552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 --2- 172.21.15.31:0/2750002465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe680ad670 0x7efe6815a650 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 -- 172.21.15.31:0/2750002465 >> 172.21.15.31:0/2750002465 conn(0x7efe68068030 msgr2=0x7efe680ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 -- 172.21.15.31:0/2750002465 shutdown_connections 2023-10-06T05:23:08.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.536+0000 7efe6d9ff700 1 -- 172.21.15.31:0/2750002465 wait complete. 2023-10-06T05:23:08.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvolume_0000000000063418 2023-10-06T05:23:08.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.754+0000 7f0de3fff700 1 Processor -- start 2023-10-06T05:23:08.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.755+0000 7f0de3fff700 1 -- start start 2023-10-06T05:23:08.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de3fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ddc153950 0x7f0ddc153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ddc0cd5d0 0x7f0ddc15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ddc0cda20 0x7f0ddc15d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de3fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ddc15bb20 con 0x7f0ddc0cda20 2023-10-06T05:23:08.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de3fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ddc15af10 con 0x7f0ddc153950 2023-10-06T05:23:08.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de3fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc15b1f0 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ddc0cda20 0x7f0ddc15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ddc0cda20 0x7f0ddc15d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49974/0 (socket says 172.21.15.31:49974) 2023-10-06T05:23:08.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de259c700 1 -- 172.21.15.31:0/2973724822 learned_addr learned my addr 172.21.15.31:0/2973724822 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:08.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de1d9b700 1 --2- 172.21.15.31:0/2973724822 >> v2:172.21.15.62:3300/0 conn(0x7f0ddc153950 0x7f0ddc153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.756+0000 7f0de159a700 1 --2- 172.21.15.31:0/2973724822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ddc0cd5d0 0x7f0ddc15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.757+0000 7f0de259c700 1 -- 172.21.15.31:0/2973724822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ddc0cd5d0 msgr2=0x7f0ddc15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:08.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.757+0000 7f0de259c700 1 --2- 172.21.15.31:0/2973724822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ddc0cd5d0 0x7f0ddc15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.757+0000 7f0de259c700 1 -- 172.21.15.31:0/2973724822 >> v2:172.21.15.62:3300/0 conn(0x7f0ddc153950 msgr2=0x7f0ddc153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:08.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.757+0000 7f0de259c700 1 --2- 172.21.15.31:0/2973724822 >> v2:172.21.15.62:3300/0 conn(0x7f0ddc153950 0x7f0ddc153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.757+0000 7f0de259c700 1 -- 172.21.15.31:0/2973724822 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc15b4d0 con 0x7f0ddc0cda20 2023-10-06T05:23:08.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.757+0000 7f0de259c700 1 --2- 172.21.15.31:0/2973724822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cda20 0x7f0ddc15d750 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0d15b0 tx=0x7f0dd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6adf53672dae08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.757+0000 7f0de0d99700 1 -- 172.21.15.31:0/2973724822 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd400e070 con 0x7f0ddc0cda20 2023-10-06T05:23:08.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.757+0000 7f0de0d99700 1 -- 172.21.15.31:0/2973724822 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd400b970 con 0x7f0ddc0cda20 2023-10-06T05:23:08.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.757+0000 7f0de0d99700 1 -- 172.21.15.31:0/2973724822 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd40092e0 con 0x7f0ddc0cda20 2023-10-06T05:23:08.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.758+0000 7f0de3fff700 1 -- 172.21.15.31:0/2973724822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cda20 msgr2=0x7f0ddc15d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.758+0000 7f0de3fff700 1 --2- 172.21.15.31:0/2973724822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cda20 0x7f0ddc15d750 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0d15b0 tx=0x7f0dd4009840 comp rx=0 tx=0).stop 2023-10-06T05:23:08.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.758+0000 7f0de3fff700 1 -- 172.21.15.31:0/2973724822 shutdown_connections 2023-10-06T05:23:08.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.758+0000 7f0de3fff700 1 --2- 172.21.15.31:0/2973724822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cda20 0x7f0ddc15d750 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.758+0000 7f0de3fff700 1 --2- 172.21.15.31:0/2973724822 >> v2:172.21.15.62:3300/0 conn(0x7f0ddc153950 0x7f0ddc153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.758+0000 7f0de3fff700 1 --2- 172.21.15.31:0/2973724822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ddc0cd5d0 0x7f0ddc15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.758+0000 7f0de3fff700 1 -- 172.21.15.31:0/2973724822 >> 172.21.15.31:0/2973724822 conn(0x7f0ddc0681b0 msgr2=0x7f0ddc0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:08.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.758+0000 7f0de3fff700 1 -- 172.21.15.31:0/2973724822 shutdown_connections 2023-10-06T05:23:08.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.758+0000 7f0de3fff700 1 -- 172.21.15.31:0/2973724822 wait complete. 2023-10-06T05:23:08.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.760+0000 7f0de3fff700 1 Processor -- start 2023-10-06T05:23:08.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.760+0000 7f0de3fff700 1 -- start start 2023-10-06T05:23:08.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cd5d0 0x7f0ddc159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ddc0cda20 0x7f0ddc0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc153950 0x7f0ddc1699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de3fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc16cff0 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de3fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0b3670 con 0x7f0ddc153950 2023-10-06T05:23:08.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de3fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0b3950 con 0x7f0ddc0cda20 2023-10-06T05:23:08.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cd5d0 0x7f0ddc159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cd5d0 0x7f0ddc159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49976/0 (socket says 172.21.15.31:49976) 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de1d9b700 1 -- 172.21.15.31:0/2007938642 learned_addr learned my addr 172.21.15.31:0/2007938642 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de259c700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc153950 0x7f0ddc1699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de159a700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ddc0cda20 0x7f0ddc0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de1d9b700 1 -- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ddc0cda20 msgr2=0x7f0ddc0b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de1d9b700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ddc0cda20 0x7f0ddc0b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de1d9b700 1 -- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc153950 msgr2=0x7f0ddc1699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de1d9b700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc153950 0x7f0ddc1699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.761+0000 7f0de1d9b700 1 -- 172.21.15.31:0/2007938642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd4009ae0 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.762+0000 7f0de1d9b700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cd5d0 0x7f0ddc159c20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800ec80 tx=0x7f0dd8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.762+0000 7f0dd1ffb700 1 -- 172.21.15.31:0/2007938642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800c9d0 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.762+0000 7f0dd1ffb700 1 -- 172.21.15.31:0/2007938642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd800cb30 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.762+0000 7f0de3fff700 1 -- 172.21.15.31:0/2007938642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc0b3c30 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.762+0000 7f0dd1ffb700 1 -- 172.21.15.31:0/2007938642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8013470 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.762+0000 7f0de3fff700 1 -- 172.21.15.31:0/2007938642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ddc0b5fa0 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.763+0000 7f0de3fff700 1 -- 172.21.15.31:0/2007938642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ddc04c380 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.764+0000 7f0dd1ffb700 1 -- 172.21.15.31:0/2007938642 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f0dd8013860 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.765+0000 7f0dd1ffb700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0dc0067230 0x7f0dc00696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:08.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.765+0000 7f0dd1ffb700 1 -- 172.21.15.31:0/2007938642 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0dd8085210 con 0x7f0ddc0cd5d0 2023-10-06T05:23:08.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.766+0000 7f0de159a700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0dc0067230 0x7f0dc00696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:08.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.766+0000 7f0de159a700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0dc0067230 0x7f0dc00696e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009ca0 tx=0x7f0dcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:08.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:08.769+0000 7f0dd1ffb700 1 -- 172.21.15.31:0/2007938642 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0dd8051f90 con 0x7f0ddc0cd5d0 2023-10-06T05:23:09.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.008+0000 7f0de3fff700 1 -- 172.21.15.31:0/2007938642 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ddc15b7b0 con 0x7f0dc0067230 2023-10-06T05:23:09.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.025+0000 7f0dd1ffb700 1 -- 172.21.15.31:0/2007938642 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7f0ddc15b7b0 con 0x7f0dc0067230 2023-10-06T05:23:09.026 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:23:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fake" 2023-10-06T05:23:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_0000000000055671" 2023-10-06T05:23:09.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:09.027 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:23:09.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 -- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0dc0067230 msgr2=0x7f0dc00696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0dc0067230 0x7f0dc00696e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009ca0 tx=0x7f0dcc008040 comp rx=0 tx=0).stop 2023-10-06T05:23:09.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 -- 172.21.15.31:0/2007938642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cd5d0 msgr2=0x7f0ddc159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cd5d0 0x7f0ddc159c20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800ec80 tx=0x7f0dd8009120 comp rx=0 tx=0).stop 2023-10-06T05:23:09.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 -- 172.21.15.31:0/2007938642 shutdown_connections 2023-10-06T05:23:09.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0dc0067230 0x7f0dc00696e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc153950 0x7f0ddc1699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ddc0cd5d0 0x7f0ddc159c20 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 --2- 172.21.15.31:0/2007938642 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ddc0cda20 0x7f0ddc0b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 -- 172.21.15.31:0/2007938642 >> 172.21.15.31:0/2007938642 conn(0x7f0ddc0681b0 msgr2=0x7f0ddc150c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:09.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 -- 172.21.15.31:0/2007938642 shutdown_connections 2023-10-06T05:23:09.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.027+0000 7f0de3fff700 1 -- 172.21.15.31:0/2007938642 wait complete. 2023-10-06T05:23:09.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000063418 snapshot_0000000000055671 2023-10-06T05:23:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.245+0000 7f5d83bff700 1 Processor -- start 2023-10-06T05:23:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.245+0000 7f5d83bff700 1 -- start start 2023-10-06T05:23:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.246+0000 7f5d83bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d7c06df30 0x7f5d7c06d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.246+0000 7f5d83bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d7c06d830 0x7f5d7c159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d83bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7c0c72b0 con 0x7f5d7c06e300 2023-10-06T05:23:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d83bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7c0aba60 con 0x7f5d7c06d830 2023-10-06T05:23:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d83bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ccea0 con 0x7f5d7c06df30 2023-10-06T05:23:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d8119a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d7c06d830 0x7f5d7c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d8119a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d7c06d830 0x7f5d7c159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49982/0 (socket says 172.21.15.31:49982) 2023-10-06T05:23:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d8119a700 1 -- 172.21.15.31:0/4044361681 learned_addr learned my addr 172.21.15.31:0/4044361681 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d8199b700 1 --2- 172.21.15.31:0/4044361681 >> v2:172.21.15.62:3300/0 conn(0x7f5d7c06df30 0x7f5d7c06d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d8119a700 1 -- 172.21.15.31:0/4044361681 >> v2:172.21.15.62:3300/0 conn(0x7f5d7c06df30 msgr2=0x7f5d7c06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d8119a700 1 --2- 172.21.15.31:0/4044361681 >> v2:172.21.15.62:3300/0 conn(0x7f5d7c06df30 0x7f5d7c06d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d8119a700 1 -- 172.21.15.31:0/4044361681 >> v1:172.21.15.62:6789/0 conn(0x7f5d7c06e300 legacy=0x7f5d7c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d8119a700 1 -- 172.21.15.31:0/4044361681 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0c7590 con 0x7f5d7c06d830 2023-10-06T05:23:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.247+0000 7f5d8199b700 1 --2- 172.21.15.31:0/4044361681 >> v2:172.21.15.62:3300/0 conn(0x7f5d7c06df30 0x7f5d7c06d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.248+0000 7f5d8119a700 1 --2- 172.21.15.31:0/4044361681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06d830 0x7f5d7c159660 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0cbfc0 tx=0x7f5d6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ff77bfc021f1df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:09.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.248+0000 7f5d80999700 1 -- 172.21.15.31:0/4044361681 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00b860 con 0x7f5d7c06d830 2023-10-06T05:23:09.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.248+0000 7f5d80999700 1 -- 172.21.15.31:0/4044361681 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c00b9c0 con 0x7f5d7c06d830 2023-10-06T05:23:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.248+0000 7f5d80999700 1 -- 172.21.15.31:0/4044361681 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00bd60 con 0x7f5d7c06d830 2023-10-06T05:23:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.248+0000 7f5d83bff700 1 -- 172.21.15.31:0/4044361681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06d830 msgr2=0x7f5d7c159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.248+0000 7f5d83bff700 1 --2- 172.21.15.31:0/4044361681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06d830 0x7f5d7c159660 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0cbfc0 tx=0x7f5d6c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.248+0000 7f5d83bff700 1 -- 172.21.15.31:0/4044361681 shutdown_connections 2023-10-06T05:23:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.248+0000 7f5d83bff700 1 --2- 172.21.15.31:0/4044361681 >> v2:172.21.15.62:3300/0 conn(0x7f5d7c06df30 0x7f5d7c06d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.248+0000 7f5d83bff700 1 --2- 172.21.15.31:0/4044361681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06d830 0x7f5d7c159660 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.248+0000 7f5d83bff700 1 -- 172.21.15.31:0/4044361681 >> 172.21.15.31:0/4044361681 conn(0x7f5d7c068030 msgr2=0x7f5d7c0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.249+0000 7f5d83bff700 1 -- 172.21.15.31:0/4044361681 shutdown_connections 2023-10-06T05:23:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.249+0000 7f5d83bff700 1 -- 172.21.15.31:0/4044361681 wait complete. 2023-10-06T05:23:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.250+0000 7f5d83bff700 1 Processor -- start 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.250+0000 7f5d83bff700 1 -- start start 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.251+0000 7f5d83bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d7c06d830 0x7f5d7c0c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.251+0000 7f5d83bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06df30 0x7f5d7c1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.251+0000 7f5d83bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d7c06e300 0x7f5d7c0b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.251+0000 7f5d83bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0c13b0 con 0x7f5d7c06df30 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.251+0000 7f5d83bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ac000 con 0x7f5d7c06e300 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.251+0000 7f5d83bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ac2e0 con 0x7f5d7c06d830 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.251+0000 7f5d8119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06df30 0x7f5d7c1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.251+0000 7f5d8119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06df30 0x7f5d7c1543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49984/0 (socket says 172.21.15.31:49984) 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.251+0000 7f5d8119a700 1 -- 172.21.15.31:0/3819160577 learned_addr learned my addr 172.21.15.31:0/3819160577 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d8199b700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d7c06d830 0x7f5d7c0c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d8219c700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d7c06e300 0x7f5d7c0b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d8119a700 1 -- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d7c06d830 msgr2=0x7f5d7c0c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d8119a700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d7c06d830 0x7f5d7c0c2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d8119a700 1 -- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d7c06e300 msgr2=0x7f5d7c0b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d8119a700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d7c06e300 0x7f5d7c0b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d8119a700 1 -- 172.21.15.31:0/3819160577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c0094f0 con 0x7f5d7c06df30 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d8119a700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06df30 0x7f5d7c1543f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009290 tx=0x7f5d6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d71ffb700 1 -- 172.21.15.31:0/3819160577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c011e60 con 0x7f5d7c06df30 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d71ffb700 1 -- 172.21.15.31:0/3819160577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c01f070 con 0x7f5d7c06df30 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d71ffb700 1 -- 172.21.15.31:0/3819160577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c01a870 con 0x7f5d7c06df30 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.252+0000 7f5d83bff700 1 -- 172.21.15.31:0/3819160577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c0ac5c0 con 0x7f5d7c06df30 2023-10-06T05:23:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.253+0000 7f5d83bff700 1 -- 172.21.15.31:0/3819160577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c154af0 con 0x7f5d7c06df30 2023-10-06T05:23:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.254+0000 7f5d83bff700 1 -- 172.21.15.31:0/3819160577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7c0ac800 con 0x7f5d7c06df30 2023-10-06T05:23:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.255+0000 7f5d71ffb700 1 -- 172.21.15.31:0/3819160577 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5d6c01abe0 con 0x7f5d7c06df30 2023-10-06T05:23:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.255+0000 7f5d71ffb700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d60067200 0x7f5d600696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.255+0000 7f5d71ffb700 1 -- 172.21.15.31:0/3819160577 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d6c021070 con 0x7f5d7c06df30 2023-10-06T05:23:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.256+0000 7f5d8199b700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d60067200 0x7f5d600696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.256+0000 7f5d8199b700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d60067200 0x7f5d600696b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5d78006ea0 tx=0x7f5d78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.260+0000 7f5d71ffb700 1 -- 172.21.15.31:0/3819160577 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d7c0ac800 con 0x7f5d7c06df30 2023-10-06T05:23:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.504+0000 7f5d83bff700 1 -- 172.21.15.31:0/3819160577 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "snap_name": "snapshot_0000000000055671", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d7c0abe80 con 0x7f5d60067200 2023-10-06T05:23:09.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.538+0000 7f5d71ffb700 1 -- 172.21.15.31:0/3819160577 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5d780097b0 con 0x7f5d60067200 2023-10-06T05:23:09.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 -- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d60067200 msgr2=0x7f5d600696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d60067200 0x7f5d600696b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5d78006ea0 tx=0x7f5d78008040 comp rx=0 tx=0).stop 2023-10-06T05:23:09.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 -- 172.21.15.31:0/3819160577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06df30 msgr2=0x7f5d7c1543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06df30 0x7f5d7c1543f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009290 tx=0x7f5d6c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:09.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 -- 172.21.15.31:0/3819160577 shutdown_connections 2023-10-06T05:23:09.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d60067200 0x7f5d600696b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d7c06e300 0x7f5d7c0b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d7c06d830 0x7f5d7c0c2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 --2- 172.21.15.31:0/3819160577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d7c06df30 0x7f5d7c1543f0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 -- 172.21.15.31:0/3819160577 >> 172.21.15.31:0/3819160577 conn(0x7f5d7c068030 msgr2=0x7f5d7c0a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:09.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 -- 172.21.15.31:0/3819160577 shutdown_connections 2023-10-06T05:23:09.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.541+0000 7f5d83bff700 1 -- 172.21.15.31:0/3819160577 wait complete. 2023-10-06T05:23:09.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000063418 fake 2023-10-06T05:23:09.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.758+0000 7ff3c4fff700 1 Processor -- start 2023-10-06T05:23:09.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.759+0000 7ff3c4fff700 1 -- start start 2023-10-06T05:23:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.760+0000 7ff3c4fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0153950 0x7ff3c0153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.760+0000 7ff3c4fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3c015d920 0x7ff3c015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.760+0000 7ff3c4fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c015bb20 con 0x7ff3c00cd5d0 2023-10-06T05:23:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.760+0000 7ff3c4fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c015af10 con 0x7ff3c015d920 2023-10-06T05:23:09.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.760+0000 7ff3c4fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c015b1f0 con 0x7ff3c0153950 2023-10-06T05:23:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.760+0000 7ff3be59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0153950 0x7ff3c0153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.760+0000 7ff3bed9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3c015d920 0x7ff3c015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.761+0000 7ff3bed9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3c015d920 0x7ff3c015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:58626/0 (socket says 172.21.15.31:58626) 2023-10-06T05:23:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.761+0000 7ff3bed9d700 1 -- 172.21.15.31:0/2790447653 learned_addr learned my addr 172.21.15.31:0/2790447653 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.761+0000 7ff3be59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0153950 0x7ff3c0153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:49480/0 (socket says 172.21.15.31:49480) 2023-10-06T05:23:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.761+0000 7ff3bd59a700 1 -- 172.21.15.31:0/2790447653 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321575357 0 0) 0x7ff3c015bb20 con 0x7ff3c00cd5d0 2023-10-06T05:23:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.761+0000 7ff3bd59a700 1 -- 172.21.15.31:0/2790447653 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a8003570 con 0x7ff3c00cd5d0 2023-10-06T05:23:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.761+0000 7ff3be59c700 1 -- 172.21.15.31:0/2790447653 >> v2:172.21.15.62:3300/0 conn(0x7ff3c015d920 msgr2=0x7ff3c015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.761+0000 7ff3be59c700 1 --2- 172.21.15.31:0/2790447653 >> v2:172.21.15.62:3300/0 conn(0x7ff3c015d920 0x7ff3c015bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.761+0000 7ff3be59c700 1 -- 172.21.15.31:0/2790447653 >> v1:172.21.15.62:6789/0 conn(0x7ff3c00cd5d0 legacy=0x7ff3c015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.761+0000 7ff3be59c700 1 -- 172.21.15.31:0/2790447653 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c015b4d0 con 0x7ff3c0153950 2023-10-06T05:23:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.761+0000 7ff3bed9d700 1 --2- 172.21.15.31:0/2790447653 >> v2:172.21.15.62:3300/0 conn(0x7ff3c015d920 0x7ff3c015bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3be59c700 1 --2- 172.21.15.31:0/2790447653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0153950 0x7ff3c0153d20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00098d0 tx=0x7ff3b0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98d7f41fb05b5993 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3bd59a700 1 -- 172.21.15.31:0/2790447653 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0005c10 con 0x7ff3c0153950 2023-10-06T05:23:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3bd59a700 1 -- 172.21.15.31:0/2790447653 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b0005d70 con 0x7ff3c0153950 2023-10-06T05:23:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3bd59a700 1 -- 172.21.15.31:0/2790447653 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0006240 con 0x7ff3c0153950 2023-10-06T05:23:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3c4fff700 1 -- 172.21.15.31:0/2790447653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0153950 msgr2=0x7ff3c0153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3c4fff700 1 --2- 172.21.15.31:0/2790447653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0153950 0x7ff3c0153d20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00098d0 tx=0x7ff3b0009120 comp rx=0 tx=0).stop 2023-10-06T05:23:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3c4fff700 1 -- 172.21.15.31:0/2790447653 shutdown_connections 2023-10-06T05:23:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3c4fff700 1 --2- 172.21.15.31:0/2790447653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0153950 0x7ff3c0153d20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3c4fff700 1 --2- 172.21.15.31:0/2790447653 >> v2:172.21.15.62:3300/0 conn(0x7ff3c015d920 0x7ff3c015bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3c4fff700 1 -- 172.21.15.31:0/2790447653 >> 172.21.15.31:0/2790447653 conn(0x7ff3c00681b0 msgr2=0x7ff3c00685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3c4fff700 1 -- 172.21.15.31:0/2790447653 shutdown_connections 2023-10-06T05:23:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.762+0000 7ff3c4fff700 1 -- 172.21.15.31:0/2790447653 wait complete. 2023-10-06T05:23:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.765+0000 7ff3c4fff700 1 Processor -- start 2023-10-06T05:23:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.765+0000 7ff3c4fff700 1 -- start start 2023-10-06T05:23:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3c4fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd5d0 0x7ff3c00c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3c4fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0153950 0x7ff3c00c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3c4fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c015d920 0x7ff3c0156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3c4fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c00c8270 con 0x7ff3c0153950 2023-10-06T05:23:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3c4fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c00c7660 con 0x7ff3c00cd5d0 2023-10-06T05:23:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3c4fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c00c7940 con 0x7ff3c015d920 2023-10-06T05:23:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3bdd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0153950 0x7ff3c00c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3bdd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0153950 0x7ff3c00c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50000/0 (socket says 172.21.15.31:50000) 2023-10-06T05:23:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3bdd9b700 1 -- 172.21.15.31:0/3391648594 learned_addr learned my addr 172.21.15.31:0/3391648594 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3be59c700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd5d0 0x7ff3c00c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.766+0000 7ff3bed9d700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c015d920 0x7ff3c0156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.767+0000 7ff3bdd9b700 1 -- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c015d920 msgr2=0x7ff3c0156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.767+0000 7ff3bdd9b700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c015d920 0x7ff3c0156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.767+0000 7ff3bdd9b700 1 -- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd5d0 msgr2=0x7ff3c00c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.767+0000 7ff3bdd9b700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd5d0 0x7ff3c00c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.767+0000 7ff3bdd9b700 1 -- 172.21.15.31:0/3391648594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b8009ae0 con 0x7ff3c0153950 2023-10-06T05:23:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.767+0000 7ff3bdd9b700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0153950 0x7ff3c00c8700 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c590 tx=0x7ff3b4007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:09.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.767+0000 7ff3ae7fc700 1 -- 172.21.15.31:0/3391648594 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4010510 con 0x7ff3c0153950 2023-10-06T05:23:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.767+0000 7ff3ae7fc700 1 -- 172.21.15.31:0/3391648594 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b4010670 con 0x7ff3c0153950 2023-10-06T05:23:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.767+0000 7ff3ae7fc700 1 -- 172.21.15.31:0/3391648594 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4010a30 con 0x7ff3c0153950 2023-10-06T05:23:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.767+0000 7ff3c4fff700 1 -- 172.21.15.31:0/3391648594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b00093c0 con 0x7ff3c0153950 2023-10-06T05:23:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.768+0000 7ff3c4fff700 1 -- 172.21.15.31:0/3391648594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c00c7f00 con 0x7ff3c0153950 2023-10-06T05:23:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.769+0000 7ff3c4fff700 1 -- 172.21.15.31:0/3391648594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c00c8420 con 0x7ff3c0153950 2023-10-06T05:23:09.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.769+0000 7ff3ae7fc700 1 -- 172.21.15.31:0/3391648594 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ff3b401b400 con 0x7ff3c0153950 2023-10-06T05:23:09.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.771+0000 7ff3ae7fc700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3a0067200 0x7ff3a00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:09.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.771+0000 7ff3ae7fc700 1 -- 172.21.15.31:0/3391648594 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff3b40845a0 con 0x7ff3c0153950 2023-10-06T05:23:09.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.771+0000 7ff3be59c700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3a0067200 0x7ff3a00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:09.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.772+0000 7ff3be59c700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3a0067200 0x7ff3a00696b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009340 tx=0x7ff3b0016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:09.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:09.775+0000 7ff3ae7fc700 1 -- 172.21.15.31:0/3391648594 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3c00c8420 con 0x7ff3c0153950 2023-10-06T05:23:10.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.016+0000 7ff3c4fff700 1 -- 172.21.15.31:0/3391648594 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "snap_name": "fake", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3c004c380 con 0x7ff3a0067200 2023-10-06T05:23:10.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.052+0000 7ff3ae7fc700 1 -- 172.21.15.31:0/3391648594 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff3c004c380 con 0x7ff3a0067200 2023-10-06T05:23:10.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.054+0000 7ff3c4fff700 1 -- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3a0067200 msgr2=0x7ff3a00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.054+0000 7ff3c4fff700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3a0067200 0x7ff3a00696b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009340 tx=0x7ff3b0016000 comp rx=0 tx=0).stop 2023-10-06T05:23:10.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.054+0000 7ff3c4fff700 1 -- 172.21.15.31:0/3391648594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0153950 msgr2=0x7ff3c00c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.054+0000 7ff3c4fff700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0153950 0x7ff3c00c8700 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c590 tx=0x7ff3b4007dc0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.054+0000 7ff3c4fff700 1 -- 172.21.15.31:0/3391648594 shutdown_connections 2023-10-06T05:23:10.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.054+0000 7ff3c4fff700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3a0067200 0x7ff3a00696b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.054+0000 7ff3c4fff700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c015d920 0x7ff3c0156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.054+0000 7ff3c4fff700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd5d0 0x7ff3c00c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.054+0000 7ff3c4fff700 1 --2- 172.21.15.31:0/3391648594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0153950 0x7ff3c00c8700 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.054+0000 7ff3c4fff700 1 -- 172.21.15.31:0/3391648594 >> 172.21.15.31:0/3391648594 conn(0x7ff3c00681b0 msgr2=0x7ff3c00b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:10.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.055+0000 7ff3c4fff700 1 -- 172.21.15.31:0/3391648594 shutdown_connections 2023-10-06T05:23:10.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.055+0000 7ff3c4fff700 1 -- 172.21.15.31:0/3391648594 wait complete. 2023-10-06T05:23:10.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:10.074 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvolume_0000000000063418/.meta) 2023-10-06T05:23:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:23:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T05:23:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T05:23:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000063418/141542f8-7142-400e-81a0-91665a5d46f1 2023-10-06T05:23:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:23:10.150 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:10.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000063418 2023-10-06T05:23:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.348+0000 7f11355ff700 1 Processor -- start 2023-10-06T05:23:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.349+0000 7f11355ff700 1 -- start start 2023-10-06T05:23:10.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f11355ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11301573a0 0x7f11300acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f11355ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f11300ae860 0x7f11301611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f11355ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1130158cf0 con 0x7f1130157770 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f11355ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11301580e0 con 0x7f11301573a0 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f11355ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11301583c0 con 0x7f11300ae860 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f112effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11301573a0 0x7f11300acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f112effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11301573a0 0x7f11300acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50016/0 (socket says 172.21.15.31:50016) 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f112effd700 1 -- 172.21.15.31:0/3483108006 learned_addr learned my addr 172.21.15.31:0/3483108006 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f112f7fe700 1 --2- 172.21.15.31:0/3483108006 >> v2:172.21.15.62:3300/0 conn(0x7f11300ae860 0x7f11301611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f112effd700 1 -- 172.21.15.31:0/3483108006 >> v2:172.21.15.62:3300/0 conn(0x7f11300ae860 msgr2=0x7f11301611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.350+0000 7f112effd700 1 --2- 172.21.15.31:0/3483108006 >> v2:172.21.15.62:3300/0 conn(0x7f11300ae860 0x7f11301611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.351+0000 7f112effd700 1 -- 172.21.15.31:0/3483108006 >> v1:172.21.15.62:6789/0 conn(0x7f1130157770 legacy=0x7f11300b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.351+0000 7f112effd700 1 -- 172.21.15.31:0/3483108006 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11301586a0 con 0x7f11301573a0 2023-10-06T05:23:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.351+0000 7f112effd700 1 --2- 172.21.15.31:0/3483108006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 0x7f11300acb70 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1130158c60 tx=0x7f1120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df369525f41d046f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.351+0000 7f112dffb700 1 -- 172.21.15.31:0/3483108006 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000b860 con 0x7f11301573a0 2023-10-06T05:23:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.351+0000 7f112dffb700 1 -- 172.21.15.31:0/3483108006 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112000b9c0 con 0x7f11301573a0 2023-10-06T05:23:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.351+0000 7f112dffb700 1 -- 172.21.15.31:0/3483108006 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000bd60 con 0x7f11301573a0 2023-10-06T05:23:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.352+0000 7f11355ff700 1 -- 172.21.15.31:0/3483108006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 msgr2=0x7f11300acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.352+0000 7f11355ff700 1 --2- 172.21.15.31:0/3483108006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 0x7f11300acb70 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1130158c60 tx=0x7f1120009250 comp rx=0 tx=0).stop 2023-10-06T05:23:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.352+0000 7f11355ff700 1 -- 172.21.15.31:0/3483108006 shutdown_connections 2023-10-06T05:23:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.352+0000 7f11355ff700 1 --2- 172.21.15.31:0/3483108006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 0x7f11300acb70 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.352+0000 7f11355ff700 1 --2- 172.21.15.31:0/3483108006 >> v2:172.21.15.62:3300/0 conn(0x7f11300ae860 0x7f11301611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.352+0000 7f11355ff700 1 -- 172.21.15.31:0/3483108006 >> 172.21.15.31:0/3483108006 conn(0x7f1130067560 msgr2=0x7f1130067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.352+0000 7f11355ff700 1 -- 172.21.15.31:0/3483108006 shutdown_connections 2023-10-06T05:23:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.352+0000 7f11355ff700 1 -- 172.21.15.31:0/3483108006 wait complete. 2023-10-06T05:23:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.353+0000 7f11355ff700 1 Processor -- start 2023-10-06T05:23:10.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.353+0000 7f11355ff700 1 -- start start 2023-10-06T05:23:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.354+0000 7f11355ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 0x7f113016a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.354+0000 7f11355ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1130157770 0x7f113015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.354+0000 7f11355ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11300ca180 0x7f1130156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.354+0000 7f11355ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11300cb510 con 0x7f11301573a0 2023-10-06T05:23:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.354+0000 7f11355ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11300ca960 con 0x7f11300ca180 2023-10-06T05:23:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.354+0000 7f11355ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11300cac40 con 0x7f1130157770 2023-10-06T05:23:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.354+0000 7f112effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 0x7f113016a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f112effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 0x7f113016a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50024/0 (socket says 172.21.15.31:50024) 2023-10-06T05:23:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f112effd700 1 -- 172.21.15.31:0/505643662 learned_addr learned my addr 172.21.15.31:0/505643662 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f112effd700 1 -- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1130157770 msgr2=0x7f113015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f112effd700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1130157770 0x7f113015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f112effd700 1 -- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11300ca180 msgr2=0x7f1130156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f112effd700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11300ca180 0x7f1130156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f112effd700 1 -- 172.21.15.31:0/505643662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11200094f0 con 0x7f11301573a0 2023-10-06T05:23:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f112effd700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 0x7f113016a180 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1130064290 tx=0x7f1120017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f1116ffd700 1 -- 172.21.15.31:0/505643662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120011e60 con 0x7f11301573a0 2023-10-06T05:23:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f1116ffd700 1 -- 172.21.15.31:0/505643662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112001a630 con 0x7f11301573a0 2023-10-06T05:23:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.355+0000 7f1116ffd700 1 -- 172.21.15.31:0/505643662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112001a9f0 con 0x7f11301573a0 2023-10-06T05:23:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.356+0000 7f11355ff700 1 -- 172.21.15.31:0/505643662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11300caf20 con 0x7f11301573a0 2023-10-06T05:23:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.356+0000 7f11355ff700 1 -- 172.21.15.31:0/505643662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11300ca620 con 0x7f11301573a0 2023-10-06T05:23:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.357+0000 7f11355ff700 1 -- 172.21.15.31:0/505643662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113004c380 con 0x7f11301573a0 2023-10-06T05:23:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.358+0000 7f1116ffd700 1 -- 172.21.15.31:0/505643662 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f112002d440 con 0x7f11301573a0 2023-10-06T05:23:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.359+0000 7f1116ffd700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f111c067200 0x7f111c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.359+0000 7f1116ffd700 1 -- 172.21.15.31:0/505643662 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1120086690 con 0x7f11301573a0 2023-10-06T05:23:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.359+0000 7f112e7fc700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f111c067200 0x7f111c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.360+0000 7f112e7fc700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f111c067200 0x7f111c0696b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f11301607f0 tx=0x7f1124007b40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.363+0000 7f1116ffd700 1 -- 172.21.15.31:0/505643662 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1120053450 con 0x7f11301573a0 2023-10-06T05:23:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.594+0000 7f11355ff700 1 -- 172.21.15.31:0/505643662 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000063418", "target": ["mon-mgr", ""]}) v1 -- 0x7f11300b1fb0 con 0x7f111c067200 2023-10-06T05:23:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.641+0000 7f1116ffd700 1 -- 172.21.15.31:0/505643662 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f11300b1fb0 con 0x7f111c067200 2023-10-06T05:23:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 -- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f111c067200 msgr2=0x7f111c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f111c067200 0x7f111c0696b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f11301607f0 tx=0x7f1124007b40 comp rx=0 tx=0).stop 2023-10-06T05:23:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 -- 172.21.15.31:0/505643662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 msgr2=0x7f113016a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 0x7f113016a180 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1130064290 tx=0x7f1120017000 comp rx=0 tx=0).stop 2023-10-06T05:23:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 -- 172.21.15.31:0/505643662 shutdown_connections 2023-10-06T05:23:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f111c067200 0x7f111c0696b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11300ca180 0x7f1130156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11301573a0 0x7f113016a180 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 --2- 172.21.15.31:0/505643662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1130157770 0x7f113015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 -- 172.21.15.31:0/505643662 >> 172.21.15.31:0/505643662 conn(0x7f1130067560 msgr2=0x7f113006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 -- 172.21.15.31:0/505643662 shutdown_connections 2023-10-06T05:23:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.644+0000 7f11355ff700 1 -- 172.21.15.31:0/505643662 wait complete. 2023-10-06T05:23:10.648 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:10.646+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:10.648 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:10.646+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:10.648 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:10.646+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:10.648 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:10.646+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:10.648 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:10.646+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:10.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs clone_0000000000030936 2023-10-06T05:23:10.666 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:10.665+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:10.667 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:10.665+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:10.667 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:10.665+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:10.667 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:10.665+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:10.667 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:10.665+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:10.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.863+0000 7f3f079ff700 1 Processor -- start 2023-10-06T05:23:10.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.863+0000 7f3f079ff700 1 -- start start 2023-10-06T05:23:10.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.864+0000 7f3f079ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f000caeb0 0x7f3f000cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.864+0000 7f3f079ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f000cc1d0 0x7f3f000afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.864+0000 7f3f079ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f00069380 con 0x7f3f000b10c0 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.864+0000 7f3f079ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f00068770 con 0x7f3f000cc1d0 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.864+0000 7f3f079ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f00068a50 con 0x7f3f000caeb0 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.865+0000 7f3f04f9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3f000b10c0 0x7f3f00159230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35356/0 (socket says 172.21.15.31:35356) 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.865+0000 7f3f04f9a700 1 -- 172.21.15.31:0/3113331411 learned_addr learned my addr 172.21.15.31:0/3113331411 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.865+0000 7f3f05f9c700 1 --2- 172.21.15.31:0/3113331411 >> v2:172.21.15.31:3300/0 conn(0x7f3f000cc1d0 0x7f3f000afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.865+0000 7f3f0579b700 1 --2- 172.21.15.31:0/3113331411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f000caeb0 0x7f3f000cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.865+0000 7f3ef7fff700 1 -- 172.21.15.31:0/3113331411 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826032427 0 0) 0x7f3f00069380 con 0x7f3f000b10c0 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.865+0000 7f3ef7fff700 1 -- 172.21.15.31:0/3113331411 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3eec003570 con 0x7f3f000b10c0 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.865+0000 7f3f05f9c700 1 -- 172.21.15.31:0/3113331411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f000caeb0 msgr2=0x7f3f000cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.865+0000 7f3f05f9c700 1 --2- 172.21.15.31:0/3113331411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f000caeb0 0x7f3f000cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.865+0000 7f3f05f9c700 1 -- 172.21.15.31:0/3113331411 >> v1:172.21.15.31:6789/0 conn(0x7f3f000b10c0 legacy=0x7f3f00159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.865+0000 7f3f05f9c700 1 -- 172.21.15.31:0/3113331411 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f00068d30 con 0x7f3f000cc1d0 2023-10-06T05:23:10.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3f05f9c700 1 --2- 172.21.15.31:0/3113331411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000cc1d0 0x7f3f000afea0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009e60 tx=0x7f3ef8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a8ff192965eb3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3ef7fff700 1 -- 172.21.15.31:0/3113331411 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef800e070 con 0x7f3f000cc1d0 2023-10-06T05:23:10.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3ef7fff700 1 -- 172.21.15.31:0/3113331411 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef800bab0 con 0x7f3f000cc1d0 2023-10-06T05:23:10.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3ef7fff700 1 -- 172.21.15.31:0/3113331411 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef800be70 con 0x7f3f000cc1d0 2023-10-06T05:23:10.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3f079ff700 1 -- 172.21.15.31:0/3113331411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000cc1d0 msgr2=0x7f3f000afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3f079ff700 1 --2- 172.21.15.31:0/3113331411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000cc1d0 0x7f3f000afea0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009e60 tx=0x7f3ef8009780 comp rx=0 tx=0).stop 2023-10-06T05:23:10.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3f079ff700 1 -- 172.21.15.31:0/3113331411 shutdown_connections 2023-10-06T05:23:10.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3f079ff700 1 --2- 172.21.15.31:0/3113331411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000cc1d0 0x7f3f000afea0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3f079ff700 1 --2- 172.21.15.31:0/3113331411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f000caeb0 0x7f3f000cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3f079ff700 1 -- 172.21.15.31:0/3113331411 >> 172.21.15.31:0/3113331411 conn(0x7f3f00097550 msgr2=0x7f3f000639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:10.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.866+0000 7f3f079ff700 1 -- 172.21.15.31:0/3113331411 shutdown_connections 2023-10-06T05:23:10.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.867+0000 7f3f079ff700 1 -- 172.21.15.31:0/3113331411 wait complete. 2023-10-06T05:23:10.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.868+0000 7f3f079ff700 1 Processor -- start 2023-10-06T05:23:10.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.868+0000 7f3f079ff700 1 -- start start 2023-10-06T05:23:10.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.869+0000 7f3f079ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f000b10c0 0x7f3f000c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.869+0000 7f3f079ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000caeb0 0x7f3f000c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.869+0000 7f3f079ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f000cc1d0 0x7f3f000c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.869+0000 7f3f079ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000c6250 con 0x7f3f000caeb0 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.869+0000 7f3f079ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0015c830 con 0x7f3f000cc1d0 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.869+0000 7f3f079ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0015bc20 con 0x7f3f000b10c0 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.869+0000 7f3f04f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000caeb0 0x7f3f000c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.869+0000 7f3f04f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000caeb0 0x7f3f000c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50048/0 (socket says 172.21.15.31:50048) 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.869+0000 7f3f04f9a700 1 -- 172.21.15.31:0/3429346424 learned_addr learned my addr 172.21.15.31:0/3429346424 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f04f9a700 1 -- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f000b10c0 msgr2=0x7f3f000c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f0579b700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f000b10c0 0x7f3f000c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f05f9c700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f000cc1d0 0x7f3f000c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f04f9a700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f000b10c0 0x7f3f000c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f04f9a700 1 -- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f000cc1d0 msgr2=0x7f3f000c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f04f9a700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f000cc1d0 0x7f3f000c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f04f9a700 1 -- 172.21.15.31:0/3429346424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8009a20 con 0x7f3f000caeb0 2023-10-06T05:23:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f05f9c700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f000cc1d0 0x7f3f000c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f04f9a700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000caeb0 0x7f3f000c7f40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c7c0 tx=0x7f3ef0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3ef57fa700 1 -- 172.21.15.31:0/3429346424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0005b30 con 0x7f3f000caeb0 2023-10-06T05:23:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3ef57fa700 1 -- 172.21.15.31:0/3429346424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef0005c90 con 0x7f3f000caeb0 2023-10-06T05:23:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3ef57fa700 1 -- 172.21.15.31:0/3429346424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0007340 con 0x7f3f000caeb0 2023-10-06T05:23:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f079ff700 1 -- 172.21.15.31:0/3429346424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0015bf00 con 0x7f3f000caeb0 2023-10-06T05:23:10.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.870+0000 7f3f079ff700 1 -- 172.21.15.31:0/3429346424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0015c4c0 con 0x7f3f000caeb0 2023-10-06T05:23:10.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.871+0000 7f3f079ff700 1 -- 172.21.15.31:0/3429346424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0015c9c0 con 0x7f3f000caeb0 2023-10-06T05:23:10.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.872+0000 7f3ef57fa700 1 -- 172.21.15.31:0/3429346424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f3ef001b400 con 0x7f3f000caeb0 2023-10-06T05:23:10.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.873+0000 7f3ef57fa700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ee4067200 0x7f3ee40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:10.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.873+0000 7f3ef57fa700 1 -- 172.21.15.31:0/3429346424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3ef0084630 con 0x7f3f000caeb0 2023-10-06T05:23:10.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.873+0000 7f3f0579b700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ee4067200 0x7f3ee40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.874+0000 7f3f0579b700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ee4067200 0x7f3ee40696b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009390 tx=0x7f3efc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:10.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:10.877+0000 7f3ef57fa700 1 -- 172.21.15.31:0/3429346424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ef0051430 con 0x7f3f000caeb0 2023-10-06T05:23:11.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.109+0000 7f3f079ff700 1 -- 172.21.15.31:0/3429346424 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "clone_0000000000030936", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f0004c3d0 con 0x7f3ee4067200 2023-10-06T05:23:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.129+0000 7f3ef57fa700 1 -- 172.21.15.31:0/3429346424 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3f0004c3d0 con 0x7f3ee4067200 2023-10-06T05:23:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 -- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ee4067200 msgr2=0x7f3ee40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ee4067200 0x7f3ee40696b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009390 tx=0x7f3efc008040 comp rx=0 tx=0).stop 2023-10-06T05:23:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 -- 172.21.15.31:0/3429346424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000caeb0 msgr2=0x7f3f000c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000caeb0 0x7f3f000c7f40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c7c0 tx=0x7f3ef0007e00 comp rx=0 tx=0).stop 2023-10-06T05:23:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 -- 172.21.15.31:0/3429346424 shutdown_connections 2023-10-06T05:23:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ee4067200 0x7f3ee40696b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f000cc1d0 0x7f3f000c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f000b10c0 0x7f3f000c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 --2- 172.21.15.31:0/3429346424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f000caeb0 0x7f3f000c7f40 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 -- 172.21.15.31:0/3429346424 >> 172.21.15.31:0/3429346424 conn(0x7f3f00097550 msgr2=0x7f3f000b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 -- 172.21.15.31:0/3429346424 shutdown_connections 2023-10-06T05:23:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.132+0000 7f3f079ff700 1 -- 172.21.15.31:0/3429346424 wait complete. 2023-10-06T05:23:11.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs clone_0000000000030937 2023-10-06T05:23:11.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.348+0000 7f0883dff700 1 Processor -- start 2023-10-06T05:23:11.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.349+0000 7f0883dff700 1 -- start start 2023-10-06T05:23:11.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.350+0000 7f0883dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f087c15ca40 0x7f087c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.350+0000 7f0883dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c068da0 con 0x7f087c0c8f40 2023-10-06T05:23:11.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.350+0000 7f0883dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c069080 con 0x7f087c06a300 2023-10-06T05:23:11.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.350+0000 7f0883dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c0cabc0 con 0x7f087c15ca40 2023-10-06T05:23:11.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.350+0000 7f0881b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f087c15ca40 0x7f087c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.350+0000 7f0881b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f087c15ca40 0x7f087c068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50056/0 (socket says 172.21.15.31:50056) 2023-10-06T05:23:11.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.350+0000 7f0881b9b700 1 -- 172.21.15.31:0/148471375 learned_addr learned my addr 172.21.15.31:0/148471375 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.351+0000 7f0881b9b700 1 -- 172.21.15.31:0/148471375 >> v1:172.21.15.62:6789/0 conn(0x7f087c06a300 legacy=0x7f087c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.351+0000 7f0881b9b700 1 -- 172.21.15.31:0/148471375 >> v1:172.21.15.31:6789/0 conn(0x7f087c0c8f40 legacy=0x7f087c0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.351+0000 7f0881b9b700 1 -- 172.21.15.31:0/148471375 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c069360 con 0x7f087c15ca40 2023-10-06T05:23:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.351+0000 7f0880b99700 1 -- 172.21.15.31:0/148471375 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735261826 0 0) 0x7f087c068da0 con 0x7f087c0c8f40 2023-10-06T05:23:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.351+0000 7f0881b9b700 1 --2- 172.21.15.31:0/148471375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 0x7f087c068670 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f08780099b0 tx=0x7f0878009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb0c33294ff07fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.351+0000 7f0880b99700 1 -- 172.21.15.31:0/148471375 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800b860 con 0x7f087c15ca40 2023-10-06T05:23:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.351+0000 7f0880b99700 1 -- 172.21.15.31:0/148471375 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800b9c0 con 0x7f087c15ca40 2023-10-06T05:23:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.351+0000 7f0880b99700 1 -- 172.21.15.31:0/148471375 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800bd60 con 0x7f087c15ca40 2023-10-06T05:23:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.352+0000 7f0883dff700 1 -- 172.21.15.31:0/148471375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 msgr2=0x7f087c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.352+0000 7f0883dff700 1 --2- 172.21.15.31:0/148471375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 0x7f087c068670 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f08780099b0 tx=0x7f0878009250 comp rx=0 tx=0).stop 2023-10-06T05:23:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.352+0000 7f0883dff700 1 -- 172.21.15.31:0/148471375 shutdown_connections 2023-10-06T05:23:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.352+0000 7f0883dff700 1 --2- 172.21.15.31:0/148471375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 0x7f087c068670 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.352+0000 7f0883dff700 1 -- 172.21.15.31:0/148471375 >> 172.21.15.31:0/148471375 conn(0x7f087c0ce290 msgr2=0x7f087c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.352+0000 7f0883dff700 1 -- 172.21.15.31:0/148471375 shutdown_connections 2023-10-06T05:23:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.352+0000 7f0883dff700 1 -- 172.21.15.31:0/148471375 wait complete. 2023-10-06T05:23:11.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.353+0000 7f0883dff700 1 Processor -- start 2023-10-06T05:23:11.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.353+0000 7f0883dff700 1 -- start start 2023-10-06T05:23:11.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.354+0000 7f0883dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f087c06a300 0x7f087c0c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.354+0000 7f0883dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087c0c8f40 0x7f087c0c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.354+0000 7f0883dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 0x7f087c1596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.354+0000 7f0883dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0c31c0 con 0x7f087c15ca40 2023-10-06T05:23:11.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.354+0000 7f0883dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0adfa0 con 0x7f087c06a300 2023-10-06T05:23:11.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.354+0000 7f0883dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0ae840 con 0x7f087c0c8f40 2023-10-06T05:23:11.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f088239c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 0x7f087c1596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f0881b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f087c06a300 0x7f087c0c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f088239c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 0x7f087c1596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50064/0 (socket says 172.21.15.31:50064) 2023-10-06T05:23:11.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f088239c700 1 -- 172.21.15.31:0/3075863391 learned_addr learned my addr 172.21.15.31:0/3075863391 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:11.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f088139a700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087c0c8f40 0x7f087c0c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f088239c700 1 -- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087c0c8f40 msgr2=0x7f087c0c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f088239c700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087c0c8f40 0x7f087c0c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f088239c700 1 -- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f087c06a300 msgr2=0x7f087c0c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f088239c700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f087c06a300 0x7f087c0c2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f088239c700 1 -- 172.21.15.31:0/3075863391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780094f0 con 0x7f087c15ca40 2023-10-06T05:23:11.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.355+0000 7f088239c700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 0x7f087c1596b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f087400c930 tx=0x7f08740016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.356+0000 7f0871ffb700 1 -- 172.21.15.31:0/3075863391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874012630 con 0x7f087c15ca40 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.356+0000 7f0871ffb700 1 -- 172.21.15.31:0/3075863391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0874012790 con 0x7f087c15ca40 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.356+0000 7f0871ffb700 1 -- 172.21.15.31:0/3075863391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874012b50 con 0x7f087c15ca40 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.356+0000 7f0883dff700 1 -- 172.21.15.31:0/3075863391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c0c6d20 con 0x7f087c15ca40 2023-10-06T05:23:11.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.356+0000 7f0883dff700 1 -- 172.21.15.31:0/3075863391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c0ae1f0 con 0x7f087c15ca40 2023-10-06T05:23:11.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.357+0000 7f0883dff700 1 -- 172.21.15.31:0/3075863391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c04c380 con 0x7f087c15ca40 2023-10-06T05:23:11.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.358+0000 7f0871ffb700 1 -- 172.21.15.31:0/3075863391 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f087401a3a0 con 0x7f087c15ca40 2023-10-06T05:23:11.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.359+0000 7f0871ffb700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0860067200 0x7f08600696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:11.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.359+0000 7f0871ffb700 1 -- 172.21.15.31:0/3075863391 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0874084d60 con 0x7f087c15ca40 2023-10-06T05:23:11.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.359+0000 7f0881b9b700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0860067200 0x7f08600696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:11.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.360+0000 7f0881b9b700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0860067200 0x7f08600696b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f08780099b0 tx=0x7f0878017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:11.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.363+0000 7f0871ffb700 1 -- 172.21.15.31:0/3075863391 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0874051ae0 con 0x7f087c15ca40 2023-10-06T05:23:11.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.598+0000 7f0883dff700 1 -- 172.21.15.31:0/3075863391 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "clone_0000000000030937", "target": ["mon-mgr", ""]}) v1 -- 0x7f087c0ca1a0 con 0x7f0860067200 2023-10-06T05:23:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.618+0000 7f0871ffb700 1 -- 172.21.15.31:0/3075863391 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f087c0ca1a0 con 0x7f0860067200 2023-10-06T05:23:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 -- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0860067200 msgr2=0x7f08600696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0860067200 0x7f08600696b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f08780099b0 tx=0x7f0878017000 comp rx=0 tx=0).stop 2023-10-06T05:23:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 -- 172.21.15.31:0/3075863391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 msgr2=0x7f087c1596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 0x7f087c1596b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f087400c930 tx=0x7f08740016f0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 -- 172.21.15.31:0/3075863391 shutdown_connections 2023-10-06T05:23:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0860067200 0x7f08600696b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f087c15ca40 0x7f087c1596b0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f087c06a300 0x7f087c0c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 --2- 172.21.15.31:0/3075863391 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087c0c8f40 0x7f087c0c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 -- 172.21.15.31:0/3075863391 >> 172.21.15.31:0/3075863391 conn(0x7f087c0ce290 msgr2=0x7f087c15e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 -- 172.21.15.31:0/3075863391 shutdown_connections 2023-10-06T05:23:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:11.621+0000 7f0883dff700 1 -- 172.21.15.31:0/3075863391 wait complete. 2023-10-06T05:23:11.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:11.640 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T05:23:11.729 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T05:23:11.730 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:23:11.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:11.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:11.940 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:23:11.940 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:23:11.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:11.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:12.051 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:23:12.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:12.051 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:12.101 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:23:12.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:12.101 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:12.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:23:12.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:12.198 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:23:12.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:12.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.440+0000 7fbc2f5ff700 1 Processor -- start 2023-10-06T05:23:12.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.440+0000 7fbc2f5ff700 1 -- start start 2023-10-06T05:23:12.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc2f5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc28159500 0x7fbc280b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc2f5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbc280cd510 0x7fbc280aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc2f5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc280c3f40 con 0x7fbc2815d2d0 2023-10-06T05:23:12.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc2f5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc280b2860 con 0x7fbc280cd510 2023-10-06T05:23:12.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc2f5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2815d7d0 con 0x7fbc28159500 2023-10-06T05:23:12.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc2db9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbc280cd510 0x7fbc280aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc2cb9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbc2815d2d0 0x7fbc280cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35364/0 (socket says 172.21.15.31:35364) 2023-10-06T05:23:12.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc2cb9a700 1 -- 172.21.15.31:0/1548362218 learned_addr learned my addr 172.21.15.31:0/1548362218 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:12.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc2d39b700 1 --2- 172.21.15.31:0/1548362218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc28159500 0x7fbc280b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc1ffff700 1 -- 172.21.15.31:0/1548362218 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485119834 0 0) 0x7fbc280c3f40 con 0x7fbc2815d2d0 2023-10-06T05:23:12.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.442+0000 7fbc1ffff700 1 -- 172.21.15.31:0/1548362218 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc14003570 con 0x7fbc2815d2d0 2023-10-06T05:23:12.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.443+0000 7fbc2d39b700 1 -- 172.21.15.31:0/1548362218 >> v2:172.21.15.31:3300/0 conn(0x7fbc280cd510 msgr2=0x7fbc280aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.443+0000 7fbc2d39b700 1 --2- 172.21.15.31:0/1548362218 >> v2:172.21.15.31:3300/0 conn(0x7fbc280cd510 0x7fbc280aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.443+0000 7fbc2d39b700 1 -- 172.21.15.31:0/1548362218 >> v1:172.21.15.31:6789/0 conn(0x7fbc2815d2d0 legacy=0x7fbc280cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.443+0000 7fbc2d39b700 1 -- 172.21.15.31:0/1548362218 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc280cb000 con 0x7fbc28159500 2023-10-06T05:23:12.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.443+0000 7fbc2db9c700 1 --2- 172.21.15.31:0/1548362218 >> v2:172.21.15.31:3300/0 conn(0x7fbc280cd510 0x7fbc280aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:12.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.443+0000 7fbc2d39b700 1 --2- 172.21.15.31:0/1548362218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc28159500 0x7fbc280b0ab0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbc240098e0 tx=0x7fbc240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d36f905f3525cf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.443+0000 7fbc1ffff700 1 -- 172.21.15.31:0/1548362218 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24011630 con 0x7fbc28159500 2023-10-06T05:23:12.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.443+0000 7fbc1ffff700 1 -- 172.21.15.31:0/1548362218 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc24011790 con 0x7fbc28159500 2023-10-06T05:23:12.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.443+0000 7fbc1ffff700 1 -- 172.21.15.31:0/1548362218 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24011b30 con 0x7fbc28159500 2023-10-06T05:23:12.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.444+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/1548362218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc28159500 msgr2=0x7fbc280b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.444+0000 7fbc2f5ff700 1 --2- 172.21.15.31:0/1548362218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc28159500 0x7fbc280b0ab0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbc240098e0 tx=0x7fbc240091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.444+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/1548362218 shutdown_connections 2023-10-06T05:23:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.444+0000 7fbc2f5ff700 1 --2- 172.21.15.31:0/1548362218 >> v2:172.21.15.31:3300/0 conn(0x7fbc280cd510 0x7fbc280aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.444+0000 7fbc2f5ff700 1 --2- 172.21.15.31:0/1548362218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc28159500 0x7fbc280b0ab0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.444+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/1548362218 >> 172.21.15.31:0/1548362218 conn(0x7fbc2806ec90 msgr2=0x7fbc2806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.444+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/1548362218 shutdown_connections 2023-10-06T05:23:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.444+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/1548362218 wait complete. 2023-10-06T05:23:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.446+0000 7fbc2f5ff700 1 Processor -- start 2023-10-06T05:23:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.446+0000 7fbc2f5ff700 1 -- start start 2023-10-06T05:23:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2f5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc280cd510 0x7fbc280c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2f5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc28159500 0x7fbc281598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2f5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2815d2d0 0x7fbc28159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2f5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc280c84c0 con 0x7fbc2815d2d0 2023-10-06T05:23:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2f5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc280c78b0 con 0x7fbc28159500 2023-10-06T05:23:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2f5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc280c7b90 con 0x7fbc280cd510 2023-10-06T05:23:12.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2db9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2815d2d0 0x7fbc28159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2db9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2815d2d0 0x7fbc28159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50072/0 (socket says 172.21.15.31:50072) 2023-10-06T05:23:12.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2db9c700 1 -- 172.21.15.31:0/204645825 learned_addr learned my addr 172.21.15.31:0/204645825 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:12.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2cb9a700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc28159500 0x7fbc281598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2d39b700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc280cd510 0x7fbc280c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2db9c700 1 -- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc280cd510 msgr2=0x7fbc280c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2db9c700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc280cd510 0x7fbc280c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2db9c700 1 -- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc28159500 msgr2=0x7fbc281598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2db9c700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc28159500 0x7fbc281598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.447+0000 7fbc2db9c700 1 -- 172.21.15.31:0/204645825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc20009a20 con 0x7fbc2815d2d0 2023-10-06T05:23:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.448+0000 7fbc2db9c700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2815d2d0 0x7fbc28159e10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009f60 tx=0x7fbc2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.448+0000 7fbc1d7fa700 1 -- 172.21.15.31:0/204645825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2000cb70 con 0x7fbc2815d2d0 2023-10-06T05:23:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.448+0000 7fbc1d7fa700 1 -- 172.21.15.31:0/204645825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2000f040 con 0x7fbc2815d2d0 2023-10-06T05:23:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.448+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/204645825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc24009420 con 0x7fbc2815d2d0 2023-10-06T05:23:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.448+0000 7fbc1d7fa700 1 -- 172.21.15.31:0/204645825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20014480 con 0x7fbc2815d2d0 2023-10-06T05:23:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.448+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/204645825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc280c8150 con 0x7fbc2815d2d0 2023-10-06T05:23:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.449+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/204645825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2804c380 con 0x7fbc2815d2d0 2023-10-06T05:23:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.450+0000 7fbc1d7fa700 1 -- 172.21.15.31:0/204645825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fbc20003020 con 0x7fbc2815d2d0 2023-10-06T05:23:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.451+0000 7fbc1d7fa700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc0c067200 0x7fbc0c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.451+0000 7fbc1d7fa700 1 -- 172.21.15.31:0/204645825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbc20085570 con 0x7fbc2815d2d0 2023-10-06T05:23:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.451+0000 7fbc2d39b700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc0c067200 0x7fbc0c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.452+0000 7fbc2d39b700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc0c067200 0x7fbc0c0696b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc240091e0 tx=0x7fbc24017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.455+0000 7fbc1d7fa700 1 -- 172.21.15.31:0/204645825 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc200521e0 con 0x7fbc2815d2d0 2023-10-06T05:23:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.701+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/204645825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc280c8600 con 0x7fbc2815d2d0 2023-10-06T05:23:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.702+0000 7fbc1d7fa700 1 -- 172.21.15.31:0/204645825 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fbc20055800 con 0x7fbc2815d2d0 2023-10-06T05:23:12.703 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:12.704 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1885821393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1885821393},{"type":"v1","addr":"172.21.15.31:6834","nonce":1885821393}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6889,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/258742412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":258742412},{"type":"v1","addr":"172.21.15.31:6835","nonce":258742412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4070877337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4070877337},{"type":"v1","addr":"172.21.15.62:6834","nonce":4070877337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.638176+0000","modified":"2023-10-06T05:22:36.157996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7059":{"gid":7059,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2094863015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2094863015},{"type":"v1","addr":"172.21.15.62:6835","nonce":2094863015}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc0c067200 msgr2=0x7fbc0c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc0c067200 0x7fbc0c0696b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc240091e0 tx=0x7fbc24017000 comp rx=0 tx=0).stop 2023-10-06T05:23:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/204645825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2815d2d0 msgr2=0x7fbc28159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2815d2d0 0x7fbc28159e10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009f60 tx=0x7fbc2000b040 comp rx=0 tx=0).stop 2023-10-06T05:23:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/204645825 shutdown_connections 2023-10-06T05:23:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc0c067200 0x7fbc0c0696b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2815d2d0 0x7fbc28159e10 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc280cd510 0x7fbc280c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 --2- 172.21.15.31:0/204645825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc28159500 0x7fbc281598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/204645825 >> 172.21.15.31:0/204645825 conn(0x7fbc2806ec90 msgr2=0x7fbc280aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/204645825 shutdown_connections 2023-10-06T05:23:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.704+0000 7fbc2f5ff700 1 -- 172.21.15.31:0/204645825 wait complete. 2023-10-06T05:23:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 68 2023-10-06T05:23:12.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:12.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.917+0000 7fad3cdff700 1 Processor -- start 2023-10-06T05:23:12.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.917+0000 7fad3cdff700 1 -- start start 2023-10-06T05:23:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.918+0000 7fad3cdff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad3813fb00 0x7fad3806c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.918+0000 7fad3cdff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fad3813fed0 0x7fad3806e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.918+0000 7fad3cdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3806ea90 0x7fad380cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.918+0000 7fad3cdff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3806ef30 con 0x7fad3813fb00 2023-10-06T05:23:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.918+0000 7fad3cdff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3806be70 con 0x7fad3813fed0 2023-10-06T05:23:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.918+0000 7fad3cdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3806f130 con 0x7fad3806ea90 2023-10-06T05:23:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.918+0000 7fad3659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad3813fb00 0x7fad3806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.919+0000 7fad3659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad3813fb00 0x7fad3806c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50088/0 (socket says 172.21.15.31:50088) 2023-10-06T05:23:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.919+0000 7fad3659c700 1 -- 172.21.15.31:0/2888008301 learned_addr learned my addr 172.21.15.31:0/2888008301 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.919+0000 7fad36d9d700 1 --2- 172.21.15.31:0/2888008301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3806ea90 0x7fad380cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.919+0000 7fad35d9b700 1 --2- 172.21.15.31:0/2888008301 >> v2:172.21.15.62:3300/0 conn(0x7fad3813fed0 0x7fad3806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.919+0000 7fad3659c700 1 -- 172.21.15.31:0/2888008301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3806ea90 msgr2=0x7fad380cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.919+0000 7fad3659c700 1 --2- 172.21.15.31:0/2888008301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3806ea90 0x7fad380cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.919+0000 7fad3659c700 1 -- 172.21.15.31:0/2888008301 >> v2:172.21.15.62:3300/0 conn(0x7fad3813fed0 msgr2=0x7fad3806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.919+0000 7fad3659c700 1 --2- 172.21.15.31:0/2888008301 >> v2:172.21.15.62:3300/0 conn(0x7fad3813fed0 0x7fad3806e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.919+0000 7fad3659c700 1 -- 172.21.15.31:0/2888008301 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3806f410 con 0x7fad3813fb00 2023-10-06T05:23:12.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.919+0000 7fad3659c700 1 --2- 172.21.15.31:0/2888008301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3813fb00 0x7fad3806c300 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fad380710f0 tx=0x7fad28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edded30d486c6ee3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3559a700 1 -- 172.21.15.31:0/2888008301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2800b860 con 0x7fad3813fb00 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3559a700 1 -- 172.21.15.31:0/2888008301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2800b9c0 con 0x7fad3813fb00 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3559a700 1 -- 172.21.15.31:0/2888008301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2800bd60 con 0x7fad3813fb00 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3cdff700 1 -- 172.21.15.31:0/2888008301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3813fb00 msgr2=0x7fad3806c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3cdff700 1 --2- 172.21.15.31:0/2888008301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3813fb00 0x7fad3806c300 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fad380710f0 tx=0x7fad28009250 comp rx=0 tx=0).stop 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3cdff700 1 -- 172.21.15.31:0/2888008301 shutdown_connections 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3cdff700 1 --2- 172.21.15.31:0/2888008301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3806ea90 0x7fad380cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3cdff700 1 --2- 172.21.15.31:0/2888008301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3813fb00 0x7fad3806c300 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3cdff700 1 --2- 172.21.15.31:0/2888008301 >> v2:172.21.15.62:3300/0 conn(0x7fad3813fed0 0x7fad3806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3cdff700 1 -- 172.21.15.31:0/2888008301 >> 172.21.15.31:0/2888008301 conn(0x7fad380cdcb0 msgr2=0x7fad380c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3cdff700 1 -- 172.21.15.31:0/2888008301 shutdown_connections 2023-10-06T05:23:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.920+0000 7fad3cdff700 1 -- 172.21.15.31:0/2888008301 wait complete. 2023-10-06T05:23:12.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.922+0000 7fad3cdff700 1 Processor -- start 2023-10-06T05:23:12.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.922+0000 7fad3cdff700 1 -- start start 2023-10-06T05:23:12.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.923+0000 7fad3cdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3806ea90 0x7fad380c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.923+0000 7fad3cdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3813fb00 0x7fad38156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.923+0000 7fad3cdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3813fed0 0x7fad380b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.923+0000 7fad3cdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad38157460 con 0x7fad3806ea90 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.923+0000 7fad3cdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380b25e0 con 0x7fad3813fb00 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.923+0000 7fad3cdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad380b28c0 con 0x7fad3813fed0 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.923+0000 7fad3659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3806ea90 0x7fad380c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.923+0000 7fad3659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3806ea90 0x7fad380c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50104/0 (socket says 172.21.15.31:50104) 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.923+0000 7fad3659c700 1 -- 172.21.15.31:0/34603518 learned_addr learned my addr 172.21.15.31:0/34603518 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad36d9d700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3813fed0 0x7fad380b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad35d9b700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3813fb00 0x7fad38156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad3659c700 1 -- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3813fed0 msgr2=0x7fad380b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad3659c700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3813fed0 0x7fad380b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad3659c700 1 -- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3813fb00 msgr2=0x7fad38156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:12.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad3659c700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3813fb00 0x7fad38156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:12.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad3659c700 1 -- 172.21.15.31:0/34603518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280094f0 con 0x7fad3806ea90 2023-10-06T05:23:12.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad36d9d700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3813fed0 0x7fad380b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:12.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad35d9b700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3813fb00 0x7fad38156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:12.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad3659c700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3806ea90 0x7fad380c5560 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fad3806da70 tx=0x7fad28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad267fc700 1 -- 172.21.15.31:0/34603518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28024440 con 0x7fad3806ea90 2023-10-06T05:23:12.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad267fc700 1 -- 172.21.15.31:0/34603518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad280245a0 con 0x7fad3806ea90 2023-10-06T05:23:12.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad267fc700 1 -- 172.21.15.31:0/34603518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28024960 con 0x7fad3806ea90 2023-10-06T05:23:12.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad3cdff700 1 -- 172.21.15.31:0/34603518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad380b2ba0 con 0x7fad3806ea90 2023-10-06T05:23:12.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.924+0000 7fad3cdff700 1 -- 172.21.15.31:0/34603518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad38169a20 con 0x7fad3806ea90 2023-10-06T05:23:12.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.925+0000 7fad3cdff700 1 -- 172.21.15.31:0/34603518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad38169be0 con 0x7fad3806ea90 2023-10-06T05:23:12.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.926+0000 7fad267fc700 1 -- 172.21.15.31:0/34603518 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fad28024d50 con 0x7fad3806ea90 2023-10-06T05:23:12.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.927+0000 7fad267fc700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad18067200 0x7fad180696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:12.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.927+0000 7fad267fc700 1 -- 172.21.15.31:0/34603518 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fad280866e0 con 0x7fad3806ea90 2023-10-06T05:23:12.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.928+0000 7fad35d9b700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad18067200 0x7fad180696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:12.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.929+0000 7fad35d9b700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad18067200 0x7fad180696b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad2c006aa0 tx=0x7fad2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:12.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:12.932+0000 7fad267fc700 1 -- 172.21.15.31:0/34603518 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad28053460 con 0x7fad3806ea90 2023-10-06T05:23:13.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.134+0000 7fad3cdff700 1 -- 172.21.15.31:0/34603518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad3804c380 con 0x7fad3806ea90 2023-10-06T05:23:13.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.136+0000 7fad267fc700 1 -- 172.21.15.31:0/34603518 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16920 (secure 0 0 0) 0x7fad2802b340 con 0x7fad3806ea90 2023-10-06T05:23:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:13.138 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":101,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:22:34.142984+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.850575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.607953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.138+0000 7fad3cdff700 1 -- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad18067200 msgr2=0x7fad180696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad18067200 0x7fad180696b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad2c006aa0 tx=0x7fad2c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 -- 172.21.15.31:0/34603518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3806ea90 msgr2=0x7fad380c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3806ea90 0x7fad380c5560 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fad3806da70 tx=0x7fad28017000 comp rx=0 tx=0).stop 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 -- 172.21.15.31:0/34603518 shutdown_connections 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad18067200 0x7fad180696b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad3813fed0 0x7fad380b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad3806ea90 0x7fad380c5560 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 --2- 172.21.15.31:0/34603518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3813fb00 0x7fad38156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 -- 172.21.15.31:0/34603518 >> 172.21.15.31:0/34603518 conn(0x7fad380cdcb0 msgr2=0x7fad3813e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 -- 172.21.15.31:0/34603518 shutdown_connections 2023-10-06T05:23:13.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.139+0000 7fad3cdff700 1 -- 172.21.15.31:0/34603518 wait complete. 2023-10-06T05:23:13.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:22:30.850575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:22:31.607953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:23:13.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:23:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.375+0000 7fbf9c9ff700 1 Processor -- start 2023-10-06T05:23:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.375+0000 7fbf9c9ff700 1 -- start start 2023-10-06T05:23:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.376+0000 7fbf9c9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbf98159560 0x7fbf980cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.376+0000 7fbf9c9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9815d2d0 con 0x7fbf980b2b80 2023-10-06T05:23:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.376+0000 7fbf9c9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9815d5b0 con 0x7fbf98159930 2023-10-06T05:23:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.376+0000 7fbf9c9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9815d890 con 0x7fbf98159560 2023-10-06T05:23:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.376+0000 7fbf96d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbf980b2b80 0x7fbf980cb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35376/0 (socket says 172.21.15.31:35376) 2023-10-06T05:23:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.377+0000 7fbf96d9d700 1 -- 172.21.15.31:0/2157005391 learned_addr learned my addr 172.21.15.31:0/2157005391 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.377+0000 7fbf9659c700 1 --2- 172.21.15.31:0/2157005391 >> v2:172.21.15.62:3300/0 conn(0x7fbf98159560 0x7fbf980cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.377+0000 7fbf9559a700 1 -- 172.21.15.31:0/2157005391 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423958372 0 0) 0x7fbf9815d2d0 con 0x7fbf980b2b80 2023-10-06T05:23:13.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.377+0000 7fbf9559a700 1 -- 172.21.15.31:0/2157005391 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf80003570 con 0x7fbf980b2b80 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.377+0000 7fbf9559a700 1 -- 172.21.15.31:0/2157005391 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815227330 0 0) 0x7fbf9815d5b0 con 0x7fbf98159930 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.377+0000 7fbf9559a700 1 -- 172.21.15.31:0/2157005391 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf9815d2d0 con 0x7fbf98159930 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.377+0000 7fbf9659c700 1 -- 172.21.15.31:0/2157005391 >> v1:172.21.15.62:6789/0 conn(0x7fbf98159930 legacy=0x7fbf980b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.377+0000 7fbf9659c700 1 -- 172.21.15.31:0/2157005391 >> v1:172.21.15.31:6789/0 conn(0x7fbf980b2b80 legacy=0x7fbf980cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.377+0000 7fbf9659c700 1 -- 172.21.15.31:0/2157005391 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9815db70 con 0x7fbf98159560 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.378+0000 7fbf9659c700 1 --2- 172.21.15.31:0/2157005391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf98159560 0x7fbf980cce70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbf880098e0 tx=0x7fbf880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc4c974b9f9374e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.378+0000 7fbf9559a700 1 -- 172.21.15.31:0/2157005391 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88011630 con 0x7fbf98159560 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.378+0000 7fbf9559a700 1 -- 172.21.15.31:0/2157005391 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf88011790 con 0x7fbf98159560 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.378+0000 7fbf9559a700 1 -- 172.21.15.31:0/2157005391 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88011b30 con 0x7fbf98159560 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.378+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/2157005391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf98159560 msgr2=0x7fbf980cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.378+0000 7fbf9c9ff700 1 --2- 172.21.15.31:0/2157005391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf98159560 0x7fbf980cce70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbf880098e0 tx=0x7fbf880091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.379+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/2157005391 shutdown_connections 2023-10-06T05:23:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.379+0000 7fbf9c9ff700 1 --2- 172.21.15.31:0/2157005391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf98159560 0x7fbf980cce70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.379+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/2157005391 >> 172.21.15.31:0/2157005391 conn(0x7fbf9806ec90 msgr2=0x7fbf9806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.379+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/2157005391 shutdown_connections 2023-10-06T05:23:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.379+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/2157005391 wait complete. 2023-10-06T05:23:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.380+0000 7fbf9c9ff700 1 Processor -- start 2023-10-06T05:23:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.380+0000 7fbf9c9ff700 1 -- start start 2023-10-06T05:23:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.381+0000 7fbf9c9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf980b2b80 0x7fbf980c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.381+0000 7fbf9c9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf98159560 0x7fbf980cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.381+0000 7fbf9c9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf98159930 0x7fbf980cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.381+0000 7fbf9c9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf980b4b20 con 0x7fbf980b2b80 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.381+0000 7fbf9c9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf98155890 con 0x7fbf98159930 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.381+0000 7fbf9c9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf980b3c00 con 0x7fbf98159560 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.381+0000 7fbf9659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf980b2b80 0x7fbf980c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.381+0000 7fbf9659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf980b2b80 0x7fbf980c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50112/0 (socket says 172.21.15.31:50112) 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.381+0000 7fbf9659c700 1 -- 172.21.15.31:0/4100290807 learned_addr learned my addr 172.21.15.31:0/4100290807 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf96d9d700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf98159930 0x7fbf980cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf95d9b700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf98159560 0x7fbf980cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf9659c700 1 -- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf98159560 msgr2=0x7fbf980cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf9659c700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf98159560 0x7fbf980cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf9659c700 1 -- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf98159930 msgr2=0x7fbf980cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf9659c700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf98159930 0x7fbf980cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf9659c700 1 -- 172.21.15.31:0/4100290807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf88009420 con 0x7fbf980b2b80 2023-10-06T05:23:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf9659c700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf980b2b80 0x7fbf980c9d30 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbf980893d0 tx=0x7fbf88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf867fc700 1 -- 172.21.15.31:0/4100290807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8801f070 con 0x7fbf980b2b80 2023-10-06T05:23:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf867fc700 1 -- 172.21.15.31:0/4100290807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf88011b10 con 0x7fbf980b2b80 2023-10-06T05:23:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/4100290807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf980b3ea0 con 0x7fbf980b2b80 2023-10-06T05:23:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.382+0000 7fbf867fc700 1 -- 172.21.15.31:0/4100290807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88023640 con 0x7fbf980b2b80 2023-10-06T05:23:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.383+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/4100290807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf980b4460 con 0x7fbf980b2b80 2023-10-06T05:23:13.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.384+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/4100290807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9804c380 con 0x7fbf980b2b80 2023-10-06T05:23:13.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.384+0000 7fbf867fc700 1 -- 172.21.15.31:0/4100290807 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbf8802c3b0 con 0x7fbf980b2b80 2023-10-06T05:23:13.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.385+0000 7fbf867fc700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf78067320 0x7fbf780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.385+0000 7fbf867fc700 1 -- 172.21.15.31:0/4100290807 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6114+0+0 (secure 0 0 0) 0x7fbf88086130 con 0x7fbf980b2b80 2023-10-06T05:23:13.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.386+0000 7fbf95d9b700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf78067320 0x7fbf780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.386+0000 7fbf95d9b700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf78067320 0x7fbf780697d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbf980b0fe0 tx=0x7fbf8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.390+0000 7fbf867fc700 1 -- 172.21.15.31:0/4100290807 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf88052be0 con 0x7fbf980b2b80 2023-10-06T05:23:13.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.591+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/4100290807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbf9815dee0 con 0x7fbf980b2b80 2023-10-06T05:23:13.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.592+0000 7fbf867fc700 1 -- 172.21.15.31:0/4100290807 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v102) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbf88056200 con 0x7fbf980b2b80 2023-10-06T05:23:13.594 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:23:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf78067320 msgr2=0x7fbf780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf78067320 0x7fbf780697d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbf980b0fe0 tx=0x7fbf8c009040 comp rx=0 tx=0).stop 2023-10-06T05:23:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/4100290807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf980b2b80 msgr2=0x7fbf980c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf980b2b80 0x7fbf980c9d30 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbf980893d0 tx=0x7fbf88017000 comp rx=0 tx=0).stop 2023-10-06T05:23:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/4100290807 shutdown_connections 2023-10-06T05:23:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf78067320 0x7fbf780697d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf98159930 0x7fbf980cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf980b2b80 0x7fbf980c9d30 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 --2- 172.21.15.31:0/4100290807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf98159560 0x7fbf980cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/4100290807 >> 172.21.15.31:0/4100290807 conn(0x7fbf9806ec90 msgr2=0x7fbf98159d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/4100290807 shutdown_connections 2023-10-06T05:23:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.594+0000 7fbf9c9ff700 1 -- 172.21.15.31:0/4100290807 wait complete. 2023-10-06T05:23:13.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:23:13.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.811+0000 7fcd9efff700 1 Processor -- start 2023-10-06T05:23:13.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.811+0000 7fcd9efff700 1 -- start start 2023-10-06T05:23:13.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.812+0000 7fcd9efff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcd98153d40 0x7fcd980b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.812+0000 7fcd9efff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd980af890 0x7fcd980adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.812+0000 7fcd9efff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd980b0e40 con 0x7fcd980b09c0 2023-10-06T05:23:13.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.812+0000 7fcd9efff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd980afcd0 con 0x7fcd98153d40 2023-10-06T05:23:13.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.812+0000 7fcd9efff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980ad690 con 0x7fcd980af890 2023-10-06T05:23:13.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.813+0000 7fcd9cd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcd98153d40 0x7fcd980b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.813+0000 7fcd97fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcd980b09c0 0x7fcd980af180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35388/0 (socket says 172.21.15.31:35388) 2023-10-06T05:23:13.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.813+0000 7fcd97fff700 1 -- 172.21.15.31:0/2678809911 learned_addr learned my addr 172.21.15.31:0/2678809911 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:13.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.813+0000 7fcd9d59c700 1 --2- 172.21.15.31:0/2678809911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd980af890 0x7fcd980adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.813+0000 7fcd9cd9b700 1 -- 172.21.15.31:0/2678809911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd980af890 msgr2=0x7fcd980adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:13.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.813+0000 7fcd9cd9b700 1 --2- 172.21.15.31:0/2678809911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd980af890 0x7fcd980adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.813+0000 7fcd9cd9b700 1 -- 172.21.15.31:0/2678809911 >> v1:172.21.15.31:6789/0 conn(0x7fcd980b09c0 legacy=0x7fcd980af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.813+0000 7fcd9cd9b700 1 -- 172.21.15.31:0/2678809911 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd980acd60 con 0x7fcd98153d40 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.813+0000 7fcd977fe700 1 -- 172.21.15.31:0/2678809911 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501312751 0 0) 0x7fcd84001110 con 0x7fcd980b09c0 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.813+0000 7fcd9cd9b700 1 --2- 172.21.15.31:0/2678809911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd98153d40 0x7fcd980b0480 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0098e0 tx=0x7fcd8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bd0b8b6f8f6a74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd977fe700 1 -- 172.21.15.31:0/2678809911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c009e80 con 0x7fcd98153d40 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd977fe700 1 -- 172.21.15.31:0/2678809911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c00b860 con 0x7fcd98153d40 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd977fe700 1 -- 172.21.15.31:0/2678809911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00bbe0 con 0x7fcd98153d40 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd9efff700 1 -- 172.21.15.31:0/2678809911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd98153d40 msgr2=0x7fcd980b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd9efff700 1 --2- 172.21.15.31:0/2678809911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd98153d40 0x7fcd980b0480 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0098e0 tx=0x7fcd8c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd9efff700 1 -- 172.21.15.31:0/2678809911 shutdown_connections 2023-10-06T05:23:13.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd9efff700 1 --2- 172.21.15.31:0/2678809911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd98153d40 0x7fcd980b0480 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd9efff700 1 --2- 172.21.15.31:0/2678809911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd980af890 0x7fcd980adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd9efff700 1 -- 172.21.15.31:0/2678809911 >> 172.21.15.31:0/2678809911 conn(0x7fcd98067560 msgr2=0x7fcd98067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:13.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd9efff700 1 -- 172.21.15.31:0/2678809911 shutdown_connections 2023-10-06T05:23:13.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.814+0000 7fcd9efff700 1 -- 172.21.15.31:0/2678809911 wait complete. 2023-10-06T05:23:13.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.816+0000 7fcd9efff700 1 Processor -- start 2023-10-06T05:23:13.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.816+0000 7fcd9efff700 1 -- start start 2023-10-06T05:23:13.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.817+0000 7fcd9efff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd980af890 0x7fcd9815c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.817+0000 7fcd9efff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd980b09c0 0x7fcd981683c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.817+0000 7fcd9efff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd98153d40 0x7fcd9814a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.817+0000 7fcd9efff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980cad70 con 0x7fcd980af890 2023-10-06T05:23:13.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.817+0000 7fcd9efff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9815c830 con 0x7fcd980b09c0 2023-10-06T05:23:13.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.817+0000 7fcd9efff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd98168900 con 0x7fcd98153d40 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.817+0000 7fcd9cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd980af890 0x7fcd9815c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.817+0000 7fcd9cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd980af890 0x7fcd9815c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50120/0 (socket says 172.21.15.31:50120) 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.817+0000 7fcd9cd9b700 1 -- 172.21.15.31:0/2886969782 learned_addr learned my addr 172.21.15.31:0/2886969782 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd97fff700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd980b09c0 0x7fcd981683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd9d59c700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd98153d40 0x7fcd9814a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd9cd9b700 1 -- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd98153d40 msgr2=0x7fcd9814a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd9cd9b700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd98153d40 0x7fcd9814a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd9cd9b700 1 -- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd980b09c0 msgr2=0x7fcd981683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd9cd9b700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd980b09c0 0x7fcd981683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd9cd9b700 1 -- 172.21.15.31:0/2886969782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c009420 con 0x7fcd980af890 2023-10-06T05:23:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd9d59c700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd98153d40 0x7fcd9814a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:13.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd9cd9b700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd980af890 0x7fcd9815c2f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fcd9806f430 tx=0x7fcd8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd94ff9700 1 -- 172.21.15.31:0/2886969782 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00bbe0 con 0x7fcd980af890 2023-10-06T05:23:13.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd94ff9700 1 -- 172.21.15.31:0/2886969782 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c011420 con 0x7fcd980af890 2023-10-06T05:23:13.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd94ff9700 1 -- 172.21.15.31:0/2886969782 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c0117e0 con 0x7fcd980af890 2023-10-06T05:23:13.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd9efff700 1 -- 172.21.15.31:0/2886969782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9814b0c0 con 0x7fcd980af890 2023-10-06T05:23:13.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.818+0000 7fcd9efff700 1 -- 172.21.15.31:0/2886969782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9814b680 con 0x7fcd980af890 2023-10-06T05:23:13.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.819+0000 7fcd9efff700 1 -- 172.21.15.31:0/2886969782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9804c380 con 0x7fcd980af890 2023-10-06T05:23:13.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.820+0000 7fcd94ff9700 1 -- 172.21.15.31:0/2886969782 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcd8c011bd0 con 0x7fcd980af890 2023-10-06T05:23:13.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.822+0000 7fcd94ff9700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd7c067320 0x7fcd7c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:13.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.822+0000 7fcd94ff9700 1 -- 172.21.15.31:0/2886969782 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6114+0+0 (secure 0 0 0) 0x7fcd8c086830 con 0x7fcd980af890 2023-10-06T05:23:13.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.822+0000 7fcd97fff700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd7c067320 0x7fcd7c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:13.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.823+0000 7fcd97fff700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd7c067320 0x7fcd7c0697d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcd9814f150 tx=0x7fcd84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:13.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:13.826+0000 7fcd94ff9700 1 -- 172.21.15.31:0/2886969782 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd8c0532e0 con 0x7fcd980af890 2023-10-06T05:23:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.028+0000 7fcd9efff700 1 -- 172.21.15.31:0/2886969782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd980ad040 con 0x7fcd980af890 2023-10-06T05:23:14.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.028+0000 7fcd94ff9700 1 -- 172.21.15.31:0/2886969782 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd8c056900 con 0x7fcd980af890 2023-10-06T05:23:14.030 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:23:14.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 -- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd7c067320 msgr2=0x7fcd7c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd7c067320 0x7fcd7c0697d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcd9814f150 tx=0x7fcd84009040 comp rx=0 tx=0).stop 2023-10-06T05:23:14.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 -- 172.21.15.31:0/2886969782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd980af890 msgr2=0x7fcd9815c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd980af890 0x7fcd9815c2f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fcd9806f430 tx=0x7fcd8c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:14.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 -- 172.21.15.31:0/2886969782 shutdown_connections 2023-10-06T05:23:14.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd7c067320 0x7fcd7c0697d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd98153d40 0x7fcd9814a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd980af890 0x7fcd9815c2f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 --2- 172.21.15.31:0/2886969782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd980b09c0 0x7fcd981683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 -- 172.21.15.31:0/2886969782 >> 172.21.15.31:0/2886969782 conn(0x7fcd98067560 msgr2=0x7fcd98152c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 -- 172.21.15.31:0/2886969782 shutdown_connections 2023-10-06T05:23:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.031+0000 7fcd9efff700 1 -- 172.21.15.31:0/2886969782 wait complete. 2023-10-06T05:23:14.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:23:14.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.248+0000 7fd35ddff700 1 Processor -- start 2023-10-06T05:23:14.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.248+0000 7fd35ddff700 1 -- start start 2023-10-06T05:23:14.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.249+0000 7fd35ddff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd3580b0b90 0x7fd3580af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.249+0000 7fd35ddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3580af6c0 0x7fd3580adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.249+0000 7fd35ddff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3580ad690 con 0x7fd358153d40 2023-10-06T05:23:14.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.249+0000 7fd35ddff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3580acd60 con 0x7fd3580b0b90 2023-10-06T05:23:14.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.249+0000 7fd35ddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580ad040 con 0x7fd3580af6c0 2023-10-06T05:23:14.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.250+0000 7fd356ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd3580b0b90 0x7fd3580af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.250+0000 7fd357fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3580af6c0 0x7fd3580adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.250+0000 7fd356ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd3580b0b90 0x7fd3580af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:58708/0 (socket says 172.21.15.31:58708) 2023-10-06T05:23:14.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.250+0000 7fd356ffd700 1 -- 172.21.15.31:0/2412581497 learned_addr learned my addr 172.21.15.31:0/2412581497 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:14.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.250+0000 7fd3577fe700 1 --1- 172.21.15.31:0/2412581497 >> v1:172.21.15.62:6789/0 conn(0x7fd358153d40 0x7fd3580b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:46592/0 (socket says 172.21.15.31:46592) 2023-10-06T05:23:14.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.250+0000 7fd356ffd700 1 -- 172.21.15.31:0/2412581497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3580af6c0 msgr2=0x7fd3580adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.250+0000 7fd356ffd700 1 --2- 172.21.15.31:0/2412581497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3580af6c0 0x7fd3580adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.250+0000 7fd356ffd700 1 -- 172.21.15.31:0/2412581497 >> v1:172.21.15.62:6789/0 conn(0x7fd358153d40 legacy=0x7fd3580b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.250+0000 7fd356ffd700 1 -- 172.21.15.31:0/2412581497 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580ad320 con 0x7fd3580b0b90 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.250+0000 7fd356ffd700 1 --2- 172.21.15.31:0/2412581497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3580b0b90 0x7fd3580af180 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0099b0 tx=0x7fd34c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6c9b19cac6961b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd3567fc700 1 -- 172.21.15.31:0/2412581497 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c009ea0 con 0x7fd3580b0b90 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd3567fc700 1 -- 172.21.15.31:0/2412581497 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c00b860 con 0x7fd3580b0b90 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd3567fc700 1 -- 172.21.15.31:0/2412581497 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00bc00 con 0x7fd3580b0b90 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd35ddff700 1 -- 172.21.15.31:0/2412581497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3580b0b90 msgr2=0x7fd3580af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd35ddff700 1 --2- 172.21.15.31:0/2412581497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3580b0b90 0x7fd3580af180 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0099b0 tx=0x7fd34c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd35ddff700 1 -- 172.21.15.31:0/2412581497 shutdown_connections 2023-10-06T05:23:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd35ddff700 1 --2- 172.21.15.31:0/2412581497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3580af6c0 0x7fd3580adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd35ddff700 1 --2- 172.21.15.31:0/2412581497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3580b0b90 0x7fd3580af180 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd35ddff700 1 -- 172.21.15.31:0/2412581497 >> 172.21.15.31:0/2412581497 conn(0x7fd358067560 msgr2=0x7fd358067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd35ddff700 1 -- 172.21.15.31:0/2412581497 shutdown_connections 2023-10-06T05:23:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.251+0000 7fd35ddff700 1 -- 172.21.15.31:0/2412581497 wait complete. 2023-10-06T05:23:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.253+0000 7fd35ddff700 1 Processor -- start 2023-10-06T05:23:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.253+0000 7fd35ddff700 1 -- start start 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd35ddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3580af6c0 0x7fd358141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd35ddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3580b0b90 0x7fd3580c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd35ddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358153d40 0x7fd3580c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd35ddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580c2210 con 0x7fd358153d40 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd35ddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3581420d0 con 0x7fd3580b0b90 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd35ddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580c9000 con 0x7fd3580af6c0 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd357fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358153d40 0x7fd3580c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd3577fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3580af6c0 0x7fd358141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd357fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358153d40 0x7fd3580c2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50122/0 (socket says 172.21.15.31:50122) 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd357fff700 1 -- 172.21.15.31:0/60947216 learned_addr learned my addr 172.21.15.31:0/60947216 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.254+0000 7fd356ffd700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3580b0b90 0x7fd3580c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd357fff700 1 -- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3580af6c0 msgr2=0x7fd358141b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd357fff700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3580af6c0 0x7fd358141b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd357fff700 1 -- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3580b0b90 msgr2=0x7fd3580c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd357fff700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3580b0b90 0x7fd3580c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd357fff700 1 -- 172.21.15.31:0/60947216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c0094f0 con 0x7fd358153d40 2023-10-06T05:23:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd357fff700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358153d40 0x7fd3580c2830 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd350009ec0 tx=0x7fd35000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd33f7fe700 1 -- 172.21.15.31:0/60947216 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35000cb70 con 0x7fd358153d40 2023-10-06T05:23:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd33f7fe700 1 -- 172.21.15.31:0/60947216 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35000f040 con 0x7fd358153d40 2023-10-06T05:23:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd35ddff700 1 -- 172.21.15.31:0/60947216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3580c1970 con 0x7fd358153d40 2023-10-06T05:23:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd33f7fe700 1 -- 172.21.15.31:0/60947216 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350014480 con 0x7fd358153d40 2023-10-06T05:23:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.255+0000 7fd35ddff700 1 -- 172.21.15.31:0/60947216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3580c1f30 con 0x7fd358153d40 2023-10-06T05:23:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.256+0000 7fd35ddff700 1 -- 172.21.15.31:0/60947216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3580c8dd0 con 0x7fd358153d40 2023-10-06T05:23:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.257+0000 7fd33f7fe700 1 -- 172.21.15.31:0/60947216 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd3500147f0 con 0x7fd358153d40 2023-10-06T05:23:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.259+0000 7fd33f7fe700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd344067320 0x7fd3440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.259+0000 7fd33f7fe700 1 -- 172.21.15.31:0/60947216 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6114+0+0 (secure 0 0 0) 0x7fd350085b00 con 0x7fd358153d40 2023-10-06T05:23:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.259+0000 7fd3577fe700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd344067320 0x7fd3440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.260+0000 7fd3577fe700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd344067320 0x7fd3440697d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd348004f40 tx=0x7fd348007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.263+0000 7fd33f7fe700 1 -- 172.21.15.31:0/60947216 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3580c8dd0 con 0x7fd358153d40 2023-10-06T05:23:14.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.464+0000 7fd35ddff700 1 -- 172.21.15.31:0/60947216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3580c8dd0 con 0x7fd358153d40 2023-10-06T05:23:14.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.465+0000 7fd33f7fe700 1 -- 172.21.15.31:0/60947216 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd3580c8dd0 con 0x7fd358153d40 2023-10-06T05:23:14.467 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:23:14.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 -- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd344067320 msgr2=0x7fd3440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd344067320 0x7fd3440697d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd348004f40 tx=0x7fd348007eb0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 -- 172.21.15.31:0/60947216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358153d40 msgr2=0x7fd3580c2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358153d40 0x7fd3580c2830 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd350009ec0 tx=0x7fd35000b040 comp rx=0 tx=0).stop 2023-10-06T05:23:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 -- 172.21.15.31:0/60947216 shutdown_connections 2023-10-06T05:23:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd344067320 0x7fd3440697d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358153d40 0x7fd3580c2830 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3580af6c0 0x7fd358141b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 --2- 172.21.15.31:0/60947216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3580b0b90 0x7fd3580c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 -- 172.21.15.31:0/60947216 >> 172.21.15.31:0/60947216 conn(0x7fd358067560 msgr2=0x7fd358152ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.467+0000 7fd35ddff700 1 -- 172.21.15.31:0/60947216 shutdown_connections 2023-10-06T05:23:14.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.468+0000 7fd35ddff700 1 -- 172.21.15.31:0/60947216 wait complete. 2023-10-06T05:23:14.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.682+0000 7f21e33ff700 1 Processor -- start 2023-10-06T05:23:14.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.682+0000 7f21e33ff700 1 -- start start 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.683+0000 7f21e33ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21dc0c8f40 0x7f21dc068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.683+0000 7f21e33ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc068da0 con 0x7f21dc06a300 2023-10-06T05:23:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.683+0000 7f21e33ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc069080 con 0x7f21dc15ca40 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.684+0000 7f21e33ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21dc0cabc0 con 0x7f21dc0c8f40 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.684+0000 7f21e099a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21dc0c8f40 0x7f21dc068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.684+0000 7f21e199c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f21dc06a300 0x7f21dc0adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35400/0 (socket says 172.21.15.31:35400) 2023-10-06T05:23:14.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.684+0000 7f21e199c700 1 -- 172.21.15.31:0/4011321832 learned_addr learned my addr 172.21.15.31:0/4011321832 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.684+0000 7f21e099a700 1 -- 172.21.15.31:0/4011321832 >> v1:172.21.15.62:6789/0 conn(0x7f21dc15ca40 legacy=0x7f21dc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.684+0000 7f21e099a700 1 -- 172.21.15.31:0/4011321832 >> v1:172.21.15.31:6789/0 conn(0x7f21dc06a300 legacy=0x7f21dc0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.684+0000 7f21e099a700 1 -- 172.21.15.31:0/4011321832 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc069360 con 0x7f21dc0c8f40 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.685+0000 7f21e099a700 1 --2- 172.21.15.31:0/4011321832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 0x7f21dc068670 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0098e0 tx=0x7f21cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb9a615b7a9c21a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.685+0000 7f21d3fff700 1 -- 172.21.15.31:0/4011321832 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc009e80 con 0x7f21dc0c8f40 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.685+0000 7f21d3fff700 1 -- 172.21.15.31:0/4011321832 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc00b860 con 0x7f21dc0c8f40 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.685+0000 7f21d3fff700 1 -- 172.21.15.31:0/4011321832 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00bbe0 con 0x7f21dc0c8f40 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.685+0000 7f21e33ff700 1 -- 172.21.15.31:0/4011321832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 msgr2=0x7f21dc068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.685+0000 7f21e33ff700 1 --2- 172.21.15.31:0/4011321832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 0x7f21dc068670 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0098e0 tx=0x7f21cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.685+0000 7f21e33ff700 1 -- 172.21.15.31:0/4011321832 shutdown_connections 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.685+0000 7f21e33ff700 1 --2- 172.21.15.31:0/4011321832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 0x7f21dc068670 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.686+0000 7f21e33ff700 1 -- 172.21.15.31:0/4011321832 >> 172.21.15.31:0/4011321832 conn(0x7f21dc0ce290 msgr2=0x7f21dc0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.686+0000 7f21e33ff700 1 -- 172.21.15.31:0/4011321832 shutdown_connections 2023-10-06T05:23:14.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.686+0000 7f21e33ff700 1 -- 172.21.15.31:0/4011321832 wait complete. 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.688+0000 7f21e33ff700 1 Processor -- start 2023-10-06T05:23:14.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.688+0000 7f21e33ff700 1 -- start start 2023-10-06T05:23:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.689+0000 7f21e33ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc06a300 0x7f21dc0c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.689+0000 7f21e33ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 0x7f21dc0b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.689+0000 7f21e33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21dc15ca40 0x7f21dc0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.689+0000 7f21e33ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0c7e90 con 0x7f21dc0c8f40 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.689+0000 7f21e33ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc16a070 con 0x7f21dc06a300 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.689+0000 7f21e33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc16a350 con 0x7f21dc15ca40 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.689+0000 7f21e099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 0x7f21dc0b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.689+0000 7f21e099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 0x7f21dc0b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50152/0 (socket says 172.21.15.31:50152) 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.689+0000 7f21e099a700 1 -- 172.21.15.31:0/2114567895 learned_addr learned my addr 172.21.15.31:0/2114567895 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.689+0000 7f21e119b700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc06a300 0x7f21dc0c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21e199c700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21dc15ca40 0x7f21dc0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21e099a700 1 -- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21dc15ca40 msgr2=0x7f21dc0b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21e099a700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21dc15ca40 0x7f21dc0b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21e099a700 1 -- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc06a300 msgr2=0x7f21dc0c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21e099a700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc06a300 0x7f21dc0c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21e099a700 1 -- 172.21.15.31:0/2114567895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc009420 con 0x7f21dc0c8f40 2023-10-06T05:23:14.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21e099a700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 0x7f21dc0b35a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f21dc06e040 tx=0x7f21cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21d17fa700 1 -- 172.21.15.31:0/2114567895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00baa0 con 0x7f21dc0c8f40 2023-10-06T05:23:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21d17fa700 1 -- 172.21.15.31:0/2114567895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc025420 con 0x7f21dc0c8f40 2023-10-06T05:23:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21d17fa700 1 -- 172.21.15.31:0/2114567895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0257c0 con 0x7f21dc0c8f40 2023-10-06T05:23:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.690+0000 7f21e33ff700 1 -- 172.21.15.31:0/2114567895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc16a630 con 0x7f21dc0c8f40 2023-10-06T05:23:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.691+0000 7f21e33ff700 1 -- 172.21.15.31:0/2114567895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc0c7ad0 con 0x7f21dc0c8f40 2023-10-06T05:23:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.692+0000 7f21e33ff700 1 -- 172.21.15.31:0/2114567895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc169ea0 con 0x7f21dc0c8f40 2023-10-06T05:23:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.692+0000 7f21d17fa700 1 -- 172.21.15.31:0/2114567895 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f21cc023070 con 0x7f21dc0c8f40 2023-10-06T05:23:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.693+0000 7f21d17fa700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21c0067320 0x7f21c00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.693+0000 7f21d17fa700 1 -- 172.21.15.31:0/2114567895 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f21cc086410 con 0x7f21dc0c8f40 2023-10-06T05:23:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.694+0000 7f21e119b700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21c0067320 0x7f21c00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:14.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.694+0000 7f21e119b700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21c0067320 0x7f21c00697d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0ae030 tx=0x7f21d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:14.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.697+0000 7f21d17fa700 1 -- 172.21.15.31:0/2114567895 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21cc052ec0 con 0x7f21dc0c8f40 2023-10-06T05:23:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.896+0000 7f21e33ff700 1 -- 172.21.15.31:0/2114567895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f21dc04c380 con 0x7f21dc0c8f40 2023-10-06T05:23:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.897+0000 7f21d17fa700 1 -- 172.21.15.31:0/2114567895 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7f21cc0564e0 con 0x7f21dc0c8f40 2023-10-06T05:23:14.899 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:23:14.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 -- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21c0067320 msgr2=0x7f21c00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21c0067320 0x7f21c00697d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0ae030 tx=0x7f21d8009040 comp rx=0 tx=0).stop 2023-10-06T05:23:14.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 -- 172.21.15.31:0/2114567895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 msgr2=0x7f21dc0b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:14.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 0x7f21dc0b35a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f21dc06e040 tx=0x7f21cc017000 comp rx=0 tx=0).stop 2023-10-06T05:23:14.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 -- 172.21.15.31:0/2114567895 shutdown_connections 2023-10-06T05:23:14.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21c0067320 0x7f21c00697d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21dc15ca40 0x7f21dc0b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc06a300 0x7f21dc0c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 --2- 172.21.15.31:0/2114567895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21dc0c8f40 0x7f21dc0b35a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:14.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 -- 172.21.15.31:0/2114567895 >> 172.21.15.31:0/2114567895 conn(0x7f21dc0ce290 msgr2=0x7f21dc15b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:14.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 -- 172.21.15.31:0/2114567895 shutdown_connections 2023-10-06T05:23:14.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:14.899+0000 7f21e33ff700 1 -- 172.21.15.31:0/2114567895 wait complete. 2023-10-06T05:23:14.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:15.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.113+0000 7f132efff700 1 Processor -- start 2023-10-06T05:23:15.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.114+0000 7f132efff700 1 -- start start 2023-10-06T05:23:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.115+0000 7f132efff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1328159500 0x7f13280b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.115+0000 7f132efff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13280cd510 0x7f13280aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.115+0000 7f132efff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13280c3f40 con 0x7f132815d2d0 2023-10-06T05:23:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.115+0000 7f132efff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13280b2860 con 0x7f13280cd510 2023-10-06T05:23:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.115+0000 7f132efff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132815d7d0 con 0x7f1328159500 2023-10-06T05:23:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.115+0000 7f132d59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13280cd510 0x7f13280aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.115+0000 7f132d59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13280cd510 0x7f13280aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50166/0 (socket says 172.21.15.31:50166) 2023-10-06T05:23:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.115+0000 7f132d59c700 1 -- 172.21.15.31:0/303382350 learned_addr learned my addr 172.21.15.31:0/303382350 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.115+0000 7f132cd9b700 1 --2- 172.21.15.31:0/303382350 >> v2:172.21.15.62:3300/0 conn(0x7f1328159500 0x7f13280b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.116+0000 7f132d59c700 1 -- 172.21.15.31:0/303382350 >> v2:172.21.15.62:3300/0 conn(0x7f1328159500 msgr2=0x7f13280b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.116+0000 7f132d59c700 1 --2- 172.21.15.31:0/303382350 >> v2:172.21.15.62:3300/0 conn(0x7f1328159500 0x7f13280b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.116+0000 7f132d59c700 1 -- 172.21.15.31:0/303382350 >> v1:172.21.15.62:6789/0 conn(0x7f132815d2d0 legacy=0x7f13280cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.116+0000 7f132d59c700 1 -- 172.21.15.31:0/303382350 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13280cb000 con 0x7f13280cd510 2023-10-06T05:23:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.116+0000 7f132d59c700 1 --2- 172.21.15.31:0/303382350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 0x7f13280aedd0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1328089840 tx=0x7f1320009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7258adbcee8bad2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.116+0000 7f13277fe700 1 -- 172.21.15.31:0/303382350 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132000e070 con 0x7f13280cd510 2023-10-06T05:23:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.116+0000 7f13277fe700 1 -- 172.21.15.31:0/303382350 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132000ba20 con 0x7f13280cd510 2023-10-06T05:23:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.116+0000 7f13277fe700 1 -- 172.21.15.31:0/303382350 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13200092e0 con 0x7f13280cd510 2023-10-06T05:23:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.117+0000 7f132efff700 1 -- 172.21.15.31:0/303382350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 msgr2=0x7f13280aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.117+0000 7f132efff700 1 --2- 172.21.15.31:0/303382350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 0x7f13280aedd0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1328089840 tx=0x7f1320009840 comp rx=0 tx=0).stop 2023-10-06T05:23:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.117+0000 7f132efff700 1 -- 172.21.15.31:0/303382350 shutdown_connections 2023-10-06T05:23:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.117+0000 7f132efff700 1 --2- 172.21.15.31:0/303382350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 0x7f13280aedd0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.117+0000 7f132efff700 1 --2- 172.21.15.31:0/303382350 >> v2:172.21.15.62:3300/0 conn(0x7f1328159500 0x7f13280b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.117+0000 7f132efff700 1 -- 172.21.15.31:0/303382350 >> 172.21.15.31:0/303382350 conn(0x7f132806ec90 msgr2=0x7f132806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.117+0000 7f132efff700 1 -- 172.21.15.31:0/303382350 shutdown_connections 2023-10-06T05:23:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.117+0000 7f132efff700 1 -- 172.21.15.31:0/303382350 wait complete. 2023-10-06T05:23:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.119+0000 7f132efff700 1 Processor -- start 2023-10-06T05:23:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.119+0000 7f132efff700 1 -- start start 2023-10-06T05:23:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.120+0000 7f132efff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 0x7f1328155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.120+0000 7f132efff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1328159500 0x7f13280b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.120+0000 7f132efff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f132815d2d0 0x7f13280c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.120+0000 7f132efff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132815a4e0 con 0x7f13280cd510 2023-10-06T05:23:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.120+0000 7f132efff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13281598d0 con 0x7f1328159500 2023-10-06T05:23:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.120+0000 7f132efff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1328159bb0 con 0x7f132815d2d0 2023-10-06T05:23:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 0x7f1328155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132cd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 0x7f1328155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50174/0 (socket says 172.21.15.31:50174) 2023-10-06T05:23:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132cd9b700 1 -- 172.21.15.31:0/1115807683 learned_addr learned my addr 172.21.15.31:0/1115807683 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f1327fff700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1328159500 0x7f13280b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132d59c700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f132815d2d0 0x7f13280c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132cd9b700 1 -- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f132815d2d0 msgr2=0x7f13280c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132cd9b700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f132815d2d0 0x7f13280c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132cd9b700 1 -- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1328159500 msgr2=0x7f13280b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132cd9b700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1328159500 0x7f13280b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132cd9b700 1 -- 172.21.15.31:0/1115807683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320009ae0 con 0x7f13280cd510 2023-10-06T05:23:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f1327fff700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1328159500 0x7f13280b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132cd9b700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 0x7f1328155710 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f131c00ec80 tx=0x7f131c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f1324ff9700 1 -- 172.21.15.31:0/1115807683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00c9d0 con 0x7f13280cd510 2023-10-06T05:23:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f1324ff9700 1 -- 172.21.15.31:0/1115807683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131c00cb30 con 0x7f13280cd510 2023-10-06T05:23:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.121+0000 7f132efff700 1 -- 172.21.15.31:0/1115807683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1328159e90 con 0x7f13280cd510 2023-10-06T05:23:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.122+0000 7f1324ff9700 1 -- 172.21.15.31:0/1115807683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c013470 con 0x7f13280cd510 2023-10-06T05:23:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.122+0000 7f132efff700 1 -- 172.21.15.31:0/1115807683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1328155c50 con 0x7f13280cd510 2023-10-06T05:23:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.123+0000 7f132efff700 1 -- 172.21.15.31:0/1115807683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132815a7c0 con 0x7f13280cd510 2023-10-06T05:23:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.123+0000 7f1324ff9700 1 -- 172.21.15.31:0/1115807683 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f131c013820 con 0x7f13280cd510 2023-10-06T05:23:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.125+0000 7f1324ff9700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f130c067320 0x7f130c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.125+0000 7f1324ff9700 1 -- 172.21.15.31:0/1115807683 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f131c085240 con 0x7f13280cd510 2023-10-06T05:23:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.125+0000 7f1327fff700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f130c067320 0x7f130c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.126+0000 7f1327fff700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f130c067320 0x7f130c0697d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f132816cfd0 tx=0x7f1314009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.129+0000 7f1324ff9700 1 -- 172.21.15.31:0/1115807683 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f131c051cf0 con 0x7f13280cd510 2023-10-06T05:23:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.380+0000 7f132efff700 1 -- 172.21.15.31:0/1115807683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f132804c380 con 0x7f13280cd510 2023-10-06T05:23:15.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.381+0000 7f1324ff9700 1 -- 172.21.15.31:0/1115807683 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f131c055310 con 0x7f13280cd510 2023-10-06T05:23:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:15.383 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1885821393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1885821393},{"type":"v1","addr":"172.21.15.31:6834","nonce":1885821393}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6889,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/258742412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":258742412},{"type":"v1","addr":"172.21.15.31:6835","nonce":258742412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4070877337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4070877337},{"type":"v1","addr":"172.21.15.62:6834","nonce":4070877337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.638176+0000","modified":"2023-10-06T05:22:36.157996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7059":{"gid":7059,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2094863015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2094863015},{"type":"v1","addr":"172.21.15.62:6835","nonce":2094863015}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:15.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 -- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f130c067320 msgr2=0x7f130c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f130c067320 0x7f130c0697d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f132816cfd0 tx=0x7f1314009040 comp rx=0 tx=0).stop 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 -- 172.21.15.31:0/1115807683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 msgr2=0x7f1328155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 0x7f1328155710 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f131c00ec80 tx=0x7f131c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 -- 172.21.15.31:0/1115807683 shutdown_connections 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f130c067320 0x7f130c0697d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f132815d2d0 0x7f13280c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13280cd510 0x7f1328155710 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 --2- 172.21.15.31:0/1115807683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1328159500 0x7f13280b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 -- 172.21.15.31:0/1115807683 >> 172.21.15.31:0/1115807683 conn(0x7f132806ec90 msgr2=0x7f13280afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.383+0000 7f132efff700 1 -- 172.21.15.31:0/1115807683 shutdown_connections 2023-10-06T05:23:15.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.384+0000 7f132efff700 1 -- 172.21.15.31:0/1115807683 wait complete. 2023-10-06T05:23:15.387 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 68 2023-10-06T05:23:15.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.597+0000 7f21c81ff700 1 Processor -- start 2023-10-06T05:23:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.597+0000 7f21c81ff700 1 -- start start 2023-10-06T05:23:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c81ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21c00ccc80 0x7f21c00cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c81ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c00cf280 con 0x7f21c00cdfe0 2023-10-06T05:23:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c81ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c00cfe20 con 0x7f21c0159580 2023-10-06T05:23:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c81ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c00ce480 con 0x7f21c00ccc80 2023-10-06T05:23:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c579a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f21c00cdfe0 0x7f21c00cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35406/0 (socket says 172.21.15.31:35406) 2023-10-06T05:23:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c579a700 1 -- 172.21.15.31:0/530507047 learned_addr learned my addr 172.21.15.31:0/530507047 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c679c700 1 --2- 172.21.15.31:0/530507047 >> v2:172.21.15.62:3300/0 conn(0x7f21c00ccc80 0x7f21c00cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c4f99700 1 -- 172.21.15.31:0/530507047 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810888999 0 0) 0x7f21c00cf280 con 0x7f21c00cdfe0 2023-10-06T05:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c4f99700 1 -- 172.21.15.31:0/530507047 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21ac003570 con 0x7f21c00cdfe0 2023-10-06T05:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c4f99700 1 -- 172.21.15.31:0/530507047 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106957268 0 0) 0x7f21c00cfe20 con 0x7f21c0159580 2023-10-06T05:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.599+0000 7f21c4f99700 1 -- 172.21.15.31:0/530507047 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c00cf280 con 0x7f21c0159580 2023-10-06T05:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.600+0000 7f21c679c700 1 -- 172.21.15.31:0/530507047 >> v1:172.21.15.62:6789/0 conn(0x7f21c0159580 legacy=0x7f21c00cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.600+0000 7f21c679c700 1 -- 172.21.15.31:0/530507047 >> v1:172.21.15.31:6789/0 conn(0x7f21c00cdfe0 legacy=0x7f21c00cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.600+0000 7f21c679c700 1 -- 172.21.15.31:0/530507047 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c00ce670 con 0x7f21c00ccc80 2023-10-06T05:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.600+0000 7f21c679c700 1 --2- 172.21.15.31:0/530507047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c00ccc80 0x7f21c00cd080 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009e60 tx=0x7f21b8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a6e6e1a66442265 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.600+0000 7f21c4f99700 1 -- 172.21.15.31:0/530507047 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b800e070 con 0x7f21c00ccc80 2023-10-06T05:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.600+0000 7f21c4f99700 1 -- 172.21.15.31:0/530507047 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b800bab0 con 0x7f21c00ccc80 2023-10-06T05:23:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.600+0000 7f21c4f99700 1 -- 172.21.15.31:0/530507047 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b800be70 con 0x7f21c00ccc80 2023-10-06T05:23:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.601+0000 7f21c81ff700 1 -- 172.21.15.31:0/530507047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c00ccc80 msgr2=0x7f21c00cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.601+0000 7f21c81ff700 1 --2- 172.21.15.31:0/530507047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c00ccc80 0x7f21c00cd080 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009e60 tx=0x7f21b8009780 comp rx=0 tx=0).stop 2023-10-06T05:23:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.601+0000 7f21c81ff700 1 -- 172.21.15.31:0/530507047 shutdown_connections 2023-10-06T05:23:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.601+0000 7f21c81ff700 1 --2- 172.21.15.31:0/530507047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c00ccc80 0x7f21c00cd080 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.601+0000 7f21c81ff700 1 -- 172.21.15.31:0/530507047 >> 172.21.15.31:0/530507047 conn(0x7f21c0099710 msgr2=0x7f21c006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.601+0000 7f21c81ff700 1 -- 172.21.15.31:0/530507047 shutdown_connections 2023-10-06T05:23:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.601+0000 7f21c81ff700 1 -- 172.21.15.31:0/530507047 wait complete. 2023-10-06T05:23:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.602+0000 7f21c81ff700 1 Processor -- start 2023-10-06T05:23:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.602+0000 7f21c81ff700 1 -- start start 2023-10-06T05:23:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.603+0000 7f21c81ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c00ccc80 0x7f21c01583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.603+0000 7f21c81ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21c00cdfe0 0x7f21c00c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.603+0000 7f21c81ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c0159580 0x7f21c00b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.603+0000 7f21c81ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c0158930 con 0x7f21c0159580 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.603+0000 7f21c81ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c00c47a0 con 0x7f21c00ccc80 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.603+0000 7f21c81ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c00c4c30 con 0x7f21c00cdfe0 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c679c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c0159580 0x7f21c00b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c679c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c0159580 0x7f21c00b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50188/0 (socket says 172.21.15.31:50188) 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c679c700 1 -- 172.21.15.31:0/1646198101 learned_addr learned my addr 172.21.15.31:0/1646198101 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c579a700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21c00cdfe0 0x7f21c00c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c5f9b700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c00ccc80 0x7f21c01583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c679c700 1 -- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21c00cdfe0 msgr2=0x7f21c00c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c679c700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21c00cdfe0 0x7f21c00c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c679c700 1 -- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c00ccc80 msgr2=0x7f21c01583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c679c700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c00ccc80 0x7f21c01583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c679c700 1 -- 172.21.15.31:0/1646198101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b8009a20 con 0x7f21c0159580 2023-10-06T05:23:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c579a700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21c00cdfe0 0x7f21c00c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c5f9b700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c00ccc80 0x7f21c01583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21c679c700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c0159580 0x7f21c00b4250 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f21c00888e0 tx=0x7f21b8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.604+0000 7f21b5ffb700 1 -- 172.21.15.31:0/1646198101 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8026480 con 0x7f21c0159580 2023-10-06T05:23:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.605+0000 7f21b5ffb700 1 -- 172.21.15.31:0/1646198101 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8026600 con 0x7f21c0159580 2023-10-06T05:23:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.605+0000 7f21c81ff700 1 -- 172.21.15.31:0/1646198101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c00c90d0 con 0x7f21c0159580 2023-10-06T05:23:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.605+0000 7f21b5ffb700 1 -- 172.21.15.31:0/1646198101 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b80269c0 con 0x7f21c0159580 2023-10-06T05:23:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.605+0000 7f21c81ff700 1 -- 172.21.15.31:0/1646198101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c00c4150 con 0x7f21c0159580 2023-10-06T05:23:15.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.606+0000 7f21c81ff700 1 -- 172.21.15.31:0/1646198101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c00c4930 con 0x7f21c0159580 2023-10-06T05:23:15.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.607+0000 7f21b5ffb700 1 -- 172.21.15.31:0/1646198101 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f21b801c400 con 0x7f21c0159580 2023-10-06T05:23:15.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.607+0000 7f21b5ffb700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21a4067320 0x7f21a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:15.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.607+0000 7f21b5ffb700 1 -- 172.21.15.31:0/1646198101 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f21b80873b0 con 0x7f21c0159580 2023-10-06T05:23:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.608+0000 7f21c5f9b700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21a4067320 0x7f21a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:15.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.608+0000 7f21c5f9b700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21a4067320 0x7f21a40697d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0038e0 tx=0x7f21bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:15.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.612+0000 7f21b5ffb700 1 -- 172.21.15.31:0/1646198101 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21b8053d50 con 0x7f21c0159580 2023-10-06T05:23:15.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.813+0000 7f21c81ff700 1 -- 172.21.15.31:0/1646198101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21c004c380 con 0x7f21c0159580 2023-10-06T05:23:15.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.814+0000 7f21b5ffb700 1 -- 172.21.15.31:0/1646198101 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+17031 (secure 0 0 0) 0x7f21b8057370 con 0x7f21c0159580 2023-10-06T05:23:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:15.816 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":102,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:23:13.133045+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.850575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.607953+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:15.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 -- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21a4067320 msgr2=0x7f21a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21a4067320 0x7f21a40697d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0038e0 tx=0x7f21bc009040 comp rx=0 tx=0).stop 2023-10-06T05:23:15.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 -- 172.21.15.31:0/1646198101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c0159580 msgr2=0x7f21c00b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:15.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c0159580 0x7f21c00b4250 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f21c00888e0 tx=0x7f21b8019000 comp rx=0 tx=0).stop 2023-10-06T05:23:15.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 -- 172.21.15.31:0/1646198101 shutdown_connections 2023-10-06T05:23:15.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21a4067320 0x7f21a40697d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c0159580 0x7f21c00b4250 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21c00ccc80 0x7f21c01583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 --2- 172.21.15.31:0/1646198101 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21c00cdfe0 0x7f21c00c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:15.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 -- 172.21.15.31:0/1646198101 >> 172.21.15.31:0/1646198101 conn(0x7f21c0099710 msgr2=0x7f21c00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:15.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 -- 172.21.15.31:0/1646198101 shutdown_connections 2023-10-06T05:23:15.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:15.817+0000 7f21c81ff700 1 -- 172.21.15.31:0/1646198101 wait complete. 2023-10-06T05:23:15.836 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:23:15.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:16.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.029+0000 7f8284dff700 1 Processor -- start 2023-10-06T05:23:16.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.029+0000 7f8284dff700 1 -- start start 2023-10-06T05:23:16.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.031+0000 7f8284dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82800c7f30 0x7f8280069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.031+0000 7f8284dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82800ca5b0 con 0x7f82800aee70 2023-10-06T05:23:16.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.031+0000 7f8284dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82800af2f0 con 0x7f82800c7b60 2023-10-06T05:23:16.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.031+0000 7f8284dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828006f740 con 0x7f82800c7f30 2023-10-06T05:23:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.031+0000 7f827ed9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f82800aee70 0x7f828013f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35414/0 (socket says 172.21.15.31:35414) 2023-10-06T05:23:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.031+0000 7f827ed9d700 1 -- 172.21.15.31:0/749831364 learned_addr learned my addr 172.21.15.31:0/749831364 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.031+0000 7f827dd9b700 1 --2- 172.21.15.31:0/749831364 >> v2:172.21.15.62:3300/0 conn(0x7f82800c7f30 0x7f8280069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.031+0000 7f827d59a700 1 -- 172.21.15.31:0/749831364 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809218919 0 0) 0x7f82800ca5b0 con 0x7f82800aee70 2023-10-06T05:23:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.032+0000 7f827d59a700 1 -- 172.21.15.31:0/749831364 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8268003570 con 0x7f82800aee70 2023-10-06T05:23:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.032+0000 7f827d59a700 1 -- 172.21.15.31:0/749831364 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740544982 0 0) 0x7f82800af2f0 con 0x7f82800c7b60 2023-10-06T05:23:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.032+0000 7f827d59a700 1 -- 172.21.15.31:0/749831364 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82800ca5b0 con 0x7f82800c7b60 2023-10-06T05:23:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.032+0000 7f827d59a700 1 -- 172.21.15.31:0/749831364 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2673590718 0 0) 0x7f8268003570 con 0x7f82800aee70 2023-10-06T05:23:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.032+0000 7f827dd9b700 1 -- 172.21.15.31:0/749831364 >> v1:172.21.15.62:6789/0 conn(0x7f82800c7b60 legacy=0x7f828006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.032+0000 7f827dd9b700 1 -- 172.21.15.31:0/749831364 >> v1:172.21.15.31:6789/0 conn(0x7f82800aee70 legacy=0x7f828013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.032+0000 7f827dd9b700 1 -- 172.21.15.31:0/749831364 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82800c9c80 con 0x7f82800c7f30 2023-10-06T05:23:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.032+0000 7f827dd9b700 1 --2- 172.21.15.31:0/749831364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82800c7f30 0x7f8280069880 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f82740098e0 tx=0x7f82740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68414d438d6b9332 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.033+0000 7f827d59a700 1 -- 172.21.15.31:0/749831364 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780017f0 con 0x7f82800c7f30 2023-10-06T05:23:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.033+0000 7f827d59a700 1 -- 172.21.15.31:0/749831364 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8274011420 con 0x7f82800c7f30 2023-10-06T05:23:16.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.033+0000 7f827d59a700 1 -- 172.21.15.31:0/749831364 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740117c0 con 0x7f82800c7f30 2023-10-06T05:23:16.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.033+0000 7f8284dff700 1 -- 172.21.15.31:0/749831364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82800c7f30 msgr2=0x7f8280069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.033+0000 7f8284dff700 1 --2- 172.21.15.31:0/749831364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82800c7f30 0x7f8280069880 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f82740098e0 tx=0x7f82740091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.033+0000 7f8284dff700 1 -- 172.21.15.31:0/749831364 shutdown_connections 2023-10-06T05:23:16.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.033+0000 7f8284dff700 1 --2- 172.21.15.31:0/749831364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82800c7f30 0x7f8280069880 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.033+0000 7f8284dff700 1 -- 172.21.15.31:0/749831364 >> 172.21.15.31:0/749831364 conn(0x7f82800cc0b0 msgr2=0x7f82800cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.033+0000 7f8284dff700 1 -- 172.21.15.31:0/749831364 shutdown_connections 2023-10-06T05:23:16.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.033+0000 7f8284dff700 1 -- 172.21.15.31:0/749831364 wait complete. 2023-10-06T05:23:16.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.035+0000 7f8284dff700 1 Processor -- start 2023-10-06T05:23:16.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.035+0000 7f8284dff700 1 -- start start 2023-10-06T05:23:16.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f8284dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82800aee70 0x7f8280139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f8284dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82800c7b60 0x7f828013a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f8284dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82800c7f30 0x7f828013aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f8284dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828013b3b0 con 0x7f82800c7f30 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f8284dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828016d2b0 con 0x7f82800aee70 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f8284dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8280156560 con 0x7f82800c7b60 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f827ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82800c7f30 0x7f828013aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f827ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82800c7f30 0x7f828013aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50196/0 (socket says 172.21.15.31:50196) 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f827ed9d700 1 -- 172.21.15.31:0/1736641531 learned_addr learned my addr 172.21.15.31:0/1736641531 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f827e59c700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82800aee70 0x7f8280139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.036+0000 7f827dd9b700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82800c7b60 0x7f828013a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f827ed9d700 1 -- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82800c7b60 msgr2=0x7f828013a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f827ed9d700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82800c7b60 0x7f828013a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f827ed9d700 1 -- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82800aee70 msgr2=0x7f8280139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f827ed9d700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82800aee70 0x7f8280139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f827ed9d700 1 -- 172.21.15.31:0/1736641531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274009420 con 0x7f82800c7f30 2023-10-06T05:23:16.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f827ed9d700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82800c7f30 0x7f828013aa50 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f827800ccc0 tx=0x7f8278007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f826e7fc700 1 -- 172.21.15.31:0/1736641531 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278004dc0 con 0x7f82800c7f30 2023-10-06T05:23:16.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f826e7fc700 1 -- 172.21.15.31:0/1736641531 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827800a5f0 con 0x7f82800c7f30 2023-10-06T05:23:16.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f8284dff700 1 -- 172.21.15.31:0/1736641531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828013b0d0 con 0x7f82800c7f30 2023-10-06T05:23:16.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f826e7fc700 1 -- 172.21.15.31:0/1736641531 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827800a9b0 con 0x7f82800c7f30 2023-10-06T05:23:16.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.037+0000 7f8284dff700 1 -- 172.21.15.31:0/1736641531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828016cf40 con 0x7f82800c7f30 2023-10-06T05:23:16.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.038+0000 7f8284dff700 1 -- 172.21.15.31:0/1736641531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828016d420 con 0x7f82800c7f30 2023-10-06T05:23:16.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.039+0000 7f826e7fc700 1 -- 172.21.15.31:0/1736641531 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8278007150 con 0x7f82800c7f30 2023-10-06T05:23:16.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.040+0000 7f826e7fc700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8260067320 0x7f82600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.040+0000 7f826e7fc700 1 -- 172.21.15.31:0/1736641531 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f82780847f0 con 0x7f82800c7f30 2023-10-06T05:23:16.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.041+0000 7f827e59c700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8260067320 0x7f82600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.041+0000 7f827e59c700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8260067320 0x7f82600697d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f828013d3a0 tx=0x7f8270007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.044+0000 7f826e7fc700 1 -- 172.21.15.31:0/1736641531 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8278051320 con 0x7f82800c7f30 2023-10-06T05:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.290+0000 7f8284dff700 1 -- 172.21.15.31:0/1736641531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f828004c380 con 0x7f82800c7f30 2023-10-06T05:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.290+0000 7f826e7fc700 1 -- 172.21.15.31:0/1736641531 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f8278054940 con 0x7f82800c7f30 2023-10-06T05:23:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:16.293 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1885821393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1885821393},{"type":"v1","addr":"172.21.15.31:6834","nonce":1885821393}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6889,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/258742412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":258742412},{"type":"v1","addr":"172.21.15.31:6835","nonce":258742412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4070877337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4070877337},{"type":"v1","addr":"172.21.15.62:6834","nonce":4070877337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.638176+0000","modified":"2023-10-06T05:22:36.157996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7059":{"gid":7059,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2094863015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2094863015},{"type":"v1","addr":"172.21.15.62:6835","nonce":2094863015}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:16.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 -- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8260067320 msgr2=0x7f82600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8260067320 0x7f82600697d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f828013d3a0 tx=0x7f8270007d40 comp rx=0 tx=0).stop 2023-10-06T05:23:16.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 -- 172.21.15.31:0/1736641531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82800c7f30 msgr2=0x7f828013aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82800c7f30 0x7f828013aa50 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f827800ccc0 tx=0x7f8278007d20 comp rx=0 tx=0).stop 2023-10-06T05:23:16.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 -- 172.21.15.31:0/1736641531 shutdown_connections 2023-10-06T05:23:16.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8260067320 0x7f82600697d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82800c7f30 0x7f828013aa50 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82800aee70 0x7f8280139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 --2- 172.21.15.31:0/1736641531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82800c7b60 0x7f828013a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 -- 172.21.15.31:0/1736641531 >> 172.21.15.31:0/1736641531 conn(0x7f82800cc0b0 msgr2=0x7f82800c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 -- 172.21.15.31:0/1736641531 shutdown_connections 2023-10-06T05:23:16.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.293+0000 7f8284dff700 1 -- 172.21.15.31:0/1736641531 wait complete. 2023-10-06T05:23:16.296 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 68 2023-10-06T05:23:16.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.506+0000 7f40039ff700 1 Processor -- start 2023-10-06T05:23:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.506+0000 7f40039ff700 1 -- start start 2023-10-06T05:23:16.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.508+0000 7f40039ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc0c7f30 0x7f3ffc069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.508+0000 7f40039ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ffc0ca5b0 con 0x7f3ffc0c7b60 2023-10-06T05:23:16.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.508+0000 7f40039ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ffc0af2f0 con 0x7f3ffc0aee70 2023-10-06T05:23:16.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.508+0000 7f40039ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc06f740 con 0x7f3ffc0c7f30 2023-10-06T05:23:16.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.508+0000 7f400179b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3ffc0c7b60 0x7f3ffc06ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35418/0 (socket says 172.21.15.31:35418) 2023-10-06T05:23:16.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.508+0000 7f400179b700 1 -- 172.21.15.31:0/264485958 learned_addr learned my addr 172.21.15.31:0/264485958 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:16.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.508+0000 7f4000f9a700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc0c7f30 0x7f3ffc069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.508+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25185525 0 0) 0x7f3ffc0ca5b0 con 0x7f3ffc0c7b60 2023-10-06T05:23:16.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.508+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fe8003570 con 0x7f3ffc0c7b60 2023-10-06T05:23:16.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253992866 0 0) 0x7f3ffc0af2f0 con 0x7f3ffc0aee70 2023-10-06T05:23:16.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ffc0ca5b0 con 0x7f3ffc0aee70 2023-10-06T05:23:16.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 879814051 0 0) 0x7f3fe8003570 con 0x7f3ffc0c7b60 2023-10-06T05:23:16.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc0c7f30 msgr2=0x7f3ffc069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:16.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc0c7f30 0x7f3ffc069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 >> v1:172.21.15.62:6789/0 conn(0x7f3ffc0aee70 legacy=0x7f3ffc13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0c9c80 con 0x7f3ffc0c7b60 2023-10-06T05:23:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f3ff80015f0 con 0x7f3ffc0c7b60 2023-10-06T05:23:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f4000f9a700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc0c7f30 0x7f3ffc069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 >> v1:172.21.15.31:6789/0 conn(0x7f3ffc0c7b60 legacy=0x7f3ffc06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe8003c20 0x7f3fe8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe8004810 0x7f3fe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc0c7f30 0x7f3fe80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc06f740 con 0x7f3fe8003c20 2023-10-06T05:23:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe800bb10 con 0x7f3fe8004810 2023-10-06T05:23:16.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe800bcd0 con 0x7f3ffc0c7f30 2023-10-06T05:23:16.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f400179b700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe8003c20 0x7f3fe8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f4000f9a700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe8004810 0x7f3fe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.509+0000 7f4001f9c700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc0c7f30 0x7f3fe80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.510+0000 7f400179b700 1 -- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc0c7f30 msgr2=0x7f3fe80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:16.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.510+0000 7f400179b700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc0c7f30 0x7f3fe80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.510+0000 7f400179b700 1 -- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe8004810 msgr2=0x7f3fe8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:16.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.510+0000 7f400179b700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe8004810 0x7f3fe8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.510+0000 7f400179b700 1 -- 172.21.15.31:0/264485958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec009420 con 0x7f3fe8003c20 2023-10-06T05:23:16.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.510+0000 7f400179b700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe8003c20 0x7f3fe8004010 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c2d0 tx=0x7f3ff80078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b514eacd0c358d31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.510+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800cc00 con 0x7f3fe8003c20 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.510+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff800cd60 con 0x7f3fe8003c20 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.510+0000 7f3ff3fff700 1 -- 172.21.15.31:0/264485958 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8007300 con 0x7f3fe8003c20 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.511+0000 7f40039ff700 1 -- 172.21.15.31:0/264485958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe8003c20 msgr2=0x7f3fe8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.511+0000 7f40039ff700 1 --2- 172.21.15.31:0/264485958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe8003c20 0x7f3fe8004010 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c2d0 tx=0x7f3ff80078a0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.511+0000 7f4001f9c700 1 -- 172.21.15.31:0/264485958 reap_dead start 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.511+0000 7f40039ff700 1 -- 172.21.15.31:0/264485958 shutdown_connections 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.511+0000 7f40039ff700 1 -- 172.21.15.31:0/264485958 >> 172.21.15.31:0/264485958 conn(0x7f3ffc0cc0b0 msgr2=0x7f3ffc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.511+0000 7f40039ff700 1 -- 172.21.15.31:0/264485958 shutdown_connections 2023-10-06T05:23:16.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.511+0000 7f40039ff700 1 -- 172.21.15.31:0/264485958 wait complete. 2023-10-06T05:23:16.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.512+0000 7f40039ff700 1 Processor -- start 2023-10-06T05:23:16.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.512+0000 7f40039ff700 1 -- start start 2023-10-06T05:23:16.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.513+0000 7f40039ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe8003c20 0x7f3ffc16ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.513+0000 7f40039ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc158a50 0x7f3ffc0b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.513+0000 7f40039ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc15c860 0x7f3ffc13e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.513+0000 7f40039ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc0ab7a0 con 0x7f3ffc158a50 2023-10-06T05:23:16.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.513+0000 7f40039ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc16d5c0 con 0x7f3fe8003c20 2023-10-06T05:23:16.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.513+0000 7f40039ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc16d8a0 con 0x7f3ffc15c860 2023-10-06T05:23:16.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.513+0000 7f4000f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc158a50 0x7f3ffc0b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.513+0000 7f4000f9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc158a50 0x7f3ffc0b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50218/0 (socket says 172.21.15.31:50218) 2023-10-06T05:23:16.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.513+0000 7f4000f9a700 1 -- 172.21.15.31:0/4149319625 learned_addr learned my addr 172.21.15.31:0/4149319625 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:16.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f400179b700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe8003c20 0x7f3ffc16ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f4001f9c700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc15c860 0x7f3ffc13e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f4000f9a700 1 -- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc15c860 msgr2=0x7f3ffc13e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f4000f9a700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc15c860 0x7f3ffc13e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f4000f9a700 1 -- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe8003c20 msgr2=0x7f3ffc16ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f4000f9a700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe8003c20 0x7f3ffc16ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f4000f9a700 1 -- 172.21.15.31:0/4149319625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0aba40 con 0x7f3ffc158a50 2023-10-06T05:23:16.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f4000f9a700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc158a50 0x7f3ffc0b0320 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3fec011cc0 tx=0x7f3fec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f3ff17fa700 1 -- 172.21.15.31:0/4149319625 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec011ed0 con 0x7f3ffc158a50 2023-10-06T05:23:16.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f3ff17fa700 1 -- 172.21.15.31:0/4149319625 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec00fcd0 con 0x7f3ffc158a50 2023-10-06T05:23:16.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f40039ff700 1 -- 172.21.15.31:0/4149319625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc16db80 con 0x7f3ffc158a50 2023-10-06T05:23:16.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.514+0000 7f3ff17fa700 1 -- 172.21.15.31:0/4149319625 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec01d6b0 con 0x7f3ffc158a50 2023-10-06T05:23:16.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.515+0000 7f40039ff700 1 -- 172.21.15.31:0/4149319625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ffc0b0860 con 0x7f3ffc158a50 2023-10-06T05:23:16.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.516+0000 7f40039ff700 1 -- 172.21.15.31:0/4149319625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ffc0abbd0 con 0x7f3ffc158a50 2023-10-06T05:23:16.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.516+0000 7f3ff17fa700 1 -- 172.21.15.31:0/4149319625 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3fec01dae0 con 0x7f3ffc158a50 2023-10-06T05:23:16.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.517+0000 7f3ff17fa700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3fe0067320 0x7f3fe00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.517+0000 7f3ff17fa700 1 -- 172.21.15.31:0/4149319625 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f3fec018070 con 0x7f3ffc158a50 2023-10-06T05:23:16.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.518+0000 7f400179b700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3fe0067320 0x7f3fe00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.519+0000 7f400179b700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3fe0067320 0x7f3fe00697d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8008f90 tx=0x7f3ff8015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:16.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.522+0000 7f3ff17fa700 1 -- 172.21.15.31:0/4149319625 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3fec052900 con 0x7f3ffc158a50 2023-10-06T05:23:16.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.775+0000 7f40039ff700 1 -- 172.21.15.31:0/4149319625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ffc04c380 con 0x7f3ffc158a50 2023-10-06T05:23:16.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.776+0000 7f3ff17fa700 1 -- 172.21.15.31:0/4149319625 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f3fec055f20 con 0x7f3ffc158a50 2023-10-06T05:23:16.778 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:16.778 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1885821393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1885821393},{"type":"v1","addr":"172.21.15.31:6834","nonce":1885821393}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6889,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/258742412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":258742412},{"type":"v1","addr":"172.21.15.31:6835","nonce":258742412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4070877337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4070877337},{"type":"v1","addr":"172.21.15.62:6834","nonce":4070877337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:22:32.638176+0000","modified":"2023-10-06T05:22:36.157996+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7059},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7059":{"gid":7059,"name":"d","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2094863015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2094863015},{"type":"v1","addr":"172.21.15.62:6835","nonce":2094863015}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T05:23:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 -- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3fe0067320 msgr2=0x7f3fe00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3fe0067320 0x7f3fe00697d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8008f90 tx=0x7f3ff8015000 comp rx=0 tx=0).stop 2023-10-06T05:23:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 -- 172.21.15.31:0/4149319625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc158a50 msgr2=0x7f3ffc0b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc158a50 0x7f3ffc0b0320 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3fec011cc0 tx=0x7f3fec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 -- 172.21.15.31:0/4149319625 shutdown_connections 2023-10-06T05:23:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3fe0067320 0x7f3fe00697d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ffc15c860 0x7f3ffc13e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe8003c20 0x7f3ffc16ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 --2- 172.21.15.31:0/4149319625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ffc158a50 0x7f3ffc0b0320 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 -- 172.21.15.31:0/4149319625 >> 172.21.15.31:0/4149319625 conn(0x7f3ffc0cc0b0 msgr2=0x7f3ffc0caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 -- 172.21.15.31:0/4149319625 shutdown_connections 2023-10-06T05:23:16.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.778+0000 7f40039ff700 1 -- 172.21.15.31:0/4149319625 wait complete. 2023-10-06T05:23:16.782 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 68 2023-10-06T05:23:16.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:16.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.991+0000 7fa6911ff700 1 Processor -- start 2023-10-06T05:23:16.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.992+0000 7fa6911ff700 1 -- start start 2023-10-06T05:23:16.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.993+0000 7fa6911ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa68c1595a0 0x7fa68c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.993+0000 7fa6911ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c068d80 0x7fa68c0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.993+0000 7fa6911ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68c0c95b0 con 0x7fa68c0cc0d0 2023-10-06T05:23:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.993+0000 7fa6911ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68c0c9890 con 0x7fa68c1595a0 2023-10-06T05:23:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.993+0000 7fa6911ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c0c9b70 con 0x7fa68c068d80 2023-10-06T05:23:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.993+0000 7fa68ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa68c1595a0 0x7fa68c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.993+0000 7fa68ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa68c1595a0 0x7fa68c159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39886/0 (socket says 172.21.15.31:39886) 2023-10-06T05:23:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.993+0000 7fa68ad9d700 1 -- 172.21.15.31:0/770237137 learned_addr learned my addr 172.21.15.31:0/770237137 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.993+0000 7fa68b59e700 1 --2- 172.21.15.31:0/770237137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c068d80 0x7fa68c0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.994+0000 7fa68ad9d700 1 -- 172.21.15.31:0/770237137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c068d80 msgr2=0x7fa68c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:16.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.994+0000 7fa68ad9d700 1 --2- 172.21.15.31:0/770237137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c068d80 0x7fa68c0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.994+0000 7fa68ad9d700 1 -- 172.21.15.31:0/770237137 >> v1:172.21.15.62:6789/0 conn(0x7fa68c0cc0d0 legacy=0x7fa68c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.994+0000 7fa68ad9d700 1 -- 172.21.15.31:0/770237137 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c0ca130 con 0x7fa68c1595a0 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.994+0000 7fa68b59e700 1 --2- 172.21.15.31:0/770237137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c068d80 0x7fa68c0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.994+0000 7fa68ad9d700 1 --2- 172.21.15.31:0/770237137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 0x7fa68c159970 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0099b0 tx=0x7fa67c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc213643a061edf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.994+0000 7fa689d9b700 1 -- 172.21.15.31:0/770237137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00b860 con 0x7fa68c1595a0 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.994+0000 7fa689d9b700 1 -- 172.21.15.31:0/770237137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67c00b9c0 con 0x7fa68c1595a0 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.994+0000 7fa689d9b700 1 -- 172.21.15.31:0/770237137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00bd60 con 0x7fa68c1595a0 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.995+0000 7fa6911ff700 1 -- 172.21.15.31:0/770237137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 msgr2=0x7fa68c159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.995+0000 7fa6911ff700 1 --2- 172.21.15.31:0/770237137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 0x7fa68c159970 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0099b0 tx=0x7fa67c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.995+0000 7fa6911ff700 1 -- 172.21.15.31:0/770237137 shutdown_connections 2023-10-06T05:23:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.995+0000 7fa6911ff700 1 --2- 172.21.15.31:0/770237137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c068d80 0x7fa68c0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.995+0000 7fa6911ff700 1 --2- 172.21.15.31:0/770237137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 0x7fa68c159970 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.995+0000 7fa6911ff700 1 -- 172.21.15.31:0/770237137 >> 172.21.15.31:0/770237137 conn(0x7fa68c06b620 msgr2=0x7fa68c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:17.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.995+0000 7fa6911ff700 1 -- 172.21.15.31:0/770237137 shutdown_connections 2023-10-06T05:23:17.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.995+0000 7fa6911ff700 1 -- 172.21.15.31:0/770237137 wait complete. 2023-10-06T05:23:17.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.997+0000 7fa6911ff700 1 Processor -- start 2023-10-06T05:23:17.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.997+0000 7fa6911ff700 1 -- start start 2023-10-06T05:23:17.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa6911ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c068d80 0x7fa68c142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa6911ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c0cc0d0 0x7fa68c0c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa6911ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 0x7fa68c0c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa6911ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c15d930 con 0x7fa68c1595a0 2023-10-06T05:23:17.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa6911ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c142e40 con 0x7fa68c068d80 2023-10-06T05:23:17.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa6911ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c0c85e0 con 0x7fa68c0cc0d0 2023-10-06T05:23:17.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa68b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 0x7fa68c0c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa68b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 0x7fa68c0c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39896/0 (socket says 172.21.15.31:39896) 2023-10-06T05:23:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa68b59e700 1 -- 172.21.15.31:0/1054393938 learned_addr learned my addr 172.21.15.31:0/1054393938 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa68ad9d700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c068d80 0x7fa68c142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa68a59c700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c0cc0d0 0x7fa68c0c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa68b59e700 1 -- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c0cc0d0 msgr2=0x7fa68c0c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa68b59e700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c0cc0d0 0x7fa68c0c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa68b59e700 1 -- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c068d80 msgr2=0x7fa68c142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa68b59e700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c068d80 0x7fa68c142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.998+0000 7fa68b59e700 1 -- 172.21.15.31:0/1054393938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c0094f0 con 0x7fa68c1595a0 2023-10-06T05:23:17.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.999+0000 7fa68b59e700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 0x7fa68c0c80a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa684009f70 tx=0x7fa6840099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.999+0000 7fa672ffd700 1 -- 172.21.15.31:0/1054393938 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68400dce0 con 0x7fa68c1595a0 2023-10-06T05:23:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.999+0000 7fa672ffd700 1 -- 172.21.15.31:0/1054393938 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68400f040 con 0x7fa68c1595a0 2023-10-06T05:23:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.999+0000 7fa6911ff700 1 -- 172.21.15.31:0/1054393938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68c0c8890 con 0x7fa68c1595a0 2023-10-06T05:23:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.999+0000 7fa672ffd700 1 -- 172.21.15.31:0/1054393938 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684015650 con 0x7fa68c1595a0 2023-10-06T05:23:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:16.999+0000 7fa6911ff700 1 -- 172.21.15.31:0/1054393938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68c0c8e50 con 0x7fa68c1595a0 2023-10-06T05:23:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.000+0000 7fa6911ff700 1 -- 172.21.15.31:0/1054393938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68c15daa0 con 0x7fa68c1595a0 2023-10-06T05:23:17.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.001+0000 7fa672ffd700 1 -- 172.21.15.31:0/1054393938 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa684015a40 con 0x7fa68c1595a0 2023-10-06T05:23:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.002+0000 7fa672ffd700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa678067320 0x7fa6780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.002+0000 7fa672ffd700 1 -- 172.21.15.31:0/1054393938 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6114+0+0 (secure 0 0 0) 0x7fa684085780 con 0x7fa68c1595a0 2023-10-06T05:23:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.003+0000 7fa68ad9d700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa678067320 0x7fa6780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.003+0000 7fa68ad9d700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa678067320 0x7fa6780697d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0099b0 tx=0x7fa67c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.006+0000 7fa672ffd700 1 -- 172.21.15.31:0/1054393938 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa68c15daa0 con 0x7fa68c1595a0 2023-10-06T05:23:17.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.208+0000 7fa6911ff700 1 -- 172.21.15.31:0/1054393938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa68c15daa0 con 0x7fa68c1595a0 2023-10-06T05:23:17.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.209+0000 7fa672ffd700 1 -- 172.21.15.31:0/1054393938 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+17031 (secure 0 0 0) 0x7fa68c15daa0 con 0x7fa68c1595a0 2023-10-06T05:23:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:17.212 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":102,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:23:13.133045+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:22:30.850575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T05:22:31.607953+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:17.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 -- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa678067320 msgr2=0x7fa6780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa678067320 0x7fa6780697d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0099b0 tx=0x7fa67c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:17.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 -- 172.21.15.31:0/1054393938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 msgr2=0x7fa68c0c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 0x7fa68c0c80a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa684009f70 tx=0x7fa6840099b0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 -- 172.21.15.31:0/1054393938 shutdown_connections 2023-10-06T05:23:17.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa678067320 0x7fa6780697d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68c1595a0 0x7fa68c0c80a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c068d80 0x7fa68c142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 --2- 172.21.15.31:0/1054393938 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa68c0cc0d0 0x7fa68c0c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 -- 172.21.15.31:0/1054393938 >> 172.21.15.31:0/1054393938 conn(0x7fa68c06b620 msgr2=0x7fa68c15a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:17.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 -- 172.21.15.31:0/1054393938 shutdown_connections 2023-10-06T05:23:17.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.212+0000 7fa6911ff700 1 -- 172.21.15.31:0/1054393938 wait complete. 2023-10-06T05:23:17.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:17.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.427+0000 7f78e9bff700 1 Processor -- start 2023-10-06T05:23:17.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.427+0000 7f78e9bff700 1 -- start start 2023-10-06T05:23:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.429+0000 7f78e9bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f78e40cd510 0x7f78e40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.429+0000 7f78e9bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e40c3f40 con 0x7f78e415d2d0 2023-10-06T05:23:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.429+0000 7f78e9bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e40cb000 con 0x7f78e4159500 2023-10-06T05:23:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.429+0000 7f78e9bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e40b2860 con 0x7f78e40cd510 2023-10-06T05:23:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.429+0000 7f78e2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f78e40cd510 0x7f78e40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.429+0000 7f78e3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f78e415d2d0 0x7f78e40aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52724/0 (socket says 172.21.15.31:52724) 2023-10-06T05:23:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.429+0000 7f78e3fff700 1 -- 172.21.15.31:0/1871529018 learned_addr learned my addr 172.21.15.31:0/1871529018 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:17.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.429+0000 7f78e2ffd700 1 -- 172.21.15.31:0/1871529018 >> v1:172.21.15.62:6789/0 conn(0x7f78e4159500 legacy=0x7f78e40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.429+0000 7f78e2ffd700 1 -- 172.21.15.31:0/1871529018 >> v1:172.21.15.31:6789/0 conn(0x7f78e415d2d0 legacy=0x7f78e40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.429+0000 7f78e2ffd700 1 -- 172.21.15.31:0/1871529018 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e415d7d0 con 0x7f78e40cd510 2023-10-06T05:23:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.430+0000 7f78e2ffd700 1 --2- 172.21.15.31:0/1871529018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e40cd510 0x7f78e40b0ab0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f78d80098d0 tx=0x7f78d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df52da26f39879d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.430+0000 7f78e27fc700 1 -- 172.21.15.31:0/1871529018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8009e70 con 0x7f78e40cd510 2023-10-06T05:23:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.430+0000 7f78e27fc700 1 -- 172.21.15.31:0/1871529018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d800b860 con 0x7f78e40cd510 2023-10-06T05:23:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.430+0000 7f78e27fc700 1 -- 172.21.15.31:0/1871529018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d800bc00 con 0x7f78e40cd510 2023-10-06T05:23:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.430+0000 7f78e9bff700 1 -- 172.21.15.31:0/1871529018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e40cd510 msgr2=0x7f78e40b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.430+0000 7f78e9bff700 1 --2- 172.21.15.31:0/1871529018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e40cd510 0x7f78e40b0ab0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f78d80098d0 tx=0x7f78d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.431+0000 7f78e9bff700 1 -- 172.21.15.31:0/1871529018 shutdown_connections 2023-10-06T05:23:17.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.431+0000 7f78e9bff700 1 --2- 172.21.15.31:0/1871529018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e40cd510 0x7f78e40b0ab0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.431+0000 7f78e9bff700 1 -- 172.21.15.31:0/1871529018 >> 172.21.15.31:0/1871529018 conn(0x7f78e406ec90 msgr2=0x7f78e406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:17.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.431+0000 7f78e9bff700 1 -- 172.21.15.31:0/1871529018 shutdown_connections 2023-10-06T05:23:17.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.431+0000 7f78e9bff700 1 -- 172.21.15.31:0/1871529018 wait complete. 2023-10-06T05:23:17.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.432+0000 7f78e9bff700 1 Processor -- start 2023-10-06T05:23:17.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.432+0000 7f78e9bff700 1 -- start start 2023-10-06T05:23:17.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.433+0000 7f78e9bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78e40cd510 0x7f78e416d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.433+0000 7f78e9bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e4159500 0x7f78e40cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.433+0000 7f78e9bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e415d2d0 0x7f78e40ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.433+0000 7f78e9bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e416a3a0 con 0x7f78e415d2d0 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.433+0000 7f78e9bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e416b1c0 con 0x7f78e4159500 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.433+0000 7f78e9bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e416d710 con 0x7f78e40cd510 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.433+0000 7f78e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e415d2d0 0x7f78e40ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.433+0000 7f78e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e415d2d0 0x7f78e40ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39904/0 (socket says 172.21.15.31:39904) 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.433+0000 7f78e3fff700 1 -- 172.21.15.31:0/2370427683 learned_addr learned my addr 172.21.15.31:0/2370427683 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78e2ffd700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e4159500 0x7f78e40cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78e37fe700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78e40cd510 0x7f78e416d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78e3fff700 1 -- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78e40cd510 msgr2=0x7f78e416d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78e3fff700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78e40cd510 0x7f78e416d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78e3fff700 1 -- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e4159500 msgr2=0x7f78e40cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78e3fff700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e4159500 0x7f78e40cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78e3fff700 1 -- 172.21.15.31:0/2370427683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80093c0 con 0x7f78e415d2d0 2023-10-06T05:23:17.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78e37fe700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78e40cd510 0x7f78e416d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:17.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78e3fff700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e415d2d0 0x7f78e40ce530 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00cbf0 tx=0x7f78dc0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78cb7fe700 1 -- 172.21.15.31:0/2370427683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00a980 con 0x7f78e415d2d0 2023-10-06T05:23:17.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78cb7fe700 1 -- 172.21.15.31:0/2370427683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc00aae0 con 0x7f78e415d2d0 2023-10-06T05:23:17.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78cb7fe700 1 -- 172.21.15.31:0/2370427683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00ae60 con 0x7f78e415d2d0 2023-10-06T05:23:17.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.434+0000 7f78e9bff700 1 -- 172.21.15.31:0/2370427683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e416a5b0 con 0x7f78e415d2d0 2023-10-06T05:23:17.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.435+0000 7f78e9bff700 1 -- 172.21.15.31:0/2370427683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e416ab70 con 0x7f78e415d2d0 2023-10-06T05:23:17.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.436+0000 7f78e9bff700 1 -- 172.21.15.31:0/2370427683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e404c380 con 0x7f78e415d2d0 2023-10-06T05:23:17.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.437+0000 7f78cb7fe700 1 -- 172.21.15.31:0/2370427683 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f78dc007300 con 0x7f78e415d2d0 2023-10-06T05:23:17.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.437+0000 7f78cb7fe700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78d0067350 0x7f78d0069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.438+0000 7f78cb7fe700 1 -- 172.21.15.31:0/2370427683 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6114+0+0 (secure 0 0 0) 0x7f78dc085190 con 0x7f78e415d2d0 2023-10-06T05:23:17.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.438+0000 7f78e37fe700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78d0067350 0x7f78d0069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.438+0000 7f78e37fe700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78d0067350 0x7f78d0069800 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f78d4001840 tx=0x7f78d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.442+0000 7f78cb7fe700 1 -- 172.21.15.31:0/2370427683 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78dc051d00 con 0x7f78e415d2d0 2023-10-06T05:23:17.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.691+0000 7f78e9bff700 1 -- 172.21.15.31:0/2370427683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f78e416b300 con 0x7f78e415d2d0 2023-10-06T05:23:17.693 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:23:17.692+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:17.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.697+0000 7f78cb7fe700 1 -- 172.21.15.31:0/2370427683 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7f78dc055320 con 0x7f78e415d2d0 2023-10-06T05:23:17.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 -- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78d0067350 msgr2=0x7f78d0069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78d0067350 0x7f78d0069800 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f78d4001840 tx=0x7f78d4009040 comp rx=0 tx=0).stop 2023-10-06T05:23:17.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 -- 172.21.15.31:0/2370427683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e415d2d0 msgr2=0x7f78e40ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e415d2d0 0x7f78e40ce530 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00cbf0 tx=0x7f78dc0019c0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 -- 172.21.15.31:0/2370427683 shutdown_connections 2023-10-06T05:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78d0067350 0x7f78d0069800 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78e415d2d0 0x7f78e40ce530 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78e40cd510 0x7f78e416d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 --2- 172.21.15.31:0/2370427683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e4159500 0x7f78e40cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 -- 172.21.15.31:0/2370427683 >> 172.21.15.31:0/2370427683 conn(0x7f78e406ec90 msgr2=0x7f78e4158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 -- 172.21.15.31:0/2370427683 shutdown_connections 2023-10-06T05:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.700+0000 7f78e9bff700 1 -- 172.21.15.31:0/2370427683 wait complete. 2023-10-06T05:23:17.704 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:17.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:23:17.768 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:23:17.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.919+0000 7f88777ff700 1 Processor -- start 2023-10-06T05:23:17.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.919+0000 7f88777ff700 1 -- start start 2023-10-06T05:23:17.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.920+0000 7f88777ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88700b2bf0 0x7f88700c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.920+0000 7f88777ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f88700cce60 0x7f88700cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.920+0000 7f88777ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88700cbae0 con 0x7f88700b2820 2023-10-06T05:23:17.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.920+0000 7f88777ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88700cb280 con 0x7f88700cce60 2023-10-06T05:23:17.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.920+0000 7f88777ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88700cd7f0 con 0x7f88700b2bf0 2023-10-06T05:23:17.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f887559b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f88700b2820 0x7f88700cea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52726/0 (socket says 172.21.15.31:52726) 2023-10-06T05:23:17.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f887559b700 1 -- 172.21.15.31:0/2897776719 learned_addr learned my addr 172.21.15.31:0/2897776719 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:17.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8875d9c700 1 --2- 172.21.15.31:0/2897776719 >> v2:172.21.15.62:3300/0 conn(0x7f88700cce60 0x7f88700cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8874d9a700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88700b2bf0 0x7f88700c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521688112 0 0) 0x7f88700cbae0 con 0x7f88700b2820 2023-10-06T05:23:17.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f885c003570 con 0x7f88700b2820 2023-10-06T05:23:17.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4195951126 0 0) 0x7f885c003570 con 0x7f88700b2820 2023-10-06T05:23:17.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88700b2bf0 msgr2=0x7f88700c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:17.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8867fff700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88700b2bf0 0x7f88700c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 >> v2:172.21.15.62:3300/0 conn(0x7f88700cce60 msgr2=0x7f88700cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:17.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8867fff700 1 --2- 172.21.15.31:0/2897776719 >> v2:172.21.15.62:3300/0 conn(0x7f88700cce60 0x7f88700cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88700cf120 con 0x7f88700b2820 2023-10-06T05:23:17.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.921+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f886c0015f0 con 0x7f88700b2820 2023-10-06T05:23:17.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 >> v1:172.21.15.31:6789/0 conn(0x7f88700b2820 legacy=0x7f88700cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8867fff700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f885c003ae0 0x7f885c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8867fff700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885c004810 0x7f885c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8867fff700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88700b2bf0 0x7f885c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88700cb280 con 0x7f885c003ae0 2023-10-06T05:23:17.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88700cd7f0 con 0x7f885c004810 2023-10-06T05:23:17.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c00bb10 con 0x7f88700b2bf0 2023-10-06T05:23:17.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f887559b700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f885c003ae0 0x7f885c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8874d9a700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885c004810 0x7f885c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8875d9c700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88700b2bf0 0x7f885c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8874d9a700 1 -- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88700b2bf0 msgr2=0x7f885c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8874d9a700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88700b2bf0 0x7f885c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8874d9a700 1 -- 172.21.15.31:0/2897776719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f885c003ae0 msgr2=0x7f885c003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8874d9a700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f885c003ae0 0x7f885c003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f8874d9a700 1 -- 172.21.15.31:0/2897776719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c003570 con 0x7f885c004810 2023-10-06T05:23:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.922+0000 7f887559b700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f885c003ae0 0x7f885c003ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.923+0000 7f8874d9a700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885c004810 0x7f885c004c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88700b2fe0 tx=0x7f88600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6ef3ecd6e4eac73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.923+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000e710 con 0x7f885c004810 2023-10-06T05:23:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.923+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886000e870 con 0x7f885c004810 2023-10-06T05:23:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.923+0000 7f8867fff700 1 -- 172.21.15.31:0/2897776719 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000ec30 con 0x7f885c004810 2023-10-06T05:23:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.923+0000 7f88777ff700 1 -- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885c004810 msgr2=0x7f885c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.923+0000 7f88777ff700 1 --2- 172.21.15.31:0/2897776719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885c004810 0x7f885c004c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88700b2fe0 tx=0x7f88600091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.923+0000 7f8875d9c700 1 -- 172.21.15.31:0/2897776719 reap_dead start 2023-10-06T05:23:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.924+0000 7f88777ff700 1 -- 172.21.15.31:0/2897776719 shutdown_connections 2023-10-06T05:23:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.924+0000 7f88777ff700 1 -- 172.21.15.31:0/2897776719 >> 172.21.15.31:0/2897776719 conn(0x7f88700996f0 msgr2=0x7f887006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:17.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.924+0000 7f88777ff700 1 -- 172.21.15.31:0/2897776719 shutdown_connections 2023-10-06T05:23:17.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.924+0000 7f88777ff700 1 -- 172.21.15.31:0/2897776719 wait complete. 2023-10-06T05:23:17.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.925+0000 7f88777ff700 1 Processor -- start 2023-10-06T05:23:17.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.925+0000 7f88777ff700 1 -- start start 2023-10-06T05:23:17.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.926+0000 7f88777ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885c004810 0x7f887013bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.926+0000 7f88777ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f887013e680 0x7f88700b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.926+0000 7f88777ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88700b58d0 0x7f88701524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.926+0000 7f88777ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88700b5d40 con 0x7f887013e680 2023-10-06T05:23:17.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.926+0000 7f88777ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8870153270 con 0x7f88700b58d0 2023-10-06T05:23:17.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.926+0000 7f88777ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8870153df0 con 0x7f885c004810 2023-10-06T05:23:17.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.926+0000 7f8874d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f887013e680 0x7f88700b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.926+0000 7f8874d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f887013e680 0x7f88700b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39918/0 (socket says 172.21.15.31:39918) 2023-10-06T05:23:17.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.926+0000 7f8874d9a700 1 -- 172.21.15.31:0/686284480 learned_addr learned my addr 172.21.15.31:0/686284480 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:17.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f8875d9c700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88700b58d0 0x7f88701524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f887559b700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885c004810 0x7f887013bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f8874d9a700 1 -- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885c004810 msgr2=0x7f887013bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f8874d9a700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885c004810 0x7f887013bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f8874d9a700 1 -- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88700b58d0 msgr2=0x7f88701524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:17.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f8874d9a700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88700b58d0 0x7f88701524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:17.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f8874d9a700 1 -- 172.21.15.31:0/686284480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88701534c0 con 0x7f887013e680 2023-10-06T05:23:17.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f887559b700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885c004810 0x7f887013bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:17.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f8874d9a700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f887013e680 0x7f88700b5390 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8860013ef0 tx=0x7f886001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f88657fa700 1 -- 172.21.15.31:0/686284480 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8860022070 con 0x7f887013e680 2023-10-06T05:23:17.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f88657fa700 1 -- 172.21.15.31:0/686284480 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886000e9e0 con 0x7f887013e680 2023-10-06T05:23:17.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f88777ff700 1 -- 172.21.15.31:0/686284480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88701537a0 con 0x7f887013e680 2023-10-06T05:23:17.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.927+0000 7f88657fa700 1 -- 172.21.15.31:0/686284480 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000e710 con 0x7f887013e680 2023-10-06T05:23:17.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.928+0000 7f88777ff700 1 -- 172.21.15.31:0/686284480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887013eaf0 con 0x7f887013e680 2023-10-06T05:23:17.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.929+0000 7f88777ff700 1 -- 172.21.15.31:0/686284480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887004c380 con 0x7f887013e680 2023-10-06T05:23:17.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.929+0000 7f88657fa700 1 -- 172.21.15.31:0/686284480 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f886001c450 con 0x7f887013e680 2023-10-06T05:23:17.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.930+0000 7f88657fa700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8854067320 0x7f88540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:17.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.930+0000 7f88657fa700 1 -- 172.21.15.31:0/686284480 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f8860086850 con 0x7f887013e680 2023-10-06T05:23:17.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.931+0000 7f887559b700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8854067320 0x7f88540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:17.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.931+0000 7f887559b700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8854067320 0x7f88540697d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f886c001760 tx=0x7f886c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:17.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:17.934+0000 7f88657fa700 1 -- 172.21.15.31:0/686284480 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8860053390 con 0x7f887013e680 2023-10-06T05:23:18.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.184+0000 7f88777ff700 1 -- 172.21.15.31:0/686284480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8870153f30 con 0x7f887013e680 2023-10-06T05:23:18.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.698+0000 7f88657fa700 1 -- 172.21.15.31:0/686284480 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7f88600568f0 con 0x7f887013e680 2023-10-06T05:23:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.700+0000 7f88777ff700 1 -- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8854067320 msgr2=0x7f88540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.700+0000 7f88777ff700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8854067320 0x7f88540697d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f886c001760 tx=0x7f886c009040 comp rx=0 tx=0).stop 2023-10-06T05:23:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.700+0000 7f88777ff700 1 -- 172.21.15.31:0/686284480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f887013e680 msgr2=0x7f88700b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.700+0000 7f88777ff700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f887013e680 0x7f88700b5390 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8860013ef0 tx=0x7f886001a000 comp rx=0 tx=0).stop 2023-10-06T05:23:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.700+0000 7f88777ff700 1 -- 172.21.15.31:0/686284480 shutdown_connections 2023-10-06T05:23:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.700+0000 7f88777ff700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8854067320 0x7f88540697d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.700+0000 7f88777ff700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88700b58d0 0x7f88701524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.700+0000 7f88777ff700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885c004810 0x7f887013bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.700+0000 7f88777ff700 1 --2- 172.21.15.31:0/686284480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f887013e680 0x7f88700b5390 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.700+0000 7f88777ff700 1 -- 172.21.15.31:0/686284480 >> 172.21.15.31:0/686284480 conn(0x7f88700996f0 msgr2=0x7f8870159890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.701+0000 7f88777ff700 1 -- 172.21.15.31:0/686284480 shutdown_connections 2023-10-06T05:23:18.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.701+0000 7f88777ff700 1 -- 172.21.15.31:0/686284480 wait complete. 2023-10-06T05:23:18.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:23:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.921+0000 7fba08fff700 1 Processor -- start 2023-10-06T05:23:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.921+0000 7fba08fff700 1 -- start start 2023-10-06T05:23:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.922+0000 7fba08fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 0x7fba040ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.922+0000 7fba08fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0406e600 con 0x7fba0406ea90 2023-10-06T05:23:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.922+0000 7fba08fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0406ef00 con 0x7fba0406a630 2023-10-06T05:23:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.922+0000 7fba08fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0406d9f0 con 0x7fba04156300 2023-10-06T05:23:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.923+0000 7fba01d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fba0406ea90 0x7fba0406d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52734/0 (socket says 172.21.15.31:52734) 2023-10-06T05:23:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.923+0000 7fba01d9b700 1 -- 172.21.15.31:0/251451548 learned_addr learned my addr 172.21.15.31:0/251451548 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.923+0000 7fba0259c700 1 --2- 172.21.15.31:0/251451548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 0x7fba040ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.923+0000 7fba0159a700 1 -- 172.21.15.31:0/251451548 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120477631 0 0) 0x7fba0406e600 con 0x7fba0406ea90 2023-10-06T05:23:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.923+0000 7fba0159a700 1 -- 172.21.15.31:0/251451548 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ec003570 con 0x7fba0406ea90 2023-10-06T05:23:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.923+0000 7fba0259c700 1 -- 172.21.15.31:0/251451548 >> v1:172.21.15.62:6789/0 conn(0x7fba0406a630 legacy=0x7fba040c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.923+0000 7fba0159a700 1 -- 172.21.15.31:0/251451548 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638766191 0 0) 0x7fba0406ef00 con 0x7fba0406a630 2023-10-06T05:23:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.923+0000 7fba0259c700 1 -- 172.21.15.31:0/251451548 >> v1:172.21.15.31:6789/0 conn(0x7fba0406ea90 legacy=0x7fba0406d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.923+0000 7fba0259c700 1 -- 172.21.15.31:0/251451548 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0406dcd0 con 0x7fba04156300 2023-10-06T05:23:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.924+0000 7fba0259c700 1 --2- 172.21.15.31:0/251451548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 0x7fba040ad670 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009900 tx=0x7fb9f4009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c524a5a2bd7e77e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.924+0000 7fba0159a700 1 -- 172.21.15.31:0/251451548 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4005c10 con 0x7fba04156300 2023-10-06T05:23:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.924+0000 7fba0159a700 1 -- 172.21.15.31:0/251451548 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f4005d70 con 0x7fba04156300 2023-10-06T05:23:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.924+0000 7fba0159a700 1 -- 172.21.15.31:0/251451548 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4006240 con 0x7fba04156300 2023-10-06T05:23:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.925+0000 7fba08fff700 1 -- 172.21.15.31:0/251451548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 msgr2=0x7fba040ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.925+0000 7fba08fff700 1 --2- 172.21.15.31:0/251451548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 0x7fba040ad670 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009900 tx=0x7fb9f4009120 comp rx=0 tx=0).stop 2023-10-06T05:23:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.925+0000 7fba08fff700 1 -- 172.21.15.31:0/251451548 shutdown_connections 2023-10-06T05:23:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.925+0000 7fba08fff700 1 --2- 172.21.15.31:0/251451548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 0x7fba040ad670 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.925+0000 7fba08fff700 1 -- 172.21.15.31:0/251451548 >> 172.21.15.31:0/251451548 conn(0x7fba04068030 msgr2=0x7fba040cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.925+0000 7fba08fff700 1 -- 172.21.15.31:0/251451548 shutdown_connections 2023-10-06T05:23:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.925+0000 7fba08fff700 1 -- 172.21.15.31:0/251451548 wait complete. 2023-10-06T05:23:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.926+0000 7fba08fff700 1 Processor -- start 2023-10-06T05:23:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.926+0000 7fba08fff700 1 -- start start 2023-10-06T05:23:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.927+0000 7fba08fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba0406a630 0x7fba041537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.927+0000 7fba08fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba0406ea90 0x7fba040c1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.927+0000 7fba08fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 0x7fba040c1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.927+0000 7fba08fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba041580f0 con 0x7fba0406a630 2023-10-06T05:23:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.927+0000 7fba08fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba041574e0 con 0x7fba0406ea90 2023-10-06T05:23:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.927+0000 7fba08fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba041577c0 con 0x7fba04156300 2023-10-06T05:23:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.927+0000 7fba0259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba0406a630 0x7fba041537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fba0259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba0406a630 0x7fba041537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39922/0 (socket says 172.21.15.31:39922) 2023-10-06T05:23:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fba0259c700 1 -- 172.21.15.31:0/2549867419 learned_addr learned my addr 172.21.15.31:0/2549867419 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fba01d9b700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba0406ea90 0x7fba040c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fba02d9d700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 0x7fba040c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fba0259c700 1 -- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 msgr2=0x7fba040c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fba0259c700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 0x7fba040c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fba0259c700 1 -- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba0406ea90 msgr2=0x7fba040c1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fba0259c700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba0406ea90 0x7fba040c1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fba0259c700 1 -- 172.21.15.31:0/2549867419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f40093c0 con 0x7fba0406a630 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fba0259c700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba0406a630 0x7fba041537a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009160 tx=0x7fb9f4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.928+0000 7fb9f27fc700 1 -- 172.21.15.31:0/2549867419 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400b800 con 0x7fba0406a630 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.929+0000 7fb9f27fc700 1 -- 172.21.15.31:0/2549867419 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f400b960 con 0x7fba0406a630 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.929+0000 7fba08fff700 1 -- 172.21.15.31:0/2549867419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba04157aa0 con 0x7fba0406a630 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.929+0000 7fb9f27fc700 1 -- 172.21.15.31:0/2549867419 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400bd20 con 0x7fba0406a630 2023-10-06T05:23:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.929+0000 7fba08fff700 1 -- 172.21.15.31:0/2549867419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba04153ce0 con 0x7fba0406a630 2023-10-06T05:23:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.930+0000 7fba08fff700 1 -- 172.21.15.31:0/2549867419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0404c380 con 0x7fba0406a630 2023-10-06T05:23:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.931+0000 7fb9f27fc700 1 -- 172.21.15.31:0/2549867419 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb9f401b690 con 0x7fba0406a630 2023-10-06T05:23:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.932+0000 7fb9f27fc700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9e4067320 0x7fb9e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.932+0000 7fb9f27fc700 1 -- 172.21.15.31:0/2549867419 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fb9f4086f50 con 0x7fba0406a630 2023-10-06T05:23:18.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.932+0000 7fba01d9b700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9e4067320 0x7fb9e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:18.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.933+0000 7fba01d9b700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9e4067320 0x7fb9e40697d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba0415ec80 tx=0x7fb9f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:18.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:18.937+0000 7fb9f27fc700 1 -- 172.21.15.31:0/2549867419 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9f4053a10 con 0x7fba0406a630 2023-10-06T05:23:19.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.140+0000 7fba08fff700 1 -- 172.21.15.31:0/2549867419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fba0406dfb0 con 0x7fba0406a630 2023-10-06T05:23:19.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.700+0000 7fb9f27fc700 1 -- 172.21.15.31:0/2549867419 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb9f4057030 con 0x7fba0406a630 2023-10-06T05:23:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.705+0000 7fba08fff700 1 -- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9e4067320 msgr2=0x7fb9e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.705+0000 7fba08fff700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9e4067320 0x7fb9e40697d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba0415ec80 tx=0x7fb9f8009040 comp rx=0 tx=0).stop 2023-10-06T05:23:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.705+0000 7fba08fff700 1 -- 172.21.15.31:0/2549867419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba0406a630 msgr2=0x7fba041537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.705+0000 7fba08fff700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba0406a630 0x7fba041537a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009160 tx=0x7fb9f4016000 comp rx=0 tx=0).stop 2023-10-06T05:23:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.705+0000 7fba08fff700 1 -- 172.21.15.31:0/2549867419 shutdown_connections 2023-10-06T05:23:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.705+0000 7fba08fff700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9e4067320 0x7fb9e40697d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.705+0000 7fba08fff700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba04156300 0x7fba040c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.705+0000 7fba08fff700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba0406a630 0x7fba041537a0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.705+0000 7fba08fff700 1 --2- 172.21.15.31:0/2549867419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba0406ea90 0x7fba040c1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.705+0000 7fba08fff700 1 -- 172.21.15.31:0/2549867419 >> 172.21.15.31:0/2549867419 conn(0x7fba04068030 msgr2=0x7fba040c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.706+0000 7fba08fff700 1 -- 172.21.15.31:0/2549867419 shutdown_connections 2023-10-06T05:23:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.706+0000 7fba08fff700 1 -- 172.21.15.31:0/2549867419 wait complete. 2023-10-06T05:23:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:23:19.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:23:19.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.930+0000 7f1a145ff700 1 Processor -- start 2023-10-06T05:23:19.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.930+0000 7f1a145ff700 1 -- start start 2023-10-06T05:23:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.931+0000 7f1a145ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a0c156300 0x7f1a0c0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.931+0000 7f1a145ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a0c06ea90 0x7f1a0c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.931+0000 7f1a145ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c06e600 con 0x7f1a0c06a630 2023-10-06T05:23:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.931+0000 7f1a145ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c06ef00 con 0x7f1a0c156300 2023-10-06T05:23:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.931+0000 7f1a145ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c06d750 con 0x7f1a0c06ea90 2023-10-06T05:23:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.931+0000 7f1a1239b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a0c156300 0x7f1a0c0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.931+0000 7f1a12b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1a0c06a630 0x7f1a0c0c6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52744/0 (socket says 172.21.15.31:52744) 2023-10-06T05:23:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.931+0000 7f1a12b9c700 1 -- 172.21.15.31:0/1362210166 learned_addr learned my addr 172.21.15.31:0/1362210166 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.932+0000 7f1a11b9a700 1 --2- 172.21.15.31:0/1362210166 >> v2:172.21.15.62:3300/0 conn(0x7f1a0c06ea90 0x7f1a0c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.932+0000 7f1a1239b700 1 -- 172.21.15.31:0/1362210166 >> v2:172.21.15.62:3300/0 conn(0x7f1a0c06ea90 msgr2=0x7f1a0c06d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.932+0000 7f1a1239b700 1 --2- 172.21.15.31:0/1362210166 >> v2:172.21.15.62:3300/0 conn(0x7f1a0c06ea90 0x7f1a0c06d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.932+0000 7f1a1239b700 1 -- 172.21.15.31:0/1362210166 >> v1:172.21.15.31:6789/0 conn(0x7f1a0c06a630 legacy=0x7f1a0c0c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.932+0000 7f1a1239b700 1 -- 172.21.15.31:0/1362210166 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c06d9f0 con 0x7f1a0c156300 2023-10-06T05:23:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.932+0000 7f1a1239b700 1 --2- 172.21.15.31:0/1362210166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 0x7f1a0c0ad670 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0c3020 tx=0x7f1a08009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652e882c8f23d265 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.932+0000 7f1a11399700 1 -- 172.21.15.31:0/1362210166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a08005c10 con 0x7f1a0c156300 2023-10-06T05:23:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.932+0000 7f1a11399700 1 -- 172.21.15.31:0/1362210166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a08005d70 con 0x7f1a0c156300 2023-10-06T05:23:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.932+0000 7f1a11399700 1 -- 172.21.15.31:0/1362210166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a08006260 con 0x7f1a0c156300 2023-10-06T05:23:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.933+0000 7f1a145ff700 1 -- 172.21.15.31:0/1362210166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 msgr2=0x7f1a0c0ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.933+0000 7f1a145ff700 1 --2- 172.21.15.31:0/1362210166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 0x7f1a0c0ad670 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0c3020 tx=0x7f1a08009120 comp rx=0 tx=0).stop 2023-10-06T05:23:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.933+0000 7f1a145ff700 1 -- 172.21.15.31:0/1362210166 shutdown_connections 2023-10-06T05:23:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.933+0000 7f1a145ff700 1 --2- 172.21.15.31:0/1362210166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 0x7f1a0c0ad670 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.933+0000 7f1a145ff700 1 --2- 172.21.15.31:0/1362210166 >> v2:172.21.15.62:3300/0 conn(0x7f1a0c06ea90 0x7f1a0c06d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.933+0000 7f1a145ff700 1 -- 172.21.15.31:0/1362210166 >> 172.21.15.31:0/1362210166 conn(0x7f1a0c068030 msgr2=0x7f1a0c0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.933+0000 7f1a145ff700 1 -- 172.21.15.31:0/1362210166 shutdown_connections 2023-10-06T05:23:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.933+0000 7f1a145ff700 1 -- 172.21.15.31:0/1362210166 wait complete. 2023-10-06T05:23:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.935+0000 7f1a145ff700 1 Processor -- start 2023-10-06T05:23:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.935+0000 7f1a145ff700 1 -- start start 2023-10-06T05:23:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a145ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0c06a630 0x7f1a0c14a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a145ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a0c06ea90 0x7f1a0c158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a145ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 0x7f1a0c0c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a145ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c157d40 con 0x7f1a0c156300 2023-10-06T05:23:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a145ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c156ca0 con 0x7f1a0c06a630 2023-10-06T05:23:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a145ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c156f80 con 0x7f1a0c06ea90 2023-10-06T05:23:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a12b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 0x7f1a0c0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a12b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 0x7f1a0c0c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39944/0 (socket says 172.21.15.31:39944) 2023-10-06T05:23:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a12b9c700 1 -- 172.21.15.31:0/3960277181 learned_addr learned my addr 172.21.15.31:0/3960277181 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a1239b700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0c06a630 0x7f1a0c14a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.936+0000 7f1a11b9a700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a0c06ea90 0x7f1a0c158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a12b9c700 1 -- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a0c06ea90 msgr2=0x7f1a0c158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a12b9c700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a0c06ea90 0x7f1a0c158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a12b9c700 1 -- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0c06a630 msgr2=0x7f1a0c14a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a12b9c700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0c06a630 0x7f1a0c14a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a12b9c700 1 -- 172.21.15.31:0/3960277181 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a080093c0 con 0x7f1a0c156300 2023-10-06T05:23:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a12b9c700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 0x7f1a0c0c74a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400cc40 tx=0x7f1a04001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a027fc700 1 -- 172.21.15.31:0/3960277181 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400a8b0 con 0x7f1a0c156300 2023-10-06T05:23:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a027fc700 1 -- 172.21.15.31:0/3960277181 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0400aa10 con 0x7f1a0c156300 2023-10-06T05:23:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a145ff700 1 -- 172.21.15.31:0/3960277181 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c157260 con 0x7f1a0c156300 2023-10-06T05:23:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a027fc700 1 -- 172.21.15.31:0/3960277181 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400add0 con 0x7f1a0c156300 2023-10-06T05:23:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.937+0000 7f1a145ff700 1 -- 172.21.15.31:0/3960277181 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c157820 con 0x7f1a0c156300 2023-10-06T05:23:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.938+0000 7f1a145ff700 1 -- 172.21.15.31:0/3960277181 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c04c380 con 0x7f1a0c156300 2023-10-06T05:23:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.939+0000 7f1a027fc700 1 -- 172.21.15.31:0/3960277181 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1a04007300 con 0x7f1a0c156300 2023-10-06T05:23:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.940+0000 7f1a027fc700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067320 0x7f19f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.940+0000 7f1a027fc700 1 -- 172.21.15.31:0/3960277181 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f1a04084f40 con 0x7f1a0c156300 2023-10-06T05:23:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.941+0000 7f1a1239b700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067320 0x7f19f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.942+0000 7f1a1239b700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067320 0x7f19f00697d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009160 tx=0x7f1a08016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:19.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:19.945+0000 7f1a027fc700 1 -- 172.21.15.31:0/3960277181 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a04051bf0 con 0x7f1a0c156300 2023-10-06T05:23:20.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.146+0000 7f1a145ff700 1 -- 172.21.15.31:0/3960277181 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1a0c06dcd0 con 0x7f1a0c156300 2023-10-06T05:23:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.701+0000 7f1a027fc700 1 -- 172.21.15.31:0/3960277181 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1a04055210 con 0x7f1a0c156300 2023-10-06T05:23:20.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.710+0000 7f1a145ff700 1 -- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067320 msgr2=0x7f19f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.710+0000 7f1a145ff700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067320 0x7f19f00697d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009160 tx=0x7f1a08016000 comp rx=0 tx=0).stop 2023-10-06T05:23:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.710+0000 7f1a145ff700 1 -- 172.21.15.31:0/3960277181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 msgr2=0x7f1a0c0c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.710+0000 7f1a145ff700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 0x7f1a0c0c74a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400cc40 tx=0x7f1a04001a50 comp rx=0 tx=0).stop 2023-10-06T05:23:20.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.711+0000 7f1a145ff700 1 -- 172.21.15.31:0/3960277181 shutdown_connections 2023-10-06T05:23:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.711+0000 7f1a145ff700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f19f0067320 0x7f19f00697d0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009160 tx=0x7f1a08016000 comp rx=0 tx=0).stop 2023-10-06T05:23:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.711+0000 7f1a145ff700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a0c156300 0x7f1a0c0c74a0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.711+0000 7f1a145ff700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0c06a630 0x7f1a0c14a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.711+0000 7f1a145ff700 1 --2- 172.21.15.31:0/3960277181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a0c06ea90 0x7f1a0c158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.711+0000 7f1a145ff700 1 -- 172.21.15.31:0/3960277181 >> 172.21.15.31:0/3960277181 conn(0x7f1a0c068030 msgr2=0x7f1a0c155470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.722+0000 7f1a145ff700 1 -- 172.21.15.31:0/3960277181 shutdown_connections 2023-10-06T05:23:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.722+0000 7f1a145ff700 1 -- 172.21.15.31:0/3960277181 wait complete. 2023-10-06T05:23:20.728 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:23:20.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:23:20.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.947+0000 7f8fa77ff700 1 Processor -- start 2023-10-06T05:23:20.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.947+0000 7f8fa77ff700 1 -- start start 2023-10-06T05:23:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.949+0000 7f8fa77ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8fa00cd580 0x7f8fa01645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.949+0000 7f8fa77ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa015d2d0 con 0x7f8fa0159560 2023-10-06T05:23:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.949+0000 7f8fa77ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa015d5b0 con 0x7f8fa0159930 2023-10-06T05:23:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.949+0000 7f8fa77ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa015d890 con 0x7f8fa00cd580 2023-10-06T05:23:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.949+0000 7f8fa559b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8fa0159560 0x7f8fa00cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52756/0 (socket says 172.21.15.31:52756) 2023-10-06T05:23:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.949+0000 7f8fa559b700 1 -- 172.21.15.31:0/35480302 learned_addr learned my addr 172.21.15.31:0/35480302 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.949+0000 7f8fa5d9c700 1 --2- 172.21.15.31:0/35480302 >> v2:172.21.15.62:3300/0 conn(0x7f8fa00cd580 0x7f8fa01645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.949+0000 7f8f97fff700 1 -- 172.21.15.31:0/35480302 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346747421 0 0) 0x7f8fa015d2d0 con 0x7f8fa0159560 2023-10-06T05:23:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.949+0000 7f8f97fff700 1 -- 172.21.15.31:0/35480302 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f8c003570 con 0x7f8fa0159560 2023-10-06T05:23:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.950+0000 7f8f97fff700 1 -- 172.21.15.31:0/35480302 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968809209 0 0) 0x7f8fa015d5b0 con 0x7f8fa0159930 2023-10-06T05:23:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.950+0000 7f8f97fff700 1 -- 172.21.15.31:0/35480302 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fa015d2d0 con 0x7f8fa0159930 2023-10-06T05:23:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.950+0000 7f8fa5d9c700 1 -- 172.21.15.31:0/35480302 >> v1:172.21.15.62:6789/0 conn(0x7f8fa0159930 legacy=0x7f8fa00b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.950+0000 7f8fa5d9c700 1 -- 172.21.15.31:0/35480302 >> v1:172.21.15.31:6789/0 conn(0x7f8fa0159560 legacy=0x7f8fa00cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.950+0000 7f8f97fff700 1 -- 172.21.15.31:0/35480302 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2809789529 0 0) 0x7f8f8c003570 con 0x7f8fa0159560 2023-10-06T05:23:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.950+0000 7f8fa5d9c700 1 -- 172.21.15.31:0/35480302 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa015db70 con 0x7f8fa00cd580 2023-10-06T05:23:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.950+0000 7f8fa5d9c700 1 --2- 172.21.15.31:0/35480302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa00cd580 0x7f8fa01645d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009e60 tx=0x7f8f98009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a725d078ad101a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.950+0000 7f8f97fff700 1 -- 172.21.15.31:0/35480302 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800e070 con 0x7f8fa00cd580 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.950+0000 7f8f97fff700 1 -- 172.21.15.31:0/35480302 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9800b7a0 con 0x7f8fa00cd580 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.950+0000 7f8f97fff700 1 -- 172.21.15.31:0/35480302 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c001470 con 0x7f8fa00cd580 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.951+0000 7f8fa77ff700 1 -- 172.21.15.31:0/35480302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa00cd580 msgr2=0x7f8fa01645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.951+0000 7f8fa77ff700 1 --2- 172.21.15.31:0/35480302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa00cd580 0x7f8fa01645d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009e60 tx=0x7f8f98009780 comp rx=0 tx=0).stop 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.951+0000 7f8fa77ff700 1 -- 172.21.15.31:0/35480302 shutdown_connections 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.951+0000 7f8fa77ff700 1 --2- 172.21.15.31:0/35480302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa00cd580 0x7f8fa01645d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.951+0000 7f8fa77ff700 1 -- 172.21.15.31:0/35480302 >> 172.21.15.31:0/35480302 conn(0x7f8fa006ec90 msgr2=0x7f8fa006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.951+0000 7f8fa77ff700 1 -- 172.21.15.31:0/35480302 shutdown_connections 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.951+0000 7f8fa77ff700 1 -- 172.21.15.31:0/35480302 wait complete. 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.953+0000 7f8fa77ff700 1 Processor -- start 2023-10-06T05:23:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.953+0000 7f8fa77ff700 1 -- start start 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa77ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa00cd580 0x7f8fa0157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa77ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa0159560 0x7f8fa00c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa77ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa0159930 0x7f8fa00d0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa77ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa00af860 con 0x7f8fa00cd580 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa77ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa016adb0 con 0x7f8fa0159560 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa77ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa016a230 con 0x7f8fa0159930 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa559b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa00cd580 0x7f8fa0157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa559b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa00cd580 0x7f8fa0157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39958/0 (socket says 172.21.15.31:39958) 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa559b700 1 -- 172.21.15.31:0/3201157571 learned_addr learned my addr 172.21.15.31:0/3201157571 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa4d9a700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa0159560 0x7f8fa00c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa5d9c700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa0159930 0x7f8fa00d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa559b700 1 -- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa0159930 msgr2=0x7f8fa00d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa559b700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa0159930 0x7f8fa00d0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa559b700 1 -- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa0159560 msgr2=0x7f8fa00c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa559b700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa0159560 0x7f8fa00c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.954+0000 7f8fa559b700 1 -- 172.21.15.31:0/3201157571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f98009a20 con 0x7f8fa00cd580 2023-10-06T05:23:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.955+0000 7f8fa559b700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa00cd580 0x7f8fa0157c40 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00cb90 tx=0x7f8f9c001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.955+0000 7f8f957fa700 1 -- 172.21.15.31:0/3201157571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00cea0 con 0x7f8fa00cd580 2023-10-06T05:23:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.955+0000 7f8f957fa700 1 -- 172.21.15.31:0/3201157571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c007e90 con 0x7f8fa00cd580 2023-10-06T05:23:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.955+0000 7f8fa77ff700 1 -- 172.21.15.31:0/3201157571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa016a510 con 0x7f8fa00cd580 2023-10-06T05:23:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.955+0000 7f8f957fa700 1 -- 172.21.15.31:0/3201157571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0075d0 con 0x7f8fa00cd580 2023-10-06T05:23:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.955+0000 7f8fa77ff700 1 -- 172.21.15.31:0/3201157571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa016aad0 con 0x7f8fa00cd580 2023-10-06T05:23:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.956+0000 7f8fa77ff700 1 -- 172.21.15.31:0/3201157571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa00c74a0 con 0x7f8fa00cd580 2023-10-06T05:23:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.957+0000 7f8f957fa700 1 -- 172.21.15.31:0/3201157571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8f9c004a40 con 0x7f8fa00cd580 2023-10-06T05:23:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.958+0000 7f8f957fa700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f84067320 0x7f8f840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.958+0000 7f8f957fa700 1 -- 172.21.15.31:0/3201157571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f8f9c083e80 con 0x7f8fa00cd580 2023-10-06T05:23:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.958+0000 7f8fa4d9a700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f84067320 0x7f8f840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.959+0000 7f8fa4d9a700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f84067320 0x7f8f840697d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00b4980 tx=0x7f8f90007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:20.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:20.962+0000 7f8f957fa700 1 -- 172.21.15.31:0/3201157571 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fa00c74a0 con 0x7f8fa00cd580 2023-10-06T05:23:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.168+0000 7f8fa77ff700 1 -- 172.21.15.31:0/3201157571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8fa00c74a0 con 0x7f8fa00cd580 2023-10-06T05:23:21.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.172+0000 7f8f957fa700 1 -- 172.21.15.31:0/3201157571 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8fa00c74a0 con 0x7f8fa00cd580 2023-10-06T05:23:21.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 -- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f84067320 msgr2=0x7f8f840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f84067320 0x7f8f840697d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00b4980 tx=0x7f8f90007d50 comp rx=0 tx=0).stop 2023-10-06T05:23:21.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 -- 172.21.15.31:0/3201157571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa00cd580 msgr2=0x7f8fa0157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa00cd580 0x7f8fa0157c40 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00cb90 tx=0x7f8f9c001b50 comp rx=0 tx=0).stop 2023-10-06T05:23:21.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 -- 172.21.15.31:0/3201157571 shutdown_connections 2023-10-06T05:23:21.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8f84067320 0x7f8f840697d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8fa0159930 0x7f8fa00d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa00cd580 0x7f8fa0157c40 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 --2- 172.21.15.31:0/3201157571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fa0159560 0x7f8fa00c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 -- 172.21.15.31:0/3201157571 >> 172.21.15.31:0/3201157571 conn(0x7f8fa006ec90 msgr2=0x7f8fa01588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:21.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 -- 172.21.15.31:0/3201157571 shutdown_connections 2023-10-06T05:23:21.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.174+0000 7f8fa77ff700 1 -- 172.21.15.31:0/3201157571 wait complete. 2023-10-06T05:23:21.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2023-10-06T05:23:21.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.387+0000 7f59283ff700 1 Processor -- start 2023-10-06T05:23:21.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.387+0000 7f59283ff700 1 -- start start 2023-10-06T05:23:21.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.388+0000 7f59283ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f59200ca770 0x7f59200c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.388+0000 7f59283ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f592006b510 0x7f59201569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.388+0000 7f59283ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5920156ef0 con 0x7f59200cab40 2023-10-06T05:23:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.388+0000 7f59283ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59200c7a70 con 0x7f59200ca770 2023-10-06T05:23:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.388+0000 7f59283ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59200c6df0 con 0x7f592006b510 2023-10-06T05:23:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.389+0000 7f592599a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f59200cab40 0x7f592006ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52768/0 (socket says 172.21.15.31:52768) 2023-10-06T05:23:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.389+0000 7f592599a700 1 -- 172.21.15.31:0/1214791778 learned_addr learned my addr 172.21.15.31:0/1214791778 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:21.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.389+0000 7f592699c700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f592006b510 0x7f59201569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.389+0000 7f592619b700 1 --2- 172.21.15.31:0/1214791778 >> v2:172.21.15.62:3300/0 conn(0x7f59200ca770 0x7f59200c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.389+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609632103 0 0) 0x7f5920156ef0 con 0x7f59200cab40 2023-10-06T05:23:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.389+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f590c004680 con 0x7f59200cab40 2023-10-06T05:23:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.389+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2528117370 0 0) 0x7f590c004680 con 0x7f59200cab40 2023-10-06T05:23:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.389+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f592006b510 msgr2=0x7f59201569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f592006b510 0x7f59201569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 >> v2:172.21.15.62:3300/0 conn(0x7f59200ca770 msgr2=0x7f59200c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 --2- 172.21.15.31:0/1214791778 >> v2:172.21.15.62:3300/0 conn(0x7f59200ca770 0x7f59200c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59200c70d0 con 0x7f59200cab40 2023-10-06T05:23:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f5910001470 con 0x7f59200cab40 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 >> v1:172.21.15.31:6789/0 conn(0x7f59200cab40 legacy=0x7f592006ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f590c004bf0 0x7f590c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f590c005920 0x7f590c005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f590c0062d0 0x7f590c00cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59200c7a70 con 0x7f590c005920 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59200c6df0 con 0x7f590c004bf0 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c00d0e0 con 0x7f590c0062d0 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f592619b700 1 --2- 172.21.15.31:0/1214791778 >> v2:172.21.15.62:3300/0 conn(0x7f59200ca770 0x7f59200c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f592699c700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f592006b510 0x7f59201569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f592599a700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f590c005920 0x7f590c005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f592699c700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f590c0062d0 0x7f590c00cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f592619b700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f590c004bf0 0x7f590c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f592599a700 1 -- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f590c0062d0 msgr2=0x7f590c00cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f592599a700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f590c0062d0 0x7f590c00cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f592599a700 1 -- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f590c004bf0 msgr2=0x7f590c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f592599a700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f590c004bf0 0x7f590c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.390+0000 7f592599a700 1 -- 172.21.15.31:0/1214791778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c004680 con 0x7f590c005920 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.391+0000 7f592599a700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f590c005920 0x7f590c005d90 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f591000c2a0 tx=0x7f5920155ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6706d7b6488f297 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.391+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591000cb70 con 0x7f590c005920 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.391+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591000ccd0 con 0x7f590c005920 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.391+0000 7f5925199700 1 -- 172.21.15.31:0/1214791778 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910005b70 con 0x7f590c005920 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.391+0000 7f59283ff700 1 -- 172.21.15.31:0/1214791778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f590c005920 msgr2=0x7f590c005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.391+0000 7f59283ff700 1 --2- 172.21.15.31:0/1214791778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f590c005920 0x7f590c005d90 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f591000c2a0 tx=0x7f5920155ff0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.391+0000 7f592699c700 1 -- 172.21.15.31:0/1214791778 reap_dead start 2023-10-06T05:23:21.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.391+0000 7f59283ff700 1 -- 172.21.15.31:0/1214791778 shutdown_connections 2023-10-06T05:23:21.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.391+0000 7f59283ff700 1 -- 172.21.15.31:0/1214791778 >> 172.21.15.31:0/1214791778 conn(0x7f59200cc0b0 msgr2=0x7f59200cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:21.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.392+0000 7f59283ff700 1 -- 172.21.15.31:0/1214791778 shutdown_connections 2023-10-06T05:23:21.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.392+0000 7f59283ff700 1 -- 172.21.15.31:0/1214791778 wait complete. 2023-10-06T05:23:21.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.394+0000 7f59283ff700 1 Processor -- start 2023-10-06T05:23:21.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.394+0000 7f59283ff700 1 -- start start 2023-10-06T05:23:21.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.395+0000 7f59283ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f590c005920 0x7f59200ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.395+0000 7f59283ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59200af0d0 0x7f59200b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.395+0000 7f59283ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59200b1be0 0x7f592014ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.395+0000 7f59283ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59200b0170 con 0x7f59200af0d0 2023-10-06T05:23:21.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.395+0000 7f59283ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59200af560 con 0x7f590c005920 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.395+0000 7f59283ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59200af840 con 0x7f59200b1be0 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.395+0000 7f592599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59200af0d0 0x7f59200b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.395+0000 7f592599a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59200af0d0 0x7f59200b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39974/0 (socket says 172.21.15.31:39974) 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.395+0000 7f592599a700 1 -- 172.21.15.31:0/2801084541 learned_addr learned my addr 172.21.15.31:0/2801084541 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.395+0000 7f592619b700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f590c005920 0x7f59200ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.396+0000 7f592699c700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59200b1be0 0x7f592014ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.396+0000 7f592599a700 1 -- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59200b1be0 msgr2=0x7f592014ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.396+0000 7f592599a700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59200b1be0 0x7f592014ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.396+0000 7f592599a700 1 -- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f590c005920 msgr2=0x7f59200ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.396+0000 7f592599a700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f590c005920 0x7f59200ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.396+0000 7f592599a700 1 -- 172.21.15.31:0/2801084541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59200afb20 con 0x7f59200af0d0 2023-10-06T05:23:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.396+0000 7f592599a700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59200af0d0 0x7f59200b16a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f590c005840 tx=0x7f5910015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.396+0000 7f59167fc700 1 -- 172.21.15.31:0/2801084541 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591000cb70 con 0x7f59200af0d0 2023-10-06T05:23:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.396+0000 7f59167fc700 1 -- 172.21.15.31:0/2801084541 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591000ce40 con 0x7f59200af0d0 2023-10-06T05:23:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.396+0000 7f59283ff700 1 -- 172.21.15.31:0/2801084541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59200afe00 con 0x7f59200af0d0 2023-10-06T05:23:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.397+0000 7f59167fc700 1 -- 172.21.15.31:0/2801084541 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910007550 con 0x7f59200af0d0 2023-10-06T05:23:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.397+0000 7f59283ff700 1 -- 172.21.15.31:0/2801084541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59200b0600 con 0x7f59200af0d0 2023-10-06T05:23:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.398+0000 7f59283ff700 1 -- 172.21.15.31:0/2801084541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59200b0320 con 0x7f59200af0d0 2023-10-06T05:23:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.399+0000 7f59167fc700 1 -- 172.21.15.31:0/2801084541 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5910019030 con 0x7f59200af0d0 2023-10-06T05:23:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.399+0000 7f59167fc700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5904067320 0x7f59040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.399+0000 7f59167fc700 1 -- 172.21.15.31:0/2801084541 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f59100860a0 con 0x7f59200af0d0 2023-10-06T05:23:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.400+0000 7f592619b700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5904067320 0x7f59040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.400+0000 7f592619b700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5904067320 0x7f59040697d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f591c009fd0 tx=0x7f591c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:21.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.404+0000 7f59167fc700 1 -- 172.21.15.31:0/2801084541 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59100b75a0 con 0x7f59200af0d0 2023-10-06T05:23:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.687+0000 7f59283ff700 1 -- 172.21.15.31:0/2801084541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7f59200b0170 con 0x7f59200af0d0 2023-10-06T05:23:21.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.711+0000 7f59167fc700 1 -- 172.21.15.31:0/2801084541 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v341) v1 ==== 149+0+0 (secure 0 0 0) 0x7f59200b0320 con 0x7f59200af0d0 2023-10-06T05:23:21.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 -- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5904067320 msgr2=0x7f59040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5904067320 0x7f59040697d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f591c009fd0 tx=0x7f591c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:21.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 -- 172.21.15.31:0/2801084541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59200af0d0 msgr2=0x7f59200b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59200af0d0 0x7f59200b16a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f590c005840 tx=0x7f5910015000 comp rx=0 tx=0).stop 2023-10-06T05:23:21.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 -- 172.21.15.31:0/2801084541 shutdown_connections 2023-10-06T05:23:21.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5904067320 0x7f59040697d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59200b1be0 0x7f592014ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f590c005920 0x7f59200ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 --2- 172.21.15.31:0/2801084541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59200af0d0 0x7f59200b16a0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 -- 172.21.15.31:0/2801084541 >> 172.21.15.31:0/2801084541 conn(0x7f59200cc0b0 msgr2=0x7f592014cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 -- 172.21.15.31:0/2801084541 shutdown_connections 2023-10-06T05:23:21.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.714+0000 7f59283ff700 1 -- 172.21.15.31:0/2801084541 wait complete. 2023-10-06T05:23:21.733 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2023-10-06T05:23:21.733 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2023-10-06T05:23:21.733 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:23:21.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2023-10-06T05:23:21.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.926+0000 7f3d1ddff700 1 Processor -- start 2023-10-06T05:23:21.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.926+0000 7f3d1ddff700 1 -- start start 2023-10-06T05:23:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.928+0000 7f3d1ddff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3d181596d0 0x7f3d18159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.928+0000 7f3d1ddff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d18067f70 con 0x7f3d180c6f80 2023-10-06T05:23:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.928+0000 7f3d1ddff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1806e040 con 0x7f3d180c16e0 2023-10-06T05:23:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.928+0000 7f3d1ddff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1806b040 con 0x7f3d181596d0 2023-10-06T05:23:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.928+0000 7f3d16ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3d180c6f80 0x7f3d1806a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52774/0 (socket says 172.21.15.31:52774) 2023-10-06T05:23:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.928+0000 7f3d16ffd700 1 -- 172.21.15.31:0/1811739952 learned_addr learned my addr 172.21.15.31:0/1811739952 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.928+0000 7f3d177fe700 1 --2- 172.21.15.31:0/1811739952 >> v2:172.21.15.62:3300/0 conn(0x7f3d181596d0 0x7f3d18159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.928+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850947295 0 0) 0x7f3d18067f70 con 0x7f3d180c6f80 2023-10-06T05:23:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.928+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d00003570 con 0x7f3d180c6f80 2023-10-06T05:23:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818289959 0 0) 0x7f3d1806e040 con 0x7f3d180c16e0 2023-10-06T05:23:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d18067f70 con 0x7f3d180c16e0 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1352009861 0 0) 0x7f3d00003570 con 0x7f3d180c6f80 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 >> v2:172.21.15.62:3300/0 conn(0x7f3d181596d0 msgr2=0x7f3d18159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 --2- 172.21.15.31:0/1811739952 >> v2:172.21.15.62:3300/0 conn(0x7f3d181596d0 0x7f3d18159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 >> v1:172.21.15.62:6789/0 conn(0x7f3d180c16e0 legacy=0x7f3d1806e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180681c0 con 0x7f3d180c6f80 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f3d0c001470 con 0x7f3d180c6f80 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d177fe700 1 --2- 172.21.15.31:0/1811739952 >> v2:172.21.15.62:3300/0 conn(0x7f3d181596d0 0x7f3d18159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 >> v1:172.21.15.31:6789/0 conn(0x7f3d180c6f80 legacy=0x7f3d1806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d00003c40 0x7f3d00004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d00005210 0x7f3d0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1806b040 con 0x7f3d00004810 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0000c130 con 0x7f3d00003c40 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0000c2c0 con 0x7f3d00005210 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d16ffd700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d177fe700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d00003c40 0x7f3d00004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.929+0000 7f3d17fff700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d00005210 0x7f3d0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.930+0000 7f3d16ffd700 1 -- 172.21.15.31:0/1811739952 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d00005210 msgr2=0x7f3d0000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.930+0000 7f3d16ffd700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d00005210 0x7f3d0000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.930+0000 7f3d16ffd700 1 -- 172.21.15.31:0/1811739952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d00003c40 msgr2=0x7f3d00004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.930+0000 7f3d16ffd700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d00003c40 0x7f3d00004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.930+0000 7f3d16ffd700 1 -- 172.21.15.31:0/1811739952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d00003570 con 0x7f3d00004810 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.930+0000 7f3d17fff700 1 -- 172.21.15.31:0/1811739952 reap_dead start 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.930+0000 7f3d16ffd700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d00004c80 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c210 tx=0x7f3d0c001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f9de3784f15814 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.930+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00cb00 con 0x7f3d00004810 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.930+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c00cca0 con 0x7f3d00004810 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.930+0000 7f3d167fc700 1 -- 172.21.15.31:0/1811739952 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c005b70 con 0x7f3d00004810 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.931+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1811739952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 msgr2=0x7f3d00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.931+0000 7f3d1ddff700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d00004c80 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c210 tx=0x7f3d0c001d40 comp rx=0 tx=0).stop 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.931+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1811739952 shutdown_connections 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.931+0000 7f3d1ddff700 1 --2- 172.21.15.31:0/1811739952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d00004c80 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.931+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1811739952 >> 172.21.15.31:0/1811739952 conn(0x7f3d180c9e80 msgr2=0x7f3d1806a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.931+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1811739952 shutdown_connections 2023-10-06T05:23:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.931+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1811739952 wait complete. 2023-10-06T05:23:21.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.932+0000 7f3d1ddff700 1 Processor -- start 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.932+0000 7f3d1ddff700 1 -- start start 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.933+0000 7f3d1ddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d18158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.933+0000 7f3d1ddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d1815ca50 0x7f3d180b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.933+0000 7f3d1ddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d1815cec0 0x7f3d180b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.933+0000 7f3d1ddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180c49e0 con 0x7f3d00004810 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.933+0000 7f3d1ddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180c3dd0 con 0x7f3d1815ca50 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.933+0000 7f3d1ddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180c40b0 con 0x7f3d1815cec0 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d18158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d18158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39986/0 (socket says 172.21.15.31:39986) 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d177fe700 1 -- 172.21.15.31:0/1783472726 learned_addr learned my addr 172.21.15.31:0/1783472726 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d16ffd700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d1815ca50 0x7f3d180b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d17fff700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d1815cec0 0x7f3d180b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d177fe700 1 -- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d1815cec0 msgr2=0x7f3d180b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d177fe700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d1815cec0 0x7f3d180b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d177fe700 1 -- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d1815ca50 msgr2=0x7f3d180b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d177fe700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d1815ca50 0x7f3d180b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d177fe700 1 -- 172.21.15.31:0/1783472726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180c4390 con 0x7f3d00004810 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.934+0000 7f3d177fe700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d18158430 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3d08011d70 tx=0x7f3d080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.935+0000 7f3cff7fe700 1 -- 172.21.15.31:0/1783472726 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08011e70 con 0x7f3d00004810 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.935+0000 7f3cff7fe700 1 -- 172.21.15.31:0/1783472726 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0800fc20 con 0x7f3d00004810 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.935+0000 7f3cff7fe700 1 -- 172.21.15.31:0/1783472726 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08014440 con 0x7f3d00004810 2023-10-06T05:23:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.935+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1783472726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d180c4670 con 0x7f3d00004810 2023-10-06T05:23:21.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.935+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1783472726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d18158970 con 0x7f3d00004810 2023-10-06T05:23:21.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.936+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1783472726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1806e2f0 con 0x7f3d00004810 2023-10-06T05:23:21.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.937+0000 7f3cff7fe700 1 -- 172.21.15.31:0/1783472726 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3d08014870 con 0x7f3d00004810 2023-10-06T05:23:21.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.938+0000 7f3cff7fe700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3d04067320 0x7f3d040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:21.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.938+0000 7f3cff7fe700 1 -- 172.21.15.31:0/1783472726 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f3d08018030 con 0x7f3d00004810 2023-10-06T05:23:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.939+0000 7f3d16ffd700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3d04067320 0x7f3d040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.939+0000 7f3d16ffd700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3d04067320 0x7f3d040697d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c210 tx=0x7f3d0c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:21.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:21.943+0000 7f3cff7fe700 1 -- 172.21.15.31:0/1783472726 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d08052790 con 0x7f3d00004810 2023-10-06T05:23:22.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.227+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1783472726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) v1 -- 0x7f3d1804c380 con 0x7f3d00004810 2023-10-06T05:23:22.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.713+0000 7f3cff7fe700 1 -- 172.21.15.31:0/1783472726 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v342) v1 ==== 151+0+0 (secure 0 0 0) 0x7f3d080220d0 con 0x7f3d00004810 2023-10-06T05:23:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.716+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3d04067320 msgr2=0x7f3d040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.716+0000 7f3d1ddff700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3d04067320 0x7f3d040697d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c210 tx=0x7f3d0c015000 comp rx=0 tx=0).stop 2023-10-06T05:23:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.716+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1783472726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 msgr2=0x7f3d18158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.716+0000 7f3d1ddff700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d18158430 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3d08011d70 tx=0x7f3d080091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.716+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1783472726 shutdown_connections 2023-10-06T05:23:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.716+0000 7f3d1ddff700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3d04067320 0x7f3d040697d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.716+0000 7f3d1ddff700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d1815cec0 0x7f3d180b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.716+0000 7f3d1ddff700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d00004810 0x7f3d18158430 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.716+0000 7f3d1ddff700 1 --2- 172.21.15.31:0/1783472726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d1815ca50 0x7f3d180b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.716+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1783472726 >> 172.21.15.31:0/1783472726 conn(0x7f3d180c9e80 msgr2=0x7f3d180c1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.717+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1783472726 shutdown_connections 2023-10-06T05:23:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.717+0000 7f3d1ddff700 1 -- 172.21.15.31:0/1783472726 wait complete. 2023-10-06T05:23:22.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:23:22.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.937+0000 7fdd6ebff700 1 Processor -- start 2023-10-06T05:23:22.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.937+0000 7fdd6ebff700 1 -- start start 2023-10-06T05:23:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.938+0000 7fdd6ebff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd68159930 0x7fdd680b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.938+0000 7fdd6ebff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd680cd580 0x7fdd680b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.938+0000 7fdd6ebff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6815d2d0 con 0x7fdd68159560 2023-10-06T05:23:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.938+0000 7fdd6ebff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6815d5b0 con 0x7fdd68159930 2023-10-06T05:23:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.938+0000 7fdd6ebff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6815d890 con 0x7fdd680cd580 2023-10-06T05:23:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.938+0000 7fdd6c99b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdd68159560 0x7fdd680cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52780/0 (socket says 172.21.15.31:52780) 2023-10-06T05:23:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.938+0000 7fdd6c99b700 1 -- 172.21.15.31:0/4247634433 learned_addr learned my addr 172.21.15.31:0/4247634433 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:22.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd6d19c700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd680cd580 0x7fdd680b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:22.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000584260 0 0) 0x7fdd6815d2d0 con 0x7fdd68159560 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd67fff700 1 --2- 172.21.15.31:0/4247634433 >> v2:172.21.15.62:3300/0 conn(0x7fdd68159930 0x7fdd680b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd58003570 con 0x7fdd68159560 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3210606849 0 0) 0x7fdd58003570 con 0x7fdd68159560 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd680cd580 msgr2=0x7fdd680b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd680cd580 0x7fdd680b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 >> v2:172.21.15.62:3300/0 conn(0x7fdd68159930 msgr2=0x7fdd680b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 --2- 172.21.15.31:0/4247634433 >> v2:172.21.15.62:3300/0 conn(0x7fdd68159930 0x7fdd680b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6815db70 con 0x7fdd68159560 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fdd5c001470 con 0x7fdd68159560 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 >> v1:172.21.15.31:6789/0 conn(0x7fdd68159560 legacy=0x7fdd680cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd680cd580 0x7fdd58003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58003fe0 0x7fdd58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58004d50 0x7fdd580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6815d5b0 con 0x7fdd58003fe0 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6815d890 con 0x7fdd58004d50 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.939+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5800ba90 con 0x7fdd680cd580 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.940+0000 7fdd67fff700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58003fe0 0x7fdd58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.940+0000 7fdd6c99b700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd680cd580 0x7fdd58003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.940+0000 7fdd6d19c700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58004d50 0x7fdd580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.940+0000 7fdd67fff700 1 -- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd680cd580 msgr2=0x7fdd58003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.940+0000 7fdd67fff700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd680cd580 0x7fdd58003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.940+0000 7fdd67fff700 1 -- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58004d50 msgr2=0x7fdd580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:22.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.940+0000 7fdd67fff700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd58004d50 0x7fdd580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.940+0000 7fdd67fff700 1 -- 172.21.15.31:0/4247634433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58003570 con 0x7fdd58003fe0 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.940+0000 7fdd67fff700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58003fe0 0x7fdd58004810 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009c00 tx=0x7fdd540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209aea05b2891ad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.941+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400e6b0 con 0x7fdd58003fe0 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.941+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5400e810 con 0x7fdd58003fe0 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.941+0000 7fdd677fe700 1 -- 172.21.15.31:0/4247634433 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400ebd0 con 0x7fdd58003fe0 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.941+0000 7fdd6ebff700 1 -- 172.21.15.31:0/4247634433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58003fe0 msgr2=0x7fdd58004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.941+0000 7fdd6ebff700 1 --2- 172.21.15.31:0/4247634433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd58003fe0 0x7fdd58004810 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009c00 tx=0x7fdd540091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.941+0000 7fdd6d19c700 1 -- 172.21.15.31:0/4247634433 reap_dead start 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.941+0000 7fdd6ebff700 1 -- 172.21.15.31:0/4247634433 shutdown_connections 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.941+0000 7fdd6ebff700 1 -- 172.21.15.31:0/4247634433 >> 172.21.15.31:0/4247634433 conn(0x7fdd6806ec90 msgr2=0x7fdd6806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.941+0000 7fdd6ebff700 1 -- 172.21.15.31:0/4247634433 shutdown_connections 2023-10-06T05:23:22.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.941+0000 7fdd6ebff700 1 -- 172.21.15.31:0/4247634433 wait complete. 2023-10-06T05:23:22.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.943+0000 7fdd6ebff700 1 Processor -- start 2023-10-06T05:23:22.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.943+0000 7fdd6ebff700 1 -- start start 2023-10-06T05:23:22.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd6ebff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd58003fe0 0x7fdd6815b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:22.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd6ebff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd68169bf0 0x7fdd68156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:22.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd6ebff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd680c95f0 0x7fdd680c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:22.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd6ebff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd680c9fa0 con 0x7fdd68169bf0 2023-10-06T05:23:22.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd6ebff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6815b960 con 0x7fdd680c95f0 2023-10-06T05:23:22.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd6ebff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6815acb0 con 0x7fdd58003fe0 2023-10-06T05:23:22.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd67fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd68169bf0 0x7fdd68156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd67fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd68169bf0 0x7fdd68156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40002/0 (socket says 172.21.15.31:40002) 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd67fff700 1 -- 172.21.15.31:0/2772961872 learned_addr learned my addr 172.21.15.31:0/2772961872 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd6c99b700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd58003fe0 0x7fdd6815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.944+0000 7fdd6d19c700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd680c95f0 0x7fdd680c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd67fff700 1 -- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd58003fe0 msgr2=0x7fdd6815b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd67fff700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd58003fe0 0x7fdd6815b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd67fff700 1 -- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd680c95f0 msgr2=0x7fdd680c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd67fff700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd680c95f0 0x7fdd680c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd67fff700 1 -- 172.21.15.31:0/2772961872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6815a0a0 con 0x7fdd68169bf0 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd6d19c700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd680c95f0 0x7fdd680c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd67fff700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd68169bf0 0x7fdd68156f40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fdd58004730 tx=0x7fdd5401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2772961872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54022070 con 0x7fdd68169bf0 2023-10-06T05:23:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2772961872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5400e980 con 0x7fdd68169bf0 2023-10-06T05:23:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd6ebff700 1 -- 172.21.15.31:0/2772961872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6815a380 con 0x7fdd68169bf0 2023-10-06T05:23:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2772961872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54027470 con 0x7fdd68169bf0 2023-10-06T05:23:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.945+0000 7fdd6ebff700 1 -- 172.21.15.31:0/2772961872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6815a940 con 0x7fdd68169bf0 2023-10-06T05:23:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.946+0000 7fdd6ebff700 1 -- 172.21.15.31:0/2772961872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6815bb80 con 0x7fdd68169bf0 2023-10-06T05:23:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.947+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2772961872 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdd5401c450 con 0x7fdd68169bf0 2023-10-06T05:23:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.948+0000 7fdd64ff9700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd4c067320 0x7fdd4c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.948+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2772961872 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fdd54086710 con 0x7fdd68169bf0 2023-10-06T05:23:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.949+0000 7fdd6c99b700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd4c067320 0x7fdd4c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.949+0000 7fdd6c99b700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd4c067320 0x7fdd4c0697d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c000e50 tx=0x7fdd5c001e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:22.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:22.952+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2772961872 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd6815bb80 con 0x7fdd68169bf0 2023-10-06T05:23:23.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.157+0000 7fdd6ebff700 1 -- 172.21.15.31:0/2772961872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdd6815bb80 con 0x7fdd68169bf0 2023-10-06T05:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.163+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2772961872 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdd6815bb80 con 0x7fdd68169bf0 2023-10-06T05:23:23.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.165+0000 7fdd6ebff700 1 -- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd4c067320 msgr2=0x7fdd4c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.165+0000 7fdd6ebff700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd4c067320 0x7fdd4c0697d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c000e50 tx=0x7fdd5c001e90 comp rx=0 tx=0).stop 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.165+0000 7fdd6ebff700 1 -- 172.21.15.31:0/2772961872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd68169bf0 msgr2=0x7fdd68156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.165+0000 7fdd6ebff700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd68169bf0 0x7fdd68156f40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fdd58004730 tx=0x7fdd5401a000 comp rx=0 tx=0).stop 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.166+0000 7fdd6ebff700 1 -- 172.21.15.31:0/2772961872 shutdown_connections 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.166+0000 7fdd6ebff700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd4c067320 0x7fdd4c0697d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.166+0000 7fdd6ebff700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd680c95f0 0x7fdd680c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.166+0000 7fdd6ebff700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd58003fe0 0x7fdd6815b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.166+0000 7fdd6ebff700 1 --2- 172.21.15.31:0/2772961872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd68169bf0 0x7fdd68156f40 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.166+0000 7fdd6ebff700 1 -- 172.21.15.31:0/2772961872 >> 172.21.15.31:0/2772961872 conn(0x7fdd6806ec90 msgr2=0x7fdd680cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.166+0000 7fdd6ebff700 1 -- 172.21.15.31:0/2772961872 shutdown_connections 2023-10-06T05:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.166+0000 7fdd6ebff700 1 -- 172.21.15.31:0/2772961872 wait complete. 2023-10-06T05:23:23.185 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:23:23.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:23.283 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:23.389 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:23:23.389 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:23:23.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:23.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.584+0000 7f172f9ff700 1 Processor -- start 2023-10-06T05:23:23.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.584+0000 7f172f9ff700 1 -- start start 2023-10-06T05:23:23.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f172f9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a5b0 0x7f172806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:23.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f172f9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172806ddc0 con 0x7f172806d980 2023-10-06T05:23:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f172f9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17280c71b0 con 0x7f172813a1e0 2023-10-06T05:23:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f172f9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17280c7490 con 0x7f172813a5b0 2023-10-06T05:23:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f172df9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f172806d980 0x7f1728069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52786/0 (socket says 172.21.15.31:52786) 2023-10-06T05:23:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f172df9c700 1 -- 172.21.15.31:0/1822723708 learned_addr learned my addr 172.21.15.31:0/1822723708 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f172cf9a700 1 --2- 172.21.15.31:0/1822723708 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a5b0 0x7f172806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f171ffff700 1 -- 172.21.15.31:0/1822723708 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884705069 0 0) 0x7f172806ddc0 con 0x7f172806d980 2023-10-06T05:23:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f171ffff700 1 -- 172.21.15.31:0/1822723708 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1714003570 con 0x7f172806d980 2023-10-06T05:23:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f171ffff700 1 -- 172.21.15.31:0/1822723708 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793077547 0 0) 0x7f17280c71b0 con 0x7f172813a1e0 2023-10-06T05:23:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.586+0000 7f171ffff700 1 -- 172.21.15.31:0/1822723708 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f172806ddc0 con 0x7f172813a1e0 2023-10-06T05:23:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.587+0000 7f172cf9a700 1 -- 172.21.15.31:0/1822723708 >> v1:172.21.15.62:6789/0 conn(0x7f172813a1e0 legacy=0x7f17280c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.587+0000 7f172cf9a700 1 -- 172.21.15.31:0/1822723708 >> v1:172.21.15.31:6789/0 conn(0x7f172806d980 legacy=0x7f1728069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.587+0000 7f172cf9a700 1 -- 172.21.15.31:0/1822723708 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280c7770 con 0x7f172813a5b0 2023-10-06T05:23:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.587+0000 7f172cf9a700 1 --2- 172.21.15.31:0/1822723708 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a5b0 0x7f172806d440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1718009910 tx=0x7f17180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c28e318302c495 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.587+0000 7f171ffff700 1 -- 172.21.15.31:0/1822723708 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718011630 con 0x7f172813a5b0 2023-10-06T05:23:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.587+0000 7f171ffff700 1 -- 172.21.15.31:0/1822723708 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1718011790 con 0x7f172813a5b0 2023-10-06T05:23:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.587+0000 7f171ffff700 1 -- 172.21.15.31:0/1822723708 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718011b10 con 0x7f172813a5b0 2023-10-06T05:23:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.588+0000 7f172f9ff700 1 -- 172.21.15.31:0/1822723708 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a5b0 msgr2=0x7f172806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.588+0000 7f172f9ff700 1 --2- 172.21.15.31:0/1822723708 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a5b0 0x7f172806d440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1718009910 tx=0x7f17180091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.588+0000 7f172f9ff700 1 -- 172.21.15.31:0/1822723708 shutdown_connections 2023-10-06T05:23:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.588+0000 7f172f9ff700 1 --2- 172.21.15.31:0/1822723708 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a5b0 0x7f172806d440 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.588+0000 7f172f9ff700 1 -- 172.21.15.31:0/1822723708 >> 172.21.15.31:0/1822723708 conn(0x7f17280cc0b0 msgr2=0x7f17280cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.588+0000 7f172f9ff700 1 -- 172.21.15.31:0/1822723708 shutdown_connections 2023-10-06T05:23:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.588+0000 7f172f9ff700 1 -- 172.21.15.31:0/1822723708 wait complete. 2023-10-06T05:23:23.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.589+0000 7f172f9ff700 1 Processor -- start 2023-10-06T05:23:23.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.589+0000 7f172f9ff700 1 -- start start 2023-10-06T05:23:23.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.590+0000 7f172f9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f172806d980 0x7f1728152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:23.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.590+0000 7f172f9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a1e0 0x7f17280cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.590+0000 7f172f9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172813a5b0 0x7f17280cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.590+0000 7f172f9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280cbc90 con 0x7f172813a5b0 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.590+0000 7f172f9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280b05d0 con 0x7f172806d980 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.590+0000 7f172f9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172814e420 con 0x7f172813a1e0 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172df9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172813a5b0 0x7f17280cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172df9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172813a5b0 0x7f17280cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40008/0 (socket says 172.21.15.31:40008) 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172df9c700 1 -- 172.21.15.31:0/214345595 learned_addr learned my addr 172.21.15.31:0/214345595 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172d79b700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f172806d980 0x7f1728152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172cf9a700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a1e0 0x7f17280cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172df9c700 1 -- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a1e0 msgr2=0x7f17280cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172df9c700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a1e0 0x7f17280cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172df9c700 1 -- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f172806d980 msgr2=0x7f1728152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172df9c700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f172806d980 0x7f1728152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172df9c700 1 -- 172.21.15.31:0/214345595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718009420 con 0x7f172813a5b0 2023-10-06T05:23:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.591+0000 7f172df9c700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172813a5b0 0x7f17280cb380 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f172000cc60 tx=0x7f1720001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.592+0000 7f171d7fa700 1 -- 172.21.15.31:0/214345595 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000a8b0 con 0x7f172813a5b0 2023-10-06T05:23:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.592+0000 7f171d7fa700 1 -- 172.21.15.31:0/214345595 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172000aa10 con 0x7f172813a5b0 2023-10-06T05:23:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.592+0000 7f171d7fa700 1 -- 172.21.15.31:0/214345595 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000adb0 con 0x7f172813a5b0 2023-10-06T05:23:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.592+0000 7f172f9ff700 1 -- 172.21.15.31:0/214345595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17280affc0 con 0x7f172813a5b0 2023-10-06T05:23:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.592+0000 7f172f9ff700 1 -- 172.21.15.31:0/214345595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17280b0260 con 0x7f172813a5b0 2023-10-06T05:23:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.593+0000 7f172f9ff700 1 -- 172.21.15.31:0/214345595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17280afe20 con 0x7f172813a5b0 2023-10-06T05:23:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.594+0000 7f171d7fa700 1 -- 172.21.15.31:0/214345595 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1720007300 con 0x7f172813a5b0 2023-10-06T05:23:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.594+0000 7f171d7fa700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f170c067320 0x7f170c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.595+0000 7f171d7fa700 1 -- 172.21.15.31:0/214345595 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f1720084e70 con 0x7f172813a5b0 2023-10-06T05:23:23.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.595+0000 7f172d79b700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f170c067320 0x7f170c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:23.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.596+0000 7f172d79b700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f170c067320 0x7f170c0697d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1724001130 tx=0x7f1724007d00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:23.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.599+0000 7f171d7fa700 1 -- 172.21.15.31:0/214345595 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17280afe20 con 0x7f172813a5b0 2023-10-06T05:23:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.845+0000 7f172f9ff700 1 -- 172.21.15.31:0/214345595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17280afe20 con 0x7f172813a5b0 2023-10-06T05:23:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.846+0000 7f171d7fa700 1 -- 172.21.15.31:0/214345595 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f17280afe20 con 0x7f172813a5b0 2023-10-06T05:23:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:23.848 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":70,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6509,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/734559578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":734559578},{"type":"v1","addr":"172.21.15.62:6835","nonce":734559578}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70},{"gid":6886,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1885821393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1885821393},{"type":"v1","addr":"172.21.15.31:6834","nonce":1885821393}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6889,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/258742412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":258742412},{"type":"v1","addr":"172.21.15.31:6835","nonce":258742412}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/4070877337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":4070877337},{"type":"v1","addr":"172.21.15.62:6834","nonce":4070877337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[]} 2023-10-06T05:23:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.848+0000 7f172f9ff700 1 -- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f170c067320 msgr2=0x7f170c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.848+0000 7f172f9ff700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f170c067320 0x7f170c0697d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1724001130 tx=0x7f1724007d00 comp rx=0 tx=0).stop 2023-10-06T05:23:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.848+0000 7f172f9ff700 1 -- 172.21.15.31:0/214345595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172813a5b0 msgr2=0x7f17280cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.848+0000 7f172f9ff700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172813a5b0 0x7f17280cb380 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f172000cc60 tx=0x7f1720001ae0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.848+0000 7f172f9ff700 1 -- 172.21.15.31:0/214345595 shutdown_connections 2023-10-06T05:23:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.848+0000 7f172f9ff700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f170c067320 0x7f170c0697d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.848+0000 7f172f9ff700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f172813a5b0 0x7f17280cb380 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.848+0000 7f172f9ff700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f172806d980 0x7f1728152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.848+0000 7f172f9ff700 1 --2- 172.21.15.31:0/214345595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f172813a1e0 0x7f17280cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.848+0000 7f172f9ff700 1 -- 172.21.15.31:0/214345595 >> 172.21.15.31:0/214345595 conn(0x7f17280cc0b0 msgr2=0x7f17280c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:23.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.849+0000 7f172f9ff700 1 -- 172.21.15.31:0/214345595 shutdown_connections 2023-10-06T05:23:23.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:23.849+0000 7f172f9ff700 1 -- 172.21.15.31:0/214345595 wait complete. 2023-10-06T05:23:23.853 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 70 2023-10-06T05:23:23.869 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:23:23.869 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:23:23.869 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:23:23.869 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:23.870 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:23:23.870 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:23:23.870 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:23:23.870 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:23.870 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:23:23.871 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:23:23.871 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:23:23.873 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:23.873 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:23:23.873 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:23:23.874 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:23:23.874 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:23.883 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:23:23.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:23:23.885 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:23:23.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:23:23.887 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:23:23.887 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:23:23.890 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:23:23.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:23:23.928 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:23:23.929 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:23.931 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:23:23.932 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:23.935 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:23:23.935 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:23.939 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:23:23.939 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:23.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:23:24.076 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:23:24.076 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:23:24.087 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:23:24.088 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:23:24.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.146+0000 7f3c721ff700 1 Processor -- start 2023-10-06T05:23:24.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.146+0000 7f3c721ff700 1 -- start start 2023-10-06T05:23:24.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.148+0000 7f3c721ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3c6c06a570 0x7f3c6c162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:24.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.148+0000 7f3c721ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6c0c71b0 con 0x7f3c6c13a5b0 2023-10-06T05:23:24.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.148+0000 7f3c721ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6c0c7490 con 0x7f3c6c13a1e0 2023-10-06T05:23:24.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.148+0000 7f3c721ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c6c0c7770 con 0x7f3c6c06a570 2023-10-06T05:23:24.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.148+0000 7f3c6affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3c6c13a5b0 0x7f3c6c06d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52808/0 (socket says 172.21.15.31:52808) 2023-10-06T05:23:24.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.148+0000 7f3c6affd700 1 -- 172.21.15.31:0/3789829529 learned_addr learned my addr 172.21.15.31:0/3789829529 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:24.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.148+0000 7f3c6bfff700 1 --2- 172.21.15.31:0/3789829529 >> v2:172.21.15.62:3300/0 conn(0x7f3c6c06a570 0x7f3c6c162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:24.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.148+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974385481 0 0) 0x7f3c6c0c71b0 con 0x7f3c6c13a5b0 2023-10-06T05:23:24.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.148+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c54003570 con 0x7f3c6c13a5b0 2023-10-06T05:23:24.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239466014 0 0) 0x7f3c6c0c7490 con 0x7f3c6c13a1e0 2023-10-06T05:23:24.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c6c0c71b0 con 0x7f3c6c13a1e0 2023-10-06T05:23:24.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1873192494 0 0) 0x7f3c54003570 con 0x7f3c6c13a5b0 2023-10-06T05:23:24.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 >> v2:172.21.15.62:3300/0 conn(0x7f3c6c06a570 msgr2=0x7f3c6c162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:24.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 --2- 172.21.15.31:0/3789829529 >> v2:172.21.15.62:3300/0 conn(0x7f3c6c06a570 0x7f3c6c162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:24.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 >> v1:172.21.15.62:6789/0 conn(0x7f3c6c13a1e0 legacy=0x7f3c6c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:24.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c0c7a50 con 0x7f3c6c13a5b0 2023-10-06T05:23:24.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f3c60001470 con 0x7f3c6c13a5b0 2023-10-06T05:23:24.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 >> v1:172.21.15.31:6789/0 conn(0x7f3c6c13a5b0 legacy=0x7f3c6c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:24.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54003bf0 0x7f3c54003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:24.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c54004810 0x7f3c54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:24.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 0x7f3c5400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:24.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c0c7770 con 0x7f3c54003bf0 2023-10-06T05:23:24.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5400c0b0 con 0x7f3c54004810 2023-10-06T05:23:24.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5400c270 con 0x7f3c540051c0 2023-10-06T05:23:24.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6b7fe700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54003bf0 0x7f3c54003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:24.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6bfff700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 0x7f3c5400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:24.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6affd700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c54004810 0x7f3c54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:24.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6bfff700 1 -- 172.21.15.31:0/3789829529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c54004810 msgr2=0x7f3c54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:24.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6bfff700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c54004810 0x7f3c54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:24.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6bfff700 1 -- 172.21.15.31:0/3789829529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54003bf0 msgr2=0x7f3c54003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:24.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.149+0000 7f3c6bfff700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c54003bf0 0x7f3c54003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:24.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.150+0000 7f3c6bfff700 1 -- 172.21.15.31:0/3789829529 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c54003570 con 0x7f3c540051c0 2023-10-06T05:23:24.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.150+0000 7f3c6bfff700 1 -- 172.21.15.31:0/3789829529 reap_dead start 2023-10-06T05:23:24.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.150+0000 7f3c6bfff700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 0x7f3c5400ba60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c06a960 tx=0x7f3c6400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e3bf98b4e321e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:24.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.150+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6400ea90 con 0x7f3c540051c0 2023-10-06T05:23:24.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.150+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6400f040 con 0x7f3c540051c0 2023-10-06T05:23:24.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.150+0000 7f3c6a7fc700 1 -- 172.21.15.31:0/3789829529 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6400ee40 con 0x7f3c540051c0 2023-10-06T05:23:24.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.150+0000 7f3c721ff700 1 -- 172.21.15.31:0/3789829529 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 msgr2=0x7f3c5400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:24.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.150+0000 7f3c721ff700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 0x7f3c5400ba60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c06a960 tx=0x7f3c6400b040 comp rx=0 tx=0).stop 2023-10-06T05:23:24.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.151+0000 7f3c721ff700 1 -- 172.21.15.31:0/3789829529 shutdown_connections 2023-10-06T05:23:24.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.151+0000 7f3c721ff700 1 --2- 172.21.15.31:0/3789829529 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 0x7f3c5400ba60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:24.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.151+0000 7f3c721ff700 1 -- 172.21.15.31:0/3789829529 >> 172.21.15.31:0/3789829529 conn(0x7f3c6c0cc0b0 msgr2=0x7f3c6c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:24.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.151+0000 7f3c721ff700 1 -- 172.21.15.31:0/3789829529 shutdown_connections 2023-10-06T05:23:24.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.151+0000 7f3c721ff700 1 -- 172.21.15.31:0/3789829529 wait complete. 2023-10-06T05:23:24.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.153+0000 7f3c721ff700 1 Processor -- start 2023-10-06T05:23:24.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.153+0000 7f3c721ff700 1 -- start start 2023-10-06T05:23:24.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.154+0000 7f3c721ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 0x7f3c6c138f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:24.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.154+0000 7f3c721ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c6c0c5860 0x7f3c6c156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.154+0000 7f3c721ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c6c157190 0x7f3c6c0c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.154+0000 7f3c721ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c158620 con 0x7f3c6c157190 2023-10-06T05:23:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.154+0000 7f3c721ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c157630 con 0x7f3c6c0c5860 2023-10-06T05:23:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.154+0000 7f3c721ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c0c9e90 con 0x7f3c540051c0 2023-10-06T05:23:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c6c157190 0x7f3c6c0c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:24.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c6c157190 0x7f3c6c0c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40076/0 (socket says 172.21.15.31:40076) 2023-10-06T05:23:24.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6bfff700 1 -- 172.21.15.31:0/1697938206 learned_addr learned my addr 172.21.15.31:0/1697938206 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:24.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6b7fe700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 0x7f3c6c138f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:24.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6affd700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c6c0c5860 0x7f3c6c156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:24.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6bfff700 1 -- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 msgr2=0x7f3c6c138f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:24.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6bfff700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 0x7f3c6c138f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:24.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6bfff700 1 -- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c6c0c5860 msgr2=0x7f3c6c156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:24.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6bfff700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c6c0c5860 0x7f3c6c156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:24.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6bfff700 1 -- 172.21.15.31:0/1697938206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c15bb80 con 0x7f3c6c157190 2023-10-06T05:23:24.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.155+0000 7f3c6bfff700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c6c157190 0x7f3c6c0c4130 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c06d210 tx=0x7f3c64019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:24.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.156+0000 7f3c537fe700 1 -- 172.21.15.31:0/1697938206 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6401d660 con 0x7f3c6c157190 2023-10-06T05:23:24.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.156+0000 7f3c537fe700 1 -- 172.21.15.31:0/1697938206 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c64022070 con 0x7f3c6c157190 2023-10-06T05:23:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.156+0000 7f3c721ff700 1 -- 172.21.15.31:0/1697938206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c15be60 con 0x7f3c6c157190 2023-10-06T05:23:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.156+0000 7f3c537fe700 1 -- 172.21.15.31:0/1697938206 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6401da30 con 0x7f3c6c157190 2023-10-06T05:23:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.156+0000 7f3c721ff700 1 -- 172.21.15.31:0/1697938206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c1394d0 con 0x7f3c6c157190 2023-10-06T05:23:24.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.157+0000 7f3c721ff700 1 -- 172.21.15.31:0/1697938206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6c0ca040 con 0x7f3c6c157190 2023-10-06T05:23:24.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.158+0000 7f3c537fe700 1 -- 172.21.15.31:0/1697938206 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3c64003020 con 0x7f3c6c157190 2023-10-06T05:23:24.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.159+0000 7f3c537fe700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c58067320 0x7f3c580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:24.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.159+0000 7f3c537fe700 1 -- 172.21.15.31:0/1697938206 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f3c64024070 con 0x7f3c6c157190 2023-10-06T05:23:24.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.160+0000 7f3c6b7fe700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c58067320 0x7f3c580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:24.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.160+0000 7f3c6b7fe700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c58067320 0x7f3c580697d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c001130 tx=0x7f3c5c007d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:24.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.163+0000 7f3c537fe700 1 -- 172.21.15.31:0/1697938206 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c6c0ca040 con 0x7f3c6c157190 2023-10-06T05:23:24.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:24.386+0000 7f3c721ff700 1 -- 172.21.15.31:0/1697938206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3c6c15c360 con 0x7f3c6c157190 2023-10-06T05:23:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.192+0000 7f3c537fe700 1 -- 172.21.15.31:0/1697938206 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v110) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3c64018080 con 0x7f3c6c157190 2023-10-06T05:23:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.195+0000 7f3c721ff700 1 -- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c58067320 msgr2=0x7f3c580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.195+0000 7f3c721ff700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c58067320 0x7f3c580697d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c001130 tx=0x7f3c5c007d80 comp rx=0 tx=0).stop 2023-10-06T05:23:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.195+0000 7f3c721ff700 1 -- 172.21.15.31:0/1697938206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c6c157190 msgr2=0x7f3c6c0c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.195+0000 7f3c721ff700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c6c157190 0x7f3c6c0c4130 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c06d210 tx=0x7f3c64019000 comp rx=0 tx=0).stop 2023-10-06T05:23:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.196+0000 7f3c721ff700 1 -- 172.21.15.31:0/1697938206 shutdown_connections 2023-10-06T05:23:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.196+0000 7f3c721ff700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c58067320 0x7f3c580697d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.196+0000 7f3c721ff700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c6c157190 0x7f3c6c0c4130 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.196+0000 7f3c721ff700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c540051c0 0x7f3c6c138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.196+0000 7f3c721ff700 1 --2- 172.21.15.31:0/1697938206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c6c0c5860 0x7f3c6c156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.196+0000 7f3c721ff700 1 -- 172.21.15.31:0/1697938206 >> 172.21.15.31:0/1697938206 conn(0x7f3c6c0cc0b0 msgr2=0x7f3c6c0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.196+0000 7f3c721ff700 1 -- 172.21.15.31:0/1697938206 shutdown_connections 2023-10-06T05:23:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.196+0000 7f3c721ff700 1 -- 172.21.15.31:0/1697938206 wait complete. 2023-10-06T05:23:25.199 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:23:25.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:23:25.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.418+0000 7f4295fff700 1 Processor -- start 2023-10-06T05:23:25.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.418+0000 7f4295fff700 1 -- start start 2023-10-06T05:23:25.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.419+0000 7f4295fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42900cd510 0x7f42900b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.419+0000 7f4295fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42900c3f40 con 0x7f429015d2d0 2023-10-06T05:23:25.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.419+0000 7f4295fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42900cb000 con 0x7f4290159500 2023-10-06T05:23:25.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.419+0000 7f4295fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42900b2860 con 0x7f42900cd510 2023-10-06T05:23:25.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.419+0000 7f428effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42900cd510 0x7f42900b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.419+0000 7f428effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42900cd510 0x7f42900b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40080/0 (socket says 172.21.15.31:40080) 2023-10-06T05:23:25.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.419+0000 7f428effd700 1 -- 172.21.15.31:0/2216724 learned_addr learned my addr 172.21.15.31:0/2216724 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:25.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.419+0000 7f428ffff700 1 --1- 172.21.15.31:0/2216724 >> v1:172.21.15.31:6789/0 conn(0x7f429015d2d0 0x7f42900aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52824/0 (socket says 172.21.15.31:52824) 2023-10-06T05:23:25.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.420+0000 7f428effd700 1 -- 172.21.15.31:0/2216724 >> v1:172.21.15.62:6789/0 conn(0x7f4290159500 legacy=0x7f42900cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.420+0000 7f428effd700 1 -- 172.21.15.31:0/2216724 >> v1:172.21.15.31:6789/0 conn(0x7f429015d2d0 legacy=0x7f42900aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.420+0000 7f428effd700 1 -- 172.21.15.31:0/2216724 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429015d7d0 con 0x7f42900cd510 2023-10-06T05:23:25.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.420+0000 7f428effd700 1 --2- 172.21.15.31:0/2216724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42900cd510 0x7f42900b0ab0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f42840099a0 tx=0x7f4284009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d70ad630b860ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:25.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.420+0000 7f428e7fc700 1 -- 172.21.15.31:0/2216724 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428400b860 con 0x7f42900cd510 2023-10-06T05:23:25.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.421+0000 7f428e7fc700 1 -- 172.21.15.31:0/2216724 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428400b9c0 con 0x7f42900cd510 2023-10-06T05:23:25.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.421+0000 7f428e7fc700 1 -- 172.21.15.31:0/2216724 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428400bd60 con 0x7f42900cd510 2023-10-06T05:23:25.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.421+0000 7f4295fff700 1 -- 172.21.15.31:0/2216724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42900cd510 msgr2=0x7f42900b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.421+0000 7f4295fff700 1 --2- 172.21.15.31:0/2216724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42900cd510 0x7f42900b0ab0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f42840099a0 tx=0x7f4284009250 comp rx=0 tx=0).stop 2023-10-06T05:23:25.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.421+0000 7f4295fff700 1 -- 172.21.15.31:0/2216724 shutdown_connections 2023-10-06T05:23:25.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.421+0000 7f4295fff700 1 --2- 172.21.15.31:0/2216724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42900cd510 0x7f42900b0ab0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.421+0000 7f4295fff700 1 -- 172.21.15.31:0/2216724 >> 172.21.15.31:0/2216724 conn(0x7f429006ec90 msgr2=0x7f429006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:25.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.421+0000 7f4295fff700 1 -- 172.21.15.31:0/2216724 shutdown_connections 2023-10-06T05:23:25.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.421+0000 7f4295fff700 1 -- 172.21.15.31:0/2216724 wait complete. 2023-10-06T05:23:25.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.423+0000 7f4295fff700 1 Processor -- start 2023-10-06T05:23:25.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.423+0000 7f4295fff700 1 -- start start 2023-10-06T05:23:25.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f4295fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42900cd510 0x7f429015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f4295fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4290159500 0x7f42900ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f4295fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429015d2d0 0x7f4290157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f4295fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429015ba90 con 0x7f429015d2d0 2023-10-06T05:23:25.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f4295fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4290158220 con 0x7f4290159500 2023-10-06T05:23:25.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f4295fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42900c9780 con 0x7f42900cd510 2023-10-06T05:23:25.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f428ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429015d2d0 0x7f4290157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f428ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429015d2d0 0x7f4290157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40092/0 (socket says 172.21.15.31:40092) 2023-10-06T05:23:25.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f428effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4290159500 0x7f42900ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f428ffff700 1 -- 172.21.15.31:0/1876391658 learned_addr learned my addr 172.21.15.31:0/1876391658 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f428f7fe700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42900cd510 0x7f429015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f428ffff700 1 -- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42900cd510 msgr2=0x7f429015a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f428ffff700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42900cd510 0x7f429015a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f428ffff700 1 -- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4290159500 msgr2=0x7f42900ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f428ffff700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4290159500 0x7f42900ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.424+0000 7f428ffff700 1 -- 172.21.15.31:0/1876391658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4284009490 con 0x7f429015d2d0 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.425+0000 7f428ffff700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429015d2d0 0x7f4290157ce0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f428800cc70 tx=0x7f4288001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.425+0000 7f42777fe700 1 -- 172.21.15.31:0/1876391658 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288005df0 con 0x7f429015d2d0 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.425+0000 7f42777fe700 1 -- 172.21.15.31:0/1876391658 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428800a5b0 con 0x7f429015d2d0 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.425+0000 7f4295fff700 1 -- 172.21.15.31:0/1876391658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429015ae80 con 0x7f429015d2d0 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.425+0000 7f42777fe700 1 -- 172.21.15.31:0/1876391658 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428800a930 con 0x7f429015d2d0 2023-10-06T05:23:25.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.425+0000 7f4295fff700 1 -- 172.21.15.31:0/1876391658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429015b440 con 0x7f429015d2d0 2023-10-06T05:23:25.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.426+0000 7f4295fff700 1 -- 172.21.15.31:0/1876391658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429004c380 con 0x7f429015d2d0 2023-10-06T05:23:25.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.427+0000 7f42777fe700 1 -- 172.21.15.31:0/1876391658 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f428800ace0 con 0x7f429015d2d0 2023-10-06T05:23:25.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.428+0000 7f42777fe700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067320 0x7f427c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.429+0000 7f42777fe700 1 -- 172.21.15.31:0/1876391658 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f4288084a70 con 0x7f429015d2d0 2023-10-06T05:23:25.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.429+0000 7f428f7fe700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067320 0x7f427c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.430+0000 7f428f7fe700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067320 0x7f427c0697d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4280003a80 tx=0x7f4280007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:25.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.433+0000 7f42777fe700 1 -- 172.21.15.31:0/1876391658 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4288051910 con 0x7f429015d2d0 2023-10-06T05:23:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.712+0000 7f4295fff700 1 -- 172.21.15.31:0/1876391658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f429015bbd0 con 0x7f429015d2d0 2023-10-06T05:23:25.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.712+0000 7f42777fe700 1 -- 172.21.15.31:0/1876391658 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4288054f30 con 0x7f429015d2d0 2023-10-06T05:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:25.715 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:23:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:23:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:23:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:23:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:23:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:23:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:23:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:23:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:23:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:23:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:23:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:23:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:23:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:23:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:23:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:23:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:23:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:23:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:23:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:23:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:23:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:23:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:23:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:23:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:23:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:23:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:23:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:23:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:23:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:23:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:23:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:23:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:23:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:23:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:23:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:23:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:23:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:23:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:23:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:23:25.738 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:23:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 -- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067320 msgr2=0x7f427c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067320 0x7f427c0697d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4280003a80 tx=0x7f4280007ec0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 -- 172.21.15.31:0/1876391658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429015d2d0 msgr2=0x7f4290157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429015d2d0 0x7f4290157ce0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f428800cc70 tx=0x7f4288001a70 comp rx=0 tx=0).stop 2023-10-06T05:23:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 -- 172.21.15.31:0/1876391658 shutdown_connections 2023-10-06T05:23:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f427c067320 0x7f427c0697d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429015d2d0 0x7f4290157ce0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42900cd510 0x7f429015a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 --2- 172.21.15.31:0/1876391658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4290159500 0x7f42900ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 -- 172.21.15.31:0/1876391658 >> 172.21.15.31:0/1876391658 conn(0x7f429006ec90 msgr2=0x7f42900c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 -- 172.21.15.31:0/1876391658 shutdown_connections 2023-10-06T05:23:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.715+0000 7f4295fff700 1 -- 172.21.15.31:0/1876391658 wait complete. 2023-10-06T05:23:25.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:25.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.936+0000 7fb2bcbff700 1 Processor -- start 2023-10-06T05:23:25.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.936+0000 7fb2bcbff700 1 -- start start 2023-10-06T05:23:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.937+0000 7fb2bcbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb2b81595a0 0x7fb2b8159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.937+0000 7fb2bcbff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b80c95b0 con 0x7fb2b8068d80 2023-10-06T05:23:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.937+0000 7fb2bcbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b80c9890 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.937+0000 7fb2bcbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b80c9b70 con 0x7fb2b81595a0 2023-10-06T05:23:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.937+0000 7fb2b659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb2b81595a0 0x7fb2b8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.937+0000 7fb2b6d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb2b8068d80 0x7fb2b80c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52840/0 (socket says 172.21.15.31:52840) 2023-10-06T05:23:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.937+0000 7fb2b6d9d700 1 -- 172.21.15.31:0/485253083 learned_addr learned my addr 172.21.15.31:0/485253083 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.937+0000 7fb2b659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb2b81595a0 0x7fb2b8159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40094/0 (socket says 172.21.15.31:40094) 2023-10-06T05:23:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.938+0000 7fb2b559a700 1 -- 172.21.15.31:0/485253083 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170081389 0 0) 0x7fb2b80c95b0 con 0x7fb2b8068d80 2023-10-06T05:23:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.938+0000 7fb2b659c700 1 -- 172.21.15.31:0/485253083 >> v1:172.21.15.62:6789/0 conn(0x7fb2b80cc0d0 legacy=0x7fb2b8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.938+0000 7fb2b659c700 1 -- 172.21.15.31:0/485253083 >> v1:172.21.15.31:6789/0 conn(0x7fb2b8068d80 legacy=0x7fb2b80c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.938+0000 7fb2b659c700 1 -- 172.21.15.31:0/485253083 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b80ca130 con 0x7fb2b81595a0 2023-10-06T05:23:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.938+0000 7fb2b659c700 1 --2- 172.21.15.31:0/485253083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b81595a0 0x7fb2b8159970 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009910 tx=0x7fb2a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d216670e3386a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.938+0000 7fb2b559a700 1 -- 172.21.15.31:0/485253083 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800b820 con 0x7fb2b81595a0 2023-10-06T05:23:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.938+0000 7fb2b559a700 1 -- 172.21.15.31:0/485253083 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a800b980 con 0x7fb2b81595a0 2023-10-06T05:23:25.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.938+0000 7fb2b559a700 1 -- 172.21.15.31:0/485253083 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800bd20 con 0x7fb2b81595a0 2023-10-06T05:23:25.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.939+0000 7fb2bcbff700 1 -- 172.21.15.31:0/485253083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b81595a0 msgr2=0x7fb2b8159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.939+0000 7fb2bcbff700 1 --2- 172.21.15.31:0/485253083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b81595a0 0x7fb2b8159970 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009910 tx=0x7fb2a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.939+0000 7fb2bcbff700 1 -- 172.21.15.31:0/485253083 shutdown_connections 2023-10-06T05:23:25.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.939+0000 7fb2bcbff700 1 --2- 172.21.15.31:0/485253083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b81595a0 0x7fb2b8159970 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.939+0000 7fb2bcbff700 1 -- 172.21.15.31:0/485253083 >> 172.21.15.31:0/485253083 conn(0x7fb2b806b620 msgr2=0x7fb2b806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:25.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.939+0000 7fb2bcbff700 1 -- 172.21.15.31:0/485253083 shutdown_connections 2023-10-06T05:23:25.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.939+0000 7fb2bcbff700 1 -- 172.21.15.31:0/485253083 wait complete. 2023-10-06T05:23:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.941+0000 7fb2bcbff700 1 Processor -- start 2023-10-06T05:23:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.941+0000 7fb2bcbff700 1 -- start start 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2bcbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2b8068d80 0x7fb2b8142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2bcbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b80cc0d0 0x7fb2b8142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2bcbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2b81595a0 0x7fb2b816d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2bcbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b81553d0 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2bcbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80c6fc0 con 0x7fb2b81595a0 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2bcbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80c72a0 con 0x7fb2b8068d80 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2b5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b80cc0d0 0x7fb2b8142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2b5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b80cc0d0 0x7fb2b8142480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40104/0 (socket says 172.21.15.31:40104) 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2b5d9b700 1 -- 172.21.15.31:0/2895498576 learned_addr learned my addr 172.21.15.31:0/2895498576 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2b659c700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2b8068d80 0x7fb2b8142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2b6d9d700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2b81595a0 0x7fb2b816d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2b5d9b700 1 -- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2b8068d80 msgr2=0x7fb2b8142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2b5d9b700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2b8068d80 0x7fb2b8142ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2b5d9b700 1 -- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2b81595a0 msgr2=0x7fb2b816d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2b5d9b700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2b81595a0 0x7fb2b816d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.942+0000 7fb2b5d9b700 1 -- 172.21.15.31:0/2895498576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a8009420 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.943+0000 7fb2b5d9b700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b80cc0d0 0x7fb2b8142480 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00c590 tx=0x7fb2ac00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.943+0000 7fb2a67fc700 1 -- 172.21.15.31:0/2895498576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac011590 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.943+0000 7fb2a67fc700 1 -- 172.21.15.31:0/2895498576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac011710 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.943+0000 7fb2bcbff700 1 -- 172.21.15.31:0/2895498576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b80c7580 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.943+0000 7fb2a67fc700 1 -- 172.21.15.31:0/2895498576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac011ad0 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.943+0000 7fb2bcbff700 1 -- 172.21.15.31:0/2895498576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b80c7b40 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.944+0000 7fb2bcbff700 1 -- 172.21.15.31:0/2895498576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b80c7d60 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.945+0000 7fb2a67fc700 1 -- 172.21.15.31:0/2895498576 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb2ac01b3b0 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.946+0000 7fb2a67fc700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb298067320 0x7fb2980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:25.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.946+0000 7fb2a67fc700 1 -- 172.21.15.31:0/2895498576 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fb2ac083ff0 con 0x7fb2b80cc0d0 2023-10-06T05:23:25.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.947+0000 7fb2b659c700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb298067320 0x7fb2980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:25.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.947+0000 7fb2b659c700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb298067320 0x7fb2980697d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80093f0 tx=0x7fb2a8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:25.950+0000 7fb2a67fc700 1 -- 172.21.15.31:0/2895498576 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2ac050e10 con 0x7fb2b80cc0d0 2023-10-06T05:23:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.154+0000 7fb2bcbff700 1 -- 172.21.15.31:0/2895498576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2b804c380 con 0x7fb2b80cc0d0 2023-10-06T05:23:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.155+0000 7fb2a67fc700 1 -- 172.21.15.31:0/2895498576 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+13744 (secure 0 0 0) 0x7fb2ac017070 con 0x7fb2b80cc0d0 2023-10-06T05:23:26.156 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:26.157 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":110,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:23:25.188225+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 -- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb298067320 msgr2=0x7fb2980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb298067320 0x7fb2980697d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80093f0 tx=0x7fb2a8017000 comp rx=0 tx=0).stop 2023-10-06T05:23:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 -- 172.21.15.31:0/2895498576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b80cc0d0 msgr2=0x7fb2b8142480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b80cc0d0 0x7fb2b8142480 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00c590 tx=0x7fb2ac00c8a0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 -- 172.21.15.31:0/2895498576 shutdown_connections 2023-10-06T05:23:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb298067320 0x7fb2980697d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2b81595a0 0x7fb2b816d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2b8068d80 0x7fb2b8142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 --2- 172.21.15.31:0/2895498576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2b80cc0d0 0x7fb2b8142480 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 -- 172.21.15.31:0/2895498576 >> 172.21.15.31:0/2895498576 conn(0x7fb2b806b620 msgr2=0x7fb2b80cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 -- 172.21.15.31:0/2895498576 shutdown_connections 2023-10-06T05:23:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.157+0000 7fb2bcbff700 1 -- 172.21.15.31:0/2895498576 wait complete. 2023-10-06T05:23:26.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:23:26.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:23:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.374+0000 7fd3e1fff700 1 Processor -- start 2023-10-06T05:23:26.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.374+0000 7fd3e1fff700 1 -- start start 2023-10-06T05:23:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3e1fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd3dc0c7190 0x7fd3dc0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3e1fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7560 0x7fd3dc0686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3e1fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3dc068250 con 0x7fd3dc0aef00 2023-10-06T05:23:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3e1fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3dc06ac80 con 0x7fd3dc0c7190 2023-10-06T05:23:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3e1fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc067f20 con 0x7fd3dc0c7560 2023-10-06T05:23:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3daffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd3dc0aef00 0x7fd3dc0af7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52850/0 (socket says 172.21.15.31:52850) 2023-10-06T05:23:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3daffd700 1 -- 172.21.15.31:0/1503580240 learned_addr learned my addr 172.21.15.31:0/1503580240 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3db7fe700 1 --2- 172.21.15.31:0/1503580240 >> v2:172.21.15.62:3300/0 conn(0x7fd3dc0c7190 0x7fd3dc0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3dbfff700 1 --2- 172.21.15.31:0/1503580240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7560 0x7fd3dc0686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3da7fc700 1 -- 172.21.15.31:0/1503580240 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128345312 0 0) 0x7fd3dc068250 con 0x7fd3dc0aef00 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.376+0000 7fd3da7fc700 1 -- 172.21.15.31:0/1503580240 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3c4003570 con 0x7fd3dc0aef00 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.377+0000 7fd3db7fe700 1 -- 172.21.15.31:0/1503580240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7560 msgr2=0x7fd3dc0686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.377+0000 7fd3db7fe700 1 --2- 172.21.15.31:0/1503580240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7560 0x7fd3dc0686e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.377+0000 7fd3db7fe700 1 -- 172.21.15.31:0/1503580240 >> v1:172.21.15.31:6789/0 conn(0x7fd3dc0aef00 legacy=0x7fd3dc0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.377+0000 7fd3db7fe700 1 -- 172.21.15.31:0/1503580240 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc06aed0 con 0x7fd3dc0c7190 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.377+0000 7fd3db7fe700 1 --2- 172.21.15.31:0/1503580240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc0c7190 0x7fd3dc0c9d40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0098e0 tx=0x7fd3cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dadb3db273f59e14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.377+0000 7fd3da7fc700 1 -- 172.21.15.31:0/1503580240 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc011630 con 0x7fd3dc0c7190 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.377+0000 7fd3da7fc700 1 -- 172.21.15.31:0/1503580240 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc011790 con 0x7fd3dc0c7190 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.377+0000 7fd3da7fc700 1 -- 172.21.15.31:0/1503580240 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc011b30 con 0x7fd3dc0c7190 2023-10-06T05:23:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.378+0000 7fd3e1fff700 1 -- 172.21.15.31:0/1503580240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc0c7190 msgr2=0x7fd3dc0c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.378+0000 7fd3e1fff700 1 --2- 172.21.15.31:0/1503580240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc0c7190 0x7fd3dc0c9d40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0098e0 tx=0x7fd3cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.378+0000 7fd3e1fff700 1 -- 172.21.15.31:0/1503580240 shutdown_connections 2023-10-06T05:23:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.378+0000 7fd3e1fff700 1 --2- 172.21.15.31:0/1503580240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc0c7190 0x7fd3dc0c9d40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.378+0000 7fd3e1fff700 1 --2- 172.21.15.31:0/1503580240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7560 0x7fd3dc0686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.378+0000 7fd3e1fff700 1 -- 172.21.15.31:0/1503580240 >> 172.21.15.31:0/1503580240 conn(0x7fd3dc0cd3d0 msgr2=0x7fd3dc0c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.378+0000 7fd3e1fff700 1 -- 172.21.15.31:0/1503580240 shutdown_connections 2023-10-06T05:23:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.378+0000 7fd3e1fff700 1 -- 172.21.15.31:0/1503580240 wait complete. 2023-10-06T05:23:26.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.380+0000 7fd3e1fff700 1 Processor -- start 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.380+0000 7fd3e1fff700 1 -- start start 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.381+0000 7fd3e1fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc0aef00 0x7fd3dc0c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.381+0000 7fd3e1fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7190 0x7fd3dc0c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3e1fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3dc0c7560 0x7fd3dc0c0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3e1fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc15bae0 con 0x7fd3dc0c7560 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3e1fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc15aed0 con 0x7fd3dc0aef00 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3e1fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc15b1b0 con 0x7fd3dc0c7190 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3dc0c7560 0x7fd3dc0c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3dc0c7560 0x7fd3dc0c0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40108/0 (socket says 172.21.15.31:40108) 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3dbfff700 1 -- 172.21.15.31:0/2486041868 learned_addr learned my addr 172.21.15.31:0/2486041868 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3db7fe700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc0aef00 0x7fd3dc0c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3daffd700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7190 0x7fd3dc0c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3dbfff700 1 -- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7190 msgr2=0x7fd3dc0c03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3dbfff700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7190 0x7fd3dc0c03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3dbfff700 1 -- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc0aef00 msgr2=0x7fd3dc0c2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3dbfff700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc0aef00 0x7fd3dc0c2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3dbfff700 1 -- 172.21.15.31:0/2486041868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc009420 con 0x7fd3dc0c7560 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3daffd700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7190 0x7fd3dc0c03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.382+0000 7fd3db7fe700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc0aef00 0x7fd3dc0c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.383+0000 7fd3dbfff700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3dc0c7560 0x7fd3dc0c0920 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009e90 tx=0x7fd3d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.383+0000 7fd3c37fe700 1 -- 172.21.15.31:0/2486041868 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400cc20 con 0x7fd3dc0c7560 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.383+0000 7fd3c37fe700 1 -- 172.21.15.31:0/2486041868 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d400f040 con 0x7fd3dc0c7560 2023-10-06T05:23:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.383+0000 7fd3e1fff700 1 -- 172.21.15.31:0/2486041868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3dc15b490 con 0x7fd3dc0c7560 2023-10-06T05:23:26.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.383+0000 7fd3c37fe700 1 -- 172.21.15.31:0/2486041868 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4014480 con 0x7fd3dc0c7560 2023-10-06T05:23:26.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.383+0000 7fd3e1fff700 1 -- 172.21.15.31:0/2486041868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3dc0c2070 con 0x7fd3dc0c7560 2023-10-06T05:23:26.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.384+0000 7fd3e1fff700 1 -- 172.21.15.31:0/2486041868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3dc04c380 con 0x7fd3dc0c7560 2023-10-06T05:23:26.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.385+0000 7fd3c37fe700 1 -- 172.21.15.31:0/2486041868 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd3d4014830 con 0x7fd3dc0c7560 2023-10-06T05:23:26.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.386+0000 7fd3c37fe700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3c8067320 0x7fd3c80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.386+0000 7fd3c37fe700 1 -- 172.21.15.31:0/2486041868 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fd3d4085680 con 0x7fd3dc0c7560 2023-10-06T05:23:26.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.386+0000 7fd3db7fe700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3c8067320 0x7fd3c80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.387+0000 7fd3db7fe700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3c8067320 0x7fd3c80697d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0091e0 tx=0x7fd3cc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.390+0000 7fd3c37fe700 1 -- 172.21.15.31:0/2486041868 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3d4055e20 con 0x7fd3dc0c7560 2023-10-06T05:23:26.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.591+0000 7fd3e1fff700 1 -- 172.21.15.31:0/2486041868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd3dc15bc20 con 0x7fd3dc0c7560 2023-10-06T05:23:26.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.592+0000 7fd3c37fe700 1 -- 172.21.15.31:0/2486041868 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v110) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd3d40559b0 con 0x7fd3dc0c7560 2023-10-06T05:23:26.594 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:23:26.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 -- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3c8067320 msgr2=0x7fd3c80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3c8067320 0x7fd3c80697d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0091e0 tx=0x7fd3cc017000 comp rx=0 tx=0).stop 2023-10-06T05:23:26.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 -- 172.21.15.31:0/2486041868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3dc0c7560 msgr2=0x7fd3dc0c0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3dc0c7560 0x7fd3dc0c0920 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009e90 tx=0x7fd3d400b040 comp rx=0 tx=0).stop 2023-10-06T05:23:26.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 -- 172.21.15.31:0/2486041868 shutdown_connections 2023-10-06T05:23:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd3c8067320 0x7fd3c80697d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3dc0c7560 0x7fd3dc0c0920 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3dc0aef00 0x7fd3dc0c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 --2- 172.21.15.31:0/2486041868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3dc0c7190 0x7fd3dc0c03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 -- 172.21.15.31:0/2486041868 >> 172.21.15.31:0/2486041868 conn(0x7fd3dc0cd3d0 msgr2=0x7fd3dc159310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 -- 172.21.15.31:0/2486041868 shutdown_connections 2023-10-06T05:23:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.594+0000 7fd3e1fff700 1 -- 172.21.15.31:0/2486041868 wait complete. 2023-10-06T05:23:26.614 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:23:26.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.811+0000 7f0933bff700 1 Processor -- start 2023-10-06T05:23:26.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.811+0000 7f0933bff700 1 -- start start 2023-10-06T05:23:26.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.812+0000 7f0933bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f092c0b0b90 0x7f092c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.812+0000 7f0933bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c0af6c0 0x7f092c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.812+0000 7f0933bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c0ad690 con 0x7f092c153d40 2023-10-06T05:23:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.812+0000 7f0933bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c0acd60 con 0x7f092c0b0b90 2023-10-06T05:23:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.812+0000 7f0933bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0ad040 con 0x7f092c0af6c0 2023-10-06T05:23:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.812+0000 7f093199b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f092c153d40 0x7f092c0b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52864/0 (socket says 172.21.15.31:52864) 2023-10-06T05:23:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f093199b700 1 -- 172.21.15.31:0/2362839959 learned_addr learned my addr 172.21.15.31:0/2362839959 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f093119a700 1 --2- 172.21.15.31:0/2362839959 >> v2:172.21.15.62:3300/0 conn(0x7f092c0b0b90 0x7f092c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f093219c700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c0af6c0 0x7f092c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286314819 0 0) 0x7f092c0ad690 con 0x7f092c153d40 2023-10-06T05:23:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0918003570 con 0x7f092c153d40 2023-10-06T05:23:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1413685882 0 0) 0x7f0918003570 con 0x7f092c153d40 2023-10-06T05:23:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c0af6c0 msgr2=0x7f092c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c0af6c0 0x7f092c0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 >> v2:172.21.15.62:3300/0 conn(0x7f092c0b0b90 msgr2=0x7f092c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 --2- 172.21.15.31:0/2362839959 >> v2:172.21.15.62:3300/0 conn(0x7f092c0b0b90 0x7f092c0af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c0ad320 con 0x7f092c153d40 2023-10-06T05:23:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f0928001470 con 0x7f092c153d40 2023-10-06T05:23:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 >> v1:172.21.15.31:6789/0 conn(0x7f092c153d40 legacy=0x7f092c0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.813+0000 7f0930999700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0918003ae0 0x7f0918003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f0930999700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 0x7f0918004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f0930999700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c0af6c0 0x7f09180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0acd60 con 0x7f0918004810 2023-10-06T05:23:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0ad040 con 0x7f0918003ae0 2023-10-06T05:23:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091800dd20 con 0x7f092c0af6c0 2023-10-06T05:23:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f093119a700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 0x7f0918004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f093199b700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0918003ae0 0x7f0918003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f093219c700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c0af6c0 0x7f09180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f093119a700 1 -- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c0af6c0 msgr2=0x7f09180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:26.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f093119a700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c0af6c0 0x7f09180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f093119a700 1 -- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0918003ae0 msgr2=0x7f0918003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:26.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f093119a700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0918003ae0 0x7f0918003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f093119a700 1 -- 172.21.15.31:0/2362839959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0918003570 con 0x7f0918004810 2023-10-06T05:23:26.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.814+0000 7f093199b700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0918003ae0 0x7f0918003ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:26.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f093119a700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 0x7f0918004c80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f092c0b0fb0 tx=0x7f091c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12721608b6414e2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c00e710 con 0x7f0918004810 2023-10-06T05:23:26.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c00e870 con 0x7f0918004810 2023-10-06T05:23:26.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f0930999700 1 -- 172.21.15.31:0/2362839959 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c00ec30 con 0x7f0918004810 2023-10-06T05:23:26.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f0933bff700 1 -- 172.21.15.31:0/2362839959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 msgr2=0x7f0918004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f0933bff700 1 --2- 172.21.15.31:0/2362839959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 0x7f0918004c80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f092c0b0fb0 tx=0x7f091c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f093219c700 1 -- 172.21.15.31:0/2362839959 reap_dead start 2023-10-06T05:23:26.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f0933bff700 1 -- 172.21.15.31:0/2362839959 shutdown_connections 2023-10-06T05:23:26.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f0933bff700 1 -- 172.21.15.31:0/2362839959 >> 172.21.15.31:0/2362839959 conn(0x7f092c067560 msgr2=0x7f092c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:26.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f0933bff700 1 -- 172.21.15.31:0/2362839959 shutdown_connections 2023-10-06T05:23:26.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.815+0000 7f0933bff700 1 -- 172.21.15.31:0/2362839959 wait complete. 2023-10-06T05:23:26.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.817+0000 7f0933bff700 1 Processor -- start 2023-10-06T05:23:26.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.817+0000 7f0933bff700 1 -- start start 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f0933bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 0x7f092c1516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f0933bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092c149d40 0x7f092c14f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f0933bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c15b230 0x7f092c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f0933bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c15bbe0 con 0x7f0918004810 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f0933bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c14cab0 con 0x7f092c149d40 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f0933bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c14f870 con 0x7f092c15b230 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f093199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 0x7f092c1516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f093199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 0x7f092c1516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40124/0 (socket says 172.21.15.31:40124) 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f093199b700 1 -- 172.21.15.31:0/2060383600 learned_addr learned my addr 172.21.15.31:0/2060383600 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f093119a700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092c149d40 0x7f092c14f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.818+0000 7f093219c700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c15b230 0x7f092c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f093199b700 1 -- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c15b230 msgr2=0x7f092c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f093199b700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c15b230 0x7f092c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f093199b700 1 -- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092c149d40 msgr2=0x7f092c14f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f093199b700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092c149d40 0x7f092c14f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f093199b700 1 -- 172.21.15.31:0/2060383600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c14fb50 con 0x7f0918004810 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f093199b700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 0x7f092c1516c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f0928007440 tx=0x7f0928001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f0921ffb700 1 -- 172.21.15.31:0/2060383600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800f850 con 0x7f0918004810 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f0921ffb700 1 -- 172.21.15.31:0/2060383600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092800f9b0 con 0x7f0918004810 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f0933bff700 1 -- 172.21.15.31:0/2060383600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c14fe30 con 0x7f0918004810 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f0921ffb700 1 -- 172.21.15.31:0/2060383600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800fd30 con 0x7f0918004810 2023-10-06T05:23:26.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.819+0000 7f0933bff700 1 -- 172.21.15.31:0/2060383600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c14f560 con 0x7f0918004810 2023-10-06T05:23:26.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.820+0000 7f0933bff700 1 -- 172.21.15.31:0/2060383600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c04c380 con 0x7f0918004810 2023-10-06T05:23:26.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.821+0000 7f0921ffb700 1 -- 172.21.15.31:0/2060383600 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0928013600 con 0x7f0918004810 2023-10-06T05:23:26.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.823+0000 7f0921ffb700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0910067320 0x7f09100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:26.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.823+0000 7f0921ffb700 1 -- 172.21.15.31:0/2060383600 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f0928084c40 con 0x7f0918004810 2023-10-06T05:23:26.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.823+0000 7f093119a700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0910067320 0x7f09100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:26.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.824+0000 7f093119a700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0910067320 0x7f09100697d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f092c0ab610 tx=0x7f091c01a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:26.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:26.827+0000 7f0921ffb700 1 -- 172.21.15.31:0/2060383600 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0928051b70 con 0x7f0918004810 2023-10-06T05:23:27.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.029+0000 7f0933bff700 1 -- 172.21.15.31:0/2060383600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f092c0afed0 con 0x7f0918004810 2023-10-06T05:23:27.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.030+0000 7f0921ffb700 1 -- 172.21.15.31:0/2060383600 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0928055190 con 0x7f0918004810 2023-10-06T05:23:27.032 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:23:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 -- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0910067320 msgr2=0x7f09100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0910067320 0x7f09100697d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f092c0ab610 tx=0x7f091c01a000 comp rx=0 tx=0).stop 2023-10-06T05:23:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 -- 172.21.15.31:0/2060383600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 msgr2=0x7f092c1516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 0x7f092c1516c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f0928007440 tx=0x7f0928001dc0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 -- 172.21.15.31:0/2060383600 shutdown_connections 2023-10-06T05:23:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0910067320 0x7f09100697d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f092c15b230 0x7f092c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0918004810 0x7f092c1516c0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 --2- 172.21.15.31:0/2060383600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092c149d40 0x7f092c14f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 -- 172.21.15.31:0/2060383600 >> 172.21.15.31:0/2060383600 conn(0x7f092c067560 msgr2=0x7f092c06de20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.032+0000 7f0933bff700 1 -- 172.21.15.31:0/2060383600 shutdown_connections 2023-10-06T05:23:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.033+0000 7f0933bff700 1 -- 172.21.15.31:0/2060383600 wait complete. 2023-10-06T05:23:27.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:23:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.246+0000 7f5489bff700 1 Processor -- start 2023-10-06T05:23:27.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.246+0000 7f5489bff700 1 -- start start 2023-10-06T05:23:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.247+0000 7f5489bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54840c7b60 0x7f5484069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.247+0000 7f5489bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f54840c7f30 0x7f54840aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.247+0000 7f5489bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54840ca5b0 con 0x7f548406ef50 2023-10-06T05:23:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.247+0000 7f5489bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548406f740 con 0x7f54840c7b60 2023-10-06T05:23:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.247+0000 7f5489bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54840c9c80 con 0x7f54840c7f30 2023-10-06T05:23:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.248+0000 7f54837fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54840c7b60 0x7f5484069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.248+0000 7f54837fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54840c7b60 0x7f5484069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57844/0 (socket says 172.21.15.31:57844) 2023-10-06T05:23:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.248+0000 7f54837fe700 1 -- 172.21.15.31:0/2938017012 learned_addr learned my addr 172.21.15.31:0/2938017012 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.248+0000 7f5483fff700 1 --1- 172.21.15.31:0/2938017012 >> v1:172.21.15.31:6789/0 conn(0x7f548406ef50 0x7f548413f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33374/0 (socket says 172.21.15.31:33374) 2023-10-06T05:23:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.248+0000 7f5482ffd700 1 --2- 172.21.15.31:0/2938017012 >> v2:172.21.15.62:3300/0 conn(0x7f54840c7f30 0x7f54840aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.248+0000 7f54837fe700 1 -- 172.21.15.31:0/2938017012 >> v2:172.21.15.62:3300/0 conn(0x7f54840c7f30 msgr2=0x7f54840aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.248+0000 7f54837fe700 1 --2- 172.21.15.31:0/2938017012 >> v2:172.21.15.62:3300/0 conn(0x7f54840c7f30 0x7f54840aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.248+0000 7f54837fe700 1 -- 172.21.15.31:0/2938017012 >> v1:172.21.15.31:6789/0 conn(0x7f548406ef50 legacy=0x7f548413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.248+0000 7f54837fe700 1 -- 172.21.15.31:0/2938017012 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840c9f60 con 0x7f54840c7b60 2023-10-06T05:23:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.248+0000 7f54837fe700 1 --2- 172.21.15.31:0/2938017012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54840c7b60 0x7f5484069880 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f5474009990 tx=0x7f5474009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ea91fa68842aba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f54827fc700 1 -- 172.21.15.31:0/2938017012 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474005c10 con 0x7f54840c7b60 2023-10-06T05:23:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f54827fc700 1 -- 172.21.15.31:0/2938017012 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5474005d70 con 0x7f54840c7b60 2023-10-06T05:23:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f54827fc700 1 -- 172.21.15.31:0/2938017012 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547400b840 con 0x7f54840c7b60 2023-10-06T05:23:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f5489bff700 1 -- 172.21.15.31:0/2938017012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54840c7b60 msgr2=0x7f5484069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f5489bff700 1 --2- 172.21.15.31:0/2938017012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54840c7b60 0x7f5484069880 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f5474009990 tx=0x7f5474009220 comp rx=0 tx=0).stop 2023-10-06T05:23:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f5489bff700 1 -- 172.21.15.31:0/2938017012 shutdown_connections 2023-10-06T05:23:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f5489bff700 1 --2- 172.21.15.31:0/2938017012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54840c7b60 0x7f5484069880 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f5489bff700 1 --2- 172.21.15.31:0/2938017012 >> v2:172.21.15.62:3300/0 conn(0x7f54840c7f30 0x7f54840aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f5489bff700 1 -- 172.21.15.31:0/2938017012 >> 172.21.15.31:0/2938017012 conn(0x7f54840cc0b0 msgr2=0x7f54840cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f5489bff700 1 -- 172.21.15.31:0/2938017012 shutdown_connections 2023-10-06T05:23:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.249+0000 7f5489bff700 1 -- 172.21.15.31:0/2938017012 wait complete. 2023-10-06T05:23:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.251+0000 7f5489bff700 1 Processor -- start 2023-10-06T05:23:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.252+0000 7f5489bff700 1 -- start start 2023-10-06T05:23:27.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f5489bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f548406ef50 0x7f5484152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f5489bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54840c7b60 0x7f54840c2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f5489bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54840c7f30 0x7f5484157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f5489bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5484169a80 con 0x7f548406ef50 2023-10-06T05:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f5489bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54840abf70 con 0x7f54840c7b60 2023-10-06T05:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f5489bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54840ac250 con 0x7f54840c7f30 2023-10-06T05:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f54837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f548406ef50 0x7f5484152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f54837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f548406ef50 0x7f5484152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57860/0 (socket says 172.21.15.31:57860) 2023-10-06T05:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f54837fe700 1 -- 172.21.15.31:0/1931629199 learned_addr learned my addr 172.21.15.31:0/1931629199 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f5482ffd700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54840c7b60 0x7f54840c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f5483fff700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54840c7f30 0x7f5484157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.253+0000 7f54837fe700 1 -- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54840c7f30 msgr2=0x7f5484157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f54837fe700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54840c7f30 0x7f5484157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f54837fe700 1 -- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54840c7b60 msgr2=0x7f54840c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f54837fe700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54840c7b60 0x7f54840c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f54837fe700 1 -- 172.21.15.31:0/1931629199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5474009480 con 0x7f548406ef50 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f5483fff700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54840c7f30 0x7f5484157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f54837fe700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f548406ef50 0x7f5484152e70 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f548406cd80 tx=0x7f5474016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f546b7fe700 1 -- 172.21.15.31:0/1931629199 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474011a20 con 0x7f548406ef50 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f546b7fe700 1 -- 172.21.15.31:0/1931629199 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5474011b80 con 0x7f548406ef50 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f546b7fe700 1 -- 172.21.15.31:0/1931629199 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547401c410 con 0x7f548406ef50 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f5489bff700 1 -- 172.21.15.31:0/1931629199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54840ac530 con 0x7f548406ef50 2023-10-06T05:23:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.254+0000 7f5489bff700 1 -- 172.21.15.31:0/1931629199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54840adef0 con 0x7f548406ef50 2023-10-06T05:23:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.255+0000 7f5489bff700 1 -- 172.21.15.31:0/1931629199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54840ac9d0 con 0x7f548406ef50 2023-10-06T05:23:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.256+0000 7f546b7fe700 1 -- 172.21.15.31:0/1931629199 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f547401c800 con 0x7f548406ef50 2023-10-06T05:23:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.257+0000 7f546b7fe700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5470067320 0x7f54700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.257+0000 7f546b7fe700 1 -- 172.21.15.31:0/1931629199 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f5474086470 con 0x7f548406ef50 2023-10-06T05:23:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.258+0000 7f5482ffd700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5470067320 0x7f54700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.259+0000 7f5482ffd700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5470067320 0x7f54700697d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5478006fd0 tx=0x7f5478008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:27.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.261+0000 7f546b7fe700 1 -- 172.21.15.31:0/1931629199 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5474053310 con 0x7f548406ef50 2023-10-06T05:23:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.467+0000 7f5489bff700 1 -- 172.21.15.31:0/1931629199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f548404c380 con 0x7f548406ef50 2023-10-06T05:23:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.468+0000 7f546b7fe700 1 -- 172.21.15.31:0/1931629199 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5474056930 con 0x7f548406ef50 2023-10-06T05:23:27.470 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:27.470 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:23:27.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:27.470 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:23:27.470 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:23:27.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:23:27.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:27.470 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:23:27.471 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:23:27.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:27.471 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:23:27.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 -- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5470067320 msgr2=0x7f54700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5470067320 0x7f54700697d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5478006fd0 tx=0x7f5478008040 comp rx=0 tx=0).stop 2023-10-06T05:23:27.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 -- 172.21.15.31:0/1931629199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f548406ef50 msgr2=0x7f5484152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f548406ef50 0x7f5484152e70 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f548406cd80 tx=0x7f5474016000 comp rx=0 tx=0).stop 2023-10-06T05:23:27.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 -- 172.21.15.31:0/1931629199 shutdown_connections 2023-10-06T05:23:27.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5470067320 0x7f54700697d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54840c7f30 0x7f5484157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f548406ef50 0x7f5484152e70 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 --2- 172.21.15.31:0/1931629199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54840c7b60 0x7f54840c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 -- 172.21.15.31:0/1931629199 >> 172.21.15.31:0/1931629199 conn(0x7f54840cc0b0 msgr2=0x7f5484155e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:27.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 -- 172.21.15.31:0/1931629199 shutdown_connections 2023-10-06T05:23:27.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.470+0000 7f5489bff700 1 -- 172.21.15.31:0/1931629199 wait complete. 2023-10-06T05:23:27.489 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:23:27.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:23:27.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.686+0000 7fb06c1ff700 1 Processor -- start 2023-10-06T05:23:27.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.686+0000 7fb06c1ff700 1 -- start start 2023-10-06T05:23:27.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.687+0000 7fb06c1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb064156300 0x7fb0640ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.687+0000 7fb06c1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb06406ea90 0x7fb06406d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.687+0000 7fb06c1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb06406a630 0x7fb06406aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.687+0000 7fb06c1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06406aff0 con 0x7fb06406ea90 2023-10-06T05:23:27.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.687+0000 7fb06c1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06406e600 con 0x7fb06406a630 2023-10-06T05:23:27.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.687+0000 7fb06c1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06406ef00 con 0x7fb064156300 2023-10-06T05:23:27.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.687+0000 7fb06979a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb06406ea90 0x7fb06406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb06a79c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb06406a630 0x7fb06406aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb069f9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb064156300 0x7fb0640ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb06979a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb06406ea90 0x7fb06406d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57868/0 (socket says 172.21.15.31:57868) 2023-10-06T05:23:27.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb06979a700 1 -- 172.21.15.31:0/3336613532 learned_addr learned my addr 172.21.15.31:0/3336613532 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:27.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb06979a700 1 -- 172.21.15.31:0/3336613532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb064156300 msgr2=0x7fb0640ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:27.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb06979a700 1 --2- 172.21.15.31:0/3336613532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb064156300 0x7fb0640ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb06979a700 1 -- 172.21.15.31:0/3336613532 >> v2:172.21.15.62:3300/0 conn(0x7fb06406a630 msgr2=0x7fb06406aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:27.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb06979a700 1 --2- 172.21.15.31:0/3336613532 >> v2:172.21.15.62:3300/0 conn(0x7fb06406a630 0x7fb06406aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb06979a700 1 -- 172.21.15.31:0/3336613532 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06406d750 con 0x7fb06406ea90 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb06979a700 1 --2- 172.21.15.31:0/3336613532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406ea90 0x7fb06406d210 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb06406d0e0 tx=0x7fb05400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c52b0dcc068f80f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.688+0000 7fb068f99700 1 -- 172.21.15.31:0/3336613532 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05400db40 con 0x7fb06406ea90 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb068f99700 1 -- 172.21.15.31:0/3336613532 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05400dca0 con 0x7fb06406ea90 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb068f99700 1 -- 172.21.15.31:0/3336613532 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0540196b0 con 0x7fb06406ea90 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb06c1ff700 1 -- 172.21.15.31:0/3336613532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406ea90 msgr2=0x7fb06406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb06c1ff700 1 --2- 172.21.15.31:0/3336613532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406ea90 0x7fb06406d210 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb06406d0e0 tx=0x7fb05400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb06c1ff700 1 -- 172.21.15.31:0/3336613532 shutdown_connections 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb06c1ff700 1 --2- 172.21.15.31:0/3336613532 >> v2:172.21.15.62:3300/0 conn(0x7fb06406a630 0x7fb06406aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb06c1ff700 1 --2- 172.21.15.31:0/3336613532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb064156300 0x7fb0640ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb06c1ff700 1 --2- 172.21.15.31:0/3336613532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406ea90 0x7fb06406d210 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb06c1ff700 1 -- 172.21.15.31:0/3336613532 >> 172.21.15.31:0/3336613532 conn(0x7fb064068030 msgr2=0x7fb0640cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:27.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb06c1ff700 1 -- 172.21.15.31:0/3336613532 shutdown_connections 2023-10-06T05:23:27.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.689+0000 7fb06c1ff700 1 -- 172.21.15.31:0/3336613532 wait complete. 2023-10-06T05:23:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.691+0000 7fb06c1ff700 1 Processor -- start 2023-10-06T05:23:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.691+0000 7fb06c1ff700 1 -- start start 2023-10-06T05:23:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb06c1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406a630 0x7fb064154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb06c1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb06406ea90 0x7fb064158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb06c1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb064156300 0x7fb064158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb06c1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06416aa50 con 0x7fb06406a630 2023-10-06T05:23:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb06c1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0641594c0 con 0x7fb06406ea90 2023-10-06T05:23:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb06c1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640aa7e0 con 0x7fb064156300 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb069f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406a630 0x7fb064154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb069f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406a630 0x7fb064154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57876/0 (socket says 172.21.15.31:57876) 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb069f9b700 1 -- 172.21.15.31:0/1284046095 learned_addr learned my addr 172.21.15.31:0/1284046095 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb06979a700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb06406ea90 0x7fb064158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb06a79c700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb064156300 0x7fb064158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb069f9b700 1 -- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb064156300 msgr2=0x7fb064158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.692+0000 7fb069f9b700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb064156300 0x7fb064158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.693+0000 7fb069f9b700 1 -- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb06406ea90 msgr2=0x7fb064158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.693+0000 7fb069f9b700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb06406ea90 0x7fb064158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.693+0000 7fb069f9b700 1 -- 172.21.15.31:0/1284046095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05400b790 con 0x7fb06406a630 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.693+0000 7fb06979a700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb06406ea90 0x7fb064158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:27.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.693+0000 7fb069f9b700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406a630 0x7fb064154640 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb06000ea00 tx=0x7fb060009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.693+0000 7fb059ffb700 1 -- 172.21.15.31:0/1284046095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06000c850 con 0x7fb06406a630 2023-10-06T05:23:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.693+0000 7fb059ffb700 1 -- 172.21.15.31:0/1284046095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06000c9b0 con 0x7fb06406a630 2023-10-06T05:23:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.693+0000 7fb06c1ff700 1 -- 172.21.15.31:0/1284046095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0640aaac0 con 0x7fb06406a630 2023-10-06T05:23:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.693+0000 7fb059ffb700 1 -- 172.21.15.31:0/1284046095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06000cd30 con 0x7fb06406a630 2023-10-06T05:23:27.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.693+0000 7fb06c1ff700 1 -- 172.21.15.31:0/1284046095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0640ab100 con 0x7fb06406a630 2023-10-06T05:23:27.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.694+0000 7fb06c1ff700 1 -- 172.21.15.31:0/1284046095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0640c68a0 con 0x7fb06406a630 2023-10-06T05:23:27.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.695+0000 7fb059ffb700 1 -- 172.21.15.31:0/1284046095 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb060013680 con 0x7fb06406a630 2023-10-06T05:23:27.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.696+0000 7fb059ffb700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb048067320 0x7fb0480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:27.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.696+0000 7fb059ffb700 1 -- 172.21.15.31:0/1284046095 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fb060085180 con 0x7fb06406a630 2023-10-06T05:23:27.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.697+0000 7fb06979a700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb048067320 0x7fb0480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:27.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.697+0000 7fb06979a700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb048067320 0x7fb0480697d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb06406d0e0 tx=0x7fb05401b000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.700+0000 7fb059ffb700 1 -- 172.21.15.31:0/1284046095 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb060051fa0 con 0x7fb06406a630 2023-10-06T05:23:27.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:27.902+0000 7fb06c1ff700 1 -- 172.21.15.31:0/1284046095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb064154be0 con 0x7fb06406a630 2023-10-06T05:23:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.105+0000 7fb059ffb700 1 -- 172.21.15.31:0/1284046095 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb0600555c0 con 0x7fb06406a630 2023-10-06T05:23:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 -- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb048067320 msgr2=0x7fb0480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb048067320 0x7fb0480697d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb06406d0e0 tx=0x7fb05401b000 comp rx=0 tx=0).stop 2023-10-06T05:23:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 -- 172.21.15.31:0/1284046095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406a630 msgr2=0x7fb064154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406a630 0x7fb064154640 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb06000ea00 tx=0x7fb060009120 comp rx=0 tx=0).stop 2023-10-06T05:23:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 -- 172.21.15.31:0/1284046095 shutdown_connections 2023-10-06T05:23:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb048067320 0x7fb0480697d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb064156300 0x7fb064158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb06406a630 0x7fb064154640 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 --2- 172.21.15.31:0/1284046095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb06406ea90 0x7fb064158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 -- 172.21.15.31:0/1284046095 >> 172.21.15.31:0/1284046095 conn(0x7fb064068030 msgr2=0x7fb0640cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 -- 172.21.15.31:0/1284046095 shutdown_connections 2023-10-06T05:23:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.109+0000 7fb06c1ff700 1 -- 172.21.15.31:0/1284046095 wait complete. 2023-10-06T05:23:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:23:28.129 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:23:28.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.333+0000 7fc8003ff700 1 Processor -- start 2023-10-06T05:23:28.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.333+0000 7fc8003ff700 1 -- start start 2023-10-06T05:23:28.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.334+0000 7fc8003ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80c7190 0x7fc7f80c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.334+0000 7fc8003ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f8068250 con 0x7fc7f80c7560 2023-10-06T05:23:28.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.334+0000 7fc8003ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f806ac80 con 0x7fc7f80aef00 2023-10-06T05:23:28.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.334+0000 7fc8003ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8067f20 con 0x7fc7f80c7190 2023-10-06T05:23:28.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.334+0000 7fc7fe99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc7f80c7560 0x7fc7f80686e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33382/0 (socket says 172.21.15.31:33382) 2023-10-06T05:23:28.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.334+0000 7fc7fe99c700 1 -- 172.21.15.31:0/1705672325 learned_addr learned my addr 172.21.15.31:0/1705672325 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:28.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.334+0000 7fc7fe19b700 1 --2- 172.21.15.31:0/1705672325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80c7190 0x7fc7f80c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.335+0000 7fc7fd199700 1 -- 172.21.15.31:0/1705672325 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196348615 0 0) 0x7fc7f8068250 con 0x7fc7f80c7560 2023-10-06T05:23:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.335+0000 7fc7fd199700 1 -- 172.21.15.31:0/1705672325 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e4003570 con 0x7fc7f80c7560 2023-10-06T05:23:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.335+0000 7fc7fd199700 1 -- 172.21.15.31:0/1705672325 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620219234 0 0) 0x7fc7f806ac80 con 0x7fc7f80aef00 2023-10-06T05:23:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.335+0000 7fc7fd199700 1 -- 172.21.15.31:0/1705672325 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f8068250 con 0x7fc7f80aef00 2023-10-06T05:23:28.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.335+0000 7fc7fe19b700 1 -- 172.21.15.31:0/1705672325 >> v1:172.21.15.62:6789/0 conn(0x7fc7f80aef00 legacy=0x7fc7f80af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.335+0000 7fc7fe19b700 1 -- 172.21.15.31:0/1705672325 >> v1:172.21.15.31:6789/0 conn(0x7fc7f80c7560 legacy=0x7fc7f80686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.335+0000 7fc7fe19b700 1 -- 172.21.15.31:0/1705672325 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f806aed0 con 0x7fc7f80c7190 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.335+0000 7fc7fe19b700 1 --2- 172.21.15.31:0/1705672325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80c7190 0x7fc7f80c9d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009910 tx=0x7fc7f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4bb665b038edd78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.336+0000 7fc7fd199700 1 -- 172.21.15.31:0/1705672325 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4011630 con 0x7fc7f80c7190 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.336+0000 7fc7fd199700 1 -- 172.21.15.31:0/1705672325 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f4011790 con 0x7fc7f80c7190 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.336+0000 7fc7fd199700 1 -- 172.21.15.31:0/1705672325 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4011b30 con 0x7fc7f80c7190 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.336+0000 7fc8003ff700 1 -- 172.21.15.31:0/1705672325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80c7190 msgr2=0x7fc7f80c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.336+0000 7fc8003ff700 1 --2- 172.21.15.31:0/1705672325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80c7190 0x7fc7f80c9d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009910 tx=0x7fc7f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.336+0000 7fc8003ff700 1 -- 172.21.15.31:0/1705672325 shutdown_connections 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.336+0000 7fc8003ff700 1 --2- 172.21.15.31:0/1705672325 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80c7190 0x7fc7f80c9d40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.336+0000 7fc8003ff700 1 -- 172.21.15.31:0/1705672325 >> 172.21.15.31:0/1705672325 conn(0x7fc7f80cd3d0 msgr2=0x7fc7f80c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:28.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.336+0000 7fc8003ff700 1 -- 172.21.15.31:0/1705672325 shutdown_connections 2023-10-06T05:23:28.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.336+0000 7fc8003ff700 1 -- 172.21.15.31:0/1705672325 wait complete. 2023-10-06T05:23:28.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.338+0000 7fc8003ff700 1 Processor -- start 2023-10-06T05:23:28.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.339+0000 7fc8003ff700 1 -- start start 2023-10-06T05:23:28.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc8003ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80aef00 0x7fc7f8157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc8003ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7f80c7190 0x7fc7f815f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc8003ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7f80c7560 0x7fc7f815f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc8003ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8169830 con 0x7fc7f80c7190 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc8003ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80c68d0 con 0x7fc7f80c7560 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc8003ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8169a10 con 0x7fc7f80aef00 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc7fd99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7f80c7190 0x7fc7f815f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc7fd99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7f80c7190 0x7fc7f815f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57882/0 (socket says 172.21.15.31:57882) 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc7fe19b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80aef00 0x7fc7f8157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc7fd99a700 1 -- 172.21.15.31:0/814149466 learned_addr learned my addr 172.21.15.31:0/814149466 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.340+0000 7fc7fe99c700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7f80c7560 0x7fc7f815f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc7fd99a700 1 -- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80aef00 msgr2=0x7fc7f8157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc7fd99a700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80aef00 0x7fc7f8157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc7fd99a700 1 -- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7f80c7560 msgr2=0x7fc7f815f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc7fd99a700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7f80c7560 0x7fc7f815f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc7fd99a700 1 -- 172.21.15.31:0/814149466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f4009420 con 0x7fc7f80c7190 2023-10-06T05:23:28.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc7fd99a700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7f80c7190 0x7fc7f815f390 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800c580 tx=0x7fc7e8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:28.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc7ee7fc700 1 -- 172.21.15.31:0/814149466 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8010510 con 0x7fc7f80c7190 2023-10-06T05:23:28.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc7ee7fc700 1 -- 172.21.15.31:0/814149466 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e8010670 con 0x7fc7f80c7190 2023-10-06T05:23:28.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc7ee7fc700 1 -- 172.21.15.31:0/814149466 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8010a30 con 0x7fc7f80c7190 2023-10-06T05:23:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc8003ff700 1 -- 172.21.15.31:0/814149466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f8169cf0 con 0x7fc7f80c7190 2023-10-06T05:23:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.341+0000 7fc8003ff700 1 -- 172.21.15.31:0/814149466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f816a2b0 con 0x7fc7f80c7190 2023-10-06T05:23:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.342+0000 7fc8003ff700 1 -- 172.21.15.31:0/814149466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f816a4d0 con 0x7fc7f80c7190 2023-10-06T05:23:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.343+0000 7fc7ee7fc700 1 -- 172.21.15.31:0/814149466 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc7e801b400 con 0x7fc7f80c7190 2023-10-06T05:23:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.344+0000 7fc7ee7fc700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc7dc067320 0x7fc7dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:28.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.344+0000 7fc7ee7fc700 1 -- 172.21.15.31:0/814149466 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5604+0+0 (secure 0 0 0) 0x7fc7e8084730 con 0x7fc7f80c7190 2023-10-06T05:23:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.345+0000 7fc7fe19b700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc7dc067320 0x7fc7dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:28.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.346+0000 7fc7fe19b700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc7dc067320 0x7fc7dc0697d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80c2b60 tx=0x7fc7f4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:28.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.349+0000 7fc7ee7fc700 1 -- 172.21.15.31:0/814149466 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7e8051440 con 0x7fc7f80c7190 2023-10-06T05:23:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:28.551+0000 7fc8003ff700 1 -- 172.21.15.31:0/814149466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc7f804c380 con 0x7fc7f80c7190 2023-10-06T05:23:29.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.114+0000 7fc7ee7fc700 1 -- 172.21.15.31:0/814149466 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc7e8054a60 con 0x7fc7f80c7190 2023-10-06T05:23:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.121+0000 7fc8003ff700 1 -- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc7dc067320 msgr2=0x7fc7dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.121+0000 7fc8003ff700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc7dc067320 0x7fc7dc0697d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80c2b60 tx=0x7fc7f4017000 comp rx=0 tx=0).stop 2023-10-06T05:23:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.121+0000 7fc8003ff700 1 -- 172.21.15.31:0/814149466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7f80c7190 msgr2=0x7fc7f815f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.121+0000 7fc8003ff700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7f80c7190 0x7fc7f815f390 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800c580 tx=0x7fc7e8007e10 comp rx=0 tx=0).stop 2023-10-06T05:23:29.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.121+0000 7fc8003ff700 1 -- 172.21.15.31:0/814149466 shutdown_connections 2023-10-06T05:23:29.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.121+0000 7fc8003ff700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc7dc067320 0x7fc7dc0697d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.121+0000 7fc8003ff700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7f80c7560 0x7fc7f815f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.121+0000 7fc8003ff700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7f80aef00 0x7fc7f8157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.121+0000 7fc8003ff700 1 --2- 172.21.15.31:0/814149466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7f80c7190 0x7fc7f815f390 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.121+0000 7fc8003ff700 1 -- 172.21.15.31:0/814149466 >> 172.21.15.31:0/814149466 conn(0x7fc7f80cd3d0 msgr2=0x7fc7f81588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.123+0000 7fc8003ff700 1 -- 172.21.15.31:0/814149466 shutdown_connections 2023-10-06T05:23:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.123+0000 7fc8003ff700 1 -- 172.21.15.31:0/814149466 wait complete. 2023-10-06T05:23:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:23:29.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:23:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.353+0000 7efd4d1ff700 1 Processor -- start 2023-10-06T05:23:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.353+0000 7efd4d1ff700 1 -- start start 2023-10-06T05:23:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4d1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efd480c8f40 0x7efd48068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4d1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efd4806a300 0x7efd480adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4d1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd48068da0 con 0x7efd4815ca40 2023-10-06T05:23:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4d1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd480cabc0 con 0x7efd480c8f40 2023-10-06T05:23:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4d1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd48068bb0 con 0x7efd4806a300 2023-10-06T05:23:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efd480c8f40 0x7efd48068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efd480c8f40 0x7efd48068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57892/0 (socket says 172.21.15.31:57892) 2023-10-06T05:23:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4659c700 1 -- 172.21.15.31:0/2924861779 learned_addr learned my addr 172.21.15.31:0/2924861779 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:29.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4759e700 1 --2- 172.21.15.31:0/2924861779 >> v2:172.21.15.62:3300/0 conn(0x7efd4806a300 0x7efd480adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4659c700 1 -- 172.21.15.31:0/2924861779 >> v2:172.21.15.62:3300/0 conn(0x7efd4806a300 msgr2=0x7efd480adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4659c700 1 --2- 172.21.15.31:0/2924861779 >> v2:172.21.15.62:3300/0 conn(0x7efd4806a300 0x7efd480adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4659c700 1 -- 172.21.15.31:0/2924861779 >> v1:172.21.15.62:6789/0 conn(0x7efd4815ca40 legacy=0x7efd480c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.355+0000 7efd4659c700 1 -- 172.21.15.31:0/2924861779 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd48069080 con 0x7efd480c8f40 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.356+0000 7efd4659c700 1 --2- 172.21.15.31:0/2924861779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd480c8f40 0x7efd48068670 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7efd4806c5f0 tx=0x7efd3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f9020ab408465f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.356+0000 7efd45d9b700 1 -- 172.21.15.31:0/2924861779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c00b860 con 0x7efd480c8f40 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.356+0000 7efd45d9b700 1 -- 172.21.15.31:0/2924861779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c00b9c0 con 0x7efd480c8f40 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.356+0000 7efd45d9b700 1 -- 172.21.15.31:0/2924861779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c00bd60 con 0x7efd480c8f40 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.356+0000 7efd4d1ff700 1 -- 172.21.15.31:0/2924861779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd480c8f40 msgr2=0x7efd48068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.356+0000 7efd4d1ff700 1 --2- 172.21.15.31:0/2924861779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd480c8f40 0x7efd48068670 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7efd4806c5f0 tx=0x7efd3c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.357+0000 7efd4d1ff700 1 -- 172.21.15.31:0/2924861779 shutdown_connections 2023-10-06T05:23:29.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.357+0000 7efd4d1ff700 1 --2- 172.21.15.31:0/2924861779 >> v2:172.21.15.62:3300/0 conn(0x7efd4806a300 0x7efd480adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.357+0000 7efd4d1ff700 1 --2- 172.21.15.31:0/2924861779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd480c8f40 0x7efd48068670 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.357+0000 7efd4d1ff700 1 -- 172.21.15.31:0/2924861779 >> 172.21.15.31:0/2924861779 conn(0x7efd480ce290 msgr2=0x7efd480ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.357+0000 7efd4d1ff700 1 -- 172.21.15.31:0/2924861779 shutdown_connections 2023-10-06T05:23:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.357+0000 7efd4d1ff700 1 -- 172.21.15.31:0/2924861779 wait complete. 2023-10-06T05:23:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.358+0000 7efd4d1ff700 1 Processor -- start 2023-10-06T05:23:29.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.358+0000 7efd4d1ff700 1 -- start start 2023-10-06T05:23:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.359+0000 7efd4d1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4806a300 0x7efd4816d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.359+0000 7efd4d1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd480c8f40 0x7efd480c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.359+0000 7efd4d1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd4815ca40 0x7efd48169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.359+0000 7efd4d1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4816df70 con 0x7efd4806a300 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.359+0000 7efd4d1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd480c1eb0 con 0x7efd480c8f40 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.359+0000 7efd4d1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd480b3fc0 con 0x7efd4815ca40 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.359+0000 7efd46d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4806a300 0x7efd4816d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd46d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4806a300 0x7efd4816d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57896/0 (socket says 172.21.15.31:57896) 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd46d9d700 1 -- 172.21.15.31:0/780827502 learned_addr learned my addr 172.21.15.31:0/780827502 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd4759e700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd4815ca40 0x7efd48169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd46d9d700 1 -- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd4815ca40 msgr2=0x7efd48169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd4659c700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd480c8f40 0x7efd480c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd46d9d700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd4815ca40 0x7efd48169a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd46d9d700 1 -- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd480c8f40 msgr2=0x7efd480c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd46d9d700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd480c8f40 0x7efd480c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd46d9d700 1 -- 172.21.15.31:0/780827502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0094f0 con 0x7efd4806a300 2023-10-06T05:23:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd4759e700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd4815ca40 0x7efd48169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd4659c700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd480c8f40 0x7efd480c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd46d9d700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4806a300 0x7efd4816d040 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c5c0 tx=0x7efd3800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd2effd700 1 -- 172.21.15.31:0/780827502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38011590 con 0x7efd4806a300 2023-10-06T05:23:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd2effd700 1 -- 172.21.15.31:0/780827502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd38011710 con 0x7efd4806a300 2023-10-06T05:23:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd2effd700 1 -- 172.21.15.31:0/780827502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38011ad0 con 0x7efd4806a300 2023-10-06T05:23:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.360+0000 7efd4d1ff700 1 -- 172.21.15.31:0/780827502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4816d6d0 con 0x7efd4806a300 2023-10-06T05:23:29.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.361+0000 7efd4d1ff700 1 -- 172.21.15.31:0/780827502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4816dc90 con 0x7efd4806a300 2023-10-06T05:23:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.362+0000 7efd4d1ff700 1 -- 172.21.15.31:0/780827502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4804c380 con 0x7efd4806a300 2023-10-06T05:23:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.363+0000 7efd2effd700 1 -- 172.21.15.31:0/780827502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efd3801b3b0 con 0x7efd4806a300 2023-10-06T05:23:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.364+0000 7efd2effd700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efd34067320 0x7efd340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.364+0000 7efd2effd700 1 -- 172.21.15.31:0/780827502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5986+0+0 (secure 0 0 0) 0x7efd38084370 con 0x7efd4806a300 2023-10-06T05:23:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.364+0000 7efd4659c700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efd34067320 0x7efd340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.365+0000 7efd4659c700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efd34067320 0x7efd340697d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efd4806c5f0 tx=0x7efd3c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:29.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.368+0000 7efd2effd700 1 -- 172.21.15.31:0/780827502 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd38050ee0 con 0x7efd4806a300 2023-10-06T05:23:29.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:29.613+0000 7efd4d1ff700 1 -- 172.21.15.31:0/780827502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efd480ca1a0 con 0x7efd4806a300 2023-10-06T05:23:30.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.131+0000 7efd2effd700 1 -- 172.21.15.31:0/780827502 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v72) v1 ==== 174+0+0 (secure 0 0 0) 0x7efd38054500 con 0x7efd4806a300 2023-10-06T05:23:30.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.137+0000 7efd4d1ff700 1 -- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efd34067320 msgr2=0x7efd340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.137+0000 7efd4d1ff700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efd34067320 0x7efd340697d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efd4806c5f0 tx=0x7efd3c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:30.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.137+0000 7efd4d1ff700 1 -- 172.21.15.31:0/780827502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4806a300 msgr2=0x7efd4816d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.137+0000 7efd4d1ff700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4806a300 0x7efd4816d040 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c5c0 tx=0x7efd3800c980 comp rx=0 tx=0).stop 2023-10-06T05:23:30.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.140+0000 7efd4d1ff700 1 -- 172.21.15.31:0/780827502 shutdown_connections 2023-10-06T05:23:30.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.140+0000 7efd4d1ff700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efd34067320 0x7efd340697d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.140+0000 7efd4d1ff700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd4815ca40 0x7efd48169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.140+0000 7efd4d1ff700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4806a300 0x7efd4816d040 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.140+0000 7efd4d1ff700 1 --2- 172.21.15.31:0/780827502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd480c8f40 0x7efd480c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.140+0000 7efd4d1ff700 1 -- 172.21.15.31:0/780827502 >> 172.21.15.31:0/780827502 conn(0x7efd480ce290 msgr2=0x7efd480aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:30.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.140+0000 7efd4d1ff700 1 -- 172.21.15.31:0/780827502 shutdown_connections 2023-10-06T05:23:30.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.140+0000 7efd4d1ff700 1 -- 172.21.15.31:0/780827502 wait complete. 2023-10-06T05:23:30.143 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 19 and data pool 20 2023-10-06T05:23:30.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.362+0000 7fb742bff700 1 Processor -- start 2023-10-06T05:23:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.363+0000 7fb742bff700 1 -- start start 2023-10-06T05:23:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.364+0000 7fb742bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb73c0c7f30 0x7fb73c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.364+0000 7fb742bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb73c0aee70 0x7fb73c13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.364+0000 7fb742bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73c0ca5b0 con 0x7fb73c0c7b60 2023-10-06T05:23:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.364+0000 7fb742bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73c0af2f0 con 0x7fb73c0aee70 2023-10-06T05:23:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.364+0000 7fb742bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73c06f740 con 0x7fb73c0c7f30 2023-10-06T05:23:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.364+0000 7fb74119c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb73c0aee70 0x7fb73c13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.364+0000 7fb74119c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb73c0aee70 0x7fb73c13f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57906/0 (socket says 172.21.15.31:57906) 2023-10-06T05:23:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.364+0000 7fb74119c700 1 -- 172.21.15.31:0/1139971912 learned_addr learned my addr 172.21.15.31:0/1139971912 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.364+0000 7fb73bfff700 1 --2- 172.21.15.31:0/1139971912 >> v2:172.21.15.62:3300/0 conn(0x7fb73c0c7f30 0x7fb73c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.365+0000 7fb74119c700 1 -- 172.21.15.31:0/1139971912 >> v2:172.21.15.62:3300/0 conn(0x7fb73c0c7f30 msgr2=0x7fb73c069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.365+0000 7fb74119c700 1 --2- 172.21.15.31:0/1139971912 >> v2:172.21.15.62:3300/0 conn(0x7fb73c0c7f30 0x7fb73c069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.365+0000 7fb74119c700 1 -- 172.21.15.31:0/1139971912 >> v1:172.21.15.62:6789/0 conn(0x7fb73c0c7b60 legacy=0x7fb73c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.365+0000 7fb74119c700 1 -- 172.21.15.31:0/1139971912 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c0c9c80 con 0x7fb73c0aee70 2023-10-06T05:23:30.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.365+0000 7fb74119c700 1 --2- 172.21.15.31:0/1139971912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0aee70 0x7fb73c13f690 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb73c06c990 tx=0x7fb734009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a560fdba736af5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.365+0000 7fb73b7fe700 1 -- 172.21.15.31:0/1139971912 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73400e070 con 0x7fb73c0aee70 2023-10-06T05:23:30.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.365+0000 7fb73b7fe700 1 -- 172.21.15.31:0/1139971912 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73400ba20 con 0x7fb73c0aee70 2023-10-06T05:23:30.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.365+0000 7fb73b7fe700 1 -- 172.21.15.31:0/1139971912 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7340092e0 con 0x7fb73c0aee70 2023-10-06T05:23:30.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.366+0000 7fb742bff700 1 -- 172.21.15.31:0/1139971912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0aee70 msgr2=0x7fb73c13f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.366+0000 7fb742bff700 1 --2- 172.21.15.31:0/1139971912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0aee70 0x7fb73c13f690 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb73c06c990 tx=0x7fb734009840 comp rx=0 tx=0).stop 2023-10-06T05:23:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.366+0000 7fb742bff700 1 -- 172.21.15.31:0/1139971912 shutdown_connections 2023-10-06T05:23:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.366+0000 7fb742bff700 1 --2- 172.21.15.31:0/1139971912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0aee70 0x7fb73c13f690 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.366+0000 7fb742bff700 1 --2- 172.21.15.31:0/1139971912 >> v2:172.21.15.62:3300/0 conn(0x7fb73c0c7f30 0x7fb73c069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.366+0000 7fb742bff700 1 -- 172.21.15.31:0/1139971912 >> 172.21.15.31:0/1139971912 conn(0x7fb73c0cc0b0 msgr2=0x7fb73c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.366+0000 7fb742bff700 1 -- 172.21.15.31:0/1139971912 shutdown_connections 2023-10-06T05:23:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.366+0000 7fb742bff700 1 -- 172.21.15.31:0/1139971912 wait complete. 2023-10-06T05:23:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.367+0000 7fb742bff700 1 Processor -- start 2023-10-06T05:23:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.368+0000 7fb742bff700 1 -- start start 2023-10-06T05:23:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.368+0000 7fb742bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73c0aee70 0x7fb73c13c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.368+0000 7fb742bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c0c7b60 0x7fb73c0a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.368+0000 7fb742bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0c7f30 0x7fb73c169530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.368+0000 7fb742bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0a5cf0 con 0x7fb73c0c7f30 2023-10-06T05:23:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.368+0000 7fb742bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c13cbf0 con 0x7fb73c0c7b60 2023-10-06T05:23:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.368+0000 7fb742bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0aa010 con 0x7fb73c0aee70 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb74119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0c7f30 0x7fb73c169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb74119c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0c7f30 0x7fb73c169530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57918/0 (socket says 172.21.15.31:57918) 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb74119c700 1 -- 172.21.15.31:0/2800307616 learned_addr learned my addr 172.21.15.31:0/2800307616 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb73bfff700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c0c7b60 0x7fb73c0a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb74099b700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73c0aee70 0x7fb73c13c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb74119c700 1 -- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73c0aee70 msgr2=0x7fb73c13c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb74119c700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73c0aee70 0x7fb73c13c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb74119c700 1 -- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c0c7b60 msgr2=0x7fb73c0a9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb74119c700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c0c7b60 0x7fb73c0a9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb74119c700 1 -- 172.21.15.31:0/2800307616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb734009ae0 con 0x7fb73c0c7f30 2023-10-06T05:23:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb74099b700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73c0aee70 0x7fb73c13c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.369+0000 7fb73bfff700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c0c7b60 0x7fb73c0a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.370+0000 7fb74119c700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0c7f30 0x7fb73c169530 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb7340072f0 tx=0x7fb734018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.370+0000 7fb738ff9700 1 -- 172.21.15.31:0/2800307616 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734008c60 con 0x7fb73c0c7f30 2023-10-06T05:23:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.370+0000 7fb738ff9700 1 -- 172.21.15.31:0/2800307616 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb734008dc0 con 0x7fb73c0c7f30 2023-10-06T05:23:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.370+0000 7fb738ff9700 1 -- 172.21.15.31:0/2800307616 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7340266b0 con 0x7fb73c0c7f30 2023-10-06T05:23:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.370+0000 7fb742bff700 1 -- 172.21.15.31:0/2800307616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73c169a70 con 0x7fb73c0c7f30 2023-10-06T05:23:30.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.370+0000 7fb742bff700 1 -- 172.21.15.31:0/2800307616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73c16a020 con 0x7fb73c0c7f30 2023-10-06T05:23:30.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.371+0000 7fb742bff700 1 -- 172.21.15.31:0/2800307616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73c04c380 con 0x7fb73c0c7f30 2023-10-06T05:23:30.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.372+0000 7fb738ff9700 1 -- 172.21.15.31:0/2800307616 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb734003040 con 0x7fb73c0c7f30 2023-10-06T05:23:30.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.373+0000 7fb738ff9700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb720067320 0x7fb7200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.373+0000 7fb738ff9700 1 -- 172.21.15.31:0/2800307616 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb734086e80 con 0x7fb73c0c7f30 2023-10-06T05:23:30.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.373+0000 7fb74099b700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb720067320 0x7fb7200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.374+0000 7fb74099b700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb720067320 0x7fb7200697d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb73c15de30 tx=0x7fb730009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.377+0000 7fb738ff9700 1 -- 172.21.15.31:0/2800307616 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb734053840 con 0x7fb73c0c7f30 2023-10-06T05:23:30.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.581+0000 7fb742bff700 1 -- 172.21.15.31:0/2800307616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb73c0c9f60 con 0x7fb73c0c7f30 2023-10-06T05:23:30.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.582+0000 7fb738ff9700 1 -- 172.21.15.31:0/2800307616 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16985 (secure 0 0 0) 0x7fb734008f30 con 0x7fb73c0c7f30 2023-10-06T05:23:30.584 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:30.584 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":113,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:23:29.615011+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:27.942469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:28.591817+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 -- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb720067320 msgr2=0x7fb7200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb720067320 0x7fb7200697d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb73c15de30 tx=0x7fb730009040 comp rx=0 tx=0).stop 2023-10-06T05:23:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 -- 172.21.15.31:0/2800307616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0c7f30 msgr2=0x7fb73c169530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0c7f30 0x7fb73c169530 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb7340072f0 tx=0x7fb734018000 comp rx=0 tx=0).stop 2023-10-06T05:23:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 -- 172.21.15.31:0/2800307616 shutdown_connections 2023-10-06T05:23:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb720067320 0x7fb7200697d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb73c0c7f30 0x7fb73c169530 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb73c0aee70 0x7fb73c13c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 --2- 172.21.15.31:0/2800307616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb73c0c7b60 0x7fb73c0a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 -- 172.21.15.31:0/2800307616 >> 172.21.15.31:0/2800307616 conn(0x7fb73c0cc0b0 msgr2=0x7fb73c0c1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 -- 172.21.15.31:0/2800307616 shutdown_connections 2023-10-06T05:23:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.585+0000 7fb742bff700 1 -- 172.21.15.31:0/2800307616 wait complete. 2023-10-06T05:23:30.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:30.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.799+0000 7fc243fff700 1 Processor -- start 2023-10-06T05:23:30.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.799+0000 7fc243fff700 1 -- start start 2023-10-06T05:23:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.801+0000 7fc243fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c06b510 0x7fc23c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.801+0000 7fc243fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc23c0cab40 0x7fc23c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.801+0000 7fc243fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc23c156ef0 con 0x7fc23c0ca770 2023-10-06T05:23:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.801+0000 7fc243fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23c0c7a70 con 0x7fc23c0cab40 2023-10-06T05:23:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.801+0000 7fc243fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c0c6df0 con 0x7fc23c06b510 2023-10-06T05:23:30.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc24159a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c06b510 0x7fc23c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc24259c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc23c0cab40 0x7fc23c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc241d9b700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc23c0ca770 0x7fc23c06ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:34636/0 (socket says 172.21.15.31:34636) 2023-10-06T05:23:30.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc241d9b700 1 -- 172.21.15.31:0/1499694782 learned_addr learned my addr 172.21.15.31:0/1499694782 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:30.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc240d99700 1 -- 172.21.15.31:0/1499694782 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64063409 0 0) 0x7fc23c156ef0 con 0x7fc23c0ca770 2023-10-06T05:23:30.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc24159a700 1 -- 172.21.15.31:0/1499694782 >> v2:172.21.15.62:3300/0 conn(0x7fc23c0cab40 msgr2=0x7fc23c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc24159a700 1 --2- 172.21.15.31:0/1499694782 >> v2:172.21.15.62:3300/0 conn(0x7fc23c0cab40 0x7fc23c1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc24159a700 1 -- 172.21.15.31:0/1499694782 >> v1:172.21.15.62:6789/0 conn(0x7fc23c0ca770 legacy=0x7fc23c06ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc24159a700 1 -- 172.21.15.31:0/1499694782 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c0c70d0 con 0x7fc23c06b510 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc24259c700 1 --2- 172.21.15.31:0/1499694782 >> v2:172.21.15.62:3300/0 conn(0x7fc23c0cab40 0x7fc23c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.802+0000 7fc24159a700 1 --2- 172.21.15.31:0/1499694782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c06b510 0x7fc23c0c66f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0098e0 tx=0x7fc22c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a301453044c84444 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.803+0000 7fc240d99700 1 -- 172.21.15.31:0/1499694782 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c00b640 con 0x7fc23c06b510 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.803+0000 7fc240d99700 1 -- 172.21.15.31:0/1499694782 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22c00b7a0 con 0x7fc23c06b510 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.803+0000 7fc240d99700 1 -- 172.21.15.31:0/1499694782 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c00bb40 con 0x7fc23c06b510 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.803+0000 7fc243fff700 1 -- 172.21.15.31:0/1499694782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c06b510 msgr2=0x7fc23c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.803+0000 7fc243fff700 1 --2- 172.21.15.31:0/1499694782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c06b510 0x7fc23c0c66f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0098e0 tx=0x7fc22c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.803+0000 7fc243fff700 1 -- 172.21.15.31:0/1499694782 shutdown_connections 2023-10-06T05:23:30.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.803+0000 7fc243fff700 1 --2- 172.21.15.31:0/1499694782 >> v2:172.21.15.62:3300/0 conn(0x7fc23c0cab40 0x7fc23c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.803+0000 7fc243fff700 1 --2- 172.21.15.31:0/1499694782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c06b510 0x7fc23c0c66f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.803+0000 7fc243fff700 1 -- 172.21.15.31:0/1499694782 >> 172.21.15.31:0/1499694782 conn(0x7fc23c0cc0b0 msgr2=0x7fc23c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:30.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.803+0000 7fc243fff700 1 -- 172.21.15.31:0/1499694782 shutdown_connections 2023-10-06T05:23:30.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.804+0000 7fc243fff700 1 -- 172.21.15.31:0/1499694782 wait complete. 2023-10-06T05:23:30.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.806+0000 7fc243fff700 1 Processor -- start 2023-10-06T05:23:30.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.806+0000 7fc243fff700 1 -- start start 2023-10-06T05:23:30.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.807+0000 7fc243fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c06b510 0x7fc23c16ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.807+0000 7fc243fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c0ca770 0x7fc23c159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.807+0000 7fc243fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc23c0cab40 0x7fc23c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.807+0000 7fc243fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c15a600 con 0x7fc23c06b510 2023-10-06T05:23:30.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.807+0000 7fc243fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c0b0b80 con 0x7fc23c0cab40 2023-10-06T05:23:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.807+0000 7fc243fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c16a900 con 0x7fc23c0ca770 2023-10-06T05:23:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.807+0000 7fc241d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c06b510 0x7fc23c16ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.807+0000 7fc241d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c06b510 0x7fc23c16ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57932/0 (socket says 172.21.15.31:57932) 2023-10-06T05:23:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.807+0000 7fc241d9b700 1 -- 172.21.15.31:0/888388623 learned_addr learned my addr 172.21.15.31:0/888388623 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc24259c700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc23c0cab40 0x7fc23c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc24159a700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c0ca770 0x7fc23c159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc241d9b700 1 -- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c0ca770 msgr2=0x7fc23c159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc241d9b700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c0ca770 0x7fc23c159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc241d9b700 1 -- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc23c0cab40 msgr2=0x7fc23c15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:30.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc241d9b700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc23c0cab40 0x7fc23c15a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:30.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc241d9b700 1 -- 172.21.15.31:0/888388623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc234009a20 con 0x7fc23c06b510 2023-10-06T05:23:30.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc241d9b700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c06b510 0x7fc23c16ad30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c7e0 tx=0x7fc238007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc231ffb700 1 -- 172.21.15.31:0/888388623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238005b80 con 0x7fc23c06b510 2023-10-06T05:23:30.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc231ffb700 1 -- 172.21.15.31:0/888388623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc238005ce0 con 0x7fc23c06b510 2023-10-06T05:23:30.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc243fff700 1 -- 172.21.15.31:0/888388623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c009420 con 0x7fc23c06b510 2023-10-06T05:23:30.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.808+0000 7fc231ffb700 1 -- 172.21.15.31:0/888388623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238007320 con 0x7fc23c06b510 2023-10-06T05:23:30.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.809+0000 7fc243fff700 1 -- 172.21.15.31:0/888388623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23c0b0810 con 0x7fc23c06b510 2023-10-06T05:23:30.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.810+0000 7fc243fff700 1 -- 172.21.15.31:0/888388623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23c04c380 con 0x7fc23c06b510 2023-10-06T05:23:30.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.811+0000 7fc231ffb700 1 -- 172.21.15.31:0/888388623 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc238007690 con 0x7fc23c06b510 2023-10-06T05:23:30.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.812+0000 7fc231ffb700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc220067320 0x7fc2200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:30.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.812+0000 7fc231ffb700 1 -- 172.21.15.31:0/888388623 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc238084280 con 0x7fc23c06b510 2023-10-06T05:23:30.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.812+0000 7fc24159a700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc220067320 0x7fc2200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:30.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.813+0000 7fc24159a700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc220067320 0x7fc2200697d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0093f0 tx=0x7fc22c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:30.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:30.816+0000 7fc231ffb700 1 -- 172.21.15.31:0/888388623 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc238050e50 con 0x7fc23c06b510 2023-10-06T05:23:31.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.018+0000 7fc243fff700 1 -- 172.21.15.31:0/888388623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc23c0cbe20 con 0x7fc23c06b510 2023-10-06T05:23:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.018+0000 7fc231ffb700 1 -- 172.21.15.31:0/888388623 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16985 (secure 0 0 0) 0x7fc238054470 con 0x7fc23c06b510 2023-10-06T05:23:31.020 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:31.021 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":113,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:23:29.615011+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:27.942469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:28.591817+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 -- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc220067320 msgr2=0x7fc2200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc220067320 0x7fc2200697d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0093f0 tx=0x7fc22c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 -- 172.21.15.31:0/888388623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c06b510 msgr2=0x7fc23c16ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c06b510 0x7fc23c16ad30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c7e0 tx=0x7fc238007e00 comp rx=0 tx=0).stop 2023-10-06T05:23:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 -- 172.21.15.31:0/888388623 shutdown_connections 2023-10-06T05:23:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc220067320 0x7fc2200697d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc23c0cab40 0x7fc23c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c06b510 0x7fc23c16ad30 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 --2- 172.21.15.31:0/888388623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc23c0ca770 0x7fc23c159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 -- 172.21.15.31:0/888388623 >> 172.21.15.31:0/888388623 conn(0x7fc23c0cc0b0 msgr2=0x7fc23c14ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:31.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 -- 172.21.15.31:0/888388623 shutdown_connections 2023-10-06T05:23:31.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.021+0000 7fc243fff700 1 -- 172.21.15.31:0/888388623 wait complete. 2023-10-06T05:23:31.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:23:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.240+0000 7f73f59ff700 1 Processor -- start 2023-10-06T05:23:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.240+0000 7f73f59ff700 1 -- start start 2023-10-06T05:23:31.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73f59ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f73f0155b40 0x7f73f00c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73f59ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f0158190 con 0x7f73f0159890 2023-10-06T05:23:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73f59ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f0157580 con 0x7f73f0159c60 2023-10-06T05:23:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73f59ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f0157860 con 0x7f73f0155b40 2023-10-06T05:23:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73eeffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f73f0159890 0x7f73f0156a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33394/0 (socket says 172.21.15.31:33394) 2023-10-06T05:23:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73eeffd700 1 -- 172.21.15.31:0/3564528885 learned_addr learned my addr 172.21.15.31:0/3564528885 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73ef7fe700 1 --2- 172.21.15.31:0/3564528885 >> v2:172.21.15.62:3300/0 conn(0x7f73f0155b40 0x7f73f00c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73edffb700 1 -- 172.21.15.31:0/3564528885 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242458438 0 0) 0x7f73f0158190 con 0x7f73f0159890 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73edffb700 1 -- 172.21.15.31:0/3564528885 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73d8004680 con 0x7f73f0159890 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73edffb700 1 -- 172.21.15.31:0/3564528885 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469651064 0 0) 0x7f73f0157580 con 0x7f73f0159c60 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.242+0000 7f73edffb700 1 -- 172.21.15.31:0/3564528885 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73f0158190 con 0x7f73f0159c60 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.243+0000 7f73ef7fe700 1 -- 172.21.15.31:0/3564528885 >> v1:172.21.15.62:6789/0 conn(0x7f73f0159c60 legacy=0x7f73f0155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.243+0000 7f73ef7fe700 1 -- 172.21.15.31:0/3564528885 >> v1:172.21.15.31:6789/0 conn(0x7f73f0159890 legacy=0x7f73f0156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.243+0000 7f73edffb700 1 -- 172.21.15.31:0/3564528885 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1870789017 0 0) 0x7f73d8004680 con 0x7f73f0159890 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.243+0000 7f73ef7fe700 1 -- 172.21.15.31:0/3564528885 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0157b40 con 0x7f73f0155b40 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.243+0000 7f73ef7fe700 1 --2- 172.21.15.31:0/3564528885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73f0155b40 0x7f73f00c89e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009e60 tx=0x7f73e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80b167de7b70efb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.243+0000 7f73edffb700 1 -- 172.21.15.31:0/3564528885 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0001470 con 0x7f73f0155b40 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.243+0000 7f73edffb700 1 -- 172.21.15.31:0/3564528885 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e800b7a0 con 0x7f73f0155b40 2023-10-06T05:23:31.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.243+0000 7f73edffb700 1 -- 172.21.15.31:0/3564528885 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e800bb40 con 0x7f73f0155b40 2023-10-06T05:23:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.244+0000 7f73f59ff700 1 -- 172.21.15.31:0/3564528885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73f0155b40 msgr2=0x7f73f00c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.244+0000 7f73f59ff700 1 --2- 172.21.15.31:0/3564528885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73f0155b40 0x7f73f00c89e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009e60 tx=0x7f73e8009780 comp rx=0 tx=0).stop 2023-10-06T05:23:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.244+0000 7f73f59ff700 1 -- 172.21.15.31:0/3564528885 shutdown_connections 2023-10-06T05:23:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.244+0000 7f73f59ff700 1 --2- 172.21.15.31:0/3564528885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73f0155b40 0x7f73f00c89e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.244+0000 7f73f59ff700 1 -- 172.21.15.31:0/3564528885 >> 172.21.15.31:0/3564528885 conn(0x7f73f0067560 msgr2=0x7f73f0067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.244+0000 7f73f59ff700 1 -- 172.21.15.31:0/3564528885 shutdown_connections 2023-10-06T05:23:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.244+0000 7f73f59ff700 1 -- 172.21.15.31:0/3564528885 wait complete. 2023-10-06T05:23:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.245+0000 7f73f59ff700 1 Processor -- start 2023-10-06T05:23:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.246+0000 7f73f59ff700 1 -- start start 2023-10-06T05:23:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.246+0000 7f73f59ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73f0155b40 0x7f73f00c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.246+0000 7f73f59ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73f0159890 0x7f73f01504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.246+0000 7f73f59ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73f0159c60 0x7f73f0150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73f59ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f00c7780 con 0x7f73f0159890 2023-10-06T05:23:31.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73f59ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f00c7a60 con 0x7f73f0159c60 2023-10-06T05:23:31.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73f59ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f00c7d40 con 0x7f73f0155b40 2023-10-06T05:23:31.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73ee7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73f0159890 0x7f73f01504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73ee7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73f0159890 0x7f73f01504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57934/0 (socket says 172.21.15.31:57934) 2023-10-06T05:23:31.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73ee7fc700 1 -- 172.21.15.31:0/766891282 learned_addr learned my addr 172.21.15.31:0/766891282 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:31.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73ef7fe700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73f0159c60 0x7f73f0150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73eeffd700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73f0155b40 0x7f73f00c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73ee7fc700 1 -- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73f0155b40 msgr2=0x7f73f00c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73ee7fc700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73f0155b40 0x7f73f00c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73ee7fc700 1 -- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73f0159c60 msgr2=0x7f73f0150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73ee7fc700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73f0159c60 0x7f73f0150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73ee7fc700 1 -- 172.21.15.31:0/766891282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e8009a20 con 0x7f73f0159890 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.247+0000 7f73ee7fc700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73f0159890 0x7f73f01504d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f73e400c790 tx=0x7f73e4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.248+0000 7f73d6ffd700 1 -- 172.21.15.31:0/766891282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4010510 con 0x7f73f0159890 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.248+0000 7f73d6ffd700 1 -- 172.21.15.31:0/766891282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e4010670 con 0x7f73f0159890 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.248+0000 7f73f59ff700 1 -- 172.21.15.31:0/766891282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f016a500 con 0x7f73f0159890 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.248+0000 7f73d6ffd700 1 -- 172.21.15.31:0/766891282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4010a30 con 0x7f73f0159890 2023-10-06T05:23:31.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.248+0000 7f73f59ff700 1 -- 172.21.15.31:0/766891282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f013c710 con 0x7f73f0159890 2023-10-06T05:23:31.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.249+0000 7f73f59ff700 1 -- 172.21.15.31:0/766891282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f016a360 con 0x7f73f0159890 2023-10-06T05:23:31.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.250+0000 7f73d6ffd700 1 -- 172.21.15.31:0/766891282 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f73e401b3b0 con 0x7f73f0159890 2023-10-06T05:23:31.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.250+0000 7f73d6ffd700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73dc067320 0x7f73dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:31.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.251+0000 7f73d6ffd700 1 -- 172.21.15.31:0/766891282 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f73e40846f0 con 0x7f73f0159890 2023-10-06T05:23:31.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.251+0000 7f73eeffd700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73dc067320 0x7f73dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:31.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.252+0000 7f73eeffd700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73dc067320 0x7f73dc0697d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f73e0001660 tx=0x7f73e0001940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.255+0000 7f73d6ffd700 1 -- 172.21.15.31:0/766891282 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73f016a360 con 0x7f73f0159890 2023-10-06T05:23:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:31.506+0000 7f73f59ff700 1 -- 172.21.15.31:0/766891282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f73f016a360 con 0x7f73f0159890 2023-10-06T05:23:31.849 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:31.904 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:23:32.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.142+0000 7f73d6ffd700 1 -- 172.21.15.31:0/766891282 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) v1 ==== 118+0+0 (secure 0 0 0) 0x7f73f016a360 con 0x7f73f0159890 2023-10-06T05:23:32.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 -- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73dc067320 msgr2=0x7f73dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73dc067320 0x7f73dc0697d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f73e0001660 tx=0x7f73e0001940 comp rx=0 tx=0).stop 2023-10-06T05:23:32.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 -- 172.21.15.31:0/766891282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73f0159890 msgr2=0x7f73f01504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73f0159890 0x7f73f01504d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f73e400c790 tx=0x7f73e4007e10 comp rx=0 tx=0).stop 2023-10-06T05:23:32.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 -- 172.21.15.31:0/766891282 shutdown_connections 2023-10-06T05:23:32.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f73dc067320 0x7f73dc0697d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73f0159c60 0x7f73f0150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73f0155b40 0x7f73f00c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 --2- 172.21.15.31:0/766891282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73f0159890 0x7f73f01504d0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 -- 172.21.15.31:0/766891282 >> 172.21.15.31:0/766891282 conn(0x7f73f0067560 msgr2=0x7f73f0158620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:32.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 -- 172.21.15.31:0/766891282 shutdown_connections 2023-10-06T05:23:32.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.145+0000 7f73f59ff700 1 -- 172.21.15.31:0/766891282 wait complete. 2023-10-06T05:23:32.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:23:32.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.363+0000 7f9a121ff700 1 Processor -- start 2023-10-06T05:23:32.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.364+0000 7f9a121ff700 1 -- start start 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.365+0000 7f9a121ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a0c153d40 0x7f9a0c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.365+0000 7f9a121ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9a0c0b09c0 0x7f9a0c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.365+0000 7f9a121ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0b0e40 con 0x7f9a0c0af6c0 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.365+0000 7f9a121ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0ad690 con 0x7f9a0c153d40 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.365+0000 7f9a121ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0acd60 con 0x7f9a0c0b09c0 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.365+0000 7f9a0b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a0c153d40 0x7f9a0c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.365+0000 7f9a0bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9a0c0af6c0 0x7f9a0c159ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33406/0 (socket says 172.21.15.31:33406) 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.365+0000 7f9a0bfff700 1 -- 172.21.15.31:0/4064001084 learned_addr learned my addr 172.21.15.31:0/4064001084 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.366+0000 7f9a0affd700 1 --2- 172.21.15.31:0/4064001084 >> v2:172.21.15.62:3300/0 conn(0x7f9a0c0b09c0 0x7f9a0c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.366+0000 7f9a0b7fe700 1 -- 172.21.15.31:0/4064001084 >> v2:172.21.15.62:3300/0 conn(0x7f9a0c0b09c0 msgr2=0x7f9a0c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.366+0000 7f9a0b7fe700 1 --2- 172.21.15.31:0/4064001084 >> v2:172.21.15.62:3300/0 conn(0x7f9a0c0b09c0 0x7f9a0c0af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.366+0000 7f9a0b7fe700 1 -- 172.21.15.31:0/4064001084 >> v1:172.21.15.31:6789/0 conn(0x7f9a0c0af6c0 legacy=0x7f9a0c159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.366+0000 7f9a0a7fc700 1 -- 172.21.15.31:0/4064001084 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253993415 0 0) 0x7f9a04001490 con 0x7f9a0c0af6c0 2023-10-06T05:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.366+0000 7f9a0b7fe700 1 -- 172.21.15.31:0/4064001084 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0ad040 con 0x7f9a0c153d40 2023-10-06T05:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.366+0000 7f9a0b7fe700 1 --2- 172.21.15.31:0/4064001084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 0x7f9a0c0b0480 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c064120 tx=0x7f99fc00b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15bbf173750c4c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.366+0000 7f9a0a7fc700 1 -- 172.21.15.31:0/4064001084 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00db70 con 0x7f9a0c153d40 2023-10-06T05:23:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.366+0000 7f9a0a7fc700 1 -- 172.21.15.31:0/4064001084 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc00dcd0 con 0x7f9a0c153d40 2023-10-06T05:23:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.366+0000 7f9a0a7fc700 1 -- 172.21.15.31:0/4064001084 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0196b0 con 0x7f9a0c153d40 2023-10-06T05:23:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.367+0000 7f9a121ff700 1 -- 172.21.15.31:0/4064001084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 msgr2=0x7f9a0c0b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.367+0000 7f9a121ff700 1 --2- 172.21.15.31:0/4064001084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 0x7f9a0c0b0480 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c064120 tx=0x7f99fc00b430 comp rx=0 tx=0).stop 2023-10-06T05:23:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.367+0000 7f9a121ff700 1 -- 172.21.15.31:0/4064001084 shutdown_connections 2023-10-06T05:23:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.367+0000 7f9a121ff700 1 --2- 172.21.15.31:0/4064001084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 0x7f9a0c0b0480 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.367+0000 7f9a121ff700 1 --2- 172.21.15.31:0/4064001084 >> v2:172.21.15.62:3300/0 conn(0x7f9a0c0b09c0 0x7f9a0c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.367+0000 7f9a121ff700 1 -- 172.21.15.31:0/4064001084 >> 172.21.15.31:0/4064001084 conn(0x7f9a0c067560 msgr2=0x7f9a0c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.367+0000 7f9a121ff700 1 -- 172.21.15.31:0/4064001084 shutdown_connections 2023-10-06T05:23:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.367+0000 7f9a121ff700 1 -- 172.21.15.31:0/4064001084 wait complete. 2023-10-06T05:23:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.369+0000 7f9a121ff700 1 Processor -- start 2023-10-06T05:23:32.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.369+0000 7f9a121ff700 1 -- start start 2023-10-06T05:23:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a121ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a0c0af6c0 0x7f9a0c166110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a121ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0b09c0 0x7f9a0c149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a121ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 0x7f9a0c150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a121ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c151a90 con 0x7f9a0c153d40 2023-10-06T05:23:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a121ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c14ea10 con 0x7f9a0c0af6c0 2023-10-06T05:23:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a121ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c14ecb0 con 0x7f9a0c0b09c0 2023-10-06T05:23:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 0x7f9a0c150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 0x7f9a0c150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57938/0 (socket says 172.21.15.31:57938) 2023-10-06T05:23:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0bfff700 1 -- 172.21.15.31:0/2502088506 learned_addr learned my addr 172.21.15.31:0/2502088506 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0b7fe700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a0c0af6c0 0x7f9a0c166110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0affd700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0b09c0 0x7f9a0c149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0bfff700 1 -- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0b09c0 msgr2=0x7f9a0c149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0bfff700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0b09c0 0x7f9a0c149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0bfff700 1 -- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a0c0af6c0 msgr2=0x7f9a0c166110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0bfff700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a0c0af6c0 0x7f9a0c166110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0bfff700 1 -- 172.21.15.31:0/2502088506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc00b6d0 con 0x7f9a0c153d40 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.371+0000 7f9a0affd700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0b09c0 0x7f9a0c149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.372+0000 7f9a0bfff700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 0x7f9a0c150830 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400ccc0 tx=0x7f9a04001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.372+0000 7f99f37fe700 1 -- 172.21.15.31:0/2502088506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400a930 con 0x7f9a0c153d40 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.372+0000 7f99f37fe700 1 -- 172.21.15.31:0/2502088506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0400aab0 con 0x7f9a0c153d40 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.372+0000 7f99f37fe700 1 -- 172.21.15.31:0/2502088506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400ae70 con 0x7f9a0c153d40 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.372+0000 7f9a121ff700 1 -- 172.21.15.31:0/2502088506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c166650 con 0x7f9a0c153d40 2023-10-06T05:23:32.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.372+0000 7f9a121ff700 1 -- 172.21.15.31:0/2502088506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c151270 con 0x7f9a0c153d40 2023-10-06T05:23:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.373+0000 7f9a121ff700 1 -- 172.21.15.31:0/2502088506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c14ee20 con 0x7f9a0c153d40 2023-10-06T05:23:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.374+0000 7f99f37fe700 1 -- 172.21.15.31:0/2502088506 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9a04007340 con 0x7f9a0c153d40 2023-10-06T05:23:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.375+0000 7f99f37fe700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f8067320 0x7f99f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.375+0000 7f99f37fe700 1 -- 172.21.15.31:0/2502088506 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9a04085110 con 0x7f9a0c153d40 2023-10-06T05:23:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.375+0000 7f9a0b7fe700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f8067320 0x7f99f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.376+0000 7f9a0b7fe700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f8067320 0x7f99f80697d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00b6a0 tx=0x7f99fc01b000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.379+0000 7f99f37fe700 1 -- 172.21.15.31:0/2502088506 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a04051c60 con 0x7f9a0c153d40 2023-10-06T05:23:32.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:32.632+0000 7f9a121ff700 1 -- 172.21.15.31:0/2502088506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9a0c0b0e40 con 0x7f9a0c153d40 2023-10-06T05:23:33.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.149+0000 7f99f37fe700 1 -- 172.21.15.31:0/2502088506 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v75) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9a0c0b0e40 con 0x7f9a0c153d40 2023-10-06T05:23:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.151+0000 7f9a121ff700 1 -- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f8067320 msgr2=0x7f99f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.151+0000 7f9a121ff700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f8067320 0x7f99f80697d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00b6a0 tx=0x7f99fc01b000 comp rx=0 tx=0).stop 2023-10-06T05:23:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.151+0000 7f9a121ff700 1 -- 172.21.15.31:0/2502088506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 msgr2=0x7f9a0c150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.151+0000 7f9a121ff700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 0x7f9a0c150830 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400ccc0 tx=0x7f9a04001a10 comp rx=0 tx=0).stop 2023-10-06T05:23:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.152+0000 7f9a121ff700 1 -- 172.21.15.31:0/2502088506 shutdown_connections 2023-10-06T05:23:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.152+0000 7f9a121ff700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f8067320 0x7f99f80697d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.152+0000 7f9a121ff700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c153d40 0x7f9a0c150830 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.152+0000 7f9a121ff700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a0c0af6c0 0x7f9a0c166110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.152+0000 7f9a121ff700 1 --2- 172.21.15.31:0/2502088506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0b09c0 0x7f9a0c149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.152+0000 7f9a121ff700 1 -- 172.21.15.31:0/2502088506 >> 172.21.15.31:0/2502088506 conn(0x7f9a0c067560 msgr2=0x7f9a0c06eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.152+0000 7f9a121ff700 1 -- 172.21.15.31:0/2502088506 shutdown_connections 2023-10-06T05:23:33.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.152+0000 7f9a121ff700 1 -- 172.21.15.31:0/2502088506 wait complete. 2023-10-06T05:23:33.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:33.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.370+0000 7f9f48dff700 1 Processor -- start 2023-10-06T05:23:33.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.370+0000 7f9f48dff700 1 -- start start 2023-10-06T05:23:33.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.371+0000 7f9f48dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f4413a1e0 0x7f9f440c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.371+0000 7f9f48dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f4406db50 0x7f9f44069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.371+0000 7f9f48dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f440c71b0 con 0x7f9f4413a5b0 2023-10-06T05:23:33.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.371+0000 7f9f48dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f440c7490 con 0x7f9f4413a1e0 2023-10-06T05:23:33.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.371+0000 7f9f48dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440c7770 con 0x7f9f4406db50 2023-10-06T05:23:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.371+0000 7f9f4259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f4413a1e0 0x7f9f440c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f41d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9f4413a5b0 0x7f9f4406d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33422/0 (socket says 172.21.15.31:33422) 2023-10-06T05:23:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f41d9b700 1 -- 172.21.15.31:0/2881049495 learned_addr learned my addr 172.21.15.31:0/2881049495 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f4259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f4413a1e0 0x7f9f440c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57954/0 (socket says 172.21.15.31:57954) 2023-10-06T05:23:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f42d9d700 1 --2- 172.21.15.31:0/2881049495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f4406db50 0x7f9f44069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f4259c700 1 -- 172.21.15.31:0/2881049495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f4406db50 msgr2=0x7f9f44069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f4259c700 1 --2- 172.21.15.31:0/2881049495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f4406db50 0x7f9f44069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f4259c700 1 -- 172.21.15.31:0/2881049495 >> v1:172.21.15.31:6789/0 conn(0x7f9f4413a5b0 legacy=0x7f9f4406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f4259c700 1 -- 172.21.15.31:0/2881049495 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f440c7a50 con 0x7f9f4413a1e0 2023-10-06T05:23:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f4159a700 1 -- 172.21.15.31:0/2881049495 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986513103 0 0) 0x7f9f440c71b0 con 0x7f9f4413a5b0 2023-10-06T05:23:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f4259c700 1 --2- 172.21.15.31:0/2881049495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 0x7f9f440c66f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f9f340098d0 tx=0x7f9f340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13447c3a1d5dec2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.372+0000 7f9f4159a700 1 -- 172.21.15.31:0/2881049495 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34009e70 con 0x7f9f4413a1e0 2023-10-06T05:23:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.373+0000 7f9f4159a700 1 -- 172.21.15.31:0/2881049495 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3400b860 con 0x7f9f4413a1e0 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.373+0000 7f9f4159a700 1 -- 172.21.15.31:0/2881049495 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3400bc00 con 0x7f9f4413a1e0 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.373+0000 7f9f48dff700 1 -- 172.21.15.31:0/2881049495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 msgr2=0x7f9f440c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.373+0000 7f9f48dff700 1 --2- 172.21.15.31:0/2881049495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 0x7f9f440c66f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f9f340098d0 tx=0x7f9f340091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.373+0000 7f9f48dff700 1 -- 172.21.15.31:0/2881049495 shutdown_connections 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.373+0000 7f9f48dff700 1 --2- 172.21.15.31:0/2881049495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 0x7f9f440c66f0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.373+0000 7f9f48dff700 1 --2- 172.21.15.31:0/2881049495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f4406db50 0x7f9f44069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.373+0000 7f9f48dff700 1 -- 172.21.15.31:0/2881049495 >> 172.21.15.31:0/2881049495 conn(0x7f9f440cc0b0 msgr2=0x7f9f440cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.373+0000 7f9f48dff700 1 -- 172.21.15.31:0/2881049495 shutdown_connections 2023-10-06T05:23:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.373+0000 7f9f48dff700 1 -- 172.21.15.31:0/2881049495 wait complete. 2023-10-06T05:23:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.375+0000 7f9f48dff700 1 Processor -- start 2023-10-06T05:23:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.375+0000 7f9f48dff700 1 -- start start 2023-10-06T05:23:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f48dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f4406db50 0x7f9f44168c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f48dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 0x7f9f440c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f48dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f4413a5b0 0x7f9f440c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f48dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44150a70 con 0x7f9f4413a1e0 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f48dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4413ad30 con 0x7f9f4413a5b0 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f48dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44150420 con 0x7f9f4406db50 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f41d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 0x7f9f440c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f41d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 0x7f9f440c32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57960/0 (socket says 172.21.15.31:57960) 2023-10-06T05:23:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f41d9b700 1 -- 172.21.15.31:0/2997201783 learned_addr learned my addr 172.21.15.31:0/2997201783 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f4259c700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f4406db50 0x7f9f44168c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.376+0000 7f9f42d9d700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f4413a5b0 0x7f9f440c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f41d9b700 1 -- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f4406db50 msgr2=0x7f9f44168c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f41d9b700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f4406db50 0x7f9f44168c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f41d9b700 1 -- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f4413a5b0 msgr2=0x7f9f440c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f41d9b700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f4413a5b0 0x7f9f440c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f41d9b700 1 -- 172.21.15.31:0/2997201783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f340093c0 con 0x7f9f4413a1e0 2023-10-06T05:23:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f41d9b700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 0x7f9f440c32c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f9f3800c5e0 tx=0x7f9f38000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f327fc700 1 -- 172.21.15.31:0/2997201783 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38005b30 con 0x7f9f4413a1e0 2023-10-06T05:23:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f327fc700 1 -- 172.21.15.31:0/2997201783 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f38005c90 con 0x7f9f4413a1e0 2023-10-06T05:23:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f327fc700 1 -- 172.21.15.31:0/2997201783 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38007300 con 0x7f9f4413a1e0 2023-10-06T05:23:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f48dff700 1 -- 172.21.15.31:0/2997201783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f44150700 con 0x7f9f4413a1e0 2023-10-06T05:23:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.377+0000 7f9f48dff700 1 -- 172.21.15.31:0/2997201783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f441691c0 con 0x7f9f4413a1e0 2023-10-06T05:23:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.378+0000 7f9f48dff700 1 -- 172.21.15.31:0/2997201783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f44150250 con 0x7f9f4413a1e0 2023-10-06T05:23:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.379+0000 7f9f327fc700 1 -- 172.21.15.31:0/2997201783 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9f380076b0 con 0x7f9f4413a1e0 2023-10-06T05:23:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.380+0000 7f9f327fc700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9f24067320 0x7f9f240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.380+0000 7f9f327fc700 1 -- 172.21.15.31:0/2997201783 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9f38084370 con 0x7f9f4413a1e0 2023-10-06T05:23:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.381+0000 7f9f4259c700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9f24067320 0x7f9f240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.381+0000 7f9f4259c700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9f24067320 0x7f9f240697d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9f340091e0 tx=0x7f9f34017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.384+0000 7f9f327fc700 1 -- 172.21.15.31:0/2997201783 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f38050e40 con 0x7f9f4413a1e0 2023-10-06T05:23:33.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.629+0000 7f9f48dff700 1 -- 172.21.15.31:0/2997201783 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f4404c380 con 0x7f9f4413a1e0 2023-10-06T05:23:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.629+0000 7f9f327fc700 1 -- 172.21.15.31:0/2997201783 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4593 (secure 0 0 0) 0x7f9f38054460 con 0x7f9f4413a1e0 2023-10-06T05:23:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:33.631 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":75,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6557,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1866061469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1866061469},{"type":"v1","addr":"172.21.15.31:6834","nonce":1866061469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7366,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/257113943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":257113943},{"type":"v1","addr":"172.21.15.31:6835","nonce":257113943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2082524951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2082524951},{"type":"v1","addr":"172.21.15.62:6834","nonce":2082524951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:29.615356+0000","modified":"2023-10-06T05:23:33.142272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7378":{"gid":7378,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/519548465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":519548465},{"type":"v1","addr":"172.21.15.62:6835","nonce":519548465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:23:33.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.631+0000 7f9f48dff700 1 -- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9f24067320 msgr2=0x7f9f240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9f24067320 0x7f9f240697d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9f340091e0 tx=0x7f9f34017000 comp rx=0 tx=0).stop 2023-10-06T05:23:33.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 -- 172.21.15.31:0/2997201783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 msgr2=0x7f9f440c32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 0x7f9f440c32c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f9f3800c5e0 tx=0x7f9f38000fe0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 -- 172.21.15.31:0/2997201783 shutdown_connections 2023-10-06T05:23:33.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9f24067320 0x7f9f240697d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f4413a5b0 0x7f9f440c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f4406db50 0x7f9f44168c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 --2- 172.21.15.31:0/2997201783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f4413a1e0 0x7f9f440c32c0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 -- 172.21.15.31:0/2997201783 >> 172.21.15.31:0/2997201783 conn(0x7f9f440cc0b0 msgr2=0x7f9f440b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 -- 172.21.15.31:0/2997201783 shutdown_connections 2023-10-06T05:23:33.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.632+0000 7f9f48dff700 1 -- 172.21.15.31:0/2997201783 wait complete. 2023-10-06T05:23:33.635 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 75 2023-10-06T05:23:33.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:33.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.844+0000 7fa330dff700 1 Processor -- start 2023-10-06T05:23:33.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.844+0000 7fa330dff700 1 -- start start 2023-10-06T05:23:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.845+0000 7fa330dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa32c1595a0 0x7fa32c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.845+0000 7fa330dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c0cc0d0 0x7fa32c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.845+0000 7fa330dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c069010 con 0x7fa32c068bb0 2023-10-06T05:23:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.845+0000 7fa330dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c0c95b0 con 0x7fa32c1595a0 2023-10-06T05:23:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.845+0000 7fa330dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c9890 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.846+0000 7fa329d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c0cc0d0 0x7fa32c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.846+0000 7fa329d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c0cc0d0 0x7fa32c068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:51560/0 (socket says 172.21.15.31:51560) 2023-10-06T05:23:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.846+0000 7fa329d9b700 1 -- 172.21.15.31:0/61345138 learned_addr learned my addr 172.21.15.31:0/61345138 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.846+0000 7fa32a59c700 1 --2- 172.21.15.31:0/61345138 >> v2:172.21.15.62:3300/0 conn(0x7fa32c1595a0 0x7fa32c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.846+0000 7fa329d9b700 1 -- 172.21.15.31:0/61345138 >> v2:172.21.15.62:3300/0 conn(0x7fa32c1595a0 msgr2=0x7fa32c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.846+0000 7fa329d9b700 1 --2- 172.21.15.31:0/61345138 >> v2:172.21.15.62:3300/0 conn(0x7fa32c1595a0 0x7fa32c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.846+0000 7fa329d9b700 1 -- 172.21.15.31:0/61345138 >> v1:172.21.15.62:6789/0 conn(0x7fa32c068bb0 legacy=0x7fa32c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.846+0000 7fa329d9b700 1 -- 172.21.15.31:0/61345138 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c0c9b70 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.847+0000 7fa329d9b700 1 --2- 172.21.15.31:0/61345138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c0cc0d0 0x7fa32c068670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa3200099b0 tx=0x7fa320009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3408189b31f285 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.847+0000 7fa32959a700 1 -- 172.21.15.31:0/61345138 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320009ea0 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.847+0000 7fa32959a700 1 -- 172.21.15.31:0/61345138 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32000b860 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.847+0000 7fa32959a700 1 -- 172.21.15.31:0/61345138 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32000bc00 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.847+0000 7fa330dff700 1 -- 172.21.15.31:0/61345138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c0cc0d0 msgr2=0x7fa32c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.847+0000 7fa330dff700 1 --2- 172.21.15.31:0/61345138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c0cc0d0 0x7fa32c068670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa3200099b0 tx=0x7fa320009250 comp rx=0 tx=0).stop 2023-10-06T05:23:33.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.847+0000 7fa330dff700 1 -- 172.21.15.31:0/61345138 shutdown_connections 2023-10-06T05:23:33.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.847+0000 7fa330dff700 1 --2- 172.21.15.31:0/61345138 >> v2:172.21.15.62:3300/0 conn(0x7fa32c1595a0 0x7fa32c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.847+0000 7fa330dff700 1 --2- 172.21.15.31:0/61345138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c0cc0d0 0x7fa32c068670 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.847+0000 7fa330dff700 1 -- 172.21.15.31:0/61345138 >> 172.21.15.31:0/61345138 conn(0x7fa32c06b620 msgr2=0x7fa32c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:33.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.848+0000 7fa330dff700 1 -- 172.21.15.31:0/61345138 shutdown_connections 2023-10-06T05:23:33.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.848+0000 7fa330dff700 1 -- 172.21.15.31:0/61345138 wait complete. 2023-10-06T05:23:33.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.849+0000 7fa330dff700 1 Processor -- start 2023-10-06T05:23:33.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.849+0000 7fa330dff700 1 -- start start 2023-10-06T05:23:33.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.850+0000 7fa330dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c068bb0 0x7fa32c1588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.850+0000 7fa330dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0cc0d0 0x7fa32c15ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.850+0000 7fa330dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa32c1595a0 0x7fa32c0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.850+0000 7fa330dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c15f160 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.850+0000 7fa330dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c15e790 con 0x7fa32c1595a0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.850+0000 7fa330dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c15db80 con 0x7fa32c068bb0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.850+0000 7fa329d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0cc0d0 0x7fa32c15ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.850+0000 7fa329d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0cc0d0 0x7fa32c15ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57964/0 (socket says 172.21.15.31:57964) 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.850+0000 7fa329d9b700 1 -- 172.21.15.31:0/2641312214 learned_addr learned my addr 172.21.15.31:0/2641312214 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa32a59c700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c068bb0 0x7fa32c1588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa329d9b700 1 -- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c068bb0 msgr2=0x7fa32c1588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa32ad9d700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa32c1595a0 0x7fa32c0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa329d9b700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c068bb0 0x7fa32c1588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa329d9b700 1 -- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa32c1595a0 msgr2=0x7fa32c0b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa329d9b700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa32c1595a0 0x7fa32c0b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:33.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa329d9b700 1 -- 172.21.15.31:0/2641312214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3200094f0 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa32a59c700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c068bb0 0x7fa32c1588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa32ad9d700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa32c1595a0 0x7fa32c0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa329d9b700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0cc0d0 0x7fa32c15ec20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0c99d0 tx=0x7fa320017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa31a7fc700 1 -- 172.21.15.31:0/2641312214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32000bbe0 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa31a7fc700 1 -- 172.21.15.31:0/2641312214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320011420 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa330dff700 1 -- 172.21.15.31:0/2641312214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c15de60 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.851+0000 7fa31a7fc700 1 -- 172.21.15.31:0/2641312214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200117c0 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.852+0000 7fa330dff700 1 -- 172.21.15.31:0/2641312214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c15e420 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.853+0000 7fa330dff700 1 -- 172.21.15.31:0/2641312214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c04c380 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.853+0000 7fa31a7fc700 1 -- 172.21.15.31:0/2641312214 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa320011bf0 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.854+0000 7fa31a7fc700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa30c067320 0x7fa30c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.855+0000 7fa31a7fc700 1 -- 172.21.15.31:0/2641312214 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fa3200867e0 con 0x7fa32c0cc0d0 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.855+0000 7fa32a59c700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa30c067320 0x7fa30c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:33.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.856+0000 7fa32a59c700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa30c067320 0x7fa30c0697d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006f00 tx=0x7fa31c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:33.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:33.858+0000 7fa31a7fc700 1 -- 172.21.15.31:0/2641312214 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3200532b0 con 0x7fa32c0cc0d0 2023-10-06T05:23:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.062+0000 7fa330dff700 1 -- 172.21.15.31:0/2641312214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa32c0cd8a0 con 0x7fa32c0cc0d0 2023-10-06T05:23:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.063+0000 7fa31a7fc700 1 -- 172.21.15.31:0/2641312214 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16972 (secure 0 0 0) 0x7fa3200568d0 con 0x7fa32c0cc0d0 2023-10-06T05:23:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:34.065 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":114,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:23:31.123716+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:27.942469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:28.591817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.065+0000 7fa330dff700 1 -- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa30c067320 msgr2=0x7fa30c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.065+0000 7fa330dff700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa30c067320 0x7fa30c0697d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006f00 tx=0x7fa31c008040 comp rx=0 tx=0).stop 2023-10-06T05:23:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.065+0000 7fa330dff700 1 -- 172.21.15.31:0/2641312214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0cc0d0 msgr2=0x7fa32c15ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.065+0000 7fa330dff700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0cc0d0 0x7fa32c15ec20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0c99d0 tx=0x7fa320017000 comp rx=0 tx=0).stop 2023-10-06T05:23:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.065+0000 7fa330dff700 1 -- 172.21.15.31:0/2641312214 shutdown_connections 2023-10-06T05:23:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.065+0000 7fa330dff700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa30c067320 0x7fa30c0697d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.065+0000 7fa330dff700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa32c1595a0 0x7fa32c0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.065+0000 7fa330dff700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa32c068bb0 0x7fa32c1588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.065+0000 7fa330dff700 1 --2- 172.21.15.31:0/2641312214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0cc0d0 0x7fa32c15ec20 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.065+0000 7fa330dff700 1 -- 172.21.15.31:0/2641312214 >> 172.21.15.31:0/2641312214 conn(0x7fa32c06b620 msgr2=0x7fa32c15b340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.066+0000 7fa330dff700 1 -- 172.21.15.31:0/2641312214 shutdown_connections 2023-10-06T05:23:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.066+0000 7fa330dff700 1 -- 172.21.15.31:0/2641312214 wait complete. 2023-10-06T05:23:34.085 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:23:34.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:23:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.282+0000 7f68d7dff700 1 Processor -- start 2023-10-06T05:23:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.282+0000 7f68d7dff700 1 -- start start 2023-10-06T05:23:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.283+0000 7f68d7dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f68d00cd510 0x7f68d00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d7dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68d015d2d0 0x7f68d00aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d7dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d00c3f40 con 0x7f68d0159500 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d7dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d00b2860 con 0x7f68d015d2d0 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d7dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d015d7d0 con 0x7f68d00cd510 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68d015d2d0 0x7f68d00aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d639c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68d015d2d0 0x7f68d00aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57966/0 (socket says 172.21.15.31:57966) 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d5b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f68d0159500 0x7f68d00cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33438/0 (socket says 172.21.15.31:33438) 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d639c700 1 -- 172.21.15.31:0/2922415301 learned_addr learned my addr 172.21.15.31:0/2922415301 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d539a700 1 --2- 172.21.15.31:0/2922415301 >> v2:172.21.15.62:3300/0 conn(0x7f68d00cd510 0x7f68d00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d639c700 1 -- 172.21.15.31:0/2922415301 >> v2:172.21.15.62:3300/0 conn(0x7f68d00cd510 msgr2=0x7f68d00b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d639c700 1 --2- 172.21.15.31:0/2922415301 >> v2:172.21.15.62:3300/0 conn(0x7f68d00cd510 0x7f68d00b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d639c700 1 -- 172.21.15.31:0/2922415301 >> v1:172.21.15.31:6789/0 conn(0x7f68d0159500 legacy=0x7f68d00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d639c700 1 -- 172.21.15.31:0/2922415301 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d00cb000 con 0x7f68d015d2d0 2023-10-06T05:23:34.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.284+0000 7f68d4b99700 1 -- 172.21.15.31:0/2922415301 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383639482 0 0) 0x7f68d00c3f40 con 0x7f68d0159500 2023-10-06T05:23:34.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.285+0000 7f68d639c700 1 --2- 172.21.15.31:0/2922415301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d015d2d0 0x7f68d00aedd0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f68d015d230 tx=0x7f68c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6b37fcc8a5590c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.285+0000 7f68d4b99700 1 -- 172.21.15.31:0/2922415301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c800e070 con 0x7f68d015d2d0 2023-10-06T05:23:34.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.285+0000 7f68d4b99700 1 -- 172.21.15.31:0/2922415301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c800ba20 con 0x7f68d015d2d0 2023-10-06T05:23:34.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.285+0000 7f68d4b99700 1 -- 172.21.15.31:0/2922415301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c80092e0 con 0x7f68d015d2d0 2023-10-06T05:23:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.285+0000 7f68d7dff700 1 -- 172.21.15.31:0/2922415301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d015d2d0 msgr2=0x7f68d00aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.285+0000 7f68d7dff700 1 --2- 172.21.15.31:0/2922415301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d015d2d0 0x7f68d00aedd0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f68d015d230 tx=0x7f68c8009840 comp rx=0 tx=0).stop 2023-10-06T05:23:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.285+0000 7f68d7dff700 1 -- 172.21.15.31:0/2922415301 shutdown_connections 2023-10-06T05:23:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.285+0000 7f68d7dff700 1 --2- 172.21.15.31:0/2922415301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d015d2d0 0x7f68d00aedd0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.285+0000 7f68d7dff700 1 --2- 172.21.15.31:0/2922415301 >> v2:172.21.15.62:3300/0 conn(0x7f68d00cd510 0x7f68d00b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.285+0000 7f68d7dff700 1 -- 172.21.15.31:0/2922415301 >> 172.21.15.31:0/2922415301 conn(0x7f68d006ec90 msgr2=0x7f68d006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.286+0000 7f68d7dff700 1 -- 172.21.15.31:0/2922415301 shutdown_connections 2023-10-06T05:23:34.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.286+0000 7f68d7dff700 1 -- 172.21.15.31:0/2922415301 wait complete. 2023-10-06T05:23:34.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.287+0000 7f68d7dff700 1 Processor -- start 2023-10-06T05:23:34.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.287+0000 7f68d7dff700 1 -- start start 2023-10-06T05:23:34.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.288+0000 7f68d7dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d00cd510 0x7f68d01575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.288+0000 7f68d7dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68d0159500 0x7f68d00b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.288+0000 7f68d7dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d015d2d0 0x7f68d00b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.288+0000 7f68d7dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d0157ae0 con 0x7f68d00cd510 2023-10-06T05:23:34.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.288+0000 7f68d7dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d015fa90 con 0x7f68d015d2d0 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.288+0000 7f68d7dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d015f160 con 0x7f68d0159500 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.288+0000 7f68d5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d00cd510 0x7f68d01575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.288+0000 7f68d5b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d00cd510 0x7f68d01575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57972/0 (socket says 172.21.15.31:57972) 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.288+0000 7f68d5b9b700 1 -- 172.21.15.31:0/3888153921 learned_addr learned my addr 172.21.15.31:0/3888153921 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d639c700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d015d2d0 0x7f68d00b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d539a700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68d0159500 0x7f68d00b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d5b9b700 1 -- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68d0159500 msgr2=0x7f68d00b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d5b9b700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68d0159500 0x7f68d00b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d5b9b700 1 -- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d015d2d0 msgr2=0x7f68d00b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d5b9b700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d015d2d0 0x7f68d00b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d5b9b700 1 -- 172.21.15.31:0/3888153921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c8009ae0 con 0x7f68d00cd510 2023-10-06T05:23:34.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d539a700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68d0159500 0x7f68d00b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:34.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d639c700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d015d2d0 0x7f68d00b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:34.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d5b9b700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d00cd510 0x7f68d01575a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0011e0 tx=0x7f68cc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68c5ffb700 1 -- 172.21.15.31:0/3888153921 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc007300 con 0x7f68d00cd510 2023-10-06T05:23:34.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68c5ffb700 1 -- 172.21.15.31:0/3888153921 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc007460 con 0x7f68d00cd510 2023-10-06T05:23:34.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68c5ffb700 1 -- 172.21.15.31:0/3888153921 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc004830 con 0x7f68d00cd510 2023-10-06T05:23:34.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.289+0000 7f68d7dff700 1 -- 172.21.15.31:0/3888153921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d015f440 con 0x7f68d00cd510 2023-10-06T05:23:34.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.290+0000 7f68d7dff700 1 -- 172.21.15.31:0/3888153921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d00c88e0 con 0x7f68d00cd510 2023-10-06T05:23:34.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.291+0000 7f68d7dff700 1 -- 172.21.15.31:0/3888153921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d015fd70 con 0x7f68d00cd510 2023-10-06T05:23:34.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.291+0000 7f68c5ffb700 1 -- 172.21.15.31:0/3888153921 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f68cc004be0 con 0x7f68d00cd510 2023-10-06T05:23:34.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.292+0000 7f68c5ffb700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f68b4067350 0x7f68b4069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.292+0000 7f68c5ffb700 1 -- 172.21.15.31:0/3888153921 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f68cc084330 con 0x7f68d00cd510 2023-10-06T05:23:34.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.293+0000 7f68d539a700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f68b4067350 0x7f68b4069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.294+0000 7f68d539a700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f68b4067350 0x7f68b4069800 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009390 tx=0x7f68c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.296+0000 7f68c5ffb700 1 -- 172.21.15.31:0/3888153921 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68cc050e80 con 0x7f68d00cd510 2023-10-06T05:23:34.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.493+0000 7f68d7dff700 1 -- 172.21.15.31:0/3888153921 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f68d015fbd0 con 0x7f68b4067350 2023-10-06T05:23:34.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.504+0000 7f68c5ffb700 1 -- 172.21.15.31:0/3888153921 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311973 (secure 0 0 0) 0x7f68c00099a0 con 0x7f68b4067350 2023-10-06T05:23:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:34.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.508+0000 7f68d7dff700 1 -- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f68b4067350 msgr2=0x7f68b4069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.508+0000 7f68d7dff700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f68b4067350 0x7f68b4069800 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009390 tx=0x7f68c0008040 comp rx=0 tx=0).stop 2023-10-06T05:23:34.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.508+0000 7f68d7dff700 1 -- 172.21.15.31:0/3888153921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d00cd510 msgr2=0x7f68d01575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.508+0000 7f68d7dff700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d00cd510 0x7f68d01575a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0011e0 tx=0x7f68cc007e10 comp rx=0 tx=0).stop 2023-10-06T05:23:34.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.508+0000 7f68d7dff700 1 -- 172.21.15.31:0/3888153921 shutdown_connections 2023-10-06T05:23:34.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.508+0000 7f68d7dff700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f68b4067350 0x7f68b4069800 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.508+0000 7f68d7dff700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d015d2d0 0x7f68d00b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.508+0000 7f68d7dff700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68d00cd510 0x7f68d01575a0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.509+0000 7f68d7dff700 1 --2- 172.21.15.31:0/3888153921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68d0159500 0x7f68d00b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.509+0000 7f68d7dff700 1 -- 172.21.15.31:0/3888153921 >> 172.21.15.31:0/3888153921 conn(0x7f68d006ec90 msgr2=0x7f68d00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.509+0000 7f68d7dff700 1 -- 172.21.15.31:0/3888153921 shutdown_connections 2023-10-06T05:23:34.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.509+0000 7f68d7dff700 1 -- 172.21.15.31:0/3888153921 wait complete. 2023-10-06T05:23:34.515 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:23:34.528 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2023-10-06T05:23:33.487924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411048,"kb_used_data":12264,"kb_used_omap":0,"kb_used_meta":2398720,"kb_avail":752563672,"statfs":{"total":773094113280,"available":770625200128,"internally_reserved":0,"allocated":12558336,"data_stored":6423028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2456289280},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.381359"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141119+0000","last_change":"2023-10-06T05:23:30.133477+0000","last_active":"2023-10-06T05:23:30.141119+0000","last_peered":"2023-10-06T05:23:30.141119+0000","last_clean":"2023-10-06T05:23:30.141119+0000","last_became_active":"2023-10-06T05:23:30.133198+0000","last_became_peered":"2023-10-06T05:23:30.133198+0000","last_unstale":"2023-10-06T05:23:30.141119+0000","last_undegraded":"2023-10-06T05:23:30.141119+0000","last_fullsized":"2023-10-06T05:23:30.141119+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:51.630441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141815+0000","last_change":"2023-10-06T05:23:30.141815+0000","last_active":"2023-10-06T05:23:30.141815+0000","last_peered":"2023-10-06T05:23:30.141815+0000","last_clean":"2023-10-06T05:23:30.141815+0000","last_became_active":"2023-10-06T05:23:30.141545+0000","last_became_peered":"2023-10-06T05:23:30.141545+0000","last_unstale":"2023-10-06T05:23:30.141815+0000","last_undegraded":"2023-10-06T05:23:30.141815+0000","last_fullsized":"2023-10-06T05:23:30.141815+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:08.653017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.137550+0000","last_change":"2023-10-06T05:23:30.137550+0000","last_active":"2023-10-06T05:23:30.137550+0000","last_peered":"2023-10-06T05:23:30.137550+0000","last_clean":"2023-10-06T05:23:30.137550+0000","last_became_active":"2023-10-06T05:23:30.136469+0000","last_became_peered":"2023-10-06T05:23:30.136469+0000","last_unstale":"2023-10-06T05:23:30.137550+0000","last_undegraded":"2023-10-06T05:23:30.137550+0000","last_fullsized":"2023-10-06T05:23:30.137550+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:35.784601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.135707+0000","last_change":"2023-10-06T05:23:30.135707+0000","last_active":"2023-10-06T05:23:30.135707+0000","last_peered":"2023-10-06T05:23:30.135707+0000","last_clean":"2023-10-06T05:23:30.135707+0000","last_became_active":"2023-10-06T05:23:30.135404+0000","last_became_peered":"2023-10-06T05:23:30.135404+0000","last_unstale":"2023-10-06T05:23:30.135707+0000","last_undegraded":"2023-10-06T05:23:30.135707+0000","last_fullsized":"2023-10-06T05:23:30.135707+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:25.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.137630+0000","last_change":"2023-10-06T05:23:30.129884+0000","last_active":"2023-10-06T05:23:30.137630+0000","last_peered":"2023-10-06T05:23:30.137630+0000","last_clean":"2023-10-06T05:23:30.137630+0000","last_became_active":"2023-10-06T05:23:30.129642+0000","last_became_peered":"2023-10-06T05:23:30.129642+0000","last_unstale":"2023-10-06T05:23:30.137630+0000","last_undegraded":"2023-10-06T05:23:30.137630+0000","last_fullsized":"2023-10-06T05:23:30.137630+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:05.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.149429+0000","last_change":"2023-10-06T05:23:30.149429+0000","last_active":"2023-10-06T05:23:30.149429+0000","last_peered":"2023-10-06T05:23:30.149429+0000","last_clean":"2023-10-06T05:23:30.149429+0000","last_became_active":"2023-10-06T05:23:30.149120+0000","last_became_peered":"2023-10-06T05:23:30.149120+0000","last_unstale":"2023-10-06T05:23:30.149429+0000","last_undegraded":"2023-10-06T05:23:30.149429+0000","last_fullsized":"2023-10-06T05:23:30.149429+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:34.159767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.156776+0000","last_change":"2023-10-06T05:23:30.156776+0000","last_active":"2023-10-06T05:23:30.156776+0000","last_peered":"2023-10-06T05:23:30.156776+0000","last_clean":"2023-10-06T05:23:30.156776+0000","last_became_active":"2023-10-06T05:23:30.156487+0000","last_became_peered":"2023-10-06T05:23:30.156487+0000","last_unstale":"2023-10-06T05:23:30.156776+0000","last_undegraded":"2023-10-06T05:23:30.156776+0000","last_fullsized":"2023-10-06T05:23:30.156776+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:36.745933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140258+0000","last_change":"2023-10-06T05:23:30.133694+0000","last_active":"2023-10-06T05:23:30.140258+0000","last_peered":"2023-10-06T05:23:30.140258+0000","last_clean":"2023-10-06T05:23:30.140258+0000","last_became_active":"2023-10-06T05:23:30.133360+0000","last_became_peered":"2023-10-06T05:23:30.133360+0000","last_unstale":"2023-10-06T05:23:30.140258+0000","last_undegraded":"2023-10-06T05:23:30.140258+0000","last_fullsized":"2023-10-06T05:23:30.140258+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:08.653017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.136027+0000","last_change":"2023-10-06T05:23:30.136027+0000","last_active":"2023-10-06T05:23:30.136027+0000","last_peered":"2023-10-06T05:23:30.136027+0000","last_clean":"2023-10-06T05:23:30.136027+0000","last_became_active":"2023-10-06T05:23:30.135739+0000","last_became_peered":"2023-10-06T05:23:30.135739+0000","last_unstale":"2023-10-06T05:23:30.136027+0000","last_undegraded":"2023-10-06T05:23:30.136027+0000","last_fullsized":"2023-10-06T05:23:30.136027+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:33.248937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.134129+0000","last_change":"2023-10-06T05:23:30.134129+0000","last_active":"2023-10-06T05:23:30.134129+0000","last_peered":"2023-10-06T05:23:30.134129+0000","last_clean":"2023-10-06T05:23:30.134129+0000","last_became_active":"2023-10-06T05:23:30.133818+0000","last_became_peered":"2023-10-06T05:23:30.133818+0000","last_unstale":"2023-10-06T05:23:30.134129+0000","last_undegraded":"2023-10-06T05:23:30.134129+0000","last_fullsized":"2023-10-06T05:23:30.134129+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:33.243604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.150817+0000","last_change":"2023-10-06T05:23:30.150817+0000","last_active":"2023-10-06T05:23:30.150817+0000","last_peered":"2023-10-06T05:23:30.150817+0000","last_clean":"2023-10-06T05:23:30.150817+0000","last_became_active":"2023-10-06T05:23:30.150532+0000","last_became_peered":"2023-10-06T05:23:30.150532+0000","last_unstale":"2023-10-06T05:23:30.150817+0000","last_undegraded":"2023-10-06T05:23:30.150817+0000","last_fullsized":"2023-10-06T05:23:30.150817+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:43.098998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141285+0000","last_change":"2023-10-06T05:23:30.133408+0000","last_active":"2023-10-06T05:23:30.141285+0000","last_peered":"2023-10-06T05:23:30.141285+0000","last_clean":"2023-10-06T05:23:30.141285+0000","last_became_active":"2023-10-06T05:23:30.133119+0000","last_became_peered":"2023-10-06T05:23:30.133119+0000","last_unstale":"2023-10-06T05:23:30.141285+0000","last_undegraded":"2023-10-06T05:23:30.141285+0000","last_fullsized":"2023-10-06T05:23:30.141285+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:23.188223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.149498+0000","last_change":"2023-10-06T05:23:30.149498+0000","last_active":"2023-10-06T05:23:30.149498+0000","last_peered":"2023-10-06T05:23:30.149498+0000","last_clean":"2023-10-06T05:23:30.149498+0000","last_became_active":"2023-10-06T05:23:30.149129+0000","last_became_peered":"2023-10-06T05:23:30.149129+0000","last_unstale":"2023-10-06T05:23:30.149498+0000","last_undegraded":"2023-10-06T05:23:30.149498+0000","last_fullsized":"2023-10-06T05:23:30.149498+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:08.653017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.143999+0000","last_change":"2023-10-06T05:23:30.140473+0000","last_active":"2023-10-06T05:23:30.143999+0000","last_peered":"2023-10-06T05:23:30.143999+0000","last_clean":"2023-10-06T05:23:30.143999+0000","last_became_active":"2023-10-06T05:23:30.140096+0000","last_became_peered":"2023-10-06T05:23:30.140096+0000","last_unstale":"2023-10-06T05:23:30.143999+0000","last_undegraded":"2023-10-06T05:23:30.143999+0000","last_fullsized":"2023-10-06T05:23:30.143999+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:34.207064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141734+0000","last_change":"2023-10-06T05:23:30.140000+0000","last_active":"2023-10-06T05:23:30.141734+0000","last_peered":"2023-10-06T05:23:30.141734+0000","last_clean":"2023-10-06T05:23:30.141734+0000","last_became_active":"2023-10-06T05:23:30.139712+0000","last_became_peered":"2023-10-06T05:23:30.139712+0000","last_unstale":"2023-10-06T05:23:30.141734+0000","last_undegraded":"2023-10-06T05:23:30.141734+0000","last_fullsized":"2023-10-06T05:23:30.141734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:35.597987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.130091+0000","last_change":"2023-10-06T05:23:30.130091+0000","last_active":"2023-10-06T05:23:30.130091+0000","last_peered":"2023-10-06T05:23:30.130091+0000","last_clean":"2023-10-06T05:23:30.130091+0000","last_became_active":"2023-10-06T05:23:30.129771+0000","last_became_peered":"2023-10-06T05:23:30.129771+0000","last_unstale":"2023-10-06T05:23:30.130091+0000","last_undegraded":"2023-10-06T05:23:30.130091+0000","last_fullsized":"2023-10-06T05:23:30.130091+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:38.581343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.129886+0000","last_change":"2023-10-06T05:23:30.129886+0000","last_active":"2023-10-06T05:23:30.129886+0000","last_peered":"2023-10-06T05:23:30.129886+0000","last_clean":"2023-10-06T05:23:30.129886+0000","last_became_active":"2023-10-06T05:23:30.129611+0000","last_became_peered":"2023-10-06T05:23:30.129611+0000","last_unstale":"2023-10-06T05:23:30.129886+0000","last_undegraded":"2023-10-06T05:23:30.129886+0000","last_fullsized":"2023-10-06T05:23:30.129886+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:59.661176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.137688+0000","last_change":"2023-10-06T05:23:30.130802+0000","last_active":"2023-10-06T05:23:30.137688+0000","last_peered":"2023-10-06T05:23:30.137688+0000","last_clean":"2023-10-06T05:23:30.137688+0000","last_became_active":"2023-10-06T05:23:30.130540+0000","last_became_peered":"2023-10-06T05:23:30.130540+0000","last_unstale":"2023-10-06T05:23:30.137688+0000","last_undegraded":"2023-10-06T05:23:30.137688+0000","last_fullsized":"2023-10-06T05:23:30.137688+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:55.292129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.136977+0000","last_change":"2023-10-06T05:23:30.136977+0000","last_active":"2023-10-06T05:23:30.136977+0000","last_peered":"2023-10-06T05:23:30.136977+0000","last_clean":"2023-10-06T05:23:30.136977+0000","last_became_active":"2023-10-06T05:23:30.136698+0000","last_became_peered":"2023-10-06T05:23:30.136698+0000","last_unstale":"2023-10-06T05:23:30.136977+0000","last_undegraded":"2023-10-06T05:23:30.136977+0000","last_fullsized":"2023-10-06T05:23:30.136977+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:39.944519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.154543+0000","last_change":"2023-10-06T05:23:30.137942+0000","last_active":"2023-10-06T05:23:30.154543+0000","last_peered":"2023-10-06T05:23:30.154543+0000","last_clean":"2023-10-06T05:23:30.154543+0000","last_became_active":"2023-10-06T05:23:30.137653+0000","last_became_peered":"2023-10-06T05:23:30.137653+0000","last_unstale":"2023-10-06T05:23:30.154543+0000","last_undegraded":"2023-10-06T05:23:30.154543+0000","last_fullsized":"2023-10-06T05:23:30.154543+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:56.407467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.134266+0000","last_change":"2023-10-06T05:23:30.134266+0000","last_active":"2023-10-06T05:23:30.134266+0000","last_peered":"2023-10-06T05:23:30.134266+0000","last_clean":"2023-10-06T05:23:30.134266+0000","last_became_active":"2023-10-06T05:23:30.133962+0000","last_became_peered":"2023-10-06T05:23:30.133962+0000","last_unstale":"2023-10-06T05:23:30.134266+0000","last_undegraded":"2023-10-06T05:23:30.134266+0000","last_fullsized":"2023-10-06T05:23:30.134266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.065283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141725+0000","last_change":"2023-10-06T05:23:30.136345+0000","last_active":"2023-10-06T05:23:30.141725+0000","last_peered":"2023-10-06T05:23:30.141725+0000","last_clean":"2023-10-06T05:23:30.141725+0000","last_became_active":"2023-10-06T05:23:30.136082+0000","last_became_peered":"2023-10-06T05:23:30.136082+0000","last_unstale":"2023-10-06T05:23:30.141725+0000","last_undegraded":"2023-10-06T05:23:30.141725+0000","last_fullsized":"2023-10-06T05:23:30.141725+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:11.703042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140811+0000","last_change":"2023-10-06T05:23:30.140811+0000","last_active":"2023-10-06T05:23:30.140811+0000","last_peered":"2023-10-06T05:23:30.140811+0000","last_clean":"2023-10-06T05:23:30.140811+0000","last_became_active":"2023-10-06T05:23:30.140501+0000","last_became_peered":"2023-10-06T05:23:30.140501+0000","last_unstale":"2023-10-06T05:23:30.140811+0000","last_undegraded":"2023-10-06T05:23:30.140811+0000","last_fullsized":"2023-10-06T05:23:30.140811+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:34.159767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.137003+0000","last_change":"2023-10-06T05:23:30.137003+0000","last_active":"2023-10-06T05:23:30.137003+0000","last_peered":"2023-10-06T05:23:30.137003+0000","last_clean":"2023-10-06T05:23:30.137003+0000","last_became_active":"2023-10-06T05:23:30.136746+0000","last_became_peered":"2023-10-06T05:23:30.136746+0000","last_unstale":"2023-10-06T05:23:30.137003+0000","last_undegraded":"2023-10-06T05:23:30.137003+0000","last_fullsized":"2023-10-06T05:23:30.137003+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:05.514516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.142165+0000","last_change":"2023-10-06T05:23:30.142165+0000","last_active":"2023-10-06T05:23:30.142165+0000","last_peered":"2023-10-06T05:23:30.142165+0000","last_clean":"2023-10-06T05:23:30.142165+0000","last_became_active":"2023-10-06T05:23:30.141897+0000","last_became_peered":"2023-10-06T05:23:30.141897+0000","last_unstale":"2023-10-06T05:23:30.142165+0000","last_undegraded":"2023-10-06T05:23:30.142165+0000","last_fullsized":"2023-10-06T05:23:30.142165+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:33.883796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.148090+0000","last_change":"2023-10-06T05:23:30.148090+0000","last_active":"2023-10-06T05:23:30.148090+0000","last_peered":"2023-10-06T05:23:30.148090+0000","last_clean":"2023-10-06T05:23:30.148090+0000","last_became_active":"2023-10-06T05:23:30.147744+0000","last_became_peered":"2023-10-06T05:23:30.147744+0000","last_unstale":"2023-10-06T05:23:30.148090+0000","last_undegraded":"2023-10-06T05:23:30.148090+0000","last_fullsized":"2023-10-06T05:23:30.148090+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:30.015241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.138418+0000","last_change":"2023-10-06T05:23:30.138418+0000","last_active":"2023-10-06T05:23:30.138418+0000","last_peered":"2023-10-06T05:23:30.138418+0000","last_clean":"2023-10-06T05:23:30.138418+0000","last_became_active":"2023-10-06T05:23:30.138139+0000","last_became_peered":"2023-10-06T05:23:30.138139+0000","last_unstale":"2023-10-06T05:23:30.138418+0000","last_undegraded":"2023-10-06T05:23:30.138418+0000","last_fullsized":"2023-10-06T05:23:30.138418+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:42.294545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.138287+0000","last_change":"2023-10-06T05:23:30.138287+0000","last_active":"2023-10-06T05:23:30.138287+0000","last_peered":"2023-10-06T05:23:30.138287+0000","last_clean":"2023-10-06T05:23:30.138287+0000","last_became_active":"2023-10-06T05:23:30.137851+0000","last_became_peered":"2023-10-06T05:23:30.137851+0000","last_unstale":"2023-10-06T05:23:30.138287+0000","last_undegraded":"2023-10-06T05:23:30.138287+0000","last_fullsized":"2023-10-06T05:23:30.138287+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:38.445699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.149225+0000","last_change":"2023-10-06T05:23:30.149225+0000","last_active":"2023-10-06T05:23:30.149225+0000","last_peered":"2023-10-06T05:23:30.149225+0000","last_clean":"2023-10-06T05:23:30.149225+0000","last_became_active":"2023-10-06T05:23:30.148911+0000","last_became_peered":"2023-10-06T05:23:30.148911+0000","last_unstale":"2023-10-06T05:23:30.149225+0000","last_undegraded":"2023-10-06T05:23:30.149225+0000","last_fullsized":"2023-10-06T05:23:30.149225+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:57.634204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141394+0000","last_change":"2023-10-06T05:23:30.141394+0000","last_active":"2023-10-06T05:23:30.141394+0000","last_peered":"2023-10-06T05:23:30.141394+0000","last_clean":"2023-10-06T05:23:30.141394+0000","last_became_active":"2023-10-06T05:23:30.141128+0000","last_became_peered":"2023-10-06T05:23:30.141128+0000","last_unstale":"2023-10-06T05:23:30.141394+0000","last_undegraded":"2023-10-06T05:23:30.141394+0000","last_fullsized":"2023-10-06T05:23:30.141394+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:56.838792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.136295+0000","last_change":"2023-10-06T05:23:30.132199+0000","last_active":"2023-10-06T05:23:30.136295+0000","last_peered":"2023-10-06T05:23:30.136295+0000","last_clean":"2023-10-06T05:23:30.136295+0000","last_became_active":"2023-10-06T05:23:30.130271+0000","last_became_peered":"2023-10-06T05:23:30.130271+0000","last_unstale":"2023-10-06T05:23:30.136295+0000","last_undegraded":"2023-10-06T05:23:30.136295+0000","last_fullsized":"2023-10-06T05:23:30.136295+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.065283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.131230+0000","last_change":"2023-10-06T05:23:30.131230+0000","last_active":"2023-10-06T05:23:30.131230+0000","last_peered":"2023-10-06T05:23:30.131230+0000","last_clean":"2023-10-06T05:23:30.131230+0000","last_became_active":"2023-10-06T05:23:30.130956+0000","last_became_peered":"2023-10-06T05:23:30.130956+0000","last_unstale":"2023-10-06T05:23:30.131230+0000","last_undegraded":"2023-10-06T05:23:30.131230+0000","last_fullsized":"2023-10-06T05:23:30.131230+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:57.471693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.155166+0000","last_change":"2023-10-06T05:23:30.148928+0000","last_active":"2023-10-06T05:23:30.155166+0000","last_peered":"2023-10-06T05:23:30.155166+0000","last_clean":"2023-10-06T05:23:30.155166+0000","last_became_active":"2023-10-06T05:23:30.148645+0000","last_became_peered":"2023-10-06T05:23:30.148645+0000","last_unstale":"2023-10-06T05:23:30.155166+0000","last_undegraded":"2023-10-06T05:23:30.155166+0000","last_fullsized":"2023-10-06T05:23:30.155166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:05.514516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"113'4","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.158491+0000","last_change":"2023-10-06T05:23:29.137035+0000","last_active":"2023-10-06T05:23:30.158491+0000","last_peered":"2023-10-06T05:23:30.158491+0000","last_clean":"2023-10-06T05:23:30.158491+0000","last_became_active":"2023-10-06T05:23:29.136668+0000","last_became_peered":"2023-10-06T05:23:29.136668+0000","last_unstale":"2023-10-06T05:23:30.158491+0000","last_undegraded":"2023-10-06T05:23:30.158491+0000","last_fullsized":"2023-10-06T05:23:30.158491+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:00.418916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141707+0000","last_change":"2023-10-06T05:23:30.136346+0000","last_active":"2023-10-06T05:23:30.141707+0000","last_peered":"2023-10-06T05:23:30.141707+0000","last_clean":"2023-10-06T05:23:30.141707+0000","last_became_active":"2023-10-06T05:23:30.136105+0000","last_became_peered":"2023-10-06T05:23:30.136105+0000","last_unstale":"2023-10-06T05:23:30.141707+0000","last_undegraded":"2023-10-06T05:23:30.141707+0000","last_fullsized":"2023-10-06T05:23:30.141707+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:26.269530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.158053+0000","last_change":"2023-10-06T05:23:29.136838+0000","last_active":"2023-10-06T05:23:30.158053+0000","last_peered":"2023-10-06T05:23:30.158053+0000","last_clean":"2023-10-06T05:23:30.158053+0000","last_became_active":"2023-10-06T05:23:29.136527+0000","last_became_peered":"2023-10-06T05:23:29.136527+0000","last_unstale":"2023-10-06T05:23:30.158053+0000","last_undegraded":"2023-10-06T05:23:30.158053+0000","last_fullsized":"2023-10-06T05:23:30.158053+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:47.815436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140221+0000","last_change":"2023-10-06T05:23:30.140221+0000","last_active":"2023-10-06T05:23:30.140221+0000","last_peered":"2023-10-06T05:23:30.140221+0000","last_clean":"2023-10-06T05:23:30.140221+0000","last_became_active":"2023-10-06T05:23:30.139934+0000","last_became_peered":"2023-10-06T05:23:30.139934+0000","last_unstale":"2023-10-06T05:23:30.140221+0000","last_undegraded":"2023-10-06T05:23:30.140221+0000","last_fullsized":"2023-10-06T05:23:30.140221+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:33.248937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"113'2","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.156236+0000","last_change":"2023-10-06T05:23:29.133876+0000","last_active":"2023-10-06T05:23:30.156236+0000","last_peered":"2023-10-06T05:23:30.156236+0000","last_clean":"2023-10-06T05:23:30.156236+0000","last_became_active":"2023-10-06T05:23:29.133509+0000","last_became_peered":"2023-10-06T05:23:29.133509+0000","last_unstale":"2023-10-06T05:23:30.156236+0000","last_undegraded":"2023-10-06T05:23:30.156236+0000","last_fullsized":"2023-10-06T05:23:30.156236+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:05.097160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.144536+0000","last_change":"2023-10-06T05:23:30.142230+0000","last_active":"2023-10-06T05:23:30.144536+0000","last_peered":"2023-10-06T05:23:30.144536+0000","last_clean":"2023-10-06T05:23:30.144536+0000","last_became_active":"2023-10-06T05:23:30.141968+0000","last_became_peered":"2023-10-06T05:23:30.141968+0000","last_unstale":"2023-10-06T05:23:30.144536+0000","last_undegraded":"2023-10-06T05:23:30.144536+0000","last_fullsized":"2023-10-06T05:23:30.144536+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:26.277530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"113'2","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.160556+0000","last_change":"2023-10-06T05:23:29.133699+0000","last_active":"2023-10-06T05:23:30.160556+0000","last_peered":"2023-10-06T05:23:30.160556+0000","last_clean":"2023-10-06T05:23:30.160556+0000","last_became_active":"2023-10-06T05:23:29.133270+0000","last_became_peered":"2023-10-06T05:23:29.133270+0000","last_unstale":"2023-10-06T05:23:30.160556+0000","last_undegraded":"2023-10-06T05:23:30.160556+0000","last_fullsized":"2023-10-06T05:23:30.160556+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:52.377269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.136344+0000","last_change":"2023-10-06T05:23:30.131605+0000","last_active":"2023-10-06T05:23:30.136344+0000","last_peered":"2023-10-06T05:23:30.136344+0000","last_clean":"2023-10-06T05:23:30.136344+0000","last_became_active":"2023-10-06T05:23:30.131320+0000","last_became_peered":"2023-10-06T05:23:30.131320+0000","last_unstale":"2023-10-06T05:23:30.136344+0000","last_undegraded":"2023-10-06T05:23:30.136344+0000","last_fullsized":"2023-10-06T05:23:30.136344+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:06.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.158395+0000","last_change":"2023-10-06T05:23:29.136365+0000","last_active":"2023-10-06T05:23:30.158395+0000","last_peered":"2023-10-06T05:23:30.158395+0000","last_clean":"2023-10-06T05:23:30.158395+0000","last_became_active":"2023-10-06T05:23:29.136076+0000","last_became_peered":"2023-10-06T05:23:29.136076+0000","last_unstale":"2023-10-06T05:23:30.158395+0000","last_undegraded":"2023-10-06T05:23:30.158395+0000","last_fullsized":"2023-10-06T05:23:30.158395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:45.591769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140318+0000","last_change":"2023-10-06T05:23:30.134366+0000","last_active":"2023-10-06T05:23:30.140318+0000","last_peered":"2023-10-06T05:23:30.140318+0000","last_clean":"2023-10-06T05:23:30.140318+0000","last_became_active":"2023-10-06T05:23:30.134065+0000","last_became_peered":"2023-10-06T05:23:30.134065+0000","last_unstale":"2023-10-06T05:23:30.140318+0000","last_undegraded":"2023-10-06T05:23:30.140318+0000","last_fullsized":"2023-10-06T05:23:30.140318+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:33.248937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.158385+0000","last_change":"2023-10-06T05:23:29.127493+0000","last_active":"2023-10-06T05:23:30.158385+0000","last_peered":"2023-10-06T05:23:30.158385+0000","last_clean":"2023-10-06T05:23:30.158385+0000","last_became_active":"2023-10-06T05:23:29.127238+0000","last_became_peered":"2023-10-06T05:23:29.127238+0000","last_unstale":"2023-10-06T05:23:30.158385+0000","last_undegraded":"2023-10-06T05:23:30.158385+0000","last_fullsized":"2023-10-06T05:23:30.158385+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:10.622995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.155680+0000","last_change":"2023-10-06T05:23:30.140809+0000","last_active":"2023-10-06T05:23:30.155680+0000","last_peered":"2023-10-06T05:23:30.155680+0000","last_clean":"2023-10-06T05:23:30.155680+0000","last_became_active":"2023-10-06T05:23:30.140518+0000","last_became_peered":"2023-10-06T05:23:30.140518+0000","last_unstale":"2023-10-06T05:23:30.155680+0000","last_undegraded":"2023-10-06T05:23:30.155680+0000","last_fullsized":"2023-10-06T05:23:30.155680+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:35.939769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.130156+0000","last_change":"2023-10-06T05:23:29.124548+0000","last_active":"2023-10-06T05:23:29.130156+0000","last_peered":"2023-10-06T05:23:29.130156+0000","last_clean":"2023-10-06T05:23:29.130156+0000","last_became_active":"2023-10-06T05:23:29.124178+0000","last_became_peered":"2023-10-06T05:23:29.124178+0000","last_unstale":"2023-10-06T05:23:29.130156+0000","last_undegraded":"2023-10-06T05:23:29.130156+0000","last_fullsized":"2023-10-06T05:23:29.130156+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:37.980654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.138062+0000","last_change":"2023-10-06T05:23:30.138062+0000","last_active":"2023-10-06T05:23:30.138062+0000","last_peered":"2023-10-06T05:23:30.138062+0000","last_clean":"2023-10-06T05:23:30.138062+0000","last_became_active":"2023-10-06T05:23:30.137735+0000","last_became_peered":"2023-10-06T05:23:30.137735+0000","last_unstale":"2023-10-06T05:23:30.138062+0000","last_undegraded":"2023-10-06T05:23:30.138062+0000","last_fullsized":"2023-10-06T05:23:30.138062+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:13.176765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.141048+0000","last_change":"2023-10-06T05:23:29.137049+0000","last_active":"2023-10-06T05:23:29.141048+0000","last_peered":"2023-10-06T05:23:29.141048+0000","last_clean":"2023-10-06T05:23:29.141048+0000","last_became_active":"2023-10-06T05:23:29.136671+0000","last_became_peered":"2023-10-06T05:23:29.136671+0000","last_unstale":"2023-10-06T05:23:29.141048+0000","last_undegraded":"2023-10-06T05:23:29.141048+0000","last_fullsized":"2023-10-06T05:23:29.141048+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:14.856202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.136905+0000","last_change":"2023-10-06T05:23:30.136905+0000","last_active":"2023-10-06T05:23:30.136905+0000","last_peered":"2023-10-06T05:23:30.136905+0000","last_clean":"2023-10-06T05:23:30.136905+0000","last_became_active":"2023-10-06T05:23:30.136438+0000","last_became_peered":"2023-10-06T05:23:30.136438+0000","last_unstale":"2023-10-06T05:23:30.136905+0000","last_undegraded":"2023-10-06T05:23:30.136905+0000","last_fullsized":"2023-10-06T05:23:30.136905+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:09.638864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"113'1","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.125579+0000","last_change":"2023-10-06T05:23:29.135237+0000","last_active":"2023-10-06T05:23:30.125579+0000","last_peered":"2023-10-06T05:23:30.125579+0000","last_clean":"2023-10-06T05:23:30.125579+0000","last_became_active":"2023-10-06T05:23:29.134930+0000","last_became_peered":"2023-10-06T05:23:29.134930+0000","last_unstale":"2023-10-06T05:23:30.125579+0000","last_undegraded":"2023-10-06T05:23:30.125579+0000","last_fullsized":"2023-10-06T05:23:30.125579+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:45.591769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.132092+0000","last_change":"2023-10-06T05:23:30.132092+0000","last_active":"2023-10-06T05:23:30.132092+0000","last_peered":"2023-10-06T05:23:30.132092+0000","last_clean":"2023-10-06T05:23:30.132092+0000","last_became_active":"2023-10-06T05:23:30.131834+0000","last_became_peered":"2023-10-06T05:23:30.131834+0000","last_unstale":"2023-10-06T05:23:30.132092+0000","last_undegraded":"2023-10-06T05:23:30.132092+0000","last_fullsized":"2023-10-06T05:23:30.132092+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:30.983647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"113'2","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.156346+0000","last_change":"2023-10-06T05:23:29.136280+0000","last_active":"2023-10-06T05:23:30.156346+0000","last_peered":"2023-10-06T05:23:30.156346+0000","last_clean":"2023-10-06T05:23:30.156346+0000","last_became_active":"2023-10-06T05:23:29.130976+0000","last_became_peered":"2023-10-06T05:23:29.130976+0000","last_unstale":"2023-10-06T05:23:30.156346+0000","last_undegraded":"2023-10-06T05:23:30.156346+0000","last_fullsized":"2023-10-06T05:23:30.156346+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:50.624645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.139406+0000","last_change":"2023-10-06T05:23:30.139406+0000","last_active":"2023-10-06T05:23:30.139406+0000","last_peered":"2023-10-06T05:23:30.139406+0000","last_clean":"2023-10-06T05:23:30.139406+0000","last_became_active":"2023-10-06T05:23:30.139007+0000","last_became_peered":"2023-10-06T05:23:30.139007+0000","last_unstale":"2023-10-06T05:23:30.139406+0000","last_undegraded":"2023-10-06T05:23:30.139406+0000","last_fullsized":"2023-10-06T05:23:30.139406+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:52.725371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"113'1","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.120313+0000","last_change":"2023-10-06T05:23:29.124137+0000","last_active":"2023-10-06T05:23:30.120313+0000","last_peered":"2023-10-06T05:23:30.120313+0000","last_clean":"2023-10-06T05:23:30.120313+0000","last_became_active":"2023-10-06T05:23:29.123822+0000","last_became_peered":"2023-10-06T05:23:29.123822+0000","last_unstale":"2023-10-06T05:23:30.120313+0000","last_undegraded":"2023-10-06T05:23:30.120313+0000","last_fullsized":"2023-10-06T05:23:30.120313+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:15.126665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.150279+0000","last_change":"2023-10-06T05:23:30.150279+0000","last_active":"2023-10-06T05:23:30.150279+0000","last_peered":"2023-10-06T05:23:30.150279+0000","last_clean":"2023-10-06T05:23:30.150279+0000","last_became_active":"2023-10-06T05:23:30.150008+0000","last_became_peered":"2023-10-06T05:23:30.150008+0000","last_unstale":"2023-10-06T05:23:30.150279+0000","last_undegraded":"2023-10-06T05:23:30.150279+0000","last_fullsized":"2023-10-06T05:23:30.150279+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:21.046071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.123568+0000","last_change":"2023-10-06T05:23:29.129736+0000","last_active":"2023-10-06T05:23:30.123568+0000","last_peered":"2023-10-06T05:23:30.123568+0000","last_clean":"2023-10-06T05:23:30.123568+0000","last_became_active":"2023-10-06T05:23:29.129414+0000","last_became_peered":"2023-10-06T05:23:29.129414+0000","last_unstale":"2023-10-06T05:23:30.123568+0000","last_undegraded":"2023-10-06T05:23:30.123568+0000","last_fullsized":"2023-10-06T05:23:30.123568+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:37.933377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.130937+0000","last_change":"2023-10-06T05:23:30.128248+0000","last_active":"2023-10-06T05:23:30.130937+0000","last_peered":"2023-10-06T05:23:30.130937+0000","last_clean":"2023-10-06T05:23:30.130937+0000","last_became_active":"2023-10-06T05:23:30.127963+0000","last_became_peered":"2023-10-06T05:23:30.127963+0000","last_unstale":"2023-10-06T05:23:30.130937+0000","last_undegraded":"2023-10-06T05:23:30.130937+0000","last_fullsized":"2023-10-06T05:23:30.130937+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:05.884400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.160389+0000","last_change":"2023-10-06T05:23:29.138352+0000","last_active":"2023-10-06T05:23:30.160389+0000","last_peered":"2023-10-06T05:23:30.160389+0000","last_clean":"2023-10-06T05:23:30.160389+0000","last_became_active":"2023-10-06T05:23:29.138114+0000","last_became_peered":"2023-10-06T05:23:29.138114+0000","last_unstale":"2023-10-06T05:23:30.160389+0000","last_undegraded":"2023-10-06T05:23:30.160389+0000","last_fullsized":"2023-10-06T05:23:30.160389+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:04.794201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141769+0000","last_change":"2023-10-06T05:23:30.140210+0000","last_active":"2023-10-06T05:23:30.141769+0000","last_peered":"2023-10-06T05:23:30.141769+0000","last_clean":"2023-10-06T05:23:30.141769+0000","last_became_active":"2023-10-06T05:23:30.139918+0000","last_became_peered":"2023-10-06T05:23:30.139918+0000","last_unstale":"2023-10-06T05:23:30.141769+0000","last_undegraded":"2023-10-06T05:23:30.141769+0000","last_fullsized":"2023-10-06T05:23:30.141769+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:18.048314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.126678+0000","last_change":"2023-10-06T05:23:29.129688+0000","last_active":"2023-10-06T05:23:30.126678+0000","last_peered":"2023-10-06T05:23:30.126678+0000","last_clean":"2023-10-06T05:23:30.126678+0000","last_became_active":"2023-10-06T05:23:29.129401+0000","last_became_peered":"2023-10-06T05:23:29.129401+0000","last_unstale":"2023-10-06T05:23:30.126678+0000","last_undegraded":"2023-10-06T05:23:30.126678+0000","last_fullsized":"2023-10-06T05:23:30.126678+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:11.185301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.155703+0000","last_change":"2023-10-06T05:23:30.149969+0000","last_active":"2023-10-06T05:23:30.155703+0000","last_peered":"2023-10-06T05:23:30.155703+0000","last_clean":"2023-10-06T05:23:30.155703+0000","last_became_active":"2023-10-06T05:23:30.149691+0000","last_became_peered":"2023-10-06T05:23:30.149691+0000","last_unstale":"2023-10-06T05:23:30.155703+0000","last_undegraded":"2023-10-06T05:23:30.155703+0000","last_fullsized":"2023-10-06T05:23:30.155703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:30.983647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.139434+0000","last_change":"2023-10-06T05:23:29.134522+0000","last_active":"2023-10-06T05:23:29.139434+0000","last_peered":"2023-10-06T05:23:29.139434+0000","last_clean":"2023-10-06T05:23:29.139434+0000","last_became_active":"2023-10-06T05:23:29.134219+0000","last_became_peered":"2023-10-06T05:23:29.134219+0000","last_unstale":"2023-10-06T05:23:29.139434+0000","last_undegraded":"2023-10-06T05:23:29.139434+0000","last_fullsized":"2023-10-06T05:23:29.139434+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:36.325949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.151402+0000","last_change":"2023-10-06T05:23:30.151402+0000","last_active":"2023-10-06T05:23:30.151402+0000","last_peered":"2023-10-06T05:23:30.151402+0000","last_clean":"2023-10-06T05:23:30.151402+0000","last_became_active":"2023-10-06T05:23:30.151143+0000","last_became_peered":"2023-10-06T05:23:30.151143+0000","last_unstale":"2023-10-06T05:23:30.151402+0000","last_undegraded":"2023-10-06T05:23:30.151402+0000","last_fullsized":"2023-10-06T05:23:30.151402+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:06.440258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.140071+0000","last_change":"2023-10-06T05:23:29.136217+0000","last_active":"2023-10-06T05:23:29.140071+0000","last_peered":"2023-10-06T05:23:29.140071+0000","last_clean":"2023-10-06T05:23:29.140071+0000","last_became_active":"2023-10-06T05:23:29.135829+0000","last_became_peered":"2023-10-06T05:23:29.135829+0000","last_unstale":"2023-10-06T05:23:29.140071+0000","last_undegraded":"2023-10-06T05:23:29.140071+0000","last_fullsized":"2023-10-06T05:23:29.140071+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:56.628408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.154632+0000","last_change":"2023-10-06T05:23:30.140251+0000","last_active":"2023-10-06T05:23:30.154632+0000","last_peered":"2023-10-06T05:23:30.154632+0000","last_clean":"2023-10-06T05:23:30.154632+0000","last_became_active":"2023-10-06T05:23:30.140003+0000","last_became_peered":"2023-10-06T05:23:30.140003+0000","last_unstale":"2023-10-06T05:23:30.154632+0000","last_undegraded":"2023-10-06T05:23:30.154632+0000","last_fullsized":"2023-10-06T05:23:30.154632+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:55.301787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.155788+0000","last_change":"2023-10-06T05:23:29.139095+0000","last_active":"2023-10-06T05:23:30.155788+0000","last_peered":"2023-10-06T05:23:30.155788+0000","last_clean":"2023-10-06T05:23:30.155788+0000","last_became_active":"2023-10-06T05:23:29.138833+0000","last_became_peered":"2023-10-06T05:23:29.138833+0000","last_unstale":"2023-10-06T05:23:30.155788+0000","last_undegraded":"2023-10-06T05:23:30.155788+0000","last_fullsized":"2023-10-06T05:23:30.155788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:16.485871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.147573+0000","last_change":"2023-10-06T05:23:30.147573+0000","last_active":"2023-10-06T05:23:30.147573+0000","last_peered":"2023-10-06T05:23:30.147573+0000","last_clean":"2023-10-06T05:23:30.147573+0000","last_became_active":"2023-10-06T05:23:30.147279+0000","last_became_peered":"2023-10-06T05:23:30.147279+0000","last_unstale":"2023-10-06T05:23:30.147573+0000","last_undegraded":"2023-10-06T05:23:30.147573+0000","last_fullsized":"2023-10-06T05:23:30.147573+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:31.416020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.119929+0000","last_change":"2023-10-06T05:23:29.133773+0000","last_active":"2023-10-06T05:23:30.119929+0000","last_peered":"2023-10-06T05:23:30.119929+0000","last_clean":"2023-10-06T05:23:30.119929+0000","last_became_active":"2023-10-06T05:23:29.133388+0000","last_became_peered":"2023-10-06T05:23:29.133388+0000","last_unstale":"2023-10-06T05:23:30.119929+0000","last_undegraded":"2023-10-06T05:23:30.119929+0000","last_fullsized":"2023-10-06T05:23:30.119929+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:10.132750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.153869+0000","last_change":"2023-10-06T05:23:30.153869+0000","last_active":"2023-10-06T05:23:30.153869+0000","last_peered":"2023-10-06T05:23:30.153869+0000","last_clean":"2023-10-06T05:23:30.153869+0000","last_became_active":"2023-10-06T05:23:30.153594+0000","last_became_peered":"2023-10-06T05:23:30.153594+0000","last_unstale":"2023-10-06T05:23:30.153869+0000","last_undegraded":"2023-10-06T05:23:30.153869+0000","last_fullsized":"2023-10-06T05:23:30.153869+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:51.494448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.120650+0000","last_change":"2023-10-06T05:23:29.130788+0000","last_active":"2023-10-06T05:23:30.120650+0000","last_peered":"2023-10-06T05:23:30.120650+0000","last_clean":"2023-10-06T05:23:30.120650+0000","last_became_active":"2023-10-06T05:23:29.130480+0000","last_became_peered":"2023-10-06T05:23:29.130480+0000","last_unstale":"2023-10-06T05:23:30.120650+0000","last_undegraded":"2023-10-06T05:23:30.120650+0000","last_fullsized":"2023-10-06T05:23:30.120650+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:02.324057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.137782+0000","last_change":"2023-10-06T05:23:30.137782+0000","last_active":"2023-10-06T05:23:30.137782+0000","last_peered":"2023-10-06T05:23:30.137782+0000","last_clean":"2023-10-06T05:23:30.137782+0000","last_became_active":"2023-10-06T05:23:30.137487+0000","last_became_peered":"2023-10-06T05:23:30.137487+0000","last_unstale":"2023-10-06T05:23:30.137782+0000","last_undegraded":"2023-10-06T05:23:30.137782+0000","last_fullsized":"2023-10-06T05:23:30.137782+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:47.002807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.133073+0000","last_change":"2023-10-06T05:23:29.132141+0000","last_active":"2023-10-06T05:23:29.133073+0000","last_peered":"2023-10-06T05:23:29.133073+0000","last_clean":"2023-10-06T05:23:29.133073+0000","last_became_active":"2023-10-06T05:23:29.131846+0000","last_became_peered":"2023-10-06T05:23:29.131846+0000","last_unstale":"2023-10-06T05:23:29.133073+0000","last_undegraded":"2023-10-06T05:23:29.133073+0000","last_fullsized":"2023-10-06T05:23:29.133073+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:12.291784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.134442+0000","last_change":"2023-10-06T05:23:30.134442+0000","last_active":"2023-10-06T05:23:30.134442+0000","last_peered":"2023-10-06T05:23:30.134442+0000","last_clean":"2023-10-06T05:23:30.134442+0000","last_became_active":"2023-10-06T05:23:30.134168+0000","last_became_peered":"2023-10-06T05:23:30.134168+0000","last_unstale":"2023-10-06T05:23:30.134442+0000","last_undegraded":"2023-10-06T05:23:30.134442+0000","last_fullsized":"2023-10-06T05:23:30.134442+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:06.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.139101+0000","last_change":"2023-10-06T05:23:29.132923+0000","last_active":"2023-10-06T05:23:29.139101+0000","last_peered":"2023-10-06T05:23:29.139101+0000","last_clean":"2023-10-06T05:23:29.139101+0000","last_became_active":"2023-10-06T05:23:29.132661+0000","last_became_peered":"2023-10-06T05:23:29.132661+0000","last_unstale":"2023-10-06T05:23:29.139101+0000","last_undegraded":"2023-10-06T05:23:29.139101+0000","last_fullsized":"2023-10-06T05:23:29.139101+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:04.794201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.134639+0000","last_change":"2023-10-06T05:23:30.134639+0000","last_active":"2023-10-06T05:23:30.134639+0000","last_peered":"2023-10-06T05:23:30.134639+0000","last_clean":"2023-10-06T05:23:30.134639+0000","last_became_active":"2023-10-06T05:23:30.134373+0000","last_became_peered":"2023-10-06T05:23:30.134373+0000","last_unstale":"2023-10-06T05:23:30.134639+0000","last_undegraded":"2023-10-06T05:23:30.134639+0000","last_fullsized":"2023-10-06T05:23:30.134639+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.065283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.139174+0000","last_change":"2023-10-06T05:23:29.134462+0000","last_active":"2023-10-06T05:23:29.139174+0000","last_peered":"2023-10-06T05:23:29.139174+0000","last_clean":"2023-10-06T05:23:29.139174+0000","last_became_active":"2023-10-06T05:23:29.133452+0000","last_became_peered":"2023-10-06T05:23:29.133452+0000","last_unstale":"2023-10-06T05:23:29.139174+0000","last_undegraded":"2023-10-06T05:23:29.139174+0000","last_fullsized":"2023-10-06T05:23:29.139174+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:40.296147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140995+0000","last_change":"2023-10-06T05:23:30.140995+0000","last_active":"2023-10-06T05:23:30.140995+0000","last_peered":"2023-10-06T05:23:30.140995+0000","last_clean":"2023-10-06T05:23:30.140995+0000","last_became_active":"2023-10-06T05:23:30.140689+0000","last_became_peered":"2023-10-06T05:23:30.140689+0000","last_unstale":"2023-10-06T05:23:30.140995+0000","last_undegraded":"2023-10-06T05:23:30.140995+0000","last_fullsized":"2023-10-06T05:23:30.140995+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:04.795929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.133062+0000","last_change":"2023-10-06T05:23:29.131644+0000","last_active":"2023-10-06T05:23:29.133062+0000","last_peered":"2023-10-06T05:23:29.133062+0000","last_clean":"2023-10-06T05:23:29.133062+0000","last_became_active":"2023-10-06T05:23:29.131345+0000","last_became_peered":"2023-10-06T05:23:29.131345+0000","last_unstale":"2023-10-06T05:23:29.133062+0000","last_undegraded":"2023-10-06T05:23:29.133062+0000","last_fullsized":"2023-10-06T05:23:29.133062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:20.579285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.156341+0000","last_change":"2023-10-06T05:23:30.156341+0000","last_active":"2023-10-06T05:23:30.156341+0000","last_peered":"2023-10-06T05:23:30.156341+0000","last_clean":"2023-10-06T05:23:30.156341+0000","last_became_active":"2023-10-06T05:23:30.156044+0000","last_became_peered":"2023-10-06T05:23:30.156044+0000","last_unstale":"2023-10-06T05:23:30.156341+0000","last_undegraded":"2023-10-06T05:23:30.156341+0000","last_fullsized":"2023-10-06T05:23:30.156341+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:06.719958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.123067+0000","last_change":"2023-10-06T05:23:29.138784+0000","last_active":"2023-10-06T05:23:30.123067+0000","last_peered":"2023-10-06T05:23:30.123067+0000","last_clean":"2023-10-06T05:23:30.123067+0000","last_became_active":"2023-10-06T05:23:29.138475+0000","last_became_peered":"2023-10-06T05:23:29.138475+0000","last_unstale":"2023-10-06T05:23:30.123067+0000","last_undegraded":"2023-10-06T05:23:30.123067+0000","last_fullsized":"2023-10-06T05:23:30.123067+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:52.462512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.149780+0000","last_change":"2023-10-06T05:23:30.149780+0000","last_active":"2023-10-06T05:23:30.149780+0000","last_peered":"2023-10-06T05:23:30.149780+0000","last_clean":"2023-10-06T05:23:30.149780+0000","last_became_active":"2023-10-06T05:23:30.149488+0000","last_became_peered":"2023-10-06T05:23:30.149488+0000","last_unstale":"2023-10-06T05:23:30.149780+0000","last_undegraded":"2023-10-06T05:23:30.149780+0000","last_fullsized":"2023-10-06T05:23:30.149780+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:03.550372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"113'3","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.160203+0000","last_change":"2023-10-06T05:23:29.137778+0000","last_active":"2023-10-06T05:23:30.160203+0000","last_peered":"2023-10-06T05:23:30.160203+0000","last_clean":"2023-10-06T05:23:30.160203+0000","last_became_active":"2023-10-06T05:23:29.137411+0000","last_became_peered":"2023-10-06T05:23:29.137411+0000","last_unstale":"2023-10-06T05:23:30.160203+0000","last_undegraded":"2023-10-06T05:23:30.160203+0000","last_fullsized":"2023-10-06T05:23:30.160203+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:43.374922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.132186+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:23:29.132186+0000","last_peered":"2023-10-06T05:23:29.132186+0000","last_clean":"2023-10-06T05:23:29.132186+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:23:29.132186+0000","last_undegraded":"2023-10-06T05:23:29.132186+0000","last_fullsized":"2023-10-06T05:23:29.132186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140403+0000","last_change":"2023-10-06T05:23:30.140403+0000","last_active":"2023-10-06T05:23:30.140403+0000","last_peered":"2023-10-06T05:23:30.140403+0000","last_clean":"2023-10-06T05:23:30.140403+0000","last_became_active":"2023-10-06T05:23:30.138925+0000","last_became_peered":"2023-10-06T05:23:30.138925+0000","last_unstale":"2023-10-06T05:23:30.140403+0000","last_undegraded":"2023-10-06T05:23:30.140403+0000","last_fullsized":"2023-10-06T05:23:30.140403+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:16.944912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"113'6","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.160728+0000","last_change":"2023-10-06T05:23:29.133359+0000","last_active":"2023-10-06T05:23:30.160728+0000","last_peered":"2023-10-06T05:23:30.160728+0000","last_clean":"2023-10-06T05:23:30.160728+0000","last_became_active":"2023-10-06T05:23:29.133060+0000","last_became_peered":"2023-10-06T05:23:29.133060+0000","last_unstale":"2023-10-06T05:23:30.160728+0000","last_undegraded":"2023-10-06T05:23:30.160728+0000","last_fullsized":"2023-10-06T05:23:30.160728+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:36.655702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.130479+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:23:29.130479+0000","last_peered":"2023-10-06T05:23:29.130479+0000","last_clean":"2023-10-06T05:23:29.130479+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:23:29.130479+0000","last_undegraded":"2023-10-06T05:23:29.130479+0000","last_fullsized":"2023-10-06T05:23:29.130479+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.138111+0000","last_change":"2023-10-06T05:23:30.138111+0000","last_active":"2023-10-06T05:23:30.138111+0000","last_peered":"2023-10-06T05:23:30.138111+0000","last_clean":"2023-10-06T05:23:30.138111+0000","last_became_active":"2023-10-06T05:23:30.137277+0000","last_became_peered":"2023-10-06T05:23:30.137277+0000","last_unstale":"2023-10-06T05:23:30.138111+0000","last_undegraded":"2023-10-06T05:23:30.138111+0000","last_fullsized":"2023-10-06T05:23:30.138111+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:20.345883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.154285+0000","last_change":"2023-10-06T05:23:29.137861+0000","last_active":"2023-10-06T05:23:30.154285+0000","last_peered":"2023-10-06T05:23:30.154285+0000","last_clean":"2023-10-06T05:23:30.154285+0000","last_became_active":"2023-10-06T05:23:29.137578+0000","last_became_peered":"2023-10-06T05:23:29.137578+0000","last_unstale":"2023-10-06T05:23:30.154285+0000","last_undegraded":"2023-10-06T05:23:30.154285+0000","last_fullsized":"2023-10-06T05:23:30.154285+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:24.006184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.122660+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:23:29.122660+0000","last_peered":"2023-10-06T05:23:29.122660+0000","last_clean":"2023-10-06T05:23:29.122660+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:23:29.122660+0000","last_undegraded":"2023-10-06T05:23:29.122660+0000","last_fullsized":"2023-10-06T05:23:29.122660+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.148510+0000","last_change":"2023-10-06T05:23:30.148510+0000","last_active":"2023-10-06T05:23:30.148510+0000","last_peered":"2023-10-06T05:23:30.148510+0000","last_clean":"2023-10-06T05:23:30.148510+0000","last_became_active":"2023-10-06T05:23:30.147288+0000","last_became_peered":"2023-10-06T05:23:30.147288+0000","last_unstale":"2023-10-06T05:23:30.148510+0000","last_undegraded":"2023-10-06T05:23:30.148510+0000","last_fullsized":"2023-10-06T05:23:30.148510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:02.373765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.139154+0000","last_change":"2023-10-06T05:23:29.133776+0000","last_active":"2023-10-06T05:23:29.139154+0000","last_peered":"2023-10-06T05:23:29.139154+0000","last_clean":"2023-10-06T05:23:29.139154+0000","last_became_active":"2023-10-06T05:23:29.133099+0000","last_became_peered":"2023-10-06T05:23:29.133099+0000","last_unstale":"2023-10-06T05:23:29.139154+0000","last_undegraded":"2023-10-06T05:23:29.139154+0000","last_fullsized":"2023-10-06T05:23:29.139154+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:50.972785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.123248+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:23:29.123248+0000","last_peered":"2023-10-06T05:23:29.123248+0000","last_clean":"2023-10-06T05:23:29.123248+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:23:29.123248+0000","last_undegraded":"2023-10-06T05:23:29.123248+0000","last_fullsized":"2023-10-06T05:23:29.123248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140732+0000","last_change":"2023-10-06T05:23:30.132892+0000","last_active":"2023-10-06T05:23:30.140732+0000","last_peered":"2023-10-06T05:23:30.140732+0000","last_clean":"2023-10-06T05:23:30.140732+0000","last_became_active":"2023-10-06T05:23:30.132541+0000","last_became_peered":"2023-10-06T05:23:30.132541+0000","last_unstale":"2023-10-06T05:23:30.140732+0000","last_undegraded":"2023-10-06T05:23:30.140732+0000","last_fullsized":"2023-10-06T05:23:30.140732+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:18.606201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"113'8","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.159615+0000","last_change":"2023-10-06T05:23:29.139053+0000","last_active":"2023-10-06T05:23:30.159615+0000","last_peered":"2023-10-06T05:23:30.159615+0000","last_clean":"2023-10-06T05:23:30.159615+0000","last_became_active":"2023-10-06T05:23:29.138771+0000","last_became_peered":"2023-10-06T05:23:29.138771+0000","last_unstale":"2023-10-06T05:23:30.159615+0000","last_undegraded":"2023-10-06T05:23:30.159615+0000","last_fullsized":"2023-10-06T05:23:30.159615+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:53.801548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":86,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.124185+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:23:29.124185+0000","last_peered":"2023-10-06T05:23:29.124185+0000","last_clean":"2023-10-06T05:23:29.124185+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:23:29.124185+0000","last_undegraded":"2023-10-06T05:23:29.124185+0000","last_fullsized":"2023-10-06T05:23:29.124185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.143656+0000","last_change":"2023-10-06T05:23:30.143656+0000","last_active":"2023-10-06T05:23:30.143656+0000","last_peered":"2023-10-06T05:23:30.143656+0000","last_clean":"2023-10-06T05:23:30.143656+0000","last_became_active":"2023-10-06T05:23:30.139683+0000","last_became_peered":"2023-10-06T05:23:30.139683+0000","last_unstale":"2023-10-06T05:23:30.143656+0000","last_undegraded":"2023-10-06T05:23:30.143656+0000","last_fullsized":"2023-10-06T05:23:30.143656+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:46.439071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.144011+0000","last_change":"2023-10-06T05:23:29.140252+0000","last_active":"2023-10-06T05:23:29.144011+0000","last_peered":"2023-10-06T05:23:29.144011+0000","last_clean":"2023-10-06T05:23:29.144011+0000","last_became_active":"2023-10-06T05:23:29.139946+0000","last_became_peered":"2023-10-06T05:23:29.139946+0000","last_unstale":"2023-10-06T05:23:29.144011+0000","last_undegraded":"2023-10-06T05:23:29.144011+0000","last_fullsized":"2023-10-06T05:23:29.144011+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:55.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":275,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.123161+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:23:29.123161+0000","last_peered":"2023-10-06T05:23:29.123161+0000","last_clean":"2023-10-06T05:23:29.123161+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:23:29.123161+0000","last_undegraded":"2023-10-06T05:23:29.123161+0000","last_fullsized":"2023-10-06T05:23:29.123161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":94,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.114510+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:23:29.114510+0000","last_peered":"2023-10-06T05:23:29.114510+0000","last_clean":"2023-10-06T05:23:29.114510+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:23:29.114510+0000","last_undegraded":"2023-10-06T05:23:29.114510+0000","last_fullsized":"2023-10-06T05:23:29.114510+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.130471+0000","last_change":"2023-10-06T05:23:30.130471+0000","last_active":"2023-10-06T05:23:30.130471+0000","last_peered":"2023-10-06T05:23:30.130471+0000","last_clean":"2023-10-06T05:23:30.130471+0000","last_became_active":"2023-10-06T05:23:30.130214+0000","last_became_peered":"2023-10-06T05:23:30.130214+0000","last_unstale":"2023-10-06T05:23:30.130471+0000","last_undegraded":"2023-10-06T05:23:30.130471+0000","last_fullsized":"2023-10-06T05:23:30.130471+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:25.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.125347+0000","last_change":"2023-10-06T05:23:29.137836+0000","last_active":"2023-10-06T05:23:30.125347+0000","last_peered":"2023-10-06T05:23:30.125347+0000","last_clean":"2023-10-06T05:23:30.125347+0000","last_became_active":"2023-10-06T05:23:29.137565+0000","last_became_peered":"2023-10-06T05:23:29.137565+0000","last_unstale":"2023-10-06T05:23:30.125347+0000","last_undegraded":"2023-10-06T05:23:30.125347+0000","last_fullsized":"2023-10-06T05:23:30.125347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:15.188594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.123658+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:23:30.123658+0000","last_peered":"2023-10-06T05:23:30.123658+0000","last_clean":"2023-10-06T05:23:30.123658+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:23:30.123658+0000","last_undegraded":"2023-10-06T05:23:30.123658+0000","last_fullsized":"2023-10-06T05:23:30.123658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141821+0000","last_change":"2023-10-06T05:23:30.139410+0000","last_active":"2023-10-06T05:23:30.141821+0000","last_peered":"2023-10-06T05:23:30.141821+0000","last_clean":"2023-10-06T05:23:30.141821+0000","last_became_active":"2023-10-06T05:23:30.139127+0000","last_became_peered":"2023-10-06T05:23:30.139127+0000","last_unstale":"2023-10-06T05:23:30.141821+0000","last_undegraded":"2023-10-06T05:23:30.141821+0000","last_fullsized":"2023-10-06T05:23:30.141821+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:01.424712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.139188+0000","last_change":"2023-10-06T05:23:29.133845+0000","last_active":"2023-10-06T05:23:29.139188+0000","last_peered":"2023-10-06T05:23:29.139188+0000","last_clean":"2023-10-06T05:23:29.139188+0000","last_became_active":"2023-10-06T05:23:29.133578+0000","last_became_peered":"2023-10-06T05:23:29.133578+0000","last_unstale":"2023-10-06T05:23:29.139188+0000","last_undegraded":"2023-10-06T05:23:29.139188+0000","last_fullsized":"2023-10-06T05:23:29.139188+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:10.132750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.120746+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:23:30.120746+0000","last_peered":"2023-10-06T05:23:30.120746+0000","last_clean":"2023-10-06T05:23:30.120746+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:23:30.120746+0000","last_undegraded":"2023-10-06T05:23:30.120746+0000","last_fullsized":"2023-10-06T05:23:30.120746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":1976,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":2023424,"data_stored":1246704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.47899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.51100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.46700000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88900000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91200000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.52000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88300000000000001,"5min":1.018,"15min":1.018},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71099999999999997,"15min":0.75},"min":{"1min":0.48999999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94199999999999995,"5min":1.056,"15min":1.056},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.746},"min":{"1min":0.53600000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":1.056,"15min":1.056},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.53600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.872,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63500000000000001,"15min":0.70699999999999996},"min":{"1min":0.498,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95199999999999996,"5min":0.95299999999999996,"15min":1.0249999999999999},"last":0.754},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71499999999999997,"15min":0.77400000000000002},"min":{"1min":0.52000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96699999999999997,"5min":1.27,"15min":1.27},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69999999999999996,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92900000000000005,"5min":0.97899999999999998,"15min":0.999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.74099999999999999,"15min":0.82399999999999995},"min":{"1min":0.47099999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.98899999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79900000000000004,"15min":0.876},"min":{"1min":0.56599999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0089999999999999,"5min":1.012,"15min":1.1040000000000001},"last":0.621},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.84599999999999997},"min":{"1min":0.57799999999999996,"5min":0.46200000000000002,"15min":0.441},"max":{"1min":0.99099999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298676,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073164,"statfs":{"total":96636764160,"available":96330919936,"internally_reserved":0,"allocated":1421312,"data_stored":655354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66800000000000004,"15min":0.68200000000000005},"min":{"1min":0.46600000000000003,"5min":0.42999999999999999,"15min":0.32300000000000001},"max":{"1min":0.84399999999999997,"5min":1.2629999999999999,"15min":1.8500000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.38900000000000001},"max":{"1min":0.89500000000000002,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70499999999999996,"15min":0.71299999999999997},"min":{"1min":0.50700000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":1.135,"15min":1.9279999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":1.073,"15min":1.4239999999999999},"last":0.754}]},{"osd":2,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94999999999999996,"5min":1.369,"15min":1.369},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.75900000000000001},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.91200000000000003,"5min":1.208,"15min":1.891},"last":0.69699999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.748},"min":{"1min":0.47799999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.86099999999999999,"5min":1.018,"15min":1.347},"last":0.63},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.53500000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":1},"last":0.499}]},{"osd":4,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.77600000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.998,"5min":1.1140000000000001,"15min":1.9670000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76900000000000002,"15min":0.81899999999999995},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.012,"5min":1.278,"15min":1.4510000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.415},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73899999999999999,"15min":0.81899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":2},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.753,"15min":0.80700000000000005},"min":{"1min":0.57299999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.751,"15min":0.77800000000000002},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.46},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303348,"kb_used_data":1964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068492,"statfs":{"total":96636764160,"available":96326135808,"internally_reserved":0,"allocated":2011136,"data_stored":1245148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.32800000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67000000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.54600000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.37,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.66000000000000003,"5min":0.77000000000000002,"15min":0.83799999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.496,"15min":0.47199999999999998},"min":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.753,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55900000000000005,"15min":0.52400000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.625,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.504},{"interface":"front","average":{"1min":0.495,"5min":0.53200000000000003,"15min":0.52700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.64200000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90200000000000002,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.81999999999999995,"15min":0.79500000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91600000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.76600000000000001,"15min":0.749},"min":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0009999999999999,"5min":1.099,"15min":1.099},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.79800000000000004,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0209999999999999,"5min":1.181,"15min":1.181},"last":0.69199999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.76800000000000002,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.86399999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81899999999999995,"15min":0.79900000000000004},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.95799999999999996,"5min":1.123,"15min":1.123},"last":0.85999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96799999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.82499999999999996,"15min":0.82399999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":0.91400000000000003,"5min":1.087,"15min":1.103},"last":0.73199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298676,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073164,"statfs":{"total":96636764160,"available":96330919936,"internally_reserved":0,"allocated":1421312,"data_stored":654766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.442,"15min":0.432},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58799999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.318},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.78900000000000003,"15min":1.042},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.499,"15min":0.49099999999999999},"min":{"1min":0.34300000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.504},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52900000000000003,"15min":0.51400000000000001},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.58699999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.52000000000000002,"15min":0.49099999999999999},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.32800000000000001},"max":{"1min":0.61399999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59099999999999997,"15min":0.57799999999999996},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.73999999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.496,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1379999999999999,"5min":1.242,"15min":1.242},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.76100000000000001,"15min":0.77800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94699999999999995,"5min":1.198,"15min":1.198},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.77300000000000002,"15min":0.76700000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":1.26,"15min":1.26},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76400000000000001,"15min":0.77400000000000002},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0029999999999999,"5min":1.163,"15min":1.163},"last":1.1299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.78300000000000003,"15min":0.79900000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.105,"5min":1.323,"15min":1.323},"last":0.999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.80500000000000005,"15min":0.79300000000000004},"min":{"1min":0.56499999999999995,"5min":0.55200000000000005,"15min":0.50600000000000001},"max":{"1min":0.98799999999999999,"5min":1.1799999999999999,"15min":1.3100000000000001},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.83399999999999996},"min":{"1min":0.57599999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.056,"5min":1.2090000000000001,"15min":1.264},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79500000000000004,"15min":0.82399999999999995},"min":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.077,"5min":1.1579999999999999,"15min":1.3680000000000001},"last":1.1719999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":655298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.43099999999999999,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.627,"5min":0.77400000000000002,"15min":0.77800000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68500000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.27400000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.502},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.31900000000000001},"max":{"1min":0.66200000000000003,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.51600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.82799999999999996,"15min":0.93600000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.45700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73599999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98499999999999999,"5min":1.022,"15min":1.1299999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.79400000000000004},"min":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96999999999999997,"5min":1.085,"15min":1.1699999999999999},"last":0.86599999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73699999999999999,"15min":0.78200000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.026,"5min":1.042,"15min":1.099},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.751,"15min":0.80000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.002,"5min":1.0649999999999999,"15min":1.163},"last":0.95999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77400000000000002,"15min":0.82999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94799999999999995,"5min":1.0960000000000001,"15min":1.21},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.998,"5min":1.022,"15min":1.042},"last":0.876}]},{"osd":7,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.77600000000000002,"15min":0.80600000000000005},"min":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.85899999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0629999999999999,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":0.84599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302780,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069060,"statfs":{"total":96636764160,"available":96326717440,"internally_reserved":0,"allocated":1429504,"data_stored":656316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44,"15min":0.46000000000000002},"min":{"1min":0.32600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67500000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53200000000000003,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70799999999999996,"5min":0.999,"15min":0.999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.74299999999999999,"5min":0.98999999999999999,"15min":1.014},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.57299999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.071,"5min":1.161,"15min":1.2669999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.57999999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.113,"5min":1.1830000000000001,"15min":1.3540000000000001},"last":0.89300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.54200000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0880000000000001,"5min":1.147,"15min":1.204},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79500000000000004,"15min":0.83299999999999996},"min":{"1min":0.59199999999999997,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0569999999999999,"5min":1.284,"15min":1.329},"last":0.95899999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78500000000000003,"15min":0.81499999999999995},"min":{"1min":0.53300000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.484},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.80800000000000005,"15min":0.84499999999999997},"min":{"1min":0.60599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.629},"last":1.0169999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80900000000000005,"15min":0.85099999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0640000000000001,"5min":1.236,"15min":1.5620000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82099999999999995,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.117,"5min":1.194,"15min":1.325},"last":0.94899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302764,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069076,"statfs":{"total":96636764160,"available":96326733824,"internally_reserved":0,"allocated":1413120,"data_stored":654710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90700000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68100000000000005,"15min":0.77700000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89100000000000001,"5min":0.95999999999999996,"15min":2.77},"last":0.872}]},{"osd":1,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.73499999999999999},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.6419999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68999999999999995,"15min":0.71999999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.036,"5min":1.1100000000000001,"15min":1.1899999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.49199999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71299999999999997,"15min":0.81200000000000006},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95599999999999996,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.52900000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95899999999999996,"5min":0.999,"15min":1.0629999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":1.026},"min":{"1min":0.59799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91200000000000003,"5min":0.97699999999999998,"15min":7.742},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68100000000000005,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91900000000000004,"5min":0.96499999999999997,"15min":1.079},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70599999999999996,"15min":0.748},"min":{"1min":0.498,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.181},"last":0.97499999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.73599999999999999},"min":{"1min":0.505,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.2669999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.747,"15min":0.81499999999999995},"min":{"1min":0.56499999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.171},"last":0.83699999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.82699999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76500000000000001,"15min":0.85299999999999998},"min":{"1min":0.64800000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.571},"last":0.76700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298672,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073168,"statfs":{"total":96636764160,"available":96330924032,"internally_reserved":0,"allocated":1417216,"data_stored":654732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.93600000000000005},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.873,"5min":1.109,"15min":9.2780000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.52500000000000002,"5min":0.438,"15min":0.434},"max":{"1min":0.88900000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68000000000000005,"15min":0.96199999999999997},"min":{"1min":0.432,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":1.079,"15min":9.25},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69599999999999995,"15min":1.022},"min":{"1min":0.44500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.54200000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66900000000000004,"15min":0.99299999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.052,"5min":1.103,"15min":10.384},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72799999999999998,"15min":1.0169999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73399999999999999,"15min":1.0589999999999999},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.76,"5min":1.76,"15min":10.205},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70499999999999996,"15min":0.72299999999999998},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.292},"last":0.625}]},{"osd":4,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96999999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.502},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72499999999999998,"15min":0.77200000000000002},"min":{"1min":0.53300000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.76300000000000001,"15min":0.81000000000000005},"min":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.008,"5min":1.1000000000000001,"15min":1.298},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78200000000000003,"15min":0.82199999999999995},"min":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77500000000000002,"15min":0.81100000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:23:34.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:23:34.726 INFO:tasks.ceph.mds.d.smithi062.stderr:2023-10-06T05:23:34.724+0000 7f955d5f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:23:34.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.761+0000 7f5986dff700 1 Processor -- start 2023-10-06T05:23:34.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.762+0000 7f5986dff700 1 -- start start 2023-10-06T05:23:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.763+0000 7f5986dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59800d0630 0x7f59800cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.763+0000 7f5986dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 0x7f5980158f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.763+0000 7f5986dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59800cdd00 con 0x7f59800ce560 2023-10-06T05:23:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f5986dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59800cead0 con 0x7f59800d0630 2023-10-06T05:23:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f5986dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59800cedb0 con 0x7f59800d0a00 2023-10-06T05:23:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f5984b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59800d0630 0x7f59800cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f5984b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59800d0630 0x7f59800cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57974/0 (socket says 172.21.15.31:57974) 2023-10-06T05:23:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f5984b9b700 1 -- 172.21.15.31:0/1620792978 learned_addr learned my addr 172.21.15.31:0/1620792978 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f598539c700 1 --2- 172.21.15.31:0/1620792978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 0x7f5980158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f5984b9b700 1 -- 172.21.15.31:0/1620792978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 msgr2=0x7f5980158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f5984b9b700 1 --2- 172.21.15.31:0/1620792978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 0x7f5980158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f5984b9b700 1 -- 172.21.15.31:0/1620792978 >> v1:172.21.15.62:6789/0 conn(0x7f59800ce560 legacy=0x7f59800cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f5984b9b700 1 -- 172.21.15.31:0/1620792978 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800cf090 con 0x7f59800d0630 2023-10-06T05:23:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.764+0000 7f598539c700 1 --2- 172.21.15.31:0/1620792978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 0x7f5980158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f5984b9b700 1 --2- 172.21.15.31:0/1620792978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 0x7f59800cd7c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f59740099b0 tx=0x7f5974009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a389fe9ee6d7a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f597f7fe700 1 -- 172.21.15.31:0/1620792978 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400b820 con 0x7f59800d0630 2023-10-06T05:23:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f597f7fe700 1 -- 172.21.15.31:0/1620792978 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597400b980 con 0x7f59800d0630 2023-10-06T05:23:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f597f7fe700 1 -- 172.21.15.31:0/1620792978 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400bd20 con 0x7f59800d0630 2023-10-06T05:23:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f5986dff700 1 -- 172.21.15.31:0/1620792978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 msgr2=0x7f59800cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f5986dff700 1 --2- 172.21.15.31:0/1620792978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 0x7f59800cd7c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f59740099b0 tx=0x7f5974009250 comp rx=0 tx=0).stop 2023-10-06T05:23:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f5986dff700 1 -- 172.21.15.31:0/1620792978 shutdown_connections 2023-10-06T05:23:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f5986dff700 1 --2- 172.21.15.31:0/1620792978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 0x7f5980158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f5986dff700 1 --2- 172.21.15.31:0/1620792978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 0x7f59800cd7c0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f5986dff700 1 -- 172.21.15.31:0/1620792978 >> 172.21.15.31:0/1620792978 conn(0x7f5980068190 msgr2=0x7f5980068590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.765+0000 7f5986dff700 1 -- 172.21.15.31:0/1620792978 shutdown_connections 2023-10-06T05:23:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.766+0000 7f5986dff700 1 -- 172.21.15.31:0/1620792978 wait complete. 2023-10-06T05:23:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.768+0000 7f5986dff700 1 Processor -- start 2023-10-06T05:23:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.768+0000 7f5986dff700 1 -- start start 2023-10-06T05:23:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f5986dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59800ce560 0x7f5980159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f5986dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 0x7f598015e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f5986dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 0x7f598015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f5986dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598015acc0 con 0x7f59800d0630 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f5986dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598015a040 con 0x7f59800ce560 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f5986dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598015a320 con 0x7f59800d0a00 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f597ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 0x7f598015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f597ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 0x7f598015e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57982/0 (socket says 172.21.15.31:57982) 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f597ffff700 1 -- 172.21.15.31:0/40925825 learned_addr learned my addr 172.21.15.31:0/40925825 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f5984b9b700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59800ce560 0x7f5980159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.769+0000 7f598539c700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 0x7f598015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f597ffff700 1 -- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 msgr2=0x7f598015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f597ffff700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 0x7f598015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f597ffff700 1 -- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59800ce560 msgr2=0x7f5980159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f597ffff700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59800ce560 0x7f5980159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f597ffff700 1 -- 172.21.15.31:0/40925825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59740094f0 con 0x7f59800d0630 2023-10-06T05:23:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f597ffff700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 0x7f598015e700 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f596c00b5e0 tx=0x7f596c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f597cff9700 1 -- 172.21.15.31:0/40925825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c00f510 con 0x7f59800d0630 2023-10-06T05:23:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f597cff9700 1 -- 172.21.15.31:0/40925825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c00f6b0 con 0x7f59800d0630 2023-10-06T05:23:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f5986dff700 1 -- 172.21.15.31:0/40925825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598015a600 con 0x7f59800d0630 2023-10-06T05:23:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f597cff9700 1 -- 172.21.15.31:0/40925825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c00fa70 con 0x7f59800d0630 2023-10-06T05:23:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.770+0000 7f5986dff700 1 -- 172.21.15.31:0/40925825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598015ba90 con 0x7f59800d0630 2023-10-06T05:23:34.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.771+0000 7f5986dff700 1 -- 172.21.15.31:0/40925825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598004c380 con 0x7f59800d0630 2023-10-06T05:23:34.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.772+0000 7f597cff9700 1 -- 172.21.15.31:0/40925825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f596c01a400 con 0x7f59800d0630 2023-10-06T05:23:34.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.773+0000 7f597cff9700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5964067320 0x7f59640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:34.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.773+0000 7f597cff9700 1 -- 172.21.15.31:0/40925825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f596c0847e0 con 0x7f59800d0630 2023-10-06T05:23:34.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.774+0000 7f5984b9b700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5964067320 0x7f59640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:34.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.775+0000 7f5984b9b700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5964067320 0x7f59640697d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f59740099b0 tx=0x7f5974016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:34.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.777+0000 7f597cff9700 1 -- 172.21.15.31:0/40925825 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f596c051330 con 0x7f59800d0630 2023-10-06T05:23:34.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.973+0000 7f5986dff700 1 -- 172.21.15.31:0/40925825 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59800cf370 con 0x7f5964067320 2023-10-06T05:23:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.983+0000 7f597cff9700 1 -- 172.21.15.31:0/40925825 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311973 (secure 0 0 0) 0x7f59800cf370 con 0x7f5964067320 2023-10-06T05:23:34.988 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 -- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5964067320 msgr2=0x7f59640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5964067320 0x7f59640697d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f59740099b0 tx=0x7f5974016000 comp rx=0 tx=0).stop 2023-10-06T05:23:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 -- 172.21.15.31:0/40925825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 msgr2=0x7f598015e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:34.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 0x7f598015e700 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f596c00b5e0 tx=0x7f596c0090f0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 -- 172.21.15.31:0/40925825 shutdown_connections 2023-10-06T05:23:34.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5964067320 0x7f59640697d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59800d0a00 0x7f598015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59800ce560 0x7f5980159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 --2- 172.21.15.31:0/40925825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800d0630 0x7f598015e700 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:34.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 -- 172.21.15.31:0/40925825 >> 172.21.15.31:0/40925825 conn(0x7f5980068190 msgr2=0x7f59800c6590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:34.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 -- 172.21.15.31:0/40925825 shutdown_connections 2023-10-06T05:23:34.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:34.987+0000 7f5986dff700 1 -- 172.21.15.31:0/40925825 wait complete. 2023-10-06T05:23:34.993 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:23:35.006 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2023-10-06T05:23:33.487924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411048,"kb_used_data":12264,"kb_used_omap":0,"kb_used_meta":2398720,"kb_avail":752563672,"statfs":{"total":773094113280,"available":770625200128,"internally_reserved":0,"allocated":12558336,"data_stored":6423028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2456289280},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.381359"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141119+0000","last_change":"2023-10-06T05:23:30.133477+0000","last_active":"2023-10-06T05:23:30.141119+0000","last_peered":"2023-10-06T05:23:30.141119+0000","last_clean":"2023-10-06T05:23:30.141119+0000","last_became_active":"2023-10-06T05:23:30.133198+0000","last_became_peered":"2023-10-06T05:23:30.133198+0000","last_unstale":"2023-10-06T05:23:30.141119+0000","last_undegraded":"2023-10-06T05:23:30.141119+0000","last_fullsized":"2023-10-06T05:23:30.141119+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:51.630441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141815+0000","last_change":"2023-10-06T05:23:30.141815+0000","last_active":"2023-10-06T05:23:30.141815+0000","last_peered":"2023-10-06T05:23:30.141815+0000","last_clean":"2023-10-06T05:23:30.141815+0000","last_became_active":"2023-10-06T05:23:30.141545+0000","last_became_peered":"2023-10-06T05:23:30.141545+0000","last_unstale":"2023-10-06T05:23:30.141815+0000","last_undegraded":"2023-10-06T05:23:30.141815+0000","last_fullsized":"2023-10-06T05:23:30.141815+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:08.653017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.137550+0000","last_change":"2023-10-06T05:23:30.137550+0000","last_active":"2023-10-06T05:23:30.137550+0000","last_peered":"2023-10-06T05:23:30.137550+0000","last_clean":"2023-10-06T05:23:30.137550+0000","last_became_active":"2023-10-06T05:23:30.136469+0000","last_became_peered":"2023-10-06T05:23:30.136469+0000","last_unstale":"2023-10-06T05:23:30.137550+0000","last_undegraded":"2023-10-06T05:23:30.137550+0000","last_fullsized":"2023-10-06T05:23:30.137550+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:35.784601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.135707+0000","last_change":"2023-10-06T05:23:30.135707+0000","last_active":"2023-10-06T05:23:30.135707+0000","last_peered":"2023-10-06T05:23:30.135707+0000","last_clean":"2023-10-06T05:23:30.135707+0000","last_became_active":"2023-10-06T05:23:30.135404+0000","last_became_peered":"2023-10-06T05:23:30.135404+0000","last_unstale":"2023-10-06T05:23:30.135707+0000","last_undegraded":"2023-10-06T05:23:30.135707+0000","last_fullsized":"2023-10-06T05:23:30.135707+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:25.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.137630+0000","last_change":"2023-10-06T05:23:30.129884+0000","last_active":"2023-10-06T05:23:30.137630+0000","last_peered":"2023-10-06T05:23:30.137630+0000","last_clean":"2023-10-06T05:23:30.137630+0000","last_became_active":"2023-10-06T05:23:30.129642+0000","last_became_peered":"2023-10-06T05:23:30.129642+0000","last_unstale":"2023-10-06T05:23:30.137630+0000","last_undegraded":"2023-10-06T05:23:30.137630+0000","last_fullsized":"2023-10-06T05:23:30.137630+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:05.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.149429+0000","last_change":"2023-10-06T05:23:30.149429+0000","last_active":"2023-10-06T05:23:30.149429+0000","last_peered":"2023-10-06T05:23:30.149429+0000","last_clean":"2023-10-06T05:23:30.149429+0000","last_became_active":"2023-10-06T05:23:30.149120+0000","last_became_peered":"2023-10-06T05:23:30.149120+0000","last_unstale":"2023-10-06T05:23:30.149429+0000","last_undegraded":"2023-10-06T05:23:30.149429+0000","last_fullsized":"2023-10-06T05:23:30.149429+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:34.159767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.156776+0000","last_change":"2023-10-06T05:23:30.156776+0000","last_active":"2023-10-06T05:23:30.156776+0000","last_peered":"2023-10-06T05:23:30.156776+0000","last_clean":"2023-10-06T05:23:30.156776+0000","last_became_active":"2023-10-06T05:23:30.156487+0000","last_became_peered":"2023-10-06T05:23:30.156487+0000","last_unstale":"2023-10-06T05:23:30.156776+0000","last_undegraded":"2023-10-06T05:23:30.156776+0000","last_fullsized":"2023-10-06T05:23:30.156776+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:36.745933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140258+0000","last_change":"2023-10-06T05:23:30.133694+0000","last_active":"2023-10-06T05:23:30.140258+0000","last_peered":"2023-10-06T05:23:30.140258+0000","last_clean":"2023-10-06T05:23:30.140258+0000","last_became_active":"2023-10-06T05:23:30.133360+0000","last_became_peered":"2023-10-06T05:23:30.133360+0000","last_unstale":"2023-10-06T05:23:30.140258+0000","last_undegraded":"2023-10-06T05:23:30.140258+0000","last_fullsized":"2023-10-06T05:23:30.140258+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:08.653017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.136027+0000","last_change":"2023-10-06T05:23:30.136027+0000","last_active":"2023-10-06T05:23:30.136027+0000","last_peered":"2023-10-06T05:23:30.136027+0000","last_clean":"2023-10-06T05:23:30.136027+0000","last_became_active":"2023-10-06T05:23:30.135739+0000","last_became_peered":"2023-10-06T05:23:30.135739+0000","last_unstale":"2023-10-06T05:23:30.136027+0000","last_undegraded":"2023-10-06T05:23:30.136027+0000","last_fullsized":"2023-10-06T05:23:30.136027+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:33.248937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.134129+0000","last_change":"2023-10-06T05:23:30.134129+0000","last_active":"2023-10-06T05:23:30.134129+0000","last_peered":"2023-10-06T05:23:30.134129+0000","last_clean":"2023-10-06T05:23:30.134129+0000","last_became_active":"2023-10-06T05:23:30.133818+0000","last_became_peered":"2023-10-06T05:23:30.133818+0000","last_unstale":"2023-10-06T05:23:30.134129+0000","last_undegraded":"2023-10-06T05:23:30.134129+0000","last_fullsized":"2023-10-06T05:23:30.134129+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:33.243604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.150817+0000","last_change":"2023-10-06T05:23:30.150817+0000","last_active":"2023-10-06T05:23:30.150817+0000","last_peered":"2023-10-06T05:23:30.150817+0000","last_clean":"2023-10-06T05:23:30.150817+0000","last_became_active":"2023-10-06T05:23:30.150532+0000","last_became_peered":"2023-10-06T05:23:30.150532+0000","last_unstale":"2023-10-06T05:23:30.150817+0000","last_undegraded":"2023-10-06T05:23:30.150817+0000","last_fullsized":"2023-10-06T05:23:30.150817+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:43.098998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141285+0000","last_change":"2023-10-06T05:23:30.133408+0000","last_active":"2023-10-06T05:23:30.141285+0000","last_peered":"2023-10-06T05:23:30.141285+0000","last_clean":"2023-10-06T05:23:30.141285+0000","last_became_active":"2023-10-06T05:23:30.133119+0000","last_became_peered":"2023-10-06T05:23:30.133119+0000","last_unstale":"2023-10-06T05:23:30.141285+0000","last_undegraded":"2023-10-06T05:23:30.141285+0000","last_fullsized":"2023-10-06T05:23:30.141285+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:23.188223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.149498+0000","last_change":"2023-10-06T05:23:30.149498+0000","last_active":"2023-10-06T05:23:30.149498+0000","last_peered":"2023-10-06T05:23:30.149498+0000","last_clean":"2023-10-06T05:23:30.149498+0000","last_became_active":"2023-10-06T05:23:30.149129+0000","last_became_peered":"2023-10-06T05:23:30.149129+0000","last_unstale":"2023-10-06T05:23:30.149498+0000","last_undegraded":"2023-10-06T05:23:30.149498+0000","last_fullsized":"2023-10-06T05:23:30.149498+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:08.653017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.143999+0000","last_change":"2023-10-06T05:23:30.140473+0000","last_active":"2023-10-06T05:23:30.143999+0000","last_peered":"2023-10-06T05:23:30.143999+0000","last_clean":"2023-10-06T05:23:30.143999+0000","last_became_active":"2023-10-06T05:23:30.140096+0000","last_became_peered":"2023-10-06T05:23:30.140096+0000","last_unstale":"2023-10-06T05:23:30.143999+0000","last_undegraded":"2023-10-06T05:23:30.143999+0000","last_fullsized":"2023-10-06T05:23:30.143999+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:34.207064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141734+0000","last_change":"2023-10-06T05:23:30.140000+0000","last_active":"2023-10-06T05:23:30.141734+0000","last_peered":"2023-10-06T05:23:30.141734+0000","last_clean":"2023-10-06T05:23:30.141734+0000","last_became_active":"2023-10-06T05:23:30.139712+0000","last_became_peered":"2023-10-06T05:23:30.139712+0000","last_unstale":"2023-10-06T05:23:30.141734+0000","last_undegraded":"2023-10-06T05:23:30.141734+0000","last_fullsized":"2023-10-06T05:23:30.141734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:35.597987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.130091+0000","last_change":"2023-10-06T05:23:30.130091+0000","last_active":"2023-10-06T05:23:30.130091+0000","last_peered":"2023-10-06T05:23:30.130091+0000","last_clean":"2023-10-06T05:23:30.130091+0000","last_became_active":"2023-10-06T05:23:30.129771+0000","last_became_peered":"2023-10-06T05:23:30.129771+0000","last_unstale":"2023-10-06T05:23:30.130091+0000","last_undegraded":"2023-10-06T05:23:30.130091+0000","last_fullsized":"2023-10-06T05:23:30.130091+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:01:38.581343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.129886+0000","last_change":"2023-10-06T05:23:30.129886+0000","last_active":"2023-10-06T05:23:30.129886+0000","last_peered":"2023-10-06T05:23:30.129886+0000","last_clean":"2023-10-06T05:23:30.129886+0000","last_became_active":"2023-10-06T05:23:30.129611+0000","last_became_peered":"2023-10-06T05:23:30.129611+0000","last_unstale":"2023-10-06T05:23:30.129886+0000","last_undegraded":"2023-10-06T05:23:30.129886+0000","last_fullsized":"2023-10-06T05:23:30.129886+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:59.661176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.137688+0000","last_change":"2023-10-06T05:23:30.130802+0000","last_active":"2023-10-06T05:23:30.137688+0000","last_peered":"2023-10-06T05:23:30.137688+0000","last_clean":"2023-10-06T05:23:30.137688+0000","last_became_active":"2023-10-06T05:23:30.130540+0000","last_became_peered":"2023-10-06T05:23:30.130540+0000","last_unstale":"2023-10-06T05:23:30.137688+0000","last_undegraded":"2023-10-06T05:23:30.137688+0000","last_fullsized":"2023-10-06T05:23:30.137688+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:30:55.292129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.136977+0000","last_change":"2023-10-06T05:23:30.136977+0000","last_active":"2023-10-06T05:23:30.136977+0000","last_peered":"2023-10-06T05:23:30.136977+0000","last_clean":"2023-10-06T05:23:30.136977+0000","last_became_active":"2023-10-06T05:23:30.136698+0000","last_became_peered":"2023-10-06T05:23:30.136698+0000","last_unstale":"2023-10-06T05:23:30.136977+0000","last_undegraded":"2023-10-06T05:23:30.136977+0000","last_fullsized":"2023-10-06T05:23:30.136977+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:39.944519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.154543+0000","last_change":"2023-10-06T05:23:30.137942+0000","last_active":"2023-10-06T05:23:30.154543+0000","last_peered":"2023-10-06T05:23:30.154543+0000","last_clean":"2023-10-06T05:23:30.154543+0000","last_became_active":"2023-10-06T05:23:30.137653+0000","last_became_peered":"2023-10-06T05:23:30.137653+0000","last_unstale":"2023-10-06T05:23:30.154543+0000","last_undegraded":"2023-10-06T05:23:30.154543+0000","last_fullsized":"2023-10-06T05:23:30.154543+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:56.407467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.134266+0000","last_change":"2023-10-06T05:23:30.134266+0000","last_active":"2023-10-06T05:23:30.134266+0000","last_peered":"2023-10-06T05:23:30.134266+0000","last_clean":"2023-10-06T05:23:30.134266+0000","last_became_active":"2023-10-06T05:23:30.133962+0000","last_became_peered":"2023-10-06T05:23:30.133962+0000","last_unstale":"2023-10-06T05:23:30.134266+0000","last_undegraded":"2023-10-06T05:23:30.134266+0000","last_fullsized":"2023-10-06T05:23:30.134266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.065283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141725+0000","last_change":"2023-10-06T05:23:30.136345+0000","last_active":"2023-10-06T05:23:30.141725+0000","last_peered":"2023-10-06T05:23:30.141725+0000","last_clean":"2023-10-06T05:23:30.141725+0000","last_became_active":"2023-10-06T05:23:30.136082+0000","last_became_peered":"2023-10-06T05:23:30.136082+0000","last_unstale":"2023-10-06T05:23:30.141725+0000","last_undegraded":"2023-10-06T05:23:30.141725+0000","last_fullsized":"2023-10-06T05:23:30.141725+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:11.703042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140811+0000","last_change":"2023-10-06T05:23:30.140811+0000","last_active":"2023-10-06T05:23:30.140811+0000","last_peered":"2023-10-06T05:23:30.140811+0000","last_clean":"2023-10-06T05:23:30.140811+0000","last_became_active":"2023-10-06T05:23:30.140501+0000","last_became_peered":"2023-10-06T05:23:30.140501+0000","last_unstale":"2023-10-06T05:23:30.140811+0000","last_undegraded":"2023-10-06T05:23:30.140811+0000","last_fullsized":"2023-10-06T05:23:30.140811+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:34.159767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.137003+0000","last_change":"2023-10-06T05:23:30.137003+0000","last_active":"2023-10-06T05:23:30.137003+0000","last_peered":"2023-10-06T05:23:30.137003+0000","last_clean":"2023-10-06T05:23:30.137003+0000","last_became_active":"2023-10-06T05:23:30.136746+0000","last_became_peered":"2023-10-06T05:23:30.136746+0000","last_unstale":"2023-10-06T05:23:30.137003+0000","last_undegraded":"2023-10-06T05:23:30.137003+0000","last_fullsized":"2023-10-06T05:23:30.137003+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:05.514516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.142165+0000","last_change":"2023-10-06T05:23:30.142165+0000","last_active":"2023-10-06T05:23:30.142165+0000","last_peered":"2023-10-06T05:23:30.142165+0000","last_clean":"2023-10-06T05:23:30.142165+0000","last_became_active":"2023-10-06T05:23:30.141897+0000","last_became_peered":"2023-10-06T05:23:30.141897+0000","last_unstale":"2023-10-06T05:23:30.142165+0000","last_undegraded":"2023-10-06T05:23:30.142165+0000","last_fullsized":"2023-10-06T05:23:30.142165+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:33.883796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.148090+0000","last_change":"2023-10-06T05:23:30.148090+0000","last_active":"2023-10-06T05:23:30.148090+0000","last_peered":"2023-10-06T05:23:30.148090+0000","last_clean":"2023-10-06T05:23:30.148090+0000","last_became_active":"2023-10-06T05:23:30.147744+0000","last_became_peered":"2023-10-06T05:23:30.147744+0000","last_unstale":"2023-10-06T05:23:30.148090+0000","last_undegraded":"2023-10-06T05:23:30.148090+0000","last_fullsized":"2023-10-06T05:23:30.148090+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:20:30.015241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.138418+0000","last_change":"2023-10-06T05:23:30.138418+0000","last_active":"2023-10-06T05:23:30.138418+0000","last_peered":"2023-10-06T05:23:30.138418+0000","last_clean":"2023-10-06T05:23:30.138418+0000","last_became_active":"2023-10-06T05:23:30.138139+0000","last_became_peered":"2023-10-06T05:23:30.138139+0000","last_unstale":"2023-10-06T05:23:30.138418+0000","last_undegraded":"2023-10-06T05:23:30.138418+0000","last_fullsized":"2023-10-06T05:23:30.138418+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:42.294545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.138287+0000","last_change":"2023-10-06T05:23:30.138287+0000","last_active":"2023-10-06T05:23:30.138287+0000","last_peered":"2023-10-06T05:23:30.138287+0000","last_clean":"2023-10-06T05:23:30.138287+0000","last_became_active":"2023-10-06T05:23:30.137851+0000","last_became_peered":"2023-10-06T05:23:30.137851+0000","last_unstale":"2023-10-06T05:23:30.138287+0000","last_undegraded":"2023-10-06T05:23:30.138287+0000","last_fullsized":"2023-10-06T05:23:30.138287+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:38.445699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.149225+0000","last_change":"2023-10-06T05:23:30.149225+0000","last_active":"2023-10-06T05:23:30.149225+0000","last_peered":"2023-10-06T05:23:30.149225+0000","last_clean":"2023-10-06T05:23:30.149225+0000","last_became_active":"2023-10-06T05:23:30.148911+0000","last_became_peered":"2023-10-06T05:23:30.148911+0000","last_unstale":"2023-10-06T05:23:30.149225+0000","last_undegraded":"2023-10-06T05:23:30.149225+0000","last_fullsized":"2023-10-06T05:23:30.149225+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:57.634204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141394+0000","last_change":"2023-10-06T05:23:30.141394+0000","last_active":"2023-10-06T05:23:30.141394+0000","last_peered":"2023-10-06T05:23:30.141394+0000","last_clean":"2023-10-06T05:23:30.141394+0000","last_became_active":"2023-10-06T05:23:30.141128+0000","last_became_peered":"2023-10-06T05:23:30.141128+0000","last_unstale":"2023-10-06T05:23:30.141394+0000","last_undegraded":"2023-10-06T05:23:30.141394+0000","last_fullsized":"2023-10-06T05:23:30.141394+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:56.838792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.136295+0000","last_change":"2023-10-06T05:23:30.132199+0000","last_active":"2023-10-06T05:23:30.136295+0000","last_peered":"2023-10-06T05:23:30.136295+0000","last_clean":"2023-10-06T05:23:30.136295+0000","last_became_active":"2023-10-06T05:23:30.130271+0000","last_became_peered":"2023-10-06T05:23:30.130271+0000","last_unstale":"2023-10-06T05:23:30.136295+0000","last_undegraded":"2023-10-06T05:23:30.136295+0000","last_fullsized":"2023-10-06T05:23:30.136295+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.065283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.131230+0000","last_change":"2023-10-06T05:23:30.131230+0000","last_active":"2023-10-06T05:23:30.131230+0000","last_peered":"2023-10-06T05:23:30.131230+0000","last_clean":"2023-10-06T05:23:30.131230+0000","last_became_active":"2023-10-06T05:23:30.130956+0000","last_became_peered":"2023-10-06T05:23:30.130956+0000","last_unstale":"2023-10-06T05:23:30.131230+0000","last_undegraded":"2023-10-06T05:23:30.131230+0000","last_fullsized":"2023-10-06T05:23:30.131230+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:57.471693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.155166+0000","last_change":"2023-10-06T05:23:30.148928+0000","last_active":"2023-10-06T05:23:30.155166+0000","last_peered":"2023-10-06T05:23:30.155166+0000","last_clean":"2023-10-06T05:23:30.155166+0000","last_became_active":"2023-10-06T05:23:30.148645+0000","last_became_peered":"2023-10-06T05:23:30.148645+0000","last_unstale":"2023-10-06T05:23:30.155166+0000","last_undegraded":"2023-10-06T05:23:30.155166+0000","last_fullsized":"2023-10-06T05:23:30.155166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:05.514516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"113'4","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.158491+0000","last_change":"2023-10-06T05:23:29.137035+0000","last_active":"2023-10-06T05:23:30.158491+0000","last_peered":"2023-10-06T05:23:30.158491+0000","last_clean":"2023-10-06T05:23:30.158491+0000","last_became_active":"2023-10-06T05:23:29.136668+0000","last_became_peered":"2023-10-06T05:23:29.136668+0000","last_unstale":"2023-10-06T05:23:30.158491+0000","last_undegraded":"2023-10-06T05:23:30.158491+0000","last_fullsized":"2023-10-06T05:23:30.158491+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:00.418916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141707+0000","last_change":"2023-10-06T05:23:30.136346+0000","last_active":"2023-10-06T05:23:30.141707+0000","last_peered":"2023-10-06T05:23:30.141707+0000","last_clean":"2023-10-06T05:23:30.141707+0000","last_became_active":"2023-10-06T05:23:30.136105+0000","last_became_peered":"2023-10-06T05:23:30.136105+0000","last_unstale":"2023-10-06T05:23:30.141707+0000","last_undegraded":"2023-10-06T05:23:30.141707+0000","last_fullsized":"2023-10-06T05:23:30.141707+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:26.269530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.158053+0000","last_change":"2023-10-06T05:23:29.136838+0000","last_active":"2023-10-06T05:23:30.158053+0000","last_peered":"2023-10-06T05:23:30.158053+0000","last_clean":"2023-10-06T05:23:30.158053+0000","last_became_active":"2023-10-06T05:23:29.136527+0000","last_became_peered":"2023-10-06T05:23:29.136527+0000","last_unstale":"2023-10-06T05:23:30.158053+0000","last_undegraded":"2023-10-06T05:23:30.158053+0000","last_fullsized":"2023-10-06T05:23:30.158053+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:47.815436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140221+0000","last_change":"2023-10-06T05:23:30.140221+0000","last_active":"2023-10-06T05:23:30.140221+0000","last_peered":"2023-10-06T05:23:30.140221+0000","last_clean":"2023-10-06T05:23:30.140221+0000","last_became_active":"2023-10-06T05:23:30.139934+0000","last_became_peered":"2023-10-06T05:23:30.139934+0000","last_unstale":"2023-10-06T05:23:30.140221+0000","last_undegraded":"2023-10-06T05:23:30.140221+0000","last_fullsized":"2023-10-06T05:23:30.140221+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:33.248937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"113'2","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.156236+0000","last_change":"2023-10-06T05:23:29.133876+0000","last_active":"2023-10-06T05:23:30.156236+0000","last_peered":"2023-10-06T05:23:30.156236+0000","last_clean":"2023-10-06T05:23:30.156236+0000","last_became_active":"2023-10-06T05:23:29.133509+0000","last_became_peered":"2023-10-06T05:23:29.133509+0000","last_unstale":"2023-10-06T05:23:30.156236+0000","last_undegraded":"2023-10-06T05:23:30.156236+0000","last_fullsized":"2023-10-06T05:23:30.156236+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:05.097160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.144536+0000","last_change":"2023-10-06T05:23:30.142230+0000","last_active":"2023-10-06T05:23:30.144536+0000","last_peered":"2023-10-06T05:23:30.144536+0000","last_clean":"2023-10-06T05:23:30.144536+0000","last_became_active":"2023-10-06T05:23:30.141968+0000","last_became_peered":"2023-10-06T05:23:30.141968+0000","last_unstale":"2023-10-06T05:23:30.144536+0000","last_undegraded":"2023-10-06T05:23:30.144536+0000","last_fullsized":"2023-10-06T05:23:30.144536+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:26.277530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"113'2","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.160556+0000","last_change":"2023-10-06T05:23:29.133699+0000","last_active":"2023-10-06T05:23:30.160556+0000","last_peered":"2023-10-06T05:23:30.160556+0000","last_clean":"2023-10-06T05:23:30.160556+0000","last_became_active":"2023-10-06T05:23:29.133270+0000","last_became_peered":"2023-10-06T05:23:29.133270+0000","last_unstale":"2023-10-06T05:23:30.160556+0000","last_undegraded":"2023-10-06T05:23:30.160556+0000","last_fullsized":"2023-10-06T05:23:30.160556+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:52.377269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.136344+0000","last_change":"2023-10-06T05:23:30.131605+0000","last_active":"2023-10-06T05:23:30.136344+0000","last_peered":"2023-10-06T05:23:30.136344+0000","last_clean":"2023-10-06T05:23:30.136344+0000","last_became_active":"2023-10-06T05:23:30.131320+0000","last_became_peered":"2023-10-06T05:23:30.131320+0000","last_unstale":"2023-10-06T05:23:30.136344+0000","last_undegraded":"2023-10-06T05:23:30.136344+0000","last_fullsized":"2023-10-06T05:23:30.136344+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:06.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.158395+0000","last_change":"2023-10-06T05:23:29.136365+0000","last_active":"2023-10-06T05:23:30.158395+0000","last_peered":"2023-10-06T05:23:30.158395+0000","last_clean":"2023-10-06T05:23:30.158395+0000","last_became_active":"2023-10-06T05:23:29.136076+0000","last_became_peered":"2023-10-06T05:23:29.136076+0000","last_unstale":"2023-10-06T05:23:30.158395+0000","last_undegraded":"2023-10-06T05:23:30.158395+0000","last_fullsized":"2023-10-06T05:23:30.158395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:45.591769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140318+0000","last_change":"2023-10-06T05:23:30.134366+0000","last_active":"2023-10-06T05:23:30.140318+0000","last_peered":"2023-10-06T05:23:30.140318+0000","last_clean":"2023-10-06T05:23:30.140318+0000","last_became_active":"2023-10-06T05:23:30.134065+0000","last_became_peered":"2023-10-06T05:23:30.134065+0000","last_unstale":"2023-10-06T05:23:30.140318+0000","last_undegraded":"2023-10-06T05:23:30.140318+0000","last_fullsized":"2023-10-06T05:23:30.140318+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:33.248937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"113'1","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.158385+0000","last_change":"2023-10-06T05:23:29.127493+0000","last_active":"2023-10-06T05:23:30.158385+0000","last_peered":"2023-10-06T05:23:30.158385+0000","last_clean":"2023-10-06T05:23:30.158385+0000","last_became_active":"2023-10-06T05:23:29.127238+0000","last_became_peered":"2023-10-06T05:23:29.127238+0000","last_unstale":"2023-10-06T05:23:30.158385+0000","last_undegraded":"2023-10-06T05:23:30.158385+0000","last_fullsized":"2023-10-06T05:23:30.158385+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:10.622995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.155680+0000","last_change":"2023-10-06T05:23:30.140809+0000","last_active":"2023-10-06T05:23:30.155680+0000","last_peered":"2023-10-06T05:23:30.155680+0000","last_clean":"2023-10-06T05:23:30.155680+0000","last_became_active":"2023-10-06T05:23:30.140518+0000","last_became_peered":"2023-10-06T05:23:30.140518+0000","last_unstale":"2023-10-06T05:23:30.155680+0000","last_undegraded":"2023-10-06T05:23:30.155680+0000","last_fullsized":"2023-10-06T05:23:30.155680+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:35.939769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.130156+0000","last_change":"2023-10-06T05:23:29.124548+0000","last_active":"2023-10-06T05:23:29.130156+0000","last_peered":"2023-10-06T05:23:29.130156+0000","last_clean":"2023-10-06T05:23:29.130156+0000","last_became_active":"2023-10-06T05:23:29.124178+0000","last_became_peered":"2023-10-06T05:23:29.124178+0000","last_unstale":"2023-10-06T05:23:29.130156+0000","last_undegraded":"2023-10-06T05:23:29.130156+0000","last_fullsized":"2023-10-06T05:23:29.130156+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:37.980654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.138062+0000","last_change":"2023-10-06T05:23:30.138062+0000","last_active":"2023-10-06T05:23:30.138062+0000","last_peered":"2023-10-06T05:23:30.138062+0000","last_clean":"2023-10-06T05:23:30.138062+0000","last_became_active":"2023-10-06T05:23:30.137735+0000","last_became_peered":"2023-10-06T05:23:30.137735+0000","last_unstale":"2023-10-06T05:23:30.138062+0000","last_undegraded":"2023-10-06T05:23:30.138062+0000","last_fullsized":"2023-10-06T05:23:30.138062+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:13.176765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.141048+0000","last_change":"2023-10-06T05:23:29.137049+0000","last_active":"2023-10-06T05:23:29.141048+0000","last_peered":"2023-10-06T05:23:29.141048+0000","last_clean":"2023-10-06T05:23:29.141048+0000","last_became_active":"2023-10-06T05:23:29.136671+0000","last_became_peered":"2023-10-06T05:23:29.136671+0000","last_unstale":"2023-10-06T05:23:29.141048+0000","last_undegraded":"2023-10-06T05:23:29.141048+0000","last_fullsized":"2023-10-06T05:23:29.141048+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:14.856202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.136905+0000","last_change":"2023-10-06T05:23:30.136905+0000","last_active":"2023-10-06T05:23:30.136905+0000","last_peered":"2023-10-06T05:23:30.136905+0000","last_clean":"2023-10-06T05:23:30.136905+0000","last_became_active":"2023-10-06T05:23:30.136438+0000","last_became_peered":"2023-10-06T05:23:30.136438+0000","last_unstale":"2023-10-06T05:23:30.136905+0000","last_undegraded":"2023-10-06T05:23:30.136905+0000","last_fullsized":"2023-10-06T05:23:30.136905+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:09.638864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"113'1","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.125579+0000","last_change":"2023-10-06T05:23:29.135237+0000","last_active":"2023-10-06T05:23:30.125579+0000","last_peered":"2023-10-06T05:23:30.125579+0000","last_clean":"2023-10-06T05:23:30.125579+0000","last_became_active":"2023-10-06T05:23:29.134930+0000","last_became_peered":"2023-10-06T05:23:29.134930+0000","last_unstale":"2023-10-06T05:23:30.125579+0000","last_undegraded":"2023-10-06T05:23:30.125579+0000","last_fullsized":"2023-10-06T05:23:30.125579+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:45.591769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.132092+0000","last_change":"2023-10-06T05:23:30.132092+0000","last_active":"2023-10-06T05:23:30.132092+0000","last_peered":"2023-10-06T05:23:30.132092+0000","last_clean":"2023-10-06T05:23:30.132092+0000","last_became_active":"2023-10-06T05:23:30.131834+0000","last_became_peered":"2023-10-06T05:23:30.131834+0000","last_unstale":"2023-10-06T05:23:30.132092+0000","last_undegraded":"2023-10-06T05:23:30.132092+0000","last_fullsized":"2023-10-06T05:23:30.132092+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:30.983647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"113'2","reported_seq":11,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.156346+0000","last_change":"2023-10-06T05:23:29.136280+0000","last_active":"2023-10-06T05:23:30.156346+0000","last_peered":"2023-10-06T05:23:30.156346+0000","last_clean":"2023-10-06T05:23:30.156346+0000","last_became_active":"2023-10-06T05:23:29.130976+0000","last_became_peered":"2023-10-06T05:23:29.130976+0000","last_unstale":"2023-10-06T05:23:30.156346+0000","last_undegraded":"2023-10-06T05:23:30.156346+0000","last_fullsized":"2023-10-06T05:23:30.156346+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:50.624645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.139406+0000","last_change":"2023-10-06T05:23:30.139406+0000","last_active":"2023-10-06T05:23:30.139406+0000","last_peered":"2023-10-06T05:23:30.139406+0000","last_clean":"2023-10-06T05:23:30.139406+0000","last_became_active":"2023-10-06T05:23:30.139007+0000","last_became_peered":"2023-10-06T05:23:30.139007+0000","last_unstale":"2023-10-06T05:23:30.139406+0000","last_undegraded":"2023-10-06T05:23:30.139406+0000","last_fullsized":"2023-10-06T05:23:30.139406+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:52.725371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"113'1","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.120313+0000","last_change":"2023-10-06T05:23:29.124137+0000","last_active":"2023-10-06T05:23:30.120313+0000","last_peered":"2023-10-06T05:23:30.120313+0000","last_clean":"2023-10-06T05:23:30.120313+0000","last_became_active":"2023-10-06T05:23:29.123822+0000","last_became_peered":"2023-10-06T05:23:29.123822+0000","last_unstale":"2023-10-06T05:23:30.120313+0000","last_undegraded":"2023-10-06T05:23:30.120313+0000","last_fullsized":"2023-10-06T05:23:30.120313+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:53:15.126665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.150279+0000","last_change":"2023-10-06T05:23:30.150279+0000","last_active":"2023-10-06T05:23:30.150279+0000","last_peered":"2023-10-06T05:23:30.150279+0000","last_clean":"2023-10-06T05:23:30.150279+0000","last_became_active":"2023-10-06T05:23:30.150008+0000","last_became_peered":"2023-10-06T05:23:30.150008+0000","last_unstale":"2023-10-06T05:23:30.150279+0000","last_undegraded":"2023-10-06T05:23:30.150279+0000","last_fullsized":"2023-10-06T05:23:30.150279+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:21.046071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.123568+0000","last_change":"2023-10-06T05:23:29.129736+0000","last_active":"2023-10-06T05:23:30.123568+0000","last_peered":"2023-10-06T05:23:30.123568+0000","last_clean":"2023-10-06T05:23:30.123568+0000","last_became_active":"2023-10-06T05:23:29.129414+0000","last_became_peered":"2023-10-06T05:23:29.129414+0000","last_unstale":"2023-10-06T05:23:30.123568+0000","last_undegraded":"2023-10-06T05:23:30.123568+0000","last_fullsized":"2023-10-06T05:23:30.123568+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:37.933377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.130937+0000","last_change":"2023-10-06T05:23:30.128248+0000","last_active":"2023-10-06T05:23:30.130937+0000","last_peered":"2023-10-06T05:23:30.130937+0000","last_clean":"2023-10-06T05:23:30.130937+0000","last_became_active":"2023-10-06T05:23:30.127963+0000","last_became_peered":"2023-10-06T05:23:30.127963+0000","last_unstale":"2023-10-06T05:23:30.130937+0000","last_undegraded":"2023-10-06T05:23:30.130937+0000","last_fullsized":"2023-10-06T05:23:30.130937+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:05.884400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.160389+0000","last_change":"2023-10-06T05:23:29.138352+0000","last_active":"2023-10-06T05:23:30.160389+0000","last_peered":"2023-10-06T05:23:30.160389+0000","last_clean":"2023-10-06T05:23:30.160389+0000","last_became_active":"2023-10-06T05:23:29.138114+0000","last_became_peered":"2023-10-06T05:23:29.138114+0000","last_unstale":"2023-10-06T05:23:30.160389+0000","last_undegraded":"2023-10-06T05:23:30.160389+0000","last_fullsized":"2023-10-06T05:23:30.160389+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:04.794201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141769+0000","last_change":"2023-10-06T05:23:30.140210+0000","last_active":"2023-10-06T05:23:30.141769+0000","last_peered":"2023-10-06T05:23:30.141769+0000","last_clean":"2023-10-06T05:23:30.141769+0000","last_became_active":"2023-10-06T05:23:30.139918+0000","last_became_peered":"2023-10-06T05:23:30.139918+0000","last_unstale":"2023-10-06T05:23:30.141769+0000","last_undegraded":"2023-10-06T05:23:30.141769+0000","last_fullsized":"2023-10-06T05:23:30.141769+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:18.048314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.126678+0000","last_change":"2023-10-06T05:23:29.129688+0000","last_active":"2023-10-06T05:23:30.126678+0000","last_peered":"2023-10-06T05:23:30.126678+0000","last_clean":"2023-10-06T05:23:30.126678+0000","last_became_active":"2023-10-06T05:23:29.129401+0000","last_became_peered":"2023-10-06T05:23:29.129401+0000","last_unstale":"2023-10-06T05:23:30.126678+0000","last_undegraded":"2023-10-06T05:23:30.126678+0000","last_fullsized":"2023-10-06T05:23:30.126678+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:11.185301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.155703+0000","last_change":"2023-10-06T05:23:30.149969+0000","last_active":"2023-10-06T05:23:30.155703+0000","last_peered":"2023-10-06T05:23:30.155703+0000","last_clean":"2023-10-06T05:23:30.155703+0000","last_became_active":"2023-10-06T05:23:30.149691+0000","last_became_peered":"2023-10-06T05:23:30.149691+0000","last_unstale":"2023-10-06T05:23:30.155703+0000","last_undegraded":"2023-10-06T05:23:30.155703+0000","last_fullsized":"2023-10-06T05:23:30.155703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:30.983647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.139434+0000","last_change":"2023-10-06T05:23:29.134522+0000","last_active":"2023-10-06T05:23:29.139434+0000","last_peered":"2023-10-06T05:23:29.139434+0000","last_clean":"2023-10-06T05:23:29.139434+0000","last_became_active":"2023-10-06T05:23:29.134219+0000","last_became_peered":"2023-10-06T05:23:29.134219+0000","last_unstale":"2023-10-06T05:23:29.139434+0000","last_undegraded":"2023-10-06T05:23:29.139434+0000","last_fullsized":"2023-10-06T05:23:29.139434+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:36.325949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.151402+0000","last_change":"2023-10-06T05:23:30.151402+0000","last_active":"2023-10-06T05:23:30.151402+0000","last_peered":"2023-10-06T05:23:30.151402+0000","last_clean":"2023-10-06T05:23:30.151402+0000","last_became_active":"2023-10-06T05:23:30.151143+0000","last_became_peered":"2023-10-06T05:23:30.151143+0000","last_unstale":"2023-10-06T05:23:30.151402+0000","last_undegraded":"2023-10-06T05:23:30.151402+0000","last_fullsized":"2023-10-06T05:23:30.151402+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:06.440258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.140071+0000","last_change":"2023-10-06T05:23:29.136217+0000","last_active":"2023-10-06T05:23:29.140071+0000","last_peered":"2023-10-06T05:23:29.140071+0000","last_clean":"2023-10-06T05:23:29.140071+0000","last_became_active":"2023-10-06T05:23:29.135829+0000","last_became_peered":"2023-10-06T05:23:29.135829+0000","last_unstale":"2023-10-06T05:23:29.140071+0000","last_undegraded":"2023-10-06T05:23:29.140071+0000","last_fullsized":"2023-10-06T05:23:29.140071+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:56.628408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.154632+0000","last_change":"2023-10-06T05:23:30.140251+0000","last_active":"2023-10-06T05:23:30.154632+0000","last_peered":"2023-10-06T05:23:30.154632+0000","last_clean":"2023-10-06T05:23:30.154632+0000","last_became_active":"2023-10-06T05:23:30.140003+0000","last_became_peered":"2023-10-06T05:23:30.140003+0000","last_unstale":"2023-10-06T05:23:30.154632+0000","last_undegraded":"2023-10-06T05:23:30.154632+0000","last_fullsized":"2023-10-06T05:23:30.154632+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:55.301787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.155788+0000","last_change":"2023-10-06T05:23:29.139095+0000","last_active":"2023-10-06T05:23:30.155788+0000","last_peered":"2023-10-06T05:23:30.155788+0000","last_clean":"2023-10-06T05:23:30.155788+0000","last_became_active":"2023-10-06T05:23:29.138833+0000","last_became_peered":"2023-10-06T05:23:29.138833+0000","last_unstale":"2023-10-06T05:23:30.155788+0000","last_undegraded":"2023-10-06T05:23:30.155788+0000","last_fullsized":"2023-10-06T05:23:30.155788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:16.485871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.147573+0000","last_change":"2023-10-06T05:23:30.147573+0000","last_active":"2023-10-06T05:23:30.147573+0000","last_peered":"2023-10-06T05:23:30.147573+0000","last_clean":"2023-10-06T05:23:30.147573+0000","last_became_active":"2023-10-06T05:23:30.147279+0000","last_became_peered":"2023-10-06T05:23:30.147279+0000","last_unstale":"2023-10-06T05:23:30.147573+0000","last_undegraded":"2023-10-06T05:23:30.147573+0000","last_fullsized":"2023-10-06T05:23:30.147573+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:31.416020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.119929+0000","last_change":"2023-10-06T05:23:29.133773+0000","last_active":"2023-10-06T05:23:30.119929+0000","last_peered":"2023-10-06T05:23:30.119929+0000","last_clean":"2023-10-06T05:23:30.119929+0000","last_became_active":"2023-10-06T05:23:29.133388+0000","last_became_peered":"2023-10-06T05:23:29.133388+0000","last_unstale":"2023-10-06T05:23:30.119929+0000","last_undegraded":"2023-10-06T05:23:30.119929+0000","last_fullsized":"2023-10-06T05:23:30.119929+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:10.132750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.153869+0000","last_change":"2023-10-06T05:23:30.153869+0000","last_active":"2023-10-06T05:23:30.153869+0000","last_peered":"2023-10-06T05:23:30.153869+0000","last_clean":"2023-10-06T05:23:30.153869+0000","last_became_active":"2023-10-06T05:23:30.153594+0000","last_became_peered":"2023-10-06T05:23:30.153594+0000","last_unstale":"2023-10-06T05:23:30.153869+0000","last_undegraded":"2023-10-06T05:23:30.153869+0000","last_fullsized":"2023-10-06T05:23:30.153869+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:51.494448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.120650+0000","last_change":"2023-10-06T05:23:29.130788+0000","last_active":"2023-10-06T05:23:30.120650+0000","last_peered":"2023-10-06T05:23:30.120650+0000","last_clean":"2023-10-06T05:23:30.120650+0000","last_became_active":"2023-10-06T05:23:29.130480+0000","last_became_peered":"2023-10-06T05:23:29.130480+0000","last_unstale":"2023-10-06T05:23:30.120650+0000","last_undegraded":"2023-10-06T05:23:30.120650+0000","last_fullsized":"2023-10-06T05:23:30.120650+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:02.324057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.137782+0000","last_change":"2023-10-06T05:23:30.137782+0000","last_active":"2023-10-06T05:23:30.137782+0000","last_peered":"2023-10-06T05:23:30.137782+0000","last_clean":"2023-10-06T05:23:30.137782+0000","last_became_active":"2023-10-06T05:23:30.137487+0000","last_became_peered":"2023-10-06T05:23:30.137487+0000","last_unstale":"2023-10-06T05:23:30.137782+0000","last_undegraded":"2023-10-06T05:23:30.137782+0000","last_fullsized":"2023-10-06T05:23:30.137782+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:47.002807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.133073+0000","last_change":"2023-10-06T05:23:29.132141+0000","last_active":"2023-10-06T05:23:29.133073+0000","last_peered":"2023-10-06T05:23:29.133073+0000","last_clean":"2023-10-06T05:23:29.133073+0000","last_became_active":"2023-10-06T05:23:29.131846+0000","last_became_peered":"2023-10-06T05:23:29.131846+0000","last_unstale":"2023-10-06T05:23:29.133073+0000","last_undegraded":"2023-10-06T05:23:29.133073+0000","last_fullsized":"2023-10-06T05:23:29.133073+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:12.291784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.134442+0000","last_change":"2023-10-06T05:23:30.134442+0000","last_active":"2023-10-06T05:23:30.134442+0000","last_peered":"2023-10-06T05:23:30.134442+0000","last_clean":"2023-10-06T05:23:30.134442+0000","last_became_active":"2023-10-06T05:23:30.134168+0000","last_became_peered":"2023-10-06T05:23:30.134168+0000","last_unstale":"2023-10-06T05:23:30.134442+0000","last_undegraded":"2023-10-06T05:23:30.134442+0000","last_fullsized":"2023-10-06T05:23:30.134442+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:06.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.139101+0000","last_change":"2023-10-06T05:23:29.132923+0000","last_active":"2023-10-06T05:23:29.139101+0000","last_peered":"2023-10-06T05:23:29.139101+0000","last_clean":"2023-10-06T05:23:29.139101+0000","last_became_active":"2023-10-06T05:23:29.132661+0000","last_became_peered":"2023-10-06T05:23:29.132661+0000","last_unstale":"2023-10-06T05:23:29.139101+0000","last_undegraded":"2023-10-06T05:23:29.139101+0000","last_fullsized":"2023-10-06T05:23:29.139101+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:04.794201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.134639+0000","last_change":"2023-10-06T05:23:30.134639+0000","last_active":"2023-10-06T05:23:30.134639+0000","last_peered":"2023-10-06T05:23:30.134639+0000","last_clean":"2023-10-06T05:23:30.134639+0000","last_became_active":"2023-10-06T05:23:30.134373+0000","last_became_peered":"2023-10-06T05:23:30.134373+0000","last_unstale":"2023-10-06T05:23:30.134639+0000","last_undegraded":"2023-10-06T05:23:30.134639+0000","last_fullsized":"2023-10-06T05:23:30.134639+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:38.065283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.139174+0000","last_change":"2023-10-06T05:23:29.134462+0000","last_active":"2023-10-06T05:23:29.139174+0000","last_peered":"2023-10-06T05:23:29.139174+0000","last_clean":"2023-10-06T05:23:29.139174+0000","last_became_active":"2023-10-06T05:23:29.133452+0000","last_became_peered":"2023-10-06T05:23:29.133452+0000","last_unstale":"2023-10-06T05:23:29.139174+0000","last_undegraded":"2023-10-06T05:23:29.139174+0000","last_fullsized":"2023-10-06T05:23:29.139174+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:53:40.296147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140995+0000","last_change":"2023-10-06T05:23:30.140995+0000","last_active":"2023-10-06T05:23:30.140995+0000","last_peered":"2023-10-06T05:23:30.140995+0000","last_clean":"2023-10-06T05:23:30.140995+0000","last_became_active":"2023-10-06T05:23:30.140689+0000","last_became_peered":"2023-10-06T05:23:30.140689+0000","last_unstale":"2023-10-06T05:23:30.140995+0000","last_undegraded":"2023-10-06T05:23:30.140995+0000","last_fullsized":"2023-10-06T05:23:30.140995+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:04.795929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.133062+0000","last_change":"2023-10-06T05:23:29.131644+0000","last_active":"2023-10-06T05:23:29.133062+0000","last_peered":"2023-10-06T05:23:29.133062+0000","last_clean":"2023-10-06T05:23:29.133062+0000","last_became_active":"2023-10-06T05:23:29.131345+0000","last_became_peered":"2023-10-06T05:23:29.131345+0000","last_unstale":"2023-10-06T05:23:29.133062+0000","last_undegraded":"2023-10-06T05:23:29.133062+0000","last_fullsized":"2023-10-06T05:23:29.133062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:20.579285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.156341+0000","last_change":"2023-10-06T05:23:30.156341+0000","last_active":"2023-10-06T05:23:30.156341+0000","last_peered":"2023-10-06T05:23:30.156341+0000","last_clean":"2023-10-06T05:23:30.156341+0000","last_became_active":"2023-10-06T05:23:30.156044+0000","last_became_peered":"2023-10-06T05:23:30.156044+0000","last_unstale":"2023-10-06T05:23:30.156341+0000","last_undegraded":"2023-10-06T05:23:30.156341+0000","last_fullsized":"2023-10-06T05:23:30.156341+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:06.719958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.123067+0000","last_change":"2023-10-06T05:23:29.138784+0000","last_active":"2023-10-06T05:23:30.123067+0000","last_peered":"2023-10-06T05:23:30.123067+0000","last_clean":"2023-10-06T05:23:30.123067+0000","last_became_active":"2023-10-06T05:23:29.138475+0000","last_became_peered":"2023-10-06T05:23:29.138475+0000","last_unstale":"2023-10-06T05:23:30.123067+0000","last_undegraded":"2023-10-06T05:23:30.123067+0000","last_fullsized":"2023-10-06T05:23:30.123067+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:52.462512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.149780+0000","last_change":"2023-10-06T05:23:30.149780+0000","last_active":"2023-10-06T05:23:30.149780+0000","last_peered":"2023-10-06T05:23:30.149780+0000","last_clean":"2023-10-06T05:23:30.149780+0000","last_became_active":"2023-10-06T05:23:30.149488+0000","last_became_peered":"2023-10-06T05:23:30.149488+0000","last_unstale":"2023-10-06T05:23:30.149780+0000","last_undegraded":"2023-10-06T05:23:30.149780+0000","last_fullsized":"2023-10-06T05:23:30.149780+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:03.550372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"113'3","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.160203+0000","last_change":"2023-10-06T05:23:29.137778+0000","last_active":"2023-10-06T05:23:30.160203+0000","last_peered":"2023-10-06T05:23:30.160203+0000","last_clean":"2023-10-06T05:23:30.160203+0000","last_became_active":"2023-10-06T05:23:29.137411+0000","last_became_peered":"2023-10-06T05:23:29.137411+0000","last_unstale":"2023-10-06T05:23:30.160203+0000","last_undegraded":"2023-10-06T05:23:30.160203+0000","last_fullsized":"2023-10-06T05:23:30.160203+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:43.374922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.132186+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:23:29.132186+0000","last_peered":"2023-10-06T05:23:29.132186+0000","last_clean":"2023-10-06T05:23:29.132186+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:23:29.132186+0000","last_undegraded":"2023-10-06T05:23:29.132186+0000","last_fullsized":"2023-10-06T05:23:29.132186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140403+0000","last_change":"2023-10-06T05:23:30.140403+0000","last_active":"2023-10-06T05:23:30.140403+0000","last_peered":"2023-10-06T05:23:30.140403+0000","last_clean":"2023-10-06T05:23:30.140403+0000","last_became_active":"2023-10-06T05:23:30.138925+0000","last_became_peered":"2023-10-06T05:23:30.138925+0000","last_unstale":"2023-10-06T05:23:30.140403+0000","last_undegraded":"2023-10-06T05:23:30.140403+0000","last_fullsized":"2023-10-06T05:23:30.140403+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:16.944912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"113'6","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.160728+0000","last_change":"2023-10-06T05:23:29.133359+0000","last_active":"2023-10-06T05:23:30.160728+0000","last_peered":"2023-10-06T05:23:30.160728+0000","last_clean":"2023-10-06T05:23:30.160728+0000","last_became_active":"2023-10-06T05:23:29.133060+0000","last_became_peered":"2023-10-06T05:23:29.133060+0000","last_unstale":"2023-10-06T05:23:30.160728+0000","last_undegraded":"2023-10-06T05:23:30.160728+0000","last_fullsized":"2023-10-06T05:23:30.160728+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:36.655702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.130479+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:23:29.130479+0000","last_peered":"2023-10-06T05:23:29.130479+0000","last_clean":"2023-10-06T05:23:29.130479+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:23:29.130479+0000","last_undegraded":"2023-10-06T05:23:29.130479+0000","last_fullsized":"2023-10-06T05:23:29.130479+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.138111+0000","last_change":"2023-10-06T05:23:30.138111+0000","last_active":"2023-10-06T05:23:30.138111+0000","last_peered":"2023-10-06T05:23:30.138111+0000","last_clean":"2023-10-06T05:23:30.138111+0000","last_became_active":"2023-10-06T05:23:30.137277+0000","last_became_peered":"2023-10-06T05:23:30.137277+0000","last_unstale":"2023-10-06T05:23:30.138111+0000","last_undegraded":"2023-10-06T05:23:30.138111+0000","last_fullsized":"2023-10-06T05:23:30.138111+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:20.345883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"113'2","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.154285+0000","last_change":"2023-10-06T05:23:29.137861+0000","last_active":"2023-10-06T05:23:30.154285+0000","last_peered":"2023-10-06T05:23:30.154285+0000","last_clean":"2023-10-06T05:23:30.154285+0000","last_became_active":"2023-10-06T05:23:29.137578+0000","last_became_peered":"2023-10-06T05:23:29.137578+0000","last_unstale":"2023-10-06T05:23:30.154285+0000","last_undegraded":"2023-10-06T05:23:30.154285+0000","last_fullsized":"2023-10-06T05:23:30.154285+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:24.006184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.122660+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:23:29.122660+0000","last_peered":"2023-10-06T05:23:29.122660+0000","last_clean":"2023-10-06T05:23:29.122660+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:23:29.122660+0000","last_undegraded":"2023-10-06T05:23:29.122660+0000","last_fullsized":"2023-10-06T05:23:29.122660+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.148510+0000","last_change":"2023-10-06T05:23:30.148510+0000","last_active":"2023-10-06T05:23:30.148510+0000","last_peered":"2023-10-06T05:23:30.148510+0000","last_clean":"2023-10-06T05:23:30.148510+0000","last_became_active":"2023-10-06T05:23:30.147288+0000","last_became_peered":"2023-10-06T05:23:30.147288+0000","last_unstale":"2023-10-06T05:23:30.148510+0000","last_undegraded":"2023-10-06T05:23:30.148510+0000","last_fullsized":"2023-10-06T05:23:30.148510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:02.373765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.139154+0000","last_change":"2023-10-06T05:23:29.133776+0000","last_active":"2023-10-06T05:23:29.139154+0000","last_peered":"2023-10-06T05:23:29.139154+0000","last_clean":"2023-10-06T05:23:29.139154+0000","last_became_active":"2023-10-06T05:23:29.133099+0000","last_became_peered":"2023-10-06T05:23:29.133099+0000","last_unstale":"2023-10-06T05:23:29.139154+0000","last_undegraded":"2023-10-06T05:23:29.139154+0000","last_fullsized":"2023-10-06T05:23:29.139154+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:50.972785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.123248+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:23:29.123248+0000","last_peered":"2023-10-06T05:23:29.123248+0000","last_clean":"2023-10-06T05:23:29.123248+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:23:29.123248+0000","last_undegraded":"2023-10-06T05:23:29.123248+0000","last_fullsized":"2023-10-06T05:23:29.123248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.140732+0000","last_change":"2023-10-06T05:23:30.132892+0000","last_active":"2023-10-06T05:23:30.140732+0000","last_peered":"2023-10-06T05:23:30.140732+0000","last_clean":"2023-10-06T05:23:30.140732+0000","last_became_active":"2023-10-06T05:23:30.132541+0000","last_became_peered":"2023-10-06T05:23:30.132541+0000","last_unstale":"2023-10-06T05:23:30.140732+0000","last_undegraded":"2023-10-06T05:23:30.140732+0000","last_fullsized":"2023-10-06T05:23:30.140732+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:18.606201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"113'8","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.159615+0000","last_change":"2023-10-06T05:23:29.139053+0000","last_active":"2023-10-06T05:23:30.159615+0000","last_peered":"2023-10-06T05:23:30.159615+0000","last_clean":"2023-10-06T05:23:30.159615+0000","last_became_active":"2023-10-06T05:23:29.138771+0000","last_became_peered":"2023-10-06T05:23:29.138771+0000","last_unstale":"2023-10-06T05:23:30.159615+0000","last_undegraded":"2023-10-06T05:23:30.159615+0000","last_fullsized":"2023-10-06T05:23:30.159615+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:53.801548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":86,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.124185+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:23:29.124185+0000","last_peered":"2023-10-06T05:23:29.124185+0000","last_clean":"2023-10-06T05:23:29.124185+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:23:29.124185+0000","last_undegraded":"2023-10-06T05:23:29.124185+0000","last_fullsized":"2023-10-06T05:23:29.124185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.143656+0000","last_change":"2023-10-06T05:23:30.143656+0000","last_active":"2023-10-06T05:23:30.143656+0000","last_peered":"2023-10-06T05:23:30.143656+0000","last_clean":"2023-10-06T05:23:30.143656+0000","last_became_active":"2023-10-06T05:23:30.139683+0000","last_became_peered":"2023-10-06T05:23:30.139683+0000","last_unstale":"2023-10-06T05:23:30.143656+0000","last_undegraded":"2023-10-06T05:23:30.143656+0000","last_fullsized":"2023-10-06T05:23:30.143656+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:46.439071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.144011+0000","last_change":"2023-10-06T05:23:29.140252+0000","last_active":"2023-10-06T05:23:29.144011+0000","last_peered":"2023-10-06T05:23:29.144011+0000","last_clean":"2023-10-06T05:23:29.144011+0000","last_became_active":"2023-10-06T05:23:29.139946+0000","last_became_peered":"2023-10-06T05:23:29.139946+0000","last_unstale":"2023-10-06T05:23:29.144011+0000","last_undegraded":"2023-10-06T05:23:29.144011+0000","last_fullsized":"2023-10-06T05:23:29.144011+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:55.604837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":275,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.123161+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:23:29.123161+0000","last_peered":"2023-10-06T05:23:29.123161+0000","last_clean":"2023-10-06T05:23:29.123161+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:23:29.123161+0000","last_undegraded":"2023-10-06T05:23:29.123161+0000","last_fullsized":"2023-10-06T05:23:29.123161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":94,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.114510+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:23:29.114510+0000","last_peered":"2023-10-06T05:23:29.114510+0000","last_clean":"2023-10-06T05:23:29.114510+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:23:29.114510+0000","last_undegraded":"2023-10-06T05:23:29.114510+0000","last_fullsized":"2023-10-06T05:23:29.114510+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.130471+0000","last_change":"2023-10-06T05:23:30.130471+0000","last_active":"2023-10-06T05:23:30.130471+0000","last_peered":"2023-10-06T05:23:30.130471+0000","last_clean":"2023-10-06T05:23:30.130471+0000","last_became_active":"2023-10-06T05:23:30.130214+0000","last_became_peered":"2023-10-06T05:23:30.130214+0000","last_unstale":"2023-10-06T05:23:30.130471+0000","last_undegraded":"2023-10-06T05:23:30.130471+0000","last_fullsized":"2023-10-06T05:23:30.130471+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:25.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.125347+0000","last_change":"2023-10-06T05:23:29.137836+0000","last_active":"2023-10-06T05:23:30.125347+0000","last_peered":"2023-10-06T05:23:30.125347+0000","last_clean":"2023-10-06T05:23:30.125347+0000","last_became_active":"2023-10-06T05:23:29.137565+0000","last_became_peered":"2023-10-06T05:23:29.137565+0000","last_unstale":"2023-10-06T05:23:30.125347+0000","last_undegraded":"2023-10-06T05:23:30.125347+0000","last_fullsized":"2023-10-06T05:23:30.125347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:15.188594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.123658+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:23:30.123658+0000","last_peered":"2023-10-06T05:23:30.123658+0000","last_clean":"2023-10-06T05:23:30.123658+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:23:30.123658+0000","last_undegraded":"2023-10-06T05:23:30.123658+0000","last_fullsized":"2023-10-06T05:23:30.123658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.141821+0000","last_change":"2023-10-06T05:23:30.139410+0000","last_active":"2023-10-06T05:23:30.141821+0000","last_peered":"2023-10-06T05:23:30.141821+0000","last_clean":"2023-10-06T05:23:30.141821+0000","last_became_active":"2023-10-06T05:23:30.139127+0000","last_became_peered":"2023-10-06T05:23:30.139127+0000","last_unstale":"2023-10-06T05:23:30.141821+0000","last_undegraded":"2023-10-06T05:23:30.141821+0000","last_fullsized":"2023-10-06T05:23:30.141821+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:29.104391+0000","last_clean_scrub_stamp":"2023-10-06T05:23:29.104391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:01.424712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-06T05:23:29.139188+0000","last_change":"2023-10-06T05:23:29.133845+0000","last_active":"2023-10-06T05:23:29.139188+0000","last_peered":"2023-10-06T05:23:29.139188+0000","last_clean":"2023-10-06T05:23:29.139188+0000","last_became_active":"2023-10-06T05:23:29.133578+0000","last_became_peered":"2023-10-06T05:23:29.133578+0000","last_unstale":"2023-10-06T05:23:29.139188+0000","last_undegraded":"2023-10-06T05:23:29.139188+0000","last_fullsized":"2023-10-06T05:23:29.139188+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:23:28.098594+0000","last_clean_scrub_stamp":"2023-10-06T05:23:28.098594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:10.132750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T05:23:30.120746+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:23:30.120746+0000","last_peered":"2023-10-06T05:23:30.120746+0000","last_clean":"2023-10-06T05:23:30.120746+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:23:30.120746+0000","last_undegraded":"2023-10-06T05:23:30.120746+0000","last_fullsized":"2023-10-06T05:23:30.120746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":1976,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":2023424,"data_stored":1246704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.47899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.51100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.46700000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88900000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91200000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.52000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88300000000000001,"5min":1.018,"15min":1.018},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71099999999999997,"15min":0.75},"min":{"1min":0.48999999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94199999999999995,"5min":1.056,"15min":1.056},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.746},"min":{"1min":0.53600000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":1.056,"15min":1.056},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.53600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.872,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63500000000000001,"15min":0.70699999999999996},"min":{"1min":0.498,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95199999999999996,"5min":0.95299999999999996,"15min":1.0249999999999999},"last":0.754},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71499999999999997,"15min":0.77400000000000002},"min":{"1min":0.52000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96699999999999997,"5min":1.27,"15min":1.27},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69999999999999996,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92900000000000005,"5min":0.97899999999999998,"15min":0.999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.74099999999999999,"15min":0.82399999999999995},"min":{"1min":0.47099999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.98899999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79900000000000004,"15min":0.876},"min":{"1min":0.56599999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0089999999999999,"5min":1.012,"15min":1.1040000000000001},"last":0.621},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.84599999999999997},"min":{"1min":0.57799999999999996,"5min":0.46200000000000002,"15min":0.441},"max":{"1min":0.99099999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298676,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073164,"statfs":{"total":96636764160,"available":96330919936,"internally_reserved":0,"allocated":1421312,"data_stored":655354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66800000000000004,"15min":0.68200000000000005},"min":{"1min":0.46600000000000003,"5min":0.42999999999999999,"15min":0.32300000000000001},"max":{"1min":0.84399999999999997,"5min":1.2629999999999999,"15min":1.8500000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.38900000000000001},"max":{"1min":0.89500000000000002,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70499999999999996,"15min":0.71299999999999997},"min":{"1min":0.50700000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":1.135,"15min":1.9279999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":1.073,"15min":1.4239999999999999},"last":0.754}]},{"osd":2,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94999999999999996,"5min":1.369,"15min":1.369},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.75900000000000001},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.91200000000000003,"5min":1.208,"15min":1.891},"last":0.69699999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.748},"min":{"1min":0.47799999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.86099999999999999,"5min":1.018,"15min":1.347},"last":0.63},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.53500000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":1},"last":0.499}]},{"osd":4,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.77600000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.998,"5min":1.1140000000000001,"15min":1.9670000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76900000000000002,"15min":0.81899999999999995},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.012,"5min":1.278,"15min":1.4510000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.415},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73899999999999999,"15min":0.81899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":2},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.753,"15min":0.80700000000000005},"min":{"1min":0.57299999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.751,"15min":0.77800000000000002},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.46},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303348,"kb_used_data":1964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068492,"statfs":{"total":96636764160,"available":96326135808,"internally_reserved":0,"allocated":2011136,"data_stored":1245148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45200000000000001,"15min":0.436},"min":{"1min":0.32800000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67000000000000004,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.54600000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.37,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.66000000000000003,"5min":0.77000000000000002,"15min":0.83799999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.496,"15min":0.47199999999999998},"min":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.753,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55900000000000005,"15min":0.52400000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.625,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.504},{"interface":"front","average":{"1min":0.495,"5min":0.53200000000000003,"15min":0.52700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.64200000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90200000000000002,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.81999999999999995,"15min":0.79500000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91600000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.76600000000000001,"15min":0.749},"min":{"1min":0.54000000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0009999999999999,"5min":1.099,"15min":1.099},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.79800000000000004,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0209999999999999,"5min":1.181,"15min":1.181},"last":0.69199999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.76800000000000002,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.86399999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81899999999999995,"15min":0.79900000000000004},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.95799999999999996,"5min":1.123,"15min":1.123},"last":0.85999999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96799999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.82499999999999996,"15min":0.82399999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":0.91400000000000003,"5min":1.087,"15min":1.103},"last":0.73199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298676,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073164,"statfs":{"total":96636764160,"available":96330919936,"internally_reserved":0,"allocated":1421312,"data_stored":654766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.442,"15min":0.432},"min":{"1min":0.28599999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58799999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.318},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.78900000000000003,"15min":1.042},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.499,"15min":0.49099999999999999},"min":{"1min":0.34300000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.504},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52900000000000003,"15min":0.51400000000000001},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.58699999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.52000000000000002,"15min":0.49099999999999999},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.32800000000000001},"max":{"1min":0.61399999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59099999999999997,"15min":0.57799999999999996},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.73999999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.496,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1379999999999999,"5min":1.242,"15min":1.242},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.76100000000000001,"15min":0.77800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94699999999999995,"5min":1.198,"15min":1.198},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.77300000000000002,"15min":0.76700000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":1.26,"15min":1.26},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.76400000000000001,"15min":0.77400000000000002},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0029999999999999,"5min":1.163,"15min":1.163},"last":1.1299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.78300000000000003,"15min":0.79900000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.105,"5min":1.323,"15min":1.323},"last":0.999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.80500000000000005,"15min":0.79300000000000004},"min":{"1min":0.56499999999999995,"5min":0.55200000000000005,"15min":0.50600000000000001},"max":{"1min":0.98799999999999999,"5min":1.1799999999999999,"15min":1.3100000000000001},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.83399999999999996},"min":{"1min":0.57599999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.056,"5min":1.2090000000000001,"15min":1.264},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79500000000000004,"15min":0.82399999999999995},"min":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.077,"5min":1.1579999999999999,"15min":1.3680000000000001},"last":1.1719999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":655298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.43099999999999999,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.627,"5min":0.77400000000000002,"15min":0.77800000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68500000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.27400000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.502},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.31900000000000001},"max":{"1min":0.66200000000000003,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.51600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.82799999999999996,"15min":0.93600000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.45700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73599999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98499999999999999,"5min":1.022,"15min":1.1299999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.79400000000000004},"min":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96999999999999997,"5min":1.085,"15min":1.1699999999999999},"last":0.86599999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73699999999999999,"15min":0.78200000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.026,"5min":1.042,"15min":1.099},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.751,"15min":0.80000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.002,"5min":1.0649999999999999,"15min":1.163},"last":0.95999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77400000000000002,"15min":0.82999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94799999999999995,"5min":1.0960000000000001,"15min":1.21},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.998,"5min":1.022,"15min":1.042},"last":0.876}]},{"osd":7,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.77600000000000002,"15min":0.80600000000000005},"min":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.85899999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0629999999999999,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":0.84599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302780,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069060,"statfs":{"total":96636764160,"available":96326717440,"internally_reserved":0,"allocated":1429504,"data_stored":656316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44,"15min":0.46000000000000002},"min":{"1min":0.32600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67500000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53200000000000003,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70799999999999996,"5min":0.999,"15min":0.999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.74299999999999999,"5min":0.98999999999999999,"15min":1.014},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.57299999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.071,"5min":1.161,"15min":1.2669999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.57999999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.113,"5min":1.1830000000000001,"15min":1.3540000000000001},"last":0.89300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.54200000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0880000000000001,"5min":1.147,"15min":1.204},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79500000000000004,"15min":0.83299999999999996},"min":{"1min":0.59199999999999997,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0569999999999999,"5min":1.284,"15min":1.329},"last":0.95899999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78500000000000003,"15min":0.81499999999999995},"min":{"1min":0.53300000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.484},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.80800000000000005,"15min":0.84499999999999997},"min":{"1min":0.60599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.629},"last":1.0169999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80900000000000005,"15min":0.85099999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0640000000000001,"5min":1.236,"15min":1.5620000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82099999999999995,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.117,"5min":1.194,"15min":1.325},"last":0.94899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302764,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069076,"statfs":{"total":96636764160,"available":96326733824,"internally_reserved":0,"allocated":1413120,"data_stored":654710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90700000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68100000000000005,"15min":0.77700000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89100000000000001,"5min":0.95999999999999996,"15min":2.77},"last":0.872}]},{"osd":1,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.73499999999999999},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.6419999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68999999999999995,"15min":0.71999999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.036,"5min":1.1100000000000001,"15min":1.1899999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.49199999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71299999999999997,"15min":0.81200000000000006},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95599999999999996,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.52900000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95899999999999996,"5min":0.999,"15min":1.0629999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":1.026},"min":{"1min":0.59799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91200000000000003,"5min":0.97699999999999998,"15min":7.742},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68100000000000005,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91900000000000004,"5min":0.96499999999999997,"15min":1.079},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70599999999999996,"15min":0.748},"min":{"1min":0.498,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.181},"last":0.97499999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.73599999999999999},"min":{"1min":0.505,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.2669999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.747,"15min":0.81499999999999995},"min":{"1min":0.56499999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.171},"last":0.83699999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.82699999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76500000000000001,"15min":0.85299999999999998},"min":{"1min":0.64800000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.571},"last":0.76700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298672,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073168,"statfs":{"total":96636764160,"available":96330924032,"internally_reserved":0,"allocated":1417216,"data_stored":654732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.93600000000000005},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.873,"5min":1.109,"15min":9.2780000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.52500000000000002,"5min":0.438,"15min":0.434},"max":{"1min":0.88900000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68000000000000005,"15min":0.96199999999999997},"min":{"1min":0.432,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":1.079,"15min":9.25},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69599999999999995,"15min":1.022},"min":{"1min":0.44500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.54200000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66900000000000004,"15min":0.99299999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.052,"5min":1.103,"15min":10.384},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72799999999999998,"15min":1.0169999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73399999999999999,"15min":1.0589999999999999},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.76,"5min":1.76,"15min":10.205},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70499999999999996,"15min":0.72299999999999998},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.097,"5min":1.097,"15min":1.292},"last":0.625}]},{"osd":4,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.43099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96999999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.502},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72499999999999998,"15min":0.77200000000000002},"min":{"1min":0.53300000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.76300000000000001,"15min":0.81000000000000005},"min":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.008,"5min":1.1000000000000001,"15min":1.298},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:23:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78200000000000003,"15min":0.82199999999999995},"min":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77500000000000002,"15min":0.81100000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:23:35.015 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:23:35.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:23:35.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.249+0000 7f00db3ff700 1 Processor -- start 2023-10-06T05:23:35.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.249+0000 7f00db3ff700 1 -- start start 2023-10-06T05:23:35.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.250+0000 7f00db3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 0x7f00d40cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.250+0000 7f00db3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f00d40b2b80 0x7f00d40cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.250+0000 7f00db3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d415d2d0 con 0x7f00d4159930 2023-10-06T05:23:35.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.250+0000 7f00db3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d415d5b0 con 0x7f00d40b2b80 2023-10-06T05:23:35.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.250+0000 7f00db3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d415d890 con 0x7f00d4159560 2023-10-06T05:23:35.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00d899a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f00d4159930 0x7f00d40b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33454/0 (socket says 172.21.15.31:33454) 2023-10-06T05:23:35.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00d899a700 1 -- 172.21.15.31:0/490280886 learned_addr learned my addr 172.21.15.31:0/490280886 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:35.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00d919b700 1 --2- 172.21.15.31:0/490280886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 0x7f00d40cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00d999c700 1 --2- 172.21.15.31:0/490280886 >> v2:172.21.15.62:3300/0 conn(0x7f00d40b2b80 0x7f00d40cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00cbfff700 1 -- 172.21.15.31:0/490280886 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962561359 0 0) 0x7f00d415d2d0 con 0x7f00d4159930 2023-10-06T05:23:35.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00cbfff700 1 -- 172.21.15.31:0/490280886 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00c0003570 con 0x7f00d4159930 2023-10-06T05:23:35.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00d999c700 1 -- 172.21.15.31:0/490280886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 msgr2=0x7f00d40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:35.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00d999c700 1 --2- 172.21.15.31:0/490280886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 0x7f00d40cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00d999c700 1 -- 172.21.15.31:0/490280886 >> v1:172.21.15.31:6789/0 conn(0x7f00d4159930 legacy=0x7f00d40b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00d999c700 1 -- 172.21.15.31:0/490280886 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d415db70 con 0x7f00d40b2b80 2023-10-06T05:23:35.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.251+0000 7f00d919b700 1 --2- 172.21.15.31:0/490280886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 0x7f00d40cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:35.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.252+0000 7f00d999c700 1 --2- 172.21.15.31:0/490280886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00d40b2b80 0x7f00d40cd3b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009e60 tx=0x7f00cc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e440db8f2b5484 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.252+0000 7f00cbfff700 1 -- 172.21.15.31:0/490280886 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc00e070 con 0x7f00d40b2b80 2023-10-06T05:23:35.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.252+0000 7f00cbfff700 1 -- 172.21.15.31:0/490280886 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc00bab0 con 0x7f00d40b2b80 2023-10-06T05:23:35.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.252+0000 7f00cbfff700 1 -- 172.21.15.31:0/490280886 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc00be70 con 0x7f00d40b2b80 2023-10-06T05:23:35.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.252+0000 7f00db3ff700 1 -- 172.21.15.31:0/490280886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00d40b2b80 msgr2=0x7f00d40cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.252+0000 7f00db3ff700 1 --2- 172.21.15.31:0/490280886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00d40b2b80 0x7f00d40cd3b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009e60 tx=0x7f00cc009780 comp rx=0 tx=0).stop 2023-10-06T05:23:35.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.253+0000 7f00db3ff700 1 -- 172.21.15.31:0/490280886 shutdown_connections 2023-10-06T05:23:35.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.253+0000 7f00db3ff700 1 --2- 172.21.15.31:0/490280886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00d40b2b80 0x7f00d40cd3b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.253+0000 7f00db3ff700 1 --2- 172.21.15.31:0/490280886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 0x7f00d40cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.253+0000 7f00db3ff700 1 -- 172.21.15.31:0/490280886 >> 172.21.15.31:0/490280886 conn(0x7f00d406ec90 msgr2=0x7f00d406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:35.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.253+0000 7f00db3ff700 1 -- 172.21.15.31:0/490280886 shutdown_connections 2023-10-06T05:23:35.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.253+0000 7f00db3ff700 1 -- 172.21.15.31:0/490280886 wait complete. 2023-10-06T05:23:35.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.255+0000 7f00db3ff700 1 Processor -- start 2023-10-06T05:23:35.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.255+0000 7f00db3ff700 1 -- start start 2023-10-06T05:23:35.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.256+0000 7f00db3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40b2b80 0x7f00d415c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.256+0000 7f00db3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 0x7f00d414a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.256+0000 7f00db3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00d4159930 0x7f00d40c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.256+0000 7f00db3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d415cca0 con 0x7f00d40b2b80 2023-10-06T05:23:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.256+0000 7f00db3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d415cab0 con 0x7f00d4159930 2023-10-06T05:23:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.256+0000 7f00db3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d414ae80 con 0x7f00d4159560 2023-10-06T05:23:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40b2b80 0x7f00d415c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40b2b80 0x7f00d415c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57984/0 (socket says 172.21.15.31:57984) 2023-10-06T05:23:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d919b700 1 -- 172.21.15.31:0/567443759 learned_addr learned my addr 172.21.15.31:0/567443759 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d999c700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00d4159930 0x7f00d40c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d899a700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 0x7f00d414a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d919b700 1 -- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 msgr2=0x7f00d414a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d919b700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 0x7f00d414a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d919b700 1 -- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00d4159930 msgr2=0x7f00d40c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d919b700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00d4159930 0x7f00d40c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d919b700 1 -- 172.21.15.31:0/567443759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0009420 con 0x7f00d40b2b80 2023-10-06T05:23:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00d919b700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40b2b80 0x7f00d415c570 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f00d000ecb0 tx=0x7f00d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00c97fa700 1 -- 172.21.15.31:0/567443759 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d000ee60 con 0x7f00d40b2b80 2023-10-06T05:23:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.257+0000 7f00c97fa700 1 -- 172.21.15.31:0/567443759 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d000cbe0 con 0x7f00d40b2b80 2023-10-06T05:23:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.258+0000 7f00db3ff700 1 -- 172.21.15.31:0/567443759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc009a20 con 0x7f00d40b2b80 2023-10-06T05:23:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.258+0000 7f00c97fa700 1 -- 172.21.15.31:0/567443759 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0013470 con 0x7f00d40b2b80 2023-10-06T05:23:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.258+0000 7f00db3ff700 1 -- 172.21.15.31:0/567443759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d40c4950 con 0x7f00d40b2b80 2023-10-06T05:23:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.259+0000 7f00db3ff700 1 -- 172.21.15.31:0/567443759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d404c380 con 0x7f00d40b2b80 2023-10-06T05:23:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.260+0000 7f00c97fa700 1 -- 172.21.15.31:0/567443759 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f00d0013820 con 0x7f00d40b2b80 2023-10-06T05:23:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.260+0000 7f00c97fa700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00b8067320 0x7f00b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.261+0000 7f00c97fa700 1 -- 172.21.15.31:0/567443759 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f00d0085220 con 0x7f00d40b2b80 2023-10-06T05:23:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.261+0000 7f00d899a700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00b8067320 0x7f00b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.262+0000 7f00d899a700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00b8067320 0x7f00b80697d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f00d4155720 tx=0x7f00c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.264+0000 7f00c97fa700 1 -- 172.21.15.31:0/567443759 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00d0051cf0 con 0x7f00d40b2b80 2023-10-06T05:23:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.554+0000 7f00db3ff700 1 -- 172.21.15.31:0/567443759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f00d415de50 con 0x7f00d40b2b80 2023-10-06T05:23:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.559+0000 7f00c97fa700 1 -- 172.21.15.31:0/567443759 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f00d0055310 con 0x7f00d40b2b80 2023-10-06T05:23:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 -- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00b8067320 msgr2=0x7f00b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00b8067320 0x7f00b80697d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f00d4155720 tx=0x7f00c4009040 comp rx=0 tx=0).stop 2023-10-06T05:23:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 -- 172.21.15.31:0/567443759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40b2b80 msgr2=0x7f00d415c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40b2b80 0x7f00d415c570 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f00d000ecb0 tx=0x7f00d00091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 -- 172.21.15.31:0/567443759 shutdown_connections 2023-10-06T05:23:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00b8067320 0x7f00b80697d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00d4159930 0x7f00d40c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40b2b80 0x7f00d415c570 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 --2- 172.21.15.31:0/567443759 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00d4159560 0x7f00d414a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 -- 172.21.15.31:0/567443759 >> 172.21.15.31:0/567443759 conn(0x7f00d406ec90 msgr2=0x7f00d41562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 -- 172.21.15.31:0/567443759 shutdown_connections 2023-10-06T05:23:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.561+0000 7f00db3ff700 1 -- 172.21.15.31:0/567443759 wait complete. 2023-10-06T05:23:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:23:35.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:35.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.775+0000 7f67c53ff700 1 Processor -- start 2023-10-06T05:23:35.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.775+0000 7f67c53ff700 1 -- start start 2023-10-06T05:23:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.776+0000 7f67c53ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f67c0159500 0x7f67c00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.776+0000 7f67c53ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c00cd510 0x7f67c00aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.776+0000 7f67c53ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c00c3f40 con 0x7f67c015d2d0 2023-10-06T05:23:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.776+0000 7f67c53ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c00b2860 con 0x7f67c0159500 2023-10-06T05:23:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.776+0000 7f67c53ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c015d7d0 con 0x7f67c00cd510 2023-10-06T05:23:35.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c00cd510 0x7f67c00aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67bf7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c00cd510 0x7f67c00aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:51640/0 (socket says 172.21.15.31:51640) 2023-10-06T05:23:35.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67be7fc700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f67c015d2d0 0x7f67c00cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:34676/0 (socket says 172.21.15.31:34676) 2023-10-06T05:23:35.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67bf7fe700 1 -- 172.21.15.31:0/214686244 learned_addr learned my addr 172.21.15.31:0/214686244 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67beffd700 1 --2- 172.21.15.31:0/214686244 >> v2:172.21.15.62:3300/0 conn(0x7f67c0159500 0x7f67c00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67bdffb700 1 -- 172.21.15.31:0/214686244 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816706260 0 0) 0x7f67c00c3f40 con 0x7f67c015d2d0 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67bdffb700 1 -- 172.21.15.31:0/214686244 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67a8003570 con 0x7f67c015d2d0 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67bf7fe700 1 -- 172.21.15.31:0/214686244 >> v2:172.21.15.62:3300/0 conn(0x7f67c0159500 msgr2=0x7f67c00b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67bf7fe700 1 --2- 172.21.15.31:0/214686244 >> v2:172.21.15.62:3300/0 conn(0x7f67c0159500 0x7f67c00b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67bf7fe700 1 -- 172.21.15.31:0/214686244 >> v1:172.21.15.62:6789/0 conn(0x7f67c015d2d0 legacy=0x7f67c00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67bf7fe700 1 -- 172.21.15.31:0/214686244 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c00cb000 con 0x7f67c00cd510 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.777+0000 7f67beffd700 1 --2- 172.21.15.31:0/214686244 >> v2:172.21.15.62:3300/0 conn(0x7f67c0159500 0x7f67c00b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.778+0000 7f67bf7fe700 1 --2- 172.21.15.31:0/214686244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c00cd510 0x7f67c00aedd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009f20 tx=0x7f67b8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207f2baa781dd25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.778+0000 7f67bdffb700 1 -- 172.21.15.31:0/214686244 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b800e070 con 0x7f67c00cd510 2023-10-06T05:23:35.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.778+0000 7f67bdffb700 1 -- 172.21.15.31:0/214686244 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b800b970 con 0x7f67c00cd510 2023-10-06T05:23:35.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.778+0000 7f67bdffb700 1 -- 172.21.15.31:0/214686244 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80092e0 con 0x7f67c00cd510 2023-10-06T05:23:35.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.778+0000 7f67c53ff700 1 -- 172.21.15.31:0/214686244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c00cd510 msgr2=0x7f67c00aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.778+0000 7f67c53ff700 1 --2- 172.21.15.31:0/214686244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c00cd510 0x7f67c00aedd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009f20 tx=0x7f67b8009840 comp rx=0 tx=0).stop 2023-10-06T05:23:35.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.778+0000 7f67c53ff700 1 -- 172.21.15.31:0/214686244 shutdown_connections 2023-10-06T05:23:35.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.778+0000 7f67c53ff700 1 --2- 172.21.15.31:0/214686244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c00cd510 0x7f67c00aedd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.778+0000 7f67c53ff700 1 --2- 172.21.15.31:0/214686244 >> v2:172.21.15.62:3300/0 conn(0x7f67c0159500 0x7f67c00b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.778+0000 7f67c53ff700 1 -- 172.21.15.31:0/214686244 >> 172.21.15.31:0/214686244 conn(0x7f67c006ec90 msgr2=0x7f67c006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:35.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.779+0000 7f67c53ff700 1 -- 172.21.15.31:0/214686244 shutdown_connections 2023-10-06T05:23:35.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.779+0000 7f67c53ff700 1 -- 172.21.15.31:0/214686244 wait complete. 2023-10-06T05:23:35.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.780+0000 7f67c53ff700 1 Processor -- start 2023-10-06T05:23:35.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.780+0000 7f67c53ff700 1 -- start start 2023-10-06T05:23:35.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.781+0000 7f67c53ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c00cd510 0x7f67c0153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.781+0000 7f67c53ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c0159500 0x7f67c00b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.781+0000 7f67c53ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c015d2d0 0x7f67c016cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.781+0000 7f67c53ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c016dee0 con 0x7f67c0159500 2023-10-06T05:23:35.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.781+0000 7f67c53ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00c5b40 con 0x7f67c00cd510 2023-10-06T05:23:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.781+0000 7f67c53ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c015e470 con 0x7f67c015d2d0 2023-10-06T05:23:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.781+0000 7f67be7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c0159500 0x7f67c00b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.781+0000 7f67be7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c0159500 0x7f67c00b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58000/0 (socket says 172.21.15.31:58000) 2023-10-06T05:23:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.781+0000 7f67be7fc700 1 -- 172.21.15.31:0/599629550 learned_addr learned my addr 172.21.15.31:0/599629550 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67beffd700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c00cd510 0x7f67c0153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67bf7fe700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c015d2d0 0x7f67c016cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67be7fc700 1 -- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c015d2d0 msgr2=0x7f67c016cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67be7fc700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c015d2d0 0x7f67c016cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67be7fc700 1 -- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c00cd510 msgr2=0x7f67c0153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67be7fc700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c00cd510 0x7f67c0153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67be7fc700 1 -- 172.21.15.31:0/599629550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b0009420 con 0x7f67c0159500 2023-10-06T05:23:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67be7fc700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c0159500 0x7f67c00b5aa0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f67b400c5b0 tx=0x7f67b4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67a6ffd700 1 -- 172.21.15.31:0/599629550 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4005ac0 con 0x7f67c0159500 2023-10-06T05:23:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67a6ffd700 1 -- 172.21.15.31:0/599629550 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b4005c20 con 0x7f67c0159500 2023-10-06T05:23:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67a6ffd700 1 -- 172.21.15.31:0/599629550 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40070d0 con 0x7f67c0159500 2023-10-06T05:23:35.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.782+0000 7f67c53ff700 1 -- 172.21.15.31:0/599629550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b8009ae0 con 0x7f67c0159500 2023-10-06T05:23:35.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.783+0000 7f67c53ff700 1 -- 172.21.15.31:0/599629550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c00c5580 con 0x7f67c0159500 2023-10-06T05:23:35.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.784+0000 7f67c53ff700 1 -- 172.21.15.31:0/599629550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c015e600 con 0x7f67c0159500 2023-10-06T05:23:35.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.784+0000 7f67a6ffd700 1 -- 172.21.15.31:0/599629550 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f67b4007480 con 0x7f67c0159500 2023-10-06T05:23:35.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.786+0000 7f67a6ffd700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67ac067320 0x7f67ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:35.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.786+0000 7f67a6ffd700 1 -- 172.21.15.31:0/599629550 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f67b40842b0 con 0x7f67c0159500 2023-10-06T05:23:35.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.786+0000 7f67beffd700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67ac067320 0x7f67ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:35.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.787+0000 7f67beffd700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67ac067320 0x7f67ac0697d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f67b00093f0 tx=0x7f67b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:35.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:35.789+0000 7f67a6ffd700 1 -- 172.21.15.31:0/599629550 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67b404b5a0 con 0x7f67c0159500 2023-10-06T05:23:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.034+0000 7f67c53ff700 1 -- 172.21.15.31:0/599629550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67c004c380 con 0x7f67c0159500 2023-10-06T05:23:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.035+0000 7f67a6ffd700 1 -- 172.21.15.31:0/599629550 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4593 (secure 0 0 0) 0x7f67b4017090 con 0x7f67c0159500 2023-10-06T05:23:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:36.037 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":75,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6557,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1866061469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1866061469},{"type":"v1","addr":"172.21.15.31:6834","nonce":1866061469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7366,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/257113943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":257113943},{"type":"v1","addr":"172.21.15.31:6835","nonce":257113943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2082524951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2082524951},{"type":"v1","addr":"172.21.15.62:6834","nonce":2082524951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:29.615356+0000","modified":"2023-10-06T05:23:33.142272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7378":{"gid":7378,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/519548465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":519548465},{"type":"v1","addr":"172.21.15.62:6835","nonce":519548465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:23:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 -- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67ac067320 msgr2=0x7f67ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67ac067320 0x7f67ac0697d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f67b00093f0 tx=0x7f67b0008040 comp rx=0 tx=0).stop 2023-10-06T05:23:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 -- 172.21.15.31:0/599629550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c0159500 msgr2=0x7f67c00b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c0159500 0x7f67c00b5aa0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f67b400c5b0 tx=0x7f67b4007df0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 -- 172.21.15.31:0/599629550 shutdown_connections 2023-10-06T05:23:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67ac067320 0x7f67ac0697d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c015d2d0 0x7f67c016cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c00cd510 0x7f67c0153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 --2- 172.21.15.31:0/599629550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c0159500 0x7f67c00b5aa0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 -- 172.21.15.31:0/599629550 >> 172.21.15.31:0/599629550 conn(0x7f67c006ec90 msgr2=0x7f67c00ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 -- 172.21.15.31:0/599629550 shutdown_connections 2023-10-06T05:23:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.037+0000 7f67c53ff700 1 -- 172.21.15.31:0/599629550 wait complete. 2023-10-06T05:23:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 75 2023-10-06T05:23:36.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:23:29.615356+0000', 'modified': '2023-10-06T05:23:33.142272+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7378}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7378': {'gid': 7378, 'name': 'd', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/519548465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 519548465}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 519548465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:23:36.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:23:36.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T05:23:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.657+0000 7f46907ff700 1 Processor -- start 2023-10-06T05:23:36.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.657+0000 7f46907ff700 1 -- start start 2023-10-06T05:23:36.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.658+0000 7f46907ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f468813a1e0 0x7f46880c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.658+0000 7f46907ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f468813a5b0 0x7f468806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.658+0000 7f46907ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46880c71b0 con 0x7f46880c6c30 2023-10-06T05:23:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.658+0000 7f46907ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880c7490 con 0x7f468813a5b0 2023-10-06T05:23:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.658+0000 7f46907ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880c7770 con 0x7f468813a1e0 2023-10-06T05:23:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.659+0000 7f468dd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f468813a5b0 0x7f468806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.659+0000 7f468dd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f468813a5b0 0x7f468806d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58010/0 (socket says 172.21.15.31:58010) 2023-10-06T05:23:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.659+0000 7f468dd9a700 1 -- 172.21.15.31:0/2540887285 learned_addr learned my addr 172.21.15.31:0/2540887285 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.659+0000 7f468e59b700 1 --2- 172.21.15.31:0/2540887285 >> v2:172.21.15.62:3300/0 conn(0x7f468813a1e0 0x7f46880c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.659+0000 7f468dd9a700 1 -- 172.21.15.31:0/2540887285 >> v2:172.21.15.62:3300/0 conn(0x7f468813a1e0 msgr2=0x7f46880c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.659+0000 7f468dd9a700 1 --2- 172.21.15.31:0/2540887285 >> v2:172.21.15.62:3300/0 conn(0x7f468813a1e0 0x7f46880c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.659+0000 7f468dd9a700 1 -- 172.21.15.31:0/2540887285 >> v1:172.21.15.62:6789/0 conn(0x7f46880c6c30 legacy=0x7f468815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.659+0000 7f468dd9a700 1 -- 172.21.15.31:0/2540887285 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880c7a50 con 0x7f468813a5b0 2023-10-06T05:23:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.660+0000 7f468dd9a700 1 --2- 172.21.15.31:0/2540887285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a5b0 0x7f468806d440 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f46880cbf90 tx=0x7f4678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d73e37d510b75f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.660+0000 7f468d599700 1 -- 172.21.15.31:0/2540887285 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467800b860 con 0x7f468813a5b0 2023-10-06T05:23:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.660+0000 7f468d599700 1 -- 172.21.15.31:0/2540887285 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467800b9c0 con 0x7f468813a5b0 2023-10-06T05:23:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.660+0000 7f468d599700 1 -- 172.21.15.31:0/2540887285 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467800bd60 con 0x7f468813a5b0 2023-10-06T05:23:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.660+0000 7f46907ff700 1 -- 172.21.15.31:0/2540887285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a5b0 msgr2=0x7f468806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.660+0000 7f46907ff700 1 --2- 172.21.15.31:0/2540887285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a5b0 0x7f468806d440 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f46880cbf90 tx=0x7f4678009250 comp rx=0 tx=0).stop 2023-10-06T05:23:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.660+0000 7f46907ff700 1 -- 172.21.15.31:0/2540887285 shutdown_connections 2023-10-06T05:23:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.660+0000 7f46907ff700 1 --2- 172.21.15.31:0/2540887285 >> v2:172.21.15.62:3300/0 conn(0x7f468813a1e0 0x7f46880c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.660+0000 7f46907ff700 1 --2- 172.21.15.31:0/2540887285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a5b0 0x7f468806d440 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.661+0000 7f46907ff700 1 -- 172.21.15.31:0/2540887285 >> 172.21.15.31:0/2540887285 conn(0x7f46880cc0b0 msgr2=0x7f46880cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.661+0000 7f46907ff700 1 -- 172.21.15.31:0/2540887285 shutdown_connections 2023-10-06T05:23:36.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.661+0000 7f46907ff700 1 -- 172.21.15.31:0/2540887285 wait complete. 2023-10-06T05:23:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.662+0000 7f46907ff700 1 Processor -- start 2023-10-06T05:23:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.662+0000 7f46907ff700 1 -- start start 2023-10-06T05:23:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.663+0000 7f46907ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46880c6c30 0x7f46881522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.663+0000 7f46907ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a1e0 0x7f468814e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.663+0000 7f46907ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f468813a5b0 0x7f4688138d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.663+0000 7f46907ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4688139280 con 0x7f468813a1e0 2023-10-06T05:23:36.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.663+0000 7f46907ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4688150ab0 con 0x7f468813a5b0 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.663+0000 7f46907ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46881501c0 con 0x7f46880c6c30 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.663+0000 7f468dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a1e0 0x7f468814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.663+0000 7f468dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a1e0 0x7f468814e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58016/0 (socket says 172.21.15.31:58016) 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.663+0000 7f468dd9a700 1 -- 172.21.15.31:0/2812362595 learned_addr learned my addr 172.21.15.31:0/2812362595 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f468e59b700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46880c6c30 0x7f46881522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f468ed9c700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f468813a5b0 0x7f4688138d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f468dd9a700 1 -- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46880c6c30 msgr2=0x7f46881522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f468dd9a700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46880c6c30 0x7f46881522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f468dd9a700 1 -- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f468813a5b0 msgr2=0x7f4688138d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f468dd9a700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f468813a5b0 0x7f4688138d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f468dd9a700 1 -- 172.21.15.31:0/2812362595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46780094f0 con 0x7f468813a1e0 2023-10-06T05:23:36.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f468e59b700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46880c6c30 0x7f46881522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f468dd9a700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a1e0 0x7f468814e210 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f46880cc010 tx=0x7f4678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f467e7fc700 1 -- 172.21.15.31:0/2812362595 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678011e60 con 0x7f468813a1e0 2023-10-06T05:23:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f467e7fc700 1 -- 172.21.15.31:0/2812362595 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467801a630 con 0x7f468813a1e0 2023-10-06T05:23:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f467e7fc700 1 -- 172.21.15.31:0/2812362595 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467801a9f0 con 0x7f468813a1e0 2023-10-06T05:23:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f46907ff700 1 -- 172.21.15.31:0/2812362595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4688154f50 con 0x7f468813a1e0 2023-10-06T05:23:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.664+0000 7f46907ff700 1 -- 172.21.15.31:0/2812362595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4688150740 con 0x7f468813a1e0 2023-10-06T05:23:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.666+0000 7f467e7fc700 1 -- 172.21.15.31:0/2812362595 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f467802d440 con 0x7f468813a1e0 2023-10-06T05:23:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.667+0000 7f467e7fc700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f466c067320 0x7f466c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.667+0000 7f467e7fc700 1 -- 172.21.15.31:0/2812362595 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f4678086910 con 0x7f468813a1e0 2023-10-06T05:23:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.668+0000 7f468e59b700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f466c067320 0x7f466c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.668+0000 7f468e59b700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f466c067320 0x7f466c0697d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4684006aa0 tx=0x7f4684008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.669+0000 7f469f023b80 1 Processor -- start 2023-10-06T05:23:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.670+0000 7f469f023b80 1 -- start start 2023-10-06T05:23:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f469f023b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05dce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f469f023b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05d9920 0x564bc05acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f469f023b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05dc910 0x564bc05dd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f469f023b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc04ca310 con 0x564bc05dc910 2023-10-06T05:23:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f469f023b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc0507130 con 0x564bc05d9920 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f469f023b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x564bc0504af0 con 0x564bc05ac930 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f468ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05dc910 0x564bc05dd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f468ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05dc910 0x564bc05dd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58030/0 (socket says 172.21.15.31:58030) 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f468ed9c700 1 -- 172.21.15.31:0/1528305794 learned_addr learned my addr 172.21.15.31:0/1528305794 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f468e59b700 1 --2- 172.21.15.31:0/1528305794 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05dce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.671+0000 7f468dd9a700 1 --2- 172.21.15.31:0/1528305794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05d9920 0x564bc05acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.672+0000 7f468ed9c700 1 -- 172.21.15.31:0/1528305794 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 msgr2=0x564bc05dce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.672+0000 7f468ed9c700 1 --2- 172.21.15.31:0/1528305794 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05dce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.672+0000 7f468ed9c700 1 -- 172.21.15.31:0/1528305794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05d9920 msgr2=0x564bc05acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.672+0000 7f468ed9c700 1 --2- 172.21.15.31:0/1528305794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05d9920 0x564bc05acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.672+0000 7f468ed9c700 1 -- 172.21.15.31:0/1528305794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bc0306790 con 0x564bc05dc910 2023-10-06T05:23:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.672+0000 7f468ed9c700 1 --2- 172.21.15.31:0/1528305794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05dc910 0x564bc05dd640 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4680003d90 tx=0x7f4680009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8b90590136fb49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.672+0000 7f466b7fe700 1 -- 172.21.15.31:0/1528305794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46800032e0 con 0x564bc05dc910 2023-10-06T05:23:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.672+0000 7f466b7fe700 1 -- 172.21.15.31:0/1528305794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000e650 con 0x564bc05dc910 2023-10-06T05:23:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.672+0000 7f466b7fe700 1 -- 172.21.15.31:0/1528305794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000e9d0 con 0x564bc05dc910 2023-10-06T05:23:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.673+0000 7f469f023b80 1 -- 172.21.15.31:0/1528305794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05dc910 msgr2=0x564bc05dd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.673+0000 7f469f023b80 1 --2- 172.21.15.31:0/1528305794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05dc910 0x564bc05dd640 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4680003d90 tx=0x7f4680009040 comp rx=0 tx=0).stop 2023-10-06T05:23:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.673+0000 7f469f023b80 1 -- 172.21.15.31:0/1528305794 shutdown_connections 2023-10-06T05:23:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.673+0000 7f469f023b80 1 --2- 172.21.15.31:0/1528305794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05dc910 0x564bc05dd640 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.673+0000 7f469f023b80 1 --2- 172.21.15.31:0/1528305794 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05dce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.673+0000 7f469f023b80 1 --2- 172.21.15.31:0/1528305794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05d9920 0x564bc05acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.673+0000 7f469f023b80 1 -- 172.21.15.31:0/1528305794 >> 172.21.15.31:0/1528305794 conn(0x564bc05fa880 msgr2=0x564bc05fac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.673+0000 7f469f023b80 1 -- 172.21.15.31:0/1528305794 shutdown_connections 2023-10-06T05:23:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.673+0000 7f469f023b80 1 -- 172.21.15.31:0/1528305794 wait complete. 2023-10-06T05:23:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.674+0000 7f469f023b80 1 -- start start 2023-10-06T05:23:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.674+0000 7f469f023b80 1 Processor -- start 2023-10-06T05:23:36.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f469f023b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f469f023b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05dc910 0x564bc05d7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f469f023b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc05d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f469f023b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc05008e0 con 0x564bc05d8130 2023-10-06T05:23:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f469f023b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc04ac980 con 0x564bc05dc910 2023-10-06T05:23:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f469f023b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x564bc04ac620 con 0x564bc05ac930 2023-10-06T05:23:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f469f023b80 10 client.0 _ll_register_callbacks cb 0x564bc030fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:23:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f468ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc05d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f468ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc05d5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58036/0 (socket says 172.21.15.31:58036) 2023-10-06T05:23:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f468ed9c700 1 -- 172.21.15.31:0/808947296 learned_addr learned my addr 172.21.15.31:0/808947296 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f468e59b700 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.675+0000 7f468dd9a700 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05dc910 0x564bc05d7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f468ed9c700 1 -- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 msgr2=0x564bc05cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f468ed9c700 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f468ed9c700 1 -- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05dc910 msgr2=0x564bc05d7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f468ed9c700 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05dc910 0x564bc05d7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f468ed9c700 1 -- 172.21.15.31:0/808947296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468000f040 con 0x564bc05d8130 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f468ed9c700 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc05d5f20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f468000e440 tx=0x7f4680019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a7fc6be0ba9730 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f4668ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f4668ff9700 1 -- 172.21.15.31:0/808947296 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000e9d0 con 0x564bc05d8130 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f4668ff9700 1 -- 172.21.15.31:0/808947296 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f468000ee80 con 0x564bc05d8130 2023-10-06T05:23:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f469f023b80 1 -- 172.21.15.31:0/808947296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bc05a4d60 con 0x564bc05d8130 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f4668ff9700 1 -- 172.21.15.31:0/808947296 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4680003540 con 0x564bc05d8130 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.676+0000 7f469f023b80 10 client.7713 fetch_fsmap learned FSMap version 75 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.677+0000 7f469f023b80 1 -- 172.21.15.31:0/808947296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x564bc03f30a0 con 0x564bc05d8130 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.677+0000 7f4668ff9700 1 -- 172.21.15.31:0/808947296 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564bc05a4d60 con 0x564bc05d8130 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.677+0000 7f4668ff9700 1 -- 172.21.15.31:0/808947296 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 75) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f46800014a0 con 0x564bc05d8130 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.677+0000 7f469f023b80 10 client.7713 fetch_fsmap finished waiting for FSMap version 75 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.677+0000 7f469f023b80 10 client.7713 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.677+0000 7f469f023b80 20 client.7713 populate_metadata read hostname 'smithi031' 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.677+0000 7f469f023b80 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:6833/519548465,v1:172.21.15.62:6835/519548465] conn(0x564bc05d4240 0x564bc05d4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.677+0000 7f469f023b80 4 client.7713 mds_command: new command op to 7378 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:23:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.677+0000 7f469f023b80 1 -- 172.21.15.31:0/808947296 --> [v2:172.21.15.62:6833/519548465,v1:172.21.15.62:6835/519548465] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bc0581b30 con 0x564bc05d4240 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.677+0000 7f468e59b700 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:6833/519548465,v1:172.21.15.62:6835/519548465] conn(0x564bc05d4240 0x564bc05d4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.678+0000 7f468e59b700 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:6833/519548465,v1:172.21.15.62:6835/519548465] conn(0x564bc05d4240 0x564bc05d4610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2177ef550c245333 server_cookie=f9fce8612e08e33f in_seq=0 out_seq=0 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.678+0000 7f4668ff9700 10 client.7713 ms_handle_connect on v2:172.21.15.62:6833/519548465 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.679+0000 7f4668ff9700 1 -- 172.21.15.31:0/808947296 <== mds.0 v2:172.21.15.62:6833/519548465 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564bc0581b30 con 0x564bc05d4240 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.679+0000 7f4668ff9700 10 client.7713 handle_command_reply: tid=0 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.679+0000 7f469f023b80 1 client.7713 shutdown 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.679+0000 7f469f023b80 1 -- 172.21.15.31:0/808947296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 msgr2=0x564bc05d5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.679+0000 7f469f023b80 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc05d5f20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f468000e440 tx=0x7f4680019000 comp rx=0 tx=0).stop 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.680+0000 7f469f023b80 1 -- 172.21.15.31:0/808947296 shutdown_connections 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.680+0000 7f469f023b80 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:6833/519548465,v1:172.21.15.62:6835/519548465] conn(0x564bc05d4240 0x564bc05d4610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.680+0000 7f469f023b80 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc05d5f20 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.680+0000 7f469f023b80 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.680+0000 7f469f023b80 1 --2- 172.21.15.31:0/808947296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05dc910 0x564bc05d7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.680+0000 7f4668ff9700 0 client.7713 ms_handle_reset on v2:172.21.15.62:6833/519548465 2023-10-06T05:23:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.680+0000 7f469f023b80 1 -- 172.21.15.31:0/808947296 >> 172.21.15.31:0/808947296 conn(0x564bc05fa880 msgr2=0x564bc05b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.680+0000 7f469f023b80 1 -- 172.21.15.31:0/808947296 shutdown_connections 2023-10-06T05:23:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.680+0000 7f469f023b80 1 -- 172.21.15.31:0/808947296 wait complete. 2023-10-06T05:23:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.680+0000 7f469f023b80 20 client.7713 trim_cache size 0 max 16384 2023-10-06T05:23:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.697+0000 7f469f023b80 1 Processor -- start 2023-10-06T05:23:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.697+0000 7f469f023b80 1 -- start start 2023-10-06T05:23:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f469f023b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 0x564bc05d4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f469f023b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f469f023b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0657310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f469f023b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc03981e0 con 0x564bc05d8130 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f469f023b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc05dc690 con 0x564bc05fa880 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f469f023b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x564bc05d3cb0 con 0x564bc05ac930 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f468ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0657310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f468ed9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0657310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58046/0 (socket says 172.21.15.31:58046) 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f468ed9c700 1 -- 172.21.15.31:0/3600960054 learned_addr learned my addr 172.21.15.31:0/3600960054 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f468dd9a700 1 --2- 172.21.15.31:0/3600960054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.698+0000 7f468e59b700 1 --2- 172.21.15.31:0/3600960054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 0x564bc05d4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.699+0000 7f468ed9c700 1 -- 172.21.15.31:0/3600960054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 msgr2=0x564bc05aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.699+0000 7f468ed9c700 1 --2- 172.21.15.31:0/3600960054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.699+0000 7f468ed9c700 1 -- 172.21.15.31:0/3600960054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 msgr2=0x564bc05d4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.699+0000 7f468ed9c700 1 --2- 172.21.15.31:0/3600960054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 0x564bc05d4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.699+0000 7f468ed9c700 1 -- 172.21.15.31:0/3600960054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bc04cf3c0 con 0x564bc05d8130 2023-10-06T05:23:36.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.699+0000 7f468e59b700 1 --2- 172.21.15.31:0/3600960054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 0x564bc05d4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:36.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.699+0000 7f468ed9c700 1 --2- 172.21.15.31:0/3600960054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0657310 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f46800012a0 tx=0x7f468001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665ab8b282be6c55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.699+0000 7f466bfff700 1 -- 172.21.15.31:0/3600960054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46800030d0 con 0x564bc05d8130 2023-10-06T05:23:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.699+0000 7f466bfff700 1 -- 172.21.15.31:0/3600960054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000ed80 con 0x564bc05d8130 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.699+0000 7f466bfff700 1 -- 172.21.15.31:0/3600960054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680003490 con 0x564bc05d8130 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.700+0000 7f469f023b80 1 -- 172.21.15.31:0/3600960054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 msgr2=0x564bc0657310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.700+0000 7f469f023b80 1 --2- 172.21.15.31:0/3600960054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0657310 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f46800012a0 tx=0x7f468001f000 comp rx=0 tx=0).stop 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.700+0000 7f469f023b80 1 -- 172.21.15.31:0/3600960054 shutdown_connections 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.700+0000 7f469f023b80 1 --2- 172.21.15.31:0/3600960054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0657310 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.700+0000 7f469f023b80 1 --2- 172.21.15.31:0/3600960054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 0x564bc05d4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.700+0000 7f469f023b80 1 --2- 172.21.15.31:0/3600960054 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc05aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.700+0000 7f469f023b80 1 -- 172.21.15.31:0/3600960054 >> 172.21.15.31:0/3600960054 conn(0x564bc05d9380 msgr2=0x564bc05a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.700+0000 7f469f023b80 1 -- 172.21.15.31:0/3600960054 shutdown_connections 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.700+0000 7f469f023b80 1 -- 172.21.15.31:0/3600960054 wait complete. 2023-10-06T05:23:36.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.700+0000 7f469f023b80 1 -- start start 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.701+0000 7f469f023b80 1 Processor -- start 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f469f023b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc061cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f469f023b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0660a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f469f023b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 0x564bc05f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f469f023b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc061d510 con 0x564bc05d8130 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f469f023b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc06615b0 con 0x564bc05fa880 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f469f023b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x564bc05cdf00 con 0x564bc05ac930 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f469f023b80 10 client.0 _ll_register_callbacks cb 0x564bc030fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f468dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0660a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f468dd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0660a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58056/0 (socket says 172.21.15.31:58056) 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f468dd9a700 1 -- 172.21.15.31:0/2731137576 learned_addr learned my addr 172.21.15.31:0/2731137576 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:36.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f468e59b700 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc061cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f468ed9c700 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 0x564bc05f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f468dd9a700 1 -- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 msgr2=0x564bc061cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f468dd9a700 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc061cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f468dd9a700 1 -- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 msgr2=0x564bc05f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f468dd9a700 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 0x564bc05f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.702+0000 7f468dd9a700 1 -- 172.21.15.31:0/2731137576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bc03f30a0 con 0x564bc05d8130 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f468dd9a700 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0660a90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f467800b3c0 tx=0x7f467800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc5f68d14646e04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f4668ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f4668ff9700 1 -- 172.21.15.31:0/2731137576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467802b340 con 0x564bc05d8130 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f4668ff9700 1 -- 172.21.15.31:0/2731137576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f4678057800 con 0x564bc05d8130 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f469f023b80 1 -- 172.21.15.31:0/2731137576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f468000e650 con 0x564bc05d8130 2023-10-06T05:23:36.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f4668ff9700 1 -- 172.21.15.31:0/2731137576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4678057100 con 0x564bc05d8130 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f469f023b80 10 client.7719 fetch_fsmap learned FSMap version 75 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f469f023b80 1 -- 172.21.15.31:0/2731137576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x564bc03671d0 con 0x564bc05d8130 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f4668ff9700 1 -- 172.21.15.31:0/2731137576 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000e650 con 0x564bc05d8130 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f4668ff9700 1 -- 172.21.15.31:0/2731137576 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 75) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4678084240 con 0x564bc05d8130 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f469f023b80 10 client.7719 fetch_fsmap finished waiting for FSMap version 75 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f469f023b80 10 client.7719 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f469f023b80 20 client.7719 populate_metadata read hostname 'smithi031' 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f469f023b80 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:6833/519548465,v1:172.21.15.62:6835/519548465] conn(0x564bc05dc910 0x564bc05fac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f469f023b80 4 client.7719 mds_command: new command op to 7378 tid=0[{"prefix": "status"}] 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.703+0000 7f469f023b80 1 -- 172.21.15.31:0/2731137576 --> [v2:172.21.15.62:6833/519548465,v1:172.21.15.62:6835/519548465] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564bc04a38d0 con 0x564bc05dc910 2023-10-06T05:23:36.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.704+0000 7f468e59b700 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:6833/519548465,v1:172.21.15.62:6835/519548465] conn(0x564bc05dc910 0x564bc05fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.704+0000 7f468e59b700 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:6833/519548465,v1:172.21.15.62:6835/519548465] conn(0x564bc05dc910 0x564bc05fac50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd30b823de5319c8 server_cookie=7612eea4d1992c0e in_seq=0 out_seq=0 2023-10-06T05:23:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.704+0000 7f4668ff9700 10 client.7719 ms_handle_connect on v2:172.21.15.62:6833/519548465 2023-10-06T05:23:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f4668ff9700 1 -- 172.21.15.31:0/2731137576 <== mds.0 v2:172.21.15.62:6833/519548465 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x564bc04a38d0 con 0x564bc05dc910 2023-10-06T05:23:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f4668ff9700 10 client.7719 handle_command_reply: tid=0 2023-10-06T05:23:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 client.7719 shutdown 2023-10-06T05:23:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 -- 172.21.15.31:0/2731137576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 msgr2=0x564bc0660a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0660a90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f467800b3c0 tx=0x7f467800b040 comp rx=0 tx=0).stop 2023-10-06T05:23:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 -- 172.21.15.31:0/2731137576 shutdown_connections 2023-10-06T05:23:36.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:6833/519548465,v1:172.21.15.62:6835/519548465] conn(0x564bc05dc910 0x564bc05fac50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564bc05fa880 0x564bc05f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564bc05ac930 0x564bc061cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f4668ff9700 0 client.7719 ms_handle_reset on v2:172.21.15.62:6833/519548465 2023-10-06T05:23:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 --2- 172.21.15.31:0/2731137576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bc05d8130 0x564bc0660a90 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 -- 172.21.15.31:0/2731137576 >> 172.21.15.31:0/2731137576 conn(0x564bc05d9380 msgr2=0x564bc06540d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 -- 172.21.15.31:0/2731137576 shutdown_connections 2023-10-06T05:23:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 1 -- 172.21.15.31:0/2731137576 wait complete. 2023-10-06T05:23:36.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.705+0000 7f469f023b80 20 client.7719 trim_cache size 0 max 16384 2023-10-06T05:23:36.712 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 7378, 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5677267419999996, 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 75, 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 114, 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 114, 2023-10-06T05:23:36.713 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.613931239999999 2023-10-06T05:23:36.714 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:23:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 -- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f466c067320 msgr2=0x7f466c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f466c067320 0x7f466c0697d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4684006aa0 tx=0x7f4684008040 comp rx=0 tx=0).stop 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 -- 172.21.15.31:0/2812362595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a1e0 msgr2=0x7f468814e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a1e0 0x7f468814e210 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f46880cc010 tx=0x7f4678017000 comp rx=0 tx=0).stop 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 -- 172.21.15.31:0/2812362595 shutdown_connections 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f466c067320 0x7f466c0697d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f468813a5b0 0x7f4688138d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46880c6c30 0x7f46881522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 --2- 172.21.15.31:0/2812362595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f468813a1e0 0x7f468814e210 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 -- 172.21.15.31:0/2812362595 >> 172.21.15.31:0/2812362595 conn(0x7f46880cc0b0 msgr2=0x7f46880c1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 -- 172.21.15.31:0/2812362595 shutdown_connections 2023-10-06T05:23:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.706+0000 7f46907ff700 1 -- 172.21.15.31:0/2812362595 wait complete. 2023-10-06T05:23:36.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.919+0000 7f29d89ff700 1 Processor -- start 2023-10-06T05:23:36.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.919+0000 7f29d89ff700 1 -- start start 2023-10-06T05:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.920+0000 7f29d89ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d40cd510 0x7f29d40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.920+0000 7f29d89ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f29d415d2d0 0x7f29d40aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.920+0000 7f29d89ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d40c3f40 con 0x7f29d4159500 2023-10-06T05:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.920+0000 7f29d89ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d40b2860 con 0x7f29d415d2d0 2023-10-06T05:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.920+0000 7f29d89ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d415d7d0 con 0x7f29d40cd510 2023-10-06T05:23:36.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.921+0000 7f29d1d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d40cd510 0x7f29d40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.921+0000 7f29d1d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d40cd510 0x7f29d40b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:46230/0 (socket says 172.21.15.31:46230) 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.921+0000 7f29d1d9b700 1 -- 172.21.15.31:0/830217705 learned_addr learned my addr 172.21.15.31:0/830217705 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.921+0000 7f29d2d9d700 1 --2- 172.21.15.31:0/830217705 >> v2:172.21.15.62:3300/0 conn(0x7f29d415d2d0 0x7f29d40aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.921+0000 7f29d1d9b700 1 -- 172.21.15.31:0/830217705 >> v2:172.21.15.62:3300/0 conn(0x7f29d415d2d0 msgr2=0x7f29d40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.921+0000 7f29d1d9b700 1 --2- 172.21.15.31:0/830217705 >> v2:172.21.15.62:3300/0 conn(0x7f29d415d2d0 0x7f29d40aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.921+0000 7f29d1d9b700 1 -- 172.21.15.31:0/830217705 >> v1:172.21.15.62:6789/0 conn(0x7f29d4159500 legacy=0x7f29d40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.921+0000 7f29d1d9b700 1 -- 172.21.15.31:0/830217705 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40cb000 con 0x7f29d40cd510 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.921+0000 7f29d159a700 1 -- 172.21.15.31:0/830217705 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888887705 0 0) 0x7f29d40c3f40 con 0x7f29d4159500 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.921+0000 7f29d1d9b700 1 --2- 172.21.15.31:0/830217705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d40cd510 0x7f29d40b0ab0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009990 tx=0x7f29c8009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d94d474edbd77939 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.922+0000 7f29d159a700 1 -- 172.21.15.31:0/830217705 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8009e80 con 0x7f29d40cd510 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.922+0000 7f29d159a700 1 -- 172.21.15.31:0/830217705 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c8005c10 con 0x7f29d40cd510 2023-10-06T05:23:36.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.922+0000 7f29d159a700 1 -- 172.21.15.31:0/830217705 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c800b650 con 0x7f29d40cd510 2023-10-06T05:23:36.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.922+0000 7f29d89ff700 1 -- 172.21.15.31:0/830217705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d40cd510 msgr2=0x7f29d40b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.922+0000 7f29d89ff700 1 --2- 172.21.15.31:0/830217705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d40cd510 0x7f29d40b0ab0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009990 tx=0x7f29c8009220 comp rx=0 tx=0).stop 2023-10-06T05:23:36.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.922+0000 7f29d89ff700 1 -- 172.21.15.31:0/830217705 shutdown_connections 2023-10-06T05:23:36.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.922+0000 7f29d89ff700 1 --2- 172.21.15.31:0/830217705 >> v2:172.21.15.62:3300/0 conn(0x7f29d415d2d0 0x7f29d40aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.922+0000 7f29d89ff700 1 --2- 172.21.15.31:0/830217705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d40cd510 0x7f29d40b0ab0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.922+0000 7f29d89ff700 1 -- 172.21.15.31:0/830217705 >> 172.21.15.31:0/830217705 conn(0x7f29d406ec90 msgr2=0x7f29d406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:36.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.923+0000 7f29d89ff700 1 -- 172.21.15.31:0/830217705 shutdown_connections 2023-10-06T05:23:36.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.923+0000 7f29d89ff700 1 -- 172.21.15.31:0/830217705 wait complete. 2023-10-06T05:23:36.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.924+0000 7f29d89ff700 1 Processor -- start 2023-10-06T05:23:36.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.924+0000 7f29d89ff700 1 -- start start 2023-10-06T05:23:36.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.925+0000 7f29d89ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d40cd510 0x7f29d40b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.925+0000 7f29d89ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29d4159500 0x7f29d41557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.925+0000 7f29d89ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d415d2d0 0x7f29d4157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.925+0000 7f29d89ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d415b3d0 con 0x7f29d4159500 2023-10-06T05:23:36.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.925+0000 7f29d89ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d415b860 con 0x7f29d40cd510 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.925+0000 7f29d89ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d415a7c0 con 0x7f29d415d2d0 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.925+0000 7f29d1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29d4159500 0x7f29d41557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.925+0000 7f29d1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29d4159500 0x7f29d41557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44142/0 (socket says 172.21.15.31:44142) 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.925+0000 7f29d1d9b700 1 -- 172.21.15.31:0/4224820620 learned_addr learned my addr 172.21.15.31:0/4224820620 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29d2d9d700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d415d2d0 0x7f29d4157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29d259c700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d40cd510 0x7f29d40b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29d1d9b700 1 -- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d415d2d0 msgr2=0x7f29d4157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29d1d9b700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d415d2d0 0x7f29d4157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29d1d9b700 1 -- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d40cd510 msgr2=0x7f29d40b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29d1d9b700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d40cd510 0x7f29d40b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:36.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29d1d9b700 1 -- 172.21.15.31:0/4224820620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c8009480 con 0x7f29d4159500 2023-10-06T05:23:36.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29d259c700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d40cd510 0x7f29d40b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:36.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29d1d9b700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29d4159500 0x7f29d41557f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f29d4067c50 tx=0x7f29c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29c27fc700 1 -- 172.21.15.31:0/4224820620 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c801dc10 con 0x7f29d4159500 2023-10-06T05:23:36.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29c27fc700 1 -- 172.21.15.31:0/4224820620 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c801dd70 con 0x7f29d4159500 2023-10-06T05:23:36.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29d89ff700 1 -- 172.21.15.31:0/4224820620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d415aaa0 con 0x7f29d4159500 2023-10-06T05:23:36.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.926+0000 7f29c27fc700 1 -- 172.21.15.31:0/4224820620 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c801a670 con 0x7f29d4159500 2023-10-06T05:23:36.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.927+0000 7f29d89ff700 1 -- 172.21.15.31:0/4224820620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d415b060 con 0x7f29d4159500 2023-10-06T05:23:36.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.928+0000 7f29d89ff700 1 -- 172.21.15.31:0/4224820620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d404c380 con 0x7f29d4159500 2023-10-06T05:23:36.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.928+0000 7f29c27fc700 1 -- 172.21.15.31:0/4224820620 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f29c8011420 con 0x7f29d4159500 2023-10-06T05:23:36.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.930+0000 7f29c27fc700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29b4067320 0x7f29b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:36.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.930+0000 7f29c27fc700 1 -- 172.21.15.31:0/4224820620 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f29c8086010 con 0x7f29d4159500 2023-10-06T05:23:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.930+0000 7f29d259c700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29b4067320 0x7f29b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.931+0000 7f29d259c700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29b4067320 0x7f29b40697d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f29c40011e0 tx=0x7f29c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:36.934+0000 7f29c27fc700 1 -- 172.21.15.31:0/4224820620 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29c8052bd0 con 0x7f29d4159500 2023-10-06T05:23:37.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.186+0000 7f29d89ff700 1 -- 172.21.15.31:0/4224820620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f29d415b510 con 0x7f29d4159500 2023-10-06T05:23:37.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.186+0000 7f29c27fc700 1 -- 172.21.15.31:0/4224820620 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7f29c80561f0 con 0x7f29d4159500 2023-10-06T05:23:37.188 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:23:37.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 -- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29b4067320 msgr2=0x7f29b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29b4067320 0x7f29b40697d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f29c40011e0 tx=0x7f29c4009040 comp rx=0 tx=0).stop 2023-10-06T05:23:37.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 -- 172.21.15.31:0/4224820620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29d4159500 msgr2=0x7f29d41557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:37.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29d4159500 0x7f29d41557f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f29d4067c50 tx=0x7f29c8016000 comp rx=0 tx=0).stop 2023-10-06T05:23:37.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 -- 172.21.15.31:0/4224820620 shutdown_connections 2023-10-06T05:23:37.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29b4067320 0x7f29b40697d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f29d415d2d0 0x7f29d4157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d40cd510 0x7f29d40b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 --2- 172.21.15.31:0/4224820620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29d4159500 0x7f29d41557f0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:37.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 -- 172.21.15.31:0/4224820620 >> 172.21.15.31:0/4224820620 conn(0x7f29d406ec90 msgr2=0x7f29d41562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:37.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 -- 172.21.15.31:0/4224820620 shutdown_connections 2023-10-06T05:23:37.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:37.189+0000 7f29d89ff700 1 -- 172.21.15.31:0/4224820620 wait complete. 2023-10-06T05:23:37.208 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:23:37.208 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:23:37.208 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:23:37.208 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:23:37.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:37.208 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:23:37.208 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:23:37.208 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:23:37.209 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:37.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:37.209 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:23:37.257 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:23:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:23:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:23:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:37.257 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:23:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:37.258 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:23:37.258 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:23:37.258 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:23:37.258 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:23:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:23:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41897sec preferred_lft 41897sec 2023-10-06T05:23:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:23:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:37.259 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:37.259 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:23:37.259 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:23:37.260 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:23:37.261 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:37.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:37.261 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:23:37.329 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:23:37.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:37.332 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:23:37.381 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:23:37.381 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:23:37.381 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:23:37.382 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:23:37.382 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:23:37.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:37.383 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:23:37.383 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:23:37.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:23:37.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:23:37.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:23:37.383 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:23:37.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:37.555 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:23:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:23:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:23:37.603 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:37.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:37.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:23:37.675 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:37.675 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:37.722 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:23:37.722 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:37.722 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:37.808 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:23:37.808 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:23:37.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:23:37.948 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:23:37.949 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:23:37.975 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:23:37.976 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:23:37.976 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:23:37.976 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:23:37.976 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:23:37.981 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:23:37.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:37.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:38.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:23:38.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.249+0000 7fb517bff700 1 Processor -- start 2023-10-06T05:23:38.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.249+0000 7fb517bff700 1 -- start start 2023-10-06T05:23:38.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.250+0000 7fb517bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5101595a0 0x7fb510159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.250+0000 7fb517bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100c95b0 con 0x7fb5100cc0d0 2023-10-06T05:23:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.250+0000 7fb517bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100c9890 con 0x7fb510068d80 2023-10-06T05:23:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.250+0000 7fb517bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100c9b70 con 0x7fb5101595a0 2023-10-06T05:23:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.251+0000 7fb51519a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb5100cc0d0 0x7fb510068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45786/0 (socket says 172.21.15.31:45786) 2023-10-06T05:23:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.251+0000 7fb51519a700 1 -- 172.21.15.31:0/2786820532 learned_addr learned my addr 172.21.15.31:0/2786820532 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.251+0000 7fb51599b700 1 --2- 172.21.15.31:0/2786820532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5101595a0 0x7fb510159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.251+0000 7fb514999700 1 -- 172.21.15.31:0/2786820532 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926750213 0 0) 0x7fb5100c95b0 con 0x7fb5100cc0d0 2023-10-06T05:23:38.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.251+0000 7fb514999700 1 -- 172.21.15.31:0/2786820532 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4fc003570 con 0x7fb5100cc0d0 2023-10-06T05:23:38.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.251+0000 7fb514999700 1 -- 172.21.15.31:0/2786820532 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776590227 0 0) 0x7fb5100c9890 con 0x7fb510068d80 2023-10-06T05:23:38.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.251+0000 7fb514999700 1 -- 172.21.15.31:0/2786820532 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5100c95b0 con 0x7fb510068d80 2023-10-06T05:23:38.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.251+0000 7fb51599b700 1 -- 172.21.15.31:0/2786820532 >> v1:172.21.15.62:6789/0 conn(0x7fb510068d80 legacy=0x7fb5100c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.251+0000 7fb51599b700 1 -- 172.21.15.31:0/2786820532 >> v1:172.21.15.31:6789/0 conn(0x7fb5100cc0d0 legacy=0x7fb510068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.251+0000 7fb51599b700 1 -- 172.21.15.31:0/2786820532 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100ca130 con 0x7fb5101595a0 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.252+0000 7fb51599b700 1 --2- 172.21.15.31:0/2786820532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5101595a0 0x7fb510159970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009910 tx=0x7fb50c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a8c8bb2384089f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.252+0000 7fb514999700 1 -- 172.21.15.31:0/2786820532 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c011630 con 0x7fb5101595a0 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.252+0000 7fb514999700 1 -- 172.21.15.31:0/2786820532 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c011790 con 0x7fb5101595a0 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.252+0000 7fb514999700 1 -- 172.21.15.31:0/2786820532 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c011b30 con 0x7fb5101595a0 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.252+0000 7fb517bff700 1 -- 172.21.15.31:0/2786820532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5101595a0 msgr2=0x7fb510159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.252+0000 7fb517bff700 1 --2- 172.21.15.31:0/2786820532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5101595a0 0x7fb510159970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009910 tx=0x7fb50c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.253+0000 7fb517bff700 1 -- 172.21.15.31:0/2786820532 shutdown_connections 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.253+0000 7fb517bff700 1 --2- 172.21.15.31:0/2786820532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5101595a0 0x7fb510159970 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.253+0000 7fb517bff700 1 -- 172.21.15.31:0/2786820532 >> 172.21.15.31:0/2786820532 conn(0x7fb51006b620 msgr2=0x7fb51006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.253+0000 7fb517bff700 1 -- 172.21.15.31:0/2786820532 shutdown_connections 2023-10-06T05:23:38.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.253+0000 7fb517bff700 1 -- 172.21.15.31:0/2786820532 wait complete. 2023-10-06T05:23:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.254+0000 7fb517bff700 1 Processor -- start 2023-10-06T05:23:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.254+0000 7fb517bff700 1 -- start start 2023-10-06T05:23:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.255+0000 7fb517bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb510068d80 0x7fb5101563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.255+0000 7fb517bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5100cc0d0 0x7fb510156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.255+0000 7fb517bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5101595a0 0x7fb510156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.255+0000 7fb517bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51016b550 con 0x7fb510068d80 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.255+0000 7fb517bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51016b830 con 0x7fb5101595a0 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.255+0000 7fb517bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51016bb10 con 0x7fb5100cc0d0 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.255+0000 7fb51599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb510068d80 0x7fb5101563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.255+0000 7fb51599b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb510068d80 0x7fb5101563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44146/0 (socket says 172.21.15.31:44146) 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.255+0000 7fb51599b700 1 -- 172.21.15.31:0/2722539116 learned_addr learned my addr 172.21.15.31:0/2722539116 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.256+0000 7fb51519a700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5100cc0d0 0x7fb510156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.256+0000 7fb51619c700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5101595a0 0x7fb510156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.256+0000 7fb51599b700 1 -- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5100cc0d0 msgr2=0x7fb510156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.256+0000 7fb51599b700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5100cc0d0 0x7fb510156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.256+0000 7fb51599b700 1 -- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5101595a0 msgr2=0x7fb510156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.256+0000 7fb51599b700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5101595a0 0x7fb510156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.256+0000 7fb51599b700 1 -- 172.21.15.31:0/2722539116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c009420 con 0x7fb510068d80 2023-10-06T05:23:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.256+0000 7fb51619c700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5101595a0 0x7fb510156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.256+0000 7fb51599b700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb510068d80 0x7fb5101563f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb51006ee00 tx=0x7fb50c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.256+0000 7fb505ffb700 1 -- 172.21.15.31:0/2722539116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c011b10 con 0x7fb510068d80 2023-10-06T05:23:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.257+0000 7fb505ffb700 1 -- 172.21.15.31:0/2722539116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c0253f0 con 0x7fb510068d80 2023-10-06T05:23:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.257+0000 7fb505ffb700 1 -- 172.21.15.31:0/2722539116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c025790 con 0x7fb510068d80 2023-10-06T05:23:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.257+0000 7fb517bff700 1 -- 172.21.15.31:0/2722539116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51016bdf0 con 0x7fb510068d80 2023-10-06T05:23:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.257+0000 7fb517bff700 1 -- 172.21.15.31:0/2722539116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51016a3a0 con 0x7fb510068d80 2023-10-06T05:23:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.258+0000 7fb517bff700 1 -- 172.21.15.31:0/2722539116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51004c380 con 0x7fb510068d80 2023-10-06T05:23:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.259+0000 7fb505ffb700 1 -- 172.21.15.31:0/2722539116 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb50c025b40 con 0x7fb510068d80 2023-10-06T05:23:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.259+0000 7fb505ffb700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4f4067320 0x7fb4f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.260+0000 7fb505ffb700 1 -- 172.21.15.31:0/2722539116 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb50c086630 con 0x7fb510068d80 2023-10-06T05:23:38.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.260+0000 7fb51519a700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4f4067320 0x7fb4f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:38.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.260+0000 7fb51519a700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4f4067320 0x7fb4f40697d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb500007fd0 tx=0x7fb500009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:38.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.264+0000 7fb505ffb700 1 -- 172.21.15.31:0/2722539116 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb50c053100 con 0x7fb510068d80 2023-10-06T05:23:38.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.509+0000 7fb517bff700 1 -- 172.21.15.31:0/2722539116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb51016b690 con 0x7fb510068d80 2023-10-06T05:23:38.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.509+0000 7fb505ffb700 1 -- 172.21.15.31:0/2722539116 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb51016b690 con 0x7fb510068d80 2023-10-06T05:23:38.514 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:23:38.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 -- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4f4067320 msgr2=0x7fb4f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4f4067320 0x7fb4f40697d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb500007fd0 tx=0x7fb500009040 comp rx=0 tx=0).stop 2023-10-06T05:23:38.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 -- 172.21.15.31:0/2722539116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb510068d80 msgr2=0x7fb5101563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:38.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb510068d80 0x7fb5101563f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb51006ee00 tx=0x7fb50c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:38.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 -- 172.21.15.31:0/2722539116 shutdown_connections 2023-10-06T05:23:38.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4f4067320 0x7fb4f40697d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5101595a0 0x7fb510156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb510068d80 0x7fb5101563f0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 --2- 172.21.15.31:0/2722539116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5100cc0d0 0x7fb510156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:38.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 -- 172.21.15.31:0/2722539116 >> 172.21.15.31:0/2722539116 conn(0x7fb51006b620 msgr2=0x7fb51006cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:38.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 -- 172.21.15.31:0/2722539116 shutdown_connections 2023-10-06T05:23:38.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:38.511+0000 7fb517bff700 1 -- 172.21.15.31:0/2722539116 wait complete. 2023-10-06T05:23:38.531 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:23:38.531 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:23:38.531 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:23:38.532 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:23:38.533 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:38.533 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:23:38.533 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:23:38.533 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:23:38.534 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:23:38.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:38.534 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:23:38.587 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:23:38.587 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:23:38.587 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:23:38.587 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:38.587 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:23:38.588 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:38.588 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:38.588 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:38.588 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:23:38.588 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:38.588 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:38.588 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:23:38.588 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:23:38.588 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:38.588 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:23:38.589 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:38.589 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:23:38.589 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41896sec preferred_lft 41896sec 2023-10-06T05:23:38.589 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:23:38.589 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:38.589 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:23:38.589 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:23:38.589 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:23:38.589 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:38.589 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:23:38.590 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:38.590 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:38.590 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:23:38.590 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:23:38.590 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:38.590 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:23:38.590 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:23:38.590 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:23:38.590 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:23:38.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:38.592 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:23:38.664 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:23:38.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:38.669 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:23:38.719 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:23:38.719 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:23:38.719 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:23:38.719 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:23:38.719 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:23:38.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:38.721 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:23:38.721 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:23:38.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:23:38.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:23:38.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:23:38.721 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:23:38.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:38.892 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:23:38.938 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:23:38.939 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:23:38.940 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:23:38.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:38.940 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:23:39.011 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:39.012 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:39.058 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:23:39.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:23:39.058 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:39.143 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:23:39.143 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:23:39.143 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:23:39.284 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:23:39.284 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:23:39.310 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:23:39.311 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:23:39.311 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:23:39.311 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:23:39.311 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:23:39.314 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:23:39.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:39.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:39.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:23:39.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.595+0000 7f52715ff700 1 Processor -- start 2023-10-06T05:23:39.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.595+0000 7f52715ff700 1 -- start start 2023-10-06T05:23:39.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.596+0000 7f52715ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c153d20 0x7f526c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.596+0000 7f52715ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526c15bb20 con 0x7f526c15d750 2023-10-06T05:23:39.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.596+0000 7f52715ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526c15af10 con 0x7f526c153950 2023-10-06T05:23:39.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.596+0000 7f52715ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526c154170 con 0x7f526c153d20 2023-10-06T05:23:39.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.596+0000 7f526b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f526c15d750 0x7f526c15bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45796/0 (socket says 172.21.15.31:45796) 2023-10-06T05:23:39.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.596+0000 7f526b7fe700 1 -- 172.21.15.31:0/1401089103 learned_addr learned my addr 172.21.15.31:0/1401089103 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:39.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.597+0000 7f526a7fc700 1 --2- 172.21.15.31:0/1401089103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c153d20 0x7f526c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.597+0000 7f5269ffb700 1 -- 172.21.15.31:0/1401089103 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836358395 0 0) 0x7f526c15bb20 con 0x7f526c15d750 2023-10-06T05:23:39.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.597+0000 7f5269ffb700 1 -- 172.21.15.31:0/1401089103 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5254003570 con 0x7f526c15d750 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.597+0000 7f5269ffb700 1 -- 172.21.15.31:0/1401089103 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263902979 0 0) 0x7f526c15af10 con 0x7f526c153950 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.597+0000 7f526a7fc700 1 -- 172.21.15.31:0/1401089103 >> v1:172.21.15.62:6789/0 conn(0x7f526c153950 legacy=0x7f526c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.597+0000 7f526a7fc700 1 -- 172.21.15.31:0/1401089103 >> v1:172.21.15.31:6789/0 conn(0x7f526c15d750 legacy=0x7f526c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.597+0000 7f526a7fc700 1 -- 172.21.15.31:0/1401089103 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c15b1f0 con 0x7f526c153d20 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.597+0000 7f526a7fc700 1 --2- 172.21.15.31:0/1401089103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c153d20 0x7f526c15d210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5260009910 tx=0x7f52600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc4fd6b769f7599b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.598+0000 7f5269ffb700 1 -- 172.21.15.31:0/1401089103 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000b680 con 0x7f526c153d20 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.598+0000 7f5269ffb700 1 -- 172.21.15.31:0/1401089103 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000b7e0 con 0x7f526c153d20 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.598+0000 7f5269ffb700 1 -- 172.21.15.31:0/1401089103 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000bba0 con 0x7f526c153d20 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.598+0000 7f52715ff700 1 -- 172.21.15.31:0/1401089103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c153d20 msgr2=0x7f526c15d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.598+0000 7f52715ff700 1 --2- 172.21.15.31:0/1401089103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c153d20 0x7f526c15d210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5260009910 tx=0x7f52600091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.598+0000 7f52715ff700 1 -- 172.21.15.31:0/1401089103 shutdown_connections 2023-10-06T05:23:39.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.598+0000 7f52715ff700 1 --2- 172.21.15.31:0/1401089103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c153d20 0x7f526c15d210 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.598+0000 7f52715ff700 1 -- 172.21.15.31:0/1401089103 >> 172.21.15.31:0/1401089103 conn(0x7f526c0681b0 msgr2=0x7f526c0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:39.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.598+0000 7f52715ff700 1 -- 172.21.15.31:0/1401089103 shutdown_connections 2023-10-06T05:23:39.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.598+0000 7f52715ff700 1 -- 172.21.15.31:0/1401089103 wait complete. 2023-10-06T05:23:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.600+0000 7f52715ff700 1 Processor -- start 2023-10-06T05:23:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.600+0000 7f52715ff700 1 -- start start 2023-10-06T05:23:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f52715ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f526c153950 0x7f526c151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f52715ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f526c153d20 0x7f526c0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f52715ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c15d750 0x7f526c169930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f52715ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c16a970 con 0x7f526c153d20 2023-10-06T05:23:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f52715ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c16a3b0 con 0x7f526c153950 2023-10-06T05:23:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f52715ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526c16a690 con 0x7f526c15d750 2023-10-06T05:23:39.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f526c153d20 0x7f526c0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f526c153d20 0x7f526c0c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44158/0 (socket says 172.21.15.31:44158) 2023-10-06T05:23:39.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526a7fc700 1 -- 172.21.15.31:0/3974274816 learned_addr learned my addr 172.21.15.31:0/3974274816 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:39.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526b7fe700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c15d750 0x7f526c169930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526affd700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f526c153950 0x7f526c151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526a7fc700 1 -- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c15d750 msgr2=0x7f526c169930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526a7fc700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c15d750 0x7f526c169930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526a7fc700 1 -- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f526c153950 msgr2=0x7f526c151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:39.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526a7fc700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f526c153950 0x7f526c151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:39.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526a7fc700 1 -- 172.21.15.31:0/3974274816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5260009420 con 0x7f526c153d20 2023-10-06T05:23:39.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.601+0000 7f526affd700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f526c153950 0x7f526c151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:39.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.602+0000 7f526a7fc700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f526c153d20 0x7f526c0c9d20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f526c067bb0 tx=0x7f5260017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:39.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.602+0000 7f5252ffd700 1 -- 172.21.15.31:0/3974274816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000bb60 con 0x7f526c153d20 2023-10-06T05:23:39.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.602+0000 7f52715ff700 1 -- 172.21.15.31:0/3974274816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526c0c9520 con 0x7f526c153d20 2023-10-06T05:23:39.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.602+0000 7f5252ffd700 1 -- 172.21.15.31:0/3974274816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5260027450 con 0x7f526c153d20 2023-10-06T05:23:39.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.602+0000 7f5252ffd700 1 -- 172.21.15.31:0/3974274816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52600277f0 con 0x7f526c153d20 2023-10-06T05:23:39.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.602+0000 7f52715ff700 1 -- 172.21.15.31:0/3974274816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526c150930 con 0x7f526c153d20 2023-10-06T05:23:39.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.603+0000 7f52715ff700 1 -- 172.21.15.31:0/3974274816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526c150b20 con 0x7f526c153d20 2023-10-06T05:23:39.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.604+0000 7f5252ffd700 1 -- 172.21.15.31:0/3974274816 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f526001a430 con 0x7f526c153d20 2023-10-06T05:23:39.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.605+0000 7f5252ffd700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5258067320 0x7f52580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:39.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.605+0000 7f5252ffd700 1 -- 172.21.15.31:0/3974274816 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f5260086d40 con 0x7f526c153d20 2023-10-06T05:23:39.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.605+0000 7f526affd700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5258067320 0x7f52580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:39.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.606+0000 7f526affd700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5258067320 0x7f52580697d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f525c0018b0 tx=0x7f525c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:39.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.609+0000 7f5252ffd700 1 -- 172.21.15.31:0/3974274816 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f526c150b20 con 0x7f526c153d20 2023-10-06T05:23:39.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:39.853+0000 7f52715ff700 1 -- 172.21.15.31:0/3974274816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f526c0c9380 con 0x7f526c153d20 2023-10-06T05:23:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.183+0000 7f5252ffd700 1 -- 172.21.15.31:0/3974274816 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 136+0+0 (secure 0 0 0) 0x7f526c0c9380 con 0x7f526c153d20 2023-10-06T05:23:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 -- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5258067320 msgr2=0x7f52580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5258067320 0x7f52580697d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f525c0018b0 tx=0x7f525c009040 comp rx=0 tx=0).stop 2023-10-06T05:23:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 -- 172.21.15.31:0/3974274816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f526c153d20 msgr2=0x7f526c0c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f526c153d20 0x7f526c0c9d20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f526c067bb0 tx=0x7f5260017000 comp rx=0 tx=0).stop 2023-10-06T05:23:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 -- 172.21.15.31:0/3974274816 shutdown_connections 2023-10-06T05:23:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5258067320 0x7f52580697d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f526c15d750 0x7f526c169930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f526c153950 0x7f526c151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 --2- 172.21.15.31:0/3974274816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f526c153d20 0x7f526c0c9d20 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 -- 172.21.15.31:0/3974274816 >> 172.21.15.31:0/3974274816 conn(0x7f526c0681b0 msgr2=0x7f526c1530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 -- 172.21.15.31:0/3974274816 shutdown_connections 2023-10-06T05:23:40.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.186+0000 7f52715ff700 1 -- 172.21.15.31:0/3974274816 wait complete. 2023-10-06T05:23:40.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:23:40.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.402+0000 7f0680bff700 1 Processor -- start 2023-10-06T05:23:40.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.402+0000 7f0680bff700 1 -- start start 2023-10-06T05:23:40.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.403+0000 7f0680bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f067c0aed50 0x7f067c069590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.403+0000 7f0680bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f067c0aca50 0x7f067c1631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.403+0000 7f0680bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067c0ab5b0 con 0x7f067c068670 2023-10-06T05:23:40.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.403+0000 7f0680bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067c159a00 con 0x7f067c0aed50 2023-10-06T05:23:40.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.403+0000 7f0680bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067c069de0 con 0x7f067c0aca50 2023-10-06T05:23:40.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.403+0000 7f067a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f067c0aed50 0x7f067c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.403+0000 7f0679d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f067c068670 0x7f067c0af120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45804/0 (socket says 172.21.15.31:45804) 2023-10-06T05:23:40.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.403+0000 7f0679d9b700 1 -- 172.21.15.31:0/2939796385 learned_addr learned my addr 172.21.15.31:0/2939796385 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:40.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.404+0000 7f067ad9d700 1 --2- 172.21.15.31:0/2939796385 >> v2:172.21.15.62:3300/0 conn(0x7f067c0aca50 0x7f067c1631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.404+0000 7f067959a700 1 -- 172.21.15.31:0/2939796385 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265802108 0 0) 0x7f067c0ab5b0 con 0x7f067c068670 2023-10-06T05:23:40.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.404+0000 7f067a59c700 1 -- 172.21.15.31:0/2939796385 >> v2:172.21.15.62:3300/0 conn(0x7f067c0aca50 msgr2=0x7f067c1631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.404+0000 7f067a59c700 1 --2- 172.21.15.31:0/2939796385 >> v2:172.21.15.62:3300/0 conn(0x7f067c0aca50 0x7f067c1631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.404+0000 7f067a59c700 1 -- 172.21.15.31:0/2939796385 >> v1:172.21.15.31:6789/0 conn(0x7f067c068670 legacy=0x7f067c0af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.404+0000 7f067a59c700 1 -- 172.21.15.31:0/2939796385 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c0aaf60 con 0x7f067c0aed50 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.404+0000 7f067a59c700 1 --2- 172.21.15.31:0/2939796385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c0aed50 0x7f067c069590 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f067c06a830 tx=0x7f066c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99878562baadeaa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.404+0000 7f067959a700 1 -- 172.21.15.31:0/2939796385 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c011630 con 0x7f067c0aed50 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.404+0000 7f067959a700 1 -- 172.21.15.31:0/2939796385 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c011790 con 0x7f067c0aed50 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.404+0000 7f067959a700 1 -- 172.21.15.31:0/2939796385 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c011b30 con 0x7f067c0aed50 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.405+0000 7f0680bff700 1 -- 172.21.15.31:0/2939796385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c0aed50 msgr2=0x7f067c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.405+0000 7f0680bff700 1 --2- 172.21.15.31:0/2939796385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c0aed50 0x7f067c069590 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f067c06a830 tx=0x7f066c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.405+0000 7f0680bff700 1 -- 172.21.15.31:0/2939796385 shutdown_connections 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.405+0000 7f0680bff700 1 --2- 172.21.15.31:0/2939796385 >> v2:172.21.15.62:3300/0 conn(0x7f067c0aca50 0x7f067c1631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.405+0000 7f0680bff700 1 --2- 172.21.15.31:0/2939796385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c0aed50 0x7f067c069590 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.405+0000 7f0680bff700 1 -- 172.21.15.31:0/2939796385 >> 172.21.15.31:0/2939796385 conn(0x7f067c097550 msgr2=0x7f067c0639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.405+0000 7f0680bff700 1 -- 172.21.15.31:0/2939796385 shutdown_connections 2023-10-06T05:23:40.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.405+0000 7f0680bff700 1 -- 172.21.15.31:0/2939796385 wait complete. 2023-10-06T05:23:40.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.407+0000 7f0680bff700 1 Processor -- start 2023-10-06T05:23:40.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.407+0000 7f0680bff700 1 -- start start 2023-10-06T05:23:40.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f0680bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c068670 0x7f067c16d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f0680bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c0aca50 0x7f067c169650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f0680bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c0aed50 0x7f067c15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f0680bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c16d060 con 0x7f067c068670 2023-10-06T05:23:40.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f0680bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c169f80 con 0x7f067c0aed50 2023-10-06T05:23:40.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f0680bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067c16a260 con 0x7f067c0aca50 2023-10-06T05:23:40.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c068670 0x7f067c16d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c068670 0x7f067c16d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44164/0 (socket says 172.21.15.31:44164) 2023-10-06T05:23:40.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067a59c700 1 -- 172.21.15.31:0/733664415 learned_addr learned my addr 172.21.15.31:0/733664415 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:40.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f0679d9b700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c0aca50 0x7f067c169650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067ad9d700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c0aed50 0x7f067c15e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067a59c700 1 -- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c0aca50 msgr2=0x7f067c169650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067a59c700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c0aca50 0x7f067c169650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067a59c700 1 -- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c0aed50 msgr2=0x7f067c15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067a59c700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c0aed50 0x7f067c15e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067a59c700 1 -- 172.21.15.31:0/733664415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c009420 con 0x7f067c068670 2023-10-06T05:23:40.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f0679d9b700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c0aca50 0x7f067c169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:40.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067ad9d700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c0aed50 0x7f067c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:40.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.408+0000 7f067a59c700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c068670 0x7f067c16d4f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f067c06f250 tx=0x7f066c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.409+0000 7f066a7fc700 1 -- 172.21.15.31:0/733664415 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c011b10 con 0x7f067c068670 2023-10-06T05:23:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.409+0000 7f066a7fc700 1 -- 172.21.15.31:0/733664415 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c0253f0 con 0x7f067c068670 2023-10-06T05:23:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.409+0000 7f066a7fc700 1 -- 172.21.15.31:0/733664415 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c025790 con 0x7f067c068670 2023-10-06T05:23:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.409+0000 7f0680bff700 1 -- 172.21.15.31:0/733664415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067c16a540 con 0x7f067c068670 2023-10-06T05:23:40.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.409+0000 7f0680bff700 1 -- 172.21.15.31:0/733664415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c16da30 con 0x7f067c068670 2023-10-06T05:23:40.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.410+0000 7f0680bff700 1 -- 172.21.15.31:0/733664415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067c04c380 con 0x7f067c068670 2023-10-06T05:23:40.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.411+0000 7f066a7fc700 1 -- 172.21.15.31:0/733664415 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f066c025b80 con 0x7f067c068670 2023-10-06T05:23:40.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.412+0000 7f066a7fc700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f065c067350 0x7f065c069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.412+0000 7f066a7fc700 1 -- 172.21.15.31:0/733664415 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f066c086660 con 0x7f067c068670 2023-10-06T05:23:40.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.412+0000 7f0679d9b700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f065c067350 0x7f065c069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.413+0000 7f0679d9b700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f065c067350 0x7f065c069800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f067c068ec0 tx=0x7f0670009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.416+0000 7f066a7fc700 1 -- 172.21.15.31:0/733664415 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f066c053130 con 0x7f067c068670 2023-10-06T05:23:40.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.647+0000 7f0680bff700 1 -- 172.21.15.31:0/733664415 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f067c16d1a0 con 0x7f065c067350 2023-10-06T05:23:40.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.650+0000 7f066a7fc700 1 -- 172.21.15.31:0/733664415 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f06700014f0 con 0x7f065c067350 2023-10-06T05:23:40.652 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:23:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:23:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:23:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:40.652 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:23:40.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.651+0000 7f0680bff700 1 -- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f065c067350 msgr2=0x7f065c069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.651+0000 7f0680bff700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f065c067350 0x7f065c069800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f067c068ec0 tx=0x7f0670009040 comp rx=0 tx=0).stop 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.651+0000 7f0680bff700 1 -- 172.21.15.31:0/733664415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c068670 msgr2=0x7f067c16d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.651+0000 7f0680bff700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c068670 0x7f067c16d4f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f067c06f250 tx=0x7f066c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.652+0000 7f0680bff700 1 -- 172.21.15.31:0/733664415 shutdown_connections 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.652+0000 7f0680bff700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f065c067350 0x7f065c069800 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.652+0000 7f0680bff700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c0aed50 0x7f067c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.652+0000 7f0680bff700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f067c068670 0x7f067c16d4f0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.652+0000 7f0680bff700 1 --2- 172.21.15.31:0/733664415 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c0aca50 0x7f067c169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.652+0000 7f0680bff700 1 -- 172.21.15.31:0/733664415 >> 172.21.15.31:0/733664415 conn(0x7f067c097550 msgr2=0x7f067c158c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.652+0000 7f0680bff700 1 -- 172.21.15.31:0/733664415 shutdown_connections 2023-10-06T05:23:40.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.652+0000 7f0680bff700 1 -- 172.21.15.31:0/733664415 wait complete. 2023-10-06T05:23:40.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:23:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.868+0000 7f0bb5bff700 1 Processor -- start 2023-10-06T05:23:40.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.868+0000 7f0bb5bff700 1 -- start start 2023-10-06T05:23:40.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.869+0000 7f0bb5bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0bb015ca40 0x7f0bb0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.869+0000 7f0bb5bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb00c8f40 0x7f0bb00adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.869+0000 7f0bb5bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb0068da0 con 0x7f0bb006a300 2023-10-06T05:23:40.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.869+0000 7f0bb5bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb00cabc0 con 0x7f0bb015ca40 2023-10-06T05:23:40.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.869+0000 7f0bb5bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0068bb0 con 0x7f0bb00c8f40 2023-10-06T05:23:40.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.869+0000 7f0baeffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0bb006a300 0x7f0bb00c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45812/0 (socket says 172.21.15.31:45812) 2023-10-06T05:23:40.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.870+0000 7f0baeffd700 1 -- 172.21.15.31:0/204778914 learned_addr learned my addr 172.21.15.31:0/204778914 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:40.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.870+0000 7f0baf7fe700 1 --2- 172.21.15.31:0/204778914 >> v2:172.21.15.62:3300/0 conn(0x7f0bb015ca40 0x7f0bb0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.870+0000 7f0baffff700 1 --2- 172.21.15.31:0/204778914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb00c8f40 0x7f0bb00adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.870+0000 7f0bae7fc700 1 -- 172.21.15.31:0/204778914 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837241562 0 0) 0x7f0bb0068da0 con 0x7f0bb006a300 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.870+0000 7f0bae7fc700 1 -- 172.21.15.31:0/204778914 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b98003570 con 0x7f0bb006a300 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.870+0000 7f0baffff700 1 -- 172.21.15.31:0/204778914 >> v2:172.21.15.62:3300/0 conn(0x7f0bb015ca40 msgr2=0x7f0bb0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.870+0000 7f0baffff700 1 --2- 172.21.15.31:0/204778914 >> v2:172.21.15.62:3300/0 conn(0x7f0bb015ca40 0x7f0bb0068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.870+0000 7f0baffff700 1 -- 172.21.15.31:0/204778914 >> v1:172.21.15.31:6789/0 conn(0x7f0bb006a300 legacy=0x7f0bb00c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.870+0000 7f0baffff700 1 -- 172.21.15.31:0/204778914 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb0069080 con 0x7f0bb00c8f40 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.870+0000 7f0baffff700 1 --2- 172.21.15.31:0/204778914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb00c8f40 0x7f0bb00adde0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009e60 tx=0x7f0ba8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac3a917fc19ebe2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bae7fc700 1 -- 172.21.15.31:0/204778914 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800e070 con 0x7f0bb00c8f40 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bae7fc700 1 -- 172.21.15.31:0/204778914 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba800bab0 con 0x7f0bb00c8f40 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bae7fc700 1 -- 172.21.15.31:0/204778914 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800be30 con 0x7f0bb00c8f40 2023-10-06T05:23:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bb5bff700 1 -- 172.21.15.31:0/204778914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb00c8f40 msgr2=0x7f0bb00adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bb5bff700 1 --2- 172.21.15.31:0/204778914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb00c8f40 0x7f0bb00adde0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009e60 tx=0x7f0ba8009780 comp rx=0 tx=0).stop 2023-10-06T05:23:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bb5bff700 1 -- 172.21.15.31:0/204778914 shutdown_connections 2023-10-06T05:23:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bb5bff700 1 --2- 172.21.15.31:0/204778914 >> v2:172.21.15.62:3300/0 conn(0x7f0bb015ca40 0x7f0bb0068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bb5bff700 1 --2- 172.21.15.31:0/204778914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb00c8f40 0x7f0bb00adde0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bb5bff700 1 -- 172.21.15.31:0/204778914 >> 172.21.15.31:0/204778914 conn(0x7f0bb00ce290 msgr2=0x7f0bb00ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bb5bff700 1 -- 172.21.15.31:0/204778914 shutdown_connections 2023-10-06T05:23:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.871+0000 7f0bb5bff700 1 -- 172.21.15.31:0/204778914 wait complete. 2023-10-06T05:23:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.873+0000 7f0bb5bff700 1 Processor -- start 2023-10-06T05:23:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.873+0000 7f0bb5bff700 1 -- start start 2023-10-06T05:23:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0bb5bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb006a300 0x7f0bb00c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0bb5bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb00c8f40 0x7f0bb014d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0bb5bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb015ca40 0x7f0bb00c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0bb5bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00b1040 con 0x7f0bb00c8f40 2023-10-06T05:23:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0bb5bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00c34d0 con 0x7f0bb006a300 2023-10-06T05:23:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0bb5bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00c37b0 con 0x7f0bb015ca40 2023-10-06T05:23:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0baeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb00c8f40 0x7f0bb014d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0baeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb00c8f40 0x7f0bb014d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44170/0 (socket says 172.21.15.31:44170) 2023-10-06T05:23:40.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0baeffd700 1 -- 172.21.15.31:0/1531680884 learned_addr learned my addr 172.21.15.31:0/1531680884 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:40.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0baf7fe700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb006a300 0x7f0bb00c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.874+0000 7f0baffff700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb015ca40 0x7f0bb00c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0baeffd700 1 -- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb015ca40 msgr2=0x7f0bb00c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0baeffd700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb015ca40 0x7f0bb00c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0baeffd700 1 -- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb006a300 msgr2=0x7f0bb00c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0baeffd700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb006a300 0x7f0bb00c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0baeffd700 1 -- 172.21.15.31:0/1531680884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba8009a20 con 0x7f0bb00c8f40 2023-10-06T05:23:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0baf7fe700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb006a300 0x7f0bb00c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0baeffd700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb00c8f40 0x7f0bb014d690 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40011e0 tx=0x7f0ba4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0b977fe700 1 -- 172.21.15.31:0/1531680884 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4005b30 con 0x7f0bb00c8f40 2023-10-06T05:23:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0b977fe700 1 -- 172.21.15.31:0/1531680884 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4005c90 con 0x7f0bb00c8f40 2023-10-06T05:23:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0bb5bff700 1 -- 172.21.15.31:0/1531680884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb00b12e0 con 0x7f0bb00c8f40 2023-10-06T05:23:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0b977fe700 1 -- 172.21.15.31:0/1531680884 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4007320 con 0x7f0bb00c8f40 2023-10-06T05:23:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.875+0000 7f0bb5bff700 1 -- 172.21.15.31:0/1531680884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb00c3110 con 0x7f0bb00c8f40 2023-10-06T05:23:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.876+0000 7f0bb5bff700 1 -- 172.21.15.31:0/1531680884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb004c380 con 0x7f0bb00c8f40 2023-10-06T05:23:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.877+0000 7f0b977fe700 1 -- 172.21.15.31:0/1531680884 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0ba4007690 con 0x7f0bb00c8f40 2023-10-06T05:23:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.878+0000 7f0b977fe700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b9c067320 0x7f0b9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.878+0000 7f0b977fe700 1 -- 172.21.15.31:0/1531680884 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f0ba40843b0 con 0x7f0bb00c8f40 2023-10-06T05:23:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.878+0000 7f0baf7fe700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b9c067320 0x7f0b9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:40.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.879+0000 7f0baf7fe700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b9c067320 0x7f0b9c0697d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009d80 tx=0x7f0ba0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:40.882+0000 7f0b977fe700 1 -- 172.21.15.31:0/1531680884 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ba4050e80 con 0x7f0bb00c8f40 2023-10-06T05:23:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.084+0000 7f0bb5bff700 1 -- 172.21.15.31:0/1531680884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0bb00ca1a0 con 0x7f0bb00c8f40 2023-10-06T05:23:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.085+0000 7f0b977fe700 1 -- 172.21.15.31:0/1531680884 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0ba40544a0 con 0x7f0bb00c8f40 2023-10-06T05:23:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 -- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b9c067320 msgr2=0x7f0b9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b9c067320 0x7f0b9c0697d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009d80 tx=0x7f0ba0008040 comp rx=0 tx=0).stop 2023-10-06T05:23:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 -- 172.21.15.31:0/1531680884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb00c8f40 msgr2=0x7f0bb014d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb00c8f40 0x7f0bb014d690 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40011e0 tx=0x7f0ba4007df0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 -- 172.21.15.31:0/1531680884 shutdown_connections 2023-10-06T05:23:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b9c067320 0x7f0b9c0697d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bb015ca40 0x7f0bb00c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bb006a300 0x7f0bb00c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 --2- 172.21.15.31:0/1531680884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb00c8f40 0x7f0bb014d690 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 -- 172.21.15.31:0/1531680884 >> 172.21.15.31:0/1531680884 conn(0x7f0bb00ce290 msgr2=0x7f0bb015a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 -- 172.21.15.31:0/1531680884 shutdown_connections 2023-10-06T05:23:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:41.087+0000 7f0bb5bff700 1 -- 172.21.15.31:0/1531680884 wait complete. 2023-10-06T05:23:41.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.109 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000851349/03e6c9e3-1945-4db6-9ea9-2459c3679f70) 2023-10-06T05:23:41.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.189 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000851349/03e6c9e3-1945-4db6-9ea9-2459c3679f70/.snap/fake) 2023-10-06T05:23:41.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.269 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T05:23:41.316 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T05:23:41.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.317 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvolume_0000000000851349/03e6c9e3-1945-4db6-9ea9-2459c3679f70) 2023-10-06T05:23:41.392 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:23:41.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvolume_0000000000851349/.meta 2023-10-06T05:23:41.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.467 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/group_0000000000059849/subvolume_0000000000851350/5001162f-e867-4ea7-8e74-627a30376d4d) 2023-10-06T05:23:41.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.583 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/group_0000000000059849/subvolume_0000000000851350/5001162f-e867-4ea7-8e74-627a30376d4d/.snap/fake) 2023-10-06T05:23:41.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.663 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T05:23:41.711 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T05:23:41.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.712 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/group_0000000000059849/subvolume_0000000000851350/5001162f-e867-4ea7-8e74-627a30376d4d) 2023-10-06T05:23:41.787 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:23:41.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/group_0000000000059849/subvolume_0000000000851350/.meta 2023-10-06T05:23:41.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.860 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000000851351/082c8b85-9546-4f55-8c73-e3e044e17983) 2023-10-06T05:23:41.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:41.975 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T05:23:42.022 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T05:23:42.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:42.023 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvolume_0000000000851351/082c8b85-9546-4f55-8c73-e3e044e17983) 2023-10-06T05:23:42.098 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:23:42.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvolume_0000000000851351/.meta 2023-10-06T05:23:42.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000851349 2023-10-06T05:23:42.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.406+0000 7f825b1ff700 1 Processor -- start 2023-10-06T05:23:42.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.406+0000 7f825b1ff700 1 -- start start 2023-10-06T05:23:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.407+0000 7f825b1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f825415ca40 0x7f8254068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.407+0000 7f825b1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82540c8f40 0x7f82540adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.407+0000 7f825b1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8254068da0 con 0x7f825406a300 2023-10-06T05:23:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.407+0000 7f825b1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82540cabc0 con 0x7f825415ca40 2023-10-06T05:23:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.407+0000 7f825b1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8254068bb0 con 0x7f82540c8f40 2023-10-06T05:23:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.408+0000 7f8258f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f825415ca40 0x7f8254068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.408+0000 7f8253fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f825406a300 0x7f82540c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45826/0 (socket says 172.21.15.31:45826) 2023-10-06T05:23:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.408+0000 7f8253fff700 1 -- 172.21.15.31:0/308118692 learned_addr learned my addr 172.21.15.31:0/308118692 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.408+0000 7f825979c700 1 --2- 172.21.15.31:0/308118692 >> v2:172.21.15.62:3300/0 conn(0x7f82540c8f40 0x7f82540adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.408+0000 7f8258f9b700 1 -- 172.21.15.31:0/308118692 >> v2:172.21.15.62:3300/0 conn(0x7f82540c8f40 msgr2=0x7f82540adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:42.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.408+0000 7f8258f9b700 1 --2- 172.21.15.31:0/308118692 >> v2:172.21.15.62:3300/0 conn(0x7f82540c8f40 0x7f82540adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.408+0000 7f8258f9b700 1 -- 172.21.15.31:0/308118692 >> v1:172.21.15.31:6789/0 conn(0x7f825406a300 legacy=0x7f82540c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.408+0000 7f8258f9b700 1 -- 172.21.15.31:0/308118692 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8254069080 con 0x7f825415ca40 2023-10-06T05:23:42.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.408+0000 7f825979c700 1 --2- 172.21.15.31:0/308118692 >> v2:172.21.15.62:3300/0 conn(0x7f82540c8f40 0x7f82540adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.408+0000 7f8258f9b700 1 --2- 172.21.15.31:0/308118692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 0x7f8254068670 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f82540c19b0 tx=0x7f82480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2c83a0783ae30c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f82537fe700 1 -- 172.21.15.31:0/308118692 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248009e80 con 0x7f825415ca40 2023-10-06T05:23:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f82537fe700 1 -- 172.21.15.31:0/308118692 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824800b820 con 0x7f825415ca40 2023-10-06T05:23:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f82537fe700 1 -- 172.21.15.31:0/308118692 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824800bba0 con 0x7f825415ca40 2023-10-06T05:23:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f825b1ff700 1 -- 172.21.15.31:0/308118692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 msgr2=0x7f8254068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f825b1ff700 1 --2- 172.21.15.31:0/308118692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 0x7f8254068670 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f82540c19b0 tx=0x7f82480091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f825b1ff700 1 -- 172.21.15.31:0/308118692 shutdown_connections 2023-10-06T05:23:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f825b1ff700 1 --2- 172.21.15.31:0/308118692 >> v2:172.21.15.62:3300/0 conn(0x7f82540c8f40 0x7f82540adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f825b1ff700 1 --2- 172.21.15.31:0/308118692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 0x7f8254068670 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f825b1ff700 1 -- 172.21.15.31:0/308118692 >> 172.21.15.31:0/308118692 conn(0x7f82540ce290 msgr2=0x7f82540ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f825b1ff700 1 -- 172.21.15.31:0/308118692 shutdown_connections 2023-10-06T05:23:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.409+0000 7f825b1ff700 1 -- 172.21.15.31:0/308118692 wait complete. 2023-10-06T05:23:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.411+0000 7f825b1ff700 1 Processor -- start 2023-10-06T05:23:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.411+0000 7f825b1ff700 1 -- start start 2023-10-06T05:23:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.412+0000 7f825b1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f825406a300 0x7f8254169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.412+0000 7f825b1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82540c8f40 0x7f82540c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.412+0000 7f825b1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 0x7f82540c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.412+0000 7f825b1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825415a140 con 0x7f825415ca40 2023-10-06T05:23:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.412+0000 7f825b1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82540c48d0 con 0x7f825406a300 2023-10-06T05:23:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.412+0000 7f825b1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82540c4bb0 con 0x7f82540c8f40 2023-10-06T05:23:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.412+0000 7f825979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 0x7f82540c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.412+0000 7f825979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 0x7f82540c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44180/0 (socket says 172.21.15.31:44180) 2023-10-06T05:23:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.412+0000 7f825979c700 1 -- 172.21.15.31:0/2215691093 learned_addr learned my addr 172.21.15.31:0/2215691093 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f8253fff700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82540c8f40 0x7f82540c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f8258f9b700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f825406a300 0x7f8254169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f825979c700 1 -- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82540c8f40 msgr2=0x7f82540c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f825979c700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82540c8f40 0x7f82540c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f825979c700 1 -- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f825406a300 msgr2=0x7f8254169940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f825979c700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f825406a300 0x7f8254169940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f825979c700 1 -- 172.21.15.31:0/2215691093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8248009420 con 0x7f825415ca40 2023-10-06T05:23:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f825979c700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 0x7f82540c7580 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f824c00d150 tx=0x7f824c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f8250ff9700 1 -- 172.21.15.31:0/2215691093 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c00ddf0 con 0x7f825415ca40 2023-10-06T05:23:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f8250ff9700 1 -- 172.21.15.31:0/2215691093 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c00f040 con 0x7f825415ca40 2023-10-06T05:23:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f825b1ff700 1 -- 172.21.15.31:0/2215691093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82540c4e90 con 0x7f825415ca40 2023-10-06T05:23:42.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.413+0000 7f8250ff9700 1 -- 172.21.15.31:0/2215691093 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c015690 con 0x7f825415ca40 2023-10-06T05:23:42.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.414+0000 7f825b1ff700 1 -- 172.21.15.31:0/2215691093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8254169e80 con 0x7f825415ca40 2023-10-06T05:23:42.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.415+0000 7f825b1ff700 1 -- 172.21.15.31:0/2215691093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825404c380 con 0x7f825415ca40 2023-10-06T05:23:42.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.415+0000 7f8250ff9700 1 -- 172.21.15.31:0/2215691093 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f824c01e3b0 con 0x7f825415ca40 2023-10-06T05:23:42.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.416+0000 7f8250ff9700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8238067320 0x7f82380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.416+0000 7f8250ff9700 1 -- 172.21.15.31:0/2215691093 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f824c085540 con 0x7f825415ca40 2023-10-06T05:23:42.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.417+0000 7f8258f9b700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8238067320 0x7f82380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.417+0000 7f8258f9b700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8238067320 0x7f82380697d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f82480093f0 tx=0x7f8248017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.421+0000 7f8250ff9700 1 -- 172.21.15.31:0/2215691093 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f824c051f00 con 0x7f825415ca40 2023-10-06T05:23:42.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.662+0000 7f825b1ff700 1 -- 172.21.15.31:0/2215691093 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000851349", "target": ["mon-mgr", ""]}) v1 -- 0x7f82540ca1a0 con 0x7f8238067320 2023-10-06T05:23:42.674 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:42.672+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:42.674 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:42.672+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:42.674 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:42.672+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:42.674 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:42.672+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:42.675 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:42.672+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.719+0000 7f8250ff9700 1 -- 172.21.15.31:0/2215691093 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f82540ca1a0 con 0x7f8238067320 2023-10-06T05:23:42.721 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000851349/03e6c9e3-1945-4db6-9ea9-2459c3679f70 2023-10-06T05:23:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 -- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8238067320 msgr2=0x7f82380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8238067320 0x7f82380697d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f82480093f0 tx=0x7f8248017000 comp rx=0 tx=0).stop 2023-10-06T05:23:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 -- 172.21.15.31:0/2215691093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 msgr2=0x7f82540c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 0x7f82540c7580 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f824c00d150 tx=0x7f824c009a50 comp rx=0 tx=0).stop 2023-10-06T05:23:42.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 -- 172.21.15.31:0/2215691093 shutdown_connections 2023-10-06T05:23:42.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8238067320 0x7f82380697d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f825415ca40 0x7f82540c7580 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f825406a300 0x7f8254169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 --2- 172.21.15.31:0/2215691093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82540c8f40 0x7f82540c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 -- 172.21.15.31:0/2215691093 >> 172.21.15.31:0/2215691093 conn(0x7f82540ce290 msgr2=0x7f82540b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:42.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 -- 172.21.15.31:0/2215691093 shutdown_connections 2023-10-06T05:23:42.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.721+0000 7f825b1ff700 1 -- 172.21.15.31:0/2215691093 wait complete. 2023-10-06T05:23:42.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000851350 group_0000000000059849 2023-10-06T05:23:42.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.938+0000 7fd64d5ff700 1 Processor -- start 2023-10-06T05:23:42.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.938+0000 7fd64d5ff700 1 -- start start 2023-10-06T05:23:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.939+0000 7fd64d5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd648153950 0x7fd648153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.939+0000 7fd64d5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd6480cd5d0 0x7fd64815d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.939+0000 7fd64d5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64815cfc0 con 0x7fd6480cda20 2023-10-06T05:23:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.939+0000 7fd64d5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64815bb20 con 0x7fd6480cd5d0 2023-10-06T05:23:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.939+0000 7fd64d5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64815af10 con 0x7fd648153950 2023-10-06T05:23:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.940+0000 7fd6467fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd6480cd5d0 0x7fd64815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.940+0000 7fd6467fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd6480cd5d0 0x7fd64815d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44186/0 (socket says 172.21.15.31:44186) 2023-10-06T05:23:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.940+0000 7fd6467fc700 1 -- 172.21.15.31:0/3126574460 learned_addr learned my addr 172.21.15.31:0/3126574460 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:42.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.940+0000 7fd646ffd700 1 --2- 172.21.15.31:0/3126574460 >> v2:172.21.15.62:3300/0 conn(0x7fd648153950 0x7fd648153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.940+0000 7fd6467fc700 1 -- 172.21.15.31:0/3126574460 >> v2:172.21.15.62:3300/0 conn(0x7fd648153950 msgr2=0x7fd648153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:42.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.940+0000 7fd6467fc700 1 --2- 172.21.15.31:0/3126574460 >> v2:172.21.15.62:3300/0 conn(0x7fd648153950 0x7fd648153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.940+0000 7fd6467fc700 1 -- 172.21.15.31:0/3126574460 >> v1:172.21.15.62:6789/0 conn(0x7fd6480cda20 legacy=0x7fd648159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.940+0000 7fd6467fc700 1 -- 172.21.15.31:0/3126574460 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64815b1f0 con 0x7fd6480cd5d0 2023-10-06T05:23:42.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.940+0000 7fd6467fc700 1 --2- 172.21.15.31:0/3126574460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6480cd5d0 0x7fd64815d210 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd6480d15b0 tx=0x7fd63c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3daed81850c990 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd645ffb700 1 -- 172.21.15.31:0/3126574460 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c009e70 con 0x7fd6480cd5d0 2023-10-06T05:23:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd645ffb700 1 -- 172.21.15.31:0/3126574460 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c00b860 con 0x7fd6480cd5d0 2023-10-06T05:23:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd645ffb700 1 -- 172.21.15.31:0/3126574460 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00bc00 con 0x7fd6480cd5d0 2023-10-06T05:23:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd64d5ff700 1 -- 172.21.15.31:0/3126574460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6480cd5d0 msgr2=0x7fd64815d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd64d5ff700 1 --2- 172.21.15.31:0/3126574460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6480cd5d0 0x7fd64815d210 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd6480d15b0 tx=0x7fd63c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd64d5ff700 1 -- 172.21.15.31:0/3126574460 shutdown_connections 2023-10-06T05:23:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd64d5ff700 1 --2- 172.21.15.31:0/3126574460 >> v2:172.21.15.62:3300/0 conn(0x7fd648153950 0x7fd648153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd64d5ff700 1 --2- 172.21.15.31:0/3126574460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6480cd5d0 0x7fd64815d210 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd64d5ff700 1 -- 172.21.15.31:0/3126574460 >> 172.21.15.31:0/3126574460 conn(0x7fd6480681b0 msgr2=0x7fd6480685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:42.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd64d5ff700 1 -- 172.21.15.31:0/3126574460 shutdown_connections 2023-10-06T05:23:42.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.941+0000 7fd64d5ff700 1 -- 172.21.15.31:0/3126574460 wait complete. 2023-10-06T05:23:42.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.943+0000 7fd64d5ff700 1 Processor -- start 2023-10-06T05:23:42.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.943+0000 7fd64d5ff700 1 -- start start 2023-10-06T05:23:42.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.944+0000 7fd64d5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6480cd5d0 0x7fd64814d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.944+0000 7fd64d5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6480cda20 0x7fd648169a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.944+0000 7fd64d5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd648153950 0x7fd64814f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.944+0000 7fd64d5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd648149a50 con 0x7fd648153950 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.944+0000 7fd64d5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64815df20 con 0x7fd6480cd5d0 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.944+0000 7fd64d5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64816a2a0 con 0x7fd6480cda20 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.944+0000 7fd6477fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd648153950 0x7fd64814f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.944+0000 7fd6477fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd648153950 0x7fd64814f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44190/0 (socket says 172.21.15.31:44190) 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.944+0000 7fd6477fe700 1 -- 172.21.15.31:0/1940044217 learned_addr learned my addr 172.21.15.31:0/1940044217 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd646ffd700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6480cd5d0 0x7fd64814d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd6467fc700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6480cda20 0x7fd648169a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd6477fe700 1 -- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6480cda20 msgr2=0x7fd648169a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd6477fe700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6480cda20 0x7fd648169a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd6477fe700 1 -- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6480cd5d0 msgr2=0x7fd64814d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:42.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd6477fe700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6480cd5d0 0x7fd64814d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:42.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd6477fe700 1 -- 172.21.15.31:0/1940044217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0094f0 con 0x7fd648153950 2023-10-06T05:23:42.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd6477fe700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd648153950 0x7fd64814f520 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd64000c8e0 tx=0x7fd640001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd62effd700 1 -- 172.21.15.31:0/1940044217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640012630 con 0x7fd648153950 2023-10-06T05:23:42.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd62effd700 1 -- 172.21.15.31:0/1940044217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd640012790 con 0x7fd648153950 2023-10-06T05:23:42.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd62effd700 1 -- 172.21.15.31:0/1940044217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640012b50 con 0x7fd648153950 2023-10-06T05:23:42.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.945+0000 7fd64d5ff700 1 -- 172.21.15.31:0/1940044217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64816a580 con 0x7fd648153950 2023-10-06T05:23:42.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.946+0000 7fd64d5ff700 1 -- 172.21.15.31:0/1940044217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64816ab40 con 0x7fd648153950 2023-10-06T05:23:42.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.946+0000 7fd64d5ff700 1 -- 172.21.15.31:0/1940044217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64815dce0 con 0x7fd648153950 2023-10-06T05:23:42.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.947+0000 7fd62effd700 1 -- 172.21.15.31:0/1940044217 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd64000a3b0 con 0x7fd648153950 2023-10-06T05:23:42.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.948+0000 7fd62effd700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd634067320 0x7fd6340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:42.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.949+0000 7fd62effd700 1 -- 172.21.15.31:0/1940044217 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd6400850c0 con 0x7fd648153950 2023-10-06T05:23:42.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.949+0000 7fd646ffd700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd634067320 0x7fd6340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:42.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.950+0000 7fd646ffd700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd634067320 0x7fd6340697d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd6480c6370 tx=0x7fd638009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:42.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:42.952+0000 7fd62effd700 1 -- 172.21.15.31:0/1940044217 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd640055620 con 0x7fd648153950 2023-10-06T05:23:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.192+0000 7fd64d5ff700 1 -- 172.21.15.31:0/1940044217 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000851350", "group_name": "group_0000000000059849", "target": ["mon-mgr", ""]}) v1 -- 0x7fd64815b4d0 con 0x7fd634067320 2023-10-06T05:23:43.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.238+0000 7fd62effd700 1 -- 172.21.15.31:0/1940044217 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7fd64815b4d0 con 0x7fd634067320 2023-10-06T05:23:43.244 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000059849/subvolume_0000000000851350/5001162f-e867-4ea7-8e74-627a30376d4d 2023-10-06T05:23:43.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.240+0000 7fd64d5ff700 1 -- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd634067320 msgr2=0x7fd6340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.240+0000 7fd64d5ff700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd634067320 0x7fd6340697d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd6480c6370 tx=0x7fd638009120 comp rx=0 tx=0).stop 2023-10-06T05:23:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.240+0000 7fd64d5ff700 1 -- 172.21.15.31:0/1940044217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd648153950 msgr2=0x7fd64814f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.240+0000 7fd64d5ff700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd648153950 0x7fd64814f520 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd64000c8e0 tx=0x7fd640001710 comp rx=0 tx=0).stop 2023-10-06T05:23:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.240+0000 7fd64d5ff700 1 -- 172.21.15.31:0/1940044217 shutdown_connections 2023-10-06T05:23:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.240+0000 7fd64d5ff700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd634067320 0x7fd6340697d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.240+0000 7fd64d5ff700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd648153950 0x7fd64814f520 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.240+0000 7fd64d5ff700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6480cd5d0 0x7fd64814d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.240+0000 7fd64d5ff700 1 --2- 172.21.15.31:0/1940044217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6480cda20 0x7fd648169a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.240+0000 7fd64d5ff700 1 -- 172.21.15.31:0/1940044217 >> 172.21.15.31:0/1940044217 conn(0x7fd6480681b0 msgr2=0x7fd648151c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.241+0000 7fd64d5ff700 1 -- 172.21.15.31:0/1940044217 shutdown_connections 2023-10-06T05:23:43.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.241+0000 7fd64d5ff700 1 -- 172.21.15.31:0/1940044217 wait complete. 2023-10-06T05:23:43.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvolume_0000000000851351 2023-10-06T05:23:43.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.456+0000 7f33c13ff700 1 Processor -- start 2023-10-06T05:23:43.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.456+0000 7f33c13ff700 1 -- start start 2023-10-06T05:23:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.457+0000 7f33c13ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f33bc159560 0x7f33bc0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.457+0000 7f33c13ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f33bc0b2b80 0x7f33bc0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.457+0000 7f33c13ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33bc15d2d0 con 0x7f33bc159930 2023-10-06T05:23:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.457+0000 7f33c13ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33bc15d5b0 con 0x7f33bc0b2b80 2023-10-06T05:23:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.457+0000 7f33c13ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33bc15d890 con 0x7f33bc159560 2023-10-06T05:23:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.458+0000 7f33ba7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f33bc159930 0x7f33bc0b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45832/0 (socket says 172.21.15.31:45832) 2023-10-06T05:23:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.458+0000 7f33ba7fc700 1 -- 172.21.15.31:0/2181873875 learned_addr learned my addr 172.21.15.31:0/2181873875 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.458+0000 7f33bb7fe700 1 --2- 172.21.15.31:0/2181873875 >> v2:172.21.15.31:3300/0 conn(0x7f33bc0b2b80 0x7f33bc0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.458+0000 7f33baffd700 1 --2- 172.21.15.31:0/2181873875 >> v2:172.21.15.62:3300/0 conn(0x7f33bc159560 0x7f33bc0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.458+0000 7f33b9ffb700 1 -- 172.21.15.31:0/2181873875 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767250982 0 0) 0x7f33bc15d2d0 con 0x7f33bc159930 2023-10-06T05:23:43.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.458+0000 7f33b9ffb700 1 -- 172.21.15.31:0/2181873875 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a4003570 con 0x7f33bc159930 2023-10-06T05:23:43.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.458+0000 7f33bb7fe700 1 -- 172.21.15.31:0/2181873875 >> v2:172.21.15.62:3300/0 conn(0x7f33bc159560 msgr2=0x7f33bc0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:43.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.458+0000 7f33bb7fe700 1 --2- 172.21.15.31:0/2181873875 >> v2:172.21.15.62:3300/0 conn(0x7f33bc159560 0x7f33bc0cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.458+0000 7f33bb7fe700 1 -- 172.21.15.31:0/2181873875 >> v1:172.21.15.31:6789/0 conn(0x7f33bc159930 legacy=0x7f33bc0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.458+0000 7f33bb7fe700 1 -- 172.21.15.31:0/2181873875 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc15db70 con 0x7f33bc0b2b80 2023-10-06T05:23:43.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.459+0000 7f33bb7fe700 1 --2- 172.21.15.31:0/2181873875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc0b2b80 0x7f33bc0cd3b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f33bc063d80 tx=0x7f33b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe144177cf824d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.459+0000 7f33b9ffb700 1 -- 172.21.15.31:0/2181873875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400e070 con 0x7f33bc0b2b80 2023-10-06T05:23:43.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.459+0000 7f33b9ffb700 1 -- 172.21.15.31:0/2181873875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b400bab0 con 0x7f33bc0b2b80 2023-10-06T05:23:43.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.459+0000 7f33b9ffb700 1 -- 172.21.15.31:0/2181873875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400be70 con 0x7f33bc0b2b80 2023-10-06T05:23:43.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.459+0000 7f33c13ff700 1 -- 172.21.15.31:0/2181873875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc0b2b80 msgr2=0x7f33bc0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.459+0000 7f33c13ff700 1 --2- 172.21.15.31:0/2181873875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc0b2b80 0x7f33bc0cd3b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f33bc063d80 tx=0x7f33b4009780 comp rx=0 tx=0).stop 2023-10-06T05:23:43.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.460+0000 7f33c13ff700 1 -- 172.21.15.31:0/2181873875 shutdown_connections 2023-10-06T05:23:43.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.460+0000 7f33c13ff700 1 --2- 172.21.15.31:0/2181873875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc0b2b80 0x7f33bc0cd3b0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.460+0000 7f33c13ff700 1 --2- 172.21.15.31:0/2181873875 >> v2:172.21.15.62:3300/0 conn(0x7f33bc159560 0x7f33bc0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.460+0000 7f33c13ff700 1 -- 172.21.15.31:0/2181873875 >> 172.21.15.31:0/2181873875 conn(0x7f33bc06ec90 msgr2=0x7f33bc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:43.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.460+0000 7f33c13ff700 1 -- 172.21.15.31:0/2181873875 shutdown_connections 2023-10-06T05:23:43.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.460+0000 7f33c13ff700 1 -- 172.21.15.31:0/2181873875 wait complete. 2023-10-06T05:23:43.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.461+0000 7f33c13ff700 1 Processor -- start 2023-10-06T05:23:43.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.461+0000 7f33c13ff700 1 -- start start 2023-10-06T05:23:43.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.462+0000 7f33c13ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33bc0b2b80 0x7f33bc0c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.462+0000 7f33c13ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33bc159560 0x7f33bc0c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.462+0000 7f33c13ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc159930 0x7f33bc0c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.462+0000 7f33c13ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc0c4870 con 0x7f33bc159930 2023-10-06T05:23:43.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.462+0000 7f33c13ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc0c5a90 con 0x7f33bc0b2b80 2023-10-06T05:23:43.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.462+0000 7f33c13ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc0c3ef0 con 0x7f33bc159560 2023-10-06T05:23:43.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.462+0000 7f33bb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc159930 0x7f33bc0c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.462+0000 7f33bb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc159930 0x7f33bc0c5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44210/0 (socket says 172.21.15.31:44210) 2023-10-06T05:23:43.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.462+0000 7f33bb7fe700 1 -- 172.21.15.31:0/1221770465 learned_addr learned my addr 172.21.15.31:0/1221770465 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:43.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33baffd700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33bc0b2b80 0x7f33bc0c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33ba7fc700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33bc159560 0x7f33bc0c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33bb7fe700 1 -- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33bc159560 msgr2=0x7f33bc0c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33bb7fe700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33bc159560 0x7f33bc0c4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33bb7fe700 1 -- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33bc0b2b80 msgr2=0x7f33bc0c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33bb7fe700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33bc0b2b80 0x7f33bc0c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33bb7fe700 1 -- 172.21.15.31:0/1221770465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4009a20 con 0x7f33bc159930 2023-10-06T05:23:43.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33baffd700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33bc0b2b80 0x7f33bc0c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:43.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33bb7fe700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc159930 0x7f33bc0c5550 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0893d0 tx=0x7f33b4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33a2ffd700 1 -- 172.21.15.31:0/1221770465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4012e60 con 0x7f33bc159930 2023-10-06T05:23:43.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33a2ffd700 1 -- 172.21.15.31:0/1221770465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b4027470 con 0x7f33bc159930 2023-10-06T05:23:43.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33a2ffd700 1 -- 172.21.15.31:0/1221770465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4027810 con 0x7f33bc159930 2023-10-06T05:23:43.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.463+0000 7f33c13ff700 1 -- 172.21.15.31:0/1221770465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33bc0c6d10 con 0x7f33bc159930 2023-10-06T05:23:43.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.464+0000 7f33c13ff700 1 -- 172.21.15.31:0/1221770465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33bc0c4490 con 0x7f33bc159930 2023-10-06T05:23:43.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.465+0000 7f33c13ff700 1 -- 172.21.15.31:0/1221770465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33bc04c380 con 0x7f33bc159930 2023-10-06T05:23:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.466+0000 7f33a2ffd700 1 -- 172.21.15.31:0/1221770465 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f33b4024070 con 0x7f33bc159930 2023-10-06T05:23:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.467+0000 7f33a2ffd700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f33a8067320 0x7f33a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.467+0000 7f33a2ffd700 1 -- 172.21.15.31:0/1221770465 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f33b4087550 con 0x7f33bc159930 2023-10-06T05:23:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.467+0000 7f33baffd700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f33a8067320 0x7f33a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.468+0000 7f33baffd700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f33a8067320 0x7f33a80697d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009d80 tx=0x7f33ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.471+0000 7f33a2ffd700 1 -- 172.21.15.31:0/1221770465 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33b4054020 con 0x7f33bc159930 2023-10-06T05:23:43.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.720+0000 7f33c13ff700 1 -- 172.21.15.31:0/1221770465 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvolume_0000000000851351", "target": ["mon-mgr", ""]}) v1 -- 0x7f33bc15de50 con 0x7f33a8067320 2023-10-06T05:23:43.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.759+0000 7f33a2ffd700 1 -- 172.21.15.31:0/1221770465 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f33bc15de50 con 0x7f33a8067320 2023-10-06T05:23:43.761 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:23:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-10-06T05:23:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "pending", 2023-10-06T05:23:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "source": { 2023-10-06T05:23:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "volume": "fake", 2023-10-06T05:23:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "subvolume": "fake", 2023-10-06T05:23:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot": "fake" 2023-10-06T05:23:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:23:43.762 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:23:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.760+0000 7f33c13ff700 1 -- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f33a8067320 msgr2=0x7f33a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.760+0000 7f33c13ff700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f33a8067320 0x7f33a80697d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009d80 tx=0x7f33ac008040 comp rx=0 tx=0).stop 2023-10-06T05:23:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.760+0000 7f33c13ff700 1 -- 172.21.15.31:0/1221770465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc159930 msgr2=0x7f33bc0c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.760+0000 7f33c13ff700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc159930 0x7f33bc0c5550 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0893d0 tx=0x7f33b4019000 comp rx=0 tx=0).stop 2023-10-06T05:23:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.760+0000 7f33c13ff700 1 -- 172.21.15.31:0/1221770465 shutdown_connections 2023-10-06T05:23:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.760+0000 7f33c13ff700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f33a8067320 0x7f33a80697d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.760+0000 7f33c13ff700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33bc159930 0x7f33bc0c5550 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.760+0000 7f33c13ff700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33bc0b2b80 0x7f33bc0c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.760+0000 7f33c13ff700 1 --2- 172.21.15.31:0/1221770465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33bc159560 0x7f33bc0c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.760+0000 7f33c13ff700 1 -- 172.21.15.31:0/1221770465 >> 172.21.15.31:0/1221770465 conn(0x7f33bc06ec90 msgr2=0x7f33bc157480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.761+0000 7f33c13ff700 1 -- 172.21.15.31:0/1221770465 shutdown_connections 2023-10-06T05:23:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.761+0000 7f33c13ff700 1 -- 172.21.15.31:0/1221770465 wait complete. 2023-10-06T05:23:43.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000851349 fake 2023-10-06T05:23:43.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.978+0000 7f9d1d7ff700 1 Processor -- start 2023-10-06T05:23:43.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.979+0000 7f9d1d7ff700 1 -- start start 2023-10-06T05:23:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.980+0000 7f9d1d7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d180cb3e0 0x7f9d181645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.980+0000 7f9d1d7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d180cc5b0 con 0x7f9d18142c20 2023-10-06T05:23:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.980+0000 7f9d1d7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1815d210 con 0x7f9d1815a990 2023-10-06T05:23:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.980+0000 7f9d1d7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d180cc810 con 0x7f9d180cb3e0 2023-10-06T05:23:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.980+0000 7f9d177fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d180cb3e0 0x7f9d181645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.980+0000 7f9d16ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9d18142c20 0x7f9d180cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45842/0 (socket says 172.21.15.31:45842) 2023-10-06T05:23:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.980+0000 7f9d16ffd700 1 -- 172.21.15.31:0/2938218794 learned_addr learned my addr 172.21.15.31:0/2938218794 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.981+0000 7f9d177fe700 1 -- 172.21.15.31:0/2938218794 >> v1:172.21.15.62:6789/0 conn(0x7f9d1815a990 legacy=0x7f9d180cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.981+0000 7f9d177fe700 1 -- 172.21.15.31:0/2938218794 >> v1:172.21.15.31:6789/0 conn(0x7f9d18142c20 legacy=0x7f9d180cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.981+0000 7f9d177fe700 1 -- 172.21.15.31:0/2938218794 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1815d4b0 con 0x7f9d180cb3e0 2023-10-06T05:23:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.981+0000 7f9d15ffb700 1 -- 172.21.15.31:0/2938218794 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220381081 0 0) 0x7f9d180cc5b0 con 0x7f9d18142c20 2023-10-06T05:23:43.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.981+0000 7f9d177fe700 1 --2- 172.21.15.31:0/2938218794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d180cb3e0 0x7f9d181645b0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009e60 tx=0x7f9d10009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfcfe36413ae3dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.981+0000 7f9d15ffb700 1 -- 172.21.15.31:0/2938218794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000e070 con 0x7f9d180cb3e0 2023-10-06T05:23:43.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.981+0000 7f9d15ffb700 1 -- 172.21.15.31:0/2938218794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1000bbf0 con 0x7f9d180cb3e0 2023-10-06T05:23:43.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.981+0000 7f9d15ffb700 1 -- 172.21.15.31:0/2938218794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10012430 con 0x7f9d180cb3e0 2023-10-06T05:23:43.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.982+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2938218794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d180cb3e0 msgr2=0x7f9d181645b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.982+0000 7f9d1d7ff700 1 --2- 172.21.15.31:0/2938218794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d180cb3e0 0x7f9d181645b0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009e60 tx=0x7f9d10009780 comp rx=0 tx=0).stop 2023-10-06T05:23:43.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.982+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2938218794 shutdown_connections 2023-10-06T05:23:43.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.982+0000 7f9d1d7ff700 1 --2- 172.21.15.31:0/2938218794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d180cb3e0 0x7f9d181645b0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.982+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2938218794 >> 172.21.15.31:0/2938218794 conn(0x7f9d180ce2d0 msgr2=0x7f9d180ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:43.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.982+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2938218794 shutdown_connections 2023-10-06T05:23:43.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.982+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2938218794 wait complete. 2023-10-06T05:23:43.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.983+0000 7f9d1d7ff700 1 Processor -- start 2023-10-06T05:23:43.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.984+0000 7f9d1d7ff700 1 -- start start 2023-10-06T05:23:43.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.984+0000 7f9d1d7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d180cb3e0 0x7f9d1813dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.984+0000 7f9d1d7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d18142c20 0x7f9d180ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d1d7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1815a990 0x7f9d18141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d1d7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180b09d0 con 0x7f9d18142c20 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d1d7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180af930 con 0x7f9d1815a990 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d1d7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180afc10 con 0x7f9d180cb3e0 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d167fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d18142c20 0x7f9d180ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d167fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d18142c20 0x7f9d180ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44230/0 (socket says 172.21.15.31:44230) 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d167fc700 1 -- 172.21.15.31:0/2021635634 learned_addr learned my addr 172.21.15.31:0/2021635634 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d16ffd700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d180cb3e0 0x7f9d1813dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d177fe700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1815a990 0x7f9d18141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d167fc700 1 -- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d180cb3e0 msgr2=0x7f9d1813dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d167fc700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d180cb3e0 0x7f9d1813dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d167fc700 1 -- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1815a990 msgr2=0x7f9d18141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d167fc700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1815a990 0x7f9d18141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.985+0000 7f9d167fc700 1 -- 172.21.15.31:0/2021635634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10009a20 con 0x7f9d18142c20 2023-10-06T05:23:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.986+0000 7f9d167fc700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d18142c20 0x7f9d180ae4b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00b5e0 tx=0x7f9d0c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.986+0000 7f9cfeffd700 1 -- 172.21.15.31:0/2021635634 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00f510 con 0x7f9d18142c20 2023-10-06T05:23:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.986+0000 7f9cfeffd700 1 -- 172.21.15.31:0/2021635634 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c00f670 con 0x7f9d18142c20 2023-10-06T05:23:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.986+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2021635634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d180afef0 con 0x7f9d18142c20 2023-10-06T05:23:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.986+0000 7f9cfeffd700 1 -- 172.21.15.31:0/2021635634 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00fa30 con 0x7f9d18142c20 2023-10-06T05:23:43.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.986+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2021635634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d180b04b0 con 0x7f9d18142c20 2023-10-06T05:23:43.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.987+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2021635634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d180b06d0 con 0x7f9d18142c20 2023-10-06T05:23:43.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.988+0000 7f9cfeffd700 1 -- 172.21.15.31:0/2021635634 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9d0c01a400 con 0x7f9d18142c20 2023-10-06T05:23:43.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.989+0000 7f9cfeffd700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9d04067320 0x7f9d040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:43.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.989+0000 7f9cfeffd700 1 -- 172.21.15.31:0/2021635634 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9d0c084880 con 0x7f9d18142c20 2023-10-06T05:23:43.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.989+0000 7f9d16ffd700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9d04067320 0x7f9d040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:43.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.991+0000 7f9d16ffd700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9d04067320 0x7f9d040697d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9d080011e0 tx=0x7f9d08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:43.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:43.993+0000 7f9cfeffd700 1 -- 172.21.15.31:0/2021635634 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d0c051490 con 0x7f9d18142c20 2023-10-06T05:23:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.239+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2021635634 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000851349", "snap_name": "fake", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d1804c380 con 0x7f9d04067320 2023-10-06T05:23:44.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.292+0000 7f9cfeffd700 1 -- 172.21.15.31:0/2021635634 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9d1804c380 con 0x7f9d04067320 2023-10-06T05:23:44.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9d04067320 msgr2=0x7f9d040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9d04067320 0x7f9d040697d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9d080011e0 tx=0x7f9d08009040 comp rx=0 tx=0).stop 2023-10-06T05:23:44.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2021635634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d18142c20 msgr2=0x7f9d180ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d18142c20 0x7f9d180ae4b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00b5e0 tx=0x7f9d0c009120 comp rx=0 tx=0).stop 2023-10-06T05:23:44.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2021635634 shutdown_connections 2023-10-06T05:23:44.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9d04067320 0x7f9d040697d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1815a990 0x7f9d18141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d180cb3e0 0x7f9d1813dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 --2- 172.21.15.31:0/2021635634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d18142c20 0x7f9d180ae4b0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2021635634 >> 172.21.15.31:0/2021635634 conn(0x7f9d180ce2d0 msgr2=0x7f9d1806d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2021635634 shutdown_connections 2023-10-06T05:23:44.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.295+0000 7f9d1d7ff700 1 -- 172.21.15.31:0/2021635634 wait complete. 2023-10-06T05:23:44.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvolume_0000000000851350 fake group_0000000000059849 2023-10-06T05:23:44.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.514+0000 7f2233bff700 1 Processor -- start 2023-10-06T05:23:44.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.514+0000 7f2233bff700 1 -- start start 2023-10-06T05:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.515+0000 7f2233bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f222c15ca40 0x7f222c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.515+0000 7f2233bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222c068da0 con 0x7f222c0c8f40 2023-10-06T05:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.515+0000 7f2233bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222c069080 con 0x7f222c06a300 2023-10-06T05:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.515+0000 7f2233bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c0cabc0 con 0x7f222c15ca40 2023-10-06T05:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.515+0000 7f223219c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f222c0c8f40 0x7f222c0adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45854/0 (socket says 172.21.15.31:45854) 2023-10-06T05:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.515+0000 7f223199b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f222c15ca40 0x7f222c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.515+0000 7f223219c700 1 -- 172.21.15.31:0/1675263007 learned_addr learned my addr 172.21.15.31:0/1675263007 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:44.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.516+0000 7f2230999700 1 -- 172.21.15.31:0/1675263007 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939238435 0 0) 0x7f222c068da0 con 0x7f222c0c8f40 2023-10-06T05:23:44.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.516+0000 7f2230999700 1 -- 172.21.15.31:0/1675263007 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2218003570 con 0x7f222c0c8f40 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.516+0000 7f223199b700 1 -- 172.21.15.31:0/1675263007 >> v1:172.21.15.62:6789/0 conn(0x7f222c06a300 legacy=0x7f222c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.516+0000 7f2230999700 1 -- 172.21.15.31:0/1675263007 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075336004 0 0) 0x7f222c069080 con 0x7f222c06a300 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.516+0000 7f223199b700 1 -- 172.21.15.31:0/1675263007 >> v1:172.21.15.31:6789/0 conn(0x7f222c0c8f40 legacy=0x7f222c0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.516+0000 7f223199b700 1 -- 172.21.15.31:0/1675263007 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c069360 con 0x7f222c15ca40 2023-10-06T05:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.516+0000 7f223199b700 1 --2- 172.21.15.31:0/1675263007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c15ca40 0x7f222c068670 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f22280098e0 tx=0x7f22280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9760525c0bff4969 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.516+0000 7f2230999700 1 -- 172.21.15.31:0/1675263007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222800b640 con 0x7f222c15ca40 2023-10-06T05:23:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.516+0000 7f2230999700 1 -- 172.21.15.31:0/1675263007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222800b7a0 con 0x7f222c15ca40 2023-10-06T05:23:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.516+0000 7f2230999700 1 -- 172.21.15.31:0/1675263007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222800bb40 con 0x7f222c15ca40 2023-10-06T05:23:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.517+0000 7f2233bff700 1 -- 172.21.15.31:0/1675263007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c15ca40 msgr2=0x7f222c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.517+0000 7f2233bff700 1 --2- 172.21.15.31:0/1675263007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c15ca40 0x7f222c068670 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f22280098e0 tx=0x7f22280091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.517+0000 7f2233bff700 1 -- 172.21.15.31:0/1675263007 shutdown_connections 2023-10-06T05:23:44.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.517+0000 7f2233bff700 1 --2- 172.21.15.31:0/1675263007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c15ca40 0x7f222c068670 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.517+0000 7f2233bff700 1 -- 172.21.15.31:0/1675263007 >> 172.21.15.31:0/1675263007 conn(0x7f222c0ce290 msgr2=0x7f222c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.517+0000 7f2233bff700 1 -- 172.21.15.31:0/1675263007 shutdown_connections 2023-10-06T05:23:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.517+0000 7f2233bff700 1 -- 172.21.15.31:0/1675263007 wait complete. 2023-10-06T05:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.519+0000 7f2233bff700 1 Processor -- start 2023-10-06T05:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.519+0000 7f2233bff700 1 -- start start 2023-10-06T05:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f2233bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f222c06a300 0x7f222c0c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f2233bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c0c8f40 0x7f222c0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f2233bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f222c15ca40 0x7f222c16c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f2233bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0c53b0 con 0x7f222c0c8f40 2023-10-06T05:23:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f2233bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0afa70 con 0x7f222c06a300 2023-10-06T05:23:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f2233bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0aec00 con 0x7f222c15ca40 2023-10-06T05:23:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c0c8f40 0x7f222c0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223119a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c0c8f40 0x7f222c0ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44250/0 (socket says 172.21.15.31:44250) 2023-10-06T05:23:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223119a700 1 -- 172.21.15.31:0/3552892641 learned_addr learned my addr 172.21.15.31:0/3552892641 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223199b700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f222c06a300 0x7f222c0c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223219c700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f222c15ca40 0x7f222c16c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223119a700 1 -- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f222c15ca40 msgr2=0x7f222c16c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223119a700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f222c15ca40 0x7f222c16c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223119a700 1 -- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f222c06a300 msgr2=0x7f222c0c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223119a700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f222c06a300 0x7f222c0c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223119a700 1 -- 172.21.15.31:0/3552892641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2228009420 con 0x7f222c0c8f40 2023-10-06T05:23:44.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.520+0000 7f223199b700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f222c06a300 0x7f222c0c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:44.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.521+0000 7f223119a700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c0c8f40 0x7f222c0ae6c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c7e0 tx=0x7f221c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.521+0000 7f2221ffb700 1 -- 172.21.15.31:0/3552892641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c012700 con 0x7f222c0c8f40 2023-10-06T05:23:44.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.521+0000 7f2221ffb700 1 -- 172.21.15.31:0/3552892641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c012880 con 0x7f222c0c8f40 2023-10-06T05:23:44.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.521+0000 7f2221ffb700 1 -- 172.21.15.31:0/3552892641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c012c40 con 0x7f222c0c8f40 2023-10-06T05:23:44.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.521+0000 7f2233bff700 1 -- 172.21.15.31:0/3552892641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c0aeec0 con 0x7f222c0c8f40 2023-10-06T05:23:44.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.521+0000 7f2233bff700 1 -- 172.21.15.31:0/3552892641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c0af700 con 0x7f222c0c8f40 2023-10-06T05:23:44.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.522+0000 7f2233bff700 1 -- 172.21.15.31:0/3552892641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c04c380 con 0x7f222c0c8f40 2023-10-06T05:23:44.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.523+0000 7f2221ffb700 1 -- 172.21.15.31:0/3552892641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f221c01b400 con 0x7f222c0c8f40 2023-10-06T05:23:44.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.524+0000 7f2221ffb700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2210067320 0x7f22100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:44.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.524+0000 7f2221ffb700 1 -- 172.21.15.31:0/3552892641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f221c084ab0 con 0x7f222c0c8f40 2023-10-06T05:23:44.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.524+0000 7f223199b700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2210067320 0x7f22100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:44.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.525+0000 7f223199b700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2210067320 0x7f22100697d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f22280093f0 tx=0x7f2228017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:44.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.528+0000 7f2221ffb700 1 -- 172.21.15.31:0/3552892641 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f221c051580 con 0x7f222c0c8f40 2023-10-06T05:23:44.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.769+0000 7f2233bff700 1 -- 172.21.15.31:0/3552892641 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000851350", "snap_name": "fake", "group_name": "group_0000000000059849", "target": ["mon-mgr", ""]}) v1 -- 0x7f222c0ca1a0 con 0x7f2210067320 2023-10-06T05:23:44.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.818+0000 7f2221ffb700 1 -- 172.21.15.31:0/3552892641 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f222c0ca1a0 con 0x7f2210067320 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.820+0000 7f2233bff700 1 -- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2210067320 msgr2=0x7f22100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.820+0000 7f2233bff700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2210067320 0x7f22100697d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f22280093f0 tx=0x7f2228017000 comp rx=0 tx=0).stop 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.821+0000 7f2233bff700 1 -- 172.21.15.31:0/3552892641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c0c8f40 msgr2=0x7f222c0ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.821+0000 7f2233bff700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c0c8f40 0x7f222c0ae6c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c7e0 tx=0x7f221c007df0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.821+0000 7f2233bff700 1 -- 172.21.15.31:0/3552892641 shutdown_connections 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.821+0000 7f2233bff700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2210067320 0x7f22100697d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.821+0000 7f2233bff700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f222c15ca40 0x7f222c16c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.821+0000 7f2233bff700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f222c06a300 0x7f222c0c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.821+0000 7f2233bff700 1 --2- 172.21.15.31:0/3552892641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f222c0c8f40 0x7f222c0ae6c0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.821+0000 7f2233bff700 1 -- 172.21.15.31:0/3552892641 >> 172.21.15.31:0/3552892641 conn(0x7f222c0ce290 msgr2=0x7f222c15a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.821+0000 7f2233bff700 1 -- 172.21.15.31:0/3552892641 shutdown_connections 2023-10-06T05:23:44.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:44.821+0000 7f2233bff700 1 -- 172.21.15.31:0/3552892641 wait complete. 2023-10-06T05:23:44.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:44.841 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvolume_0000000000851349/.meta) 2023-10-06T05:23:44.913 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:23:44.913 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T05:23:44.913 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T05:23:44.913 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000851349/03e6c9e3-1945-4db6-9ea9-2459c3679f70 2023-10-06T05:23:44.913 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:23:44.913 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:44.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:44.916 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/group_0000000000059849/subvolume_0000000000851350/.meta) 2023-10-06T05:23:44.988 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:23:44.988 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T05:23:44.988 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T05:23:44.988 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/group_0000000000059849/subvolume_0000000000851350/5001162f-e867-4ea7-8e74-627a30376d4d 2023-10-06T05:23:44.989 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:23:44.989 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:44.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000851349 2023-10-06T05:23:45.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.189+0000 7fe9e85ff700 1 Processor -- start 2023-10-06T05:23:45.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.189+0000 7fe9e85ff700 1 -- start start 2023-10-06T05:23:45.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.191+0000 7fe9e85ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe9e00c7190 0x7fe9e00c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.191+0000 7fe9e85ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9e00aef00 0x7fe9e00af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.191+0000 7fe9e85ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e0068250 con 0x7fe9e00afcf0 2023-10-06T05:23:45.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.191+0000 7fe9e85ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e006ac80 con 0x7fe9e00aef00 2023-10-06T05:23:45.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.191+0000 7fe9e85ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e0067f20 con 0x7fe9e00c7190 2023-10-06T05:23:45.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.191+0000 7fe9e6b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe9e00afcf0 0x7fe9e00686e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45862/0 (socket says 172.21.15.31:45862) 2023-10-06T05:23:45.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.191+0000 7fe9e6b9c700 1 -- 172.21.15.31:0/882117218 learned_addr learned my addr 172.21.15.31:0/882117218 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:45.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.191+0000 7fe9e5b9a700 1 --2- 172.21.15.31:0/882117218 >> v2:172.21.15.31:3300/0 conn(0x7fe9e00aef00 0x7fe9e00af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.191+0000 7fe9e639b700 1 --2- 172.21.15.31:0/882117218 >> v2:172.21.15.62:3300/0 conn(0x7fe9e00c7190 0x7fe9e00c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.191+0000 7fe9e5399700 1 -- 172.21.15.31:0/882117218 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942486824 0 0) 0x7fe9e0068250 con 0x7fe9e00afcf0 2023-10-06T05:23:45.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.192+0000 7fe9e5399700 1 -- 172.21.15.31:0/882117218 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9cc003570 con 0x7fe9e00afcf0 2023-10-06T05:23:45.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.192+0000 7fe9e5b9a700 1 -- 172.21.15.31:0/882117218 >> v2:172.21.15.62:3300/0 conn(0x7fe9e00c7190 msgr2=0x7fe9e00c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:45.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.192+0000 7fe9e5b9a700 1 --2- 172.21.15.31:0/882117218 >> v2:172.21.15.62:3300/0 conn(0x7fe9e00c7190 0x7fe9e00c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.192+0000 7fe9e5b9a700 1 -- 172.21.15.31:0/882117218 >> v1:172.21.15.31:6789/0 conn(0x7fe9e00afcf0 legacy=0x7fe9e00686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.192+0000 7fe9e5b9a700 1 -- 172.21.15.31:0/882117218 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e006aed0 con 0x7fe9e00aef00 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.192+0000 7fe9e5b9a700 1 --2- 172.21.15.31:0/882117218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 0x7fe9e00af7b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe9e006f4e0 tx=0x7fe9d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178cc1bd55d45316 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.192+0000 7fe9e5399700 1 -- 172.21.15.31:0/882117218 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0011630 con 0x7fe9e00aef00 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.192+0000 7fe9e5399700 1 -- 172.21.15.31:0/882117218 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d00117b0 con 0x7fe9e00aef00 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.192+0000 7fe9e5399700 1 -- 172.21.15.31:0/882117218 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0011b30 con 0x7fe9e00aef00 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.193+0000 7fe9e85ff700 1 -- 172.21.15.31:0/882117218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 msgr2=0x7fe9e00af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.193+0000 7fe9e85ff700 1 --2- 172.21.15.31:0/882117218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 0x7fe9e00af7b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe9e006f4e0 tx=0x7fe9d00091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.193+0000 7fe9e85ff700 1 -- 172.21.15.31:0/882117218 shutdown_connections 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.193+0000 7fe9e85ff700 1 --2- 172.21.15.31:0/882117218 >> v2:172.21.15.62:3300/0 conn(0x7fe9e00c7190 0x7fe9e00c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.193+0000 7fe9e85ff700 1 --2- 172.21.15.31:0/882117218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 0x7fe9e00af7b0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.193+0000 7fe9e85ff700 1 -- 172.21.15.31:0/882117218 >> 172.21.15.31:0/882117218 conn(0x7fe9e00cd3d0 msgr2=0x7fe9e00c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:45.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.193+0000 7fe9e85ff700 1 -- 172.21.15.31:0/882117218 shutdown_connections 2023-10-06T05:23:45.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.193+0000 7fe9e85ff700 1 -- 172.21.15.31:0/882117218 wait complete. 2023-10-06T05:23:45.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.195+0000 7fe9e85ff700 1 Processor -- start 2023-10-06T05:23:45.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.195+0000 7fe9e85ff700 1 -- start start 2023-10-06T05:23:45.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.196+0000 7fe9e85ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 0x7fe9e0144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.196+0000 7fe9e85ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e00afcf0 0x7fe9e00c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e85ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9e00c7190 0x7fe9e00c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e85ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e0150fb0 con 0x7fe9e00aef00 2023-10-06T05:23:45.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e85ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e00c6920 con 0x7fe9e00afcf0 2023-10-06T05:23:45.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e85ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e00c6c00 con 0x7fe9e00c7190 2023-10-06T05:23:45.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e639b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 0x7fe9e0144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e639b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 0x7fe9e0144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44264/0 (socket says 172.21.15.31:44264) 2023-10-06T05:23:45.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e639b700 1 -- 172.21.15.31:0/2501009160 learned_addr learned my addr 172.21.15.31:0/2501009160 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:45.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e5b9a700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e00afcf0 0x7fe9e00c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e6b9c700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9e00c7190 0x7fe9e00c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e639b700 1 -- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9e00c7190 msgr2=0x7fe9e00c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e639b700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9e00c7190 0x7fe9e00c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e639b700 1 -- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e00afcf0 msgr2=0x7fe9e00c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e639b700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e00afcf0 0x7fe9e00c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e639b700 1 -- 172.21.15.31:0/2501009160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d0009420 con 0x7fe9e00aef00 2023-10-06T05:23:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.197+0000 7fe9e5b9a700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e00afcf0 0x7fe9e00c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.198+0000 7fe9e639b700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 0x7fe9e0144db0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00ec80 tx=0x7fe9dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.200+0000 7fe9d67fc700 1 -- 172.21.15.31:0/2501009160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00c9d0 con 0x7fe9e00aef00 2023-10-06T05:23:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.200+0000 7fe9d67fc700 1 -- 172.21.15.31:0/2501009160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc00cb30 con 0x7fe9e00aef00 2023-10-06T05:23:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.200+0000 7fe9d67fc700 1 -- 172.21.15.31:0/2501009160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc013470 con 0x7fe9e00aef00 2023-10-06T05:23:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.200+0000 7fe9e85ff700 1 -- 172.21.15.31:0/2501009160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e0150960 con 0x7fe9e00aef00 2023-10-06T05:23:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.200+0000 7fe9e85ff700 1 -- 172.21.15.31:0/2501009160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e01452f0 con 0x7fe9e00aef00 2023-10-06T05:23:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.201+0000 7fe9e85ff700 1 -- 172.21.15.31:0/2501009160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e0151290 con 0x7fe9e00aef00 2023-10-06T05:23:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.202+0000 7fe9d67fc700 1 -- 172.21.15.31:0/2501009160 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe9dc013820 con 0x7fe9e00aef00 2023-10-06T05:23:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.203+0000 7fe9d67fc700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe9c40673f0 0x7fe9c40698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.203+0000 7fe9d67fc700 1 -- 172.21.15.31:0/2501009160 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe9dc085180 con 0x7fe9e00aef00 2023-10-06T05:23:45.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.203+0000 7fe9e5b9a700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe9c40673f0 0x7fe9c40698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.204+0000 7fe9e5b9a700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe9c40673f0 0x7fe9c40698a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe9e006f4e0 tx=0x7fe9d0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.207+0000 7fe9d67fc700 1 -- 172.21.15.31:0/2501009160 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9dc051cd0 con 0x7fe9e00aef00 2023-10-06T05:23:45.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.210+0000 7fe9d67fc700 1 -- 172.21.15.31:0/2501009160 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 18) v1 ==== 85148+0+0 (secure 0 0 0) 0x7fe9dc0520a0 con 0x7fe9e00aef00 2023-10-06T05:23:45.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.446+0000 7fe9e85ff700 1 -- 172.21.15.31:0/2501009160 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000851349", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9e01510f0 con 0x7fe9c40673f0 2023-10-06T05:23:45.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.498+0000 7fe9d67fc700 1 -- 172.21.15.31:0/2501009160 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe9d0000d10 con 0x7fe9c40673f0 2023-10-06T05:23:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.500+0000 7fe9e85ff700 1 -- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe9c40673f0 msgr2=0x7fe9c40698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.500+0000 7fe9e85ff700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe9c40673f0 0x7fe9c40698a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe9e006f4e0 tx=0x7fe9d0017000 comp rx=0 tx=0).stop 2023-10-06T05:23:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.500+0000 7fe9e85ff700 1 -- 172.21.15.31:0/2501009160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 msgr2=0x7fe9e0144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.500+0000 7fe9e85ff700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 0x7fe9e0144db0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00ec80 tx=0x7fe9dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.501+0000 7fe9e85ff700 1 -- 172.21.15.31:0/2501009160 shutdown_connections 2023-10-06T05:23:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.501+0000 7fe9e85ff700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe9c40673f0 0x7fe9c40698a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.501+0000 7fe9e85ff700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9e00c7190 0x7fe9e00c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.501+0000 7fe9e85ff700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e00aef00 0x7fe9e0144db0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.501+0000 7fe9e85ff700 1 --2- 172.21.15.31:0/2501009160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9e00afcf0 0x7fe9e00c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.501+0000 7fe9e85ff700 1 -- 172.21.15.31:0/2501009160 >> 172.21.15.31:0/2501009160 conn(0x7fe9e00cd3d0 msgr2=0x7fe9e00ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:45.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.501+0000 7fe9e85ff700 1 -- 172.21.15.31:0/2501009160 shutdown_connections 2023-10-06T05:23:45.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.501+0000 7fe9e85ff700 1 -- 172.21.15.31:0/2501009160 wait complete. 2023-10-06T05:23:45.507 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:45.505+0000 7f2e46181700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:45.507 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:45.505+0000 7f2e46181700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:45.508 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:45.505+0000 7f2e46181700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:45.508 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:45.505+0000 7f2e46181700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:45.508 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:45.505+0000 7f2e46181700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:45.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000851350 group_0000000000059849 2023-10-06T05:23:45.525 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:45.523+0000 7f2e46982700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:45.525 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:45.523+0000 7f2e46982700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:45.525 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:45.523+0000 7f2e46982700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:45.525 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:45.523+0000 7f2e46982700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:45.525 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:45.523+0000 7f2e46982700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:23:45.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.718+0000 7fbaab7ff700 1 Processor -- start 2023-10-06T05:23:45.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.718+0000 7fbaab7ff700 1 -- start start 2023-10-06T05:23:45.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.720+0000 7fbaab7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbaa40c6680 0x7fbaa406eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.720+0000 7fbaab7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa40c0f70 0x7fbaa4156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.720+0000 7fbaab7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa40ca300 con 0x7fbaa4156940 2023-10-06T05:23:45.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.720+0000 7fbaab7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa406dc00 con 0x7fbaa40c6680 2023-10-06T05:23:45.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.720+0000 7fbaab7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa406f0d0 con 0x7fbaa40c0f70 2023-10-06T05:23:45.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.720+0000 7fbaa8d9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbaa40c6680 0x7fbaa406eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.720+0000 7fbaa9d9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa40c0f70 0x7fbaa4156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.720+0000 7fbaa8d9a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbaa40c6680 0x7fbaa406eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:47380/0 (socket says 172.21.15.31:47380) 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.720+0000 7fbaa8d9a700 1 -- 172.21.15.31:0/3831508885 learned_addr learned my addr 172.21.15.31:0/3831508885 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.721+0000 7fba9bfff700 1 -- 172.21.15.31:0/3831508885 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134888763 0 0) 0x7fbaa40ca300 con 0x7fbaa4156940 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.721+0000 7fba9bfff700 1 -- 172.21.15.31:0/3831508885 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba90003570 con 0x7fbaa4156940 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.721+0000 7fbaa8d9a700 1 -- 172.21.15.31:0/3831508885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa40c0f70 msgr2=0x7fbaa4156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.721+0000 7fbaa8d9a700 1 --2- 172.21.15.31:0/3831508885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa40c0f70 0x7fbaa4156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.721+0000 7fbaa8d9a700 1 -- 172.21.15.31:0/3831508885 >> v1:172.21.15.62:6789/0 conn(0x7fbaa4156940 legacy=0x7fbaa40ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.721+0000 7fbaa8d9a700 1 -- 172.21.15.31:0/3831508885 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa406f2c0 con 0x7fbaa40c6680 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.721+0000 7fbaa9d9c700 1 --2- 172.21.15.31:0/3831508885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa40c0f70 0x7fbaa4156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.721+0000 7fbaa8d9a700 1 --2- 172.21.15.31:0/3831508885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa40c6680 0x7fbaa406eb90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fba9400bc50 tx=0x7fba9400b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f90154b328be27f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.721+0000 7fba9bfff700 1 -- 172.21.15.31:0/3831508885 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9400db40 con 0x7fbaa40c6680 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.722+0000 7fba9bfff700 1 -- 172.21.15.31:0/3831508885 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9400dca0 con 0x7fbaa40c6680 2023-10-06T05:23:45.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.722+0000 7fba9bfff700 1 -- 172.21.15.31:0/3831508885 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba940196d0 con 0x7fbaa40c6680 2023-10-06T05:23:45.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.722+0000 7fbaab7ff700 1 -- 172.21.15.31:0/3831508885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa40c6680 msgr2=0x7fbaa406eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.722+0000 7fbaab7ff700 1 --2- 172.21.15.31:0/3831508885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa40c6680 0x7fbaa406eb90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fba9400bc50 tx=0x7fba9400b4f0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.722+0000 7fbaab7ff700 1 -- 172.21.15.31:0/3831508885 shutdown_connections 2023-10-06T05:23:45.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.722+0000 7fbaab7ff700 1 --2- 172.21.15.31:0/3831508885 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa40c0f70 0x7fbaa4156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.722+0000 7fbaab7ff700 1 --2- 172.21.15.31:0/3831508885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa40c6680 0x7fbaa406eb90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.722+0000 7fbaab7ff700 1 -- 172.21.15.31:0/3831508885 >> 172.21.15.31:0/3831508885 conn(0x7fbaa40c8ab0 msgr2=0x7fbaa40c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:45.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.722+0000 7fbaab7ff700 1 -- 172.21.15.31:0/3831508885 shutdown_connections 2023-10-06T05:23:45.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.722+0000 7fbaab7ff700 1 -- 172.21.15.31:0/3831508885 wait complete. 2023-10-06T05:23:45.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.724+0000 7fbaab7ff700 1 Processor -- start 2023-10-06T05:23:45.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.724+0000 7fbaab7ff700 1 -- start start 2023-10-06T05:23:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaab7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaa40c0f70 0x7fbaa4169480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaab7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa40c6680 0x7fbaa40c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaab7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa4156940 0x7fbaa413b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaab7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa41699c0 con 0x7fbaa40c0f70 2023-10-06T05:23:45.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaab7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa4158160 con 0x7fbaa40c6680 2023-10-06T05:23:45.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaab7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa4157b50 con 0x7fbaa4156940 2023-10-06T05:23:45.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaa959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaa40c0f70 0x7fbaa4169480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaa959b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaa40c0f70 0x7fbaa4169480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44272/0 (socket says 172.21.15.31:44272) 2023-10-06T05:23:45.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaa959b700 1 -- 172.21.15.31:0/497999245 learned_addr learned my addr 172.21.15.31:0/497999245 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:45.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaa9d9c700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa4156940 0x7fbaa413b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.725+0000 7fbaa8d9a700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa40c6680 0x7fbaa40c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fbaa9d9c700 1 -- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa40c6680 msgr2=0x7fbaa40c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fbaa9d9c700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa40c6680 0x7fbaa40c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fbaa9d9c700 1 -- 172.21.15.31:0/497999245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaa40c0f70 msgr2=0x7fbaa4169480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:45.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fbaa9d9c700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaa40c0f70 0x7fbaa4169480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:45.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fbaa9d9c700 1 -- 172.21.15.31:0/497999245 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c009a20 con 0x7fbaa4156940 2023-10-06T05:23:45.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fbaa959b700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaa40c0f70 0x7fbaa4169480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:45.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fbaa9d9c700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa4156940 0x7fbaa413b790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009f40 tx=0x7fba9c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fba997fa700 1 -- 172.21.15.31:0/497999245 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c00cb10 con 0x7fbaa4156940 2023-10-06T05:23:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fba997fa700 1 -- 172.21.15.31:0/497999245 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c00f040 con 0x7fbaa4156940 2023-10-06T05:23:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fba997fa700 1 -- 172.21.15.31:0/497999245 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c014480 con 0x7fbaa4156940 2023-10-06T05:23:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fbaab7ff700 1 -- 172.21.15.31:0/497999245 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9400b790 con 0x7fbaa4156940 2023-10-06T05:23:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.726+0000 7fbaab7ff700 1 -- 172.21.15.31:0/497999245 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa4157df0 con 0x7fbaa4156940 2023-10-06T05:23:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.727+0000 7fbaab7ff700 1 -- 172.21.15.31:0/497999245 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa404c380 con 0x7fbaa4156940 2023-10-06T05:23:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.728+0000 7fba997fa700 1 -- 172.21.15.31:0/497999245 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 18) v1 ==== 85148+0+0 (secure 0 0 0) 0x7fba9c003020 con 0x7fbaa4156940 2023-10-06T05:23:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.729+0000 7fba997fa700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fba880673b0 0x7fba88069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.729+0000 7fba997fa700 1 -- 172.21.15.31:0/497999245 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fba9c085840 con 0x7fbaa4156940 2023-10-06T05:23:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.730+0000 7fbaa959b700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fba880673b0 0x7fba88069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:45.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.730+0000 7fbaa959b700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fba880673b0 0x7fba88069860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0007fd0 tx=0x7fbaa0007e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.735+0000 7fba997fa700 1 -- 172.21.15.31:0/497999245 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba9c052180 con 0x7fbaa4156940 2023-10-06T05:23:45.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:45.973+0000 7fbaab7ff700 1 -- 172.21.15.31:0/497999245 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000851350", "group_name": "group_0000000000059849", "target": ["mon-mgr", ""]}) v1 -- 0x7fbaa406f5a0 con 0x7fba880673b0 2023-10-06T05:23:46.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.021+0000 7fba997fa700 1 -- 172.21.15.31:0/497999245 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbaa406f5a0 con 0x7fba880673b0 2023-10-06T05:23:46.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.023+0000 7fbaab7ff700 1 -- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fba880673b0 msgr2=0x7fba88069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.023+0000 7fbaab7ff700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fba880673b0 0x7fba88069860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0007fd0 tx=0x7fbaa0007e10 comp rx=0 tx=0).stop 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.023+0000 7fbaab7ff700 1 -- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa4156940 msgr2=0x7fbaa413b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.023+0000 7fbaab7ff700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa4156940 0x7fbaa413b790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009f40 tx=0x7fba9c00b040 comp rx=0 tx=0).stop 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.024+0000 7fbaab7ff700 1 -- 172.21.15.31:0/497999245 shutdown_connections 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.024+0000 7fbaab7ff700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fba880673b0 0x7fba88069860 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.024+0000 7fbaab7ff700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbaa4156940 0x7fbaa413b790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.024+0000 7fbaab7ff700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaa40c0f70 0x7fbaa4169480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.024+0000 7fbaab7ff700 1 --2- 172.21.15.31:0/497999245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaa40c6680 0x7fbaa40c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.024+0000 7fbaab7ff700 1 -- 172.21.15.31:0/497999245 >> 172.21.15.31:0/497999245 conn(0x7fbaa40c8ab0 msgr2=0x7fbaa4157310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.024+0000 7fbaab7ff700 1 -- 172.21.15.31:0/497999245 shutdown_connections 2023-10-06T05:23:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.024+0000 7fbaab7ff700 1 -- 172.21.15.31:0/497999245 wait complete. 2023-10-06T05:23:46.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000851351 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.246+0000 7fc0391ff700 1 Processor -- start 2023-10-06T05:23:46.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.247+0000 7fc0391ff700 1 -- start start 2023-10-06T05:23:46.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.248+0000 7fc0391ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0340b0b90 0x7fc0340af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.248+0000 7fc0391ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc0340af6c0 0x7fc0340adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.248+0000 7fc0391ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0340ad690 con 0x7fc034153d40 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.248+0000 7fc0391ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0340acd60 con 0x7fc0340af6c0 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.248+0000 7fc0391ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340ad040 con 0x7fc0340b0b90 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.248+0000 7fc032d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc034153d40 0x7fc0340b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45868/0 (socket says 172.21.15.31:45868) 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.248+0000 7fc032d9d700 1 -- 172.21.15.31:0/411964390 learned_addr learned my addr 172.21.15.31:0/411964390 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.248+0000 7fc03259c700 1 --2- 172.21.15.31:0/411964390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0340b0b90 0x7fc0340af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.248+0000 7fc03359e700 1 --2- 172.21.15.31:0/411964390 >> v2:172.21.15.62:3300/0 conn(0x7fc0340af6c0 0x7fc0340adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.249+0000 7fc031d9b700 1 -- 172.21.15.31:0/411964390 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322034002 0 0) 0x7fc0340ad690 con 0x7fc034153d40 2023-10-06T05:23:46.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.249+0000 7fc031d9b700 1 -- 172.21.15.31:0/411964390 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc01c003570 con 0x7fc034153d40 2023-10-06T05:23:46.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.249+0000 7fc03259c700 1 -- 172.21.15.31:0/411964390 >> v2:172.21.15.62:3300/0 conn(0x7fc0340af6c0 msgr2=0x7fc0340adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:46.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.249+0000 7fc03259c700 1 --2- 172.21.15.31:0/411964390 >> v2:172.21.15.62:3300/0 conn(0x7fc0340af6c0 0x7fc0340adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.249+0000 7fc03259c700 1 -- 172.21.15.31:0/411964390 >> v1:172.21.15.31:6789/0 conn(0x7fc034153d40 legacy=0x7fc0340b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.249+0000 7fc03259c700 1 -- 172.21.15.31:0/411964390 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0340ad320 con 0x7fc0340b0b90 2023-10-06T05:23:46.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.249+0000 7fc03259c700 1 --2- 172.21.15.31:0/411964390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0340b0b90 0x7fc0340af180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc0280098e0 tx=0x7fc0280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d573ad75e8a26e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.249+0000 7fc031d9b700 1 -- 172.21.15.31:0/411964390 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028011630 con 0x7fc0340b0b90 2023-10-06T05:23:46.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.249+0000 7fc031d9b700 1 -- 172.21.15.31:0/411964390 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc028011790 con 0x7fc0340b0b90 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.249+0000 7fc031d9b700 1 -- 172.21.15.31:0/411964390 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028011b30 con 0x7fc0340b0b90 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.250+0000 7fc0391ff700 1 -- 172.21.15.31:0/411964390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0340b0b90 msgr2=0x7fc0340af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.250+0000 7fc0391ff700 1 --2- 172.21.15.31:0/411964390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0340b0b90 0x7fc0340af180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc0280098e0 tx=0x7fc0280091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.250+0000 7fc0391ff700 1 -- 172.21.15.31:0/411964390 shutdown_connections 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.250+0000 7fc0391ff700 1 --2- 172.21.15.31:0/411964390 >> v2:172.21.15.62:3300/0 conn(0x7fc0340af6c0 0x7fc0340adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.250+0000 7fc0391ff700 1 --2- 172.21.15.31:0/411964390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0340b0b90 0x7fc0340af180 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.250+0000 7fc0391ff700 1 -- 172.21.15.31:0/411964390 >> 172.21.15.31:0/411964390 conn(0x7fc034067560 msgr2=0x7fc034067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.250+0000 7fc0391ff700 1 -- 172.21.15.31:0/411964390 shutdown_connections 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.250+0000 7fc0391ff700 1 -- 172.21.15.31:0/411964390 wait complete. 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.251+0000 7fc0391ff700 1 Processor -- start 2023-10-06T05:23:46.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.252+0000 7fc0391ff700 1 -- start start 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.252+0000 7fc0391ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0340af6c0 0x7fc03413f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.252+0000 7fc0391ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0340b0b90 0x7fc03413fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.252+0000 7fc0391ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc034153d40 0x7fc03414a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.252+0000 7fc0391ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340ab240 con 0x7fc034153d40 2023-10-06T05:23:46.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.252+0000 7fc0391ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03413f9b0 con 0x7fc0340af6c0 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.252+0000 7fc0391ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340aa4f0 con 0x7fc0340b0b90 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.253+0000 7fc03359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc034153d40 0x7fc03414a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.253+0000 7fc03359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc034153d40 0x7fc03414a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44282/0 (socket says 172.21.15.31:44282) 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.253+0000 7fc03359e700 1 -- 172.21.15.31:0/4044194912 learned_addr learned my addr 172.21.15.31:0/4044194912 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.253+0000 7fc032d9d700 1 --2- 172.21.15.31:0/4044194912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0340af6c0 0x7fc03413f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.253+0000 7fc03259c700 1 --2- 172.21.15.31:0/4044194912 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0340b0b90 0x7fc03413fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.253+0000 7fc03359e700 1 -- 172.21.15.31:0/4044194912 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0340b0b90 msgr2=0x7fc03413fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.253+0000 7fc03359e700 1 --2- 172.21.15.31:0/4044194912 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0340b0b90 0x7fc03413fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.253+0000 7fc03359e700 1 -- 172.21.15.31:0/4044194912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0340af6c0 msgr2=0x7fc03413f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.253+0000 7fc03359e700 1 --2- 172.21.15.31:0/4044194912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0340af6c0 0x7fc03413f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.253+0000 7fc03359e700 1 -- 172.21.15.31:0/4044194912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc028009420 con 0x7fc034153d40 2023-10-06T05:23:46.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.254+0000 7fc03359e700 1 --2- 172.21.15.31:0/4044194912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc034153d40 0x7fc03414a780 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009e90 tx=0x7fc02c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.254+0000 7fc01affd700 1 -- 172.21.15.31:0/4044194912 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c00cb70 con 0x7fc034153d40 2023-10-06T05:23:46.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.254+0000 7fc01affd700 1 -- 172.21.15.31:0/4044194912 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c00f040 con 0x7fc034153d40 2023-10-06T05:23:46.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.254+0000 7fc01affd700 1 -- 172.21.15.31:0/4044194912 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c014480 con 0x7fc034153d40 2023-10-06T05:23:46.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.254+0000 7fc0391ff700 1 -- 172.21.15.31:0/4044194912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0340aa7d0 con 0x7fc034153d40 2023-10-06T05:23:46.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.254+0000 7fc0391ff700 1 -- 172.21.15.31:0/4044194912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc034149f80 con 0x7fc034153d40 2023-10-06T05:23:46.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.255+0000 7fc0391ff700 1 -- 172.21.15.31:0/4044194912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03404c380 con 0x7fc034153d40 2023-10-06T05:23:46.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.256+0000 7fc01affd700 1 -- 172.21.15.31:0/4044194912 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 85148+0+0 (secure 0 0 0) 0x7fc02c0147f0 con 0x7fc034153d40 2023-10-06T05:23:46.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.258+0000 7fc01affd700 1 --2- 172.21.15.31:0/4044194912 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc0200673b0 0x7fc020069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.258+0000 7fc01affd700 1 -- 172.21.15.31:0/4044194912 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc02c085d00 con 0x7fc034153d40 2023-10-06T05:23:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.258+0000 7fc032d9d700 1 --2- 172.21.15.31:0/4044194912 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc0200673b0 0x7fc020069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.259+0000 7fc032d9d700 1 --2- 172.21.15.31:0/4044194912 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc0200673b0 0x7fc020069860 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc024007fd0 tx=0x7fc024009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.262+0000 7fc01affd700 1 -- 172.21.15.31:0/4044194912 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc02c0525e0 con 0x7fc034153d40 2023-10-06T05:23:46.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.499+0000 7fc0391ff700 1 -- 172.21.15.31:0/4044194912 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000851351", "target": ["mon-mgr", ""]}) v1 -- 0x7fc03406b600 con 0x7fc0200673b0 2023-10-06T05:23:46.524 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:23:46.522+0000 7f2e4417d700 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvolume_0000000000851351' is not ready for operation rm (use --force to override) 2023-10-06T05:23:46.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.523+0000 7fc01affd700 1 -- 172.21.15.31:0/4044194912 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvolume_0000000000851351' is not ready for operation rm (use --force to override)) v1 ==== 102+0+0 (secure 0 0 0) 0x7fc03406b600 con 0x7fc0200673b0 2023-10-06T05:23:46.528 INFO:teuthology.orchestra.run.smithi031.stderr:Error EAGAIN: subvolume 'subvolume_0000000000851351' is not ready for operation rm (use --force to override) 2023-10-06T05:23:46.534 DEBUG:teuthology.orchestra.run:got remote process result: 11 2023-10-06T05:23:46.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:46.535 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvolume_0000000000851351/.meta) 2023-10-06T05:23:46.606 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:23:46.606 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T05:23:46.606 INFO:teuthology.orchestra.run.smithi031.stdout:type = clone 2023-10-06T05:23:46.606 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000851351/082c8b85-9546-4f55-8c73-e3e044e17983 2023-10-06T05:23:46.606 INFO:teuthology.orchestra.run.smithi031.stdout:state = pending 2023-10-06T05:23:46.606 INFO:teuthology.orchestra.run.smithi031.stdout:[source] 2023-10-06T05:23:46.606 INFO:teuthology.orchestra.run.smithi031.stdout:volume = fake 2023-10-06T05:23:46.607 INFO:teuthology.orchestra.run.smithi031.stdout:subvolume = fake 2023-10-06T05:23:46.607 INFO:teuthology.orchestra.run.smithi031.stdout:snapshot = fake 2023-10-06T05:23:46.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000851351 --force 2023-10-06T05:23:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.808+0000 7f3ef65ff700 1 Processor -- start 2023-10-06T05:23:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.809+0000 7f3ef65ff700 1 -- start start 2023-10-06T05:23:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.810+0000 7f3ef65ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3ef0153d20 0x7f3ef015d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.810+0000 7f3ef65ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef015d750 0x7f3ef015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.810+0000 7f3ef65ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef015dbb0 con 0x7f3ef0153950 2023-10-06T05:23:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.810+0000 7f3ef65ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef0154170 con 0x7f3ef0153d20 2023-10-06T05:23:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.810+0000 7f3ef65ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef015bb20 con 0x7f3ef015d750 2023-10-06T05:23:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.810+0000 7f3eeffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3ef0153950 0x7f3ef00cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45884/0 (socket says 172.21.15.31:45884) 2023-10-06T05:23:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.810+0000 7f3eeffff700 1 -- 172.21.15.31:0/1345792667 learned_addr learned my addr 172.21.15.31:0/1345792667 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.810+0000 7f3eef7fe700 1 --2- 172.21.15.31:0/1345792667 >> v2:172.21.15.62:3300/0 conn(0x7f3ef0153d20 0x7f3ef015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.810+0000 7f3ef4b9c700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef015d750 0x7f3ef015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476148366 0 0) 0x7f3ee0001290 con 0x7f3ef0153950 2023-10-06T05:23:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed8003570 con 0x7f3ef0153950 2023-10-06T05:23:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2691344493 0 0) 0x7f3ed8003570 con 0x7f3ef0153950 2023-10-06T05:23:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef015d750 msgr2=0x7f3ef015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef015d750 0x7f3ef015bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 >> v2:172.21.15.62:3300/0 conn(0x7f3ef0153d20 msgr2=0x7f3ef015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 --2- 172.21.15.31:0/1345792667 >> v2:172.21.15.62:3300/0 conn(0x7f3ef0153d20 0x7f3ef015d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef015af10 con 0x7f3ef0153950 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f3ee0001780 con 0x7f3ef0153950 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 >> v1:172.21.15.31:6789/0 conn(0x7f3ef0153950 legacy=0x7f3ef00cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed8003ae0 0x7f3ed8003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef015d750 0x7f3ed8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed8004d50 0x7f3ed80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef015bb20 con 0x7f3ed8003ae0 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed800bb10 con 0x7f3ed8004d50 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed800bc80 con 0x7f3ef015d750 2023-10-06T05:23:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.811+0000 7f3eeffff700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed8003ae0 0x7f3ed8003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3eef7fe700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef015d750 0x7f3ed8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3ef4b9c700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed8004d50 0x7f3ed80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3eeffff700 1 -- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef015d750 msgr2=0x7f3ed8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3eeffff700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef015d750 0x7f3ed8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3eeffff700 1 -- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed8004d50 msgr2=0x7f3ed80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3eeffff700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed8004d50 0x7f3ed80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3eeffff700 1 -- 172.21.15.31:0/1345792667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed8003570 con 0x7f3ed8003ae0 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3ef4b9c700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed8004d50 0x7f3ed80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3eeffff700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed8003ae0 0x7f3ed8003ef0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c410 tx=0x7f3ef015ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20d9e78f00a7dce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0007cb0 con 0x7f3ed8003ae0 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee000cca0 con 0x7f3ed8003ae0 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.812+0000 7f3eeeffd700 1 -- 172.21.15.31:0/1345792667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0007320 con 0x7f3ed8003ae0 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.813+0000 7f3ef65ff700 1 -- 172.21.15.31:0/1345792667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed8003ae0 msgr2=0x7f3ed8003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.813+0000 7f3ef65ff700 1 --2- 172.21.15.31:0/1345792667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ed8003ae0 0x7f3ed8003ef0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c410 tx=0x7f3ef015ac20 comp rx=0 tx=0).stop 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.813+0000 7f3ef4b9c700 1 -- 172.21.15.31:0/1345792667 reap_dead start 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.813+0000 7f3ef65ff700 1 -- 172.21.15.31:0/1345792667 shutdown_connections 2023-10-06T05:23:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.813+0000 7f3ef65ff700 1 -- 172.21.15.31:0/1345792667 >> 172.21.15.31:0/1345792667 conn(0x7f3ef00681b0 msgr2=0x7f3ef00685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.813+0000 7f3ef65ff700 1 -- 172.21.15.31:0/1345792667 shutdown_connections 2023-10-06T05:23:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.813+0000 7f3ef65ff700 1 -- 172.21.15.31:0/1345792667 wait complete. 2023-10-06T05:23:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.815+0000 7f3ef65ff700 1 Processor -- start 2023-10-06T05:23:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.815+0000 7f3ef65ff700 1 -- start start 2023-10-06T05:23:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3ef65ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ed8003ae0 0x7f3ef016aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3ef65ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef016b3e0 0x7f3ef00cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3ef65ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00cfe20 0x7f3ef00b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3ef65ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00d02a0 con 0x7f3ef016b3e0 2023-10-06T05:23:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3ef65ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00ce8d0 con 0x7f3ef00cfe20 2023-10-06T05:23:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3ef65ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef016ab90 con 0x7f3ed8003ae0 2023-10-06T05:23:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3eef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef016b3e0 0x7f3ef00cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3eef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef016b3e0 0x7f3ef00cf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44300/0 (socket says 172.21.15.31:44300) 2023-10-06T05:23:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ed8003ae0 0x7f3ef016aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2577398805 learned_addr learned my addr 172.21.15.31:0/2577398805 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3ef4b9c700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00cfe20 0x7f3ef00b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ed8003ae0 msgr2=0x7f3ef016aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3eef7fe700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ed8003ae0 0x7f3ef016aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00cfe20 msgr2=0x7f3ef00b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3eef7fe700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00cfe20 0x7f3ef00b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.816+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2577398805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00ceb20 con 0x7f3ef016b3e0 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.817+0000 7f3eef7fe700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef016b3e0 0x7f3ef00cf8e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4011d40 tx=0x7f3ee40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.817+0000 7f3ed7fff700 1 -- 172.21.15.31:0/2577398805 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400f990 con 0x7f3ef016b3e0 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.817+0000 7f3ed7fff700 1 -- 172.21.15.31:0/2577398805 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee400faf0 con 0x7f3ef016b3e0 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.817+0000 7f3ed7fff700 1 -- 172.21.15.31:0/2577398805 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400fe70 con 0x7f3ef016b3e0 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.817+0000 7f3ef65ff700 1 -- 172.21.15.31:0/2577398805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef00cee00 con 0x7f3ef016b3e0 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.817+0000 7f3ef65ff700 1 -- 172.21.15.31:0/2577398805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef00cf3c0 con 0x7f3ef016b3e0 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.818+0000 7f3ef65ff700 1 -- 172.21.15.31:0/2577398805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef00cf5b0 con 0x7f3ef016b3e0 2023-10-06T05:23:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.819+0000 7f3ed7fff700 1 -- 172.21.15.31:0/2577398805 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 85148+0+0 (secure 0 0 0) 0x7f3ee4014650 con 0x7f3ef016b3e0 2023-10-06T05:23:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.820+0000 7f3ed7fff700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3edc0673b0 0x7f3edc069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.820+0000 7f3ed7fff700 1 -- 172.21.15.31:0/2577398805 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f3ee4018030 con 0x7f3ef016b3e0 2023-10-06T05:23:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.821+0000 7f3eeffff700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3edc0673b0 0x7f3edc069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.821+0000 7f3eeffff700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3edc0673b0 0x7f3edc069860 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0008fd0 tx=0x7f3ee0015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:46.824+0000 7f3ed7fff700 1 -- 172.21.15.31:0/2577398805 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ee4052260 con 0x7f3ef016b3e0 2023-10-06T05:23:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.061+0000 7f3ef65ff700 1 -- 172.21.15.31:0/2577398805 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000851351", "force": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3ef004c380 con 0x7f3edc0673b0 2023-10-06T05:23:47.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.092+0000 7f3ed7fff700 1 -- 172.21.15.31:0/2577398805 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3ef004c380 con 0x7f3edc0673b0 2023-10-06T05:23:47.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 -- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3edc0673b0 msgr2=0x7f3edc069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3edc0673b0 0x7f3edc069860 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0008fd0 tx=0x7f3ee0015000 comp rx=0 tx=0).stop 2023-10-06T05:23:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 -- 172.21.15.31:0/2577398805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef016b3e0 msgr2=0x7f3ef00cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef016b3e0 0x7f3ef00cf8e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4011d40 tx=0x7f3ee40091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 -- 172.21.15.31:0/2577398805 shutdown_connections 2023-10-06T05:23:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3edc0673b0 0x7f3edc069860 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00cfe20 0x7f3ef00b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ed8003ae0 0x7f3ef016aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 --2- 172.21.15.31:0/2577398805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef016b3e0 0x7f3ef00cf8e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 -- 172.21.15.31:0/2577398805 >> 172.21.15.31:0/2577398805 conn(0x7f3ef00681b0 msgr2=0x7f3ef00c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 -- 172.21.15.31:0/2577398805 shutdown_connections 2023-10-06T05:23:47.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.095+0000 7f3ef65ff700 1 -- 172.21.15.31:0/2577398805 wait complete. 2023-10-06T05:23:47.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2023-10-06T05:23:47.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.313+0000 7f83cd9ff700 1 Processor -- start 2023-10-06T05:23:47.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.313+0000 7f83cd9ff700 1 -- start start 2023-10-06T05:23:47.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.314+0000 7f83cd9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f83c815ca40 0x7f83c8068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.314+0000 7f83cd9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83c80c8f40 0x7f83c80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.314+0000 7f83cd9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c8068da0 con 0x7f83c806a300 2023-10-06T05:23:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.314+0000 7f83cd9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c80cabc0 con 0x7f83c815ca40 2023-10-06T05:23:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.314+0000 7f83cd9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c8068bb0 con 0x7f83c80c8f40 2023-10-06T05:23:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f83c815ca40 0x7f83c8068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c6ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f83c815ca40 0x7f83c8068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54798/0 (socket says 172.21.15.31:54798) 2023-10-06T05:23:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c6ffd700 1 -- 172.21.15.31:0/2838646021 learned_addr learned my addr 172.21.15.31:0/2838646021 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c67fc700 1 --1- 172.21.15.31:0/2838646021 >> v1:172.21.15.31:6789/0 conn(0x7f83c806a300 0x7f83c80c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36328/0 (socket says 172.21.15.31:36328) 2023-10-06T05:23:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c77fe700 1 --2- 172.21.15.31:0/2838646021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83c80c8f40 0x7f83c80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c6ffd700 1 -- 172.21.15.31:0/2838646021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83c80c8f40 msgr2=0x7f83c80adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c6ffd700 1 --2- 172.21.15.31:0/2838646021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83c80c8f40 0x7f83c80adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c6ffd700 1 -- 172.21.15.31:0/2838646021 >> v1:172.21.15.31:6789/0 conn(0x7f83c806a300 legacy=0x7f83c80c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c6ffd700 1 -- 172.21.15.31:0/2838646021 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8069080 con 0x7f83c815ca40 2023-10-06T05:23:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c5ffb700 1 -- 172.21.15.31:0/2838646021 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800882713 0 0) 0x7f83c8068da0 con 0x7f83c806a300 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.315+0000 7f83c6ffd700 1 --2- 172.21.15.31:0/2838646021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c815ca40 0x7f83c8068670 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f83b80099e0 tx=0x7f83b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef974f05af3a2ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83c5ffb700 1 -- 172.21.15.31:0/2838646021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800b820 con 0x7f83c815ca40 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83c5ffb700 1 -- 172.21.15.31:0/2838646021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b800b980 con 0x7f83c815ca40 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83c5ffb700 1 -- 172.21.15.31:0/2838646021 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800bd20 con 0x7f83c815ca40 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2838646021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c815ca40 msgr2=0x7f83c8068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83cd9ff700 1 --2- 172.21.15.31:0/2838646021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c815ca40 0x7f83c8068670 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f83b80099e0 tx=0x7f83b8009250 comp rx=0 tx=0).stop 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2838646021 shutdown_connections 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83cd9ff700 1 --2- 172.21.15.31:0/2838646021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c815ca40 0x7f83c8068670 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83cd9ff700 1 --2- 172.21.15.31:0/2838646021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83c80c8f40 0x7f83c80adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2838646021 >> 172.21.15.31:0/2838646021 conn(0x7f83c80ce290 msgr2=0x7f83c80ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2838646021 shutdown_connections 2023-10-06T05:23:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.316+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2838646021 wait complete. 2023-10-06T05:23:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.318+0000 7f83cd9ff700 1 Processor -- start 2023-10-06T05:23:47.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.318+0000 7f83cd9ff700 1 -- start start 2023-10-06T05:23:47.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83cd9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83c806a300 0x7f83c8159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83cd9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c80c8f40 0x7f83c80c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83cd9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83c815ca40 0x7f83c80c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83cd9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c815a150 con 0x7f83c80c8f40 2023-10-06T05:23:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83cd9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80c1a00 con 0x7f83c815ca40 2023-10-06T05:23:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83cd9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80c5d60 con 0x7f83c806a300 2023-10-06T05:23:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83c67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c80c8f40 0x7f83c80c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83c6ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83c806a300 0x7f83c8159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83c67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c80c8f40 0x7f83c80c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54814/0 (socket says 172.21.15.31:54814) 2023-10-06T05:23:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83c67fc700 1 -- 172.21.15.31:0/2208768742 learned_addr learned my addr 172.21.15.31:0/2208768742 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.319+0000 7f83c77fe700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83c815ca40 0x7f83c80c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83c67fc700 1 -- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83c806a300 msgr2=0x7f83c8159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83c67fc700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83c806a300 0x7f83c8159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83c67fc700 1 -- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83c815ca40 msgr2=0x7f83c80c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83c67fc700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83c815ca40 0x7f83c80c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83c67fc700 1 -- 172.21.15.31:0/2208768742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b80094f0 con 0x7f83c80c8f40 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83c67fc700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c80c8f40 0x7f83c80c52a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0011e0 tx=0x7f83bc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83aeffd700 1 -- 172.21.15.31:0/2208768742 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc005a70 con 0x7f83c80c8f40 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83aeffd700 1 -- 172.21.15.31:0/2208768742 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83bc005bd0 con 0x7f83c80c8f40 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83aeffd700 1 -- 172.21.15.31:0/2208768742 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0070d0 con 0x7f83c80c8f40 2023-10-06T05:23:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2208768742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c80c6040 con 0x7f83c80c8f40 2023-10-06T05:23:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.320+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2208768742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c80c6600 con 0x7f83c80c8f40 2023-10-06T05:23:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.321+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2208768742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c804c380 con 0x7f83c80c8f40 2023-10-06T05:23:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.322+0000 7f83aeffd700 1 -- 172.21.15.31:0/2208768742 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f83bc007480 con 0x7f83c80c8f40 2023-10-06T05:23:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.323+0000 7f83aeffd700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f83b40674d0 0x7f83b4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.323+0000 7f83aeffd700 1 -- 172.21.15.31:0/2208768742 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f83bc084610 con 0x7f83c80c8f40 2023-10-06T05:23:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.324+0000 7f83c6ffd700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f83b40674d0 0x7f83b4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.324+0000 7f83c6ffd700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f83b40674d0 0x7f83b4069980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f83b80099e0 tx=0x7f83b8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.327+0000 7f83aeffd700 1 -- 172.21.15.31:0/2208768742 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83bc050e50 con 0x7f83c80c8f40 2023-10-06T05:23:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.558+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2208768742 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f83c80ca1a0 con 0x7f83b40674d0 2023-10-06T05:23:47.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.563+0000 7f83aeffd700 1 -- 172.21.15.31:0/2208768742 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f83c80ca1a0 con 0x7f83b40674d0 2023-10-06T05:23:47.565 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-10-06T05:23:47.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f83b40674d0 msgr2=0x7f83b4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f83b40674d0 0x7f83b4069980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f83b80099e0 tx=0x7f83b8017000 comp rx=0 tx=0).stop 2023-10-06T05:23:47.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2208768742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c80c8f40 msgr2=0x7f83c80c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:47.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c80c8f40 0x7f83c80c52a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0011e0 tx=0x7f83bc007e10 comp rx=0 tx=0).stop 2023-10-06T05:23:47.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2208768742 shutdown_connections 2023-10-06T05:23:47.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f83b40674d0 0x7f83b4069980 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83c815ca40 0x7f83c80c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f83c806a300 0x7f83c8159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 --2- 172.21.15.31:0/2208768742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83c80c8f40 0x7f83c80c52a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:47.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2208768742 >> 172.21.15.31:0/2208768742 conn(0x7f83c80ce290 msgr2=0x7f83c815a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:47.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2208768742 shutdown_connections 2023-10-06T05:23:47.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:47.565+0000 7f83cd9ff700 1 -- 172.21.15.31:0/2208768742 wait complete. 2023-10-06T05:23:47.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:47.585 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T05:23:47.679 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T05:23:47.679 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:23:47.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:47.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:47.893 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:23:47.893 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:23:47.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:47.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:23:48.003 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:23:48.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:48.003 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:23:48.053 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:23:48.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:23:48.054 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:23:48.149 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:23:48.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:23:48.150 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:23:48.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:48.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.392+0000 7f32af3ff700 1 Processor -- start 2023-10-06T05:23:48.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.392+0000 7f32af3ff700 1 -- start start 2023-10-06T05:23:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.393+0000 7f32af3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 0x7f32a8159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.393+0000 7f32af3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f32a80cc0d0 0x7f32a8068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.393+0000 7f32af3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32a8068bb0 0x7f32a80c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.393+0000 7f32af3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a8069010 con 0x7f32a8068bb0 2023-10-06T05:23:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.393+0000 7f32af3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a80c95b0 con 0x7f32a80cc0d0 2023-10-06T05:23:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.393+0000 7f32af3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80c9890 con 0x7f32a81595a0 2023-10-06T05:23:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32a8068bb0 0x7f32a80c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32a8068bb0 0x7f32a80c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54824/0 (socket says 172.21.15.31:54824) 2023-10-06T05:23:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad99c700 1 -- 172.21.15.31:0/1764681544 learned_addr learned my addr 172.21.15.31:0/1764681544 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ac99a700 1 --2- 172.21.15.31:0/1764681544 >> v2:172.21.15.62:3300/0 conn(0x7f32a80cc0d0 0x7f32a8068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad19b700 1 --2- 172.21.15.31:0/1764681544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 0x7f32a8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad99c700 1 -- 172.21.15.31:0/1764681544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 msgr2=0x7f32a8159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad99c700 1 --2- 172.21.15.31:0/1764681544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 0x7f32a8159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad99c700 1 -- 172.21.15.31:0/1764681544 >> v2:172.21.15.62:3300/0 conn(0x7f32a80cc0d0 msgr2=0x7f32a8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad99c700 1 --2- 172.21.15.31:0/1764681544 >> v2:172.21.15.62:3300/0 conn(0x7f32a80cc0d0 0x7f32a8068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad99c700 1 -- 172.21.15.31:0/1764681544 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a80c9b70 con 0x7f32a8068bb0 2023-10-06T05:23:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad19b700 1 --2- 172.21.15.31:0/1764681544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 0x7f32a8159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.394+0000 7f32ad99c700 1 --2- 172.21.15.31:0/1764681544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a8068bb0 0x7f32a80c8930 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f32a806f290 tx=0x7f32a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d832ab16ec35a7e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f329ffff700 1 -- 172.21.15.31:0/1764681544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a000e070 con 0x7f32a8068bb0 2023-10-06T05:23:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f329ffff700 1 -- 172.21.15.31:0/1764681544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a000ba20 con 0x7f32a8068bb0 2023-10-06T05:23:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f329ffff700 1 -- 172.21.15.31:0/1764681544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a00092e0 con 0x7f32a8068bb0 2023-10-06T05:23:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f32af3ff700 1 -- 172.21.15.31:0/1764681544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a8068bb0 msgr2=0x7f32a80c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f32af3ff700 1 --2- 172.21.15.31:0/1764681544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a8068bb0 0x7f32a80c8930 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f32a806f290 tx=0x7f32a0009840 comp rx=0 tx=0).stop 2023-10-06T05:23:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f32af3ff700 1 -- 172.21.15.31:0/1764681544 shutdown_connections 2023-10-06T05:23:48.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f32af3ff700 1 --2- 172.21.15.31:0/1764681544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a8068bb0 0x7f32a80c8930 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f32af3ff700 1 --2- 172.21.15.31:0/1764681544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 0x7f32a8159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f32af3ff700 1 --2- 172.21.15.31:0/1764681544 >> v2:172.21.15.62:3300/0 conn(0x7f32a80cc0d0 0x7f32a8068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f32af3ff700 1 -- 172.21.15.31:0/1764681544 >> 172.21.15.31:0/1764681544 conn(0x7f32a806b620 msgr2=0x7f32a806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f32af3ff700 1 -- 172.21.15.31:0/1764681544 shutdown_connections 2023-10-06T05:23:48.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.395+0000 7f32af3ff700 1 -- 172.21.15.31:0/1764681544 wait complete. 2023-10-06T05:23:48.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.397+0000 7f32af3ff700 1 Processor -- start 2023-10-06T05:23:48.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.397+0000 7f32af3ff700 1 -- start start 2023-10-06T05:23:48.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32af3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a8068bb0 0x7f32a8152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32af3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a80cc0d0 0x7f32a80b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32af3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 0x7f32a815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32af3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80c93b0 con 0x7f32a80cc0d0 2023-10-06T05:23:48.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32af3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a8152f70 con 0x7f32a8068bb0 2023-10-06T05:23:48.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32af3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80b1cb0 con 0x7f32a81595a0 2023-10-06T05:23:48.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32ac99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a80cc0d0 0x7f32a80b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32ac99a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a80cc0d0 0x7f32a80b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54836/0 (socket says 172.21.15.31:54836) 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32ac99a700 1 -- 172.21.15.31:0/2870136484 learned_addr learned my addr 172.21.15.31:0/2870136484 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32ad19b700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a8068bb0 0x7f32a8152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.398+0000 7f32ad99c700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 0x7f32a815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f32ac99a700 1 -- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 msgr2=0x7f32a815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f32ac99a700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 0x7f32a815b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f32ac99a700 1 -- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a8068bb0 msgr2=0x7f32a8152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f32ac99a700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a8068bb0 0x7f32a8152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f32ac99a700 1 -- 172.21.15.31:0/2870136484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0009ae0 con 0x7f32a80cc0d0 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f32ad19b700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a8068bb0 0x7f32a8152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f32ac99a700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a80cc0d0 0x7f32a80b2e20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f329800ecb0 tx=0x7f32980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f329d7fa700 1 -- 172.21.15.31:0/2870136484 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329800c880 con 0x7f32a80cc0d0 2023-10-06T05:23:48.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f329d7fa700 1 -- 172.21.15.31:0/2870136484 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329800c9e0 con 0x7f32a80cc0d0 2023-10-06T05:23:48.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f329d7fa700 1 -- 172.21.15.31:0/2870136484 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329800cd80 con 0x7f32a80cc0d0 2023-10-06T05:23:48.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f32af3ff700 1 -- 172.21.15.31:0/2870136484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a80b3360 con 0x7f32a80cc0d0 2023-10-06T05:23:48.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.399+0000 7f32af3ff700 1 -- 172.21.15.31:0/2870136484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a80c90d0 con 0x7f32a80cc0d0 2023-10-06T05:23:48.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.400+0000 7f32af3ff700 1 -- 172.21.15.31:0/2870136484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a80b1e40 con 0x7f32a80cc0d0 2023-10-06T05:23:48.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.401+0000 7f329d7fa700 1 -- 172.21.15.31:0/2870136484 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f3298013680 con 0x7f32a80cc0d0 2023-10-06T05:23:48.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.402+0000 7f329d7fa700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f328c0674d0 0x7f328c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.402+0000 7f329d7fa700 1 -- 172.21.15.31:0/2870136484 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f3298085770 con 0x7f32a80cc0d0 2023-10-06T05:23:48.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.403+0000 7f32ad19b700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f328c0674d0 0x7f328c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.403+0000 7f32ad19b700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f328c0674d0 0x7f328c069980 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f32a4006f00 tx=0x7f32a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.406+0000 7f329d7fa700 1 -- 172.21.15.31:0/2870136484 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3298051fb0 con 0x7f32a80cc0d0 2023-10-06T05:23:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.658+0000 7f32af3ff700 1 -- 172.21.15.31:0/2870136484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32a804c380 con 0x7f32a80cc0d0 2023-10-06T05:23:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.658+0000 7f329d7fa700 1 -- 172.21.15.31:0/2870136484 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4593 (secure 0 0 0) 0x7f32980555d0 con 0x7f32a80cc0d0 2023-10-06T05:23:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:48.660 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6557,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1866061469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1866061469},{"type":"v1","addr":"172.21.15.31:6834","nonce":1866061469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7366,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/257113943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":257113943},{"type":"v1","addr":"172.21.15.31:6835","nonce":257113943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2082524951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2082524951},{"type":"v1","addr":"172.21.15.62:6834","nonce":2082524951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:29.615356+0000","modified":"2023-10-06T05:23:33.142272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7378":{"gid":7378,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/519548465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":519548465},{"type":"v1","addr":"172.21.15.62:6835","nonce":519548465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:23:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.660+0000 7f32af3ff700 1 -- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f328c0674d0 msgr2=0x7f328c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.660+0000 7f32af3ff700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f328c0674d0 0x7f328c069980 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f32a4006f00 tx=0x7f32a4008040 comp rx=0 tx=0).stop 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.660+0000 7f32af3ff700 1 -- 172.21.15.31:0/2870136484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a80cc0d0 msgr2=0x7f32a80b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.660+0000 7f32af3ff700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a80cc0d0 0x7f32a80b2e20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f329800ecb0 tx=0x7f32980091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.660+0000 7f32af3ff700 1 -- 172.21.15.31:0/2870136484 shutdown_connections 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.660+0000 7f32af3ff700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f328c0674d0 0x7f328c069980 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.660+0000 7f32af3ff700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f32a81595a0 0x7f32a815b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.660+0000 7f32af3ff700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32a8068bb0 0x7f32a8152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.660+0000 7f32af3ff700 1 --2- 172.21.15.31:0/2870136484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32a80cc0d0 0x7f32a80b2e20 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.661+0000 7f32af3ff700 1 -- 172.21.15.31:0/2870136484 >> 172.21.15.31:0/2870136484 conn(0x7f32a806b620 msgr2=0x7f32a8157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.661+0000 7f32af3ff700 1 -- 172.21.15.31:0/2870136484 shutdown_connections 2023-10-06T05:23:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.661+0000 7f32af3ff700 1 -- 172.21.15.31:0/2870136484 wait complete. 2023-10-06T05:23:48.664 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 76 2023-10-06T05:23:48.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.874+0000 7f4c2a3ff700 1 Processor -- start 2023-10-06T05:23:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.874+0000 7f4c2a3ff700 1 -- start start 2023-10-06T05:23:48.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.875+0000 7f4c2a3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4c241595a0 0x7f4c24159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.875+0000 7f4c2a3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c240c95b0 con 0x7f4c24068d80 2023-10-06T05:23:48.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.875+0000 7f4c2a3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c240c9890 con 0x7f4c240cc0d0 2023-10-06T05:23:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.875+0000 7f4c2a3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c240c9b70 con 0x7f4c241595a0 2023-10-06T05:23:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.875+0000 7f4c23fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4c241595a0 0x7f4c24159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.875+0000 7f4c2899c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4c24068d80 0x7f4c240c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36342/0 (socket says 172.21.15.31:36342) 2023-10-06T05:23:48.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.876+0000 7f4c2899c700 1 -- 172.21.15.31:0/4192920666 learned_addr learned my addr 172.21.15.31:0/4192920666 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:48.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.876+0000 7f4c23fff700 1 -- 172.21.15.31:0/4192920666 >> v1:172.21.15.62:6789/0 conn(0x7f4c240cc0d0 legacy=0x7f4c24068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:48.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.876+0000 7f4c23fff700 1 -- 172.21.15.31:0/4192920666 >> v1:172.21.15.31:6789/0 conn(0x7f4c24068d80 legacy=0x7f4c240c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.876+0000 7f4c23fff700 1 -- 172.21.15.31:0/4192920666 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240ca130 con 0x7f4c241595a0 2023-10-06T05:23:48.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.876+0000 7f4c22ffd700 1 -- 172.21.15.31:0/4192920666 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253122281 0 0) 0x7f4c240c95b0 con 0x7f4c24068d80 2023-10-06T05:23:48.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.876+0000 7f4c23fff700 1 --2- 172.21.15.31:0/4192920666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c241595a0 0x7f4c24159970 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009910 tx=0x7f4c140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f0cc6ffa3a1a36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.876+0000 7f4c22ffd700 1 -- 172.21.15.31:0/4192920666 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400b7e0 con 0x7f4c241595a0 2023-10-06T05:23:48.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.876+0000 7f4c22ffd700 1 -- 172.21.15.31:0/4192920666 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1400b940 con 0x7f4c241595a0 2023-10-06T05:23:48.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.876+0000 7f4c22ffd700 1 -- 172.21.15.31:0/4192920666 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400bce0 con 0x7f4c241595a0 2023-10-06T05:23:48.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.877+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/4192920666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c241595a0 msgr2=0x7f4c24159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.877+0000 7f4c2a3ff700 1 --2- 172.21.15.31:0/4192920666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c241595a0 0x7f4c24159970 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009910 tx=0x7f4c140091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.877+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/4192920666 shutdown_connections 2023-10-06T05:23:48.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.877+0000 7f4c2a3ff700 1 --2- 172.21.15.31:0/4192920666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c241595a0 0x7f4c24159970 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.877+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/4192920666 >> 172.21.15.31:0/4192920666 conn(0x7f4c2406b620 msgr2=0x7f4c2406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:48.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.877+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/4192920666 shutdown_connections 2023-10-06T05:23:48.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.877+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/4192920666 wait complete. 2023-10-06T05:23:48.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.879+0000 7f4c2a3ff700 1 Processor -- start 2023-10-06T05:23:48.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.879+0000 7f4c2a3ff700 1 -- start start 2023-10-06T05:23:48.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c2a3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c24068d80 0x7f4c24152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c2a3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c240cc0d0 0x7f4c240b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c2a3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c241595a0 0x7f4c240c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c2a3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24157e50 con 0x7f4c241595a0 2023-10-06T05:23:48.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c2a3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241530f0 con 0x7f4c24068d80 2023-10-06T05:23:48.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c2a3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24158080 con 0x7f4c240cc0d0 2023-10-06T05:23:48.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c2899c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c241595a0 0x7f4c240c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c24068d80 0x7f4c24152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c24068d80 0x7f4c24152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:57968/0 (socket says 172.21.15.31:57968) 2023-10-06T05:23:48.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c23fff700 1 -- 172.21.15.31:0/3134219660 learned_addr learned my addr 172.21.15.31:0/3134219660 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:48.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.880+0000 7f4c237fe700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c240cc0d0 0x7f4c240b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c23fff700 1 -- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c240cc0d0 msgr2=0x7f4c240b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c23fff700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c240cc0d0 0x7f4c240b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c23fff700 1 -- 172.21.15.31:0/3134219660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c241595a0 msgr2=0x7f4c240c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:48.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c23fff700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c241595a0 0x7f4c240c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:48.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c23fff700 1 -- 172.21.15.31:0/3134219660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c14009420 con 0x7f4c24068d80 2023-10-06T05:23:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c2899c700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c241595a0 0x7f4c240c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c23fff700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c24068d80 0x7f4c24152bb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4c140030c0 tx=0x7f4c14016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c0bfff700 1 -- 172.21.15.31:0/3134219660 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14011c90 con 0x7f4c24068d80 2023-10-06T05:23:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c0bfff700 1 -- 172.21.15.31:0/3134219660 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c14020070 con 0x7f4c24068d80 2023-10-06T05:23:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/3134219660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c24158360 con 0x7f4c24068d80 2023-10-06T05:23:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c0bfff700 1 -- 172.21.15.31:0/3134219660 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1401b450 con 0x7f4c24068d80 2023-10-06T05:23:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.881+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/3134219660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c24158920 con 0x7f4c24068d80 2023-10-06T05:23:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.882+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/3134219660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c240b2b20 con 0x7f4c24068d80 2023-10-06T05:23:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.884+0000 7f4c0bfff700 1 -- 172.21.15.31:0/3134219660 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f4c14022030 con 0x7f4c24068d80 2023-10-06T05:23:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.884+0000 7f4c0bfff700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4c100674d0 0x7f4c10069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.885+0000 7f4c0bfff700 1 -- 172.21.15.31:0/3134219660 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f4c140874d0 con 0x7f4c24068d80 2023-10-06T05:23:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.885+0000 7f4c237fe700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4c100674d0 0x7f4c10069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.885+0000 7f4c237fe700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4c100674d0 0x7f4c10069980 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c240688a0 tx=0x7f4c18007ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:48.889+0000 7f4c0bfff700 1 -- 172.21.15.31:0/3134219660 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c14053d90 con 0x7f4c24068d80 2023-10-06T05:23:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.092+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/3134219660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c240b2b20 con 0x7f4c24068d80 2023-10-06T05:23:49.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.094+0000 7f4c0bfff700 1 -- 172.21.15.31:0/3134219660 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16972 (secure 0 0 0) 0x7f4c140573b0 con 0x7f4c24068d80 2023-10-06T05:23:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:49.096 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":114,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:23:31.123716+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:27.942469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:28.591817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:49.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4c100674d0 msgr2=0x7f4c10069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4c100674d0 0x7f4c10069980 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c240688a0 tx=0x7f4c18007ea0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c24068d80 msgr2=0x7f4c24152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c24068d80 0x7f4c24152bb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4c140030c0 tx=0x7f4c14016000 comp rx=0 tx=0).stop 2023-10-06T05:23:49.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/3134219660 shutdown_connections 2023-10-06T05:23:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4c100674d0 0x7f4c10069980 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c241595a0 0x7f4c240c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c24068d80 0x7f4c24152bb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 --2- 172.21.15.31:0/3134219660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c240cc0d0 0x7f4c240b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/3134219660 >> 172.21.15.31:0/3134219660 conn(0x7f4c2406b620 msgr2=0x7f4c240c3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/3134219660 shutdown_connections 2023-10-06T05:23:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.096+0000 7f4c2a3ff700 1 -- 172.21.15.31:0/3134219660 wait complete. 2023-10-06T05:23:49.117 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:23:27.942469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:23:28.591817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:23:49.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:23:49.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.320+0000 7f53aa5ff700 1 Processor -- start 2023-10-06T05:23:49.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.320+0000 7f53aa5ff700 1 -- start start 2023-10-06T05:23:49.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.321+0000 7f53aa5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53a4153d40 0x7f53a40b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.321+0000 7f53aa5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a40b09c0 0x7f53a40af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.321+0000 7f53aa5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a40ad690 con 0x7f53a40af6c0 2023-10-06T05:23:49.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.321+0000 7f53aa5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a40b0e10 con 0x7f53a4153d40 2023-10-06T05:23:49.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.321+0000 7f53aa5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a40acd60 con 0x7f53a40b09c0 2023-10-06T05:23:49.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.322+0000 7f53a3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53a4153d40 0x7f53a40b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.322+0000 7f53a3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53a4153d40 0x7f53a40b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54862/0 (socket says 172.21.15.31:54862) 2023-10-06T05:23:49.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.322+0000 7f53a3fff700 1 -- 172.21.15.31:0/3159814315 learned_addr learned my addr 172.21.15.31:0/3159814315 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:49.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.322+0000 7f53a37fe700 1 --2- 172.21.15.31:0/3159814315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a40b09c0 0x7f53a40af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.322+0000 7f53a3fff700 1 -- 172.21.15.31:0/3159814315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a40b09c0 msgr2=0x7f53a40af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:49.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.322+0000 7f53a3fff700 1 --2- 172.21.15.31:0/3159814315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a40b09c0 0x7f53a40af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.322+0000 7f53a3fff700 1 -- 172.21.15.31:0/3159814315 >> v1:172.21.15.62:6789/0 conn(0x7f53a40af6c0 legacy=0x7f53a4159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.322+0000 7f53a3fff700 1 -- 172.21.15.31:0/3159814315 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a40ad040 con 0x7f53a4153d40 2023-10-06T05:23:49.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.322+0000 7f53a37fe700 1 --2- 172.21.15.31:0/3159814315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a40b09c0 0x7f53a40af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:49.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.322+0000 7f53a3fff700 1 --2- 172.21.15.31:0/3159814315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a4153d40 0x7f53a40b0480 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f53940099b0 tx=0x7f5394009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a464dfc923c23af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53a2ffd700 1 -- 172.21.15.31:0/3159814315 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400b860 con 0x7f53a4153d40 2023-10-06T05:23:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53a2ffd700 1 -- 172.21.15.31:0/3159814315 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400b9c0 con 0x7f53a4153d40 2023-10-06T05:23:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53a2ffd700 1 -- 172.21.15.31:0/3159814315 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400bd60 con 0x7f53a4153d40 2023-10-06T05:23:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53aa5ff700 1 -- 172.21.15.31:0/3159814315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a4153d40 msgr2=0x7f53a40b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53aa5ff700 1 --2- 172.21.15.31:0/3159814315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a4153d40 0x7f53a40b0480 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f53940099b0 tx=0x7f5394009250 comp rx=0 tx=0).stop 2023-10-06T05:23:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53aa5ff700 1 -- 172.21.15.31:0/3159814315 shutdown_connections 2023-10-06T05:23:49.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53aa5ff700 1 --2- 172.21.15.31:0/3159814315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a4153d40 0x7f53a40b0480 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53aa5ff700 1 --2- 172.21.15.31:0/3159814315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a40b09c0 0x7f53a40af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53aa5ff700 1 -- 172.21.15.31:0/3159814315 >> 172.21.15.31:0/3159814315 conn(0x7f53a4067560 msgr2=0x7f53a4067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53aa5ff700 1 -- 172.21.15.31:0/3159814315 shutdown_connections 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.323+0000 7f53aa5ff700 1 -- 172.21.15.31:0/3159814315 wait complete. 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.325+0000 7f53aa5ff700 1 Processor -- start 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.325+0000 7f53aa5ff700 1 -- start start 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53aa5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a40af6c0 0x7f53a413dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53aa5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a40b09c0 0x7f53a4150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53aa5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a4153d40 0x7f53a4151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53aa5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a40c97a0 con 0x7f53a40af6c0 2023-10-06T05:23:49.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53aa5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a413e130 con 0x7f53a4153d40 2023-10-06T05:23:49.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53aa5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41501c0 con 0x7f53a40b09c0 2023-10-06T05:23:49.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a40af6c0 0x7f53a413dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a40af6c0 0x7f53a413dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54866/0 (socket says 172.21.15.31:54866) 2023-10-06T05:23:49.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53a3fff700 1 -- 172.21.15.31:0/617977537 learned_addr learned my addr 172.21.15.31:0/617977537 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:49.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53a3fff700 1 -- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a40b09c0 msgr2=0x7f53a4150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53a3fff700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a40b09c0 0x7f53a4150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53a3fff700 1 -- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a4153d40 msgr2=0x7f53a4151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.326+0000 7f53a8b9c700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a4153d40 0x7f53a4151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.327+0000 7f53a3fff700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a4153d40 0x7f53a4151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.327+0000 7f53a3fff700 1 -- 172.21.15.31:0/617977537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940094f0 con 0x7f53a40af6c0 2023-10-06T05:23:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.327+0000 7f53a8b9c700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a4153d40 0x7f53a4151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.327+0000 7f53a3fff700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a40af6c0 0x7f53a413dbf0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f53940090b0 tx=0x7f5394017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.327+0000 7f538bfff700 1 -- 172.21.15.31:0/617977537 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394026480 con 0x7f53a40af6c0 2023-10-06T05:23:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.327+0000 7f538bfff700 1 -- 172.21.15.31:0/617977537 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539401f070 con 0x7f53a40af6c0 2023-10-06T05:23:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.327+0000 7f538bfff700 1 -- 172.21.15.31:0/617977537 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394026830 con 0x7f53a40af6c0 2023-10-06T05:23:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.327+0000 7f53aa5ff700 1 -- 172.21.15.31:0/617977537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a40c8e70 con 0x7f53a40af6c0 2023-10-06T05:23:49.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.327+0000 7f53aa5ff700 1 -- 172.21.15.31:0/617977537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a40c9430 con 0x7f53a40af6c0 2023-10-06T05:23:49.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.328+0000 7f53aa5ff700 1 -- 172.21.15.31:0/617977537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a404c380 con 0x7f53a40af6c0 2023-10-06T05:23:49.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.329+0000 7f538bfff700 1 -- 172.21.15.31:0/617977537 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f5394026ba0 con 0x7f53a40af6c0 2023-10-06T05:23:49.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.330+0000 7f538bfff700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53900674d0 0x7f5390069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.330+0000 7f538bfff700 1 -- 172.21.15.31:0/617977537 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6142+0+0 (secure 0 0 0) 0x7f5394021070 con 0x7f53a40af6c0 2023-10-06T05:23:49.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.330+0000 7f53a37fe700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53900674d0 0x7f5390069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.331+0000 7f53a37fe700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53900674d0 0x7f5390069980 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5398006fd0 tx=0x7f5398008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.335+0000 7f538bfff700 1 -- 172.21.15.31:0/617977537 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5394053810 con 0x7f53a40af6c0 2023-10-06T05:23:49.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.535+0000 7f53aa5ff700 1 -- 172.21.15.31:0/617977537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f53a40ad320 con 0x7f53a40af6c0 2023-10-06T05:23:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.536+0000 7f538bfff700 1 -- 172.21.15.31:0/617977537 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v115) v1 ==== 92+0+10 (secure 0 0 0) 0x7f539402b0a0 con 0x7f53a40af6c0 2023-10-06T05:23:49.538 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:23:49.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.538+0000 7f53aa5ff700 1 -- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53900674d0 msgr2=0x7f5390069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.538+0000 7f53aa5ff700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53900674d0 0x7f5390069980 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5398006fd0 tx=0x7f5398008040 comp rx=0 tx=0).stop 2023-10-06T05:23:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.538+0000 7f53aa5ff700 1 -- 172.21.15.31:0/617977537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a40af6c0 msgr2=0x7f53a413dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.538+0000 7f53aa5ff700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a40af6c0 0x7f53a413dbf0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f53940090b0 tx=0x7f5394017000 comp rx=0 tx=0).stop 2023-10-06T05:23:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.538+0000 7f53aa5ff700 1 -- 172.21.15.31:0/617977537 shutdown_connections 2023-10-06T05:23:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.538+0000 7f53aa5ff700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53900674d0 0x7f5390069980 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.538+0000 7f53aa5ff700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a4153d40 0x7f53a4151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.538+0000 7f53aa5ff700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a40af6c0 0x7f53a413dbf0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.538+0000 7f53aa5ff700 1 --2- 172.21.15.31:0/617977537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a40b09c0 0x7f53a4150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.538+0000 7f53aa5ff700 1 -- 172.21.15.31:0/617977537 >> 172.21.15.31:0/617977537 conn(0x7f53a4067560 msgr2=0x7f53a4154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.539+0000 7f53aa5ff700 1 -- 172.21.15.31:0/617977537 shutdown_connections 2023-10-06T05:23:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.539+0000 7f53aa5ff700 1 -- 172.21.15.31:0/617977537 wait complete. 2023-10-06T05:23:49.558 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:23:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.761+0000 7fdf381ff700 1 Processor -- start 2023-10-06T05:23:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.761+0000 7fdf381ff700 1 -- start start 2023-10-06T05:23:49.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.762+0000 7fdf381ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf300c8f40 0x7fdf30068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.762+0000 7fdf381ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdf3006a300 0x7fdf300adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.762+0000 7fdf381ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf30068da0 con 0x7fdf3015ca40 2023-10-06T05:23:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.762+0000 7fdf381ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf300cabc0 con 0x7fdf3006a300 2023-10-06T05:23:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.762+0000 7fdf381ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30068bb0 con 0x7fdf300c8f40 2023-10-06T05:23:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.763+0000 7fdf3679c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdf3006a300 0x7fdf300adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.763+0000 7fdf3579a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf300c8f40 0x7fdf30068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.763+0000 7fdf3679c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdf3006a300 0x7fdf300adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:57986/0 (socket says 172.21.15.31:57986) 2023-10-06T05:23:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.763+0000 7fdf3579a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf300c8f40 0x7fdf30068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:47806/0 (socket says 172.21.15.31:47806) 2023-10-06T05:23:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.763+0000 7fdf3679c700 1 -- 172.21.15.31:0/2315321857 learned_addr learned my addr 172.21.15.31:0/2315321857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:49.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.763+0000 7fdf3579a700 1 -- 172.21.15.31:0/2315321857 >> v2:172.21.15.62:3300/0 conn(0x7fdf3006a300 msgr2=0x7fdf300adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:49.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.763+0000 7fdf3579a700 1 --2- 172.21.15.31:0/2315321857 >> v2:172.21.15.62:3300/0 conn(0x7fdf3006a300 0x7fdf300adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.763+0000 7fdf3579a700 1 -- 172.21.15.31:0/2315321857 >> v1:172.21.15.62:6789/0 conn(0x7fdf3015ca40 legacy=0x7fdf300c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.763+0000 7fdf3579a700 1 -- 172.21.15.31:0/2315321857 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30069080 con 0x7fdf300c8f40 2023-10-06T05:23:49.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.763+0000 7fdf3679c700 1 --2- 172.21.15.31:0/2315321857 >> v2:172.21.15.62:3300/0 conn(0x7fdf3006a300 0x7fdf300adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:49.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.764+0000 7fdf3579a700 1 --2- 172.21.15.31:0/2315321857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf300c8f40 0x7fdf30068670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf200098e0 tx=0x7fdf200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31b5c8ea64967c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.764+0000 7fdf34f99700 1 -- 172.21.15.31:0/2315321857 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2000b640 con 0x7fdf300c8f40 2023-10-06T05:23:49.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.764+0000 7fdf34f99700 1 -- 172.21.15.31:0/2315321857 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2000b7a0 con 0x7fdf300c8f40 2023-10-06T05:23:49.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.764+0000 7fdf34f99700 1 -- 172.21.15.31:0/2315321857 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2000bb40 con 0x7fdf300c8f40 2023-10-06T05:23:49.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.764+0000 7fdf381ff700 1 -- 172.21.15.31:0/2315321857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf300c8f40 msgr2=0x7fdf30068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.764+0000 7fdf381ff700 1 --2- 172.21.15.31:0/2315321857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf300c8f40 0x7fdf30068670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf200098e0 tx=0x7fdf200091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.765+0000 7fdf381ff700 1 -- 172.21.15.31:0/2315321857 shutdown_connections 2023-10-06T05:23:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.765+0000 7fdf381ff700 1 --2- 172.21.15.31:0/2315321857 >> v2:172.21.15.62:3300/0 conn(0x7fdf3006a300 0x7fdf300adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.765+0000 7fdf381ff700 1 --2- 172.21.15.31:0/2315321857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf300c8f40 0x7fdf30068670 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.765+0000 7fdf381ff700 1 -- 172.21.15.31:0/2315321857 >> 172.21.15.31:0/2315321857 conn(0x7fdf300ce290 msgr2=0x7fdf300ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.765+0000 7fdf381ff700 1 -- 172.21.15.31:0/2315321857 shutdown_connections 2023-10-06T05:23:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.765+0000 7fdf381ff700 1 -- 172.21.15.31:0/2315321857 wait complete. 2023-10-06T05:23:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.766+0000 7fdf381ff700 1 Processor -- start 2023-10-06T05:23:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.766+0000 7fdf381ff700 1 -- start start 2023-10-06T05:23:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.767+0000 7fdf381ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf3006a300 0x7fdf301507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.767+0000 7fdf381ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf300c8f40 0x7fdf3016a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.767+0000 7fdf381ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf3015ca40 0x7fdf300c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.767+0000 7fdf381ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30150d00 con 0x7fdf3006a300 2023-10-06T05:23:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.767+0000 7fdf381ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300c6470 con 0x7fdf300c8f40 2023-10-06T05:23:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.767+0000 7fdf381ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300c6900 con 0x7fdf3015ca40 2023-10-06T05:23:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.767+0000 7fdf35f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf3006a300 0x7fdf301507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf35f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf3006a300 0x7fdf301507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54878/0 (socket says 172.21.15.31:54878) 2023-10-06T05:23:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf35f9b700 1 -- 172.21.15.31:0/595497417 learned_addr learned my addr 172.21.15.31:0/595497417 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf3579a700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf300c8f40 0x7fdf3016a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf3679c700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf3015ca40 0x7fdf300c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf35f9b700 1 -- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf3015ca40 msgr2=0x7fdf300c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf35f9b700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf3015ca40 0x7fdf300c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf35f9b700 1 -- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf300c8f40 msgr2=0x7fdf3016a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf35f9b700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf300c8f40 0x7fdf3016a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf35f9b700 1 -- 172.21.15.31:0/595497417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28009ae0 con 0x7fdf3006a300 2023-10-06T05:23:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf3679c700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf3015ca40 0x7fdf300c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf35f9b700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf3006a300 0x7fdf301507c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c590 tx=0x7fdf2c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf25ffb700 1 -- 172.21.15.31:0/595497417 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c011590 con 0x7fdf3006a300 2023-10-06T05:23:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf25ffb700 1 -- 172.21.15.31:0/595497417 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c0116f0 con 0x7fdf3006a300 2023-10-06T05:23:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.768+0000 7fdf381ff700 1 -- 172.21.15.31:0/595497417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf20009420 con 0x7fdf3006a300 2023-10-06T05:23:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.769+0000 7fdf25ffb700 1 -- 172.21.15.31:0/595497417 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c011ab0 con 0x7fdf3006a300 2023-10-06T05:23:49.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.769+0000 7fdf381ff700 1 -- 172.21.15.31:0/595497417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf300c5b40 con 0x7fdf3006a300 2023-10-06T05:23:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.770+0000 7fdf381ff700 1 -- 172.21.15.31:0/595497417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf300c6750 con 0x7fdf3006a300 2023-10-06T05:23:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.771+0000 7fdf25ffb700 1 -- 172.21.15.31:0/595497417 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fdf2c01b3b0 con 0x7fdf3006a300 2023-10-06T05:23:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.771+0000 7fdf25ffb700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdf140674d0 0x7fdf14069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.772+0000 7fdf25ffb700 1 -- 172.21.15.31:0/595497417 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6142+0+0 (secure 0 0 0) 0x7fdf2c084720 con 0x7fdf3006a300 2023-10-06T05:23:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.772+0000 7fdf3579a700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdf140674d0 0x7fdf14069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.773+0000 7fdf3579a700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdf140674d0 0x7fdf14069980 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf200093f0 tx=0x7fdf20017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:49.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.776+0000 7fdf25ffb700 1 -- 172.21.15.31:0/595497417 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf2c050f30 con 0x7fdf3006a300 2023-10-06T05:23:49.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.979+0000 7fdf381ff700 1 -- 172.21.15.31:0/595497417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf300cef10 con 0x7fdf3006a300 2023-10-06T05:23:49.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.979+0000 7fdf25ffb700 1 -- 172.21.15.31:0/595497417 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf2c054550 con 0x7fdf3006a300 2023-10-06T05:23:49.981 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 -- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdf140674d0 msgr2=0x7fdf14069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdf140674d0 0x7fdf14069980 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf200093f0 tx=0x7fdf20017000 comp rx=0 tx=0).stop 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 -- 172.21.15.31:0/595497417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf3006a300 msgr2=0x7fdf301507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf3006a300 0x7fdf301507c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c590 tx=0x7fdf2c00c950 comp rx=0 tx=0).stop 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 -- 172.21.15.31:0/595497417 shutdown_connections 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdf140674d0 0x7fdf14069980 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf3015ca40 0x7fdf300c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf3006a300 0x7fdf301507c0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 --2- 172.21.15.31:0/595497417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf300c8f40 0x7fdf3016a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 -- 172.21.15.31:0/595497417 >> 172.21.15.31:0/595497417 conn(0x7fdf300ce290 msgr2=0x7fdf300cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:49.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 -- 172.21.15.31:0/595497417 shutdown_connections 2023-10-06T05:23:49.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:49.981+0000 7fdf381ff700 1 -- 172.21.15.31:0/595497417 wait complete. 2023-10-06T05:23:50.000 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:23:50.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.201+0000 7f76617ff700 1 Processor -- start 2023-10-06T05:23:50.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.201+0000 7f76617ff700 1 -- start start 2023-10-06T05:23:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.202+0000 7f76617ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c8f40 0x7f765c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.202+0000 7f76617ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f765c06a300 0x7f765c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.202+0000 7f76617ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765c068da0 con 0x7f765c15ca40 2023-10-06T05:23:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.202+0000 7f76617ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c0cabc0 con 0x7f765c06a300 2023-10-06T05:23:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.202+0000 7f76617ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c068bb0 con 0x7f765c0c8f40 2023-10-06T05:23:50.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.202+0000 7f765affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f765c15ca40 0x7f765c0c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36356/0 (socket says 172.21.15.31:36356) 2023-10-06T05:23:50.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.202+0000 7f765affd700 1 -- 172.21.15.31:0/2826323908 learned_addr learned my addr 172.21.15.31:0/2826323908 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:50.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230021166 0 0) 0x7f765c068da0 con 0x7f765c15ca40 2023-10-06T05:23:50.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f765b7fe700 1 --2- 172.21.15.31:0/2826323908 >> v2:172.21.15.62:3300/0 conn(0x7f765c06a300 0x7f765c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f765a7fc700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c8f40 0x7f765c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7644003570 con 0x7f765c15ca40 2023-10-06T05:23:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3464147625 0 0) 0x7f7644003570 con 0x7f765c15ca40 2023-10-06T05:23:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c8f40 msgr2=0x7f765c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c8f40 0x7f765c068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 >> v2:172.21.15.62:3300/0 conn(0x7f765c06a300 msgr2=0x7f765c0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 --2- 172.21.15.31:0/2826323908 >> v2:172.21.15.62:3300/0 conn(0x7f765c06a300 0x7f765c0adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c069080 con 0x7f765c15ca40 2023-10-06T05:23:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f764c001470 con 0x7f765c15ca40 2023-10-06T05:23:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 >> v1:172.21.15.31:6789/0 conn(0x7f765c15ca40 legacy=0x7f765c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c8f40 0x7f7644003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644003fe0 0x7f7644004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 0x7f76440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c0cabc0 con 0x7f7644004d50 2023-10-06T05:23:50.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764400ba60 con 0x7f7644003fe0 2023-10-06T05:23:50.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.203+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764400bbd0 con 0x7f765c0c8f40 2023-10-06T05:23:50.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f765b7fe700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 0x7f76440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f765a7fc700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644003fe0 0x7f7644004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f765affd700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c8f40 0x7f7644003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f765b7fe700 1 -- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c8f40 msgr2=0x7f7644003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:50.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f765b7fe700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c8f40 0x7f7644003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f765b7fe700 1 -- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644003fe0 msgr2=0x7f7644004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:50.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f765b7fe700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644003fe0 0x7f7644004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f765b7fe700 1 -- 172.21.15.31:0/2826323908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7644003570 con 0x7f7644004d50 2023-10-06T05:23:50.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f765a7fc700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644003fe0 0x7f7644004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:50.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f765b7fe700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 0x7f76440051c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f765400e210 tx=0x7f765400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455718271c8a3dff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:50.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.204+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765400ea60 con 0x7f7644004d50 2023-10-06T05:23:50.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.205+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765400f040 con 0x7f7644004d50 2023-10-06T05:23:50.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.205+0000 7f7659ffb700 1 -- 172.21.15.31:0/2826323908 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765400ee10 con 0x7f7644004d50 2023-10-06T05:23:50.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.205+0000 7f76617ff700 1 -- 172.21.15.31:0/2826323908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 msgr2=0x7f76440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.205+0000 7f76617ff700 1 --2- 172.21.15.31:0/2826323908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 0x7f76440051c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f765400e210 tx=0x7f765400b040 comp rx=0 tx=0).stop 2023-10-06T05:23:50.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.205+0000 7f765b7fe700 1 -- 172.21.15.31:0/2826323908 reap_dead start 2023-10-06T05:23:50.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.205+0000 7f76617ff700 1 -- 172.21.15.31:0/2826323908 shutdown_connections 2023-10-06T05:23:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.205+0000 7f76617ff700 1 -- 172.21.15.31:0/2826323908 >> 172.21.15.31:0/2826323908 conn(0x7f765c0ce290 msgr2=0x7f765c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.205+0000 7f76617ff700 1 -- 172.21.15.31:0/2826323908 shutdown_connections 2023-10-06T05:23:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.205+0000 7f76617ff700 1 -- 172.21.15.31:0/2826323908 wait complete. 2023-10-06T05:23:50.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.207+0000 7f76617ff700 1 Processor -- start 2023-10-06T05:23:50.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.208+0000 7f76617ff700 1 -- start start 2023-10-06T05:23:50.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f76617ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 0x7f765c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f76617ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765c15ad50 0x7f765c0c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f76617ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c4380 0x7f765c0adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f76617ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c0c47f0 con 0x7f7644004d50 2023-10-06T05:23:50.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f76617ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c15ce10 con 0x7f765c15ad50 2023-10-06T05:23:50.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f76617ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c15dc70 con 0x7f765c0c4380 2023-10-06T05:23:50.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f765affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 0x7f765c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f765affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 0x7f765c0afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54896/0 (socket says 172.21.15.31:54896) 2023-10-06T05:23:50.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f765affd700 1 -- 172.21.15.31:0/372222633 learned_addr learned my addr 172.21.15.31:0/372222633 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:50.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f765a7fc700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765c15ad50 0x7f765c0c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.209+0000 7f765b7fe700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c4380 0x7f765c0adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f765affd700 1 -- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c4380 msgr2=0x7f765c0adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f765affd700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c4380 0x7f765c0adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f765affd700 1 -- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765c15ad50 msgr2=0x7f765c0c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f765affd700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765c15ad50 0x7f765c0c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f765affd700 1 -- 172.21.15.31:0/372222633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0c8460 con 0x7f7644004d50 2023-10-06T05:23:50.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f765affd700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 0x7f765c0afda0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f764c0079e0 tx=0x7f764c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:50.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f7642ffd700 1 -- 172.21.15.31:0/372222633 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c018030 con 0x7f7644004d50 2023-10-06T05:23:50.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f7642ffd700 1 -- 172.21.15.31:0/372222633 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c00f8a0 con 0x7f7644004d50 2023-10-06T05:23:50.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f76617ff700 1 -- 172.21.15.31:0/372222633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c15d060 con 0x7f7644004d50 2023-10-06T05:23:50.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f7642ffd700 1 -- 172.21.15.31:0/372222633 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00fc60 con 0x7f7644004d50 2023-10-06T05:23:50.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.210+0000 7f76617ff700 1 -- 172.21.15.31:0/372222633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c15d620 con 0x7f7644004d50 2023-10-06T05:23:50.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.211+0000 7f76617ff700 1 -- 172.21.15.31:0/372222633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c04c380 con 0x7f7644004d50 2023-10-06T05:23:50.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.212+0000 7f7642ffd700 1 -- 172.21.15.31:0/372222633 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f764c01b070 con 0x7f7644004d50 2023-10-06T05:23:50.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.213+0000 7f7642ffd700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7648067550 0x7f7648069a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.213+0000 7f7642ffd700 1 -- 172.21.15.31:0/372222633 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6142+0+0 (secure 0 0 0) 0x7f764c019070 con 0x7f7644004d50 2023-10-06T05:23:50.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.214+0000 7f765a7fc700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7648067550 0x7f7648069a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.214+0000 7f765a7fc700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7648067550 0x7f7648069a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f765c0af410 tx=0x7f76500091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:50.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.217+0000 7f7642ffd700 1 -- 172.21.15.31:0/372222633 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f764c0518a0 con 0x7f7644004d50 2023-10-06T05:23:50.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.421+0000 7f76617ff700 1 -- 172.21.15.31:0/372222633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f765c0ca1a0 con 0x7f7644004d50 2023-10-06T05:23:50.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.421+0000 7f7642ffd700 1 -- 172.21.15.31:0/372222633 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7f764c054ec0 con 0x7f7644004d50 2023-10-06T05:23:50.423 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:23:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.423+0000 7f76617ff700 1 -- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7648067550 msgr2=0x7f7648069a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.423+0000 7f76617ff700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7648067550 0x7f7648069a00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f765c0af410 tx=0x7f76500091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.423+0000 7f76617ff700 1 -- 172.21.15.31:0/372222633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 msgr2=0x7f765c0afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.423+0000 7f76617ff700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 0x7f765c0afda0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f764c0079e0 tx=0x7f764c001e90 comp rx=0 tx=0).stop 2023-10-06T05:23:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.424+0000 7f76617ff700 1 -- 172.21.15.31:0/372222633 shutdown_connections 2023-10-06T05:23:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.424+0000 7f76617ff700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7648067550 0x7f7648069a00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.424+0000 7f76617ff700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f765c0c4380 0x7f765c0adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.424+0000 7f76617ff700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7644004d50 0x7f765c0afda0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.424+0000 7f76617ff700 1 --2- 172.21.15.31:0/372222633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765c15ad50 0x7f765c0c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.424+0000 7f76617ff700 1 -- 172.21.15.31:0/372222633 >> 172.21.15.31:0/372222633 conn(0x7f765c0ce290 msgr2=0x7f765c06cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.424+0000 7f76617ff700 1 -- 172.21.15.31:0/372222633 shutdown_connections 2023-10-06T05:23:50.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.424+0000 7f76617ff700 1 -- 172.21.15.31:0/372222633 wait complete. 2023-10-06T05:23:50.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:23:50.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.640+0000 7fb29bbff700 1 Processor -- start 2023-10-06T05:23:50.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.640+0000 7fb29bbff700 1 -- start start 2023-10-06T05:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.641+0000 7fb29bbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb294159890 0x7fb2940c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.641+0000 7fb29bbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb294159c60 0x7fb294156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.641+0000 7fb29bbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb294156fa0 0x7fb294155430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.641+0000 7fb29bbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb294158190 con 0x7fb294159890 2023-10-06T05:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.641+0000 7fb29bbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb294157580 con 0x7fb294156fa0 2023-10-06T05:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.641+0000 7fb29bbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb294157860 con 0x7fb294159c60 2023-10-06T05:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.641+0000 7fb29999b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb294159890 0x7fb2940c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.641+0000 7fb29999b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb294159890 0x7fb2940c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54904/0 (socket says 172.21.15.31:54904) 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.641+0000 7fb29999b700 1 -- 172.21.15.31:0/1726299577 learned_addr learned my addr 172.21.15.31:0/1726299577 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb29919a700 1 --2- 172.21.15.31:0/1726299577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb294159c60 0x7fb294156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb29a19c700 1 --2- 172.21.15.31:0/1726299577 >> v2:172.21.15.62:3300/0 conn(0x7fb294156fa0 0x7fb294155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb29999b700 1 -- 172.21.15.31:0/1726299577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb294159c60 msgr2=0x7fb294156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb29999b700 1 --2- 172.21.15.31:0/1726299577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb294159c60 0x7fb294156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb29999b700 1 -- 172.21.15.31:0/1726299577 >> v2:172.21.15.62:3300/0 conn(0x7fb294156fa0 msgr2=0x7fb294155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb29999b700 1 --2- 172.21.15.31:0/1726299577 >> v2:172.21.15.62:3300/0 conn(0x7fb294156fa0 0x7fb294155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb29999b700 1 -- 172.21.15.31:0/1726299577 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294157b40 con 0x7fb294159890 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb29a19c700 1 --2- 172.21.15.31:0/1726299577 >> v2:172.21.15.62:3300/0 conn(0x7fb294156fa0 0x7fb294155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb29999b700 1 --2- 172.21.15.31:0/1726299577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 0x7fb2940c89e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb29000bc50 tx=0x7fb29000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dbe5d66c2f9ad1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb298999700 1 -- 172.21.15.31:0/1726299577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29000be60 con 0x7fb294159890 2023-10-06T05:23:50.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb298999700 1 -- 172.21.15.31:0/1726299577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29000ddc0 con 0x7fb294159890 2023-10-06T05:23:50.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.642+0000 7fb298999700 1 -- 172.21.15.31:0/1726299577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900196b0 con 0x7fb294159890 2023-10-06T05:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.643+0000 7fb29bbff700 1 -- 172.21.15.31:0/1726299577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 msgr2=0x7fb2940c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.643+0000 7fb29bbff700 1 --2- 172.21.15.31:0/1726299577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 0x7fb2940c89e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb29000bc50 tx=0x7fb29000b4f0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.643+0000 7fb29bbff700 1 -- 172.21.15.31:0/1726299577 shutdown_connections 2023-10-06T05:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.643+0000 7fb29bbff700 1 --2- 172.21.15.31:0/1726299577 >> v2:172.21.15.62:3300/0 conn(0x7fb294156fa0 0x7fb294155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.643+0000 7fb29bbff700 1 --2- 172.21.15.31:0/1726299577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 0x7fb2940c89e0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.643+0000 7fb29bbff700 1 --2- 172.21.15.31:0/1726299577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb294159c60 0x7fb294156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.643+0000 7fb29bbff700 1 -- 172.21.15.31:0/1726299577 >> 172.21.15.31:0/1726299577 conn(0x7fb294067560 msgr2=0x7fb294067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.643+0000 7fb29bbff700 1 -- 172.21.15.31:0/1726299577 shutdown_connections 2023-10-06T05:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.643+0000 7fb29bbff700 1 -- 172.21.15.31:0/1726299577 wait complete. 2023-10-06T05:23:50.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.645+0000 7fb29bbff700 1 Processor -- start 2023-10-06T05:23:50.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.645+0000 7fb29bbff700 1 -- start start 2023-10-06T05:23:50.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29bbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb294156fa0 0x7fb294148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29bbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 0x7fb2941658a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29bbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb294159c60 0x7fb294165de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29bbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2941531e0 con 0x7fb294159890 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29bbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29414bee0 con 0x7fb294159c60 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29bbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29414c1c0 con 0x7fb294156fa0 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 0x7fb2941658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29919a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 0x7fb2941658a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54912/0 (socket says 172.21.15.31:54912) 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29919a700 1 -- 172.21.15.31:0/2805060245 learned_addr learned my addr 172.21.15.31:0/2805060245 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29999b700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb294156fa0 0x7fb294148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29a19c700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb294159c60 0x7fb294165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29919a700 1 -- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb294156fa0 msgr2=0x7fb294148610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29919a700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb294156fa0 0x7fb294148610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29919a700 1 -- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb294159c60 msgr2=0x7fb294165de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29919a700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb294159c60 0x7fb294165de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.646+0000 7fb29919a700 1 -- 172.21.15.31:0/2805060245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29000b790 con 0x7fb294159890 2023-10-06T05:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.647+0000 7fb29919a700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 0x7fb2941658a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fb28400d860 tx=0x7fb284008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.647+0000 7fb289ffb700 1 -- 172.21.15.31:0/2805060245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2840125f0 con 0x7fb294159890 2023-10-06T05:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.647+0000 7fb289ffb700 1 -- 172.21.15.31:0/2805060245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb284012750 con 0x7fb294159890 2023-10-06T05:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.647+0000 7fb289ffb700 1 -- 172.21.15.31:0/2805060245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb284012b10 con 0x7fb294159890 2023-10-06T05:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.647+0000 7fb29bbff700 1 -- 172.21.15.31:0/2805060245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29414c4a0 con 0x7fb294159890 2023-10-06T05:23:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.647+0000 7fb29bbff700 1 -- 172.21.15.31:0/2805060245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb294148d10 con 0x7fb294159890 2023-10-06T05:23:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.648+0000 7fb29bbff700 1 -- 172.21.15.31:0/2805060245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb294153390 con 0x7fb294159890 2023-10-06T05:23:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.649+0000 7fb289ffb700 1 -- 172.21.15.31:0/2805060245 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb28401e460 con 0x7fb294159890 2023-10-06T05:23:50.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.650+0000 7fb289ffb700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb2780674d0 0x7fb278069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:50.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.651+0000 7fb289ffb700 1 -- 172.21.15.31:0/2805060245 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6142+0+0 (secure 0 0 0) 0x7fb284085500 con 0x7fb294159890 2023-10-06T05:23:50.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.651+0000 7fb29999b700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb2780674d0 0x7fb278069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:50.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.652+0000 7fb29999b700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb2780674d0 0x7fb278069980 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb29000bc50 tx=0x7fb29001b000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:50.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.655+0000 7fb289ffb700 1 -- 172.21.15.31:0/2805060245 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb294153390 con 0x7fb294159890 2023-10-06T05:23:50.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.857+0000 7fb29bbff700 1 -- 172.21.15.31:0/2805060245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb294148d10 con 0x7fb294159890 2023-10-06T05:23:50.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.858+0000 7fb289ffb700 1 -- 172.21.15.31:0/2805060245 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb294148d10 con 0x7fb294159890 2023-10-06T05:23:50.860 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:23:50.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 -- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb2780674d0 msgr2=0x7fb278069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb2780674d0 0x7fb278069980 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb29000bc50 tx=0x7fb29001b000 comp rx=0 tx=0).stop 2023-10-06T05:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 -- 172.21.15.31:0/2805060245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 msgr2=0x7fb2941658a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 0x7fb2941658a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fb28400d860 tx=0x7fb284008040 comp rx=0 tx=0).stop 2023-10-06T05:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 -- 172.21.15.31:0/2805060245 shutdown_connections 2023-10-06T05:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb2780674d0 0x7fb278069980 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb294159c60 0x7fb294165de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb294156fa0 0x7fb294148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 --2- 172.21.15.31:0/2805060245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb294159890 0x7fb2941658a0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:50.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 -- 172.21.15.31:0/2805060245 >> 172.21.15.31:0/2805060245 conn(0x7fb294067560 msgr2=0x7fb294155200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.860+0000 7fb29bbff700 1 -- 172.21.15.31:0/2805060245 shutdown_connections 2023-10-06T05:23:50.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:50.861+0000 7fb29bbff700 1 -- 172.21.15.31:0/2805060245 wait complete. 2023-10-06T05:23:50.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.073+0000 7f43905ff700 1 Processor -- start 2023-10-06T05:23:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.073+0000 7f43905ff700 1 -- start start 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f43905ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43880c7b60 0x7f4388069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f43905ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43880c7f30 0x7f43880aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f43905ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43880ca5b0 con 0x7f438806ef50 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f43905ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438806f740 con 0x7f43880c7b60 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f43905ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43880c9c80 con 0x7f43880c7f30 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f438e39b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43880c7b60 0x7f4388069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f438e39b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f43880c7b60 0x7f4388069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54916/0 (socket says 172.21.15.31:54916) 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f438e39b700 1 -- 172.21.15.31:0/3474488010 learned_addr learned my addr 172.21.15.31:0/3474488010 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f438db9a700 1 --2- 172.21.15.31:0/3474488010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43880c7f30 0x7f43880aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f438e39b700 1 -- 172.21.15.31:0/3474488010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43880c7f30 msgr2=0x7f43880aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:51.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.075+0000 7f438e39b700 1 --2- 172.21.15.31:0/3474488010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43880c7f30 0x7f43880aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.076+0000 7f438e39b700 1 -- 172.21.15.31:0/3474488010 >> v1:172.21.15.62:6789/0 conn(0x7f438806ef50 legacy=0x7f438813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.076+0000 7f438e39b700 1 -- 172.21.15.31:0/3474488010 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43880c9f60 con 0x7f43880c7b60 2023-10-06T05:23:51.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.076+0000 7f438db9a700 1 --2- 172.21.15.31:0/3474488010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43880c7f30 0x7f43880aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:51.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.076+0000 7f438e39b700 1 --2- 172.21.15.31:0/3474488010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7b60 0x7f4388069880 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f438400bc30 tx=0x7f438400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e681fb22a518fe0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.076+0000 7f438d399700 1 -- 172.21.15.31:0/3474488010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438400dc50 con 0x7f43880c7b60 2023-10-06T05:23:51.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.076+0000 7f438d399700 1 -- 172.21.15.31:0/3474488010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438400ddb0 con 0x7f43880c7b60 2023-10-06T05:23:51.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.076+0000 7f438d399700 1 -- 172.21.15.31:0/3474488010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384005c30 con 0x7f43880c7b60 2023-10-06T05:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.077+0000 7f43905ff700 1 -- 172.21.15.31:0/3474488010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7b60 msgr2=0x7f4388069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.077+0000 7f43905ff700 1 --2- 172.21.15.31:0/3474488010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7b60 0x7f4388069880 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f438400bc30 tx=0x7f438400b4c0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.077+0000 7f43905ff700 1 -- 172.21.15.31:0/3474488010 shutdown_connections 2023-10-06T05:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.077+0000 7f43905ff700 1 --2- 172.21.15.31:0/3474488010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7b60 0x7f4388069880 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.077+0000 7f43905ff700 1 --2- 172.21.15.31:0/3474488010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43880c7f30 0x7f43880aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.077+0000 7f43905ff700 1 -- 172.21.15.31:0/3474488010 >> 172.21.15.31:0/3474488010 conn(0x7f43880cc0b0 msgr2=0x7f43880cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.077+0000 7f43905ff700 1 -- 172.21.15.31:0/3474488010 shutdown_connections 2023-10-06T05:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.077+0000 7f43905ff700 1 -- 172.21.15.31:0/3474488010 wait complete. 2023-10-06T05:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.078+0000 7f43905ff700 1 Processor -- start 2023-10-06T05:23:51.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.078+0000 7f43905ff700 1 -- start start 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.079+0000 7f43905ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f438806ef50 0x7f43880c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.079+0000 7f43905ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43880c7b60 0x7f4388157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.079+0000 7f43905ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7f30 0x7f43880c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.079+0000 7f43905ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43880c4820 con 0x7f43880c7f30 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.079+0000 7f43905ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4388157590 con 0x7f43880c7b60 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.079+0000 7f43905ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43880c3c10 con 0x7f438806ef50 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438eb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7f30 0x7f43880c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438eb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7f30 0x7f43880c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54926/0 (socket says 172.21.15.31:54926) 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438eb9c700 1 -- 172.21.15.31:0/572550047 learned_addr learned my addr 172.21.15.31:0/572550047 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438e39b700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f438806ef50 0x7f43880c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438db9a700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43880c7b60 0x7f4388157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438eb9c700 1 -- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f438806ef50 msgr2=0x7f43880c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438eb9c700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f438806ef50 0x7f43880c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438eb9c700 1 -- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43880c7b60 msgr2=0x7f4388157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438eb9c700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43880c7b60 0x7f4388157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438eb9c700 1 -- 172.21.15.31:0/572550047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438400b720 con 0x7f43880c7f30 2023-10-06T05:23:51.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.080+0000 7f438eb9c700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7f30 0x7f43880c66b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f438000c840 tx=0x7f4380001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.081+0000 7f437e7fc700 1 -- 172.21.15.31:0/572550047 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380012630 con 0x7f43880c7f30 2023-10-06T05:23:51.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.081+0000 7f437e7fc700 1 -- 172.21.15.31:0/572550047 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43800127b0 con 0x7f43880c7f30 2023-10-06T05:23:51.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.081+0000 7f437e7fc700 1 -- 172.21.15.31:0/572550047 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380012b70 con 0x7f43880c7f30 2023-10-06T05:23:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.081+0000 7f43905ff700 1 -- 172.21.15.31:0/572550047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43880c3ef0 con 0x7f43880c7f30 2023-10-06T05:23:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.081+0000 7f43905ff700 1 -- 172.21.15.31:0/572550047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43880c44b0 con 0x7f43880c7f30 2023-10-06T05:23:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.082+0000 7f43905ff700 1 -- 172.21.15.31:0/572550047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438804c380 con 0x7f43880c7f30 2023-10-06T05:23:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.083+0000 7f437e7fc700 1 -- 172.21.15.31:0/572550047 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f438001a3a0 con 0x7f43880c7f30 2023-10-06T05:23:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.084+0000 7f437e7fc700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f436c0674d0 0x7f436c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.084+0000 7f437e7fc700 1 -- 172.21.15.31:0/572550047 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6142+0+0 (secure 0 0 0) 0x7f43800852f0 con 0x7f43880c7f30 2023-10-06T05:23:51.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.084+0000 7f438e39b700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f436c0674d0 0x7f436c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.085+0000 7f438e39b700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f436c0674d0 0x7f436c069980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f438400b500 tx=0x7f438401a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.087+0000 7f437e7fc700 1 -- 172.21.15.31:0/572550047 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4380051b00 con 0x7f43880c7f30 2023-10-06T05:23:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.335+0000 7f43905ff700 1 -- 172.21.15.31:0/572550047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f438806bc90 con 0x7f43880c7f30 2023-10-06T05:23:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.336+0000 7f437e7fc700 1 -- 172.21.15.31:0/572550047 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4593 (secure 0 0 0) 0x7f4380055120 con 0x7f43880c7f30 2023-10-06T05:23:51.338 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:51.338 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6557,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1866061469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1866061469},{"type":"v1","addr":"172.21.15.31:6834","nonce":1866061469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7366,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/257113943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":257113943},{"type":"v1","addr":"172.21.15.31:6835","nonce":257113943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2082524951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2082524951},{"type":"v1","addr":"172.21.15.62:6834","nonce":2082524951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:29.615356+0000","modified":"2023-10-06T05:23:33.142272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7378":{"gid":7378,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/519548465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":519548465},{"type":"v1","addr":"172.21.15.62:6835","nonce":519548465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:23:51.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 -- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f436c0674d0 msgr2=0x7f436c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f436c0674d0 0x7f436c069980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f438400b500 tx=0x7f438401a000 comp rx=0 tx=0).stop 2023-10-06T05:23:51.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 -- 172.21.15.31:0/572550047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7f30 msgr2=0x7f43880c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7f30 0x7f43880c66b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f438000c840 tx=0x7f4380001670 comp rx=0 tx=0).stop 2023-10-06T05:23:51.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 -- 172.21.15.31:0/572550047 shutdown_connections 2023-10-06T05:23:51.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f436c0674d0 0x7f436c069980 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43880c7f30 0x7f43880c66b0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f438806ef50 0x7f43880c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 --2- 172.21.15.31:0/572550047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43880c7b60 0x7f4388157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 -- 172.21.15.31:0/572550047 >> 172.21.15.31:0/572550047 conn(0x7f43880cc0b0 msgr2=0x7f438815adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 -- 172.21.15.31:0/572550047 shutdown_connections 2023-10-06T05:23:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.338+0000 7f43905ff700 1 -- 172.21.15.31:0/572550047 wait complete. 2023-10-06T05:23:51.342 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 76 2023-10-06T05:23:51.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.553+0000 7f4e437ff700 1 Processor -- start 2023-10-06T05:23:51.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.553+0000 7f4e437ff700 1 -- start start 2023-10-06T05:23:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.554+0000 7f4e437ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c068d80 0x7f4e3c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.554+0000 7f4e437ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4e3c0cc0d0 0x7f4e3c0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.554+0000 7f4e437ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3c0c95b0 con 0x7f4e3c1595a0 2023-10-06T05:23:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.554+0000 7f4e437ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e3c0c9890 con 0x7f4e3c0cc0d0 2023-10-06T05:23:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.554+0000 7f4e437ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0c9b70 con 0x7f4e3c068d80 2023-10-06T05:23:51.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.554+0000 7f4e4159b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4e3c1595a0 0x7f4e3c068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36360/0 (socket says 172.21.15.31:36360) 2023-10-06T05:23:51.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e4159b700 1 -- 172.21.15.31:0/1701580558 learned_addr learned my addr 172.21.15.31:0/1701580558 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:51.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204683798 0 0) 0x7f4e3c0c95b0 con 0x7f4e3c1595a0 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e41d9c700 1 --2- 172.21.15.31:0/1701580558 >> v2:172.21.15.62:3300/0 conn(0x7f4e3c0cc0d0 0x7f4e3c0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e40d9a700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c068d80 0x7f4e3c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e28003570 con 0x7f4e3c1595a0 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 625164268 0 0) 0x7f4e28003570 con 0x7f4e3c1595a0 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c068d80 msgr2=0x7f4e3c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e33fff700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c068d80 0x7f4e3c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 >> v2:172.21.15.62:3300/0 conn(0x7f4e3c0cc0d0 msgr2=0x7f4e3c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e33fff700 1 --2- 172.21.15.31:0/1701580558 >> v2:172.21.15.62:3300/0 conn(0x7f4e3c0cc0d0 0x7f4e3c0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c0ca130 con 0x7f4e3c1595a0 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f4e38001470 con 0x7f4e3c1595a0 2023-10-06T05:23:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.555+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 >> v1:172.21.15.31:6789/0 conn(0x7f4e3c1595a0 legacy=0x7f4e3c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e33fff700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e28003a60 0x7f4e28003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e33fff700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e28004810 0x7f4e28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e33fff700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c068d80 0x7f4e280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0c9890 con 0x7f4e28003a60 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0c9b70 con 0x7f4e28004810 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2800ba60 con 0x7f4e3c068d80 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e4159b700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e28003a60 0x7f4e28003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e40d9a700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e28004810 0x7f4e28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e41d9c700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c068d80 0x7f4e280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e4159b700 1 -- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c068d80 msgr2=0x7f4e280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e4159b700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c068d80 0x7f4e280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e4159b700 1 -- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e28004810 msgr2=0x7f4e28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e4159b700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e28004810 0x7f4e28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.556+0000 7f4e4159b700 1 -- 172.21.15.31:0/1701580558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e28003570 con 0x7f4e28003a60 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.557+0000 7f4e4159b700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e28003a60 0x7f4e28003e70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800c3f0 tx=0x7f4e38001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8192e92a21ca3498 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.557+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800cc80 con 0x7f4e28003a60 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.557+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3800ce20 con 0x7f4e28003a60 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.557+0000 7f4e33fff700 1 -- 172.21.15.31:0/1701580558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38005b70 con 0x7f4e28003a60 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.557+0000 7f4e437ff700 1 -- 172.21.15.31:0/1701580558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e28003a60 msgr2=0x7f4e28003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.557+0000 7f4e437ff700 1 --2- 172.21.15.31:0/1701580558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e28003a60 0x7f4e28003e70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800c3f0 tx=0x7f4e38001e90 comp rx=0 tx=0).stop 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.557+0000 7f4e41d9c700 1 -- 172.21.15.31:0/1701580558 reap_dead start 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.557+0000 7f4e437ff700 1 -- 172.21.15.31:0/1701580558 shutdown_connections 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.557+0000 7f4e437ff700 1 -- 172.21.15.31:0/1701580558 >> 172.21.15.31:0/1701580558 conn(0x7f4e3c06b620 msgr2=0x7f4e3c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.558+0000 7f4e437ff700 1 -- 172.21.15.31:0/1701580558 shutdown_connections 2023-10-06T05:23:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.558+0000 7f4e437ff700 1 -- 172.21.15.31:0/1701580558 wait complete. 2023-10-06T05:23:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.559+0000 7f4e437ff700 1 Processor -- start 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.559+0000 7f4e437ff700 1 -- start start 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.560+0000 7f4e437ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e28003a60 0x7f4e3c14d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.560+0000 7f4e437ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c1539f0 0x7f4e3c0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.560+0000 7f4e437ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e3c15cb60 0x7f4e3c0c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.560+0000 7f4e437ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c154570 con 0x7f4e3c15cb60 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.560+0000 7f4e437ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c15c6d0 con 0x7f4e28003a60 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.560+0000 7f4e437ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c153fb0 con 0x7f4e3c1539f0 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.560+0000 7f4e41d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e3c15cb60 0x7f4e3c0c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.560+0000 7f4e41d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e3c15cb60 0x7f4e3c0c3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54950/0 (socket says 172.21.15.31:54950) 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.560+0000 7f4e41d9c700 1 -- 172.21.15.31:0/455999271 learned_addr learned my addr 172.21.15.31:0/455999271 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e4159b700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e28003a60 0x7f4e3c14d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e41d9c700 1 -- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c1539f0 msgr2=0x7f4e3c0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e40d9a700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c1539f0 0x7f4e3c0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e41d9c700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c1539f0 0x7f4e3c0b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e41d9c700 1 -- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e28003a60 msgr2=0x7f4e3c14d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e41d9c700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e28003a60 0x7f4e3c14d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e41d9c700 1 -- 172.21.15.31:0/455999271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c154290 con 0x7f4e3c15cb60 2023-10-06T05:23:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e4159b700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e28003a60 0x7f4e3c14d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e40d9a700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c1539f0 0x7f4e3c0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e41d9c700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e3c15cb60 0x7f4e3c0c3d60 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c06a080 tx=0x7f4e3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e317fa700 1 -- 172.21.15.31:0/455999271 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400e040 con 0x7f4e3c15cb60 2023-10-06T05:23:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e317fa700 1 -- 172.21.15.31:0/455999271 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3400f040 con 0x7f4e3c15cb60 2023-10-06T05:23:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e437ff700 1 -- 172.21.15.31:0/455999271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c15c360 con 0x7f4e3c15cb60 2023-10-06T05:23:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e317fa700 1 -- 172.21.15.31:0/455999271 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400fe20 con 0x7f4e3c15cb60 2023-10-06T05:23:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.561+0000 7f4e437ff700 1 -- 172.21.15.31:0/455999271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c15cfd0 con 0x7f4e3c15cb60 2023-10-06T05:23:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.562+0000 7f4e437ff700 1 -- 172.21.15.31:0/455999271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3c04c380 con 0x7f4e3c15cb60 2023-10-06T05:23:51.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.563+0000 7f4e317fa700 1 -- 172.21.15.31:0/455999271 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f4e34015640 con 0x7f4e3c15cb60 2023-10-06T05:23:51.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.564+0000 7f4e317fa700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4e200674d0 0x7f4e20069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.564+0000 7f4e317fa700 1 -- 172.21.15.31:0/455999271 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6142+0+0 (secure 0 0 0) 0x7f4e34085bb0 con 0x7f4e3c15cb60 2023-10-06T05:23:51.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.565+0000 7f4e4159b700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4e200674d0 0x7f4e20069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.565+0000 7f4e4159b700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4e200674d0 0x7f4e20069980 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800c3f0 tx=0x7f4e38015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.569+0000 7f4e317fa700 1 -- 172.21.15.31:0/455999271 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e340522b0 con 0x7f4e3c15cb60 2023-10-06T05:23:51.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.770+0000 7f4e437ff700 1 -- 172.21.15.31:0/455999271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e3c15c810 con 0x7f4e3c15cb60 2023-10-06T05:23:51.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.771+0000 7f4e317fa700 1 -- 172.21.15.31:0/455999271 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f4e34019090 con 0x7f4e3c15cb60 2023-10-06T05:23:51.773 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:51.774 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":115,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:23:49.228370+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:27.942469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:28.591817+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:51.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 -- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4e200674d0 msgr2=0x7f4e20069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4e200674d0 0x7f4e20069980 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800c3f0 tx=0x7f4e38015000 comp rx=0 tx=0).stop 2023-10-06T05:23:51.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 -- 172.21.15.31:0/455999271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e3c15cb60 msgr2=0x7f4e3c0c3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e3c15cb60 0x7f4e3c0c3d60 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c06a080 tx=0x7f4e3400b040 comp rx=0 tx=0).stop 2023-10-06T05:23:51.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 -- 172.21.15.31:0/455999271 shutdown_connections 2023-10-06T05:23:51.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4e200674d0 0x7f4e20069980 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e3c15cb60 0x7f4e3c0c3d60 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e28003a60 0x7f4e3c14d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 --2- 172.21.15.31:0/455999271 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e3c1539f0 0x7f4e3c0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 -- 172.21.15.31:0/455999271 >> 172.21.15.31:0/455999271 conn(0x7f4e3c06b620 msgr2=0x7f4e3c15b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:51.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 -- 172.21.15.31:0/455999271 shutdown_connections 2023-10-06T05:23:51.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.774+0000 7f4e437ff700 1 -- 172.21.15.31:0/455999271 wait complete. 2023-10-06T05:23:51.793 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:23:51.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:51.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.988+0000 7f6a2b1ff700 1 Processor -- start 2023-10-06T05:23:51.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.988+0000 7f6a2b1ff700 1 -- start start 2023-10-06T05:23:51.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.989+0000 7f6a2b1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6a2413fb00 0x7f6a2406c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.989+0000 7f6a2b1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6a2413fed0 0x7f6a2406e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.989+0000 7f6a2b1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6a2406ea90 0x7f6a240cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.989+0000 7f6a2b1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2406ef30 con 0x7f6a2413fb00 2023-10-06T05:23:51.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.989+0000 7f6a2b1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2406be70 con 0x7f6a2406ea90 2023-10-06T05:23:51.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.989+0000 7f6a2b1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2406f130 con 0x7f6a2413fed0 2023-10-06T05:23:51.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.989+0000 7f6a28f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6a2413fb00 0x7f6a2406c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.989+0000 7f6a28f9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6a2413fb00 0x7f6a2406c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54966/0 (socket says 172.21.15.31:54966) 2023-10-06T05:23:51.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.989+0000 7f6a28f9b700 1 -- 172.21.15.31:0/1450951675 learned_addr learned my addr 172.21.15.31:0/1450951675 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:51.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.989+0000 7f6a2979c700 1 --2- 172.21.15.31:0/1450951675 >> v2:172.21.15.62:3300/0 conn(0x7f6a2406ea90 0x7f6a240cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.990+0000 7f6a23fff700 1 --2- 172.21.15.31:0/1450951675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6a2413fed0 0x7f6a2406e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.990+0000 7f6a28f9b700 1 -- 172.21.15.31:0/1450951675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6a2413fed0 msgr2=0x7f6a2406e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:51.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.990+0000 7f6a28f9b700 1 --2- 172.21.15.31:0/1450951675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6a2413fed0 0x7f6a2406e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.990+0000 7f6a28f9b700 1 -- 172.21.15.31:0/1450951675 >> v2:172.21.15.62:3300/0 conn(0x7f6a2406ea90 msgr2=0x7f6a240cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:51.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.990+0000 7f6a28f9b700 1 --2- 172.21.15.31:0/1450951675 >> v2:172.21.15.62:3300/0 conn(0x7f6a2406ea90 0x7f6a240cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.990+0000 7f6a28f9b700 1 -- 172.21.15.31:0/1450951675 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2406f410 con 0x7f6a2413fb00 2023-10-06T05:23:51.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.990+0000 7f6a28f9b700 1 --2- 172.21.15.31:0/1450951675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fb00 0x7f6a2406c300 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6a2406bde0 tx=0x7f6a18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbf298e01890f8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.990+0000 7f6a237fe700 1 -- 172.21.15.31:0/1450951675 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18009e70 con 0x7f6a2413fb00 2023-10-06T05:23:51.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.990+0000 7f6a237fe700 1 -- 172.21.15.31:0/1450951675 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1800b860 con 0x7f6a2413fb00 2023-10-06T05:23:51.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.990+0000 7f6a237fe700 1 -- 172.21.15.31:0/1450951675 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800bc00 con 0x7f6a2413fb00 2023-10-06T05:23:51.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.991+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/1450951675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fb00 msgr2=0x7f6a2406c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.991+0000 7f6a2b1ff700 1 --2- 172.21.15.31:0/1450951675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fb00 0x7f6a2406c300 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6a2406bde0 tx=0x7f6a18009250 comp rx=0 tx=0).stop 2023-10-06T05:23:51.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.991+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/1450951675 shutdown_connections 2023-10-06T05:23:51.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.991+0000 7f6a2b1ff700 1 --2- 172.21.15.31:0/1450951675 >> v2:172.21.15.62:3300/0 conn(0x7f6a2406ea90 0x7f6a240cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.991+0000 7f6a2b1ff700 1 --2- 172.21.15.31:0/1450951675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fb00 0x7f6a2406c300 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.991+0000 7f6a2b1ff700 1 --2- 172.21.15.31:0/1450951675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6a2413fed0 0x7f6a2406e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.991+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/1450951675 >> 172.21.15.31:0/1450951675 conn(0x7f6a240cdcb0 msgr2=0x7f6a240c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:51.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.991+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/1450951675 shutdown_connections 2023-10-06T05:23:51.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.991+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/1450951675 wait complete. 2023-10-06T05:23:51.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.992+0000 7f6a2b1ff700 1 Processor -- start 2023-10-06T05:23:51.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.993+0000 7f6a2b1ff700 1 -- start start 2023-10-06T05:23:51.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.993+0000 7f6a2b1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a2406ea90 0x7f6a241417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2b1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6a2413fb00 0x7f6a240b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2b1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fed0 0x7f6a240b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:51.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2b1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24169e10 con 0x7f6a2413fed0 2023-10-06T05:23:51.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2b1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2416a020 con 0x7f6a2406ea90 2023-10-06T05:23:51.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2b1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2416a300 con 0x7f6a2413fb00 2023-10-06T05:23:51.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fed0 0x7f6a240b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2979c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fed0 0x7f6a240b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54982/0 (socket says 172.21.15.31:54982) 2023-10-06T05:23:51.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2979c700 1 -- 172.21.15.31:0/2934724301 learned_addr learned my addr 172.21.15.31:0/2934724301 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:51.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a28f9b700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a2406ea90 0x7f6a241417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a23fff700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6a2413fb00 0x7f6a240b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:51.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2979c700 1 -- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6a2413fb00 msgr2=0x7f6a240b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2979c700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6a2413fb00 0x7f6a240b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2979c700 1 -- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a2406ea90 msgr2=0x7f6a241417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:51.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2979c700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a2406ea90 0x7f6a241417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:51.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2979c700 1 -- 172.21.15.31:0/2934724301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a180094f0 con 0x7f6a2413fed0 2023-10-06T05:23:51.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.994+0000 7f6a2979c700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fed0 0x7f6a240b21b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009df0 tx=0x7f6a1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:51.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.995+0000 7f6a20ff9700 1 -- 172.21.15.31:0/2934724301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c00cb50 con 0x7f6a2413fed0 2023-10-06T05:23:52.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.995+0000 7f6a20ff9700 1 -- 172.21.15.31:0/2934724301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c00f040 con 0x7f6a2413fed0 2023-10-06T05:23:52.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.995+0000 7f6a20ff9700 1 -- 172.21.15.31:0/2934724301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c014480 con 0x7f6a2413fed0 2023-10-06T05:23:52.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.995+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/2934724301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2416a5e0 con 0x7f6a2413fed0 2023-10-06T05:23:52.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.995+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/2934724301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2416aba0 con 0x7f6a2413fed0 2023-10-06T05:23:52.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.996+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/2934724301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2404c380 con 0x7f6a2413fed0 2023-10-06T05:23:52.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.997+0000 7f6a20ff9700 1 -- 172.21.15.31:0/2934724301 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f6a1c014830 con 0x7f6a2413fed0 2023-10-06T05:23:52.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.997+0000 7f6a20ff9700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6a080674d0 0x7f6a08069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.998+0000 7f6a20ff9700 1 -- 172.21.15.31:0/2934724301 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6142+0+0 (secure 0 0 0) 0x7f6a1c085be0 con 0x7f6a2413fed0 2023-10-06T05:23:52.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.998+0000 7f6a28f9b700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6a080674d0 0x7f6a08069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:51.999+0000 7f6a28f9b700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6a080674d0 0x7f6a08069980 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6a2406bde0 tx=0x7f6a18017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:52.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.002+0000 7f6a20ff9700 1 -- 172.21.15.31:0/2934724301 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a1c052320 con 0x7f6a2413fed0 2023-10-06T05:23:52.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.248+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/2934724301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a2415e010 con 0x7f6a2413fed0 2023-10-06T05:23:52.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.249+0000 7f6a20ff9700 1 -- 172.21.15.31:0/2934724301 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4593 (secure 0 0 0) 0x7f6a1c055940 con 0x7f6a2413fed0 2023-10-06T05:23:52.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:52.251 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6557,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1866061469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1866061469},{"type":"v1","addr":"172.21.15.31:6834","nonce":1866061469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7366,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/257113943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":257113943},{"type":"v1","addr":"172.21.15.31:6835","nonce":257113943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2082524951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2082524951},{"type":"v1","addr":"172.21.15.62:6834","nonce":2082524951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:29.615356+0000","modified":"2023-10-06T05:23:33.142272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7378":{"gid":7378,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/519548465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":519548465},{"type":"v1","addr":"172.21.15.62:6835","nonce":519548465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:23:52.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6a080674d0 msgr2=0x7f6a08069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6a080674d0 0x7f6a08069980 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6a2406bde0 tx=0x7f6a18017000 comp rx=0 tx=0).stop 2023-10-06T05:23:52.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/2934724301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fed0 msgr2=0x7f6a240b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fed0 0x7f6a240b21b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009df0 tx=0x7f6a1c00b040 comp rx=0 tx=0).stop 2023-10-06T05:23:52.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/2934724301 shutdown_connections 2023-10-06T05:23:52.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6a080674d0 0x7f6a08069980 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2413fed0 0x7f6a240b21b0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a2406ea90 0x7f6a241417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 --2- 172.21.15.31:0/2934724301 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6a2413fb00 0x7f6a240b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/2934724301 >> 172.21.15.31:0/2934724301 conn(0x7f6a240cdcb0 msgr2=0x7f6a24159080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:52.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/2934724301 shutdown_connections 2023-10-06T05:23:52.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.251+0000 7f6a2b1ff700 1 -- 172.21.15.31:0/2934724301 wait complete. 2023-10-06T05:23:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 76 2023-10-06T05:23:52.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.465+0000 7fd07fbff700 1 Processor -- start 2023-10-06T05:23:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.465+0000 7fd07fbff700 1 -- start start 2023-10-06T05:23:52.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.466+0000 7fd07fbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd07815a930 0x7fd0780cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.466+0000 7fd07fbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0780cb1b0 0x7fd0780ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.466+0000 7fd07fbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cd270 0x7fd0780ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.466+0000 7fd07fbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0780ce090 con 0x7fd0780cb1b0 2023-10-06T05:23:52.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.466+0000 7fd07fbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0780cb6f0 con 0x7fd07815a930 2023-10-06T05:23:52.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.466+0000 7fd07fbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780cb9d0 con 0x7fd0780cd270 2023-10-06T05:23:52.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.466+0000 7fd07d19a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0780cb1b0 0x7fd0780ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07d19a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0780cb1b0 0x7fd0780ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54988/0 (socket says 172.21.15.31:54988) 2023-10-06T05:23:52.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07d19a700 1 -- 172.21.15.31:0/1479681408 learned_addr learned my addr 172.21.15.31:0/1479681408 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:52.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07d99b700 1 --2- 172.21.15.31:0/1479681408 >> v2:172.21.15.62:3300/0 conn(0x7fd07815a930 0x7fd0780cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07e19c700 1 --2- 172.21.15.31:0/1479681408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cd270 0x7fd0780ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07d19a700 1 -- 172.21.15.31:0/1479681408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cd270 msgr2=0x7fd0780ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07d19a700 1 --2- 172.21.15.31:0/1479681408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cd270 0x7fd0780ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07d19a700 1 -- 172.21.15.31:0/1479681408 >> v2:172.21.15.62:3300/0 conn(0x7fd07815a930 msgr2=0x7fd0780cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07d19a700 1 --2- 172.21.15.31:0/1479681408 >> v2:172.21.15.62:3300/0 conn(0x7fd07815a930 0x7fd0780cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07d19a700 1 -- 172.21.15.31:0/1479681408 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780cbcb0 con 0x7fd0780cb1b0 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07e19c700 1 --2- 172.21.15.31:0/1479681408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cd270 0x7fd0780ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.467+0000 7fd07d19a700 1 --2- 172.21.15.31:0/1479681408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cb1b0 0x7fd0780ccd30 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd078088820 tx=0x7fd068009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc055338332317f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.468+0000 7fd07c999700 1 -- 172.21.15.31:0/1479681408 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068009e70 con 0x7fd0780cb1b0 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.468+0000 7fd07c999700 1 -- 172.21.15.31:0/1479681408 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06800b8f0 con 0x7fd0780cb1b0 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.468+0000 7fd07c999700 1 -- 172.21.15.31:0/1479681408 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800bc90 con 0x7fd0780cb1b0 2023-10-06T05:23:52.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.468+0000 7fd07fbff700 1 -- 172.21.15.31:0/1479681408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cb1b0 msgr2=0x7fd0780ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.468+0000 7fd07fbff700 1 --2- 172.21.15.31:0/1479681408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cb1b0 0x7fd0780ccd30 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd078088820 tx=0x7fd068009250 comp rx=0 tx=0).stop 2023-10-06T05:23:52.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.468+0000 7fd07fbff700 1 -- 172.21.15.31:0/1479681408 shutdown_connections 2023-10-06T05:23:52.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.468+0000 7fd07fbff700 1 --2- 172.21.15.31:0/1479681408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cd270 0x7fd0780ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.468+0000 7fd07fbff700 1 --2- 172.21.15.31:0/1479681408 >> v2:172.21.15.62:3300/0 conn(0x7fd07815a930 0x7fd0780cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.468+0000 7fd07fbff700 1 --2- 172.21.15.31:0/1479681408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cb1b0 0x7fd0780ccd30 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.468+0000 7fd07fbff700 1 -- 172.21.15.31:0/1479681408 >> 172.21.15.31:0/1479681408 conn(0x7fd078099720 msgr2=0x7fd07806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:52.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.469+0000 7fd07fbff700 1 -- 172.21.15.31:0/1479681408 shutdown_connections 2023-10-06T05:23:52.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.469+0000 7fd07fbff700 1 -- 172.21.15.31:0/1479681408 wait complete. 2023-10-06T05:23:52.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.470+0000 7fd07fbff700 1 Processor -- start 2023-10-06T05:23:52.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.470+0000 7fd07fbff700 1 -- start start 2023-10-06T05:23:52.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.471+0000 7fd07fbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cb1b0 0x7fd07809a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.471+0000 7fd07fbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cd270 0x7fd0780c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.471+0000 7fd07fbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd07815a930 0x7fd0780682b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.471+0000 7fd07fbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07809a6d0 con 0x7fd0780cd270 2023-10-06T05:23:52.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.471+0000 7fd07fbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780c48c0 con 0x7fd07815a930 2023-10-06T05:23:52.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.471+0000 7fd07fbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780c4ba0 con 0x7fd0780cb1b0 2023-10-06T05:23:52.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.471+0000 7fd07d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cd270 0x7fd0780c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.471+0000 7fd07d19a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cd270 0x7fd0780c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54996/0 (socket says 172.21.15.31:54996) 2023-10-06T05:23:52.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.471+0000 7fd07d19a700 1 -- 172.21.15.31:0/4133421457 learned_addr learned my addr 172.21.15.31:0/4133421457 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:52.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd07d99b700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cb1b0 0x7fd07809a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd07e19c700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd07815a930 0x7fd0780682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd07d19a700 1 -- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cb1b0 msgr2=0x7fd07809a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd07d19a700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cb1b0 0x7fd07809a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd07d19a700 1 -- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd07815a930 msgr2=0x7fd0780682b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd07d19a700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd07815a930 0x7fd0780682b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd07d19a700 1 -- 172.21.15.31:0/4133421457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0680094f0 con 0x7fd0780cd270 2023-10-06T05:23:52.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd07e19c700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd07815a930 0x7fd0780682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:52.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd07d19a700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cd270 0x7fd0780c5c40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd068009290 tx=0x7fd068017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:52.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd06dffb700 1 -- 172.21.15.31:0/4133421457 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068026480 con 0x7fd0780cd270 2023-10-06T05:23:52.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd06dffb700 1 -- 172.21.15.31:0/4133421457 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06801f070 con 0x7fd0780cd270 2023-10-06T05:23:52.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd06dffb700 1 -- 172.21.15.31:0/4133421457 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068026850 con 0x7fd0780cd270 2023-10-06T05:23:52.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.472+0000 7fd07fbff700 1 -- 172.21.15.31:0/4133421457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0780c4e80 con 0x7fd0780cd270 2023-10-06T05:23:52.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.473+0000 7fd07fbff700 1 -- 172.21.15.31:0/4133421457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0780c5440 con 0x7fd0780cd270 2023-10-06T05:23:52.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.473+0000 7fd07fbff700 1 -- 172.21.15.31:0/4133421457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07804c380 con 0x7fd0780cd270 2023-10-06T05:23:52.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.474+0000 7fd06dffb700 1 -- 172.21.15.31:0/4133421457 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fd06801a430 con 0x7fd0780cd270 2023-10-06T05:23:52.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.475+0000 7fd06dffb700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd05c0674d0 0x7fd05c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.475+0000 7fd06dffb700 1 -- 172.21.15.31:0/4133421457 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6142+0+0 (secure 0 0 0) 0x7fd068021070 con 0x7fd0780cd270 2023-10-06T05:23:52.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.476+0000 7fd07d99b700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd05c0674d0 0x7fd05c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.476+0000 7fd07d99b700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd05c0674d0 0x7fd05c069980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd074009c90 tx=0x7fd074008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:52.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.479+0000 7fd06dffb700 1 -- 172.21.15.31:0/4133421457 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd068053db0 con 0x7fd0780cd270 2023-10-06T05:23:52.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.727+0000 7fd07fbff700 1 -- 172.21.15.31:0/4133421457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0780cf200 con 0x7fd0780cd270 2023-10-06T05:23:52.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.728+0000 7fd06dffb700 1 -- 172.21.15.31:0/4133421457 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4593 (secure 0 0 0) 0x7fd0680573d0 con 0x7fd0780cd270 2023-10-06T05:23:52.730 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:52.730 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6557,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1866061469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1866061469},{"type":"v1","addr":"172.21.15.31:6834","nonce":1866061469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7366,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/257113943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":257113943},{"type":"v1","addr":"172.21.15.31:6835","nonce":257113943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2082524951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2082524951},{"type":"v1","addr":"172.21.15.62:6834","nonce":2082524951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:23:29.615356+0000","modified":"2023-10-06T05:23:33.142272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7378":{"gid":7378,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/519548465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":519548465},{"type":"v1","addr":"172.21.15.62:6835","nonce":519548465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T05:23:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 -- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd05c0674d0 msgr2=0x7fd05c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd05c0674d0 0x7fd05c069980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd074009c90 tx=0x7fd074008040 comp rx=0 tx=0).stop 2023-10-06T05:23:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 -- 172.21.15.31:0/4133421457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cd270 msgr2=0x7fd0780c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cd270 0x7fd0780c5c40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd068009290 tx=0x7fd068017000 comp rx=0 tx=0).stop 2023-10-06T05:23:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 -- 172.21.15.31:0/4133421457 shutdown_connections 2023-10-06T05:23:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd05c0674d0 0x7fd05c069980 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd07815a930 0x7fd0780682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd0780cb1b0 0x7fd07809a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 --2- 172.21.15.31:0/4133421457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780cd270 0x7fd0780c5c40 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 -- 172.21.15.31:0/4133421457 >> 172.21.15.31:0/4133421457 conn(0x7fd078099720 msgr2=0x7fd0780ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 -- 172.21.15.31:0/4133421457 shutdown_connections 2023-10-06T05:23:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.730+0000 7fd07fbff700 1 -- 172.21.15.31:0/4133421457 wait complete. 2023-10-06T05:23:52.733 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 76 2023-10-06T05:23:52.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:23:52.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.943+0000 7f7318dff700 1 Processor -- start 2023-10-06T05:23:52.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.943+0000 7f7318dff700 1 -- start start 2023-10-06T05:23:52.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.944+0000 7f7318dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7314159500 0x7f73140b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.944+0000 7f7318dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73140cd510 0x7f73140aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.944+0000 7f7318dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73140c3f40 con 0x7f731415d2d0 2023-10-06T05:23:52.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.944+0000 7f7318dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73140b2860 con 0x7f7314159500 2023-10-06T05:23:52.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.944+0000 7f7318dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731415d7d0 con 0x7f73140cd510 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.944+0000 7f731259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7314159500 0x7f73140b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.945+0000 7f731259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7314159500 0x7f73140b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55006/0 (socket says 172.21.15.31:55006) 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.945+0000 7f731259c700 1 -- 172.21.15.31:0/4037777487 learned_addr learned my addr 172.21.15.31:0/4037777487 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.945+0000 7f7312d9d700 1 --2- 172.21.15.31:0/4037777487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73140cd510 0x7f73140aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.945+0000 7f731259c700 1 -- 172.21.15.31:0/4037777487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73140cd510 msgr2=0x7f73140aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.945+0000 7f731259c700 1 --2- 172.21.15.31:0/4037777487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73140cd510 0x7f73140aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.945+0000 7f731259c700 1 -- 172.21.15.31:0/4037777487 >> v1:172.21.15.62:6789/0 conn(0x7f731415d2d0 legacy=0x7f73140cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.945+0000 7f731259c700 1 -- 172.21.15.31:0/4037777487 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73140cb000 con 0x7f7314159500 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.945+0000 7f7312d9d700 1 --2- 172.21.15.31:0/4037777487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73140cd510 0x7f73140aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.945+0000 7f731259c700 1 --2- 172.21.15.31:0/4037777487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 0x7f73140b0ab0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f73040099b0 tx=0x7f7304009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bf86f6836e70cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.945+0000 7f731159a700 1 -- 172.21.15.31:0/4037777487 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400b820 con 0x7f7314159500 2023-10-06T05:23:52.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.946+0000 7f731159a700 1 -- 172.21.15.31:0/4037777487 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730400b980 con 0x7f7314159500 2023-10-06T05:23:52.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.946+0000 7f731159a700 1 -- 172.21.15.31:0/4037777487 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400bd20 con 0x7f7314159500 2023-10-06T05:23:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.946+0000 7f7318dff700 1 -- 172.21.15.31:0/4037777487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 msgr2=0x7f73140b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.946+0000 7f7318dff700 1 --2- 172.21.15.31:0/4037777487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 0x7f73140b0ab0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f73040099b0 tx=0x7f7304009250 comp rx=0 tx=0).stop 2023-10-06T05:23:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.946+0000 7f7318dff700 1 -- 172.21.15.31:0/4037777487 shutdown_connections 2023-10-06T05:23:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.946+0000 7f7318dff700 1 --2- 172.21.15.31:0/4037777487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f73140cd510 0x7f73140aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.946+0000 7f7318dff700 1 --2- 172.21.15.31:0/4037777487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 0x7f73140b0ab0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.946+0000 7f7318dff700 1 -- 172.21.15.31:0/4037777487 >> 172.21.15.31:0/4037777487 conn(0x7f731406ec90 msgr2=0x7f731406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.946+0000 7f7318dff700 1 -- 172.21.15.31:0/4037777487 shutdown_connections 2023-10-06T05:23:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.946+0000 7f7318dff700 1 -- 172.21.15.31:0/4037777487 wait complete. 2023-10-06T05:23:52.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.948+0000 7f7318dff700 1 Processor -- start 2023-10-06T05:23:52.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.948+0000 7f7318dff700 1 -- start start 2023-10-06T05:23:52.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7318dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73140cd510 0x7f73141598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7318dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 0x7f7314150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7318dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f731415d2d0 0x7f73140cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7318dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73140c8550 con 0x7f7314159500 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7318dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731415e0c0 con 0x7f73140cd510 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7318dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731415e920 con 0x7f731415d2d0 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7311d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 0x7f7314150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7311d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 0x7f7314150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55022/0 (socket says 172.21.15.31:55022) 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7311d9b700 1 -- 172.21.15.31:0/2801758268 learned_addr learned my addr 172.21.15.31:0/2801758268 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f731259c700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73140cd510 0x7f73141598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7312d9d700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f731415d2d0 0x7f73140cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7311d9b700 1 -- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f731415d2d0 msgr2=0x7f73140cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7311d9b700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f731415d2d0 0x7f73140cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7311d9b700 1 -- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73140cd510 msgr2=0x7f73141598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7311d9b700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73140cd510 0x7f73141598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.949+0000 7f7311d9b700 1 -- 172.21.15.31:0/2801758268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73040094f0 con 0x7f7314159500 2023-10-06T05:23:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.950+0000 7f7311d9b700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 0x7f7314150a80 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f730800b5e0 tx=0x7f73080090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.950+0000 7f73027fc700 1 -- 172.21.15.31:0/2801758268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730800f510 con 0x7f7314159500 2023-10-06T05:23:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.950+0000 7f73027fc700 1 -- 172.21.15.31:0/2801758268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730800f670 con 0x7f7314159500 2023-10-06T05:23:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.950+0000 7f73027fc700 1 -- 172.21.15.31:0/2801758268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730800fa30 con 0x7f7314159500 2023-10-06T05:23:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.950+0000 7f7318dff700 1 -- 172.21.15.31:0/2801758268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73140c8270 con 0x7f7314159500 2023-10-06T05:23:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.950+0000 7f7318dff700 1 -- 172.21.15.31:0/2801758268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731415e5b0 con 0x7f7314159500 2023-10-06T05:23:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.951+0000 7f7318dff700 1 -- 172.21.15.31:0/2801758268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731415ead0 con 0x7f7314159500 2023-10-06T05:23:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.952+0000 7f73027fc700 1 -- 172.21.15.31:0/2801758268 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f730801a400 con 0x7f7314159500 2023-10-06T05:23:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.953+0000 7f73027fc700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72f40674d0 0x7f72f4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.953+0000 7f73027fc700 1 -- 172.21.15.31:0/2801758268 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6142+0+0 (secure 0 0 0) 0x7f7308084ab0 con 0x7f7314159500 2023-10-06T05:23:52.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.953+0000 7f731259c700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72f40674d0 0x7f72f4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.954+0000 7f731259c700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72f40674d0 0x7f72f4069980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f73040099b0 tx=0x7f7304016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:52.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:52.957+0000 7f73027fc700 1 -- 172.21.15.31:0/2801758268 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f731415ead0 con 0x7f7314159500 2023-10-06T05:23:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.159+0000 7f7318dff700 1 -- 172.21.15.31:0/2801758268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f731415ea90 con 0x7f7314159500 2023-10-06T05:23:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.160+0000 7f73027fc700 1 -- 172.21.15.31:0/2801758268 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f7308051340 con 0x7f7314159500 2023-10-06T05:23:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:53.163 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":115,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:23:49.228370+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:23:27.942469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T05:23:28.591817+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]},{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:23:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 -- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72f40674d0 msgr2=0x7f72f4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72f40674d0 0x7f72f4069980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f73040099b0 tx=0x7f7304016000 comp rx=0 tx=0).stop 2023-10-06T05:23:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 -- 172.21.15.31:0/2801758268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 msgr2=0x7f7314150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 0x7f7314150a80 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f730800b5e0 tx=0x7f73080090f0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 -- 172.21.15.31:0/2801758268 shutdown_connections 2023-10-06T05:23:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f72f40674d0 0x7f72f4069980 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f731415d2d0 0x7f73140cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73140cd510 0x7f73141598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 --2- 172.21.15.31:0/2801758268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7314159500 0x7f7314150a80 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 -- 172.21.15.31:0/2801758268 >> 172.21.15.31:0/2801758268 conn(0x7f731406ec90 msgr2=0x7f73140b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 -- 172.21.15.31:0/2801758268 shutdown_connections 2023-10-06T05:23:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.163+0000 7f7318dff700 1 -- 172.21.15.31:0/2801758268 wait complete. 2023-10-06T05:23:53.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:23:53.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.378+0000 7fddbddff700 1 Processor -- start 2023-10-06T05:23:53.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.378+0000 7fddbddff700 1 -- start start 2023-10-06T05:23:53.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.379+0000 7fddbddff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fddb80cd510 0x7fddb80b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.379+0000 7fddbddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb815d2d0 0x7fddb80aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.379+0000 7fddbddff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb80c3f40 con 0x7fddb8159500 2023-10-06T05:23:53.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.379+0000 7fddbddff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb80b2860 con 0x7fddb80cd510 2023-10-06T05:23:53.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.379+0000 7fddbddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb815d7d0 con 0x7fddb815d2d0 2023-10-06T05:23:53.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb6ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fddb80cd510 0x7fddb80b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb77fe700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fddb8159500 0x7fddb80cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:42668/0 (socket says 172.21.15.31:42668) 2023-10-06T05:23:53.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb815d2d0 0x7fddb80aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb77fe700 1 -- 172.21.15.31:0/687854282 learned_addr learned my addr 172.21.15.31:0/687854282 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb7fff700 1 -- 172.21.15.31:0/687854282 >> v2:172.21.15.62:3300/0 conn(0x7fddb80cd510 msgr2=0x7fddb80b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb67fc700 1 -- 172.21.15.31:0/687854282 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066718395 0 0) 0x7fddb80c3f40 con 0x7fddb8159500 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb7fff700 1 --2- 172.21.15.31:0/687854282 >> v2:172.21.15.62:3300/0 conn(0x7fddb80cd510 0x7fddb80b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb7fff700 1 -- 172.21.15.31:0/687854282 >> v1:172.21.15.62:6789/0 conn(0x7fddb8159500 legacy=0x7fddb80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb7fff700 1 -- 172.21.15.31:0/687854282 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb80cb000 con 0x7fddb815d2d0 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb6ffd700 1 --2- 172.21.15.31:0/687854282 >> v2:172.21.15.62:3300/0 conn(0x7fddb80cd510 0x7fddb80b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.380+0000 7fddb7fff700 1 --2- 172.21.15.31:0/687854282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb815d2d0 0x7fddb80aedd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009e60 tx=0x7fddb0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8afc18c12dc0539 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.381+0000 7fddb67fc700 1 -- 172.21.15.31:0/687854282 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000e070 con 0x7fddb815d2d0 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.381+0000 7fddb67fc700 1 -- 172.21.15.31:0/687854282 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb000bb60 con 0x7fddb815d2d0 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.381+0000 7fddb67fc700 1 -- 172.21.15.31:0/687854282 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0012430 con 0x7fddb815d2d0 2023-10-06T05:23:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.381+0000 7fddbddff700 1 -- 172.21.15.31:0/687854282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb815d2d0 msgr2=0x7fddb80aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.381+0000 7fddbddff700 1 --2- 172.21.15.31:0/687854282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb815d2d0 0x7fddb80aedd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009e60 tx=0x7fddb0009780 comp rx=0 tx=0).stop 2023-10-06T05:23:53.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.381+0000 7fddbddff700 1 -- 172.21.15.31:0/687854282 shutdown_connections 2023-10-06T05:23:53.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.381+0000 7fddbddff700 1 --2- 172.21.15.31:0/687854282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb815d2d0 0x7fddb80aedd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.381+0000 7fddbddff700 1 --2- 172.21.15.31:0/687854282 >> v2:172.21.15.62:3300/0 conn(0x7fddb80cd510 0x7fddb80b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.381+0000 7fddbddff700 1 -- 172.21.15.31:0/687854282 >> 172.21.15.31:0/687854282 conn(0x7fddb806ec90 msgr2=0x7fddb806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:53.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.382+0000 7fddbddff700 1 -- 172.21.15.31:0/687854282 shutdown_connections 2023-10-06T05:23:53.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.382+0000 7fddbddff700 1 -- 172.21.15.31:0/687854282 wait complete. 2023-10-06T05:23:53.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.383+0000 7fddbddff700 1 Processor -- start 2023-10-06T05:23:53.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.383+0000 7fddbddff700 1 -- start start 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddbddff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb80cd510 0x7fddb8147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddbddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb8159500 0x7fddb80aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddbddff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb815d2d0 0x7fddb80a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddbddff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb815acb0 con 0x7fddb815d2d0 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddbddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb815af90 con 0x7fddb8159500 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddbddff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb815b270 con 0x7fddb80cd510 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddb7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb815d2d0 0x7fddb80a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddb7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb815d2d0 0x7fddb80a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55028/0 (socket says 172.21.15.31:55028) 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddb7fff700 1 -- 172.21.15.31:0/3855524999 learned_addr learned my addr 172.21.15.31:0/3855524999 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddb77fe700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb80cd510 0x7fddb8147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.384+0000 7fddb6ffd700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb8159500 0x7fddb80aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.385+0000 7fddb7fff700 1 -- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb80cd510 msgr2=0x7fddb8147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.385+0000 7fddb7fff700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb80cd510 0x7fddb8147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.385+0000 7fddb7fff700 1 -- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb8159500 msgr2=0x7fddb80aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.385+0000 7fddb7fff700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb8159500 0x7fddb80aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.385+0000 7fddb7fff700 1 -- 172.21.15.31:0/3855524999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac0093c0 con 0x7fddb815d2d0 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.385+0000 7fddb7fff700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb815d2d0 0x7fddb80a6dc0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fddb815d910 tx=0x7fddb0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.385+0000 7fdd9f7fe700 1 -- 172.21.15.31:0/3855524999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0012ea0 con 0x7fddb815d2d0 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.385+0000 7fdd9f7fe700 1 -- 172.21.15.31:0/3855524999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb0027470 con 0x7fddb815d2d0 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.385+0000 7fddbddff700 1 -- 172.21.15.31:0/3855524999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb0009a20 con 0x7fddb815d2d0 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.385+0000 7fdd9f7fe700 1 -- 172.21.15.31:0/3855524999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0027830 con 0x7fddb815d2d0 2023-10-06T05:23:53.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.386+0000 7fddbddff700 1 -- 172.21.15.31:0/3855524999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb815b830 con 0x7fddb815d2d0 2023-10-06T05:23:53.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.387+0000 7fddbddff700 1 -- 172.21.15.31:0/3855524999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb815ba50 con 0x7fddb815d2d0 2023-10-06T05:23:53.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.387+0000 7fdd9f7fe700 1 -- 172.21.15.31:0/3855524999 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fddb0024070 con 0x7fddb815d2d0 2023-10-06T05:23:53.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.388+0000 7fdd9f7fe700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdda40674d0 0x7fdda4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.388+0000 7fdd9f7fe700 1 -- 172.21.15.31:0/3855524999 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6142+0+0 (secure 0 0 0) 0x7fddb0087800 con 0x7fddb815d2d0 2023-10-06T05:23:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.389+0000 7fddb77fe700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdda40674d0 0x7fdda4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.389+0000 7fddb77fe700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdda40674d0 0x7fdda4069980 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdda8007fd0 tx=0x7fdda8007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:53.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.392+0000 7fdd9f7fe700 1 -- 172.21.15.31:0/3855524999 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddb0054050 con 0x7fddb815d2d0 2023-10-06T05:23:53.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.640+0000 7fddbddff700 1 -- 172.21.15.31:0/3855524999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fddb804c380 con 0x7fddb815d2d0 2023-10-06T05:23:53.643 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:23:53.641+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:23:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.645+0000 7fdd9f7fe700 1 -- 172.21.15.31:0/3855524999 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) v1 ==== 161+0+0 (secure 0 0 0) 0x7fddb0057670 con 0x7fddb815d2d0 2023-10-06T05:23:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 -- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdda40674d0 msgr2=0x7fdda4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdda40674d0 0x7fdda4069980 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdda8007fd0 tx=0x7fdda8007d30 comp rx=0 tx=0).stop 2023-10-06T05:23:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 -- 172.21.15.31:0/3855524999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb815d2d0 msgr2=0x7fddb80a6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb815d2d0 0x7fddb80a6dc0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fddb815d910 tx=0x7fddb0019000 comp rx=0 tx=0).stop 2023-10-06T05:23:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 -- 172.21.15.31:0/3855524999 shutdown_connections 2023-10-06T05:23:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdda40674d0 0x7fdda4069980 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddb815d2d0 0x7fddb80a6dc0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddb80cd510 0x7fddb8147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 --2- 172.21.15.31:0/3855524999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb8159500 0x7fddb80aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 -- 172.21.15.31:0/3855524999 >> 172.21.15.31:0/3855524999 conn(0x7fddb806ec90 msgr2=0x7fddb80cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.648+0000 7fddbddff700 1 -- 172.21.15.31:0/3855524999 shutdown_connections 2023-10-06T05:23:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.649+0000 7fddbddff700 1 -- 172.21.15.31:0/3855524999 wait complete. 2023-10-06T05:23:53.652 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:23:53.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:23:53.717 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:23:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.867+0000 7f53a75ff700 1 Processor -- start 2023-10-06T05:23:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.867+0000 7f53a75ff700 1 -- start start 2023-10-06T05:23:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.868+0000 7f53a75ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f53a00ae860 0x7f53a0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.868+0000 7f53a75ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a0158cf0 con 0x7f53a01573a0 2023-10-06T05:23:53.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.868+0000 7f53a75ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a01580e0 con 0x7f53a00b1260 2023-10-06T05:23:53.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.868+0000 7f53a75ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a01583c0 con 0x7f53a00ae860 2023-10-06T05:23:53.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f53a539b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f53a01573a0 0x7f53a00acb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36364/0 (socket says 172.21.15.31:36364) 2023-10-06T05:23:53.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f53a539b700 1 -- 172.21.15.31:0/2826119710 learned_addr learned my addr 172.21.15.31:0/2826119710 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:53.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f53a4b9a700 1 --2- 172.21.15.31:0/2826119710 >> v2:172.21.15.62:3300/0 conn(0x7f53a00ae860 0x7f53a0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906209771 0 0) 0x7f53a0158cf0 con 0x7f53a01573a0 2023-10-06T05:23:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f538c003570 con 0x7f53a01573a0 2023-10-06T05:23:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665056236 0 0) 0x7f53a01580e0 con 0x7f53a00b1260 2023-10-06T05:23:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a0158cf0 con 0x7f53a00b1260 2023-10-06T05:23:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 113859119 0 0) 0x7f538c003570 con 0x7f53a01573a0 2023-10-06T05:23:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 >> v2:172.21.15.62:3300/0 conn(0x7f53a00ae860 msgr2=0x7f53a0157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 --2- 172.21.15.31:0/2826119710 >> v2:172.21.15.62:3300/0 conn(0x7f53a00ae860 0x7f53a0157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 >> v1:172.21.15.62:6789/0 conn(0x7f53a00b1260 legacy=0x7f53a01611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a01586a0 con 0x7f53a01573a0 2023-10-06T05:23:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f539c001470 con 0x7f53a01573a0 2023-10-06T05:23:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f53a4b9a700 1 --2- 172.21.15.31:0/2826119710 >> v2:172.21.15.62:3300/0 conn(0x7f53a00ae860 0x7f53a0157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:23:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.869+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 >> v1:172.21.15.31:6789/0 conn(0x7f53a01573a0 legacy=0x7f53a00acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f5397fff700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f538c003bf0 0x7f538c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f5397fff700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f538c004810 0x7f538c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f5397fff700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f538c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a01583c0 con 0x7f538c0051c0 2023-10-06T05:23:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c00c0b0 con 0x7f538c003bf0 2023-10-06T05:23:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538c00c270 con 0x7f538c004810 2023-10-06T05:23:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f53a5b9c700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f538c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f53a539b700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f538c003bf0 0x7f538c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f53a4b9a700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f538c004810 0x7f538c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f53a5b9c700 1 -- 172.21.15.31:0/2826119710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f538c004810 msgr2=0x7f538c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f53a5b9c700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f538c004810 0x7f538c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f53a5b9c700 1 -- 172.21.15.31:0/2826119710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f538c003bf0 msgr2=0x7f538c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f53a5b9c700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f538c003bf0 0x7f538c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f53a5b9c700 1 -- 172.21.15.31:0/2826119710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c003570 con 0x7f538c0051c0 2023-10-06T05:23:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.870+0000 7f53a5b9c700 1 -- 172.21.15.31:0/2826119710 reap_dead start 2023-10-06T05:23:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.871+0000 7f53a5b9c700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f538c00ba60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5398002120 tx=0x7f53980017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efc1e01383c34d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.871+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539800c980 con 0x7f538c0051c0 2023-10-06T05:23:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.871+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539800cae0 con 0x7f538c0051c0 2023-10-06T05:23:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.871+0000 7f5397fff700 1 -- 172.21.15.31:0/2826119710 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539800ce80 con 0x7f538c0051c0 2023-10-06T05:23:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.871+0000 7f53a75ff700 1 -- 172.21.15.31:0/2826119710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 msgr2=0x7f538c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.871+0000 7f53a75ff700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f538c00ba60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5398002120 tx=0x7f53980017a0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.871+0000 7f53a75ff700 1 -- 172.21.15.31:0/2826119710 shutdown_connections 2023-10-06T05:23:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.871+0000 7f53a75ff700 1 --2- 172.21.15.31:0/2826119710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f538c00ba60 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.871+0000 7f53a75ff700 1 -- 172.21.15.31:0/2826119710 >> 172.21.15.31:0/2826119710 conn(0x7f53a0067560 msgr2=0x7f53a0067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.871+0000 7f53a75ff700 1 -- 172.21.15.31:0/2826119710 shutdown_connections 2023-10-06T05:23:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.872+0000 7f53a75ff700 1 -- 172.21.15.31:0/2826119710 wait complete. 2023-10-06T05:23:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.873+0000 7f53a75ff700 1 Processor -- start 2023-10-06T05:23:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.873+0000 7f53a75ff700 1 -- start start 2023-10-06T05:23:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.874+0000 7f53a75ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f53a015e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.874+0000 7f53a75ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a015c300 0x7f53a0156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.874+0000 7f53a75ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a015c770 0x7f53a0156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.874+0000 7f53a75ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a015e000 con 0x7f538c0051c0 2023-10-06T05:23:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.874+0000 7f53a75ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a015e9d0 con 0x7f53a015c770 2023-10-06T05:23:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.874+0000 7f53a75ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a015d3f0 con 0x7f53a015c300 2023-10-06T05:23:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.874+0000 7f53a539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f53a015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.874+0000 7f53a539b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f53a015e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55048/0 (socket says 172.21.15.31:55048) 2023-10-06T05:23:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.874+0000 7f53a539b700 1 -- 172.21.15.31:0/2502051359 learned_addr learned my addr 172.21.15.31:0/2502051359 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53a4b9a700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a015c300 0x7f53a0156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53a5b9c700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a015c770 0x7f53a0156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53a539b700 1 -- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a015c300 msgr2=0x7f53a0156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53a539b700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a015c300 0x7f53a0156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53a539b700 1 -- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a015c770 msgr2=0x7f53a0156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53a539b700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a015c770 0x7f53a0156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53a539b700 1 -- 172.21.15.31:0/2502051359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a015d6d0 con 0x7f538c0051c0 2023-10-06T05:23:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53a539b700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f53a015e490 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f539c007390 tx=0x7f539c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53957fa700 1 -- 172.21.15.31:0/2502051359 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c018030 con 0x7f538c0051c0 2023-10-06T05:23:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53957fa700 1 -- 172.21.15.31:0/2502051359 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539c00f6c0 con 0x7f538c0051c0 2023-10-06T05:23:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53957fa700 1 -- 172.21.15.31:0/2502051359 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00fa60 con 0x7f538c0051c0 2023-10-06T05:23:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53a75ff700 1 -- 172.21.15.31:0/2502051359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a015d9b0 con 0x7f538c0051c0 2023-10-06T05:23:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.875+0000 7f53a75ff700 1 -- 172.21.15.31:0/2502051359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a015efd0 con 0x7f538c0051c0 2023-10-06T05:23:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.876+0000 7f53a75ff700 1 -- 172.21.15.31:0/2502051359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a004c380 con 0x7f538c0051c0 2023-10-06T05:23:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.877+0000 7f53957fa700 1 -- 172.21.15.31:0/2502051359 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f539c01b070 con 0x7f538c0051c0 2023-10-06T05:23:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.878+0000 7f53957fa700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53840674d0 0x7f5384069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.878+0000 7f53957fa700 1 -- 172.21.15.31:0/2502051359 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f539c019070 con 0x7f538c0051c0 2023-10-06T05:23:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.878+0000 7f53a4b9a700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53840674d0 0x7f5384069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.879+0000 7f53a4b9a700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53840674d0 0x7f5384069980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f53a01595a0 tx=0x7f53900091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:53.882+0000 7f53957fa700 1 -- 172.21.15.31:0/2502051359 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f539c0551e0 con 0x7f538c0051c0 2023-10-06T05:23:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.131+0000 7f53a75ff700 1 -- 172.21.15.31:0/2502051359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f53a00cce60 con 0x7f538c0051c0 2023-10-06T05:23:54.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.647+0000 7f53957fa700 1 -- 172.21.15.31:0/2502051359 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) v1 ==== 104+0+0 (secure 0 0 0) 0x7f539c054d70 con 0x7f538c0051c0 2023-10-06T05:23:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 -- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53840674d0 msgr2=0x7f5384069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53840674d0 0x7f5384069980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f53a01595a0 tx=0x7f53900091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 -- 172.21.15.31:0/2502051359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 msgr2=0x7f53a015e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f53a015e490 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f539c007390 tx=0x7f539c001e10 comp rx=0 tx=0).stop 2023-10-06T05:23:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 -- 172.21.15.31:0/2502051359 shutdown_connections 2023-10-06T05:23:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f53840674d0 0x7f5384069980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53a015c770 0x7f53a0156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f538c0051c0 0x7f53a015e490 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 --2- 172.21.15.31:0/2502051359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53a015c300 0x7f53a0156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 -- 172.21.15.31:0/2502051359 >> 172.21.15.31:0/2502051359 conn(0x7f53a0067560 msgr2=0x7f53a0157770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 -- 172.21.15.31:0/2502051359 shutdown_connections 2023-10-06T05:23:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.650+0000 7f53a75ff700 1 -- 172.21.15.31:0/2502051359 wait complete. 2023-10-06T05:23:54.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:23:54.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.873+0000 7f85a97ff700 1 Processor -- start 2023-10-06T05:23:54.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.873+0000 7f85a97ff700 1 -- start start 2023-10-06T05:23:54.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.874+0000 7f85a97ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f85a406df30 0x7f85a406d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.874+0000 7f85a97ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f85a406d830 0x7f85a4159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.874+0000 7f85a97ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a40c72b0 con 0x7f85a406e300 2023-10-06T05:23:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.874+0000 7f85a97ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a40aba60 con 0x7f85a406d830 2023-10-06T05:23:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.874+0000 7f85a97ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a40c7590 con 0x7f85a406df30 2023-10-06T05:23:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.875+0000 7f85a27fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f85a406d830 0x7f85a4159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.875+0000 7f85a27fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f85a406d830 0x7f85a4159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55050/0 (socket says 172.21.15.31:55050) 2023-10-06T05:23:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.875+0000 7f85a27fc700 1 -- 172.21.15.31:0/3630283066 learned_addr learned my addr 172.21.15.31:0/3630283066 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.875+0000 7f85a2ffd700 1 --2- 172.21.15.31:0/3630283066 >> v2:172.21.15.62:3300/0 conn(0x7f85a406df30 0x7f85a406d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.875+0000 7f85a27fc700 1 -- 172.21.15.31:0/3630283066 >> v2:172.21.15.62:3300/0 conn(0x7f85a406df30 msgr2=0x7f85a406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.875+0000 7f85a27fc700 1 --2- 172.21.15.31:0/3630283066 >> v2:172.21.15.62:3300/0 conn(0x7f85a406df30 0x7f85a406d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.875+0000 7f85a27fc700 1 -- 172.21.15.31:0/3630283066 >> v1:172.21.15.62:6789/0 conn(0x7f85a406e300 legacy=0x7f85a4152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.875+0000 7f85a27fc700 1 -- 172.21.15.31:0/3630283066 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a40c7870 con 0x7f85a406d830 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.876+0000 7f85a27fc700 1 --2- 172.21.15.31:0/3630283066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406d830 0x7f85a4159660 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f85a40cbfc0 tx=0x7f8598009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656253159ef1b4f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.876+0000 7f85a1ffb700 1 -- 172.21.15.31:0/3630283066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598009e70 con 0x7f85a406d830 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.876+0000 7f85a1ffb700 1 -- 172.21.15.31:0/3630283066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859800b860 con 0x7f85a406d830 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.876+0000 7f85a1ffb700 1 -- 172.21.15.31:0/3630283066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859800bc00 con 0x7f85a406d830 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.876+0000 7f85a97ff700 1 -- 172.21.15.31:0/3630283066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406d830 msgr2=0x7f85a4159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.876+0000 7f85a97ff700 1 --2- 172.21.15.31:0/3630283066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406d830 0x7f85a4159660 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f85a40cbfc0 tx=0x7f8598009250 comp rx=0 tx=0).stop 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.876+0000 7f85a97ff700 1 -- 172.21.15.31:0/3630283066 shutdown_connections 2023-10-06T05:23:54.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.876+0000 7f85a97ff700 1 --2- 172.21.15.31:0/3630283066 >> v2:172.21.15.62:3300/0 conn(0x7f85a406df30 0x7f85a406d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.876+0000 7f85a97ff700 1 --2- 172.21.15.31:0/3630283066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406d830 0x7f85a4159660 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.876+0000 7f85a97ff700 1 -- 172.21.15.31:0/3630283066 >> 172.21.15.31:0/3630283066 conn(0x7f85a4068030 msgr2=0x7f85a40cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:54.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.877+0000 7f85a97ff700 1 -- 172.21.15.31:0/3630283066 shutdown_connections 2023-10-06T05:23:54.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.877+0000 7f85a97ff700 1 -- 172.21.15.31:0/3630283066 wait complete. 2023-10-06T05:23:54.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.878+0000 7f85a97ff700 1 Processor -- start 2023-10-06T05:23:54.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.878+0000 7f85a97ff700 1 -- start start 2023-10-06T05:23:54.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.879+0000 7f85a97ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85a406d830 0x7f85a40c0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.879+0000 7f85a97ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a406df30 0x7f85a40c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.879+0000 7f85a97ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406e300 0x7f85a40c2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.879+0000 7f85a97ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40c2a20 con 0x7f85a406e300 2023-10-06T05:23:54.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.879+0000 7f85a97ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a416a050 con 0x7f85a406df30 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.879+0000 7f85a97ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40c0760 con 0x7f85a406d830 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.879+0000 7f85a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406e300 0x7f85a40c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406e300 0x7f85a40c2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55064/0 (socket says 172.21.15.31:55064) 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a37fe700 1 -- 172.21.15.31:0/3600984787 learned_addr learned my addr 172.21.15.31:0/3600984787 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a27fc700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a406df30 0x7f85a40c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a2ffd700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85a406d830 0x7f85a40c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a37fe700 1 -- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85a406d830 msgr2=0x7f85a40c0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a37fe700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85a406d830 0x7f85a40c0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a37fe700 1 -- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a406df30 msgr2=0x7f85a40c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a37fe700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a406df30 0x7f85a40c1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:54.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a37fe700 1 -- 172.21.15.31:0/3600984787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85980094f0 con 0x7f85a406e300 2023-10-06T05:23:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a37fe700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406e300 0x7f85a40c2130 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f859c00c840 tx=0x7f859c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f858affd700 1 -- 172.21.15.31:0/3600984787 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c012630 con 0x7f85a406e300 2023-10-06T05:23:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f858affd700 1 -- 172.21.15.31:0/3600984787 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c0127b0 con 0x7f85a406e300 2023-10-06T05:23:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f85a97ff700 1 -- 172.21.15.31:0/3600984787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a40c2740 con 0x7f85a406e300 2023-10-06T05:23:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.880+0000 7f858affd700 1 -- 172.21.15.31:0/3600984787 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c012b70 con 0x7f85a406e300 2023-10-06T05:23:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.881+0000 7f85a97ff700 1 -- 172.21.15.31:0/3600984787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a416a5d0 con 0x7f85a406e300 2023-10-06T05:23:54.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.882+0000 7f85a97ff700 1 -- 172.21.15.31:0/3600984787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a416a7f0 con 0x7f85a406e300 2023-10-06T05:23:54.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.883+0000 7f858affd700 1 -- 172.21.15.31:0/3600984787 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f859c003030 con 0x7f85a406e300 2023-10-06T05:23:54.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.884+0000 7f858affd700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85900674d0 0x7f8590069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:54.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.884+0000 7f858affd700 1 -- 172.21.15.31:0/3600984787 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f859c085150 con 0x7f85a406e300 2023-10-06T05:23:54.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.884+0000 7f85a2ffd700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85900674d0 0x7f8590069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:54.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.885+0000 7f85a2ffd700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85900674d0 0x7f8590069980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8594009390 tx=0x7f8594008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:54.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:54.888+0000 7f858affd700 1 -- 172.21.15.31:0/3600984787 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f859c051920 con 0x7f85a406e300 2023-10-06T05:23:55.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.093+0000 7f85a97ff700 1 -- 172.21.15.31:0/3600984787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f85a404c380 con 0x7f85a406e300 2023-10-06T05:23:55.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.653+0000 7f858affd700 1 -- 172.21.15.31:0/3600984787 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) v1 ==== 181+0+0 (secure 0 0 0) 0x7f859c054f40 con 0x7f85a406e300 2023-10-06T05:23:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.657+0000 7f85a97ff700 1 -- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85900674d0 msgr2=0x7f8590069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.657+0000 7f85a97ff700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85900674d0 0x7f8590069980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8594009390 tx=0x7f8594008040 comp rx=0 tx=0).stop 2023-10-06T05:23:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.657+0000 7f85a97ff700 1 -- 172.21.15.31:0/3600984787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406e300 msgr2=0x7f85a40c2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.657+0000 7f85a97ff700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406e300 0x7f85a40c2130 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f859c00c840 tx=0x7f859c001670 comp rx=0 tx=0).stop 2023-10-06T05:23:55.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.658+0000 7f85a97ff700 1 -- 172.21.15.31:0/3600984787 shutdown_connections 2023-10-06T05:23:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.658+0000 7f85a97ff700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85900674d0 0x7f8590069980 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.658+0000 7f85a97ff700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a406e300 0x7f85a40c2130 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.658+0000 7f85a97ff700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85a406d830 0x7f85a40c0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.658+0000 7f85a97ff700 1 --2- 172.21.15.31:0/3600984787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a406df30 0x7f85a40c1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.658+0000 7f85a97ff700 1 -- 172.21.15.31:0/3600984787 >> 172.21.15.31:0/3600984787 conn(0x7f85a4068030 msgr2=0x7f85a4156920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.658+0000 7f85a97ff700 1 -- 172.21.15.31:0/3600984787 shutdown_connections 2023-10-06T05:23:55.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.658+0000 7f85a97ff700 1 -- 172.21.15.31:0/3600984787 wait complete. 2023-10-06T05:23:55.661 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:23:55.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:23:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.881+0000 7f294e3ff700 1 Processor -- start 2023-10-06T05:23:55.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.881+0000 7f294e3ff700 1 -- start start 2023-10-06T05:23:55.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.882+0000 7f294e3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29480c6680 0x7f294806eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.882+0000 7f294e3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f29480c0f70 0x7f2948156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.882+0000 7f294e3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29480ca300 con 0x7f2948156940 2023-10-06T05:23:55.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.882+0000 7f294e3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294806dc00 con 0x7f29480c6680 2023-10-06T05:23:55.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.882+0000 7f294e3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294806f0d0 con 0x7f29480c0f70 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.882+0000 7f29477fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29480c6680 0x7f294806eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f29477fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29480c6680 0x7f294806eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55072/0 (socket says 172.21.15.31:55072) 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f29477fe700 1 -- 172.21.15.31:0/470658535 learned_addr learned my addr 172.21.15.31:0/470658535 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f2947fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2948156940 0x7f29480ae410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36372/0 (socket says 172.21.15.31:36372) 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f29477fe700 1 -- 172.21.15.31:0/470658535 >> v2:172.21.15.62:3300/0 conn(0x7f29480c0f70 msgr2=0x7f2948156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f294c99c700 1 --2- 172.21.15.31:0/470658535 >> v2:172.21.15.62:3300/0 conn(0x7f29480c0f70 0x7f2948156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f2946ffd700 1 -- 172.21.15.31:0/470658535 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181277922 0 0) 0x7f29480ca300 con 0x7f2948156940 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f29477fe700 1 --2- 172.21.15.31:0/470658535 >> v2:172.21.15.62:3300/0 conn(0x7f29480c0f70 0x7f2948156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f29477fe700 1 -- 172.21.15.31:0/470658535 >> v1:172.21.15.31:6789/0 conn(0x7f2948156940 legacy=0x7f29480ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f29477fe700 1 -- 172.21.15.31:0/470658535 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294806f2c0 con 0x7f29480c6680 2023-10-06T05:23:55.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f294c99c700 1 --2- 172.21.15.31:0/470658535 >> v2:172.21.15.62:3300/0 conn(0x7f29480c0f70 0x7f2948156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.883+0000 7f29477fe700 1 --2- 172.21.15.31:0/470658535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c6680 0x7f294806eb90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f294806d980 tx=0x7f293c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a05560b80cfc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f2946ffd700 1 -- 172.21.15.31:0/470658535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00b860 con 0x7f29480c6680 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f2946ffd700 1 -- 172.21.15.31:0/470658535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c00b9c0 con 0x7f29480c6680 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f2946ffd700 1 -- 172.21.15.31:0/470658535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00bd80 con 0x7f29480c6680 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f294e3ff700 1 -- 172.21.15.31:0/470658535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c6680 msgr2=0x7f294806eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f294e3ff700 1 --2- 172.21.15.31:0/470658535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c6680 0x7f294806eb90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f294806d980 tx=0x7f293c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f294e3ff700 1 -- 172.21.15.31:0/470658535 shutdown_connections 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f294e3ff700 1 --2- 172.21.15.31:0/470658535 >> v2:172.21.15.62:3300/0 conn(0x7f29480c0f70 0x7f2948156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f294e3ff700 1 --2- 172.21.15.31:0/470658535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c6680 0x7f294806eb90 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f294e3ff700 1 -- 172.21.15.31:0/470658535 >> 172.21.15.31:0/470658535 conn(0x7f29480c8ab0 msgr2=0x7f29480c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:55.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f294e3ff700 1 -- 172.21.15.31:0/470658535 shutdown_connections 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.884+0000 7f294e3ff700 1 -- 172.21.15.31:0/470658535 wait complete. 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.886+0000 7f294e3ff700 1 Processor -- start 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.886+0000 7f294e3ff700 1 -- start start 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f294e3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c0f70 0x7f294814b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f294e3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29480c6680 0x7f2948098230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f294e3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2948156940 0x7f29480b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f294e3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2948098770 con 0x7f29480c0f70 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f294e3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2948097da0 con 0x7f29480c6680 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f294e3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29480b1c60 con 0x7f2948156940 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f2947fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c0f70 0x7f294814b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f2947fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c0f70 0x7f294814b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55076/0 (socket says 172.21.15.31:55076) 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f2947fff700 1 -- 172.21.15.31:0/557927704 learned_addr learned my addr 172.21.15.31:0/557927704 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f29477fe700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29480c6680 0x7f2948098230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.887+0000 7f294c99c700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2948156940 0x7f29480b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f2947fff700 1 -- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2948156940 msgr2=0x7f29480b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f2947fff700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2948156940 0x7f29480b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f2947fff700 1 -- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29480c6680 msgr2=0x7f2948098230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f2947fff700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29480c6680 0x7f2948098230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:55.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f2947fff700 1 -- 172.21.15.31:0/557927704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0094f0 con 0x7f29480c0f70 2023-10-06T05:23:55.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f2947fff700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c0f70 0x7f294814b3d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f293800c5b0 tx=0x7f2938001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:55.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f292ffff700 1 -- 172.21.15.31:0/557927704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938005a70 con 0x7f29480c0f70 2023-10-06T05:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f292ffff700 1 -- 172.21.15.31:0/557927704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2938005bd0 con 0x7f29480c0f70 2023-10-06T05:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f292ffff700 1 -- 172.21.15.31:0/557927704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29380070d0 con 0x7f29480c0f70 2023-10-06T05:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f294e3ff700 1 -- 172.21.15.31:0/557927704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2948097470 con 0x7f29480c0f70 2023-10-06T05:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.888+0000 7f294e3ff700 1 -- 172.21.15.31:0/557927704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2948097a30 con 0x7f29480c0f70 2023-10-06T05:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.889+0000 7f294e3ff700 1 -- 172.21.15.31:0/557927704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2948098080 con 0x7f29480c0f70 2023-10-06T05:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.890+0000 7f292ffff700 1 -- 172.21.15.31:0/557927704 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f2938007440 con 0x7f29480c0f70 2023-10-06T05:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.891+0000 7f292ffff700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29340674d0 0x7f2934069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.891+0000 7f292ffff700 1 -- 172.21.15.31:0/557927704 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5762+0+0 (secure 0 0 0) 0x7f29380844d0 con 0x7f29480c0f70 2023-10-06T05:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.892+0000 7f29477fe700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29340674d0 0x7f2934069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.892+0000 7f29477fe700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29340674d0 0x7f2934069980 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f294806d980 tx=0x7f293c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:55.895+0000 7f292ffff700 1 -- 172.21.15.31:0/557927704 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2938050e90 con 0x7f29480c0f70 2023-10-06T05:23:56.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.099+0000 7f294e3ff700 1 -- 172.21.15.31:0/557927704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f294804c380 con 0x7f29480c0f70 2023-10-06T05:23:56.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.105+0000 7f292ffff700 1 -- 172.21.15.31:0/557927704 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) v1 ==== 169+0+0 (secure 0 0 0) 0x7f29380544b0 con 0x7f29480c0f70 2023-10-06T05:23:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 -- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29340674d0 msgr2=0x7f2934069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29340674d0 0x7f2934069980 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f294806d980 tx=0x7f293c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 -- 172.21.15.31:0/557927704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c0f70 msgr2=0x7f294814b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c0f70 0x7f294814b3d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f293800c5b0 tx=0x7f2938001040 comp rx=0 tx=0).stop 2023-10-06T05:23:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 -- 172.21.15.31:0/557927704 shutdown_connections 2023-10-06T05:23:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f29340674d0 0x7f2934069980 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2948156940 0x7f29480b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29480c0f70 0x7f294814b3d0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 --2- 172.21.15.31:0/557927704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29480c6680 0x7f2948098230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 -- 172.21.15.31:0/557927704 >> 172.21.15.31:0/557927704 conn(0x7f29480c8ab0 msgr2=0x7f29480c1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.109+0000 7f294e3ff700 1 -- 172.21.15.31:0/557927704 shutdown_connections 2023-10-06T05:23:56.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.110+0000 7f294e3ff700 1 -- 172.21.15.31:0/557927704 wait complete. 2023-10-06T05:23:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:23:56.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:23:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.341+0000 7fe2a19ff700 1 Processor -- start 2023-10-06T05:23:56.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.342+0000 7fe2a19ff700 1 -- start start 2023-10-06T05:23:56.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.343+0000 7fe2a19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 0x7fe29c0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.343+0000 7fe2a19ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe29c159930 0x7fe29c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.343+0000 7fe2a19ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c15d2d0 con 0x7fe29c0b0d60 2023-10-06T05:23:56.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.343+0000 7fe2a19ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29c15d5b0 con 0x7fe29c159930 2023-10-06T05:23:56.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.343+0000 7fe2a19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c15d890 con 0x7fe29c159560 2023-10-06T05:23:56.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.343+0000 7fe29affd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 0x7fe29c0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.343+0000 7fe29a7fc700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe29c159930 0x7fe29c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.343+0000 7fe29affd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 0x7fe29c0cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:48004/0 (socket says 172.21.15.31:48004) 2023-10-06T05:23:56.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.343+0000 7fe29affd700 1 -- 172.21.15.31:0/1614573509 learned_addr learned my addr 172.21.15.31:0/1614573509 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.344+0000 7fe29affd700 1 -- 172.21.15.31:0/1614573509 >> v2:172.21.15.62:3300/0 conn(0x7fe29c159930 msgr2=0x7fe29c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.344+0000 7fe29affd700 1 --2- 172.21.15.31:0/1614573509 >> v2:172.21.15.62:3300/0 conn(0x7fe29c159930 0x7fe29c0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.344+0000 7fe29affd700 1 -- 172.21.15.31:0/1614573509 >> v1:172.21.15.62:6789/0 conn(0x7fe29c0b0d60 legacy=0x7fe29c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.344+0000 7fe29affd700 1 -- 172.21.15.31:0/1614573509 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c15db70 con 0x7fe29c159560 2023-10-06T05:23:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.344+0000 7fe29affd700 1 --2- 172.21.15.31:0/1614573509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 0x7fe29c0cce70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0099b0 tx=0x7fe28c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e08d5c713f2f092 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe299ffb700 1 -- 172.21.15.31:0/1614573509 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c009ea0 con 0x7fe29c159560 2023-10-06T05:23:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe299ffb700 1 -- 172.21.15.31:0/1614573509 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c00b860 con 0x7fe29c159560 2023-10-06T05:23:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe299ffb700 1 -- 172.21.15.31:0/1614573509 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00bc00 con 0x7fe29c159560 2023-10-06T05:23:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1614573509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 msgr2=0x7fe29c0cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe2a19ff700 1 --2- 172.21.15.31:0/1614573509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 0x7fe29c0cce70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0099b0 tx=0x7fe28c009250 comp rx=0 tx=0).stop 2023-10-06T05:23:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1614573509 shutdown_connections 2023-10-06T05:23:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe2a19ff700 1 --2- 172.21.15.31:0/1614573509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 0x7fe29c0cce70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe2a19ff700 1 --2- 172.21.15.31:0/1614573509 >> v2:172.21.15.62:3300/0 conn(0x7fe29c159930 0x7fe29c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1614573509 >> 172.21.15.31:0/1614573509 conn(0x7fe29c06ec90 msgr2=0x7fe29c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1614573509 shutdown_connections 2023-10-06T05:23:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.345+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1614573509 wait complete. 2023-10-06T05:23:56.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.347+0000 7fe2a19ff700 1 Processor -- start 2023-10-06T05:23:56.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.347+0000 7fe2a19ff700 1 -- start start 2023-10-06T05:23:56.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe2a19ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe29c0b0d60 0x7fe29c0c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe2a19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 0x7fe29c0c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe2a19ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c159930 0x7fe29c0ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe2a19ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c1570d0 con 0x7fe29c0b0d60 2023-10-06T05:23:56.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe2a19ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c16dd60 con 0x7fe29c159930 2023-10-06T05:23:56.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe2a19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c16d150 con 0x7fe29c159560 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe29affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe29c0b0d60 0x7fe29c0c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe29affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe29c0b0d60 0x7fe29c0c4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55082/0 (socket says 172.21.15.31:55082) 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe29affd700 1 -- 172.21.15.31:0/1974519217 learned_addr learned my addr 172.21.15.31:0/1974519217 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe29a7fc700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 0x7fe29c0c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.348+0000 7fe29b7fe700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c159930 0x7fe29c0ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe29affd700 1 -- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 msgr2=0x7fe29c0c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe29affd700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 0x7fe29c0c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe29affd700 1 -- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c159930 msgr2=0x7fe29c0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe29affd700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c159930 0x7fe29c0ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe29affd700 1 -- 172.21.15.31:0/1974519217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0094f0 con 0x7fe29c0b0d60 2023-10-06T05:23:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe29affd700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe29c0b0d60 0x7fe29c0c4dc0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe29c15d9d0 tx=0x7fe28c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe282ffd700 1 -- 172.21.15.31:0/1974519217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c000d10 con 0x7fe29c0b0d60 2023-10-06T05:23:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe282ffd700 1 -- 172.21.15.31:0/1974519217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c00bbe0 con 0x7fe29c0b0d60 2023-10-06T05:23:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe282ffd700 1 -- 172.21.15.31:0/1974519217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c011670 con 0x7fe29c0b0d60 2023-10-06T05:23:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1974519217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c16d430 con 0x7fe29c0b0d60 2023-10-06T05:23:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.349+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1974519217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c16d9f0 con 0x7fe29c0b0d60 2023-10-06T05:23:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.350+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1974519217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c169a20 con 0x7fe29c0b0d60 2023-10-06T05:23:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.351+0000 7fe282ffd700 1 -- 172.21.15.31:0/1974519217 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fe28c011aa0 con 0x7fe29c0b0d60 2023-10-06T05:23:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.352+0000 7fe282ffd700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2880674d0 0x7fe288069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.353+0000 7fe282ffd700 1 -- 172.21.15.31:0/1974519217 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5348+0+0 (secure 0 0 0) 0x7fe28c0866e0 con 0x7fe29c0b0d60 2023-10-06T05:23:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.353+0000 7fe29a7fc700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2880674d0 0x7fe288069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.354+0000 7fe29a7fc700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2880674d0 0x7fe288069980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe29c157960 tx=0x7fe2900091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.357+0000 7fe282ffd700 1 -- 172.21.15.31:0/1974519217 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe28c053200 con 0x7fe29c0b0d60 2023-10-06T05:23:56.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.560+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1974519217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe29c04c380 con 0x7fe29c0b0d60 2023-10-06T05:23:56.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.564+0000 7fe282ffd700 1 -- 172.21.15.31:0/1974519217 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe28c056820 con 0x7fe29c0b0d60 2023-10-06T05:23:56.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2880674d0 msgr2=0x7fe288069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2880674d0 0x7fe288069980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe29c157960 tx=0x7fe2900091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1974519217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe29c0b0d60 msgr2=0x7fe29c0c4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe29c0b0d60 0x7fe29c0c4dc0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe29c15d9d0 tx=0x7fe28c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:56.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1974519217 shutdown_connections 2023-10-06T05:23:56.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2880674d0 0x7fe288069980 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe29c159930 0x7fe29c0ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe29c0b0d60 0x7fe29c0c4dc0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 --2- 172.21.15.31:0/1974519217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe29c159560 0x7fe29c0c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1974519217 >> 172.21.15.31:0/1974519217 conn(0x7fe29c06ec90 msgr2=0x7fe29c0b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:56.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1974519217 shutdown_connections 2023-10-06T05:23:56.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.566+0000 7fe2a19ff700 1 -- 172.21.15.31:0/1974519217 wait complete. 2023-10-06T05:23:56.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2023-10-06T05:23:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.780+0000 7f5b82fff700 1 Processor -- start 2023-10-06T05:23:56.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.780+0000 7f5b82fff700 1 -- start start 2023-10-06T05:23:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.781+0000 7f5b82fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b7c159500 0x7f5b7c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.781+0000 7f5b82fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5b7c15d2d0 0x7f5b7c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.781+0000 7f5b82fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5b7c1598d0 0x7f5b7c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.781+0000 7f5b82fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7c0c3f40 con 0x7f5b7c15d2d0 2023-10-06T05:23:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.781+0000 7f5b82fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7c0cb000 con 0x7f5b7c1598d0 2023-10-06T05:23:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.781+0000 7f5b82fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c0b2860 con 0x7f5b7c159500 2023-10-06T05:23:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b7bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5b7c15d2d0 0x7f5b7c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b7bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5b7c15d2d0 0x7f5b7c0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55094/0 (socket says 172.21.15.31:55094) 2023-10-06T05:23:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b7bfff700 1 -- 172.21.15.31:0/2763746170 learned_addr learned my addr 172.21.15.31:0/2763746170 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b80d9b700 1 --2- 172.21.15.31:0/2763746170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b7c159500 0x7f5b7c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b8159c700 1 --2- 172.21.15.31:0/2763746170 >> v2:172.21.15.62:3300/0 conn(0x7f5b7c1598d0 0x7f5b7c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b7bfff700 1 -- 172.21.15.31:0/2763746170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b7c159500 msgr2=0x7f5b7c0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b7bfff700 1 --2- 172.21.15.31:0/2763746170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b7c159500 0x7f5b7c0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b7bfff700 1 -- 172.21.15.31:0/2763746170 >> v2:172.21.15.62:3300/0 conn(0x7f5b7c1598d0 msgr2=0x7f5b7c0cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b7bfff700 1 --2- 172.21.15.31:0/2763746170 >> v2:172.21.15.62:3300/0 conn(0x7f5b7c1598d0 0x7f5b7c0cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b7bfff700 1 -- 172.21.15.31:0/2763746170 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c15d7d0 con 0x7f5b7c15d2d0 2023-10-06T05:23:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.782+0000 7f5b7bfff700 1 --2- 172.21.15.31:0/2763746170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c15d2d0 0x7f5b7c0cce00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0c7440 tx=0x7f5b68009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895ef9914f2baf5d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b7b7fe700 1 -- 172.21.15.31:0/2763746170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68005c10 con 0x7f5b7c15d2d0 2023-10-06T05:23:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b7b7fe700 1 -- 172.21.15.31:0/2763746170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b68005d70 con 0x7f5b7c15d2d0 2023-10-06T05:23:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b7b7fe700 1 -- 172.21.15.31:0/2763746170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68006260 con 0x7f5b7c15d2d0 2023-10-06T05:23:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b82fff700 1 -- 172.21.15.31:0/2763746170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c15d2d0 msgr2=0x7f5b7c0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b82fff700 1 --2- 172.21.15.31:0/2763746170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c15d2d0 0x7f5b7c0cce00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0c7440 tx=0x7f5b68009220 comp rx=0 tx=0).stop 2023-10-06T05:23:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b82fff700 1 -- 172.21.15.31:0/2763746170 shutdown_connections 2023-10-06T05:23:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b82fff700 1 --2- 172.21.15.31:0/2763746170 >> v2:172.21.15.62:3300/0 conn(0x7f5b7c1598d0 0x7f5b7c0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b82fff700 1 --2- 172.21.15.31:0/2763746170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b7c159500 0x7f5b7c0b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b82fff700 1 --2- 172.21.15.31:0/2763746170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c15d2d0 0x7f5b7c0cce00 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b82fff700 1 -- 172.21.15.31:0/2763746170 >> 172.21.15.31:0/2763746170 conn(0x7f5b7c06ec90 msgr2=0x7f5b7c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b82fff700 1 -- 172.21.15.31:0/2763746170 shutdown_connections 2023-10-06T05:23:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.783+0000 7f5b82fff700 1 -- 172.21.15.31:0/2763746170 wait complete. 2023-10-06T05:23:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.785+0000 7f5b82fff700 1 Processor -- start 2023-10-06T05:23:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.785+0000 7f5b82fff700 1 -- start start 2023-10-06T05:23:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.786+0000 7f5b82fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b7c159500 0x7f5b7c0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.786+0000 7f5b82fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c1598d0 0x7f5b7c1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.786+0000 7f5b82fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b7c15d2d0 0x7f5b7c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.786+0000 7f5b82fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c0c8e60 con 0x7f5b7c1598d0 2023-10-06T05:23:56.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.786+0000 7f5b82fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c0b4570 con 0x7f5b7c159500 2023-10-06T05:23:56.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.786+0000 7f5b82fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c0c8250 con 0x7f5b7c15d2d0 2023-10-06T05:23:56.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.786+0000 7f5b7bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c1598d0 0x7f5b7c1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.786+0000 7f5b7bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c1598d0 0x7f5b7c1589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55102/0 (socket says 172.21.15.31:55102) 2023-10-06T05:23:56.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.786+0000 7f5b7bfff700 1 -- 172.21.15.31:0/484725264 learned_addr learned my addr 172.21.15.31:0/484725264 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:56.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.786+0000 7f5b80d9b700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b7c159500 0x7f5b7c0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b8159c700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b7c15d2d0 0x7f5b7c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b7bfff700 1 -- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b7c15d2d0 msgr2=0x7f5b7c0b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b7bfff700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b7c15d2d0 0x7f5b7c0b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b7bfff700 1 -- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b7c159500 msgr2=0x7f5b7c0c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:56.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b7bfff700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b7c159500 0x7f5b7c0c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:56.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b7bfff700 1 -- 172.21.15.31:0/484725264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b68009480 con 0x7f5b7c1598d0 2023-10-06T05:23:56.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b7bfff700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c1598d0 0x7f5b7c1589f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c067c50 tx=0x7f5b68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b78ff9700 1 -- 172.21.15.31:0/484725264 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b680116c0 con 0x7f5b7c1598d0 2023-10-06T05:23:56.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b78ff9700 1 -- 172.21.15.31:0/484725264 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b68020070 con 0x7f5b7c1598d0 2023-10-06T05:23:56.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b78ff9700 1 -- 172.21.15.31:0/484725264 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68011a50 con 0x7f5b7c1598d0 2023-10-06T05:23:56.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.787+0000 7f5b82fff700 1 -- 172.21.15.31:0/484725264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c0c8530 con 0x7f5b7c1598d0 2023-10-06T05:23:56.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.788+0000 7f5b82fff700 1 -- 172.21.15.31:0/484725264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c0c8af0 con 0x7f5b7c1598d0 2023-10-06T05:23:56.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.789+0000 7f5b82fff700 1 -- 172.21.15.31:0/484725264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7c0c8c80 con 0x7f5b7c1598d0 2023-10-06T05:23:56.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.789+0000 7f5b78ff9700 1 -- 172.21.15.31:0/484725264 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f5b68022030 con 0x7f5b7c1598d0 2023-10-06T05:23:56.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.790+0000 7f5b78ff9700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b600674d0 0x7f5b60069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:56.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.790+0000 7f5b78ff9700 1 -- 172.21.15.31:0/484725264 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5348+0+0 (secure 0 0 0) 0x7f5b680869a0 con 0x7f5b7c1598d0 2023-10-06T05:23:56.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.791+0000 7f5b80d9b700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b600674d0 0x7f5b60069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:56.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.791+0000 7f5b80d9b700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b600674d0 0x7f5b60069980 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009db0 tx=0x7f5b700091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:56.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:56.795+0000 7f5b78ff9700 1 -- 172.21.15.31:0/484725264 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b68053500 con 0x7f5b7c1598d0 2023-10-06T05:23:57.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.080+0000 7f5b82fff700 1 -- 172.21.15.31:0/484725264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) v1 -- 0x7f5b7c04c380 con 0x7f5b7c1598d0 2023-10-06T05:23:57.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.659+0000 7f5b78ff9700 1 -- 172.21.15.31:0/484725264 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v376) v1 ==== 148+0+0 (secure 0 0 0) 0x7f5b68056b20 con 0x7f5b7c1598d0 2023-10-06T05:23:57.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.661+0000 7f5b82fff700 1 -- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b600674d0 msgr2=0x7f5b60069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.661+0000 7f5b82fff700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b600674d0 0x7f5b60069980 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009db0 tx=0x7f5b700091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.661+0000 7f5b82fff700 1 -- 172.21.15.31:0/484725264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c1598d0 msgr2=0x7f5b7c1589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.661+0000 7f5b82fff700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c1598d0 0x7f5b7c1589f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c067c50 tx=0x7f5b68016000 comp rx=0 tx=0).stop 2023-10-06T05:23:57.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.662+0000 7f5b82fff700 1 -- 172.21.15.31:0/484725264 shutdown_connections 2023-10-06T05:23:57.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.662+0000 7f5b82fff700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b600674d0 0x7f5b60069980 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.662+0000 7f5b82fff700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b7c15d2d0 0x7f5b7c0b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.662+0000 7f5b82fff700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b7c159500 0x7f5b7c0c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.662+0000 7f5b82fff700 1 --2- 172.21.15.31:0/484725264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b7c1598d0 0x7f5b7c1589f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.662+0000 7f5b82fff700 1 -- 172.21.15.31:0/484725264 >> 172.21.15.31:0/484725264 conn(0x7f5b7c06ec90 msgr2=0x7f5b7c0cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.662+0000 7f5b82fff700 1 -- 172.21.15.31:0/484725264 shutdown_connections 2023-10-06T05:23:57.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.662+0000 7f5b82fff700 1 -- 172.21.15.31:0/484725264 wait complete. 2023-10-06T05:23:57.684 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2023-10-06T05:23:57.684 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:23:57.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2023-10-06T05:23:57.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.883+0000 7f114b7ff700 1 Processor -- start 2023-10-06T05:23:57.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.883+0000 7f114b7ff700 1 -- start start 2023-10-06T05:23:57.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f114b7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1144068d80 0x7f1144159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f114b7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11440cc0d0 0x7f11440c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f114b7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11440c95b0 con 0x7f11441595a0 2023-10-06T05:23:57.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f114b7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440c9890 con 0x7f11440cc0d0 2023-10-06T05:23:57.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f114b7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440c9b70 con 0x7f1144068d80 2023-10-06T05:23:57.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f1149d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11440cc0d0 0x7f11440c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f1149d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11440cc0d0 0x7f11440c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35840/0 (socket says 172.21.15.31:35840) 2023-10-06T05:23:57.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f1149d9c700 1 -- 172.21.15.31:0/3929656264 learned_addr learned my addr 172.21.15.31:0/3929656264 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:57.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f1148d9a700 1 --2- 172.21.15.31:0/3929656264 >> v2:172.21.15.62:3300/0 conn(0x7f1144068d80 0x7f1144159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f1149d9c700 1 -- 172.21.15.31:0/3929656264 >> v2:172.21.15.62:3300/0 conn(0x7f1144068d80 msgr2=0x7f1144159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f1149d9c700 1 --2- 172.21.15.31:0/3929656264 >> v2:172.21.15.62:3300/0 conn(0x7f1144068d80 0x7f1144159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f1149d9c700 1 -- 172.21.15.31:0/3929656264 >> v1:172.21.15.62:6789/0 conn(0x7f11441595a0 legacy=0x7f1144068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:57.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.885+0000 7f1149d9c700 1 -- 172.21.15.31:0/3929656264 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440ca130 con 0x7f11440cc0d0 2023-10-06T05:23:57.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.886+0000 7f1148d9a700 1 --2- 172.21.15.31:0/3929656264 >> v2:172.21.15.62:3300/0 conn(0x7f1144068d80 0x7f1144159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:23:57.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.886+0000 7f1149d9c700 1 --2- 172.21.15.31:0/3929656264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11440cc0d0 0x7f11440c8930 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f113c009f20 tx=0x7f113c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7319e953881d55a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.886+0000 7f113bfff700 1 -- 172.21.15.31:0/3929656264 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c00e070 con 0x7f11440cc0d0 2023-10-06T05:23:57.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.886+0000 7f113bfff700 1 -- 172.21.15.31:0/3929656264 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c00ba00 con 0x7f11440cc0d0 2023-10-06T05:23:57.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.886+0000 7f113bfff700 1 -- 172.21.15.31:0/3929656264 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c0092e0 con 0x7f11440cc0d0 2023-10-06T05:23:57.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.886+0000 7f114b7ff700 1 -- 172.21.15.31:0/3929656264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11440cc0d0 msgr2=0x7f11440c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.886+0000 7f114b7ff700 1 --2- 172.21.15.31:0/3929656264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11440cc0d0 0x7f11440c8930 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f113c009f20 tx=0x7f113c009840 comp rx=0 tx=0).stop 2023-10-06T05:23:57.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.887+0000 7f114b7ff700 1 -- 172.21.15.31:0/3929656264 shutdown_connections 2023-10-06T05:23:57.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.887+0000 7f114b7ff700 1 --2- 172.21.15.31:0/3929656264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11440cc0d0 0x7f11440c8930 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.887+0000 7f114b7ff700 1 --2- 172.21.15.31:0/3929656264 >> v2:172.21.15.62:3300/0 conn(0x7f1144068d80 0x7f1144159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.887+0000 7f114b7ff700 1 -- 172.21.15.31:0/3929656264 >> 172.21.15.31:0/3929656264 conn(0x7f114406b620 msgr2=0x7f114406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:57.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.887+0000 7f114b7ff700 1 -- 172.21.15.31:0/3929656264 shutdown_connections 2023-10-06T05:23:57.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.887+0000 7f114b7ff700 1 -- 172.21.15.31:0/3929656264 wait complete. 2023-10-06T05:23:57.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.888+0000 7f114b7ff700 1 Processor -- start 2023-10-06T05:23:57.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.888+0000 7f114b7ff700 1 -- start start 2023-10-06T05:23:57.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.889+0000 7f114b7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1144068d80 0x7f1144152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.889+0000 7f114b7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11440cc0d0 0x7f11440c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.889+0000 7f114b7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11441595a0 0x7f1144155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.889+0000 7f114b7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440c8a20 con 0x7f11441595a0 2023-10-06T05:23:57.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.889+0000 7f114b7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440c6160 con 0x7f11440cc0d0 2023-10-06T05:23:57.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.889+0000 7f114b7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11441525b0 con 0x7f1144068d80 2023-10-06T05:23:57.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1149d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11441595a0 0x7f1144155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1149d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11441595a0 0x7f1144155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35850/0 (socket says 172.21.15.31:35850) 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1149d9c700 1 -- 172.21.15.31:0/486255001 learned_addr learned my addr 172.21.15.31:0/486255001 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1148d9a700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11440cc0d0 0x7f11440c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f114959b700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1144068d80 0x7f1144152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1149d9c700 1 -- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1144068d80 msgr2=0x7f1144152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1149d9c700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1144068d80 0x7f1144152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1149d9c700 1 -- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11440cc0d0 msgr2=0x7f11440c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1149d9c700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11440cc0d0 0x7f11440c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1149d9c700 1 -- 172.21.15.31:0/486255001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c009ae0 con 0x7f11441595a0 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1148d9a700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11440cc0d0 0x7f11440c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.890+0000 7f1149d9c700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11441595a0 0x7f1144155bd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f113c00b6c0 tx=0x7f113c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.891+0000 7f11397fa700 1 -- 172.21.15.31:0/486255001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c012e90 con 0x7f11441595a0 2023-10-06T05:23:57.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.891+0000 7f11397fa700 1 -- 172.21.15.31:0/486255001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c0092e0 con 0x7f11441595a0 2023-10-06T05:23:57.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.891+0000 7f114b7ff700 1 -- 172.21.15.31:0/486255001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11440c7e10 con 0x7f11441595a0 2023-10-06T05:23:57.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.891+0000 7f11397fa700 1 -- 172.21.15.31:0/486255001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c01d650 con 0x7f11441595a0 2023-10-06T05:23:57.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.891+0000 7f114b7ff700 1 -- 172.21.15.31:0/486255001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11440c83d0 con 0x7f11441595a0 2023-10-06T05:23:57.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.892+0000 7f114b7ff700 1 -- 172.21.15.31:0/486255001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11440c8bd0 con 0x7f11441595a0 2023-10-06T05:23:57.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.893+0000 7f11397fa700 1 -- 172.21.15.31:0/486255001 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f113c01da40 con 0x7f11441595a0 2023-10-06T05:23:57.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.893+0000 7f11397fa700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f11280674d0 0x7f1128069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:57.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.894+0000 7f11397fa700 1 -- 172.21.15.31:0/486255001 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5348+0+0 (secure 0 0 0) 0x7f113c086910 con 0x7f11441595a0 2023-10-06T05:23:57.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.894+0000 7f114959b700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f11280674d0 0x7f1128069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:57.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.894+0000 7f114959b700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f11280674d0 0x7f1128069980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1140006fd0 tx=0x7f1140008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:57.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:57.898+0000 7f11397fa700 1 -- 172.21.15.31:0/486255001 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11440c8bd0 con 0x7f11441595a0 2023-10-06T05:23:58.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.177+0000 7f114b7ff700 1 -- 172.21.15.31:0/486255001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) v1 -- 0x7f11440c8bd0 con 0x7f11441595a0 2023-10-06T05:23:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.661+0000 7f11397fa700 1 -- 172.21.15.31:0/486255001 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v377) v1 ==== 154+0+0 (secure 0 0 0) 0x7f11440c8bd0 con 0x7f11441595a0 2023-10-06T05:23:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.663+0000 7f114b7ff700 1 -- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f11280674d0 msgr2=0x7f1128069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.663+0000 7f114b7ff700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f11280674d0 0x7f1128069980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1140006fd0 tx=0x7f1140008040 comp rx=0 tx=0).stop 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.663+0000 7f114b7ff700 1 -- 172.21.15.31:0/486255001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11441595a0 msgr2=0x7f1144155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.663+0000 7f114b7ff700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11441595a0 0x7f1144155bd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f113c00b6c0 tx=0x7f113c017000 comp rx=0 tx=0).stop 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.663+0000 7f114b7ff700 1 -- 172.21.15.31:0/486255001 shutdown_connections 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.663+0000 7f114b7ff700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f11280674d0 0x7f1128069980 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.663+0000 7f114b7ff700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11441595a0 0x7f1144155bd0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.663+0000 7f114b7ff700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1144068d80 0x7f1144152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.663+0000 7f114b7ff700 1 --2- 172.21.15.31:0/486255001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11440cc0d0 0x7f11440c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.663+0000 7f114b7ff700 1 -- 172.21.15.31:0/486255001 >> 172.21.15.31:0/486255001 conn(0x7f114406b620 msgr2=0x7f114406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.664+0000 7f114b7ff700 1 -- 172.21.15.31:0/486255001 shutdown_connections 2023-10-06T05:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.664+0000 7f114b7ff700 1 -- 172.21.15.31:0/486255001 wait complete. 2023-10-06T05:23:58.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:23:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.883+0000 7f03bcdff700 1 Processor -- start 2023-10-06T05:23:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.883+0000 7f03bcdff700 1 -- start start 2023-10-06T05:23:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.884+0000 7f03bcdff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03b8153d40 0x7f03b80b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.884+0000 7f03bcdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03b80b09c0 0x7f03b80af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03bcdff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b80ad690 con 0x7f03b80af6c0 2023-10-06T05:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03bcdff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b80b0e10 con 0x7f03b8153d40 2023-10-06T05:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03bcdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80acd60 con 0x7f03b80b09c0 2023-10-06T05:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03b659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03b8153d40 0x7f03b80b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03b6d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f03b80af6c0 0x7f03b8159ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47380/0 (socket says 172.21.15.31:47380) 2023-10-06T05:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03b6d9d700 1 -- 172.21.15.31:0/740824155 learned_addr learned my addr 172.21.15.31:0/740824155 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03b5d9b700 1 --2- 172.21.15.31:0/740824155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03b80b09c0 0x7f03b80af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03b659c700 1 -- 172.21.15.31:0/740824155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03b80b09c0 msgr2=0x7f03b80af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03b659c700 1 --2- 172.21.15.31:0/740824155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03b80b09c0 0x7f03b80af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03b659c700 1 -- 172.21.15.31:0/740824155 >> v1:172.21.15.31:6789/0 conn(0x7f03b80af6c0 legacy=0x7f03b8159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.885+0000 7f03b659c700 1 -- 172.21.15.31:0/740824155 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b80ad040 con 0x7f03b8153d40 2023-10-06T05:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.886+0000 7f03b659c700 1 --2- 172.21.15.31:0/740824155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 0x7f03b80b0480 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f03a80098e0 tx=0x7f03a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc68be34cf9f8ce5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.886+0000 7f03b559a700 1 -- 172.21.15.31:0/740824155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8009e80 con 0x7f03b8153d40 2023-10-06T05:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.886+0000 7f03b559a700 1 -- 172.21.15.31:0/740824155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a800b860 con 0x7f03b8153d40 2023-10-06T05:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.886+0000 7f03b559a700 1 -- 172.21.15.31:0/740824155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a800bbe0 con 0x7f03b8153d40 2023-10-06T05:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.886+0000 7f03bcdff700 1 -- 172.21.15.31:0/740824155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 msgr2=0x7f03b80b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.886+0000 7f03bcdff700 1 --2- 172.21.15.31:0/740824155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 0x7f03b80b0480 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f03a80098e0 tx=0x7f03a80091a0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.886+0000 7f03bcdff700 1 -- 172.21.15.31:0/740824155 shutdown_connections 2023-10-06T05:23:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.886+0000 7f03bcdff700 1 --2- 172.21.15.31:0/740824155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 0x7f03b80b0480 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.886+0000 7f03bcdff700 1 --2- 172.21.15.31:0/740824155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03b80b09c0 0x7f03b80af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.887+0000 7f03bcdff700 1 -- 172.21.15.31:0/740824155 >> 172.21.15.31:0/740824155 conn(0x7f03b8067560 msgr2=0x7f03b8067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.887+0000 7f03bcdff700 1 -- 172.21.15.31:0/740824155 shutdown_connections 2023-10-06T05:23:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.887+0000 7f03bcdff700 1 -- 172.21.15.31:0/740824155 wait complete. 2023-10-06T05:23:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.889+0000 7f03bcdff700 1 Processor -- start 2023-10-06T05:23:58.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.889+0000 7f03bcdff700 1 -- start start 2023-10-06T05:23:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03bcdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b80af6c0 0x7f03b8142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03bcdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03b80b09c0 0x7f03b814b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03bcdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 0x7f03b80c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03bcdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b814a740 con 0x7f03b8153d40 2023-10-06T05:23:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03bcdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b814b560 con 0x7f03b80af6c0 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03bcdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b814a260 con 0x7f03b80b09c0 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03b6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 0x7f03b80c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03b6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 0x7f03b80c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35872/0 (socket says 172.21.15.31:35872) 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03b6d9d700 1 -- 172.21.15.31:0/2135797921 learned_addr learned my addr 172.21.15.31:0/2135797921 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03b659c700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b80af6c0 0x7f03b8142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03b5d9b700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03b80b09c0 0x7f03b814b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03b6d9d700 1 -- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03b80b09c0 msgr2=0x7f03b814b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03b6d9d700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03b80b09c0 0x7f03b814b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03b6d9d700 1 -- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b80af6c0 msgr2=0x7f03b8142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03b6d9d700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b80af6c0 0x7f03b8142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.890+0000 7f03b6d9d700 1 -- 172.21.15.31:0/2135797921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a8009420 con 0x7f03b8153d40 2023-10-06T05:23:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.891+0000 7f03b6d9d700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 0x7f03b80c95e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f03b000cc40 tx=0x7f03b0001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.891+0000 7f03a67fc700 1 -- 172.21.15.31:0/2135797921 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b000a8b0 con 0x7f03b8153d40 2023-10-06T05:23:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.891+0000 7f03a67fc700 1 -- 172.21.15.31:0/2135797921 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b000aa10 con 0x7f03b8153d40 2023-10-06T05:23:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.891+0000 7f03a67fc700 1 -- 172.21.15.31:0/2135797921 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b000add0 con 0x7f03b8153d40 2023-10-06T05:23:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.891+0000 7f03bcdff700 1 -- 172.21.15.31:0/2135797921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b814a950 con 0x7f03b8153d40 2023-10-06T05:23:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.891+0000 7f03bcdff700 1 -- 172.21.15.31:0/2135797921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b814af10 con 0x7f03b8153d40 2023-10-06T05:23:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.892+0000 7f03bcdff700 1 -- 172.21.15.31:0/2135797921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b814b6f0 con 0x7f03b8153d40 2023-10-06T05:23:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.893+0000 7f03a67fc700 1 -- 172.21.15.31:0/2135797921 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f03b0007300 con 0x7f03b8153d40 2023-10-06T05:23:58.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.894+0000 7f03a67fc700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03980674d0 0x7f0398069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:58.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.894+0000 7f03a67fc700 1 -- 172.21.15.31:0/2135797921 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5348+0+0 (secure 0 0 0) 0x7f03b0085170 con 0x7f03b8153d40 2023-10-06T05:23:58.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.895+0000 7f03b659c700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03980674d0 0x7f0398069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:58.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.896+0000 7f03b659c700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03980674d0 0x7f0398069980 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03a80091e0 tx=0x7f03a8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:58.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:58.899+0000 7f03a67fc700 1 -- 172.21.15.31:0/2135797921 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03b0051dc0 con 0x7f03b8153d40 2023-10-06T05:23:59.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.101+0000 7f03bcdff700 1 -- 172.21.15.31:0/2135797921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f03b804c380 con 0x7f03b8153d40 2023-10-06T05:23:59.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.105+0000 7f03a67fc700 1 -- 172.21.15.31:0/2135797921 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f03b00553e0 con 0x7f03b8153d40 2023-10-06T05:23:59.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.107+0000 7f03bcdff700 1 -- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03980674d0 msgr2=0x7f0398069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.107+0000 7f03bcdff700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03980674d0 0x7f0398069980 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03a80091e0 tx=0x7f03a8017000 comp rx=0 tx=0).stop 2023-10-06T05:23:59.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.107+0000 7f03bcdff700 1 -- 172.21.15.31:0/2135797921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 msgr2=0x7f03b80c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.107+0000 7f03bcdff700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 0x7f03b80c95e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f03b000cc40 tx=0x7f03b0001a50 comp rx=0 tx=0).stop 2023-10-06T05:23:59.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.107+0000 7f03bcdff700 1 -- 172.21.15.31:0/2135797921 shutdown_connections 2023-10-06T05:23:59.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.107+0000 7f03bcdff700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03980674d0 0x7f0398069980 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.107+0000 7f03bcdff700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b8153d40 0x7f03b80c95e0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.107+0000 7f03bcdff700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b80af6c0 0x7f03b8142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.107+0000 7f03bcdff700 1 --2- 172.21.15.31:0/2135797921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03b80b09c0 0x7f03b814b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.107+0000 7f03bcdff700 1 -- 172.21.15.31:0/2135797921 >> 172.21.15.31:0/2135797921 conn(0x7f03b8067560 msgr2=0x7f03b80abef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:59.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.108+0000 7f03bcdff700 1 -- 172.21.15.31:0/2135797921 shutdown_connections 2023-10-06T05:23:59.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.108+0000 7f03bcdff700 1 -- 172.21.15.31:0/2135797921 wait complete. 2023-10-06T05:23:59.126 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:23:59.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:59.226 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:23:59.340 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:23:59.340 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:23:59.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:23:59.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.534+0000 7ffa767ff700 1 Processor -- start 2023-10-06T05:23:59.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.534+0000 7ffa767ff700 1 -- start start 2023-10-06T05:23:59.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.536+0000 7ffa767ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffa700c7b60 0x7ffa70069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.536+0000 7ffa767ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700ca5b0 con 0x7ffa700aee70 2023-10-06T05:23:59.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.536+0000 7ffa767ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700af2d0 con 0x7ffa700c7f30 2023-10-06T05:23:59.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.536+0000 7ffa767ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7006f740 con 0x7ffa700c7b60 2023-10-06T05:23:59.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.536+0000 7ffa74d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffa700aee70 0x7ffa7013f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47390/0 (socket says 172.21.15.31:47390) 2023-10-06T05:23:59.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.536+0000 7ffa74d9c700 1 -- 172.21.15.31:0/3634344303 learned_addr learned my addr 172.21.15.31:0/3634344303 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.536+0000 7ffa6ffff700 1 --2- 172.21.15.31:0/3634344303 >> v2:172.21.15.62:3300/0 conn(0x7ffa700c7b60 0x7ffa70069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.536+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363436910 0 0) 0x7ffa700ca5b0 con 0x7ffa700aee70 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.536+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa58003570 con 0x7ffa700aee70 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2369879325 0 0) 0x7ffa58003570 con 0x7ffa700aee70 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 >> v2:172.21.15.62:3300/0 conn(0x7ffa700c7b60 msgr2=0x7ffa70069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 --2- 172.21.15.31:0/3634344303 >> v2:172.21.15.62:3300/0 conn(0x7ffa700c7b60 0x7ffa70069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 >> v1:172.21.15.62:6789/0 conn(0x7ffa700c7f30 legacy=0x7ffa7006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700c9c80 con 0x7ffa700aee70 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7ffa680017f0 con 0x7ffa700aee70 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 >> v1:172.21.15.31:6789/0 conn(0x7ffa700aee70 legacy=0x7ffa7013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa58003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa58004810 0x7ffa58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa58005210 0x7ffa5800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7006f740 con 0x7ffa58003ab0 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5800c130 con 0x7ffa58004810 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5800c2f0 con 0x7ffa58005210 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6ffff700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa58003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa6f7fe700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa58004810 0x7ffa58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.537+0000 7ffa74d9c700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa58005210 0x7ffa5800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.538+0000 7ffa6ffff700 1 -- 172.21.15.31:0/3634344303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa58005210 msgr2=0x7ffa5800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.538+0000 7ffa6ffff700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa58005210 0x7ffa5800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.538+0000 7ffa6ffff700 1 -- 172.21.15.31:0/3634344303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa58004810 msgr2=0x7ffa58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.538+0000 7ffa6ffff700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa58004810 0x7ffa58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.538+0000 7ffa6ffff700 1 -- 172.21.15.31:0/3634344303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58003570 con 0x7ffa58003ab0 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.538+0000 7ffa74d9c700 1 -- 172.21.15.31:0/3634344303 reap_dead start 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.538+0000 7ffa6ffff700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa58003ec0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009bf0 tx=0x7ffa60009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1274b61d0f1b33c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.538+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60005a00 con 0x7ffa58003ab0 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.538+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60005b60 con 0x7ffa58003ab0 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.538+0000 7ffa6effd700 1 -- 172.21.15.31:0/3634344303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000e710 con 0x7ffa58003ab0 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.539+0000 7ffa767ff700 1 -- 172.21.15.31:0/3634344303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 msgr2=0x7ffa58003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.539+0000 7ffa767ff700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa58003ec0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009bf0 tx=0x7ffa60009120 comp rx=0 tx=0).stop 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.539+0000 7ffa767ff700 1 -- 172.21.15.31:0/3634344303 shutdown_connections 2023-10-06T05:23:59.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.539+0000 7ffa767ff700 1 --2- 172.21.15.31:0/3634344303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa58003ec0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.539+0000 7ffa767ff700 1 -- 172.21.15.31:0/3634344303 >> 172.21.15.31:0/3634344303 conn(0x7ffa700cc0b0 msgr2=0x7ffa700cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:59.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.539+0000 7ffa767ff700 1 -- 172.21.15.31:0/3634344303 shutdown_connections 2023-10-06T05:23:59.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.539+0000 7ffa767ff700 1 -- 172.21.15.31:0/3634344303 wait complete. 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.540+0000 7ffa767ff700 1 Processor -- start 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.540+0000 7ffa767ff700 1 -- start start 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.541+0000 7ffa767ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa700adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.541+0000 7ffa767ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa700c57e0 0x7ffa700c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.541+0000 7ffa767ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7013de50 0x7ffa7013e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.541+0000 7ffa767ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700ab700 con 0x7ffa58003ab0 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.541+0000 7ffa767ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa70153bd0 con 0x7ffa7013de50 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.541+0000 7ffa767ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7013e7e0 con 0x7ffa700c57e0 2023-10-06T05:23:59.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa700adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa700adbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35878/0 (socket says 172.21.15.31:35878) 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa6ffff700 1 -- 172.21.15.31:0/3799290493 learned_addr learned my addr 172.21.15.31:0/3799290493 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa6f7fe700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa700c57e0 0x7ffa700c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa74d9c700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7013de50 0x7ffa7013e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa6ffff700 1 -- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa700c57e0 msgr2=0x7ffa700c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa6ffff700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa700c57e0 0x7ffa700c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa6ffff700 1 -- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7013de50 msgr2=0x7ffa7013e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa6ffff700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7013de50 0x7ffa7013e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa6ffff700 1 -- 172.21.15.31:0/3799290493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700c3ef0 con 0x7ffa58003ab0 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.542+0000 7ffa6ffff700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa700adbe0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ffa580057d0 tx=0x7ffa60019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.543+0000 7ffa57fff700 1 -- 172.21.15.31:0/3799290493 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60022070 con 0x7ffa58003ab0 2023-10-06T05:23:59.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.543+0000 7ffa57fff700 1 -- 172.21.15.31:0/3799290493 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60006240 con 0x7ffa58003ab0 2023-10-06T05:23:59.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.543+0000 7ffa57fff700 1 -- 172.21.15.31:0/3799290493 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60005a00 con 0x7ffa58003ab0 2023-10-06T05:23:59.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.543+0000 7ffa767ff700 1 -- 172.21.15.31:0/3799290493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa700c41d0 con 0x7ffa58003ab0 2023-10-06T05:23:59.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.543+0000 7ffa767ff700 1 -- 172.21.15.31:0/3799290493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa70153860 con 0x7ffa58003ab0 2023-10-06T05:23:59.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.544+0000 7ffa767ff700 1 -- 172.21.15.31:0/3799290493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7004c380 con 0x7ffa58003ab0 2023-10-06T05:23:59.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.545+0000 7ffa57fff700 1 -- 172.21.15.31:0/3799290493 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ffa60013470 con 0x7ffa58003ab0 2023-10-06T05:23:59.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.545+0000 7ffa57fff700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa5c0674d0 0x7ffa5c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:23:59.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.546+0000 7ffa57fff700 1 -- 172.21.15.31:0/3799290493 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5348+0+0 (secure 0 0 0) 0x7ffa60086280 con 0x7ffa58003ab0 2023-10-06T05:23:59.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.546+0000 7ffa6f7fe700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa5c0674d0 0x7ffa5c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:23:59.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.546+0000 7ffa6f7fe700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa5c0674d0 0x7ffa5c069980 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa700c7450 tx=0x7ffa64007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:23:59.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.550+0000 7ffa57fff700 1 -- 172.21.15.31:0/3799290493 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa60052de0 con 0x7ffa58003ab0 2023-10-06T05:23:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.803+0000 7ffa767ff700 1 -- 172.21.15.31:0/3799290493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa700ab700 con 0x7ffa58003ab0 2023-10-06T05:23:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.804+0000 7ffa57fff700 1 -- 172.21.15.31:0/3799290493 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+3526 (secure 0 0 0) 0x7ffa60006930 con 0x7ffa58003ab0 2023-10-06T05:23:59.806 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:23:59.806 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":78,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6557,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1866061469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1866061469},{"type":"v1","addr":"172.21.15.31:6834","nonce":1866061469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":6803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4084862869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":4084862869},{"type":"v1","addr":"172.21.15.62:6835","nonce":4084862869}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78},{"gid":7366,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/257113943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":257113943},{"type":"v1","addr":"172.21.15.31:6835","nonce":257113943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2082524951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2082524951},{"type":"v1","addr":"172.21.15.62:6834","nonce":2082524951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[]} 2023-10-06T05:23:59.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 -- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa5c0674d0 msgr2=0x7ffa5c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa5c0674d0 0x7ffa5c069980 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa700c7450 tx=0x7ffa64007d30 comp rx=0 tx=0).stop 2023-10-06T05:23:59.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 -- 172.21.15.31:0/3799290493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 msgr2=0x7ffa700adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:23:59.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa700adbe0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ffa580057d0 tx=0x7ffa60019000 comp rx=0 tx=0).stop 2023-10-06T05:23:59.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 -- 172.21.15.31:0/3799290493 shutdown_connections 2023-10-06T05:23:59.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa5c0674d0 0x7ffa5c069980 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7013de50 0x7ffa7013e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa58003ab0 0x7ffa700adbe0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 --2- 172.21.15.31:0/3799290493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa700c57e0 0x7ffa700c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:23:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 -- 172.21.15.31:0/3799290493 >> 172.21.15.31:0/3799290493 conn(0x7ffa700cc0b0 msgr2=0x7ffa700cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:23:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 -- 172.21.15.31:0/3799290493 shutdown_connections 2023-10-06T05:23:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:23:59.806+0000 7ffa767ff700 1 -- 172.21.15.31:0/3799290493 wait complete. 2023-10-06T05:23:59.809 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 78 2023-10-06T05:23:59.826 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:23:59.826 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:23:59.826 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:23:59.826 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:59.826 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:23:59.826 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:23:59.827 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:23:59.827 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:59.827 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:23:59.827 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:23:59.827 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:23:59.827 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:59.827 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:23:59.828 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:23:59.828 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:23:59.828 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:23:59.839 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:23:59.840 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:23:59.840 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:23:59.841 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:23:59.847 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:23:59.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:23:59.848 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:23:59.848 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:23:59.884 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:23:59.885 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:59.887 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:23:59.887 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:59.892 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:23:59.893 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:59.895 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:23:59.895 DEBUG:teuthology.parallel:result is None 2023-10-06T05:23:59.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:24:00.035 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:24:00.035 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:24:00.046 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:24:00.046 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:24:00.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.108+0000 7fb329dff700 1 Processor -- start 2023-10-06T05:24:00.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.108+0000 7fb329dff700 1 -- start start 2023-10-06T05:24:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.109+0000 7fb329dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3240c7b60 0x7fb324069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.109+0000 7fb329dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3240aee70 0x7fb32413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.109+0000 7fb329dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3240ca5b0 con 0x7fb3240c7f30 2023-10-06T05:24:00.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.109+0000 7fb329dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3240af2d0 con 0x7fb3240aee70 2023-10-06T05:24:00.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.109+0000 7fb329dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32406f740 con 0x7fb3240c7b60 2023-10-06T05:24:00.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.110+0000 7fb323fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3240aee70 0x7fb32413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.110+0000 7fb3237fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3240c7b60 0x7fb324069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.110+0000 7fb323fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3240aee70 0x7fb32413f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35898/0 (socket says 172.21.15.31:35898) 2023-10-06T05:24:00.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.110+0000 7fb323fff700 1 -- 172.21.15.31:0/3169528599 learned_addr learned my addr 172.21.15.31:0/3169528599 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:00.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.110+0000 7fb323fff700 1 -- 172.21.15.31:0/3169528599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3240c7b60 msgr2=0x7fb324069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.110+0000 7fb323fff700 1 --2- 172.21.15.31:0/3169528599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3240c7b60 0x7fb324069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.110+0000 7fb323fff700 1 -- 172.21.15.31:0/3169528599 >> v1:172.21.15.31:6789/0 conn(0x7fb3240c7f30 legacy=0x7fb32406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.110+0000 7fb323fff700 1 -- 172.21.15.31:0/3169528599 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3240c9c80 con 0x7fb3240aee70 2023-10-06T05:24:00.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.110+0000 7fb323fff700 1 --2- 172.21.15.31:0/3169528599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3240aee70 0x7fb32413f690 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009f20 tx=0x7fb31c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3427600139d0f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.113+0000 7fb3227fc700 1 -- 172.21.15.31:0/3169528599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c00e070 con 0x7fb3240aee70 2023-10-06T05:24:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.113+0000 7fb3227fc700 1 -- 172.21.15.31:0/3169528599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c00b9d0 con 0x7fb3240aee70 2023-10-06T05:24:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.113+0000 7fb3227fc700 1 -- 172.21.15.31:0/3169528599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c0092e0 con 0x7fb3240aee70 2023-10-06T05:24:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.114+0000 7fb329dff700 1 -- 172.21.15.31:0/3169528599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3240aee70 msgr2=0x7fb32413f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.114+0000 7fb329dff700 1 --2- 172.21.15.31:0/3169528599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3240aee70 0x7fb32413f690 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009f20 tx=0x7fb31c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:00.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.114+0000 7fb329dff700 1 -- 172.21.15.31:0/3169528599 shutdown_connections 2023-10-06T05:24:00.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.114+0000 7fb329dff700 1 --2- 172.21.15.31:0/3169528599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3240aee70 0x7fb32413f690 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.114+0000 7fb329dff700 1 --2- 172.21.15.31:0/3169528599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3240c7b60 0x7fb324069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.114+0000 7fb329dff700 1 -- 172.21.15.31:0/3169528599 >> 172.21.15.31:0/3169528599 conn(0x7fb3240cc0b0 msgr2=0x7fb3240cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:00.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.114+0000 7fb329dff700 1 -- 172.21.15.31:0/3169528599 shutdown_connections 2023-10-06T05:24:00.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.114+0000 7fb329dff700 1 -- 172.21.15.31:0/3169528599 wait complete. 2023-10-06T05:24:00.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.115+0000 7fb329dff700 1 Processor -- start 2023-10-06T05:24:00.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.116+0000 7fb329dff700 1 -- start start 2023-10-06T05:24:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.117+0000 7fb329dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3240aee70 0x7fb3240ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.117+0000 7fb329dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3240c7b60 0x7fb3240c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.117+0000 7fb329dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3240c7f30 0x7fb324154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.117+0000 7fb329dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240ae7a0 con 0x7fb3240c7b60 2023-10-06T05:24:00.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.117+0000 7fb329dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240adfb0 con 0x7fb3240aee70 2023-10-06T05:24:00.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.117+0000 7fb329dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240c4cc0 con 0x7fb3240c7f30 2023-10-06T05:24:00.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb3237fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3240aee70 0x7fb3240ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb322ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3240c7b60 0x7fb3240c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb3237fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3240aee70 0x7fb3240ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:50380/0 (socket says 172.21.15.31:50380) 2023-10-06T05:24:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb3237fe700 1 -- 172.21.15.31:0/3742111864 learned_addr learned my addr 172.21.15.31:0/3742111864 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb323fff700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3240c7f30 0x7fb324154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb322ffd700 1 -- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3240c7f30 msgr2=0x7fb324154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb322ffd700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3240c7f30 0x7fb324154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb322ffd700 1 -- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3240aee70 msgr2=0x7fb3240ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:00.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb322ffd700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3240aee70 0x7fb3240ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:00.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb322ffd700 1 -- 172.21.15.31:0/3742111864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c009ae0 con 0x7fb3240c7b60 2023-10-06T05:24:00.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.118+0000 7fb322ffd700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3240c7b60 0x7fb3240c4780 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fb31800c490 tx=0x7fb318007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.121+0000 7fb30b7fe700 1 -- 172.21.15.31:0/3742111864 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318012420 con 0x7fb3240c7b60 2023-10-06T05:24:00.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.121+0000 7fb30b7fe700 1 -- 172.21.15.31:0/3742111864 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3180125a0 con 0x7fb3240c7b60 2023-10-06T05:24:00.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.121+0000 7fb329dff700 1 -- 172.21.15.31:0/3742111864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32412b810 con 0x7fb3240c7b60 2023-10-06T05:24:00.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.121+0000 7fb30b7fe700 1 -- 172.21.15.31:0/3742111864 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318012960 con 0x7fb3240c7b60 2023-10-06T05:24:00.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.121+0000 7fb329dff700 1 -- 172.21.15.31:0/3742111864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3240ae4c0 con 0x7fb3240c7b60 2023-10-06T05:24:00.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.122+0000 7fb329dff700 1 -- 172.21.15.31:0/3742111864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32412b9a0 con 0x7fb3240c7b60 2023-10-06T05:24:00.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.123+0000 7fb30b7fe700 1 -- 172.21.15.31:0/3742111864 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb31801b3b0 con 0x7fb3240c7b60 2023-10-06T05:24:00.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.124+0000 7fb30b7fe700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3100674d0 0x7fb310069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:00.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.124+0000 7fb30b7fe700 1 -- 172.21.15.31:0/3742111864 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fb3180843c0 con 0x7fb3240c7b60 2023-10-06T05:24:00.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.124+0000 7fb3237fe700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3100674d0 0x7fb310069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:00.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.125+0000 7fb3237fe700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3100674d0 0x7fb310069980 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb32412e440 tx=0x7fb314009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:00.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.128+0000 7fb30b7fe700 1 -- 172.21.15.31:0/3742111864 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb318050f00 con 0x7fb3240c7b60 2023-10-06T05:24:00.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:00.348+0000 7fb329dff700 1 -- 172.21.15.31:0/3742111864 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb32404c3d0 con 0x7fb3240c7b60 2023-10-06T05:24:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.126+0000 7fb30b7fe700 1 -- 172.21.15.31:0/3742111864 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v123) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb318054520 con 0x7fb3240c7b60 2023-10-06T05:24:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 -- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3100674d0 msgr2=0x7fb310069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3100674d0 0x7fb310069980 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb32412e440 tx=0x7fb314009120 comp rx=0 tx=0).stop 2023-10-06T05:24:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 -- 172.21.15.31:0/3742111864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3240c7b60 msgr2=0x7fb3240c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3240c7b60 0x7fb3240c4780 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fb31800c490 tx=0x7fb318007ec0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 -- 172.21.15.31:0/3742111864 shutdown_connections 2023-10-06T05:24:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3100674d0 0x7fb310069980 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3240c7f30 0x7fb324154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3240aee70 0x7fb3240ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 --2- 172.21.15.31:0/3742111864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3240c7b60 0x7fb3240c4780 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 -- 172.21.15.31:0/3742111864 >> 172.21.15.31:0/3742111864 conn(0x7fb3240cc0b0 msgr2=0x7fb3240cb140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 -- 172.21.15.31:0/3742111864 shutdown_connections 2023-10-06T05:24:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.129+0000 7fb329dff700 1 -- 172.21.15.31:0/3742111864 wait complete. 2023-10-06T05:24:01.132 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:24:01.153 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:24:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.357+0000 7ff4417ff700 1 Processor -- start 2023-10-06T05:24:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.358+0000 7ff4417ff700 1 -- start start 2023-10-06T05:24:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.359+0000 7ff4417ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff43c0c7b60 0x7ff43c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.359+0000 7ff4417ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c0ca5b0 con 0x7ff43c0aee70 2023-10-06T05:24:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.359+0000 7ff4417ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c0af2d0 con 0x7ff43c0c7f30 2023-10-06T05:24:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.359+0000 7ff4417ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c06f740 con 0x7ff43c0c7b60 2023-10-06T05:24:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.359+0000 7ff43affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff43c0c7b60 0x7ff43c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.359+0000 7ff43affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff43c0c7b60 0x7ff43c069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35918/0 (socket says 172.21.15.31:35918) 2023-10-06T05:24:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.359+0000 7ff43affd700 1 -- 172.21.15.31:0/1750356651 learned_addr learned my addr 172.21.15.31:0/1750356651 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.360+0000 7ff43affd700 1 -- 172.21.15.31:0/1750356651 >> v1:172.21.15.62:6789/0 conn(0x7ff43c0c7f30 legacy=0x7ff43c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.360+0000 7ff43affd700 1 -- 172.21.15.31:0/1750356651 >> v1:172.21.15.31:6789/0 conn(0x7ff43c0aee70 legacy=0x7ff43c13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.360+0000 7ff43affd700 1 -- 172.21.15.31:0/1750356651 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0c9c80 con 0x7ff43c0c7b60 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.360+0000 7ff43affd700 1 --2- 172.21.15.31:0/1750356651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 0x7ff43c069880 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009990 tx=0x7ff42c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db071fc3f563d21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.360+0000 7ff439ffb700 1 -- 172.21.15.31:0/1750356651 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c005c10 con 0x7ff43c0c7b60 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.360+0000 7ff439ffb700 1 -- 172.21.15.31:0/1750356651 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c005d70 con 0x7ff43c0c7b60 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.360+0000 7ff439ffb700 1 -- 172.21.15.31:0/1750356651 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c00b840 con 0x7ff43c0c7b60 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.361+0000 7ff4417ff700 1 -- 172.21.15.31:0/1750356651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 msgr2=0x7ff43c069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.361+0000 7ff4417ff700 1 --2- 172.21.15.31:0/1750356651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 0x7ff43c069880 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009990 tx=0x7ff42c009220 comp rx=0 tx=0).stop 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.361+0000 7ff4417ff700 1 -- 172.21.15.31:0/1750356651 shutdown_connections 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.361+0000 7ff4417ff700 1 --2- 172.21.15.31:0/1750356651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 0x7ff43c069880 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.361+0000 7ff4417ff700 1 -- 172.21.15.31:0/1750356651 >> 172.21.15.31:0/1750356651 conn(0x7ff43c0cc0b0 msgr2=0x7ff43c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.361+0000 7ff4417ff700 1 -- 172.21.15.31:0/1750356651 shutdown_connections 2023-10-06T05:24:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.361+0000 7ff4417ff700 1 -- 172.21.15.31:0/1750356651 wait complete. 2023-10-06T05:24:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.362+0000 7ff4417ff700 1 Processor -- start 2023-10-06T05:24:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.363+0000 7ff4417ff700 1 -- start start 2023-10-06T05:24:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.363+0000 7ff4417ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff43c0aee70 0x7ff43c16bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.363+0000 7ff4417ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 0x7ff43c15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.363+0000 7ff4417ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff43c0c7f30 0x7ff43c0add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff4417ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0c46f0 con 0x7ff43c0c7b60 2023-10-06T05:24:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff4417ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c15ce20 con 0x7ff43c0aee70 2023-10-06T05:24:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff4417ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0ad010 con 0x7ff43c0c7f30 2023-10-06T05:24:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 0x7ff43c15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 0x7ff43c15c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35922/0 (socket says 172.21.15.31:35922) 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43a7fc700 1 -- 172.21.15.31:0/1934870947 learned_addr learned my addr 172.21.15.31:0/1934870947 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43b7fe700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff43c0c7f30 0x7ff43c0add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43affd700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff43c0aee70 0x7ff43c16bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43a7fc700 1 -- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff43c0c7f30 msgr2=0x7ff43c0add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43a7fc700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff43c0c7f30 0x7ff43c0add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43a7fc700 1 -- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff43c0aee70 msgr2=0x7ff43c16bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43a7fc700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff43c0aee70 0x7ff43c16bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43a7fc700 1 -- 172.21.15.31:0/1934870947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c009480 con 0x7ff43c0c7b60 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43affd700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff43c0aee70 0x7ff43c16bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.364+0000 7ff43a7fc700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 0x7ff43c15c8e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b5e0 tx=0x7ff430009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:01.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.365+0000 7ff422ffd700 1 -- 172.21.15.31:0/1934870947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43000f510 con 0x7ff43c0c7b60 2023-10-06T05:24:01.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.365+0000 7ff422ffd700 1 -- 172.21.15.31:0/1934870947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43000f670 con 0x7ff43c0c7b60 2023-10-06T05:24:01.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.365+0000 7ff422ffd700 1 -- 172.21.15.31:0/1934870947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43000fa30 con 0x7ff43c0c7b60 2023-10-06T05:24:01.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.365+0000 7ff4417ff700 1 -- 172.21.15.31:0/1934870947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c0ad2f0 con 0x7ff43c0c7b60 2023-10-06T05:24:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.365+0000 7ff4417ff700 1 -- 172.21.15.31:0/1934870947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c0c4380 con 0x7ff43c0c7b60 2023-10-06T05:24:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.366+0000 7ff4417ff700 1 -- 172.21.15.31:0/1934870947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c0c48a0 con 0x7ff43c0c7b60 2023-10-06T05:24:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.367+0000 7ff422ffd700 1 -- 172.21.15.31:0/1934870947 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ff43001a400 con 0x7ff43c0c7b60 2023-10-06T05:24:01.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.368+0000 7ff422ffd700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff4280674d0 0x7ff428069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.368+0000 7ff422ffd700 1 -- 172.21.15.31:0/1934870947 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5234+0+0 (secure 0 0 0) 0x7ff430084870 con 0x7ff43c0c7b60 2023-10-06T05:24:01.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.369+0000 7ff43affd700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff4280674d0 0x7ff428069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.369+0000 7ff43affd700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff4280674d0 0x7ff428069980 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009260 tx=0x7ff42c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:01.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.372+0000 7ff422ffd700 1 -- 172.21.15.31:0/1934870947 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff43c0c48a0 con 0x7ff43c0c7b60 2023-10-06T05:24:01.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.652+0000 7ff4417ff700 1 -- 172.21.15.31:0/1934870947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff43c04c380 con 0x7ff43c0c7b60 2023-10-06T05:24:01.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.652+0000 7ff422ffd700 1 -- 172.21.15.31:0/1934870947 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff4300514c0 con 0x7ff43c0c7b60 2023-10-06T05:24:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:01.657 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:24:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:24:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:24:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:24:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:24:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:24:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:24:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:24:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:24:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:24:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:24:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:24:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:24:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:24:01.673 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:24:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 -- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff4280674d0 msgr2=0x7ff428069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff4280674d0 0x7ff428069980 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009260 tx=0x7ff42c015000 comp rx=0 tx=0).stop 2023-10-06T05:24:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 -- 172.21.15.31:0/1934870947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 msgr2=0x7ff43c15c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 0x7ff43c15c8e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b5e0 tx=0x7ff430009120 comp rx=0 tx=0).stop 2023-10-06T05:24:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 -- 172.21.15.31:0/1934870947 shutdown_connections 2023-10-06T05:24:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff4280674d0 0x7ff428069980 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff43c0c7f30 0x7ff43c0add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff43c0aee70 0x7ff43c16bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 --2- 172.21.15.31:0/1934870947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff43c0c7b60 0x7ff43c15c8e0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 -- 172.21.15.31:0/1934870947 >> 172.21.15.31:0/1934870947 conn(0x7ff43c0cc0b0 msgr2=0x7ff43c13ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 -- 172.21.15.31:0/1934870947 shutdown_connections 2023-10-06T05:24:01.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.655+0000 7ff4417ff700 1 -- 172.21.15.31:0/1934870947 wait complete. 2023-10-06T05:24:01.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:01.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.869+0000 7f006c5ff700 1 Processor -- start 2023-10-06T05:24:01.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.869+0000 7f006c5ff700 1 -- start start 2023-10-06T05:24:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.871+0000 7f006c5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f006413a1e0 0x7f00640c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.871+0000 7f006c5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a5b0 0x7f006406d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.871+0000 7f006c5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00640c71b0 con 0x7f00640c6c30 2023-10-06T05:24:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.871+0000 7f006c5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00640c7490 con 0x7f006413a1e0 2023-10-06T05:24:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.871+0000 7f006c5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00640c7770 con 0x7f006413a5b0 2023-10-06T05:24:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.871+0000 7f006ab9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f00640c6c30 0x7f006415afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47444/0 (socket says 172.21.15.31:47444) 2023-10-06T05:24:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.871+0000 7f006ab9c700 1 -- 172.21.15.31:0/1434290122 learned_addr learned my addr 172.21.15.31:0/1434290122 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.871+0000 7f0069b9a700 1 --2- 172.21.15.31:0/1434290122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a5b0 0x7f006406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.871+0000 7f006a39b700 1 --2- 172.21.15.31:0/1434290122 >> v2:172.21.15.62:3300/0 conn(0x7f006413a1e0 0x7f00640c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.872+0000 7f0069399700 1 -- 172.21.15.31:0/1434290122 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211438448 0 0) 0x7f00640c71b0 con 0x7f00640c6c30 2023-10-06T05:24:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.872+0000 7f0069399700 1 -- 172.21.15.31:0/1434290122 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0050003570 con 0x7f00640c6c30 2023-10-06T05:24:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.872+0000 7f0069b9a700 1 -- 172.21.15.31:0/1434290122 >> v2:172.21.15.62:3300/0 conn(0x7f006413a1e0 msgr2=0x7f00640c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.872+0000 7f0069b9a700 1 --2- 172.21.15.31:0/1434290122 >> v2:172.21.15.62:3300/0 conn(0x7f006413a1e0 0x7f00640c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.872+0000 7f0069b9a700 1 -- 172.21.15.31:0/1434290122 >> v1:172.21.15.31:6789/0 conn(0x7f00640c6c30 legacy=0x7f006415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.872+0000 7f0069b9a700 1 -- 172.21.15.31:0/1434290122 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00640c7a50 con 0x7f006413a5b0 2023-10-06T05:24:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.872+0000 7f0069b9a700 1 --2- 172.21.15.31:0/1434290122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a5b0 0x7f006406d440 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00540098e0 tx=0x7f00540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfeafe56ba16dc33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.872+0000 7f0069399700 1 -- 172.21.15.31:0/1434290122 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054011630 con 0x7f006413a5b0 2023-10-06T05:24:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.873+0000 7f0069399700 1 -- 172.21.15.31:0/1434290122 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0054011790 con 0x7f006413a5b0 2023-10-06T05:24:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.873+0000 7f0069399700 1 -- 172.21.15.31:0/1434290122 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054011b30 con 0x7f006413a5b0 2023-10-06T05:24:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.873+0000 7f006c5ff700 1 -- 172.21.15.31:0/1434290122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a5b0 msgr2=0x7f006406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.873+0000 7f006c5ff700 1 --2- 172.21.15.31:0/1434290122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a5b0 0x7f006406d440 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00540098e0 tx=0x7f00540091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.873+0000 7f006c5ff700 1 -- 172.21.15.31:0/1434290122 shutdown_connections 2023-10-06T05:24:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.873+0000 7f006c5ff700 1 --2- 172.21.15.31:0/1434290122 >> v2:172.21.15.62:3300/0 conn(0x7f006413a1e0 0x7f00640c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.873+0000 7f006c5ff700 1 --2- 172.21.15.31:0/1434290122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a5b0 0x7f006406d440 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.873+0000 7f006c5ff700 1 -- 172.21.15.31:0/1434290122 >> 172.21.15.31:0/1434290122 conn(0x7f00640cc0b0 msgr2=0x7f00640cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.873+0000 7f006c5ff700 1 -- 172.21.15.31:0/1434290122 shutdown_connections 2023-10-06T05:24:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.873+0000 7f006c5ff700 1 -- 172.21.15.31:0/1434290122 wait complete. 2023-10-06T05:24:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.875+0000 7f006c5ff700 1 Processor -- start 2023-10-06T05:24:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.875+0000 7f006c5ff700 1 -- start start 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006c5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00640c6c30 0x7f00640afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006c5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a1e0 0x7f00640c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006c5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f006413a5b0 0x7f00640c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006c5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00640c3850 con 0x7f00640c6c30 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006c5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00641545d0 con 0x7f006413a5b0 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006c5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00641542f0 con 0x7f006413a1e0 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006a39b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00640c6c30 0x7f00640afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006a39b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00640c6c30 0x7f00640afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35928/0 (socket says 172.21.15.31:35928) 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006a39b700 1 -- 172.21.15.31:0/3051324475 learned_addr learned my addr 172.21.15.31:0/3051324475 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:01.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f0069b9a700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a1e0 0x7f00640c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006a39b700 1 -- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a1e0 msgr2=0x7f00640c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006ab9c700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f006413a5b0 0x7f00640c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006a39b700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a1e0 0x7f00640c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006a39b700 1 -- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f006413a5b0 msgr2=0x7f00640c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006a39b700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f006413a5b0 0x7f00640c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f006a39b700 1 -- 172.21.15.31:0/3051324475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0054009420 con 0x7f00640c6c30 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.876+0000 7f0069b9a700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a1e0 0x7f00640c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.877+0000 7f006a39b700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00640c6c30 0x7f00640afb80 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f006000ec80 tx=0x7f0060009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.877+0000 7f005a7fc700 1 -- 172.21.15.31:0/3051324475 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000ca60 con 0x7f00640c6c30 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.877+0000 7f005a7fc700 1 -- 172.21.15.31:0/3051324475 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006000cbc0 con 0x7f00640c6c30 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.877+0000 7f005a7fc700 1 -- 172.21.15.31:0/3051324475 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060013470 con 0x7f00640c6c30 2023-10-06T05:24:01.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.877+0000 7f006c5ff700 1 -- 172.21.15.31:0/3051324475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00640c3200 con 0x7f00640c6c30 2023-10-06T05:24:01.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.877+0000 7f006c5ff700 1 -- 172.21.15.31:0/3051324475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00640c5e60 con 0x7f00640c6c30 2023-10-06T05:24:01.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.878+0000 7f006c5ff700 1 -- 172.21.15.31:0/3051324475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006404c380 con 0x7f00640c6c30 2023-10-06T05:24:01.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.879+0000 7f005a7fc700 1 -- 172.21.15.31:0/3051324475 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0060013860 con 0x7f00640c6c30 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.880+0000 7f005a7fc700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00480674d0 0x7f0048069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.880+0000 7f005a7fc700 1 -- 172.21.15.31:0/3051324475 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f00600854e0 con 0x7f00640c6c30 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.880+0000 7f0069b9a700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00480674d0 0x7f0048069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:01.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.881+0000 7f0069b9a700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00480674d0 0x7f0048069980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f006416d1a0 tx=0x7f0054017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:01.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:01.884+0000 7f005a7fc700 1 -- 172.21.15.31:0/3051324475 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0060052130 con 0x7f00640c6c30 2023-10-06T05:24:02.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:02.010 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:02.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.092+0000 7f006c5ff700 1 -- 172.21.15.31:0/3051324475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00640c7d30 con 0x7f00640c6c30 2023-10-06T05:24:02.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.093+0000 7f005a7fc700 1 -- 172.21.15.31:0/3051324475 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+13752 (secure 0 0 0) 0x7f0060055750 con 0x7f00640c6c30 2023-10-06T05:24:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:02.095 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":123,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:01.119619+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 -- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00480674d0 msgr2=0x7f0048069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00480674d0 0x7f0048069980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f006416d1a0 tx=0x7f0054017000 comp rx=0 tx=0).stop 2023-10-06T05:24:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 -- 172.21.15.31:0/3051324475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00640c6c30 msgr2=0x7f00640afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00640c6c30 0x7f00640afb80 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f006000ec80 tx=0x7f0060009120 comp rx=0 tx=0).stop 2023-10-06T05:24:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 -- 172.21.15.31:0/3051324475 shutdown_connections 2023-10-06T05:24:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f00480674d0 0x7f0048069980 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f006413a5b0 0x7f00640c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00640c6c30 0x7f00640afb80 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 --2- 172.21.15.31:0/3051324475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f006413a1e0 0x7f00640c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 -- 172.21.15.31:0/3051324475 >> 172.21.15.31:0/3051324475 conn(0x7f00640cc0b0 msgr2=0x7f00640b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 -- 172.21.15.31:0/3051324475 shutdown_connections 2023-10-06T05:24:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.095+0000 7f006c5ff700 1 -- 172.21.15.31:0/3051324475 wait complete. 2023-10-06T05:24:02.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:24:02.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.352+0000 7fe17c5ff700 1 Processor -- start 2023-10-06T05:24:02.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.352+0000 7fe17c5ff700 1 -- start start 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.353+0000 7fe17c5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe1740af890 0x7fe174161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.353+0000 7fe17c5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1740afc90 con 0x7fe174153d40 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.353+0000 7fe17c5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1740ad690 con 0x7fe1740b0b90 2023-10-06T05:24:02.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.353+0000 7fe17c5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740acd60 con 0x7fe1740af890 2023-10-06T05:24:02.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe17a39b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe174153d40 0x7fe1740b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47450/0 (socket says 172.21.15.31:47450) 2023-10-06T05:24:02.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe17a39b700 1 -- 172.21.15.31:0/4014156117 learned_addr learned my addr 172.21.15.31:0/4014156117 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe17ab9c700 1 --2- 172.21.15.31:0/4014156117 >> v2:172.21.15.62:3300/0 conn(0x7fe1740af890 0x7fe174161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669140571 0 0) 0x7fe170001110 con 0x7fe174153d40 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe160003570 con 0x7fe174153d40 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3995438417 0 0) 0x7fe160003570 con 0x7fe174153d40 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 >> v2:172.21.15.62:3300/0 conn(0x7fe1740af890 msgr2=0x7fe174161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe179399700 1 --2- 172.21.15.31:0/4014156117 >> v2:172.21.15.62:3300/0 conn(0x7fe1740af890 0x7fe174161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 >> v1:172.21.15.62:6789/0 conn(0x7fe1740b0b90 legacy=0x7fe1740af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1740ad040 con 0x7fe174153d40 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fe170001600 con 0x7fe174153d40 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.354+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 >> v1:172.21.15.31:6789/0 conn(0x7fe174153d40 legacy=0x7fe1740b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe179399700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe160003a60 0x7fe160003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe179399700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe160004810 0x7fe160004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe179399700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1600051c0 0x7fe16000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1740acd60 con 0x7fe160003a60 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16000c050 con 0x7fe160004810 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16000c210 con 0x7fe1600051c0 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe17a39b700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe160003a60 0x7fe160003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe179b9a700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe160004810 0x7fe160004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe17ab9c700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1600051c0 0x7fe16000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe17a39b700 1 -- 172.21.15.31:0/4014156117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1600051c0 msgr2=0x7fe16000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe17a39b700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1600051c0 0x7fe16000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe17a39b700 1 -- 172.21.15.31:0/4014156117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe160004810 msgr2=0x7fe160004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe17a39b700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe160004810 0x7fe160004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe17a39b700 1 -- 172.21.15.31:0/4014156117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe160003570 con 0x7fe160003a60 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe17ab9c700 1 -- 172.21.15.31:0/4014156117 reap_dead start 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.355+0000 7fe17a39b700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe160003a60 0x7fe160003e30 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fe17000c5f0 tx=0x7fe170007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b303badb27b4d91e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.356+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000cdf0 con 0x7fe160003a60 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.356+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe170005920 con 0x7fe160003a60 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.356+0000 7fe179399700 1 -- 172.21.15.31:0/4014156117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170005ce0 con 0x7fe160003a60 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.356+0000 7fe17c5ff700 1 -- 172.21.15.31:0/4014156117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe160003a60 msgr2=0x7fe160003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.356+0000 7fe17c5ff700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe160003a60 0x7fe160003e30 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fe17000c5f0 tx=0x7fe170007a50 comp rx=0 tx=0).stop 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.356+0000 7fe17c5ff700 1 -- 172.21.15.31:0/4014156117 shutdown_connections 2023-10-06T05:24:02.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.356+0000 7fe17c5ff700 1 --2- 172.21.15.31:0/4014156117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe160003a60 0x7fe160003e30 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.356+0000 7fe17c5ff700 1 -- 172.21.15.31:0/4014156117 >> 172.21.15.31:0/4014156117 conn(0x7fe174067560 msgr2=0x7fe174067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.356+0000 7fe17c5ff700 1 -- 172.21.15.31:0/4014156117 shutdown_connections 2023-10-06T05:24:02.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.356+0000 7fe17c5ff700 1 -- 172.21.15.31:0/4014156117 wait complete. 2023-10-06T05:24:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.358+0000 7fe17c5ff700 1 Processor -- start 2023-10-06T05:24:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.358+0000 7fe17c5ff700 1 -- start start 2023-10-06T05:24:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe17c5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe160003a60 0x7fe1740a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe17c5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1740aaf20 0x7fe1740c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe17c5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe17416ce30 0x7fe17416d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe17c5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17416d810 con 0x7fe1740aaf20 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe17c5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1740a9e80 con 0x7fe17416ce30 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe17c5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1740aa160 con 0x7fe160003a60 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe179b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1740aaf20 0x7fe1740c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe179b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1740aaf20 0x7fe1740c2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35948/0 (socket says 172.21.15.31:35948) 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe179b9a700 1 -- 172.21.15.31:0/176321119 learned_addr learned my addr 172.21.15.31:0/176321119 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe17a39b700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe160003a60 0x7fe1740a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.359+0000 7fe17ab9c700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe17416ce30 0x7fe17416d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe179b9a700 1 -- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe160003a60 msgr2=0x7fe1740a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe179b9a700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe160003a60 0x7fe1740a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe179b9a700 1 -- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe17416ce30 msgr2=0x7fe17416d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe179b9a700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe17416ce30 0x7fe17416d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe179b9a700 1 -- 172.21.15.31:0/176321119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1740aa440 con 0x7fe1740aaf20 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe179b9a700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1740aaf20 0x7fe1740c2950 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fe164002b00 tx=0x7fe164007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe16a7fc700 1 -- 172.21.15.31:0/176321119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16400f850 con 0x7fe1740aaf20 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe16a7fc700 1 -- 172.21.15.31:0/176321119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16400f9b0 con 0x7fe1740aaf20 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe17c5ff700 1 -- 172.21.15.31:0/176321119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1740aa720 con 0x7fe1740aaf20 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe16a7fc700 1 -- 172.21.15.31:0/176321119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16400fd70 con 0x7fe1740aaf20 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.360+0000 7fe17c5ff700 1 -- 172.21.15.31:0/176321119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1740c0ed0 con 0x7fe1740aaf20 2023-10-06T05:24:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.361+0000 7fe17c5ff700 1 -- 172.21.15.31:0/176321119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1740aac40 con 0x7fe1740aaf20 2023-10-06T05:24:02.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.362+0000 7fe16a7fc700 1 -- 172.21.15.31:0/176321119 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fe164013600 con 0x7fe1740aaf20 2023-10-06T05:24:02.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.363+0000 7fe16a7fc700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe1580674d0 0x7fe158069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.363+0000 7fe16a7fc700 1 -- 172.21.15.31:0/176321119 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fe164085330 con 0x7fe1740aaf20 2023-10-06T05:24:02.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.364+0000 7fe17a39b700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe1580674d0 0x7fe158069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.364+0000 7fe17a39b700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe1580674d0 0x7fe158069980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe170007e50 tx=0x7fe170015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.367+0000 7fe16a7fc700 1 -- 172.21.15.31:0/176321119 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1740aac40 con 0x7fe1740aaf20 2023-10-06T05:24:02.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.572+0000 7fe17c5ff700 1 -- 172.21.15.31:0/176321119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe1740aac40 con 0x7fe1740aaf20 2023-10-06T05:24:02.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.573+0000 7fe16a7fc700 1 -- 172.21.15.31:0/176321119 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v123) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe1740aac40 con 0x7fe1740aaf20 2023-10-06T05:24:02.575 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:24:02.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 -- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe1580674d0 msgr2=0x7fe158069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe1580674d0 0x7fe158069980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe170007e50 tx=0x7fe170015000 comp rx=0 tx=0).stop 2023-10-06T05:24:02.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 -- 172.21.15.31:0/176321119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1740aaf20 msgr2=0x7fe1740c2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1740aaf20 0x7fe1740c2950 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fe164002b00 tx=0x7fe164007d10 comp rx=0 tx=0).stop 2023-10-06T05:24:02.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 -- 172.21.15.31:0/176321119 shutdown_connections 2023-10-06T05:24:02.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe1580674d0 0x7fe158069980 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe17416ce30 0x7fe17416d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe160003a60 0x7fe1740a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 --2- 172.21.15.31:0/176321119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1740aaf20 0x7fe1740c2950 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 -- 172.21.15.31:0/176321119 >> 172.21.15.31:0/176321119 conn(0x7fe174067560 msgr2=0x7fe1740c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 -- 172.21.15.31:0/176321119 shutdown_connections 2023-10-06T05:24:02.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.575+0000 7fe17c5ff700 1 -- 172.21.15.31:0/176321119 wait complete. 2023-10-06T05:24:02.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:24:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.791+0000 7f6b5d1ff700 1 Processor -- start 2023-10-06T05:24:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.791+0000 7f6b5d1ff700 1 -- start start 2023-10-06T05:24:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.792+0000 7f6b5d1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b58159890 0x7f6b58156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.792+0000 7f6b5d1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159d00 0x7f6b58156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.792+0000 7f6b5d1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58158190 con 0x7f6b580c89e0 2023-10-06T05:24:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.792+0000 7f6b5d1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b580c8db0 con 0x7f6b58159890 2023-10-06T05:24:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.792+0000 7f6b5d1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580cc780 con 0x7f6b58159d00 2023-10-06T05:24:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.792+0000 7f6b5659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b58159890 0x7f6b58156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.793+0000 7f6b5659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b58159890 0x7f6b58156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35952/0 (socket says 172.21.15.31:35952) 2023-10-06T05:24:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.793+0000 7f6b5659c700 1 -- 172.21.15.31:0/564156354 learned_addr learned my addr 172.21.15.31:0/564156354 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.793+0000 7f6b5759e700 1 --2- 172.21.15.31:0/564156354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159d00 0x7f6b58156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.793+0000 7f6b5659c700 1 -- 172.21.15.31:0/564156354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159d00 msgr2=0x7f6b58156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.793+0000 7f6b5659c700 1 --2- 172.21.15.31:0/564156354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159d00 0x7f6b58156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.793+0000 7f6b5659c700 1 -- 172.21.15.31:0/564156354 >> v1:172.21.15.31:6789/0 conn(0x7f6b580c89e0 legacy=0x7f6b580cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.793+0000 7f6b5659c700 1 -- 172.21.15.31:0/564156354 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b58157580 con 0x7f6b58159890 2023-10-06T05:24:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.793+0000 7f6b5759e700 1 --2- 172.21.15.31:0/564156354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159d00 0x7f6b58156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.793+0000 7f6b5659c700 1 --2- 172.21.15.31:0/564156354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159890 0x7f6b58156a60 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0099b0 tx=0x7f6b4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7370d2801a7fe46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b55d9b700 1 -- 172.21.15.31:0/564156354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c00b860 con 0x7f6b58159890 2023-10-06T05:24:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b55d9b700 1 -- 172.21.15.31:0/564156354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c00b9c0 con 0x7f6b58159890 2023-10-06T05:24:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b55d9b700 1 -- 172.21.15.31:0/564156354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c00bd60 con 0x7f6b58159890 2023-10-06T05:24:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/564156354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159890 msgr2=0x7f6b58156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b5d1ff700 1 --2- 172.21.15.31:0/564156354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159890 0x7f6b58156a60 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0099b0 tx=0x7f6b4c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/564156354 shutdown_connections 2023-10-06T05:24:02.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b5d1ff700 1 --2- 172.21.15.31:0/564156354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159d00 0x7f6b58156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b5d1ff700 1 --2- 172.21.15.31:0/564156354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159890 0x7f6b58156a60 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/564156354 >> 172.21.15.31:0/564156354 conn(0x7f6b58067560 msgr2=0x7f6b58067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:02.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/564156354 shutdown_connections 2023-10-06T05:24:02.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.794+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/564156354 wait complete. 2023-10-06T05:24:02.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.796+0000 7f6b5d1ff700 1 Processor -- start 2023-10-06T05:24:02.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.796+0000 7f6b5d1ff700 1 -- start start 2023-10-06T05:24:02.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.797+0000 7f6b5d1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b580c89e0 0x7f6b58146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.797+0000 7f6b5d1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159890 0x7f6b5814c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.797+0000 7f6b5d1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159d00 0x7f6b580af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.797+0000 7f6b5d1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580cb520 con 0x7f6b58159d00 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.797+0000 7f6b5d1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580cb770 con 0x7f6b580c89e0 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.797+0000 7f6b5d1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580cba50 con 0x7f6b58159890 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.797+0000 7f6b5759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159d00 0x7f6b580af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.797+0000 7f6b5759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159d00 0x7f6b580af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35954/0 (socket says 172.21.15.31:35954) 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.797+0000 7f6b5759e700 1 -- 172.21.15.31:0/2263599626 learned_addr learned my addr 172.21.15.31:0/2263599626 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b5659c700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159890 0x7f6b5814c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b56d9d700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b580c89e0 0x7f6b58146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b5759e700 1 -- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159890 msgr2=0x7f6b5814c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b5759e700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159890 0x7f6b5814c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b5759e700 1 -- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b580c89e0 msgr2=0x7f6b58146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:02.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b5759e700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b580c89e0 0x7f6b58146db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b5759e700 1 -- 172.21.15.31:0/2263599626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c0094f0 con 0x7f6b58159d00 2023-10-06T05:24:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b56d9d700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b580c89e0 0x7f6b58146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b5759e700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159d00 0x7f6b580af280 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009fc0 tx=0x7f6b500099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b3effd700 1 -- 172.21.15.31:0/2263599626 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5000dd40 con 0x7f6b58159d00 2023-10-06T05:24:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b3effd700 1 -- 172.21.15.31:0/2263599626 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5000f040 con 0x7f6b58159d00 2023-10-06T05:24:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b3effd700 1 -- 172.21.15.31:0/2263599626 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50015690 con 0x7f6b58159d00 2023-10-06T05:24:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.798+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/2263599626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b580cbd30 con 0x7f6b58159d00 2023-10-06T05:24:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.799+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/2263599626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b580cc2f0 con 0x7f6b58159d00 2023-10-06T05:24:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.800+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/2263599626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5804c380 con 0x7f6b58159d00 2023-10-06T05:24:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.801+0000 7f6b3effd700 1 -- 172.21.15.31:0/2263599626 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f6b5001e3b0 con 0x7f6b58159d00 2023-10-06T05:24:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.802+0000 7f6b3effd700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b440674d0 0x7f6b44069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.802+0000 7f6b3effd700 1 -- 172.21.15.31:0/2263599626 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f6b50085470 con 0x7f6b58159d00 2023-10-06T05:24:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.802+0000 7f6b56d9d700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b440674d0 0x7f6b44069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.803+0000 7f6b56d9d700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b440674d0 0x7f6b44069980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6b480017a0 tx=0x7f6b48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:02.805+0000 7f6b3effd700 1 -- 172.21.15.31:0/2263599626 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b50051ef0 con 0x7f6b58159d00 2023-10-06T05:24:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.006+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/2263599626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b580cc4e0 con 0x7f6b58159d00 2023-10-06T05:24:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.007+0000 7f6b3effd700 1 -- 172.21.15.31:0/2263599626 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b50055510 con 0x7f6b58159d00 2023-10-06T05:24:03.009 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:24:03.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b440674d0 msgr2=0x7f6b44069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b440674d0 0x7f6b44069980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6b480017a0 tx=0x7f6b48009040 comp rx=0 tx=0).stop 2023-10-06T05:24:03.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/2263599626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159d00 msgr2=0x7f6b580af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159d00 0x7f6b580af280 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009fc0 tx=0x7f6b500099b0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/2263599626 shutdown_connections 2023-10-06T05:24:03.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b440674d0 0x7f6b44069980 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b58159d00 0x7f6b580af280 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b580c89e0 0x7f6b58146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 --2- 172.21.15.31:0/2263599626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b58159890 0x7f6b5814c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/2263599626 >> 172.21.15.31:0/2263599626 conn(0x7f6b58067560 msgr2=0x7f6b5806e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/2263599626 shutdown_connections 2023-10-06T05:24:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.009+0000 7f6b5d1ff700 1 -- 172.21.15.31:0/2263599626 wait complete. 2023-10-06T05:24:03.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:24:03.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.222+0000 7fac393ff700 1 Processor -- start 2023-10-06T05:24:03.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.222+0000 7fac393ff700 1 -- start start 2023-10-06T05:24:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.223+0000 7fac393ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac340ca770 0x7fac340c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.223+0000 7fac393ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fac3406b510 0x7fac341569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.223+0000 7fac393ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac34156ef0 con 0x7fac340cab40 2023-10-06T05:24:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.223+0000 7fac393ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac340c7a70 con 0x7fac340ca770 2023-10-06T05:24:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.223+0000 7fac393ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac340c6df0 con 0x7fac3406b510 2023-10-06T05:24:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.224+0000 7fac32ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac340ca770 0x7fac340c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.224+0000 7fac32ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac340ca770 0x7fac340c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35956/0 (socket says 172.21.15.31:35956) 2023-10-06T05:24:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.224+0000 7fac32ffd700 1 -- 172.21.15.31:0/1389910247 learned_addr learned my addr 172.21.15.31:0/1389910247 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.224+0000 7fac327fc700 1 --1- 172.21.15.31:0/1389910247 >> v1:172.21.15.31:6789/0 conn(0x7fac340cab40 0x7fac3406ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47470/0 (socket says 172.21.15.31:47470) 2023-10-06T05:24:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.224+0000 7fac337fe700 1 --2- 172.21.15.31:0/1389910247 >> v2:172.21.15.62:3300/0 conn(0x7fac3406b510 0x7fac341569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.224+0000 7fac32ffd700 1 -- 172.21.15.31:0/1389910247 >> v2:172.21.15.62:3300/0 conn(0x7fac3406b510 msgr2=0x7fac341569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.224+0000 7fac32ffd700 1 --2- 172.21.15.31:0/1389910247 >> v2:172.21.15.62:3300/0 conn(0x7fac3406b510 0x7fac341569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.224+0000 7fac32ffd700 1 -- 172.21.15.31:0/1389910247 >> v1:172.21.15.31:6789/0 conn(0x7fac340cab40 legacy=0x7fac3406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.224+0000 7fac32ffd700 1 -- 172.21.15.31:0/1389910247 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340c70d0 con 0x7fac340ca770 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.224+0000 7fac32ffd700 1 --2- 172.21.15.31:0/1389910247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340ca770 0x7fac340c66f0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fac34150590 tx=0x7fac24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6538378b0abe3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.225+0000 7fac31ffb700 1 -- 172.21.15.31:0/1389910247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2400b860 con 0x7fac340ca770 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.225+0000 7fac31ffb700 1 -- 172.21.15.31:0/1389910247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2400b9c0 con 0x7fac340ca770 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.225+0000 7fac31ffb700 1 -- 172.21.15.31:0/1389910247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2400bd60 con 0x7fac340ca770 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.225+0000 7fac393ff700 1 -- 172.21.15.31:0/1389910247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340ca770 msgr2=0x7fac340c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.225+0000 7fac393ff700 1 --2- 172.21.15.31:0/1389910247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340ca770 0x7fac340c66f0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fac34150590 tx=0x7fac24009250 comp rx=0 tx=0).stop 2023-10-06T05:24:03.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.225+0000 7fac393ff700 1 -- 172.21.15.31:0/1389910247 shutdown_connections 2023-10-06T05:24:03.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.225+0000 7fac393ff700 1 --2- 172.21.15.31:0/1389910247 >> v2:172.21.15.62:3300/0 conn(0x7fac3406b510 0x7fac341569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.225+0000 7fac393ff700 1 --2- 172.21.15.31:0/1389910247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340ca770 0x7fac340c66f0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.225+0000 7fac393ff700 1 -- 172.21.15.31:0/1389910247 >> 172.21.15.31:0/1389910247 conn(0x7fac340cc0b0 msgr2=0x7fac340cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.225+0000 7fac393ff700 1 -- 172.21.15.31:0/1389910247 shutdown_connections 2023-10-06T05:24:03.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.226+0000 7fac393ff700 1 -- 172.21.15.31:0/1389910247 wait complete. 2023-10-06T05:24:03.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.227+0000 7fac393ff700 1 Processor -- start 2023-10-06T05:24:03.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.227+0000 7fac393ff700 1 -- start start 2023-10-06T05:24:03.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.228+0000 7fac393ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac3406b510 0x7fac340c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.228+0000 7fac393ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac340ca770 0x7fac341496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.228+0000 7fac393ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340cab40 0x7fac34149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.228+0000 7fac393ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac34143760 con 0x7fac340cab40 2023-10-06T05:24:03.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.228+0000 7fac393ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac341445c0 con 0x7fac340ca770 2023-10-06T05:24:03.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.228+0000 7fac393ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac341439b0 con 0x7fac3406b510 2023-10-06T05:24:03.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.228+0000 7fac337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340cab40 0x7fac34149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.228+0000 7fac337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340cab40 0x7fac34149c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35960/0 (socket says 172.21.15.31:35960) 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.229+0000 7fac337fe700 1 -- 172.21.15.31:0/3115053870 learned_addr learned my addr 172.21.15.31:0/3115053870 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.229+0000 7fac327fc700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac340ca770 0x7fac341496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.229+0000 7fac32ffd700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac3406b510 0x7fac340c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.229+0000 7fac337fe700 1 -- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac3406b510 msgr2=0x7fac340c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.229+0000 7fac337fe700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac3406b510 0x7fac340c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.229+0000 7fac337fe700 1 -- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac340ca770 msgr2=0x7fac341496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.229+0000 7fac337fe700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac340ca770 0x7fac341496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.229+0000 7fac337fe700 1 -- 172.21.15.31:0/3115053870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac24009490 con 0x7fac340cab40 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.229+0000 7fac337fe700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340cab40 0x7fac34149c10 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fac2c007ef0 tx=0x7fac2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.229+0000 7fac1affd700 1 -- 172.21.15.31:0/3115053870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c00eb30 con 0x7fac340cab40 2023-10-06T05:24:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.230+0000 7fac1affd700 1 -- 172.21.15.31:0/3115053870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2c00ec90 con 0x7fac340cab40 2023-10-06T05:24:03.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.230+0000 7fac1affd700 1 -- 172.21.15.31:0/3115053870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c014670 con 0x7fac340cab40 2023-10-06T05:24:03.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.230+0000 7fac393ff700 1 -- 172.21.15.31:0/3115053870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac34143c90 con 0x7fac340cab40 2023-10-06T05:24:03.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.230+0000 7fac393ff700 1 -- 172.21.15.31:0/3115053870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac34144250 con 0x7fac340cab40 2023-10-06T05:24:03.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.231+0000 7fac393ff700 1 -- 172.21.15.31:0/3115053870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac34144730 con 0x7fac340cab40 2023-10-06T05:24:03.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.232+0000 7fac1affd700 1 -- 172.21.15.31:0/3115053870 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fac2c014a60 con 0x7fac340cab40 2023-10-06T05:24:03.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.233+0000 7fac1affd700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac200674d0 0x7fac20069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.233+0000 7fac1affd700 1 -- 172.21.15.31:0/3115053870 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fac2c0853b0 con 0x7fac340cab40 2023-10-06T05:24:03.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.233+0000 7fac32ffd700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac200674d0 0x7fac20069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.234+0000 7fac32ffd700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac200674d0 0x7fac20069980 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac34150590 tx=0x7fac24017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.237+0000 7fac1affd700 1 -- 172.21.15.31:0/3115053870 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac2c051f40 con 0x7fac340cab40 2023-10-06T05:24:03.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.439+0000 7fac393ff700 1 -- 172.21.15.31:0/3115053870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fac3404c380 con 0x7fac340cab40 2023-10-06T05:24:03.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.439+0000 7fac1affd700 1 -- 172.21.15.31:0/3115053870 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v123) v1 ==== 84+0+126 (secure 0 0 0) 0x7fac2c009980 con 0x7fac340cab40 2023-10-06T05:24:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:03.441 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:24:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:24:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:24:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:24:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:24:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:03.442 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:24:03.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.441+0000 7fac393ff700 1 -- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac200674d0 msgr2=0x7fac20069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.441+0000 7fac393ff700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac200674d0 0x7fac20069980 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac34150590 tx=0x7fac24017000 comp rx=0 tx=0).stop 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.441+0000 7fac393ff700 1 -- 172.21.15.31:0/3115053870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340cab40 msgr2=0x7fac34149c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.441+0000 7fac393ff700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340cab40 0x7fac34149c10 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fac2c007ef0 tx=0x7fac2c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.442+0000 7fac393ff700 1 -- 172.21.15.31:0/3115053870 shutdown_connections 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.442+0000 7fac393ff700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac200674d0 0x7fac20069980 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.442+0000 7fac393ff700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac340cab40 0x7fac34149c10 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.442+0000 7fac393ff700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac3406b510 0x7fac340c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.442+0000 7fac393ff700 1 --2- 172.21.15.31:0/3115053870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac340ca770 0x7fac341496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.442+0000 7fac393ff700 1 -- 172.21.15.31:0/3115053870 >> 172.21.15.31:0/3115053870 conn(0x7fac340cc0b0 msgr2=0x7fac340c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.442+0000 7fac393ff700 1 -- 172.21.15.31:0/3115053870 shutdown_connections 2023-10-06T05:24:03.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.442+0000 7fac393ff700 1 -- 172.21.15.31:0/3115053870 wait complete. 2023-10-06T05:24:03.461 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:24:03.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:24:03.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.660+0000 7f4b25bff700 1 Processor -- start 2023-10-06T05:24:03.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.660+0000 7f4b25bff700 1 -- start start 2023-10-06T05:24:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.661+0000 7f4b25bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b20159cb0 0x7f4b200caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.661+0000 7f4b25bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4b200cafd0 0x7f4b2015a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.661+0000 7f4b25bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b200cb670 con 0x7f4b2015a5c0 2023-10-06T05:24:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.661+0000 7f4b25bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b200cd080 con 0x7f4b20159cb0 2023-10-06T05:24:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.661+0000 7f4b25bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b200cb450 con 0x7f4b200cafd0 2023-10-06T05:24:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.661+0000 7f4b1f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b20159cb0 0x7f4b200caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.661+0000 7f4b1f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b20159cb0 0x7f4b200caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35968/0 (socket says 172.21.15.31:35968) 2023-10-06T05:24:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.661+0000 7f4b1f7fe700 1 -- 172.21.15.31:0/1813711144 learned_addr learned my addr 172.21.15.31:0/1813711144 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.661+0000 7f4b1effd700 1 --2- 172.21.15.31:0/1813711144 >> v2:172.21.15.62:3300/0 conn(0x7f4b200cafd0 0x7f4b2015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.662+0000 7f4b1f7fe700 1 -- 172.21.15.31:0/1813711144 >> v2:172.21.15.62:3300/0 conn(0x7f4b200cafd0 msgr2=0x7f4b2015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.662+0000 7f4b1f7fe700 1 --2- 172.21.15.31:0/1813711144 >> v2:172.21.15.62:3300/0 conn(0x7f4b200cafd0 0x7f4b2015a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.662+0000 7f4b1f7fe700 1 -- 172.21.15.31:0/1813711144 >> v1:172.21.15.31:6789/0 conn(0x7f4b2015a5c0 legacy=0x7f4b200ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.662+0000 7f4b1f7fe700 1 -- 172.21.15.31:0/1813711144 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b200cb950 con 0x7f4b20159cb0 2023-10-06T05:24:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.662+0000 7f4b1f7fe700 1 --2- 172.21.15.31:0/1813711144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 0x7f4b200caa90 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4b20089360 tx=0x7f4b10009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74945d339ecd6f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.662+0000 7f4b1e7fc700 1 -- 172.21.15.31:0/1813711144 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10005c10 con 0x7f4b20159cb0 2023-10-06T05:24:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.662+0000 7f4b1e7fc700 1 -- 172.21.15.31:0/1813711144 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b10005d70 con 0x7f4b20159cb0 2023-10-06T05:24:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.662+0000 7f4b1e7fc700 1 -- 172.21.15.31:0/1813711144 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1000b840 con 0x7f4b20159cb0 2023-10-06T05:24:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.663+0000 7f4b25bff700 1 -- 172.21.15.31:0/1813711144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 msgr2=0x7f4b200caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.663+0000 7f4b25bff700 1 --2- 172.21.15.31:0/1813711144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 0x7f4b200caa90 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4b20089360 tx=0x7f4b10009220 comp rx=0 tx=0).stop 2023-10-06T05:24:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.663+0000 7f4b25bff700 1 -- 172.21.15.31:0/1813711144 shutdown_connections 2023-10-06T05:24:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.663+0000 7f4b25bff700 1 --2- 172.21.15.31:0/1813711144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 0x7f4b200caa90 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.663+0000 7f4b25bff700 1 --2- 172.21.15.31:0/1813711144 >> v2:172.21.15.62:3300/0 conn(0x7f4b200cafd0 0x7f4b2015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.663+0000 7f4b25bff700 1 -- 172.21.15.31:0/1813711144 >> 172.21.15.31:0/1813711144 conn(0x7f4b20099710 msgr2=0x7f4b2006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.663+0000 7f4b25bff700 1 -- 172.21.15.31:0/1813711144 shutdown_connections 2023-10-06T05:24:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.663+0000 7f4b25bff700 1 -- 172.21.15.31:0/1813711144 wait complete. 2023-10-06T05:24:03.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.664+0000 7f4b25bff700 1 Processor -- start 2023-10-06T05:24:03.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.665+0000 7f4b25bff700 1 -- start start 2023-10-06T05:24:03.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.665+0000 7f4b25bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b200cafd0 0x7f4b200b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b25bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 0x7f4b2016d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b25bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b2015a5c0 0x7f4b2016d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b25bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200b0450 con 0x7f4b20159cb0 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b25bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200afbb0 con 0x7f4b200cafd0 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b25bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200b3a70 con 0x7f4b2015a5c0 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 0x7f4b2016d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 0x7f4b2016d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35974/0 (socket says 172.21.15.31:35974) 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1effd700 1 -- 172.21.15.31:0/1701033449 learned_addr learned my addr 172.21.15.31:0/1701033449 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1ffff700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b2015a5c0 0x7f4b2016d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1f7fe700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b200cafd0 0x7f4b200b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1effd700 1 -- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b2015a5c0 msgr2=0x7f4b2016d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1effd700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b2015a5c0 0x7f4b2016d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1effd700 1 -- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b200cafd0 msgr2=0x7f4b200b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1effd700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b200cafd0 0x7f4b200b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1effd700 1 -- 172.21.15.31:0/1701033449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b10009480 con 0x7f4b20159cb0 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1ffff700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b2015a5c0 0x7f4b2016d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1f7fe700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b200cafd0 0x7f4b200b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.666+0000 7f4b1effd700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 0x7f4b2016d190 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400d890 tx=0x7f4b14008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.667+0000 7f4b077fe700 1 -- 172.21.15.31:0/1701033449 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400b600 con 0x7f4b20159cb0 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.667+0000 7f4b077fe700 1 -- 172.21.15.31:0/1701033449 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1400b760 con 0x7f4b20159cb0 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.667+0000 7f4b077fe700 1 -- 172.21.15.31:0/1701033449 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400bae0 con 0x7f4b20159cb0 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.667+0000 7f4b25bff700 1 -- 172.21.15.31:0/1701033449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b200afe00 con 0x7f4b20159cb0 2023-10-06T05:24:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.667+0000 7f4b25bff700 1 -- 172.21.15.31:0/1701033449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b200b28c0 con 0x7f4b20159cb0 2023-10-06T05:24:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.668+0000 7f4b25bff700 1 -- 172.21.15.31:0/1701033449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2004c380 con 0x7f4b20159cb0 2023-10-06T05:24:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.669+0000 7f4b077fe700 1 -- 172.21.15.31:0/1701033449 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f4b1401e460 con 0x7f4b20159cb0 2023-10-06T05:24:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.670+0000 7f4b077fe700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b0c0674d0 0x7f4b0c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.670+0000 7f4b077fe700 1 -- 172.21.15.31:0/1701033449 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f4b14085460 con 0x7f4b20159cb0 2023-10-06T05:24:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.670+0000 7f4b1f7fe700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b0c0674d0 0x7f4b0c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:03.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.671+0000 7f4b1f7fe700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b0c0674d0 0x7f4b0c069980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009260 tx=0x7f4b10016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.674+0000 7f4b077fe700 1 -- 172.21.15.31:0/1701033449 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b14052030 con 0x7f4b20159cb0 2023-10-06T05:24:03.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:03.881+0000 7f4b25bff700 1 -- 172.21.15.31:0/1701033449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4b200cbc30 con 0x7f4b20159cb0 2023-10-06T05:24:04.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.697+0000 7f4b077fe700 1 -- 172.21.15.31:0/1701033449 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v124) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4b14055650 con 0x7f4b20159cb0 2023-10-06T05:24:04.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.701+0000 7f4b25bff700 1 -- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b0c0674d0 msgr2=0x7f4b0c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.701+0000 7f4b25bff700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b0c0674d0 0x7f4b0c069980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009260 tx=0x7f4b10016000 comp rx=0 tx=0).stop 2023-10-06T05:24:04.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.701+0000 7f4b25bff700 1 -- 172.21.15.31:0/1701033449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 msgr2=0x7f4b2016d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.701+0000 7f4b25bff700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 0x7f4b2016d190 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400d890 tx=0x7f4b14008040 comp rx=0 tx=0).stop 2023-10-06T05:24:04.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.701+0000 7f4b25bff700 1 -- 172.21.15.31:0/1701033449 shutdown_connections 2023-10-06T05:24:04.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.701+0000 7f4b25bff700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b0c0674d0 0x7f4b0c069980 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.701+0000 7f4b25bff700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b2015a5c0 0x7f4b2016d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.701+0000 7f4b25bff700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b200cafd0 0x7f4b200b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.701+0000 7f4b25bff700 1 --2- 172.21.15.31:0/1701033449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b20159cb0 0x7f4b2016d190 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.701+0000 7f4b25bff700 1 -- 172.21.15.31:0/1701033449 >> 172.21.15.31:0/1701033449 conn(0x7f4b20099710 msgr2=0x7f4b20158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:04.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.702+0000 7f4b25bff700 1 -- 172.21.15.31:0/1701033449 shutdown_connections 2023-10-06T05:24:04.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.702+0000 7f4b25bff700 1 -- 172.21.15.31:0/1701033449 wait complete. 2023-10-06T05:24:04.706 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:24:04.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:24:04.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.930+0000 7f63a19ff700 1 Processor -- start 2023-10-06T05:24:04.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.930+0000 7f63a19ff700 1 -- start start 2023-10-06T05:24:04.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.931+0000 7f63a19ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f639c0cc340 0x7f639c0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.931+0000 7f63a19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 0x7f639c0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.932+0000 7f63a19ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639c15a020 con 0x7f639c0cc710 2023-10-06T05:24:04.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.932+0000 7f63a19ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639c0b07e0 con 0x7f639c0cc340 2023-10-06T05:24:04.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.932+0000 7f63a19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c159cf0 con 0x7f639c15a4b0 2023-10-06T05:24:04.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.932+0000 7f639a7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f639c0cc710 0x7f639c0b00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47488/0 (socket says 172.21.15.31:47488) 2023-10-06T05:24:04.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.932+0000 7f639a7fc700 1 -- 172.21.15.31:0/1557132086 learned_addr learned my addr 172.21.15.31:0/1557132086 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:04.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.932+0000 7f639affd700 1 --2- 172.21.15.31:0/1557132086 >> v2:172.21.15.62:3300/0 conn(0x7f639c0cc340 0x7f639c0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.932+0000 7f639b7fe700 1 --2- 172.21.15.31:0/1557132086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 0x7f639c0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.932+0000 7f6399ffb700 1 -- 172.21.15.31:0/1557132086 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354347968 0 0) 0x7f639c15a020 con 0x7f639c0cc710 2023-10-06T05:24:04.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.932+0000 7f6399ffb700 1 -- 172.21.15.31:0/1557132086 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6384003570 con 0x7f639c0cc710 2023-10-06T05:24:04.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.933+0000 7f639b7fe700 1 -- 172.21.15.31:0/1557132086 >> v2:172.21.15.62:3300/0 conn(0x7f639c0cc340 msgr2=0x7f639c0b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:04.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.933+0000 7f639b7fe700 1 --2- 172.21.15.31:0/1557132086 >> v2:172.21.15.62:3300/0 conn(0x7f639c0cc340 0x7f639c0b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.933+0000 7f639b7fe700 1 -- 172.21.15.31:0/1557132086 >> v1:172.21.15.31:6789/0 conn(0x7f639c0cc710 legacy=0x7f639c0b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.933+0000 7f639b7fe700 1 -- 172.21.15.31:0/1557132086 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c0ccb90 con 0x7f639c15a4b0 2023-10-06T05:24:04.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.933+0000 7f639b7fe700 1 --2- 172.21.15.31:0/1557132086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 0x7f639c0aecd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6394009e60 tx=0x7f6394009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3b7e565adc9d5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.933+0000 7f6399ffb700 1 -- 172.21.15.31:0/1557132086 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639400e070 con 0x7f639c15a4b0 2023-10-06T05:24:04.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.933+0000 7f6399ffb700 1 -- 172.21.15.31:0/1557132086 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639400bab0 con 0x7f639c15a4b0 2023-10-06T05:24:04.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.933+0000 7f6399ffb700 1 -- 172.21.15.31:0/1557132086 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639400be70 con 0x7f639c15a4b0 2023-10-06T05:24:04.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.934+0000 7f63a19ff700 1 -- 172.21.15.31:0/1557132086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 msgr2=0x7f639c0aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.934+0000 7f63a19ff700 1 --2- 172.21.15.31:0/1557132086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 0x7f639c0aecd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6394009e60 tx=0x7f6394009780 comp rx=0 tx=0).stop 2023-10-06T05:24:04.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.934+0000 7f63a19ff700 1 -- 172.21.15.31:0/1557132086 shutdown_connections 2023-10-06T05:24:04.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.934+0000 7f63a19ff700 1 --2- 172.21.15.31:0/1557132086 >> v2:172.21.15.62:3300/0 conn(0x7f639c0cc340 0x7f639c0b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.934+0000 7f63a19ff700 1 --2- 172.21.15.31:0/1557132086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 0x7f639c0aecd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.934+0000 7f63a19ff700 1 -- 172.21.15.31:0/1557132086 >> 172.21.15.31:0/1557132086 conn(0x7f639c068670 msgr2=0x7f639c068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:04.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.934+0000 7f63a19ff700 1 -- 172.21.15.31:0/1557132086 shutdown_connections 2023-10-06T05:24:04.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.934+0000 7f63a19ff700 1 -- 172.21.15.31:0/1557132086 wait complete. 2023-10-06T05:24:04.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.935+0000 7f63a19ff700 1 Processor -- start 2023-10-06T05:24:04.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.935+0000 7f63a19ff700 1 -- start start 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.936+0000 7f63a19ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c0cc340 0x7f639c14b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.936+0000 7f63a19ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c0cc710 0x7f639c0ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.936+0000 7f63a19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 0x7f639c16c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.936+0000 7f63a19ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c0af660 con 0x7f639c0cc340 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.936+0000 7f63a19ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c1541c0 con 0x7f639c0cc710 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.936+0000 7f63a19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c153360 con 0x7f639c15a4b0 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c0cc340 0x7f639c14b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c0cc340 0x7f639c14b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35990/0 (socket says 172.21.15.31:35990) 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639affd700 1 -- 172.21.15.31:0/2179046592 learned_addr learned my addr 172.21.15.31:0/2179046592 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639a7fc700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c0cc710 0x7f639c0ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639b7fe700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 0x7f639c16c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639affd700 1 -- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 msgr2=0x7f639c16c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639affd700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 0x7f639c16c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639affd700 1 -- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c0cc710 msgr2=0x7f639c0ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639affd700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c0cc710 0x7f639c0ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639affd700 1 -- 172.21.15.31:0/2179046592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394009a20 con 0x7f639c0cc340 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639a7fc700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c0cc710 0x7f639c0ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.937+0000 7f639affd700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c0cc340 0x7f639c14b860 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f638c00ea30 tx=0x7f638c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.938+0000 7f6382ffd700 1 -- 172.21.15.31:0/2179046592 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00c850 con 0x7f639c0cc340 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.938+0000 7f6382ffd700 1 -- 172.21.15.31:0/2179046592 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c00c9b0 con 0x7f639c0cc340 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.938+0000 7f6382ffd700 1 -- 172.21.15.31:0/2179046592 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00cd30 con 0x7f639c0cc340 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.938+0000 7f63a19ff700 1 -- 172.21.15.31:0/2179046592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c153ee0 con 0x7f639c0cc340 2023-10-06T05:24:04.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.938+0000 7f63a19ff700 1 -- 172.21.15.31:0/2179046592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c153890 con 0x7f639c0cc340 2023-10-06T05:24:04.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.939+0000 7f63a19ff700 1 -- 172.21.15.31:0/2179046592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c04c380 con 0x7f639c0cc340 2023-10-06T05:24:04.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.940+0000 7f6382ffd700 1 -- 172.21.15.31:0/2179046592 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f638c013680 con 0x7f639c0cc340 2023-10-06T05:24:04.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.941+0000 7f6382ffd700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f63880674d0 0x7f6388069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:04.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.941+0000 7f6382ffd700 1 -- 172.21.15.31:0/2179046592 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5604+0+0 (secure 0 0 0) 0x7f638c0855a0 con 0x7f639c0cc340 2023-10-06T05:24:04.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.942+0000 7f639a7fc700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f63880674d0 0x7f6388069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:04.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.942+0000 7f639a7fc700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f63880674d0 0x7f6388069980 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f639c154d00 tx=0x7f6390009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:04.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:04.946+0000 7f6382ffd700 1 -- 172.21.15.31:0/2179046592 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f638c051fc0 con 0x7f639c0cc340 2023-10-06T05:24:05.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.150+0000 7f63a19ff700 1 -- 172.21.15.31:0/2179046592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f639c0b2b40 con 0x7f639c0cc340 2023-10-06T05:24:05.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.704+0000 7f6382ffd700 1 -- 172.21.15.31:0/2179046592 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v125) v1 ==== 170+0+0 (secure 0 0 0) 0x7f638c0555e0 con 0x7f639c0cc340 2023-10-06T05:24:05.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.713+0000 7f63a19ff700 1 -- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f63880674d0 msgr2=0x7f6388069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.713+0000 7f63a19ff700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f63880674d0 0x7f6388069980 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f639c154d00 tx=0x7f6390009040 comp rx=0 tx=0).stop 2023-10-06T05:24:05.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.713+0000 7f63a19ff700 1 -- 172.21.15.31:0/2179046592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c0cc340 msgr2=0x7f639c14b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.713+0000 7f63a19ff700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c0cc340 0x7f639c14b860 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f638c00ea30 tx=0x7f638c009120 comp rx=0 tx=0).stop 2023-10-06T05:24:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.715+0000 7f63a19ff700 1 -- 172.21.15.31:0/2179046592 shutdown_connections 2023-10-06T05:24:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.715+0000 7f63a19ff700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f63880674d0 0x7f6388069980 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.715+0000 7f63a19ff700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f639c15a4b0 0x7f639c16c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.715+0000 7f63a19ff700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c0cc340 0x7f639c14b860 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.715+0000 7f63a19ff700 1 --2- 172.21.15.31:0/2179046592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c0cc710 0x7f639c0ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.715+0000 7f63a19ff700 1 -- 172.21.15.31:0/2179046592 >> 172.21.15.31:0/2179046592 conn(0x7f639c068670 msgr2=0x7f639c154da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.716+0000 7f63a19ff700 1 -- 172.21.15.31:0/2179046592 shutdown_connections 2023-10-06T05:24:05.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.716+0000 7f63a19ff700 1 -- 172.21.15.31:0/2179046592 wait complete. 2023-10-06T05:24:05.722 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:24:05.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:24:05.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.942+0000 7f94d9bff700 1 Processor -- start 2023-10-06T05:24:05.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.942+0000 7f94d9bff700 1 -- start start 2023-10-06T05:24:05.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.943+0000 7f94d9bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94d4159c60 0x7f94d40c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.943+0000 7f94d9bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d4158190 con 0x7f94d4155430 2023-10-06T05:24:05.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.943+0000 7f94d9bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d4157580 con 0x7f94d4159890 2023-10-06T05:24:05.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.943+0000 7f94d9bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d4157860 con 0x7f94d4159c60 2023-10-06T05:24:05.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.944+0000 7f94d2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94d4159c60 0x7f94d40c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.944+0000 7f94d2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f94d4159c60 0x7f94d40c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36000/0 (socket says 172.21.15.31:36000) 2023-10-06T05:24:05.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.944+0000 7f94d2ffd700 1 -- 172.21.15.31:0/3708573388 learned_addr learned my addr 172.21.15.31:0/3708573388 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.944+0000 7f94d2ffd700 1 -- 172.21.15.31:0/3708573388 >> v1:172.21.15.62:6789/0 conn(0x7f94d4159890 legacy=0x7f94d4156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.944+0000 7f94d27fc700 1 -- 172.21.15.31:0/3708573388 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801156706 0 0) 0x7f94d4157580 con 0x7f94d4159890 2023-10-06T05:24:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.944+0000 7f94d2ffd700 1 -- 172.21.15.31:0/3708573388 >> v1:172.21.15.31:6789/0 conn(0x7f94d4155430 legacy=0x7f94d4155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.944+0000 7f94d2ffd700 1 -- 172.21.15.31:0/3708573388 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d4157b40 con 0x7f94d4159c60 2023-10-06T05:24:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.945+0000 7f94d2ffd700 1 --2- 172.21.15.31:0/3708573388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 0x7f94d40c89e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f94c80099b0 tx=0x7f94c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d7f87d2a998801 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.945+0000 7f94d27fc700 1 -- 172.21.15.31:0/3708573388 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c800b860 con 0x7f94d4159c60 2023-10-06T05:24:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.945+0000 7f94d27fc700 1 -- 172.21.15.31:0/3708573388 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c800b9c0 con 0x7f94d4159c60 2023-10-06T05:24:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.945+0000 7f94d27fc700 1 -- 172.21.15.31:0/3708573388 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c800bd60 con 0x7f94d4159c60 2023-10-06T05:24:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.945+0000 7f94d9bff700 1 -- 172.21.15.31:0/3708573388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 msgr2=0x7f94d40c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.945+0000 7f94d9bff700 1 --2- 172.21.15.31:0/3708573388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 0x7f94d40c89e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f94c80099b0 tx=0x7f94c8009250 comp rx=0 tx=0).stop 2023-10-06T05:24:05.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.945+0000 7f94d9bff700 1 -- 172.21.15.31:0/3708573388 shutdown_connections 2023-10-06T05:24:05.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.945+0000 7f94d9bff700 1 --2- 172.21.15.31:0/3708573388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 0x7f94d40c89e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.945+0000 7f94d9bff700 1 -- 172.21.15.31:0/3708573388 >> 172.21.15.31:0/3708573388 conn(0x7f94d4067560 msgr2=0x7f94d4067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:05.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.946+0000 7f94d9bff700 1 -- 172.21.15.31:0/3708573388 shutdown_connections 2023-10-06T05:24:05.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.946+0000 7f94d9bff700 1 -- 172.21.15.31:0/3708573388 wait complete. 2023-10-06T05:24:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.947+0000 7f94d9bff700 1 Processor -- start 2023-10-06T05:24:05.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.947+0000 7f94d9bff700 1 -- start start 2023-10-06T05:24:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.948+0000 7f94d9bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94d4155430 0x7f94d4150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.948+0000 7f94d9bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d4159890 0x7f94d41694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.948+0000 7f94d9bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 0x7f94d40c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.948+0000 7f94d9bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40cb020 con 0x7f94d4159c60 2023-10-06T05:24:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.948+0000 7f94d9bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d415b030 con 0x7f94d4159890 2023-10-06T05:24:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.948+0000 7f94d9bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40ca410 con 0x7f94d4155430 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.948+0000 7f94d3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 0x7f94d40c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.948+0000 7f94d3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 0x7f94d40c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36004/0 (socket says 172.21.15.31:36004) 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.948+0000 7f94d3fff700 1 -- 172.21.15.31:0/2507429290 learned_addr learned my addr 172.21.15.31:0/2507429290 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94d2ffd700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d4159890 0x7f94d41694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94d37fe700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94d4155430 0x7f94d4150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94d3fff700 1 -- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94d4155430 msgr2=0x7f94d4150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94d3fff700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94d4155430 0x7f94d4150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94d3fff700 1 -- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d4159890 msgr2=0x7f94d41694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94d3fff700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d4159890 0x7f94d41694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94d3fff700 1 -- 172.21.15.31:0/2507429290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c80094f0 con 0x7f94d4159c60 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94d2ffd700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d4159890 0x7f94d41694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:05.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94d3fff700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 0x7f94d40c9920 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00c840 tx=0x7f94cc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94bb7fe700 1 -- 172.21.15.31:0/2507429290 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc012630 con 0x7f94d4159c60 2023-10-06T05:24:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94bb7fe700 1 -- 172.21.15.31:0/2507429290 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc0127b0 con 0x7f94d4159c60 2023-10-06T05:24:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94bb7fe700 1 -- 172.21.15.31:0/2507429290 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc012b70 con 0x7f94d4159c60 2023-10-06T05:24:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.949+0000 7f94d9bff700 1 -- 172.21.15.31:0/2507429290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d40ca6f0 con 0x7f94d4159c60 2023-10-06T05:24:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.950+0000 7f94d9bff700 1 -- 172.21.15.31:0/2507429290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d40cacb0 con 0x7f94d4159c60 2023-10-06T05:24:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.950+0000 7f94d9bff700 1 -- 172.21.15.31:0/2507429290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d41566b0 con 0x7f94d4159c60 2023-10-06T05:24:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.953+0000 7f94bb7fe700 1 -- 172.21.15.31:0/2507429290 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f94cc003030 con 0x7f94d4159c60 2023-10-06T05:24:05.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.953+0000 7f94bb7fe700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94c00674d0 0x7f94c0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:05.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.954+0000 7f94bb7fe700 1 -- 172.21.15.31:0/2507429290 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5986+0+0 (secure 0 0 0) 0x7f94cc0856e0 con 0x7f94d4159c60 2023-10-06T05:24:05.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.954+0000 7f94d37fe700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94c00674d0 0x7f94c0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:05.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.955+0000 7f94d37fe700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94c00674d0 0x7f94c0069980 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94c40011e0 tx=0x7f94c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:05.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:05.957+0000 7f94bb7fe700 1 -- 172.21.15.31:0/2507429290 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94cc052140 con 0x7f94d4159c60 2023-10-06T05:24:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:06.202+0000 7f94d9bff700 1 -- 172.21.15.31:0/2507429290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f94d404c380 con 0x7f94d4159c60 2023-10-06T05:24:07.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.108+0000 7f94bb7fe700 1 -- 172.21.15.31:0/2507429290 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v80) v1 ==== 174+0+0 (secure 0 0 0) 0x7f94cc055760 con 0x7f94d4159c60 2023-10-06T05:24:07.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 -- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94c00674d0 msgr2=0x7f94c0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94c00674d0 0x7f94c0069980 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94c40011e0 tx=0x7f94c4009040 comp rx=0 tx=0).stop 2023-10-06T05:24:07.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 -- 172.21.15.31:0/2507429290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 msgr2=0x7f94d40c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 0x7f94d40c9920 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00c840 tx=0x7f94cc001670 comp rx=0 tx=0).stop 2023-10-06T05:24:07.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 -- 172.21.15.31:0/2507429290 shutdown_connections 2023-10-06T05:24:07.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94c00674d0 0x7f94c0069980 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94d4159c60 0x7f94d40c9920 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94d4155430 0x7f94d4150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 --2- 172.21.15.31:0/2507429290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94d4159890 0x7f94d41694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 -- 172.21.15.31:0/2507429290 >> 172.21.15.31:0/2507429290 conn(0x7f94d4067560 msgr2=0x7f94d4158620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 -- 172.21.15.31:0/2507429290 shutdown_connections 2023-10-06T05:24:07.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.112+0000 7f94d9bff700 1 -- 172.21.15.31:0/2507429290 wait complete. 2023-10-06T05:24:07.118 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 21 and data pool 22 2023-10-06T05:24:07.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:07.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.336+0000 7fb1a79ff700 1 Processor -- start 2023-10-06T05:24:07.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.337+0000 7fb1a79ff700 1 -- start start 2023-10-06T05:24:07.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.338+0000 7fb1a79ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1a01596d0 0x7fb1a0159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.338+0000 7fb1a79ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb1a00c6f80 0x7fb1a006a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.338+0000 7fb1a79ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a0067f70 con 0x7fb1a00c16e0 2023-10-06T05:24:07.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.338+0000 7fb1a79ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a006e040 con 0x7fb1a01596d0 2023-10-06T05:24:07.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.338+0000 7fb1a79ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a00681c0 con 0x7fb1a00c6f80 2023-10-06T05:24:07.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.338+0000 7fb1a579b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1a01596d0 0x7fb1a0159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.338+0000 7fb1a579b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1a01596d0 0x7fb1a0159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49276/0 (socket says 172.21.15.31:49276) 2023-10-06T05:24:07.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.338+0000 7fb1a579b700 1 -- 172.21.15.31:0/2669635907 learned_addr learned my addr 172.21.15.31:0/2669635907 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:07.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.339+0000 7fb1a4f9a700 1 --2- 172.21.15.31:0/2669635907 >> v2:172.21.15.62:3300/0 conn(0x7fb1a00c6f80 0x7fb1a006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.339+0000 7fb1a579b700 1 -- 172.21.15.31:0/2669635907 >> v2:172.21.15.62:3300/0 conn(0x7fb1a00c6f80 msgr2=0x7fb1a006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:07.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.339+0000 7fb1a579b700 1 --2- 172.21.15.31:0/2669635907 >> v2:172.21.15.62:3300/0 conn(0x7fb1a00c6f80 0x7fb1a006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.339+0000 7fb1a579b700 1 -- 172.21.15.31:0/2669635907 >> v1:172.21.15.31:6789/0 conn(0x7fb1a00c16e0 legacy=0x7fb1a015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.339+0000 7fb1a579b700 1 -- 172.21.15.31:0/2669635907 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a00684a0 con 0x7fb1a01596d0 2023-10-06T05:24:07.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.339+0000 7fb1a579b700 1 --2- 172.21.15.31:0/2669635907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a01596d0 0x7fb1a0159aa0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb1a006dbe0 tx=0x7fb19c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c49d131538d297 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.339+0000 7fb197fff700 1 -- 172.21.15.31:0/2669635907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00b860 con 0x7fb1a01596d0 2023-10-06T05:24:07.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.339+0000 7fb197fff700 1 -- 172.21.15.31:0/2669635907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c00b9c0 con 0x7fb1a01596d0 2023-10-06T05:24:07.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.339+0000 7fb197fff700 1 -- 172.21.15.31:0/2669635907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00bd60 con 0x7fb1a01596d0 2023-10-06T05:24:07.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.340+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2669635907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a01596d0 msgr2=0x7fb1a0159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.340+0000 7fb1a79ff700 1 --2- 172.21.15.31:0/2669635907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a01596d0 0x7fb1a0159aa0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb1a006dbe0 tx=0x7fb19c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:07.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.340+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2669635907 shutdown_connections 2023-10-06T05:24:07.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.340+0000 7fb1a79ff700 1 --2- 172.21.15.31:0/2669635907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a01596d0 0x7fb1a0159aa0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.340+0000 7fb1a79ff700 1 --2- 172.21.15.31:0/2669635907 >> v2:172.21.15.62:3300/0 conn(0x7fb1a00c6f80 0x7fb1a006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.340+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2669635907 >> 172.21.15.31:0/2669635907 conn(0x7fb1a00c9e80 msgr2=0x7fb1a006a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.340+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2669635907 shutdown_connections 2023-10-06T05:24:07.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.340+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2669635907 wait complete. 2023-10-06T05:24:07.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.341+0000 7fb1a79ff700 1 Processor -- start 2023-10-06T05:24:07.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.342+0000 7fb1a79ff700 1 -- start start 2023-10-06T05:24:07.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.342+0000 7fb1a79ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c16e0 0x7fb1a0169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.342+0000 7fb1a79ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1a00c6f80 0x7fb1a0146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.342+0000 7fb1a79ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a01596d0 0x7fb1a00b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.342+0000 7fb1a79ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a016a410 con 0x7fb1a00c16e0 2023-10-06T05:24:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.342+0000 7fb1a79ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00af740 con 0x7fb1a01596d0 2023-10-06T05:24:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.342+0000 7fb1a79ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00c5130 con 0x7fb1a00c6f80 2023-10-06T05:24:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.343+0000 7fb1a579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c16e0 0x7fb1a0169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.343+0000 7fb1a579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c16e0 0x7fb1a0169ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49286/0 (socket says 172.21.15.31:49286) 2023-10-06T05:24:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.343+0000 7fb1a579b700 1 -- 172.21.15.31:0/2926864535 learned_addr learned my addr 172.21.15.31:0/2926864535 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.343+0000 7fb1a579b700 1 -- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1a00c6f80 msgr2=0x7fb1a0146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.343+0000 7fb1a579b700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1a00c6f80 0x7fb1a0146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.343+0000 7fb1a579b700 1 -- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a01596d0 msgr2=0x7fb1a00b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.343+0000 7fb1a579b700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a01596d0 0x7fb1a00b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.343+0000 7fb1a579b700 1 -- 172.21.15.31:0/2926864535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c0094f0 con 0x7fb1a00c16e0 2023-10-06T05:24:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.343+0000 7fb1a579b700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c16e0 0x7fb1a0169ed0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0069a60 tx=0x7fb19c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.344+0000 7fb1957fa700 1 -- 172.21.15.31:0/2926864535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c011e60 con 0x7fb1a00c16e0 2023-10-06T05:24:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.344+0000 7fb1957fa700 1 -- 172.21.15.31:0/2926864535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c01a630 con 0x7fb1a00c16e0 2023-10-06T05:24:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.344+0000 7fb1957fa700 1 -- 172.21.15.31:0/2926864535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c01a9f0 con 0x7fb1a00c16e0 2023-10-06T05:24:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.344+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2926864535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a00c5410 con 0x7fb1a00c16e0 2023-10-06T05:24:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.344+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2926864535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a00c59d0 con 0x7fb1a00c16e0 2023-10-06T05:24:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.345+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2926864535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a00af8b0 con 0x7fb1a00c16e0 2023-10-06T05:24:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.346+0000 7fb1957fa700 1 -- 172.21.15.31:0/2926864535 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb19c02d440 con 0x7fb1a00c16e0 2023-10-06T05:24:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.346+0000 7fb1957fa700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1840674d0 0x7fb184069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.347+0000 7fb1957fa700 1 -- 172.21.15.31:0/2926864535 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb19c086d40 con 0x7fb1a00c16e0 2023-10-06T05:24:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.347+0000 7fb1a4f9a700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1840674d0 0x7fb184069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.348+0000 7fb1a4f9a700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1840674d0 0x7fb184069980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00c6ea0 tx=0x7fb190008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.351+0000 7fb1957fa700 1 -- 172.21.15.31:0/2926864535 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb19c053580 con 0x7fb1a00c16e0 2023-10-06T05:24:07.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.552+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2926864535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1a00b0cc0 con 0x7fb1a00c16e0 2023-10-06T05:24:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.553+0000 7fb1957fa700 1 -- 172.21.15.31:0/2926864535 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16985 (secure 0 0 0) 0x7fb19c015090 con 0x7fb1a00c16e0 2023-10-06T05:24:07.555 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:07.555 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":126,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:06.204064+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:03.921897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:05.190243+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:07.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1840674d0 msgr2=0x7fb184069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1840674d0 0x7fb184069980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00c6ea0 tx=0x7fb190008040 comp rx=0 tx=0).stop 2023-10-06T05:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2926864535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c16e0 msgr2=0x7fb1a0169ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c16e0 0x7fb1a0169ed0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0069a60 tx=0x7fb19c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2926864535 shutdown_connections 2023-10-06T05:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1840674d0 0x7fb184069980 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a01596d0 0x7fb1a00b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c16e0 0x7fb1a0169ed0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 --2- 172.21.15.31:0/2926864535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1a00c6f80 0x7fb1a0146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2926864535 >> 172.21.15.31:0/2926864535 conn(0x7fb1a00c9e80 msgr2=0x7fb1a00c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.555+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2926864535 shutdown_connections 2023-10-06T05:24:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.556+0000 7fb1a79ff700 1 -- 172.21.15.31:0/2926864535 wait complete. 2023-10-06T05:24:07.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:07.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.770+0000 7f50035ff700 1 Processor -- start 2023-10-06T05:24:07.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.770+0000 7f50035ff700 1 -- start start 2023-10-06T05:24:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.772+0000 7f50035ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4ffc13a1e0 0x7f4ffc0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.772+0000 7f50035ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc0c71b0 con 0x7f4ffc06db50 2023-10-06T05:24:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.772+0000 7f50035ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc0c7490 con 0x7f4ffc13a5b0 2023-10-06T05:24:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.772+0000 7f50035ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ffc0c7770 con 0x7f4ffc13a1e0 2023-10-06T05:24:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.772+0000 7f5001b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4ffc06db50 0x7f4ffc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39586/0 (socket says 172.21.15.31:39586) 2023-10-06T05:24:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.772+0000 7f5001b9c700 1 -- 172.21.15.31:0/2655849071 learned_addr learned my addr 172.21.15.31:0/2655849071 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.772+0000 7f500139b700 1 --2- 172.21.15.31:0/2655849071 >> v2:172.21.15.62:3300/0 conn(0x7f4ffc13a1e0 0x7f4ffc0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.773+0000 7f4ff3fff700 1 -- 172.21.15.31:0/2655849071 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508831584 0 0) 0x7f4ffc0c71b0 con 0x7f4ffc06db50 2023-10-06T05:24:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.773+0000 7f4ff3fff700 1 -- 172.21.15.31:0/2655849071 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe8003570 con 0x7f4ffc06db50 2023-10-06T05:24:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.773+0000 7f500139b700 1 -- 172.21.15.31:0/2655849071 >> v1:172.21.15.62:6789/0 conn(0x7f4ffc13a5b0 legacy=0x7f4ffc06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.773+0000 7f500139b700 1 -- 172.21.15.31:0/2655849071 >> v1:172.21.15.31:6789/0 conn(0x7f4ffc06db50 legacy=0x7f4ffc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.773+0000 7f500139b700 1 -- 172.21.15.31:0/2655849071 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc0c7a50 con 0x7f4ffc13a1e0 2023-10-06T05:24:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.773+0000 7f500139b700 1 --2- 172.21.15.31:0/2655849071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc13a1e0 0x7f4ffc0c66f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80098d0 tx=0x7f4ff8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22070185ebce674a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.773+0000 7f4ff3fff700 1 -- 172.21.15.31:0/2655849071 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8005c10 con 0x7f4ffc13a1e0 2023-10-06T05:24:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.773+0000 7f4ff3fff700 1 -- 172.21.15.31:0/2655849071 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff8005d70 con 0x7f4ffc13a1e0 2023-10-06T05:24:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.773+0000 7f4ff3fff700 1 -- 172.21.15.31:0/2655849071 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8006280 con 0x7f4ffc13a1e0 2023-10-06T05:24:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.774+0000 7f50035ff700 1 -- 172.21.15.31:0/2655849071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc13a1e0 msgr2=0x7f4ffc0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.774+0000 7f50035ff700 1 --2- 172.21.15.31:0/2655849071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc13a1e0 0x7f4ffc0c66f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80098d0 tx=0x7f4ff8009120 comp rx=0 tx=0).stop 2023-10-06T05:24:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.774+0000 7f50035ff700 1 -- 172.21.15.31:0/2655849071 shutdown_connections 2023-10-06T05:24:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.774+0000 7f50035ff700 1 --2- 172.21.15.31:0/2655849071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc13a1e0 0x7f4ffc0c66f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.774+0000 7f50035ff700 1 -- 172.21.15.31:0/2655849071 >> 172.21.15.31:0/2655849071 conn(0x7f4ffc0cc0b0 msgr2=0x7f4ffc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.774+0000 7f50035ff700 1 -- 172.21.15.31:0/2655849071 shutdown_connections 2023-10-06T05:24:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.774+0000 7f50035ff700 1 -- 172.21.15.31:0/2655849071 wait complete. 2023-10-06T05:24:07.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.776+0000 7f50035ff700 1 Processor -- start 2023-10-06T05:24:07.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.776+0000 7f50035ff700 1 -- start start 2023-10-06T05:24:07.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f50035ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ffc06db50 0x7f4ffc1391d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f50035ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ffc13a1e0 0x7f4ffc0c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f50035ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc13a5b0 0x7f4ffc0c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f50035ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc13d760 con 0x7f4ffc06db50 2023-10-06T05:24:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f50035ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0c59a0 con 0x7f4ffc13a5b0 2023-10-06T05:24:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f50035ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0c5e30 con 0x7f4ffc13a1e0 2023-10-06T05:24:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f500139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ffc06db50 0x7f4ffc1391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f500139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ffc06db50 0x7f4ffc1391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49290/0 (socket says 172.21.15.31:49290) 2023-10-06T05:24:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f500139b700 1 -- 172.21.15.31:0/2023024853 learned_addr learned my addr 172.21.15.31:0/2023024853 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f5000b9a700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ffc13a1e0 0x7f4ffc0c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f5001b9c700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc13a5b0 0x7f4ffc0c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f500139b700 1 -- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ffc13a1e0 msgr2=0x7f4ffc0c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f500139b700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ffc13a1e0 0x7f4ffc0c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f500139b700 1 -- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc13a5b0 msgr2=0x7f4ffc0c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f500139b700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc13a5b0 0x7f4ffc0c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.777+0000 7f500139b700 1 -- 172.21.15.31:0/2023024853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff80093c0 con 0x7f4ffc06db50 2023-10-06T05:24:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.778+0000 7f500139b700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ffc06db50 0x7f4ffc1391d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0cc010 tx=0x7f4ff8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.778+0000 7f4ff17fa700 1 -- 172.21.15.31:0/2023024853 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff801d780 con 0x7f4ffc06db50 2023-10-06T05:24:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.778+0000 7f4ff17fa700 1 -- 172.21.15.31:0/2023024853 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff801d900 con 0x7f4ffc06db50 2023-10-06T05:24:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.778+0000 7f50035ff700 1 -- 172.21.15.31:0/2023024853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc0c4d90 con 0x7f4ffc06db50 2023-10-06T05:24:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.778+0000 7f4ff17fa700 1 -- 172.21.15.31:0/2023024853 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff801dcc0 con 0x7f4ffc06db50 2023-10-06T05:24:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.778+0000 7f50035ff700 1 -- 172.21.15.31:0/2023024853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc0c5350 con 0x7f4ffc06db50 2023-10-06T05:24:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.779+0000 7f50035ff700 1 -- 172.21.15.31:0/2023024853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ffc0c5c80 con 0x7f4ffc06db50 2023-10-06T05:24:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.780+0000 7f4ff17fa700 1 -- 172.21.15.31:0/2023024853 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f4ff801b690 con 0x7f4ffc06db50 2023-10-06T05:24:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.781+0000 7f4ff17fa700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4fe00674d0 0x7f4fe0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.781+0000 7f4ff17fa700 1 -- 172.21.15.31:0/2023024853 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f4ff8086c80 con 0x7f4ffc06db50 2023-10-06T05:24:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.781+0000 7f5000b9a700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4fe00674d0 0x7f4fe0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:07.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.782+0000 7f5000b9a700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4fe00674d0 0x7f4fe0069980 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0ae930 tx=0x7f4fec007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:07.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.785+0000 7f4ff17fa700 1 -- 172.21.15.31:0/2023024853 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ff8053500 con 0x7f4ffc06db50 2023-10-06T05:24:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.991+0000 7f50035ff700 1 -- 172.21.15.31:0/2023024853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ffc04c380 con 0x7f4ffc06db50 2023-10-06T05:24:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.992+0000 7f4ff17fa700 1 -- 172.21.15.31:0/2023024853 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16972 (secure 0 0 0) 0x7f4ff8056b20 con 0x7f4ffc06db50 2023-10-06T05:24:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:07.994 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":127,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:07.702885+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:03.921897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:05.190243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 -- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4fe00674d0 msgr2=0x7f4fe0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4fe00674d0 0x7f4fe0069980 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0ae930 tx=0x7f4fec007ec0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 -- 172.21.15.31:0/2023024853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ffc06db50 msgr2=0x7f4ffc1391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ffc06db50 0x7f4ffc1391d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0cc010 tx=0x7f4ff8016000 comp rx=0 tx=0).stop 2023-10-06T05:24:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 -- 172.21.15.31:0/2023024853 shutdown_connections 2023-10-06T05:24:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4fe00674d0 0x7f4fe0069980 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc13a5b0 0x7f4ffc0c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ffc06db50 0x7f4ffc1391d0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 --2- 172.21.15.31:0/2023024853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4ffc13a1e0 0x7f4ffc0c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 -- 172.21.15.31:0/2023024853 >> 172.21.15.31:0/2023024853 conn(0x7f4ffc0cc0b0 msgr2=0x7f4ffc069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 -- 172.21.15.31:0/2023024853 shutdown_connections 2023-10-06T05:24:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:07.994+0000 7f50035ff700 1 -- 172.21.15.31:0/2023024853 wait complete. 2023-10-06T05:24:08.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:24:08.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.215+0000 7ff7b25ff700 1 Processor -- start 2023-10-06T05:24:08.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.215+0000 7ff7b25ff700 1 -- start start 2023-10-06T05:24:08.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b25ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc340 0x7ff7ac0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b25ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff7ac15a4b0 0x7ff7ac0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b25ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac15a020 con 0x7ff7ac0cc710 2023-10-06T05:24:08.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b25ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac0b07e0 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b25ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac159cf0 con 0x7ff7ac0cc340 2023-10-06T05:24:08.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b0b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff7ac15a4b0 0x7ff7ac0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b0b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff7ac15a4b0 0x7ff7ac0aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49302/0 (socket says 172.21.15.31:49302) 2023-10-06T05:24:08.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b0b9c700 1 -- 172.21.15.31:0/2940860396 learned_addr learned my addr 172.21.15.31:0/2940860396 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:08.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7abfff700 1 --2- 172.21.15.31:0/2940860396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc340 0x7ff7ac0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b0b9c700 1 -- 172.21.15.31:0/2940860396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc340 msgr2=0x7ff7ac0b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b0b9c700 1 --2- 172.21.15.31:0/2940860396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc340 0x7ff7ac0b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b0b9c700 1 -- 172.21.15.31:0/2940860396 >> v1:172.21.15.62:6789/0 conn(0x7ff7ac0cc710 legacy=0x7ff7ac0b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.217+0000 7ff7b0b9c700 1 -- 172.21.15.31:0/2940860396 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac0ccb60 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.218+0000 7ff7b0b9c700 1 --2- 172.21.15.31:0/2940860396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 0x7ff7ac0aecd0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009f20 tx=0x7ff7a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2630e4020ef9242 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.218+0000 7ff7aaffd700 1 -- 172.21.15.31:0/2940860396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400e070 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.218+0000 7ff7aaffd700 1 -- 172.21.15.31:0/2940860396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a400ba40 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.218+0000 7ff7aaffd700 1 -- 172.21.15.31:0/2940860396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40092e0 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.218+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2940860396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 msgr2=0x7ff7ac0aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.218+0000 7ff7b25ff700 1 --2- 172.21.15.31:0/2940860396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 0x7ff7ac0aecd0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009f20 tx=0x7ff7a4009840 comp rx=0 tx=0).stop 2023-10-06T05:24:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.219+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2940860396 shutdown_connections 2023-10-06T05:24:08.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.219+0000 7ff7b25ff700 1 --2- 172.21.15.31:0/2940860396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 0x7ff7ac0aecd0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.219+0000 7ff7b25ff700 1 --2- 172.21.15.31:0/2940860396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc340 0x7ff7ac0b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.219+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2940860396 >> 172.21.15.31:0/2940860396 conn(0x7ff7ac068670 msgr2=0x7ff7ac068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:08.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.219+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2940860396 shutdown_connections 2023-10-06T05:24:08.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.219+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2940860396 wait complete. 2023-10-06T05:24:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.220+0000 7ff7b25ff700 1 Processor -- start 2023-10-06T05:24:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.220+0000 7ff7b25ff700 1 -- start start 2023-10-06T05:24:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.221+0000 7ff7b25ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7ac0cc340 0x7ff7ac15b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.221+0000 7ff7b25ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc710 0x7ff7ac146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.221+0000 7ff7b25ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 0x7ff7ac169b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.221+0000 7ff7b25ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac16aa30 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.221+0000 7ff7b25ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac16a470 con 0x7ff7ac0cc340 2023-10-06T05:24:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.221+0000 7ff7b25ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac16a750 con 0x7ff7ac0cc710 2023-10-06T05:24:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7b0b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 0x7ff7ac169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7b0b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 0x7ff7ac169b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49312/0 (socket says 172.21.15.31:49312) 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7b0b9c700 1 -- 172.21.15.31:0/2110824268 learned_addr learned my addr 172.21.15.31:0/2110824268 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7abfff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7ac0cc340 0x7ff7ac15b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7ab7fe700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc710 0x7ff7ac146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7b0b9c700 1 -- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc710 msgr2=0x7ff7ac146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7b0b9c700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc710 0x7ff7ac146b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7b0b9c700 1 -- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7ac0cc340 msgr2=0x7ff7ac15b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7b0b9c700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7ac0cc340 0x7ff7ac15b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7b0b9c700 1 -- 172.21.15.31:0/2110824268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4009ae0 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7ab7fe700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc710 0x7ff7ac146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7abfff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7ac0cc340 0x7ff7ac15b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7b0b9c700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 0x7ff7ac169b20 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac06dd80 tx=0x7ff7a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff793fff700 1 -- 172.21.15.31:0/2110824268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40032f0 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff793fff700 1 -- 172.21.15.31:0/2110824268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a4012e60 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.222+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2110824268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac15a9e0 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.223+0000 7ff793fff700 1 -- 172.21.15.31:0/2110824268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a401e680 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.223+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2110824268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac15b720 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.224+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2110824268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ac15aee0 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.225+0000 7ff793fff700 1 -- 172.21.15.31:0/2110824268 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ff7a401ea70 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.225+0000 7ff793fff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff7980674d0 0x7ff798069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.225+0000 7ff793fff700 1 -- 172.21.15.31:0/2110824268 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff7a4086e30 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.226+0000 7ff7abfff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff7980674d0 0x7ff798069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.227+0000 7ff7abfff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff7980674d0 0x7ff798069980 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff79c005fd0 tx=0x7ff79c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.230+0000 7ff793fff700 1 -- 172.21.15.31:0/2110824268 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7a4053560 con 0x7ff7ac15a4b0 2023-10-06T05:24:08.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:08.475+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2110824268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff7ac04c380 con 0x7ff7ac15a4b0 2023-10-06T05:24:09.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.122+0000 7ff793fff700 1 -- 172.21.15.31:0/2110824268 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff7a4056b80 con 0x7ff7ac15a4b0 2023-10-06T05:24:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff7980674d0 msgr2=0x7ff798069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff7980674d0 0x7ff798069980 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff79c005fd0 tx=0x7ff79c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2110824268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 msgr2=0x7ff7ac169b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 0x7ff7ac169b20 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac06dd80 tx=0x7ff7a4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2110824268 shutdown_connections 2023-10-06T05:24:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff7980674d0 0x7ff798069980 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7ac15a4b0 0x7ff7ac169b20 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7ac0cc340 0x7ff7ac15b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 --2- 172.21.15.31:0/2110824268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7ac0cc710 0x7ff7ac146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2110824268 >> 172.21.15.31:0/2110824268 conn(0x7ff7ac068670 msgr2=0x7ff7ac15bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2110824268 shutdown_connections 2023-10-06T05:24:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.125+0000 7ff7b25ff700 1 -- 172.21.15.31:0/2110824268 wait complete. 2023-10-06T05:24:09.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:24:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.347+0000 7f59bf3ff700 1 Processor -- start 2023-10-06T05:24:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.347+0000 7f59bf3ff700 1 -- start start 2023-10-06T05:24:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.348+0000 7f59bf3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59b81573a0 0x7f59b80acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.348+0000 7f59bf3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f59b8157770 0x7f59b80b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.348+0000 7f59bf3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b8158cf0 con 0x7f59b8157c10 2023-10-06T05:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.348+0000 7f59bf3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b81580e0 con 0x7f59b81573a0 2023-10-06T05:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.348+0000 7f59bf3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b81583c0 con 0x7f59b8157770 2023-10-06T05:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.349+0000 7f59bd19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59b81573a0 0x7f59b80acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.349+0000 7f59bd19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59b81573a0 0x7f59b80acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49324/0 (socket says 172.21.15.31:49324) 2023-10-06T05:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.349+0000 7f59bd19b700 1 -- 172.21.15.31:0/3836756901 learned_addr learned my addr 172.21.15.31:0/3836756901 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.349+0000 7f59bc99a700 1 --2- 172.21.15.31:0/3836756901 >> v2:172.21.15.62:3300/0 conn(0x7f59b8157770 0x7f59b80b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.349+0000 7f59bd19b700 1 -- 172.21.15.31:0/3836756901 >> v2:172.21.15.62:3300/0 conn(0x7f59b8157770 msgr2=0x7f59b80b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.349+0000 7f59bd19b700 1 --2- 172.21.15.31:0/3836756901 >> v2:172.21.15.62:3300/0 conn(0x7f59b8157770 0x7f59b80b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.349+0000 7f59bd19b700 1 -- 172.21.15.31:0/3836756901 >> v1:172.21.15.62:6789/0 conn(0x7f59b8157c10 legacy=0x7f59b80ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.349+0000 7f59bd19b700 1 -- 172.21.15.31:0/3836756901 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b81586a0 con 0x7f59b81573a0 2023-10-06T05:24:09.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.349+0000 7f59bd19b700 1 --2- 172.21.15.31:0/3836756901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 0x7f59b80acb70 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f59b806b3a0 tx=0x7f59b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729c24bd07f11370 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59affff700 1 -- 172.21.15.31:0/3836756901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400b860 con 0x7f59b81573a0 2023-10-06T05:24:09.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59affff700 1 -- 172.21.15.31:0/3836756901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b400b9c0 con 0x7f59b81573a0 2023-10-06T05:24:09.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59affff700 1 -- 172.21.15.31:0/3836756901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400bd60 con 0x7f59b81573a0 2023-10-06T05:24:09.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59bf3ff700 1 -- 172.21.15.31:0/3836756901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 msgr2=0x7f59b80acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59bf3ff700 1 --2- 172.21.15.31:0/3836756901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 0x7f59b80acb70 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f59b806b3a0 tx=0x7f59b4009250 comp rx=0 tx=0).stop 2023-10-06T05:24:09.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59bf3ff700 1 -- 172.21.15.31:0/3836756901 shutdown_connections 2023-10-06T05:24:09.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59bf3ff700 1 --2- 172.21.15.31:0/3836756901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 0x7f59b80acb70 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59bf3ff700 1 --2- 172.21.15.31:0/3836756901 >> v2:172.21.15.62:3300/0 conn(0x7f59b8157770 0x7f59b80b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59bf3ff700 1 -- 172.21.15.31:0/3836756901 >> 172.21.15.31:0/3836756901 conn(0x7f59b8067560 msgr2=0x7f59b8067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:09.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59bf3ff700 1 -- 172.21.15.31:0/3836756901 shutdown_connections 2023-10-06T05:24:09.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.350+0000 7f59bf3ff700 1 -- 172.21.15.31:0/3836756901 wait complete. 2023-10-06T05:24:09.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.353+0000 7f59bf3ff700 1 Processor -- start 2023-10-06T05:24:09.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.353+0000 7f59bf3ff700 1 -- start start 2023-10-06T05:24:09.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.354+0000 7f59bf3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 0x7f59b8143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.354+0000 7f59bf3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b8157770 0x7f59b80ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.354+0000 7f59bf3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59b8157c10 0x7f59b80add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.354+0000 7f59bf3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8146c50 con 0x7f59b81573a0 2023-10-06T05:24:09.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.354+0000 7f59bf3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8146040 con 0x7f59b8157c10 2023-10-06T05:24:09.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.354+0000 7f59bf3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8146320 con 0x7f59b8157770 2023-10-06T05:24:09.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.354+0000 7f59bd19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 0x7f59b8143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.354+0000 7f59bd19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 0x7f59b8143490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49336/0 (socket says 172.21.15.31:49336) 2023-10-06T05:24:09.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.354+0000 7f59bd19b700 1 -- 172.21.15.31:0/2260148502 learned_addr learned my addr 172.21.15.31:0/2260148502 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:09.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59bd19b700 1 -- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b8157770 msgr2=0x7f59b80ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59bd19b700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b8157770 0x7f59b80ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59bd19b700 1 -- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59b8157c10 msgr2=0x7f59b80add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:09.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59bd19b700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59b8157c10 0x7f59b80add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:09.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59bd19b700 1 -- 172.21.15.31:0/2260148502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40094f0 con 0x7f59b81573a0 2023-10-06T05:24:09.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59bd19b700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 0x7f59b8143490 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f59b8064290 tx=0x7f59b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59ad7fa700 1 -- 172.21.15.31:0/2260148502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4011e60 con 0x7f59b81573a0 2023-10-06T05:24:09.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59ad7fa700 1 -- 172.21.15.31:0/2260148502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b401a630 con 0x7f59b81573a0 2023-10-06T05:24:09.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59ad7fa700 1 -- 172.21.15.31:0/2260148502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b401a9f0 con 0x7f59b81573a0 2023-10-06T05:24:09.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59bf3ff700 1 -- 172.21.15.31:0/2260148502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b8146600 con 0x7f59b81573a0 2023-10-06T05:24:09.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.355+0000 7f59bf3ff700 1 -- 172.21.15.31:0/2260148502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b81439d0 con 0x7f59b81573a0 2023-10-06T05:24:09.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.356+0000 7f59bf3ff700 1 -- 172.21.15.31:0/2260148502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b804c380 con 0x7f59b81573a0 2023-10-06T05:24:09.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.357+0000 7f59ad7fa700 1 -- 172.21.15.31:0/2260148502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f59b402d440 con 0x7f59b81573a0 2023-10-06T05:24:09.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.359+0000 7f59ad7fa700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f599c0674d0 0x7f599c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:09.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.359+0000 7f59ad7fa700 1 -- 172.21.15.31:0/2260148502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f59b4086d00 con 0x7f59b81573a0 2023-10-06T05:24:09.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.359+0000 7f59bc99a700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f599c0674d0 0x7f599c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:09.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.360+0000 7f59bc99a700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f599c0674d0 0x7f599c069980 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59b80c8fb0 tx=0x7f59a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.363+0000 7f59ad7fa700 1 -- 172.21.15.31:0/2260148502 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59b4053580 con 0x7f59b81573a0 2023-10-06T05:24:09.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:09.613+0000 7f59bf3ff700 1 -- 172.21.15.31:0/2260148502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f59b80b1fb0 con 0x7f59b81573a0 2023-10-06T05:24:10.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.128+0000 7f59ad7fa700 1 -- 172.21.15.31:0/2260148502 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) v1 ==== 122+0+0 (secure 0 0 0) 0x7f59b4056ba0 con 0x7f59b81573a0 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 -- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f599c0674d0 msgr2=0x7f599c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f599c0674d0 0x7f599c069980 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59b80c8fb0 tx=0x7f59a8008040 comp rx=0 tx=0).stop 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 -- 172.21.15.31:0/2260148502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 msgr2=0x7f59b8143490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 0x7f59b8143490 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f59b8064290 tx=0x7f59b4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 -- 172.21.15.31:0/2260148502 shutdown_connections 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f599c0674d0 0x7f599c069980 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59b8157c10 0x7f59b80add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b81573a0 0x7f59b8143490 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 --2- 172.21.15.31:0/2260148502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59b8157770 0x7f59b80ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 -- 172.21.15.31:0/2260148502 >> 172.21.15.31:0/2260148502 conn(0x7f59b8067560 msgr2=0x7f59b815a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 -- 172.21.15.31:0/2260148502 shutdown_connections 2023-10-06T05:24:10.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.131+0000 7f59bf3ff700 1 -- 172.21.15.31:0/2260148502 wait complete. 2023-10-06T05:24:10.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.348+0000 7ff6537ff700 1 Processor -- start 2023-10-06T05:24:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.348+0000 7ff6537ff700 1 -- start start 2023-10-06T05:24:10.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.350+0000 7ff6537ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff64c0c7b60 0x7ff64c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.350+0000 7ff6537ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff64c0c7f30 0x7ff64c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.350+0000 7ff6537ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64c0ca5b0 con 0x7ff64c06ef50 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.350+0000 7ff6537ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64c06f740 con 0x7ff64c0c7f30 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.350+0000 7ff6537ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64c0c9c80 con 0x7ff64c0c7b60 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.350+0000 7ff650d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff64c0c7f30 0x7ff64c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.350+0000 7ff650d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff64c0c7f30 0x7ff64c0aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49338/0 (socket says 172.21.15.31:49338) 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.350+0000 7ff650d9a700 1 -- 172.21.15.31:0/508133596 learned_addr learned my addr 172.21.15.31:0/508133596 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.350+0000 7ff65159b700 1 --2- 172.21.15.31:0/508133596 >> v2:172.21.15.62:3300/0 conn(0x7ff64c0c7b60 0x7ff64c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.351+0000 7ff650d9a700 1 -- 172.21.15.31:0/508133596 >> v2:172.21.15.62:3300/0 conn(0x7ff64c0c7b60 msgr2=0x7ff64c069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.351+0000 7ff650d9a700 1 --2- 172.21.15.31:0/508133596 >> v2:172.21.15.62:3300/0 conn(0x7ff64c0c7b60 0x7ff64c069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.351+0000 7ff650d9a700 1 -- 172.21.15.31:0/508133596 >> v1:172.21.15.62:6789/0 conn(0x7ff64c06ef50 legacy=0x7ff64c13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.351+0000 7ff650d9a700 1 -- 172.21.15.31:0/508133596 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c0c9f60 con 0x7ff64c0c7f30 2023-10-06T05:24:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.351+0000 7ff650d9a700 1 --2- 172.21.15.31:0/508133596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7f30 0x7ff64c0aee70 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00bc50 tx=0x7ff63c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b994bb34c0ed57a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.351+0000 7ff643fff700 1 -- 172.21.15.31:0/508133596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00dbb0 con 0x7ff64c0c7f30 2023-10-06T05:24:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.351+0000 7ff643fff700 1 -- 172.21.15.31:0/508133596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c00dd10 con 0x7ff64c0c7f30 2023-10-06T05:24:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.351+0000 7ff643fff700 1 -- 172.21.15.31:0/508133596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c0196b0 con 0x7ff64c0c7f30 2023-10-06T05:24:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.352+0000 7ff6537ff700 1 -- 172.21.15.31:0/508133596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7f30 msgr2=0x7ff64c0aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.352+0000 7ff6537ff700 1 --2- 172.21.15.31:0/508133596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7f30 0x7ff64c0aee70 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00bc50 tx=0x7ff63c00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.352+0000 7ff6537ff700 1 -- 172.21.15.31:0/508133596 shutdown_connections 2023-10-06T05:24:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.352+0000 7ff6537ff700 1 --2- 172.21.15.31:0/508133596 >> v2:172.21.15.62:3300/0 conn(0x7ff64c0c7b60 0x7ff64c069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.352+0000 7ff6537ff700 1 --2- 172.21.15.31:0/508133596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7f30 0x7ff64c0aee70 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.352+0000 7ff6537ff700 1 -- 172.21.15.31:0/508133596 >> 172.21.15.31:0/508133596 conn(0x7ff64c0cc0b0 msgr2=0x7ff64c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.352+0000 7ff6537ff700 1 -- 172.21.15.31:0/508133596 shutdown_connections 2023-10-06T05:24:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.352+0000 7ff6537ff700 1 -- 172.21.15.31:0/508133596 wait complete. 2023-10-06T05:24:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.354+0000 7ff6537ff700 1 Processor -- start 2023-10-06T05:24:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.354+0000 7ff6537ff700 1 -- start start 2023-10-06T05:24:10.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.355+0000 7ff6537ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff64c06ef50 0x7ff64c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.355+0000 7ff6537ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7b60 0x7ff64c157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.355+0000 7ff6537ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff64c0c7f30 0x7ff64c13de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.355+0000 7ff6537ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c158000 con 0x7ff64c0c7b60 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.355+0000 7ff6537ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0aca90 con 0x7ff64c06ef50 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.355+0000 7ff6537ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0acd30 con 0x7ff64c0c7f30 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff650d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7b60 0x7ff64c157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff650d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7b60 0x7ff64c157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49340/0 (socket says 172.21.15.31:49340) 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff650d9a700 1 -- 172.21.15.31:0/2126156817 learned_addr learned my addr 172.21.15.31:0/2126156817 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff65159b700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff64c06ef50 0x7ff64c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff651d9c700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff64c0c7f30 0x7ff64c13de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff650d9a700 1 -- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff64c0c7f30 msgr2=0x7ff64c13de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff650d9a700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff64c0c7f30 0x7ff64c13de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff650d9a700 1 -- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff64c06ef50 msgr2=0x7ff64c155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff650d9a700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff64c06ef50 0x7ff64c155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff650d9a700 1 -- 172.21.15.31:0/2126156817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c00b790 con 0x7ff64c0c7b60 2023-10-06T05:24:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.356+0000 7ff650d9a700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7b60 0x7ff64c157ac0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7ff64c06cd80 tx=0x7ff63c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.357+0000 7ff6417fa700 1 -- 172.21.15.31:0/2126156817 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c010470 con 0x7ff64c0c7b60 2023-10-06T05:24:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.357+0000 7ff6417fa700 1 -- 172.21.15.31:0/2126156817 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c0105d0 con 0x7ff64c0c7b60 2023-10-06T05:24:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.357+0000 7ff6417fa700 1 -- 172.21.15.31:0/2126156817 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c010970 con 0x7ff64c0c7b60 2023-10-06T05:24:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.357+0000 7ff6537ff700 1 -- 172.21.15.31:0/2126156817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64c0ad010 con 0x7ff64c0c7b60 2023-10-06T05:24:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.357+0000 7ff6537ff700 1 -- 172.21.15.31:0/2126156817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64c0ad5d0 con 0x7ff64c0c7b60 2023-10-06T05:24:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.358+0000 7ff6537ff700 1 -- 172.21.15.31:0/2126156817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64c04c380 con 0x7ff64c0c7b60 2023-10-06T05:24:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.359+0000 7ff6417fa700 1 -- 172.21.15.31:0/2126156817 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ff63c022030 con 0x7ff64c0c7b60 2023-10-06T05:24:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.359+0000 7ff6417fa700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6300674d0 0x7ff630069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.360+0000 7ff6417fa700 1 -- 172.21.15.31:0/2126156817 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff63c014070 con 0x7ff64c0c7b60 2023-10-06T05:24:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.360+0000 7ff65159b700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6300674d0 0x7ff630069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.361+0000 7ff65159b700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6300674d0 0x7ff630069980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0cbc70 tx=0x7ff648009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.364+0000 7ff6417fa700 1 -- 172.21.15.31:0/2126156817 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff63c055b70 con 0x7ff64c0c7b60 2023-10-06T05:24:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.614+0000 7ff6537ff700 1 -- 172.21.15.31:0/2126156817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff64c0c7540 con 0x7ff64c0c7b60 2023-10-06T05:24:10.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.615+0000 7ff6417fa700 1 -- 172.21.15.31:0/2126156817 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4601 (secure 0 0 0) 0x7ff63c059190 con 0x7ff64c0c7b60 2023-10-06T05:24:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:10.617 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":84,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3816040413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3816040413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3816040413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1295480750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1295480750},{"type":"v1","addr":"172.21.15.31:6834","nonce":1295480750}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2807869230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2807869230},{"type":"v1","addr":"172.21.15.62:6834","nonce":2807869230}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:06.204405+0000","modified":"2023-10-06T05:24:10.122253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7995":{"gid":7995,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1928465499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1928465499},{"type":"v1","addr":"172.21.15.62:6835","nonce":1928465499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:10.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 -- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6300674d0 msgr2=0x7ff630069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6300674d0 0x7ff630069980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0cbc70 tx=0x7ff648009120 comp rx=0 tx=0).stop 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 -- 172.21.15.31:0/2126156817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7b60 msgr2=0x7ff64c157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7b60 0x7ff64c157ac0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7ff64c06cd80 tx=0x7ff63c01b000 comp rx=0 tx=0).stop 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 -- 172.21.15.31:0/2126156817 shutdown_connections 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6300674d0 0x7ff630069980 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff64c0c7f30 0x7ff64c13de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff64c06ef50 0x7ff64c155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 --2- 172.21.15.31:0/2126156817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff64c0c7b60 0x7ff64c157ac0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 -- 172.21.15.31:0/2126156817 >> 172.21.15.31:0/2126156817 conn(0x7ff64c0cc0b0 msgr2=0x7ff64c0c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.617+0000 7ff6537ff700 1 -- 172.21.15.31:0/2126156817 shutdown_connections 2023-10-06T05:24:10.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.618+0000 7ff6537ff700 1 -- 172.21.15.31:0/2126156817 wait complete. 2023-10-06T05:24:10.621 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 84 2023-10-06T05:24:10.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:10.671 INFO:tasks.ceph.mds.b.smithi062.stderr:2023-10-06T05:24:10.669+0000 7fb673ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:24:10.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.831+0000 7ff1bedff700 1 Processor -- start 2023-10-06T05:24:10.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.831+0000 7ff1bedff700 1 -- start start 2023-10-06T05:24:10.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.832+0000 7ff1bedff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff1b81595a0 0x7ff1b8159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.832+0000 7ff1bedff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b80c95b0 con 0x7ff1b80cc0d0 2023-10-06T05:24:10.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.832+0000 7ff1bedff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b80c9890 con 0x7ff1b8068d80 2023-10-06T05:24:10.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.832+0000 7ff1bedff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b80c9b70 con 0x7ff1b81595a0 2023-10-06T05:24:10.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.832+0000 7ff1b7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff1b80cc0d0 0x7ff1b8068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39588/0 (socket says 172.21.15.31:39588) 2023-10-06T05:24:10.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.832+0000 7ff1b7fff700 1 -- 172.21.15.31:0/114455713 learned_addr learned my addr 172.21.15.31:0/114455713 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:10.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1bcb9b700 1 --2- 172.21.15.31:0/114455713 >> v2:172.21.15.62:3300/0 conn(0x7ff1b81595a0 0x7ff1b8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108236490 0 0) 0x7ff1b80c95b0 con 0x7ff1b80cc0d0 2023-10-06T05:24:10.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1a8003570 con 0x7ff1b80cc0d0 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1987855973 0 0) 0x7ff1a8003570 con 0x7ff1b80cc0d0 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 >> v2:172.21.15.62:3300/0 conn(0x7ff1b81595a0 msgr2=0x7ff1b8159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 --2- 172.21.15.31:0/114455713 >> v2:172.21.15.62:3300/0 conn(0x7ff1b81595a0 0x7ff1b8159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 >> v1:172.21.15.62:6789/0 conn(0x7ff1b8068d80 legacy=0x7ff1b80c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b80ca130 con 0x7ff1b80cc0d0 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7ff1a4001470 con 0x7ff1b80cc0d0 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1bcb9b700 1 --2- 172.21.15.31:0/114455713 >> v2:172.21.15.62:3300/0 conn(0x7ff1b81595a0 0x7ff1b8159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 >> v1:172.21.15.31:6789/0 conn(0x7ff1b80cc0d0 legacy=0x7ff1b8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff1a8003a60 0x7ff1a8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1a8004810 0x7ff1a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a80051c0 0x7ff1a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b80c9b70 con 0x7ff1a8004810 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a800c0d0 con 0x7ff1a80051c0 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.833+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a800c290 con 0x7ff1a8003a60 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.834+0000 7ff1b7fff700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1a8004810 0x7ff1a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.834+0000 7ff1bcb9b700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff1a8003a60 0x7ff1a8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.834+0000 7ff1bd39c700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a80051c0 0x7ff1a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.834+0000 7ff1b7fff700 1 -- 172.21.15.31:0/114455713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff1a8003a60 msgr2=0x7ff1a8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.834+0000 7ff1b7fff700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff1a8003a60 0x7ff1a8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.834+0000 7ff1b7fff700 1 -- 172.21.15.31:0/114455713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a80051c0 msgr2=0x7ff1a800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.834+0000 7ff1b7fff700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a80051c0 0x7ff1a800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.834+0000 7ff1b7fff700 1 -- 172.21.15.31:0/114455713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a8003570 con 0x7ff1a8004810 2023-10-06T05:24:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.834+0000 7ff1bd39c700 1 -- 172.21.15.31:0/114455713 reap_dead start 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.834+0000 7ff1b7fff700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1a8004810 0x7ff1a8004c80 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400c3c0 tx=0x7ff1a4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528f55b8364d8c1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.835+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a400cc90 con 0x7ff1a8004810 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.835+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a400cdf0 con 0x7ff1a8004810 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.835+0000 7ff1b77fe700 1 -- 172.21.15.31:0/114455713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4005b50 con 0x7ff1a8004810 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.835+0000 7ff1bedff700 1 -- 172.21.15.31:0/114455713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1a8004810 msgr2=0x7ff1a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.835+0000 7ff1bedff700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1a8004810 0x7ff1a8004c80 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400c3c0 tx=0x7ff1a4001e50 comp rx=0 tx=0).stop 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.835+0000 7ff1bedff700 1 -- 172.21.15.31:0/114455713 shutdown_connections 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.835+0000 7ff1bedff700 1 --2- 172.21.15.31:0/114455713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1a8004810 0x7ff1a8004c80 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.835+0000 7ff1bedff700 1 -- 172.21.15.31:0/114455713 >> 172.21.15.31:0/114455713 conn(0x7ff1b806b620 msgr2=0x7ff1b806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.835+0000 7ff1bedff700 1 -- 172.21.15.31:0/114455713 shutdown_connections 2023-10-06T05:24:10.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.835+0000 7ff1bedff700 1 -- 172.21.15.31:0/114455713 wait complete. 2023-10-06T05:24:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.837+0000 7ff1bedff700 1 Processor -- start 2023-10-06T05:24:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.837+0000 7ff1bedff700 1 -- start start 2023-10-06T05:24:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bedff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a8004810 0x7ff1b815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bedff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff1b80b3ad0 0x7ff1b814f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bedff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1b80cce90 0x7ff1b80cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bedff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b80cd870 con 0x7ff1b80cce90 2023-10-06T05:24:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bedff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b815f950 con 0x7ff1a8004810 2023-10-06T05:24:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bedff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b815fde0 con 0x7ff1b80b3ad0 2023-10-06T05:24:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bd39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1b80cce90 0x7ff1b80cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bd39c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1b80cce90 0x7ff1b80cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49356/0 (socket says 172.21.15.31:49356) 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bd39c700 1 -- 172.21.15.31:0/210409068 learned_addr learned my addr 172.21.15.31:0/210409068 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bcb9b700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a8004810 0x7ff1b815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1b7fff700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff1b80b3ad0 0x7ff1b814f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bd39c700 1 -- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff1b80b3ad0 msgr2=0x7ff1b814f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bd39c700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff1b80b3ad0 0x7ff1b814f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bd39c700 1 -- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a8004810 msgr2=0x7ff1b815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bd39c700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a8004810 0x7ff1b815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.838+0000 7ff1bd39c700 1 -- 172.21.15.31:0/210409068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b815ed40 con 0x7ff1b80cce90 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.839+0000 7ff1bd39c700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1b80cce90 0x7ff1b80cd330 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8069180 tx=0x7ff1b0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.839+0000 7ff1b4ff9700 1 -- 172.21.15.31:0/210409068 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0006e80 con 0x7ff1b80cce90 2023-10-06T05:24:10.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.839+0000 7ff1b4ff9700 1 -- 172.21.15.31:0/210409068 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b000c5d0 con 0x7ff1b80cce90 2023-10-06T05:24:10.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.839+0000 7ff1bedff700 1 -- 172.21.15.31:0/210409068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b815f020 con 0x7ff1b80cce90 2023-10-06T05:24:10.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.839+0000 7ff1b4ff9700 1 -- 172.21.15.31:0/210409068 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000c970 con 0x7ff1b80cce90 2023-10-06T05:24:10.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.839+0000 7ff1bedff700 1 -- 172.21.15.31:0/210409068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b815f5e0 con 0x7ff1b80cce90 2023-10-06T05:24:10.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.840+0000 7ff1bedff700 1 -- 172.21.15.31:0/210409068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b804c380 con 0x7ff1b80cce90 2023-10-06T05:24:10.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.841+0000 7ff1b4ff9700 1 -- 172.21.15.31:0/210409068 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ff1b00133e0 con 0x7ff1b80cce90 2023-10-06T05:24:10.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.842+0000 7ff1b4ff9700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff19c0674d0 0x7ff19c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:10.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.842+0000 7ff1b4ff9700 1 -- 172.21.15.31:0/210409068 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff1b0084c10 con 0x7ff1b80cce90 2023-10-06T05:24:10.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.842+0000 7ff1bcb9b700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff19c0674d0 0x7ff19c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:10.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.843+0000 7ff1bcb9b700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff19c0674d0 0x7ff19c069980 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff1b806c900 tx=0x7ff1ac0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:10.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:10.846+0000 7ff1b4ff9700 1 -- 172.21.15.31:0/210409068 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1b0051490 con 0x7ff1b80cce90 2023-10-06T05:24:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.048+0000 7ff1bedff700 1 -- 172.21.15.31:0/210409068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1b815fa90 con 0x7ff1b80cce90 2023-10-06T05:24:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.049+0000 7ff1b4ff9700 1 -- 172.21.15.31:0/210409068 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16972 (secure 0 0 0) 0x7ff1b0019090 con 0x7ff1b80cce90 2023-10-06T05:24:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":127,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:07.702885+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:03.921897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:05.190243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:11.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 -- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff19c0674d0 msgr2=0x7ff19c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff19c0674d0 0x7ff19c069980 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff1b806c900 tx=0x7ff1ac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 -- 172.21.15.31:0/210409068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1b80cce90 msgr2=0x7ff1b80cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1b80cce90 0x7ff1b80cd330 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8069180 tx=0x7ff1b0001640 comp rx=0 tx=0).stop 2023-10-06T05:24:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 -- 172.21.15.31:0/210409068 shutdown_connections 2023-10-06T05:24:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff19c0674d0 0x7ff19c069980 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1b80cce90 0x7ff1b80cd330 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1a8004810 0x7ff1b815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 --2- 172.21.15.31:0/210409068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff1b80b3ad0 0x7ff1b814f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 -- 172.21.15.31:0/210409068 >> 172.21.15.31:0/210409068 conn(0x7ff1b806b620 msgr2=0x7ff1b80ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 -- 172.21.15.31:0/210409068 shutdown_connections 2023-10-06T05:24:11.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.051+0000 7ff1bedff700 1 -- 172.21.15.31:0/210409068 wait complete. 2023-10-06T05:24:11.071 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:24:11.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:24:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.266+0000 7f745b7ff700 1 Processor -- start 2023-10-06T05:24:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.266+0000 7f745b7ff700 1 -- start start 2023-10-06T05:24:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.268+0000 7f745b7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 0x7f74540c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.268+0000 7f745b7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74540cab40 0x7f74541569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.268+0000 7f745b7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7454156ef0 con 0x7f74540ca770 2023-10-06T05:24:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.268+0000 7f745b7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74540c7a70 con 0x7f74540cab40 2023-10-06T05:24:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.268+0000 7f745b7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74540c6df0 con 0x7f745406b510 2023-10-06T05:24:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.268+0000 7f7459d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74540cab40 0x7f74541569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.268+0000 7f7459d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74540cab40 0x7f74541569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49358/0 (socket says 172.21.15.31:49358) 2023-10-06T05:24:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.268+0000 7f7459d9c700 1 -- 172.21.15.31:0/3543650433 learned_addr learned my addr 172.21.15.31:0/3543650433 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.268+0000 7f7458d9a700 1 --2- 172.21.15.31:0/3543650433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 0x7f74540c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.269+0000 7f7459d9c700 1 -- 172.21.15.31:0/3543650433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 msgr2=0x7f74540c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:11.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.269+0000 7f7459d9c700 1 --2- 172.21.15.31:0/3543650433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 0x7f74540c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.269+0000 7f7459d9c700 1 -- 172.21.15.31:0/3543650433 >> v1:172.21.15.62:6789/0 conn(0x7f74540ca770 legacy=0x7f745406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.269+0000 7f7459d9c700 1 -- 172.21.15.31:0/3543650433 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74540c70d0 con 0x7f74540cab40 2023-10-06T05:24:11.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.269+0000 7f7459d9c700 1 --2- 172.21.15.31:0/3543650433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540cab40 0x7f74541569b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f744c00d1a0 tx=0x7f744c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393c13e6498d7223 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.269+0000 7f744bfff700 1 -- 172.21.15.31:0/3543650433 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c010070 con 0x7f74540cab40 2023-10-06T05:24:11.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.269+0000 7f744bfff700 1 -- 172.21.15.31:0/3543650433 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c00dcb0 con 0x7f74540cab40 2023-10-06T05:24:11.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.269+0000 7f744bfff700 1 -- 172.21.15.31:0/3543650433 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c016630 con 0x7f74540cab40 2023-10-06T05:24:11.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.270+0000 7f745b7ff700 1 -- 172.21.15.31:0/3543650433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540cab40 msgr2=0x7f74541569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.270+0000 7f745b7ff700 1 --2- 172.21.15.31:0/3543650433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540cab40 0x7f74541569b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f744c00d1a0 tx=0x7f744c00ba50 comp rx=0 tx=0).stop 2023-10-06T05:24:11.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.270+0000 7f745b7ff700 1 -- 172.21.15.31:0/3543650433 shutdown_connections 2023-10-06T05:24:11.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.270+0000 7f745b7ff700 1 --2- 172.21.15.31:0/3543650433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540cab40 0x7f74541569b0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.270+0000 7f745b7ff700 1 --2- 172.21.15.31:0/3543650433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 0x7f74540c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.270+0000 7f745b7ff700 1 -- 172.21.15.31:0/3543650433 >> 172.21.15.31:0/3543650433 conn(0x7f74540cc0b0 msgr2=0x7f74540cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.270+0000 7f745b7ff700 1 -- 172.21.15.31:0/3543650433 shutdown_connections 2023-10-06T05:24:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.270+0000 7f745b7ff700 1 -- 172.21.15.31:0/3543650433 wait complete. 2023-10-06T05:24:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.271+0000 7f745b7ff700 1 Processor -- start 2023-10-06T05:24:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.272+0000 7f745b7ff700 1 -- start start 2023-10-06T05:24:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.272+0000 7f745b7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 0x7f74540b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f745b7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540ca770 0x7f74540c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f745b7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74540cab40 0x7f745414bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f745b7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540ad700 con 0x7f74540ca770 2023-10-06T05:24:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f745b7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540acb30 con 0x7f74540cab40 2023-10-06T05:24:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f745b7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74540acdd0 con 0x7f745406b510 2023-10-06T05:24:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f7458d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540ca770 0x7f74540c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f7458d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540ca770 0x7f74540c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49370/0 (socket says 172.21.15.31:49370) 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f7458d9a700 1 -- 172.21.15.31:0/1899012825 learned_addr learned my addr 172.21.15.31:0/1899012825 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f7459d9c700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74540cab40 0x7f745414bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f745959b700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 0x7f74540b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f7458d9a700 1 -- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 msgr2=0x7f74540b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f7458d9a700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 0x7f74540b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f7458d9a700 1 -- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74540cab40 msgr2=0x7f745414bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f7458d9a700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74540cab40 0x7f745414bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f7458d9a700 1 -- 172.21.15.31:0/1899012825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c00bcf0 con 0x7f74540ca770 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.273+0000 7f745959b700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 0x7f74540b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.274+0000 7f7458d9a700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540ca770 0x7f74540c63a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f744400d860 tx=0x7f7444008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.274+0000 7f74497fa700 1 -- 172.21.15.31:0/1899012825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444009610 con 0x7f74540ca770 2023-10-06T05:24:11.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.274+0000 7f74497fa700 1 -- 172.21.15.31:0/1899012825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7444009770 con 0x7f74540ca770 2023-10-06T05:24:11.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.274+0000 7f745b7ff700 1 -- 172.21.15.31:0/1899012825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74540ad0b0 con 0x7f74540ca770 2023-10-06T05:24:11.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.274+0000 7f74497fa700 1 -- 172.21.15.31:0/1899012825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444009b30 con 0x7f74540ca770 2023-10-06T05:24:11.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.274+0000 7f745b7ff700 1 -- 172.21.15.31:0/1899012825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74540c68e0 con 0x7f74540ca770 2023-10-06T05:24:11.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.275+0000 7f745b7ff700 1 -- 172.21.15.31:0/1899012825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745404c380 con 0x7f74540ca770 2023-10-06T05:24:11.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.276+0000 7f74497fa700 1 -- 172.21.15.31:0/1899012825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f744401e460 con 0x7f74540ca770 2023-10-06T05:24:11.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.277+0000 7f74497fa700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f74380674d0 0x7f7438069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.277+0000 7f74497fa700 1 -- 172.21.15.31:0/1899012825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f74440857d0 con 0x7f74540ca770 2023-10-06T05:24:11.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.277+0000 7f745959b700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f74380674d0 0x7f7438069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.278+0000 7f745959b700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f74380674d0 0x7f7438069980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7450001870 tx=0x7f7450009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.281+0000 7f74497fa700 1 -- 172.21.15.31:0/1899012825 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7444052010 con 0x7f74540ca770 2023-10-06T05:24:11.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.485+0000 7f745b7ff700 1 -- 172.21.15.31:0/1899012825 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7454150370 con 0x7f74380674d0 2023-10-06T05:24:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.495+0000 7f74497fa700 1 -- 172.21.15.31:0/1899012825 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308554 (secure 0 0 0) 0x7f7454150370 con 0x7f74380674d0 2023-10-06T05:24:11.501 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.499+0000 7f745b7ff700 1 -- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f74380674d0 msgr2=0x7f7438069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.499+0000 7f745b7ff700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f74380674d0 0x7f7438069980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7450001870 tx=0x7f7450009040 comp rx=0 tx=0).stop 2023-10-06T05:24:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.500+0000 7f745b7ff700 1 -- 172.21.15.31:0/1899012825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540ca770 msgr2=0x7f74540c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.500+0000 7f745b7ff700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540ca770 0x7f74540c63a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f744400d860 tx=0x7f7444008040 comp rx=0 tx=0).stop 2023-10-06T05:24:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.500+0000 7f745b7ff700 1 -- 172.21.15.31:0/1899012825 shutdown_connections 2023-10-06T05:24:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.500+0000 7f745b7ff700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f74380674d0 0x7f7438069980 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.500+0000 7f745b7ff700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74540cab40 0x7f745414bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.500+0000 7f745b7ff700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f745406b510 0x7f74540b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.500+0000 7f745b7ff700 1 --2- 172.21.15.31:0/1899012825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74540ca770 0x7f74540c63a0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.500+0000 7f745b7ff700 1 -- 172.21.15.31:0/1899012825 >> 172.21.15.31:0/1899012825 conn(0x7f74540cc0b0 msgr2=0x7f74540cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.500+0000 7f745b7ff700 1 -- 172.21.15.31:0/1899012825 shutdown_connections 2023-10-06T05:24:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.500+0000 7f745b7ff700 1 -- 172.21.15.31:0/1899012825 wait complete. 2023-10-06T05:24:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:24:11.519 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2023-10-06T05:24:09.497074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2420380,"kb_used_data":13404,"kb_used_omap":0,"kb_used_meta":2406912,"kb_avail":752554340,"statfs":{"total":773094113280,"available":770615644160,"internally_reserved":0,"allocated":13725696,"data_stored":6993084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2464677888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.802987"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.711607+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:11.497121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.711374+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:45.374302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.712393+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:22.989870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.713750+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:25.833053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.707645+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:21.503302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.706669+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:27.986026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.709493+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:32.207879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.713981+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:06.740890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.708300+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:54.129394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.709044+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:58.940036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.709125+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:28.567275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.714244+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:15.034393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.706565+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:24.525339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.715960+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:57.634766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.711003+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:37.593003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.705435+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:08.465102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.712229+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:43.308652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710669+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:29.971759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710005+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:21.503302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.706866+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:58.940036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.705673+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:50.725523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.709166+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:59.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710541+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:37.680967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.708790+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:36:14.250192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.711079+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:24.149580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.714082+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:02.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.706634+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:51.773348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.708846+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:17.890637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.720214+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:14.382412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710021+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:20.969412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.707650+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:22.276001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.713499+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:21.753322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.711674+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:14.382412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.713591+0000","last_change":"2023-10-06T05:24:05.713591+0000","last_active":"2023-10-06T05:24:05.713591+0000","last_peered":"2023-10-06T05:24:05.713591+0000","last_clean":"2023-10-06T05:24:05.713591+0000","last_became_active":"2023-10-06T05:24:05.713310+0000","last_became_peered":"2023-10-06T05:24:05.713310+0000","last_unstale":"2023-10-06T05:24:05.713591+0000","last_undegraded":"2023-10-06T05:24:05.713591+0000","last_fullsized":"2023-10-06T05:24:05.713591+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:35.466776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.711921+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:59.983040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.738451+0000","last_change":"2023-10-06T05:24:05.738451+0000","last_active":"2023-10-06T05:24:05.738451+0000","last_peered":"2023-10-06T05:24:05.738451+0000","last_clean":"2023-10-06T05:24:05.738451+0000","last_became_active":"2023-10-06T05:24:05.738139+0000","last_became_peered":"2023-10-06T05:24:05.738139+0000","last_unstale":"2023-10-06T05:24:05.738451+0000","last_undegraded":"2023-10-06T05:24:05.738451+0000","last_fullsized":"2023-10-06T05:24:05.738451+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:49.715080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.709915+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:54.021504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.728447+0000","last_change":"2023-10-06T05:24:05.728447+0000","last_active":"2023-10-06T05:24:05.728447+0000","last_peered":"2023-10-06T05:24:05.728447+0000","last_clean":"2023-10-06T05:24:05.728447+0000","last_became_active":"2023-10-06T05:24:05.728167+0000","last_became_peered":"2023-10-06T05:24:05.728167+0000","last_unstale":"2023-10-06T05:24:05.728447+0000","last_undegraded":"2023-10-06T05:24:05.728447+0000","last_fullsized":"2023-10-06T05:24:05.728447+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:40.510468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.708786+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:04.206537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.727449+0000","last_change":"2023-10-06T05:24:05.727449+0000","last_active":"2023-10-06T05:24:05.727449+0000","last_peered":"2023-10-06T05:24:05.727449+0000","last_clean":"2023-10-06T05:24:05.727449+0000","last_became_active":"2023-10-06T05:24:05.727177+0000","last_became_peered":"2023-10-06T05:24:05.727177+0000","last_unstale":"2023-10-06T05:24:05.727449+0000","last_undegraded":"2023-10-06T05:24:05.727449+0000","last_fullsized":"2023-10-06T05:24:05.727449+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:26.823343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.705277+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:48.790796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.727924+0000","last_change":"2023-10-06T05:24:05.727924+0000","last_active":"2023-10-06T05:24:05.727924+0000","last_peered":"2023-10-06T05:24:05.727924+0000","last_clean":"2023-10-06T05:24:05.727924+0000","last_became_active":"2023-10-06T05:24:05.727521+0000","last_became_peered":"2023-10-06T05:24:05.727521+0000","last_unstale":"2023-10-06T05:24:05.727924+0000","last_undegraded":"2023-10-06T05:24:05.727924+0000","last_fullsized":"2023-10-06T05:24:05.727924+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:29.277714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.712831+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:37.917944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.717504+0000","last_change":"2023-10-06T05:24:05.717504+0000","last_active":"2023-10-06T05:24:05.717504+0000","last_peered":"2023-10-06T05:24:05.717504+0000","last_clean":"2023-10-06T05:24:05.717504+0000","last_became_active":"2023-10-06T05:24:05.717220+0000","last_became_peered":"2023-10-06T05:24:05.717220+0000","last_unstale":"2023-10-06T05:24:05.717504+0000","last_undegraded":"2023-10-06T05:24:05.717504+0000","last_fullsized":"2023-10-06T05:24:05.717504+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:11.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.709834+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:55.900781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.720465+0000","last_change":"2023-10-06T05:24:05.720465+0000","last_active":"2023-10-06T05:24:05.720465+0000","last_peered":"2023-10-06T05:24:05.720465+0000","last_clean":"2023-10-06T05:24:05.720465+0000","last_became_active":"2023-10-06T05:24:05.720063+0000","last_became_peered":"2023-10-06T05:24:05.720063+0000","last_unstale":"2023-10-06T05:24:05.720465+0000","last_undegraded":"2023-10-06T05:24:05.720465+0000","last_fullsized":"2023-10-06T05:24:05.720465+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:20.742878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710130+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:43.129819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.721747+0000","last_change":"2023-10-06T05:24:05.721747+0000","last_active":"2023-10-06T05:24:05.721747+0000","last_peered":"2023-10-06T05:24:05.721747+0000","last_clean":"2023-10-06T05:24:05.721747+0000","last_became_active":"2023-10-06T05:24:05.720794+0000","last_became_peered":"2023-10-06T05:24:05.720794+0000","last_unstale":"2023-10-06T05:24:05.721747+0000","last_undegraded":"2023-10-06T05:24:05.721747+0000","last_fullsized":"2023-10-06T05:24:05.721747+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:10.314272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.708929+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:22.276001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.719260+0000","last_change":"2023-10-06T05:24:05.719260+0000","last_active":"2023-10-06T05:24:05.719260+0000","last_peered":"2023-10-06T05:24:05.719260+0000","last_clean":"2023-10-06T05:24:05.719260+0000","last_became_active":"2023-10-06T05:24:05.718959+0000","last_became_peered":"2023-10-06T05:24:05.718959+0000","last_unstale":"2023-10-06T05:24:05.719260+0000","last_undegraded":"2023-10-06T05:24:05.719260+0000","last_fullsized":"2023-10-06T05:24:05.719260+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:14.517424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.708843+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:00.462676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.724542+0000","last_change":"2023-10-06T05:24:05.724542+0000","last_active":"2023-10-06T05:24:05.724542+0000","last_peered":"2023-10-06T05:24:05.724542+0000","last_clean":"2023-10-06T05:24:05.724542+0000","last_became_active":"2023-10-06T05:24:05.724236+0000","last_became_peered":"2023-10-06T05:24:05.724236+0000","last_unstale":"2023-10-06T05:24:05.724542+0000","last_undegraded":"2023-10-06T05:24:05.724542+0000","last_fullsized":"2023-10-06T05:24:05.724542+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:07.454659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.713560+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:28.288794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.721144+0000","last_change":"2023-10-06T05:24:05.721144+0000","last_active":"2023-10-06T05:24:05.721144+0000","last_peered":"2023-10-06T05:24:05.721144+0000","last_clean":"2023-10-06T05:24:05.721144+0000","last_became_active":"2023-10-06T05:24:05.720848+0000","last_became_peered":"2023-10-06T05:24:05.720848+0000","last_unstale":"2023-10-06T05:24:05.721144+0000","last_undegraded":"2023-10-06T05:24:05.721144+0000","last_fullsized":"2023-10-06T05:24:05.721144+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:53.556572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710523+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:08.701111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.728217+0000","last_change":"2023-10-06T05:24:05.728217+0000","last_active":"2023-10-06T05:24:05.728217+0000","last_peered":"2023-10-06T05:24:05.728217+0000","last_clean":"2023-10-06T05:24:05.728217+0000","last_became_active":"2023-10-06T05:24:05.727916+0000","last_became_peered":"2023-10-06T05:24:05.727916+0000","last_unstale":"2023-10-06T05:24:05.728217+0000","last_undegraded":"2023-10-06T05:24:05.728217+0000","last_fullsized":"2023-10-06T05:24:05.728217+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:15.522770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.708699+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:51.773348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.729185+0000","last_change":"2023-10-06T05:24:05.729185+0000","last_active":"2023-10-06T05:24:05.729185+0000","last_peered":"2023-10-06T05:24:05.729185+0000","last_clean":"2023-10-06T05:24:05.729185+0000","last_became_active":"2023-10-06T05:24:05.728912+0000","last_became_peered":"2023-10-06T05:24:05.728912+0000","last_unstale":"2023-10-06T05:24:05.729185+0000","last_undegraded":"2023-10-06T05:24:05.729185+0000","last_fullsized":"2023-10-06T05:24:05.729185+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:21.265558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.709690+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:37.680967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.720003+0000","last_change":"2023-10-06T05:24:05.720003+0000","last_active":"2023-10-06T05:24:05.720003+0000","last_peered":"2023-10-06T05:24:05.720003+0000","last_clean":"2023-10-06T05:24:05.720003+0000","last_became_active":"2023-10-06T05:24:05.719744+0000","last_became_peered":"2023-10-06T05:24:05.719744+0000","last_unstale":"2023-10-06T05:24:05.720003+0000","last_undegraded":"2023-10-06T05:24:05.720003+0000","last_fullsized":"2023-10-06T05:24:05.720003+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:26.823343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.708698+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:27.986026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.719375+0000","last_change":"2023-10-06T05:24:05.719375+0000","last_active":"2023-10-06T05:24:05.719375+0000","last_peered":"2023-10-06T05:24:05.719375+0000","last_clean":"2023-10-06T05:24:05.719375+0000","last_became_active":"2023-10-06T05:24:05.719098+0000","last_became_peered":"2023-10-06T05:24:05.719098+0000","last_unstale":"2023-10-06T05:24:05.719375+0000","last_undegraded":"2023-10-06T05:24:05.719375+0000","last_fullsized":"2023-10-06T05:24:05.719375+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:03.196093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710298+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:21.753322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.726797+0000","last_change":"2023-10-06T05:24:05.726797+0000","last_active":"2023-10-06T05:24:05.726797+0000","last_peered":"2023-10-06T05:24:05.726797+0000","last_clean":"2023-10-06T05:24:05.726797+0000","last_became_active":"2023-10-06T05:24:05.726504+0000","last_became_peered":"2023-10-06T05:24:05.726504+0000","last_unstale":"2023-10-06T05:24:05.726797+0000","last_undegraded":"2023-10-06T05:24:05.726797+0000","last_fullsized":"2023-10-06T05:24:05.726797+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:53.556572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.706940+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:00.462676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.729691+0000","last_change":"2023-10-06T05:24:05.729691+0000","last_active":"2023-10-06T05:24:05.729691+0000","last_peered":"2023-10-06T05:24:05.729691+0000","last_clean":"2023-10-06T05:24:05.729691+0000","last_became_active":"2023-10-06T05:24:05.729129+0000","last_became_peered":"2023-10-06T05:24:05.729129+0000","last_unstale":"2023-10-06T05:24:05.729691+0000","last_undegraded":"2023-10-06T05:24:05.729691+0000","last_fullsized":"2023-10-06T05:24:05.729691+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:22.017322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.711643+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:28.083142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.725667+0000","last_change":"2023-10-06T05:24:05.725667+0000","last_active":"2023-10-06T05:24:05.725667+0000","last_peered":"2023-10-06T05:24:05.725667+0000","last_clean":"2023-10-06T05:24:05.725667+0000","last_became_active":"2023-10-06T05:24:05.725396+0000","last_became_peered":"2023-10-06T05:24:05.725396+0000","last_unstale":"2023-10-06T05:24:05.725667+0000","last_undegraded":"2023-10-06T05:24:05.725667+0000","last_fullsized":"2023-10-06T05:24:05.725667+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:13.371969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.713066+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:26.599528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.728225+0000","last_change":"2023-10-06T05:24:05.728225+0000","last_active":"2023-10-06T05:24:05.728225+0000","last_peered":"2023-10-06T05:24:05.728225+0000","last_clean":"2023-10-06T05:24:05.728225+0000","last_became_active":"2023-10-06T05:24:05.727932+0000","last_became_peered":"2023-10-06T05:24:05.727932+0000","last_unstale":"2023-10-06T05:24:05.728225+0000","last_undegraded":"2023-10-06T05:24:05.728225+0000","last_fullsized":"2023-10-06T05:24:05.728225+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:29.277714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.706871+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:04.206537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.727450+0000","last_change":"2023-10-06T05:24:05.727450+0000","last_active":"2023-10-06T05:24:05.727450+0000","last_peered":"2023-10-06T05:24:05.727450+0000","last_clean":"2023-10-06T05:24:05.727450+0000","last_became_active":"2023-10-06T05:24:05.727184+0000","last_became_peered":"2023-10-06T05:24:05.727184+0000","last_unstale":"2023-10-06T05:24:05.727450+0000","last_undegraded":"2023-10-06T05:24:05.727450+0000","last_fullsized":"2023-10-06T05:24:05.727450+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:59.507404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.712434+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:55.900781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.725732+0000","last_change":"2023-10-06T05:24:05.725732+0000","last_active":"2023-10-06T05:24:05.725732+0000","last_peered":"2023-10-06T05:24:05.725732+0000","last_clean":"2023-10-06T05:24:05.725732+0000","last_became_active":"2023-10-06T05:24:05.725468+0000","last_became_peered":"2023-10-06T05:24:05.725468+0000","last_unstale":"2023-10-06T05:24:05.725732+0000","last_undegraded":"2023-10-06T05:24:05.725732+0000","last_fullsized":"2023-10-06T05:24:05.725732+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:05.730446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.712308+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:50.670475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.728985+0000","last_change":"2023-10-06T05:24:05.728985+0000","last_active":"2023-10-06T05:24:05.728985+0000","last_peered":"2023-10-06T05:24:05.728985+0000","last_clean":"2023-10-06T05:24:05.728985+0000","last_became_active":"2023-10-06T05:24:05.728637+0000","last_became_peered":"2023-10-06T05:24:05.728637+0000","last_unstale":"2023-10-06T05:24:05.728985+0000","last_undegraded":"2023-10-06T05:24:05.728985+0000","last_fullsized":"2023-10-06T05:24:05.728985+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:29.046559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.709323+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:07.068167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.720333+0000","last_change":"2023-10-06T05:24:05.720333+0000","last_active":"2023-10-06T05:24:05.720333+0000","last_peered":"2023-10-06T05:24:05.720333+0000","last_clean":"2023-10-06T05:24:05.720333+0000","last_became_active":"2023-10-06T05:24:05.719660+0000","last_became_peered":"2023-10-06T05:24:05.719660+0000","last_unstale":"2023-10-06T05:24:05.720333+0000","last_undegraded":"2023-10-06T05:24:05.720333+0000","last_fullsized":"2023-10-06T05:24:05.720333+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:36.582559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.716085+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:45.374302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.724366+0000","last_change":"2023-10-06T05:24:05.724366+0000","last_active":"2023-10-06T05:24:05.724366+0000","last_peered":"2023-10-06T05:24:05.724366+0000","last_clean":"2023-10-06T05:24:05.724366+0000","last_became_active":"2023-10-06T05:24:05.724056+0000","last_became_peered":"2023-10-06T05:24:05.724056+0000","last_unstale":"2023-10-06T05:24:05.724366+0000","last_undegraded":"2023-10-06T05:24:05.724366+0000","last_fullsized":"2023-10-06T05:24:05.724366+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:59.507404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.712970+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:20.258322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.714026+0000","last_change":"2023-10-06T05:24:05.714026+0000","last_active":"2023-10-06T05:24:05.714026+0000","last_peered":"2023-10-06T05:24:05.714026+0000","last_clean":"2023-10-06T05:24:05.714026+0000","last_became_active":"2023-10-06T05:24:05.713754+0000","last_became_peered":"2023-10-06T05:24:05.713754+0000","last_unstale":"2023-10-06T05:24:05.714026+0000","last_undegraded":"2023-10-06T05:24:05.714026+0000","last_fullsized":"2023-10-06T05:24:05.714026+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:39.135386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-06T05:24:04.701736+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:24:04.701736+0000","last_peered":"2023-10-06T05:24:04.701736+0000","last_clean":"2023-10-06T05:24:04.701736+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:24:04.701736+0000","last_undegraded":"2023-10-06T05:24:04.701736+0000","last_fullsized":"2023-10-06T05:24:04.701736+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710241+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:20.258322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.729545+0000","last_change":"2023-10-06T05:24:05.729545+0000","last_active":"2023-10-06T05:24:05.729545+0000","last_peered":"2023-10-06T05:24:05.729545+0000","last_clean":"2023-10-06T05:24:05.729545+0000","last_became_active":"2023-10-06T05:24:05.729229+0000","last_became_peered":"2023-10-06T05:24:05.729229+0000","last_unstale":"2023-10-06T05:24:05.729545+0000","last_undegraded":"2023-10-06T05:24:05.729545+0000","last_fullsized":"2023-10-06T05:24:05.729545+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:28.303622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-06T05:24:04.700654+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:24:04.700654+0000","last_peered":"2023-10-06T05:24:04.700654+0000","last_clean":"2023-10-06T05:24:04.700654+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:24:04.700654+0000","last_undegraded":"2023-10-06T05:24:04.700654+0000","last_fullsized":"2023-10-06T05:24:04.700654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710120+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:42.691650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.713794+0000","last_change":"2023-10-06T05:24:05.713794+0000","last_active":"2023-10-06T05:24:05.713794+0000","last_peered":"2023-10-06T05:24:05.713794+0000","last_clean":"2023-10-06T05:24:05.713794+0000","last_became_active":"2023-10-06T05:24:05.713539+0000","last_became_peered":"2023-10-06T05:24:05.713539+0000","last_unstale":"2023-10-06T05:24:05.713794+0000","last_undegraded":"2023-10-06T05:24:05.713794+0000","last_fullsized":"2023-10-06T05:24:05.713794+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:47.207042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-06T05:24:04.697829+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:24:04.697829+0000","last_peered":"2023-10-06T05:24:04.697829+0000","last_clean":"2023-10-06T05:24:04.697829+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:24:04.697829+0000","last_undegraded":"2023-10-06T05:24:04.697829+0000","last_fullsized":"2023-10-06T05:24:04.697829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710401+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:37:07.068167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.727464+0000","last_change":"2023-10-06T05:24:05.727464+0000","last_active":"2023-10-06T05:24:05.727464+0000","last_peered":"2023-10-06T05:24:05.727464+0000","last_clean":"2023-10-06T05:24:05.727464+0000","last_became_active":"2023-10-06T05:24:05.727158+0000","last_became_peered":"2023-10-06T05:24:05.727158+0000","last_unstale":"2023-10-06T05:24:05.727464+0000","last_undegraded":"2023-10-06T05:24:05.727464+0000","last_fullsized":"2023-10-06T05:24:05.727464+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:59.452232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":94,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-06T05:24:04.701594+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:24:04.701594+0000","last_peered":"2023-10-06T05:24:04.701594+0000","last_clean":"2023-10-06T05:24:04.701594+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:24:04.701594+0000","last_undegraded":"2023-10-06T05:24:04.701594+0000","last_fullsized":"2023-10-06T05:24:04.701594+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.707891+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:13.920439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.724587+0000","last_change":"2023-10-06T05:24:05.724587+0000","last_active":"2023-10-06T05:24:05.724587+0000","last_peered":"2023-10-06T05:24:05.724587+0000","last_clean":"2023-10-06T05:24:05.724587+0000","last_became_active":"2023-10-06T05:24:05.724331+0000","last_became_peered":"2023-10-06T05:24:05.724331+0000","last_unstale":"2023-10-06T05:24:05.724587+0000","last_undegraded":"2023-10-06T05:24:05.724587+0000","last_fullsized":"2023-10-06T05:24:05.724587+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:44.363859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-06T05:24:04.701503+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:24:04.701503+0000","last_peered":"2023-10-06T05:24:04.701503+0000","last_clean":"2023-10-06T05:24:04.701503+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:24:04.701503+0000","last_undegraded":"2023-10-06T05:24:04.701503+0000","last_fullsized":"2023-10-06T05:24:04.701503+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.715437+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:37.593003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.725681+0000","last_change":"2023-10-06T05:24:05.725681+0000","last_active":"2023-10-06T05:24:05.725681+0000","last_peered":"2023-10-06T05:24:05.725681+0000","last_clean":"2023-10-06T05:24:05.725681+0000","last_became_active":"2023-10-06T05:24:05.725320+0000","last_became_peered":"2023-10-06T05:24:05.725320+0000","last_unstale":"2023-10-06T05:24:05.725681+0000","last_undegraded":"2023-10-06T05:24:05.725681+0000","last_fullsized":"2023-10-06T05:24:05.725681+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:55.075685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":283,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-06T05:24:04.701490+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:24:04.701490+0000","last_peered":"2023-10-06T05:24:04.701490+0000","last_clean":"2023-10-06T05:24:04.701490+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:24:04.701490+0000","last_undegraded":"2023-10-06T05:24:04.701490+0000","last_fullsized":"2023-10-06T05:24:04.701490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":105,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-06T05:24:04.697248+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:24:04.697248+0000","last_peered":"2023-10-06T05:24:04.697248+0000","last_clean":"2023-10-06T05:24:04.697248+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:24:04.697248+0000","last_undegraded":"2023-10-06T05:24:04.697248+0000","last_fullsized":"2023-10-06T05:24:04.697248+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.710745+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:06.740890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.722602+0000","last_change":"2023-10-06T05:24:05.722602+0000","last_active":"2023-10-06T05:24:05.722602+0000","last_peered":"2023-10-06T05:24:05.722602+0000","last_clean":"2023-10-06T05:24:05.722602+0000","last_became_active":"2023-10-06T05:24:05.722289+0000","last_became_peered":"2023-10-06T05:24:05.722289+0000","last_unstale":"2023-10-06T05:24:05.722602+0000","last_undegraded":"2023-10-06T05:24:05.722602+0000","last_fullsized":"2023-10-06T05:24:05.722602+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:58.972597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-06T05:24:04.699471+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:24:04.699471+0000","last_peered":"2023-10-06T05:24:04.699471+0000","last_clean":"2023-10-06T05:24:04.699471+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:24:04.699471+0000","last_undegraded":"2023-10-06T05:24:04.699471+0000","last_fullsized":"2023-10-06T05:24:04.699471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:24:05.693085+0000","last_change":"2023-10-06T05:24:05.706101+0000","last_active":"2023-10-06T05:24:05.693085+0000","last_peered":"2023-10-06T05:24:05.693085+0000","last_clean":"2023-10-06T05:24:05.693085+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:24:05.693085+0000","last_undegraded":"2023-10-06T05:24:05.693085+0000","last_fullsized":"2023-10-06T05:24:05.693085+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:56.086128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":8,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-06T05:24:05.725825+0000","last_change":"2023-10-06T05:24:05.725825+0000","last_active":"2023-10-06T05:24:05.725825+0000","last_peered":"2023-10-06T05:24:05.725825+0000","last_clean":"2023-10-06T05:24:05.725825+0000","last_became_active":"2023-10-06T05:24:05.725475+0000","last_became_peered":"2023-10-06T05:24:05.725475+0000","last_unstale":"2023-10-06T05:24:05.725825+0000","last_undegraded":"2023-10-06T05:24:05.725825+0000","last_fullsized":"2023-10-06T05:24:05.725825+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:48.755016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-06T05:24:04.704518+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:24:04.704518+0000","last_peered":"2023-10-06T05:24:04.704518+0000","last_clean":"2023-10-06T05:24:04.704518+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:24:04.704518+0000","last_undegraded":"2023-10-06T05:24:04.704518+0000","last_fullsized":"2023-10-06T05:24:04.704518+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303496,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068344,"statfs":{"total":96636764160,"available":96325984256,"internally_reserved":0,"allocated":2162688,"data_stored":1317742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.38700000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.57599999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.60899999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.87,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67900000000000005,"15min":0.67800000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70399999999999996,"15min":0.74299999999999999},"min":{"1min":0.46899999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":1.056,"15min":1.056},"last":0.68400000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.55200000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99099999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64100000000000001,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":1.0249999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.77000000000000002},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0629999999999999,"5min":1.27,"15min":1.27},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68999999999999995,"15min":0.73299999999999998},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":0.97899999999999998,"15min":0.999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.74199999999999999,"15min":0.81599999999999995},"min":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.98399999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78900000000000003,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.1040000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79800000000000004,"15min":0.84199999999999997},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.441},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.751}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302916,"kb_used_data":1532,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068924,"statfs":{"total":96636764160,"available":96326578176,"internally_reserved":0,"allocated":1568768,"data_stored":727374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66800000000000004,"15min":0.68200000000000005},"min":{"1min":0.46600000000000003,"5min":0.42999999999999999,"15min":0.32300000000000001},"max":{"1min":0.84399999999999997,"5min":1.2629999999999999,"15min":1.8500000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.38900000000000001},"max":{"1min":0.89500000000000002,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.90500000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70499999999999996,"15min":0.71299999999999997},"min":{"1min":0.50700000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":1.135,"15min":1.9279999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":1.073,"15min":1.4239999999999999},"last":0.998}]},{"osd":2,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94999999999999996,"5min":1.369,"15min":1.369},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.75900000000000001},"min":{"1min":0.45700000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.91200000000000003,"5min":1.208,"15min":1.891},"last":0.81000000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72499999999999998,"15min":0.748},"min":{"1min":0.47799999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.86099999999999999,"5min":1.018,"15min":1.347},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.53500000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":1},"last":0.97699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.77600000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.998,"5min":1.1140000000000001,"15min":1.9670000000000001},"last":0.498},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76900000000000002,"15min":0.81899999999999995},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.012,"5min":1.278,"15min":1.4510000000000001},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.415},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73899999999999999,"15min":0.81899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":2},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.753,"15min":0.80700000000000005},"min":{"1min":0.57299999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.748},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.751,"15min":0.77800000000000002},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.46},"last":0.85499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303496,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068344,"statfs":{"total":96636764160,"available":96325984256,"internally_reserved":0,"allocated":2162688,"data_stored":1317742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.438},"min":{"1min":0.32900000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57799999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.498},{"interface":"front","average":{"1min":0.442,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.59599999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54700000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.33900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71399999999999997,"5min":0.77000000000000002,"15min":0.83799999999999997},"last":0.621},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.504,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.35399999999999998,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66000000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76500000000000001,"15min":0.78800000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.79900000000000004,"15min":0.79200000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.3169999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.753,"15min":0.746},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.143},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.2809999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.749,"15min":0.747},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.387}]},{"osd":7,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.80700000000000005,"15min":0.81399999999999995},"min":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.50600000000000001},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":1.244},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.59099999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.103},"last":1.1000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302916,"kb_used_data":1532,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068924,"statfs":{"total":96636764160,"available":96326578176,"internally_reserved":0,"allocated":1568768,"data_stored":727374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.434,"15min":0.42999999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56899999999999995,"5min":0.71499999999999997,"15min":0.84499999999999997},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.48399999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60599999999999998,"5min":0.78900000000000003,"15min":1.042},"last":0.27300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.497,"15min":0.48899999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66200000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52400000000000002,"15min":0.51200000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65800000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.311}]},{"osd":3,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52000000000000002,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.443},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.79700000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.497}]},{"osd":4,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0880000000000001,"5min":1.242,"15min":1.242},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.748,"15min":0.77200000000000002},"min":{"1min":0.54200000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.198,"15min":1.198},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76300000000000001,"15min":0.76400000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96499999999999997,"5min":1.26,"15min":1.26},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1299999999999999,"5min":1.163,"15min":1.163},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.999,"5min":1.323,"15min":1.323},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92300000000000004,"5min":1.1799999999999999,"15min":1.3100000000000001},"last":0.47199999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.82999999999999996},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.52400000000000002},"max":{"1min":0.999,"5min":1.2090000000000001,"15min":1.264},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78700000000000003,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":0.52800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302916,"kb_used_data":1532,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068924,"statfs":{"total":96636764160,"available":96326578176,"internally_reserved":0,"allocated":1568768,"data_stored":727374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.43099999999999999,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.627,"5min":0.77400000000000002,"15min":0.77800000000000002},"last":0.376},{"interface":"front","average":{"1min":0.496,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68500000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.27400000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70899999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.498,"15min":0.502},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.31900000000000001},"max":{"1min":0.66200000000000003,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.503}]},{"osd":3,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.82799999999999996,"15min":0.93600000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.45700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73599999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.75800000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98499999999999999,"5min":1.022,"15min":1.1299999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.79400000000000004},"min":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96999999999999997,"5min":1.085,"15min":1.1699999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73699999999999999,"15min":0.78200000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.026,"5min":1.042,"15min":1.099},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.751,"15min":0.80000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.002,"5min":1.0649999999999999,"15min":1.163},"last":0.86299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77400000000000002,"15min":0.82999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94799999999999995,"5min":1.0960000000000001,"15min":1.21},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.998,"5min":1.022,"15min":1.042},"last":0.751}]},{"osd":7,"last update":"Fri Oct 6 05:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.77600000000000002,"15min":0.80600000000000005},"min":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.85899999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0629999999999999,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":0.73799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302904,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068936,"statfs":{"total":96636764160,"available":96326590464,"internally_reserved":0,"allocated":1556480,"data_stored":720730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.58899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47099999999999997,"15min":0.46800000000000003},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.499},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60399999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70299999999999996,"5min":0.999,"15min":0.999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.63200000000000001,"5min":0.98999999999999999,"15min":1.014},"last":0.45200000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0349999999999999,"5min":1.161,"15min":1.2669999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.004,"5min":1.1830000000000001,"15min":1.3540000000000001},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.754,"15min":0.76400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.093,"5min":1.147,"15min":1.204},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.208,"5min":1.284,"15min":1.329},"last":0.71899999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77900000000000003,"15min":0.80800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92700000000000005,"5min":1.0880000000000001,"15min":1.484},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0169999999999999,"5min":1.2809999999999999,"15min":1.629},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81000000000000005,"15min":0.84499999999999997},"min":{"1min":0.58299999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1379999999999999,"5min":1.236,"15min":1.5620000000000001},"last":0.75},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.624,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":0.96399999999999997,"5min":1.194,"15min":1.325},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302916,"kb_used_data":1532,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068924,"statfs":{"total":96636764160,"available":96326578176,"internally_reserved":0,"allocated":1568768,"data_stored":727374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90700000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68100000000000005,"15min":0.77700000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89100000000000001,"5min":0.95999999999999996,"15min":2.77},"last":0.61199999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.73499999999999999},"min":{"1min":0.52100000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.6419999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68999999999999995,"15min":0.71999999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.036,"5min":1.1100000000000001,"15min":1.1899999999999999},"last":0.497}]},{"osd":2,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.49199999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71299999999999997,"15min":0.81200000000000006},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95599999999999996,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.65400000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.52900000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95899999999999996,"5min":0.999,"15min":1.0629999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.70999999999999996,"15min":1.026},"min":{"1min":0.59799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91200000000000003,"5min":0.97699999999999998,"15min":7.742},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68100000000000005,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91900000000000004,"5min":0.96499999999999997,"15min":1.079},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70599999999999996,"15min":0.748},"min":{"1min":0.498,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.181},"last":1.03}]},{"osd":6,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.73599999999999999},"min":{"1min":0.505,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.2669999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.747,"15min":0.81499999999999995},"min":{"1min":0.56499999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.171},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.82699999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76500000000000001,"15min":0.85299999999999998},"min":{"1min":0.64800000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.571},"last":0.85999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298820,"kb_used_data":1532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073020,"statfs":{"total":96636764160,"available":96330772480,"internally_reserved":0,"allocated":1568768,"data_stored":727374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65400000000000003,"15min":0.91000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.98199999999999998,"5min":1.109,"15min":9.2780000000000005},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.91300000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.0449999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.93500000000000005},"min":{"1min":0.52500000000000002,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":1.0329999999999999,"5min":1.079,"15min":9.25},"last":0.874},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.98899999999999999},"min":{"1min":0.46500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93200000000000005,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.93799999999999994}]},{"osd":2,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66400000000000003,"15min":0.95999999999999996},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98499999999999999,"5min":1.052,"15min":10.384},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.98699999999999999},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.097,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.97299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":1.0249999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.495},"max":{"1min":0.88200000000000001,"5min":1.76,"15min":10.205},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.55300000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.097,"15min":1.292},"last":0.95799999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68300000000000005,"15min":0.70099999999999996},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70699999999999996,"15min":0.76500000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.008,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70299999999999996,"15min":0.73499999999999999},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90900000000000003,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.80500000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95899999999999996,"5min":1.008,"15min":1.298},"last":1.0289999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.81799999999999995},"min":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0529999999999999,"5min":1.556,"15min":1.556},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.80700000000000005},"min":{"1min":0.498,"5min":0.432,"15min":0.432},"max":{"1min":1.04,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:24:11.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:24:11.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.753+0000 7fc96c7ff700 1 Processor -- start 2023-10-06T05:24:11.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.753+0000 7fc96c7ff700 1 -- start start 2023-10-06T05:24:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.754+0000 7fc96c7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9640c7f30 0x7fc964069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.754+0000 7fc96c7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc9640aee70 0x7fc96413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.754+0000 7fc96c7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9640ca5b0 con 0x7fc9640c7b60 2023-10-06T05:24:11.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.754+0000 7fc96c7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9640af2f0 con 0x7fc9640aee70 2023-10-06T05:24:11.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.754+0000 7fc96c7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96406f740 con 0x7fc9640c7f30 2023-10-06T05:24:11.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.755+0000 7fc96ad9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc9640aee70 0x7fc96413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.755+0000 7fc96ad9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc9640aee70 0x7fc96413f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49384/0 (socket says 172.21.15.31:49384) 2023-10-06T05:24:11.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.755+0000 7fc96ad9c700 1 -- 172.21.15.31:0/2164866488 learned_addr learned my addr 172.21.15.31:0/2164866488 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:11.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.755+0000 7fc969d9a700 1 --2- 172.21.15.31:0/2164866488 >> v2:172.21.15.62:3300/0 conn(0x7fc9640c7f30 0x7fc964069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.755+0000 7fc96ad9c700 1 -- 172.21.15.31:0/2164866488 >> v2:172.21.15.62:3300/0 conn(0x7fc9640c7f30 msgr2=0x7fc964069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.755+0000 7fc96ad9c700 1 --2- 172.21.15.31:0/2164866488 >> v2:172.21.15.62:3300/0 conn(0x7fc9640c7f30 0x7fc964069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.755+0000 7fc96ad9c700 1 -- 172.21.15.31:0/2164866488 >> v1:172.21.15.62:6789/0 conn(0x7fc9640c7b60 legacy=0x7fc96406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.755+0000 7fc96ad9c700 1 -- 172.21.15.31:0/2164866488 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9640c9c80 con 0x7fc9640aee70 2023-10-06T05:24:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.755+0000 7fc96ad9c700 1 --2- 172.21.15.31:0/2164866488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 0x7fc96413f690 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc96406c990 tx=0x7fc95c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39fc5934e4caeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc969599700 1 -- 172.21.15.31:0/2164866488 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00e070 con 0x7fc9640aee70 2023-10-06T05:24:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc969599700 1 -- 172.21.15.31:0/2164866488 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95c00ba10 con 0x7fc9640aee70 2023-10-06T05:24:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc969599700 1 -- 172.21.15.31:0/2164866488 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0092e0 con 0x7fc9640aee70 2023-10-06T05:24:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc96c7ff700 1 -- 172.21.15.31:0/2164866488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 msgr2=0x7fc96413f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc96c7ff700 1 --2- 172.21.15.31:0/2164866488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 0x7fc96413f690 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc96406c990 tx=0x7fc95c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc96c7ff700 1 -- 172.21.15.31:0/2164866488 shutdown_connections 2023-10-06T05:24:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc96c7ff700 1 --2- 172.21.15.31:0/2164866488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 0x7fc96413f690 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc96c7ff700 1 --2- 172.21.15.31:0/2164866488 >> v2:172.21.15.62:3300/0 conn(0x7fc9640c7f30 0x7fc964069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc96c7ff700 1 -- 172.21.15.31:0/2164866488 >> 172.21.15.31:0/2164866488 conn(0x7fc9640cc0b0 msgr2=0x7fc9640cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc96c7ff700 1 -- 172.21.15.31:0/2164866488 shutdown_connections 2023-10-06T05:24:11.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.756+0000 7fc96c7ff700 1 -- 172.21.15.31:0/2164866488 wait complete. 2023-10-06T05:24:11.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.758+0000 7fc96c7ff700 1 Processor -- start 2023-10-06T05:24:11.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.759+0000 7fc96c7ff700 1 -- start start 2023-10-06T05:24:11.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.760+0000 7fc96c7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 0x7fc964155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.760+0000 7fc96c7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9640c7b60 0x7fc9640ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.760+0000 7fc96c7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9640c7f30 0x7fc9640adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.760+0000 7fc96c7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9640c4ee0 con 0x7fc9640aee70 2023-10-06T05:24:11.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.760+0000 7fc96c7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9641563e0 con 0x7fc9640c7f30 2023-10-06T05:24:11.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.760+0000 7fc96c7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc964169af0 con 0x7fc9640c7b60 2023-10-06T05:24:11.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.760+0000 7fc96a59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 0x7fc964155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.760+0000 7fc96a59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 0x7fc964155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49390/0 (socket says 172.21.15.31:49390) 2023-10-06T05:24:11.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.760+0000 7fc96a59b700 1 -- 172.21.15.31:0/1852873571 learned_addr learned my addr 172.21.15.31:0/1852873571 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc96ad9c700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9640c7f30 0x7fc9640adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc969d9a700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9640c7b60 0x7fc9640ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc96a59b700 1 -- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9640c7b60 msgr2=0x7fc9640ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc96a59b700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9640c7b60 0x7fc9640ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc96a59b700 1 -- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9640c7f30 msgr2=0x7fc9640adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc96a59b700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9640c7f30 0x7fc9640adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc96a59b700 1 -- 172.21.15.31:0/1852873571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c009ae0 con 0x7fc9640aee70 2023-10-06T05:24:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc96a59b700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 0x7fc964155ea0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc96000c5b0 tx=0x7fc9600020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc95a7fc700 1 -- 172.21.15.31:0/1852873571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960011590 con 0x7fc9640aee70 2023-10-06T05:24:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc95a7fc700 1 -- 172.21.15.31:0/1852873571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9600116f0 con 0x7fc9640aee70 2023-10-06T05:24:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc96c7ff700 1 -- 172.21.15.31:0/1852873571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc964169dd0 con 0x7fc9640aee70 2023-10-06T05:24:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc95a7fc700 1 -- 172.21.15.31:0/1852873571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960011a90 con 0x7fc9640aee70 2023-10-06T05:24:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.761+0000 7fc96c7ff700 1 -- 172.21.15.31:0/1852873571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9640c4890 con 0x7fc9640aee70 2023-10-06T05:24:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.762+0000 7fc96c7ff700 1 -- 172.21.15.31:0/1852873571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96404c380 con 0x7fc9640aee70 2023-10-06T05:24:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.763+0000 7fc95a7fc700 1 -- 172.21.15.31:0/1852873571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fc960012420 con 0x7fc9640aee70 2023-10-06T05:24:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.764+0000 7fc95a7fc700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9480674d0 0x7fc948069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.764+0000 7fc95a7fc700 1 -- 172.21.15.31:0/1852873571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc960084bc0 con 0x7fc9640aee70 2023-10-06T05:24:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.765+0000 7fc969d9a700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9480674d0 0x7fc948069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.765+0000 7fc969d9a700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9480674d0 0x7fc948069980 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc964130690 tx=0x7fc954008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:11.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.768+0000 7fc95a7fc700 1 -- 172.21.15.31:0/1852873571 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc960051400 con 0x7fc9640aee70 2023-10-06T05:24:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.967+0000 7fc96c7ff700 1 -- 172.21.15.31:0/1852873571 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9640c9f60 con 0x7fc9480674d0 2023-10-06T05:24:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.978+0000 7fc95a7fc700 1 -- 172.21.15.31:0/1852873571 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311639 (secure 0 0 0) 0x7fc9640c9f60 con 0x7fc9480674d0 2023-10-06T05:24:11.984 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 -- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9480674d0 msgr2=0x7fc948069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9480674d0 0x7fc948069980 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc964130690 tx=0x7fc954008040 comp rx=0 tx=0).stop 2023-10-06T05:24:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 -- 172.21.15.31:0/1852873571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 msgr2=0x7fc964155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 0x7fc964155ea0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc96000c5b0 tx=0x7fc9600020e0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 -- 172.21.15.31:0/1852873571 shutdown_connections 2023-10-06T05:24:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9480674d0 0x7fc948069980 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9640c7f30 0x7fc9640adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9640aee70 0x7fc964155ea0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 --2- 172.21.15.31:0/1852873571 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc9640c7b60 0x7fc9640ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 -- 172.21.15.31:0/1852873571 >> 172.21.15.31:0/1852873571 conn(0x7fc9640cc0b0 msgr2=0x7fc9640b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 -- 172.21.15.31:0/1852873571 shutdown_connections 2023-10-06T05:24:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:11.982+0000 7fc96c7ff700 1 -- 172.21.15.31:0/1852873571 wait complete. 2023-10-06T05:24:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:24:12.001 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2023-10-06T05:24:11.497450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":200,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2424744,"kb_used_data":13672,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752549976,"statfs":{"total":773094113280,"available":770611175424,"internally_reserved":0,"allocated":14000128,"data_stored":7114844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001212"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.738627+0000","last_change":"2023-10-06T05:24:06.733075+0000","last_active":"2023-10-06T05:24:06.738627+0000","last_peered":"2023-10-06T05:24:06.738627+0000","last_clean":"2023-10-06T05:24:06.738627+0000","last_became_active":"2023-10-06T05:24:06.732745+0000","last_became_peered":"2023-10-06T05:24:06.732745+0000","last_unstale":"2023-10-06T05:24:06.738627+0000","last_undegraded":"2023-10-06T05:24:06.738627+0000","last_fullsized":"2023-10-06T05:24:06.738627+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:12.189447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.722854+0000","last_change":"2023-10-06T05:24:06.717028+0000","last_active":"2023-10-06T05:24:06.722854+0000","last_peered":"2023-10-06T05:24:06.722854+0000","last_clean":"2023-10-06T05:24:06.722854+0000","last_became_active":"2023-10-06T05:24:06.716742+0000","last_became_peered":"2023-10-06T05:24:06.716742+0000","last_unstale":"2023-10-06T05:24:06.722854+0000","last_undegraded":"2023-10-06T05:24:06.722854+0000","last_fullsized":"2023-10-06T05:24:06.722854+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:08.701111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.732089+0000","last_change":"2023-10-06T05:24:06.730115+0000","last_active":"2023-10-06T05:24:06.732089+0000","last_peered":"2023-10-06T05:24:06.732089+0000","last_clean":"2023-10-06T05:24:06.732089+0000","last_became_active":"2023-10-06T05:24:06.729769+0000","last_became_peered":"2023-10-06T05:24:06.729769+0000","last_unstale":"2023-10-06T05:24:06.732089+0000","last_undegraded":"2023-10-06T05:24:06.732089+0000","last_fullsized":"2023-10-06T05:24:06.732089+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:02.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733452+0000","last_change":"2023-10-06T05:24:06.715083+0000","last_active":"2023-10-06T05:24:06.733452+0000","last_peered":"2023-10-06T05:24:06.733452+0000","last_clean":"2023-10-06T05:24:06.733452+0000","last_became_active":"2023-10-06T05:24:06.714752+0000","last_became_peered":"2023-10-06T05:24:06.714752+0000","last_unstale":"2023-10-06T05:24:06.733452+0000","last_undegraded":"2023-10-06T05:24:06.733452+0000","last_fullsized":"2023-10-06T05:24:06.733452+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:46.830863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.729412+0000","last_change":"2023-10-06T05:24:06.727096+0000","last_active":"2023-10-06T05:24:06.729412+0000","last_peered":"2023-10-06T05:24:06.729412+0000","last_clean":"2023-10-06T05:24:06.729412+0000","last_became_active":"2023-10-06T05:24:06.726797+0000","last_became_peered":"2023-10-06T05:24:06.726797+0000","last_unstale":"2023-10-06T05:24:06.729412+0000","last_undegraded":"2023-10-06T05:24:06.729412+0000","last_fullsized":"2023-10-06T05:24:06.729412+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:28.882751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.729395+0000","last_change":"2023-10-06T05:24:06.727158+0000","last_active":"2023-10-06T05:24:06.729395+0000","last_peered":"2023-10-06T05:24:06.729395+0000","last_clean":"2023-10-06T05:24:06.729395+0000","last_became_active":"2023-10-06T05:24:06.726840+0000","last_became_peered":"2023-10-06T05:24:06.726840+0000","last_unstale":"2023-10-06T05:24:06.729395+0000","last_undegraded":"2023-10-06T05:24:06.729395+0000","last_fullsized":"2023-10-06T05:24:06.729395+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:29.843001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.734780+0000","last_change":"2023-10-06T05:24:06.730513+0000","last_active":"2023-10-06T05:24:06.734780+0000","last_peered":"2023-10-06T05:24:06.734780+0000","last_clean":"2023-10-06T05:24:06.734780+0000","last_became_active":"2023-10-06T05:24:06.730221+0000","last_became_peered":"2023-10-06T05:24:06.730221+0000","last_unstale":"2023-10-06T05:24:06.734780+0000","last_undegraded":"2023-10-06T05:24:06.734780+0000","last_fullsized":"2023-10-06T05:24:06.734780+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:50.048745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733406+0000","last_change":"2023-10-06T05:24:06.726860+0000","last_active":"2023-10-06T05:24:06.733406+0000","last_peered":"2023-10-06T05:24:06.733406+0000","last_clean":"2023-10-06T05:24:06.733406+0000","last_became_active":"2023-10-06T05:24:06.726196+0000","last_became_peered":"2023-10-06T05:24:06.726196+0000","last_unstale":"2023-10-06T05:24:06.733406+0000","last_undegraded":"2023-10-06T05:24:06.733406+0000","last_fullsized":"2023-10-06T05:24:06.733406+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:43.129819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733307+0000","last_change":"2023-10-06T05:24:06.731121+0000","last_active":"2023-10-06T05:24:06.733307+0000","last_peered":"2023-10-06T05:24:06.733307+0000","last_clean":"2023-10-06T05:24:06.733307+0000","last_became_active":"2023-10-06T05:24:06.730836+0000","last_became_peered":"2023-10-06T05:24:06.730836+0000","last_unstale":"2023-10-06T05:24:06.733307+0000","last_undegraded":"2023-10-06T05:24:06.733307+0000","last_fullsized":"2023-10-06T05:24:06.733307+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:01.789601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.729157+0000","last_change":"2023-10-06T05:24:06.724573+0000","last_active":"2023-10-06T05:24:06.729157+0000","last_peered":"2023-10-06T05:24:06.729157+0000","last_clean":"2023-10-06T05:24:06.729157+0000","last_became_active":"2023-10-06T05:24:06.724233+0000","last_became_peered":"2023-10-06T05:24:06.724233+0000","last_unstale":"2023-10-06T05:24:06.729157+0000","last_undegraded":"2023-10-06T05:24:06.729157+0000","last_fullsized":"2023-10-06T05:24:06.729157+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:57.137421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733142+0000","last_change":"2023-10-06T05:24:06.728148+0000","last_active":"2023-10-06T05:24:06.733142+0000","last_peered":"2023-10-06T05:24:06.733142+0000","last_clean":"2023-10-06T05:24:06.733142+0000","last_became_active":"2023-10-06T05:24:06.727841+0000","last_became_peered":"2023-10-06T05:24:06.727841+0000","last_unstale":"2023-10-06T05:24:06.733142+0000","last_undegraded":"2023-10-06T05:24:06.733142+0000","last_fullsized":"2023-10-06T05:24:06.733142+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:55.194895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733474+0000","last_change":"2023-10-06T05:24:06.722060+0000","last_active":"2023-10-06T05:24:06.733474+0000","last_peered":"2023-10-06T05:24:06.733474+0000","last_clean":"2023-10-06T05:24:06.733474+0000","last_became_active":"2023-10-06T05:24:06.721177+0000","last_became_peered":"2023-10-06T05:24:06.721177+0000","last_unstale":"2023-10-06T05:24:06.733474+0000","last_undegraded":"2023-10-06T05:24:06.733474+0000","last_fullsized":"2023-10-06T05:24:06.733474+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:56.934578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.724591+0000","last_change":"2023-10-06T05:24:06.721413+0000","last_active":"2023-10-06T05:24:06.724591+0000","last_peered":"2023-10-06T05:24:06.724591+0000","last_clean":"2023-10-06T05:24:06.724591+0000","last_became_active":"2023-10-06T05:24:06.721126+0000","last_became_peered":"2023-10-06T05:24:06.721126+0000","last_unstale":"2023-10-06T05:24:06.724591+0000","last_undegraded":"2023-10-06T05:24:06.724591+0000","last_fullsized":"2023-10-06T05:24:06.724591+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:48.790796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.729573+0000","last_change":"2023-10-06T05:24:06.720576+0000","last_active":"2023-10-06T05:24:06.729573+0000","last_peered":"2023-10-06T05:24:06.729573+0000","last_clean":"2023-10-06T05:24:06.729573+0000","last_became_active":"2023-10-06T05:24:06.720241+0000","last_became_peered":"2023-10-06T05:24:06.720241+0000","last_unstale":"2023-10-06T05:24:06.729573+0000","last_undegraded":"2023-10-06T05:24:06.729573+0000","last_fullsized":"2023-10-06T05:24:06.729573+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:38.867828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.722884+0000","last_change":"2023-10-06T05:24:06.717855+0000","last_active":"2023-10-06T05:24:06.722884+0000","last_peered":"2023-10-06T05:24:06.722884+0000","last_clean":"2023-10-06T05:24:06.722884+0000","last_became_active":"2023-10-06T05:24:06.717558+0000","last_became_peered":"2023-10-06T05:24:06.717558+0000","last_unstale":"2023-10-06T05:24:06.722884+0000","last_undegraded":"2023-10-06T05:24:06.722884+0000","last_fullsized":"2023-10-06T05:24:06.722884+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:28.288794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.730861+0000","last_change":"2023-10-06T05:24:06.728475+0000","last_active":"2023-10-06T05:24:06.730861+0000","last_peered":"2023-10-06T05:24:06.730861+0000","last_clean":"2023-10-06T05:24:06.730861+0000","last_became_active":"2023-10-06T05:24:06.728205+0000","last_became_peered":"2023-10-06T05:24:06.728205+0000","last_unstale":"2023-10-06T05:24:06.730861+0000","last_undegraded":"2023-10-06T05:24:06.730861+0000","last_fullsized":"2023-10-06T05:24:06.730861+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:17.932150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.729535+0000","last_change":"2023-10-06T05:24:06.725033+0000","last_active":"2023-10-06T05:24:06.729535+0000","last_peered":"2023-10-06T05:24:06.729535+0000","last_clean":"2023-10-06T05:24:06.729535+0000","last_became_active":"2023-10-06T05:24:06.724732+0000","last_became_peered":"2023-10-06T05:24:06.724732+0000","last_unstale":"2023-10-06T05:24:06.729535+0000","last_undegraded":"2023-10-06T05:24:06.729535+0000","last_fullsized":"2023-10-06T05:24:06.729535+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:34.060387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733144+0000","last_change":"2023-10-06T05:24:06.728285+0000","last_active":"2023-10-06T05:24:06.733144+0000","last_peered":"2023-10-06T05:24:06.733144+0000","last_clean":"2023-10-06T05:24:06.733144+0000","last_became_active":"2023-10-06T05:24:06.726703+0000","last_became_peered":"2023-10-06T05:24:06.726703+0000","last_unstale":"2023-10-06T05:24:06.733144+0000","last_undegraded":"2023-10-06T05:24:06.733144+0000","last_fullsized":"2023-10-06T05:24:06.733144+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:21:47.211634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.741269+0000","last_change":"2023-10-06T05:24:06.734969+0000","last_active":"2023-10-06T05:24:06.741269+0000","last_peered":"2023-10-06T05:24:06.741269+0000","last_clean":"2023-10-06T05:24:06.741269+0000","last_became_active":"2023-10-06T05:24:06.734500+0000","last_became_peered":"2023-10-06T05:24:06.734500+0000","last_unstale":"2023-10-06T05:24:06.741269+0000","last_undegraded":"2023-10-06T05:24:06.741269+0000","last_fullsized":"2023-10-06T05:24:06.741269+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:48.790796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.729380+0000","last_change":"2023-10-06T05:24:06.723686+0000","last_active":"2023-10-06T05:24:06.729380+0000","last_peered":"2023-10-06T05:24:06.729380+0000","last_clean":"2023-10-06T05:24:06.729380+0000","last_became_active":"2023-10-06T05:24:06.723317+0000","last_became_peered":"2023-10-06T05:24:06.723317+0000","last_unstale":"2023-10-06T05:24:06.729380+0000","last_undegraded":"2023-10-06T05:24:06.729380+0000","last_fullsized":"2023-10-06T05:24:06.729380+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:07.247292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733322+0000","last_change":"2023-10-06T05:24:06.731168+0000","last_active":"2023-10-06T05:24:06.733322+0000","last_peered":"2023-10-06T05:24:06.733322+0000","last_clean":"2023-10-06T05:24:06.733322+0000","last_became_active":"2023-10-06T05:24:06.730850+0000","last_became_peered":"2023-10-06T05:24:06.730850+0000","last_unstale":"2023-10-06T05:24:06.733322+0000","last_undegraded":"2023-10-06T05:24:06.733322+0000","last_fullsized":"2023-10-06T05:24:06.733322+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:04.744072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.738675+0000","last_change":"2023-10-06T05:24:06.733129+0000","last_active":"2023-10-06T05:24:06.738675+0000","last_peered":"2023-10-06T05:24:06.738675+0000","last_clean":"2023-10-06T05:24:06.738675+0000","last_became_active":"2023-10-06T05:24:06.732815+0000","last_became_peered":"2023-10-06T05:24:06.732815+0000","last_unstale":"2023-10-06T05:24:06.738675+0000","last_undegraded":"2023-10-06T05:24:06.738675+0000","last_fullsized":"2023-10-06T05:24:06.738675+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:26:18.258781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.734165+0000","last_change":"2023-10-06T05:24:06.731560+0000","last_active":"2023-10-06T05:24:06.734165+0000","last_peered":"2023-10-06T05:24:06.734165+0000","last_clean":"2023-10-06T05:24:06.734165+0000","last_became_active":"2023-10-06T05:24:06.731268+0000","last_became_peered":"2023-10-06T05:24:06.731268+0000","last_unstale":"2023-10-06T05:24:06.734165+0000","last_undegraded":"2023-10-06T05:24:06.734165+0000","last_fullsized":"2023-10-06T05:24:06.734165+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:22.989870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.716869+0000","last_change":"2023-10-06T05:24:06.713141+0000","last_active":"2023-10-06T05:24:06.716869+0000","last_peered":"2023-10-06T05:24:06.716869+0000","last_clean":"2023-10-06T05:24:06.716869+0000","last_became_active":"2023-10-06T05:24:06.712857+0000","last_became_peered":"2023-10-06T05:24:06.712857+0000","last_unstale":"2023-10-06T05:24:06.716869+0000","last_undegraded":"2023-10-06T05:24:06.716869+0000","last_fullsized":"2023-10-06T05:24:06.716869+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:32.996162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.732936+0000","last_change":"2023-10-06T05:24:06.727256+0000","last_active":"2023-10-06T05:24:06.732936+0000","last_peered":"2023-10-06T05:24:06.732936+0000","last_clean":"2023-10-06T05:24:06.732936+0000","last_became_active":"2023-10-06T05:24:06.726954+0000","last_became_peered":"2023-10-06T05:24:06.726954+0000","last_unstale":"2023-10-06T05:24:06.732936+0000","last_undegraded":"2023-10-06T05:24:06.732936+0000","last_fullsized":"2023-10-06T05:24:06.732936+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:59.774251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.730790+0000","last_change":"2023-10-06T05:24:06.726591+0000","last_active":"2023-10-06T05:24:06.730790+0000","last_peered":"2023-10-06T05:24:06.730790+0000","last_clean":"2023-10-06T05:24:06.730790+0000","last_became_active":"2023-10-06T05:24:06.726323+0000","last_became_peered":"2023-10-06T05:24:06.726323+0000","last_unstale":"2023-10-06T05:24:06.730790+0000","last_undegraded":"2023-10-06T05:24:06.730790+0000","last_fullsized":"2023-10-06T05:24:06.730790+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:09.426593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.731886+0000","last_change":"2023-10-06T05:24:06.730303+0000","last_active":"2023-10-06T05:24:06.731886+0000","last_peered":"2023-10-06T05:24:06.731886+0000","last_clean":"2023-10-06T05:24:06.731886+0000","last_became_active":"2023-10-06T05:24:06.729974+0000","last_became_peered":"2023-10-06T05:24:06.729974+0000","last_unstale":"2023-10-06T05:24:06.731886+0000","last_undegraded":"2023-10-06T05:24:06.731886+0000","last_fullsized":"2023-10-06T05:24:06.731886+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:33.576489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.724269+0000","last_change":"2023-10-06T05:24:06.719730+0000","last_active":"2023-10-06T05:24:06.724269+0000","last_peered":"2023-10-06T05:24:06.724269+0000","last_clean":"2023-10-06T05:24:06.724269+0000","last_became_active":"2023-10-06T05:24:06.719436+0000","last_became_peered":"2023-10-06T05:24:06.719436+0000","last_unstale":"2023-10-06T05:24:06.724269+0000","last_undegraded":"2023-10-06T05:24:06.724269+0000","last_fullsized":"2023-10-06T05:24:06.724269+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:47.464913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733458+0000","last_change":"2023-10-06T05:24:06.725866+0000","last_active":"2023-10-06T05:24:06.733458+0000","last_peered":"2023-10-06T05:24:06.733458+0000","last_clean":"2023-10-06T05:24:06.733458+0000","last_became_active":"2023-10-06T05:24:06.725486+0000","last_became_peered":"2023-10-06T05:24:06.725486+0000","last_unstale":"2023-10-06T05:24:06.733458+0000","last_undegraded":"2023-10-06T05:24:06.733458+0000","last_fullsized":"2023-10-06T05:24:06.733458+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:28.288794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.716536+0000","last_change":"2023-10-06T05:24:06.714800+0000","last_active":"2023-10-06T05:24:06.716536+0000","last_peered":"2023-10-06T05:24:06.716536+0000","last_clean":"2023-10-06T05:24:06.716536+0000","last_became_active":"2023-10-06T05:24:06.714521+0000","last_became_peered":"2023-10-06T05:24:06.714521+0000","last_unstale":"2023-10-06T05:24:06.716536+0000","last_undegraded":"2023-10-06T05:24:06.716536+0000","last_fullsized":"2023-10-06T05:24:06.716536+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:41.971658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.729189+0000","last_change":"2023-10-06T05:24:06.725612+0000","last_active":"2023-10-06T05:24:06.729189+0000","last_peered":"2023-10-06T05:24:06.729189+0000","last_clean":"2023-10-06T05:24:06.729189+0000","last_became_active":"2023-10-06T05:24:06.725325+0000","last_became_peered":"2023-10-06T05:24:06.725325+0000","last_unstale":"2023-10-06T05:24:06.729189+0000","last_undegraded":"2023-10-06T05:24:06.729189+0000","last_fullsized":"2023-10-06T05:24:06.729189+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:07.247292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733525+0000","last_change":"2023-10-06T05:24:06.726981+0000","last_active":"2023-10-06T05:24:06.733525+0000","last_peered":"2023-10-06T05:24:06.733525+0000","last_clean":"2023-10-06T05:24:06.733525+0000","last_became_active":"2023-10-06T05:24:06.726547+0000","last_became_peered":"2023-10-06T05:24:06.726547+0000","last_unstale":"2023-10-06T05:24:06.733525+0000","last_undegraded":"2023-10-06T05:24:06.733525+0000","last_fullsized":"2023-10-06T05:24:06.733525+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:32.207879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.732050+0000","last_change":"2023-10-06T05:24:06.730040+0000","last_active":"2023-10-06T05:24:06.732050+0000","last_peered":"2023-10-06T05:24:06.732050+0000","last_clean":"2023-10-06T05:24:06.732050+0000","last_became_active":"2023-10-06T05:24:06.729729+0000","last_became_peered":"2023-10-06T05:24:06.729729+0000","last_unstale":"2023-10-06T05:24:06.732050+0000","last_undegraded":"2023-10-06T05:24:06.732050+0000","last_fullsized":"2023-10-06T05:24:06.732050+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:09.426593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"126'4","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.120038+0000","last_change":"2023-10-06T05:24:05.713591+0000","last_active":"2023-10-06T05:24:07.120038+0000","last_peered":"2023-10-06T05:24:07.120038+0000","last_clean":"2023-10-06T05:24:07.120038+0000","last_became_active":"2023-10-06T05:24:05.713310+0000","last_became_peered":"2023-10-06T05:24:05.713310+0000","last_unstale":"2023-10-06T05:24:07.120038+0000","last_undegraded":"2023-10-06T05:24:07.120038+0000","last_fullsized":"2023-10-06T05:24:07.120038+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:35.466776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.722814+0000","last_change":"2023-10-06T05:24:06.717787+0000","last_active":"2023-10-06T05:24:06.722814+0000","last_peered":"2023-10-06T05:24:06.722814+0000","last_clean":"2023-10-06T05:24:06.722814+0000","last_became_active":"2023-10-06T05:24:06.717511+0000","last_became_peered":"2023-10-06T05:24:06.717511+0000","last_unstale":"2023-10-06T05:24:06.722814+0000","last_undegraded":"2023-10-06T05:24:06.722814+0000","last_fullsized":"2023-10-06T05:24:06.722814+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:59.774251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"126'1","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.120302+0000","last_change":"2023-10-06T05:24:05.738451+0000","last_active":"2023-10-06T05:24:07.120302+0000","last_peered":"2023-10-06T05:24:07.120302+0000","last_clean":"2023-10-06T05:24:07.120302+0000","last_became_active":"2023-10-06T05:24:05.738139+0000","last_became_peered":"2023-10-06T05:24:05.738139+0000","last_unstale":"2023-10-06T05:24:07.120302+0000","last_undegraded":"2023-10-06T05:24:07.120302+0000","last_fullsized":"2023-10-06T05:24:07.120302+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:49.715080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.734274+0000","last_change":"2023-10-06T05:24:06.732255+0000","last_active":"2023-10-06T05:24:06.734274+0000","last_peered":"2023-10-06T05:24:06.734274+0000","last_clean":"2023-10-06T05:24:06.734274+0000","last_became_active":"2023-10-06T05:24:06.731953+0000","last_became_peered":"2023-10-06T05:24:06.731953+0000","last_unstale":"2023-10-06T05:24:06.734274+0000","last_undegraded":"2023-10-06T05:24:06.734274+0000","last_fullsized":"2023-10-06T05:24:06.734274+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:59.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"126'2","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.120233+0000","last_change":"2023-10-06T05:24:05.728447+0000","last_active":"2023-10-06T05:24:07.120233+0000","last_peered":"2023-10-06T05:24:07.120233+0000","last_clean":"2023-10-06T05:24:07.120233+0000","last_became_active":"2023-10-06T05:24:05.728167+0000","last_became_peered":"2023-10-06T05:24:05.728167+0000","last_unstale":"2023-10-06T05:24:07.120233+0000","last_undegraded":"2023-10-06T05:24:07.120233+0000","last_fullsized":"2023-10-06T05:24:07.120233+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:40.510468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.741322+0000","last_change":"2023-10-06T05:24:06.735309+0000","last_active":"2023-10-06T05:24:06.741322+0000","last_peered":"2023-10-06T05:24:06.741322+0000","last_clean":"2023-10-06T05:24:06.741322+0000","last_became_active":"2023-10-06T05:24:06.735095+0000","last_became_peered":"2023-10-06T05:24:06.735095+0000","last_unstale":"2023-10-06T05:24:06.741322+0000","last_undegraded":"2023-10-06T05:24:06.741322+0000","last_fullsized":"2023-10-06T05:24:06.741322+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:57.137421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"126'2","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.121238+0000","last_change":"2023-10-06T05:24:05.727449+0000","last_active":"2023-10-06T05:24:07.121238+0000","last_peered":"2023-10-06T05:24:07.121238+0000","last_clean":"2023-10-06T05:24:07.121238+0000","last_became_active":"2023-10-06T05:24:05.727177+0000","last_became_peered":"2023-10-06T05:24:05.727177+0000","last_unstale":"2023-10-06T05:24:07.121238+0000","last_undegraded":"2023-10-06T05:24:07.121238+0000","last_fullsized":"2023-10-06T05:24:07.121238+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:26.823343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.726152+0000","last_change":"2023-10-06T05:24:06.722345+0000","last_active":"2023-10-06T05:24:06.726152+0000","last_peered":"2023-10-06T05:24:06.726152+0000","last_clean":"2023-10-06T05:24:06.726152+0000","last_became_active":"2023-10-06T05:24:06.722075+0000","last_became_peered":"2023-10-06T05:24:06.722075+0000","last_unstale":"2023-10-06T05:24:06.726152+0000","last_undegraded":"2023-10-06T05:24:06.726152+0000","last_fullsized":"2023-10-06T05:24:06.726152+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:57.137421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"126'1","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.118966+0000","last_change":"2023-10-06T05:24:05.727924+0000","last_active":"2023-10-06T05:24:07.118966+0000","last_peered":"2023-10-06T05:24:07.118966+0000","last_clean":"2023-10-06T05:24:07.118966+0000","last_became_active":"2023-10-06T05:24:05.727521+0000","last_became_peered":"2023-10-06T05:24:05.727521+0000","last_unstale":"2023-10-06T05:24:07.118966+0000","last_undegraded":"2023-10-06T05:24:07.118966+0000","last_fullsized":"2023-10-06T05:24:07.118966+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:29.277714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.727322+0000","last_change":"2023-10-06T05:24:06.713632+0000","last_active":"2023-10-06T05:24:06.727322+0000","last_peered":"2023-10-06T05:24:06.727322+0000","last_clean":"2023-10-06T05:24:06.727322+0000","last_became_active":"2023-10-06T05:24:06.713255+0000","last_became_peered":"2023-10-06T05:24:06.713255+0000","last_unstale":"2023-10-06T05:24:06.727322+0000","last_undegraded":"2023-10-06T05:24:06.727322+0000","last_fullsized":"2023-10-06T05:24:06.727322+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:21.213252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"126'1","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.118637+0000","last_change":"2023-10-06T05:24:05.717504+0000","last_active":"2023-10-06T05:24:07.118637+0000","last_peered":"2023-10-06T05:24:07.118637+0000","last_clean":"2023-10-06T05:24:07.118637+0000","last_became_active":"2023-10-06T05:24:05.717220+0000","last_became_peered":"2023-10-06T05:24:05.717220+0000","last_unstale":"2023-10-06T05:24:07.118637+0000","last_undegraded":"2023-10-06T05:24:07.118637+0000","last_fullsized":"2023-10-06T05:24:07.118637+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:11.179004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.723918+0000","last_change":"2023-10-06T05:24:06.719804+0000","last_active":"2023-10-06T05:24:06.723918+0000","last_peered":"2023-10-06T05:24:06.723918+0000","last_clean":"2023-10-06T05:24:06.723918+0000","last_became_active":"2023-10-06T05:24:06.719404+0000","last_became_peered":"2023-10-06T05:24:06.719404+0000","last_unstale":"2023-10-06T05:24:06.723918+0000","last_undegraded":"2023-10-06T05:24:06.723918+0000","last_fullsized":"2023-10-06T05:24:06.723918+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:32.207879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.716416+0000","last_change":"2023-10-06T05:24:05.720465+0000","last_active":"2023-10-06T05:24:06.716416+0000","last_peered":"2023-10-06T05:24:06.716416+0000","last_clean":"2023-10-06T05:24:06.716416+0000","last_became_active":"2023-10-06T05:24:05.720063+0000","last_became_peered":"2023-10-06T05:24:05.720063+0000","last_unstale":"2023-10-06T05:24:06.716416+0000","last_undegraded":"2023-10-06T05:24:06.716416+0000","last_fullsized":"2023-10-06T05:24:06.716416+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:20.742878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.738516+0000","last_change":"2023-10-06T05:24:06.732388+0000","last_active":"2023-10-06T05:24:06.738516+0000","last_peered":"2023-10-06T05:24:06.738516+0000","last_clean":"2023-10-06T05:24:06.738516+0000","last_became_active":"2023-10-06T05:24:06.732091+0000","last_became_peered":"2023-10-06T05:24:06.732091+0000","last_unstale":"2023-10-06T05:24:06.738516+0000","last_undegraded":"2023-10-06T05:24:06.738516+0000","last_fullsized":"2023-10-06T05:24:06.738516+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:15.550776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.710006+0000","last_change":"2023-10-06T05:24:05.721747+0000","last_active":"2023-10-06T05:24:06.710006+0000","last_peered":"2023-10-06T05:24:06.710006+0000","last_clean":"2023-10-06T05:24:06.710006+0000","last_became_active":"2023-10-06T05:24:05.720794+0000","last_became_peered":"2023-10-06T05:24:05.720794+0000","last_unstale":"2023-10-06T05:24:06.710006+0000","last_undegraded":"2023-10-06T05:24:06.710006+0000","last_fullsized":"2023-10-06T05:24:06.710006+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:10.314272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.731739+0000","last_change":"2023-10-06T05:24:06.729181+0000","last_active":"2023-10-06T05:24:06.731739+0000","last_peered":"2023-10-06T05:24:06.731739+0000","last_clean":"2023-10-06T05:24:06.731739+0000","last_became_active":"2023-10-06T05:24:06.728882+0000","last_became_peered":"2023-10-06T05:24:06.728882+0000","last_unstale":"2023-10-06T05:24:06.731739+0000","last_undegraded":"2023-10-06T05:24:06.731739+0000","last_fullsized":"2023-10-06T05:24:06.731739+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:04.744072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"126'1","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.120305+0000","last_change":"2023-10-06T05:24:05.719260+0000","last_active":"2023-10-06T05:24:07.120305+0000","last_peered":"2023-10-06T05:24:07.120305+0000","last_clean":"2023-10-06T05:24:07.120305+0000","last_became_active":"2023-10-06T05:24:05.718959+0000","last_became_peered":"2023-10-06T05:24:05.718959+0000","last_unstale":"2023-10-06T05:24:07.120305+0000","last_undegraded":"2023-10-06T05:24:07.120305+0000","last_fullsized":"2023-10-06T05:24:07.120305+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:14.517424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.741244+0000","last_change":"2023-10-06T05:24:06.734780+0000","last_active":"2023-10-06T05:24:06.741244+0000","last_peered":"2023-10-06T05:24:06.741244+0000","last_clean":"2023-10-06T05:24:06.741244+0000","last_became_active":"2023-10-06T05:24:06.734107+0000","last_became_peered":"2023-10-06T05:24:06.734107+0000","last_unstale":"2023-10-06T05:24:06.741244+0000","last_undegraded":"2023-10-06T05:24:06.741244+0000","last_fullsized":"2023-10-06T05:24:06.741244+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:54.129394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"126'2","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.119416+0000","last_change":"2023-10-06T05:24:05.724542+0000","last_active":"2023-10-06T05:24:07.119416+0000","last_peered":"2023-10-06T05:24:07.119416+0000","last_clean":"2023-10-06T05:24:07.119416+0000","last_became_active":"2023-10-06T05:24:05.724236+0000","last_became_peered":"2023-10-06T05:24:05.724236+0000","last_unstale":"2023-10-06T05:24:07.119416+0000","last_undegraded":"2023-10-06T05:24:07.119416+0000","last_fullsized":"2023-10-06T05:24:07.119416+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:07.454659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.731882+0000","last_change":"2023-10-06T05:24:06.727982+0000","last_active":"2023-10-06T05:24:06.731882+0000","last_peered":"2023-10-06T05:24:06.731882+0000","last_clean":"2023-10-06T05:24:06.731882+0000","last_became_active":"2023-10-06T05:24:06.727711+0000","last_became_peered":"2023-10-06T05:24:06.727711+0000","last_unstale":"2023-10-06T05:24:06.731882+0000","last_undegraded":"2023-10-06T05:24:06.731882+0000","last_fullsized":"2023-10-06T05:24:06.731882+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:22.450805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"126'1","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.119053+0000","last_change":"2023-10-06T05:24:05.721144+0000","last_active":"2023-10-06T05:24:07.119053+0000","last_peered":"2023-10-06T05:24:07.119053+0000","last_clean":"2023-10-06T05:24:07.119053+0000","last_became_active":"2023-10-06T05:24:05.720848+0000","last_became_peered":"2023-10-06T05:24:05.720848+0000","last_unstale":"2023-10-06T05:24:07.119053+0000","last_undegraded":"2023-10-06T05:24:07.119053+0000","last_fullsized":"2023-10-06T05:24:07.119053+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:53.556572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.741183+0000","last_change":"2023-10-06T05:24:06.735269+0000","last_active":"2023-10-06T05:24:06.741183+0000","last_peered":"2023-10-06T05:24:06.741183+0000","last_clean":"2023-10-06T05:24:06.741183+0000","last_became_active":"2023-10-06T05:24:06.734962+0000","last_became_peered":"2023-10-06T05:24:06.734962+0000","last_unstale":"2023-10-06T05:24:06.741183+0000","last_undegraded":"2023-10-06T05:24:06.741183+0000","last_fullsized":"2023-10-06T05:24:06.741183+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:11.324716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.710885+0000","last_change":"2023-10-06T05:24:05.728217+0000","last_active":"2023-10-06T05:24:06.710885+0000","last_peered":"2023-10-06T05:24:06.710885+0000","last_clean":"2023-10-06T05:24:06.710885+0000","last_became_active":"2023-10-06T05:24:05.727916+0000","last_became_peered":"2023-10-06T05:24:05.727916+0000","last_unstale":"2023-10-06T05:24:06.710885+0000","last_undegraded":"2023-10-06T05:24:06.710885+0000","last_fullsized":"2023-10-06T05:24:06.710885+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:15.522770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.741259+0000","last_change":"2023-10-06T05:24:06.734763+0000","last_active":"2023-10-06T05:24:06.741259+0000","last_peered":"2023-10-06T05:24:06.741259+0000","last_clean":"2023-10-06T05:24:06.741259+0000","last_became_active":"2023-10-06T05:24:06.734457+0000","last_became_peered":"2023-10-06T05:24:06.734457+0000","last_unstale":"2023-10-06T05:24:06.741259+0000","last_undegraded":"2023-10-06T05:24:06.741259+0000","last_fullsized":"2023-10-06T05:24:06.741259+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:07.247292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"126'2","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.121385+0000","last_change":"2023-10-06T05:24:05.729185+0000","last_active":"2023-10-06T05:24:07.121385+0000","last_peered":"2023-10-06T05:24:07.121385+0000","last_clean":"2023-10-06T05:24:07.121385+0000","last_became_active":"2023-10-06T05:24:05.728912+0000","last_became_peered":"2023-10-06T05:24:05.728912+0000","last_unstale":"2023-10-06T05:24:07.121385+0000","last_undegraded":"2023-10-06T05:24:07.121385+0000","last_fullsized":"2023-10-06T05:24:07.121385+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:21.265558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.731569+0000","last_change":"2023-10-06T05:24:06.726332+0000","last_active":"2023-10-06T05:24:06.731569+0000","last_peered":"2023-10-06T05:24:06.731569+0000","last_clean":"2023-10-06T05:24:06.731569+0000","last_became_active":"2023-10-06T05:24:06.725957+0000","last_became_peered":"2023-10-06T05:24:06.725957+0000","last_unstale":"2023-10-06T05:24:06.731569+0000","last_undegraded":"2023-10-06T05:24:06.731569+0000","last_fullsized":"2023-10-06T05:24:06.731569+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:59.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.718925+0000","last_change":"2023-10-06T05:24:05.720003+0000","last_active":"2023-10-06T05:24:06.718925+0000","last_peered":"2023-10-06T05:24:06.718925+0000","last_clean":"2023-10-06T05:24:06.718925+0000","last_became_active":"2023-10-06T05:24:05.719744+0000","last_became_peered":"2023-10-06T05:24:05.719744+0000","last_unstale":"2023-10-06T05:24:06.718925+0000","last_undegraded":"2023-10-06T05:24:06.718925+0000","last_fullsized":"2023-10-06T05:24:06.718925+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:26.823343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.731407+0000","last_change":"2023-10-06T05:24:06.727121+0000","last_active":"2023-10-06T05:24:06.731407+0000","last_peered":"2023-10-06T05:24:06.731407+0000","last_clean":"2023-10-06T05:24:06.731407+0000","last_became_active":"2023-10-06T05:24:06.726780+0000","last_became_peered":"2023-10-06T05:24:06.726780+0000","last_unstale":"2023-10-06T05:24:06.731407+0000","last_undegraded":"2023-10-06T05:24:06.731407+0000","last_fullsized":"2023-10-06T05:24:06.731407+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:58.940036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.715409+0000","last_change":"2023-10-06T05:24:05.719375+0000","last_active":"2023-10-06T05:24:06.715409+0000","last_peered":"2023-10-06T05:24:06.715409+0000","last_clean":"2023-10-06T05:24:06.715409+0000","last_became_active":"2023-10-06T05:24:05.719098+0000","last_became_peered":"2023-10-06T05:24:05.719098+0000","last_unstale":"2023-10-06T05:24:06.715409+0000","last_undegraded":"2023-10-06T05:24:06.715409+0000","last_fullsized":"2023-10-06T05:24:06.715409+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:03.196093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.722820+0000","last_change":"2023-10-06T05:24:06.717046+0000","last_active":"2023-10-06T05:24:06.722820+0000","last_peered":"2023-10-06T05:24:06.722820+0000","last_clean":"2023-10-06T05:24:06.722820+0000","last_became_active":"2023-10-06T05:24:06.716759+0000","last_became_peered":"2023-10-06T05:24:06.716759+0000","last_unstale":"2023-10-06T05:24:06.722820+0000","last_undegraded":"2023-10-06T05:24:06.722820+0000","last_fullsized":"2023-10-06T05:24:06.722820+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:26.599528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.718264+0000","last_change":"2023-10-06T05:24:05.726797+0000","last_active":"2023-10-06T05:24:06.718264+0000","last_peered":"2023-10-06T05:24:06.718264+0000","last_clean":"2023-10-06T05:24:06.718264+0000","last_became_active":"2023-10-06T05:24:05.726504+0000","last_became_peered":"2023-10-06T05:24:05.726504+0000","last_unstale":"2023-10-06T05:24:06.718264+0000","last_undegraded":"2023-10-06T05:24:06.718264+0000","last_fullsized":"2023-10-06T05:24:06.718264+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:53.556572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.730757+0000","last_change":"2023-10-06T05:24:06.727700+0000","last_active":"2023-10-06T05:24:06.730757+0000","last_peered":"2023-10-06T05:24:06.730757+0000","last_clean":"2023-10-06T05:24:06.730757+0000","last_became_active":"2023-10-06T05:24:06.727351+0000","last_became_peered":"2023-10-06T05:24:06.727351+0000","last_unstale":"2023-10-06T05:24:06.730757+0000","last_undegraded":"2023-10-06T05:24:06.730757+0000","last_fullsized":"2023-10-06T05:24:06.730757+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:12.125031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"126'2","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.120655+0000","last_change":"2023-10-06T05:24:05.729691+0000","last_active":"2023-10-06T05:24:07.120655+0000","last_peered":"2023-10-06T05:24:07.120655+0000","last_clean":"2023-10-06T05:24:07.120655+0000","last_became_active":"2023-10-06T05:24:05.729129+0000","last_became_peered":"2023-10-06T05:24:05.729129+0000","last_unstale":"2023-10-06T05:24:07.120655+0000","last_undegraded":"2023-10-06T05:24:07.120655+0000","last_fullsized":"2023-10-06T05:24:07.120655+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:22.017322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733526+0000","last_change":"2023-10-06T05:24:06.720867+0000","last_active":"2023-10-06T05:24:06.733526+0000","last_peered":"2023-10-06T05:24:06.733526+0000","last_clean":"2023-10-06T05:24:06.733526+0000","last_became_active":"2023-10-06T05:24:06.720326+0000","last_became_peered":"2023-10-06T05:24:06.720326+0000","last_unstale":"2023-10-06T05:24:06.733526+0000","last_undegraded":"2023-10-06T05:24:06.733526+0000","last_fullsized":"2023-10-06T05:24:06.733526+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:23.591501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"127'1","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.865328+0000","last_change":"2023-10-06T05:24:05.725667+0000","last_active":"2023-10-06T05:24:07.865328+0000","last_peered":"2023-10-06T05:24:07.865328+0000","last_clean":"2023-10-06T05:24:07.865328+0000","last_became_active":"2023-10-06T05:24:05.725396+0000","last_became_peered":"2023-10-06T05:24:05.725396+0000","last_unstale":"2023-10-06T05:24:07.865328+0000","last_undegraded":"2023-10-06T05:24:07.865328+0000","last_fullsized":"2023-10-06T05:24:07.865328+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:13.371969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.741194+0000","last_change":"2023-10-06T05:24:06.735292+0000","last_active":"2023-10-06T05:24:06.741194+0000","last_peered":"2023-10-06T05:24:06.741194+0000","last_clean":"2023-10-06T05:24:06.741194+0000","last_became_active":"2023-10-06T05:24:06.734988+0000","last_became_peered":"2023-10-06T05:24:06.734988+0000","last_unstale":"2023-10-06T05:24:06.741194+0000","last_undegraded":"2023-10-06T05:24:06.741194+0000","last_fullsized":"2023-10-06T05:24:06.741194+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:40.145829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.711065+0000","last_change":"2023-10-06T05:24:05.728225+0000","last_active":"2023-10-06T05:24:06.711065+0000","last_peered":"2023-10-06T05:24:06.711065+0000","last_clean":"2023-10-06T05:24:06.711065+0000","last_became_active":"2023-10-06T05:24:05.727932+0000","last_became_peered":"2023-10-06T05:24:05.727932+0000","last_unstale":"2023-10-06T05:24:06.711065+0000","last_undegraded":"2023-10-06T05:24:06.711065+0000","last_fullsized":"2023-10-06T05:24:06.711065+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:29.277714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.718321+0000","last_change":"2023-10-06T05:24:06.716203+0000","last_active":"2023-10-06T05:24:06.718321+0000","last_peered":"2023-10-06T05:24:06.718321+0000","last_clean":"2023-10-06T05:24:06.718321+0000","last_became_active":"2023-10-06T05:24:06.715950+0000","last_became_peered":"2023-10-06T05:24:06.715950+0000","last_unstale":"2023-10-06T05:24:06.718321+0000","last_undegraded":"2023-10-06T05:24:06.718321+0000","last_fullsized":"2023-10-06T05:24:06.718321+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:14.355277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.709855+0000","last_change":"2023-10-06T05:24:05.727450+0000","last_active":"2023-10-06T05:24:06.709855+0000","last_peered":"2023-10-06T05:24:06.709855+0000","last_clean":"2023-10-06T05:24:06.709855+0000","last_became_active":"2023-10-06T05:24:05.727184+0000","last_became_peered":"2023-10-06T05:24:05.727184+0000","last_unstale":"2023-10-06T05:24:06.709855+0000","last_undegraded":"2023-10-06T05:24:06.709855+0000","last_fullsized":"2023-10-06T05:24:06.709855+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:59.507404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.734394+0000","last_change":"2023-10-06T05:24:06.729692+0000","last_active":"2023-10-06T05:24:06.734394+0000","last_peered":"2023-10-06T05:24:06.734394+0000","last_clean":"2023-10-06T05:24:06.734394+0000","last_became_active":"2023-10-06T05:24:06.727964+0000","last_became_peered":"2023-10-06T05:24:06.727964+0000","last_unstale":"2023-10-06T05:24:06.734394+0000","last_undegraded":"2023-10-06T05:24:06.734394+0000","last_fullsized":"2023-10-06T05:24:06.734394+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:26.599528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.717603+0000","last_change":"2023-10-06T05:24:05.725732+0000","last_active":"2023-10-06T05:24:06.717603+0000","last_peered":"2023-10-06T05:24:06.717603+0000","last_clean":"2023-10-06T05:24:06.717603+0000","last_became_active":"2023-10-06T05:24:05.725468+0000","last_became_peered":"2023-10-06T05:24:05.725468+0000","last_unstale":"2023-10-06T05:24:06.717603+0000","last_undegraded":"2023-10-06T05:24:06.717603+0000","last_fullsized":"2023-10-06T05:24:06.717603+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:05.730446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.729609+0000","last_change":"2023-10-06T05:24:06.717099+0000","last_active":"2023-10-06T05:24:06.729609+0000","last_peered":"2023-10-06T05:24:06.729609+0000","last_clean":"2023-10-06T05:24:06.729609+0000","last_became_active":"2023-10-06T05:24:06.716826+0000","last_became_peered":"2023-10-06T05:24:06.716826+0000","last_unstale":"2023-10-06T05:24:06.729609+0000","last_undegraded":"2023-10-06T05:24:06.729609+0000","last_fullsized":"2023-10-06T05:24:06.729609+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:09.832298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.705089+0000","last_change":"2023-10-06T05:24:05.728985+0000","last_active":"2023-10-06T05:24:06.705089+0000","last_peered":"2023-10-06T05:24:06.705089+0000","last_clean":"2023-10-06T05:24:06.705089+0000","last_became_active":"2023-10-06T05:24:05.728637+0000","last_became_peered":"2023-10-06T05:24:05.728637+0000","last_unstale":"2023-10-06T05:24:06.705089+0000","last_undegraded":"2023-10-06T05:24:06.705089+0000","last_fullsized":"2023-10-06T05:24:06.705089+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:29.046559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.731440+0000","last_change":"2023-10-06T05:24:06.726322+0000","last_active":"2023-10-06T05:24:06.731440+0000","last_peered":"2023-10-06T05:24:06.731440+0000","last_clean":"2023-10-06T05:24:06.731440+0000","last_became_active":"2023-10-06T05:24:06.725423+0000","last_became_peered":"2023-10-06T05:24:06.725423+0000","last_unstale":"2023-10-06T05:24:06.731440+0000","last_undegraded":"2023-10-06T05:24:06.731440+0000","last_fullsized":"2023-10-06T05:24:06.731440+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:24.149580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.716298+0000","last_change":"2023-10-06T05:24:05.720333+0000","last_active":"2023-10-06T05:24:06.716298+0000","last_peered":"2023-10-06T05:24:06.716298+0000","last_clean":"2023-10-06T05:24:06.716298+0000","last_became_active":"2023-10-06T05:24:05.719660+0000","last_became_peered":"2023-10-06T05:24:05.719660+0000","last_unstale":"2023-10-06T05:24:06.716298+0000","last_undegraded":"2023-10-06T05:24:06.716298+0000","last_fullsized":"2023-10-06T05:24:06.716298+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:13:36.582559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733436+0000","last_change":"2023-10-06T05:24:06.725718+0000","last_active":"2023-10-06T05:24:06.733436+0000","last_peered":"2023-10-06T05:24:06.733436+0000","last_clean":"2023-10-06T05:24:06.733436+0000","last_became_active":"2023-10-06T05:24:06.725005+0000","last_became_peered":"2023-10-06T05:24:06.725005+0000","last_unstale":"2023-10-06T05:24:06.733436+0000","last_undegraded":"2023-10-06T05:24:06.733436+0000","last_fullsized":"2023-10-06T05:24:06.733436+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:24.149580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.715968+0000","last_change":"2023-10-06T05:24:05.724366+0000","last_active":"2023-10-06T05:24:06.715968+0000","last_peered":"2023-10-06T05:24:06.715968+0000","last_clean":"2023-10-06T05:24:06.715968+0000","last_became_active":"2023-10-06T05:24:05.724056+0000","last_became_peered":"2023-10-06T05:24:05.724056+0000","last_unstale":"2023-10-06T05:24:06.715968+0000","last_undegraded":"2023-10-06T05:24:06.715968+0000","last_fullsized":"2023-10-06T05:24:06.715968+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:59.507404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.736540+0000","last_change":"2023-10-06T05:24:06.733663+0000","last_active":"2023-10-06T05:24:06.736540+0000","last_peered":"2023-10-06T05:24:06.736540+0000","last_clean":"2023-10-06T05:24:06.736540+0000","last_became_active":"2023-10-06T05:24:06.733149+0000","last_became_peered":"2023-10-06T05:24:06.733149+0000","last_unstale":"2023-10-06T05:24:06.736540+0000","last_undegraded":"2023-10-06T05:24:06.736540+0000","last_fullsized":"2023-10-06T05:24:06.736540+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:11.497121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"126'2","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.119948+0000","last_change":"2023-10-06T05:24:05.714026+0000","last_active":"2023-10-06T05:24:07.119948+0000","last_peered":"2023-10-06T05:24:07.119948+0000","last_clean":"2023-10-06T05:24:07.119948+0000","last_became_active":"2023-10-06T05:24:05.713754+0000","last_became_peered":"2023-10-06T05:24:05.713754+0000","last_unstale":"2023-10-06T05:24:07.119948+0000","last_undegraded":"2023-10-06T05:24:07.119948+0000","last_fullsized":"2023-10-06T05:24:07.119948+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:39.135386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.710559+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:24:06.710559+0000","last_peered":"2023-10-06T05:24:06.710559+0000","last_clean":"2023-10-06T05:24:06.710559+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:24:06.710559+0000","last_undegraded":"2023-10-06T05:24:06.710559+0000","last_fullsized":"2023-10-06T05:24:06.710559+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.718547+0000","last_change":"2023-10-06T05:24:06.714699+0000","last_active":"2023-10-06T05:24:06.718547+0000","last_peered":"2023-10-06T05:24:06.718547+0000","last_clean":"2023-10-06T05:24:06.718547+0000","last_became_active":"2023-10-06T05:24:06.714398+0000","last_became_peered":"2023-10-06T05:24:06.714398+0000","last_unstale":"2023-10-06T05:24:06.718547+0000","last_undegraded":"2023-10-06T05:24:06.718547+0000","last_fullsized":"2023-10-06T05:24:06.718547+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:11.497121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.710484+0000","last_change":"2023-10-06T05:24:05.729545+0000","last_active":"2023-10-06T05:24:06.710484+0000","last_peered":"2023-10-06T05:24:06.710484+0000","last_clean":"2023-10-06T05:24:06.710484+0000","last_became_active":"2023-10-06T05:24:05.729229+0000","last_became_peered":"2023-10-06T05:24:05.729229+0000","last_unstale":"2023-10-06T05:24:06.710484+0000","last_undegraded":"2023-10-06T05:24:06.710484+0000","last_fullsized":"2023-10-06T05:24:06.710484+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:28.303622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.709304+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:24:06.709304+0000","last_peered":"2023-10-06T05:24:06.709304+0000","last_clean":"2023-10-06T05:24:06.709304+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:24:06.709304+0000","last_undegraded":"2023-10-06T05:24:06.709304+0000","last_fullsized":"2023-10-06T05:24:06.709304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.731808+0000","last_change":"2023-10-06T05:24:06.727098+0000","last_active":"2023-10-06T05:24:06.731808+0000","last_peered":"2023-10-06T05:24:06.731808+0000","last_clean":"2023-10-06T05:24:06.731808+0000","last_became_active":"2023-10-06T05:24:06.726629+0000","last_became_peered":"2023-10-06T05:24:06.726629+0000","last_unstale":"2023-10-06T05:24:06.731808+0000","last_undegraded":"2023-10-06T05:24:06.731808+0000","last_fullsized":"2023-10-06T05:24:06.731808+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:31.890482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"126'3","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.120444+0000","last_change":"2023-10-06T05:24:05.713794+0000","last_active":"2023-10-06T05:24:07.120444+0000","last_peered":"2023-10-06T05:24:07.120444+0000","last_clean":"2023-10-06T05:24:07.120444+0000","last_became_active":"2023-10-06T05:24:05.713539+0000","last_became_peered":"2023-10-06T05:24:05.713539+0000","last_unstale":"2023-10-06T05:24:07.120444+0000","last_undegraded":"2023-10-06T05:24:07.120444+0000","last_fullsized":"2023-10-06T05:24:07.120444+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:47.207042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.716499+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:24:06.716499+0000","last_peered":"2023-10-06T05:24:06.716499+0000","last_clean":"2023-10-06T05:24:06.716499+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:24:06.716499+0000","last_undegraded":"2023-10-06T05:24:06.716499+0000","last_fullsized":"2023-10-06T05:24:06.716499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733090+0000","last_change":"2023-10-06T05:24:06.726607+0000","last_active":"2023-10-06T05:24:06.733090+0000","last_peered":"2023-10-06T05:24:06.733090+0000","last_clean":"2023-10-06T05:24:06.733090+0000","last_became_active":"2023-10-06T05:24:06.726213+0000","last_became_peered":"2023-10-06T05:24:06.726213+0000","last_unstale":"2023-10-06T05:24:06.733090+0000","last_undegraded":"2023-10-06T05:24:06.733090+0000","last_fullsized":"2023-10-06T05:24:06.733090+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:59.983040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"126'6","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.122903+0000","last_change":"2023-10-06T05:24:05.727464+0000","last_active":"2023-10-06T05:24:07.122903+0000","last_peered":"2023-10-06T05:24:07.122903+0000","last_clean":"2023-10-06T05:24:07.122903+0000","last_became_active":"2023-10-06T05:24:05.727158+0000","last_became_peered":"2023-10-06T05:24:05.727158+0000","last_unstale":"2023-10-06T05:24:07.122903+0000","last_undegraded":"2023-10-06T05:24:07.122903+0000","last_fullsized":"2023-10-06T05:24:07.122903+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:59.452232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.716178+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:24:06.716178+0000","last_peered":"2023-10-06T05:24:06.716178+0000","last_clean":"2023-10-06T05:24:06.716178+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:24:06.716178+0000","last_undegraded":"2023-10-06T05:24:06.716178+0000","last_fullsized":"2023-10-06T05:24:06.716178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.733179+0000","last_change":"2023-10-06T05:24:06.728569+0000","last_active":"2023-10-06T05:24:06.733179+0000","last_peered":"2023-10-06T05:24:06.733179+0000","last_clean":"2023-10-06T05:24:06.733179+0000","last_became_active":"2023-10-06T05:24:06.728271+0000","last_became_peered":"2023-10-06T05:24:06.728271+0000","last_unstale":"2023-10-06T05:24:06.733179+0000","last_undegraded":"2023-10-06T05:24:06.733179+0000","last_fullsized":"2023-10-06T05:24:06.733179+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:32:05.177459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.722510+0000","last_change":"2023-10-06T05:24:05.724587+0000","last_active":"2023-10-06T05:24:06.722510+0000","last_peered":"2023-10-06T05:24:06.722510+0000","last_clean":"2023-10-06T05:24:06.722510+0000","last_became_active":"2023-10-06T05:24:05.724331+0000","last_became_peered":"2023-10-06T05:24:05.724331+0000","last_unstale":"2023-10-06T05:24:06.722510+0000","last_undegraded":"2023-10-06T05:24:06.722510+0000","last_fullsized":"2023-10-06T05:24:06.722510+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:44.363859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":96,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.716443+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:24:06.716443+0000","last_peered":"2023-10-06T05:24:06.716443+0000","last_clean":"2023-10-06T05:24:06.716443+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:24:06.716443+0000","last_undegraded":"2023-10-06T05:24:06.716443+0000","last_fullsized":"2023-10-06T05:24:06.716443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.734636+0000","last_change":"2023-10-06T05:24:06.731648+0000","last_active":"2023-10-06T05:24:06.734636+0000","last_peered":"2023-10-06T05:24:06.734636+0000","last_clean":"2023-10-06T05:24:06.734636+0000","last_became_active":"2023-10-06T05:24:06.731358+0000","last_became_peered":"2023-10-06T05:24:06.731358+0000","last_unstale":"2023-10-06T05:24:06.734636+0000","last_undegraded":"2023-10-06T05:24:06.734636+0000","last_fullsized":"2023-10-06T05:24:06.734636+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:08.701111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.724490+0000","last_change":"2023-10-06T05:24:05.725681+0000","last_active":"2023-10-06T05:24:06.724490+0000","last_peered":"2023-10-06T05:24:06.724490+0000","last_clean":"2023-10-06T05:24:06.724490+0000","last_became_active":"2023-10-06T05:24:05.725320+0000","last_became_peered":"2023-10-06T05:24:05.725320+0000","last_unstale":"2023-10-06T05:24:06.724490+0000","last_undegraded":"2023-10-06T05:24:06.724490+0000","last_fullsized":"2023-10-06T05:24:06.724490+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:55.075685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":285,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.716087+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:24:06.716087+0000","last_peered":"2023-10-06T05:24:06.716087+0000","last_clean":"2023-10-06T05:24:06.716087+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:24:06.716087+0000","last_undegraded":"2023-10-06T05:24:06.716087+0000","last_fullsized":"2023-10-06T05:24:06.716087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":107,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.711704+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:24:06.711704+0000","last_peered":"2023-10-06T05:24:06.711704+0000","last_clean":"2023-10-06T05:24:06.711704+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:24:06.711704+0000","last_undegraded":"2023-10-06T05:24:06.711704+0000","last_fullsized":"2023-10-06T05:24:06.711704+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.723813+0000","last_change":"2023-10-06T05:24:06.719055+0000","last_active":"2023-10-06T05:24:06.723813+0000","last_peered":"2023-10-06T05:24:06.723813+0000","last_clean":"2023-10-06T05:24:06.723813+0000","last_became_active":"2023-10-06T05:24:06.718773+0000","last_became_peered":"2023-10-06T05:24:06.718773+0000","last_unstale":"2023-10-06T05:24:06.723813+0000","last_undegraded":"2023-10-06T05:24:06.723813+0000","last_fullsized":"2023-10-06T05:24:06.723813+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:43.129819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"126'8","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:07.121402+0000","last_change":"2023-10-06T05:24:05.722602+0000","last_active":"2023-10-06T05:24:07.121402+0000","last_peered":"2023-10-06T05:24:07.121402+0000","last_clean":"2023-10-06T05:24:07.121402+0000","last_became_active":"2023-10-06T05:24:05.722289+0000","last_became_peered":"2023-10-06T05:24:05.722289+0000","last_unstale":"2023-10-06T05:24:07.121402+0000","last_undegraded":"2023-10-06T05:24:07.121402+0000","last_fullsized":"2023-10-06T05:24:07.121402+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:58.972597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":120,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.709949+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:24:06.709949+0000","last_peered":"2023-10-06T05:24:06.709949+0000","last_clean":"2023-10-06T05:24:06.709949+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:24:06.709949+0000","last_undegraded":"2023-10-06T05:24:06.709949+0000","last_fullsized":"2023-10-06T05:24:06.709949+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.718297+0000","last_change":"2023-10-06T05:24:06.713588+0000","last_active":"2023-10-06T05:24:06.718297+0000","last_peered":"2023-10-06T05:24:06.718297+0000","last_clean":"2023-10-06T05:24:06.718297+0000","last_became_active":"2023-10-06T05:24:06.713090+0000","last_became_peered":"2023-10-06T05:24:06.713090+0000","last_unstale":"2023-10-06T05:24:06.718297+0000","last_undegraded":"2023-10-06T05:24:06.718297+0000","last_fullsized":"2023-10-06T05:24:06.718297+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:05.693085+0000","last_clean_scrub_stamp":"2023-10-06T05:24:05.693085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:18.043864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.709956+0000","last_change":"2023-10-06T05:24:05.725825+0000","last_active":"2023-10-06T05:24:06.709956+0000","last_peered":"2023-10-06T05:24:06.709956+0000","last_clean":"2023-10-06T05:24:06.709956+0000","last_became_active":"2023-10-06T05:24:05.725475+0000","last_became_peered":"2023-10-06T05:24:05.725475+0000","last_unstale":"2023-10-06T05:24:06.709956+0000","last_undegraded":"2023-10-06T05:24:06.709956+0000","last_fullsized":"2023-10-06T05:24:06.709956+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:04.682641+0000","last_clean_scrub_stamp":"2023-10-06T05:24:04.682641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:48.755016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-06T05:24:06.716637+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:24:06.716637+0000","last_peered":"2023-10-06T05:24:06.716637+0000","last_clean":"2023-10-06T05:24:06.716637+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:24:06.716637+0000","last_undegraded":"2023-10-06T05:24:06.716637+0000","last_fullsized":"2023-10-06T05:24:06.716637+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303524,"kb_used_data":2140,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068316,"statfs":{"total":96636764160,"available":96325955584,"internally_reserved":0,"allocated":2191360,"data_stored":1331577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.38700000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.57599999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.87,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67900000000000005,"15min":0.67800000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70399999999999996,"15min":0.74299999999999999},"min":{"1min":0.46899999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":1.056,"15min":1.056},"last":0.81899999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.55200000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99099999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64100000000000001,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":1.0249999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.77000000000000002},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0629999999999999,"5min":1.27,"15min":1.27},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68999999999999995,"15min":0.73299999999999998},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":0.97899999999999998,"15min":0.999},"last":0.502},{"interface":"front","average":{"1min":0.747,"5min":0.74199999999999999,"15min":0.81599999999999995},"min":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.98399999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.58899999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78900000000000003,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.1040000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79800000000000004,"15min":0.84199999999999997},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.441},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.55800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302944,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068896,"statfs":{"total":96636764160,"available":96326549504,"internally_reserved":0,"allocated":1597440,"data_stored":741257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65400000000000003,"15min":0.67500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.32300000000000001},"max":{"1min":0.88200000000000001,"5min":1.006,"15min":1.8500000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68000000000000005,"15min":0.69799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90500000000000003,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.177,"5min":1.177,"15min":1.9279999999999999},"last":0.755},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.998,"5min":1.073,"15min":1.4239999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65500000000000003,"15min":0.65000000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92700000000000005,"5min":1.093,"15min":1.369},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.754},"min":{"1min":0.47799999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.085,"5min":1.085,"15min":1.891},"last":0.81699999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.71699999999999997,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95999999999999996,"5min":1.018,"15min":1.347},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.71599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.97699999999999998,"5min":0.99299999999999999,"15min":1},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69899999999999995,"15min":0.76700000000000002},"min":{"1min":0.40600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.107,"5min":1.1140000000000001,"15min":1.9670000000000001},"last":0.871},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.81399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.997,"5min":1.278,"15min":1.4510000000000001},"last":0.84899999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.415},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73799999999999999,"15min":0.81000000000000005},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":2},"last":0.92400000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.76000000000000001,"15min":0.80200000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.032,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.77700000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.46},"last":0.93600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303524,"kb_used_data":2140,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068316,"statfs":{"total":96636764160,"available":96325955584,"internally_reserved":0,"allocated":2191360,"data_stored":1331625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.438},"min":{"1min":0.32900000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57799999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.59599999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.33900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71399999999999997,"5min":0.77000000000000002,"15min":0.83799999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.504,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.35399999999999998,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.38600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66000000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40100000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76500000000000001,"15min":0.78800000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.79900000000000004,"15min":0.79200000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.753,"15min":0.746},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.749,"15min":0.747},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.80700000000000005,"15min":0.81399999999999995},"min":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.50600000000000001},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.59099999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.103},"last":0.72499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302964,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068876,"statfs":{"total":96636764160,"available":96326529024,"internally_reserved":0,"allocated":1617920,"data_stored":743401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.434,"15min":0.42999999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56899999999999995,"5min":0.71499999999999997,"15min":0.84499999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.48399999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60599999999999998,"5min":0.78900000000000003,"15min":1.042},"last":0.32700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.497,"15min":0.48899999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66200000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52400000000000002,"15min":0.51200000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65800000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52000000000000002,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.79700000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0880000000000001,"5min":1.242,"15min":1.242},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.748,"15min":0.77200000000000002},"min":{"1min":0.54200000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.198,"15min":1.198},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76300000000000001,"15min":0.76400000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96499999999999997,"5min":1.26,"15min":1.26},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1299999999999999,"5min":1.163,"15min":1.163},"last":0.51200000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.999,"5min":1.323,"15min":1.323},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92300000000000004,"5min":1.1799999999999999,"15min":1.3100000000000001},"last":0.76300000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.82999999999999996},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.52400000000000002},"max":{"1min":0.999,"5min":1.2090000000000001,"15min":1.264},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78700000000000003,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":0.69499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302948,"kb_used_data":1564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068892,"statfs":{"total":96636764160,"available":96326545408,"internally_reserved":0,"allocated":1601536,"data_stored":741299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.42699999999999999},"min":{"1min":0.32000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51000000000000001,"5min":0.77400000000000002,"15min":0.77800000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48699999999999999,"15min":0.48399999999999999},"min":{"1min":0.32100000000000001,"5min":0.31,"15min":0.28399999999999997},"max":{"1min":0.60599999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45500000000000002,"15min":0.46200000000000002},"min":{"1min":0.29899999999999999,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.53700000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.498,"15min":0.5},"min":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.31900000000000001},"max":{"1min":0.61799999999999999,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.61799999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.55300000000000005,"5min":0.82799999999999996,"15min":0.93600000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.372,"15min":0.36199999999999999},"max":{"1min":0.626,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73199999999999998,"15min":0.753},"min":{"1min":0.58899999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88800000000000001,"5min":1.022,"15min":1.1299999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73399999999999999,"15min":0.78600000000000003},"min":{"1min":0.52800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92200000000000004,"5min":1.085,"15min":1.1699999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73499999999999999,"15min":0.77600000000000002},"min":{"1min":0.496,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.042,"15min":1.099},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.747,"15min":0.79500000000000004},"min":{"1min":0.56499999999999995,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97899999999999998,"5min":1.0649999999999999,"15min":1.163},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76800000000000002,"15min":0.82299999999999995},"min":{"1min":0.60599999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96299999999999997,"5min":1.0960000000000001,"15min":1.21},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.76600000000000001,"15min":0.80500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.876,"5min":1.022,"15min":1.042},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77900000000000003,"15min":0.80200000000000005},"min":{"1min":0.64400000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.997,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79000000000000004,"15min":0.84799999999999998},"min":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.93899999999999995,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":0.77400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302948,"kb_used_data":1564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068892,"statfs":{"total":96636764160,"available":96326545408,"internally_reserved":0,"allocated":1601536,"data_stored":741775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.58899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47099999999999997,"15min":0.46800000000000003},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.499},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60399999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70299999999999996,"5min":0.999,"15min":0.999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.63200000000000001,"5min":0.98999999999999999,"15min":1.014},"last":0.45200000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0349999999999999,"5min":1.161,"15min":1.2669999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.004,"5min":1.1830000000000001,"15min":1.3540000000000001},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.754,"15min":0.76400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.093,"5min":1.147,"15min":1.204},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.208,"5min":1.284,"15min":1.329},"last":0.71899999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77900000000000003,"15min":0.80800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92700000000000005,"5min":1.0880000000000001,"15min":1.484},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0169999999999999,"5min":1.2809999999999999,"15min":1.629},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81000000000000005,"15min":0.84499999999999997},"min":{"1min":0.58299999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1379999999999999,"5min":1.236,"15min":1.5620000000000001},"last":0.75},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.624,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":0.96399999999999997,"5min":1.194,"15min":1.325},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302952,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068888,"statfs":{"total":96636764160,"available":96326541312,"internally_reserved":0,"allocated":1605632,"data_stored":742723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.51700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89400000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.76800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.02,"5min":1.02,"15min":2.77},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69899999999999995,"15min":0.73299999999999998},"min":{"1min":0.48799999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.6419999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68899999999999995,"15min":0.71499999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.85799999999999998,"5min":1.1100000000000001,"15min":1.1899999999999999},"last":0.77400000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.497,"5min":0.47599999999999998,"15min":0.39800000000000002},"max":{"1min":0.79400000000000004,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72499999999999998,"15min":0.80300000000000005},"min":{"1min":0.57399999999999995,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97599999999999998,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71399999999999997,"15min":0.71699999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88800000000000001,"5min":0.95899999999999996,"15min":1.0629999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.996},"min":{"1min":0.56399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":7.742},"last":0.627}]},{"osd":5,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.73199999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93700000000000006,"5min":0.96499999999999997,"15min":1.079},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.747},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.03,"5min":1.04,"15min":1.181},"last":0.52300000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.73099999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98299999999999998,"5min":1.0029999999999999,"15min":1.2669999999999999},"last":0.498},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76100000000000001,"15min":0.81200000000000006},"min":{"1min":0.56399999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.093,"5min":1.093,"15min":1.171},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.82199999999999995},"min":{"1min":0.504,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.016,"5min":1.1899999999999999,"15min":1.228},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77300000000000002,"15min":0.84599999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.571},"last":0.56599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302940,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068900,"statfs":{"total":96636764160,"available":96326553600,"internally_reserved":0,"allocated":1593344,"data_stored":741187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65400000000000003,"15min":0.91000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.98199999999999998,"5min":1.109,"15min":9.2780000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.91300000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.93500000000000005},"min":{"1min":0.52500000000000002,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":1.0329999999999999,"5min":1.079,"15min":9.25},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.98899999999999999},"min":{"1min":0.46500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93200000000000005,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.85899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66400000000000003,"15min":0.95999999999999996},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98499999999999999,"5min":1.052,"15min":10.384},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.98699999999999999},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.097,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":1.0249999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.495},"max":{"1min":0.88200000000000001,"5min":1.76,"15min":10.205},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.55300000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.097,"15min":1.292},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68300000000000005,"15min":0.70099999999999996},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70699999999999996,"15min":0.76500000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.008,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70299999999999996,"15min":0.73499999999999999},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90900000000000003,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.80500000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95899999999999996,"5min":1.008,"15min":1.298},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.81799999999999995},"min":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0529999999999999,"5min":1.556,"15min":1.556},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.80700000000000005},"min":{"1min":0.498,"5min":0.432,"15min":0.432},"max":{"1min":1.04,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.89900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:24:12.008 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:24:12.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:24:12.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.243+0000 7fdbcc5ff700 1 Processor -- start 2023-10-06T05:24:12.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.243+0000 7fdbcc5ff700 1 -- start start 2023-10-06T05:24:12.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbcc5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdbc4159930 0x7fdbc40b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbcc5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdbc40cd580 0x7fdbc40b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbcc5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc415d2d0 con 0x7fdbc4159560 2023-10-06T05:24:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbcc5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc415d5b0 con 0x7fdbc4159930 2023-10-06T05:24:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbcc5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc415d890 con 0x7fdbc40cd580 2023-10-06T05:24:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbc9b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdbc4159930 0x7fdbc40b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbc9b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdbc4159930 0x7fdbc40b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49394/0 (socket says 172.21.15.31:49394) 2023-10-06T05:24:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbc9b9a700 1 -- 172.21.15.31:0/2341183564 learned_addr learned my addr 172.21.15.31:0/2341183564 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbca39b700 1 --1- 172.21.15.31:0/2341183564 >> v1:172.21.15.31:6789/0 conn(0x7fdbc4159560 0x7fdbc40cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39596/0 (socket says 172.21.15.31:39596) 2023-10-06T05:24:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbcab9c700 1 --2- 172.21.15.31:0/2341183564 >> v2:172.21.15.62:3300/0 conn(0x7fdbc40cd580 0x7fdbc40b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbc9b9a700 1 -- 172.21.15.31:0/2341183564 >> v2:172.21.15.62:3300/0 conn(0x7fdbc40cd580 msgr2=0x7fdbc40b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbc9b9a700 1 --2- 172.21.15.31:0/2341183564 >> v2:172.21.15.62:3300/0 conn(0x7fdbc40cd580 0x7fdbc40b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbc9b9a700 1 -- 172.21.15.31:0/2341183564 >> v1:172.21.15.31:6789/0 conn(0x7fdbc4159560 legacy=0x7fdbc40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbc9b9a700 1 -- 172.21.15.31:0/2341183564 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc415db70 con 0x7fdbc4159930 2023-10-06T05:24:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.245+0000 7fdbcab9c700 1 --2- 172.21.15.31:0/2341183564 >> v2:172.21.15.62:3300/0 conn(0x7fdbc40cd580 0x7fdbc40b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.246+0000 7fdbc9b9a700 1 --2- 172.21.15.31:0/2341183564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159930 0x7fdbc40b2470 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40b1740 tx=0x7fdbb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4b4b8009696508 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.246+0000 7fdbc9399700 1 -- 172.21.15.31:0/2341183564 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb400b860 con 0x7fdbc4159930 2023-10-06T05:24:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.246+0000 7fdbc9399700 1 -- 172.21.15.31:0/2341183564 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb400b9c0 con 0x7fdbc4159930 2023-10-06T05:24:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.246+0000 7fdbc9399700 1 -- 172.21.15.31:0/2341183564 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb400bd60 con 0x7fdbc4159930 2023-10-06T05:24:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.246+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/2341183564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159930 msgr2=0x7fdbc40b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.246+0000 7fdbcc5ff700 1 --2- 172.21.15.31:0/2341183564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159930 0x7fdbc40b2470 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40b1740 tx=0x7fdbb4009250 comp rx=0 tx=0).stop 2023-10-06T05:24:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.246+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/2341183564 shutdown_connections 2023-10-06T05:24:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.246+0000 7fdbcc5ff700 1 --2- 172.21.15.31:0/2341183564 >> v2:172.21.15.62:3300/0 conn(0x7fdbc40cd580 0x7fdbc40b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.247+0000 7fdbcc5ff700 1 --2- 172.21.15.31:0/2341183564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159930 0x7fdbc40b2470 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.247+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/2341183564 >> 172.21.15.31:0/2341183564 conn(0x7fdbc406ec90 msgr2=0x7fdbc406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.247+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/2341183564 shutdown_connections 2023-10-06T05:24:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.247+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/2341183564 wait complete. 2023-10-06T05:24:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.248+0000 7fdbcc5ff700 1 Processor -- start 2023-10-06T05:24:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.248+0000 7fdbcc5ff700 1 -- start start 2023-10-06T05:24:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.249+0000 7fdbcc5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc40cd580 0x7fdbc40c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.249+0000 7fdbcc5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159560 0x7fdbc415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.249+0000 7fdbcc5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc4159930 0x7fdbc415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.249+0000 7fdbcc5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc4169ea0 con 0x7fdbc4159560 2023-10-06T05:24:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.249+0000 7fdbcc5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40d02d0 con 0x7fdbc40cd580 2023-10-06T05:24:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.249+0000 7fdbcc5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc40d05b0 con 0x7fdbc4159930 2023-10-06T05:24:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.249+0000 7fdbc9b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159560 0x7fdbc415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbc9b9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159560 0x7fdbc415bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49406/0 (socket says 172.21.15.31:49406) 2023-10-06T05:24:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbc9b9a700 1 -- 172.21.15.31:0/3469177562 learned_addr learned my addr 172.21.15.31:0/3469177562 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbca39b700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc40cd580 0x7fdbc40c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbcab9c700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc4159930 0x7fdbc415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbc9b9a700 1 -- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc4159930 msgr2=0x7fdbc415c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbc9b9a700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc4159930 0x7fdbc415c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbc9b9a700 1 -- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc40cd580 msgr2=0x7fdbc40c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbc9b9a700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc40cd580 0x7fdbc40c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbc9b9a700 1 -- 172.21.15.31:0/3469177562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb40094f0 con 0x7fdbc4159560 2023-10-06T05:24:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbcab9c700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc4159930 0x7fdbc415c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbc9b9a700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159560 0x7fdbc415bb50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40893d0 tx=0x7fdbb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbba7fc700 1 -- 172.21.15.31:0/3469177562 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4011e60 con 0x7fdbc4159560 2023-10-06T05:24:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbba7fc700 1 -- 172.21.15.31:0/3469177562 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb401a630 con 0x7fdbc4159560 2023-10-06T05:24:12.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbba7fc700 1 -- 172.21.15.31:0/3469177562 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb401a9f0 con 0x7fdbc4159560 2023-10-06T05:24:12.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.250+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/3469177562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc40d0890 con 0x7fdbc4159560 2023-10-06T05:24:12.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.251+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/3469177562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc40c5920 con 0x7fdbc4159560 2023-10-06T05:24:12.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.252+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/3469177562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc416a070 con 0x7fdbc4159560 2023-10-06T05:24:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.253+0000 7fdbba7fc700 1 -- 172.21.15.31:0/3469177562 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fdbb402d440 con 0x7fdbc4159560 2023-10-06T05:24:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.253+0000 7fdbba7fc700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdba80674d0 0x7fdba8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.253+0000 7fdbba7fc700 1 -- 172.21.15.31:0/3469177562 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fdbb4086ba0 con 0x7fdbc4159560 2023-10-06T05:24:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.254+0000 7fdbca39b700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdba80674d0 0x7fdba8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.255+0000 7fdbca39b700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdba80674d0 0x7fdba8069980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0007fd0 tx=0x7fdbc0007d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.257+0000 7fdbba7fc700 1 -- 172.21.15.31:0/3469177562 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbc416a070 con 0x7fdbc4159560 2023-10-06T05:24:12.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.538+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/3469177562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdbc416a040 con 0x7fdbc4159560 2023-10-06T05:24:12.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.542+0000 7fdbba7fc700 1 -- 172.21.15.31:0/3469177562 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7fdbb40150e0 con 0x7fdbc4159560 2023-10-06T05:24:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.544+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdba80674d0 msgr2=0x7fdba8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.544+0000 7fdbcc5ff700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdba80674d0 0x7fdba8069980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0007fd0 tx=0x7fdbc0007d70 comp rx=0 tx=0).stop 2023-10-06T05:24:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.544+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/3469177562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159560 msgr2=0x7fdbc415bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.544+0000 7fdbcc5ff700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159560 0x7fdbc415bb50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40893d0 tx=0x7fdbb4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.545+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/3469177562 shutdown_connections 2023-10-06T05:24:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.545+0000 7fdbcc5ff700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdba80674d0 0x7fdba8069980 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.545+0000 7fdbcc5ff700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc4159930 0x7fdbc415c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.545+0000 7fdbcc5ff700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc40cd580 0x7fdbc40c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.545+0000 7fdbcc5ff700 1 --2- 172.21.15.31:0/3469177562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4159560 0x7fdbc415bb50 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.545+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/3469177562 >> 172.21.15.31:0/3469177562 conn(0x7fdbc406ec90 msgr2=0x7fdbc4158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.545+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/3469177562 shutdown_connections 2023-10-06T05:24:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.545+0000 7fdbcc5ff700 1 -- 172.21.15.31:0/3469177562 wait complete. 2023-10-06T05:24:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:24:12.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.757+0000 7f624e1ff700 1 Processor -- start 2023-10-06T05:24:12.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.757+0000 7f624e1ff700 1 -- start start 2023-10-06T05:24:12.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.758+0000 7f624e1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6248159500 0x7f62480b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.758+0000 7f624e1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f624815d2d0 0x7f62480cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.758+0000 7f624e1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62480c3f40 con 0x7f62481598d0 2023-10-06T05:24:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.758+0000 7f624e1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62480b2860 con 0x7f6248159500 2023-10-06T05:24:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.758+0000 7f624e1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624815d7d0 con 0x7f624815d2d0 2023-10-06T05:24:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.758+0000 7f62477fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6248159500 0x7f62480b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.759+0000 7f62477fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6248159500 0x7f62480b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49408/0 (socket says 172.21.15.31:49408) 2023-10-06T05:24:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.759+0000 7f62477fe700 1 -- 172.21.15.31:0/4210677077 learned_addr learned my addr 172.21.15.31:0/4210677077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.759+0000 7f6246ffd700 1 --2- 172.21.15.31:0/4210677077 >> v2:172.21.15.62:3300/0 conn(0x7f624815d2d0 0x7f62480cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.759+0000 7f62477fe700 1 -- 172.21.15.31:0/4210677077 >> v2:172.21.15.62:3300/0 conn(0x7f624815d2d0 msgr2=0x7f62480cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.759+0000 7f62477fe700 1 --2- 172.21.15.31:0/4210677077 >> v2:172.21.15.62:3300/0 conn(0x7f624815d2d0 0x7f62480cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.759+0000 7f62477fe700 1 -- 172.21.15.31:0/4210677077 >> v1:172.21.15.62:6789/0 conn(0x7f62481598d0 legacy=0x7f62480aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.759+0000 7f62477fe700 1 -- 172.21.15.31:0/4210677077 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62480cb000 con 0x7f6248159500 2023-10-06T05:24:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.759+0000 7f62477fe700 1 --2- 172.21.15.31:0/4210677077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6248159500 0x7f62480b0ab0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f624815d200 tx=0x7f6238009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3d74995c6f01f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.759+0000 7f62467fc700 1 -- 172.21.15.31:0/4210677077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623800b860 con 0x7f6248159500 2023-10-06T05:24:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.760+0000 7f62467fc700 1 -- 172.21.15.31:0/4210677077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623800b9c0 con 0x7f6248159500 2023-10-06T05:24:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.760+0000 7f62467fc700 1 -- 172.21.15.31:0/4210677077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623800bd60 con 0x7f6248159500 2023-10-06T05:24:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.760+0000 7f624e1ff700 1 -- 172.21.15.31:0/4210677077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6248159500 msgr2=0x7f62480b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.760+0000 7f624e1ff700 1 --2- 172.21.15.31:0/4210677077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6248159500 0x7f62480b0ab0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f624815d200 tx=0x7f6238009250 comp rx=0 tx=0).stop 2023-10-06T05:24:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.760+0000 7f624e1ff700 1 -- 172.21.15.31:0/4210677077 shutdown_connections 2023-10-06T05:24:12.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.760+0000 7f624e1ff700 1 --2- 172.21.15.31:0/4210677077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6248159500 0x7f62480b0ab0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.760+0000 7f624e1ff700 1 --2- 172.21.15.31:0/4210677077 >> v2:172.21.15.62:3300/0 conn(0x7f624815d2d0 0x7f62480cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.760+0000 7f624e1ff700 1 -- 172.21.15.31:0/4210677077 >> 172.21.15.31:0/4210677077 conn(0x7f624806ec90 msgr2=0x7f624806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.760+0000 7f624e1ff700 1 -- 172.21.15.31:0/4210677077 shutdown_connections 2023-10-06T05:24:12.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.760+0000 7f624e1ff700 1 -- 172.21.15.31:0/4210677077 wait complete. 2023-10-06T05:24:12.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.762+0000 7f624e1ff700 1 Processor -- start 2023-10-06T05:24:12.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.762+0000 7f624e1ff700 1 -- start start 2023-10-06T05:24:12.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f624e1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6248159500 0x7f6248159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f624e1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62481598d0 0x7f62480c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f624e1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f624815d2d0 0x7f62481579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f624e1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62480c5b60 con 0x7f624815d2d0 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f624e1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624815d020 con 0x7f6248159500 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f624e1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624815caa0 con 0x7f62481598d0 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f6247fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f624815d2d0 0x7f62481579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f62477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6248159500 0x7f6248159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f6247fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f624815d2d0 0x7f62481579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49410/0 (socket says 172.21.15.31:49410) 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f6246ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62481598d0 0x7f62480c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.763+0000 7f6247fff700 1 -- 172.21.15.31:0/866304457 learned_addr learned my addr 172.21.15.31:0/866304457 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f6247fff700 1 -- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62481598d0 msgr2=0x7f62480c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f6247fff700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62481598d0 0x7f62480c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f6247fff700 1 -- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6248159500 msgr2=0x7f6248159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:12.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f6247fff700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6248159500 0x7f6248159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:12.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f6247fff700 1 -- 172.21.15.31:0/866304457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62380094f0 con 0x7f624815d2d0 2023-10-06T05:24:12.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f6247fff700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f624815d2d0 0x7f62481579c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f624000c8e0 tx=0x7f6240001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f622f7fe700 1 -- 172.21.15.31:0/866304457 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240012630 con 0x7f624815d2d0 2023-10-06T05:24:12.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f622f7fe700 1 -- 172.21.15.31:0/866304457 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6240012790 con 0x7f624815d2d0 2023-10-06T05:24:12.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f622f7fe700 1 -- 172.21.15.31:0/866304457 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240012b50 con 0x7f624815d2d0 2023-10-06T05:24:12.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f624e1ff700 1 -- 172.21.15.31:0/866304457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624815cd40 con 0x7f624815d2d0 2023-10-06T05:24:12.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.764+0000 7f624e1ff700 1 -- 172.21.15.31:0/866304457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62481697e0 con 0x7f624815d2d0 2023-10-06T05:24:12.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.765+0000 7f624e1ff700 1 -- 172.21.15.31:0/866304457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624804c380 con 0x7f624815d2d0 2023-10-06T05:24:12.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.766+0000 7f622f7fe700 1 -- 172.21.15.31:0/866304457 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f624001a3a0 con 0x7f624815d2d0 2023-10-06T05:24:12.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.767+0000 7f622f7fe700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f62340674d0 0x7f6234069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:12.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.767+0000 7f622f7fe700 1 -- 172.21.15.31:0/866304457 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f62400852a0 con 0x7f624815d2d0 2023-10-06T05:24:12.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.768+0000 7f62477fe700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f62340674d0 0x7f6234069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:12.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.768+0000 7f62477fe700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f62340674d0 0x7f6234069980 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f624815d200 tx=0x7f6238017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:12.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:12.771+0000 7f622f7fe700 1 -- 172.21.15.31:0/866304457 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6240055570 con 0x7f624815d2d0 2023-10-06T05:24:13.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.019+0000 7f624e1ff700 1 -- 172.21.15.31:0/866304457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62480c5ca0 con 0x7f624815d2d0 2023-10-06T05:24:13.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.019+0000 7f622f7fe700 1 -- 172.21.15.31:0/866304457 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f6240055100 con 0x7f624815d2d0 2023-10-06T05:24:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:13.021 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":84,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3816040413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3816040413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3816040413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1295480750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1295480750},{"type":"v1","addr":"172.21.15.31:6834","nonce":1295480750}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2807869230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2807869230},{"type":"v1","addr":"172.21.15.62:6834","nonce":2807869230}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:06.204405+0000","modified":"2023-10-06T05:24:10.122253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7995":{"gid":7995,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1928465499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1928465499},{"type":"v1","addr":"172.21.15.62:6835","nonce":1928465499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:13.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.021+0000 7f624e1ff700 1 -- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f62340674d0 msgr2=0x7f6234069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.021+0000 7f624e1ff700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f62340674d0 0x7f6234069980 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f624815d200 tx=0x7f6238017000 comp rx=0 tx=0).stop 2023-10-06T05:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.021+0000 7f624e1ff700 1 -- 172.21.15.31:0/866304457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f624815d2d0 msgr2=0x7f62481579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.021+0000 7f624e1ff700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f624815d2d0 0x7f62481579c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f624000c8e0 tx=0x7f6240001710 comp rx=0 tx=0).stop 2023-10-06T05:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.022+0000 7f624e1ff700 1 -- 172.21.15.31:0/866304457 shutdown_connections 2023-10-06T05:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.022+0000 7f624e1ff700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f62340674d0 0x7f6234069980 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.022+0000 7f624e1ff700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f624815d2d0 0x7f62481579c0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.022+0000 7f624e1ff700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6248159500 0x7f6248159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.022+0000 7f624e1ff700 1 --2- 172.21.15.31:0/866304457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62481598d0 0x7f62480c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.022+0000 7f624e1ff700 1 -- 172.21.15.31:0/866304457 >> 172.21.15.31:0/866304457 conn(0x7f624806ec90 msgr2=0x7f62480cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.022+0000 7f624e1ff700 1 -- 172.21.15.31:0/866304457 shutdown_connections 2023-10-06T05:24:13.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.022+0000 7f624e1ff700 1 -- 172.21.15.31:0/866304457 wait complete. 2023-10-06T05:24:13.025 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 84 2023-10-06T05:24:13.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:24:06.204405+0000', 'modified': '2023-10-06T05:24:10.122253+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7995}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7995': {'gid': 7995, 'name': 'b', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/1928465499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 1928465499}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 1928465499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:24:13.444 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:24:13.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T05:24:13.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.641+0000 7f40eb3ff700 1 Processor -- start 2023-10-06T05:24:13.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.642+0000 7f40eb3ff700 1 -- start start 2023-10-06T05:24:13.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.643+0000 7f40eb3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40e4159500 0x7f40e40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.643+0000 7f40eb3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 0x7f40e40aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.643+0000 7f40eb3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e40c3f40 con 0x7f40e415d2d0 2023-10-06T05:24:13.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.643+0000 7f40eb3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e40b2860 con 0x7f40e4159500 2023-10-06T05:24:13.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.643+0000 7f40eb3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e415d7d0 con 0x7f40e40cd510 2023-10-06T05:24:13.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.643+0000 7f40e919b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40e4159500 0x7f40e40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.643+0000 7f40e919b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40e4159500 0x7f40e40b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49412/0 (socket says 172.21.15.31:49412) 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.643+0000 7f40e919b700 1 -- 172.21.15.31:0/396398073 learned_addr learned my addr 172.21.15.31:0/396398073 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.644+0000 7f40e919b700 1 -- 172.21.15.31:0/396398073 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 msgr2=0x7f40e40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.644+0000 7f40e999c700 1 --2- 172.21.15.31:0/396398073 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 0x7f40e40aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.644+0000 7f40e919b700 1 --2- 172.21.15.31:0/396398073 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 0x7f40e40aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.644+0000 7f40e919b700 1 -- 172.21.15.31:0/396398073 >> v1:172.21.15.31:6789/0 conn(0x7f40e415d2d0 legacy=0x7f40e40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.644+0000 7f40e919b700 1 -- 172.21.15.31:0/396398073 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e40cb000 con 0x7f40e4159500 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.644+0000 7f40e919b700 1 --2- 172.21.15.31:0/396398073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 0x7f40e40b0ab0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f40e00099b0 tx=0x7f40e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d473bfe8a77a20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.644+0000 7f40dbfff700 1 -- 172.21.15.31:0/396398073 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000b860 con 0x7f40e4159500 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.644+0000 7f40dbfff700 1 -- 172.21.15.31:0/396398073 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e000b9c0 con 0x7f40e4159500 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.644+0000 7f40dbfff700 1 -- 172.21.15.31:0/396398073 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000bd60 con 0x7f40e4159500 2023-10-06T05:24:13.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.645+0000 7f40eb3ff700 1 -- 172.21.15.31:0/396398073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 msgr2=0x7f40e40b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.645+0000 7f40eb3ff700 1 --2- 172.21.15.31:0/396398073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 0x7f40e40b0ab0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f40e00099b0 tx=0x7f40e0009250 comp rx=0 tx=0).stop 2023-10-06T05:24:13.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.645+0000 7f40eb3ff700 1 -- 172.21.15.31:0/396398073 shutdown_connections 2023-10-06T05:24:13.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.645+0000 7f40eb3ff700 1 --2- 172.21.15.31:0/396398073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 0x7f40e40b0ab0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.645+0000 7f40eb3ff700 1 --2- 172.21.15.31:0/396398073 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 0x7f40e40aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.645+0000 7f40eb3ff700 1 -- 172.21.15.31:0/396398073 >> 172.21.15.31:0/396398073 conn(0x7f40e406ec90 msgr2=0x7f40e406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.645+0000 7f40eb3ff700 1 -- 172.21.15.31:0/396398073 shutdown_connections 2023-10-06T05:24:13.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.645+0000 7f40eb3ff700 1 -- 172.21.15.31:0/396398073 wait complete. 2023-10-06T05:24:13.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.646+0000 7f40eb3ff700 1 Processor -- start 2023-10-06T05:24:13.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.646+0000 7f40eb3ff700 1 -- start start 2023-10-06T05:24:13.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.647+0000 7f40eb3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 0x7f40e40c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.647+0000 7f40eb3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 0x7f40e40b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.647+0000 7f40eb3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40e415d2d0 0x7f40e40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.647+0000 7f40eb3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e41513d0 con 0x7f40e4159500 2023-10-06T05:24:13.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.647+0000 7f40eb3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40c53e0 con 0x7f40e415d2d0 2023-10-06T05:24:13.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.647+0000 7f40eb3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40c56c0 con 0x7f40e40cd510 2023-10-06T05:24:13.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 0x7f40e40b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 0x7f40e40b5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49422/0 (socket says 172.21.15.31:49422) 2023-10-06T05:24:13.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e899a700 1 -- 172.21.15.31:0/4007649881 learned_addr learned my addr 172.21.15.31:0/4007649881 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e919b700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 0x7f40e40c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e999c700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40e415d2d0 0x7f40e40b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e899a700 1 -- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 msgr2=0x7f40e40c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e899a700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 0x7f40e40c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e899a700 1 -- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40e415d2d0 msgr2=0x7f40e40b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e899a700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40e415d2d0 0x7f40e40b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e899a700 1 -- 172.21.15.31:0/4007649881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e00094f0 con 0x7f40e4159500 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e919b700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 0x7f40e40c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.648+0000 7f40e899a700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 0x7f40e40b5020 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c540 tx=0x7f40d4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.649+0000 7f40d97fa700 1 -- 172.21.15.31:0/4007649881 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400f470 con 0x7f40e4159500 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.649+0000 7f40d97fa700 1 -- 172.21.15.31:0/4007649881 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d400f5d0 con 0x7f40e4159500 2023-10-06T05:24:13.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.649+0000 7f40d97fa700 1 -- 172.21.15.31:0/4007649881 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400f990 con 0x7f40e4159500 2023-10-06T05:24:13.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.649+0000 7f40eb3ff700 1 -- 172.21.15.31:0/4007649881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e4151670 con 0x7f40e4159500 2023-10-06T05:24:13.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.649+0000 7f40eb3ff700 1 -- 172.21.15.31:0/4007649881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e40c59a0 con 0x7f40e4159500 2023-10-06T05:24:13.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.651+0000 7f40d97fa700 1 -- 172.21.15.31:0/4007649881 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f40d400fd40 con 0x7f40e4159500 2023-10-06T05:24:13.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.652+0000 7f40d97fa700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40c80674d0 0x7f40c8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.652+0000 7f40d97fa700 1 -- 172.21.15.31:0/4007649881 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f40d4084690 con 0x7f40e4159500 2023-10-06T05:24:13.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.652+0000 7f40e919b700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40c80674d0 0x7f40c8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.653+0000 7f40e919b700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40c80674d0 0x7f40c8069980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40e00099b0 tx=0x7f40e0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.653+0000 7f40f9ab2b80 1 Processor -- start 2023-10-06T05:24:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.654+0000 7f40f9ab2b80 1 -- start start 2023-10-06T05:24:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.654+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b10f450 0x55d86b10f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.654+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b117f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.654+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b1184d0 0x55d86b118940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86afe2310 con 0x55d86b0f4f40 2023-10-06T05:24:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86b01f130 con 0x55d86b10f450 2023-10-06T05:24:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d86b01caf0 con 0x55d86b1184d0 2023-10-06T05:24:13.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b117f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b117f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49436/0 (socket says 172.21.15.31:49436) 2023-10-06T05:24:13.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e899a700 1 -- 172.21.15.31:0/2113502596 learned_addr learned my addr 172.21.15.31:0/2113502596 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:13.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e919b700 1 --2- 172.21.15.31:0/2113502596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b10f450 0x55d86b10f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e999c700 1 --2- 172.21.15.31:0/2113502596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b1184d0 0x55d86b118940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e899a700 1 -- 172.21.15.31:0/2113502596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b1184d0 msgr2=0x55d86b118940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e899a700 1 --2- 172.21.15.31:0/2113502596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b1184d0 0x55d86b118940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e899a700 1 -- 172.21.15.31:0/2113502596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b10f450 msgr2=0x55d86b10f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e899a700 1 --2- 172.21.15.31:0/2113502596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b10f450 0x55d86b10f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e899a700 1 -- 172.21.15.31:0/2113502596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86ae1e790 con 0x55d86b0f4f40 2023-10-06T05:24:13.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e919b700 1 --2- 172.21.15.31:0/2113502596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b10f450 0x55d86b10f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:13.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.655+0000 7f40e899a700 1 --2- 172.21.15.31:0/2113502596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b117f90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f40d4051090 tx=0x7f40d4054960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c60060c76dbdc86 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40ce7fc700 1 -- 172.21.15.31:0/2113502596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40820e0 con 0x55d86b0f4f40 2023-10-06T05:24:13.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40ce7fc700 1 -- 172.21.15.31:0/2113502596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4054c10 con 0x55d86b0f4f40 2023-10-06T05:24:13.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40ce7fc700 1 -- 172.21.15.31:0/2113502596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4057de0 con 0x55d86b0f4f40 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2113502596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 msgr2=0x55d86b117f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/2113502596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b117f90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f40d4051090 tx=0x7f40d4054960 comp rx=0 tx=0).stop 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2113502596 shutdown_connections 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/2113502596 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b1184d0 0x55d86b118940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/2113502596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b10f450 0x55d86b10f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/2113502596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b117f90 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2113502596 >> 172.21.15.31:0/2113502596 conn(0x55d86b112880 msgr2=0x55d86b112c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2113502596 shutdown_connections 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.656+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2113502596 wait complete. 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.657+0000 7f40f9ab2b80 1 -- start start 2023-10-06T05:24:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.657+0000 7f40f9ab2b80 1 Processor -- start 2023-10-06T05:24:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.658+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b0f4f40 0x55d86b0e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.658+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b10f450 0x55d86b0ea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.658+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b1184d0 0x55d86b0ed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.658+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86b0188e0 con 0x55d86b1184d0 2023-10-06T05:24:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.658+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86afc4980 con 0x55d86b0f4f40 2023-10-06T05:24:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.658+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d86afc4620 con 0x55d86b10f450 2023-10-06T05:24:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.658+0000 7f40f9ab2b80 10 client.0 _ll_register_callbacks cb 0x55d86ae27a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b1184d0 0x55d86b0ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b1184d0 0x55d86b0ed750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49446/0 (socket says 172.21.15.31:49446) 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e919b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b0f4f40 0x55d86b0e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e999c700 1 -- 172.21.15.31:0/1024322244 learned_addr learned my addr 172.21.15.31:0/1024322244 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e899a700 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b10f450 0x55d86b0ea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e999c700 1 -- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b10f450 msgr2=0x55d86b0ea4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e999c700 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b10f450 0x55d86b0ea4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e999c700 1 -- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b0f4f40 msgr2=0x55d86b0e93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e999c700 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b0f4f40 0x55d86b0e93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e999c700 1 -- 172.21.15.31:0/1024322244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e40c59a0 con 0x55d86b1184d0 2023-10-06T05:24:13.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40e999c700 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b1184d0 0x55d86b0ed750 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f40dc010f30 tx=0x7f40dc0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3f09b85634f7bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.659+0000 7f40c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:24:13.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40c37fe700 1 -- 172.21.15.31:0/1024322244 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00ec20 con 0x55d86b1184d0 2023-10-06T05:24:13.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40c37fe700 1 -- 172.21.15.31:0/1024322244 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f40dc00ed80 con 0x55d86b1184d0 2023-10-06T05:24:13.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/1024322244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d86b0bcd60 con 0x55d86b1184d0 2023-10-06T05:24:13.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40c37fe700 1 -- 172.21.15.31:0/1024322244 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40dc018600 con 0x55d86b1184d0 2023-10-06T05:24:13.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40f9ab2b80 10 client.8145 fetch_fsmap learned FSMap version 84 2023-10-06T05:24:13.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/1024322244 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x55d86af0b0a0 con 0x55d86b1184d0 2023-10-06T05:24:13.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40c37fe700 1 -- 172.21.15.31:0/1024322244 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d86b0bcd60 con 0x55d86b1184d0 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40c37fe700 1 -- 172.21.15.31:0/1024322244 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 84) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f40dc018760 con 0x55d86b1184d0 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40f9ab2b80 10 client.8145 fetch_fsmap finished waiting for FSMap version 84 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40f9ab2b80 10 client.8145 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40f9ab2b80 20 client.8145 populate_metadata read hostname 'smithi031' 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:6833/1928465499,v1:172.21.15.62:6835/1928465499] conn(0x55d86b0ee950 0x55d86b0eed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40f9ab2b80 4 client.8145 mds_command: new command op to 7995 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.660+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/1024322244 --> [v2:172.21.15.62:6833/1928465499,v1:172.21.15.62:6835/1928465499] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d86b099b30 con 0x55d86b0ee950 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.661+0000 7f40e919b700 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:6833/1928465499,v1:172.21.15.62:6835/1928465499] conn(0x55d86b0ee950 0x55d86b0eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.661+0000 7f40e919b700 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:6833/1928465499,v1:172.21.15.62:6835/1928465499] conn(0x55d86b0ee950 0x55d86b0eed20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4f6610842ea512 server_cookie=4d0a1e3b72668552 in_seq=0 out_seq=0 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.661+0000 7f40c37fe700 10 client.8145 ms_handle_connect on v2:172.21.15.62:6833/1928465499 2023-10-06T05:24:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.662+0000 7f40c37fe700 1 -- 172.21.15.31:0/1024322244 <== mds.0 v2:172.21.15.62:6833/1928465499 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d86b099b30 con 0x55d86b0ee950 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.662+0000 7f40c37fe700 10 client.8145 handle_command_reply: tid=0 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 client.8145 shutdown 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/1024322244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b1184d0 msgr2=0x55d86b0ed750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b1184d0 0x55d86b0ed750 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f40dc010f30 tx=0x7f40dc0077b0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/1024322244 shutdown_connections 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b1184d0 0x55d86b0ed750 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:6833/1928465499,v1:172.21.15.62:6835/1928465499] conn(0x55d86b0ee950 0x55d86b0eed20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b0f4f40 0x55d86b0e93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/1024322244 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b10f450 0x55d86b0ea4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/1024322244 >> 172.21.15.31:0/1024322244 conn(0x55d86b112880 msgr2=0x55d86b0c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40c37fe700 0 client.8145 ms_handle_reset on v2:172.21.15.62:6833/1928465499 2023-10-06T05:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/1024322244 shutdown_connections 2023-10-06T05:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/1024322244 wait complete. 2023-10-06T05:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.663+0000 7f40f9ab2b80 20 client.8145 trim_cache size 0 max 16384 2023-10-06T05:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.681+0000 7f40f9ab2b80 1 Processor -- start 2023-10-06T05:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.681+0000 7f40f9ab2b80 1 -- start start 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b0f4f40 0x55d86b0c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b10f450 0x55d86b0bea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b1184d0 0x55d86b127c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86aeb01e0 con 0x55d86b10f450 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86b0bcd00 con 0x55d86b0f4f40 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d86b0e8d30 con 0x55d86b1184d0 2023-10-06T05:24:13.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40e899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b10f450 0x55d86b0bea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40e899a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b10f450 0x55d86b0bea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49462/0 (socket says 172.21.15.31:49462) 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40e899a700 1 -- 172.21.15.31:0/3025185013 learned_addr learned my addr 172.21.15.31:0/3025185013 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40e919b700 1 --2- 172.21.15.31:0/3025185013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b0f4f40 0x55d86b0c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40e999c700 1 --2- 172.21.15.31:0/3025185013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b1184d0 0x55d86b127c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40e899a700 1 -- 172.21.15.31:0/3025185013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b1184d0 msgr2=0x55d86b127c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40e899a700 1 --2- 172.21.15.31:0/3025185013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b1184d0 0x55d86b127c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40e899a700 1 -- 172.21.15.31:0/3025185013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b0f4f40 msgr2=0x55d86b0c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40e899a700 1 --2- 172.21.15.31:0/3025185013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b0f4f40 0x55d86b0c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.682+0000 7f40e899a700 1 -- 172.21.15.31:0/3025185013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86afe73c0 con 0x55d86b10f450 2023-10-06T05:24:13.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40e899a700 1 --2- 172.21.15.31:0/3025185013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b10f450 0x55d86b0bea40 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f40d4052ee0 tx=0x7f40d406a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d219c148695d6075 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40ceffd700 1 -- 172.21.15.31:0/3025185013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4057de0 con 0x55d86b10f450 2023-10-06T05:24:13.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40ceffd700 1 -- 172.21.15.31:0/3025185013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4054290 con 0x55d86b10f450 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40ceffd700 1 -- 172.21.15.31:0/3025185013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4052f30 con 0x55d86b10f450 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/3025185013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b10f450 msgr2=0x55d86b0bea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/3025185013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b10f450 0x55d86b0bea40 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f40d4052ee0 tx=0x7f40d406a000 comp rx=0 tx=0).stop 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/3025185013 shutdown_connections 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/3025185013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b1184d0 0x55d86b127c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/3025185013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b0f4f40 0x55d86b0c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/3025185013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b10f450 0x55d86b0bea40 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/3025185013 >> 172.21.15.31:0/3025185013 conn(0x55d86b112880 msgr2=0x55d86b0e8150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/3025185013 shutdown_connections 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.683+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/3025185013 wait complete. 2023-10-06T05:24:13.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.684+0000 7f40f9ab2b80 1 -- start start 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.684+0000 7f40f9ab2b80 1 Processor -- start 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.685+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b16e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.685+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b10f450 0x55d86b114c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.685+0000 7f40f9ab2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b1184d0 0x55d86b1151a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.685+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86b0c5150 con 0x55d86b0f4f40 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.685+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86b115c90 con 0x55d86b1184d0 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.685+0000 7f40f9ab2b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d86b0e9f70 con 0x55d86b10f450 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.685+0000 7f40f9ab2b80 10 client.0 _ll_register_callbacks cb 0x55d86ae27a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b16e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b16e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49470/0 (socket says 172.21.15.31:49470) 2023-10-06T05:24:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e919b700 1 -- 172.21.15.31:0/2390945916 learned_addr learned my addr 172.21.15.31:0/2390945916 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e899a700 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b10f450 0x55d86b114c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e999c700 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b1184d0 0x55d86b1151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e919b700 1 -- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b10f450 msgr2=0x55d86b114c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e919b700 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b10f450 0x55d86b114c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e919b700 1 -- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b1184d0 msgr2=0x55d86b1151a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e919b700 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b1184d0 0x55d86b1151a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e919b700 1 -- 172.21.15.31:0/2390945916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d86ae1e790 con 0x55d86b0f4f40 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40e919b700 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b16e6f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f40e000b0b0 tx=0x7f40e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1596c17ce2973cee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.686+0000 7f40c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40c37fe700 1 -- 172.21.15.31:0/2390945916 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0038070 con 0x55d86b0f4f40 2023-10-06T05:24:13.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40c37fe700 1 -- 172.21.15.31:0/2390945916 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f40e0033490 con 0x55d86b0f4f40 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2390945916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40d4054c10 con 0x55d86b0f4f40 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40c37fe700 1 -- 172.21.15.31:0/2390945916 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40e002b040 con 0x55d86b0f4f40 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40f9ab2b80 10 client.8151 fetch_fsmap learned FSMap version 84 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2390945916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x55d86ae7f1d0 con 0x55d86b0f4f40 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40c37fe700 1 -- 172.21.15.31:0/2390945916 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4054c10 con 0x55d86b0f4f40 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40c37fe700 1 -- 172.21.15.31:0/2390945916 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 84) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f40e003b020 con 0x55d86b0f4f40 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40f9ab2b80 10 client.8151 fetch_fsmap finished waiting for FSMap version 84 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40f9ab2b80 10 client.8151 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40f9ab2b80 20 client.8151 populate_metadata read hostname 'smithi031' 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:6833/1928465499,v1:172.21.15.62:6835/1928465499] conn(0x55d86b0c8800 0x55d86b0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40f9ab2b80 4 client.8151 mds_command: new command op to 7995 tid=0[{"prefix": "status"}] 2023-10-06T05:24:13.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.687+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2390945916 --> [v2:172.21.15.62:6833/1928465499,v1:172.21.15.62:6835/1928465499] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d86afbb8d0 con 0x55d86b0c8800 2023-10-06T05:24:13.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.688+0000 7f40e899a700 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:6833/1928465499,v1:172.21.15.62:6835/1928465499] conn(0x55d86b0c8800 0x55d86b0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.688+0000 7f40e899a700 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:6833/1928465499,v1:172.21.15.62:6835/1928465499] conn(0x55d86b0c8800 0x55d86b0c8bd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=828ce62256803d48 server_cookie=63921e1b0da6332f in_seq=0 out_seq=0 2023-10-06T05:24:13.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.688+0000 7f40c37fe700 10 client.8151 ms_handle_connect on v2:172.21.15.62:6833/1928465499 2023-10-06T05:24:13.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.688+0000 7f40c37fe700 1 -- 172.21.15.31:0/2390945916 <== mds.0 v2:172.21.15.62:6833/1928465499 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55d86afbb8d0 con 0x55d86b0c8800 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.688+0000 7f40c37fe700 10 client.8151 handle_command_reply: tid=0 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 client.8151 shutdown 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2390945916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 msgr2=0x55d86b16e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b16e6f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f40e000b0b0 tx=0x7f40e000b040 comp rx=0 tx=0).stop 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2390945916 shutdown_connections 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:6833/1928465499,v1:172.21.15.62:6835/1928465499] conn(0x55d86b0c8800 0x55d86b0c8bd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b1184d0 0x55d86b1151a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d86b0f4f40 0x55d86b16e6f0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 --2- 172.21.15.31:0/2390945916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b10f450 0x55d86b114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40c37fe700 0 client.8151 ms_handle_reset on v2:172.21.15.62:6833/1928465499 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2390945916 >> 172.21.15.31:0/2390945916 conn(0x55d86b112880 msgr2=0x55d86b10fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2390945916 shutdown_connections 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 1 -- 172.21.15.31:0/2390945916 wait complete. 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.689+0000 7f40f9ab2b80 20 client.8151 trim_cache size 0 max 16384 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 7995, 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:24:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:24:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5817913880000001, 2023-10-06T05:24:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 84, 2023-10-06T05:24:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 127, 2023-10-06T05:24:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 127, 2023-10-06T05:24:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 13.64064675 2023-10-06T05:24:13.698 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:24:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 -- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40c80674d0 msgr2=0x7f40c8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40c80674d0 0x7f40c8069980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40e00099b0 tx=0x7f40e0017000 comp rx=0 tx=0).stop 2023-10-06T05:24:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 -- 172.21.15.31:0/4007649881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 msgr2=0x7f40e40b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 0x7f40e40b5020 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c540 tx=0x7f40d4007f60 comp rx=0 tx=0).stop 2023-10-06T05:24:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 -- 172.21.15.31:0/4007649881 shutdown_connections 2023-10-06T05:24:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f40c80674d0 0x7f40c8069980 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40e415d2d0 0x7f40e40b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40e40cd510 0x7f40e40c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 --2- 172.21.15.31:0/4007649881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4159500 0x7f40e40b5020 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 -- 172.21.15.31:0/4007649881 >> 172.21.15.31:0/4007649881 conn(0x7f40e406ec90 msgr2=0x7f40e4158b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 -- 172.21.15.31:0/4007649881 shutdown_connections 2023-10-06T05:24:13.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.690+0000 7f40eb3ff700 1 -- 172.21.15.31:0/4007649881 wait complete. 2023-10-06T05:24:13.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:24:13.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.904+0000 7f4257fff700 1 Processor -- start 2023-10-06T05:24:13.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.904+0000 7f4257fff700 1 -- start start 2023-10-06T05:24:13.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.905+0000 7f4257fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f425013fb00 0x7f425006c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.905+0000 7f4257fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f425013fed0 0x7f425006e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.905+0000 7f4257fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425006be70 con 0x7f425006ea90 2023-10-06T05:24:13.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.905+0000 7f4257fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425006ef30 con 0x7f425013fed0 2023-10-06T05:24:13.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.905+0000 7f4257fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425006f130 con 0x7f425013fb00 2023-10-06T05:24:13.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.906+0000 7f425559a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f425013fed0 0x7f425006e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.906+0000 7f425559a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f425013fed0 0x7f425006e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49484/0 (socket says 172.21.15.31:49484) 2023-10-06T05:24:13.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.906+0000 7f425559a700 1 -- 172.21.15.31:0/2223900500 learned_addr learned my addr 172.21.15.31:0/2223900500 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:13.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.906+0000 7f4255d9b700 1 --2- 172.21.15.31:0/2223900500 >> v2:172.21.15.62:3300/0 conn(0x7f425013fb00 0x7f425006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.906+0000 7f4254d99700 1 -- 172.21.15.31:0/2223900500 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293971374 0 0) 0x7f425006be70 con 0x7f425006ea90 2023-10-06T05:24:13.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.906+0000 7f4254d99700 1 -- 172.21.15.31:0/2223900500 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f423c003570 con 0x7f425006ea90 2023-10-06T05:24:13.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.906+0000 7f425559a700 1 -- 172.21.15.31:0/2223900500 >> v2:172.21.15.62:3300/0 conn(0x7f425013fb00 msgr2=0x7f425006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:13.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.906+0000 7f425559a700 1 --2- 172.21.15.31:0/2223900500 >> v2:172.21.15.62:3300/0 conn(0x7f425013fb00 0x7f425006c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.906+0000 7f425559a700 1 -- 172.21.15.31:0/2223900500 >> v1:172.21.15.31:6789/0 conn(0x7f425006ea90 legacy=0x7f42500cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.906+0000 7f425559a700 1 -- 172.21.15.31:0/2223900500 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425006f410 con 0x7f425013fed0 2023-10-06T05:24:13.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.907+0000 7f425559a700 1 --2- 172.21.15.31:0/2223900500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 0x7f425006e550 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f42500711c0 tx=0x7f4240009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25beb3227ecb197d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.907+0000 7f4254d99700 1 -- 172.21.15.31:0/2223900500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424000b640 con 0x7f425013fed0 2023-10-06T05:24:13.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.907+0000 7f4254d99700 1 -- 172.21.15.31:0/2223900500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424000b7a0 con 0x7f425013fed0 2023-10-06T05:24:13.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.907+0000 7f4254d99700 1 -- 172.21.15.31:0/2223900500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424000bb20 con 0x7f425013fed0 2023-10-06T05:24:13.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.907+0000 7f4257fff700 1 -- 172.21.15.31:0/2223900500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 msgr2=0x7f425006e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.907+0000 7f4257fff700 1 --2- 172.21.15.31:0/2223900500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 0x7f425006e550 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f42500711c0 tx=0x7f4240009250 comp rx=0 tx=0).stop 2023-10-06T05:24:13.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.908+0000 7f4257fff700 1 -- 172.21.15.31:0/2223900500 shutdown_connections 2023-10-06T05:24:13.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.908+0000 7f4257fff700 1 --2- 172.21.15.31:0/2223900500 >> v2:172.21.15.62:3300/0 conn(0x7f425013fb00 0x7f425006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.908+0000 7f4257fff700 1 --2- 172.21.15.31:0/2223900500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 0x7f425006e550 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.908+0000 7f4257fff700 1 -- 172.21.15.31:0/2223900500 >> 172.21.15.31:0/2223900500 conn(0x7f42500cdcb0 msgr2=0x7f42500c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:13.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.908+0000 7f4257fff700 1 -- 172.21.15.31:0/2223900500 shutdown_connections 2023-10-06T05:24:13.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.908+0000 7f4257fff700 1 -- 172.21.15.31:0/2223900500 wait complete. 2023-10-06T05:24:13.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.909+0000 7f4257fff700 1 Processor -- start 2023-10-06T05:24:13.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.909+0000 7f4257fff700 1 -- start start 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.910+0000 7f4257fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f425006ea90 0x7f4250159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.910+0000 7f4257fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f425013fb00 0x7f4250158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.910+0000 7f4257fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 0x7f425016a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.910+0000 7f4257fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425016b970 con 0x7f425013fed0 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.910+0000 7f4257fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425015e9a0 con 0x7f425006ea90 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.910+0000 7f4257fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425016ad60 con 0x7f425013fb00 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.910+0000 7f425659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 0x7f425016a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.911+0000 7f425659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 0x7f425016a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49488/0 (socket says 172.21.15.31:49488) 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.911+0000 7f425659c700 1 -- 172.21.15.31:0/2788644018 learned_addr learned my addr 172.21.15.31:0/2788644018 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.911+0000 7f4255d9b700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f425006ea90 0x7f4250159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.911+0000 7f425559a700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f425013fb00 0x7f4250158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.911+0000 7f425659c700 1 -- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f425013fb00 msgr2=0x7f4250158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.911+0000 7f425659c700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f425013fb00 0x7f4250158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.911+0000 7f425659c700 1 -- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f425006ea90 msgr2=0x7f4250159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.911+0000 7f425659c700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f425006ea90 0x7f4250159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.911+0000 7f425659c700 1 -- 172.21.15.31:0/2788644018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4240009490 con 0x7f425013fed0 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.911+0000 7f425659c700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 0x7f425016a1f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f424800c860 tx=0x7f4248002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.912+0000 7f4245ffb700 1 -- 172.21.15.31:0/2788644018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248012630 con 0x7f425013fed0 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.912+0000 7f4245ffb700 1 -- 172.21.15.31:0/2788644018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42480127b0 con 0x7f425013fed0 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.912+0000 7f4245ffb700 1 -- 172.21.15.31:0/2788644018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248012b70 con 0x7f425013fed0 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.912+0000 7f4257fff700 1 -- 172.21.15.31:0/2788644018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425016b040 con 0x7f425013fed0 2023-10-06T05:24:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.912+0000 7f4257fff700 1 -- 172.21.15.31:0/2788644018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425016b600 con 0x7f425013fed0 2023-10-06T05:24:13.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.913+0000 7f4257fff700 1 -- 172.21.15.31:0/2788644018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425016bae0 con 0x7f425013fed0 2023-10-06T05:24:13.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.914+0000 7f4245ffb700 1 -- 172.21.15.31:0/2788644018 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f424801b400 con 0x7f425013fed0 2023-10-06T05:24:13.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.914+0000 7f4245ffb700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f42340674d0 0x7f4234069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:13.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.915+0000 7f4245ffb700 1 -- 172.21.15.31:0/2788644018 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f4248085320 con 0x7f425013fed0 2023-10-06T05:24:13.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.915+0000 7f4255d9b700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f42340674d0 0x7f4234069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:13.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.916+0000 7f4255d9b700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f42340674d0 0x7f4234069980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f424c009db0 tx=0x7f424c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:13.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:13.919+0000 7f4245ffb700 1 -- 172.21.15.31:0/2788644018 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4248051b60 con 0x7f425013fed0 2023-10-06T05:24:14.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.165+0000 7f4257fff700 1 -- 172.21.15.31:0/2788644018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f425004c380 con 0x7f425013fed0 2023-10-06T05:24:14.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.165+0000 7f4245ffb700 1 -- 172.21.15.31:0/2788644018 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4248055180 con 0x7f425013fed0 2023-10-06T05:24:14.167 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:24:14.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 -- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f42340674d0 msgr2=0x7f4234069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:14.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f42340674d0 0x7f4234069980 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f424c009db0 tx=0x7f424c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 -- 172.21.15.31:0/2788644018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 msgr2=0x7f425016a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:14.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 0x7f425016a1f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f424800c860 tx=0x7f4248002700 comp rx=0 tx=0).stop 2023-10-06T05:24:14.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 -- 172.21.15.31:0/2788644018 shutdown_connections 2023-10-06T05:24:14.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f42340674d0 0x7f4234069980 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f425013fed0 0x7f425016a1f0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f425006ea90 0x7f4250159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 --2- 172.21.15.31:0/2788644018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f425013fb00 0x7f4250158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:14.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 -- 172.21.15.31:0/2788644018 >> 172.21.15.31:0/2788644018 conn(0x7f42500cdcb0 msgr2=0x7f42500b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:14.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 -- 172.21.15.31:0/2788644018 shutdown_connections 2023-10-06T05:24:14.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:14.168+0000 7f4257fff700 1 -- 172.21.15.31:0/2788644018 wait complete. 2023-10-06T05:24:14.187 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:24:14.187 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:24:14.187 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:24:14.187 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:24:14.187 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:14.187 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:24:14.187 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:24:14.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:24:14.188 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:14.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:14.188 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:24:14.236 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:24:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:24:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:24:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:24:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:14.237 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:24:14.237 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:24:14.237 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:24:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:24:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:24:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41860sec preferred_lft 41860sec 2023-10-06T05:24:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:24:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:14.238 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:24:14.238 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:24:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:24:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:24:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:14.239 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:24:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:24:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:14.240 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:24:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:24:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:14.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:14.241 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:24:14.309 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:24:14.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:14.312 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:24:14.361 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:24:14.361 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:24:14.361 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:24:14.361 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:24:14.362 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:24:14.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:14.363 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:24:14.363 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:24:14.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:24:14.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:24:14.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:24:14.363 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:24:14.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:14.536 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:24:14.583 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:24:14.583 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:24:14.583 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:14.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:14.583 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:24:14.695 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:14.695 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:14.743 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:14.743 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:14.743 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:14.789 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:24:14.789 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:24:14.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:14.930 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:14.930 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:24:14.957 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:14.957 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:24:14.957 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:24:14.957 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:24:14.957 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:24:14.963 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:24:14.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:14.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:15.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:24:15.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.228+0000 7ff387fff700 1 Processor -- start 2023-10-06T05:24:15.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.229+0000 7ff387fff700 1 -- start start 2023-10-06T05:24:15.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.230+0000 7ff387fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3800c7f30 0x7ff380069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.230+0000 7ff387fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800aee70 0x7ff38013f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.230+0000 7ff387fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3800ca5b0 con 0x7ff3800c7b60 2023-10-06T05:24:15.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.230+0000 7ff387fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3800af2f0 con 0x7ff3800c7f30 2023-10-06T05:24:15.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.230+0000 7ff387fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38006f740 con 0x7ff3800aee70 2023-10-06T05:24:15.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.230+0000 7ff38559a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3800c7f30 0x7ff380069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.230+0000 7ff38559a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3800c7f30 0x7ff380069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49492/0 (socket says 172.21.15.31:49492) 2023-10-06T05:24:15.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.230+0000 7ff38559a700 1 -- 172.21.15.31:0/159060386 learned_addr learned my addr 172.21.15.31:0/159060386 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:15.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.230+0000 7ff38659c700 1 --2- 172.21.15.31:0/159060386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800aee70 0x7ff38013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.231+0000 7ff38559a700 1 -- 172.21.15.31:0/159060386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800aee70 msgr2=0x7ff38013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:15.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.231+0000 7ff38559a700 1 --2- 172.21.15.31:0/159060386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800aee70 0x7ff38013f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.231+0000 7ff38559a700 1 -- 172.21.15.31:0/159060386 >> v1:172.21.15.62:6789/0 conn(0x7ff3800c7b60 legacy=0x7ff38006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.231+0000 7ff38559a700 1 -- 172.21.15.31:0/159060386 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3800c9c80 con 0x7ff3800c7f30 2023-10-06T05:24:15.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.231+0000 7ff38659c700 1 --2- 172.21.15.31:0/159060386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800aee70 0x7ff38013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:15.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.231+0000 7ff38559a700 1 --2- 172.21.15.31:0/159060386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800c7f30 0x7ff380069880 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff3700099b0 tx=0x7ff370009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39110396b3c20c7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.231+0000 7ff384d99700 1 -- 172.21.15.31:0/159060386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37000b860 con 0x7ff3800c7f30 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.231+0000 7ff384d99700 1 -- 172.21.15.31:0/159060386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37000b9c0 con 0x7ff3800c7f30 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.231+0000 7ff384d99700 1 -- 172.21.15.31:0/159060386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37000bd60 con 0x7ff3800c7f30 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.232+0000 7ff387fff700 1 -- 172.21.15.31:0/159060386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800c7f30 msgr2=0x7ff380069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.232+0000 7ff387fff700 1 --2- 172.21.15.31:0/159060386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800c7f30 0x7ff380069880 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff3700099b0 tx=0x7ff370009250 comp rx=0 tx=0).stop 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.232+0000 7ff387fff700 1 -- 172.21.15.31:0/159060386 shutdown_connections 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.232+0000 7ff387fff700 1 --2- 172.21.15.31:0/159060386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800aee70 0x7ff38013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.232+0000 7ff387fff700 1 --2- 172.21.15.31:0/159060386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800c7f30 0x7ff380069880 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.232+0000 7ff387fff700 1 -- 172.21.15.31:0/159060386 >> 172.21.15.31:0/159060386 conn(0x7ff3800cc0b0 msgr2=0x7ff3800cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.232+0000 7ff387fff700 1 -- 172.21.15.31:0/159060386 shutdown_connections 2023-10-06T05:24:15.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.232+0000 7ff387fff700 1 -- 172.21.15.31:0/159060386 wait complete. 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.234+0000 7ff387fff700 1 Processor -- start 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.234+0000 7ff387fff700 1 -- start start 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff387fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800aee70 0x7ff38016a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff387fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3800c7b60 0x7ff3800ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff387fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800c7f30 0x7ff3800cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff387fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38015a6f0 con 0x7ff3800aee70 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff387fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38015c6c0 con 0x7ff3800c7b60 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff387fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38015bb10 con 0x7ff3800c7f30 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff385d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800aee70 0x7ff38016a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff385d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800aee70 0x7ff38016a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49498/0 (socket says 172.21.15.31:49498) 2023-10-06T05:24:15.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff385d9b700 1 -- 172.21.15.31:0/931942933 learned_addr learned my addr 172.21.15.31:0/931942933 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff385d9b700 1 -- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800c7f30 msgr2=0x7ff3800cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff38659c700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800c7f30 0x7ff3800cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff38559a700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3800c7b60 0x7ff3800ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff385d9b700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800c7f30 0x7ff3800cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff385d9b700 1 -- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3800c7b60 msgr2=0x7ff3800ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff385d9b700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3800c7b60 0x7ff3800ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.235+0000 7ff385d9b700 1 -- 172.21.15.31:0/931942933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3700094f0 con 0x7ff3800aee70 2023-10-06T05:24:15.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.236+0000 7ff38559a700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3800c7b60 0x7ff3800ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:15.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.236+0000 7ff38659c700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800c7f30 0x7ff3800cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:15.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.236+0000 7ff385d9b700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800aee70 0x7ff38016a3e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00c7f0 tx=0x7ff37c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:15.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.236+0000 7ff375ffb700 1 -- 172.21.15.31:0/931942933 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c011590 con 0x7ff3800aee70 2023-10-06T05:24:15.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.236+0000 7ff375ffb700 1 -- 172.21.15.31:0/931942933 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c011710 con 0x7ff3800aee70 2023-10-06T05:24:15.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.236+0000 7ff375ffb700 1 -- 172.21.15.31:0/931942933 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c011ad0 con 0x7ff3800aee70 2023-10-06T05:24:15.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.236+0000 7ff387fff700 1 -- 172.21.15.31:0/931942933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38015bd90 con 0x7ff3800aee70 2023-10-06T05:24:15.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.236+0000 7ff387fff700 1 -- 172.21.15.31:0/931942933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38015c350 con 0x7ff3800aee70 2023-10-06T05:24:15.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.237+0000 7ff387fff700 1 -- 172.21.15.31:0/931942933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38004c380 con 0x7ff3800aee70 2023-10-06T05:24:15.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.238+0000 7ff375ffb700 1 -- 172.21.15.31:0/931942933 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ff37c01b3b0 con 0x7ff3800aee70 2023-10-06T05:24:15.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.239+0000 7ff375ffb700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3640674d0 0x7ff364069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:15.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.239+0000 7ff375ffb700 1 -- 172.21.15.31:0/931942933 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff37c084660 con 0x7ff3800aee70 2023-10-06T05:24:15.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.240+0000 7ff38559a700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3640674d0 0x7ff364069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:15.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.240+0000 7ff38559a700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3640674d0 0x7ff364069980 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff380069dc0 tx=0x7ff370017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:15.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.243+0000 7ff375ffb700 1 -- 172.21.15.31:0/931942933 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff37c050f50 con 0x7ff3800aee70 2023-10-06T05:24:15.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.490+0000 7ff387fff700 1 -- 172.21.15.31:0/931942933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff3800c9f60 con 0x7ff3800aee70 2023-10-06T05:24:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.490+0000 7ff375ffb700 1 -- 172.21.15.31:0/931942933 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff37c017070 con 0x7ff3800aee70 2023-10-06T05:24:15.492 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:24:15.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 -- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3640674d0 msgr2=0x7ff364069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3640674d0 0x7ff364069980 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff380069dc0 tx=0x7ff370017000 comp rx=0 tx=0).stop 2023-10-06T05:24:15.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 -- 172.21.15.31:0/931942933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800aee70 msgr2=0x7ff38016a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:15.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800aee70 0x7ff38016a3e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00c7f0 tx=0x7ff37c007f70 comp rx=0 tx=0).stop 2023-10-06T05:24:15.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 -- 172.21.15.31:0/931942933 shutdown_connections 2023-10-06T05:24:15.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3640674d0 0x7ff364069980 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3800c7f30 0x7ff3800cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3800aee70 0x7ff38016a3e0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 --2- 172.21.15.31:0/931942933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3800c7b60 0x7ff3800ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 -- 172.21.15.31:0/931942933 >> 172.21.15.31:0/931942933 conn(0x7ff3800cc0b0 msgr2=0x7ff38013e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 -- 172.21.15.31:0/931942933 shutdown_connections 2023-10-06T05:24:15.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:15.493+0000 7ff387fff700 1 -- 172.21.15.31:0/931942933 wait complete. 2023-10-06T05:24:15.512 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:24:15.512 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:24:15.512 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:24:15.512 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:24:15.512 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:15.512 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:24:15.512 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:24:15.513 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:24:15.513 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:24:15.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:15.513 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:24:15.566 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:24:15.566 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:24:15.566 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:24:15.566 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:15.566 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:24:15.566 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:15.566 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:15.566 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:15.566 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41859sec preferred_lft 41859sec 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:15.567 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:24:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:24:15.569 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:15.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:15.570 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:24:15.641 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:24:15.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:15.644 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:24:15.696 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:24:15.696 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:24:15.696 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:24:15.696 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:24:15.696 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:24:15.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:15.697 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:24:15.697 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:24:15.697 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:24:15.697 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:24:15.697 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:24:15.697 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:24:15.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:15.869 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:24:15.915 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:24:15.915 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:24:15.916 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:24:15.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:15.916 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:24:15.988 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:15.988 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:16.035 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:24:16.035 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:16.036 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:16.121 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:24:16.121 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:24:16.121 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:16.262 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:16.262 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:24:16.288 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:16.288 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:24:16.288 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:24:16.288 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:24:16.288 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:24:16.292 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:24:16.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:16.292 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:16.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:24:16.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.571+0000 7fe2823ff700 1 Processor -- start 2023-10-06T05:24:16.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.571+0000 7fe2823ff700 1 -- start start 2023-10-06T05:24:16.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.572+0000 7fe2823ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c142c20 0x7fe27c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.572+0000 7fe2823ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c15d210 con 0x7fe27c15a990 2023-10-06T05:24:16.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.572+0000 7fe2823ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c0cc810 con 0x7fe27c0cb3e0 2023-10-06T05:24:16.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.572+0000 7fe2823ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c15d4b0 con 0x7fe27c142c20 2023-10-06T05:24:16.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.572+0000 7fe27b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe27c15a990 0x7fe27c0cacd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39618/0 (socket says 172.21.15.31:39618) 2023-10-06T05:24:16.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.572+0000 7fe27b7fe700 1 -- 172.21.15.31:0/102415775 learned_addr learned my addr 172.21.15.31:0/102415775 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:16.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.573+0000 7fe27bfff700 1 --2- 172.21.15.31:0/102415775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c142c20 0x7fe27c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.573+0000 7fe27affd700 1 -- 172.21.15.31:0/102415775 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225556670 0 0) 0x7fe27c15d210 con 0x7fe27c15a990 2023-10-06T05:24:16.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.573+0000 7fe27affd700 1 -- 172.21.15.31:0/102415775 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe264003570 con 0x7fe27c15a990 2023-10-06T05:24:16.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.573+0000 7fe27affd700 1 -- 172.21.15.31:0/102415775 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591346024 0 0) 0x7fe27c0cc810 con 0x7fe27c0cb3e0 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.573+0000 7fe27affd700 1 -- 172.21.15.31:0/102415775 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe27c15d210 con 0x7fe27c0cb3e0 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.573+0000 7fe27bfff700 1 -- 172.21.15.31:0/102415775 >> v1:172.21.15.62:6789/0 conn(0x7fe27c0cb3e0 legacy=0x7fe27c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.573+0000 7fe27bfff700 1 -- 172.21.15.31:0/102415775 >> v1:172.21.15.31:6789/0 conn(0x7fe27c15a990 legacy=0x7fe27c0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.573+0000 7fe27bfff700 1 -- 172.21.15.31:0/102415775 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c15d790 con 0x7fe27c142c20 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.573+0000 7fe27bfff700 1 --2- 172.21.15.31:0/102415775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c142c20 0x7fe27c0cbea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009910 tx=0x7fe26c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eee598768b8a63c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.574+0000 7fe27affd700 1 -- 172.21.15.31:0/102415775 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c011630 con 0x7fe27c142c20 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.574+0000 7fe27affd700 1 -- 172.21.15.31:0/102415775 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c011790 con 0x7fe27c142c20 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.574+0000 7fe27affd700 1 -- 172.21.15.31:0/102415775 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c011b30 con 0x7fe27c142c20 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.574+0000 7fe2823ff700 1 -- 172.21.15.31:0/102415775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c142c20 msgr2=0x7fe27c0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.574+0000 7fe2823ff700 1 --2- 172.21.15.31:0/102415775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c142c20 0x7fe27c0cbea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009910 tx=0x7fe26c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.574+0000 7fe2823ff700 1 -- 172.21.15.31:0/102415775 shutdown_connections 2023-10-06T05:24:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.574+0000 7fe2823ff700 1 --2- 172.21.15.31:0/102415775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c142c20 0x7fe27c0cbea0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.574+0000 7fe2823ff700 1 -- 172.21.15.31:0/102415775 >> 172.21.15.31:0/102415775 conn(0x7fe27c0ce2d0 msgr2=0x7fe27c0ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.574+0000 7fe2823ff700 1 -- 172.21.15.31:0/102415775 shutdown_connections 2023-10-06T05:24:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.575+0000 7fe2823ff700 1 -- 172.21.15.31:0/102415775 wait complete. 2023-10-06T05:24:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.576+0000 7fe2823ff700 1 Processor -- start 2023-10-06T05:24:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.576+0000 7fe2823ff700 1 -- start start 2023-10-06T05:24:16.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.577+0000 7fe2823ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c0cb3e0 0x7fe27c16cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.577+0000 7fe2823ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe27c142c20 0x7fe27c0af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.577+0000 7fe2823ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe27c15a990 0x7fe27c0b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.577+0000 7fe2823ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c16d530 con 0x7fe27c142c20 2023-10-06T05:24:16.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.577+0000 7fe2823ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0b1070 con 0x7fe27c15a990 2023-10-06T05:24:16.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.577+0000 7fe2823ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0b1350 con 0x7fe27c0cb3e0 2023-10-06T05:24:16.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe27c142c20 0x7fe27c0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe27c142c20 0x7fe27c0af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49508/0 (socket says 172.21.15.31:49508) 2023-10-06T05:24:16.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe27b7fe700 1 -- 172.21.15.31:0/3959658862 learned_addr learned my addr 172.21.15.31:0/3959658862 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe27bfff700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c0cb3e0 0x7fe27c16cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe28099c700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe27c15a990 0x7fe27c0b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe27b7fe700 1 -- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c0cb3e0 msgr2=0x7fe27c16cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe27b7fe700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c0cb3e0 0x7fe27c16cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe27b7fe700 1 -- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe27c15a990 msgr2=0x7fe27c0b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe27b7fe700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe27c15a990 0x7fe27c0b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe27b7fe700 1 -- 172.21.15.31:0/3959658862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c009420 con 0x7fe27c142c20 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.578+0000 7fe27b7fe700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe27c142c20 0x7fe27c0af800 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe27000c590 tx=0x7fe270007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.579+0000 7fe263fff700 1 -- 172.21.15.31:0/3959658862 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270005b30 con 0x7fe27c142c20 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.579+0000 7fe263fff700 1 -- 172.21.15.31:0/3959658862 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe270005cb0 con 0x7fe27c142c20 2023-10-06T05:24:16.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.579+0000 7fe263fff700 1 -- 172.21.15.31:0/3959658862 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270007320 con 0x7fe27c142c20 2023-10-06T05:24:16.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.579+0000 7fe2823ff700 1 -- 172.21.15.31:0/3959658862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c16d760 con 0x7fe27c142c20 2023-10-06T05:24:16.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.579+0000 7fe2823ff700 1 -- 172.21.15.31:0/3959658862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c16dd20 con 0x7fe27c142c20 2023-10-06T05:24:16.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.580+0000 7fe2823ff700 1 -- 172.21.15.31:0/3959658862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c04c380 con 0x7fe27c142c20 2023-10-06T05:24:16.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.581+0000 7fe263fff700 1 -- 172.21.15.31:0/3959658862 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fe27001b3b0 con 0x7fe27c142c20 2023-10-06T05:24:16.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.582+0000 7fe263fff700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2680674a0 0x7fe268069950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:16.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.582+0000 7fe263fff700 1 -- 172.21.15.31:0/3959658862 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe2700844d0 con 0x7fe27c142c20 2023-10-06T05:24:16.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.583+0000 7fe27bfff700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2680674a0 0x7fe268069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:16.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.583+0000 7fe27bfff700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2680674a0 0x7fe268069950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009860 tx=0x7fe26c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:16.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.586+0000 7fe263fff700 1 -- 172.21.15.31:0/3959658862 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe270050d10 con 0x7fe27c142c20 2023-10-06T05:24:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:16.835+0000 7fe2823ff700 1 -- 172.21.15.31:0/3959658862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe27c15da70 con 0x7fe27c142c20 2023-10-06T05:24:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.146+0000 7fe263fff700 1 -- 172.21.15.31:0/3959658862 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe270054330 con 0x7fe27c142c20 2023-10-06T05:24:17.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 -- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2680674a0 msgr2=0x7fe268069950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2680674a0 0x7fe268069950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009860 tx=0x7fe26c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:17.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 -- 172.21.15.31:0/3959658862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe27c142c20 msgr2=0x7fe27c0af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe27c142c20 0x7fe27c0af800 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe27000c590 tx=0x7fe270007dd0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 -- 172.21.15.31:0/3959658862 shutdown_connections 2023-10-06T05:24:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe2680674a0 0x7fe268069950 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe27c15a990 0x7fe27c0b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe27c0cb3e0 0x7fe27c16cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 --2- 172.21.15.31:0/3959658862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe27c142c20 0x7fe27c0af800 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 -- 172.21.15.31:0/3959658862 >> 172.21.15.31:0/3959658862 conn(0x7fe27c0ce2d0 msgr2=0x7fe27c1533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 -- 172.21.15.31:0/3959658862 shutdown_connections 2023-10-06T05:24:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.149+0000 7fe2823ff700 1 -- 172.21.15.31:0/3959658862 wait complete. 2023-10-06T05:24:17.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:24:17.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.366+0000 7f7b8d3ff700 1 Processor -- start 2023-10-06T05:24:17.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.366+0000 7f7b8d3ff700 1 -- start start 2023-10-06T05:24:17.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.367+0000 7f7b8d3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7b88159580 0x7f7b880cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.367+0000 7f7b8d3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880ccc80 0x7f7b880cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.367+0000 7f7b8d3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b880cf280 con 0x7f7b880cb9f0 2023-10-06T05:24:17.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.367+0000 7f7b8d3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b880cfe20 con 0x7f7b88159580 2023-10-06T05:24:17.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.367+0000 7f7b8d3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880ce670 con 0x7f7b880ccc80 2023-10-06T05:24:17.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.367+0000 7f7b877fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880ccc80 0x7f7b880cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.367+0000 7f7b86ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7b88159580 0x7f7b880cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.367+0000 7f7b86ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7b88159580 0x7f7b880cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53158/0 (socket says 172.21.15.31:53158) 2023-10-06T05:24:17.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.367+0000 7f7b877fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880ccc80 0x7f7b880cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:60178/0 (socket says 172.21.15.31:60178) 2023-10-06T05:24:17.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.368+0000 7f7b86ffd700 1 -- 172.21.15.31:0/873959262 learned_addr learned my addr 172.21.15.31:0/873959262 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:17.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.367+0000 7f7b867fc700 1 --1- 172.21.15.31:0/873959262 >> v1:172.21.15.62:6789/0 conn(0x7f7b880cb9f0 0x7f7b880cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:60596/0 (socket says 172.21.15.31:60596) 2023-10-06T05:24:17.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.368+0000 7f7b85ffb700 1 -- 172.21.15.31:0/873959262 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427627594 0 0) 0x7f7b880cf280 con 0x7f7b880cb9f0 2023-10-06T05:24:17.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.368+0000 7f7b85ffb700 1 -- 172.21.15.31:0/873959262 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b70003570 con 0x7f7b880cb9f0 2023-10-06T05:24:17.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.368+0000 7f7b877fe700 1 -- 172.21.15.31:0/873959262 >> v2:172.21.15.62:3300/0 conn(0x7f7b88159580 msgr2=0x7f7b880cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:17.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.368+0000 7f7b877fe700 1 --2- 172.21.15.31:0/873959262 >> v2:172.21.15.62:3300/0 conn(0x7f7b88159580 0x7f7b880cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.368+0000 7f7b877fe700 1 -- 172.21.15.31:0/873959262 >> v1:172.21.15.62:6789/0 conn(0x7f7b880cb9f0 legacy=0x7f7b880cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.368+0000 7f7b877fe700 1 -- 172.21.15.31:0/873959262 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b880ce950 con 0x7f7b880ccc80 2023-10-06T05:24:17.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.368+0000 7f7b86ffd700 1 --2- 172.21.15.31:0/873959262 >> v2:172.21.15.62:3300/0 conn(0x7f7b88159580 0x7f7b880cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:17.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.368+0000 7f7b877fe700 1 --2- 172.21.15.31:0/873959262 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880ccc80 0x7f7b880cd0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009e60 tx=0x7f7b80009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355f81334cb96d6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b85ffb700 1 -- 172.21.15.31:0/873959262 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000e070 con 0x7f7b880ccc80 2023-10-06T05:24:17.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b85ffb700 1 -- 172.21.15.31:0/873959262 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8000bab0 con 0x7f7b880ccc80 2023-10-06T05:24:17.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b85ffb700 1 -- 172.21.15.31:0/873959262 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000be70 con 0x7f7b880ccc80 2023-10-06T05:24:17.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/873959262 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880ccc80 msgr2=0x7f7b880cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b8d3ff700 1 --2- 172.21.15.31:0/873959262 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880ccc80 0x7f7b880cd0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009e60 tx=0x7f7b80009780 comp rx=0 tx=0).stop 2023-10-06T05:24:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/873959262 shutdown_connections 2023-10-06T05:24:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b8d3ff700 1 --2- 172.21.15.31:0/873959262 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880ccc80 0x7f7b880cd0a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b8d3ff700 1 --2- 172.21.15.31:0/873959262 >> v2:172.21.15.62:3300/0 conn(0x7f7b88159580 0x7f7b880cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/873959262 >> 172.21.15.31:0/873959262 conn(0x7f7b88099710 msgr2=0x7f7b8806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/873959262 shutdown_connections 2023-10-06T05:24:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.369+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/873959262 wait complete. 2023-10-06T05:24:17.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.371+0000 7f7b8d3ff700 1 Processor -- start 2023-10-06T05:24:17.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.371+0000 7f7b8d3ff700 1 -- start start 2023-10-06T05:24:17.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b8d3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880cb9f0 0x7f7b880c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b8d3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b880ccc80 0x7f7b88156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b8d3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b88159580 0x7f7b880693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b8d3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8815b9d0 con 0x7f7b88159580 2023-10-06T05:24:17.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b8d3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8815ab40 con 0x7f7b880ccc80 2023-10-06T05:24:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b8d3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8815be60 con 0x7f7b880cb9f0 2023-10-06T05:24:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b88159580 0x7f7b880693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b88159580 0x7f7b880693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37458/0 (socket says 172.21.15.31:37458) 2023-10-06T05:24:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b867fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b880ccc80 0x7f7b88156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b877fe700 1 -- 172.21.15.31:0/1988514611 learned_addr learned my addr 172.21.15.31:0/1988514611 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.372+0000 7f7b86ffd700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880cb9f0 0x7f7b880c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b877fe700 1 -- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880cb9f0 msgr2=0x7f7b880c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b877fe700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880cb9f0 0x7f7b880c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b877fe700 1 -- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b880ccc80 msgr2=0x7f7b88156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b877fe700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b880ccc80 0x7f7b88156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b877fe700 1 -- 172.21.15.31:0/1988514611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b78009480 con 0x7f7b88159580 2023-10-06T05:24:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b877fe700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b88159580 0x7f7b880693d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000b610 tx=0x7f7b80019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b6effd700 1 -- 172.21.15.31:0/1988514611 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80012dd0 con 0x7f7b88159580 2023-10-06T05:24:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b6effd700 1 -- 172.21.15.31:0/1988514611 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b80027470 con 0x7f7b88159580 2023-10-06T05:24:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/1988514611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b80009a20 con 0x7f7b88159580 2023-10-06T05:24:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b6effd700 1 -- 172.21.15.31:0/1988514611 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80027830 con 0x7f7b88159580 2023-10-06T05:24:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.373+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/1988514611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8815b0a0 con 0x7f7b88159580 2023-10-06T05:24:17.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.374+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/1988514611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b8804c380 con 0x7f7b88159580 2023-10-06T05:24:17.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.375+0000 7f7b6effd700 1 -- 172.21.15.31:0/1988514611 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f7b80024070 con 0x7f7b88159580 2023-10-06T05:24:17.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.376+0000 7f7b6effd700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b740674d0 0x7f7b74069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.376+0000 7f7b6effd700 1 -- 172.21.15.31:0/1988514611 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f7b80087ac0 con 0x7f7b88159580 2023-10-06T05:24:17.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.377+0000 7f7b86ffd700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b740674d0 0x7f7b74069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.377+0000 7f7b86ffd700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b740674d0 0x7f7b74069980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009e90 tx=0x7f7b78009220 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.380+0000 7f7b6effd700 1 -- 172.21.15.31:0/1988514611 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b800543b0 con 0x7f7b88159580 2023-10-06T05:24:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.609+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/1988514611 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b880cec30 con 0x7f7b740674d0 2023-10-06T05:24:17.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.611+0000 7f7b6effd700 1 -- 172.21.15.31:0/1988514611 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f7b880cec30 con 0x7f7b740674d0 2023-10-06T05:24:17.612 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:24:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:24:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:17.613 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b740674d0 msgr2=0x7f7b74069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b740674d0 0x7f7b74069980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009e90 tx=0x7f7b78009220 comp rx=0 tx=0).stop 2023-10-06T05:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/1988514611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b88159580 msgr2=0x7f7b880693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b88159580 0x7f7b880693d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000b610 tx=0x7f7b80019000 comp rx=0 tx=0).stop 2023-10-06T05:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/1988514611 shutdown_connections 2023-10-06T05:24:17.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b740674d0 0x7f7b74069980 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b88159580 0x7f7b880693d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b880cb9f0 0x7f7b880c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 --2- 172.21.15.31:0/1988514611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b880ccc80 0x7f7b88156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/1988514611 >> 172.21.15.31:0/1988514611 conn(0x7f7b88099710 msgr2=0x7f7b880b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/1988514611 shutdown_connections 2023-10-06T05:24:17.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.613+0000 7f7b8d3ff700 1 -- 172.21.15.31:0/1988514611 wait complete. 2023-10-06T05:24:17.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:24:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.828+0000 7f580f9ff700 1 Processor -- start 2023-10-06T05:24:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.828+0000 7f580f9ff700 1 -- start start 2023-10-06T05:24:17.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.829+0000 7f580f9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58081573a0 0x7f58080acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.829+0000 7f580f9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5808157770 0x7f58080b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580f9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5808158cf0 con 0x7f5808157c10 2023-10-06T05:24:17.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580f9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58081580e0 con 0x7f5808157770 2023-10-06T05:24:17.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580f9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58081583c0 con 0x7f58081573a0 2023-10-06T05:24:17.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580cf9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5808157770 0x7f58080b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580df9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5808157c10 0x7f58080ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41028/0 (socket says 172.21.15.31:41028) 2023-10-06T05:24:17.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580df9c700 1 -- 172.21.15.31:0/2743453899 learned_addr learned my addr 172.21.15.31:0/2743453899 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:17.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580d79b700 1 --2- 172.21.15.31:0/2743453899 >> v2:172.21.15.62:3300/0 conn(0x7f58081573a0 0x7f58080acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580cf9a700 1 -- 172.21.15.31:0/2743453899 >> v2:172.21.15.62:3300/0 conn(0x7f58081573a0 msgr2=0x7f58080acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:17.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580cf9a700 1 --2- 172.21.15.31:0/2743453899 >> v2:172.21.15.62:3300/0 conn(0x7f58081573a0 0x7f58080acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f57fffff700 1 -- 172.21.15.31:0/2743453899 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001093449 0 0) 0x7f5808158cf0 con 0x7f5808157c10 2023-10-06T05:24:17.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580cf9a700 1 -- 172.21.15.31:0/2743453899 >> v1:172.21.15.31:6789/0 conn(0x7f5808157c10 legacy=0x7f58080ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580cf9a700 1 -- 172.21.15.31:0/2743453899 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58081586a0 con 0x7f5808157770 2023-10-06T05:24:17.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.830+0000 7f580d79b700 1 --2- 172.21.15.31:0/2743453899 >> v2:172.21.15.62:3300/0 conn(0x7f58081573a0 0x7f58080acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:17.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.831+0000 7f580cf9a700 1 --2- 172.21.15.31:0/2743453899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157770 0x7f58080b1260 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f58080cd110 tx=0x7f57f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6c0ec640ccef4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.831+0000 7f57fffff700 1 -- 172.21.15.31:0/2743453899 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f800b6b0 con 0x7f5808157770 2023-10-06T05:24:17.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.831+0000 7f57fffff700 1 -- 172.21.15.31:0/2743453899 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f800b810 con 0x7f5808157770 2023-10-06T05:24:17.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.831+0000 7f57fffff700 1 -- 172.21.15.31:0/2743453899 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f800bbb0 con 0x7f5808157770 2023-10-06T05:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.831+0000 7f580f9ff700 1 -- 172.21.15.31:0/2743453899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157770 msgr2=0x7f58080b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.831+0000 7f580f9ff700 1 --2- 172.21.15.31:0/2743453899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157770 0x7f58080b1260 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f58080cd110 tx=0x7f57f80091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.832+0000 7f580f9ff700 1 -- 172.21.15.31:0/2743453899 shutdown_connections 2023-10-06T05:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.832+0000 7f580f9ff700 1 --2- 172.21.15.31:0/2743453899 >> v2:172.21.15.62:3300/0 conn(0x7f58081573a0 0x7f58080acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.832+0000 7f580f9ff700 1 --2- 172.21.15.31:0/2743453899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157770 0x7f58080b1260 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.832+0000 7f580f9ff700 1 -- 172.21.15.31:0/2743453899 >> 172.21.15.31:0/2743453899 conn(0x7f5808067560 msgr2=0x7f5808067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.832+0000 7f580f9ff700 1 -- 172.21.15.31:0/2743453899 shutdown_connections 2023-10-06T05:24:17.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.832+0000 7f580f9ff700 1 -- 172.21.15.31:0/2743453899 wait complete. 2023-10-06T05:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.833+0000 7f580f9ff700 1 Processor -- start 2023-10-06T05:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.833+0000 7f580f9ff700 1 -- start start 2023-10-06T05:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.834+0000 7f580f9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58081573a0 0x7f580816dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.834+0000 7f580f9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5808157770 0x7f5808145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.834+0000 7f580f9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157c10 0x7f580816d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.834+0000 7f580f9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580815ba10 con 0x7f5808157c10 2023-10-06T05:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.834+0000 7f580f9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080ab0e0 con 0x7f58081573a0 2023-10-06T05:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.834+0000 7f580f9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580815b0e0 con 0x7f5808157770 2023-10-06T05:24:17.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580df9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157c10 0x7f580816d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580df9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157c10 0x7f580816d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37472/0 (socket says 172.21.15.31:37472) 2023-10-06T05:24:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580df9c700 1 -- 172.21.15.31:0/2081892200 learned_addr learned my addr 172.21.15.31:0/2081892200 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580d79b700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58081573a0 0x7f580816dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580df9c700 1 -- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5808157770 msgr2=0x7f5808145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580cf9a700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5808157770 0x7f5808145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580df9c700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5808157770 0x7f5808145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580df9c700 1 -- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58081573a0 msgr2=0x7f580816dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580df9c700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58081573a0 0x7f580816dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580df9c700 1 -- 172.21.15.31:0/2081892200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f8009420 con 0x7f5808157c10 2023-10-06T05:24:17.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580d79b700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58081573a0 0x7f580816dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580cf9a700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5808157770 0x7f5808145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f580df9c700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157c10 0x7f580816d150 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f580000ccc0 tx=0x7f5800001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.835+0000 7f57fd7fa700 1 -- 172.21.15.31:0/2081892200 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580000ab10 con 0x7f5808157c10 2023-10-06T05:24:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.836+0000 7f57fd7fa700 1 -- 172.21.15.31:0/2081892200 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580000ac70 con 0x7f5808157c10 2023-10-06T05:24:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.836+0000 7f57fd7fa700 1 -- 172.21.15.31:0/2081892200 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000070f0 con 0x7f5808157c10 2023-10-06T05:24:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.836+0000 7f580f9ff700 1 -- 172.21.15.31:0/2081892200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580815b3c0 con 0x7f5808157c10 2023-10-06T05:24:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.836+0000 7f580f9ff700 1 -- 172.21.15.31:0/2081892200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5808145fa0 con 0x7f5808157c10 2023-10-06T05:24:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.837+0000 7f580f9ff700 1 -- 172.21.15.31:0/2081892200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580815bbc0 con 0x7f5808157c10 2023-10-06T05:24:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.838+0000 7f57fd7fa700 1 -- 172.21.15.31:0/2081892200 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f5800007520 con 0x7f5808157c10 2023-10-06T05:24:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.838+0000 7f57fd7fa700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ec0674d0 0x7f57ec069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.838+0000 7f57fd7fa700 1 -- 172.21.15.31:0/2081892200 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f58000855f0 con 0x7f5808157c10 2023-10-06T05:24:17.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.839+0000 7f580d79b700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ec0674d0 0x7f57ec069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:17.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.839+0000 7f580d79b700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ec0674d0 0x7f57ec069980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5804006f00 tx=0x7f5804008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:17.843+0000 7f57fd7fa700 1 -- 172.21.15.31:0/2081892200 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f580815bbc0 con 0x7f5808157c10 2023-10-06T05:24:18.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.043+0000 7f580f9ff700 1 -- 172.21.15.31:0/2081892200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f580804c380 con 0x7f5808157c10 2023-10-06T05:24:18.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.044+0000 7f57fd7fa700 1 -- 172.21.15.31:0/2081892200 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5800051ef0 con 0x7f5808157c10 2023-10-06T05:24:18.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 -- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ec0674d0 msgr2=0x7f57ec069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ec0674d0 0x7f57ec069980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5804006f00 tx=0x7f5804008040 comp rx=0 tx=0).stop 2023-10-06T05:24:18.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 -- 172.21.15.31:0/2081892200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157c10 msgr2=0x7f580816d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157c10 0x7f580816d150 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f580000ccc0 tx=0x7f5800001a20 comp rx=0 tx=0).stop 2023-10-06T05:24:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 -- 172.21.15.31:0/2081892200 shutdown_connections 2023-10-06T05:24:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ec0674d0 0x7f57ec069980 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808157c10 0x7f580816d150 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58081573a0 0x7f580816dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 --2- 172.21.15.31:0/2081892200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5808157770 0x7f5808145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 -- 172.21.15.31:0/2081892200 >> 172.21.15.31:0/2081892200 conn(0x7f5808067560 msgr2=0x7f58080ca330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 -- 172.21.15.31:0/2081892200 shutdown_connections 2023-10-06T05:24:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.046+0000 7f580f9ff700 1 -- 172.21.15.31:0/2081892200 wait complete. 2023-10-06T05:24:18.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2023-10-06T05:24:18.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.263+0000 7f0a1a7ff700 1 Processor -- start 2023-10-06T05:24:18.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.263+0000 7f0a1a7ff700 1 -- start start 2023-10-06T05:24:18.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.264+0000 7f0a1a7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0a141573a0 0x7f0a140acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.264+0000 7f0a1a7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a140ae860 0x7f0a141611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.264+0000 7f0a1a7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a14158cf0 con 0x7f0a14157770 2023-10-06T05:24:18.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.264+0000 7f0a1a7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a141580e0 con 0x7f0a140ae860 2023-10-06T05:24:18.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.264+0000 7f0a1a7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a141583c0 con 0x7f0a141573a0 2023-10-06T05:24:18.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.265+0000 7f0a18d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a140ae860 0x7f0a141611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.265+0000 7f0a18d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a140ae860 0x7f0a141611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37476/0 (socket says 172.21.15.31:37476) 2023-10-06T05:24:18.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.265+0000 7f0a18d9c700 1 -- 172.21.15.31:0/4150878641 learned_addr learned my addr 172.21.15.31:0/4150878641 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:18.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.265+0000 7f0a13fff700 1 --2- 172.21.15.31:0/4150878641 >> v2:172.21.15.62:3300/0 conn(0x7f0a141573a0 0x7f0a140acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.265+0000 7f0a18d9c700 1 -- 172.21.15.31:0/4150878641 >> v2:172.21.15.62:3300/0 conn(0x7f0a141573a0 msgr2=0x7f0a140acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:18.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.265+0000 7f0a18d9c700 1 --2- 172.21.15.31:0/4150878641 >> v2:172.21.15.62:3300/0 conn(0x7f0a141573a0 0x7f0a140acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.265+0000 7f0a18d9c700 1 -- 172.21.15.31:0/4150878641 >> v1:172.21.15.62:6789/0 conn(0x7f0a14157770 legacy=0x7f0a140b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.265+0000 7f0a18d9c700 1 -- 172.21.15.31:0/4150878641 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a141586a0 con 0x7f0a140ae860 2023-10-06T05:24:18.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.265+0000 7f0a18d9c700 1 --2- 172.21.15.31:0/4150878641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a140ae860 0x7f0a141611e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f0a140cd110 tx=0x7f0a0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1b8b3e843a9b5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.266+0000 7f0a12ffd700 1 -- 172.21.15.31:0/4150878641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c00e070 con 0x7f0a140ae860 2023-10-06T05:24:18.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.266+0000 7f0a12ffd700 1 -- 172.21.15.31:0/4150878641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c00ba20 con 0x7f0a140ae860 2023-10-06T05:24:18.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.266+0000 7f0a12ffd700 1 -- 172.21.15.31:0/4150878641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0092e0 con 0x7f0a140ae860 2023-10-06T05:24:18.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.266+0000 7f0a1a7ff700 1 -- 172.21.15.31:0/4150878641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a140ae860 msgr2=0x7f0a141611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.266+0000 7f0a1a7ff700 1 --2- 172.21.15.31:0/4150878641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a140ae860 0x7f0a141611e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f0a140cd110 tx=0x7f0a0c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:18.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.266+0000 7f0a1a7ff700 1 -- 172.21.15.31:0/4150878641 shutdown_connections 2023-10-06T05:24:18.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.266+0000 7f0a1a7ff700 1 --2- 172.21.15.31:0/4150878641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a140ae860 0x7f0a141611e0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.266+0000 7f0a1a7ff700 1 --2- 172.21.15.31:0/4150878641 >> v2:172.21.15.62:3300/0 conn(0x7f0a141573a0 0x7f0a140acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.266+0000 7f0a1a7ff700 1 -- 172.21.15.31:0/4150878641 >> 172.21.15.31:0/4150878641 conn(0x7f0a14067560 msgr2=0x7f0a14067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:18.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.266+0000 7f0a1a7ff700 1 -- 172.21.15.31:0/4150878641 shutdown_connections 2023-10-06T05:24:18.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.267+0000 7f0a1a7ff700 1 -- 172.21.15.31:0/4150878641 wait complete. 2023-10-06T05:24:18.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.268+0000 7f0a1a7ff700 1 Processor -- start 2023-10-06T05:24:18.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.268+0000 7f0a1a7ff700 1 -- start start 2023-10-06T05:24:18.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.269+0000 7f0a1a7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a141573a0 0x7f0a141567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.269+0000 7f0a1a7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a14157770 0x7f0a1415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.269+0000 7f0a1a7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a1415e3f0 0x7f0a14169850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.269+0000 7f0a1a7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1415c740 con 0x7f0a141573a0 2023-10-06T05:24:18.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.269+0000 7f0a1a7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1415e900 con 0x7f0a14157770 2023-10-06T05:24:18.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.269+0000 7f0a1a7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1415ebe0 con 0x7f0a1415e3f0 2023-10-06T05:24:18.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.269+0000 7f0a13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a141573a0 0x7f0a141567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.269+0000 7f0a13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a141573a0 0x7f0a141567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37482/0 (socket says 172.21.15.31:37482) 2023-10-06T05:24:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.269+0000 7f0a13fff700 1 -- 172.21.15.31:0/1938888624 learned_addr learned my addr 172.21.15.31:0/1938888624 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f0a18d9c700 1 --2- 172.21.15.31:0/1938888624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a1415e3f0 0x7f0a14169850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f0a137fe700 1 --2- 172.21.15.31:0/1938888624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a14157770 0x7f0a1415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f0a13fff700 1 -- 172.21.15.31:0/1938888624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a1415e3f0 msgr2=0x7f0a14169850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f0a13fff700 1 --2- 172.21.15.31:0/1938888624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a1415e3f0 0x7f0a14169850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f0a13fff700 1 -- 172.21.15.31:0/1938888624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a14157770 msgr2=0x7f0a1415deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f0a13fff700 1 --2- 172.21.15.31:0/1938888624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a14157770 0x7f0a1415deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f0a13fff700 1 -- 172.21.15.31:0/1938888624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c009ae0 con 0x7f0a141573a0 2023-10-06T05:24:18.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f0a13fff700 1 --2- 172.21.15.31:0/1938888624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a141573a0 0x7f0a141567a0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400ec80 tx=0x7f0a040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f09fbfff700 1 -- 172.21.15.31:0/1938888624 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400c9d0 con 0x7f0a141573a0 2023-10-06T05:24:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f09fbfff700 1 -- 172.21.15.31:0/1938888624 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0400cb50 con 0x7f0a141573a0 2023-10-06T05:24:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f0a1a7ff700 1 -- 172.21.15.31:0/1938888624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1415c3d0 con 0x7f0a141573a0 2023-10-06T05:24:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.270+0000 7f09fbfff700 1 -- 172.21.15.31:0/1938888624 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04013470 con 0x7f0a141573a0 2023-10-06T05:24:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.271+0000 7f0a1a7ff700 1 -- 172.21.15.31:0/1938888624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a14156ce0 con 0x7f0a141573a0 2023-10-06T05:24:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.272+0000 7f0a1a7ff700 1 -- 172.21.15.31:0/1938888624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a140abb20 con 0x7f0a141573a0 2023-10-06T05:24:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.272+0000 7f09fbfff700 1 -- 172.21.15.31:0/1938888624 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0a04013860 con 0x7f0a141573a0 2023-10-06T05:24:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.274+0000 7f09fbfff700 1 --2- 172.21.15.31:0/1938888624 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0a000674d0 0x7f0a00069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.274+0000 7f09fbfff700 1 -- 172.21.15.31:0/1938888624 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f0a04085530 con 0x7f0a141573a0 2023-10-06T05:24:18.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.274+0000 7f0a137fe700 1 --2- 172.21.15.31:0/1938888624 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0a000674d0 0x7f0a00069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.275+0000 7f0a137fe700 1 --2- 172.21.15.31:0/1938888624 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0a000674d0 0x7f0a00069980 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0a140c8130 tx=0x7f0a08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.278+0000 7f09fbfff700 1 -- 172.21.15.31:0/1938888624 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a04051d70 con 0x7f0a141573a0 2023-10-06T05:24:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.514+0000 7f0a1a7ff700 1 -- 172.21.15.31:0/1938888624 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a1415c880 con 0x7f0a000674d0 2023-10-06T05:24:18.517 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:18.516+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.516+0000 7f09fbfff700 1 -- 172.21.15.31:0/1938888624 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f0a08006820 con 0x7f0a000674d0 2023-10-06T05:24:18.519 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:18.527 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:18.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2023-10-06T05:24:18.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.724+0000 7f9b1c9ff700 1 Processor -- start 2023-10-06T05:24:18.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.724+0000 7f9b1c9ff700 1 -- start start 2023-10-06T05:24:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.725+0000 7f9b1c9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9b181573a0 0x7f9b180acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1c9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b180ae860 0x7f9b181611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1c9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b18158cf0 con 0x7f9b18157770 2023-10-06T05:24:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1c9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b181580e0 con 0x7f9b181573a0 2023-10-06T05:24:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1c9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b181583c0 con 0x7f9b180ae860 2023-10-06T05:24:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9b181573a0 0x7f9b180acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9b181573a0 0x7f9b180acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37488/0 (socket says 172.21.15.31:37488) 2023-10-06T05:24:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1659c700 1 -- 172.21.15.31:0/2524322328 learned_addr learned my addr 172.21.15.31:0/2524322328 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b16d9d700 1 --2- 172.21.15.31:0/2524322328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b180ae860 0x7f9b181611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1659c700 1 -- 172.21.15.31:0/2524322328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b180ae860 msgr2=0x7f9b181611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1659c700 1 --2- 172.21.15.31:0/2524322328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b180ae860 0x7f9b181611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1659c700 1 -- 172.21.15.31:0/2524322328 >> v1:172.21.15.62:6789/0 conn(0x7f9b18157770 legacy=0x7f9b180b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.726+0000 7f9b1659c700 1 -- 172.21.15.31:0/2524322328 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b181586a0 con 0x7f9b181573a0 2023-10-06T05:24:18.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.727+0000 7f9b1659c700 1 --2- 172.21.15.31:0/2524322328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b181573a0 0x7f9b180acb70 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f9b080099b0 tx=0x7f9b08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6963a5e074b188be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.727+0000 7f9b1559a700 1 -- 172.21.15.31:0/2524322328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08009ea0 con 0x7f9b181573a0 2023-10-06T05:24:18.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.727+0000 7f9b1559a700 1 -- 172.21.15.31:0/2524322328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0800b860 con 0x7f9b181573a0 2023-10-06T05:24:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.727+0000 7f9b1559a700 1 -- 172.21.15.31:0/2524322328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0800bc00 con 0x7f9b181573a0 2023-10-06T05:24:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.727+0000 7f9b1c9ff700 1 -- 172.21.15.31:0/2524322328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b181573a0 msgr2=0x7f9b180acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.727+0000 7f9b1c9ff700 1 --2- 172.21.15.31:0/2524322328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b181573a0 0x7f9b180acb70 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f9b080099b0 tx=0x7f9b08009250 comp rx=0 tx=0).stop 2023-10-06T05:24:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.728+0000 7f9b1c9ff700 1 -- 172.21.15.31:0/2524322328 shutdown_connections 2023-10-06T05:24:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.728+0000 7f9b1c9ff700 1 --2- 172.21.15.31:0/2524322328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b180ae860 0x7f9b181611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.728+0000 7f9b1c9ff700 1 --2- 172.21.15.31:0/2524322328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b181573a0 0x7f9b180acb70 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.728+0000 7f9b1c9ff700 1 -- 172.21.15.31:0/2524322328 >> 172.21.15.31:0/2524322328 conn(0x7f9b18067560 msgr2=0x7f9b18067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.728+0000 7f9b1c9ff700 1 -- 172.21.15.31:0/2524322328 shutdown_connections 2023-10-06T05:24:18.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.728+0000 7f9b1c9ff700 1 -- 172.21.15.31:0/2524322328 wait complete. 2023-10-06T05:24:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.729+0000 7f9b1c9ff700 1 Processor -- start 2023-10-06T05:24:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.729+0000 7f9b1c9ff700 1 -- start start 2023-10-06T05:24:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.730+0000 7f9b1c9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b181573a0 0x7f9b180ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.730+0000 7f9b1c9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18157770 0x7f9b180cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.730+0000 7f9b1c9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b1816caf0 0x7f9b18169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.730+0000 7f9b1c9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b180cbb50 con 0x7f9b1816caf0 2023-10-06T05:24:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.730+0000 7f9b1c9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b180cb080 con 0x7f9b18157770 2023-10-06T05:24:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.730+0000 7f9b1c9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b180af180 con 0x7f9b181573a0 2023-10-06T05:24:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.730+0000 7f9b16d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b1816caf0 0x7f9b18169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b16d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b1816caf0 0x7f9b18169d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37496/0 (socket says 172.21.15.31:37496) 2023-10-06T05:24:18.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b16d9d700 1 -- 172.21.15.31:0/942583013 learned_addr learned my addr 172.21.15.31:0/942583013 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b1659c700 1 --2- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b181573a0 0x7f9b180ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b15d9b700 1 --2- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18157770 0x7f9b180cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b16d9d700 1 -- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b181573a0 msgr2=0x7f9b180ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b16d9d700 1 --2- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b181573a0 0x7f9b180ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b16d9d700 1 -- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18157770 msgr2=0x7f9b180cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b16d9d700 1 --2- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18157770 0x7f9b180cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:18.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b16d9d700 1 -- 172.21.15.31:0/942583013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b080094f0 con 0x7f9b1816caf0 2023-10-06T05:24:18.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b1659c700 1 --2- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b181573a0 0x7f9b180ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:18.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b15d9b700 1 --2- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18157770 0x7f9b180cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:18.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b16d9d700 1 --2- 172.21.15.31:0/942583013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b1816caf0 0x7f9b18169d30 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009da0 tx=0x7f9b1000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b067fc700 1 -- 172.21.15.31:0/942583013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1000cb50 con 0x7f9b1816caf0 2023-10-06T05:24:18.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b067fc700 1 -- 172.21.15.31:0/942583013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1000f040 con 0x7f9b1816caf0 2023-10-06T05:24:18.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.731+0000 7f9b067fc700 1 -- 172.21.15.31:0/942583013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10014480 con 0x7f9b1816caf0 2023-10-06T05:24:18.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.732+0000 7f9b1c9ff700 1 -- 172.21.15.31:0/942583013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b180ca7e0 con 0x7f9b1816caf0 2023-10-06T05:24:18.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.732+0000 7f9b1c9ff700 1 -- 172.21.15.31:0/942583013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b180cada0 con 0x7f9b1816caf0 2023-10-06T05:24:18.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.733+0000 7f9b1c9ff700 1 -- 172.21.15.31:0/942583013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1804c380 con 0x7f9b1816caf0 2023-10-06T05:24:18.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.734+0000 7f9b067fc700 1 -- 172.21.15.31:0/942583013 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9b10014830 con 0x7f9b1816caf0 2023-10-06T05:24:18.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.735+0000 7f9b067fc700 1 --2- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9af80674d0 0x7f9af8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:18.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.735+0000 7f9b067fc700 1 -- 172.21.15.31:0/942583013 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9b10085ed0 con 0x7f9b1816caf0 2023-10-06T05:24:18.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.735+0000 7f9b1659c700 1 --2- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9af80674d0 0x7f9af8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:18.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.736+0000 7f9b1659c700 1 --2- 172.21.15.31:0/942583013 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9af80674d0 0x7f9af8069980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9b080099b0 tx=0x7f9b08017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:18.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.739+0000 7f9b067fc700 1 -- 172.21.15.31:0/942583013 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b10052640 con 0x7f9b1816caf0 2023-10-06T05:24:18.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.973+0000 7f9b1c9ff700 1 -- 172.21.15.31:0/942583013 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b180af2c0 con 0x7f9af80674d0 2023-10-06T05:24:18.977 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:18.975+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:18.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:18.976+0000 7f9b067fc700 1 -- 172.21.15.31:0/942583013 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f9b180af2c0 con 0x7f9af80674d0 2023-10-06T05:24:18.979 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:18.986 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:18.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2023-10-06T05:24:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.184+0000 7fb44c7ff700 1 Processor -- start 2023-10-06T05:24:19.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.184+0000 7fb44c7ff700 1 -- start start 2023-10-06T05:24:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.185+0000 7fb44c7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb444159930 0x7fb4440b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.185+0000 7fb44c7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4440cd580 0x7fb4440b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.185+0000 7fb44c7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44415d2d0 con 0x7fb444159560 2023-10-06T05:24:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.185+0000 7fb44c7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44415d5b0 con 0x7fb4440cd580 2023-10-06T05:24:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.185+0000 7fb44c7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44415d890 con 0x7fb444159930 2023-10-06T05:24:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.185+0000 7fb44ad9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4440cd580 0x7fb4440b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.186+0000 7fb44ad9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4440cd580 0x7fb4440b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37506/0 (socket says 172.21.15.31:37506) 2023-10-06T05:24:19.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.186+0000 7fb44ad9c700 1 -- 172.21.15.31:0/18348793 learned_addr learned my addr 172.21.15.31:0/18348793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.186+0000 7fb449d9a700 1 --2- 172.21.15.31:0/18348793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb444159930 0x7fb4440b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.186+0000 7fb44ad9c700 1 -- 172.21.15.31:0/18348793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb444159930 msgr2=0x7fb4440b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.186+0000 7fb44ad9c700 1 --2- 172.21.15.31:0/18348793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb444159930 0x7fb4440b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.186+0000 7fb44ad9c700 1 -- 172.21.15.31:0/18348793 >> v1:172.21.15.62:6789/0 conn(0x7fb444159560 legacy=0x7fb4440cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.186+0000 7fb449599700 1 -- 172.21.15.31:0/18348793 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271152971 0 0) 0x7fb44415d2d0 con 0x7fb444159560 2023-10-06T05:24:19.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.186+0000 7fb44ad9c700 1 -- 172.21.15.31:0/18348793 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44415db70 con 0x7fb4440cd580 2023-10-06T05:24:19.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.186+0000 7fb44ad9c700 1 --2- 172.21.15.31:0/18348793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4440cd580 0x7fb4440b29b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb43c009f20 tx=0x7fb43c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0c18f7cc4991e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb449599700 1 -- 172.21.15.31:0/18348793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c00e070 con 0x7fb4440cd580 2023-10-06T05:24:19.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb449599700 1 -- 172.21.15.31:0/18348793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c00b9d0 con 0x7fb4440cd580 2023-10-06T05:24:19.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb449599700 1 -- 172.21.15.31:0/18348793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0092e0 con 0x7fb4440cd580 2023-10-06T05:24:19.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb44c7ff700 1 -- 172.21.15.31:0/18348793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4440cd580 msgr2=0x7fb4440b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb44c7ff700 1 --2- 172.21.15.31:0/18348793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4440cd580 0x7fb4440b29b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb43c009f20 tx=0x7fb43c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:19.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb44c7ff700 1 -- 172.21.15.31:0/18348793 shutdown_connections 2023-10-06T05:24:19.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb44c7ff700 1 --2- 172.21.15.31:0/18348793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4440cd580 0x7fb4440b29b0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb44c7ff700 1 --2- 172.21.15.31:0/18348793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb444159930 0x7fb4440b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb44c7ff700 1 -- 172.21.15.31:0/18348793 >> 172.21.15.31:0/18348793 conn(0x7fb44406ec90 msgr2=0x7fb44406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb44c7ff700 1 -- 172.21.15.31:0/18348793 shutdown_connections 2023-10-06T05:24:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.187+0000 7fb44c7ff700 1 -- 172.21.15.31:0/18348793 wait complete. 2023-10-06T05:24:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.189+0000 7fb44c7ff700 1 Processor -- start 2023-10-06T05:24:19.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.189+0000 7fb44c7ff700 1 -- start start 2023-10-06T05:24:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44c7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4440cd580 0x7fb4440ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44c7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb444159560 0x7fb4440d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44c7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444159930 0x7fb44414a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44c7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440d0e20 con 0x7fb444159930 2023-10-06T05:24:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44c7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440cefb0 con 0x7fb4440cd580 2023-10-06T05:24:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44c7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440cf290 con 0x7fb444159560 2023-10-06T05:24:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44ad9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444159930 0x7fb44414a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44ad9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444159930 0x7fb44414a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37508/0 (socket says 172.21.15.31:37508) 2023-10-06T05:24:19.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44ad9c700 1 -- 172.21.15.31:0/2342049426 learned_addr learned my addr 172.21.15.31:0/2342049426 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:19.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44a59b700 1 --2- 172.21.15.31:0/2342049426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4440cd580 0x7fb4440ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb449d9a700 1 --2- 172.21.15.31:0/2342049426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb444159560 0x7fb4440d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44ad9c700 1 -- 172.21.15.31:0/2342049426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb444159560 msgr2=0x7fb4440d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44ad9c700 1 --2- 172.21.15.31:0/2342049426 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb444159560 0x7fb4440d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44ad9c700 1 -- 172.21.15.31:0/2342049426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4440cd580 msgr2=0x7fb4440ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44ad9c700 1 --2- 172.21.15.31:0/2342049426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4440cd580 0x7fb4440ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44ad9c700 1 -- 172.21.15.31:0/2342049426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c009ae0 con 0x7fb444159930 2023-10-06T05:24:19.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.190+0000 7fb44a59b700 1 --2- 172.21.15.31:0/2342049426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4440cd580 0x7fb4440ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:19.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.191+0000 7fb44ad9c700 1 --2- 172.21.15.31:0/2342049426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444159930 0x7fb44414a7a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb43c012000 tx=0x7fb43c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.191+0000 7fb43a7fc700 1 -- 172.21.15.31:0/2342049426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c008bc0 con 0x7fb444159930 2023-10-06T05:24:19.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.191+0000 7fb43a7fc700 1 -- 172.21.15.31:0/2342049426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c008d20 con 0x7fb444159930 2023-10-06T05:24:19.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.191+0000 7fb43a7fc700 1 -- 172.21.15.31:0/2342049426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0266b0 con 0x7fb444159930 2023-10-06T05:24:19.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.191+0000 7fb44c7ff700 1 -- 172.21.15.31:0/2342049426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4440cf570 con 0x7fb444159930 2023-10-06T05:24:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.191+0000 7fb44c7ff700 1 -- 172.21.15.31:0/2342049426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4440c80c0 con 0x7fb444159930 2023-10-06T05:24:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.192+0000 7fb44c7ff700 1 -- 172.21.15.31:0/2342049426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4440c7f20 con 0x7fb444159930 2023-10-06T05:24:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.193+0000 7fb43a7fc700 1 -- 172.21.15.31:0/2342049426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb43c003040 con 0x7fb444159930 2023-10-06T05:24:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.194+0000 7fb43a7fc700 1 --2- 172.21.15.31:0/2342049426 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4280674d0 0x7fb428069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.194+0000 7fb43a7fc700 1 -- 172.21.15.31:0/2342049426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb43c0870a0 con 0x7fb444159930 2023-10-06T05:24:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.195+0000 7fb44a59b700 1 --2- 172.21.15.31:0/2342049426 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4280674d0 0x7fb428069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.195+0000 7fb44a59b700 1 --2- 172.21.15.31:0/2342049426 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4280674d0 0x7fb428069980 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb4400011e0 tx=0x7fb440009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.198+0000 7fb43a7fc700 1 -- 172.21.15.31:0/2342049426 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4440c7f20 con 0x7fb444159930 2023-10-06T05:24:19.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.439+0000 7fb44c7ff700 1 -- 172.21.15.31:0/2342049426 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4440c7f20 con 0x7fb4280674d0 2023-10-06T05:24:19.443 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:19.441+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.442+0000 7fb43a7fc700 1 -- 172.21.15.31:0/2342049426 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fb4440c7f20 con 0x7fb4280674d0 2023-10-06T05:24:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:19.452 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:19.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2023-10-06T05:24:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.649+0000 7fadb13ff700 1 Processor -- start 2023-10-06T05:24:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.649+0000 7fadb13ff700 1 -- start start 2023-10-06T05:24:19.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.650+0000 7fadb13ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 0x7fadac06db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.650+0000 7fadb13ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fadac06e0d0 0x7fadac0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.650+0000 7fadb13ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadac06f500 con 0x7fadac0c8dd0 2023-10-06T05:24:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.650+0000 7fadb13ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadac06e8f0 con 0x7fadac06e0d0 2023-10-06T05:24:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.650+0000 7fadb13ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac06ebd0 con 0x7fadac155a10 2023-10-06T05:24:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.650+0000 7fadaaffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fadac0c8dd0 0x7fadac0c91a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41034/0 (socket says 172.21.15.31:41034) 2023-10-06T05:24:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.650+0000 7fadaaffd700 1 -- 172.21.15.31:0/1167679208 learned_addr learned my addr 172.21.15.31:0/1167679208 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.651+0000 7fadab7fe700 1 --2- 172.21.15.31:0/1167679208 >> v2:172.21.15.62:3300/0 conn(0x7fadac06e0d0 0x7fadac0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.651+0000 7fadaa7fc700 1 --2- 172.21.15.31:0/1167679208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 0x7fadac06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.651+0000 7fada9ffb700 1 -- 172.21.15.31:0/1167679208 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709023422 0 0) 0x7fadac06f500 con 0x7fadac0c8dd0 2023-10-06T05:24:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.651+0000 7fada9ffb700 1 -- 172.21.15.31:0/1167679208 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad94003570 con 0x7fadac0c8dd0 2023-10-06T05:24:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.651+0000 7fadaa7fc700 1 -- 172.21.15.31:0/1167679208 >> v2:172.21.15.62:3300/0 conn(0x7fadac06e0d0 msgr2=0x7fadac0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.651+0000 7fadaa7fc700 1 --2- 172.21.15.31:0/1167679208 >> v2:172.21.15.62:3300/0 conn(0x7fadac06e0d0 0x7fadac0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.651+0000 7fadaa7fc700 1 -- 172.21.15.31:0/1167679208 >> v1:172.21.15.31:6789/0 conn(0x7fadac0c8dd0 legacy=0x7fadac0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.651+0000 7fadaa7fc700 1 -- 172.21.15.31:0/1167679208 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac06eeb0 con 0x7fadac155a10 2023-10-06T05:24:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.651+0000 7fadaa7fc700 1 --2- 172.21.15.31:0/1167679208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 0x7fadac06db90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fada00098e0 tx=0x7fada00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=986714154d153696 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.652+0000 7fada9ffb700 1 -- 172.21.15.31:0/1167679208 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0011630 con 0x7fadac155a10 2023-10-06T05:24:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.652+0000 7fada9ffb700 1 -- 172.21.15.31:0/1167679208 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada0011790 con 0x7fadac155a10 2023-10-06T05:24:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.652+0000 7fada9ffb700 1 -- 172.21.15.31:0/1167679208 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0011b30 con 0x7fadac155a10 2023-10-06T05:24:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.653+0000 7fadb13ff700 1 -- 172.21.15.31:0/1167679208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 msgr2=0x7fadac06db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.653+0000 7fadb13ff700 1 --2- 172.21.15.31:0/1167679208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 0x7fadac06db90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fada00098e0 tx=0x7fada00091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.653+0000 7fadb13ff700 1 -- 172.21.15.31:0/1167679208 shutdown_connections 2023-10-06T05:24:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.653+0000 7fadb13ff700 1 --2- 172.21.15.31:0/1167679208 >> v2:172.21.15.62:3300/0 conn(0x7fadac06e0d0 0x7fadac0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.653+0000 7fadb13ff700 1 --2- 172.21.15.31:0/1167679208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 0x7fadac06db90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.653+0000 7fadb13ff700 1 -- 172.21.15.31:0/1167679208 >> 172.21.15.31:0/1167679208 conn(0x7fadac068030 msgr2=0x7fadac0697d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.653+0000 7fadb13ff700 1 -- 172.21.15.31:0/1167679208 shutdown_connections 2023-10-06T05:24:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.653+0000 7fadb13ff700 1 -- 172.21.15.31:0/1167679208 wait complete. 2023-10-06T05:24:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.654+0000 7fadb13ff700 1 Processor -- start 2023-10-06T05:24:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.654+0000 7fadb13ff700 1 -- start start 2023-10-06T05:24:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.655+0000 7fadb13ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac06e0d0 0x7fadac154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.655+0000 7fadb13ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadac0c8dd0 0x7fadac0ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.655+0000 7fadb13ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 0x7fadac0b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.655+0000 7fadb13ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0ae8f0 con 0x7fadac0c8dd0 2023-10-06T05:24:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.655+0000 7fadb13ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0aa590 con 0x7fadac06e0d0 2023-10-06T05:24:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.655+0000 7fadb13ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0aa870 con 0x7fadac155a10 2023-10-06T05:24:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadac0c8dd0 0x7fadac0ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadac0c8dd0 0x7fadac0ad620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37522/0 (socket says 172.21.15.31:37522) 2023-10-06T05:24:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaa7fc700 1 -- 172.21.15.31:0/1342666916 learned_addr learned my addr 172.21.15.31:0/1342666916 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadab7fe700 1 --2- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 0x7fadac0b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaaffd700 1 --2- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac06e0d0 0x7fadac154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaa7fc700 1 -- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 msgr2=0x7fadac0b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaa7fc700 1 --2- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 0x7fadac0b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaa7fc700 1 -- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac06e0d0 msgr2=0x7fadac154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaa7fc700 1 --2- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac06e0d0 0x7fadac154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaa7fc700 1 -- 172.21.15.31:0/1342666916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada0009420 con 0x7fadac0c8dd0 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadab7fe700 1 --2- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac155a10 0x7fadac0b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaaffd700 1 --2- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac06e0d0 0x7fadac154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fadaa7fc700 1 --2- 172.21.15.31:0/1342666916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadac0c8dd0 0x7fadac0ad620 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fadac06ed10 tx=0x7fada0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fad92ffd700 1 -- 172.21.15.31:0/1342666916 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0011b10 con 0x7fadac0c8dd0 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.656+0000 7fad92ffd700 1 -- 172.21.15.31:0/1342666916 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada00253f0 con 0x7fadac0c8dd0 2023-10-06T05:24:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.657+0000 7fad92ffd700 1 -- 172.21.15.31:0/1342666916 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0025790 con 0x7fadac0c8dd0 2023-10-06T05:24:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.657+0000 7fadb13ff700 1 -- 172.21.15.31:0/1342666916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac0aab50 con 0x7fadac0c8dd0 2023-10-06T05:24:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.657+0000 7fadb13ff700 1 -- 172.21.15.31:0/1342666916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac0ab110 con 0x7fadac0c8dd0 2023-10-06T05:24:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.658+0000 7fadb13ff700 1 -- 172.21.15.31:0/1342666916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac156fa0 con 0x7fadac0c8dd0 2023-10-06T05:24:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.659+0000 7fad92ffd700 1 -- 172.21.15.31:0/1342666916 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fada0025b80 con 0x7fadac0c8dd0 2023-10-06T05:24:19.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.660+0000 7fad92ffd700 1 --2- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad980674d0 0x7fad98069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:19.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.661+0000 7fad92ffd700 1 -- 172.21.15.31:0/1342666916 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fada0086820 con 0x7fadac0c8dd0 2023-10-06T05:24:19.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.661+0000 7fadaaffd700 1 --2- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad980674d0 0x7fad98069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:19.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.662+0000 7fadaaffd700 1 --2- 172.21.15.31:0/1342666916 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fad980674d0 0x7fad98069980 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0058d0 tx=0x7fad9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.665+0000 7fad92ffd700 1 -- 172.21.15.31:0/1342666916 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fada00531a0 con 0x7fadac0c8dd0 2023-10-06T05:24:19.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.902+0000 7fadb13ff700 1 -- 172.21.15.31:0/1342666916 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fadac04c380 con 0x7fad980674d0 2023-10-06T05:24:19.905 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:19.904+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:19.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:19.904+0000 7fad92ffd700 1 -- 172.21.15.31:0/1342666916 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fadac04c380 con 0x7fad980674d0 2023-10-06T05:24:19.907 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:19.915 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:19.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2023-10-06T05:24:20.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.114+0000 7f30c6bff700 1 Processor -- start 2023-10-06T05:24:20.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.114+0000 7f30c6bff700 1 -- start start 2023-10-06T05:24:20.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.116+0000 7f30c6bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c0153950 0x7f30c0153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.116+0000 7f30c6bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30c00cd5d0 0x7f30c015d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.116+0000 7f30c6bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c015cfc0 con 0x7f30c00cda40 2023-10-06T05:24:20.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.116+0000 7f30c6bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c015bb20 con 0x7f30c00cd5d0 2023-10-06T05:24:20.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.116+0000 7f30c6bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c015af10 con 0x7f30c0153950 2023-10-06T05:24:20.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.116+0000 7f30c499b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c0153950 0x7f30c0153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.116+0000 7f30bffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30c00cd5d0 0x7f30c015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.116+0000 7f30c499b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c0153950 0x7f30c0153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:60278/0 (socket says 172.21.15.31:60278) 2023-10-06T05:24:20.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.116+0000 7f30c499b700 1 -- 172.21.15.31:0/3177427795 learned_addr learned my addr 172.21.15.31:0/3177427795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:20.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.116+0000 7f30bffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30c00cd5d0 0x7f30c015d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53226/0 (socket says 172.21.15.31:53226) 2023-10-06T05:24:20.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.117+0000 7f30c499b700 1 -- 172.21.15.31:0/3177427795 >> v2:172.21.15.62:3300/0 conn(0x7f30c00cd5d0 msgr2=0x7f30c015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:20.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.117+0000 7f30c499b700 1 --2- 172.21.15.31:0/3177427795 >> v2:172.21.15.62:3300/0 conn(0x7f30c00cd5d0 0x7f30c015d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.117+0000 7f30c499b700 1 -- 172.21.15.31:0/3177427795 >> v1:172.21.15.62:6789/0 conn(0x7f30c00cda40 legacy=0x7f30c0159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.117+0000 7f30c499b700 1 -- 172.21.15.31:0/3177427795 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c015b1f0 con 0x7f30c0153950 2023-10-06T05:24:20.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.117+0000 7f30bffff700 1 --2- 172.21.15.31:0/3177427795 >> v2:172.21.15.62:3300/0 conn(0x7f30c00cd5d0 0x7f30c015d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:20.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.117+0000 7f30c499b700 1 --2- 172.21.15.31:0/3177427795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c0153950 0x7f30c0153d20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009990 tx=0x7f30b4009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e542945359ad02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.117+0000 7f30bf7fe700 1 -- 172.21.15.31:0/3177427795 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4009e80 con 0x7f30c0153950 2023-10-06T05:24:20.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.117+0000 7f30bf7fe700 1 -- 172.21.15.31:0/3177427795 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b4005c10 con 0x7f30c0153950 2023-10-06T05:24:20.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.117+0000 7f30bf7fe700 1 -- 172.21.15.31:0/3177427795 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b400b650 con 0x7f30c0153950 2023-10-06T05:24:20.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.118+0000 7f30c6bff700 1 -- 172.21.15.31:0/3177427795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c0153950 msgr2=0x7f30c0153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.118+0000 7f30c6bff700 1 --2- 172.21.15.31:0/3177427795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c0153950 0x7f30c0153d20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009990 tx=0x7f30b4009220 comp rx=0 tx=0).stop 2023-10-06T05:24:20.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.118+0000 7f30c6bff700 1 -- 172.21.15.31:0/3177427795 shutdown_connections 2023-10-06T05:24:20.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.118+0000 7f30c6bff700 1 --2- 172.21.15.31:0/3177427795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c0153950 0x7f30c0153d20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.118+0000 7f30c6bff700 1 --2- 172.21.15.31:0/3177427795 >> v2:172.21.15.62:3300/0 conn(0x7f30c00cd5d0 0x7f30c015d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.118+0000 7f30c6bff700 1 -- 172.21.15.31:0/3177427795 >> 172.21.15.31:0/3177427795 conn(0x7f30c00681b0 msgr2=0x7f30c00685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:20.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.118+0000 7f30c6bff700 1 -- 172.21.15.31:0/3177427795 shutdown_connections 2023-10-06T05:24:20.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.118+0000 7f30c6bff700 1 -- 172.21.15.31:0/3177427795 wait complete. 2023-10-06T05:24:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.120+0000 7f30c6bff700 1 Processor -- start 2023-10-06T05:24:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.120+0000 7f30c6bff700 1 -- start start 2023-10-06T05:24:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c6bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c00cd5d0 0x7f30c0151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c6bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c00cda40 0x7f30c00b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c6bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30c0153950 0x7f30c015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c6bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01517b0 con 0x7f30c0153950 2023-10-06T05:24:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c6bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c00ca600 con 0x7f30c00cd5d0 2023-10-06T05:24:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c6bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c0155450 con 0x7f30c00cda40 2023-10-06T05:24:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30c0153950 0x7f30c015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30c0153950 0x7f30c015e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37538/0 (socket says 172.21.15.31:37538) 2023-10-06T05:24:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c519c700 1 -- 172.21.15.31:0/984297319 learned_addr learned my addr 172.21.15.31:0/984297319 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30bffff700 1 --2- 172.21.15.31:0/984297319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c00cda40 0x7f30c00b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c499b700 1 --2- 172.21.15.31:0/984297319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c00cd5d0 0x7f30c0151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c519c700 1 -- 172.21.15.31:0/984297319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c00cda40 msgr2=0x7f30c00b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c519c700 1 --2- 172.21.15.31:0/984297319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30c00cda40 0x7f30c00b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c519c700 1 -- 172.21.15.31:0/984297319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c00cd5d0 msgr2=0x7f30c0151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c519c700 1 --2- 172.21.15.31:0/984297319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c00cd5d0 0x7f30c0151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.121+0000 7f30c519c700 1 -- 172.21.15.31:0/984297319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac009420 con 0x7f30c0153950 2023-10-06T05:24:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.122+0000 7f30c519c700 1 --2- 172.21.15.31:0/984297319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30c0153950 0x7f30c015e460 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c910 tx=0x7f30b8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.122+0000 7f30bcff9700 1 -- 172.21.15.31:0/984297319 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8006ba0 con 0x7f30c0153950 2023-10-06T05:24:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.122+0000 7f30bcff9700 1 -- 172.21.15.31:0/984297319 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8006d20 con 0x7f30c0153950 2023-10-06T05:24:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.122+0000 7f30bcff9700 1 -- 172.21.15.31:0/984297319 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8012880 con 0x7f30c0153950 2023-10-06T05:24:20.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.122+0000 7f30c6bff700 1 -- 172.21.15.31:0/984297319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b4009480 con 0x7f30c0153950 2023-10-06T05:24:20.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.122+0000 7f30c6bff700 1 -- 172.21.15.31:0/984297319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c00c9d60 con 0x7f30c0153950 2023-10-06T05:24:20.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.123+0000 7f30c6bff700 1 -- 172.21.15.31:0/984297319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c01555c0 con 0x7f30c0153950 2023-10-06T05:24:20.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.124+0000 7f30bcff9700 1 -- 172.21.15.31:0/984297319 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f30b8012c30 con 0x7f30c0153950 2023-10-06T05:24:20.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.125+0000 7f30bcff9700 1 --2- 172.21.15.31:0/984297319 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f30a40674d0 0x7f30a4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.125+0000 7f30bcff9700 1 -- 172.21.15.31:0/984297319 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f30b8084ec0 con 0x7f30c0153950 2023-10-06T05:24:20.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.126+0000 7f30c499b700 1 --2- 172.21.15.31:0/984297319 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f30a40674d0 0x7f30a4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.126+0000 7f30c499b700 1 --2- 172.21.15.31:0/984297319 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f30a40674d0 0x7f30a4069980 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f30b4007fd0 tx=0x7f30b4016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.129+0000 7f30bcff9700 1 -- 172.21.15.31:0/984297319 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30b8055190 con 0x7f30c0153950 2023-10-06T05:24:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.377+0000 7f30c6bff700 1 -- 172.21.15.31:0/984297319 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) v1 -- 0x7f30c015b570 con 0x7f30a40674d0 2023-10-06T05:24:20.380 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:20.379+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.379+0000 7f30bcff9700 1 -- 172.21.15.31:0/984297319 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f30b4009ed0 con 0x7f30a40674d0 2023-10-06T05:24:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:20.390 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:20.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2023-10-06T05:24:20.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.589+0000 7f6fad5ff700 1 Processor -- start 2023-10-06T05:24:20.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.589+0000 7f6fad5ff700 1 -- start start 2023-10-06T05:24:20.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.590+0000 7f6fad5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6fa8142c20 0x7f6fa80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.590+0000 7f6fad5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa815a990 0x7f6fa80cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fad5ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fa8142ff0 0x7f6fa80cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fad5ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa815d210 con 0x7f6fa8142ff0 2023-10-06T05:24:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fad5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa80cc810 con 0x7f6fa8142c20 2023-10-06T05:24:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fad5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa815d4b0 con 0x7f6fa815a990 2023-10-06T05:24:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fa8142ff0 0x7f6fa80cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fa8142ff0 0x7f6fa80cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37548/0 (socket says 172.21.15.31:37548) 2023-10-06T05:24:20.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa77fe700 1 -- 172.21.15.31:0/2225448682 learned_addr learned my addr 172.21.15.31:0/2225448682 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa6ffd700 1 --2- 172.21.15.31:0/2225448682 >> v2:172.21.15.62:3300/0 conn(0x7f6fa8142c20 0x7f6fa80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa67fc700 1 --2- 172.21.15.31:0/2225448682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa815a990 0x7f6fa80cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa77fe700 1 -- 172.21.15.31:0/2225448682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa815a990 msgr2=0x7f6fa80cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa77fe700 1 --2- 172.21.15.31:0/2225448682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa815a990 0x7f6fa80cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa77fe700 1 -- 172.21.15.31:0/2225448682 >> v2:172.21.15.62:3300/0 conn(0x7f6fa8142c20 msgr2=0x7f6fa80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa77fe700 1 --2- 172.21.15.31:0/2225448682 >> v2:172.21.15.62:3300/0 conn(0x7f6fa8142c20 0x7f6fa80cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa77fe700 1 -- 172.21.15.31:0/2225448682 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa815d790 con 0x7f6fa8142ff0 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.591+0000 7f6fa67fc700 1 --2- 172.21.15.31:0/2225448682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa815a990 0x7f6fa80cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.592+0000 7f6fa77fe700 1 --2- 172.21.15.31:0/2225448682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa8142ff0 0x7f6fa80cb210 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6fa806f3a0 tx=0x7f6fa0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341e61921cb6a2f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.592+0000 7f6fa5ffb700 1 -- 172.21.15.31:0/2225448682 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000e070 con 0x7f6fa8142ff0 2023-10-06T05:24:20.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.592+0000 7f6fa5ffb700 1 -- 172.21.15.31:0/2225448682 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa000ba20 con 0x7f6fa8142ff0 2023-10-06T05:24:20.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.592+0000 7f6fa5ffb700 1 -- 172.21.15.31:0/2225448682 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00092e0 con 0x7f6fa8142ff0 2023-10-06T05:24:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.592+0000 7f6fad5ff700 1 -- 172.21.15.31:0/2225448682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa8142ff0 msgr2=0x7f6fa80cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.592+0000 7f6fad5ff700 1 --2- 172.21.15.31:0/2225448682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa8142ff0 0x7f6fa80cb210 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6fa806f3a0 tx=0x7f6fa0009840 comp rx=0 tx=0).stop 2023-10-06T05:24:20.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.593+0000 7f6fad5ff700 1 -- 172.21.15.31:0/2225448682 shutdown_connections 2023-10-06T05:24:20.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.593+0000 7f6fad5ff700 1 --2- 172.21.15.31:0/2225448682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa8142ff0 0x7f6fa80cb210 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.593+0000 7f6fad5ff700 1 --2- 172.21.15.31:0/2225448682 >> v2:172.21.15.62:3300/0 conn(0x7f6fa8142c20 0x7f6fa80cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.593+0000 7f6fad5ff700 1 --2- 172.21.15.31:0/2225448682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa815a990 0x7f6fa80cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.593+0000 7f6fad5ff700 1 -- 172.21.15.31:0/2225448682 >> 172.21.15.31:0/2225448682 conn(0x7f6fa80ce2d0 msgr2=0x7f6fa80ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:20.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.593+0000 7f6fad5ff700 1 -- 172.21.15.31:0/2225448682 shutdown_connections 2023-10-06T05:24:20.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.593+0000 7f6fad5ff700 1 -- 172.21.15.31:0/2225448682 wait complete. 2023-10-06T05:24:20.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.594+0000 7f6fad5ff700 1 Processor -- start 2023-10-06T05:24:20.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.594+0000 7f6fad5ff700 1 -- start start 2023-10-06T05:24:20.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.595+0000 7f6fad5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fa8142c20 0x7f6fa81408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.595+0000 7f6fad5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa8142ff0 0x7f6fa80b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.595+0000 7f6fad5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa815a990 0x7f6fa80b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.595+0000 7f6fad5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8141370 con 0x7f6fa815a990 2023-10-06T05:24:20.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.595+0000 7f6fad5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa80b2bb0 con 0x7f6fa8142c20 2023-10-06T05:24:20.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.595+0000 7f6fad5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa80b2e90 con 0x7f6fa8142ff0 2023-10-06T05:24:20.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.595+0000 7f6fa77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa815a990 0x7f6fa80b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa815a990 0x7f6fa80b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37554/0 (socket says 172.21.15.31:37554) 2023-10-06T05:24:20.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa77fe700 1 -- 172.21.15.31:0/1793296639 learned_addr learned my addr 172.21.15.31:0/1793296639 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:20.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa67fc700 1 --2- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa8142ff0 0x7f6fa80b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa6ffd700 1 --2- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fa8142c20 0x7f6fa81408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa77fe700 1 -- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa8142ff0 msgr2=0x7f6fa80b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa77fe700 1 --2- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa8142ff0 0x7f6fa80b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa77fe700 1 -- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fa8142c20 msgr2=0x7f6fa81408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:20.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa77fe700 1 --2- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fa8142c20 0x7f6fa81408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:20.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa77fe700 1 -- 172.21.15.31:0/1793296639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa0009ae0 con 0x7f6fa815a990 2023-10-06T05:24:20.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa67fc700 1 --2- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fa8142ff0 0x7f6fa80b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:20.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa6ffd700 1 --2- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fa8142c20 0x7f6fa81408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:20.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fa77fe700 1 --2- 172.21.15.31:0/1793296639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa815a990 0x7f6fa80b2580 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80659f0 tx=0x7f6fa0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6f8effd700 1 -- 172.21.15.31:0/1793296639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00032f0 con 0x7f6fa815a990 2023-10-06T05:24:20.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6f8effd700 1 -- 172.21.15.31:0/1793296639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa0026460 con 0x7f6fa815a990 2023-10-06T05:24:20.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6f8effd700 1 -- 172.21.15.31:0/1793296639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0026800 con 0x7f6fa815a990 2023-10-06T05:24:20.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.596+0000 7f6fad5ff700 1 -- 172.21.15.31:0/1793296639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa80b3170 con 0x7f6fa815a990 2023-10-06T05:24:20.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.597+0000 7f6fad5ff700 1 -- 172.21.15.31:0/1793296639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa8140e10 con 0x7f6fa815a990 2023-10-06T05:24:20.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.598+0000 7f6fad5ff700 1 -- 172.21.15.31:0/1793296639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa8142430 con 0x7f6fa815a990 2023-10-06T05:24:20.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.599+0000 7f6f8effd700 1 -- 172.21.15.31:0/1793296639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f6fa0003040 con 0x7f6fa815a990 2023-10-06T05:24:20.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.600+0000 7f6f8effd700 1 --2- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f940674d0 0x7f6f94069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:20.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.600+0000 7f6f8effd700 1 -- 172.21.15.31:0/1793296639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6fa00877c0 con 0x7f6fa815a990 2023-10-06T05:24:20.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.600+0000 7f6fa6ffd700 1 --2- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f940674d0 0x7f6f94069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:20.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.601+0000 7f6fa6ffd700 1 --2- 172.21.15.31:0/1793296639 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f940674d0 0x7f6f94069980 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6f98006f00 tx=0x7f6f98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:20.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.604+0000 7f6f8effd700 1 -- 172.21.15.31:0/1793296639 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fa0053ef0 con 0x7f6fa815a990 2023-10-06T05:24:20.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.847+0000 7f6fad5ff700 1 -- 172.21.15.31:0/1793296639 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fa804c380 con 0x7f6f940674d0 2023-10-06T05:24:20.850 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:20.849+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:20.849+0000 7f6f8effd700 1 -- 172.21.15.31:0/1793296639 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f6fa804c380 con 0x7f6f940674d0 2023-10-06T05:24:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:20.859 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:20.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2023-10-06T05:24:21.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.056+0000 7fdfcb9ff700 1 Processor -- start 2023-10-06T05:24:21.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.056+0000 7fdfcb9ff700 1 -- start start 2023-10-06T05:24:21.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.057+0000 7fdfcb9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdfc4159560 0x7fdfc40cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.058+0000 7fdfcb9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc415d2d0 con 0x7fdfc40b2b80 2023-10-06T05:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.058+0000 7fdfcb9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc415d5b0 con 0x7fdfc4159930 2023-10-06T05:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.058+0000 7fdfcb9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc415d890 con 0x7fdfc4159560 2023-10-06T05:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.058+0000 7fdfc9f9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdfc40b2b80 0x7fdfc40cb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41050/0 (socket says 172.21.15.31:41050) 2023-10-06T05:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.058+0000 7fdfc9f9c700 1 -- 172.21.15.31:0/3523561468 learned_addr learned my addr 172.21.15.31:0/3523561468 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:21.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.058+0000 7fdfc979b700 1 --2- 172.21.15.31:0/3523561468 >> v2:172.21.15.62:3300/0 conn(0x7fdfc4159560 0x7fdfc40cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.058+0000 7fdfbbfff700 1 -- 172.21.15.31:0/3523561468 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435704852 0 0) 0x7fdfc415d2d0 con 0x7fdfc40b2b80 2023-10-06T05:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.058+0000 7fdfbbfff700 1 -- 172.21.15.31:0/3523561468 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb0003570 con 0x7fdfc40b2b80 2023-10-06T05:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.058+0000 7fdfbbfff700 1 -- 172.21.15.31:0/3523561468 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975701062 0 0) 0x7fdfc415d5b0 con 0x7fdfc4159930 2023-10-06T05:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.058+0000 7fdfbbfff700 1 -- 172.21.15.31:0/3523561468 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfc415d2d0 con 0x7fdfc4159930 2023-10-06T05:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.059+0000 7fdfc979b700 1 -- 172.21.15.31:0/3523561468 >> v1:172.21.15.62:6789/0 conn(0x7fdfc4159930 legacy=0x7fdfc40b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.059+0000 7fdfc979b700 1 -- 172.21.15.31:0/3523561468 >> v1:172.21.15.31:6789/0 conn(0x7fdfc40b2b80 legacy=0x7fdfc40cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.059+0000 7fdfc979b700 1 -- 172.21.15.31:0/3523561468 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc415db70 con 0x7fdfc4159560 2023-10-06T05:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.059+0000 7fdfc979b700 1 --2- 172.21.15.31:0/3523561468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4159560 0x7fdfc40cce70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00098e0 tx=0x7fdfc00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73932fb3b9f6a4d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.059+0000 7fdfbbfff700 1 -- 172.21.15.31:0/3523561468 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0011630 con 0x7fdfc4159560 2023-10-06T05:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.059+0000 7fdfbbfff700 1 -- 172.21.15.31:0/3523561468 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc0011790 con 0x7fdfc4159560 2023-10-06T05:24:21.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.059+0000 7fdfbbfff700 1 -- 172.21.15.31:0/3523561468 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0011b30 con 0x7fdfc4159560 2023-10-06T05:24:21.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.060+0000 7fdfcb9ff700 1 -- 172.21.15.31:0/3523561468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4159560 msgr2=0x7fdfc40cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.060+0000 7fdfcb9ff700 1 --2- 172.21.15.31:0/3523561468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4159560 0x7fdfc40cce70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00098e0 tx=0x7fdfc00091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.060+0000 7fdfcb9ff700 1 -- 172.21.15.31:0/3523561468 shutdown_connections 2023-10-06T05:24:21.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.060+0000 7fdfcb9ff700 1 --2- 172.21.15.31:0/3523561468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc4159560 0x7fdfc40cce70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.060+0000 7fdfcb9ff700 1 -- 172.21.15.31:0/3523561468 >> 172.21.15.31:0/3523561468 conn(0x7fdfc406ec90 msgr2=0x7fdfc406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:21.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.060+0000 7fdfcb9ff700 1 -- 172.21.15.31:0/3523561468 shutdown_connections 2023-10-06T05:24:21.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.060+0000 7fdfcb9ff700 1 -- 172.21.15.31:0/3523561468 wait complete. 2023-10-06T05:24:21.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.061+0000 7fdfcb9ff700 1 Processor -- start 2023-10-06T05:24:21.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.062+0000 7fdfcb9ff700 1 -- start start 2023-10-06T05:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.062+0000 7fdfcb9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc40b2b80 0x7fdfc40c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.062+0000 7fdfcb9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfc4159560 0x7fdfc40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.062+0000 7fdfcb9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfc4159930 0x7fdfc40cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.062+0000 7fdfcb9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40b4b20 con 0x7fdfc4159930 2023-10-06T05:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.062+0000 7fdfcb9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc4155890 con 0x7fdfc40b2b80 2023-10-06T05:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.062+0000 7fdfcb9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40b3c00 con 0x7fdfc4159560 2023-10-06T05:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc9f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfc4159930 0x7fdfc40cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc9f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfc4159930 0x7fdfc40cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37566/0 (socket says 172.21.15.31:37566) 2023-10-06T05:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc9f9c700 1 -- 172.21.15.31:0/3068661449 learned_addr learned my addr 172.21.15.31:0/3068661449 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc979b700 1 --2- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc40b2b80 0x7fdfc40c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc8f9a700 1 --2- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfc4159560 0x7fdfc40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc9f9c700 1 -- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfc4159560 msgr2=0x7fdfc40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc9f9c700 1 --2- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfc4159560 0x7fdfc40cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc9f9c700 1 -- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc40b2b80 msgr2=0x7fdfc40c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc9f9c700 1 --2- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc40b2b80 0x7fdfc40c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc9f9c700 1 -- 172.21.15.31:0/3068661449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc0009420 con 0x7fdfc4159930 2023-10-06T05:24:21.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc979b700 1 --2- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc40b2b80 0x7fdfc40c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:21.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc8f9a700 1 --2- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfc4159560 0x7fdfc40cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:21.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.063+0000 7fdfc9f9c700 1 --2- 172.21.15.31:0/3068661449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfc4159930 0x7fdfc40cfd80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00cc80 tx=0x7fdfbc001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.064+0000 7fdfb97fa700 1 -- 172.21.15.31:0/3068661449 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00ce90 con 0x7fdfc4159930 2023-10-06T05:24:21.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.064+0000 7fdfb97fa700 1 -- 172.21.15.31:0/3068661449 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc00ab70 con 0x7fdfc4159930 2023-10-06T05:24:21.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.064+0000 7fdfcb9ff700 1 -- 172.21.15.31:0/3068661449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc40b3ea0 con 0x7fdfc4159930 2023-10-06T05:24:21.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.064+0000 7fdfb97fa700 1 -- 172.21.15.31:0/3068661449 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0070f0 con 0x7fdfc4159930 2023-10-06T05:24:21.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.064+0000 7fdfcb9ff700 1 -- 172.21.15.31:0/3068661449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc40b4460 con 0x7fdfc4159930 2023-10-06T05:24:21.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.065+0000 7fdfcb9ff700 1 -- 172.21.15.31:0/3068661449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc40b4c60 con 0x7fdfc4159930 2023-10-06T05:24:21.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.066+0000 7fdfb97fa700 1 -- 172.21.15.31:0/3068661449 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fdfbc0074a0 con 0x7fdfc4159930 2023-10-06T05:24:21.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.066+0000 7fdfb97fa700 1 --2- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdfa80674d0 0x7fdfa8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.067+0000 7fdfb97fa700 1 -- 172.21.15.31:0/3068661449 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fdfbc085620 con 0x7fdfc4159930 2023-10-06T05:24:21.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.067+0000 7fdfc979b700 1 --2- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdfa80674d0 0x7fdfa8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.068+0000 7fdfc979b700 1 --2- 172.21.15.31:0/3068661449 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdfa80674d0 0x7fdfa8069980 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdfc416ce70 tx=0x7fdfc0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.071+0000 7fdfb97fa700 1 -- 172.21.15.31:0/3068661449 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfc40b4c60 con 0x7fdfc4159930 2023-10-06T05:24:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.307+0000 7fdfcb9ff700 1 -- 172.21.15.31:0/3068661449 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfc40b4c60 con 0x7fdfa80674d0 2023-10-06T05:24:21.311 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:21.309+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.310+0000 7fdfb97fa700 1 -- 172.21.15.31:0/3068661449 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fdfc40b4c60 con 0x7fdfa80674d0 2023-10-06T05:24:21.313 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:21.320 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:21.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2023-10-06T05:24:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.519+0000 7f44d5bff700 1 Processor -- start 2023-10-06T05:24:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.520+0000 7f44d5bff700 1 -- start start 2023-10-06T05:24:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.521+0000 7f44d5bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f44d015a990 0x7f44d00cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.521+0000 7f44d5bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d00cc5b0 con 0x7f44d0142ff0 2023-10-06T05:24:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.521+0000 7f44d5bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d015d210 con 0x7f44d0142c20 2023-10-06T05:24:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.521+0000 7f44d5bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d00cc810 con 0x7f44d015a990 2023-10-06T05:24:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.521+0000 7f44cffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f44d0142ff0 0x7f44d0068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41062/0 (socket says 172.21.15.31:41062) 2023-10-06T05:24:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.521+0000 7f44cffff700 1 -- 172.21.15.31:0/1303218759 learned_addr learned my addr 172.21.15.31:0/1303218759 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.521+0000 7f44ceffd700 1 --2- 172.21.15.31:0/1303218759 >> v2:172.21.15.62:3300/0 conn(0x7f44d015a990 0x7f44d00cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 961207347 0 0) 0x7f44d00cc5b0 con 0x7f44d0142ff0 2023-10-06T05:24:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b8003570 con 0x7f44d0142ff0 2023-10-06T05:24:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125363464 0 0) 0x7f44d015d210 con 0x7f44d0142c20 2023-10-06T05:24:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d00cc5b0 con 0x7f44d0142c20 2023-10-06T05:24:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4196178174 0 0) 0x7f44b8003570 con 0x7f44d0142ff0 2023-10-06T05:24:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 >> v2:172.21.15.62:3300/0 conn(0x7f44d015a990 msgr2=0x7f44d00cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 --2- 172.21.15.31:0/1303218759 >> v2:172.21.15.62:3300/0 conn(0x7f44d015a990 0x7f44d00cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 >> v1:172.21.15.62:6789/0 conn(0x7f44d0142c20 legacy=0x7f44d00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d015d4b0 con 0x7f44d0142ff0 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f44c80017f0 con 0x7f44d0142ff0 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 >> v1:172.21.15.31:6789/0 conn(0x7f44d0142ff0 legacy=0x7f44d0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44b8003bf0 0x7f44b8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b8004810 0x7f44b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44b80051c0 0x7f44b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00cc810 con 0x7f44b8004810 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b800c0b0 con 0x7f44b8003bf0 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b800c270 con 0x7f44b80051c0 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.522+0000 7f44ceffd700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b8004810 0x7f44b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44cf7fe700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44b8003bf0 0x7f44b8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44cffff700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44b80051c0 0x7f44b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44ceffd700 1 -- 172.21.15.31:0/1303218759 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44b80051c0 msgr2=0x7f44b800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44ceffd700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44b80051c0 0x7f44b800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44ceffd700 1 -- 172.21.15.31:0/1303218759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44b8003bf0 msgr2=0x7f44b8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44ceffd700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44b8003bf0 0x7f44b8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44ceffd700 1 -- 172.21.15.31:0/1303218759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b8003570 con 0x7f44b8004810 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44cffff700 1 -- 172.21.15.31:0/1303218759 reap_dead start 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44ceffd700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b8004810 0x7f44b8004c80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009c00 tx=0x7f44c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a071a22086d3fc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c400e710 con 0x7f44b8004810 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c400e870 con 0x7f44b8004810 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.523+0000 7f44ce7fc700 1 -- 172.21.15.31:0/1303218759 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c400ec30 con 0x7f44b8004810 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.524+0000 7f44d5bff700 1 -- 172.21.15.31:0/1303218759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b8004810 msgr2=0x7f44b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.524+0000 7f44d5bff700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b8004810 0x7f44b8004c80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009c00 tx=0x7f44c40091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.524+0000 7f44d5bff700 1 -- 172.21.15.31:0/1303218759 shutdown_connections 2023-10-06T05:24:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.524+0000 7f44d5bff700 1 --2- 172.21.15.31:0/1303218759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b8004810 0x7f44b8004c80 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.524+0000 7f44d5bff700 1 -- 172.21.15.31:0/1303218759 >> 172.21.15.31:0/1303218759 conn(0x7f44d00ce2d0 msgr2=0x7f44d00ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.524+0000 7f44d5bff700 1 -- 172.21.15.31:0/1303218759 shutdown_connections 2023-10-06T05:24:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.524+0000 7f44d5bff700 1 -- 172.21.15.31:0/1303218759 wait complete. 2023-10-06T05:24:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.526+0000 7f44d5bff700 1 Processor -- start 2023-10-06T05:24:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.526+0000 7f44d5bff700 1 -- start start 2023-10-06T05:24:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44d5bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b8004810 0x7f44d0140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44d5bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44d00afac0 0x7f44d00aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44d5bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44d00aa300 0x7f44d00aa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44d5bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00b21b0 con 0x7f44b8004810 2023-10-06T05:24:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44d5bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00b0470 con 0x7f44d00aa300 2023-10-06T05:24:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44d5bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00b1d20 con 0x7f44d00afac0 2023-10-06T05:24:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b8004810 0x7f44d0140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b8004810 0x7f44d0140200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37586/0 (socket says 172.21.15.31:37586) 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44cf7fe700 1 -- 172.21.15.31:0/3139486168 learned_addr learned my addr 172.21.15.31:0/3139486168 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44cffff700 1 --2- 172.21.15.31:0/3139486168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44d00aa300 0x7f44d00aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44ceffd700 1 --2- 172.21.15.31:0/3139486168 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44d00afac0 0x7f44d00aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44cf7fe700 1 -- 172.21.15.31:0/3139486168 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44d00afac0 msgr2=0x7f44d00aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44cf7fe700 1 --2- 172.21.15.31:0/3139486168 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44d00afac0 0x7f44d00aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44cf7fe700 1 -- 172.21.15.31:0/3139486168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44d00aa300 msgr2=0x7f44d00aa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44cf7fe700 1 --2- 172.21.15.31:0/3139486168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44d00aa300 0x7f44d00aa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.527+0000 7f44cf7fe700 1 -- 172.21.15.31:0/3139486168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d00b1110 con 0x7f44b8004810 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.528+0000 7f44cf7fe700 1 --2- 172.21.15.31:0/3139486168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b8004810 0x7f44d0140200 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f44c0002ac0 tx=0x7f44c0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.528+0000 7f44b77fe700 1 -- 172.21.15.31:0/3139486168 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0006a30 con 0x7f44b8004810 2023-10-06T05:24:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.528+0000 7f44b77fe700 1 -- 172.21.15.31:0/3139486168 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c0006bb0 con 0x7f44b8004810 2023-10-06T05:24:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.528+0000 7f44b77fe700 1 -- 172.21.15.31:0/3139486168 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0007050 con 0x7f44b8004810 2023-10-06T05:24:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.528+0000 7f44d5bff700 1 -- 172.21.15.31:0/3139486168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d00b13f0 con 0x7f44b8004810 2023-10-06T05:24:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.528+0000 7f44d5bff700 1 -- 172.21.15.31:0/3139486168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d00b19b0 con 0x7f44b8004810 2023-10-06T05:24:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.529+0000 7f44d5bff700 1 -- 172.21.15.31:0/3139486168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d00cdee0 con 0x7f44b8004810 2023-10-06T05:24:21.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.530+0000 7f44b77fe700 1 -- 172.21.15.31:0/3139486168 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f44c0007440 con 0x7f44b8004810 2023-10-06T05:24:21.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.531+0000 7f44b77fe700 1 --2- 172.21.15.31:0/3139486168 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f44bc0674d0 0x7f44bc069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.531+0000 7f44b77fe700 1 -- 172.21.15.31:0/3139486168 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f44c0085440 con 0x7f44b8004810 2023-10-06T05:24:21.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.532+0000 7f44ceffd700 1 --2- 172.21.15.31:0/3139486168 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f44bc0674d0 0x7f44bc069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.533+0000 7f44ceffd700 1 --2- 172.21.15.31:0/3139486168 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f44bc0674d0 0x7f44bc069980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f44c40091e0 tx=0x7f44c401a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.536+0000 7f44b77fe700 1 -- 172.21.15.31:0/3139486168 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44c0051cc0 con 0x7f44b8004810 2023-10-06T05:24:21.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.777+0000 7f44d5bff700 1 -- 172.21.15.31:0/3139486168 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f44d004c380 con 0x7f44bc0674d0 2023-10-06T05:24:21.780 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:21.778+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.779+0000 7f44b77fe700 1 -- 172.21.15.31:0/3139486168 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f44d004c380 con 0x7f44bc0674d0 2023-10-06T05:24:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:21.789 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:21.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2023-10-06T05:24:21.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.988+0000 7f7c6bfff700 1 Processor -- start 2023-10-06T05:24:21.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.989+0000 7f7c6bfff700 1 -- start start 2023-10-06T05:24:21.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.990+0000 7f7c6bfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7c6406df30 0x7f7c6406d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.990+0000 7f7c6bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c64159d70 0x7f7c64152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.991+0000 7f7c6bfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c640c72b0 con 0x7f7c6406d830 2023-10-06T05:24:21.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.991+0000 7f7c6bfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c640aba60 con 0x7f7c6406df30 2023-10-06T05:24:21.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.991+0000 7f7c6bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640c7590 con 0x7f7c64159d70 2023-10-06T05:24:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.991+0000 7f7c69d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7c6406df30 0x7f7c6406d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.991+0000 7f7c6a59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c64159d70 0x7f7c64152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.991+0000 7f7c69d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7c6406df30 0x7f7c6406d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:53318/0 (socket says 172.21.15.31:53318) 2023-10-06T05:24:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.991+0000 7f7c69d9b700 1 -- 172.21.15.31:0/2677956994 learned_addr learned my addr 172.21.15.31:0/2677956994 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.991+0000 7f7c6959a700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7c6406d830 0x7f7c64159660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:60666/0 (socket says 172.21.15.31:60666) 2023-10-06T05:24:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.992+0000 7f7c6a59c700 1 -- 172.21.15.31:0/2677956994 >> v2:172.21.15.62:3300/0 conn(0x7f7c6406df30 msgr2=0x7f7c6406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.992+0000 7f7c6a59c700 1 --2- 172.21.15.31:0/2677956994 >> v2:172.21.15.62:3300/0 conn(0x7f7c6406df30 0x7f7c6406d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.992+0000 7f7c6a59c700 1 -- 172.21.15.31:0/2677956994 >> v1:172.21.15.62:6789/0 conn(0x7f7c6406d830 legacy=0x7f7c64159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.992+0000 7f7c6a59c700 1 -- 172.21.15.31:0/2677956994 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640c7870 con 0x7f7c64159d70 2023-10-06T05:24:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.992+0000 7f7c6a59c700 1 --2- 172.21.15.31:0/2677956994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c64159d70 0x7f7c64152cb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009e60 tx=0x7f7c5c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d319dead45bd88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.992+0000 7f7c68d99700 1 -- 172.21.15.31:0/2677956994 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c00e070 con 0x7f7c64159d70 2023-10-06T05:24:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.992+0000 7f7c68d99700 1 -- 172.21.15.31:0/2677956994 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c00bb60 con 0x7f7c64159d70 2023-10-06T05:24:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.992+0000 7f7c68d99700 1 -- 172.21.15.31:0/2677956994 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c012430 con 0x7f7c64159d70 2023-10-06T05:24:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.993+0000 7f7c6bfff700 1 -- 172.21.15.31:0/2677956994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c64159d70 msgr2=0x7f7c64152cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.993+0000 7f7c6bfff700 1 --2- 172.21.15.31:0/2677956994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c64159d70 0x7f7c64152cb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009e60 tx=0x7f7c5c009780 comp rx=0 tx=0).stop 2023-10-06T05:24:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.993+0000 7f7c6bfff700 1 -- 172.21.15.31:0/2677956994 shutdown_connections 2023-10-06T05:24:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.993+0000 7f7c6bfff700 1 --2- 172.21.15.31:0/2677956994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c64159d70 0x7f7c64152cb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.993+0000 7f7c6bfff700 1 --2- 172.21.15.31:0/2677956994 >> v2:172.21.15.62:3300/0 conn(0x7f7c6406df30 0x7f7c6406d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.993+0000 7f7c6bfff700 1 -- 172.21.15.31:0/2677956994 >> 172.21.15.31:0/2677956994 conn(0x7f7c64068030 msgr2=0x7f7c640cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.993+0000 7f7c6bfff700 1 -- 172.21.15.31:0/2677956994 shutdown_connections 2023-10-06T05:24:21.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.993+0000 7f7c6bfff700 1 -- 172.21.15.31:0/2677956994 wait complete. 2023-10-06T05:24:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.994+0000 7f7c6bfff700 1 Processor -- start 2023-10-06T05:24:21.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.995+0000 7f7c6bfff700 1 -- start start 2023-10-06T05:24:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c6406d830 0x7f7c640c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6bfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c6406df30 0x7f7c6414f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c64159d70 0x7f7c640c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6bfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640c0470 con 0x7f7c64159d70 2023-10-06T05:24:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640af330 con 0x7f7c6406d830 2023-10-06T05:24:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6bfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640bfb80 con 0x7f7c6406df30 2023-10-06T05:24:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c64159d70 0x7f7c640c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c64159d70 0x7f7c640c55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37590/0 (socket says 172.21.15.31:37590) 2023-10-06T05:24:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6a59c700 1 -- 172.21.15.31:0/2070505213 learned_addr learned my addr 172.21.15.31:0/2070505213 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c69d9b700 1 --2- 172.21.15.31:0/2070505213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c6406d830 0x7f7c640c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:21.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6959a700 1 --2- 172.21.15.31:0/2070505213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c6406df30 0x7f7c6414f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6a59c700 1 -- 172.21.15.31:0/2070505213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c6406df30 msgr2=0x7f7c6414f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6a59c700 1 --2- 172.21.15.31:0/2070505213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c6406df30 0x7f7c6414f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6a59c700 1 -- 172.21.15.31:0/2070505213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c6406d830 msgr2=0x7f7c640c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6a59c700 1 --2- 172.21.15.31:0/2070505213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c6406d830 0x7f7c640c3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.996+0000 7f7c6a59c700 1 -- 172.21.15.31:0/2070505213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c009a20 con 0x7f7c64159d70 2023-10-06T05:24:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.997+0000 7f7c6a59c700 1 --2- 172.21.15.31:0/2070505213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c64159d70 0x7f7c640c55c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c013000 tx=0x7f7c5c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.997+0000 7f7c59ffb700 1 -- 172.21.15.31:0/2070505213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c012ea0 con 0x7f7c64159d70 2023-10-06T05:24:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.997+0000 7f7c59ffb700 1 -- 172.21.15.31:0/2070505213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c027470 con 0x7f7c64159d70 2023-10-06T05:24:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.997+0000 7f7c6bfff700 1 -- 172.21.15.31:0/2070505213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c640bfe20 con 0x7f7c64159d70 2023-10-06T05:24:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.997+0000 7f7c59ffb700 1 -- 172.21.15.31:0/2070505213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c027850 con 0x7f7c64159d70 2023-10-06T05:24:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.997+0000 7f7c6bfff700 1 -- 172.21.15.31:0/2070505213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c640c5e60 con 0x7f7c64159d70 2023-10-06T05:24:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.998+0000 7f7c6bfff700 1 -- 172.21.15.31:0/2070505213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c640c05e0 con 0x7f7c64159d70 2023-10-06T05:24:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:21.999+0000 7f7c59ffb700 1 -- 172.21.15.31:0/2070505213 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f7c5c024070 con 0x7f7c64159d70 2023-10-06T05:24:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.000+0000 7f7c59ffb700 1 --2- 172.21.15.31:0/2070505213 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7c480674d0 0x7f7c48069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.000+0000 7f7c59ffb700 1 -- 172.21.15.31:0/2070505213 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f7c5c087b60 con 0x7f7c64159d70 2023-10-06T05:24:22.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.000+0000 7f7c69d9b700 1 --2- 172.21.15.31:0/2070505213 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7c480674d0 0x7f7c48069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.001+0000 7f7c69d9b700 1 --2- 172.21.15.31:0/2070505213 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7c480674d0 0x7f7c48069980 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009da0 tx=0x7f7c60009220 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.004+0000 7f7c59ffb700 1 -- 172.21.15.31:0/2070505213 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c5c0543a0 con 0x7f7c64159d70 2023-10-06T05:24:22.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.251+0000 7f7c6bfff700 1 -- 172.21.15.31:0/2070505213 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c6404c380 con 0x7f7c480674d0 2023-10-06T05:24:22.255 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:22.253+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:22.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.254+0000 7f7c59ffb700 1 -- 172.21.15.31:0/2070505213 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f7c6404c380 con 0x7f7c480674d0 2023-10-06T05:24:22.257 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:22.264 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:22.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2023-10-06T05:24:22.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.463+0000 7fde8c3ff700 1 Processor -- start 2023-10-06T05:24:22.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.463+0000 7fde8c3ff700 1 -- start start 2023-10-06T05:24:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.464+0000 7fde8c3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde84156300 0x7fde840ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.464+0000 7fde8c3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde8406a630 0x7fde8406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8c3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8406e600 con 0x7fde8406ea90 2023-10-06T05:24:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8c3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8406b010 con 0x7fde84156300 2023-10-06T05:24:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8c3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8406ef00 con 0x7fde8406a630 2023-10-06T05:24:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8a19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde84156300 0x7fde840ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8a19b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde84156300 0x7fde840ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37604/0 (socket says 172.21.15.31:37604) 2023-10-06T05:24:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8a19b700 1 -- 172.21.15.31:0/2800287560 learned_addr learned my addr 172.21.15.31:0/2800287560 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8a99c700 1 --2- 172.21.15.31:0/2800287560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde8406a630 0x7fde8406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8a19b700 1 -- 172.21.15.31:0/2800287560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde8406a630 msgr2=0x7fde8406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:22.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8a19b700 1 --2- 172.21.15.31:0/2800287560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde8406a630 0x7fde8406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8a19b700 1 -- 172.21.15.31:0/2800287560 >> v1:172.21.15.62:6789/0 conn(0x7fde8406ea90 legacy=0x7fde8406d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8a19b700 1 -- 172.21.15.31:0/2800287560 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8406d9f0 con 0x7fde84156300 2023-10-06T05:24:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.465+0000 7fde8a19b700 1 --2- 172.21.15.31:0/2800287560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde84156300 0x7fde840ad670 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fde80009990 tx=0x7fde80009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d8a66039c6f399 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde89199700 1 -- 172.21.15.31:0/2800287560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80005c10 con 0x7fde84156300 2023-10-06T05:24:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde89199700 1 -- 172.21.15.31:0/2800287560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde80005d70 con 0x7fde84156300 2023-10-06T05:24:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde89199700 1 -- 172.21.15.31:0/2800287560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000b880 con 0x7fde84156300 2023-10-06T05:24:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde8c3ff700 1 -- 172.21.15.31:0/2800287560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde84156300 msgr2=0x7fde840ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde8c3ff700 1 --2- 172.21.15.31:0/2800287560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde84156300 0x7fde840ad670 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fde80009990 tx=0x7fde80009220 comp rx=0 tx=0).stop 2023-10-06T05:24:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde8c3ff700 1 -- 172.21.15.31:0/2800287560 shutdown_connections 2023-10-06T05:24:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde8c3ff700 1 --2- 172.21.15.31:0/2800287560 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde8406a630 0x7fde8406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde8c3ff700 1 --2- 172.21.15.31:0/2800287560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde84156300 0x7fde840ad670 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde8c3ff700 1 -- 172.21.15.31:0/2800287560 >> 172.21.15.31:0/2800287560 conn(0x7fde84068030 msgr2=0x7fde840cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:22.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde8c3ff700 1 -- 172.21.15.31:0/2800287560 shutdown_connections 2023-10-06T05:24:22.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.466+0000 7fde8c3ff700 1 -- 172.21.15.31:0/2800287560 wait complete. 2023-10-06T05:24:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.468+0000 7fde8c3ff700 1 Processor -- start 2023-10-06T05:24:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.468+0000 7fde8c3ff700 1 -- start start 2023-10-06T05:24:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8c3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde8406a630 0x7fde84157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8c3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde8406ea90 0x7fde841539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8c3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde84156300 0x7fde8416a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8c3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde841576c0 con 0x7fde8406ea90 2023-10-06T05:24:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8c3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde84169c40 con 0x7fde84156300 2023-10-06T05:24:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8c3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde84153f00 con 0x7fde8406a630 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde8406ea90 0x7fde841539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde8406ea90 0x7fde841539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37614/0 (socket says 172.21.15.31:37614) 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8999a700 1 -- 172.21.15.31:0/3374756379 learned_addr learned my addr 172.21.15.31:0/3374756379 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8a19b700 1 --2- 172.21.15.31:0/3374756379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde8406a630 0x7fde84157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.469+0000 7fde8a99c700 1 --2- 172.21.15.31:0/3374756379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde84156300 0x7fde8416a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde8999a700 1 -- 172.21.15.31:0/3374756379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde8406a630 msgr2=0x7fde84157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde8999a700 1 --2- 172.21.15.31:0/3374756379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde8406a630 0x7fde84157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde8999a700 1 -- 172.21.15.31:0/3374756379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde84156300 msgr2=0x7fde8416a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde8999a700 1 --2- 172.21.15.31:0/3374756379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde84156300 0x7fde8416a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde8999a700 1 -- 172.21.15.31:0/3374756379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde80009480 con 0x7fde8406ea90 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde8999a700 1 --2- 172.21.15.31:0/3374756379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde8406ea90 0x7fde841539c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fde7400c5c0 tx=0x7fde74007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde7a7fc700 1 -- 172.21.15.31:0/3374756379 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74011590 con 0x7fde8406ea90 2023-10-06T05:24:22.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde7a7fc700 1 -- 172.21.15.31:0/3374756379 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde74011710 con 0x7fde8406ea90 2023-10-06T05:24:22.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde8c3ff700 1 -- 172.21.15.31:0/3374756379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde841578f0 con 0x7fde8406ea90 2023-10-06T05:24:22.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde7a7fc700 1 -- 172.21.15.31:0/3374756379 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74011ad0 con 0x7fde8406ea90 2023-10-06T05:24:22.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.470+0000 7fde8c3ff700 1 -- 172.21.15.31:0/3374756379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde84157eb0 con 0x7fde8406ea90 2023-10-06T05:24:22.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.471+0000 7fde8c3ff700 1 -- 172.21.15.31:0/3374756379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8404c380 con 0x7fde8406ea90 2023-10-06T05:24:22.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.472+0000 7fde7a7fc700 1 -- 172.21.15.31:0/3374756379 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fde7401b3b0 con 0x7fde8406ea90 2023-10-06T05:24:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.473+0000 7fde7a7fc700 1 --2- 172.21.15.31:0/3374756379 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fde680674d0 0x7fde68069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.473+0000 7fde7a7fc700 1 -- 172.21.15.31:0/3374756379 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fde74084460 con 0x7fde8406ea90 2023-10-06T05:24:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.474+0000 7fde8a19b700 1 --2- 172.21.15.31:0/3374756379 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fde680674d0 0x7fde68069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.474+0000 7fde8a19b700 1 --2- 172.21.15.31:0/3374756379 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fde680674d0 0x7fde68069980 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fde80009260 tx=0x7fde80016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.477+0000 7fde7a7fc700 1 -- 172.21.15.31:0/3374756379 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde74050ca0 con 0x7fde8406ea90 2023-10-06T05:24:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.718+0000 7fde8c3ff700 1 -- 172.21.15.31:0/3374756379 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fde840c2ac0 con 0x7fde680674d0 2023-10-06T05:24:22.721 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:22.719+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:22.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.719+0000 7fde7a7fc700 1 -- 172.21.15.31:0/3374756379 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fde840c2ac0 con 0x7fde680674d0 2023-10-06T05:24:22.723 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:22.731 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:22.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2023-10-06T05:24:22.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.930+0000 7fe436dff700 1 Processor -- start 2023-10-06T05:24:22.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.930+0000 7fe436dff700 1 -- start start 2023-10-06T05:24:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe436dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe430142c20 0x7fe4300cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe436dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe4300cb3e0 0x7fe430068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe436dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43015d210 con 0x7fe43015a990 2023-10-06T05:24:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe436dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4300cc810 con 0x7fe430142c20 2023-10-06T05:24:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe436dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe43015d4b0 con 0x7fe4300cb3e0 2023-10-06T05:24:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe434b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe430142c20 0x7fe4300cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe427fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe43015a990 0x7fe4300cacd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41074/0 (socket says 172.21.15.31:41074) 2023-10-06T05:24:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe427fff700 1 -- 172.21.15.31:0/633488870 learned_addr learned my addr 172.21.15.31:0/633488870 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe434b9b700 1 -- 172.21.15.31:0/633488870 >> v2:172.21.15.62:3300/0 conn(0x7fe4300cb3e0 msgr2=0x7fe430068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe434b9b700 1 --2- 172.21.15.31:0/633488870 >> v2:172.21.15.62:3300/0 conn(0x7fe4300cb3e0 0x7fe430068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.932+0000 7fe434b9b700 1 -- 172.21.15.31:0/633488870 >> v1:172.21.15.31:6789/0 conn(0x7fe43015a990 legacy=0x7fe4300cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.933+0000 7fe434b9b700 1 -- 172.21.15.31:0/633488870 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43015d790 con 0x7fe430142c20 2023-10-06T05:24:22.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.933+0000 7fe434b9b700 1 --2- 172.21.15.31:0/633488870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe430142c20 0x7fe4300cbea0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe43006f050 tx=0x7fe42c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6492e3ea3d421bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.933+0000 7fe4277fe700 1 -- 172.21.15.31:0/633488870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c011630 con 0x7fe430142c20 2023-10-06T05:24:22.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.933+0000 7fe4277fe700 1 -- 172.21.15.31:0/633488870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c011790 con 0x7fe430142c20 2023-10-06T05:24:22.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.933+0000 7fe4277fe700 1 -- 172.21.15.31:0/633488870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c011b30 con 0x7fe430142c20 2023-10-06T05:24:22.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.934+0000 7fe436dff700 1 -- 172.21.15.31:0/633488870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe430142c20 msgr2=0x7fe4300cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.934+0000 7fe436dff700 1 --2- 172.21.15.31:0/633488870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe430142c20 0x7fe4300cbea0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe43006f050 tx=0x7fe42c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.934+0000 7fe436dff700 1 -- 172.21.15.31:0/633488870 shutdown_connections 2023-10-06T05:24:22.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.934+0000 7fe436dff700 1 --2- 172.21.15.31:0/633488870 >> v2:172.21.15.62:3300/0 conn(0x7fe4300cb3e0 0x7fe430068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.934+0000 7fe436dff700 1 --2- 172.21.15.31:0/633488870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe430142c20 0x7fe4300cbea0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.934+0000 7fe436dff700 1 -- 172.21.15.31:0/633488870 >> 172.21.15.31:0/633488870 conn(0x7fe4300ce2d0 msgr2=0x7fe4300ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:22.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.934+0000 7fe436dff700 1 -- 172.21.15.31:0/633488870 shutdown_connections 2023-10-06T05:24:22.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.934+0000 7fe436dff700 1 -- 172.21.15.31:0/633488870 wait complete. 2023-10-06T05:24:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.935+0000 7fe436dff700 1 Processor -- start 2023-10-06T05:24:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.936+0000 7fe436dff700 1 -- start start 2023-10-06T05:24:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.936+0000 7fe436dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4300cb3e0 0x7fe4300b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.936+0000 7fe436dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe430142c20 0x7fe4300b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.936+0000 7fe436dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43015a990 0x7fe4300b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.936+0000 7fe436dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300b15b0 con 0x7fe4300cb3e0 2023-10-06T05:24:22.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe436dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300b3ee0 con 0x7fe43015a990 2023-10-06T05:24:22.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe436dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300b3310 con 0x7fe430142c20 2023-10-06T05:24:22.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe434b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4300cb3e0 0x7fe4300b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe434b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4300cb3e0 0x7fe4300b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37634/0 (socket says 172.21.15.31:37634) 2023-10-06T05:24:22.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe434b9b700 1 -- 172.21.15.31:0/2078146120 learned_addr learned my addr 172.21.15.31:0/2078146120 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:22.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe427fff700 1 --2- 172.21.15.31:0/2078146120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe430142c20 0x7fe4300b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe43539c700 1 --2- 172.21.15.31:0/2078146120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43015a990 0x7fe4300b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe434b9b700 1 -- 172.21.15.31:0/2078146120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe430142c20 msgr2=0x7fe4300b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe434b9b700 1 --2- 172.21.15.31:0/2078146120 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe430142c20 0x7fe4300b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe434b9b700 1 -- 172.21.15.31:0/2078146120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43015a990 msgr2=0x7fe4300b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:22.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe434b9b700 1 --2- 172.21.15.31:0/2078146120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43015a990 0x7fe4300b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:22.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe434b9b700 1 -- 172.21.15.31:0/2078146120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c009420 con 0x7fe4300cb3e0 2023-10-06T05:24:22.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.937+0000 7fe434b9b700 1 --2- 172.21.15.31:0/2078146120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4300cb3e0 0x7fe4300b1070 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe4300659f0 tx=0x7fe42c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.938+0000 7fe424ff9700 1 -- 172.21.15.31:0/2078146120 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c000c90 con 0x7fe4300cb3e0 2023-10-06T05:24:22.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.938+0000 7fe424ff9700 1 -- 172.21.15.31:0/2078146120 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c01a450 con 0x7fe4300cb3e0 2023-10-06T05:24:22.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.938+0000 7fe424ff9700 1 -- 172.21.15.31:0/2078146120 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c01a810 con 0x7fe4300cb3e0 2023-10-06T05:24:22.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.938+0000 7fe436dff700 1 -- 172.21.15.31:0/2078146120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4300b35b0 con 0x7fe4300cb3e0 2023-10-06T05:24:22.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.938+0000 7fe436dff700 1 -- 172.21.15.31:0/2078146120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4300b3b70 con 0x7fe4300cb3e0 2023-10-06T05:24:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.939+0000 7fe436dff700 1 -- 172.21.15.31:0/2078146120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43004c380 con 0x7fe4300cb3e0 2023-10-06T05:24:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.940+0000 7fe424ff9700 1 -- 172.21.15.31:0/2078146120 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fe42c01ab80 con 0x7fe4300cb3e0 2023-10-06T05:24:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.941+0000 7fe424ff9700 1 --2- 172.21.15.31:0/2078146120 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe41c0674d0 0x7fe41c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.941+0000 7fe424ff9700 1 -- 172.21.15.31:0/2078146120 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe42c087180 con 0x7fe4300cb3e0 2023-10-06T05:24:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.941+0000 7fe427fff700 1 --2- 172.21.15.31:0/2078146120 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe41c0674d0 0x7fe41c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:22.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.942+0000 7fe427fff700 1 --2- 172.21.15.31:0/2078146120 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe41c0674d0 0x7fe41c069980 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe4200058d0 tx=0x7fe420009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:22.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:22.945+0000 7fe424ff9700 1 -- 172.21.15.31:0/2078146120 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe42c053a00 con 0x7fe4300cb3e0 2023-10-06T05:24:23.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.187+0000 7fe436dff700 1 -- 172.21.15.31:0/2078146120 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe43004c380 con 0x7fe41c0674d0 2023-10-06T05:24:23.190 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:23.188+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:23.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.189+0000 7fe424ff9700 1 -- 172.21.15.31:0/2078146120 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fe43004c380 con 0x7fe41c0674d0 2023-10-06T05:24:23.192 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:23.200 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:23.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2023-10-06T05:24:23.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.399+0000 7f9decbff700 1 Processor -- start 2023-10-06T05:24:23.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.399+0000 7f9decbff700 1 -- start start 2023-10-06T05:24:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.400+0000 7f9decbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9de8156300 0x7f9de80ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.400+0000 7f9decbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9de806ea90 0x7f9de806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9decbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9de806a630 0x7f9de806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9decbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de806aff0 con 0x7f9de806ea90 2023-10-06T05:24:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9decbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de806e600 con 0x7f9de806a630 2023-10-06T05:24:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9decbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de806ef00 con 0x7f9de8156300 2023-10-06T05:24:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de5d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9de806ea90 0x7f9de806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de5d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9de806ea90 0x7f9de806d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37644/0 (socket says 172.21.15.31:37644) 2023-10-06T05:24:23.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de5d9b700 1 -- 172.21.15.31:0/4239451156 learned_addr learned my addr 172.21.15.31:0/4239451156 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:23.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de6d9d700 1 --2- 172.21.15.31:0/4239451156 >> v2:172.21.15.62:3300/0 conn(0x7f9de806a630 0x7f9de806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de659c700 1 --2- 172.21.15.31:0/4239451156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9de8156300 0x7f9de80ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de5d9b700 1 -- 172.21.15.31:0/4239451156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9de8156300 msgr2=0x7f9de80ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de5d9b700 1 --2- 172.21.15.31:0/4239451156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9de8156300 0x7f9de80ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de5d9b700 1 -- 172.21.15.31:0/4239451156 >> v2:172.21.15.62:3300/0 conn(0x7f9de806a630 msgr2=0x7f9de806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de5d9b700 1 --2- 172.21.15.31:0/4239451156 >> v2:172.21.15.62:3300/0 conn(0x7f9de806a630 0x7f9de806aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de5d9b700 1 -- 172.21.15.31:0/4239451156 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de806d750 con 0x7f9de806ea90 2023-10-06T05:24:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.401+0000 7f9de6d9d700 1 --2- 172.21.15.31:0/4239451156 >> v2:172.21.15.62:3300/0 conn(0x7f9de806a630 0x7f9de806aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.402+0000 7f9de5d9b700 1 --2- 172.21.15.31:0/4239451156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9de806ea90 0x7f9de806d210 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f9de806d0e0 tx=0x7f9ddc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b0f0e0be6ad1f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.402+0000 7f9de559a700 1 -- 172.21.15.31:0/4239451156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00b860 con 0x7f9de806ea90 2023-10-06T05:24:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.402+0000 7f9de559a700 1 -- 172.21.15.31:0/4239451156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc00b9c0 con 0x7f9de806ea90 2023-10-06T05:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.402+0000 7f9de559a700 1 -- 172.21.15.31:0/4239451156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00bd60 con 0x7f9de806ea90 2023-10-06T05:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.402+0000 7f9decbff700 1 -- 172.21.15.31:0/4239451156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9de806ea90 msgr2=0x7f9de806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.402+0000 7f9decbff700 1 --2- 172.21.15.31:0/4239451156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9de806ea90 0x7f9de806d210 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f9de806d0e0 tx=0x7f9ddc009250 comp rx=0 tx=0).stop 2023-10-06T05:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.403+0000 7f9decbff700 1 -- 172.21.15.31:0/4239451156 shutdown_connections 2023-10-06T05:24:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.403+0000 7f9decbff700 1 --2- 172.21.15.31:0/4239451156 >> v2:172.21.15.62:3300/0 conn(0x7f9de806a630 0x7f9de806aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.403+0000 7f9decbff700 1 --2- 172.21.15.31:0/4239451156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9de8156300 0x7f9de80ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.403+0000 7f9decbff700 1 --2- 172.21.15.31:0/4239451156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9de806ea90 0x7f9de806d210 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.403+0000 7f9decbff700 1 -- 172.21.15.31:0/4239451156 >> 172.21.15.31:0/4239451156 conn(0x7f9de8068030 msgr2=0x7f9de80cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.403+0000 7f9decbff700 1 -- 172.21.15.31:0/4239451156 shutdown_connections 2023-10-06T05:24:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.403+0000 7f9decbff700 1 -- 172.21.15.31:0/4239451156 wait complete. 2023-10-06T05:24:23.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.404+0000 7f9decbff700 1 Processor -- start 2023-10-06T05:24:23.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.404+0000 7f9decbff700 1 -- start start 2023-10-06T05:24:23.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.405+0000 7f9decbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9de806a630 0x7f9de81443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.405+0000 7f9decbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de806ea90 0x7f9de80c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.405+0000 7f9decbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9de8156300 0x7f9de80c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.405+0000 7f9decbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80b0830 con 0x7f9de806a630 2023-10-06T05:24:23.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.405+0000 7f9decbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80c4a40 con 0x7f9de806ea90 2023-10-06T05:24:23.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.405+0000 7f9decbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80c4d20 con 0x7f9de8156300 2023-10-06T05:24:23.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9de806a630 0x7f9de81443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9de806a630 0x7f9de81443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37658/0 (socket says 172.21.15.31:37658) 2023-10-06T05:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de659c700 1 -- 172.21.15.31:0/1319936865 learned_addr learned my addr 172.21.15.31:0/1319936865 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de6d9d700 1 --2- 172.21.15.31:0/1319936865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9de8156300 0x7f9de80c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de5d9b700 1 --2- 172.21.15.31:0/1319936865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de806ea90 0x7f9de80c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de659c700 1 -- 172.21.15.31:0/1319936865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9de8156300 msgr2=0x7f9de80c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de659c700 1 --2- 172.21.15.31:0/1319936865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9de8156300 0x7f9de80c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de659c700 1 -- 172.21.15.31:0/1319936865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de806ea90 msgr2=0x7f9de80c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de659c700 1 --2- 172.21.15.31:0/1319936865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de806ea90 0x7f9de80c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de659c700 1 -- 172.21.15.31:0/1319936865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc0094f0 con 0x7f9de806a630 2023-10-06T05:24:23.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.406+0000 7f9de659c700 1 --2- 172.21.15.31:0/1319936865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9de806a630 0x7f9de81443a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800d840 tx=0x7f9dd8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.407+0000 7f9dd67fc700 1 -- 172.21.15.31:0/1319936865 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8014620 con 0x7f9de806a630 2023-10-06T05:24:23.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.407+0000 7f9dd67fc700 1 -- 172.21.15.31:0/1319936865 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd8014780 con 0x7f9de806a630 2023-10-06T05:24:23.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.407+0000 7f9dd67fc700 1 -- 172.21.15.31:0/1319936865 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8014b40 con 0x7f9de806a630 2023-10-06T05:24:23.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.407+0000 7f9decbff700 1 -- 172.21.15.31:0/1319936865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de80c5000 con 0x7f9de806a630 2023-10-06T05:24:23.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.407+0000 7f9decbff700 1 -- 172.21.15.31:0/1319936865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de81448e0 con 0x7f9de806a630 2023-10-06T05:24:23.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.408+0000 7f9decbff700 1 -- 172.21.15.31:0/1319936865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de80b0b10 con 0x7f9de806a630 2023-10-06T05:24:23.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.409+0000 7f9dd67fc700 1 -- 172.21.15.31:0/1319936865 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9dd801d400 con 0x7f9de806a630 2023-10-06T05:24:23.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.410+0000 7f9dd67fc700 1 --2- 172.21.15.31:0/1319936865 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9dc80674d0 0x7f9dc8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.410+0000 7f9dd67fc700 1 -- 172.21.15.31:0/1319936865 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9dd8085610 con 0x7f9de806a630 2023-10-06T05:24:23.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.410+0000 7f9de5d9b700 1 --2- 172.21.15.31:0/1319936865 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9dc80674d0 0x7f9dc8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.411+0000 7f9de5d9b700 1 --2- 172.21.15.31:0/1319936865 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9dc80674d0 0x7f9dc8069980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9de806d0e0 tx=0x7f9ddc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.414+0000 7f9dd67fc700 1 -- 172.21.15.31:0/1319936865 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dd8051e50 con 0x7f9de806a630 2023-10-06T05:24:23.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.657+0000 7f9decbff700 1 -- 172.21.15.31:0/1319936865 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9de80b0970 con 0x7f9dc80674d0 2023-10-06T05:24:23.660 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:23.659+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:23.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.660+0000 7f9dd67fc700 1 -- 172.21.15.31:0/1319936865 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f9ddc00bb30 con 0x7f9dc80674d0 2023-10-06T05:24:23.663 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:23.670 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:23.671 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2023-10-06T05:24:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.869+0000 7f4843dff700 1 Processor -- start 2023-10-06T05:24:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.869+0000 7f4843dff700 1 -- start start 2023-10-06T05:24:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.870+0000 7f4843dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f483c1550a0 0x7f483c0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.870+0000 7f4843dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c06ee10 0x7f483c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.870+0000 7f4843dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483c06f640 con 0x7f483c154cd0 2023-10-06T05:24:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.870+0000 7f4843dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f483c06e610 con 0x7f483c1550a0 2023-10-06T05:24:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.870+0000 7f4843dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483c0adf60 con 0x7f483c06ee10 2023-10-06T05:24:23.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.871+0000 7f484239c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c06ee10 0x7f483c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.871+0000 7f484139a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f483c1550a0 0x7f483c0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.871+0000 7f484239c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c06ee10 0x7f483c06d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:60410/0 (socket says 172.21.15.31:60410) 2023-10-06T05:24:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.871+0000 7f484239c700 1 -- 172.21.15.31:0/3075589128 learned_addr learned my addr 172.21.15.31:0/3075589128 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.871+0000 7f4841b9b700 1 --1- 172.21.15.31:0/3075589128 >> v1:172.21.15.62:6789/0 conn(0x7f483c154cd0 0x7f483c1583e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:60698/0 (socket says 172.21.15.31:60698) 2023-10-06T05:24:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.871+0000 7f484239c700 1 -- 172.21.15.31:0/3075589128 >> v2:172.21.15.62:3300/0 conn(0x7f483c1550a0 msgr2=0x7f483c0ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.871+0000 7f484239c700 1 --2- 172.21.15.31:0/3075589128 >> v2:172.21.15.62:3300/0 conn(0x7f483c1550a0 0x7f483c0ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.871+0000 7f484239c700 1 -- 172.21.15.31:0/3075589128 >> v1:172.21.15.62:6789/0 conn(0x7f483c154cd0 legacy=0x7f483c1583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.871+0000 7f484239c700 1 -- 172.21.15.31:0/3075589128 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c06f360 con 0x7f483c06ee10 2023-10-06T05:24:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.871+0000 7f484239c700 1 --2- 172.21.15.31:0/3075589128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c06ee10 0x7f483c06d7c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4834009f20 tx=0x7f4834009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529743474728647c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4840b99700 1 -- 172.21.15.31:0/3075589128 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483400e070 con 0x7f483c06ee10 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4840b99700 1 -- 172.21.15.31:0/3075589128 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483400b9d0 con 0x7f483c06ee10 2023-10-06T05:24:23.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4840b99700 1 -- 172.21.15.31:0/3075589128 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48340092e0 con 0x7f483c06ee10 2023-10-06T05:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4843dff700 1 -- 172.21.15.31:0/3075589128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c06ee10 msgr2=0x7f483c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4843dff700 1 --2- 172.21.15.31:0/3075589128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c06ee10 0x7f483c06d7c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4834009f20 tx=0x7f4834009840 comp rx=0 tx=0).stop 2023-10-06T05:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4843dff700 1 -- 172.21.15.31:0/3075589128 shutdown_connections 2023-10-06T05:24:23.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4843dff700 1 --2- 172.21.15.31:0/3075589128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c06ee10 0x7f483c06d7c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4843dff700 1 --2- 172.21.15.31:0/3075589128 >> v2:172.21.15.62:3300/0 conn(0x7f483c1550a0 0x7f483c0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4843dff700 1 -- 172.21.15.31:0/3075589128 >> 172.21.15.31:0/3075589128 conn(0x7f483c068030 msgr2=0x7f483c0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4843dff700 1 -- 172.21.15.31:0/3075589128 shutdown_connections 2023-10-06T05:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.872+0000 7f4843dff700 1 -- 172.21.15.31:0/3075589128 wait complete. 2023-10-06T05:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.874+0000 7f4843dff700 1 Processor -- start 2023-10-06T05:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.874+0000 7f4843dff700 1 -- start start 2023-10-06T05:24:23.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f4843dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c06ee10 0x7f483c15bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f4843dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c154cd0 0x7f483c14a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f4843dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f483c1550a0 0x7f483c0afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f4843dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c14ede0 con 0x7f483c06ee10 2023-10-06T05:24:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f4843dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c15d290 con 0x7f483c1550a0 2023-10-06T05:24:23.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f4843dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483c15c680 con 0x7f483c154cd0 2023-10-06T05:24:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f4841b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c06ee10 0x7f483c15bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f4841b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c06ee10 0x7f483c15bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37674/0 (socket says 172.21.15.31:37674) 2023-10-06T05:24:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f4841b9b700 1 -- 172.21.15.31:0/3581791117 learned_addr learned my addr 172.21.15.31:0/3581791117 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f484139a700 1 --2- 172.21.15.31:0/3581791117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c154cd0 0x7f483c14a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.875+0000 7f484239c700 1 --2- 172.21.15.31:0/3581791117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f483c1550a0 0x7f483c0afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4841b9b700 1 -- 172.21.15.31:0/3581791117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c154cd0 msgr2=0x7f483c14a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4841b9b700 1 --2- 172.21.15.31:0/3581791117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f483c154cd0 0x7f483c14a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4841b9b700 1 -- 172.21.15.31:0/3581791117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f483c1550a0 msgr2=0x7f483c0afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4841b9b700 1 --2- 172.21.15.31:0/3581791117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f483c1550a0 0x7f483c0afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4841b9b700 1 -- 172.21.15.31:0/3581791117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4834009ae0 con 0x7f483c06ee10 2023-10-06T05:24:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4841b9b700 1 --2- 172.21.15.31:0/3581791117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c06ee10 0x7f483c15bfa0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f483800c7c0 tx=0x7f4838007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4831ffb700 1 -- 172.21.15.31:0/3581791117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838005ac0 con 0x7f483c06ee10 2023-10-06T05:24:23.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4831ffb700 1 -- 172.21.15.31:0/3581791117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4838005c20 con 0x7f483c06ee10 2023-10-06T05:24:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4831ffb700 1 -- 172.21.15.31:0/3581791117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380070d0 con 0x7f483c06ee10 2023-10-06T05:24:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4843dff700 1 -- 172.21.15.31:0/3581791117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483c15c960 con 0x7f483c06ee10 2023-10-06T05:24:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.876+0000 7f4843dff700 1 -- 172.21.15.31:0/3581791117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483c15cf20 con 0x7f483c06ee10 2023-10-06T05:24:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.877+0000 7f4843dff700 1 -- 172.21.15.31:0/3581791117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483c04c380 con 0x7f483c06ee10 2023-10-06T05:24:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.878+0000 7f4831ffb700 1 -- 172.21.15.31:0/3581791117 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f4838007500 con 0x7f483c06ee10 2023-10-06T05:24:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.879+0000 7f4831ffb700 1 --2- 172.21.15.31:0/3581791117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f48200674d0 0x7f4820069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.879+0000 7f4831ffb700 1 -- 172.21.15.31:0/3581791117 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f4838084570 con 0x7f483c06ee10 2023-10-06T05:24:23.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.880+0000 7f484139a700 1 --2- 172.21.15.31:0/3581791117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f48200674d0 0x7f4820069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:23.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.880+0000 7f484139a700 1 --2- 172.21.15.31:0/3581791117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f48200674d0 0x7f4820069980 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f483c16caf0 tx=0x7f482c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:23.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:23.883+0000 7f4831ffb700 1 -- 172.21.15.31:0/3581791117 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4838050db0 con 0x7f483c06ee10 2023-10-06T05:24:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.121+0000 7f4843dff700 1 -- 172.21.15.31:0/3581791117 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f483c15d3d0 con 0x7f48200674d0 2023-10-06T05:24:24.124 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:24.123+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.123+0000 7f4831ffb700 1 -- 172.21.15.31:0/3581791117 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f482c009ad0 con 0x7f48200674d0 2023-10-06T05:24:24.126 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:24.134 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:24.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2023-10-06T05:24:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.337+0000 7f877f7ff700 1 Processor -- start 2023-10-06T05:24:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.338+0000 7f877f7ff700 1 -- start start 2023-10-06T05:24:24.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.339+0000 7f877f7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8778153d40 0x7f87780b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.339+0000 7f877f7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 0x7f87780af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.339+0000 7f877f7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87780ad690 con 0x7f87780af6c0 2023-10-06T05:24:24.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.339+0000 7f877f7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87780b0e10 con 0x7f8778153d40 2023-10-06T05:24:24.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.339+0000 7f877f7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87780acd60 con 0x7f87780b09c0 2023-10-06T05:24:24.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.339+0000 7f877d59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8778153d40 0x7f87780b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.339+0000 7f877d59b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8778153d40 0x7f87780b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37682/0 (socket says 172.21.15.31:37682) 2023-10-06T05:24:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.339+0000 7f877d59b700 1 -- 172.21.15.31:0/1886386848 learned_addr learned my addr 172.21.15.31:0/1886386848 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f877cd9a700 1 --2- 172.21.15.31:0/1886386848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 0x7f87780af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f877d59b700 1 -- 172.21.15.31:0/1886386848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 msgr2=0x7f87780af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f877d59b700 1 --2- 172.21.15.31:0/1886386848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 0x7f87780af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f877d59b700 1 -- 172.21.15.31:0/1886386848 >> v1:172.21.15.31:6789/0 conn(0x7f87780af6c0 legacy=0x7f8778159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f877d59b700 1 -- 172.21.15.31:0/1886386848 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87780ad040 con 0x7f8778153d40 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f876ffff700 1 -- 172.21.15.31:0/1886386848 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977858489 0 0) 0x7f87780ad690 con 0x7f87780af6c0 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f877cd9a700 1 --2- 172.21.15.31:0/1886386848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 0x7f87780af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f877d59b700 1 --2- 172.21.15.31:0/1886386848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8778153d40 0x7f87780b0480 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f87740099b0 tx=0x7f8774009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259b77cfed40bc00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f876ffff700 1 -- 172.21.15.31:0/1886386848 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877400b860 con 0x7f8778153d40 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f876ffff700 1 -- 172.21.15.31:0/1886386848 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877400b9c0 con 0x7f8778153d40 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.340+0000 7f876ffff700 1 -- 172.21.15.31:0/1886386848 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877400bd60 con 0x7f8778153d40 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.341+0000 7f877f7ff700 1 -- 172.21.15.31:0/1886386848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8778153d40 msgr2=0x7f87780b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.341+0000 7f877f7ff700 1 --2- 172.21.15.31:0/1886386848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8778153d40 0x7f87780b0480 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f87740099b0 tx=0x7f8774009250 comp rx=0 tx=0).stop 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.341+0000 7f877f7ff700 1 -- 172.21.15.31:0/1886386848 shutdown_connections 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.341+0000 7f877f7ff700 1 --2- 172.21.15.31:0/1886386848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8778153d40 0x7f87780b0480 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.341+0000 7f877f7ff700 1 --2- 172.21.15.31:0/1886386848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 0x7f87780af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.341+0000 7f877f7ff700 1 -- 172.21.15.31:0/1886386848 >> 172.21.15.31:0/1886386848 conn(0x7f8778067560 msgr2=0x7f8778067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.341+0000 7f877f7ff700 1 -- 172.21.15.31:0/1886386848 shutdown_connections 2023-10-06T05:24:24.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.341+0000 7f877f7ff700 1 -- 172.21.15.31:0/1886386848 wait complete. 2023-10-06T05:24:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.342+0000 7f877f7ff700 1 Processor -- start 2023-10-06T05:24:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.343+0000 7f877f7ff700 1 -- start start 2023-10-06T05:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.343+0000 7f877f7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87780af6c0 0x7f87781459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.343+0000 7f877f7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 0x7f8778145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.343+0000 7f877f7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8778153d40 0x7f8778146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.343+0000 7f877f7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8778146970 con 0x7f8778153d40 2023-10-06T05:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.343+0000 7f877f7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877814c780 con 0x7f87780af6c0 2023-10-06T05:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.343+0000 7f877f7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877814c1c0 con 0x7f87780b09c0 2023-10-06T05:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8778153d40 0x7f8778146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877dd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8778153d40 0x7f8778146430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37684/0 (socket says 172.21.15.31:37684) 2023-10-06T05:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877dd9c700 1 -- 172.21.15.31:0/3293984878 learned_addr learned my addr 172.21.15.31:0/3293984878 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877cd9a700 1 --2- 172.21.15.31:0/3293984878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 0x7f8778145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877dd9c700 1 -- 172.21.15.31:0/3293984878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 msgr2=0x7f8778145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877dd9c700 1 --2- 172.21.15.31:0/3293984878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 0x7f8778145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877dd9c700 1 -- 172.21.15.31:0/3293984878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87780af6c0 msgr2=0x7f87781459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877dd9c700 1 --2- 172.21.15.31:0/3293984878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87780af6c0 0x7f87781459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877dd9c700 1 -- 172.21.15.31:0/3293984878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87740094f0 con 0x7f8778153d40 2023-10-06T05:24:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877cd9a700 1 --2- 172.21.15.31:0/3293984878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87780b09c0 0x7f8778145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.344+0000 7f877dd9c700 1 --2- 172.21.15.31:0/3293984878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8778153d40 0x7f8778146430 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f877000c890 tx=0x7f8770001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.345+0000 7f876d7fa700 1 -- 172.21.15.31:0/3293984878 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770017070 con 0x7f8778153d40 2023-10-06T05:24:24.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.345+0000 7f876d7fa700 1 -- 172.21.15.31:0/3293984878 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8770006ba0 con 0x7f8778153d40 2023-10-06T05:24:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.345+0000 7f876d7fa700 1 -- 172.21.15.31:0/3293984878 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770012630 con 0x7f8778153d40 2023-10-06T05:24:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.345+0000 7f877f7ff700 1 -- 172.21.15.31:0/3293984878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877814c4a0 con 0x7f8778153d40 2023-10-06T05:24:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.345+0000 7f877f7ff700 1 -- 172.21.15.31:0/3293984878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87780c94b0 con 0x7f8778153d40 2023-10-06T05:24:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.346+0000 7f877f7ff700 1 -- 172.21.15.31:0/3293984878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877804c380 con 0x7f8778153d40 2023-10-06T05:24:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.347+0000 7f876d7fa700 1 -- 172.21.15.31:0/3293984878 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f8770012a20 con 0x7f8778153d40 2023-10-06T05:24:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.347+0000 7f876d7fa700 1 --2- 172.21.15.31:0/3293984878 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f875c0674d0 0x7f875c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.348+0000 7f876d7fa700 1 -- 172.21.15.31:0/3293984878 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f8770084f50 con 0x7f8778153d40 2023-10-06T05:24:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.348+0000 7f877d59b700 1 --2- 172.21.15.31:0/3293984878 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f875c0674d0 0x7f875c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.349+0000 7f877d59b700 1 --2- 172.21.15.31:0/3293984878 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f875c0674d0 0x7f875c069980 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f87740099b0 tx=0x7f8774017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.352+0000 7f876d7fa700 1 -- 172.21.15.31:0/3293984878 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8770051790 con 0x7f8778153d40 2023-10-06T05:24:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.594+0000 7f877f7ff700 1 -- 172.21.15.31:0/3293984878 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f877806b600 con 0x7f875c0674d0 2023-10-06T05:24:24.597 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:24.596+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.596+0000 7f876d7fa700 1 -- 172.21.15.31:0/3293984878 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f877806b600 con 0x7f875c0674d0 2023-10-06T05:24:24.599 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:24.606 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:24.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2023-10-06T05:24:24.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.804+0000 7fa1f25ff700 1 Processor -- start 2023-10-06T05:24:24.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.804+0000 7fa1f25ff700 1 -- start start 2023-10-06T05:24:24.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.805+0000 7fa1f25ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ec06b790 0x7fa1ec0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.805+0000 7fa1f25ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa1ec0ca5f0 0x7fa1ec0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.805+0000 7fa1f25ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec0b0970 con 0x7fa1ec06bb60 2023-10-06T05:24:24.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.805+0000 7fa1f25ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ec0af150 con 0x7fa1ec0ca5f0 2023-10-06T05:24:24.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.805+0000 7fa1f25ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec0c8690 con 0x7fa1ec06b790 2023-10-06T05:24:24.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.806+0000 7fa1f0b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa1ec06bb60 0x7fa1ec15a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41094/0 (socket says 172.21.15.31:41094) 2023-10-06T05:24:24.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.806+0000 7fa1f0b9c700 1 -- 172.21.15.31:0/3783462711 learned_addr learned my addr 172.21.15.31:0/3783462711 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:24.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.806+0000 7fa1ebfff700 1 --2- 172.21.15.31:0/3783462711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ec06b790 0x7fa1ec0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.806+0000 7fa1eb7fe700 1 --2- 172.21.15.31:0/3783462711 >> v2:172.21.15.62:3300/0 conn(0x7fa1ec0ca5f0 0x7fa1ec0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.806+0000 7fa1eaffd700 1 -- 172.21.15.31:0/3783462711 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850104510 0 0) 0x7fa1ec0b0970 con 0x7fa1ec06bb60 2023-10-06T05:24:24.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.806+0000 7fa1eaffd700 1 -- 172.21.15.31:0/3783462711 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d4003570 con 0x7fa1ec06bb60 2023-10-06T05:24:24.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.806+0000 7fa1ebfff700 1 -- 172.21.15.31:0/3783462711 >> v2:172.21.15.62:3300/0 conn(0x7fa1ec0ca5f0 msgr2=0x7fa1ec0c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:24.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.806+0000 7fa1ebfff700 1 --2- 172.21.15.31:0/3783462711 >> v2:172.21.15.62:3300/0 conn(0x7fa1ec0ca5f0 0x7fa1ec0c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.806+0000 7fa1ebfff700 1 -- 172.21.15.31:0/3783462711 >> v1:172.21.15.31:6789/0 conn(0x7fa1ec06bb60 legacy=0x7fa1ec15a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.806+0000 7fa1ebfff700 1 -- 172.21.15.31:0/3783462711 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec0c9bd0 con 0x7fa1ec06b790 2023-10-06T05:24:24.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.807+0000 7fa1ebfff700 1 --2- 172.21.15.31:0/3783462711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ec06b790 0x7fa1ec0b0430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0098e0 tx=0x7fa1dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8f4d67ca0a51b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.807+0000 7fa1eaffd700 1 -- 172.21.15.31:0/3783462711 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc011630 con 0x7fa1ec06b790 2023-10-06T05:24:24.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.807+0000 7fa1eaffd700 1 -- 172.21.15.31:0/3783462711 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc011790 con 0x7fa1ec06b790 2023-10-06T05:24:24.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.807+0000 7fa1eaffd700 1 -- 172.21.15.31:0/3783462711 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc011b10 con 0x7fa1ec06b790 2023-10-06T05:24:24.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.808+0000 7fa1f25ff700 1 -- 172.21.15.31:0/3783462711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ec06b790 msgr2=0x7fa1ec0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.808+0000 7fa1f25ff700 1 --2- 172.21.15.31:0/3783462711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ec06b790 0x7fa1ec0b0430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0098e0 tx=0x7fa1dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.808+0000 7fa1f25ff700 1 -- 172.21.15.31:0/3783462711 shutdown_connections 2023-10-06T05:24:24.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.808+0000 7fa1f25ff700 1 --2- 172.21.15.31:0/3783462711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ec06b790 0x7fa1ec0b0430 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.808+0000 7fa1f25ff700 1 --2- 172.21.15.31:0/3783462711 >> v2:172.21.15.62:3300/0 conn(0x7fa1ec0ca5f0 0x7fa1ec0c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.808+0000 7fa1f25ff700 1 -- 172.21.15.31:0/3783462711 >> 172.21.15.31:0/3783462711 conn(0x7fa1ec0ce290 msgr2=0x7fa1ec0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:24.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.808+0000 7fa1f25ff700 1 -- 172.21.15.31:0/3783462711 shutdown_connections 2023-10-06T05:24:24.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.808+0000 7fa1f25ff700 1 -- 172.21.15.31:0/3783462711 wait complete. 2023-10-06T05:24:24.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.810+0000 7fa1f25ff700 1 Processor -- start 2023-10-06T05:24:24.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.810+0000 7fa1f25ff700 1 -- start start 2023-10-06T05:24:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.811+0000 7fa1f25ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ec06b790 0x7fa1ec149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.811+0000 7fa1f25ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ec06bb60 0x7fa1ec0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.811+0000 7fa1f25ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec0ca5f0 0x7fa1ec0c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.811+0000 7fa1f25ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec16a700 con 0x7fa1ec06bb60 2023-10-06T05:24:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.811+0000 7fa1f25ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec0c6980 con 0x7fa1ec0ca5f0 2023-10-06T05:24:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.811+0000 7fa1f25ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec149bb0 con 0x7fa1ec06b790 2023-10-06T05:24:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1eb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ec06bb60 0x7fa1ec0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1eb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ec06bb60 0x7fa1ec0b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37698/0 (socket says 172.21.15.31:37698) 2023-10-06T05:24:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1eb7fe700 1 -- 172.21.15.31:0/2581266044 learned_addr learned my addr 172.21.15.31:0/2581266044 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1ebfff700 1 --2- 172.21.15.31:0/2581266044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ec06b790 0x7fa1ec149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1f0b9c700 1 --2- 172.21.15.31:0/2581266044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec0ca5f0 0x7fa1ec0c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1eb7fe700 1 -- 172.21.15.31:0/2581266044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ec06b790 msgr2=0x7fa1ec149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1eb7fe700 1 --2- 172.21.15.31:0/2581266044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ec06b790 0x7fa1ec149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1eb7fe700 1 -- 172.21.15.31:0/2581266044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec0ca5f0 msgr2=0x7fa1ec0c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:24.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1eb7fe700 1 --2- 172.21.15.31:0/2581266044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec0ca5f0 0x7fa1ec0c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:24.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1eb7fe700 1 -- 172.21.15.31:0/2581266044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc009420 con 0x7fa1ec06bb60 2023-10-06T05:24:24.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1f0b9c700 1 --2- 172.21.15.31:0/2581266044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec0ca5f0 0x7fa1ec0c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:24.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1eb7fe700 1 --2- 172.21.15.31:0/2581266044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ec06bb60 0x7fa1ec0b21c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000ec80 tx=0x7fa1e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1d3fff700 1 -- 172.21.15.31:0/2581266044 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000c9d0 con 0x7fa1ec06bb60 2023-10-06T05:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1d3fff700 1 -- 172.21.15.31:0/2581266044 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e000cb30 con 0x7fa1ec06bb60 2023-10-06T05:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.812+0000 7fa1d3fff700 1 -- 172.21.15.31:0/2581266044 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0013470 con 0x7fa1ec06bb60 2023-10-06T05:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.813+0000 7fa1f25ff700 1 -- 172.21.15.31:0/2581266044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec16a420 con 0x7fa1ec06bb60 2023-10-06T05:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.813+0000 7fa1f25ff700 1 -- 172.21.15.31:0/2581266044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec0c6f00 con 0x7fa1ec06bb60 2023-10-06T05:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.814+0000 7fa1f25ff700 1 -- 172.21.15.31:0/2581266044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec04c380 con 0x7fa1ec06bb60 2023-10-06T05:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.815+0000 7fa1d3fff700 1 -- 172.21.15.31:0/2581266044 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fa1e0013820 con 0x7fa1ec06bb60 2023-10-06T05:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.816+0000 7fa1d3fff700 1 --2- 172.21.15.31:0/2581266044 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa1d8067500 0x7fa1d80699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:24.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.816+0000 7fa1d3fff700 1 -- 172.21.15.31:0/2581266044 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fa1e0085480 con 0x7fa1ec06bb60 2023-10-06T05:24:24.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.816+0000 7fa1ebfff700 1 --2- 172.21.15.31:0/2581266044 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa1d8067500 0x7fa1d80699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:24.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.817+0000 7fa1ebfff700 1 --2- 172.21.15.31:0/2581266044 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa1d8067500 0x7fa1d80699b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0c4a40 tx=0x7fa1dc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:24.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:24.820+0000 7fa1d3fff700 1 -- 172.21.15.31:0/2581266044 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1e0051cc0 con 0x7fa1ec06bb60 2023-10-06T05:24:25.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.066+0000 7fa1f25ff700 1 -- 172.21.15.31:0/2581266044 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1ec0b0ba0 con 0x7fa1d8067500 2023-10-06T05:24:25.069 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:25.068+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.068+0000 7fa1d3fff700 1 -- 172.21.15.31:0/2581266044 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fa1ec0b0ba0 con 0x7fa1d8067500 2023-10-06T05:24:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:25.079 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:25.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2023-10-06T05:24:25.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.278+0000 7f0c44fff700 1 Processor -- start 2023-10-06T05:24:25.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.278+0000 7f0c44fff700 1 -- start start 2023-10-06T05:24:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.279+0000 7f0c44fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0c40159560 0x7f0c400cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.279+0000 7f0c44fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0c40159930 0x7f0c400cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.279+0000 7f0c44fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c400b0d60 0x7f0c400b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.279+0000 7f0c44fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4015d2d0 con 0x7f0c40159560 2023-10-06T05:24:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c44fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4015d5b0 con 0x7f0c40159930 2023-10-06T05:24:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c44fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4015d890 con 0x7f0c400b0d60 2023-10-06T05:24:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0c40159560 0x7f0c400cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0c40159560 0x7f0c400cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37714/0 (socket says 172.21.15.31:37714) 2023-10-06T05:24:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3e59c700 1 -- 172.21.15.31:0/2421990795 learned_addr learned my addr 172.21.15.31:0/2421990795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3ed9d700 1 --2- 172.21.15.31:0/2421990795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c400b0d60 0x7f0c400b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3dd9b700 1 --2- 172.21.15.31:0/2421990795 >> v2:172.21.15.62:3300/0 conn(0x7f0c40159930 0x7f0c400cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3e59c700 1 -- 172.21.15.31:0/2421990795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c400b0d60 msgr2=0x7f0c400b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:25.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3e59c700 1 --2- 172.21.15.31:0/2421990795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c400b0d60 0x7f0c400b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3e59c700 1 -- 172.21.15.31:0/2421990795 >> v2:172.21.15.62:3300/0 conn(0x7f0c40159930 msgr2=0x7f0c400cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:25.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3e59c700 1 --2- 172.21.15.31:0/2421990795 >> v2:172.21.15.62:3300/0 conn(0x7f0c40159930 0x7f0c400cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3e59c700 1 -- 172.21.15.31:0/2421990795 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4015db70 con 0x7f0c40159560 2023-10-06T05:24:25.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3dd9b700 1 --2- 172.21.15.31:0/2421990795 >> v2:172.21.15.62:3300/0 conn(0x7f0c40159930 0x7f0c400cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:25.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.280+0000 7f0c3e59c700 1 --2- 172.21.15.31:0/2421990795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c40159560 0x7f0c400cce70 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f0c40089160 tx=0x7f0c30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c527816534a570ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c3d59a700 1 -- 172.21.15.31:0/2421990795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000b860 con 0x7f0c40159560 2023-10-06T05:24:25.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c3d59a700 1 -- 172.21.15.31:0/2421990795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3000b9c0 con 0x7f0c40159560 2023-10-06T05:24:25.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c3d59a700 1 -- 172.21.15.31:0/2421990795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000bd60 con 0x7f0c40159560 2023-10-06T05:24:25.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c44fff700 1 -- 172.21.15.31:0/2421990795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c40159560 msgr2=0x7f0c400cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c44fff700 1 --2- 172.21.15.31:0/2421990795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c40159560 0x7f0c400cce70 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f0c40089160 tx=0x7f0c30009250 comp rx=0 tx=0).stop 2023-10-06T05:24:25.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c44fff700 1 -- 172.21.15.31:0/2421990795 shutdown_connections 2023-10-06T05:24:25.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c44fff700 1 --2- 172.21.15.31:0/2421990795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c400b0d60 0x7f0c400b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c44fff700 1 --2- 172.21.15.31:0/2421990795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c40159560 0x7f0c400cce70 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c44fff700 1 --2- 172.21.15.31:0/2421990795 >> v2:172.21.15.62:3300/0 conn(0x7f0c40159930 0x7f0c400cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c44fff700 1 -- 172.21.15.31:0/2421990795 >> 172.21.15.31:0/2421990795 conn(0x7f0c4006ec90 msgr2=0x7f0c4006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c44fff700 1 -- 172.21.15.31:0/2421990795 shutdown_connections 2023-10-06T05:24:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.281+0000 7f0c44fff700 1 -- 172.21.15.31:0/2421990795 wait complete. 2023-10-06T05:24:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.283+0000 7f0c44fff700 1 Processor -- start 2023-10-06T05:24:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.283+0000 7f0c44fff700 1 -- start start 2023-10-06T05:24:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.284+0000 7f0c44fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c400b0d60 0x7f0c400c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.284+0000 7f0c44fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c40159560 0x7f0c4016a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.284+0000 7f0c44fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c40159930 0x7f0c40150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.284+0000 7f0c44fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4016ae20 con 0x7f0c40159930 2023-10-06T05:24:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.284+0000 7f0c44fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4016d4d0 con 0x7f0c400b0d60 2023-10-06T05:24:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.284+0000 7f0c44fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4016d7b0 con 0x7f0c40159560 2023-10-06T05:24:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.284+0000 7f0c3ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c40159930 0x7f0c40150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.284+0000 7f0c3ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c40159930 0x7f0c40150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37728/0 (socket says 172.21.15.31:37728) 2023-10-06T05:24:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.284+0000 7f0c3ed9d700 1 -- 172.21.15.31:0/2859812386 learned_addr learned my addr 172.21.15.31:0/2859812386 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c3dd9b700 1 --2- 172.21.15.31:0/2859812386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c40159560 0x7f0c4016a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c3e59c700 1 --2- 172.21.15.31:0/2859812386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c400b0d60 0x7f0c400c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c3ed9d700 1 -- 172.21.15.31:0/2859812386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c40159560 msgr2=0x7f0c4016a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c3ed9d700 1 --2- 172.21.15.31:0/2859812386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c40159560 0x7f0c4016a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c3ed9d700 1 -- 172.21.15.31:0/2859812386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c400b0d60 msgr2=0x7f0c400c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c3ed9d700 1 --2- 172.21.15.31:0/2859812386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c400b0d60 0x7f0c400c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c3ed9d700 1 -- 172.21.15.31:0/2859812386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c300094f0 con 0x7f0c40159930 2023-10-06T05:24:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c3ed9d700 1 --2- 172.21.15.31:0/2859812386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c40159930 0x7f0c40150570 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0c38007f30 tx=0x7f0c380098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c2e7fc700 1 -- 172.21.15.31:0/2859812386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800ec40 con 0x7f0c40159930 2023-10-06T05:24:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c2e7fc700 1 -- 172.21.15.31:0/2859812386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3800eda0 con 0x7f0c40159930 2023-10-06T05:24:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c2e7fc700 1 -- 172.21.15.31:0/2859812386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38014670 con 0x7f0c40159930 2023-10-06T05:24:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c44fff700 1 -- 172.21.15.31:0/2859812386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4016da90 con 0x7f0c40159930 2023-10-06T05:24:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.285+0000 7f0c44fff700 1 -- 172.21.15.31:0/2859812386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c400ca200 con 0x7f0c40159930 2023-10-06T05:24:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.286+0000 7f0c44fff700 1 -- 172.21.15.31:0/2859812386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4004c380 con 0x7f0c40159930 2023-10-06T05:24:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.287+0000 7f0c2e7fc700 1 -- 172.21.15.31:0/2859812386 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0c38014a20 con 0x7f0c40159930 2023-10-06T05:24:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.289+0000 7f0c2e7fc700 1 --2- 172.21.15.31:0/2859812386 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0c200674d0 0x7f0c20069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.289+0000 7f0c2e7fc700 1 -- 172.21.15.31:0/2859812386 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f0c380857d0 con 0x7f0c40159930 2023-10-06T05:24:25.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.289+0000 7f0c3e59c700 1 --2- 172.21.15.31:0/2859812386 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0c200674d0 0x7f0c20069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.290+0000 7f0c3e59c700 1 --2- 172.21.15.31:0/2859812386 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0c200674d0 0x7f0c20069980 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0c40089160 tx=0x7f0c30017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.292+0000 7f0c2e7fc700 1 -- 172.21.15.31:0/2859812386 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c380520d0 con 0x7f0c40159930 2023-10-06T05:24:25.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.522+0000 7f0c44fff700 1 -- 172.21.15.31:0/2859812386 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c4015de50 con 0x7f0c200674d0 2023-10-06T05:24:25.526 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:25.524+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:25.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.524+0000 7f0c2e7fc700 1 -- 172.21.15.31:0/2859812386 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f0c4015de50 con 0x7f0c200674d0 2023-10-06T05:24:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:25.535 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:25.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2023-10-06T05:24:25.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.732+0000 7fc30cfff700 1 Processor -- start 2023-10-06T05:24:25.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.732+0000 7fc30cfff700 1 -- start start 2023-10-06T05:24:25.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.733+0000 7fc30cfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc308159560 0x7fc3080cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.733+0000 7fc30cfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3080b2b80 0x7fc3080cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.733+0000 7fc30cfff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30815d2d0 con 0x7fc308159930 2023-10-06T05:24:25.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.733+0000 7fc30cfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30815d5b0 con 0x7fc3080b2b80 2023-10-06T05:24:25.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.733+0000 7fc30cfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30815d890 con 0x7fc308159560 2023-10-06T05:24:25.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.733+0000 7fc306d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3080b2b80 0x7fc3080cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.733+0000 7fc306d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3080b2b80 0x7fc3080cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37738/0 (socket says 172.21.15.31:37738) 2023-10-06T05:24:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.733+0000 7fc306d9d700 1 -- 172.21.15.31:0/18123417 learned_addr learned my addr 172.21.15.31:0/18123417 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.733+0000 7fc30659c700 1 --2- 172.21.15.31:0/18123417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc308159560 0x7fc3080cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.734+0000 7fc306d9d700 1 -- 172.21.15.31:0/18123417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc308159560 msgr2=0x7fc3080cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.734+0000 7fc306d9d700 1 --2- 172.21.15.31:0/18123417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc308159560 0x7fc3080cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.734+0000 7fc306d9d700 1 -- 172.21.15.31:0/18123417 >> v1:172.21.15.62:6789/0 conn(0x7fc308159930 legacy=0x7fc3080b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.734+0000 7fc306d9d700 1 -- 172.21.15.31:0/18123417 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30815db70 con 0x7fc3080b2b80 2023-10-06T05:24:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.734+0000 7fc306d9d700 1 --2- 172.21.15.31:0/18123417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3080b2b80 0x7fc3080cd3b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc300009f20 tx=0x7fc300009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d12404dfe56c06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.734+0000 7fc30559a700 1 -- 172.21.15.31:0/18123417 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000e070 con 0x7fc3080b2b80 2023-10-06T05:24:25.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.734+0000 7fc30559a700 1 -- 172.21.15.31:0/18123417 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30000ba80 con 0x7fc3080b2b80 2023-10-06T05:24:25.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.734+0000 7fc30559a700 1 -- 172.21.15.31:0/18123417 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3000092e0 con 0x7fc3080b2b80 2023-10-06T05:24:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.735+0000 7fc30cfff700 1 -- 172.21.15.31:0/18123417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3080b2b80 msgr2=0x7fc3080cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.735+0000 7fc30cfff700 1 --2- 172.21.15.31:0/18123417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3080b2b80 0x7fc3080cd3b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc300009f20 tx=0x7fc300009840 comp rx=0 tx=0).stop 2023-10-06T05:24:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.735+0000 7fc30cfff700 1 -- 172.21.15.31:0/18123417 shutdown_connections 2023-10-06T05:24:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.735+0000 7fc30cfff700 1 --2- 172.21.15.31:0/18123417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3080b2b80 0x7fc3080cd3b0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.735+0000 7fc30cfff700 1 --2- 172.21.15.31:0/18123417 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc308159560 0x7fc3080cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.735+0000 7fc30cfff700 1 -- 172.21.15.31:0/18123417 >> 172.21.15.31:0/18123417 conn(0x7fc30806ec90 msgr2=0x7fc30806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.735+0000 7fc30cfff700 1 -- 172.21.15.31:0/18123417 shutdown_connections 2023-10-06T05:24:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.735+0000 7fc30cfff700 1 -- 172.21.15.31:0/18123417 wait complete. 2023-10-06T05:24:25.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.736+0000 7fc30cfff700 1 Processor -- start 2023-10-06T05:24:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.737+0000 7fc30cfff700 1 -- start start 2023-10-06T05:24:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.737+0000 7fc30cfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3080b2b80 0x7fc308155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.737+0000 7fc30cfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc308159560 0x7fc3080cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30cfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308159930 0x7fc3080cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30cfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30816e180 con 0x7fc3080b2b80 2023-10-06T05:24:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30cfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30815c3b0 con 0x7fc308159930 2023-10-06T05:24:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30cfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30816d3c0 con 0x7fc308159560 2023-10-06T05:24:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3080b2b80 0x7fc308155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3080b2b80 0x7fc308155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37746/0 (socket says 172.21.15.31:37746) 2023-10-06T05:24:25.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30659c700 1 -- 172.21.15.31:0/3261363570 learned_addr learned my addr 172.21.15.31:0/3261363570 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc306d9d700 1 --2- 172.21.15.31:0/3261363570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308159930 0x7fc3080cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc305d9b700 1 --2- 172.21.15.31:0/3261363570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc308159560 0x7fc3080cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30659c700 1 -- 172.21.15.31:0/3261363570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc308159560 msgr2=0x7fc3080cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30659c700 1 --2- 172.21.15.31:0/3261363570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc308159560 0x7fc3080cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30659c700 1 -- 172.21.15.31:0/3261363570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308159930 msgr2=0x7fc3080cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30659c700 1 --2- 172.21.15.31:0/3261363570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308159930 0x7fc3080cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30659c700 1 -- 172.21.15.31:0/3261363570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300009ae0 con 0x7fc3080b2b80 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc305d9b700 1 --2- 172.21.15.31:0/3261363570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc308159560 0x7fc3080cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.738+0000 7fc30659c700 1 --2- 172.21.15.31:0/3261363570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3080b2b80 0x7fc308155a90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800d890 tx=0x7fc2f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.739+0000 7fc2f67fc700 1 -- 172.21.15.31:0/3261363570 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80125f0 con 0x7fc3080b2b80 2023-10-06T05:24:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.739+0000 7fc2f67fc700 1 -- 172.21.15.31:0/3261363570 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f8012770 con 0x7fc3080b2b80 2023-10-06T05:24:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.739+0000 7fc2f67fc700 1 -- 172.21.15.31:0/3261363570 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8012b30 con 0x7fc3080b2b80 2023-10-06T05:24:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.739+0000 7fc30cfff700 1 -- 172.21.15.31:0/3261363570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30816d6a0 con 0x7fc3080b2b80 2023-10-06T05:24:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.739+0000 7fc30cfff700 1 -- 172.21.15.31:0/3261363570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30816dc60 con 0x7fc3080b2b80 2023-10-06T05:24:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.740+0000 7fc30cfff700 1 -- 172.21.15.31:0/3261363570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30804c380 con 0x7fc3080b2b80 2023-10-06T05:24:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.741+0000 7fc2f67fc700 1 -- 172.21.15.31:0/3261363570 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fc2f801e460 con 0x7fc3080b2b80 2023-10-06T05:24:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.742+0000 7fc2f67fc700 1 --2- 172.21.15.31:0/3261363570 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc2e80674d0 0x7fc2e8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.742+0000 7fc2f67fc700 1 -- 172.21.15.31:0/3261363570 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc2f8085760 con 0x7fc3080b2b80 2023-10-06T05:24:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.743+0000 7fc305d9b700 1 --2- 172.21.15.31:0/3261363570 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc2e80674d0 0x7fc2e8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:25.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.744+0000 7fc305d9b700 1 --2- 172.21.15.31:0/3261363570 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc2e80674d0 0x7fc2e8069980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc007fd0 tx=0x7fc2fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:25.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.747+0000 7fc2f67fc700 1 -- 172.21.15.31:0/3261363570 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2f8051fa0 con 0x7fc3080b2b80 2023-10-06T05:24:25.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.981+0000 7fc30cfff700 1 -- 172.21.15.31:0/3261363570 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc30815de50 con 0x7fc2e80674d0 2023-10-06T05:24:25.984 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:25.983+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:25.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:25.983+0000 7fc2f67fc700 1 -- 172.21.15.31:0/3261363570 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fc30815de50 con 0x7fc2e80674d0 2023-10-06T05:24:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:25.994 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:25.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2023-10-06T05:24:26.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.193+0000 7f2bf09ff700 1 Processor -- start 2023-10-06T05:24:26.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.193+0000 7f2bf09ff700 1 -- start start 2023-10-06T05:24:26.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.194+0000 7f2bf09ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bec159930 0x7f2bec0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.194+0000 7f2bf09ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2bec0cd580 0x7f2bec0b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.194+0000 7f2bf09ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec15d2d0 con 0x7f2bec159560 2023-10-06T05:24:26.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.194+0000 7f2bf09ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec15d5b0 con 0x7f2bec0cd580 2023-10-06T05:24:26.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.194+0000 7f2bf09ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec15d890 con 0x7f2bec159930 2023-10-06T05:24:26.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.194+0000 7f2bea59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2bec159560 0x7f2bec0cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41104/0 (socket says 172.21.15.31:41104) 2023-10-06T05:24:26.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.194+0000 7f2bea59c700 1 -- 172.21.15.31:0/4006531795 learned_addr learned my addr 172.21.15.31:0/4006531795 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:26.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.194+0000 7f2bead9d700 1 --2- 172.21.15.31:0/4006531795 >> v2:172.21.15.62:3300/0 conn(0x7f2bec0cd580 0x7f2bec0b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.194+0000 7f2be9d9b700 1 --2- 172.21.15.31:0/4006531795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bec159930 0x7f2bec0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.195+0000 7f2be959a700 1 -- 172.21.15.31:0/4006531795 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113168873 0 0) 0x7f2bec15d2d0 con 0x7f2bec159560 2023-10-06T05:24:26.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.195+0000 7f2be959a700 1 -- 172.21.15.31:0/4006531795 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd4003570 con 0x7f2bec159560 2023-10-06T05:24:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.195+0000 7f2bead9d700 1 -- 172.21.15.31:0/4006531795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bec159930 msgr2=0x7f2bec0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.195+0000 7f2bead9d700 1 --2- 172.21.15.31:0/4006531795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bec159930 0x7f2bec0b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.195+0000 7f2bead9d700 1 -- 172.21.15.31:0/4006531795 >> v1:172.21.15.31:6789/0 conn(0x7f2bec159560 legacy=0x7f2bec0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.195+0000 7f2bead9d700 1 -- 172.21.15.31:0/4006531795 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec15db70 con 0x7f2bec0cd580 2023-10-06T05:24:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.195+0000 7f2be9d9b700 1 --2- 172.21.15.31:0/4006531795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bec159930 0x7f2bec0b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:26.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.195+0000 7f2bead9d700 1 --2- 172.21.15.31:0/4006531795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bec0cd580 0x7f2bec0b29b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009e60 tx=0x7f2be4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9addbc873b944011 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2be959a700 1 -- 172.21.15.31:0/4006531795 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400e070 con 0x7f2bec0cd580 2023-10-06T05:24:26.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2be959a700 1 -- 172.21.15.31:0/4006531795 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be400bab0 con 0x7f2bec0cd580 2023-10-06T05:24:26.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2be959a700 1 -- 172.21.15.31:0/4006531795 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400be70 con 0x7f2bec0cd580 2023-10-06T05:24:26.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2bf09ff700 1 -- 172.21.15.31:0/4006531795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bec0cd580 msgr2=0x7f2bec0b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2bf09ff700 1 --2- 172.21.15.31:0/4006531795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bec0cd580 0x7f2bec0b29b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009e60 tx=0x7f2be4009780 comp rx=0 tx=0).stop 2023-10-06T05:24:26.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2bf09ff700 1 -- 172.21.15.31:0/4006531795 shutdown_connections 2023-10-06T05:24:26.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2bf09ff700 1 --2- 172.21.15.31:0/4006531795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bec0cd580 0x7f2bec0b29b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2bf09ff700 1 --2- 172.21.15.31:0/4006531795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bec159930 0x7f2bec0b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2bf09ff700 1 -- 172.21.15.31:0/4006531795 >> 172.21.15.31:0/4006531795 conn(0x7f2bec06ec90 msgr2=0x7f2bec06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:26.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2bf09ff700 1 -- 172.21.15.31:0/4006531795 shutdown_connections 2023-10-06T05:24:26.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.196+0000 7f2bf09ff700 1 -- 172.21.15.31:0/4006531795 wait complete. 2023-10-06T05:24:26.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.198+0000 7f2bf09ff700 1 Processor -- start 2023-10-06T05:24:26.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.198+0000 7f2bf09ff700 1 -- start start 2023-10-06T05:24:26.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2bf09ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bec0cd580 0x7f2bec0c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2bf09ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec159560 0x7f2bec0c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2bf09ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bec159930 0x7f2bec0c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2bf09ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0cf6d0 con 0x7f2bec159560 2023-10-06T05:24:26.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2bf09ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0ceac0 con 0x7f2bec0cd580 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2bf09ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0ceda0 con 0x7f2bec159930 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2be9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec159560 0x7f2bec0c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2be9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec159560 0x7f2bec0c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37752/0 (socket says 172.21.15.31:37752) 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2be9d9b700 1 -- 172.21.15.31:0/3066322736 learned_addr learned my addr 172.21.15.31:0/3066322736 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2bead9d700 1 --2- 172.21.15.31:0/3066322736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bec159930 0x7f2bec0c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2bea59c700 1 --2- 172.21.15.31:0/3066322736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bec0cd580 0x7f2bec0c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.199+0000 7f2be9d9b700 1 -- 172.21.15.31:0/3066322736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bec159930 msgr2=0x7f2bec0c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2be9d9b700 1 --2- 172.21.15.31:0/3066322736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bec159930 0x7f2bec0c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2be9d9b700 1 -- 172.21.15.31:0/3066322736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bec0cd580 msgr2=0x7f2bec0c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2be9d9b700 1 --2- 172.21.15.31:0/3066322736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bec0cd580 0x7f2bec0c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2be9d9b700 1 -- 172.21.15.31:0/3066322736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be0009420 con 0x7f2bec159560 2023-10-06T05:24:26.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2bea59c700 1 --2- 172.21.15.31:0/3066322736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bec0cd580 0x7f2bec0c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:26.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2be9d9b700 1 --2- 172.21.15.31:0/3066322736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec159560 0x7f2bec0c4580 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2be000ec80 tx=0x7f2be00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2bda7fc700 1 -- 172.21.15.31:0/3066322736 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be000c9d0 con 0x7f2bec159560 2023-10-06T05:24:26.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2bda7fc700 1 -- 172.21.15.31:0/3066322736 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be000cb30 con 0x7f2bec159560 2023-10-06T05:24:26.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2bda7fc700 1 -- 172.21.15.31:0/3066322736 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be000ceb0 con 0x7f2bec159560 2023-10-06T05:24:26.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2bf09ff700 1 -- 172.21.15.31:0/3066322736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be4009a20 con 0x7f2bec159560 2023-10-06T05:24:26.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.200+0000 7f2bf09ff700 1 -- 172.21.15.31:0/3066322736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec0cf360 con 0x7f2bec159560 2023-10-06T05:24:26.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.201+0000 7f2bf09ff700 1 -- 172.21.15.31:0/3066322736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec04c380 con 0x7f2bec159560 2023-10-06T05:24:26.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.202+0000 7f2bda7fc700 1 -- 172.21.15.31:0/3066322736 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f2be0013680 con 0x7f2bec159560 2023-10-06T05:24:26.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.203+0000 7f2bda7fc700 1 --2- 172.21.15.31:0/3066322736 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2bcc0674d0 0x7f2bcc069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.203+0000 7f2bda7fc700 1 -- 172.21.15.31:0/3066322736 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f2be0085550 con 0x7f2bec159560 2023-10-06T05:24:26.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.204+0000 7f2bea59c700 1 --2- 172.21.15.31:0/3066322736 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2bcc0674d0 0x7f2bcc069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.204+0000 7f2bea59c700 1 --2- 172.21.15.31:0/3066322736 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2bcc0674d0 0x7f2bcc069980 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc007fd0 tx=0x7f2bdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.207+0000 7f2bda7fc700 1 -- 172.21.15.31:0/3066322736 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2be0051d90 con 0x7f2bec159560 2023-10-06T05:24:26.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.440+0000 7f2bf09ff700 1 -- 172.21.15.31:0/3066322736 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bec0cf810 con 0x7f2bcc0674d0 2023-10-06T05:24:26.443 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:26.442+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:26.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.442+0000 7f2bda7fc700 1 -- 172.21.15.31:0/3066322736 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7f2bec0cf810 con 0x7f2bcc0674d0 2023-10-06T05:24:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:26.453 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:26.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2023-10-06T05:24:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.651+0000 7fdd13fff700 1 Processor -- start 2023-10-06T05:24:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.652+0000 7fdd13fff700 1 -- start start 2023-10-06T05:24:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.653+0000 7fdd13fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd0c06df30 0x7fdd0c06d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.653+0000 7fdd13fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd0c06d830 0x7fdd0c159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.653+0000 7fdd13fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c0c72b0 con 0x7fdd0c06e300 2023-10-06T05:24:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.653+0000 7fdd13fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0c0aba60 con 0x7fdd0c06d830 2023-10-06T05:24:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.653+0000 7fdd13fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0c0c7590 con 0x7fdd0c06df30 2023-10-06T05:24:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.653+0000 7fdd1159a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd0c06d830 0x7fdd0c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.653+0000 7fdd1159a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd0c06d830 0x7fdd0c159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37758/0 (socket says 172.21.15.31:37758) 2023-10-06T05:24:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.653+0000 7fdd1159a700 1 -- 172.21.15.31:0/132124179 learned_addr learned my addr 172.21.15.31:0/132124179 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.653+0000 7fdd11d9b700 1 --2- 172.21.15.31:0/132124179 >> v2:172.21.15.62:3300/0 conn(0x7fdd0c06df30 0x7fdd0c06d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.654+0000 7fdd1159a700 1 -- 172.21.15.31:0/132124179 >> v2:172.21.15.62:3300/0 conn(0x7fdd0c06df30 msgr2=0x7fdd0c06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.654+0000 7fdd1159a700 1 --2- 172.21.15.31:0/132124179 >> v2:172.21.15.62:3300/0 conn(0x7fdd0c06df30 0x7fdd0c06d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.654+0000 7fdd1159a700 1 -- 172.21.15.31:0/132124179 >> v1:172.21.15.62:6789/0 conn(0x7fdd0c06e300 legacy=0x7fdd0c152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.654+0000 7fdd1159a700 1 -- 172.21.15.31:0/132124179 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c0c7870 con 0x7fdd0c06d830 2023-10-06T05:24:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.654+0000 7fdd10d99700 1 -- 172.21.15.31:0/132124179 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316592934 0 0) 0x7fdd0c0c72b0 con 0x7fdd0c06e300 2023-10-06T05:24:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.654+0000 7fdd1159a700 1 --2- 172.21.15.31:0/132124179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c06d830 0x7fdd0c159660 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0cbfc0 tx=0x7fdcfc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c73cc245af20baa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.654+0000 7fdd10d99700 1 -- 172.21.15.31:0/132124179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc009e70 con 0x7fdd0c06d830 2023-10-06T05:24:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.654+0000 7fdd10d99700 1 -- 172.21.15.31:0/132124179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc00b860 con 0x7fdd0c06d830 2023-10-06T05:24:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.654+0000 7fdd10d99700 1 -- 172.21.15.31:0/132124179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc00bc00 con 0x7fdd0c06d830 2023-10-06T05:24:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.655+0000 7fdd13fff700 1 -- 172.21.15.31:0/132124179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c06d830 msgr2=0x7fdd0c159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.655+0000 7fdd13fff700 1 --2- 172.21.15.31:0/132124179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c06d830 0x7fdd0c159660 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0cbfc0 tx=0x7fdcfc009250 comp rx=0 tx=0).stop 2023-10-06T05:24:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.655+0000 7fdd13fff700 1 -- 172.21.15.31:0/132124179 shutdown_connections 2023-10-06T05:24:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.655+0000 7fdd13fff700 1 --2- 172.21.15.31:0/132124179 >> v2:172.21.15.62:3300/0 conn(0x7fdd0c06df30 0x7fdd0c06d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.655+0000 7fdd13fff700 1 --2- 172.21.15.31:0/132124179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c06d830 0x7fdd0c159660 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.655+0000 7fdd13fff700 1 -- 172.21.15.31:0/132124179 >> 172.21.15.31:0/132124179 conn(0x7fdd0c068030 msgr2=0x7fdd0c0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.655+0000 7fdd13fff700 1 -- 172.21.15.31:0/132124179 shutdown_connections 2023-10-06T05:24:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.655+0000 7fdd13fff700 1 -- 172.21.15.31:0/132124179 wait complete. 2023-10-06T05:24:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.656+0000 7fdd13fff700 1 Processor -- start 2023-10-06T05:24:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.657+0000 7fdd13fff700 1 -- start start 2023-10-06T05:24:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.657+0000 7fdd13fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd0c06d830 0x7fdd0c0bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd13fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c06df30 0x7fdd0c14b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c06e300 0x7fdd0c158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd13fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0b0cf0 con 0x7fdd0c06e300 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd13fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0c0020 con 0x7fdd0c06d830 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd13fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0afe40 con 0x7fdd0c06df30 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd1259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c06e300 0x7fdd0c158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd1259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c06e300 0x7fdd0c158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37770/0 (socket says 172.21.15.31:37770) 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd1259c700 1 -- 172.21.15.31:0/2335268039 learned_addr learned my addr 172.21.15.31:0/2335268039 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd11d9b700 1 --2- 172.21.15.31:0/2335268039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd0c06d830 0x7fdd0c0bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd1159a700 1 --2- 172.21.15.31:0/2335268039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c06df30 0x7fdd0c14b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd1259c700 1 -- 172.21.15.31:0/2335268039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c06df30 msgr2=0x7fdd0c14b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd1259c700 1 --2- 172.21.15.31:0/2335268039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd0c06df30 0x7fdd0c14b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd1259c700 1 -- 172.21.15.31:0/2335268039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd0c06d830 msgr2=0x7fdd0c0bfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd1259c700 1 --2- 172.21.15.31:0/2335268039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd0c06d830 0x7fdd0c0bfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.658+0000 7fdd1259c700 1 -- 172.21.15.31:0/2335268039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc0094f0 con 0x7fdd0c06e300 2023-10-06T05:24:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.659+0000 7fdd1259c700 1 --2- 172.21.15.31:0/2335268039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd0c06e300 0x7fdd0c158650 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400c890 tx=0x7fdd04001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.659+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2335268039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04012630 con 0x7fdd0c06e300 2023-10-06T05:24:26.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.659+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2335268039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd040127b0 con 0x7fdd0c06e300 2023-10-06T05:24:26.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.659+0000 7fdd13fff700 1 -- 172.21.15.31:0/2335268039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c0b00e0 con 0x7fdd0c06e300 2023-10-06T05:24:26.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.659+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2335268039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04012b70 con 0x7fdd0c06e300 2023-10-06T05:24:26.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.659+0000 7fdd13fff700 1 -- 172.21.15.31:0/2335268039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c0b06a0 con 0x7fdd0c06e300 2023-10-06T05:24:26.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.660+0000 7fdd13fff700 1 -- 172.21.15.31:0/2335268039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0c0b0e60 con 0x7fdd0c06e300 2023-10-06T05:24:26.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.661+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2335268039 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fdd04003030 con 0x7fdd0c06e300 2023-10-06T05:24:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.662+0000 7fdd01ffb700 1 --2- 172.21.15.31:0/2335268039 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcf00675a0 0x7fdcf0069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.662+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2335268039 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fdd04085020 con 0x7fdd0c06e300 2023-10-06T05:24:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.662+0000 7fdd11d9b700 1 --2- 172.21.15.31:0/2335268039 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcf00675a0 0x7fdcf0069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.663+0000 7fdd11d9b700 1 --2- 172.21.15.31:0/2335268039 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcf00675a0 0x7fdcf0069a50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800ae10 tx=0x7fdd0800a2e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.666+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2335268039 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd040518a0 con 0x7fdd0c06e300 2023-10-06T05:24:26.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.901+0000 7fdd13fff700 1 -- 172.21.15.31:0/2335268039 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd0c04c380 con 0x7fdcf00675a0 2023-10-06T05:24:26.905 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:26.903+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:26.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:26.904+0000 7fdd01ffb700 1 -- 172.21.15.31:0/2335268039 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fdd0c04c380 con 0x7fdcf00675a0 2023-10-06T05:24:26.907 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:26.914 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:26.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2023-10-06T05:24:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.110+0000 7fd49cdff700 1 Processor -- start 2023-10-06T05:24:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.111+0000 7fd49cdff700 1 -- start start 2023-10-06T05:24:27.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.112+0000 7fd49cdff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd498159890 0x7fd4980c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd49cdff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd498158190 con 0x7fd498159c60 2023-10-06T05:24:27.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd49cdff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd498157580 con 0x7fd498155430 2023-10-06T05:24:27.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd49cdff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd498157860 con 0x7fd498159890 2023-10-06T05:24:27.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd49659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd498159890 0x7fd4980c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd49659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd498159890 0x7fd4980c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59334/0 (socket says 172.21.15.31:59334) 2023-10-06T05:24:27.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd49659c700 1 -- 172.21.15.31:0/916670893 learned_addr learned my addr 172.21.15.31:0/916670893 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd495d9b700 1 --1- 172.21.15.31:0/916670893 >> v1:172.21.15.31:6789/0 conn(0x7fd498159c60 0x7fd498156a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33708/0 (socket says 172.21.15.31:33708) 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd49659c700 1 -- 172.21.15.31:0/916670893 >> v1:172.21.15.62:6789/0 conn(0x7fd498155430 legacy=0x7fd498155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd49659c700 1 -- 172.21.15.31:0/916670893 >> v1:172.21.15.31:6789/0 conn(0x7fd498159c60 legacy=0x7fd498156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd49659c700 1 -- 172.21.15.31:0/916670893 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd498157b40 con 0x7fd498159890 2023-10-06T05:24:27.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.113+0000 7fd49559a700 1 -- 172.21.15.31:0/916670893 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803603056 0 0) 0x7fd498158190 con 0x7fd498159c60 2023-10-06T05:24:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.114+0000 7fd49659c700 1 --2- 172.21.15.31:0/916670893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498159890 0x7fd4980c89e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd4880099b0 tx=0x7fd488009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0b2bb57e9b4efd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.114+0000 7fd49559a700 1 -- 172.21.15.31:0/916670893 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800b860 con 0x7fd498159890 2023-10-06T05:24:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.114+0000 7fd49559a700 1 -- 172.21.15.31:0/916670893 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800b9c0 con 0x7fd498159890 2023-10-06T05:24:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.114+0000 7fd49559a700 1 -- 172.21.15.31:0/916670893 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800bd60 con 0x7fd498159890 2023-10-06T05:24:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.114+0000 7fd49cdff700 1 -- 172.21.15.31:0/916670893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498159890 msgr2=0x7fd4980c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.114+0000 7fd49cdff700 1 --2- 172.21.15.31:0/916670893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498159890 0x7fd4980c89e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd4880099b0 tx=0x7fd488009250 comp rx=0 tx=0).stop 2023-10-06T05:24:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.114+0000 7fd49cdff700 1 -- 172.21.15.31:0/916670893 shutdown_connections 2023-10-06T05:24:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.114+0000 7fd49cdff700 1 --2- 172.21.15.31:0/916670893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498159890 0x7fd4980c89e0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.114+0000 7fd49cdff700 1 -- 172.21.15.31:0/916670893 >> 172.21.15.31:0/916670893 conn(0x7fd498067560 msgr2=0x7fd498067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.114+0000 7fd49cdff700 1 -- 172.21.15.31:0/916670893 shutdown_connections 2023-10-06T05:24:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.115+0000 7fd49cdff700 1 -- 172.21.15.31:0/916670893 wait complete. 2023-10-06T05:24:27.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.117+0000 7fd49cdff700 1 Processor -- start 2023-10-06T05:24:27.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.117+0000 7fd49cdff700 1 -- start start 2023-10-06T05:24:27.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.118+0000 7fd49cdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498155430 0x7fd498163a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.118+0000 7fd49cdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498159890 0x7fd498147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.118+0000 7fd49cdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd498159c60 0x7fd498154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.118+0000 7fd49cdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd498163fc0 con 0x7fd498155430 2023-10-06T05:24:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.118+0000 7fd49cdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd498141940 con 0x7fd498159890 2023-10-06T05:24:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.118+0000 7fd49cdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4981482c0 con 0x7fd498159c60 2023-10-06T05:24:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.118+0000 7fd49659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498155430 0x7fd498163a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.118+0000 7fd49659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498155430 0x7fd498163a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59348/0 (socket says 172.21.15.31:59348) 2023-10-06T05:24:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.118+0000 7fd49659c700 1 -- 172.21.15.31:0/4119822610 learned_addr learned my addr 172.21.15.31:0/4119822610 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd495d9b700 1 --2- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498159890 0x7fd498147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd496d9d700 1 --2- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd498159c60 0x7fd498154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd49659c700 1 -- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd498159c60 msgr2=0x7fd498154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd49659c700 1 --2- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd498159c60 0x7fd498154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd49659c700 1 -- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498159890 msgr2=0x7fd498147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd49659c700 1 --2- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498159890 0x7fd498147870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd49659c700 1 -- 172.21.15.31:0/4119822610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4880094f0 con 0x7fd498155430 2023-10-06T05:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd495d9b700 1 --2- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498159890 0x7fd498147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd496d9d700 1 --2- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd498159c60 0x7fd498154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:27.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd49659c700 1 --2- 172.21.15.31:0/4119822610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498155430 0x7fd498163a80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd488006f50 tx=0x7fd488017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd4867fc700 1 -- 172.21.15.31:0/4119822610 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48801f070 con 0x7fd498155430 2023-10-06T05:24:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd4867fc700 1 -- 172.21.15.31:0/4119822610 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd488011d10 con 0x7fd498155430 2023-10-06T05:24:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd4867fc700 1 -- 172.21.15.31:0/4119822610 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488025670 con 0x7fd498155430 2023-10-06T05:24:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.119+0000 7fd49cdff700 1 -- 172.21.15.31:0/4119822610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4981485a0 con 0x7fd498155430 2023-10-06T05:24:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.120+0000 7fd49cdff700 1 -- 172.21.15.31:0/4119822610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd498148b60 con 0x7fd498155430 2023-10-06T05:24:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.120+0000 7fd49cdff700 1 -- 172.21.15.31:0/4119822610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49804c380 con 0x7fd498155430 2023-10-06T05:24:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.121+0000 7fd4867fc700 1 -- 172.21.15.31:0/4119822610 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fd488023070 con 0x7fd498155430 2023-10-06T05:24:27.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.122+0000 7fd4867fc700 1 --2- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd4780674d0 0x7fd478069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.122+0000 7fd4867fc700 1 -- 172.21.15.31:0/4119822610 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd4880863e0 con 0x7fd498155430 2023-10-06T05:24:27.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.123+0000 7fd495d9b700 1 --2- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd4780674d0 0x7fd478069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.123+0000 7fd495d9b700 1 --2- 172.21.15.31:0/4119822610 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd4780674d0 0x7fd478069980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd498143600 tx=0x7fd48c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.126+0000 7fd4867fc700 1 -- 172.21.15.31:0/4119822610 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd488052cd0 con 0x7fd498155430 2023-10-06T05:24:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.358+0000 7fd49cdff700 1 -- 172.21.15.31:0/4119822610 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) v1 -- 0x7fd498141a80 con 0x7fd4780674d0 2023-10-06T05:24:27.361 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:27.359+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:27.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.360+0000 7fd4867fc700 1 -- 172.21.15.31:0/4119822610 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fd48c001ce0 con 0x7fd4780674d0 2023-10-06T05:24:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:27.370 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:27.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2023-10-06T05:24:27.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.571+0000 7fb8e63ff700 1 Processor -- start 2023-10-06T05:24:27.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.571+0000 7fb8e63ff700 1 -- start start 2023-10-06T05:24:27.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.572+0000 7fb8e63ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8e015d340 0x7fb8e015d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.573+0000 7fb8e63ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e00cc610 con 0x7fb8e00ccaa0 2023-10-06T05:24:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.573+0000 7fb8e63ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e015ac50 con 0x7fb8e0159870 2023-10-06T05:24:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.573+0000 7fb8e63ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e015dd20 con 0x7fb8e015d340 2023-10-06T05:24:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.573+0000 7fb8df7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8e015d340 0x7fb8e015d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.573+0000 7fb8e499c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb8e00ccaa0 0x7fb8e00ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33720/0 (socket says 172.21.15.31:33720) 2023-10-06T05:24:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.573+0000 7fb8e499c700 1 -- 172.21.15.31:0/1971974194 learned_addr learned my addr 172.21.15.31:0/1971974194 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.573+0000 7fb8df7fe700 1 -- 172.21.15.31:0/1971974194 >> v1:172.21.15.62:6789/0 conn(0x7fb8e0159870 legacy=0x7fb8e015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.573+0000 7fb8df7fe700 1 -- 172.21.15.31:0/1971974194 >> v1:172.21.15.31:6789/0 conn(0x7fb8e00ccaa0 legacy=0x7fb8e00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.573+0000 7fb8df7fe700 1 -- 172.21.15.31:0/1971974194 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e00c9e40 con 0x7fb8e015d340 2023-10-06T05:24:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.573+0000 7fb8deffd700 1 -- 172.21.15.31:0/1971974194 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806248575 0 0) 0x7fb8e00cc610 con 0x7fb8e00ccaa0 2023-10-06T05:24:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.574+0000 7fb8df7fe700 1 --2- 172.21.15.31:0/1971974194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e015d340 0x7fb8e015d7e0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40098e0 tx=0x7fb8d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad308995e6724633 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.574+0000 7fb8deffd700 1 -- 172.21.15.31:0/1971974194 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4009e80 con 0x7fb8e015d340 2023-10-06T05:24:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.574+0000 7fb8deffd700 1 -- 172.21.15.31:0/1971974194 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d400b860 con 0x7fb8e015d340 2023-10-06T05:24:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.574+0000 7fb8deffd700 1 -- 172.21.15.31:0/1971974194 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400bbe0 con 0x7fb8e015d340 2023-10-06T05:24:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.574+0000 7fb8e63ff700 1 -- 172.21.15.31:0/1971974194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e015d340 msgr2=0x7fb8e015d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.574+0000 7fb8e63ff700 1 --2- 172.21.15.31:0/1971974194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e015d340 0x7fb8e015d7e0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40098e0 tx=0x7fb8d40091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.575+0000 7fb8e63ff700 1 -- 172.21.15.31:0/1971974194 shutdown_connections 2023-10-06T05:24:27.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.575+0000 7fb8e63ff700 1 --2- 172.21.15.31:0/1971974194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e015d340 0x7fb8e015d7e0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.575+0000 7fb8e63ff700 1 -- 172.21.15.31:0/1971974194 >> 172.21.15.31:0/1971974194 conn(0x7fb8e0087fc0 msgr2=0x7fb8e00639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:27.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.575+0000 7fb8e63ff700 1 -- 172.21.15.31:0/1971974194 shutdown_connections 2023-10-06T05:24:27.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.575+0000 7fb8e63ff700 1 -- 172.21.15.31:0/1971974194 wait complete. 2023-10-06T05:24:27.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.576+0000 7fb8e63ff700 1 Processor -- start 2023-10-06T05:24:27.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.576+0000 7fb8e63ff700 1 -- start start 2023-10-06T05:24:27.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.577+0000 7fb8e63ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e00ccaa0 0x7fb8e00c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.577+0000 7fb8e63ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e0159870 0x7fb8e01580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.577+0000 7fb8e63ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e015d340 0x7fb8e00abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.577+0000 7fb8e63ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e00b2d80 con 0x7fb8e0159870 2023-10-06T05:24:27.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.577+0000 7fb8e63ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e016d900 con 0x7fb8e00ccaa0 2023-10-06T05:24:27.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.577+0000 7fb8e63ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e016dbe0 con 0x7fb8e015d340 2023-10-06T05:24:27.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e0159870 0x7fb8e01580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e0159870 0x7fb8e01580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59374/0 (socket says 172.21.15.31:59374) 2023-10-06T05:24:27.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8df7fe700 1 -- 172.21.15.31:0/1517090508 learned_addr learned my addr 172.21.15.31:0/1517090508 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:27.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8e499c700 1 --2- 172.21.15.31:0/1517090508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e015d340 0x7fb8e00abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8dffff700 1 --2- 172.21.15.31:0/1517090508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e00ccaa0 0x7fb8e00c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8df7fe700 1 -- 172.21.15.31:0/1517090508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e015d340 msgr2=0x7fb8e00abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8df7fe700 1 --2- 172.21.15.31:0/1517090508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e015d340 0x7fb8e00abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8df7fe700 1 -- 172.21.15.31:0/1517090508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e00ccaa0 msgr2=0x7fb8e00c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:27.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8df7fe700 1 --2- 172.21.15.31:0/1517090508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e00ccaa0 0x7fb8e00c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:27.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8df7fe700 1 -- 172.21.15.31:0/1517090508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d4009420 con 0x7fb8e0159870 2023-10-06T05:24:27.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.578+0000 7fb8df7fe700 1 --2- 172.21.15.31:0/1517090508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8e0159870 0x7fb8e01580d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb8e006e810 tx=0x7fb8d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.579+0000 7fb8c7fff700 1 -- 172.21.15.31:0/1517090508 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400bbe0 con 0x7fb8e0159870 2023-10-06T05:24:27.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.579+0000 7fb8c7fff700 1 -- 172.21.15.31:0/1517090508 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d4011420 con 0x7fb8e0159870 2023-10-06T05:24:27.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.579+0000 7fb8c7fff700 1 -- 172.21.15.31:0/1517090508 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d40117c0 con 0x7fb8e0159870 2023-10-06T05:24:27.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.579+0000 7fb8e63ff700 1 -- 172.21.15.31:0/1517090508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e016dec0 con 0x7fb8e0159870 2023-10-06T05:24:27.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.579+0000 7fb8e63ff700 1 -- 172.21.15.31:0/1517090508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e0158610 con 0x7fb8e0159870 2023-10-06T05:24:27.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.580+0000 7fb8e63ff700 1 -- 172.21.15.31:0/1517090508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e004c380 con 0x7fb8e0159870 2023-10-06T05:24:27.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.581+0000 7fb8c7fff700 1 -- 172.21.15.31:0/1517090508 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb8d4011b30 con 0x7fb8e0159870 2023-10-06T05:24:27.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.582+0000 7fb8c7fff700 1 --2- 172.21.15.31:0/1517090508 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb8cc0674d0 0x7fb8cc069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:27.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.582+0000 7fb8c7fff700 1 -- 172.21.15.31:0/1517090508 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb8d4086770 con 0x7fb8e0159870 2023-10-06T05:24:27.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.582+0000 7fb8dffff700 1 --2- 172.21.15.31:0/1517090508 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb8cc0674d0 0x7fb8cc069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:27.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.583+0000 7fb8dffff700 1 --2- 172.21.15.31:0/1517090508 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb8cc0674d0 0x7fb8cc069980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00039e0 tx=0x7fb8d0007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.586+0000 7fb8c7fff700 1 -- 172.21.15.31:0/1517090508 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8d4053030 con 0x7fb8e0159870 2023-10-06T05:24:27.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.819+0000 7fb8e63ff700 1 -- 172.21.15.31:0/1517090508 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8e00b2ec0 con 0x7fb8cc0674d0 2023-10-06T05:24:27.822 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:27.821+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:27.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:27.821+0000 7fb8c7fff700 1 -- 172.21.15.31:0/1517090508 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fb8e00b2ec0 con 0x7fb8cc0674d0 2023-10-06T05:24:27.825 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:27.832 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:27.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2023-10-06T05:24:28.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.031+0000 7fc712dff700 1 Processor -- start 2023-10-06T05:24:28.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.031+0000 7fc712dff700 1 -- start start 2023-10-06T05:24:28.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.032+0000 7fc712dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc70c15d920 0x7fc70c1645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.032+0000 7fc712dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70c15cfc0 con 0x7fc70c153d20 2023-10-06T05:24:28.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.032+0000 7fc712dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70c15bb20 con 0x7fc70c153950 2023-10-06T05:24:28.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.032+0000 7fc712dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c154170 con 0x7fc70c15d920 2023-10-06T05:24:28.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.032+0000 7fc70bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc70c153d20 0x7fc70c15d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33732/0 (socket says 172.21.15.31:33732) 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.032+0000 7fc70bfff700 1 -- 172.21.15.31:0/1581929525 learned_addr learned my addr 172.21.15.31:0/1581929525 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.033+0000 7fc71139c700 1 --2- 172.21.15.31:0/1581929525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc70c15d920 0x7fc70c1645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.033+0000 7fc70b7fe700 1 -- 172.21.15.31:0/1581929525 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21203210 0 0) 0x7fc70c15cfc0 con 0x7fc70c153d20 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.033+0000 7fc70b7fe700 1 -- 172.21.15.31:0/1581929525 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6fc003570 con 0x7fc70c153d20 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.033+0000 7fc70b7fe700 1 -- 172.21.15.31:0/1581929525 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471635552 0 0) 0x7fc70c15bb20 con 0x7fc70c153950 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.033+0000 7fc70b7fe700 1 -- 172.21.15.31:0/1581929525 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc70c15cfc0 con 0x7fc70c153950 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.033+0000 7fc71139c700 1 -- 172.21.15.31:0/1581929525 >> v1:172.21.15.62:6789/0 conn(0x7fc70c153950 legacy=0x7fc70c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.033+0000 7fc71139c700 1 -- 172.21.15.31:0/1581929525 >> v1:172.21.15.31:6789/0 conn(0x7fc70c153d20 legacy=0x7fc70c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.033+0000 7fc71139c700 1 -- 172.21.15.31:0/1581929525 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c15af10 con 0x7fc70c15d920 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.034+0000 7fc71139c700 1 --2- 172.21.15.31:0/1581929525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc70c15d920 0x7fc70c1645b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc704009e90 tx=0x7fc704009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0fe271f577f8269 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.034+0000 7fc70b7fe700 1 -- 172.21.15.31:0/1581929525 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70400e070 con 0x7fc70c15d920 2023-10-06T05:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.034+0000 7fc70b7fe700 1 -- 172.21.15.31:0/1581929525 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70400bb80 con 0x7fc70c15d920 2023-10-06T05:24:28.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.034+0000 7fc70b7fe700 1 -- 172.21.15.31:0/1581929525 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704012430 con 0x7fc70c15d920 2023-10-06T05:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.034+0000 7fc712dff700 1 -- 172.21.15.31:0/1581929525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc70c15d920 msgr2=0x7fc70c1645b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.034+0000 7fc712dff700 1 --2- 172.21.15.31:0/1581929525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc70c15d920 0x7fc70c1645b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc704009e90 tx=0x7fc704009780 comp rx=0 tx=0).stop 2023-10-06T05:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.034+0000 7fc712dff700 1 -- 172.21.15.31:0/1581929525 shutdown_connections 2023-10-06T05:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.034+0000 7fc712dff700 1 --2- 172.21.15.31:0/1581929525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc70c15d920 0x7fc70c1645b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.034+0000 7fc712dff700 1 -- 172.21.15.31:0/1581929525 >> 172.21.15.31:0/1581929525 conn(0x7fc70c0681b0 msgr2=0x7fc70c0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.035+0000 7fc712dff700 1 -- 172.21.15.31:0/1581929525 shutdown_connections 2023-10-06T05:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.035+0000 7fc712dff700 1 -- 172.21.15.31:0/1581929525 wait complete. 2023-10-06T05:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.036+0000 7fc712dff700 1 Processor -- start 2023-10-06T05:24:28.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.036+0000 7fc712dff700 1 -- start start 2023-10-06T05:24:28.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.037+0000 7fc712dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc70c153950 0x7fc70c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.037+0000 7fc712dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c153d20 0x7fc70c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.037+0000 7fc712dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc70c15d920 0x7fc70c0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.037+0000 7fc712dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c0cf350 con 0x7fc70c153d20 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.037+0000 7fc712dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c159900 con 0x7fc70c153950 2023-10-06T05:24:28.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.037+0000 7fc712dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c158cf0 con 0x7fc70c15d920 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.037+0000 7fc70bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c153d20 0x7fc70c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.037+0000 7fc70bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c153d20 0x7fc70c0ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59380/0 (socket says 172.21.15.31:59380) 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.037+0000 7fc70bfff700 1 -- 172.21.15.31:0/3970667983 learned_addr learned my addr 172.21.15.31:0/3970667983 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc71139c700 1 --2- 172.21.15.31:0/3970667983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc70c15d920 0x7fc70c0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc710b9b700 1 --2- 172.21.15.31:0/3970667983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc70c153950 0x7fc70c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc70bfff700 1 -- 172.21.15.31:0/3970667983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc70c15d920 msgr2=0x7fc70c0cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc70bfff700 1 --2- 172.21.15.31:0/3970667983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc70c15d920 0x7fc70c0cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc70bfff700 1 -- 172.21.15.31:0/3970667983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc70c153950 msgr2=0x7fc70c159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc70bfff700 1 --2- 172.21.15.31:0/3970667983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc70c153950 0x7fc70c159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc70bfff700 1 -- 172.21.15.31:0/3970667983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc704009a20 con 0x7fc70c153d20 2023-10-06T05:24:28.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc710b9b700 1 --2- 172.21.15.31:0/3970667983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc70c153950 0x7fc70c159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:28.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc70bfff700 1 --2- 172.21.15.31:0/3970667983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c153d20 0x7fc70c0ce8d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800c790 tx=0x7fc6f8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc708ff9700 1 -- 172.21.15.31:0/3970667983 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8005b30 con 0x7fc70c153d20 2023-10-06T05:24:28.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc708ff9700 1 -- 172.21.15.31:0/3970667983 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f8005c90 con 0x7fc70c153d20 2023-10-06T05:24:28.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc708ff9700 1 -- 172.21.15.31:0/3970667983 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8007300 con 0x7fc70c153d20 2023-10-06T05:24:28.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.038+0000 7fc712dff700 1 -- 172.21.15.31:0/3970667983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c158fd0 con 0x7fc70c153d20 2023-10-06T05:24:28.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.039+0000 7fc712dff700 1 -- 172.21.15.31:0/3970667983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c159590 con 0x7fc70c153d20 2023-10-06T05:24:28.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.039+0000 7fc712dff700 1 -- 172.21.15.31:0/3970667983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70c159a90 con 0x7fc70c153d20 2023-10-06T05:24:28.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.040+0000 7fc708ff9700 1 -- 172.21.15.31:0/3970667983 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fc6f801b400 con 0x7fc70c153d20 2023-10-06T05:24:28.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.041+0000 7fc708ff9700 1 --2- 172.21.15.31:0/3970667983 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6f00674d0 0x7fc6f0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.041+0000 7fc708ff9700 1 -- 172.21.15.31:0/3970667983 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc6f8084d10 con 0x7fc70c153d20 2023-10-06T05:24:28.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.042+0000 7fc710b9b700 1 --2- 172.21.15.31:0/3970667983 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6f00674d0 0x7fc6f0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.043+0000 7fc710b9b700 1 --2- 172.21.15.31:0/3970667983 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6f00674d0 0x7fc6f0069980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc700002870 tx=0x7fc700009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.046+0000 7fc708ff9700 1 -- 172.21.15.31:0/3970667983 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6f8051680 con 0x7fc70c153d20 2023-10-06T05:24:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.280+0000 7fc712dff700 1 -- 172.21.15.31:0/3970667983 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc70c04c380 con 0x7fc6f00674d0 2023-10-06T05:24:28.367 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:28.282+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:28.368 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:28.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.282+0000 7fc708ff9700 1 -- 172.21.15.31:0/3970667983 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fc70c04c380 con 0x7fc6f00674d0 2023-10-06T05:24:28.369 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:28.369 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2023-10-06T05:24:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.566+0000 7fb4941ff700 1 Processor -- start 2023-10-06T05:24:28.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.567+0000 7fb4941ff700 1 -- start start 2023-10-06T05:24:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.568+0000 7fb4941ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb48c153d20 0x7fb48c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.568+0000 7fb4941ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c15bb20 con 0x7fb48c15d750 2023-10-06T05:24:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.568+0000 7fb4941ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c15af10 con 0x7fb48c153950 2023-10-06T05:24:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.568+0000 7fb4941ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c154170 con 0x7fb48c153d20 2023-10-06T05:24:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.568+0000 7fb49279c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb48c15d750 0x7fb48c15bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33744/0 (socket says 172.21.15.31:33744) 2023-10-06T05:24:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.568+0000 7fb49279c700 1 -- 172.21.15.31:0/341810902 learned_addr learned my addr 172.21.15.31:0/341810902 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:28.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.568+0000 7fb49179a700 1 --2- 172.21.15.31:0/341810902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb48c153d20 0x7fb48c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.569+0000 7fb490f99700 1 -- 172.21.15.31:0/341810902 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703222600 0 0) 0x7fb48c15bb20 con 0x7fb48c15d750 2023-10-06T05:24:28.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.569+0000 7fb490f99700 1 -- 172.21.15.31:0/341810902 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb478003570 con 0x7fb48c15d750 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.569+0000 7fb490f99700 1 -- 172.21.15.31:0/341810902 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630604855 0 0) 0x7fb48c15af10 con 0x7fb48c153950 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.569+0000 7fb490f99700 1 -- 172.21.15.31:0/341810902 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb48c15bb20 con 0x7fb48c153950 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.569+0000 7fb49179a700 1 -- 172.21.15.31:0/341810902 >> v1:172.21.15.62:6789/0 conn(0x7fb48c153950 legacy=0x7fb48c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.569+0000 7fb49179a700 1 -- 172.21.15.31:0/341810902 >> v1:172.21.15.31:6789/0 conn(0x7fb48c15d750 legacy=0x7fb48c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.569+0000 7fb49179a700 1 -- 172.21.15.31:0/341810902 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c15b1f0 con 0x7fb48c153d20 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.569+0000 7fb49179a700 1 --2- 172.21.15.31:0/341810902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb48c153d20 0x7fb48c15d210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009910 tx=0x7fb47c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7420ea6a69a2862 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.569+0000 7fb490f99700 1 -- 172.21.15.31:0/341810902 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c011630 con 0x7fb48c153d20 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.570+0000 7fb490f99700 1 -- 172.21.15.31:0/341810902 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c011790 con 0x7fb48c153d20 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.570+0000 7fb490f99700 1 -- 172.21.15.31:0/341810902 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c011b30 con 0x7fb48c153d20 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.570+0000 7fb4941ff700 1 -- 172.21.15.31:0/341810902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb48c153d20 msgr2=0x7fb48c15d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.570+0000 7fb4941ff700 1 --2- 172.21.15.31:0/341810902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb48c153d20 0x7fb48c15d210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009910 tx=0x7fb47c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.570+0000 7fb4941ff700 1 -- 172.21.15.31:0/341810902 shutdown_connections 2023-10-06T05:24:28.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.570+0000 7fb4941ff700 1 --2- 172.21.15.31:0/341810902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb48c153d20 0x7fb48c15d210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.570+0000 7fb4941ff700 1 -- 172.21.15.31:0/341810902 >> 172.21.15.31:0/341810902 conn(0x7fb48c0681b0 msgr2=0x7fb48c0685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:28.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.570+0000 7fb4941ff700 1 -- 172.21.15.31:0/341810902 shutdown_connections 2023-10-06T05:24:28.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.570+0000 7fb4941ff700 1 -- 172.21.15.31:0/341810902 wait complete. 2023-10-06T05:24:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.572+0000 7fb4941ff700 1 Processor -- start 2023-10-06T05:24:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.572+0000 7fb4941ff700 1 -- start start 2023-10-06T05:24:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb4941ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb48c153950 0x7fb48c16dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb4941ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48c153d20 0x7fb48c16a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb4941ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb48c15d750 0x7fb48c169340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb4941ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c156450 con 0x7fb48c153d20 2023-10-06T05:24:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb4941ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c16aac0 con 0x7fb48c15d750 2023-10-06T05:24:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb4941ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c157250 con 0x7fb48c153950 2023-10-06T05:24:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb49179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48c153d20 0x7fb48c16a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb49179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48c153d20 0x7fb48c16a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59382/0 (socket says 172.21.15.31:59382) 2023-10-06T05:24:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb49179a700 1 -- 172.21.15.31:0/2254702406 learned_addr learned my addr 172.21.15.31:0/2254702406 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb49279c700 1 --2- 172.21.15.31:0/2254702406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb48c15d750 0x7fb48c169340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb491f9b700 1 --2- 172.21.15.31:0/2254702406 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb48c153950 0x7fb48c16dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb49179a700 1 -- 172.21.15.31:0/2254702406 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb48c153950 msgr2=0x7fb48c16dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb49179a700 1 --2- 172.21.15.31:0/2254702406 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb48c153950 0x7fb48c16dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb49179a700 1 -- 172.21.15.31:0/2254702406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb48c15d750 msgr2=0x7fb48c169340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb49179a700 1 --2- 172.21.15.31:0/2254702406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb48c15d750 0x7fb48c169340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.573+0000 7fb49179a700 1 -- 172.21.15.31:0/2254702406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c009420 con 0x7fb48c153d20 2023-10-06T05:24:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.574+0000 7fb49179a700 1 --2- 172.21.15.31:0/2254702406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48c153d20 0x7fb48c16a580 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009f00 tx=0x7fb47c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.574+0000 7fb481ffb700 1 -- 172.21.15.31:0/2254702406 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c01f070 con 0x7fb48c153d20 2023-10-06T05:24:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.574+0000 7fb481ffb700 1 -- 172.21.15.31:0/2254702406 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c011900 con 0x7fb48c153d20 2023-10-06T05:24:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.574+0000 7fb4941ff700 1 -- 172.21.15.31:0/2254702406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c156f70 con 0x7fb48c153d20 2023-10-06T05:24:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.574+0000 7fb481ffb700 1 -- 172.21.15.31:0/2254702406 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c023410 con 0x7fb48c153d20 2023-10-06T05:24:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.574+0000 7fb4941ff700 1 -- 172.21.15.31:0/2254702406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c16d4b0 con 0x7fb48c153d20 2023-10-06T05:24:28.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.575+0000 7fb4941ff700 1 -- 172.21.15.31:0/2254702406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c04c380 con 0x7fb48c153d20 2023-10-06T05:24:28.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.576+0000 7fb481ffb700 1 -- 172.21.15.31:0/2254702406 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb47c023800 con 0x7fb48c153d20 2023-10-06T05:24:28.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.577+0000 7fb481ffb700 1 --2- 172.21.15.31:0/2254702406 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4700674d0 0x7fb470069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:28.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.577+0000 7fb481ffb700 1 -- 172.21.15.31:0/2254702406 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb47c0861e0 con 0x7fb48c153d20 2023-10-06T05:24:28.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.578+0000 7fb491f9b700 1 --2- 172.21.15.31:0/2254702406 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4700674d0 0x7fb470069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:28.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.578+0000 7fb491f9b700 1 --2- 172.21.15.31:0/2254702406 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4700674d0 0x7fb470069980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb488003820 tx=0x7fb488009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.582+0000 7fb481ffb700 1 -- 172.21.15.31:0/2254702406 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb47c052a20 con 0x7fb48c153d20 2023-10-06T05:24:28.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.813+0000 7fb4941ff700 1 -- 172.21.15.31:0/2254702406 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb48c0ce480 con 0x7fb4700674d0 2023-10-06T05:24:28.816 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:28.815+0000 7f2e4417d700 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2023-10-06T05:24:28.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:28.816+0000 7fb481ffb700 1 -- 172.21.15.31:0/2254702406 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) v1 ==== 45+0+0 (secure 0 0 0) 0x7fb48c0ce480 con 0x7fb4700674d0 2023-10-06T05:24:28.818 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2023-10-06T05:24:28.825 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T05:24:28.826 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:24:28.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:28.924 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:29.033 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:24:29.033 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:24:29.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:29.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:29.119 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:24:29.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:29.120 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:24:29.167 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:24:29.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:29.167 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:29.260 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:24:29.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:29.260 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:24:29.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:29.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.501+0000 7f952e5ff700 1 Processor -- start 2023-10-06T05:24:29.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.502+0000 7f952e5ff700 1 -- start start 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.503+0000 7f952e5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95281595a0 0x7f9528159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.503+0000 7f952e5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95280c95b0 con 0x7f95280cc0d0 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.503+0000 7f952e5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95280c9890 con 0x7f9528068d80 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.503+0000 7f952e5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95280c9b70 con 0x7f95281595a0 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.503+0000 7f95277fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f95280cc0d0 0x7f9528068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33756/0 (socket says 172.21.15.31:33756) 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.503+0000 7f95277fe700 1 -- 172.21.15.31:0/996827353 learned_addr learned my addr 172.21.15.31:0/996827353 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.503+0000 7f9527fff700 1 --2- 172.21.15.31:0/996827353 >> v2:172.21.15.62:3300/0 conn(0x7f95281595a0 0x7f9528159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.504+0000 7f9526ffd700 1 -- 172.21.15.31:0/996827353 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462253244 0 0) 0x7f95280c95b0 con 0x7f95280cc0d0 2023-10-06T05:24:29.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.504+0000 7f9526ffd700 1 -- 172.21.15.31:0/996827353 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9510003570 con 0x7f95280cc0d0 2023-10-06T05:24:29.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.504+0000 7f9527fff700 1 -- 172.21.15.31:0/996827353 >> v1:172.21.15.62:6789/0 conn(0x7f9528068d80 legacy=0x7f95280c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.504+0000 7f9527fff700 1 -- 172.21.15.31:0/996827353 >> v1:172.21.15.31:6789/0 conn(0x7f95280cc0d0 legacy=0x7f9528068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.504+0000 7f9526ffd700 1 -- 172.21.15.31:0/996827353 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2995892886 0 0) 0x7f9510003570 con 0x7f95280cc0d0 2023-10-06T05:24:29.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.504+0000 7f9527fff700 1 -- 172.21.15.31:0/996827353 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95280ca130 con 0x7f95281595a0 2023-10-06T05:24:29.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.504+0000 7f9527fff700 1 --2- 172.21.15.31:0/996827353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95281595a0 0x7f9528159970 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9518009910 tx=0x7f95180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a0d692da4c49d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.505+0000 7f9526ffd700 1 -- 172.21.15.31:0/996827353 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c001470 con 0x7f95281595a0 2023-10-06T05:24:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.505+0000 7f9526ffd700 1 -- 172.21.15.31:0/996827353 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9518011420 con 0x7f95281595a0 2023-10-06T05:24:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.505+0000 7f9526ffd700 1 -- 172.21.15.31:0/996827353 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95180117c0 con 0x7f95281595a0 2023-10-06T05:24:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.505+0000 7f952e5ff700 1 -- 172.21.15.31:0/996827353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95281595a0 msgr2=0x7f9528159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.505+0000 7f952e5ff700 1 --2- 172.21.15.31:0/996827353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95281595a0 0x7f9528159970 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9518009910 tx=0x7f95180091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.505+0000 7f952e5ff700 1 -- 172.21.15.31:0/996827353 shutdown_connections 2023-10-06T05:24:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.505+0000 7f952e5ff700 1 --2- 172.21.15.31:0/996827353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95281595a0 0x7f9528159970 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.505+0000 7f952e5ff700 1 -- 172.21.15.31:0/996827353 >> 172.21.15.31:0/996827353 conn(0x7f952806b620 msgr2=0x7f952806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.505+0000 7f952e5ff700 1 -- 172.21.15.31:0/996827353 shutdown_connections 2023-10-06T05:24:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.505+0000 7f952e5ff700 1 -- 172.21.15.31:0/996827353 wait complete. 2023-10-06T05:24:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.507+0000 7f952e5ff700 1 Processor -- start 2023-10-06T05:24:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.507+0000 7f952e5ff700 1 -- start start 2023-10-06T05:24:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f952e5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9528068d80 0x7f9528147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f952e5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95280cc0d0 0x7f952815bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f952e5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95281595a0 0x7f9528153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f952e5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95280c73c0 con 0x7f95280cc0d0 2023-10-06T05:24:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f952e5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95280c6820 con 0x7f9528068d80 2023-10-06T05:24:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f952e5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95281482f0 con 0x7f95281595a0 2023-10-06T05:24:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f95277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95280cc0d0 0x7f952815bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f95277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95280cc0d0 0x7f952815bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59396/0 (socket says 172.21.15.31:59396) 2023-10-06T05:24:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f95277fe700 1 -- 172.21.15.31:0/1509155282 learned_addr learned my addr 172.21.15.31:0/1509155282 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f9527fff700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9528068d80 0x7f9528147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.508+0000 7f952cb9c700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95281595a0 0x7f9528153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f95277fe700 1 -- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95281595a0 msgr2=0x7f9528153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f95277fe700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95281595a0 0x7f9528153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f95277fe700 1 -- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9528068d80 msgr2=0x7f9528147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f95277fe700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9528068d80 0x7f9528147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f95277fe700 1 -- 172.21.15.31:0/1509155282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9518009420 con 0x7f95280cc0d0 2023-10-06T05:24:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f95277fe700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95280cc0d0 0x7f952815bd90 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f951c00cb90 tx=0x7f951c001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f950ffff700 1 -- 172.21.15.31:0/1509155282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c007e90 con 0x7f95280cc0d0 2023-10-06T05:24:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f950ffff700 1 -- 172.21.15.31:0/1509155282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c007360 con 0x7f95280cc0d0 2023-10-06T05:24:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f950ffff700 1 -- 172.21.15.31:0/1509155282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c004830 con 0x7f95280cc0d0 2023-10-06T05:24:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f952e5ff700 1 -- 172.21.15.31:0/1509155282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95280c6a90 con 0x7f95280cc0d0 2023-10-06T05:24:29.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.509+0000 7f952e5ff700 1 -- 172.21.15.31:0/1509155282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95280c7050 con 0x7f95280cc0d0 2023-10-06T05:24:29.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.510+0000 7f952e5ff700 1 -- 172.21.15.31:0/1509155282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952804c380 con 0x7f95280cc0d0 2023-10-06T05:24:29.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.512+0000 7f950ffff700 1 -- 172.21.15.31:0/1509155282 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f951c004be0 con 0x7f95280cc0d0 2023-10-06T05:24:29.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.512+0000 7f950ffff700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f95140674d0 0x7f9514069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.513+0000 7f950ffff700 1 -- 172.21.15.31:0/1509155282 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f951c084600 con 0x7f95280cc0d0 2023-10-06T05:24:29.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.513+0000 7f9527fff700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f95140674d0 0x7f9514069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.513+0000 7f9527fff700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f95140674d0 0x7f9514069980 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f95180093f0 tx=0x7f9518016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.517+0000 7f950ffff700 1 -- 172.21.15.31:0/1509155282 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f951c050ec0 con 0x7f95280cc0d0 2023-10-06T05:24:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.762+0000 7f952e5ff700 1 -- 172.21.15.31:0/1509155282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95280c7500 con 0x7f95280cc0d0 2023-10-06T05:24:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.762+0000 7f950ffff700 1 -- 172.21.15.31:0/1509155282 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f951c0544e0 con 0x7f95280cc0d0 2023-10-06T05:24:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:29.764 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3816040413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3816040413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3816040413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1295480750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1295480750},{"type":"v1","addr":"172.21.15.31:6834","nonce":1295480750}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2807869230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2807869230},{"type":"v1","addr":"172.21.15.62:6834","nonce":2807869230}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:06.204405+0000","modified":"2023-10-06T05:24:10.122253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7995":{"gid":7995,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1928465499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1928465499},{"type":"v1","addr":"172.21.15.62:6835","nonce":1928465499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 -- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f95140674d0 msgr2=0x7f9514069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f95140674d0 0x7f9514069980 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f95180093f0 tx=0x7f9518016000 comp rx=0 tx=0).stop 2023-10-06T05:24:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 -- 172.21.15.31:0/1509155282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95280cc0d0 msgr2=0x7f952815bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95280cc0d0 0x7f952815bd90 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f951c00cb90 tx=0x7f951c001b50 comp rx=0 tx=0).stop 2023-10-06T05:24:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 -- 172.21.15.31:0/1509155282 shutdown_connections 2023-10-06T05:24:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f95140674d0 0x7f9514069980 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f95281595a0 0x7f9528153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9528068d80 0x7f9528147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 --2- 172.21.15.31:0/1509155282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95280cc0d0 0x7f952815bd90 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 -- 172.21.15.31:0/1509155282 >> 172.21.15.31:0/1509155282 conn(0x7f952806b620 msgr2=0x7f9528068670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 -- 172.21.15.31:0/1509155282 shutdown_connections 2023-10-06T05:24:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.765+0000 7f952e5ff700 1 -- 172.21.15.31:0/1509155282 wait complete. 2023-10-06T05:24:29.768 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 85 2023-10-06T05:24:29.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:29.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.978+0000 7ff606dff700 1 Processor -- start 2023-10-06T05:24:29.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.978+0000 7ff606dff700 1 -- start start 2023-10-06T05:24:29.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.979+0000 7ff606dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff600068d80 0x7ff600159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.979+0000 7ff606dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6000c95b0 con 0x7ff6000cc0d0 2023-10-06T05:24:29.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.979+0000 7ff606dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6000c9890 con 0x7ff6001595a0 2023-10-06T05:24:29.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.979+0000 7ff606dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000c9b70 con 0x7ff600068d80 2023-10-06T05:24:29.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.980+0000 7ff60539c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff6000cc0d0 0x7ff6000c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33760/0 (socket says 172.21.15.31:33760) 2023-10-06T05:24:29.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.980+0000 7ff60539c700 1 -- 172.21.15.31:0/2030014116 learned_addr learned my addr 172.21.15.31:0/2030014116 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:29.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.980+0000 7ff5fffff700 1 --2- 172.21.15.31:0/2030014116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff600068d80 0x7ff600159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.980+0000 7ff5ff7fe700 1 -- 172.21.15.31:0/2030014116 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231516278 0 0) 0x7ff6000c95b0 con 0x7ff6000cc0d0 2023-10-06T05:24:29.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.980+0000 7ff5ff7fe700 1 -- 172.21.15.31:0/2030014116 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f0003570 con 0x7ff6000cc0d0 2023-10-06T05:24:29.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.980+0000 7ff5ff7fe700 1 -- 172.21.15.31:0/2030014116 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055098947 0 0) 0x7ff6000c9890 con 0x7ff6001595a0 2023-10-06T05:24:29.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.980+0000 7ff5ff7fe700 1 -- 172.21.15.31:0/2030014116 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6000c95b0 con 0x7ff6001595a0 2023-10-06T05:24:29.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.980+0000 7ff5fffff700 1 -- 172.21.15.31:0/2030014116 >> v1:172.21.15.62:6789/0 conn(0x7ff6001595a0 legacy=0x7ff600068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.980+0000 7ff5fffff700 1 -- 172.21.15.31:0/2030014116 >> v1:172.21.15.31:6789/0 conn(0x7ff6000cc0d0 legacy=0x7ff6000c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.980+0000 7ff5fffff700 1 -- 172.21.15.31:0/2030014116 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6000ca130 con 0x7ff600068d80 2023-10-06T05:24:29.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.981+0000 7ff5fffff700 1 --2- 172.21.15.31:0/2030014116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff600068d80 0x7ff600159970 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009910 tx=0x7ff5ec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41035e7904528343 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.981+0000 7ff5ff7fe700 1 -- 172.21.15.31:0/2030014116 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec011630 con 0x7ff600068d80 2023-10-06T05:24:29.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.981+0000 7ff5ff7fe700 1 -- 172.21.15.31:0/2030014116 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec011790 con 0x7ff600068d80 2023-10-06T05:24:29.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.981+0000 7ff5ff7fe700 1 -- 172.21.15.31:0/2030014116 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec011b30 con 0x7ff600068d80 2023-10-06T05:24:29.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.981+0000 7ff606dff700 1 -- 172.21.15.31:0/2030014116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff600068d80 msgr2=0x7ff600159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.981+0000 7ff606dff700 1 --2- 172.21.15.31:0/2030014116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff600068d80 0x7ff600159970 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009910 tx=0x7ff5ec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.982+0000 7ff606dff700 1 -- 172.21.15.31:0/2030014116 shutdown_connections 2023-10-06T05:24:29.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.982+0000 7ff606dff700 1 --2- 172.21.15.31:0/2030014116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff600068d80 0x7ff600159970 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.982+0000 7ff606dff700 1 -- 172.21.15.31:0/2030014116 >> 172.21.15.31:0/2030014116 conn(0x7ff60006b620 msgr2=0x7ff60006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:29.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.982+0000 7ff606dff700 1 -- 172.21.15.31:0/2030014116 shutdown_connections 2023-10-06T05:24:29.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.982+0000 7ff606dff700 1 -- 172.21.15.31:0/2030014116 wait complete. 2023-10-06T05:24:29.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.983+0000 7ff606dff700 1 Processor -- start 2023-10-06T05:24:29.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.983+0000 7ff606dff700 1 -- start start 2023-10-06T05:24:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.984+0000 7ff606dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff600068d80 0x7ff600155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.984+0000 7ff606dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000cc0d0 0x7ff6000c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.984+0000 7ff606dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6001595a0 0x7ff6000c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.984+0000 7ff606dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000c7b30 con 0x7ff6000cc0d0 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.984+0000 7ff606dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000c6cb0 con 0x7ff600068d80 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.984+0000 7ff606dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff600142900 con 0x7ff6001595a0 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000cc0d0 0x7ff6000c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000cc0d0 0x7ff6000c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59412/0 (socket says 172.21.15.31:59412) 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff5fffff700 1 -- 172.21.15.31:0/4013619056 learned_addr learned my addr 172.21.15.31:0/4013619056 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff60539c700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6001595a0 0x7ff6000c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff604b9b700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff600068d80 0x7ff600155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff5fffff700 1 -- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6001595a0 msgr2=0x7ff6000c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff5fffff700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6001595a0 0x7ff6000c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff5fffff700 1 -- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff600068d80 msgr2=0x7ff600155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff5fffff700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff600068d80 0x7ff600155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff5fffff700 1 -- 172.21.15.31:0/4013619056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec009420 con 0x7ff6000cc0d0 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff604b9b700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff600068d80 0x7ff600155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff60539c700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6001595a0 0x7ff6000c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff5fffff700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000cc0d0 0x7ff6000c6230 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009fd0 tx=0x7ff5ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.985+0000 7ff5fcff9700 1 -- 172.21.15.31:0/4013619056 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec01f070 con 0x7ff6000cc0d0 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.986+0000 7ff5fcff9700 1 -- 172.21.15.31:0/4013619056 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec000bc0 con 0x7ff6000cc0d0 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.986+0000 7ff5fcff9700 1 -- 172.21.15.31:0/4013619056 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec025420 con 0x7ff6000cc0d0 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.986+0000 7ff606dff700 1 -- 172.21.15.31:0/4013619056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6000c6f20 con 0x7ff6000cc0d0 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.986+0000 7ff606dff700 1 -- 172.21.15.31:0/4013619056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6000c74e0 con 0x7ff6000cc0d0 2023-10-06T05:24:29.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.987+0000 7ff606dff700 1 -- 172.21.15.31:0/4013619056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60004c380 con 0x7ff6000cc0d0 2023-10-06T05:24:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.988+0000 7ff5fcff9700 1 -- 172.21.15.31:0/4013619056 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ff5ec023070 con 0x7ff6000cc0d0 2023-10-06T05:24:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.988+0000 7ff5fcff9700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5e40674d0 0x7ff5e4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:29.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.988+0000 7ff5fcff9700 1 -- 172.21.15.31:0/4013619056 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff5ec0864d0 con 0x7ff6000cc0d0 2023-10-06T05:24:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.989+0000 7ff604b9b700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5e40674d0 0x7ff5e4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.989+0000 7ff604b9b700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5e40674d0 0x7ff5e4069980 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4007fd0 tx=0x7ff5f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:29.993+0000 7ff5fcff9700 1 -- 172.21.15.31:0/4013619056 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5ec052d10 con 0x7ff6000cc0d0 2023-10-06T05:24:30.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.194+0000 7ff606dff700 1 -- 172.21.15.31:0/4013619056 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6000c7c70 con 0x7ff6000cc0d0 2023-10-06T05:24:30.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.194+0000 7ff5fcff9700 1 -- 172.21.15.31:0/4013619056 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16972 (secure 0 0 0) 0x7ff5ec056330 con 0x7ff6000cc0d0 2023-10-06T05:24:30.197 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:30.197 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":127,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:07.702885+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:03.921897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:05.190243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:30.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 -- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5e40674d0 msgr2=0x7ff5e4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5e40674d0 0x7ff5e4069980 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4007fd0 tx=0x7ff5f4009040 comp rx=0 tx=0).stop 2023-10-06T05:24:30.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 -- 172.21.15.31:0/4013619056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000cc0d0 msgr2=0x7ff6000c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000cc0d0 0x7ff6000c6230 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009fd0 tx=0x7ff5ec017000 comp rx=0 tx=0).stop 2023-10-06T05:24:30.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 -- 172.21.15.31:0/4013619056 shutdown_connections 2023-10-06T05:24:30.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5e40674d0 0x7ff5e4069980 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6001595a0 0x7ff6000c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff600068d80 0x7ff600155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 --2- 172.21.15.31:0/4013619056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000cc0d0 0x7ff6000c6230 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 -- 172.21.15.31:0/4013619056 >> 172.21.15.31:0/4013619056 conn(0x7ff60006b620 msgr2=0x7ff6000ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 -- 172.21.15.31:0/4013619056 shutdown_connections 2023-10-06T05:24:30.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.197+0000 7ff606dff700 1 -- 172.21.15.31:0/4013619056 wait complete. 2023-10-06T05:24:30.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:24:03.921897+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:24:05.190243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:24:30.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:24:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.414+0000 7f9a12dff700 1 Processor -- start 2023-10-06T05:24:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.414+0000 7f9a12dff700 1 -- start start 2023-10-06T05:24:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a12dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c15ca40 0x7f9a0c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a12dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a0c0c8f40 0x7f9a0c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a12dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c068da0 con 0x7f9a0c06a300 2023-10-06T05:24:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a12dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cabc0 con 0x7f9a0c0c8f40 2023-10-06T05:24:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a12dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c068bb0 con 0x7f9a0c15ca40 2023-10-06T05:24:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a1139c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a0c0c8f40 0x7f9a0c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a1139c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a0c0c8f40 0x7f9a0c0adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59418/0 (socket says 172.21.15.31:59418) 2023-10-06T05:24:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a1139c700 1 -- 172.21.15.31:0/278862177 learned_addr learned my addr 172.21.15.31:0/278862177 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a10b9b700 1 --2- 172.21.15.31:0/278862177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c15ca40 0x7f9a0c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a1139c700 1 -- 172.21.15.31:0/278862177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c15ca40 msgr2=0x7f9a0c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a1139c700 1 --2- 172.21.15.31:0/278862177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c15ca40 0x7f9a0c068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a1139c700 1 -- 172.21.15.31:0/278862177 >> v1:172.21.15.62:6789/0 conn(0x7f9a0c06a300 legacy=0x7f9a0c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.416+0000 7f9a1139c700 1 -- 172.21.15.31:0/278862177 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c069080 con 0x7f9a0c0c8f40 2023-10-06T05:24:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.417+0000 7f9a1139c700 1 --2- 172.21.15.31:0/278862177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c0c8f40 0x7f9a0c0adde0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009f20 tx=0x7f9a04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edcaf41f97b290a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.417+0000 7f9a0b7fe700 1 -- 172.21.15.31:0/278862177 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400e070 con 0x7f9a0c0c8f40 2023-10-06T05:24:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.417+0000 7f9a0b7fe700 1 -- 172.21.15.31:0/278862177 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0400b9d0 con 0x7f9a0c0c8f40 2023-10-06T05:24:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.417+0000 7f9a0b7fe700 1 -- 172.21.15.31:0/278862177 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040092e0 con 0x7f9a0c0c8f40 2023-10-06T05:24:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.417+0000 7f9a12dff700 1 -- 172.21.15.31:0/278862177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c0c8f40 msgr2=0x7f9a0c0adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.417+0000 7f9a12dff700 1 --2- 172.21.15.31:0/278862177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c0c8f40 0x7f9a0c0adde0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009f20 tx=0x7f9a04009840 comp rx=0 tx=0).stop 2023-10-06T05:24:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.418+0000 7f9a12dff700 1 -- 172.21.15.31:0/278862177 shutdown_connections 2023-10-06T05:24:30.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.418+0000 7f9a12dff700 1 --2- 172.21.15.31:0/278862177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c0c8f40 0x7f9a0c0adde0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.418+0000 7f9a12dff700 1 --2- 172.21.15.31:0/278862177 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c15ca40 0x7f9a0c068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.418+0000 7f9a12dff700 1 -- 172.21.15.31:0/278862177 >> 172.21.15.31:0/278862177 conn(0x7f9a0c0ce290 msgr2=0x7f9a0c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.418+0000 7f9a12dff700 1 -- 172.21.15.31:0/278862177 shutdown_connections 2023-10-06T05:24:30.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.418+0000 7f9a12dff700 1 -- 172.21.15.31:0/278862177 wait complete. 2023-10-06T05:24:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.419+0000 7f9a12dff700 1 Processor -- start 2023-10-06T05:24:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.419+0000 7f9a12dff700 1 -- start start 2023-10-06T05:24:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.420+0000 7f9a12dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a0c06a300 0x7f9a0c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.420+0000 7f9a12dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0c8f40 0x7f9a0c14b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.420+0000 7f9a12dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c15ca40 0x7f9a0c159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.420+0000 7f9a12dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0b1c50 con 0x7f9a0c15ca40 2023-10-06T05:24:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.420+0000 7f9a12dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0b1040 con 0x7f9a0c06a300 2023-10-06T05:24:30.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.420+0000 7f9a12dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0b1320 con 0x7f9a0c0c8f40 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.420+0000 7f9a1139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c15ca40 0x7f9a0c159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.420+0000 7f9a1139c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c15ca40 0x7f9a0c159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59426/0 (socket says 172.21.15.31:59426) 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.420+0000 7f9a1139c700 1 -- 172.21.15.31:0/3187069197 learned_addr learned my addr 172.21.15.31:0/3187069197 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.420+0000 7f9a10b9b700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a0c06a300 0x7f9a0c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a0bfff700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0c8f40 0x7f9a0c14b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a1139c700 1 -- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0c8f40 msgr2=0x7f9a0c14b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a1139c700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0c8f40 0x7f9a0c14b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a1139c700 1 -- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a0c06a300 msgr2=0x7f9a0c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a1139c700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a0c06a300 0x7f9a0c15e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a1139c700 1 -- 172.21.15.31:0/3187069197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04009ae0 con 0x7f9a0c15ca40 2023-10-06T05:24:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a0bfff700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0c8f40 0x7f9a0c14b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a1139c700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c15ca40 0x7f9a0c159740 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c06e040 tx=0x7f9a04018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a08ff9700 1 -- 172.21.15.31:0/3187069197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04008b80 con 0x7f9a0c15ca40 2023-10-06T05:24:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a08ff9700 1 -- 172.21.15.31:0/3187069197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04008ce0 con 0x7f9a0c15ca40 2023-10-06T05:24:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a08ff9700 1 -- 172.21.15.31:0/3187069197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040266b0 con 0x7f9a0c15ca40 2023-10-06T05:24:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.421+0000 7f9a12dff700 1 -- 172.21.15.31:0/3187069197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c0b1600 con 0x7f9a0c15ca40 2023-10-06T05:24:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.422+0000 7f9a12dff700 1 -- 172.21.15.31:0/3187069197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c15ea40 con 0x7f9a0c15ca40 2023-10-06T05:24:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.423+0000 7f9a12dff700 1 -- 172.21.15.31:0/3187069197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c0b1e00 con 0x7f9a0c15ca40 2023-10-06T05:24:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.423+0000 7f9a08ff9700 1 -- 172.21.15.31:0/3187069197 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9a04003040 con 0x7f9a0c15ca40 2023-10-06T05:24:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.424+0000 7f9a08ff9700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f00674d0 0x7f99f0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.424+0000 7f9a08ff9700 1 -- 172.21.15.31:0/3187069197 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9a04086b80 con 0x7f9a0c15ca40 2023-10-06T05:24:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.425+0000 7f9a10b9b700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f00674d0 0x7f99f0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.426+0000 7f9a10b9b700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f00674d0 0x7f99f0069980 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009db0 tx=0x7f9a000091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.429+0000 7f9a08ff9700 1 -- 172.21.15.31:0/3187069197 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a0c0b1e00 con 0x7f9a0c15ca40 2023-10-06T05:24:30.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.632+0000 7f9a12dff700 1 -- 172.21.15.31:0/3187069197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9a0c04c380 con 0x7f9a0c15ca40 2023-10-06T05:24:30.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.633+0000 7f9a08ff9700 1 -- 172.21.15.31:0/3187069197 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v127) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9a040532b0 con 0x7f9a0c15ca40 2023-10-06T05:24:30.635 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:24:30.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 -- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f00674d0 msgr2=0x7f99f0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f00674d0 0x7f99f0069980 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009db0 tx=0x7f9a000091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 -- 172.21.15.31:0/3187069197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c15ca40 msgr2=0x7f9a0c159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c15ca40 0x7f9a0c159740 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c06e040 tx=0x7f9a04018000 comp rx=0 tx=0).stop 2023-10-06T05:24:30.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 -- 172.21.15.31:0/3187069197 shutdown_connections 2023-10-06T05:24:30.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99f00674d0 0x7f99f0069980 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a0c15ca40 0x7f9a0c159740 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a0c06a300 0x7f9a0c15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 --2- 172.21.15.31:0/3187069197 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a0c0c8f40 0x7f9a0c14b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 -- 172.21.15.31:0/3187069197 >> 172.21.15.31:0/3187069197 conn(0x7f9a0c0ce290 msgr2=0x7f9a0c15a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 -- 172.21.15.31:0/3187069197 shutdown_connections 2023-10-06T05:24:30.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.635+0000 7f9a12dff700 1 -- 172.21.15.31:0/3187069197 wait complete. 2023-10-06T05:24:30.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:24:30.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.860+0000 7f8c51dff700 1 Processor -- start 2023-10-06T05:24:30.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.861+0000 7f8c51dff700 1 -- start start 2023-10-06T05:24:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.862+0000 7f8c51dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8c4c06a300 0x7f8c4c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.862+0000 7f8c51dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4c068da0 con 0x7f8c4c15ca40 2023-10-06T05:24:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.862+0000 7f8c51dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4c069080 con 0x7f8c4c0c8f40 2023-10-06T05:24:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.862+0000 7f8c51dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4c0cabc0 con 0x7f8c4c06a300 2023-10-06T05:24:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.862+0000 7f8c4bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8c4c06a300 0x7f8c4c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.862+0000 7f8c4bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8c4c06a300 0x7f8c4c068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59434/0 (socket says 172.21.15.31:59434) 2023-10-06T05:24:30.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.862+0000 7f8c4bfff700 1 -- 172.21.15.31:0/2722459717 learned_addr learned my addr 172.21.15.31:0/2722459717 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.862+0000 7f8c4bfff700 1 -- 172.21.15.31:0/2722459717 >> v1:172.21.15.62:6789/0 conn(0x7f8c4c0c8f40 legacy=0x7f8c4c0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.862+0000 7f8c4bfff700 1 -- 172.21.15.31:0/2722459717 >> v1:172.21.15.31:6789/0 conn(0x7f8c4c15ca40 legacy=0x7f8c4c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.862+0000 7f8c4bfff700 1 -- 172.21.15.31:0/2722459717 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c069360 con 0x7f8c4c06a300 2023-10-06T05:24:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.863+0000 7f8c4bfff700 1 --2- 172.21.15.31:0/2722459717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c06a300 0x7f8c4c068670 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009f20 tx=0x7f8c44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d1278df5b4c9bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.863+0000 7f8c4a7fc700 1 -- 172.21.15.31:0/2722459717 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400e070 con 0x7f8c4c06a300 2023-10-06T05:24:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.863+0000 7f8c4a7fc700 1 -- 172.21.15.31:0/2722459717 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4400ba80 con 0x7f8c4c06a300 2023-10-06T05:24:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.863+0000 7f8c4a7fc700 1 -- 172.21.15.31:0/2722459717 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c440092e0 con 0x7f8c4c06a300 2023-10-06T05:24:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.863+0000 7f8c51dff700 1 -- 172.21.15.31:0/2722459717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c06a300 msgr2=0x7f8c4c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.863+0000 7f8c51dff700 1 --2- 172.21.15.31:0/2722459717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c06a300 0x7f8c4c068670 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009f20 tx=0x7f8c44009840 comp rx=0 tx=0).stop 2023-10-06T05:24:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.864+0000 7f8c51dff700 1 -- 172.21.15.31:0/2722459717 shutdown_connections 2023-10-06T05:24:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.864+0000 7f8c51dff700 1 --2- 172.21.15.31:0/2722459717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c06a300 0x7f8c4c068670 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.864+0000 7f8c51dff700 1 -- 172.21.15.31:0/2722459717 >> 172.21.15.31:0/2722459717 conn(0x7f8c4c0ce290 msgr2=0x7f8c4c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.864+0000 7f8c51dff700 1 -- 172.21.15.31:0/2722459717 shutdown_connections 2023-10-06T05:24:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.864+0000 7f8c51dff700 1 -- 172.21.15.31:0/2722459717 wait complete. 2023-10-06T05:24:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.865+0000 7f8c51dff700 1 Processor -- start 2023-10-06T05:24:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.865+0000 7f8c51dff700 1 -- start start 2023-10-06T05:24:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.866+0000 7f8c51dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c06a300 0x7f8c4c0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.866+0000 7f8c51dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0c8f40 0x7f8c4c0b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.866+0000 7f8c51dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c15ca40 0x7f8c4c14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.866+0000 7f8c51dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0ad690 con 0x7f8c4c15ca40 2023-10-06T05:24:30.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.866+0000 7f8c51dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0b1170 con 0x7f8c4c06a300 2023-10-06T05:24:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.866+0000 7f8c51dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0b2d80 con 0x7f8c4c0c8f40 2023-10-06T05:24:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.866+0000 7f8c4bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c15ca40 0x7f8c4c14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.866+0000 7f8c4bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c15ca40 0x7f8c4c14d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59450/0 (socket says 172.21.15.31:59450) 2023-10-06T05:24:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.866+0000 7f8c4bfff700 1 -- 172.21.15.31:0/565638214 learned_addr learned my addr 172.21.15.31:0/565638214 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c4affd700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0c8f40 0x7f8c4c0b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c4b7fe700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c06a300 0x7f8c4c0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c4bfff700 1 -- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0c8f40 msgr2=0x7f8c4c0b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c4bfff700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0c8f40 0x7f8c4c0b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c4bfff700 1 -- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c06a300 msgr2=0x7f8c4c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c4bfff700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c06a300 0x7f8c4c0ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c4bfff700 1 -- 172.21.15.31:0/565638214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c44009ae0 con 0x7f8c4c15ca40 2023-10-06T05:24:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c4affd700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0c8f40 0x7f8c4c0b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c4bfff700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c15ca40 0x7f8c4c14d210 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c06e040 tx=0x7f8c44018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c337fe700 1 -- 172.21.15.31:0/565638214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44008e80 con 0x7f8c4c15ca40 2023-10-06T05:24:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c337fe700 1 -- 172.21.15.31:0/565638214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c44026460 con 0x7f8c4c15ca40 2023-10-06T05:24:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c337fe700 1 -- 172.21.15.31:0/565638214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44026800 con 0x7f8c4c15ca40 2023-10-06T05:24:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.867+0000 7f8c51dff700 1 -- 172.21.15.31:0/565638214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c0b36b0 con 0x7f8c4c15ca40 2023-10-06T05:24:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.868+0000 7f8c51dff700 1 -- 172.21.15.31:0/565638214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c0b3c70 con 0x7f8c4c15ca40 2023-10-06T05:24:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.869+0000 7f8c51dff700 1 -- 172.21.15.31:0/565638214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4c04c380 con 0x7f8c4c15ca40 2023-10-06T05:24:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.869+0000 7f8c337fe700 1 -- 172.21.15.31:0/565638214 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f8c44003040 con 0x7f8c4c15ca40 2023-10-06T05:24:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.870+0000 7f8c337fe700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c380674d0 0x7f8c38069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.870+0000 7f8c337fe700 1 -- 172.21.15.31:0/565638214 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f8c44087410 con 0x7f8c4c15ca40 2023-10-06T05:24:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.871+0000 7f8c4b7fe700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c380674d0 0x7f8c38069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.871+0000 7f8c4b7fe700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c380674d0 0x7f8c38069980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0037c0 tx=0x7f8c3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:30.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:30.874+0000 7f8c337fe700 1 -- 172.21.15.31:0/565638214 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c44053b40 con 0x7f8c4c15ca40 2023-10-06T05:24:31.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.081+0000 7f8c51dff700 1 -- 172.21.15.31:0/565638214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c4c0b12b0 con 0x7f8c4c15ca40 2023-10-06T05:24:31.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.081+0000 7f8c337fe700 1 -- 172.21.15.31:0/565638214 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c44057160 con 0x7f8c4c15ca40 2023-10-06T05:24:31.083 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:24:31.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 -- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c380674d0 msgr2=0x7f8c38069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c380674d0 0x7f8c38069980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0037c0 tx=0x7f8c3c009040 comp rx=0 tx=0).stop 2023-10-06T05:24:31.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 -- 172.21.15.31:0/565638214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c15ca40 msgr2=0x7f8c4c14d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c15ca40 0x7f8c4c14d210 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c06e040 tx=0x7f8c44018000 comp rx=0 tx=0).stop 2023-10-06T05:24:31.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 -- 172.21.15.31:0/565638214 shutdown_connections 2023-10-06T05:24:31.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8c380674d0 0x7f8c38069980 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c15ca40 0x7f8c4c14d210 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c4c06a300 0x7f8c4c0ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 --2- 172.21.15.31:0/565638214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8c4c0c8f40 0x7f8c4c0b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 -- 172.21.15.31:0/565638214 >> 172.21.15.31:0/565638214 conn(0x7f8c4c0ce290 msgr2=0x7f8c4c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 -- 172.21.15.31:0/565638214 shutdown_connections 2023-10-06T05:24:31.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.084+0000 7f8c51dff700 1 -- 172.21.15.31:0/565638214 wait complete. 2023-10-06T05:24:31.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:24:31.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.302+0000 7f6c489ff700 1 Processor -- start 2023-10-06T05:24:31.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.302+0000 7f6c489ff700 1 -- start start 2023-10-06T05:24:31.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.303+0000 7f6c489ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c440b0b90 0x7f6c440af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.303+0000 7f6c489ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c440af6c0 0x7f6c440adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.303+0000 7f6c489ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c440ad690 con 0x7f6c44153d40 2023-10-06T05:24:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.304+0000 7f6c489ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440acd60 con 0x7f6c440af6c0 2023-10-06T05:24:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.304+0000 7f6c489ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440ad040 con 0x7f6c440b0b90 2023-10-06T05:24:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.304+0000 7f6c42d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c440af6c0 0x7f6c440adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.304+0000 7f6c42d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c440af6c0 0x7f6c440adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59456/0 (socket says 172.21.15.31:59456) 2023-10-06T05:24:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.304+0000 7f6c42d9d700 1 -- 172.21.15.31:0/3748541981 learned_addr learned my addr 172.21.15.31:0/3748541981 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:31.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.304+0000 7f6c41d9b700 1 --2- 172.21.15.31:0/3748541981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c440b0b90 0x7f6c440af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.304+0000 7f6c42d9d700 1 -- 172.21.15.31:0/3748541981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c440b0b90 msgr2=0x7f6c440af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.304+0000 7f6c42d9d700 1 --2- 172.21.15.31:0/3748541981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c440b0b90 0x7f6c440af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.304+0000 7f6c42d9d700 1 -- 172.21.15.31:0/3748541981 >> v1:172.21.15.62:6789/0 conn(0x7f6c44153d40 legacy=0x7f6c440b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.304+0000 7f6c42d9d700 1 -- 172.21.15.31:0/3748541981 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440ad320 con 0x7f6c440af6c0 2023-10-06T05:24:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.305+0000 7f6c42d9d700 1 --2- 172.21.15.31:0/3748541981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 0x7f6c440adb20 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009f20 tx=0x7f6c3c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dc7fb9111941a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.305+0000 7f6c4159a700 1 -- 172.21.15.31:0/3748541981 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00e070 con 0x7f6c440af6c0 2023-10-06T05:24:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.305+0000 7f6c4159a700 1 -- 172.21.15.31:0/3748541981 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c00ba80 con 0x7f6c440af6c0 2023-10-06T05:24:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.305+0000 7f6c4159a700 1 -- 172.21.15.31:0/3748541981 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0092e0 con 0x7f6c440af6c0 2023-10-06T05:24:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.305+0000 7f6c489ff700 1 -- 172.21.15.31:0/3748541981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 msgr2=0x7f6c440adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.305+0000 7f6c489ff700 1 --2- 172.21.15.31:0/3748541981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 0x7f6c440adb20 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009f20 tx=0x7f6c3c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.306+0000 7f6c489ff700 1 -- 172.21.15.31:0/3748541981 shutdown_connections 2023-10-06T05:24:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.306+0000 7f6c489ff700 1 --2- 172.21.15.31:0/3748541981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 0x7f6c440adb20 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.306+0000 7f6c489ff700 1 --2- 172.21.15.31:0/3748541981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c440b0b90 0x7f6c440af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.306+0000 7f6c489ff700 1 -- 172.21.15.31:0/3748541981 >> 172.21.15.31:0/3748541981 conn(0x7f6c44067560 msgr2=0x7f6c44067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.306+0000 7f6c489ff700 1 -- 172.21.15.31:0/3748541981 shutdown_connections 2023-10-06T05:24:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.306+0000 7f6c489ff700 1 -- 172.21.15.31:0/3748541981 wait complete. 2023-10-06T05:24:31.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.307+0000 7f6c489ff700 1 Processor -- start 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.307+0000 7f6c489ff700 1 -- start start 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.308+0000 7f6c489ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 0x7f6c440c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.308+0000 7f6c489ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c440b0b90 0x7f6c4406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.308+0000 7f6c489ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c44153d40 0x7f6c4416c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.308+0000 7f6c489ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440c2ce0 con 0x7f6c440af6c0 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.308+0000 7f6c489ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440c1f30 con 0x7f6c440b0b90 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.308+0000 7f6c489ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440c1320 con 0x7f6c44153d40 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.308+0000 7f6c4259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 0x7f6c440c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.308+0000 7f6c4259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 0x7f6c440c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59470/0 (socket says 172.21.15.31:59470) 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.308+0000 7f6c4259c700 1 -- 172.21.15.31:0/3388228505 learned_addr learned my addr 172.21.15.31:0/3388228505 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:31.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c42d9d700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c44153d40 0x7f6c4416c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c41d9b700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c440b0b90 0x7f6c4406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c4259c700 1 -- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c44153d40 msgr2=0x7f6c4416c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c4259c700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c44153d40 0x7f6c4416c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c4259c700 1 -- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c440b0b90 msgr2=0x7f6c4406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c4259c700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c440b0b90 0x7f6c4406d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c4259c700 1 -- 172.21.15.31:0/3388228505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c009ae0 con 0x7f6c440af6c0 2023-10-06T05:24:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c41d9b700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c440b0b90 0x7f6c4406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c4259c700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 0x7f6c440c2f20 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c4e0 tx=0x7f6c3400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c327fc700 1 -- 172.21.15.31:0/3388228505 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c34011590 con 0x7f6c440af6c0 2023-10-06T05:24:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c327fc700 1 -- 172.21.15.31:0/3388228505 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c34011710 con 0x7f6c440af6c0 2023-10-06T05:24:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c489ff700 1 -- 172.21.15.31:0/3388228505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c440c1600 con 0x7f6c440af6c0 2023-10-06T05:24:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c327fc700 1 -- 172.21.15.31:0/3388228505 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c34011ad0 con 0x7f6c440af6c0 2023-10-06T05:24:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.309+0000 7f6c489ff700 1 -- 172.21.15.31:0/3388228505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c440c1c40 con 0x7f6c440af6c0 2023-10-06T05:24:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.310+0000 7f6c489ff700 1 -- 172.21.15.31:0/3388228505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4404c380 con 0x7f6c440af6c0 2023-10-06T05:24:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.311+0000 7f6c327fc700 1 -- 172.21.15.31:0/3388228505 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f6c3401b3b0 con 0x7f6c440af6c0 2023-10-06T05:24:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.312+0000 7f6c327fc700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c240674d0 0x7f6c24069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.313+0000 7f6c327fc700 1 -- 172.21.15.31:0/3388228505 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6c340845f0 con 0x7f6c440af6c0 2023-10-06T05:24:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.313+0000 7f6c41d9b700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c240674d0 0x7f6c24069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.314+0000 7f6c41d9b700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c240674d0 0x7f6c24069980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6c440cb7a0 tx=0x7f6c38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.317+0000 7f6c327fc700 1 -- 172.21.15.31:0/3388228505 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c34050e30 con 0x7f6c440af6c0 2023-10-06T05:24:31.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.518+0000 7f6c489ff700 1 -- 172.21.15.31:0/3388228505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c440afed0 con 0x7f6c440af6c0 2023-10-06T05:24:31.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.519+0000 7f6c327fc700 1 -- 172.21.15.31:0/3388228505 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c34017020 con 0x7f6c440af6c0 2023-10-06T05:24:31.521 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:24:31.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 -- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c240674d0 msgr2=0x7f6c24069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c240674d0 0x7f6c24069980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6c440cb7a0 tx=0x7f6c38008040 comp rx=0 tx=0).stop 2023-10-06T05:24:31.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 -- 172.21.15.31:0/3388228505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 msgr2=0x7f6c440c2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 0x7f6c440c2f20 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c4e0 tx=0x7f6c3400c7f0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 -- 172.21.15.31:0/3388228505 shutdown_connections 2023-10-06T05:24:31.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c240674d0 0x7f6c24069980 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c44153d40 0x7f6c4416c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440af6c0 0x7f6c440c2f20 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 --2- 172.21.15.31:0/3388228505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c440b0b90 0x7f6c4406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 -- 172.21.15.31:0/3388228505 >> 172.21.15.31:0/3388228505 conn(0x7f6c44067560 msgr2=0x7f6c44152820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.521+0000 7f6c489ff700 1 -- 172.21.15.31:0/3388228505 shutdown_connections 2023-10-06T05:24:31.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.522+0000 7f6c489ff700 1 -- 172.21.15.31:0/3388228505 wait complete. 2023-10-06T05:24:31.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:24:31.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.738+0000 7f9814fff700 1 Processor -- start 2023-10-06T05:24:31.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.738+0000 7f9814fff700 1 -- start start 2023-10-06T05:24:31.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.739+0000 7f9814fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9810159560 0x7f98100cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.739+0000 7f9814fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9810159930 0x7f98100cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.739+0000 7f9814fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981015d2d0 con 0x7f98100b0d60 2023-10-06T05:24:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.739+0000 7f9814fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981015d5b0 con 0x7f9810159930 2023-10-06T05:24:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.739+0000 7f9814fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981015d890 con 0x7f9810159560 2023-10-06T05:24:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980ed9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f98100b0d60 0x7f98100b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33778/0 (socket says 172.21.15.31:33778) 2023-10-06T05:24:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980ed9d700 1 -- 172.21.15.31:0/1703827857 learned_addr learned my addr 172.21.15.31:0/1703827857 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980e59c700 1 --2- 172.21.15.31:0/1703827857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9810159560 0x7f98100cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980dd9b700 1 --2- 172.21.15.31:0/1703827857 >> v2:172.21.15.62:3300/0 conn(0x7f9810159930 0x7f98100cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980d59a700 1 -- 172.21.15.31:0/1703827857 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053570339 0 0) 0x7f981015d2d0 con 0x7f98100b0d60 2023-10-06T05:24:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980d59a700 1 -- 172.21.15.31:0/1703827857 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f8003570 con 0x7f98100b0d60 2023-10-06T05:24:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980e59c700 1 -- 172.21.15.31:0/1703827857 >> v2:172.21.15.62:3300/0 conn(0x7f9810159930 msgr2=0x7f98100cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980e59c700 1 --2- 172.21.15.31:0/1703827857 >> v2:172.21.15.62:3300/0 conn(0x7f9810159930 0x7f98100cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980e59c700 1 -- 172.21.15.31:0/1703827857 >> v1:172.21.15.31:6789/0 conn(0x7f98100b0d60 legacy=0x7f98100b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980e59c700 1 -- 172.21.15.31:0/1703827857 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981015db70 con 0x7f9810159560 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.740+0000 7f980dd9b700 1 --2- 172.21.15.31:0/1703827857 >> v2:172.21.15.62:3300/0 conn(0x7f9810159930 0x7f98100cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.741+0000 7f980e59c700 1 --2- 172.21.15.31:0/1703827857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9810159560 0x7f98100cce70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98000098e0 tx=0x7f98000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a28e57acd405514b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.741+0000 7f980d59a700 1 -- 172.21.15.31:0/1703827857 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800011630 con 0x7f9810159560 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.741+0000 7f980d59a700 1 -- 172.21.15.31:0/1703827857 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9800011790 con 0x7f9810159560 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.741+0000 7f980d59a700 1 -- 172.21.15.31:0/1703827857 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800011b30 con 0x7f9810159560 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.742+0000 7f9814fff700 1 -- 172.21.15.31:0/1703827857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9810159560 msgr2=0x7f98100cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.742+0000 7f9814fff700 1 --2- 172.21.15.31:0/1703827857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9810159560 0x7f98100cce70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98000098e0 tx=0x7f98000091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.742+0000 7f9814fff700 1 -- 172.21.15.31:0/1703827857 shutdown_connections 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.742+0000 7f9814fff700 1 --2- 172.21.15.31:0/1703827857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9810159560 0x7f98100cce70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.742+0000 7f9814fff700 1 --2- 172.21.15.31:0/1703827857 >> v2:172.21.15.62:3300/0 conn(0x7f9810159930 0x7f98100cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.742+0000 7f9814fff700 1 -- 172.21.15.31:0/1703827857 >> 172.21.15.31:0/1703827857 conn(0x7f981006ec90 msgr2=0x7f981006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.742+0000 7f9814fff700 1 -- 172.21.15.31:0/1703827857 shutdown_connections 2023-10-06T05:24:31.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.742+0000 7f9814fff700 1 -- 172.21.15.31:0/1703827857 wait complete. 2023-10-06T05:24:31.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.743+0000 7f9814fff700 1 Processor -- start 2023-10-06T05:24:31.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.743+0000 7f9814fff700 1 -- start start 2023-10-06T05:24:31.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.744+0000 7f9814fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98100b0d60 0x7f981015c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.744+0000 7f9814fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810159560 0x7f9810154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.744+0000 7f9814fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9810159930 0x7f98101553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.744+0000 7f9814fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810155dc0 con 0x7f9810159560 2023-10-06T05:24:31.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.744+0000 7f9814fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98100ce8f0 con 0x7f9810159930 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.744+0000 7f9814fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98100c4280 con 0x7f98100b0d60 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810159560 0x7f9810154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810159560 0x7f9810154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59484/0 (socket says 172.21.15.31:59484) 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980dd9b700 1 -- 172.21.15.31:0/638457162 learned_addr learned my addr 172.21.15.31:0/638457162 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980e59c700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98100b0d60 0x7f981015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980ed9d700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9810159930 0x7f98101553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980dd9b700 1 -- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98100b0d60 msgr2=0x7f981015c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980dd9b700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98100b0d60 0x7f981015c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980dd9b700 1 -- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9810159930 msgr2=0x7f98101553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980dd9b700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9810159930 0x7f98101553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980dd9b700 1 -- 172.21.15.31:0/638457162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9804009420 con 0x7f9810159560 2023-10-06T05:24:31.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.745+0000 7f980dd9b700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810159560 0x7f9810154e80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f980400ec80 tx=0x7f98040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.746+0000 7f97fe7fc700 1 -- 172.21.15.31:0/638457162 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980400c850 con 0x7f9810159560 2023-10-06T05:24:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.746+0000 7f97fe7fc700 1 -- 172.21.15.31:0/638457162 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980400c9b0 con 0x7f9810159560 2023-10-06T05:24:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.746+0000 7f97fe7fc700 1 -- 172.21.15.31:0/638457162 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980400cd30 con 0x7f9810159560 2023-10-06T05:24:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.746+0000 7f9814fff700 1 -- 172.21.15.31:0/638457162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9800009420 con 0x7f9810159560 2023-10-06T05:24:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.746+0000 7f9814fff700 1 -- 172.21.15.31:0/638457162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98100ce2a0 con 0x7f9810159560 2023-10-06T05:24:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.747+0000 7f9814fff700 1 -- 172.21.15.31:0/638457162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98100ceaa0 con 0x7f9810159560 2023-10-06T05:24:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.748+0000 7f97fe7fc700 1 -- 172.21.15.31:0/638457162 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9804013680 con 0x7f9810159560 2023-10-06T05:24:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.749+0000 7f97fe7fc700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f97f00674d0 0x7f97f0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.749+0000 7f97fe7fc700 1 -- 172.21.15.31:0/638457162 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9804085670 con 0x7f9810159560 2023-10-06T05:24:31.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.750+0000 7f980e59c700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f97f00674d0 0x7f97f0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:31.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.750+0000 7f980e59c700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f97f00674d0 0x7f97f0069980 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f98000091e0 tx=0x7f9800017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:31.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.753+0000 7f97fe7fc700 1 -- 172.21.15.31:0/638457162 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98100ceaa0 con 0x7f9810159560 2023-10-06T05:24:31.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.956+0000 7f9814fff700 1 -- 172.21.15.31:0/638457162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98100cea60 con 0x7f9810159560 2023-10-06T05:24:31.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.957+0000 7f97fe7fc700 1 -- 172.21.15.31:0/638457162 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9804055940 con 0x7f9810159560 2023-10-06T05:24:31.959 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:24:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 -- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f97f00674d0 msgr2=0x7f97f0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f97f00674d0 0x7f97f0069980 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f98000091e0 tx=0x7f9800017000 comp rx=0 tx=0).stop 2023-10-06T05:24:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 -- 172.21.15.31:0/638457162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810159560 msgr2=0x7f9810154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810159560 0x7f9810154e80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f980400ec80 tx=0x7f98040091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 -- 172.21.15.31:0/638457162 shutdown_connections 2023-10-06T05:24:31.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f97f00674d0 0x7f97f0069980 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9810159930 0x7f98101553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98100b0d60 0x7f981015c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 --2- 172.21.15.31:0/638457162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810159560 0x7f9810154e80 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:31.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 -- 172.21.15.31:0/638457162 >> 172.21.15.31:0/638457162 conn(0x7f981006ec90 msgr2=0x7f98100b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:31.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.959+0000 7f9814fff700 1 -- 172.21.15.31:0/638457162 shutdown_connections 2023-10-06T05:24:31.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:31.960+0000 7f9814fff700 1 -- 172.21.15.31:0/638457162 wait complete. 2023-10-06T05:24:31.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:32.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:32.118 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:24:32.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.176+0000 7fd965fff700 1 Processor -- start 2023-10-06T05:24:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.176+0000 7fd965fff700 1 -- start start 2023-10-06T05:24:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.177+0000 7fd965fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd96013fda0 0x7fd960069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.177+0000 7fd965fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9600ca5b0 con 0x7fd9600c7b60 2023-10-06T05:24:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.177+0000 7fd965fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96006f740 con 0x7fd9600c7f30 2023-10-06T05:24:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.177+0000 7fd965fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600c9c80 con 0x7fd96013fda0 2023-10-06T05:24:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.178+0000 7fd95f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd9600c7b60 0x7fd96006ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33790/0 (socket says 172.21.15.31:33790) 2023-10-06T05:24:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.178+0000 7fd95f7fe700 1 -- 172.21.15.31:0/112112478 learned_addr learned my addr 172.21.15.31:0/112112478 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.178+0000 7fd95ffff700 1 --2- 172.21.15.31:0/112112478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd96013fda0 0x7fd960069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.178+0000 7fd95e7fc700 1 -- 172.21.15.31:0/112112478 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023907399 0 0) 0x7fd9600ca5b0 con 0x7fd9600c7b60 2023-10-06T05:24:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.178+0000 7fd95e7fc700 1 -- 172.21.15.31:0/112112478 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd948003570 con 0x7fd9600c7b60 2023-10-06T05:24:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.178+0000 7fd95e7fc700 1 -- 172.21.15.31:0/112112478 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406279414 0 0) 0x7fd96006f740 con 0x7fd9600c7f30 2023-10-06T05:24:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.178+0000 7fd95e7fc700 1 -- 172.21.15.31:0/112112478 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9600ca5b0 con 0x7fd9600c7f30 2023-10-06T05:24:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.178+0000 7fd95ffff700 1 -- 172.21.15.31:0/112112478 >> v1:172.21.15.62:6789/0 conn(0x7fd9600c7f30 legacy=0x7fd96013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.178+0000 7fd95ffff700 1 -- 172.21.15.31:0/112112478 >> v1:172.21.15.31:6789/0 conn(0x7fd9600c7b60 legacy=0x7fd96006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.178+0000 7fd95ffff700 1 -- 172.21.15.31:0/112112478 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9600c9f60 con 0x7fd96013fda0 2023-10-06T05:24:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.179+0000 7fd95ffff700 1 --2- 172.21.15.31:0/112112478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd96013fda0 0x7fd960069880 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd958009e90 tx=0x7fd958009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df469e31ad60c5f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.179+0000 7fd95e7fc700 1 -- 172.21.15.31:0/112112478 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95800e070 con 0x7fd96013fda0 2023-10-06T05:24:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.179+0000 7fd95e7fc700 1 -- 172.21.15.31:0/112112478 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95800bb80 con 0x7fd96013fda0 2023-10-06T05:24:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.179+0000 7fd95e7fc700 1 -- 172.21.15.31:0/112112478 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd958012430 con 0x7fd96013fda0 2023-10-06T05:24:32.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.180+0000 7fd965fff700 1 -- 172.21.15.31:0/112112478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd96013fda0 msgr2=0x7fd960069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.180+0000 7fd965fff700 1 --2- 172.21.15.31:0/112112478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd96013fda0 0x7fd960069880 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd958009e90 tx=0x7fd958009780 comp rx=0 tx=0).stop 2023-10-06T05:24:32.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.180+0000 7fd965fff700 1 -- 172.21.15.31:0/112112478 shutdown_connections 2023-10-06T05:24:32.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.180+0000 7fd965fff700 1 --2- 172.21.15.31:0/112112478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd96013fda0 0x7fd960069880 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.180+0000 7fd965fff700 1 -- 172.21.15.31:0/112112478 >> 172.21.15.31:0/112112478 conn(0x7fd9600cc0b0 msgr2=0x7fd9600cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.180+0000 7fd965fff700 1 -- 172.21.15.31:0/112112478 shutdown_connections 2023-10-06T05:24:32.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.180+0000 7fd965fff700 1 -- 172.21.15.31:0/112112478 wait complete. 2023-10-06T05:24:32.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.181+0000 7fd965fff700 1 Processor -- start 2023-10-06T05:24:32.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.181+0000 7fd965fff700 1 -- start start 2023-10-06T05:24:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.182+0000 7fd965fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9600c7b60 0x7fd9600ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.182+0000 7fd965fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9600c7f30 0x7fd9600abed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.182+0000 7fd965fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96013fda0 0x7fd9600aa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.182+0000 7fd965fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600c54b0 con 0x7fd96013fda0 2023-10-06T05:24:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.182+0000 7fd965fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600ae5e0 con 0x7fd9600c7f30 2023-10-06T05:24:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.182+0000 7fd965fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600c4b10 con 0x7fd9600c7b60 2023-10-06T05:24:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9600c7b60 0x7fd9600ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96013fda0 0x7fd9600aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9600c7b60 0x7fd9600ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:35762/0 (socket says 172.21.15.31:35762) 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95f7fe700 1 -- 172.21.15.31:0/2755751875 learned_addr learned my addr 172.21.15.31:0/2755751875 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95effd700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9600c7f30 0x7fd9600abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95f7fe700 1 -- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9600c7f30 msgr2=0x7fd9600abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95f7fe700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9600c7f30 0x7fd9600abed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95f7fe700 1 -- 172.21.15.31:0/2755751875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96013fda0 msgr2=0x7fd9600aa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95f7fe700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96013fda0 0x7fd9600aa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95f7fe700 1 -- 172.21.15.31:0/2755751875 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd958009a20 con 0x7fd9600c7b60 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.183+0000 7fd95effd700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9600c7f30 0x7fd9600abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.184+0000 7fd95f7fe700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9600c7b60 0x7fd9600ae000 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd95000c790 tx=0x7fd950007ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.184+0000 7fd9477fe700 1 -- 172.21.15.31:0/2755751875 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9500072e0 con 0x7fd9600c7b60 2023-10-06T05:24:32.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.184+0000 7fd9477fe700 1 -- 172.21.15.31:0/2755751875 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950007440 con 0x7fd9600c7b60 2023-10-06T05:24:32.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.184+0000 7fd9477fe700 1 -- 172.21.15.31:0/2755751875 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9500057f0 con 0x7fd9600c7b60 2023-10-06T05:24:32.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.184+0000 7fd965fff700 1 -- 172.21.15.31:0/2755751875 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9600c4df0 con 0x7fd9600c7b60 2023-10-06T05:24:32.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.184+0000 7fd965fff700 1 -- 172.21.15.31:0/2755751875 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9600ac410 con 0x7fd9600c7b60 2023-10-06T05:24:32.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.185+0000 7fd965fff700 1 -- 172.21.15.31:0/2755751875 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96004c380 con 0x7fd9600c7b60 2023-10-06T05:24:32.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.186+0000 7fd9477fe700 1 -- 172.21.15.31:0/2755751875 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fd950005ba0 con 0x7fd9600c7b60 2023-10-06T05:24:32.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.187+0000 7fd9477fe700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd94c0674d0 0x7fd94c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.187+0000 7fd9477fe700 1 -- 172.21.15.31:0/2755751875 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd950084790 con 0x7fd9600c7b60 2023-10-06T05:24:32.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.187+0000 7fd95effd700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd94c0674d0 0x7fd94c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.188+0000 7fd95effd700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd94c0674d0 0x7fd94c069980 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd9600c4760 tx=0x7fd954009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.194+0000 7fd9477fe700 1 -- 172.21.15.31:0/2755751875 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9500549d0 con 0x7fd9600c7b60 2023-10-06T05:24:32.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.440+0000 7fd965fff700 1 -- 172.21.15.31:0/2755751875 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9600ca240 con 0x7fd9600c7b60 2023-10-06T05:24:32.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.441+0000 7fd9477fe700 1 -- 172.21.15.31:0/2755751875 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fd950054560 con 0x7fd9600c7b60 2023-10-06T05:24:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:32.443 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3816040413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3816040413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3816040413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1295480750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1295480750},{"type":"v1","addr":"172.21.15.31:6834","nonce":1295480750}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2807869230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2807869230},{"type":"v1","addr":"172.21.15.62:6834","nonce":2807869230}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:06.204405+0000","modified":"2023-10-06T05:24:10.122253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7995":{"gid":7995,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1928465499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1928465499},{"type":"v1","addr":"172.21.15.62:6835","nonce":1928465499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.443+0000 7fd965fff700 1 -- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd94c0674d0 msgr2=0x7fd94c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.443+0000 7fd965fff700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd94c0674d0 0x7fd94c069980 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd9600c4760 tx=0x7fd954009040 comp rx=0 tx=0).stop 2023-10-06T05:24:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.443+0000 7fd965fff700 1 -- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9600c7b60 msgr2=0x7fd9600ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.443+0000 7fd965fff700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9600c7b60 0x7fd9600ae000 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd95000c790 tx=0x7fd950007ce0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.443+0000 7fd965fff700 1 -- 172.21.15.31:0/2755751875 shutdown_connections 2023-10-06T05:24:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.443+0000 7fd965fff700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd94c0674d0 0x7fd94c069980 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.443+0000 7fd965fff700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96013fda0 0x7fd9600aa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.443+0000 7fd965fff700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9600c7b60 0x7fd9600ae000 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.443+0000 7fd965fff700 1 --2- 172.21.15.31:0/2755751875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9600c7f30 0x7fd9600abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.443+0000 7fd965fff700 1 -- 172.21.15.31:0/2755751875 >> 172.21.15.31:0/2755751875 conn(0x7fd9600cc0b0 msgr2=0x7fd96013e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.444+0000 7fd965fff700 1 -- 172.21.15.31:0/2755751875 shutdown_connections 2023-10-06T05:24:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.444+0000 7fd965fff700 1 -- 172.21.15.31:0/2755751875 wait complete. 2023-10-06T05:24:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 85 2023-10-06T05:24:32.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.657+0000 7f110c9ff700 1 Processor -- start 2023-10-06T05:24:32.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.657+0000 7f110c9ff700 1 -- start start 2023-10-06T05:24:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.659+0000 7f110c9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1108068d80 0x7f1108159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.659+0000 7f110c9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11080c95b0 con 0x7f11081595a0 2023-10-06T05:24:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.659+0000 7f110c9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11080c9890 con 0x7f11080cc0d0 2023-10-06T05:24:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.659+0000 7f110c9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11080c9b70 con 0x7f1108068d80 2023-10-06T05:24:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.659+0000 7f110659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f11081595a0 0x7f1108068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33804/0 (socket says 172.21.15.31:33804) 2023-10-06T05:24:32.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.659+0000 7f110659c700 1 -- 172.21.15.31:0/2825609983 learned_addr learned my addr 172.21.15.31:0/2825609983 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.659+0000 7f1105d9b700 1 --2- 172.21.15.31:0/2825609983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1108068d80 0x7f1108159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.659+0000 7f110559a700 1 -- 172.21.15.31:0/2825609983 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230984317 0 0) 0x7f11080c95b0 con 0x7f11081595a0 2023-10-06T05:24:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.659+0000 7f110559a700 1 -- 172.21.15.31:0/2825609983 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10f0003570 con 0x7f11081595a0 2023-10-06T05:24:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.659+0000 7f110559a700 1 -- 172.21.15.31:0/2825609983 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802638885 0 0) 0x7f11080c9890 con 0x7f11080cc0d0 2023-10-06T05:24:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.660+0000 7f110559a700 1 -- 172.21.15.31:0/2825609983 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11080c95b0 con 0x7f11080cc0d0 2023-10-06T05:24:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.660+0000 7f110559a700 1 -- 172.21.15.31:0/2825609983 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3544825172 0 0) 0x7f10f0003570 con 0x7f11081595a0 2023-10-06T05:24:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.660+0000 7f1105d9b700 1 -- 172.21.15.31:0/2825609983 >> v1:172.21.15.62:6789/0 conn(0x7f11080cc0d0 legacy=0x7f11080c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.660+0000 7f1105d9b700 1 -- 172.21.15.31:0/2825609983 >> v1:172.21.15.31:6789/0 conn(0x7f11081595a0 legacy=0x7f1108068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.660+0000 7f1105d9b700 1 -- 172.21.15.31:0/2825609983 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11080ca130 con 0x7f1108068d80 2023-10-06T05:24:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.660+0000 7f1105d9b700 1 --2- 172.21.15.31:0/2825609983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1108068d80 0x7f1108159970 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009910 tx=0x7f10fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b28f6acc011e0751 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.660+0000 7f110559a700 1 -- 172.21.15.31:0/2825609983 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8001470 con 0x7f1108068d80 2023-10-06T05:24:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.660+0000 7f110559a700 1 -- 172.21.15.31:0/2825609983 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc011420 con 0x7f1108068d80 2023-10-06T05:24:32.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.660+0000 7f110559a700 1 -- 172.21.15.31:0/2825609983 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0117c0 con 0x7f1108068d80 2023-10-06T05:24:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.661+0000 7f110c9ff700 1 -- 172.21.15.31:0/2825609983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1108068d80 msgr2=0x7f1108159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.661+0000 7f110c9ff700 1 --2- 172.21.15.31:0/2825609983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1108068d80 0x7f1108159970 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009910 tx=0x7f10fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.661+0000 7f110c9ff700 1 -- 172.21.15.31:0/2825609983 shutdown_connections 2023-10-06T05:24:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.661+0000 7f110c9ff700 1 --2- 172.21.15.31:0/2825609983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1108068d80 0x7f1108159970 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.661+0000 7f110c9ff700 1 -- 172.21.15.31:0/2825609983 >> 172.21.15.31:0/2825609983 conn(0x7f110806b620 msgr2=0x7f110806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.661+0000 7f110c9ff700 1 -- 172.21.15.31:0/2825609983 shutdown_connections 2023-10-06T05:24:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.661+0000 7f110c9ff700 1 -- 172.21.15.31:0/2825609983 wait complete. 2023-10-06T05:24:32.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.662+0000 7f110c9ff700 1 Processor -- start 2023-10-06T05:24:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.663+0000 7f110c9ff700 1 -- start start 2023-10-06T05:24:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.663+0000 7f110c9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1108068d80 0x7f110815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.663+0000 7f110c9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11080cc0d0 0x7f110815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110c9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11081595a0 0x7f110815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110c9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110815deb0 con 0x7f1108068d80 2023-10-06T05:24:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110c9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110815e190 con 0x7f11081595a0 2023-10-06T05:24:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110c9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110815e470 con 0x7f11080cc0d0 2023-10-06T05:24:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1108068d80 0x7f110815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1108068d80 0x7f110815b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59496/0 (socket says 172.21.15.31:59496) 2023-10-06T05:24:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110659c700 1 -- 172.21.15.31:0/3392586240 learned_addr learned my addr 172.21.15.31:0/3392586240 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f1105d9b700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11080cc0d0 0x7f110815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f1106d9d700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11081595a0 0x7f110815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110659c700 1 -- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11080cc0d0 msgr2=0x7f110815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110659c700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11080cc0d0 0x7f110815d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110659c700 1 -- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11081595a0 msgr2=0x7f110815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110659c700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11081595a0 0x7f110815d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110659c700 1 -- 172.21.15.31:0/3392586240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc009420 con 0x7f1108068d80 2023-10-06T05:24:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.664+0000 7f110659c700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1108068d80 0x7f110815b590 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c7f0 tx=0x7f10f8001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.665+0000 7f10f67fc700 1 -- 172.21.15.31:0/3392586240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8007360 con 0x7f1108068d80 2023-10-06T05:24:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.665+0000 7f10f67fc700 1 -- 172.21.15.31:0/3392586240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f80074e0 con 0x7f1108068d80 2023-10-06T05:24:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.665+0000 7f10f67fc700 1 -- 172.21.15.31:0/3392586240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8004830 con 0x7f1108068d80 2023-10-06T05:24:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.665+0000 7f110c9ff700 1 -- 172.21.15.31:0/3392586240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110815e750 con 0x7f1108068d80 2023-10-06T05:24:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.665+0000 7f110c9ff700 1 -- 172.21.15.31:0/3392586240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1108151e10 con 0x7f1108068d80 2023-10-06T05:24:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.666+0000 7f110c9ff700 1 -- 172.21.15.31:0/3392586240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110804c380 con 0x7f1108068d80 2023-10-06T05:24:32.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.667+0000 7f10f67fc700 1 -- 172.21.15.31:0/3392586240 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f10f8004c00 con 0x7f1108068d80 2023-10-06T05:24:32.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.668+0000 7f10f67fc700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10e80674d0 0x7f10e8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:32.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.668+0000 7f10f67fc700 1 -- 172.21.15.31:0/3392586240 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f10f8084820 con 0x7f1108068d80 2023-10-06T05:24:32.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.668+0000 7f1105d9b700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10e80674d0 0x7f10e8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:32.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.669+0000 7f1105d9b700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10e80674d0 0x7f10e8069980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0030c0 tx=0x7f10fc016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:32.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.672+0000 7f10f67fc700 1 -- 172.21.15.31:0/3392586240 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10f8051140 con 0x7f1108068d80 2023-10-06T05:24:32.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.872+0000 7f110c9ff700 1 -- 172.21.15.31:0/3392586240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f110806bb90 con 0x7f1108068d80 2023-10-06T05:24:32.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.873+0000 7f10f67fc700 1 -- 172.21.15.31:0/3392586240 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16972 (secure 0 0 0) 0x7f10f8018020 con 0x7f1108068d80 2023-10-06T05:24:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:32.875 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":127,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:07.702885+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:03.921897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:05.190243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:32.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 -- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10e80674d0 msgr2=0x7f10e8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10e80674d0 0x7f10e8069980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0030c0 tx=0x7f10fc016000 comp rx=0 tx=0).stop 2023-10-06T05:24:32.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 -- 172.21.15.31:0/3392586240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1108068d80 msgr2=0x7f110815b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:32.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1108068d80 0x7f110815b590 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c7f0 tx=0x7f10f8001a10 comp rx=0 tx=0).stop 2023-10-06T05:24:32.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 -- 172.21.15.31:0/3392586240 shutdown_connections 2023-10-06T05:24:32.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10e80674d0 0x7f10e8069980 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11081595a0 0x7f110815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1108068d80 0x7f110815b590 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 --2- 172.21.15.31:0/3392586240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11080cc0d0 0x7f110815d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:32.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 -- 172.21.15.31:0/3392586240 >> 172.21.15.31:0/3392586240 conn(0x7f110806b620 msgr2=0x7f11081538d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:32.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.875+0000 7f110c9ff700 1 -- 172.21.15.31:0/3392586240 shutdown_connections 2023-10-06T05:24:32.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:32.876+0000 7f110c9ff700 1 -- 172.21.15.31:0/3392586240 wait complete. 2023-10-06T05:24:32.895 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:24:32.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:33.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.090+0000 7fd425dff700 1 Processor -- start 2023-10-06T05:24:33.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.090+0000 7fd425dff700 1 -- start start 2023-10-06T05:24:33.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.091+0000 7fd425dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd420159500 0x7fd4200b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.091+0000 7fd425dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 0x7fd4200cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.091+0000 7fd425dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200c3f40 con 0x7fd4201598d0 2023-10-06T05:24:33.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.091+0000 7fd425dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4200b2860 con 0x7fd420159500 2023-10-06T05:24:33.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.091+0000 7fd425dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42015d7d0 con 0x7fd42015d2d0 2023-10-06T05:24:33.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41effd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 0x7fd4200cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41f7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd420159500 0x7fd4200b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41f7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd420159500 0x7fd4200b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:41192/0 (socket says 172.21.15.31:41192) 2023-10-06T05:24:33.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41f7fe700 1 -- 172.21.15.31:0/3584109 learned_addr learned my addr 172.21.15.31:0/3584109 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:33.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41ffff700 1 --1- 172.21.15.31:0/3584109 >> v1:172.21.15.62:6789/0 conn(0x7fd4201598d0 0x7fd4200aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:34866/0 (socket says 172.21.15.31:34866) 2023-10-06T05:24:33.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41e7fc700 1 -- 172.21.15.31:0/3584109 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858630594 0 0) 0x7fd4200c3f40 con 0x7fd4201598d0 2023-10-06T05:24:33.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41e7fc700 1 -- 172.21.15.31:0/3584109 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd408003570 con 0x7fd4201598d0 2023-10-06T05:24:33.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41f7fe700 1 -- 172.21.15.31:0/3584109 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 msgr2=0x7fd4200cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:33.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41f7fe700 1 --2- 172.21.15.31:0/3584109 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 0x7fd4200cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41f7fe700 1 -- 172.21.15.31:0/3584109 >> v1:172.21.15.62:6789/0 conn(0x7fd4201598d0 legacy=0x7fd4200aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41f7fe700 1 -- 172.21.15.31:0/3584109 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4200cb000 con 0x7fd420159500 2023-10-06T05:24:33.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.092+0000 7fd41effd700 1 --2- 172.21.15.31:0/3584109 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 0x7fd4200cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:33.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.093+0000 7fd41f7fe700 1 --2- 172.21.15.31:0/3584109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd420159500 0x7fd4200b0ab0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd4100099b0 tx=0x7fd410009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fb968b6cc2eae90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.093+0000 7fd41e7fc700 1 -- 172.21.15.31:0/3584109 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41000b680 con 0x7fd420159500 2023-10-06T05:24:33.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.093+0000 7fd41e7fc700 1 -- 172.21.15.31:0/3584109 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41000b7e0 con 0x7fd420159500 2023-10-06T05:24:33.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.093+0000 7fd41e7fc700 1 -- 172.21.15.31:0/3584109 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41000bb80 con 0x7fd420159500 2023-10-06T05:24:33.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.094+0000 7fd425dff700 1 -- 172.21.15.31:0/3584109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd420159500 msgr2=0x7fd4200b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.094+0000 7fd425dff700 1 --2- 172.21.15.31:0/3584109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd420159500 0x7fd4200b0ab0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd4100099b0 tx=0x7fd410009250 comp rx=0 tx=0).stop 2023-10-06T05:24:33.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.094+0000 7fd425dff700 1 -- 172.21.15.31:0/3584109 shutdown_connections 2023-10-06T05:24:33.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.094+0000 7fd425dff700 1 --2- 172.21.15.31:0/3584109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd420159500 0x7fd4200b0ab0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.094+0000 7fd425dff700 1 --2- 172.21.15.31:0/3584109 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 0x7fd4200cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.094+0000 7fd425dff700 1 -- 172.21.15.31:0/3584109 >> 172.21.15.31:0/3584109 conn(0x7fd42006ec90 msgr2=0x7fd42006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:33.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.094+0000 7fd425dff700 1 -- 172.21.15.31:0/3584109 shutdown_connections 2023-10-06T05:24:33.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.094+0000 7fd425dff700 1 -- 172.21.15.31:0/3584109 wait complete. 2023-10-06T05:24:33.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.095+0000 7fd425dff700 1 Processor -- start 2023-10-06T05:24:33.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.095+0000 7fd425dff700 1 -- start start 2023-10-06T05:24:33.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.096+0000 7fd425dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd420159500 0x7fd42016a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.096+0000 7fd425dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4201598d0 0x7fd42014c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.096+0000 7fd425dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 0x7fd42015b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.096+0000 7fd425dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42014ced0 con 0x7fd420159500 2023-10-06T05:24:33.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.096+0000 7fd425dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd420157cc0 con 0x7fd4201598d0 2023-10-06T05:24:33.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.096+0000 7fd425dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42015a590 con 0x7fd42015d2d0 2023-10-06T05:24:33.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.096+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd420159500 0x7fd42016a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.096+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd420159500 0x7fd42016a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59500/0 (socket says 172.21.15.31:59500) 2023-10-06T05:24:33.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.096+0000 7fd41f7fe700 1 -- 172.21.15.31:0/2271394550 learned_addr learned my addr 172.21.15.31:0/2271394550 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:33.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd41effd700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4201598d0 0x7fd42014c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd41ffff700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 0x7fd42015b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd41f7fe700 1 -- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 msgr2=0x7fd42015b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd41f7fe700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 0x7fd42015b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd41f7fe700 1 -- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4201598d0 msgr2=0x7fd42014c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd41f7fe700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4201598d0 0x7fd42014c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd41f7fe700 1 -- 172.21.15.31:0/2271394550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4140093c0 con 0x7fd420159500 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd41effd700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4201598d0 0x7fd42014c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd41ffff700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 0x7fd42015b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd41f7fe700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd420159500 0x7fd42016a160 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fd410009290 tx=0x7fd410017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd4077fe700 1 -- 172.21.15.31:0/2271394550 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41000b950 con 0x7fd420159500 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd4077fe700 1 -- 172.21.15.31:0/2271394550 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41000bad0 con 0x7fd420159500 2023-10-06T05:24:33.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd4077fe700 1 -- 172.21.15.31:0/2271394550 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41001a680 con 0x7fd420159500 2023-10-06T05:24:33.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.097+0000 7fd425dff700 1 -- 172.21.15.31:0/2271394550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4100094f0 con 0x7fd420159500 2023-10-06T05:24:33.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.098+0000 7fd425dff700 1 -- 172.21.15.31:0/2271394550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42015ab50 con 0x7fd420159500 2023-10-06T05:24:33.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.099+0000 7fd425dff700 1 -- 172.21.15.31:0/2271394550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42015a2e0 con 0x7fd420159500 2023-10-06T05:24:33.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.099+0000 7fd4077fe700 1 -- 172.21.15.31:0/2271394550 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fd41001a9f0 con 0x7fd420159500 2023-10-06T05:24:33.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.100+0000 7fd4077fe700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd40c0674d0 0x7fd40c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.100+0000 7fd4077fe700 1 -- 172.21.15.31:0/2271394550 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd410086d20 con 0x7fd420159500 2023-10-06T05:24:33.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.101+0000 7fd41effd700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd40c0674d0 0x7fd40c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.102+0000 7fd41effd700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd40c0674d0 0x7fd40c069980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd42016a6a0 tx=0x7fd414008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.105+0000 7fd4077fe700 1 -- 172.21.15.31:0/2271394550 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4100535a0 con 0x7fd420159500 2023-10-06T05:24:33.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.352+0000 7fd425dff700 1 -- 172.21.15.31:0/2271394550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd42004c380 con 0x7fd420159500 2023-10-06T05:24:33.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.352+0000 7fd4077fe700 1 -- 172.21.15.31:0/2271394550 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fd410056bc0 con 0x7fd420159500 2023-10-06T05:24:33.355 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:33.355 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3816040413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3816040413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3816040413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1295480750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1295480750},{"type":"v1","addr":"172.21.15.31:6834","nonce":1295480750}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2807869230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2807869230},{"type":"v1","addr":"172.21.15.62:6834","nonce":2807869230}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:06.204405+0000","modified":"2023-10-06T05:24:10.122253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7995":{"gid":7995,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1928465499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1928465499},{"type":"v1","addr":"172.21.15.62:6835","nonce":1928465499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:33.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.354+0000 7fd425dff700 1 -- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd40c0674d0 msgr2=0x7fd40c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.354+0000 7fd425dff700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd40c0674d0 0x7fd40c069980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd42016a6a0 tx=0x7fd414008040 comp rx=0 tx=0).stop 2023-10-06T05:24:33.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.354+0000 7fd425dff700 1 -- 172.21.15.31:0/2271394550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd420159500 msgr2=0x7fd42016a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.354+0000 7fd425dff700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd420159500 0x7fd42016a160 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fd410009290 tx=0x7fd410017000 comp rx=0 tx=0).stop 2023-10-06T05:24:33.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.354+0000 7fd425dff700 1 -- 172.21.15.31:0/2271394550 shutdown_connections 2023-10-06T05:24:33.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.354+0000 7fd425dff700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd40c0674d0 0x7fd40c069980 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.355+0000 7fd425dff700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd42015d2d0 0x7fd42015b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.355+0000 7fd425dff700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd420159500 0x7fd42016a160 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.355+0000 7fd425dff700 1 --2- 172.21.15.31:0/2271394550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4201598d0 0x7fd42014c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.355+0000 7fd425dff700 1 -- 172.21.15.31:0/2271394550 >> 172.21.15.31:0/2271394550 conn(0x7fd42006ec90 msgr2=0x7fd420158990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:33.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.355+0000 7fd425dff700 1 -- 172.21.15.31:0/2271394550 shutdown_connections 2023-10-06T05:24:33.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.355+0000 7fd425dff700 1 -- 172.21.15.31:0/2271394550 wait complete. 2023-10-06T05:24:33.358 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 85 2023-10-06T05:24:33.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.572+0000 7fcce8bff700 1 Processor -- start 2023-10-06T05:24:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.572+0000 7fcce8bff700 1 -- start start 2023-10-06T05:24:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.573+0000 7fcce8bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcce40c7f30 0x7fcce4069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.573+0000 7fcce8bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcce40aee70 0x7fcce413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.573+0000 7fcce8bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce40ca5b0 con 0x7fcce40c7b60 2023-10-06T05:24:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.573+0000 7fcce8bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce40af2f0 con 0x7fcce40c7f30 2023-10-06T05:24:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.573+0000 7fcce8bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce406f740 con 0x7fcce40aee70 2023-10-06T05:24:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.574+0000 7fcce1d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcce40c7f30 0x7fcce4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.574+0000 7fcce1d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcce40c7f30 0x7fcce4069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59514/0 (socket says 172.21.15.31:59514) 2023-10-06T05:24:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.574+0000 7fcce1d9b700 1 -- 172.21.15.31:0/2803923099 learned_addr learned my addr 172.21.15.31:0/2803923099 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.574+0000 7fcce2d9d700 1 --2- 172.21.15.31:0/2803923099 >> v2:172.21.15.62:3300/0 conn(0x7fcce40aee70 0x7fcce413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.574+0000 7fcce1d9b700 1 -- 172.21.15.31:0/2803923099 >> v2:172.21.15.62:3300/0 conn(0x7fcce40aee70 msgr2=0x7fcce413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.574+0000 7fcce1d9b700 1 --2- 172.21.15.31:0/2803923099 >> v2:172.21.15.62:3300/0 conn(0x7fcce40aee70 0x7fcce413f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.574+0000 7fcce1d9b700 1 -- 172.21.15.31:0/2803923099 >> v1:172.21.15.62:6789/0 conn(0x7fcce40c7b60 legacy=0x7fcce406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.574+0000 7fcce1d9b700 1 -- 172.21.15.31:0/2803923099 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40c9c80 con 0x7fcce40c7f30 2023-10-06T05:24:33.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.574+0000 7fcce1d9b700 1 --2- 172.21.15.31:0/2803923099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40c7f30 0x7fcce4069880 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fcce406d6e0 tx=0x7fccd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea8023035b08927 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce159a700 1 -- 172.21.15.31:0/2803923099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8009e70 con 0x7fcce40c7f30 2023-10-06T05:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce159a700 1 -- 172.21.15.31:0/2803923099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd800b860 con 0x7fcce40c7f30 2023-10-06T05:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce159a700 1 -- 172.21.15.31:0/2803923099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd800bc00 con 0x7fcce40c7f30 2023-10-06T05:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce8bff700 1 -- 172.21.15.31:0/2803923099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40c7f30 msgr2=0x7fcce4069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce8bff700 1 --2- 172.21.15.31:0/2803923099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40c7f30 0x7fcce4069880 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fcce406d6e0 tx=0x7fccd8009250 comp rx=0 tx=0).stop 2023-10-06T05:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce8bff700 1 -- 172.21.15.31:0/2803923099 shutdown_connections 2023-10-06T05:24:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce8bff700 1 --2- 172.21.15.31:0/2803923099 >> v2:172.21.15.62:3300/0 conn(0x7fcce40aee70 0x7fcce413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce8bff700 1 --2- 172.21.15.31:0/2803923099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40c7f30 0x7fcce4069880 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce8bff700 1 -- 172.21.15.31:0/2803923099 >> 172.21.15.31:0/2803923099 conn(0x7fcce40cc0b0 msgr2=0x7fcce40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce8bff700 1 -- 172.21.15.31:0/2803923099 shutdown_connections 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.575+0000 7fcce8bff700 1 -- 172.21.15.31:0/2803923099 wait complete. 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.577+0000 7fcce8bff700 1 Processor -- start 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.578+0000 7fcce8bff700 1 -- start start 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce8bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40aee70 0x7fcce40aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce8bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcce40c7b60 0x7fcce413ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce8bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcce40c7f30 0x7fcce4069df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce8bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce416a9a0 con 0x7fcce40aee70 2023-10-06T05:24:33.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce8bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce413f160 con 0x7fcce40c7f30 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce8bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4156900 con 0x7fcce40c7b60 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40aee70 0x7fcce40aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40aee70 0x7fcce40aad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59518/0 (socket says 172.21.15.31:59518) 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce259c700 1 -- 172.21.15.31:0/635464502 learned_addr learned my addr 172.21.15.31:0/635464502 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce2d9d700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcce40c7f30 0x7fcce4069df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.579+0000 7fcce1d9b700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcce40c7b60 0x7fcce413ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fcce259c700 1 -- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcce40c7b60 msgr2=0x7fcce413ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fcce259c700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcce40c7b60 0x7fcce413ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fcce259c700 1 -- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcce40c7f30 msgr2=0x7fcce4069df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fcce259c700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcce40c7f30 0x7fcce4069df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fcce259c700 1 -- 172.21.15.31:0/635464502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd80094f0 con 0x7fcce40aee70 2023-10-06T05:24:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fcce2d9d700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcce40c7f30 0x7fcce4069df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:33.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fcce259c700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40aee70 0x7fcce40aad50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fccd400c7f0 tx=0x7fccd4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fccd27fc700 1 -- 172.21.15.31:0/635464502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd4011590 con 0x7fcce40aee70 2023-10-06T05:24:33.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fccd27fc700 1 -- 172.21.15.31:0/635464502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd4011710 con 0x7fcce40aee70 2023-10-06T05:24:33.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fcce8bff700 1 -- 172.21.15.31:0/635464502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce4169db0 con 0x7fcce40aee70 2023-10-06T05:24:33.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fccd27fc700 1 -- 172.21.15.31:0/635464502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd4011ad0 con 0x7fcce40aee70 2023-10-06T05:24:33.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.580+0000 7fcce8bff700 1 -- 172.21.15.31:0/635464502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce416a350 con 0x7fcce40aee70 2023-10-06T05:24:33.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.581+0000 7fcce8bff700 1 -- 172.21.15.31:0/635464502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce404c380 con 0x7fcce40aee70 2023-10-06T05:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.582+0000 7fccd27fc700 1 -- 172.21.15.31:0/635464502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fccd401b3b0 con 0x7fcce40aee70 2023-10-06T05:24:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.583+0000 7fccd27fc700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fccc40674d0 0x7fccc4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.584+0000 7fccd27fc700 1 -- 172.21.15.31:0/635464502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fccd4084660 con 0x7fcce40aee70 2023-10-06T05:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.584+0000 7fcce1d9b700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fccc40674d0 0x7fccc4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.585+0000 7fcce1d9b700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fccc40674d0 0x7fccc4069980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcce4069dc0 tx=0x7fccd8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:33.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.587+0000 7fccd27fc700 1 -- 172.21.15.31:0/635464502 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccd4050f50 con 0x7fcce40aee70 2023-10-06T05:24:33.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.832+0000 7fcce8bff700 1 -- 172.21.15.31:0/635464502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcce40c9f60 con 0x7fcce40aee70 2023-10-06T05:24:33.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.833+0000 7fccd27fc700 1 -- 172.21.15.31:0/635464502 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fccd4017090 con 0x7fcce40aee70 2023-10-06T05:24:33.835 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:33.835 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3816040413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3816040413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3816040413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1295480750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1295480750},{"type":"v1","addr":"172.21.15.31:6834","nonce":1295480750}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2807869230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2807869230},{"type":"v1","addr":"172.21.15.62:6834","nonce":2807869230}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:06.204405+0000","modified":"2023-10-06T05:24:10.122253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7995":{"gid":7995,"name":"b","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/1928465499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1928465499},{"type":"v1","addr":"172.21.15.62:6835","nonce":1928465499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:24:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 -- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fccc40674d0 msgr2=0x7fccc4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fccc40674d0 0x7fccc4069980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcce4069dc0 tx=0x7fccd8017000 comp rx=0 tx=0).stop 2023-10-06T05:24:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 -- 172.21.15.31:0/635464502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40aee70 msgr2=0x7fcce40aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40aee70 0x7fcce40aad50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fccd400c7f0 tx=0x7fccd4007f70 comp rx=0 tx=0).stop 2023-10-06T05:24:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 -- 172.21.15.31:0/635464502 shutdown_connections 2023-10-06T05:24:33.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fccc40674d0 0x7fccc4069980 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcce40c7f30 0x7fcce4069df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce40aee70 0x7fcce40aad50 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 --2- 172.21.15.31:0/635464502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcce40c7b60 0x7fcce413ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 -- 172.21.15.31:0/635464502 >> 172.21.15.31:0/635464502 conn(0x7fcce40cc0b0 msgr2=0x7fcce40c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 -- 172.21.15.31:0/635464502 shutdown_connections 2023-10-06T05:24:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:33.835+0000 7fcce8bff700 1 -- 172.21.15.31:0/635464502 wait complete. 2023-10-06T05:24:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 85 2023-10-06T05:24:33.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.050+0000 7f1e3d5ff700 1 Processor -- start 2023-10-06T05:24:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.050+0000 7f1e3d5ff700 1 -- start start 2023-10-06T05:24:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.051+0000 7f1e3d5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3806a570 0x7f1e38162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.051+0000 7f1e3d5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380c71b0 con 0x7f1e3813a1e0 2023-10-06T05:24:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.051+0000 7f1e3d5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380c7490 con 0x7f1e3813a5b0 2023-10-06T05:24:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.052+0000 7f1e3d5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380c7770 con 0x7f1e3806a570 2023-10-06T05:24:34.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.052+0000 7f1e36ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1e3813a1e0 0x7f1e380c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33810/0 (socket says 172.21.15.31:33810) 2023-10-06T05:24:34.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.052+0000 7f1e36ffd700 1 -- 172.21.15.31:0/795193818 learned_addr learned my addr 172.21.15.31:0/795193818 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:34.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.052+0000 7f1e377fe700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3806a570 0x7f1e38162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.052+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269062410 0 0) 0x7f1e380c71b0 con 0x7f1e3813a1e0 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.052+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e20003570 con 0x7f1e3813a1e0 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.052+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787599729 0 0) 0x7f1e380c7490 con 0x7f1e3813a5b0 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.052+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e380c71b0 con 0x7f1e3813a5b0 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2486867280 0 0) 0x7f1e20003570 con 0x7f1e3813a1e0 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3806a570 msgr2=0x7f1e38162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:34.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3806a570 0x7f1e38162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 >> v1:172.21.15.62:6789/0 conn(0x7f1e3813a5b0 legacy=0x7f1e3806d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380c7a50 con 0x7f1e3813a1e0 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f1e280015f0 con 0x7f1e3813a1e0 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 >> v1:172.21.15.31:6789/0 conn(0x7f1e3813a1e0 legacy=0x7f1e380c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3806a570 0x7f1e20003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e200041d0 0x7f1e20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20004d50 0x7f1e200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380c7770 con 0x7f1e200041d0 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2000bbf0 con 0x7f1e20004d50 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2000bdb0 con 0x7f1e3806a570 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e367fc700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e200041d0 0x7f1e20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e377fe700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20004d50 0x7f1e200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e36ffd700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3806a570 0x7f1e20003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.053+0000 7f1e367fc700 1 -- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3806a570 msgr2=0x7f1e20003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.054+0000 7f1e367fc700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3806a570 0x7f1e20003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.054+0000 7f1e367fc700 1 -- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20004d50 msgr2=0x7f1e200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.054+0000 7f1e367fc700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e20004d50 0x7f1e200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.054+0000 7f1e367fc700 1 -- 172.21.15.31:0/795193818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e20003570 con 0x7f1e200041d0 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.054+0000 7f1e367fc700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e200041d0 0x7f1e20004810 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c001a20 tx=0x7f1e2c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e47d59ac347235 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.054+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00c5d0 con 0x7f1e200041d0 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.054+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c00c730 con 0x7f1e200041d0 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.054+0000 7f1e35ffb700 1 -- 172.21.15.31:0/795193818 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00cad0 con 0x7f1e200041d0 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.054+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/795193818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e200041d0 msgr2=0x7f1e20004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.054+0000 7f1e3d5ff700 1 --2- 172.21.15.31:0/795193818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e200041d0 0x7f1e20004810 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c001a20 tx=0x7f1e2c007d30 comp rx=0 tx=0).stop 2023-10-06T05:24:34.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.055+0000 7f1e377fe700 1 -- 172.21.15.31:0/795193818 reap_dead start 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.055+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/795193818 shutdown_connections 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.055+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/795193818 >> 172.21.15.31:0/795193818 conn(0x7f1e380cc0b0 msgr2=0x7f1e380cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.055+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/795193818 shutdown_connections 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.055+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/795193818 wait complete. 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.056+0000 7f1e3d5ff700 1 Processor -- start 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.056+0000 7f1e3d5ff700 1 -- start start 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.057+0000 7f1e3d5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e200041d0 0x7f1e3813b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.057+0000 7f1e3d5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3813d1e0 0x7f1e381398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.057+0000 7f1e3d5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3814fa20 0x7f1e3814fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.057+0000 7f1e3d5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3813da10 con 0x7f1e3813d1e0 2023-10-06T05:24:34.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.057+0000 7f1e3d5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3813bdf0 con 0x7f1e200041d0 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.057+0000 7f1e3d5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380aff80 con 0x7f1e3814fa20 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.057+0000 7f1e367fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3813d1e0 0x7f1e381398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e367fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3813d1e0 0x7f1e381398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59530/0 (socket says 172.21.15.31:59530) 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e367fc700 1 -- 172.21.15.31:0/3224093214 learned_addr learned my addr 172.21.15.31:0/3224093214 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e36ffd700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e200041d0 0x7f1e3813b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e377fe700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3814fa20 0x7f1e3814fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e367fc700 1 -- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3814fa20 msgr2=0x7f1e3814fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e367fc700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3814fa20 0x7f1e3814fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e367fc700 1 -- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e200041d0 msgr2=0x7f1e3813b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e367fc700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e200041d0 0x7f1e3813b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e367fc700 1 -- 172.21.15.31:0/3224093214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380b0260 con 0x7f1e3813d1e0 2023-10-06T05:24:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e36ffd700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e200041d0 0x7f1e3813b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e367fc700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3813d1e0 0x7f1e381398f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1e200057f0 tx=0x7f1e2c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e1effd700 1 -- 172.21.15.31:0/3224093214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0013a0 con 0x7f1e3813d1e0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e1effd700 1 -- 172.21.15.31:0/3224093214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c000fe0 con 0x7f1e3813d1e0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/3224093214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e380b0540 con 0x7f1e3813d1e0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.058+0000 7f1e1effd700 1 -- 172.21.15.31:0/3224093214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00cad0 con 0x7f1e3813d1e0 2023-10-06T05:24:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.059+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/3224093214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e38139e30 con 0x7f1e3813d1e0 2023-10-06T05:24:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.060+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/3224093214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e380b0a60 con 0x7f1e3813d1e0 2023-10-06T05:24:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.061+0000 7f1e1effd700 1 -- 172.21.15.31:0/3224093214 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f1e2c019070 con 0x7f1e3813d1e0 2023-10-06T05:24:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.061+0000 7f1e1effd700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e240674d0 0x7f1e24069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.061+0000 7f1e1effd700 1 -- 172.21.15.31:0/3224093214 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f1e2c086600 con 0x7f1e3813d1e0 2023-10-06T05:24:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.062+0000 7f1e36ffd700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e240674d0 0x7f1e24069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.062+0000 7f1e36ffd700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e240674d0 0x7f1e24069980 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1e28002d40 tx=0x7f1e28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.066+0000 7f1e1effd700 1 -- 172.21.15.31:0/3224093214 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e380b0a60 con 0x7f1e3813d1e0 2023-10-06T05:24:34.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.268+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/3224093214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e380b0a60 con 0x7f1e3813d1e0 2023-10-06T05:24:34.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.269+0000 7f1e1effd700 1 -- 172.21.15.31:0/3224093214 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16972 (secure 0 0 0) 0x7f1e380b0a60 con 0x7f1e3813d1e0 2023-10-06T05:24:34.271 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:34.271 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":127,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:07.702885+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:03.921897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:05.190243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:34.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e240674d0 msgr2=0x7f1e24069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e240674d0 0x7f1e24069980 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1e28002d40 tx=0x7f1e28009040 comp rx=0 tx=0).stop 2023-10-06T05:24:34.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/3224093214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3813d1e0 msgr2=0x7f1e381398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3813d1e0 0x7f1e381398f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1e200057f0 tx=0x7f1e2c015000 comp rx=0 tx=0).stop 2023-10-06T05:24:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/3224093214 shutdown_connections 2023-10-06T05:24:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1e240674d0 0x7f1e24069980 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e3814fa20 0x7f1e3814fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e200041d0 0x7f1e3813b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 --2- 172.21.15.31:0/3224093214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3813d1e0 0x7f1e381398f0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/3224093214 >> 172.21.15.31:0/3224093214 conn(0x7f1e380cc0b0 msgr2=0x7f1e380c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/3224093214 shutdown_connections 2023-10-06T05:24:34.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.271+0000 7f1e3d5ff700 1 -- 172.21.15.31:0/3224093214 wait complete. 2023-10-06T05:24:34.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:24:34.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.486+0000 7feef83ff700 1 Processor -- start 2023-10-06T05:24:34.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.486+0000 7feef83ff700 1 -- start start 2023-10-06T05:24:34.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.487+0000 7feef83ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7b60 0x7feef0069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.487+0000 7feef83ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feef00c7f30 0x7feef00aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.487+0000 7feef83ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef00ca5b0 con 0x7feef006ef50 2023-10-06T05:24:34.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.487+0000 7feef83ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef006f740 con 0x7feef00c7f30 2023-10-06T05:24:34.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.487+0000 7feef83ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef00c9c80 con 0x7feef00c7b60 2023-10-06T05:24:34.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.488+0000 7feef599a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feef00c7f30 0x7feef00aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.488+0000 7feef599a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feef00c7f30 0x7feef00aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:41262/0 (socket says 172.21.15.31:41262) 2023-10-06T05:24:34.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.488+0000 7feef619b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7b60 0x7feef0069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.488+0000 7feef599a700 1 -- 172.21.15.31:0/3380149142 learned_addr learned my addr 172.21.15.31:0/3380149142 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:34.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.488+0000 7feef619b700 1 -- 172.21.15.31:0/3380149142 >> v2:172.21.15.62:3300/0 conn(0x7feef00c7f30 msgr2=0x7feef00aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:34.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.488+0000 7feef619b700 1 --2- 172.21.15.31:0/3380149142 >> v2:172.21.15.62:3300/0 conn(0x7feef00c7f30 0x7feef00aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.488+0000 7feef619b700 1 -- 172.21.15.31:0/3380149142 >> v1:172.21.15.62:6789/0 conn(0x7feef006ef50 legacy=0x7feef013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.488+0000 7feef619b700 1 -- 172.21.15.31:0/3380149142 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef00c9f60 con 0x7feef00c7b60 2023-10-06T05:24:34.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.488+0000 7feef599a700 1 --2- 172.21.15.31:0/3380149142 >> v2:172.21.15.62:3300/0 conn(0x7feef00c7f30 0x7feef00aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:34.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.489+0000 7feef619b700 1 --2- 172.21.15.31:0/3380149142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7b60 0x7feef0069880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feeec0098d0 tx=0x7feeec009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b56bda0ca1c43465 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.489+0000 7feef5199700 1 -- 172.21.15.31:0/3380149142 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec005c10 con 0x7feef00c7b60 2023-10-06T05:24:34.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.489+0000 7feef5199700 1 -- 172.21.15.31:0/3380149142 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec005d70 con 0x7feef00c7b60 2023-10-06T05:24:34.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.489+0000 7feef5199700 1 -- 172.21.15.31:0/3380149142 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec006280 con 0x7feef00c7b60 2023-10-06T05:24:34.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.489+0000 7feef83ff700 1 -- 172.21.15.31:0/3380149142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7b60 msgr2=0x7feef0069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.489+0000 7feef83ff700 1 --2- 172.21.15.31:0/3380149142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7b60 0x7feef0069880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feeec0098d0 tx=0x7feeec009120 comp rx=0 tx=0).stop 2023-10-06T05:24:34.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.490+0000 7feef83ff700 1 -- 172.21.15.31:0/3380149142 shutdown_connections 2023-10-06T05:24:34.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.490+0000 7feef83ff700 1 --2- 172.21.15.31:0/3380149142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7b60 0x7feef0069880 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.490+0000 7feef83ff700 1 --2- 172.21.15.31:0/3380149142 >> v2:172.21.15.62:3300/0 conn(0x7feef00c7f30 0x7feef00aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.490+0000 7feef83ff700 1 -- 172.21.15.31:0/3380149142 >> 172.21.15.31:0/3380149142 conn(0x7feef00cc0b0 msgr2=0x7feef00cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:34.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.490+0000 7feef83ff700 1 -- 172.21.15.31:0/3380149142 shutdown_connections 2023-10-06T05:24:34.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.490+0000 7feef83ff700 1 -- 172.21.15.31:0/3380149142 wait complete. 2023-10-06T05:24:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.491+0000 7feef83ff700 1 Processor -- start 2023-10-06T05:24:34.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.491+0000 7feef83ff700 1 -- start start 2023-10-06T05:24:34.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.492+0000 7feef83ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef006ef50 0x7feef0154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.492+0000 7feef83ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef00c7b60 0x7feef00aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.492+0000 7feef83ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7f30 0x7feef0155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.492+0000 7feef83ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00c1780 con 0x7feef006ef50 2023-10-06T05:24:34.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.492+0000 7feef83ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00ab1e0 con 0x7feef00c7b60 2023-10-06T05:24:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.492+0000 7feef83ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef00c0e50 con 0x7feef00c7f30 2023-10-06T05:24:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef619b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef006ef50 0x7feef0154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef619b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef006ef50 0x7feef0154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59546/0 (socket says 172.21.15.31:59546) 2023-10-06T05:24:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef619b700 1 -- 172.21.15.31:0/3640163949 learned_addr learned my addr 172.21.15.31:0/3640163949 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef599a700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef00c7b60 0x7feef00aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef699c700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7f30 0x7feef0155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef619b700 1 -- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7f30 msgr2=0x7feef0155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef619b700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7f30 0x7feef0155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef619b700 1 -- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef00c7b60 msgr2=0x7feef00aa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef619b700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef00c7b60 0x7feef00aa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef619b700 1 -- 172.21.15.31:0/3640163949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee000b790 con 0x7feef006ef50 2023-10-06T05:24:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.493+0000 7feef619b700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef006ef50 0x7feef0154120 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7feeec009f70 tx=0x7feeec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.494+0000 7feee67fc700 1 -- 172.21.15.31:0/3640163949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec01b760 con 0x7feef006ef50 2023-10-06T05:24:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.494+0000 7feee67fc700 1 -- 172.21.15.31:0/3640163949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec01b8c0 con 0x7feef006ef50 2023-10-06T05:24:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.494+0000 7feef83ff700 1 -- 172.21.15.31:0/3640163949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec0093c0 con 0x7feef006ef50 2023-10-06T05:24:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.494+0000 7feee67fc700 1 -- 172.21.15.31:0/3640163949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec01bc60 con 0x7feef006ef50 2023-10-06T05:24:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.494+0000 7feef83ff700 1 -- 172.21.15.31:0/3640163949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef00c1410 con 0x7feef006ef50 2023-10-06T05:24:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.495+0000 7feef83ff700 1 -- 172.21.15.31:0/3640163949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef004c380 con 0x7feef006ef50 2023-10-06T05:24:34.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.496+0000 7feee67fc700 1 -- 172.21.15.31:0/3640163949 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7feeec011630 con 0x7feef006ef50 2023-10-06T05:24:34.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.496+0000 7feee67fc700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feed40674d0 0x7feed4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.497+0000 7feee67fc700 1 -- 172.21.15.31:0/3640163949 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6086+0+0 (secure 0 0 0) 0x7feeec086e10 con 0x7feef006ef50 2023-10-06T05:24:34.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.497+0000 7feef599a700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feed40674d0 0x7feed4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.497+0000 7feef599a700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feed40674d0 0x7feed4069980 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7feee000b240 tx=0x7feee0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.501+0000 7feee67fc700 1 -- 172.21.15.31:0/3640163949 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feeec053690 con 0x7feef006ef50 2023-10-06T05:24:34.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.748+0000 7feef83ff700 1 -- 172.21.15.31:0/3640163949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feef006bcf0 con 0x7feef006ef50 2023-10-06T05:24:34.750 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:24:34.749+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:24:34.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.753+0000 7feee67fc700 1 -- 172.21.15.31:0/3640163949 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7feeec056cb0 con 0x7feef006ef50 2023-10-06T05:24:34.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 -- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feed40674d0 msgr2=0x7feed4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feed40674d0 0x7feed4069980 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7feee000b240 tx=0x7feee0008040 comp rx=0 tx=0).stop 2023-10-06T05:24:34.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 -- 172.21.15.31:0/3640163949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef006ef50 msgr2=0x7feef0154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef006ef50 0x7feef0154120 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7feeec009f70 tx=0x7feeec016000 comp rx=0 tx=0).stop 2023-10-06T05:24:34.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 -- 172.21.15.31:0/3640163949 shutdown_connections 2023-10-06T05:24:34.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feed40674d0 0x7feed4069980 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef00c7f30 0x7feef0155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef006ef50 0x7feef0154120 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 --2- 172.21.15.31:0/3640163949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef00c7b60 0x7feef00aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 -- 172.21.15.31:0/3640163949 >> 172.21.15.31:0/3640163949 conn(0x7feef00cc0b0 msgr2=0x7feef00cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:34.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 -- 172.21.15.31:0/3640163949 shutdown_connections 2023-10-06T05:24:34.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.756+0000 7feef83ff700 1 -- 172.21.15.31:0/3640163949 wait complete. 2023-10-06T05:24:34.760 INFO:tasks.ceph.mds.b.smithi062.stderr: -1268> 2023-10-06T05:24:10.669+0000 7fb673ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:24:34.760 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:24:34.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:24:34.801 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:24:34.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.973+0000 7faabcfff700 1 Processor -- start 2023-10-06T05:24:34.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.973+0000 7faabcfff700 1 -- start start 2023-10-06T05:24:34.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.974+0000 7faabcfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faab8159930 0x7faab80b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.974+0000 7faabcfff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faab80cd580 0x7faab80b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.974+0000 7faabcfff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab815d2d0 con 0x7faab8159560 2023-10-06T05:24:34.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.974+0000 7faabcfff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab815d5b0 con 0x7faab8159930 2023-10-06T05:24:34.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.974+0000 7faabcfff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab815d890 con 0x7faab80cd580 2023-10-06T05:24:34.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.974+0000 7faab5d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faab8159930 0x7faab80b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.974+0000 7faab659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faab8159560 0x7faab80cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33820/0 (socket says 172.21.15.31:33820) 2023-10-06T05:24:34.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.974+0000 7faab659c700 1 -- 172.21.15.31:0/1881566475 learned_addr learned my addr 172.21.15.31:0/1881566475 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:34.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.974+0000 7faab6d9d700 1 --2- 172.21.15.31:0/1881566475 >> v2:172.21.15.62:3300/0 conn(0x7faab80cd580 0x7faab80b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.975+0000 7faab5d9b700 1 -- 172.21.15.31:0/1881566475 >> v2:172.21.15.62:3300/0 conn(0x7faab80cd580 msgr2=0x7faab80b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:34.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.975+0000 7faab5d9b700 1 --2- 172.21.15.31:0/1881566475 >> v2:172.21.15.62:3300/0 conn(0x7faab80cd580 0x7faab80b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.975+0000 7faab5d9b700 1 -- 172.21.15.31:0/1881566475 >> v1:172.21.15.31:6789/0 conn(0x7faab8159560 legacy=0x7faab80cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.975+0000 7faab5d9b700 1 -- 172.21.15.31:0/1881566475 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab815db70 con 0x7faab8159930 2023-10-06T05:24:34.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.975+0000 7faab5d9b700 1 --2- 172.21.15.31:0/1881566475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab8159930 0x7faab80b2470 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7faab80b1740 tx=0x7faaac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165ef3a70e213bdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.975+0000 7faab559a700 1 -- 172.21.15.31:0/1881566475 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac011630 con 0x7faab8159930 2023-10-06T05:24:34.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.975+0000 7faab559a700 1 -- 172.21.15.31:0/1881566475 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaac011790 con 0x7faab8159930 2023-10-06T05:24:34.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.976+0000 7faab559a700 1 -- 172.21.15.31:0/1881566475 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac011b30 con 0x7faab8159930 2023-10-06T05:24:34.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.976+0000 7faabcfff700 1 -- 172.21.15.31:0/1881566475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab8159930 msgr2=0x7faab80b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.976+0000 7faabcfff700 1 --2- 172.21.15.31:0/1881566475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab8159930 0x7faab80b2470 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7faab80b1740 tx=0x7faaac0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.976+0000 7faabcfff700 1 -- 172.21.15.31:0/1881566475 shutdown_connections 2023-10-06T05:24:34.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.976+0000 7faabcfff700 1 --2- 172.21.15.31:0/1881566475 >> v2:172.21.15.62:3300/0 conn(0x7faab80cd580 0x7faab80b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.976+0000 7faabcfff700 1 --2- 172.21.15.31:0/1881566475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab8159930 0x7faab80b2470 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.976+0000 7faabcfff700 1 -- 172.21.15.31:0/1881566475 >> 172.21.15.31:0/1881566475 conn(0x7faab806ec90 msgr2=0x7faab806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:34.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.976+0000 7faabcfff700 1 -- 172.21.15.31:0/1881566475 shutdown_connections 2023-10-06T05:24:34.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.976+0000 7faabcfff700 1 -- 172.21.15.31:0/1881566475 wait complete. 2023-10-06T05:24:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.978+0000 7faabcfff700 1 Processor -- start 2023-10-06T05:24:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.978+0000 7faabcfff700 1 -- start start 2023-10-06T05:24:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faabcfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab80cd580 0x7faab814d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faabcfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab8159560 0x7faab80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faabcfff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faab8159930 0x7faab80c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faabcfff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab8169990 con 0x7faab80cd580 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faabcfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab816d440 con 0x7faab8159560 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faabcfff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab816d720 con 0x7faab8159930 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab80cd580 0x7faab814d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab80cd580 0x7faab814d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59576/0 (socket says 172.21.15.31:59576) 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab659c700 1 -- 172.21.15.31:0/2710316328 learned_addr learned my addr 172.21.15.31:0/2710316328 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab5d9b700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab8159560 0x7faab80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab6d9d700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faab8159930 0x7faab80c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab659c700 1 -- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faab8159930 msgr2=0x7faab80c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab659c700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faab8159930 0x7faab80c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab659c700 1 -- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab8159560 msgr2=0x7faab80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:34.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab659c700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab8159560 0x7faab80cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab659c700 1 -- 172.21.15.31:0/2710316328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac009420 con 0x7faab80cd580 2023-10-06T05:24:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab6d9d700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faab8159930 0x7faab80c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.979+0000 7faab5d9b700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab8159560 0x7faab80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.980+0000 7faab659c700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab80cd580 0x7faab814d900 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7faaa800c7c0 tx=0x7faaa8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.980+0000 7faaa67fc700 1 -- 172.21.15.31:0/2710316328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8005b30 con 0x7faab80cd580 2023-10-06T05:24:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.980+0000 7faaa67fc700 1 -- 172.21.15.31:0/2710316328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa8005cb0 con 0x7faab80cd580 2023-10-06T05:24:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.980+0000 7faaa67fc700 1 -- 172.21.15.31:0/2710316328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8004a80 con 0x7faab80cd580 2023-10-06T05:24:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.980+0000 7faabcfff700 1 -- 172.21.15.31:0/2710316328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab816da00 con 0x7faab80cd580 2023-10-06T05:24:34.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.980+0000 7faabcfff700 1 -- 172.21.15.31:0/2710316328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab814de40 con 0x7faab80cd580 2023-10-06T05:24:34.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.981+0000 7faabcfff700 1 -- 172.21.15.31:0/2710316328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab8169b00 con 0x7faab80cd580 2023-10-06T05:24:34.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.982+0000 7faaa67fc700 1 -- 172.21.15.31:0/2710316328 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7faaa80070f0 con 0x7faab80cd580 2023-10-06T05:24:34.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.983+0000 7faaa67fc700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faa980674d0 0x7faa98069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:34.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.983+0000 7faaa67fc700 1 -- 172.21.15.31:0/2710316328 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6172+0+0 (secure 0 0 0) 0x7faaa80845a0 con 0x7faab80cd580 2023-10-06T05:24:34.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.983+0000 7faab5d9b700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faa980674d0 0x7faa98069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.984+0000 7faab5d9b700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faa980674d0 0x7faa98069980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faaac0093f0 tx=0x7faaac017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:34.987+0000 7faaa67fc700 1 -- 172.21.15.31:0/2710316328 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faaa8050e10 con 0x7faab80cd580 2023-10-06T05:24:35.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.235+0000 7faabcfff700 1 -- 172.21.15.31:0/2710316328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faab804c380 con 0x7faab80cd580 2023-10-06T05:24:35.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.754+0000 7faaa67fc700 1 -- 172.21.15.31:0/2710316328 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7faaa8054430 con 0x7faab80cd580 2023-10-06T05:24:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.756+0000 7faabcfff700 1 -- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faa980674d0 msgr2=0x7faa98069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.756+0000 7faabcfff700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faa980674d0 0x7faa98069980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faaac0093f0 tx=0x7faaac017000 comp rx=0 tx=0).stop 2023-10-06T05:24:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.757+0000 7faabcfff700 1 -- 172.21.15.31:0/2710316328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab80cd580 msgr2=0x7faab814d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.757+0000 7faabcfff700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab80cd580 0x7faab814d900 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7faaa800c7c0 tx=0x7faaa8007f70 comp rx=0 tx=0).stop 2023-10-06T05:24:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.757+0000 7faabcfff700 1 -- 172.21.15.31:0/2710316328 shutdown_connections 2023-10-06T05:24:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.757+0000 7faabcfff700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faa980674d0 0x7faa98069980 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.757+0000 7faabcfff700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faab8159930 0x7faab80c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.757+0000 7faabcfff700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faab80cd580 0x7faab814d900 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.757+0000 7faabcfff700 1 --2- 172.21.15.31:0/2710316328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faab8159560 0x7faab80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.757+0000 7faabcfff700 1 -- 172.21.15.31:0/2710316328 >> 172.21.15.31:0/2710316328 conn(0x7faab806ec90 msgr2=0x7faab8158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:35.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.757+0000 7faabcfff700 1 -- 172.21.15.31:0/2710316328 shutdown_connections 2023-10-06T05:24:35.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.757+0000 7faabcfff700 1 -- 172.21.15.31:0/2710316328 wait complete. 2023-10-06T05:24:35.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:24:35.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.974+0000 7fcdc89ff700 1 Processor -- start 2023-10-06T05:24:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.974+0000 7fcdc89ff700 1 -- start start 2023-10-06T05:24:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.976+0000 7fcdc89ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcdc4153d20 0x7fcdc415d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.976+0000 7fcdc89ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc415d750 0x7fcdc415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.976+0000 7fcdc89ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc415dbb0 con 0x7fcdc4153950 2023-10-06T05:24:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.976+0000 7fcdc89ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc4154170 con 0x7fcdc4153d20 2023-10-06T05:24:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.976+0000 7fcdc89ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc415bb20 con 0x7fcdc415d750 2023-10-06T05:24:35.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.976+0000 7fcdc1d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcdc4153d20 0x7fcdc415d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.976+0000 7fcdc2d9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc415d750 0x7fcdc415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.976+0000 7fcdc1d9b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcdc4153d20 0x7fcdc415d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:41310/0 (socket says 172.21.15.31:41310) 2023-10-06T05:24:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.976+0000 7fcdc259c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fcdc4153950 0x7fcdc40cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:34906/0 (socket says 172.21.15.31:34906) 2023-10-06T05:24:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.976+0000 7fcdc1d9b700 1 -- 172.21.15.31:0/2231107691 learned_addr learned my addr 172.21.15.31:0/2231107691 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:35.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.977+0000 7fcdc2d9d700 1 -- 172.21.15.31:0/2231107691 >> v2:172.21.15.62:3300/0 conn(0x7fcdc4153d20 msgr2=0x7fcdc415d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.977+0000 7fcdc2d9d700 1 --2- 172.21.15.31:0/2231107691 >> v2:172.21.15.62:3300/0 conn(0x7fcdc4153d20 0x7fcdc415d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.977+0000 7fcdc2d9d700 1 -- 172.21.15.31:0/2231107691 >> v1:172.21.15.62:6789/0 conn(0x7fcdc4153950 legacy=0x7fcdc40cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.977+0000 7fcdc2d9d700 1 -- 172.21.15.31:0/2231107691 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc415af10 con 0x7fcdc415d750 2023-10-06T05:24:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.977+0000 7fcdc1d9b700 1 --2- 172.21.15.31:0/2231107691 >> v2:172.21.15.62:3300/0 conn(0x7fcdc4153d20 0x7fcdc415d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:35.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.977+0000 7fcdc2d9d700 1 --2- 172.21.15.31:0/2231107691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc415d750 0x7fcdc415bfb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009e60 tx=0x7fcdbc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63cbdea3abddb45f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.977+0000 7fcdc159a700 1 -- 172.21.15.31:0/2231107691 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc00e070 con 0x7fcdc415d750 2023-10-06T05:24:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.978+0000 7fcdc159a700 1 -- 172.21.15.31:0/2231107691 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc00bb60 con 0x7fcdc415d750 2023-10-06T05:24:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.978+0000 7fcdc159a700 1 -- 172.21.15.31:0/2231107691 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc012430 con 0x7fcdc415d750 2023-10-06T05:24:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.978+0000 7fcdc89ff700 1 -- 172.21.15.31:0/2231107691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc415d750 msgr2=0x7fcdc415bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.978+0000 7fcdc89ff700 1 --2- 172.21.15.31:0/2231107691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc415d750 0x7fcdc415bfb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009e60 tx=0x7fcdbc009780 comp rx=0 tx=0).stop 2023-10-06T05:24:35.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.978+0000 7fcdc89ff700 1 -- 172.21.15.31:0/2231107691 shutdown_connections 2023-10-06T05:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.978+0000 7fcdc89ff700 1 --2- 172.21.15.31:0/2231107691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc415d750 0x7fcdc415bfb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.978+0000 7fcdc89ff700 1 --2- 172.21.15.31:0/2231107691 >> v2:172.21.15.62:3300/0 conn(0x7fcdc4153d20 0x7fcdc415d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.978+0000 7fcdc89ff700 1 -- 172.21.15.31:0/2231107691 >> 172.21.15.31:0/2231107691 conn(0x7fcdc40681b0 msgr2=0x7fcdc40685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.978+0000 7fcdc89ff700 1 -- 172.21.15.31:0/2231107691 shutdown_connections 2023-10-06T05:24:35.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.978+0000 7fcdc89ff700 1 -- 172.21.15.31:0/2231107691 wait complete. 2023-10-06T05:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.980+0000 7fcdc89ff700 1 Processor -- start 2023-10-06T05:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.981+0000 7fcdc89ff700 1 -- start start 2023-10-06T05:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc89ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdc4153950 0x7fcdc40c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc89ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc4153d20 0x7fcdc40c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc89ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc415d750 0x7fcdc40c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc89ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40c81a0 con 0x7fcdc4153950 2023-10-06T05:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc89ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40b5570 con 0x7fcdc415d750 2023-10-06T05:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc89ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40b5850 con 0x7fcdc4153d20 2023-10-06T05:24:35.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdc4153950 0x7fcdc40c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdc4153950 0x7fcdc40c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59582/0 (socket says 172.21.15.31:59582) 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc259c700 1 -- 172.21.15.31:0/282173189 learned_addr learned my addr 172.21.15.31:0/282173189 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc2d9d700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc415d750 0x7fcdc40c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.982+0000 7fcdc1d9b700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc4153d20 0x7fcdc40c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdc259c700 1 -- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc4153d20 msgr2=0x7fcdc40c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdc259c700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc4153d20 0x7fcdc40c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdc259c700 1 -- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc415d750 msgr2=0x7fcdc40c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdc259c700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc415d750 0x7fcdc40c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdc259c700 1 -- 172.21.15.31:0/282173189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb80094f0 con 0x7fcdc4153950 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdc259c700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdc4153950 0x7fcdc40c8a40 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400c590 tx=0x7fcdb4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdb27fc700 1 -- 172.21.15.31:0/282173189 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4005ac0 con 0x7fcdc4153950 2023-10-06T05:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdb27fc700 1 -- 172.21.15.31:0/282173189 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb4005c20 con 0x7fcdc4153950 2023-10-06T05:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdb27fc700 1 -- 172.21.15.31:0/282173189 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb40070d0 con 0x7fcdc4153950 2023-10-06T05:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdc89ff700 1 -- 172.21.15.31:0/282173189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc009a20 con 0x7fcdc4153950 2023-10-06T05:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.983+0000 7fcdc89ff700 1 -- 172.21.15.31:0/282173189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc40b5e10 con 0x7fcdc4153950 2023-10-06T05:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.984+0000 7fcdc89ff700 1 -- 172.21.15.31:0/282173189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc404c380 con 0x7fcdc4153950 2023-10-06T05:24:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.985+0000 7fcdb27fc700 1 -- 172.21.15.31:0/282173189 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fcdb4007440 con 0x7fcdc4153950 2023-10-06T05:24:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.986+0000 7fcdb27fc700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcda40674d0 0x7fcda4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.986+0000 7fcdb27fc700 1 -- 172.21.15.31:0/282173189 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fcdb40843d0 con 0x7fcdc4153950 2023-10-06T05:24:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.987+0000 7fcdc1d9b700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcda40674d0 0x7fcda4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.987+0000 7fcdc1d9b700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcda40674d0 0x7fcda4069980 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40b4640 tx=0x7fcdb8009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:35.990+0000 7fcdb27fc700 1 -- 172.21.15.31:0/282173189 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdb4050bc0 con 0x7fcdc4153950 2023-10-06T05:24:36.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.197+0000 7fcdc89ff700 1 -- 172.21.15.31:0/282173189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcdc415dbb0 con 0x7fcdc4153950 2023-10-06T05:24:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.237+0000 7fcdb27fc700 1 -- 172.21.15.31:0/282173189 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcdc415dbb0 con 0x7fcdc4153950 2023-10-06T05:24:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.241+0000 7fcdc89ff700 1 -- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcda40674d0 msgr2=0x7fcda4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.241+0000 7fcdc89ff700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcda40674d0 0x7fcda4069980 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40b4640 tx=0x7fcdb8009250 comp rx=0 tx=0).stop 2023-10-06T05:24:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.241+0000 7fcdc89ff700 1 -- 172.21.15.31:0/282173189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdc4153950 msgr2=0x7fcdc40c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.241+0000 7fcdc89ff700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdc4153950 0x7fcdc40c8a40 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400c590 tx=0x7fcdb4001070 comp rx=0 tx=0).stop 2023-10-06T05:24:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.241+0000 7fcdc89ff700 1 -- 172.21.15.31:0/282173189 shutdown_connections 2023-10-06T05:24:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.241+0000 7fcdc89ff700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcda40674d0 0x7fcda4069980 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.241+0000 7fcdc89ff700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdc415d750 0x7fcdc40c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.241+0000 7fcdc89ff700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdc4153950 0x7fcdc40c8a40 secure :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400c590 tx=0x7fcdb4001070 comp rx=0 tx=0).stop 2023-10-06T05:24:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.241+0000 7fcdc89ff700 1 --2- 172.21.15.31:0/282173189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcdc4153d20 0x7fcdc40c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.242+0000 7fcdc89ff700 1 -- 172.21.15.31:0/282173189 >> 172.21.15.31:0/282173189 conn(0x7fcdc40681b0 msgr2=0x7fcdc40cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.242+0000 7fcdc89ff700 1 -- 172.21.15.31:0/282173189 shutdown_connections 2023-10-06T05:24:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.242+0000 7fcdc89ff700 1 -- 172.21.15.31:0/282173189 wait complete. 2023-10-06T05:24:36.246 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:24:36.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:24:36.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.463+0000 7fabb73ff700 1 Processor -- start 2023-10-06T05:24:36.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.463+0000 7fabb73ff700 1 -- start start 2023-10-06T05:24:36.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.464+0000 7fabb73ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb015d920 0x7fabb01645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.465+0000 7fabb73ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb015cfc0 con 0x7fabb0153950 2023-10-06T05:24:36.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.465+0000 7fabb73ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb015bb20 con 0x7fabb0153d20 2023-10-06T05:24:36.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.465+0000 7fabb73ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb0154170 con 0x7fabb015d920 2023-10-06T05:24:36.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.465+0000 7fabb519b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fabb0153950 0x7fabb00cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33826/0 (socket says 172.21.15.31:33826) 2023-10-06T05:24:36.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.465+0000 7fabb519b700 1 -- 172.21.15.31:0/4006622111 learned_addr learned my addr 172.21.15.31:0/4006622111 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:36.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.465+0000 7fabb599c700 1 --2- 172.21.15.31:0/4006622111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb015d920 0x7fabb01645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.465+0000 7faba7fff700 1 -- 172.21.15.31:0/4006622111 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617151058 0 0) 0x7fabb015cfc0 con 0x7fabb0153950 2023-10-06T05:24:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.465+0000 7faba7fff700 1 -- 172.21.15.31:0/4006622111 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab9c003570 con 0x7fabb0153950 2023-10-06T05:24:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.465+0000 7faba7fff700 1 -- 172.21.15.31:0/4006622111 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842056520 0 0) 0x7fabb015bb20 con 0x7fabb0153d20 2023-10-06T05:24:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.465+0000 7faba7fff700 1 -- 172.21.15.31:0/4006622111 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb015cfc0 con 0x7fabb0153d20 2023-10-06T05:24:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.466+0000 7fabb599c700 1 -- 172.21.15.31:0/4006622111 >> v1:172.21.15.62:6789/0 conn(0x7fabb0153d20 legacy=0x7fabb015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.466+0000 7fabb599c700 1 -- 172.21.15.31:0/4006622111 >> v1:172.21.15.31:6789/0 conn(0x7fabb0153950 legacy=0x7fabb00cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.466+0000 7faba7fff700 1 -- 172.21.15.31:0/4006622111 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 175230670 0 0) 0x7fab9c003570 con 0x7fabb0153950 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.466+0000 7fabb599c700 1 -- 172.21.15.31:0/4006622111 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb015af10 con 0x7fabb015d920 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.466+0000 7fabb599c700 1 --2- 172.21.15.31:0/4006622111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb015d920 0x7fabb01645b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faba8009e90 tx=0x7faba8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346e1e418150fc83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.466+0000 7faba7fff700 1 -- 172.21.15.31:0/4006622111 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba800e070 con 0x7fabb015d920 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.466+0000 7faba7fff700 1 -- 172.21.15.31:0/4006622111 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba800b870 con 0x7fabb015d920 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.466+0000 7faba7fff700 1 -- 172.21.15.31:0/4006622111 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac0015f0 con 0x7fabb015d920 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.467+0000 7fabb73ff700 1 -- 172.21.15.31:0/4006622111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb015d920 msgr2=0x7fabb01645b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.467+0000 7fabb73ff700 1 --2- 172.21.15.31:0/4006622111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb015d920 0x7fabb01645b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faba8009e90 tx=0x7faba8009780 comp rx=0 tx=0).stop 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.467+0000 7fabb73ff700 1 -- 172.21.15.31:0/4006622111 shutdown_connections 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.467+0000 7fabb73ff700 1 --2- 172.21.15.31:0/4006622111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb015d920 0x7fabb01645b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.467+0000 7fabb73ff700 1 -- 172.21.15.31:0/4006622111 >> 172.21.15.31:0/4006622111 conn(0x7fabb00681b0 msgr2=0x7fabb00685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.467+0000 7fabb73ff700 1 -- 172.21.15.31:0/4006622111 shutdown_connections 2023-10-06T05:24:36.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.467+0000 7fabb73ff700 1 -- 172.21.15.31:0/4006622111 wait complete. 2023-10-06T05:24:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.468+0000 7fabb73ff700 1 Processor -- start 2023-10-06T05:24:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.469+0000 7fabb73ff700 1 -- start start 2023-10-06T05:24:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb73ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb0153950 0x7fabb016c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb73ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb0153d20 0x7fabb00b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb73ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb015d920 0x7fabb00cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb73ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00c8dd0 con 0x7fabb0153950 2023-10-06T05:24:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb73ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00c81c0 con 0x7fabb015d920 2023-10-06T05:24:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb73ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00c84a0 con 0x7fabb0153d20 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb0153950 0x7fabb016c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb0153950 0x7fabb016c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59596/0 (socket says 172.21.15.31:59596) 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb519b700 1 -- 172.21.15.31:0/3879776960 learned_addr learned my addr 172.21.15.31:0/3879776960 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb499a700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb0153d20 0x7fabb00b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb599c700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb015d920 0x7fabb00cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb519b700 1 -- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb0153d20 msgr2=0x7fabb00b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb519b700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb0153d20 0x7fabb00b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb519b700 1 -- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb015d920 msgr2=0x7fabb00cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb519b700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb015d920 0x7fabb00cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb519b700 1 -- 172.21.15.31:0/3879776960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8009a20 con 0x7fabb0153950 2023-10-06T05:24:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb499a700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb0153d20 0x7fabb00b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb599c700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb015d920 0x7fabb00cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.470+0000 7fabb519b700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb0153950 0x7fabb016c730 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fabac00cb90 tx=0x7fabac002b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.471+0000 7faba57fa700 1 -- 172.21.15.31:0/3879776960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac0070f0 con 0x7fabb0153950 2023-10-06T05:24:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.471+0000 7faba57fa700 1 -- 172.21.15.31:0/3879776960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac007270 con 0x7fabb0153950 2023-10-06T05:24:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.471+0000 7fabb73ff700 1 -- 172.21.15.31:0/3879776960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb00c8780 con 0x7fabb0153950 2023-10-06T05:24:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.471+0000 7faba57fa700 1 -- 172.21.15.31:0/3879776960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac007630 con 0x7fabb0153950 2023-10-06T05:24:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.471+0000 7fabb73ff700 1 -- 172.21.15.31:0/3879776960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb00b4190 con 0x7fabb0153950 2023-10-06T05:24:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.472+0000 7fabb73ff700 1 -- 172.21.15.31:0/3879776960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb00c8f80 con 0x7fabb0153950 2023-10-06T05:24:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.473+0000 7faba57fa700 1 -- 172.21.15.31:0/3879776960 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fabac005a00 con 0x7fabb0153950 2023-10-06T05:24:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.474+0000 7faba57fa700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab940674d0 0x7fab94069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.474+0000 7faba57fa700 1 -- 172.21.15.31:0/3879776960 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fabac084d10 con 0x7fabb0153950 2023-10-06T05:24:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.474+0000 7fabb499a700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab940674d0 0x7fab94069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:36.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.475+0000 7fabb499a700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab940674d0 0x7fab94069980 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fabb00c79e0 tx=0x7faba0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:36.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.478+0000 7faba57fa700 1 -- 172.21.15.31:0/3879776960 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabac051700 con 0x7fabb0153950 2023-10-06T05:24:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:36.685+0000 7fabb73ff700 1 -- 172.21.15.31:0/3879776960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fabb004c380 con 0x7fabb0153950 2023-10-06T05:24:37.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.240+0000 7faba57fa700 1 -- 172.21.15.31:0/3879776960 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7fabac005cb0 con 0x7fabb0153950 2023-10-06T05:24:37.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.245+0000 7fabb73ff700 1 -- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab940674d0 msgr2=0x7fab94069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.245+0000 7fabb73ff700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab940674d0 0x7fab94069980 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fabb00c79e0 tx=0x7faba0009040 comp rx=0 tx=0).stop 2023-10-06T05:24:37.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.245+0000 7fabb73ff700 1 -- 172.21.15.31:0/3879776960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb0153950 msgr2=0x7fabb016c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.245+0000 7fabb73ff700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb0153950 0x7fabb016c730 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fabac00cb90 tx=0x7fabac002b10 comp rx=0 tx=0).stop 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.248+0000 7fabb73ff700 1 -- 172.21.15.31:0/3879776960 shutdown_connections 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.248+0000 7fabb73ff700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab940674d0 0x7fab94069980 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.248+0000 7fabb73ff700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb015d920 0x7fabb00cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.248+0000 7fabb73ff700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb0153950 0x7fabb016c730 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.248+0000 7fabb73ff700 1 --2- 172.21.15.31:0/3879776960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabb0153d20 0x7fabb00b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.248+0000 7fabb73ff700 1 -- 172.21.15.31:0/3879776960 >> 172.21.15.31:0/3879776960 conn(0x7fabb00681b0 msgr2=0x7fabb00cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:37.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.250+0000 7fabb73ff700 1 -- 172.21.15.31:0/3879776960 shutdown_connections 2023-10-06T05:24:37.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.250+0000 7fabb73ff700 1 -- 172.21.15.31:0/3879776960 wait complete. 2023-10-06T05:24:37.256 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:24:37.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:24:37.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.474+0000 7f67cd7ff700 1 Processor -- start 2023-10-06T05:24:37.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.475+0000 7f67cd7ff700 1 -- start start 2023-10-06T05:24:37.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.476+0000 7f67cd7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f67c80af890 0x7f67c8161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.476+0000 7f67cd7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c80afc90 con 0x7f67c8153d40 2023-10-06T05:24:37.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.476+0000 7f67cd7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c80ad690 con 0x7f67c80b0b90 2023-10-06T05:24:37.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.476+0000 7f67cd7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c80acd60 con 0x7f67c80af890 2023-10-06T05:24:37.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.476+0000 7f67c77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f67c80af890 0x7f67c8161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.476+0000 7f67c77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f67c80af890 0x7f67c8161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53442/0 (socket says 172.21.15.31:53442) 2023-10-06T05:24:37.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.476+0000 7f67c77fe700 1 -- 172.21.15.31:0/2049786165 learned_addr learned my addr 172.21.15.31:0/2049786165 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:37.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.476+0000 7f67c6ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f67c8153d40 0x7f67c80b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60048/0 (socket says 172.21.15.31:60048) 2023-10-06T05:24:37.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.477+0000 7f67c77fe700 1 -- 172.21.15.31:0/2049786165 >> v1:172.21.15.62:6789/0 conn(0x7f67c80b0b90 legacy=0x7f67c80af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.477+0000 7f67c77fe700 1 -- 172.21.15.31:0/2049786165 >> v1:172.21.15.31:6789/0 conn(0x7f67c8153d40 legacy=0x7f67c80b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.477+0000 7f67c77fe700 1 -- 172.21.15.31:0/2049786165 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c80ad040 con 0x7f67c80af890 2023-10-06T05:24:37.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.477+0000 7f67c5ffb700 1 -- 172.21.15.31:0/2049786165 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745642690 0 0) 0x7f67b8001110 con 0x7f67c8153d40 2023-10-06T05:24:37.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.477+0000 7f67c77fe700 1 --2- 172.21.15.31:0/2049786165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c80af890 0x7f67c8161240 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009f20 tx=0x7f67c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a5085372f873c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.477+0000 7f67c5ffb700 1 -- 172.21.15.31:0/2049786165 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c000e070 con 0x7f67c80af890 2023-10-06T05:24:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.477+0000 7f67c5ffb700 1 -- 172.21.15.31:0/2049786165 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c000ba80 con 0x7f67c80af890 2023-10-06T05:24:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.477+0000 7f67c5ffb700 1 -- 172.21.15.31:0/2049786165 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c00092e0 con 0x7f67c80af890 2023-10-06T05:24:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.478+0000 7f67cd7ff700 1 -- 172.21.15.31:0/2049786165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c80af890 msgr2=0x7f67c8161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.478+0000 7f67cd7ff700 1 --2- 172.21.15.31:0/2049786165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c80af890 0x7f67c8161240 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009f20 tx=0x7f67c0009840 comp rx=0 tx=0).stop 2023-10-06T05:24:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.478+0000 7f67cd7ff700 1 -- 172.21.15.31:0/2049786165 shutdown_connections 2023-10-06T05:24:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.478+0000 7f67cd7ff700 1 --2- 172.21.15.31:0/2049786165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c80af890 0x7f67c8161240 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.478+0000 7f67cd7ff700 1 -- 172.21.15.31:0/2049786165 >> 172.21.15.31:0/2049786165 conn(0x7f67c8067560 msgr2=0x7f67c8067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.478+0000 7f67cd7ff700 1 -- 172.21.15.31:0/2049786165 shutdown_connections 2023-10-06T05:24:37.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.478+0000 7f67cd7ff700 1 -- 172.21.15.31:0/2049786165 wait complete. 2023-10-06T05:24:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.479+0000 7f67cd7ff700 1 Processor -- start 2023-10-06T05:24:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.479+0000 7f67cd7ff700 1 -- start start 2023-10-06T05:24:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.480+0000 7f67cd7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80af890 0x7f67c81668e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.480+0000 7f67cd7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c80b0b90 0x7f67c8169c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.480+0000 7f67cd7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c8153d40 0x7f67c80c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.480+0000 7f67cd7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80c6020 con 0x7f67c8153d40 2023-10-06T05:24:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.480+0000 7f67cd7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c816a190 con 0x7f67c80af890 2023-10-06T05:24:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.480+0000 7f67cd7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80c5410 con 0x7f67c80b0b90 2023-10-06T05:24:37.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c8153d40 0x7f67c80c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c8153d40 0x7f67c80c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53448/0 (socket says 172.21.15.31:53448) 2023-10-06T05:24:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c77fe700 1 -- 172.21.15.31:0/1467513477 learned_addr learned my addr 172.21.15.31:0/1467513477 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c6ffd700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80af890 0x7f67c81668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c67fc700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c80b0b90 0x7f67c8169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c77fe700 1 -- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c80b0b90 msgr2=0x7f67c8169c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c77fe700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c80b0b90 0x7f67c8169c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c77fe700 1 -- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80af890 msgr2=0x7f67c81668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c77fe700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80af890 0x7f67c81668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c77fe700 1 -- 172.21.15.31:0/1467513477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c0009ae0 con 0x7f67c8153d40 2023-10-06T05:24:37.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.481+0000 7f67c77fe700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c8153d40 0x7f67c80c68b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f67c806f430 tx=0x7f67c0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.482+0000 7f67aeffd700 1 -- 172.21.15.31:0/1467513477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0007e70 con 0x7f67c8153d40 2023-10-06T05:24:37.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.482+0000 7f67aeffd700 1 -- 172.21.15.31:0/1467513477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c0026460 con 0x7f67c8153d40 2023-10-06T05:24:37.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.482+0000 7f67cd7ff700 1 -- 172.21.15.31:0/1467513477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c80c56f0 con 0x7f67c8153d40 2023-10-06T05:24:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.482+0000 7f67aeffd700 1 -- 172.21.15.31:0/1467513477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0026840 con 0x7f67c8153d40 2023-10-06T05:24:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.482+0000 7f67cd7ff700 1 -- 172.21.15.31:0/1467513477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c80c5cb0 con 0x7f67c8153d40 2023-10-06T05:24:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.483+0000 7f67cd7ff700 1 -- 172.21.15.31:0/1467513477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c804c380 con 0x7f67c8153d40 2023-10-06T05:24:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.484+0000 7f67aeffd700 1 -- 172.21.15.31:0/1467513477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f67c0003020 con 0x7f67c8153d40 2023-10-06T05:24:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.485+0000 7f67aeffd700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67b4067500 0x7f67b40699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.485+0000 7f67aeffd700 1 -- 172.21.15.31:0/1467513477 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f67c0086c60 con 0x7f67c8153d40 2023-10-06T05:24:37.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.485+0000 7f67c6ffd700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67b4067500 0x7f67b40699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.486+0000 7f67c6ffd700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67b4067500 0x7f67b40699b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f67b8007fd0 tx=0x7f67b8007db0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.489+0000 7f67aeffd700 1 -- 172.21.15.31:0/1467513477 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67c00536d0 con 0x7f67c8153d40 2023-10-06T05:24:37.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.689+0000 7f67cd7ff700 1 -- 172.21.15.31:0/1467513477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f67c80ad320 con 0x7f67c8153d40 2023-10-06T05:24:37.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.693+0000 7f67aeffd700 1 -- 172.21.15.31:0/1467513477 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f67c0056cf0 con 0x7f67c8153d40 2023-10-06T05:24:37.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 -- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67b4067500 msgr2=0x7f67b40699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67b4067500 0x7f67b40699b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f67b8007fd0 tx=0x7f67b8007db0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 -- 172.21.15.31:0/1467513477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c8153d40 msgr2=0x7f67c80c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c8153d40 0x7f67c80c68b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f67c806f430 tx=0x7f67c0018000 comp rx=0 tx=0).stop 2023-10-06T05:24:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 -- 172.21.15.31:0/1467513477 shutdown_connections 2023-10-06T05:24:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f67b4067500 0x7f67b40699b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c8153d40 0x7f67c80c68b0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80af890 0x7f67c81668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 --2- 172.21.15.31:0/1467513477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67c80b0b90 0x7f67c8169c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 -- 172.21.15.31:0/1467513477 >> 172.21.15.31:0/1467513477 conn(0x7f67c8067560 msgr2=0x7f67c80c8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 -- 172.21.15.31:0/1467513477 shutdown_connections 2023-10-06T05:24:37.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.696+0000 7f67cd7ff700 1 -- 172.21.15.31:0/1467513477 wait complete. 2023-10-06T05:24:37.716 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2023-10-06T05:24:37.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.913+0000 7f0e4a3ff700 1 Processor -- start 2023-10-06T05:24:37.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.913+0000 7f0e4a3ff700 1 -- start start 2023-10-06T05:24:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.914+0000 7f0e4a3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0e44159500 0x7f0e440b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.914+0000 7f0e4a3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e440cd510 0x7f0e440aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.914+0000 7f0e4a3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e440c3f40 con 0x7f0e4415d2d0 2023-10-06T05:24:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.914+0000 7f0e4a3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e440b2860 con 0x7f0e44159500 2023-10-06T05:24:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.914+0000 7f0e4a3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4415d7d0 con 0x7f0e440cd510 2023-10-06T05:24:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e437fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0e4415d2d0 0x7f0e440cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60058/0 (socket says 172.21.15.31:60058) 2023-10-06T05:24:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e437fe700 1 -- 172.21.15.31:0/3649907192 learned_addr learned my addr 172.21.15.31:0/3649907192 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e43fff700 1 --2- 172.21.15.31:0/3649907192 >> v2:172.21.15.62:3300/0 conn(0x7f0e44159500 0x7f0e440b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e4899c700 1 --2- 172.21.15.31:0/3649907192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e440cd510 0x7f0e440aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e42ffd700 1 -- 172.21.15.31:0/3649907192 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441653093 0 0) 0x7f0e440c3f40 con 0x7f0e4415d2d0 2023-10-06T05:24:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e42ffd700 1 -- 172.21.15.31:0/3649907192 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e2c003570 con 0x7f0e4415d2d0 2023-10-06T05:24:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e42ffd700 1 -- 172.21.15.31:0/3649907192 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1363001060 0 0) 0x7f0e2c003570 con 0x7f0e4415d2d0 2023-10-06T05:24:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e43fff700 1 -- 172.21.15.31:0/3649907192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e440cd510 msgr2=0x7f0e440aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e43fff700 1 --2- 172.21.15.31:0/3649907192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e440cd510 0x7f0e440aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e43fff700 1 -- 172.21.15.31:0/3649907192 >> v1:172.21.15.31:6789/0 conn(0x7f0e4415d2d0 legacy=0x7f0e440cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.915+0000 7f0e43fff700 1 -- 172.21.15.31:0/3649907192 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e440cb000 con 0x7f0e44159500 2023-10-06T05:24:37.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.916+0000 7f0e43fff700 1 --2- 172.21.15.31:0/3649907192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e44159500 0x7f0e440b0ab0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0e340098e0 tx=0x7f0e340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14efb7a938472aa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.916+0000 7f0e42ffd700 1 -- 172.21.15.31:0/3649907192 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380015f0 con 0x7f0e44159500 2023-10-06T05:24:37.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.916+0000 7f0e42ffd700 1 -- 172.21.15.31:0/3649907192 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e34011420 con 0x7f0e44159500 2023-10-06T05:24:37.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.916+0000 7f0e42ffd700 1 -- 172.21.15.31:0/3649907192 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340117c0 con 0x7f0e44159500 2023-10-06T05:24:37.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.916+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3649907192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e44159500 msgr2=0x7f0e440b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.916+0000 7f0e4a3ff700 1 --2- 172.21.15.31:0/3649907192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e44159500 0x7f0e440b0ab0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0e340098e0 tx=0x7f0e340091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.917+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3649907192 shutdown_connections 2023-10-06T05:24:37.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.917+0000 7f0e4a3ff700 1 --2- 172.21.15.31:0/3649907192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e44159500 0x7f0e440b0ab0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.917+0000 7f0e4a3ff700 1 --2- 172.21.15.31:0/3649907192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e440cd510 0x7f0e440aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.917+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3649907192 >> 172.21.15.31:0/3649907192 conn(0x7f0e4406ec90 msgr2=0x7f0e4406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:37.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.917+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3649907192 shutdown_connections 2023-10-06T05:24:37.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.917+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3649907192 wait complete. 2023-10-06T05:24:37.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.918+0000 7f0e4a3ff700 1 Processor -- start 2023-10-06T05:24:37.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.918+0000 7f0e4a3ff700 1 -- start start 2023-10-06T05:24:37.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.919+0000 7f0e4a3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e440cd510 0x7f0e44150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.919+0000 7f0e4a3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e44159500 0x7f0e4415fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.919+0000 7f0e4a3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4415d2d0 0x7f0e4415ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.919+0000 7f0e4a3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440b5d70 con 0x7f0e440cd510 2023-10-06T05:24:37.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.919+0000 7f0e4a3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440b6200 con 0x7f0e44159500 2023-10-06T05:24:37.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.919+0000 7f0e4a3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440b5160 con 0x7f0e4415d2d0 2023-10-06T05:24:37.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e43fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e440cd510 0x7f0e44150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e43fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e440cd510 0x7f0e44150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53458/0 (socket says 172.21.15.31:53458) 2023-10-06T05:24:37.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e43fff700 1 -- 172.21.15.31:0/3828634041 learned_addr learned my addr 172.21.15.31:0/3828634041 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:37.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e437fe700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e44159500 0x7f0e4415fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e4899c700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4415d2d0 0x7f0e4415ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e43fff700 1 -- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4415d2d0 msgr2=0x7f0e4415ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e43fff700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4415d2d0 0x7f0e4415ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e43fff700 1 -- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e44159500 msgr2=0x7f0e4415fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:37.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e43fff700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e44159500 0x7f0e4415fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:37.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e43fff700 1 -- 172.21.15.31:0/3828634041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e34009420 con 0x7f0e440cd510 2023-10-06T05:24:37.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e4899c700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4415d2d0 0x7f0e4415ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:37.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e437fe700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e44159500 0x7f0e4415fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:37.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.920+0000 7f0e43fff700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e440cd510 0x7f0e44150960 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f0e44067c50 tx=0x7f0e34016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.921+0000 7f0e2bfff700 1 -- 172.21.15.31:0/3828634041 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3401cc50 con 0x7f0e440cd510 2023-10-06T05:24:37.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.921+0000 7f0e2bfff700 1 -- 172.21.15.31:0/3828634041 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3401cdd0 con 0x7f0e440cd510 2023-10-06T05:24:37.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.921+0000 7f0e2bfff700 1 -- 172.21.15.31:0/3828634041 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3401a6a0 con 0x7f0e440cd510 2023-10-06T05:24:37.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.921+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3828634041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e440b5440 con 0x7f0e440cd510 2023-10-06T05:24:37.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.921+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3828634041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e440b5a00 con 0x7f0e440cd510 2023-10-06T05:24:37.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.922+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3828634041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e440b6050 con 0x7f0e440cd510 2023-10-06T05:24:37.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.923+0000 7f0e2bfff700 1 -- 172.21.15.31:0/3828634041 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0e3401aa50 con 0x7f0e440cd510 2023-10-06T05:24:37.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.924+0000 7f0e2bfff700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0e30067500 0x7f0e300699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:37.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.924+0000 7f0e2bfff700 1 -- 172.21.15.31:0/3828634041 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f0e34086800 con 0x7f0e440cd510 2023-10-06T05:24:37.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.924+0000 7f0e437fe700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0e30067500 0x7f0e300699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:37.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.925+0000 7f0e437fe700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0e30067500 0x7f0e300699b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0e4415f9a0 tx=0x7f0e38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:37.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:37.928+0000 7f0e2bfff700 1 -- 172.21.15.31:0/3828634041 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e340533c0 con 0x7f0e440cd510 2023-10-06T05:24:38.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.208+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3828634041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) v1 -- 0x7f0e4404c380 con 0x7f0e440cd510 2023-10-06T05:24:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.260+0000 7f0e2bfff700 1 -- 172.21.15.31:0/3828634041 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v414) v1 ==== 151+0+0 (secure 0 0 0) 0x7f0e34056960 con 0x7f0e440cd510 2023-10-06T05:24:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0e30067500 msgr2=0x7f0e300699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0e30067500 0x7f0e300699b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0e4415f9a0 tx=0x7f0e38009040 comp rx=0 tx=0).stop 2023-10-06T05:24:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3828634041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e440cd510 msgr2=0x7f0e44150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e440cd510 0x7f0e44150960 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f0e44067c50 tx=0x7f0e34016000 comp rx=0 tx=0).stop 2023-10-06T05:24:38.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3828634041 shutdown_connections 2023-10-06T05:24:38.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0e30067500 0x7f0e300699b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e4415d2d0 0x7f0e4415ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e440cd510 0x7f0e44150960 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 --2- 172.21.15.31:0/3828634041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e44159500 0x7f0e4415fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3828634041 >> 172.21.15.31:0/3828634041 conn(0x7f0e4406ec90 msgr2=0x7f0e440cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:38.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3828634041 shutdown_connections 2023-10-06T05:24:38.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.262+0000 7f0e4a3ff700 1 -- 172.21.15.31:0/3828634041 wait complete. 2023-10-06T05:24:38.281 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-10-06T05:24:38.282 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2023-10-06T05:24:38.282 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:24:38.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2023-10-06T05:24:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.476+0000 7f25e33ff700 1 Processor -- start 2023-10-06T05:24:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.476+0000 7f25e33ff700 1 -- start start 2023-10-06T05:24:38.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e33ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25dc068d80 0x7f25dc159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc0cc0d0 0x7f25dc0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e33ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25dc0c95b0 con 0x7f25dc1595a0 2023-10-06T05:24:38.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e33ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25dc0c9890 con 0x7f25dc068d80 2023-10-06T05:24:38.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc0c9b70 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e099a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25dc068d80 0x7f25dc159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e099a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25dc068d80 0x7f25dc159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53472/0 (socket says 172.21.15.31:53472) 2023-10-06T05:24:38.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e099a700 1 -- 172.21.15.31:0/2067769765 learned_addr learned my addr 172.21.15.31:0/2067769765 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e199c700 1 --2- 172.21.15.31:0/2067769765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc0cc0d0 0x7f25dc0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e099a700 1 -- 172.21.15.31:0/2067769765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc0cc0d0 msgr2=0x7f25dc0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:38.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e099a700 1 --2- 172.21.15.31:0/2067769765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc0cc0d0 0x7f25dc0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e099a700 1 -- 172.21.15.31:0/2067769765 >> v1:172.21.15.62:6789/0 conn(0x7f25dc1595a0 legacy=0x7f25dc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e099a700 1 -- 172.21.15.31:0/2067769765 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc0ca130 con 0x7f25dc068d80 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.478+0000 7f25e199c700 1 --2- 172.21.15.31:0/2067769765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc0cc0d0 0x7f25dc0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.479+0000 7f25e099a700 1 --2- 172.21.15.31:0/2067769765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc068d80 0x7f25dc159970 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0099e0 tx=0x7f25cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76768320635d4aa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.479+0000 7f25d3fff700 1 -- 172.21.15.31:0/2067769765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00b820 con 0x7f25dc068d80 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.479+0000 7f25d3fff700 1 -- 172.21.15.31:0/2067769765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc00b980 con 0x7f25dc068d80 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.479+0000 7f25d3fff700 1 -- 172.21.15.31:0/2067769765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00bd20 con 0x7f25dc068d80 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.480+0000 7f25e33ff700 1 -- 172.21.15.31:0/2067769765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc068d80 msgr2=0x7f25dc159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.480+0000 7f25e33ff700 1 --2- 172.21.15.31:0/2067769765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc068d80 0x7f25dc159970 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0099e0 tx=0x7f25cc009250 comp rx=0 tx=0).stop 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.480+0000 7f25e33ff700 1 -- 172.21.15.31:0/2067769765 shutdown_connections 2023-10-06T05:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.480+0000 7f25e33ff700 1 --2- 172.21.15.31:0/2067769765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc0cc0d0 0x7f25dc0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.480+0000 7f25e33ff700 1 --2- 172.21.15.31:0/2067769765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc068d80 0x7f25dc159970 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.480+0000 7f25e33ff700 1 -- 172.21.15.31:0/2067769765 >> 172.21.15.31:0/2067769765 conn(0x7f25dc06b620 msgr2=0x7f25dc06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.480+0000 7f25e33ff700 1 -- 172.21.15.31:0/2067769765 shutdown_connections 2023-10-06T05:24:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.480+0000 7f25e33ff700 1 -- 172.21.15.31:0/2067769765 wait complete. 2023-10-06T05:24:38.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.481+0000 7f25e33ff700 1 Processor -- start 2023-10-06T05:24:38.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.481+0000 7f25e33ff700 1 -- start start 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.482+0000 7f25e33ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25dc068d80 0x7f25dc147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.482+0000 7f25e33ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc0cc0d0 0x7f25dc155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.482+0000 7f25e33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc1595a0 0x7f25dc1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.482+0000 7f25e33ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc169a20 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.482+0000 7f25e33ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc15d830 con 0x7f25dc068d80 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.482+0000 7f25e33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc15cf90 con 0x7f25dc1595a0 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.482+0000 7f25e099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc0cc0d0 0x7f25dc155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.483+0000 7f25e099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc0cc0d0 0x7f25dc155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53482/0 (socket says 172.21.15.31:53482) 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.483+0000 7f25e099a700 1 -- 172.21.15.31:0/1996083122 learned_addr learned my addr 172.21.15.31:0/1996083122 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.483+0000 7f25e199c700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc1595a0 0x7f25dc1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.483+0000 7f25e119b700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25dc068d80 0x7f25dc147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.483+0000 7f25e099a700 1 -- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc1595a0 msgr2=0x7f25dc1560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.483+0000 7f25e099a700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc1595a0 0x7f25dc1560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.483+0000 7f25e099a700 1 -- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25dc068d80 msgr2=0x7f25dc147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.483+0000 7f25e099a700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25dc068d80 0x7f25dc147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.483+0000 7f25e099a700 1 -- 172.21.15.31:0/1996083122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0094f0 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.483+0000 7f25e099a700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc0cc0d0 0x7f25dc155b70 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0090b0 tx=0x7f25cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.484+0000 7f25d17fa700 1 -- 172.21.15.31:0/1996083122 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc026480 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.484+0000 7f25d17fa700 1 -- 172.21.15.31:0/1996083122 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc01f070 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.484+0000 7f25e33ff700 1 -- 172.21.15.31:0/1996083122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc15d270 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.484+0000 7f25d17fa700 1 -- 172.21.15.31:0/1996083122 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc026850 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.484+0000 7f25e33ff700 1 -- 172.21.15.31:0/1996083122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc15c2a0 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.485+0000 7f25e33ff700 1 -- 172.21.15.31:0/1996083122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25dc169830 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.486+0000 7f25d17fa700 1 -- 172.21.15.31:0/1996083122 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f25cc01a430 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.486+0000 7f25d17fa700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f25c00674d0 0x7f25c0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.486+0000 7f25d17fa700 1 -- 172.21.15.31:0/1996083122 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f25cc021070 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.487+0000 7f25e119b700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f25c00674d0 0x7f25c0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.488+0000 7f25e119b700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f25c00674d0 0x7f25c0069980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f25d8003860 tx=0x7f25d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.491+0000 7f25d17fa700 1 -- 172.21.15.31:0/1996083122 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25dc169830 con 0x7f25dc0cc0d0 2023-10-06T05:24:38.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:38.770+0000 7f25e33ff700 1 -- 172.21.15.31:0/1996083122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) v1 -- 0x7f25dc169830 con 0x7f25dc0cc0d0 2023-10-06T05:24:39.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.262+0000 7f25d17fa700 1 -- 172.21.15.31:0/1996083122 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v415) v1 ==== 152+0+0 (secure 0 0 0) 0x7f25dc169830 con 0x7f25dc0cc0d0 2023-10-06T05:24:39.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.264+0000 7f25e33ff700 1 -- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f25c00674d0 msgr2=0x7f25c0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.264+0000 7f25e33ff700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f25c00674d0 0x7f25c0069980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f25d8003860 tx=0x7f25d8009040 comp rx=0 tx=0).stop 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.264+0000 7f25e33ff700 1 -- 172.21.15.31:0/1996083122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc0cc0d0 msgr2=0x7f25dc155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.264+0000 7f25e33ff700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc0cc0d0 0x7f25dc155b70 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0090b0 tx=0x7f25cc017000 comp rx=0 tx=0).stop 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.264+0000 7f25e33ff700 1 -- 172.21.15.31:0/1996083122 shutdown_connections 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.264+0000 7f25e33ff700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f25c00674d0 0x7f25c0069980 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.264+0000 7f25e33ff700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f25dc1595a0 0x7f25dc1560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.264+0000 7f25e33ff700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25dc068d80 0x7f25dc147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.264+0000 7f25e33ff700 1 --2- 172.21.15.31:0/1996083122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25dc0cc0d0 0x7f25dc155b70 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.264+0000 7f25e33ff700 1 -- 172.21.15.31:0/1996083122 >> 172.21.15.31:0/1996083122 conn(0x7f25dc06b620 msgr2=0x7f25dc158790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.265+0000 7f25e33ff700 1 -- 172.21.15.31:0/1996083122 shutdown_connections 2023-10-06T05:24:39.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.265+0000 7f25e33ff700 1 -- 172.21.15.31:0/1996083122 wait complete. 2023-10-06T05:24:39.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:24:39.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.483+0000 7fce737ff700 1 Processor -- start 2023-10-06T05:24:39.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.483+0000 7fce737ff700 1 -- start start 2023-10-06T05:24:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.484+0000 7fce737ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce6c15ca40 0x7fce6c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.484+0000 7fce737ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6c068da0 con 0x7fce6c06a300 2023-10-06T05:24:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.484+0000 7fce737ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6c069080 con 0x7fce6c0c8f40 2023-10-06T05:24:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.484+0000 7fce737ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6c0cabc0 con 0x7fce6c15ca40 2023-10-06T05:24:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.485+0000 7fce70d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fce6c06a300 0x7fce6c0c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60070/0 (socket says 172.21.15.31:60070) 2023-10-06T05:24:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.485+0000 7fce70d9a700 1 -- 172.21.15.31:0/3973594234 learned_addr learned my addr 172.21.15.31:0/3973594234 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.485+0000 7fce7159b700 1 --2- 172.21.15.31:0/3973594234 >> v2:172.21.15.62:3300/0 conn(0x7fce6c15ca40 0x7fce6c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.485+0000 7fce63fff700 1 -- 172.21.15.31:0/3973594234 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918163108 0 0) 0x7fce6c068da0 con 0x7fce6c06a300 2023-10-06T05:24:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.485+0000 7fce63fff700 1 -- 172.21.15.31:0/3973594234 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce58003570 con 0x7fce6c06a300 2023-10-06T05:24:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.485+0000 7fce7159b700 1 -- 172.21.15.31:0/3973594234 >> v1:172.21.15.62:6789/0 conn(0x7fce6c0c8f40 legacy=0x7fce6c0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.485+0000 7fce7159b700 1 -- 172.21.15.31:0/3973594234 >> v1:172.21.15.31:6789/0 conn(0x7fce6c06a300 legacy=0x7fce6c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.485+0000 7fce7159b700 1 -- 172.21.15.31:0/3973594234 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c069360 con 0x7fce6c15ca40 2023-10-06T05:24:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.486+0000 7fce7159b700 1 --2- 172.21.15.31:0/3973594234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce6c15ca40 0x7fce6c068670 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fce680098e0 tx=0x7fce680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a778624303e92e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:39.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.486+0000 7fce63fff700 1 -- 172.21.15.31:0/3973594234 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800b640 con 0x7fce6c15ca40 2023-10-06T05:24:39.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.486+0000 7fce63fff700 1 -- 172.21.15.31:0/3973594234 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6800b7a0 con 0x7fce6c15ca40 2023-10-06T05:24:39.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.486+0000 7fce63fff700 1 -- 172.21.15.31:0/3973594234 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800bb40 con 0x7fce6c15ca40 2023-10-06T05:24:39.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.486+0000 7fce737ff700 1 -- 172.21.15.31:0/3973594234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce6c15ca40 msgr2=0x7fce6c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.486+0000 7fce737ff700 1 --2- 172.21.15.31:0/3973594234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce6c15ca40 0x7fce6c068670 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fce680098e0 tx=0x7fce680091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.487+0000 7fce737ff700 1 -- 172.21.15.31:0/3973594234 shutdown_connections 2023-10-06T05:24:39.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.487+0000 7fce737ff700 1 --2- 172.21.15.31:0/3973594234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce6c15ca40 0x7fce6c068670 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.487+0000 7fce737ff700 1 -- 172.21.15.31:0/3973594234 >> 172.21.15.31:0/3973594234 conn(0x7fce6c0ce290 msgr2=0x7fce6c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:39.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.487+0000 7fce737ff700 1 -- 172.21.15.31:0/3973594234 shutdown_connections 2023-10-06T05:24:39.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.487+0000 7fce737ff700 1 -- 172.21.15.31:0/3973594234 wait complete. 2023-10-06T05:24:39.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.488+0000 7fce737ff700 1 Processor -- start 2023-10-06T05:24:39.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.488+0000 7fce737ff700 1 -- start start 2023-10-06T05:24:39.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.489+0000 7fce737ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce6c06a300 0x7fce6c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.489+0000 7fce737ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce6c0c8f40 0x7fce6c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.489+0000 7fce737ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce6c15ca40 0x7fce6c0afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.489+0000 7fce737ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0c2620 con 0x7fce6c15ca40 2023-10-06T05:24:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.489+0000 7fce737ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0c2ec0 con 0x7fce6c0c8f40 2023-10-06T05:24:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.489+0000 7fce737ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0b0a60 con 0x7fce6c06a300 2023-10-06T05:24:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.490+0000 7fce71d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce6c15ca40 0x7fce6c0afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.490+0000 7fce71d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce6c15ca40 0x7fce6c0afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53498/0 (socket says 172.21.15.31:53498) 2023-10-06T05:24:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.490+0000 7fce71d9c700 1 -- 172.21.15.31:0/926585436 learned_addr learned my addr 172.21.15.31:0/926585436 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.490+0000 7fce70d9a700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce6c0c8f40 0x7fce6c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.490+0000 7fce7159b700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce6c06a300 0x7fce6c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.490+0000 7fce71d9c700 1 -- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce6c06a300 msgr2=0x7fce6c158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.490+0000 7fce71d9c700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce6c06a300 0x7fce6c158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.490+0000 7fce71d9c700 1 -- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce6c0c8f40 msgr2=0x7fce6c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.490+0000 7fce71d9c700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce6c0c8f40 0x7fce6c0c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.490+0000 7fce71d9c700 1 -- 172.21.15.31:0/926585436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68009420 con 0x7fce6c15ca40 2023-10-06T05:24:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.491+0000 7fce71d9c700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce6c15ca40 0x7fce6c0afea0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fce6400c940 tx=0x7fce640016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.491+0000 7fce617fa700 1 -- 172.21.15.31:0/926585436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64012630 con 0x7fce6c15ca40 2023-10-06T05:24:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.491+0000 7fce617fa700 1 -- 172.21.15.31:0/926585436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64012790 con 0x7fce6c15ca40 2023-10-06T05:24:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.491+0000 7fce617fa700 1 -- 172.21.15.31:0/926585436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64012b50 con 0x7fce6c15ca40 2023-10-06T05:24:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.491+0000 7fce737ff700 1 -- 172.21.15.31:0/926585436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c0c2870 con 0x7fce6c15ca40 2023-10-06T05:24:39.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.491+0000 7fce737ff700 1 -- 172.21.15.31:0/926585436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c158eb0 con 0x7fce6c15ca40 2023-10-06T05:24:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.492+0000 7fce737ff700 1 -- 172.21.15.31:0/926585436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c0c3050 con 0x7fce6c15ca40 2023-10-06T05:24:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.493+0000 7fce617fa700 1 -- 172.21.15.31:0/926585436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fce6401b450 con 0x7fce6c15ca40 2023-10-06T05:24:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.494+0000 7fce617fa700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce500674d0 0x7fce50069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.494+0000 7fce617fa700 1 -- 172.21.15.31:0/926585436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fce64084f60 con 0x7fce6c15ca40 2023-10-06T05:24:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.495+0000 7fce7159b700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce500674d0 0x7fce50069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:39.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.495+0000 7fce7159b700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce500674d0 0x7fce50069980 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fce680093f0 tx=0x7fce68017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.498+0000 7fce617fa700 1 -- 172.21.15.31:0/926585436 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce64055530 con 0x7fce6c15ca40 2023-10-06T05:24:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.702+0000 7fce737ff700 1 -- 172.21.15.31:0/926585436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fce6c04c380 con 0x7fce6c15ca40 2023-10-06T05:24:39.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.706+0000 7fce617fa700 1 -- 172.21.15.31:0/926585436 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fce640160c0 con 0x7fce6c15ca40 2023-10-06T05:24:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.708+0000 7fce737ff700 1 -- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce500674d0 msgr2=0x7fce50069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.708+0000 7fce737ff700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce500674d0 0x7fce50069980 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fce680093f0 tx=0x7fce68017000 comp rx=0 tx=0).stop 2023-10-06T05:24:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.708+0000 7fce737ff700 1 -- 172.21.15.31:0/926585436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce6c15ca40 msgr2=0x7fce6c0afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.708+0000 7fce737ff700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce6c15ca40 0x7fce6c0afea0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fce6400c940 tx=0x7fce640016f0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.708+0000 7fce737ff700 1 -- 172.21.15.31:0/926585436 shutdown_connections 2023-10-06T05:24:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.708+0000 7fce737ff700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce500674d0 0x7fce50069980 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.708+0000 7fce737ff700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce6c15ca40 0x7fce6c0afea0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.708+0000 7fce737ff700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce6c06a300 0x7fce6c158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.708+0000 7fce737ff700 1 --2- 172.21.15.31:0/926585436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce6c0c8f40 0x7fce6c0c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:39.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.709+0000 7fce737ff700 1 -- 172.21.15.31:0/926585436 >> 172.21.15.31:0/926585436 conn(0x7fce6c0ce290 msgr2=0x7fce6c159790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:39.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.709+0000 7fce737ff700 1 -- 172.21.15.31:0/926585436 shutdown_connections 2023-10-06T05:24:39.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:39.709+0000 7fce737ff700 1 -- 172.21.15.31:0/926585436 wait complete. 2023-10-06T05:24:39.728 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:24:39.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:39.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:24:39.936 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:24:39.936 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:24:39.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:40.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.130+0000 7f9ae29ff700 1 Processor -- start 2023-10-06T05:24:40.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.131+0000 7f9ae29ff700 1 -- start start 2023-10-06T05:24:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.132+0000 7f9ae29ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9adc13fda0 0x7f9adc069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.132+0000 7f9ae29ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc0ca5b0 con 0x7f9adc0c7f30 2023-10-06T05:24:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.132+0000 7f9ae29ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc06f740 con 0x7f9adc0c7b60 2023-10-06T05:24:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.132+0000 7f9ae29ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc0c9c80 con 0x7f9adc13fda0 2023-10-06T05:24:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.132+0000 7f9adb7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9adc0c7f30 0x7f9adc13f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60074/0 (socket says 172.21.15.31:60074) 2023-10-06T05:24:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.132+0000 7f9ae0f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9adc13fda0 0x7f9adc069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.132+0000 7f9adb7fe700 1 -- 172.21.15.31:0/2474185838 learned_addr learned my addr 172.21.15.31:0/2474185838 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.133+0000 7f9ae0f9c700 1 -- 172.21.15.31:0/2474185838 >> v1:172.21.15.62:6789/0 conn(0x7f9adc0c7b60 legacy=0x7f9adc06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.133+0000 7f9ae0f9c700 1 -- 172.21.15.31:0/2474185838 >> v1:172.21.15.31:6789/0 conn(0x7f9adc0c7f30 legacy=0x7f9adc13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.133+0000 7f9ae0f9c700 1 -- 172.21.15.31:0/2474185838 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc0c9f60 con 0x7f9adc13fda0 2023-10-06T05:24:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.133+0000 7f9adaffd700 1 -- 172.21.15.31:0/2474185838 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023180608 0 0) 0x7f9adc06f740 con 0x7f9adc0c7b60 2023-10-06T05:24:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.133+0000 7f9ae0f9c700 1 --2- 172.21.15.31:0/2474185838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc13fda0 0x7f9adc069880 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009e60 tx=0x7f9ad4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60502279d8cfcea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.133+0000 7f9adaffd700 1 -- 172.21.15.31:0/2474185838 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400e070 con 0x7f9adc13fda0 2023-10-06T05:24:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.133+0000 7f9adaffd700 1 -- 172.21.15.31:0/2474185838 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad400bb30 con 0x7f9adc13fda0 2023-10-06T05:24:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.133+0000 7f9adaffd700 1 -- 172.21.15.31:0/2474185838 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4012430 con 0x7f9adc13fda0 2023-10-06T05:24:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.134+0000 7f9ae29ff700 1 -- 172.21.15.31:0/2474185838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc13fda0 msgr2=0x7f9adc069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.134+0000 7f9ae29ff700 1 --2- 172.21.15.31:0/2474185838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc13fda0 0x7f9adc069880 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009e60 tx=0x7f9ad4009780 comp rx=0 tx=0).stop 2023-10-06T05:24:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.134+0000 7f9ae29ff700 1 -- 172.21.15.31:0/2474185838 shutdown_connections 2023-10-06T05:24:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.134+0000 7f9ae29ff700 1 --2- 172.21.15.31:0/2474185838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc13fda0 0x7f9adc069880 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.134+0000 7f9ae29ff700 1 -- 172.21.15.31:0/2474185838 >> 172.21.15.31:0/2474185838 conn(0x7f9adc0cc0b0 msgr2=0x7f9adc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.134+0000 7f9ae29ff700 1 -- 172.21.15.31:0/2474185838 shutdown_connections 2023-10-06T05:24:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.134+0000 7f9ae29ff700 1 -- 172.21.15.31:0/2474185838 wait complete. 2023-10-06T05:24:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.135+0000 7f9ae29ff700 1 Processor -- start 2023-10-06T05:24:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.136+0000 7f9ae29ff700 1 -- start start 2023-10-06T05:24:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.136+0000 7f9ae29ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc0c7b60 0x7f9adc13eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9ae29ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9adc0c7f30 0x7f9adc1572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9ae29ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc13fda0 0x7f9adc1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9ae29ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc158a20 con 0x7f9adc0c7b60 2023-10-06T05:24:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9ae29ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc13f040 con 0x7f9adc13fda0 2023-10-06T05:24:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9ae29ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc0c7560 con 0x7f9adc0c7f30 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9adbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc0c7b60 0x7f9adc13eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9adbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc0c7b60 0x7f9adc13eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53506/0 (socket says 172.21.15.31:53506) 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9adbfff700 1 -- 172.21.15.31:0/531578886 learned_addr learned my addr 172.21.15.31:0/531578886 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9ae0f9c700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc13fda0 0x7f9adc1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9adb7fe700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9adc0c7f30 0x7f9adc1572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9adbfff700 1 -- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9adc0c7f30 msgr2=0x7f9adc1572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9adbfff700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9adc0c7f30 0x7f9adc1572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9adbfff700 1 -- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc13fda0 msgr2=0x7f9adc1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9adbfff700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc13fda0 0x7f9adc1577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.137+0000 7f9adbfff700 1 -- 172.21.15.31:0/531578886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009a20 con 0x7f9adc0c7b60 2023-10-06T05:24:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.138+0000 7f9adbfff700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc0c7b60 0x7f9adc13eb00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00c810 tx=0x7f9acc007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.138+0000 7f9ac3fff700 1 -- 172.21.15.31:0/531578886 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc012700 con 0x7f9adc0c7b60 2023-10-06T05:24:40.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.138+0000 7f9ac3fff700 1 -- 172.21.15.31:0/531578886 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc0128a0 con 0x7f9adc0c7b60 2023-10-06T05:24:40.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.138+0000 7f9ac3fff700 1 -- 172.21.15.31:0/531578886 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc012c60 con 0x7f9adc0c7b60 2023-10-06T05:24:40.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.138+0000 7f9ae29ff700 1 -- 172.21.15.31:0/531578886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc157e10 con 0x7f9adc0c7b60 2023-10-06T05:24:40.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.138+0000 7f9ae29ff700 1 -- 172.21.15.31:0/531578886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc1583d0 con 0x7f9adc0c7b60 2023-10-06T05:24:40.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.139+0000 7f9ae29ff700 1 -- 172.21.15.31:0/531578886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc04c380 con 0x7f9adc0c7b60 2023-10-06T05:24:40.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.140+0000 7f9ac3fff700 1 -- 172.21.15.31:0/531578886 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9acc01b400 con 0x7f9adc0c7b60 2023-10-06T05:24:40.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.141+0000 7f9ac3fff700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9ac80674d0 0x7f9ac8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.141+0000 7f9ac3fff700 1 -- 172.21.15.31:0/531578886 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f9acc0849b0 con 0x7f9adc0c7b60 2023-10-06T05:24:40.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.141+0000 7f9adb7fe700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9ac80674d0 0x7f9ac8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.142+0000 7f9adb7fe700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9ac80674d0 0x7f9ac8069980 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0b1830 tx=0x7f9ad0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.145+0000 7f9ac3fff700 1 -- 172.21.15.31:0/531578886 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9acc0514f0 con 0x7f9adc0c7b60 2023-10-06T05:24:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.396+0000 7f9ae29ff700 1 -- 172.21.15.31:0/531578886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9adc0ca240 con 0x7f9adc0c7b60 2023-10-06T05:24:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.397+0000 7f9ac3fff700 1 -- 172.21.15.31:0/531578886 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f9acc054b10 con 0x7f9adc0c7b60 2023-10-06T05:24:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:40.399 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6845,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3816040413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3816040413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3816040413}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1295480750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1295480750},{"type":"v1","addr":"172.21.15.31:6834","nonce":1295480750}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2807869230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2807869230},{"type":"v1","addr":"172.21.15.62:6834","nonce":2807869230}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":8122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3973540010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3973540010},{"type":"v1","addr":"172.21.15.62:6835","nonce":3973540010}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[]} 2023-10-06T05:24:40.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 -- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9ac80674d0 msgr2=0x7f9ac8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9ac80674d0 0x7f9ac8069980 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0b1830 tx=0x7f9ad0009040 comp rx=0 tx=0).stop 2023-10-06T05:24:40.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 -- 172.21.15.31:0/531578886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc0c7b60 msgr2=0x7f9adc13eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc0c7b60 0x7f9adc13eb00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00c810 tx=0x7f9acc007dd0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 -- 172.21.15.31:0/531578886 shutdown_connections 2023-10-06T05:24:40.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9ac80674d0 0x7f9ac8069980 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc13fda0 0x7f9adc1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9adc0c7b60 0x7f9adc13eb00 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 --2- 172.21.15.31:0/531578886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9adc0c7f30 0x7f9adc1572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 -- 172.21.15.31:0/531578886 >> 172.21.15.31:0/531578886 conn(0x7f9adc0cc0b0 msgr2=0x7f9adc15a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:40.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 -- 172.21.15.31:0/531578886 shutdown_connections 2023-10-06T05:24:40.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.399+0000 7f9ae29ff700 1 -- 172.21.15.31:0/531578886 wait complete. 2023-10-06T05:24:40.402 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 87 2023-10-06T05:24:40.419 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:24:40.419 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:24:40.419 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:24:40.419 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:40.419 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:24:40.419 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:24:40.420 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:24:40.420 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:40.420 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:24:40.420 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:24:40.420 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:24:40.421 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:40.421 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:24:40.421 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:24:40.421 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:24:40.421 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:24:40.436 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:24:40.437 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:24:40.437 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:24:40.437 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:24:40.438 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:24:40.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:24:40.439 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:24:40.440 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:24:40.481 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:24:40.481 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:40.483 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:24:40.483 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:40.484 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:24:40.484 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:40.486 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:24:40.486 DEBUG:teuthology.parallel:result is None 2023-10-06T05:24:40.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:24:40.630 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:24:40.631 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:24:40.633 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:24:40.633 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:24:40.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.693+0000 7f32747ff700 1 Processor -- start 2023-10-06T05:24:40.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.693+0000 7f32747ff700 1 -- start start 2023-10-06T05:24:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.695+0000 7f32747ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f326c159430 0x7f326c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.695+0000 7f32747ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326c068650 con 0x7f326c067560 2023-10-06T05:24:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.695+0000 7f32747ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326c067a40 con 0x7f326c069da0 2023-10-06T05:24:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.695+0000 7f32747ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326c067d20 con 0x7f326c159430 2023-10-06T05:24:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.695+0000 7f3271d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f326c067560 0x7f326c069690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60100/0 (socket says 172.21.15.31:60100) 2023-10-06T05:24:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.695+0000 7f3271d9a700 1 -- 172.21.15.31:0/1614506595 learned_addr learned my addr 172.21.15.31:0/1614506595 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:40.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.695+0000 7f327259b700 1 --2- 172.21.15.31:0/1614506595 >> v2:172.21.15.62:3300/0 conn(0x7f326c159430 0x7f326c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f3271599700 1 -- 172.21.15.31:0/1614506595 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617021764 0 0) 0x7f326c068650 con 0x7f326c067560 2023-10-06T05:24:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f3271599700 1 -- 172.21.15.31:0/1614506595 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3258003570 con 0x7f326c067560 2023-10-06T05:24:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f3271599700 1 -- 172.21.15.31:0/1614506595 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588801916 0 0) 0x7f326c067a40 con 0x7f326c069da0 2023-10-06T05:24:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f3271599700 1 -- 172.21.15.31:0/1614506595 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f326c068650 con 0x7f326c069da0 2023-10-06T05:24:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f327259b700 1 -- 172.21.15.31:0/1614506595 >> v1:172.21.15.62:6789/0 conn(0x7f326c069da0 legacy=0x7f326c0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f327259b700 1 -- 172.21.15.31:0/1614506595 >> v1:172.21.15.31:6789/0 conn(0x7f326c067560 legacy=0x7f326c069690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f327259b700 1 -- 172.21.15.31:0/1614506595 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c068000 con 0x7f326c159430 2023-10-06T05:24:40.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f327259b700 1 --2- 172.21.15.31:0/1614506595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c159430 0x7f326c068ae0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f32680098e0 tx=0x7f32680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8bc1241a3b7d7c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f3271599700 1 -- 172.21.15.31:0/1614506595 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3268011630 con 0x7f326c159430 2023-10-06T05:24:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f3271599700 1 -- 172.21.15.31:0/1614506595 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3268011790 con 0x7f326c159430 2023-10-06T05:24:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.696+0000 7f3271599700 1 -- 172.21.15.31:0/1614506595 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3268011b30 con 0x7f326c159430 2023-10-06T05:24:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.697+0000 7f32747ff700 1 -- 172.21.15.31:0/1614506595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c159430 msgr2=0x7f326c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.697+0000 7f32747ff700 1 --2- 172.21.15.31:0/1614506595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c159430 0x7f326c068ae0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f32680098e0 tx=0x7f32680091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.697+0000 7f32747ff700 1 -- 172.21.15.31:0/1614506595 shutdown_connections 2023-10-06T05:24:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.697+0000 7f32747ff700 1 --2- 172.21.15.31:0/1614506595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c159430 0x7f326c068ae0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.697+0000 7f32747ff700 1 -- 172.21.15.31:0/1614506595 >> 172.21.15.31:0/1614506595 conn(0x7f326c0c9320 msgr2=0x7f326c0c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.697+0000 7f32747ff700 1 -- 172.21.15.31:0/1614506595 shutdown_connections 2023-10-06T05:24:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.697+0000 7f32747ff700 1 -- 172.21.15.31:0/1614506595 wait complete. 2023-10-06T05:24:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.698+0000 7f32747ff700 1 Processor -- start 2023-10-06T05:24:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.699+0000 7f32747ff700 1 -- start start 2023-10-06T05:24:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.699+0000 7f32747ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f326c067560 0x7f326c1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.699+0000 7f32747ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c069da0 0x7f326c0ab690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f32747ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f326c159430 0x7f326c0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f32747ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c169960 con 0x7f326c159430 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f32747ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c16a930 con 0x7f326c069da0 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f32747ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c16a4a0 con 0x7f326c067560 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f3271d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c069da0 0x7f326c0ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f3272d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f326c159430 0x7f326c0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f3271d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c069da0 0x7f326c0ab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:60124/0 (socket says 172.21.15.31:60124) 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f3271d9a700 1 -- 172.21.15.31:0/312191946 learned_addr learned my addr 172.21.15.31:0/312191946 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f327259b700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f326c067560 0x7f326c1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f3271d9a700 1 -- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f326c067560 msgr2=0x7f326c1471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f3271d9a700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f326c067560 0x7f326c1471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f3271d9a700 1 -- 172.21.15.31:0/312191946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f326c159430 msgr2=0x7f326c0abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:40.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f3271d9a700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f326c159430 0x7f326c0abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:40.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.700+0000 7f3271d9a700 1 -- 172.21.15.31:0/312191946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268009420 con 0x7f326c069da0 2023-10-06T05:24:40.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.701+0000 7f3271d9a700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c069da0 0x7f326c0ab690 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f325c00c580 tx=0x7f325c007e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.701+0000 7f32627fc700 1 -- 172.21.15.31:0/312191946 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c005b30 con 0x7f326c069da0 2023-10-06T05:24:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.701+0000 7f32627fc700 1 -- 172.21.15.31:0/312191946 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c005cb0 con 0x7f326c069da0 2023-10-06T05:24:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.701+0000 7f32627fc700 1 -- 172.21.15.31:0/312191946 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c007320 con 0x7f326c069da0 2023-10-06T05:24:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.701+0000 7f32747ff700 1 -- 172.21.15.31:0/312191946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c169b70 con 0x7f326c069da0 2023-10-06T05:24:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.701+0000 7f32747ff700 1 -- 172.21.15.31:0/312191946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c16a130 con 0x7f326c069da0 2023-10-06T05:24:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.702+0000 7f32747ff700 1 -- 172.21.15.31:0/312191946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c16a630 con 0x7f326c069da0 2023-10-06T05:24:40.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.703+0000 7f32627fc700 1 -- 172.21.15.31:0/312191946 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f325c01b3b0 con 0x7f326c069da0 2023-10-06T05:24:40.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.704+0000 7f32627fc700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f32500674d0 0x7f3250069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:40.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.704+0000 7f32627fc700 1 -- 172.21.15.31:0/312191946 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f325c084250 con 0x7f326c069da0 2023-10-06T05:24:40.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.704+0000 7f327259b700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f32500674d0 0x7f3250069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:40.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.705+0000 7f327259b700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f32500674d0 0x7f3250069980 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f32680093f0 tx=0x7f3268017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:40.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.709+0000 7f32627fc700 1 -- 172.21.15.31:0/312191946 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f325c050dd0 con 0x7f326c069da0 2023-10-06T05:24:40.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:40.929+0000 7f32747ff700 1 -- 172.21.15.31:0/312191946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f326c04c380 con 0x7f326c069da0 2023-10-06T05:24:42.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.137+0000 7f32627fc700 1 -- 172.21.15.31:0/312191946 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v135) v1 ==== 97+0+0 (secure 0 0 0) 0x7f325c0543f0 con 0x7f326c069da0 2023-10-06T05:24:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 -- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f32500674d0 msgr2=0x7f3250069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f32500674d0 0x7f3250069980 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f32680093f0 tx=0x7f3268017000 comp rx=0 tx=0).stop 2023-10-06T05:24:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 -- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c069da0 msgr2=0x7f326c0ab690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c069da0 0x7f326c0ab690 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f325c00c580 tx=0x7f325c007e20 comp rx=0 tx=0).stop 2023-10-06T05:24:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 -- 172.21.15.31:0/312191946 shutdown_connections 2023-10-06T05:24:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f32500674d0 0x7f3250069980 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f326c159430 0x7f326c0abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f326c067560 0x7f326c1471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 --2- 172.21.15.31:0/312191946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c069da0 0x7f326c0ab690 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 -- 172.21.15.31:0/312191946 >> 172.21.15.31:0/312191946 conn(0x7f326c0c9320 msgr2=0x7f326c158680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 -- 172.21.15.31:0/312191946 shutdown_connections 2023-10-06T05:24:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.140+0000 7f32747ff700 1 -- 172.21.15.31:0/312191946 wait complete. 2023-10-06T05:24:42.144 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:24:42.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:24:42.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.362+0000 7f26b2fff700 1 Processor -- start 2023-10-06T05:24:42.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.363+0000 7f26b2fff700 1 -- start start 2023-10-06T05:24:42.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.364+0000 7f26b2fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26ac13a1e0 0x7f26ac0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.364+0000 7f26b2fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26ac06db50 0x7f26ac069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.364+0000 7f26b2fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac0c71b0 con 0x7f26ac13a5b0 2023-10-06T05:24:42.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.364+0000 7f26b2fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ac0c7490 con 0x7f26ac06db50 2023-10-06T05:24:42.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.364+0000 7f26b2fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0c7770 con 0x7f26ac13a1e0 2023-10-06T05:24:42.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.364+0000 7f26b0d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26ac13a1e0 0x7f26ac0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.364+0000 7f26b159c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26ac06db50 0x7f26ac069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.365+0000 7f26b159c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26ac06db50 0x7f26ac069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:60128/0 (socket says 172.21.15.31:60128) 2023-10-06T05:24:42.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.365+0000 7f26b159c700 1 -- 172.21.15.31:0/111220443 learned_addr learned my addr 172.21.15.31:0/111220443 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:42.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.365+0000 7f26abfff700 1 --1- 172.21.15.31:0/111220443 >> v1:172.21.15.62:6789/0 conn(0x7f26ac13a5b0 0x7f26ac06d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:50398/0 (socket says 172.21.15.31:50398) 2023-10-06T05:24:42.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.365+0000 7f26b159c700 1 -- 172.21.15.31:0/111220443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26ac13a1e0 msgr2=0x7f26ac0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:42.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.365+0000 7f26b159c700 1 --2- 172.21.15.31:0/111220443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26ac13a1e0 0x7f26ac0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.365+0000 7f26b159c700 1 -- 172.21.15.31:0/111220443 >> v1:172.21.15.62:6789/0 conn(0x7f26ac13a5b0 legacy=0x7f26ac06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.365+0000 7f26b159c700 1 -- 172.21.15.31:0/111220443 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac0c7a50 con 0x7f26ac06db50 2023-10-06T05:24:42.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.365+0000 7f26b159c700 1 --2- 172.21.15.31:0/111220443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26ac06db50 0x7f26ac069560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009f20 tx=0x7f26a4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa0b25527b55d3f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:42.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26ab7fe700 1 -- 172.21.15.31:0/111220443 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a400e070 con 0x7f26ac06db50 2023-10-06T05:24:42.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26ab7fe700 1 -- 172.21.15.31:0/111220443 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a400b9d0 con 0x7f26ac06db50 2023-10-06T05:24:42.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26ab7fe700 1 -- 172.21.15.31:0/111220443 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a40092e0 con 0x7f26ac06db50 2023-10-06T05:24:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26b2fff700 1 -- 172.21.15.31:0/111220443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26ac06db50 msgr2=0x7f26ac069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26b2fff700 1 --2- 172.21.15.31:0/111220443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26ac06db50 0x7f26ac069560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009f20 tx=0x7f26a4009840 comp rx=0 tx=0).stop 2023-10-06T05:24:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26b2fff700 1 -- 172.21.15.31:0/111220443 shutdown_connections 2023-10-06T05:24:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26b2fff700 1 --2- 172.21.15.31:0/111220443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26ac13a1e0 0x7f26ac0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26b2fff700 1 --2- 172.21.15.31:0/111220443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26ac06db50 0x7f26ac069560 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26b2fff700 1 -- 172.21.15.31:0/111220443 >> 172.21.15.31:0/111220443 conn(0x7f26ac0cc0b0 msgr2=0x7f26ac0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26b2fff700 1 -- 172.21.15.31:0/111220443 shutdown_connections 2023-10-06T05:24:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.366+0000 7f26b2fff700 1 -- 172.21.15.31:0/111220443 wait complete. 2023-10-06T05:24:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.368+0000 7f26b2fff700 1 Processor -- start 2023-10-06T05:24:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.368+0000 7f26b2fff700 1 -- start start 2023-10-06T05:24:42.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.369+0000 7f26b2fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26ac06db50 0x7f26ac0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.369+0000 7f26b2fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26ac13a1e0 0x7f26ac0c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.369+0000 7f26b2fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26ac13a5b0 0x7f26ac0c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.369+0000 7f26b2fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac159f70 con 0x7f26ac13a1e0 2023-10-06T05:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.369+0000 7f26b2fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac15a250 con 0x7f26ac06db50 2023-10-06T05:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.369+0000 7f26b2fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac15a530 con 0x7f26ac13a5b0 2023-10-06T05:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.369+0000 7f26abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26ac13a1e0 0x7f26ac0c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.369+0000 7f26abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26ac13a1e0 0x7f26ac0c22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53564/0 (socket says 172.21.15.31:53564) 2023-10-06T05:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.369+0000 7f26abfff700 1 -- 172.21.15.31:0/1862386787 learned_addr learned my addr 172.21.15.31:0/1862386787 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.369+0000 7f26b0d9b700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26ac06db50 0x7f26ac0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.370+0000 7f26b159c700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26ac13a5b0 0x7f26ac0c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.370+0000 7f26abfff700 1 -- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26ac13a5b0 msgr2=0x7f26ac0c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.370+0000 7f26abfff700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26ac13a5b0 0x7f26ac0c2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.370+0000 7f26abfff700 1 -- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26ac06db50 msgr2=0x7f26ac0cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.370+0000 7f26abfff700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26ac06db50 0x7f26ac0cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.370+0000 7f26abfff700 1 -- 172.21.15.31:0/1862386787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a4009ae0 con 0x7f26ac13a1e0 2023-10-06T05:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.370+0000 7f26abfff700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26ac13a1e0 0x7f26ac0c22f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f269800c560 tx=0x7f2698007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.370+0000 7f26a8ff9700 1 -- 172.21.15.31:0/1862386787 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698005a70 con 0x7f26ac13a1e0 2023-10-06T05:24:42.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.370+0000 7f26a8ff9700 1 -- 172.21.15.31:0/1862386787 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2698005bd0 con 0x7f26ac13a1e0 2023-10-06T05:24:42.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.370+0000 7f26b2fff700 1 -- 172.21.15.31:0/1862386787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac15a810 con 0x7f26ac13a1e0 2023-10-06T05:24:42.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.371+0000 7f26a8ff9700 1 -- 172.21.15.31:0/1862386787 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26980070d0 con 0x7f26ac13a1e0 2023-10-06T05:24:42.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.371+0000 7f26b2fff700 1 -- 172.21.15.31:0/1862386787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac14e8a0 con 0x7f26ac13a1e0 2023-10-06T05:24:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.372+0000 7f26b2fff700 1 -- 172.21.15.31:0/1862386787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac04c380 con 0x7f26ac13a1e0 2023-10-06T05:24:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.373+0000 7f26a8ff9700 1 -- 172.21.15.31:0/1862386787 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f2698007480 con 0x7f26ac13a1e0 2023-10-06T05:24:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.373+0000 7f26a8ff9700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f26900674d0 0x7f2690069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.373+0000 7f26a8ff9700 1 -- 172.21.15.31:0/1862386787 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f2698084180 con 0x7f26ac13a1e0 2023-10-06T05:24:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.374+0000 7f26b0d9b700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f26900674d0 0x7f2690069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.375+0000 7f26b0d9b700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f26900674d0 0x7f2690069980 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009da0 tx=0x7f26a0009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:42.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.378+0000 7f26a8ff9700 1 -- 172.21.15.31:0/1862386787 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2698050d10 con 0x7f26ac13a1e0 2023-10-06T05:24:42.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.659+0000 7f26b2fff700 1 -- 172.21.15.31:0/1862386787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f26ac0c7d30 con 0x7f26ac13a1e0 2023-10-06T05:24:42.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.660+0000 7f26a8ff9700 1 -- 172.21.15.31:0/1862386787 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2698054330 con 0x7f26ac13a1e0 2023-10-06T05:24:42.663 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:42.663 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:24:42.663 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:24:42.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.663 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:24:42.663 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:24:42.663 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:42.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:24:42.666 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:24:42.668 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:42.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:42.670 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.670 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:24:42.670 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:24:42.670 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:42.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:24:42.673 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:24:42.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:24:42.675 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:24:42.677 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:24:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:24:42.679 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:24:42.679 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:24:42.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:42.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:24:42.679 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:24:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.662+0000 7f26b2fff700 1 -- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f26900674d0 msgr2=0x7f2690069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.662+0000 7f26b2fff700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f26900674d0 0x7f2690069980 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009da0 tx=0x7f26a0009120 comp rx=0 tx=0).stop 2023-10-06T05:24:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.663+0000 7f26b2fff700 1 -- 172.21.15.31:0/1862386787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26ac13a1e0 msgr2=0x7f26ac0c22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.663+0000 7f26b2fff700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26ac13a1e0 0x7f26ac0c22f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f269800c560 tx=0x7f2698007f70 comp rx=0 tx=0).stop 2023-10-06T05:24:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.663+0000 7f26b2fff700 1 -- 172.21.15.31:0/1862386787 shutdown_connections 2023-10-06T05:24:42.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.663+0000 7f26b2fff700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f26900674d0 0x7f2690069980 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.663+0000 7f26b2fff700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26ac13a5b0 0x7f26ac0c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.663+0000 7f26b2fff700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26ac06db50 0x7f26ac0cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.663+0000 7f26b2fff700 1 --2- 172.21.15.31:0/1862386787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26ac13a1e0 0x7f26ac0c22f0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.663+0000 7f26b2fff700 1 -- 172.21.15.31:0/1862386787 >> 172.21.15.31:0/1862386787 conn(0x7f26ac0cc0b0 msgr2=0x7f26ac0af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.663+0000 7f26b2fff700 1 -- 172.21.15.31:0/1862386787 shutdown_connections 2023-10-06T05:24:42.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.663+0000 7f26b2fff700 1 -- 172.21.15.31:0/1862386787 wait complete. 2023-10-06T05:24:42.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.876+0000 7fba065ff700 1 Processor -- start 2023-10-06T05:24:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.876+0000 7fba065ff700 1 -- start start 2023-10-06T05:24:42.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.877+0000 7fba065ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fba000c7f30 0x7fba00069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.877+0000 7fba065ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba000aee70 0x7fba0013f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.877+0000 7fba065ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba000ca5b0 con 0x7fba000c7b60 2023-10-06T05:24:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.877+0000 7fba065ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba000af2f0 con 0x7fba000c7f30 2023-10-06T05:24:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.877+0000 7fba065ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0006f740 con 0x7fba000aee70 2023-10-06T05:24:42.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.878+0000 7fb9ff7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fba000c7f30 0x7fba00069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.878+0000 7fba04b9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba000aee70 0x7fba0013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.878+0000 7fb9ff7fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fba000c7f30 0x7fba00069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:60150/0 (socket says 172.21.15.31:60150) 2023-10-06T05:24:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.878+0000 7fb9ff7fe700 1 -- 172.21.15.31:0/3688706861 learned_addr learned my addr 172.21.15.31:0/3688706861 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.878+0000 7fb9ff7fe700 1 -- 172.21.15.31:0/3688706861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba000aee70 msgr2=0x7fba0013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.878+0000 7fb9ff7fe700 1 --2- 172.21.15.31:0/3688706861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba000aee70 0x7fba0013f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.878+0000 7fb9ff7fe700 1 -- 172.21.15.31:0/3688706861 >> v1:172.21.15.62:6789/0 conn(0x7fba000c7b60 legacy=0x7fba0006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.878+0000 7fb9ff7fe700 1 -- 172.21.15.31:0/3688706861 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba000c9c80 con 0x7fba000c7f30 2023-10-06T05:24:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.878+0000 7fb9feffd700 1 -- 172.21.15.31:0/3688706861 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097567733 0 0) 0x7fba000ca5b0 con 0x7fba000c7b60 2023-10-06T05:24:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.879+0000 7fb9ff7fe700 1 --2- 172.21.15.31:0/3688706861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba000c7f30 0x7fba00069880 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40099b0 tx=0x7fb9f4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9eb4c60861d3779 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.879+0000 7fb9feffd700 1 -- 172.21.15.31:0/3688706861 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4009ea0 con 0x7fba000c7f30 2023-10-06T05:24:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.879+0000 7fb9feffd700 1 -- 172.21.15.31:0/3688706861 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f400b860 con 0x7fba000c7f30 2023-10-06T05:24:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.879+0000 7fb9feffd700 1 -- 172.21.15.31:0/3688706861 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400bc00 con 0x7fba000c7f30 2023-10-06T05:24:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.880+0000 7fba065ff700 1 -- 172.21.15.31:0/3688706861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba000c7f30 msgr2=0x7fba00069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.880+0000 7fba065ff700 1 --2- 172.21.15.31:0/3688706861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba000c7f30 0x7fba00069880 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40099b0 tx=0x7fb9f4009250 comp rx=0 tx=0).stop 2023-10-06T05:24:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.880+0000 7fba065ff700 1 -- 172.21.15.31:0/3688706861 shutdown_connections 2023-10-06T05:24:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.880+0000 7fba065ff700 1 --2- 172.21.15.31:0/3688706861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba000aee70 0x7fba0013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.880+0000 7fba065ff700 1 --2- 172.21.15.31:0/3688706861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba000c7f30 0x7fba00069880 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.880+0000 7fba065ff700 1 -- 172.21.15.31:0/3688706861 >> 172.21.15.31:0/3688706861 conn(0x7fba000cc0b0 msgr2=0x7fba000cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:42.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.880+0000 7fba065ff700 1 -- 172.21.15.31:0/3688706861 shutdown_connections 2023-10-06T05:24:42.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.880+0000 7fba065ff700 1 -- 172.21.15.31:0/3688706861 wait complete. 2023-10-06T05:24:42.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.881+0000 7fba065ff700 1 Processor -- start 2023-10-06T05:24:42.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.881+0000 7fba065ff700 1 -- start start 2023-10-06T05:24:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.882+0000 7fba065ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba000aee70 0x7fba00167870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.882+0000 7fba065ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba000c7b60 0x7fba000ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.882+0000 7fba065ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba000c7f30 0x7fba000ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.882+0000 7fba065ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba000ac8e0 con 0x7fba000aee70 2023-10-06T05:24:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.882+0000 7fba065ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba000acb50 con 0x7fba000c7b60 2023-10-06T05:24:42.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.882+0000 7fba065ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba000ace30 con 0x7fba000c7f30 2023-10-06T05:24:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9fffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba000aee70 0x7fba00167870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9fffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba000aee70 0x7fba00167870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53570/0 (socket says 172.21.15.31:53570) 2023-10-06T05:24:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9fffff700 1 -- 172.21.15.31:0/2456919016 learned_addr learned my addr 172.21.15.31:0/2456919016 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9fffff700 1 -- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba000c7f30 msgr2=0x7fba000ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9fffff700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba000c7f30 0x7fba000ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9fffff700 1 -- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba000c7b60 msgr2=0x7fba000ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:42.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9fffff700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba000c7b60 0x7fba000ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:42.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9fffff700 1 -- 172.21.15.31:0/2456919016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f40094f0 con 0x7fba000aee70 2023-10-06T05:24:42.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9fffff700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba000aee70 0x7fba00167870 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000c810 tx=0x7fb9f0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9e7fff700 1 -- 172.21.15.31:0/2456919016 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0012700 con 0x7fba000aee70 2023-10-06T05:24:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fb9e7fff700 1 -- 172.21.15.31:0/2456919016 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f00128a0 con 0x7fba000aee70 2023-10-06T05:24:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.883+0000 7fba065ff700 1 -- 172.21.15.31:0/2456919016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba000ad110 con 0x7fba000aee70 2023-10-06T05:24:42.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.884+0000 7fb9e7fff700 1 -- 172.21.15.31:0/2456919016 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0012c60 con 0x7fba000aee70 2023-10-06T05:24:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.884+0000 7fba065ff700 1 -- 172.21.15.31:0/2456919016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba000ab110 con 0x7fba000aee70 2023-10-06T05:24:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.885+0000 7fba065ff700 1 -- 172.21.15.31:0/2456919016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba000ab300 con 0x7fba000aee70 2023-10-06T05:24:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.886+0000 7fb9e7fff700 1 -- 172.21.15.31:0/2456919016 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb9f001b400 con 0x7fba000aee70 2023-10-06T05:24:42.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.886+0000 7fb9e7fff700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9ec0674d0 0x7fb9ec069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:42.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.886+0000 7fb9e7fff700 1 -- 172.21.15.31:0/2456919016 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fb9f0084980 con 0x7fba000aee70 2023-10-06T05:24:42.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.887+0000 7fb9ff7fe700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9ec0674d0 0x7fb9ec069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:42.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.888+0000 7fb9ff7fe700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9ec0674d0 0x7fb9ec069980 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fba00069dc0 tx=0x7fb9f4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:42.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:42.891+0000 7fb9e7fff700 1 -- 172.21.15.31:0/2456919016 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9f0051510 con 0x7fba000aee70 2023-10-06T05:24:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.092+0000 7fba065ff700 1 -- 172.21.15.31:0/2456919016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba0004c380 con 0x7fba000aee70 2023-10-06T05:24:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.093+0000 7fb9e7fff700 1 -- 172.21.15.31:0/2456919016 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+13752 (secure 0 0 0) 0x7fb9f0054b30 con 0x7fba000aee70 2023-10-06T05:24:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:43.096 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":135,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:42.133381+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 -- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9ec0674d0 msgr2=0x7fb9ec069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9ec0674d0 0x7fb9ec069980 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fba00069dc0 tx=0x7fb9f4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 -- 172.21.15.31:0/2456919016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba000aee70 msgr2=0x7fba00167870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba000aee70 0x7fba00167870 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000c810 tx=0x7fb9f0007dd0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 -- 172.21.15.31:0/2456919016 shutdown_connections 2023-10-06T05:24:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb9ec0674d0 0x7fb9ec069980 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba000c7f30 0x7fba000ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba000aee70 0x7fba00167870 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 --2- 172.21.15.31:0/2456919016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba000c7b60 0x7fba000ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 -- 172.21.15.31:0/2456919016 >> 172.21.15.31:0/2456919016 conn(0x7fba000cc0b0 msgr2=0x7fba000caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 -- 172.21.15.31:0/2456919016 shutdown_connections 2023-10-06T05:24:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.095+0000 7fba065ff700 1 -- 172.21.15.31:0/2456919016 wait complete. 2023-10-06T05:24:43.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:24:43.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:24:43.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.312+0000 7ff5ab5ff700 1 Processor -- start 2023-10-06T05:24:43.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.313+0000 7ff5ab5ff700 1 -- start start 2023-10-06T05:24:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.314+0000 7ff5ab5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a4153d40 0x7ff5a40b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.314+0000 7ff5ab5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a40ad690 con 0x7ff5a40af890 2023-10-06T05:24:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.314+0000 7ff5ab5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a40b0e30 con 0x7ff5a40b09c0 2023-10-06T05:24:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.314+0000 7ff5ab5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40afcd0 con 0x7ff5a4153d40 2023-10-06T05:24:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.314+0000 7ff5a9b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff5a40af890 0x7ff5a40adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60114/0 (socket says 172.21.15.31:60114) 2023-10-06T05:24:43.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.314+0000 7ff5a9b9c700 1 -- 172.21.15.31:0/2867924222 learned_addr learned my addr 172.21.15.31:0/2867924222 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:43.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.314+0000 7ff5a939b700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a4153d40 0x7ff5a40b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327556392 0 0) 0x7ff5a40ad690 con 0x7ff5a40af890 2023-10-06T05:24:43.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff590003570 con 0x7ff5a40af890 2023-10-06T05:24:43.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536729091 0 0) 0x7ff5a40b0e30 con 0x7ff5a40b09c0 2023-10-06T05:24:43.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5a40ad690 con 0x7ff5a40b09c0 2023-10-06T05:24:43.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 911025806 0 0) 0x7ff590003570 con 0x7ff5a40af890 2023-10-06T05:24:43.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a4153d40 msgr2=0x7ff5a40b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a4153d40 0x7ff5a40b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 >> v1:172.21.15.62:6789/0 conn(0x7ff5a40b09c0 legacy=0x7ff5a40af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a40acd60 con 0x7ff5a40af890 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7ff59c0017f0 con 0x7ff5a40af890 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 >> v1:172.21.15.31:6789/0 conn(0x7ff5a40af890 legacy=0x7ff5a40adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff590003c20 0x7ff590004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff590004810 0x7ff590004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5900051c0 0x7ff59000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40afcd0 con 0x7ff5900051c0 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59000c080 con 0x7ff590004810 2023-10-06T05:24:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59000c240 con 0x7ff590003c20 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff5a939b700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5a4153d40 0x7ff5a40b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.315+0000 7ff5a9b9c700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5900051c0 0x7ff59000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff5a8b9a700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff590004810 0x7ff590004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff5a939b700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff590003c20 0x7ff590004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff5a9b9c700 1 -- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff590003c20 msgr2=0x7ff590004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff5a9b9c700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff590003c20 0x7ff590004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff5a9b9c700 1 -- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff590004810 msgr2=0x7ff590004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff5a9b9c700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff590004810 0x7ff590004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff5a9b9c700 1 -- 172.21.15.31:0/2867924222 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff590003570 con 0x7ff5900051c0 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff5a8b9a700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff590004810 0x7ff590004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff5a9b9c700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5900051c0 0x7ff59000ba60 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00a820 tx=0x7ff59c0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693a3ce0b7fe2e93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:43.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0070f0 con 0x7ff5900051c0 2023-10-06T05:24:43.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c007250 con 0x7ff5900051c0 2023-10-06T05:24:43.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.316+0000 7ff59bfff700 1 -- 172.21.15.31:0/2867924222 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c007610 con 0x7ff5900051c0 2023-10-06T05:24:43.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.317+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/2867924222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5900051c0 msgr2=0x7ff59000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.317+0000 7ff5ab5ff700 1 --2- 172.21.15.31:0/2867924222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5900051c0 0x7ff59000ba60 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00a820 tx=0x7ff59c0020c0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.317+0000 7ff5a9b9c700 1 -- 172.21.15.31:0/2867924222 reap_dead start 2023-10-06T05:24:43.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.317+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/2867924222 shutdown_connections 2023-10-06T05:24:43.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.317+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/2867924222 >> 172.21.15.31:0/2867924222 conn(0x7ff5a4067560 msgr2=0x7ff5a4067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:43.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.317+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/2867924222 shutdown_connections 2023-10-06T05:24:43.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.317+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/2867924222 wait complete. 2023-10-06T05:24:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.319+0000 7ff5ab5ff700 1 Processor -- start 2023-10-06T05:24:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.319+0000 7ff5ab5ff700 1 -- start start 2023-10-06T05:24:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5ab5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5900051c0 0x7ff5a4149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5ab5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4149ab0 0x7ff5a40cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5ab5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a40cc710 0x7ff5a4144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5ab5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40ccb80 con 0x7ff5a40cc710 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5ab5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4146700 con 0x7ff5a4149ab0 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5ab5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4145660 con 0x7ff5900051c0 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5a9b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a40cc710 0x7ff5a4144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5a9b9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a40cc710 0x7ff5a4144890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53588/0 (socket says 172.21.15.31:53588) 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5a9b9c700 1 -- 172.21.15.31:0/1515188982 learned_addr learned my addr 172.21.15.31:0/1515188982 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5a939b700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5900051c0 0x7ff5a4149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.320+0000 7ff5a8b9a700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4149ab0 0x7ff5a40cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5a9b9c700 1 -- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5900051c0 msgr2=0x7ff5a4149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5a9b9c700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5900051c0 0x7ff5a4149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5a9b9c700 1 -- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4149ab0 msgr2=0x7ff5a40cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5a9b9c700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4149ab0 0x7ff5a40cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5a9b9c700 1 -- 172.21.15.31:0/1515188982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4145940 con 0x7ff5a40cc710 2023-10-06T05:24:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5a8b9a700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4149ab0 0x7ff5a40cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5a9b9c700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a40cc710 0x7ff5a4144890 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff59c012000 tx=0x7ff59c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5997fa700 1 -- 172.21.15.31:0/1515188982 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0011c0 con 0x7ff5a40cc710 2023-10-06T05:24:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5997fa700 1 -- 172.21.15.31:0/1515188982 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c007070 con 0x7ff5a40cc710 2023-10-06T05:24:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5997fa700 1 -- 172.21.15.31:0/1515188982 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c007980 con 0x7ff5a40cc710 2023-10-06T05:24:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/1515188982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a4145c20 con 0x7ff5a40cc710 2023-10-06T05:24:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.321+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/1515188982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a41461e0 con 0x7ff5a40cc710 2023-10-06T05:24:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.322+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/1515188982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a404c380 con 0x7ff5a40cc710 2023-10-06T05:24:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.323+0000 7ff5997fa700 1 -- 172.21.15.31:0/1515188982 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ff59c004bb0 con 0x7ff5a40cc710 2023-10-06T05:24:43.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.324+0000 7ff5997fa700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5880674d0 0x7ff588069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.324+0000 7ff5997fa700 1 -- 172.21.15.31:0/1515188982 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5234+0+0 (secure 0 0 0) 0x7ff59c0874b0 con 0x7ff5a40cc710 2023-10-06T05:24:43.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.324+0000 7ff5a939b700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5880674d0 0x7ff588069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.325+0000 7ff5a939b700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5880674d0 0x7ff588069980 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00093f0 tx=0x7ff5a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:43.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.328+0000 7ff5997fa700 1 -- 172.21.15.31:0/1515188982 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff59c054080 con 0x7ff5a40cc710 2023-10-06T05:24:43.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.531+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/1515188982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff5a40ad040 con 0x7ff5a40cc710 2023-10-06T05:24:43.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.532+0000 7ff5997fa700 1 -- 172.21.15.31:0/1515188982 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v135) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff59c0075d0 con 0x7ff5a40cc710 2023-10-06T05:24:43.535 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:24:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5880674d0 msgr2=0x7ff588069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5880674d0 0x7ff588069980 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00093f0 tx=0x7ff5a0008040 comp rx=0 tx=0).stop 2023-10-06T05:24:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/1515188982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a40cc710 msgr2=0x7ff5a4144890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a40cc710 0x7ff5a4144890 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff59c012000 tx=0x7ff59c016000 comp rx=0 tx=0).stop 2023-10-06T05:24:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/1515188982 shutdown_connections 2023-10-06T05:24:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff5880674d0 0x7ff588069980 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5a40cc710 0x7ff5a4144890 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff5900051c0 0x7ff5a4149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 --2- 172.21.15.31:0/1515188982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5a4149ab0 0x7ff5a40cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/1515188982 >> 172.21.15.31:0/1515188982 conn(0x7ff5a4067560 msgr2=0x7ff5a406d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/1515188982 shutdown_connections 2023-10-06T05:24:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.534+0000 7ff5ab5ff700 1 -- 172.21.15.31:0/1515188982 wait complete. 2023-10-06T05:24:43.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:24:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.751+0000 7f51833ff700 1 Processor -- start 2023-10-06T05:24:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.751+0000 7f51833ff700 1 -- start start 2023-10-06T05:24:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.752+0000 7f51833ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c0c8f40 0x7f517c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.752+0000 7f51833ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f517c06a300 0x7f517c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.752+0000 7f51833ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c068da0 con 0x7f517c15ca40 2023-10-06T05:24:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.752+0000 7f51833ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c0cabc0 con 0x7f517c06a300 2023-10-06T05:24:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.752+0000 7f51833ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517c068bb0 con 0x7f517c0c8f40 2023-10-06T05:24:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f518199c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f517c06a300 0x7f517c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f518199c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f517c06a300 0x7f517c0adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53604/0 (socket says 172.21.15.31:53604) 2023-10-06T05:24:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f518199c700 1 -- 172.21.15.31:0/673847401 learned_addr learned my addr 172.21.15.31:0/673847401 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:43.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f518099a700 1 --2- 172.21.15.31:0/673847401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c0c8f40 0x7f517c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f518199c700 1 -- 172.21.15.31:0/673847401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c0c8f40 msgr2=0x7f517c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:43.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f518199c700 1 --2- 172.21.15.31:0/673847401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c0c8f40 0x7f517c068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f518199c700 1 -- 172.21.15.31:0/673847401 >> v1:172.21.15.62:6789/0 conn(0x7f517c15ca40 legacy=0x7f517c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f518199c700 1 -- 172.21.15.31:0/673847401 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c069080 con 0x7f517c06a300 2023-10-06T05:24:43.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f518099a700 1 --2- 172.21.15.31:0/673847401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c0c8f40 0x7f517c068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:43.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f518199c700 1 --2- 172.21.15.31:0/673847401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 0x7f517c0adde0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5174009f20 tx=0x7f5174009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d620b862a64412 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.753+0000 7f5173fff700 1 -- 172.21.15.31:0/673847401 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517400e070 con 0x7f517c06a300 2023-10-06T05:24:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.754+0000 7f5173fff700 1 -- 172.21.15.31:0/673847401 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517400ba80 con 0x7f517c06a300 2023-10-06T05:24:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.754+0000 7f5173fff700 1 -- 172.21.15.31:0/673847401 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740092e0 con 0x7f517c06a300 2023-10-06T05:24:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.754+0000 7f51833ff700 1 -- 172.21.15.31:0/673847401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 msgr2=0x7f517c0adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.754+0000 7f51833ff700 1 --2- 172.21.15.31:0/673847401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 0x7f517c0adde0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5174009f20 tx=0x7f5174009840 comp rx=0 tx=0).stop 2023-10-06T05:24:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.754+0000 7f51833ff700 1 -- 172.21.15.31:0/673847401 shutdown_connections 2023-10-06T05:24:43.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.754+0000 7f51833ff700 1 --2- 172.21.15.31:0/673847401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 0x7f517c0adde0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.754+0000 7f51833ff700 1 --2- 172.21.15.31:0/673847401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c0c8f40 0x7f517c068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.754+0000 7f51833ff700 1 -- 172.21.15.31:0/673847401 >> 172.21.15.31:0/673847401 conn(0x7f517c0ce290 msgr2=0x7f517c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.754+0000 7f51833ff700 1 -- 172.21.15.31:0/673847401 shutdown_connections 2023-10-06T05:24:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.754+0000 7f51833ff700 1 -- 172.21.15.31:0/673847401 wait complete. 2023-10-06T05:24:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.756+0000 7f51833ff700 1 Processor -- start 2023-10-06T05:24:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.757+0000 7f51833ff700 1 -- start start 2023-10-06T05:24:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f51833ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 0x7f517c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f51833ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f517c0c8f40 0x7f517c0b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f51833ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c15ca40 0x7f517c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f51833ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0c4aa0 con 0x7f517c06a300 2023-10-06T05:24:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f51833ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0b12a0 con 0x7f517c0c8f40 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f51833ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517c157910 con 0x7f517c15ca40 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f518119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 0x7f517c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f518119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 0x7f517c0c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53616/0 (socket says 172.21.15.31:53616) 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f518119b700 1 -- 172.21.15.31:0/4218551776 learned_addr learned my addr 172.21.15.31:0/4218551776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f518099a700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f517c0c8f40 0x7f517c0b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.758+0000 7f518199c700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c15ca40 0x7f517c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f518119b700 1 -- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c15ca40 msgr2=0x7f517c0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f518119b700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c15ca40 0x7f517c0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f518119b700 1 -- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f517c0c8f40 msgr2=0x7f517c0b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f518119b700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f517c0c8f40 0x7f517c0b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f518119b700 1 -- 172.21.15.31:0/4218551776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174009ae0 con 0x7f517c06a300 2023-10-06T05:24:43.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f518119b700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 0x7f517c0c52a0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f517800c4e0 tx=0x7f517800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:43.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f51717fa700 1 -- 172.21.15.31:0/4218551776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517800cea0 con 0x7f517c06a300 2023-10-06T05:24:43.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f51717fa700 1 -- 172.21.15.31:0/4218551776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5178011590 con 0x7f517c06a300 2023-10-06T05:24:43.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f51717fa700 1 -- 172.21.15.31:0/4218551776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178011930 con 0x7f517c06a300 2023-10-06T05:24:43.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f51833ff700 1 -- 172.21.15.31:0/4218551776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c0c7dc0 con 0x7f517c06a300 2023-10-06T05:24:43.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.759+0000 7f51833ff700 1 -- 172.21.15.31:0/4218551776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c0c8380 con 0x7f517c06a300 2023-10-06T05:24:43.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.760+0000 7f51833ff700 1 -- 172.21.15.31:0/4218551776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c04c380 con 0x7f517c06a300 2023-10-06T05:24:43.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.761+0000 7f51717fa700 1 -- 172.21.15.31:0/4218551776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f5178011d20 con 0x7f517c06a300 2023-10-06T05:24:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.762+0000 7f51717fa700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f51600674d0 0x7f5160069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.762+0000 7f51717fa700 1 -- 172.21.15.31:0/4218551776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f51780840f0 con 0x7f517c06a300 2023-10-06T05:24:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.763+0000 7f518099a700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f51600674d0 0x7f5160069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.763+0000 7f518099a700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f51600674d0 0x7f5160069980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f517c0cd100 tx=0x7f516c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.766+0000 7f51717fa700 1 -- 172.21.15.31:0/4218551776 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5178050c80 con 0x7f517c06a300 2023-10-06T05:24:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.970+0000 7f51833ff700 1 -- 172.21.15.31:0/4218551776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f517c0ca1a0 con 0x7f517c06a300 2023-10-06T05:24:43.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.971+0000 7f51717fa700 1 -- 172.21.15.31:0/4218551776 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5178017020 con 0x7f517c06a300 2023-10-06T05:24:43.974 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:24:43.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 -- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f51600674d0 msgr2=0x7f5160069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f51600674d0 0x7f5160069980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f517c0cd100 tx=0x7f516c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 -- 172.21.15.31:0/4218551776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 msgr2=0x7f517c0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 0x7f517c0c52a0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f517800c4e0 tx=0x7f517800c7f0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 -- 172.21.15.31:0/4218551776 shutdown_connections 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f51600674d0 0x7f5160069980 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f517c15ca40 0x7f517c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f517c06a300 0x7f517c0c52a0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 --2- 172.21.15.31:0/4218551776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f517c0c8f40 0x7f517c0b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 -- 172.21.15.31:0/4218551776 >> 172.21.15.31:0/4218551776 conn(0x7f517c0ce290 msgr2=0x7f517c0b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 -- 172.21.15.31:0/4218551776 shutdown_connections 2023-10-06T05:24:43.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:43.973+0000 7f51833ff700 1 -- 172.21.15.31:0/4218551776 wait complete. 2023-10-06T05:24:43.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:24:44.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.187+0000 7f30519ff700 1 Processor -- start 2023-10-06T05:24:44.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.188+0000 7f30519ff700 1 -- start start 2023-10-06T05:24:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.189+0000 7f30519ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f304c13a1e0 0x7f304c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.189+0000 7f30519ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c06db50 0x7f304c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.189+0000 7f30519ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c0c71b0 con 0x7f304c13a5b0 2023-10-06T05:24:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.189+0000 7f30519ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c0c7490 con 0x7f304c13a1e0 2023-10-06T05:24:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.189+0000 7f30519ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0c7770 con 0x7f304c06db50 2023-10-06T05:24:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.189+0000 7f304affd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f304c13a1e0 0x7f304c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.189+0000 7f304b7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c06db50 0x7f304c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.189+0000 7f304affd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f304c13a1e0 0x7f304c0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:60194/0 (socket says 172.21.15.31:60194) 2023-10-06T05:24:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.189+0000 7f304affd700 1 -- 172.21.15.31:0/1535327935 learned_addr learned my addr 172.21.15.31:0/1535327935 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.190+0000 7f304b7fe700 1 -- 172.21.15.31:0/1535327935 >> v2:172.21.15.62:3300/0 conn(0x7f304c13a1e0 msgr2=0x7f304c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.190+0000 7f304b7fe700 1 --2- 172.21.15.31:0/1535327935 >> v2:172.21.15.62:3300/0 conn(0x7f304c13a1e0 0x7f304c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.190+0000 7f304b7fe700 1 -- 172.21.15.31:0/1535327935 >> v1:172.21.15.62:6789/0 conn(0x7f304c13a5b0 legacy=0x7f304c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.190+0000 7f304b7fe700 1 -- 172.21.15.31:0/1535327935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c0c7a50 con 0x7f304c06db50 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.190+0000 7f304affd700 1 --2- 172.21.15.31:0/1535327935 >> v2:172.21.15.62:3300/0 conn(0x7f304c13a1e0 0x7f304c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.190+0000 7f304b7fe700 1 --2- 172.21.15.31:0/1535327935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c06db50 0x7f304c069560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3044009e60 tx=0x7f3044009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80973dea906934bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f3049ffb700 1 -- 172.21.15.31:0/1535327935 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304400e070 con 0x7f304c06db50 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f3049ffb700 1 -- 172.21.15.31:0/1535327935 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304400bb60 con 0x7f304c06db50 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f3049ffb700 1 -- 172.21.15.31:0/1535327935 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044012430 con 0x7f304c06db50 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f30519ff700 1 -- 172.21.15.31:0/1535327935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c06db50 msgr2=0x7f304c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f30519ff700 1 --2- 172.21.15.31:0/1535327935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c06db50 0x7f304c069560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3044009e60 tx=0x7f3044009780 comp rx=0 tx=0).stop 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f30519ff700 1 -- 172.21.15.31:0/1535327935 shutdown_connections 2023-10-06T05:24:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f30519ff700 1 --2- 172.21.15.31:0/1535327935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c06db50 0x7f304c069560 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f30519ff700 1 --2- 172.21.15.31:0/1535327935 >> v2:172.21.15.62:3300/0 conn(0x7f304c13a1e0 0x7f304c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f30519ff700 1 -- 172.21.15.31:0/1535327935 >> 172.21.15.31:0/1535327935 conn(0x7f304c0cc0b0 msgr2=0x7f304c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f30519ff700 1 -- 172.21.15.31:0/1535327935 shutdown_connections 2023-10-06T05:24:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.191+0000 7f30519ff700 1 -- 172.21.15.31:0/1535327935 wait complete. 2023-10-06T05:24:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.193+0000 7f30519ff700 1 Processor -- start 2023-10-06T05:24:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.193+0000 7f30519ff700 1 -- start start 2023-10-06T05:24:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f30519ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f304c06db50 0x7f304c16cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f30519ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13a1e0 0x7f304c16d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f30519ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c13a5b0 0x7f304c157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f30519ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c156bd0 con 0x7f304c13a1e0 2023-10-06T05:24:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f30519ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c14fbb0 con 0x7f304c06db50 2023-10-06T05:24:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f30519ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c14fe90 con 0x7f304c13a5b0 2023-10-06T05:24:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f304a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13a1e0 0x7f304c16d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f304a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13a1e0 0x7f304c16d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53618/0 (socket says 172.21.15.31:53618) 2023-10-06T05:24:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f304a7fc700 1 -- 172.21.15.31:0/1479726809 learned_addr learned my addr 172.21.15.31:0/1479726809 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f304affd700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f304c06db50 0x7f304c16cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.194+0000 7f304b7fe700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c13a5b0 0x7f304c157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f304a7fc700 1 -- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c13a5b0 msgr2=0x7f304c157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f304a7fc700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c13a5b0 0x7f304c157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f304a7fc700 1 -- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f304c06db50 msgr2=0x7f304c16cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f304a7fc700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f304c06db50 0x7f304c16cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f304a7fc700 1 -- 172.21.15.31:0/1479726809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c009480 con 0x7f304c13a1e0 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f304a7fc700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13a1e0 0x7f304c16d300 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f304000c590 tx=0x7f3040007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f3032ffd700 1 -- 172.21.15.31:0/1479726809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040011590 con 0x7f304c13a1e0 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f3032ffd700 1 -- 172.21.15.31:0/1479726809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3040011710 con 0x7f304c13a1e0 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f30519ff700 1 -- 172.21.15.31:0/1479726809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3044009a20 con 0x7f304c13a1e0 2023-10-06T05:24:44.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f3032ffd700 1 -- 172.21.15.31:0/1479726809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040011ad0 con 0x7f304c13a1e0 2023-10-06T05:24:44.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.195+0000 7f30519ff700 1 -- 172.21.15.31:0/1479726809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c156860 con 0x7f304c13a1e0 2023-10-06T05:24:44.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.196+0000 7f30519ff700 1 -- 172.21.15.31:0/1479726809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c0aea80 con 0x7f304c13a1e0 2023-10-06T05:24:44.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.197+0000 7f3032ffd700 1 -- 172.21.15.31:0/1479726809 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f304001b3b0 con 0x7f304c13a1e0 2023-10-06T05:24:44.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.199+0000 7f3032ffd700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f30380674d0 0x7f3038069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.199+0000 7f3032ffd700 1 -- 172.21.15.31:0/1479726809 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f3040084150 con 0x7f304c13a1e0 2023-10-06T05:24:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.199+0000 7f304affd700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f30380674d0 0x7f3038069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.200+0000 7f304affd700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f30380674d0 0x7f3038069980 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f303c009f30 tx=0x7f303c009220 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.203+0000 7f3032ffd700 1 -- 172.21.15.31:0/1479726809 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3040050ce0 con 0x7f304c13a1e0 2023-10-06T05:24:44.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.406+0000 7f30519ff700 1 -- 172.21.15.31:0/1479726809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f304c04c380 con 0x7f304c13a1e0 2023-10-06T05:24:44.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.407+0000 7f3032ffd700 1 -- 172.21.15.31:0/1479726809 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v135) v1 ==== 84+0+126 (secure 0 0 0) 0x7f3040017020 con 0x7f304c13a1e0 2023-10-06T05:24:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:44.410 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:24:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:24:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:24:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:24:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:24:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:24:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:44.411 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:24:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.409+0000 7f30519ff700 1 -- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f30380674d0 msgr2=0x7f3038069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.409+0000 7f30519ff700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f30380674d0 0x7f3038069980 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f303c009f30 tx=0x7f303c009220 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.409+0000 7f30519ff700 1 -- 172.21.15.31:0/1479726809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13a1e0 msgr2=0x7f304c16d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.409+0000 7f30519ff700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13a1e0 0x7f304c16d300 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f304000c590 tx=0x7f3040007f50 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.409+0000 7f30519ff700 1 -- 172.21.15.31:0/1479726809 shutdown_connections 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.409+0000 7f30519ff700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f30380674d0 0x7f3038069980 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.409+0000 7f30519ff700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f304c13a5b0 0x7f304c157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.409+0000 7f30519ff700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f304c06db50 0x7f304c16cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.409+0000 7f30519ff700 1 --2- 172.21.15.31:0/1479726809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13a1e0 0x7f304c16d300 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.409+0000 7f30519ff700 1 -- 172.21.15.31:0/1479726809 >> 172.21.15.31:0/1479726809 conn(0x7f304c0cc0b0 msgr2=0x7f304c0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.410+0000 7f30519ff700 1 -- 172.21.15.31:0/1479726809 shutdown_connections 2023-10-06T05:24:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.410+0000 7f30519ff700 1 -- 172.21.15.31:0/1479726809 wait complete. 2023-10-06T05:24:44.430 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:24:44.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:24:44.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.626+0000 7f9cb23ff700 1 Processor -- start 2023-10-06T05:24:44.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.626+0000 7f9cb23ff700 1 -- start start 2023-10-06T05:24:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.627+0000 7f9cb23ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac1550a0 0x7f9cac0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.627+0000 7f9cb23ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9cac06ee10 0x7f9cac06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.627+0000 7f9cb23ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac06f640 con 0x7f9cac154cd0 2023-10-06T05:24:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.627+0000 7f9cb23ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cac06e610 con 0x7f9cac06ee10 2023-10-06T05:24:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.627+0000 7f9cb23ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0adf60 con 0x7f9cac1550a0 2023-10-06T05:24:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.628+0000 7f9cb099c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9cac06ee10 0x7f9cac06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.628+0000 7f9cab7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac1550a0 0x7f9cac0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.628+0000 7f9cb099c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9cac06ee10 0x7f9cac06d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:60220/0 (socket says 172.21.15.31:60220) 2023-10-06T05:24:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.628+0000 7f9cb099c700 1 -- 172.21.15.31:0/356457218 learned_addr learned my addr 172.21.15.31:0/356457218 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.628+0000 7f9cab7fe700 1 --2- 172.21.15.31:0/356457218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac1550a0 0x7f9cac0ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:51200/0 (socket says 172.21.15.31:51200) 2023-10-06T05:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.628+0000 7f9cb099c700 1 -- 172.21.15.31:0/356457218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac1550a0 msgr2=0x7f9cac0ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.628+0000 7f9cb099c700 1 --2- 172.21.15.31:0/356457218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac1550a0 0x7f9cac0ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.628+0000 7f9cb099c700 1 -- 172.21.15.31:0/356457218 >> v1:172.21.15.62:6789/0 conn(0x7f9cac154cd0 legacy=0x7f9cac1583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.628+0000 7f9cb099c700 1 -- 172.21.15.31:0/356457218 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac06f360 con 0x7f9cac06ee10 2023-10-06T05:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.628+0000 7f9cab7fe700 1 --2- 172.21.15.31:0/356457218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac1550a0 0x7f9cac0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.629+0000 7f9cb099c700 1 --2- 172.21.15.31:0/356457218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cac06ee10 0x7f9cac06d7c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009f20 tx=0x7f9ca4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3179b0051240267e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.629+0000 7f9caaffd700 1 -- 172.21.15.31:0/356457218 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400e070 con 0x7f9cac06ee10 2023-10-06T05:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.629+0000 7f9caaffd700 1 -- 172.21.15.31:0/356457218 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca400b9d0 con 0x7f9cac06ee10 2023-10-06T05:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.629+0000 7f9caaffd700 1 -- 172.21.15.31:0/356457218 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40092e0 con 0x7f9cac06ee10 2023-10-06T05:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.629+0000 7f9cb23ff700 1 -- 172.21.15.31:0/356457218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cac06ee10 msgr2=0x7f9cac06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.629+0000 7f9cb23ff700 1 --2- 172.21.15.31:0/356457218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cac06ee10 0x7f9cac06d7c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009f20 tx=0x7f9ca4009840 comp rx=0 tx=0).stop 2023-10-06T05:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.630+0000 7f9cb23ff700 1 -- 172.21.15.31:0/356457218 shutdown_connections 2023-10-06T05:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.630+0000 7f9cb23ff700 1 --2- 172.21.15.31:0/356457218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cac06ee10 0x7f9cac06d7c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.630+0000 7f9cb23ff700 1 --2- 172.21.15.31:0/356457218 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac1550a0 0x7f9cac0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.630+0000 7f9cb23ff700 1 -- 172.21.15.31:0/356457218 >> 172.21.15.31:0/356457218 conn(0x7f9cac068030 msgr2=0x7f9cac0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.630+0000 7f9cb23ff700 1 -- 172.21.15.31:0/356457218 shutdown_connections 2023-10-06T05:24:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.630+0000 7f9cb23ff700 1 -- 172.21.15.31:0/356457218 wait complete. 2023-10-06T05:24:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.631+0000 7f9cb23ff700 1 Processor -- start 2023-10-06T05:24:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.631+0000 7f9cb23ff700 1 -- start start 2023-10-06T05:24:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.632+0000 7f9cb23ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac06ee10 0x7f9cac150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.632+0000 7f9cb23ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cac154cd0 0x7f9cac159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.632+0000 7f9cb23ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac1550a0 0x7f9cac1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.632+0000 7f9cb23ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac15b200 con 0x7f9cac1550a0 2023-10-06T05:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.632+0000 7f9cb23ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac15a160 con 0x7f9cac154cd0 2023-10-06T05:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.632+0000 7f9cb23ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac15a440 con 0x7f9cac06ee10 2023-10-06T05:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.632+0000 7f9cb099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac1550a0 0x7f9cac1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9cb099c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac1550a0 0x7f9cac1525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53628/0 (socket says 172.21.15.31:53628) 2023-10-06T05:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9cb099c700 1 -- 172.21.15.31:0/499861208 learned_addr learned my addr 172.21.15.31:0/499861208 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9cabfff700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac06ee10 0x7f9cac150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9cab7fe700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cac154cd0 0x7f9cac159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9cb099c700 1 -- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac06ee10 msgr2=0x7f9cac150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9cb099c700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac06ee10 0x7f9cac150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9cb099c700 1 -- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cac154cd0 msgr2=0x7f9cac159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:44.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9cb099c700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cac154cd0 0x7f9cac159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:44.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9cb099c700 1 -- 172.21.15.31:0/499861208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca0009420 con 0x7f9cac1550a0 2023-10-06T05:24:44.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9cb099c700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac1550a0 0x7f9cac1525a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0c8920 tx=0x7f9ca4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.633+0000 7f9c93fff700 1 -- 172.21.15.31:0/499861208 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4008ab0 con 0x7f9cac1550a0 2023-10-06T05:24:44.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.634+0000 7f9c93fff700 1 -- 172.21.15.31:0/499861208 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca4008c10 con 0x7f9cac1550a0 2023-10-06T05:24:44.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.634+0000 7f9cb23ff700 1 -- 172.21.15.31:0/499861208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca4009ae0 con 0x7f9cac1550a0 2023-10-06T05:24:44.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.634+0000 7f9c93fff700 1 -- 172.21.15.31:0/499861208 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4026460 con 0x7f9cac1550a0 2023-10-06T05:24:44.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.634+0000 7f9cb23ff700 1 -- 172.21.15.31:0/499861208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac15aa00 con 0x7f9cac1550a0 2023-10-06T05:24:44.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.635+0000 7f9cb23ff700 1 -- 172.21.15.31:0/499861208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac04c380 con 0x7f9cac1550a0 2023-10-06T05:24:44.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.636+0000 7f9c93fff700 1 -- 172.21.15.31:0/499861208 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9ca4003040 con 0x7f9cac1550a0 2023-10-06T05:24:44.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.637+0000 7f9c93fff700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c980674d0 0x7f9c98069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:44.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.637+0000 7f9c93fff700 1 -- 172.21.15.31:0/499861208 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f9ca4087040 con 0x7f9cac1550a0 2023-10-06T05:24:44.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.637+0000 7f9cabfff700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c980674d0 0x7f9c98069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:44.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.638+0000 7f9cabfff700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c980674d0 0x7f9c98069980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c007fd0 tx=0x7f9c9c007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:44.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.641+0000 7f9c93fff700 1 -- 172.21.15.31:0/499861208 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ca4053b00 con 0x7f9cac1550a0 2023-10-06T05:24:44.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:44.846+0000 7f9cb23ff700 1 -- 172.21.15.31:0/499861208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9cac0ccaa0 con 0x7f9cac1550a0 2023-10-06T05:24:45.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.301+0000 7f9c93fff700 1 -- 172.21.15.31:0/499861208 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v136) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9ca4057120 con 0x7f9cac1550a0 2023-10-06T05:24:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.306+0000 7f9cb23ff700 1 -- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c980674d0 msgr2=0x7f9c98069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.306+0000 7f9cb23ff700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c980674d0 0x7f9c98069980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c007fd0 tx=0x7f9c9c007eb0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.306+0000 7f9cb23ff700 1 -- 172.21.15.31:0/499861208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac1550a0 msgr2=0x7f9cac1525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.306+0000 7f9cb23ff700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac1550a0 0x7f9cac1525a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0c8920 tx=0x7f9ca4018000 comp rx=0 tx=0).stop 2023-10-06T05:24:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.307+0000 7f9cb23ff700 1 -- 172.21.15.31:0/499861208 shutdown_connections 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.307+0000 7f9cb23ff700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c980674d0 0x7f9c98069980 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.307+0000 7f9cb23ff700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac1550a0 0x7f9cac1525a0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.307+0000 7f9cb23ff700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9cac06ee10 0x7f9cac150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.307+0000 7f9cb23ff700 1 --2- 172.21.15.31:0/499861208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cac154cd0 0x7f9cac159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.307+0000 7f9cb23ff700 1 -- 172.21.15.31:0/499861208 >> 172.21.15.31:0/499861208 conn(0x7f9cac068030 msgr2=0x7f9cac0af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.308+0000 7f9cb23ff700 1 -- 172.21.15.31:0/499861208 shutdown_connections 2023-10-06T05:24:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.308+0000 7f9cb23ff700 1 -- 172.21.15.31:0/499861208 wait complete. 2023-10-06T05:24:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:24:45.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:24:45.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.534+0000 7fde455ff700 1 Processor -- start 2023-10-06T05:24:45.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.534+0000 7fde455ff700 1 -- start start 2023-10-06T05:24:45.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde455ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde400ae860 0x7fde40157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde455ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde40158cf0 con 0x7fde401573a0 2023-10-06T05:24:45.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde455ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde401580e0 con 0x7fde400b1260 2023-10-06T05:24:45.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde455ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde401583c0 con 0x7fde400ae860 2023-10-06T05:24:45.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde3effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fde401573a0 0x7fde400acb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60128/0 (socket says 172.21.15.31:60128) 2023-10-06T05:24:45.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde3effd700 1 -- 172.21.15.31:0/2904907600 learned_addr learned my addr 172.21.15.31:0/2904907600 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:45.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde3e7fc700 1 --2- 172.21.15.31:0/2904907600 >> v2:172.21.15.31:3300/0 conn(0x7fde400ae860 0x7fde40157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde3dffb700 1 -- 172.21.15.31:0/2904907600 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306325926 0 0) 0x7fde40158cf0 con 0x7fde401573a0 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde3dffb700 1 -- 172.21.15.31:0/2904907600 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde28003570 con 0x7fde401573a0 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde3e7fc700 1 -- 172.21.15.31:0/2904907600 >> v1:172.21.15.62:6789/0 conn(0x7fde400b1260 legacy=0x7fde401611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde3e7fc700 1 -- 172.21.15.31:0/2904907600 >> v1:172.21.15.31:6789/0 conn(0x7fde401573a0 legacy=0x7fde400acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.536+0000 7fde3e7fc700 1 -- 172.21.15.31:0/2904907600 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde401586a0 con 0x7fde400ae860 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.537+0000 7fde3e7fc700 1 --2- 172.21.15.31:0/2904907600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400ae860 0x7fde40157770 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fde340098e0 tx=0x7fde340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3633b6c421aca20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.537+0000 7fde3dffb700 1 -- 172.21.15.31:0/2904907600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34011630 con 0x7fde400ae860 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.537+0000 7fde3dffb700 1 -- 172.21.15.31:0/2904907600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde34011790 con 0x7fde400ae860 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.537+0000 7fde3dffb700 1 -- 172.21.15.31:0/2904907600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34011b30 con 0x7fde400ae860 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.537+0000 7fde455ff700 1 -- 172.21.15.31:0/2904907600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400ae860 msgr2=0x7fde40157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.537+0000 7fde455ff700 1 --2- 172.21.15.31:0/2904907600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400ae860 0x7fde40157770 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fde340098e0 tx=0x7fde340091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.538+0000 7fde455ff700 1 -- 172.21.15.31:0/2904907600 shutdown_connections 2023-10-06T05:24:45.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.538+0000 7fde455ff700 1 --2- 172.21.15.31:0/2904907600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400ae860 0x7fde40157770 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.538+0000 7fde455ff700 1 -- 172.21.15.31:0/2904907600 >> 172.21.15.31:0/2904907600 conn(0x7fde40067560 msgr2=0x7fde40067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:45.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.538+0000 7fde455ff700 1 -- 172.21.15.31:0/2904907600 shutdown_connections 2023-10-06T05:24:45.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.538+0000 7fde455ff700 1 -- 172.21.15.31:0/2904907600 wait complete. 2023-10-06T05:24:45.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.539+0000 7fde455ff700 1 Processor -- start 2023-10-06T05:24:45.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.539+0000 7fde455ff700 1 -- start start 2023-10-06T05:24:45.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.540+0000 7fde455ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400b1260 0x7fde400aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.540+0000 7fde455ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde401573a0 0x7fde40142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.540+0000 7fde455ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde401549a0 0x7fde40154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.540+0000 7fde455ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40155360 con 0x7fde400b1260 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.540+0000 7fde455ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde401439e0 con 0x7fde401549a0 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.540+0000 7fde455ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde40142dd0 con 0x7fde401573a0 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.540+0000 7fde3effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400b1260 0x7fde400aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde3effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400b1260 0x7fde400aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53634/0 (socket says 172.21.15.31:53634) 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde3effd700 1 -- 172.21.15.31:0/3770043811 learned_addr learned my addr 172.21.15.31:0/3770043811 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde3e7fc700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde401573a0 0x7fde40142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde3f7fe700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde401549a0 0x7fde40154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde3effd700 1 -- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde401573a0 msgr2=0x7fde40142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde3effd700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde401573a0 0x7fde40142460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde3effd700 1 -- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde401549a0 msgr2=0x7fde40154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:45.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde3effd700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde401549a0 0x7fde40154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:45.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde3effd700 1 -- 172.21.15.31:0/3770043811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34009420 con 0x7fde400b1260 2023-10-06T05:24:45.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde3effd700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400b1260 0x7fde400aabb0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fde3000c7c0 tx=0x7fde30007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.541+0000 7fde26ffd700 1 -- 172.21.15.31:0/3770043811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30005b30 con 0x7fde400b1260 2023-10-06T05:24:45.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.542+0000 7fde26ffd700 1 -- 172.21.15.31:0/3770043811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde30005c90 con 0x7fde400b1260 2023-10-06T05:24:45.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.542+0000 7fde455ff700 1 -- 172.21.15.31:0/3770043811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde401430b0 con 0x7fde400b1260 2023-10-06T05:24:45.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.542+0000 7fde26ffd700 1 -- 172.21.15.31:0/3770043811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30007340 con 0x7fde400b1260 2023-10-06T05:24:45.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.542+0000 7fde455ff700 1 -- 172.21.15.31:0/3770043811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde40143670 con 0x7fde400b1260 2023-10-06T05:24:45.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.543+0000 7fde455ff700 1 -- 172.21.15.31:0/3770043811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4004c380 con 0x7fde400b1260 2023-10-06T05:24:45.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.545+0000 7fde26ffd700 1 -- 172.21.15.31:0/3770043811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fde30025430 con 0x7fde400b1260 2023-10-06T05:24:45.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.545+0000 7fde26ffd700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fde2c0674d0 0x7fde2c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:45.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.546+0000 7fde26ffd700 1 -- 172.21.15.31:0/3770043811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5604+0+0 (secure 0 0 0) 0x7fde300855f0 con 0x7fde400b1260 2023-10-06T05:24:45.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.546+0000 7fde3e7fc700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fde2c0674d0 0x7fde2c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:45.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.547+0000 7fde3e7fc700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fde2c0674d0 0x7fde2c069980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fde340091e0 tx=0x7fde34016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:45.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.550+0000 7fde26ffd700 1 -- 172.21.15.31:0/3770043811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde300521c0 con 0x7fde400b1260 2023-10-06T05:24:45.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:45.753+0000 7fde455ff700 1 -- 172.21.15.31:0/3770043811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fde400b1fb0 con 0x7fde400b1260 2023-10-06T05:24:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.312+0000 7fde26ffd700 1 -- 172.21.15.31:0/3770043811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v137) v1 ==== 170+0+0 (secure 0 0 0) 0x7fde300557e0 con 0x7fde400b1260 2023-10-06T05:24:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.317+0000 7fde455ff700 1 -- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fde2c0674d0 msgr2=0x7fde2c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.317+0000 7fde455ff700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fde2c0674d0 0x7fde2c069980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fde340091e0 tx=0x7fde34016000 comp rx=0 tx=0).stop 2023-10-06T05:24:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.317+0000 7fde455ff700 1 -- 172.21.15.31:0/3770043811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400b1260 msgr2=0x7fde400aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.317+0000 7fde455ff700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400b1260 0x7fde400aabb0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fde3000c7c0 tx=0x7fde30007e00 comp rx=0 tx=0).stop 2023-10-06T05:24:46.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.318+0000 7fde455ff700 1 -- 172.21.15.31:0/3770043811 shutdown_connections 2023-10-06T05:24:46.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.318+0000 7fde455ff700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fde2c0674d0 0x7fde2c069980 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.318+0000 7fde455ff700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde401549a0 0x7fde40154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.318+0000 7fde455ff700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde400b1260 0x7fde400aabb0 secure :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fde3000c7c0 tx=0x7fde30007e00 comp rx=0 tx=0).stop 2023-10-06T05:24:46.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.318+0000 7fde455ff700 1 --2- 172.21.15.31:0/3770043811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fde401573a0 0x7fde40142460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.318+0000 7fde455ff700 1 -- 172.21.15.31:0/3770043811 >> 172.21.15.31:0/3770043811 conn(0x7fde40067560 msgr2=0x7fde400ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:46.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.318+0000 7fde455ff700 1 -- 172.21.15.31:0/3770043811 shutdown_connections 2023-10-06T05:24:46.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.327+0000 7fde455ff700 1 -- 172.21.15.31:0/3770043811 wait complete. 2023-10-06T05:24:46.340 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:24:46.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:24:46.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.554+0000 7fc931bff700 1 Processor -- start 2023-10-06T05:24:46.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.555+0000 7fc931bff700 1 -- start start 2023-10-06T05:24:46.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.556+0000 7fc931bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc92c153d40 0x7fc92c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.556+0000 7fc931bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc92c0b09c0 0x7fc92c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.556+0000 7fc931bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92c0b0e40 con 0x7fc92c0af6c0 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.556+0000 7fc931bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c0ad690 con 0x7fc92c0b09c0 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.556+0000 7fc931bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c0acd60 con 0x7fc92c153d40 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.556+0000 7fc92affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc92c0b09c0 0x7fc92c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.556+0000 7fc92affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc92c0b09c0 0x7fc92c0af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53644/0 (socket says 172.21.15.31:53644) 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.556+0000 7fc92affd700 1 -- 172.21.15.31:0/4014485343 learned_addr learned my addr 172.21.15.31:0/4014485343 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.556+0000 7fc92b7fe700 1 --2- 172.21.15.31:0/4014485343 >> v2:172.21.15.62:3300/0 conn(0x7fc92c153d40 0x7fc92c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.557+0000 7fc92affd700 1 -- 172.21.15.31:0/4014485343 >> v2:172.21.15.62:3300/0 conn(0x7fc92c153d40 msgr2=0x7fc92c0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.557+0000 7fc92affd700 1 --2- 172.21.15.31:0/4014485343 >> v2:172.21.15.62:3300/0 conn(0x7fc92c153d40 0x7fc92c0b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.557+0000 7fc92affd700 1 -- 172.21.15.31:0/4014485343 >> v1:172.21.15.62:6789/0 conn(0x7fc92c0af6c0 legacy=0x7fc92c159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.557+0000 7fc92affd700 1 -- 172.21.15.31:0/4014485343 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c0ad040 con 0x7fc92c0b09c0 2023-10-06T05:24:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.557+0000 7fc92b7fe700 1 --2- 172.21.15.31:0/4014485343 >> v2:172.21.15.62:3300/0 conn(0x7fc92c153d40 0x7fc92c0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:46.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.557+0000 7fc92affd700 1 --2- 172.21.15.31:0/4014485343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 0x7fc92c0af180 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fc92c06d1d0 tx=0x7fc920009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42858f3b9325f1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.557+0000 7fc92a7fc700 1 -- 172.21.15.31:0/4014485343 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92000b860 con 0x7fc92c0b09c0 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.557+0000 7fc92a7fc700 1 -- 172.21.15.31:0/4014485343 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92000b9c0 con 0x7fc92c0b09c0 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.557+0000 7fc92a7fc700 1 -- 172.21.15.31:0/4014485343 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92000bd60 con 0x7fc92c0b09c0 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.559+0000 7fc931bff700 1 -- 172.21.15.31:0/4014485343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 msgr2=0x7fc92c0af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.559+0000 7fc931bff700 1 --2- 172.21.15.31:0/4014485343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 0x7fc92c0af180 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fc92c06d1d0 tx=0x7fc920009250 comp rx=0 tx=0).stop 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.559+0000 7fc931bff700 1 -- 172.21.15.31:0/4014485343 shutdown_connections 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.559+0000 7fc931bff700 1 --2- 172.21.15.31:0/4014485343 >> v2:172.21.15.62:3300/0 conn(0x7fc92c153d40 0x7fc92c0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.559+0000 7fc931bff700 1 --2- 172.21.15.31:0/4014485343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 0x7fc92c0af180 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.559+0000 7fc931bff700 1 -- 172.21.15.31:0/4014485343 >> 172.21.15.31:0/4014485343 conn(0x7fc92c067560 msgr2=0x7fc92c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.559+0000 7fc931bff700 1 -- 172.21.15.31:0/4014485343 shutdown_connections 2023-10-06T05:24:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.559+0000 7fc931bff700 1 -- 172.21.15.31:0/4014485343 wait complete. 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.560+0000 7fc931bff700 1 Processor -- start 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.560+0000 7fc931bff700 1 -- start start 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc931bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc92c0af6c0 0x7fc92c142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc931bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 0x7fc92c142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc931bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc92c153d40 0x7fc92c143280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc931bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c14c960 con 0x7fc92c0b09c0 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc931bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c14bd90 con 0x7fc92c0af6c0 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc931bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c14c030 con 0x7fc92c153d40 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc92affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 0x7fc92c142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc92affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 0x7fc92c142d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53646/0 (socket says 172.21.15.31:53646) 2023-10-06T05:24:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc92affd700 1 -- 172.21.15.31:0/2195441665 learned_addr learned my addr 172.21.15.31:0/2195441665 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc92b7fe700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc92c0af6c0 0x7fc92c142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc92bfff700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc92c153d40 0x7fc92c143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc92affd700 1 -- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc92c153d40 msgr2=0x7fc92c143280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc92affd700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc92c153d40 0x7fc92c143280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc92affd700 1 -- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc92c0af6c0 msgr2=0x7fc92c142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc92affd700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc92c0af6c0 0x7fc92c142800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.562+0000 7fc92affd700 1 -- 172.21.15.31:0/2195441665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9200094f0 con 0x7fc92c0b09c0 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.563+0000 7fc92affd700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 0x7fc92c142d40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc920009290 tx=0x7fc920017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.563+0000 7fc9137fe700 1 -- 172.21.15.31:0/2195441665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920011e60 con 0x7fc92c0b09c0 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.563+0000 7fc9137fe700 1 -- 172.21.15.31:0/2195441665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92001a660 con 0x7fc92c0b09c0 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.563+0000 7fc931bff700 1 -- 172.21.15.31:0/2195441665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c14c310 con 0x7fc92c0b09c0 2023-10-06T05:24:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.563+0000 7fc9137fe700 1 -- 172.21.15.31:0/2195441665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92001aa20 con 0x7fc92c0b09c0 2023-10-06T05:24:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.563+0000 7fc931bff700 1 -- 172.21.15.31:0/2195441665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c142410 con 0x7fc92c0b09c0 2023-10-06T05:24:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.564+0000 7fc931bff700 1 -- 172.21.15.31:0/2195441665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c04c380 con 0x7fc92c0b09c0 2023-10-06T05:24:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.565+0000 7fc9137fe700 1 -- 172.21.15.31:0/2195441665 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fc92002f480 con 0x7fc92c0b09c0 2023-10-06T05:24:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.565+0000 7fc9137fe700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9180674d0 0x7fc918069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.565+0000 7fc9137fe700 1 -- 172.21.15.31:0/2195441665 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5986+0+0 (secure 0 0 0) 0x7fc920086c70 con 0x7fc92c0b09c0 2023-10-06T05:24:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.567+0000 7fc92b7fe700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9180674d0 0x7fc918069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:46.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.567+0000 7fc92b7fe700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9180674d0 0x7fc918069980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc91c006f00 tx=0x7fc91c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:46.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.569+0000 7fc9137fe700 1 -- 172.21.15.31:0/2195441665 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc920053590 con 0x7fc92c0b09c0 2023-10-06T05:24:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:46.822+0000 7fc931bff700 1 -- 172.21.15.31:0/2195441665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc92c14caa0 con 0x7fc92c0b09c0 2023-10-06T05:24:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.318+0000 7fc9137fe700 1 -- 172.21.15.31:0/2195441665 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v89) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc9200150c0 con 0x7fc92c0b09c0 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.329+0000 7fc931bff700 1 -- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9180674d0 msgr2=0x7fc918069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.329+0000 7fc931bff700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9180674d0 0x7fc918069980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc91c006f00 tx=0x7fc91c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.329+0000 7fc931bff700 1 -- 172.21.15.31:0/2195441665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 msgr2=0x7fc92c142d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.329+0000 7fc931bff700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 0x7fc92c142d40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc920009290 tx=0x7fc920017000 comp rx=0 tx=0).stop 2023-10-06T05:24:47.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.331+0000 7fc931bff700 1 -- 172.21.15.31:0/2195441665 shutdown_connections 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.331+0000 7fc931bff700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc9180674d0 0x7fc918069980 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc91c006f00 tx=0x7fc91c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.331+0000 7fc931bff700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc92c153d40 0x7fc92c143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.331+0000 7fc931bff700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc92c0af6c0 0x7fc92c142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.331+0000 7fc931bff700 1 --2- 172.21.15.31:0/2195441665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc92c0b09c0 0x7fc92c142d40 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.331+0000 7fc931bff700 1 -- 172.21.15.31:0/2195441665 >> 172.21.15.31:0/2195441665 conn(0x7fc92c067560 msgr2=0x7fc92c0abbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.340+0000 7fc931bff700 1 -- 172.21.15.31:0/2195441665 shutdown_connections 2023-10-06T05:24:47.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.345+0000 7fc931bff700 1 -- 172.21.15.31:0/2195441665 wait complete. 2023-10-06T05:24:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 23 and data pool 24 2023-10-06T05:24:47.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:47.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.569+0000 7f3c221ff700 1 Processor -- start 2023-10-06T05:24:47.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.570+0000 7f3c221ff700 1 -- start start 2023-10-06T05:24:47.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.571+0000 7f3c221ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3c1c1595a0 0x7f3c1c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.571+0000 7f3c221ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c1c0cc0d0 0x7f3c1c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.571+0000 7f3c221ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c1c069010 con 0x7f3c1c068bb0 2023-10-06T05:24:47.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.571+0000 7f3c221ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1c0c95b0 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.571+0000 7f3c221ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1c0c9890 con 0x7f3c1c1595a0 2023-10-06T05:24:47.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.571+0000 7f3c1affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c1c0cc0d0 0x7f3c1c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.571+0000 7f3c1affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c1c0cc0d0 0x7f3c1c068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55486/0 (socket says 172.21.15.31:55486) 2023-10-06T05:24:47.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.571+0000 7f3c1affd700 1 -- 172.21.15.31:0/3918874862 learned_addr learned my addr 172.21.15.31:0/3918874862 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:47.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.571+0000 7f3c1b7fe700 1 --2- 172.21.15.31:0/3918874862 >> v2:172.21.15.62:3300/0 conn(0x7f3c1c1595a0 0x7f3c1c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.572+0000 7f3c1affd700 1 -- 172.21.15.31:0/3918874862 >> v2:172.21.15.62:3300/0 conn(0x7f3c1c1595a0 msgr2=0x7f3c1c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:47.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.572+0000 7f3c1affd700 1 --2- 172.21.15.31:0/3918874862 >> v2:172.21.15.62:3300/0 conn(0x7f3c1c1595a0 0x7f3c1c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.572+0000 7f3c1affd700 1 -- 172.21.15.31:0/3918874862 >> v1:172.21.15.62:6789/0 conn(0x7f3c1c068bb0 legacy=0x7f3c1c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:47.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.572+0000 7f3c1affd700 1 -- 172.21.15.31:0/3918874862 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c0c9b70 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.572+0000 7f3c1b7fe700 1 --2- 172.21.15.31:0/3918874862 >> v2:172.21.15.62:3300/0 conn(0x7f3c1c1595a0 0x7f3c1c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:47.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.572+0000 7f3c1affd700 1 --2- 172.21.15.31:0/3918874862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 0x7f3c1c068670 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c06c8d0 tx=0x7f3c10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240ea4b8ea86af1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.572+0000 7f3c1a7fc700 1 -- 172.21.15.31:0/3918874862 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000b820 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.572+0000 7f3c1a7fc700 1 -- 172.21.15.31:0/3918874862 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1000b980 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.572+0000 7f3c1a7fc700 1 -- 172.21.15.31:0/3918874862 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000bd20 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.573+0000 7f3c221ff700 1 -- 172.21.15.31:0/3918874862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 msgr2=0x7f3c1c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.573+0000 7f3c221ff700 1 --2- 172.21.15.31:0/3918874862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 0x7f3c1c068670 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c06c8d0 tx=0x7f3c10009250 comp rx=0 tx=0).stop 2023-10-06T05:24:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.573+0000 7f3c221ff700 1 -- 172.21.15.31:0/3918874862 shutdown_connections 2023-10-06T05:24:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.573+0000 7f3c221ff700 1 --2- 172.21.15.31:0/3918874862 >> v2:172.21.15.62:3300/0 conn(0x7f3c1c1595a0 0x7f3c1c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.573+0000 7f3c221ff700 1 --2- 172.21.15.31:0/3918874862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 0x7f3c1c068670 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.573+0000 7f3c221ff700 1 -- 172.21.15.31:0/3918874862 >> 172.21.15.31:0/3918874862 conn(0x7f3c1c06b620 msgr2=0x7f3c1c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.573+0000 7f3c221ff700 1 -- 172.21.15.31:0/3918874862 shutdown_connections 2023-10-06T05:24:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.573+0000 7f3c221ff700 1 -- 172.21.15.31:0/3918874862 wait complete. 2023-10-06T05:24:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.575+0000 7f3c221ff700 1 Processor -- start 2023-10-06T05:24:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.575+0000 7f3c221ff700 1 -- start start 2023-10-06T05:24:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.576+0000 7f3c221ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c1c068bb0 0x7f3c1c14e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.576+0000 7f3c221ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 0x7f3c1c158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.576+0000 7f3c221ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c1595a0 0x7f3c1c158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.576+0000 7f3c221ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c169ae0 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.576+0000 7f3c221ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c152a00 con 0x7f3c1c1595a0 2023-10-06T05:24:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.576+0000 7f3c221ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c1504e0 con 0x7f3c1c068bb0 2023-10-06T05:24:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 0x7f3c1c158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 0x7f3c1c158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55488/0 (socket says 172.21.15.31:55488) 2023-10-06T05:24:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1affd700 1 -- 172.21.15.31:0/3432814981 learned_addr learned my addr 172.21.15.31:0/3432814981 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1b7fe700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c1c068bb0 0x7f3c1c14e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1bfff700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c1595a0 0x7f3c1c158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1affd700 1 -- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c1c068bb0 msgr2=0x7f3c1c14e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1affd700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c1c068bb0 0x7f3c1c14e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1affd700 1 -- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c1595a0 msgr2=0x7f3c1c158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1affd700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c1595a0 0x7f3c1c158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1affd700 1 -- 172.21.15.31:0/3432814981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c100094f0 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1bfff700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c1595a0 0x7f3c1c158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.577+0000 7f3c1affd700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 0x7f3c1c158140 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c06ee00 tx=0x7f3c10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.578+0000 7f3c037fe700 1 -- 172.21.15.31:0/3432814981 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10011e20 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.578+0000 7f3c037fe700 1 -- 172.21.15.31:0/3432814981 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10020070 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.578+0000 7f3c221ff700 1 -- 172.21.15.31:0/3432814981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1c151df0 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.578+0000 7f3c037fe700 1 -- 172.21.15.31:0/3432814981 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1001b6a0 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.578+0000 7f3c221ff700 1 -- 172.21.15.31:0/3432814981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1c152180 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.579+0000 7f3c221ff700 1 -- 172.21.15.31:0/3432814981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1c04c380 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.580+0000 7f3c037fe700 1 -- 172.21.15.31:0/3432814981 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f3c10022030 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.581+0000 7f3c037fe700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c080674d0 0x7f3c08069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.581+0000 7f3c037fe700 1 -- 172.21.15.31:0/3432814981 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f3c10086f30 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.581+0000 7f3c1b7fe700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c080674d0 0x7f3c08069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.582+0000 7f3c1b7fe700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c080674d0 0x7f3c08069980 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00ac00 tx=0x7f3c0c006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:47.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.585+0000 7f3c037fe700 1 -- 172.21.15.31:0/3432814981 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c100537b0 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.787+0000 7f3c221ff700 1 -- 172.21.15.31:0/3432814981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c1c069010 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.788+0000 7f3c037fe700 1 -- 172.21.15.31:0/3432814981 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16985 (secure 0 0 0) 0x7f3c1c069010 con 0x7f3c1c0cc0d0 2023-10-06T05:24:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:47.790 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":138,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:46.824301+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:44.887621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:45.794431+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 -- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c080674d0 msgr2=0x7f3c08069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c080674d0 0x7f3c08069980 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00ac00 tx=0x7f3c0c006f90 comp rx=0 tx=0).stop 2023-10-06T05:24:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 -- 172.21.15.31:0/3432814981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 msgr2=0x7f3c1c158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 0x7f3c1c158140 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c06ee00 tx=0x7f3c10016000 comp rx=0 tx=0).stop 2023-10-06T05:24:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 -- 172.21.15.31:0/3432814981 shutdown_connections 2023-10-06T05:24:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3c080674d0 0x7f3c08069980 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c1595a0 0x7f3c1c158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c1c068bb0 0x7f3c1c14e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 --2- 172.21.15.31:0/3432814981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c1c0cc0d0 0x7f3c1c158140 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 -- 172.21.15.31:0/3432814981 >> 172.21.15.31:0/3432814981 conn(0x7f3c1c06b620 msgr2=0x7f3c1c14ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.790+0000 7f3c221ff700 1 -- 172.21.15.31:0/3432814981 shutdown_connections 2023-10-06T05:24:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:47.791+0000 7f3c221ff700 1 -- 172.21.15.31:0/3432814981 wait complete. 2023-10-06T05:24:47.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.004+0000 7f31ed1ff700 1 Processor -- start 2023-10-06T05:24:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.004+0000 7f31ed1ff700 1 -- start start 2023-10-06T05:24:48.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.005+0000 7f31ed1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f31e806a570 0x7f31e8162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.005+0000 7f31ed1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e80c71b0 con 0x7f31e813a5b0 2023-10-06T05:24:48.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.005+0000 7f31ed1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e80c7490 con 0x7f31e813a1e0 2023-10-06T05:24:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.005+0000 7f31ed1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e80c7770 con 0x7f31e806a570 2023-10-06T05:24:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.006+0000 7f31e759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f31e806a570 0x7f31e8162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.006+0000 7f31e659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f31e813a5b0 0x7f31e806d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42310/0 (socket says 172.21.15.31:42310) 2023-10-06T05:24:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.006+0000 7f31e659c700 1 -- 172.21.15.31:0/3292679562 learned_addr learned my addr 172.21.15.31:0/3292679562 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.006+0000 7f31e759e700 1 -- 172.21.15.31:0/3292679562 >> v1:172.21.15.62:6789/0 conn(0x7f31e813a1e0 legacy=0x7f31e80c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.006+0000 7f31e759e700 1 -- 172.21.15.31:0/3292679562 >> v1:172.21.15.31:6789/0 conn(0x7f31e813a5b0 legacy=0x7f31e806d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.006+0000 7f31e759e700 1 -- 172.21.15.31:0/3292679562 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e80c7a50 con 0x7f31e806a570 2023-10-06T05:24:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.006+0000 7f31e5d9b700 1 -- 172.21.15.31:0/3292679562 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406648169 0 0) 0x7f31e80c71b0 con 0x7f31e813a5b0 2023-10-06T05:24:48.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.006+0000 7f31e759e700 1 --2- 172.21.15.31:0/3292679562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e806a570 0x7f31e8162350 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009e60 tx=0x7f31e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42dcdd3e5936f3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.007+0000 7f31e5d9b700 1 -- 172.21.15.31:0/3292679562 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e000e070 con 0x7f31e806a570 2023-10-06T05:24:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.007+0000 7f31e5d9b700 1 -- 172.21.15.31:0/3292679562 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e000bc30 con 0x7f31e806a570 2023-10-06T05:24:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.007+0000 7f31e5d9b700 1 -- 172.21.15.31:0/3292679562 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0012430 con 0x7f31e806a570 2023-10-06T05:24:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.007+0000 7f31ed1ff700 1 -- 172.21.15.31:0/3292679562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e806a570 msgr2=0x7f31e8162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.007+0000 7f31ed1ff700 1 --2- 172.21.15.31:0/3292679562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e806a570 0x7f31e8162350 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009e60 tx=0x7f31e0009780 comp rx=0 tx=0).stop 2023-10-06T05:24:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.007+0000 7f31ed1ff700 1 -- 172.21.15.31:0/3292679562 shutdown_connections 2023-10-06T05:24:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.007+0000 7f31ed1ff700 1 --2- 172.21.15.31:0/3292679562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e806a570 0x7f31e8162350 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.007+0000 7f31ed1ff700 1 -- 172.21.15.31:0/3292679562 >> 172.21.15.31:0/3292679562 conn(0x7f31e80cc0b0 msgr2=0x7f31e80cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.007+0000 7f31ed1ff700 1 -- 172.21.15.31:0/3292679562 shutdown_connections 2023-10-06T05:24:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.007+0000 7f31ed1ff700 1 -- 172.21.15.31:0/3292679562 wait complete. 2023-10-06T05:24:48.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.009+0000 7f31ed1ff700 1 Processor -- start 2023-10-06T05:24:48.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.009+0000 7f31ed1ff700 1 -- start start 2023-10-06T05:24:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.010+0000 7f31ed1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31e806a570 0x7f31e8154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.010+0000 7f31ed1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e813a1e0 0x7f31e80c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.010+0000 7f31ed1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e813a5b0 0x7f31e8152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.010+0000 7f31ed1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e8153040 con 0x7f31e813a1e0 2023-10-06T05:24:48.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.010+0000 7f31ed1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80c4640 con 0x7f31e813a5b0 2023-10-06T05:24:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.010+0000 7f31ed1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e8152a80 con 0x7f31e806a570 2023-10-06T05:24:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.010+0000 7f31e659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e813a1e0 0x7f31e80c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.010+0000 7f31e659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e813a1e0 0x7f31e80c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55498/0 (socket says 172.21.15.31:55498) 2023-10-06T05:24:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.010+0000 7f31e659c700 1 -- 172.21.15.31:0/2663579233 learned_addr learned my addr 172.21.15.31:0/2663579233 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.010+0000 7f31e6d9d700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31e806a570 0x7f31e8154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31e759e700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e813a5b0 0x7f31e8152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31e659c700 1 -- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31e806a570 msgr2=0x7f31e8154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31e659c700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31e806a570 0x7f31e8154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31e659c700 1 -- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e813a5b0 msgr2=0x7f31e8152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31e659c700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e813a5b0 0x7f31e8152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31e659c700 1 -- 172.21.15.31:0/2663579233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e0009a20 con 0x7f31e813a1e0 2023-10-06T05:24:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31e659c700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e813a1e0 0x7f31e80c4ad0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c5e0 tx=0x7f31dc000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31ceffd700 1 -- 172.21.15.31:0/2663579233 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc005b30 con 0x7f31e813a1e0 2023-10-06T05:24:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31ceffd700 1 -- 172.21.15.31:0/2663579233 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc005c90 con 0x7f31e813a1e0 2023-10-06T05:24:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31ceffd700 1 -- 172.21.15.31:0/2663579233 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc007300 con 0x7f31e813a1e0 2023-10-06T05:24:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31ed1ff700 1 -- 172.21.15.31:0/2663579233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e8152d60 con 0x7f31e813a1e0 2023-10-06T05:24:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.011+0000 7f31ed1ff700 1 -- 172.21.15.31:0/2663579233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e8154a80 con 0x7f31e813a1e0 2023-10-06T05:24:48.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.012+0000 7f31ed1ff700 1 -- 172.21.15.31:0/2663579233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e80c4130 con 0x7f31e813a1e0 2023-10-06T05:24:48.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.013+0000 7f31ceffd700 1 -- 172.21.15.31:0/2663579233 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f31dc0076b0 con 0x7f31e813a1e0 2023-10-06T05:24:48.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.014+0000 7f31ceffd700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31d40675a0 0x7f31d4069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.014+0000 7f31ceffd700 1 -- 172.21.15.31:0/2663579233 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f31dc084670 con 0x7f31e813a1e0 2023-10-06T05:24:48.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.015+0000 7f31e6d9d700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31d40675a0 0x7f31d4069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.015+0000 7f31e6d9d700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31d40675a0 0x7f31d4069a50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31d8000c00 tx=0x7f31d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.018+0000 7f31ceffd700 1 -- 172.21.15.31:0/2663579233 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31e80c4130 con 0x7f31e813a1e0 2023-10-06T05:24:48.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.220+0000 7f31ed1ff700 1 -- 172.21.15.31:0/2663579233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31e80c4130 con 0x7f31e813a1e0 2023-10-06T05:24:48.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.221+0000 7f31ceffd700 1 -- 172.21.15.31:0/2663579233 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16985 (secure 0 0 0) 0x7f31e80c4130 con 0x7f31e813a1e0 2023-10-06T05:24:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:48.223 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":138,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:46.824301+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:44.887621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:45.794431+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 -- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31d40675a0 msgr2=0x7f31d4069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31d40675a0 0x7f31d4069a50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31d8000c00 tx=0x7f31d8009040 comp rx=0 tx=0).stop 2023-10-06T05:24:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 -- 172.21.15.31:0/2663579233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e813a1e0 msgr2=0x7f31e80c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e813a1e0 0x7f31e80c4ad0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c5e0 tx=0x7f31dc000fe0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 -- 172.21.15.31:0/2663579233 shutdown_connections 2023-10-06T05:24:48.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f31d40675a0 0x7f31d4069a50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e813a5b0 0x7f31e8152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31e806a570 0x7f31e8154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 --2- 172.21.15.31:0/2663579233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e813a1e0 0x7f31e80c4ad0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 -- 172.21.15.31:0/2663579233 >> 172.21.15.31:0/2663579233 conn(0x7f31e80cc0b0 msgr2=0x7f31e80c0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 -- 172.21.15.31:0/2663579233 shutdown_connections 2023-10-06T05:24:48.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.223+0000 7f31ed1ff700 1 -- 172.21.15.31:0/2663579233 wait complete. 2023-10-06T05:24:48.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:24:48.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.442+0000 7f0bb19ff700 1 Processor -- start 2023-10-06T05:24:48.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.442+0000 7f0bb19ff700 1 -- start start 2023-10-06T05:24:48.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.444+0000 7f0bb19ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0bac0cc340 0x7f0bac0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.444+0000 7f0bb19ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bac15a020 con 0x7f0bac15a4b0 2023-10-06T05:24:48.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.444+0000 7f0bb19ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bac159cf0 con 0x7f0bac0cc710 2023-10-06T05:24:48.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.444+0000 7f0bb19ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bac0b07e0 con 0x7f0bac0cc340 2023-10-06T05:24:48.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.444+0000 7f0bab7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0bac15a4b0 0x7f0bac0aecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42324/0 (socket says 172.21.15.31:42324) 2023-10-06T05:24:48.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.444+0000 7f0bab7fe700 1 -- 172.21.15.31:0/2634894072 learned_addr learned my addr 172.21.15.31:0/2634894072 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:48.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.445+0000 7f0baaffd700 1 --2- 172.21.15.31:0/2634894072 >> v2:172.21.15.62:3300/0 conn(0x7f0bac0cc340 0x7f0bac0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.445+0000 7f0ba9ffb700 1 -- 172.21.15.31:0/2634894072 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467971267 0 0) 0x7f0bac15a020 con 0x7f0bac15a4b0 2023-10-06T05:24:48.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.445+0000 7f0ba9ffb700 1 -- 172.21.15.31:0/2634894072 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b94003570 con 0x7f0bac15a4b0 2023-10-06T05:24:48.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.445+0000 7f0ba9ffb700 1 -- 172.21.15.31:0/2634894072 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 604960304 0 0) 0x7f0b94003570 con 0x7f0bac15a4b0 2023-10-06T05:24:48.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.445+0000 7f0baaffd700 1 -- 172.21.15.31:0/2634894072 >> v1:172.21.15.62:6789/0 conn(0x7f0bac0cc710 legacy=0x7f0bac0b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.445+0000 7f0baaffd700 1 -- 172.21.15.31:0/2634894072 >> v1:172.21.15.31:6789/0 conn(0x7f0bac15a4b0 legacy=0x7f0bac0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.445+0000 7f0baaffd700 1 -- 172.21.15.31:0/2634894072 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac0ccb90 con 0x7f0bac0cc340 2023-10-06T05:24:48.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.446+0000 7f0baaffd700 1 --2- 172.21.15.31:0/2634894072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac0cc340 0x7f0bac0b2540 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009900 tx=0x7f0b9c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=318c6a8d2d1da9e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.446+0000 7f0ba9ffb700 1 -- 172.21.15.31:0/2634894072 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40017f0 con 0x7f0bac0cc340 2023-10-06T05:24:48.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.446+0000 7f0ba9ffb700 1 -- 172.21.15.31:0/2634894072 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c011420 con 0x7f0bac0cc340 2023-10-06T05:24:48.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.446+0000 7f0ba9ffb700 1 -- 172.21.15.31:0/2634894072 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c0117c0 con 0x7f0bac0cc340 2023-10-06T05:24:48.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.446+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2634894072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac0cc340 msgr2=0x7f0bac0b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.446+0000 7f0bb19ff700 1 --2- 172.21.15.31:0/2634894072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac0cc340 0x7f0bac0b2540 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009900 tx=0x7f0b9c009120 comp rx=0 tx=0).stop 2023-10-06T05:24:48.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.446+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2634894072 shutdown_connections 2023-10-06T05:24:48.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.446+0000 7f0bb19ff700 1 --2- 172.21.15.31:0/2634894072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac0cc340 0x7f0bac0b2540 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.446+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2634894072 >> 172.21.15.31:0/2634894072 conn(0x7f0bac068670 msgr2=0x7f0bac068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:48.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.447+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2634894072 shutdown_connections 2023-10-06T05:24:48.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.447+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2634894072 wait complete. 2023-10-06T05:24:48.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.448+0000 7f0bb19ff700 1 Processor -- start 2023-10-06T05:24:48.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.448+0000 7f0bb19ff700 1 -- start start 2023-10-06T05:24:48.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.449+0000 7f0bb19ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac0cc340 0x7f0bac13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.449+0000 7f0bb19ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bac0cc710 0x7f0bac0ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.449+0000 7f0bb19ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac15a4b0 0x7f0bac16ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.449+0000 7f0bb19ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac140270 con 0x7f0bac0cc340 2023-10-06T05:24:48.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.449+0000 7f0bb19ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0adbb0 con 0x7f0bac15a4b0 2023-10-06T05:24:48.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.449+0000 7f0bb19ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac151f00 con 0x7f0bac0cc710 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.449+0000 7f0baaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac0cc340 0x7f0bac13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.449+0000 7f0baaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac0cc340 0x7f0bac13fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55508/0 (socket says 172.21.15.31:55508) 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.450+0000 7f0baaffd700 1 -- 172.21.15.31:0/2054126344 learned_addr learned my addr 172.21.15.31:0/2054126344 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.450+0000 7f0bab7fe700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac15a4b0 0x7f0bac16ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.450+0000 7f0baa7fc700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bac0cc710 0x7f0bac0ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.450+0000 7f0baaffd700 1 -- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bac0cc710 msgr2=0x7f0bac0ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.450+0000 7f0baaffd700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bac0cc710 0x7f0bac0ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.450+0000 7f0baaffd700 1 -- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac15a4b0 msgr2=0x7f0bac16ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.450+0000 7f0baaffd700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac15a4b0 0x7f0bac16ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.450+0000 7f0baaffd700 1 -- 172.21.15.31:0/2054126344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0093c0 con 0x7f0bac0cc340 2023-10-06T05:24:48.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.450+0000 7f0baa7fc700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bac0cc710 0x7f0bac0ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:48.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.450+0000 7f0baaffd700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac0cc340 0x7f0bac13fd30 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f0bac06dd80 tx=0x7f0b9c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.451+0000 7f0b92ffd700 1 -- 172.21.15.31:0/2054126344 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c01bc60 con 0x7f0bac0cc340 2023-10-06T05:24:48.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.451+0000 7f0b92ffd700 1 -- 172.21.15.31:0/2054126344 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c01be00 con 0x7f0bac0cc340 2023-10-06T05:24:48.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.451+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2054126344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac152a80 con 0x7f0bac0cc340 2023-10-06T05:24:48.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.451+0000 7f0b92ffd700 1 -- 172.21.15.31:0/2054126344 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c01c6a0 con 0x7f0bac0cc340 2023-10-06T05:24:48.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.451+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2054126344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac152200 con 0x7f0bac0cc340 2023-10-06T05:24:48.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.452+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2054126344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bac152830 con 0x7f0bac0cc340 2023-10-06T05:24:48.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.453+0000 7f0b92ffd700 1 -- 172.21.15.31:0/2054126344 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f0b9c01ca10 con 0x7f0bac0cc340 2023-10-06T05:24:48.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.454+0000 7f0b92ffd700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b980674d0 0x7f0b98069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:48.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.454+0000 7f0b92ffd700 1 -- 172.21.15.31:0/2054126344 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f0b9c086580 con 0x7f0bac0cc340 2023-10-06T05:24:48.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.454+0000 7f0baa7fc700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b980674d0 0x7f0b98069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:48.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.455+0000 7f0baa7fc700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b980674d0 0x7f0b98069980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0007fd0 tx=0x7f0ba0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.458+0000 7f0b92ffd700 1 -- 172.21.15.31:0/2054126344 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b9c052e00 con 0x7f0bac0cc340 2023-10-06T05:24:48.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:48.709+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2054126344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0bac04c380 con 0x7f0bac0cc340 2023-10-06T05:24:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.335+0000 7f0b92ffd700 1 -- 172.21.15.31:0/2054126344 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v92) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0b9c056420 con 0x7f0bac0cc340 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.338+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b980674d0 msgr2=0x7f0b98069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.338+0000 7f0bb19ff700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b980674d0 0x7f0b98069980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0007fd0 tx=0x7f0ba0009040 comp rx=0 tx=0).stop 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.338+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2054126344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac0cc340 msgr2=0x7f0bac13fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.338+0000 7f0bb19ff700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac0cc340 0x7f0bac13fd30 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f0bac06dd80 tx=0x7f0b9c016000 comp rx=0 tx=0).stop 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.339+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2054126344 shutdown_connections 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.339+0000 7f0bb19ff700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0b980674d0 0x7f0b98069980 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.339+0000 7f0bb19ff700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac15a4b0 0x7f0bac16ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.339+0000 7f0bb19ff700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac0cc340 0x7f0bac13fd30 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.339+0000 7f0bb19ff700 1 --2- 172.21.15.31:0/2054126344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bac0cc710 0x7f0bac0ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.339+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2054126344 >> 172.21.15.31:0/2054126344 conn(0x7f0bac068670 msgr2=0x7f0bac1549e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:49.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.339+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2054126344 shutdown_connections 2023-10-06T05:24:49.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.339+0000 7f0bb19ff700 1 -- 172.21.15.31:0/2054126344 wait complete. 2023-10-06T05:24:49.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:24:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.561+0000 7ffbe33ff700 1 Processor -- start 2023-10-06T05:24:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.562+0000 7ffbe33ff700 1 -- start start 2023-10-06T05:24:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.563+0000 7ffbe33ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffbdc159c60 0x7ffbdc0c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 0x7ffbdc1558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe33ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc158190 con 0x7ffbdc159890 2023-10-06T05:24:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe33ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbdc157580 con 0x7ffbdc159c60 2023-10-06T05:24:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc157860 con 0x7ffbdc155430 2023-10-06T05:24:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe099a700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffbdc159c60 0x7ffbdc0c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe199c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 0x7ffbdc1558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe199c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 0x7ffbdc1558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:40796/0 (socket says 172.21.15.31:40796) 2023-10-06T05:24:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe199c700 1 -- 172.21.15.31:0/3041811829 learned_addr learned my addr 172.21.15.31:0/3041811829 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe119b700 1 --1- 172.21.15.31:0/3041811829 >> v1:172.21.15.62:6789/0 conn(0x7ffbdc159890 0x7ffbdc156a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:46578/0 (socket says 172.21.15.31:46578) 2023-10-06T05:24:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe099a700 1 -- 172.21.15.31:0/3041811829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 msgr2=0x7ffbdc1558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe099a700 1 --2- 172.21.15.31:0/3041811829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 0x7ffbdc1558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe099a700 1 -- 172.21.15.31:0/3041811829 >> v1:172.21.15.62:6789/0 conn(0x7ffbdc159890 legacy=0x7ffbdc156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.564+0000 7ffbe099a700 1 -- 172.21.15.31:0/3041811829 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc157b40 con 0x7ffbdc159c60 2023-10-06T05:24:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.565+0000 7ffbe099a700 1 --2- 172.21.15.31:0/3041811829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc159c60 0x7ffbdc0c89e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0098e0 tx=0x7ffbcc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185a9f9ee9e0db98 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.565+0000 7ffbd3fff700 1 -- 172.21.15.31:0/3041811829 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00b640 con 0x7ffbdc159c60 2023-10-06T05:24:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.565+0000 7ffbd3fff700 1 -- 172.21.15.31:0/3041811829 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc00b7a0 con 0x7ffbdc159c60 2023-10-06T05:24:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.565+0000 7ffbd3fff700 1 -- 172.21.15.31:0/3041811829 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00bb40 con 0x7ffbdc159c60 2023-10-06T05:24:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.566+0000 7ffbe33ff700 1 -- 172.21.15.31:0/3041811829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc159c60 msgr2=0x7ffbdc0c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.566+0000 7ffbe33ff700 1 --2- 172.21.15.31:0/3041811829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc159c60 0x7ffbdc0c89e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0098e0 tx=0x7ffbcc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.566+0000 7ffbe33ff700 1 -- 172.21.15.31:0/3041811829 shutdown_connections 2023-10-06T05:24:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.566+0000 7ffbe33ff700 1 --2- 172.21.15.31:0/3041811829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 0x7ffbdc1558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.566+0000 7ffbe33ff700 1 --2- 172.21.15.31:0/3041811829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc159c60 0x7ffbdc0c89e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.566+0000 7ffbe33ff700 1 -- 172.21.15.31:0/3041811829 >> 172.21.15.31:0/3041811829 conn(0x7ffbdc067560 msgr2=0x7ffbdc067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.566+0000 7ffbe33ff700 1 -- 172.21.15.31:0/3041811829 shutdown_connections 2023-10-06T05:24:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.566+0000 7ffbe33ff700 1 -- 172.21.15.31:0/3041811829 wait complete. 2023-10-06T05:24:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.567+0000 7ffbe33ff700 1 Processor -- start 2023-10-06T05:24:49.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.568+0000 7ffbe33ff700 1 -- start start 2023-10-06T05:24:49.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.568+0000 7ffbe33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 0x7ffbdc14b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe33ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbdc159890 0x7ffbdc15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe33ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc159c60 0x7ffbdc15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe33ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc14b630 con 0x7ffbdc159890 2023-10-06T05:24:49.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe33ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc14c310 con 0x7ffbdc159c60 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc14b9e0 con 0x7ffbdc155430 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbdc159890 0x7ffbdc15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe119b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 0x7ffbdc14b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe099a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbdc159890 0x7ffbdc15aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55518/0 (socket says 172.21.15.31:55518) 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe099a700 1 -- 172.21.15.31:0/1282042747 learned_addr learned my addr 172.21.15.31:0/1282042747 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe199c700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc159c60 0x7ffbdc15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe099a700 1 -- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 msgr2=0x7ffbdc14b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe099a700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 0x7ffbdc14b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe099a700 1 -- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc159c60 msgr2=0x7ffbdc15b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe099a700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc159c60 0x7ffbdc15b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:49.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.569+0000 7ffbe099a700 1 -- 172.21.15.31:0/1282042747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc009420 con 0x7ffbdc159890 2023-10-06T05:24:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.570+0000 7ffbe099a700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbdc159890 0x7ffbdc15aeb0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00a010 tx=0x7ffbcc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.570+0000 7ffbd17fa700 1 -- 172.21.15.31:0/1282042747 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc01bc40 con 0x7ffbdc159890 2023-10-06T05:24:49.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.570+0000 7ffbd17fa700 1 -- 172.21.15.31:0/1282042747 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc01bde0 con 0x7ffbdc159890 2023-10-06T05:24:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.570+0000 7ffbd17fa700 1 -- 172.21.15.31:0/1282042747 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc011670 con 0x7ffbdc159890 2023-10-06T05:24:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.570+0000 7ffbe33ff700 1 -- 172.21.15.31:0/1282042747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc14bcc0 con 0x7ffbdc159890 2023-10-06T05:24:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.570+0000 7ffbe33ff700 1 -- 172.21.15.31:0/1282042747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc14c7a0 con 0x7ffbdc159890 2023-10-06T05:24:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.571+0000 7ffbe33ff700 1 -- 172.21.15.31:0/1282042747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbdc14c4a0 con 0x7ffbdc159890 2023-10-06T05:24:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.572+0000 7ffbd17fa700 1 -- 172.21.15.31:0/1282042747 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ffbcc011a20 con 0x7ffbdc159890 2023-10-06T05:24:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.573+0000 7ffbd17fa700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffbc00674d0 0x7ffbc0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.573+0000 7ffbd17fa700 1 -- 172.21.15.31:0/1282042747 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ffbcc0867c0 con 0x7ffbdc159890 2023-10-06T05:24:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.573+0000 7ffbe119b700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffbc00674d0 0x7ffbc0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:49.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.574+0000 7ffbe119b700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffbc00674d0 0x7ffbc0069980 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0aeaf0 tx=0x7ffbd8007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.577+0000 7ffbd17fa700 1 -- 172.21.15.31:0/1282042747 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbcc053000 con 0x7ffbdc159890 2023-10-06T05:24:49.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:49.827+0000 7ffbe33ff700 1 -- 172.21.15.31:0/1282042747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ffbdc04c380 con 0x7ffbdc159890 2023-10-06T05:24:50.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.341+0000 7ffbd17fa700 1 -- 172.21.15.31:0/1282042747 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v93) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffbcc056620 con 0x7ffbdc159890 2023-10-06T05:24:50.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 -- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffbc00674d0 msgr2=0x7ffbc0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffbc00674d0 0x7ffbc0069980 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0aeaf0 tx=0x7ffbd8007eb0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 -- 172.21.15.31:0/1282042747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbdc159890 msgr2=0x7ffbdc15aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbdc159890 0x7ffbdc15aeb0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00a010 tx=0x7ffbcc017000 comp rx=0 tx=0).stop 2023-10-06T05:24:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 -- 172.21.15.31:0/1282042747 shutdown_connections 2023-10-06T05:24:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffbc00674d0 0x7ffbc0069980 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc159c60 0x7ffbdc15b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc155430 0x7ffbdc14b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 --2- 172.21.15.31:0/1282042747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbdc159890 0x7ffbdc15aeb0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 -- 172.21.15.31:0/1282042747 >> 172.21.15.31:0/1282042747 conn(0x7ffbdc067560 msgr2=0x7ffbdc06d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 -- 172.21.15.31:0/1282042747 shutdown_connections 2023-10-06T05:24:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.344+0000 7ffbe33ff700 1 -- 172.21.15.31:0/1282042747 wait complete. 2023-10-06T05:24:50.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:50.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.560+0000 7f7b7c7ff700 1 Processor -- start 2023-10-06T05:24:50.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.561+0000 7f7b7c7ff700 1 -- start start 2023-10-06T05:24:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.562+0000 7f7b7c7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b7413a5b0 0x7f7b7406d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.562+0000 7f7b7c7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 0x7f7b74069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.562+0000 7f7b7c7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7406ddb0 con 0x7f7b7413a1e0 2023-10-06T05:24:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.562+0000 7f7b7c7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b740c71b0 con 0x7f7b7413a5b0 2023-10-06T05:24:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.562+0000 7f7b7c7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740c7490 con 0x7f7b7406d980 2023-10-06T05:24:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.562+0000 7f7b79d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b7413a5b0 0x7f7b7406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.562+0000 7f7b79d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b7413a5b0 0x7f7b7406d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55530/0 (socket says 172.21.15.31:55530) 2023-10-06T05:24:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.562+0000 7f7b79d9a700 1 -- 172.21.15.31:0/476658428 learned_addr learned my addr 172.21.15.31:0/476658428 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.562+0000 7f7b7ad9c700 1 --2- 172.21.15.31:0/476658428 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 0x7f7b74069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.563+0000 7f7b79d9a700 1 -- 172.21.15.31:0/476658428 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 msgr2=0x7f7b74069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.563+0000 7f7b79d9a700 1 --2- 172.21.15.31:0/476658428 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 0x7f7b74069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.563+0000 7f7b79d9a700 1 -- 172.21.15.31:0/476658428 >> v1:172.21.15.62:6789/0 conn(0x7f7b7413a1e0 legacy=0x7f7b740c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.563+0000 7f7b79d9a700 1 -- 172.21.15.31:0/476658428 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b740c7770 con 0x7f7b7413a5b0 2023-10-06T05:24:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.563+0000 7f7b79d9a700 1 --2- 172.21.15.31:0/476658428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a5b0 0x7f7b7406d440 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7b640099b0 tx=0x7f7b64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c9199bfc3defd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.563+0000 7f7b79599700 1 -- 172.21.15.31:0/476658428 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400b860 con 0x7f7b7413a5b0 2023-10-06T05:24:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.563+0000 7f7b79599700 1 -- 172.21.15.31:0/476658428 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6400b9c0 con 0x7f7b7413a5b0 2023-10-06T05:24:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.563+0000 7f7b79599700 1 -- 172.21.15.31:0/476658428 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400bd60 con 0x7f7b7413a5b0 2023-10-06T05:24:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.564+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/476658428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a5b0 msgr2=0x7f7b7406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.564+0000 7f7b7c7ff700 1 --2- 172.21.15.31:0/476658428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a5b0 0x7f7b7406d440 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7b640099b0 tx=0x7f7b64009250 comp rx=0 tx=0).stop 2023-10-06T05:24:50.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.564+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/476658428 shutdown_connections 2023-10-06T05:24:50.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.564+0000 7f7b7c7ff700 1 --2- 172.21.15.31:0/476658428 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 0x7f7b74069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.564+0000 7f7b7c7ff700 1 --2- 172.21.15.31:0/476658428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a5b0 0x7f7b7406d440 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.564+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/476658428 >> 172.21.15.31:0/476658428 conn(0x7f7b740cc0b0 msgr2=0x7f7b740cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:50.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.564+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/476658428 shutdown_connections 2023-10-06T05:24:50.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.564+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/476658428 wait complete. 2023-10-06T05:24:50.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.566+0000 7f7b7c7ff700 1 Processor -- start 2023-10-06T05:24:50.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.566+0000 7f7b7c7ff700 1 -- start start 2023-10-06T05:24:50.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b7c7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 0x7f7b741398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b7c7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a1e0 0x7f7b740c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b7c7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b7413a5b0 0x7f7b740c1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b7c7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7413d5b0 con 0x7f7b7413a1e0 2023-10-06T05:24:50.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b7c7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74139e30 con 0x7f7b7413a5b0 2023-10-06T05:24:50.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b7c7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741581d0 con 0x7f7b7406d980 2023-10-06T05:24:50.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b79d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a1e0 0x7f7b740c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b79d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a1e0 0x7f7b740c0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55534/0 (socket says 172.21.15.31:55534) 2023-10-06T05:24:50.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b79d9a700 1 -- 172.21.15.31:0/1538824587 learned_addr learned my addr 172.21.15.31:0/1538824587 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:50.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b7ad9c700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b7413a5b0 0x7f7b740c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b7a59b700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 0x7f7b741398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b79d9a700 1 -- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 msgr2=0x7f7b741398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b79d9a700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 0x7f7b741398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b79d9a700 1 -- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b7413a5b0 msgr2=0x7f7b740c1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b79d9a700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b7413a5b0 0x7f7b740c1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b79d9a700 1 -- 172.21.15.31:0/1538824587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b640094f0 con 0x7f7b7413a1e0 2023-10-06T05:24:50.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.567+0000 7f7b7a59b700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 0x7f7b741398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:50.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.568+0000 7f7b79d9a700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a1e0 0x7f7b740c0e50 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009290 tx=0x7f7b64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.568+0000 7f7b6a7fc700 1 -- 172.21.15.31:0/1538824587 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64024440 con 0x7f7b7413a1e0 2023-10-06T05:24:50.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.568+0000 7f7b6a7fc700 1 -- 172.21.15.31:0/1538824587 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b640245c0 con 0x7f7b7413a1e0 2023-10-06T05:24:50.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.568+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/1538824587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b741575c0 con 0x7f7b7413a1e0 2023-10-06T05:24:50.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.568+0000 7f7b6a7fc700 1 -- 172.21.15.31:0/1538824587 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64024980 con 0x7f7b7413a1e0 2023-10-06T05:24:50.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.568+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/1538824587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b74157b80 con 0x7f7b7413a1e0 2023-10-06T05:24:50.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.569+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/1538824587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b74158380 con 0x7f7b7413a1e0 2023-10-06T05:24:50.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.570+0000 7f7b6a7fc700 1 -- 172.21.15.31:0/1538824587 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f7b6401a420 con 0x7f7b7413a1e0 2023-10-06T05:24:50.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.571+0000 7f7b6a7fc700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b580674d0 0x7f7b58069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:50.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.571+0000 7f7b6a7fc700 1 -- 172.21.15.31:0/1538824587 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f7b64086e30 con 0x7f7b7413a1e0 2023-10-06T05:24:50.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.571+0000 7f7b7a59b700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b580674d0 0x7f7b58069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:50.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.572+0000 7f7b7a59b700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b580674d0 0x7f7b58069980 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7b700018f0 tx=0x7f7b70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:50.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.575+0000 7f7b6a7fc700 1 -- 172.21.15.31:0/1538824587 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b74158380 con 0x7f7b7413a1e0 2023-10-06T05:24:50.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.821+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/1538824587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b74158340 con 0x7f7b7413a1e0 2023-10-06T05:24:50.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.821+0000 7f7b6a7fc700 1 -- 172.21.15.31:0/1538824587 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f7b640536b0 con 0x7f7b7413a1e0 2023-10-06T05:24:50.824 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:50.824 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1352198494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1352198494},{"type":"v1","addr":"172.21.15.62:6835","nonce":1352198494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8194,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/243154980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":243154980},{"type":"v1","addr":"172.21.15.62:6834","nonce":243154980}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/240917476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":240917476},{"type":"v1","addr":"172.21.15.31:6834","nonce":240917476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:46.824751+0000","modified":"2023-10-06T05:24:50.335585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3829644591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3829644591},{"type":"v1","addr":"172.21.15.31:6835","nonce":3829644591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:24:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b580674d0 msgr2=0x7f7b58069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b580674d0 0x7f7b58069980 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7b700018f0 tx=0x7f7b70009040 comp rx=0 tx=0).stop 2023-10-06T05:24:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/1538824587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a1e0 msgr2=0x7f7b740c0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a1e0 0x7f7b740c0e50 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009290 tx=0x7f7b64017000 comp rx=0 tx=0).stop 2023-10-06T05:24:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/1538824587 shutdown_connections 2023-10-06T05:24:50.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b580674d0 0x7f7b58069980 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b7413a5b0 0x7f7b740c1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b7406d980 0x7f7b741398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 --2- 172.21.15.31:0/1538824587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b7413a1e0 0x7f7b740c0e50 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:50.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/1538824587 >> 172.21.15.31:0/1538824587 conn(0x7f7b740cc0b0 msgr2=0x7f7b740b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:50.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/1538824587 shutdown_connections 2023-10-06T05:24:50.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:50.824+0000 7f7b7c7ff700 1 -- 172.21.15.31:0/1538824587 wait complete. 2023-10-06T05:24:50.829 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 93 2023-10-06T05:24:50.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:24:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.037+0000 7ff3257ff700 1 Processor -- start 2023-10-06T05:24:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.037+0000 7ff3257ff700 1 -- start start 2023-10-06T05:24:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.039+0000 7ff3257ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3201595a0 0x7ff320159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.039+0000 7ff3257ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff320068d80 0x7ff3200c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.039+0000 7ff3257ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3200c95b0 con 0x7ff3200cc0d0 2023-10-06T05:24:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.039+0000 7ff3257ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3200c9890 con 0x7ff320068d80 2023-10-06T05:24:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.039+0000 7ff3257ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3200c9b70 con 0x7ff3201595a0 2023-10-06T05:24:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.039+0000 7ff31f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff320068d80 0x7ff3200c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.039+0000 7ff31e7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff3200cc0d0 0x7ff320068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42336/0 (socket says 172.21.15.31:42336) 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.039+0000 7ff31e7fc700 1 -- 172.21.15.31:0/3221869061 learned_addr learned my addr 172.21.15.31:0/3221869061 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.039+0000 7ff31effd700 1 --2- 172.21.15.31:0/3221869061 >> v2:172.21.15.62:3300/0 conn(0x7ff3201595a0 0x7ff320159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.040+0000 7ff31dffb700 1 -- 172.21.15.31:0/3221869061 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628241090 0 0) 0x7ff3200c95b0 con 0x7ff3200cc0d0 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.040+0000 7ff31f7fe700 1 -- 172.21.15.31:0/3221869061 >> v2:172.21.15.62:3300/0 conn(0x7ff3201595a0 msgr2=0x7ff320159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.040+0000 7ff31f7fe700 1 --2- 172.21.15.31:0/3221869061 >> v2:172.21.15.62:3300/0 conn(0x7ff3201595a0 0x7ff320159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.040+0000 7ff31f7fe700 1 -- 172.21.15.31:0/3221869061 >> v1:172.21.15.31:6789/0 conn(0x7ff3200cc0d0 legacy=0x7ff320068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.040+0000 7ff31f7fe700 1 -- 172.21.15.31:0/3221869061 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3200ca130 con 0x7ff320068d80 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.040+0000 7ff31f7fe700 1 --2- 172.21.15.31:0/3221869061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 0x7ff3200c8930 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7ff318009e60 tx=0x7ff318009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d7c0b003fca63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.040+0000 7ff31dffb700 1 -- 172.21.15.31:0/3221869061 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800e070 con 0x7ff320068d80 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.040+0000 7ff31dffb700 1 -- 172.21.15.31:0/3221869061 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31800bb20 con 0x7ff320068d80 2023-10-06T05:24:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.040+0000 7ff31dffb700 1 -- 172.21.15.31:0/3221869061 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318012430 con 0x7ff320068d80 2023-10-06T05:24:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.041+0000 7ff3257ff700 1 -- 172.21.15.31:0/3221869061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 msgr2=0x7ff3200c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.041+0000 7ff3257ff700 1 --2- 172.21.15.31:0/3221869061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 0x7ff3200c8930 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7ff318009e60 tx=0x7ff318009780 comp rx=0 tx=0).stop 2023-10-06T05:24:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.041+0000 7ff3257ff700 1 -- 172.21.15.31:0/3221869061 shutdown_connections 2023-10-06T05:24:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.041+0000 7ff3257ff700 1 --2- 172.21.15.31:0/3221869061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 0x7ff3200c8930 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.041+0000 7ff3257ff700 1 --2- 172.21.15.31:0/3221869061 >> v2:172.21.15.62:3300/0 conn(0x7ff3201595a0 0x7ff320159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.041+0000 7ff3257ff700 1 -- 172.21.15.31:0/3221869061 >> 172.21.15.31:0/3221869061 conn(0x7ff32006b620 msgr2=0x7ff32006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.041+0000 7ff3257ff700 1 -- 172.21.15.31:0/3221869061 shutdown_connections 2023-10-06T05:24:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.041+0000 7ff3257ff700 1 -- 172.21.15.31:0/3221869061 wait complete. 2023-10-06T05:24:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.042+0000 7ff3257ff700 1 Processor -- start 2023-10-06T05:24:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.042+0000 7ff3257ff700 1 -- start start 2023-10-06T05:24:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.043+0000 7ff3257ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 0x7ff320156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.043+0000 7ff3257ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3200cc0d0 0x7ff3200b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.043+0000 7ff3257ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3201595a0 0x7ff32014bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.043+0000 7ff3257ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200b30e0 con 0x7ff320068d80 2023-10-06T05:24:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.043+0000 7ff3257ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200b33c0 con 0x7ff3200cc0d0 2023-10-06T05:24:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.043+0000 7ff3257ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200b36a0 con 0x7ff3201595a0 2023-10-06T05:24:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 0x7ff320156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 0x7ff320156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55560/0 (socket says 172.21.15.31:55560) 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31effd700 1 -- 172.21.15.31:0/2853231935 learned_addr learned my addr 172.21.15.31:0/2853231935 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31f7fe700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3201595a0 0x7ff32014bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31e7fc700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3200cc0d0 0x7ff3200b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31effd700 1 -- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3201595a0 msgr2=0x7ff32014bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31effd700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3201595a0 0x7ff32014bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31effd700 1 -- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3200cc0d0 msgr2=0x7ff3200b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31effd700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3200cc0d0 0x7ff3200b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31effd700 1 -- 172.21.15.31:0/2853231935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318009a20 con 0x7ff320068d80 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31e7fc700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3200cc0d0 0x7ff3200b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff31effd700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 0x7ff320156a50 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff31000ec80 tx=0x7ff3100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.044+0000 7ff306ffd700 1 -- 172.21.15.31:0/2853231935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31000c9d0 con 0x7ff320068d80 2023-10-06T05:24:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.045+0000 7ff306ffd700 1 -- 172.21.15.31:0/2853231935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31000cb30 con 0x7ff320068d80 2023-10-06T05:24:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.045+0000 7ff3257ff700 1 -- 172.21.15.31:0/2853231935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3200b3980 con 0x7ff320068d80 2023-10-06T05:24:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.045+0000 7ff306ffd700 1 -- 172.21.15.31:0/2853231935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310013470 con 0x7ff320068d80 2023-10-06T05:24:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.045+0000 7ff3257ff700 1 -- 172.21.15.31:0/2853231935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff320152e80 con 0x7ff320068d80 2023-10-06T05:24:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.046+0000 7ff3257ff700 1 -- 172.21.15.31:0/2853231935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32004c380 con 0x7ff320068d80 2023-10-06T05:24:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.047+0000 7ff306ffd700 1 -- 172.21.15.31:0/2853231935 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7ff310013820 con 0x7ff320068d80 2023-10-06T05:24:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.047+0000 7ff306ffd700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff30c0674d0 0x7ff30c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.047+0000 7ff306ffd700 1 -- 172.21.15.31:0/2853231935 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff3100854b0 con 0x7ff320068d80 2023-10-06T05:24:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.048+0000 7ff31e7fc700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff30c0674d0 0x7ff30c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.048+0000 7ff31e7fc700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff30c0674d0 0x7ff30c069980 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff32015a7a0 tx=0x7ff314009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.052+0000 7ff306ffd700 1 -- 172.21.15.31:0/2853231935 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff310051d30 con 0x7ff320068d80 2023-10-06T05:24:51.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.254+0000 7ff3257ff700 1 -- 172.21.15.31:0/2853231935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3200cdd20 con 0x7ff320068d80 2023-10-06T05:24:51.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.255+0000 7ff306ffd700 1 -- 172.21.15.31:0/2853231935 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16972 (secure 0 0 0) 0x7ff310055350 con 0x7ff320068d80 2023-10-06T05:24:51.257 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:51.257 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":139,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:48.323184+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:44.887621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:45.794431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:24:51.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 -- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff30c0674d0 msgr2=0x7ff30c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff30c0674d0 0x7ff30c069980 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff32015a7a0 tx=0x7ff314009040 comp rx=0 tx=0).stop 2023-10-06T05:24:51.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 -- 172.21.15.31:0/2853231935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 msgr2=0x7ff320156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 0x7ff320156a50 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff31000ec80 tx=0x7ff3100091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 -- 172.21.15.31:0/2853231935 shutdown_connections 2023-10-06T05:24:51.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff30c0674d0 0x7ff30c069980 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3201595a0 0x7ff32014bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff320068d80 0x7ff320156a50 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 --2- 172.21.15.31:0/2853231935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3200cc0d0 0x7ff3200b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 -- 172.21.15.31:0/2853231935 >> 172.21.15.31:0/2853231935 conn(0x7ff32006b620 msgr2=0x7ff320157470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:51.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 -- 172.21.15.31:0/2853231935 shutdown_connections 2023-10-06T05:24:51.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.258+0000 7ff3257ff700 1 -- 172.21.15.31:0/2853231935 wait complete. 2023-10-06T05:24:51.278 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:24:51.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:24:51.279 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-10-06T05:24:51.275+0000 7f534e3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:24:51.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.472+0000 7f8e197ff700 1 Processor -- start 2023-10-06T05:24:51.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.472+0000 7f8e197ff700 1 -- start start 2023-10-06T05:24:51.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.473+0000 7f8e197ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8e140c7b60 0x7f8e14069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.473+0000 7f8e197ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e140c7f30 0x7f8e140aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.473+0000 7f8e197ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e1406ef50 0x7f8e1413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.473+0000 7f8e197ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e140ca5b0 con 0x7f8e1406ef50 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.473+0000 7f8e197ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1406f740 con 0x7f8e140c7b60 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.473+0000 7f8e197ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140c9c80 con 0x7f8e140c7f30 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e137fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e1406ef50 0x7f8e1413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e137fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e1406ef50 0x7f8e1413f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55572/0 (socket says 172.21.15.31:55572) 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e137fe700 1 -- 172.21.15.31:0/4077965778 learned_addr learned my addr 172.21.15.31:0/4077965778 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e127fc700 1 --2- 172.21.15.31:0/4077965778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e140c7f30 0x7f8e140aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e12ffd700 1 --2- 172.21.15.31:0/4077965778 >> v2:172.21.15.62:3300/0 conn(0x7f8e140c7b60 0x7f8e14069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e137fe700 1 -- 172.21.15.31:0/4077965778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e140c7f30 msgr2=0x7f8e140aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e137fe700 1 --2- 172.21.15.31:0/4077965778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e140c7f30 0x7f8e140aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e137fe700 1 -- 172.21.15.31:0/4077965778 >> v2:172.21.15.62:3300/0 conn(0x7f8e140c7b60 msgr2=0x7f8e14069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e137fe700 1 --2- 172.21.15.31:0/4077965778 >> v2:172.21.15.62:3300/0 conn(0x7f8e140c7b60 0x7f8e14069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e137fe700 1 -- 172.21.15.31:0/4077965778 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e140c9f60 con 0x7f8e1406ef50 2023-10-06T05:24:51.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.474+0000 7f8e137fe700 1 --2- 172.21.15.31:0/4077965778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1406ef50 0x7f8e1413f690 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8e1406fa80 tx=0x7f8e0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3191002406fd2c8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e11ffb700 1 -- 172.21.15.31:0/4077965778 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00e070 con 0x7f8e1406ef50 2023-10-06T05:24:51.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e11ffb700 1 -- 172.21.15.31:0/4077965778 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c00b970 con 0x7f8e1406ef50 2023-10-06T05:24:51.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e11ffb700 1 -- 172.21.15.31:0/4077965778 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0092e0 con 0x7f8e1406ef50 2023-10-06T05:24:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e197ff700 1 -- 172.21.15.31:0/4077965778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1406ef50 msgr2=0x7f8e1413f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e197ff700 1 --2- 172.21.15.31:0/4077965778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1406ef50 0x7f8e1413f690 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8e1406fa80 tx=0x7f8e0c009840 comp rx=0 tx=0).stop 2023-10-06T05:24:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e197ff700 1 -- 172.21.15.31:0/4077965778 shutdown_connections 2023-10-06T05:24:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e197ff700 1 --2- 172.21.15.31:0/4077965778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e1406ef50 0x7f8e1413f690 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e197ff700 1 --2- 172.21.15.31:0/4077965778 >> v2:172.21.15.62:3300/0 conn(0x7f8e140c7b60 0x7f8e14069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e197ff700 1 --2- 172.21.15.31:0/4077965778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e140c7f30 0x7f8e140aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e197ff700 1 -- 172.21.15.31:0/4077965778 >> 172.21.15.31:0/4077965778 conn(0x7f8e140cc0b0 msgr2=0x7f8e140cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e197ff700 1 -- 172.21.15.31:0/4077965778 shutdown_connections 2023-10-06T05:24:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.475+0000 7f8e197ff700 1 -- 172.21.15.31:0/4077965778 wait complete. 2023-10-06T05:24:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.477+0000 7f8e197ff700 1 Processor -- start 2023-10-06T05:24:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.477+0000 7f8e197ff700 1 -- start start 2023-10-06T05:24:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e197ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1406ef50 0x7f8e1413dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e197ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e140c7b60 0x7f8e140afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e197ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e140c7f30 0x7f8e140b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e197ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140b1530 con 0x7f8e140c7f30 2023-10-06T05:24:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e197ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140ae190 con 0x7f8e1406ef50 2023-10-06T05:24:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e197ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140b0920 con 0x7f8e140c7b60 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e140c7f30 0x7f8e140b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e140c7f30 0x7f8e140b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55586/0 (socket says 172.21.15.31:55586) 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e137fe700 1 -- 172.21.15.31:0/541014504 learned_addr learned my addr 172.21.15.31:0/541014504 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e12ffd700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1406ef50 0x7f8e1413dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.478+0000 7f8e127fc700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e140c7b60 0x7f8e140afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.479+0000 7f8e137fe700 1 -- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e140c7b60 msgr2=0x7f8e140afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.479+0000 7f8e137fe700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e140c7b60 0x7f8e140afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.479+0000 7f8e137fe700 1 -- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1406ef50 msgr2=0x7f8e1413dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.479+0000 7f8e137fe700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1406ef50 0x7f8e1413dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.479+0000 7f8e137fe700 1 -- 172.21.15.31:0/541014504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c009ae0 con 0x7f8e140c7f30 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.479+0000 7f8e137fe700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e140c7f30 0x7f8e140b02f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009880 tx=0x7f8e0c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.479+0000 7f8dfaffd700 1 -- 172.21.15.31:0/541014504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c008b10 con 0x7f8e140c7f30 2023-10-06T05:24:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.479+0000 7f8dfaffd700 1 -- 172.21.15.31:0/541014504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c008c70 con 0x7f8e140c7f30 2023-10-06T05:24:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.479+0000 7f8dfaffd700 1 -- 172.21.15.31:0/541014504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c026460 con 0x7f8e140c7f30 2023-10-06T05:24:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.479+0000 7f8e197ff700 1 -- 172.21.15.31:0/541014504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e140b0c00 con 0x7f8e140c7f30 2023-10-06T05:24:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.480+0000 7f8e197ff700 1 -- 172.21.15.31:0/541014504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e140b11c0 con 0x7f8e140c7f30 2023-10-06T05:24:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.481+0000 7f8e197ff700 1 -- 172.21.15.31:0/541014504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e140b16e0 con 0x7f8e140c7f30 2023-10-06T05:24:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.481+0000 7f8dfaffd700 1 -- 172.21.15.31:0/541014504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f8e0c003040 con 0x7f8e140c7f30 2023-10-06T05:24:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.482+0000 7f8dfaffd700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8e000674d0 0x7f8e00069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.482+0000 7f8dfaffd700 1 -- 172.21.15.31:0/541014504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f8e0c086aa0 con 0x7f8e140c7f30 2023-10-06T05:24:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.483+0000 7f8e12ffd700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8e000674d0 0x7f8e00069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.483+0000 7f8e12ffd700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8e000674d0 0x7f8e00069980 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009da0 tx=0x7f8e04009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.487+0000 7f8dfaffd700 1 -- 172.21.15.31:0/541014504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e0c053280 con 0x7f8e140c7f30 2023-10-06T05:24:51.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.684+0000 7f8e197ff700 1 -- 172.21.15.31:0/541014504 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e1404c380 con 0x7f8e000674d0 2023-10-06T05:24:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.694+0000 7f8dfaffd700 1 -- 172.21.15.31:0/541014504 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311588 (secure 0 0 0) 0x7f8e1404c380 con 0x7f8e000674d0 2023-10-06T05:24:51.700 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 -- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8e000674d0 msgr2=0x7f8e00069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8e000674d0 0x7f8e00069980 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009da0 tx=0x7f8e04009120 comp rx=0 tx=0).stop 2023-10-06T05:24:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 -- 172.21.15.31:0/541014504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e140c7f30 msgr2=0x7f8e140b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e140c7f30 0x7f8e140b02f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009880 tx=0x7f8e0c018000 comp rx=0 tx=0).stop 2023-10-06T05:24:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 -- 172.21.15.31:0/541014504 shutdown_connections 2023-10-06T05:24:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8e000674d0 0x7f8e00069980 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e140c7f30 0x7f8e140b02f0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e1406ef50 0x7f8e1413dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 --2- 172.21.15.31:0/541014504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e140c7b60 0x7f8e140afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 -- 172.21.15.31:0/541014504 >> 172.21.15.31:0/541014504 conn(0x7f8e140cc0b0 msgr2=0x7f8e140c1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 -- 172.21.15.31:0/541014504 shutdown_connections 2023-10-06T05:24:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.699+0000 7f8e197ff700 1 -- 172.21.15.31:0/541014504 wait complete. 2023-10-06T05:24:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:24:51.719 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2023-10-06T05:24:51.507952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425960,"kb_used_data":14888,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548760,"statfs":{"total":773094113280,"available":770609930240,"internally_reserved":0,"allocated":15245312,"data_stored":7699652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001358"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.333968+0000","last_change":"2023-10-06T05:24:47.332457+0000","last_active":"2023-10-06T05:24:47.333968+0000","last_peered":"2023-10-06T05:24:47.333968+0000","last_clean":"2023-10-06T05:24:47.333968+0000","last_became_active":"2023-10-06T05:24:47.332187+0000","last_became_peered":"2023-10-06T05:24:47.332187+0000","last_unstale":"2023-10-06T05:24:47.333968+0000","last_undegraded":"2023-10-06T05:24:47.333968+0000","last_fullsized":"2023-10-06T05:24:47.333968+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:52.030713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.338204+0000","last_change":"2023-10-06T05:24:47.335225+0000","last_active":"2023-10-06T05:24:47.338204+0000","last_peered":"2023-10-06T05:24:47.338204+0000","last_clean":"2023-10-06T05:24:47.338204+0000","last_became_active":"2023-10-06T05:24:47.334471+0000","last_became_peered":"2023-10-06T05:24:47.334471+0000","last_unstale":"2023-10-06T05:24:47.338204+0000","last_undegraded":"2023-10-06T05:24:47.338204+0000","last_fullsized":"2023-10-06T05:24:47.338204+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:58.514634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349104+0000","last_change":"2023-10-06T05:24:47.341601+0000","last_active":"2023-10-06T05:24:47.349104+0000","last_peered":"2023-10-06T05:24:47.349104+0000","last_clean":"2023-10-06T05:24:47.349104+0000","last_became_active":"2023-10-06T05:24:47.341291+0000","last_became_peered":"2023-10-06T05:24:47.341291+0000","last_unstale":"2023-10-06T05:24:47.349104+0000","last_undegraded":"2023-10-06T05:24:47.349104+0000","last_fullsized":"2023-10-06T05:24:47.349104+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:16.023915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.333054+0000","last_change":"2023-10-06T05:24:47.326865+0000","last_active":"2023-10-06T05:24:47.333054+0000","last_peered":"2023-10-06T05:24:47.333054+0000","last_clean":"2023-10-06T05:24:47.333054+0000","last_became_active":"2023-10-06T05:24:47.326511+0000","last_became_peered":"2023-10-06T05:24:47.326511+0000","last_unstale":"2023-10-06T05:24:47.333054+0000","last_undegraded":"2023-10-06T05:24:47.333054+0000","last_fullsized":"2023-10-06T05:24:47.333054+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:18.401365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349879+0000","last_change":"2023-10-06T05:24:47.337371+0000","last_active":"2023-10-06T05:24:47.349879+0000","last_peered":"2023-10-06T05:24:47.349879+0000","last_clean":"2023-10-06T05:24:47.349879+0000","last_became_active":"2023-10-06T05:24:47.337097+0000","last_became_peered":"2023-10-06T05:24:47.337097+0000","last_unstale":"2023-10-06T05:24:47.349879+0000","last_undegraded":"2023-10-06T05:24:47.349879+0000","last_fullsized":"2023-10-06T05:24:47.349879+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:58.514634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.338574+0000","last_change":"2023-10-06T05:24:47.336850+0000","last_active":"2023-10-06T05:24:47.338574+0000","last_peered":"2023-10-06T05:24:47.338574+0000","last_clean":"2023-10-06T05:24:47.338574+0000","last_became_active":"2023-10-06T05:24:47.336541+0000","last_became_peered":"2023-10-06T05:24:47.336541+0000","last_unstale":"2023-10-06T05:24:47.338574+0000","last_undegraded":"2023-10-06T05:24:47.338574+0000","last_fullsized":"2023-10-06T05:24:47.338574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:26.575986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344381+0000","last_change":"2023-10-06T05:24:47.343543+0000","last_active":"2023-10-06T05:24:47.344381+0000","last_peered":"2023-10-06T05:24:47.344381+0000","last_clean":"2023-10-06T05:24:47.344381+0000","last_became_active":"2023-10-06T05:24:47.343084+0000","last_became_peered":"2023-10-06T05:24:47.343084+0000","last_unstale":"2023-10-06T05:24:47.344381+0000","last_undegraded":"2023-10-06T05:24:47.344381+0000","last_fullsized":"2023-10-06T05:24:47.344381+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:59.864576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344658+0000","last_change":"2023-10-06T05:24:47.325215+0000","last_active":"2023-10-06T05:24:47.344658+0000","last_peered":"2023-10-06T05:24:47.344658+0000","last_clean":"2023-10-06T05:24:47.344658+0000","last_became_active":"2023-10-06T05:24:47.324885+0000","last_became_peered":"2023-10-06T05:24:47.324885+0000","last_unstale":"2023-10-06T05:24:47.344658+0000","last_undegraded":"2023-10-06T05:24:47.344658+0000","last_fullsized":"2023-10-06T05:24:47.344658+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:57.933932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344265+0000","last_change":"2023-10-06T05:24:47.335108+0000","last_active":"2023-10-06T05:24:47.344265+0000","last_peered":"2023-10-06T05:24:47.344265+0000","last_clean":"2023-10-06T05:24:47.344265+0000","last_became_active":"2023-10-06T05:24:47.334459+0000","last_became_peered":"2023-10-06T05:24:47.334459+0000","last_unstale":"2023-10-06T05:24:47.344265+0000","last_undegraded":"2023-10-06T05:24:47.344265+0000","last_fullsized":"2023-10-06T05:24:47.344265+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:39.100980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.338081+0000","last_change":"2023-10-06T05:24:47.323198+0000","last_active":"2023-10-06T05:24:47.338081+0000","last_peered":"2023-10-06T05:24:47.338081+0000","last_clean":"2023-10-06T05:24:47.338081+0000","last_became_active":"2023-10-06T05:24:47.322941+0000","last_became_peered":"2023-10-06T05:24:47.322941+0000","last_unstale":"2023-10-06T05:24:47.338081+0000","last_undegraded":"2023-10-06T05:24:47.338081+0000","last_fullsized":"2023-10-06T05:24:47.338081+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:59.334226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344436+0000","last_change":"2023-10-06T05:24:47.336513+0000","last_active":"2023-10-06T05:24:47.344436+0000","last_peered":"2023-10-06T05:24:47.344436+0000","last_clean":"2023-10-06T05:24:47.344436+0000","last_became_active":"2023-10-06T05:24:47.336113+0000","last_became_peered":"2023-10-06T05:24:47.336113+0000","last_unstale":"2023-10-06T05:24:47.344436+0000","last_undegraded":"2023-10-06T05:24:47.344436+0000","last_fullsized":"2023-10-06T05:24:47.344436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:48.856362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.339222+0000","last_change":"2023-10-06T05:24:47.337114+0000","last_active":"2023-10-06T05:24:47.339222+0000","last_peered":"2023-10-06T05:24:47.339222+0000","last_clean":"2023-10-06T05:24:47.339222+0000","last_became_active":"2023-10-06T05:24:47.336815+0000","last_became_peered":"2023-10-06T05:24:47.336815+0000","last_unstale":"2023-10-06T05:24:47.339222+0000","last_undegraded":"2023-10-06T05:24:47.339222+0000","last_fullsized":"2023-10-06T05:24:47.339222+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:56.692763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.335523+0000","last_change":"2023-10-06T05:24:47.331372+0000","last_active":"2023-10-06T05:24:47.335523+0000","last_peered":"2023-10-06T05:24:47.335523+0000","last_clean":"2023-10-06T05:24:47.335523+0000","last_became_active":"2023-10-06T05:24:47.331110+0000","last_became_peered":"2023-10-06T05:24:47.331110+0000","last_unstale":"2023-10-06T05:24:47.335523+0000","last_undegraded":"2023-10-06T05:24:47.335523+0000","last_fullsized":"2023-10-06T05:24:47.335523+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:55.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351664+0000","last_change":"2023-10-06T05:24:47.342447+0000","last_active":"2023-10-06T05:24:47.351664+0000","last_peered":"2023-10-06T05:24:47.351664+0000","last_clean":"2023-10-06T05:24:47.351664+0000","last_became_active":"2023-10-06T05:24:47.342115+0000","last_became_peered":"2023-10-06T05:24:47.342115+0000","last_unstale":"2023-10-06T05:24:47.351664+0000","last_undegraded":"2023-10-06T05:24:47.351664+0000","last_fullsized":"2023-10-06T05:24:47.351664+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:16.072487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348513+0000","last_change":"2023-10-06T05:24:47.347168+0000","last_active":"2023-10-06T05:24:47.348513+0000","last_peered":"2023-10-06T05:24:47.348513+0000","last_clean":"2023-10-06T05:24:47.348513+0000","last_became_active":"2023-10-06T05:24:47.346844+0000","last_became_peered":"2023-10-06T05:24:47.346844+0000","last_unstale":"2023-10-06T05:24:47.348513+0000","last_undegraded":"2023-10-06T05:24:47.348513+0000","last_fullsized":"2023-10-06T05:24:47.348513+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:28.298824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.333456+0000","last_change":"2023-10-06T05:24:47.327590+0000","last_active":"2023-10-06T05:24:47.333456+0000","last_peered":"2023-10-06T05:24:47.333456+0000","last_clean":"2023-10-06T05:24:47.333456+0000","last_became_active":"2023-10-06T05:24:47.327220+0000","last_became_peered":"2023-10-06T05:24:47.327220+0000","last_unstale":"2023-10-06T05:24:47.333456+0000","last_undegraded":"2023-10-06T05:24:47.333456+0000","last_fullsized":"2023-10-06T05:24:47.333456+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:04.183491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344289+0000","last_change":"2023-10-06T05:24:47.335289+0000","last_active":"2023-10-06T05:24:47.344289+0000","last_peered":"2023-10-06T05:24:47.344289+0000","last_clean":"2023-10-06T05:24:47.344289+0000","last_became_active":"2023-10-06T05:24:47.334827+0000","last_became_peered":"2023-10-06T05:24:47.334827+0000","last_unstale":"2023-10-06T05:24:47.344289+0000","last_undegraded":"2023-10-06T05:24:47.344289+0000","last_fullsized":"2023-10-06T05:24:47.344289+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:34.867620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348079+0000","last_change":"2023-10-06T05:24:47.346575+0000","last_active":"2023-10-06T05:24:47.348079+0000","last_peered":"2023-10-06T05:24:47.348079+0000","last_clean":"2023-10-06T05:24:47.348079+0000","last_became_active":"2023-10-06T05:24:47.346288+0000","last_became_peered":"2023-10-06T05:24:47.346288+0000","last_unstale":"2023-10-06T05:24:47.348079+0000","last_undegraded":"2023-10-06T05:24:47.348079+0000","last_fullsized":"2023-10-06T05:24:47.348079+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:04.821590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348484+0000","last_change":"2023-10-06T05:24:47.347119+0000","last_active":"2023-10-06T05:24:47.348484+0000","last_peered":"2023-10-06T05:24:47.348484+0000","last_clean":"2023-10-06T05:24:47.348484+0000","last_became_active":"2023-10-06T05:24:47.346692+0000","last_became_peered":"2023-10-06T05:24:47.346692+0000","last_unstale":"2023-10-06T05:24:47.348484+0000","last_undegraded":"2023-10-06T05:24:47.348484+0000","last_fullsized":"2023-10-06T05:24:47.348484+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:57.933932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344646+0000","last_change":"2023-10-06T05:24:47.339163+0000","last_active":"2023-10-06T05:24:47.344646+0000","last_peered":"2023-10-06T05:24:47.344646+0000","last_clean":"2023-10-06T05:24:47.344646+0000","last_became_active":"2023-10-06T05:24:47.338824+0000","last_became_peered":"2023-10-06T05:24:47.338824+0000","last_unstale":"2023-10-06T05:24:47.344646+0000","last_undegraded":"2023-10-06T05:24:47.344646+0000","last_fullsized":"2023-10-06T05:24:47.344646+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:29.399416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.340994+0000","last_change":"2023-10-06T05:24:47.335344+0000","last_active":"2023-10-06T05:24:47.340994+0000","last_peered":"2023-10-06T05:24:47.340994+0000","last_clean":"2023-10-06T05:24:47.340994+0000","last_became_active":"2023-10-06T05:24:47.335053+0000","last_became_peered":"2023-10-06T05:24:47.335053+0000","last_unstale":"2023-10-06T05:24:47.340994+0000","last_undegraded":"2023-10-06T05:24:47.340994+0000","last_fullsized":"2023-10-06T05:24:47.340994+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:13.343995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.345963+0000","last_change":"2023-10-06T05:24:47.342356+0000","last_active":"2023-10-06T05:24:47.345963+0000","last_peered":"2023-10-06T05:24:47.345963+0000","last_clean":"2023-10-06T05:24:47.345963+0000","last_became_active":"2023-10-06T05:24:47.342072+0000","last_became_peered":"2023-10-06T05:24:47.342072+0000","last_unstale":"2023-10-06T05:24:47.345963+0000","last_undegraded":"2023-10-06T05:24:47.345963+0000","last_fullsized":"2023-10-06T05:24:47.345963+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:33.341535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351792+0000","last_change":"2023-10-06T05:24:47.345949+0000","last_active":"2023-10-06T05:24:47.351792+0000","last_peered":"2023-10-06T05:24:47.351792+0000","last_clean":"2023-10-06T05:24:47.351792+0000","last_became_active":"2023-10-06T05:24:47.345653+0000","last_became_peered":"2023-10-06T05:24:47.345653+0000","last_unstale":"2023-10-06T05:24:47.351792+0000","last_undegraded":"2023-10-06T05:24:47.351792+0000","last_fullsized":"2023-10-06T05:24:47.351792+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:48.148918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.333435+0000","last_change":"2023-10-06T05:24:47.327448+0000","last_active":"2023-10-06T05:24:47.333435+0000","last_peered":"2023-10-06T05:24:47.333435+0000","last_clean":"2023-10-06T05:24:47.333435+0000","last_became_active":"2023-10-06T05:24:47.327156+0000","last_became_peered":"2023-10-06T05:24:47.327156+0000","last_unstale":"2023-10-06T05:24:47.333435+0000","last_undegraded":"2023-10-06T05:24:47.333435+0000","last_fullsized":"2023-10-06T05:24:47.333435+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:30.384201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.340894+0000","last_change":"2023-10-06T05:24:47.336880+0000","last_active":"2023-10-06T05:24:47.340894+0000","last_peered":"2023-10-06T05:24:47.340894+0000","last_clean":"2023-10-06T05:24:47.340894+0000","last_became_active":"2023-10-06T05:24:47.336573+0000","last_became_peered":"2023-10-06T05:24:47.336573+0000","last_unstale":"2023-10-06T05:24:47.340894+0000","last_undegraded":"2023-10-06T05:24:47.340894+0000","last_fullsized":"2023-10-06T05:24:47.340894+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:10.768676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344553+0000","last_change":"2023-10-06T05:24:47.336807+0000","last_active":"2023-10-06T05:24:47.344553+0000","last_peered":"2023-10-06T05:24:47.344553+0000","last_clean":"2023-10-06T05:24:47.344553+0000","last_became_active":"2023-10-06T05:24:47.336564+0000","last_became_peered":"2023-10-06T05:24:47.336564+0000","last_unstale":"2023-10-06T05:24:47.344553+0000","last_undegraded":"2023-10-06T05:24:47.344553+0000","last_fullsized":"2023-10-06T05:24:47.344553+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:41.071296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348263+0000","last_change":"2023-10-06T05:24:47.343654+0000","last_active":"2023-10-06T05:24:47.348263+0000","last_peered":"2023-10-06T05:24:47.348263+0000","last_clean":"2023-10-06T05:24:47.348263+0000","last_became_active":"2023-10-06T05:24:47.343242+0000","last_became_peered":"2023-10-06T05:24:47.343242+0000","last_unstale":"2023-10-06T05:24:47.348263+0000","last_undegraded":"2023-10-06T05:24:47.348263+0000","last_fullsized":"2023-10-06T05:24:47.348263+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:48.148918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.346175+0000","last_change":"2023-10-06T05:24:47.337090+0000","last_active":"2023-10-06T05:24:47.346175+0000","last_peered":"2023-10-06T05:24:47.346175+0000","last_clean":"2023-10-06T05:24:47.346175+0000","last_became_active":"2023-10-06T05:24:47.335577+0000","last_became_peered":"2023-10-06T05:24:47.335577+0000","last_unstale":"2023-10-06T05:24:47.346175+0000","last_undegraded":"2023-10-06T05:24:47.346175+0000","last_fullsized":"2023-10-06T05:24:47.346175+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:41.663801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.340194+0000","last_change":"2023-10-06T05:24:47.339113+0000","last_active":"2023-10-06T05:24:47.340194+0000","last_peered":"2023-10-06T05:24:47.340194+0000","last_clean":"2023-10-06T05:24:47.340194+0000","last_became_active":"2023-10-06T05:24:47.338787+0000","last_became_peered":"2023-10-06T05:24:47.338787+0000","last_unstale":"2023-10-06T05:24:47.340194+0000","last_undegraded":"2023-10-06T05:24:47.340194+0000","last_fullsized":"2023-10-06T05:24:47.340194+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:26.414974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348952+0000","last_change":"2023-10-06T05:24:47.347188+0000","last_active":"2023-10-06T05:24:47.348952+0000","last_peered":"2023-10-06T05:24:47.348952+0000","last_clean":"2023-10-06T05:24:47.348952+0000","last_became_active":"2023-10-06T05:24:47.346912+0000","last_became_peered":"2023-10-06T05:24:47.346912+0000","last_unstale":"2023-10-06T05:24:47.348952+0000","last_undegraded":"2023-10-06T05:24:47.348952+0000","last_fullsized":"2023-10-06T05:24:47.348952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:54.963898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.345285+0000","last_change":"2023-10-06T05:24:47.334182+0000","last_active":"2023-10-06T05:24:47.345285+0000","last_peered":"2023-10-06T05:24:47.345285+0000","last_clean":"2023-10-06T05:24:47.345285+0000","last_became_active":"2023-10-06T05:24:47.333671+0000","last_became_peered":"2023-10-06T05:24:47.333671+0000","last_unstale":"2023-10-06T05:24:47.345285+0000","last_undegraded":"2023-10-06T05:24:47.345285+0000","last_fullsized":"2023-10-06T05:24:47.345285+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:56.177616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348011+0000","last_change":"2023-10-06T05:24:47.345904+0000","last_active":"2023-10-06T05:24:47.348011+0000","last_peered":"2023-10-06T05:24:47.348011+0000","last_clean":"2023-10-06T05:24:47.348011+0000","last_became_active":"2023-10-06T05:24:47.345613+0000","last_became_peered":"2023-10-06T05:24:47.345613+0000","last_unstale":"2023-10-06T05:24:47.348011+0000","last_undegraded":"2023-10-06T05:24:47.348011+0000","last_fullsized":"2023-10-06T05:24:47.348011+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:09.491372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.343302+0000","last_change":"2023-10-06T05:24:47.341126+0000","last_active":"2023-10-06T05:24:47.343302+0000","last_peered":"2023-10-06T05:24:47.343302+0000","last_clean":"2023-10-06T05:24:47.343302+0000","last_became_active":"2023-10-06T05:24:47.340663+0000","last_became_peered":"2023-10-06T05:24:47.340663+0000","last_unstale":"2023-10-06T05:24:47.343302+0000","last_undegraded":"2023-10-06T05:24:47.343302+0000","last_fullsized":"2023-10-06T05:24:47.343302+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:56.170571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"138'4","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349096+0000","last_change":"2023-10-06T05:24:46.332641+0000","last_active":"2023-10-06T05:24:47.349096+0000","last_peered":"2023-10-06T05:24:47.349096+0000","last_clean":"2023-10-06T05:24:47.349096+0000","last_became_active":"2023-10-06T05:24:46.332313+0000","last_became_peered":"2023-10-06T05:24:46.332313+0000","last_unstale":"2023-10-06T05:24:47.349096+0000","last_undegraded":"2023-10-06T05:24:47.349096+0000","last_fullsized":"2023-10-06T05:24:47.349096+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:16.966888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.342199+0000","last_change":"2023-10-06T05:24:47.333533+0000","last_active":"2023-10-06T05:24:47.342199+0000","last_peered":"2023-10-06T05:24:47.342199+0000","last_clean":"2023-10-06T05:24:47.342199+0000","last_became_active":"2023-10-06T05:24:47.333273+0000","last_became_peered":"2023-10-06T05:24:47.333273+0000","last_unstale":"2023-10-06T05:24:47.342199+0000","last_undegraded":"2023-10-06T05:24:47.342199+0000","last_fullsized":"2023-10-06T05:24:47.342199+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:06.143271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351401+0000","last_change":"2023-10-06T05:24:46.336614+0000","last_active":"2023-10-06T05:24:47.351401+0000","last_peered":"2023-10-06T05:24:47.351401+0000","last_clean":"2023-10-06T05:24:47.351401+0000","last_became_active":"2023-10-06T05:24:46.336336+0000","last_became_peered":"2023-10-06T05:24:46.336336+0000","last_unstale":"2023-10-06T05:24:47.351401+0000","last_undegraded":"2023-10-06T05:24:47.351401+0000","last_fullsized":"2023-10-06T05:24:47.351401+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:46.165907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351682+0000","last_change":"2023-10-06T05:24:47.342701+0000","last_active":"2023-10-06T05:24:47.351682+0000","last_peered":"2023-10-06T05:24:47.351682+0000","last_clean":"2023-10-06T05:24:47.351682+0000","last_became_active":"2023-10-06T05:24:47.341833+0000","last_became_peered":"2023-10-06T05:24:47.341833+0000","last_unstale":"2023-10-06T05:24:47.351682+0000","last_undegraded":"2023-10-06T05:24:47.351682+0000","last_fullsized":"2023-10-06T05:24:47.351682+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:15.459769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.350664+0000","last_change":"2023-10-06T05:24:46.334143+0000","last_active":"2023-10-06T05:24:47.350664+0000","last_peered":"2023-10-06T05:24:47.350664+0000","last_clean":"2023-10-06T05:24:47.350664+0000","last_became_active":"2023-10-06T05:24:46.333594+0000","last_became_peered":"2023-10-06T05:24:46.333594+0000","last_unstale":"2023-10-06T05:24:47.350664+0000","last_undegraded":"2023-10-06T05:24:47.350664+0000","last_fullsized":"2023-10-06T05:24:47.350664+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:36.402888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348230+0000","last_change":"2023-10-06T05:24:47.343401+0000","last_active":"2023-10-06T05:24:47.348230+0000","last_peered":"2023-10-06T05:24:47.348230+0000","last_clean":"2023-10-06T05:24:47.348230+0000","last_became_active":"2023-10-06T05:24:47.342896+0000","last_became_peered":"2023-10-06T05:24:47.342896+0000","last_unstale":"2023-10-06T05:24:47.348230+0000","last_undegraded":"2023-10-06T05:24:47.348230+0000","last_fullsized":"2023-10-06T05:24:47.348230+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:19.107039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348647+0000","last_change":"2023-10-06T05:24:46.337938+0000","last_active":"2023-10-06T05:24:47.348647+0000","last_peered":"2023-10-06T05:24:47.348647+0000","last_clean":"2023-10-06T05:24:47.348647+0000","last_became_active":"2023-10-06T05:24:46.337677+0000","last_became_peered":"2023-10-06T05:24:46.337677+0000","last_unstale":"2023-10-06T05:24:47.348647+0000","last_undegraded":"2023-10-06T05:24:47.348647+0000","last_fullsized":"2023-10-06T05:24:47.348647+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:51.789343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344327+0000","last_change":"2023-10-06T05:24:47.330026+0000","last_active":"2023-10-06T05:24:47.344327+0000","last_peered":"2023-10-06T05:24:47.344327+0000","last_clean":"2023-10-06T05:24:47.344327+0000","last_became_active":"2023-10-06T05:24:47.329790+0000","last_became_peered":"2023-10-06T05:24:47.329790+0000","last_unstale":"2023-10-06T05:24:47.344327+0000","last_undegraded":"2023-10-06T05:24:47.344327+0000","last_fullsized":"2023-10-06T05:24:47.344327+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:28.298824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344379+0000","last_change":"2023-10-06T05:24:46.336585+0000","last_active":"2023-10-06T05:24:47.344379+0000","last_peered":"2023-10-06T05:24:47.344379+0000","last_clean":"2023-10-06T05:24:47.344379+0000","last_became_active":"2023-10-06T05:24:46.336301+0000","last_became_peered":"2023-10-06T05:24:46.336301+0000","last_unstale":"2023-10-06T05:24:47.344379+0000","last_undegraded":"2023-10-06T05:24:47.344379+0000","last_fullsized":"2023-10-06T05:24:47.344379+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:09.220830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344588+0000","last_change":"2023-10-06T05:24:47.339218+0000","last_active":"2023-10-06T05:24:47.344588+0000","last_peered":"2023-10-06T05:24:47.344588+0000","last_clean":"2023-10-06T05:24:47.344588+0000","last_became_active":"2023-10-06T05:24:47.338505+0000","last_became_peered":"2023-10-06T05:24:47.338505+0000","last_unstale":"2023-10-06T05:24:47.344588+0000","last_undegraded":"2023-10-06T05:24:47.344588+0000","last_fullsized":"2023-10-06T05:24:47.344588+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:44.815157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.350982+0000","last_change":"2023-10-06T05:24:46.333864+0000","last_active":"2023-10-06T05:24:47.350982+0000","last_peered":"2023-10-06T05:24:47.350982+0000","last_clean":"2023-10-06T05:24:47.350982+0000","last_became_active":"2023-10-06T05:24:46.333602+0000","last_became_peered":"2023-10-06T05:24:46.333602+0000","last_unstale":"2023-10-06T05:24:47.350982+0000","last_undegraded":"2023-10-06T05:24:47.350982+0000","last_fullsized":"2023-10-06T05:24:47.350982+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:27.684682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344531+0000","last_change":"2023-10-06T05:24:47.336264+0000","last_active":"2023-10-06T05:24:47.344531+0000","last_peered":"2023-10-06T05:24:47.344531+0000","last_clean":"2023-10-06T05:24:47.344531+0000","last_became_active":"2023-10-06T05:24:47.335853+0000","last_became_peered":"2023-10-06T05:24:47.335853+0000","last_unstale":"2023-10-06T05:24:47.344531+0000","last_undegraded":"2023-10-06T05:24:47.344531+0000","last_fullsized":"2023-10-06T05:24:47.344531+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:39.548657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.331322+0000","last_change":"2023-10-06T05:24:46.336039+0000","last_active":"2023-10-06T05:24:47.331322+0000","last_peered":"2023-10-06T05:24:47.331322+0000","last_clean":"2023-10-06T05:24:47.331322+0000","last_became_active":"2023-10-06T05:24:46.335768+0000","last_became_peered":"2023-10-06T05:24:46.335768+0000","last_unstale":"2023-10-06T05:24:47.331322+0000","last_undegraded":"2023-10-06T05:24:47.331322+0000","last_fullsized":"2023-10-06T05:24:47.331322+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:50.924611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.335284+0000","last_change":"2023-10-06T05:24:47.333204+0000","last_active":"2023-10-06T05:24:47.335284+0000","last_peered":"2023-10-06T05:24:47.335284+0000","last_clean":"2023-10-06T05:24:47.335284+0000","last_became_active":"2023-10-06T05:24:47.332808+0000","last_became_peered":"2023-10-06T05:24:47.332808+0000","last_unstale":"2023-10-06T05:24:47.335284+0000","last_undegraded":"2023-10-06T05:24:47.335284+0000","last_fullsized":"2023-10-06T05:24:47.335284+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:05.590883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.316063+0000","last_change":"2023-10-06T05:24:46.334199+0000","last_active":"2023-10-06T05:24:47.316063+0000","last_peered":"2023-10-06T05:24:47.316063+0000","last_clean":"2023-10-06T05:24:47.316063+0000","last_became_active":"2023-10-06T05:24:46.333874+0000","last_became_peered":"2023-10-06T05:24:46.333874+0000","last_unstale":"2023-10-06T05:24:47.316063+0000","last_undegraded":"2023-10-06T05:24:47.316063+0000","last_fullsized":"2023-10-06T05:24:47.316063+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:20.047556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344694+0000","last_change":"2023-10-06T05:24:47.338891+0000","last_active":"2023-10-06T05:24:47.344694+0000","last_peered":"2023-10-06T05:24:47.344694+0000","last_clean":"2023-10-06T05:24:47.344694+0000","last_became_active":"2023-10-06T05:24:47.338620+0000","last_became_peered":"2023-10-06T05:24:47.338620+0000","last_unstale":"2023-10-06T05:24:47.344694+0000","last_undegraded":"2023-10-06T05:24:47.344694+0000","last_fullsized":"2023-10-06T05:24:47.344694+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:39.611070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.350634+0000","last_change":"2023-10-06T05:24:46.335671+0000","last_active":"2023-10-06T05:24:47.350634+0000","last_peered":"2023-10-06T05:24:47.350634+0000","last_clean":"2023-10-06T05:24:47.350634+0000","last_became_active":"2023-10-06T05:24:46.335263+0000","last_became_peered":"2023-10-06T05:24:46.335263+0000","last_unstale":"2023-10-06T05:24:47.350634+0000","last_undegraded":"2023-10-06T05:24:47.350634+0000","last_fullsized":"2023-10-06T05:24:47.350634+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:25.406249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.335445+0000","last_change":"2023-10-06T05:24:47.333905+0000","last_active":"2023-10-06T05:24:47.335445+0000","last_peered":"2023-10-06T05:24:47.335445+0000","last_clean":"2023-10-06T05:24:47.335445+0000","last_became_active":"2023-10-06T05:24:47.333514+0000","last_became_peered":"2023-10-06T05:24:47.333514+0000","last_unstale":"2023-10-06T05:24:47.335445+0000","last_undegraded":"2023-10-06T05:24:47.335445+0000","last_fullsized":"2023-10-06T05:24:47.335445+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:15.122983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.346596+0000","last_change":"2023-10-06T05:24:46.324073+0000","last_active":"2023-10-06T05:24:47.346596+0000","last_peered":"2023-10-06T05:24:47.346596+0000","last_clean":"2023-10-06T05:24:47.346596+0000","last_became_active":"2023-10-06T05:24:46.323780+0000","last_became_peered":"2023-10-06T05:24:46.323780+0000","last_unstale":"2023-10-06T05:24:47.346596+0000","last_undegraded":"2023-10-06T05:24:47.346596+0000","last_fullsized":"2023-10-06T05:24:47.346596+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.164152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344458+0000","last_change":"2023-10-06T05:24:47.337385+0000","last_active":"2023-10-06T05:24:47.344458+0000","last_peered":"2023-10-06T05:24:47.344458+0000","last_clean":"2023-10-06T05:24:47.344458+0000","last_became_active":"2023-10-06T05:24:47.337083+0000","last_became_peered":"2023-10-06T05:24:47.337083+0000","last_unstale":"2023-10-06T05:24:47.344458+0000","last_undegraded":"2023-10-06T05:24:47.344458+0000","last_fullsized":"2023-10-06T05:24:47.344458+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:59.334226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349898+0000","last_change":"2023-10-06T05:24:46.339404+0000","last_active":"2023-10-06T05:24:47.349898+0000","last_peered":"2023-10-06T05:24:47.349898+0000","last_clean":"2023-10-06T05:24:47.349898+0000","last_became_active":"2023-10-06T05:24:46.339116+0000","last_became_peered":"2023-10-06T05:24:46.339116+0000","last_unstale":"2023-10-06T05:24:47.349898+0000","last_undegraded":"2023-10-06T05:24:47.349898+0000","last_fullsized":"2023-10-06T05:24:47.349898+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:24.271761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344484+0000","last_change":"2023-10-06T05:24:47.335818+0000","last_active":"2023-10-06T05:24:47.344484+0000","last_peered":"2023-10-06T05:24:47.344484+0000","last_clean":"2023-10-06T05:24:47.344484+0000","last_became_active":"2023-10-06T05:24:47.335516+0000","last_became_peered":"2023-10-06T05:24:47.335516+0000","last_unstale":"2023-10-06T05:24:47.344484+0000","last_undegraded":"2023-10-06T05:24:47.344484+0000","last_fullsized":"2023-10-06T05:24:47.344484+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:59.864576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.318095+0000","last_change":"2023-10-06T05:24:46.341377+0000","last_active":"2023-10-06T05:24:47.318095+0000","last_peered":"2023-10-06T05:24:47.318095+0000","last_clean":"2023-10-06T05:24:47.318095+0000","last_became_active":"2023-10-06T05:24:46.341139+0000","last_became_peered":"2023-10-06T05:24:46.341139+0000","last_unstale":"2023-10-06T05:24:47.318095+0000","last_undegraded":"2023-10-06T05:24:47.318095+0000","last_fullsized":"2023-10-06T05:24:47.318095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:23.398834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.346193+0000","last_change":"2023-10-06T05:24:47.335826+0000","last_active":"2023-10-06T05:24:47.346193+0000","last_peered":"2023-10-06T05:24:47.346193+0000","last_clean":"2023-10-06T05:24:47.346193+0000","last_became_active":"2023-10-06T05:24:47.335226+0000","last_became_peered":"2023-10-06T05:24:47.335226+0000","last_unstale":"2023-10-06T05:24:47.346193+0000","last_undegraded":"2023-10-06T05:24:47.346193+0000","last_fullsized":"2023-10-06T05:24:47.346193+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:27.769222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351083+0000","last_change":"2023-10-06T05:24:46.340480+0000","last_active":"2023-10-06T05:24:47.351083+0000","last_peered":"2023-10-06T05:24:47.351083+0000","last_clean":"2023-10-06T05:24:47.351083+0000","last_became_active":"2023-10-06T05:24:46.340211+0000","last_became_peered":"2023-10-06T05:24:46.340211+0000","last_unstale":"2023-10-06T05:24:47.351083+0000","last_undegraded":"2023-10-06T05:24:47.351083+0000","last_fullsized":"2023-10-06T05:24:47.351083+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:30.470575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351897+0000","last_change":"2023-10-06T05:24:47.346488+0000","last_active":"2023-10-06T05:24:47.351897+0000","last_peered":"2023-10-06T05:24:47.351897+0000","last_clean":"2023-10-06T05:24:47.351897+0000","last_became_active":"2023-10-06T05:24:47.346216+0000","last_became_peered":"2023-10-06T05:24:47.346216+0000","last_unstale":"2023-10-06T05:24:47.351897+0000","last_undegraded":"2023-10-06T05:24:47.351897+0000","last_fullsized":"2023-10-06T05:24:47.351897+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:51.823979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321921+0000","last_change":"2023-10-06T05:24:46.337982+0000","last_active":"2023-10-06T05:24:47.321921+0000","last_peered":"2023-10-06T05:24:47.321921+0000","last_clean":"2023-10-06T05:24:47.321921+0000","last_became_active":"2023-10-06T05:24:46.337576+0000","last_became_peered":"2023-10-06T05:24:46.337576+0000","last_unstale":"2023-10-06T05:24:47.321921+0000","last_undegraded":"2023-10-06T05:24:47.321921+0000","last_fullsized":"2023-10-06T05:24:47.321921+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:32.286601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351884+0000","last_change":"2023-10-06T05:24:47.347538+0000","last_active":"2023-10-06T05:24:47.351884+0000","last_peered":"2023-10-06T05:24:47.351884+0000","last_clean":"2023-10-06T05:24:47.351884+0000","last_became_active":"2023-10-06T05:24:47.347206+0000","last_became_peered":"2023-10-06T05:24:47.347206+0000","last_unstale":"2023-10-06T05:24:47.351884+0000","last_undegraded":"2023-10-06T05:24:47.351884+0000","last_fullsized":"2023-10-06T05:24:47.351884+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:33.341535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.331417+0000","last_change":"2023-10-06T05:24:46.333226+0000","last_active":"2023-10-06T05:24:47.331417+0000","last_peered":"2023-10-06T05:24:47.331417+0000","last_clean":"2023-10-06T05:24:47.331417+0000","last_became_active":"2023-10-06T05:24:46.332920+0000","last_became_peered":"2023-10-06T05:24:46.332920+0000","last_unstale":"2023-10-06T05:24:47.331417+0000","last_undegraded":"2023-10-06T05:24:47.331417+0000","last_fullsized":"2023-10-06T05:24:47.331417+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:19.745725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.345915+0000","last_change":"2023-10-06T05:24:47.342333+0000","last_active":"2023-10-06T05:24:47.345915+0000","last_peered":"2023-10-06T05:24:47.345915+0000","last_clean":"2023-10-06T05:24:47.345915+0000","last_became_active":"2023-10-06T05:24:47.342055+0000","last_became_peered":"2023-10-06T05:24:47.342055+0000","last_unstale":"2023-10-06T05:24:47.345915+0000","last_undegraded":"2023-10-06T05:24:47.345915+0000","last_fullsized":"2023-10-06T05:24:47.345915+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:16.072487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.322226+0000","last_change":"2023-10-06T05:24:46.339259+0000","last_active":"2023-10-06T05:24:47.322226+0000","last_peered":"2023-10-06T05:24:47.322226+0000","last_clean":"2023-10-06T05:24:47.322226+0000","last_became_active":"2023-10-06T05:24:46.338824+0000","last_became_peered":"2023-10-06T05:24:46.338824+0000","last_unstale":"2023-10-06T05:24:47.322226+0000","last_undegraded":"2023-10-06T05:24:47.322226+0000","last_fullsized":"2023-10-06T05:24:47.322226+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:56.925208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344584+0000","last_change":"2023-10-06T05:24:47.332587+0000","last_active":"2023-10-06T05:24:47.344584+0000","last_peered":"2023-10-06T05:24:47.344584+0000","last_clean":"2023-10-06T05:24:47.344584+0000","last_became_active":"2023-10-06T05:24:47.332135+0000","last_became_peered":"2023-10-06T05:24:47.332135+0000","last_unstale":"2023-10-06T05:24:47.344584+0000","last_undegraded":"2023-10-06T05:24:47.344584+0000","last_fullsized":"2023-10-06T05:24:47.344584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:34.867620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351545+0000","last_change":"2023-10-06T05:24:46.334560+0000","last_active":"2023-10-06T05:24:47.351545+0000","last_peered":"2023-10-06T05:24:47.351545+0000","last_clean":"2023-10-06T05:24:47.351545+0000","last_became_active":"2023-10-06T05:24:46.334316+0000","last_became_peered":"2023-10-06T05:24:46.334316+0000","last_unstale":"2023-10-06T05:24:47.351545+0000","last_undegraded":"2023-10-06T05:24:47.351545+0000","last_fullsized":"2023-10-06T05:24:47.351545+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:02.439238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344365+0000","last_change":"2023-10-06T05:24:47.343379+0000","last_active":"2023-10-06T05:24:47.344365+0000","last_peered":"2023-10-06T05:24:47.344365+0000","last_clean":"2023-10-06T05:24:47.344365+0000","last_became_active":"2023-10-06T05:24:47.342741+0000","last_became_peered":"2023-10-06T05:24:47.342741+0000","last_unstale":"2023-10-06T05:24:47.344365+0000","last_undegraded":"2023-10-06T05:24:47.344365+0000","last_fullsized":"2023-10-06T05:24:47.344365+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:48.856362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T05:24:48.332019+0000","last_change":"2023-10-06T05:24:46.333979+0000","last_active":"2023-10-06T05:24:48.332019+0000","last_peered":"2023-10-06T05:24:48.332019+0000","last_clean":"2023-10-06T05:24:48.332019+0000","last_became_active":"2023-10-06T05:24:46.333482+0000","last_became_peered":"2023-10-06T05:24:46.333482+0000","last_unstale":"2023-10-06T05:24:48.332019+0000","last_undegraded":"2023-10-06T05:24:48.332019+0000","last_fullsized":"2023-10-06T05:24:48.332019+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:02.180811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351901+0000","last_change":"2023-10-06T05:24:47.349434+0000","last_active":"2023-10-06T05:24:47.351901+0000","last_peered":"2023-10-06T05:24:47.351901+0000","last_clean":"2023-10-06T05:24:47.351901+0000","last_became_active":"2023-10-06T05:24:47.349152+0000","last_became_peered":"2023-10-06T05:24:47.349152+0000","last_unstale":"2023-10-06T05:24:47.351901+0000","last_undegraded":"2023-10-06T05:24:47.351901+0000","last_fullsized":"2023-10-06T05:24:47.351901+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:37.746041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321140+0000","last_change":"2023-10-06T05:24:46.339888+0000","last_active":"2023-10-06T05:24:47.321140+0000","last_peered":"2023-10-06T05:24:47.321140+0000","last_clean":"2023-10-06T05:24:47.321140+0000","last_became_active":"2023-10-06T05:24:46.339698+0000","last_became_peered":"2023-10-06T05:24:46.339698+0000","last_unstale":"2023-10-06T05:24:47.321140+0000","last_undegraded":"2023-10-06T05:24:47.321140+0000","last_fullsized":"2023-10-06T05:24:47.321140+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:55.161847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344432+0000","last_change":"2023-10-06T05:24:47.339402+0000","last_active":"2023-10-06T05:24:47.344432+0000","last_peered":"2023-10-06T05:24:47.344432+0000","last_clean":"2023-10-06T05:24:47.344432+0000","last_became_active":"2023-10-06T05:24:47.339153+0000","last_became_peered":"2023-10-06T05:24:47.339153+0000","last_unstale":"2023-10-06T05:24:47.344432+0000","last_undegraded":"2023-10-06T05:24:47.344432+0000","last_fullsized":"2023-10-06T05:24:47.344432+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:04.183491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.325457+0000","last_change":"2023-10-06T05:24:46.329841+0000","last_active":"2023-10-06T05:24:47.325457+0000","last_peered":"2023-10-06T05:24:47.325457+0000","last_clean":"2023-10-06T05:24:47.325457+0000","last_became_active":"2023-10-06T05:24:46.329603+0000","last_became_peered":"2023-10-06T05:24:46.329603+0000","last_unstale":"2023-10-06T05:24:47.325457+0000","last_undegraded":"2023-10-06T05:24:47.325457+0000","last_fullsized":"2023-10-06T05:24:47.325457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:55.150672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.336089+0000","last_change":"2023-10-06T05:24:47.334602+0000","last_active":"2023-10-06T05:24:47.336089+0000","last_peered":"2023-10-06T05:24:47.336089+0000","last_clean":"2023-10-06T05:24:47.336089+0000","last_became_active":"2023-10-06T05:24:47.334270+0000","last_became_peered":"2023-10-06T05:24:47.334270+0000","last_unstale":"2023-10-06T05:24:47.336089+0000","last_undegraded":"2023-10-06T05:24:47.336089+0000","last_fullsized":"2023-10-06T05:24:47.336089+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:59.184637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.317461+0000","last_change":"2023-10-06T05:24:46.335114+0000","last_active":"2023-10-06T05:24:47.317461+0000","last_peered":"2023-10-06T05:24:47.317461+0000","last_clean":"2023-10-06T05:24:47.317461+0000","last_became_active":"2023-10-06T05:24:46.334789+0000","last_became_peered":"2023-10-06T05:24:46.334789+0000","last_unstale":"2023-10-06T05:24:47.317461+0000","last_undegraded":"2023-10-06T05:24:47.317461+0000","last_fullsized":"2023-10-06T05:24:47.317461+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.164152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348282+0000","last_change":"2023-10-06T05:24:47.343948+0000","last_active":"2023-10-06T05:24:47.348282+0000","last_peered":"2023-10-06T05:24:47.348282+0000","last_clean":"2023-10-06T05:24:47.348282+0000","last_became_active":"2023-10-06T05:24:47.343648+0000","last_became_peered":"2023-10-06T05:24:47.343648+0000","last_unstale":"2023-10-06T05:24:47.348282+0000","last_undegraded":"2023-10-06T05:24:47.348282+0000","last_fullsized":"2023-10-06T05:24:47.348282+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:15.459769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321358+0000","last_change":"2023-10-06T05:24:46.330481+0000","last_active":"2023-10-06T05:24:47.321358+0000","last_peered":"2023-10-06T05:24:47.321358+0000","last_clean":"2023-10-06T05:24:47.321358+0000","last_became_active":"2023-10-06T05:24:46.330219+0000","last_became_peered":"2023-10-06T05:24:46.330219+0000","last_unstale":"2023-10-06T05:24:47.321358+0000","last_undegraded":"2023-10-06T05:24:47.321358+0000","last_fullsized":"2023-10-06T05:24:47.321358+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:25.567261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.340918+0000","last_change":"2023-10-06T05:24:47.334740+0000","last_active":"2023-10-06T05:24:47.340918+0000","last_peered":"2023-10-06T05:24:47.340918+0000","last_clean":"2023-10-06T05:24:47.340918+0000","last_became_active":"2023-10-06T05:24:47.334446+0000","last_became_peered":"2023-10-06T05:24:47.334446+0000","last_unstale":"2023-10-06T05:24:47.340918+0000","last_undegraded":"2023-10-06T05:24:47.340918+0000","last_fullsized":"2023-10-06T05:24:47.340918+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:53.422415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.323711+0000","last_change":"2023-10-06T05:24:46.334796+0000","last_active":"2023-10-06T05:24:47.323711+0000","last_peered":"2023-10-06T05:24:47.323711+0000","last_clean":"2023-10-06T05:24:47.323711+0000","last_became_active":"2023-10-06T05:24:46.334304+0000","last_became_peered":"2023-10-06T05:24:46.334304+0000","last_unstale":"2023-10-06T05:24:47.323711+0000","last_undegraded":"2023-10-06T05:24:47.323711+0000","last_fullsized":"2023-10-06T05:24:47.323711+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:17.883023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349783+0000","last_change":"2023-10-06T05:24:47.342065+0000","last_active":"2023-10-06T05:24:47.349783+0000","last_peered":"2023-10-06T05:24:47.349783+0000","last_clean":"2023-10-06T05:24:47.349783+0000","last_became_active":"2023-10-06T05:24:47.341786+0000","last_became_peered":"2023-10-06T05:24:47.341786+0000","last_unstale":"2023-10-06T05:24:47.349783+0000","last_undegraded":"2023-10-06T05:24:47.349783+0000","last_fullsized":"2023-10-06T05:24:47.349783+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:34.738014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.327864+0000","last_change":"2023-10-06T05:24:46.339897+0000","last_active":"2023-10-06T05:24:47.327864+0000","last_peered":"2023-10-06T05:24:47.327864+0000","last_clean":"2023-10-06T05:24:47.327864+0000","last_became_active":"2023-10-06T05:24:46.339648+0000","last_became_peered":"2023-10-06T05:24:46.339648+0000","last_unstale":"2023-10-06T05:24:47.327864+0000","last_undegraded":"2023-10-06T05:24:47.327864+0000","last_fullsized":"2023-10-06T05:24:47.327864+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:54.398405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.345250+0000","last_change":"2023-10-06T05:24:47.333042+0000","last_active":"2023-10-06T05:24:47.345250+0000","last_peered":"2023-10-06T05:24:47.345250+0000","last_clean":"2023-10-06T05:24:47.345250+0000","last_became_active":"2023-10-06T05:24:47.332729+0000","last_became_peered":"2023-10-06T05:24:47.332729+0000","last_unstale":"2023-10-06T05:24:47.345250+0000","last_undegraded":"2023-10-06T05:24:47.345250+0000","last_fullsized":"2023-10-06T05:24:47.345250+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:39.100980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.323579+0000","last_change":"2023-10-06T05:24:46.340463+0000","last_active":"2023-10-06T05:24:47.323579+0000","last_peered":"2023-10-06T05:24:47.323579+0000","last_clean":"2023-10-06T05:24:47.323579+0000","last_became_active":"2023-10-06T05:24:46.340179+0000","last_became_peered":"2023-10-06T05:24:46.340179+0000","last_unstale":"2023-10-06T05:24:47.323579+0000","last_undegraded":"2023-10-06T05:24:47.323579+0000","last_fullsized":"2023-10-06T05:24:47.323579+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:46.588480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.325274+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:24:47.325274+0000","last_peered":"2023-10-06T05:24:47.325274+0000","last_clean":"2023-10-06T05:24:47.325274+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:24:47.325274+0000","last_undegraded":"2023-10-06T05:24:47.325274+0000","last_fullsized":"2023-10-06T05:24:47.325274+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344823+0000","last_change":"2023-10-06T05:24:47.337388+0000","last_active":"2023-10-06T05:24:47.344823+0000","last_peered":"2023-10-06T05:24:47.344823+0000","last_clean":"2023-10-06T05:24:47.344823+0000","last_became_active":"2023-10-06T05:24:47.337141+0000","last_became_peered":"2023-10-06T05:24:47.337141+0000","last_unstale":"2023-10-06T05:24:47.344823+0000","last_undegraded":"2023-10-06T05:24:47.344823+0000","last_fullsized":"2023-10-06T05:24:47.344823+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:02.111923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"138'8","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.346089+0000","last_change":"2023-10-06T05:24:46.330721+0000","last_active":"2023-10-06T05:24:47.346089+0000","last_peered":"2023-10-06T05:24:47.346089+0000","last_clean":"2023-10-06T05:24:47.346089+0000","last_became_active":"2023-10-06T05:24:46.330472+0000","last_became_peered":"2023-10-06T05:24:46.330472+0000","last_unstale":"2023-10-06T05:24:47.346089+0000","last_undegraded":"2023-10-06T05:24:47.346089+0000","last_fullsized":"2023-10-06T05:24:47.346089+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:17.883023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.326469+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:24:47.326469+0000","last_peered":"2023-10-06T05:24:47.326469+0000","last_clean":"2023-10-06T05:24:47.326469+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:24:47.326469+0000","last_undegraded":"2023-10-06T05:24:47.326469+0000","last_fullsized":"2023-10-06T05:24:47.326469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.343282+0000","last_change":"2023-10-06T05:24:47.340930+0000","last_active":"2023-10-06T05:24:47.343282+0000","last_peered":"2023-10-06T05:24:47.343282+0000","last_clean":"2023-10-06T05:24:47.343282+0000","last_became_active":"2023-10-06T05:24:47.340278+0000","last_became_peered":"2023-10-06T05:24:47.340278+0000","last_unstale":"2023-10-06T05:24:47.343282+0000","last_undegraded":"2023-10-06T05:24:47.343282+0000","last_fullsized":"2023-10-06T05:24:47.343282+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:03.189536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321400+0000","last_change":"2023-10-06T05:24:46.337393+0000","last_active":"2023-10-06T05:24:47.321400+0000","last_peered":"2023-10-06T05:24:47.321400+0000","last_clean":"2023-10-06T05:24:47.321400+0000","last_became_active":"2023-10-06T05:24:46.337135+0000","last_became_peered":"2023-10-06T05:24:46.337135+0000","last_unstale":"2023-10-06T05:24:47.321400+0000","last_undegraded":"2023-10-06T05:24:47.321400+0000","last_fullsized":"2023-10-06T05:24:47.321400+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:18.098315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.323805+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:24:47.323805+0000","last_peered":"2023-10-06T05:24:47.323805+0000","last_clean":"2023-10-06T05:24:47.323805+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:24:47.323805+0000","last_undegraded":"2023-10-06T05:24:47.323805+0000","last_fullsized":"2023-10-06T05:24:47.323805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.339130+0000","last_change":"2023-10-06T05:24:47.335477+0000","last_active":"2023-10-06T05:24:47.339130+0000","last_peered":"2023-10-06T05:24:47.339130+0000","last_clean":"2023-10-06T05:24:47.339130+0000","last_became_active":"2023-10-06T05:24:47.335176+0000","last_became_peered":"2023-10-06T05:24:47.335176+0000","last_unstale":"2023-10-06T05:24:47.339130+0000","last_undegraded":"2023-10-06T05:24:47.339130+0000","last_fullsized":"2023-10-06T05:24:47.339130+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:55.407129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321132+0000","last_change":"2023-10-06T05:24:46.333768+0000","last_active":"2023-10-06T05:24:47.321132+0000","last_peered":"2023-10-06T05:24:47.321132+0000","last_clean":"2023-10-06T05:24:47.321132+0000","last_became_active":"2023-10-06T05:24:46.333119+0000","last_became_peered":"2023-10-06T05:24:46.333119+0000","last_unstale":"2023-10-06T05:24:47.321132+0000","last_undegraded":"2023-10-06T05:24:47.321132+0000","last_fullsized":"2023-10-06T05:24:47.321132+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:14.451044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.322490+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:24:47.322490+0000","last_peered":"2023-10-06T05:24:47.322490+0000","last_clean":"2023-10-06T05:24:47.322490+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:24:47.322490+0000","last_undegraded":"2023-10-06T05:24:47.322490+0000","last_fullsized":"2023-10-06T05:24:47.322490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349868+0000","last_change":"2023-10-06T05:24:47.341919+0000","last_active":"2023-10-06T05:24:47.349868+0000","last_peered":"2023-10-06T05:24:47.349868+0000","last_clean":"2023-10-06T05:24:47.349868+0000","last_became_active":"2023-10-06T05:24:47.341704+0000","last_became_peered":"2023-10-06T05:24:47.341704+0000","last_unstale":"2023-10-06T05:24:47.349868+0000","last_undegraded":"2023-10-06T05:24:47.349868+0000","last_fullsized":"2023-10-06T05:24:47.349868+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:02.884622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"138'6","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.354645+0000","last_change":"2023-10-06T05:24:46.333318+0000","last_active":"2023-10-06T05:24:47.354645+0000","last_peered":"2023-10-06T05:24:47.354645+0000","last_clean":"2023-10-06T05:24:47.354645+0000","last_became_active":"2023-10-06T05:24:46.333050+0000","last_became_peered":"2023-10-06T05:24:46.333050+0000","last_unstale":"2023-10-06T05:24:47.354645+0000","last_undegraded":"2023-10-06T05:24:47.354645+0000","last_fullsized":"2023-10-06T05:24:47.354645+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:55.684039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":105,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.323987+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:24:47.323987+0000","last_peered":"2023-10-06T05:24:47.323987+0000","last_clean":"2023-10-06T05:24:47.323987+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:24:47.323987+0000","last_undegraded":"2023-10-06T05:24:47.323987+0000","last_fullsized":"2023-10-06T05:24:47.323987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.335184+0000","last_change":"2023-10-06T05:24:47.326406+0000","last_active":"2023-10-06T05:24:47.335184+0000","last_peered":"2023-10-06T05:24:47.335184+0000","last_clean":"2023-10-06T05:24:47.335184+0000","last_became_active":"2023-10-06T05:24:47.326126+0000","last_became_peered":"2023-10-06T05:24:47.326126+0000","last_unstale":"2023-10-06T05:24:47.335184+0000","last_undegraded":"2023-10-06T05:24:47.335184+0000","last_fullsized":"2023-10-06T05:24:47.335184+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:56.968222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"138'3","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349161+0000","last_change":"2023-10-06T05:24:46.332461+0000","last_active":"2023-10-06T05:24:47.349161+0000","last_peered":"2023-10-06T05:24:47.349161+0000","last_clean":"2023-10-06T05:24:47.349161+0000","last_became_active":"2023-10-06T05:24:46.332111+0000","last_became_peered":"2023-10-06T05:24:46.332111+0000","last_unstale":"2023-10-06T05:24:47.349161+0000","last_undegraded":"2023-10-06T05:24:47.349161+0000","last_fullsized":"2023-10-06T05:24:47.349161+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:13.124314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":294,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.322649+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:24:47.322649+0000","last_peered":"2023-10-06T05:24:47.322649+0000","last_clean":"2023-10-06T05:24:47.322649+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:24:47.322649+0000","last_undegraded":"2023-10-06T05:24:47.322649+0000","last_fullsized":"2023-10-06T05:24:47.322649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":118,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.316676+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:24:47.316676+0000","last_peered":"2023-10-06T05:24:47.316676+0000","last_clean":"2023-10-06T05:24:47.316676+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:24:47.316676+0000","last_undegraded":"2023-10-06T05:24:47.316676+0000","last_fullsized":"2023-10-06T05:24:47.316676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344253+0000","last_change":"2023-10-06T05:24:47.335395+0000","last_active":"2023-10-06T05:24:47.344253+0000","last_peered":"2023-10-06T05:24:47.344253+0000","last_clean":"2023-10-06T05:24:47.344253+0000","last_became_active":"2023-10-06T05:24:47.335113+0000","last_became_peered":"2023-10-06T05:24:47.335113+0000","last_unstale":"2023-10-06T05:24:47.344253+0000","last_undegraded":"2023-10-06T05:24:47.344253+0000","last_fullsized":"2023-10-06T05:24:47.344253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:56.177616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321268+0000","last_change":"2023-10-06T05:24:46.333882+0000","last_active":"2023-10-06T05:24:47.321268+0000","last_peered":"2023-10-06T05:24:47.321268+0000","last_clean":"2023-10-06T05:24:47.321268+0000","last_became_active":"2023-10-06T05:24:46.333624+0000","last_became_peered":"2023-10-06T05:24:46.333624+0000","last_unstale":"2023-10-06T05:24:47.321268+0000","last_undegraded":"2023-10-06T05:24:47.321268+0000","last_fullsized":"2023-10-06T05:24:47.321268+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:49.432859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":132,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321592+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:24:47.321592+0000","last_peered":"2023-10-06T05:24:47.321592+0000","last_clean":"2023-10-06T05:24:47.321592+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:24:47.321592+0000","last_undegraded":"2023-10-06T05:24:47.321592+0000","last_fullsized":"2023-10-06T05:24:47.321592+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344542+0000","last_change":"2023-10-06T05:24:47.334579+0000","last_active":"2023-10-06T05:24:47.344542+0000","last_peered":"2023-10-06T05:24:47.344542+0000","last_clean":"2023-10-06T05:24:47.344542+0000","last_became_active":"2023-10-06T05:24:47.334268+0000","last_became_peered":"2023-10-06T05:24:47.334268+0000","last_unstale":"2023-10-06T05:24:47.344542+0000","last_undegraded":"2023-10-06T05:24:47.344542+0000","last_fullsized":"2023-10-06T05:24:47.344542+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:58.540770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351612+0000","last_change":"2023-10-06T05:24:46.333412+0000","last_active":"2023-10-06T05:24:47.351612+0000","last_peered":"2023-10-06T05:24:47.351612+0000","last_clean":"2023-10-06T05:24:47.351612+0000","last_became_active":"2023-10-06T05:24:46.333106+0000","last_became_peered":"2023-10-06T05:24:46.333106+0000","last_unstale":"2023-10-06T05:24:47.351612+0000","last_undegraded":"2023-10-06T05:24:47.351612+0000","last_fullsized":"2023-10-06T05:24:47.351612+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:26.988427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.317712+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:24:47.317712+0000","last_peered":"2023-10-06T05:24:47.317712+0000","last_clean":"2023-10-06T05:24:47.317712+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:24:47.317712+0000","last_undegraded":"2023-10-06T05:24:47.317712+0000","last_fullsized":"2023-10-06T05:24:47.317712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303672,"kb_used_data":2288,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068168,"statfs":{"total":96636764160,"available":96325804032,"internally_reserved":0,"allocated":2342912,"data_stored":1404656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.38700000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.57599999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.87,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67900000000000005,"15min":0.67800000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70399999999999996,"15min":0.74299999999999999},"min":{"1min":0.46899999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":1.056,"15min":1.056},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.55200000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99099999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64100000000000001,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":1.0249999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.77000000000000002},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0629999999999999,"5min":1.27,"15min":1.27},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68999999999999995,"15min":0.73299999999999998},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":0.97899999999999998,"15min":0.999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.74199999999999999,"15min":0.81599999999999995},"min":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.98399999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78900000000000003,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.1040000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79800000000000004,"15min":0.84199999999999997},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.441},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.83699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574948,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303092,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068748,"statfs":{"total":96636764160,"available":96326397952,"internally_reserved":0,"allocated":1748992,"data_stored":814288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65400000000000003,"15min":0.67500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.32300000000000001},"max":{"1min":0.88200000000000001,"5min":1.006,"15min":1.8500000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68000000000000005,"15min":0.69799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90500000000000003,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.177,"5min":1.177,"15min":1.9279999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.998,"5min":1.073,"15min":1.4239999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65500000000000003,"15min":0.65000000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92700000000000005,"5min":1.093,"15min":1.369},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.754},"min":{"1min":0.47799999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.085,"5min":1.085,"15min":1.891},"last":0.86699999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.71699999999999997,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95999999999999996,"5min":1.018,"15min":1.347},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.71599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.97699999999999998,"5min":0.99299999999999999,"15min":1},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69899999999999995,"15min":0.76700000000000002},"min":{"1min":0.40600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.107,"5min":1.1140000000000001,"15min":1.9670000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.81399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.997,"5min":1.278,"15min":1.4510000000000001},"last":1.0209999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.415},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73799999999999999,"15min":0.81000000000000005},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":2},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.76000000000000001,"15min":0.80200000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.032,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.77700000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.46},"last":0.69399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303684,"kb_used_data":2300,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068156,"statfs":{"total":96636764160,"available":96325791744,"internally_reserved":0,"allocated":2355200,"data_stored":1406192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.438},"min":{"1min":0.32900000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57799999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.59599999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.44400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.33900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71399999999999997,"5min":0.77000000000000002,"15min":0.83799999999999997},"last":0.497},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.504,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.35399999999999998,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.38800000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66000000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76500000000000001,"15min":0.78800000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.79900000000000004,"15min":0.79200000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.0189999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.753,"15min":0.746},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.169},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.98699999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.749,"15min":0.747},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.1140000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.80700000000000005,"15min":0.81399999999999995},"min":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.50600000000000001},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":1.044},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.59099999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.103},"last":1.1339999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574948,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":814392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.434,"15min":0.42999999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56899999999999995,"5min":0.71499999999999997,"15min":0.84499999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.48399999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60599999999999998,"5min":0.78900000000000003,"15min":1.042},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.497,"15min":0.48899999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66200000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52400000000000002,"15min":0.51200000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65800000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52000000000000002,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.79700000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0880000000000001,"5min":1.242,"15min":1.242},"last":0.753},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.748,"15min":0.77200000000000002},"min":{"1min":0.54200000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.198,"15min":1.198},"last":0.95099999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76300000000000001,"15min":0.76400000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96499999999999997,"5min":1.26,"15min":1.26},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1299999999999999,"5min":1.163,"15min":1.163},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.999,"5min":1.323,"15min":1.323},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92300000000000004,"5min":1.1799999999999999,"15min":1.3100000000000001},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.82999999999999996},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.52400000000000002},"max":{"1min":0.999,"5min":1.2090000000000001,"15min":1.264},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78700000000000003,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":0.83399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303092,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068748,"statfs":{"total":96636764160,"available":96326397952,"internally_reserved":0,"allocated":1748992,"data_stored":814288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.42699999999999999},"min":{"1min":0.32000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51000000000000001,"5min":0.77400000000000002,"15min":0.77800000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48699999999999999,"15min":0.48399999999999999},"min":{"1min":0.32100000000000001,"5min":0.31,"15min":0.28399999999999997},"max":{"1min":0.60599999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39300000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45500000000000002,"15min":0.46200000000000002},"min":{"1min":0.29899999999999999,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.53700000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.498,"15min":0.5},"min":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.31900000000000001},"max":{"1min":0.61799999999999999,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.55300000000000005,"5min":0.82799999999999996,"15min":0.93600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.372,"15min":0.36199999999999999},"max":{"1min":0.626,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73199999999999998,"15min":0.753},"min":{"1min":0.58899999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88800000000000001,"5min":1.022,"15min":1.1299999999999999},"last":0.997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73399999999999999,"15min":0.78600000000000003},"min":{"1min":0.52800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92200000000000004,"5min":1.085,"15min":1.1699999999999999},"last":1.018}]},{"osd":5,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73499999999999999,"15min":0.77600000000000002},"min":{"1min":0.496,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.042,"15min":1.099},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.747,"15min":0.79500000000000004},"min":{"1min":0.56499999999999995,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97899999999999998,"5min":1.0649999999999999,"15min":1.163},"last":1.052}]},{"osd":6,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76800000000000002,"15min":0.82299999999999995},"min":{"1min":0.60599999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96299999999999997,"5min":1.0960000000000001,"15min":1.21},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.76600000000000001,"15min":0.80500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.876,"5min":1.022,"15min":1.042},"last":1.0860000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77900000000000003,"15min":0.80200000000000005},"min":{"1min":0.64400000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.997,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79000000000000004,"15min":0.84799999999999998},"min":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.93899999999999995,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":0.97099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1769472,"data_stored":816400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.58899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47099999999999997,"15min":0.46800000000000003},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60399999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.501}]},{"osd":2,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70299999999999996,"5min":0.999,"15min":0.999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.63200000000000001,"5min":0.98999999999999999,"15min":1.014},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0349999999999999,"5min":1.161,"15min":1.2669999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.004,"5min":1.1830000000000001,"15min":1.3540000000000001},"last":0.83599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.754,"15min":0.76400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.093,"5min":1.147,"15min":1.204},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.208,"5min":1.284,"15min":1.329},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77900000000000003,"15min":0.80800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92700000000000005,"5min":1.0880000000000001,"15min":1.484},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0169999999999999,"5min":1.2809999999999999,"15min":1.629},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81000000000000005,"15min":0.84499999999999997},"min":{"1min":0.58299999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1379999999999999,"5min":1.236,"15min":1.5620000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.624,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":0.96399999999999997,"5min":1.194,"15min":1.325},"last":0.86299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303108,"kb_used_data":1724,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068732,"statfs":{"total":96636764160,"available":96326381568,"internally_reserved":0,"allocated":1765376,"data_stored":815024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.51700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89400000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.76800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.02,"5min":1.02,"15min":2.77},"last":0.95899999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69899999999999995,"15min":0.73299999999999998},"min":{"1min":0.48799999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.6419999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68899999999999995,"15min":0.71499999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.85799999999999998,"5min":1.1100000000000001,"15min":1.1899999999999999},"last":0.99199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.497,"5min":0.47599999999999998,"15min":0.39800000000000002},"max":{"1min":0.79400000000000004,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72499999999999998,"15min":0.80300000000000005},"min":{"1min":0.57399999999999995,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97599999999999998,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71399999999999997,"15min":0.71699999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88800000000000001,"5min":0.95899999999999996,"15min":1.0629999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.996},"min":{"1min":0.56399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":7.742},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.73199999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93700000000000006,"5min":0.96499999999999997,"15min":1.079},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.747},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.03,"5min":1.04,"15min":1.181},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.73099999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98299999999999998,"5min":1.0029999999999999,"15min":1.2669999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76100000000000001,"15min":0.81200000000000006},"min":{"1min":0.56399999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.093,"5min":1.093,"15min":1.171},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.82199999999999995},"min":{"1min":0.504,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.016,"5min":1.1899999999999999,"15min":1.228},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77300000000000002,"15min":0.84599999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.571},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574948,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":814412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65400000000000003,"15min":0.91000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.98199999999999998,"5min":1.109,"15min":9.2780000000000005},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.91300000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.93500000000000005},"min":{"1min":0.52500000000000002,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":1.0329999999999999,"5min":1.079,"15min":9.25},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.98899999999999999},"min":{"1min":0.46500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93200000000000005,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66400000000000003,"15min":0.95999999999999996},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98499999999999999,"5min":1.052,"15min":10.384},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.98699999999999999},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.097,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":1.0249999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.495},"max":{"1min":0.88200000000000001,"5min":1.76,"15min":10.205},"last":0.753},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.55300000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.097,"15min":1.292},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68300000000000005,"15min":0.70099999999999996},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70699999999999996,"15min":0.76500000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.008,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.876}]},{"osd":6,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70299999999999996,"15min":0.73499999999999999},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90900000000000003,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.80500000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95899999999999996,"5min":1.008,"15min":1.298},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.81799999999999995},"min":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0529999999999999,"5min":1.556,"15min":1.556},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.80700000000000005},"min":{"1min":0.498,"5min":0.432,"15min":0.432},"max":{"1min":1.04,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.79700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:24:51.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:24:51.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.952+0000 7f55aafff700 1 Processor -- start 2023-10-06T05:24:51.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.952+0000 7f55aafff700 1 -- start start 2023-10-06T05:24:51.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.953+0000 7f55aafff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55a413fda0 0x7f55a4069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.953+0000 7f55aafff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a40ca5b0 con 0x7f55a40c7f30 2023-10-06T05:24:51.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.953+0000 7f55aafff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a406f740 con 0x7f55a40c7b60 2023-10-06T05:24:51.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.953+0000 7f55aafff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a40c9c80 con 0x7f55a413fda0 2023-10-06T05:24:51.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.954+0000 7f55a3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f55a40c7f30 0x7f55a413f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42338/0 (socket says 172.21.15.31:42338) 2023-10-06T05:24:51.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.954+0000 7f55a3fff700 1 -- 172.21.15.31:0/3664284785 learned_addr learned my addr 172.21.15.31:0/3664284785 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:51.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.954+0000 7f55a959c700 1 --2- 172.21.15.31:0/3664284785 >> v2:172.21.15.31:3300/0 conn(0x7f55a413fda0 0x7f55a4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.954+0000 7f55a959c700 1 -- 172.21.15.31:0/3664284785 >> v1:172.21.15.62:6789/0 conn(0x7f55a40c7b60 legacy=0x7f55a406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.954+0000 7f55a959c700 1 -- 172.21.15.31:0/3664284785 >> v1:172.21.15.31:6789/0 conn(0x7f55a40c7f30 legacy=0x7f55a413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.954+0000 7f55a959c700 1 -- 172.21.15.31:0/3664284785 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a40c9f60 con 0x7f55a413fda0 2023-10-06T05:24:51.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.954+0000 7f55a959c700 1 --2- 172.21.15.31:0/3664284785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a413fda0 0x7f55a4069880 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f559c009e60 tx=0x7f559c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f932dcb70fa943a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.955+0000 7f55a37fe700 1 -- 172.21.15.31:0/3664284785 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c00e070 con 0x7f55a413fda0 2023-10-06T05:24:51.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.955+0000 7f55a37fe700 1 -- 172.21.15.31:0/3664284785 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559c00bc30 con 0x7f55a413fda0 2023-10-06T05:24:51.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.955+0000 7f55a37fe700 1 -- 172.21.15.31:0/3664284785 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c012430 con 0x7f55a413fda0 2023-10-06T05:24:51.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.955+0000 7f55aafff700 1 -- 172.21.15.31:0/3664284785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a413fda0 msgr2=0x7f55a4069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.955+0000 7f55aafff700 1 --2- 172.21.15.31:0/3664284785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a413fda0 0x7f55a4069880 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f559c009e60 tx=0x7f559c009780 comp rx=0 tx=0).stop 2023-10-06T05:24:51.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.955+0000 7f55aafff700 1 -- 172.21.15.31:0/3664284785 shutdown_connections 2023-10-06T05:24:51.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.955+0000 7f55aafff700 1 --2- 172.21.15.31:0/3664284785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a413fda0 0x7f55a4069880 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.955+0000 7f55aafff700 1 -- 172.21.15.31:0/3664284785 >> 172.21.15.31:0/3664284785 conn(0x7f55a40cc0b0 msgr2=0x7f55a40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:51.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.955+0000 7f55aafff700 1 -- 172.21.15.31:0/3664284785 shutdown_connections 2023-10-06T05:24:51.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.955+0000 7f55aafff700 1 -- 172.21.15.31:0/3664284785 wait complete. 2023-10-06T05:24:51.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.957+0000 7f55aafff700 1 Processor -- start 2023-10-06T05:24:51.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.957+0000 7f55aafff700 1 -- start start 2023-10-06T05:24:51.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55aafff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55a40c7b60 0x7f55a40ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55aafff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a40c7f30 0x7f55a4159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55aafff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55a413fda0 0x7f55a413e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55aafff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40ae520 con 0x7f55a40c7f30 2023-10-06T05:24:51.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55aafff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40ae240 con 0x7f55a413fda0 2023-10-06T05:24:51.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55aafff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40ab830 con 0x7f55a40c7b60 2023-10-06T05:24:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a40c7f30 0x7f55a4159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a40c7f30 0x7f55a4159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55612/0 (socket says 172.21.15.31:55612) 2023-10-06T05:24:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55a3fff700 1 -- 172.21.15.31:0/2156902276 learned_addr learned my addr 172.21.15.31:0/2156902276 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55a8d9b700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55a40c7b60 0x7f55a40ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.958+0000 7f55a959c700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55a413fda0 0x7f55a413e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55a3fff700 1 -- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55a40c7b60 msgr2=0x7f55a40ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55a3fff700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55a40c7b60 0x7f55a40ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55a3fff700 1 -- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55a413fda0 msgr2=0x7f55a413e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:51.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55a3fff700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55a413fda0 0x7f55a413e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55a3fff700 1 -- 172.21.15.31:0/2156902276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c009a20 con 0x7f55a40c7f30 2023-10-06T05:24:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55a3fff700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a40c7f30 0x7f55a4159e50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f559000c7c0 tx=0x7f5590007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55a0ff9700 1 -- 172.21.15.31:0/2156902276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590005b30 con 0x7f55a40c7f30 2023-10-06T05:24:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55a0ff9700 1 -- 172.21.15.31:0/2156902276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5590005c90 con 0x7f55a40c7f30 2023-10-06T05:24:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55aafff700 1 -- 172.21.15.31:0/2156902276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a40abb10 con 0x7f55a40c7f30 2023-10-06T05:24:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55a0ff9700 1 -- 172.21.15.31:0/2156902276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590004a80 con 0x7f55a40c7f30 2023-10-06T05:24:51.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.959+0000 7f55aafff700 1 -- 172.21.15.31:0/2156902276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a40adf30 con 0x7f55a40c7f30 2023-10-06T05:24:51.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.960+0000 7f55aafff700 1 -- 172.21.15.31:0/2156902276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a40ac030 con 0x7f55a40c7f30 2023-10-06T05:24:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.961+0000 7f55a0ff9700 1 -- 172.21.15.31:0/2156902276 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f55900070d0 con 0x7f55a40c7f30 2023-10-06T05:24:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.962+0000 7f55a0ff9700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55880674d0 0x7f5588069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.962+0000 7f55a0ff9700 1 -- 172.21.15.31:0/2156902276 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f5590084410 con 0x7f55a40c7f30 2023-10-06T05:24:51.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.963+0000 7f55a8d9b700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55880674d0 0x7f5588069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:51.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.963+0000 7f55a8d9b700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55880674d0 0x7f5588069980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5598000fb0 tx=0x7f5598007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:51.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:51.967+0000 7f55a0ff9700 1 -- 172.21.15.31:0/2156902276 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f559004b5a0 con 0x7f55a40c7f30 2023-10-06T05:24:52.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.169+0000 7f55aafff700 1 -- 172.21.15.31:0/2156902276 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55a404c380 con 0x7f55880674d0 2023-10-06T05:24:52.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.180+0000 7f55a0ff9700 1 -- 172.21.15.31:0/2156902276 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311588 (secure 0 0 0) 0x7f55a404c380 con 0x7f55880674d0 2023-10-06T05:24:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:52.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 -- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55880674d0 msgr2=0x7f5588069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55880674d0 0x7f5588069980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5598000fb0 tx=0x7f5598007e90 comp rx=0 tx=0).stop 2023-10-06T05:24:52.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 -- 172.21.15.31:0/2156902276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a40c7f30 msgr2=0x7f55a4159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a40c7f30 0x7f55a4159e50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f559000c7c0 tx=0x7f5590007f70 comp rx=0 tx=0).stop 2023-10-06T05:24:52.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 -- 172.21.15.31:0/2156902276 shutdown_connections 2023-10-06T05:24:52.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f55880674d0 0x7f5588069980 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55a413fda0 0x7f55a413e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55a40c7b60 0x7f55a40ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 --2- 172.21.15.31:0/2156902276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a40c7f30 0x7f55a4159e50 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 -- 172.21.15.31:0/2156902276 >> 172.21.15.31:0/2156902276 conn(0x7f55a40cc0b0 msgr2=0x7f55a415b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 -- 172.21.15.31:0/2156902276 shutdown_connections 2023-10-06T05:24:52.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.184+0000 7f55aafff700 1 -- 172.21.15.31:0/2156902276 wait complete. 2023-10-06T05:24:52.189 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:24:52.203 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2023-10-06T05:24:51.507952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425960,"kb_used_data":14888,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548760,"statfs":{"total":773094113280,"available":770609930240,"internally_reserved":0,"allocated":15245312,"data_stored":7699652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001358"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.333968+0000","last_change":"2023-10-06T05:24:47.332457+0000","last_active":"2023-10-06T05:24:47.333968+0000","last_peered":"2023-10-06T05:24:47.333968+0000","last_clean":"2023-10-06T05:24:47.333968+0000","last_became_active":"2023-10-06T05:24:47.332187+0000","last_became_peered":"2023-10-06T05:24:47.332187+0000","last_unstale":"2023-10-06T05:24:47.333968+0000","last_undegraded":"2023-10-06T05:24:47.333968+0000","last_fullsized":"2023-10-06T05:24:47.333968+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:52.030713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.338204+0000","last_change":"2023-10-06T05:24:47.335225+0000","last_active":"2023-10-06T05:24:47.338204+0000","last_peered":"2023-10-06T05:24:47.338204+0000","last_clean":"2023-10-06T05:24:47.338204+0000","last_became_active":"2023-10-06T05:24:47.334471+0000","last_became_peered":"2023-10-06T05:24:47.334471+0000","last_unstale":"2023-10-06T05:24:47.338204+0000","last_undegraded":"2023-10-06T05:24:47.338204+0000","last_fullsized":"2023-10-06T05:24:47.338204+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:58.514634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349104+0000","last_change":"2023-10-06T05:24:47.341601+0000","last_active":"2023-10-06T05:24:47.349104+0000","last_peered":"2023-10-06T05:24:47.349104+0000","last_clean":"2023-10-06T05:24:47.349104+0000","last_became_active":"2023-10-06T05:24:47.341291+0000","last_became_peered":"2023-10-06T05:24:47.341291+0000","last_unstale":"2023-10-06T05:24:47.349104+0000","last_undegraded":"2023-10-06T05:24:47.349104+0000","last_fullsized":"2023-10-06T05:24:47.349104+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:16.023915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.333054+0000","last_change":"2023-10-06T05:24:47.326865+0000","last_active":"2023-10-06T05:24:47.333054+0000","last_peered":"2023-10-06T05:24:47.333054+0000","last_clean":"2023-10-06T05:24:47.333054+0000","last_became_active":"2023-10-06T05:24:47.326511+0000","last_became_peered":"2023-10-06T05:24:47.326511+0000","last_unstale":"2023-10-06T05:24:47.333054+0000","last_undegraded":"2023-10-06T05:24:47.333054+0000","last_fullsized":"2023-10-06T05:24:47.333054+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:18.401365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349879+0000","last_change":"2023-10-06T05:24:47.337371+0000","last_active":"2023-10-06T05:24:47.349879+0000","last_peered":"2023-10-06T05:24:47.349879+0000","last_clean":"2023-10-06T05:24:47.349879+0000","last_became_active":"2023-10-06T05:24:47.337097+0000","last_became_peered":"2023-10-06T05:24:47.337097+0000","last_unstale":"2023-10-06T05:24:47.349879+0000","last_undegraded":"2023-10-06T05:24:47.349879+0000","last_fullsized":"2023-10-06T05:24:47.349879+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:16:58.514634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.338574+0000","last_change":"2023-10-06T05:24:47.336850+0000","last_active":"2023-10-06T05:24:47.338574+0000","last_peered":"2023-10-06T05:24:47.338574+0000","last_clean":"2023-10-06T05:24:47.338574+0000","last_became_active":"2023-10-06T05:24:47.336541+0000","last_became_peered":"2023-10-06T05:24:47.336541+0000","last_unstale":"2023-10-06T05:24:47.338574+0000","last_undegraded":"2023-10-06T05:24:47.338574+0000","last_fullsized":"2023-10-06T05:24:47.338574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:26.575986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344381+0000","last_change":"2023-10-06T05:24:47.343543+0000","last_active":"2023-10-06T05:24:47.344381+0000","last_peered":"2023-10-06T05:24:47.344381+0000","last_clean":"2023-10-06T05:24:47.344381+0000","last_became_active":"2023-10-06T05:24:47.343084+0000","last_became_peered":"2023-10-06T05:24:47.343084+0000","last_unstale":"2023-10-06T05:24:47.344381+0000","last_undegraded":"2023-10-06T05:24:47.344381+0000","last_fullsized":"2023-10-06T05:24:47.344381+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:59.864576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344658+0000","last_change":"2023-10-06T05:24:47.325215+0000","last_active":"2023-10-06T05:24:47.344658+0000","last_peered":"2023-10-06T05:24:47.344658+0000","last_clean":"2023-10-06T05:24:47.344658+0000","last_became_active":"2023-10-06T05:24:47.324885+0000","last_became_peered":"2023-10-06T05:24:47.324885+0000","last_unstale":"2023-10-06T05:24:47.344658+0000","last_undegraded":"2023-10-06T05:24:47.344658+0000","last_fullsized":"2023-10-06T05:24:47.344658+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:57.933932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344265+0000","last_change":"2023-10-06T05:24:47.335108+0000","last_active":"2023-10-06T05:24:47.344265+0000","last_peered":"2023-10-06T05:24:47.344265+0000","last_clean":"2023-10-06T05:24:47.344265+0000","last_became_active":"2023-10-06T05:24:47.334459+0000","last_became_peered":"2023-10-06T05:24:47.334459+0000","last_unstale":"2023-10-06T05:24:47.344265+0000","last_undegraded":"2023-10-06T05:24:47.344265+0000","last_fullsized":"2023-10-06T05:24:47.344265+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:39.100980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.338081+0000","last_change":"2023-10-06T05:24:47.323198+0000","last_active":"2023-10-06T05:24:47.338081+0000","last_peered":"2023-10-06T05:24:47.338081+0000","last_clean":"2023-10-06T05:24:47.338081+0000","last_became_active":"2023-10-06T05:24:47.322941+0000","last_became_peered":"2023-10-06T05:24:47.322941+0000","last_unstale":"2023-10-06T05:24:47.338081+0000","last_undegraded":"2023-10-06T05:24:47.338081+0000","last_fullsized":"2023-10-06T05:24:47.338081+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:59.334226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344436+0000","last_change":"2023-10-06T05:24:47.336513+0000","last_active":"2023-10-06T05:24:47.344436+0000","last_peered":"2023-10-06T05:24:47.344436+0000","last_clean":"2023-10-06T05:24:47.344436+0000","last_became_active":"2023-10-06T05:24:47.336113+0000","last_became_peered":"2023-10-06T05:24:47.336113+0000","last_unstale":"2023-10-06T05:24:47.344436+0000","last_undegraded":"2023-10-06T05:24:47.344436+0000","last_fullsized":"2023-10-06T05:24:47.344436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:48.856362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.339222+0000","last_change":"2023-10-06T05:24:47.337114+0000","last_active":"2023-10-06T05:24:47.339222+0000","last_peered":"2023-10-06T05:24:47.339222+0000","last_clean":"2023-10-06T05:24:47.339222+0000","last_became_active":"2023-10-06T05:24:47.336815+0000","last_became_peered":"2023-10-06T05:24:47.336815+0000","last_unstale":"2023-10-06T05:24:47.339222+0000","last_undegraded":"2023-10-06T05:24:47.339222+0000","last_fullsized":"2023-10-06T05:24:47.339222+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:56.692763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.335523+0000","last_change":"2023-10-06T05:24:47.331372+0000","last_active":"2023-10-06T05:24:47.335523+0000","last_peered":"2023-10-06T05:24:47.335523+0000","last_clean":"2023-10-06T05:24:47.335523+0000","last_became_active":"2023-10-06T05:24:47.331110+0000","last_became_peered":"2023-10-06T05:24:47.331110+0000","last_unstale":"2023-10-06T05:24:47.335523+0000","last_undegraded":"2023-10-06T05:24:47.335523+0000","last_fullsized":"2023-10-06T05:24:47.335523+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:55.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351664+0000","last_change":"2023-10-06T05:24:47.342447+0000","last_active":"2023-10-06T05:24:47.351664+0000","last_peered":"2023-10-06T05:24:47.351664+0000","last_clean":"2023-10-06T05:24:47.351664+0000","last_became_active":"2023-10-06T05:24:47.342115+0000","last_became_peered":"2023-10-06T05:24:47.342115+0000","last_unstale":"2023-10-06T05:24:47.351664+0000","last_undegraded":"2023-10-06T05:24:47.351664+0000","last_fullsized":"2023-10-06T05:24:47.351664+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:16.072487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348513+0000","last_change":"2023-10-06T05:24:47.347168+0000","last_active":"2023-10-06T05:24:47.348513+0000","last_peered":"2023-10-06T05:24:47.348513+0000","last_clean":"2023-10-06T05:24:47.348513+0000","last_became_active":"2023-10-06T05:24:47.346844+0000","last_became_peered":"2023-10-06T05:24:47.346844+0000","last_unstale":"2023-10-06T05:24:47.348513+0000","last_undegraded":"2023-10-06T05:24:47.348513+0000","last_fullsized":"2023-10-06T05:24:47.348513+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:28.298824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.333456+0000","last_change":"2023-10-06T05:24:47.327590+0000","last_active":"2023-10-06T05:24:47.333456+0000","last_peered":"2023-10-06T05:24:47.333456+0000","last_clean":"2023-10-06T05:24:47.333456+0000","last_became_active":"2023-10-06T05:24:47.327220+0000","last_became_peered":"2023-10-06T05:24:47.327220+0000","last_unstale":"2023-10-06T05:24:47.333456+0000","last_undegraded":"2023-10-06T05:24:47.333456+0000","last_fullsized":"2023-10-06T05:24:47.333456+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:04.183491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344289+0000","last_change":"2023-10-06T05:24:47.335289+0000","last_active":"2023-10-06T05:24:47.344289+0000","last_peered":"2023-10-06T05:24:47.344289+0000","last_clean":"2023-10-06T05:24:47.344289+0000","last_became_active":"2023-10-06T05:24:47.334827+0000","last_became_peered":"2023-10-06T05:24:47.334827+0000","last_unstale":"2023-10-06T05:24:47.344289+0000","last_undegraded":"2023-10-06T05:24:47.344289+0000","last_fullsized":"2023-10-06T05:24:47.344289+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:34.867620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348079+0000","last_change":"2023-10-06T05:24:47.346575+0000","last_active":"2023-10-06T05:24:47.348079+0000","last_peered":"2023-10-06T05:24:47.348079+0000","last_clean":"2023-10-06T05:24:47.348079+0000","last_became_active":"2023-10-06T05:24:47.346288+0000","last_became_peered":"2023-10-06T05:24:47.346288+0000","last_unstale":"2023-10-06T05:24:47.348079+0000","last_undegraded":"2023-10-06T05:24:47.348079+0000","last_fullsized":"2023-10-06T05:24:47.348079+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:04.821590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348484+0000","last_change":"2023-10-06T05:24:47.347119+0000","last_active":"2023-10-06T05:24:47.348484+0000","last_peered":"2023-10-06T05:24:47.348484+0000","last_clean":"2023-10-06T05:24:47.348484+0000","last_became_active":"2023-10-06T05:24:47.346692+0000","last_became_peered":"2023-10-06T05:24:47.346692+0000","last_unstale":"2023-10-06T05:24:47.348484+0000","last_undegraded":"2023-10-06T05:24:47.348484+0000","last_fullsized":"2023-10-06T05:24:47.348484+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:57.933932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344646+0000","last_change":"2023-10-06T05:24:47.339163+0000","last_active":"2023-10-06T05:24:47.344646+0000","last_peered":"2023-10-06T05:24:47.344646+0000","last_clean":"2023-10-06T05:24:47.344646+0000","last_became_active":"2023-10-06T05:24:47.338824+0000","last_became_peered":"2023-10-06T05:24:47.338824+0000","last_unstale":"2023-10-06T05:24:47.344646+0000","last_undegraded":"2023-10-06T05:24:47.344646+0000","last_fullsized":"2023-10-06T05:24:47.344646+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:29.399416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.340994+0000","last_change":"2023-10-06T05:24:47.335344+0000","last_active":"2023-10-06T05:24:47.340994+0000","last_peered":"2023-10-06T05:24:47.340994+0000","last_clean":"2023-10-06T05:24:47.340994+0000","last_became_active":"2023-10-06T05:24:47.335053+0000","last_became_peered":"2023-10-06T05:24:47.335053+0000","last_unstale":"2023-10-06T05:24:47.340994+0000","last_undegraded":"2023-10-06T05:24:47.340994+0000","last_fullsized":"2023-10-06T05:24:47.340994+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:13.343995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.345963+0000","last_change":"2023-10-06T05:24:47.342356+0000","last_active":"2023-10-06T05:24:47.345963+0000","last_peered":"2023-10-06T05:24:47.345963+0000","last_clean":"2023-10-06T05:24:47.345963+0000","last_became_active":"2023-10-06T05:24:47.342072+0000","last_became_peered":"2023-10-06T05:24:47.342072+0000","last_unstale":"2023-10-06T05:24:47.345963+0000","last_undegraded":"2023-10-06T05:24:47.345963+0000","last_fullsized":"2023-10-06T05:24:47.345963+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:33.341535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351792+0000","last_change":"2023-10-06T05:24:47.345949+0000","last_active":"2023-10-06T05:24:47.351792+0000","last_peered":"2023-10-06T05:24:47.351792+0000","last_clean":"2023-10-06T05:24:47.351792+0000","last_became_active":"2023-10-06T05:24:47.345653+0000","last_became_peered":"2023-10-06T05:24:47.345653+0000","last_unstale":"2023-10-06T05:24:47.351792+0000","last_undegraded":"2023-10-06T05:24:47.351792+0000","last_fullsized":"2023-10-06T05:24:47.351792+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:48.148918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.333435+0000","last_change":"2023-10-06T05:24:47.327448+0000","last_active":"2023-10-06T05:24:47.333435+0000","last_peered":"2023-10-06T05:24:47.333435+0000","last_clean":"2023-10-06T05:24:47.333435+0000","last_became_active":"2023-10-06T05:24:47.327156+0000","last_became_peered":"2023-10-06T05:24:47.327156+0000","last_unstale":"2023-10-06T05:24:47.333435+0000","last_undegraded":"2023-10-06T05:24:47.333435+0000","last_fullsized":"2023-10-06T05:24:47.333435+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:30.384201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.340894+0000","last_change":"2023-10-06T05:24:47.336880+0000","last_active":"2023-10-06T05:24:47.340894+0000","last_peered":"2023-10-06T05:24:47.340894+0000","last_clean":"2023-10-06T05:24:47.340894+0000","last_became_active":"2023-10-06T05:24:47.336573+0000","last_became_peered":"2023-10-06T05:24:47.336573+0000","last_unstale":"2023-10-06T05:24:47.340894+0000","last_undegraded":"2023-10-06T05:24:47.340894+0000","last_fullsized":"2023-10-06T05:24:47.340894+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:10.768676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344553+0000","last_change":"2023-10-06T05:24:47.336807+0000","last_active":"2023-10-06T05:24:47.344553+0000","last_peered":"2023-10-06T05:24:47.344553+0000","last_clean":"2023-10-06T05:24:47.344553+0000","last_became_active":"2023-10-06T05:24:47.336564+0000","last_became_peered":"2023-10-06T05:24:47.336564+0000","last_unstale":"2023-10-06T05:24:47.344553+0000","last_undegraded":"2023-10-06T05:24:47.344553+0000","last_fullsized":"2023-10-06T05:24:47.344553+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:41.071296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348263+0000","last_change":"2023-10-06T05:24:47.343654+0000","last_active":"2023-10-06T05:24:47.348263+0000","last_peered":"2023-10-06T05:24:47.348263+0000","last_clean":"2023-10-06T05:24:47.348263+0000","last_became_active":"2023-10-06T05:24:47.343242+0000","last_became_peered":"2023-10-06T05:24:47.343242+0000","last_unstale":"2023-10-06T05:24:47.348263+0000","last_undegraded":"2023-10-06T05:24:47.348263+0000","last_fullsized":"2023-10-06T05:24:47.348263+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:48.148918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.346175+0000","last_change":"2023-10-06T05:24:47.337090+0000","last_active":"2023-10-06T05:24:47.346175+0000","last_peered":"2023-10-06T05:24:47.346175+0000","last_clean":"2023-10-06T05:24:47.346175+0000","last_became_active":"2023-10-06T05:24:47.335577+0000","last_became_peered":"2023-10-06T05:24:47.335577+0000","last_unstale":"2023-10-06T05:24:47.346175+0000","last_undegraded":"2023-10-06T05:24:47.346175+0000","last_fullsized":"2023-10-06T05:24:47.346175+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:41.663801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.340194+0000","last_change":"2023-10-06T05:24:47.339113+0000","last_active":"2023-10-06T05:24:47.340194+0000","last_peered":"2023-10-06T05:24:47.340194+0000","last_clean":"2023-10-06T05:24:47.340194+0000","last_became_active":"2023-10-06T05:24:47.338787+0000","last_became_peered":"2023-10-06T05:24:47.338787+0000","last_unstale":"2023-10-06T05:24:47.340194+0000","last_undegraded":"2023-10-06T05:24:47.340194+0000","last_fullsized":"2023-10-06T05:24:47.340194+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:26.414974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348952+0000","last_change":"2023-10-06T05:24:47.347188+0000","last_active":"2023-10-06T05:24:47.348952+0000","last_peered":"2023-10-06T05:24:47.348952+0000","last_clean":"2023-10-06T05:24:47.348952+0000","last_became_active":"2023-10-06T05:24:47.346912+0000","last_became_peered":"2023-10-06T05:24:47.346912+0000","last_unstale":"2023-10-06T05:24:47.348952+0000","last_undegraded":"2023-10-06T05:24:47.348952+0000","last_fullsized":"2023-10-06T05:24:47.348952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:54.963898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.345285+0000","last_change":"2023-10-06T05:24:47.334182+0000","last_active":"2023-10-06T05:24:47.345285+0000","last_peered":"2023-10-06T05:24:47.345285+0000","last_clean":"2023-10-06T05:24:47.345285+0000","last_became_active":"2023-10-06T05:24:47.333671+0000","last_became_peered":"2023-10-06T05:24:47.333671+0000","last_unstale":"2023-10-06T05:24:47.345285+0000","last_undegraded":"2023-10-06T05:24:47.345285+0000","last_fullsized":"2023-10-06T05:24:47.345285+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:56.177616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348011+0000","last_change":"2023-10-06T05:24:47.345904+0000","last_active":"2023-10-06T05:24:47.348011+0000","last_peered":"2023-10-06T05:24:47.348011+0000","last_clean":"2023-10-06T05:24:47.348011+0000","last_became_active":"2023-10-06T05:24:47.345613+0000","last_became_peered":"2023-10-06T05:24:47.345613+0000","last_unstale":"2023-10-06T05:24:47.348011+0000","last_undegraded":"2023-10-06T05:24:47.348011+0000","last_fullsized":"2023-10-06T05:24:47.348011+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:34:09.491372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.343302+0000","last_change":"2023-10-06T05:24:47.341126+0000","last_active":"2023-10-06T05:24:47.343302+0000","last_peered":"2023-10-06T05:24:47.343302+0000","last_clean":"2023-10-06T05:24:47.343302+0000","last_became_active":"2023-10-06T05:24:47.340663+0000","last_became_peered":"2023-10-06T05:24:47.340663+0000","last_unstale":"2023-10-06T05:24:47.343302+0000","last_undegraded":"2023-10-06T05:24:47.343302+0000","last_fullsized":"2023-10-06T05:24:47.343302+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:56.170571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"138'4","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349096+0000","last_change":"2023-10-06T05:24:46.332641+0000","last_active":"2023-10-06T05:24:47.349096+0000","last_peered":"2023-10-06T05:24:47.349096+0000","last_clean":"2023-10-06T05:24:47.349096+0000","last_became_active":"2023-10-06T05:24:46.332313+0000","last_became_peered":"2023-10-06T05:24:46.332313+0000","last_unstale":"2023-10-06T05:24:47.349096+0000","last_undegraded":"2023-10-06T05:24:47.349096+0000","last_fullsized":"2023-10-06T05:24:47.349096+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:16.966888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.342199+0000","last_change":"2023-10-06T05:24:47.333533+0000","last_active":"2023-10-06T05:24:47.342199+0000","last_peered":"2023-10-06T05:24:47.342199+0000","last_clean":"2023-10-06T05:24:47.342199+0000","last_became_active":"2023-10-06T05:24:47.333273+0000","last_became_peered":"2023-10-06T05:24:47.333273+0000","last_unstale":"2023-10-06T05:24:47.342199+0000","last_undegraded":"2023-10-06T05:24:47.342199+0000","last_fullsized":"2023-10-06T05:24:47.342199+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:06.143271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351401+0000","last_change":"2023-10-06T05:24:46.336614+0000","last_active":"2023-10-06T05:24:47.351401+0000","last_peered":"2023-10-06T05:24:47.351401+0000","last_clean":"2023-10-06T05:24:47.351401+0000","last_became_active":"2023-10-06T05:24:46.336336+0000","last_became_peered":"2023-10-06T05:24:46.336336+0000","last_unstale":"2023-10-06T05:24:47.351401+0000","last_undegraded":"2023-10-06T05:24:47.351401+0000","last_fullsized":"2023-10-06T05:24:47.351401+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:46.165907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351682+0000","last_change":"2023-10-06T05:24:47.342701+0000","last_active":"2023-10-06T05:24:47.351682+0000","last_peered":"2023-10-06T05:24:47.351682+0000","last_clean":"2023-10-06T05:24:47.351682+0000","last_became_active":"2023-10-06T05:24:47.341833+0000","last_became_peered":"2023-10-06T05:24:47.341833+0000","last_unstale":"2023-10-06T05:24:47.351682+0000","last_undegraded":"2023-10-06T05:24:47.351682+0000","last_fullsized":"2023-10-06T05:24:47.351682+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:15.459769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.350664+0000","last_change":"2023-10-06T05:24:46.334143+0000","last_active":"2023-10-06T05:24:47.350664+0000","last_peered":"2023-10-06T05:24:47.350664+0000","last_clean":"2023-10-06T05:24:47.350664+0000","last_became_active":"2023-10-06T05:24:46.333594+0000","last_became_peered":"2023-10-06T05:24:46.333594+0000","last_unstale":"2023-10-06T05:24:47.350664+0000","last_undegraded":"2023-10-06T05:24:47.350664+0000","last_fullsized":"2023-10-06T05:24:47.350664+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:36.402888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348230+0000","last_change":"2023-10-06T05:24:47.343401+0000","last_active":"2023-10-06T05:24:47.348230+0000","last_peered":"2023-10-06T05:24:47.348230+0000","last_clean":"2023-10-06T05:24:47.348230+0000","last_became_active":"2023-10-06T05:24:47.342896+0000","last_became_peered":"2023-10-06T05:24:47.342896+0000","last_unstale":"2023-10-06T05:24:47.348230+0000","last_undegraded":"2023-10-06T05:24:47.348230+0000","last_fullsized":"2023-10-06T05:24:47.348230+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:19.107039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348647+0000","last_change":"2023-10-06T05:24:46.337938+0000","last_active":"2023-10-06T05:24:47.348647+0000","last_peered":"2023-10-06T05:24:47.348647+0000","last_clean":"2023-10-06T05:24:47.348647+0000","last_became_active":"2023-10-06T05:24:46.337677+0000","last_became_peered":"2023-10-06T05:24:46.337677+0000","last_unstale":"2023-10-06T05:24:47.348647+0000","last_undegraded":"2023-10-06T05:24:47.348647+0000","last_fullsized":"2023-10-06T05:24:47.348647+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:51.789343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344327+0000","last_change":"2023-10-06T05:24:47.330026+0000","last_active":"2023-10-06T05:24:47.344327+0000","last_peered":"2023-10-06T05:24:47.344327+0000","last_clean":"2023-10-06T05:24:47.344327+0000","last_became_active":"2023-10-06T05:24:47.329790+0000","last_became_peered":"2023-10-06T05:24:47.329790+0000","last_unstale":"2023-10-06T05:24:47.344327+0000","last_undegraded":"2023-10-06T05:24:47.344327+0000","last_fullsized":"2023-10-06T05:24:47.344327+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:28.298824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344379+0000","last_change":"2023-10-06T05:24:46.336585+0000","last_active":"2023-10-06T05:24:47.344379+0000","last_peered":"2023-10-06T05:24:47.344379+0000","last_clean":"2023-10-06T05:24:47.344379+0000","last_became_active":"2023-10-06T05:24:46.336301+0000","last_became_peered":"2023-10-06T05:24:46.336301+0000","last_unstale":"2023-10-06T05:24:47.344379+0000","last_undegraded":"2023-10-06T05:24:47.344379+0000","last_fullsized":"2023-10-06T05:24:47.344379+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:09.220830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344588+0000","last_change":"2023-10-06T05:24:47.339218+0000","last_active":"2023-10-06T05:24:47.344588+0000","last_peered":"2023-10-06T05:24:47.344588+0000","last_clean":"2023-10-06T05:24:47.344588+0000","last_became_active":"2023-10-06T05:24:47.338505+0000","last_became_peered":"2023-10-06T05:24:47.338505+0000","last_unstale":"2023-10-06T05:24:47.344588+0000","last_undegraded":"2023-10-06T05:24:47.344588+0000","last_fullsized":"2023-10-06T05:24:47.344588+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:44.815157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.350982+0000","last_change":"2023-10-06T05:24:46.333864+0000","last_active":"2023-10-06T05:24:47.350982+0000","last_peered":"2023-10-06T05:24:47.350982+0000","last_clean":"2023-10-06T05:24:47.350982+0000","last_became_active":"2023-10-06T05:24:46.333602+0000","last_became_peered":"2023-10-06T05:24:46.333602+0000","last_unstale":"2023-10-06T05:24:47.350982+0000","last_undegraded":"2023-10-06T05:24:47.350982+0000","last_fullsized":"2023-10-06T05:24:47.350982+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:27.684682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344531+0000","last_change":"2023-10-06T05:24:47.336264+0000","last_active":"2023-10-06T05:24:47.344531+0000","last_peered":"2023-10-06T05:24:47.344531+0000","last_clean":"2023-10-06T05:24:47.344531+0000","last_became_active":"2023-10-06T05:24:47.335853+0000","last_became_peered":"2023-10-06T05:24:47.335853+0000","last_unstale":"2023-10-06T05:24:47.344531+0000","last_undegraded":"2023-10-06T05:24:47.344531+0000","last_fullsized":"2023-10-06T05:24:47.344531+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:39.548657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.331322+0000","last_change":"2023-10-06T05:24:46.336039+0000","last_active":"2023-10-06T05:24:47.331322+0000","last_peered":"2023-10-06T05:24:47.331322+0000","last_clean":"2023-10-06T05:24:47.331322+0000","last_became_active":"2023-10-06T05:24:46.335768+0000","last_became_peered":"2023-10-06T05:24:46.335768+0000","last_unstale":"2023-10-06T05:24:47.331322+0000","last_undegraded":"2023-10-06T05:24:47.331322+0000","last_fullsized":"2023-10-06T05:24:47.331322+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:50.924611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.335284+0000","last_change":"2023-10-06T05:24:47.333204+0000","last_active":"2023-10-06T05:24:47.335284+0000","last_peered":"2023-10-06T05:24:47.335284+0000","last_clean":"2023-10-06T05:24:47.335284+0000","last_became_active":"2023-10-06T05:24:47.332808+0000","last_became_peered":"2023-10-06T05:24:47.332808+0000","last_unstale":"2023-10-06T05:24:47.335284+0000","last_undegraded":"2023-10-06T05:24:47.335284+0000","last_fullsized":"2023-10-06T05:24:47.335284+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:05.590883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.316063+0000","last_change":"2023-10-06T05:24:46.334199+0000","last_active":"2023-10-06T05:24:47.316063+0000","last_peered":"2023-10-06T05:24:47.316063+0000","last_clean":"2023-10-06T05:24:47.316063+0000","last_became_active":"2023-10-06T05:24:46.333874+0000","last_became_peered":"2023-10-06T05:24:46.333874+0000","last_unstale":"2023-10-06T05:24:47.316063+0000","last_undegraded":"2023-10-06T05:24:47.316063+0000","last_fullsized":"2023-10-06T05:24:47.316063+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:20.047556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344694+0000","last_change":"2023-10-06T05:24:47.338891+0000","last_active":"2023-10-06T05:24:47.344694+0000","last_peered":"2023-10-06T05:24:47.344694+0000","last_clean":"2023-10-06T05:24:47.344694+0000","last_became_active":"2023-10-06T05:24:47.338620+0000","last_became_peered":"2023-10-06T05:24:47.338620+0000","last_unstale":"2023-10-06T05:24:47.344694+0000","last_undegraded":"2023-10-06T05:24:47.344694+0000","last_fullsized":"2023-10-06T05:24:47.344694+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:39.611070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.350634+0000","last_change":"2023-10-06T05:24:46.335671+0000","last_active":"2023-10-06T05:24:47.350634+0000","last_peered":"2023-10-06T05:24:47.350634+0000","last_clean":"2023-10-06T05:24:47.350634+0000","last_became_active":"2023-10-06T05:24:46.335263+0000","last_became_peered":"2023-10-06T05:24:46.335263+0000","last_unstale":"2023-10-06T05:24:47.350634+0000","last_undegraded":"2023-10-06T05:24:47.350634+0000","last_fullsized":"2023-10-06T05:24:47.350634+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:25.406249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.335445+0000","last_change":"2023-10-06T05:24:47.333905+0000","last_active":"2023-10-06T05:24:47.335445+0000","last_peered":"2023-10-06T05:24:47.335445+0000","last_clean":"2023-10-06T05:24:47.335445+0000","last_became_active":"2023-10-06T05:24:47.333514+0000","last_became_peered":"2023-10-06T05:24:47.333514+0000","last_unstale":"2023-10-06T05:24:47.335445+0000","last_undegraded":"2023-10-06T05:24:47.335445+0000","last_fullsized":"2023-10-06T05:24:47.335445+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:15.122983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.346596+0000","last_change":"2023-10-06T05:24:46.324073+0000","last_active":"2023-10-06T05:24:47.346596+0000","last_peered":"2023-10-06T05:24:47.346596+0000","last_clean":"2023-10-06T05:24:47.346596+0000","last_became_active":"2023-10-06T05:24:46.323780+0000","last_became_peered":"2023-10-06T05:24:46.323780+0000","last_unstale":"2023-10-06T05:24:47.346596+0000","last_undegraded":"2023-10-06T05:24:47.346596+0000","last_fullsized":"2023-10-06T05:24:47.346596+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.164152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344458+0000","last_change":"2023-10-06T05:24:47.337385+0000","last_active":"2023-10-06T05:24:47.344458+0000","last_peered":"2023-10-06T05:24:47.344458+0000","last_clean":"2023-10-06T05:24:47.344458+0000","last_became_active":"2023-10-06T05:24:47.337083+0000","last_became_peered":"2023-10-06T05:24:47.337083+0000","last_unstale":"2023-10-06T05:24:47.344458+0000","last_undegraded":"2023-10-06T05:24:47.344458+0000","last_fullsized":"2023-10-06T05:24:47.344458+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:59.334226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349898+0000","last_change":"2023-10-06T05:24:46.339404+0000","last_active":"2023-10-06T05:24:47.349898+0000","last_peered":"2023-10-06T05:24:47.349898+0000","last_clean":"2023-10-06T05:24:47.349898+0000","last_became_active":"2023-10-06T05:24:46.339116+0000","last_became_peered":"2023-10-06T05:24:46.339116+0000","last_unstale":"2023-10-06T05:24:47.349898+0000","last_undegraded":"2023-10-06T05:24:47.349898+0000","last_fullsized":"2023-10-06T05:24:47.349898+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:24.271761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344484+0000","last_change":"2023-10-06T05:24:47.335818+0000","last_active":"2023-10-06T05:24:47.344484+0000","last_peered":"2023-10-06T05:24:47.344484+0000","last_clean":"2023-10-06T05:24:47.344484+0000","last_became_active":"2023-10-06T05:24:47.335516+0000","last_became_peered":"2023-10-06T05:24:47.335516+0000","last_unstale":"2023-10-06T05:24:47.344484+0000","last_undegraded":"2023-10-06T05:24:47.344484+0000","last_fullsized":"2023-10-06T05:24:47.344484+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:59.864576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.318095+0000","last_change":"2023-10-06T05:24:46.341377+0000","last_active":"2023-10-06T05:24:47.318095+0000","last_peered":"2023-10-06T05:24:47.318095+0000","last_clean":"2023-10-06T05:24:47.318095+0000","last_became_active":"2023-10-06T05:24:46.341139+0000","last_became_peered":"2023-10-06T05:24:46.341139+0000","last_unstale":"2023-10-06T05:24:47.318095+0000","last_undegraded":"2023-10-06T05:24:47.318095+0000","last_fullsized":"2023-10-06T05:24:47.318095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:23.398834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.346193+0000","last_change":"2023-10-06T05:24:47.335826+0000","last_active":"2023-10-06T05:24:47.346193+0000","last_peered":"2023-10-06T05:24:47.346193+0000","last_clean":"2023-10-06T05:24:47.346193+0000","last_became_active":"2023-10-06T05:24:47.335226+0000","last_became_peered":"2023-10-06T05:24:47.335226+0000","last_unstale":"2023-10-06T05:24:47.346193+0000","last_undegraded":"2023-10-06T05:24:47.346193+0000","last_fullsized":"2023-10-06T05:24:47.346193+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:27.769222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351083+0000","last_change":"2023-10-06T05:24:46.340480+0000","last_active":"2023-10-06T05:24:47.351083+0000","last_peered":"2023-10-06T05:24:47.351083+0000","last_clean":"2023-10-06T05:24:47.351083+0000","last_became_active":"2023-10-06T05:24:46.340211+0000","last_became_peered":"2023-10-06T05:24:46.340211+0000","last_unstale":"2023-10-06T05:24:47.351083+0000","last_undegraded":"2023-10-06T05:24:47.351083+0000","last_fullsized":"2023-10-06T05:24:47.351083+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:30.470575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351897+0000","last_change":"2023-10-06T05:24:47.346488+0000","last_active":"2023-10-06T05:24:47.351897+0000","last_peered":"2023-10-06T05:24:47.351897+0000","last_clean":"2023-10-06T05:24:47.351897+0000","last_became_active":"2023-10-06T05:24:47.346216+0000","last_became_peered":"2023-10-06T05:24:47.346216+0000","last_unstale":"2023-10-06T05:24:47.351897+0000","last_undegraded":"2023-10-06T05:24:47.351897+0000","last_fullsized":"2023-10-06T05:24:47.351897+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:51.823979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321921+0000","last_change":"2023-10-06T05:24:46.337982+0000","last_active":"2023-10-06T05:24:47.321921+0000","last_peered":"2023-10-06T05:24:47.321921+0000","last_clean":"2023-10-06T05:24:47.321921+0000","last_became_active":"2023-10-06T05:24:46.337576+0000","last_became_peered":"2023-10-06T05:24:46.337576+0000","last_unstale":"2023-10-06T05:24:47.321921+0000","last_undegraded":"2023-10-06T05:24:47.321921+0000","last_fullsized":"2023-10-06T05:24:47.321921+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:32.286601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351884+0000","last_change":"2023-10-06T05:24:47.347538+0000","last_active":"2023-10-06T05:24:47.351884+0000","last_peered":"2023-10-06T05:24:47.351884+0000","last_clean":"2023-10-06T05:24:47.351884+0000","last_became_active":"2023-10-06T05:24:47.347206+0000","last_became_peered":"2023-10-06T05:24:47.347206+0000","last_unstale":"2023-10-06T05:24:47.351884+0000","last_undegraded":"2023-10-06T05:24:47.351884+0000","last_fullsized":"2023-10-06T05:24:47.351884+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:33.341535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.331417+0000","last_change":"2023-10-06T05:24:46.333226+0000","last_active":"2023-10-06T05:24:47.331417+0000","last_peered":"2023-10-06T05:24:47.331417+0000","last_clean":"2023-10-06T05:24:47.331417+0000","last_became_active":"2023-10-06T05:24:46.332920+0000","last_became_peered":"2023-10-06T05:24:46.332920+0000","last_unstale":"2023-10-06T05:24:47.331417+0000","last_undegraded":"2023-10-06T05:24:47.331417+0000","last_fullsized":"2023-10-06T05:24:47.331417+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:19.745725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.345915+0000","last_change":"2023-10-06T05:24:47.342333+0000","last_active":"2023-10-06T05:24:47.345915+0000","last_peered":"2023-10-06T05:24:47.345915+0000","last_clean":"2023-10-06T05:24:47.345915+0000","last_became_active":"2023-10-06T05:24:47.342055+0000","last_became_peered":"2023-10-06T05:24:47.342055+0000","last_unstale":"2023-10-06T05:24:47.345915+0000","last_undegraded":"2023-10-06T05:24:47.345915+0000","last_fullsized":"2023-10-06T05:24:47.345915+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:16.072487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.322226+0000","last_change":"2023-10-06T05:24:46.339259+0000","last_active":"2023-10-06T05:24:47.322226+0000","last_peered":"2023-10-06T05:24:47.322226+0000","last_clean":"2023-10-06T05:24:47.322226+0000","last_became_active":"2023-10-06T05:24:46.338824+0000","last_became_peered":"2023-10-06T05:24:46.338824+0000","last_unstale":"2023-10-06T05:24:47.322226+0000","last_undegraded":"2023-10-06T05:24:47.322226+0000","last_fullsized":"2023-10-06T05:24:47.322226+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:51:56.925208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344584+0000","last_change":"2023-10-06T05:24:47.332587+0000","last_active":"2023-10-06T05:24:47.344584+0000","last_peered":"2023-10-06T05:24:47.344584+0000","last_clean":"2023-10-06T05:24:47.344584+0000","last_became_active":"2023-10-06T05:24:47.332135+0000","last_became_peered":"2023-10-06T05:24:47.332135+0000","last_unstale":"2023-10-06T05:24:47.344584+0000","last_undegraded":"2023-10-06T05:24:47.344584+0000","last_fullsized":"2023-10-06T05:24:47.344584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:34.867620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351545+0000","last_change":"2023-10-06T05:24:46.334560+0000","last_active":"2023-10-06T05:24:47.351545+0000","last_peered":"2023-10-06T05:24:47.351545+0000","last_clean":"2023-10-06T05:24:47.351545+0000","last_became_active":"2023-10-06T05:24:46.334316+0000","last_became_peered":"2023-10-06T05:24:46.334316+0000","last_unstale":"2023-10-06T05:24:47.351545+0000","last_undegraded":"2023-10-06T05:24:47.351545+0000","last_fullsized":"2023-10-06T05:24:47.351545+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:02.439238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344365+0000","last_change":"2023-10-06T05:24:47.343379+0000","last_active":"2023-10-06T05:24:47.344365+0000","last_peered":"2023-10-06T05:24:47.344365+0000","last_clean":"2023-10-06T05:24:47.344365+0000","last_became_active":"2023-10-06T05:24:47.342741+0000","last_became_peered":"2023-10-06T05:24:47.342741+0000","last_unstale":"2023-10-06T05:24:47.344365+0000","last_undegraded":"2023-10-06T05:24:47.344365+0000","last_fullsized":"2023-10-06T05:24:47.344365+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:48.856362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T05:24:48.332019+0000","last_change":"2023-10-06T05:24:46.333979+0000","last_active":"2023-10-06T05:24:48.332019+0000","last_peered":"2023-10-06T05:24:48.332019+0000","last_clean":"2023-10-06T05:24:48.332019+0000","last_became_active":"2023-10-06T05:24:46.333482+0000","last_became_peered":"2023-10-06T05:24:46.333482+0000","last_unstale":"2023-10-06T05:24:48.332019+0000","last_undegraded":"2023-10-06T05:24:48.332019+0000","last_fullsized":"2023-10-06T05:24:48.332019+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:02.180811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351901+0000","last_change":"2023-10-06T05:24:47.349434+0000","last_active":"2023-10-06T05:24:47.351901+0000","last_peered":"2023-10-06T05:24:47.351901+0000","last_clean":"2023-10-06T05:24:47.351901+0000","last_became_active":"2023-10-06T05:24:47.349152+0000","last_became_peered":"2023-10-06T05:24:47.349152+0000","last_unstale":"2023-10-06T05:24:47.351901+0000","last_undegraded":"2023-10-06T05:24:47.351901+0000","last_fullsized":"2023-10-06T05:24:47.351901+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:37.746041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321140+0000","last_change":"2023-10-06T05:24:46.339888+0000","last_active":"2023-10-06T05:24:47.321140+0000","last_peered":"2023-10-06T05:24:47.321140+0000","last_clean":"2023-10-06T05:24:47.321140+0000","last_became_active":"2023-10-06T05:24:46.339698+0000","last_became_peered":"2023-10-06T05:24:46.339698+0000","last_unstale":"2023-10-06T05:24:47.321140+0000","last_undegraded":"2023-10-06T05:24:47.321140+0000","last_fullsized":"2023-10-06T05:24:47.321140+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:55.161847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344432+0000","last_change":"2023-10-06T05:24:47.339402+0000","last_active":"2023-10-06T05:24:47.344432+0000","last_peered":"2023-10-06T05:24:47.344432+0000","last_clean":"2023-10-06T05:24:47.344432+0000","last_became_active":"2023-10-06T05:24:47.339153+0000","last_became_peered":"2023-10-06T05:24:47.339153+0000","last_unstale":"2023-10-06T05:24:47.344432+0000","last_undegraded":"2023-10-06T05:24:47.344432+0000","last_fullsized":"2023-10-06T05:24:47.344432+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:04.183491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.325457+0000","last_change":"2023-10-06T05:24:46.329841+0000","last_active":"2023-10-06T05:24:47.325457+0000","last_peered":"2023-10-06T05:24:47.325457+0000","last_clean":"2023-10-06T05:24:47.325457+0000","last_became_active":"2023-10-06T05:24:46.329603+0000","last_became_peered":"2023-10-06T05:24:46.329603+0000","last_unstale":"2023-10-06T05:24:47.325457+0000","last_undegraded":"2023-10-06T05:24:47.325457+0000","last_fullsized":"2023-10-06T05:24:47.325457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:55.150672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.336089+0000","last_change":"2023-10-06T05:24:47.334602+0000","last_active":"2023-10-06T05:24:47.336089+0000","last_peered":"2023-10-06T05:24:47.336089+0000","last_clean":"2023-10-06T05:24:47.336089+0000","last_became_active":"2023-10-06T05:24:47.334270+0000","last_became_peered":"2023-10-06T05:24:47.334270+0000","last_unstale":"2023-10-06T05:24:47.336089+0000","last_undegraded":"2023-10-06T05:24:47.336089+0000","last_fullsized":"2023-10-06T05:24:47.336089+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:59.184637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.317461+0000","last_change":"2023-10-06T05:24:46.335114+0000","last_active":"2023-10-06T05:24:47.317461+0000","last_peered":"2023-10-06T05:24:47.317461+0000","last_clean":"2023-10-06T05:24:47.317461+0000","last_became_active":"2023-10-06T05:24:46.334789+0000","last_became_peered":"2023-10-06T05:24:46.334789+0000","last_unstale":"2023-10-06T05:24:47.317461+0000","last_undegraded":"2023-10-06T05:24:47.317461+0000","last_fullsized":"2023-10-06T05:24:47.317461+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:52.164152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.348282+0000","last_change":"2023-10-06T05:24:47.343948+0000","last_active":"2023-10-06T05:24:47.348282+0000","last_peered":"2023-10-06T05:24:47.348282+0000","last_clean":"2023-10-06T05:24:47.348282+0000","last_became_active":"2023-10-06T05:24:47.343648+0000","last_became_peered":"2023-10-06T05:24:47.343648+0000","last_unstale":"2023-10-06T05:24:47.348282+0000","last_undegraded":"2023-10-06T05:24:47.348282+0000","last_fullsized":"2023-10-06T05:24:47.348282+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:15.459769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321358+0000","last_change":"2023-10-06T05:24:46.330481+0000","last_active":"2023-10-06T05:24:47.321358+0000","last_peered":"2023-10-06T05:24:47.321358+0000","last_clean":"2023-10-06T05:24:47.321358+0000","last_became_active":"2023-10-06T05:24:46.330219+0000","last_became_peered":"2023-10-06T05:24:46.330219+0000","last_unstale":"2023-10-06T05:24:47.321358+0000","last_undegraded":"2023-10-06T05:24:47.321358+0000","last_fullsized":"2023-10-06T05:24:47.321358+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:32:25.567261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.340918+0000","last_change":"2023-10-06T05:24:47.334740+0000","last_active":"2023-10-06T05:24:47.340918+0000","last_peered":"2023-10-06T05:24:47.340918+0000","last_clean":"2023-10-06T05:24:47.340918+0000","last_became_active":"2023-10-06T05:24:47.334446+0000","last_became_peered":"2023-10-06T05:24:47.334446+0000","last_unstale":"2023-10-06T05:24:47.340918+0000","last_undegraded":"2023-10-06T05:24:47.340918+0000","last_fullsized":"2023-10-06T05:24:47.340918+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:53.422415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.323711+0000","last_change":"2023-10-06T05:24:46.334796+0000","last_active":"2023-10-06T05:24:47.323711+0000","last_peered":"2023-10-06T05:24:47.323711+0000","last_clean":"2023-10-06T05:24:47.323711+0000","last_became_active":"2023-10-06T05:24:46.334304+0000","last_became_peered":"2023-10-06T05:24:46.334304+0000","last_unstale":"2023-10-06T05:24:47.323711+0000","last_undegraded":"2023-10-06T05:24:47.323711+0000","last_fullsized":"2023-10-06T05:24:47.323711+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:17.883023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349783+0000","last_change":"2023-10-06T05:24:47.342065+0000","last_active":"2023-10-06T05:24:47.349783+0000","last_peered":"2023-10-06T05:24:47.349783+0000","last_clean":"2023-10-06T05:24:47.349783+0000","last_became_active":"2023-10-06T05:24:47.341786+0000","last_became_peered":"2023-10-06T05:24:47.341786+0000","last_unstale":"2023-10-06T05:24:47.349783+0000","last_undegraded":"2023-10-06T05:24:47.349783+0000","last_fullsized":"2023-10-06T05:24:47.349783+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:34.738014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.327864+0000","last_change":"2023-10-06T05:24:46.339897+0000","last_active":"2023-10-06T05:24:47.327864+0000","last_peered":"2023-10-06T05:24:47.327864+0000","last_clean":"2023-10-06T05:24:47.327864+0000","last_became_active":"2023-10-06T05:24:46.339648+0000","last_became_peered":"2023-10-06T05:24:46.339648+0000","last_unstale":"2023-10-06T05:24:47.327864+0000","last_undegraded":"2023-10-06T05:24:47.327864+0000","last_fullsized":"2023-10-06T05:24:47.327864+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:54.398405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.345250+0000","last_change":"2023-10-06T05:24:47.333042+0000","last_active":"2023-10-06T05:24:47.345250+0000","last_peered":"2023-10-06T05:24:47.345250+0000","last_clean":"2023-10-06T05:24:47.345250+0000","last_became_active":"2023-10-06T05:24:47.332729+0000","last_became_peered":"2023-10-06T05:24:47.332729+0000","last_unstale":"2023-10-06T05:24:47.345250+0000","last_undegraded":"2023-10-06T05:24:47.345250+0000","last_fullsized":"2023-10-06T05:24:47.345250+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:39.100980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.323579+0000","last_change":"2023-10-06T05:24:46.340463+0000","last_active":"2023-10-06T05:24:47.323579+0000","last_peered":"2023-10-06T05:24:47.323579+0000","last_clean":"2023-10-06T05:24:47.323579+0000","last_became_active":"2023-10-06T05:24:46.340179+0000","last_became_peered":"2023-10-06T05:24:46.340179+0000","last_unstale":"2023-10-06T05:24:47.323579+0000","last_undegraded":"2023-10-06T05:24:47.323579+0000","last_fullsized":"2023-10-06T05:24:47.323579+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:46.588480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.325274+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:24:47.325274+0000","last_peered":"2023-10-06T05:24:47.325274+0000","last_clean":"2023-10-06T05:24:47.325274+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:24:47.325274+0000","last_undegraded":"2023-10-06T05:24:47.325274+0000","last_fullsized":"2023-10-06T05:24:47.325274+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344823+0000","last_change":"2023-10-06T05:24:47.337388+0000","last_active":"2023-10-06T05:24:47.344823+0000","last_peered":"2023-10-06T05:24:47.344823+0000","last_clean":"2023-10-06T05:24:47.344823+0000","last_became_active":"2023-10-06T05:24:47.337141+0000","last_became_peered":"2023-10-06T05:24:47.337141+0000","last_unstale":"2023-10-06T05:24:47.344823+0000","last_undegraded":"2023-10-06T05:24:47.344823+0000","last_fullsized":"2023-10-06T05:24:47.344823+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:02.111923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"138'8","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.346089+0000","last_change":"2023-10-06T05:24:46.330721+0000","last_active":"2023-10-06T05:24:47.346089+0000","last_peered":"2023-10-06T05:24:47.346089+0000","last_clean":"2023-10-06T05:24:47.346089+0000","last_became_active":"2023-10-06T05:24:46.330472+0000","last_became_peered":"2023-10-06T05:24:46.330472+0000","last_unstale":"2023-10-06T05:24:47.346089+0000","last_undegraded":"2023-10-06T05:24:47.346089+0000","last_fullsized":"2023-10-06T05:24:47.346089+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:17.883023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.326469+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:24:47.326469+0000","last_peered":"2023-10-06T05:24:47.326469+0000","last_clean":"2023-10-06T05:24:47.326469+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:24:47.326469+0000","last_undegraded":"2023-10-06T05:24:47.326469+0000","last_fullsized":"2023-10-06T05:24:47.326469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.343282+0000","last_change":"2023-10-06T05:24:47.340930+0000","last_active":"2023-10-06T05:24:47.343282+0000","last_peered":"2023-10-06T05:24:47.343282+0000","last_clean":"2023-10-06T05:24:47.343282+0000","last_became_active":"2023-10-06T05:24:47.340278+0000","last_became_peered":"2023-10-06T05:24:47.340278+0000","last_unstale":"2023-10-06T05:24:47.343282+0000","last_undegraded":"2023-10-06T05:24:47.343282+0000","last_fullsized":"2023-10-06T05:24:47.343282+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:03.189536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321400+0000","last_change":"2023-10-06T05:24:46.337393+0000","last_active":"2023-10-06T05:24:47.321400+0000","last_peered":"2023-10-06T05:24:47.321400+0000","last_clean":"2023-10-06T05:24:47.321400+0000","last_became_active":"2023-10-06T05:24:46.337135+0000","last_became_peered":"2023-10-06T05:24:46.337135+0000","last_unstale":"2023-10-06T05:24:47.321400+0000","last_undegraded":"2023-10-06T05:24:47.321400+0000","last_fullsized":"2023-10-06T05:24:47.321400+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:18.098315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.323805+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:24:47.323805+0000","last_peered":"2023-10-06T05:24:47.323805+0000","last_clean":"2023-10-06T05:24:47.323805+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:24:47.323805+0000","last_undegraded":"2023-10-06T05:24:47.323805+0000","last_fullsized":"2023-10-06T05:24:47.323805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.339130+0000","last_change":"2023-10-06T05:24:47.335477+0000","last_active":"2023-10-06T05:24:47.339130+0000","last_peered":"2023-10-06T05:24:47.339130+0000","last_clean":"2023-10-06T05:24:47.339130+0000","last_became_active":"2023-10-06T05:24:47.335176+0000","last_became_peered":"2023-10-06T05:24:47.335176+0000","last_unstale":"2023-10-06T05:24:47.339130+0000","last_undegraded":"2023-10-06T05:24:47.339130+0000","last_fullsized":"2023-10-06T05:24:47.339130+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:55.407129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321132+0000","last_change":"2023-10-06T05:24:46.333768+0000","last_active":"2023-10-06T05:24:47.321132+0000","last_peered":"2023-10-06T05:24:47.321132+0000","last_clean":"2023-10-06T05:24:47.321132+0000","last_became_active":"2023-10-06T05:24:46.333119+0000","last_became_peered":"2023-10-06T05:24:46.333119+0000","last_unstale":"2023-10-06T05:24:47.321132+0000","last_undegraded":"2023-10-06T05:24:47.321132+0000","last_fullsized":"2023-10-06T05:24:47.321132+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:14.451044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.322490+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:24:47.322490+0000","last_peered":"2023-10-06T05:24:47.322490+0000","last_clean":"2023-10-06T05:24:47.322490+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:24:47.322490+0000","last_undegraded":"2023-10-06T05:24:47.322490+0000","last_fullsized":"2023-10-06T05:24:47.322490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349868+0000","last_change":"2023-10-06T05:24:47.341919+0000","last_active":"2023-10-06T05:24:47.349868+0000","last_peered":"2023-10-06T05:24:47.349868+0000","last_clean":"2023-10-06T05:24:47.349868+0000","last_became_active":"2023-10-06T05:24:47.341704+0000","last_became_peered":"2023-10-06T05:24:47.341704+0000","last_unstale":"2023-10-06T05:24:47.349868+0000","last_undegraded":"2023-10-06T05:24:47.349868+0000","last_fullsized":"2023-10-06T05:24:47.349868+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:02.884622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"138'6","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.354645+0000","last_change":"2023-10-06T05:24:46.333318+0000","last_active":"2023-10-06T05:24:47.354645+0000","last_peered":"2023-10-06T05:24:47.354645+0000","last_clean":"2023-10-06T05:24:47.354645+0000","last_became_active":"2023-10-06T05:24:46.333050+0000","last_became_peered":"2023-10-06T05:24:46.333050+0000","last_unstale":"2023-10-06T05:24:47.354645+0000","last_undegraded":"2023-10-06T05:24:47.354645+0000","last_fullsized":"2023-10-06T05:24:47.354645+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:55.684039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":105,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.323987+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:24:47.323987+0000","last_peered":"2023-10-06T05:24:47.323987+0000","last_clean":"2023-10-06T05:24:47.323987+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:24:47.323987+0000","last_undegraded":"2023-10-06T05:24:47.323987+0000","last_fullsized":"2023-10-06T05:24:47.323987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.335184+0000","last_change":"2023-10-06T05:24:47.326406+0000","last_active":"2023-10-06T05:24:47.335184+0000","last_peered":"2023-10-06T05:24:47.335184+0000","last_clean":"2023-10-06T05:24:47.335184+0000","last_became_active":"2023-10-06T05:24:47.326126+0000","last_became_peered":"2023-10-06T05:24:47.326126+0000","last_unstale":"2023-10-06T05:24:47.335184+0000","last_undegraded":"2023-10-06T05:24:47.335184+0000","last_fullsized":"2023-10-06T05:24:47.335184+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:56.968222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"138'3","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.349161+0000","last_change":"2023-10-06T05:24:46.332461+0000","last_active":"2023-10-06T05:24:47.349161+0000","last_peered":"2023-10-06T05:24:47.349161+0000","last_clean":"2023-10-06T05:24:47.349161+0000","last_became_active":"2023-10-06T05:24:46.332111+0000","last_became_peered":"2023-10-06T05:24:46.332111+0000","last_unstale":"2023-10-06T05:24:47.349161+0000","last_undegraded":"2023-10-06T05:24:47.349161+0000","last_fullsized":"2023-10-06T05:24:47.349161+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:13.124314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":294,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.322649+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:24:47.322649+0000","last_peered":"2023-10-06T05:24:47.322649+0000","last_clean":"2023-10-06T05:24:47.322649+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:24:47.322649+0000","last_undegraded":"2023-10-06T05:24:47.322649+0000","last_fullsized":"2023-10-06T05:24:47.322649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":118,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.316676+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:24:47.316676+0000","last_peered":"2023-10-06T05:24:47.316676+0000","last_clean":"2023-10-06T05:24:47.316676+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:24:47.316676+0000","last_undegraded":"2023-10-06T05:24:47.316676+0000","last_fullsized":"2023-10-06T05:24:47.316676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344253+0000","last_change":"2023-10-06T05:24:47.335395+0000","last_active":"2023-10-06T05:24:47.344253+0000","last_peered":"2023-10-06T05:24:47.344253+0000","last_clean":"2023-10-06T05:24:47.344253+0000","last_became_active":"2023-10-06T05:24:47.335113+0000","last_became_peered":"2023-10-06T05:24:47.335113+0000","last_unstale":"2023-10-06T05:24:47.344253+0000","last_undegraded":"2023-10-06T05:24:47.344253+0000","last_fullsized":"2023-10-06T05:24:47.344253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:56.177616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321268+0000","last_change":"2023-10-06T05:24:46.333882+0000","last_active":"2023-10-06T05:24:47.321268+0000","last_peered":"2023-10-06T05:24:47.321268+0000","last_clean":"2023-10-06T05:24:47.321268+0000","last_became_active":"2023-10-06T05:24:46.333624+0000","last_became_peered":"2023-10-06T05:24:46.333624+0000","last_unstale":"2023-10-06T05:24:47.321268+0000","last_undegraded":"2023-10-06T05:24:47.321268+0000","last_fullsized":"2023-10-06T05:24:47.321268+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:49.432859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":132,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.321592+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:24:47.321592+0000","last_peered":"2023-10-06T05:24:47.321592+0000","last_clean":"2023-10-06T05:24:47.321592+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:24:47.321592+0000","last_undegraded":"2023-10-06T05:24:47.321592+0000","last_fullsized":"2023-10-06T05:24:47.321592+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.344542+0000","last_change":"2023-10-06T05:24:47.334579+0000","last_active":"2023-10-06T05:24:47.344542+0000","last_peered":"2023-10-06T05:24:47.344542+0000","last_clean":"2023-10-06T05:24:47.344542+0000","last_became_active":"2023-10-06T05:24:47.334268+0000","last_became_peered":"2023-10-06T05:24:47.334268+0000","last_unstale":"2023-10-06T05:24:47.344542+0000","last_undegraded":"2023-10-06T05:24:47.344542+0000","last_fullsized":"2023-10-06T05:24:47.344542+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:46.301705+0000","last_clean_scrub_stamp":"2023-10-06T05:24:46.301705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:53:58.540770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.351612+0000","last_change":"2023-10-06T05:24:46.333412+0000","last_active":"2023-10-06T05:24:47.351612+0000","last_peered":"2023-10-06T05:24:47.351612+0000","last_clean":"2023-10-06T05:24:47.351612+0000","last_became_active":"2023-10-06T05:24:46.333106+0000","last_became_peered":"2023-10-06T05:24:46.333106+0000","last_unstale":"2023-10-06T05:24:47.351612+0000","last_undegraded":"2023-10-06T05:24:47.351612+0000","last_fullsized":"2023-10-06T05:24:47.351612+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:24:45.292980+0000","last_clean_scrub_stamp":"2023-10-06T05:24:45.292980+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:26.988427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T05:24:47.317712+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:24:47.317712+0000","last_peered":"2023-10-06T05:24:47.317712+0000","last_clean":"2023-10-06T05:24:47.317712+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:24:47.317712+0000","last_undegraded":"2023-10-06T05:24:47.317712+0000","last_fullsized":"2023-10-06T05:24:47.317712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303672,"kb_used_data":2288,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068168,"statfs":{"total":96636764160,"available":96325804032,"internally_reserved":0,"allocated":2342912,"data_stored":1404656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.38700000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.57599999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.48299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.87,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67900000000000005,"15min":0.67800000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91300000000000003,"5min":1.018,"15min":1.018},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70399999999999996,"15min":0.74299999999999999},"min":{"1min":0.46899999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":1.056,"15min":1.056},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.55200000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.99099999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64100000000000001,"15min":0.70499999999999996},"min":{"1min":0.48799999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":1.0249999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.77000000000000002},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0629999999999999,"5min":1.27,"15min":1.27},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68999999999999995,"15min":0.73299999999999998},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":0.97899999999999998,"15min":0.999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.74199999999999999,"15min":0.81599999999999995},"min":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.98399999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78900000000000003,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.1040000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.79800000000000004,"15min":0.84199999999999997},"min":{"1min":0.503,"5min":0.46200000000000002,"15min":0.441},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.83699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574948,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303092,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068748,"statfs":{"total":96636764160,"available":96326397952,"internally_reserved":0,"allocated":1748992,"data_stored":814288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65400000000000003,"15min":0.67500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.32300000000000001},"max":{"1min":0.88200000000000001,"5min":1.006,"15min":1.8500000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68000000000000005,"15min":0.69799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90500000000000003,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.177,"5min":1.177,"15min":1.9279999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.998,"5min":1.073,"15min":1.4239999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65500000000000003,"15min":0.65000000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92700000000000005,"5min":1.093,"15min":1.369},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":0.754},"min":{"1min":0.47799999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.085,"5min":1.085,"15min":1.891},"last":0.86699999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.71699999999999997,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95999999999999996,"5min":1.018,"15min":1.347},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.71599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.97699999999999998,"5min":0.99299999999999999,"15min":1},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69899999999999995,"15min":0.76700000000000002},"min":{"1min":0.40600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.107,"5min":1.1140000000000001,"15min":1.9670000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.81399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.997,"5min":1.278,"15min":1.4510000000000001},"last":1.0209999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.415},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73799999999999999,"15min":0.81000000000000005},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":2},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.76000000000000001,"15min":0.80200000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.032,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.77700000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.46},"last":0.69399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303684,"kb_used_data":2300,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068156,"statfs":{"total":96636764160,"available":96325791744,"internally_reserved":0,"allocated":2355200,"data_stored":1406192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.438},"min":{"1min":0.32900000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57799999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.59599999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.44400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.33900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71399999999999997,"5min":0.77000000000000002,"15min":0.83799999999999997},"last":0.497},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.504,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.35399999999999998,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.38800000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.52500000000000002},"min":{"1min":0.42099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66000000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76500000000000001,"15min":0.78800000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.79900000000000004,"15min":0.79200000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.0189999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.753,"15min":0.746},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.169},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78900000000000003,"15min":0.81299999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.98699999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.749,"15min":0.747},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79700000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.1140000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.80700000000000005,"15min":0.81399999999999995},"min":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.50600000000000001},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":1.044},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.59099999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.103},"last":1.1339999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574948,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":814392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.434,"15min":0.42999999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56899999999999995,"5min":0.71499999999999997,"15min":0.84499999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.48399999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60599999999999998,"5min":0.78900000000000003,"15min":1.042},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.497,"15min":0.48899999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66200000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52400000000000002,"15min":0.51200000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65800000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52000000000000002,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.79700000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.497,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0880000000000001,"5min":1.242,"15min":1.242},"last":0.753},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.748,"15min":0.77200000000000002},"min":{"1min":0.54200000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.198,"15min":1.198},"last":0.95099999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76300000000000001,"15min":0.76400000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96499999999999997,"5min":1.26,"15min":1.26},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1299999999999999,"5min":1.163,"15min":1.163},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.999,"5min":1.323,"15min":1.323},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92300000000000004,"5min":1.1799999999999999,"15min":1.3100000000000001},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.82999999999999996},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.52400000000000002},"max":{"1min":0.999,"5min":1.2090000000000001,"15min":1.264},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78700000000000003,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":0.83399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303092,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068748,"statfs":{"total":96636764160,"available":96326397952,"internally_reserved":0,"allocated":1748992,"data_stored":814288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.42699999999999999},"min":{"1min":0.32000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51000000000000001,"5min":0.77400000000000002,"15min":0.77800000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48699999999999999,"15min":0.48399999999999999},"min":{"1min":0.32100000000000001,"5min":0.31,"15min":0.28399999999999997},"max":{"1min":0.60599999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39300000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45500000000000002,"15min":0.46200000000000002},"min":{"1min":0.29899999999999999,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.53700000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.498,"15min":0.5},"min":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.31900000000000001},"max":{"1min":0.61799999999999999,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.55300000000000005,"5min":0.82799999999999996,"15min":0.93600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.372,"15min":0.36199999999999999},"max":{"1min":0.626,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.73199999999999998,"15min":0.753},"min":{"1min":0.58899999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88800000000000001,"5min":1.022,"15min":1.1299999999999999},"last":0.997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73399999999999999,"15min":0.78600000000000003},"min":{"1min":0.52800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92200000000000004,"5min":1.085,"15min":1.1699999999999999},"last":1.018}]},{"osd":5,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73499999999999999,"15min":0.77600000000000002},"min":{"1min":0.496,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93700000000000006,"5min":1.042,"15min":1.099},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.747,"15min":0.79500000000000004},"min":{"1min":0.56499999999999995,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97899999999999998,"5min":1.0649999999999999,"15min":1.163},"last":1.052}]},{"osd":6,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76800000000000002,"15min":0.82299999999999995},"min":{"1min":0.60599999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96299999999999997,"5min":1.0960000000000001,"15min":1.21},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.76600000000000001,"15min":0.80500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.876,"5min":1.022,"15min":1.042},"last":1.0860000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77900000000000003,"15min":0.80200000000000005},"min":{"1min":0.64400000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.997,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79000000000000004,"15min":0.84799999999999998},"min":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.93899999999999995,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":0.97099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1769472,"data_stored":816400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.58899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47099999999999997,"15min":0.46800000000000003},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60399999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.501}]},{"osd":2,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70299999999999996,"5min":0.999,"15min":0.999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.63200000000000001,"5min":0.98999999999999999,"15min":1.014},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0349999999999999,"5min":1.161,"15min":1.2669999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.60999999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.004,"5min":1.1830000000000001,"15min":1.3540000000000001},"last":0.83599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.754,"15min":0.76400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.093,"5min":1.147,"15min":1.204},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.208,"5min":1.284,"15min":1.329},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77900000000000003,"15min":0.80800000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.92700000000000005,"5min":1.0880000000000001,"15min":1.484},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0169999999999999,"5min":1.2809999999999999,"15min":1.629},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81000000000000005,"15min":0.84499999999999997},"min":{"1min":0.58299999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1379999999999999,"5min":1.236,"15min":1.5620000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.624,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":0.96399999999999997,"5min":1.194,"15min":1.325},"last":0.86299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303108,"kb_used_data":1724,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068732,"statfs":{"total":96636764160,"available":96326381568,"internally_reserved":0,"allocated":1765376,"data_stored":815024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.51700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89400000000000002,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.76800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.02,"5min":1.02,"15min":2.77},"last":0.95899999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69899999999999995,"15min":0.73299999999999998},"min":{"1min":0.48799999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.6419999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68899999999999995,"15min":0.71499999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.85799999999999998,"5min":1.1100000000000001,"15min":1.1899999999999999},"last":0.99199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.497,"5min":0.47599999999999998,"15min":0.39800000000000002},"max":{"1min":0.79400000000000004,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72499999999999998,"15min":0.80300000000000005},"min":{"1min":0.57399999999999995,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97599999999999998,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71399999999999997,"15min":0.71699999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88800000000000001,"5min":0.95899999999999996,"15min":1.0629999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.996},"min":{"1min":0.56399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":7.742},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.73199999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93700000000000006,"5min":0.96499999999999997,"15min":1.079},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.747},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.03,"5min":1.04,"15min":1.181},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.73099999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98299999999999998,"5min":1.0029999999999999,"15min":1.2669999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76100000000000001,"15min":0.81200000000000006},"min":{"1min":0.56399999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.093,"5min":1.093,"15min":1.171},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.82199999999999995},"min":{"1min":0.504,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.016,"5min":1.1899999999999999,"15min":1.228},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77300000000000002,"15min":0.84599999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.571},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574948,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":814412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65400000000000003,"15min":0.91000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.98199999999999998,"5min":1.109,"15min":9.2780000000000005},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.91300000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.93500000000000005},"min":{"1min":0.52500000000000002,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":1.0329999999999999,"5min":1.079,"15min":9.25},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.98899999999999999},"min":{"1min":0.46500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93200000000000005,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66400000000000003,"15min":0.95999999999999996},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98499999999999999,"5min":1.052,"15min":10.384},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.98699999999999999},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.097,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72699999999999998,"15min":1.0249999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.495},"max":{"1min":0.88200000000000001,"5min":1.76,"15min":10.205},"last":0.753},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70999999999999996,"15min":0.72399999999999998},"min":{"1min":0.55300000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.097,"15min":1.292},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68300000000000005,"15min":0.70099999999999996},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70699999999999996,"15min":0.76500000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.008,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.876}]},{"osd":6,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70299999999999996,"15min":0.73499999999999999},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90900000000000003,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.80500000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95899999999999996,"5min":1.008,"15min":1.298},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.81799999999999995},"min":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0529999999999999,"5min":1.556,"15min":1.556},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.80700000000000005},"min":{"1min":0.498,"5min":0.432,"15min":0.432},"max":{"1min":1.04,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.79700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:24:52.211 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:24:52.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:24:52.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.445+0000 7f6af99ff700 1 Processor -- start 2023-10-06T05:24:52.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.445+0000 7f6af99ff700 1 -- start start 2023-10-06T05:24:52.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.446+0000 7f6af99ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6af4087a00 0x7f6af40af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.446+0000 7f6af99ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40cc740 0x7f6af40ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.446+0000 7f6af99ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af40c9bb0 con 0x7f6af40c8aa0 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.446+0000 7f6af99ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af4087dd0 con 0x7f6af4087a00 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.446+0000 7f6af99ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4166650 con 0x7f6af40cc740 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.447+0000 7f6af2ffd700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6af4087a00 0x7f6af40af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.447+0000 7f6af27fc700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6af40c8aa0 0x7f6af40cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:46596/0 (socket says 172.21.15.31:46596) 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.447+0000 7f6af27fc700 1 -- 172.21.15.31:0/2764949107 learned_addr learned my addr 172.21.15.31:0/2764949107 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.447+0000 7f6af37fe700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40cc740 0x7f6af40ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.448+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682854184 0 0) 0x7f6af40c9bb0 con 0x7f6af40c8aa0 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6adc003550 con 0x7f6af40c8aa0 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 <== mon.1 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4183126057 0 0) 0x7f6adc003550 con 0x7f6af40c8aa0 2023-10-06T05:24:52.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40cc740 msgr2=0x7f6af40ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40cc740 0x7f6af40ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 >> v2:172.21.15.62:3300/0 conn(0x7f6af4087a00 msgr2=0x7f6af40af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 --2- 172.21.15.31:0/2764949107 >> v2:172.21.15.62:3300/0 conn(0x7f6af4087a00 0x7f6af40af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af40c8fa0 con 0x7f6af40c8aa0 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 <== mon.1 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f6ae8001470 con 0x7f6af40c8aa0 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 >> v1:172.21.15.62:6789/0 conn(0x7f6af40c8aa0 legacy=0x7f6af40cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6adc0081b0 0x7f6adc00a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6adc00ab90 0x7f6adc00d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc00d680 0x7f6adc00fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4087dd0 con 0x7f6adc00d680 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4166650 con 0x7f6adc00ab90 2023-10-06T05:24:52.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc010460 con 0x7f6adc0081b0 2023-10-06T05:24:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/2764949107 >> v2:172.21.15.62:3300/0 conn(0x7f6af4087a00 0x7f6af40af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.454+0000 7f6af37fe700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6af40cc740 0x7f6af40ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.455+0000 7f6af27fc700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6adc00ab90 0x7f6adc00d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.455+0000 7f6af37fe700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc00d680 0x7f6adc00fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.455+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6adc0081b0 0x7f6adc00a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.455+0000 7f6af27fc700 1 -- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6adc0081b0 msgr2=0x7f6adc00a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.455+0000 7f6af27fc700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6adc0081b0 0x7f6adc00a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.455+0000 7f6af27fc700 1 -- 172.21.15.31:0/2764949107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc00d680 msgr2=0x7f6adc00fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.455+0000 7f6af27fc700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc00d680 0x7f6adc00fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.455+0000 7f6af27fc700 1 -- 172.21.15.31:0/2764949107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc003550 con 0x7f6adc00ab90 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.456+0000 7f6af27fc700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6adc00ab90 0x7f6adc00d010 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800c2a0 tx=0x7f6ae8001dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed5471cb50a7cce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.456+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800cb70 con 0x7f6adc00ab90 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.456+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae800ccd0 con 0x7f6adc00ab90 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.456+0000 7f6af1ffb700 1 -- 172.21.15.31:0/2764949107 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8005b70 con 0x7f6adc00ab90 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.456+0000 7f6af99ff700 1 -- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6adc00ab90 msgr2=0x7f6adc00d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.456+0000 7f6af99ff700 1 --2- 172.21.15.31:0/2764949107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6adc00ab90 0x7f6adc00d010 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800c2a0 tx=0x7f6ae8001dc0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.456+0000 7f6af37fe700 1 -- 172.21.15.31:0/2764949107 reap_dead start 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.457+0000 7f6af99ff700 1 -- 172.21.15.31:0/2764949107 shutdown_connections 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.457+0000 7f6af99ff700 1 -- 172.21.15.31:0/2764949107 >> 172.21.15.31:0/2764949107 conn(0x7f6af4068030 msgr2=0x7f6af406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.457+0000 7f6af99ff700 1 -- 172.21.15.31:0/2764949107 shutdown_connections 2023-10-06T05:24:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.457+0000 7f6af99ff700 1 -- 172.21.15.31:0/2764949107 wait complete. 2023-10-06T05:24:52.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.458+0000 7f6af99ff700 1 Processor -- start 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.458+0000 7f6af99ff700 1 -- start start 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.459+0000 7f6af99ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6adc00ab90 0x7f6af40c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.459+0000 7f6af99ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6af40c6520 0x7f6af40c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.459+0000 7f6af99ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af40c4900 0x7f6af40c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.459+0000 7f6af99ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40c52b0 con 0x7f6af40c4900 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.459+0000 7f6af99ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4163640 con 0x7f6af40c6520 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.459+0000 7f6af99ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40c6a00 con 0x7f6adc00ab90 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af40c4900 0x7f6af40c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af40c4900 0x7f6af40c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55630/0 (socket says 172.21.15.31:55630) 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6af40c6520 0x7f6af40c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af37fe700 1 -- 172.21.15.31:0/2415985361 learned_addr learned my addr 172.21.15.31:0/2415985361 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6adc00ab90 0x7f6af40c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af37fe700 1 -- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6adc00ab90 msgr2=0x7f6af40c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af37fe700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6adc00ab90 0x7f6af40c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af37fe700 1 -- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6af40c6520 msgr2=0x7f6af40c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af37fe700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6af40c6520 0x7f6af40c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af37fe700 1 -- 172.21.15.31:0/2415985361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af40c6ce0 con 0x7f6af40c4900 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6af37fe700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af40c4900 0x7f6af40c4d70 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6af40ccb10 tx=0x7f6aec00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6adaffd700 1 -- 172.21.15.31:0/2415985361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec010040 con 0x7f6af40c4900 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6adaffd700 1 -- 172.21.15.31:0/2415985361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec00b2e0 con 0x7f6af40c4900 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.460+0000 7f6adaffd700 1 -- 172.21.15.31:0/2415985361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec007cc0 con 0x7f6af40c4900 2023-10-06T05:24:52.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.461+0000 7f6af99ff700 1 -- 172.21.15.31:0/2415985361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af40c6fc0 con 0x7f6af40c4900 2023-10-06T05:24:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.461+0000 7f6af99ff700 1 -- 172.21.15.31:0/2415985361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af40c2ae0 con 0x7f6af40c4900 2023-10-06T05:24:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.462+0000 7f6af99ff700 1 -- 172.21.15.31:0/2415985361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af40c74e0 con 0x7f6af40c4900 2023-10-06T05:24:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.463+0000 7f6adaffd700 1 -- 172.21.15.31:0/2415985361 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f6aec009760 con 0x7f6af40c4900 2023-10-06T05:24:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.464+0000 7f6adaffd700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6ae00674d0 0x7f6ae0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.464+0000 7f6adaffd700 1 -- 172.21.15.31:0/2415985361 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6aec085800 con 0x7f6af40c4900 2023-10-06T05:24:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.464+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6ae00674d0 0x7f6ae0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.465+0000 7f6af2ffd700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6ae00674d0 0x7f6ae0069980 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4006f00 tx=0x7f6ae40080e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.468+0000 7f6adaffd700 1 -- 172.21.15.31:0/2415985361 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6aec051f70 con 0x7f6af40c4900 2023-10-06T05:24:52.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.748+0000 7f6af99ff700 1 -- 172.21.15.31:0/2415985361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6af41637b0 con 0x7f6af40c4900 2023-10-06T05:24:52.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.754+0000 7f6adaffd700 1 -- 172.21.15.31:0/2415985361 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6aec055590 con 0x7f6af40c4900 2023-10-06T05:24:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 -- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6ae00674d0 msgr2=0x7f6ae0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6ae00674d0 0x7f6ae0069980 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4006f00 tx=0x7f6ae40080e0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 -- 172.21.15.31:0/2415985361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af40c4900 msgr2=0x7f6af40c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af40c4900 0x7f6af40c4d70 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6af40ccb10 tx=0x7f6aec00d040 comp rx=0 tx=0).stop 2023-10-06T05:24:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 -- 172.21.15.31:0/2415985361 shutdown_connections 2023-10-06T05:24:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6ae00674d0 0x7f6ae0069980 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6af40c4900 0x7f6af40c4d70 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6adc00ab90 0x7f6af40c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 --2- 172.21.15.31:0/2415985361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6af40c6520 0x7f6af40c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 -- 172.21.15.31:0/2415985361 >> 172.21.15.31:0/2415985361 conn(0x7f6af4068030 msgr2=0x7f6af41657f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.756+0000 7f6af99ff700 1 -- 172.21.15.31:0/2415985361 shutdown_connections 2023-10-06T05:24:52.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.757+0000 7f6af99ff700 1 -- 172.21.15.31:0/2415985361 wait complete. 2023-10-06T05:24:52.760 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:24:52.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:24:52.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.969+0000 7f85c63ff700 1 Processor -- start 2023-10-06T05:24:52.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.969+0000 7f85c63ff700 1 -- start start 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.970+0000 7f85c63ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f85c00c1df0 0x7f85c0159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.970+0000 7f85c63ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c0067f70 con 0x7f85c01596d0 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.970+0000 7f85c63ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c006e040 con 0x7f85c00c6f80 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.970+0000 7f85c63ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c006b040 con 0x7f85c00c1df0 2023-10-06T05:24:52.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.970+0000 7f85bffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f85c01596d0 0x7f85c006a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42348/0 (socket says 172.21.15.31:42348) 2023-10-06T05:24:52.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85bffff700 1 -- 172.21.15.31:0/332502051 learned_addr learned my addr 172.21.15.31:0/332502051 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:52.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85c499c700 1 --2- 172.21.15.31:0/332502051 >> v2:172.21.15.62:3300/0 conn(0x7f85c00c1df0 0x7f85c0159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219701863 0 0) 0x7f85c0067f70 con 0x7f85c01596d0 2023-10-06T05:24:52.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a8003570 con 0x7f85c01596d0 2023-10-06T05:24:52.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1460260145 0 0) 0x7f85a8003570 con 0x7f85c01596d0 2023-10-06T05:24:52.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 >> v2:172.21.15.62:3300/0 conn(0x7f85c00c1df0 msgr2=0x7f85c0159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 --2- 172.21.15.31:0/332502051 >> v2:172.21.15.62:3300/0 conn(0x7f85c00c1df0 0x7f85c0159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 >> v1:172.21.15.62:6789/0 conn(0x7f85c00c6f80 legacy=0x7f85c00c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c00681c0 con 0x7f85c01596d0 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f85b0001470 con 0x7f85c01596d0 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 >> v1:172.21.15.31:6789/0 conn(0x7f85c01596d0 legacy=0x7f85c006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a8003a60 0x7f85a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85a8004810 0x7f85a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a80051c0 0x7f85a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c006b040 con 0x7f85a80051c0 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.971+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a800c050 con 0x7f85a8003a60 2023-10-06T05:24:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a800c210 con 0x7f85a8004810 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85bffff700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a8003a60 0x7f85a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85c499c700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a80051c0 0x7f85a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85bf7fe700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85a8004810 0x7f85a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85c499c700 1 -- 172.21.15.31:0/332502051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85a8004810 msgr2=0x7f85a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85c499c700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85a8004810 0x7f85a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85c499c700 1 -- 172.21.15.31:0/332502051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a8003a60 msgr2=0x7f85a8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85c499c700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a8003a60 0x7f85a8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85c499c700 1 -- 172.21.15.31:0/332502051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8003570 con 0x7f85a80051c0 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85c499c700 1 -- 172.21.15.31:0/332502051 reap_dead start 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.972+0000 7f85c499c700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a80051c0 0x7f85a800ba60 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f85c00b2940 tx=0x7f85b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d09affcd9ddc2c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.973+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800ea60 con 0x7f85a80051c0 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.973+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b800f040 con 0x7f85a80051c0 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.973+0000 7f85beffd700 1 -- 172.21.15.31:0/332502051 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800ee10 con 0x7f85a80051c0 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.973+0000 7f85c63ff700 1 -- 172.21.15.31:0/332502051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a80051c0 msgr2=0x7f85a800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.973+0000 7f85c63ff700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a80051c0 0x7f85a800ba60 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f85c00b2940 tx=0x7f85b800b040 comp rx=0 tx=0).stop 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.973+0000 7f85c63ff700 1 -- 172.21.15.31:0/332502051 shutdown_connections 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.973+0000 7f85c63ff700 1 --2- 172.21.15.31:0/332502051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a80051c0 0x7f85a800ba60 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.973+0000 7f85c63ff700 1 -- 172.21.15.31:0/332502051 >> 172.21.15.31:0/332502051 conn(0x7f85c00c9e80 msgr2=0x7f85c006a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.973+0000 7f85c63ff700 1 -- 172.21.15.31:0/332502051 shutdown_connections 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.973+0000 7f85c63ff700 1 -- 172.21.15.31:0/332502051 wait complete. 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.975+0000 7f85c63ff700 1 Processor -- start 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.975+0000 7f85c63ff700 1 -- start start 2023-10-06T05:24:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85c63ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a80051c0 0x7f85c0156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85c63ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85c00b0ff0 0x7f85c00b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85c63ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85c01500b0 0x7f85c0150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85c63ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00b0630 con 0x7f85c00b0ff0 2023-10-06T05:24:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85c63ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0150a60 con 0x7f85a80051c0 2023-10-06T05:24:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85c63ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00af740 con 0x7f85c01500b0 2023-10-06T05:24:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85c00b0ff0 0x7f85c00b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85c00b0ff0 0x7f85c00b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55638/0 (socket says 172.21.15.31:55638) 2023-10-06T05:24:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85bf7fe700 1 -- 172.21.15.31:0/1033613982 learned_addr learned my addr 172.21.15.31:0/1033613982 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85bffff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a80051c0 0x7f85c0156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.976+0000 7f85c499c700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85c01500b0 0x7f85c0150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85bf7fe700 1 -- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85c01500b0 msgr2=0x7f85c0150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85bf7fe700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85c01500b0 0x7f85c0150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85bf7fe700 1 -- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a80051c0 msgr2=0x7f85c0156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85bf7fe700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a80051c0 0x7f85c0156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85bf7fe700 1 -- 172.21.15.31:0/1033613982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c00afa20 con 0x7f85c00b0ff0 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85bffff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a80051c0 0x7f85c0156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85bf7fe700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85c00b0ff0 0x7f85c00b1460 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f85b400f8d0 tx=0x7f85b4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85a7fff700 1 -- 172.21.15.31:0/1033613982 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400d6d0 con 0x7f85c00b0ff0 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85a7fff700 1 -- 172.21.15.31:0/1033613982 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b400d850 con 0x7f85c00b0ff0 2023-10-06T05:24:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85a7fff700 1 -- 172.21.15.31:0/1033613982 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400dc10 con 0x7f85c00b0ff0 2023-10-06T05:24:52.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85c63ff700 1 -- 172.21.15.31:0/1033613982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c00afd00 con 0x7f85c00b0ff0 2023-10-06T05:24:52.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.977+0000 7f85c63ff700 1 -- 172.21.15.31:0/1033613982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c00b02c0 con 0x7f85c00b0ff0 2023-10-06T05:24:52.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.978+0000 7f85c63ff700 1 -- 172.21.15.31:0/1033613982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c004c380 con 0x7f85c00b0ff0 2023-10-06T05:24:52.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.979+0000 7f85a7fff700 1 -- 172.21.15.31:0/1033613982 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f85b401c400 con 0x7f85c00b0ff0 2023-10-06T05:24:52.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.980+0000 7f85a7fff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85ac0674d0 0x7f85ac069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:52.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.980+0000 7f85a7fff700 1 -- 172.21.15.31:0/1033613982 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f85b40854b0 con 0x7f85c00b0ff0 2023-10-06T05:24:52.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.981+0000 7f85bffff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85ac0674d0 0x7f85ac069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:52.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.981+0000 7f85bffff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85ac0674d0 0x7f85ac069980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f85b00015a0 tx=0x7f85b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:52.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:52.984+0000 7f85a7fff700 1 -- 172.21.15.31:0/1033613982 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85b4051da0 con 0x7f85c00b0ff0 2023-10-06T05:24:53.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.232+0000 7f85c63ff700 1 -- 172.21.15.31:0/1033613982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85c0063760 con 0x7f85c00b0ff0 2023-10-06T05:24:53.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.233+0000 7f85a7fff700 1 -- 172.21.15.31:0/1033613982 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f85b40553c0 con 0x7f85c00b0ff0 2023-10-06T05:24:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:24:53.235 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1352198494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1352198494},{"type":"v1","addr":"172.21.15.62:6835","nonce":1352198494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8194,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/243154980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":243154980},{"type":"v1","addr":"172.21.15.62:6834","nonce":243154980}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/240917476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":240917476},{"type":"v1","addr":"172.21.15.31:6834","nonce":240917476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:46.824751+0000","modified":"2023-10-06T05:24:50.335585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3829644591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3829644591},{"type":"v1","addr":"172.21.15.31:6835","nonce":3829644591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:24:53.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.235+0000 7f85c63ff700 1 -- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85ac0674d0 msgr2=0x7f85ac069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.235+0000 7f85c63ff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85ac0674d0 0x7f85ac069980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f85b00015a0 tx=0x7f85b0009040 comp rx=0 tx=0).stop 2023-10-06T05:24:53.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.235+0000 7f85c63ff700 1 -- 172.21.15.31:0/1033613982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85c00b0ff0 msgr2=0x7f85c00b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.235+0000 7f85c63ff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85c00b0ff0 0x7f85c00b1460 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f85b400f8d0 tx=0x7f85b4007d30 comp rx=0 tx=0).stop 2023-10-06T05:24:53.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.236+0000 7f85c63ff700 1 -- 172.21.15.31:0/1033613982 shutdown_connections 2023-10-06T05:24:53.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.236+0000 7f85c63ff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85ac0674d0 0x7f85ac069980 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.236+0000 7f85c63ff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85c01500b0 0x7f85c0150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.236+0000 7f85c63ff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a80051c0 0x7f85c0156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.236+0000 7f85c63ff700 1 --2- 172.21.15.31:0/1033613982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85c00b0ff0 0x7f85c00b1460 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.236+0000 7f85c63ff700 1 -- 172.21.15.31:0/1033613982 >> 172.21.15.31:0/1033613982 conn(0x7f85c00c9e80 msgr2=0x7f85c006b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.236+0000 7f85c63ff700 1 -- 172.21.15.31:0/1033613982 shutdown_connections 2023-10-06T05:24:53.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.236+0000 7f85c63ff700 1 -- 172.21.15.31:0/1033613982 wait complete. 2023-10-06T05:24:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 93 2023-10-06T05:24:53.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:24:46.824751+0000', 'modified': '2023-10-06T05:24:50.335585+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8478': {'gid': 8478, 'name': 'c', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/3829644591', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6833', 'nonce': 3829644591}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 3829644591}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:24:53.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:24:53.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:24:53.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.855+0000 7f57d0fff700 1 Processor -- start 2023-10-06T05:24:53.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.855+0000 7f57d0fff700 1 -- start start 2023-10-06T05:24:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.857+0000 7f57d0fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc159500 0x7f57cc0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.857+0000 7f57d0fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f57cc0cd510 0x7f57cc0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.857+0000 7f57d0fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57cc0c3f40 con 0x7f57cc15d2d0 2023-10-06T05:24:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.857+0000 7f57d0fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57cc0b2860 con 0x7f57cc0cd510 2023-10-06T05:24:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.857+0000 7f57d0fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc15d7d0 con 0x7f57cc159500 2023-10-06T05:24:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.857+0000 7f57ca59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc159500 0x7f57cc0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.857+0000 7f57cad9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f57cc0cd510 0x7f57cc0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.857+0000 7f57ca59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc159500 0x7f57cc0b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:40924/0 (socket says 172.21.15.31:40924) 2023-10-06T05:24:53.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.857+0000 7f57cad9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f57cc0cd510 0x7f57cc0aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:55540/0 (socket says 172.21.15.31:55540) 2023-10-06T05:24:53.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.857+0000 7f57ca59c700 1 -- 172.21.15.31:0/4253891537 learned_addr learned my addr 172.21.15.31:0/4253891537 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.858+0000 7f57ca59c700 1 -- 172.21.15.31:0/4253891537 >> v2:172.21.15.62:3300/0 conn(0x7f57cc0cd510 msgr2=0x7f57cc0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.858+0000 7f57ca59c700 1 --2- 172.21.15.31:0/4253891537 >> v2:172.21.15.62:3300/0 conn(0x7f57cc0cd510 0x7f57cc0aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.858+0000 7f57ca59c700 1 -- 172.21.15.31:0/4253891537 >> v1:172.21.15.62:6789/0 conn(0x7f57cc15d2d0 legacy=0x7f57cc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.858+0000 7f57ca59c700 1 -- 172.21.15.31:0/4253891537 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc0cb000 con 0x7f57cc159500 2023-10-06T05:24:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.858+0000 7f57cad9d700 1 --2- 172.21.15.31:0/4253891537 >> v2:172.21.15.62:3300/0 conn(0x7f57cc0cd510 0x7f57cc0aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.858+0000 7f57ca59c700 1 --2- 172.21.15.31:0/4253891537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc159500 0x7f57cc0b0ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0099b0 tx=0x7f57bc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acfaad6747349365 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.858+0000 7f57c959a700 1 -- 172.21.15.31:0/4253891537 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc009ea0 con 0x7f57cc159500 2023-10-06T05:24:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.858+0000 7f57c959a700 1 -- 172.21.15.31:0/4253891537 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00b860 con 0x7f57cc159500 2023-10-06T05:24:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.858+0000 7f57c959a700 1 -- 172.21.15.31:0/4253891537 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00bc00 con 0x7f57cc159500 2023-10-06T05:24:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.859+0000 7f57d0fff700 1 -- 172.21.15.31:0/4253891537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc159500 msgr2=0x7f57cc0b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.859+0000 7f57d0fff700 1 --2- 172.21.15.31:0/4253891537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc159500 0x7f57cc0b0ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0099b0 tx=0x7f57bc009250 comp rx=0 tx=0).stop 2023-10-06T05:24:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.859+0000 7f57d0fff700 1 -- 172.21.15.31:0/4253891537 shutdown_connections 2023-10-06T05:24:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.859+0000 7f57d0fff700 1 --2- 172.21.15.31:0/4253891537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc159500 0x7f57cc0b0ab0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.859+0000 7f57d0fff700 1 --2- 172.21.15.31:0/4253891537 >> v2:172.21.15.62:3300/0 conn(0x7f57cc0cd510 0x7f57cc0aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.859+0000 7f57d0fff700 1 -- 172.21.15.31:0/4253891537 >> 172.21.15.31:0/4253891537 conn(0x7f57cc06ec90 msgr2=0x7f57cc06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.859+0000 7f57d0fff700 1 -- 172.21.15.31:0/4253891537 shutdown_connections 2023-10-06T05:24:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.859+0000 7f57d0fff700 1 -- 172.21.15.31:0/4253891537 wait complete. 2023-10-06T05:24:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.860+0000 7f57d0fff700 1 Processor -- start 2023-10-06T05:24:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.860+0000 7f57d0fff700 1 -- start start 2023-10-06T05:24:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.861+0000 7f57d0fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc0cd510 0x7f57cc0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.861+0000 7f57d0fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57cc159500 0x7f57cc157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.861+0000 7f57d0fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57cc15d2d0 0x7f57cc157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.861+0000 7f57d0fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0b6030 con 0x7f57cc15d2d0 2023-10-06T05:24:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.861+0000 7f57d0fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc157fd0 con 0x7f57cc159500 2023-10-06T05:24:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.861+0000 7f57d0fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0b5a70 con 0x7f57cc0cd510 2023-10-06T05:24:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57cad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57cc15d2d0 0x7f57cc157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57cad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57cc15d2d0 0x7f57cc157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55652/0 (socket says 172.21.15.31:55652) 2023-10-06T05:24:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57cad9d700 1 -- 172.21.15.31:0/2483261041 learned_addr learned my addr 172.21.15.31:0/2483261041 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57ca59c700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc0cd510 0x7f57cc0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57c9d9b700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57cc159500 0x7f57cc157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57cad9d700 1 -- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc0cd510 msgr2=0x7f57cc0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57cad9d700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc0cd510 0x7f57cc0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57cad9d700 1 -- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57cc159500 msgr2=0x7f57cc157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57cad9d700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57cc159500 0x7f57cc157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57cad9d700 1 -- 172.21.15.31:0/2483261041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4009a20 con 0x7f57cc15d2d0 2023-10-06T05:24:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.862+0000 7f57cad9d700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57cc15d2d0 0x7f57cc157a90 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009f60 tx=0x7f57c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.863+0000 7f57ba7fc700 1 -- 172.21.15.31:0/2483261041 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c400ccd0 con 0x7f57cc15d2d0 2023-10-06T05:24:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.863+0000 7f57ba7fc700 1 -- 172.21.15.31:0/2483261041 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c400f040 con 0x7f57cc15d2d0 2023-10-06T05:24:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.863+0000 7f57d0fff700 1 -- 172.21.15.31:0/2483261041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57bc0094f0 con 0x7f57cc15d2d0 2023-10-06T05:24:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.863+0000 7f57ba7fc700 1 -- 172.21.15.31:0/2483261041 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40146d0 con 0x7f57cc15d2d0 2023-10-06T05:24:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.863+0000 7f57d0fff700 1 -- 172.21.15.31:0/2483261041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc0c88b0 con 0x7f57cc15d2d0 2023-10-06T05:24:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.865+0000 7f57ba7fc700 1 -- 172.21.15.31:0/2483261041 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f57c4003020 con 0x7f57cc15d2d0 2023-10-06T05:24:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.866+0000 7f57ba7fc700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ac0674d0 0x7f57ac069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.866+0000 7f57ba7fc700 1 -- 172.21.15.31:0/2483261041 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f57c4085c30 con 0x7f57cc15d2d0 2023-10-06T05:24:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.866+0000 7f57ca59c700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ac0674d0 0x7f57ac069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.867+0000 7f57ca59c700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ac0674d0 0x7f57ac069980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0099b0 tx=0x7f57bc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.868+0000 7f57df646b80 1 Processor -- start 2023-10-06T05:24:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.868+0000 7f57df646b80 1 -- start start 2023-10-06T05:24:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57df646b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bee450 0x55ac17bee820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57df646b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 0x55ac17bf6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57df646b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bf74d0 0x55ac17bf7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57df646b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac17ac1310 con 0x55ac17bee450 2023-10-06T05:24:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57df646b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac17afe130 con 0x55ac17bd3f40 2023-10-06T05:24:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57df646b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac17afbaf0 con 0x55ac17bf74d0 2023-10-06T05:24:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57ca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bee450 0x55ac17bee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57ca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bee450 0x55ac17bee820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55658/0 (socket says 172.21.15.31:55658) 2023-10-06T05:24:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57ca59c700 1 -- 172.21.15.31:0/4277706506 learned_addr learned my addr 172.21.15.31:0/4277706506 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57c9d9b700 1 --2- 172.21.15.31:0/4277706506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 0x55ac17bf6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57cad9d700 1 --2- 172.21.15.31:0/4277706506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bf74d0 0x55ac17bf7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57ca59c700 1 -- 172.21.15.31:0/4277706506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bf74d0 msgr2=0x55ac17bf7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57ca59c700 1 --2- 172.21.15.31:0/4277706506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bf74d0 0x55ac17bf7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57ca59c700 1 -- 172.21.15.31:0/4277706506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 msgr2=0x55ac17bf6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57ca59c700 1 --2- 172.21.15.31:0/4277706506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 0x55ac17bf6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57ca59c700 1 -- 172.21.15.31:0/4277706506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac178fd790 con 0x55ac17bee450 2023-10-06T05:24:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.869+0000 7f57c9d9b700 1 --2- 172.21.15.31:0/4277706506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 0x55ac17bf6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.870+0000 7f57ca59c700 1 --2- 172.21.15.31:0/4277706506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bee450 0x55ac17bee820 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00b3c0 tx=0x7f57bc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6b2f0c24112b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.870+0000 7f57ab7fe700 1 -- 172.21.15.31:0/4277706506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc020070 con 0x55ac17bee450 2023-10-06T05:24:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.870+0000 7f57ab7fe700 1 -- 172.21.15.31:0/4277706506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc029990 con 0x55ac17bee450 2023-10-06T05:24:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.870+0000 7f57ab7fe700 1 -- 172.21.15.31:0/4277706506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc029b00 con 0x55ac17bee450 2023-10-06T05:24:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.870+0000 7f57df646b80 1 -- 172.21.15.31:0/4277706506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bee450 msgr2=0x55ac17bee820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.870+0000 7f57df646b80 1 --2- 172.21.15.31:0/4277706506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bee450 0x55ac17bee820 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00b3c0 tx=0x7f57bc020000 comp rx=0 tx=0).stop 2023-10-06T05:24:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.871+0000 7f57df646b80 1 -- 172.21.15.31:0/4277706506 shutdown_connections 2023-10-06T05:24:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.871+0000 7f57df646b80 1 --2- 172.21.15.31:0/4277706506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bf74d0 0x55ac17bf7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.871+0000 7f57df646b80 1 --2- 172.21.15.31:0/4277706506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bee450 0x55ac17bee820 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.871+0000 7f57df646b80 1 --2- 172.21.15.31:0/4277706506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 0x55ac17bf6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.871+0000 7f57df646b80 1 -- 172.21.15.31:0/4277706506 >> 172.21.15.31:0/4277706506 conn(0x55ac17bf1880 msgr2=0x55ac17bf1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.871+0000 7f57df646b80 1 -- 172.21.15.31:0/4277706506 shutdown_connections 2023-10-06T05:24:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.871+0000 7f57df646b80 1 -- 172.21.15.31:0/4277706506 wait complete. 2023-10-06T05:24:53.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.872+0000 7f57df646b80 1 -- start start 2023-10-06T05:24:53.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.872+0000 7f57df646b80 1 Processor -- start 2023-10-06T05:24:53.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57df646b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bd3f40 0x55ac17bed920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57df646b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 0x55ac17bc8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57df646b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17bc8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57df646b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac17af78e0 con 0x55ac17bf74d0 2023-10-06T05:24:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57df646b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac17aa3980 con 0x55ac17bee450 2023-10-06T05:24:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57df646b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac17aa3620 con 0x55ac17bd3f40 2023-10-06T05:24:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57df646b80 10 client.0 _ll_register_callbacks cb 0x55ac17906a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:24:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57cad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17bc8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57ca59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bd3f40 0x55ac17bed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57c9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 0x55ac17bc8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57cad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17bc8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55662/0 (socket says 172.21.15.31:55662) 2023-10-06T05:24:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.873+0000 7f57cad9d700 1 -- 172.21.15.31:0/3553346645 learned_addr learned my addr 172.21.15.31:0/3553346645 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57cad9d700 1 -- 172.21.15.31:0/3553346645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bd3f40 msgr2=0x55ac17bed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57cad9d700 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bd3f40 0x55ac17bed920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57cad9d700 1 -- 172.21.15.31:0/3553346645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 msgr2=0x55ac17bc8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57cad9d700 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 0x55ac17bc8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57cad9d700 1 -- 172.21.15.31:0/3553346645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc0cb000 con 0x55ac17bf74d0 2023-10-06T05:24:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57cad9d700 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17bc8ea0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f57c4052520 tx=0x7f57c4059040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a11d96e8b8300e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:24:53.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57a8ff9700 1 -- 172.21.15.31:0/3553346645 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4083570 con 0x55ac17bf74d0 2023-10-06T05:24:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57a8ff9700 1 -- 172.21.15.31:0/3553346645 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f57c405cfc0 con 0x55ac17bf74d0 2023-10-06T05:24:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57df646b80 1 -- 172.21.15.31:0/3553346645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac17b9bd60 con 0x55ac17bf74d0 2023-10-06T05:24:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57a8ff9700 1 -- 172.21.15.31:0/3553346645 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57c406c3e0 con 0x55ac17bf74d0 2023-10-06T05:24:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57df646b80 10 client.8595 fetch_fsmap learned FSMap version 93 2023-10-06T05:24:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57df646b80 1 -- 172.21.15.31:0/3553346645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55ac179ea0a0 con 0x55ac17bf74d0 2023-10-06T05:24:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.874+0000 7f57a8ff9700 1 -- 172.21.15.31:0/3553346645 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac17b9bd60 con 0x55ac17bf74d0 2023-10-06T05:24:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.875+0000 7f57a8ff9700 1 -- 172.21.15.31:0/3553346645 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 93) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f57c4083170 con 0x55ac17bf74d0 2023-10-06T05:24:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.875+0000 7f57df646b80 10 client.8595 fetch_fsmap finished waiting for FSMap version 93 2023-10-06T05:24:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.875+0000 7f57df646b80 10 client.8595 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:24:53.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.875+0000 7f57df646b80 20 client.8595 populate_metadata read hostname 'smithi031' 2023-10-06T05:24:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.875+0000 7f57df646b80 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.31:6833/3829644591,v1:172.21.15.31:6835/3829644591] conn(0x55ac17bcb260 0x55ac17bcb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.875+0000 7f57df646b80 4 client.8595 mds_command: new command op to 8478 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:24:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.875+0000 7f57df646b80 1 -- 172.21.15.31:0/3553346645 --> [v2:172.21.15.31:6833/3829644591,v1:172.21.15.31:6835/3829644591] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac17b78b30 con 0x55ac17bcb260 2023-10-06T05:24:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.875+0000 7f57ca59c700 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.31:6833/3829644591,v1:172.21.15.31:6835/3829644591] conn(0x55ac17bcb260 0x55ac17bcb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.875+0000 7f57ca59c700 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.31:6833/3829644591,v1:172.21.15.31:6835/3829644591] conn(0x55ac17bcb260 0x55ac17bcb630 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c69387eae646c2 server_cookie=651f5a2f4637ce8e in_seq=0 out_seq=0 2023-10-06T05:24:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.875+0000 7f57a8ff9700 10 client.8595 ms_handle_connect on v2:172.21.15.31:6833/3829644591 2023-10-06T05:24:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.876+0000 7f57a8ff9700 1 -- 172.21.15.31:0/3553346645 <== mds.0 v2:172.21.15.31:6833/3829644591 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ac17b78b30 con 0x55ac17bcb260 2023-10-06T05:24:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.876+0000 7f57a8ff9700 10 client.8595 handle_command_reply: tid=0 2023-10-06T05:24:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.876+0000 7f57df646b80 1 client.8595 shutdown 2023-10-06T05:24:53.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.876+0000 7f57df646b80 1 -- 172.21.15.31:0/3553346645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 msgr2=0x55ac17bc8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.876+0000 7f57df646b80 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17bc8ea0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f57c4052520 tx=0x7f57c4059040 comp rx=0 tx=0).stop 2023-10-06T05:24:53.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.877+0000 7f57df646b80 1 -- 172.21.15.31:0/3553346645 shutdown_connections 2023-10-06T05:24:53.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.877+0000 7f57df646b80 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.31:6833/3829644591,v1:172.21.15.31:6835/3829644591] conn(0x55ac17bcb260 0x55ac17bcb630 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.877+0000 7f57df646b80 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17bc8ea0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.877+0000 7f57df646b80 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bd3f40 0x55ac17bed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.877+0000 7f57a8ff9700 0 client.8595 ms_handle_reset on v2:172.21.15.31:6833/3829644591 2023-10-06T05:24:53.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.877+0000 7f57df646b80 1 --2- 172.21.15.31:0/3553346645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 0x55ac17bc8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.877+0000 7f57df646b80 1 -- 172.21.15.31:0/3553346645 >> 172.21.15.31:0/3553346645 conn(0x55ac17bf1880 msgr2=0x55ac17ba3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.877+0000 7f57df646b80 1 -- 172.21.15.31:0/3553346645 shutdown_connections 2023-10-06T05:24:53.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.877+0000 7f57df646b80 1 -- 172.21.15.31:0/3553346645 wait complete. 2023-10-06T05:24:53.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.877+0000 7f57df646b80 20 client.8595 trim_cache size 0 max 16384 2023-10-06T05:24:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.894+0000 7f57df646b80 1 Processor -- start 2023-10-06T05:24:53.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.894+0000 7f57df646b80 1 -- start start 2023-10-06T05:24:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57df646b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 0x55ac17ba7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57df646b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bee450 0x55ac17ba81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57df646b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17ba8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57df646b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac17ba3a90 con 0x55ac17bf74d0 2023-10-06T05:24:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57df646b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1798f1e0 con 0x55ac17bd3f40 2023-10-06T05:24:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57df646b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac17bed330 con 0x55ac17bee450 2023-10-06T05:24:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57cad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17ba8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57cad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17ba8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55676/0 (socket says 172.21.15.31:55676) 2023-10-06T05:24:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57cad9d700 1 -- 172.21.15.31:0/4069651865 learned_addr learned my addr 172.21.15.31:0/4069651865 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57c9d9b700 1 --2- 172.21.15.31:0/4069651865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bee450 0x55ac17ba81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.895+0000 7f57ca59c700 1 --2- 172.21.15.31:0/4069651865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 0x55ac17ba7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.896+0000 7f57cad9d700 1 -- 172.21.15.31:0/4069651865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bee450 msgr2=0x55ac17ba81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.896+0000 7f57cad9d700 1 --2- 172.21.15.31:0/4069651865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bee450 0x55ac17ba81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.896+0000 7f57cad9d700 1 -- 172.21.15.31:0/4069651865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 msgr2=0x55ac17ba7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.896+0000 7f57cad9d700 1 --2- 172.21.15.31:0/4069651865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 0x55ac17ba7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.896+0000 7f57cad9d700 1 -- 172.21.15.31:0/4069651865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac17ac63c0 con 0x55ac17bf74d0 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.896+0000 7f57cad9d700 1 --2- 172.21.15.31:0/4069651865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17ba8720 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f57c4052a40 tx=0x7f57c4069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9131303123e361a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.896+0000 7f57abfff700 1 -- 172.21.15.31:0/4069651865 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4052360 con 0x55ac17bf74d0 2023-10-06T05:24:53.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.896+0000 7f57abfff700 1 -- 172.21.15.31:0/4069651865 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c405cfc0 con 0x55ac17bf74d0 2023-10-06T05:24:53.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.896+0000 7f57abfff700 1 -- 172.21.15.31:0/4069651865 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c406c3e0 con 0x55ac17bf74d0 2023-10-06T05:24:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.897+0000 7f57df646b80 1 -- 172.21.15.31:0/4069651865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 msgr2=0x55ac17ba8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.897+0000 7f57df646b80 1 --2- 172.21.15.31:0/4069651865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17ba8720 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f57c4052a40 tx=0x7f57c4069000 comp rx=0 tx=0).stop 2023-10-06T05:24:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.897+0000 7f57df646b80 1 -- 172.21.15.31:0/4069651865 shutdown_connections 2023-10-06T05:24:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.897+0000 7f57df646b80 1 --2- 172.21.15.31:0/4069651865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bf74d0 0x55ac17ba8720 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.897+0000 7f57df646b80 1 --2- 172.21.15.31:0/4069651865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bd3f40 0x55ac17ba7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.897+0000 7f57df646b80 1 --2- 172.21.15.31:0/4069651865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bee450 0x55ac17ba81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.897+0000 7f57df646b80 1 -- 172.21.15.31:0/4069651865 >> 172.21.15.31:0/4069651865 conn(0x55ac17bf1880 msgr2=0x55ac17ba47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.897+0000 7f57df646b80 1 -- 172.21.15.31:0/4069651865 shutdown_connections 2023-10-06T05:24:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.897+0000 7f57df646b80 1 -- 172.21.15.31:0/4069651865 wait complete. 2023-10-06T05:24:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.897+0000 7f57df646b80 1 -- start start 2023-10-06T05:24:53.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.898+0000 7f57df646b80 1 Processor -- start 2023-10-06T05:24:53.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.899+0000 7f57df646b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bd3f40 0x55ac17bcbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.899+0000 7f57df646b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 0x55ac17bcc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57df646b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bf74d0 0x55ac17bcf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57df646b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac17baa090 con 0x55ac17bd3f40 2023-10-06T05:24:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57df646b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac17c543b0 con 0x55ac17bee450 2023-10-06T05:24:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57df646b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac17c007a0 con 0x55ac17bf74d0 2023-10-06T05:24:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57df646b80 10 client.0 _ll_register_callbacks cb 0x55ac17906a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:24:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57ca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bd3f40 0x55ac17bcbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57ca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bd3f40 0x55ac17bcbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55678/0 (socket says 172.21.15.31:55678) 2023-10-06T05:24:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57ca59c700 1 -- 172.21.15.31:0/731997802 learned_addr learned my addr 172.21.15.31:0/731997802 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57cad9d700 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bf74d0 0x55ac17bcf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57c9d9b700 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 0x55ac17bcc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57ca59c700 1 -- 172.21.15.31:0/731997802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bf74d0 msgr2=0x55ac17bcf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57ca59c700 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bf74d0 0x55ac17bcf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57ca59c700 1 -- 172.21.15.31:0/731997802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 msgr2=0x55ac17bcc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57ca59c700 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 0x55ac17bcc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57ca59c700 1 -- 172.21.15.31:0/731997802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c406c040 con 0x55ac17bd3f40 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.900+0000 7f57c9d9b700 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 0x55ac17bcc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57ca59c700 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bd3f40 0x55ac17bcbfd0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f57bc003010 tx=0x7f57bc02b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129e5e22ebf373de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57a8ff9700 1 -- 172.21.15.31:0/731997802 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc038040 con 0x55ac17bd3f40 2023-10-06T05:24:53.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57a8ff9700 1 -- 172.21.15.31:0/731997802 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f57bc03e6a0 con 0x55ac17bd3f40 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57df646b80 1 -- 172.21.15.31:0/731997802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57bc029990 con 0x55ac17bd3f40 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57a8ff9700 1 -- 172.21.15.31:0/731997802 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57bc020040 con 0x55ac17bd3f40 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57df646b80 10 client.8601 fetch_fsmap learned FSMap version 93 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57df646b80 1 -- 172.21.15.31:0/731997802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55ac1795e1d0 con 0x55ac17bd3f40 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57a8ff9700 1 -- 172.21.15.31:0/731997802 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc029990 con 0x55ac17bd3f40 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57a8ff9700 1 -- 172.21.15.31:0/731997802 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 93) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f57bc03ea70 con 0x55ac17bd3f40 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57df646b80 10 client.8601 fetch_fsmap finished waiting for FSMap version 93 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57df646b80 10 client.8601 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57df646b80 20 client.8601 populate_metadata read hostname 'smithi031' 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57df646b80 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.31:6833/3829644591,v1:172.21.15.31:6835/3829644591] conn(0x55ac17bcf7a0 0x55ac17bcfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:53.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57df646b80 4 client.8601 mds_command: new command op to 8478 tid=0[{"prefix": "status"}] 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.901+0000 7f57df646b80 1 -- 172.21.15.31:0/731997802 --> [v2:172.21.15.31:6833/3829644591,v1:172.21.15.31:6835/3829644591] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ac17a9a8d0 con 0x55ac17bcf7a0 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.902+0000 7f57c9d9b700 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.31:6833/3829644591,v1:172.21.15.31:6835/3829644591] conn(0x55ac17bcf7a0 0x55ac17bcfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.902+0000 7f57c9d9b700 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.31:6833/3829644591,v1:172.21.15.31:6835/3829644591] conn(0x55ac17bcf7a0 0x55ac17bcfb70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=badfa2af74e7a21b server_cookie=5df7d4f8fba061f6 in_seq=0 out_seq=0 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.902+0000 7f57a8ff9700 10 client.8601 ms_handle_connect on v2:172.21.15.31:6833/3829644591 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.902+0000 7f57a8ff9700 1 -- 172.21.15.31:0/731997802 <== mds.0 v2:172.21.15.31:6833/3829644591 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ac17a9a8d0 con 0x55ac17bcf7a0 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.902+0000 7f57a8ff9700 10 client.8601 handle_command_reply: tid=0 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.902+0000 7f57df646b80 1 client.8601 shutdown 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.902+0000 7f57df646b80 1 -- 172.21.15.31:0/731997802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bd3f40 msgr2=0x55ac17bcbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.902+0000 7f57df646b80 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bd3f40 0x55ac17bcbfd0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f57bc003010 tx=0x7f57bc02b040 comp rx=0 tx=0).stop 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57df646b80 1 -- 172.21.15.31:0/731997802 shutdown_connections 2023-10-06T05:24:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57df646b80 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.31:6833/3829644591,v1:172.21.15.31:6835/3829644591] conn(0x55ac17bcf7a0 0x55ac17bcfb70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57df646b80 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ac17bf74d0 0x55ac17bcf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57df646b80 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac17bd3f40 0x55ac17bcbfd0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57a8ff9700 0 client.8601 ms_handle_reset on v2:172.21.15.31:6833/3829644591 2023-10-06T05:24:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57df646b80 1 --2- 172.21.15.31:0/731997802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac17bee450 0x55ac17bcc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57df646b80 1 -- 172.21.15.31:0/731997802 >> 172.21.15.31:0/731997802 conn(0x55ac17bf1880 msgr2=0x55ac17ba4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57df646b80 1 -- 172.21.15.31:0/731997802 shutdown_connections 2023-10-06T05:24:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57df646b80 1 -- 172.21.15.31:0/731997802 wait complete. 2023-10-06T05:24:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57df646b80 20 client.8601 trim_cache size 0 max 16384 2023-10-06T05:24:53.911 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:24:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:24:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:24:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 8478, 2023-10-06T05:24:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:24:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:24:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:24:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5900064570000003, 2023-10-06T05:24:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 93, 2023-10-06T05:24:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 139, 2023-10-06T05:24:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 139, 2023-10-06T05:24:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 13.266932872 2023-10-06T05:24:53.912 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:24:53.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57d0fff700 1 -- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ac0674d0 msgr2=0x7f57ac069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57d0fff700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ac0674d0 0x7f57ac069980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0099b0 tx=0x7f57bc017000 comp rx=0 tx=0).stop 2023-10-06T05:24:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57d0fff700 1 -- 172.21.15.31:0/2483261041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57cc15d2d0 msgr2=0x7f57cc157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.903+0000 7f57d0fff700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57cc15d2d0 0x7f57cc157a90 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009f60 tx=0x7f57c400b040 comp rx=0 tx=0).stop 2023-10-06T05:24:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.904+0000 7f57d0fff700 1 -- 172.21.15.31:0/2483261041 shutdown_connections 2023-10-06T05:24:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.904+0000 7f57d0fff700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f57ac0674d0 0x7f57ac069980 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.904+0000 7f57d0fff700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57cc15d2d0 0x7f57cc157a90 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.904+0000 7f57d0fff700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57cc0cd510 0x7f57cc0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.904+0000 7f57d0fff700 1 --2- 172.21.15.31:0/2483261041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57cc159500 0x7f57cc157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.904+0000 7f57d0fff700 1 -- 172.21.15.31:0/2483261041 >> 172.21.15.31:0/2483261041 conn(0x7f57cc06ec90 msgr2=0x7f57cc158770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.904+0000 7f57d0fff700 1 -- 172.21.15.31:0/2483261041 shutdown_connections 2023-10-06T05:24:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:53.904+0000 7f57d0fff700 1 -- 172.21.15.31:0/2483261041 wait complete. 2023-10-06T05:24:53.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:24:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.116+0000 7f623efff700 1 Processor -- start 2023-10-06T05:24:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.116+0000 7f623efff700 1 -- start start 2023-10-06T05:24:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623efff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62381595a0 0x7f6238159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623efff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6238068d80 0x7f62380c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623efff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62380c95b0 con 0x7f62380cc0d0 2023-10-06T05:24:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623efff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62380c9890 con 0x7f6238068d80 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623efff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62380c9b70 con 0x7f62381595a0 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623d59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6238068d80 0x7f62380c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623d59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6238068d80 0x7f62380c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55692/0 (socket says 172.21.15.31:55692) 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623d59c700 1 -- 172.21.15.31:0/2553238958 learned_addr learned my addr 172.21.15.31:0/2553238958 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623cd9b700 1 --2- 172.21.15.31:0/2553238958 >> v2:172.21.15.62:3300/0 conn(0x7f62381595a0 0x7f6238159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623d59c700 1 -- 172.21.15.31:0/2553238958 >> v2:172.21.15.62:3300/0 conn(0x7f62381595a0 msgr2=0x7f6238159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623d59c700 1 --2- 172.21.15.31:0/2553238958 >> v2:172.21.15.62:3300/0 conn(0x7f62381595a0 0x7f6238159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623d59c700 1 -- 172.21.15.31:0/2553238958 >> v1:172.21.15.62:6789/0 conn(0x7f62380cc0d0 legacy=0x7f6238068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623d59c700 1 -- 172.21.15.31:0/2553238958 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62380ca130 con 0x7f6238068d80 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.118+0000 7f623cd9b700 1 --2- 172.21.15.31:0/2553238958 >> v2:172.21.15.62:3300/0 conn(0x7f62381595a0 0x7f6238159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.119+0000 7f623d59c700 1 --2- 172.21.15.31:0/2553238958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6238068d80 0x7f62380c8930 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6230009f20 tx=0x7f6230009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b6509cfbf555fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.119+0000 7f62377fe700 1 -- 172.21.15.31:0/2553238958 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623000e070 con 0x7f6238068d80 2023-10-06T05:24:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.119+0000 7f62377fe700 1 -- 172.21.15.31:0/2553238958 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623000ba00 con 0x7f6238068d80 2023-10-06T05:24:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.119+0000 7f62377fe700 1 -- 172.21.15.31:0/2553238958 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62300092e0 con 0x7f6238068d80 2023-10-06T05:24:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.119+0000 7f623efff700 1 -- 172.21.15.31:0/2553238958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6238068d80 msgr2=0x7f62380c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.119+0000 7f623efff700 1 --2- 172.21.15.31:0/2553238958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6238068d80 0x7f62380c8930 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6230009f20 tx=0x7f6230009840 comp rx=0 tx=0).stop 2023-10-06T05:24:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.120+0000 7f623efff700 1 -- 172.21.15.31:0/2553238958 shutdown_connections 2023-10-06T05:24:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.120+0000 7f623efff700 1 --2- 172.21.15.31:0/2553238958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6238068d80 0x7f62380c8930 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.120+0000 7f623efff700 1 --2- 172.21.15.31:0/2553238958 >> v2:172.21.15.62:3300/0 conn(0x7f62381595a0 0x7f6238159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.120+0000 7f623efff700 1 -- 172.21.15.31:0/2553238958 >> 172.21.15.31:0/2553238958 conn(0x7f623806b620 msgr2=0x7f623806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.120+0000 7f623efff700 1 -- 172.21.15.31:0/2553238958 shutdown_connections 2023-10-06T05:24:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.120+0000 7f623efff700 1 -- 172.21.15.31:0/2553238958 wait complete. 2023-10-06T05:24:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.121+0000 7f623efff700 1 Processor -- start 2023-10-06T05:24:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.121+0000 7f623efff700 1 -- start start 2023-10-06T05:24:54.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.122+0000 7f623efff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238068d80 0x7f62380c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.122+0000 7f623efff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380cc0d0 0x7f62380b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.122+0000 7f623efff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62381595a0 0x7f62380b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.122+0000 7f623efff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62380c92d0 con 0x7f62380cc0d0 2023-10-06T05:24:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.122+0000 7f623efff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62380c64f0 con 0x7f6238068d80 2023-10-06T05:24:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.122+0000 7f623efff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623816a580 con 0x7f62381595a0 2023-10-06T05:24:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.122+0000 7f6237fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380cc0d0 0x7f62380b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.122+0000 7f6237fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380cc0d0 0x7f62380b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55694/0 (socket says 172.21.15.31:55694) 2023-10-06T05:24:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.122+0000 7f6237fff700 1 -- 172.21.15.31:0/2601168900 learned_addr learned my addr 172.21.15.31:0/2601168900 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.123+0000 7f623cd9b700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238068d80 0x7f62380c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.123+0000 7f623d59c700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62381595a0 0x7f62380b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.123+0000 7f6237fff700 1 -- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62381595a0 msgr2=0x7f62380b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.123+0000 7f6237fff700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62381595a0 0x7f62380b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.123+0000 7f6237fff700 1 -- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238068d80 msgr2=0x7f62380c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.123+0000 7f6237fff700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238068d80 0x7f62380c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.123+0000 7f6237fff700 1 -- 172.21.15.31:0/2601168900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6230009ae0 con 0x7f62380cc0d0 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.123+0000 7f6237fff700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380cc0d0 0x7f62380b2710 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f622400c600 tx=0x7f6224007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.124+0000 7f6234ff9700 1 -- 172.21.15.31:0/2601168900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224011590 con 0x7f62380cc0d0 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.124+0000 7f6234ff9700 1 -- 172.21.15.31:0/2601168900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6224011710 con 0x7f62380cc0d0 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.124+0000 7f6234ff9700 1 -- 172.21.15.31:0/2601168900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224011ad0 con 0x7f62380cc0d0 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.124+0000 7f623efff700 1 -- 172.21.15.31:0/2601168900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62380c5b20 con 0x7f62380cc0d0 2023-10-06T05:24:54.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.124+0000 7f623efff700 1 -- 172.21.15.31:0/2601168900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623816a2a0 con 0x7f62380cc0d0 2023-10-06T05:24:54.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.125+0000 7f623efff700 1 -- 172.21.15.31:0/2601168900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62380c5cb0 con 0x7f62380cc0d0 2023-10-06T05:24:54.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.126+0000 7f6234ff9700 1 -- 172.21.15.31:0/2601168900 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f622401c460 con 0x7f62380cc0d0 2023-10-06T05:24:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.127+0000 7f6234ff9700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f621c0674d0 0x7f621c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.127+0000 7f6234ff9700 1 -- 172.21.15.31:0/2601168900 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6224084e00 con 0x7f62380cc0d0 2023-10-06T05:24:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.127+0000 7f623cd9b700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f621c0674d0 0x7f621c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.128+0000 7f623cd9b700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f621c0674d0 0x7f621c069980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6238071010 tx=0x7f622c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:54.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.131+0000 7f6234ff9700 1 -- 172.21.15.31:0/2601168900 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6224051640 con 0x7f62380cc0d0 2023-10-06T05:24:54.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.381+0000 7f623efff700 1 -- 172.21.15.31:0/2601168900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f623804c380 con 0x7f62380cc0d0 2023-10-06T05:24:54.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.382+0000 7f6234ff9700 1 -- 172.21.15.31:0/2601168900 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6224054c60 con 0x7f62380cc0d0 2023-10-06T05:24:54.384 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:24:54.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.384+0000 7f623efff700 1 -- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f621c0674d0 msgr2=0x7f621c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.384+0000 7f623efff700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f621c0674d0 0x7f621c069980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6238071010 tx=0x7f622c008040 comp rx=0 tx=0).stop 2023-10-06T05:24:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.384+0000 7f623efff700 1 -- 172.21.15.31:0/2601168900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380cc0d0 msgr2=0x7f62380b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.384+0000 7f623efff700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380cc0d0 0x7f62380b2710 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f622400c600 tx=0x7f6224007f60 comp rx=0 tx=0).stop 2023-10-06T05:24:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.384+0000 7f623efff700 1 -- 172.21.15.31:0/2601168900 shutdown_connections 2023-10-06T05:24:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.384+0000 7f623efff700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f621c0674d0 0x7f621c069980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.384+0000 7f623efff700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f62381595a0 0x7f62380b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.384+0000 7f623efff700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238068d80 0x7f62380c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.384+0000 7f623efff700 1 --2- 172.21.15.31:0/2601168900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380cc0d0 0x7f62380b2710 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.384+0000 7f623efff700 1 -- 172.21.15.31:0/2601168900 >> 172.21.15.31:0/2601168900 conn(0x7f623806b620 msgr2=0x7f62380ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.385+0000 7f623efff700 1 -- 172.21.15.31:0/2601168900 shutdown_connections 2023-10-06T05:24:54.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:54.385+0000 7f623efff700 1 -- 172.21.15.31:0/2601168900 wait complete. 2023-10-06T05:24:54.404 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:24:54.404 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:24:54.404 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:24:54.404 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:24:54.404 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:54.404 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:24:54.404 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:24:54.405 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:24:54.405 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:54.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:54.405 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:24:54.452 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:24:54.452 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:24:54.452 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:24:54.453 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41820sec preferred_lft 41820sec 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:24:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:54.455 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:24:54.455 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:54.455 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:54.455 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:24:54.455 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:24:54.455 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:54.455 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:54.456 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:24:54.456 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:24:54.456 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:54.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:54.456 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:24:54.524 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:24:54.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:54.527 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:24:54.576 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:24:54.576 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:24:54.576 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:24:54.577 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:24:54.577 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:24:54.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:54.577 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:24:54.577 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:24:54.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:24:54.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:24:54.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:24:54.578 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:24:54.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:54.750 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:24:54.796 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:24:54.796 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:24:54.797 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:54.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:54.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:24:54.870 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:54.870 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:54.916 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:24:54.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:54.917 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:55.002 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:24:55.002 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:24:55.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:55.142 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:55.143 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:24:55.168 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:55.169 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:24:55.169 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:24:55.169 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:24:55.169 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:24:55.174 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:24:55.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:55.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:24:55.245 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:24:55.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.444+0000 7fb4061ff700 1 Processor -- start 2023-10-06T05:24:55.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.444+0000 7fb4061ff700 1 -- start start 2023-10-06T05:24:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.445+0000 7fb4061ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb400159580 0x7fb4000cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.445+0000 7fb4061ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4000cb9f0 0x7fb4000cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.445+0000 7fb4061ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4000ccc80 0x7fb4000cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.445+0000 7fb4061ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4000cf280 con 0x7fb4000cb9f0 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.445+0000 7fb4061ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4000ce670 con 0x7fb400159580 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.445+0000 7fb4061ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4000ce950 con 0x7fb4000ccc80 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.445+0000 7fb3feffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4000cb9f0 0x7fb4000cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.445+0000 7fb3feffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4000cb9f0 0x7fb4000cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55702/0 (socket says 172.21.15.31:55702) 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.445+0000 7fb3feffd700 1 -- 172.21.15.31:0/1906140195 learned_addr learned my addr 172.21.15.31:0/1906140195 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3ff7fe700 1 --2- 172.21.15.31:0/1906140195 >> v2:172.21.15.62:3300/0 conn(0x7fb400159580 0x7fb4000cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3fffff700 1 --2- 172.21.15.31:0/1906140195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4000ccc80 0x7fb4000cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3feffd700 1 -- 172.21.15.31:0/1906140195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4000ccc80 msgr2=0x7fb4000cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3feffd700 1 --2- 172.21.15.31:0/1906140195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4000ccc80 0x7fb4000cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3feffd700 1 -- 172.21.15.31:0/1906140195 >> v2:172.21.15.62:3300/0 conn(0x7fb400159580 msgr2=0x7fb4000cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3feffd700 1 --2- 172.21.15.31:0/1906140195 >> v2:172.21.15.62:3300/0 conn(0x7fb400159580 0x7fb4000cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3feffd700 1 -- 172.21.15.31:0/1906140195 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4000cec30 con 0x7fb4000cb9f0 2023-10-06T05:24:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3feffd700 1 --2- 172.21.15.31:0/1906140195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000cb9f0 0x7fb4000cf710 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb4000889e0 tx=0x7fb3f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c6699f5f6aa0bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3fe7fc700 1 -- 172.21.15.31:0/1906140195 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400b860 con 0x7fb4000cb9f0 2023-10-06T05:24:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3fe7fc700 1 -- 172.21.15.31:0/1906140195 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f400b9c0 con 0x7fb4000cb9f0 2023-10-06T05:24:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.446+0000 7fb3fe7fc700 1 -- 172.21.15.31:0/1906140195 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400bd60 con 0x7fb4000cb9f0 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.447+0000 7fb4061ff700 1 -- 172.21.15.31:0/1906140195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000cb9f0 msgr2=0x7fb4000cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.447+0000 7fb4061ff700 1 --2- 172.21.15.31:0/1906140195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000cb9f0 0x7fb4000cf710 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb4000889e0 tx=0x7fb3f4009250 comp rx=0 tx=0).stop 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.447+0000 7fb4061ff700 1 -- 172.21.15.31:0/1906140195 shutdown_connections 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.447+0000 7fb4061ff700 1 --2- 172.21.15.31:0/1906140195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4000ccc80 0x7fb4000cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.447+0000 7fb4061ff700 1 --2- 172.21.15.31:0/1906140195 >> v2:172.21.15.62:3300/0 conn(0x7fb400159580 0x7fb4000cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.447+0000 7fb4061ff700 1 --2- 172.21.15.31:0/1906140195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000cb9f0 0x7fb4000cf710 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.447+0000 7fb4061ff700 1 -- 172.21.15.31:0/1906140195 >> 172.21.15.31:0/1906140195 conn(0x7fb400099710 msgr2=0x7fb40006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.447+0000 7fb4061ff700 1 -- 172.21.15.31:0/1906140195 shutdown_connections 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.447+0000 7fb4061ff700 1 -- 172.21.15.31:0/1906140195 wait complete. 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.449+0000 7fb4061ff700 1 Processor -- start 2023-10-06T05:24:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.449+0000 7fb4061ff700 1 -- start start 2023-10-06T05:24:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb4061ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4000cb9f0 0x7fb4000c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb4061ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000ccc80 0x7fb400169510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb4061ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb400159580 0x7fb4000b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb4061ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb400169a50 con 0x7fb4000ccc80 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb4061ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb400169cf0 con 0x7fb4000cb9f0 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb4061ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb400169fd0 con 0x7fb400159580 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000ccc80 0x7fb400169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000ccc80 0x7fb400169510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55708/0 (socket says 172.21.15.31:55708) 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3feffd700 1 -- 172.21.15.31:0/4037741542 learned_addr learned my addr 172.21.15.31:0/4037741542 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3ff7fe700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4000cb9f0 0x7fb4000c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3fffff700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb400159580 0x7fb4000b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3feffd700 1 -- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb400159580 msgr2=0x7fb4000b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3feffd700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb400159580 0x7fb4000b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3feffd700 1 -- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4000cb9f0 msgr2=0x7fb4000c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3feffd700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4000cb9f0 0x7fb4000c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3feffd700 1 -- 172.21.15.31:0/4037741542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f40094f0 con 0x7fb4000ccc80 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.450+0000 7fb3ff7fe700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4000cb9f0 0x7fb4000c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.451+0000 7fb3feffd700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000ccc80 0x7fb400169510 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb4000888e0 tx=0x7fb3f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.451+0000 7fb3e77fe700 1 -- 172.21.15.31:0/4037741542 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4024440 con 0x7fb4000ccc80 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.451+0000 7fb3e77fe700 1 -- 172.21.15.31:0/4037741542 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f40245a0 con 0x7fb4000ccc80 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.451+0000 7fb4061ff700 1 -- 172.21.15.31:0/4037741542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40016a2b0 con 0x7fb4000ccc80 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.451+0000 7fb3e77fe700 1 -- 172.21.15.31:0/4037741542 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4024960 con 0x7fb4000ccc80 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.451+0000 7fb4061ff700 1 -- 172.21.15.31:0/4037741542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4000b18e0 con 0x7fb4000ccc80 2023-10-06T05:24:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.452+0000 7fb4061ff700 1 -- 172.21.15.31:0/4037741542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4000c74c0 con 0x7fb4000ccc80 2023-10-06T05:24:55.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.453+0000 7fb3e77fe700 1 -- 172.21.15.31:0/4037741542 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb3f401a420 con 0x7fb4000ccc80 2023-10-06T05:24:55.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.455+0000 7fb3e77fe700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3ec0674d0 0x7fb3ec069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:55.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.455+0000 7fb3e77fe700 1 -- 172.21.15.31:0/4037741542 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb3f4086f40 con 0x7fb4000ccc80 2023-10-06T05:24:55.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.455+0000 7fb3ff7fe700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3ec0674d0 0x7fb3ec069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:55.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.456+0000 7fb3ff7fe700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3ec0674d0 0x7fb3ec069980 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0005fd0 tx=0x7fb3f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:55.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.459+0000 7fb3e77fe700 1 -- 172.21.15.31:0/4037741542 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4000c74c0 con 0x7fb4000ccc80 2023-10-06T05:24:55.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.703+0000 7fb4061ff700 1 -- 172.21.15.31:0/4037741542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb4000c74c0 con 0x7fb4000ccc80 2023-10-06T05:24:55.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.704+0000 7fb3e77fe700 1 -- 172.21.15.31:0/4037741542 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb3f4015060 con 0x7fb4000ccc80 2023-10-06T05:24:55.706 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:24:55.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 -- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3ec0674d0 msgr2=0x7fb3ec069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3ec0674d0 0x7fb3ec069980 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0005fd0 tx=0x7fb3f0008040 comp rx=0 tx=0).stop 2023-10-06T05:24:55.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 -- 172.21.15.31:0/4037741542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000ccc80 msgr2=0x7fb400169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:55.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000ccc80 0x7fb400169510 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb4000888e0 tx=0x7fb3f4017000 comp rx=0 tx=0).stop 2023-10-06T05:24:55.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 -- 172.21.15.31:0/4037741542 shutdown_connections 2023-10-06T05:24:55.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb3ec0674d0 0x7fb3ec069980 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb400159580 0x7fb4000b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4000cb9f0 0x7fb4000c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 --2- 172.21.15.31:0/4037741542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4000ccc80 0x7fb400169510 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:55.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 -- 172.21.15.31:0/4037741542 >> 172.21.15.31:0/4037741542 conn(0x7fb400099710 msgr2=0x7fb400068250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:55.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 -- 172.21.15.31:0/4037741542 shutdown_connections 2023-10-06T05:24:55.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:55.706+0000 7fb4061ff700 1 -- 172.21.15.31:0/4037741542 wait complete. 2023-10-06T05:24:55.725 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:24:55.726 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:24:55.726 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:24:55.726 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:24:55.726 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:55.726 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:24:55.726 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:24:55.726 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:24:55.726 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:24:55.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:55.727 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:24:55.779 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:24:55.781 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:24:55.781 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:24:55.781 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:55.781 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:24:55.781 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:55.782 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41819sec preferred_lft 41819sec 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:24:55.783 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:24:55.784 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:55.784 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:24:55.784 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:24:55.784 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:24:55.784 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:24:55.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:55.785 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:24:55.857 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:24:55.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:55.861 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:24:55.910 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:24:55.910 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:24:55.910 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:24:55.910 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:24:55.910 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:24:55.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:55.911 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:24:55.911 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:24:55.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:24:55.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:24:55.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:24:55.912 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:24:56.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:56.085 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:24:56.131 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:24:56.131 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:24:56.132 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:24:56.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:56.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:24:56.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:56.204 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:56.250 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:24:56.251 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:24:56.251 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:56.337 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:24:56.337 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:24:56.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:56.438 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:56.439 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:24:56.465 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:24:56.465 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:24:56.465 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:24:56.465 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:24:56.465 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:24:56.470 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:24:56.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:24:56.470 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:24:56.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:24:56.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.750+0000 7fdcb07ff700 1 Processor -- start 2023-10-06T05:24:56.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.750+0000 7fdcb07ff700 1 -- start start 2023-10-06T05:24:56.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.751+0000 7fdcb07ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca815a990 0x7fdca80cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:56.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.751+0000 7fdcb07ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca80cc5b0 con 0x7fdca8142c20 2023-10-06T05:24:56.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcb07ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca815d210 con 0x7fdca8142ff0 2023-10-06T05:24:56.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcb07ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80cc810 con 0x7fdca815a990 2023-10-06T05:24:56.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcae59b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdca8142c20 0x7fdca80cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42358/0 (socket says 172.21.15.31:42358) 2023-10-06T05:24:56.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcae59b700 1 -- 172.21.15.31:0/975400221 learned_addr learned my addr 172.21.15.31:0/975400221 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:56.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcadd9a700 1 --2- 172.21.15.31:0/975400221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca815a990 0x7fdca80cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:56.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcad599700 1 -- 172.21.15.31:0/975400221 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432206306 0 0) 0x7fdca80cc5b0 con 0x7fdca8142c20 2023-10-06T05:24:56.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcad599700 1 -- 172.21.15.31:0/975400221 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc94003570 con 0x7fdca8142c20 2023-10-06T05:24:56.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcad599700 1 -- 172.21.15.31:0/975400221 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112296249 0 0) 0x7fdca815d210 con 0x7fdca8142ff0 2023-10-06T05:24:56.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcad599700 1 -- 172.21.15.31:0/975400221 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca80cc5b0 con 0x7fdca8142ff0 2023-10-06T05:24:56.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcadd9a700 1 -- 172.21.15.31:0/975400221 >> v1:172.21.15.62:6789/0 conn(0x7fdca8142ff0 legacy=0x7fdca8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcadd9a700 1 -- 172.21.15.31:0/975400221 >> v1:172.21.15.31:6789/0 conn(0x7fdca8142c20 legacy=0x7fdca80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.752+0000 7fdcadd9a700 1 -- 172.21.15.31:0/975400221 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca815d4b0 con 0x7fdca815a990 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.753+0000 7fdcadd9a700 1 --2- 172.21.15.31:0/975400221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca815a990 0x7fdca80cacd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009910 tx=0x7fdc980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ff51aae2982296 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.753+0000 7fdcad599700 1 -- 172.21.15.31:0/975400221 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98011630 con 0x7fdca815a990 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.753+0000 7fdcad599700 1 -- 172.21.15.31:0/975400221 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc98011790 con 0x7fdca815a990 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.753+0000 7fdcad599700 1 -- 172.21.15.31:0/975400221 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98011b30 con 0x7fdca815a990 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.753+0000 7fdcb07ff700 1 -- 172.21.15.31:0/975400221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca815a990 msgr2=0x7fdca80cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.753+0000 7fdcb07ff700 1 --2- 172.21.15.31:0/975400221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca815a990 0x7fdca80cacd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009910 tx=0x7fdc980091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.754+0000 7fdcb07ff700 1 -- 172.21.15.31:0/975400221 shutdown_connections 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.754+0000 7fdcb07ff700 1 --2- 172.21.15.31:0/975400221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca815a990 0x7fdca80cacd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.754+0000 7fdcb07ff700 1 -- 172.21.15.31:0/975400221 >> 172.21.15.31:0/975400221 conn(0x7fdca80ce2d0 msgr2=0x7fdca80ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:56.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.754+0000 7fdcb07ff700 1 -- 172.21.15.31:0/975400221 shutdown_connections 2023-10-06T05:24:56.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.754+0000 7fdcb07ff700 1 -- 172.21.15.31:0/975400221 wait complete. 2023-10-06T05:24:56.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.755+0000 7fdcb07ff700 1 Processor -- start 2023-10-06T05:24:56.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.755+0000 7fdcb07ff700 1 -- start start 2023-10-06T05:24:56.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.756+0000 7fdcb07ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca8142c20 0x7fdca80b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:56.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.756+0000 7fdcb07ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca8142ff0 0x7fdca80adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:56.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.756+0000 7fdcb07ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdca815a990 0x7fdca80ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.756+0000 7fdcb07ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80b1410 con 0x7fdca8142c20 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.756+0000 7fdcb07ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80b16f0 con 0x7fdca815a990 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.756+0000 7fdcb07ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80b19d0 con 0x7fdca8142ff0 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.756+0000 7fdcae59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca8142c20 0x7fdca80b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.756+0000 7fdcae59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca8142c20 0x7fdca80b3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55710/0 (socket says 172.21.15.31:55710) 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.756+0000 7fdcae59b700 1 -- 172.21.15.31:0/4131182558 learned_addr learned my addr 172.21.15.31:0/4131182558 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdcadd9a700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca8142ff0 0x7fdca80adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdcaed9c700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdca815a990 0x7fdca80ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdcae59b700 1 -- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca8142ff0 msgr2=0x7fdca80adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdcae59b700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca8142ff0 0x7fdca80adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdcae59b700 1 -- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdca815a990 msgr2=0x7fdca80ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:56.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdcae59b700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdca815a990 0x7fdca80ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:56.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdcae59b700 1 -- 172.21.15.31:0/4131182558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98009420 con 0x7fdca8142c20 2023-10-06T05:24:56.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdcae59b700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca8142c20 0x7fdca80b3770 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fdca400c790 tx=0x7fdca4007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:56.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdc9e7fc700 1 -- 172.21.15.31:0/4131182558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4005b30 con 0x7fdca8142c20 2023-10-06T05:24:56.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdc9e7fc700 1 -- 172.21.15.31:0/4131182558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca4005c90 con 0x7fdca8142c20 2023-10-06T05:24:56.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdcb07ff700 1 -- 172.21.15.31:0/4131182558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca80b1cb0 con 0x7fdca8142c20 2023-10-06T05:24:56.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.757+0000 7fdc9e7fc700 1 -- 172.21.15.31:0/4131182558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4007300 con 0x7fdca8142c20 2023-10-06T05:24:56.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.758+0000 7fdcb07ff700 1 -- 172.21.15.31:0/4131182558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca80b1070 con 0x7fdca8142c20 2023-10-06T05:24:56.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.758+0000 7fdcb07ff700 1 -- 172.21.15.31:0/4131182558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca80b21d0 con 0x7fdca8142c20 2023-10-06T05:24:56.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.759+0000 7fdc9e7fc700 1 -- 172.21.15.31:0/4131182558 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fdca401b400 con 0x7fdca8142c20 2023-10-06T05:24:56.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.760+0000 7fdc9e7fc700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c0674d0 0x7fdc8c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:56.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.760+0000 7fdc9e7fc700 1 -- 172.21.15.31:0/4131182558 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fdca4084c60 con 0x7fdca8142c20 2023-10-06T05:24:56.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.761+0000 7fdcadd9a700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c0674d0 0x7fdc8c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:56.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.761+0000 7fdcadd9a700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c0674d0 0x7fdc8c069980 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdca815ad60 tx=0x7fdc98017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:56.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:56.765+0000 7fdc9e7fc700 1 -- 172.21.15.31:0/4131182558 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdca40515d0 con 0x7fdca8142c20 2023-10-06T05:24:57.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.011+0000 7fdcb07ff700 1 -- 172.21.15.31:0/4131182558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdca804c380 con 0x7fdca8142c20 2023-10-06T05:24:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.378+0000 7fdc9e7fc700 1 -- 172.21.15.31:0/4131182558 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v94) v1 ==== 136+0+0 (secure 0 0 0) 0x7fdca405a030 con 0x7fdca8142c20 2023-10-06T05:24:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.380+0000 7fdcb07ff700 1 -- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c0674d0 msgr2=0x7fdc8c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.380+0000 7fdcb07ff700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c0674d0 0x7fdc8c069980 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdca815ad60 tx=0x7fdc98017000 comp rx=0 tx=0).stop 2023-10-06T05:24:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.380+0000 7fdcb07ff700 1 -- 172.21.15.31:0/4131182558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca8142c20 msgr2=0x7fdca80b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.380+0000 7fdcb07ff700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca8142c20 0x7fdca80b3770 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fdca400c790 tx=0x7fdca4007ea0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.380+0000 7fdcb07ff700 1 -- 172.21.15.31:0/4131182558 shutdown_connections 2023-10-06T05:24:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.380+0000 7fdcb07ff700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c0674d0 0x7fdc8c069980 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.380+0000 7fdcb07ff700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdca815a990 0x7fdca80ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.380+0000 7fdcb07ff700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca8142c20 0x7fdca80b3770 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.380+0000 7fdcb07ff700 1 --2- 172.21.15.31:0/4131182558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca8142ff0 0x7fdca80adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.380+0000 7fdcb07ff700 1 -- 172.21.15.31:0/4131182558 >> 172.21.15.31:0/4131182558 conn(0x7fdca80ce2d0 msgr2=0x7fdca815cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.381+0000 7fdcb07ff700 1 -- 172.21.15.31:0/4131182558 shutdown_connections 2023-10-06T05:24:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.381+0000 7fdcb07ff700 1 -- 172.21.15.31:0/4131182558 wait complete. 2023-10-06T05:24:57.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:24:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.596+0000 7f99989ff700 1 Processor -- start 2023-10-06T05:24:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.596+0000 7f99989ff700 1 -- start start 2023-10-06T05:24:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.598+0000 7f99989ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f99940b1610 0x7f99940b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.598+0000 7f99989ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9994068f50 con 0x7f999415dcc0 2023-10-06T05:24:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.598+0000 7f99989ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99940b3ab0 con 0x7f99940b19e0 2023-10-06T05:24:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.598+0000 7f99989ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99940b3d90 con 0x7f99940b1610 2023-10-06T05:24:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.598+0000 7f999259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f99940b1610 0x7f99940b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.598+0000 7f9992d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f999415dcc0 0x7f9994159140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41842/0 (socket says 172.21.15.31:41842) 2023-10-06T05:24:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.598+0000 7f9992d9d700 1 -- 172.21.15.31:0/2106013166 learned_addr learned my addr 172.21.15.31:0/2106013166 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.598+0000 7f999159a700 1 -- 172.21.15.31:0/2106013166 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468681424 0 0) 0x7f9994068f50 con 0x7f999415dcc0 2023-10-06T05:24:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.598+0000 7f999259c700 1 -- 172.21.15.31:0/2106013166 >> v1:172.21.15.62:6789/0 conn(0x7f99940b19e0 legacy=0x7f99940cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.598+0000 7f999259c700 1 -- 172.21.15.31:0/2106013166 >> v1:172.21.15.31:6789/0 conn(0x7f999415dcc0 legacy=0x7f9994159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.599+0000 7f999259c700 1 -- 172.21.15.31:0/2106013166 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994068be0 con 0x7f99940b1610 2023-10-06T05:24:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.599+0000 7f999259c700 1 --2- 172.21.15.31:0/2106013166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 0x7f99940b33d0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f99840098e0 tx=0x7f99840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86db08e710e11da5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.599+0000 7f999159a700 1 -- 172.21.15.31:0/2106013166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984009e80 con 0x7f99940b1610 2023-10-06T05:24:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.599+0000 7f999159a700 1 -- 172.21.15.31:0/2106013166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998400b860 con 0x7f99940b1610 2023-10-06T05:24:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.599+0000 7f999159a700 1 -- 172.21.15.31:0/2106013166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998400bbe0 con 0x7f99940b1610 2023-10-06T05:24:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.600+0000 7f99989ff700 1 -- 172.21.15.31:0/2106013166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 msgr2=0x7f99940b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.600+0000 7f99989ff700 1 --2- 172.21.15.31:0/2106013166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 0x7f99940b33d0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f99840098e0 tx=0x7f99840091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.600+0000 7f99989ff700 1 -- 172.21.15.31:0/2106013166 shutdown_connections 2023-10-06T05:24:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.600+0000 7f99989ff700 1 --2- 172.21.15.31:0/2106013166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 0x7f99940b33d0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.600+0000 7f99989ff700 1 -- 172.21.15.31:0/2106013166 >> 172.21.15.31:0/2106013166 conn(0x7f999406b5b0 msgr2=0x7f999406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.600+0000 7f99989ff700 1 -- 172.21.15.31:0/2106013166 shutdown_connections 2023-10-06T05:24:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.600+0000 7f99989ff700 1 -- 172.21.15.31:0/2106013166 wait complete. 2023-10-06T05:24:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.601+0000 7f99989ff700 1 Processor -- start 2023-10-06T05:24:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.601+0000 7f99989ff700 1 -- start start 2023-10-06T05:24:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.602+0000 7f99989ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 0x7f99940c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.602+0000 7f99989ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99940b19e0 0x7f9994155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.602+0000 7f99989ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999415dcc0 0x7f99940cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.602+0000 7f99989ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9994140830 con 0x7f99940b1610 2023-10-06T05:24:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.602+0000 7f99989ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999416dcb0 con 0x7f999415dcc0 2023-10-06T05:24:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.602+0000 7f99989ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9994151f80 con 0x7f99940b19e0 2023-10-06T05:24:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.602+0000 7f999259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 0x7f99940c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.603+0000 7f999259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 0x7f99940c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36734/0 (socket says 172.21.15.31:36734) 2023-10-06T05:24:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.603+0000 7f999259c700 1 -- 172.21.15.31:0/711249497 learned_addr learned my addr 172.21.15.31:0/711249497 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.603+0000 7f9992d9d700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999415dcc0 0x7f99940cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.603+0000 7f9991d9b700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99940b19e0 0x7f9994155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.603+0000 7f999259c700 1 -- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99940b19e0 msgr2=0x7f9994155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.603+0000 7f999259c700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99940b19e0 0x7f9994155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.603+0000 7f999259c700 1 -- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999415dcc0 msgr2=0x7f99940cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.603+0000 7f999259c700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999415dcc0 0x7f99940cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.603+0000 7f999259c700 1 -- 172.21.15.31:0/711249497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9984009420 con 0x7f99940b1610 2023-10-06T05:24:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.603+0000 7f999259c700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 0x7f99940c7a20 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f998400b3c0 tx=0x7f9984017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.604+0000 7f99827fc700 1 -- 172.21.15.31:0/711249497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998401f070 con 0x7f99940b1610 2023-10-06T05:24:57.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.604+0000 7f99827fc700 1 -- 172.21.15.31:0/711249497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998400b9d0 con 0x7f99940b1610 2023-10-06T05:24:57.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.604+0000 7f99827fc700 1 -- 172.21.15.31:0/711249497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984025650 con 0x7f99940b1610 2023-10-06T05:24:57.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.604+0000 7f99989ff700 1 -- 172.21.15.31:0/711249497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999416cdc0 con 0x7f99940b1610 2023-10-06T05:24:57.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.604+0000 7f99989ff700 1 -- 172.21.15.31:0/711249497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999416d380 con 0x7f99940b1610 2023-10-06T05:24:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.605+0000 7f99989ff700 1 -- 172.21.15.31:0/711249497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999406eac0 con 0x7f99940b1610 2023-10-06T05:24:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.606+0000 7f99827fc700 1 -- 172.21.15.31:0/711249497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7f9984023070 con 0x7f99940b1610 2023-10-06T05:24:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.607+0000 7f99827fc700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99740674d0 0x7f9974069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.607+0000 7f99827fc700 1 -- 172.21.15.31:0/711249497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9984086720 con 0x7f99940b1610 2023-10-06T05:24:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.607+0000 7f9991d9b700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99740674d0 0x7f9974069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.608+0000 7f9991d9b700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99740674d0 0x7f9974069980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f999416cd20 tx=0x7f9988009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:57.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.611+0000 7f99827fc700 1 -- 172.21.15.31:0/711249497 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9984052f60 con 0x7f99940b1610 2023-10-06T05:24:57.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.841+0000 7f99989ff700 1 -- 172.21.15.31:0/711249497 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f999415e140 con 0x7f99740674d0 2023-10-06T05:24:57.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.843+0000 7f99827fc700 1 -- 172.21.15.31:0/711249497 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f999415e140 con 0x7f99740674d0 2023-10-06T05:24:57.845 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:24:57.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:24:57.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:24:57.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:24:57.846 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:24:57.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 -- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99740674d0 msgr2=0x7f9974069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99740674d0 0x7f9974069980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f999416cd20 tx=0x7f9988009040 comp rx=0 tx=0).stop 2023-10-06T05:24:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 -- 172.21.15.31:0/711249497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 msgr2=0x7f99940c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 0x7f99940c7a20 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f998400b3c0 tx=0x7f9984017000 comp rx=0 tx=0).stop 2023-10-06T05:24:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 -- 172.21.15.31:0/711249497 shutdown_connections 2023-10-06T05:24:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99740674d0 0x7f9974069980 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f999415dcc0 0x7f99940cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99940b1610 0x7f99940c7a20 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 --2- 172.21.15.31:0/711249497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99940b19e0 0x7f9994155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 -- 172.21.15.31:0/711249497 >> 172.21.15.31:0/711249497 conn(0x7f999406b5b0 msgr2=0x7f9994152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 -- 172.21.15.31:0/711249497 shutdown_connections 2023-10-06T05:24:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:57.845+0000 7f99989ff700 1 -- 172.21.15.31:0/711249497 wait complete. 2023-10-06T05:24:57.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:24:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.062+0000 7fcba29ff700 1 Processor -- start 2023-10-06T05:24:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.062+0000 7fcba29ff700 1 -- start start 2023-10-06T05:24:58.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.063+0000 7fcba29ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb9c1573a0 0x7fcb9c0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcba29ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb9c157770 0x7fcb9c0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcba29ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcb9c157be0 0x7fcb9c0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcba29ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c158cf0 con 0x7fcb9c1573a0 2023-10-06T05:24:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcba29ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c1580e0 con 0x7fcb9c157be0 2023-10-06T05:24:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcba29ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c1583c0 con 0x7fcb9c157770 2023-10-06T05:24:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcb9bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb9c1573a0 0x7fcb9c0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcb9bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb9c1573a0 0x7fcb9c0acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36738/0 (socket says 172.21.15.31:36738) 2023-10-06T05:24:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcb9bfff700 1 -- 172.21.15.31:0/4255888643 learned_addr learned my addr 172.21.15.31:0/4255888643 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcb9b7fe700 1 --2- 172.21.15.31:0/4255888643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb9c157770 0x7fcb9c0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcba0f9c700 1 --2- 172.21.15.31:0/4255888643 >> v2:172.21.15.62:3300/0 conn(0x7fcb9c157be0 0x7fcb9c0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcb9bfff700 1 -- 172.21.15.31:0/4255888643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb9c157770 msgr2=0x7fcb9c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcb9bfff700 1 --2- 172.21.15.31:0/4255888643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb9c157770 0x7fcb9c0b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcb9bfff700 1 -- 172.21.15.31:0/4255888643 >> v2:172.21.15.62:3300/0 conn(0x7fcb9c157be0 msgr2=0x7fcb9c0b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcb9bfff700 1 --2- 172.21.15.31:0/4255888643 >> v2:172.21.15.62:3300/0 conn(0x7fcb9c157be0 0x7fcb9c0b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcb9bfff700 1 -- 172.21.15.31:0/4255888643 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c1586a0 con 0x7fcb9c1573a0 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.064+0000 7fcba0f9c700 1 --2- 172.21.15.31:0/4255888643 >> v2:172.21.15.62:3300/0 conn(0x7fcb9c157be0 0x7fcb9c0b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.065+0000 7fcb9bfff700 1 --2- 172.21.15.31:0/4255888643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c1573a0 0x7fcb9c0acb70 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c06b3a0 tx=0x7fcb8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aea49a7ab6cad03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.065+0000 7fcb9affd700 1 -- 172.21.15.31:0/4255888643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c009e70 con 0x7fcb9c1573a0 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.065+0000 7fcb9affd700 1 -- 172.21.15.31:0/4255888643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c00b860 con 0x7fcb9c1573a0 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.065+0000 7fcb9affd700 1 -- 172.21.15.31:0/4255888643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00bc00 con 0x7fcb9c1573a0 2023-10-06T05:24:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.065+0000 7fcba29ff700 1 -- 172.21.15.31:0/4255888643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c1573a0 msgr2=0x7fcb9c0acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.065+0000 7fcba29ff700 1 --2- 172.21.15.31:0/4255888643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c1573a0 0x7fcb9c0acb70 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c06b3a0 tx=0x7fcb8c009250 comp rx=0 tx=0).stop 2023-10-06T05:24:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.066+0000 7fcba29ff700 1 -- 172.21.15.31:0/4255888643 shutdown_connections 2023-10-06T05:24:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.066+0000 7fcba29ff700 1 --2- 172.21.15.31:0/4255888643 >> v2:172.21.15.62:3300/0 conn(0x7fcb9c157be0 0x7fcb9c0b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.066+0000 7fcba29ff700 1 --2- 172.21.15.31:0/4255888643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c1573a0 0x7fcb9c0acb70 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.066+0000 7fcba29ff700 1 --2- 172.21.15.31:0/4255888643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb9c157770 0x7fcb9c0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.066+0000 7fcba29ff700 1 -- 172.21.15.31:0/4255888643 >> 172.21.15.31:0/4255888643 conn(0x7fcb9c067560 msgr2=0x7fcb9c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.066+0000 7fcba29ff700 1 -- 172.21.15.31:0/4255888643 shutdown_connections 2023-10-06T05:24:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.066+0000 7fcba29ff700 1 -- 172.21.15.31:0/4255888643 wait complete. 2023-10-06T05:24:58.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.067+0000 7fcba29ff700 1 Processor -- start 2023-10-06T05:24:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.067+0000 7fcba29ff700 1 -- start start 2023-10-06T05:24:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.068+0000 7fcba29ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb9c1573a0 0x7fcb9c147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.068+0000 7fcba29ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb9c157770 0x7fcb9c0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.068+0000 7fcba29ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c157be0 0x7fcb9c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.068+0000 7fcba29ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0aa820 con 0x7fcb9c157be0 2023-10-06T05:24:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.068+0000 7fcba29ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0aab00 con 0x7fcb9c1573a0 2023-10-06T05:24:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.068+0000 7fcba29ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0aade0 con 0x7fcb9c157770 2023-10-06T05:24:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.068+0000 7fcba0f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c157be0 0x7fcb9c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.068+0000 7fcba0f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c157be0 0x7fcb9c0cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36742/0 (socket says 172.21.15.31:36742) 2023-10-06T05:24:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.068+0000 7fcba0f9c700 1 -- 172.21.15.31:0/18337790 learned_addr learned my addr 172.21.15.31:0/18337790 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcb9bfff700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb9c1573a0 0x7fcb9c147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcb9b7fe700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb9c157770 0x7fcb9c0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcba0f9c700 1 -- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb9c157770 msgr2=0x7fcb9c0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcba0f9c700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb9c157770 0x7fcb9c0cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcba0f9c700 1 -- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb9c1573a0 msgr2=0x7fcb9c147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcba0f9c700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb9c1573a0 0x7fcb9c147a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcba0f9c700 1 -- 172.21.15.31:0/18337790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0094f0 con 0x7fcb9c157be0 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcba0f9c700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c157be0 0x7fcb9c0cc6d0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009fc0 tx=0x7fcb940099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcb83fff700 1 -- 172.21.15.31:0/18337790 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400dd40 con 0x7fcb9c157be0 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcb83fff700 1 -- 172.21.15.31:0/18337790 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9400f040 con 0x7fcb9c157be0 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcb83fff700 1 -- 172.21.15.31:0/18337790 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94015690 con 0x7fcb9c157be0 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.069+0000 7fcba29ff700 1 -- 172.21.15.31:0/18337790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9c0ab0c0 con 0x7fcb9c157be0 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.070+0000 7fcba29ff700 1 -- 172.21.15.31:0/18337790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c147f70 con 0x7fcb9c157be0 2023-10-06T05:24:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.070+0000 7fcba29ff700 1 -- 172.21.15.31:0/18337790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9c04c380 con 0x7fcb9c157be0 2023-10-06T05:24:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.071+0000 7fcb83fff700 1 -- 172.21.15.31:0/18337790 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fcb9401e3b0 con 0x7fcb9c157be0 2023-10-06T05:24:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.072+0000 7fcb83fff700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb880674d0 0x7fcb88069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.072+0000 7fcb83fff700 1 -- 172.21.15.31:0/18337790 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fcb940857d0 con 0x7fcb9c157be0 2023-10-06T05:24:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.073+0000 7fcb9bfff700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb880674d0 0x7fcb88069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.073+0000 7fcb9bfff700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb880674d0 0x7fcb88069980 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c06b3a0 tx=0x7fcb8c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.077+0000 7fcb83fff700 1 -- 172.21.15.31:0/18337790 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb94051f00 con 0x7fcb9c157be0 2023-10-06T05:24:58.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.280+0000 7fcba29ff700 1 -- 172.21.15.31:0/18337790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcb9c0b1fb0 con 0x7fcb9c157be0 2023-10-06T05:24:58.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.280+0000 7fcb83fff700 1 -- 172.21.15.31:0/18337790 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcb94055520 con 0x7fcb9c157be0 2023-10-06T05:24:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 -- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb880674d0 msgr2=0x7fcb88069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb880674d0 0x7fcb88069980 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c06b3a0 tx=0x7fcb8c017000 comp rx=0 tx=0).stop 2023-10-06T05:24:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 -- 172.21.15.31:0/18337790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c157be0 msgr2=0x7fcb9c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c157be0 0x7fcb9c0cc6d0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009fc0 tx=0x7fcb940099b0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 -- 172.21.15.31:0/18337790 shutdown_connections 2023-10-06T05:24:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb880674d0 0x7fcb88069980 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb9c157be0 0x7fcb9c0cc6d0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb9c1573a0 0x7fcb9c147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 --2- 172.21.15.31:0/18337790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb9c157770 0x7fcb9c0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 -- 172.21.15.31:0/18337790 >> 172.21.15.31:0/18337790 conn(0x7fcb9c067560 msgr2=0x7fcb9c06e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.282+0000 7fcba29ff700 1 -- 172.21.15.31:0/18337790 shutdown_connections 2023-10-06T05:24:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.283+0000 7fcba29ff700 1 -- 172.21.15.31:0/18337790 wait complete. 2023-10-06T05:24:58.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:58.304 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvolume_0000000000085904) 2023-10-06T05:24:58.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:58.383 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/group_0000000000276376/subvolume_0000000000085905) 2023-10-06T05:24:58.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000085904 2023-10-06T05:24:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.656+0000 7feea15ff700 1 Processor -- start 2023-10-06T05:24:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.656+0000 7feea15ff700 1 -- start start 2023-10-06T05:24:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.657+0000 7feea15ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fee9c0af890 0x7fee9c161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.657+0000 7feea15ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9c0afc90 con 0x7fee9c153d40 2023-10-06T05:24:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.657+0000 7feea15ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9c0ad690 con 0x7fee9c0b0b90 2023-10-06T05:24:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.657+0000 7feea15ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9c0acd60 con 0x7fee9c0af890 2023-10-06T05:24:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee9affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fee9c153d40 0x7fee9c0b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41846/0 (socket says 172.21.15.31:41846) 2023-10-06T05:24:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee9affd700 1 -- 172.21.15.31:0/1166833122 learned_addr learned my addr 172.21.15.31:0/1166833122 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:24:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee9b7fe700 1 --2- 172.21.15.31:0/1166833122 >> v2:172.21.15.62:3300/0 conn(0x7fee9c0af890 0x7fee9c161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527163524 0 0) 0x7fee8c001110 con 0x7fee9c153d40 2023-10-06T05:24:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee84003570 con 0x7fee9c153d40 2023-10-06T05:24:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2827377881 0 0) 0x7fee84003570 con 0x7fee9c153d40 2023-10-06T05:24:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 >> v2:172.21.15.62:3300/0 conn(0x7fee9c0af890 msgr2=0x7fee9c161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee99ffb700 1 --2- 172.21.15.31:0/1166833122 >> v2:172.21.15.62:3300/0 conn(0x7fee9c0af890 0x7fee9c161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 >> v1:172.21.15.62:6789/0 conn(0x7fee9c0b0b90 legacy=0x7fee9c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c0ad040 con 0x7fee9c153d40 2023-10-06T05:24:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.658+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fee8c001600 con 0x7fee9c153d40 2023-10-06T05:24:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 >> v1:172.21.15.31:6789/0 conn(0x7fee9c153d40 legacy=0x7fee9c0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee99ffb700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee84003a60 0x7fee84003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee99ffb700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee84004810 0x7fee84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee99ffb700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee8400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0acd60 con 0x7fee840051c0 2023-10-06T05:24:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8400c0d0 con 0x7fee84003a60 2023-10-06T05:24:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8400c290 con 0x7fee84004810 2023-10-06T05:24:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee9b7fe700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee8400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee9affd700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee84003a60 0x7fee84003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee9a7fc700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee84004810 0x7fee84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee9b7fe700 1 -- 172.21.15.31:0/1166833122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee84004810 msgr2=0x7fee84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee9b7fe700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee84004810 0x7fee84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee9b7fe700 1 -- 172.21.15.31:0/1166833122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee84003a60 msgr2=0x7fee84003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee9b7fe700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee84003a60 0x7fee84003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.659+0000 7fee9b7fe700 1 -- 172.21.15.31:0/1166833122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee84003570 con 0x7fee840051c0 2023-10-06T05:24:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.660+0000 7fee9b7fe700 1 -- 172.21.15.31:0/1166833122 reap_dead start 2023-10-06T05:24:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.660+0000 7fee9b7fe700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee8400ba60 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fee9400e210 tx=0x7fee9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587346e08e98bfb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.660+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9400eac0 con 0x7fee840051c0 2023-10-06T05:24:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.660+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9400f040 con 0x7fee840051c0 2023-10-06T05:24:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.660+0000 7fee99ffb700 1 -- 172.21.15.31:0/1166833122 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9400ee90 con 0x7fee840051c0 2023-10-06T05:24:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.660+0000 7feea15ff700 1 -- 172.21.15.31:0/1166833122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 msgr2=0x7fee8400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.660+0000 7feea15ff700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee8400ba60 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fee9400e210 tx=0x7fee9400b040 comp rx=0 tx=0).stop 2023-10-06T05:24:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.661+0000 7feea15ff700 1 -- 172.21.15.31:0/1166833122 shutdown_connections 2023-10-06T05:24:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.661+0000 7feea15ff700 1 --2- 172.21.15.31:0/1166833122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee8400ba60 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.661+0000 7feea15ff700 1 -- 172.21.15.31:0/1166833122 >> 172.21.15.31:0/1166833122 conn(0x7fee9c067560 msgr2=0x7fee9c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.661+0000 7feea15ff700 1 -- 172.21.15.31:0/1166833122 shutdown_connections 2023-10-06T05:24:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.661+0000 7feea15ff700 1 -- 172.21.15.31:0/1166833122 wait complete. 2023-10-06T05:24:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.662+0000 7feea15ff700 1 Processor -- start 2023-10-06T05:24:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.662+0000 7feea15ff700 1 -- start start 2023-10-06T05:24:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.663+0000 7feea15ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee9c1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.663+0000 7feea15ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee9c15a9d0 0x7fee9c158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.663+0000 7feea15ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee9c1557d0 0x7fee9c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.663+0000 7feea15ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c1654a0 con 0x7fee840051c0 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.663+0000 7feea15ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c150f20 con 0x7fee9c1557d0 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.663+0000 7feea15ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c156180 con 0x7fee9c15a9d0 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.663+0000 7fee9affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee9c1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.663+0000 7fee9affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee9c1509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36752/0 (socket says 172.21.15.31:36752) 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee9affd700 1 -- 172.21.15.31:0/3720200156 learned_addr learned my addr 172.21.15.31:0/3720200156 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee9b7fe700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee9c1557d0 0x7fee9c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee9a7fc700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee9c15a9d0 0x7fee9c158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee9affd700 1 -- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee9c15a9d0 msgr2=0x7fee9c158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee9affd700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee9c15a9d0 0x7fee9c158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee9affd700 1 -- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee9c1557d0 msgr2=0x7fee9c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee9affd700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee9c1557d0 0x7fee9c155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee9affd700 1 -- 172.21.15.31:0/3720200156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c165720 con 0x7fee840051c0 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee9a7fc700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee9c15a9d0 0x7fee9c158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee9affd700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee9c1509e0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fee8c007390 tx=0x7fee8c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee82ffd700 1 -- 172.21.15.31:0/3720200156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00f8d0 con 0x7fee840051c0 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee82ffd700 1 -- 172.21.15.31:0/3720200156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c00fa70 con 0x7fee840051c0 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7fee82ffd700 1 -- 172.21.15.31:0/3720200156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00fe30 con 0x7fee840051c0 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.664+0000 7feea15ff700 1 -- 172.21.15.31:0/3720200156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c159100 con 0x7fee840051c0 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.665+0000 7feea15ff700 1 -- 172.21.15.31:0/3720200156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c1596c0 con 0x7fee840051c0 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.666+0000 7feea15ff700 1 -- 172.21.15.31:0/3720200156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9c1598e0 con 0x7fee840051c0 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.666+0000 7fee82ffd700 1 -- 172.21.15.31:0/3720200156 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fee8c013600 con 0x7fee840051c0 2023-10-06T05:24:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.667+0000 7fee82ffd700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee880674d0 0x7fee88069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.668+0000 7fee82ffd700 1 -- 172.21.15.31:0/3720200156 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fee8c0852b0 con 0x7fee840051c0 2023-10-06T05:24:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.668+0000 7fee9a7fc700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee880674d0 0x7fee88069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.669+0000 7fee9a7fc700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee880674d0 0x7fee88069980 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee90000f60 tx=0x7fee90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.671+0000 7fee82ffd700 1 -- 172.21.15.31:0/3720200156 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee8c051af0 con 0x7fee840051c0 2023-10-06T05:24:58.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.905+0000 7feea15ff700 1 -- 172.21.15.31:0/3720200156 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000085904", "target": ["mon-mgr", ""]}) v1 -- 0x7fee9c0afc90 con 0x7fee880674d0 2023-10-06T05:24:58.963 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:58.914+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:24:58.964 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:58.914+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:24:58.964 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:58.914+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:24:58.964 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:58.914+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:24:58.964 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:24:58.914+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:24:58.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.948+0000 7fee82ffd700 1 -- 172.21.15.31:0/3720200156 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7fee9c0afc90 con 0x7fee880674d0 2023-10-06T05:24:58.964 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000085904 2023-10-06T05:24:58.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 -- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee880674d0 msgr2=0x7fee88069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee880674d0 0x7fee88069980 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee90000f60 tx=0x7fee90009040 comp rx=0 tx=0).stop 2023-10-06T05:24:58.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 -- 172.21.15.31:0/3720200156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 msgr2=0x7fee9c1509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:58.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee9c1509e0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fee8c007390 tx=0x7fee8c007a50 comp rx=0 tx=0).stop 2023-10-06T05:24:58.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 -- 172.21.15.31:0/3720200156 shutdown_connections 2023-10-06T05:24:58.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee880674d0 0x7fee88069980 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee9c1557d0 0x7fee9c155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee840051c0 0x7fee9c1509e0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 --2- 172.21.15.31:0/3720200156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee9c15a9d0 0x7fee9c158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:58.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 -- 172.21.15.31:0/3720200156 >> 172.21.15.31:0/3720200156 conn(0x7fee9c067560 msgr2=0x7fee9c154320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:58.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 -- 172.21.15.31:0/3720200156 shutdown_connections 2023-10-06T05:24:58.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:58.950+0000 7feea15ff700 1 -- 172.21.15.31:0/3720200156 wait complete. 2023-10-06T05:24:58.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000085905 group_0000000000276376 2023-10-06T05:24:59.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.166+0000 7fcd711ff700 1 Processor -- start 2023-10-06T05:24:59.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.167+0000 7fcd711ff700 1 -- start start 2023-10-06T05:24:59.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd711ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcd6c0cf7a0 0x7fcd6c15a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd711ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 0x7fcd6c163230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd711ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6c0cd050 con 0x7fcd6c159c70 2023-10-06T05:24:59.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd711ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6c0cde90 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd711ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0ce020 con 0x7fcd6c15a580 2023-10-06T05:24:59.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd6a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcd6c0cf7a0 0x7fcd6c15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd6a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcd6c0cf7a0 0x7fcd6c15a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36764/0 (socket says 172.21.15.31:36764) 2023-10-06T05:24:59.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd6a59c700 1 -- 172.21.15.31:0/3090651981 learned_addr learned my addr 172.21.15.31:0/3090651981 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:59.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd6b59e700 1 --2- 172.21.15.31:0/3090651981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 0x7fcd6c163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd6a59c700 1 -- 172.21.15.31:0/3090651981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 msgr2=0x7fcd6c163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:59.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.168+0000 7fcd6a59c700 1 --2- 172.21.15.31:0/3090651981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 0x7fcd6c163230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.169+0000 7fcd6a59c700 1 -- 172.21.15.31:0/3090651981 >> v1:172.21.15.62:6789/0 conn(0x7fcd6c159c70 legacy=0x7fcd6c0caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.169+0000 7fcd6a59c700 1 -- 172.21.15.31:0/3090651981 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c0ce300 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.169+0000 7fcd6a59c700 1 --2- 172.21.15.31:0/3090651981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 0x7fcd6c15a040 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fcd600099b0 tx=0x7fcd60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b1175d242718d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:59.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.169+0000 7fcd69d9b700 1 -- 172.21.15.31:0/3090651981 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60009ea0 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.169+0000 7fcd69d9b700 1 -- 172.21.15.31:0/3090651981 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6000b860 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.169+0000 7fcd69d9b700 1 -- 172.21.15.31:0/3090651981 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000bc00 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.170+0000 7fcd711ff700 1 -- 172.21.15.31:0/3090651981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 msgr2=0x7fcd6c15a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.170+0000 7fcd711ff700 1 --2- 172.21.15.31:0/3090651981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 0x7fcd6c15a040 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fcd600099b0 tx=0x7fcd60009250 comp rx=0 tx=0).stop 2023-10-06T05:24:59.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.170+0000 7fcd711ff700 1 -- 172.21.15.31:0/3090651981 shutdown_connections 2023-10-06T05:24:59.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.170+0000 7fcd711ff700 1 --2- 172.21.15.31:0/3090651981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 0x7fcd6c163230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.170+0000 7fcd711ff700 1 --2- 172.21.15.31:0/3090651981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 0x7fcd6c15a040 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.170+0000 7fcd711ff700 1 -- 172.21.15.31:0/3090651981 >> 172.21.15.31:0/3090651981 conn(0x7fcd6c099710 msgr2=0x7fcd6c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:59.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.170+0000 7fcd711ff700 1 -- 172.21.15.31:0/3090651981 shutdown_connections 2023-10-06T05:24:59.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.170+0000 7fcd711ff700 1 -- 172.21.15.31:0/3090651981 wait complete. 2023-10-06T05:24:59.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.171+0000 7fcd711ff700 1 Processor -- start 2023-10-06T05:24:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.171+0000 7fcd711ff700 1 -- start start 2023-10-06T05:24:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.172+0000 7fcd711ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 0x7fcd6c0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.172+0000 7fcd711ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd6c159c70 0x7fcd6c0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.172+0000 7fcd711ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 0x7fcd6c16c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.172+0000 7fcd711ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0b3960 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.172+0000 7fcd711ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0b3f20 con 0x7fcd6c159c70 2023-10-06T05:24:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.172+0000 7fcd711ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c16d3e0 con 0x7fcd6c15a580 2023-10-06T05:24:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 0x7fcd6c0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 0x7fcd6c0ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36772/0 (socket says 172.21.15.31:36772) 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6ad9d700 1 -- 172.21.15.31:0/1207192650 learned_addr learned my addr 172.21.15.31:0/1207192650 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6b59e700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 0x7fcd6c16c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6a59c700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd6c159c70 0x7fcd6c0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6ad9d700 1 -- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 msgr2=0x7fcd6c16c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6ad9d700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 0x7fcd6c16c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6ad9d700 1 -- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd6c159c70 msgr2=0x7fcd6c0c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6ad9d700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd6c159c70 0x7fcd6c0c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6ad9d700 1 -- 172.21.15.31:0/1207192650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd600094f0 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6b59e700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 0x7fcd6c16c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6a59c700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd6c159c70 0x7fcd6c0c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:24:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.173+0000 7fcd6ad9d700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 0x7fcd6c0ca050 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c7c0 tx=0x7fcd5c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.174+0000 7fcd52ffd700 1 -- 172.21.15.31:0/1207192650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c011590 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.174+0000 7fcd52ffd700 1 -- 172.21.15.31:0/1207192650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c011710 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.174+0000 7fcd52ffd700 1 -- 172.21.15.31:0/1207192650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c011ad0 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.174+0000 7fcd711ff700 1 -- 172.21.15.31:0/1207192650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6c16d6c0 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.174+0000 7fcd711ff700 1 -- 172.21.15.31:0/1207192650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6c0c7730 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.175+0000 7fcd711ff700 1 -- 172.21.15.31:0/1207192650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6c04c380 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.176+0000 7fcd52ffd700 1 -- 172.21.15.31:0/1207192650 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fcd5c01b3b0 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.177+0000 7fcd52ffd700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd580674d0 0x7fcd58069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.177+0000 7fcd52ffd700 1 -- 172.21.15.31:0/1207192650 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fcd5c084670 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.177+0000 7fcd6a59c700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd580674d0 0x7fcd58069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.178+0000 7fcd6a59c700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd580674d0 0x7fcd58069980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcd600099b0 tx=0x7fcd60017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:59.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.181+0000 7fcd52ffd700 1 -- 172.21.15.31:0/1207192650 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd5c050f60 con 0x7fcd6c0cf7a0 2023-10-06T05:24:59.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.418+0000 7fcd711ff700 1 -- 172.21.15.31:0/1207192650 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000085905", "group_name": "group_0000000000276376", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd6c0cb810 con 0x7fcd580674d0 2023-10-06T05:24:59.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.438+0000 7fcd52ffd700 1 -- 172.21.15.31:0/1207192650 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fcd6c0cb810 con 0x7fcd580674d0 2023-10-06T05:24:59.440 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000276376/subvolume_0000000000085905 2023-10-06T05:24:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 -- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd580674d0 msgr2=0x7fcd58069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd580674d0 0x7fcd58069980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcd600099b0 tx=0x7fcd60017000 comp rx=0 tx=0).stop 2023-10-06T05:24:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 -- 172.21.15.31:0/1207192650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 msgr2=0x7fcd6c0ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 0x7fcd6c0ca050 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c7c0 tx=0x7fcd5c007f70 comp rx=0 tx=0).stop 2023-10-06T05:24:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 -- 172.21.15.31:0/1207192650 shutdown_connections 2023-10-06T05:24:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcd580674d0 0x7fcd58069980 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd6c15a580 0x7fcd6c16c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd6c0cf7a0 0x7fcd6c0ca050 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 --2- 172.21.15.31:0/1207192650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd6c159c70 0x7fcd6c0c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 -- 172.21.15.31:0/1207192650 >> 172.21.15.31:0/1207192650 conn(0x7fcd6c099710 msgr2=0x7fcd6c157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:59.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 -- 172.21.15.31:0/1207192650 shutdown_connections 2023-10-06T05:24:59.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.441+0000 7fcd711ff700 1 -- 172.21.15.31:0/1207192650 wait complete. 2023-10-06T05:24:59.460 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000085904 2023-10-06T05:24:59.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.657+0000 7fb09e7ff700 1 Processor -- start 2023-10-06T05:24:59.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.657+0000 7fb09e7ff700 1 -- start start 2023-10-06T05:24:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.658+0000 7fb09e7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb09815ca40 0x7fb098068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.658+0000 7fb09e7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb09806a300 0x7fb0980c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.658+0000 7fb09e7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb098068da0 con 0x7fb0980c8d70 2023-10-06T05:24:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.658+0000 7fb09e7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0980cabc0 con 0x7fb09815ca40 2023-10-06T05:24:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.658+0000 7fb09e7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb098068bb0 con 0x7fb09806a300 2023-10-06T05:24:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.658+0000 7fb0977fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb09806a300 0x7fb0980c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.658+0000 7fb097fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb09815ca40 0x7fb098068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.658+0000 7fb097fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb09815ca40 0x7fb098068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:57368/0 (socket says 172.21.15.31:57368) 2023-10-06T05:24:59.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.658+0000 7fb097fff700 1 -- 172.21.15.31:0/2028973949 learned_addr learned my addr 172.21.15.31:0/2028973949 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:59.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.659+0000 7fb097fff700 1 -- 172.21.15.31:0/2028973949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb09806a300 msgr2=0x7fb0980c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:24:59.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.659+0000 7fb097fff700 1 --2- 172.21.15.31:0/2028973949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb09806a300 0x7fb0980c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.659+0000 7fb097fff700 1 -- 172.21.15.31:0/2028973949 >> v1:172.21.15.62:6789/0 conn(0x7fb0980c8d70 legacy=0x7fb09806cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.659+0000 7fb097fff700 1 -- 172.21.15.31:0/2028973949 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb098069080 con 0x7fb09815ca40 2023-10-06T05:24:59.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.659+0000 7fb0977fe700 1 --2- 172.21.15.31:0/2028973949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb09806a300 0x7fb0980c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:24:59.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.659+0000 7fb097fff700 1 --2- 172.21.15.31:0/2028973949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09815ca40 0x7fb098068670 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb0880099b0 tx=0x7fb088009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2941f174ab9f16cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:59.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb096ffd700 1 -- 172.21.15.31:0/2028973949 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088009ea0 con 0x7fb09815ca40 2023-10-06T05:24:59.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb096ffd700 1 -- 172.21.15.31:0/2028973949 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08800b860 con 0x7fb09815ca40 2023-10-06T05:24:59.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb096ffd700 1 -- 172.21.15.31:0/2028973949 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800bc00 con 0x7fb09815ca40 2023-10-06T05:24:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb09e7ff700 1 -- 172.21.15.31:0/2028973949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09815ca40 msgr2=0x7fb098068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb09e7ff700 1 --2- 172.21.15.31:0/2028973949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09815ca40 0x7fb098068670 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb0880099b0 tx=0x7fb088009250 comp rx=0 tx=0).stop 2023-10-06T05:24:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb09e7ff700 1 -- 172.21.15.31:0/2028973949 shutdown_connections 2023-10-06T05:24:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb09e7ff700 1 --2- 172.21.15.31:0/2028973949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09815ca40 0x7fb098068670 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb09e7ff700 1 --2- 172.21.15.31:0/2028973949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb09806a300 0x7fb0980c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb09e7ff700 1 -- 172.21.15.31:0/2028973949 >> 172.21.15.31:0/2028973949 conn(0x7fb0980ce290 msgr2=0x7fb0980ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb09e7ff700 1 -- 172.21.15.31:0/2028973949 shutdown_connections 2023-10-06T05:24:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.660+0000 7fb09e7ff700 1 -- 172.21.15.31:0/2028973949 wait complete. 2023-10-06T05:24:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.662+0000 7fb09e7ff700 1 Processor -- start 2023-10-06T05:24:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.662+0000 7fb09e7ff700 1 -- start start 2023-10-06T05:24:59.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb09e7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb09806a300 0x7fb0980c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb09e7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0980c8d70 0x7fb09816ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb09e7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09815ca40 0x7fb09816b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb09e7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0980c5fc0 con 0x7fb09806a300 2023-10-06T05:24:59.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb09e7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0980c5340 con 0x7fb09815ca40 2023-10-06T05:24:59.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb09e7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0980c5620 con 0x7fb0980c8d70 2023-10-06T05:24:59.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb097fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb09806a300 0x7fb0980c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb097fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb09806a300 0x7fb0980c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36786/0 (socket says 172.21.15.31:36786) 2023-10-06T05:24:59.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb097fff700 1 -- 172.21.15.31:0/1940081175 learned_addr learned my addr 172.21.15.31:0/1940081175 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:24:59.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb0977fe700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0980c8d70 0x7fb09816ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb09cd9c700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09815ca40 0x7fb09816b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb097fff700 1 -- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0980c8d70 msgr2=0x7fb09816ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb097fff700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0980c8d70 0x7fb09816ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb097fff700 1 -- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09815ca40 msgr2=0x7fb09816b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb097fff700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09815ca40 0x7fb09816b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.663+0000 7fb097fff700 1 -- 172.21.15.31:0/1940081175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c009420 con 0x7fb09806a300 2023-10-06T05:24:59.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.664+0000 7fb097fff700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb09806a300 0x7fb0980c63e0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fb09806e040 tx=0x7fb088017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:59.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.664+0000 7fb07ffff700 1 -- 172.21.15.31:0/1940081175 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088000d10 con 0x7fb09806a300 2023-10-06T05:24:59.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.664+0000 7fb07ffff700 1 -- 172.21.15.31:0/1940081175 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08800bbe0 con 0x7fb09806a300 2023-10-06T05:24:59.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.664+0000 7fb07ffff700 1 -- 172.21.15.31:0/1940081175 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088011670 con 0x7fb09806a300 2023-10-06T05:24:59.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.664+0000 7fb09e7ff700 1 -- 172.21.15.31:0/1940081175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0880094f0 con 0x7fb09806a300 2023-10-06T05:24:59.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.664+0000 7fb09e7ff700 1 -- 172.21.15.31:0/1940081175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0980c5be0 con 0x7fb09806a300 2023-10-06T05:24:59.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.665+0000 7fb09e7ff700 1 -- 172.21.15.31:0/1940081175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0980c6100 con 0x7fb09806a300 2023-10-06T05:24:59.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.666+0000 7fb07ffff700 1 -- 172.21.15.31:0/1940081175 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 85250+0+0 (secure 0 0 0) 0x7fb088011aa0 con 0x7fb09806a300 2023-10-06T05:24:59.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.667+0000 7fb07ffff700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb0840674d0 0x7fb084069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:24:59.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.667+0000 7fb07ffff700 1 -- 172.21.15.31:0/1940081175 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb088086800 con 0x7fb09806a300 2023-10-06T05:24:59.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.667+0000 7fb0977fe700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb0840674d0 0x7fb084069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:24:59.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.668+0000 7fb0977fe700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb0840674d0 0x7fb084069980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009f40 tx=0x7fb08c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:24:59.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.671+0000 7fb07ffff700 1 -- 172.21.15.31:0/1940081175 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0980c6100 con 0x7fb09806a300 2023-10-06T05:24:59.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.903+0000 7fb09e7ff700 1 -- 172.21.15.31:0/1940081175 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000085904", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0980c6100 con 0x7fb0840674d0 2023-10-06T05:24:59.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.909+0000 7fb07ffff700 1 -- 172.21.15.31:0/1940081175 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7fb0980c6100 con 0x7fb0840674d0 2023-10-06T05:24:59.911 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000000085904 2023-10-06T05:24:59.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.911+0000 7fb09e7ff700 1 -- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb0840674d0 msgr2=0x7fb084069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.911+0000 7fb09e7ff700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb0840674d0 0x7fb084069980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009f40 tx=0x7fb08c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.911+0000 7fb09e7ff700 1 -- 172.21.15.31:0/1940081175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb09806a300 msgr2=0x7fb0980c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:24:59.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.911+0000 7fb09e7ff700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb09806a300 0x7fb0980c63e0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fb09806e040 tx=0x7fb088017000 comp rx=0 tx=0).stop 2023-10-06T05:24:59.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.912+0000 7fb09e7ff700 1 -- 172.21.15.31:0/1940081175 shutdown_connections 2023-10-06T05:24:59.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.912+0000 7fb09e7ff700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb0840674d0 0x7fb084069980 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.912+0000 7fb09e7ff700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09815ca40 0x7fb09816b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.912+0000 7fb09e7ff700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb09806a300 0x7fb0980c63e0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.912+0000 7fb09e7ff700 1 --2- 172.21.15.31:0/1940081175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0980c8d70 0x7fb09816ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:24:59.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.912+0000 7fb09e7ff700 1 -- 172.21.15.31:0/1940081175 >> 172.21.15.31:0/1940081175 conn(0x7fb0980ce290 msgr2=0x7fb09815a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:24:59.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.912+0000 7fb09e7ff700 1 -- 172.21.15.31:0/1940081175 shutdown_connections 2023-10-06T05:24:59.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:24:59.912+0000 7fb09e7ff700 1 -- 172.21.15.31:0/1940081175 wait complete. 2023-10-06T05:24:59.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:24:59.931 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/d2133e086190364ef72e8a08f804d35a.meta) 2023-10-06T05:25:00.648 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:25:00.648 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T05:25:00.648 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T05:25:00.648 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000000085904 2023-10-06T05:25:00.648 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:25:00.649 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:00.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000085905 group_0000000000276376 2023-10-06T05:25:00.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.850+0000 7f1185bff700 1 Processor -- start 2023-10-06T05:25:00.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.851+0000 7f1185bff700 1 -- start start 2023-10-06T05:25:00.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.852+0000 7f1185bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1180159410 0x7f11800c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f1185bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11800c7830 con 0x7f11800c73c0 2023-10-06T05:25:00.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f1185bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1180069750 con 0x7f118006b4c0 2023-10-06T05:25:00.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f1185bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1180069a30 con 0x7f1180159410 2023-10-06T05:25:00.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f117effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f11800c73c0 0x7f118006adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41848/0 (socket says 172.21.15.31:41848) 2023-10-06T05:25:00.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f117effd700 1 -- 172.21.15.31:0/3289141551 learned_addr learned my addr 172.21.15.31:0/3289141551 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f117f7fe700 1 --2- 172.21.15.31:0/3289141551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1180159410 0x7f11800c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f117e7fc700 1 -- 172.21.15.31:0/3289141551 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270660746 0 0) 0x7f11800c7830 con 0x7f11800c73c0 2023-10-06T05:25:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f117e7fc700 1 -- 172.21.15.31:0/3289141551 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1168003570 con 0x7f11800c73c0 2023-10-06T05:25:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f117e7fc700 1 -- 172.21.15.31:0/3289141551 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645185014 0 0) 0x7f1180069750 con 0x7f118006b4c0 2023-10-06T05:25:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f117e7fc700 1 -- 172.21.15.31:0/3289141551 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11800c7830 con 0x7f118006b4c0 2023-10-06T05:25:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f117f7fe700 1 -- 172.21.15.31:0/3289141551 >> v1:172.21.15.62:6789/0 conn(0x7f118006b4c0 legacy=0x7f11801597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f117f7fe700 1 -- 172.21.15.31:0/3289141551 >> v1:172.21.15.31:6789/0 conn(0x7f11800c73c0 legacy=0x7f118006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.853+0000 7f117f7fe700 1 -- 172.21.15.31:0/3289141551 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1180069d10 con 0x7f1180159410 2023-10-06T05:25:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.854+0000 7f117f7fe700 1 --2- 172.21.15.31:0/3289141551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1180159410 0x7f11800c6e80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1170009910 tx=0x7f11700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70b6b2741222ad64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.854+0000 7f117e7fc700 1 -- 172.21.15.31:0/3289141551 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1170011630 con 0x7f1180159410 2023-10-06T05:25:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.854+0000 7f117e7fc700 1 -- 172.21.15.31:0/3289141551 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1170011790 con 0x7f1180159410 2023-10-06T05:25:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.854+0000 7f117e7fc700 1 -- 172.21.15.31:0/3289141551 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1170011b30 con 0x7f1180159410 2023-10-06T05:25:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.855+0000 7f1185bff700 1 -- 172.21.15.31:0/3289141551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1180159410 msgr2=0x7f11800c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.855+0000 7f1185bff700 1 --2- 172.21.15.31:0/3289141551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1180159410 0x7f11800c6e80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1170009910 tx=0x7f11700091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.855+0000 7f1185bff700 1 -- 172.21.15.31:0/3289141551 shutdown_connections 2023-10-06T05:25:00.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.855+0000 7f1185bff700 1 --2- 172.21.15.31:0/3289141551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1180159410 0x7f11800c6e80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.855+0000 7f1185bff700 1 -- 172.21.15.31:0/3289141551 >> 172.21.15.31:0/3289141551 conn(0x7f11800c9350 msgr2=0x7f11800c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:00.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.855+0000 7f1185bff700 1 -- 172.21.15.31:0/3289141551 shutdown_connections 2023-10-06T05:25:00.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.855+0000 7f1185bff700 1 -- 172.21.15.31:0/3289141551 wait complete. 2023-10-06T05:25:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.857+0000 7f1185bff700 1 Processor -- start 2023-10-06T05:25:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.857+0000 7f1185bff700 1 -- start start 2023-10-06T05:25:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.858+0000 7f1185bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f118006b4c0 0x7f118016a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.858+0000 7f1185bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11800c73c0 0x7f11800ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.858+0000 7f1185bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180159410 0x7f1180158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.858+0000 7f1185bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118016a760 con 0x7f11800c73c0 2023-10-06T05:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.858+0000 7f1185bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11800ae2e0 con 0x7f1180159410 2023-10-06T05:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.858+0000 7f1185bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11800ad6d0 con 0x7f118006b4c0 2023-10-06T05:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11800c73c0 0x7f11800ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11800c73c0 0x7f11800ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36794/0 (socket says 172.21.15.31:36794) 2023-10-06T05:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117effd700 1 -- 172.21.15.31:0/2408639893 learned_addr learned my addr 172.21.15.31:0/2408639893 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117f7fe700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f118006b4c0 0x7f118016a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117ffff700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180159410 0x7f1180158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117effd700 1 -- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f118006b4c0 msgr2=0x7f118016a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117effd700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f118006b4c0 0x7f118016a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117effd700 1 -- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180159410 msgr2=0x7f1180158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117effd700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180159410 0x7f1180158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117effd700 1 -- 172.21.15.31:0/2408639893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1170009420 con 0x7f11800c73c0 2023-10-06T05:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.859+0000 7f117effd700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11800c73c0 0x7f11800ae770 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f117400c790 tx=0x7f1174007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.860+0000 7f11677fe700 1 -- 172.21.15.31:0/2408639893 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174005b30 con 0x7f11800c73c0 2023-10-06T05:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.860+0000 7f11677fe700 1 -- 172.21.15.31:0/2408639893 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1174005c90 con 0x7f11800c73c0 2023-10-06T05:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.860+0000 7f1185bff700 1 -- 172.21.15.31:0/2408639893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11800ad9b0 con 0x7f11800c73c0 2023-10-06T05:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.860+0000 7f11677fe700 1 -- 172.21.15.31:0/2408639893 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174007300 con 0x7f11800c73c0 2023-10-06T05:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.860+0000 7f1185bff700 1 -- 172.21.15.31:0/2408639893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11800adf70 con 0x7f11800c73c0 2023-10-06T05:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.861+0000 7f1185bff700 1 -- 172.21.15.31:0/2408639893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118004c380 con 0x7f11800c73c0 2023-10-06T05:25:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.862+0000 7f11677fe700 1 -- 172.21.15.31:0/2408639893 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85301+0+0 (secure 0 0 0) 0x7f117401b3b0 con 0x7f11800c73c0 2023-10-06T05:25:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.863+0000 7f11677fe700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f116c067560 0x7f116c069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.863+0000 7f11677fe700 1 -- 172.21.15.31:0/2408639893 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f1174084740 con 0x7f11800c73c0 2023-10-06T05:25:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.863+0000 7f117f7fe700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f116c067560 0x7f116c069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.864+0000 7f117f7fe700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f116c067560 0x7f116c069a10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f11700091e0 tx=0x7f1170017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:00.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:00.867+0000 7f11677fe700 1 -- 172.21.15.31:0/2408639893 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1174050eb0 con 0x7f11800c73c0 2023-10-06T05:25:01.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.103+0000 7f1185bff700 1 -- 172.21.15.31:0/2408639893 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000085905", "group_name": "group_0000000000276376", "target": ["mon-mgr", ""]}) v1 -- 0x7f11800ae420 con 0x7f116c067560 2023-10-06T05:25:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.111+0000 7f11677fe700 1 -- 172.21.15.31:0/2408639893 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f11800ae420 con 0x7f116c067560 2023-10-06T05:25:01.114 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000276376/subvolume_0000000000085905 2023-10-06T05:25:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 -- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f116c067560 msgr2=0x7f116c069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f116c067560 0x7f116c069a10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f11700091e0 tx=0x7f1170017000 comp rx=0 tx=0).stop 2023-10-06T05:25:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 -- 172.21.15.31:0/2408639893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11800c73c0 msgr2=0x7f11800ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11800c73c0 0x7f11800ae770 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f117400c790 tx=0x7f1174007ea0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 -- 172.21.15.31:0/2408639893 shutdown_connections 2023-10-06T05:25:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f116c067560 0x7f116c069a10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180159410 0x7f1180158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f118006b4c0 0x7f118016a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 --2- 172.21.15.31:0/2408639893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11800c73c0 0x7f11800ae770 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 -- 172.21.15.31:0/2408639893 >> 172.21.15.31:0/2408639893 conn(0x7f11800c9350 msgr2=0x7f118015d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 -- 172.21.15.31:0/2408639893 shutdown_connections 2023-10-06T05:25:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:01.114+0000 7f1185bff700 1 -- 172.21.15.31:0/2408639893 wait complete. 2023-10-06T05:25:01.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:01.134 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/a765368d11e5a5acda24e2339e8ee8b7.meta) 2023-10-06T05:25:02.221 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:02.226 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:05.647 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:25:05.648 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T05:25:05.648 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T05:25:05.648 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/group_0000000000276376/subvolume_0000000000085905 2023-10-06T05:25:05.648 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:25:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:05.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000085904 2023-10-06T05:25:05.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.857+0000 7fc79f3ff700 1 Processor -- start 2023-10-06T05:25:05.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.857+0000 7fc79f3ff700 1 -- start start 2023-10-06T05:25:05.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.858+0000 7fc79f3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79815ca40 0x7fc798068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.858+0000 7fc79f3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc79806a300 0x7fc7980c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.858+0000 7fc79f3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc798068da0 con 0x7fc7980c8d70 2023-10-06T05:25:05.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.858+0000 7fc79f3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980cabc0 con 0x7fc79806a300 2023-10-06T05:25:05.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.858+0000 7fc79f3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc798068bb0 con 0x7fc79815ca40 2023-10-06T05:25:05.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.858+0000 7fc79c99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc79806a300 0x7fc7980c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.858+0000 7fc79d99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc7980c8d70 0x7fc79806cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41854/0 (socket says 172.21.15.31:41854) 2023-10-06T05:25:05.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.858+0000 7fc79d99c700 1 -- 172.21.15.31:0/2447051544 learned_addr learned my addr 172.21.15.31:0/2447051544 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.859+0000 7fc79d19b700 1 --2- 172.21.15.31:0/2447051544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79815ca40 0x7fc798068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.859+0000 7fc79c99a700 1 -- 172.21.15.31:0/2447051544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79815ca40 msgr2=0x7fc798068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.859+0000 7fc79c99a700 1 --2- 172.21.15.31:0/2447051544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79815ca40 0x7fc798068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.859+0000 7fc79c99a700 1 -- 172.21.15.31:0/2447051544 >> v1:172.21.15.31:6789/0 conn(0x7fc7980c8d70 legacy=0x7fc79806cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.859+0000 7fc79c99a700 1 -- 172.21.15.31:0/2447051544 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc798069080 con 0x7fc79806a300 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.859+0000 7fc79c99a700 1 --2- 172.21.15.31:0/2447051544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79806a300 0x7fc7980c8830 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc7880098e0 tx=0x7fc7880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e182ce171f4adcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.859+0000 7fc78ffff700 1 -- 172.21.15.31:0/2447051544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788009e80 con 0x7fc79806a300 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.859+0000 7fc78ffff700 1 -- 172.21.15.31:0/2447051544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78800b860 con 0x7fc79806a300 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.859+0000 7fc78ffff700 1 -- 172.21.15.31:0/2447051544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78800bbe0 con 0x7fc79806a300 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.860+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2447051544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79806a300 msgr2=0x7fc7980c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.860+0000 7fc79f3ff700 1 --2- 172.21.15.31:0/2447051544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79806a300 0x7fc7980c8830 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc7880098e0 tx=0x7fc7880091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.860+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2447051544 shutdown_connections 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.860+0000 7fc79f3ff700 1 --2- 172.21.15.31:0/2447051544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79815ca40 0x7fc798068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.860+0000 7fc79f3ff700 1 --2- 172.21.15.31:0/2447051544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79806a300 0x7fc7980c8830 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.860+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2447051544 >> 172.21.15.31:0/2447051544 conn(0x7fc7980ce290 msgr2=0x7fc7980ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.860+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2447051544 shutdown_connections 2023-10-06T05:25:05.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.860+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2447051544 wait complete. 2023-10-06T05:25:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.861+0000 7fc79f3ff700 1 Processor -- start 2023-10-06T05:25:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.862+0000 7fc79f3ff700 1 -- start start 2023-10-06T05:25:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79f3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79806a300 0x7fc7980ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79f3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7980c8d70 0x7fc7980c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79f3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79815ca40 0x7fc7980c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79f3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980ad700 con 0x7fc79815ca40 2023-10-06T05:25:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79f3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980ad9d0 con 0x7fc7980c8d70 2023-10-06T05:25:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79f3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980adcb0 con 0x7fc79806a300 2023-10-06T05:25:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79815ca40 0x7fc7980c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79d19b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79806a300 0x7fc7980ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79d99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79815ca40 0x7fc7980c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36806/0 (socket says 172.21.15.31:36806) 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79d99c700 1 -- 172.21.15.31:0/2287408459 learned_addr learned my addr 172.21.15.31:0/2287408459 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79c99a700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7980c8d70 0x7fc7980c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79d99c700 1 -- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79806a300 msgr2=0x7fc7980ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79d99c700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79806a300 0x7fc7980ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79d99c700 1 -- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7980c8d70 msgr2=0x7fc7980c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79d99c700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7980c8d70 0x7fc7980c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.863+0000 7fc79d99c700 1 -- 172.21.15.31:0/2287408459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc788009420 con 0x7fc79815ca40 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.864+0000 7fc79d99c700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79815ca40 0x7fc7980c82e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fc79000cc40 tx=0x7fc790001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.864+0000 7fc78d7fa700 1 -- 172.21.15.31:0/2287408459 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79000a8b0 con 0x7fc79815ca40 2023-10-06T05:25:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.864+0000 7fc78d7fa700 1 -- 172.21.15.31:0/2287408459 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79000aa10 con 0x7fc79815ca40 2023-10-06T05:25:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.864+0000 7fc78d7fa700 1 -- 172.21.15.31:0/2287408459 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79000add0 con 0x7fc79815ca40 2023-10-06T05:25:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.864+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2287408459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7980adf90 con 0x7fc79815ca40 2023-10-06T05:25:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.864+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2287408459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7980c6a20 con 0x7fc79815ca40 2023-10-06T05:25:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.865+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2287408459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7980c7c20 con 0x7fc79815ca40 2023-10-06T05:25:05.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.866+0000 7fc78d7fa700 1 -- 172.21.15.31:0/2287408459 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85301+0+0 (secure 0 0 0) 0x7fc790007300 con 0x7fc79815ca40 2023-10-06T05:25:05.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.867+0000 7fc78d7fa700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc77c067560 0x7fc77c069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:05.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.867+0000 7fc78d7fa700 1 -- 172.21.15.31:0/2287408459 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc790085560 con 0x7fc79815ca40 2023-10-06T05:25:05.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.867+0000 7fc79d19b700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc77c067560 0x7fc77c069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:05.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.868+0000 7fc79d19b700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc77c067560 0x7fc77c069a10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc794006fd0 tx=0x7fc794008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:05.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:05.871+0000 7fc78d7fa700 1 -- 172.21.15.31:0/2287408459 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7980c7c20 con 0x7fc79815ca40 2023-10-06T05:25:06.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.103+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2287408459 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000085904", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7980c7c20 con 0x7fc77c067560 2023-10-06T05:25:06.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.122+0000 7fc78d7fa700 1 -- 172.21.15.31:0/2287408459 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc7980c7c20 con 0x7fc77c067560 2023-10-06T05:25:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc77c067560 msgr2=0x7fc77c069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc77c067560 0x7fc77c069a10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc794006fd0 tx=0x7fc794008040 comp rx=0 tx=0).stop 2023-10-06T05:25:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2287408459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79815ca40 msgr2=0x7fc7980c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79815ca40 0x7fc7980c82e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fc79000cc40 tx=0x7fc790001a50 comp rx=0 tx=0).stop 2023-10-06T05:25:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2287408459 shutdown_connections 2023-10-06T05:25:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc77c067560 0x7fc77c069a10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc79815ca40 0x7fc7980c82e0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc79806a300 0x7fc7980ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 --2- 172.21.15.31:0/2287408459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7980c8d70 0x7fc7980c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2287408459 >> 172.21.15.31:0/2287408459 conn(0x7fc7980ce290 msgr2=0x7fc79815a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2287408459 shutdown_connections 2023-10-06T05:25:06.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.125+0000 7fc79f3ff700 1 -- 172.21.15.31:0/2287408459 wait complete. 2023-10-06T05:25:06.129 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:06.127+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:06.129 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:06.127+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:06.129 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:06.127+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:06.129 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:06.127+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:06.129 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:06.127+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:06.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000085905 group_0000000000276376 2023-10-06T05:25:06.148 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:06.145+0000 7f2e46982700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:06.148 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:06.145+0000 7f2e46982700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:06.148 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:06.145+0000 7f2e46982700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:06.148 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:06.145+0000 7f2e46982700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:06.148 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:06.145+0000 7f2e46982700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:06.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.346+0000 7f12589ff700 1 Processor -- start 2023-10-06T05:25:06.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.347+0000 7f12589ff700 1 -- start start 2023-10-06T05:25:06.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.348+0000 7f12589ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f125406ee10 0x7f125406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.348+0000 7f12589ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125406f640 con 0x7f1254154cd0 2023-10-06T05:25:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.348+0000 7f12589ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12540adf60 con 0x7f12541550a0 2023-10-06T05:25:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.348+0000 7f12589ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125406e610 con 0x7f125406ee10 2023-10-06T05:25:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.348+0000 7f1252d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f125406ee10 0x7f125406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.348+0000 7f125259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1254154cd0 0x7f12541583e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41866/0 (socket says 172.21.15.31:41866) 2023-10-06T05:25:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.348+0000 7f125259c700 1 -- 172.21.15.31:0/4035474762 learned_addr learned my addr 172.21.15.31:0/4035474762 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.348+0000 7f1251d9b700 1 -- 172.21.15.31:0/4035474762 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963578930 0 0) 0x7f125406f640 con 0x7f1254154cd0 2023-10-06T05:25:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.349+0000 7f1252d9d700 1 -- 172.21.15.31:0/4035474762 >> v1:172.21.15.62:6789/0 conn(0x7f12541550a0 legacy=0x7f12540ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.349+0000 7f1252d9d700 1 -- 172.21.15.31:0/4035474762 >> v1:172.21.15.31:6789/0 conn(0x7f1254154cd0 legacy=0x7f12541583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.349+0000 7f1252d9d700 1 -- 172.21.15.31:0/4035474762 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125406f360 con 0x7f125406ee10 2023-10-06T05:25:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.349+0000 7f1252d9d700 1 --2- 172.21.15.31:0/4035474762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 0x7f125406d7c0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f124c009e60 tx=0x7f124c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba04988c1f66ca1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.349+0000 7f1251d9b700 1 -- 172.21.15.31:0/4035474762 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00e070 con 0x7f125406ee10 2023-10-06T05:25:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.349+0000 7f1251d9b700 1 -- 172.21.15.31:0/4035474762 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c00bc30 con 0x7f125406ee10 2023-10-06T05:25:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.349+0000 7f1251d9b700 1 -- 172.21.15.31:0/4035474762 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c012430 con 0x7f125406ee10 2023-10-06T05:25:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.350+0000 7f12589ff700 1 -- 172.21.15.31:0/4035474762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 msgr2=0x7f125406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.350+0000 7f12589ff700 1 --2- 172.21.15.31:0/4035474762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 0x7f125406d7c0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f124c009e60 tx=0x7f124c009780 comp rx=0 tx=0).stop 2023-10-06T05:25:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.350+0000 7f12589ff700 1 -- 172.21.15.31:0/4035474762 shutdown_connections 2023-10-06T05:25:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.350+0000 7f12589ff700 1 --2- 172.21.15.31:0/4035474762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 0x7f125406d7c0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.350+0000 7f12589ff700 1 -- 172.21.15.31:0/4035474762 >> 172.21.15.31:0/4035474762 conn(0x7f1254068030 msgr2=0x7f12540cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.350+0000 7f12589ff700 1 -- 172.21.15.31:0/4035474762 shutdown_connections 2023-10-06T05:25:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.350+0000 7f12589ff700 1 -- 172.21.15.31:0/4035474762 wait complete. 2023-10-06T05:25:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.351+0000 7f12589ff700 1 Processor -- start 2023-10-06T05:25:06.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.351+0000 7f12589ff700 1 -- start start 2023-10-06T05:25:06.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.352+0000 7f12589ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 0x7f125416a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.352+0000 7f12589ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1254154cd0 0x7f12540ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.352+0000 7f12589ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12541550a0 0x7f12540ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.352+0000 7f12589ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125416b770 con 0x7f125406ee10 2023-10-06T05:25:06.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.352+0000 7f12589ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125416a900 con 0x7f12541550a0 2023-10-06T05:25:06.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.352+0000 7f12589ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125416ab60 con 0x7f1254154cd0 2023-10-06T05:25:06.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.352+0000 7f125259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 0x7f125416a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f125259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 0x7f125416a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36820/0 (socket says 172.21.15.31:36820) 2023-10-06T05:25:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f125259c700 1 -- 172.21.15.31:0/895189729 learned_addr learned my addr 172.21.15.31:0/895189729 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f1252d9d700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12541550a0 0x7f12540ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f124bfff700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1254154cd0 0x7f12540ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f125259c700 1 -- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1254154cd0 msgr2=0x7f12540ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f125259c700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1254154cd0 0x7f12540ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f125259c700 1 -- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12541550a0 msgr2=0x7f12540ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f125259c700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12541550a0 0x7f12540ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f125259c700 1 -- 172.21.15.31:0/895189729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c009a20 con 0x7f125406ee10 2023-10-06T05:25:06.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f125259c700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 0x7f125416a3c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f124400c790 tx=0x7f1244007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.353+0000 7f124a7fc700 1 -- 172.21.15.31:0/895189729 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244001ea0 con 0x7f125406ee10 2023-10-06T05:25:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.354+0000 7f124a7fc700 1 -- 172.21.15.31:0/895189729 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12440072e0 con 0x7f125406ee10 2023-10-06T05:25:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.354+0000 7f12589ff700 1 -- 172.21.15.31:0/895189729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125416ae40 con 0x7f125406ee10 2023-10-06T05:25:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.354+0000 7f124a7fc700 1 -- 172.21.15.31:0/895189729 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244007680 con 0x7f125406ee10 2023-10-06T05:25:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.354+0000 7f12589ff700 1 -- 172.21.15.31:0/895189729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125416b400 con 0x7f125406ee10 2023-10-06T05:25:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.355+0000 7f12589ff700 1 -- 172.21.15.31:0/895189729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125404c380 con 0x7f125406ee10 2023-10-06T05:25:06.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.356+0000 7f124a7fc700 1 -- 172.21.15.31:0/895189729 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85301+0+0 (secure 0 0 0) 0x7f1244005a00 con 0x7f125406ee10 2023-10-06T05:25:06.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.356+0000 7f124a7fc700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1234067560 0x7f1234069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.357+0000 7f124a7fc700 1 -- 172.21.15.31:0/895189729 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f1244084620 con 0x7f125406ee10 2023-10-06T05:25:06.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.357+0000 7f124bfff700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1234067560 0x7f1234069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.358+0000 7f124bfff700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1234067560 0x7f1234069a10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1254157580 tx=0x7f123c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.361+0000 7f124a7fc700 1 -- 172.21.15.31:0/895189729 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f124404b5a0 con 0x7f125406ee10 2023-10-06T05:25:06.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.594+0000 7f12589ff700 1 -- 172.21.15.31:0/895189729 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000085905", "group_name": "group_0000000000276376", "target": ["mon-mgr", ""]}) v1 -- 0x7f125416b8b0 con 0x7f1234067560 2023-10-06T05:25:06.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.613+0000 7f124a7fc700 1 -- 172.21.15.31:0/895189729 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f123c006820 con 0x7f1234067560 2023-10-06T05:25:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.615+0000 7f12589ff700 1 -- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1234067560 msgr2=0x7f1234069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.615+0000 7f12589ff700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1234067560 0x7f1234069a10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1254157580 tx=0x7f123c009040 comp rx=0 tx=0).stop 2023-10-06T05:25:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.616+0000 7f12589ff700 1 -- 172.21.15.31:0/895189729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 msgr2=0x7f125416a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.616+0000 7f12589ff700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 0x7f125416a3c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f124400c790 tx=0x7f1244007c80 comp rx=0 tx=0).stop 2023-10-06T05:25:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.616+0000 7f12589ff700 1 -- 172.21.15.31:0/895189729 shutdown_connections 2023-10-06T05:25:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.616+0000 7f12589ff700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1234067560 0x7f1234069a10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.616+0000 7f12589ff700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12541550a0 0x7f12540ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.616+0000 7f12589ff700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125406ee10 0x7f125416a3c0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.616+0000 7f12589ff700 1 --2- 172.21.15.31:0/895189729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1254154cd0 0x7f12540ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.616+0000 7f12589ff700 1 -- 172.21.15.31:0/895189729 >> 172.21.15.31:0/895189729 conn(0x7f1254068030 msgr2=0x7f1254155520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.616+0000 7f12589ff700 1 -- 172.21.15.31:0/895189729 shutdown_connections 2023-10-06T05:25:06.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.616+0000 7f12589ff700 1 -- 172.21.15.31:0/895189729 wait complete. 2023-10-06T05:25:06.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:06.635 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T05:25:06.727 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T05:25:06.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000276376 2023-10-06T05:25:06.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.925+0000 7fe5691ff700 1 Processor -- start 2023-10-06T05:25:06.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.925+0000 7fe5691ff700 1 -- start start 2023-10-06T05:25:06.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe5691ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe56415ca40 0x7fe564068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe5691ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe56406a300 0x7fe5640c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe5691ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5640c91b0 con 0x7fe5640c8d70 2023-10-06T05:25:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe5691ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5640cabc0 con 0x7fe56415ca40 2023-10-06T05:25:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe5691ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe564068bb0 con 0x7fe56406a300 2023-10-06T05:25:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe562d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe56415ca40 0x7fe564068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe562d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe56415ca40 0x7fe564068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37888/0 (socket says 172.21.15.31:37888) 2023-10-06T05:25:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe562d9d700 1 -- 172.21.15.31:0/655841695 learned_addr learned my addr 172.21.15.31:0/655841695 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe56359e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe5640c8d70 0x7fe56406cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55940/0 (socket says 172.21.15.31:55940) 2023-10-06T05:25:06.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe56259c700 1 --2- 172.21.15.31:0/655841695 >> v2:172.21.15.62:3300/0 conn(0x7fe56406a300 0x7fe5640c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe562d9d700 1 -- 172.21.15.31:0/655841695 >> v2:172.21.15.62:3300/0 conn(0x7fe56406a300 msgr2=0x7fe5640c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe562d9d700 1 --2- 172.21.15.31:0/655841695 >> v2:172.21.15.62:3300/0 conn(0x7fe56406a300 0x7fe5640c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe562d9d700 1 -- 172.21.15.31:0/655841695 >> v1:172.21.15.31:6789/0 conn(0x7fe5640c8d70 legacy=0x7fe56406cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe562d9d700 1 -- 172.21.15.31:0/655841695 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe564068da0 con 0x7fe56415ca40 2023-10-06T05:25:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe56259c700 1 --2- 172.21.15.31:0/655841695 >> v2:172.21.15.62:3300/0 conn(0x7fe56406a300 0x7fe5640c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.927+0000 7fe561d9b700 1 -- 172.21.15.31:0/655841695 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1776437648 0 0) 0x7fe5640c91b0 con 0x7fe5640c8d70 2023-10-06T05:25:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.928+0000 7fe562d9d700 1 --2- 172.21.15.31:0/655841695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe56415ca40 0x7fe564068670 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe56406e950 tx=0x7fe554009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fffc2f7c0b4fed1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.928+0000 7fe561d9b700 1 -- 172.21.15.31:0/655841695 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55400b860 con 0x7fe56415ca40 2023-10-06T05:25:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.928+0000 7fe561d9b700 1 -- 172.21.15.31:0/655841695 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55400b9c0 con 0x7fe56415ca40 2023-10-06T05:25:06.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.928+0000 7fe561d9b700 1 -- 172.21.15.31:0/655841695 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55400bd60 con 0x7fe56415ca40 2023-10-06T05:25:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.928+0000 7fe5691ff700 1 -- 172.21.15.31:0/655841695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe56415ca40 msgr2=0x7fe564068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.928+0000 7fe5691ff700 1 --2- 172.21.15.31:0/655841695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe56415ca40 0x7fe564068670 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe56406e950 tx=0x7fe554009250 comp rx=0 tx=0).stop 2023-10-06T05:25:06.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.929+0000 7fe5691ff700 1 -- 172.21.15.31:0/655841695 shutdown_connections 2023-10-06T05:25:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.929+0000 7fe5691ff700 1 --2- 172.21.15.31:0/655841695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe56415ca40 0x7fe564068670 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.929+0000 7fe5691ff700 1 --2- 172.21.15.31:0/655841695 >> v2:172.21.15.62:3300/0 conn(0x7fe56406a300 0x7fe5640c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.929+0000 7fe5691ff700 1 -- 172.21.15.31:0/655841695 >> 172.21.15.31:0/655841695 conn(0x7fe5640ce290 msgr2=0x7fe5640ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.929+0000 7fe5691ff700 1 -- 172.21.15.31:0/655841695 shutdown_connections 2023-10-06T05:25:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.929+0000 7fe5691ff700 1 -- 172.21.15.31:0/655841695 wait complete. 2023-10-06T05:25:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.930+0000 7fe5691ff700 1 Processor -- start 2023-10-06T05:25:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.930+0000 7fe5691ff700 1 -- start start 2023-10-06T05:25:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.931+0000 7fe5691ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe56406a300 0x7fe564155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.931+0000 7fe5691ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5640c8d70 0x7fe5640c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.931+0000 7fe5691ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56415ca40 0x7fe5640c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.931+0000 7fe5691ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640b1e30 con 0x7fe5640c8d70 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.931+0000 7fe5691ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5641563e0 con 0x7fe56415ca40 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.931+0000 7fe5691ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640c4f70 con 0x7fe56406a300 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe56259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5640c8d70 0x7fe5640c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe56259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5640c8d70 0x7fe5640c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37890/0 (socket says 172.21.15.31:37890) 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe56259c700 1 -- 172.21.15.31:0/3229399643 learned_addr learned my addr 172.21.15.31:0/3229399643 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe56359e700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56415ca40 0x7fe5640c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe562d9d700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe56406a300 0x7fe564155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe56259c700 1 -- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe56406a300 msgr2=0x7fe564155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe56259c700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe56406a300 0x7fe564155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe56259c700 1 -- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56415ca40 msgr2=0x7fe5640c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe56259c700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56415ca40 0x7fe5640c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe56259c700 1 -- 172.21.15.31:0/3229399643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5540094f0 con 0x7fe5640c8d70 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.932+0000 7fe56259c700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5640c8d70 0x7fe5640c4a30 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe55800e8d0 tx=0x7fe558008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.933+0000 7fe54affd700 1 -- 172.21.15.31:0/3229399643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800ee70 con 0x7fe5640c8d70 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.933+0000 7fe54affd700 1 -- 172.21.15.31:0/3229399643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55800c5e0 con 0x7fe5640c8d70 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.933+0000 7fe5691ff700 1 -- 172.21.15.31:0/3229399643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5640c2a50 con 0x7fe5640c8d70 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.933+0000 7fe54affd700 1 -- 172.21.15.31:0/3229399643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800c980 con 0x7fe5640c8d70 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.933+0000 7fe5691ff700 1 -- 172.21.15.31:0/3229399643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5640b17e0 con 0x7fe5640c8d70 2023-10-06T05:25:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.934+0000 7fe5691ff700 1 -- 172.21.15.31:0/3229399643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56404c380 con 0x7fe5640c8d70 2023-10-06T05:25:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.935+0000 7fe54affd700 1 -- 172.21.15.31:0/3229399643 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) v1 ==== 85301+0+0 (secure 0 0 0) 0x7fe558013410 con 0x7fe5640c8d70 2023-10-06T05:25:06.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.936+0000 7fe54affd700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe550067560 0x7fe550069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:06.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.936+0000 7fe54affd700 1 -- 172.21.15.31:0/3229399643 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fe558085160 con 0x7fe5640c8d70 2023-10-06T05:25:06.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.937+0000 7fe562d9d700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe550067560 0x7fe550069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:06.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.937+0000 7fe562d9d700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe550067560 0x7fe550069a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe56406e950 tx=0x7fe554017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:06.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:06.940+0000 7fe54affd700 1 -- 172.21.15.31:0/3229399643 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5580517c0 con 0x7fe5640c8d70 2023-10-06T05:25:07.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.125+0000 7fe54affd700 1 -- 172.21.15.31:0/3229399643 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fe558051b90 con 0x7fe5640c8d70 2023-10-06T05:25:07.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.176+0000 7fe5691ff700 1 -- 172.21.15.31:0/3229399643 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "group_0000000000276376", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5640b1f70 con 0x7fe550067560 2023-10-06T05:25:07.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.181+0000 7fe54affd700 1 -- 172.21.15.31:0/3229399643 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe5640b1f70 con 0x7fe550067560 2023-10-06T05:25:07.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 -- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe550067560 msgr2=0x7fe550069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe550067560 0x7fe550069a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe56406e950 tx=0x7fe554017000 comp rx=0 tx=0).stop 2023-10-06T05:25:07.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 -- 172.21.15.31:0/3229399643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5640c8d70 msgr2=0x7fe5640c4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5640c8d70 0x7fe5640c4a30 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe55800e8d0 tx=0x7fe558008040 comp rx=0 tx=0).stop 2023-10-06T05:25:07.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 -- 172.21.15.31:0/3229399643 shutdown_connections 2023-10-06T05:25:07.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe550067560 0x7fe550069a10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56415ca40 0x7fe5640c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe56406a300 0x7fe564155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 --2- 172.21.15.31:0/3229399643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5640c8d70 0x7fe5640c4a30 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 -- 172.21.15.31:0/3229399643 >> 172.21.15.31:0/3229399643 conn(0x7fe5640ce290 msgr2=0x7fe56406cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:07.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 -- 172.21.15.31:0/3229399643 shutdown_connections 2023-10-06T05:25:07.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.184+0000 7fe5691ff700 1 -- 172.21.15.31:0/3229399643 wait complete. 2023-10-06T05:25:07.203 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:25:07.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:25:07.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:25:07.429 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:25:07.429 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:25:07.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:07.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:07.517 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:25:07.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:07.518 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:25:07.565 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:25:07.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:07.565 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:07.663 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:25:07.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:07.664 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:25:07.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.906+0000 7fc6e13ff700 1 Processor -- start 2023-10-06T05:25:07.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.907+0000 7fc6e13ff700 1 -- start start 2023-10-06T05:25:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.908+0000 7fc6e13ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc6dc0c7f30 0x7fc6dc069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.908+0000 7fc6e13ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6dc0aee70 0x7fc6dc13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.908+0000 7fc6e13ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc0ca5b0 con 0x7fc6dc0c7b60 2023-10-06T05:25:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.908+0000 7fc6e13ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc0af2f0 con 0x7fc6dc0aee70 2023-10-06T05:25:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.908+0000 7fc6e13ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc06f740 con 0x7fc6dc0c7f30 2023-10-06T05:25:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.908+0000 7fc6db7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6dc0aee70 0x7fc6dc13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.908+0000 7fc6db7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6dc0aee70 0x7fc6dc13f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37904/0 (socket says 172.21.15.31:37904) 2023-10-06T05:25:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.908+0000 7fc6db7fe700 1 -- 172.21.15.31:0/831316389 learned_addr learned my addr 172.21.15.31:0/831316389 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.908+0000 7fc6da7fc700 1 --2- 172.21.15.31:0/831316389 >> v2:172.21.15.62:3300/0 conn(0x7fc6dc0c7f30 0x7fc6dc069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.909+0000 7fc6db7fe700 1 -- 172.21.15.31:0/831316389 >> v2:172.21.15.62:3300/0 conn(0x7fc6dc0c7f30 msgr2=0x7fc6dc069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.909+0000 7fc6db7fe700 1 --2- 172.21.15.31:0/831316389 >> v2:172.21.15.62:3300/0 conn(0x7fc6dc0c7f30 0x7fc6dc069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.909+0000 7fc6db7fe700 1 -- 172.21.15.31:0/831316389 >> v1:172.21.15.62:6789/0 conn(0x7fc6dc0c7b60 legacy=0x7fc6dc06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:07.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.909+0000 7fc6db7fe700 1 -- 172.21.15.31:0/831316389 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc0c9c80 con 0x7fc6dc0aee70 2023-10-06T05:25:07.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.909+0000 7fc6da7fc700 1 --2- 172.21.15.31:0/831316389 >> v2:172.21.15.62:3300/0 conn(0x7fc6dc0c7f30 0x7fc6dc069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:07.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.909+0000 7fc6db7fe700 1 --2- 172.21.15.31:0/831316389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 0x7fc6dc13f690 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc06c990 tx=0x7fc6d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a8bbfff8fdc6ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:07.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.909+0000 7fc6d9ffb700 1 -- 172.21.15.31:0/831316389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d400e070 con 0x7fc6dc0aee70 2023-10-06T05:25:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.909+0000 7fc6d9ffb700 1 -- 172.21.15.31:0/831316389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d400ba50 con 0x7fc6dc0aee70 2023-10-06T05:25:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.909+0000 7fc6d9ffb700 1 -- 172.21.15.31:0/831316389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d40092e0 con 0x7fc6dc0aee70 2023-10-06T05:25:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.910+0000 7fc6e13ff700 1 -- 172.21.15.31:0/831316389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 msgr2=0x7fc6dc13f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.910+0000 7fc6e13ff700 1 --2- 172.21.15.31:0/831316389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 0x7fc6dc13f690 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc06c990 tx=0x7fc6d4009840 comp rx=0 tx=0).stop 2023-10-06T05:25:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.910+0000 7fc6e13ff700 1 -- 172.21.15.31:0/831316389 shutdown_connections 2023-10-06T05:25:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.910+0000 7fc6e13ff700 1 --2- 172.21.15.31:0/831316389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 0x7fc6dc13f690 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.910+0000 7fc6e13ff700 1 --2- 172.21.15.31:0/831316389 >> v2:172.21.15.62:3300/0 conn(0x7fc6dc0c7f30 0x7fc6dc069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.910+0000 7fc6e13ff700 1 -- 172.21.15.31:0/831316389 >> 172.21.15.31:0/831316389 conn(0x7fc6dc0cc0b0 msgr2=0x7fc6dc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.910+0000 7fc6e13ff700 1 -- 172.21.15.31:0/831316389 shutdown_connections 2023-10-06T05:25:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.910+0000 7fc6e13ff700 1 -- 172.21.15.31:0/831316389 wait complete. 2023-10-06T05:25:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.911+0000 7fc6e13ff700 1 Processor -- start 2023-10-06T05:25:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.912+0000 7fc6e13ff700 1 -- start start 2023-10-06T05:25:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6e13ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 0x7fc6dc13c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6e13ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6dc0c7b60 0x7fc6dc06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6e13ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc0c7f30 0x7fc6dc06ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6e13ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0aa7e0 con 0x7fc6dc0aee70 2023-10-06T05:25:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6e13ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0aaa80 con 0x7fc6dc0c7f30 2023-10-06T05:25:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6e13ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0aad60 con 0x7fc6dc0c7b60 2023-10-06T05:25:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 0x7fc6dc13c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6daffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 0x7fc6dc13c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37916/0 (socket says 172.21.15.31:37916) 2023-10-06T05:25:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6daffd700 1 -- 172.21.15.31:0/1970221223 learned_addr learned my addr 172.21.15.31:0/1970221223 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6da7fc700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6dc0c7b60 0x7fc6dc06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6db7fe700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc0c7f30 0x7fc6dc06ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6daffd700 1 -- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6dc0c7b60 msgr2=0x7fc6dc06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6daffd700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6dc0c7b60 0x7fc6dc06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6daffd700 1 -- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc0c7f30 msgr2=0x7fc6dc06ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6daffd700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc0c7f30 0x7fc6dc06ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.913+0000 7fc6daffd700 1 -- 172.21.15.31:0/1970221223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d4009ae0 con 0x7fc6dc0aee70 2023-10-06T05:25:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.914+0000 7fc6daffd700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 0x7fc6dc13c2d0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00c7c0 tx=0x7fc6cc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.914+0000 7fc6c2ffd700 1 -- 172.21.15.31:0/1970221223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc011590 con 0x7fc6dc0aee70 2023-10-06T05:25:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.914+0000 7fc6c2ffd700 1 -- 172.21.15.31:0/1970221223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc0116f0 con 0x7fc6dc0aee70 2023-10-06T05:25:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.914+0000 7fc6e13ff700 1 -- 172.21.15.31:0/1970221223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc0ab040 con 0x7fc6dc0aee70 2023-10-06T05:25:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.914+0000 7fc6c2ffd700 1 -- 172.21.15.31:0/1970221223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc011ab0 con 0x7fc6dc0aee70 2023-10-06T05:25:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.914+0000 7fc6e13ff700 1 -- 172.21.15.31:0/1970221223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc13c9d0 con 0x7fc6dc0aee70 2023-10-06T05:25:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.915+0000 7fc6e13ff700 1 -- 172.21.15.31:0/1970221223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6dc04c380 con 0x7fc6dc0aee70 2023-10-06T05:25:07.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.916+0000 7fc6c2ffd700 1 -- 172.21.15.31:0/1970221223 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fc6cc01b3b0 con 0x7fc6dc0aee70 2023-10-06T05:25:07.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.917+0000 7fc6c2ffd700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6c8067680 0x7fc6c8069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:07.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.917+0000 7fc6c2ffd700 1 -- 172.21.15.31:0/1970221223 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc6cc084c40 con 0x7fc6dc0aee70 2023-10-06T05:25:07.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.918+0000 7fc6da7fc700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6c8067680 0x7fc6c8069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:07.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.918+0000 7fc6da7fc700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6c8067680 0x7fc6c8069b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0ae770 tx=0x7fc6d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:07.922+0000 7fc6c2ffd700 1 -- 172.21.15.31:0/1970221223 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6cc082000 con 0x7fc6dc0aee70 2023-10-06T05:25:08.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.169+0000 7fc6e13ff700 1 -- 172.21.15.31:0/1970221223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6dc0a5490 con 0x7fc6dc0aee70 2023-10-06T05:25:08.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.169+0000 7fc6c2ffd700 1 -- 172.21.15.31:0/1970221223 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7fc6cc017070 con 0x7fc6dc0aee70 2023-10-06T05:25:08.171 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:08.171 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1352198494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1352198494},{"type":"v1","addr":"172.21.15.62:6835","nonce":1352198494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8194,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/243154980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":243154980},{"type":"v1","addr":"172.21.15.62:6834","nonce":243154980}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/240917476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":240917476},{"type":"v1","addr":"172.21.15.31:6834","nonce":240917476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:46.824751+0000","modified":"2023-10-06T05:24:50.335585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3829644591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3829644591},{"type":"v1","addr":"172.21.15.31:6835","nonce":3829644591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:08.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 -- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6c8067680 msgr2=0x7fc6c8069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6c8067680 0x7fc6c8069b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0ae770 tx=0x7fc6d0008040 comp rx=0 tx=0).stop 2023-10-06T05:25:08.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 -- 172.21.15.31:0/1970221223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 msgr2=0x7fc6dc13c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 0x7fc6dc13c2d0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00c7c0 tx=0x7fc6cc007f70 comp rx=0 tx=0).stop 2023-10-06T05:25:08.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 -- 172.21.15.31:0/1970221223 shutdown_connections 2023-10-06T05:25:08.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6c8067680 0x7fc6c8069b30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6dc0c7f30 0x7fc6dc06ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6dc0aee70 0x7fc6dc13c2d0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 --2- 172.21.15.31:0/1970221223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6dc0c7b60 0x7fc6dc06e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 -- 172.21.15.31:0/1970221223 >> 172.21.15.31:0/1970221223 conn(0x7fc6dc0cc0b0 msgr2=0x7fc6dc0c1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 -- 172.21.15.31:0/1970221223 shutdown_connections 2023-10-06T05:25:08.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.172+0000 7fc6e13ff700 1 -- 172.21.15.31:0/1970221223 wait complete. 2023-10-06T05:25:08.175 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 94 2023-10-06T05:25:08.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:08.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.387+0000 7f3ef6dff700 1 Processor -- start 2023-10-06T05:25:08.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.387+0000 7f3ef6dff700 1 -- start start 2023-10-06T05:25:08.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.388+0000 7f3ef6dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00ccc80 0x7f3ef00cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.388+0000 7f3ef6dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef00cf280 con 0x7f3ef0159580 2023-10-06T05:25:08.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.388+0000 7f3ef6dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef00cfe20 con 0x7f3ef00cdfe0 2023-10-06T05:25:08.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.388+0000 7f3ef6dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00ce480 con 0x7f3ef00ccc80 2023-10-06T05:25:08.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.388+0000 7f3ef4b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3ef0159580 0x7f3ef00cb9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55948/0 (socket says 172.21.15.31:55948) 2023-10-06T05:25:08.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.388+0000 7f3ef4b9b700 1 -- 172.21.15.31:0/1365316134 learned_addr learned my addr 172.21.15.31:0/1365316134 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:08.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3ef539c700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00ccc80 0x7f3ef00cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087743106 0 0) 0x7f3ef00cf280 con 0x7f3ef0159580 2023-10-06T05:25:08.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee0003570 con 0x7f3ef0159580 2023-10-06T05:25:08.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017361899 0 0) 0x7f3ef00cfe20 con 0x7f3ef00cdfe0 2023-10-06T05:25:08.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef00cf280 con 0x7f3ef00cdfe0 2023-10-06T05:25:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 499890255 0 0) 0x7f3ee0003570 con 0x7f3ef0159580 2023-10-06T05:25:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00ccc80 msgr2=0x7f3ef00cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00ccc80 0x7f3ef00cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 >> v1:172.21.15.62:6789/0 conn(0x7f3ef00cdfe0 legacy=0x7f3ef00cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00ce670 con 0x7f3ef0159580 2023-10-06T05:25:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f3ee40015f0 con 0x7f3ef0159580 2023-10-06T05:25:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 >> v1:172.21.15.31:6789/0 conn(0x7f3ef0159580 legacy=0x7f3ef00cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00ccc80 0x7f3ee0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ee0004180 0x7f3ee0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee0004d50 0x7f3ee00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee000bb70 con 0x7f3ee0004180 2023-10-06T05:25:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee000bce0 con 0x7f3ee0004d50 2023-10-06T05:25:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.389+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee000bea0 con 0x7f3ef00ccc80 2023-10-06T05:25:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.390+0000 7f3eeffff700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ee0004180 0x7f3ee0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.390+0000 7f3ef4b9b700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00ccc80 0x7f3ee0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.390+0000 7f3ef539c700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee0004d50 0x7f3ee00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.390+0000 7f3eeffff700 1 -- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00ccc80 msgr2=0x7f3ee0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.390+0000 7f3eeffff700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00ccc80 0x7f3ee0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.390+0000 7f3eeffff700 1 -- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee0004d50 msgr2=0x7f3ee00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.390+0000 7f3eeffff700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee0004d50 0x7f3ee00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.390+0000 7f3eeffff700 1 -- 172.21.15.31:0/1365316134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0003570 con 0x7f3ee0004180 2023-10-06T05:25:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.390+0000 7f3eeffff700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ee0004180 0x7f3ee0004810 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f3edc001a10 tx=0x7f3edc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffed53386c34d72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.391+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00c5d0 con 0x7f3ee0004180 2023-10-06T05:25:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.391+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc00c730 con 0x7f3ee0004180 2023-10-06T05:25:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.391+0000 7f3eef7fe700 1 -- 172.21.15.31:0/1365316134 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00cad0 con 0x7f3ee0004180 2023-10-06T05:25:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.391+0000 7f3ef6dff700 1 -- 172.21.15.31:0/1365316134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ee0004180 msgr2=0x7f3ee0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.391+0000 7f3ef6dff700 1 --2- 172.21.15.31:0/1365316134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ee0004180 0x7f3ee0004810 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f3edc001a10 tx=0x7f3edc007d30 comp rx=0 tx=0).stop 2023-10-06T05:25:08.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.391+0000 7f3ef539c700 1 -- 172.21.15.31:0/1365316134 reap_dead start 2023-10-06T05:25:08.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.391+0000 7f3ef6dff700 1 -- 172.21.15.31:0/1365316134 shutdown_connections 2023-10-06T05:25:08.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.391+0000 7f3ef6dff700 1 -- 172.21.15.31:0/1365316134 >> 172.21.15.31:0/1365316134 conn(0x7f3ef0099710 msgr2=0x7f3ef006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.391+0000 7f3ef6dff700 1 -- 172.21.15.31:0/1365316134 shutdown_connections 2023-10-06T05:25:08.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.391+0000 7f3ef6dff700 1 -- 172.21.15.31:0/1365316134 wait complete. 2023-10-06T05:25:08.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.393+0000 7f3ef6dff700 1 Processor -- start 2023-10-06T05:25:08.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.394+0000 7f3ef6dff700 1 -- start start 2023-10-06T05:25:08.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.395+0000 7f3ef6dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ee0004180 0x7f3ef00c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.395+0000 7f3ef6dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef00c97c0 0x7f3ef015c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.395+0000 7f3ef6dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00c8490 0x7f3ef00c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.395+0000 7f3ef6dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00c6c90 con 0x7f3ef00c97c0 2023-10-06T05:25:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.395+0000 7f3ef6dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef015bc80 con 0x7f3ef00c8490 2023-10-06T05:25:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.395+0000 7f3ef6dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef015b670 con 0x7f3ee0004180 2023-10-06T05:25:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.395+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef00c97c0 0x7f3ef015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.395+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef00c97c0 0x7f3ef015c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37938/0 (socket says 172.21.15.31:37938) 2023-10-06T05:25:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.395+0000 7f3eeffff700 1 -- 172.21.15.31:0/139650335 learned_addr learned my addr 172.21.15.31:0/139650335 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3eeffff700 1 -- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ee0004180 msgr2=0x7f3ef00c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3eeffff700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ee0004180 0x7f3ef00c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3eeffff700 1 -- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00c8490 msgr2=0x7f3ef00c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3eeffff700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00c8490 0x7f3ef00c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3eeffff700 1 -- 172.21.15.31:0/139650335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef016d590 con 0x7f3ef00c97c0 2023-10-06T05:25:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3eeffff700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef00c97c0 0x7f3ef015c110 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000bcb0 tx=0x7f3edc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3eecff9700 1 -- 172.21.15.31:0/139650335 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00c3f0 con 0x7f3ef00c97c0 2023-10-06T05:25:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3eecff9700 1 -- 172.21.15.31:0/139650335 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc001360 con 0x7f3ef00c97c0 2023-10-06T05:25:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3eecff9700 1 -- 172.21.15.31:0/139650335 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00cad0 con 0x7f3ef00c97c0 2023-10-06T05:25:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3ef6dff700 1 -- 172.21.15.31:0/139650335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef016d870 con 0x7f3ef00c97c0 2023-10-06T05:25:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.396+0000 7f3ef6dff700 1 -- 172.21.15.31:0/139650335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef00c8e20 con 0x7f3ef00c97c0 2023-10-06T05:25:08.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.397+0000 7f3ef6dff700 1 -- 172.21.15.31:0/139650335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef015bf60 con 0x7f3ef00c97c0 2023-10-06T05:25:08.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.398+0000 7f3eecff9700 1 -- 172.21.15.31:0/139650335 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f3edc019070 con 0x7f3ef00c97c0 2023-10-06T05:25:08.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.399+0000 7f3eecff9700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ed4067680 0x7f3ed4069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.399+0000 7f3eecff9700 1 -- 172.21.15.31:0/139650335 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f3edc0870f0 con 0x7f3ef00c97c0 2023-10-06T05:25:08.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.400+0000 7f3ef4b9b700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ed4067680 0x7f3ed4069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.400+0000 7f3ef4b9b700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ed4067680 0x7f3ed4069b30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4000e10 tx=0x7f3ee4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.404+0000 7f3eecff9700 1 -- 172.21.15.31:0/139650335 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3edc0844b0 con 0x7f3ef00c97c0 2023-10-06T05:25:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.606+0000 7f3ef6dff700 1 -- 172.21.15.31:0/139650335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ef004c380 con 0x7f3ef00c97c0 2023-10-06T05:25:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.607+0000 7f3eecff9700 1 -- 172.21.15.31:0/139650335 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16972 (secure 0 0 0) 0x7f3edc0532a0 con 0x7f3ef00c97c0 2023-10-06T05:25:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:08.609 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":139,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:48.323184+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:44.887621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:45.794431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 -- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ed4067680 msgr2=0x7f3ed4069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ed4067680 0x7f3ed4069b30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4000e10 tx=0x7f3ee4009040 comp rx=0 tx=0).stop 2023-10-06T05:25:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 -- 172.21.15.31:0/139650335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef00c97c0 msgr2=0x7f3ef015c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef00c97c0 0x7f3ef015c110 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000bcb0 tx=0x7f3edc015000 comp rx=0 tx=0).stop 2023-10-06T05:25:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 -- 172.21.15.31:0/139650335 shutdown_connections 2023-10-06T05:25:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3ed4067680 0x7f3ed4069b30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00c8490 0x7f3ef00c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ee0004180 0x7f3ef00c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 --2- 172.21.15.31:0/139650335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ef00c97c0 0x7f3ef015c110 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 -- 172.21.15.31:0/139650335 >> 172.21.15.31:0/139650335 conn(0x7f3ef0099710 msgr2=0x7f3ef00b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 -- 172.21.15.31:0/139650335 shutdown_connections 2023-10-06T05:25:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.609+0000 7f3ef6dff700 1 -- 172.21.15.31:0/139650335 wait complete. 2023-10-06T05:25:08.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:24:44.887621+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:24:45.794431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:25:08.629 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:25:08.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.825+0000 7f5c8e7ff700 1 Processor -- start 2023-10-06T05:25:08.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.826+0000 7f5c8e7ff700 1 -- start start 2023-10-06T05:25:08.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.827+0000 7f5c8e7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c880aef00 0x7f5c880c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.827+0000 7f5c8e7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c88068250 con 0x7f5c880c7560 2023-10-06T05:25:08.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.827+0000 7f5c8e7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8806ac80 con 0x7f5c880c7190 2023-10-06T05:25:08.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.827+0000 7f5c8e7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c88067f20 con 0x7f5c880aef00 2023-10-06T05:25:08.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.827+0000 7f5c877fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c880aef00 0x7f5c880c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.827+0000 7f5c877fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c880aef00 0x7f5c880c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37954/0 (socket says 172.21.15.31:37954) 2023-10-06T05:25:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.827+0000 7f5c877fe700 1 -- 172.21.15.31:0/1643117034 learned_addr learned my addr 172.21.15.31:0/1643117034 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.827+0000 7f5c8cd9c700 1 --1- 172.21.15.31:0/1643117034 >> v1:172.21.15.31:6789/0 conn(0x7f5c880c7560 0x7f5c880686e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55964/0 (socket says 172.21.15.31:55964) 2023-10-06T05:25:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.828+0000 7f5c877fe700 1 -- 172.21.15.31:0/1643117034 >> v1:172.21.15.62:6789/0 conn(0x7f5c880c7190 legacy=0x7f5c880af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.828+0000 7f5c877fe700 1 -- 172.21.15.31:0/1643117034 >> v1:172.21.15.31:6789/0 conn(0x7f5c880c7560 legacy=0x7f5c880686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.828+0000 7f5c877fe700 1 -- 172.21.15.31:0/1643117034 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8806aed0 con 0x7f5c880aef00 2023-10-06T05:25:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.828+0000 7f5c877fe700 1 --2- 172.21.15.31:0/1643117034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880aef00 0x7f5c880c9d40 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0099b0 tx=0x7f5c7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c441d72e701c00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.828+0000 7f5c86ffd700 1 -- 172.21.15.31:0/1643117034 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c00b860 con 0x7f5c880aef00 2023-10-06T05:25:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.828+0000 7f5c86ffd700 1 -- 172.21.15.31:0/1643117034 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c00b9c0 con 0x7f5c880aef00 2023-10-06T05:25:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.828+0000 7f5c86ffd700 1 -- 172.21.15.31:0/1643117034 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c00bd60 con 0x7f5c880aef00 2023-10-06T05:25:08.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.829+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/1643117034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880aef00 msgr2=0x7f5c880c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.829+0000 7f5c8e7ff700 1 --2- 172.21.15.31:0/1643117034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880aef00 0x7f5c880c9d40 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0099b0 tx=0x7f5c7c009250 comp rx=0 tx=0).stop 2023-10-06T05:25:08.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.829+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/1643117034 shutdown_connections 2023-10-06T05:25:08.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.829+0000 7f5c8e7ff700 1 --2- 172.21.15.31:0/1643117034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880aef00 0x7f5c880c9d40 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.829+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/1643117034 >> 172.21.15.31:0/1643117034 conn(0x7f5c880cd3d0 msgr2=0x7f5c880c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:08.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.829+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/1643117034 shutdown_connections 2023-10-06T05:25:08.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.829+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/1643117034 wait complete. 2023-10-06T05:25:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.830+0000 7f5c8e7ff700 1 Processor -- start 2023-10-06T05:25:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.830+0000 7f5c8e7ff700 1 -- start start 2023-10-06T05:25:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.831+0000 7f5c8e7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c880aef00 0x7f5c880c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.831+0000 7f5c8e7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c880c7190 0x7f5c8815f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.831+0000 7f5c8e7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880c7560 0x7f5c88157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.831+0000 7f5c8e7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815adb0 con 0x7f5c880c7560 2023-10-06T05:25:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.831+0000 7f5c8e7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c880c1480 con 0x7f5c880aef00 2023-10-06T05:25:08.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.831+0000 7f5c8e7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c880c1760 con 0x7f5c880c7190 2023-10-06T05:25:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c8cd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880c7560 0x7f5c88157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c8cd9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880c7560 0x7f5c88157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37966/0 (socket says 172.21.15.31:37966) 2023-10-06T05:25:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c8cd9c700 1 -- 172.21.15.31:0/2890696460 learned_addr learned my addr 172.21.15.31:0/2890696460 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c877fe700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c880c7190 0x7f5c8815f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c87fff700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c880aef00 0x7f5c880c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c8cd9c700 1 -- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c880c7190 msgr2=0x7f5c8815f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c8cd9c700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c880c7190 0x7f5c8815f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c8cd9c700 1 -- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c880aef00 msgr2=0x7f5c880c0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:08.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c8cd9c700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c880aef00 0x7f5c880c0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:08.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c8cd9c700 1 -- 172.21.15.31:0/2890696460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c0094f0 con 0x7f5c880c7560 2023-10-06T05:25:08.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c877fe700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c880c7190 0x7f5c8815f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:08.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.832+0000 7f5c8cd9c700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880c7560 0x7f5c88157c70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000cc70 tx=0x7f5c80001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.833+0000 7f5c6ffff700 1 -- 172.21.15.31:0/2890696460 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c80004bb0 con 0x7f5c880c7560 2023-10-06T05:25:08.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.833+0000 7f5c6ffff700 1 -- 172.21.15.31:0/2890696460 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c80004d10 con 0x7f5c880c7560 2023-10-06T05:25:08.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.833+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/2890696460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c880c1a40 con 0x7f5c880c7560 2023-10-06T05:25:08.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.833+0000 7f5c6ffff700 1 -- 172.21.15.31:0/2890696460 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8000ab90 con 0x7f5c880c7560 2023-10-06T05:25:08.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.833+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/2890696460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c880c1090 con 0x7f5c880c7560 2023-10-06T05:25:08.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.834+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/2890696460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8815af20 con 0x7f5c880c7560 2023-10-06T05:25:08.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.835+0000 7f5c6ffff700 1 -- 172.21.15.31:0/2890696460 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f5c800070f0 con 0x7f5c880c7560 2023-10-06T05:25:08.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.836+0000 7f5c6ffff700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5c74067680 0x7f5c74069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.836+0000 7f5c6ffff700 1 -- 172.21.15.31:0/2890696460 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f5c80085350 con 0x7f5c880c7560 2023-10-06T05:25:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.836+0000 7f5c87fff700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5c74067680 0x7f5c74069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.837+0000 7f5c87fff700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5c74067680 0x7f5c74069b30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5c78003860 tx=0x7f5c78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:08.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:08.840+0000 7f5c6ffff700 1 -- 172.21.15.31:0/2890696460 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c80082710 con 0x7f5c880c7560 2023-10-06T05:25:09.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.044+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/2890696460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c8804c380 con 0x7f5c880c7560 2023-10-06T05:25:09.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.044+0000 7f5c6ffff700 1 -- 172.21.15.31:0/2890696460 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v139) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5c80018020 con 0x7f5c880c7560 2023-10-06T05:25:09.046 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:25:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5c74067680 msgr2=0x7f5c74069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5c74067680 0x7f5c74069b30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5c78003860 tx=0x7f5c78009040 comp rx=0 tx=0).stop 2023-10-06T05:25:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/2890696460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880c7560 msgr2=0x7f5c88157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880c7560 0x7f5c88157c70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000cc70 tx=0x7f5c80001a70 comp rx=0 tx=0).stop 2023-10-06T05:25:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/2890696460 shutdown_connections 2023-10-06T05:25:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5c74067680 0x7f5c74069b30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c880c7560 0x7f5c88157c70 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c880aef00 0x7f5c880c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 --2- 172.21.15.31:0/2890696460 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5c880c7190 0x7f5c8815f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/2890696460 >> 172.21.15.31:0/2890696460 conn(0x7f5c880cd3d0 msgr2=0x7f5c881589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/2890696460 shutdown_connections 2023-10-06T05:25:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.047+0000 7f5c8e7ff700 1 -- 172.21.15.31:0/2890696460 wait complete. 2023-10-06T05:25:09.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:25:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.261+0000 7fee1e3ff700 1 Processor -- start 2023-10-06T05:25:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.261+0000 7fee1e3ff700 1 -- start start 2023-10-06T05:25:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.262+0000 7fee1e3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fee18159560 0x7fee180cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.263+0000 7fee1e3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1815d2d0 con 0x7fee180b2b80 2023-10-06T05:25:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.263+0000 7fee1e3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1815d5b0 con 0x7fee18159930 2023-10-06T05:25:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.263+0000 7fee1e3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1815d890 con 0x7fee18159560 2023-10-06T05:25:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.263+0000 7fee1c99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fee180b2b80 0x7fee180cb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55974/0 (socket says 172.21.15.31:55974) 2023-10-06T05:25:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.263+0000 7fee1c99c700 1 -- 172.21.15.31:0/3272345826 learned_addr learned my addr 172.21.15.31:0/3272345826 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.263+0000 7fee17fff700 1 --2- 172.21.15.31:0/3272345826 >> v2:172.21.15.62:3300/0 conn(0x7fee18159560 0x7fee180cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.263+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273153779 0 0) 0x7fee1815d2d0 con 0x7fee180b2b80 2023-10-06T05:25:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.263+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee00003570 con 0x7fee180b2b80 2023-10-06T05:25:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1805244312 0 0) 0x7fee00003570 con 0x7fee180b2b80 2023-10-06T05:25:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 >> v2:172.21.15.62:3300/0 conn(0x7fee18159560 msgr2=0x7fee180cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 --2- 172.21.15.31:0/3272345826 >> v2:172.21.15.62:3300/0 conn(0x7fee18159560 0x7fee180cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 >> v1:172.21.15.62:6789/0 conn(0x7fee18159930 legacy=0x7fee180b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1815db70 con 0x7fee180b2b80 2023-10-06T05:25:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fee100017f0 con 0x7fee180b2b80 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 >> v1:172.21.15.31:6789/0 conn(0x7fee180b2b80 legacy=0x7fee180cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee00003a60 0x7fee00003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee00004810 0x7fee00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee000051c0 0x7fee0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1815d890 con 0x7fee000051c0 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0000c130 con 0x7fee00003a60 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0000c2f0 con 0x7fee00004810 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee1c99c700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee000051c0 0x7fee0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee17fff700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee00003a60 0x7fee00003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.264+0000 7fee177fe700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee00004810 0x7fee00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.265+0000 7fee1c99c700 1 -- 172.21.15.31:0/3272345826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee00004810 msgr2=0x7fee00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.265+0000 7fee1c99c700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee00004810 0x7fee00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.265+0000 7fee1c99c700 1 -- 172.21.15.31:0/3272345826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee00003a60 msgr2=0x7fee00003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.265+0000 7fee1c99c700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee00003a60 0x7fee00003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.265+0000 7fee1c99c700 1 -- 172.21.15.31:0/3272345826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee00003570 con 0x7fee000051c0 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.265+0000 7fee1c99c700 1 -- 172.21.15.31:0/3272345826 reap_dead start 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.265+0000 7fee1c99c700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee000051c0 0x7fee0000ba60 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fee1000a850 tx=0x7fee10002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1717fdb1f3c1c4cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.265+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1000a210 con 0x7fee000051c0 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.265+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee100070f0 con 0x7fee000051c0 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.265+0000 7fee16ffd700 1 -- 172.21.15.31:0/3272345826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10007490 con 0x7fee000051c0 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.266+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3272345826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee000051c0 msgr2=0x7fee0000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.266+0000 7fee1e3ff700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee000051c0 0x7fee0000ba60 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fee1000a850 tx=0x7fee10002120 comp rx=0 tx=0).stop 2023-10-06T05:25:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.266+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3272345826 shutdown_connections 2023-10-06T05:25:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.266+0000 7fee1e3ff700 1 --2- 172.21.15.31:0/3272345826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee000051c0 0x7fee0000ba60 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.266+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3272345826 >> 172.21.15.31:0/3272345826 conn(0x7fee1806ec90 msgr2=0x7fee1806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.266+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3272345826 shutdown_connections 2023-10-06T05:25:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.266+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3272345826 wait complete. 2023-10-06T05:25:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.268+0000 7fee1e3ff700 1 Processor -- start 2023-10-06T05:25:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.268+0000 7fee1e3ff700 1 -- start start 2023-10-06T05:25:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1e3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee000051c0 0x7fee1816a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1e3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee180c74a0 0x7fee180c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1e3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee180c93e0 0x7fee180c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1e3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee180c50b0 con 0x7fee180c93e0 2023-10-06T05:25:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1e3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee180c7e50 con 0x7fee180c74a0 2023-10-06T05:25:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1e3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee180caab0 con 0x7fee000051c0 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee180c93e0 0x7fee180c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee180c93e0 0x7fee180c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37984/0 (socket says 172.21.15.31:37984) 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1c99c700 1 -- 172.21.15.31:0/3457078514 learned_addr learned my addr 172.21.15.31:0/3457078514 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee17fff700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee000051c0 0x7fee1816a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee177fe700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee180c74a0 0x7fee180c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1c99c700 1 -- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee000051c0 msgr2=0x7fee1816a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1c99c700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee000051c0 0x7fee1816a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1c99c700 1 -- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee180c74a0 msgr2=0x7fee180c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1c99c700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee180c74a0 0x7fee180c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.269+0000 7fee1c99c700 1 -- 172.21.15.31:0/3457078514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee180ca210 con 0x7fee180c93e0 2023-10-06T05:25:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.270+0000 7fee1c99c700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee180c93e0 0x7fee180c9840 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fee1806fbc0 tx=0x7fee10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.270+0000 7fedfffff700 1 -- 172.21.15.31:0/3457078514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10007a10 con 0x7fee180c93e0 2023-10-06T05:25:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.270+0000 7fedfffff700 1 -- 172.21.15.31:0/3457078514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1001e430 con 0x7fee180c93e0 2023-10-06T05:25:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.270+0000 7fedfffff700 1 -- 172.21.15.31:0/3457078514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1001e7f0 con 0x7fee180c93e0 2023-10-06T05:25:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.270+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3457078514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee180ca4f0 con 0x7fee180c93e0 2023-10-06T05:25:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.270+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3457078514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1816a670 con 0x7fee180c93e0 2023-10-06T05:25:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.271+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3457078514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee180c51f0 con 0x7fee180c93e0 2023-10-06T05:25:09.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.272+0000 7fedfffff700 1 -- 172.21.15.31:0/3457078514 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fee1001eb60 con 0x7fee180c93e0 2023-10-06T05:25:09.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.273+0000 7fedfffff700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee04067680 0x7fee04069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.274+0000 7fedfffff700 1 -- 172.21.15.31:0/3457078514 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fee10087070 con 0x7fee180c93e0 2023-10-06T05:25:09.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.274+0000 7fee17fff700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee04067680 0x7fee04069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.275+0000 7fee17fff700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee04067680 0x7fee04069b30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fee180b44e0 tx=0x7fee080091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.278+0000 7fedfffff700 1 -- 172.21.15.31:0/3457078514 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee180c51f0 con 0x7fee180c93e0 2023-10-06T05:25:09.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.482+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3457078514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee180c51f0 con 0x7fee180c93e0 2023-10-06T05:25:09.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.483+0000 7fedfffff700 1 -- 172.21.15.31:0/3457078514 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee180c51f0 con 0x7fee180c93e0 2023-10-06T05:25:09.484 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:25:09.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee04067680 msgr2=0x7fee04069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee04067680 0x7fee04069b30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fee180b44e0 tx=0x7fee080091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3457078514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee180c93e0 msgr2=0x7fee180c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee180c93e0 0x7fee180c9840 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fee1806fbc0 tx=0x7fee10016000 comp rx=0 tx=0).stop 2023-10-06T05:25:09.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3457078514 shutdown_connections 2023-10-06T05:25:09.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fee04067680 0x7fee04069b30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee180c93e0 0x7fee180c9840 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee000051c0 0x7fee1816a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 --2- 172.21.15.31:0/3457078514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee180c74a0 0x7fee180c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3457078514 >> 172.21.15.31:0/3457078514 conn(0x7fee1806ec90 msgr2=0x7fee180cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3457078514 shutdown_connections 2023-10-06T05:25:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.485+0000 7fee1e3ff700 1 -- 172.21.15.31:0/3457078514 wait complete. 2023-10-06T05:25:09.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:25:09.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.701+0000 7f940b3ff700 1 Processor -- start 2023-10-06T05:25:09.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.701+0000 7f940b3ff700 1 -- start start 2023-10-06T05:25:09.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.702+0000 7f940b3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9404159930 0x7f94040b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.702+0000 7f940b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94040cd580 0x7f94040b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.702+0000 7f940b3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f940415d2d0 con 0x7f9404159560 2023-10-06T05:25:09.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.702+0000 7f940b3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940415d5b0 con 0x7f9404159930 2023-10-06T05:25:09.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.702+0000 7f940b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940415d890 con 0x7f94040cd580 2023-10-06T05:25:09.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.702+0000 7f940919b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9404159560 0x7f94040cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55986/0 (socket says 172.21.15.31:55986) 2023-10-06T05:25:09.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.702+0000 7f940919b700 1 -- 172.21.15.31:0/2649820130 learned_addr learned my addr 172.21.15.31:0/2649820130 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:09.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.703+0000 7f940999c700 1 --2- 172.21.15.31:0/2649820130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94040cd580 0x7f94040b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.703+0000 7f940899a700 1 --2- 172.21.15.31:0/2649820130 >> v2:172.21.15.62:3300/0 conn(0x7f9404159930 0x7f94040b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.703+0000 7f93fbfff700 1 -- 172.21.15.31:0/2649820130 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633922220 0 0) 0x7f940415d2d0 con 0x7f9404159560 2023-10-06T05:25:09.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.703+0000 7f93fbfff700 1 -- 172.21.15.31:0/2649820130 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93f0003570 con 0x7f9404159560 2023-10-06T05:25:09.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.703+0000 7f940899a700 1 -- 172.21.15.31:0/2649820130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94040cd580 msgr2=0x7f94040b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.703+0000 7f940899a700 1 --2- 172.21.15.31:0/2649820130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94040cd580 0x7f94040b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.703+0000 7f940899a700 1 -- 172.21.15.31:0/2649820130 >> v1:172.21.15.31:6789/0 conn(0x7f9404159560 legacy=0x7f94040cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.703+0000 7f93fbfff700 1 -- 172.21.15.31:0/2649820130 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4254505255 0 0) 0x7f93f0003570 con 0x7f9404159560 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.703+0000 7f940899a700 1 -- 172.21.15.31:0/2649820130 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940415db70 con 0x7f9404159930 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.704+0000 7f940899a700 1 --2- 172.21.15.31:0/2649820130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9404159930 0x7f94040b2470 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f93f40098e0 tx=0x7f93f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0790dfd752985fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.704+0000 7f93fbfff700 1 -- 172.21.15.31:0/2649820130 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400001470 con 0x7f9404159930 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.704+0000 7f93fbfff700 1 -- 172.21.15.31:0/2649820130 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f4011420 con 0x7f9404159930 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.704+0000 7f93fbfff700 1 -- 172.21.15.31:0/2649820130 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40117c0 con 0x7f9404159930 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.704+0000 7f940b3ff700 1 -- 172.21.15.31:0/2649820130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9404159930 msgr2=0x7f94040b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.704+0000 7f940b3ff700 1 --2- 172.21.15.31:0/2649820130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9404159930 0x7f94040b2470 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f93f40098e0 tx=0x7f93f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.704+0000 7f940b3ff700 1 -- 172.21.15.31:0/2649820130 shutdown_connections 2023-10-06T05:25:09.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.704+0000 7f940b3ff700 1 --2- 172.21.15.31:0/2649820130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94040cd580 0x7f94040b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.704+0000 7f940b3ff700 1 --2- 172.21.15.31:0/2649820130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9404159930 0x7f94040b2470 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.704+0000 7f940b3ff700 1 -- 172.21.15.31:0/2649820130 >> 172.21.15.31:0/2649820130 conn(0x7f940406ec90 msgr2=0x7f940406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:09.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.705+0000 7f940b3ff700 1 -- 172.21.15.31:0/2649820130 shutdown_connections 2023-10-06T05:25:09.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.705+0000 7f940b3ff700 1 -- 172.21.15.31:0/2649820130 wait complete. 2023-10-06T05:25:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.706+0000 7f940b3ff700 1 Processor -- start 2023-10-06T05:25:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.706+0000 7f940b3ff700 1 -- start start 2023-10-06T05:25:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.707+0000 7f940b3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94040cd580 0x7f9404144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.707+0000 7f940b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9404159560 0x7f94040d0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.707+0000 7f940b3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9404159930 0x7f94040d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.707+0000 7f940b3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9404144a80 con 0x7f9404159930 2023-10-06T05:25:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.707+0000 7f940b3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94040c4ff0 con 0x7f94040cd580 2023-10-06T05:25:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.707+0000 7f940b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94040c52d0 con 0x7f9404159560 2023-10-06T05:25:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.707+0000 7f940999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9404159930 0x7f94040d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.707+0000 7f940999c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9404159930 0x7f94040d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37990/0 (socket says 172.21.15.31:37990) 2023-10-06T05:25:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.707+0000 7f940999c700 1 -- 172.21.15.31:0/1618032294 learned_addr learned my addr 172.21.15.31:0/1618032294 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.708+0000 7f940919b700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94040cd580 0x7f9404144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.708+0000 7f940899a700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9404159560 0x7f94040d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.708+0000 7f940999c700 1 -- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9404159560 msgr2=0x7f94040d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.708+0000 7f940999c700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9404159560 0x7f94040d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.708+0000 7f940999c700 1 -- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94040cd580 msgr2=0x7f9404144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.708+0000 7f940999c700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94040cd580 0x7f9404144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.708+0000 7f940999c700 1 -- 172.21.15.31:0/1618032294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4009420 con 0x7f9404159930 2023-10-06T05:25:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.708+0000 7f940999c700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9404159930 0x7f94040d0bc0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009ef0 tx=0x7f93fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.709+0000 7f93f97fa700 1 -- 172.21.15.31:0/1618032294 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc00cc20 con 0x7f9404159930 2023-10-06T05:25:09.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.709+0000 7f93f97fa700 1 -- 172.21.15.31:0/1618032294 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93fc00f040 con 0x7f9404159930 2023-10-06T05:25:09.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.709+0000 7f93f97fa700 1 -- 172.21.15.31:0/1618032294 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc014480 con 0x7f9404159930 2023-10-06T05:25:09.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.709+0000 7f940b3ff700 1 -- 172.21.15.31:0/1618032294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94040c55b0 con 0x7f9404159930 2023-10-06T05:25:09.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.709+0000 7f940b3ff700 1 -- 172.21.15.31:0/1618032294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94040c4280 con 0x7f9404159930 2023-10-06T05:25:09.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.710+0000 7f940b3ff700 1 -- 172.21.15.31:0/1618032294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940404c380 con 0x7f9404159930 2023-10-06T05:25:09.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.711+0000 7f93f97fa700 1 -- 172.21.15.31:0/1618032294 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f93fc014830 con 0x7f9404159930 2023-10-06T05:25:09.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.712+0000 7f93f97fa700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f93e8067750 0x7f93e8069c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:09.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.712+0000 7f93f97fa700 1 -- 172.21.15.31:0/1618032294 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f93fc086210 con 0x7f9404159930 2023-10-06T05:25:09.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.712+0000 7f940919b700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f93e8067750 0x7f93e8069c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:09.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.713+0000 7f940919b700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f93e8067750 0x7f93e8069c00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f94040d0540 tx=0x7f94000019d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.716+0000 7f93f97fa700 1 -- 172.21.15.31:0/1618032294 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93fc0838d0 con 0x7f9404159930 2023-10-06T05:25:09.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.920+0000 7f940b3ff700 1 -- 172.21.15.31:0/1618032294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f940415de50 con 0x7f9404159930 2023-10-06T05:25:09.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.921+0000 7f93f97fa700 1 -- 172.21.15.31:0/1618032294 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93fc052740 con 0x7f9404159930 2023-10-06T05:25:09.923 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:25:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 -- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f93e8067750 msgr2=0x7f93e8069c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f93e8067750 0x7f93e8069c00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f94040d0540 tx=0x7f94000019d0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 -- 172.21.15.31:0/1618032294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9404159930 msgr2=0x7f94040d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9404159930 0x7f94040d0bc0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009ef0 tx=0x7f93fc00b040 comp rx=0 tx=0).stop 2023-10-06T05:25:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 -- 172.21.15.31:0/1618032294 shutdown_connections 2023-10-06T05:25:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f93e8067750 0x7f93e8069c00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9404159930 0x7f94040d0bc0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94040cd580 0x7f9404144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 --2- 172.21.15.31:0/1618032294 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9404159560 0x7f94040d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:09.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 -- 172.21.15.31:0/1618032294 >> 172.21.15.31:0/1618032294 conn(0x7f940406ec90 msgr2=0x7f94040b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:09.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 -- 172.21.15.31:0/1618032294 shutdown_connections 2023-10-06T05:25:09.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:09.923+0000 7f940b3ff700 1 -- 172.21.15.31:0/1618032294 wait complete. 2023-10-06T05:25:09.942 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:25:10.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.139+0000 7f804e3ff700 1 Processor -- start 2023-10-06T05:25:10.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.139+0000 7f804e3ff700 1 -- start start 2023-10-06T05:25:10.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.141+0000 7f804e3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80480c52d0 0x7f80480cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.141+0000 7f804e3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f80480cf110 0x7f80480cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.141+0000 7f804e3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80480cb800 con 0x7f80480af590 2023-10-06T05:25:10.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.141+0000 7f804e3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80480cf5b0 con 0x7f80480cf110 2023-10-06T05:25:10.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.141+0000 7f804e3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80480cb280 con 0x7f80480c52d0 2023-10-06T05:25:10.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.142+0000 7f8047fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f80480af590 0x7f80480c6160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56000/0 (socket says 172.21.15.31:56000) 2023-10-06T05:25:10.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.142+0000 7f8047fff700 1 -- 172.21.15.31:0/1278255624 learned_addr learned my addr 172.21.15.31:0/1278255624 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:10.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.142+0000 7f804c99c700 1 --2- 172.21.15.31:0/1278255624 >> v2:172.21.15.62:3300/0 conn(0x7f80480cf110 0x7f80480cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.142+0000 7f80477fe700 1 --2- 172.21.15.31:0/1278255624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80480c52d0 0x7f80480cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.142+0000 7f804c99c700 1 -- 172.21.15.31:0/1278255624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80480c52d0 msgr2=0x7f80480cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:10.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.142+0000 7f804c99c700 1 --2- 172.21.15.31:0/1278255624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80480c52d0 0x7f80480cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.142+0000 7f804c99c700 1 -- 172.21.15.31:0/1278255624 >> v1:172.21.15.31:6789/0 conn(0x7f80480af590 legacy=0x7f80480c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.142+0000 7f804c99c700 1 -- 172.21.15.31:0/1278255624 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80480c6870 con 0x7f80480cf110 2023-10-06T05:25:10.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.143+0000 7f804c99c700 1 --2- 172.21.15.31:0/1278255624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80480cf110 0x7f80480cbc90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8040009e60 tx=0x7f8040009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e3aac8fe3481d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.145+0000 7f8046ffd700 1 -- 172.21.15.31:0/1278255624 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804000e070 con 0x7f80480cf110 2023-10-06T05:25:10.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.145+0000 7f8046ffd700 1 -- 172.21.15.31:0/1278255624 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804000bb60 con 0x7f80480cf110 2023-10-06T05:25:10.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.145+0000 7f8046ffd700 1 -- 172.21.15.31:0/1278255624 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040012430 con 0x7f80480cf110 2023-10-06T05:25:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.146+0000 7f804e3ff700 1 -- 172.21.15.31:0/1278255624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80480cf110 msgr2=0x7f80480cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.146+0000 7f804e3ff700 1 --2- 172.21.15.31:0/1278255624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80480cf110 0x7f80480cbc90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8040009e60 tx=0x7f8040009780 comp rx=0 tx=0).stop 2023-10-06T05:25:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.146+0000 7f804e3ff700 1 -- 172.21.15.31:0/1278255624 shutdown_connections 2023-10-06T05:25:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.146+0000 7f804e3ff700 1 --2- 172.21.15.31:0/1278255624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80480cf110 0x7f80480cbc90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.146+0000 7f804e3ff700 1 --2- 172.21.15.31:0/1278255624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80480c52d0 0x7f80480cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.146+0000 7f804e3ff700 1 -- 172.21.15.31:0/1278255624 >> 172.21.15.31:0/1278255624 conn(0x7f80480996f0 msgr2=0x7f804806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.146+0000 7f804e3ff700 1 -- 172.21.15.31:0/1278255624 shutdown_connections 2023-10-06T05:25:10.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.146+0000 7f804e3ff700 1 -- 172.21.15.31:0/1278255624 wait complete. 2023-10-06T05:25:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.147+0000 7f804e3ff700 1 Processor -- start 2023-10-06T05:25:10.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.148+0000 7f804e3ff700 1 -- start start 2023-10-06T05:25:10.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804e3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80480af590 0x7f80480b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804e3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80480c52d0 0x7f80480c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804e3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80480cf110 0x7f80480c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804e3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80480b6060 con 0x7f80480cf110 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804e3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80480b0200 con 0x7f80480af590 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804e3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80480b5aa0 con 0x7f80480c52d0 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80480cf110 0x7f80480c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f8047fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80480af590 0x7f80480b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f80477fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80480c52d0 0x7f80480c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f8047fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80480af590 0x7f80480b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:59766/0 (socket says 172.21.15.31:59766) 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f8047fff700 1 -- 172.21.15.31:0/3250302300 learned_addr learned my addr 172.21.15.31:0/3250302300 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804c99c700 1 -- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80480c52d0 msgr2=0x7f80480c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804c99c700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80480c52d0 0x7f80480c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804c99c700 1 -- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80480af590 msgr2=0x7f80480b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.149+0000 7f804c99c700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80480af590 0x7f80480b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.150+0000 7f804c99c700 1 -- 172.21.15.31:0/3250302300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040009a20 con 0x7f80480cf110 2023-10-06T05:25:10.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.150+0000 7f804c99c700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80480cf110 0x7f80480c60a0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8040016000 tx=0x7f8040019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.150+0000 7f802ffff700 1 -- 172.21.15.31:0/3250302300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040012ea0 con 0x7f80480cf110 2023-10-06T05:25:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.150+0000 7f802ffff700 1 -- 172.21.15.31:0/3250302300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8040027470 con 0x7f80480cf110 2023-10-06T05:25:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.150+0000 7f802ffff700 1 -- 172.21.15.31:0/3250302300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040027850 con 0x7f80480cf110 2023-10-06T05:25:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.150+0000 7f804e3ff700 1 -- 172.21.15.31:0/3250302300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80480b5d80 con 0x7f80480cf110 2023-10-06T05:25:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.150+0000 7f804e3ff700 1 -- 172.21.15.31:0/3250302300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80481508f0 con 0x7f80480cf110 2023-10-06T05:25:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.151+0000 7f804e3ff700 1 -- 172.21.15.31:0/3250302300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80480b0370 con 0x7f80480cf110 2023-10-06T05:25:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.152+0000 7f802ffff700 1 -- 172.21.15.31:0/3250302300 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f8040024070 con 0x7f80480cf110 2023-10-06T05:25:10.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.153+0000 7f802ffff700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8034067680 0x7f8034069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.153+0000 7f802ffff700 1 -- 172.21.15.31:0/3250302300 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f8040087e00 con 0x7f80480cf110 2023-10-06T05:25:10.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.153+0000 7f8047fff700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8034067680 0x7f8034069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.154+0000 7f8047fff700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8034067680 0x7f8034069b30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8048063790 tx=0x7f8038009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.157+0000 7f802ffff700 1 -- 172.21.15.31:0/3250302300 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80400851c0 con 0x7f80480cf110 2023-10-06T05:25:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.364+0000 7f804e3ff700 1 -- 172.21.15.31:0/3250302300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f804804c380 con 0x7f80480cf110 2023-10-06T05:25:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.364+0000 7f802ffff700 1 -- 172.21.15.31:0/3250302300 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7f804002c020 con 0x7f80480cf110 2023-10-06T05:25:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:25:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 -- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8034067680 msgr2=0x7f8034069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8034067680 0x7f8034069b30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8048063790 tx=0x7f8038009040 comp rx=0 tx=0).stop 2023-10-06T05:25:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 -- 172.21.15.31:0/3250302300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80480cf110 msgr2=0x7f80480c60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80480cf110 0x7f80480c60a0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8040016000 tx=0x7f8040019000 comp rx=0 tx=0).stop 2023-10-06T05:25:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 -- 172.21.15.31:0/3250302300 shutdown_connections 2023-10-06T05:25:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8034067680 0x7f8034069b30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80480cf110 0x7f80480c60a0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80480af590 0x7f80480b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 --2- 172.21.15.31:0/3250302300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80480c52d0 0x7f80480c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 -- 172.21.15.31:0/3250302300 >> 172.21.15.31:0/3250302300 conn(0x7f80480996f0 msgr2=0x7f80480cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 -- 172.21.15.31:0/3250302300 shutdown_connections 2023-10-06T05:25:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.367+0000 7f804e3ff700 1 -- 172.21.15.31:0/3250302300 wait complete. 2023-10-06T05:25:10.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.579+0000 7f09587ff700 1 Processor -- start 2023-10-06T05:25:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.579+0000 7f09587ff700 1 -- start start 2023-10-06T05:25:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f09587ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09501596d0 0x7f0950159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f09587ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c6f80 0x7f095006a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f09587ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f09500c16e0 0x7f09500c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f09587ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09500c20c0 con 0x7f09501596d0 2023-10-06T05:25:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f09587ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0950067f70 con 0x7f09500c16e0 2023-10-06T05:25:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f09587ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095006e040 con 0x7f09500c6f80 2023-10-06T05:25:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f095659b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09501596d0 0x7f0950159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f095659b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09501596d0 0x7f0950159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37996/0 (socket says 172.21.15.31:37996) 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f095659b700 1 -- 172.21.15.31:0/2663461775 learned_addr learned my addr 172.21.15.31:0/2663461775 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f0955d9a700 1 --2- 172.21.15.31:0/2663461775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c6f80 0x7f095006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.581+0000 7f0956d9c700 1 --2- 172.21.15.31:0/2663461775 >> v2:172.21.15.62:3300/0 conn(0x7f09500c16e0 0x7f09500c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f095659b700 1 -- 172.21.15.31:0/2663461775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c6f80 msgr2=0x7f095006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f095659b700 1 --2- 172.21.15.31:0/2663461775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c6f80 0x7f095006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f095659b700 1 -- 172.21.15.31:0/2663461775 >> v2:172.21.15.62:3300/0 conn(0x7f09500c16e0 msgr2=0x7f09500c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f095659b700 1 --2- 172.21.15.31:0/2663461775 >> v2:172.21.15.62:3300/0 conn(0x7f09500c16e0 0x7f09500c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f095659b700 1 -- 172.21.15.31:0/2663461775 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500681c0 con 0x7f09501596d0 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f0955d9a700 1 --2- 172.21.15.31:0/2663461775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c6f80 0x7f095006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f0956d9c700 1 --2- 172.21.15.31:0/2663461775 >> v2:172.21.15.62:3300/0 conn(0x7f09500c16e0 0x7f09500c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f095659b700 1 --2- 172.21.15.31:0/2663461775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09501596d0 0x7f0950159aa0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f09500c6f10 tx=0x7f094c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b62fed493050651 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f0955599700 1 -- 172.21.15.31:0/2663461775 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00b860 con 0x7f09501596d0 2023-10-06T05:25:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f0955599700 1 -- 172.21.15.31:0/2663461775 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c00b9c0 con 0x7f09501596d0 2023-10-06T05:25:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.582+0000 7f0955599700 1 -- 172.21.15.31:0/2663461775 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00bd60 con 0x7f09501596d0 2023-10-06T05:25:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.583+0000 7f09587ff700 1 -- 172.21.15.31:0/2663461775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09501596d0 msgr2=0x7f0950159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.583+0000 7f09587ff700 1 --2- 172.21.15.31:0/2663461775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09501596d0 0x7f0950159aa0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f09500c6f10 tx=0x7f094c009250 comp rx=0 tx=0).stop 2023-10-06T05:25:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.583+0000 7f09587ff700 1 -- 172.21.15.31:0/2663461775 shutdown_connections 2023-10-06T05:25:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.583+0000 7f09587ff700 1 --2- 172.21.15.31:0/2663461775 >> v2:172.21.15.62:3300/0 conn(0x7f09500c16e0 0x7f09500c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.583+0000 7f09587ff700 1 --2- 172.21.15.31:0/2663461775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09501596d0 0x7f0950159aa0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.583+0000 7f09587ff700 1 --2- 172.21.15.31:0/2663461775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c6f80 0x7f095006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.583+0000 7f09587ff700 1 -- 172.21.15.31:0/2663461775 >> 172.21.15.31:0/2663461775 conn(0x7f09500c9e80 msgr2=0x7f095006a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.583+0000 7f09587ff700 1 -- 172.21.15.31:0/2663461775 shutdown_connections 2023-10-06T05:25:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.583+0000 7f09587ff700 1 -- 172.21.15.31:0/2663461775 wait complete. 2023-10-06T05:25:10.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.584+0000 7f09587ff700 1 Processor -- start 2023-10-06T05:25:10.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.585+0000 7f09587ff700 1 -- start start 2023-10-06T05:25:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f09587ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c16e0 0x7f0950151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f09587ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09500c6f80 0x7f09500c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f09587ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09501596d0 0x7f09500c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f09587ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0950155a70 con 0x7f09500c6f80 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f09587ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500c6c10 con 0x7f09501596d0 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f09587ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0950151db0 con 0x7f09500c16e0 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f0955d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09500c6f80 0x7f09500c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f0955d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09500c6f80 0x7f09500c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38000/0 (socket says 172.21.15.31:38000) 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f0955d9a700 1 -- 172.21.15.31:0/4207399428 learned_addr learned my addr 172.21.15.31:0/4207399428 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f095659b700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c16e0 0x7f0950151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f0956d9c700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09501596d0 0x7f09500c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f0955d9a700 1 -- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c16e0 msgr2=0x7f0950151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f0955d9a700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c16e0 0x7f0950151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f0955d9a700 1 -- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09501596d0 msgr2=0x7f09500c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f0955d9a700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09501596d0 0x7f09500c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.586+0000 7f0955d9a700 1 -- 172.21.15.31:0/4207399428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c0094f0 con 0x7f09500c6f80 2023-10-06T05:25:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.587+0000 7f0955d9a700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09500c6f80 0x7f09500c6190 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f094000e850 tx=0x7f0940008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.587+0000 7f09467fc700 1 -- 172.21.15.31:0/4207399428 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940013410 con 0x7f09500c6f80 2023-10-06T05:25:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.587+0000 7f09467fc700 1 -- 172.21.15.31:0/4207399428 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0940013570 con 0x7f09500c6f80 2023-10-06T05:25:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.587+0000 7f09467fc700 1 -- 172.21.15.31:0/4207399428 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940013910 con 0x7f09500c6f80 2023-10-06T05:25:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.587+0000 7f09587ff700 1 -- 172.21.15.31:0/4207399428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09500b1590 con 0x7f09500c6f80 2023-10-06T05:25:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.587+0000 7f09587ff700 1 -- 172.21.15.31:0/4207399428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09500b1b50 con 0x7f09500c6f80 2023-10-06T05:25:10.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.588+0000 7f09587ff700 1 -- 172.21.15.31:0/4207399428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0950155c20 con 0x7f09500c6f80 2023-10-06T05:25:10.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.589+0000 7f09467fc700 1 -- 172.21.15.31:0/4207399428 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f0940013d00 con 0x7f09500c6f80 2023-10-06T05:25:10.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.590+0000 7f09467fc700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0934067680 0x7f0934069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:10.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.590+0000 7f09467fc700 1 -- 172.21.15.31:0/4207399428 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f0940085600 con 0x7f09500c6f80 2023-10-06T05:25:10.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.590+0000 7f095659b700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0934067680 0x7f0934069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:10.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.591+0000 7f095659b700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0934067680 0x7f0934069b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f09500c6f10 tx=0x7f094c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.594+0000 7f09467fc700 1 -- 172.21.15.31:0/4207399428 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0950155c20 con 0x7f09500c6f80 2023-10-06T05:25:10.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.836+0000 7f09587ff700 1 -- 172.21.15.31:0/4207399428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f095004c380 con 0x7f09500c6f80 2023-10-06T05:25:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.836+0000 7f09467fc700 1 -- 172.21.15.31:0/4207399428 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f09400828b0 con 0x7f09500c6f80 2023-10-06T05:25:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:10.838 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1352198494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1352198494},{"type":"v1","addr":"172.21.15.62:6835","nonce":1352198494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8194,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/243154980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":243154980},{"type":"v1","addr":"172.21.15.62:6834","nonce":243154980}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/240917476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":240917476},{"type":"v1","addr":"172.21.15.31:6834","nonce":240917476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:46.824751+0000","modified":"2023-10-06T05:24:50.335585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3829644591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3829644591},{"type":"v1","addr":"172.21.15.31:6835","nonce":3829644591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.838+0000 7f09587ff700 1 -- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0934067680 msgr2=0x7f0934069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.838+0000 7f09587ff700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0934067680 0x7f0934069b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f09500c6f10 tx=0x7f094c017000 comp rx=0 tx=0).stop 2023-10-06T05:25:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.838+0000 7f09587ff700 1 -- 172.21.15.31:0/4207399428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09500c6f80 msgr2=0x7f09500c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.838+0000 7f09587ff700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09500c6f80 0x7f09500c6190 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f094000e850 tx=0x7f0940008040 comp rx=0 tx=0).stop 2023-10-06T05:25:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.839+0000 7f09587ff700 1 -- 172.21.15.31:0/4207399428 shutdown_connections 2023-10-06T05:25:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.839+0000 7f09587ff700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0934067680 0x7f0934069b30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.839+0000 7f09587ff700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09501596d0 0x7f09500c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.839+0000 7f09587ff700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09500c16e0 0x7f0950151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.839+0000 7f09587ff700 1 --2- 172.21.15.31:0/4207399428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09500c6f80 0x7f09500c6190 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.839+0000 7f09587ff700 1 -- 172.21.15.31:0/4207399428 >> 172.21.15.31:0/4207399428 conn(0x7f09500c9e80 msgr2=0x7f0950067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.839+0000 7f09587ff700 1 -- 172.21.15.31:0/4207399428 shutdown_connections 2023-10-06T05:25:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:10.839+0000 7f09587ff700 1 -- 172.21.15.31:0/4207399428 wait complete. 2023-10-06T05:25:10.845 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 94 2023-10-06T05:25:10.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:11.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.050+0000 7fbc29bff700 1 Processor -- start 2023-10-06T05:25:11.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.051+0000 7fbc29bff700 1 -- start start 2023-10-06T05:25:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.052+0000 7fbc29bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2406a570 0x7fbc24162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.052+0000 7fbc29bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240c71b0 con 0x7fbc2413a1e0 2023-10-06T05:25:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.052+0000 7fbc29bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240c7490 con 0x7fbc2413a5b0 2023-10-06T05:25:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.052+0000 7fbc29bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c7770 con 0x7fbc2406a570 2023-10-06T05:25:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.052+0000 7fbc237fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbc2413a1e0 0x7fbc240c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56012/0 (socket says 172.21.15.31:56012) 2023-10-06T05:25:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.052+0000 7fbc237fe700 1 -- 172.21.15.31:0/2997178701 learned_addr learned my addr 172.21.15.31:0/2997178701 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:11.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc23fff700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2406a570 0x7fbc24162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592973608 0 0) 0x7fbc240c71b0 con 0x7fbc2413a1e0 2023-10-06T05:25:11.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc0c003570 con 0x7fbc2413a1e0 2023-10-06T05:25:11.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996136571 0 0) 0x7fbc0c003570 con 0x7fbc2413a1e0 2023-10-06T05:25:11.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2406a570 msgr2=0x7fbc24162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:11.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2406a570 0x7fbc24162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 >> v1:172.21.15.62:6789/0 conn(0x7fbc2413a5b0 legacy=0x7fbc2406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc240c7a50 con 0x7fbc2413a1e0 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fbc140015f0 con 0x7fbc2413a1e0 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 >> v1:172.21.15.31:6789/0 conn(0x7fbc2413a1e0 legacy=0x7fbc240c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c003a90 0x7fbc0c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2406a570 0x7fbc0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc0c004d50 0x7fbc0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c7770 con 0x7fbc0c003a90 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c00bb10 con 0x7fbc0c004d50 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c00bcd0 con 0x7fbc2406a570 2023-10-06T05:25:11.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.053+0000 7fbc237fe700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c003a90 0x7fbc0c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc23fff700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc0c004d50 0x7fbc0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc22ffd700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2406a570 0x7fbc0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc237fe700 1 -- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2406a570 msgr2=0x7fbc0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc237fe700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2406a570 0x7fbc0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc237fe700 1 -- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc0c004d50 msgr2=0x7fbc0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc237fe700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc0c004d50 0x7fbc0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc237fe700 1 -- 172.21.15.31:0/2997178701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c003570 con 0x7fbc0c003a90 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc22ffd700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2406a570 0x7fbc0c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc23fff700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc0c004d50 0x7fbc0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc237fe700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c003a90 0x7fbc0c003e60 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400c300 tx=0x7fbc14007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832c1aaa7c6746e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400cc30 con 0x7fbc0c003a90 2023-10-06T05:25:11.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1400cd90 con 0x7fbc0c003a90 2023-10-06T05:25:11.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.054+0000 7fbc227fc700 1 -- 172.21.15.31:0/2997178701 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14007320 con 0x7fbc0c003a90 2023-10-06T05:25:11.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.055+0000 7fbc29bff700 1 -- 172.21.15.31:0/2997178701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c003a90 msgr2=0x7fbc0c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.055+0000 7fbc29bff700 1 --2- 172.21.15.31:0/2997178701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c003a90 0x7fbc0c003e60 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400c300 tx=0x7fbc14007920 comp rx=0 tx=0).stop 2023-10-06T05:25:11.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.055+0000 7fbc23fff700 1 -- 172.21.15.31:0/2997178701 reap_dead start 2023-10-06T05:25:11.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.055+0000 7fbc29bff700 1 -- 172.21.15.31:0/2997178701 shutdown_connections 2023-10-06T05:25:11.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.055+0000 7fbc29bff700 1 -- 172.21.15.31:0/2997178701 >> 172.21.15.31:0/2997178701 conn(0x7fbc240cc0b0 msgr2=0x7fbc240cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:11.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.055+0000 7fbc29bff700 1 -- 172.21.15.31:0/2997178701 shutdown_connections 2023-10-06T05:25:11.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.055+0000 7fbc29bff700 1 -- 172.21.15.31:0/2997178701 wait complete. 2023-10-06T05:25:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.056+0000 7fbc29bff700 1 Processor -- start 2023-10-06T05:25:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.057+0000 7fbc29bff700 1 -- start start 2023-10-06T05:25:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.057+0000 7fbc29bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc0c003a90 0x7fbc24139c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc29bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc240c4510 0x7fbc240b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc29bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc240c4980 0x7fbc240b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc29bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c5b20 con 0x7fbc240c4510 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc29bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c4f10 con 0x7fbc240c4980 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc29bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c51f0 con 0x7fbc0c003a90 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc237fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc0c003a90 0x7fbc24139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc22ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc240c4510 0x7fbc240b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc237fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc0c003a90 0x7fbc24139c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57100/0 (socket says 172.21.15.31:57100) 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc240c4980 0x7fbc240b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc22ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc240c4510 0x7fbc240b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38016/0 (socket says 172.21.15.31:38016) 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc237fe700 1 -- 172.21.15.31:0/1623409380 learned_addr learned my addr 172.21.15.31:0/1623409380 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc22ffd700 1 -- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc0c003a90 msgr2=0x7fbc24139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc22ffd700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc0c003a90 0x7fbc24139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc22ffd700 1 -- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc240c4980 msgr2=0x7fbc240b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc22ffd700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc240c4980 0x7fbc240b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.058+0000 7fbc22ffd700 1 -- 172.21.15.31:0/1623409380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc240c54d0 con 0x7fbc240c4510 2023-10-06T05:25:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.059+0000 7fbc22ffd700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc240c4510 0x7fbc240b02f0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800e810 tx=0x7fbc18009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.059+0000 7fbc0b7fe700 1 -- 172.21.15.31:0/1623409380 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18010520 con 0x7fbc240c4510 2023-10-06T05:25:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.059+0000 7fbc0b7fe700 1 -- 172.21.15.31:0/1623409380 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc18010680 con 0x7fbc240c4510 2023-10-06T05:25:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.059+0000 7fbc0b7fe700 1 -- 172.21.15.31:0/1623409380 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18010a20 con 0x7fbc240c4510 2023-10-06T05:25:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.059+0000 7fbc29bff700 1 -- 172.21.15.31:0/1623409380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc240c57b0 con 0x7fbc240c4510 2023-10-06T05:25:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.059+0000 7fbc29bff700 1 -- 172.21.15.31:0/1623409380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc240b0fb0 con 0x7fbc240c4510 2023-10-06T05:25:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.060+0000 7fbc29bff700 1 -- 172.21.15.31:0/1623409380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2404c380 con 0x7fbc240c4510 2023-10-06T05:25:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.061+0000 7fbc0b7fe700 1 -- 172.21.15.31:0/1623409380 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fbc18014480 con 0x7fbc240c4510 2023-10-06T05:25:11.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.062+0000 7fbc0b7fe700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc10067680 0x7fbc10069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.062+0000 7fbc0b7fe700 1 -- 172.21.15.31:0/1623409380 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fbc18084d80 con 0x7fbc240c4510 2023-10-06T05:25:11.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.063+0000 7fbc237fe700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc10067680 0x7fbc10069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.063+0000 7fbc237fe700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc10067680 0x7fbc10069b30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbc14000ef0 tx=0x7fbc14015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.067+0000 7fbc0b7fe700 1 -- 172.21.15.31:0/1623409380 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc1804d5a0 con 0x7fbc240c4510 2023-10-06T05:25:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.269+0000 7fbc29bff700 1 -- 172.21.15.31:0/1623409380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc240c7d30 con 0x7fbc240c4510 2023-10-06T05:25:11.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.270+0000 7fbc0b7fe700 1 -- 172.21.15.31:0/1623409380 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16972 (secure 0 0 0) 0x7fbc18019020 con 0x7fbc240c4510 2023-10-06T05:25:11.272 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:11.273 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":139,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:48.323184+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:44.887621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:45.794431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.272+0000 7fbc29bff700 1 -- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc10067680 msgr2=0x7fbc10069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.272+0000 7fbc29bff700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc10067680 0x7fbc10069b30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbc14000ef0 tx=0x7fbc14015000 comp rx=0 tx=0).stop 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.272+0000 7fbc29bff700 1 -- 172.21.15.31:0/1623409380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc240c4510 msgr2=0x7fbc240b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.272+0000 7fbc29bff700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc240c4510 0x7fbc240b02f0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800e810 tx=0x7fbc18009040 comp rx=0 tx=0).stop 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.272+0000 7fbc29bff700 1 -- 172.21.15.31:0/1623409380 shutdown_connections 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.272+0000 7fbc29bff700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbc10067680 0x7fbc10069b30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.272+0000 7fbc29bff700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc240c4980 0x7fbc240b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.272+0000 7fbc29bff700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc0c003a90 0x7fbc24139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.272+0000 7fbc29bff700 1 --2- 172.21.15.31:0/1623409380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc240c4510 0x7fbc240b02f0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.272+0000 7fbc29bff700 1 -- 172.21.15.31:0/1623409380 >> 172.21.15.31:0/1623409380 conn(0x7fbc240cc0b0 msgr2=0x7fbc2406d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.273+0000 7fbc29bff700 1 -- 172.21.15.31:0/1623409380 shutdown_connections 2023-10-06T05:25:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.273+0000 7fbc29bff700 1 -- 172.21.15.31:0/1623409380 wait complete. 2023-10-06T05:25:11.293 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:25:11.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:11.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.490+0000 7fc1489ff700 1 Processor -- start 2023-10-06T05:25:11.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.490+0000 7fc1489ff700 1 -- start start 2023-10-06T05:25:11.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.492+0000 7fc1489ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc14406a570 0x7fc144162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.492+0000 7fc1489ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1440c71b0 con 0x7fc14413a5b0 2023-10-06T05:25:11.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.492+0000 7fc1489ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1440c7490 con 0x7fc14413a1e0 2023-10-06T05:25:11.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.492+0000 7fc1489ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1440c7770 con 0x7fc14406a570 2023-10-06T05:25:11.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.492+0000 7fc141d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc14413a5b0 0x7fc14406d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56020/0 (socket says 172.21.15.31:56020) 2023-10-06T05:25:11.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.492+0000 7fc141d9b700 1 -- 172.21.15.31:0/2565857676 learned_addr learned my addr 172.21.15.31:0/2565857676 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.492+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070875973 0 0) 0x7fc1440c71b0 con 0x7fc14413a5b0 2023-10-06T05:25:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.492+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc12c003570 con 0x7fc14413a5b0 2023-10-06T05:25:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.492+0000 7fc142d9d700 1 --2- 172.21.15.31:0/2565857676 >> v2:172.21.15.62:3300/0 conn(0x7fc14406a570 0x7fc144162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2359369485 0 0) 0x7fc12c003570 con 0x7fc14413a5b0 2023-10-06T05:25:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 >> v2:172.21.15.62:3300/0 conn(0x7fc14406a570 msgr2=0x7fc144162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 --2- 172.21.15.31:0/2565857676 >> v2:172.21.15.62:3300/0 conn(0x7fc14406a570 0x7fc144162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 >> v1:172.21.15.62:6789/0 conn(0x7fc14413a1e0 legacy=0x7fc1440c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1440c7a50 con 0x7fc14413a5b0 2023-10-06T05:25:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fc138001470 con 0x7fc14413a5b0 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 >> v1:172.21.15.31:6789/0 conn(0x7fc14413a5b0 legacy=0x7fc14406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c003ab0 0x7fc12c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc12c004810 0x7fc12c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc12c005210 0x7fc12c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1440c7770 con 0x7fc12c005210 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c00c130 con 0x7fc12c003ab0 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c00c2f0 con 0x7fc12c004810 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc142d9d700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc12c005210 0x7fc12c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc14259c700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c003ab0 0x7fc12c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.493+0000 7fc141d9b700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc12c004810 0x7fc12c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc142d9d700 1 -- 172.21.15.31:0/2565857676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc12c004810 msgr2=0x7fc12c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:11.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc142d9d700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc12c004810 0x7fc12c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc142d9d700 1 -- 172.21.15.31:0/2565857676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c003ab0 msgr2=0x7fc12c003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc142d9d700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c003ab0 0x7fc12c003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc142d9d700 1 -- 172.21.15.31:0/2565857676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c003570 con 0x7fc12c005210 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc141d9b700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc12c004810 0x7fc12c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc142d9d700 1 -- 172.21.15.31:0/2565857676 reap_dead start 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc142d9d700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc12c005210 0x7fc12c00ba60 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc14406a960 tx=0x7fc13c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8edad99949e793 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c00ea90 con 0x7fc12c005210 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c00f040 con 0x7fc12c005210 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.494+0000 7fc14159a700 1 -- 172.21.15.31:0/2565857676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c00ee40 con 0x7fc12c005210 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.495+0000 7fc1489ff700 1 -- 172.21.15.31:0/2565857676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc12c005210 msgr2=0x7fc12c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.495+0000 7fc1489ff700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc12c005210 0x7fc12c00ba60 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc14406a960 tx=0x7fc13c00b040 comp rx=0 tx=0).stop 2023-10-06T05:25:11.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.495+0000 7fc1489ff700 1 -- 172.21.15.31:0/2565857676 shutdown_connections 2023-10-06T05:25:11.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.495+0000 7fc1489ff700 1 --2- 172.21.15.31:0/2565857676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc12c005210 0x7fc12c00ba60 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.495+0000 7fc1489ff700 1 -- 172.21.15.31:0/2565857676 >> 172.21.15.31:0/2565857676 conn(0x7fc1440cc0b0 msgr2=0x7fc1440cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:11.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.495+0000 7fc1489ff700 1 -- 172.21.15.31:0/2565857676 shutdown_connections 2023-10-06T05:25:11.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.495+0000 7fc1489ff700 1 -- 172.21.15.31:0/2565857676 wait complete. 2023-10-06T05:25:11.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.496+0000 7fc1489ff700 1 Processor -- start 2023-10-06T05:25:11.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.496+0000 7fc1489ff700 1 -- start start 2023-10-06T05:25:11.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.497+0000 7fc1489ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c005210 0x7fc144168ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.497+0000 7fc1489ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1440b07d0 0x7fc1440b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.497+0000 7fc1489ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc14416cb90 0x7fc14416d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.497+0000 7fc1489ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14416d570 con 0x7fc14416cb90 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.497+0000 7fc1489ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14414f210 con 0x7fc12c005210 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.497+0000 7fc1489ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc144169410 con 0x7fc1440b07d0 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc142d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc14416cb90 0x7fc14416d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc142d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc14416cb90 0x7fc14416d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38044/0 (socket says 172.21.15.31:38044) 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc141d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1440b07d0 0x7fc1440b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc142d9d700 1 -- 172.21.15.31:0/1388346770 learned_addr learned my addr 172.21.15.31:0/1388346770 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc14259c700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c005210 0x7fc144168ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc142d9d700 1 -- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1440b07d0 msgr2=0x7fc1440b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc142d9d700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1440b07d0 0x7fc1440b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc142d9d700 1 -- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c005210 msgr2=0x7fc144168ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc142d9d700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c005210 0x7fc144168ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc142d9d700 1 -- 172.21.15.31:0/1388346770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1440b12b0 con 0x7fc14416cb90 2023-10-06T05:25:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.498+0000 7fc142d9d700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc14416cb90 0x7fc14416d030 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00afd0 tx=0x7fc13c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.499+0000 7fc1327fc700 1 -- 172.21.15.31:0/1388346770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c01d660 con 0x7fc14416cb90 2023-10-06T05:25:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.499+0000 7fc1327fc700 1 -- 172.21.15.31:0/1388346770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c022070 con 0x7fc14416cb90 2023-10-06T05:25:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.499+0000 7fc1489ff700 1 -- 172.21.15.31:0/1388346770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1440b1590 con 0x7fc14416cb90 2023-10-06T05:25:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.499+0000 7fc1327fc700 1 -- 172.21.15.31:0/1388346770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c01da30 con 0x7fc14416cb90 2023-10-06T05:25:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.499+0000 7fc1489ff700 1 -- 172.21.15.31:0/1388346770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14414eea0 con 0x7fc14416cb90 2023-10-06T05:25:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.500+0000 7fc1489ff700 1 -- 172.21.15.31:0/1388346770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14414f3c0 con 0x7fc14416cb90 2023-10-06T05:25:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.501+0000 7fc1327fc700 1 -- 172.21.15.31:0/1388346770 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fc13c003020 con 0x7fc14416cb90 2023-10-06T05:25:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.502+0000 7fc1327fc700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc124067680 0x7fc124069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.502+0000 7fc1327fc700 1 -- 172.21.15.31:0/1388346770 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fc13c024070 con 0x7fc14416cb90 2023-10-06T05:25:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.502+0000 7fc14259c700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc124067680 0x7fc124069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.503+0000 7fc14259c700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc124067680 0x7fc124069b30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc134000c00 tx=0x7fc134007d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.506+0000 7fc1327fc700 1 -- 172.21.15.31:0/1388346770 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc14414f3c0 con 0x7fc14416cb90 2023-10-06T05:25:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.754+0000 7fc1489ff700 1 -- 172.21.15.31:0/1388346770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc14414f380 con 0x7fc14416cb90 2023-10-06T05:25:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.755+0000 7fc1327fc700 1 -- 172.21.15.31:0/1388346770 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7fc13c018020 con 0x7fc14416cb90 2023-10-06T05:25:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:11.757 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1352198494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1352198494},{"type":"v1","addr":"172.21.15.62:6835","nonce":1352198494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8194,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/243154980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":243154980},{"type":"v1","addr":"172.21.15.62:6834","nonce":243154980}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/240917476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":240917476},{"type":"v1","addr":"172.21.15.31:6834","nonce":240917476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:46.824751+0000","modified":"2023-10-06T05:24:50.335585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3829644591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3829644591},{"type":"v1","addr":"172.21.15.31:6835","nonce":3829644591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 -- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc124067680 msgr2=0x7fc124069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc124067680 0x7fc124069b30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc134000c00 tx=0x7fc134007d80 comp rx=0 tx=0).stop 2023-10-06T05:25:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 -- 172.21.15.31:0/1388346770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc14416cb90 msgr2=0x7fc14416d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc14416cb90 0x7fc14416d030 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00afd0 tx=0x7fc13c019000 comp rx=0 tx=0).stop 2023-10-06T05:25:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 -- 172.21.15.31:0/1388346770 shutdown_connections 2023-10-06T05:25:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc124067680 0x7fc124069b30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc14416cb90 0x7fc14416d030 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c005210 0x7fc144168ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 --2- 172.21.15.31:0/1388346770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc1440b07d0 0x7fc1440b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 -- 172.21.15.31:0/1388346770 >> 172.21.15.31:0/1388346770 conn(0x7fc1440cc0b0 msgr2=0x7fc1440c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:11.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.757+0000 7fc1489ff700 1 -- 172.21.15.31:0/1388346770 shutdown_connections 2023-10-06T05:25:11.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.758+0000 7fc1489ff700 1 -- 172.21.15.31:0/1388346770 wait complete. 2023-10-06T05:25:11.763 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 94 2023-10-06T05:25:11.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.971+0000 7fa9c8dff700 1 Processor -- start 2023-10-06T05:25:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.971+0000 7fa9c8dff700 1 -- start start 2023-10-06T05:25:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.973+0000 7fa9c8dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa9c40cd510 0x7fa9c40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.973+0000 7fa9c8dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c40c3f40 con 0x7fa9c4159500 2023-10-06T05:25:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.973+0000 7fa9c8dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c40cb000 con 0x7fa9c415d2d0 2023-10-06T05:25:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.973+0000 7fa9c8dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c40b2860 con 0x7fa9c40cd510 2023-10-06T05:25:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.973+0000 7fa9c259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa9c4159500 0x7fa9c40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56026/0 (socket says 172.21.15.31:56026) 2023-10-06T05:25:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.973+0000 7fa9c259c700 1 -- 172.21.15.31:0/496389392 learned_addr learned my addr 172.21.15.31:0/496389392 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.973+0000 7fa9c1d9b700 1 --2- 172.21.15.31:0/496389392 >> v2:172.21.15.62:3300/0 conn(0x7fa9c40cd510 0x7fa9c40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.973+0000 7fa9c159a700 1 -- 172.21.15.31:0/496389392 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051738540 0 0) 0x7fa9c40c3f40 con 0x7fa9c4159500 2023-10-06T05:25:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.973+0000 7fa9c159a700 1 -- 172.21.15.31:0/496389392 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9ac003570 con 0x7fa9c4159500 2023-10-06T05:25:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.974+0000 7fa9c159a700 1 -- 172.21.15.31:0/496389392 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761514680 0 0) 0x7fa9c40cb000 con 0x7fa9c415d2d0 2023-10-06T05:25:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.974+0000 7fa9c159a700 1 -- 172.21.15.31:0/496389392 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c40c3f40 con 0x7fa9c415d2d0 2023-10-06T05:25:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.974+0000 7fa9c1d9b700 1 -- 172.21.15.31:0/496389392 >> v1:172.21.15.62:6789/0 conn(0x7fa9c415d2d0 legacy=0x7fa9c40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.974+0000 7fa9c1d9b700 1 -- 172.21.15.31:0/496389392 >> v1:172.21.15.31:6789/0 conn(0x7fa9c4159500 legacy=0x7fa9c40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.974+0000 7fa9c159a700 1 -- 172.21.15.31:0/496389392 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3590212027 0 0) 0x7fa9ac003570 con 0x7fa9c4159500 2023-10-06T05:25:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.974+0000 7fa9c1d9b700 1 -- 172.21.15.31:0/496389392 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c415d7d0 con 0x7fa9c40cd510 2023-10-06T05:25:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.974+0000 7fa9c1d9b700 1 --2- 172.21.15.31:0/496389392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c40cd510 0x7fa9c40b0ab0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80098d0 tx=0x7fa9b8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51b240b5d4a5311 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.974+0000 7fa9c159a700 1 -- 172.21.15.31:0/496389392 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4001470 con 0x7fa9c40cd510 2023-10-06T05:25:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.974+0000 7fa9c159a700 1 -- 172.21.15.31:0/496389392 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b8011420 con 0x7fa9c40cd510 2023-10-06T05:25:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.974+0000 7fa9c159a700 1 -- 172.21.15.31:0/496389392 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80117e0 con 0x7fa9c40cd510 2023-10-06T05:25:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.975+0000 7fa9c8dff700 1 -- 172.21.15.31:0/496389392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c40cd510 msgr2=0x7fa9c40b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.975+0000 7fa9c8dff700 1 --2- 172.21.15.31:0/496389392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c40cd510 0x7fa9c40b0ab0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80098d0 tx=0x7fa9b8009120 comp rx=0 tx=0).stop 2023-10-06T05:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.975+0000 7fa9c8dff700 1 -- 172.21.15.31:0/496389392 shutdown_connections 2023-10-06T05:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.975+0000 7fa9c8dff700 1 --2- 172.21.15.31:0/496389392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c40cd510 0x7fa9c40b0ab0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.975+0000 7fa9c8dff700 1 -- 172.21.15.31:0/496389392 >> 172.21.15.31:0/496389392 conn(0x7fa9c406ec90 msgr2=0x7fa9c406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.975+0000 7fa9c8dff700 1 -- 172.21.15.31:0/496389392 shutdown_connections 2023-10-06T05:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.975+0000 7fa9c8dff700 1 -- 172.21.15.31:0/496389392 wait complete. 2023-10-06T05:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.976+0000 7fa9c8dff700 1 Processor -- start 2023-10-06T05:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.977+0000 7fa9c8dff700 1 -- start start 2023-10-06T05:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.977+0000 7fa9c8dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40cd510 0x7fa9c4156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.977+0000 7fa9c8dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c4159500 0x7fa9c415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.977+0000 7fa9c8dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9c415d2d0 0x7fa9c40c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.977+0000 7fa9c8dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40b3f00 con 0x7fa9c40cd510 2023-10-06T05:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.977+0000 7fa9c8dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40b3330 con 0x7fa9c4159500 2023-10-06T05:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.977+0000 7fa9c8dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40b35d0 con 0x7fa9c415d2d0 2023-10-06T05:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40cd510 0x7fa9c4156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40cd510 0x7fa9c4156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38052/0 (socket says 172.21.15.31:38052) 2023-10-06T05:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c259c700 1 -- 172.21.15.31:0/1976336563 learned_addr learned my addr 172.21.15.31:0/1976336563 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c2d9d700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9c415d2d0 0x7fa9c40c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c1d9b700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c4159500 0x7fa9c415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c259c700 1 -- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9c415d2d0 msgr2=0x7fa9c40c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c259c700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9c415d2d0 0x7fa9c40c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c259c700 1 -- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c4159500 msgr2=0x7fa9c415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c259c700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c4159500 0x7fa9c415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c259c700 1 -- 172.21.15.31:0/1976336563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b80093c0 con 0x7fa9c40cd510 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.978+0000 7fa9c259c700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40cd510 0x7fa9c4156ad0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400cb70 tx=0x7fa9b4001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.979+0000 7fa9b27fc700 1 -- 172.21.15.31:0/1976336563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4001d00 con 0x7fa9c40cd510 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.979+0000 7fa9b27fc700 1 -- 172.21.15.31:0/1976336563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b4007360 con 0x7fa9c40cd510 2023-10-06T05:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.979+0000 7fa9b27fc700 1 -- 172.21.15.31:0/1976336563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4004830 con 0x7fa9c40cd510 2023-10-06T05:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.979+0000 7fa9c8dff700 1 -- 172.21.15.31:0/1976336563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c40b38b0 con 0x7fa9c40cd510 2023-10-06T05:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.979+0000 7fa9c8dff700 1 -- 172.21.15.31:0/1976336563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c4157010 con 0x7fa9c40cd510 2023-10-06T05:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.980+0000 7fa9c8dff700 1 -- 172.21.15.31:0/1976336563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c404c380 con 0x7fa9c40cd510 2023-10-06T05:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.981+0000 7fa9b27fc700 1 -- 172.21.15.31:0/1976336563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fa9b4004ba0 con 0x7fa9c40cd510 2023-10-06T05:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.982+0000 7fa9b27fc700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9a4067680 0x7fa9a4069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.982+0000 7fa9b27fc700 1 -- 172.21.15.31:0/1976336563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fa9b4085110 con 0x7fa9c40cd510 2023-10-06T05:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.982+0000 7fa9c1d9b700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9a4067680 0x7fa9a4069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.983+0000 7fa9c1d9b700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9a4067680 0x7fa9a4069b30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40cd910 tx=0x7fa9b8016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:11.986+0000 7fa9b27fc700 1 -- 172.21.15.31:0/1976336563 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9b40824d0 con 0x7fa9c40cd510 2023-10-06T05:25:12.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.233+0000 7fa9c8dff700 1 -- 172.21.15.31:0/1976336563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9c40cade0 con 0x7fa9c40cd510 2023-10-06T05:25:12.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.234+0000 7fa9b27fc700 1 -- 172.21.15.31:0/1976336563 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7fa9b4018090 con 0x7fa9c40cd510 2023-10-06T05:25:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:12.236 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1352198494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1352198494},{"type":"v1","addr":"172.21.15.62:6835","nonce":1352198494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8194,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/243154980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":243154980},{"type":"v1","addr":"172.21.15.62:6834","nonce":243154980}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/240917476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":240917476},{"type":"v1","addr":"172.21.15.31:6834","nonce":240917476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:24:46.824751+0000","modified":"2023-10-06T05:24:50.335585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8478":{"gid":8478,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3829644591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3829644591},{"type":"v1","addr":"172.21.15.31:6835","nonce":3829644591}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:25:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 -- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9a4067680 msgr2=0x7fa9a4069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9a4067680 0x7fa9a4069b30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40cd910 tx=0x7fa9b8016000 comp rx=0 tx=0).stop 2023-10-06T05:25:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 -- 172.21.15.31:0/1976336563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40cd510 msgr2=0x7fa9c4156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40cd510 0x7fa9c4156ad0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400cb70 tx=0x7fa9b4001ab0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 -- 172.21.15.31:0/1976336563 shutdown_connections 2023-10-06T05:25:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa9a4067680 0x7fa9a4069b30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9c415d2d0 0x7fa9c40c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40cd510 0x7fa9c4156ad0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 --2- 172.21.15.31:0/1976336563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c4159500 0x7fa9c415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 -- 172.21.15.31:0/1976336563 >> 172.21.15.31:0/1976336563 conn(0x7fa9c406ec90 msgr2=0x7fa9c4158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 -- 172.21.15.31:0/1976336563 shutdown_connections 2023-10-06T05:25:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.236+0000 7fa9c8dff700 1 -- 172.21.15.31:0/1976336563 wait complete. 2023-10-06T05:25:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 94 2023-10-06T05:25:12.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.449+0000 7ffb38dff700 1 Processor -- start 2023-10-06T05:25:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.449+0000 7ffb38dff700 1 -- start start 2023-10-06T05:25:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.450+0000 7ffb38dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3413a5b0 0x7ffb3406d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.450+0000 7ffb38dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffb3406d980 0x7ffb34069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.450+0000 7ffb38dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3406ddc0 con 0x7ffb3413a1e0 2023-10-06T05:25:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.450+0000 7ffb38dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb340c71b0 con 0x7ffb3406d980 2023-10-06T05:25:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.450+0000 7ffb38dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340c7490 con 0x7ffb3413a5b0 2023-10-06T05:25:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.451+0000 7ffb31d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3413a5b0 0x7ffb3406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.451+0000 7ffb32d9d700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffb3406d980 0x7ffb34069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.451+0000 7ffb31d9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3413a5b0 0x7ffb3406d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57140/0 (socket says 172.21.15.31:57140) 2023-10-06T05:25:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.451+0000 7ffb31d9b700 1 -- 172.21.15.31:0/1209889870 learned_addr learned my addr 172.21.15.31:0/1209889870 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.451+0000 7ffb3259c700 1 --1- 172.21.15.31:0/1209889870 >> v1:172.21.15.62:6789/0 conn(0x7ffb3413a1e0 0x7ffb340c66f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:48130/0 (socket says 172.21.15.31:48130) 2023-10-06T05:25:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.451+0000 7ffb31d9b700 1 -- 172.21.15.31:0/1209889870 >> v2:172.21.15.62:3300/0 conn(0x7ffb3406d980 msgr2=0x7ffb34069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.451+0000 7ffb31d9b700 1 --2- 172.21.15.31:0/1209889870 >> v2:172.21.15.62:3300/0 conn(0x7ffb3406d980 0x7ffb34069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.451+0000 7ffb31d9b700 1 -- 172.21.15.31:0/1209889870 >> v1:172.21.15.62:6789/0 conn(0x7ffb3413a1e0 legacy=0x7ffb340c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.451+0000 7ffb31d9b700 1 -- 172.21.15.31:0/1209889870 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340c7770 con 0x7ffb3413a5b0 2023-10-06T05:25:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.451+0000 7ffb32d9d700 1 --2- 172.21.15.31:0/1209889870 >> v2:172.21.15.62:3300/0 conn(0x7ffb3406d980 0x7ffb34069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:12.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.452+0000 7ffb31d9b700 1 --2- 172.21.15.31:0/1209889870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3413a5b0 0x7ffb3406d440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffb280099b0 tx=0x7ffb28009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66bc06ddea0b21b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.452+0000 7ffb3159a700 1 -- 172.21.15.31:0/1209889870 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28009ea0 con 0x7ffb3413a5b0 2023-10-06T05:25:12.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.452+0000 7ffb3159a700 1 -- 172.21.15.31:0/1209889870 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2800b860 con 0x7ffb3413a5b0 2023-10-06T05:25:12.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.452+0000 7ffb3159a700 1 -- 172.21.15.31:0/1209889870 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2800bc00 con 0x7ffb3413a5b0 2023-10-06T05:25:12.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.453+0000 7ffb38dff700 1 -- 172.21.15.31:0/1209889870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3413a5b0 msgr2=0x7ffb3406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.453+0000 7ffb38dff700 1 --2- 172.21.15.31:0/1209889870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3413a5b0 0x7ffb3406d440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffb280099b0 tx=0x7ffb28009250 comp rx=0 tx=0).stop 2023-10-06T05:25:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.453+0000 7ffb38dff700 1 -- 172.21.15.31:0/1209889870 shutdown_connections 2023-10-06T05:25:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.453+0000 7ffb38dff700 1 --2- 172.21.15.31:0/1209889870 >> v2:172.21.15.62:3300/0 conn(0x7ffb3406d980 0x7ffb34069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.453+0000 7ffb38dff700 1 --2- 172.21.15.31:0/1209889870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3413a5b0 0x7ffb3406d440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.453+0000 7ffb38dff700 1 -- 172.21.15.31:0/1209889870 >> 172.21.15.31:0/1209889870 conn(0x7ffb340cc0b0 msgr2=0x7ffb340cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:12.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.453+0000 7ffb38dff700 1 -- 172.21.15.31:0/1209889870 shutdown_connections 2023-10-06T05:25:12.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.453+0000 7ffb38dff700 1 -- 172.21.15.31:0/1209889870 wait complete. 2023-10-06T05:25:12.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.454+0000 7ffb38dff700 1 Processor -- start 2023-10-06T05:25:12.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.455+0000 7ffb38dff700 1 -- start start 2023-10-06T05:25:12.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.455+0000 7ffb38dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3406d980 0x7ffb34137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb38dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb3413a1e0 0x7ffb34150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb38dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb3413a5b0 0x7ffb34150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb38dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb341502c0 con 0x7ffb3413a5b0 2023-10-06T05:25:12.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb38dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34138040 con 0x7ffb3413a1e0 2023-10-06T05:25:12.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb38dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3414f6b0 con 0x7ffb3406d980 2023-10-06T05:25:12.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb32d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb3413a5b0 0x7ffb34150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb32d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb3413a5b0 0x7ffb34150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38058/0 (socket says 172.21.15.31:38058) 2023-10-06T05:25:12.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb32d9d700 1 -- 172.21.15.31:0/3238888383 learned_addr learned my addr 172.21.15.31:0/3238888383 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:12.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb3259c700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3406d980 0x7ffb34137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb31d9b700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb3413a1e0 0x7ffb34150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb32d9d700 1 -- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3406d980 msgr2=0x7ffb34137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb32d9d700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3406d980 0x7ffb34137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb32d9d700 1 -- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb3413a1e0 msgr2=0x7ffb34150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb32d9d700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb3413a1e0 0x7ffb34150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb32d9d700 1 -- 172.21.15.31:0/3238888383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c00de30 con 0x7ffb3413a5b0 2023-10-06T05:25:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.456+0000 7ffb3259c700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3406d980 0x7ffb34137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.457+0000 7ffb32d9d700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb3413a5b0 0x7ffb34150f90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0124f0 tx=0x7ffb2c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.457+0000 7ffb227fc700 1 -- 172.21.15.31:0/3238888383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c012e70 con 0x7ffb3413a5b0 2023-10-06T05:25:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.457+0000 7ffb227fc700 1 -- 172.21.15.31:0/3238888383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c013040 con 0x7ffb3413a5b0 2023-10-06T05:25:12.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.457+0000 7ffb38dff700 1 -- 172.21.15.31:0/3238888383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb280094f0 con 0x7ffb3413a5b0 2023-10-06T05:25:12.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.457+0000 7ffb227fc700 1 -- 172.21.15.31:0/3238888383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c017860 con 0x7ffb3413a5b0 2023-10-06T05:25:12.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.457+0000 7ffb38dff700 1 -- 172.21.15.31:0/3238888383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3414fc70 con 0x7ffb3413a5b0 2023-10-06T05:25:12.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.458+0000 7ffb38dff700 1 -- 172.21.15.31:0/3238888383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3404c380 con 0x7ffb3413a5b0 2023-10-06T05:25:12.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.459+0000 7ffb227fc700 1 -- 172.21.15.31:0/3238888383 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7ffb2c017bd0 con 0x7ffb3413a5b0 2023-10-06T05:25:12.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.460+0000 7ffb227fc700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffb14067680 0x7ffb14069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.460+0000 7ffb227fc700 1 -- 172.21.15.31:0/3238888383 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ffb2c08a830 con 0x7ffb3413a5b0 2023-10-06T05:25:12.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.460+0000 7ffb3259c700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffb14067680 0x7ffb14069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.461+0000 7ffb3259c700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffb14067680 0x7ffb14069b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffb24007120 tx=0x7ffb24007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.464+0000 7ffb227fc700 1 -- 172.21.15.31:0/3238888383 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb2c087ae0 con 0x7ffb3413a5b0 2023-10-06T05:25:12.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.665+0000 7ffb38dff700 1 -- 172.21.15.31:0/3238888383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb340c7a50 con 0x7ffb3413a5b0 2023-10-06T05:25:12.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.666+0000 7ffb227fc700 1 -- 172.21.15.31:0/3238888383 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16972 (secure 0 0 0) 0x7ffb2c05a3e0 con 0x7ffb3413a5b0 2023-10-06T05:25:12.668 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:12.669 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":139,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:24:48.323184+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:24:44.887621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:24:45.794431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:12.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.668+0000 7ffb38dff700 1 -- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffb14067680 msgr2=0x7ffb14069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.668+0000 7ffb38dff700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffb14067680 0x7ffb14069b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffb24007120 tx=0x7ffb24007dc0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.668+0000 7ffb38dff700 1 -- 172.21.15.31:0/3238888383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb3413a5b0 msgr2=0x7ffb34150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.668+0000 7ffb38dff700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb3413a5b0 0x7ffb34150f90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0124f0 tx=0x7ffb2c00db90 comp rx=0 tx=0).stop 2023-10-06T05:25:12.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.669+0000 7ffb38dff700 1 -- 172.21.15.31:0/3238888383 shutdown_connections 2023-10-06T05:25:12.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.669+0000 7ffb38dff700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffb14067680 0x7ffb14069b30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.669+0000 7ffb38dff700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb3413a5b0 0x7ffb34150f90 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.669+0000 7ffb38dff700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffb3406d980 0x7ffb34137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.669+0000 7ffb38dff700 1 --2- 172.21.15.31:0/3238888383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb3413a1e0 0x7ffb34150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.669+0000 7ffb38dff700 1 -- 172.21.15.31:0/3238888383 >> 172.21.15.31:0/3238888383 conn(0x7ffb340cc0b0 msgr2=0x7ffb34069460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:12.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.669+0000 7ffb38dff700 1 -- 172.21.15.31:0/3238888383 shutdown_connections 2023-10-06T05:25:12.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.669+0000 7ffb38dff700 1 -- 172.21.15.31:0/3238888383 wait complete. 2023-10-06T05:25:12.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:25:12.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.883+0000 7f760c7ff700 1 Processor -- start 2023-10-06T05:25:12.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.883+0000 7f760c7ff700 1 -- start start 2023-10-06T05:25:12.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.884+0000 7f760c7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f76041595a0 0x7f7604159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.884+0000 7f760c7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76040cc0d0 0x7f7604068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.885+0000 7f760c7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7604069010 con 0x7f7604068bb0 2023-10-06T05:25:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.885+0000 7f760c7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76040c95b0 con 0x7f76041595a0 2023-10-06T05:25:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.885+0000 7f760c7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76040c9890 con 0x7f76040cc0d0 2023-10-06T05:25:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.885+0000 7f760a59b700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f76041595a0 0x7f7604159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.885+0000 7f7609d9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76040cc0d0 0x7f7604068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.885+0000 7f760ad9c700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7604068bb0 0x7f76040c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:48134/0 (socket says 172.21.15.31:48134) 2023-10-06T05:25:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.885+0000 7f760ad9c700 1 -- 172.21.15.31:0/3566293705 learned_addr learned my addr 172.21.15.31:0/3566293705 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.886+0000 7f760a59b700 1 -- 172.21.15.31:0/3566293705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76040cc0d0 msgr2=0x7f7604068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.885+0000 7f7609599700 1 -- 172.21.15.31:0/3566293705 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600079326 0 0) 0x7f75fc001490 con 0x7f7604068bb0 2023-10-06T05:25:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.886+0000 7f760a59b700 1 --2- 172.21.15.31:0/3566293705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76040cc0d0 0x7f7604068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.886+0000 7f760a59b700 1 -- 172.21.15.31:0/3566293705 >> v1:172.21.15.62:6789/0 conn(0x7f7604068bb0 legacy=0x7f76040c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.886+0000 7f760a59b700 1 -- 172.21.15.31:0/3566293705 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76040c9b70 con 0x7f76041595a0 2023-10-06T05:25:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.886+0000 7f760a59b700 1 --2- 172.21.15.31:0/3566293705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76041595a0 0x7f7604159970 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f76000098e0 tx=0x7f76000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffb63aaaaab64edd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.886+0000 7f7609599700 1 -- 172.21.15.31:0/3566293705 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760000b640 con 0x7f76041595a0 2023-10-06T05:25:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.886+0000 7f7609599700 1 -- 172.21.15.31:0/3566293705 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760000b7a0 con 0x7f76041595a0 2023-10-06T05:25:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.886+0000 7f7609599700 1 -- 172.21.15.31:0/3566293705 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760000bb40 con 0x7f76041595a0 2023-10-06T05:25:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.887+0000 7f760c7ff700 1 -- 172.21.15.31:0/3566293705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76041595a0 msgr2=0x7f7604159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.887+0000 7f760c7ff700 1 --2- 172.21.15.31:0/3566293705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76041595a0 0x7f7604159970 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f76000098e0 tx=0x7f76000091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.887+0000 7f760c7ff700 1 -- 172.21.15.31:0/3566293705 shutdown_connections 2023-10-06T05:25:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.887+0000 7f760c7ff700 1 --2- 172.21.15.31:0/3566293705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76041595a0 0x7f7604159970 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.887+0000 7f760c7ff700 1 --2- 172.21.15.31:0/3566293705 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76040cc0d0 0x7f7604068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.887+0000 7f760c7ff700 1 -- 172.21.15.31:0/3566293705 >> 172.21.15.31:0/3566293705 conn(0x7f760406b620 msgr2=0x7f760406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.887+0000 7f760c7ff700 1 -- 172.21.15.31:0/3566293705 shutdown_connections 2023-10-06T05:25:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.887+0000 7f760c7ff700 1 -- 172.21.15.31:0/3566293705 wait complete. 2023-10-06T05:25:12.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.888+0000 7f760c7ff700 1 Processor -- start 2023-10-06T05:25:12.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.889+0000 7f760c7ff700 1 -- start start 2023-10-06T05:25:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760c7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7604068bb0 0x7f76041553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760c7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76040cc0d0 0x7f76040c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760c7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76041595a0 0x7f76040b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760c7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76040b2d60 con 0x7f76040cc0d0 2023-10-06T05:25:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760c7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7604155930 con 0x7f76041595a0 2023-10-06T05:25:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760c7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76040b2190 con 0x7f7604068bb0 2023-10-06T05:25:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f7609d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76040cc0d0 0x7f76040c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760a59b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7604068bb0 0x7f76041553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760a59b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7604068bb0 0x7f76041553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:57160/0 (socket says 172.21.15.31:57160) 2023-10-06T05:25:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760a59b700 1 -- 172.21.15.31:0/4120932274 learned_addr learned my addr 172.21.15.31:0/4120932274 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:12.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f7609d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76040cc0d0 0x7f76040c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38060/0 (socket says 172.21.15.31:38060) 2023-10-06T05:25:12.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760ad9c700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76041595a0 0x7f76040b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f7609d9a700 1 -- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7604068bb0 msgr2=0x7f76041553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f7609d9a700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7604068bb0 0x7f76041553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f7609d9a700 1 -- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76041595a0 msgr2=0x7f76040b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:12.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f7609d9a700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76041595a0 0x7f76040b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:12.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f7609d9a700 1 -- 172.21.15.31:0/4120932274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7600009420 con 0x7f76040cc0d0 2023-10-06T05:25:12.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.890+0000 7f760ad9c700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76041595a0 0x7f76040b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:12.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.891+0000 7f7609d9a700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76040cc0d0 0x7f76040c5890 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f75f400ea50 tx=0x7f75f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.891+0000 7f75fa7fc700 1 -- 172.21.15.31:0/4120932274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f40095d0 con 0x7f76040cc0d0 2023-10-06T05:25:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.891+0000 7f75fa7fc700 1 -- 172.21.15.31:0/4120932274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f400c7f0 con 0x7f76040cc0d0 2023-10-06T05:25:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.891+0000 7f75fa7fc700 1 -- 172.21.15.31:0/4120932274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f400cbb0 con 0x7f76040cc0d0 2023-10-06T05:25:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.891+0000 7f760c7ff700 1 -- 172.21.15.31:0/4120932274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76040b2430 con 0x7f76040cc0d0 2023-10-06T05:25:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.891+0000 7f760c7ff700 1 -- 172.21.15.31:0/4120932274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76040b29f0 con 0x7f76040cc0d0 2023-10-06T05:25:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.892+0000 7f760c7ff700 1 -- 172.21.15.31:0/4120932274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76040b2ed0 con 0x7f76040cc0d0 2023-10-06T05:25:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.893+0000 7f75fa7fc700 1 -- 172.21.15.31:0/4120932274 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f75f4013470 con 0x7f76040cc0d0 2023-10-06T05:25:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.894+0000 7f75fa7fc700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75e8067680 0x7f75e8069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.894+0000 7f75fa7fc700 1 -- 172.21.15.31:0/4120932274 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f75f4083060 con 0x7f76040cc0d0 2023-10-06T05:25:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.895+0000 7f760a59b700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75e8067680 0x7f75e8069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.895+0000 7f760a59b700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75e8067680 0x7f75e8069b30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7600003080 tx=0x7f7600017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:12.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:12.898+0000 7f75fa7fc700 1 -- 172.21.15.31:0/4120932274 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75f4082d70 con 0x7f76040cc0d0 2023-10-06T05:25:13.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.145+0000 7f760c7ff700 1 -- 172.21.15.31:0/4120932274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f760404c380 con 0x7f76040cc0d0 2023-10-06T05:25:13.147 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:25:13.146+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:25:13.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.153+0000 7f75fa7fc700 1 -- 172.21.15.31:0/4120932274 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) v1 ==== 161+0+0 (secure 0 0 0) 0x7f75f4055360 con 0x7f76040cc0d0 2023-10-06T05:25:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 -- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75e8067680 msgr2=0x7f75e8069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75e8067680 0x7f75e8069b30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7600003080 tx=0x7f7600017000 comp rx=0 tx=0).stop 2023-10-06T05:25:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 -- 172.21.15.31:0/4120932274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76040cc0d0 msgr2=0x7f76040c5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76040cc0d0 0x7f76040c5890 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f75f400ea50 tx=0x7f75f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 -- 172.21.15.31:0/4120932274 shutdown_connections 2023-10-06T05:25:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75e8067680 0x7f75e8069b30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76041595a0 0x7f76040b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7604068bb0 0x7f76041553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 --2- 172.21.15.31:0/4120932274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76040cc0d0 0x7f76040c5890 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 -- 172.21.15.31:0/4120932274 >> 172.21.15.31:0/4120932274 conn(0x7f760406b620 msgr2=0x7f76041561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 -- 172.21.15.31:0/4120932274 shutdown_connections 2023-10-06T05:25:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.156+0000 7f760c7ff700 1 -- 172.21.15.31:0/4120932274 wait complete. 2023-10-06T05:25:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:25:13.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:25:13.219 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:25:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.374+0000 7fdd15fff700 1 Processor -- start 2023-10-06T05:25:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.374+0000 7fdd15fff700 1 -- start start 2023-10-06T05:25:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.375+0000 7fdd15fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd100cd580 0x7fdd101645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.375+0000 7fdd15fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1015d2d0 con 0x7fdd10159560 2023-10-06T05:25:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.375+0000 7fdd15fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1015d5b0 con 0x7fdd10159930 2023-10-06T05:25:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.375+0000 7fdd15fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1015d890 con 0x7fdd100cd580 2023-10-06T05:25:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.375+0000 7fdd0ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd100cd580 0x7fdd101645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.375+0000 7fdd0f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdd10159560 0x7fdd100cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56050/0 (socket says 172.21.15.31:56050) 2023-10-06T05:25:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.376+0000 7fdd0f7fe700 1 -- 172.21.15.31:0/3420375197 learned_addr learned my addr 172.21.15.31:0/3420375197 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.375+0000 7fdd0ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd100cd580 0x7fdd101645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38102/0 (socket says 172.21.15.31:38102) 2023-10-06T05:25:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.376+0000 7fdd0ffff700 1 -- 172.21.15.31:0/3420375197 >> v1:172.21.15.62:6789/0 conn(0x7fdd10159930 legacy=0x7fdd100b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.376+0000 7fdd0ffff700 1 -- 172.21.15.31:0/3420375197 >> v1:172.21.15.31:6789/0 conn(0x7fdd10159560 legacy=0x7fdd100cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.376+0000 7fdd0ffff700 1 -- 172.21.15.31:0/3420375197 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1015db70 con 0x7fdd100cd580 2023-10-06T05:25:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.376+0000 7fdd0e7fc700 1 -- 172.21.15.31:0/3420375197 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593695696 0 0) 0x7fdd1015d2d0 con 0x7fdd10159560 2023-10-06T05:25:13.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.376+0000 7fdd0ffff700 1 --2- 172.21.15.31:0/3420375197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd100cd580 0x7fdd101645d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009e60 tx=0x7fdd08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324e75feb1100df7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.376+0000 7fdd0e7fc700 1 -- 172.21.15.31:0/3420375197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0800e070 con 0x7fdd100cd580 2023-10-06T05:25:13.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.377+0000 7fdd0e7fc700 1 -- 172.21.15.31:0/3420375197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0800bc30 con 0x7fdd100cd580 2023-10-06T05:25:13.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.377+0000 7fdd0e7fc700 1 -- 172.21.15.31:0/3420375197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08012430 con 0x7fdd100cd580 2023-10-06T05:25:13.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.377+0000 7fdd15fff700 1 -- 172.21.15.31:0/3420375197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd100cd580 msgr2=0x7fdd101645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.377+0000 7fdd15fff700 1 --2- 172.21.15.31:0/3420375197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd100cd580 0x7fdd101645d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009e60 tx=0x7fdd08009780 comp rx=0 tx=0).stop 2023-10-06T05:25:13.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.377+0000 7fdd15fff700 1 -- 172.21.15.31:0/3420375197 shutdown_connections 2023-10-06T05:25:13.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.377+0000 7fdd15fff700 1 --2- 172.21.15.31:0/3420375197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd100cd580 0x7fdd101645d0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.377+0000 7fdd15fff700 1 -- 172.21.15.31:0/3420375197 >> 172.21.15.31:0/3420375197 conn(0x7fdd1006ec90 msgr2=0x7fdd1006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.377+0000 7fdd15fff700 1 -- 172.21.15.31:0/3420375197 shutdown_connections 2023-10-06T05:25:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.377+0000 7fdd15fff700 1 -- 172.21.15.31:0/3420375197 wait complete. 2023-10-06T05:25:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.379+0000 7fdd15fff700 1 Processor -- start 2023-10-06T05:25:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.379+0000 7fdd15fff700 1 -- start start 2023-10-06T05:25:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd15fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd100cd580 0x7fdd100d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd15fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd10159560 0x7fdd100c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd15fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd10159930 0x7fdd100c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd15fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100c8080 con 0x7fdd10159560 2023-10-06T05:25:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd15fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100c8360 con 0x7fdd10159930 2023-10-06T05:25:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd15fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100c8640 con 0x7fdd100cd580 2023-10-06T05:25:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd0effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd10159560 0x7fdd100c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd0effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd10159560 0x7fdd100c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38108/0 (socket says 172.21.15.31:38108) 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd0effd700 1 -- 172.21.15.31:0/383563649 learned_addr learned my addr 172.21.15.31:0/383563649 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd0f7fe700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd100cd580 0x7fdd100d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd0ffff700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd10159930 0x7fdd100c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd0effd700 1 -- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd100cd580 msgr2=0x7fdd100d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd0effd700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd100cd580 0x7fdd100d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd0effd700 1 -- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd10159930 msgr2=0x7fdd100c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd0effd700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd10159930 0x7fdd100c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.380+0000 7fdd0effd700 1 -- 172.21.15.31:0/383563649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd08009a20 con 0x7fdd10159560 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.381+0000 7fdd0effd700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd10159560 0x7fdd100c74a0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400c590 tx=0x7fdd04007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.381+0000 7fdcf77fe700 1 -- 172.21.15.31:0/383563649 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04011590 con 0x7fdd10159560 2023-10-06T05:25:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.381+0000 7fdcf77fe700 1 -- 172.21.15.31:0/383563649 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd04011730 con 0x7fdd10159560 2023-10-06T05:25:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.381+0000 7fdd15fff700 1 -- 172.21.15.31:0/383563649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd100c8920 con 0x7fdd10159560 2023-10-06T05:25:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.381+0000 7fdcf77fe700 1 -- 172.21.15.31:0/383563649 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04011af0 con 0x7fdd10159560 2023-10-06T05:25:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.381+0000 7fdd15fff700 1 -- 172.21.15.31:0/383563649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd100d08a0 con 0x7fdd10159560 2023-10-06T05:25:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.382+0000 7fdd15fff700 1 -- 172.21.15.31:0/383563649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd100d0ac0 con 0x7fdd10159560 2023-10-06T05:25:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.383+0000 7fdcf77fe700 1 -- 172.21.15.31:0/383563649 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fdd04012420 con 0x7fdd10159560 2023-10-06T05:25:13.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.384+0000 7fdcf77fe700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcfc067680 0x7fdcfc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:13.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.385+0000 7fdcf77fe700 1 -- 172.21.15.31:0/383563649 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fdd04084bf0 con 0x7fdd10159560 2023-10-06T05:25:13.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.385+0000 7fdd0f7fe700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcfc067680 0x7fdcfc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:13.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.387+0000 7fdd0f7fe700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcfc067680 0x7fdcfc069b30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdd100b01c0 tx=0x7fdd00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.389+0000 7fdcf77fe700 1 -- 172.21.15.31:0/383563649 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd0404b5a0 con 0x7fdd10159560 2023-10-06T05:25:13.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:13.632+0000 7fdd15fff700 1 -- 172.21.15.31:0/383563649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdd1004c380 con 0x7fdd10159560 2023-10-06T05:25:14.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.166+0000 7fdcf77fe700 1 -- 172.21.15.31:0/383563649 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd04017020 con 0x7fdd10159560 2023-10-06T05:25:14.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.169+0000 7fdd15fff700 1 -- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcfc067680 msgr2=0x7fdcfc069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.169+0000 7fdd15fff700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcfc067680 0x7fdcfc069b30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdd100b01c0 tx=0x7fdd00009040 comp rx=0 tx=0).stop 2023-10-06T05:25:14.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.169+0000 7fdd15fff700 1 -- 172.21.15.31:0/383563649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd10159560 msgr2=0x7fdd100c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.169+0000 7fdd15fff700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd10159560 0x7fdd100c74a0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400c590 tx=0x7fdd04007f30 comp rx=0 tx=0).stop 2023-10-06T05:25:14.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.170+0000 7fdd15fff700 1 -- 172.21.15.31:0/383563649 shutdown_connections 2023-10-06T05:25:14.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.170+0000 7fdd15fff700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdcfc067680 0x7fdcfc069b30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.170+0000 7fdd15fff700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd10159930 0x7fdd100c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.170+0000 7fdd15fff700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd100cd580 0x7fdd100d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.170+0000 7fdd15fff700 1 --2- 172.21.15.31:0/383563649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd10159560 0x7fdd100c74a0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.170+0000 7fdd15fff700 1 -- 172.21.15.31:0/383563649 >> 172.21.15.31:0/383563649 conn(0x7fdd1006ec90 msgr2=0x7fdd101587d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:14.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.170+0000 7fdd15fff700 1 -- 172.21.15.31:0/383563649 shutdown_connections 2023-10-06T05:25:14.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.170+0000 7fdd15fff700 1 -- 172.21.15.31:0/383563649 wait complete. 2023-10-06T05:25:14.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:25:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.437+0000 7f885a7ff700 1 Processor -- start 2023-10-06T05:25:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.437+0000 7f885a7ff700 1 -- start start 2023-10-06T05:25:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.438+0000 7f885a7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8854156300 0x7f88540ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.438+0000 7f885a7ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885406e600 con 0x7f885406ea90 2023-10-06T05:25:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.438+0000 7f885a7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885406ef00 con 0x7f885406a630 2023-10-06T05:25:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.438+0000 7f885a7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885406d9f0 con 0x7f8854156300 2023-10-06T05:25:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.439+0000 7f8853fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8854156300 0x7f88540ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.439+0000 7f88537fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f885406ea90 0x7f885406d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56060/0 (socket says 172.21.15.31:56060) 2023-10-06T05:25:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.439+0000 7f8853fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8854156300 0x7f88540ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38118/0 (socket says 172.21.15.31:38118) 2023-10-06T05:25:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.439+0000 7f88537fe700 1 -- 172.21.15.31:0/748505595 learned_addr learned my addr 172.21.15.31:0/748505595 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.439+0000 7f8853fff700 1 -- 172.21.15.31:0/748505595 >> v1:172.21.15.62:6789/0 conn(0x7f885406a630 legacy=0x7f88540c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.439+0000 7f8853fff700 1 -- 172.21.15.31:0/748505595 >> v1:172.21.15.31:6789/0 conn(0x7f885406ea90 legacy=0x7f885406d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.439+0000 7f8853fff700 1 -- 172.21.15.31:0/748505595 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885406dcd0 con 0x7f8854156300 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.439+0000 7f8852ffd700 1 -- 172.21.15.31:0/748505595 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697118711 0 0) 0x7f885406e600 con 0x7f885406ea90 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.439+0000 7f8853fff700 1 --2- 172.21.15.31:0/748505595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 0x7f88540ad670 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f88440098d0 tx=0x7f8844009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61fadb23b0e09a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.440+0000 7f8852ffd700 1 -- 172.21.15.31:0/748505595 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844009e70 con 0x7f8854156300 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.440+0000 7f8852ffd700 1 -- 172.21.15.31:0/748505595 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8844005c10 con 0x7f8854156300 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.440+0000 7f8852ffd700 1 -- 172.21.15.31:0/748505595 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884400b610 con 0x7f8854156300 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.440+0000 7f885a7ff700 1 -- 172.21.15.31:0/748505595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 msgr2=0x7f88540ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.440+0000 7f885a7ff700 1 --2- 172.21.15.31:0/748505595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 0x7f88540ad670 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f88440098d0 tx=0x7f8844009120 comp rx=0 tx=0).stop 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.440+0000 7f885a7ff700 1 -- 172.21.15.31:0/748505595 shutdown_connections 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.440+0000 7f885a7ff700 1 --2- 172.21.15.31:0/748505595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 0x7f88540ad670 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.440+0000 7f885a7ff700 1 -- 172.21.15.31:0/748505595 >> 172.21.15.31:0/748505595 conn(0x7f8854068030 msgr2=0x7f88540cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.440+0000 7f885a7ff700 1 -- 172.21.15.31:0/748505595 shutdown_connections 2023-10-06T05:25:14.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.440+0000 7f885a7ff700 1 -- 172.21.15.31:0/748505595 wait complete. 2023-10-06T05:25:14.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.442+0000 7f885a7ff700 1 Processor -- start 2023-10-06T05:25:14.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.442+0000 7f885a7ff700 1 -- start start 2023-10-06T05:25:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.443+0000 7f885a7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885406a630 0x7f88540aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.443+0000 7f885a7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885406ea90 0x7f88540b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.443+0000 7f885a7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 0x7f88540c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.443+0000 7f885a7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b0060 con 0x7f8854156300 2023-10-06T05:25:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.443+0000 7f885a7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b0a30 con 0x7f885406a630 2023-10-06T05:25:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.443+0000 7f885a7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88540af450 con 0x7f885406ea90 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.443+0000 7f8858d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 0x7f88540c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.443+0000 7f8858d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 0x7f88540c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38120/0 (socket says 172.21.15.31:38120) 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.443+0000 7f8858d9c700 1 -- 172.21.15.31:0/3262359947 learned_addr learned my addr 172.21.15.31:0/3262359947 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.443+0000 7f8853fff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885406a630 0x7f88540aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f8858d9c700 1 -- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885406ea90 msgr2=0x7f88540b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f88537fe700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885406ea90 0x7f88540b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f8858d9c700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885406ea90 0x7f88540b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f8858d9c700 1 -- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885406a630 msgr2=0x7f88540aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f8858d9c700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885406a630 0x7f88540aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f8858d9c700 1 -- 172.21.15.31:0/3262359947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88440093c0 con 0x7f8854156300 2023-10-06T05:25:14.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f8853fff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885406a630 0x7f88540aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:14.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f8858d9c700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 0x7f88540c3b50 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f884c00c930 tx=0x7f884c0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f883bfff700 1 -- 172.21.15.31:0/3262359947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c00ce70 con 0x7f8854156300 2023-10-06T05:25:14.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f883bfff700 1 -- 172.21.15.31:0/3262359947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c00a7e0 con 0x7f8854156300 2023-10-06T05:25:14.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f883bfff700 1 -- 172.21.15.31:0/3262359947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c00aba0 con 0x7f8854156300 2023-10-06T05:25:14.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f885a7ff700 1 -- 172.21.15.31:0/3262359947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88540af730 con 0x7f8854156300 2023-10-06T05:25:14.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.444+0000 7f885a7ff700 1 -- 172.21.15.31:0/3262359947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88540afcf0 con 0x7f8854156300 2023-10-06T05:25:14.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.445+0000 7f885a7ff700 1 -- 172.21.15.31:0/3262359947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88540cbe10 con 0x7f8854156300 2023-10-06T05:25:14.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.446+0000 7f883bfff700 1 -- 172.21.15.31:0/3262359947 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f884c012420 con 0x7f8854156300 2023-10-06T05:25:14.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.447+0000 7f883bfff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8840067680 0x7f8840069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:14.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.447+0000 7f883bfff700 1 -- 172.21.15.31:0/3262359947 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f884c085cc0 con 0x7f8854156300 2023-10-06T05:25:14.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.448+0000 7f8853fff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8840067680 0x7f8840069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:14.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.448+0000 7f8853fff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8840067680 0x7f8840069b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8844009340 tx=0x7f8844015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:14.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.451+0000 7f883bfff700 1 -- 172.21.15.31:0/3262359947 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f884c083030 con 0x7f8854156300 2023-10-06T05:25:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:14.657+0000 7f885a7ff700 1 -- 172.21.15.31:0/3262359947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f885404c380 con 0x7f8854156300 2023-10-06T05:25:15.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.172+0000 7f883bfff700 1 -- 172.21.15.31:0/3262359947 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v141) v1 ==== 181+0+0 (secure 0 0 0) 0x7f884c0160d0 con 0x7f8854156300 2023-10-06T05:25:15.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.177+0000 7f885a7ff700 1 -- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8840067680 msgr2=0x7f8840069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.177+0000 7f885a7ff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8840067680 0x7f8840069b30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8844009340 tx=0x7f8844015000 comp rx=0 tx=0).stop 2023-10-06T05:25:15.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.177+0000 7f885a7ff700 1 -- 172.21.15.31:0/3262359947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 msgr2=0x7f88540c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.177+0000 7f885a7ff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 0x7f88540c3b50 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f884c00c930 tx=0x7f884c0026e0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.178+0000 7f885a7ff700 1 -- 172.21.15.31:0/3262359947 shutdown_connections 2023-10-06T05:25:15.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.178+0000 7f885a7ff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8840067680 0x7f8840069b30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.178+0000 7f885a7ff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8854156300 0x7f88540c3b50 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.178+0000 7f885a7ff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f885406a630 0x7f88540aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.178+0000 7f885a7ff700 1 --2- 172.21.15.31:0/3262359947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f885406ea90 0x7f88540b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.178+0000 7f885a7ff700 1 -- 172.21.15.31:0/3262359947 >> 172.21.15.31:0/3262359947 conn(0x7f8854068030 msgr2=0x7f88540cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:15.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.178+0000 7f885a7ff700 1 -- 172.21.15.31:0/3262359947 shutdown_connections 2023-10-06T05:25:15.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.178+0000 7f885a7ff700 1 -- 172.21.15.31:0/3262359947 wait complete. 2023-10-06T05:25:15.182 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:25:15.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:25:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.403+0000 7f2c5f1ff700 1 Processor -- start 2023-10-06T05:25:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.403+0000 7f2c5f1ff700 1 -- start start 2023-10-06T05:25:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.404+0000 7f2c5f1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c58161140 0x7f2c58069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.404+0000 7f2c5f1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2c5815ed70 0x7f2c5806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.404+0000 7f2c5f1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5806aad0 con 0x7f2c5806d750 2023-10-06T05:25:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.404+0000 7f2c5f1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5806db90 con 0x7f2c58161140 2023-10-06T05:25:15.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.404+0000 7f2c5f1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c58069ec0 con 0x7f2c5815ed70 2023-10-06T05:25:15.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.405+0000 7f2c5cf9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c58161140 0x7f2c58069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.405+0000 7f2c5d79c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2c5806d750 0x7f2c581615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56072/0 (socket says 172.21.15.31:56072) 2023-10-06T05:25:15.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.405+0000 7f2c5d79c700 1 -- 172.21.15.31:0/753267512 learned_addr learned my addr 172.21.15.31:0/753267512 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:15.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.405+0000 7f2c57fff700 1 --2- 172.21.15.31:0/753267512 >> v2:172.21.15.62:3300/0 conn(0x7f2c5815ed70 0x7f2c5806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.405+0000 7f2c577fe700 1 -- 172.21.15.31:0/753267512 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177938514 0 0) 0x7f2c5806aad0 con 0x7f2c5806d750 2023-10-06T05:25:15.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.405+0000 7f2c5cf9b700 1 -- 172.21.15.31:0/753267512 >> v2:172.21.15.62:3300/0 conn(0x7f2c5815ed70 msgr2=0x7f2c5806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:15.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.405+0000 7f2c5cf9b700 1 --2- 172.21.15.31:0/753267512 >> v2:172.21.15.62:3300/0 conn(0x7f2c5815ed70 0x7f2c5806d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.405+0000 7f2c5cf9b700 1 -- 172.21.15.31:0/753267512 >> v1:172.21.15.31:6789/0 conn(0x7f2c5806d750 legacy=0x7f2c581615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.405+0000 7f2c5cf9b700 1 -- 172.21.15.31:0/753267512 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5806a1a0 con 0x7f2c58161140 2023-10-06T05:25:15.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.405+0000 7f2c5cf9b700 1 --2- 172.21.15.31:0/753267512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c58161140 0x7f2c58069520 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f2c5806b820 tx=0x7f2c4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f555742ba8d1a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:15.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c577fe700 1 -- 172.21.15.31:0/753267512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c011630 con 0x7f2c58161140 2023-10-06T05:25:15.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c577fe700 1 -- 172.21.15.31:0/753267512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c011790 con 0x7f2c58161140 2023-10-06T05:25:15.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c577fe700 1 -- 172.21.15.31:0/753267512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c011b30 con 0x7f2c58161140 2023-10-06T05:25:15.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/753267512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c58161140 msgr2=0x7f2c58069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c5f1ff700 1 --2- 172.21.15.31:0/753267512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c58161140 0x7f2c58069520 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f2c5806b820 tx=0x7f2c4c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/753267512 shutdown_connections 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c5f1ff700 1 --2- 172.21.15.31:0/753267512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c58161140 0x7f2c58069520 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c5f1ff700 1 --2- 172.21.15.31:0/753267512 >> v2:172.21.15.62:3300/0 conn(0x7f2c5815ed70 0x7f2c5806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/753267512 >> 172.21.15.31:0/753267512 conn(0x7f2c58068030 msgr2=0x7f2c580cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/753267512 shutdown_connections 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.406+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/753267512 wait complete. 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.408+0000 7f2c5f1ff700 1 Processor -- start 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.408+0000 7f2c5f1ff700 1 -- start start 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c5f1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c5806d750 0x7f2c5816f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c5f1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c5815ed70 0x7f2c580c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c5f1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c58161140 0x7f2c580bd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c5f1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c580bffb0 con 0x7f2c5806d750 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c5f1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c580bf9a0 con 0x7f2c58161140 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c5f1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c580bd770 con 0x7f2c5815ed70 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c5cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c5806d750 0x7f2c5816f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c5cf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c5806d750 0x7f2c5816f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38144/0 (socket says 172.21.15.31:38144) 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c5cf9b700 1 -- 172.21.15.31:0/3420083352 learned_addr learned my addr 172.21.15.31:0/3420083352 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c57fff700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c5815ed70 0x7f2c580c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.409+0000 7f2c5d79c700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c58161140 0x7f2c580bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c5cf9b700 1 -- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c5815ed70 msgr2=0x7f2c580c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c5cf9b700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c5815ed70 0x7f2c580c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c5cf9b700 1 -- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c58161140 msgr2=0x7f2c580bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c5cf9b700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c58161140 0x7f2c580bd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c5cf9b700 1 -- 172.21.15.31:0/3420083352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c0093c0 con 0x7f2c5806d750 2023-10-06T05:25:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c5d79c700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c58161140 0x7f2c580bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c5cf9b700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c5806d750 0x7f2c5816f400 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f2c580641a0 tx=0x7f2c4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c54ff9700 1 -- 172.21.15.31:0/3420083352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c011900 con 0x7f2c5806d750 2023-10-06T05:25:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c54ff9700 1 -- 172.21.15.31:0/3420083352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c011a60 con 0x7f2c5806d750 2023-10-06T05:25:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c54ff9700 1 -- 172.21.15.31:0/3420083352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c025670 con 0x7f2c5806d750 2023-10-06T05:25:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/3420083352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c580bfc40 con 0x7f2c5806d750 2023-10-06T05:25:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.410+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/3420083352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c58171140 con 0x7f2c5806d750 2023-10-06T05:25:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.411+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/3420083352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5804c380 con 0x7f2c5806d750 2023-10-06T05:25:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.412+0000 7f2c54ff9700 1 -- 172.21.15.31:0/3420083352 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f2c4c023070 con 0x7f2c5806d750 2023-10-06T05:25:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.413+0000 7f2c54ff9700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c3c067680 0x7f2c3c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.413+0000 7f2c54ff9700 1 -- 172.21.15.31:0/3420083352 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f2c4c086c60 con 0x7f2c5806d750 2023-10-06T05:25:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.414+0000 7f2c57fff700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c3c067680 0x7f2c3c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:15.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.414+0000 7f2c57fff700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c3c067680 0x7f2c3c069b30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2c580ca530 tx=0x7f2c44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:15.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.417+0000 7f2c54ff9700 1 -- 172.21.15.31:0/3420083352 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c4c084180 con 0x7f2c5806d750 2023-10-06T05:25:15.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:15.617+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/3420083352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c580ccf00 con 0x7f2c5806d750 2023-10-06T05:25:16.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.191+0000 7f2c54ff9700 1 -- 172.21.15.31:0/3420083352 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v142) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2c4c052f70 con 0x7f2c5806d750 2023-10-06T05:25:16.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.196+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c3c067680 msgr2=0x7f2c3c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.196+0000 7f2c5f1ff700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c3c067680 0x7f2c3c069b30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2c580ca530 tx=0x7f2c44008040 comp rx=0 tx=0).stop 2023-10-06T05:25:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.196+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/3420083352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c5806d750 msgr2=0x7f2c5816f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.196+0000 7f2c5f1ff700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c5806d750 0x7f2c5816f400 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f2c580641a0 tx=0x7f2c4c017000 comp rx=0 tx=0).stop 2023-10-06T05:25:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.197+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/3420083352 shutdown_connections 2023-10-06T05:25:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.197+0000 7f2c5f1ff700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c3c067680 0x7f2c3c069b30 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2c580ca530 tx=0x7f2c44008040 comp rx=0 tx=0).stop 2023-10-06T05:25:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.197+0000 7f2c5f1ff700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c58161140 0x7f2c580bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.197+0000 7f2c5f1ff700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c5806d750 0x7f2c5816f400 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.197+0000 7f2c5f1ff700 1 --2- 172.21.15.31:0/3420083352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c5815ed70 0x7f2c580c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.197+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/3420083352 >> 172.21.15.31:0/3420083352 conn(0x7f2c58068030 msgr2=0x7f2c5815d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:16.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.198+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/3420083352 shutdown_connections 2023-10-06T05:25:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.200+0000 7f2c5f1ff700 1 -- 172.21.15.31:0/3420083352 wait complete. 2023-10-06T05:25:16.204 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:25:16.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:25:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.423+0000 7f11541ff700 1 Processor -- start 2023-10-06T05:25:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.423+0000 7f11541ff700 1 -- start start 2023-10-06T05:25:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f11541ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f114c0ae860 0x7f114c157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f11541ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f114c0b1260 0x7f114c0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f11541ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114c158cf0 con 0x7f114c1573a0 2023-10-06T05:25:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f11541ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c0b1c40 con 0x7f114c0ae860 2023-10-06T05:25:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f11541ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c1580e0 con 0x7f114c0b1260 2023-10-06T05:25:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f115179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f114c0ae860 0x7f114c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f1151f9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f114c1573a0 0x7f114c0acb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56080/0 (socket says 172.21.15.31:56080) 2023-10-06T05:25:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f1151f9b700 1 -- 172.21.15.31:0/517087029 learned_addr learned my addr 172.21.15.31:0/517087029 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f115279c700 1 --2- 172.21.15.31:0/517087029 >> v2:172.21.15.62:3300/0 conn(0x7f114c0b1260 0x7f114c0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f115179a700 1 -- 172.21.15.31:0/517087029 >> v2:172.21.15.62:3300/0 conn(0x7f114c0b1260 msgr2=0x7f114c0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f115179a700 1 --2- 172.21.15.31:0/517087029 >> v2:172.21.15.62:3300/0 conn(0x7f114c0b1260 0x7f114c0b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f115179a700 1 -- 172.21.15.31:0/517087029 >> v1:172.21.15.31:6789/0 conn(0x7f114c1573a0 legacy=0x7f114c0acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f115179a700 1 -- 172.21.15.31:0/517087029 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c1583c0 con 0x7f114c0ae860 2023-10-06T05:25:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.425+0000 7f1150f99700 1 -- 172.21.15.31:0/517087029 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944752745 0 0) 0x7f114c158cf0 con 0x7f114c1573a0 2023-10-06T05:25:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.426+0000 7f115179a700 1 --2- 172.21.15.31:0/517087029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c0ae860 0x7f114c157770 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f114c0c93c0 tx=0x7f113c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ece3cc470e112 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.426+0000 7f1150f99700 1 -- 172.21.15.31:0/517087029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c00b6b0 con 0x7f114c0ae860 2023-10-06T05:25:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.426+0000 7f1150f99700 1 -- 172.21.15.31:0/517087029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c00b810 con 0x7f114c0ae860 2023-10-06T05:25:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.426+0000 7f1150f99700 1 -- 172.21.15.31:0/517087029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c00bbb0 con 0x7f114c0ae860 2023-10-06T05:25:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.426+0000 7f11541ff700 1 -- 172.21.15.31:0/517087029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c0ae860 msgr2=0x7f114c157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.426+0000 7f11541ff700 1 --2- 172.21.15.31:0/517087029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c0ae860 0x7f114c157770 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f114c0c93c0 tx=0x7f113c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.426+0000 7f11541ff700 1 -- 172.21.15.31:0/517087029 shutdown_connections 2023-10-06T05:25:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.426+0000 7f11541ff700 1 --2- 172.21.15.31:0/517087029 >> v2:172.21.15.62:3300/0 conn(0x7f114c0b1260 0x7f114c0b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.427+0000 7f11541ff700 1 --2- 172.21.15.31:0/517087029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c0ae860 0x7f114c157770 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.427+0000 7f11541ff700 1 -- 172.21.15.31:0/517087029 >> 172.21.15.31:0/517087029 conn(0x7f114c067560 msgr2=0x7f114c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.427+0000 7f11541ff700 1 -- 172.21.15.31:0/517087029 shutdown_connections 2023-10-06T05:25:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.427+0000 7f11541ff700 1 -- 172.21.15.31:0/517087029 wait complete. 2023-10-06T05:25:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.428+0000 7f11541ff700 1 Processor -- start 2023-10-06T05:25:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.428+0000 7f11541ff700 1 -- start start 2023-10-06T05:25:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.429+0000 7f11541ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f114c0b1260 0x7f114c0a7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.429+0000 7f11541ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c1573a0 0x7f114c167640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.429+0000 7f11541ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c1553e0 0x7f114c0a84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.429+0000 7f11541ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c168630 con 0x7f114c1573a0 2023-10-06T05:25:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.429+0000 7f11541ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c15c550 con 0x7f114c1553e0 2023-10-06T05:25:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.429+0000 7f11541ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c168070 con 0x7f114c0b1260 2023-10-06T05:25:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.429+0000 7f115179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c1573a0 0x7f114c167640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f115179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c1573a0 0x7f114c167640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38164/0 (socket says 172.21.15.31:38164) 2023-10-06T05:25:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f115179a700 1 -- 172.21.15.31:0/1882187295 learned_addr learned my addr 172.21.15.31:0/1882187295 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f1151f9b700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f114c0b1260 0x7f114c0a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f115279c700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c1553e0 0x7f114c0a84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f115179a700 1 -- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f114c0b1260 msgr2=0x7f114c0a7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f115179a700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f114c0b1260 0x7f114c0a7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f115179a700 1 -- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c1553e0 msgr2=0x7f114c0a84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f115179a700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c1553e0 0x7f114c0a84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f115179a700 1 -- 172.21.15.31:0/1882187295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c009420 con 0x7f114c1573a0 2023-10-06T05:25:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f115279c700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c1553e0 0x7f114c0a84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f115179a700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c1573a0 0x7f114c167640 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f114c064290 tx=0x7f113c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.430+0000 7f1141ffb700 1 -- 172.21.15.31:0/1882187295 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c011420 con 0x7f114c1573a0 2023-10-06T05:25:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.431+0000 7f1141ffb700 1 -- 172.21.15.31:0/1882187295 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c011580 con 0x7f114c1573a0 2023-10-06T05:25:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.431+0000 7f1141ffb700 1 -- 172.21.15.31:0/1882187295 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c011940 con 0x7f114c1573a0 2023-10-06T05:25:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.431+0000 7f11541ff700 1 -- 172.21.15.31:0/1882187295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c168350 con 0x7f114c1573a0 2023-10-06T05:25:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.431+0000 7f11541ff700 1 -- 172.21.15.31:0/1882187295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c0a7c80 con 0x7f114c1573a0 2023-10-06T05:25:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.432+0000 7f11541ff700 1 -- 172.21.15.31:0/1882187295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c04c380 con 0x7f114c1573a0 2023-10-06T05:25:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.433+0000 7f1141ffb700 1 -- 172.21.15.31:0/1882187295 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f113c011d30 con 0x7f114c1573a0 2023-10-06T05:25:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.434+0000 7f1141ffb700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1130067680 0x7f1130069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.434+0000 7f1141ffb700 1 -- 172.21.15.31:0/1882187295 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f113c086ab0 con 0x7f114c1573a0 2023-10-06T05:25:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.434+0000 7f1151f9b700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1130067680 0x7f1130069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.435+0000 7f1151f9b700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1130067680 0x7f1130069b30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f114c0c8960 tx=0x7f1148009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:16.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.438+0000 7f1141ffb700 1 -- 172.21.15.31:0/1882187295 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f113c084170 con 0x7f114c1573a0 2023-10-06T05:25:16.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.641+0000 7f11541ff700 1 -- 172.21.15.31:0/1882187295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f114c0cc380 con 0x7f114c1573a0 2023-10-06T05:25:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.645+0000 7f1141ffb700 1 -- 172.21.15.31:0/1882187295 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f113c052f60 con 0x7f114c1573a0 2023-10-06T05:25:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 -- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1130067680 msgr2=0x7f1130069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1130067680 0x7f1130069b30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f114c0c8960 tx=0x7f1148009040 comp rx=0 tx=0).stop 2023-10-06T05:25:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 -- 172.21.15.31:0/1882187295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c1573a0 msgr2=0x7f114c167640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c1573a0 0x7f114c167640 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f114c064290 tx=0x7f113c017000 comp rx=0 tx=0).stop 2023-10-06T05:25:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 -- 172.21.15.31:0/1882187295 shutdown_connections 2023-10-06T05:25:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1130067680 0x7f1130069b30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f114c1553e0 0x7f114c0a84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f114c0b1260 0x7f114c0a7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 --2- 172.21.15.31:0/1882187295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c1573a0 0x7f114c167640 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 -- 172.21.15.31:0/1882187295 >> 172.21.15.31:0/1882187295 conn(0x7f114c067560 msgr2=0x7f114c0cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:16.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 -- 172.21.15.31:0/1882187295 shutdown_connections 2023-10-06T05:25:16.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.647+0000 7f11541ff700 1 -- 172.21.15.31:0/1882187295 wait complete. 2023-10-06T05:25:16.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2023-10-06T05:25:16.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.860+0000 7fbfc35ff700 1 Processor -- start 2023-10-06T05:25:16.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.860+0000 7fbfc35ff700 1 -- start start 2023-10-06T05:25:16.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc35ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbfbc0c7b60 0x7fbfbc069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc35ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbfbc0c7f30 0x7fbfbc0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc35ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfbc0ca5b0 con 0x7fbfbc06ef50 2023-10-06T05:25:16.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc35ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc06f740 con 0x7fbfbc0c7f30 2023-10-06T05:25:16.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc35ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc0c9c80 con 0x7fbfbc0c7b60 2023-10-06T05:25:16.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc0b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbfbc0c7f30 0x7fbfbc0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc0b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbfbc0c7f30 0x7fbfbc0aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38166/0 (socket says 172.21.15.31:38166) 2023-10-06T05:25:16.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc0b9a700 1 -- 172.21.15.31:0/324714938 learned_addr learned my addr 172.21.15.31:0/324714938 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:16.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc139b700 1 --2- 172.21.15.31:0/324714938 >> v2:172.21.15.62:3300/0 conn(0x7fbfbc0c7b60 0x7fbfbc069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc0b9a700 1 -- 172.21.15.31:0/324714938 >> v2:172.21.15.62:3300/0 conn(0x7fbfbc0c7b60 msgr2=0x7fbfbc069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:16.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc0b9a700 1 --2- 172.21.15.31:0/324714938 >> v2:172.21.15.62:3300/0 conn(0x7fbfbc0c7b60 0x7fbfbc069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc0b9a700 1 -- 172.21.15.31:0/324714938 >> v1:172.21.15.62:6789/0 conn(0x7fbfbc06ef50 legacy=0x7fbfbc13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.862+0000 7fbfc0b9a700 1 -- 172.21.15.31:0/324714938 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0c9f60 con 0x7fbfbc0c7f30 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.863+0000 7fbfc0b9a700 1 --2- 172.21.15.31:0/324714938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc0c7f30 0x7fbfbc0aee70 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0099b0 tx=0x7fbfac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75577a772ef64da7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.863+0000 7fbfb3fff700 1 -- 172.21.15.31:0/324714938 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00b820 con 0x7fbfbc0c7f30 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.863+0000 7fbfb3fff700 1 -- 172.21.15.31:0/324714938 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac00b980 con 0x7fbfbc0c7f30 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.863+0000 7fbfb3fff700 1 -- 172.21.15.31:0/324714938 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00bd20 con 0x7fbfbc0c7f30 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.864+0000 7fbfc35ff700 1 -- 172.21.15.31:0/324714938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc0c7f30 msgr2=0x7fbfbc0aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.864+0000 7fbfc35ff700 1 --2- 172.21.15.31:0/324714938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc0c7f30 0x7fbfbc0aee70 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0099b0 tx=0x7fbfac009250 comp rx=0 tx=0).stop 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.864+0000 7fbfc35ff700 1 -- 172.21.15.31:0/324714938 shutdown_connections 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.864+0000 7fbfc35ff700 1 --2- 172.21.15.31:0/324714938 >> v2:172.21.15.62:3300/0 conn(0x7fbfbc0c7b60 0x7fbfbc069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.864+0000 7fbfc35ff700 1 --2- 172.21.15.31:0/324714938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc0c7f30 0x7fbfbc0aee70 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.864+0000 7fbfc35ff700 1 -- 172.21.15.31:0/324714938 >> 172.21.15.31:0/324714938 conn(0x7fbfbc0cc0b0 msgr2=0x7fbfbc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.864+0000 7fbfc35ff700 1 -- 172.21.15.31:0/324714938 shutdown_connections 2023-10-06T05:25:16.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.864+0000 7fbfc35ff700 1 -- 172.21.15.31:0/324714938 wait complete. 2023-10-06T05:25:16.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.865+0000 7fbfc35ff700 1 Processor -- start 2023-10-06T05:25:16.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.865+0000 7fbfc35ff700 1 -- start start 2023-10-06T05:25:16.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.866+0000 7fbfc35ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc06ef50 0x7fbfbc13e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.866+0000 7fbfc35ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbfbc0c7b60 0x7fbfbc0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.866+0000 7fbfc35ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfbc0c7f30 0x7fbfbc0c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.866+0000 7fbfc35ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc16a7b0 con 0x7fbfbc06ef50 2023-10-06T05:25:16.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.866+0000 7fbfc35ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0ae4f0 con 0x7fbfbc0c7f30 2023-10-06T05:25:16.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.866+0000 7fbfc35ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0adb30 con 0x7fbfbc0c7b60 2023-10-06T05:25:16.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc06ef50 0x7fbfbc13e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc06ef50 0x7fbfbc13e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38174/0 (socket says 172.21.15.31:38174) 2023-10-06T05:25:16.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc139b700 1 -- 172.21.15.31:0/1226220966 learned_addr learned my addr 172.21.15.31:0/1226220966 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:16.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc0b9a700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbfbc0c7b60 0x7fbfbc0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc1b9c700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfbc0c7f30 0x7fbfbc0c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc139b700 1 -- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbfbc0c7b60 msgr2=0x7fbfbc0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc139b700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbfbc0c7b60 0x7fbfbc0adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc139b700 1 -- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfbc0c7f30 msgr2=0x7fbfbc0c17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:16.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc139b700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfbc0c7f30 0x7fbfbc0c17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:16.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc139b700 1 -- 172.21.15.31:0/1226220966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac0094f0 con 0x7fbfbc06ef50 2023-10-06T05:25:16.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc1b9c700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfbc0c7f30 0x7fbfbc0c17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:16.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.867+0000 7fbfc139b700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc06ef50 0x7fbfbc13e050 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800d8c0 tx=0x7fbfb8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:16.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.868+0000 7fbfb17fa700 1 -- 172.21.15.31:0/1226220966 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8005a00 con 0x7fbfbc06ef50 2023-10-06T05:25:16.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.868+0000 7fbfb17fa700 1 -- 172.21.15.31:0/1226220966 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb8005ba0 con 0x7fbfbc06ef50 2023-10-06T05:25:16.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.868+0000 7fbfb17fa700 1 -- 172.21.15.31:0/1226220966 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8006030 con 0x7fbfbc06ef50 2023-10-06T05:25:16.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.868+0000 7fbfc35ff700 1 -- 172.21.15.31:0/1226220966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc169e80 con 0x7fbfbc06ef50 2023-10-06T05:25:16.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.868+0000 7fbfc35ff700 1 -- 172.21.15.31:0/1226220966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc16a440 con 0x7fbfbc06ef50 2023-10-06T05:25:16.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.869+0000 7fbfc35ff700 1 -- 172.21.15.31:0/1226220966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc04c380 con 0x7fbfbc06ef50 2023-10-06T05:25:16.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.870+0000 7fbfb17fa700 1 -- 172.21.15.31:0/1226220966 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fbfb80063a0 con 0x7fbfbc06ef50 2023-10-06T05:25:16.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.870+0000 7fbfb17fa700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbfa0067680 0x7fbfa0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:16.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.871+0000 7fbfb17fa700 1 -- 172.21.15.31:0/1226220966 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fbfb80858a0 con 0x7fbfbc06ef50 2023-10-06T05:25:16.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.871+0000 7fbfc0b9a700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbfa0067680 0x7fbfa0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:16.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.872+0000 7fbfc0b9a700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbfa0067680 0x7fbfa0069b30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0099b0 tx=0x7fbfac017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:16.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:16.875+0000 7fbfb17fa700 1 -- 172.21.15.31:0/1226220966 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfb8082f60 con 0x7fbfbc06ef50 2023-10-06T05:25:17.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.175+0000 7fbfc35ff700 1 -- 172.21.15.31:0/1226220966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) v1 -- 0x7fbfbc15fa30 con 0x7fbfbc06ef50 2023-10-06T05:25:17.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.191+0000 7fbfb17fa700 1 -- 172.21.15.31:0/1226220966 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v450) v1 ==== 149+0+0 (secure 0 0 0) 0x7fbfb8051d50 con 0x7fbfbc06ef50 2023-10-06T05:25:17.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 -- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbfa0067680 msgr2=0x7fbfa0069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbfa0067680 0x7fbfa0069b30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0099b0 tx=0x7fbfac017000 comp rx=0 tx=0).stop 2023-10-06T05:25:17.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 -- 172.21.15.31:0/1226220966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc06ef50 msgr2=0x7fbfbc13e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc06ef50 0x7fbfbc13e050 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800d8c0 tx=0x7fbfb8008040 comp rx=0 tx=0).stop 2023-10-06T05:25:17.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 -- 172.21.15.31:0/1226220966 shutdown_connections 2023-10-06T05:25:17.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbfa0067680 0x7fbfa0069b30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfbc0c7f30 0x7fbfbc0c17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc06ef50 0x7fbfbc13e050 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 --2- 172.21.15.31:0/1226220966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbfbc0c7b60 0x7fbfbc0adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 -- 172.21.15.31:0/1226220966 >> 172.21.15.31:0/1226220966 conn(0x7fbfbc0cc0b0 msgr2=0x7fbfbc15f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:17.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 -- 172.21.15.31:0/1226220966 shutdown_connections 2023-10-06T05:25:17.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.193+0000 7fbfc35ff700 1 -- 172.21.15.31:0/1226220966 wait complete. 2023-10-06T05:25:17.213 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2023-10-06T05:25:17.213 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2023-10-06T05:25:17.213 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:25:17.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2023-10-06T05:25:17.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.407+0000 7fc71c9ff700 1 Processor -- start 2023-10-06T05:25:17.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.407+0000 7fc71c9ff700 1 -- start start 2023-10-06T05:25:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.409+0000 7fc71c9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc7181595a0 0x7fc718159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.409+0000 7fc71c9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7180cc0d0 0x7fc718068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.409+0000 7fc71c9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc718069010 con 0x7fc718068bb0 2023-10-06T05:25:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.409+0000 7fc71c9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7180c95b0 con 0x7fc7180cc0d0 2023-10-06T05:25:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.409+0000 7fc71c9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7180c9890 con 0x7fc7181595a0 2023-10-06T05:25:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.409+0000 7fc715d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7180cc0d0 0x7fc718068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.409+0000 7fc715d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7180cc0d0 0x7fc718068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50658/0 (socket says 172.21.15.31:50658) 2023-10-06T05:25:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.409+0000 7fc715d9b700 1 -- 172.21.15.31:0/3982367378 learned_addr learned my addr 172.21.15.31:0/3982367378 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.409+0000 7fc71659c700 1 --2- 172.21.15.31:0/3982367378 >> v2:172.21.15.62:3300/0 conn(0x7fc7181595a0 0x7fc718159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.410+0000 7fc715d9b700 1 -- 172.21.15.31:0/3982367378 >> v2:172.21.15.62:3300/0 conn(0x7fc7181595a0 msgr2=0x7fc718159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.410+0000 7fc715d9b700 1 --2- 172.21.15.31:0/3982367378 >> v2:172.21.15.62:3300/0 conn(0x7fc7181595a0 0x7fc718159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.410+0000 7fc715d9b700 1 -- 172.21.15.31:0/3982367378 >> v1:172.21.15.62:6789/0 conn(0x7fc718068bb0 legacy=0x7fc7180c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.410+0000 7fc715d9b700 1 -- 172.21.15.31:0/3982367378 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7180c9b70 con 0x7fc7180cc0d0 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.410+0000 7fc715d9b700 1 --2- 172.21.15.31:0/3982367378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 0x7fc718068670 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc71806c8d0 tx=0x7fc70c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188fc9852dafebf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.410+0000 7fc71559a700 1 -- 172.21.15.31:0/3982367378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00b860 con 0x7fc7180cc0d0 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.410+0000 7fc71559a700 1 -- 172.21.15.31:0/3982367378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c00b9c0 con 0x7fc7180cc0d0 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.410+0000 7fc71559a700 1 -- 172.21.15.31:0/3982367378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00bd60 con 0x7fc7180cc0d0 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.411+0000 7fc71c9ff700 1 -- 172.21.15.31:0/3982367378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 msgr2=0x7fc718068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.411+0000 7fc71c9ff700 1 --2- 172.21.15.31:0/3982367378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 0x7fc718068670 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc71806c8d0 tx=0x7fc70c009250 comp rx=0 tx=0).stop 2023-10-06T05:25:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.411+0000 7fc71c9ff700 1 -- 172.21.15.31:0/3982367378 shutdown_connections 2023-10-06T05:25:17.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.411+0000 7fc71c9ff700 1 --2- 172.21.15.31:0/3982367378 >> v2:172.21.15.62:3300/0 conn(0x7fc7181595a0 0x7fc718159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.411+0000 7fc71c9ff700 1 --2- 172.21.15.31:0/3982367378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 0x7fc718068670 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.411+0000 7fc71c9ff700 1 -- 172.21.15.31:0/3982367378 >> 172.21.15.31:0/3982367378 conn(0x7fc71806b620 msgr2=0x7fc71806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:17.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.411+0000 7fc71c9ff700 1 -- 172.21.15.31:0/3982367378 shutdown_connections 2023-10-06T05:25:17.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.411+0000 7fc71c9ff700 1 -- 172.21.15.31:0/3982367378 wait complete. 2023-10-06T05:25:17.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.412+0000 7fc71c9ff700 1 Processor -- start 2023-10-06T05:25:17.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.413+0000 7fc71c9ff700 1 -- start start 2023-10-06T05:25:17.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.413+0000 7fc71c9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc718068bb0 0x7fc71816a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.413+0000 7fc71c9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 0x7fc718150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc71c9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7181595a0 0x7fc7180c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc71c9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181558c0 con 0x7fc7180cc0d0 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc71c9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc718157530 con 0x7fc7181595a0 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc71c9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc718157810 con 0x7fc718068bb0 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc715d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 0x7fc718150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc715d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 0x7fc718150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50672/0 (socket says 172.21.15.31:50672) 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc715d9b700 1 -- 172.21.15.31:0/1568217957 learned_addr learned my addr 172.21.15.31:0/1568217957 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc71659c700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc718068bb0 0x7fc71816a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc716d9d700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7181595a0 0x7fc7180c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc715d9b700 1 -- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc718068bb0 msgr2=0x7fc71816a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc715d9b700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc718068bb0 0x7fc71816a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc715d9b700 1 -- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7181595a0 msgr2=0x7fc7180c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:17.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc715d9b700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7181595a0 0x7fc7180c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:17.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc715d9b700 1 -- 172.21.15.31:0/1568217957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c0094f0 con 0x7fc7180cc0d0 2023-10-06T05:25:17.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc71659c700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc718068bb0 0x7fc71816a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:17.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc716d9d700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7181595a0 0x7fc7180c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:17.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.414+0000 7fc715d9b700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 0x7fc718150c10 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc71806ee00 tx=0x7fc70c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.415+0000 7fc7067fc700 1 -- 172.21.15.31:0/1568217957 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c011e60 con 0x7fc7180cc0d0 2023-10-06T05:25:17.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.415+0000 7fc7067fc700 1 -- 172.21.15.31:0/1568217957 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c01a630 con 0x7fc7180cc0d0 2023-10-06T05:25:17.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.415+0000 7fc7067fc700 1 -- 172.21.15.31:0/1568217957 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c01a9f0 con 0x7fc7180cc0d0 2023-10-06T05:25:17.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.415+0000 7fc71c9ff700 1 -- 172.21.15.31:0/1568217957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc718155550 con 0x7fc7180cc0d0 2023-10-06T05:25:17.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.415+0000 7fc71c9ff700 1 -- 172.21.15.31:0/1568217957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc718151150 con 0x7fc7180cc0d0 2023-10-06T05:25:17.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.416+0000 7fc71c9ff700 1 -- 172.21.15.31:0/1568217957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc718155a30 con 0x7fc7180cc0d0 2023-10-06T05:25:17.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.417+0000 7fc7067fc700 1 -- 172.21.15.31:0/1568217957 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fc70c02d440 con 0x7fc7180cc0d0 2023-10-06T05:25:17.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.418+0000 7fc7067fc700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6f8067680 0x7fc6f8069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:17.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.418+0000 7fc7067fc700 1 -- 172.21.15.31:0/1568217957 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fc70c086da0 con 0x7fc7180cc0d0 2023-10-06T05:25:17.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.418+0000 7fc71659c700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6f8067680 0x7fc6f8069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:17.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.419+0000 7fc71659c700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6f8067680 0x7fc6f8069b30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc708006fd0 tx=0x7fc708008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:17.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.422+0000 7fc7067fc700 1 -- 172.21.15.31:0/1568217957 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc70c084870 con 0x7fc7180cc0d0 2023-10-06T05:25:17.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:17.703+0000 7fc71c9ff700 1 -- 172.21.15.31:0/1568217957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) v1 -- 0x7fc7181556e0 con 0x7fc7180cc0d0 2023-10-06T05:25:18.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.199+0000 7fc7067fc700 1 -- 172.21.15.31:0/1568217957 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v451) v1 ==== 148+0+0 (secure 0 0 0) 0x7fc70c053660 con 0x7fc7180cc0d0 2023-10-06T05:25:18.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 -- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6f8067680 msgr2=0x7fc6f8069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6f8067680 0x7fc6f8069b30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc708006fd0 tx=0x7fc708008040 comp rx=0 tx=0).stop 2023-10-06T05:25:18.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 -- 172.21.15.31:0/1568217957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 msgr2=0x7fc718150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 0x7fc718150c10 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc71806ee00 tx=0x7fc70c017000 comp rx=0 tx=0).stop 2023-10-06T05:25:18.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 -- 172.21.15.31:0/1568217957 shutdown_connections 2023-10-06T05:25:18.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6f8067680 0x7fc6f8069b30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7181595a0 0x7fc7180c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc718068bb0 0x7fc71816a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 --2- 172.21.15.31:0/1568217957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7180cc0d0 0x7fc718150c10 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 -- 172.21.15.31:0/1568217957 >> 172.21.15.31:0/1568217957 conn(0x7fc71806b620 msgr2=0x7fc718158000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:18.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 -- 172.21.15.31:0/1568217957 shutdown_connections 2023-10-06T05:25:18.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.202+0000 7fc71c9ff700 1 -- 172.21.15.31:0/1568217957 wait complete. 2023-10-06T05:25:18.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:25:18.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.425+0000 7f5b719ff700 1 Processor -- start 2023-10-06T05:25:18.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.425+0000 7f5b719ff700 1 -- start start 2023-10-06T05:25:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.427+0000 7f5b719ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b6c0b2820 0x7f5b6c0c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.427+0000 7f5b719ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5b6c0b2bf0 0x7f5b6c0cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.427+0000 7f5b719ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6c0cbae0 con 0x7f5b6c0cef50 2023-10-06T05:25:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.427+0000 7f5b719ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b6c0cb280 con 0x7f5b6c0b2bf0 2023-10-06T05:25:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.427+0000 7f5b719ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c0cf3f0 con 0x7f5b6c0b2820 2023-10-06T05:25:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.427+0000 7f5b6a7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5b6c0b2bf0 0x7f5b6c0cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.427+0000 7f5b6b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5b6c0cef50 0x7f5b6c0cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36754/0 (socket says 172.21.15.31:36754) 2023-10-06T05:25:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.427+0000 7f5b6b7fe700 1 -- 172.21.15.31:0/2196414809 learned_addr learned my addr 172.21.15.31:0/2196414809 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.427+0000 7f5b6affd700 1 --2- 172.21.15.31:0/2196414809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b6c0b2820 0x7f5b6c0c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.427+0000 7f5b69ffb700 1 -- 172.21.15.31:0/2196414809 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26639431 0 0) 0x7f5b6c0cbae0 con 0x7f5b6c0cef50 2023-10-06T05:25:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.428+0000 7f5b69ffb700 1 -- 172.21.15.31:0/2196414809 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b54003570 con 0x7f5b6c0cef50 2023-10-06T05:25:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.428+0000 7f5b6a7fc700 1 -- 172.21.15.31:0/2196414809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b6c0b2820 msgr2=0x7f5b6c0c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.428+0000 7f5b6a7fc700 1 --2- 172.21.15.31:0/2196414809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b6c0b2820 0x7f5b6c0c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.428+0000 7f5b6a7fc700 1 -- 172.21.15.31:0/2196414809 >> v1:172.21.15.31:6789/0 conn(0x7f5b6c0cef50 legacy=0x7f5b6c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.428+0000 7f5b6a7fc700 1 -- 172.21.15.31:0/2196414809 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c0cb490 con 0x7f5b6c0b2bf0 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.428+0000 7f5b6a7fc700 1 --2- 172.21.15.31:0/2196414809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0b2bf0 0x7f5b6c0cea10 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f5b600098e0 tx=0x7f5b600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5e62425f91c3a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.428+0000 7f5b69ffb700 1 -- 172.21.15.31:0/2196414809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60011630 con 0x7f5b6c0b2bf0 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.428+0000 7f5b69ffb700 1 -- 172.21.15.31:0/2196414809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b60011790 con 0x7f5b6c0b2bf0 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.428+0000 7f5b69ffb700 1 -- 172.21.15.31:0/2196414809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60011b30 con 0x7f5b6c0b2bf0 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.429+0000 7f5b719ff700 1 -- 172.21.15.31:0/2196414809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0b2bf0 msgr2=0x7f5b6c0cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.429+0000 7f5b719ff700 1 --2- 172.21.15.31:0/2196414809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0b2bf0 0x7f5b6c0cea10 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f5b600098e0 tx=0x7f5b600091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.429+0000 7f5b719ff700 1 -- 172.21.15.31:0/2196414809 shutdown_connections 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.429+0000 7f5b719ff700 1 --2- 172.21.15.31:0/2196414809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b6c0b2820 0x7f5b6c0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.429+0000 7f5b719ff700 1 --2- 172.21.15.31:0/2196414809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0b2bf0 0x7f5b6c0cea10 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.429+0000 7f5b719ff700 1 -- 172.21.15.31:0/2196414809 >> 172.21.15.31:0/2196414809 conn(0x7f5b6c0996f0 msgr2=0x7f5b6c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:18.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.429+0000 7f5b719ff700 1 -- 172.21.15.31:0/2196414809 shutdown_connections 2023-10-06T05:25:18.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.429+0000 7f5b719ff700 1 -- 172.21.15.31:0/2196414809 wait complete. 2023-10-06T05:25:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.430+0000 7f5b719ff700 1 Processor -- start 2023-10-06T05:25:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.431+0000 7f5b719ff700 1 -- start start 2023-10-06T05:25:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.431+0000 7f5b719ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b6c0b2820 0x7f5b6c16a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.431+0000 7f5b719ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6c0b2bf0 0x7f5b6c0697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.431+0000 7f5b719ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0cef50 0x7f5b6c147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.431+0000 7f5b719ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c16a7e0 con 0x7f5b6c0cef50 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.431+0000 7f5b719ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c09ae00 con 0x7f5b6c0b2bf0 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.431+0000 7f5b719ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c09ce90 con 0x7f5b6c0b2820 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0cef50 0x7f5b6c147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0cef50 0x7f5b6c147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50690/0 (socket says 172.21.15.31:50690) 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6b7fe700 1 -- 172.21.15.31:0/202523896 learned_addr learned my addr 172.21.15.31:0/202523896 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6affd700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b6c0b2820 0x7f5b6c16a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6a7fc700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6c0b2bf0 0x7f5b6c0697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6b7fe700 1 -- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b6c0b2820 msgr2=0x7f5b6c16a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6b7fe700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b6c0b2820 0x7f5b6c16a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6b7fe700 1 -- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6c0b2bf0 msgr2=0x7f5b6c0697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6b7fe700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6c0b2bf0 0x7f5b6c0697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6b7fe700 1 -- 172.21.15.31:0/202523896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b60009420 con 0x7f5b6c0cef50 2023-10-06T05:25:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.432+0000 7f5b6a7fc700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6c0b2bf0 0x7f5b6c0697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.433+0000 7f5b6b7fe700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0cef50 0x7f5b6c147c70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f5b64001460 tx=0x7f5b640019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.433+0000 7f5b52ffd700 1 -- 172.21.15.31:0/202523896 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400a880 con 0x7f5b6c0cef50 2023-10-06T05:25:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.433+0000 7f5b52ffd700 1 -- 172.21.15.31:0/202523896 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6400a9e0 con 0x7f5b6c0cef50 2023-10-06T05:25:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.433+0000 7f5b52ffd700 1 -- 172.21.15.31:0/202523896 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400ad60 con 0x7f5b6c0cef50 2023-10-06T05:25:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.433+0000 7f5b719ff700 1 -- 172.21.15.31:0/202523896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6c09d170 con 0x7f5b6c0cef50 2023-10-06T05:25:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.433+0000 7f5b719ff700 1 -- 172.21.15.31:0/202523896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6c09d730 con 0x7f5b6c0cef50 2023-10-06T05:25:18.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.434+0000 7f5b719ff700 1 -- 172.21.15.31:0/202523896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6c04c380 con 0x7f5b6c0cef50 2023-10-06T05:25:18.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.435+0000 7f5b52ffd700 1 -- 172.21.15.31:0/202523896 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f5b64007300 con 0x7f5b6c0cef50 2023-10-06T05:25:18.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.436+0000 7f5b52ffd700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b58067680 0x7f5b58069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:18.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.436+0000 7f5b52ffd700 1 -- 172.21.15.31:0/202523896 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f5b64083200 con 0x7f5b6c0cef50 2023-10-06T05:25:18.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.437+0000 7f5b6affd700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b58067680 0x7f5b58069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:18.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.437+0000 7f5b6affd700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b58067680 0x7f5b58069b30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c156330 tx=0x7f5b5c009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:18.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.440+0000 7f5b52ffd700 1 -- 172.21.15.31:0/202523896 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b64082f10 con 0x7f5b6c0cef50 2023-10-06T05:25:18.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.641+0000 7f5b719ff700 1 -- 172.21.15.31:0/202523896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5b6c0cb770 con 0x7f5b6c0cef50 2023-10-06T05:25:18.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.645+0000 7f5b52ffd700 1 -- 172.21.15.31:0/202523896 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5b64051d80 con 0x7f5b6c0cef50 2023-10-06T05:25:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 -- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b58067680 msgr2=0x7f5b58069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b58067680 0x7f5b58069b30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c156330 tx=0x7f5b5c009120 comp rx=0 tx=0).stop 2023-10-06T05:25:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 -- 172.21.15.31:0/202523896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0cef50 msgr2=0x7f5b6c147c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:18.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0cef50 0x7f5b6c147c70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f5b64001460 tx=0x7f5b640019e0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 -- 172.21.15.31:0/202523896 shutdown_connections 2023-10-06T05:25:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5b58067680 0x7f5b58069b30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b6c0cef50 0x7f5b6c147c70 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b6c0b2820 0x7f5b6c16a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 --2- 172.21.15.31:0/202523896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6c0b2bf0 0x7f5b6c0697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 -- 172.21.15.31:0/202523896 >> 172.21.15.31:0/202523896 conn(0x7f5b6c0996f0 msgr2=0x7f5b6c0cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 -- 172.21.15.31:0/202523896 shutdown_connections 2023-10-06T05:25:18.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:18.647+0000 7f5b719ff700 1 -- 172.21.15.31:0/202523896 wait complete. 2023-10-06T05:25:18.667 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:25:18.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:25:18.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:25:18.883 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:25:18.883 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:25:18.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.077+0000 7f2c85fff700 1 Processor -- start 2023-10-06T05:25:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.077+0000 7f2c85fff700 1 -- start start 2023-10-06T05:25:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.078+0000 7f2c85fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2c801595a0 0x7f2c80159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.078+0000 7f2c85fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c800c95b0 con 0x7f2c80068d80 2023-10-06T05:25:19.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.078+0000 7f2c85fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c800c9890 con 0x7f2c800cc0d0 2023-10-06T05:25:19.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.078+0000 7f2c85fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c800c9b70 con 0x7f2c801595a0 2023-10-06T05:25:19.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2c80068d80 0x7f2c800c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36756/0 (socket says 172.21.15.31:36756) 2023-10-06T05:25:19.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7ffff700 1 -- 172.21.15.31:0/3536524116 learned_addr learned my addr 172.21.15.31:0/3536524116 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:19.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7f7fe700 1 --2- 172.21.15.31:0/3536524116 >> v2:172.21.15.62:3300/0 conn(0x7f2c801595a0 0x7f2c80159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197670985 0 0) 0x7f2c800c95b0 con 0x7f2c80068d80 2023-10-06T05:25:19.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c68003570 con 0x7f2c80068d80 2023-10-06T05:25:19.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282817646 0 0) 0x7f2c800c9890 con 0x7f2c800cc0d0 2023-10-06T05:25:19.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c800c95b0 con 0x7f2c800cc0d0 2023-10-06T05:25:19.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 791674244 0 0) 0x7f2c68003570 con 0x7f2c80068d80 2023-10-06T05:25:19.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 >> v2:172.21.15.62:3300/0 conn(0x7f2c801595a0 msgr2=0x7f2c80159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:19.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7e7fc700 1 --2- 172.21.15.31:0/3536524116 >> v2:172.21.15.62:3300/0 conn(0x7f2c801595a0 0x7f2c80159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 >> v1:172.21.15.62:6789/0 conn(0x7f2c800cc0d0 legacy=0x7f2c80068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.079+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800ca130 con 0x7f2c80068d80 2023-10-06T05:25:19.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f2c780017f0 con 0x7f2c80068d80 2023-10-06T05:25:19.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 >> v1:172.21.15.31:6789/0 conn(0x7f2c80068d80 legacy=0x7f2c800c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7e7fc700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c68003bf0 0x7f2c68003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7e7fc700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7e7fc700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c680051c0 0x7f2c6800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800c9b70 con 0x7f2c68004810 2023-10-06T05:25:19.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6800c0b0 con 0x7f2c68003bf0 2023-10-06T05:25:19.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6800c270 con 0x7f2c680051c0 2023-10-06T05:25:19.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7effd700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7f7fe700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c68003bf0 0x7f2c68003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7ffff700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c680051c0 0x7f2c6800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7effd700 1 -- 172.21.15.31:0/3536524116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c680051c0 msgr2=0x7f2c6800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7effd700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c680051c0 0x7f2c6800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7effd700 1 -- 172.21.15.31:0/3536524116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c68003bf0 msgr2=0x7f2c68003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7effd700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c68003bf0 0x7f2c68003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.080+0000 7f2c7effd700 1 -- 172.21.15.31:0/3536524116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c68003570 con 0x7f2c68004810 2023-10-06T05:25:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.081+0000 7f2c7ffff700 1 -- 172.21.15.31:0/3536524116 reap_dead start 2023-10-06T05:25:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.081+0000 7f2c7effd700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c68004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f2c74001a10 tx=0x7f2c74007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d565cf18caaea2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.081+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400c5d0 con 0x7f2c68004810 2023-10-06T05:25:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.081+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7400c730 con 0x7f2c68004810 2023-10-06T05:25:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.081+0000 7f2c7e7fc700 1 -- 172.21.15.31:0/3536524116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400cad0 con 0x7f2c68004810 2023-10-06T05:25:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.081+0000 7f2c85fff700 1 -- 172.21.15.31:0/3536524116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 msgr2=0x7f2c68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.081+0000 7f2c85fff700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c68004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f2c74001a10 tx=0x7f2c74007d30 comp rx=0 tx=0).stop 2023-10-06T05:25:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.082+0000 7f2c85fff700 1 -- 172.21.15.31:0/3536524116 shutdown_connections 2023-10-06T05:25:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.082+0000 7f2c85fff700 1 --2- 172.21.15.31:0/3536524116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c68004c80 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.082+0000 7f2c85fff700 1 -- 172.21.15.31:0/3536524116 >> 172.21.15.31:0/3536524116 conn(0x7f2c8006b620 msgr2=0x7f2c8006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.082+0000 7f2c85fff700 1 -- 172.21.15.31:0/3536524116 shutdown_connections 2023-10-06T05:25:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.082+0000 7f2c85fff700 1 -- 172.21.15.31:0/3536524116 wait complete. 2023-10-06T05:25:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.083+0000 7f2c85fff700 1 Processor -- start 2023-10-06T05:25:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.083+0000 7f2c85fff700 1 -- start start 2023-10-06T05:25:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.084+0000 7f2c85fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c8015e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.084+0000 7f2c85fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c80140e70 0x7f2c8015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.084+0000 7f2c85fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c8015d350 0x7f2c80157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.084+0000 7f2c85fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8015ee70 con 0x7f2c68004810 2023-10-06T05:25:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.084+0000 7f2c85fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8015e4a0 con 0x7f2c8015d350 2023-10-06T05:25:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.084+0000 7f2c85fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8015d890 con 0x7f2c80140e70 2023-10-06T05:25:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.084+0000 7f2c7f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c8015e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.084+0000 7f2c7f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c8015e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50706/0 (socket says 172.21.15.31:50706) 2023-10-06T05:25:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.084+0000 7f2c7f7fe700 1 -- 172.21.15.31:0/3311871944 learned_addr learned my addr 172.21.15.31:0/3311871944 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:19.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c7effd700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c80140e70 0x7f2c8015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c7ffff700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c8015d350 0x7f2c80157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c7f7fe700 1 -- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c80140e70 msgr2=0x7f2c8015ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c7f7fe700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c80140e70 0x7f2c8015ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c7f7fe700 1 -- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c8015d350 msgr2=0x7f2c80157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c7f7fe700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c8015d350 0x7f2c80157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c7f7fe700 1 -- 172.21.15.31:0/3311871944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8015db70 con 0x7f2c68004810 2023-10-06T05:25:19.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c7f7fe700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c8015e930 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f2c70011d40 tx=0x7f2c700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:19.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c677fe700 1 -- 172.21.15.31:0/3311871944 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000fac0 con 0x7f2c68004810 2023-10-06T05:25:19.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c677fe700 1 -- 172.21.15.31:0/3311871944 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7000fc20 con 0x7f2c68004810 2023-10-06T05:25:19.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c677fe700 1 -- 172.21.15.31:0/3311871944 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70014440 con 0x7f2c68004810 2023-10-06T05:25:19.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.085+0000 7f2c85fff700 1 -- 172.21.15.31:0/3311871944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8015de50 con 0x7f2c68004810 2023-10-06T05:25:19.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.086+0000 7f2c85fff700 1 -- 172.21.15.31:0/3311871944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c800b2cd0 con 0x7f2c68004810 2023-10-06T05:25:19.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.087+0000 7f2c85fff700 1 -- 172.21.15.31:0/3311871944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8004c380 con 0x7f2c68004810 2023-10-06T05:25:19.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.088+0000 7f2c677fe700 1 -- 172.21.15.31:0/3311871944 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f2c70014a80 con 0x7f2c68004810 2023-10-06T05:25:19.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.089+0000 7f2c677fe700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c6c067650 0x7f2c6c069b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.089+0000 7f2c677fe700 1 -- 172.21.15.31:0/3311871944 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f2c70018030 con 0x7f2c68004810 2023-10-06T05:25:19.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.089+0000 7f2c7effd700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c6c067650 0x7f2c6c069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.090+0000 7f2c7effd700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c6c067650 0x7f2c6c069b00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2c800c9400 tx=0x7f2c74015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:19.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.093+0000 7f2c677fe700 1 -- 172.21.15.31:0/3311871944 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c70083830 con 0x7f2c68004810 2023-10-06T05:25:19.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.344+0000 7f2c85fff700 1 -- 172.21.15.31:0/3311871944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c8015e780 con 0x7f2c68004810 2023-10-06T05:25:19.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.344+0000 7f2c677fe700 1 -- 172.21.15.31:0/3311871944 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f2c70022070 con 0x7f2c68004810 2023-10-06T05:25:19.346 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:19.346 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1352198494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1352198494},{"type":"v1","addr":"172.21.15.62:6835","nonce":1352198494}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8194,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/243154980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":243154980},{"type":"v1","addr":"172.21.15.62:6834","nonce":243154980}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/240917476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":240917476},{"type":"v1","addr":"172.21.15.31:6834","nonce":240917476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8778,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4058637367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4058637367},{"type":"v1","addr":"172.21.15.31:6835","nonce":4058637367}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[]} 2023-10-06T05:25:19.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.346+0000 7f2c85fff700 1 -- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c6c067650 msgr2=0x7f2c6c069b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.346+0000 7f2c85fff700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c6c067650 0x7f2c6c069b00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2c800c9400 tx=0x7f2c74015000 comp rx=0 tx=0).stop 2023-10-06T05:25:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.346+0000 7f2c85fff700 1 -- 172.21.15.31:0/3311871944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 msgr2=0x7f2c8015e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.346+0000 7f2c85fff700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c8015e930 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f2c70011d40 tx=0x7f2c700091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.346+0000 7f2c85fff700 1 -- 172.21.15.31:0/3311871944 shutdown_connections 2023-10-06T05:25:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.346+0000 7f2c85fff700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c6c067650 0x7f2c6c069b00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.346+0000 7f2c85fff700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c8015d350 0x7f2c80157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.346+0000 7f2c85fff700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68004810 0x7f2c8015e930 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.346+0000 7f2c85fff700 1 --2- 172.21.15.31:0/3311871944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c80140e70 0x7f2c8015ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.346+0000 7f2c85fff700 1 -- 172.21.15.31:0/3311871944 >> 172.21.15.31:0/3311871944 conn(0x7f2c8006b620 msgr2=0x7f2c8006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:19.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.347+0000 7f2c85fff700 1 -- 172.21.15.31:0/3311871944 shutdown_connections 2023-10-06T05:25:19.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.347+0000 7f2c85fff700 1 -- 172.21.15.31:0/3311871944 wait complete. 2023-10-06T05:25:19.350 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 96 2023-10-06T05:25:19.366 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:25:19.366 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:25:19.367 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:25:19.367 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:25:19.367 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:25:19.367 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:25:19.367 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:25:19.367 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:25:19.368 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:25:19.368 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:25:19.368 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:25:19.368 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:25:19.368 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:25:19.368 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:25:19.369 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:25:19.369 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:25:19.380 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:25:19.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:25:19.381 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:25:19.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:25:19.383 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:25:19.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:25:19.384 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:25:19.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:25:19.425 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:25:19.426 DEBUG:teuthology.parallel:result is None 2023-10-06T05:25:19.428 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:25:19.428 DEBUG:teuthology.parallel:result is None 2023-10-06T05:25:19.429 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:25:19.429 DEBUG:teuthology.parallel:result is None 2023-10-06T05:25:19.431 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:25:19.431 DEBUG:teuthology.parallel:result is None 2023-10-06T05:25:19.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:25:19.575 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:25:19.576 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:25:19.580 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:25:19.580 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:25:19.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.634+0000 7ff127dff700 1 Processor -- start 2023-10-06T05:25:19.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.635+0000 7ff127dff700 1 -- start start 2023-10-06T05:25:19.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.636+0000 7ff127dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff12013fed0 0x7ff12006e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.636+0000 7ff127dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff12006ea90 0x7ff1200cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.636+0000 7ff127dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12006be70 con 0x7ff12013fb00 2023-10-06T05:25:19.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.636+0000 7ff127dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12006ef30 con 0x7ff12013fed0 2023-10-06T05:25:19.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.636+0000 7ff127dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12006ca10 con 0x7ff12006ea90 2023-10-06T05:25:19.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.636+0000 7ff12539a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff12013fed0 0x7ff12006e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.636+0000 7ff12539a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff12013fed0 0x7ff12006e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50750/0 (socket says 172.21.15.31:50750) 2023-10-06T05:25:19.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.636+0000 7ff12539a700 1 -- 172.21.15.31:0/1997579197 learned_addr learned my addr 172.21.15.31:0/1997579197 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:19.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.636+0000 7ff12639c700 1 --2- 172.21.15.31:0/1997579197 >> v2:172.21.15.62:3300/0 conn(0x7ff12006ea90 0x7ff1200cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.637+0000 7ff12539a700 1 -- 172.21.15.31:0/1997579197 >> v2:172.21.15.62:3300/0 conn(0x7ff12006ea90 msgr2=0x7ff1200cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:19.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.637+0000 7ff12539a700 1 --2- 172.21.15.31:0/1997579197 >> v2:172.21.15.62:3300/0 conn(0x7ff12006ea90 0x7ff1200cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.637+0000 7ff12539a700 1 -- 172.21.15.31:0/1997579197 >> v1:172.21.15.62:6789/0 conn(0x7ff12013fb00 legacy=0x7ff12006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.637+0000 7ff12539a700 1 -- 172.21.15.31:0/1997579197 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12006f130 con 0x7ff12013fed0 2023-10-06T05:25:19.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.637+0000 7ff12639c700 1 --2- 172.21.15.31:0/1997579197 >> v2:172.21.15.62:3300/0 conn(0x7ff12006ea90 0x7ff1200cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:19.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.637+0000 7ff12539a700 1 --2- 172.21.15.31:0/1997579197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12013fed0 0x7ff12006e550 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff1200cba10 tx=0x7ff110009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cfacef3389015b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:19.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.637+0000 7ff124b99700 1 -- 172.21.15.31:0/1997579197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110005c10 con 0x7ff12013fed0 2023-10-06T05:25:19.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.637+0000 7ff124b99700 1 -- 172.21.15.31:0/1997579197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff110005d70 con 0x7ff12013fed0 2023-10-06T05:25:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.637+0000 7ff124b99700 1 -- 172.21.15.31:0/1997579197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11000b880 con 0x7ff12013fed0 2023-10-06T05:25:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.638+0000 7ff127dff700 1 -- 172.21.15.31:0/1997579197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12013fed0 msgr2=0x7ff12006e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.638+0000 7ff127dff700 1 --2- 172.21.15.31:0/1997579197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12013fed0 0x7ff12006e550 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff1200cba10 tx=0x7ff110009220 comp rx=0 tx=0).stop 2023-10-06T05:25:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.638+0000 7ff127dff700 1 -- 172.21.15.31:0/1997579197 shutdown_connections 2023-10-06T05:25:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.638+0000 7ff127dff700 1 --2- 172.21.15.31:0/1997579197 >> v2:172.21.15.62:3300/0 conn(0x7ff12006ea90 0x7ff1200cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.638+0000 7ff127dff700 1 --2- 172.21.15.31:0/1997579197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12013fed0 0x7ff12006e550 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.638+0000 7ff127dff700 1 -- 172.21.15.31:0/1997579197 >> 172.21.15.31:0/1997579197 conn(0x7ff1200cdcb0 msgr2=0x7ff1200c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.638+0000 7ff127dff700 1 -- 172.21.15.31:0/1997579197 shutdown_connections 2023-10-06T05:25:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.638+0000 7ff127dff700 1 -- 172.21.15.31:0/1997579197 wait complete. 2023-10-06T05:25:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.639+0000 7ff127dff700 1 Processor -- start 2023-10-06T05:25:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.640+0000 7ff127dff700 1 -- start start 2023-10-06T05:25:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.640+0000 7ff127dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12006ea90 0x7ff12015cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.640+0000 7ff127dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff12013fb00 0x7ff1201421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff127dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff12013fed0 0x7ff1201590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff127dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200b0670 con 0x7ff12006ea90 2023-10-06T05:25:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff127dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff120142810 con 0x7ff12013fb00 2023-10-06T05:25:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff127dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff120142af0 con 0x7ff12013fed0 2023-10-06T05:25:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff125b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12006ea90 0x7ff12015cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff125b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12006ea90 0x7ff12015cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50754/0 (socket says 172.21.15.31:50754) 2023-10-06T05:25:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff125b9b700 1 -- 172.21.15.31:0/3060595669 learned_addr learned my addr 172.21.15.31:0/3060595669 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff12539a700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff12013fb00 0x7ff1201421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff12639c700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff12013fed0 0x7ff1201590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff125b9b700 1 -- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff12013fed0 msgr2=0x7ff1201590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff125b9b700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff12013fed0 0x7ff1201590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff125b9b700 1 -- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff12013fb00 msgr2=0x7ff1201421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff125b9b700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff12013fb00 0x7ff1201421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff125b9b700 1 -- 172.21.15.31:0/3060595669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff110009480 con 0x7ff12006ea90 2023-10-06T05:25:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff12539a700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff12013fb00 0x7ff1201421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff12639c700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff12013fed0 0x7ff1201590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.641+0000 7ff125b9b700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12006ea90 0x7ff12015cb40 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c590 tx=0x7ff11c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.642+0000 7ff115ffb700 1 -- 172.21.15.31:0/3060595669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c011590 con 0x7ff12006ea90 2023-10-06T05:25:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.642+0000 7ff115ffb700 1 -- 172.21.15.31:0/3060595669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c0116f0 con 0x7ff12006ea90 2023-10-06T05:25:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.642+0000 7ff115ffb700 1 -- 172.21.15.31:0/3060595669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c011ab0 con 0x7ff12006ea90 2023-10-06T05:25:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.642+0000 7ff127dff700 1 -- 172.21.15.31:0/3060595669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1200b0020 con 0x7ff12006ea90 2023-10-06T05:25:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.642+0000 7ff127dff700 1 -- 172.21.15.31:0/3060595669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1200b22c0 con 0x7ff12006ea90 2023-10-06T05:25:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.643+0000 7ff127dff700 1 -- 172.21.15.31:0/3060595669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12004c380 con 0x7ff12006ea90 2023-10-06T05:25:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.644+0000 7ff115ffb700 1 -- 172.21.15.31:0/3060595669 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7ff11c01b3b0 con 0x7ff12006ea90 2023-10-06T05:25:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.645+0000 7ff115ffb700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff104067680 0x7ff104069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.645+0000 7ff115ffb700 1 -- 172.21.15.31:0/3060595669 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5320+0+0 (secure 0 0 0) 0x7ff11c084b30 con 0x7ff12006ea90 2023-10-06T05:25:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.645+0000 7ff12539a700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff104067680 0x7ff104069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.646+0000 7ff12539a700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff104067680 0x7ff104069b30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff120140330 tx=0x7ff110016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.649+0000 7ff115ffb700 1 -- 172.21.15.31:0/3060595669 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff11c0821f0 con 0x7ff12006ea90 2023-10-06T05:25:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:19.865+0000 7ff127dff700 1 -- 172.21.15.31:0/3060595669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff12015a260 con 0x7ff12006ea90 2023-10-06T05:25:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.212+0000 7ff115ffb700 1 -- 172.21.15.31:0/3060595669 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v144) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff11c017080 con 0x7ff12006ea90 2023-10-06T05:25:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.215+0000 7ff127dff700 1 -- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff104067680 msgr2=0x7ff104069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.215+0000 7ff127dff700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff104067680 0x7ff104069b30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff120140330 tx=0x7ff110016000 comp rx=0 tx=0).stop 2023-10-06T05:25:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.215+0000 7ff127dff700 1 -- 172.21.15.31:0/3060595669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12006ea90 msgr2=0x7ff12015cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.215+0000 7ff127dff700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12006ea90 0x7ff12015cb40 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c590 tx=0x7ff11c00c950 comp rx=0 tx=0).stop 2023-10-06T05:25:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.216+0000 7ff127dff700 1 -- 172.21.15.31:0/3060595669 shutdown_connections 2023-10-06T05:25:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.216+0000 7ff127dff700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff104067680 0x7ff104069b30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.216+0000 7ff127dff700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff12013fed0 0x7ff1201590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.216+0000 7ff127dff700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12006ea90 0x7ff12015cb40 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.216+0000 7ff127dff700 1 --2- 172.21.15.31:0/3060595669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff12013fb00 0x7ff1201421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.216+0000 7ff127dff700 1 -- 172.21.15.31:0/3060595669 >> 172.21.15.31:0/3060595669 conn(0x7ff1200cdcb0 msgr2=0x7ff12013c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.216+0000 7ff127dff700 1 -- 172.21.15.31:0/3060595669 shutdown_connections 2023-10-06T05:25:20.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.216+0000 7ff127dff700 1 -- 172.21.15.31:0/3060595669 wait complete. 2023-10-06T05:25:20.219 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:25:20.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:25:20.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.432+0000 7f973fdff700 1 Processor -- start 2023-10-06T05:25:20.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.432+0000 7f973fdff700 1 -- start start 2023-10-06T05:25:20.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.434+0000 7f973fdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f973806b510 0x7f97380c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.434+0000 7f973fdff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f97380cab40 0x7f97381569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.434+0000 7f973fdff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9738156ef0 con 0x7f97380ca770 2023-10-06T05:25:20.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.434+0000 7f973fdff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97380c7a70 con 0x7f97380cab40 2023-10-06T05:25:20.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.434+0000 7f973fdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97380c6df0 con 0x7f973806b510 2023-10-06T05:25:20.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973db9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f97380ca770 0x7f973806ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36782/0 (socket says 172.21.15.31:36782) 2023-10-06T05:25:20.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973db9b700 1 -- 172.21.15.31:0/4238801164 learned_addr learned my addr 172.21.15.31:0/4238801164 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:20.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973e39c700 1 --2- 172.21.15.31:0/4238801164 >> v2:172.21.15.62:3300/0 conn(0x7f97380cab40 0x7f97381569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973d39a700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f973806b510 0x7f97380c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428699672 0 0) 0x7f9738156ef0 con 0x7f97380ca770 2023-10-06T05:25:20.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9724003570 con 0x7f97380ca770 2023-10-06T05:25:20.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1485708284 0 0) 0x7f9724003570 con 0x7f97380ca770 2023-10-06T05:25:20.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f973806b510 msgr2=0x7f97380c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:20.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f973806b510 0x7f97380c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 >> v2:172.21.15.62:3300/0 conn(0x7f97380cab40 msgr2=0x7f97381569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:20.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 --2- 172.21.15.31:0/4238801164 >> v2:172.21.15.62:3300/0 conn(0x7f97380cab40 0x7f97381569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380c70d0 con 0x7f97380ca770 2023-10-06T05:25:20.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f97340015f0 con 0x7f97380ca770 2023-10-06T05:25:20.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973e39c700 1 --2- 172.21.15.31:0/4238801164 >> v2:172.21.15.62:3300/0 conn(0x7f97380cab40 0x7f97381569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:20.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973d39a700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f973806b510 0x7f97380c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:20.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 >> v1:172.21.15.31:6789/0 conn(0x7f97380ca770 legacy=0x7f973806ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.435+0000 7f973cb99700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9724003ae0 0x7f9724003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973cb99700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9724004810 0x7f9724004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973cb99700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f973806b510 0x7f97240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380c7a70 con 0x7f9724004810 2023-10-06T05:25:20.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380c6df0 con 0x7f9724003ae0 2023-10-06T05:25:20.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972400dcf0 con 0x7f973806b510 2023-10-06T05:25:20.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973d39a700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9724004810 0x7f9724004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973db9b700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9724003ae0 0x7f9724003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973e39c700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f973806b510 0x7f97240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973d39a700 1 -- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f973806b510 msgr2=0x7f97240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:20.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973d39a700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f973806b510 0x7f97240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973d39a700 1 -- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9724003ae0 msgr2=0x7f9724003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973d39a700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9724003ae0 0x7f9724003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973d39a700 1 -- 172.21.15.31:0/4238801164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9728009420 con 0x7f9724004810 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.436+0000 7f973d39a700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9724004810 0x7f9724004c80 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f9728009df0 tx=0x7f97280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9956a2242805f4e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.437+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728009740 con 0x7f9724004810 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.437+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972800e920 con 0x7f9724004810 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.437+0000 7f973cb99700 1 -- 172.21.15.31:0/4238801164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972800ece0 con 0x7f9724004810 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.437+0000 7f973fdff700 1 -- 172.21.15.31:0/4238801164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9724004810 msgr2=0x7f9724004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.437+0000 7f973fdff700 1 --2- 172.21.15.31:0/4238801164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9724004810 0x7f9724004c80 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f9728009df0 tx=0x7f97280091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.437+0000 7f973e39c700 1 -- 172.21.15.31:0/4238801164 reap_dead start 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.437+0000 7f973fdff700 1 -- 172.21.15.31:0/4238801164 shutdown_connections 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.437+0000 7f973fdff700 1 -- 172.21.15.31:0/4238801164 >> 172.21.15.31:0/4238801164 conn(0x7f97380cc0b0 msgr2=0x7f97380cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:20.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.437+0000 7f973fdff700 1 -- 172.21.15.31:0/4238801164 shutdown_connections 2023-10-06T05:25:20.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.437+0000 7f973fdff700 1 -- 172.21.15.31:0/4238801164 wait complete. 2023-10-06T05:25:20.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.439+0000 7f973fdff700 1 Processor -- start 2023-10-06T05:25:20.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.439+0000 7f973fdff700 1 -- start start 2023-10-06T05:25:20.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.440+0000 7f973fdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9724004810 0x7f9738149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.440+0000 7f973fdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97380b1170 0x7f97380b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.440+0000 7f973fdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97380ac920 0x7f97380acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.440+0000 7f973fdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380b1b20 con 0x7f97380b1170 2023-10-06T05:25:20.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.440+0000 7f973fdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973816a790 con 0x7f9724004810 2023-10-06T05:25:20.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.440+0000 7f973fdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97380ade10 con 0x7f97380ac920 2023-10-06T05:25:20.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.440+0000 7f973d39a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97380b1170 0x7f97380b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.440+0000 7f973d39a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97380b1170 0x7f97380b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50784/0 (socket says 172.21.15.31:50784) 2023-10-06T05:25:20.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.441+0000 7f973d39a700 1 -- 172.21.15.31:0/3911919236 learned_addr learned my addr 172.21.15.31:0/3911919236 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:20.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.441+0000 7f973db9b700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9724004810 0x7f9738149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.441+0000 7f973e39c700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97380ac920 0x7f97380acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.441+0000 7f973d39a700 1 -- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97380ac920 msgr2=0x7f97380acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.441+0000 7f973d39a700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97380ac920 0x7f97380acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.441+0000 7f973d39a700 1 -- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9724004810 msgr2=0x7f9738149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.441+0000 7f973d39a700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9724004810 0x7f9738149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.441+0000 7f973d39a700 1 -- 172.21.15.31:0/3911919236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380ad300 con 0x7f97380b1170 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.441+0000 7f973d39a700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97380b1170 0x7f97380b15e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f97280091e0 tx=0x7f972801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.441+0000 7f972dffb700 1 -- 172.21.15.31:0/3911919236 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728022070 con 0x7f97380b1170 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.442+0000 7f972dffb700 1 -- 172.21.15.31:0/3911919236 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9728013ab0 con 0x7f97380b1170 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.442+0000 7f973fdff700 1 -- 172.21.15.31:0/3911919236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973814a160 con 0x7f97380b1170 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.442+0000 7f972dffb700 1 -- 172.21.15.31:0/3911919236 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728013e70 con 0x7f97380b1170 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.442+0000 7f973fdff700 1 -- 172.21.15.31:0/3911919236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97380ad570 con 0x7f97380b1170 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.443+0000 7f973fdff700 1 -- 172.21.15.31:0/3911919236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97380af1f0 con 0x7f97380b1170 2023-10-06T05:25:20.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.444+0000 7f972dffb700 1 -- 172.21.15.31:0/3911919236 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f9728026070 con 0x7f97380b1170 2023-10-06T05:25:20.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.444+0000 7f972dffb700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f971c067680 0x7f971c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.445+0000 7f972dffb700 1 -- 172.21.15.31:0/3911919236 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f97280845a0 con 0x7f97380b1170 2023-10-06T05:25:20.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.445+0000 7f973db9b700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f971c067680 0x7f971c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.446+0000 7f973db9b700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f971c067680 0x7f971c069b30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9734002ee0 tx=0x7f9734009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:20.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.449+0000 7f972dffb700 1 -- 172.21.15.31:0/3911919236 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97380af1f0 con 0x7f97380b1170 2023-10-06T05:25:20.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.727+0000 7f973fdff700 1 -- 172.21.15.31:0/3911919236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f97380af1f0 con 0x7f97380b1170 2023-10-06T05:25:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.727+0000 7f972dffb700 1 -- 172.21.15.31:0/3911919236 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f97380af1f0 con 0x7f97380b1170 2023-10-06T05:25:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:20.729 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:25:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:25:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:25:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:25:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:25:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:25:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:25:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:25:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:25:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:25:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:25:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:25:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:25:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:25:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:25:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:25:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:25:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:25:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:25:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:25:20.747 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:25:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.729+0000 7f973fdff700 1 -- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f971c067680 msgr2=0x7f971c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.729+0000 7f973fdff700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f971c067680 0x7f971c069b30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9734002ee0 tx=0x7f9734009040 comp rx=0 tx=0).stop 2023-10-06T05:25:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.729+0000 7f973fdff700 1 -- 172.21.15.31:0/3911919236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97380b1170 msgr2=0x7f97380b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.729+0000 7f973fdff700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97380b1170 0x7f97380b15e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f97280091e0 tx=0x7f972801a000 comp rx=0 tx=0).stop 2023-10-06T05:25:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.729+0000 7f973fdff700 1 -- 172.21.15.31:0/3911919236 shutdown_connections 2023-10-06T05:25:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.729+0000 7f973fdff700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f971c067680 0x7f971c069b30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.729+0000 7f973fdff700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f97380ac920 0x7f97380acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.729+0000 7f973fdff700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9724004810 0x7f9738149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.729+0000 7f973fdff700 1 --2- 172.21.15.31:0/3911919236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97380b1170 0x7f97380b15e0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.729+0000 7f973fdff700 1 -- 172.21.15.31:0/3911919236 >> 172.21.15.31:0/3911919236 conn(0x7f97380cc0b0 msgr2=0x7f973806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:20.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.730+0000 7f973fdff700 1 -- 172.21.15.31:0/3911919236 shutdown_connections 2023-10-06T05:25:20.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.730+0000 7f973fdff700 1 -- 172.21.15.31:0/3911919236 wait complete. 2023-10-06T05:25:20.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:20.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.945+0000 7fef36bff700 1 Processor -- start 2023-10-06T05:25:20.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.945+0000 7fef36bff700 1 -- start start 2023-10-06T05:25:20.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.946+0000 7fef36bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 0x7fef300b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.946+0000 7fef36bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef300c3f40 con 0x7fef300cd510 2023-10-06T05:25:20.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef36bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef300cb000 con 0x7fef3015d2d0 2023-10-06T05:25:20.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef36bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef300b2860 con 0x7fef30159500 2023-10-06T05:25:20.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef3519c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fef300cd510 0x7fef300aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36792/0 (socket says 172.21.15.31:36792) 2023-10-06T05:25:20.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef3519c700 1 -- 172.21.15.31:0/3388502872 learned_addr learned my addr 172.21.15.31:0/3388502872 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:20.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef3499b700 1 --2- 172.21.15.31:0/3388502872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 0x7fef300b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef2f7fe700 1 -- 172.21.15.31:0/3388502872 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984874636 0 0) 0x7fef300c3f40 con 0x7fef300cd510 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef2f7fe700 1 -- 172.21.15.31:0/3388502872 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef20003570 con 0x7fef300cd510 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef2f7fe700 1 -- 172.21.15.31:0/3388502872 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160270393 0 0) 0x7fef300cb000 con 0x7fef3015d2d0 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef2f7fe700 1 -- 172.21.15.31:0/3388502872 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef300c3f40 con 0x7fef3015d2d0 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef3499b700 1 -- 172.21.15.31:0/3388502872 >> v1:172.21.15.62:6789/0 conn(0x7fef3015d2d0 legacy=0x7fef300cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef3499b700 1 -- 172.21.15.31:0/3388502872 >> v1:172.21.15.31:6789/0 conn(0x7fef300cd510 legacy=0x7fef300aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.947+0000 7fef3499b700 1 -- 172.21.15.31:0/3388502872 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3015d7d0 con 0x7fef30159500 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.948+0000 7fef3499b700 1 --2- 172.21.15.31:0/3388502872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 0x7fef300b0ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fef24009910 tx=0x7fef240091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5bc1a832a0a7e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.948+0000 7fef2f7fe700 1 -- 172.21.15.31:0/3388502872 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24011630 con 0x7fef30159500 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.948+0000 7fef2f7fe700 1 -- 172.21.15.31:0/3388502872 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef24011790 con 0x7fef30159500 2023-10-06T05:25:20.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.948+0000 7fef2f7fe700 1 -- 172.21.15.31:0/3388502872 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24011b10 con 0x7fef30159500 2023-10-06T05:25:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.949+0000 7fef36bff700 1 -- 172.21.15.31:0/3388502872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 msgr2=0x7fef300b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.949+0000 7fef36bff700 1 --2- 172.21.15.31:0/3388502872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 0x7fef300b0ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fef24009910 tx=0x7fef240091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.949+0000 7fef36bff700 1 -- 172.21.15.31:0/3388502872 shutdown_connections 2023-10-06T05:25:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.949+0000 7fef36bff700 1 --2- 172.21.15.31:0/3388502872 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 0x7fef300b0ab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.949+0000 7fef36bff700 1 -- 172.21.15.31:0/3388502872 >> 172.21.15.31:0/3388502872 conn(0x7fef3006ec90 msgr2=0x7fef3006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.949+0000 7fef36bff700 1 -- 172.21.15.31:0/3388502872 shutdown_connections 2023-10-06T05:25:20.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.949+0000 7fef36bff700 1 -- 172.21.15.31:0/3388502872 wait complete. 2023-10-06T05:25:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.950+0000 7fef36bff700 1 Processor -- start 2023-10-06T05:25:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.951+0000 7fef36bff700 1 -- start start 2023-10-06T05:25:20.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef36bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef300cd510 0x7fef3015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef36bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 0x7fef3015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef36bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef3015d2d0 0x7fef3015ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef36bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef300ae840 con 0x7fef3015d2d0 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef36bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3016d610 con 0x7fef300cd510 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef36bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3016d8f0 con 0x7fef30159500 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef3519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef3015d2d0 0x7fef3015ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef3519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef3015d2d0 0x7fef3015ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50796/0 (socket says 172.21.15.31:50796) 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef3519c700 1 -- 172.21.15.31:0/2339768196 learned_addr learned my addr 172.21.15.31:0/2339768196 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef3499b700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef300cd510 0x7fef3015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.952+0000 7fef2ffff700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 0x7fef3015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef3499b700 1 -- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 msgr2=0x7fef3015a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef3499b700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 0x7fef3015a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef3499b700 1 -- 172.21.15.31:0/2339768196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef3015d2d0 msgr2=0x7fef3015ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef3499b700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef3015d2d0 0x7fef3015ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef3499b700 1 -- 172.21.15.31:0/2339768196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef24009420 con 0x7fef300cd510 2023-10-06T05:25:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef3519c700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef3015d2d0 0x7fef3015ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef3499b700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef300cd510 0x7fef3015ca10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fef24016000 tx=0x7fef24017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef2cff9700 1 -- 172.21.15.31:0/2339768196 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24000c90 con 0x7fef300cd510 2023-10-06T05:25:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef2cff9700 1 -- 172.21.15.31:0/2339768196 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef24011900 con 0x7fef300cd510 2023-10-06T05:25:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef36bff700 1 -- 172.21.15.31:0/2339768196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3016dbd0 con 0x7fef300cd510 2023-10-06T05:25:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.953+0000 7fef2cff9700 1 -- 172.21.15.31:0/2339768196 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24025670 con 0x7fef300cd510 2023-10-06T05:25:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.954+0000 7fef36bff700 1 -- 172.21.15.31:0/2339768196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3016e190 con 0x7fef300cd510 2023-10-06T05:25:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.954+0000 7fef36bff700 1 -- 172.21.15.31:0/2339768196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef300c6770 con 0x7fef300cd510 2023-10-06T05:25:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.956+0000 7fef2cff9700 1 -- 172.21.15.31:0/2339768196 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fef24023070 con 0x7fef300cd510 2023-10-06T05:25:20.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.956+0000 7fef2cff9700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fef14067680 0x7fef14069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:20.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.956+0000 7fef2cff9700 1 -- 172.21.15.31:0/2339768196 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fef24083d80 con 0x7fef300cd510 2023-10-06T05:25:20.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.957+0000 7fef2ffff700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fef14067680 0x7fef14069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:20.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.957+0000 7fef2ffff700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fef14067680 0x7fef14069b30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fef30067e30 tx=0x7fef1c007d20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:20.960+0000 7fef2cff9700 1 -- 172.21.15.31:0/2339768196 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef24083a70 con 0x7fef300cd510 2023-10-06T05:25:21.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.164+0000 7fef36bff700 1 -- 172.21.15.31:0/2339768196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef3004c380 con 0x7fef300cd510 2023-10-06T05:25:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.166+0000 7fef2cff9700 1 -- 172.21.15.31:0/2339768196 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+13752 (secure 0 0 0) 0x7fef24015090 con 0x7fef300cd510 2023-10-06T05:25:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:21.168 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":146,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:25:20.219687+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 -- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fef14067680 msgr2=0x7fef14069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fef14067680 0x7fef14069b30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fef30067e30 tx=0x7fef1c007d20 comp rx=0 tx=0).stop 2023-10-06T05:25:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 -- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef300cd510 msgr2=0x7fef3015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef300cd510 0x7fef3015ca10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fef24016000 tx=0x7fef24017000 comp rx=0 tx=0).stop 2023-10-06T05:25:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 -- 172.21.15.31:0/2339768196 shutdown_connections 2023-10-06T05:25:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fef14067680 0x7fef14069b30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef3015d2d0 0x7fef3015ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef300cd510 0x7fef3015ca10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 --2- 172.21.15.31:0/2339768196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fef30159500 0x7fef3015a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 -- 172.21.15.31:0/2339768196 >> 172.21.15.31:0/2339768196 conn(0x7fef3006ec90 msgr2=0x7fef300cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:21.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 -- 172.21.15.31:0/2339768196 shutdown_connections 2023-10-06T05:25:21.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.168+0000 7fef36bff700 1 -- 172.21.15.31:0/2339768196 wait complete. 2023-10-06T05:25:21.188 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:25:21.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:25:21.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.386+0000 7f162cdff700 1 Processor -- start 2023-10-06T05:25:21.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.386+0000 7f162cdff700 1 -- start start 2023-10-06T05:25:21.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.387+0000 7f162cdff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16280c7190 0x7f16280c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.387+0000 7f162cdff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f16280aef00 0x7f16280af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.387+0000 7f162cdff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1628068250 con 0x7f16280afcf0 2023-10-06T05:25:21.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.387+0000 7f162cdff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162806ac80 con 0x7f16280c7190 2023-10-06T05:25:21.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.387+0000 7f162cdff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1628067f20 con 0x7f16280aef00 2023-10-06T05:25:21.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.388+0000 7f162659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16280c7190 0x7f16280c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.388+0000 7f162659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16280c7190 0x7f16280c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50808/0 (socket says 172.21.15.31:50808) 2023-10-06T05:25:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.388+0000 7f162659c700 1 -- 172.21.15.31:0/2382334382 learned_addr learned my addr 172.21.15.31:0/2382334382 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.388+0000 7f1625d9b700 1 --2- 172.21.15.31:0/2382334382 >> v2:172.21.15.62:3300/0 conn(0x7f16280aef00 0x7f16280af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.388+0000 7f162659c700 1 -- 172.21.15.31:0/2382334382 >> v2:172.21.15.62:3300/0 conn(0x7f16280aef00 msgr2=0x7f16280af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.388+0000 7f162659c700 1 --2- 172.21.15.31:0/2382334382 >> v2:172.21.15.62:3300/0 conn(0x7f16280aef00 0x7f16280af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.388+0000 7f162659c700 1 -- 172.21.15.31:0/2382334382 >> v1:172.21.15.62:6789/0 conn(0x7f16280afcf0 legacy=0x7f16280686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.388+0000 7f162659c700 1 -- 172.21.15.31:0/2382334382 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162806aed0 con 0x7f16280c7190 2023-10-06T05:25:21.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.388+0000 7f1625d9b700 1 --2- 172.21.15.31:0/2382334382 >> v2:172.21.15.62:3300/0 conn(0x7f16280aef00 0x7f16280af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:21.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.388+0000 7f162659c700 1 --2- 172.21.15.31:0/2382334382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280c7190 0x7f16280c9d40 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f16280c9340 tx=0x7f1618009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c89175066cb4f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.389+0000 7f162559a700 1 -- 172.21.15.31:0/2382334382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161800b860 con 0x7f16280c7190 2023-10-06T05:25:21.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.389+0000 7f162559a700 1 -- 172.21.15.31:0/2382334382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161800b9c0 con 0x7f16280c7190 2023-10-06T05:25:21.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.389+0000 7f162559a700 1 -- 172.21.15.31:0/2382334382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161800bd60 con 0x7f16280c7190 2023-10-06T05:25:21.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.389+0000 7f162cdff700 1 -- 172.21.15.31:0/2382334382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280c7190 msgr2=0x7f16280c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.389+0000 7f162cdff700 1 --2- 172.21.15.31:0/2382334382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280c7190 0x7f16280c9d40 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f16280c9340 tx=0x7f1618009250 comp rx=0 tx=0).stop 2023-10-06T05:25:21.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.389+0000 7f162cdff700 1 -- 172.21.15.31:0/2382334382 shutdown_connections 2023-10-06T05:25:21.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.389+0000 7f162cdff700 1 --2- 172.21.15.31:0/2382334382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280c7190 0x7f16280c9d40 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.389+0000 7f162cdff700 1 --2- 172.21.15.31:0/2382334382 >> v2:172.21.15.62:3300/0 conn(0x7f16280aef00 0x7f16280af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.389+0000 7f162cdff700 1 -- 172.21.15.31:0/2382334382 >> 172.21.15.31:0/2382334382 conn(0x7f16280cd3d0 msgr2=0x7f16280c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.390+0000 7f162cdff700 1 -- 172.21.15.31:0/2382334382 shutdown_connections 2023-10-06T05:25:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.390+0000 7f162cdff700 1 -- 172.21.15.31:0/2382334382 wait complete. 2023-10-06T05:25:21.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.391+0000 7f162cdff700 1 Processor -- start 2023-10-06T05:25:21.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.391+0000 7f162cdff700 1 -- start start 2023-10-06T05:25:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.392+0000 7f162cdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16280aef00 0x7f162815de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.392+0000 7f162cdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280afcf0 0x7f1628147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.392+0000 7f162cdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16280c7190 0x7f16280c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.392+0000 7f162cdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16280c03e0 con 0x7f16280afcf0 2023-10-06T05:25:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.392+0000 7f162cdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16280c4400 con 0x7f16280c7190 2023-10-06T05:25:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.392+0000 7f162cdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16280c46e0 con 0x7f16280aef00 2023-10-06T05:25:21.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.392+0000 7f1625d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280afcf0 0x7f1628147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.392+0000 7f1625d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280afcf0 0x7f1628147c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50822/0 (socket says 172.21.15.31:50822) 2023-10-06T05:25:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.392+0000 7f1625d9b700 1 -- 172.21.15.31:0/2143571178 learned_addr learned my addr 172.21.15.31:0/2143571178 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f162659c700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16280aef00 0x7f162815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f1626d9d700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16280c7190 0x7f16280c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f1625d9b700 1 -- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16280aef00 msgr2=0x7f162815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f1625d9b700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16280aef00 0x7f162815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f1625d9b700 1 -- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16280c7190 msgr2=0x7f16280c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f1625d9b700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16280c7190 0x7f16280c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f1625d9b700 1 -- 172.21.15.31:0/2143571178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16180094f0 con 0x7f16280afcf0 2023-10-06T05:25:21.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f1626d9d700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16280c7190 0x7f16280c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:21.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f1625d9b700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280afcf0 0x7f1628147c20 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f161c00d860 tx=0x7f161c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f16167fc700 1 -- 172.21.15.31:0/2143571178 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c0125f0 con 0x7f16280afcf0 2023-10-06T05:25:21.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f16167fc700 1 -- 172.21.15.31:0/2143571178 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c012750 con 0x7f16280afcf0 2023-10-06T05:25:21.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.393+0000 7f162cdff700 1 -- 172.21.15.31:0/2143571178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16280c49c0 con 0x7f16280afcf0 2023-10-06T05:25:21.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.394+0000 7f16167fc700 1 -- 172.21.15.31:0/2143571178 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c012af0 con 0x7f16280afcf0 2023-10-06T05:25:21.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.394+0000 7f162cdff700 1 -- 172.21.15.31:0/2143571178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162815e3c0 con 0x7f16280afcf0 2023-10-06T05:25:21.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.394+0000 7f162cdff700 1 -- 172.21.15.31:0/2143571178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162804c380 con 0x7f16280afcf0 2023-10-06T05:25:21.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.396+0000 7f16167fc700 1 -- 172.21.15.31:0/2143571178 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f161c014430 con 0x7f16280afcf0 2023-10-06T05:25:21.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.397+0000 7f16167fc700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1608067680 0x7f1608069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.397+0000 7f16167fc700 1 -- 172.21.15.31:0/2143571178 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f161c0858d0 con 0x7f16280afcf0 2023-10-06T05:25:21.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.397+0000 7f162659c700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1608067680 0x7f1608069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.398+0000 7f162659c700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1608067680 0x7f1608069b30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f16280c9340 tx=0x7f1618017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.401+0000 7f16167fc700 1 -- 172.21.15.31:0/2143571178 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f161c082fe0 con 0x7f16280afcf0 2023-10-06T05:25:21.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.605+0000 7f162cdff700 1 -- 172.21.15.31:0/2143571178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f16280cc6e0 con 0x7f16280afcf0 2023-10-06T05:25:21.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.606+0000 7f16167fc700 1 -- 172.21.15.31:0/2143571178 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v146) v1 ==== 92+0+10 (secure 0 0 0) 0x7f161c051dd0 con 0x7f16280afcf0 2023-10-06T05:25:21.608 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:25:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.608+0000 7f162cdff700 1 -- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1608067680 msgr2=0x7f1608069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.608+0000 7f162cdff700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1608067680 0x7f1608069b30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f16280c9340 tx=0x7f1618017000 comp rx=0 tx=0).stop 2023-10-06T05:25:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.608+0000 7f162cdff700 1 -- 172.21.15.31:0/2143571178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280afcf0 msgr2=0x7f1628147c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.608+0000 7f162cdff700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280afcf0 0x7f1628147c20 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f161c00d860 tx=0x7f161c008040 comp rx=0 tx=0).stop 2023-10-06T05:25:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.608+0000 7f162cdff700 1 -- 172.21.15.31:0/2143571178 shutdown_connections 2023-10-06T05:25:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.608+0000 7f162cdff700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f1608067680 0x7f1608069b30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.608+0000 7f162cdff700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16280c7190 0x7f16280c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.608+0000 7f162cdff700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16280aef00 0x7f162815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.608+0000 7f162cdff700 1 --2- 172.21.15.31:0/2143571178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16280afcf0 0x7f1628147c20 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.608+0000 7f162cdff700 1 -- 172.21.15.31:0/2143571178 >> 172.21.15.31:0/2143571178 conn(0x7f16280cd3d0 msgr2=0x7f16280686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:21.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.609+0000 7f162cdff700 1 -- 172.21.15.31:0/2143571178 shutdown_connections 2023-10-06T05:25:21.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.609+0000 7f162cdff700 1 -- 172.21.15.31:0/2143571178 wait complete. 2023-10-06T05:25:21.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:25:21.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.825+0000 7f53819ff700 1 Processor -- start 2023-10-06T05:25:21.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.825+0000 7f53819ff700 1 -- start start 2023-10-06T05:25:21.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.826+0000 7f53819ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f537c153d40 0x7f537c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.826+0000 7f53819ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f537c0af890 0x7f537c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.826+0000 7f53819ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c0b0e40 con 0x7f537c0b09c0 2023-10-06T05:25:21.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.826+0000 7f53819ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537c0afc90 con 0x7f537c153d40 2023-10-06T05:25:21.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.826+0000 7f53819ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537c0ad690 con 0x7f537c0af890 2023-10-06T05:25:21.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.826+0000 7f537affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f537c153d40 0x7f537c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.826+0000 7f537affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f537c153d40 0x7f537c0b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50830/0 (socket says 172.21.15.31:50830) 2023-10-06T05:25:21.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.826+0000 7f537affd700 1 -- 172.21.15.31:0/837154314 learned_addr learned my addr 172.21.15.31:0/837154314 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:21.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.826+0000 7f537a7fc700 1 --1- 172.21.15.31:0/837154314 >> v1:172.21.15.31:6789/0 conn(0x7f537c0b09c0 0x7f537c0af180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36798/0 (socket says 172.21.15.31:36798) 2023-10-06T05:25:21.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f537b7fe700 1 --2- 172.21.15.31:0/837154314 >> v2:172.21.15.62:3300/0 conn(0x7f537c0af890 0x7f537c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f537affd700 1 -- 172.21.15.31:0/837154314 >> v2:172.21.15.62:3300/0 conn(0x7f537c0af890 msgr2=0x7f537c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f537affd700 1 --2- 172.21.15.31:0/837154314 >> v2:172.21.15.62:3300/0 conn(0x7f537c0af890 0x7f537c0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f537affd700 1 -- 172.21.15.31:0/837154314 >> v1:172.21.15.31:6789/0 conn(0x7f537c0b09c0 legacy=0x7f537c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f537affd700 1 -- 172.21.15.31:0/837154314 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0acd60 con 0x7f537c153d40 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f537b7fe700 1 --2- 172.21.15.31:0/837154314 >> v2:172.21.15.62:3300/0 conn(0x7f537c0af890 0x7f537c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f5379ffb700 1 -- 172.21.15.31:0/837154314 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469044177 0 0) 0x7f5370001110 con 0x7f537c0b09c0 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f537affd700 1 --2- 172.21.15.31:0/837154314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c153d40 0x7f537c0b0480 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f537c06b310 tx=0x7f536c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1a9c473dab283e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f5379ffb700 1 -- 172.21.15.31:0/837154314 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00b860 con 0x7f537c153d40 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f5379ffb700 1 -- 172.21.15.31:0/837154314 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c00b9c0 con 0x7f537c153d40 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.827+0000 7f5379ffb700 1 -- 172.21.15.31:0/837154314 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00bd60 con 0x7f537c153d40 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.828+0000 7f53819ff700 1 -- 172.21.15.31:0/837154314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c153d40 msgr2=0x7f537c0b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.828+0000 7f53819ff700 1 --2- 172.21.15.31:0/837154314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c153d40 0x7f537c0b0480 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f537c06b310 tx=0x7f536c009250 comp rx=0 tx=0).stop 2023-10-06T05:25:21.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.828+0000 7f53819ff700 1 -- 172.21.15.31:0/837154314 shutdown_connections 2023-10-06T05:25:21.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.828+0000 7f53819ff700 1 --2- 172.21.15.31:0/837154314 >> v2:172.21.15.62:3300/0 conn(0x7f537c0af890 0x7f537c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.828+0000 7f53819ff700 1 --2- 172.21.15.31:0/837154314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c153d40 0x7f537c0b0480 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.828+0000 7f53819ff700 1 -- 172.21.15.31:0/837154314 >> 172.21.15.31:0/837154314 conn(0x7f537c067560 msgr2=0x7f537c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:21.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.828+0000 7f53819ff700 1 -- 172.21.15.31:0/837154314 shutdown_connections 2023-10-06T05:25:21.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.828+0000 7f53819ff700 1 -- 172.21.15.31:0/837154314 wait complete. 2023-10-06T05:25:21.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.830+0000 7f53819ff700 1 Processor -- start 2023-10-06T05:25:21.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.830+0000 7f53819ff700 1 -- start start 2023-10-06T05:25:21.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f53819ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c0af890 0x7f537c144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f53819ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537c0b09c0 0x7f537c0cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f53819ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f537c153d40 0x7f537c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f53819ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0c2050 con 0x7f537c0af890 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f53819ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0c1440 con 0x7f537c153d40 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f53819ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0c1720 con 0x7f537c0b09c0 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c0af890 0x7f537c144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c0af890 0x7f537c144c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50840/0 (socket says 172.21.15.31:50840) 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537affd700 1 -- 172.21.15.31:0/1911509665 learned_addr learned my addr 172.21.15.31:0/1911509665 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537b7fe700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f537c153d40 0x7f537c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537a7fc700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537c0b09c0 0x7f537c0cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537affd700 1 -- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537c0b09c0 msgr2=0x7f537c0cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537affd700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537c0b09c0 0x7f537c0cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537affd700 1 -- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f537c153d40 msgr2=0x7f537c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537affd700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f537c153d40 0x7f537c0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537affd700 1 -- 172.21.15.31:0/1911509665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0094f0 con 0x7f537c0af890 2023-10-06T05:25:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.831+0000 7f537a7fc700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537c0b09c0 0x7f537c0cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.832+0000 7f537affd700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c0af890 0x7f537c144c50 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f536c0090b0 tx=0x7f536c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.832+0000 7f5362ffd700 1 -- 172.21.15.31:0/1911509665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c026480 con 0x7f537c0af890 2023-10-06T05:25:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.832+0000 7f5362ffd700 1 -- 172.21.15.31:0/1911509665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c01f070 con 0x7f537c0af890 2023-10-06T05:25:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.832+0000 7f53819ff700 1 -- 172.21.15.31:0/1911509665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c0c1a00 con 0x7f537c0af890 2023-10-06T05:25:21.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.832+0000 7f5362ffd700 1 -- 172.21.15.31:0/1911509665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c026830 con 0x7f537c0af890 2023-10-06T05:25:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.832+0000 7f53819ff700 1 -- 172.21.15.31:0/1911509665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c145190 con 0x7f537c0af890 2023-10-06T05:25:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.833+0000 7f53819ff700 1 -- 172.21.15.31:0/1911509665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c04c380 con 0x7f537c0af890 2023-10-06T05:25:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.834+0000 7f5362ffd700 1 -- 172.21.15.31:0/1911509665 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f536c01a430 con 0x7f537c0af890 2023-10-06T05:25:21.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.835+0000 7f5362ffd700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5368067680 0x7f5368069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.836+0000 7f5362ffd700 1 -- 172.21.15.31:0/1911509665 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f536c021070 con 0x7f537c0af890 2023-10-06T05:25:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.836+0000 7f537a7fc700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5368067680 0x7f5368069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:21.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.837+0000 7f537a7fc700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5368067680 0x7f5368069b30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f537c0cb3a0 tx=0x7f5370007ad0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:21.840+0000 7f5362ffd700 1 -- 172.21.15.31:0/1911509665 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f536c084b60 con 0x7f537c0af890 2023-10-06T05:25:22.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.043+0000 7f53819ff700 1 -- 172.21.15.31:0/1911509665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f537c0ad040 con 0x7f537c0af890 2023-10-06T05:25:22.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.044+0000 7f5362ffd700 1 -- 172.21.15.31:0/1911509665 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v146) v1 ==== 93+0+10 (secure 0 0 0) 0x7f536c02b070 con 0x7f537c0af890 2023-10-06T05:25:22.046 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:25:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 -- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5368067680 msgr2=0x7f5368069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5368067680 0x7f5368069b30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f537c0cb3a0 tx=0x7f5370007ad0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 -- 172.21.15.31:0/1911509665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c0af890 msgr2=0x7f537c144c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c0af890 0x7f537c144c50 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f536c0090b0 tx=0x7f536c017000 comp rx=0 tx=0).stop 2023-10-06T05:25:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 -- 172.21.15.31:0/1911509665 shutdown_connections 2023-10-06T05:25:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5368067680 0x7f5368069b30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f537c153d40 0x7f537c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f537c0af890 0x7f537c144c50 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 --2- 172.21.15.31:0/1911509665 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537c0b09c0 0x7f537c0cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 -- 172.21.15.31:0/1911509665 >> 172.21.15.31:0/1911509665 conn(0x7f537c067560 msgr2=0x7f537c06e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 -- 172.21.15.31:0/1911509665 shutdown_connections 2023-10-06T05:25:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.046+0000 7f53819ff700 1 -- 172.21.15.31:0/1911509665 wait complete. 2023-10-06T05:25:22.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:25:22.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.262+0000 7fc6cf1ff700 1 Processor -- start 2023-10-06T05:25:22.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.262+0000 7fc6cf1ff700 1 -- start start 2023-10-06T05:25:22.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.263+0000 7fc6cf1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6c815d2d0 0x7fc6c80b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.263+0000 7fc6cf1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c80c3f40 con 0x7fc6c8159500 2023-10-06T05:25:22.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.263+0000 7fc6cf1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c80cb000 con 0x7fc6c80cd510 2023-10-06T05:25:22.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.263+0000 7fc6cf1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c80b2860 con 0x7fc6c815d2d0 2023-10-06T05:25:22.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.263+0000 7fc6cd79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6c815d2d0 0x7fc6c80b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.263+0000 7fc6ccf9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc6c8159500 0x7fc6c80cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36806/0 (socket says 172.21.15.31:36806) 2023-10-06T05:25:22.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.263+0000 7fc6ccf9b700 1 -- 172.21.15.31:0/754862514 learned_addr learned my addr 172.21.15.31:0/754862514 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.264+0000 7fc6cd79c700 1 -- 172.21.15.31:0/754862514 >> v1:172.21.15.62:6789/0 conn(0x7fc6c80cd510 legacy=0x7fc6c80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.264+0000 7fc6cd79c700 1 -- 172.21.15.31:0/754862514 >> v1:172.21.15.31:6789/0 conn(0x7fc6c8159500 legacy=0x7fc6c80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.264+0000 7fc6cd79c700 1 -- 172.21.15.31:0/754862514 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c815d7d0 con 0x7fc6c815d2d0 2023-10-06T05:25:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.264+0000 7fc6c77fe700 1 -- 172.21.15.31:0/754862514 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155768032 0 0) 0x7fc6c80c3f40 con 0x7fc6c8159500 2023-10-06T05:25:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.264+0000 7fc6cd79c700 1 --2- 172.21.15.31:0/754862514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c815d2d0 0x7fc6c80b0ab0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009e60 tx=0x7fc6c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafe6ad19e3819f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.264+0000 7fc6c77fe700 1 -- 172.21.15.31:0/754862514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000e070 con 0x7fc6c815d2d0 2023-10-06T05:25:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.264+0000 7fc6c77fe700 1 -- 172.21.15.31:0/754862514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c000bc30 con 0x7fc6c815d2d0 2023-10-06T05:25:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.264+0000 7fc6c77fe700 1 -- 172.21.15.31:0/754862514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0012430 con 0x7fc6c815d2d0 2023-10-06T05:25:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.265+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/754862514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c815d2d0 msgr2=0x7fc6c80b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.265+0000 7fc6cf1ff700 1 --2- 172.21.15.31:0/754862514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c815d2d0 0x7fc6c80b0ab0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009e60 tx=0x7fc6c0009780 comp rx=0 tx=0).stop 2023-10-06T05:25:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.265+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/754862514 shutdown_connections 2023-10-06T05:25:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.265+0000 7fc6cf1ff700 1 --2- 172.21.15.31:0/754862514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c815d2d0 0x7fc6c80b0ab0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.265+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/754862514 >> 172.21.15.31:0/754862514 conn(0x7fc6c806ec90 msgr2=0x7fc6c806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.265+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/754862514 shutdown_connections 2023-10-06T05:25:22.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.265+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/754862514 wait complete. 2023-10-06T05:25:22.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.266+0000 7fc6cf1ff700 1 Processor -- start 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.267+0000 7fc6cf1ff700 1 -- start start 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6cf1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c80cd510 0x7fc6c80b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6cf1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8159500 0x7fc6c8154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6cf1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c815d2d0 0x7fc6c80b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6cf1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80b5d30 con 0x7fc6c80cd510 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6cf1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80b58a0 con 0x7fc6c8159500 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6cf1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80b49f0 con 0x7fc6c815d2d0 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6ccf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c80cd510 0x7fc6c80b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6ccf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c80cd510 0x7fc6c80b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50850/0 (socket says 172.21.15.31:50850) 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6ccf9b700 1 -- 172.21.15.31:0/3800708798 learned_addr learned my addr 172.21.15.31:0/3800708798 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6c7fff700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8159500 0x7fc6c8154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6cd79c700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c815d2d0 0x7fc6c80b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6ccf9b700 1 -- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c815d2d0 msgr2=0x7fc6c80b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6ccf9b700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c815d2d0 0x7fc6c80b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6ccf9b700 1 -- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8159500 msgr2=0x7fc6c8154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6ccf9b700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8159500 0x7fc6c8154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.268+0000 7fc6ccf9b700 1 -- 172.21.15.31:0/3800708798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c0009a20 con 0x7fc6c80cd510 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.269+0000 7fc6ccf9b700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c80cd510 0x7fc6c80b44b0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00c7e0 tx=0x7fc6bc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.269+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/3800708798 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc005b30 con 0x7fc6c80cd510 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.269+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/3800708798 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc005c90 con 0x7fc6c80cd510 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.269+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/3800708798 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc007300 con 0x7fc6c80cd510 2023-10-06T05:25:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.269+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/3800708798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c80b4c90 con 0x7fc6c80cd510 2023-10-06T05:25:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.269+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/3800708798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c80b5250 con 0x7fc6c80cd510 2023-10-06T05:25:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.270+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/3800708798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c80b5b80 con 0x7fc6c80cd510 2023-10-06T05:25:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.271+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/3800708798 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fc6bc007670 con 0x7fc6c80cd510 2023-10-06T05:25:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.272+0000 7fc6c4ff9700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6ac067680 0x7fc6ac069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.272+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/3800708798 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fc6bc084ab0 con 0x7fc6c80cd510 2023-10-06T05:25:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.272+0000 7fc6c7fff700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6ac067680 0x7fc6ac069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.273+0000 7fc6c7fff700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6ac067680 0x7fc6ac069b30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80c6470 tx=0x7fc6b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.276+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/3800708798 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6bc0821c0 con 0x7fc6c80cd510 2023-10-06T05:25:22.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.484+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/3800708798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc6c804c380 con 0x7fc6c80cd510 2023-10-06T05:25:22.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.485+0000 7fc6c4ff9700 1 -- 172.21.15.31:0/3800708798 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v146) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc6bc017090 con 0x7fc6c80cd510 2023-10-06T05:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:25:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:25:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:22.488 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:25:22.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6ac067680 msgr2=0x7fc6ac069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6ac067680 0x7fc6ac069b30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80c6470 tx=0x7fc6b4009040 comp rx=0 tx=0).stop 2023-10-06T05:25:22.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/3800708798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c80cd510 msgr2=0x7fc6c80b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c80cd510 0x7fc6c80b44b0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00c7e0 tx=0x7fc6bc007df0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/3800708798 shutdown_connections 2023-10-06T05:25:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc6ac067680 0x7fc6ac069b30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6c815d2d0 0x7fc6c80b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c80cd510 0x7fc6c80b44b0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 --2- 172.21.15.31:0/3800708798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c8159500 0x7fc6c8154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/3800708798 >> 172.21.15.31:0/3800708798 conn(0x7fc6c806ec90 msgr2=0x7fc6c80cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/3800708798 shutdown_connections 2023-10-06T05:25:22.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.487+0000 7fc6cf1ff700 1 -- 172.21.15.31:0/3800708798 wait complete. 2023-10-06T05:25:22.507 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:25:22.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:25:22.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.705+0000 7f8b555ff700 1 Processor -- start 2023-10-06T05:25:22.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.705+0000 7f8b555ff700 1 -- start start 2023-10-06T05:25:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.706+0000 7f8b555ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8b50159cb0 0x7f8b500caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.706+0000 7f8b555ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b500cf5f0 0x7f8b50163270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.706+0000 7f8b555ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b500cb670 con 0x7f8b500cafd0 2023-10-06T05:25:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.706+0000 7f8b555ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b500cd080 con 0x7f8b500cf5f0 2023-10-06T05:25:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.706+0000 7f8b555ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b500cb450 con 0x7f8b50159cb0 2023-10-06T05:25:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.706+0000 7f8b4f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b500cf5f0 0x7f8b50163270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.706+0000 7f8b4e7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8b500cafd0 0x7f8b5015a080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36810/0 (socket says 172.21.15.31:36810) 2023-10-06T05:25:22.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.706+0000 7f8b4f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b500cf5f0 0x7f8b50163270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50866/0 (socket says 172.21.15.31:50866) 2023-10-06T05:25:22.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.706+0000 7f8b4e7fc700 1 -- 172.21.15.31:0/1908168489 learned_addr learned my addr 172.21.15.31:0/1908168489 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:22.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4effd700 1 --2- 172.21.15.31:0/1908168489 >> v2:172.21.15.62:3300/0 conn(0x7f8b50159cb0 0x7f8b500caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4f7fe700 1 -- 172.21.15.31:0/1908168489 >> v2:172.21.15.62:3300/0 conn(0x7f8b50159cb0 msgr2=0x7f8b500caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:22.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4f7fe700 1 --2- 172.21.15.31:0/1908168489 >> v2:172.21.15.62:3300/0 conn(0x7f8b50159cb0 0x7f8b500caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4f7fe700 1 -- 172.21.15.31:0/1908168489 >> v1:172.21.15.31:6789/0 conn(0x7f8b500cafd0 legacy=0x7f8b5015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4f7fe700 1 -- 172.21.15.31:0/1908168489 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500cb950 con 0x7f8b500cf5f0 2023-10-06T05:25:22.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4effd700 1 --2- 172.21.15.31:0/1908168489 >> v2:172.21.15.62:3300/0 conn(0x7f8b50159cb0 0x7f8b500caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:22.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4dffb700 1 -- 172.21.15.31:0/1908168489 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009578531 0 0) 0x7f8b500cb670 con 0x7f8b500cafd0 2023-10-06T05:25:22.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4f7fe700 1 --2- 172.21.15.31:0/1908168489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b500cf5f0 0x7f8b50163270 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f8b500892c0 tx=0x7f8b48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebefd497ed22486 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:22.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4dffb700 1 -- 172.21.15.31:0/1908168489 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4800e070 con 0x7f8b500cf5f0 2023-10-06T05:25:22.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4dffb700 1 -- 172.21.15.31:0/1908168489 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4800bbf0 con 0x7f8b500cf5f0 2023-10-06T05:25:22.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.707+0000 7f8b4dffb700 1 -- 172.21.15.31:0/1908168489 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48012430 con 0x7f8b500cf5f0 2023-10-06T05:25:22.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.708+0000 7f8b555ff700 1 -- 172.21.15.31:0/1908168489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b500cf5f0 msgr2=0x7f8b50163270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.708+0000 7f8b555ff700 1 --2- 172.21.15.31:0/1908168489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b500cf5f0 0x7f8b50163270 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f8b500892c0 tx=0x7f8b48009780 comp rx=0 tx=0).stop 2023-10-06T05:25:22.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.708+0000 7f8b555ff700 1 -- 172.21.15.31:0/1908168489 shutdown_connections 2023-10-06T05:25:22.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.708+0000 7f8b555ff700 1 --2- 172.21.15.31:0/1908168489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b500cf5f0 0x7f8b50163270 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.708+0000 7f8b555ff700 1 --2- 172.21.15.31:0/1908168489 >> v2:172.21.15.62:3300/0 conn(0x7f8b50159cb0 0x7f8b500caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.708+0000 7f8b555ff700 1 -- 172.21.15.31:0/1908168489 >> 172.21.15.31:0/1908168489 conn(0x7f8b50099710 msgr2=0x7f8b5006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:22.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.708+0000 7f8b555ff700 1 -- 172.21.15.31:0/1908168489 shutdown_connections 2023-10-06T05:25:22.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.708+0000 7f8b555ff700 1 -- 172.21.15.31:0/1908168489 wait complete. 2023-10-06T05:25:22.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.710+0000 7f8b555ff700 1 Processor -- start 2023-10-06T05:25:22.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.710+0000 7f8b555ff700 1 -- start start 2023-10-06T05:25:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b555ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b500cafd0 0x7f8b50154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b555ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b500cf5f0 0x7f8b50155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b555ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b50159cb0 0x7f8b50155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b555ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500b5e10 con 0x7f8b50159cb0 2023-10-06T05:25:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b555ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50154940 con 0x7f8b500cafd0 2023-10-06T05:25:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b555ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500b5570 con 0x7f8b500cf5f0 2023-10-06T05:25:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b50159cb0 0x7f8b50155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b50159cb0 0x7f8b50155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50876/0 (socket says 172.21.15.31:50876) 2023-10-06T05:25:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4f7fe700 1 -- 172.21.15.31:0/3482351265 learned_addr learned my addr 172.21.15.31:0/3482351265 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4effd700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b500cafd0 0x7f8b50154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4e7fc700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b500cf5f0 0x7f8b50155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4f7fe700 1 -- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b500cf5f0 msgr2=0x7f8b50155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4f7fe700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b500cf5f0 0x7f8b50155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4f7fe700 1 -- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b500cafd0 msgr2=0x7f8b50154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4f7fe700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b500cafd0 0x7f8b50154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4f7fe700 1 -- 172.21.15.31:0/3482351265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b48009a20 con 0x7f8b50159cb0 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.711+0000 7f8b4e7fc700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b500cf5f0 0x7f8b50155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.712+0000 7f8b4f7fe700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b50159cb0 0x7f8b50155850 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f8b48012ad0 tx=0x7f8b48019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.712+0000 7f8b36ffd700 1 -- 172.21.15.31:0/3482351265 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48012e70 con 0x7f8b50159cb0 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.712+0000 7f8b36ffd700 1 -- 172.21.15.31:0/3482351265 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4801d630 con 0x7f8b50159cb0 2023-10-06T05:25:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.712+0000 7f8b36ffd700 1 -- 172.21.15.31:0/3482351265 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4801d9d0 con 0x7f8b50159cb0 2023-10-06T05:25:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.712+0000 7f8b555ff700 1 -- 172.21.15.31:0/3482351265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b500b5850 con 0x7f8b50159cb0 2023-10-06T05:25:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.712+0000 7f8b555ff700 1 -- 172.21.15.31:0/3482351265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b500b5190 con 0x7f8b50159cb0 2023-10-06T05:25:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.713+0000 7f8b555ff700 1 -- 172.21.15.31:0/3482351265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b50154ad0 con 0x7f8b50159cb0 2023-10-06T05:25:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.714+0000 7f8b36ffd700 1 -- 172.21.15.31:0/3482351265 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f8b48024070 con 0x7f8b50159cb0 2023-10-06T05:25:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.715+0000 7f8b36ffd700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b3c067680 0x7f8b3c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.715+0000 7f8b36ffd700 1 -- 172.21.15.31:0/3482351265 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f8b48087d70 con 0x7f8b50159cb0 2023-10-06T05:25:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.716+0000 7f8b4effd700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b3c067680 0x7f8b3c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.716+0000 7f8b4effd700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b3c067680 0x7f8b3c069b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000aca0 tx=0x7f8b40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:22.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.719+0000 7f8b36ffd700 1 -- 172.21.15.31:0/3482351265 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b48085480 con 0x7f8b50159cb0 2023-10-06T05:25:22.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:22.924+0000 7f8b555ff700 1 -- 172.21.15.31:0/3482351265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8b5004c380 con 0x7f8b50159cb0 2023-10-06T05:25:23.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.232+0000 7f8b36ffd700 1 -- 172.21.15.31:0/3482351265 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v147) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8b4802c0b0 con 0x7f8b50159cb0 2023-10-06T05:25:23.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.236+0000 7f8b555ff700 1 -- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b3c067680 msgr2=0x7f8b3c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:23.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.236+0000 7f8b555ff700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b3c067680 0x7f8b3c069b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000aca0 tx=0x7f8b40008040 comp rx=0 tx=0).stop 2023-10-06T05:25:23.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.236+0000 7f8b555ff700 1 -- 172.21.15.31:0/3482351265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b50159cb0 msgr2=0x7f8b50155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:23.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.236+0000 7f8b555ff700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b50159cb0 0x7f8b50155850 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f8b48012ad0 tx=0x7f8b48019000 comp rx=0 tx=0).stop 2023-10-06T05:25:23.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.237+0000 7f8b555ff700 1 -- 172.21.15.31:0/3482351265 shutdown_connections 2023-10-06T05:25:23.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.237+0000 7f8b555ff700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8b3c067680 0x7f8b3c069b30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.237+0000 7f8b555ff700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b50159cb0 0x7f8b50155850 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.237+0000 7f8b555ff700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b500cafd0 0x7f8b50154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.237+0000 7f8b555ff700 1 --2- 172.21.15.31:0/3482351265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b500cf5f0 0x7f8b50155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.237+0000 7f8b555ff700 1 -- 172.21.15.31:0/3482351265 >> 172.21.15.31:0/3482351265 conn(0x7f8b50099710 msgr2=0x7f8b50153800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:23.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.237+0000 7f8b555ff700 1 -- 172.21.15.31:0/3482351265 shutdown_connections 2023-10-06T05:25:23.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.237+0000 7f8b555ff700 1 -- 172.21.15.31:0/3482351265 wait complete. 2023-10-06T05:25:23.242 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:25:23.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:25:23.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.462+0000 7fe3237ff700 1 Processor -- start 2023-10-06T05:25:23.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.462+0000 7fe3237ff700 1 -- start start 2023-10-06T05:25:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.463+0000 7fe3237ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe31c153d40 0x7fe31c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.463+0000 7fe3237ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe31c0b09c0 0x7fe31c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe3237ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 0x7fe31c159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe3237ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c0b0e40 con 0x7fe31c0b09c0 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe3237ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c0ad690 con 0x7fe31c153d40 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe3237ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0acd60 con 0x7fe31c0af6c0 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe320d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe31c0b09c0 0x7fe31c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe320d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe31c0b09c0 0x7fe31c0af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50880/0 (socket says 172.21.15.31:50880) 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe320d9a700 1 -- 172.21.15.31:0/1344591420 learned_addr learned my addr 172.21.15.31:0/1344591420 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe321d9c700 1 --2- 172.21.15.31:0/1344591420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 0x7fe31c159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe32159b700 1 --2- 172.21.15.31:0/1344591420 >> v2:172.21.15.62:3300/0 conn(0x7fe31c153d40 0x7fe31c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe321d9c700 1 -- 172.21.15.31:0/1344591420 >> v2:172.21.15.62:3300/0 conn(0x7fe31c153d40 msgr2=0x7fe31c0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe321d9c700 1 --2- 172.21.15.31:0/1344591420 >> v2:172.21.15.62:3300/0 conn(0x7fe31c153d40 0x7fe31c0b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe321d9c700 1 -- 172.21.15.31:0/1344591420 >> v2:172.21.15.31:3300/0 conn(0x7fe31c0b09c0 msgr2=0x7fe31c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe321d9c700 1 --2- 172.21.15.31:0/1344591420 >> v2:172.21.15.31:3300/0 conn(0x7fe31c0b09c0 0x7fe31c0af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe321d9c700 1 -- 172.21.15.31:0/1344591420 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c0ad040 con 0x7fe31c0af6c0 2023-10-06T05:25:23.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.464+0000 7fe320d9a700 1 --2- 172.21.15.31:0/1344591420 >> v2:172.21.15.31:3300/0 conn(0x7fe31c0b09c0 0x7fe31c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:23.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.465+0000 7fe321d9c700 1 --2- 172.21.15.31:0/1344591420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 0x7fe31c159ea0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe31c06d1d0 tx=0x7fe314009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef4f63eccd2d336 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.465+0000 7fe313fff700 1 -- 172.21.15.31:0/1344591420 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31400e070 con 0x7fe31c0af6c0 2023-10-06T05:25:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.465+0000 7fe313fff700 1 -- 172.21.15.31:0/1344591420 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31400bab0 con 0x7fe31c0af6c0 2023-10-06T05:25:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.465+0000 7fe313fff700 1 -- 172.21.15.31:0/1344591420 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31400be70 con 0x7fe31c0af6c0 2023-10-06T05:25:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.466+0000 7fe3237ff700 1 -- 172.21.15.31:0/1344591420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 msgr2=0x7fe31c159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.466+0000 7fe3237ff700 1 --2- 172.21.15.31:0/1344591420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 0x7fe31c159ea0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe31c06d1d0 tx=0x7fe314009780 comp rx=0 tx=0).stop 2023-10-06T05:25:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.466+0000 7fe3237ff700 1 -- 172.21.15.31:0/1344591420 shutdown_connections 2023-10-06T05:25:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.466+0000 7fe3237ff700 1 --2- 172.21.15.31:0/1344591420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 0x7fe31c159ea0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.466+0000 7fe3237ff700 1 --2- 172.21.15.31:0/1344591420 >> v2:172.21.15.62:3300/0 conn(0x7fe31c153d40 0x7fe31c0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.466+0000 7fe3237ff700 1 --2- 172.21.15.31:0/1344591420 >> v2:172.21.15.31:3300/0 conn(0x7fe31c0b09c0 0x7fe31c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.466+0000 7fe3237ff700 1 -- 172.21.15.31:0/1344591420 >> 172.21.15.31:0/1344591420 conn(0x7fe31c067560 msgr2=0x7fe31c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.466+0000 7fe3237ff700 1 -- 172.21.15.31:0/1344591420 shutdown_connections 2023-10-06T05:25:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.466+0000 7fe3237ff700 1 -- 172.21.15.31:0/1344591420 wait complete. 2023-10-06T05:25:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.467+0000 7fe3237ff700 1 Processor -- start 2023-10-06T05:25:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.467+0000 7fe3237ff700 1 -- start start 2023-10-06T05:25:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.468+0000 7fe3237ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 0x7fe31c13e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.468+0000 7fe3237ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe31c0b09c0 0x7fe31c148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.468+0000 7fe3237ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c153d40 0x7fe31c149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.468+0000 7fe3237ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c13fa60 con 0x7fe31c153d40 2023-10-06T05:25:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.468+0000 7fe3237ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c144a70 con 0x7fe31c0b09c0 2023-10-06T05:25:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.468+0000 7fe3237ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c144760 con 0x7fe31c0af6c0 2023-10-06T05:25:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe321d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c153d40 0x7fe31c149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe32159b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 0x7fe31c13e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe321d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c153d40 0x7fe31c149020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50884/0 (socket says 172.21.15.31:50884) 2023-10-06T05:25:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe321d9c700 1 -- 172.21.15.31:0/2611278412 learned_addr learned my addr 172.21.15.31:0/2611278412 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe320d9a700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe31c0b09c0 0x7fe31c148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe321d9c700 1 -- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 msgr2=0x7fe31c13e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe321d9c700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 0x7fe31c13e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe321d9c700 1 -- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe31c0b09c0 msgr2=0x7fe31c148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe321d9c700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe31c0b09c0 0x7fe31c148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe321d9c700 1 -- 172.21.15.31:0/2611278412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0094f0 con 0x7fe31c153d40 2023-10-06T05:25:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.469+0000 7fe321d9c700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c153d40 0x7fe31c149020 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0acea0 tx=0x7fe314019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.470+0000 7fe3117fa700 1 -- 172.21.15.31:0/2611278412 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314012dd0 con 0x7fe31c153d40 2023-10-06T05:25:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.470+0000 7fe3117fa700 1 -- 172.21.15.31:0/2611278412 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe314027470 con 0x7fe31c153d40 2023-10-06T05:25:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.470+0000 7fe3237ff700 1 -- 172.21.15.31:0/2611278412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe314009a20 con 0x7fe31c153d40 2023-10-06T05:25:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.470+0000 7fe3117fa700 1 -- 172.21.15.31:0/2611278412 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314027830 con 0x7fe31c153d40 2023-10-06T05:25:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.470+0000 7fe3237ff700 1 -- 172.21.15.31:0/2611278412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c13ff30 con 0x7fe31c153d40 2023-10-06T05:25:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.471+0000 7fe3237ff700 1 -- 172.21.15.31:0/2611278412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c04c380 con 0x7fe31c153d40 2023-10-06T05:25:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.473+0000 7fe3117fa700 1 -- 172.21.15.31:0/2611278412 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fe314024070 con 0x7fe31c153d40 2023-10-06T05:25:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.473+0000 7fe3117fa700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe300067680 0x7fe300069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.473+0000 7fe3117fa700 1 -- 172.21.15.31:0/2611278412 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5604+0+0 (secure 0 0 0) 0x7fe314087e60 con 0x7fe31c153d40 2023-10-06T05:25:23.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.474+0000 7fe32159b700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe300067680 0x7fe300069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:23.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.475+0000 7fe32159b700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe300067680 0x7fe300069b30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe318010e50 tx=0x7fe318012040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.477+0000 7fe3117fa700 1 -- 172.21.15.31:0/2611278412 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe314085400 con 0x7fe31c153d40 2023-10-06T05:25:23.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:23.678+0000 7fe3237ff700 1 -- 172.21.15.31:0/2611278412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe31c06b600 con 0x7fe31c153d40 2023-10-06T05:25:24.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.243+0000 7fe3117fa700 1 -- 172.21.15.31:0/2611278412 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v148) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe314054270 con 0x7fe31c153d40 2023-10-06T05:25:24.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.248+0000 7fe3237ff700 1 -- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe300067680 msgr2=0x7fe300069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:24.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.248+0000 7fe3237ff700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe300067680 0x7fe300069b30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe318010e50 tx=0x7fe318012040 comp rx=0 tx=0).stop 2023-10-06T05:25:24.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.249+0000 7fe3237ff700 1 -- 172.21.15.31:0/2611278412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c153d40 msgr2=0x7fe31c149020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:24.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.249+0000 7fe3237ff700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c153d40 0x7fe31c149020 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0acea0 tx=0x7fe314019000 comp rx=0 tx=0).stop 2023-10-06T05:25:24.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.249+0000 7fe3237ff700 1 -- 172.21.15.31:0/2611278412 shutdown_connections 2023-10-06T05:25:24.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.249+0000 7fe3237ff700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe300067680 0x7fe300069b30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:24.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.249+0000 7fe3237ff700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c153d40 0x7fe31c149020 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:24.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.249+0000 7fe3237ff700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe31c0af6c0 0x7fe31c13e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:24.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.249+0000 7fe3237ff700 1 --2- 172.21.15.31:0/2611278412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe31c0b09c0 0x7fe31c148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:24.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.249+0000 7fe3237ff700 1 -- 172.21.15.31:0/2611278412 >> 172.21.15.31:0/2611278412 conn(0x7fe31c067560 msgr2=0x7fe31c0c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:24.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.254+0000 7fe3237ff700 1 -- 172.21.15.31:0/2611278412 shutdown_connections 2023-10-06T05:25:24.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.255+0000 7fe3237ff700 1 -- 172.21.15.31:0/2611278412 wait complete. 2023-10-06T05:25:24.269 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:25:24.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:25:24.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.485+0000 7fa41e5ff700 1 Processor -- start 2023-10-06T05:25:24.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.485+0000 7fa41e5ff700 1 -- start start 2023-10-06T05:25:24.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.486+0000 7fa41e5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa418159560 0x7fa4180cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:24.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.486+0000 7fa41e5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4180b2b80 0x7fa4180cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:24.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.487+0000 7fa41e5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41815d2d0 con 0x7fa418159930 2023-10-06T05:25:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.487+0000 7fa41e5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41815d5b0 con 0x7fa418159560 2023-10-06T05:25:24.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.487+0000 7fa41e5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41815d890 con 0x7fa4180b2b80 2023-10-06T05:25:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.487+0000 7fa417fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa418159560 0x7fa4180cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.487+0000 7fa41cb9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4180b2b80 0x7fa4180cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.487+0000 7fa41cb9c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4180b2b80 0x7fa4180cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:35426/0 (socket says 172.21.15.31:35426) 2023-10-06T05:25:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.487+0000 7fa41cb9c700 1 -- 172.21.15.31:0/1224571512 learned_addr learned my addr 172.21.15.31:0/1224571512 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.488+0000 7fa417fff700 1 -- 172.21.15.31:0/1224571512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4180b2b80 msgr2=0x7fa4180cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.488+0000 7fa417fff700 1 --2- 172.21.15.31:0/1224571512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4180b2b80 0x7fa4180cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.488+0000 7fa417fff700 1 -- 172.21.15.31:0/1224571512 >> v1:172.21.15.62:6789/0 conn(0x7fa418159930 legacy=0x7fa4180b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:24.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.488+0000 7fa417fff700 1 -- 172.21.15.31:0/1224571512 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41815db70 con 0x7fa418159560 2023-10-06T05:25:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.488+0000 7fa417fff700 1 --2- 172.21.15.31:0/1224571512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa418159560 0x7fa4180cce70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa4080098e0 tx=0x7fa4080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7cd26eebc3ca9e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.488+0000 7fa416ffd700 1 -- 172.21.15.31:0/1224571512 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800b640 con 0x7fa418159560 2023-10-06T05:25:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.488+0000 7fa416ffd700 1 -- 172.21.15.31:0/1224571512 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40800b7a0 con 0x7fa418159560 2023-10-06T05:25:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.488+0000 7fa416ffd700 1 -- 172.21.15.31:0/1224571512 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800bb40 con 0x7fa418159560 2023-10-06T05:25:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.489+0000 7fa41e5ff700 1 -- 172.21.15.31:0/1224571512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa418159560 msgr2=0x7fa4180cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.489+0000 7fa41e5ff700 1 --2- 172.21.15.31:0/1224571512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa418159560 0x7fa4180cce70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa4080098e0 tx=0x7fa4080091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.489+0000 7fa41e5ff700 1 -- 172.21.15.31:0/1224571512 shutdown_connections 2023-10-06T05:25:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.489+0000 7fa41e5ff700 1 --2- 172.21.15.31:0/1224571512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4180b2b80 0x7fa4180cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.489+0000 7fa41e5ff700 1 --2- 172.21.15.31:0/1224571512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa418159560 0x7fa4180cce70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.489+0000 7fa41e5ff700 1 -- 172.21.15.31:0/1224571512 >> 172.21.15.31:0/1224571512 conn(0x7fa41806ec90 msgr2=0x7fa41806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.489+0000 7fa41e5ff700 1 -- 172.21.15.31:0/1224571512 shutdown_connections 2023-10-06T05:25:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.489+0000 7fa41e5ff700 1 -- 172.21.15.31:0/1224571512 wait complete. 2023-10-06T05:25:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.490+0000 7fa41e5ff700 1 Processor -- start 2023-10-06T05:25:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.491+0000 7fa41e5ff700 1 -- start start 2023-10-06T05:25:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa41e5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4180b2b80 0x7fa41816adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa41e5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa418159560 0x7fa4180d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa41e5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa418159930 0x7fa4180c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa41e5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41816b300 con 0x7fa4180b2b80 2023-10-06T05:25:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa41e5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4180c8bd0 con 0x7fa418159930 2023-10-06T05:25:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa41e5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4180c7fc0 con 0x7fa418159560 2023-10-06T05:25:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa417fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4180b2b80 0x7fa41816adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa417fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4180b2b80 0x7fa41816adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50898/0 (socket says 172.21.15.31:50898) 2023-10-06T05:25:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa417fff700 1 -- 172.21.15.31:0/638112936 learned_addr learned my addr 172.21.15.31:0/638112936 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa41cb9c700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa418159930 0x7fa4180c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa4177fe700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa418159560 0x7fa4180d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa417fff700 1 -- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa418159560 msgr2=0x7fa4180d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa417fff700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa418159560 0x7fa4180d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa417fff700 1 -- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa418159930 msgr2=0x7fa4180c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa417fff700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa418159930 0x7fa4180c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa417fff700 1 -- 172.21.15.31:0/638112936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408009420 con 0x7fa4180b2b80 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.492+0000 7fa4177fe700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa418159560 0x7fa4180d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.493+0000 7fa417fff700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4180b2b80 0x7fa41816adc0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fa4180893d0 tx=0x7fa408017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.493+0000 7fa3fffff700 1 -- 172.21.15.31:0/638112936 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40801bcf0 con 0x7fa4180b2b80 2023-10-06T05:25:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.493+0000 7fa3fffff700 1 -- 172.21.15.31:0/638112936 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40801be90 con 0x7fa4180b2b80 2023-10-06T05:25:24.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.493+0000 7fa41e5ff700 1 -- 172.21.15.31:0/638112936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4180c82a0 con 0x7fa4180b2b80 2023-10-06T05:25:24.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.493+0000 7fa3fffff700 1 -- 172.21.15.31:0/638112936 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408011670 con 0x7fa4180b2b80 2023-10-06T05:25:24.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.493+0000 7fa41e5ff700 1 -- 172.21.15.31:0/638112936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4180c8860 con 0x7fa4180b2b80 2023-10-06T05:25:24.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.494+0000 7fa41e5ff700 1 -- 172.21.15.31:0/638112936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41804c380 con 0x7fa4180b2b80 2023-10-06T05:25:24.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.495+0000 7fa3fffff700 1 -- 172.21.15.31:0/638112936 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fa4080119e0 con 0x7fa4180b2b80 2023-10-06T05:25:24.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.496+0000 7fa3fffff700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa4040676b0 0x7fa404069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:24.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.496+0000 7fa3fffff700 1 -- 172.21.15.31:0/638112936 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5986+0+0 (secure 0 0 0) 0x7fa4080845a0 con 0x7fa4180b2b80 2023-10-06T05:25:24.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.497+0000 7fa4177fe700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa4040676b0 0x7fa404069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:24.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.497+0000 7fa4177fe700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa4040676b0 0x7fa404069b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa40c007fd0 tx=0x7fa40c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:24.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.500+0000 7fa3fffff700 1 -- 172.21.15.31:0/638112936 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4080842b0 con 0x7fa4180b2b80 2023-10-06T05:25:24.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:24.745+0000 7fa41e5ff700 1 -- 172.21.15.31:0/638112936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa41815de50 con 0x7fa4180b2b80 2023-10-06T05:25:25.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.247+0000 7fa3fffff700 1 -- 172.21.15.31:0/638112936 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v99) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa408053120 con 0x7fa4180b2b80 2023-10-06T05:25:25.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.256+0000 7fa41e5ff700 1 -- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa4040676b0 msgr2=0x7fa404069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.256+0000 7fa41e5ff700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa4040676b0 0x7fa404069b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa40c007fd0 tx=0x7fa40c009040 comp rx=0 tx=0).stop 2023-10-06T05:25:25.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.256+0000 7fa41e5ff700 1 -- 172.21.15.31:0/638112936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4180b2b80 msgr2=0x7fa41816adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.256+0000 7fa41e5ff700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4180b2b80 0x7fa41816adc0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fa4180893d0 tx=0x7fa408017000 comp rx=0 tx=0).stop 2023-10-06T05:25:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.259+0000 7fa41e5ff700 1 -- 172.21.15.31:0/638112936 shutdown_connections 2023-10-06T05:25:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.259+0000 7fa41e5ff700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa4040676b0 0x7fa404069b60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.259+0000 7fa41e5ff700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa418159930 0x7fa4180c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.259+0000 7fa41e5ff700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4180b2b80 0x7fa41816adc0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.259+0000 7fa41e5ff700 1 --2- 172.21.15.31:0/638112936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa418159560 0x7fa4180d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.259+0000 7fa41e5ff700 1 -- 172.21.15.31:0/638112936 >> 172.21.15.31:0/638112936 conn(0x7fa41806ec90 msgr2=0x7fa4181567d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.264+0000 7fa41e5ff700 1 -- 172.21.15.31:0/638112936 shutdown_connections 2023-10-06T05:25:25.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.267+0000 7fa41e5ff700 1 -- 172.21.15.31:0/638112936 wait complete. 2023-10-06T05:25:25.279 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 25 and data pool 26 2023-10-06T05:25:25.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.494+0000 7f6c6c1ff700 1 Processor -- start 2023-10-06T05:25:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.494+0000 7f6c6c1ff700 1 -- start start 2023-10-06T05:25:25.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.495+0000 7f6c6c1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c640c7f30 0x7f6c64069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.495+0000 7f6c6c1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c640aee70 0x7f6c6413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.495+0000 7f6c6c1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c640ca5b0 con 0x7f6c640c7b60 2023-10-06T05:25:25.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.495+0000 7f6c6c1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c640af2f0 con 0x7f6c640aee70 2023-10-06T05:25:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.495+0000 7f6c6c1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6406f740 con 0x7f6c640c7f30 2023-10-06T05:25:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.496+0000 7f6c6a79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c640aee70 0x7f6c6413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.496+0000 7f6c6a79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c640aee70 0x7f6c6413f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50912/0 (socket says 172.21.15.31:50912) 2023-10-06T05:25:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.496+0000 7f6c6a79c700 1 -- 172.21.15.31:0/242839613 learned_addr learned my addr 172.21.15.31:0/242839613 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:25.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.496+0000 7f6c6979a700 1 --2- 172.21.15.31:0/242839613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c640c7f30 0x7f6c64069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.496+0000 7f6c6a79c700 1 -- 172.21.15.31:0/242839613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c640c7f30 msgr2=0x7f6c64069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.496+0000 7f6c6a79c700 1 --2- 172.21.15.31:0/242839613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c640c7f30 0x7f6c64069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.496+0000 7f6c6a79c700 1 -- 172.21.15.31:0/242839613 >> v1:172.21.15.62:6789/0 conn(0x7f6c640c7b60 legacy=0x7f6c6406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.496+0000 7f6c6a79c700 1 -- 172.21.15.31:0/242839613 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c640c9c80 con 0x7f6c640aee70 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.496+0000 7f6c68f99700 1 -- 172.21.15.31:0/242839613 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442123938 0 0) 0x7f6c640ca5b0 con 0x7f6c640c7b60 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.496+0000 7f6c6a79c700 1 --2- 172.21.15.31:0/242839613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640aee70 0x7f6c6413f690 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009f20 tx=0x7f6c5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a640e4a2c3c3f632 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c68f99700 1 -- 172.21.15.31:0/242839613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00e070 con 0x7f6c640aee70 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c68f99700 1 -- 172.21.15.31:0/242839613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c00b9d0 con 0x7f6c640aee70 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c68f99700 1 -- 172.21.15.31:0/242839613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0092e0 con 0x7f6c640aee70 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/242839613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640aee70 msgr2=0x7f6c6413f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c6c1ff700 1 --2- 172.21.15.31:0/242839613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640aee70 0x7f6c6413f690 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009f20 tx=0x7f6c5c009840 comp rx=0 tx=0).stop 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/242839613 shutdown_connections 2023-10-06T05:25:25.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c6c1ff700 1 --2- 172.21.15.31:0/242839613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640aee70 0x7f6c6413f690 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c6c1ff700 1 --2- 172.21.15.31:0/242839613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c640c7f30 0x7f6c64069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/242839613 >> 172.21.15.31:0/242839613 conn(0x7f6c640cc0b0 msgr2=0x7f6c640cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/242839613 shutdown_connections 2023-10-06T05:25:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.497+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/242839613 wait complete. 2023-10-06T05:25:25.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.499+0000 7f6c6c1ff700 1 Processor -- start 2023-10-06T05:25:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.499+0000 7f6c6c1ff700 1 -- start start 2023-10-06T05:25:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.500+0000 7f6c6c1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c640aee70 0x7f6c64155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.500+0000 7f6c6c1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c640c7b60 0x7f6c640c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.500+0000 7f6c6c1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640c7f30 0x7f6c640cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.500+0000 7f6c6c1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6413cbc0 con 0x7f6c640c7f30 2023-10-06T05:25:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.500+0000 7f6c6c1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c0f80 con 0x7f6c640aee70 2023-10-06T05:25:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.500+0000 7f6c6c1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c5aa0 con 0x7f6c640c7b60 2023-10-06T05:25:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.500+0000 7f6c6a79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640c7f30 0x7f6c640cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.500+0000 7f6c6a79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640c7f30 0x7f6c640cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50928/0 (socket says 172.21.15.31:50928) 2023-10-06T05:25:25.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.500+0000 7f6c6a79c700 1 -- 172.21.15.31:0/3626031193 learned_addr learned my addr 172.21.15.31:0/3626031193 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c6979a700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c640c7b60 0x7f6c640c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c69f9b700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c640aee70 0x7f6c64155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c6a79c700 1 -- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c640c7b60 msgr2=0x7f6c640c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c6a79c700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c640c7b60 0x7f6c640c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c6a79c700 1 -- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c640aee70 msgr2=0x7f6c64155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c6a79c700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c640aee70 0x7f6c64155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c6a79c700 1 -- 172.21.15.31:0/3626031193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c009ae0 con 0x7f6c640c7f30 2023-10-06T05:25:25.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c6a79c700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640c7f30 0x7f6c640cad80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b610 tx=0x7f6c5c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c59ffb700 1 -- 172.21.15.31:0/3626031193 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c008c30 con 0x7f6c640c7f30 2023-10-06T05:25:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c59ffb700 1 -- 172.21.15.31:0/3626031193 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c008dd0 con 0x7f6c640c7f30 2023-10-06T05:25:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c59ffb700 1 -- 172.21.15.31:0/3626031193 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0266b0 con 0x7f6c640c7f30 2023-10-06T05:25:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/3626031193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c640c5d80 con 0x7f6c640c7f30 2023-10-06T05:25:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.501+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/3626031193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c640c6340 con 0x7f6c640c7f30 2023-10-06T05:25:25.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.502+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/3626031193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6404c380 con 0x7f6c640c7f30 2023-10-06T05:25:25.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.503+0000 7f6c59ffb700 1 -- 172.21.15.31:0/3626031193 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f6c5c003040 con 0x7f6c640c7f30 2023-10-06T05:25:25.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.505+0000 7f6c59ffb700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c48067680 0x7f6c48069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.505+0000 7f6c59ffb700 1 -- 172.21.15.31:0/3626031193 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6c5c087990 con 0x7f6c640c7f30 2023-10-06T05:25:25.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.505+0000 7f6c69f9b700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c48067680 0x7f6c48069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.506+0000 7f6c69f9b700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c48067680 0x7f6c48069b30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6c600038f0 tx=0x7f6c60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:25.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.509+0000 7f6c59ffb700 1 -- 172.21.15.31:0/3626031193 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c5c084c40 con 0x7f6c640c7f30 2023-10-06T05:25:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.715+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/3626031193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c640c10c0 con 0x7f6c640c7f30 2023-10-06T05:25:25.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.716+0000 7f6c59ffb700 1 -- 172.21.15.31:0/3626031193 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16985 (secure 0 0 0) 0x7f6c5c053ae0 con 0x7f6c640c7f30 2023-10-06T05:25:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:25.718 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":149,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:25:24.746365+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:25:22.964655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:23.719177+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:25.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c48067680 msgr2=0x7f6c48069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c48067680 0x7f6c48069b30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6c600038f0 tx=0x7f6c60009040 comp rx=0 tx=0).stop 2023-10-06T05:25:25.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/3626031193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640c7f30 msgr2=0x7f6c640cad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640c7f30 0x7f6c640cad80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b610 tx=0x7f6c5c018000 comp rx=0 tx=0).stop 2023-10-06T05:25:25.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/3626031193 shutdown_connections 2023-10-06T05:25:25.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6c48067680 0x7f6c48069b30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c640c7f30 0x7f6c640cad80 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c640aee70 0x7f6c64155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 --2- 172.21.15.31:0/3626031193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c640c7b60 0x7f6c640c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/3626031193 >> 172.21.15.31:0/3626031193 conn(0x7f6c640cc0b0 msgr2=0x7f6c6413e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:25.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/3626031193 shutdown_connections 2023-10-06T05:25:25.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.718+0000 7f6c6c1ff700 1 -- 172.21.15.31:0/3626031193 wait complete. 2023-10-06T05:25:25.738 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:25.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.932+0000 7fbdf9bff700 1 Processor -- start 2023-10-06T05:25:25.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.933+0000 7fbdf9bff700 1 -- start start 2023-10-06T05:25:25.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.934+0000 7fbdf9bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf41595a0 0x7fbdf4159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.934+0000 7fbdf9bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbdf4068d80 0x7fbdf40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.934+0000 7fbdf9bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf40c95b0 con 0x7fbdf40cc0d0 2023-10-06T05:25:25.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.934+0000 7fbdf9bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf40c9890 con 0x7fbdf4068d80 2023-10-06T05:25:25.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.934+0000 7fbdf9bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf40c9b70 con 0x7fbdf41595a0 2023-10-06T05:25:25.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.934+0000 7fbdf3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbdf4068d80 0x7fbdf40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.934+0000 7fbdf3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbdf4068d80 0x7fbdf40c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50934/0 (socket says 172.21.15.31:50934) 2023-10-06T05:25:25.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.934+0000 7fbdf3fff700 1 -- 172.21.15.31:0/64192791 learned_addr learned my addr 172.21.15.31:0/64192791 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:25.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.934+0000 7fbdf37fe700 1 --2- 172.21.15.31:0/64192791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf41595a0 0x7fbdf4159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.935+0000 7fbdf3fff700 1 -- 172.21.15.31:0/64192791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf41595a0 msgr2=0x7fbdf4159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:25.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.935+0000 7fbdf3fff700 1 --2- 172.21.15.31:0/64192791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf41595a0 0x7fbdf4159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.935+0000 7fbdf3fff700 1 -- 172.21.15.31:0/64192791 >> v1:172.21.15.62:6789/0 conn(0x7fbdf40cc0d0 legacy=0x7fbdf4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.935+0000 7fbdf3fff700 1 -- 172.21.15.31:0/64192791 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf40ca130 con 0x7fbdf4068d80 2023-10-06T05:25:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.935+0000 7fbdf3fff700 1 --2- 172.21.15.31:0/64192791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf4068d80 0x7fbdf40c8930 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009f20 tx=0x7fbdec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350ed44fecef08d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.935+0000 7fbdf27fc700 1 -- 172.21.15.31:0/64192791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00e070 con 0x7fbdf4068d80 2023-10-06T05:25:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.935+0000 7fbdf27fc700 1 -- 172.21.15.31:0/64192791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdec00b9d0 con 0x7fbdf4068d80 2023-10-06T05:25:25.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.935+0000 7fbdf27fc700 1 -- 172.21.15.31:0/64192791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0092e0 con 0x7fbdf4068d80 2023-10-06T05:25:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.936+0000 7fbdf9bff700 1 -- 172.21.15.31:0/64192791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf4068d80 msgr2=0x7fbdf40c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.936+0000 7fbdf9bff700 1 --2- 172.21.15.31:0/64192791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf4068d80 0x7fbdf40c8930 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009f20 tx=0x7fbdec009840 comp rx=0 tx=0).stop 2023-10-06T05:25:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.936+0000 7fbdf9bff700 1 -- 172.21.15.31:0/64192791 shutdown_connections 2023-10-06T05:25:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.936+0000 7fbdf9bff700 1 --2- 172.21.15.31:0/64192791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf4068d80 0x7fbdf40c8930 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.936+0000 7fbdf9bff700 1 --2- 172.21.15.31:0/64192791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf41595a0 0x7fbdf4159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.936+0000 7fbdf9bff700 1 -- 172.21.15.31:0/64192791 >> 172.21.15.31:0/64192791 conn(0x7fbdf406b620 msgr2=0x7fbdf406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.936+0000 7fbdf9bff700 1 -- 172.21.15.31:0/64192791 shutdown_connections 2023-10-06T05:25:25.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.936+0000 7fbdf9bff700 1 -- 172.21.15.31:0/64192791 wait complete. 2023-10-06T05:25:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.938+0000 7fbdf9bff700 1 Processor -- start 2023-10-06T05:25:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.938+0000 7fbdf9bff700 1 -- start start 2023-10-06T05:25:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf9bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf4068d80 0x7fbdf4169d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf9bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf40cc0d0 0x7fbdf416d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf9bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf41595a0 0x7fbdf41519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf9bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf416ace0 con 0x7fbdf41595a0 2023-10-06T05:25:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf9bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf40b2e70 con 0x7fbdf40cc0d0 2023-10-06T05:25:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf9bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf4158e30 con 0x7fbdf4068d80 2023-10-06T05:25:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf41595a0 0x7fbdf41519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf41595a0 0x7fbdf41519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50946/0 (socket says 172.21.15.31:50946) 2023-10-06T05:25:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf3fff700 1 -- 172.21.15.31:0/1510291412 learned_addr learned my addr 172.21.15.31:0/1510291412 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf37fe700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf4068d80 0x7fbdf4169d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf2ffd700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf40cc0d0 0x7fbdf416d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf3fff700 1 -- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf4068d80 msgr2=0x7fbdf4169d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf3fff700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf4068d80 0x7fbdf4169d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf3fff700 1 -- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf40cc0d0 msgr2=0x7fbdf416d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf3fff700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf40cc0d0 0x7fbdf416d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf3fff700 1 -- 172.21.15.31:0/1510291412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec009ae0 con 0x7fbdf41595a0 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf37fe700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf4068d80 0x7fbdf4169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.940+0000 7fbdf2ffd700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf40cc0d0 0x7fbdf416d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.941+0000 7fbdf3fff700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf41595a0 0x7fbdf41519f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406ee00 tx=0x7fbdec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.941+0000 7fbddb7fe700 1 -- 172.21.15.31:0/1510291412 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec008c30 con 0x7fbdf41595a0 2023-10-06T05:25:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.941+0000 7fbddb7fe700 1 -- 172.21.15.31:0/1510291412 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdec008d90 con 0x7fbdf41595a0 2023-10-06T05:25:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.941+0000 7fbddb7fe700 1 -- 172.21.15.31:0/1510291412 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec01c660 con 0x7fbdf41595a0 2023-10-06T05:25:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.941+0000 7fbdf9bff700 1 -- 172.21.15.31:0/1510291412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf416da40 con 0x7fbdf41595a0 2023-10-06T05:25:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.941+0000 7fbdf9bff700 1 -- 172.21.15.31:0/1510291412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf416e000 con 0x7fbdf41595a0 2023-10-06T05:25:25.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.942+0000 7fbdf9bff700 1 -- 172.21.15.31:0/1510291412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf40b3000 con 0x7fbdf41595a0 2023-10-06T05:25:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.943+0000 7fbddb7fe700 1 -- 172.21.15.31:0/1510291412 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fbdec003040 con 0x7fbdf41595a0 2023-10-06T05:25:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.944+0000 7fbddb7fe700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbde0067680 0x7fbde0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.944+0000 7fbddb7fe700 1 -- 172.21.15.31:0/1510291412 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fbdec087590 con 0x7fbdf41595a0 2023-10-06T05:25:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.944+0000 7fbdf37fe700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbde0067680 0x7fbde0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.945+0000 7fbdf37fe700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbde0067680 0x7fbde0069b30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbde4006f00 tx=0x7fbde4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:25.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:25.948+0000 7fbddb7fe700 1 -- 172.21.15.31:0/1510291412 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdec084840 con 0x7fbdf41595a0 2023-10-06T05:25:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.153+0000 7fbdf9bff700 1 -- 172.21.15.31:0/1510291412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdf404c380 con 0x7fbdf41595a0 2023-10-06T05:25:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.154+0000 7fbddb7fe700 1 -- 172.21.15.31:0/1510291412 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16985 (secure 0 0 0) 0x7fbdec053630 con 0x7fbdf41595a0 2023-10-06T05:25:26.156 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:26.157 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":149,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:25:24.746365+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:25:22.964655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:23.719177+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 -- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbde0067680 msgr2=0x7fbde0069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbde0067680 0x7fbde0069b30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbde4006f00 tx=0x7fbde4008040 comp rx=0 tx=0).stop 2023-10-06T05:25:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 -- 172.21.15.31:0/1510291412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf41595a0 msgr2=0x7fbdf41519f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf41595a0 0x7fbdf41519f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406ee00 tx=0x7fbdec018000 comp rx=0 tx=0).stop 2023-10-06T05:25:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 -- 172.21.15.31:0/1510291412 shutdown_connections 2023-10-06T05:25:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbde0067680 0x7fbde0069b30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf41595a0 0x7fbdf41519f0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbdf4068d80 0x7fbdf4169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 --2- 172.21.15.31:0/1510291412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdf40cc0d0 0x7fbdf416d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 -- 172.21.15.31:0/1510291412 >> 172.21.15.31:0/1510291412 conn(0x7fbdf406b620 msgr2=0x7fbdf40cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 -- 172.21.15.31:0/1510291412 shutdown_connections 2023-10-06T05:25:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.157+0000 7fbdf9bff700 1 -- 172.21.15.31:0/1510291412 wait complete. 2023-10-06T05:25:26.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:25:26.223 INFO:tasks.ceph.mds.a.smithi031.stderr:2023-10-06T05:25:26.221+0000 7f686b5f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:25:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.375+0000 7ff796bff700 1 Processor -- start 2023-10-06T05:25:26.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.375+0000 7ff796bff700 1 -- start start 2023-10-06T05:25:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.376+0000 7ff796bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff790159560 0x7ff7900cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.376+0000 7ff796bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7900b2b80 0x7ff7900cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:26.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.376+0000 7ff796bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79015d2d0 con 0x7ff790159930 2023-10-06T05:25:26.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.376+0000 7ff796bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79015d5b0 con 0x7ff790159560 2023-10-06T05:25:26.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.376+0000 7ff796bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79015d890 con 0x7ff7900b2b80 2023-10-06T05:25:26.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.376+0000 7ff79499b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff790159560 0x7ff7900cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:26.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.377+0000 7ff79499b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff790159560 0x7ff7900cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50952/0 (socket says 172.21.15.31:50952) 2023-10-06T05:25:26.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.377+0000 7ff79499b700 1 -- 172.21.15.31:0/278289192 learned_addr learned my addr 172.21.15.31:0/278289192 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:26.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.377+0000 7ff78ffff700 1 --1- 172.21.15.31:0/278289192 >> v1:172.21.15.31:6789/0 conn(0x7ff790159930 0x7ff7900b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36812/0 (socket says 172.21.15.31:36812) 2023-10-06T05:25:26.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.377+0000 7ff79519c700 1 --2- 172.21.15.31:0/278289192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7900b2b80 0x7ff7900cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.377+0000 7ff79499b700 1 -- 172.21.15.31:0/278289192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7900b2b80 msgr2=0x7ff7900cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.377+0000 7ff79499b700 1 --2- 172.21.15.31:0/278289192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7900b2b80 0x7ff7900cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.377+0000 7ff79499b700 1 -- 172.21.15.31:0/278289192 >> v1:172.21.15.31:6789/0 conn(0x7ff790159930 legacy=0x7ff7900b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.377+0000 7ff79499b700 1 -- 172.21.15.31:0/278289192 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79015db70 con 0x7ff790159560 2023-10-06T05:25:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.377+0000 7ff78f7fe700 1 -- 172.21.15.31:0/278289192 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113640714 0 0) 0x7ff79015d2d0 con 0x7ff790159930 2023-10-06T05:25:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.377+0000 7ff79499b700 1 --2- 172.21.15.31:0/278289192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159560 0x7ff7900cce70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ff7840099b0 tx=0x7ff784009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce31ffc7306c9f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff78f7fe700 1 -- 172.21.15.31:0/278289192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400b860 con 0x7ff790159560 2023-10-06T05:25:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff78f7fe700 1 -- 172.21.15.31:0/278289192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78400b9c0 con 0x7ff790159560 2023-10-06T05:25:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff78f7fe700 1 -- 172.21.15.31:0/278289192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400bd60 con 0x7ff790159560 2023-10-06T05:25:26.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff796bff700 1 -- 172.21.15.31:0/278289192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159560 msgr2=0x7ff7900cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff796bff700 1 --2- 172.21.15.31:0/278289192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159560 0x7ff7900cce70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ff7840099b0 tx=0x7ff784009250 comp rx=0 tx=0).stop 2023-10-06T05:25:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff796bff700 1 -- 172.21.15.31:0/278289192 shutdown_connections 2023-10-06T05:25:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff796bff700 1 --2- 172.21.15.31:0/278289192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159560 0x7ff7900cce70 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff796bff700 1 --2- 172.21.15.31:0/278289192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7900b2b80 0x7ff7900cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff796bff700 1 -- 172.21.15.31:0/278289192 >> 172.21.15.31:0/278289192 conn(0x7ff79006ec90 msgr2=0x7ff79006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff796bff700 1 -- 172.21.15.31:0/278289192 shutdown_connections 2023-10-06T05:25:26.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.378+0000 7ff796bff700 1 -- 172.21.15.31:0/278289192 wait complete. 2023-10-06T05:25:26.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.380+0000 7ff796bff700 1 Processor -- start 2023-10-06T05:25:26.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.380+0000 7ff796bff700 1 -- start start 2023-10-06T05:25:26.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.381+0000 7ff796bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7900b2b80 0x7ff790153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.381+0000 7ff796bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff790159560 0x7ff790169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.381+0000 7ff796bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159930 0x7ff79016a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.381+0000 7ff796bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900d0bb0 con 0x7ff790159930 2023-10-06T05:25:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.381+0000 7ff796bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79016b560 con 0x7ff7900b2b80 2023-10-06T05:25:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.381+0000 7ff796bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79016a920 con 0x7ff790159560 2023-10-06T05:25:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.381+0000 7ff79519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159930 0x7ff79016a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.381+0000 7ff79519c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159930 0x7ff79016a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50956/0 (socket says 172.21.15.31:50956) 2023-10-06T05:25:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.381+0000 7ff79519c700 1 -- 172.21.15.31:0/3201334431 learned_addr learned my addr 172.21.15.31:0/3201334431 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff79519c700 1 -- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff790159560 msgr2=0x7ff790169d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:26.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff79499b700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7900b2b80 0x7ff790153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff78ffff700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff790159560 0x7ff790169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff79519c700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff790159560 0x7ff790169d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff79519c700 1 -- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7900b2b80 msgr2=0x7ff790153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff79519c700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7900b2b80 0x7ff790153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff79519c700 1 -- 172.21.15.31:0/3201334431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7840094f0 con 0x7ff790159930 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff79499b700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7900b2b80 0x7ff790153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff78ffff700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff790159560 0x7ff790169d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff79519c700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159930 0x7ff79016a280 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7ff788009fa0 tx=0x7ff788009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff78cff9700 1 -- 172.21.15.31:0/3201334431 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800ed20 con 0x7ff790159930 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff78cff9700 1 -- 172.21.15.31:0/3201334431 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800ee80 con 0x7ff790159930 2023-10-06T05:25:26.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff796bff700 1 -- 172.21.15.31:0/3201334431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79016ac00 con 0x7ff790159930 2023-10-06T05:25:26.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff78cff9700 1 -- 172.21.15.31:0/3201334431 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788014670 con 0x7ff790159930 2023-10-06T05:25:26.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.382+0000 7ff796bff700 1 -- 172.21.15.31:0/3201334431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79016b1f0 con 0x7ff790159930 2023-10-06T05:25:26.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.383+0000 7ff796bff700 1 -- 172.21.15.31:0/3201334431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79016b6d0 con 0x7ff790159930 2023-10-06T05:25:26.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.384+0000 7ff78cff9700 1 -- 172.21.15.31:0/3201334431 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7ff7880149e0 con 0x7ff790159930 2023-10-06T05:25:26.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.385+0000 7ff78cff9700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff774067680 0x7ff774069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:26.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.385+0000 7ff78cff9700 1 -- 172.21.15.31:0/3201334431 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff788085f70 con 0x7ff790159930 2023-10-06T05:25:26.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.386+0000 7ff79499b700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff774067680 0x7ff774069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:26.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.387+0000 7ff79499b700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff774067680 0x7ff774069b30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff7840099b0 tx=0x7ff784017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:26.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.389+0000 7ff78cff9700 1 -- 172.21.15.31:0/3201334431 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff788083330 con 0x7ff790159930 2023-10-06T05:25:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:26.638+0000 7ff796bff700 1 -- 172.21.15.31:0/3201334431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff79004c380 con 0x7ff790159930 2023-10-06T05:25:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.266+0000 7ff78cff9700 1 -- 172.21.15.31:0/3201334431 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v101) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff788055bb0 con 0x7ff790159930 2023-10-06T05:25:27.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.269+0000 7ff796bff700 1 -- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff774067680 msgr2=0x7ff774069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:27.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.269+0000 7ff796bff700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff774067680 0x7ff774069b30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff7840099b0 tx=0x7ff784017000 comp rx=0 tx=0).stop 2023-10-06T05:25:27.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.269+0000 7ff796bff700 1 -- 172.21.15.31:0/3201334431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159930 msgr2=0x7ff79016a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:27.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.269+0000 7ff796bff700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159930 0x7ff79016a280 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7ff788009fa0 tx=0x7ff788009900 comp rx=0 tx=0).stop 2023-10-06T05:25:27.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.270+0000 7ff796bff700 1 -- 172.21.15.31:0/3201334431 shutdown_connections 2023-10-06T05:25:27.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.270+0000 7ff796bff700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff774067680 0x7ff774069b30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.270+0000 7ff796bff700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff790159930 0x7ff79016a280 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.270+0000 7ff796bff700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7900b2b80 0x7ff790153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.270+0000 7ff796bff700 1 --2- 172.21.15.31:0/3201334431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff790159560 0x7ff790169d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.270+0000 7ff796bff700 1 -- 172.21.15.31:0/3201334431 >> 172.21.15.31:0/3201334431 conn(0x7ff79006ec90 msgr2=0x7ff7901568b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:27.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.270+0000 7ff796bff700 1 -- 172.21.15.31:0/3201334431 shutdown_connections 2023-10-06T05:25:27.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.270+0000 7ff796bff700 1 -- 172.21.15.31:0/3201334431 wait complete. 2023-10-06T05:25:27.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:25:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.535+0000 7fab269ff700 1 Processor -- start 2023-10-06T05:25:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.535+0000 7fab269ff700 1 -- start start 2023-10-06T05:25:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.536+0000 7fab269ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fab2015ca40 0x7fab20068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.536+0000 7fab269ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2006a300 0x7fab200c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.536+0000 7fab269ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab200c8d70 0x7fab2006cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.536+0000 7fab269ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab200cabc0 con 0x7fab200c8d70 2023-10-06T05:25:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.536+0000 7fab269ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab20068bb0 con 0x7fab2015ca40 2023-10-06T05:25:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.536+0000 7fab269ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab200c91f0 con 0x7fab2006a300 2023-10-06T05:25:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab24f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab200c8d70 0x7fab2006cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab1f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2006a300 0x7fab200c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab1ffff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fab2015ca40 0x7fab20068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab24f9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab200c8d70 0x7fab2006cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47100/0 (socket says 172.21.15.31:47100) 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab24f9c700 1 -- 172.21.15.31:0/3887078679 learned_addr learned my addr 172.21.15.31:0/3887078679 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab1f7fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2006a300 0x7fab200c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:47996/0 (socket says 172.21.15.31:47996) 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab24f9c700 1 -- 172.21.15.31:0/3887078679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2006a300 msgr2=0x7fab200c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab24f9c700 1 --2- 172.21.15.31:0/3887078679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2006a300 0x7fab200c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab24f9c700 1 -- 172.21.15.31:0/3887078679 >> v2:172.21.15.62:3300/0 conn(0x7fab2015ca40 msgr2=0x7fab20068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab24f9c700 1 --2- 172.21.15.31:0/3887078679 >> v2:172.21.15.62:3300/0 conn(0x7fab2015ca40 0x7fab20068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab24f9c700 1 -- 172.21.15.31:0/3887078679 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab20068da0 con 0x7fab200c8d70 2023-10-06T05:25:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.537+0000 7fab24f9c700 1 --2- 172.21.15.31:0/3887078679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 0x7fab2006cf40 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fab20067f80 tx=0x7fab18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6503310fd86256 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab1effd700 1 -- 172.21.15.31:0/3887078679 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1800e070 con 0x7fab200c8d70 2023-10-06T05:25:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab1effd700 1 -- 172.21.15.31:0/3887078679 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1800b930 con 0x7fab200c8d70 2023-10-06T05:25:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab1effd700 1 -- 172.21.15.31:0/3887078679 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab180092e0 con 0x7fab200c8d70 2023-10-06T05:25:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab269ff700 1 -- 172.21.15.31:0/3887078679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 msgr2=0x7fab2006cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab269ff700 1 --2- 172.21.15.31:0/3887078679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 0x7fab2006cf40 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fab20067f80 tx=0x7fab18009840 comp rx=0 tx=0).stop 2023-10-06T05:25:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab269ff700 1 -- 172.21.15.31:0/3887078679 shutdown_connections 2023-10-06T05:25:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab269ff700 1 --2- 172.21.15.31:0/3887078679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 0x7fab2006cf40 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab269ff700 1 --2- 172.21.15.31:0/3887078679 >> v2:172.21.15.62:3300/0 conn(0x7fab2015ca40 0x7fab20068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab269ff700 1 --2- 172.21.15.31:0/3887078679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2006a300 0x7fab200c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab269ff700 1 -- 172.21.15.31:0/3887078679 >> 172.21.15.31:0/3887078679 conn(0x7fab200ce290 msgr2=0x7fab200ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab269ff700 1 -- 172.21.15.31:0/3887078679 shutdown_connections 2023-10-06T05:25:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.538+0000 7fab269ff700 1 -- 172.21.15.31:0/3887078679 wait complete. 2023-10-06T05:25:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.540+0000 7fab269ff700 1 Processor -- start 2023-10-06T05:25:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.540+0000 7fab269ff700 1 -- start start 2023-10-06T05:25:27.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab269ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2006a300 0x7fab200c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:27.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab269ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 0x7fab20159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:27.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab269ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015ca40 0x7fab200af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:27.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab269ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab200c2ae0 con 0x7fab200c8d70 2023-10-06T05:25:27.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab269ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab200c4d80 con 0x7fab2006a300 2023-10-06T05:25:27.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab269ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab200b2690 con 0x7fab2015ca40 2023-10-06T05:25:27.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab1f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 0x7fab20159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:27.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab1f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 0x7fab20159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47110/0 (socket says 172.21.15.31:47110) 2023-10-06T05:25:27.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab1f7fe700 1 -- 172.21.15.31:0/3432530479 learned_addr learned my addr 172.21.15.31:0/3432530479 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab1ffff700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2006a300 0x7fab200c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.541+0000 7fab24f9c700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015ca40 0x7fab200af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab1f7fe700 1 -- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015ca40 msgr2=0x7fab200af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab1f7fe700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015ca40 0x7fab200af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab1f7fe700 1 -- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2006a300 msgr2=0x7fab200c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab1f7fe700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2006a300 0x7fab200c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab1f7fe700 1 -- 172.21.15.31:0/3432530479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18009ae0 con 0x7fab200c8d70 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab24f9c700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015ca40 0x7fab200af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab1f7fe700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 0x7fab20159760 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fab1400e950 tx=0x7fab140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab07fff700 1 -- 172.21.15.31:0/3432530479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1400c850 con 0x7fab200c8d70 2023-10-06T05:25:27.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab07fff700 1 -- 172.21.15.31:0/3432530479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1400c9b0 con 0x7fab200c8d70 2023-10-06T05:25:27.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab07fff700 1 -- 172.21.15.31:0/3432530479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1400cdb0 con 0x7fab200c8d70 2023-10-06T05:25:27.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab269ff700 1 -- 172.21.15.31:0/3432530479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab200c2cc0 con 0x7fab200c8d70 2023-10-06T05:25:27.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.542+0000 7fab269ff700 1 -- 172.21.15.31:0/3432530479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab200c3280 con 0x7fab200c8d70 2023-10-06T05:25:27.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.543+0000 7fab269ff700 1 -- 172.21.15.31:0/3432530479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2006fb10 con 0x7fab200c8d70 2023-10-06T05:25:27.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.544+0000 7fab07fff700 1 -- 172.21.15.31:0/3432530479 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fab14013680 con 0x7fab200c8d70 2023-10-06T05:25:27.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.545+0000 7fab07fff700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab0c067680 0x7fab0c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:27.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.545+0000 7fab07fff700 1 -- 172.21.15.31:0/3432530479 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fab14085ab0 con 0x7fab200c8d70 2023-10-06T05:25:27.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.546+0000 7fab1ffff700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab0c067680 0x7fab0c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:27.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.546+0000 7fab1ffff700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab0c067680 0x7fab0c069b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fab10009d40 tx=0x7fab10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:27.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.549+0000 7fab07fff700 1 -- 172.21.15.31:0/3432530479 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab14082e70 con 0x7fab200c8d70 2023-10-06T05:25:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:27.794+0000 7fab269ff700 1 -- 172.21.15.31:0/3432530479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fab2004c380 con 0x7fab200c8d70 2023-10-06T05:25:28.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.275+0000 7fab07fff700 1 -- 172.21.15.31:0/3432530479 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v102) v1 ==== 122+0+0 (secure 0 0 0) 0x7fab14051c60 con 0x7fab200c8d70 2023-10-06T05:25:28.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 -- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab0c067680 msgr2=0x7fab0c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab0c067680 0x7fab0c069b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fab10009d40 tx=0x7fab10008040 comp rx=0 tx=0).stop 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 -- 172.21.15.31:0/3432530479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 msgr2=0x7fab20159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 0x7fab20159760 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fab1400e950 tx=0x7fab140091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 -- 172.21.15.31:0/3432530479 shutdown_connections 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab0c067680 0x7fab0c069b30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015ca40 0x7fab200af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2006a300 0x7fab200c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 --2- 172.21.15.31:0/3432530479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab200c8d70 0x7fab20159760 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 -- 172.21.15.31:0/3432530479 >> 172.21.15.31:0/3432530479 conn(0x7fab200ce290 msgr2=0x7fab200695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 -- 172.21.15.31:0/3432530479 shutdown_connections 2023-10-06T05:25:28.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.278+0000 7fab269ff700 1 -- 172.21.15.31:0/3432530479 wait complete. 2023-10-06T05:25:28.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.493+0000 7f43eefff700 1 Processor -- start 2023-10-06T05:25:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.493+0000 7f43eefff700 1 -- start start 2023-10-06T05:25:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.495+0000 7f43eefff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 0x7f43e8159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.495+0000 7f43eefff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e8067f70 con 0x7f43e80c6f80 2023-10-06T05:25:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.495+0000 7f43eefff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e806e040 con 0x7f43e81596d0 2023-10-06T05:25:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.495+0000 7f43eefff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e806b040 con 0x7f43e80c1df0 2023-10-06T05:25:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.495+0000 7f43e7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f43e80c6f80 0x7f43e80c16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59232/0 (socket says 172.21.15.31:59232) 2023-10-06T05:25:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.495+0000 7f43e7fff700 1 -- 172.21.15.31:0/726876870 learned_addr learned my addr 172.21.15.31:0/726876870 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.495+0000 7f43ed59c700 1 --2- 172.21.15.31:0/726876870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 0x7f43e8159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.495+0000 7f43e77fe700 1 -- 172.21.15.31:0/726876870 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167734191 0 0) 0x7f43e8067f70 con 0x7f43e80c6f80 2023-10-06T05:25:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.495+0000 7f43e77fe700 1 -- 172.21.15.31:0/726876870 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d8003570 con 0x7f43e80c6f80 2023-10-06T05:25:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.496+0000 7f43e77fe700 1 -- 172.21.15.31:0/726876870 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3671382376 0 0) 0x7f43e806e040 con 0x7f43e81596d0 2023-10-06T05:25:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.496+0000 7f43e77fe700 1 -- 172.21.15.31:0/726876870 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43e8067f70 con 0x7f43e81596d0 2023-10-06T05:25:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.496+0000 7f43ed59c700 1 -- 172.21.15.31:0/726876870 >> v1:172.21.15.62:6789/0 conn(0x7f43e81596d0 legacy=0x7f43e806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.496+0000 7f43ed59c700 1 -- 172.21.15.31:0/726876870 >> v1:172.21.15.31:6789/0 conn(0x7f43e80c6f80 legacy=0x7f43e80c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.496+0000 7f43e77fe700 1 -- 172.21.15.31:0/726876870 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 215788498 0 0) 0x7f43d8003570 con 0x7f43e80c6f80 2023-10-06T05:25:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.496+0000 7f43ed59c700 1 -- 172.21.15.31:0/726876870 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e80681c0 con 0x7f43e80c1df0 2023-10-06T05:25:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.496+0000 7f43ed59c700 1 --2- 172.21.15.31:0/726876870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 0x7f43e8159aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009e90 tx=0x7f43e0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683a56f77b5f7d33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.496+0000 7f43e77fe700 1 -- 172.21.15.31:0/726876870 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000e070 con 0x7f43e80c1df0 2023-10-06T05:25:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.496+0000 7f43e77fe700 1 -- 172.21.15.31:0/726876870 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e000b870 con 0x7f43e80c1df0 2023-10-06T05:25:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.496+0000 7f43e77fe700 1 -- 172.21.15.31:0/726876870 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4001470 con 0x7f43e80c1df0 2023-10-06T05:25:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.497+0000 7f43eefff700 1 -- 172.21.15.31:0/726876870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 msgr2=0x7f43e8159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.497+0000 7f43eefff700 1 --2- 172.21.15.31:0/726876870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 0x7f43e8159aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009e90 tx=0x7f43e0009780 comp rx=0 tx=0).stop 2023-10-06T05:25:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.497+0000 7f43eefff700 1 -- 172.21.15.31:0/726876870 shutdown_connections 2023-10-06T05:25:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.497+0000 7f43eefff700 1 --2- 172.21.15.31:0/726876870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 0x7f43e8159aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.497+0000 7f43eefff700 1 -- 172.21.15.31:0/726876870 >> 172.21.15.31:0/726876870 conn(0x7f43e80c9e80 msgr2=0x7f43e806a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.497+0000 7f43eefff700 1 -- 172.21.15.31:0/726876870 shutdown_connections 2023-10-06T05:25:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.497+0000 7f43eefff700 1 -- 172.21.15.31:0/726876870 wait complete. 2023-10-06T05:25:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.498+0000 7f43eefff700 1 Processor -- start 2023-10-06T05:25:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.499+0000 7f43eefff700 1 -- start start 2023-10-06T05:25:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.499+0000 7f43eefff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 0x7f43e8142f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.499+0000 7f43eefff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e80c6f80 0x7f43e80cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43eefff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e81596d0 0x7f43e80cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43eefff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e815cba0 con 0x7f43e81596d0 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43eefff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8169500 con 0x7f43e80c6f80 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43eefff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81434b0 con 0x7f43e80c1df0 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ed59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e81596d0 0x7f43e80cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ed59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e81596d0 0x7f43e80cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47114/0 (socket says 172.21.15.31:47114) 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ed59c700 1 -- 172.21.15.31:0/88597964 learned_addr learned my addr 172.21.15.31:0/88597964 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ecd9b700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 0x7f43e8142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43e7fff700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e80c6f80 0x7f43e80cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ed59c700 1 -- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 msgr2=0x7f43e8142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ed59c700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 0x7f43e8142f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ed59c700 1 -- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e80c6f80 msgr2=0x7f43e80cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ed59c700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e80c6f80 0x7f43e80cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ed59c700 1 -- 172.21.15.31:0/88597964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e0009a20 con 0x7f43e81596d0 2023-10-06T05:25:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ecd9b700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 0x7f43e8142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43e7fff700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e80c6f80 0x7f43e80cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.500+0000 7f43ed59c700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e81596d0 0x7f43e80cc2a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f43e0003000 tx=0x7f43e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.501+0000 7f43e4ff9700 1 -- 172.21.15.31:0/88597964 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0012d40 con 0x7f43e81596d0 2023-10-06T05:25:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.501+0000 7f43e4ff9700 1 -- 172.21.15.31:0/88597964 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e001e450 con 0x7f43e81596d0 2023-10-06T05:25:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.501+0000 7f43eefff700 1 -- 172.21.15.31:0/88597964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e815bf90 con 0x7f43e81596d0 2023-10-06T05:25:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.501+0000 7f43e4ff9700 1 -- 172.21.15.31:0/88597964 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0012ea0 con 0x7f43e81596d0 2023-10-06T05:25:28.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.501+0000 7f43eefff700 1 -- 172.21.15.31:0/88597964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e815c550 con 0x7f43e81596d0 2023-10-06T05:25:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.502+0000 7f43eefff700 1 -- 172.21.15.31:0/88597964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e804c380 con 0x7f43e81596d0 2023-10-06T05:25:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.503+0000 7f43e4ff9700 1 -- 172.21.15.31:0/88597964 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f43e001eae0 con 0x7f43e81596d0 2023-10-06T05:25:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.504+0000 7f43e4ff9700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f43cc067680 0x7f43cc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.504+0000 7f43e4ff9700 1 -- 172.21.15.31:0/88597964 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f43e0087470 con 0x7f43e81596d0 2023-10-06T05:25:28.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.505+0000 7f43ecd9b700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f43cc067680 0x7f43cc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.505+0000 7f43ecd9b700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f43cc067680 0x7f43cc069b30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f43dc003940 tx=0x7f43dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.509+0000 7f43e4ff9700 1 -- 172.21.15.31:0/88597964 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43e0084b30 con 0x7f43e81596d0 2023-10-06T05:25:28.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.757+0000 7f43eefff700 1 -- 172.21.15.31:0/88597964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43e8063760 con 0x7f43e81596d0 2023-10-06T05:25:28.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.758+0000 7f43e4ff9700 1 -- 172.21.15.31:0/88597964 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f43e0053920 con 0x7f43e81596d0 2023-10-06T05:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1489945066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1489945066},{"type":"v1","addr":"172.21.15.62:6834","nonce":1489945066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/660206899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":660206899},{"type":"v1","addr":"172.21.15.62:6835","nonce":660206899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8862,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1056123914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1056123914},{"type":"v1","addr":"172.21.15.31:6834","nonce":1056123914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:24.746699+0000","modified":"2023-10-06T05:25:28.266401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8865},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8865":{"gid":8865,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1535525741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1535525741},{"type":"v1","addr":"172.21.15.31:6835","nonce":1535525741}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:25:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.760+0000 7f43eefff700 1 -- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f43cc067680 msgr2=0x7f43cc069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.760+0000 7f43eefff700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f43cc067680 0x7f43cc069b30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f43dc003940 tx=0x7f43dc009040 comp rx=0 tx=0).stop 2023-10-06T05:25:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.760+0000 7f43eefff700 1 -- 172.21.15.31:0/88597964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e81596d0 msgr2=0x7f43e80cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.760+0000 7f43eefff700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e81596d0 0x7f43e80cc2a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f43e0003000 tx=0x7f43e0017000 comp rx=0 tx=0).stop 2023-10-06T05:25:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.761+0000 7f43eefff700 1 -- 172.21.15.31:0/88597964 shutdown_connections 2023-10-06T05:25:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.761+0000 7f43eefff700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f43cc067680 0x7f43cc069b30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.761+0000 7f43eefff700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43e81596d0 0x7f43e80cc2a0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.761+0000 7f43eefff700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e80c1df0 0x7f43e8142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.761+0000 7f43eefff700 1 --2- 172.21.15.31:0/88597964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e80c6f80 0x7f43e80cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.761+0000 7f43eefff700 1 -- 172.21.15.31:0/88597964 >> 172.21.15.31:0/88597964 conn(0x7f43e80c9e80 msgr2=0x7f43e80c7420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.761+0000 7f43eefff700 1 -- 172.21.15.31:0/88597964 shutdown_connections 2023-10-06T05:25:28.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.761+0000 7f43eefff700 1 -- 172.21.15.31:0/88597964 wait complete. 2023-10-06T05:25:28.764 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 102 2023-10-06T05:25:28.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.974+0000 7fa8915ff700 1 Processor -- start 2023-10-06T05:25:28.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.974+0000 7fa8915ff700 1 -- start start 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.975+0000 7fa8915ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa88c0c7b60 0x7fa88c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.975+0000 7fa8915ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7f30 0x7fa88c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa8915ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c0ca5b0 con 0x7fa88c06ef50 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa8915ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88c06f740 con 0x7fa88c0c7b60 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa8915ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0c9c80 con 0x7fa88c0c7f30 2023-10-06T05:25:28.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa88affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa88c0c7b60 0x7fa88c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa88affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa88c0c7b60 0x7fa88c069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47126/0 (socket says 172.21.15.31:47126) 2023-10-06T05:25:28.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa88b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa88c06ef50 0x7fa88c13f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59238/0 (socket says 172.21.15.31:59238) 2023-10-06T05:25:28.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa88affd700 1 -- 172.21.15.31:0/1524379233 learned_addr learned my addr 172.21.15.31:0/1524379233 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:28.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa88a7fc700 1 --2- 172.21.15.31:0/1524379233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7f30 0x7fa88c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa88affd700 1 -- 172.21.15.31:0/1524379233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7f30 msgr2=0x7fa88c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:28.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa88affd700 1 --2- 172.21.15.31:0/1524379233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7f30 0x7fa88c0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa88affd700 1 -- 172.21.15.31:0/1524379233 >> v1:172.21.15.31:6789/0 conn(0x7fa88c06ef50 legacy=0x7fa88c13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa88affd700 1 -- 172.21.15.31:0/1524379233 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c0c9f60 con 0x7fa88c0c7b60 2023-10-06T05:25:28.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa88a7fc700 1 --2- 172.21.15.31:0/1524379233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7f30 0x7fa88c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:28.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.976+0000 7fa889ffb700 1 -- 172.21.15.31:0/1524379233 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723072292 0 0) 0x7fa88c0ca5b0 con 0x7fa88c06ef50 2023-10-06T05:25:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.977+0000 7fa88affd700 1 --2- 172.21.15.31:0/1524379233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7b60 0x7fa88c069880 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009990 tx=0x7fa87c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17754d87ceca705 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.977+0000 7fa889ffb700 1 -- 172.21.15.31:0/1524379233 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c005c10 con 0x7fa88c0c7b60 2023-10-06T05:25:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.977+0000 7fa889ffb700 1 -- 172.21.15.31:0/1524379233 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c005d70 con 0x7fa88c0c7b60 2023-10-06T05:25:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.977+0000 7fa889ffb700 1 -- 172.21.15.31:0/1524379233 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00b880 con 0x7fa88c0c7b60 2023-10-06T05:25:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.977+0000 7fa8915ff700 1 -- 172.21.15.31:0/1524379233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7b60 msgr2=0x7fa88c069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.977+0000 7fa8915ff700 1 --2- 172.21.15.31:0/1524379233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7b60 0x7fa88c069880 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009990 tx=0x7fa87c009220 comp rx=0 tx=0).stop 2023-10-06T05:25:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.977+0000 7fa8915ff700 1 -- 172.21.15.31:0/1524379233 shutdown_connections 2023-10-06T05:25:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.977+0000 7fa8915ff700 1 --2- 172.21.15.31:0/1524379233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7b60 0x7fa88c069880 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.977+0000 7fa8915ff700 1 --2- 172.21.15.31:0/1524379233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7f30 0x7fa88c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.977+0000 7fa8915ff700 1 -- 172.21.15.31:0/1524379233 >> 172.21.15.31:0/1524379233 conn(0x7fa88c0cc0b0 msgr2=0x7fa88c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:28.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.978+0000 7fa8915ff700 1 -- 172.21.15.31:0/1524379233 shutdown_connections 2023-10-06T05:25:28.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.978+0000 7fa8915ff700 1 -- 172.21.15.31:0/1524379233 wait complete. 2023-10-06T05:25:28.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.979+0000 7fa8915ff700 1 Processor -- start 2023-10-06T05:25:28.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.979+0000 7fa8915ff700 1 -- start start 2023-10-06T05:25:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.980+0000 7fa8915ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c06ef50 0x7fa88c0c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.980+0000 7fa8915ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7b60 0x7fa88c0c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.980+0000 7fa8915ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7f30 0x7fa88c0c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.980+0000 7fa8915ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c15aad0 con 0x7fa88c0c7f30 2023-10-06T05:25:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.980+0000 7fa8915ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c15a7f0 con 0x7fa88c06ef50 2023-10-06T05:25:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.980+0000 7fa8915ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0abf70 con 0x7fa88c0c7b60 2023-10-06T05:25:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.980+0000 7fa88b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7f30 0x7fa88c0c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.980+0000 7fa88b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7f30 0x7fa88c0c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47134/0 (socket says 172.21.15.31:47134) 2023-10-06T05:25:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.980+0000 7fa88b7fe700 1 -- 172.21.15.31:0/352837566 learned_addr learned my addr 172.21.15.31:0/352837566 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa88a7fc700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7b60 0x7fa88c0c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa88affd700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c06ef50 0x7fa88c0c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa88b7fe700 1 -- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7b60 msgr2=0x7fa88c0c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa88b7fe700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7b60 0x7fa88c0c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa88b7fe700 1 -- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c06ef50 msgr2=0x7fa88c0c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa88b7fe700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c06ef50 0x7fa88c0c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa88b7fe700 1 -- 172.21.15.31:0/352837566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c009480 con 0x7fa88c0c7f30 2023-10-06T05:25:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa88a7fc700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7b60 0x7fa88c0c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa88affd700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c06ef50 0x7fa88c0c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa88b7fe700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7f30 0x7fa88c0c4d60 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa88400ccc0 tx=0x7fa884001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa872ffd700 1 -- 172.21.15.31:0/352837566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400ce00 con 0x7fa88c0c7f30 2023-10-06T05:25:28.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa872ffd700 1 -- 172.21.15.31:0/352837566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa884005df0 con 0x7fa88c0c7f30 2023-10-06T05:25:28.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa872ffd700 1 -- 172.21.15.31:0/352837566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400abb0 con 0x7fa88c0c7f30 2023-10-06T05:25:28.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa8915ff700 1 -- 172.21.15.31:0/352837566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c0ac250 con 0x7fa88c0c7f30 2023-10-06T05:25:28.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.981+0000 7fa8915ff700 1 -- 172.21.15.31:0/352837566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c157760 con 0x7fa88c0c7f30 2023-10-06T05:25:28.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.982+0000 7fa8915ff700 1 -- 172.21.15.31:0/352837566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c0abdd0 con 0x7fa88c0c7f30 2023-10-06T05:25:28.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.983+0000 7fa872ffd700 1 -- 172.21.15.31:0/352837566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fa8840070f0 con 0x7fa88c0c7f30 2023-10-06T05:25:28.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.984+0000 7fa872ffd700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa878067680 0x7fa878069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:28.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.985+0000 7fa872ffd700 1 -- 172.21.15.31:0/352837566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fa884085a10 con 0x7fa88c0c7f30 2023-10-06T05:25:28.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.985+0000 7fa88affd700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa878067680 0x7fa878069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:28.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.986+0000 7fa88affd700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa878067680 0x7fa878069b30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009260 tx=0x7fa87c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:28.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:28.988+0000 7fa872ffd700 1 -- 172.21.15.31:0/352837566 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa88c0abdd0 con 0x7fa88c0c7f30 2023-10-06T05:25:29.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.188+0000 7fa8915ff700 1 -- 172.21.15.31:0/352837566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa88c0abdd0 con 0x7fa88c0c7f30 2023-10-06T05:25:29.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.189+0000 7fa872ffd700 1 -- 172.21.15.31:0/352837566 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16972 (secure 0 0 0) 0x7fa88c0abdd0 con 0x7fa88c0c7f30 2023-10-06T05:25:29.191 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:29.191 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":150,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:25:26.253002+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:25:22.964655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:23.719177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:29.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 -- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa878067680 msgr2=0x7fa878069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa878067680 0x7fa878069b30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009260 tx=0x7fa87c016000 comp rx=0 tx=0).stop 2023-10-06T05:25:29.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 -- 172.21.15.31:0/352837566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7f30 msgr2=0x7fa88c0c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7f30 0x7fa88c0c4d60 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa88400ccc0 tx=0x7fa884001a70 comp rx=0 tx=0).stop 2023-10-06T05:25:29.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 -- 172.21.15.31:0/352837566 shutdown_connections 2023-10-06T05:25:29.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa878067680 0x7fa878069b30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa88c0c7f30 0x7fa88c0c4d60 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c06ef50 0x7fa88c0c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 --2- 172.21.15.31:0/352837566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0c7b60 0x7fa88c0c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 -- 172.21.15.31:0/352837566 >> 172.21.15.31:0/352837566 conn(0x7fa88c0cc0b0 msgr2=0x7fa88c0b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:29.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 -- 172.21.15.31:0/352837566 shutdown_connections 2023-10-06T05:25:29.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.191+0000 7fa8915ff700 1 -- 172.21.15.31:0/352837566 wait complete. 2023-10-06T05:25:29.211 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:25:29.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:25:29.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.404+0000 7fce871ff700 1 Processor -- start 2023-10-06T05:25:29.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.404+0000 7fce871ff700 1 -- start start 2023-10-06T05:25:29.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.406+0000 7fce871ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006ec30 0x7fce8014ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:29.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.406+0000 7fce871ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800c6df0 con 0x7fce8006b510 2023-10-06T05:25:29.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.406+0000 7fce871ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800c70d0 con 0x7fce800ca770 2023-10-06T05:25:29.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.406+0000 7fce871ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce800c7a70 con 0x7fce8006ec30 2023-10-06T05:25:29.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.406+0000 7fce7ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fce8006b510 0x7fce8014c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59252/0 (socket says 172.21.15.31:59252) 2023-10-06T05:25:29.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.406+0000 7fce7ffff700 1 -- 172.21.15.31:0/460375797 learned_addr learned my addr 172.21.15.31:0/460375797 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:29.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.406+0000 7fce8579c700 1 --2- 172.21.15.31:0/460375797 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006ec30 0x7fce8014ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:29.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.406+0000 7fce7f7fe700 1 -- 172.21.15.31:0/460375797 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638066078 0 0) 0x7fce800c6df0 con 0x7fce8006b510 2023-10-06T05:25:29.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.407+0000 7fce7f7fe700 1 -- 172.21.15.31:0/460375797 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce70003570 con 0x7fce8006b510 2023-10-06T05:25:29.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.407+0000 7fce7f7fe700 1 -- 172.21.15.31:0/460375797 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447278863 0 0) 0x7fce800c70d0 con 0x7fce800ca770 2023-10-06T05:25:29.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.407+0000 7fce7f7fe700 1 -- 172.21.15.31:0/460375797 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce800c6df0 con 0x7fce800ca770 2023-10-06T05:25:29.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.407+0000 7fce8579c700 1 -- 172.21.15.31:0/460375797 >> v1:172.21.15.62:6789/0 conn(0x7fce800ca770 legacy=0x7fce8006ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.407+0000 7fce8579c700 1 -- 172.21.15.31:0/460375797 >> v1:172.21.15.31:6789/0 conn(0x7fce8006b510 legacy=0x7fce8014c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.407+0000 7fce8579c700 1 -- 172.21.15.31:0/460375797 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800c73b0 con 0x7fce8006ec30 2023-10-06T05:25:29.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.407+0000 7fce8579c700 1 --2- 172.21.15.31:0/460375797 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006ec30 0x7fce8014ca10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce7800afa0 tx=0x7fce7800a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cdd69f9453bbe4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:29.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.407+0000 7fce7f7fe700 1 -- 172.21.15.31:0/460375797 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7800f070 con 0x7fce8006ec30 2023-10-06T05:25:29.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.407+0000 7fce7f7fe700 1 -- 172.21.15.31:0/460375797 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7800a2e0 con 0x7fce8006ec30 2023-10-06T05:25:29.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.407+0000 7fce7f7fe700 1 -- 172.21.15.31:0/460375797 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7800cbf0 con 0x7fce8006ec30 2023-10-06T05:25:29.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.408+0000 7fce871ff700 1 -- 172.21.15.31:0/460375797 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006ec30 msgr2=0x7fce8014ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.408+0000 7fce871ff700 1 --2- 172.21.15.31:0/460375797 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006ec30 0x7fce8014ca10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce7800afa0 tx=0x7fce7800a890 comp rx=0 tx=0).stop 2023-10-06T05:25:29.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.408+0000 7fce871ff700 1 -- 172.21.15.31:0/460375797 shutdown_connections 2023-10-06T05:25:29.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.408+0000 7fce871ff700 1 --2- 172.21.15.31:0/460375797 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006ec30 0x7fce8014ca10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.408+0000 7fce871ff700 1 -- 172.21.15.31:0/460375797 >> 172.21.15.31:0/460375797 conn(0x7fce800cc0b0 msgr2=0x7fce800cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:29.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.408+0000 7fce871ff700 1 -- 172.21.15.31:0/460375797 shutdown_connections 2023-10-06T05:25:29.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.408+0000 7fce871ff700 1 -- 172.21.15.31:0/460375797 wait complete. 2023-10-06T05:25:29.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.409+0000 7fce871ff700 1 Processor -- start 2023-10-06T05:25:29.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.410+0000 7fce871ff700 1 -- start start 2023-10-06T05:25:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.410+0000 7fce871ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006b510 0x7fce801412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.410+0000 7fce871ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8006ec30 0x7fce800ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.410+0000 7fce871ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce800ca770 0x7fce800c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.410+0000 7fce871ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce801417f0 con 0x7fce800ca770 2023-10-06T05:25:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.410+0000 7fce871ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce800ad090 con 0x7fce8006ec30 2023-10-06T05:25:29.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.410+0000 7fce871ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce800ada60 con 0x7fce8006b510 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce8579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce800ca770 0x7fce800c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce8579c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce800ca770 0x7fce800c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47138/0 (socket says 172.21.15.31:47138) 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce8579c700 1 -- 172.21.15.31:0/1385027320 learned_addr learned my addr 172.21.15.31:0/1385027320 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce7ffff700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8006ec30 0x7fce800ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce84f9b700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006b510 0x7fce801412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce8579c700 1 -- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006b510 msgr2=0x7fce801412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce8579c700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006b510 0x7fce801412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce8579c700 1 -- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8006ec30 msgr2=0x7fce800ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce8579c700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8006ec30 0x7fce800ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce8579c700 1 -- 172.21.15.31:0/1385027320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7800ab30 con 0x7fce800ca770 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.411+0000 7fce7ffff700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8006ec30 0x7fce800ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:29.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.412+0000 7fce8579c700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce800ca770 0x7fce800c6450 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fce7800ab00 tx=0x7fce78019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:29.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.412+0000 7fce7cff9700 1 -- 172.21.15.31:0/1385027320 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78009bd0 con 0x7fce800ca770 2023-10-06T05:25:29.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.412+0000 7fce7cff9700 1 -- 172.21.15.31:0/1385027320 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce78009d70 con 0x7fce800ca770 2023-10-06T05:25:29.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.412+0000 7fce7cff9700 1 -- 172.21.15.31:0/1385027320 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78027660 con 0x7fce800ca770 2023-10-06T05:25:29.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.412+0000 7fce871ff700 1 -- 172.21.15.31:0/1385027320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce800ac480 con 0x7fce800ca770 2023-10-06T05:25:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.412+0000 7fce871ff700 1 -- 172.21.15.31:0/1385027320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce800aca40 con 0x7fce800ca770 2023-10-06T05:25:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.413+0000 7fce871ff700 1 -- 172.21.15.31:0/1385027320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8004c380 con 0x7fce800ca770 2023-10-06T05:25:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.414+0000 7fce7cff9700 1 -- 172.21.15.31:0/1385027320 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fce78027a10 con 0x7fce800ca770 2023-10-06T05:25:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.415+0000 7fce7cff9700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce64067680 0x7fce64069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.415+0000 7fce7cff9700 1 -- 172.21.15.31:0/1385027320 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fce78087180 con 0x7fce800ca770 2023-10-06T05:25:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.415+0000 7fce84f9b700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce64067680 0x7fce64069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.416+0000 7fce84f9b700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce64067680 0x7fce64069b30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fce74003a40 tx=0x7fce74007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:29.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.419+0000 7fce7cff9700 1 -- 172.21.15.31:0/1385027320 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce78084430 con 0x7fce800ca770 2023-10-06T05:25:29.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.618+0000 7fce871ff700 1 -- 172.21.15.31:0/1385027320 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce800ad1d0 con 0x7fce64067680 2023-10-06T05:25:29.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.628+0000 7fce7cff9700 1 -- 172.21.15.31:0/1385027320 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311791 (secure 0 0 0) 0x7fce800ad1d0 con 0x7fce64067680 2023-10-06T05:25:29.634 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:29.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 -- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce64067680 msgr2=0x7fce64069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce64067680 0x7fce64069b30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fce74003a40 tx=0x7fce74007d40 comp rx=0 tx=0).stop 2023-10-06T05:25:29.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 -- 172.21.15.31:0/1385027320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce800ca770 msgr2=0x7fce800c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce800ca770 0x7fce800c6450 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fce7800ab00 tx=0x7fce78019000 comp rx=0 tx=0).stop 2023-10-06T05:25:29.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 -- 172.21.15.31:0/1385027320 shutdown_connections 2023-10-06T05:25:29.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fce64067680 0x7fce64069b30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce800ca770 0x7fce800c6450 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce8006b510 0x7fce801412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 --2- 172.21.15.31:0/1385027320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce8006ec30 0x7fce800ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 -- 172.21.15.31:0/1385027320 >> 172.21.15.31:0/1385027320 conn(0x7fce800cc0b0 msgr2=0x7fce80069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:29.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 -- 172.21.15.31:0/1385027320 shutdown_connections 2023-10-06T05:25:29.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.633+0000 7fce871ff700 1 -- 172.21.15.31:0/1385027320 wait complete. 2023-10-06T05:25:29.638 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:25:29.653 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2023-10-06T05:25:29.518354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2426996,"kb_used_data":15924,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547724,"statfs":{"total":773094113280,"available":770608869376,"internally_reserved":0,"allocated":16306176,"data_stored":8191041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001493"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279618+0000","last_change":"2023-10-06T05:25:25.279618+0000","last_active":"2023-10-06T05:25:25.279618+0000","last_peered":"2023-10-06T05:25:25.279618+0000","last_clean":"2023-10-06T05:25:25.279618+0000","last_became_active":"2023-10-06T05:25:25.279327+0000","last_became_peered":"2023-10-06T05:25:25.279327+0000","last_unstale":"2023-10-06T05:25:25.279618+0000","last_undegraded":"2023-10-06T05:25:25.279618+0000","last_fullsized":"2023-10-06T05:25:25.279618+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:21.801742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.272362+0000","last_change":"2023-10-06T05:25:25.272362+0000","last_active":"2023-10-06T05:25:25.272362+0000","last_peered":"2023-10-06T05:25:25.272362+0000","last_clean":"2023-10-06T05:25:25.272362+0000","last_became_active":"2023-10-06T05:25:25.272106+0000","last_became_peered":"2023-10-06T05:25:25.272106+0000","last_unstale":"2023-10-06T05:25:25.272362+0000","last_undegraded":"2023-10-06T05:25:25.272362+0000","last_fullsized":"2023-10-06T05:25:25.272362+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:37.990044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276221+0000","last_change":"2023-10-06T05:25:25.276221+0000","last_active":"2023-10-06T05:25:25.276221+0000","last_peered":"2023-10-06T05:25:25.276221+0000","last_clean":"2023-10-06T05:25:25.276221+0000","last_became_active":"2023-10-06T05:25:25.275933+0000","last_became_peered":"2023-10-06T05:25:25.275933+0000","last_unstale":"2023-10-06T05:25:25.276221+0000","last_undegraded":"2023-10-06T05:25:25.276221+0000","last_fullsized":"2023-10-06T05:25:25.276221+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:57.038031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.281061+0000","last_change":"2023-10-06T05:25:25.281061+0000","last_active":"2023-10-06T05:25:25.281061+0000","last_peered":"2023-10-06T05:25:25.281061+0000","last_clean":"2023-10-06T05:25:25.281061+0000","last_became_active":"2023-10-06T05:25:25.280629+0000","last_became_peered":"2023-10-06T05:25:25.280629+0000","last_unstale":"2023-10-06T05:25:25.281061+0000","last_undegraded":"2023-10-06T05:25:25.281061+0000","last_fullsized":"2023-10-06T05:25:25.281061+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:00.819753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.275077+0000","last_change":"2023-10-06T05:25:25.266779+0000","last_active":"2023-10-06T05:25:25.275077+0000","last_peered":"2023-10-06T05:25:25.275077+0000","last_clean":"2023-10-06T05:25:25.275077+0000","last_became_active":"2023-10-06T05:25:25.266537+0000","last_became_peered":"2023-10-06T05:25:25.266537+0000","last_unstale":"2023-10-06T05:25:25.275077+0000","last_undegraded":"2023-10-06T05:25:25.275077+0000","last_fullsized":"2023-10-06T05:25:25.275077+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:58.136155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278653+0000","last_change":"2023-10-06T05:25:25.276076+0000","last_active":"2023-10-06T05:25:25.278653+0000","last_peered":"2023-10-06T05:25:25.278653+0000","last_clean":"2023-10-06T05:25:25.278653+0000","last_became_active":"2023-10-06T05:25:25.275557+0000","last_became_peered":"2023-10-06T05:25:25.275557+0000","last_unstale":"2023-10-06T05:25:25.278653+0000","last_undegraded":"2023-10-06T05:25:25.278653+0000","last_fullsized":"2023-10-06T05:25:25.278653+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:32.494803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.266658+0000","last_change":"2023-10-06T05:25:25.263637+0000","last_active":"2023-10-06T05:25:25.266658+0000","last_peered":"2023-10-06T05:25:25.266658+0000","last_clean":"2023-10-06T05:25:25.266658+0000","last_became_active":"2023-10-06T05:25:25.263361+0000","last_became_peered":"2023-10-06T05:25:25.263361+0000","last_unstale":"2023-10-06T05:25:25.266658+0000","last_undegraded":"2023-10-06T05:25:25.266658+0000","last_fullsized":"2023-10-06T05:25:25.266658+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:05.928144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.271745+0000","last_change":"2023-10-06T05:25:25.271745+0000","last_active":"2023-10-06T05:25:25.271745+0000","last_peered":"2023-10-06T05:25:25.271745+0000","last_clean":"2023-10-06T05:25:25.271745+0000","last_became_active":"2023-10-06T05:25:25.271364+0000","last_became_peered":"2023-10-06T05:25:25.271364+0000","last_unstale":"2023-10-06T05:25:25.271745+0000","last_undegraded":"2023-10-06T05:25:25.271745+0000","last_fullsized":"2023-10-06T05:25:25.271745+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:32.728256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.270002+0000","last_change":"2023-10-06T05:25:25.270002+0000","last_active":"2023-10-06T05:25:25.270002+0000","last_peered":"2023-10-06T05:25:25.270002+0000","last_clean":"2023-10-06T05:25:25.270002+0000","last_became_active":"2023-10-06T05:25:25.269691+0000","last_became_peered":"2023-10-06T05:25:25.269691+0000","last_unstale":"2023-10-06T05:25:25.270002+0000","last_undegraded":"2023-10-06T05:25:25.270002+0000","last_fullsized":"2023-10-06T05:25:25.270002+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:15.338977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278385+0000","last_change":"2023-10-06T05:25:25.274134+0000","last_active":"2023-10-06T05:25:25.278385+0000","last_peered":"2023-10-06T05:25:25.278385+0000","last_clean":"2023-10-06T05:25:25.278385+0000","last_became_active":"2023-10-06T05:25:25.273887+0000","last_became_peered":"2023-10-06T05:25:25.273887+0000","last_unstale":"2023-10-06T05:25:25.278385+0000","last_undegraded":"2023-10-06T05:25:25.278385+0000","last_fullsized":"2023-10-06T05:25:25.278385+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:07.780135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.268341+0000","last_change":"2023-10-06T05:25:25.268341+0000","last_active":"2023-10-06T05:25:25.268341+0000","last_peered":"2023-10-06T05:25:25.268341+0000","last_clean":"2023-10-06T05:25:25.268341+0000","last_became_active":"2023-10-06T05:25:25.268003+0000","last_became_peered":"2023-10-06T05:25:25.268003+0000","last_unstale":"2023-10-06T05:25:25.268341+0000","last_undegraded":"2023-10-06T05:25:25.268341+0000","last_fullsized":"2023-10-06T05:25:25.268341+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:16.299063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279352+0000","last_change":"2023-10-06T05:25:25.279352+0000","last_active":"2023-10-06T05:25:25.279352+0000","last_peered":"2023-10-06T05:25:25.279352+0000","last_clean":"2023-10-06T05:25:25.279352+0000","last_became_active":"2023-10-06T05:25:25.278891+0000","last_became_peered":"2023-10-06T05:25:25.278891+0000","last_unstale":"2023-10-06T05:25:25.279352+0000","last_undegraded":"2023-10-06T05:25:25.279352+0000","last_fullsized":"2023-10-06T05:25:25.279352+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:22.000678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279271+0000","last_change":"2023-10-06T05:25:25.268572+0000","last_active":"2023-10-06T05:25:25.279271+0000","last_peered":"2023-10-06T05:25:25.279271+0000","last_clean":"2023-10-06T05:25:25.279271+0000","last_became_active":"2023-10-06T05:25:25.268247+0000","last_became_peered":"2023-10-06T05:25:25.268247+0000","last_unstale":"2023-10-06T05:25:25.279271+0000","last_undegraded":"2023-10-06T05:25:25.279271+0000","last_fullsized":"2023-10-06T05:25:25.279271+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:28.441607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.258739+0000","last_change":"2023-10-06T05:25:25.254819+0000","last_active":"2023-10-06T05:25:25.258739+0000","last_peered":"2023-10-06T05:25:25.258739+0000","last_clean":"2023-10-06T05:25:25.258739+0000","last_became_active":"2023-10-06T05:25:25.254506+0000","last_became_peered":"2023-10-06T05:25:25.254506+0000","last_unstale":"2023-10-06T05:25:25.258739+0000","last_undegraded":"2023-10-06T05:25:25.258739+0000","last_fullsized":"2023-10-06T05:25:25.258739+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:33.338121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.282102+0000","last_change":"2023-10-06T05:25:25.282102+0000","last_active":"2023-10-06T05:25:25.282102+0000","last_peered":"2023-10-06T05:25:25.282102+0000","last_clean":"2023-10-06T05:25:25.282102+0000","last_became_active":"2023-10-06T05:25:25.281684+0000","last_became_peered":"2023-10-06T05:25:25.281684+0000","last_unstale":"2023-10-06T05:25:25.282102+0000","last_undegraded":"2023-10-06T05:25:25.282102+0000","last_fullsized":"2023-10-06T05:25:25.282102+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:16.693212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.274422+0000","last_change":"2023-10-06T05:25:25.267323+0000","last_active":"2023-10-06T05:25:25.274422+0000","last_peered":"2023-10-06T05:25:25.274422+0000","last_clean":"2023-10-06T05:25:25.274422+0000","last_became_active":"2023-10-06T05:25:25.267076+0000","last_became_peered":"2023-10-06T05:25:25.267076+0000","last_unstale":"2023-10-06T05:25:25.274422+0000","last_undegraded":"2023-10-06T05:25:25.274422+0000","last_fullsized":"2023-10-06T05:25:25.274422+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:35.837407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273034+0000","last_change":"2023-10-06T05:25:25.266216+0000","last_active":"2023-10-06T05:25:25.273034+0000","last_peered":"2023-10-06T05:25:25.273034+0000","last_clean":"2023-10-06T05:25:25.273034+0000","last_became_active":"2023-10-06T05:25:25.265897+0000","last_became_peered":"2023-10-06T05:25:25.265897+0000","last_unstale":"2023-10-06T05:25:25.273034+0000","last_undegraded":"2023-10-06T05:25:25.273034+0000","last_fullsized":"2023-10-06T05:25:25.273034+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:13.691031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278662+0000","last_change":"2023-10-06T05:25:25.266699+0000","last_active":"2023-10-06T05:25:25.278662+0000","last_peered":"2023-10-06T05:25:25.278662+0000","last_clean":"2023-10-06T05:25:25.278662+0000","last_became_active":"2023-10-06T05:25:25.265891+0000","last_became_peered":"2023-10-06T05:25:25.265891+0000","last_unstale":"2023-10-06T05:25:25.278662+0000","last_undegraded":"2023-10-06T05:25:25.278662+0000","last_fullsized":"2023-10-06T05:25:25.278662+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:02.659477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273588+0000","last_change":"2023-10-06T05:25:25.264736+0000","last_active":"2023-10-06T05:25:25.273588+0000","last_peered":"2023-10-06T05:25:25.273588+0000","last_clean":"2023-10-06T05:25:25.273588+0000","last_became_active":"2023-10-06T05:25:25.264293+0000","last_became_peered":"2023-10-06T05:25:25.264293+0000","last_unstale":"2023-10-06T05:25:25.273588+0000","last_undegraded":"2023-10-06T05:25:25.273588+0000","last_fullsized":"2023-10-06T05:25:25.273588+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:21.819526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.267966+0000","last_change":"2023-10-06T05:25:25.262232+0000","last_active":"2023-10-06T05:25:25.267966+0000","last_peered":"2023-10-06T05:25:25.267966+0000","last_clean":"2023-10-06T05:25:25.267966+0000","last_became_active":"2023-10-06T05:25:25.261971+0000","last_became_peered":"2023-10-06T05:25:25.261971+0000","last_unstale":"2023-10-06T05:25:25.267966+0000","last_undegraded":"2023-10-06T05:25:25.267966+0000","last_fullsized":"2023-10-06T05:25:25.267966+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:31.654943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.271010+0000","last_change":"2023-10-06T05:25:25.271010+0000","last_active":"2023-10-06T05:25:25.271010+0000","last_peered":"2023-10-06T05:25:25.271010+0000","last_clean":"2023-10-06T05:25:25.271010+0000","last_became_active":"2023-10-06T05:25:25.270642+0000","last_became_peered":"2023-10-06T05:25:25.270642+0000","last_unstale":"2023-10-06T05:25:25.271010+0000","last_undegraded":"2023-10-06T05:25:25.271010+0000","last_fullsized":"2023-10-06T05:25:25.271010+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:55.497869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.270418+0000","last_change":"2023-10-06T05:25:25.270418+0000","last_active":"2023-10-06T05:25:25.270418+0000","last_peered":"2023-10-06T05:25:25.270418+0000","last_clean":"2023-10-06T05:25:25.270418+0000","last_became_active":"2023-10-06T05:25:25.270088+0000","last_became_peered":"2023-10-06T05:25:25.270088+0000","last_unstale":"2023-10-06T05:25:25.270418+0000","last_undegraded":"2023-10-06T05:25:25.270418+0000","last_fullsized":"2023-10-06T05:25:25.270418+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:45.636431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.272713+0000","last_change":"2023-10-06T05:25:25.272713+0000","last_active":"2023-10-06T05:25:25.272713+0000","last_peered":"2023-10-06T05:25:25.272713+0000","last_clean":"2023-10-06T05:25:25.272713+0000","last_became_active":"2023-10-06T05:25:25.272413+0000","last_became_peered":"2023-10-06T05:25:25.272413+0000","last_unstale":"2023-10-06T05:25:25.272713+0000","last_undegraded":"2023-10-06T05:25:25.272713+0000","last_fullsized":"2023-10-06T05:25:25.272713+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:04.468911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.264064+0000","last_change":"2023-10-06T05:25:25.264064+0000","last_active":"2023-10-06T05:25:25.264064+0000","last_peered":"2023-10-06T05:25:25.264064+0000","last_clean":"2023-10-06T05:25:25.264064+0000","last_became_active":"2023-10-06T05:25:25.263751+0000","last_became_peered":"2023-10-06T05:25:25.263751+0000","last_unstale":"2023-10-06T05:25:25.264064+0000","last_undegraded":"2023-10-06T05:25:25.264064+0000","last_fullsized":"2023-10-06T05:25:25.264064+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:35.509669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.265141+0000","last_change":"2023-10-06T05:25:25.259082+0000","last_active":"2023-10-06T05:25:25.265141+0000","last_peered":"2023-10-06T05:25:25.265141+0000","last_clean":"2023-10-06T05:25:25.265141+0000","last_became_active":"2023-10-06T05:25:25.258790+0000","last_became_peered":"2023-10-06T05:25:25.258790+0000","last_unstale":"2023-10-06T05:25:25.265141+0000","last_undegraded":"2023-10-06T05:25:25.265141+0000","last_fullsized":"2023-10-06T05:25:25.265141+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:28.372575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.267319+0000","last_change":"2023-10-06T05:25:25.267319+0000","last_active":"2023-10-06T05:25:25.267319+0000","last_peered":"2023-10-06T05:25:25.267319+0000","last_clean":"2023-10-06T05:25:25.267319+0000","last_became_active":"2023-10-06T05:25:25.266840+0000","last_became_peered":"2023-10-06T05:25:25.266840+0000","last_unstale":"2023-10-06T05:25:25.267319+0000","last_undegraded":"2023-10-06T05:25:25.267319+0000","last_fullsized":"2023-10-06T05:25:25.267319+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:21.399027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.268661+0000","last_change":"2023-10-06T05:25:25.268661+0000","last_active":"2023-10-06T05:25:25.268661+0000","last_peered":"2023-10-06T05:25:25.268661+0000","last_clean":"2023-10-06T05:25:25.268661+0000","last_became_active":"2023-10-06T05:25:25.268402+0000","last_became_peered":"2023-10-06T05:25:25.268402+0000","last_unstale":"2023-10-06T05:25:25.268661+0000","last_undegraded":"2023-10-06T05:25:25.268661+0000","last_fullsized":"2023-10-06T05:25:25.268661+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:28.415161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.271407+0000","last_change":"2023-10-06T05:25:25.268099+0000","last_active":"2023-10-06T05:25:25.271407+0000","last_peered":"2023-10-06T05:25:25.271407+0000","last_clean":"2023-10-06T05:25:25.271407+0000","last_became_active":"2023-10-06T05:25:25.267749+0000","last_became_peered":"2023-10-06T05:25:25.267749+0000","last_unstale":"2023-10-06T05:25:25.271407+0000","last_undegraded":"2023-10-06T05:25:25.271407+0000","last_fullsized":"2023-10-06T05:25:25.271407+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:28.593301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.274651+0000","last_change":"2023-10-06T05:25:25.274651+0000","last_active":"2023-10-06T05:25:25.274651+0000","last_peered":"2023-10-06T05:25:25.274651+0000","last_clean":"2023-10-06T05:25:25.274651+0000","last_became_active":"2023-10-06T05:25:25.273907+0000","last_became_peered":"2023-10-06T05:25:25.273907+0000","last_unstale":"2023-10-06T05:25:25.274651+0000","last_undegraded":"2023-10-06T05:25:25.274651+0000","last_fullsized":"2023-10-06T05:25:25.274651+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:03.051773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.274662+0000","last_change":"2023-10-06T05:25:25.274662+0000","last_active":"2023-10-06T05:25:25.274662+0000","last_peered":"2023-10-06T05:25:25.274662+0000","last_clean":"2023-10-06T05:25:25.274662+0000","last_became_active":"2023-10-06T05:25:25.273703+0000","last_became_peered":"2023-10-06T05:25:25.273703+0000","last_unstale":"2023-10-06T05:25:25.274662+0000","last_undegraded":"2023-10-06T05:25:25.274662+0000","last_fullsized":"2023-10-06T05:25:25.274662+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:21.801742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.277033+0000","last_change":"2023-10-06T05:25:25.267988+0000","last_active":"2023-10-06T05:25:25.277033+0000","last_peered":"2023-10-06T05:25:25.277033+0000","last_clean":"2023-10-06T05:25:25.277033+0000","last_became_active":"2023-10-06T05:25:25.267548+0000","last_became_peered":"2023-10-06T05:25:25.267548+0000","last_unstale":"2023-10-06T05:25:25.277033+0000","last_undegraded":"2023-10-06T05:25:25.277033+0000","last_fullsized":"2023-10-06T05:25:25.277033+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:06.624399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.261751+0000","last_change":"2023-10-06T05:25:25.257552+0000","last_active":"2023-10-06T05:25:25.261751+0000","last_peered":"2023-10-06T05:25:25.261751+0000","last_clean":"2023-10-06T05:25:25.261751+0000","last_became_active":"2023-10-06T05:25:25.254730+0000","last_became_peered":"2023-10-06T05:25:25.254730+0000","last_unstale":"2023-10-06T05:25:25.261751+0000","last_undegraded":"2023-10-06T05:25:25.261751+0000","last_fullsized":"2023-10-06T05:25:25.261751+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.113900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278798+0000","last_change":"2023-10-06T05:25:25.278798+0000","last_active":"2023-10-06T05:25:25.278798+0000","last_peered":"2023-10-06T05:25:25.278798+0000","last_clean":"2023-10-06T05:25:25.278798+0000","last_became_active":"2023-10-06T05:25:25.278526+0000","last_became_peered":"2023-10-06T05:25:25.278526+0000","last_unstale":"2023-10-06T05:25:25.278798+0000","last_undegraded":"2023-10-06T05:25:25.278798+0000","last_fullsized":"2023-10-06T05:25:25.278798+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:56.175405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.253649+0000","last_change":"2023-10-06T05:25:24.265692+0000","last_active":"2023-10-06T05:25:25.253649+0000","last_peered":"2023-10-06T05:25:25.253649+0000","last_clean":"2023-10-06T05:25:25.253649+0000","last_became_active":"2023-10-06T05:25:24.265417+0000","last_became_peered":"2023-10-06T05:25:24.265417+0000","last_unstale":"2023-10-06T05:25:25.253649+0000","last_undegraded":"2023-10-06T05:25:25.253649+0000","last_fullsized":"2023-10-06T05:25:25.253649+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:33.593154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278624+0000","last_change":"2023-10-06T05:25:25.274994+0000","last_active":"2023-10-06T05:25:25.278624+0000","last_peered":"2023-10-06T05:25:25.278624+0000","last_clean":"2023-10-06T05:25:25.278624+0000","last_became_active":"2023-10-06T05:25:25.274721+0000","last_became_peered":"2023-10-06T05:25:25.274721+0000","last_unstale":"2023-10-06T05:25:25.278624+0000","last_undegraded":"2023-10-06T05:25:25.278624+0000","last_fullsized":"2023-10-06T05:25:25.278624+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:59.999831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.268807+0000","last_change":"2023-10-06T05:25:24.266665+0000","last_active":"2023-10-06T05:25:24.268807+0000","last_peered":"2023-10-06T05:25:24.268807+0000","last_clean":"2023-10-06T05:25:24.268807+0000","last_became_active":"2023-10-06T05:25:24.266401+0000","last_became_peered":"2023-10-06T05:25:24.266401+0000","last_unstale":"2023-10-06T05:25:24.268807+0000","last_undegraded":"2023-10-06T05:25:24.268807+0000","last_fullsized":"2023-10-06T05:25:24.268807+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:43.133010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273511+0000","last_change":"2023-10-06T05:25:25.264240+0000","last_active":"2023-10-06T05:25:25.273511+0000","last_peered":"2023-10-06T05:25:25.273511+0000","last_clean":"2023-10-06T05:25:25.273511+0000","last_became_active":"2023-10-06T05:25:25.263873+0000","last_became_peered":"2023-10-06T05:25:25.263873+0000","last_unstale":"2023-10-06T05:25:25.273511+0000","last_undegraded":"2023-10-06T05:25:25.273511+0000","last_fullsized":"2023-10-06T05:25:25.273511+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:53.407207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"149'3","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279258+0000","last_change":"2023-10-06T05:25:24.265708+0000","last_active":"2023-10-06T05:25:25.279258+0000","last_peered":"2023-10-06T05:25:25.279258+0000","last_clean":"2023-10-06T05:25:25.279258+0000","last_became_active":"2023-10-06T05:25:24.265420+0000","last_became_peered":"2023-10-06T05:25:24.265420+0000","last_unstale":"2023-10-06T05:25:25.279258+0000","last_undegraded":"2023-10-06T05:25:25.279258+0000","last_fullsized":"2023-10-06T05:25:25.279258+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:41.113364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.271832+0000","last_change":"2023-10-06T05:25:25.271832+0000","last_active":"2023-10-06T05:25:25.271832+0000","last_peered":"2023-10-06T05:25:25.271832+0000","last_clean":"2023-10-06T05:25:25.271832+0000","last_became_active":"2023-10-06T05:25:25.271567+0000","last_became_peered":"2023-10-06T05:25:25.271567+0000","last_unstale":"2023-10-06T05:25:25.271832+0000","last_undegraded":"2023-10-06T05:25:25.271832+0000","last_fullsized":"2023-10-06T05:25:25.271832+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:24.247757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"149'6","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.284314+0000","last_change":"2023-10-06T05:25:24.265409+0000","last_active":"2023-10-06T05:25:25.284314+0000","last_peered":"2023-10-06T05:25:25.284314+0000","last_clean":"2023-10-06T05:25:25.284314+0000","last_became_active":"2023-10-06T05:25:24.262911+0000","last_became_peered":"2023-10-06T05:25:24.262911+0000","last_unstale":"2023-10-06T05:25:25.284314+0000","last_undegraded":"2023-10-06T05:25:25.284314+0000","last_fullsized":"2023-10-06T05:25:25.284314+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:22.263580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.266626+0000","last_change":"2023-10-06T05:25:25.263652+0000","last_active":"2023-10-06T05:25:25.266626+0000","last_peered":"2023-10-06T05:25:25.266626+0000","last_clean":"2023-10-06T05:25:25.266626+0000","last_became_active":"2023-10-06T05:25:25.263322+0000","last_became_peered":"2023-10-06T05:25:25.263322+0000","last_unstale":"2023-10-06T05:25:25.266626+0000","last_undegraded":"2023-10-06T05:25:25.266626+0000","last_fullsized":"2023-10-06T05:25:25.266626+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:04.506978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"149'2","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.280097+0000","last_change":"2023-10-06T05:25:24.263326+0000","last_active":"2023-10-06T05:25:25.280097+0000","last_peered":"2023-10-06T05:25:25.280097+0000","last_clean":"2023-10-06T05:25:25.280097+0000","last_became_active":"2023-10-06T05:25:24.263036+0000","last_became_peered":"2023-10-06T05:25:24.263036+0000","last_unstale":"2023-10-06T05:25:25.280097+0000","last_undegraded":"2023-10-06T05:25:25.280097+0000","last_fullsized":"2023-10-06T05:25:25.280097+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.474397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.268838+0000","last_change":"2023-10-06T05:25:25.265525+0000","last_active":"2023-10-06T05:25:25.268838+0000","last_peered":"2023-10-06T05:25:25.268838+0000","last_clean":"2023-10-06T05:25:25.268838+0000","last_became_active":"2023-10-06T05:25:25.265257+0000","last_became_peered":"2023-10-06T05:25:25.265257+0000","last_unstale":"2023-10-06T05:25:25.268838+0000","last_undegraded":"2023-10-06T05:25:25.268838+0000","last_fullsized":"2023-10-06T05:25:25.268838+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:13.765245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.268748+0000","last_change":"2023-10-06T05:25:24.266271+0000","last_active":"2023-10-06T05:25:24.268748+0000","last_peered":"2023-10-06T05:25:24.268748+0000","last_clean":"2023-10-06T05:25:24.268748+0000","last_became_active":"2023-10-06T05:25:24.265983+0000","last_became_peered":"2023-10-06T05:25:24.265983+0000","last_unstale":"2023-10-06T05:25:24.268748+0000","last_undegraded":"2023-10-06T05:25:24.268748+0000","last_fullsized":"2023-10-06T05:25:24.268748+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:01.290511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278954+0000","last_change":"2023-10-06T05:25:25.273367+0000","last_active":"2023-10-06T05:25:25.278954+0000","last_peered":"2023-10-06T05:25:25.278954+0000","last_clean":"2023-10-06T05:25:25.278954+0000","last_became_active":"2023-10-06T05:25:25.273080+0000","last_became_peered":"2023-10-06T05:25:25.273080+0000","last_unstale":"2023-10-06T05:25:25.278954+0000","last_undegraded":"2023-10-06T05:25:25.278954+0000","last_fullsized":"2023-10-06T05:25:25.278954+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:51.992960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.273029+0000","last_change":"2023-10-06T05:25:24.270592+0000","last_active":"2023-10-06T05:25:24.273029+0000","last_peered":"2023-10-06T05:25:24.273029+0000","last_clean":"2023-10-06T05:25:24.273029+0000","last_became_active":"2023-10-06T05:25:24.270317+0000","last_became_peered":"2023-10-06T05:25:24.270317+0000","last_unstale":"2023-10-06T05:25:24.273029+0000","last_undegraded":"2023-10-06T05:25:24.273029+0000","last_fullsized":"2023-10-06T05:25:24.273029+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:27.406131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.280148+0000","last_change":"2023-10-06T05:25:25.274535+0000","last_active":"2023-10-06T05:25:25.280148+0000","last_peered":"2023-10-06T05:25:25.280148+0000","last_clean":"2023-10-06T05:25:25.280148+0000","last_became_active":"2023-10-06T05:25:25.274282+0000","last_became_peered":"2023-10-06T05:25:25.274282+0000","last_unstale":"2023-10-06T05:25:25.280148+0000","last_undegraded":"2023-10-06T05:25:25.280148+0000","last_fullsized":"2023-10-06T05:25:25.280148+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:10.525443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"149'8","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.275762+0000","last_change":"2023-10-06T05:25:24.263949+0000","last_active":"2023-10-06T05:25:25.275762+0000","last_peered":"2023-10-06T05:25:25.275762+0000","last_clean":"2023-10-06T05:25:25.275762+0000","last_became_active":"2023-10-06T05:25:24.263667+0000","last_became_peered":"2023-10-06T05:25:24.263667+0000","last_unstale":"2023-10-06T05:25:25.275762+0000","last_undegraded":"2023-10-06T05:25:25.275762+0000","last_fullsized":"2023-10-06T05:25:25.275762+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:37.924827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.277071+0000","last_change":"2023-10-06T05:25:25.272698+0000","last_active":"2023-10-06T05:25:25.277071+0000","last_peered":"2023-10-06T05:25:25.277071+0000","last_clean":"2023-10-06T05:25:25.277071+0000","last_became_active":"2023-10-06T05:25:25.272392+0000","last_became_peered":"2023-10-06T05:25:25.272392+0000","last_unstale":"2023-10-06T05:25:25.277071+0000","last_undegraded":"2023-10-06T05:25:25.277071+0000","last_fullsized":"2023-10-06T05:25:25.277071+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:41.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"149'1","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.260997+0000","last_change":"2023-10-06T05:25:24.264898+0000","last_active":"2023-10-06T05:25:25.260997+0000","last_peered":"2023-10-06T05:25:25.260997+0000","last_clean":"2023-10-06T05:25:25.260997+0000","last_became_active":"2023-10-06T05:25:24.264626+0000","last_became_peered":"2023-10-06T05:25:24.264626+0000","last_unstale":"2023-10-06T05:25:25.260997+0000","last_undegraded":"2023-10-06T05:25:25.260997+0000","last_fullsized":"2023-10-06T05:25:25.260997+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:02.255681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.260838+0000","last_change":"2023-10-06T05:25:25.258865+0000","last_active":"2023-10-06T05:25:25.260838+0000","last_peered":"2023-10-06T05:25:25.260838+0000","last_clean":"2023-10-06T05:25:25.260838+0000","last_became_active":"2023-10-06T05:25:25.258562+0000","last_became_peered":"2023-10-06T05:25:25.258562+0000","last_unstale":"2023-10-06T05:25:25.260838+0000","last_undegraded":"2023-10-06T05:25:25.260838+0000","last_fullsized":"2023-10-06T05:25:25.260838+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:33.454838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"149'2","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279806+0000","last_change":"2023-10-06T05:25:24.269985+0000","last_active":"2023-10-06T05:25:25.279806+0000","last_peered":"2023-10-06T05:25:25.279806+0000","last_clean":"2023-10-06T05:25:25.279806+0000","last_became_active":"2023-10-06T05:25:24.269630+0000","last_became_peered":"2023-10-06T05:25:24.269630+0000","last_unstale":"2023-10-06T05:25:25.279806+0000","last_undegraded":"2023-10-06T05:25:25.279806+0000","last_fullsized":"2023-10-06T05:25:25.279806+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:47.109597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.267134+0000","last_change":"2023-10-06T05:25:25.267134+0000","last_active":"2023-10-06T05:25:25.267134+0000","last_peered":"2023-10-06T05:25:25.267134+0000","last_clean":"2023-10-06T05:25:25.267134+0000","last_became_active":"2023-10-06T05:25:25.266511+0000","last_became_peered":"2023-10-06T05:25:25.266511+0000","last_unstale":"2023-10-06T05:25:25.267134+0000","last_undegraded":"2023-10-06T05:25:25.267134+0000","last_fullsized":"2023-10-06T05:25:25.267134+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:14.198941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"149'1","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.261360+0000","last_change":"2023-10-06T05:25:24.265327+0000","last_active":"2023-10-06T05:25:25.261360+0000","last_peered":"2023-10-06T05:25:25.261360+0000","last_clean":"2023-10-06T05:25:25.261360+0000","last_became_active":"2023-10-06T05:25:24.265012+0000","last_became_peered":"2023-10-06T05:25:24.265012+0000","last_unstale":"2023-10-06T05:25:25.261360+0000","last_undegraded":"2023-10-06T05:25:25.261360+0000","last_fullsized":"2023-10-06T05:25:25.261360+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:32.329091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273088+0000","last_change":"2023-10-06T05:25:25.273088+0000","last_active":"2023-10-06T05:25:25.273088+0000","last_peered":"2023-10-06T05:25:25.273088+0000","last_clean":"2023-10-06T05:25:25.273088+0000","last_became_active":"2023-10-06T05:25:25.272791+0000","last_became_peered":"2023-10-06T05:25:25.272791+0000","last_unstale":"2023-10-06T05:25:25.273088+0000","last_undegraded":"2023-10-06T05:25:25.273088+0000","last_fullsized":"2023-10-06T05:25:25.273088+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:16.299063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.274205+0000","last_change":"2023-10-06T05:25:24.267882+0000","last_active":"2023-10-06T05:25:24.274205+0000","last_peered":"2023-10-06T05:25:24.274205+0000","last_clean":"2023-10-06T05:25:24.274205+0000","last_became_active":"2023-10-06T05:25:24.267591+0000","last_became_peered":"2023-10-06T05:25:24.267591+0000","last_unstale":"2023-10-06T05:25:24.274205+0000","last_undegraded":"2023-10-06T05:25:24.274205+0000","last_fullsized":"2023-10-06T05:25:24.274205+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.390286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273403+0000","last_change":"2023-10-06T05:25:25.273403+0000","last_active":"2023-10-06T05:25:25.273403+0000","last_peered":"2023-10-06T05:25:25.273403+0000","last_clean":"2023-10-06T05:25:25.273403+0000","last_became_active":"2023-10-06T05:25:25.273143+0000","last_became_peered":"2023-10-06T05:25:25.273143+0000","last_unstale":"2023-10-06T05:25:25.273403+0000","last_undegraded":"2023-10-06T05:25:25.273403+0000","last_fullsized":"2023-10-06T05:25:25.273403+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:04.717165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"149'2","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279979+0000","last_change":"2023-10-06T05:25:24.264237+0000","last_active":"2023-10-06T05:25:25.279979+0000","last_peered":"2023-10-06T05:25:25.279979+0000","last_clean":"2023-10-06T05:25:25.279979+0000","last_became_active":"2023-10-06T05:25:24.263952+0000","last_became_peered":"2023-10-06T05:25:24.263952+0000","last_unstale":"2023-10-06T05:25:25.279979+0000","last_undegraded":"2023-10-06T05:25:25.279979+0000","last_fullsized":"2023-10-06T05:25:25.279979+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:23.669063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279631+0000","last_change":"2023-10-06T05:25:25.279631+0000","last_active":"2023-10-06T05:25:25.279631+0000","last_peered":"2023-10-06T05:25:25.279631+0000","last_clean":"2023-10-06T05:25:25.279631+0000","last_became_active":"2023-10-06T05:25:25.279214+0000","last_became_peered":"2023-10-06T05:25:25.279214+0000","last_unstale":"2023-10-06T05:25:25.279631+0000","last_undegraded":"2023-10-06T05:25:25.279631+0000","last_fullsized":"2023-10-06T05:25:25.279631+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:24.568428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.272967+0000","last_change":"2023-10-06T05:25:24.269898+0000","last_active":"2023-10-06T05:25:24.272967+0000","last_peered":"2023-10-06T05:25:24.272967+0000","last_clean":"2023-10-06T05:25:24.272967+0000","last_became_active":"2023-10-06T05:25:24.269579+0000","last_became_peered":"2023-10-06T05:25:24.269579+0000","last_unstale":"2023-10-06T05:25:24.272967+0000","last_undegraded":"2023-10-06T05:25:24.272967+0000","last_fullsized":"2023-10-06T05:25:24.272967+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:26.412224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276557+0000","last_change":"2023-10-06T05:25:25.276557+0000","last_active":"2023-10-06T05:25:25.276557+0000","last_peered":"2023-10-06T05:25:25.276557+0000","last_clean":"2023-10-06T05:25:25.276557+0000","last_became_active":"2023-10-06T05:25:25.276291+0000","last_became_peered":"2023-10-06T05:25:25.276291+0000","last_unstale":"2023-10-06T05:25:25.276557+0000","last_undegraded":"2023-10-06T05:25:25.276557+0000","last_fullsized":"2023-10-06T05:25:25.276557+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:52.140788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.247116+0000","last_change":"2023-10-06T05:25:24.254817+0000","last_active":"2023-10-06T05:25:25.247116+0000","last_peered":"2023-10-06T05:25:25.247116+0000","last_clean":"2023-10-06T05:25:25.247116+0000","last_became_active":"2023-10-06T05:25:24.254555+0000","last_became_peered":"2023-10-06T05:25:24.254555+0000","last_unstale":"2023-10-06T05:25:25.247116+0000","last_undegraded":"2023-10-06T05:25:25.247116+0000","last_fullsized":"2023-10-06T05:25:25.247116+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:16.533031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.264321+0000","last_change":"2023-10-06T05:25:25.260487+0000","last_active":"2023-10-06T05:25:25.264321+0000","last_peered":"2023-10-06T05:25:25.264321+0000","last_clean":"2023-10-06T05:25:25.264321+0000","last_became_active":"2023-10-06T05:25:25.260211+0000","last_became_peered":"2023-10-06T05:25:25.260211+0000","last_unstale":"2023-10-06T05:25:25.264321+0000","last_undegraded":"2023-10-06T05:25:25.264321+0000","last_fullsized":"2023-10-06T05:25:25.264321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:09.814818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.271595+0000","last_change":"2023-10-06T05:25:24.267602+0000","last_active":"2023-10-06T05:25:24.271595+0000","last_peered":"2023-10-06T05:25:24.271595+0000","last_clean":"2023-10-06T05:25:24.271595+0000","last_became_active":"2023-10-06T05:25:24.267296+0000","last_became_peered":"2023-10-06T05:25:24.267296+0000","last_unstale":"2023-10-06T05:25:24.271595+0000","last_undegraded":"2023-10-06T05:25:24.271595+0000","last_fullsized":"2023-10-06T05:25:24.271595+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:05.615368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.275003+0000","last_change":"2023-10-06T05:25:25.265537+0000","last_active":"2023-10-06T05:25:25.275003+0000","last_peered":"2023-10-06T05:25:25.275003+0000","last_clean":"2023-10-06T05:25:25.275003+0000","last_became_active":"2023-10-06T05:25:25.265297+0000","last_became_peered":"2023-10-06T05:25:25.265297+0000","last_unstale":"2023-10-06T05:25:25.275003+0000","last_undegraded":"2023-10-06T05:25:25.275003+0000","last_fullsized":"2023-10-06T05:25:25.275003+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:57.038031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"149'2","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278874+0000","last_change":"2023-10-06T05:25:24.268505+0000","last_active":"2023-10-06T05:25:25.278874+0000","last_peered":"2023-10-06T05:25:25.278874+0000","last_clean":"2023-10-06T05:25:25.278874+0000","last_became_active":"2023-10-06T05:25:24.268207+0000","last_became_peered":"2023-10-06T05:25:24.268207+0000","last_unstale":"2023-10-06T05:25:25.278874+0000","last_undegraded":"2023-10-06T05:25:25.278874+0000","last_fullsized":"2023-10-06T05:25:25.278874+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:03.459880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.270619+0000","last_change":"2023-10-06T05:25:25.270619+0000","last_active":"2023-10-06T05:25:25.270619+0000","last_peered":"2023-10-06T05:25:25.270619+0000","last_clean":"2023-10-06T05:25:25.270619+0000","last_became_active":"2023-10-06T05:25:25.270310+0000","last_became_peered":"2023-10-06T05:25:25.270310+0000","last_unstale":"2023-10-06T05:25:25.270619+0000","last_undegraded":"2023-10-06T05:25:25.270619+0000","last_fullsized":"2023-10-06T05:25:25.270619+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:03.051773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.246389+0000","last_change":"2023-10-06T05:25:24.253669+0000","last_active":"2023-10-06T05:25:25.246389+0000","last_peered":"2023-10-06T05:25:25.246389+0000","last_clean":"2023-10-06T05:25:25.246389+0000","last_became_active":"2023-10-06T05:25:24.253397+0000","last_became_peered":"2023-10-06T05:25:24.253397+0000","last_unstale":"2023-10-06T05:25:25.246389+0000","last_undegraded":"2023-10-06T05:25:25.246389+0000","last_fullsized":"2023-10-06T05:25:25.246389+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:25.778323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.269995+0000","last_change":"2023-10-06T05:25:25.269995+0000","last_active":"2023-10-06T05:25:25.269995+0000","last_peered":"2023-10-06T05:25:25.269995+0000","last_clean":"2023-10-06T05:25:25.269995+0000","last_became_active":"2023-10-06T05:25:25.269740+0000","last_became_peered":"2023-10-06T05:25:25.269740+0000","last_unstale":"2023-10-06T05:25:25.269995+0000","last_undegraded":"2023-10-06T05:25:25.269995+0000","last_fullsized":"2023-10-06T05:25:25.269995+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:52.826187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.253506+0000","last_change":"2023-10-06T05:25:24.267381+0000","last_active":"2023-10-06T05:25:25.253506+0000","last_peered":"2023-10-06T05:25:25.253506+0000","last_clean":"2023-10-06T05:25:25.253506+0000","last_became_active":"2023-10-06T05:25:24.267117+0000","last_became_peered":"2023-10-06T05:25:24.267117+0000","last_unstale":"2023-10-06T05:25:25.253506+0000","last_undegraded":"2023-10-06T05:25:25.253506+0000","last_fullsized":"2023-10-06T05:25:25.253506+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:20.991647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276999+0000","last_change":"2023-10-06T05:25:25.270469+0000","last_active":"2023-10-06T05:25:25.276999+0000","last_peered":"2023-10-06T05:25:25.276999+0000","last_clean":"2023-10-06T05:25:25.276999+0000","last_became_active":"2023-10-06T05:25:25.269903+0000","last_became_peered":"2023-10-06T05:25:25.269903+0000","last_unstale":"2023-10-06T05:25:25.276999+0000","last_undegraded":"2023-10-06T05:25:25.276999+0000","last_fullsized":"2023-10-06T05:25:25.276999+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:34.623755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.253798+0000","last_change":"2023-10-06T05:25:24.263152+0000","last_active":"2023-10-06T05:25:25.253798+0000","last_peered":"2023-10-06T05:25:25.253798+0000","last_clean":"2023-10-06T05:25:25.253798+0000","last_became_active":"2023-10-06T05:25:24.262892+0000","last_became_peered":"2023-10-06T05:25:24.262892+0000","last_unstale":"2023-10-06T05:25:25.253798+0000","last_undegraded":"2023-10-06T05:25:25.253798+0000","last_fullsized":"2023-10-06T05:25:25.253798+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:51.131757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.274699+0000","last_change":"2023-10-06T05:25:25.274699+0000","last_active":"2023-10-06T05:25:25.274699+0000","last_peered":"2023-10-06T05:25:25.274699+0000","last_clean":"2023-10-06T05:25:25.274699+0000","last_became_active":"2023-10-06T05:25:25.274375+0000","last_became_peered":"2023-10-06T05:25:25.274375+0000","last_unstale":"2023-10-06T05:25:25.274699+0000","last_undegraded":"2023-10-06T05:25:25.274699+0000","last_fullsized":"2023-10-06T05:25:25.274699+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:34.623755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.272252+0000","last_change":"2023-10-06T05:25:24.267786+0000","last_active":"2023-10-06T05:25:24.272252+0000","last_peered":"2023-10-06T05:25:24.272252+0000","last_clean":"2023-10-06T05:25:24.272252+0000","last_became_active":"2023-10-06T05:25:24.267442+0000","last_became_peered":"2023-10-06T05:25:24.267442+0000","last_unstale":"2023-10-06T05:25:24.272252+0000","last_undegraded":"2023-10-06T05:25:24.272252+0000","last_fullsized":"2023-10-06T05:25:24.272252+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:44.627400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276289+0000","last_change":"2023-10-06T05:25:25.271266+0000","last_active":"2023-10-06T05:25:25.276289+0000","last_peered":"2023-10-06T05:25:25.276289+0000","last_clean":"2023-10-06T05:25:25.276289+0000","last_became_active":"2023-10-06T05:25:25.268309+0000","last_became_peered":"2023-10-06T05:25:25.268309+0000","last_unstale":"2023-10-06T05:25:25.276289+0000","last_undegraded":"2023-10-06T05:25:25.276289+0000","last_fullsized":"2023-10-06T05:25:25.276289+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:28.940256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.271850+0000","last_change":"2023-10-06T05:25:24.265924+0000","last_active":"2023-10-06T05:25:24.271850+0000","last_peered":"2023-10-06T05:25:24.271850+0000","last_clean":"2023-10-06T05:25:24.271850+0000","last_became_active":"2023-10-06T05:25:24.265553+0000","last_became_peered":"2023-10-06T05:25:24.265553+0000","last_unstale":"2023-10-06T05:25:24.271850+0000","last_undegraded":"2023-10-06T05:25:24.271850+0000","last_fullsized":"2023-10-06T05:25:24.271850+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:23.238726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273028+0000","last_change":"2023-10-06T05:25:25.273028+0000","last_active":"2023-10-06T05:25:25.273028+0000","last_peered":"2023-10-06T05:25:25.273028+0000","last_clean":"2023-10-06T05:25:25.273028+0000","last_became_active":"2023-10-06T05:25:25.272695+0000","last_became_peered":"2023-10-06T05:25:25.272695+0000","last_unstale":"2023-10-06T05:25:25.273028+0000","last_undegraded":"2023-10-06T05:25:25.273028+0000","last_fullsized":"2023-10-06T05:25:25.273028+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:22.247139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.247557+0000","last_change":"2023-10-06T05:25:24.262759+0000","last_active":"2023-10-06T05:25:25.247557+0000","last_peered":"2023-10-06T05:25:25.247557+0000","last_clean":"2023-10-06T05:25:25.247557+0000","last_became_active":"2023-10-06T05:25:24.262434+0000","last_became_peered":"2023-10-06T05:25:24.262434+0000","last_unstale":"2023-10-06T05:25:25.247557+0000","last_undegraded":"2023-10-06T05:25:25.247557+0000","last_fullsized":"2023-10-06T05:25:25.247557+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:36.786537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.272667+0000","last_change":"2023-10-06T05:25:25.272667+0000","last_active":"2023-10-06T05:25:25.272667+0000","last_peered":"2023-10-06T05:25:25.272667+0000","last_clean":"2023-10-06T05:25:25.272667+0000","last_became_active":"2023-10-06T05:25:25.272351+0000","last_became_peered":"2023-10-06T05:25:25.272351+0000","last_unstale":"2023-10-06T05:25:25.272667+0000","last_undegraded":"2023-10-06T05:25:25.272667+0000","last_fullsized":"2023-10-06T05:25:25.272667+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:48.118628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.269328+0000","last_change":"2023-10-06T05:25:24.265828+0000","last_active":"2023-10-06T05:25:24.269328+0000","last_peered":"2023-10-06T05:25:24.269328+0000","last_clean":"2023-10-06T05:25:24.269328+0000","last_became_active":"2023-10-06T05:25:24.265526+0000","last_became_peered":"2023-10-06T05:25:24.265526+0000","last_unstale":"2023-10-06T05:25:24.269328+0000","last_undegraded":"2023-10-06T05:25:24.269328+0000","last_fullsized":"2023-10-06T05:25:24.269328+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:24.096593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273894+0000","last_change":"2023-10-06T05:25:25.273894+0000","last_active":"2023-10-06T05:25:25.273894+0000","last_peered":"2023-10-06T05:25:25.273894+0000","last_clean":"2023-10-06T05:25:25.273894+0000","last_became_active":"2023-10-06T05:25:25.273622+0000","last_became_peered":"2023-10-06T05:25:25.273622+0000","last_unstale":"2023-10-06T05:25:25.273894+0000","last_undegraded":"2023-10-06T05:25:25.273894+0000","last_fullsized":"2023-10-06T05:25:25.273894+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:08.677801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"149'4","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.281212+0000","last_change":"2023-10-06T05:25:24.268047+0000","last_active":"2023-10-06T05:25:25.281212+0000","last_peered":"2023-10-06T05:25:25.281212+0000","last_clean":"2023-10-06T05:25:25.281212+0000","last_became_active":"2023-10-06T05:25:24.267767+0000","last_became_peered":"2023-10-06T05:25:24.267767+0000","last_unstale":"2023-10-06T05:25:25.281212+0000","last_undegraded":"2023-10-06T05:25:25.281212+0000","last_fullsized":"2023-10-06T05:25:25.281212+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.293280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.256100+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:25:24.256100+0000","last_peered":"2023-10-06T05:25:24.256100+0000","last_clean":"2023-10-06T05:25:24.256100+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:25:24.256100+0000","last_undegraded":"2023-10-06T05:25:24.256100+0000","last_fullsized":"2023-10-06T05:25:24.256100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.266799+0000","last_change":"2023-10-06T05:25:25.263932+0000","last_active":"2023-10-06T05:25:25.266799+0000","last_peered":"2023-10-06T05:25:25.266799+0000","last_clean":"2023-10-06T05:25:25.266799+0000","last_became_active":"2023-10-06T05:25:25.263652+0000","last_became_peered":"2023-10-06T05:25:25.263652+0000","last_unstale":"2023-10-06T05:25:25.266799+0000","last_undegraded":"2023-10-06T05:25:25.266799+0000","last_fullsized":"2023-10-06T05:25:25.266799+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:03.211478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"149'1","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276175+0000","last_change":"2023-10-06T05:25:24.267912+0000","last_active":"2023-10-06T05:25:25.276175+0000","last_peered":"2023-10-06T05:25:25.276175+0000","last_clean":"2023-10-06T05:25:25.276175+0000","last_became_active":"2023-10-06T05:25:24.267555+0000","last_became_peered":"2023-10-06T05:25:24.267555+0000","last_unstale":"2023-10-06T05:25:25.276175+0000","last_undegraded":"2023-10-06T05:25:25.276175+0000","last_fullsized":"2023-10-06T05:25:25.276175+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:23.559397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.254899+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:25:24.254899+0000","last_peered":"2023-10-06T05:25:24.254899+0000","last_clean":"2023-10-06T05:25:24.254899+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:25:24.254899+0000","last_undegraded":"2023-10-06T05:25:24.254899+0000","last_fullsized":"2023-10-06T05:25:24.254899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.280761+0000","last_change":"2023-10-06T05:25:25.280761+0000","last_active":"2023-10-06T05:25:25.280761+0000","last_peered":"2023-10-06T05:25:25.280761+0000","last_clean":"2023-10-06T05:25:25.280761+0000","last_became_active":"2023-10-06T05:25:25.280478+0000","last_became_peered":"2023-10-06T05:25:25.280478+0000","last_unstale":"2023-10-06T05:25:25.280761+0000","last_undegraded":"2023-10-06T05:25:25.280761+0000","last_fullsized":"2023-10-06T05:25:25.280761+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:14.198941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.250303+0000","last_change":"2023-10-06T05:25:24.263518+0000","last_active":"2023-10-06T05:25:25.250303+0000","last_peered":"2023-10-06T05:25:25.250303+0000","last_clean":"2023-10-06T05:25:25.250303+0000","last_became_active":"2023-10-06T05:25:24.263218+0000","last_became_peered":"2023-10-06T05:25:24.263218+0000","last_unstale":"2023-10-06T05:25:25.250303+0000","last_undegraded":"2023-10-06T05:25:25.250303+0000","last_fullsized":"2023-10-06T05:25:25.250303+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:54.016171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.254803+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:25:24.254803+0000","last_peered":"2023-10-06T05:25:24.254803+0000","last_clean":"2023-10-06T05:25:24.254803+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:25:24.254803+0000","last_undegraded":"2023-10-06T05:25:24.254803+0000","last_fullsized":"2023-10-06T05:25:24.254803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.271103+0000","last_change":"2023-10-06T05:25:25.271103+0000","last_active":"2023-10-06T05:25:25.271103+0000","last_peered":"2023-10-06T05:25:25.271103+0000","last_clean":"2023-10-06T05:25:25.271103+0000","last_became_active":"2023-10-06T05:25:25.270803+0000","last_became_peered":"2023-10-06T05:25:25.270803+0000","last_unstale":"2023-10-06T05:25:25.271103+0000","last_undegraded":"2023-10-06T05:25:25.271103+0000","last_fullsized":"2023-10-06T05:25:25.271103+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:34.101563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.272411+0000","last_change":"2023-10-06T05:25:24.265008+0000","last_active":"2023-10-06T05:25:24.272411+0000","last_peered":"2023-10-06T05:25:24.272411+0000","last_clean":"2023-10-06T05:25:24.272411+0000","last_became_active":"2023-10-06T05:25:24.264440+0000","last_became_peered":"2023-10-06T05:25:24.264440+0000","last_unstale":"2023-10-06T05:25:24.272411+0000","last_undegraded":"2023-10-06T05:25:24.272411+0000","last_fullsized":"2023-10-06T05:25:24.272411+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:10.490754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.252680+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:25:24.252680+0000","last_peered":"2023-10-06T05:25:24.252680+0000","last_clean":"2023-10-06T05:25:24.252680+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:25:24.252680+0000","last_undegraded":"2023-10-06T05:25:24.252680+0000","last_fullsized":"2023-10-06T05:25:24.252680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.263379+0000","last_change":"2023-10-06T05:25:25.263379+0000","last_active":"2023-10-06T05:25:25.263379+0000","last_peered":"2023-10-06T05:25:25.263379+0000","last_clean":"2023-10-06T05:25:25.263379+0000","last_became_active":"2023-10-06T05:25:25.262958+0000","last_became_peered":"2023-10-06T05:25:25.262958+0000","last_unstale":"2023-10-06T05:25:25.263379+0000","last_undegraded":"2023-10-06T05:25:25.263379+0000","last_fullsized":"2023-10-06T05:25:25.263379+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:21.399027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"149'1","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.262816+0000","last_change":"2023-10-06T05:25:24.266518+0000","last_active":"2023-10-06T05:25:25.262816+0000","last_peered":"2023-10-06T05:25:25.262816+0000","last_clean":"2023-10-06T05:25:25.262816+0000","last_became_active":"2023-10-06T05:25:24.266198+0000","last_became_peered":"2023-10-06T05:25:24.266198+0000","last_unstale":"2023-10-06T05:25:25.262816+0000","last_undegraded":"2023-10-06T05:25:25.262816+0000","last_fullsized":"2023-10-06T05:25:25.262816+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:49.522768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":113,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.251377+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:25:24.251377+0000","last_peered":"2023-10-06T05:25:24.251377+0000","last_clean":"2023-10-06T05:25:24.251377+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:25:24.251377+0000","last_undegraded":"2023-10-06T05:25:24.251377+0000","last_fullsized":"2023-10-06T05:25:24.251377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276459+0000","last_change":"2023-10-06T05:25:25.267298+0000","last_active":"2023-10-06T05:25:25.276459+0000","last_peered":"2023-10-06T05:25:25.276459+0000","last_clean":"2023-10-06T05:25:25.276459+0000","last_became_active":"2023-10-06T05:25:25.266888+0000","last_became_peered":"2023-10-06T05:25:25.266888+0000","last_unstale":"2023-10-06T05:25:25.276459+0000","last_undegraded":"2023-10-06T05:25:25.276459+0000","last_fullsized":"2023-10-06T05:25:25.276459+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:48.283153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"149'1","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.274716+0000","last_change":"2023-10-06T05:25:24.267289+0000","last_active":"2023-10-06T05:25:25.274716+0000","last_peered":"2023-10-06T05:25:25.274716+0000","last_clean":"2023-10-06T05:25:25.274716+0000","last_became_active":"2023-10-06T05:25:24.266998+0000","last_became_peered":"2023-10-06T05:25:24.266998+0000","last_unstale":"2023-10-06T05:25:25.274716+0000","last_undegraded":"2023-10-06T05:25:25.274716+0000","last_fullsized":"2023-10-06T05:25:25.274716+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:59.810722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":302,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.252596+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:25:24.252596+0000","last_peered":"2023-10-06T05:25:24.252596+0000","last_clean":"2023-10-06T05:25:24.252596+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:25:24.252596+0000","last_undegraded":"2023-10-06T05:25:24.252596+0000","last_fullsized":"2023-10-06T05:25:24.252596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":127,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.242827+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:25:24.242827+0000","last_peered":"2023-10-06T05:25:24.242827+0000","last_clean":"2023-10-06T05:25:24.242827+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:25:24.242827+0000","last_undegraded":"2023-10-06T05:25:24.242827+0000","last_fullsized":"2023-10-06T05:25:24.242827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.277127+0000","last_change":"2023-10-06T05:25:25.277127+0000","last_active":"2023-10-06T05:25:25.277127+0000","last_peered":"2023-10-06T05:25:25.277127+0000","last_clean":"2023-10-06T05:25:25.277127+0000","last_became_active":"2023-10-06T05:25:25.276802+0000","last_became_peered":"2023-10-06T05:25:25.276802+0000","last_unstale":"2023-10-06T05:25:25.277127+0000","last_undegraded":"2023-10-06T05:25:25.277127+0000","last_fullsized":"2023-10-06T05:25:25.277127+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:41.120528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"149'2","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279875+0000","last_change":"2023-10-06T05:25:24.257400+0000","last_active":"2023-10-06T05:25:25.279875+0000","last_peered":"2023-10-06T05:25:25.279875+0000","last_clean":"2023-10-06T05:25:25.279875+0000","last_became_active":"2023-10-06T05:25:24.257112+0000","last_became_peered":"2023-10-06T05:25:24.257112+0000","last_unstale":"2023-10-06T05:25:25.279875+0000","last_undegraded":"2023-10-06T05:25:25.279875+0000","last_fullsized":"2023-10-06T05:25:25.279875+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:04.407229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":143,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.253889+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:25:25.253889+0000","last_peered":"2023-10-06T05:25:25.253889+0000","last_clean":"2023-10-06T05:25:25.253889+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:25:25.253889+0000","last_undegraded":"2023-10-06T05:25:25.253889+0000","last_fullsized":"2023-10-06T05:25:25.253889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279012+0000","last_change":"2023-10-06T05:25:25.279012+0000","last_active":"2023-10-06T05:25:25.279012+0000","last_peered":"2023-10-06T05:25:25.279012+0000","last_clean":"2023-10-06T05:25:25.279012+0000","last_became_active":"2023-10-06T05:25:25.278683+0000","last_became_peered":"2023-10-06T05:25:25.278683+0000","last_unstale":"2023-10-06T05:25:25.279012+0000","last_undegraded":"2023-10-06T05:25:25.279012+0000","last_fullsized":"2023-10-06T05:25:25.279012+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:04.717165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"149'2","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276250+0000","last_change":"2023-10-06T05:25:24.269896+0000","last_active":"2023-10-06T05:25:25.276250+0000","last_peered":"2023-10-06T05:25:25.276250+0000","last_clean":"2023-10-06T05:25:25.276250+0000","last_became_active":"2023-10-06T05:25:24.269631+0000","last_became_peered":"2023-10-06T05:25:24.269631+0000","last_unstale":"2023-10-06T05:25:25.276250+0000","last_undegraded":"2023-10-06T05:25:25.276250+0000","last_fullsized":"2023-10-06T05:25:25.276250+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:20.810496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":131,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.251241+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:25:25.251241+0000","last_peered":"2023-10-06T05:25:25.251241+0000","last_clean":"2023-10-06T05:25:25.251241+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:25:25.251241+0000","last_undegraded":"2023-10-06T05:25:25.251241+0000","last_fullsized":"2023-10-06T05:25:25.251241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303800,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068040,"statfs":{"total":96636764160,"available":96325672960,"internally_reserved":0,"allocated":2473984,"data_stored":1465246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.50800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71199999999999997,"15min":0.72099999999999997},"min":{"1min":0.51800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83899999999999997,"5min":1.081,"15min":1.081},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70499999999999996,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.48099999999999998,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91700000000000004,"5min":0.93600000000000005,"15min":1.018},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.73699999999999999},"min":{"1min":0.51200000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90000000000000002,"5min":0.97799999999999998,"15min":1.056},"last":0.70599999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71099999999999997,"15min":0.73999999999999999},"min":{"1min":0.53400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94199999999999995,"5min":1.149,"15min":1.149},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.72699999999999998,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65700000000000003,"15min":0.69999999999999996},"min":{"1min":0.37,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86499999999999999,"5min":0.96199999999999997,"15min":1.0249999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.76600000000000001},"min":{"1min":0.46899999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97499999999999998,"5min":1.27,"15min":1.27},"last":0.41999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69699999999999995,"15min":0.72899999999999998},"min":{"1min":0.502,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91300000000000003,"5min":0.97899999999999998,"15min":0.999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75,"15min":0.80900000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90600000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.64000000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79500000000000004,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96499999999999997,"5min":1.0449999999999999,"15min":1.1040000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.52200000000000002,"5min":0.46200000000000002,"15min":0.441},"max":{"1min":1.0089999999999999,"5min":1.248,"15min":1.248},"last":0.51700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303224,"kb_used_data":1840,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068616,"statfs":{"total":96636764160,"available":96326262784,"internally_reserved":0,"allocated":1884160,"data_stored":874948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63900000000000001,"15min":0.67000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.89800000000000002,"15min":1.8500000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66500000000000004,"15min":0.69699999999999995},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0369999999999999,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69099999999999995,"15min":0.70699999999999996},"min":{"1min":0.50800000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93000000000000005,"5min":1.177,"15min":1.9279999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95799999999999996,"5min":1.073,"15min":1.4239999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.65200000000000002},"min":{"1min":0.47299999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.877,"5min":1.093,"15min":1.369},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71299999999999997,"15min":0.751},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.038,"5min":1.085,"15min":1.891},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71599999999999997,"15min":0.74099999999999999},"min":{"1min":0.49299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.347},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.71399999999999997},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.98699999999999999,"15min":1},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70999999999999996,"15min":0.75900000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.9670000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77800000000000002,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0209999999999999,"5min":1.278,"15min":1.4510000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.50700000000000001,"5min":0.41999999999999998,"15min":0.41499999999999998},"max":{"1min":0.82899999999999996,"5min":1.0680000000000001,"15min":1.415},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.753,"15min":0.80400000000000005},"min":{"1min":0.52300000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1020000000000001,"5min":1.2350000000000001,"15min":2},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77200000000000002,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.016,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76400000000000001,"15min":0.77500000000000002},"min":{"1min":0.54900000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95799999999999996,"5min":1.133,"15min":1.46},"last":0.82099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303800,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068040,"statfs":{"total":96636764160,"available":96325672960,"internally_reserved":0,"allocated":2473984,"data_stored":1465246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.44,"15min":0.436},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.621,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.48599999999999999},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.504,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.434}]},{"osd":2,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.54100000000000004,"5min":0.77000000000000002,"15min":0.83799999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.59999999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.44900000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.58499999999999996,"5min":0.80500000000000005,"15min":0.92400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51000000000000001,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.57299999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.495}]},{"osd":4,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.52400000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0680000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79200000000000004,"15min":0.79300000000000004},"min":{"1min":0.63600000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0189999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74099999999999999,"15min":0.74399999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.81000000000000005},"min":{"1min":0.61599999999999999,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":1.002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73399999999999999,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93799999999999994,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1140000000000001,"5min":1.387,"15min":1.387},"last":0.86799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.50600000000000001},"max":{"1min":1.044,"5min":1.244,"15min":1.244},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81399999999999995,"15min":0.82199999999999995},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303236,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068604,"statfs":{"total":96636764160,"available":96326250496,"internally_reserved":0,"allocated":1896448,"data_stored":875578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41499999999999998,"15min":0.42799999999999999},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.51300000000000001,"5min":0.64500000000000002,"15min":0.84499999999999997},"last":0.436},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.44500000000000001,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":1.042},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.48799999999999999},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.624,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.63,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.48199999999999998,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.67500000000000004,"5min":0.80100000000000005,"15min":0.90500000000000003},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.745},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.80000000000000004,"5min":1.1519999999999999,"15min":1.242},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.76700000000000002},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97299999999999998,"5min":0.97899999999999998,"15min":1.198},"last":0.78900000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73399999999999999,"15min":0.76100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.90900000000000003,"5min":1.1659999999999999,"15min":1.26},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73099999999999998,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84099999999999997,"5min":1.1299999999999999,"15min":1.163},"last":0.874}]},{"osd":6,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73499999999999999,"15min":0.78600000000000003},"min":{"1min":0.55300000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92500000000000004,"5min":1.105,"15min":1.323},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76300000000000001,"15min":0.78600000000000003},"min":{"1min":0.54600000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95099999999999996,"5min":1.123,"15min":1.3100000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77400000000000002,"15min":0.82199999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.52400000000000002},"max":{"1min":0.84999999999999998,"5min":1.2010000000000001,"15min":1.264},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.81100000000000005},"min":{"1min":0.58199999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":0.77300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303224,"kb_used_data":1840,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068616,"statfs":{"total":96636764160,"available":96326262784,"internally_reserved":0,"allocated":1884160,"data_stored":875432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.77800000000000002},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.46000000000000002,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.55400000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45100000000000001,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.57399999999999995,"5min":0.71699999999999997,"15min":0.88700000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.31900000000000001},"max":{"1min":0.57799999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.34100000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.627,"5min":0.80600000000000005,"15min":0.93600000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.38500000000000001,"5min":0.372,"15min":0.36199999999999999},"max":{"1min":0.57299999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.30499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.997,"5min":1.022,"15min":1.1299999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.77800000000000002},"min":{"1min":0.53300000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.018,"5min":1.085,"15min":1.1699999999999999},"last":0.875}]},{"osd":5,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.52400000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91600000000000004,"5min":1.042,"15min":1.099},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73699999999999999,"15min":0.78900000000000003},"min":{"1min":0.55600000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.163},"last":0.92500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.81799999999999995},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98599999999999999,"5min":1.056,"15min":1.21},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.095}]},{"osd":7,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.79900000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1200000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.77100000000000002,"15min":0.83899999999999997},"min":{"1min":0.58999999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97099999999999997,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":1.0309999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303228,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068612,"statfs":{"total":96636764160,"available":96326258688,"internally_reserved":0,"allocated":1888256,"data_stored":875038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42999999999999999,"15min":0.45300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65500000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.58299999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.45100000000000001,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54000000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.497,"15min":0.52300000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72199999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.504,"15min":0.53500000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.57699999999999996,"5min":0.999,"15min":0.999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.53400000000000003,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.58599999999999997,"5min":0.98999999999999999,"15min":1.014},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":0.80100000000000005},"min":{"1min":0.53300000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.0800000000000001,"5min":1.145,"15min":1.2669999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.82299999999999995},"min":{"1min":0.64000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.016,"5min":1.1830000000000001,"15min":1.3540000000000001},"last":0.628}]},{"osd":5,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72899999999999998,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93100000000000005,"5min":1.093,"15min":1.204},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77100000000000002,"15min":0.81999999999999995},"min":{"1min":0.52400000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":1.208,"15min":1.329},"last":0.57599999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75600000000000001,"15min":0.80000000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.96899999999999997,"5min":1.0880000000000001,"15min":1.484},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77400000000000002,"15min":0.82899999999999996},"min":{"1min":0.55900000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98299999999999998,"5min":1.2809999999999999,"15min":1.629},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78300000000000003,"15min":0.83599999999999997},"min":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0469999999999999,"5min":1.1950000000000001,"15min":1.5620000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.79100000000000004,"15min":0.82199999999999995},"min":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":1.0309999999999999,"5min":1.194,"15min":1.325},"last":0.59299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303248,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068592,"statfs":{"total":96636764160,"available":96326238208,"internally_reserved":0,"allocated":1908736,"data_stored":883117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.75600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.95899999999999996,"5min":1.02,"15min":2.77},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.46899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.6419999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67900000000000005,"15min":0.71099999999999997},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.99199999999999999,"5min":1.1100000000000001,"15min":1.1899999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.39800000000000002},"max":{"1min":0.93799999999999994,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71799999999999997,"15min":0.79300000000000004},"min":{"1min":0.52100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.52900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89000000000000001,"5min":0.95899999999999996,"15min":1.0629999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70799999999999996,"15min":0.96299999999999997},"min":{"1min":0.52700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97899999999999998,"5min":0.99299999999999999,"15min":7.742},"last":0.745}]},{"osd":5,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.51000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.079},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.72499999999999998,"15min":0.746},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.012,"5min":1.04,"15min":1.181},"last":0.627}]},{"osd":6,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92900000000000005,"5min":1.0029999999999999,"15min":1.2669999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77300000000000002,"15min":0.80800000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.436},"max":{"1min":0.95399999999999996,"5min":1.093,"15min":1.171},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78600000000000003,"15min":0.81999999999999995},"min":{"1min":0.69599999999999995,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0589999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78400000000000003,"15min":0.83999999999999997},"min":{"1min":0.59199999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.403,"5min":1.403,"15min":1.571},"last":0.69899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303236,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068604,"statfs":{"total":96636764160,"available":96326250496,"internally_reserved":0,"allocated":1896448,"data_stored":876436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63400000000000001,"15min":0.878},"min":{"1min":0.38400000000000001,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.95599999999999996,"5min":1.109,"15min":9.2780000000000005},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0469999999999999},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66800000000000004,"15min":0.90400000000000003},"min":{"1min":0.433,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.874,"5min":1.079,"15min":9.25},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67800000000000005,"15min":0.95399999999999996},"min":{"1min":0.48699999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.92500000000000004},"min":{"1min":0.51600000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84999999999999998,"5min":1.052,"15min":10.384},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71099999999999997,"15min":0.95399999999999996},"min":{"1min":0.48199999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98699999999999999,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.98799999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.495},"max":{"1min":0.88900000000000001,"5min":1.76,"15min":10.205},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70299999999999996,"15min":0.71999999999999997},"min":{"1min":0.51600000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.024,"5min":1.097,"15min":1.292},"last":0.749}]},{"osd":4,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.40600000000000003},"max":{"1min":0.95699999999999996,"5min":1.069,"15min":1.069},"last":0.496},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0469999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.73299999999999998},"min":{"1min":0.55300000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91300000000000003,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76400000000000001,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.298},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78000000000000003,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98699999999999999,"5min":1.556,"15min":1.556},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76800000000000002,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.121,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.64800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:25:29.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:25:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.886+0000 7f7a553ff700 1 Processor -- start 2023-10-06T05:25:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.887+0000 7f7a553ff700 1 -- start start 2023-10-06T05:25:29.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a553ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7a500ca770 0x7f7a500c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:29.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a553ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7a500cab40 0x7f7a501569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a553ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a5006ae00 0x7f7a5006b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a553ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a500c7a70 con 0x7f7a500ca770 2023-10-06T05:25:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a553ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a50156ef0 con 0x7f7a500cab40 2023-10-06T05:25:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a553ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5006b7c0 con 0x7f7a5006ae00 2023-10-06T05:25:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a4effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7a500ca770 0x7f7a500c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a4effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7a500ca770 0x7f7a500c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47144/0 (socket says 172.21.15.31:47144) 2023-10-06T05:25:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a4effd700 1 -- 172.21.15.31:0/2761255137 learned_addr learned my addr 172.21.15.31:0/2761255137 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a4f7fe700 1 --2- 172.21.15.31:0/2761255137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a5006ae00 0x7f7a5006b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.888+0000 7f7a4e7fc700 1 --2- 172.21.15.31:0/2761255137 >> v2:172.21.15.62:3300/0 conn(0x7f7a500cab40 0x7f7a501569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:29.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.889+0000 7f7a4effd700 1 -- 172.21.15.31:0/2761255137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a5006ae00 msgr2=0x7f7a5006b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:29.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.889+0000 7f7a4effd700 1 --2- 172.21.15.31:0/2761255137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a5006ae00 0x7f7a5006b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.889+0000 7f7a4effd700 1 -- 172.21.15.31:0/2761255137 >> v2:172.21.15.62:3300/0 conn(0x7f7a500cab40 msgr2=0x7f7a501569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:29.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.889+0000 7f7a4effd700 1 --2- 172.21.15.31:0/2761255137 >> v2:172.21.15.62:3300/0 conn(0x7f7a500cab40 0x7f7a501569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.889+0000 7f7a4effd700 1 -- 172.21.15.31:0/2761255137 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500c6df0 con 0x7f7a500ca770 2023-10-06T05:25:29.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.889+0000 7f7a4effd700 1 --2- 172.21.15.31:0/2761255137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500ca770 0x7f7a500c66f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f7a500ca030 tx=0x7f7a4000b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a293dc22a865fd6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:29.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.889+0000 7f7a4dffb700 1 -- 172.21.15.31:0/2761255137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40005c10 con 0x7f7a500ca770 2023-10-06T05:25:29.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.889+0000 7f7a4dffb700 1 -- 172.21.15.31:0/2761255137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a40005d70 con 0x7f7a500ca770 2023-10-06T05:25:29.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.889+0000 7f7a4dffb700 1 -- 172.21.15.31:0/2761255137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000db20 con 0x7f7a500ca770 2023-10-06T05:25:29.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.890+0000 7f7a553ff700 1 -- 172.21.15.31:0/2761255137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500ca770 msgr2=0x7f7a500c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:29.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.890+0000 7f7a553ff700 1 --2- 172.21.15.31:0/2761255137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500ca770 0x7f7a500c66f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f7a500ca030 tx=0x7f7a4000b4c0 comp rx=0 tx=0).stop 2023-10-06T05:25:29.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.890+0000 7f7a553ff700 1 -- 172.21.15.31:0/2761255137 shutdown_connections 2023-10-06T05:25:29.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.890+0000 7f7a553ff700 1 --2- 172.21.15.31:0/2761255137 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a5006ae00 0x7f7a5006b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.890+0000 7f7a553ff700 1 --2- 172.21.15.31:0/2761255137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500ca770 0x7f7a500c66f0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.890+0000 7f7a553ff700 1 --2- 172.21.15.31:0/2761255137 >> v2:172.21.15.62:3300/0 conn(0x7f7a500cab40 0x7f7a501569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.890+0000 7f7a553ff700 1 -- 172.21.15.31:0/2761255137 >> 172.21.15.31:0/2761255137 conn(0x7f7a500cc0b0 msgr2=0x7f7a500cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.890+0000 7f7a553ff700 1 -- 172.21.15.31:0/2761255137 shutdown_connections 2023-10-06T05:25:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.890+0000 7f7a553ff700 1 -- 172.21.15.31:0/2761255137 wait complete. 2023-10-06T05:25:30.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.892+0000 7f7a553ff700 1 Processor -- start 2023-10-06T05:25:30.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.892+0000 7f7a553ff700 1 -- start start 2023-10-06T05:25:30.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a553ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a5006ae00 0x7f7a50158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a553ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a500ca770 0x7f7a50158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a553ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500cab40 0x7f7a500ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a553ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500ad090 con 0x7f7a500cab40 2023-10-06T05:25:30.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a553ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500ad370 con 0x7f7a500ca770 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a553ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500ad650 con 0x7f7a5006ae00 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a4f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500cab40 0x7f7a500ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a4f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500cab40 0x7f7a500ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47154/0 (socket says 172.21.15.31:47154) 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a4f7fe700 1 -- 172.21.15.31:0/882042494 learned_addr learned my addr 172.21.15.31:0/882042494 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a4effd700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a5006ae00 0x7f7a50158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a4e7fc700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a500ca770 0x7f7a50158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a4f7fe700 1 -- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a5006ae00 msgr2=0x7f7a50158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a4f7fe700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a5006ae00 0x7f7a50158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a4f7fe700 1 -- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a500ca770 msgr2=0x7f7a50158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.893+0000 7f7a4f7fe700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a500ca770 0x7f7a50158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.894+0000 7f7a4f7fe700 1 -- 172.21.15.31:0/882042494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4000b720 con 0x7f7a500cab40 2023-10-06T05:25:30.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.894+0000 7f7a4f7fe700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500cab40 0x7f7a500ac2f0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009e90 tx=0x7f7a4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.894+0000 7f7a36ffd700 1 -- 172.21.15.31:0/882042494 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4800cb70 con 0x7f7a500cab40 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.894+0000 7f7a36ffd700 1 -- 172.21.15.31:0/882042494 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4800f040 con 0x7f7a500cab40 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.894+0000 7f7a553ff700 1 -- 172.21.15.31:0/882042494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a500ad930 con 0x7f7a500cab40 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.894+0000 7f7a36ffd700 1 -- 172.21.15.31:0/882042494 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48014480 con 0x7f7a500cab40 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.894+0000 7f7a553ff700 1 -- 172.21.15.31:0/882042494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a500b1aa0 con 0x7f7a500cab40 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.895+0000 7f7a553ff700 1 -- 172.21.15.31:0/882042494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5004c380 con 0x7f7a500cab40 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.896+0000 7f7a36ffd700 1 -- 172.21.15.31:0/882042494 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f7a48014830 con 0x7f7a500cab40 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.897+0000 7f7a36ffd700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7a3c067680 0x7f7a3c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.897+0000 7f7a36ffd700 1 -- 172.21.15.31:0/882042494 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f7a48085f50 con 0x7f7a500cab40 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.898+0000 7f7a4effd700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7a3c067680 0x7f7a3c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.899+0000 7f7a4effd700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7a3c067680 0x7f7a3c069b30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000b500 tx=0x7f7a4001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:30.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:29.902+0000 7f7a36ffd700 1 -- 172.21.15.31:0/882042494 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a48083610 con 0x7f7a500cab40 2023-10-06T05:25:30.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.102+0000 7f7a553ff700 1 -- 172.21.15.31:0/882042494 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a500af830 con 0x7f7a3c067680 2023-10-06T05:25:30.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.113+0000 7f7a36ffd700 1 -- 172.21.15.31:0/882042494 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311791 (secure 0 0 0) 0x7f7a500af830 con 0x7f7a3c067680 2023-10-06T05:25:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 -- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7a3c067680 msgr2=0x7f7a3c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7a3c067680 0x7f7a3c069b30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000b500 tx=0x7f7a4001a000 comp rx=0 tx=0).stop 2023-10-06T05:25:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 -- 172.21.15.31:0/882042494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500cab40 msgr2=0x7f7a500ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500cab40 0x7f7a500ac2f0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009e90 tx=0x7f7a4800b040 comp rx=0 tx=0).stop 2023-10-06T05:25:30.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 -- 172.21.15.31:0/882042494 shutdown_connections 2023-10-06T05:25:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7a3c067680 0x7f7a3c069b30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a500cab40 0x7f7a500ac2f0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7a5006ae00 0x7f7a50158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 --2- 172.21.15.31:0/882042494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a500ca770 0x7f7a50158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 -- 172.21.15.31:0/882042494 >> 172.21.15.31:0/882042494 conn(0x7f7a500cc0b0 msgr2=0x7f7a500aab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 -- 172.21.15.31:0/882042494 shutdown_connections 2023-10-06T05:25:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.117+0000 7f7a553ff700 1 -- 172.21.15.31:0/882042494 wait complete. 2023-10-06T05:25:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:25:30.137 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2023-10-06T05:25:29.518354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2426996,"kb_used_data":15924,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547724,"statfs":{"total":773094113280,"available":770608869376,"internally_reserved":0,"allocated":16306176,"data_stored":8191041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001493"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279618+0000","last_change":"2023-10-06T05:25:25.279618+0000","last_active":"2023-10-06T05:25:25.279618+0000","last_peered":"2023-10-06T05:25:25.279618+0000","last_clean":"2023-10-06T05:25:25.279618+0000","last_became_active":"2023-10-06T05:25:25.279327+0000","last_became_peered":"2023-10-06T05:25:25.279327+0000","last_unstale":"2023-10-06T05:25:25.279618+0000","last_undegraded":"2023-10-06T05:25:25.279618+0000","last_fullsized":"2023-10-06T05:25:25.279618+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:21.801742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.272362+0000","last_change":"2023-10-06T05:25:25.272362+0000","last_active":"2023-10-06T05:25:25.272362+0000","last_peered":"2023-10-06T05:25:25.272362+0000","last_clean":"2023-10-06T05:25:25.272362+0000","last_became_active":"2023-10-06T05:25:25.272106+0000","last_became_peered":"2023-10-06T05:25:25.272106+0000","last_unstale":"2023-10-06T05:25:25.272362+0000","last_undegraded":"2023-10-06T05:25:25.272362+0000","last_fullsized":"2023-10-06T05:25:25.272362+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:37.990044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276221+0000","last_change":"2023-10-06T05:25:25.276221+0000","last_active":"2023-10-06T05:25:25.276221+0000","last_peered":"2023-10-06T05:25:25.276221+0000","last_clean":"2023-10-06T05:25:25.276221+0000","last_became_active":"2023-10-06T05:25:25.275933+0000","last_became_peered":"2023-10-06T05:25:25.275933+0000","last_unstale":"2023-10-06T05:25:25.276221+0000","last_undegraded":"2023-10-06T05:25:25.276221+0000","last_fullsized":"2023-10-06T05:25:25.276221+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:57.038031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.281061+0000","last_change":"2023-10-06T05:25:25.281061+0000","last_active":"2023-10-06T05:25:25.281061+0000","last_peered":"2023-10-06T05:25:25.281061+0000","last_clean":"2023-10-06T05:25:25.281061+0000","last_became_active":"2023-10-06T05:25:25.280629+0000","last_became_peered":"2023-10-06T05:25:25.280629+0000","last_unstale":"2023-10-06T05:25:25.281061+0000","last_undegraded":"2023-10-06T05:25:25.281061+0000","last_fullsized":"2023-10-06T05:25:25.281061+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:51:00.819753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.275077+0000","last_change":"2023-10-06T05:25:25.266779+0000","last_active":"2023-10-06T05:25:25.275077+0000","last_peered":"2023-10-06T05:25:25.275077+0000","last_clean":"2023-10-06T05:25:25.275077+0000","last_became_active":"2023-10-06T05:25:25.266537+0000","last_became_peered":"2023-10-06T05:25:25.266537+0000","last_unstale":"2023-10-06T05:25:25.275077+0000","last_undegraded":"2023-10-06T05:25:25.275077+0000","last_fullsized":"2023-10-06T05:25:25.275077+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:58.136155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278653+0000","last_change":"2023-10-06T05:25:25.276076+0000","last_active":"2023-10-06T05:25:25.278653+0000","last_peered":"2023-10-06T05:25:25.278653+0000","last_clean":"2023-10-06T05:25:25.278653+0000","last_became_active":"2023-10-06T05:25:25.275557+0000","last_became_peered":"2023-10-06T05:25:25.275557+0000","last_unstale":"2023-10-06T05:25:25.278653+0000","last_undegraded":"2023-10-06T05:25:25.278653+0000","last_fullsized":"2023-10-06T05:25:25.278653+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:32.494803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.266658+0000","last_change":"2023-10-06T05:25:25.263637+0000","last_active":"2023-10-06T05:25:25.266658+0000","last_peered":"2023-10-06T05:25:25.266658+0000","last_clean":"2023-10-06T05:25:25.266658+0000","last_became_active":"2023-10-06T05:25:25.263361+0000","last_became_peered":"2023-10-06T05:25:25.263361+0000","last_unstale":"2023-10-06T05:25:25.266658+0000","last_undegraded":"2023-10-06T05:25:25.266658+0000","last_fullsized":"2023-10-06T05:25:25.266658+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:05.928144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.271745+0000","last_change":"2023-10-06T05:25:25.271745+0000","last_active":"2023-10-06T05:25:25.271745+0000","last_peered":"2023-10-06T05:25:25.271745+0000","last_clean":"2023-10-06T05:25:25.271745+0000","last_became_active":"2023-10-06T05:25:25.271364+0000","last_became_peered":"2023-10-06T05:25:25.271364+0000","last_unstale":"2023-10-06T05:25:25.271745+0000","last_undegraded":"2023-10-06T05:25:25.271745+0000","last_fullsized":"2023-10-06T05:25:25.271745+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:32.728256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.270002+0000","last_change":"2023-10-06T05:25:25.270002+0000","last_active":"2023-10-06T05:25:25.270002+0000","last_peered":"2023-10-06T05:25:25.270002+0000","last_clean":"2023-10-06T05:25:25.270002+0000","last_became_active":"2023-10-06T05:25:25.269691+0000","last_became_peered":"2023-10-06T05:25:25.269691+0000","last_unstale":"2023-10-06T05:25:25.270002+0000","last_undegraded":"2023-10-06T05:25:25.270002+0000","last_fullsized":"2023-10-06T05:25:25.270002+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:15.338977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278385+0000","last_change":"2023-10-06T05:25:25.274134+0000","last_active":"2023-10-06T05:25:25.278385+0000","last_peered":"2023-10-06T05:25:25.278385+0000","last_clean":"2023-10-06T05:25:25.278385+0000","last_became_active":"2023-10-06T05:25:25.273887+0000","last_became_peered":"2023-10-06T05:25:25.273887+0000","last_unstale":"2023-10-06T05:25:25.278385+0000","last_undegraded":"2023-10-06T05:25:25.278385+0000","last_fullsized":"2023-10-06T05:25:25.278385+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:07.780135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.268341+0000","last_change":"2023-10-06T05:25:25.268341+0000","last_active":"2023-10-06T05:25:25.268341+0000","last_peered":"2023-10-06T05:25:25.268341+0000","last_clean":"2023-10-06T05:25:25.268341+0000","last_became_active":"2023-10-06T05:25:25.268003+0000","last_became_peered":"2023-10-06T05:25:25.268003+0000","last_unstale":"2023-10-06T05:25:25.268341+0000","last_undegraded":"2023-10-06T05:25:25.268341+0000","last_fullsized":"2023-10-06T05:25:25.268341+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:16.299063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279352+0000","last_change":"2023-10-06T05:25:25.279352+0000","last_active":"2023-10-06T05:25:25.279352+0000","last_peered":"2023-10-06T05:25:25.279352+0000","last_clean":"2023-10-06T05:25:25.279352+0000","last_became_active":"2023-10-06T05:25:25.278891+0000","last_became_peered":"2023-10-06T05:25:25.278891+0000","last_unstale":"2023-10-06T05:25:25.279352+0000","last_undegraded":"2023-10-06T05:25:25.279352+0000","last_fullsized":"2023-10-06T05:25:25.279352+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:22.000678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279271+0000","last_change":"2023-10-06T05:25:25.268572+0000","last_active":"2023-10-06T05:25:25.279271+0000","last_peered":"2023-10-06T05:25:25.279271+0000","last_clean":"2023-10-06T05:25:25.279271+0000","last_became_active":"2023-10-06T05:25:25.268247+0000","last_became_peered":"2023-10-06T05:25:25.268247+0000","last_unstale":"2023-10-06T05:25:25.279271+0000","last_undegraded":"2023-10-06T05:25:25.279271+0000","last_fullsized":"2023-10-06T05:25:25.279271+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:02:28.441607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.258739+0000","last_change":"2023-10-06T05:25:25.254819+0000","last_active":"2023-10-06T05:25:25.258739+0000","last_peered":"2023-10-06T05:25:25.258739+0000","last_clean":"2023-10-06T05:25:25.258739+0000","last_became_active":"2023-10-06T05:25:25.254506+0000","last_became_peered":"2023-10-06T05:25:25.254506+0000","last_unstale":"2023-10-06T05:25:25.258739+0000","last_undegraded":"2023-10-06T05:25:25.258739+0000","last_fullsized":"2023-10-06T05:25:25.258739+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:33.338121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.282102+0000","last_change":"2023-10-06T05:25:25.282102+0000","last_active":"2023-10-06T05:25:25.282102+0000","last_peered":"2023-10-06T05:25:25.282102+0000","last_clean":"2023-10-06T05:25:25.282102+0000","last_became_active":"2023-10-06T05:25:25.281684+0000","last_became_peered":"2023-10-06T05:25:25.281684+0000","last_unstale":"2023-10-06T05:25:25.282102+0000","last_undegraded":"2023-10-06T05:25:25.282102+0000","last_fullsized":"2023-10-06T05:25:25.282102+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:16.693212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.274422+0000","last_change":"2023-10-06T05:25:25.267323+0000","last_active":"2023-10-06T05:25:25.274422+0000","last_peered":"2023-10-06T05:25:25.274422+0000","last_clean":"2023-10-06T05:25:25.274422+0000","last_became_active":"2023-10-06T05:25:25.267076+0000","last_became_peered":"2023-10-06T05:25:25.267076+0000","last_unstale":"2023-10-06T05:25:25.274422+0000","last_undegraded":"2023-10-06T05:25:25.274422+0000","last_fullsized":"2023-10-06T05:25:25.274422+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:35.837407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273034+0000","last_change":"2023-10-06T05:25:25.266216+0000","last_active":"2023-10-06T05:25:25.273034+0000","last_peered":"2023-10-06T05:25:25.273034+0000","last_clean":"2023-10-06T05:25:25.273034+0000","last_became_active":"2023-10-06T05:25:25.265897+0000","last_became_peered":"2023-10-06T05:25:25.265897+0000","last_unstale":"2023-10-06T05:25:25.273034+0000","last_undegraded":"2023-10-06T05:25:25.273034+0000","last_fullsized":"2023-10-06T05:25:25.273034+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:13.691031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278662+0000","last_change":"2023-10-06T05:25:25.266699+0000","last_active":"2023-10-06T05:25:25.278662+0000","last_peered":"2023-10-06T05:25:25.278662+0000","last_clean":"2023-10-06T05:25:25.278662+0000","last_became_active":"2023-10-06T05:25:25.265891+0000","last_became_peered":"2023-10-06T05:25:25.265891+0000","last_unstale":"2023-10-06T05:25:25.278662+0000","last_undegraded":"2023-10-06T05:25:25.278662+0000","last_fullsized":"2023-10-06T05:25:25.278662+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:02.659477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273588+0000","last_change":"2023-10-06T05:25:25.264736+0000","last_active":"2023-10-06T05:25:25.273588+0000","last_peered":"2023-10-06T05:25:25.273588+0000","last_clean":"2023-10-06T05:25:25.273588+0000","last_became_active":"2023-10-06T05:25:25.264293+0000","last_became_peered":"2023-10-06T05:25:25.264293+0000","last_unstale":"2023-10-06T05:25:25.273588+0000","last_undegraded":"2023-10-06T05:25:25.273588+0000","last_fullsized":"2023-10-06T05:25:25.273588+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:21.819526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.267966+0000","last_change":"2023-10-06T05:25:25.262232+0000","last_active":"2023-10-06T05:25:25.267966+0000","last_peered":"2023-10-06T05:25:25.267966+0000","last_clean":"2023-10-06T05:25:25.267966+0000","last_became_active":"2023-10-06T05:25:25.261971+0000","last_became_peered":"2023-10-06T05:25:25.261971+0000","last_unstale":"2023-10-06T05:25:25.267966+0000","last_undegraded":"2023-10-06T05:25:25.267966+0000","last_fullsized":"2023-10-06T05:25:25.267966+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:31.654943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.271010+0000","last_change":"2023-10-06T05:25:25.271010+0000","last_active":"2023-10-06T05:25:25.271010+0000","last_peered":"2023-10-06T05:25:25.271010+0000","last_clean":"2023-10-06T05:25:25.271010+0000","last_became_active":"2023-10-06T05:25:25.270642+0000","last_became_peered":"2023-10-06T05:25:25.270642+0000","last_unstale":"2023-10-06T05:25:25.271010+0000","last_undegraded":"2023-10-06T05:25:25.271010+0000","last_fullsized":"2023-10-06T05:25:25.271010+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:55.497869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.270418+0000","last_change":"2023-10-06T05:25:25.270418+0000","last_active":"2023-10-06T05:25:25.270418+0000","last_peered":"2023-10-06T05:25:25.270418+0000","last_clean":"2023-10-06T05:25:25.270418+0000","last_became_active":"2023-10-06T05:25:25.270088+0000","last_became_peered":"2023-10-06T05:25:25.270088+0000","last_unstale":"2023-10-06T05:25:25.270418+0000","last_undegraded":"2023-10-06T05:25:25.270418+0000","last_fullsized":"2023-10-06T05:25:25.270418+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:45.636431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.272713+0000","last_change":"2023-10-06T05:25:25.272713+0000","last_active":"2023-10-06T05:25:25.272713+0000","last_peered":"2023-10-06T05:25:25.272713+0000","last_clean":"2023-10-06T05:25:25.272713+0000","last_became_active":"2023-10-06T05:25:25.272413+0000","last_became_peered":"2023-10-06T05:25:25.272413+0000","last_unstale":"2023-10-06T05:25:25.272713+0000","last_undegraded":"2023-10-06T05:25:25.272713+0000","last_fullsized":"2023-10-06T05:25:25.272713+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:04.468911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.264064+0000","last_change":"2023-10-06T05:25:25.264064+0000","last_active":"2023-10-06T05:25:25.264064+0000","last_peered":"2023-10-06T05:25:25.264064+0000","last_clean":"2023-10-06T05:25:25.264064+0000","last_became_active":"2023-10-06T05:25:25.263751+0000","last_became_peered":"2023-10-06T05:25:25.263751+0000","last_unstale":"2023-10-06T05:25:25.264064+0000","last_undegraded":"2023-10-06T05:25:25.264064+0000","last_fullsized":"2023-10-06T05:25:25.264064+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:35.509669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.265141+0000","last_change":"2023-10-06T05:25:25.259082+0000","last_active":"2023-10-06T05:25:25.265141+0000","last_peered":"2023-10-06T05:25:25.265141+0000","last_clean":"2023-10-06T05:25:25.265141+0000","last_became_active":"2023-10-06T05:25:25.258790+0000","last_became_peered":"2023-10-06T05:25:25.258790+0000","last_unstale":"2023-10-06T05:25:25.265141+0000","last_undegraded":"2023-10-06T05:25:25.265141+0000","last_fullsized":"2023-10-06T05:25:25.265141+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:11:28.372575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.267319+0000","last_change":"2023-10-06T05:25:25.267319+0000","last_active":"2023-10-06T05:25:25.267319+0000","last_peered":"2023-10-06T05:25:25.267319+0000","last_clean":"2023-10-06T05:25:25.267319+0000","last_became_active":"2023-10-06T05:25:25.266840+0000","last_became_peered":"2023-10-06T05:25:25.266840+0000","last_unstale":"2023-10-06T05:25:25.267319+0000","last_undegraded":"2023-10-06T05:25:25.267319+0000","last_fullsized":"2023-10-06T05:25:25.267319+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:21.399027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.268661+0000","last_change":"2023-10-06T05:25:25.268661+0000","last_active":"2023-10-06T05:25:25.268661+0000","last_peered":"2023-10-06T05:25:25.268661+0000","last_clean":"2023-10-06T05:25:25.268661+0000","last_became_active":"2023-10-06T05:25:25.268402+0000","last_became_peered":"2023-10-06T05:25:25.268402+0000","last_unstale":"2023-10-06T05:25:25.268661+0000","last_undegraded":"2023-10-06T05:25:25.268661+0000","last_fullsized":"2023-10-06T05:25:25.268661+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:28.415161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.271407+0000","last_change":"2023-10-06T05:25:25.268099+0000","last_active":"2023-10-06T05:25:25.271407+0000","last_peered":"2023-10-06T05:25:25.271407+0000","last_clean":"2023-10-06T05:25:25.271407+0000","last_became_active":"2023-10-06T05:25:25.267749+0000","last_became_peered":"2023-10-06T05:25:25.267749+0000","last_unstale":"2023-10-06T05:25:25.271407+0000","last_undegraded":"2023-10-06T05:25:25.271407+0000","last_fullsized":"2023-10-06T05:25:25.271407+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:28.593301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.274651+0000","last_change":"2023-10-06T05:25:25.274651+0000","last_active":"2023-10-06T05:25:25.274651+0000","last_peered":"2023-10-06T05:25:25.274651+0000","last_clean":"2023-10-06T05:25:25.274651+0000","last_became_active":"2023-10-06T05:25:25.273907+0000","last_became_peered":"2023-10-06T05:25:25.273907+0000","last_unstale":"2023-10-06T05:25:25.274651+0000","last_undegraded":"2023-10-06T05:25:25.274651+0000","last_fullsized":"2023-10-06T05:25:25.274651+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:03.051773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.274662+0000","last_change":"2023-10-06T05:25:25.274662+0000","last_active":"2023-10-06T05:25:25.274662+0000","last_peered":"2023-10-06T05:25:25.274662+0000","last_clean":"2023-10-06T05:25:25.274662+0000","last_became_active":"2023-10-06T05:25:25.273703+0000","last_became_peered":"2023-10-06T05:25:25.273703+0000","last_unstale":"2023-10-06T05:25:25.274662+0000","last_undegraded":"2023-10-06T05:25:25.274662+0000","last_fullsized":"2023-10-06T05:25:25.274662+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:21.801742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.277033+0000","last_change":"2023-10-06T05:25:25.267988+0000","last_active":"2023-10-06T05:25:25.277033+0000","last_peered":"2023-10-06T05:25:25.277033+0000","last_clean":"2023-10-06T05:25:25.277033+0000","last_became_active":"2023-10-06T05:25:25.267548+0000","last_became_peered":"2023-10-06T05:25:25.267548+0000","last_unstale":"2023-10-06T05:25:25.277033+0000","last_undegraded":"2023-10-06T05:25:25.277033+0000","last_fullsized":"2023-10-06T05:25:25.277033+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:06.624399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.261751+0000","last_change":"2023-10-06T05:25:25.257552+0000","last_active":"2023-10-06T05:25:25.261751+0000","last_peered":"2023-10-06T05:25:25.261751+0000","last_clean":"2023-10-06T05:25:25.261751+0000","last_became_active":"2023-10-06T05:25:25.254730+0000","last_became_peered":"2023-10-06T05:25:25.254730+0000","last_unstale":"2023-10-06T05:25:25.261751+0000","last_undegraded":"2023-10-06T05:25:25.261751+0000","last_fullsized":"2023-10-06T05:25:25.261751+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:12:36.113900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278798+0000","last_change":"2023-10-06T05:25:25.278798+0000","last_active":"2023-10-06T05:25:25.278798+0000","last_peered":"2023-10-06T05:25:25.278798+0000","last_clean":"2023-10-06T05:25:25.278798+0000","last_became_active":"2023-10-06T05:25:25.278526+0000","last_became_peered":"2023-10-06T05:25:25.278526+0000","last_unstale":"2023-10-06T05:25:25.278798+0000","last_undegraded":"2023-10-06T05:25:25.278798+0000","last_fullsized":"2023-10-06T05:25:25.278798+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:56.175405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.253649+0000","last_change":"2023-10-06T05:25:24.265692+0000","last_active":"2023-10-06T05:25:25.253649+0000","last_peered":"2023-10-06T05:25:25.253649+0000","last_clean":"2023-10-06T05:25:25.253649+0000","last_became_active":"2023-10-06T05:25:24.265417+0000","last_became_peered":"2023-10-06T05:25:24.265417+0000","last_unstale":"2023-10-06T05:25:25.253649+0000","last_undegraded":"2023-10-06T05:25:25.253649+0000","last_fullsized":"2023-10-06T05:25:25.253649+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:33.593154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278624+0000","last_change":"2023-10-06T05:25:25.274994+0000","last_active":"2023-10-06T05:25:25.278624+0000","last_peered":"2023-10-06T05:25:25.278624+0000","last_clean":"2023-10-06T05:25:25.278624+0000","last_became_active":"2023-10-06T05:25:25.274721+0000","last_became_peered":"2023-10-06T05:25:25.274721+0000","last_unstale":"2023-10-06T05:25:25.278624+0000","last_undegraded":"2023-10-06T05:25:25.278624+0000","last_fullsized":"2023-10-06T05:25:25.278624+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:59.999831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.268807+0000","last_change":"2023-10-06T05:25:24.266665+0000","last_active":"2023-10-06T05:25:24.268807+0000","last_peered":"2023-10-06T05:25:24.268807+0000","last_clean":"2023-10-06T05:25:24.268807+0000","last_became_active":"2023-10-06T05:25:24.266401+0000","last_became_peered":"2023-10-06T05:25:24.266401+0000","last_unstale":"2023-10-06T05:25:24.268807+0000","last_undegraded":"2023-10-06T05:25:24.268807+0000","last_fullsized":"2023-10-06T05:25:24.268807+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:43.133010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273511+0000","last_change":"2023-10-06T05:25:25.264240+0000","last_active":"2023-10-06T05:25:25.273511+0000","last_peered":"2023-10-06T05:25:25.273511+0000","last_clean":"2023-10-06T05:25:25.273511+0000","last_became_active":"2023-10-06T05:25:25.263873+0000","last_became_peered":"2023-10-06T05:25:25.263873+0000","last_unstale":"2023-10-06T05:25:25.273511+0000","last_undegraded":"2023-10-06T05:25:25.273511+0000","last_fullsized":"2023-10-06T05:25:25.273511+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:53.407207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"149'3","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279258+0000","last_change":"2023-10-06T05:25:24.265708+0000","last_active":"2023-10-06T05:25:25.279258+0000","last_peered":"2023-10-06T05:25:25.279258+0000","last_clean":"2023-10-06T05:25:25.279258+0000","last_became_active":"2023-10-06T05:25:24.265420+0000","last_became_peered":"2023-10-06T05:25:24.265420+0000","last_unstale":"2023-10-06T05:25:25.279258+0000","last_undegraded":"2023-10-06T05:25:25.279258+0000","last_fullsized":"2023-10-06T05:25:25.279258+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:41.113364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.271832+0000","last_change":"2023-10-06T05:25:25.271832+0000","last_active":"2023-10-06T05:25:25.271832+0000","last_peered":"2023-10-06T05:25:25.271832+0000","last_clean":"2023-10-06T05:25:25.271832+0000","last_became_active":"2023-10-06T05:25:25.271567+0000","last_became_peered":"2023-10-06T05:25:25.271567+0000","last_unstale":"2023-10-06T05:25:25.271832+0000","last_undegraded":"2023-10-06T05:25:25.271832+0000","last_fullsized":"2023-10-06T05:25:25.271832+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:24.247757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"149'6","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.284314+0000","last_change":"2023-10-06T05:25:24.265409+0000","last_active":"2023-10-06T05:25:25.284314+0000","last_peered":"2023-10-06T05:25:25.284314+0000","last_clean":"2023-10-06T05:25:25.284314+0000","last_became_active":"2023-10-06T05:25:24.262911+0000","last_became_peered":"2023-10-06T05:25:24.262911+0000","last_unstale":"2023-10-06T05:25:25.284314+0000","last_undegraded":"2023-10-06T05:25:25.284314+0000","last_fullsized":"2023-10-06T05:25:25.284314+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:22.263580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.266626+0000","last_change":"2023-10-06T05:25:25.263652+0000","last_active":"2023-10-06T05:25:25.266626+0000","last_peered":"2023-10-06T05:25:25.266626+0000","last_clean":"2023-10-06T05:25:25.266626+0000","last_became_active":"2023-10-06T05:25:25.263322+0000","last_became_peered":"2023-10-06T05:25:25.263322+0000","last_unstale":"2023-10-06T05:25:25.266626+0000","last_undegraded":"2023-10-06T05:25:25.266626+0000","last_fullsized":"2023-10-06T05:25:25.266626+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:04.506978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"149'2","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.280097+0000","last_change":"2023-10-06T05:25:24.263326+0000","last_active":"2023-10-06T05:25:25.280097+0000","last_peered":"2023-10-06T05:25:25.280097+0000","last_clean":"2023-10-06T05:25:25.280097+0000","last_became_active":"2023-10-06T05:25:24.263036+0000","last_became_peered":"2023-10-06T05:25:24.263036+0000","last_unstale":"2023-10-06T05:25:25.280097+0000","last_undegraded":"2023-10-06T05:25:25.280097+0000","last_fullsized":"2023-10-06T05:25:25.280097+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:56.474397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.268838+0000","last_change":"2023-10-06T05:25:25.265525+0000","last_active":"2023-10-06T05:25:25.268838+0000","last_peered":"2023-10-06T05:25:25.268838+0000","last_clean":"2023-10-06T05:25:25.268838+0000","last_became_active":"2023-10-06T05:25:25.265257+0000","last_became_peered":"2023-10-06T05:25:25.265257+0000","last_unstale":"2023-10-06T05:25:25.268838+0000","last_undegraded":"2023-10-06T05:25:25.268838+0000","last_fullsized":"2023-10-06T05:25:25.268838+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:13.765245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.268748+0000","last_change":"2023-10-06T05:25:24.266271+0000","last_active":"2023-10-06T05:25:24.268748+0000","last_peered":"2023-10-06T05:25:24.268748+0000","last_clean":"2023-10-06T05:25:24.268748+0000","last_became_active":"2023-10-06T05:25:24.265983+0000","last_became_peered":"2023-10-06T05:25:24.265983+0000","last_unstale":"2023-10-06T05:25:24.268748+0000","last_undegraded":"2023-10-06T05:25:24.268748+0000","last_fullsized":"2023-10-06T05:25:24.268748+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:01.290511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278954+0000","last_change":"2023-10-06T05:25:25.273367+0000","last_active":"2023-10-06T05:25:25.278954+0000","last_peered":"2023-10-06T05:25:25.278954+0000","last_clean":"2023-10-06T05:25:25.278954+0000","last_became_active":"2023-10-06T05:25:25.273080+0000","last_became_peered":"2023-10-06T05:25:25.273080+0000","last_unstale":"2023-10-06T05:25:25.278954+0000","last_undegraded":"2023-10-06T05:25:25.278954+0000","last_fullsized":"2023-10-06T05:25:25.278954+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:51.992960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.273029+0000","last_change":"2023-10-06T05:25:24.270592+0000","last_active":"2023-10-06T05:25:24.273029+0000","last_peered":"2023-10-06T05:25:24.273029+0000","last_clean":"2023-10-06T05:25:24.273029+0000","last_became_active":"2023-10-06T05:25:24.270317+0000","last_became_peered":"2023-10-06T05:25:24.270317+0000","last_unstale":"2023-10-06T05:25:24.273029+0000","last_undegraded":"2023-10-06T05:25:24.273029+0000","last_fullsized":"2023-10-06T05:25:24.273029+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:27.406131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.280148+0000","last_change":"2023-10-06T05:25:25.274535+0000","last_active":"2023-10-06T05:25:25.280148+0000","last_peered":"2023-10-06T05:25:25.280148+0000","last_clean":"2023-10-06T05:25:25.280148+0000","last_became_active":"2023-10-06T05:25:25.274282+0000","last_became_peered":"2023-10-06T05:25:25.274282+0000","last_unstale":"2023-10-06T05:25:25.280148+0000","last_undegraded":"2023-10-06T05:25:25.280148+0000","last_fullsized":"2023-10-06T05:25:25.280148+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:10.525443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"149'8","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.275762+0000","last_change":"2023-10-06T05:25:24.263949+0000","last_active":"2023-10-06T05:25:25.275762+0000","last_peered":"2023-10-06T05:25:25.275762+0000","last_clean":"2023-10-06T05:25:25.275762+0000","last_became_active":"2023-10-06T05:25:24.263667+0000","last_became_peered":"2023-10-06T05:25:24.263667+0000","last_unstale":"2023-10-06T05:25:25.275762+0000","last_undegraded":"2023-10-06T05:25:25.275762+0000","last_fullsized":"2023-10-06T05:25:25.275762+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:37.924827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.277071+0000","last_change":"2023-10-06T05:25:25.272698+0000","last_active":"2023-10-06T05:25:25.277071+0000","last_peered":"2023-10-06T05:25:25.277071+0000","last_clean":"2023-10-06T05:25:25.277071+0000","last_became_active":"2023-10-06T05:25:25.272392+0000","last_became_peered":"2023-10-06T05:25:25.272392+0000","last_unstale":"2023-10-06T05:25:25.277071+0000","last_undegraded":"2023-10-06T05:25:25.277071+0000","last_fullsized":"2023-10-06T05:25:25.277071+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:41.378955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"149'1","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.260997+0000","last_change":"2023-10-06T05:25:24.264898+0000","last_active":"2023-10-06T05:25:25.260997+0000","last_peered":"2023-10-06T05:25:25.260997+0000","last_clean":"2023-10-06T05:25:25.260997+0000","last_became_active":"2023-10-06T05:25:24.264626+0000","last_became_peered":"2023-10-06T05:25:24.264626+0000","last_unstale":"2023-10-06T05:25:25.260997+0000","last_undegraded":"2023-10-06T05:25:25.260997+0000","last_fullsized":"2023-10-06T05:25:25.260997+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:02.255681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.260838+0000","last_change":"2023-10-06T05:25:25.258865+0000","last_active":"2023-10-06T05:25:25.260838+0000","last_peered":"2023-10-06T05:25:25.260838+0000","last_clean":"2023-10-06T05:25:25.260838+0000","last_became_active":"2023-10-06T05:25:25.258562+0000","last_became_peered":"2023-10-06T05:25:25.258562+0000","last_unstale":"2023-10-06T05:25:25.260838+0000","last_undegraded":"2023-10-06T05:25:25.260838+0000","last_fullsized":"2023-10-06T05:25:25.260838+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:33.454838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"149'2","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279806+0000","last_change":"2023-10-06T05:25:24.269985+0000","last_active":"2023-10-06T05:25:25.279806+0000","last_peered":"2023-10-06T05:25:25.279806+0000","last_clean":"2023-10-06T05:25:25.279806+0000","last_became_active":"2023-10-06T05:25:24.269630+0000","last_became_peered":"2023-10-06T05:25:24.269630+0000","last_unstale":"2023-10-06T05:25:25.279806+0000","last_undegraded":"2023-10-06T05:25:25.279806+0000","last_fullsized":"2023-10-06T05:25:25.279806+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:47.109597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.267134+0000","last_change":"2023-10-06T05:25:25.267134+0000","last_active":"2023-10-06T05:25:25.267134+0000","last_peered":"2023-10-06T05:25:25.267134+0000","last_clean":"2023-10-06T05:25:25.267134+0000","last_became_active":"2023-10-06T05:25:25.266511+0000","last_became_peered":"2023-10-06T05:25:25.266511+0000","last_unstale":"2023-10-06T05:25:25.267134+0000","last_undegraded":"2023-10-06T05:25:25.267134+0000","last_fullsized":"2023-10-06T05:25:25.267134+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:14.198941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"149'1","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.261360+0000","last_change":"2023-10-06T05:25:24.265327+0000","last_active":"2023-10-06T05:25:25.261360+0000","last_peered":"2023-10-06T05:25:25.261360+0000","last_clean":"2023-10-06T05:25:25.261360+0000","last_became_active":"2023-10-06T05:25:24.265012+0000","last_became_peered":"2023-10-06T05:25:24.265012+0000","last_unstale":"2023-10-06T05:25:25.261360+0000","last_undegraded":"2023-10-06T05:25:25.261360+0000","last_fullsized":"2023-10-06T05:25:25.261360+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:32.329091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273088+0000","last_change":"2023-10-06T05:25:25.273088+0000","last_active":"2023-10-06T05:25:25.273088+0000","last_peered":"2023-10-06T05:25:25.273088+0000","last_clean":"2023-10-06T05:25:25.273088+0000","last_became_active":"2023-10-06T05:25:25.272791+0000","last_became_peered":"2023-10-06T05:25:25.272791+0000","last_unstale":"2023-10-06T05:25:25.273088+0000","last_undegraded":"2023-10-06T05:25:25.273088+0000","last_fullsized":"2023-10-06T05:25:25.273088+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:16.299063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.274205+0000","last_change":"2023-10-06T05:25:24.267882+0000","last_active":"2023-10-06T05:25:24.274205+0000","last_peered":"2023-10-06T05:25:24.274205+0000","last_clean":"2023-10-06T05:25:24.274205+0000","last_became_active":"2023-10-06T05:25:24.267591+0000","last_became_peered":"2023-10-06T05:25:24.267591+0000","last_unstale":"2023-10-06T05:25:24.274205+0000","last_undegraded":"2023-10-06T05:25:24.274205+0000","last_fullsized":"2023-10-06T05:25:24.274205+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:00.390286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273403+0000","last_change":"2023-10-06T05:25:25.273403+0000","last_active":"2023-10-06T05:25:25.273403+0000","last_peered":"2023-10-06T05:25:25.273403+0000","last_clean":"2023-10-06T05:25:25.273403+0000","last_became_active":"2023-10-06T05:25:25.273143+0000","last_became_peered":"2023-10-06T05:25:25.273143+0000","last_unstale":"2023-10-06T05:25:25.273403+0000","last_undegraded":"2023-10-06T05:25:25.273403+0000","last_fullsized":"2023-10-06T05:25:25.273403+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:04.717165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"149'2","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279979+0000","last_change":"2023-10-06T05:25:24.264237+0000","last_active":"2023-10-06T05:25:25.279979+0000","last_peered":"2023-10-06T05:25:25.279979+0000","last_clean":"2023-10-06T05:25:25.279979+0000","last_became_active":"2023-10-06T05:25:24.263952+0000","last_became_peered":"2023-10-06T05:25:24.263952+0000","last_unstale":"2023-10-06T05:25:25.279979+0000","last_undegraded":"2023-10-06T05:25:25.279979+0000","last_fullsized":"2023-10-06T05:25:25.279979+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:23.669063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279631+0000","last_change":"2023-10-06T05:25:25.279631+0000","last_active":"2023-10-06T05:25:25.279631+0000","last_peered":"2023-10-06T05:25:25.279631+0000","last_clean":"2023-10-06T05:25:25.279631+0000","last_became_active":"2023-10-06T05:25:25.279214+0000","last_became_peered":"2023-10-06T05:25:25.279214+0000","last_unstale":"2023-10-06T05:25:25.279631+0000","last_undegraded":"2023-10-06T05:25:25.279631+0000","last_fullsized":"2023-10-06T05:25:25.279631+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:24.568428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.272967+0000","last_change":"2023-10-06T05:25:24.269898+0000","last_active":"2023-10-06T05:25:24.272967+0000","last_peered":"2023-10-06T05:25:24.272967+0000","last_clean":"2023-10-06T05:25:24.272967+0000","last_became_active":"2023-10-06T05:25:24.269579+0000","last_became_peered":"2023-10-06T05:25:24.269579+0000","last_unstale":"2023-10-06T05:25:24.272967+0000","last_undegraded":"2023-10-06T05:25:24.272967+0000","last_fullsized":"2023-10-06T05:25:24.272967+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:26.412224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276557+0000","last_change":"2023-10-06T05:25:25.276557+0000","last_active":"2023-10-06T05:25:25.276557+0000","last_peered":"2023-10-06T05:25:25.276557+0000","last_clean":"2023-10-06T05:25:25.276557+0000","last_became_active":"2023-10-06T05:25:25.276291+0000","last_became_peered":"2023-10-06T05:25:25.276291+0000","last_unstale":"2023-10-06T05:25:25.276557+0000","last_undegraded":"2023-10-06T05:25:25.276557+0000","last_fullsized":"2023-10-06T05:25:25.276557+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:52.140788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.247116+0000","last_change":"2023-10-06T05:25:24.254817+0000","last_active":"2023-10-06T05:25:25.247116+0000","last_peered":"2023-10-06T05:25:25.247116+0000","last_clean":"2023-10-06T05:25:25.247116+0000","last_became_active":"2023-10-06T05:25:24.254555+0000","last_became_peered":"2023-10-06T05:25:24.254555+0000","last_unstale":"2023-10-06T05:25:25.247116+0000","last_undegraded":"2023-10-06T05:25:25.247116+0000","last_fullsized":"2023-10-06T05:25:25.247116+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:16.533031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.264321+0000","last_change":"2023-10-06T05:25:25.260487+0000","last_active":"2023-10-06T05:25:25.264321+0000","last_peered":"2023-10-06T05:25:25.264321+0000","last_clean":"2023-10-06T05:25:25.264321+0000","last_became_active":"2023-10-06T05:25:25.260211+0000","last_became_peered":"2023-10-06T05:25:25.260211+0000","last_unstale":"2023-10-06T05:25:25.264321+0000","last_undegraded":"2023-10-06T05:25:25.264321+0000","last_fullsized":"2023-10-06T05:25:25.264321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:09.814818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.271595+0000","last_change":"2023-10-06T05:25:24.267602+0000","last_active":"2023-10-06T05:25:24.271595+0000","last_peered":"2023-10-06T05:25:24.271595+0000","last_clean":"2023-10-06T05:25:24.271595+0000","last_became_active":"2023-10-06T05:25:24.267296+0000","last_became_peered":"2023-10-06T05:25:24.267296+0000","last_unstale":"2023-10-06T05:25:24.271595+0000","last_undegraded":"2023-10-06T05:25:24.271595+0000","last_fullsized":"2023-10-06T05:25:24.271595+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:07:05.615368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.275003+0000","last_change":"2023-10-06T05:25:25.265537+0000","last_active":"2023-10-06T05:25:25.275003+0000","last_peered":"2023-10-06T05:25:25.275003+0000","last_clean":"2023-10-06T05:25:25.275003+0000","last_became_active":"2023-10-06T05:25:25.265297+0000","last_became_peered":"2023-10-06T05:25:25.265297+0000","last_unstale":"2023-10-06T05:25:25.275003+0000","last_undegraded":"2023-10-06T05:25:25.275003+0000","last_fullsized":"2023-10-06T05:25:25.275003+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:06:57.038031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"149'2","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.278874+0000","last_change":"2023-10-06T05:25:24.268505+0000","last_active":"2023-10-06T05:25:25.278874+0000","last_peered":"2023-10-06T05:25:25.278874+0000","last_clean":"2023-10-06T05:25:25.278874+0000","last_became_active":"2023-10-06T05:25:24.268207+0000","last_became_peered":"2023-10-06T05:25:24.268207+0000","last_unstale":"2023-10-06T05:25:25.278874+0000","last_undegraded":"2023-10-06T05:25:25.278874+0000","last_fullsized":"2023-10-06T05:25:25.278874+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:03.459880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.270619+0000","last_change":"2023-10-06T05:25:25.270619+0000","last_active":"2023-10-06T05:25:25.270619+0000","last_peered":"2023-10-06T05:25:25.270619+0000","last_clean":"2023-10-06T05:25:25.270619+0000","last_became_active":"2023-10-06T05:25:25.270310+0000","last_became_peered":"2023-10-06T05:25:25.270310+0000","last_unstale":"2023-10-06T05:25:25.270619+0000","last_undegraded":"2023-10-06T05:25:25.270619+0000","last_fullsized":"2023-10-06T05:25:25.270619+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:03.051773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.246389+0000","last_change":"2023-10-06T05:25:24.253669+0000","last_active":"2023-10-06T05:25:25.246389+0000","last_peered":"2023-10-06T05:25:25.246389+0000","last_clean":"2023-10-06T05:25:25.246389+0000","last_became_active":"2023-10-06T05:25:24.253397+0000","last_became_peered":"2023-10-06T05:25:24.253397+0000","last_unstale":"2023-10-06T05:25:25.246389+0000","last_undegraded":"2023-10-06T05:25:25.246389+0000","last_fullsized":"2023-10-06T05:25:25.246389+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:25.778323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.269995+0000","last_change":"2023-10-06T05:25:25.269995+0000","last_active":"2023-10-06T05:25:25.269995+0000","last_peered":"2023-10-06T05:25:25.269995+0000","last_clean":"2023-10-06T05:25:25.269995+0000","last_became_active":"2023-10-06T05:25:25.269740+0000","last_became_peered":"2023-10-06T05:25:25.269740+0000","last_unstale":"2023-10-06T05:25:25.269995+0000","last_undegraded":"2023-10-06T05:25:25.269995+0000","last_fullsized":"2023-10-06T05:25:25.269995+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:52.826187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.253506+0000","last_change":"2023-10-06T05:25:24.267381+0000","last_active":"2023-10-06T05:25:25.253506+0000","last_peered":"2023-10-06T05:25:25.253506+0000","last_clean":"2023-10-06T05:25:25.253506+0000","last_became_active":"2023-10-06T05:25:24.267117+0000","last_became_peered":"2023-10-06T05:25:24.267117+0000","last_unstale":"2023-10-06T05:25:25.253506+0000","last_undegraded":"2023-10-06T05:25:25.253506+0000","last_fullsized":"2023-10-06T05:25:25.253506+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:20.991647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276999+0000","last_change":"2023-10-06T05:25:25.270469+0000","last_active":"2023-10-06T05:25:25.276999+0000","last_peered":"2023-10-06T05:25:25.276999+0000","last_clean":"2023-10-06T05:25:25.276999+0000","last_became_active":"2023-10-06T05:25:25.269903+0000","last_became_peered":"2023-10-06T05:25:25.269903+0000","last_unstale":"2023-10-06T05:25:25.276999+0000","last_undegraded":"2023-10-06T05:25:25.276999+0000","last_fullsized":"2023-10-06T05:25:25.276999+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:34.623755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.253798+0000","last_change":"2023-10-06T05:25:24.263152+0000","last_active":"2023-10-06T05:25:25.253798+0000","last_peered":"2023-10-06T05:25:25.253798+0000","last_clean":"2023-10-06T05:25:25.253798+0000","last_became_active":"2023-10-06T05:25:24.262892+0000","last_became_peered":"2023-10-06T05:25:24.262892+0000","last_unstale":"2023-10-06T05:25:25.253798+0000","last_undegraded":"2023-10-06T05:25:25.253798+0000","last_fullsized":"2023-10-06T05:25:25.253798+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:51.131757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.274699+0000","last_change":"2023-10-06T05:25:25.274699+0000","last_active":"2023-10-06T05:25:25.274699+0000","last_peered":"2023-10-06T05:25:25.274699+0000","last_clean":"2023-10-06T05:25:25.274699+0000","last_became_active":"2023-10-06T05:25:25.274375+0000","last_became_peered":"2023-10-06T05:25:25.274375+0000","last_unstale":"2023-10-06T05:25:25.274699+0000","last_undegraded":"2023-10-06T05:25:25.274699+0000","last_fullsized":"2023-10-06T05:25:25.274699+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:34.623755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.272252+0000","last_change":"2023-10-06T05:25:24.267786+0000","last_active":"2023-10-06T05:25:24.272252+0000","last_peered":"2023-10-06T05:25:24.272252+0000","last_clean":"2023-10-06T05:25:24.272252+0000","last_became_active":"2023-10-06T05:25:24.267442+0000","last_became_peered":"2023-10-06T05:25:24.267442+0000","last_unstale":"2023-10-06T05:25:24.272252+0000","last_undegraded":"2023-10-06T05:25:24.272252+0000","last_fullsized":"2023-10-06T05:25:24.272252+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:44.627400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276289+0000","last_change":"2023-10-06T05:25:25.271266+0000","last_active":"2023-10-06T05:25:25.276289+0000","last_peered":"2023-10-06T05:25:25.276289+0000","last_clean":"2023-10-06T05:25:25.276289+0000","last_became_active":"2023-10-06T05:25:25.268309+0000","last_became_peered":"2023-10-06T05:25:25.268309+0000","last_unstale":"2023-10-06T05:25:25.276289+0000","last_undegraded":"2023-10-06T05:25:25.276289+0000","last_fullsized":"2023-10-06T05:25:25.276289+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:28.940256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.271850+0000","last_change":"2023-10-06T05:25:24.265924+0000","last_active":"2023-10-06T05:25:24.271850+0000","last_peered":"2023-10-06T05:25:24.271850+0000","last_clean":"2023-10-06T05:25:24.271850+0000","last_became_active":"2023-10-06T05:25:24.265553+0000","last_became_peered":"2023-10-06T05:25:24.265553+0000","last_unstale":"2023-10-06T05:25:24.271850+0000","last_undegraded":"2023-10-06T05:25:24.271850+0000","last_fullsized":"2023-10-06T05:25:24.271850+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:23.238726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273028+0000","last_change":"2023-10-06T05:25:25.273028+0000","last_active":"2023-10-06T05:25:25.273028+0000","last_peered":"2023-10-06T05:25:25.273028+0000","last_clean":"2023-10-06T05:25:25.273028+0000","last_became_active":"2023-10-06T05:25:25.272695+0000","last_became_peered":"2023-10-06T05:25:25.272695+0000","last_unstale":"2023-10-06T05:25:25.273028+0000","last_undegraded":"2023-10-06T05:25:25.273028+0000","last_fullsized":"2023-10-06T05:25:25.273028+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:22.247139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.247557+0000","last_change":"2023-10-06T05:25:24.262759+0000","last_active":"2023-10-06T05:25:25.247557+0000","last_peered":"2023-10-06T05:25:25.247557+0000","last_clean":"2023-10-06T05:25:25.247557+0000","last_became_active":"2023-10-06T05:25:24.262434+0000","last_became_peered":"2023-10-06T05:25:24.262434+0000","last_unstale":"2023-10-06T05:25:25.247557+0000","last_undegraded":"2023-10-06T05:25:25.247557+0000","last_fullsized":"2023-10-06T05:25:25.247557+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:36.786537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.272667+0000","last_change":"2023-10-06T05:25:25.272667+0000","last_active":"2023-10-06T05:25:25.272667+0000","last_peered":"2023-10-06T05:25:25.272667+0000","last_clean":"2023-10-06T05:25:25.272667+0000","last_became_active":"2023-10-06T05:25:25.272351+0000","last_became_peered":"2023-10-06T05:25:25.272351+0000","last_unstale":"2023-10-06T05:25:25.272667+0000","last_undegraded":"2023-10-06T05:25:25.272667+0000","last_fullsized":"2023-10-06T05:25:25.272667+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:48.118628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.269328+0000","last_change":"2023-10-06T05:25:24.265828+0000","last_active":"2023-10-06T05:25:24.269328+0000","last_peered":"2023-10-06T05:25:24.269328+0000","last_clean":"2023-10-06T05:25:24.269328+0000","last_became_active":"2023-10-06T05:25:24.265526+0000","last_became_peered":"2023-10-06T05:25:24.265526+0000","last_unstale":"2023-10-06T05:25:24.269328+0000","last_undegraded":"2023-10-06T05:25:24.269328+0000","last_fullsized":"2023-10-06T05:25:24.269328+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:24.096593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.273894+0000","last_change":"2023-10-06T05:25:25.273894+0000","last_active":"2023-10-06T05:25:25.273894+0000","last_peered":"2023-10-06T05:25:25.273894+0000","last_clean":"2023-10-06T05:25:25.273894+0000","last_became_active":"2023-10-06T05:25:25.273622+0000","last_became_peered":"2023-10-06T05:25:25.273622+0000","last_unstale":"2023-10-06T05:25:25.273894+0000","last_undegraded":"2023-10-06T05:25:25.273894+0000","last_fullsized":"2023-10-06T05:25:25.273894+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:08.677801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"149'4","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.281212+0000","last_change":"2023-10-06T05:25:24.268047+0000","last_active":"2023-10-06T05:25:25.281212+0000","last_peered":"2023-10-06T05:25:25.281212+0000","last_clean":"2023-10-06T05:25:25.281212+0000","last_became_active":"2023-10-06T05:25:24.267767+0000","last_became_peered":"2023-10-06T05:25:24.267767+0000","last_unstale":"2023-10-06T05:25:25.281212+0000","last_undegraded":"2023-10-06T05:25:25.281212+0000","last_fullsized":"2023-10-06T05:25:25.281212+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.293280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.256100+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:25:24.256100+0000","last_peered":"2023-10-06T05:25:24.256100+0000","last_clean":"2023-10-06T05:25:24.256100+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:25:24.256100+0000","last_undegraded":"2023-10-06T05:25:24.256100+0000","last_fullsized":"2023-10-06T05:25:24.256100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.266799+0000","last_change":"2023-10-06T05:25:25.263932+0000","last_active":"2023-10-06T05:25:25.266799+0000","last_peered":"2023-10-06T05:25:25.266799+0000","last_clean":"2023-10-06T05:25:25.266799+0000","last_became_active":"2023-10-06T05:25:25.263652+0000","last_became_peered":"2023-10-06T05:25:25.263652+0000","last_unstale":"2023-10-06T05:25:25.266799+0000","last_undegraded":"2023-10-06T05:25:25.266799+0000","last_fullsized":"2023-10-06T05:25:25.266799+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:03.211478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"149'1","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276175+0000","last_change":"2023-10-06T05:25:24.267912+0000","last_active":"2023-10-06T05:25:25.276175+0000","last_peered":"2023-10-06T05:25:25.276175+0000","last_clean":"2023-10-06T05:25:25.276175+0000","last_became_active":"2023-10-06T05:25:24.267555+0000","last_became_peered":"2023-10-06T05:25:24.267555+0000","last_unstale":"2023-10-06T05:25:25.276175+0000","last_undegraded":"2023-10-06T05:25:25.276175+0000","last_fullsized":"2023-10-06T05:25:25.276175+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:23.559397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.254899+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:25:24.254899+0000","last_peered":"2023-10-06T05:25:24.254899+0000","last_clean":"2023-10-06T05:25:24.254899+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:25:24.254899+0000","last_undegraded":"2023-10-06T05:25:24.254899+0000","last_fullsized":"2023-10-06T05:25:24.254899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.280761+0000","last_change":"2023-10-06T05:25:25.280761+0000","last_active":"2023-10-06T05:25:25.280761+0000","last_peered":"2023-10-06T05:25:25.280761+0000","last_clean":"2023-10-06T05:25:25.280761+0000","last_became_active":"2023-10-06T05:25:25.280478+0000","last_became_peered":"2023-10-06T05:25:25.280478+0000","last_unstale":"2023-10-06T05:25:25.280761+0000","last_undegraded":"2023-10-06T05:25:25.280761+0000","last_fullsized":"2023-10-06T05:25:25.280761+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:14.198941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.250303+0000","last_change":"2023-10-06T05:25:24.263518+0000","last_active":"2023-10-06T05:25:25.250303+0000","last_peered":"2023-10-06T05:25:25.250303+0000","last_clean":"2023-10-06T05:25:25.250303+0000","last_became_active":"2023-10-06T05:25:24.263218+0000","last_became_peered":"2023-10-06T05:25:24.263218+0000","last_unstale":"2023-10-06T05:25:25.250303+0000","last_undegraded":"2023-10-06T05:25:25.250303+0000","last_fullsized":"2023-10-06T05:25:25.250303+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:54.016171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.254803+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:25:24.254803+0000","last_peered":"2023-10-06T05:25:24.254803+0000","last_clean":"2023-10-06T05:25:24.254803+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:25:24.254803+0000","last_undegraded":"2023-10-06T05:25:24.254803+0000","last_fullsized":"2023-10-06T05:25:24.254803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.271103+0000","last_change":"2023-10-06T05:25:25.271103+0000","last_active":"2023-10-06T05:25:25.271103+0000","last_peered":"2023-10-06T05:25:25.271103+0000","last_clean":"2023-10-06T05:25:25.271103+0000","last_became_active":"2023-10-06T05:25:25.270803+0000","last_became_peered":"2023-10-06T05:25:25.270803+0000","last_unstale":"2023-10-06T05:25:25.271103+0000","last_undegraded":"2023-10-06T05:25:25.271103+0000","last_fullsized":"2023-10-06T05:25:25.271103+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:34.101563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.272411+0000","last_change":"2023-10-06T05:25:24.265008+0000","last_active":"2023-10-06T05:25:24.272411+0000","last_peered":"2023-10-06T05:25:24.272411+0000","last_clean":"2023-10-06T05:25:24.272411+0000","last_became_active":"2023-10-06T05:25:24.264440+0000","last_became_peered":"2023-10-06T05:25:24.264440+0000","last_unstale":"2023-10-06T05:25:24.272411+0000","last_undegraded":"2023-10-06T05:25:24.272411+0000","last_fullsized":"2023-10-06T05:25:24.272411+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:10.490754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.252680+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:25:24.252680+0000","last_peered":"2023-10-06T05:25:24.252680+0000","last_clean":"2023-10-06T05:25:24.252680+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:25:24.252680+0000","last_undegraded":"2023-10-06T05:25:24.252680+0000","last_fullsized":"2023-10-06T05:25:24.252680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.263379+0000","last_change":"2023-10-06T05:25:25.263379+0000","last_active":"2023-10-06T05:25:25.263379+0000","last_peered":"2023-10-06T05:25:25.263379+0000","last_clean":"2023-10-06T05:25:25.263379+0000","last_became_active":"2023-10-06T05:25:25.262958+0000","last_became_peered":"2023-10-06T05:25:25.262958+0000","last_unstale":"2023-10-06T05:25:25.263379+0000","last_undegraded":"2023-10-06T05:25:25.263379+0000","last_fullsized":"2023-10-06T05:25:25.263379+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:21.399027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"149'1","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.262816+0000","last_change":"2023-10-06T05:25:24.266518+0000","last_active":"2023-10-06T05:25:25.262816+0000","last_peered":"2023-10-06T05:25:25.262816+0000","last_clean":"2023-10-06T05:25:25.262816+0000","last_became_active":"2023-10-06T05:25:24.266198+0000","last_became_peered":"2023-10-06T05:25:24.266198+0000","last_unstale":"2023-10-06T05:25:25.262816+0000","last_undegraded":"2023-10-06T05:25:25.262816+0000","last_fullsized":"2023-10-06T05:25:25.262816+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:49.522768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":113,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.251377+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:25:24.251377+0000","last_peered":"2023-10-06T05:25:24.251377+0000","last_clean":"2023-10-06T05:25:24.251377+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:25:24.251377+0000","last_undegraded":"2023-10-06T05:25:24.251377+0000","last_fullsized":"2023-10-06T05:25:24.251377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276459+0000","last_change":"2023-10-06T05:25:25.267298+0000","last_active":"2023-10-06T05:25:25.276459+0000","last_peered":"2023-10-06T05:25:25.276459+0000","last_clean":"2023-10-06T05:25:25.276459+0000","last_became_active":"2023-10-06T05:25:25.266888+0000","last_became_peered":"2023-10-06T05:25:25.266888+0000","last_unstale":"2023-10-06T05:25:25.276459+0000","last_undegraded":"2023-10-06T05:25:25.276459+0000","last_fullsized":"2023-10-06T05:25:25.276459+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:48.283153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"149'1","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.274716+0000","last_change":"2023-10-06T05:25:24.267289+0000","last_active":"2023-10-06T05:25:25.274716+0000","last_peered":"2023-10-06T05:25:25.274716+0000","last_clean":"2023-10-06T05:25:25.274716+0000","last_became_active":"2023-10-06T05:25:24.266998+0000","last_became_peered":"2023-10-06T05:25:24.266998+0000","last_unstale":"2023-10-06T05:25:25.274716+0000","last_undegraded":"2023-10-06T05:25:25.274716+0000","last_fullsized":"2023-10-06T05:25:25.274716+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:50:59.810722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":302,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.252596+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:25:24.252596+0000","last_peered":"2023-10-06T05:25:24.252596+0000","last_clean":"2023-10-06T05:25:24.252596+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:25:24.252596+0000","last_undegraded":"2023-10-06T05:25:24.252596+0000","last_fullsized":"2023-10-06T05:25:24.252596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":127,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T05:25:24.242827+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:25:24.242827+0000","last_peered":"2023-10-06T05:25:24.242827+0000","last_clean":"2023-10-06T05:25:24.242827+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:25:24.242827+0000","last_undegraded":"2023-10-06T05:25:24.242827+0000","last_fullsized":"2023-10-06T05:25:24.242827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.277127+0000","last_change":"2023-10-06T05:25:25.277127+0000","last_active":"2023-10-06T05:25:25.277127+0000","last_peered":"2023-10-06T05:25:25.277127+0000","last_clean":"2023-10-06T05:25:25.277127+0000","last_became_active":"2023-10-06T05:25:25.276802+0000","last_became_peered":"2023-10-06T05:25:25.276802+0000","last_unstale":"2023-10-06T05:25:25.277127+0000","last_undegraded":"2023-10-06T05:25:25.277127+0000","last_fullsized":"2023-10-06T05:25:25.277127+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:41.120528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"149'2","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279875+0000","last_change":"2023-10-06T05:25:24.257400+0000","last_active":"2023-10-06T05:25:25.279875+0000","last_peered":"2023-10-06T05:25:25.279875+0000","last_clean":"2023-10-06T05:25:25.279875+0000","last_became_active":"2023-10-06T05:25:24.257112+0000","last_became_peered":"2023-10-06T05:25:24.257112+0000","last_unstale":"2023-10-06T05:25:25.279875+0000","last_undegraded":"2023-10-06T05:25:25.279875+0000","last_fullsized":"2023-10-06T05:25:25.279875+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:04.407229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":143,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.253889+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:25:25.253889+0000","last_peered":"2023-10-06T05:25:25.253889+0000","last_clean":"2023-10-06T05:25:25.253889+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:25:25.253889+0000","last_undegraded":"2023-10-06T05:25:25.253889+0000","last_fullsized":"2023-10-06T05:25:25.253889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":8,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.279012+0000","last_change":"2023-10-06T05:25:25.279012+0000","last_active":"2023-10-06T05:25:25.279012+0000","last_peered":"2023-10-06T05:25:25.279012+0000","last_clean":"2023-10-06T05:25:25.279012+0000","last_became_active":"2023-10-06T05:25:25.278683+0000","last_became_peered":"2023-10-06T05:25:25.278683+0000","last_unstale":"2023-10-06T05:25:25.279012+0000","last_undegraded":"2023-10-06T05:25:25.279012+0000","last_fullsized":"2023-10-06T05:25:25.279012+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:24.232697+0000","last_clean_scrub_stamp":"2023-10-06T05:25:24.232697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:04.717165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"149'2","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.276250+0000","last_change":"2023-10-06T05:25:24.269896+0000","last_active":"2023-10-06T05:25:25.276250+0000","last_peered":"2023-10-06T05:25:25.276250+0000","last_clean":"2023-10-06T05:25:25.276250+0000","last_became_active":"2023-10-06T05:25:24.269631+0000","last_became_peered":"2023-10-06T05:25:24.269631+0000","last_unstale":"2023-10-06T05:25:25.276250+0000","last_undegraded":"2023-10-06T05:25:25.276250+0000","last_fullsized":"2023-10-06T05:25:25.276250+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:25:23.223666+0000","last_clean_scrub_stamp":"2023-10-06T05:25:23.223666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:20.810496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":131,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-06T05:25:25.251241+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:25:25.251241+0000","last_peered":"2023-10-06T05:25:25.251241+0000","last_clean":"2023-10-06T05:25:25.251241+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:25:25.251241+0000","last_undegraded":"2023-10-06T05:25:25.251241+0000","last_fullsized":"2023-10-06T05:25:25.251241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303800,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068040,"statfs":{"total":96636764160,"available":96325672960,"internally_reserved":0,"allocated":2473984,"data_stored":1465246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.50800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71199999999999997,"15min":0.72099999999999997},"min":{"1min":0.51800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83899999999999997,"5min":1.081,"15min":1.081},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70499999999999996,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.48099999999999998,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91700000000000004,"5min":0.93600000000000005,"15min":1.018},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.73699999999999999},"min":{"1min":0.51200000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90000000000000002,"5min":0.97799999999999998,"15min":1.056},"last":0.70599999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71099999999999997,"15min":0.73999999999999999},"min":{"1min":0.53400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94199999999999995,"5min":1.149,"15min":1.149},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.72699999999999998,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65700000000000003,"15min":0.69999999999999996},"min":{"1min":0.37,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86499999999999999,"5min":0.96199999999999997,"15min":1.0249999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.76600000000000001},"min":{"1min":0.46899999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97499999999999998,"5min":1.27,"15min":1.27},"last":0.41999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69699999999999995,"15min":0.72899999999999998},"min":{"1min":0.502,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91300000000000003,"5min":0.97899999999999998,"15min":0.999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75,"15min":0.80900000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90600000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.64000000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79500000000000004,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96499999999999997,"5min":1.0449999999999999,"15min":1.1040000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.52200000000000002,"5min":0.46200000000000002,"15min":0.441},"max":{"1min":1.0089999999999999,"5min":1.248,"15min":1.248},"last":0.51700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303224,"kb_used_data":1840,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068616,"statfs":{"total":96636764160,"available":96326262784,"internally_reserved":0,"allocated":1884160,"data_stored":874948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63900000000000001,"15min":0.67000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.89800000000000002,"15min":1.8500000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66500000000000004,"15min":0.69699999999999995},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0369999999999999,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69099999999999995,"15min":0.70699999999999996},"min":{"1min":0.50800000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93000000000000005,"5min":1.177,"15min":1.9279999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95799999999999996,"5min":1.073,"15min":1.4239999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.65200000000000002},"min":{"1min":0.47299999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.877,"5min":1.093,"15min":1.369},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71299999999999997,"15min":0.751},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.038,"5min":1.085,"15min":1.891},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71599999999999997,"15min":0.74099999999999999},"min":{"1min":0.49299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.347},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.71399999999999997},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.98699999999999999,"15min":1},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70999999999999996,"15min":0.75900000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.9670000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77800000000000002,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0209999999999999,"5min":1.278,"15min":1.4510000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.50700000000000001,"5min":0.41999999999999998,"15min":0.41499999999999998},"max":{"1min":0.82899999999999996,"5min":1.0680000000000001,"15min":1.415},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.753,"15min":0.80400000000000005},"min":{"1min":0.52300000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1020000000000001,"5min":1.2350000000000001,"15min":2},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77200000000000002,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.016,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76400000000000001,"15min":0.77500000000000002},"min":{"1min":0.54900000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95799999999999996,"5min":1.133,"15min":1.46},"last":0.82099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303800,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068040,"statfs":{"total":96636764160,"available":96325672960,"internally_reserved":0,"allocated":2473984,"data_stored":1465246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.44,"15min":0.436},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.621,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.48599999999999999},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.504,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.434}]},{"osd":2,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.54100000000000004,"5min":0.77000000000000002,"15min":0.83799999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.59999999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.44900000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.58499999999999996,"5min":0.80500000000000005,"15min":0.92400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51000000000000001,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.57299999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.495}]},{"osd":4,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.52400000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0680000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79200000000000004,"15min":0.79300000000000004},"min":{"1min":0.63600000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0189999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74099999999999999,"15min":0.74399999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.81000000000000005},"min":{"1min":0.61599999999999999,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":1.002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73399999999999999,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93799999999999994,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1140000000000001,"5min":1.387,"15min":1.387},"last":0.86799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.50600000000000001},"max":{"1min":1.044,"5min":1.244,"15min":1.244},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81399999999999995,"15min":0.82199999999999995},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303236,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068604,"statfs":{"total":96636764160,"available":96326250496,"internally_reserved":0,"allocated":1896448,"data_stored":875578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41499999999999998,"15min":0.42799999999999999},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.51300000000000001,"5min":0.64500000000000002,"15min":0.84499999999999997},"last":0.436},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.44500000000000001,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":1.042},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.48799999999999999},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.624,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.63,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.48199999999999998,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.67500000000000004,"5min":0.80100000000000005,"15min":0.90500000000000003},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.745},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.80000000000000004,"5min":1.1519999999999999,"15min":1.242},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.76700000000000002},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97299999999999998,"5min":0.97899999999999998,"15min":1.198},"last":0.78900000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73399999999999999,"15min":0.76100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.90900000000000003,"5min":1.1659999999999999,"15min":1.26},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73099999999999998,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84099999999999997,"5min":1.1299999999999999,"15min":1.163},"last":0.874}]},{"osd":6,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73499999999999999,"15min":0.78600000000000003},"min":{"1min":0.55300000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92500000000000004,"5min":1.105,"15min":1.323},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76300000000000001,"15min":0.78600000000000003},"min":{"1min":0.54600000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95099999999999996,"5min":1.123,"15min":1.3100000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77400000000000002,"15min":0.82199999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.52400000000000002},"max":{"1min":0.84999999999999998,"5min":1.2010000000000001,"15min":1.264},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.81100000000000005},"min":{"1min":0.58199999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":0.77300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303224,"kb_used_data":1840,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068616,"statfs":{"total":96636764160,"available":96326262784,"internally_reserved":0,"allocated":1884160,"data_stored":875432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.77800000000000002},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.46000000000000002,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.55400000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45100000000000001,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.57399999999999995,"5min":0.71699999999999997,"15min":0.88700000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.31900000000000001},"max":{"1min":0.57799999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.34100000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.627,"5min":0.80600000000000005,"15min":0.93600000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.38500000000000001,"5min":0.372,"15min":0.36199999999999999},"max":{"1min":0.57299999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.30499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.997,"5min":1.022,"15min":1.1299999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.77800000000000002},"min":{"1min":0.53300000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.018,"5min":1.085,"15min":1.1699999999999999},"last":0.875}]},{"osd":5,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.52400000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91600000000000004,"5min":1.042,"15min":1.099},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73699999999999999,"15min":0.78900000000000003},"min":{"1min":0.55600000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.163},"last":0.92500000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.81799999999999995},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98599999999999999,"5min":1.056,"15min":1.21},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.095}]},{"osd":7,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.79900000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1200000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.77100000000000002,"15min":0.83899999999999997},"min":{"1min":0.58999999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97099999999999997,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":1.0309999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303228,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068612,"statfs":{"total":96636764160,"available":96326258688,"internally_reserved":0,"allocated":1888256,"data_stored":875038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42999999999999999,"15min":0.45300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65500000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.58299999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.45100000000000001,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54000000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.497,"15min":0.52300000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72199999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.504,"15min":0.53500000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.57699999999999996,"5min":0.999,"15min":0.999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.53400000000000003,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.58599999999999997,"5min":0.98999999999999999,"15min":1.014},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":0.80100000000000005},"min":{"1min":0.53300000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.0800000000000001,"5min":1.145,"15min":1.2669999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.82299999999999995},"min":{"1min":0.64000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.016,"5min":1.1830000000000001,"15min":1.3540000000000001},"last":0.628}]},{"osd":5,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72899999999999998,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93100000000000005,"5min":1.093,"15min":1.204},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77100000000000002,"15min":0.81999999999999995},"min":{"1min":0.52400000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":1.208,"15min":1.329},"last":0.57599999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75600000000000001,"15min":0.80000000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.96899999999999997,"5min":1.0880000000000001,"15min":1.484},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77400000000000002,"15min":0.82899999999999996},"min":{"1min":0.55900000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98299999999999998,"5min":1.2809999999999999,"15min":1.629},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78300000000000003,"15min":0.83599999999999997},"min":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0469999999999999,"5min":1.1950000000000001,"15min":1.5620000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.79100000000000004,"15min":0.82199999999999995},"min":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":1.0309999999999999,"5min":1.194,"15min":1.325},"last":0.59299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303248,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068592,"statfs":{"total":96636764160,"available":96326238208,"internally_reserved":0,"allocated":1908736,"data_stored":883117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.75600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.95899999999999996,"5min":1.02,"15min":2.77},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.46899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.6419999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67900000000000005,"15min":0.71099999999999997},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.99199999999999999,"5min":1.1100000000000001,"15min":1.1899999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.39800000000000002},"max":{"1min":0.93799999999999994,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71799999999999997,"15min":0.79300000000000004},"min":{"1min":0.52100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.52900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89000000000000001,"5min":0.95899999999999996,"15min":1.0629999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70799999999999996,"15min":0.96299999999999997},"min":{"1min":0.52700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97899999999999998,"5min":0.99299999999999999,"15min":7.742},"last":0.745}]},{"osd":5,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.51000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.079},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.72499999999999998,"15min":0.746},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.012,"5min":1.04,"15min":1.181},"last":0.627}]},{"osd":6,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92900000000000005,"5min":1.0029999999999999,"15min":1.2669999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77300000000000002,"15min":0.80800000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.436},"max":{"1min":0.95399999999999996,"5min":1.093,"15min":1.171},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78600000000000003,"15min":0.81999999999999995},"min":{"1min":0.69599999999999995,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0589999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78400000000000003,"15min":0.83999999999999997},"min":{"1min":0.59199999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.403,"5min":1.403,"15min":1.571},"last":0.69899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303236,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068604,"statfs":{"total":96636764160,"available":96326250496,"internally_reserved":0,"allocated":1896448,"data_stored":876436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63400000000000001,"15min":0.878},"min":{"1min":0.38400000000000001,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.95599999999999996,"5min":1.109,"15min":9.2780000000000005},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0469999999999999},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66800000000000004,"15min":0.90400000000000003},"min":{"1min":0.433,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.874,"5min":1.079,"15min":9.25},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67800000000000005,"15min":0.95399999999999996},"min":{"1min":0.48699999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.92500000000000004},"min":{"1min":0.51600000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84999999999999998,"5min":1.052,"15min":10.384},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71099999999999997,"15min":0.95399999999999996},"min":{"1min":0.48199999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98699999999999999,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.98799999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.495},"max":{"1min":0.88900000000000001,"5min":1.76,"15min":10.205},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70299999999999996,"15min":0.71999999999999997},"min":{"1min":0.51600000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.024,"5min":1.097,"15min":1.292},"last":0.749}]},{"osd":4,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.40600000000000003},"max":{"1min":0.95699999999999996,"5min":1.069,"15min":1.069},"last":0.496},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0469999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.73299999999999998},"min":{"1min":0.55300000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91300000000000003,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76400000000000001,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.298},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78000000000000003,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98699999999999999,"5min":1.556,"15min":1.556},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76800000000000002,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.121,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.64800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:25:30.145 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:25:30.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:25:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.379+0000 7fac541ff700 1 Processor -- start 2023-10-06T05:25:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.379+0000 7fac541ff700 1 -- start start 2023-10-06T05:25:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.380+0000 7fac541ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 0x7fac4c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.380+0000 7fac541ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c15d2d0 con 0x7fac4c0cd580 2023-10-06T05:25:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.380+0000 7fac541ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c15d5b0 con 0x7fac4c159560 2023-10-06T05:25:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.380+0000 7fac541ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c15d890 con 0x7fac4c159930 2023-10-06T05:25:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac5279c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fac4c0cd580 0x7fac4c0cb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59268/0 (socket says 172.21.15.31:59268) 2023-10-06T05:25:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac5279c700 1 -- 172.21.15.31:0/2041833697 learned_addr learned my addr 172.21.15.31:0/2041833697 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac5179a700 1 --2- 172.21.15.31:0/2041833697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 0x7fac4c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac50f99700 1 -- 172.21.15.31:0/2041833697 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268207034 0 0) 0x7fac4c15d2d0 con 0x7fac4c0cd580 2023-10-06T05:25:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac50f99700 1 -- 172.21.15.31:0/2041833697 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac38003570 con 0x7fac4c0cd580 2023-10-06T05:25:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac50f99700 1 -- 172.21.15.31:0/2041833697 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062494155 0 0) 0x7fac4c15d5b0 con 0x7fac4c159560 2023-10-06T05:25:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac50f99700 1 -- 172.21.15.31:0/2041833697 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac4c15d2d0 con 0x7fac4c159560 2023-10-06T05:25:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac5179a700 1 -- 172.21.15.31:0/2041833697 >> v1:172.21.15.62:6789/0 conn(0x7fac4c159560 legacy=0x7fac4c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac5179a700 1 -- 172.21.15.31:0/2041833697 >> v1:172.21.15.31:6789/0 conn(0x7fac4c0cd580 legacy=0x7fac4c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac50f99700 1 -- 172.21.15.31:0/2041833697 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4126144443 0 0) 0x7fac38003570 con 0x7fac4c0cd580 2023-10-06T05:25:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.381+0000 7fac5179a700 1 -- 172.21.15.31:0/2041833697 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c15db70 con 0x7fac4c159930 2023-10-06T05:25:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.382+0000 7fac5179a700 1 --2- 172.21.15.31:0/2041833697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 0x7fac4c0b2470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009910 tx=0x7fac3c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d408dbf1d31fd3f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.382+0000 7fac50f99700 1 -- 172.21.15.31:0/2041833697 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440017f0 con 0x7fac4c159930 2023-10-06T05:25:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.382+0000 7fac50f99700 1 -- 172.21.15.31:0/2041833697 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c011420 con 0x7fac4c159930 2023-10-06T05:25:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.382+0000 7fac50f99700 1 -- 172.21.15.31:0/2041833697 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0117c0 con 0x7fac4c159930 2023-10-06T05:25:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.382+0000 7fac541ff700 1 -- 172.21.15.31:0/2041833697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 msgr2=0x7fac4c0b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.382+0000 7fac541ff700 1 --2- 172.21.15.31:0/2041833697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 0x7fac4c0b2470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009910 tx=0x7fac3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.383+0000 7fac541ff700 1 -- 172.21.15.31:0/2041833697 shutdown_connections 2023-10-06T05:25:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.383+0000 7fac541ff700 1 --2- 172.21.15.31:0/2041833697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 0x7fac4c0b2470 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.383+0000 7fac541ff700 1 -- 172.21.15.31:0/2041833697 >> 172.21.15.31:0/2041833697 conn(0x7fac4c06ec90 msgr2=0x7fac4c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.383+0000 7fac541ff700 1 -- 172.21.15.31:0/2041833697 shutdown_connections 2023-10-06T05:25:30.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.383+0000 7fac541ff700 1 -- 172.21.15.31:0/2041833697 wait complete. 2023-10-06T05:25:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.384+0000 7fac541ff700 1 Processor -- start 2023-10-06T05:25:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.384+0000 7fac541ff700 1 -- start start 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.385+0000 7fac541ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac4c0cd580 0x7fac4c14fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.385+0000 7fac541ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c159560 0x7fac4c0cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.385+0000 7fac541ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 0x7fac4c0ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.385+0000 7fac541ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0c88c0 con 0x7fac4c159560 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.385+0000 7fac541ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0c5690 con 0x7fac4c0cd580 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.385+0000 7fac541ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0c4df0 con 0x7fac4c159930 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.385+0000 7fac5179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c159560 0x7fac4c0cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.385+0000 7fac5179a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c159560 0x7fac4c0cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47164/0 (socket says 172.21.15.31:47164) 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.385+0000 7fac5179a700 1 -- 172.21.15.31:0/1577875481 learned_addr learned my addr 172.21.15.31:0/1577875481 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac5279c700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 0x7fac4c0ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac51f9b700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac4c0cd580 0x7fac4c14fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac5179a700 1 -- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 msgr2=0x7fac4c0ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac5179a700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 0x7fac4c0ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac5179a700 1 -- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac4c0cd580 msgr2=0x7fac4c14fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac5179a700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac4c0cd580 0x7fac4c14fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac5179a700 1 -- 172.21.15.31:0/1577875481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c009420 con 0x7fac4c159560 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac51f9b700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac4c0cd580 0x7fac4c14fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac5279c700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 0x7fac4c0ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac5179a700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c159560 0x7fac4c0cca40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0091e0 tx=0x7fac3c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac41ffb700 1 -- 172.21.15.31:0/1577875481 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c01cc90 con 0x7fac4c159560 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac41ffb700 1 -- 172.21.15.31:0/1577875481 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c01cdf0 con 0x7fac4c159560 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac41ffb700 1 -- 172.21.15.31:0/1577875481 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c01a6a0 con 0x7fac4c159560 2023-10-06T05:25:30.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.386+0000 7fac541ff700 1 -- 172.21.15.31:0/1577875481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c0c50d0 con 0x7fac4c159560 2023-10-06T05:25:30.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.387+0000 7fac541ff700 1 -- 172.21.15.31:0/1577875481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c0c8af0 con 0x7fac4c159560 2023-10-06T05:25:30.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.388+0000 7fac541ff700 1 -- 172.21.15.31:0/1577875481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c0c8ce0 con 0x7fac4c159560 2023-10-06T05:25:30.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.389+0000 7fac41ffb700 1 -- 172.21.15.31:0/1577875481 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fac3c01aad0 con 0x7fac4c159560 2023-10-06T05:25:30.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.389+0000 7fac41ffb700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac30067680 0x7fac30069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.389+0000 7fac41ffb700 1 -- 172.21.15.31:0/1577875481 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fac3c084e00 con 0x7fac4c159560 2023-10-06T05:25:30.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.390+0000 7fac51f9b700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac30067680 0x7fac30069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.390+0000 7fac51f9b700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac30067680 0x7fac30069b30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fac48007fd0 tx=0x7fac48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:30.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.393+0000 7fac41ffb700 1 -- 172.21.15.31:0/1577875481 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac3c084b10 con 0x7fac4c159560 2023-10-06T05:25:30.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.676+0000 7fac541ff700 1 -- 172.21.15.31:0/1577875481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fac4c04c380 con 0x7fac4c159560 2023-10-06T05:25:30.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.680+0000 7fac41ffb700 1 -- 172.21.15.31:0/1577875481 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7fac3c053900 con 0x7fac4c159560 2023-10-06T05:25:30.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 -- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac30067680 msgr2=0x7fac30069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac30067680 0x7fac30069b30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fac48007fd0 tx=0x7fac48009040 comp rx=0 tx=0).stop 2023-10-06T05:25:30.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 -- 172.21.15.31:0/1577875481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c159560 msgr2=0x7fac4c0cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c159560 0x7fac4c0cca40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0091e0 tx=0x7fac3c016000 comp rx=0 tx=0).stop 2023-10-06T05:25:30.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 -- 172.21.15.31:0/1577875481 shutdown_connections 2023-10-06T05:25:30.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac30067680 0x7fac30069b30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac4c159930 0x7fac4c0ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac4c0cd580 0x7fac4c14fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 --2- 172.21.15.31:0/1577875481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac4c159560 0x7fac4c0cca40 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 -- 172.21.15.31:0/1577875481 >> 172.21.15.31:0/1577875481 conn(0x7fac4c06ec90 msgr2=0x7fac4c158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:30.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 -- 172.21.15.31:0/1577875481 shutdown_connections 2023-10-06T05:25:30.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.683+0000 7fac541ff700 1 -- 172.21.15.31:0/1577875481 wait complete. 2023-10-06T05:25:30.687 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:25:30.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:30.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.896+0000 7f9fcc3ff700 1 Processor -- start 2023-10-06T05:25:30.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.897+0000 7f9fcc3ff700 1 -- start start 2023-10-06T05:25:30.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.898+0000 7f9fcc3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9fc4068d80 0x7f9fc4159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.898+0000 7f9fcc3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40c95b0 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.898+0000 7f9fcc3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40c9890 con 0x7f9fc41595a0 2023-10-06T05:25:30.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.898+0000 7f9fcc3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc40c9b70 con 0x7f9fc4068d80 2023-10-06T05:25:30.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.898+0000 7f9fca99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9fc40cc0d0 0x7f9fc40c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59270/0 (socket says 172.21.15.31:59270) 2023-10-06T05:25:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.898+0000 7f9fca99c700 1 -- 172.21.15.31:0/4136340474 learned_addr learned my addr 172.21.15.31:0/4136340474 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.898+0000 7f9fc999a700 1 --2- 172.21.15.31:0/4136340474 >> v2:172.21.15.62:3300/0 conn(0x7f9fc4068d80 0x7f9fc4159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.899+0000 7f9fc9199700 1 -- 172.21.15.31:0/4136340474 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742029342 0 0) 0x7f9fc40c95b0 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.899+0000 7f9fc9199700 1 -- 172.21.15.31:0/4136340474 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb0003570 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.899+0000 7f9fc999a700 1 -- 172.21.15.31:0/4136340474 >> v1:172.21.15.62:6789/0 conn(0x7f9fc41595a0 legacy=0x7f9fc4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.899+0000 7f9fc999a700 1 -- 172.21.15.31:0/4136340474 >> v1:172.21.15.31:6789/0 conn(0x7f9fc40cc0d0 legacy=0x7f9fc40c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.899+0000 7f9fc999a700 1 -- 172.21.15.31:0/4136340474 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40ca130 con 0x7f9fc4068d80 2023-10-06T05:25:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.899+0000 7f9fc999a700 1 --2- 172.21.15.31:0/4136340474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068d80 0x7f9fc4159970 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009910 tx=0x7f9fb40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8602ecce3c32829 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.900+0000 7f9fc9199700 1 -- 172.21.15.31:0/4136340474 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400b640 con 0x7f9fc4068d80 2023-10-06T05:25:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.900+0000 7f9fc9199700 1 -- 172.21.15.31:0/4136340474 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb400b7a0 con 0x7f9fc4068d80 2023-10-06T05:25:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.900+0000 7f9fc9199700 1 -- 172.21.15.31:0/4136340474 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400bb60 con 0x7f9fc4068d80 2023-10-06T05:25:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.900+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/4136340474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068d80 msgr2=0x7f9fc4159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.900+0000 7f9fcc3ff700 1 --2- 172.21.15.31:0/4136340474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068d80 0x7f9fc4159970 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009910 tx=0x7f9fb40091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.900+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/4136340474 shutdown_connections 2023-10-06T05:25:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.900+0000 7f9fcc3ff700 1 --2- 172.21.15.31:0/4136340474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4068d80 0x7f9fc4159970 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.900+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/4136340474 >> 172.21.15.31:0/4136340474 conn(0x7f9fc406b620 msgr2=0x7f9fc406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.901+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/4136340474 shutdown_connections 2023-10-06T05:25:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.901+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/4136340474 wait complete. 2023-10-06T05:25:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.902+0000 7f9fcc3ff700 1 Processor -- start 2023-10-06T05:25:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.902+0000 7f9fcc3ff700 1 -- start start 2023-10-06T05:25:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fcc3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fc4068d80 0x7f9fc414f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fcc3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fc40cc0d0 0x7f9fc406cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fcc3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc41595a0 0x7f9fc406d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fcc3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40b2a00 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fcc3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40b3540 con 0x7f9fc41595a0 2023-10-06T05:25:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fcc3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc414fa90 con 0x7f9fc4068d80 2023-10-06T05:25:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fc999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fc40cc0d0 0x7f9fc406cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fc999a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fc40cc0d0 0x7f9fc406cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47166/0 (socket says 172.21.15.31:47166) 2023-10-06T05:25:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fc999a700 1 -- 172.21.15.31:0/640616915 learned_addr learned my addr 172.21.15.31:0/640616915 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fca19b700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fc4068d80 0x7f9fc414f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.903+0000 7f9fca99c700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc41595a0 0x7f9fc406d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fc999a700 1 -- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fc4068d80 msgr2=0x7f9fc414f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fc999a700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fc4068d80 0x7f9fc414f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fc999a700 1 -- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc41595a0 msgr2=0x7f9fc406d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fc999a700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc41595a0 0x7f9fc406d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fc999a700 1 -- 172.21.15.31:0/640616915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb4009420 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fca99c700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc41595a0 0x7f9fc406d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fc999a700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fc40cc0d0 0x7f9fc406cda0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40093f0 tx=0x7f9fb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:30.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fba7fc700 1 -- 172.21.15.31:0/640616915 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb401f070 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fba7fc700 1 -- 172.21.15.31:0/640616915 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb4000bc0 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fba7fc700 1 -- 172.21.15.31:0/640616915 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400bb60 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.904+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/640616915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc40b2c10 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.905+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/640616915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc40b31d0 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.906+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/640616915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc404c380 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.906+0000 7f9fba7fc700 1 -- 172.21.15.31:0/640616915 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f9fb4023070 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.907+0000 7f9fba7fc700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9fa8067680 0x7f9fa8069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:30.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.907+0000 7f9fba7fc700 1 -- 172.21.15.31:0/640616915 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9fb4086c10 con 0x7f9fc40cc0d0 2023-10-06T05:25:30.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.908+0000 7f9fca19b700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9fa8067680 0x7f9fa8069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:30.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.908+0000 7f9fca19b700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9fa8067680 0x7f9fa8069b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00039e0 tx=0x7f9fc0007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:30.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:30.911+0000 7f9fba7fc700 1 -- 172.21.15.31:0/640616915 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fb4083fd0 con 0x7f9fc40cc0d0 2023-10-06T05:25:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.157+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/640616915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fc40b40c0 con 0x7f9fc40cc0d0 2023-10-06T05:25:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.157+0000 7f9fba7fc700 1 -- 172.21.15.31:0/640616915 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f9fb4052dc0 con 0x7f9fc40cc0d0 2023-10-06T05:25:31.160 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:31.160 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1489945066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1489945066},{"type":"v1","addr":"172.21.15.62:6834","nonce":1489945066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/660206899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":660206899},{"type":"v1","addr":"172.21.15.62:6835","nonce":660206899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8862,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1056123914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1056123914},{"type":"v1","addr":"172.21.15.31:6834","nonce":1056123914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:24.746699+0000","modified":"2023-10-06T05:25:28.266401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8865},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8865":{"gid":8865,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1535525741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1535525741},{"type":"v1","addr":"172.21.15.31:6835","nonce":1535525741}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:25:31.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9fa8067680 msgr2=0x7f9fa8069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9fa8067680 0x7f9fa8069b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00039e0 tx=0x7f9fc0007ec0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/640616915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fc40cc0d0 msgr2=0x7f9fc406cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fc40cc0d0 0x7f9fc406cda0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40093f0 tx=0x7f9fb4017000 comp rx=0 tx=0).stop 2023-10-06T05:25:31.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/640616915 shutdown_connections 2023-10-06T05:25:31.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9fa8067680 0x7f9fa8069b30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc41595a0 0x7f9fc406d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9fc4068d80 0x7f9fc414f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 --2- 172.21.15.31:0/640616915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fc40cc0d0 0x7f9fc406cda0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/640616915 >> 172.21.15.31:0/640616915 conn(0x7f9fc406b620 msgr2=0x7f9fc406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:31.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/640616915 shutdown_connections 2023-10-06T05:25:31.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.160+0000 7f9fcc3ff700 1 -- 172.21.15.31:0/640616915 wait complete. 2023-10-06T05:25:31.164 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 102 2023-10-06T05:25:31.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:25:24.746699+0000', 'modified': '2023-10-06T05:25:28.266401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8865}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8865': {'gid': 8865, 'name': 'a', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/1535525741', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6833', 'nonce': 1535525741}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 1535525741}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:25:31.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:25:31.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:25:31.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.781+0000 7fdcaf1ff700 1 Processor -- start 2023-10-06T05:25:31.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.781+0000 7fdcaf1ff700 1 -- start start 2023-10-06T05:25:31.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.782+0000 7fdcaf1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca81596d0 0x7fdca8159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.782+0000 7fdcaf1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca8067f70 con 0x7fdca80c16e0 2023-10-06T05:25:31.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.782+0000 7fdcaf1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca806e040 con 0x7fdca80c6f80 2023-10-06T05:25:31.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.782+0000 7fdcaf1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca806b040 con 0x7fdca81596d0 2023-10-06T05:25:31.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.782+0000 7fdcad79c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdca80c16e0 0x7fdca806e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59284/0 (socket says 172.21.15.31:59284) 2023-10-06T05:25:31.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.782+0000 7fdcad79c700 1 -- 172.21.15.31:0/453892960 learned_addr learned my addr 172.21.15.31:0/453892960 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca81596d0 0x7fdca8159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665700241 0 0) 0x7fdca8067f70 con 0x7fdca80c16e0 2023-10-06T05:25:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc98003570 con 0x7fdca80c16e0 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4284369714 0 0) 0x7fdc98003570 con 0x7fdca80c16e0 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca81596d0 msgr2=0x7fdca8159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca81596d0 0x7fdca8159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 >> v1:172.21.15.62:6789/0 conn(0x7fdca80c6f80 legacy=0x7fdca806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca80681c0 con 0x7fdca80c16e0 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fdca00017f0 con 0x7fdca80c16e0 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 >> v1:172.21.15.31:6789/0 conn(0x7fdca80c16e0 legacy=0x7fdca806e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc98003a90 0x7fdc98003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc98004810 0x7fdc98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca81596d0 0x7fdc98005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca806b040 con 0x7fdc98003a90 2023-10-06T05:25:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9800bbf0 con 0x7fdc98004810 2023-10-06T05:25:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.783+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9800bd80 con 0x7fdca81596d0 2023-10-06T05:25:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc98003a90 0x7fdc98003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdca7fff700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc98004810 0x7fdc98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdcad79c700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca81596d0 0x7fdc98005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdcacf9b700 1 -- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca81596d0 msgr2=0x7fdc98005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdca81596d0 0x7fdc98005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdcacf9b700 1 -- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc98004810 msgr2=0x7fdc98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc98004810 0x7fdc98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdcacf9b700 1 -- 172.21.15.31:0/453892960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98003570 con 0x7fdc98003a90 2023-10-06T05:25:31.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdca7fff700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc98004810 0x7fdc98004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:31.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc98003a90 0x7fdc98003ea0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009c30 tx=0x7fdc9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9561ec19ca21f24a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:31.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00e710 con 0x7fdc98003a90 2023-10-06T05:25:31.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c00e870 con 0x7fdc98003a90 2023-10-06T05:25:31.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.784+0000 7fdca77fe700 1 -- 172.21.15.31:0/453892960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00ec30 con 0x7fdc98003a90 2023-10-06T05:25:31.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.785+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/453892960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc98003a90 msgr2=0x7fdc98003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.785+0000 7fdcaf1ff700 1 --2- 172.21.15.31:0/453892960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc98003a90 0x7fdc98003ea0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009c30 tx=0x7fdc9c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.785+0000 7fdcad79c700 1 -- 172.21.15.31:0/453892960 reap_dead start 2023-10-06T05:25:31.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.785+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/453892960 shutdown_connections 2023-10-06T05:25:31.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.785+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/453892960 >> 172.21.15.31:0/453892960 conn(0x7fdca80c9e80 msgr2=0x7fdca806a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:31.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.785+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/453892960 shutdown_connections 2023-10-06T05:25:31.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.785+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/453892960 wait complete. 2023-10-06T05:25:31.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.787+0000 7fdcaf1ff700 1 Processor -- start 2023-10-06T05:25:31.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.787+0000 7fdcaf1ff700 1 -- start start 2023-10-06T05:25:31.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcaf1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc98003a90 0x7fdca8169b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcaf1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdca80b1f20 0x7fdca80b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcaf1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80c5280 0x7fdca80c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcaf1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80b0c50 con 0x7fdca80c5280 2023-10-06T05:25:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcaf1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80b1a90 con 0x7fdca80b1f20 2023-10-06T05:25:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcaf1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80b0e80 con 0x7fdc98003a90 2023-10-06T05:25:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcad79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80c5280 0x7fdca80c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcad79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80c5280 0x7fdca80c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47178/0 (socket says 172.21.15.31:47178) 2023-10-06T05:25:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcad79c700 1 -- 172.21.15.31:0/4281266517 learned_addr learned my addr 172.21.15.31:0/4281266517 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc98003a90 0x7fdca8169b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdca7fff700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdca80b1f20 0x7fdca80b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcad79c700 1 -- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc98003a90 msgr2=0x7fdca8169b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcad79c700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc98003a90 0x7fdca8169b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcad79c700 1 -- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdca80b1f20 msgr2=0x7fdca80b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcad79c700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdca80b1f20 0x7fdca80b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.788+0000 7fdcad79c700 1 -- 172.21.15.31:0/4281266517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca80b1160 con 0x7fdca80c5280 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.789+0000 7fdcad79c700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80c5280 0x7fdca80c5fb0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fdca00013d0 tx=0x7fdca0001870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.789+0000 7fdca4ff9700 1 -- 172.21.15.31:0/4281266517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000ebc0 con 0x7fdca80c5280 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.789+0000 7fdca4ff9700 1 -- 172.21.15.31:0/4281266517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca000ed20 con 0x7fdca80c5280 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.789+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/4281266517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca80b1440 con 0x7fdca80c5280 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.789+0000 7fdca4ff9700 1 -- 172.21.15.31:0/4281266517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0013670 con 0x7fdca80c5280 2023-10-06T05:25:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.789+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/4281266517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca80c56d0 con 0x7fdca80c5280 2023-10-06T05:25:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.791+0000 7fdca4ff9700 1 -- 172.21.15.31:0/4281266517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fdca00139e0 con 0x7fdca80c5280 2023-10-06T05:25:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.792+0000 7fdca4ff9700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c067680 0x7fdc8c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.792+0000 7fdca4ff9700 1 -- 172.21.15.31:0/4281266517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fdca0085c70 con 0x7fdca80c5280 2023-10-06T05:25:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.793+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c067680 0x7fdc8c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.793+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c067680 0x7fdc8c069b30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0091e0 tx=0x7fdc9c01a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.794+0000 7fdcbd9d7b80 1 Processor -- start 2023-10-06T05:25:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.794+0000 7fdcbd9d7b80 1 -- start start 2023-10-06T05:25:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.795+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de6d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.795+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 0x56379de75f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de76940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379dd40310 con 0x56379de6d450 2023-10-06T05:25:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379dd7d130 con 0x56379de764d0 2023-10-06T05:25:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56379dd7aaf0 con 0x56379de52f40 2023-10-06T05:25:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcacf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de6d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcacf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de6d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47186/0 (socket says 172.21.15.31:47186) 2023-10-06T05:25:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcacf9b700 1 -- 172.21.15.31:0/216583625 learned_addr learned my addr 172.21.15.31:0/216583625 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdca7fff700 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 0x56379de75f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcad79c700 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de76940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcacf9b700 1 -- 172.21.15.31:0/216583625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 msgr2=0x56379de75f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 0x56379de75f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcacf9b700 1 -- 172.21.15.31:0/216583625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 msgr2=0x56379de76940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de76940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcacf9b700 1 -- 172.21.15.31:0/216583625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56379db7c790 con 0x56379de6d450 2023-10-06T05:25:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdca7fff700 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 0x56379de75f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.796+0000 7fdcad79c700 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de76940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.797+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de6d820 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e4b0 tx=0x7fdc9c026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1744fe570849e1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.797+0000 7fdc91ffb700 1 -- 172.21.15.31:0/216583625 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c013540 con 0x56379de6d450 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.797+0000 7fdc91ffb700 1 -- 172.21.15.31:0/216583625 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c024040 con 0x56379de6d450 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.797+0000 7fdc91ffb700 1 -- 172.21.15.31:0/216583625 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0138b0 con 0x56379de6d450 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.797+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/216583625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 msgr2=0x56379de6d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.797+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de6d820 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e4b0 tx=0x7fdc9c026000 comp rx=0 tx=0).stop 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.797+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/216583625 shutdown_connections 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.798+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de76940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.798+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de6d820 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.798+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/216583625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 0x56379de75f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.798+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/216583625 >> 172.21.15.31:0/216583625 conn(0x56379de70880 msgr2=0x56379de70c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:31.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.798+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/216583625 shutdown_connections 2023-10-06T05:25:31.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.798+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/216583625 wait complete. 2023-10-06T05:25:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.798+0000 7fdcbd9d7b80 1 -- start start 2023-10-06T05:25:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.799+0000 7fdcbd9d7b80 1 Processor -- start 2023-10-06T05:25:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de4c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de6d450 0x56379de46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de4b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379dd768e0 con 0x56379de52f40 2023-10-06T05:25:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379dd22980 con 0x56379de764d0 2023-10-06T05:25:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56379dd22620 con 0x56379de6d450 2023-10-06T05:25:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcbd9d7b80 10 client.0 _ll_register_callbacks cb 0x56379db85a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcacf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de4c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcacf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de4c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47190/0 (socket says 172.21.15.31:47190) 2023-10-06T05:25:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcacf9b700 1 -- 172.21.15.31:0/4187261023 learned_addr learned my addr 172.21.15.31:0/4187261023 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdca7fff700 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de6d450 0x56379de46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcad79c700 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de4b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcacf9b700 1 -- 172.21.15.31:0/4187261023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de6d450 msgr2=0x56379de46f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de6d450 0x56379de46f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcacf9b700 1 -- 172.21.15.31:0/4187261023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 msgr2=0x56379de4b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de4b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.801+0000 7fdcacf9b700 1 -- 172.21.15.31:0/4187261023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc98003570 con 0x56379de52f40 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.802+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de4c250 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c038040 tx=0x7fdc9c02f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63dcd30c5dd5d425 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.802+0000 7fdc86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.802+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4187261023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c005a00 con 0x56379de52f40 2023-10-06T05:25:31.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.802+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4187261023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fdc9c005b60 con 0x56379de52f40 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.802+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4187261023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56379de1ad60 con 0x56379de52f40 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.802+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4187261023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc9c006040 con 0x56379de52f40 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.802+0000 7fdcbd9d7b80 10 client.9009 fetch_fsmap learned FSMap version 102 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.802+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4187261023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x56379dc690a0 con 0x56379de52f40 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.802+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4187261023 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56379de1ad60 con 0x56379de52f40 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.802+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4187261023 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 102) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdc9c0061a0 con 0x56379de52f40 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.803+0000 7fdcbd9d7b80 10 client.9009 fetch_fsmap finished waiting for FSMap version 102 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.803+0000 7fdcbd9d7b80 10 client.9009 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.803+0000 7fdcbd9d7b80 20 client.9009 populate_metadata read hostname 'smithi031' 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.803+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.31:6833/1535525741,v1:172.21.15.31:6835/1535525741] conn(0x56379de4a090 0x56379de4a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.803+0000 7fdcbd9d7b80 4 client.9009 mds_command: new command op to 8865 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.803+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4187261023 --> [v2:172.21.15.31:6833/1535525741,v1:172.21.15.31:6835/1535525741] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56379ddf7b30 con 0x56379de4a090 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.803+0000 7fdca7fff700 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.31:6833/1535525741,v1:172.21.15.31:6835/1535525741] conn(0x56379de4a090 0x56379de4a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.803+0000 7fdca7fff700 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.31:6833/1535525741,v1:172.21.15.31:6835/1535525741] conn(0x56379de4a090 0x56379de4a460 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b139c5254a97f052 server_cookie=db339b314558a225 in_seq=0 out_seq=0 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.803+0000 7fdc86ffd700 10 client.9009 ms_handle_connect on v2:172.21.15.31:6833/1535525741 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4187261023 <== mds.0 v2:172.21.15.31:6833/1535525741 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56379ddf7b30 con 0x56379de4a090 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdc86ffd700 10 client.9009 handle_command_reply: tid=0 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdcbd9d7b80 1 client.9009 shutdown 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4187261023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 msgr2=0x56379de4c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de4c250 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c038040 tx=0x7fdc9c02f050 comp rx=0 tx=0).stop 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4187261023 shutdown_connections 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de4b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.31:6833/1535525741,v1:172.21.15.31:6835/1535525741] conn(0x56379de4a090 0x56379de4a460 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de4c250 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4187261023 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de6d450 0x56379de46f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdc86ffd700 0 client.9009 ms_handle_reset on v2:172.21.15.31:6833/1535525741 2023-10-06T05:25:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.804+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4187261023 >> 172.21.15.31:0/4187261023 conn(0x56379de70880 msgr2=0x56379de22c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.805+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4187261023 shutdown_connections 2023-10-06T05:25:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.805+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4187261023 wait complete. 2023-10-06T05:25:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.805+0000 7fdcbd9d7b80 20 client.9009 trim_cache size 0 max 16384 2023-10-06T05:25:31.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.822+0000 7fdcbd9d7b80 1 Processor -- start 2023-10-06T05:25:31.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.822+0000 7fdcbd9d7b80 1 -- start start 2023-10-06T05:25:31.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.823+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de1c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.823+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de6d450 0x56379de1cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.823+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de764d0 0x56379de27440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.823+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379dc0e1e0 con 0x56379de52f40 2023-10-06T05:25:31.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.823+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379de24470 con 0x56379de6d450 2023-10-06T05:25:31.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.823+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56379de24070 con 0x56379de764d0 2023-10-06T05:25:31.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.823+0000 7fdcacf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de1c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.823+0000 7fdcacf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de1c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47206/0 (socket says 172.21.15.31:47206) 2023-10-06T05:25:31.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.823+0000 7fdcacf9b700 1 -- 172.21.15.31:0/2859064319 learned_addr learned my addr 172.21.15.31:0/2859064319 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:31.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdcad79c700 1 --2- 172.21.15.31:0/2859064319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de764d0 0x56379de27440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdca7fff700 1 --2- 172.21.15.31:0/2859064319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de6d450 0x56379de1cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdcacf9b700 1 -- 172.21.15.31:0/2859064319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de764d0 msgr2=0x56379de27440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/2859064319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de764d0 0x56379de27440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdcacf9b700 1 -- 172.21.15.31:0/2859064319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de6d450 msgr2=0x56379de1cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/2859064319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de6d450 0x56379de1cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdcacf9b700 1 -- 172.21.15.31:0/2859064319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56379de6e700 con 0x56379de52f40 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdca7fff700 1 --2- 172.21.15.31:0/2859064319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de6d450 0x56379de1cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/2859064319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de1c870 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00ed40 tx=0x7fdc9c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e73e3883aef1042 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdc927fc700 1 -- 172.21.15.31:0/2859064319 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c006660 con 0x56379de52f40 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdc927fc700 1 -- 172.21.15.31:0/2859064319 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c013470 con 0x56379de52f40 2023-10-06T05:25:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.824+0000 7fdc927fc700 1 -- 172.21.15.31:0/2859064319 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c005f30 con 0x56379de52f40 2023-10-06T05:25:31.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.825+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/2859064319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 msgr2=0x56379de1c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.825+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/2859064319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de1c870 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00ed40 tx=0x7fdc9c02c070 comp rx=0 tx=0).stop 2023-10-06T05:25:31.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.825+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/2859064319 shutdown_connections 2023-10-06T05:25:31.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.825+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/2859064319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de764d0 0x56379de27440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.825+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/2859064319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de52f40 0x56379de1c870 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.825+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/2859064319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de6d450 0x56379de1cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.825+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/2859064319 >> 172.21.15.31:0/2859064319 conn(0x56379de70880 msgr2=0x56379de45f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:31.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.825+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/2859064319 shutdown_connections 2023-10-06T05:25:31.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.825+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/2859064319 wait complete. 2023-10-06T05:25:31.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.825+0000 7fdcbd9d7b80 1 -- start start 2023-10-06T05:25:31.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.826+0000 7fdcbd9d7b80 1 Processor -- start 2023-10-06T05:25:31.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.827+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 0x56379de753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.827+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de55760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.827+0000 7fdcbd9d7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de55ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379ded1f80 con 0x56379de6d450 2023-10-06T05:25:31.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379de4b730 con 0x56379de764d0 2023-10-06T05:25:31.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdcbd9d7b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x56379de46830 con 0x56379de52f40 2023-10-06T05:25:31.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdcbd9d7b80 10 client.0 _ll_register_callbacks cb 0x56379db85a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:25:31.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdca7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de55760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdcad79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de55ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdca7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de55760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47222/0 (socket says 172.21.15.31:47222) 2023-10-06T05:25:31.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdca7fff700 1 -- 172.21.15.31:0/4105767494 learned_addr learned my addr 172.21.15.31:0/4105767494 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:31.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 0x56379de753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdca7fff700 1 -- 172.21.15.31:0/4105767494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 msgr2=0x56379de753b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdca7fff700 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 0x56379de753b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdca7fff700 1 -- 172.21.15.31:0/4105767494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 msgr2=0x56379de55ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdca7fff700 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de55ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdca7fff700 1 -- 172.21.15.31:0/4105767494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56379dc690a0 con 0x56379de6d450 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdca7fff700 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de55760 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fdc94000c00 tx=0x7fdc9401d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ce802683488ded server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.828+0000 7fdc86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4105767494 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94003aa0 con 0x56379de6d450 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4105767494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc9c024040 con 0x56379de6d450 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4105767494 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fdc94017de0 con 0x56379de6d450 2023-10-06T05:25:31.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4105767494 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc94020650 con 0x56379de6d450 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdcbd9d7b80 10 client.9015 fetch_fsmap learned FSMap version 102 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4105767494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x56379dd453c0 con 0x56379de6d450 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4105767494 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c024040 con 0x56379de6d450 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4105767494 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 102) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdc94027060 con 0x56379de6d450 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdcbd9d7b80 10 client.9015 fetch_fsmap finished waiting for FSMap version 102 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdcbd9d7b80 10 client.9015 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdcbd9d7b80 20 client.9015 populate_metadata read hostname 'smithi031' 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.31:6833/1535525741,v1:172.21.15.31:6835/1535525741] conn(0x56379ded1130 0x56379ded1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdcbd9d7b80 4 client.9015 mds_command: new command op to 8865 tid=0[{"prefix": "status"}] 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.829+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4105767494 --> [v2:172.21.15.31:6833/1535525741,v1:172.21.15.31:6835/1535525741] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56379dd198d0 con 0x56379ded1130 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.830+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.31:6833/1535525741,v1:172.21.15.31:6835/1535525741] conn(0x56379ded1130 0x56379ded1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:31.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.830+0000 7fdcacf9b700 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.31:6833/1535525741,v1:172.21.15.31:6835/1535525741] conn(0x56379ded1130 0x56379ded1500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99118ebebb2168c4 server_cookie=449def6327282899 in_seq=0 out_seq=0 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.830+0000 7fdc86ffd700 10 client.9015 ms_handle_connect on v2:172.21.15.31:6833/1535525741 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.830+0000 7fdc86ffd700 1 -- 172.21.15.31:0/4105767494 <== mds.0 v2:172.21.15.31:6833/1535525741 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x56379dd198d0 con 0x56379ded1130 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.830+0000 7fdc86ffd700 10 client.9015 handle_command_reply: tid=0 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.830+0000 7fdcbd9d7b80 1 client.9015 shutdown 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4105767494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 msgr2=0x56379de55760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de55760 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fdc94000c00 tx=0x7fdc9401d250 comp rx=0 tx=0).stop 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4105767494 shutdown_connections 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56379de764d0 0x56379de55ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56379de52f40 0x56379de753b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.31:6833/1535525741,v1:172.21.15.31:6835/1535525741] conn(0x56379ded1130 0x56379ded1500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 1 --2- 172.21.15.31:0/4105767494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56379de6d450 0x56379de55760 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4105767494 >> 172.21.15.31:0/4105767494 conn(0x56379de70880 msgr2=0x56379de45f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdc86ffd700 0 client.9015 ms_handle_reset on v2:172.21.15.31:6833/1535525741 2023-10-06T05:25:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4105767494 shutdown_connections 2023-10-06T05:25:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 1 -- 172.21.15.31:0/4105767494 wait complete. 2023-10-06T05:25:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.831+0000 7fdcbd9d7b80 20 client.9015 trim_cache size 0 max 16384 2023-10-06T05:25:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:25:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:25:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:25:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 8865, 2023-10-06T05:25:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:25:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:25:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:25:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5877509889999999, 2023-10-06T05:25:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 102, 2023-10-06T05:25:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 150, 2023-10-06T05:25:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 150, 2023-10-06T05:25:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.252423723 2023-10-06T05:25:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:25:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c067680 msgr2=0x7fdc8c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c067680 0x7fdc8c069b30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0091e0 tx=0x7fdc9c01a000 comp rx=0 tx=0).stop 2023-10-06T05:25:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/4281266517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80c5280 msgr2=0x7fdca80c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80c5280 0x7fdca80c5fb0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fdca00013d0 tx=0x7fdca0001870 comp rx=0 tx=0).stop 2023-10-06T05:25:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/4281266517 shutdown_connections 2023-10-06T05:25:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdc8c067680 0x7fdc8c069b30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80c5280 0x7fdca80c5fb0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc98003a90 0x7fdca8169b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 --2- 172.21.15.31:0/4281266517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdca80b1f20 0x7fdca80b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/4281266517 >> 172.21.15.31:0/4281266517 conn(0x7fdca80c9e80 msgr2=0x7fdca8067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/4281266517 shutdown_connections 2023-10-06T05:25:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:31.832+0000 7fdcaf1ff700 1 -- 172.21.15.31:0/4281266517 wait complete. 2023-10-06T05:25:31.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:25:32.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.043+0000 7fb210dff700 1 Processor -- start 2023-10-06T05:25:32.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.043+0000 7fb210dff700 1 -- start start 2023-10-06T05:25:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.044+0000 7fb210dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb20c1596d0 0x7fb20c159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.044+0000 7fb210dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c6f80 0x7fb20c06a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.044+0000 7fb210dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20c067f70 con 0x7fb20c0c16e0 2023-10-06T05:25:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.044+0000 7fb210dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20c06e040 con 0x7fb20c1596d0 2023-10-06T05:25:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.044+0000 7fb210dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0681c0 con 0x7fb20c0c6f80 2023-10-06T05:25:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.045+0000 7fb20ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb20c0c16e0 0x7fb20c15d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59290/0 (socket says 172.21.15.31:59290) 2023-10-06T05:25:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.045+0000 7fb20ad9d700 1 -- 172.21.15.31:0/4009573534 learned_addr learned my addr 172.21.15.31:0/4009573534 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.045+0000 7fb209d9b700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c6f80 0x7fb20c06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.045+0000 7fb20a59c700 1 --2- 172.21.15.31:0/4009573534 >> v2:172.21.15.62:3300/0 conn(0x7fb20c1596d0 0x7fb20c159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.045+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517784873 0 0) 0x7fb20c067f70 con 0x7fb20c0c16e0 2023-10-06T05:25:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.045+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f4003570 con 0x7fb20c0c16e0 2023-10-06T05:25:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3933146942 0 0) 0x7fb1f4003570 con 0x7fb20c0c16e0 2023-10-06T05:25:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c6f80 msgr2=0x7fb20c06a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c6f80 0x7fb20c06a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 >> v2:172.21.15.62:3300/0 conn(0x7fb20c1596d0 msgr2=0x7fb20c159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:32.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 --2- 172.21.15.31:0/4009573534 >> v2:172.21.15.62:3300/0 conn(0x7fb20c1596d0 0x7fb20c159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0684a0 con 0x7fb20c0c16e0 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fb2040017f0 con 0x7fb20c0c16e0 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 >> v1:172.21.15.31:6789/0 conn(0x7fb20c0c16e0 legacy=0x7fb20c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c6f80 0x7fb1f4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f4003fe0 0x7fb1f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1f4004d50 0x7fb1f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c06e040 con 0x7fb1f4004d50 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0681c0 con 0x7fb1f4003fe0 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f400ba90 con 0x7fb20c0c6f80 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20ad9d700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1f4004d50 0x7fb1f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb209d9b700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f4003fe0 0x7fb1f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:32.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.046+0000 7fb20a59c700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c6f80 0x7fb1f4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.047+0000 7fb20ad9d700 1 -- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c6f80 msgr2=0x7fb1f4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.047+0000 7fb20ad9d700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c6f80 0x7fb1f4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.047+0000 7fb20ad9d700 1 -- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f4003fe0 msgr2=0x7fb1f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.047+0000 7fb20ad9d700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f4003fe0 0x7fb1f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.047+0000 7fb20ad9d700 1 -- 172.21.15.31:0/4009573534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f4003570 con 0x7fb1f4004d50 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.047+0000 7fb20ad9d700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1f4004d50 0x7fb1f40051c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fb20400a970 tx=0x7fb204002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b490e9fa5b5d7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.047+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040070d0 con 0x7fb1f4004d50 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.047+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb204007270 con 0x7fb1f4004d50 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.047+0000 7fb20959a700 1 -- 172.21.15.31:0/4009573534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204007630 con 0x7fb1f4004d50 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.048+0000 7fb210dff700 1 -- 172.21.15.31:0/4009573534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1f4004d50 msgr2=0x7fb1f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.048+0000 7fb210dff700 1 --2- 172.21.15.31:0/4009573534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1f4004d50 0x7fb1f40051c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fb20400a970 tx=0x7fb204002210 comp rx=0 tx=0).stop 2023-10-06T05:25:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.048+0000 7fb20ad9d700 1 -- 172.21.15.31:0/4009573534 reap_dead start 2023-10-06T05:25:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.048+0000 7fb210dff700 1 -- 172.21.15.31:0/4009573534 shutdown_connections 2023-10-06T05:25:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.048+0000 7fb210dff700 1 -- 172.21.15.31:0/4009573534 >> 172.21.15.31:0/4009573534 conn(0x7fb20c0c9e80 msgr2=0x7fb20c06a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.048+0000 7fb210dff700 1 -- 172.21.15.31:0/4009573534 shutdown_connections 2023-10-06T05:25:32.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.048+0000 7fb210dff700 1 -- 172.21.15.31:0/4009573534 wait complete. 2023-10-06T05:25:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.049+0000 7fb210dff700 1 Processor -- start 2023-10-06T05:25:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.050+0000 7fb210dff700 1 -- start start 2023-10-06T05:25:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.050+0000 7fb210dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f4004d50 0x7fb20c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb210dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c8570 0x7fb20c1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb210dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb20c0c89e0 0x7fb20c153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb210dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0c5000 con 0x7fb20c0c89e0 2023-10-06T05:25:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb210dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0c5e20 con 0x7fb1f4004d50 2023-10-06T05:25:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb210dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0c5210 con 0x7fb20c0c8570 2023-10-06T05:25:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb20ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb20c0c89e0 0x7fb20c153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb20ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb20c0c89e0 0x7fb20c153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47228/0 (socket says 172.21.15.31:47228) 2023-10-06T05:25:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb20ad9d700 1 -- 172.21.15.31:0/1575202361 learned_addr learned my addr 172.21.15.31:0/1575202361 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb209d9b700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c8570 0x7fb20c1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb20a59c700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f4004d50 0x7fb20c0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb20ad9d700 1 -- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c8570 msgr2=0x7fb20c1530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb20ad9d700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c8570 0x7fb20c1530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb20ad9d700 1 -- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f4004d50 msgr2=0x7fb20c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb20ad9d700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f4004d50 0x7fb20c0b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.051+0000 7fb20ad9d700 1 -- 172.21.15.31:0/1575202361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0c54f0 con 0x7fb20c0c89e0 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.052+0000 7fb20ad9d700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb20c0c89e0 0x7fb20c153610 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb2040077e0 tx=0x7fb204015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.052+0000 7fb1fa7fc700 1 -- 172.21.15.31:0/1575202361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204023070 con 0x7fb20c0c89e0 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.052+0000 7fb1fa7fc700 1 -- 172.21.15.31:0/1575202361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2040017b0 con 0x7fb20c0c89e0 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.052+0000 7fb210dff700 1 -- 172.21.15.31:0/1575202361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c0c57d0 con 0x7fb20c0c89e0 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.052+0000 7fb1fa7fc700 1 -- 172.21.15.31:0/1575202361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20401e8b0 con 0x7fb20c0c89e0 2023-10-06T05:25:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.052+0000 7fb210dff700 1 -- 172.21.15.31:0/1575202361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c0b1200 con 0x7fb20c0c89e0 2023-10-06T05:25:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.053+0000 7fb210dff700 1 -- 172.21.15.31:0/1575202361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20c04c380 con 0x7fb20c0c89e0 2023-10-06T05:25:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.054+0000 7fb1fa7fc700 1 -- 172.21.15.31:0/1575202361 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fb20401ec20 con 0x7fb20c0c89e0 2023-10-06T05:25:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.055+0000 7fb1fa7fc700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1ec067680 0x7fb1ec069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.055+0000 7fb1fa7fc700 1 -- 172.21.15.31:0/1575202361 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fb204086df0 con 0x7fb20c0c89e0 2023-10-06T05:25:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.056+0000 7fb20a59c700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1ec067680 0x7fb1ec069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.056+0000 7fb20a59c700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1ec067680 0x7fb1ec069b30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006f00 tx=0x7fb1fc0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.059+0000 7fb1fa7fc700 1 -- 172.21.15.31:0/1575202361 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2040845c0 con 0x7fb20c0c89e0 2023-10-06T05:25:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.309+0000 7fb210dff700 1 -- 172.21.15.31:0/1575202361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb20c06e290 con 0x7fb20c0c89e0 2023-10-06T05:25:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.309+0000 7fb1fa7fc700 1 -- 172.21.15.31:0/1575202361 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb204016060 con 0x7fb20c0c89e0 2023-10-06T05:25:32.311 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:25:32.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 -- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1ec067680 msgr2=0x7fb1ec069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:32.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1ec067680 0x7fb1ec069b30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006f00 tx=0x7fb1fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 -- 172.21.15.31:0/1575202361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb20c0c89e0 msgr2=0x7fb20c153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:32.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb20c0c89e0 0x7fb20c153610 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb2040077e0 tx=0x7fb204015000 comp rx=0 tx=0).stop 2023-10-06T05:25:32.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 -- 172.21.15.31:0/1575202361 shutdown_connections 2023-10-06T05:25:32.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb1ec067680 0x7fb1ec069b30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb20c0c89e0 0x7fb20c153610 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f4004d50 0x7fb20c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 --2- 172.21.15.31:0/1575202361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb20c0c8570 0x7fb20c1530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 -- 172.21.15.31:0/1575202361 >> 172.21.15.31:0/1575202361 conn(0x7fb20c0c9e80 msgr2=0x7fb20c0cafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 -- 172.21.15.31:0/1575202361 shutdown_connections 2023-10-06T05:25:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:32.312+0000 7fb210dff700 1 -- 172.21.15.31:0/1575202361 wait complete. 2023-10-06T05:25:32.331 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:25:32.331 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:25:32.331 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:25:32.331 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:25:32.331 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:32.331 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:25:32.332 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:25:32.332 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:25:32.332 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:32.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:32.332 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:25:32.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:32.340 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:25:32.381 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:32.381 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:25:32.381 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:25:32.382 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:25:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:25:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:25:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41782sec preferred_lft 41782sec 2023-10-06T05:25:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:25:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:32.382 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:25:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:32.384 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:25:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:25:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:32.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:32.385 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:25:32.494 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:25:32.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:32.497 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:25:32.546 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:25:32.547 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:25:32.547 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:25:32.547 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:25:32.547 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:25:32.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:32.547 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:25:32.547 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:25:32.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:25:32.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:25:32.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:25:32.548 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:25:32.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:32.720 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:25:32.767 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:25:32.767 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:25:32.768 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:32.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:32.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:25:32.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:32.839 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:32.886 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:25:32.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:32.887 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:32.972 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:25:32.972 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:25:32.973 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:25:33.114 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:25:33.114 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:25:33.139 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:25:33.140 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:25:33.140 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:25:33.140 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:25:33.140 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:25:33.143 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:25:33.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:33.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:33.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:25:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.414+0000 7f93b01ff700 1 Processor -- start 2023-10-06T05:25:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.414+0000 7f93b01ff700 1 -- start start 2023-10-06T05:25:33.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.415+0000 7f93b01ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93a81596e0 0x7f93a815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:33.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.416+0000 7f93b01ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a80ccb30 con 0x7f93a8067c70 2023-10-06T05:25:33.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.416+0000 7f93b01ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a8159ab0 con 0x7f93a8159310 2023-10-06T05:25:33.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.416+0000 7f93b01ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a80c8990 con 0x7f93a81596e0 2023-10-06T05:25:33.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.416+0000 7f93ad79a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f93a8067c70 0x7f93a806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59298/0 (socket says 172.21.15.31:59298) 2023-10-06T05:25:33.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.416+0000 7f93ad79a700 1 -- 172.21.15.31:0/2909043816 learned_addr learned my addr 172.21.15.31:0/2909043816 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:33.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.416+0000 7f93ae79c700 1 --2- 172.21.15.31:0/2909043816 >> v2:172.21.15.31:3300/0 conn(0x7f93a81596e0 0x7f93a815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.416+0000 7f93acf99700 1 -- 172.21.15.31:0/2909043816 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704769374 0 0) 0x7f93a80ccb30 con 0x7f93a8067c70 2023-10-06T05:25:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.416+0000 7f93ae79c700 1 -- 172.21.15.31:0/2909043816 >> v1:172.21.15.62:6789/0 conn(0x7f93a8159310 legacy=0x7f93a8067560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.416+0000 7f93ae79c700 1 -- 172.21.15.31:0/2909043816 >> v1:172.21.15.31:6789/0 conn(0x7f93a8067c70 legacy=0x7f93a806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.416+0000 7f93ae79c700 1 -- 172.21.15.31:0/2909043816 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a80ccfc0 con 0x7f93a81596e0 2023-10-06T05:25:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.417+0000 7f93ae79c700 1 --2- 172.21.15.31:0/2909043816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a81596e0 0x7f93a815dd50 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009e60 tx=0x7f93a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a656f13912949d4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.417+0000 7f93acf99700 1 -- 172.21.15.31:0/2909043816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a000e070 con 0x7f93a81596e0 2023-10-06T05:25:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.417+0000 7f93acf99700 1 -- 172.21.15.31:0/2909043816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a000bc30 con 0x7f93a81596e0 2023-10-06T05:25:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.417+0000 7f93acf99700 1 -- 172.21.15.31:0/2909043816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0012430 con 0x7f93a81596e0 2023-10-06T05:25:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.417+0000 7f93b01ff700 1 -- 172.21.15.31:0/2909043816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a81596e0 msgr2=0x7f93a815dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.417+0000 7f93b01ff700 1 --2- 172.21.15.31:0/2909043816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a81596e0 0x7f93a815dd50 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009e60 tx=0x7f93a0009780 comp rx=0 tx=0).stop 2023-10-06T05:25:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.417+0000 7f93b01ff700 1 -- 172.21.15.31:0/2909043816 shutdown_connections 2023-10-06T05:25:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.417+0000 7f93b01ff700 1 --2- 172.21.15.31:0/2909043816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a81596e0 0x7f93a815dd50 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.417+0000 7f93b01ff700 1 -- 172.21.15.31:0/2909043816 >> 172.21.15.31:0/2909043816 conn(0x7f93a80cdc50 msgr2=0x7f93a80c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.417+0000 7f93b01ff700 1 -- 172.21.15.31:0/2909043816 shutdown_connections 2023-10-06T05:25:33.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.418+0000 7f93b01ff700 1 -- 172.21.15.31:0/2909043816 wait complete. 2023-10-06T05:25:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.419+0000 7f93b01ff700 1 Processor -- start 2023-10-06T05:25:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.419+0000 7f93b01ff700 1 -- start start 2023-10-06T05:25:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.420+0000 7f93b01ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93a8067c70 0x7f93a80b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.420+0000 7f93b01ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a8159310 0x7f93a815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.420+0000 7f93b01ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a81596e0 0x7f93a815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.420+0000 7f93b01ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a815ac10 con 0x7f93a8159310 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.420+0000 7f93b01ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a815c630 con 0x7f93a81596e0 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.420+0000 7f93b01ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a815adf0 con 0x7f93a8067c70 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.420+0000 7f93ad79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a8159310 0x7f93a815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.420+0000 7f93ad79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a8159310 0x7f93a815bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47250/0 (socket says 172.21.15.31:47250) 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.420+0000 7f93ad79a700 1 -- 172.21.15.31:0/3390687059 learned_addr learned my addr 172.21.15.31:0/3390687059 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93ae79c700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a81596e0 0x7f93a815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93adf9b700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93a8067c70 0x7f93a80b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93ad79a700 1 -- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93a8067c70 msgr2=0x7f93a80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93ad79a700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93a8067c70 0x7f93a80b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93ad79a700 1 -- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a81596e0 msgr2=0x7f93a815c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93ad79a700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a81596e0 0x7f93a815c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93ad79a700 1 -- 172.21.15.31:0/3390687059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0009a20 con 0x7f93a8159310 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93ae79c700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a81596e0 0x7f93a815c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93adf9b700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93a8067c70 0x7f93a80b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93ad79a700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a8159310 0x7f93a815bbb0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f939800c810 tx=0x7f9398007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f939dffb700 1 -- 172.21.15.31:0/3390687059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398005c00 con 0x7f93a8159310 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f939dffb700 1 -- 172.21.15.31:0/3390687059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9398005d60 con 0x7f93a8159310 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f939dffb700 1 -- 172.21.15.31:0/3390687059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398007320 con 0x7f93a8159310 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.421+0000 7f93b01ff700 1 -- 172.21.15.31:0/3390687059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a815b0d0 con 0x7f93a8159310 2023-10-06T05:25:33.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.422+0000 7f93b01ff700 1 -- 172.21.15.31:0/3390687059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a815b710 con 0x7f93a8159310 2023-10-06T05:25:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.423+0000 7f93b01ff700 1 -- 172.21.15.31:0/3390687059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a815b8a0 con 0x7f93a8159310 2023-10-06T05:25:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.423+0000 7f939dffb700 1 -- 172.21.15.31:0/3390687059 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f939801b400 con 0x7f93a8159310 2023-10-06T05:25:33.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.424+0000 7f939dffb700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f938c067680 0x7f938c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:33.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.424+0000 7f939dffb700 1 -- 172.21.15.31:0/3390687059 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9398085360 con 0x7f93a8159310 2023-10-06T05:25:33.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.425+0000 7f93adf9b700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f938c067680 0x7f938c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:33.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.426+0000 7f93adf9b700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f938c067680 0x7f938c069b30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f93a815c7d0 tx=0x7f93a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:33.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.429+0000 7f939dffb700 1 -- 172.21.15.31:0/3390687059 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9398082720 con 0x7f93a8159310 2023-10-06T05:25:33.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.680+0000 7f93b01ff700 1 -- 172.21.15.31:0/3390687059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f93a804c380 con 0x7f93a8159310 2023-10-06T05:25:33.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.681+0000 7f939dffb700 1 -- 172.21.15.31:0/3390687059 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9398051590 con 0x7f93a8159310 2023-10-06T05:25:33.683 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:25:33.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.683+0000 7f93b01ff700 1 -- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f938c067680 msgr2=0x7f938c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.683+0000 7f93b01ff700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f938c067680 0x7f938c069b30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f93a815c7d0 tx=0x7f93a4009040 comp rx=0 tx=0).stop 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.683+0000 7f93b01ff700 1 -- 172.21.15.31:0/3390687059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a8159310 msgr2=0x7f93a815bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.683+0000 7f93b01ff700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a8159310 0x7f93a815bbb0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f939800c810 tx=0x7f9398007db0 comp rx=0 tx=0).stop 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.683+0000 7f93b01ff700 1 -- 172.21.15.31:0/3390687059 shutdown_connections 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.683+0000 7f93b01ff700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f938c067680 0x7f938c069b30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.683+0000 7f93b01ff700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93a81596e0 0x7f93a815c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.683+0000 7f93b01ff700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93a8067c70 0x7f93a80b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.683+0000 7f93b01ff700 1 --2- 172.21.15.31:0/3390687059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a8159310 0x7f93a815bbb0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.683+0000 7f93b01ff700 1 -- 172.21.15.31:0/3390687059 >> 172.21.15.31:0/3390687059 conn(0x7f93a80cdc50 msgr2=0x7f93a80c1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.684+0000 7f93b01ff700 1 -- 172.21.15.31:0/3390687059 shutdown_connections 2023-10-06T05:25:33.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:33.684+0000 7f93b01ff700 1 -- 172.21.15.31:0/3390687059 wait complete. 2023-10-06T05:25:33.703 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:25:33.703 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:25:33.703 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:25:33.703 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:25:33.704 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:33.704 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:25:33.704 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:25:33.704 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:25:33.704 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:25:33.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:33.704 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:25:33.758 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:25:33.758 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:25:33.759 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:25:33.759 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:33.759 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:25:33.759 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:33.759 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:33.760 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:33.760 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:25:33.760 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:33.760 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:33.760 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:25:33.761 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:25:33.761 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:33.761 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:25:33.761 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:33.761 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:25:33.762 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41781sec preferred_lft 41781sec 2023-10-06T05:25:33.762 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:25:33.762 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:33.762 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:25:33.762 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:25:33.762 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:25:33.763 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:33.763 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:25:33.763 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:33.763 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:33.763 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:25:33.764 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:25:33.764 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:33.764 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:25:33.764 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:25:33.764 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:25:33.764 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:25:33.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:33.765 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:25:33.837 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:25:33.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:33.841 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:25:33.890 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:25:33.891 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:25:33.891 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:25:33.891 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:25:33.891 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:25:33.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:33.893 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:25:33.893 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:25:33.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:25:33.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:25:33.894 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:25:33.894 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:25:34.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:34.065 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:25:34.112 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:25:34.113 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:25:34.113 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:25:34.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:34.113 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:25:34.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:34.185 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:34.231 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:25:34.232 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:25:34.232 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:34.319 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:25:34.319 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:25:34.319 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:25:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:25:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:25:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:25:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:25:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:25:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:25:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:25:34.449 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:25:34.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:34.450 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:34.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:25:34.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.728+0000 7fbbfe5ff700 1 Processor -- start 2023-10-06T05:25:34.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.728+0000 7fbbfe5ff700 1 -- start start 2023-10-06T05:25:34.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.729+0000 7fbbfe5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142c20 0x7fbbf80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:34.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbfe5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbbf815a990 0x7fbbf80cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:34.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbfe5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf815d210 con 0x7fbbf8142ff0 2023-10-06T05:25:34.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbfe5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80cc810 con 0x7fbbf815a990 2023-10-06T05:25:34.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbfe5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf815d4b0 con 0x7fbbf8142c20 2023-10-06T05:25:34.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbf77fe700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbbf815a990 0x7fbbf80cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:34.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbf7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142c20 0x7fbbf80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:34.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbf7fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142c20 0x7fbbf80cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:48226/0 (socket says 172.21.15.31:48226) 2023-10-06T05:25:34.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbf7fff700 1 -- 172.21.15.31:0/1820857345 learned_addr learned my addr 172.21.15.31:0/1820857345 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:34.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbf7fff700 1 -- 172.21.15.31:0/1820857345 >> v2:172.21.15.62:3300/0 conn(0x7fbbf815a990 msgr2=0x7fbbf80cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:34.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbf7fff700 1 --2- 172.21.15.31:0/1820857345 >> v2:172.21.15.62:3300/0 conn(0x7fbbf815a990 0x7fbbf80cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:34.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbf7fff700 1 -- 172.21.15.31:0/1820857345 >> v1:172.21.15.62:6789/0 conn(0x7fbbf8142ff0 legacy=0x7fbbf806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:34.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.730+0000 7fbbf7fff700 1 -- 172.21.15.31:0/1820857345 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf815d790 con 0x7fbbf8142c20 2023-10-06T05:25:34.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.731+0000 7fbbf7fff700 1 --2- 172.21.15.31:0/1820857345 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142c20 0x7fbbf80cbea0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80099b0 tx=0x7fbbe8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29eba9bf35fc7c82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:34.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.731+0000 7fbbf6ffd700 1 -- 172.21.15.31:0/1820857345 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8009ea0 con 0x7fbbf8142c20 2023-10-06T05:25:34.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.731+0000 7fbbf6ffd700 1 -- 172.21.15.31:0/1820857345 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe800b860 con 0x7fbbf8142c20 2023-10-06T05:25:34.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.731+0000 7fbbf6ffd700 1 -- 172.21.15.31:0/1820857345 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800bc00 con 0x7fbbf8142c20 2023-10-06T05:25:34.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.732+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/1820857345 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142c20 msgr2=0x7fbbf80cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:34.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.732+0000 7fbbfe5ff700 1 --2- 172.21.15.31:0/1820857345 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142c20 0x7fbbf80cbea0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80099b0 tx=0x7fbbe8009250 comp rx=0 tx=0).stop 2023-10-06T05:25:34.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.732+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/1820857345 shutdown_connections 2023-10-06T05:25:34.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.732+0000 7fbbfe5ff700 1 --2- 172.21.15.31:0/1820857345 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142c20 0x7fbbf80cbea0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:34.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.732+0000 7fbbfe5ff700 1 --2- 172.21.15.31:0/1820857345 >> v2:172.21.15.62:3300/0 conn(0x7fbbf815a990 0x7fbbf80cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:34.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.732+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/1820857345 >> 172.21.15.31:0/1820857345 conn(0x7fbbf80ce2d0 msgr2=0x7fbbf80ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:34.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.732+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/1820857345 shutdown_connections 2023-10-06T05:25:34.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.732+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/1820857345 wait complete. 2023-10-06T05:25:34.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.733+0000 7fbbfe5ff700 1 Processor -- start 2023-10-06T05:25:34.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.733+0000 7fbbfe5ff700 1 -- start start 2023-10-06T05:25:34.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.734+0000 7fbbfe5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf8142c20 0x7fbbf80ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:34.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.734+0000 7fbbfe5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142ff0 0x7fbbf80ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:34.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.734+0000 7fbbfe5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbf815a990 0x7fbbf813f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:34.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.734+0000 7fbbfe5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80b1760 con 0x7fbbf815a990 2023-10-06T05:25:34.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.734+0000 7fbbfe5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80b1a40 con 0x7fbbf8142c20 2023-10-06T05:25:34.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.734+0000 7fbbfe5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80b1d20 con 0x7fbbf8142ff0 2023-10-06T05:25:34.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbfcb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbf815a990 0x7fbbf813f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:34.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbfcb9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbf815a990 0x7fbbf813f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47262/0 (socket says 172.21.15.31:47262) 2023-10-06T05:25:34.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbfcb9c700 1 -- 172.21.15.31:0/473957574 learned_addr learned my addr 172.21.15.31:0/473957574 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:34.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbf7fff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf8142c20 0x7fbbf80ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:34.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbf77fe700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142ff0 0x7fbbf80ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:34.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbfcb9c700 1 -- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142ff0 msgr2=0x7fbbf80ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:34.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbfcb9c700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142ff0 0x7fbbf80ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:34.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbfcb9c700 1 -- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf8142c20 msgr2=0x7fbbf80ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:34.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbfcb9c700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf8142c20 0x7fbbf80ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:34.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbfcb9c700 1 -- 172.21.15.31:0/473957574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80094f0 con 0x7fbbf815a990 2023-10-06T05:25:34.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbf7fff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf8142c20 0x7fbbf80ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:34.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbf77fe700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142ff0 0x7fbbf80ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:34.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.735+0000 7fbbfcb9c700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbf815a990 0x7fbbf813f430 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c8e0 tx=0x7fbbf0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:34.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.736+0000 7fbbdffff700 1 -- 172.21.15.31:0/473957574 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0012630 con 0x7fbbf815a990 2023-10-06T05:25:34.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.736+0000 7fbbdffff700 1 -- 172.21.15.31:0/473957574 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf0012790 con 0x7fbbf815a990 2023-10-06T05:25:34.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.736+0000 7fbbdffff700 1 -- 172.21.15.31:0/473957574 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0012b50 con 0x7fbbf815a990 2023-10-06T05:25:34.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.736+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/473957574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf80b2000 con 0x7fbbf815a990 2023-10-06T05:25:34.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.736+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/473957574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf80b1380 con 0x7fbbf815a990 2023-10-06T05:25:34.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.737+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/473957574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf80cbc80 con 0x7fbbf815a990 2023-10-06T05:25:34.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.738+0000 7fbbdffff700 1 -- 172.21.15.31:0/473957574 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fbbf001a3a0 con 0x7fbbf815a990 2023-10-06T05:25:34.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.739+0000 7fbbdffff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbbe4067680 0x7fbbe4069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:34.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.739+0000 7fbbdffff700 1 -- 172.21.15.31:0/473957574 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fbbf00830a0 con 0x7fbbf815a990 2023-10-06T05:25:34.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.739+0000 7fbbf7fff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbbe4067680 0x7fbbe4069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:34.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.740+0000 7fbbf7fff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbbe4067680 0x7fbbe4069b30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80099b0 tx=0x7fbbe8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:34.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.743+0000 7fbbdffff700 1 -- 172.21.15.31:0/473957574 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbf0082db0 con 0x7fbbf815a990 2023-10-06T05:25:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:34.990+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/473957574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbbf80b2520 con 0x7fbbf815a990 2023-10-06T05:25:35.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.306+0000 7fbbdffff700 1 -- 172.21.15.31:0/473957574 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v103) v1 ==== 136+0+0 (secure 0 0 0) 0x7fbbf0085020 con 0x7fbbf815a990 2023-10-06T05:25:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbbe4067680 msgr2=0x7fbbe4069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbbe4067680 0x7fbbe4069b30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80099b0 tx=0x7fbbe8017000 comp rx=0 tx=0).stop 2023-10-06T05:25:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/473957574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbf815a990 msgr2=0x7fbbf813f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbf815a990 0x7fbbf813f430 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c8e0 tx=0x7fbbf0001710 comp rx=0 tx=0).stop 2023-10-06T05:25:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/473957574 shutdown_connections 2023-10-06T05:25:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbbe4067680 0x7fbbe4069b30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbf815a990 0x7fbbf813f430 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbf8142c20 0x7fbbf80ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 --2- 172.21.15.31:0/473957574 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbf8142ff0 0x7fbbf80ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/473957574 >> 172.21.15.31:0/473957574 conn(0x7fbbf80ce2d0 msgr2=0x7fbbf80cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:35.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/473957574 shutdown_connections 2023-10-06T05:25:35.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.309+0000 7fbbfe5ff700 1 -- 172.21.15.31:0/473957574 wait complete. 2023-10-06T05:25:35.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:25:35.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.525+0000 7f6f211ff700 1 Processor -- start 2023-10-06T05:25:35.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.525+0000 7f6f211ff700 1 -- start start 2023-10-06T05:25:35.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.526+0000 7f6f211ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c13a1e0 0x7f6f1c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:35.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.526+0000 7f6f211ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c0c71b0 con 0x7f6f1c13a5b0 2023-10-06T05:25:35.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.526+0000 7f6f211ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c0c7490 con 0x7f6f1c06db50 2023-10-06T05:25:35.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.526+0000 7f6f211ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0c7770 con 0x7f6f1c13a1e0 2023-10-06T05:25:35.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.526+0000 7f6f1a59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6f1c13a5b0 0x7f6f1c06d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59312/0 (socket says 172.21.15.31:59312) 2023-10-06T05:25:35.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.526+0000 7f6f1a59c700 1 -- 172.21.15.31:0/1733910544 learned_addr learned my addr 172.21.15.31:0/1733910544 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:35.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f1ad9d700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c13a1e0 0x7f6f1c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:35.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 603141731 0 0) 0x7f6f1c0c71b0 con 0x7f6f1c13a5b0 2023-10-06T05:25:35.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f04003570 con 0x7f6f1c13a5b0 2023-10-06T05:25:35.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2918521649 0 0) 0x7f6f04003570 con 0x7f6f1c13a5b0 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c13a1e0 msgr2=0x7f6f1c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c13a1e0 0x7f6f1c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 >> v1:172.21.15.62:6789/0 conn(0x7f6f1c06db50 legacy=0x7f6f1c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c0c7a50 con 0x7f6f1c13a5b0 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f6f10001470 con 0x7f6f1c13a5b0 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 >> v1:172.21.15.31:6789/0 conn(0x7f6f1c13a5b0 legacy=0x7f6f1c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f04003a90 0x7f6f04003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c13a1e0 0x7f6f04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 0x7f6f040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0c7770 con 0x7f6f04004d50 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0400bb40 con 0x7f6f04003a90 2023-10-06T05:25:35.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.527+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0400bd00 con 0x7f6f1c13a1e0 2023-10-06T05:25:35.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.528+0000 7f6f1b59e700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 0x7f6f040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:35.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.528+0000 7f6f1ad9d700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f04003a90 0x7f6f04003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:35.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.528+0000 7f6f1a59c700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c13a1e0 0x7f6f04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:35.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.528+0000 7f6f1b59e700 1 -- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c13a1e0 msgr2=0x7f6f04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:35.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.528+0000 7f6f1b59e700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c13a1e0 0x7f6f04004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.528+0000 7f6f1b59e700 1 -- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f04003a90 msgr2=0x7f6f04003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:35.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.528+0000 7f6f1b59e700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f04003a90 0x7f6f04003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.528+0000 7f6f1b59e700 1 -- 172.21.15.31:0/1733910544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f04003570 con 0x7f6f04004d50 2023-10-06T05:25:35.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.528+0000 7f6f1b59e700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 0x7f6f040051c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f6f14001f90 tx=0x7f6f14001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6da65749d0fc085 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:35.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.529+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14001950 con 0x7f6f04004d50 2023-10-06T05:25:35.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.529+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1400c8b0 con 0x7f6f04004d50 2023-10-06T05:25:35.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.529+0000 7f6f19d9b700 1 -- 172.21.15.31:0/1733910544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400cc50 con 0x7f6f04004d50 2023-10-06T05:25:35.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.529+0000 7f6f211ff700 1 -- 172.21.15.31:0/1733910544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 msgr2=0x7f6f040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.529+0000 7f6f211ff700 1 --2- 172.21.15.31:0/1733910544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 0x7f6f040051c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f6f14001f90 tx=0x7f6f14001640 comp rx=0 tx=0).stop 2023-10-06T05:25:35.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.529+0000 7f6f1b59e700 1 -- 172.21.15.31:0/1733910544 reap_dead start 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.529+0000 7f6f211ff700 1 -- 172.21.15.31:0/1733910544 shutdown_connections 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.529+0000 7f6f211ff700 1 -- 172.21.15.31:0/1733910544 >> 172.21.15.31:0/1733910544 conn(0x7f6f1c0cc0b0 msgr2=0x7f6f1c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.529+0000 7f6f211ff700 1 -- 172.21.15.31:0/1733910544 shutdown_connections 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.529+0000 7f6f211ff700 1 -- 172.21.15.31:0/1733910544 wait complete. 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.531+0000 7f6f211ff700 1 Processor -- start 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.531+0000 7f6f211ff700 1 -- start start 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f211ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 0x7f6f1c1692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f211ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c15ad80 0x7f6f1c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f211ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1c156450 0x7f6f1c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f211ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c156e00 con 0x7f6f04004d50 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f211ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0cb640 con 0x7f6f1c156450 2023-10-06T05:25:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f211ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0caa30 con 0x7f6f1c15ad80 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 0x7f6f1c1692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 0x7f6f1c1692b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47284/0 (socket says 172.21.15.31:47284) 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1ad9d700 1 -- 172.21.15.31:0/931183589 learned_addr learned my addr 172.21.15.31:0/931183589 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1a59c700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c15ad80 0x7f6f1c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1b59e700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1c156450 0x7f6f1c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1ad9d700 1 -- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c15ad80 msgr2=0x7f6f1c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1ad9d700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c15ad80 0x7f6f1c15b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1ad9d700 1 -- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1c156450 msgr2=0x7f6f1c1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1ad9d700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1c156450 0x7f6f1c1568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1ad9d700 1 -- 172.21.15.31:0/931183589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c0cad10 con 0x7f6f04004d50 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1a59c700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c15ad80 0x7f6f1c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.532+0000 7f6f1b59e700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1c156450 0x7f6f1c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.533+0000 7f6f1ad9d700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 0x7f6f1c1692b0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c011b10 tx=0x7f6f0c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.533+0000 7f6f02ffd700 1 -- 172.21.15.31:0/931183589 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c005c10 con 0x7f6f04004d50 2023-10-06T05:25:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.533+0000 7f6f02ffd700 1 -- 172.21.15.31:0/931183589 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c005d70 con 0x7f6f04004d50 2023-10-06T05:25:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.533+0000 7f6f02ffd700 1 -- 172.21.15.31:0/931183589 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00f9b0 con 0x7f6f04004d50 2023-10-06T05:25:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.533+0000 7f6f211ff700 1 -- 172.21.15.31:0/931183589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c0caff0 con 0x7f6f04004d50 2023-10-06T05:25:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.533+0000 7f6f211ff700 1 -- 172.21.15.31:0/931183589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1c0ca690 con 0x7f6f04004d50 2023-10-06T05:25:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.534+0000 7f6f211ff700 1 -- 172.21.15.31:0/931183589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1c04c380 con 0x7f6f04004d50 2023-10-06T05:25:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.535+0000 7f6f02ffd700 1 -- 172.21.15.31:0/931183589 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f6f0c00fd60 con 0x7f6f04004d50 2023-10-06T05:25:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.536+0000 7f6f02ffd700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f08067680 0x7f6f08069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.536+0000 7f6f02ffd700 1 -- 172.21.15.31:0/931183589 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6f0c086200 con 0x7f6f04004d50 2023-10-06T05:25:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.537+0000 7f6f1a59c700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f08067680 0x7f6f08069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.537+0000 7f6f1a59c700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f08067680 0x7f6f08069b30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6f100019e0 tx=0x7f6f1000a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:35.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.540+0000 7f6f02ffd700 1 -- 172.21.15.31:0/931183589 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f0c0835c0 con 0x7f6f04004d50 2023-10-06T05:25:35.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.771+0000 7f6f211ff700 1 -- 172.21.15.31:0/931183589 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f1c0c7d30 con 0x7f6f08067680 2023-10-06T05:25:35.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.774+0000 7f6f02ffd700 1 -- 172.21.15.31:0/931183589 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6f1c0c7d30 con 0x7f6f08067680 2023-10-06T05:25:35.775 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:25:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:25:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:35.776 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:25:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 -- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f08067680 msgr2=0x7f6f08069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f08067680 0x7f6f08069b30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6f100019e0 tx=0x7f6f1000a040 comp rx=0 tx=0).stop 2023-10-06T05:25:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 -- 172.21.15.31:0/931183589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 msgr2=0x7f6f1c1692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 0x7f6f1c1692b0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c011b10 tx=0x7f6f0c009120 comp rx=0 tx=0).stop 2023-10-06T05:25:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 -- 172.21.15.31:0/931183589 shutdown_connections 2023-10-06T05:25:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f08067680 0x7f6f08069b30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f1c156450 0x7f6f1c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f04004d50 0x7f6f1c1692b0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 --2- 172.21.15.31:0/931183589 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f1c15ad80 0x7f6f1c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 -- 172.21.15.31:0/931183589 >> 172.21.15.31:0/931183589 conn(0x7f6f1c0cc0b0 msgr2=0x7f6f1c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 -- 172.21.15.31:0/931183589 shutdown_connections 2023-10-06T05:25:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.776+0000 7f6f211ff700 1 -- 172.21.15.31:0/931183589 wait complete. 2023-10-06T05:25:35.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:25:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.992+0000 7f6f4e3ff700 1 Processor -- start 2023-10-06T05:25:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.992+0000 7f6f4e3ff700 1 -- start start 2023-10-06T05:25:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.993+0000 7f6f4e3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6f480c8f40 0x7f6f48068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.993+0000 7f6f4e3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f48068da0 con 0x7f6f4806a300 2023-10-06T05:25:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.993+0000 7f6f4e3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f48069080 con 0x7f6f4815ca40 2023-10-06T05:25:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.993+0000 7f6f4e3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f480cabc0 con 0x7f6f480c8f40 2023-10-06T05:25:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.994+0000 7f6f4c99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6f4806a300 0x7f6f480adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59326/0 (socket says 172.21.15.31:59326) 2023-10-06T05:25:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.994+0000 7f6f4c99c700 1 -- 172.21.15.31:0/315865800 learned_addr learned my addr 172.21.15.31:0/315865800 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.994+0000 7f6f477fe700 1 --2- 172.21.15.31:0/315865800 >> v2:172.21.15.62:3300/0 conn(0x7f6f480c8f40 0x7f6f48068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.994+0000 7f6f46ffd700 1 -- 172.21.15.31:0/315865800 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220484205 0 0) 0x7f6f48068da0 con 0x7f6f4806a300 2023-10-06T05:25:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.994+0000 7f6f46ffd700 1 -- 172.21.15.31:0/315865800 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f30003570 con 0x7f6f4806a300 2023-10-06T05:25:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.994+0000 7f6f477fe700 1 -- 172.21.15.31:0/315865800 >> v1:172.21.15.62:6789/0 conn(0x7f6f4815ca40 legacy=0x7f6f480c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.994+0000 7f6f477fe700 1 -- 172.21.15.31:0/315865800 >> v1:172.21.15.31:6789/0 conn(0x7f6f4806a300 legacy=0x7f6f480adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.994+0000 7f6f477fe700 1 -- 172.21.15.31:0/315865800 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f48069360 con 0x7f6f480c8f40 2023-10-06T05:25:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.995+0000 7f6f477fe700 1 --2- 172.21.15.31:0/315865800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f480c8f40 0x7f6f48068670 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0098e0 tx=0x7f6f3c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5043cfd9e72b45b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.995+0000 7f6f46ffd700 1 -- 172.21.15.31:0/315865800 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00b640 con 0x7f6f480c8f40 2023-10-06T05:25:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.995+0000 7f6f46ffd700 1 -- 172.21.15.31:0/315865800 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c00b7a0 con 0x7f6f480c8f40 2023-10-06T05:25:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.995+0000 7f6f46ffd700 1 -- 172.21.15.31:0/315865800 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00bb40 con 0x7f6f480c8f40 2023-10-06T05:25:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.996+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/315865800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f480c8f40 msgr2=0x7f6f48068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.996+0000 7f6f4e3ff700 1 --2- 172.21.15.31:0/315865800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f480c8f40 0x7f6f48068670 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0098e0 tx=0x7f6f3c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.996+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/315865800 shutdown_connections 2023-10-06T05:25:35.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.996+0000 7f6f4e3ff700 1 --2- 172.21.15.31:0/315865800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f480c8f40 0x7f6f48068670 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:35.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.996+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/315865800 >> 172.21.15.31:0/315865800 conn(0x7f6f480ce290 msgr2=0x7f6f480ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:35.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.996+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/315865800 shutdown_connections 2023-10-06T05:25:35.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.996+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/315865800 wait complete. 2023-10-06T05:25:35.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.998+0000 7f6f4e3ff700 1 Processor -- start 2023-10-06T05:25:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.998+0000 7f6f4e3ff700 1 -- start start 2023-10-06T05:25:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.999+0000 7f6f4e3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4806a300 0x7f6f480c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.999+0000 7f6f4e3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f480c8f40 0x7f6f481597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.999+0000 7f6f4e3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f4815ca40 0x7f6f480c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.999+0000 7f6f4e3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480a98b0 con 0x7f6f4815ca40 2023-10-06T05:25:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.999+0000 7f6f4e3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480c67f0 con 0x7f6f4806a300 2023-10-06T05:25:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:35.999+0000 7f6f4e3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480c7090 con 0x7f6f480c8f40 2023-10-06T05:25:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f4c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f4815ca40 0x7f6f480c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f4c99c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f4815ca40 0x7f6f480c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47294/0 (socket says 172.21.15.31:47294) 2023-10-06T05:25:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f4c99c700 1 -- 172.21.15.31:0/3820253214 learned_addr learned my addr 172.21.15.31:0/3820253214 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f477fe700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f480c8f40 0x7f6f481597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f47fff700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4806a300 0x7f6f480c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f4c99c700 1 -- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f480c8f40 msgr2=0x7f6f481597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f4c99c700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f480c8f40 0x7f6f481597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f4c99c700 1 -- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4806a300 msgr2=0x7f6f480c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f4c99c700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4806a300 0x7f6f480c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f4c99c700 1 -- 172.21.15.31:0/3820253214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c009420 con 0x7f6f4815ca40 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.000+0000 7f6f4c99c700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f4815ca40 0x7f6f480c41a0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f6f40001460 tx=0x7f6f40001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.001+0000 7f6f2ffff700 1 -- 172.21.15.31:0/3820253214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000a830 con 0x7f6f4815ca40 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.001+0000 7f6f2ffff700 1 -- 172.21.15.31:0/3820253214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4000a990 con 0x7f6f4815ca40 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.001+0000 7f6f2ffff700 1 -- 172.21.15.31:0/3820253214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000ad10 con 0x7f6f4815ca40 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.001+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/3820253214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f48159d20 con 0x7f6f4815ca40 2023-10-06T05:25:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.001+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/3820253214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f480c6ac0 con 0x7f6f4815ca40 2023-10-06T05:25:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.002+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/3820253214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f480c7200 con 0x7f6f4815ca40 2023-10-06T05:25:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.003+0000 7f6f2ffff700 1 -- 172.21.15.31:0/3820253214 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f6f40007300 con 0x7f6f4815ca40 2023-10-06T05:25:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.004+0000 7f6f2ffff700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f34067680 0x7f6f34069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.004+0000 7f6f2ffff700 1 -- 172.21.15.31:0/3820253214 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f6f40085be0 con 0x7f6f4815ca40 2023-10-06T05:25:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.004+0000 7f6f47fff700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f34067680 0x7f6f34069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.005+0000 7f6f47fff700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f34067680 0x7f6f34069b30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6f380039e0 tx=0x7f6f38007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:36.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.008+0000 7f6f2ffff700 1 -- 172.21.15.31:0/3820253214 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f40082f10 con 0x7f6f4815ca40 2023-10-06T05:25:36.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.211+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/3820253214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6f4804c380 con 0x7f6f4815ca40 2023-10-06T05:25:36.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.212+0000 7f6f2ffff700 1 -- 172.21.15.31:0/3820253214 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6f40051d80 con 0x7f6f4815ca40 2023-10-06T05:25:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.214+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f34067680 msgr2=0x7f6f34069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.214+0000 7f6f4e3ff700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f34067680 0x7f6f34069b30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6f380039e0 tx=0x7f6f38007ec0 comp rx=0 tx=0).stop 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.214+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/3820253214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f4815ca40 msgr2=0x7f6f480c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.214+0000 7f6f4e3ff700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f4815ca40 0x7f6f480c41a0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f6f40001460 tx=0x7f6f40001ae0 comp rx=0 tx=0).stop 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.214+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/3820253214 shutdown_connections 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.214+0000 7f6f4e3ff700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6f34067680 0x7f6f34069b30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.214+0000 7f6f4e3ff700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f4815ca40 0x7f6f480c41a0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.214+0000 7f6f4e3ff700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4806a300 0x7f6f480c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.214+0000 7f6f4e3ff700 1 --2- 172.21.15.31:0/3820253214 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f480c8f40 0x7f6f481597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.214+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/3820253214 >> 172.21.15.31:0/3820253214 conn(0x7f6f480ce290 msgr2=0x7f6f480aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.215+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/3820253214 shutdown_connections 2023-10-06T05:25:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:36.215+0000 7f6f4e3ff700 1 -- 172.21.15.31:0/3820253214 wait complete. 2023-10-06T05:25:36.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:36.234 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvolume_0000000001034764/de8cd73b-4f67-409c-a28f-1d3c7a5c95fd) 2023-10-06T05:25:36.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:36.315 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T05:25:36.362 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T05:25:36.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:36.363 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvolume_0000000001034764/de8cd73b-4f67-409c-a28f-1d3c7a5c95fd) 2023-10-06T05:25:36.438 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:25:36.438 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvolume_0000000001034764/.meta 2023-10-06T05:25:36.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:36.513 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/group_0000000000630279/subvolume_0000000001034765/a50a2bc0-11dd-48b6-9469-3d2aae96e70d) 2023-10-06T05:25:36.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:36.628 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T05:25:36.675 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T05:25:36.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:36.676 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/group_0000000000630279/subvolume_0000000001034765/a50a2bc0-11dd-48b6-9469-3d2aae96e70d) 2023-10-06T05:25:36.753 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:25:36.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/group_0000000000630279/subvolume_0000000001034765/.meta 2023-10-06T05:25:36.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000001034764 2023-10-06T05:25:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.061+0000 7fdd899ff700 1 Processor -- start 2023-10-06T05:25:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.061+0000 7fdd899ff700 1 -- start start 2023-10-06T05:25:37.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.063+0000 7fdd899ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd84155b40 0x7fdd840c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.063+0000 7fdd899ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd84158190 con 0x7fdd84159c60 2023-10-06T05:25:37.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.063+0000 7fdd899ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd84157580 con 0x7fdd84159890 2023-10-06T05:25:37.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.063+0000 7fdd899ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84157860 con 0x7fdd84155b40 2023-10-06T05:25:37.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.063+0000 7fdd827fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdd84159c60 0x7fdd84155430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46494/0 (socket says 172.21.15.31:46494) 2023-10-06T05:25:37.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.063+0000 7fdd827fc700 1 -- 172.21.15.31:0/2565386918 learned_addr learned my addr 172.21.15.31:0/2565386918 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:37.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.063+0000 7fdd837fe700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd84155b40 0x7fdd840c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.063+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074722834 0 0) 0x7fdd84158190 con 0x7fdd84159c60 2023-10-06T05:25:37.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.063+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd6c003570 con 0x7fdd84159c60 2023-10-06T05:25:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 158866846 0 0) 0x7fdd6c003570 con 0x7fdd84159c60 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd84155b40 msgr2=0x7fdd840c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd84155b40 0x7fdd840c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 >> v1:172.21.15.62:6789/0 conn(0x7fdd84159890 legacy=0x7fdd84156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84157b40 con 0x7fdd84159c60 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fdd78001470 con 0x7fdd84159c60 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 >> v1:172.21.15.31:6789/0 conn(0x7fdd84159c60 legacy=0x7fdd84155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd6c003a90 0x7fdd6c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 0x7fdd6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd84155b40 0x7fdd6c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84157860 con 0x7fdd6c004810 2023-10-06T05:25:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c00bbf0 con 0x7fdd6c003a90 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c00df90 con 0x7fdd84155b40 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd827fc700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 0x7fdd6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd837fe700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd84155b40 0x7fdd6c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.064+0000 7fdd82ffd700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd6c003a90 0x7fdd6c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.065+0000 7fdd827fc700 1 -- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd84155b40 msgr2=0x7fdd6c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.065+0000 7fdd827fc700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd84155b40 0x7fdd6c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.065+0000 7fdd827fc700 1 -- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd6c003a90 msgr2=0x7fdd6c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.065+0000 7fdd827fc700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd6c003a90 0x7fdd6c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.065+0000 7fdd827fc700 1 -- 172.21.15.31:0/2565386918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c003570 con 0x7fdd6c004810 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.065+0000 7fdd82ffd700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd6c003a90 0x7fdd6c003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.065+0000 7fdd827fc700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 0x7fdd6c004c80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c300 tx=0x7fdd78001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165febd40d67ffe2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.065+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7800cb90 con 0x7fdd6c004810 2023-10-06T05:25:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.065+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7800cd10 con 0x7fdd6c004810 2023-10-06T05:25:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.065+0000 7fdd81ffb700 1 -- 172.21.15.31:0/2565386918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78005b70 con 0x7fdd6c004810 2023-10-06T05:25:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.066+0000 7fdd899ff700 1 -- 172.21.15.31:0/2565386918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 msgr2=0x7fdd6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.066+0000 7fdd899ff700 1 --2- 172.21.15.31:0/2565386918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 0x7fdd6c004c80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c300 tx=0x7fdd78001d80 comp rx=0 tx=0).stop 2023-10-06T05:25:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.066+0000 7fdd837fe700 1 -- 172.21.15.31:0/2565386918 reap_dead start 2023-10-06T05:25:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.066+0000 7fdd899ff700 1 -- 172.21.15.31:0/2565386918 shutdown_connections 2023-10-06T05:25:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.066+0000 7fdd899ff700 1 -- 172.21.15.31:0/2565386918 >> 172.21.15.31:0/2565386918 conn(0x7fdd84067560 msgr2=0x7fdd84067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.066+0000 7fdd899ff700 1 -- 172.21.15.31:0/2565386918 shutdown_connections 2023-10-06T05:25:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.066+0000 7fdd899ff700 1 -- 172.21.15.31:0/2565386918 wait complete. 2023-10-06T05:25:37.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.067+0000 7fdd899ff700 1 Processor -- start 2023-10-06T05:25:37.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.067+0000 7fdd899ff700 1 -- start start 2023-10-06T05:25:37.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.068+0000 7fdd899ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 0x7fdd840ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.068+0000 7fdd899ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd840c9340 0x7fdd84149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.068+0000 7fdd899ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd8414ecf0 0x7fdd8414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.068+0000 7fdd899ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8414e860 con 0x7fdd6c004810 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.068+0000 7fdd899ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8414dc50 con 0x7fdd8414ecf0 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.068+0000 7fdd899ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8414df30 con 0x7fdd840c9340 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd82ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 0x7fdd840ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd82ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 0x7fdd840ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38918/0 (socket says 172.21.15.31:38918) 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd82ffd700 1 -- 172.21.15.31:0/326071300 learned_addr learned my addr 172.21.15.31:0/326071300 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd827fc700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd840c9340 0x7fdd84149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd837fe700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd8414ecf0 0x7fdd8414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd82ffd700 1 -- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd840c9340 msgr2=0x7fdd84149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd82ffd700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd840c9340 0x7fdd84149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd82ffd700 1 -- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd8414ecf0 msgr2=0x7fdd8414f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd82ffd700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd8414ecf0 0x7fdd8414f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd82ffd700 1 -- 172.21.15.31:0/326071300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8414e210 con 0x7fdd6c004810 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.069+0000 7fdd82ffd700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 0x7fdd840ca260 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400e840 tx=0x7fdd74009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.070+0000 7fdd6affd700 1 -- 172.21.15.31:0/326071300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd74017070 con 0x7fdd6c004810 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.070+0000 7fdd6affd700 1 -- 172.21.15.31:0/326071300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd74006820 con 0x7fdd6c004810 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.070+0000 7fdd899ff700 1 -- 172.21.15.31:0/326071300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8414e4f0 con 0x7fdd6c004810 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.070+0000 7fdd6affd700 1 -- 172.21.15.31:0/326071300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd74006be0 con 0x7fdd6c004810 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.070+0000 7fdd899ff700 1 -- 172.21.15.31:0/326071300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd840cb290 con 0x7fdd6c004810 2023-10-06T05:25:37.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.071+0000 7fdd899ff700 1 -- 172.21.15.31:0/326071300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8404c380 con 0x7fdd6c004810 2023-10-06T05:25:37.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.072+0000 7fdd6affd700 1 -- 172.21.15.31:0/326071300 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fdd74006d40 con 0x7fdd6c004810 2023-10-06T05:25:37.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.072+0000 7fdd6affd700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd70067680 0x7fdd70069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.072+0000 7fdd6affd700 1 -- 172.21.15.31:0/326071300 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fdd740853d0 con 0x7fdd6c004810 2023-10-06T05:25:37.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.073+0000 7fdd827fc700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd70067680 0x7fdd70069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.073+0000 7fdd827fc700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd70067680 0x7fdd70069b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdd78007c00 tx=0x7fdd78015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.077+0000 7fdd6affd700 1 -- 172.21.15.31:0/326071300 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd74082790 con 0x7fdd6c004810 2023-10-06T05:25:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.316+0000 7fdd899ff700 1 -- 172.21.15.31:0/326071300 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000001034764", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd8414e9a0 con 0x7fdd70067680 2023-10-06T05:25:37.326 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:37.324+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:37.326 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:37.324+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:37.326 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:37.324+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:37.327 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:37.324+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:37.327 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:37.324+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:37.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.411+0000 7fdd6affd700 1 -- 172.21.15.31:0/326071300 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7fdd78023070 con 0x7fdd70067680 2023-10-06T05:25:37.414 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvolume_0000000001034764/de8cd73b-4f67-409c-a28f-1d3c7a5c95fd 2023-10-06T05:25:37.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 -- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd70067680 msgr2=0x7fdd70069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd70067680 0x7fdd70069b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdd78007c00 tx=0x7fdd78015000 comp rx=0 tx=0).stop 2023-10-06T05:25:37.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 -- 172.21.15.31:0/326071300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 msgr2=0x7fdd840ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 0x7fdd840ca260 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400e840 tx=0x7fdd74009040 comp rx=0 tx=0).stop 2023-10-06T05:25:37.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 -- 172.21.15.31:0/326071300 shutdown_connections 2023-10-06T05:25:37.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdd70067680 0x7fdd70069b30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd8414ecf0 0x7fdd8414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd6c004810 0x7fdd840ca260 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 --2- 172.21.15.31:0/326071300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdd840c9340 0x7fdd84149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 -- 172.21.15.31:0/326071300 >> 172.21.15.31:0/326071300 conn(0x7fdd84067560 msgr2=0x7fdd84156b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 -- 172.21.15.31:0/326071300 shutdown_connections 2023-10-06T05:25:37.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.414+0000 7fdd899ff700 1 -- 172.21.15.31:0/326071300 wait complete. 2023-10-06T05:25:37.433 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000001034765 group_0000000000630279 2023-10-06T05:25:37.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.631+0000 7f859b9ff700 1 Processor -- start 2023-10-06T05:25:37.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.631+0000 7f859b9ff700 1 -- start start 2023-10-06T05:25:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.632+0000 7f859b9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8594159380 0x7f8594067560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.632+0000 7f859b9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940b2d20 0x7f8594067aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.632+0000 7f859b9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859415deb0 con 0x7f8594159750 2023-10-06T05:25:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.632+0000 7f859b9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859415d2a0 con 0x7f8594159380 2023-10-06T05:25:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.632+0000 7f859b9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859415d580 con 0x7f85940b2d20 2023-10-06T05:25:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f8598f9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8594159750 0x7f85940b2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46504/0 (socket says 172.21.15.31:46504) 2023-10-06T05:25:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f8598f9a700 1 -- 172.21.15.31:0/1627898921 learned_addr learned my addr 172.21.15.31:0/1627898921 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:37.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f859979b700 1 --2- 172.21.15.31:0/1627898921 >> v2:172.21.15.62:3300/0 conn(0x7f8594159380 0x7f8594067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f8599f9c700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940b2d20 0x7f8594067aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701384095 0 0) 0x7f859415deb0 con 0x7f8594159750 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8580003570 con 0x7f8594159750 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3566146357 0 0) 0x7f8580003570 con 0x7f8594159750 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940b2d20 msgr2=0x7f8594067aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f858bfff700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940b2d20 0x7f8594067aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 >> v2:172.21.15.62:3300/0 conn(0x7f8594159380 msgr2=0x7f8594067560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f858bfff700 1 --2- 172.21.15.31:0/1627898921 >> v2:172.21.15.62:3300/0 conn(0x7f8594159380 0x7f8594067560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859415d860 con 0x7f8594159750 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.633+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f8584001470 con 0x7f8594159750 2023-10-06T05:25:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 >> v1:172.21.15.31:6789/0 conn(0x7f8594159750 legacy=0x7f85940b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f858bfff700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 0x7f8580003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f858bfff700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8580004810 0x7f8580004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f858bfff700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940b2d20 0x7f85800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859415d2a0 con 0x7f8580003a60 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859415d580 con 0x7f8580004810 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858000ba60 con 0x7f85940b2d20 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f859979b700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 0x7f8580003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f8598f9a700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8580004810 0x7f8580004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f8599f9c700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940b2d20 0x7f85800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f859979b700 1 -- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940b2d20 msgr2=0x7f85800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f859979b700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940b2d20 0x7f85800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f859979b700 1 -- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8580004810 msgr2=0x7f8580004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f859979b700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8580004810 0x7f8580004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f859979b700 1 -- 172.21.15.31:0/1627898921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8580003570 con 0x7f8580003a60 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.634+0000 7f8599f9c700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940b2d20 0x7f85800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.635+0000 7f859979b700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 0x7f8580003e70 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8590009c30 tx=0x7f85900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc0cc38c23fa76e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.635+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859000e710 con 0x7f8580003a60 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.635+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859000e870 con 0x7f8580003a60 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.635+0000 7f858bfff700 1 -- 172.21.15.31:0/1627898921 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859000ec30 con 0x7f8580003a60 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.635+0000 7f859b9ff700 1 -- 172.21.15.31:0/1627898921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 msgr2=0x7f8580003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.635+0000 7f859b9ff700 1 --2- 172.21.15.31:0/1627898921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 0x7f8580003e70 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8590009c30 tx=0x7f85900091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.635+0000 7f8599f9c700 1 -- 172.21.15.31:0/1627898921 reap_dead start 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.635+0000 7f859b9ff700 1 -- 172.21.15.31:0/1627898921 shutdown_connections 2023-10-06T05:25:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.635+0000 7f859b9ff700 1 -- 172.21.15.31:0/1627898921 >> 172.21.15.31:0/1627898921 conn(0x7f8594069140 msgr2=0x7f85940aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.636+0000 7f859b9ff700 1 -- 172.21.15.31:0/1627898921 shutdown_connections 2023-10-06T05:25:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.636+0000 7f859b9ff700 1 -- 172.21.15.31:0/1627898921 wait complete. 2023-10-06T05:25:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.637+0000 7f859b9ff700 1 Processor -- start 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.637+0000 7f859b9ff700 1 -- start start 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.638+0000 7f859b9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 0x7f85941489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.638+0000 7f859b9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859414a2d0 0x7f859416a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.638+0000 7f859b9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940ab010 0x7f85940ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.638+0000 7f859b9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85940ab9c0 con 0x7f8580003a60 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.638+0000 7f859b9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859414a870 con 0x7f859414a2d0 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.638+0000 7f859b9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859414ab50 con 0x7f85940ab010 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.638+0000 7f859979b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 0x7f85941489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.638+0000 7f859979b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 0x7f85941489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38930/0 (socket says 172.21.15.31:38930) 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.638+0000 7f859979b700 1 -- 172.21.15.31:0/2724623810 learned_addr learned my addr 172.21.15.31:0/2724623810 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f8598f9a700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859414a2d0 0x7f859416a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f8599f9c700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940ab010 0x7f85940ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f859979b700 1 -- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940ab010 msgr2=0x7f85940ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f859979b700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940ab010 0x7f85940ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f859979b700 1 -- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859414a2d0 msgr2=0x7f859416a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f859979b700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859414a2d0 0x7f859416a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f859979b700 1 -- 172.21.15.31:0/2724623810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859414ae30 con 0x7f8580003a60 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f8599f9c700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940ab010 0x7f85940ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f859979b700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 0x7f85941489e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f85800046d0 tx=0x7f859001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f85897fa700 1 -- 172.21.15.31:0/2724623810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590022070 con 0x7f8580003a60 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f85897fa700 1 -- 172.21.15.31:0/2724623810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859000e9e0 con 0x7f8580003a60 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f85897fa700 1 -- 172.21.15.31:0/2724623810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859001c450 con 0x7f8580003a60 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.639+0000 7f859b9ff700 1 -- 172.21.15.31:0/2724623810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859414b110 con 0x7f8580003a60 2023-10-06T05:25:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.640+0000 7f859b9ff700 1 -- 172.21.15.31:0/2724623810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859414b480 con 0x7f8580003a60 2023-10-06T05:25:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.641+0000 7f859b9ff700 1 -- 172.21.15.31:0/2724623810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859404c380 con 0x7f8580003a60 2023-10-06T05:25:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.641+0000 7f85897fa700 1 -- 172.21.15.31:0/2724623810 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f859001c820 con 0x7f8580003a60 2023-10-06T05:25:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.642+0000 7f85897fa700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8578067680 0x7f8578069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.642+0000 7f85897fa700 1 -- 172.21.15.31:0/2724623810 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f8590086ff0 con 0x7f8580003a60 2023-10-06T05:25:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.643+0000 7f8598f9a700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8578067680 0x7f8578069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.643+0000 7f8598f9a700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8578067680 0x7f8578069b30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f85940abc80 tx=0x7f8594068580 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.647+0000 7f85897fa700 1 -- 172.21.15.31:0/2724623810 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8590084340 con 0x7f8580003a60 2023-10-06T05:25:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.885+0000 7f859b9ff700 1 -- 172.21.15.31:0/2724623810 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000001034765", "group_name": "group_0000000000630279", "target": ["mon-mgr", ""]}) v1 -- 0x7f85940aa0b0 con 0x7f8578067680 2023-10-06T05:25:37.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.952+0000 7f85897fa700 1 -- 172.21.15.31:0/2724623810 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7f85940aa0b0 con 0x7f8578067680 2023-10-06T05:25:37.954 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000630279/subvolume_0000000001034765/a50a2bc0-11dd-48b6-9469-3d2aae96e70d 2023-10-06T05:25:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 -- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8578067680 msgr2=0x7f8578069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8578067680 0x7f8578069b30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f85940abc80 tx=0x7f8594068580 comp rx=0 tx=0).stop 2023-10-06T05:25:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 -- 172.21.15.31:0/2724623810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 msgr2=0x7f85941489e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 0x7f85941489e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f85800046d0 tx=0x7f859001a000 comp rx=0 tx=0).stop 2023-10-06T05:25:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 -- 172.21.15.31:0/2724623810 shutdown_connections 2023-10-06T05:25:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8578067680 0x7f8578069b30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85940ab010 0x7f85940ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8580003a60 0x7f85941489e0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 --2- 172.21.15.31:0/2724623810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859414a2d0 0x7f859416a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:37.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 -- 172.21.15.31:0/2724623810 >> 172.21.15.31:0/2724623810 conn(0x7f8594069140 msgr2=0x7f8594149ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:37.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 -- 172.21.15.31:0/2724623810 shutdown_connections 2023-10-06T05:25:37.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:37.955+0000 7f859b9ff700 1 -- 172.21.15.31:0/2724623810 wait complete. 2023-10-06T05:25:37.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:37.974 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvolume_0000000001034764/.meta) 2023-10-06T05:25:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:25:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:version = 2 2023-10-06T05:25:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T05:25:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvolume_0000000001034764/de8cd73b-4f67-409c-a28f-1d3c7a5c95fd 2023-10-06T05:25:38.045 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:25:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:38.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:38.048 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/group_0000000000630279/subvolume_0000000001034765/.meta) 2023-10-06T05:25:38.118 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:25:38.118 INFO:teuthology.orchestra.run.smithi031.stdout:version = 2 2023-10-06T05:25:38.118 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T05:25:38.118 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/group_0000000000630279/subvolume_0000000001034765/a50a2bc0-11dd-48b6-9469-3d2aae96e70d 2023-10-06T05:25:38.119 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:25:38.119 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:38.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000001034764 2023-10-06T05:25:38.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.322+0000 7f988cdff700 1 Processor -- start 2023-10-06T05:25:38.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.322+0000 7f988cdff700 1 -- start start 2023-10-06T05:25:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.323+0000 7f988cdff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9888159890 0x7f98880c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.323+0000 7f988cdff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9888159c60 0x7f9888156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.323+0000 7f988cdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9888156fa0 0x7f9888155430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.323+0000 7f988cdff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9888158190 con 0x7f9888159c60 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.323+0000 7f988cdff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9888157580 con 0x7f9888159890 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.323+0000 7f988cdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9888157860 con 0x7f9888156fa0 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.324+0000 7f9885d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9888159c60 0x7f9888156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.324+0000 7f9885d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9888159c60 0x7f9888156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38936/0 (socket says 172.21.15.31:38936) 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.324+0000 7f9885d9b700 1 -- 172.21.15.31:0/732107538 learned_addr learned my addr 172.21.15.31:0/732107538 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.324+0000 7f988659c700 1 --2- 172.21.15.31:0/732107538 >> v2:172.21.15.62:3300/0 conn(0x7f9888159890 0x7f98880c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.324+0000 7f9886d9d700 1 --2- 172.21.15.31:0/732107538 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9888156fa0 0x7f9888155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.324+0000 7f9885d9b700 1 -- 172.21.15.31:0/732107538 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9888156fa0 msgr2=0x7f9888155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.324+0000 7f9885d9b700 1 --2- 172.21.15.31:0/732107538 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9888156fa0 0x7f9888155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.324+0000 7f9885d9b700 1 -- 172.21.15.31:0/732107538 >> v2:172.21.15.62:3300/0 conn(0x7f9888159890 msgr2=0x7f98880c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.324+0000 7f9885d9b700 1 --2- 172.21.15.31:0/732107538 >> v2:172.21.15.62:3300/0 conn(0x7f9888159890 0x7f98880c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.324+0000 7f9885d9b700 1 -- 172.21.15.31:0/732107538 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9888157b40 con 0x7f9888159c60 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f9885d9b700 1 --2- 172.21.15.31:0/732107538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888159c60 0x7f9888156a60 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9888064120 tx=0x7f987c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152657823bdd6a92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f988559a700 1 -- 172.21.15.31:0/732107538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c00b640 con 0x7f9888159c60 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f988559a700 1 -- 172.21.15.31:0/732107538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c00b7a0 con 0x7f9888159c60 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f988559a700 1 -- 172.21.15.31:0/732107538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c00bb40 con 0x7f9888159c60 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f988cdff700 1 -- 172.21.15.31:0/732107538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888159c60 msgr2=0x7f9888156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f988cdff700 1 --2- 172.21.15.31:0/732107538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888159c60 0x7f9888156a60 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9888064120 tx=0x7f987c009250 comp rx=0 tx=0).stop 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f988cdff700 1 -- 172.21.15.31:0/732107538 shutdown_connections 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f988cdff700 1 --2- 172.21.15.31:0/732107538 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9888156fa0 0x7f9888155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f988cdff700 1 --2- 172.21.15.31:0/732107538 >> v2:172.21.15.62:3300/0 conn(0x7f9888159890 0x7f98880c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f988cdff700 1 --2- 172.21.15.31:0/732107538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888159c60 0x7f9888156a60 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.325+0000 7f988cdff700 1 -- 172.21.15.31:0/732107538 >> 172.21.15.31:0/732107538 conn(0x7f9888067560 msgr2=0x7f9888067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:38.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.326+0000 7f988cdff700 1 -- 172.21.15.31:0/732107538 shutdown_connections 2023-10-06T05:25:38.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.326+0000 7f988cdff700 1 -- 172.21.15.31:0/732107538 wait complete. 2023-10-06T05:25:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.327+0000 7f988cdff700 1 Processor -- start 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.327+0000 7f988cdff700 1 -- start start 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.328+0000 7f988cdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888156fa0 0x7f988814bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.328+0000 7f988cdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9888159890 0x7f9888149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.328+0000 7f988cdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9888159c60 0x7f98881489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.328+0000 7f988cdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988816dea0 con 0x7f9888156fa0 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.328+0000 7f988cdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9888150bf0 con 0x7f9888159c60 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.328+0000 7f988cdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988816d290 con 0x7f9888159890 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f988659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888156fa0 0x7f988814bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f988659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888156fa0 0x7f988814bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38944/0 (socket says 172.21.15.31:38944) 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f988659c700 1 -- 172.21.15.31:0/711793411 learned_addr learned my addr 172.21.15.31:0/711793411 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f9885d9b700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9888159890 0x7f9888149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f9886d9d700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9888159c60 0x7f98881489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f988659c700 1 -- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9888159890 msgr2=0x7f9888149550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f988659c700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9888159890 0x7f9888149550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f988659c700 1 -- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9888159c60 msgr2=0x7f98881489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f988659c700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9888159c60 0x7f98881489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f988659c700 1 -- 172.21.15.31:0/711793411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c0094f0 con 0x7f9888156fa0 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.329+0000 7f988659c700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888156fa0 0x7f988814bf60 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f987800ec80 tx=0x7f98780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.330+0000 7f98767fc700 1 -- 172.21.15.31:0/711793411 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987800c850 con 0x7f9888156fa0 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.330+0000 7f98767fc700 1 -- 172.21.15.31:0/711793411 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987800c9b0 con 0x7f9888156fa0 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.330+0000 7f98767fc700 1 -- 172.21.15.31:0/711793411 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987800cd50 con 0x7f9888156fa0 2023-10-06T05:25:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.330+0000 7f988cdff700 1 -- 172.21.15.31:0/711793411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988816d570 con 0x7f9888156fa0 2023-10-06T05:25:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.330+0000 7f988cdff700 1 -- 172.21.15.31:0/711793411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988816db30 con 0x7f9888156fa0 2023-10-06T05:25:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.331+0000 7f988cdff700 1 -- 172.21.15.31:0/711793411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988804c380 con 0x7f9888156fa0 2023-10-06T05:25:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.332+0000 7f98767fc700 1 -- 172.21.15.31:0/711793411 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7f9878013680 con 0x7f9888156fa0 2023-10-06T05:25:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.333+0000 7f98767fc700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9868067680 0x7f9868069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.333+0000 7f98767fc700 1 -- 172.21.15.31:0/711793411 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f9878085b10 con 0x7f9888156fa0 2023-10-06T05:25:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.333+0000 7f9885d9b700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9868067680 0x7f9868069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.334+0000 7f9885d9b700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9868067680 0x7f9868069b30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9888064120 tx=0x7f987c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.337+0000 7f98767fc700 1 -- 172.21.15.31:0/711793411 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9878082e60 con 0x7f9888156fa0 2023-10-06T05:25:38.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.570+0000 7f988cdff700 1 -- 172.21.15.31:0/711793411 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000001034764", "target": ["mon-mgr", ""]}) v1 -- 0x7f988816dfe0 con 0x7f9868067680 2023-10-06T05:25:38.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.592+0000 7f98767fc700 1 -- 172.21.15.31:0/711793411 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f987c000d00 con 0x7f9868067680 2023-10-06T05:25:38.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 -- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9868067680 msgr2=0x7f9868069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9868067680 0x7f9868069b30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9888064120 tx=0x7f987c017000 comp rx=0 tx=0).stop 2023-10-06T05:25:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 -- 172.21.15.31:0/711793411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888156fa0 msgr2=0x7f988814bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888156fa0 0x7f988814bf60 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f987800ec80 tx=0x7f98780091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 -- 172.21.15.31:0/711793411 shutdown_connections 2023-10-06T05:25:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9868067680 0x7f9868069b30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9888159c60 0x7f98881489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9888156fa0 0x7f988814bf60 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 --2- 172.21.15.31:0/711793411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9888159890 0x7f9888149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 -- 172.21.15.31:0/711793411 >> 172.21.15.31:0/711793411 conn(0x7f9888067560 msgr2=0x7f98881518c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 -- 172.21.15.31:0/711793411 shutdown_connections 2023-10-06T05:25:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.594+0000 7f988cdff700 1 -- 172.21.15.31:0/711793411 wait complete. 2023-10-06T05:25:38.600 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:38.599+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:38.600 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:38.599+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:38.601 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:38.599+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:38.601 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:38.599+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:38.601 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:38.599+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:38.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000001034765 group_0000000000630279 2023-10-06T05:25:38.620 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:38.618+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:38.620 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:38.618+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:38.620 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:38.618+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:38.620 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:38.618+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:38.621 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:25:38.618+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:25:38.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.815+0000 7fe4965ff700 1 Processor -- start 2023-10-06T05:25:38.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.815+0000 7fe4965ff700 1 -- start start 2023-10-06T05:25:38.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.816+0000 7fe4965ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4900ad670 0x7fe49006ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.816+0000 7fe4965ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49006e600 con 0x7fe49006a630 2023-10-06T05:25:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.816+0000 7fe4965ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49006d9f0 con 0x7fe490156300 2023-10-06T05:25:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.816+0000 7fe4965ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900adae0 con 0x7fe4900ad670 2023-10-06T05:25:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.817+0000 7fe494b9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe49006a630 0x7fe4900c6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46518/0 (socket says 172.21.15.31:46518) 2023-10-06T05:25:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.817+0000 7fe494b9c700 1 -- 172.21.15.31:0/4251046228 learned_addr learned my addr 172.21.15.31:0/4251046228 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.817+0000 7fe48f7fe700 1 --2- 172.21.15.31:0/4251046228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4900ad670 0x7fe49006ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.817+0000 7fe48effd700 1 -- 172.21.15.31:0/4251046228 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986692843 0 0) 0x7fe49006e600 con 0x7fe49006a630 2023-10-06T05:25:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.817+0000 7fe48effd700 1 -- 172.21.15.31:0/4251046228 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe478003570 con 0x7fe49006a630 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.817+0000 7fe48effd700 1 -- 172.21.15.31:0/4251046228 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138391571 0 0) 0x7fe49006d9f0 con 0x7fe490156300 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.817+0000 7fe48effd700 1 -- 172.21.15.31:0/4251046228 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe49006e600 con 0x7fe490156300 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.817+0000 7fe48f7fe700 1 -- 172.21.15.31:0/4251046228 >> v1:172.21.15.62:6789/0 conn(0x7fe490156300 legacy=0x7fe49006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.817+0000 7fe48f7fe700 1 -- 172.21.15.31:0/4251046228 >> v1:172.21.15.31:6789/0 conn(0x7fe49006a630 legacy=0x7fe4900c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.817+0000 7fe48f7fe700 1 -- 172.21.15.31:0/4251046228 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49006dcd0 con 0x7fe4900ad670 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.818+0000 7fe48f7fe700 1 --2- 172.21.15.31:0/4251046228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4900ad670 0x7fe49006ea90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe484009910 tx=0x7fe4840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ebf55ba0c2f01ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.818+0000 7fe48effd700 1 -- 172.21.15.31:0/4251046228 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484011630 con 0x7fe4900ad670 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.818+0000 7fe48effd700 1 -- 172.21.15.31:0/4251046228 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe484011790 con 0x7fe4900ad670 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.818+0000 7fe48effd700 1 -- 172.21.15.31:0/4251046228 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484011b30 con 0x7fe4900ad670 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.818+0000 7fe4965ff700 1 -- 172.21.15.31:0/4251046228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4900ad670 msgr2=0x7fe49006ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.818+0000 7fe4965ff700 1 --2- 172.21.15.31:0/4251046228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4900ad670 0x7fe49006ea90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe484009910 tx=0x7fe4840091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.819+0000 7fe4965ff700 1 -- 172.21.15.31:0/4251046228 shutdown_connections 2023-10-06T05:25:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.819+0000 7fe4965ff700 1 --2- 172.21.15.31:0/4251046228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe4900ad670 0x7fe49006ea90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.819+0000 7fe4965ff700 1 -- 172.21.15.31:0/4251046228 >> 172.21.15.31:0/4251046228 conn(0x7fe490068030 msgr2=0x7fe4900cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.819+0000 7fe4965ff700 1 -- 172.21.15.31:0/4251046228 shutdown_connections 2023-10-06T05:25:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.819+0000 7fe4965ff700 1 -- 172.21.15.31:0/4251046228 wait complete. 2023-10-06T05:25:38.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.820+0000 7fe4965ff700 1 Processor -- start 2023-10-06T05:25:38.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.820+0000 7fe4965ff700 1 -- start start 2023-10-06T05:25:38.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.821+0000 7fe4965ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49006a630 0x7fe490169680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.821+0000 7fe4965ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4900ad670 0x7fe4900b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.821+0000 7fe4965ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490156300 0x7fe4900aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.821+0000 7fe4965ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900b1fa0 con 0x7fe4900ad670 2023-10-06T05:25:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.821+0000 7fe4965ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900b1700 con 0x7fe49006a630 2023-10-06T05:25:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.821+0000 7fe4965ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900b19e0 con 0x7fe490156300 2023-10-06T05:25:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.821+0000 7fe48f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4900ad670 0x7fe4900b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.821+0000 7fe48f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4900ad670 0x7fe4900b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38956/0 (socket says 172.21.15.31:38956) 2023-10-06T05:25:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.821+0000 7fe48f7fe700 1 -- 172.21.15.31:0/1040849736 learned_addr learned my addr 172.21.15.31:0/1040849736 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe494b9c700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490156300 0x7fe4900aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe48ffff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49006a630 0x7fe490169680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe48f7fe700 1 -- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490156300 msgr2=0x7fe4900aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe48f7fe700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490156300 0x7fe4900aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe48f7fe700 1 -- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49006a630 msgr2=0x7fe490169680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe48f7fe700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49006a630 0x7fe490169680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe48f7fe700 1 -- 172.21.15.31:0/1040849736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe484009420 con 0x7fe4900ad670 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe494b9c700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490156300 0x7fe4900aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe48ffff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49006a630 0x7fe490169680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe48f7fe700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4900ad670 0x7fe4900b0d40 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe484009f00 tx=0x7fe484017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe477fff700 1 -- 172.21.15.31:0/1040849736 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48401f070 con 0x7fe4900ad670 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe477fff700 1 -- 172.21.15.31:0/1040849736 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe484000bc0 con 0x7fe4900ad670 2023-10-06T05:25:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.822+0000 7fe4965ff700 1 -- 172.21.15.31:0/1040849736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4900b1cc0 con 0x7fe4900ad670 2023-10-06T05:25:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.823+0000 7fe477fff700 1 -- 172.21.15.31:0/1040849736 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484025420 con 0x7fe4900ad670 2023-10-06T05:25:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.823+0000 7fe4965ff700 1 -- 172.21.15.31:0/1040849736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe490169bc0 con 0x7fe4900ad670 2023-10-06T05:25:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.824+0000 7fe4965ff700 1 -- 172.21.15.31:0/1040849736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4900c4580 con 0x7fe4900ad670 2023-10-06T05:25:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.825+0000 7fe477fff700 1 -- 172.21.15.31:0/1040849736 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 85403+0+0 (secure 0 0 0) 0x7fe484023070 con 0x7fe4900ad670 2023-10-06T05:25:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.826+0000 7fe477fff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe47c0676b0 0x7fe47c069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.826+0000 7fe477fff700 1 -- 172.21.15.31:0/1040849736 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fe484086c90 con 0x7fe4900ad670 2023-10-06T05:25:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.826+0000 7fe48ffff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe47c0676b0 0x7fe47c069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:38.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.827+0000 7fe48ffff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe47c0676b0 0x7fe47c069b60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe480003980 tx=0x7fe480009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:38.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:38.831+0000 7fe477fff700 1 -- 172.21.15.31:0/1040849736 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4900c4580 con 0x7fe4900ad670 2023-10-06T05:25:39.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:39.063+0000 7fe4965ff700 1 -- 172.21.15.31:0/1040849736 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000001034765", "group_name": "group_0000000000630279", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4900c4580 con 0x7fe47c0676b0 2023-10-06T05:25:39.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:39.144+0000 7fe477fff700 1 -- 172.21.15.31:0/1040849736 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 22) v1 ==== 85454+0+0 (secure 0 0 0) 0x7fe4840531a0 con 0x7fe4900ad670 2023-10-06T05:25:41.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:41.157+0000 7fe477fff700 1 -- 172.21.15.31:0/1040849736 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fe4840598b0 con 0x7fe4900ad670 2023-10-06T05:25:44.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.598+0000 7fe477fff700 1 -- 172.21.15.31:0/1040849736 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe4900c4580 con 0x7fe47c0676b0 2023-10-06T05:25:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.601+0000 7fe4965ff700 1 -- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe47c0676b0 msgr2=0x7fe47c069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.601+0000 7fe4965ff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe47c0676b0 0x7fe47c069b60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe480003980 tx=0x7fe480009040 comp rx=0 tx=0).stop 2023-10-06T05:25:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.601+0000 7fe4965ff700 1 -- 172.21.15.31:0/1040849736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4900ad670 msgr2=0x7fe4900b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.601+0000 7fe4965ff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4900ad670 0x7fe4900b0d40 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe484009f00 tx=0x7fe484017000 comp rx=0 tx=0).stop 2023-10-06T05:25:44.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.601+0000 7fe4965ff700 1 -- 172.21.15.31:0/1040849736 shutdown_connections 2023-10-06T05:25:44.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.601+0000 7fe4965ff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe47c0676b0 0x7fe47c069b60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:44.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.601+0000 7fe4965ff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490156300 0x7fe4900aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:44.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.601+0000 7fe4965ff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49006a630 0x7fe490169680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:44.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.601+0000 7fe4965ff700 1 --2- 172.21.15.31:0/1040849736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4900ad670 0x7fe4900b0d40 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:44.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.601+0000 7fe4965ff700 1 -- 172.21.15.31:0/1040849736 >> 172.21.15.31:0/1040849736 conn(0x7fe490068030 msgr2=0x7fe490155560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:44.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.602+0000 7fe4965ff700 1 -- 172.21.15.31:0/1040849736 shutdown_connections 2023-10-06T05:25:44.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:44.602+0000 7fe4965ff700 1 -- 172.21.15.31:0/1040849736 wait complete. 2023-10-06T05:25:44.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:44.630 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T05:25:44.725 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T05:25:44.727 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:25:44.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:25:44.828 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:25:44.938 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:25:44.938 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:25:44.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:44.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:25:45.055 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:25:45.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:45.056 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:25:45.102 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:25:45.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:25:45.103 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:25:45.202 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:25:45.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:25:45.203 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:25:45.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:45.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.444+0000 7f3c0afff700 1 Processor -- start 2023-10-06T05:25:45.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.444+0000 7f3c0afff700 1 -- start start 2023-10-06T05:25:45.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.445+0000 7f3c0afff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3c0406b510 0x7f3c040c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.445+0000 7f3c0afff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c040cab40 0x7f3c041569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.445+0000 7f3c0afff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c04156ef0 con 0x7f3c040ca770 2023-10-06T05:25:45.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.445+0000 7f3c0afff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c040c7a70 con 0x7f3c040cab40 2023-10-06T05:25:45.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.445+0000 7f3c0afff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c040c6df0 con 0x7f3c0406b510 2023-10-06T05:25:45.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c0959c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c040cab40 0x7f3c041569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c0959c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3c040cab40 0x7f3c041569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38966/0 (socket says 172.21.15.31:38966) 2023-10-06T05:25:45.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c0959c700 1 -- 172.21.15.31:0/2853275559 learned_addr learned my addr 172.21.15.31:0/2853275559 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:45.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c08d9b700 1 --1- 172.21.15.31:0/2853275559 >> v1:172.21.15.31:6789/0 conn(0x7f3c040ca770 0x7f3c0406ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46532/0 (socket says 172.21.15.31:46532) 2023-10-06T05:25:45.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c03fff700 1 --2- 172.21.15.31:0/2853275559 >> v2:172.21.15.62:3300/0 conn(0x7f3c0406b510 0x7f3c040c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c0959c700 1 -- 172.21.15.31:0/2853275559 >> v2:172.21.15.62:3300/0 conn(0x7f3c0406b510 msgr2=0x7f3c040c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:45.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c0959c700 1 --2- 172.21.15.31:0/2853275559 >> v2:172.21.15.62:3300/0 conn(0x7f3c0406b510 0x7f3c040c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c0959c700 1 -- 172.21.15.31:0/2853275559 >> v1:172.21.15.31:6789/0 conn(0x7f3c040ca770 legacy=0x7f3c0406ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c0959c700 1 -- 172.21.15.31:0/2853275559 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c040c70d0 con 0x7f3c040cab40 2023-10-06T05:25:45.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c037fe700 1 -- 172.21.15.31:0/2853275559 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094683930 0 0) 0x7f3c04156ef0 con 0x7f3c040ca770 2023-10-06T05:25:45.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.446+0000 7f3c0959c700 1 --2- 172.21.15.31:0/2853275559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040cab40 0x7f3c041569b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f3c040ca030 tx=0x7f3bfc00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c043810a1e8ec098 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:45.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c037fe700 1 -- 172.21.15.31:0/2853275559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc010070 con 0x7f3c040cab40 2023-10-06T05:25:45.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c037fe700 1 -- 172.21.15.31:0/2853275559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bfc00dc80 con 0x7f3c040cab40 2023-10-06T05:25:45.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c037fe700 1 -- 172.21.15.31:0/2853275559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc016400 con 0x7f3c040cab40 2023-10-06T05:25:45.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c0afff700 1 -- 172.21.15.31:0/2853275559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040cab40 msgr2=0x7f3c041569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c0afff700 1 --2- 172.21.15.31:0/2853275559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040cab40 0x7f3c041569b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f3c040ca030 tx=0x7f3bfc00ba50 comp rx=0 tx=0).stop 2023-10-06T05:25:45.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c0afff700 1 -- 172.21.15.31:0/2853275559 shutdown_connections 2023-10-06T05:25:45.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c0afff700 1 --2- 172.21.15.31:0/2853275559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040cab40 0x7f3c041569b0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c0afff700 1 --2- 172.21.15.31:0/2853275559 >> v2:172.21.15.62:3300/0 conn(0x7f3c0406b510 0x7f3c040c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c0afff700 1 -- 172.21.15.31:0/2853275559 >> 172.21.15.31:0/2853275559 conn(0x7f3c040cc0b0 msgr2=0x7f3c040cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:45.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c0afff700 1 -- 172.21.15.31:0/2853275559 shutdown_connections 2023-10-06T05:25:45.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.447+0000 7f3c0afff700 1 -- 172.21.15.31:0/2853275559 wait complete. 2023-10-06T05:25:45.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.449+0000 7f3c0afff700 1 Processor -- start 2023-10-06T05:25:45.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.449+0000 7f3c0afff700 1 -- start start 2023-10-06T05:25:45.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c0afff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c0406b510 0x7f3c040afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c0afff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040ca770 0x7f3c040b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c0afff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c040cab40 0x7f3c041695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c0afff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c040b0a70 con 0x7f3c040ca770 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c0afff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0416a8e0 con 0x7f3c040cab40 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c0afff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0416abc0 con 0x7f3c0406b510 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040ca770 0x7f3c040b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040ca770 0x7f3c040b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38968/0 (socket says 172.21.15.31:38968) 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c03fff700 1 -- 172.21.15.31:0/890609118 learned_addr learned my addr 172.21.15.31:0/890609118 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c0959c700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c040cab40 0x7f3c041695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.450+0000 7f3c08d9b700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c0406b510 0x7f3c040afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c03fff700 1 -- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c0406b510 msgr2=0x7f3c040afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c03fff700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c0406b510 0x7f3c040afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c03fff700 1 -- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c040cab40 msgr2=0x7f3c041695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c03fff700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c040cab40 0x7f3c041695e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c03fff700 1 -- 172.21.15.31:0/890609118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc00bcf0 con 0x7f3c040ca770 2023-10-06T05:25:45.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c03fff700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040ca770 0x7f3c040b01f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000d860 tx=0x7f3bf0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:45.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c00ff9700 1 -- 172.21.15.31:0/890609118 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00125f0 con 0x7f3c040ca770 2023-10-06T05:25:45.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c00ff9700 1 -- 172.21.15.31:0/890609118 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf0012750 con 0x7f3c040ca770 2023-10-06T05:25:45.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c00ff9700 1 -- 172.21.15.31:0/890609118 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0012b10 con 0x7f3c040ca770 2023-10-06T05:25:45.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c0afff700 1 -- 172.21.15.31:0/890609118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0416aea0 con 0x7f3c040ca770 2023-10-06T05:25:45.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.451+0000 7f3c0afff700 1 -- 172.21.15.31:0/890609118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0413a790 con 0x7f3c040ca770 2023-10-06T05:25:45.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.453+0000 7f3c0afff700 1 -- 172.21.15.31:0/890609118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0413a950 con 0x7f3c040ca770 2023-10-06T05:25:45.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.453+0000 7f3c00ff9700 1 -- 172.21.15.31:0/890609118 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f3bf001e460 con 0x7f3c040ca770 2023-10-06T05:25:45.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.454+0000 7f3c00ff9700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3be8067830 0x7f3be8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.454+0000 7f3c00ff9700 1 -- 172.21.15.31:0/890609118 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f3bf0083570 con 0x7f3c040ca770 2023-10-06T05:25:45.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.455+0000 7f3c08d9b700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3be8067830 0x7f3be8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.455+0000 7f3c08d9b700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3be8067830 0x7f3be8069ce0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80012a0 tx=0x7f3bf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:45.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.458+0000 7f3c00ff9700 1 -- 172.21.15.31:0/890609118 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bf0055a50 con 0x7f3c040ca770 2023-10-06T05:25:45.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.707+0000 7f3c0afff700 1 -- 172.21.15.31:0/890609118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c0416a740 con 0x7f3c040ca770 2023-10-06T05:25:45.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.708+0000 7f3c00ff9700 1 -- 172.21.15.31:0/890609118 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f3c0416a740 con 0x7f3c040ca770 2023-10-06T05:25:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:45.711 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1489945066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1489945066},{"type":"v1","addr":"172.21.15.62:6834","nonce":1489945066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/660206899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":660206899},{"type":"v1","addr":"172.21.15.62:6835","nonce":660206899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8862,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1056123914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1056123914},{"type":"v1","addr":"172.21.15.31:6834","nonce":1056123914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:24.746699+0000","modified":"2023-10-06T05:25:28.266401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8865},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8865":{"gid":8865,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1535525741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1535525741},{"type":"v1","addr":"172.21.15.31:6835","nonce":1535525741}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:25:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 -- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3be8067830 msgr2=0x7f3be8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3be8067830 0x7f3be8069ce0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80012a0 tx=0x7f3bf8009040 comp rx=0 tx=0).stop 2023-10-06T05:25:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 -- 172.21.15.31:0/890609118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040ca770 msgr2=0x7f3c040b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040ca770 0x7f3c040b01f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000d860 tx=0x7f3bf0008040 comp rx=0 tx=0).stop 2023-10-06T05:25:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 -- 172.21.15.31:0/890609118 shutdown_connections 2023-10-06T05:25:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3be8067830 0x7f3be8069ce0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c040cab40 0x7f3c041695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c0406b510 0x7f3c040afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 --2- 172.21.15.31:0/890609118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c040ca770 0x7f3c040b01f0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 -- 172.21.15.31:0/890609118 >> 172.21.15.31:0/890609118 conn(0x7f3c040cc0b0 msgr2=0x7f3c040c17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.710+0000 7f3c0afff700 1 -- 172.21.15.31:0/890609118 shutdown_connections 2023-10-06T05:25:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.711+0000 7f3c0afff700 1 -- 172.21.15.31:0/890609118 wait complete. 2023-10-06T05:25:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 103 2023-10-06T05:25:45.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:45.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.923+0000 7fd19abff700 1 Processor -- start 2023-10-06T05:25:45.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.923+0000 7fd19abff700 1 -- start start 2023-10-06T05:25:45.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.925+0000 7fd19abff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1940c6f80 0x7fd194159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.925+0000 7fd19abff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd194067f70 con 0x7fd1941596d0 2023-10-06T05:25:45.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.925+0000 7fd19abff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19406e040 con 0x7fd1940c16e0 2023-10-06T05:25:45.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.925+0000 7fd19abff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19406b040 con 0x7fd1940c6f80 2023-10-06T05:25:45.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.925+0000 7fd193fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1940c6f80 0x7fd194159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.925+0000 7fd193fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1940c6f80 0x7fd194159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38976/0 (socket says 172.21.15.31:38976) 2023-10-06T05:25:45.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.925+0000 7fd193fff700 1 -- 172.21.15.31:0/204216974 learned_addr learned my addr 172.21.15.31:0/204216974 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:45.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.925+0000 7fd19899b700 1 --1- 172.21.15.31:0/204216974 >> v1:172.21.15.31:6789/0 conn(0x7fd1941596d0 0x7fd19406a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46534/0 (socket says 172.21.15.31:46534) 2023-10-06T05:25:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.926+0000 7fd193fff700 1 -- 172.21.15.31:0/204216974 >> v1:172.21.15.62:6789/0 conn(0x7fd1940c16e0 legacy=0x7fd19406e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.926+0000 7fd193fff700 1 -- 172.21.15.31:0/204216974 >> v1:172.21.15.31:6789/0 conn(0x7fd1941596d0 legacy=0x7fd19406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.926+0000 7fd193fff700 1 -- 172.21.15.31:0/204216974 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1940681c0 con 0x7fd1940c6f80 2023-10-06T05:25:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.926+0000 7fd1937fe700 1 -- 172.21.15.31:0/204216974 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809213621 0 0) 0x7fd194067f70 con 0x7fd1941596d0 2023-10-06T05:25:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.926+0000 7fd193fff700 1 --2- 172.21.15.31:0/204216974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1940c6f80 0x7fd194159aa0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fd1800099b0 tx=0x7fd180009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc738c521a3795d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.926+0000 7fd1937fe700 1 -- 172.21.15.31:0/204216974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000b860 con 0x7fd1940c6f80 2023-10-06T05:25:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.926+0000 7fd1937fe700 1 -- 172.21.15.31:0/204216974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18000b9c0 con 0x7fd1940c6f80 2023-10-06T05:25:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.926+0000 7fd1937fe700 1 -- 172.21.15.31:0/204216974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000bd60 con 0x7fd1940c6f80 2023-10-06T05:25:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.927+0000 7fd19abff700 1 -- 172.21.15.31:0/204216974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1940c6f80 msgr2=0x7fd194159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.927+0000 7fd19abff700 1 --2- 172.21.15.31:0/204216974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1940c6f80 0x7fd194159aa0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fd1800099b0 tx=0x7fd180009250 comp rx=0 tx=0).stop 2023-10-06T05:25:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.927+0000 7fd19abff700 1 -- 172.21.15.31:0/204216974 shutdown_connections 2023-10-06T05:25:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.927+0000 7fd19abff700 1 --2- 172.21.15.31:0/204216974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1940c6f80 0x7fd194159aa0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.927+0000 7fd19abff700 1 -- 172.21.15.31:0/204216974 >> 172.21.15.31:0/204216974 conn(0x7fd1940c9e80 msgr2=0x7fd19406a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.927+0000 7fd19abff700 1 -- 172.21.15.31:0/204216974 shutdown_connections 2023-10-06T05:25:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.927+0000 7fd19abff700 1 -- 172.21.15.31:0/204216974 wait complete. 2023-10-06T05:25:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.928+0000 7fd19abff700 1 Processor -- start 2023-10-06T05:25:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.929+0000 7fd19abff700 1 -- start start 2023-10-06T05:25:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.929+0000 7fd19abff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1940c16e0 0x7fd194144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.929+0000 7fd19abff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1940c6f80 0x7fd1940c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.929+0000 7fd19abff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1941596d0 0x7fd1940c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.929+0000 7fd19abff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd194156e20 con 0x7fd1941596d0 2023-10-06T05:25:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.929+0000 7fd19abff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd194157100 con 0x7fd1940c6f80 2023-10-06T05:25:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.929+0000 7fd19abff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940c3a90 con 0x7fd1940c16e0 2023-10-06T05:25:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd19919c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1941596d0 0x7fd1940c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd19919c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1941596d0 0x7fd1940c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38980/0 (socket says 172.21.15.31:38980) 2023-10-06T05:25:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd19919c700 1 -- 172.21.15.31:0/1318760134 learned_addr learned my addr 172.21.15.31:0/1318760134 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd19899b700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1940c16e0 0x7fd194144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd193fff700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1940c6f80 0x7fd1940c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd19919c700 1 -- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1940c16e0 msgr2=0x7fd194144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd19919c700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1940c16e0 0x7fd194144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd19919c700 1 -- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1940c6f80 msgr2=0x7fd1940c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd19919c700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1940c6f80 0x7fd1940c43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd19919c700 1 -- 172.21.15.31:0/1318760134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800094f0 con 0x7fd1941596d0 2023-10-06T05:25:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.930+0000 7fd19919c700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1941596d0 0x7fd1940c5fc0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00c8a0 tx=0x7fd18c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:45.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.931+0000 7fd190ff9700 1 -- 172.21.15.31:0/1318760134 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c012630 con 0x7fd1941596d0 2023-10-06T05:25:45.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.931+0000 7fd190ff9700 1 -- 172.21.15.31:0/1318760134 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c0127b0 con 0x7fd1941596d0 2023-10-06T05:25:45.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.931+0000 7fd190ff9700 1 -- 172.21.15.31:0/1318760134 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c012b70 con 0x7fd1941596d0 2023-10-06T05:25:45.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.931+0000 7fd19abff700 1 -- 172.21.15.31:0/1318760134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1940c3d00 con 0x7fd1941596d0 2023-10-06T05:25:45.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.931+0000 7fd19abff700 1 -- 172.21.15.31:0/1318760134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd194144890 con 0x7fd1941596d0 2023-10-06T05:25:45.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.932+0000 7fd19abff700 1 -- 172.21.15.31:0/1318760134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1940c4220 con 0x7fd1941596d0 2023-10-06T05:25:45.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.933+0000 7fd190ff9700 1 -- 172.21.15.31:0/1318760134 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fd18c003030 con 0x7fd1941596d0 2023-10-06T05:25:45.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.933+0000 7fd190ff9700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd178067830 0x7fd178069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:45.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.934+0000 7fd190ff9700 1 -- 172.21.15.31:0/1318760134 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fd18c082e50 con 0x7fd1941596d0 2023-10-06T05:25:45.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.934+0000 7fd19899b700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd178067830 0x7fd178069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:45.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.935+0000 7fd19899b700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd178067830 0x7fd178069ce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd1880011e0 tx=0x7fd188007bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:45.938+0000 7fd190ff9700 1 -- 172.21.15.31:0/1318760134 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd18c051a90 con 0x7fd1941596d0 2023-10-06T05:25:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.146+0000 7fd19abff700 1 -- 172.21.15.31:0/1318760134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd19404c380 con 0x7fd1941596d0 2023-10-06T05:25:46.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.147+0000 7fd190ff9700 1 -- 172.21.15.31:0/1318760134 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+17164 (secure 0 0 0) 0x7fd18c0550b0 con 0x7fd1941596d0 2023-10-06T05:25:46.150 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:46.150 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":152,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:25:38.128777+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:25:22.964655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:23.719177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.3","mappings":[{"from":1,"to":6}]},{"pgid":"26.20","mappings":[{"from":3,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 -- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd178067830 msgr2=0x7fd178069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd178067830 0x7fd178069ce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd1880011e0 tx=0x7fd188007bd0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 -- 172.21.15.31:0/1318760134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1941596d0 msgr2=0x7fd1940c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1941596d0 0x7fd1940c5fc0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00c8a0 tx=0x7fd18c001670 comp rx=0 tx=0).stop 2023-10-06T05:25:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 -- 172.21.15.31:0/1318760134 shutdown_connections 2023-10-06T05:25:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd178067830 0x7fd178069ce0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1941596d0 0x7fd1940c5fc0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1940c16e0 0x7fd194144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 --2- 172.21.15.31:0/1318760134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1940c6f80 0x7fd1940c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 -- 172.21.15.31:0/1318760134 >> 172.21.15.31:0/1318760134 conn(0x7fd1940c9e80 msgr2=0x7fd19406b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 -- 172.21.15.31:0/1318760134 shutdown_connections 2023-10-06T05:25:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.149+0000 7fd19abff700 1 -- 172.21.15.31:0/1318760134 wait complete. 2023-10-06T05:25:46.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:25:22.964655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:25:23.719177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:25:46.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:25:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.368+0000 7fb091bff700 1 Processor -- start 2023-10-06T05:25:46.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.368+0000 7fb091bff700 1 -- start start 2023-10-06T05:25:46.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.369+0000 7fb091bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb08c159930 0x7fb08c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb091bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08c15d2d0 con 0x7fb08c159560 2023-10-06T05:25:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb091bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08c15d5b0 con 0x7fb08c0cd580 2023-10-06T05:25:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb091bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08c15d890 con 0x7fb08c159930 2023-10-06T05:25:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb08b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb08c159560 0x7fb08c0cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46550/0 (socket says 172.21.15.31:46550) 2023-10-06T05:25:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb08b7fe700 1 -- 172.21.15.31:0/2683276960 learned_addr learned my addr 172.21.15.31:0/2683276960 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:46.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb08affd700 1 --2- 172.21.15.31:0/2683276960 >> v2:172.21.15.62:3300/0 conn(0x7fb08c159930 0x7fb08c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb08a7fc700 1 -- 172.21.15.31:0/2683276960 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016856507 0 0) 0x7fb08c15d2d0 con 0x7fb08c159560 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb08a7fc700 1 -- 172.21.15.31:0/2683276960 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb074003570 con 0x7fb08c159560 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb08a7fc700 1 -- 172.21.15.31:0/2683276960 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258486804 0 0) 0x7fb08c15d5b0 con 0x7fb08c0cd580 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb08a7fc700 1 -- 172.21.15.31:0/2683276960 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb08c15d2d0 con 0x7fb08c0cd580 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb08affd700 1 -- 172.21.15.31:0/2683276960 >> v1:172.21.15.62:6789/0 conn(0x7fb08c0cd580 legacy=0x7fb08c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.370+0000 7fb08affd700 1 -- 172.21.15.31:0/2683276960 >> v1:172.21.15.31:6789/0 conn(0x7fb08c159560 legacy=0x7fb08c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.371+0000 7fb08affd700 1 -- 172.21.15.31:0/2683276960 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c15db70 con 0x7fb08c159930 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.371+0000 7fb08affd700 1 --2- 172.21.15.31:0/2683276960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08c159930 0x7fb08c0b2470 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb0800098e0 tx=0x7fb0800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d55111782575da0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.371+0000 7fb08a7fc700 1 -- 172.21.15.31:0/2683276960 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080011630 con 0x7fb08c159930 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.371+0000 7fb08a7fc700 1 -- 172.21.15.31:0/2683276960 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080011790 con 0x7fb08c159930 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.371+0000 7fb08a7fc700 1 -- 172.21.15.31:0/2683276960 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080011b30 con 0x7fb08c159930 2023-10-06T05:25:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.372+0000 7fb091bff700 1 -- 172.21.15.31:0/2683276960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08c159930 msgr2=0x7fb08c0b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.372+0000 7fb091bff700 1 --2- 172.21.15.31:0/2683276960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08c159930 0x7fb08c0b2470 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb0800098e0 tx=0x7fb0800091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.372+0000 7fb091bff700 1 -- 172.21.15.31:0/2683276960 shutdown_connections 2023-10-06T05:25:46.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.372+0000 7fb091bff700 1 --2- 172.21.15.31:0/2683276960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08c159930 0x7fb08c0b2470 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.372+0000 7fb091bff700 1 -- 172.21.15.31:0/2683276960 >> 172.21.15.31:0/2683276960 conn(0x7fb08c06ec90 msgr2=0x7fb08c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:46.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.372+0000 7fb091bff700 1 -- 172.21.15.31:0/2683276960 shutdown_connections 2023-10-06T05:25:46.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.372+0000 7fb091bff700 1 -- 172.21.15.31:0/2683276960 wait complete. 2023-10-06T05:25:46.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.373+0000 7fb091bff700 1 Processor -- start 2023-10-06T05:25:46.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.373+0000 7fb091bff700 1 -- start start 2023-10-06T05:25:46.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.374+0000 7fb091bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08c0cd580 0x7fb08c0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.374+0000 7fb091bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08c159560 0x7fb08c150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.374+0000 7fb091bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb08c159930 0x7fb08c0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.374+0000 7fb091bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0c74a0 con 0x7fb08c0cd580 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.374+0000 7fb091bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0c86d0 con 0x7fb08c159560 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.374+0000 7fb091bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0c7d00 con 0x7fb08c159930 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08c0cd580 0x7fb08c0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08c0cd580 0x7fb08c0c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38984/0 (socket says 172.21.15.31:38984) 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08b7fe700 1 -- 172.21.15.31:0/2416754947 learned_addr learned my addr 172.21.15.31:0/2416754947 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08affd700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08c159560 0x7fb08c150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08b7fe700 1 -- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb08c159930 msgr2=0x7fb08c0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08bfff700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb08c159930 0x7fb08c0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08b7fe700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb08c159930 0x7fb08c0d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08b7fe700 1 -- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08c159560 msgr2=0x7fb08c150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08b7fe700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08c159560 0x7fb08c150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08b7fe700 1 -- 172.21.15.31:0/2416754947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb080009420 con 0x7fb08c0cd580 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08affd700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08c159560 0x7fb08c150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08bfff700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb08c159930 0x7fb08c0d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb08b7fe700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08c0cd580 0x7fb08c0c8190 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00c7c0 tx=0x7fb07c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.375+0000 7fb0737fe700 1 -- 172.21.15.31:0/2416754947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c005b30 con 0x7fb08c0cd580 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.376+0000 7fb0737fe700 1 -- 172.21.15.31:0/2416754947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c005c90 con 0x7fb08c0cd580 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.376+0000 7fb091bff700 1 -- 172.21.15.31:0/2416754947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08c150e70 con 0x7fb08c0cd580 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.376+0000 7fb0737fe700 1 -- 172.21.15.31:0/2416754947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c007300 con 0x7fb08c0cd580 2023-10-06T05:25:46.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.376+0000 7fb091bff700 1 -- 172.21.15.31:0/2416754947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08c0c76b0 con 0x7fb08c0cd580 2023-10-06T05:25:46.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.377+0000 7fb091bff700 1 -- 172.21.15.31:0/2416754947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08c04c380 con 0x7fb08c0cd580 2023-10-06T05:25:46.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.377+0000 7fb0737fe700 1 -- 172.21.15.31:0/2416754947 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fb07c01b400 con 0x7fb08c0cd580 2023-10-06T05:25:46.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.378+0000 7fb0737fe700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb078067830 0x7fb078069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.378+0000 7fb0737fe700 1 -- 172.21.15.31:0/2416754947 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fb07c082a70 con 0x7fb08c0cd580 2023-10-06T05:25:46.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.379+0000 7fb08affd700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb078067830 0x7fb078069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.380+0000 7fb08affd700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb078067830 0x7fb078069ce0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0cd500 tx=0x7fb080017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.383+0000 7fb0737fe700 1 -- 172.21.15.31:0/2416754947 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb07c0516b0 con 0x7fb08c0cd580 2023-10-06T05:25:46.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.587+0000 7fb091bff700 1 -- 172.21.15.31:0/2416754947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb08c15de50 con 0x7fb08c0cd580 2023-10-06T05:25:46.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.588+0000 7fb0737fe700 1 -- 172.21.15.31:0/2416754947 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v152) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb07c054cd0 con 0x7fb08c0cd580 2023-10-06T05:25:46.591 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:25:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.590+0000 7fb091bff700 1 -- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb078067830 msgr2=0x7fb078069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.590+0000 7fb091bff700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb078067830 0x7fb078069ce0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0cd500 tx=0x7fb080017000 comp rx=0 tx=0).stop 2023-10-06T05:25:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.590+0000 7fb091bff700 1 -- 172.21.15.31:0/2416754947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08c0cd580 msgr2=0x7fb08c0c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.590+0000 7fb091bff700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08c0cd580 0x7fb08c0c8190 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00c7c0 tx=0x7fb07c007ea0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.590+0000 7fb091bff700 1 -- 172.21.15.31:0/2416754947 shutdown_connections 2023-10-06T05:25:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.590+0000 7fb091bff700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb078067830 0x7fb078069ce0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.590+0000 7fb091bff700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb08c159930 0x7fb08c0d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.590+0000 7fb091bff700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08c0cd580 0x7fb08c0c8190 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.590+0000 7fb091bff700 1 --2- 172.21.15.31:0/2416754947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb08c159560 0x7fb08c150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.590+0000 7fb091bff700 1 -- 172.21.15.31:0/2416754947 >> 172.21.15.31:0/2416754947 conn(0x7fb08c06ec90 msgr2=0x7fb08c0b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:46.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.591+0000 7fb091bff700 1 -- 172.21.15.31:0/2416754947 shutdown_connections 2023-10-06T05:25:46.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.591+0000 7fb091bff700 1 -- 172.21.15.31:0/2416754947 wait complete. 2023-10-06T05:25:46.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:25:46.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.806+0000 7efedadff700 1 Processor -- start 2023-10-06T05:25:46.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.806+0000 7efedadff700 1 -- start start 2023-10-06T05:25:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.807+0000 7efedadff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efed40cd580 0x7efed41645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.807+0000 7efedadff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed415d2d0 con 0x7efed4159560 2023-10-06T05:25:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.807+0000 7efedadff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed415d5b0 con 0x7efed4159930 2023-10-06T05:25:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.807+0000 7efedadff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed415d890 con 0x7efed40cd580 2023-10-06T05:25:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.807+0000 7efed8b9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efed4159560 0x7efed40cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46552/0 (socket says 172.21.15.31:46552) 2023-10-06T05:25:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.807+0000 7efed8b9b700 1 -- 172.21.15.31:0/3099789975 learned_addr learned my addr 172.21.15.31:0/3099789975 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed939c700 1 --2- 172.21.15.31:0/3099789975 >> v2:172.21.15.62:3300/0 conn(0x7efed40cd580 0x7efed41645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784284306 0 0) 0x7efed415d2d0 con 0x7efed4159560 2023-10-06T05:25:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec4003570 con 0x7efed4159560 2023-10-06T05:25:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2377330762 0 0) 0x7efec4003570 con 0x7efed4159560 2023-10-06T05:25:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 >> v2:172.21.15.62:3300/0 conn(0x7efed40cd580 msgr2=0x7efed41645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 --2- 172.21.15.31:0/3099789975 >> v2:172.21.15.62:3300/0 conn(0x7efed40cd580 0x7efed41645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 >> v1:172.21.15.62:6789/0 conn(0x7efed4159930 legacy=0x7efed40b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed415db70 con 0x7efed4159560 2023-10-06T05:25:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7efec8001470 con 0x7efed4159560 2023-10-06T05:25:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 >> v1:172.21.15.31:6789/0 conn(0x7efed4159560 legacy=0x7efed40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4003a60 0x7efec4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efec4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efec40051c0 0x7efec400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed415d890 con 0x7efec4004810 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec400c0e0 con 0x7efec4003a60 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.808+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec400c2a0 con 0x7efec40051c0 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed3fff700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efec4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed8b9b700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4003a60 0x7efec4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed939c700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efec40051c0 0x7efec400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed3fff700 1 -- 172.21.15.31:0/3099789975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efec40051c0 msgr2=0x7efec400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed3fff700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efec40051c0 0x7efec400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed3fff700 1 -- 172.21.15.31:0/3099789975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4003a60 msgr2=0x7efec4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed3fff700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4003a60 0x7efec4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed3fff700 1 -- 172.21.15.31:0/3099789975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4003570 con 0x7efec4004810 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed939c700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efec40051c0 0x7efec400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed939c700 1 -- 172.21.15.31:0/3099789975 reap_dead start 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.809+0000 7efed3fff700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efec4004c80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7efec0001a60 tx=0x7efec0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbac748fffac7f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.810+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0001ea0 con 0x7efec4004810 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.810+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec000c590 con 0x7efec4004810 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.810+0000 7efed37fe700 1 -- 172.21.15.31:0/3099789975 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000c930 con 0x7efec4004810 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.810+0000 7efedadff700 1 -- 172.21.15.31:0/3099789975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 msgr2=0x7efec4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.810+0000 7efedadff700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efec4004c80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7efec0001a60 tx=0x7efec0007d30 comp rx=0 tx=0).stop 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.810+0000 7efedadff700 1 -- 172.21.15.31:0/3099789975 shutdown_connections 2023-10-06T05:25:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.810+0000 7efedadff700 1 --2- 172.21.15.31:0/3099789975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efec4004c80 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.810+0000 7efedadff700 1 -- 172.21.15.31:0/3099789975 >> 172.21.15.31:0/3099789975 conn(0x7efed406ec90 msgr2=0x7efed406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.810+0000 7efedadff700 1 -- 172.21.15.31:0/3099789975 shutdown_connections 2023-10-06T05:25:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.810+0000 7efedadff700 1 -- 172.21.15.31:0/3099789975 wait complete. 2023-10-06T05:25:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.812+0000 7efedadff700 1 Processor -- start 2023-10-06T05:25:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.812+0000 7efedadff700 1 -- start start 2023-10-06T05:25:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.813+0000 7efedadff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efed4155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.813+0000 7efedadff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efed40c9e80 0x7efed40c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.813+0000 7efedadff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed4169810 0x7efed41585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.813+0000 7efedadff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed415b120 con 0x7efec4004810 2023-10-06T05:25:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.813+0000 7efedadff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed40ca2f0 con 0x7efed4169810 2023-10-06T05:25:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.813+0000 7efedadff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed416a030 con 0x7efed40c9e80 2023-10-06T05:25:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.813+0000 7efed8b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efed4155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.813+0000 7efed8b9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efed4155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39006/0 (socket says 172.21.15.31:39006) 2023-10-06T05:25:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.813+0000 7efed8b9b700 1 -- 172.21.15.31:0/453150663 learned_addr learned my addr 172.21.15.31:0/453150663 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efed8b9b700 1 -- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efed40c9e80 msgr2=0x7efed40c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efed8b9b700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efed40c9e80 0x7efed40c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efed8b9b700 1 -- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed4169810 msgr2=0x7efed41585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efed8b9b700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed4169810 0x7efed41585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efed8b9b700 1 -- 172.21.15.31:0/453150663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed416a310 con 0x7efec4004810 2023-10-06T05:25:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efed8b9b700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efed4155980 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7efec80073c0 tx=0x7efec8001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efed0ff9700 1 -- 172.21.15.31:0/453150663 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8018030 con 0x7efec4004810 2023-10-06T05:25:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efed0ff9700 1 -- 172.21.15.31:0/453150663 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec8001040 con 0x7efec4004810 2023-10-06T05:25:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efed0ff9700 1 -- 172.21.15.31:0/453150663 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec800fb20 con 0x7efec4004810 2023-10-06T05:25:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efedadff700 1 -- 172.21.15.31:0/453150663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed416a5f0 con 0x7efec4004810 2023-10-06T05:25:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.814+0000 7efedadff700 1 -- 172.21.15.31:0/453150663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed4169c80 con 0x7efec4004810 2023-10-06T05:25:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.815+0000 7efedadff700 1 -- 172.21.15.31:0/453150663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed416a810 con 0x7efec4004810 2023-10-06T05:25:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.816+0000 7efed0ff9700 1 -- 172.21.15.31:0/453150663 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7efec801b070 con 0x7efec4004810 2023-10-06T05:25:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.817+0000 7efed0ff9700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efeb8067830 0x7efeb8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.817+0000 7efed0ff9700 1 -- 172.21.15.31:0/453150663 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7efec8019070 con 0x7efec4004810 2023-10-06T05:25:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.818+0000 7efed3fff700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efeb8067830 0x7efeb8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.818+0000 7efed3fff700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efeb8067830 0x7efeb8069ce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efed4159110 tx=0x7efec0015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:46.821+0000 7efed0ff9700 1 -- 172.21.15.31:0/453150663 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efec80519b0 con 0x7efec4004810 2023-10-06T05:25:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.024+0000 7efedadff700 1 -- 172.21.15.31:0/453150663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efed404c380 con 0x7efec4004810 2023-10-06T05:25:47.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.024+0000 7efed0ff9700 1 -- 172.21.15.31:0/453150663 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v152) v1 ==== 93+0+10 (secure 0 0 0) 0x7efec8054fd0 con 0x7efec4004810 2023-10-06T05:25:47.027 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:25:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.026+0000 7efedadff700 1 -- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efeb8067830 msgr2=0x7efeb8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.026+0000 7efedadff700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efeb8067830 0x7efeb8069ce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efed4159110 tx=0x7efec0015000 comp rx=0 tx=0).stop 2023-10-06T05:25:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.027+0000 7efedadff700 1 -- 172.21.15.31:0/453150663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 msgr2=0x7efed4155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.027+0000 7efedadff700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efed4155980 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7efec80073c0 tx=0x7efec8001df0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.027+0000 7efedadff700 1 -- 172.21.15.31:0/453150663 shutdown_connections 2023-10-06T05:25:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.027+0000 7efedadff700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efeb8067830 0x7efeb8069ce0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.027+0000 7efedadff700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed4169810 0x7efed41585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.027+0000 7efedadff700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4004810 0x7efed4155980 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.027+0000 7efedadff700 1 --2- 172.21.15.31:0/453150663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efed40c9e80 0x7efed40c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.027+0000 7efedadff700 1 -- 172.21.15.31:0/453150663 >> 172.21.15.31:0/453150663 conn(0x7efed406ec90 msgr2=0x7efed41565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.027+0000 7efedadff700 1 -- 172.21.15.31:0/453150663 shutdown_connections 2023-10-06T05:25:47.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.027+0000 7efedadff700 1 -- 172.21.15.31:0/453150663 wait complete. 2023-10-06T05:25:47.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:25:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.244+0000 7f24559ff700 1 Processor -- start 2023-10-06T05:25:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.244+0000 7f24559ff700 1 -- start start 2023-10-06T05:25:47.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.245+0000 7f24559ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24500b0b90 0x7f24500af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f24559ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24500ad690 con 0x7f2450153d40 2023-10-06T05:25:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f24559ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24500acd60 con 0x7f24500af6c0 2023-10-06T05:25:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f24559ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24500ad040 con 0x7f24500b0b90 2023-10-06T05:25:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f244effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2450153d40 0x7f24500b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48602/0 (socket says 172.21.15.31:48602) 2023-10-06T05:25:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f244effd700 1 -- 172.21.15.31:0/4123515334 learned_addr learned my addr 172.21.15.31:0/4123515334 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f244e7fc700 1 --2- 172.21.15.31:0/4123515334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24500b0b90 0x7f24500af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f244dffb700 1 -- 172.21.15.31:0/4123515334 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208339787 0 0) 0x7f24500ad690 con 0x7f2450153d40 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f244dffb700 1 -- 172.21.15.31:0/4123515334 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2438003570 con 0x7f2450153d40 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f244dffb700 1 -- 172.21.15.31:0/4123515334 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765052774 0 0) 0x7f24500acd60 con 0x7f24500af6c0 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f244dffb700 1 -- 172.21.15.31:0/4123515334 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24500ad690 con 0x7f24500af6c0 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f244e7fc700 1 -- 172.21.15.31:0/4123515334 >> v1:172.21.15.62:6789/0 conn(0x7f24500af6c0 legacy=0x7f24500adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f244e7fc700 1 -- 172.21.15.31:0/4123515334 >> v1:172.21.15.31:6789/0 conn(0x7f2450153d40 legacy=0x7f24500b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.246+0000 7f244e7fc700 1 -- 172.21.15.31:0/4123515334 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24500ad320 con 0x7f24500b0b90 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.247+0000 7f244e7fc700 1 --2- 172.21.15.31:0/4123515334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24500b0b90 0x7f24500af180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2444009910 tx=0x7f24440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9a3d10a5d66f7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.247+0000 7f244dffb700 1 -- 172.21.15.31:0/4123515334 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444011630 con 0x7f24500b0b90 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.247+0000 7f244dffb700 1 -- 172.21.15.31:0/4123515334 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2444011790 con 0x7f24500b0b90 2023-10-06T05:25:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.247+0000 7f244dffb700 1 -- 172.21.15.31:0/4123515334 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444011b30 con 0x7f24500b0b90 2023-10-06T05:25:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.248+0000 7f24559ff700 1 -- 172.21.15.31:0/4123515334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24500b0b90 msgr2=0x7f24500af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.248+0000 7f24559ff700 1 --2- 172.21.15.31:0/4123515334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24500b0b90 0x7f24500af180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2444009910 tx=0x7f24440091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.248+0000 7f24559ff700 1 -- 172.21.15.31:0/4123515334 shutdown_connections 2023-10-06T05:25:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.248+0000 7f24559ff700 1 --2- 172.21.15.31:0/4123515334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24500b0b90 0x7f24500af180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.248+0000 7f24559ff700 1 -- 172.21.15.31:0/4123515334 >> 172.21.15.31:0/4123515334 conn(0x7f2450067560 msgr2=0x7f2450067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.248+0000 7f24559ff700 1 -- 172.21.15.31:0/4123515334 shutdown_connections 2023-10-06T05:25:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.248+0000 7f24559ff700 1 -- 172.21.15.31:0/4123515334 wait complete. 2023-10-06T05:25:47.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.249+0000 7f24559ff700 1 Processor -- start 2023-10-06T05:25:47.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.250+0000 7f24559ff700 1 -- start start 2023-10-06T05:25:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.250+0000 7f24559ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24500af6c0 0x7f24500a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.250+0000 7f24559ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24500b0b90 0x7f24501469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.250+0000 7f24559ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2450153d40 0x7f245015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f24559ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245016a7b0 con 0x7f24500af6c0 2023-10-06T05:25:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f24559ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245016a4d0 con 0x7f24500b0b90 2023-10-06T05:25:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f24559ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24500a55c0 con 0x7f2450153d40 2023-10-06T05:25:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24500af6c0 0x7f24500a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24500af6c0 0x7f24500a60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53500/0 (socket says 172.21.15.31:53500) 2023-10-06T05:25:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244effd700 1 -- 172.21.15.31:0/2580082905 learned_addr learned my addr 172.21.15.31:0/2580082905 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244e7fc700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24500b0b90 0x7f24501469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244f7fe700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2450153d40 0x7f245015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244effd700 1 -- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2450153d40 msgr2=0x7f245015c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244effd700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2450153d40 0x7f245015c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244effd700 1 -- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24500b0b90 msgr2=0x7f24501469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244effd700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24500b0b90 0x7f24501469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244effd700 1 -- 172.21.15.31:0/2580082905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2444009420 con 0x7f24500af6c0 2023-10-06T05:25:47.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.251+0000 7f244effd700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24500af6c0 0x7f24500a60a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f244000c790 tx=0x7f2440007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:47.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.252+0000 7f2436ffd700 1 -- 172.21.15.31:0/2580082905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440005b30 con 0x7f24500af6c0 2023-10-06T05:25:47.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.252+0000 7f2436ffd700 1 -- 172.21.15.31:0/2580082905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2440005c90 con 0x7f24500af6c0 2023-10-06T05:25:47.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.252+0000 7f24559ff700 1 -- 172.21.15.31:0/2580082905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24500a58a0 con 0x7f24500af6c0 2023-10-06T05:25:47.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.252+0000 7f2436ffd700 1 -- 172.21.15.31:0/2580082905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440007300 con 0x7f24500af6c0 2023-10-06T05:25:47.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.252+0000 7f24559ff700 1 -- 172.21.15.31:0/2580082905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24500a65e0 con 0x7f24500af6c0 2023-10-06T05:25:47.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.253+0000 7f24559ff700 1 -- 172.21.15.31:0/2580082905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245004c380 con 0x7f24500af6c0 2023-10-06T05:25:47.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.254+0000 7f2436ffd700 1 -- 172.21.15.31:0/2580082905 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f244001b400 con 0x7f24500af6c0 2023-10-06T05:25:47.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.255+0000 7f2436ffd700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f243c067830 0x7f243c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.255+0000 7f2436ffd700 1 -- 172.21.15.31:0/2580082905 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f24400829c0 con 0x7f24500af6c0 2023-10-06T05:25:47.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.255+0000 7f244e7fc700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f243c067830 0x7f243c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.256+0000 7f244e7fc700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f243c067830 0x7f243c069ce0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2444009860 tx=0x7f2444017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:47.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.259+0000 7f2436ffd700 1 -- 172.21.15.31:0/2580082905 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24400551c0 con 0x7f24500af6c0 2023-10-06T05:25:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.460+0000 7f24559ff700 1 -- 172.21.15.31:0/2580082905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f24500afed0 con 0x7f24500af6c0 2023-10-06T05:25:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.461+0000 7f2436ffd700 1 -- 172.21.15.31:0/2580082905 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v152) v1 ==== 104+0+11 (secure 0 0 0) 0x7f24400076f0 con 0x7f24500af6c0 2023-10-06T05:25:47.464 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:25:47.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 -- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f243c067830 msgr2=0x7f243c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f243c067830 0x7f243c069ce0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2444009860 tx=0x7f2444017000 comp rx=0 tx=0).stop 2023-10-06T05:25:47.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 -- 172.21.15.31:0/2580082905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24500af6c0 msgr2=0x7f24500a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24500af6c0 0x7f24500a60a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f244000c790 tx=0x7f2440007ea0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 -- 172.21.15.31:0/2580082905 shutdown_connections 2023-10-06T05:25:47.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f243c067830 0x7f243c069ce0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2450153d40 0x7f245015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24500af6c0 0x7f24500a60a0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 --2- 172.21.15.31:0/2580082905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24500b0b90 0x7f24501469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 -- 172.21.15.31:0/2580082905 >> 172.21.15.31:0/2580082905 conn(0x7f2450067560 msgr2=0x7f24500c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 -- 172.21.15.31:0/2580082905 shutdown_connections 2023-10-06T05:25:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.463+0000 7f24559ff700 1 -- 172.21.15.31:0/2580082905 wait complete. 2023-10-06T05:25:47.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:25:47.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.679+0000 7fa1331ff700 1 Processor -- start 2023-10-06T05:25:47.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.679+0000 7fa1331ff700 1 -- start start 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.680+0000 7fa1331ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c159890 0x7fa12c0c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.680+0000 7fa1331ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa12c159c60 0x7fa12c156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.680+0000 7fa1331ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12c158190 con 0x7fa12c156fa0 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.680+0000 7fa1331ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12c157580 con 0x7fa12c159c60 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.680+0000 7fa1331ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c157860 con 0x7fa12c159890 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.680+0000 7fa12bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa12c159c60 0x7fa12c156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.680+0000 7fa12bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa12c159c60 0x7fa12c156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53508/0 (socket says 172.21.15.31:53508) 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.680+0000 7fa12bfff700 1 -- 172.21.15.31:0/1835684754 learned_addr learned my addr 172.21.15.31:0/1835684754 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.681+0000 7fa130f9b700 1 --2- 172.21.15.31:0/1835684754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c159890 0x7fa12c0c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.681+0000 7fa12bfff700 1 -- 172.21.15.31:0/1835684754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c159890 msgr2=0x7fa12c0c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.681+0000 7fa12bfff700 1 --2- 172.21.15.31:0/1835684754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c159890 0x7fa12c0c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.681+0000 7fa12bfff700 1 -- 172.21.15.31:0/1835684754 >> v1:172.21.15.31:6789/0 conn(0x7fa12c156fa0 legacy=0x7fa12c155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.681+0000 7fa12bfff700 1 -- 172.21.15.31:0/1835684754 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c157b40 con 0x7fa12c159c60 2023-10-06T05:25:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.681+0000 7fa12b7fe700 1 -- 172.21.15.31:0/1835684754 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002968230 0 0) 0x7fa12c158190 con 0x7fa12c156fa0 2023-10-06T05:25:47.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.681+0000 7fa12bfff700 1 --2- 172.21.15.31:0/1835684754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 0x7fa12c156a60 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fa1180099e0 tx=0x7fa118009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cc79a88560839 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:47.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.681+0000 7fa12b7fe700 1 -- 172.21.15.31:0/1835684754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800b820 con 0x7fa12c159c60 2023-10-06T05:25:47.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.681+0000 7fa12b7fe700 1 -- 172.21.15.31:0/1835684754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11800b980 con 0x7fa12c159c60 2023-10-06T05:25:47.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.681+0000 7fa12b7fe700 1 -- 172.21.15.31:0/1835684754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800bd20 con 0x7fa12c159c60 2023-10-06T05:25:47.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.682+0000 7fa1331ff700 1 -- 172.21.15.31:0/1835684754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 msgr2=0x7fa12c156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.682+0000 7fa1331ff700 1 --2- 172.21.15.31:0/1835684754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 0x7fa12c156a60 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fa1180099e0 tx=0x7fa118009250 comp rx=0 tx=0).stop 2023-10-06T05:25:47.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.682+0000 7fa1331ff700 1 -- 172.21.15.31:0/1835684754 shutdown_connections 2023-10-06T05:25:47.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.682+0000 7fa1331ff700 1 --2- 172.21.15.31:0/1835684754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c159890 0x7fa12c0c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.682+0000 7fa1331ff700 1 --2- 172.21.15.31:0/1835684754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 0x7fa12c156a60 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.682+0000 7fa1331ff700 1 -- 172.21.15.31:0/1835684754 >> 172.21.15.31:0/1835684754 conn(0x7fa12c067560 msgr2=0x7fa12c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:47.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.682+0000 7fa1331ff700 1 -- 172.21.15.31:0/1835684754 shutdown_connections 2023-10-06T05:25:47.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.682+0000 7fa1331ff700 1 -- 172.21.15.31:0/1835684754 wait complete. 2023-10-06T05:25:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.685+0000 7fa1331ff700 1 Processor -- start 2023-10-06T05:25:47.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.685+0000 7fa1331ff700 1 -- start start 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa1331ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c156fa0 0x7fa12c14a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa1331ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12c159890 0x7fa12c0c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa1331ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 0x7fa12c14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa1331ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c14c0e0 con 0x7fa12c159c60 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa1331ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0c5630 con 0x7fa12c159890 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa1331ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0c52e0 con 0x7fa12c156fa0 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa13179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 0x7fa12c14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa130f9b700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c156fa0 0x7fa12c14a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa13179c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 0x7fa12c14d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53510/0 (socket says 172.21.15.31:53510) 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa13179c700 1 -- 172.21.15.31:0/1466778082 learned_addr learned my addr 172.21.15.31:0/1466778082 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa12bfff700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12c159890 0x7fa12c0c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa13179c700 1 -- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c156fa0 msgr2=0x7fa12c14a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa13179c700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c156fa0 0x7fa12c14a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa13179c700 1 -- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12c159890 msgr2=0x7fa12c0c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa13179c700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12c159890 0x7fa12c0c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa13179c700 1 -- 172.21.15.31:0/1466778082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180094f0 con 0x7fa12c159c60 2023-10-06T05:25:47.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa13179c700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 0x7fa12c14d810 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fa12400c910 tx=0x7fa1240016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:47.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa128ff9700 1 -- 172.21.15.31:0/1466778082 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124012630 con 0x7fa12c159c60 2023-10-06T05:25:47.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa128ff9700 1 -- 172.21.15.31:0/1466778082 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1240127b0 con 0x7fa12c159c60 2023-10-06T05:25:47.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa128ff9700 1 -- 172.21.15.31:0/1466778082 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124012b70 con 0x7fa12c159c60 2023-10-06T05:25:47.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa1331ff700 1 -- 172.21.15.31:0/1466778082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c169950 con 0x7fa12c159c60 2023-10-06T05:25:47.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.686+0000 7fa1331ff700 1 -- 172.21.15.31:0/1466778082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c14c610 con 0x7fa12c159c60 2023-10-06T05:25:47.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.687+0000 7fa1331ff700 1 -- 172.21.15.31:0/1466778082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12c0c57c0 con 0x7fa12c159c60 2023-10-06T05:25:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.688+0000 7fa128ff9700 1 -- 172.21.15.31:0/1466778082 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fa124003030 con 0x7fa12c159c60 2023-10-06T05:25:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.689+0000 7fa128ff9700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa110067830 0x7fa110069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.689+0000 7fa128ff9700 1 -- 172.21.15.31:0/1466778082 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fa124082e50 con 0x7fa12c159c60 2023-10-06T05:25:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.691+0000 7fa130f9b700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa110067830 0x7fa110069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.692+0000 7fa130f9b700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa110067830 0x7fa110069ce0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa120006fd0 tx=0x7fa120008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.694+0000 7fa128ff9700 1 -- 172.21.15.31:0/1466778082 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa124051a90 con 0x7fa12c159c60 2023-10-06T05:25:47.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.894+0000 7fa1331ff700 1 -- 172.21.15.31:0/1466778082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa12c04c380 con 0x7fa12c159c60 2023-10-06T05:25:47.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.894+0000 7fa128ff9700 1 -- 172.21.15.31:0/1466778082 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v152) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa1240550b0 con 0x7fa12c159c60 2023-10-06T05:25:47.898 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:25:47.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.896+0000 7fa1331ff700 1 -- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa110067830 msgr2=0x7fa110069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.896+0000 7fa1331ff700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa110067830 0x7fa110069ce0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa120006fd0 tx=0x7fa120008040 comp rx=0 tx=0).stop 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.896+0000 7fa1331ff700 1 -- 172.21.15.31:0/1466778082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 msgr2=0x7fa12c14d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.896+0000 7fa1331ff700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 0x7fa12c14d810 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fa12400c910 tx=0x7fa1240016c0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.897+0000 7fa1331ff700 1 -- 172.21.15.31:0/1466778082 shutdown_connections 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.897+0000 7fa1331ff700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fa110067830 0x7fa110069ce0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.897+0000 7fa1331ff700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12c159c60 0x7fa12c14d810 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.897+0000 7fa1331ff700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c156fa0 0x7fa12c14a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.897+0000 7fa1331ff700 1 --2- 172.21.15.31:0/1466778082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12c159890 0x7fa12c0c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.897+0000 7fa1331ff700 1 -- 172.21.15.31:0/1466778082 >> 172.21.15.31:0/1466778082 conn(0x7fa12c067560 msgr2=0x7fa12c06d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.897+0000 7fa1331ff700 1 -- 172.21.15.31:0/1466778082 shutdown_connections 2023-10-06T05:25:47.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:47.897+0000 7fa1331ff700 1 -- 172.21.15.31:0/1466778082 wait complete. 2023-10-06T05:25:47.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:48.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.110+0000 7fda58bff700 1 Processor -- start 2023-10-06T05:25:48.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.110+0000 7fda58bff700 1 -- start start 2023-10-06T05:25:48.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.110+0000 7fda58bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda54159500 0x7fda540b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.110+0000 7fda58bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda540c3f40 con 0x7fda540cd510 2023-10-06T05:25:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.110+0000 7fda58bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda540cb000 con 0x7fda5415d2d0 2023-10-06T05:25:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.110+0000 7fda58bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda540b2860 con 0x7fda54159500 2023-10-06T05:25:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.111+0000 7fda5259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda54159500 0x7fda540b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.111+0000 7fda5259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda54159500 0x7fda540b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53516/0 (socket says 172.21.15.31:53516) 2023-10-06T05:25:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.111+0000 7fda5259c700 1 -- 172.21.15.31:0/2046693442 learned_addr learned my addr 172.21.15.31:0/2046693442 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.111+0000 7fda52d9d700 1 --1- 172.21.15.31:0/2046693442 >> v1:172.21.15.31:6789/0 conn(0x7fda540cd510 0x7fda540aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48614/0 (socket says 172.21.15.31:48614) 2023-10-06T05:25:48.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.111+0000 7fda5259c700 1 -- 172.21.15.31:0/2046693442 >> v1:172.21.15.62:6789/0 conn(0x7fda5415d2d0 legacy=0x7fda540cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.111+0000 7fda5259c700 1 -- 172.21.15.31:0/2046693442 >> v1:172.21.15.31:6789/0 conn(0x7fda540cd510 legacy=0x7fda540aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.111+0000 7fda5259c700 1 -- 172.21.15.31:0/2046693442 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5415d7d0 con 0x7fda54159500 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.112+0000 7fda5259c700 1 --2- 172.21.15.31:0/2046693442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda54159500 0x7fda540b0ab0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fda440099b0 tx=0x7fda44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cd5f4fc5259045 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.112+0000 7fda5159a700 1 -- 172.21.15.31:0/2046693442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4400b860 con 0x7fda54159500 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.112+0000 7fda5159a700 1 -- 172.21.15.31:0/2046693442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4400b9c0 con 0x7fda54159500 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.112+0000 7fda5159a700 1 -- 172.21.15.31:0/2046693442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4400bd60 con 0x7fda54159500 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.112+0000 7fda58bff700 1 -- 172.21.15.31:0/2046693442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda54159500 msgr2=0x7fda540b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.112+0000 7fda58bff700 1 --2- 172.21.15.31:0/2046693442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda54159500 0x7fda540b0ab0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fda440099b0 tx=0x7fda44009250 comp rx=0 tx=0).stop 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.112+0000 7fda58bff700 1 -- 172.21.15.31:0/2046693442 shutdown_connections 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.112+0000 7fda58bff700 1 --2- 172.21.15.31:0/2046693442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda54159500 0x7fda540b0ab0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.112+0000 7fda58bff700 1 -- 172.21.15.31:0/2046693442 >> 172.21.15.31:0/2046693442 conn(0x7fda5406ec90 msgr2=0x7fda5406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.113+0000 7fda58bff700 1 -- 172.21.15.31:0/2046693442 shutdown_connections 2023-10-06T05:25:48.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.113+0000 7fda58bff700 1 -- 172.21.15.31:0/2046693442 wait complete. 2023-10-06T05:25:48.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.114+0000 7fda58bff700 1 Processor -- start 2023-10-06T05:25:48.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.114+0000 7fda58bff700 1 -- start start 2023-10-06T05:25:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.115+0000 7fda58bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda540cd510 0x7fda54155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.115+0000 7fda58bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda54159500 0x7fda540b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.115+0000 7fda58bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda5415d2d0 0x7fda540c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.115+0000 7fda58bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda541512c0 con 0x7fda540cd510 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.115+0000 7fda58bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda540b4e30 con 0x7fda54159500 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.115+0000 7fda58bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda540b5090 con 0x7fda5415d2d0 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.115+0000 7fda5259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda540cd510 0x7fda54155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.115+0000 7fda5259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda540cd510 0x7fda54155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53530/0 (socket says 172.21.15.31:53530) 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.115+0000 7fda5259c700 1 -- 172.21.15.31:0/3028071323 learned_addr learned my addr 172.21.15.31:0/3028071323 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda51d9b700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda54159500 0x7fda540b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda52d9d700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda5415d2d0 0x7fda540c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda5259c700 1 -- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda5415d2d0 msgr2=0x7fda540c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda5259c700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda5415d2d0 0x7fda540c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda5259c700 1 -- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda54159500 msgr2=0x7fda540b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda5259c700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda54159500 0x7fda540b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda5259c700 1 -- 172.21.15.31:0/3028071323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440094f0 con 0x7fda540cd510 2023-10-06T05:25:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda5259c700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda540cd510 0x7fda54155550 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fda54067c50 tx=0x7fda44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda427fc700 1 -- 172.21.15.31:0/3028071323 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44011ea0 con 0x7fda540cd510 2023-10-06T05:25:48.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda427fc700 1 -- 172.21.15.31:0/3028071323 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4401a630 con 0x7fda540cd510 2023-10-06T05:25:48.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda427fc700 1 -- 172.21.15.31:0/3028071323 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4401a9d0 con 0x7fda540cd510 2023-10-06T05:25:48.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.116+0000 7fda58bff700 1 -- 172.21.15.31:0/3028071323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda540b5370 con 0x7fda540cd510 2023-10-06T05:25:48.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.117+0000 7fda58bff700 1 -- 172.21.15.31:0/3028071323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda540b5930 con 0x7fda540cd510 2023-10-06T05:25:48.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.118+0000 7fda58bff700 1 -- 172.21.15.31:0/3028071323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda540b5ac0 con 0x7fda540cd510 2023-10-06T05:25:48.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.118+0000 7fda427fc700 1 -- 172.21.15.31:0/3028071323 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fda4402d440 con 0x7fda540cd510 2023-10-06T05:25:48.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.119+0000 7fda427fc700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fda34067830 0x7fda34069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.120+0000 7fda427fc700 1 -- 172.21.15.31:0/3028071323 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fda44084a00 con 0x7fda540cd510 2023-10-06T05:25:48.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.120+0000 7fda51d9b700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fda34067830 0x7fda34069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.122+0000 7fda51d9b700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fda34067830 0x7fda34069ce0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda540b4560 tx=0x7fda48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.124+0000 7fda427fc700 1 -- 172.21.15.31:0/3028071323 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda44053640 con 0x7fda540cd510 2023-10-06T05:25:48.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.372+0000 7fda58bff700 1 -- 172.21.15.31:0/3028071323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda5404c380 con 0x7fda540cd510 2023-10-06T05:25:48.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.372+0000 7fda427fc700 1 -- 172.21.15.31:0/3028071323 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4600 (secure 0 0 0) 0x7fda44056c60 con 0x7fda540cd510 2023-10-06T05:25:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:48.374 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1489945066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1489945066},{"type":"v1","addr":"172.21.15.62:6834","nonce":1489945066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/660206899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":660206899},{"type":"v1","addr":"172.21.15.62:6835","nonce":660206899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8862,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1056123914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1056123914},{"type":"v1","addr":"172.21.15.31:6834","nonce":1056123914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:24.746699+0000","modified":"2023-10-06T05:25:28.266401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8865},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8865":{"gid":8865,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1535525741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1535525741},{"type":"v1","addr":"172.21.15.31:6835","nonce":1535525741}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:25:48.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.374+0000 7fda58bff700 1 -- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fda34067830 msgr2=0x7fda34069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.374+0000 7fda58bff700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fda34067830 0x7fda34069ce0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda540b4560 tx=0x7fda48009040 comp rx=0 tx=0).stop 2023-10-06T05:25:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.374+0000 7fda58bff700 1 -- 172.21.15.31:0/3028071323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda540cd510 msgr2=0x7fda54155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.374+0000 7fda58bff700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda540cd510 0x7fda54155550 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fda54067c50 tx=0x7fda44017000 comp rx=0 tx=0).stop 2023-10-06T05:25:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.374+0000 7fda58bff700 1 -- 172.21.15.31:0/3028071323 shutdown_connections 2023-10-06T05:25:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.374+0000 7fda58bff700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fda34067830 0x7fda34069ce0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.374+0000 7fda58bff700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda5415d2d0 0x7fda540c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.374+0000 7fda58bff700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda540cd510 0x7fda54155550 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.374+0000 7fda58bff700 1 --2- 172.21.15.31:0/3028071323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda54159500 0x7fda540b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.374+0000 7fda58bff700 1 -- 172.21.15.31:0/3028071323 >> 172.21.15.31:0/3028071323 conn(0x7fda5406ec90 msgr2=0x7fda540aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:48.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.375+0000 7fda58bff700 1 -- 172.21.15.31:0/3028071323 shutdown_connections 2023-10-06T05:25:48.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.375+0000 7fda58bff700 1 -- 172.21.15.31:0/3028071323 wait complete. 2023-10-06T05:25:48.378 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 103 2023-10-06T05:25:48.394 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:48.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.588+0000 7fe34f1ff700 1 Processor -- start 2023-10-06T05:25:48.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.588+0000 7fe34f1ff700 1 -- start start 2023-10-06T05:25:48.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.589+0000 7fe34f1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348159500 0x7fe3480b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.589+0000 7fe34f1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3480c3f40 con 0x7fe3480cd510 2023-10-06T05:25:48.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.590+0000 7fe34f1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3480cb000 con 0x7fe34815d2d0 2023-10-06T05:25:48.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.590+0000 7fe34f1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3480b2860 con 0x7fe348159500 2023-10-06T05:25:48.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.590+0000 7fe34d79c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe3480cd510 0x7fe3480aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48622/0 (socket says 172.21.15.31:48622) 2023-10-06T05:25:48.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.590+0000 7fe34d79c700 1 -- 172.21.15.31:0/2143906643 learned_addr learned my addr 172.21.15.31:0/2143906643 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:48.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.590+0000 7fe34cf9b700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348159500 0x7fe3480b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.590+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097426184 0 0) 0x7fe3480c3f40 con 0x7fe3480cd510 2023-10-06T05:25:48.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.590+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe338003570 con 0x7fe3480cd510 2023-10-06T05:25:48.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.590+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322490731 0 0) 0x7fe3480cb000 con 0x7fe34815d2d0 2023-10-06T05:25:48.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.590+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3480c3f40 con 0x7fe34815d2d0 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4157523197 0 0) 0x7fe338003570 con 0x7fe3480cd510 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348159500 msgr2=0x7fe3480b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348159500 0x7fe3480b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 >> v1:172.21.15.62:6789/0 conn(0x7fe34815d2d0 legacy=0x7fe3480cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34815d7d0 con 0x7fe3480cd510 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7fe3400017f0 con 0x7fe3480cd510 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 >> v1:172.21.15.31:6789/0 conn(0x7fe3480cd510 legacy=0x7fe3480aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe338003c20 0x7fe338004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348159500 0x7fe338004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338004d50 0x7fe3380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3480b2860 con 0x7fe338004d50 2023-10-06T05:25:48.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33800bbf0 con 0x7fe338003c20 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33800bd60 con 0x7fe348159500 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe34d79c700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338004d50 0x7fe3380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe34cf9b700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe338003c20 0x7fe338004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe347fff700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348159500 0x7fe338004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe34d79c700 1 -- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348159500 msgr2=0x7fe338004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe34d79c700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348159500 0x7fe338004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe34d79c700 1 -- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe338003c20 msgr2=0x7fe338004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe34d79c700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe338003c20 0x7fe338004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.591+0000 7fe34d79c700 1 -- 172.21.15.31:0/2143906643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe338003570 con 0x7fe338004d50 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.592+0000 7fe34d79c700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338004d50 0x7fe3380051c0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe3480b3030 tx=0x7fe3400020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf39e47de6ab57c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.592+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3400070d0 con 0x7fe338004d50 2023-10-06T05:25:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.592+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe340007250 con 0x7fe338004d50 2023-10-06T05:25:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.592+0000 7fe3477fe700 1 -- 172.21.15.31:0/2143906643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340007610 con 0x7fe338004d50 2023-10-06T05:25:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.592+0000 7fe34f1ff700 1 -- 172.21.15.31:0/2143906643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338004d50 msgr2=0x7fe3380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.592+0000 7fe34f1ff700 1 --2- 172.21.15.31:0/2143906643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338004d50 0x7fe3380051c0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe3480b3030 tx=0x7fe3400020e0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.592+0000 7fe34d79c700 1 -- 172.21.15.31:0/2143906643 reap_dead start 2023-10-06T05:25:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.592+0000 7fe34f1ff700 1 -- 172.21.15.31:0/2143906643 shutdown_connections 2023-10-06T05:25:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.592+0000 7fe34f1ff700 1 -- 172.21.15.31:0/2143906643 >> 172.21.15.31:0/2143906643 conn(0x7fe34806ec90 msgr2=0x7fe34806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.593+0000 7fe34f1ff700 1 -- 172.21.15.31:0/2143906643 shutdown_connections 2023-10-06T05:25:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.593+0000 7fe34f1ff700 1 -- 172.21.15.31:0/2143906643 wait complete. 2023-10-06T05:25:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.595+0000 7fe34f1ff700 1 Processor -- start 2023-10-06T05:25:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.595+0000 7fe34f1ff700 1 -- start start 2023-10-06T05:25:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.596+0000 7fe34f1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe338004d50 0x7fe34816ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.596+0000 7fe34f1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34815c7e0 0x7fe3480c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.596+0000 7fe34f1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348157780 0x7fe348157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.596+0000 7fe34f1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34815cfe0 con 0x7fe34815c7e0 2023-10-06T05:25:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.596+0000 7fe34f1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe348158130 con 0x7fe338004d50 2023-10-06T05:25:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.596+0000 7fe34f1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3480cff40 con 0x7fe348157780 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.596+0000 7fe347fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34815c7e0 0x7fe3480c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.596+0000 7fe347fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34815c7e0 0x7fe3480c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53558/0 (socket says 172.21.15.31:53558) 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.596+0000 7fe347fff700 1 -- 172.21.15.31:0/3163825624 learned_addr learned my addr 172.21.15.31:0/3163825624 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.596+0000 7fe34cf9b700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe338004d50 0x7fe34816ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe34d79c700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348157780 0x7fe348157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe347fff700 1 -- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348157780 msgr2=0x7fe348157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe347fff700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348157780 0x7fe348157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe347fff700 1 -- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe338004d50 msgr2=0x7fe34816ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe347fff700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe338004d50 0x7fe34816ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe347fff700 1 -- 172.21.15.31:0/3163825624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3480d0220 con 0x7fe34815c7e0 2023-10-06T05:25:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe34d79c700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348157780 0x7fe348157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe347fff700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34815c7e0 0x7fe3480c8dc0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe33400e7b0 tx=0x7fe334009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe344ff9700 1 -- 172.21.15.31:0/3163825624 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334006870 con 0x7fe34815c7e0 2023-10-06T05:25:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe344ff9700 1 -- 172.21.15.31:0/3163825624 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe334010040 con 0x7fe34815c7e0 2023-10-06T05:25:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe344ff9700 1 -- 172.21.15.31:0/3163825624 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334006c20 con 0x7fe34815c7e0 2023-10-06T05:25:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe34f1ff700 1 -- 172.21.15.31:0/3163825624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3480d0500 con 0x7fe34815c7e0 2023-10-06T05:25:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.597+0000 7fe34f1ff700 1 -- 172.21.15.31:0/3163825624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3480d0d00 con 0x7fe34815c7e0 2023-10-06T05:25:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.598+0000 7fe34f1ff700 1 -- 172.21.15.31:0/3163825624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34804c380 con 0x7fe34815c7e0 2023-10-06T05:25:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.599+0000 7fe344ff9700 1 -- 172.21.15.31:0/3163825624 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fe334007050 con 0x7fe34815c7e0 2023-10-06T05:25:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.601+0000 7fe344ff9700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe32c067830 0x7fe32c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.601+0000 7fe344ff9700 1 -- 172.21.15.31:0/3163825624 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fe334085870 con 0x7fe34815c7e0 2023-10-06T05:25:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.601+0000 7fe34cf9b700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe32c067830 0x7fe32c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.602+0000 7fe34cf9b700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe32c067830 0x7fe32c069ce0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe33c006f00 tx=0x7fe33c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.605+0000 7fe344ff9700 1 -- 172.21.15.31:0/3163825624 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe334051970 con 0x7fe34815c7e0 2023-10-06T05:25:48.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.808+0000 7fe34f1ff700 1 -- 172.21.15.31:0/3163825624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3480cade0 con 0x7fe34815c7e0 2023-10-06T05:25:48.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.809+0000 7fe344ff9700 1 -- 172.21.15.31:0/3163825624 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+17164 (secure 0 0 0) 0x7fe334054f90 con 0x7fe34815c7e0 2023-10-06T05:25:48.811 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:48.812 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":152,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:25:38.128777+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:25:22.964655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:23.719177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.3","mappings":[{"from":1,"to":6}]},{"pgid":"26.20","mappings":[{"from":3,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:48.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 -- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe32c067830 msgr2=0x7fe32c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe32c067830 0x7fe32c069ce0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe33c006f00 tx=0x7fe33c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 -- 172.21.15.31:0/3163825624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34815c7e0 msgr2=0x7fe3480c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:48.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34815c7e0 0x7fe3480c8dc0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe33400e7b0 tx=0x7fe334009040 comp rx=0 tx=0).stop 2023-10-06T05:25:48.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 -- 172.21.15.31:0/3163825624 shutdown_connections 2023-10-06T05:25:48.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fe32c067830 0x7fe32c069ce0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe348157780 0x7fe348157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe338004d50 0x7fe34816ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 --2- 172.21.15.31:0/3163825624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe34815c7e0 0x7fe3480c8dc0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:48.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 -- 172.21.15.31:0/3163825624 >> 172.21.15.31:0/3163825624 conn(0x7fe34806ec90 msgr2=0x7fe348158990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:48.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 -- 172.21.15.31:0/3163825624 shutdown_connections 2023-10-06T05:25:48.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:48.812+0000 7fe34f1ff700 1 -- 172.21.15.31:0/3163825624 wait complete. 2023-10-06T05:25:48.831 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:25:48.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.025+0000 7f42a47ff700 1 Processor -- start 2023-10-06T05:25:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.025+0000 7f42a47ff700 1 -- start start 2023-10-06T05:25:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.026+0000 7f42a47ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c0ca770 0x7f429c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.026+0000 7f42a47ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f429c06b510 0x7f429c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.026+0000 7f42a47ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429c156ef0 con 0x7f429c0cab40 2023-10-06T05:25:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.026+0000 7f42a47ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f429c0c7a70 con 0x7f429c06b510 2023-10-06T05:25:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.026+0000 7f42a47ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0c6df0 con 0x7f429c0ca770 2023-10-06T05:25:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a1d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f429c0cab40 0x7f429c06ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48624/0 (socket says 172.21.15.31:48624) 2023-10-06T05:25:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a1d9a700 1 -- 172.21.15.31:0/2315901884 learned_addr learned my addr 172.21.15.31:0/2315901884 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a2d9c700 1 --2- 172.21.15.31:0/2315901884 >> v2:172.21.15.62:3300/0 conn(0x7f429c06b510 0x7f429c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a259b700 1 --2- 172.21.15.31:0/2315901884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c0ca770 0x7f429c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a1599700 1 -- 172.21.15.31:0/2315901884 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667467780 0 0) 0x7f429c156ef0 con 0x7f429c0cab40 2023-10-06T05:25:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a1599700 1 -- 172.21.15.31:0/2315901884 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4288003570 con 0x7f429c0cab40 2023-10-06T05:25:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a2d9c700 1 -- 172.21.15.31:0/2315901884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c0ca770 msgr2=0x7f429c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a2d9c700 1 --2- 172.21.15.31:0/2315901884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c0ca770 0x7f429c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a2d9c700 1 -- 172.21.15.31:0/2315901884 >> v1:172.21.15.31:6789/0 conn(0x7f429c0cab40 legacy=0x7f429c06ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a2d9c700 1 -- 172.21.15.31:0/2315901884 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c0c70d0 con 0x7f429c06b510 2023-10-06T05:25:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.027+0000 7f42a259b700 1 --2- 172.21.15.31:0/2315901884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c0ca770 0x7f429c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.028+0000 7f42a2d9c700 1 --2- 172.21.15.31:0/2315901884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f429c06b510 0x7f429c1569b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4294009e60 tx=0x7f4294009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc76fb1adce63550 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.028+0000 7f42a1599700 1 -- 172.21.15.31:0/2315901884 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400e070 con 0x7f429c06b510 2023-10-06T05:25:49.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.028+0000 7f42a1599700 1 -- 172.21.15.31:0/2315901884 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429400bab0 con 0x7f429c06b510 2023-10-06T05:25:49.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.028+0000 7f42a1599700 1 -- 172.21.15.31:0/2315901884 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400be70 con 0x7f429c06b510 2023-10-06T05:25:49.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.029+0000 7f42a47ff700 1 -- 172.21.15.31:0/2315901884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f429c06b510 msgr2=0x7f429c1569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.029+0000 7f42a47ff700 1 --2- 172.21.15.31:0/2315901884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f429c06b510 0x7f429c1569b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4294009e60 tx=0x7f4294009780 comp rx=0 tx=0).stop 2023-10-06T05:25:49.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.029+0000 7f42a47ff700 1 -- 172.21.15.31:0/2315901884 shutdown_connections 2023-10-06T05:25:49.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.029+0000 7f42a47ff700 1 --2- 172.21.15.31:0/2315901884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f429c06b510 0x7f429c1569b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.029+0000 7f42a47ff700 1 --2- 172.21.15.31:0/2315901884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c0ca770 0x7f429c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.029+0000 7f42a47ff700 1 -- 172.21.15.31:0/2315901884 >> 172.21.15.31:0/2315901884 conn(0x7f429c0cc0b0 msgr2=0x7f429c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:49.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.029+0000 7f42a47ff700 1 -- 172.21.15.31:0/2315901884 shutdown_connections 2023-10-06T05:25:49.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.029+0000 7f42a47ff700 1 -- 172.21.15.31:0/2315901884 wait complete. 2023-10-06T05:25:49.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.030+0000 7f42a47ff700 1 Processor -- start 2023-10-06T05:25:49.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.031+0000 7f42a47ff700 1 -- start start 2023-10-06T05:25:49.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.031+0000 7f42a47ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c06b510 0x7f429c14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.031+0000 7f42a47ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429c0ca770 0x7f429c0c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.031+0000 7f42a47ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f429c0cab40 0x7f429c13fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.031+0000 7f42a47ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0c5820 con 0x7f429c0ca770 2023-10-06T05:25:49.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.031+0000 7f42a47ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c1400c0 con 0x7f429c0cab40 2023-10-06T05:25:49.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.031+0000 7f42a47ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0afa60 con 0x7f429c06b510 2023-10-06T05:25:49.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.032+0000 7f42a1d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429c0ca770 0x7f429c0c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.032+0000 7f42a1d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429c0ca770 0x7f429c0c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53570/0 (socket says 172.21.15.31:53570) 2023-10-06T05:25:49.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.032+0000 7f42a1d9a700 1 -- 172.21.15.31:0/3598863157 learned_addr learned my addr 172.21.15.31:0/3598863157 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:49.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.032+0000 7f42a2d9c700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f429c0cab40 0x7f429c13fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.032+0000 7f42a259b700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c06b510 0x7f429c14ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.032+0000 7f42a1d9a700 1 -- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c06b510 msgr2=0x7f429c14ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.032+0000 7f42a1d9a700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c06b510 0x7f429c14ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.032+0000 7f42a1d9a700 1 -- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f429c0cab40 msgr2=0x7f429c13fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.032+0000 7f42a1d9a700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f429c0cab40 0x7f429c13fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.032+0000 7f42a1d9a700 1 -- 172.21.15.31:0/3598863157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42980093c0 con 0x7f429c0ca770 2023-10-06T05:25:49.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.033+0000 7f42a1d9a700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429c0ca770 0x7f429c0c5d20 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c7f0 tx=0x7f428c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.033+0000 7f42927fc700 1 -- 172.21.15.31:0/3598863157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c005cd0 con 0x7f429c0ca770 2023-10-06T05:25:49.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.033+0000 7f42927fc700 1 -- 172.21.15.31:0/3598863157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c005e50 con 0x7f429c0ca770 2023-10-06T05:25:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.033+0000 7f42927fc700 1 -- 172.21.15.31:0/3598863157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c007320 con 0x7f429c0ca770 2023-10-06T05:25:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.033+0000 7f42a47ff700 1 -- 172.21.15.31:0/3598863157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4294009a20 con 0x7f429c0ca770 2023-10-06T05:25:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.033+0000 7f42a47ff700 1 -- 172.21.15.31:0/3598863157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429c0af130 con 0x7f429c0ca770 2023-10-06T05:25:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.034+0000 7f42a47ff700 1 -- 172.21.15.31:0/3598863157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429c0afc10 con 0x7f429c0ca770 2023-10-06T05:25:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.035+0000 7f42927fc700 1 -- 172.21.15.31:0/3598863157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f428c007690 con 0x7f429c0ca770 2023-10-06T05:25:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.036+0000 7f42927fc700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4280067830 0x7f4280069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.036+0000 7f42927fc700 1 -- 172.21.15.31:0/3598863157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f428c0824a0 con 0x7f429c0ca770 2023-10-06T05:25:49.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.036+0000 7f42a259b700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4280067830 0x7f4280069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.037+0000 7f42a259b700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4280067830 0x7f4280069ce0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4298009f30 tx=0x7f4298009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.040+0000 7f42927fc700 1 -- 172.21.15.31:0/3598863157 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f429c0afc10 con 0x7f429c0ca770 2023-10-06T05:25:49.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.294+0000 7f42a47ff700 1 -- 172.21.15.31:0/3598863157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f429c0afbd0 con 0x7f429c0ca770 2023-10-06T05:25:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.295+0000 7f42927fc700 1 -- 172.21.15.31:0/3598863157 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4600 (secure 0 0 0) 0x7f428c017020 con 0x7f429c0ca770 2023-10-06T05:25:49.297 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:49.297 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1489945066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1489945066},{"type":"v1","addr":"172.21.15.62:6834","nonce":1489945066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/660206899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":660206899},{"type":"v1","addr":"172.21.15.62:6835","nonce":660206899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8862,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1056123914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1056123914},{"type":"v1","addr":"172.21.15.31:6834","nonce":1056123914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:24.746699+0000","modified":"2023-10-06T05:25:28.266401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8865},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8865":{"gid":8865,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1535525741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1535525741},{"type":"v1","addr":"172.21.15.31:6835","nonce":1535525741}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:25:49.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 -- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4280067830 msgr2=0x7f4280069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4280067830 0x7f4280069ce0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4298009f30 tx=0x7f4298009120 comp rx=0 tx=0).stop 2023-10-06T05:25:49.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 -- 172.21.15.31:0/3598863157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429c0ca770 msgr2=0x7f429c0c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429c0ca770 0x7f429c0c5d20 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c7f0 tx=0x7f428c007e00 comp rx=0 tx=0).stop 2023-10-06T05:25:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 -- 172.21.15.31:0/3598863157 shutdown_connections 2023-10-06T05:25:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4280067830 0x7f4280069ce0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f429c0cab40 0x7f429c13fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f429c06b510 0x7f429c14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 --2- 172.21.15.31:0/3598863157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f429c0ca770 0x7f429c0c5d20 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 -- 172.21.15.31:0/3598863157 >> 172.21.15.31:0/3598863157 conn(0x7f429c0cc0b0 msgr2=0x7f429c06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 -- 172.21.15.31:0/3598863157 shutdown_connections 2023-10-06T05:25:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.297+0000 7f42a47ff700 1 -- 172.21.15.31:0/3598863157 wait complete. 2023-10-06T05:25:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 103 2023-10-06T05:25:49.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.509+0000 7fc4c4bff700 1 Processor -- start 2023-10-06T05:25:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.510+0000 7fc4c4bff700 1 -- start start 2023-10-06T05:25:49.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.511+0000 7fc4c4bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc4c01595a0 0x7fc4c0159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.511+0000 7fc4c4bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 0x7fc4c00c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.511+0000 7fc4c4bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c00c95b0 con 0x7fc4c00cc0d0 2023-10-06T05:25:49.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.511+0000 7fc4c4bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c00c9890 con 0x7fc4c01595a0 2023-10-06T05:25:49.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.511+0000 7fc4c4bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00c9b70 con 0x7fc4c0068d80 2023-10-06T05:25:49.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.511+0000 7fc4bdd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc4c00cc0d0 0x7fc4c0068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48628/0 (socket says 172.21.15.31:48628) 2023-10-06T05:25:49.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.511+0000 7fc4bdd9b700 1 -- 172.21.15.31:0/3036771100 learned_addr learned my addr 172.21.15.31:0/3036771100 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:49.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.511+0000 7fc4bed9d700 1 --2- 172.21.15.31:0/3036771100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 0x7fc4c00c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.511+0000 7fc4be59c700 1 --2- 172.21.15.31:0/3036771100 >> v2:172.21.15.62:3300/0 conn(0x7fc4c01595a0 0x7fc4c0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.512+0000 7fc4bd59a700 1 -- 172.21.15.31:0/3036771100 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255617784 0 0) 0x7fc4c00c95b0 con 0x7fc4c00cc0d0 2023-10-06T05:25:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.512+0000 7fc4bd59a700 1 -- 172.21.15.31:0/3036771100 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a8003570 con 0x7fc4c00cc0d0 2023-10-06T05:25:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.512+0000 7fc4bed9d700 1 -- 172.21.15.31:0/3036771100 >> v2:172.21.15.62:3300/0 conn(0x7fc4c01595a0 msgr2=0x7fc4c0159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.512+0000 7fc4bed9d700 1 --2- 172.21.15.31:0/3036771100 >> v2:172.21.15.62:3300/0 conn(0x7fc4c01595a0 0x7fc4c0159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.512+0000 7fc4bed9d700 1 -- 172.21.15.31:0/3036771100 >> v1:172.21.15.31:6789/0 conn(0x7fc4c00cc0d0 legacy=0x7fc4c0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.512+0000 7fc4bd59a700 1 -- 172.21.15.31:0/3036771100 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3134045770 0 0) 0x7fc4a8003570 con 0x7fc4c00cc0d0 2023-10-06T05:25:49.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.512+0000 7fc4bed9d700 1 -- 172.21.15.31:0/3036771100 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c00ca130 con 0x7fc4c0068d80 2023-10-06T05:25:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.512+0000 7fc4bed9d700 1 --2- 172.21.15.31:0/3036771100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 0x7fc4c00c8930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009e60 tx=0x7fc4b8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7218f365c030be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4bd59a700 1 -- 172.21.15.31:0/3036771100 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800e070 con 0x7fc4c0068d80 2023-10-06T05:25:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4bd59a700 1 -- 172.21.15.31:0/3036771100 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b800b7a0 con 0x7fc4c0068d80 2023-10-06T05:25:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4bd59a700 1 -- 172.21.15.31:0/3036771100 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4001470 con 0x7fc4c0068d80 2023-10-06T05:25:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3036771100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 msgr2=0x7fc4c00c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4c4bff700 1 --2- 172.21.15.31:0/3036771100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 0x7fc4c00c8930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009e60 tx=0x7fc4b8009780 comp rx=0 tx=0).stop 2023-10-06T05:25:49.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3036771100 shutdown_connections 2023-10-06T05:25:49.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4c4bff700 1 --2- 172.21.15.31:0/3036771100 >> v2:172.21.15.62:3300/0 conn(0x7fc4c01595a0 0x7fc4c0159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4c4bff700 1 --2- 172.21.15.31:0/3036771100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 0x7fc4c00c8930 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3036771100 >> 172.21.15.31:0/3036771100 conn(0x7fc4c006b620 msgr2=0x7fc4c006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:49.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3036771100 shutdown_connections 2023-10-06T05:25:49.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.513+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3036771100 wait complete. 2023-10-06T05:25:49.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.515+0000 7fc4c4bff700 1 Processor -- start 2023-10-06T05:25:49.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.515+0000 7fc4c4bff700 1 -- start start 2023-10-06T05:25:49.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4c4bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 0x7fc4c013f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4c4bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c00cc0d0 0x7fc4c01546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4c4bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4c01595a0 0x7fc4c0154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4c4bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c015c050 con 0x7fc4c01595a0 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4c4bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c015c330 con 0x7fc4c00cc0d0 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4c4bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c015c610 con 0x7fc4c0068d80 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4bed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4c01595a0 0x7fc4c0154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4be59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 0x7fc4c013f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4be59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 0x7fc4c013f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:37048/0 (socket says 172.21.15.31:37048) 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4bdd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c00cc0d0 0x7fc4c01546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4be59c700 1 -- 172.21.15.31:0/3454454477 learned_addr learned my addr 172.21.15.31:0/3454454477 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.516+0000 7fc4bed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4c01595a0 0x7fc4c0154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53576/0 (socket says 172.21.15.31:53576) 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4bed9d700 1 -- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 msgr2=0x7fc4c013f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4bed9d700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 0x7fc4c013f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4bed9d700 1 -- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c00cc0d0 msgr2=0x7fc4c01546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4bed9d700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c00cc0d0 0x7fc4c01546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4bed9d700 1 -- 172.21.15.31:0/3454454477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b8009a20 con 0x7fc4c01595a0 2023-10-06T05:25:49.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4bed9d700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4c01595a0 0x7fc4c0154bf0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8015000 tx=0x7fc4b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4ae7fc700 1 -- 172.21.15.31:0/3454454477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8012bc0 con 0x7fc4c01595a0 2023-10-06T05:25:49.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4ae7fc700 1 -- 172.21.15.31:0/3454454477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8012d60 con 0x7fc4c01595a0 2023-10-06T05:25:49.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4ae7fc700 1 -- 172.21.15.31:0/3454454477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b801d680 con 0x7fc4c01595a0 2023-10-06T05:25:49.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3454454477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c015c8f0 con 0x7fc4c01595a0 2023-10-06T05:25:49.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.517+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3454454477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c013fa60 con 0x7fc4c01595a0 2023-10-06T05:25:49.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.518+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3454454477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c015ce10 con 0x7fc4c01595a0 2023-10-06T05:25:49.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.519+0000 7fc4ae7fc700 1 -- 172.21.15.31:0/3454454477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fc4b8024070 con 0x7fc4c01595a0 2023-10-06T05:25:49.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.520+0000 7fc4ae7fc700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc4a0067830 0x7fc4a0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.520+0000 7fc4ae7fc700 1 -- 172.21.15.31:0/3454454477 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fc4b8084f60 con 0x7fc4c01595a0 2023-10-06T05:25:49.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.521+0000 7fc4be59c700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc4a0067830 0x7fc4a0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.521+0000 7fc4be59c700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc4a0067830 0x7fc4a0069ce0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009dd0 tx=0x7fc4b00091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.525+0000 7fc4ae7fc700 1 -- 172.21.15.31:0/3454454477 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4b8053ba0 con 0x7fc4c01595a0 2023-10-06T05:25:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.769+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3454454477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4c004c380 con 0x7fc4c01595a0 2023-10-06T05:25:49.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.770+0000 7fc4ae7fc700 1 -- 172.21.15.31:0/3454454477 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4600 (secure 0 0 0) 0x7fc4b80571c0 con 0x7fc4c01595a0 2023-10-06T05:25:49.772 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:49.772 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1489945066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1489945066},{"type":"v1","addr":"172.21.15.62:6834","nonce":1489945066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/660206899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":660206899},{"type":"v1","addr":"172.21.15.62:6835","nonce":660206899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8862,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1056123914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1056123914},{"type":"v1","addr":"172.21.15.31:6834","nonce":1056123914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:25:24.746699+0000","modified":"2023-10-06T05:25:28.266401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8865},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8865":{"gid":8865,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1535525741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1535525741},{"type":"v1","addr":"172.21.15.31:6835","nonce":1535525741}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:25:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc4a0067830 msgr2=0x7fc4a0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc4a0067830 0x7fc4a0069ce0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009dd0 tx=0x7fc4b00091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3454454477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4c01595a0 msgr2=0x7fc4c0154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4c01595a0 0x7fc4c0154bf0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8015000 tx=0x7fc4b8018000 comp rx=0 tx=0).stop 2023-10-06T05:25:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3454454477 shutdown_connections 2023-10-06T05:25:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc4a0067830 0x7fc4a0069ce0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4c01595a0 0x7fc4c0154bf0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c0068d80 0x7fc4c013f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 --2- 172.21.15.31:0/3454454477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c00cc0d0 0x7fc4c01546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3454454477 >> 172.21.15.31:0/3454454477 conn(0x7fc4c006b620 msgr2=0x7fc4c006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3454454477 shutdown_connections 2023-10-06T05:25:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.772+0000 7fc4c4bff700 1 -- 172.21.15.31:0/3454454477 wait complete. 2023-10-06T05:25:49.775 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 103 2023-10-06T05:25:49.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:49.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.984+0000 7fddfbbff700 1 Processor -- start 2023-10-06T05:25:49.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.984+0000 7fddfbbff700 1 -- start start 2023-10-06T05:25:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.985+0000 7fddfbbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddf40c7b60 0x7fddf4069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.985+0000 7fddfbbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7f30 0x7fddf40aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.985+0000 7fddfbbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf40ca5b0 con 0x7fddf406ef50 2023-10-06T05:25:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.985+0000 7fddfbbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf406f740 con 0x7fddf40c7b60 2023-10-06T05:25:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.985+0000 7fddfbbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40c9c80 con 0x7fddf40c7f30 2023-10-06T05:25:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.986+0000 7fddf999b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddf40c7b60 0x7fddf4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.986+0000 7fddf999b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddf40c7b60 0x7fddf4069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53580/0 (socket says 172.21.15.31:53580) 2023-10-06T05:25:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.986+0000 7fddf999b700 1 -- 172.21.15.31:0/2712219734 learned_addr learned my addr 172.21.15.31:0/2712219734 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.986+0000 7fddf919a700 1 --2- 172.21.15.31:0/2712219734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7f30 0x7fddf40aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.986+0000 7fddf999b700 1 -- 172.21.15.31:0/2712219734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7f30 msgr2=0x7fddf40aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.986+0000 7fddf999b700 1 --2- 172.21.15.31:0/2712219734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7f30 0x7fddf40aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.986+0000 7fddf999b700 1 -- 172.21.15.31:0/2712219734 >> v1:172.21.15.62:6789/0 conn(0x7fddf406ef50 legacy=0x7fddf413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.986+0000 7fddf999b700 1 -- 172.21.15.31:0/2712219734 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40c9f60 con 0x7fddf40c7b60 2023-10-06T05:25:49.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.986+0000 7fddf919a700 1 --2- 172.21.15.31:0/2712219734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7f30 0x7fddf40aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:49.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.986+0000 7fddf999b700 1 --2- 172.21.15.31:0/2712219734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf40c7b60 0x7fddf4069880 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009990 tx=0x7fddf0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d12ddea6f058189 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddf8999700 1 -- 172.21.15.31:0/2712219734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0005c10 con 0x7fddf40c7b60 2023-10-06T05:25:49.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddf8999700 1 -- 172.21.15.31:0/2712219734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf0005d70 con 0x7fddf40c7b60 2023-10-06T05:25:49.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddf8999700 1 -- 172.21.15.31:0/2712219734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf000b840 con 0x7fddf40c7b60 2023-10-06T05:25:49.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddfbbff700 1 -- 172.21.15.31:0/2712219734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf40c7b60 msgr2=0x7fddf4069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddfbbff700 1 --2- 172.21.15.31:0/2712219734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf40c7b60 0x7fddf4069880 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009990 tx=0x7fddf0009220 comp rx=0 tx=0).stop 2023-10-06T05:25:49.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddfbbff700 1 -- 172.21.15.31:0/2712219734 shutdown_connections 2023-10-06T05:25:49.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddfbbff700 1 --2- 172.21.15.31:0/2712219734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf40c7b60 0x7fddf4069880 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddfbbff700 1 --2- 172.21.15.31:0/2712219734 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7f30 0x7fddf40aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddfbbff700 1 -- 172.21.15.31:0/2712219734 >> 172.21.15.31:0/2712219734 conn(0x7fddf40cc0b0 msgr2=0x7fddf40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:49.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddfbbff700 1 -- 172.21.15.31:0/2712219734 shutdown_connections 2023-10-06T05:25:49.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.987+0000 7fddfbbff700 1 -- 172.21.15.31:0/2712219734 wait complete. 2023-10-06T05:25:49.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.989+0000 7fddfbbff700 1 Processor -- start 2023-10-06T05:25:49.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.990+0000 7fddfbbff700 1 -- start start 2023-10-06T05:25:49.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddfbbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddf406ef50 0x7fddf413eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddfbbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7b60 0x7fddf415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddfbbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf40c7f30 0x7fddf415f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddfbbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4158820 con 0x7fddf40c7f30 2023-10-06T05:25:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddfbbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf413f090 con 0x7fddf406ef50 2023-10-06T05:25:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddfbbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40abac0 con 0x7fddf40c7b60 2023-10-06T05:25:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddfa19c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf40c7f30 0x7fddf415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddf999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddf406ef50 0x7fddf413eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddf999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddf406ef50 0x7fddf413eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:54392/0 (socket says 172.21.15.31:54392) 2023-10-06T05:25:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddf999b700 1 -- 172.21.15.31:0/894951136 learned_addr learned my addr 172.21.15.31:0/894951136 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.991+0000 7fddf919a700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7b60 0x7fddf415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fddfa19c700 1 -- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7b60 msgr2=0x7fddf415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fddfa19c700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7b60 0x7fddf415e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fddfa19c700 1 -- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddf406ef50 msgr2=0x7fddf413eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fddfa19c700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddf406ef50 0x7fddf413eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fddfa19c700 1 -- 172.21.15.31:0/894951136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf0009480 con 0x7fddf40c7f30 2023-10-06T05:25:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fddf999b700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddf406ef50 0x7fddf413eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:25:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fddfa19c700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf40c7f30 0x7fddf415f220 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fddec00bad0 tx=0x7fddec009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fdde9ffb700 1 -- 172.21.15.31:0/894951136 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec00c760 con 0x7fddf40c7f30 2023-10-06T05:25:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fdde9ffb700 1 -- 172.21.15.31:0/894951136 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec00c8c0 con 0x7fddf40c7f30 2023-10-06T05:25:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fddfbbff700 1 -- 172.21.15.31:0/894951136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf40abda0 con 0x7fddf40c7f30 2023-10-06T05:25:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fdde9ffb700 1 -- 172.21.15.31:0/894951136 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec00cc80 con 0x7fddf40c7f30 2023-10-06T05:25:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.992+0000 7fddfbbff700 1 -- 172.21.15.31:0/894951136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf40ac360 con 0x7fddf40c7f30 2023-10-06T05:25:49.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.994+0000 7fddfbbff700 1 -- 172.21.15.31:0/894951136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf404c380 con 0x7fddf40c7f30 2023-10-06T05:25:49.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.994+0000 7fdde9ffb700 1 -- 172.21.15.31:0/894951136 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fddec014620 con 0x7fddf40c7f30 2023-10-06T05:25:49.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.995+0000 7fdde9ffb700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fddd8067830 0x7fddd8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:49.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.995+0000 7fdde9ffb700 1 -- 172.21.15.31:0/894951136 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fddec082be0 con 0x7fddf40c7f30 2023-10-06T05:25:49.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.996+0000 7fddf999b700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fddd8067830 0x7fddd8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:49.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.996+0000 7fddf999b700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fddd8067830 0x7fddd8069ce0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009fd0 tx=0x7fddf0015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:50.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:49.999+0000 7fdde9ffb700 1 -- 172.21.15.31:0/894951136 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddec04c5a0 con 0x7fddf40c7f30 2023-10-06T05:25:50.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.207+0000 7fddfbbff700 1 -- 172.21.15.31:0/894951136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddf406e590 con 0x7fddf40c7f30 2023-10-06T05:25:50.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.208+0000 7fdde9ffb700 1 -- 172.21.15.31:0/894951136 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+17164 (secure 0 0 0) 0x7fddec0552b0 con 0x7fddf40c7f30 2023-10-06T05:25:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:50.210 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":152,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:25:38.128777+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:25:22.964655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:25:23.719177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.3","mappings":[{"from":1,"to":6}]},{"pgid":"26.20","mappings":[{"from":3,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:50.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 -- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fddd8067830 msgr2=0x7fddd8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fddd8067830 0x7fddd8069ce0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009fd0 tx=0x7fddf0015000 comp rx=0 tx=0).stop 2023-10-06T05:25:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 -- 172.21.15.31:0/894951136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf40c7f30 msgr2=0x7fddf415f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf40c7f30 0x7fddf415f220 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fddec00bad0 tx=0x7fddec009760 comp rx=0 tx=0).stop 2023-10-06T05:25:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 -- 172.21.15.31:0/894951136 shutdown_connections 2023-10-06T05:25:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fddd8067830 0x7fddd8069ce0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf40c7f30 0x7fddf415f220 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddf406ef50 0x7fddf413eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 --2- 172.21.15.31:0/894951136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddf40c7b60 0x7fddf415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 -- 172.21.15.31:0/894951136 >> 172.21.15.31:0/894951136 conn(0x7fddf40cc0b0 msgr2=0x7fddf40aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 -- 172.21.15.31:0/894951136 shutdown_connections 2023-10-06T05:25:50.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.210+0000 7fddfbbff700 1 -- 172.21.15.31:0/894951136 wait complete. 2023-10-06T05:25:50.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:25:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.425+0000 7fda18bff700 1 Processor -- start 2023-10-06T05:25:50.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.425+0000 7fda18bff700 1 -- start start 2023-10-06T05:25:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.426+0000 7fda18bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda141596d0 0x7fda14159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.426+0000 7fda18bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda140c6f80 0x7fda1406a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.426+0000 7fda18bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda14067f70 con 0x7fda140c16e0 2023-10-06T05:25:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.426+0000 7fda18bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1406e040 con 0x7fda141596d0 2023-10-06T05:25:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.426+0000 7fda18bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda140681c0 con 0x7fda140c6f80 2023-10-06T05:25:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.426+0000 7fda1259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda141596d0 0x7fda14159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.427+0000 7fda1259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda141596d0 0x7fda14159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53594/0 (socket says 172.21.15.31:53594) 2023-10-06T05:25:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.427+0000 7fda1259c700 1 -- 172.21.15.31:0/1140337864 learned_addr learned my addr 172.21.15.31:0/1140337864 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.427+0000 7fda11d9b700 1 --2- 172.21.15.31:0/1140337864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda140c6f80 0x7fda1406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.427+0000 7fda1259c700 1 -- 172.21.15.31:0/1140337864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda140c6f80 msgr2=0x7fda1406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:50.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.427+0000 7fda1259c700 1 --2- 172.21.15.31:0/1140337864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda140c6f80 0x7fda1406a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.427+0000 7fda1259c700 1 -- 172.21.15.31:0/1140337864 >> v1:172.21.15.62:6789/0 conn(0x7fda140c16e0 legacy=0x7fda1415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.427+0000 7fda1259c700 1 -- 172.21.15.31:0/1140337864 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda140684a0 con 0x7fda141596d0 2023-10-06T05:25:50.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.427+0000 7fda1259c700 1 --2- 172.21.15.31:0/1140337864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda141596d0 0x7fda14159aa0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fda040099b0 tx=0x7fda04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0e89220262d6e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:50.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda1159a700 1 -- 172.21.15.31:0/1140337864 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04009ea0 con 0x7fda141596d0 2023-10-06T05:25:50.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda1159a700 1 -- 172.21.15.31:0/1140337864 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0400b860 con 0x7fda141596d0 2023-10-06T05:25:50.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda1159a700 1 -- 172.21.15.31:0/1140337864 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400bc00 con 0x7fda141596d0 2023-10-06T05:25:50.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda18bff700 1 -- 172.21.15.31:0/1140337864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda141596d0 msgr2=0x7fda14159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda18bff700 1 --2- 172.21.15.31:0/1140337864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda141596d0 0x7fda14159aa0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fda040099b0 tx=0x7fda04009250 comp rx=0 tx=0).stop 2023-10-06T05:25:50.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda18bff700 1 -- 172.21.15.31:0/1140337864 shutdown_connections 2023-10-06T05:25:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda18bff700 1 --2- 172.21.15.31:0/1140337864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda141596d0 0x7fda14159aa0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda18bff700 1 --2- 172.21.15.31:0/1140337864 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda140c6f80 0x7fda1406a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda18bff700 1 -- 172.21.15.31:0/1140337864 >> 172.21.15.31:0/1140337864 conn(0x7fda140c9e80 msgr2=0x7fda1406a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda18bff700 1 -- 172.21.15.31:0/1140337864 shutdown_connections 2023-10-06T05:25:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.428+0000 7fda18bff700 1 -- 172.21.15.31:0/1140337864 wait complete. 2023-10-06T05:25:50.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.430+0000 7fda18bff700 1 Processor -- start 2023-10-06T05:25:50.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.430+0000 7fda18bff700 1 -- start start 2023-10-06T05:25:50.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.431+0000 7fda18bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda140c16e0 0x7fda1415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.431+0000 7fda18bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda140c6f80 0x7fda1416d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.431+0000 7fda18bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda141596d0 0x7fda1416da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.431+0000 7fda18bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1415c6e0 con 0x7fda140c16e0 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.431+0000 7fda18bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1415c9c0 con 0x7fda141596d0 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.431+0000 7fda18bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1415cca0 con 0x7fda140c6f80 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.431+0000 7fda1259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda140c16e0 0x7fda1415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.431+0000 7fda1259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda140c16e0 0x7fda1415be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53600/0 (socket says 172.21.15.31:53600) 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.431+0000 7fda1259c700 1 -- 172.21.15.31:0/1251688246 learned_addr learned my addr 172.21.15.31:0/1251688246 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda11d9b700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda140c6f80 0x7fda1416d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda12d9d700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda141596d0 0x7fda1416da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda1259c700 1 -- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda140c6f80 msgr2=0x7fda1416d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda1259c700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda140c6f80 0x7fda1416d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda1259c700 1 -- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda141596d0 msgr2=0x7fda1416da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda1259c700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda141596d0 0x7fda1416da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda1259c700 1 -- 172.21.15.31:0/1251688246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040094f0 con 0x7fda140c16e0 2023-10-06T05:25:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda1259c700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda140c16e0 0x7fda1415be00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fda14069a60 tx=0x7fda04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda027fc700 1 -- 172.21.15.31:0/1251688246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400b9d0 con 0x7fda140c16e0 2023-10-06T05:25:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda027fc700 1 -- 172.21.15.31:0/1251688246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0400bb70 con 0x7fda140c16e0 2023-10-06T05:25:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda027fc700 1 -- 172.21.15.31:0/1251688246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04025670 con 0x7fda140c16e0 2023-10-06T05:25:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.432+0000 7fda18bff700 1 -- 172.21.15.31:0/1251688246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1415cf80 con 0x7fda140c16e0 2023-10-06T05:25:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.433+0000 7fda18bff700 1 -- 172.21.15.31:0/1251688246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1416dfb0 con 0x7fda140c16e0 2023-10-06T05:25:50.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.434+0000 7fda18bff700 1 -- 172.21.15.31:0/1251688246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1404c380 con 0x7fda140c16e0 2023-10-06T05:25:50.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.434+0000 7fda027fc700 1 -- 172.21.15.31:0/1251688246 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fda04023070 con 0x7fda140c16e0 2023-10-06T05:25:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.435+0000 7fda027fc700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9f4067830 0x7fd9f4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.436+0000 7fda027fc700 1 -- 172.21.15.31:0/1251688246 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fda04084390 con 0x7fda140c16e0 2023-10-06T05:25:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.436+0000 7fda11d9b700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9f4067830 0x7fd9f4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.437+0000 7fda11d9b700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9f4067830 0x7fd9f4069ce0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fda140b2880 tx=0x7fda08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:50.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.439+0000 7fda027fc700 1 -- 172.21.15.31:0/1251688246 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda04053080 con 0x7fda140c16e0 2023-10-06T05:25:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.691+0000 7fda18bff700 1 -- 172.21.15.31:0/1251688246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fda14063760 con 0x7fda140c16e0 2023-10-06T05:25:50.693 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:25:50.692+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:25:50.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.697+0000 7fda027fc700 1 -- 172.21.15.31:0/1251688246 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) v1 ==== 161+0+0 (secure 0 0 0) 0x7fda040566a0 con 0x7fda140c16e0 2023-10-06T05:25:50.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 -- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9f4067830 msgr2=0x7fd9f4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9f4067830 0x7fd9f4069ce0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fda140b2880 tx=0x7fda08008040 comp rx=0 tx=0).stop 2023-10-06T05:25:50.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 -- 172.21.15.31:0/1251688246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda140c16e0 msgr2=0x7fda1415be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda140c16e0 0x7fda1415be00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fda14069a60 tx=0x7fda04017000 comp rx=0 tx=0).stop 2023-10-06T05:25:50.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 -- 172.21.15.31:0/1251688246 shutdown_connections 2023-10-06T05:25:50.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd9f4067830 0x7fd9f4069ce0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda141596d0 0x7fda1416da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda140c16e0 0x7fda1415be00 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 --2- 172.21.15.31:0/1251688246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda140c6f80 0x7fda1416d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 -- 172.21.15.31:0/1251688246 >> 172.21.15.31:0/1251688246 conn(0x7fda140c9e80 msgr2=0x7fda14067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:50.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 -- 172.21.15.31:0/1251688246 shutdown_connections 2023-10-06T05:25:50.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.700+0000 7fda18bff700 1 -- 172.21.15.31:0/1251688246 wait complete. 2023-10-06T05:25:50.704 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:25:50.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:25:50.764 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:25:50.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.921+0000 7ff9f2bff700 1 Processor -- start 2023-10-06T05:25:50.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.921+0000 7ff9f2bff700 1 -- start start 2023-10-06T05:25:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.922+0000 7ff9f2bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9ec0cc340 0x7ff9ec0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.922+0000 7ff9f2bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ec0cc710 0x7ff9ec0b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.922+0000 7ff9f2bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff9ec15a4b0 0x7ff9ec0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.922+0000 7ff9f2bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ec15a020 con 0x7ff9ec0cc340 2023-10-06T05:25:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.922+0000 7ff9f2bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ec159cf0 con 0x7ff9ec15a4b0 2023-10-06T05:25:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.922+0000 7ff9f2bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0ccb60 con 0x7ff9ec0cc710 2023-10-06T05:25:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.922+0000 7ff9f099b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9ec0cc340 0x7ff9ec0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9f099b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9ec0cc340 0x7ff9ec0b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53616/0 (socket says 172.21.15.31:53616) 2023-10-06T05:25:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9f099b700 1 -- 172.21.15.31:0/1360648020 learned_addr learned my addr 172.21.15.31:0/1360648020 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9ebfff700 1 --2- 172.21.15.31:0/1360648020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ec0cc710 0x7ff9ec0b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9f119c700 1 --2- 172.21.15.31:0/1360648020 >> v2:172.21.15.62:3300/0 conn(0x7ff9ec15a4b0 0x7ff9ec0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9f099b700 1 -- 172.21.15.31:0/1360648020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ec0cc710 msgr2=0x7ff9ec0b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9f099b700 1 --2- 172.21.15.31:0/1360648020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ec0cc710 0x7ff9ec0b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9f099b700 1 -- 172.21.15.31:0/1360648020 >> v2:172.21.15.62:3300/0 conn(0x7ff9ec15a4b0 msgr2=0x7ff9ec0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9f099b700 1 --2- 172.21.15.31:0/1360648020 >> v2:172.21.15.62:3300/0 conn(0x7ff9ec15a4b0 0x7ff9ec0aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9f099b700 1 -- 172.21.15.31:0/1360648020 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0b2b40 con 0x7ff9ec0cc340 2023-10-06T05:25:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9f099b700 1 --2- 172.21.15.31:0/1360648020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 0x7ff9ec0b2540 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0ac1b0 tx=0x7ff9e0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f624041594a8677 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:50.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.923+0000 7ff9eb7fe700 1 -- 172.21.15.31:0/1360648020 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0005c10 con 0x7ff9ec0cc340 2023-10-06T05:25:50.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9eb7fe700 1 -- 172.21.15.31:0/1360648020 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e0005d70 con 0x7ff9ec0cc340 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9eb7fe700 1 -- 172.21.15.31:0/1360648020 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0006260 con 0x7ff9ec0cc340 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9f2bff700 1 -- 172.21.15.31:0/1360648020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 msgr2=0x7ff9ec0b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9f2bff700 1 --2- 172.21.15.31:0/1360648020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 0x7ff9ec0b2540 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0ac1b0 tx=0x7ff9e0009220 comp rx=0 tx=0).stop 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9f2bff700 1 -- 172.21.15.31:0/1360648020 shutdown_connections 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9f2bff700 1 --2- 172.21.15.31:0/1360648020 >> v2:172.21.15.62:3300/0 conn(0x7ff9ec15a4b0 0x7ff9ec0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9f2bff700 1 --2- 172.21.15.31:0/1360648020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 0x7ff9ec0b2540 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9f2bff700 1 --2- 172.21.15.31:0/1360648020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ec0cc710 0x7ff9ec0b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9f2bff700 1 -- 172.21.15.31:0/1360648020 >> 172.21.15.31:0/1360648020 conn(0x7ff9ec068670 msgr2=0x7ff9ec068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9f2bff700 1 -- 172.21.15.31:0/1360648020 shutdown_connections 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.924+0000 7ff9f2bff700 1 -- 172.21.15.31:0/1360648020 wait complete. 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.926+0000 7ff9f2bff700 1 Processor -- start 2023-10-06T05:25:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.926+0000 7ff9f2bff700 1 -- start start 2023-10-06T05:25:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f2bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 0x7ff9ec0ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f2bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9ec0cc710 0x7ff9ec151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f2bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ec15a4b0 0x7ff9ec1523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f2bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0acea0 con 0x7ff9ec0cc340 2023-10-06T05:25:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f2bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec153820 con 0x7ff9ec0cc710 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f2bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec153260 con 0x7ff9ec15a4b0 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f099b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 0x7ff9ec0ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f099b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 0x7ff9ec0ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53618/0 (socket says 172.21.15.31:53618) 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f099b700 1 -- 172.21.15.31:0/811889707 learned_addr learned my addr 172.21.15.31:0/811889707 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9ebfff700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9ec0cc710 0x7ff9ec151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f119c700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ec15a4b0 0x7ff9ec1523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f099b700 1 -- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ec15a4b0 msgr2=0x7ff9ec1523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f099b700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ec15a4b0 0x7ff9ec1523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f099b700 1 -- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9ec0cc710 msgr2=0x7ff9ec151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f099b700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9ec0cc710 0x7ff9ec151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.927+0000 7ff9f099b700 1 -- 172.21.15.31:0/811889707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0009480 con 0x7ff9ec0cc340 2023-10-06T05:25:50.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.928+0000 7ff9f099b700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 0x7ff9ec0ad560 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009f50 tx=0x7ff9e0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:50.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.928+0000 7ff9e8ff9700 1 -- 172.21.15.31:0/811889707 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0011750 con 0x7ff9ec0cc340 2023-10-06T05:25:50.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.928+0000 7ff9e8ff9700 1 -- 172.21.15.31:0/811889707 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e0020070 con 0x7ff9ec0cc340 2023-10-06T05:25:50.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.928+0000 7ff9f2bff700 1 -- 172.21.15.31:0/811889707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ec153540 con 0x7ff9ec0cc340 2023-10-06T05:25:50.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.928+0000 7ff9e8ff9700 1 -- 172.21.15.31:0/811889707 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0011ae0 con 0x7ff9ec0cc340 2023-10-06T05:25:50.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.928+0000 7ff9f2bff700 1 -- 172.21.15.31:0/811889707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec15b030 con 0x7ff9ec0cc340 2023-10-06T05:25:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.929+0000 7ff9f2bff700 1 -- 172.21.15.31:0/811889707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ec15b220 con 0x7ff9ec0cc340 2023-10-06T05:25:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.930+0000 7ff9e8ff9700 1 -- 172.21.15.31:0/811889707 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7ff9e0022030 con 0x7ff9ec0cc340 2023-10-06T05:25:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.931+0000 7ff9e8ff9700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff9d0067830 0x7ff9d0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.931+0000 7ff9e8ff9700 1 -- 172.21.15.31:0/811889707 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6288+0+0 (secure 0 0 0) 0x7ff9e0084900 con 0x7ff9ec0cc340 2023-10-06T05:25:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.932+0000 7ff9ebfff700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff9d0067830 0x7ff9d0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.933+0000 7ff9ebfff700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff9d0067830 0x7ff9d0069ce0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0ad470 tx=0x7ff9d80091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:50.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:50.936+0000 7ff9e8ff9700 1 -- 172.21.15.31:0/811889707 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9e0053560 con 0x7ff9ec0cc340 2023-10-06T05:25:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.183+0000 7ff9f2bff700 1 -- 172.21.15.31:0/811889707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff9ec04c380 con 0x7ff9ec0cc340 2023-10-06T05:25:51.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.698+0000 7ff9e8ff9700 1 -- 172.21.15.31:0/811889707 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v105) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff9e0056b80 con 0x7ff9ec0cc340 2023-10-06T05:25:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 -- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff9d0067830 msgr2=0x7ff9d0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff9d0067830 0x7ff9d0069ce0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0ad470 tx=0x7ff9d80091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 -- 172.21.15.31:0/811889707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 msgr2=0x7ff9ec0ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 0x7ff9ec0ad560 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009f50 tx=0x7ff9e0016000 comp rx=0 tx=0).stop 2023-10-06T05:25:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 -- 172.21.15.31:0/811889707 shutdown_connections 2023-10-06T05:25:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff9d0067830 0x7ff9d0069ce0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ec15a4b0 0x7ff9ec1523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ec0cc340 0x7ff9ec0ad560 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 --2- 172.21.15.31:0/811889707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9ec0cc710 0x7ff9ec151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 -- 172.21.15.31:0/811889707 >> 172.21.15.31:0/811889707 conn(0x7ff9ec068670 msgr2=0x7ff9ec154950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 -- 172.21.15.31:0/811889707 shutdown_connections 2023-10-06T05:25:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.701+0000 7ff9f2bff700 1 -- 172.21.15.31:0/811889707 wait complete. 2023-10-06T05:25:51.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:25:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.964+0000 7f85edbff700 1 Processor -- start 2023-10-06T05:25:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.965+0000 7f85edbff700 1 -- start start 2023-10-06T05:25:51.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.966+0000 7f85edbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e8159410 0x7f85e80c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.966+0000 7f85edbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f85e806b4c0 0x7f85e81597e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.966+0000 7f85edbff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e80c7830 con 0x7f85e80c73c0 2023-10-06T05:25:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.966+0000 7f85edbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e8069750 con 0x7f85e806b4c0 2023-10-06T05:25:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.966+0000 7f85edbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e8069a30 con 0x7f85e8159410 2023-10-06T05:25:51.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.966+0000 7f85e7fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f85e806b4c0 0x7f85e81597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:51.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.966+0000 7f85e77fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e8159410 0x7f85e80c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:51.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.966+0000 7f85e7fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f85e806b4c0 0x7f85e81597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:54436/0 (socket says 172.21.15.31:54436) 2023-10-06T05:25:51.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.966+0000 7f85e7fff700 1 -- 172.21.15.31:0/2174045842 learned_addr learned my addr 172.21.15.31:0/2174045842 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:51.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.967+0000 7f85e7fff700 1 -- 172.21.15.31:0/2174045842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e8159410 msgr2=0x7f85e80c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:51.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.967+0000 7f85e7fff700 1 --2- 172.21.15.31:0/2174045842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e8159410 0x7f85e80c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:51.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.967+0000 7f85e7fff700 1 -- 172.21.15.31:0/2174045842 >> v1:172.21.15.62:6789/0 conn(0x7f85e80c73c0 legacy=0x7f85e806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:51.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.967+0000 7f85e7fff700 1 -- 172.21.15.31:0/2174045842 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8069d10 con 0x7f85e806b4c0 2023-10-06T05:25:51.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.967+0000 7f85e7fff700 1 --2- 172.21.15.31:0/2174045842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85e806b4c0 0x7f85e81597e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009f20 tx=0x7f85e0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e07cb230b33f129 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:51.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.967+0000 7f85e67fc700 1 -- 172.21.15.31:0/2174045842 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e000e070 con 0x7f85e806b4c0 2023-10-06T05:25:51.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.968+0000 7f85e67fc700 1 -- 172.21.15.31:0/2174045842 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e000b9d0 con 0x7f85e806b4c0 2023-10-06T05:25:51.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.968+0000 7f85e67fc700 1 -- 172.21.15.31:0/2174045842 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e00092e0 con 0x7f85e806b4c0 2023-10-06T05:25:51.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.968+0000 7f85edbff700 1 -- 172.21.15.31:0/2174045842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85e806b4c0 msgr2=0x7f85e81597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.968+0000 7f85edbff700 1 --2- 172.21.15.31:0/2174045842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85e806b4c0 0x7f85e81597e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009f20 tx=0x7f85e0009840 comp rx=0 tx=0).stop 2023-10-06T05:25:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.968+0000 7f85edbff700 1 -- 172.21.15.31:0/2174045842 shutdown_connections 2023-10-06T05:25:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.968+0000 7f85edbff700 1 --2- 172.21.15.31:0/2174045842 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e8159410 0x7f85e80c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.968+0000 7f85edbff700 1 --2- 172.21.15.31:0/2174045842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85e806b4c0 0x7f85e81597e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.968+0000 7f85edbff700 1 -- 172.21.15.31:0/2174045842 >> 172.21.15.31:0/2174045842 conn(0x7f85e80c9350 msgr2=0x7f85e80c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.968+0000 7f85edbff700 1 -- 172.21.15.31:0/2174045842 shutdown_connections 2023-10-06T05:25:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.968+0000 7f85edbff700 1 -- 172.21.15.31:0/2174045842 wait complete. 2023-10-06T05:25:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.970+0000 7f85edbff700 1 Processor -- start 2023-10-06T05:25:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.970+0000 7f85edbff700 1 -- start start 2023-10-06T05:25:51.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85edbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e806b4c0 0x7f85e8152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:51.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85edbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85e80c73c0 0x7f85e80c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:51.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85edbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85e8159410 0x7f85e80aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85edbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80c01e0 con 0x7f85e8159410 2023-10-06T05:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85edbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80c2ff0 con 0x7f85e80c73c0 2023-10-06T05:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85edbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e8152630 con 0x7f85e806b4c0 2023-10-06T05:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85e7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85e8159410 0x7f85e80aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85e7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85e8159410 0x7f85e80aa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53634/0 (socket says 172.21.15.31:53634) 2023-10-06T05:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85e7fff700 1 -- 172.21.15.31:0/395606995 learned_addr learned my addr 172.21.15.31:0/395606995 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85e77fe700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e806b4c0 0x7f85e8152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85e6ffd700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85e80c73c0 0x7f85e80c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85e7fff700 1 -- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e806b4c0 msgr2=0x7f85e8152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85e7fff700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e806b4c0 0x7f85e8152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85e7fff700 1 -- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85e80c73c0 msgr2=0x7f85e80c2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85e7fff700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85e80c73c0 0x7f85e80c2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.971+0000 7f85e7fff700 1 -- 172.21.15.31:0/395606995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0009ae0 con 0x7f85e8159410 2023-10-06T05:25:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.972+0000 7f85e77fe700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e806b4c0 0x7f85e8152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.972+0000 7f85e7fff700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85e8159410 0x7f85e80aa2c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f85e806c600 tx=0x7f85e0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.972+0000 7f85cf7fe700 1 -- 172.21.15.31:0/395606995 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0008c70 con 0x7f85e8159410 2023-10-06T05:25:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.972+0000 7f85cf7fe700 1 -- 172.21.15.31:0/395606995 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0008e10 con 0x7f85e8159410 2023-10-06T05:25:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.972+0000 7f85cf7fe700 1 -- 172.21.15.31:0/395606995 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e00256b0 con 0x7f85e8159410 2023-10-06T05:25:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.972+0000 7f85edbff700 1 -- 172.21.15.31:0/395606995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e80aead0 con 0x7f85e8159410 2023-10-06T05:25:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.972+0000 7f85edbff700 1 -- 172.21.15.31:0/395606995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e8152110 con 0x7f85e8159410 2023-10-06T05:25:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.973+0000 7f85edbff700 1 -- 172.21.15.31:0/395606995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e8152300 con 0x7f85e8159410 2023-10-06T05:25:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.974+0000 7f85cf7fe700 1 -- 172.21.15.31:0/395606995 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f85e0003040 con 0x7f85e8159410 2023-10-06T05:25:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.975+0000 7f85cf7fe700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85d4067860 0x7f85d4069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.975+0000 7f85cf7fe700 1 -- 172.21.15.31:0/395606995 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f85e0084450 con 0x7f85e8159410 2023-10-06T05:25:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.975+0000 7f85e77fe700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85d4067860 0x7f85d4069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.976+0000 7f85e77fe700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85d4067860 0x7f85d4069d10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009d80 tx=0x7f85d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:51.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:51.980+0000 7f85cf7fe700 1 -- 172.21.15.31:0/395606995 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85e0053160 con 0x7f85e8159410 2023-10-06T05:25:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.184+0000 7f85edbff700 1 -- 172.21.15.31:0/395606995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f85e804c380 con 0x7f85e8159410 2023-10-06T05:25:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.701+0000 7f85cf7fe700 1 -- 172.21.15.31:0/395606995 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v154) v1 ==== 181+0+0 (secure 0 0 0) 0x7f85e0056780 con 0x7f85e8159410 2023-10-06T05:25:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.704+0000 7f85edbff700 1 -- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85d4067860 msgr2=0x7f85d4069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.704+0000 7f85edbff700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85d4067860 0x7f85d4069d10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009d80 tx=0x7f85d8008040 comp rx=0 tx=0).stop 2023-10-06T05:25:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.705+0000 7f85edbff700 1 -- 172.21.15.31:0/395606995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85e8159410 msgr2=0x7f85e80aa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.705+0000 7f85edbff700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85e8159410 0x7f85e80aa2c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f85e806c600 tx=0x7f85e0018000 comp rx=0 tx=0).stop 2023-10-06T05:25:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.705+0000 7f85edbff700 1 -- 172.21.15.31:0/395606995 shutdown_connections 2023-10-06T05:25:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.705+0000 7f85edbff700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85d4067860 0x7f85d4069d10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.705+0000 7f85edbff700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85e8159410 0x7f85e80aa2c0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.705+0000 7f85edbff700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f85e806b4c0 0x7f85e8152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.705+0000 7f85edbff700 1 --2- 172.21.15.31:0/395606995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85e80c73c0 0x7f85e80c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.705+0000 7f85edbff700 1 -- 172.21.15.31:0/395606995 >> 172.21.15.31:0/395606995 conn(0x7f85e80c9350 msgr2=0x7f85e81531a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:52.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.705+0000 7f85edbff700 1 -- 172.21.15.31:0/395606995 shutdown_connections 2023-10-06T05:25:52.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.705+0000 7f85edbff700 1 -- 172.21.15.31:0/395606995 wait complete. 2023-10-06T05:25:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:25:52.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:25:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.925+0000 7f2a19bff700 1 Processor -- start 2023-10-06T05:25:52.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.926+0000 7f2a19bff700 1 -- start start 2023-10-06T05:25:52.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.927+0000 7f2a19bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2a14154cd0 0x7f2a141583e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.927+0000 7f2a19bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2a141550a0 0x7f2a140ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.927+0000 7f2a19bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1406f640 con 0x7f2a1406ee10 2023-10-06T05:25:52.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.927+0000 7f2a19bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a1406e610 con 0x7f2a14154cd0 2023-10-06T05:25:52.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.927+0000 7f2a19bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a140adf60 con 0x7f2a141550a0 2023-10-06T05:25:52.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.927+0000 7f2a137fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2a14154cd0 0x7f2a141583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.927+0000 7f2a137fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2a14154cd0 0x7f2a141583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53642/0 (socket says 172.21.15.31:53642) 2023-10-06T05:25:52.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.927+0000 7f2a137fe700 1 -- 172.21.15.31:0/968246065 learned_addr learned my addr 172.21.15.31:0/968246065 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:52.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.927+0000 7f2a13fff700 1 --1- 172.21.15.31:0/968246065 >> v1:172.21.15.31:6789/0 conn(0x7f2a1406ee10 0x7f2a1406d7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48640/0 (socket says 172.21.15.31:48640) 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a12ffd700 1 --2- 172.21.15.31:0/968246065 >> v2:172.21.15.62:3300/0 conn(0x7f2a141550a0 0x7f2a140ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a137fe700 1 -- 172.21.15.31:0/968246065 >> v2:172.21.15.62:3300/0 conn(0x7f2a141550a0 msgr2=0x7f2a140ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a137fe700 1 --2- 172.21.15.31:0/968246065 >> v2:172.21.15.62:3300/0 conn(0x7f2a141550a0 0x7f2a140ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a137fe700 1 -- 172.21.15.31:0/968246065 >> v1:172.21.15.31:6789/0 conn(0x7f2a1406ee10 legacy=0x7f2a1406d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a137fe700 1 -- 172.21.15.31:0/968246065 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a14158920 con 0x7f2a14154cd0 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a127fc700 1 -- 172.21.15.31:0/968246065 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828532276 0 0) 0x7f2a1406f640 con 0x7f2a1406ee10 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a12ffd700 1 --2- 172.21.15.31:0/968246065 >> v2:172.21.15.62:3300/0 conn(0x7f2a141550a0 0x7f2a140ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a137fe700 1 --2- 172.21.15.31:0/968246065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a14154cd0 0x7f2a141583e0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f2a1406ceb0 tx=0x7f2a04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68fa05d4496fbfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a127fc700 1 -- 172.21.15.31:0/968246065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400b860 con 0x7f2a14154cd0 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a127fc700 1 -- 172.21.15.31:0/968246065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0400b9c0 con 0x7f2a14154cd0 2023-10-06T05:25:52.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.928+0000 7f2a127fc700 1 -- 172.21.15.31:0/968246065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400bd60 con 0x7f2a14154cd0 2023-10-06T05:25:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.929+0000 7f2a19bff700 1 -- 172.21.15.31:0/968246065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a14154cd0 msgr2=0x7f2a141583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.929+0000 7f2a19bff700 1 --2- 172.21.15.31:0/968246065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a14154cd0 0x7f2a141583e0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f2a1406ceb0 tx=0x7f2a04009250 comp rx=0 tx=0).stop 2023-10-06T05:25:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.929+0000 7f2a19bff700 1 -- 172.21.15.31:0/968246065 shutdown_connections 2023-10-06T05:25:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.929+0000 7f2a19bff700 1 --2- 172.21.15.31:0/968246065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a14154cd0 0x7f2a141583e0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.929+0000 7f2a19bff700 1 --2- 172.21.15.31:0/968246065 >> v2:172.21.15.62:3300/0 conn(0x7f2a141550a0 0x7f2a140ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.929+0000 7f2a19bff700 1 -- 172.21.15.31:0/968246065 >> 172.21.15.31:0/968246065 conn(0x7f2a14068030 msgr2=0x7f2a140cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.929+0000 7f2a19bff700 1 -- 172.21.15.31:0/968246065 shutdown_connections 2023-10-06T05:25:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.929+0000 7f2a19bff700 1 -- 172.21.15.31:0/968246065 wait complete. 2023-10-06T05:25:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.930+0000 7f2a19bff700 1 Processor -- start 2023-10-06T05:25:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.931+0000 7f2a19bff700 1 -- start start 2023-10-06T05:25:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.931+0000 7f2a19bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a1406ee10 0x7f2a140b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.931+0000 7f2a19bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a14154cd0 0x7f2a14150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a19bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a141550a0 0x7f2a140c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a19bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140b1350 con 0x7f2a1406ee10 2023-10-06T05:25:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a19bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140b0740 con 0x7f2a14154cd0 2023-10-06T05:25:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a19bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140b0a20 con 0x7f2a141550a0 2023-10-06T05:25:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a1406ee10 0x7f2a140b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a1406ee10 0x7f2a140b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53656/0 (socket says 172.21.15.31:53656) 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a137fe700 1 -- 172.21.15.31:0/183671916 learned_addr learned my addr 172.21.15.31:0/183671916 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a12ffd700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a14154cd0 0x7f2a14150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a13fff700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a141550a0 0x7f2a140c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a137fe700 1 -- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a141550a0 msgr2=0x7f2a140c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a137fe700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a141550a0 0x7f2a140c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a137fe700 1 -- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a14154cd0 msgr2=0x7f2a14150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a137fe700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a14154cd0 0x7f2a14150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.932+0000 7f2a137fe700 1 -- 172.21.15.31:0/183671916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04009490 con 0x7f2a1406ee10 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.933+0000 7f2a137fe700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a1406ee10 0x7f2a140b17e0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2a04006fd0 tx=0x7f2a04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.933+0000 7f29fb7fe700 1 -- 172.21.15.31:0/183671916 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04024440 con 0x7f2a1406ee10 2023-10-06T05:25:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.933+0000 7f29fb7fe700 1 -- 172.21.15.31:0/183671916 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a040245a0 con 0x7f2a1406ee10 2023-10-06T05:25:52.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.933+0000 7f29fb7fe700 1 -- 172.21.15.31:0/183671916 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04024940 con 0x7f2a1406ee10 2023-10-06T05:25:52.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.933+0000 7f2a19bff700 1 -- 172.21.15.31:0/183671916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a140b0d00 con 0x7f2a1406ee10 2023-10-06T05:25:52.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.933+0000 7f2a19bff700 1 -- 172.21.15.31:0/183671916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a14151270 con 0x7f2a1406ee10 2023-10-06T05:25:52.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.934+0000 7f2a19bff700 1 -- 172.21.15.31:0/183671916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1404c380 con 0x7f2a1406ee10 2023-10-06T05:25:52.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.935+0000 7f29fb7fe700 1 -- 172.21.15.31:0/183671916 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f2a0401a420 con 0x7f2a1406ee10 2023-10-06T05:25:52.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.936+0000 7f29fb7fe700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2a00067830 0x7f2a00069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:52.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.936+0000 7f29fb7fe700 1 -- 172.21.15.31:0/183671916 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f2a04084d60 con 0x7f2a1406ee10 2023-10-06T05:25:52.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.937+0000 7f2a12ffd700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2a00067830 0x7f2a00069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:52.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.937+0000 7f2a12ffd700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2a00067830 0x7f2a00069ce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2a1416cb20 tx=0x7f2a08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:52.941+0000 7f29fb7fe700 1 -- 172.21.15.31:0/183671916 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a04053a20 con 0x7f2a1406ee10 2023-10-06T05:25:53.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.149+0000 7f2a19bff700 1 -- 172.21.15.31:0/183671916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2a140b1630 con 0x7f2a1406ee10 2023-10-06T05:25:53.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.703+0000 7f29fb7fe700 1 -- 172.21.15.31:0/183671916 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v155) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2a04057040 con 0x7f2a1406ee10 2023-10-06T05:25:53.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.709+0000 7f2a19bff700 1 -- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2a00067830 msgr2=0x7f2a00069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.709+0000 7f2a19bff700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2a00067830 0x7f2a00069ce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2a1416cb20 tx=0x7f2a08008040 comp rx=0 tx=0).stop 2023-10-06T05:25:53.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.709+0000 7f2a19bff700 1 -- 172.21.15.31:0/183671916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a1406ee10 msgr2=0x7f2a140b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.709+0000 7f2a19bff700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a1406ee10 0x7f2a140b17e0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2a04006fd0 tx=0x7f2a04017000 comp rx=0 tx=0).stop 2023-10-06T05:25:53.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.710+0000 7f2a19bff700 1 -- 172.21.15.31:0/183671916 shutdown_connections 2023-10-06T05:25:53.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.710+0000 7f2a19bff700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2a00067830 0x7f2a00069ce0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.710+0000 7f2a19bff700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a141550a0 0x7f2a140c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.710+0000 7f2a19bff700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a1406ee10 0x7f2a140b17e0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.710+0000 7f2a19bff700 1 --2- 172.21.15.31:0/183671916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a14154cd0 0x7f2a14150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.710+0000 7f2a19bff700 1 -- 172.21.15.31:0/183671916 >> 172.21.15.31:0/183671916 conn(0x7f2a14068030 msgr2=0x7f2a14153800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:53.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.711+0000 7f2a19bff700 1 -- 172.21.15.31:0/183671916 shutdown_connections 2023-10-06T05:25:53.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.711+0000 7f2a19bff700 1 -- 172.21.15.31:0/183671916 wait complete. 2023-10-06T05:25:53.716 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:25:53.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:25:53.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.939+0000 7f4a20fff700 1 Processor -- start 2023-10-06T05:25:53.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.939+0000 7f4a20fff700 1 -- start start 2023-10-06T05:25:53.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.941+0000 7f4a20fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159930 0x7f4a1c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.941+0000 7f4a20fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4a1c0cd580 0x7f4a1c0b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.941+0000 7f4a20fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c15d2d0 con 0x7f4a1c159560 2023-10-06T05:25:53.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.941+0000 7f4a20fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c15d5b0 con 0x7f4a1c0cd580 2023-10-06T05:25:53.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.941+0000 7f4a20fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c15d890 con 0x7f4a1c159930 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.941+0000 7f4a1ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4a1c0cd580 0x7f4a1c0b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.942+0000 7f4a1ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4a1c0cd580 0x7f4a1c0b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53666/0 (socket says 172.21.15.31:53666) 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.942+0000 7f4a1ad9d700 1 -- 172.21.15.31:0/1585773240 learned_addr learned my addr 172.21.15.31:0/1585773240 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.942+0000 7f4a19d9b700 1 --2- 172.21.15.31:0/1585773240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159930 0x7f4a1c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.942+0000 7f4a1a59c700 1 --1- 172.21.15.31:0/1585773240 >> v1:172.21.15.62:6789/0 conn(0x7f4a1c159560 0x7f4a1c0cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:44088/0 (socket says 172.21.15.31:44088) 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.942+0000 7f4a1ad9d700 1 -- 172.21.15.31:0/1585773240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159930 msgr2=0x7f4a1c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.942+0000 7f4a1ad9d700 1 --2- 172.21.15.31:0/1585773240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159930 0x7f4a1c0b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.942+0000 7f4a1ad9d700 1 -- 172.21.15.31:0/1585773240 >> v1:172.21.15.62:6789/0 conn(0x7f4a1c159560 legacy=0x7f4a1c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.942+0000 7f4a1ad9d700 1 -- 172.21.15.31:0/1585773240 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c15db70 con 0x7f4a1c0cd580 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.942+0000 7f4a1959a700 1 -- 172.21.15.31:0/1585773240 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248649112 0 0) 0x7f4a1c15d2d0 con 0x7f4a1c159560 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.942+0000 7f4a1ad9d700 1 --2- 172.21.15.31:0/1585773240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 0x7f4a1c0b29b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009f20 tx=0x7f4a14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f63663190315f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a1959a700 1 -- 172.21.15.31:0/1585773240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400e070 con 0x7f4a1c0cd580 2023-10-06T05:25:53.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a1959a700 1 -- 172.21.15.31:0/1585773240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1400b9d0 con 0x7f4a1c0cd580 2023-10-06T05:25:53.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a1959a700 1 -- 172.21.15.31:0/1585773240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140092e0 con 0x7f4a1c0cd580 2023-10-06T05:25:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a20fff700 1 -- 172.21.15.31:0/1585773240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 msgr2=0x7f4a1c0b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a20fff700 1 --2- 172.21.15.31:0/1585773240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 0x7f4a1c0b29b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009f20 tx=0x7f4a14009840 comp rx=0 tx=0).stop 2023-10-06T05:25:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a20fff700 1 -- 172.21.15.31:0/1585773240 shutdown_connections 2023-10-06T05:25:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a20fff700 1 --2- 172.21.15.31:0/1585773240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 0x7f4a1c0b29b0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a20fff700 1 --2- 172.21.15.31:0/1585773240 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159930 0x7f4a1c0b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a20fff700 1 -- 172.21.15.31:0/1585773240 >> 172.21.15.31:0/1585773240 conn(0x7f4a1c06ec90 msgr2=0x7f4a1c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a20fff700 1 -- 172.21.15.31:0/1585773240 shutdown_connections 2023-10-06T05:25:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.943+0000 7f4a20fff700 1 -- 172.21.15.31:0/1585773240 wait complete. 2023-10-06T05:25:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.945+0000 7f4a20fff700 1 Processor -- start 2023-10-06T05:25:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.945+0000 7f4a20fff700 1 -- start start 2023-10-06T05:25:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a20fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 0x7f4a1c0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a20fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159560 0x7f4a1c169880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a20fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a1c159930 0x7f4a1c1497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a20fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c16d1c0 con 0x7f4a1c0cd580 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a20fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c16a0c0 con 0x7f4a1c159930 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a20fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c16a3a0 con 0x7f4a1c159560 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a1a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 0x7f4a1c0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a1a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 0x7f4a1c0c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53678/0 (socket says 172.21.15.31:53678) 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a1a59c700 1 -- 172.21.15.31:0/2492015504 learned_addr learned my addr 172.21.15.31:0/2492015504 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a19d9b700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159560 0x7f4a1c169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a1ad9d700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a1c159930 0x7f4a1c1497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a1a59c700 1 -- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159560 msgr2=0x7f4a1c169880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a1a59c700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159560 0x7f4a1c169880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a1a59c700 1 -- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a1c159930 msgr2=0x7f4a1c1497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a1a59c700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a1c159930 0x7f4a1c1497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a1a59c700 1 -- 172.21.15.31:0/2492015504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14009ae0 con 0x7f4a1c0cd580 2023-10-06T05:25:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a19d9b700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159560 0x7f4a1c169880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.946+0000 7f4a1ad9d700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a1c159930 0x7f4a1c1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.947+0000 7f4a1a59c700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 0x7f4a1c0c6770 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0011e0 tx=0x7f4a0c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.947+0000 7f4a0a7fc700 1 -- 172.21.15.31:0/2492015504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c007300 con 0x7f4a1c0cd580 2023-10-06T05:25:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.947+0000 7f4a0a7fc700 1 -- 172.21.15.31:0/2492015504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c007460 con 0x7f4a1c0cd580 2023-10-06T05:25:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.947+0000 7f4a0a7fc700 1 -- 172.21.15.31:0/2492015504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c004830 con 0x7f4a1c0cd580 2023-10-06T05:25:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.947+0000 7f4a20fff700 1 -- 172.21.15.31:0/2492015504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c16a680 con 0x7f4a1c0cd580 2023-10-06T05:25:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.947+0000 7f4a20fff700 1 -- 172.21.15.31:0/2492015504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c169dc0 con 0x7f4a1c0cd580 2023-10-06T05:25:53.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.948+0000 7f4a20fff700 1 -- 172.21.15.31:0/2492015504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c16dd00 con 0x7f4a1c0cd580 2023-10-06T05:25:53.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.949+0000 7f4a0a7fc700 1 -- 172.21.15.31:0/2492015504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f4a0c004c20 con 0x7f4a1c0cd580 2023-10-06T05:25:53.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.950+0000 7f4a0a7fc700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f49fc067830 0x7f49fc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:53.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.951+0000 7f4a0a7fc700 1 -- 172.21.15.31:0/2492015504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f4a0c082350 con 0x7f4a1c0cd580 2023-10-06T05:25:53.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.951+0000 7f4a19d9b700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f49fc067830 0x7f49fc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:53.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.952+0000 7f4a19d9b700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f49fc067830 0x7f49fc069ce0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0c53f0 tx=0x7f4a10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:53.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:53.954+0000 7f4a0a7fc700 1 -- 172.21.15.31:0/2492015504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a1c16dd00 con 0x7f4a1c0cd580 2023-10-06T05:25:54.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.160+0000 7f4a20fff700 1 -- 172.21.15.31:0/2492015504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4a1c04c380 con 0x7f4a1c0cd580 2023-10-06T05:25:54.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.164+0000 7f4a0a7fc700 1 -- 172.21.15.31:0/2492015504 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4a0c051010 con 0x7f4a1c0cd580 2023-10-06T05:25:54.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 -- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f49fc067830 msgr2=0x7f49fc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f49fc067830 0x7f49fc069ce0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0c53f0 tx=0x7f4a10008040 comp rx=0 tx=0).stop 2023-10-06T05:25:54.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 -- 172.21.15.31:0/2492015504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 msgr2=0x7f4a1c0c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 0x7f4a1c0c6770 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0011e0 tx=0x7f4a0c007e10 comp rx=0 tx=0).stop 2023-10-06T05:25:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 -- 172.21.15.31:0/2492015504 shutdown_connections 2023-10-06T05:25:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f49fc067830 0x7f49fc069ce0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a1c159930 0x7f4a1c1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1c0cd580 0x7f4a1c0c6770 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 --2- 172.21.15.31:0/2492015504 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a1c159560 0x7f4a1c169880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 -- 172.21.15.31:0/2492015504 >> 172.21.15.31:0/2492015504 conn(0x7f4a1c06ec90 msgr2=0x7f4a1c159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 -- 172.21.15.31:0/2492015504 shutdown_connections 2023-10-06T05:25:54.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.166+0000 7f4a20fff700 1 -- 172.21.15.31:0/2492015504 wait complete. 2023-10-06T05:25:54.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2023-10-06T05:25:54.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.380+0000 7f70eefff700 1 Processor -- start 2023-10-06T05:25:54.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.381+0000 7f70eefff700 1 -- start start 2023-10-06T05:25:54.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.382+0000 7f70eefff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f70e813fb00 0x7f70e806c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.382+0000 7f70eefff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 0x7f70e80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.382+0000 7f70eefff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e806be70 con 0x7f70e813fed0 2023-10-06T05:25:54.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.382+0000 7f70eefff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e806f130 con 0x7f70e813fb00 2023-10-06T05:25:54.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.382+0000 7f70eefff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e806f410 con 0x7f70e806ec60 2023-10-06T05:25:54.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.382+0000 7f70ecd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f70e813fb00 0x7f70e806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.382+0000 7f70e7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f70e813fed0 0x7f70e806e550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48656/0 (socket says 172.21.15.31:48656) 2023-10-06T05:25:54.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.382+0000 7f70e7fff700 1 -- 172.21.15.31:0/945996483 learned_addr learned my addr 172.21.15.31:0/945996483 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:54.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.382+0000 7f70ed59c700 1 --2- 172.21.15.31:0/945996483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 0x7f70e80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.383+0000 7f70ecd9b700 1 -- 172.21.15.31:0/945996483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 msgr2=0x7f70e80cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:54.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.383+0000 7f70ecd9b700 1 --2- 172.21.15.31:0/945996483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 0x7f70e80cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.383+0000 7f70ecd9b700 1 -- 172.21.15.31:0/945996483 >> v1:172.21.15.31:6789/0 conn(0x7f70e813fed0 legacy=0x7f70e806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.383+0000 7f70ecd9b700 1 -- 172.21.15.31:0/945996483 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e806bb00 con 0x7f70e813fb00 2023-10-06T05:25:54.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.383+0000 7f70ed59c700 1 --2- 172.21.15.31:0/945996483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 0x7f70e80cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.383+0000 7f70ecd9b700 1 --2- 172.21.15.31:0/945996483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 0x7f70e806c300 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0098e0 tx=0x7f70dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd4a2aba2b89037 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.383+0000 7f70e77fe700 1 -- 172.21.15.31:0/945996483 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc009e80 con 0x7f70e813fb00 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.383+0000 7f70e77fe700 1 -- 172.21.15.31:0/945996483 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc00b860 con 0x7f70e813fb00 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.383+0000 7f70e77fe700 1 -- 172.21.15.31:0/945996483 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc00bbe0 con 0x7f70e813fb00 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.384+0000 7f70eefff700 1 -- 172.21.15.31:0/945996483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 msgr2=0x7f70e806c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.384+0000 7f70eefff700 1 --2- 172.21.15.31:0/945996483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 0x7f70e806c300 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0098e0 tx=0x7f70dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.384+0000 7f70eefff700 1 -- 172.21.15.31:0/945996483 shutdown_connections 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.384+0000 7f70eefff700 1 --2- 172.21.15.31:0/945996483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 0x7f70e806c300 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.384+0000 7f70eefff700 1 --2- 172.21.15.31:0/945996483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 0x7f70e80cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.384+0000 7f70eefff700 1 -- 172.21.15.31:0/945996483 >> 172.21.15.31:0/945996483 conn(0x7f70e80cdcb0 msgr2=0x7f70e80c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.384+0000 7f70eefff700 1 -- 172.21.15.31:0/945996483 shutdown_connections 2023-10-06T05:25:54.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.384+0000 7f70eefff700 1 -- 172.21.15.31:0/945996483 wait complete. 2023-10-06T05:25:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.385+0000 7f70eefff700 1 Processor -- start 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.386+0000 7f70eefff700 1 -- start start 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70eefff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 0x7f70e813c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70eefff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 0x7f70e815d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70eefff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e813fed0 0x7f70e816a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70eefff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e815dfd0 con 0x7f70e813fb00 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70eefff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e816ad50 con 0x7f70e813fed0 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70eefff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e815dcf0 con 0x7f70e806ec60 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70e7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 0x7f70e815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70e7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 0x7f70e815d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53692/0 (socket says 172.21.15.31:53692) 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70e7fff700 1 -- 172.21.15.31:0/3338385018 learned_addr learned my addr 172.21.15.31:0/3338385018 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70ed59c700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e813fed0 0x7f70e816a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70ecd9b700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 0x7f70e813c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70e7fff700 1 -- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 msgr2=0x7f70e813c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70e7fff700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 0x7f70e813c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70e7fff700 1 -- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e813fed0 msgr2=0x7f70e816a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70e7fff700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e813fed0 0x7f70e816a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70e7fff700 1 -- 172.21.15.31:0/3338385018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc009420 con 0x7f70e813fb00 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.387+0000 7f70e7fff700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 0x7f70e815d660 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f70d400c560 tx=0x7f70d4007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.388+0000 7f70e4ff9700 1 -- 172.21.15.31:0/3338385018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4005b30 con 0x7f70e813fb00 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.388+0000 7f70e4ff9700 1 -- 172.21.15.31:0/3338385018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4005c90 con 0x7f70e813fb00 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.388+0000 7f70e4ff9700 1 -- 172.21.15.31:0/3338385018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4007320 con 0x7f70e813fb00 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.388+0000 7f70eefff700 1 -- 172.21.15.31:0/3338385018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e815bea0 con 0x7f70e813fb00 2023-10-06T05:25:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.388+0000 7f70eefff700 1 -- 172.21.15.31:0/3338385018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e815c460 con 0x7f70e813fb00 2023-10-06T05:25:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.389+0000 7f70eefff700 1 -- 172.21.15.31:0/3338385018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e815bd00 con 0x7f70e813fb00 2023-10-06T05:25:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.390+0000 7f70e4ff9700 1 -- 172.21.15.31:0/3338385018 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f70d401b3b0 con 0x7f70e813fb00 2023-10-06T05:25:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.391+0000 7f70e4ff9700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f70cc067830 0x7f70cc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.391+0000 7f70e4ff9700 1 -- 172.21.15.31:0/3338385018 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f70d4082340 con 0x7f70e813fb00 2023-10-06T05:25:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.392+0000 7f70ecd9b700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f70cc067830 0x7f70cc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.392+0000 7f70ecd9b700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f70cc067830 0x7f70cc069ce0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0091e0 tx=0x7f70dc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.395+0000 7f70e4ff9700 1 -- 172.21.15.31:0/3338385018 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70e815bd00 con 0x7f70e813fb00 2023-10-06T05:25:54.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.678+0000 7f70eefff700 1 -- 172.21.15.31:0/3338385018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) v1 -- 0x7f70e815bd00 con 0x7f70e813fb00 2023-10-06T05:25:54.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.703+0000 7f70e4ff9700 1 -- 172.21.15.31:0/3338385018 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v485) v1 ==== 145+0+0 (secure 0 0 0) 0x7f70e815bd00 con 0x7f70e813fb00 2023-10-06T05:25:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 -- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f70cc067830 msgr2=0x7f70cc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f70cc067830 0x7f70cc069ce0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0091e0 tx=0x7f70dc017000 comp rx=0 tx=0).stop 2023-10-06T05:25:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 -- 172.21.15.31:0/3338385018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 msgr2=0x7f70e815d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 0x7f70e815d660 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f70d400c560 tx=0x7f70d4007f40 comp rx=0 tx=0).stop 2023-10-06T05:25:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 -- 172.21.15.31:0/3338385018 shutdown_connections 2023-10-06T05:25:54.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f70cc067830 0x7f70cc069ce0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e813fed0 0x7f70e816a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e806ec60 0x7f70e813c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 --2- 172.21.15.31:0/3338385018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e813fb00 0x7f70e815d660 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 -- 172.21.15.31:0/3338385018 >> 172.21.15.31:0/3338385018 conn(0x7f70e80cdcb0 msgr2=0x7f70e8158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 -- 172.21.15.31:0/3338385018 shutdown_connections 2023-10-06T05:25:54.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.706+0000 7f70eefff700 1 -- 172.21.15.31:0/3338385018 wait complete. 2023-10-06T05:25:54.725 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2023-10-06T05:25:54.725 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2023-10-06T05:25:54.725 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-10-06T05:25:54.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2023-10-06T05:25:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.920+0000 7f9540bff700 1 Processor -- start 2023-10-06T05:25:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.920+0000 7f9540bff700 1 -- start start 2023-10-06T05:25:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f9540bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f953c068d80 0x7f953c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f9540bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f953c0cc0d0 0x7f953c0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f9540bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953c0c95b0 con 0x7f953c1595a0 2023-10-06T05:25:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f9540bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f953c0c9890 con 0x7f953c068d80 2023-10-06T05:25:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f9540bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f953c0c9b70 con 0x7f953c0cc0d0 2023-10-06T05:25:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f9539d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f953c068d80 0x7f953c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f9539d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f953c068d80 0x7f953c159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53704/0 (socket says 172.21.15.31:53704) 2023-10-06T05:25:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f9539d9b700 1 -- 172.21.15.31:0/1828402775 learned_addr learned my addr 172.21.15.31:0/1828402775 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f953a59c700 1 --1- 172.21.15.31:0/1828402775 >> v1:172.21.15.31:6789/0 conn(0x7f953c1595a0 0x7f953c068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48672/0 (socket says 172.21.15.31:48672) 2023-10-06T05:25:54.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f953ad9d700 1 --2- 172.21.15.31:0/1828402775 >> v2:172.21.15.62:3300/0 conn(0x7f953c0cc0d0 0x7f953c0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f9539d9b700 1 -- 172.21.15.31:0/1828402775 >> v2:172.21.15.62:3300/0 conn(0x7f953c0cc0d0 msgr2=0x7f953c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:54.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.922+0000 7f9539d9b700 1 --2- 172.21.15.31:0/1828402775 >> v2:172.21.15.62:3300/0 conn(0x7f953c0cc0d0 0x7f953c0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.923+0000 7f9539d9b700 1 -- 172.21.15.31:0/1828402775 >> v1:172.21.15.31:6789/0 conn(0x7f953c1595a0 legacy=0x7f953c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.923+0000 7f9539d9b700 1 -- 172.21.15.31:0/1828402775 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c0ca130 con 0x7f953c068d80 2023-10-06T05:25:54.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.923+0000 7f953959a700 1 -- 172.21.15.31:0/1828402775 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998656967 0 0) 0x7f953c0c95b0 con 0x7f953c1595a0 2023-10-06T05:25:54.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.923+0000 7f9539d9b700 1 --2- 172.21.15.31:0/1828402775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c068d80 0x7f953c159970 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f95300099b0 tx=0x7f9530009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2b9f9c014ccceb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.923+0000 7f953959a700 1 -- 172.21.15.31:0/1828402775 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953000b820 con 0x7f953c068d80 2023-10-06T05:25:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.923+0000 7f953959a700 1 -- 172.21.15.31:0/1828402775 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953000b980 con 0x7f953c068d80 2023-10-06T05:25:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.923+0000 7f953959a700 1 -- 172.21.15.31:0/1828402775 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953000bd20 con 0x7f953c068d80 2023-10-06T05:25:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.924+0000 7f9540bff700 1 -- 172.21.15.31:0/1828402775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c068d80 msgr2=0x7f953c159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.924+0000 7f9540bff700 1 --2- 172.21.15.31:0/1828402775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c068d80 0x7f953c159970 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f95300099b0 tx=0x7f9530009250 comp rx=0 tx=0).stop 2023-10-06T05:25:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.924+0000 7f9540bff700 1 -- 172.21.15.31:0/1828402775 shutdown_connections 2023-10-06T05:25:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.924+0000 7f9540bff700 1 --2- 172.21.15.31:0/1828402775 >> v2:172.21.15.62:3300/0 conn(0x7f953c0cc0d0 0x7f953c0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.924+0000 7f9540bff700 1 --2- 172.21.15.31:0/1828402775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c068d80 0x7f953c159970 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.924+0000 7f9540bff700 1 -- 172.21.15.31:0/1828402775 >> 172.21.15.31:0/1828402775 conn(0x7f953c06b620 msgr2=0x7f953c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.924+0000 7f9540bff700 1 -- 172.21.15.31:0/1828402775 shutdown_connections 2023-10-06T05:25:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.924+0000 7f9540bff700 1 -- 172.21.15.31:0/1828402775 wait complete. 2023-10-06T05:25:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.925+0000 7f9540bff700 1 Processor -- start 2023-10-06T05:25:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.925+0000 7f9540bff700 1 -- start start 2023-10-06T05:25:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.926+0000 7f9540bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953c068d80 0x7f953c0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.926+0000 7f9540bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953c0cc0d0 0x7f953c0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.926+0000 7f9540bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c1595a0 0x7f953c147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.926+0000 7f9540bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c169420 con 0x7f953c1595a0 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.926+0000 7f9540bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c0b2260 con 0x7f953c0cc0d0 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.926+0000 7f9540bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953c0b3520 con 0x7f953c068d80 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f953ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c1595a0 0x7f953c147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f953ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c1595a0 0x7f953c147b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53718/0 (socket says 172.21.15.31:53718) 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f953ad9d700 1 -- 172.21.15.31:0/2851199733 learned_addr learned my addr 172.21.15.31:0/2851199733 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f953a59c700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953c068d80 0x7f953c0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f9539d9b700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953c0cc0d0 0x7f953c0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f953ad9d700 1 -- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953c068d80 msgr2=0x7f953c0b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f953ad9d700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953c068d80 0x7f953c0b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f953ad9d700 1 -- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953c0cc0d0 msgr2=0x7f953c0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:54.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f953ad9d700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953c0cc0d0 0x7f953c0b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:54.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f953ad9d700 1 -- 172.21.15.31:0/2851199733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95300094f0 con 0x7f953c1595a0 2023-10-06T05:25:54.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f9539d9b700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953c0cc0d0 0x7f953c0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:54.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f953ad9d700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c1595a0 0x7f953c147b60 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9534009fa0 tx=0x7f9534009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f952a7fc700 1 -- 172.21.15.31:0/2851199733 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953400ed20 con 0x7f953c1595a0 2023-10-06T05:25:54.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f952a7fc700 1 -- 172.21.15.31:0/2851199733 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953400ee80 con 0x7f953c1595a0 2023-10-06T05:25:54.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f952a7fc700 1 -- 172.21.15.31:0/2851199733 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534014670 con 0x7f953c1595a0 2023-10-06T05:25:54.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.927+0000 7f9540bff700 1 -- 172.21.15.31:0/2851199733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953c0b16e0 con 0x7f953c1595a0 2023-10-06T05:25:54.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.928+0000 7f9540bff700 1 -- 172.21.15.31:0/2851199733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953c0b1ca0 con 0x7f953c1595a0 2023-10-06T05:25:54.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.928+0000 7f9540bff700 1 -- 172.21.15.31:0/2851199733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953c169560 con 0x7f953c1595a0 2023-10-06T05:25:54.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.929+0000 7f952a7fc700 1 -- 172.21.15.31:0/2851199733 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f95340149e0 con 0x7f953c1595a0 2023-10-06T05:25:54.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.931+0000 7f952a7fc700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f951c067830 0x7f951c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:54.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.931+0000 7f952a7fc700 1 -- 172.21.15.31:0/2851199733 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f9534083680 con 0x7f953c1595a0 2023-10-06T05:25:54.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.932+0000 7f953a59c700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f951c067830 0x7f951c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:54.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.933+0000 7f953a59c700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f951c067830 0x7f951c069ce0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f952c0011e0 tx=0x7f952c007bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:54.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:54.935+0000 7f952a7fc700 1 -- 172.21.15.31:0/2851199733 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f953c169560 con 0x7f953c1595a0 2023-10-06T05:25:55.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.220+0000 7f9540bff700 1 -- 172.21.15.31:0/2851199733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f953c169560 con 0x7f953c1595a0 2023-10-06T05:25:55.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.713+0000 7f952a7fc700 1 -- 172.21.15.31:0/2851199733 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v486) v1 ==== 166+0+0 (secure 0 0 0) 0x7f953c169560 con 0x7f953c1595a0 2023-10-06T05:25:55.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 -- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f951c067830 msgr2=0x7f951c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f951c067830 0x7f951c069ce0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f952c0011e0 tx=0x7f952c007bf0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 -- 172.21.15.31:0/2851199733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c1595a0 msgr2=0x7f953c147b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c1595a0 0x7f953c147b60 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9534009fa0 tx=0x7f9534009900 comp rx=0 tx=0).stop 2023-10-06T05:25:55.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 -- 172.21.15.31:0/2851199733 shutdown_connections 2023-10-06T05:25:55.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f951c067830 0x7f951c069ce0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f953c1595a0 0x7f953c147b60 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953c068d80 0x7f953c0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 --2- 172.21.15.31:0/2851199733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f953c0cc0d0 0x7f953c0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 -- 172.21.15.31:0/2851199733 >> 172.21.15.31:0/2851199733 conn(0x7f953c06b620 msgr2=0x7f953c157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:55.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 -- 172.21.15.31:0/2851199733 shutdown_connections 2023-10-06T05:25:55.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.716+0000 7f9540bff700 1 -- 172.21.15.31:0/2851199733 wait complete. 2023-10-06T05:25:55.738 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:25:55.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.938+0000 7fab7b3ff700 1 Processor -- start 2023-10-06T05:25:55.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.938+0000 7fab7b3ff700 1 -- start start 2023-10-06T05:25:55.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.939+0000 7fab7b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab74159890 0x7fab740c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.939+0000 7fab7b3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab74159c60 0x7fab74156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.939+0000 7fab7b3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab74158190 con 0x7fab74156fa0 2023-10-06T05:25:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.939+0000 7fab7b3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab740cc7b0 con 0x7fab74159c60 2023-10-06T05:25:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.939+0000 7fab7b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab74157580 con 0x7fab74159890 2023-10-06T05:25:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.940+0000 7fab7899a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab74159c60 0x7fab74156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.940+0000 7fab7999c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fab74156fa0 0x7fab740cc0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48674/0 (socket says 172.21.15.31:48674) 2023-10-06T05:25:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.940+0000 7fab7999c700 1 -- 172.21.15.31:0/1099684039 learned_addr learned my addr 172.21.15.31:0/1099684039 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.940+0000 7fab7919b700 1 --2- 172.21.15.31:0/1099684039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab74159890 0x7fab740c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.940+0000 7fab7899a700 1 -- 172.21.15.31:0/1099684039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab74159890 msgr2=0x7fab740c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:55.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.940+0000 7fab7899a700 1 --2- 172.21.15.31:0/1099684039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab74159890 0x7fab740c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.940+0000 7fab7899a700 1 -- 172.21.15.31:0/1099684039 >> v1:172.21.15.31:6789/0 conn(0x7fab74156fa0 legacy=0x7fab740cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.940+0000 7fab6bfff700 1 -- 172.21.15.31:0/1099684039 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426922719 0 0) 0x7fab74158190 con 0x7fab74156fa0 2023-10-06T05:25:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.940+0000 7fab7899a700 1 -- 172.21.15.31:0/1099684039 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab74157860 con 0x7fab74159c60 2023-10-06T05:25:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.941+0000 7fab7899a700 1 --2- 172.21.15.31:0/1099684039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74159c60 0x7fab74156a60 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fab64009910 tx=0x7fab640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ee8e8395e33514 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.941+0000 7fab6bfff700 1 -- 172.21.15.31:0/1099684039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400b820 con 0x7fab74159c60 2023-10-06T05:25:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.941+0000 7fab6bfff700 1 -- 172.21.15.31:0/1099684039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6400b980 con 0x7fab74159c60 2023-10-06T05:25:55.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.941+0000 7fab6bfff700 1 -- 172.21.15.31:0/1099684039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400bd20 con 0x7fab74159c60 2023-10-06T05:25:55.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.941+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1099684039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74159c60 msgr2=0x7fab74156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.941+0000 7fab7b3ff700 1 --2- 172.21.15.31:0/1099684039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74159c60 0x7fab74156a60 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fab64009910 tx=0x7fab640091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.941+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1099684039 shutdown_connections 2023-10-06T05:25:55.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.941+0000 7fab7b3ff700 1 --2- 172.21.15.31:0/1099684039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab74159890 0x7fab740c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.941+0000 7fab7b3ff700 1 --2- 172.21.15.31:0/1099684039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74159c60 0x7fab74156a60 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.941+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1099684039 >> 172.21.15.31:0/1099684039 conn(0x7fab74067560 msgr2=0x7fab74067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:55.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.942+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1099684039 shutdown_connections 2023-10-06T05:25:55.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.942+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1099684039 wait complete. 2023-10-06T05:25:55.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.943+0000 7fab7b3ff700 1 Processor -- start 2023-10-06T05:25:55.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.943+0000 7fab7b3ff700 1 -- start start 2023-10-06T05:25:55.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.944+0000 7fab7b3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74156fa0 0x7fab740c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.944+0000 7fab7b3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab74159890 0x7fab7416e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.944+0000 7fab7b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab74159c60 0x7fab7414d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.944+0000 7fab7b3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7416e8e0 con 0x7fab74156fa0 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.944+0000 7fab7b3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740c7fc0 con 0x7fab74159890 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.944+0000 7fab7b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab740c7720 con 0x7fab74159c60 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.944+0000 7fab7919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74156fa0 0x7fab740c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.944+0000 7fab7919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74156fa0 0x7fab740c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53740/0 (socket says 172.21.15.31:53740) 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.944+0000 7fab7919b700 1 -- 172.21.15.31:0/1768031609 learned_addr learned my addr 172.21.15.31:0/1768031609 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab7999c700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab74159c60 0x7fab7414d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab7899a700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab74159890 0x7fab7416e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab7919b700 1 -- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab74159c60 msgr2=0x7fab7414d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab7919b700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab74159c60 0x7fab7414d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab7919b700 1 -- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab74159890 msgr2=0x7fab7416e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:55.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab7919b700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab74159890 0x7fab7416e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:55.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab7919b700 1 -- 172.21.15.31:0/1768031609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab64009420 con 0x7fab74156fa0 2023-10-06T05:25:55.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab7919b700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74156fa0 0x7fab740c7020 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fab7000d890 tx=0x7fab70008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:55.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab697fa700 1 -- 172.21.15.31:0/1768031609 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab700125f0 con 0x7fab74156fa0 2023-10-06T05:25:55.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab697fa700 1 -- 172.21.15.31:0/1768031609 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab70012750 con 0x7fab74156fa0 2023-10-06T05:25:55.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab697fa700 1 -- 172.21.15.31:0/1768031609 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70012ad0 con 0x7fab74156fa0 2023-10-06T05:25:55.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1768031609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab740c7a00 con 0x7fab74156fa0 2023-10-06T05:25:55.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.945+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1768031609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7415a550 con 0x7fab74156fa0 2023-10-06T05:25:55.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.946+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1768031609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7415a3b0 con 0x7fab74156fa0 2023-10-06T05:25:55.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.947+0000 7fab697fa700 1 -- 172.21.15.31:0/1768031609 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fab7001e460 con 0x7fab74156fa0 2023-10-06T05:25:55.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.948+0000 7fab697fa700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab58067830 0x7fab58069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:55.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.948+0000 7fab697fa700 1 -- 172.21.15.31:0/1768031609 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fab700836a0 con 0x7fab74156fa0 2023-10-06T05:25:55.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.948+0000 7fab7899a700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab58067830 0x7fab58069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:55.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.949+0000 7fab7899a700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab58067830 0x7fab58069ce0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fab640091e0 tx=0x7fab64017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:55.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:55.952+0000 7fab697fa700 1 -- 172.21.15.31:0/1768031609 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab7415a3b0 con 0x7fab74156fa0 2023-10-06T05:25:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.160+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1768031609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fab7415a3b0 con 0x7fab74156fa0 2023-10-06T05:25:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.164+0000 7fab697fa700 1 -- 172.21.15.31:0/1768031609 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fab7415a3b0 con 0x7fab74156fa0 2023-10-06T05:25:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab58067830 msgr2=0x7fab58069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab58067830 0x7fab58069ce0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fab640091e0 tx=0x7fab64017000 comp rx=0 tx=0).stop 2023-10-06T05:25:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1768031609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74156fa0 msgr2=0x7fab740c7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74156fa0 0x7fab740c7020 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fab7000d890 tx=0x7fab70008040 comp rx=0 tx=0).stop 2023-10-06T05:25:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1768031609 shutdown_connections 2023-10-06T05:25:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fab58067830 0x7fab58069ce0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab74159c60 0x7fab7414d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab74156fa0 0x7fab740c7020 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 --2- 172.21.15.31:0/1768031609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab74159890 0x7fab7416e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1768031609 >> 172.21.15.31:0/1768031609 conn(0x7fab74067560 msgr2=0x7fab74151ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1768031609 shutdown_connections 2023-10-06T05:25:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.166+0000 7fab7b3ff700 1 -- 172.21.15.31:0/1768031609 wait complete. 2023-10-06T05:25:56.186 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:25:56.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:25:56.290 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:25:56.400 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:25:56.400 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:25:56.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:25:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.594+0000 7fc40d3ff700 1 Processor -- start 2023-10-06T05:25:56.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.594+0000 7fc40d3ff700 1 -- start start 2023-10-06T05:25:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.595+0000 7fc40d3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4081595a0 0x7fc408159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.595+0000 7fc40d3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc4080cc0d0 0x7fc408068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.595+0000 7fc40d3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 0x7fc4080c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.595+0000 7fc40d3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc408069010 con 0x7fc4081595a0 2023-10-06T05:25:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.595+0000 7fc40d3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4080c95b0 con 0x7fc4080cc0d0 2023-10-06T05:25:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.595+0000 7fc40d3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080c9890 con 0x7fc408068bb0 2023-10-06T05:25:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.595+0000 7fc406ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4081595a0 0x7fc408159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.596+0000 7fc406ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4081595a0 0x7fc408159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53756/0 (socket says 172.21.15.31:53756) 2023-10-06T05:25:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.596+0000 7fc406ffd700 1 -- 172.21.15.31:0/2996979879 learned_addr learned my addr 172.21.15.31:0/2996979879 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:56.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.596+0000 7fc4077fe700 1 --2- 172.21.15.31:0/2996979879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 0x7fc4080c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.596+0000 7fc4067fc700 1 --2- 172.21.15.31:0/2996979879 >> v2:172.21.15.62:3300/0 conn(0x7fc4080cc0d0 0x7fc408068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.596+0000 7fc406ffd700 1 -- 172.21.15.31:0/2996979879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 msgr2=0x7fc4080c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.596+0000 7fc406ffd700 1 --2- 172.21.15.31:0/2996979879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 0x7fc4080c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.596+0000 7fc406ffd700 1 -- 172.21.15.31:0/2996979879 >> v2:172.21.15.62:3300/0 conn(0x7fc4080cc0d0 msgr2=0x7fc408068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:56.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.596+0000 7fc406ffd700 1 --2- 172.21.15.31:0/2996979879 >> v2:172.21.15.62:3300/0 conn(0x7fc4080cc0d0 0x7fc408068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.596+0000 7fc406ffd700 1 -- 172.21.15.31:0/2996979879 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4080c9b70 con 0x7fc4081595a0 2023-10-06T05:25:56.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.596+0000 7fc406ffd700 1 --2- 172.21.15.31:0/2996979879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4081595a0 0x7fc408159970 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fc40806a360 tx=0x7fc3f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd153f9107dc492 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc405ffb700 1 -- 172.21.15.31:0/2996979879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f800b860 con 0x7fc4081595a0 2023-10-06T05:25:56.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc405ffb700 1 -- 172.21.15.31:0/2996979879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f800b9c0 con 0x7fc4081595a0 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc405ffb700 1 -- 172.21.15.31:0/2996979879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f800bd60 con 0x7fc4081595a0 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc40d3ff700 1 -- 172.21.15.31:0/2996979879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4081595a0 msgr2=0x7fc408159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc40d3ff700 1 --2- 172.21.15.31:0/2996979879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4081595a0 0x7fc408159970 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fc40806a360 tx=0x7fc3f8009250 comp rx=0 tx=0).stop 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc40d3ff700 1 -- 172.21.15.31:0/2996979879 shutdown_connections 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc40d3ff700 1 --2- 172.21.15.31:0/2996979879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 0x7fc4080c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc40d3ff700 1 --2- 172.21.15.31:0/2996979879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4081595a0 0x7fc408159970 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc40d3ff700 1 --2- 172.21.15.31:0/2996979879 >> v2:172.21.15.62:3300/0 conn(0x7fc4080cc0d0 0x7fc408068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc40d3ff700 1 -- 172.21.15.31:0/2996979879 >> 172.21.15.31:0/2996979879 conn(0x7fc40806b620 msgr2=0x7fc40806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc40d3ff700 1 -- 172.21.15.31:0/2996979879 shutdown_connections 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.597+0000 7fc40d3ff700 1 -- 172.21.15.31:0/2996979879 wait complete. 2023-10-06T05:25:56.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.599+0000 7fc40d3ff700 1 Processor -- start 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.599+0000 7fc40d3ff700 1 -- start start 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.600+0000 7fc40d3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 0x7fc408169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.600+0000 7fc40d3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4080cc0d0 0x7fc40815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.600+0000 7fc40d3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4081595a0 0x7fc408152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.600+0000 7fc40d3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc408158ed0 con 0x7fc4080cc0d0 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.600+0000 7fc40d3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4081580d0 con 0x7fc4081595a0 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.600+0000 7fc40d3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc408156c40 con 0x7fc408068bb0 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.600+0000 7fc4067fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4080cc0d0 0x7fc40815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.600+0000 7fc4067fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4080cc0d0 0x7fc40815ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53772/0 (socket says 172.21.15.31:53772) 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.600+0000 7fc4067fc700 1 -- 172.21.15.31:0/3853851492 learned_addr learned my addr 172.21.15.31:0/3853851492 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:56.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc4067fc700 1 -- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 msgr2=0x7fc408169940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc406ffd700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 0x7fc408169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc4077fe700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4081595a0 0x7fc408152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc4067fc700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 0x7fc408169940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc4067fc700 1 -- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4081595a0 msgr2=0x7fc408152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc4067fc700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4081595a0 0x7fc408152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc4067fc700 1 -- 172.21.15.31:0/3853851492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f80094f0 con 0x7fc4080cc0d0 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc4077fe700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4081595a0 0x7fc408152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc406ffd700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 0x7fc408169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc4067fc700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4080cc0d0 0x7fc40815ebc0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00d890 tx=0x7fc3fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc3eeffd700 1 -- 172.21.15.31:0/3853851492 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc009610 con 0x7fc4080cc0d0 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc3eeffd700 1 -- 172.21.15.31:0/3853851492 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc009770 con 0x7fc4080cc0d0 2023-10-06T05:25:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc3eeffd700 1 -- 172.21.15.31:0/3853851492 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc009b30 con 0x7fc4080cc0d0 2023-10-06T05:25:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.601+0000 7fc40d3ff700 1 -- 172.21.15.31:0/3853851492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc408158630 con 0x7fc4080cc0d0 2023-10-06T05:25:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.602+0000 7fc40d3ff700 1 -- 172.21.15.31:0/3853851492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc408158bf0 con 0x7fc4080cc0d0 2023-10-06T05:25:56.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.603+0000 7fc40d3ff700 1 -- 172.21.15.31:0/3853851492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40815c2f0 con 0x7fc4080cc0d0 2023-10-06T05:25:56.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.603+0000 7fc3eeffd700 1 -- 172.21.15.31:0/3853851492 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fc3fc01e460 con 0x7fc4080cc0d0 2023-10-06T05:25:56.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.604+0000 7fc3eeffd700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc3f4067830 0x7fc3f4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:56.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.605+0000 7fc3eeffd700 1 -- 172.21.15.31:0/3853851492 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fc3fc0836f0 con 0x7fc4080cc0d0 2023-10-06T05:25:56.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.605+0000 7fc406ffd700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc3f4067830 0x7fc3f4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:56.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.606+0000 7fc406ffd700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc3f4067830 0x7fc3f4069ce0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc40806a360 tx=0x7fc3f8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:56.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.609+0000 7fc3eeffd700 1 -- 172.21.15.31:0/3853851492 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc40815c2f0 con 0x7fc4080cc0d0 2023-10-06T05:25:56.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.854+0000 7fc40d3ff700 1 -- 172.21.15.31:0/3853851492 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc40815c2f0 con 0x7fc4080cc0d0 2023-10-06T05:25:56.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.855+0000 7fc3eeffd700 1 -- 172.21.15.31:0/3853851492 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+3528 (secure 0 0 0) 0x7fc40815c2f0 con 0x7fc4080cc0d0 2023-10-06T05:25:56.857 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:56.857 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7424,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1489945066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1489945066},{"type":"v1","addr":"172.21.15.62:6834","nonce":1489945066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/660206899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":660206899},{"type":"v1","addr":"172.21.15.62:6835","nonce":660206899}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8862,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1056123914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1056123914},{"type":"v1","addr":"172.21.15.31:6834","nonce":1056123914}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97},{"gid":9180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1264694350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1264694350},{"type":"v1","addr":"172.21.15.31:6835","nonce":1264694350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[]} 2023-10-06T05:25:56.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 -- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc3f4067830 msgr2=0x7fc3f4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc3f4067830 0x7fc3f4069ce0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc40806a360 tx=0x7fc3f8017000 comp rx=0 tx=0).stop 2023-10-06T05:25:56.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 -- 172.21.15.31:0/3853851492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4080cc0d0 msgr2=0x7fc40815ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:56.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4080cc0d0 0x7fc40815ebc0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00d890 tx=0x7fc3fc008040 comp rx=0 tx=0).stop 2023-10-06T05:25:56.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 -- 172.21.15.31:0/3853851492 shutdown_connections 2023-10-06T05:25:56.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc3f4067830 0x7fc3f4069ce0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4081595a0 0x7fc408152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc408068bb0 0x7fc408169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 --2- 172.21.15.31:0/3853851492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4080cc0d0 0x7fc40815ebc0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:56.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 -- 172.21.15.31:0/3853851492 >> 172.21.15.31:0/3853851492 conn(0x7fc40806b620 msgr2=0x7fc40806cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:56.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 -- 172.21.15.31:0/3853851492 shutdown_connections 2023-10-06T05:25:56.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:56.857+0000 7fc40d3ff700 1 -- 172.21.15.31:0/3853851492 wait complete. 2023-10-06T05:25:56.861 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 105 2023-10-06T05:25:56.877 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:25:56.877 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:25:56.877 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:25:56.877 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:25:56.878 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:25:56.878 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:25:56.878 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:25:56.878 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:25:56.878 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:25:56.878 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:25:56.879 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:25:56.879 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:25:56.879 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:25:56.879 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:25:56.879 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:25:56.879 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:25:56.891 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:25:56.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:25:56.891 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:25:56.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:25:56.895 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:25:56.895 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:25:56.896 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:25:56.896 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:25:56.935 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:25:56.936 DEBUG:teuthology.parallel:result is None 2023-10-06T05:25:56.938 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:25:56.938 DEBUG:teuthology.parallel:result is None 2023-10-06T05:25:56.940 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:25:56.941 DEBUG:teuthology.parallel:result is None 2023-10-06T05:25:56.944 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:25:56.944 DEBUG:teuthology.parallel:result is None 2023-10-06T05:25:56.944 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:25:57.083 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:25:57.083 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-10-06T05:25:57.089 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2023-10-06T05:25:57.090 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2023-10-06T05:25:57.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.152+0000 7feb261ff700 1 Processor -- start 2023-10-06T05:25:57.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.152+0000 7feb261ff700 1 -- start start 2023-10-06T05:25:57.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.153+0000 7feb261ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feb201595a0 0x7feb20159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.153+0000 7feb261ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb200cc0d0 0x7feb20068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.153+0000 7feb261ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb20069010 con 0x7feb20068bb0 2023-10-06T05:25:57.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb261ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200c95b0 con 0x7feb200cc0d0 2023-10-06T05:25:57.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb261ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200c9890 con 0x7feb201595a0 2023-10-06T05:25:57.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb1effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb200cc0d0 0x7feb20068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb1ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7feb20068bb0 0x7feb200c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38426/0 (socket says 172.21.15.31:38426) 2023-10-06T05:25:57.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb1effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb200cc0d0 0x7feb20068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51644/0 (socket says 172.21.15.31:51644) 2023-10-06T05:25:57.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb1ffff700 1 -- 172.21.15.31:0/1775970220 learned_addr learned my addr 172.21.15.31:0/1775970220 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:57.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb1f7fe700 1 --2- 172.21.15.31:0/1775970220 >> v2:172.21.15.62:3300/0 conn(0x7feb201595a0 0x7feb20159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb1effd700 1 -- 172.21.15.31:0/1775970220 >> v2:172.21.15.62:3300/0 conn(0x7feb201595a0 msgr2=0x7feb20159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:57.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb1effd700 1 --2- 172.21.15.31:0/1775970220 >> v2:172.21.15.62:3300/0 conn(0x7feb201595a0 0x7feb20159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb1effd700 1 -- 172.21.15.31:0/1775970220 >> v1:172.21.15.31:6789/0 conn(0x7feb20068bb0 legacy=0x7feb200c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb1effd700 1 -- 172.21.15.31:0/1775970220 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200c9b70 con 0x7feb200cc0d0 2023-10-06T05:25:57.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.154+0000 7feb1e7fc700 1 -- 172.21.15.31:0/1775970220 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235329605 0 0) 0x7feb18001490 con 0x7feb20068bb0 2023-10-06T05:25:57.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.155+0000 7feb1effd700 1 --2- 172.21.15.31:0/1775970220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 0x7feb20068670 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7feb2006c8d0 tx=0x7feb140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311e415dae327335 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.155+0000 7feb1e7fc700 1 -- 172.21.15.31:0/1775970220 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14011630 con 0x7feb200cc0d0 2023-10-06T05:25:57.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.155+0000 7feb1e7fc700 1 -- 172.21.15.31:0/1775970220 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb14011790 con 0x7feb200cc0d0 2023-10-06T05:25:57.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.155+0000 7feb1e7fc700 1 -- 172.21.15.31:0/1775970220 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14011b30 con 0x7feb200cc0d0 2023-10-06T05:25:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.155+0000 7feb261ff700 1 -- 172.21.15.31:0/1775970220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 msgr2=0x7feb20068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.155+0000 7feb261ff700 1 --2- 172.21.15.31:0/1775970220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 0x7feb20068670 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7feb2006c8d0 tx=0x7feb140091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.155+0000 7feb261ff700 1 -- 172.21.15.31:0/1775970220 shutdown_connections 2023-10-06T05:25:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.156+0000 7feb261ff700 1 --2- 172.21.15.31:0/1775970220 >> v2:172.21.15.62:3300/0 conn(0x7feb201595a0 0x7feb20159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.156+0000 7feb261ff700 1 --2- 172.21.15.31:0/1775970220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 0x7feb20068670 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.156+0000 7feb261ff700 1 -- 172.21.15.31:0/1775970220 >> 172.21.15.31:0/1775970220 conn(0x7feb2006b620 msgr2=0x7feb2006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.156+0000 7feb261ff700 1 -- 172.21.15.31:0/1775970220 shutdown_connections 2023-10-06T05:25:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.156+0000 7feb261ff700 1 -- 172.21.15.31:0/1775970220 wait complete. 2023-10-06T05:25:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.157+0000 7feb261ff700 1 Processor -- start 2023-10-06T05:25:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.157+0000 7feb261ff700 1 -- start start 2023-10-06T05:25:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.158+0000 7feb261ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb20068bb0 0x7feb2014be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.158+0000 7feb261ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 0x7feb20150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.158+0000 7feb261ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb201595a0 0x7feb20152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.158+0000 7feb261ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200b1090 con 0x7feb200cc0d0 2023-10-06T05:25:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.158+0000 7feb261ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2016a870 con 0x7feb20068bb0 2023-10-06T05:25:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.158+0000 7feb261ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb201515c0 con 0x7feb201595a0 2023-10-06T05:25:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.158+0000 7feb1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 0x7feb20150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.158+0000 7feb1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 0x7feb20150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51650/0 (socket says 172.21.15.31:51650) 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.158+0000 7feb1effd700 1 -- 172.21.15.31:0/1007201737 learned_addr learned my addr 172.21.15.31:0/1007201737 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb1f7fe700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb20068bb0 0x7feb2014be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb1effd700 1 -- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb201595a0 msgr2=0x7feb20152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb1ffff700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb201595a0 0x7feb20152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb1effd700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb201595a0 0x7feb20152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb1effd700 1 -- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb20068bb0 msgr2=0x7feb2014be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb1effd700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb20068bb0 0x7feb2014be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb1effd700 1 -- 172.21.15.31:0/1007201737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14009420 con 0x7feb200cc0d0 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb1ffff700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb201595a0 0x7feb20152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb1effd700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 0x7feb20150be0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7feb2006ee00 tx=0x7feb14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb077fe700 1 -- 172.21.15.31:0/1007201737 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14011b10 con 0x7feb200cc0d0 2023-10-06T05:25:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb077fe700 1 -- 172.21.15.31:0/1007201737 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb140253f0 con 0x7feb200cc0d0 2023-10-06T05:25:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.159+0000 7feb077fe700 1 -- 172.21.15.31:0/1007201737 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14025790 con 0x7feb200cc0d0 2023-10-06T05:25:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.160+0000 7feb261ff700 1 -- 172.21.15.31:0/1007201737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2016a220 con 0x7feb200cc0d0 2023-10-06T05:25:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.160+0000 7feb261ff700 1 -- 172.21.15.31:0/1007201737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2016ad00 con 0x7feb200cc0d0 2023-10-06T05:25:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.161+0000 7feb261ff700 1 -- 172.21.15.31:0/1007201737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2004c380 con 0x7feb200cc0d0 2023-10-06T05:25:57.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.162+0000 7feb077fe700 1 -- 172.21.15.31:0/1007201737 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7feb14025b80 con 0x7feb200cc0d0 2023-10-06T05:25:57.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.162+0000 7feb077fe700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feb0c067830 0x7feb0c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.162+0000 7feb077fe700 1 -- 172.21.15.31:0/1007201737 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5320+0+0 (secure 0 0 0) 0x7feb14084800 con 0x7feb200cc0d0 2023-10-06T05:25:57.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.163+0000 7feb1f7fe700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feb0c067830 0x7feb0c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.163+0000 7feb1f7fe700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feb0c067830 0x7feb0c069ce0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feb10009cb0 tx=0x7feb10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.167+0000 7feb077fe700 1 -- 172.21.15.31:0/1007201737 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb14053440 con 0x7feb200cc0d0 2023-10-06T05:25:57.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.385+0000 7feb261ff700 1 -- 172.21.15.31:0/1007201737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7feb20069e40 con 0x7feb200cc0d0 2023-10-06T05:25:57.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.734+0000 7feb077fe700 1 -- 172.21.15.31:0/1007201737 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v157) v1 ==== 97+0+0 (secure 0 0 0) 0x7feb14015080 con 0x7feb200cc0d0 2023-10-06T05:25:57.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.737+0000 7feb261ff700 1 -- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feb0c067830 msgr2=0x7feb0c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.737+0000 7feb261ff700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feb0c067830 0x7feb0c069ce0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feb10009cb0 tx=0x7feb10008040 comp rx=0 tx=0).stop 2023-10-06T05:25:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.737+0000 7feb261ff700 1 -- 172.21.15.31:0/1007201737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 msgr2=0x7feb20150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.737+0000 7feb261ff700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 0x7feb20150be0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7feb2006ee00 tx=0x7feb14017000 comp rx=0 tx=0).stop 2023-10-06T05:25:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.737+0000 7feb261ff700 1 -- 172.21.15.31:0/1007201737 shutdown_connections 2023-10-06T05:25:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.737+0000 7feb261ff700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feb0c067830 0x7feb0c069ce0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.737+0000 7feb261ff700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb201595a0 0x7feb20152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.737+0000 7feb261ff700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb20068bb0 0x7feb2014be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.737+0000 7feb261ff700 1 --2- 172.21.15.31:0/1007201737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb200cc0d0 0x7feb20150be0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.737+0000 7feb261ff700 1 -- 172.21.15.31:0/1007201737 >> 172.21.15.31:0/1007201737 conn(0x7feb2006b620 msgr2=0x7feb200cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.738+0000 7feb261ff700 1 -- 172.21.15.31:0/1007201737 shutdown_connections 2023-10-06T05:25:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.738+0000 7feb261ff700 1 -- 172.21.15.31:0/1007201737 wait complete. 2023-10-06T05:25:57.741 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-10-06T05:25:57.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:25:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.953+0000 7f303c1ff700 1 Processor -- start 2023-10-06T05:25:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.954+0000 7f303c1ff700 1 -- start start 2023-10-06T05:25:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.955+0000 7f303c1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7f30 0x7f3034069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.955+0000 7f303c1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30340aee70 0x7f303413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.955+0000 7f303c1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30340ca5b0 con 0x7f30340c7b60 2023-10-06T05:25:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.955+0000 7f303c1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30340af2f0 con 0x7f30340aee70 2023-10-06T05:25:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.955+0000 7f303c1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303406f740 con 0x7f30340c7f30 2023-10-06T05:25:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.955+0000 7f303a79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30340aee70 0x7f303413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.955+0000 7f303a79c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30340aee70 0x7f303413f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51654/0 (socket says 172.21.15.31:51654) 2023-10-06T05:25:57.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.955+0000 7f303a79c700 1 -- 172.21.15.31:0/2004705804 learned_addr learned my addr 172.21.15.31:0/2004705804 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.955+0000 7f303979a700 1 --2- 172.21.15.31:0/2004705804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7f30 0x7f3034069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.956+0000 7f303a79c700 1 -- 172.21.15.31:0/2004705804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7f30 msgr2=0x7f3034069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.956+0000 7f303a79c700 1 --2- 172.21.15.31:0/2004705804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7f30 0x7f3034069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.956+0000 7f303a79c700 1 -- 172.21.15.31:0/2004705804 >> v1:172.21.15.62:6789/0 conn(0x7f30340c7b60 legacy=0x7f303406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.956+0000 7f303a79c700 1 -- 172.21.15.31:0/2004705804 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340c9c80 con 0x7f30340aee70 2023-10-06T05:25:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.956+0000 7f303979a700 1 --2- 172.21.15.31:0/2004705804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7f30 0x7f3034069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.956+0000 7f303a79c700 1 --2- 172.21.15.31:0/2004705804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340aee70 0x7f303413f690 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f302c009f20 tx=0x7f302c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b41204b345a55ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.956+0000 7f3038f99700 1 -- 172.21.15.31:0/2004705804 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c00e070 con 0x7f30340aee70 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.956+0000 7f3038f99700 1 -- 172.21.15.31:0/2004705804 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c00ba40 con 0x7f30340aee70 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.956+0000 7f3038f99700 1 -- 172.21.15.31:0/2004705804 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c0092e0 con 0x7f30340aee70 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.957+0000 7f303c1ff700 1 -- 172.21.15.31:0/2004705804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340aee70 msgr2=0x7f303413f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.957+0000 7f303c1ff700 1 --2- 172.21.15.31:0/2004705804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340aee70 0x7f303413f690 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f302c009f20 tx=0x7f302c009840 comp rx=0 tx=0).stop 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.957+0000 7f303c1ff700 1 -- 172.21.15.31:0/2004705804 shutdown_connections 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.957+0000 7f303c1ff700 1 --2- 172.21.15.31:0/2004705804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340aee70 0x7f303413f690 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.957+0000 7f303c1ff700 1 --2- 172.21.15.31:0/2004705804 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7f30 0x7f3034069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.957+0000 7f303c1ff700 1 -- 172.21.15.31:0/2004705804 >> 172.21.15.31:0/2004705804 conn(0x7f30340cc0b0 msgr2=0x7f30340cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.957+0000 7f303c1ff700 1 -- 172.21.15.31:0/2004705804 shutdown_connections 2023-10-06T05:25:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.957+0000 7f303c1ff700 1 -- 172.21.15.31:0/2004705804 wait complete. 2023-10-06T05:25:57.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.958+0000 7f303c1ff700 1 Processor -- start 2023-10-06T05:25:57.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.958+0000 7f303c1ff700 1 -- start start 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.959+0000 7f303c1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30340aee70 0x7f30340ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.959+0000 7f303c1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7b60 0x7f3034155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.959+0000 7f303c1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340c7f30 0x7f30341563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.959+0000 7f303c1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340c4a60 con 0x7f30340c7f30 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.959+0000 7f303c1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340c3e50 con 0x7f30340aee70 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.959+0000 7f303c1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30340c4130 con 0x7f30340c7b60 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f303a79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340c7f30 0x7f30341563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f303a79c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340c7f30 0x7f30341563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51668/0 (socket says 172.21.15.31:51668) 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f303a79c700 1 -- 172.21.15.31:0/2918856812 learned_addr learned my addr 172.21.15.31:0/2918856812 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f303979a700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7b60 0x7f3034155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f3039f9b700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30340aee70 0x7f30340ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f303a79c700 1 -- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7b60 msgr2=0x7f3034155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f303a79c700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7b60 0x7f3034155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f303a79c700 1 -- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30340aee70 msgr2=0x7f30340ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f303a79c700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30340aee70 0x7f30340ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f303a79c700 1 -- 172.21.15.31:0/2918856812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c009ae0 con 0x7f30340c7f30 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.960+0000 7f303a79c700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340c7f30 0x7f30341563a0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f303406cd80 tx=0x7f302c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.961+0000 7f3029ffb700 1 -- 172.21.15.31:0/2918856812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c0032f0 con 0x7f30340c7f30 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.961+0000 7f3029ffb700 1 -- 172.21.15.31:0/2918856812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c012d00 con 0x7f30340c7f30 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.961+0000 7f303c1ff700 1 -- 172.21.15.31:0/2918856812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30340c4410 con 0x7f30340c7f30 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.961+0000 7f3029ffb700 1 -- 172.21.15.31:0/2918856812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c01d670 con 0x7f30340c7f30 2023-10-06T05:25:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.961+0000 7f303c1ff700 1 -- 172.21.15.31:0/2918856812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30340adb20 con 0x7f30340c7f30 2023-10-06T05:25:57.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.962+0000 7f303c1ff700 1 -- 172.21.15.31:0/2918856812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30340add40 con 0x7f30340c7f30 2023-10-06T05:25:57.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.963+0000 7f3029ffb700 1 -- 172.21.15.31:0/2918856812 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f302c01da60 con 0x7f30340c7f30 2023-10-06T05:25:57.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.963+0000 7f3029ffb700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3018067830 0x7f3018069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:57.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.964+0000 7f3029ffb700 1 -- 172.21.15.31:0/2918856812 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f302c084630 con 0x7f30340c7f30 2023-10-06T05:25:57.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.964+0000 7f3039f9b700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3018067830 0x7f3018069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:57.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.964+0000 7f3039f9b700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3018067830 0x7f3018069ce0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3030003a70 tx=0x7f3030007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:57.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:57.968+0000 7f3029ffb700 1 -- 172.21.15.31:0/2918856812 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f302c08c020 con 0x7f30340c7f30 2023-10-06T05:25:58.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.254+0000 7f303c1ff700 1 -- 172.21.15.31:0/2918856812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f303404c380 con 0x7f30340c7f30 2023-10-06T05:25:58.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.254+0000 7f3029ffb700 1 -- 172.21.15.31:0/2918856812 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f302c053270 con 0x7f30340c7f30 2023-10-06T05:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-10-06T05:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lennOJBAACsmMH/Zc9qDsf9HL3qfAGw==", 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-10-06T05:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lrc5dABAAFCfqqeXVBlU64T/6IRNdyQ==", 2023-10-06T05:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-10-06T05:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lgnhZMBAAV0xZjTCM7B0XemSrnhhDVA==", 2023-10-06T05:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-10-06T05:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lM5/iCxAAtBhgSFwlfRB1pGYpM5fxhA==", 2023-10-06T05:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-10-06T05:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAYmB9lr9lPCxAAX6S7CoK10EvblxMOTUlAjA==", 2023-10-06T05:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-10-06T05:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAbmB9lVth2HBAADm9OyzQEa0Vh66IwVwiNXQ==", 2023-10-06T05:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-10-06T05:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAemB9lR6unIBAAUnpo7j1McHYhz/LCcUlrxA==", 2023-10-06T05:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-10-06T05:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAhmB9l7NY4JRAAYEIB2ZssH75ogWkMTDowWg==", 2023-10-06T05:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-10-06T05:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAomB9lg44RARAAvesHK3/w0PX0LnglVRNeOw==", 2023-10-06T05:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-10-06T05:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQArmB9lNNMhBBAAHaM6z43qSwCMZU9nlwqoVw==", 2023-10-06T05:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-10-06T05:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAumB9lV/wiChAAhYPXU1WKhZz09w2uZaIFFQ==", 2023-10-06T05:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-10-06T05:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAxmB9lOc9UDhAAKs279mJLnk98zL8WRxFNFQ==", 2023-10-06T05:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-10-06T05:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-10-06T05:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-10-06T05:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lPQL9FRAAcwI69ckMpIFpzQE0jUZm6g==", 2023-10-06T05:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-10-06T05:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-10-06T05:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lAnKqJBAA5KB5OrclyPPNuQRz2Mj88w==", 2023-10-06T05:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-10-06T05:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lbE1RHRAAAdF+SdhqrqE41cg0P1TGRQ==", 2023-10-06T05:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-10-06T05:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAUmB9lsdTzKxAAHW29jWbb9cE27Em6C1clng==", 2023-10-06T05:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-10-06T05:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-10-06T05:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-10-06T05:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-10-06T05:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-10-06T05:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQASmB9l0KHHKBAApE+v71SKF1xQaZZzKLorog==", 2023-10-06T05:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-10-06T05:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-10-06T05:25:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-10-06T05:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9lVVRcHRAAej/8eHwhY2K4plUUsQEcQg==", 2023-10-06T05:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-10-06T05:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQATmB9llFFzFBAAIkZOHG62gT9wEVYv8ltptg==", 2023-10-06T05:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-10-06T05:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-10-06T05:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-10-06T05:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-10-06T05:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-10-06T05:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:25:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.256+0000 7f303c1ff700 1 -- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3018067830 msgr2=0x7f3018069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.256+0000 7f303c1ff700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3018067830 0x7f3018069ce0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3030003a70 tx=0x7f3030007ef0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.256+0000 7f303c1ff700 1 -- 172.21.15.31:0/2918856812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340c7f30 msgr2=0x7f30341563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.256+0000 7f303c1ff700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340c7f30 0x7f30341563a0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f303406cd80 tx=0x7f302c017000 comp rx=0 tx=0).stop 2023-10-06T05:25:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.257+0000 7f303c1ff700 1 -- 172.21.15.31:0/2918856812 shutdown_connections 2023-10-06T05:25:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.257+0000 7f303c1ff700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3018067830 0x7f3018069ce0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.257+0000 7f303c1ff700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30340c7f30 0x7f30341563a0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.257+0000 7f303c1ff700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30340aee70 0x7f30340ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.257+0000 7f303c1ff700 1 --2- 172.21.15.31:0/2918856812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30340c7b60 0x7f3034155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.257+0000 7f303c1ff700 1 -- 172.21.15.31:0/2918856812 >> 172.21.15.31:0/2918856812 conn(0x7f30340cc0b0 msgr2=0x7f30340b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.257+0000 7f303c1ff700 1 -- 172.21.15.31:0/2918856812 shutdown_connections 2023-10-06T05:25:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.257+0000 7f303c1ff700 1 -- 172.21.15.31:0/2918856812 wait complete. 2023-10-06T05:25:58.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:25:58.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.479+0000 7f79f0dff700 1 Processor -- start 2023-10-06T05:25:58.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.479+0000 7f79f0dff700 1 -- start start 2023-10-06T05:25:58.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.480+0000 7f79f0dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f79ec1595a0 0x7f79ec159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.480+0000 7f79f0dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79ec0cc0d0 0x7f79ec068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.480+0000 7f79f0dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ec069010 con 0x7f79ec068bb0 2023-10-06T05:25:58.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.480+0000 7f79f0dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0c95b0 con 0x7f79ec0cc0d0 2023-10-06T05:25:58.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.480+0000 7f79f0dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0c9890 con 0x7f79ec1595a0 2023-10-06T05:25:58.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79e9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79ec0cc0d0 0x7f79ec068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79e9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79ec0cc0d0 0x7f79ec068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51676/0 (socket says 172.21.15.31:51676) 2023-10-06T05:25:58.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79e9d9b700 1 -- 172.21.15.31:0/2682381606 learned_addr learned my addr 172.21.15.31:0/2682381606 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:58.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79ead9d700 1 --1- 172.21.15.31:0/2682381606 >> v1:172.21.15.31:6789/0 conn(0x7f79ec068bb0 0x7f79ec0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38432/0 (socket says 172.21.15.31:38432) 2023-10-06T05:25:58.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79ea59c700 1 --2- 172.21.15.31:0/2682381606 >> v2:172.21.15.62:3300/0 conn(0x7f79ec1595a0 0x7f79ec159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79e9d9b700 1 -- 172.21.15.31:0/2682381606 >> v2:172.21.15.62:3300/0 conn(0x7f79ec1595a0 msgr2=0x7f79ec159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:58.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79e9d9b700 1 --2- 172.21.15.31:0/2682381606 >> v2:172.21.15.62:3300/0 conn(0x7f79ec1595a0 0x7f79ec159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79e9d9b700 1 -- 172.21.15.31:0/2682381606 >> v1:172.21.15.31:6789/0 conn(0x7f79ec068bb0 legacy=0x7f79ec0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79e9d9b700 1 -- 172.21.15.31:0/2682381606 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0c9b70 con 0x7f79ec0cc0d0 2023-10-06T05:25:58.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79e959a700 1 -- 172.21.15.31:0/2682381606 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502234237 0 0) 0x7f79e4001490 con 0x7f79ec068bb0 2023-10-06T05:25:58.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.481+0000 7f79e9d9b700 1 --2- 172.21.15.31:0/2682381606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec0cc0d0 0x7f79ec068670 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f79ec06c8d0 tx=0x7f79e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353ae6d507267278 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:58.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79e959a700 1 -- 172.21.15.31:0/2682381606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0009e70 con 0x7f79ec0cc0d0 2023-10-06T05:25:58.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79e959a700 1 -- 172.21.15.31:0/2682381606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e000b860 con 0x7f79ec0cc0d0 2023-10-06T05:25:58.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79e959a700 1 -- 172.21.15.31:0/2682381606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000bc00 con 0x7f79ec0cc0d0 2023-10-06T05:25:58.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79f0dff700 1 -- 172.21.15.31:0/2682381606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec0cc0d0 msgr2=0x7f79ec068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79f0dff700 1 --2- 172.21.15.31:0/2682381606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec0cc0d0 0x7f79ec068670 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f79ec06c8d0 tx=0x7f79e0009250 comp rx=0 tx=0).stop 2023-10-06T05:25:58.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79f0dff700 1 -- 172.21.15.31:0/2682381606 shutdown_connections 2023-10-06T05:25:58.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79f0dff700 1 --2- 172.21.15.31:0/2682381606 >> v2:172.21.15.62:3300/0 conn(0x7f79ec1595a0 0x7f79ec159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79f0dff700 1 --2- 172.21.15.31:0/2682381606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec0cc0d0 0x7f79ec068670 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79f0dff700 1 -- 172.21.15.31:0/2682381606 >> 172.21.15.31:0/2682381606 conn(0x7f79ec06b620 msgr2=0x7f79ec06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:58.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79f0dff700 1 -- 172.21.15.31:0/2682381606 shutdown_connections 2023-10-06T05:25:58.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.482+0000 7f79f0dff700 1 -- 172.21.15.31:0/2682381606 wait complete. 2023-10-06T05:25:58.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.484+0000 7f79f0dff700 1 Processor -- start 2023-10-06T05:25:58.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.484+0000 7f79f0dff700 1 -- start start 2023-10-06T05:25:58.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79f0dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec068bb0 0x7f79ec155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79f0dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79ec0cc0d0 0x7f79ec0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79f0dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ec1595a0 0x7f79ec15b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79f0dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1555f0 con 0x7f79ec068bb0 2023-10-06T05:25:58.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79f0dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec155fc0 con 0x7f79ec1595a0 2023-10-06T05:25:58.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79f0dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1549e0 con 0x7f79ec0cc0d0 2023-10-06T05:25:58.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec068bb0 0x7f79ec155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec068bb0 0x7f79ec155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51684/0 (socket says 172.21.15.31:51684) 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79ea59c700 1 -- 172.21.15.31:0/681999666 learned_addr learned my addr 172.21.15.31:0/681999666 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79ead9d700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ec1595a0 0x7f79ec15b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79e9d9b700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79ec0cc0d0 0x7f79ec0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79ea59c700 1 -- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79ec0cc0d0 msgr2=0x7f79ec0b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79ea59c700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79ec0cc0d0 0x7f79ec0b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79ea59c700 1 -- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ec1595a0 msgr2=0x7f79ec15b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79ea59c700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ec1595a0 0x7f79ec15b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79ea59c700 1 -- 172.21.15.31:0/681999666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e00094f0 con 0x7f79ec068bb0 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79ead9d700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ec1595a0 0x7f79ec15b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:58.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.485+0000 7f79e9d9b700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79ec0cc0d0 0x7f79ec0b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:58.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.486+0000 7f79ea59c700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec068bb0 0x7f79ec155a80 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00ec80 tx=0x7f79dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:58.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.486+0000 7f79da7fc700 1 -- 172.21.15.31:0/681999666 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc00c9d0 con 0x7f79ec068bb0 2023-10-06T05:25:58.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.486+0000 7f79da7fc700 1 -- 172.21.15.31:0/681999666 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc00cb50 con 0x7f79ec068bb0 2023-10-06T05:25:58.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.486+0000 7f79da7fc700 1 -- 172.21.15.31:0/681999666 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc013470 con 0x7f79ec068bb0 2023-10-06T05:25:58.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.486+0000 7f79f0dff700 1 -- 172.21.15.31:0/681999666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec154cc0 con 0x7f79ec068bb0 2023-10-06T05:25:58.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.486+0000 7f79f0dff700 1 -- 172.21.15.31:0/681999666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec155280 con 0x7f79ec068bb0 2023-10-06T05:25:58.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.487+0000 7f79f0dff700 1 -- 172.21.15.31:0/681999666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec1558d0 con 0x7f79ec068bb0 2023-10-06T05:25:58.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.488+0000 7f79da7fc700 1 -- 172.21.15.31:0/681999666 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f79dc013860 con 0x7f79ec068bb0 2023-10-06T05:25:58.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.489+0000 7f79da7fc700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f79cc067830 0x7f79cc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.489+0000 7f79da7fc700 1 -- 172.21.15.31:0/681999666 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f79dc083450 con 0x7f79ec068bb0 2023-10-06T05:25:58.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.490+0000 7f79e9d9b700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f79cc067830 0x7f79cc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.490+0000 7f79e9d9b700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f79cc067830 0x7f79cc069ce0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f79ec06c8d0 tx=0x7f79e0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.493+0000 7f79da7fc700 1 -- 172.21.15.31:0/681999666 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79dc051ea0 con 0x7f79ec068bb0 2023-10-06T05:25:58.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.697+0000 7f79f0dff700 1 -- 172.21.15.31:0/681999666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79ec04c380 con 0x7f79ec068bb0 2023-10-06T05:25:58.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.698+0000 7f79da7fc700 1 -- 172.21.15.31:0/681999666 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+13752 (secure 0 0 0) 0x7f79dc017090 con 0x7f79ec068bb0 2023-10-06T05:25:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:25:58.700 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":159,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:25:57.742187+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:25:58.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.700+0000 7f79f0dff700 1 -- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f79cc067830 msgr2=0x7f79cc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.700+0000 7f79f0dff700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f79cc067830 0x7f79cc069ce0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f79ec06c8d0 tx=0x7f79e0017000 comp rx=0 tx=0).stop 2023-10-06T05:25:58.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.700+0000 7f79f0dff700 1 -- 172.21.15.31:0/681999666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec068bb0 msgr2=0x7f79ec155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.700+0000 7f79f0dff700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec068bb0 0x7f79ec155a80 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00ec80 tx=0x7f79dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.701+0000 7f79f0dff700 1 -- 172.21.15.31:0/681999666 shutdown_connections 2023-10-06T05:25:58.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.701+0000 7f79f0dff700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f79cc067830 0x7f79cc069ce0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.701+0000 7f79f0dff700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79ec1595a0 0x7f79ec15b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.701+0000 7f79f0dff700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79ec068bb0 0x7f79ec155a80 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.701+0000 7f79f0dff700 1 --2- 172.21.15.31:0/681999666 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79ec0cc0d0 0x7f79ec0b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.701+0000 7f79f0dff700 1 -- 172.21.15.31:0/681999666 >> 172.21.15.31:0/681999666 conn(0x7f79ec06b620 msgr2=0x7f79ec1537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:58.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.701+0000 7f79f0dff700 1 -- 172.21.15.31:0/681999666 shutdown_connections 2023-10-06T05:25:58.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.701+0000 7f79f0dff700 1 -- 172.21.15.31:0/681999666 wait complete. 2023-10-06T05:25:58.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:25:58.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:25:58.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.917+0000 7f52a9bff700 1 Processor -- start 2023-10-06T05:25:58.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.917+0000 7f52a9bff700 1 -- start start 2023-10-06T05:25:58.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a9bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52a40b0b90 0x7f52a40af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a9bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a40af6c0 0x7f52a40adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a9bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a40ad690 con 0x7f52a4153d40 2023-10-06T05:25:58.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a9bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a40acd60 con 0x7f52a40b0b90 2023-10-06T05:25:58.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a9bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40ad040 con 0x7f52a40af6c0 2023-10-06T05:25:58.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52a40b0b90 0x7f52a40af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a37fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52a4153d40 0x7f52a40b0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38448/0 (socket says 172.21.15.31:38448) 2023-10-06T05:25:58.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a37fe700 1 -- 172.21.15.31:0/2170427543 learned_addr learned my addr 172.21.15.31:0/2170427543 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:58.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a3fff700 1 --2- 172.21.15.31:0/2170427543 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a40af6c0 0x7f52a40adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a2ffd700 1 -- 172.21.15.31:0/2170427543 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a40af6c0 msgr2=0x7f52a40adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:58.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a2ffd700 1 --2- 172.21.15.31:0/2170427543 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a40af6c0 0x7f52a40adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a2ffd700 1 -- 172.21.15.31:0/2170427543 >> v1:172.21.15.31:6789/0 conn(0x7f52a4153d40 legacy=0x7f52a40b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.920+0000 7f52a2ffd700 1 -- 172.21.15.31:0/2170427543 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a40ad320 con 0x7f52a40b0b90 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.919+0000 7f52a27fc700 1 -- 172.21.15.31:0/2170427543 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225617466 0 0) 0x7f52a40ad690 con 0x7f52a4153d40 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.920+0000 7f52a2ffd700 1 --2- 172.21.15.31:0/2170427543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40b0b90 0x7f52a40af180 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f52980098e0 tx=0x7f52980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238fe641ea6cbb35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.920+0000 7f52a27fc700 1 -- 172.21.15.31:0/2170427543 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298009e80 con 0x7f52a40b0b90 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.920+0000 7f52a27fc700 1 -- 172.21.15.31:0/2170427543 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529800b860 con 0x7f52a40b0b90 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.920+0000 7f52a27fc700 1 -- 172.21.15.31:0/2170427543 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529800bbe0 con 0x7f52a40b0b90 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.920+0000 7f52a9bff700 1 -- 172.21.15.31:0/2170427543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40b0b90 msgr2=0x7f52a40af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.920+0000 7f52a9bff700 1 --2- 172.21.15.31:0/2170427543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40b0b90 0x7f52a40af180 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f52980098e0 tx=0x7f52980091a0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.921+0000 7f52a9bff700 1 -- 172.21.15.31:0/2170427543 shutdown_connections 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.921+0000 7f52a9bff700 1 --2- 172.21.15.31:0/2170427543 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a40af6c0 0x7f52a40adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.921+0000 7f52a9bff700 1 --2- 172.21.15.31:0/2170427543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40b0b90 0x7f52a40af180 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.921+0000 7f52a9bff700 1 -- 172.21.15.31:0/2170427543 >> 172.21.15.31:0/2170427543 conn(0x7f52a4067560 msgr2=0x7f52a4067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.921+0000 7f52a9bff700 1 -- 172.21.15.31:0/2170427543 shutdown_connections 2023-10-06T05:25:58.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.921+0000 7f52a9bff700 1 -- 172.21.15.31:0/2170427543 wait complete. 2023-10-06T05:25:58.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.922+0000 7f52a9bff700 1 Processor -- start 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.922+0000 7f52a9bff700 1 -- start start 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.923+0000 7f52a9bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40af6c0 0x7f52a414bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.923+0000 7f52a9bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52a40b0b90 0x7f52a415c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.923+0000 7f52a9bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a4153d40 0x7f52a40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.923+0000 7f52a9bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a416bb90 con 0x7f52a40af6c0 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.923+0000 7f52a9bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a416af80 con 0x7f52a40b0b90 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.923+0000 7f52a9bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a416b260 con 0x7f52a4153d40 2023-10-06T05:25:58.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40af6c0 0x7f52a414bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40af6c0 0x7f52a414bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51712/0 (socket says 172.21.15.31:51712) 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a37fe700 1 -- 172.21.15.31:0/1374930879 learned_addr learned my addr 172.21.15.31:0/1374930879 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a3fff700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a4153d40 0x7f52a40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a2ffd700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52a40b0b90 0x7f52a415c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a37fe700 1 -- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a4153d40 msgr2=0x7f52a40cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a37fe700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a4153d40 0x7f52a40cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a37fe700 1 -- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52a40b0b90 msgr2=0x7f52a415c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a37fe700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52a40b0b90 0x7f52a415c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a37fe700 1 -- 172.21.15.31:0/1374930879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5298009420 con 0x7f52a40af6c0 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a2ffd700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52a40b0b90 0x7f52a415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:58.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a3fff700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a4153d40 0x7f52a40cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.924+0000 7f52a37fe700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40af6c0 0x7f52a414bf60 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f529400c5b0 tx=0x7f5294001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.925+0000 7f528b7fe700 1 -- 172.21.15.31:0/1374930879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294005b30 con 0x7f52a40af6c0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.925+0000 7f528b7fe700 1 -- 172.21.15.31:0/1374930879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5294005c90 con 0x7f52a40af6c0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.925+0000 7f528b7fe700 1 -- 172.21.15.31:0/1374930879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294007300 con 0x7f52a40af6c0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.925+0000 7f52a9bff700 1 -- 172.21.15.31:0/1374930879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a416b540 con 0x7f52a40af6c0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.925+0000 7f52a9bff700 1 -- 172.21.15.31:0/1374930879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a4157020 con 0x7f52a40af6c0 2023-10-06T05:25:58.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.926+0000 7f52a9bff700 1 -- 172.21.15.31:0/1374930879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a404c380 con 0x7f52a40af6c0 2023-10-06T05:25:58.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.927+0000 7f528b7fe700 1 -- 172.21.15.31:0/1374930879 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f529401b3b0 con 0x7f52a40af6c0 2023-10-06T05:25:58.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.927+0000 7f528b7fe700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5290067830 0x7f5290069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:58.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.928+0000 7f528b7fe700 1 -- 172.21.15.31:0/1374930879 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f52940823c0 con 0x7f52a40af6c0 2023-10-06T05:25:58.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.928+0000 7f52a2ffd700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5290067830 0x7f5290069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:58.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.929+0000 7f52a2ffd700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5290067830 0x7f5290069ce0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f52a40b0fb0 tx=0x7f5298017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:58.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:58.932+0000 7f528b7fe700 1 -- 172.21.15.31:0/1374930879 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5294051000 con 0x7f52a40af6c0 2023-10-06T05:25:59.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.130+0000 7f52a9bff700 1 -- 172.21.15.31:0/1374930879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f52a40afed0 con 0x7f52a40af6c0 2023-10-06T05:25:59.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.130+0000 7f528b7fe700 1 -- 172.21.15.31:0/1374930879 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v159) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5294054620 con 0x7f52a40af6c0 2023-10-06T05:25:59.132 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:25:59.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.132+0000 7f52a9bff700 1 -- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5290067830 msgr2=0x7f5290069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.132+0000 7f52a9bff700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5290067830 0x7f5290069ce0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f52a40b0fb0 tx=0x7f5298017000 comp rx=0 tx=0).stop 2023-10-06T05:25:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.132+0000 7f52a9bff700 1 -- 172.21.15.31:0/1374930879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40af6c0 msgr2=0x7f52a414bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.132+0000 7f52a9bff700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40af6c0 0x7f52a414bf60 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f529400c5b0 tx=0x7f5294001070 comp rx=0 tx=0).stop 2023-10-06T05:25:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.132+0000 7f52a9bff700 1 -- 172.21.15.31:0/1374930879 shutdown_connections 2023-10-06T05:25:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.132+0000 7f52a9bff700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5290067830 0x7f5290069ce0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.132+0000 7f52a9bff700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52a4153d40 0x7f52a40cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.132+0000 7f52a9bff700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40af6c0 0x7f52a414bf60 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.132+0000 7f52a9bff700 1 --2- 172.21.15.31:0/1374930879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52a40b0b90 0x7f52a415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.132+0000 7f52a9bff700 1 -- 172.21.15.31:0/1374930879 >> 172.21.15.31:0/1374930879 conn(0x7f52a4067560 msgr2=0x7f52a414c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.133+0000 7f52a9bff700 1 -- 172.21.15.31:0/1374930879 shutdown_connections 2023-10-06T05:25:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.133+0000 7f52a9bff700 1 -- 172.21.15.31:0/1374930879 wait complete. 2023-10-06T05:25:59.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:25:59.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.347+0000 7fd99e3ff700 1 Processor -- start 2023-10-06T05:25:59.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.347+0000 7fd99e3ff700 1 -- start start 2023-10-06T05:25:59.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.348+0000 7fd99e3ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd998159930 0x7fd9980b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.348+0000 7fd99e3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9980cd580 0x7fd9980b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.348+0000 7fd99e3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99815d2d0 con 0x7fd998159560 2023-10-06T05:25:59.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.348+0000 7fd99e3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99815d5b0 con 0x7fd9980cd580 2023-10-06T05:25:59.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.349+0000 7fd99e3ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99815d890 con 0x7fd998159930 2023-10-06T05:25:59.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.349+0000 7fd99c99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9980cd580 0x7fd9980b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.349+0000 7fd99c99c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9980cd580 0x7fd9980b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51726/0 (socket says 172.21.15.31:51726) 2023-10-06T05:25:59.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.349+0000 7fd99c99c700 1 -- 172.21.15.31:0/486142474 learned_addr learned my addr 172.21.15.31:0/486142474 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:59.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.349+0000 7fd9977fe700 1 --2- 172.21.15.31:0/486142474 >> v2:172.21.15.62:3300/0 conn(0x7fd998159930 0x7fd9980b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.349+0000 7fd99c99c700 1 -- 172.21.15.31:0/486142474 >> v2:172.21.15.62:3300/0 conn(0x7fd998159930 msgr2=0x7fd9980b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:25:59.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.349+0000 7fd99c99c700 1 --2- 172.21.15.31:0/486142474 >> v2:172.21.15.62:3300/0 conn(0x7fd998159930 0x7fd9980b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.349+0000 7fd99c99c700 1 -- 172.21.15.31:0/486142474 >> v1:172.21.15.62:6789/0 conn(0x7fd998159560 legacy=0x7fd9980cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.349+0000 7fd99c99c700 1 -- 172.21.15.31:0/486142474 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99815db70 con 0x7fd9980cd580 2023-10-06T05:25:59.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.350+0000 7fd99c99c700 1 --2- 172.21.15.31:0/486142474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9980cd580 0x7fd9980b29b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd998089160 tx=0x7fd990009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c6a0c339aedc07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:59.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.350+0000 7fd996ffd700 1 -- 172.21.15.31:0/486142474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99000e070 con 0x7fd9980cd580 2023-10-06T05:25:59.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.350+0000 7fd996ffd700 1 -- 172.21.15.31:0/486142474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99000ba50 con 0x7fd9980cd580 2023-10-06T05:25:59.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.350+0000 7fd996ffd700 1 -- 172.21.15.31:0/486142474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9900092e0 con 0x7fd9980cd580 2023-10-06T05:25:59.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.350+0000 7fd99e3ff700 1 -- 172.21.15.31:0/486142474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9980cd580 msgr2=0x7fd9980b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.350+0000 7fd99e3ff700 1 --2- 172.21.15.31:0/486142474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9980cd580 0x7fd9980b29b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd998089160 tx=0x7fd990009840 comp rx=0 tx=0).stop 2023-10-06T05:25:59.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.350+0000 7fd99e3ff700 1 -- 172.21.15.31:0/486142474 shutdown_connections 2023-10-06T05:25:59.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.350+0000 7fd99e3ff700 1 --2- 172.21.15.31:0/486142474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9980cd580 0x7fd9980b29b0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.351+0000 7fd99e3ff700 1 --2- 172.21.15.31:0/486142474 >> v2:172.21.15.62:3300/0 conn(0x7fd998159930 0x7fd9980b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.351+0000 7fd99e3ff700 1 -- 172.21.15.31:0/486142474 >> 172.21.15.31:0/486142474 conn(0x7fd99806ec90 msgr2=0x7fd99806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:59.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.351+0000 7fd99e3ff700 1 -- 172.21.15.31:0/486142474 shutdown_connections 2023-10-06T05:25:59.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.351+0000 7fd99e3ff700 1 -- 172.21.15.31:0/486142474 wait complete. 2023-10-06T05:25:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.352+0000 7fd99e3ff700 1 Processor -- start 2023-10-06T05:25:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.352+0000 7fd99e3ff700 1 -- start start 2023-10-06T05:25:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.353+0000 7fd99e3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9980cd580 0x7fd9980cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.353+0000 7fd99e3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd998159560 0x7fd9980c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.353+0000 7fd99e3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd998159930 0x7fd99816a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.353+0000 7fd99e3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9980d0b70 con 0x7fd998159560 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.353+0000 7fd99e3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9980cf0d0 con 0x7fd998159930 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.353+0000 7fd99e3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9980c59e0 con 0x7fd9980cd580 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.353+0000 7fd9977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd998159560 0x7fd9980c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.353+0000 7fd9977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd998159560 0x7fd9980c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51730/0 (socket says 172.21.15.31:51730) 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.353+0000 7fd9977fe700 1 -- 172.21.15.31:0/2503597929 learned_addr learned my addr 172.21.15.31:0/2503597929 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd99c99c700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd998159930 0x7fd99816a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd9977fe700 1 -- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9980cd580 msgr2=0x7fd9980cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd997fff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9980cd580 0x7fd9980cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd9977fe700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9980cd580 0x7fd9980cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd9977fe700 1 -- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd998159930 msgr2=0x7fd99816a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd9977fe700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd998159930 0x7fd99816a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd9977fe700 1 -- 172.21.15.31:0/2503597929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd990009ae0 con 0x7fd998159560 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd99c99c700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd998159930 0x7fd99816a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd997fff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9980cd580 0x7fd9980cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd9977fe700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd998159560 0x7fd9980c54a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00d860 tx=0x7fd98c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd97ffff700 1 -- 172.21.15.31:0/2503597929 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c009610 con 0x7fd998159560 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd97ffff700 1 -- 172.21.15.31:0/2503597929 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c009770 con 0x7fd998159560 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd99e3ff700 1 -- 172.21.15.31:0/2503597929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99816a9b0 con 0x7fd998159560 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.354+0000 7fd97ffff700 1 -- 172.21.15.31:0/2503597929 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c009b30 con 0x7fd998159560 2023-10-06T05:25:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.355+0000 7fd99e3ff700 1 -- 172.21.15.31:0/2503597929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9980ce7a0 con 0x7fd998159560 2023-10-06T05:25:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.355+0000 7fd99e3ff700 1 -- 172.21.15.31:0/2503597929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99804c380 con 0x7fd998159560 2023-10-06T05:25:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.356+0000 7fd97ffff700 1 -- 172.21.15.31:0/2503597929 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fd98c01e460 con 0x7fd998159560 2023-10-06T05:25:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.357+0000 7fd97ffff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd984067830 0x7fd984069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.357+0000 7fd97ffff700 1 -- 172.21.15.31:0/2503597929 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fd98c0836f0 con 0x7fd998159560 2023-10-06T05:25:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.358+0000 7fd997fff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd984067830 0x7fd984069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.358+0000 7fd997fff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd984067830 0x7fd984069ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd99816abd0 tx=0x7fd988009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:59.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.362+0000 7fd97ffff700 1 -- 172.21.15.31:0/2503597929 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd98c08c080 con 0x7fd998159560 2023-10-06T05:25:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.564+0000 7fd99e3ff700 1 -- 172.21.15.31:0/2503597929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9980cf210 con 0x7fd998159560 2023-10-06T05:25:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.565+0000 7fd97ffff700 1 -- 172.21.15.31:0/2503597929 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd98c052140 con 0x7fd998159560 2023-10-06T05:25:59.567 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:25:59.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 -- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd984067830 msgr2=0x7fd984069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd984067830 0x7fd984069ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd99816abd0 tx=0x7fd988009040 comp rx=0 tx=0).stop 2023-10-06T05:25:59.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 -- 172.21.15.31:0/2503597929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd998159560 msgr2=0x7fd9980c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd998159560 0x7fd9980c54a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00d860 tx=0x7fd98c008040 comp rx=0 tx=0).stop 2023-10-06T05:25:59.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 -- 172.21.15.31:0/2503597929 shutdown_connections 2023-10-06T05:25:59.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd984067830 0x7fd984069ce0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd998159930 0x7fd99816a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9980cd580 0x7fd9980cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 --2- 172.21.15.31:0/2503597929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd998159560 0x7fd9980c54a0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 -- 172.21.15.31:0/2503597929 >> 172.21.15.31:0/2503597929 conn(0x7fd99806ec90 msgr2=0x7fd9980cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:59.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 -- 172.21.15.31:0/2503597929 shutdown_connections 2023-10-06T05:25:59.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.567+0000 7fd99e3ff700 1 -- 172.21.15.31:0/2503597929 wait complete. 2023-10-06T05:25:59.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:25:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.780+0000 7f2c987ff700 1 Processor -- start 2023-10-06T05:25:59.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.780+0000 7f2c987ff700 1 -- start start 2023-10-06T05:25:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.781+0000 7f2c987ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c9013a1e0 0x7f2c900c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.781+0000 7f2c987ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c900c71b0 con 0x7f2c9013a5b0 2023-10-06T05:25:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.781+0000 7f2c987ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c900c7490 con 0x7f2c9006db50 2023-10-06T05:25:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.781+0000 7f2c987ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c900c7770 con 0x7f2c9013a1e0 2023-10-06T05:25:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.782+0000 7f2c9659b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c9013a1e0 0x7f2c900c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.782+0000 7f2c95d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2c9013a5b0 0x7f2c9006d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38456/0 (socket says 172.21.15.31:38456) 2023-10-06T05:25:59.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.782+0000 7f2c95d9a700 1 -- 172.21.15.31:0/1772347159 learned_addr learned my addr 172.21.15.31:0/1772347159 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:25:59.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.782+0000 7f2c9659b700 1 -- 172.21.15.31:0/1772347159 >> v1:172.21.15.62:6789/0 conn(0x7f2c9006db50 legacy=0x7f2c90069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.782+0000 7f2c9659b700 1 -- 172.21.15.31:0/1772347159 >> v1:172.21.15.31:6789/0 conn(0x7f2c9013a5b0 legacy=0x7f2c9006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.782+0000 7f2c9659b700 1 -- 172.21.15.31:0/1772347159 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c900c7a50 con 0x7f2c9013a1e0 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.782+0000 7f2c95599700 1 -- 172.21.15.31:0/1772347159 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166643942 0 0) 0x7f2c900c71b0 con 0x7f2c9013a5b0 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.783+0000 7f2c9659b700 1 --2- 172.21.15.31:0/1772347159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9013a1e0 0x7f2c900c66f0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0098d0 tx=0x7f2c8c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fb6421b2eacfae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.783+0000 7f2c95599700 1 -- 172.21.15.31:0/1772347159 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c009e70 con 0x7f2c9013a1e0 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.783+0000 7f2c95599700 1 -- 172.21.15.31:0/1772347159 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c005c10 con 0x7f2c9013a1e0 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.783+0000 7f2c95599700 1 -- 172.21.15.31:0/1772347159 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00b650 con 0x7f2c9013a1e0 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.783+0000 7f2c987ff700 1 -- 172.21.15.31:0/1772347159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9013a1e0 msgr2=0x7f2c900c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.783+0000 7f2c987ff700 1 --2- 172.21.15.31:0/1772347159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9013a1e0 0x7f2c900c66f0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0098d0 tx=0x7f2c8c009120 comp rx=0 tx=0).stop 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.783+0000 7f2c987ff700 1 -- 172.21.15.31:0/1772347159 shutdown_connections 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.783+0000 7f2c987ff700 1 --2- 172.21.15.31:0/1772347159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9013a1e0 0x7f2c900c66f0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.783+0000 7f2c987ff700 1 -- 172.21.15.31:0/1772347159 >> 172.21.15.31:0/1772347159 conn(0x7f2c900cc0b0 msgr2=0x7f2c900cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:25:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.784+0000 7f2c987ff700 1 -- 172.21.15.31:0/1772347159 shutdown_connections 2023-10-06T05:25:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.784+0000 7f2c987ff700 1 -- 172.21.15.31:0/1772347159 wait complete. 2023-10-06T05:25:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.785+0000 7f2c987ff700 1 Processor -- start 2023-10-06T05:25:59.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.785+0000 7f2c987ff700 1 -- start start 2023-10-06T05:25:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c987ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9006db50 0x7f2c9013a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c987ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9013a1e0 0x7f2c9016af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c987ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9013a5b0 0x7f2c9013d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c987ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c901512c0 con 0x7f2c9006db50 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c987ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9013b940 con 0x7f2c9013a1e0 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c987ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9013cb70 con 0x7f2c9013a5b0 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c9659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9006db50 0x7f2c9013a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c9659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9006db50 0x7f2c9013a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51750/0 (socket says 172.21.15.31:51750) 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c9659b700 1 -- 172.21.15.31:0/221125100 learned_addr learned my addr 172.21.15.31:0/221125100 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c95d9a700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9013a1e0 0x7f2c9016af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.786+0000 7f2c96d9c700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9013a5b0 0x7f2c9013d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c9659b700 1 -- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9013a5b0 msgr2=0x7f2c9013d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c9659b700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9013a5b0 0x7f2c9013d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c9659b700 1 -- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9013a1e0 msgr2=0x7f2c9016af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:25:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c9659b700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9013a1e0 0x7f2c9016af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:25:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c9659b700 1 -- 172.21.15.31:0/221125100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0093c0 con 0x7f2c9006db50 2023-10-06T05:25:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c96d9c700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9013a5b0 0x7f2c9013d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:25:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c9659b700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9006db50 0x7f2c9013a9c0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009390 tx=0x7f2c8c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c867fc700 1 -- 172.21.15.31:0/221125100 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c006460 con 0x7f2c9006db50 2023-10-06T05:25:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c867fc700 1 -- 172.21.15.31:0/221125100 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c0065c0 con 0x7f2c9006db50 2023-10-06T05:25:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c867fc700 1 -- 172.21.15.31:0/221125100 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c01b480 con 0x7f2c9006db50 2023-10-06T05:25:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.787+0000 7f2c987ff700 1 -- 172.21.15.31:0/221125100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9013b0a0 con 0x7f2c9006db50 2023-10-06T05:25:59.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.788+0000 7f2c987ff700 1 -- 172.21.15.31:0/221125100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9013b660 con 0x7f2c9006db50 2023-10-06T05:25:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.788+0000 7f2c987ff700 1 -- 172.21.15.31:0/221125100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9004c380 con 0x7f2c9006db50 2023-10-06T05:25:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.789+0000 7f2c867fc700 1 -- 172.21.15.31:0/221125100 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f2c8c01b830 con 0x7f2c9006db50 2023-10-06T05:25:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.790+0000 7f2c867fc700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c74067830 0x7f2c74069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:25:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.791+0000 7f2c867fc700 1 -- 172.21.15.31:0/221125100 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f2c8c085010 con 0x7f2c9006db50 2023-10-06T05:25:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.791+0000 7f2c95d9a700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c74067830 0x7f2c74069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:25:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.792+0000 7f2c95d9a700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c74067830 0x7f2c74069ce0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2c900aec10 tx=0x7f2c80007f30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:25:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:25:59.795+0000 7f2c867fc700 1 -- 172.21.15.31:0/221125100 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c8c053c50 con 0x7f2c9006db50 2023-10-06T05:26:00.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.000+0000 7f2c987ff700 1 -- 172.21.15.31:0/221125100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2c901577c0 con 0x7f2c9006db50 2023-10-06T05:26:00.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.001+0000 7f2c867fc700 1 -- 172.21.15.31:0/221125100 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v159) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2c8c057270 con 0x7f2c9006db50 2023-10-06T05:26:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:00.003 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:26:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:26:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-10-06T05:26:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "rbd" 2023-10-06T05:26:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-10-06T05:26:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:26:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 2, 2023-10-06T05:26:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-10-06T05:26:00.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:26:00.004 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:26:00.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 -- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c74067830 msgr2=0x7f2c74069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c74067830 0x7f2c74069ce0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2c900aec10 tx=0x7f2c80007f30 comp rx=0 tx=0).stop 2023-10-06T05:26:00.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 -- 172.21.15.31:0/221125100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9006db50 msgr2=0x7f2c9013a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9006db50 0x7f2c9013a9c0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009390 tx=0x7f2c8c016000 comp rx=0 tx=0).stop 2023-10-06T05:26:00.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 -- 172.21.15.31:0/221125100 shutdown_connections 2023-10-06T05:26:00.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c74067830 0x7f2c74069ce0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9013a5b0 0x7f2c9013d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9006db50 0x7f2c9013a9c0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 --2- 172.21.15.31:0/221125100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9013a1e0 0x7f2c9016af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 -- 172.21.15.31:0/221125100 >> 172.21.15.31:0/221125100 conn(0x7f2c900cc0b0 msgr2=0x7f2c900c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:00.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 -- 172.21.15.31:0/221125100 shutdown_connections 2023-10-06T05:26:00.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.003+0000 7f2c987ff700 1 -- 172.21.15.31:0/221125100 wait complete. 2023-10-06T05:26:00.023 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:26:00.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:26:00.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.220+0000 7faee7bff700 1 Processor -- start 2023-10-06T05:26:00.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.220+0000 7faee7bff700 1 -- start start 2023-10-06T05:26:00.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.222+0000 7faee7bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faee0159750 0x7faee00b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.222+0000 7faee7bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee015deb0 con 0x7faee0067c70 2023-10-06T05:26:00.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.222+0000 7faee7bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee015d2a0 con 0x7faee0159380 2023-10-06T05:26:00.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.222+0000 7faee7bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee015d580 con 0x7faee0159750 2023-10-06T05:26:00.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.222+0000 7faee619c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faee0159750 0x7faee00b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.222+0000 7faee619c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faee0159750 0x7faee00b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51752/0 (socket says 172.21.15.31:51752) 2023-10-06T05:26:00.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.222+0000 7faee619c700 1 -- 172.21.15.31:0/400004907 learned_addr learned my addr 172.21.15.31:0/400004907 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:00.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.222+0000 7faee619c700 1 -- 172.21.15.31:0/400004907 >> v1:172.21.15.62:6789/0 conn(0x7faee0159380 legacy=0x7faee0067560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.223+0000 7faee619c700 1 -- 172.21.15.31:0/400004907 >> v1:172.21.15.31:6789/0 conn(0x7faee0067c70 legacy=0x7faee00b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.223+0000 7faee619c700 1 -- 172.21.15.31:0/400004907 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee015d860 con 0x7faee0159750 2023-10-06T05:26:00.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.223+0000 7faee4999700 1 -- 172.21.15.31:0/400004907 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944126649 0 0) 0x7faee015d2a0 con 0x7faee0159380 2023-10-06T05:26:00.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.223+0000 7faee619c700 1 --2- 172.21.15.31:0/400004907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 0x7faee00b2d20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7faed8009f20 tx=0x7faed8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7054c853dee01f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:00.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.223+0000 7faee4999700 1 -- 172.21.15.31:0/400004907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed800e070 con 0x7faee0159750 2023-10-06T05:26:00.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.223+0000 7faee4999700 1 -- 172.21.15.31:0/400004907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed800b980 con 0x7faee0159750 2023-10-06T05:26:00.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.223+0000 7faee4999700 1 -- 172.21.15.31:0/400004907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed80092e0 con 0x7faee0159750 2023-10-06T05:26:00.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.224+0000 7faee7bff700 1 -- 172.21.15.31:0/400004907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 msgr2=0x7faee00b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.224+0000 7faee7bff700 1 --2- 172.21.15.31:0/400004907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 0x7faee00b2d20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7faed8009f20 tx=0x7faed8009840 comp rx=0 tx=0).stop 2023-10-06T05:26:00.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.224+0000 7faee7bff700 1 -- 172.21.15.31:0/400004907 shutdown_connections 2023-10-06T05:26:00.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.224+0000 7faee7bff700 1 --2- 172.21.15.31:0/400004907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 0x7faee00b2d20 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.224+0000 7faee7bff700 1 -- 172.21.15.31:0/400004907 >> 172.21.15.31:0/400004907 conn(0x7faee0069140 msgr2=0x7faee00aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:00.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.224+0000 7faee7bff700 1 -- 172.21.15.31:0/400004907 shutdown_connections 2023-10-06T05:26:00.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.224+0000 7faee7bff700 1 -- 172.21.15.31:0/400004907 wait complete. 2023-10-06T05:26:00.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.225+0000 7faee7bff700 1 Processor -- start 2023-10-06T05:26:00.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.225+0000 7faee7bff700 1 -- start start 2023-10-06T05:26:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.226+0000 7faee7bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faee0067c70 0x7faee0148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.226+0000 7faee7bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee0159380 0x7faee00ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.226+0000 7faee7bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 0x7faee00cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.226+0000 7faee7bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee00acb90 con 0x7faee0159750 2023-10-06T05:26:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.226+0000 7faee7bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee00ad9f0 con 0x7faee0159380 2023-10-06T05:26:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.226+0000 7faee7bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee015c720 con 0x7faee0067c70 2023-10-06T05:26:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee619c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 0x7faee00cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee619c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 0x7faee00cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51766/0 (socket says 172.21.15.31:51766) 2023-10-06T05:26:00.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee619c700 1 -- 172.21.15.31:0/214098470 learned_addr learned my addr 172.21.15.31:0/214098470 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:00.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee519a700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee0159380 0x7faee00ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee599b700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faee0067c70 0x7faee0148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee619c700 1 -- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faee0067c70 msgr2=0x7faee0148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee619c700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faee0067c70 0x7faee0148b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee619c700 1 -- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee0159380 msgr2=0x7faee00ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee619c700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee0159380 0x7faee00ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee619c700 1 -- 172.21.15.31:0/214098470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8009ae0 con 0x7faee0159750 2023-10-06T05:26:00.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.227+0000 7faee619c700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 0x7faee00cd080 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7faed8009ab0 tx=0x7faed8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:00.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.228+0000 7faed5ffb700 1 -- 172.21.15.31:0/214098470 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8012b30 con 0x7faee0159750 2023-10-06T05:26:00.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.228+0000 7faed5ffb700 1 -- 172.21.15.31:0/214098470 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed8012c90 con 0x7faee0159750 2023-10-06T05:26:00.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.228+0000 7faee7bff700 1 -- 172.21.15.31:0/214098470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee00acde0 con 0x7faee0159750 2023-10-06T05:26:00.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.228+0000 7faed5ffb700 1 -- 172.21.15.31:0/214098470 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed801e680 con 0x7faee0159750 2023-10-06T05:26:00.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.228+0000 7faee7bff700 1 -- 172.21.15.31:0/214098470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee00ad3a0 con 0x7faee0159750 2023-10-06T05:26:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.229+0000 7faee7bff700 1 -- 172.21.15.31:0/214098470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee004c380 con 0x7faee0159750 2023-10-06T05:26:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.230+0000 7faed5ffb700 1 -- 172.21.15.31:0/214098470 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7faed801eab0 con 0x7faee0159750 2023-10-06T05:26:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.231+0000 7faed5ffb700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faec4067830 0x7faec4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.231+0000 7faed5ffb700 1 -- 172.21.15.31:0/214098470 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5234+0+0 (secure 0 0 0) 0x7faed8084a00 con 0x7faee0159750 2023-10-06T05:26:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.231+0000 7faee599b700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faec4067830 0x7faec4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.232+0000 7faee599b700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faec4067830 0x7faec4069ce0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faedc0011e0 tx=0x7faedc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.235+0000 7faed5ffb700 1 -- 172.21.15.31:0/214098470 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faed8053640 con 0x7faee0159750 2023-10-06T05:26:00.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.440+0000 7faee7bff700 1 -- 172.21.15.31:0/214098470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faee00af6b0 con 0x7faee0159750 2023-10-06T05:26:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.748+0000 7faed5ffb700 1 -- 172.21.15.31:0/214098470 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v160) v1 ==== 138+0+0 (secure 0 0 0) 0x7faed8056c60 con 0x7faee0159750 2023-10-06T05:26:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.751+0000 7faee7bff700 1 -- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faec4067830 msgr2=0x7faec4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.751+0000 7faee7bff700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faec4067830 0x7faec4069ce0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faedc0011e0 tx=0x7faedc009040 comp rx=0 tx=0).stop 2023-10-06T05:26:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.751+0000 7faee7bff700 1 -- 172.21.15.31:0/214098470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 msgr2=0x7faee00cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.751+0000 7faee7bff700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 0x7faee00cd080 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7faed8009ab0 tx=0x7faed8017000 comp rx=0 tx=0).stop 2023-10-06T05:26:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.753+0000 7faee7bff700 1 -- 172.21.15.31:0/214098470 shutdown_connections 2023-10-06T05:26:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.753+0000 7faee7bff700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faec4067830 0x7faec4069ce0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.753+0000 7faee7bff700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee0159750 0x7faee00cd080 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.753+0000 7faee7bff700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faee0067c70 0x7faee0148b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.753+0000 7faee7bff700 1 --2- 172.21.15.31:0/214098470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee0159380 0x7faee00ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.753+0000 7faee7bff700 1 -- 172.21.15.31:0/214098470 >> 172.21.15.31:0/214098470 conn(0x7faee0069140 msgr2=0x7faee0149570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.754+0000 7faee7bff700 1 -- 172.21.15.31:0/214098470 shutdown_connections 2023-10-06T05:26:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.754+0000 7faee7bff700 1 -- 172.21.15.31:0/214098470 wait complete. 2023-10-06T05:26:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-10-06T05:26:00.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:26:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.976+0000 7f5af3fff700 1 Processor -- start 2023-10-06T05:26:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.976+0000 7f5af3fff700 1 -- start start 2023-10-06T05:26:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.978+0000 7f5af3fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5aec153d40 0x7f5aec0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.978+0000 7f5af3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5aec0af890 0x7f5aec0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.978+0000 7f5af3fff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec0b0e40 con 0x7f5aec0b09c0 2023-10-06T05:26:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.978+0000 7f5af3fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec0afc90 con 0x7f5aec0af890 2023-10-06T05:26:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.978+0000 7f5af3fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec0ad690 con 0x7f5aec153d40 2023-10-06T05:26:00.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.978+0000 7f5af259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5aec0af890 0x7f5aec0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.978+0000 7f5af259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5aec0af890 0x7f5aec0adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51770/0 (socket says 172.21.15.31:51770) 2023-10-06T05:26:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.978+0000 7f5af259c700 1 -- 172.21.15.31:0/169665544 learned_addr learned my addr 172.21.15.31:0/169665544 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.978+0000 7f5af1d9b700 1 --2- 172.21.15.31:0/169665544 >> v2:172.21.15.62:3300/0 conn(0x7f5aec153d40 0x7f5aec0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.979+0000 7f5af259c700 1 -- 172.21.15.31:0/169665544 >> v2:172.21.15.62:3300/0 conn(0x7f5aec153d40 msgr2=0x7f5aec0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.979+0000 7f5af259c700 1 --2- 172.21.15.31:0/169665544 >> v2:172.21.15.62:3300/0 conn(0x7f5aec153d40 0x7f5aec0b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.979+0000 7f5af259c700 1 -- 172.21.15.31:0/169665544 >> v1:172.21.15.31:6789/0 conn(0x7f5aec0b09c0 legacy=0x7f5aec0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.979+0000 7f5af259c700 1 -- 172.21.15.31:0/169665544 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0acd60 con 0x7f5aec0af890 2023-10-06T05:26:00.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.979+0000 7f5af259c700 1 --2- 172.21.15.31:0/169665544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 0x7f5aec0adb20 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f5aec06d1d0 tx=0x7f5ae4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49ece81799804b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:00.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.979+0000 7f5af0d99700 1 -- 172.21.15.31:0/169665544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400e070 con 0x7f5aec0af890 2023-10-06T05:26:00.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.979+0000 7f5af0d99700 1 -- 172.21.15.31:0/169665544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae400ba50 con 0x7f5aec0af890 2023-10-06T05:26:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.979+0000 7f5af0d99700 1 -- 172.21.15.31:0/169665544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40092e0 con 0x7f5aec0af890 2023-10-06T05:26:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.980+0000 7f5af3fff700 1 -- 172.21.15.31:0/169665544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 msgr2=0x7f5aec0adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.980+0000 7f5af3fff700 1 --2- 172.21.15.31:0/169665544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 0x7f5aec0adb20 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f5aec06d1d0 tx=0x7f5ae4009840 comp rx=0 tx=0).stop 2023-10-06T05:26:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.980+0000 7f5af3fff700 1 -- 172.21.15.31:0/169665544 shutdown_connections 2023-10-06T05:26:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.980+0000 7f5af3fff700 1 --2- 172.21.15.31:0/169665544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 0x7f5aec0adb20 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.980+0000 7f5af3fff700 1 --2- 172.21.15.31:0/169665544 >> v2:172.21.15.62:3300/0 conn(0x7f5aec153d40 0x7f5aec0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.980+0000 7f5af3fff700 1 -- 172.21.15.31:0/169665544 >> 172.21.15.31:0/169665544 conn(0x7f5aec067560 msgr2=0x7f5aec067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.980+0000 7f5af3fff700 1 -- 172.21.15.31:0/169665544 shutdown_connections 2023-10-06T05:26:00.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.980+0000 7f5af3fff700 1 -- 172.21.15.31:0/169665544 wait complete. 2023-10-06T05:26:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.981+0000 7f5af3fff700 1 Processor -- start 2023-10-06T05:26:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.981+0000 7f5af3fff700 1 -- start start 2023-10-06T05:26:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.982+0000 7f5af3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 0x7f5aec15b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.982+0000 7f5af3fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5aec0b09c0 0x7f5aec169760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.982+0000 7f5af3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aec153d40 0x7f5aec16a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.982+0000 7f5af3fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec140600 con 0x7f5aec0af890 2023-10-06T05:26:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.982+0000 7f5af3fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec16a700 con 0x7f5aec153d40 2023-10-06T05:26:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.982+0000 7f5af3fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec15b7e0 con 0x7f5aec0b09c0 2023-10-06T05:26:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 0x7f5aec15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 0x7f5aec15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51778/0 (socket says 172.21.15.31:51778) 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af1d9b700 1 -- 172.21.15.31:0/3282009886 learned_addr learned my addr 172.21.15.31:0/3282009886 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af259c700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aec153d40 0x7f5aec16a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af159a700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5aec0b09c0 0x7f5aec169760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af1d9b700 1 -- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5aec0b09c0 msgr2=0x7f5aec169760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af1d9b700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5aec0b09c0 0x7f5aec169760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af1d9b700 1 -- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aec153d40 msgr2=0x7f5aec16a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af1d9b700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aec153d40 0x7f5aec16a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af1d9b700 1 -- 172.21.15.31:0/3282009886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4009ae0 con 0x7f5aec0af890 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af159a700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5aec0b09c0 0x7f5aec169760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.983+0000 7f5af1d9b700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 0x7f5aec15b200 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800d860 tx=0x7f5ae8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:00.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.984+0000 7f5ae1ffb700 1 -- 172.21.15.31:0/3282009886 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8009610 con 0x7f5aec0af890 2023-10-06T05:26:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.984+0000 7f5ae1ffb700 1 -- 172.21.15.31:0/3282009886 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae8009770 con 0x7f5aec0af890 2023-10-06T05:26:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.984+0000 7f5ae1ffb700 1 -- 172.21.15.31:0/3282009886 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8009b30 con 0x7f5aec0af890 2023-10-06T05:26:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.984+0000 7f5af3fff700 1 -- 172.21.15.31:0/3282009886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec15bac0 con 0x7f5aec0af890 2023-10-06T05:26:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.984+0000 7f5af3fff700 1 -- 172.21.15.31:0/3282009886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec15c080 con 0x7f5aec0af890 2023-10-06T05:26:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.985+0000 7f5af3fff700 1 -- 172.21.15.31:0/3282009886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec15c6d0 con 0x7f5aec0af890 2023-10-06T05:26:00.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.986+0000 7f5ae1ffb700 1 -- 172.21.15.31:0/3282009886 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f5ae8028430 con 0x7f5aec0af890 2023-10-06T05:26:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.987+0000 7f5ae1ffb700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5ad0067830 0x7f5ad0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.987+0000 7f5ae1ffb700 1 -- 172.21.15.31:0/3282009886 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5604+0+0 (secure 0 0 0) 0x7f5ae8084090 con 0x7f5aec0af890 2023-10-06T05:26:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.987+0000 7f5af159a700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5ad0067830 0x7f5ad0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:00.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.988+0000 7f5af159a700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5ad0067830 0x7f5ad0069ce0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5adc001840 tx=0x7f5adc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:00.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:00.991+0000 7f5ae1ffb700 1 -- 172.21.15.31:0/3282009886 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ae8052b60 con 0x7f5aec0af890 2023-10-06T05:26:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.198+0000 7f5af3fff700 1 -- 172.21.15.31:0/3282009886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5aec0ad0d0 con 0x7f5aec0af890 2023-10-06T05:26:01.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.757+0000 7f5ae1ffb700 1 -- 172.21.15.31:0/3282009886 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v161) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5ae8056180 con 0x7f5aec0af890 2023-10-06T05:26:01.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.770+0000 7f5af3fff700 1 -- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5ad0067830 msgr2=0x7f5ad0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.770+0000 7f5af3fff700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5ad0067830 0x7f5ad0069ce0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5adc001840 tx=0x7f5adc009040 comp rx=0 tx=0).stop 2023-10-06T05:26:01.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.770+0000 7f5af3fff700 1 -- 172.21.15.31:0/3282009886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 msgr2=0x7f5aec15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.770+0000 7f5af3fff700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 0x7f5aec15b200 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800d860 tx=0x7f5ae8008040 comp rx=0 tx=0).stop 2023-10-06T05:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.770+0000 7f5af3fff700 1 -- 172.21.15.31:0/3282009886 shutdown_connections 2023-10-06T05:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.770+0000 7f5af3fff700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5ad0067830 0x7f5ad0069ce0 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5adc001840 tx=0x7f5adc009040 comp rx=0 tx=0).stop 2023-10-06T05:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.770+0000 7f5af3fff700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aec153d40 0x7f5aec16a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.770+0000 7f5af3fff700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aec0af890 0x7f5aec15b200 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.770+0000 7f5af3fff700 1 --2- 172.21.15.31:0/3282009886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5aec0b09c0 0x7f5aec169760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.770+0000 7f5af3fff700 1 -- 172.21.15.31:0/3282009886 >> 172.21.15.31:0/3282009886 conn(0x7f5aec067560 msgr2=0x7f5aec152730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.773+0000 7f5af3fff700 1 -- 172.21.15.31:0/3282009886 shutdown_connections 2023-10-06T05:26:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.773+0000 7f5af3fff700 1 -- 172.21.15.31:0/3282009886 wait complete. 2023-10-06T05:26:01.778 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-10-06T05:26:01.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:26:01.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.996+0000 7faafffff700 1 Processor -- start 2023-10-06T05:26:01.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.996+0000 7faafffff700 1 -- start start 2023-10-06T05:26:02.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.998+0000 7faafffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faaf8159890 0x7faaf80c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.998+0000 7faafffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faaf8155430 0x7faaf81558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.998+0000 7faafffff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf8158190 con 0x7faaf8159c60 2023-10-06T05:26:02.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.998+0000 7faafffff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf8157580 con 0x7faaf8155430 2023-10-06T05:26:02.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.998+0000 7faafffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf8157860 con 0x7faaf8159890 2023-10-06T05:26:02.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.998+0000 7faafd59a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faaf8159c60 0x7faaf8156a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38478/0 (socket says 172.21.15.31:38478) 2023-10-06T05:26:02.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.998+0000 7faafd59a700 1 -- 172.21.15.31:0/2839615064 learned_addr learned my addr 172.21.15.31:0/2839615064 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.999+0000 7faafe59c700 1 --2- 172.21.15.31:0/2839615064 >> v2:172.21.15.31:3300/0 conn(0x7faaf8155430 0x7faaf81558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.999+0000 7faafdd9b700 1 --2- 172.21.15.31:0/2839615064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faaf8159890 0x7faaf80c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.999+0000 7faafcd99700 1 -- 172.21.15.31:0/2839615064 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 860101403 0 0) 0x7faaf8158190 con 0x7faaf8159c60 2023-10-06T05:26:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.999+0000 7faafcd99700 1 -- 172.21.15.31:0/2839615064 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faae4003570 con 0x7faaf8159c60 2023-10-06T05:26:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.999+0000 7faafe59c700 1 -- 172.21.15.31:0/2839615064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faaf8159890 msgr2=0x7faaf80c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.999+0000 7faafe59c700 1 --2- 172.21.15.31:0/2839615064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faaf8159890 0x7faaf80c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.999+0000 7faafe59c700 1 -- 172.21.15.31:0/2839615064 >> v1:172.21.15.31:6789/0 conn(0x7faaf8159c60 legacy=0x7faaf8156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.999+0000 7faafe59c700 1 -- 172.21.15.31:0/2839615064 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8157b40 con 0x7faaf8155430 2023-10-06T05:26:02.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:01.999+0000 7faafe59c700 1 --2- 172.21.15.31:0/2839615064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8155430 0x7faaf81558b0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7faaf000afa0 tx=0x7faaf000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58444d0177276812 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:02.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafcd99700 1 -- 172.21.15.31:0/2839615064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf000f070 con 0x7faaf8155430 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafcd99700 1 -- 172.21.15.31:0/2839615064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf000a2e0 con 0x7faaf8155430 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafcd99700 1 -- 172.21.15.31:0/2839615064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf000cbb0 con 0x7faaf8155430 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafffff700 1 -- 172.21.15.31:0/2839615064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8155430 msgr2=0x7faaf81558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafffff700 1 --2- 172.21.15.31:0/2839615064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8155430 0x7faaf81558b0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7faaf000afa0 tx=0x7faaf000a890 comp rx=0 tx=0).stop 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafffff700 1 -- 172.21.15.31:0/2839615064 shutdown_connections 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafffff700 1 --2- 172.21.15.31:0/2839615064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8155430 0x7faaf81558b0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafffff700 1 --2- 172.21.15.31:0/2839615064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faaf8159890 0x7faaf80c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafffff700 1 -- 172.21.15.31:0/2839615064 >> 172.21.15.31:0/2839615064 conn(0x7faaf8067560 msgr2=0x7faaf8067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafffff700 1 -- 172.21.15.31:0/2839615064 shutdown_connections 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.000+0000 7faafffff700 1 -- 172.21.15.31:0/2839615064 wait complete. 2023-10-06T05:26:02.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.002+0000 7faafffff700 1 Processor -- start 2023-10-06T05:26:02.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.002+0000 7faafffff700 1 -- start start 2023-10-06T05:26:02.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.003+0000 7faafffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faaf8155430 0x7faaf80ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.003+0000 7faafffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf8159890 0x7faaf80c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.003+0000 7faafffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8159c60 0x7faaf80c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.003+0000 7faafffff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80cc270 con 0x7faaf8159c60 2023-10-06T05:26:02.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.003+0000 7faafffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80c9b20 con 0x7faaf8159890 2023-10-06T05:26:02.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.003+0000 7faafffff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80c9fc0 con 0x7faaf8155430 2023-10-06T05:26:02.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.003+0000 7faafe59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8159c60 0x7faaf80c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.003+0000 7faafe59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8159c60 0x7faaf80c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51798/0 (socket says 172.21.15.31:51798) 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.003+0000 7faafe59c700 1 -- 172.21.15.31:0/3935004493 learned_addr learned my addr 172.21.15.31:0/3935004493 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faafdd9b700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faaf8155430 0x7faaf80ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faafd59a700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf8159890 0x7faaf80c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faafe59c700 1 -- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faaf8155430 msgr2=0x7faaf80ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faafe59c700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faaf8155430 0x7faaf80ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faafe59c700 1 -- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf8159890 msgr2=0x7faaf80c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faafe59c700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf8159890 0x7faaf80c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faafe59c700 1 -- 172.21.15.31:0/3935004493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf000ab30 con 0x7faaf8159c60 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faafd59a700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf8159890 0x7faaf80c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:02.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faafe59c700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8159c60 0x7faaf80c9480 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7faaf000afd0 tx=0x7faaf0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faaedffb700 1 -- 172.21.15.31:0/3935004493 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0009c80 con 0x7faaf8159c60 2023-10-06T05:26:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faaedffb700 1 -- 172.21.15.31:0/3935004493 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf0009e00 con 0x7faaf8159c60 2023-10-06T05:26:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.004+0000 7faaedffb700 1 -- 172.21.15.31:0/3935004493 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0027660 con 0x7faaf8159c60 2023-10-06T05:26:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.005+0000 7faafffff700 1 -- 172.21.15.31:0/3935004493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf80ca2a0 con 0x7faaf8159c60 2023-10-06T05:26:02.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.005+0000 7faafffff700 1 -- 172.21.15.31:0/3935004493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf80ca860 con 0x7faaf8159c60 2023-10-06T05:26:02.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.006+0000 7faafffff700 1 -- 172.21.15.31:0/3935004493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf804c380 con 0x7faaf8159c60 2023-10-06T05:26:02.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.007+0000 7faaedffb700 1 -- 172.21.15.31:0/3935004493 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7faaf001d410 con 0x7faaf8159c60 2023-10-06T05:26:02.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.008+0000 7faaedffb700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faadc067830 0x7faadc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:02.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.008+0000 7faaedffb700 1 -- 172.21.15.31:0/3935004493 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5986+0+0 (secure 0 0 0) 0x7faaf0084fa0 con 0x7faaf8159c60 2023-10-06T05:26:02.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.008+0000 7faafdd9b700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faadc067830 0x7faadc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:02.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.009+0000 7faafdd9b700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faadc067830 0x7faadc069ce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009d40 tx=0x7faaf4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:02.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.012+0000 7faaedffb700 1 -- 172.21.15.31:0/3935004493 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faaf0053c60 con 0x7faaf8159c60 2023-10-06T05:26:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:02.258+0000 7faafffff700 1 -- 172.21.15.31:0/3935004493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faaf8157e20 con 0x7faaf8159c60 2023-10-06T05:26:02.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:02.456 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:03.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.137+0000 7faaedffb700 1 -- 172.21.15.31:0/3935004493 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v108) v1 ==== 174+0+0 (secure 0 0 0) 0x7faaf000a470 con 0x7faaf8159c60 2023-10-06T05:26:03.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.141+0000 7faafffff700 1 -- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faadc067830 msgr2=0x7faadc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.141+0000 7faafffff700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faadc067830 0x7faadc069ce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009d40 tx=0x7faaf4008040 comp rx=0 tx=0).stop 2023-10-06T05:26:03.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.141+0000 7faafffff700 1 -- 172.21.15.31:0/3935004493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8159c60 msgr2=0x7faaf80c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.141+0000 7faafffff700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8159c60 0x7faaf80c9480 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7faaf000afd0 tx=0x7faaf0019000 comp rx=0 tx=0).stop 2023-10-06T05:26:03.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.142+0000 7faafffff700 1 -- 172.21.15.31:0/3935004493 shutdown_connections 2023-10-06T05:26:03.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.142+0000 7faafffff700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7faadc067830 0x7faadc069ce0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.142+0000 7faafffff700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaf8159c60 0x7faaf80c9480 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.142+0000 7faafffff700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faaf8155430 0x7faaf80ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.142+0000 7faafffff700 1 --2- 172.21.15.31:0/3935004493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf8159890 0x7faaf80c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.142+0000 7faafffff700 1 -- 172.21.15.31:0/3935004493 >> 172.21.15.31:0/3935004493 conn(0x7faaf8067560 msgr2=0x7faaf806e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:03.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.144+0000 7faafffff700 1 -- 172.21.15.31:0/3935004493 shutdown_connections 2023-10-06T05:26:03.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.144+0000 7faafffff700 1 -- 172.21.15.31:0/3935004493 wait complete. 2023-10-06T05:26:03.149 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 27 and data pool 28 2023-10-06T05:26:03.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:03.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.364+0000 7ff9b2dff700 1 Processor -- start 2023-10-06T05:26:03.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.364+0000 7ff9b2dff700 1 -- start start 2023-10-06T05:26:03.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.366+0000 7ff9b2dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ac13fb00 0x7ff9ac06c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.366+0000 7ff9b2dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9ac13fed0 0x7ff9ac06e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.366+0000 7ff9b2dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac06be70 con 0x7ff9ac06ea90 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.366+0000 7ff9b2dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac06ef30 con 0x7ff9ac13fed0 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.366+0000 7ff9b2dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac06f130 con 0x7ff9ac13fb00 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.366+0000 7ff9abfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9ac13fed0 0x7ff9ac06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.366+0000 7ff9abfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9ac13fed0 0x7ff9ac06e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51800/0 (socket says 172.21.15.31:51800) 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.366+0000 7ff9abfff700 1 -- 172.21.15.31:0/155462676 learned_addr learned my addr 172.21.15.31:0/155462676 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.366+0000 7ff9b0b9b700 1 --2- 172.21.15.31:0/155462676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ac13fb00 0x7ff9ac06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.366+0000 7ff9abfff700 1 -- 172.21.15.31:0/155462676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ac13fb00 msgr2=0x7ff9ac06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.367+0000 7ff9abfff700 1 --2- 172.21.15.31:0/155462676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ac13fb00 0x7ff9ac06c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.367+0000 7ff9abfff700 1 -- 172.21.15.31:0/155462676 >> v1:172.21.15.62:6789/0 conn(0x7ff9ac06ea90 legacy=0x7ff9ac0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.367+0000 7ff9abfff700 1 -- 172.21.15.31:0/155462676 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac06f410 con 0x7ff9ac13fed0 2023-10-06T05:26:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.367+0000 7ff9abfff700 1 --2- 172.21.15.31:0/155462676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fed0 0x7ff9ac06e550 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7ff9980099a0 tx=0x7ff998009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64849febec2d6bec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.367+0000 7ff9ab7fe700 1 -- 172.21.15.31:0/155462676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99800b860 con 0x7ff9ac13fed0 2023-10-06T05:26:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.367+0000 7ff9ab7fe700 1 -- 172.21.15.31:0/155462676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800b9c0 con 0x7ff9ac13fed0 2023-10-06T05:26:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.367+0000 7ff9ab7fe700 1 -- 172.21.15.31:0/155462676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99800bd80 con 0x7ff9ac13fed0 2023-10-06T05:26:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.368+0000 7ff9b2dff700 1 -- 172.21.15.31:0/155462676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fed0 msgr2=0x7ff9ac06e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.368+0000 7ff9b2dff700 1 --2- 172.21.15.31:0/155462676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fed0 0x7ff9ac06e550 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7ff9980099a0 tx=0x7ff998009250 comp rx=0 tx=0).stop 2023-10-06T05:26:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.368+0000 7ff9b2dff700 1 -- 172.21.15.31:0/155462676 shutdown_connections 2023-10-06T05:26:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.368+0000 7ff9b2dff700 1 --2- 172.21.15.31:0/155462676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ac13fb00 0x7ff9ac06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.368+0000 7ff9b2dff700 1 --2- 172.21.15.31:0/155462676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fed0 0x7ff9ac06e550 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.368+0000 7ff9b2dff700 1 -- 172.21.15.31:0/155462676 >> 172.21.15.31:0/155462676 conn(0x7ff9ac0cdcb0 msgr2=0x7ff9ac0c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.368+0000 7ff9b2dff700 1 -- 172.21.15.31:0/155462676 shutdown_connections 2023-10-06T05:26:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.368+0000 7ff9b2dff700 1 -- 172.21.15.31:0/155462676 wait complete. 2023-10-06T05:26:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.369+0000 7ff9b2dff700 1 Processor -- start 2023-10-06T05:26:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.369+0000 7ff9b2dff700 1 -- start start 2023-10-06T05:26:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.370+0000 7ff9b2dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ac06ea90 0x7ff9ac0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.370+0000 7ff9b2dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fb00 0x7ff9ac158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.370+0000 7ff9b2dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9ac13fed0 0x7ff9ac140cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.370+0000 7ff9b2dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac140b90 con 0x7ff9ac13fb00 2023-10-06T05:26:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.370+0000 7ff9b2dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0b1f10 con 0x7ff9ac13fed0 2023-10-06T05:26:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.370+0000 7ff9b2dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0b28e0 con 0x7ff9ac06ea90 2023-10-06T05:26:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.371+0000 7ff9abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fb00 0x7ff9ac158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.371+0000 7ff9abfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fb00 0x7ff9ac158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51806/0 (socket says 172.21.15.31:51806) 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.371+0000 7ff9abfff700 1 -- 172.21.15.31:0/3886532119 learned_addr learned my addr 172.21.15.31:0/3886532119 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.371+0000 7ff9b0b9b700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ac06ea90 0x7ff9ac0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.371+0000 7ff9b139c700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9ac13fed0 0x7ff9ac140cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.371+0000 7ff9abfff700 1 -- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ac06ea90 msgr2=0x7ff9ac0b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.371+0000 7ff9abfff700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ac06ea90 0x7ff9ac0b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.371+0000 7ff9abfff700 1 -- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9ac13fed0 msgr2=0x7ff9ac140cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.371+0000 7ff9abfff700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9ac13fed0 0x7ff9ac140cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.371+0000 7ff9abfff700 1 -- 172.21.15.31:0/3886532119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff998009490 con 0x7ff9ac13fb00 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.372+0000 7ff9abfff700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fb00 0x7ff9ac158fa0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac06da70 tx=0x7ff998017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.372+0000 7ff9a8ff9700 1 -- 172.21.15.31:0/3886532119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99801f070 con 0x7ff9ac13fb00 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.372+0000 7ff9a8ff9700 1 -- 172.21.15.31:0/3886532119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800bd40 con 0x7ff9ac13fb00 2023-10-06T05:26:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.372+0000 7ff9a8ff9700 1 -- 172.21.15.31:0/3886532119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99801a670 con 0x7ff9ac13fb00 2023-10-06T05:26:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.372+0000 7ff9b2dff700 1 -- 172.21.15.31:0/3886532119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac0b1300 con 0x7ff9ac13fb00 2023-10-06T05:26:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.372+0000 7ff9b2dff700 1 -- 172.21.15.31:0/3886532119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac0b18c0 con 0x7ff9ac13fb00 2023-10-06T05:26:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.373+0000 7ff9b2dff700 1 -- 172.21.15.31:0/3886532119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac04c380 con 0x7ff9ac13fb00 2023-10-06T05:26:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.374+0000 7ff9a8ff9700 1 -- 172.21.15.31:0/3886532119 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7ff99801aaa0 con 0x7ff9ac13fb00 2023-10-06T05:26:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.375+0000 7ff9a8ff9700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff990067830 0x7ff990069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.375+0000 7ff9a8ff9700 1 -- 172.21.15.31:0/3886532119 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff9980843a0 con 0x7ff9ac13fb00 2023-10-06T05:26:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.375+0000 7ff9b0b9b700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff990067830 0x7ff990069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.376+0000 7ff9b0b9b700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff990067830 0x7ff990069ce0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0006fd0 tx=0x7ff9a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.379+0000 7ff9a8ff9700 1 -- 172.21.15.31:0/3886532119 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff99808d080 con 0x7ff9ac13fb00 2023-10-06T05:26:03.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.583+0000 7ff9b2dff700 1 -- 172.21.15.31:0/3886532119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9ac15a080 con 0x7ff9ac13fb00 2023-10-06T05:26:03.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.584+0000 7ff9a8ff9700 1 -- 172.21.15.31:0/3886532119 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16985 (secure 0 0 0) 0x7ff998053060 con 0x7ff9ac13fb00 2023-10-06T05:26:03.586 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:03.587 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":162,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:26:02.260195+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:00.480986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:01.238722+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 -- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff990067830 msgr2=0x7ff990069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff990067830 0x7ff990069ce0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0006fd0 tx=0x7ff9a0008040 comp rx=0 tx=0).stop 2023-10-06T05:26:03.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 -- 172.21.15.31:0/3886532119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fb00 msgr2=0x7ff9ac158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fb00 0x7ff9ac158fa0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac06da70 tx=0x7ff998017000 comp rx=0 tx=0).stop 2023-10-06T05:26:03.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 -- 172.21.15.31:0/3886532119 shutdown_connections 2023-10-06T05:26:03.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff990067830 0x7ff990069ce0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9ac13fed0 0x7ff9ac140cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9ac06ea90 0x7ff9ac0b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 --2- 172.21.15.31:0/3886532119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac13fb00 0x7ff9ac158fa0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 -- 172.21.15.31:0/3886532119 >> 172.21.15.31:0/3886532119 conn(0x7ff9ac0cdcb0 msgr2=0x7ff9ac0c3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:03.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 -- 172.21.15.31:0/3886532119 shutdown_connections 2023-10-06T05:26:03.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.586+0000 7ff9b2dff700 1 -- 172.21.15.31:0/3886532119 wait complete. 2023-10-06T05:26:03.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:03.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.802+0000 7f94fabff700 1 Processor -- start 2023-10-06T05:26:03.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.803+0000 7f94fabff700 1 -- start start 2023-10-06T05:26:03.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.804+0000 7f94fabff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f94f40b08c0 0x7f94f40c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.804+0000 7f94fabff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f4069170 con 0x7f94f40cae30 2023-10-06T05:26:03.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.804+0000 7f94fabff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f40685f0 con 0x7f94f40cb200 2023-10-06T05:26:03.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.804+0000 7f94fabff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f4068840 con 0x7f94f40b08c0 2023-10-06T05:26:03.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.804+0000 7f94f899b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f94f40cae30 0x7f94f40ca090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38488/0 (socket says 172.21.15.31:38488) 2023-10-06T05:26:03.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.804+0000 7f94f899b700 1 -- 172.21.15.31:0/2893737207 learned_addr learned my addr 172.21.15.31:0/2893737207 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:03.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.804+0000 7f94f919c700 1 --2- 172.21.15.31:0/2893737207 >> v2:172.21.15.62:3300/0 conn(0x7f94f40b08c0 0x7f94f40c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.804+0000 7f94f37fe700 1 -- 172.21.15.31:0/2893737207 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099732521 0 0) 0x7f94f4069170 con 0x7f94f40cae30 2023-10-06T05:26:03.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.805+0000 7f94f37fe700 1 -- 172.21.15.31:0/2893737207 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e4003570 con 0x7f94f40cae30 2023-10-06T05:26:03.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.805+0000 7f94f919c700 1 -- 172.21.15.31:0/2893737207 >> v1:172.21.15.62:6789/0 conn(0x7f94f40cb200 legacy=0x7f94f40b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.805+0000 7f94f919c700 1 -- 172.21.15.31:0/2893737207 >> v1:172.21.15.31:6789/0 conn(0x7f94f40cae30 legacy=0x7f94f40ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.805+0000 7f94f919c700 1 -- 172.21.15.31:0/2893737207 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f4068b20 con 0x7f94f40b08c0 2023-10-06T05:26:03.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.805+0000 7f94f919c700 1 --2- 172.21.15.31:0/2893737207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94f40b08c0 0x7f94f40c90a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009e60 tx=0x7f94ec009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd7dc024bfaf9b52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.805+0000 7f94f37fe700 1 -- 172.21.15.31:0/2893737207 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec00e070 con 0x7f94f40b08c0 2023-10-06T05:26:03.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.805+0000 7f94f37fe700 1 -- 172.21.15.31:0/2893737207 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ec00bb60 con 0x7f94f40b08c0 2023-10-06T05:26:03.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.805+0000 7f94f37fe700 1 -- 172.21.15.31:0/2893737207 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec012430 con 0x7f94f40b08c0 2023-10-06T05:26:03.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.806+0000 7f94fabff700 1 -- 172.21.15.31:0/2893737207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94f40b08c0 msgr2=0x7f94f40c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.806+0000 7f94fabff700 1 --2- 172.21.15.31:0/2893737207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94f40b08c0 0x7f94f40c90a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009e60 tx=0x7f94ec009780 comp rx=0 tx=0).stop 2023-10-06T05:26:03.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.806+0000 7f94fabff700 1 -- 172.21.15.31:0/2893737207 shutdown_connections 2023-10-06T05:26:03.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.806+0000 7f94fabff700 1 --2- 172.21.15.31:0/2893737207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94f40b08c0 0x7f94f40c90a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.806+0000 7f94fabff700 1 -- 172.21.15.31:0/2893737207 >> 172.21.15.31:0/2893737207 conn(0x7f94f4087fc0 msgr2=0x7f94f40639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:03.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.806+0000 7f94fabff700 1 -- 172.21.15.31:0/2893737207 shutdown_connections 2023-10-06T05:26:03.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.806+0000 7f94fabff700 1 -- 172.21.15.31:0/2893737207 wait complete. 2023-10-06T05:26:03.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.808+0000 7f94fabff700 1 Processor -- start 2023-10-06T05:26:03.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.808+0000 7f94fabff700 1 -- start start 2023-10-06T05:26:03.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.809+0000 7f94fabff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f40b08c0 0x7f94f41556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94fabff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94f40cae30 0x7f94f40b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94fabff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94f40cb200 0x7f94f4158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94fabff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40b1a60 con 0x7f94f40b08c0 2023-10-06T05:26:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94fabff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40b1d40 con 0x7f94f40cb200 2023-10-06T05:26:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94fabff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40b2020 con 0x7f94f40cae30 2023-10-06T05:26:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94f899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f40b08c0 0x7f94f41556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94f899b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f40b08c0 0x7f94f41556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51812/0 (socket says 172.21.15.31:51812) 2023-10-06T05:26:03.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94f899b700 1 -- 172.21.15.31:0/282995220 learned_addr learned my addr 172.21.15.31:0/282995220 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94f919c700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94f40cb200 0x7f94f4158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94f3fff700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94f40cae30 0x7f94f40b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94f899b700 1 -- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94f40cae30 msgr2=0x7f94f40b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94f899b700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94f40cae30 0x7f94f40b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94f899b700 1 -- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94f40cb200 msgr2=0x7f94f4158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94f899b700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94f40cb200 0x7f94f4158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.810+0000 7f94f899b700 1 -- 172.21.15.31:0/282995220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec009a20 con 0x7f94f40b08c0 2023-10-06T05:26:03.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.811+0000 7f94f899b700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f40b08c0 0x7f94f41556b0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f94e800c790 tx=0x7f94e8007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.811+0000 7f94f0ff9700 1 -- 172.21.15.31:0/282995220 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e80072e0 con 0x7f94f40b08c0 2023-10-06T05:26:03.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.811+0000 7f94f0ff9700 1 -- 172.21.15.31:0/282995220 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8007440 con 0x7f94f40b08c0 2023-10-06T05:26:03.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.811+0000 7f94fabff700 1 -- 172.21.15.31:0/282995220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f40b2300 con 0x7f94f40b08c0 2023-10-06T05:26:03.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.811+0000 7f94f0ff9700 1 -- 172.21.15.31:0/282995220 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e80057f0 con 0x7f94f40b08c0 2023-10-06T05:26:03.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.811+0000 7f94fabff700 1 -- 172.21.15.31:0/282995220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f415b060 con 0x7f94f40b08c0 2023-10-06T05:26:03.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.812+0000 7f94fabff700 1 -- 172.21.15.31:0/282995220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f40b2820 con 0x7f94f40b08c0 2023-10-06T05:26:03.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.813+0000 7f94f0ff9700 1 -- 172.21.15.31:0/282995220 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f94e8005be0 con 0x7f94f40b08c0 2023-10-06T05:26:03.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.814+0000 7f94f0ff9700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94d8067830 0x7f94d8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:03.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.814+0000 7f94f0ff9700 1 -- 172.21.15.31:0/282995220 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f94e8082140 con 0x7f94f40b08c0 2023-10-06T05:26:03.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.814+0000 7f94f3fff700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94d8067830 0x7f94d8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:03.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.815+0000 7f94f3fff700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94d8067830 0x7f94d8069ce0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f94f406cb60 tx=0x7f94e0007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:03.818+0000 7f94f0ff9700 1 -- 172.21.15.31:0/282995220 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94e804b5a0 con 0x7f94f40b08c0 2023-10-06T05:26:04.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.025+0000 7f94fabff700 1 -- 172.21.15.31:0/282995220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94f404c380 con 0x7f94f40b08c0 2023-10-06T05:26:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.026+0000 7f94f0ff9700 1 -- 172.21.15.31:0/282995220 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+16972 (secure 0 0 0) 0x7f94e8017020 con 0x7f94f40b08c0 2023-10-06T05:26:04.029 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:04.029 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":163,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:26:03.756620+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:00.480986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:01.238722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:04.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 -- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94d8067830 msgr2=0x7f94d8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:04.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94d8067830 0x7f94d8069ce0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f94f406cb60 tx=0x7f94e0007ec0 comp rx=0 tx=0).stop 2023-10-06T05:26:04.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 -- 172.21.15.31:0/282995220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f40b08c0 msgr2=0x7f94f41556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:04.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f40b08c0 0x7f94f41556b0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f94e800c790 tx=0x7f94e8007c80 comp rx=0 tx=0).stop 2023-10-06T05:26:04.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 -- 172.21.15.31:0/282995220 shutdown_connections 2023-10-06T05:26:04.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f94d8067830 0x7f94d8069ce0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:04.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94f40cb200 0x7f94f4158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:04.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94f40b08c0 0x7f94f41556b0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:04.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 --2- 172.21.15.31:0/282995220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f94f40cae30 0x7f94f40b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:04.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 -- 172.21.15.31:0/282995220 >> 172.21.15.31:0/282995220 conn(0x7f94f4087fc0 msgr2=0x7f94f406b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:04.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 -- 172.21.15.31:0/282995220 shutdown_connections 2023-10-06T05:26:04.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.029+0000 7f94fabff700 1 -- 172.21.15.31:0/282995220 wait complete. 2023-10-06T05:26:04.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:26:04.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.248+0000 7f75e4fff700 1 Processor -- start 2023-10-06T05:26:04.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.248+0000 7f75e4fff700 1 -- start start 2023-10-06T05:26:04.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.250+0000 7f75e4fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75e0159890 0x7f75e00c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:04.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.250+0000 7f75e4fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f75e0155430 0x7f75e0155890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:04.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.250+0000 7f75e4fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e0158190 con 0x7f75e0159c60 2023-10-06T05:26:04.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.250+0000 7f75e4fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e0157580 con 0x7f75e0159890 2023-10-06T05:26:04.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.250+0000 7f75e4fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e0157860 con 0x7f75e0155430 2023-10-06T05:26:04.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.250+0000 7f75de59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75e0159890 0x7f75e00c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:04.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.250+0000 7f75de59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75e0159890 0x7f75e00c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51818/0 (socket says 172.21.15.31:51818) 2023-10-06T05:26:04.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.250+0000 7f75de59c700 1 -- 172.21.15.31:0/3680759214 learned_addr learned my addr 172.21.15.31:0/3680759214 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:04.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.251+0000 7f75ded9d700 1 --2- 172.21.15.31:0/3680759214 >> v2:172.21.15.62:3300/0 conn(0x7f75e0155430 0x7f75e0155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:04.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.251+0000 7f75de59c700 1 -- 172.21.15.31:0/3680759214 >> v2:172.21.15.62:3300/0 conn(0x7f75e0155430 msgr2=0x7f75e0155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:04.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.251+0000 7f75de59c700 1 --2- 172.21.15.31:0/3680759214 >> v2:172.21.15.62:3300/0 conn(0x7f75e0155430 0x7f75e0155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:04.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.251+0000 7f75de59c700 1 -- 172.21.15.31:0/3680759214 >> v1:172.21.15.62:6789/0 conn(0x7f75e0159c60 legacy=0x7f75e0156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:04.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.251+0000 7f75de59c700 1 -- 172.21.15.31:0/3680759214 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0157b40 con 0x7f75e0159890 2023-10-06T05:26:04.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.251+0000 7f75ded9d700 1 --2- 172.21.15.31:0/3680759214 >> v2:172.21.15.62:3300/0 conn(0x7f75e0155430 0x7f75e0155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:04.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.251+0000 7f75de59c700 1 --2- 172.21.15.31:0/3680759214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 0x7f75e00c89e0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f75e00cd190 tx=0x7f75d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbc069545bc3d18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:04.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.251+0000 7f75dd59a700 1 -- 172.21.15.31:0/3680759214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000b820 con 0x7f75e0159890 2023-10-06T05:26:04.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.251+0000 7f75dd59a700 1 -- 172.21.15.31:0/3680759214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d000b980 con 0x7f75e0159890 2023-10-06T05:26:04.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.251+0000 7f75dd59a700 1 -- 172.21.15.31:0/3680759214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000bd20 con 0x7f75e0159890 2023-10-06T05:26:04.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.252+0000 7f75e4fff700 1 -- 172.21.15.31:0/3680759214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 msgr2=0x7f75e00c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:04.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.252+0000 7f75e4fff700 1 --2- 172.21.15.31:0/3680759214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 0x7f75e00c89e0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f75e00cd190 tx=0x7f75d0009250 comp rx=0 tx=0).stop 2023-10-06T05:26:04.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.252+0000 7f75e4fff700 1 -- 172.21.15.31:0/3680759214 shutdown_connections 2023-10-06T05:26:04.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.252+0000 7f75e4fff700 1 --2- 172.21.15.31:0/3680759214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 0x7f75e00c89e0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:04.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.252+0000 7f75e4fff700 1 --2- 172.21.15.31:0/3680759214 >> v2:172.21.15.62:3300/0 conn(0x7f75e0155430 0x7f75e0155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:04.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.252+0000 7f75e4fff700 1 -- 172.21.15.31:0/3680759214 >> 172.21.15.31:0/3680759214 conn(0x7f75e0067560 msgr2=0x7f75e0067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:04.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.252+0000 7f75e4fff700 1 -- 172.21.15.31:0/3680759214 shutdown_connections 2023-10-06T05:26:04.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.252+0000 7f75e4fff700 1 -- 172.21.15.31:0/3680759214 wait complete. 2023-10-06T05:26:04.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.254+0000 7f75e4fff700 1 Processor -- start 2023-10-06T05:26:04.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.254+0000 7f75e4fff700 1 -- start start 2023-10-06T05:26:04.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75e4fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e0155430 0x7f75e0152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:04.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75e4fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 0x7f75e014ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:04.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75e4fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e0159c60 0x7f75e014d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75e4fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0165ce0 con 0x7f75e0159890 2023-10-06T05:26:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75e4fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e014b290 con 0x7f75e0159c60 2023-10-06T05:26:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75e4fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0153590 con 0x7f75e0155430 2023-10-06T05:26:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75ddd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 0x7f75e014ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75de59c700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e0155430 0x7f75e0152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75ddd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 0x7f75e014ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51820/0 (socket says 172.21.15.31:51820) 2023-10-06T05:26:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75ddd9b700 1 -- 172.21.15.31:0/3232282248 learned_addr learned my addr 172.21.15.31:0/3232282248 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75ded9d700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e0159c60 0x7f75e014d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75ddd9b700 1 -- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e0155430 msgr2=0x7f75e0152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75ddd9b700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e0155430 0x7f75e0152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75ddd9b700 1 -- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e0159c60 msgr2=0x7f75e014d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75ddd9b700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e0159c60 0x7f75e014d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.256+0000 7f75ddd9b700 1 -- 172.21.15.31:0/3232282248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d00094f0 con 0x7f75e0159890 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.257+0000 7f75ddd9b700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 0x7f75e014ad50 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f75d400b5e0 tx=0x7f75d4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.257+0000 7f75ce7fc700 1 -- 172.21.15.31:0/3232282248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400f510 con 0x7f75e0159890 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.257+0000 7f75ce7fc700 1 -- 172.21.15.31:0/3232282248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d400f670 con 0x7f75e0159890 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.257+0000 7f75ce7fc700 1 -- 172.21.15.31:0/3232282248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400fa30 con 0x7f75e0159890 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.257+0000 7f75e4fff700 1 -- 172.21.15.31:0/3232282248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e0153870 con 0x7f75e0159890 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.257+0000 7f75e4fff700 1 -- 172.21.15.31:0/3232282248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e0153e30 con 0x7f75e0159890 2023-10-06T05:26:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.258+0000 7f75e4fff700 1 -- 172.21.15.31:0/3232282248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e014b5f0 con 0x7f75e0159890 2023-10-06T05:26:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.259+0000 7f75ce7fc700 1 -- 172.21.15.31:0/3232282248 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f75d401a400 con 0x7f75e0159890 2023-10-06T05:26:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.260+0000 7f75ce7fc700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75c0067830 0x7f75c0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.260+0000 7f75ce7fc700 1 -- 172.21.15.31:0/3232282248 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f75d4082880 con 0x7f75e0159890 2023-10-06T05:26:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.261+0000 7f75de59c700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75c0067830 0x7f75c0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.262+0000 7f75de59c700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75c0067830 0x7f75c0069ce0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f75e00cd190 tx=0x7f75d0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.264+0000 7f75ce7fc700 1 -- 172.21.15.31:0/3232282248 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75e014b5f0 con 0x7f75e0159890 2023-10-06T05:26:04.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:04.514+0000 7f75e4fff700 1 -- 172.21.15.31:0/3232282248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f75e014b5f0 con 0x7f75e0159890 2023-10-06T05:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.150+0000 7f75ce7fc700 1 -- 172.21.15.31:0/3232282248 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v111) v1 ==== 118+0+0 (secure 0 0 0) 0x7f75e014b5f0 con 0x7f75e0159890 2023-10-06T05:26:05.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 -- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75c0067830 msgr2=0x7f75c0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75c0067830 0x7f75c0069ce0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f75e00cd190 tx=0x7f75d0017000 comp rx=0 tx=0).stop 2023-10-06T05:26:05.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 -- 172.21.15.31:0/3232282248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 msgr2=0x7f75e014ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 0x7f75e014ad50 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f75d400b5e0 tx=0x7f75d4009120 comp rx=0 tx=0).stop 2023-10-06T05:26:05.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 -- 172.21.15.31:0/3232282248 shutdown_connections 2023-10-06T05:26:05.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f75c0067830 0x7f75c0069ce0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e0159c60 0x7f75e014d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e0155430 0x7f75e0152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 --2- 172.21.15.31:0/3232282248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e0159890 0x7f75e014ad50 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 -- 172.21.15.31:0/3232282248 >> 172.21.15.31:0/3232282248 conn(0x7f75e0067560 msgr2=0x7f75e0151610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 -- 172.21.15.31:0/3232282248 shutdown_connections 2023-10-06T05:26:05.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.153+0000 7f75e4fff700 1 -- 172.21.15.31:0/3232282248 wait complete. 2023-10-06T05:26:05.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:26:05.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.377+0000 7f37003ff700 1 Processor -- start 2023-10-06T05:26:05.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.377+0000 7f37003ff700 1 -- start start 2023-10-06T05:26:05.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.378+0000 7f37003ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f36f8159560 0x7f36f80cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.378+0000 7f37003ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f815d2d0 con 0x7f36f80b2b80 2023-10-06T05:26:05.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.378+0000 7f37003ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f815d5b0 con 0x7f36f8159930 2023-10-06T05:26:05.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.378+0000 7f37003ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f815d890 con 0x7f36f8159560 2023-10-06T05:26:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.379+0000 7f36fe99c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f36f80b2b80 0x7f36f80cb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38502/0 (socket says 172.21.15.31:38502) 2023-10-06T05:26:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.379+0000 7f36fe99c700 1 -- 172.21.15.31:0/64672333 learned_addr learned my addr 172.21.15.31:0/64672333 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.379+0000 7f36fe19b700 1 --2- 172.21.15.31:0/64672333 >> v2:172.21.15.62:3300/0 conn(0x7f36f8159560 0x7f36f80cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.379+0000 7f36fd199700 1 -- 172.21.15.31:0/64672333 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92777865 0 0) 0x7f36f815d2d0 con 0x7f36f80b2b80 2023-10-06T05:26:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.379+0000 7f36fd199700 1 -- 172.21.15.31:0/64672333 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e4003570 con 0x7f36f80b2b80 2023-10-06T05:26:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.379+0000 7f36fe19b700 1 -- 172.21.15.31:0/64672333 >> v1:172.21.15.62:6789/0 conn(0x7f36f8159930 legacy=0x7f36f80b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.379+0000 7f36fe19b700 1 -- 172.21.15.31:0/64672333 >> v1:172.21.15.31:6789/0 conn(0x7f36f80b2b80 legacy=0x7f36f80cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.379+0000 7f36fe19b700 1 -- 172.21.15.31:0/64672333 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f815db70 con 0x7f36f8159560 2023-10-06T05:26:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.380+0000 7f36fe19b700 1 --2- 172.21.15.31:0/64672333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36f8159560 0x7f36f80cce70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f36f40098e0 tx=0x7f36f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcad738a4c471adb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.380+0000 7f36fd199700 1 -- 172.21.15.31:0/64672333 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400b640 con 0x7f36f8159560 2023-10-06T05:26:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.380+0000 7f36fd199700 1 -- 172.21.15.31:0/64672333 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f400b7a0 con 0x7f36f8159560 2023-10-06T05:26:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.380+0000 7f36fd199700 1 -- 172.21.15.31:0/64672333 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400bb40 con 0x7f36f8159560 2023-10-06T05:26:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.381+0000 7f37003ff700 1 -- 172.21.15.31:0/64672333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36f8159560 msgr2=0x7f36f80cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.381+0000 7f37003ff700 1 --2- 172.21.15.31:0/64672333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36f8159560 0x7f36f80cce70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f36f40098e0 tx=0x7f36f40091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.381+0000 7f37003ff700 1 -- 172.21.15.31:0/64672333 shutdown_connections 2023-10-06T05:26:05.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.381+0000 7f37003ff700 1 --2- 172.21.15.31:0/64672333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36f8159560 0x7f36f80cce70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.381+0000 7f37003ff700 1 -- 172.21.15.31:0/64672333 >> 172.21.15.31:0/64672333 conn(0x7f36f806ec90 msgr2=0x7f36f806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:05.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.381+0000 7f37003ff700 1 -- 172.21.15.31:0/64672333 shutdown_connections 2023-10-06T05:26:05.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.381+0000 7f37003ff700 1 -- 172.21.15.31:0/64672333 wait complete. 2023-10-06T05:26:05.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.382+0000 7f37003ff700 1 Processor -- start 2023-10-06T05:26:05.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.382+0000 7f37003ff700 1 -- start start 2023-10-06T05:26:05.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.383+0000 7f37003ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36f80b2b80 0x7f36f80c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.383+0000 7f37003ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36f8159560 0x7f36f80d0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.383+0000 7f37003ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36f8159930 0x7f36f816c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.383+0000 7f37003ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f80c8700 con 0x7f36f80b2b80 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.383+0000 7f37003ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f80c7af0 con 0x7f36f8159560 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.383+0000 7f37003ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f80c7dd0 con 0x7f36f8159930 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.383+0000 7f36fe19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36f80b2b80 0x7f36f80c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.383+0000 7f36fe19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36f80b2b80 0x7f36f80c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51830/0 (socket says 172.21.15.31:51830) 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.383+0000 7f36fe19b700 1 -- 172.21.15.31:0/3998050835 learned_addr learned my addr 172.21.15.31:0/3998050835 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f36fe19b700 1 -- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36f8159930 msgr2=0x7f36f816c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f36fe19b700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36f8159930 0x7f36f816c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f36fe19b700 1 -- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36f8159560 msgr2=0x7f36f80d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f36fe19b700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36f8159560 0x7f36f80d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f36fe19b700 1 -- 172.21.15.31:0/3998050835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f4009420 con 0x7f36f80b2b80 2023-10-06T05:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f36fe19b700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36f80b2b80 0x7f36f80c74a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f36f80893d0 tx=0x7f36f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f36ee7fc700 1 -- 172.21.15.31:0/3998050835 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400bb20 con 0x7f36f80b2b80 2023-10-06T05:26:05.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f36ee7fc700 1 -- 172.21.15.31:0/3998050835 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f4011420 con 0x7f36f80b2b80 2023-10-06T05:26:05.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f36ee7fc700 1 -- 172.21.15.31:0/3998050835 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f40117c0 con 0x7f36f80b2b80 2023-10-06T05:26:05.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f37003ff700 1 -- 172.21.15.31:0/3998050835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f80c80b0 con 0x7f36f80b2b80 2023-10-06T05:26:05.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.384+0000 7f37003ff700 1 -- 172.21.15.31:0/3998050835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f80c9f40 con 0x7f36f80b2b80 2023-10-06T05:26:05.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.385+0000 7f37003ff700 1 -- 172.21.15.31:0/3998050835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f804c380 con 0x7f36f80b2b80 2023-10-06T05:26:05.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.386+0000 7f36ee7fc700 1 -- 172.21.15.31:0/3998050835 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f36f4011b70 con 0x7f36f80b2b80 2023-10-06T05:26:05.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.388+0000 7f36ee7fc700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f36dc067830 0x7f36dc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:05.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.388+0000 7f36ee7fc700 1 -- 172.21.15.31:0/3998050835 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f36f4084560 con 0x7f36f80b2b80 2023-10-06T05:26:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.388+0000 7f36fd99a700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f36dc067830 0x7f36dc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.389+0000 7f36fd99a700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f36dc067830 0x7f36dc069ce0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f36f80d0fc0 tx=0x7f36e8007b40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:05.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.392+0000 7f36ee7fc700 1 -- 172.21.15.31:0/3998050835 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36f40531a0 con 0x7f36f80b2b80 2023-10-06T05:26:05.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:05.644+0000 7f37003ff700 1 -- 172.21.15.31:0/3998050835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f36f815de50 con 0x7f36f80b2b80 2023-10-06T05:26:06.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.155+0000 7f36ee7fc700 1 -- 172.21.15.31:0/3998050835 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v112) v1 ==== 122+0+0 (secure 0 0 0) 0x7f36f40567c0 con 0x7f36f80b2b80 2023-10-06T05:26:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 -- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f36dc067830 msgr2=0x7f36dc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f36dc067830 0x7f36dc069ce0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f36f80d0fc0 tx=0x7f36e8007b40 comp rx=0 tx=0).stop 2023-10-06T05:26:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 -- 172.21.15.31:0/3998050835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36f80b2b80 msgr2=0x7f36f80c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36f80b2b80 0x7f36f80c74a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f36f80893d0 tx=0x7f36f4017000 comp rx=0 tx=0).stop 2023-10-06T05:26:06.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 -- 172.21.15.31:0/3998050835 shutdown_connections 2023-10-06T05:26:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f36dc067830 0x7f36dc069ce0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36f8159930 0x7f36f816c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36f80b2b80 0x7f36f80c74a0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 --2- 172.21.15.31:0/3998050835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36f8159560 0x7f36f80d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 -- 172.21.15.31:0/3998050835 >> 172.21.15.31:0/3998050835 conn(0x7f36f806ec90 msgr2=0x7f36f8157740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 -- 172.21.15.31:0/3998050835 shutdown_connections 2023-10-06T05:26:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.158+0000 7f37003ff700 1 -- 172.21.15.31:0/3998050835 wait complete. 2023-10-06T05:26:06.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:06.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.375+0000 7f19719ff700 1 Processor -- start 2023-10-06T05:26:06.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.375+0000 7f19719ff700 1 -- start start 2023-10-06T05:26:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.376+0000 7f19719ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c159580 0x7f196c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.376+0000 7f19719ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f196c0ccc80 0x7f196c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.376+0000 7f19719ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0cf280 con 0x7f196c0cb9f0 2023-10-06T05:26:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.376+0000 7f19719ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c0cfe20 con 0x7f196c0ccc80 2023-10-06T05:26:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.376+0000 7f19719ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0ce670 con 0x7f196c159580 2023-10-06T05:26:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.376+0000 7f1963fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f196c0cb9f0 0x7f196c0cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38514/0 (socket says 172.21.15.31:38514) 2023-10-06T05:26:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.376+0000 7f1963fff700 1 -- 172.21.15.31:0/848334672 learned_addr learned my addr 172.21.15.31:0/848334672 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196affd700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c159580 0x7f196c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196b7fe700 1 --2- 172.21.15.31:0/848334672 >> v2:172.21.15.62:3300/0 conn(0x7f196c0ccc80 0x7f196c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210962651 0 0) 0x7f196c0cf280 con 0x7f196c0cb9f0 2023-10-06T05:26:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1958003570 con 0x7f196c0cb9f0 2023-10-06T05:26:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 954181638 0 0) 0x7f1958003570 con 0x7f196c0cb9f0 2023-10-06T05:26:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c159580 msgr2=0x7f196c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c159580 0x7f196c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 >> v2:172.21.15.62:3300/0 conn(0x7f196c0ccc80 msgr2=0x7f196c0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 --2- 172.21.15.31:0/848334672 >> v2:172.21.15.62:3300/0 conn(0x7f196c0ccc80 0x7f196c0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0ce950 con 0x7f196c0cb9f0 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f1954001470 con 0x7f196c0cb9f0 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 >> v1:172.21.15.31:6789/0 conn(0x7f196c0cb9f0 legacy=0x7f196c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c159580 0x7f1958003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19580040b0 0x7f1958004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 0x7f19580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0cfe20 con 0x7f1958004d50 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0ce670 con 0x7f19580040b0 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.377+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195800bb60 con 0x7f196c159580 2023-10-06T05:26:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196b7fe700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 0x7f19580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196affd700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c159580 0x7f1958003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f1963fff700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19580040b0 0x7f1958004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196b7fe700 1 -- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c159580 msgr2=0x7f1958003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196b7fe700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c159580 0x7f1958003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196b7fe700 1 -- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19580040b0 msgr2=0x7f1958004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196b7fe700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19580040b0 0x7f1958004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196b7fe700 1 -- 172.21.15.31:0/848334672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1958003570 con 0x7f1958004d50 2023-10-06T05:26:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196b7fe700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 0x7f19580051c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f19640094d0 tx=0x7f196400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf4ba373a09b0e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400ea90 con 0x7f1958004d50 2023-10-06T05:26:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196400f040 con 0x7f1958004d50 2023-10-06T05:26:06.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.378+0000 7f196a7fc700 1 -- 172.21.15.31:0/848334672 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400ee20 con 0x7f1958004d50 2023-10-06T05:26:06.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.379+0000 7f19719ff700 1 -- 172.21.15.31:0/848334672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 msgr2=0x7f19580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.379+0000 7f19719ff700 1 --2- 172.21.15.31:0/848334672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 0x7f19580051c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f19640094d0 tx=0x7f196400b040 comp rx=0 tx=0).stop 2023-10-06T05:26:06.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.379+0000 7f196b7fe700 1 -- 172.21.15.31:0/848334672 reap_dead start 2023-10-06T05:26:06.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.379+0000 7f19719ff700 1 -- 172.21.15.31:0/848334672 shutdown_connections 2023-10-06T05:26:06.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.379+0000 7f19719ff700 1 -- 172.21.15.31:0/848334672 >> 172.21.15.31:0/848334672 conn(0x7f196c099710 msgr2=0x7f196c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:06.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.379+0000 7f19719ff700 1 -- 172.21.15.31:0/848334672 shutdown_connections 2023-10-06T05:26:06.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.379+0000 7f19719ff700 1 -- 172.21.15.31:0/848334672 wait complete. 2023-10-06T05:26:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.381+0000 7f19719ff700 1 Processor -- start 2023-10-06T05:26:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.381+0000 7f19719ff700 1 -- start start 2023-10-06T05:26:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f19719ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 0x7f196c156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f19719ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c0c9590 0x7f196c15e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f19719ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c0ade40 0x7f196c0ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f19719ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0c3dd0 con 0x7f1958004d50 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f19719ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0ae820 con 0x7f196c0ade40 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f19719ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c15c510 con 0x7f196c0c9590 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f196affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 0x7f196c156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f196affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 0x7f196c156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51852/0 (socket says 172.21.15.31:51852) 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f196affd700 1 -- 172.21.15.31:0/1357522992 learned_addr learned my addr 172.21.15.31:0/1357522992 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f196b7fe700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c0ade40 0x7f196c0ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f1963fff700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c0c9590 0x7f196c15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f196affd700 1 -- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c0c9590 msgr2=0x7f196c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f196affd700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c0c9590 0x7f196c15e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f196affd700 1 -- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c0ade40 msgr2=0x7f196c0ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f196affd700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c0ade40 0x7f196c0ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f196affd700 1 -- 172.21.15.31:0/1357522992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c157090 con 0x7f1958004d50 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.382+0000 7f196b7fe700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c0ade40 0x7f196c0ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.383+0000 7f196affd700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 0x7f196c156b50 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f195c011b10 tx=0x7f195c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.383+0000 7f1962ffd700 1 -- 172.21.15.31:0/1357522992 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c005c10 con 0x7f1958004d50 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.383+0000 7f1962ffd700 1 -- 172.21.15.31:0/1357522992 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c005d70 con 0x7f1958004d50 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.383+0000 7f1962ffd700 1 -- 172.21.15.31:0/1357522992 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c00fa10 con 0x7f1958004d50 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.383+0000 7f19719ff700 1 -- 172.21.15.31:0/1357522992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c0c9b50 con 0x7f1958004d50 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.383+0000 7f19719ff700 1 -- 172.21.15.31:0/1357522992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c0ca110 con 0x7f1958004d50 2023-10-06T05:26:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.384+0000 7f19719ff700 1 -- 172.21.15.31:0/1357522992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c15c2e0 con 0x7f1958004d50 2023-10-06T05:26:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.385+0000 7f1962ffd700 1 -- 172.21.15.31:0/1357522992 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f195c006050 con 0x7f1958004d50 2023-10-06T05:26:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.386+0000 7f1962ffd700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f194c067830 0x7f194c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.386+0000 7f1962ffd700 1 -- 172.21.15.31:0/1357522992 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f195c083890 con 0x7f1958004d50 2023-10-06T05:26:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.386+0000 7f1963fff700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f194c067830 0x7f194c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.387+0000 7f1963fff700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f194c067830 0x7f194c069ce0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1954001c50 tx=0x7f1954009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.390+0000 7f1962ffd700 1 -- 172.21.15.31:0/1357522992 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f196c15c2e0 con 0x7f1958004d50 2023-10-06T05:26:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.637+0000 7f19719ff700 1 -- 172.21.15.31:0/1357522992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f196c15c2e0 con 0x7f1958004d50 2023-10-06T05:26:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.638+0000 7f1962ffd700 1 -- 172.21.15.31:0/1357522992 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f196c15c2e0 con 0x7f1958004d50 2023-10-06T05:26:06.640 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:06.640 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":112,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4242992570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4242992570},{"type":"v1","addr":"172.21.15.31:6835","nonce":4242992570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":7706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2405114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2405114362},{"type":"v1","addr":"172.21.15.62:6834","nonce":2405114362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2676480318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2676480318},{"type":"v1","addr":"172.21.15.62:6835","nonce":2676480318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:02.260523+0000","modified":"2023-10-06T05:26:06.149545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9249":{"gid":9249,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3001391649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3001391649},{"type":"v1","addr":"172.21.15.31:6834","nonce":3001391649}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:26:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 -- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f194c067830 msgr2=0x7f194c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f194c067830 0x7f194c069ce0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1954001c50 tx=0x7f1954009040 comp rx=0 tx=0).stop 2023-10-06T05:26:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 -- 172.21.15.31:0/1357522992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 msgr2=0x7f196c156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 0x7f196c156b50 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f195c011b10 tx=0x7f195c009120 comp rx=0 tx=0).stop 2023-10-06T05:26:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 -- 172.21.15.31:0/1357522992 shutdown_connections 2023-10-06T05:26:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f194c067830 0x7f194c069ce0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c0ade40 0x7f196c0ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1958004d50 0x7f196c156b50 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 --2- 172.21.15.31:0/1357522992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f196c0c9590 0x7f196c15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 -- 172.21.15.31:0/1357522992 >> 172.21.15.31:0/1357522992 conn(0x7f196c099710 msgr2=0x7f196c158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 -- 172.21.15.31:0/1357522992 shutdown_connections 2023-10-06T05:26:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.640+0000 7f19719ff700 1 -- 172.21.15.31:0/1357522992 wait complete. 2023-10-06T05:26:06.644 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 112 2023-10-06T05:26:06.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:06.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.854+0000 7f9c93dff700 1 Processor -- start 2023-10-06T05:26:06.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.854+0000 7f9c93dff700 1 -- start start 2023-10-06T05:26:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.855+0000 7f9c93dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c8c0c7b60 0x7f9c8c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.855+0000 7f9c93dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c8c0c7f30 0x7f9c8c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.855+0000 7f9c93dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8c0ca5b0 con 0x7f9c8c06ef50 2023-10-06T05:26:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.855+0000 7f9c93dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8c06f740 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.855+0000 7f9c93dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8c0c9c80 con 0x7f9c8c0c7f30 2023-10-06T05:26:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.855+0000 7f9c91b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c8c0c7b60 0x7f9c8c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.856+0000 7f9c91b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c8c0c7b60 0x7f9c8c069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51856/0 (socket says 172.21.15.31:51856) 2023-10-06T05:26:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.856+0000 7f9c91b9b700 1 -- 172.21.15.31:0/3041892794 learned_addr learned my addr 172.21.15.31:0/3041892794 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.856+0000 7f9c9139a700 1 --2- 172.21.15.31:0/3041892794 >> v2:172.21.15.62:3300/0 conn(0x7f9c8c0c7f30 0x7f9c8c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.856+0000 7f9c91b9b700 1 -- 172.21.15.31:0/3041892794 >> v2:172.21.15.62:3300/0 conn(0x7f9c8c0c7f30 msgr2=0x7f9c8c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.856+0000 7f9c91b9b700 1 --2- 172.21.15.31:0/3041892794 >> v2:172.21.15.62:3300/0 conn(0x7f9c8c0c7f30 0x7f9c8c0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.856+0000 7f9c91b9b700 1 -- 172.21.15.31:0/3041892794 >> v1:172.21.15.62:6789/0 conn(0x7f9c8c06ef50 legacy=0x7f9c8c13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.856+0000 7f9c91b9b700 1 -- 172.21.15.31:0/3041892794 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c0c9f60 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.856+0000 7f9c91b9b700 1 --2- 172.21.15.31:0/3041892794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 0x7f9c8c069880 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009990 tx=0x7f9c88009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbbe2bef357a756 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:06.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.856+0000 7f9c90b99700 1 -- 172.21.15.31:0/3041892794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88009e80 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.857+0000 7f9c90b99700 1 -- 172.21.15.31:0/3041892794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c88005c10 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.857+0000 7f9c90b99700 1 -- 172.21.15.31:0/3041892794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800b610 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.857+0000 7f9c93dff700 1 -- 172.21.15.31:0/3041892794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 msgr2=0x7f9c8c069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.857+0000 7f9c93dff700 1 --2- 172.21.15.31:0/3041892794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 0x7f9c8c069880 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009990 tx=0x7f9c88009220 comp rx=0 tx=0).stop 2023-10-06T05:26:06.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.857+0000 7f9c93dff700 1 -- 172.21.15.31:0/3041892794 shutdown_connections 2023-10-06T05:26:06.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.857+0000 7f9c93dff700 1 --2- 172.21.15.31:0/3041892794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 0x7f9c8c069880 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.857+0000 7f9c93dff700 1 --2- 172.21.15.31:0/3041892794 >> v2:172.21.15.62:3300/0 conn(0x7f9c8c0c7f30 0x7f9c8c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.857+0000 7f9c93dff700 1 -- 172.21.15.31:0/3041892794 >> 172.21.15.31:0/3041892794 conn(0x7f9c8c0cc0b0 msgr2=0x7f9c8c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:06.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.857+0000 7f9c93dff700 1 -- 172.21.15.31:0/3041892794 shutdown_connections 2023-10-06T05:26:06.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.857+0000 7f9c93dff700 1 -- 172.21.15.31:0/3041892794 wait complete. 2023-10-06T05:26:06.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.859+0000 7f9c93dff700 1 Processor -- start 2023-10-06T05:26:06.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.859+0000 7f9c93dff700 1 -- start start 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c93dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c8c06ef50 0x7f9c8c157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c93dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 0x7f9c8c158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c93dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c8c0c7f30 0x7f9c8c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c93dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c15dde0 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c93dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c157b80 con 0x7f9c8c0c7f30 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c93dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c15d4b0 con 0x7f9c8c06ef50 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c9139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 0x7f9c8c158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c9139a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 0x7f9c8c158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51862/0 (socket says 172.21.15.31:51862) 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c9139a700 1 -- 172.21.15.31:0/3227507190 learned_addr learned my addr 172.21.15.31:0/3227507190 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c9239c700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c8c0c7f30 0x7f9c8c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.860+0000 7f9c91b9b700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c8c06ef50 0x7f9c8c157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c9139a700 1 -- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c8c06ef50 msgr2=0x7f9c8c157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c9139a700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c8c06ef50 0x7f9c8c157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c9139a700 1 -- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c8c0c7f30 msgr2=0x7f9c8c15e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c9139a700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c8c0c7f30 0x7f9c8c15e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c9139a700 1 -- 172.21.15.31:0/3227507190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c88009480 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c91b9b700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c8c06ef50 0x7f9c8c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c9239c700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c8c0c7f30 0x7f9c8c15e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c9139a700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 0x7f9c8c158fe0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00ecb0 tx=0x7f9c7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c81ffb700 1 -- 172.21.15.31:0/3227507190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00c9d0 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c81ffb700 1 -- 172.21.15.31:0/3227507190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c00cb30 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c81ffb700 1 -- 172.21.15.31:0/3227507190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00ced0 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c93dff700 1 -- 172.21.15.31:0/3227507190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c15d790 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.861+0000 7f9c93dff700 1 -- 172.21.15.31:0/3227507190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c157890 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.862+0000 7f9c93dff700 1 -- 172.21.15.31:0/3227507190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8c15df90 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.863+0000 7f9c81ffb700 1 -- 172.21.15.31:0/3227507190 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f9c7c013680 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.864+0000 7f9c81ffb700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c70067830 0x7f9c70069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.864+0000 7f9c81ffb700 1 -- 172.21.15.31:0/3227507190 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f9c7c083440 con 0x7f9c8c0c7b60 2023-10-06T05:26:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.865+0000 7f9c91b9b700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c70067830 0x7f9c70069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.866+0000 7f9c91b9b700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c70067830 0x7f9c70069ce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009260 tx=0x7f9c88016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:06.868+0000 7f9c81ffb700 1 -- 172.21.15.31:0/3227507190 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c8c15df90 con 0x7f9c8c0c7b60 2023-10-06T05:26:07.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.071+0000 7f9c93dff700 1 -- 172.21.15.31:0/3227507190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c8c15df90 con 0x7f9c8c0c7b60 2023-10-06T05:26:07.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.072+0000 7f9c81ffb700 1 -- 172.21.15.31:0/3227507190 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+16972 (secure 0 0 0) 0x7f9c7c071260 con 0x7f9c8c0c7b60 2023-10-06T05:26:07.074 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:07.074 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":163,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:26:03.756620+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:00.480986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:01.238722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:07.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.074+0000 7f9c93dff700 1 -- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c70067830 msgr2=0x7f9c70069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.074+0000 7f9c93dff700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c70067830 0x7f9c70069ce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009260 tx=0x7f9c88016000 comp rx=0 tx=0).stop 2023-10-06T05:26:07.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.074+0000 7f9c93dff700 1 -- 172.21.15.31:0/3227507190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 msgr2=0x7f9c8c158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.074+0000 7f9c93dff700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 0x7f9c8c158fe0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00ecb0 tx=0x7f9c7c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.075+0000 7f9c93dff700 1 -- 172.21.15.31:0/3227507190 shutdown_connections 2023-10-06T05:26:07.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.075+0000 7f9c93dff700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f9c70067830 0x7f9c70069ce0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.075+0000 7f9c93dff700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c8c0c7f30 0x7f9c8c15e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.075+0000 7f9c93dff700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c8c06ef50 0x7f9c8c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.075+0000 7f9c93dff700 1 --2- 172.21.15.31:0/3227507190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c8c0c7b60 0x7f9c8c158fe0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.075+0000 7f9c93dff700 1 -- 172.21.15.31:0/3227507190 >> 172.21.15.31:0/3227507190 conn(0x7f9c8c0cc0b0 msgr2=0x7f9c8c0c1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:07.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.075+0000 7f9c93dff700 1 -- 172.21.15.31:0/3227507190 shutdown_connections 2023-10-06T05:26:07.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.075+0000 7f9c93dff700 1 -- 172.21.15.31:0/3227507190 wait complete. 2023-10-06T05:26:07.095 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:26:07.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:26:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.290+0000 7f07345ff700 1 Processor -- start 2023-10-06T05:26:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.290+0000 7f07345ff700 1 -- start start 2023-10-06T05:26:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.291+0000 7f07345ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f072c13a1e0 0x7f072c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.291+0000 7f07345ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f072c06db50 0x7f072c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.291+0000 7f07345ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c0c71b0 con 0x7f072c13a5b0 2023-10-06T05:26:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.291+0000 7f07345ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072c0c7490 con 0x7f072c13a1e0 2023-10-06T05:26:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.291+0000 7f07345ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072c0c7770 con 0x7f072c06db50 2023-10-06T05:26:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.292+0000 7f073239b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f072c13a1e0 0x7f072c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.292+0000 7f073239b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f072c13a1e0 0x7f072c0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55142/0 (socket says 172.21.15.31:55142) 2023-10-06T05:26:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.292+0000 7f073239b700 1 -- 172.21.15.31:0/2381017667 learned_addr learned my addr 172.21.15.31:0/2381017667 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.292+0000 7f0732b9c700 1 --2- 172.21.15.31:0/2381017667 >> v2:172.21.15.62:3300/0 conn(0x7f072c06db50 0x7f072c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.292+0000 7f073239b700 1 -- 172.21.15.31:0/2381017667 >> v2:172.21.15.62:3300/0 conn(0x7f072c06db50 msgr2=0x7f072c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.292+0000 7f073239b700 1 --2- 172.21.15.31:0/2381017667 >> v2:172.21.15.62:3300/0 conn(0x7f072c06db50 0x7f072c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.292+0000 7f073239b700 1 -- 172.21.15.31:0/2381017667 >> v1:172.21.15.62:6789/0 conn(0x7f072c13a5b0 legacy=0x7f072c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.292+0000 7f073239b700 1 -- 172.21.15.31:0/2381017667 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0c7a50 con 0x7f072c13a1e0 2023-10-06T05:26:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.292+0000 7f073239b700 1 --2- 172.21.15.31:0/2381017667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c13a1e0 0x7f072c0c66f0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f072c06b920 tx=0x7f0728009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7346761760cde7c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f0731399700 1 -- 172.21.15.31:0/2381017667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072800b820 con 0x7f072c13a1e0 2023-10-06T05:26:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f0731399700 1 -- 172.21.15.31:0/2381017667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072800b980 con 0x7f072c13a1e0 2023-10-06T05:26:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f0731399700 1 -- 172.21.15.31:0/2381017667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072800bd20 con 0x7f072c13a1e0 2023-10-06T05:26:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f07345ff700 1 -- 172.21.15.31:0/2381017667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c13a1e0 msgr2=0x7f072c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f07345ff700 1 --2- 172.21.15.31:0/2381017667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c13a1e0 0x7f072c0c66f0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f072c06b920 tx=0x7f0728009250 comp rx=0 tx=0).stop 2023-10-06T05:26:07.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f07345ff700 1 -- 172.21.15.31:0/2381017667 shutdown_connections 2023-10-06T05:26:07.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f07345ff700 1 --2- 172.21.15.31:0/2381017667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c13a1e0 0x7f072c0c66f0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f07345ff700 1 --2- 172.21.15.31:0/2381017667 >> v2:172.21.15.62:3300/0 conn(0x7f072c06db50 0x7f072c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f07345ff700 1 -- 172.21.15.31:0/2381017667 >> 172.21.15.31:0/2381017667 conn(0x7f072c0cc0b0 msgr2=0x7f072c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:07.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f07345ff700 1 -- 172.21.15.31:0/2381017667 shutdown_connections 2023-10-06T05:26:07.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.293+0000 7f07345ff700 1 -- 172.21.15.31:0/2381017667 wait complete. 2023-10-06T05:26:07.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.295+0000 7f07345ff700 1 Processor -- start 2023-10-06T05:26:07.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.295+0000 7f07345ff700 1 -- start start 2023-10-06T05:26:07.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.296+0000 7f07345ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c06db50 0x7f072c0c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.296+0000 7f07345ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f072c13a1e0 0x7f072c138960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.296+0000 7f07345ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c13a5b0 0x7f072c0b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.296+0000 7f07345ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0af1a0 con 0x7f072c06db50 2023-10-06T05:26:07.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.296+0000 7f07345ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0c3160 con 0x7f072c13a5b0 2023-10-06T05:26:07.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.296+0000 7f07345ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0c5ff0 con 0x7f072c13a1e0 2023-10-06T05:26:07.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.296+0000 7f073239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c06db50 0x7f072c0c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.296+0000 7f073239b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c06db50 0x7f072c0c5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55152/0 (socket says 172.21.15.31:55152) 2023-10-06T05:26:07.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.296+0000 7f073239b700 1 -- 172.21.15.31:0/609781058 learned_addr learned my addr 172.21.15.31:0/609781058 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f0732b9c700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c13a5b0 0x7f072c0b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f073239b700 1 -- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f072c13a1e0 msgr2=0x7f072c138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f0731b9a700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f072c13a1e0 0x7f072c138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f073239b700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f072c13a1e0 0x7f072c138960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f073239b700 1 -- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c13a5b0 msgr2=0x7f072c0b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f073239b700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c13a5b0 0x7f072c0b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f073239b700 1 -- 172.21.15.31:0/609781058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0728009490 con 0x7f072c06db50 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f0732b9c700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c13a5b0 0x7f072c0b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f0731b9a700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f072c13a1e0 0x7f072c138960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f073239b700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c06db50 0x7f072c0c5ab0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f072c0cc010 tx=0x7f0728016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f07227fc700 1 -- 172.21.15.31:0/609781058 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728011c80 con 0x7f072c06db50 2023-10-06T05:26:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f07227fc700 1 -- 172.21.15.31:0/609781058 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728020070 con 0x7f072c06db50 2023-10-06T05:26:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f07227fc700 1 -- 172.21.15.31:0/609781058 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072801b6a0 con 0x7f072c06db50 2023-10-06T05:26:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.297+0000 7f07345ff700 1 -- 172.21.15.31:0/609781058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c0cb190 con 0x7f072c06db50 2023-10-06T05:26:07.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.298+0000 7f07345ff700 1 -- 172.21.15.31:0/609781058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c0c33b0 con 0x7f072c06db50 2023-10-06T05:26:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.299+0000 7f07345ff700 1 -- 172.21.15.31:0/609781058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c04c380 con 0x7f072c06db50 2023-10-06T05:26:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.299+0000 7f07227fc700 1 -- 172.21.15.31:0/609781058 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f0728022030 con 0x7f072c06db50 2023-10-06T05:26:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.300+0000 7f07227fc700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0710067830 0x7f0710069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.300+0000 7f07227fc700 1 -- 172.21.15.31:0/609781058 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f0728084dc0 con 0x7f072c06db50 2023-10-06T05:26:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.301+0000 7f0731b9a700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0710067830 0x7f0710069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.301+0000 7f0731b9a700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0710067830 0x7f0710069ce0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f071c006fd0 tx=0x7f071c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.305+0000 7f07227fc700 1 -- 172.21.15.31:0/609781058 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0728053a00 con 0x7f072c06db50 2023-10-06T05:26:07.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.503+0000 7f07345ff700 1 -- 172.21.15.31:0/609781058 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f072c04c380 con 0x7f0710067830 2023-10-06T05:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.513+0000 7f07227fc700 1 -- 172.21.15.31:0/609781058 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300790 (secure 0 0 0) 0x7f072c04c380 con 0x7f0710067830 2023-10-06T05:26:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:07.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 -- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0710067830 msgr2=0x7f0710069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0710067830 0x7f0710069ce0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f071c006fd0 tx=0x7f071c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:07.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 -- 172.21.15.31:0/609781058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c06db50 msgr2=0x7f072c0c5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c06db50 0x7f072c0c5ab0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f072c0cc010 tx=0x7f0728016000 comp rx=0 tx=0).stop 2023-10-06T05:26:07.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 -- 172.21.15.31:0/609781058 shutdown_connections 2023-10-06T05:26:07.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0710067830 0x7f0710069ce0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c13a5b0 0x7f072c0b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f072c06db50 0x7f072c0c5ab0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 --2- 172.21.15.31:0/609781058 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f072c13a1e0 0x7f072c138960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 -- 172.21.15.31:0/609781058 >> 172.21.15.31:0/609781058 conn(0x7f072c0cc0b0 msgr2=0x7f072c0c6920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:07.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 -- 172.21.15.31:0/609781058 shutdown_connections 2023-10-06T05:26:07.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.517+0000 7f07345ff700 1 -- 172.21.15.31:0/609781058 wait complete. 2023-10-06T05:26:07.525 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:26:07.536 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2023-10-06T05:26:05.526982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427992,"kb_used_data":16920,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546728,"statfs":{"total":773094113280,"available":770607849472,"internally_reserved":0,"allocated":17326080,"data_stored":8688544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.778791"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.753467+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:39.559510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.752894+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:08.246444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.754094+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:29.218407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.753120+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:18.890885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.750186+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:00.405513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.749774+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:39.456096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.754896+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:20.881918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.754113+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:57.906765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.754319+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:43.139452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.753796+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:44.287873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.750123+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:28.927922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.753387+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:39.167215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.753454+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:20.152967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.750049+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:23.204982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.751011+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:50.272983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.752998+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:34.643893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.754997+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:42.090169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.757171+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:39.559510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.750924+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:09.962576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.753351+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:05.101039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.757289+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:50.706678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.753995+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:45.500409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.752944+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:12.345145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.753546+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:50.706678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.750640+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:18.580399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T05:25:58.336209+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:25:58.336209+0000","last_peered":"2023-10-06T05:25:58.336209+0000","last_clean":"2023-10-06T05:25:58.336209+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:25:58.336209+0000","last_undegraded":"2023-10-06T05:25:58.336209+0000","last_fullsized":"2023-10-06T05:25:58.336209+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.753515+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:42.139699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":123,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T05:25:58.336209+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:25:58.336209+0000","last_peered":"2023-10-06T05:25:58.336209+0000","last_clean":"2023-10-06T05:25:58.336209+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:25:58.336209+0000","last_undegraded":"2023-10-06T05:25:58.336209+0000","last_fullsized":"2023-10-06T05:25:58.336209+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.753533+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:44.013558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T05:25:58.339398+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:25:58.339398+0000","last_peered":"2023-10-06T05:25:58.339398+0000","last_clean":"2023-10-06T05:25:58.339398+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:25:58.339398+0000","last_undegraded":"2023-10-06T05:25:58.339398+0000","last_fullsized":"2023-10-06T05:25:58.339398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.754445+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:36.392335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":122,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T05:25:58.337067+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:25:58.337067+0000","last_peered":"2023-10-06T05:25:58.337067+0000","last_clean":"2023-10-06T05:25:58.337067+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:25:58.337067+0000","last_undegraded":"2023-10-06T05:25:58.337067+0000","last_fullsized":"2023-10-06T05:25:58.337067+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.749768+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:11.047523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":122,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T05:25:58.337085+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:25:58.337085+0000","last_peered":"2023-10-06T05:25:58.337085+0000","last_clean":"2023-10-06T05:25:58.337085+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:25:58.337085+0000","last_undegraded":"2023-10-06T05:25:58.337085+0000","last_fullsized":"2023-10-06T05:25:58.337085+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.752911+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:53.070646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":311,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T05:25:58.337082+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:25:58.337082+0000","last_peered":"2023-10-06T05:25:58.337082+0000","last_clean":"2023-10-06T05:25:58.337082+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:25:58.337082+0000","last_undegraded":"2023-10-06T05:25:58.337082+0000","last_fullsized":"2023-10-06T05:25:58.337082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T05:25:57.749874+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:25:57.749874+0000","last_peered":"2023-10-06T05:25:57.749874+0000","last_clean":"2023-10-06T05:25:57.749874+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:25:57.749874+0000","last_undegraded":"2023-10-06T05:25:57.749874+0000","last_fullsized":"2023-10-06T05:25:57.749874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.754423+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:02.869039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":153,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T05:25:57.749336+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:25:57.749336+0000","last_peered":"2023-10-06T05:25:57.749336+0000","last_clean":"2023-10-06T05:25:57.749336+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:25:57.749336+0000","last_undegraded":"2023-10-06T05:25:57.749336+0000","last_fullsized":"2023-10-06T05:25:57.749336+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T05:26:01.747396+0000","last_change":"2023-10-06T05:26:01.747396+0000","last_active":"2023-10-06T05:26:01.747396+0000","last_peered":"2023-10-06T05:26:01.747396+0000","last_clean":"2023-10-06T05:26:01.747396+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:01.747396+0000","last_undegraded":"2023-10-06T05:26:01.747396+0000","last_fullsized":"2023-10-06T05:26:01.747396+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T05:26:00.740435+0000","last_change":"2023-10-06T05:26:00.750544+0000","last_active":"2023-10-06T05:26:00.740435+0000","last_peered":"2023-10-06T05:26:00.740435+0000","last_clean":"2023-10-06T05:26:00.740435+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T05:26:00.740435+0000","last_undegraded":"2023-10-06T05:26:00.740435+0000","last_fullsized":"2023-10-06T05:26:00.740435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:46.322556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":139,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T05:25:58.336567+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:25:58.336567+0000","last_peered":"2023-10-06T05:25:58.336567+0000","last_clean":"2023-10-06T05:25:58.336567+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:25:58.336567+0000","last_undegraded":"2023-10-06T05:25:58.336567+0000","last_fullsized":"2023-10-06T05:25:58.336567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303940,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067900,"statfs":{"total":96636764160,"available":96325529600,"internally_reserved":0,"allocated":2617344,"data_stored":1531937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.50800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71199999999999997,"15min":0.72099999999999997},"min":{"1min":0.51800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83899999999999997,"5min":1.081,"15min":1.081},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70499999999999996,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.48099999999999998,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91700000000000004,"5min":0.93600000000000005,"15min":1.018},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.73699999999999999},"min":{"1min":0.51200000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90000000000000002,"5min":0.97799999999999998,"15min":1.056},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71099999999999997,"15min":0.73999999999999999},"min":{"1min":0.53400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94199999999999995,"5min":1.149,"15min":1.149},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.72699999999999998,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65700000000000003,"15min":0.69999999999999996},"min":{"1min":0.37,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86499999999999999,"5min":0.96199999999999997,"15min":1.0249999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.76600000000000001},"min":{"1min":0.46899999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97499999999999998,"5min":1.27,"15min":1.27},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69699999999999995,"15min":0.72899999999999998},"min":{"1min":0.502,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91300000000000003,"5min":0.97899999999999998,"15min":0.999},"last":0.747},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75,"15min":0.80900000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90600000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79500000000000004,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96499999999999997,"5min":1.0449999999999999,"15min":1.1040000000000001},"last":0.626},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.52200000000000002,"5min":0.46200000000000002,"15min":0.441},"max":{"1min":1.0089999999999999,"5min":1.248,"15min":1.248},"last":0.76900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303348,"kb_used_data":1964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068492,"statfs":{"total":96636764160,"available":96326135808,"internally_reserved":0,"allocated":2011136,"data_stored":935383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63900000000000001,"15min":0.67000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.89800000000000002,"15min":1.8500000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66500000000000004,"15min":0.69699999999999995},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0369999999999999,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69099999999999995,"15min":0.70699999999999996},"min":{"1min":0.50800000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93000000000000005,"5min":1.177,"15min":1.9279999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95799999999999996,"5min":1.073,"15min":1.4239999999999999},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.65200000000000002},"min":{"1min":0.47299999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.877,"5min":1.093,"15min":1.369},"last":0.75},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71299999999999997,"15min":0.751},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.038,"5min":1.085,"15min":1.891},"last":0.68100000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71599999999999997,"15min":0.74099999999999999},"min":{"1min":0.49299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.347},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.71399999999999997},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.98699999999999999,"15min":1},"last":0.82799999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70999999999999996,"15min":0.75900000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.9670000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77800000000000002,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0209999999999999,"5min":1.278,"15min":1.4510000000000001},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.50700000000000001,"5min":0.41999999999999998,"15min":0.41499999999999998},"max":{"1min":0.82899999999999996,"5min":1.0680000000000001,"15min":1.415},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.753,"15min":0.80400000000000005},"min":{"1min":0.52300000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1020000000000001,"5min":1.2350000000000001,"15min":2},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77200000000000002,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.016,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76400000000000001,"15min":0.77500000000000002},"min":{"1min":0.54900000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95799999999999996,"5min":1.133,"15min":1.46},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303928,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067912,"statfs":{"total":96636764160,"available":96325541888,"internally_reserved":0,"allocated":2605056,"data_stored":1525751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.44,"15min":0.436},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.621,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.48599999999999999},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.504,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.5}]},{"osd":2,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.54100000000000004,"5min":0.77000000000000002,"15min":0.83799999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.59999999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.44900000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.58499999999999996,"5min":0.80500000000000005,"15min":0.92400000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51000000000000001,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.57299999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.52400000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0680000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79200000000000004,"15min":0.79300000000000004},"min":{"1min":0.63600000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0189999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74099999999999999,"15min":0.74399999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.81000000000000005},"min":{"1min":0.61599999999999999,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":1.002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.84799999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73399999999999999,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93799999999999994,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1140000000000001,"5min":1.387,"15min":1.387},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.50600000000000001},"max":{"1min":1.044,"5min":1.244,"15min":1.244},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81399999999999995,"15min":0.82199999999999995},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.81200000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574962,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303348,"kb_used_data":1964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068492,"statfs":{"total":96636764160,"available":96326135808,"internally_reserved":0,"allocated":2011136,"data_stored":935383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41499999999999998,"15min":0.42799999999999999},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.51300000000000001,"5min":0.64500000000000002,"15min":0.84499999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.44500000000000001,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":1.042},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.48799999999999999},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.624,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.63,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.498},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.48199999999999998,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.67500000000000004,"5min":0.80100000000000005,"15min":0.90500000000000003},"last":0.53100000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.745},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.80000000000000004,"5min":1.1519999999999999,"15min":1.242},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.76700000000000002},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97299999999999998,"5min":0.97899999999999998,"15min":1.198},"last":0.751}]},{"osd":5,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73399999999999999,"15min":0.76100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.90900000000000003,"5min":1.1659999999999999,"15min":1.26},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73099999999999998,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84099999999999997,"5min":1.1299999999999999,"15min":1.163},"last":0.82399999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73499999999999999,"15min":0.78600000000000003},"min":{"1min":0.55300000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92500000000000004,"5min":1.105,"15min":1.323},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76300000000000001,"15min":0.78600000000000003},"min":{"1min":0.54600000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95099999999999996,"5min":1.123,"15min":1.3100000000000001},"last":0.88}]},{"osd":7,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77400000000000002,"15min":0.82199999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.52400000000000002},"max":{"1min":0.84999999999999998,"5min":1.2010000000000001,"15min":1.264},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.81100000000000005},"min":{"1min":0.58199999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":0.77500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":1976,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":2023424,"data_stored":941569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.77800000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.46000000000000002,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.55400000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45100000000000001,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.57399999999999995,"5min":0.71699999999999997,"15min":0.88700000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.31900000000000001},"max":{"1min":0.57799999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.627,"5min":0.80600000000000005,"15min":0.93600000000000005},"last":0.501},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.38500000000000001,"5min":0.372,"15min":0.36199999999999999},"max":{"1min":0.57299999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.997,"5min":1.022,"15min":1.1299999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.77800000000000002},"min":{"1min":0.53300000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.018,"5min":1.085,"15min":1.1699999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.52400000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91600000000000004,"5min":1.042,"15min":1.099},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73699999999999999,"15min":0.78900000000000003},"min":{"1min":0.55600000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.163},"last":1.0229999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.81799999999999995},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98599999999999999,"5min":1.056,"15min":1.21},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.95399999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.79900000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1200000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.997},{"interface":"front","average":{"1min":0.747,"5min":0.77100000000000002,"15min":0.83899999999999997},"min":{"1min":0.58999999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97099999999999997,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":0.878}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":1976,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":2023424,"data_stored":941569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42999999999999999,"15min":0.45300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.65500000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.58299999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.45100000000000001,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54000000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.497,"15min":0.52300000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72199999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.504,"15min":0.53500000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.57699999999999996,"5min":0.999,"15min":0.999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.53400000000000003,"15min":0.56599999999999995},"min":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.58599999999999997,"5min":0.98999999999999999,"15min":1.014},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.77400000000000002,"15min":0.80100000000000005},"min":{"1min":0.53300000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.0800000000000001,"5min":1.145,"15min":1.2669999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.82299999999999995},"min":{"1min":0.64000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.016,"5min":1.1830000000000001,"15min":1.3540000000000001},"last":0.90600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72899999999999998,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93100000000000005,"5min":1.093,"15min":1.204},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77100000000000002,"15min":0.81999999999999995},"min":{"1min":0.52400000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":1.208,"15min":1.329},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75600000000000001,"15min":0.80000000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.96899999999999997,"5min":1.0880000000000001,"15min":1.484},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77400000000000002,"15min":0.82899999999999996},"min":{"1min":0.55900000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98299999999999998,"5min":1.2809999999999999,"15min":1.629},"last":0.88600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78300000000000003,"15min":0.83599999999999997},"min":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0469999999999999,"5min":1.1950000000000001,"15min":1.5620000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.79100000000000004,"15min":0.82199999999999995},"min":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":1.0309999999999999,"5min":1.194,"15min":1.325},"last":0.995}]}]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303360,"kb_used_data":1976,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068480,"statfs":{"total":96636764160,"available":96326123520,"internally_reserved":0,"allocated":2023424,"data_stored":941569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.751},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.75600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.95899999999999996,"5min":1.02,"15min":2.77},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.46899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.6419999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67900000000000005,"15min":0.71099999999999997},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.99199999999999999,"5min":1.1100000000000001,"15min":1.1899999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.39800000000000002},"max":{"1min":0.93799999999999994,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71799999999999997,"15min":0.79300000000000004},"min":{"1min":0.52100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.52900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89000000000000001,"5min":0.95899999999999996,"15min":1.0629999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70799999999999996,"15min":0.96299999999999997},"min":{"1min":0.52700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97899999999999998,"5min":0.99299999999999999,"15min":7.742},"last":0.82899999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.51000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.079},"last":0.63},{"interface":"front","average":{"1min":0.755,"5min":0.72499999999999998,"15min":0.746},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.012,"5min":1.04,"15min":1.181},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92900000000000005,"5min":1.0029999999999999,"15min":1.2669999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77300000000000002,"15min":0.80800000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.436},"max":{"1min":0.95399999999999996,"5min":1.093,"15min":1.171},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78600000000000003,"15min":0.81999999999999995},"min":{"1min":0.69599999999999995,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0589999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78400000000000003,"15min":0.83999999999999997},"min":{"1min":0.59199999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.403,"5min":1.403,"15min":1.571},"last":0.88100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574962,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303348,"kb_used_data":1964,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068492,"statfs":{"total":96636764160,"available":96326135808,"internally_reserved":0,"allocated":2011136,"data_stored":935383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63400000000000001,"15min":0.878},"min":{"1min":0.38400000000000001,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.95599999999999996,"5min":1.109,"15min":9.2780000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0469999999999999},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66800000000000004,"15min":0.90400000000000003},"min":{"1min":0.433,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.874,"5min":1.079,"15min":9.25},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67800000000000005,"15min":0.95399999999999996},"min":{"1min":0.48699999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.63}]},{"osd":2,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.92500000000000004},"min":{"1min":0.51600000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84999999999999998,"5min":1.052,"15min":10.384},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71099999999999997,"15min":0.95399999999999996},"min":{"1min":0.48199999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.98699999999999999,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.98799999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.495},"max":{"1min":0.88900000000000001,"5min":1.76,"15min":10.205},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70299999999999996,"15min":0.71999999999999997},"min":{"1min":0.51600000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.024,"5min":1.097,"15min":1.292},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.40600000000000003},"max":{"1min":0.95699999999999996,"5min":1.069,"15min":1.069},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0469999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.443}]},{"osd":6,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.73299999999999998},"min":{"1min":0.55300000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91300000000000003,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76400000000000001,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.298},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78000000000000003,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98699999999999999,"5min":1.556,"15min":1.556},"last":0.751},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76800000000000002,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.121,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:26:07.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:26:07.734 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-10-06T05:26:07.732+0000 7f45149f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:26:07.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.771+0000 7fd5b99ff700 1 Processor -- start 2023-10-06T05:26:07.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.771+0000 7fd5b99ff700 1 -- start start 2023-10-06T05:26:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.772+0000 7fd5b99ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b413a1e0 0x7fd5b40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.772+0000 7fd5b99ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd5b413a5b0 0x7fd5b406d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.772+0000 7fd5b99ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b40c71b0 con 0x7fd5b40c6c30 2023-10-06T05:26:07.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.772+0000 7fd5b99ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b40c7490 con 0x7fd5b413a5b0 2023-10-06T05:26:07.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.772+0000 7fd5b99ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40c7770 con 0x7fd5b413a1e0 2023-10-06T05:26:07.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.773+0000 7fd5b27fc700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd5b413a5b0 0x7fd5b406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.773+0000 7fd5b2ffd700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b413a1e0 0x7fd5b40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.773+0000 7fd5b27fc700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd5b413a5b0 0x7fd5b406d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:50688/0 (socket says 172.21.15.31:50688) 2023-10-06T05:26:07.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.773+0000 7fd5b27fc700 1 -- 172.21.15.31:0/1171714297 learned_addr learned my addr 172.21.15.31:0/1171714297 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.773+0000 7fd5b27fc700 1 -- 172.21.15.31:0/1171714297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b413a1e0 msgr2=0x7fd5b40c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.773+0000 7fd5b27fc700 1 --2- 172.21.15.31:0/1171714297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b413a1e0 0x7fd5b40c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.773+0000 7fd5b27fc700 1 -- 172.21.15.31:0/1171714297 >> v1:172.21.15.62:6789/0 conn(0x7fd5b40c6c30 legacy=0x7fd5b415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.773+0000 7fd5b27fc700 1 -- 172.21.15.31:0/1171714297 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b40c7a50 con 0x7fd5b413a5b0 2023-10-06T05:26:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.774+0000 7fd5b27fc700 1 --2- 172.21.15.31:0/1171714297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b413a5b0 0x7fd5b406d440 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80099b0 tx=0x7fd5a8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26eb10e589caff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.774+0000 7fd5b1ffb700 1 -- 172.21.15.31:0/1171714297 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8009ea0 con 0x7fd5b413a5b0 2023-10-06T05:26:07.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.774+0000 7fd5b1ffb700 1 -- 172.21.15.31:0/1171714297 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a800b860 con 0x7fd5b413a5b0 2023-10-06T05:26:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.774+0000 7fd5b1ffb700 1 -- 172.21.15.31:0/1171714297 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800bc00 con 0x7fd5b413a5b0 2023-10-06T05:26:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.775+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1171714297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b413a5b0 msgr2=0x7fd5b406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.775+0000 7fd5b99ff700 1 --2- 172.21.15.31:0/1171714297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b413a5b0 0x7fd5b406d440 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80099b0 tx=0x7fd5a8009250 comp rx=0 tx=0).stop 2023-10-06T05:26:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.775+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1171714297 shutdown_connections 2023-10-06T05:26:07.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.775+0000 7fd5b99ff700 1 --2- 172.21.15.31:0/1171714297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b413a1e0 0x7fd5b40c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.775+0000 7fd5b99ff700 1 --2- 172.21.15.31:0/1171714297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b413a5b0 0x7fd5b406d440 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.775+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1171714297 >> 172.21.15.31:0/1171714297 conn(0x7fd5b40cc0b0 msgr2=0x7fd5b40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:07.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.775+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1171714297 shutdown_connections 2023-10-06T05:26:07.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.775+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1171714297 wait complete. 2023-10-06T05:26:07.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.777+0000 7fd5b99ff700 1 Processor -- start 2023-10-06T05:26:07.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.777+0000 7fd5b99ff700 1 -- start start 2023-10-06T05:26:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b99ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b40c6c30 0x7fd5b40c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b99ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b413a1e0 0x7fd5b40b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b99ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b413a5b0 0x7fd5b40b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b99ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40aff90 con 0x7fd5b40c6c30 2023-10-06T05:26:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b99ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40c6010 con 0x7fd5b413a1e0 2023-10-06T05:26:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b99ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40af660 con 0x7fd5b413a5b0 2023-10-06T05:26:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b40c6c30 0x7fd5b40c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b40c6c30 0x7fd5b40c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55162/0 (socket says 172.21.15.31:55162) 2023-10-06T05:26:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b2ffd700 1 -- 172.21.15.31:0/1370944745 learned_addr learned my addr 172.21.15.31:0/1370944745 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b27fc700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b413a1e0 0x7fd5b40b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b37fe700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b413a5b0 0x7fd5b40b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b2ffd700 1 -- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b413a5b0 msgr2=0x7fd5b40b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b2ffd700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b413a5b0 0x7fd5b40b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b2ffd700 1 -- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b413a1e0 msgr2=0x7fd5b40b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b2ffd700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b413a1e0 0x7fd5b40b0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.778+0000 7fd5b2ffd700 1 -- 172.21.15.31:0/1370944745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a80094f0 con 0x7fd5b40c6c30 2023-10-06T05:26:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.779+0000 7fd5b2ffd700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b40c6c30 0x7fd5b40c57e0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400ec80 tx=0x7fd5a4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.779+0000 7fd59affd700 1 -- 172.21.15.31:0/1370944745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a400c850 con 0x7fd5b40c6c30 2023-10-06T05:26:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.779+0000 7fd59affd700 1 -- 172.21.15.31:0/1370944745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a400c9b0 con 0x7fd5b40c6c30 2023-10-06T05:26:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.779+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1370944745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b40af940 con 0x7fd5b40c6c30 2023-10-06T05:26:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.779+0000 7fd59affd700 1 -- 172.21.15.31:0/1370944745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a400cd30 con 0x7fd5b40c6c30 2023-10-06T05:26:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.779+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1370944745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b40c5d20 con 0x7fd5b40c6c30 2023-10-06T05:26:07.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.780+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1370944745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b404c380 con 0x7fd5b40c6c30 2023-10-06T05:26:07.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.781+0000 7fd59affd700 1 -- 172.21.15.31:0/1370944745 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fd5a4013680 con 0x7fd5b40c6c30 2023-10-06T05:26:07.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.782+0000 7fd59affd700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd5a0067830 0x7fd5a0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:07.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.782+0000 7fd59affd700 1 -- 172.21.15.31:0/1370944745 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd5a4083600 con 0x7fd5b40c6c30 2023-10-06T05:26:07.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.783+0000 7fd5b27fc700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd5a0067830 0x7fd5a0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:07.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.783+0000 7fd5b27fc700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd5a0067830 0x7fd5a0069ce0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80099b0 tx=0x7fd5a8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:07.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.786+0000 7fd59affd700 1 -- 172.21.15.31:0/1370944745 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5a4052050 con 0x7fd5b40c6c30 2023-10-06T05:26:07.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.982+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1370944745 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5b40b00d0 con 0x7fd5a0067830 2023-10-06T05:26:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.993+0000 7fd59affd700 1 -- 172.21.15.31:0/1370944745 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311823 (secure 0 0 0) 0x7fd5a800b9d0 con 0x7fd5a0067830 2023-10-06T05:26:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.996+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd5a0067830 msgr2=0x7fd5a0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.996+0000 7fd5b99ff700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd5a0067830 0x7fd5a0069ce0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80099b0 tx=0x7fd5a8017000 comp rx=0 tx=0).stop 2023-10-06T05:26:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.997+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1370944745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b40c6c30 msgr2=0x7fd5b40c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.997+0000 7fd5b99ff700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b40c6c30 0x7fd5b40c57e0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400ec80 tx=0x7fd5a4009120 comp rx=0 tx=0).stop 2023-10-06T05:26:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.997+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1370944745 shutdown_connections 2023-10-06T05:26:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.997+0000 7fd5b99ff700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd5a0067830 0x7fd5a0069ce0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.997+0000 7fd5b99ff700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5b413a5b0 0x7fd5b40b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.997+0000 7fd5b99ff700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b40c6c30 0x7fd5b40c57e0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.997+0000 7fd5b99ff700 1 --2- 172.21.15.31:0/1370944745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b413a1e0 0x7fd5b40b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.997+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1370944745 >> 172.21.15.31:0/1370944745 conn(0x7fd5b40cc0b0 msgr2=0x7fd5b40c64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.997+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1370944745 shutdown_connections 2023-10-06T05:26:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:07.997+0000 7fd5b99ff700 1 -- 172.21.15.31:0/1370944745 wait complete. 2023-10-06T05:26:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:26:08.016 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2023-10-06T05:26:07.527442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428392,"kb_used_data":17320,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546328,"statfs":{"total":773094113280,"available":770607439872,"internally_reserved":0,"allocated":17735680,"data_stored":8881556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001494"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.789369+0000","last_change":"2023-10-06T05:26:02.787048+0000","last_active":"2023-10-06T05:26:02.789369+0000","last_peered":"2023-10-06T05:26:02.789369+0000","last_clean":"2023-10-06T05:26:02.789369+0000","last_became_active":"2023-10-06T05:26:02.786757+0000","last_became_peered":"2023-10-06T05:26:02.786757+0000","last_unstale":"2023-10-06T05:26:02.789369+0000","last_undegraded":"2023-10-06T05:26:02.789369+0000","last_fullsized":"2023-10-06T05:26:02.789369+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:57.372182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.780483+0000","last_change":"2023-10-06T05:26:02.772033+0000","last_active":"2023-10-06T05:26:02.780483+0000","last_peered":"2023-10-06T05:26:02.780483+0000","last_clean":"2023-10-06T05:26:02.780483+0000","last_became_active":"2023-10-06T05:26:02.771636+0000","last_became_peered":"2023-10-06T05:26:02.771636+0000","last_unstale":"2023-10-06T05:26:02.780483+0000","last_undegraded":"2023-10-06T05:26:02.780483+0000","last_fullsized":"2023-10-06T05:26:02.780483+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:48.242816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.774366+0000","last_change":"2023-10-06T05:26:02.769916+0000","last_active":"2023-10-06T05:26:02.774366+0000","last_peered":"2023-10-06T05:26:02.774366+0000","last_clean":"2023-10-06T05:26:02.774366+0000","last_became_active":"2023-10-06T05:26:02.769645+0000","last_became_peered":"2023-10-06T05:26:02.769645+0000","last_unstale":"2023-10-06T05:26:02.774366+0000","last_undegraded":"2023-10-06T05:26:02.774366+0000","last_fullsized":"2023-10-06T05:26:02.774366+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:46.655381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.774620+0000","last_change":"2023-10-06T05:26:02.770908+0000","last_active":"2023-10-06T05:26:02.774620+0000","last_peered":"2023-10-06T05:26:02.774620+0000","last_clean":"2023-10-06T05:26:02.774620+0000","last_became_active":"2023-10-06T05:26:02.770607+0000","last_became_peered":"2023-10-06T05:26:02.770607+0000","last_unstale":"2023-10-06T05:26:02.774620+0000","last_undegraded":"2023-10-06T05:26:02.774620+0000","last_fullsized":"2023-10-06T05:26:02.774620+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:15.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.778586+0000","last_change":"2023-10-06T05:26:02.775177+0000","last_active":"2023-10-06T05:26:02.778586+0000","last_peered":"2023-10-06T05:26:02.778586+0000","last_clean":"2023-10-06T05:26:02.778586+0000","last_became_active":"2023-10-06T05:26:02.774697+0000","last_became_peered":"2023-10-06T05:26:02.774697+0000","last_unstale":"2023-10-06T05:26:02.778586+0000","last_undegraded":"2023-10-06T05:26:02.778586+0000","last_fullsized":"2023-10-06T05:26:02.778586+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:04.817010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.778540+0000","last_change":"2023-10-06T05:26:02.773286+0000","last_active":"2023-10-06T05:26:02.778540+0000","last_peered":"2023-10-06T05:26:02.778540+0000","last_clean":"2023-10-06T05:26:02.778540+0000","last_became_active":"2023-10-06T05:26:02.772882+0000","last_became_peered":"2023-10-06T05:26:02.772882+0000","last_unstale":"2023-10-06T05:26:02.778540+0000","last_undegraded":"2023-10-06T05:26:02.778540+0000","last_fullsized":"2023-10-06T05:26:02.778540+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:28.139414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.776645+0000","last_change":"2023-10-06T05:26:02.774729+0000","last_active":"2023-10-06T05:26:02.776645+0000","last_peered":"2023-10-06T05:26:02.776645+0000","last_clean":"2023-10-06T05:26:02.776645+0000","last_became_active":"2023-10-06T05:26:02.774453+0000","last_became_peered":"2023-10-06T05:26:02.774453+0000","last_unstale":"2023-10-06T05:26:02.776645+0000","last_undegraded":"2023-10-06T05:26:02.776645+0000","last_fullsized":"2023-10-06T05:26:02.776645+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:28.046497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.787381+0000","last_change":"2023-10-06T05:26:02.779202+0000","last_active":"2023-10-06T05:26:02.787381+0000","last_peered":"2023-10-06T05:26:02.787381+0000","last_clean":"2023-10-06T05:26:02.787381+0000","last_became_active":"2023-10-06T05:26:02.778920+0000","last_became_peered":"2023-10-06T05:26:02.778920+0000","last_unstale":"2023-10-06T05:26:02.787381+0000","last_undegraded":"2023-10-06T05:26:02.787381+0000","last_fullsized":"2023-10-06T05:26:02.787381+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:30.340886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.782818+0000","last_change":"2023-10-06T05:26:02.779804+0000","last_active":"2023-10-06T05:26:02.782818+0000","last_peered":"2023-10-06T05:26:02.782818+0000","last_clean":"2023-10-06T05:26:02.782818+0000","last_became_active":"2023-10-06T05:26:02.779493+0000","last_became_peered":"2023-10-06T05:26:02.779493+0000","last_unstale":"2023-10-06T05:26:02.782818+0000","last_undegraded":"2023-10-06T05:26:02.782818+0000","last_fullsized":"2023-10-06T05:26:02.782818+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:28.301293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.785379+0000","last_change":"2023-10-06T05:26:02.781526+0000","last_active":"2023-10-06T05:26:02.785379+0000","last_peered":"2023-10-06T05:26:02.785379+0000","last_clean":"2023-10-06T05:26:02.785379+0000","last_became_active":"2023-10-06T05:26:02.781258+0000","last_became_peered":"2023-10-06T05:26:02.781258+0000","last_unstale":"2023-10-06T05:26:02.785379+0000","last_undegraded":"2023-10-06T05:26:02.785379+0000","last_fullsized":"2023-10-06T05:26:02.785379+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:04.817010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.785944+0000","last_change":"2023-10-06T05:26:02.782024+0000","last_active":"2023-10-06T05:26:02.785944+0000","last_peered":"2023-10-06T05:26:02.785944+0000","last_clean":"2023-10-06T05:26:02.785944+0000","last_became_active":"2023-10-06T05:26:02.781751+0000","last_became_peered":"2023-10-06T05:26:02.781751+0000","last_unstale":"2023-10-06T05:26:02.785944+0000","last_undegraded":"2023-10-06T05:26:02.785944+0000","last_fullsized":"2023-10-06T05:26:02.785944+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:21.217371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.775578+0000","last_change":"2023-10-06T05:26:02.773418+0000","last_active":"2023-10-06T05:26:02.775578+0000","last_peered":"2023-10-06T05:26:02.775578+0000","last_clean":"2023-10-06T05:26:02.775578+0000","last_became_active":"2023-10-06T05:26:02.773139+0000","last_became_peered":"2023-10-06T05:26:02.773139+0000","last_unstale":"2023-10-06T05:26:02.775578+0000","last_undegraded":"2023-10-06T05:26:02.775578+0000","last_fullsized":"2023-10-06T05:26:02.775578+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:19.637094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.782537+0000","last_change":"2023-10-06T05:26:02.777355+0000","last_active":"2023-10-06T05:26:02.782537+0000","last_peered":"2023-10-06T05:26:02.782537+0000","last_clean":"2023-10-06T05:26:02.782537+0000","last_became_active":"2023-10-06T05:26:02.776392+0000","last_became_peered":"2023-10-06T05:26:02.776392+0000","last_unstale":"2023-10-06T05:26:02.782537+0000","last_undegraded":"2023-10-06T05:26:02.782537+0000","last_fullsized":"2023-10-06T05:26:02.782537+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:01.412474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.791741+0000","last_change":"2023-10-06T05:26:02.785188+0000","last_active":"2023-10-06T05:26:02.791741+0000","last_peered":"2023-10-06T05:26:02.791741+0000","last_clean":"2023-10-06T05:26:02.791741+0000","last_became_active":"2023-10-06T05:26:02.784879+0000","last_became_peered":"2023-10-06T05:26:02.784879+0000","last_unstale":"2023-10-06T05:26:02.791741+0000","last_undegraded":"2023-10-06T05:26:02.791741+0000","last_fullsized":"2023-10-06T05:26:02.791741+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:42:05.929860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.787206+0000","last_change":"2023-10-06T05:26:02.784592+0000","last_active":"2023-10-06T05:26:02.787206+0000","last_peered":"2023-10-06T05:26:02.787206+0000","last_clean":"2023-10-06T05:26:02.787206+0000","last_became_active":"2023-10-06T05:26:02.784324+0000","last_became_peered":"2023-10-06T05:26:02.784324+0000","last_unstale":"2023-10-06T05:26:02.787206+0000","last_undegraded":"2023-10-06T05:26:02.787206+0000","last_fullsized":"2023-10-06T05:26:02.787206+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:56.000052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.767304+0000","last_change":"2023-10-06T05:26:02.763680+0000","last_active":"2023-10-06T05:26:02.767304+0000","last_peered":"2023-10-06T05:26:02.767304+0000","last_clean":"2023-10-06T05:26:02.767304+0000","last_became_active":"2023-10-06T05:26:02.763412+0000","last_became_peered":"2023-10-06T05:26:02.763412+0000","last_unstale":"2023-10-06T05:26:02.767304+0000","last_undegraded":"2023-10-06T05:26:02.767304+0000","last_fullsized":"2023-10-06T05:26:02.767304+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:11.790081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.786971+0000","last_change":"2023-10-06T05:26:02.786200+0000","last_active":"2023-10-06T05:26:02.786971+0000","last_peered":"2023-10-06T05:26:02.786971+0000","last_clean":"2023-10-06T05:26:02.786971+0000","last_became_active":"2023-10-06T05:26:02.785915+0000","last_became_peered":"2023-10-06T05:26:02.785915+0000","last_unstale":"2023-10-06T05:26:02.786971+0000","last_undegraded":"2023-10-06T05:26:02.786971+0000","last_fullsized":"2023-10-06T05:26:02.786971+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:47.514173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.774020+0000","last_change":"2023-10-06T05:26:02.771290+0000","last_active":"2023-10-06T05:26:02.774020+0000","last_peered":"2023-10-06T05:26:02.774020+0000","last_clean":"2023-10-06T05:26:02.774020+0000","last_became_active":"2023-10-06T05:26:02.770990+0000","last_became_peered":"2023-10-06T05:26:02.770990+0000","last_unstale":"2023-10-06T05:26:02.774020+0000","last_undegraded":"2023-10-06T05:26:02.774020+0000","last_fullsized":"2023-10-06T05:26:02.774020+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:48.242816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.785344+0000","last_change":"2023-10-06T05:26:02.780371+0000","last_active":"2023-10-06T05:26:02.785344+0000","last_peered":"2023-10-06T05:26:02.785344+0000","last_clean":"2023-10-06T05:26:02.785344+0000","last_became_active":"2023-10-06T05:26:02.779972+0000","last_became_peered":"2023-10-06T05:26:02.779972+0000","last_unstale":"2023-10-06T05:26:02.785344+0000","last_undegraded":"2023-10-06T05:26:02.785344+0000","last_fullsized":"2023-10-06T05:26:02.785344+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:33.690103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.791662+0000","last_change":"2023-10-06T05:26:02.784236+0000","last_active":"2023-10-06T05:26:02.791662+0000","last_peered":"2023-10-06T05:26:02.791662+0000","last_clean":"2023-10-06T05:26:02.791662+0000","last_became_active":"2023-10-06T05:26:02.783896+0000","last_became_peered":"2023-10-06T05:26:02.783896+0000","last_unstale":"2023-10-06T05:26:02.791662+0000","last_undegraded":"2023-10-06T05:26:02.791662+0000","last_fullsized":"2023-10-06T05:26:02.791662+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:59.515376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.791679+0000","last_change":"2023-10-06T05:26:02.784032+0000","last_active":"2023-10-06T05:26:02.791679+0000","last_peered":"2023-10-06T05:26:02.791679+0000","last_clean":"2023-10-06T05:26:02.791679+0000","last_became_active":"2023-10-06T05:26:02.783488+0000","last_became_peered":"2023-10-06T05:26:02.783488+0000","last_unstale":"2023-10-06T05:26:02.791679+0000","last_undegraded":"2023-10-06T05:26:02.791679+0000","last_fullsized":"2023-10-06T05:26:02.791679+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:41.983609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.791771+0000","last_change":"2023-10-06T05:26:02.785627+0000","last_active":"2023-10-06T05:26:02.791771+0000","last_peered":"2023-10-06T05:26:02.791771+0000","last_clean":"2023-10-06T05:26:02.791771+0000","last_became_active":"2023-10-06T05:26:02.785217+0000","last_became_peered":"2023-10-06T05:26:02.785217+0000","last_unstale":"2023-10-06T05:26:02.791771+0000","last_undegraded":"2023-10-06T05:26:02.791771+0000","last_fullsized":"2023-10-06T05:26:02.791771+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:25.633327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.785797+0000","last_change":"2023-10-06T05:26:02.781600+0000","last_active":"2023-10-06T05:26:02.785797+0000","last_peered":"2023-10-06T05:26:02.785797+0000","last_clean":"2023-10-06T05:26:02.785797+0000","last_became_active":"2023-10-06T05:26:02.781318+0000","last_became_peered":"2023-10-06T05:26:02.781318+0000","last_unstale":"2023-10-06T05:26:02.785797+0000","last_undegraded":"2023-10-06T05:26:02.785797+0000","last_fullsized":"2023-10-06T05:26:02.785797+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:29.642060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.783387+0000","last_change":"2023-10-06T05:26:02.779759+0000","last_active":"2023-10-06T05:26:02.783387+0000","last_peered":"2023-10-06T05:26:02.783387+0000","last_clean":"2023-10-06T05:26:02.783387+0000","last_became_active":"2023-10-06T05:26:02.779476+0000","last_became_peered":"2023-10-06T05:26:02.779476+0000","last_unstale":"2023-10-06T05:26:02.783387+0000","last_undegraded":"2023-10-06T05:26:02.783387+0000","last_fullsized":"2023-10-06T05:26:02.783387+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:04.935953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.774209+0000","last_change":"2023-10-06T05:26:02.772476+0000","last_active":"2023-10-06T05:26:02.774209+0000","last_peered":"2023-10-06T05:26:02.774209+0000","last_clean":"2023-10-06T05:26:02.774209+0000","last_became_active":"2023-10-06T05:26:02.772116+0000","last_became_peered":"2023-10-06T05:26:02.772116+0000","last_unstale":"2023-10-06T05:26:02.774209+0000","last_undegraded":"2023-10-06T05:26:02.774209+0000","last_fullsized":"2023-10-06T05:26:02.774209+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:02.754179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.787389+0000","last_change":"2023-10-06T05:26:02.781735+0000","last_active":"2023-10-06T05:26:02.787389+0000","last_peered":"2023-10-06T05:26:02.787389+0000","last_clean":"2023-10-06T05:26:02.787389+0000","last_became_active":"2023-10-06T05:26:02.781446+0000","last_became_peered":"2023-10-06T05:26:02.781446+0000","last_unstale":"2023-10-06T05:26:02.787389+0000","last_undegraded":"2023-10-06T05:26:02.787389+0000","last_fullsized":"2023-10-06T05:26:02.787389+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:46.655381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.785905+0000","last_change":"2023-10-06T05:26:02.781179+0000","last_active":"2023-10-06T05:26:02.785905+0000","last_peered":"2023-10-06T05:26:02.785905+0000","last_clean":"2023-10-06T05:26:02.785905+0000","last_became_active":"2023-10-06T05:26:02.780902+0000","last_became_peered":"2023-10-06T05:26:02.780902+0000","last_unstale":"2023-10-06T05:26:02.785905+0000","last_undegraded":"2023-10-06T05:26:02.785905+0000","last_fullsized":"2023-10-06T05:26:02.785905+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:57.122833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.772939+0000","last_change":"2023-10-06T05:26:02.771447+0000","last_active":"2023-10-06T05:26:02.772939+0000","last_peered":"2023-10-06T05:26:02.772939+0000","last_clean":"2023-10-06T05:26:02.772939+0000","last_became_active":"2023-10-06T05:26:02.771162+0000","last_became_peered":"2023-10-06T05:26:02.771162+0000","last_unstale":"2023-10-06T05:26:02.772939+0000","last_undegraded":"2023-10-06T05:26:02.772939+0000","last_fullsized":"2023-10-06T05:26:02.772939+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:41.983609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.778485+0000","last_change":"2023-10-06T05:26:02.776574+0000","last_active":"2023-10-06T05:26:02.778485+0000","last_peered":"2023-10-06T05:26:02.778485+0000","last_clean":"2023-10-06T05:26:02.778485+0000","last_became_active":"2023-10-06T05:26:02.776316+0000","last_became_peered":"2023-10-06T05:26:02.776316+0000","last_unstale":"2023-10-06T05:26:02.778485+0000","last_undegraded":"2023-10-06T05:26:02.778485+0000","last_fullsized":"2023-10-06T05:26:02.778485+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:47.816392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.769036+0000","last_change":"2023-10-06T05:26:02.763932+0000","last_active":"2023-10-06T05:26:02.769036+0000","last_peered":"2023-10-06T05:26:02.769036+0000","last_clean":"2023-10-06T05:26:02.769036+0000","last_became_active":"2023-10-06T05:26:02.763658+0000","last_became_peered":"2023-10-06T05:26:02.763658+0000","last_unstale":"2023-10-06T05:26:02.769036+0000","last_undegraded":"2023-10-06T05:26:02.769036+0000","last_fullsized":"2023-10-06T05:26:02.769036+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:55.664301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.783122+0000","last_change":"2023-10-06T05:26:02.778551+0000","last_active":"2023-10-06T05:26:02.783122+0000","last_peered":"2023-10-06T05:26:02.783122+0000","last_clean":"2023-10-06T05:26:02.783122+0000","last_became_active":"2023-10-06T05:26:02.778272+0000","last_became_peered":"2023-10-06T05:26:02.778272+0000","last_unstale":"2023-10-06T05:26:02.783122+0000","last_undegraded":"2023-10-06T05:26:02.783122+0000","last_fullsized":"2023-10-06T05:26:02.783122+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:12.472111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.782469+0000","last_change":"2023-10-06T05:26:02.777634+0000","last_active":"2023-10-06T05:26:02.782469+0000","last_peered":"2023-10-06T05:26:02.782469+0000","last_clean":"2023-10-06T05:26:02.782469+0000","last_became_active":"2023-10-06T05:26:02.777345+0000","last_became_peered":"2023-10-06T05:26:02.777345+0000","last_unstale":"2023-10-06T05:26:02.782469+0000","last_undegraded":"2023-10-06T05:26:02.782469+0000","last_fullsized":"2023-10-06T05:26:02.782469+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:19.587360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.776079+0000","last_change":"2023-10-06T05:26:02.770979+0000","last_active":"2023-10-06T05:26:02.776079+0000","last_peered":"2023-10-06T05:26:02.776079+0000","last_clean":"2023-10-06T05:26:02.776079+0000","last_became_active":"2023-10-06T05:26:02.770101+0000","last_became_peered":"2023-10-06T05:26:02.770101+0000","last_unstale":"2023-10-06T05:26:02.776079+0000","last_undegraded":"2023-10-06T05:26:02.776079+0000","last_fullsized":"2023-10-06T05:26:02.776079+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:40.463056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.764373+0000","last_change":"2023-10-06T05:26:01.778065+0000","last_active":"2023-10-06T05:26:02.764373+0000","last_peered":"2023-10-06T05:26:02.764373+0000","last_clean":"2023-10-06T05:26:02.764373+0000","last_became_active":"2023-10-06T05:26:01.777756+0000","last_became_peered":"2023-10-06T05:26:01.777756+0000","last_unstale":"2023-10-06T05:26:02.764373+0000","last_undegraded":"2023-10-06T05:26:02.764373+0000","last_fullsized":"2023-10-06T05:26:02.764373+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:12.017407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.774462+0000","last_change":"2023-10-06T05:26:02.772316+0000","last_active":"2023-10-06T05:26:02.774462+0000","last_peered":"2023-10-06T05:26:02.774462+0000","last_clean":"2023-10-06T05:26:02.774462+0000","last_became_active":"2023-10-06T05:26:02.772071+0000","last_became_peered":"2023-10-06T05:26:02.772071+0000","last_unstale":"2023-10-06T05:26:02.774462+0000","last_undegraded":"2023-10-06T05:26:02.774462+0000","last_fullsized":"2023-10-06T05:26:02.774462+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:15.064865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.767303+0000","last_change":"2023-10-06T05:26:01.782852+0000","last_active":"2023-10-06T05:26:02.767303+0000","last_peered":"2023-10-06T05:26:02.767303+0000","last_clean":"2023-10-06T05:26:02.767303+0000","last_became_active":"2023-10-06T05:26:01.782576+0000","last_became_peered":"2023-10-06T05:26:01.782576+0000","last_unstale":"2023-10-06T05:26:02.767303+0000","last_undegraded":"2023-10-06T05:26:02.767303+0000","last_fullsized":"2023-10-06T05:26:02.767303+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:48.374182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.782834+0000","last_change":"2023-10-06T05:26:02.779453+0000","last_active":"2023-10-06T05:26:02.782834+0000","last_peered":"2023-10-06T05:26:02.782834+0000","last_clean":"2023-10-06T05:26:02.782834+0000","last_became_active":"2023-10-06T05:26:02.779200+0000","last_became_peered":"2023-10-06T05:26:02.779200+0000","last_unstale":"2023-10-06T05:26:02.782834+0000","last_undegraded":"2023-10-06T05:26:02.782834+0000","last_fullsized":"2023-10-06T05:26:02.782834+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:25.212451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"162'3","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.151488+0000","last_change":"2023-10-06T05:26:01.773473+0000","last_active":"2023-10-06T05:26:03.151488+0000","last_peered":"2023-10-06T05:26:03.151488+0000","last_clean":"2023-10-06T05:26:03.151488+0000","last_became_active":"2023-10-06T05:26:01.773196+0000","last_became_peered":"2023-10-06T05:26:01.773196+0000","last_unstale":"2023-10-06T05:26:03.151488+0000","last_undegraded":"2023-10-06T05:26:03.151488+0000","last_fullsized":"2023-10-06T05:26:03.151488+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:27.132454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.783350+0000","last_change":"2023-10-06T05:26:02.778678+0000","last_active":"2023-10-06T05:26:02.783350+0000","last_peered":"2023-10-06T05:26:02.783350+0000","last_clean":"2023-10-06T05:26:02.783350+0000","last_became_active":"2023-10-06T05:26:02.778362+0000","last_became_peered":"2023-10-06T05:26:02.778362+0000","last_unstale":"2023-10-06T05:26:02.783350+0000","last_undegraded":"2023-10-06T05:26:02.783350+0000","last_fullsized":"2023-10-06T05:26:02.783350+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:25.633327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"162'6","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.154988+0000","last_change":"2023-10-06T05:26:01.781978+0000","last_active":"2023-10-06T05:26:03.154988+0000","last_peered":"2023-10-06T05:26:03.154988+0000","last_clean":"2023-10-06T05:26:03.154988+0000","last_became_active":"2023-10-06T05:26:01.781719+0000","last_became_peered":"2023-10-06T05:26:01.781719+0000","last_unstale":"2023-10-06T05:26:03.154988+0000","last_undegraded":"2023-10-06T05:26:03.154988+0000","last_fullsized":"2023-10-06T05:26:03.154988+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:48.222794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.780635+0000","last_change":"2023-10-06T05:26:02.773984+0000","last_active":"2023-10-06T05:26:02.780635+0000","last_peered":"2023-10-06T05:26:02.780635+0000","last_clean":"2023-10-06T05:26:02.780635+0000","last_became_active":"2023-10-06T05:26:02.773721+0000","last_became_peered":"2023-10-06T05:26:02.773721+0000","last_unstale":"2023-10-06T05:26:02.780635+0000","last_undegraded":"2023-10-06T05:26:02.780635+0000","last_fullsized":"2023-10-06T05:26:02.780635+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:35.450119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.148354+0000","last_change":"2023-10-06T05:26:01.771916+0000","last_active":"2023-10-06T05:26:03.148354+0000","last_peered":"2023-10-06T05:26:03.148354+0000","last_clean":"2023-10-06T05:26:03.148354+0000","last_became_active":"2023-10-06T05:26:01.771650+0000","last_became_peered":"2023-10-06T05:26:01.771650+0000","last_unstale":"2023-10-06T05:26:03.148354+0000","last_undegraded":"2023-10-06T05:26:03.148354+0000","last_fullsized":"2023-10-06T05:26:03.148354+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:56.030300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.774388+0000","last_change":"2023-10-06T05:26:02.770807+0000","last_active":"2023-10-06T05:26:02.774388+0000","last_peered":"2023-10-06T05:26:02.774388+0000","last_clean":"2023-10-06T05:26:02.774388+0000","last_became_active":"2023-10-06T05:26:02.770564+0000","last_became_peered":"2023-10-06T05:26:02.770564+0000","last_unstale":"2023-10-06T05:26:02.774388+0000","last_undegraded":"2023-10-06T05:26:02.774388+0000","last_fullsized":"2023-10-06T05:26:02.774388+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:57.372182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.759337+0000","last_change":"2023-10-06T05:26:01.767573+0000","last_active":"2023-10-06T05:26:02.759337+0000","last_peered":"2023-10-06T05:26:02.759337+0000","last_clean":"2023-10-06T05:26:02.759337+0000","last_became_active":"2023-10-06T05:26:01.767282+0000","last_became_peered":"2023-10-06T05:26:01.767282+0000","last_unstale":"2023-10-06T05:26:02.759337+0000","last_undegraded":"2023-10-06T05:26:02.759337+0000","last_fullsized":"2023-10-06T05:26:02.759337+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:54.993091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.778439+0000","last_change":"2023-10-06T05:26:02.775836+0000","last_active":"2023-10-06T05:26:02.778439+0000","last_peered":"2023-10-06T05:26:02.778439+0000","last_clean":"2023-10-06T05:26:02.778439+0000","last_became_active":"2023-10-06T05:26:02.775520+0000","last_became_peered":"2023-10-06T05:26:02.775520+0000","last_unstale":"2023-10-06T05:26:02.778439+0000","last_undegraded":"2023-10-06T05:26:02.778439+0000","last_fullsized":"2023-10-06T05:26:02.778439+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:04.543249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.768215+0000","last_change":"2023-10-06T05:26:01.774286+0000","last_active":"2023-10-06T05:26:02.768215+0000","last_peered":"2023-10-06T05:26:02.768215+0000","last_clean":"2023-10-06T05:26:02.768215+0000","last_became_active":"2023-10-06T05:26:01.774036+0000","last_became_peered":"2023-10-06T05:26:01.774036+0000","last_unstale":"2023-10-06T05:26:02.768215+0000","last_undegraded":"2023-10-06T05:26:02.768215+0000","last_fullsized":"2023-10-06T05:26:02.768215+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:14.173535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.788925+0000","last_change":"2023-10-06T05:26:02.783640+0000","last_active":"2023-10-06T05:26:02.788925+0000","last_peered":"2023-10-06T05:26:02.788925+0000","last_clean":"2023-10-06T05:26:02.788925+0000","last_became_active":"2023-10-06T05:26:02.783331+0000","last_became_peered":"2023-10-06T05:26:02.783331+0000","last_unstale":"2023-10-06T05:26:02.788925+0000","last_undegraded":"2023-10-06T05:26:02.788925+0000","last_fullsized":"2023-10-06T05:26:02.788925+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:51.369769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"162'8","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.152441+0000","last_change":"2023-10-06T05:26:01.781500+0000","last_active":"2023-10-06T05:26:03.152441+0000","last_peered":"2023-10-06T05:26:03.152441+0000","last_clean":"2023-10-06T05:26:03.152441+0000","last_became_active":"2023-10-06T05:26:01.781182+0000","last_became_peered":"2023-10-06T05:26:01.781182+0000","last_unstale":"2023-10-06T05:26:03.152441+0000","last_undegraded":"2023-10-06T05:26:03.152441+0000","last_fullsized":"2023-10-06T05:26:03.152441+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:58.754876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.787417+0000","last_change":"2023-10-06T05:26:02.781974+0000","last_active":"2023-10-06T05:26:02.787417+0000","last_peered":"2023-10-06T05:26:02.787417+0000","last_clean":"2023-10-06T05:26:02.787417+0000","last_became_active":"2023-10-06T05:26:02.781482+0000","last_became_peered":"2023-10-06T05:26:02.781482+0000","last_unstale":"2023-10-06T05:26:02.787417+0000","last_undegraded":"2023-10-06T05:26:02.787417+0000","last_fullsized":"2023-10-06T05:26:02.787417+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:29.655487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.148680+0000","last_change":"2023-10-06T05:26:01.771442+0000","last_active":"2023-10-06T05:26:03.148680+0000","last_peered":"2023-10-06T05:26:03.148680+0000","last_clean":"2023-10-06T05:26:03.148680+0000","last_became_active":"2023-10-06T05:26:01.771168+0000","last_became_peered":"2023-10-06T05:26:01.771168+0000","last_unstale":"2023-10-06T05:26:03.148680+0000","last_undegraded":"2023-10-06T05:26:03.148680+0000","last_fullsized":"2023-10-06T05:26:03.148680+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:36.121944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.787280+0000","last_change":"2023-10-06T05:26:02.784840+0000","last_active":"2023-10-06T05:26:02.787280+0000","last_peered":"2023-10-06T05:26:02.787280+0000","last_clean":"2023-10-06T05:26:02.787280+0000","last_became_active":"2023-10-06T05:26:02.784554+0000","last_became_peered":"2023-10-06T05:26:02.784554+0000","last_unstale":"2023-10-06T05:26:02.787280+0000","last_undegraded":"2023-10-06T05:26:02.787280+0000","last_fullsized":"2023-10-06T05:26:02.787280+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:57.122833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.148346+0000","last_change":"2023-10-06T05:26:01.780356+0000","last_active":"2023-10-06T05:26:03.148346+0000","last_peered":"2023-10-06T05:26:03.148346+0000","last_clean":"2023-10-06T05:26:03.148346+0000","last_became_active":"2023-10-06T05:26:01.780012+0000","last_became_peered":"2023-10-06T05:26:01.780012+0000","last_unstale":"2023-10-06T05:26:03.148346+0000","last_undegraded":"2023-10-06T05:26:03.148346+0000","last_fullsized":"2023-10-06T05:26:03.148346+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:54.657340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.785228+0000","last_change":"2023-10-06T05:26:02.781543+0000","last_active":"2023-10-06T05:26:02.785228+0000","last_peered":"2023-10-06T05:26:02.785228+0000","last_clean":"2023-10-06T05:26:02.785228+0000","last_became_active":"2023-10-06T05:26:02.781256+0000","last_became_peered":"2023-10-06T05:26:02.781256+0000","last_unstale":"2023-10-06T05:26:02.785228+0000","last_undegraded":"2023-10-06T05:26:02.785228+0000","last_fullsized":"2023-10-06T05:26:02.785228+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:11.790081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.148201+0000","last_change":"2023-10-06T05:26:01.773704+0000","last_active":"2023-10-06T05:26:03.148201+0000","last_peered":"2023-10-06T05:26:03.148201+0000","last_clean":"2023-10-06T05:26:03.148201+0000","last_became_active":"2023-10-06T05:26:01.773448+0000","last_became_peered":"2023-10-06T05:26:01.773448+0000","last_unstale":"2023-10-06T05:26:03.148201+0000","last_undegraded":"2023-10-06T05:26:03.148201+0000","last_fullsized":"2023-10-06T05:26:03.148201+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:03.488364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.791656+0000","last_change":"2023-10-06T05:26:02.784170+0000","last_active":"2023-10-06T05:26:02.791656+0000","last_peered":"2023-10-06T05:26:02.791656+0000","last_clean":"2023-10-06T05:26:02.791656+0000","last_became_active":"2023-10-06T05:26:02.783875+0000","last_became_peered":"2023-10-06T05:26:02.783875+0000","last_unstale":"2023-10-06T05:26:02.791656+0000","last_undegraded":"2023-10-06T05:26:02.791656+0000","last_fullsized":"2023-10-06T05:26:02.791656+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:10.242955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.764764+0000","last_change":"2023-10-06T05:26:01.770977+0000","last_active":"2023-10-06T05:26:02.764764+0000","last_peered":"2023-10-06T05:26:02.764764+0000","last_clean":"2023-10-06T05:26:02.764764+0000","last_became_active":"2023-10-06T05:26:01.770682+0000","last_became_peered":"2023-10-06T05:26:01.770682+0000","last_unstale":"2023-10-06T05:26:02.764764+0000","last_undegraded":"2023-10-06T05:26:02.764764+0000","last_fullsized":"2023-10-06T05:26:02.764764+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:41.224903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.780613+0000","last_change":"2023-10-06T05:26:02.771669+0000","last_active":"2023-10-06T05:26:02.780613+0000","last_peered":"2023-10-06T05:26:02.780613+0000","last_clean":"2023-10-06T05:26:02.780613+0000","last_became_active":"2023-10-06T05:26:02.771397+0000","last_became_peered":"2023-10-06T05:26:02.771397+0000","last_unstale":"2023-10-06T05:26:02.780613+0000","last_undegraded":"2023-10-06T05:26:02.780613+0000","last_fullsized":"2023-10-06T05:26:02.780613+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:25.212451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.147508+0000","last_change":"2023-10-06T05:26:01.769449+0000","last_active":"2023-10-06T05:26:03.147508+0000","last_peered":"2023-10-06T05:26:03.147508+0000","last_clean":"2023-10-06T05:26:03.147508+0000","last_became_active":"2023-10-06T05:26:01.769162+0000","last_became_peered":"2023-10-06T05:26:01.769162+0000","last_unstale":"2023-10-06T05:26:03.147508+0000","last_undegraded":"2023-10-06T05:26:03.147508+0000","last_fullsized":"2023-10-06T05:26:03.147508+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:58.309480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.783243+0000","last_change":"2023-10-06T05:26:02.778566+0000","last_active":"2023-10-06T05:26:02.783243+0000","last_peered":"2023-10-06T05:26:02.783243+0000","last_clean":"2023-10-06T05:26:02.783243+0000","last_became_active":"2023-10-06T05:26:02.778295+0000","last_became_peered":"2023-10-06T05:26:02.778295+0000","last_unstale":"2023-10-06T05:26:02.783243+0000","last_undegraded":"2023-10-06T05:26:02.783243+0000","last_fullsized":"2023-10-06T05:26:02.783243+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:16.617434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.760176+0000","last_change":"2023-10-06T05:26:01.769722+0000","last_active":"2023-10-06T05:26:02.760176+0000","last_peered":"2023-10-06T05:26:02.760176+0000","last_clean":"2023-10-06T05:26:02.760176+0000","last_became_active":"2023-10-06T05:26:01.769389+0000","last_became_peered":"2023-10-06T05:26:01.769389+0000","last_unstale":"2023-10-06T05:26:02.760176+0000","last_undegraded":"2023-10-06T05:26:02.760176+0000","last_fullsized":"2023-10-06T05:26:02.760176+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:48.397918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.774040+0000","last_change":"2023-10-06T05:26:02.771805+0000","last_active":"2023-10-06T05:26:02.774040+0000","last_peered":"2023-10-06T05:26:02.774040+0000","last_clean":"2023-10-06T05:26:02.774040+0000","last_became_active":"2023-10-06T05:26:02.771551+0000","last_became_peered":"2023-10-06T05:26:02.771551+0000","last_unstale":"2023-10-06T05:26:02.774040+0000","last_undegraded":"2023-10-06T05:26:02.774040+0000","last_fullsized":"2023-10-06T05:26:02.774040+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:29.642060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.760549+0000","last_change":"2023-10-06T05:26:01.779420+0000","last_active":"2023-10-06T05:26:02.760549+0000","last_peered":"2023-10-06T05:26:02.760549+0000","last_clean":"2023-10-06T05:26:02.760549+0000","last_became_active":"2023-10-06T05:26:01.779148+0000","last_became_peered":"2023-10-06T05:26:01.779148+0000","last_unstale":"2023-10-06T05:26:02.760549+0000","last_undegraded":"2023-10-06T05:26:02.760549+0000","last_fullsized":"2023-10-06T05:26:02.760549+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:25:08.162681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.791597+0000","last_change":"2023-10-06T05:26:02.783543+0000","last_active":"2023-10-06T05:26:02.791597+0000","last_peered":"2023-10-06T05:26:02.791597+0000","last_clean":"2023-10-06T05:26:02.791597+0000","last_became_active":"2023-10-06T05:26:02.783262+0000","last_became_peered":"2023-10-06T05:26:02.783262+0000","last_unstale":"2023-10-06T05:26:02.791597+0000","last_undegraded":"2023-10-06T05:26:02.791597+0000","last_fullsized":"2023-10-06T05:26:02.791597+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:46.192499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.767043+0000","last_change":"2023-10-06T05:26:01.769920+0000","last_active":"2023-10-06T05:26:02.767043+0000","last_peered":"2023-10-06T05:26:02.767043+0000","last_clean":"2023-10-06T05:26:02.767043+0000","last_became_active":"2023-10-06T05:26:01.769512+0000","last_became_peered":"2023-10-06T05:26:01.769512+0000","last_unstale":"2023-10-06T05:26:02.767043+0000","last_undegraded":"2023-10-06T05:26:02.767043+0000","last_fullsized":"2023-10-06T05:26:02.767043+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:11.047523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.787400+0000","last_change":"2023-10-06T05:26:02.781763+0000","last_active":"2023-10-06T05:26:02.787400+0000","last_peered":"2023-10-06T05:26:02.787400+0000","last_clean":"2023-10-06T05:26:02.787400+0000","last_became_active":"2023-10-06T05:26:02.781118+0000","last_became_peered":"2023-10-06T05:26:02.781118+0000","last_unstale":"2023-10-06T05:26:02.787400+0000","last_undegraded":"2023-10-06T05:26:02.787400+0000","last_fullsized":"2023-10-06T05:26:02.787400+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:12.116883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.149753+0000","last_change":"2023-10-06T05:26:01.781831+0000","last_active":"2023-10-06T05:26:03.149753+0000","last_peered":"2023-10-06T05:26:03.149753+0000","last_clean":"2023-10-06T05:26:03.149753+0000","last_became_active":"2023-10-06T05:26:01.781494+0000","last_became_peered":"2023-10-06T05:26:01.781494+0000","last_unstale":"2023-10-06T05:26:03.149753+0000","last_undegraded":"2023-10-06T05:26:03.149753+0000","last_fullsized":"2023-10-06T05:26:03.149753+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:15.610474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.786473+0000","last_change":"2023-10-06T05:26:02.784500+0000","last_active":"2023-10-06T05:26:02.786473+0000","last_peered":"2023-10-06T05:26:02.786473+0000","last_clean":"2023-10-06T05:26:02.786473+0000","last_became_active":"2023-10-06T05:26:02.784260+0000","last_became_peered":"2023-10-06T05:26:02.784260+0000","last_unstale":"2023-10-06T05:26:02.786473+0000","last_undegraded":"2023-10-06T05:26:02.786473+0000","last_fullsized":"2023-10-06T05:26:02.786473+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:30.439492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.769025+0000","last_change":"2023-10-06T05:26:01.778961+0000","last_active":"2023-10-06T05:26:02.769025+0000","last_peered":"2023-10-06T05:26:02.769025+0000","last_clean":"2023-10-06T05:26:02.769025+0000","last_became_active":"2023-10-06T05:26:01.778660+0000","last_became_peered":"2023-10-06T05:26:01.778660+0000","last_unstale":"2023-10-06T05:26:02.769025+0000","last_undegraded":"2023-10-06T05:26:02.769025+0000","last_fullsized":"2023-10-06T05:26:02.769025+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:00:57.906765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.778630+0000","last_change":"2023-10-06T05:26:02.775865+0000","last_active":"2023-10-06T05:26:02.778630+0000","last_peered":"2023-10-06T05:26:02.778630+0000","last_clean":"2023-10-06T05:26:02.778630+0000","last_became_active":"2023-10-06T05:26:02.775579+0000","last_became_peered":"2023-10-06T05:26:02.775579+0000","last_unstale":"2023-10-06T05:26:02.778630+0000","last_undegraded":"2023-10-06T05:26:02.778630+0000","last_fullsized":"2023-10-06T05:26:02.778630+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:37:35.450119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.760495+0000","last_change":"2023-10-06T05:26:01.777364+0000","last_active":"2023-10-06T05:26:02.760495+0000","last_peered":"2023-10-06T05:26:02.760495+0000","last_clean":"2023-10-06T05:26:02.760495+0000","last_became_active":"2023-10-06T05:26:01.776712+0000","last_became_peered":"2023-10-06T05:26:01.776712+0000","last_unstale":"2023-10-06T05:26:02.760495+0000","last_undegraded":"2023-10-06T05:26:02.760495+0000","last_fullsized":"2023-10-06T05:26:02.760495+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:12.621638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.781976+0000","last_change":"2023-10-06T05:26:02.775180+0000","last_active":"2023-10-06T05:26:02.781976+0000","last_peered":"2023-10-06T05:26:02.781976+0000","last_clean":"2023-10-06T05:26:02.781976+0000","last_became_active":"2023-10-06T05:26:02.774885+0000","last_became_peered":"2023-10-06T05:26:02.774885+0000","last_unstale":"2023-10-06T05:26:02.781976+0000","last_undegraded":"2023-10-06T05:26:02.781976+0000","last_fullsized":"2023-10-06T05:26:02.781976+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:30.439492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.765934+0000","last_change":"2023-10-06T05:26:01.775470+0000","last_active":"2023-10-06T05:26:02.765934+0000","last_peered":"2023-10-06T05:26:02.765934+0000","last_clean":"2023-10-06T05:26:02.765934+0000","last_became_active":"2023-10-06T05:26:01.775184+0000","last_became_peered":"2023-10-06T05:26:01.775184+0000","last_unstale":"2023-10-06T05:26:02.765934+0000","last_undegraded":"2023-10-06T05:26:02.765934+0000","last_fullsized":"2023-10-06T05:26:02.765934+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:39.456096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.785306+0000","last_change":"2023-10-06T05:26:02.778624+0000","last_active":"2023-10-06T05:26:02.785306+0000","last_peered":"2023-10-06T05:26:02.785306+0000","last_clean":"2023-10-06T05:26:02.785306+0000","last_became_active":"2023-10-06T05:26:02.778330+0000","last_became_peered":"2023-10-06T05:26:02.778330+0000","last_unstale":"2023-10-06T05:26:02.785306+0000","last_undegraded":"2023-10-06T05:26:02.785306+0000","last_fullsized":"2023-10-06T05:26:02.785306+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:28.046497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.767138+0000","last_change":"2023-10-06T05:26:01.778705+0000","last_active":"2023-10-06T05:26:02.767138+0000","last_peered":"2023-10-06T05:26:02.767138+0000","last_clean":"2023-10-06T05:26:02.767138+0000","last_became_active":"2023-10-06T05:26:01.778345+0000","last_became_peered":"2023-10-06T05:26:01.778345+0000","last_unstale":"2023-10-06T05:26:02.767138+0000","last_undegraded":"2023-10-06T05:26:02.767138+0000","last_fullsized":"2023-10-06T05:26:02.767138+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:58.309480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.779610+0000","last_change":"2023-10-06T05:26:02.774702+0000","last_active":"2023-10-06T05:26:02.779610+0000","last_peered":"2023-10-06T05:26:02.779610+0000","last_clean":"2023-10-06T05:26:02.779610+0000","last_became_active":"2023-10-06T05:26:02.774262+0000","last_became_peered":"2023-10-06T05:26:02.774262+0000","last_unstale":"2023-10-06T05:26:02.779610+0000","last_undegraded":"2023-10-06T05:26:02.779610+0000","last_fullsized":"2023-10-06T05:26:02.779610+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:29.934883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.767490+0000","last_change":"2023-10-06T05:26:01.773684+0000","last_active":"2023-10-06T05:26:02.767490+0000","last_peered":"2023-10-06T05:26:02.767490+0000","last_clean":"2023-10-06T05:26:02.767490+0000","last_became_active":"2023-10-06T05:26:01.773135+0000","last_became_peered":"2023-10-06T05:26:01.773135+0000","last_unstale":"2023-10-06T05:26:02.767490+0000","last_undegraded":"2023-10-06T05:26:02.767490+0000","last_fullsized":"2023-10-06T05:26:02.767490+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:11.465150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.782664+0000","last_change":"2023-10-06T05:26:02.776249+0000","last_active":"2023-10-06T05:26:02.782664+0000","last_peered":"2023-10-06T05:26:02.782664+0000","last_clean":"2023-10-06T05:26:02.782664+0000","last_became_active":"2023-10-06T05:26:02.775953+0000","last_became_peered":"2023-10-06T05:26:02.775953+0000","last_unstale":"2023-10-06T05:26:02.782664+0000","last_undegraded":"2023-10-06T05:26:02.782664+0000","last_fullsized":"2023-10-06T05:26:02.782664+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:24.211943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.767970+0000","last_change":"2023-10-06T05:26:01.780163+0000","last_active":"2023-10-06T05:26:02.767970+0000","last_peered":"2023-10-06T05:26:02.767970+0000","last_clean":"2023-10-06T05:26:02.767970+0000","last_became_active":"2023-10-06T05:26:01.779863+0000","last_became_peered":"2023-10-06T05:26:01.779863+0000","last_unstale":"2023-10-06T05:26:02.767970+0000","last_undegraded":"2023-10-06T05:26:02.767970+0000","last_fullsized":"2023-10-06T05:26:02.767970+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:08:13.612197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.786446+0000","last_change":"2023-10-06T05:26:02.784496+0000","last_active":"2023-10-06T05:26:02.786446+0000","last_peered":"2023-10-06T05:26:02.786446+0000","last_clean":"2023-10-06T05:26:02.786446+0000","last_became_active":"2023-10-06T05:26:02.784254+0000","last_became_peered":"2023-10-06T05:26:02.784254+0000","last_unstale":"2023-10-06T05:26:02.786446+0000","last_undegraded":"2023-10-06T05:26:02.786446+0000","last_fullsized":"2023-10-06T05:26:02.786446+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:09.253405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.764733+0000","last_change":"2023-10-06T05:26:01.778150+0000","last_active":"2023-10-06T05:26:02.764733+0000","last_peered":"2023-10-06T05:26:02.764733+0000","last_clean":"2023-10-06T05:26:02.764733+0000","last_became_active":"2023-10-06T05:26:01.777828+0000","last_became_peered":"2023-10-06T05:26:01.777828+0000","last_unstale":"2023-10-06T05:26:02.764733+0000","last_undegraded":"2023-10-06T05:26:02.764733+0000","last_fullsized":"2023-10-06T05:26:02.764733+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:52.806800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.785408+0000","last_change":"2023-10-06T05:26:02.778093+0000","last_active":"2023-10-06T05:26:02.785408+0000","last_peered":"2023-10-06T05:26:02.785408+0000","last_clean":"2023-10-06T05:26:02.785408+0000","last_became_active":"2023-10-06T05:26:02.777811+0000","last_became_peered":"2023-10-06T05:26:02.777811+0000","last_unstale":"2023-10-06T05:26:02.785408+0000","last_undegraded":"2023-10-06T05:26:02.785408+0000","last_fullsized":"2023-10-06T05:26:02.785408+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:19.637094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.151650+0000","last_change":"2023-10-06T05:26:01.766285+0000","last_active":"2023-10-06T05:26:03.151650+0000","last_peered":"2023-10-06T05:26:03.151650+0000","last_clean":"2023-10-06T05:26:03.151650+0000","last_became_active":"2023-10-06T05:26:01.766023+0000","last_became_peered":"2023-10-06T05:26:01.766023+0000","last_unstale":"2023-10-06T05:26:03.151650+0000","last_undegraded":"2023-10-06T05:26:03.151650+0000","last_fullsized":"2023-10-06T05:26:03.151650+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:17.090014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":126,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.774000+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:26:02.774000+0000","last_peered":"2023-10-06T05:26:02.774000+0000","last_clean":"2023-10-06T05:26:02.774000+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:26:02.774000+0000","last_undegraded":"2023-10-06T05:26:02.774000+0000","last_fullsized":"2023-10-06T05:26:02.774000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.776747+0000","last_change":"2023-10-06T05:26:02.775044+0000","last_active":"2023-10-06T05:26:02.776747+0000","last_peered":"2023-10-06T05:26:02.776747+0000","last_clean":"2023-10-06T05:26:02.776747+0000","last_became_active":"2023-10-06T05:26:02.774386+0000","last_became_peered":"2023-10-06T05:26:02.774386+0000","last_unstale":"2023-10-06T05:26:02.776747+0000","last_undegraded":"2023-10-06T05:26:02.776747+0000","last_fullsized":"2023-10-06T05:26:02.776747+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:39.829280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.148605+0000","last_change":"2023-10-06T05:26:01.772354+0000","last_active":"2023-10-06T05:26:03.148605+0000","last_peered":"2023-10-06T05:26:03.148605+0000","last_clean":"2023-10-06T05:26:03.148605+0000","last_became_active":"2023-10-06T05:26:01.772101+0000","last_became_peered":"2023-10-06T05:26:01.772101+0000","last_unstale":"2023-10-06T05:26:03.148605+0000","last_undegraded":"2023-10-06T05:26:03.148605+0000","last_fullsized":"2023-10-06T05:26:03.148605+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:13.404213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":126,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.769145+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:26:02.769145+0000","last_peered":"2023-10-06T05:26:02.769145+0000","last_clean":"2023-10-06T05:26:02.769145+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:26:02.769145+0000","last_undegraded":"2023-10-06T05:26:02.769145+0000","last_fullsized":"2023-10-06T05:26:02.769145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.768960+0000","last_change":"2023-10-06T05:26:02.763927+0000","last_active":"2023-10-06T05:26:02.768960+0000","last_peered":"2023-10-06T05:26:02.768960+0000","last_clean":"2023-10-06T05:26:02.768960+0000","last_became_active":"2023-10-06T05:26:02.763599+0000","last_became_peered":"2023-10-06T05:26:02.763599+0000","last_unstale":"2023-10-06T05:26:02.768960+0000","last_undegraded":"2023-10-06T05:26:02.768960+0000","last_fullsized":"2023-10-06T05:26:02.768960+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:03.876000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.147057+0000","last_change":"2023-10-06T05:26:01.780325+0000","last_active":"2023-10-06T05:26:03.147057+0000","last_peered":"2023-10-06T05:26:03.147057+0000","last_clean":"2023-10-06T05:26:03.147057+0000","last_became_active":"2023-10-06T05:26:01.780034+0000","last_became_peered":"2023-10-06T05:26:01.780034+0000","last_unstale":"2023-10-06T05:26:03.147057+0000","last_undegraded":"2023-10-06T05:26:03.147057+0000","last_fullsized":"2023-10-06T05:26:03.147057+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:53:52.794343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":124,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.768082+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:26:02.768082+0000","last_peered":"2023-10-06T05:26:02.768082+0000","last_clean":"2023-10-06T05:26:02.768082+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:26:02.768082+0000","last_undegraded":"2023-10-06T05:26:02.768082+0000","last_fullsized":"2023-10-06T05:26:02.768082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.789417+0000","last_change":"2023-10-06T05:26:02.787907+0000","last_active":"2023-10-06T05:26:02.789417+0000","last_peered":"2023-10-06T05:26:02.789417+0000","last_clean":"2023-10-06T05:26:02.789417+0000","last_became_active":"2023-10-06T05:26:02.787637+0000","last_became_peered":"2023-10-06T05:26:02.787637+0000","last_unstale":"2023-10-06T05:26:02.789417+0000","last_undegraded":"2023-10-06T05:26:02.789417+0000","last_fullsized":"2023-10-06T05:26:02.789417+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:21.217371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.147805+0000","last_change":"2023-10-06T05:26:01.774256+0000","last_active":"2023-10-06T05:26:03.147805+0000","last_peered":"2023-10-06T05:26:03.147805+0000","last_clean":"2023-10-06T05:26:03.147805+0000","last_became_active":"2023-10-06T05:26:01.773436+0000","last_became_peered":"2023-10-06T05:26:01.773436+0000","last_unstale":"2023-10-06T05:26:03.147805+0000","last_undegraded":"2023-10-06T05:26:03.147805+0000","last_fullsized":"2023-10-06T05:26:03.147805+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:55:38.635156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":125,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.765746+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:26:02.765746+0000","last_peered":"2023-10-06T05:26:02.765746+0000","last_clean":"2023-10-06T05:26:02.765746+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:26:02.765746+0000","last_undegraded":"2023-10-06T05:26:02.765746+0000","last_fullsized":"2023-10-06T05:26:02.765746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.780359+0000","last_change":"2023-10-06T05:26:02.777192+0000","last_active":"2023-10-06T05:26:02.780359+0000","last_peered":"2023-10-06T05:26:02.780359+0000","last_clean":"2023-10-06T05:26:02.780359+0000","last_became_active":"2023-10-06T05:26:02.776880+0000","last_became_peered":"2023-10-06T05:26:02.776880+0000","last_unstale":"2023-10-06T05:26:02.780359+0000","last_undegraded":"2023-10-06T05:26:02.780359+0000","last_fullsized":"2023-10-06T05:26:02.780359+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:00.760477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.759246+0000","last_change":"2023-10-06T05:26:01.777958+0000","last_active":"2023-10-06T05:26:02.759246+0000","last_peered":"2023-10-06T05:26:02.759246+0000","last_clean":"2023-10-06T05:26:02.759246+0000","last_became_active":"2023-10-06T05:26:01.777647+0000","last_became_peered":"2023-10-06T05:26:01.777647+0000","last_unstale":"2023-10-06T05:26:02.759246+0000","last_undegraded":"2023-10-06T05:26:02.759246+0000","last_fullsized":"2023-10-06T05:26:02.759246+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:36.121944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":125,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.766219+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:26:02.766219+0000","last_peered":"2023-10-06T05:26:02.766219+0000","last_clean":"2023-10-06T05:26:02.766219+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:26:02.766219+0000","last_undegraded":"2023-10-06T05:26:02.766219+0000","last_fullsized":"2023-10-06T05:26:02.766219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.776099+0000","last_change":"2023-10-06T05:26:02.772261+0000","last_active":"2023-10-06T05:26:02.776099+0000","last_peered":"2023-10-06T05:26:02.776099+0000","last_clean":"2023-10-06T05:26:02.776099+0000","last_became_active":"2023-10-06T05:26:02.771976+0000","last_became_peered":"2023-10-06T05:26:02.771976+0000","last_unstale":"2023-10-06T05:26:02.776099+0000","last_undegraded":"2023-10-06T05:26:02.776099+0000","last_fullsized":"2023-10-06T05:26:02.776099+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:12.054484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.765691+0000","last_change":"2023-10-06T05:26:01.783620+0000","last_active":"2023-10-06T05:26:02.765691+0000","last_peered":"2023-10-06T05:26:02.765691+0000","last_clean":"2023-10-06T05:26:02.765691+0000","last_became_active":"2023-10-06T05:26:01.783344+0000","last_became_peered":"2023-10-06T05:26:01.783344+0000","last_unstale":"2023-10-06T05:26:02.765691+0000","last_undegraded":"2023-10-06T05:26:02.765691+0000","last_fullsized":"2023-10-06T05:26:02.765691+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:04.675376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":314,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.765657+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:26:02.765657+0000","last_peered":"2023-10-06T05:26:02.765657+0000","last_clean":"2023-10-06T05:26:02.765657+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:26:02.765657+0000","last_undegraded":"2023-10-06T05:26:02.765657+0000","last_fullsized":"2023-10-06T05:26:02.765657+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":140,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.759389+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:26:02.759389+0000","last_peered":"2023-10-06T05:26:02.759389+0000","last_clean":"2023-10-06T05:26:02.759389+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:26:02.759389+0000","last_undegraded":"2023-10-06T05:26:02.759389+0000","last_fullsized":"2023-10-06T05:26:02.759389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.782564+0000","last_change":"2023-10-06T05:26:02.779174+0000","last_active":"2023-10-06T05:26:02.782564+0000","last_peered":"2023-10-06T05:26:02.782564+0000","last_clean":"2023-10-06T05:26:02.782564+0000","last_became_active":"2023-10-06T05:26:02.778850+0000","last_became_peered":"2023-10-06T05:26:02.778850+0000","last_unstale":"2023-10-06T05:26:02.782564+0000","last_undegraded":"2023-10-06T05:26:02.782564+0000","last_fullsized":"2023-10-06T05:26:02.782564+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:15.064865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.147809+0000","last_change":"2023-10-06T05:26:01.775267+0000","last_active":"2023-10-06T05:26:03.147809+0000","last_peered":"2023-10-06T05:26:03.147809+0000","last_clean":"2023-10-06T05:26:03.147809+0000","last_became_active":"2023-10-06T05:26:01.774977+0000","last_became_peered":"2023-10-06T05:26:01.774977+0000","last_unstale":"2023-10-06T05:26:03.147809+0000","last_undegraded":"2023-10-06T05:26:03.147809+0000","last_fullsized":"2023-10-06T05:26:03.147809+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:51:54.993091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":156,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.760716+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:26:02.760716+0000","last_peered":"2023-10-06T05:26:02.760716+0000","last_clean":"2023-10-06T05:26:02.760716+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:26:02.760716+0000","last_undegraded":"2023-10-06T05:26:02.760716+0000","last_fullsized":"2023-10-06T05:26:02.760716+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.780446+0000","last_change":"2023-10-06T05:26:02.776800+0000","last_active":"2023-10-06T05:26:02.780446+0000","last_peered":"2023-10-06T05:26:02.780446+0000","last_clean":"2023-10-06T05:26:02.780446+0000","last_became_active":"2023-10-06T05:26:02.776447+0000","last_became_peered":"2023-10-06T05:26:02.776447+0000","last_unstale":"2023-10-06T05:26:02.780446+0000","last_undegraded":"2023-10-06T05:26:02.780446+0000","last_fullsized":"2023-10-06T05:26:02.780446+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:01.747396+0000","last_clean_scrub_stamp":"2023-10-06T05:26:01.747396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:02.754179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"162'4","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.149842+0000","last_change":"2023-10-06T05:26:01.777709+0000","last_active":"2023-10-06T05:26:03.149842+0000","last_peered":"2023-10-06T05:26:03.149842+0000","last_clean":"2023-10-06T05:26:03.149842+0000","last_became_active":"2023-10-06T05:26:01.777391+0000","last_became_peered":"2023-10-06T05:26:01.777391+0000","last_unstale":"2023-10-06T05:26:03.149842+0000","last_undegraded":"2023-10-06T05:26:03.149842+0000","last_fullsized":"2023-10-06T05:26:03.149842+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:00.740435+0000","last_clean_scrub_stamp":"2023-10-06T05:26:00.740435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:50.198769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-06T05:26:02.760587+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:26:02.760587+0000","last_peered":"2023-10-06T05:26:02.760587+0000","last_clean":"2023-10-06T05:26:02.760587+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:26:02.760587+0000","last_undegraded":"2023-10-06T05:26:02.760587+0000","last_fullsized":"2023-10-06T05:26:02.760587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303992,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067848,"statfs":{"total":96636764160,"available":96325476352,"internally_reserved":0,"allocated":2670592,"data_stored":1554000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.50800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89700000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71199999999999997,"15min":0.72099999999999997},"min":{"1min":0.51800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83899999999999997,"5min":1.081,"15min":1.081},"last":0.77400000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70499999999999996,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.73099999999999998},"min":{"1min":0.53800000000000003,"5min":0.48099999999999998,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.90400000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.67800000000000005},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91700000000000004,"5min":0.93600000000000005,"15min":1.018},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.73699999999999999},"min":{"1min":0.51200000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90000000000000002,"5min":0.97799999999999998,"15min":1.056},"last":0.68000000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71099999999999997,"15min":0.73999999999999999},"min":{"1min":0.53400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94199999999999995,"5min":1.149,"15min":1.149},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.72699999999999998,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65700000000000003,"15min":0.69999999999999996},"min":{"1min":0.37,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86499999999999999,"5min":0.96199999999999997,"15min":1.0249999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.76600000000000001},"min":{"1min":0.46899999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97499999999999998,"5min":1.27,"15min":1.27},"last":0.79000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69699999999999995,"15min":0.72899999999999998},"min":{"1min":0.502,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91300000000000003,"5min":0.97899999999999998,"15min":0.999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75,"15min":0.80900000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90600000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.876}]},{"osd":6,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79500000000000004,"15min":0.85899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96499999999999997,"5min":1.0449999999999999,"15min":1.1040000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.52200000000000002,"5min":0.46200000000000002,"15min":0.441},"max":{"1min":1.0089999999999999,"5min":1.248,"15min":1.248},"last":0.92400000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303412,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068428,"statfs":{"total":96636764160,"available":96326070272,"internally_reserved":0,"allocated":2076672,"data_stored":964116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63900000000000001,"15min":0.67000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":0.89800000000000002,"15min":1.8500000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66500000000000004,"15min":0.69699999999999995},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0369999999999999,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69099999999999995,"15min":0.70699999999999996},"min":{"1min":0.50800000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93000000000000005,"5min":1.177,"15min":1.9279999999999999},"last":0.628},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95799999999999996,"5min":1.073,"15min":1.4239999999999999},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65100000000000002,"15min":0.65200000000000002},"min":{"1min":0.47299999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.877,"5min":1.093,"15min":1.369},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71299999999999997,"15min":0.751},"min":{"1min":0.50600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.038,"5min":1.085,"15min":1.891},"last":0.436}]},{"osd":3,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71599999999999997,"15min":0.74099999999999999},"min":{"1min":0.49299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0089999999999999,"5min":1.018,"15min":1.347},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.71399999999999997},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.98699999999999999,"15min":1},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70999999999999996,"15min":0.75900000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.9670000000000001},"last":0.5},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77800000000000002,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0209999999999999,"5min":1.278,"15min":1.4510000000000001},"last":0.75}]},{"osd":5,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.50700000000000001,"5min":0.41999999999999998,"15min":0.41499999999999998},"max":{"1min":0.82899999999999996,"5min":1.0680000000000001,"15min":1.415},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.753,"15min":0.80400000000000005},"min":{"1min":0.52300000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1020000000000001,"5min":1.2350000000000001,"15min":2},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:25:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77200000000000002,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.016,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76400000000000001,"15min":0.77500000000000002},"min":{"1min":0.54900000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95799999999999996,"5min":1.133,"15min":1.46},"last":0.84299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303976,"kb_used_data":2592,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067864,"statfs":{"total":96636764160,"available":96325492736,"internally_reserved":0,"allocated":2654208,"data_stored":1552394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.44,"15min":0.436},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.621,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.48599999999999999},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.504,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.497}]},{"osd":2,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.377,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.54100000000000004,"5min":0.77000000000000002,"15min":0.83799999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.59999999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.44900000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.58499999999999996,"5min":0.80500000000000005,"15min":0.92400000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.51000000000000001,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.57299999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.52400000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0680000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79200000000000004,"15min":0.79300000000000004},"min":{"1min":0.63600000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0189999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.73399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74099999999999999,"15min":0.74399999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.81000000000000005},"min":{"1min":0.61599999999999999,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":1.002,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73399999999999999,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93799999999999994,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.58799999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1140000000000001,"5min":1.387,"15min":1.387},"last":0.625}]},{"osd":7,"last update":"Fri Oct 6 05:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.81100000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.50600000000000001},"max":{"1min":1.044,"5min":1.244,"15min":1.244},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81399999999999995,"15min":0.82199999999999995},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.69499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303400,"kb_used_data":2016,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068440,"statfs":{"total":96636764160,"available":96326082560,"internally_reserved":0,"allocated":2064384,"data_stored":962116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41499999999999998,"15min":0.42799999999999999},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.51300000000000001,"5min":0.64500000000000002,"15min":0.84499999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.44500000000000001,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70999999999999996,"5min":0.755,"15min":1.042},"last":0.55600000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.48799999999999999},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.624,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.32800000000000001},"max":{"1min":0.63,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.48199999999999998,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.67500000000000004,"5min":0.80100000000000005,"15min":0.90500000000000003},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.745},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.80000000000000004,"5min":1.1519999999999999,"15min":1.242},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.76700000000000002},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97299999999999998,"5min":0.97899999999999998,"15min":1.198},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73399999999999999,"15min":0.76100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.90900000000000003,"5min":1.1659999999999999,"15min":1.26},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73099999999999998,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.84099999999999997,"5min":1.1299999999999999,"15min":1.163},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73499999999999999,"15min":0.78600000000000003},"min":{"1min":0.55300000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92500000000000004,"5min":1.105,"15min":1.323},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76300000000000001,"15min":0.78600000000000003},"min":{"1min":0.54600000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95099999999999996,"5min":1.123,"15min":1.3100000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77400000000000002,"15min":0.82199999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.52400000000000002},"max":{"1min":0.84999999999999998,"5min":1.2010000000000001,"15min":1.264},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.81100000000000005},"min":{"1min":0.58199999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":0.68500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303400,"kb_used_data":2016,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068440,"statfs":{"total":96636764160,"available":96326082560,"internally_reserved":0,"allocated":2064384,"data_stored":962116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.77800000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.46000000000000002,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.55400000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45100000000000001,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.57399999999999995,"5min":0.71699999999999997,"15min":0.88700000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.31900000000000001},"max":{"1min":0.57799999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.627,"5min":0.80600000000000005,"15min":0.93600000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.38500000000000001,"5min":0.372,"15min":0.36199999999999999},"max":{"1min":0.57299999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.997,"5min":1.022,"15min":1.1299999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.77800000000000002},"min":{"1min":0.53300000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.018,"5min":1.085,"15min":1.1699999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.52400000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91600000000000004,"5min":1.042,"15min":1.099},"last":0.75},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73699999999999999,"15min":0.78900000000000003},"min":{"1min":0.55600000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.163},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.81799999999999995},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98599999999999999,"5min":1.056,"15min":1.21},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.79900000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1200000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.77100000000000002,"15min":0.83899999999999997},"min":{"1min":0.58999999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97099999999999997,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":0.60399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303408,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068432,"statfs":{"total":96636764160,"available":96326074368,"internally_reserved":0,"allocated":2072576,"data_stored":962152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.435,"15min":0.45100000000000001},"min":{"1min":0.35199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70799999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.76200000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.57399999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.65300000000000002,"5min":0.999,"15min":0.999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.98999999999999999,"15min":1.014},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.752,"15min":0.79400000000000004},"min":{"1min":0.52400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.88300000000000001,"5min":1.1220000000000001,"15min":1.2669999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78900000000000003,"15min":0.81899999999999995},"min":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.54200000000000004},"max":{"1min":0.93200000000000005,"5min":1.113,"15min":1.3540000000000001},"last":0.94599999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97199999999999998,"5min":1.093,"15min":1.204},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.81399999999999995},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98499999999999999,"5min":1.208,"15min":1.329},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.79400000000000004},"min":{"1min":0.52300000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.96099999999999997,"5min":1.0880000000000001,"15min":1.484},"last":0.88},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.82099999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.2809999999999999,"15min":1.629},"last":0.872}]},{"osd":7,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89500000000000002,"5min":1.1379999999999999,"15min":1.5620000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79500000000000004,"15min":0.81999999999999995},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.325},"last":0.97499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303400,"kb_used_data":2016,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068440,"statfs":{"total":96636764160,"available":96326082560,"internally_reserved":0,"allocated":2064384,"data_stored":962580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.75600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.95899999999999996,"5min":1.02,"15min":2.77},"last":0.622}]},{"osd":1,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.46899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.6419999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67900000000000005,"15min":0.71099999999999997},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.99199999999999999,"5min":1.1100000000000001,"15min":1.1899999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.39800000000000002},"max":{"1min":0.93799999999999994,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71799999999999997,"15min":0.79300000000000004},"min":{"1min":0.52100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.442}]},{"osd":3,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.52900000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89000000000000001,"5min":0.95899999999999996,"15min":1.0629999999999999},"last":0.75},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70799999999999996,"15min":0.96299999999999997},"min":{"1min":0.52700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97899999999999998,"5min":0.99299999999999999,"15min":7.742},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.51000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.079},"last":0.503},{"interface":"front","average":{"1min":0.755,"5min":0.72499999999999998,"15min":0.746},"min":{"1min":0.59999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.012,"5min":1.04,"15min":1.181},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70299999999999996,"15min":0.72899999999999998},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92900000000000005,"5min":1.0029999999999999,"15min":1.2669999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77300000000000002,"15min":0.80800000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.436},"max":{"1min":0.95399999999999996,"5min":1.093,"15min":1.171},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78600000000000003,"15min":0.81999999999999995},"min":{"1min":0.69599999999999995,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0589999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78400000000000003,"15min":0.83999999999999997},"min":{"1min":0.59199999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.403,"5min":1.403,"15min":1.571},"last":0.64200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":962082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63900000000000001,"15min":0.85099999999999998},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.98099999999999998,"5min":0.98199999999999998,"15min":9.2780000000000005},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65400000000000003,"15min":0.66600000000000004},"min":{"1min":0.40699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":1.0449999999999999,"15min":1.0469999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66700000000000004,"15min":0.875},"min":{"1min":0.437,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":1.0329999999999999,"15min":9.25},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.91900000000000004},"min":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0549999999999999,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81000000000000005,"5min":1.052,"15min":10.384},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70999999999999996,"15min":0.92500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.67600000000000005}]},{"osd":3,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.95399999999999996},"min":{"1min":0.52800000000000002,"5min":0.502,"15min":0.495},"max":{"1min":0.879,"5min":1.76,"15min":10.205},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.46100000000000002},"max":{"1min":0.92600000000000005,"5min":1.097,"15min":1.292},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.069,"15min":1.069},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0409999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77700000000000002,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.298},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95899999999999996,"5min":1.556,"15min":1.556},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.79900000000000004},"min":{"1min":0.47999999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.91800000000000004,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:26:08.024 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:26:08.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:26:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.258+0000 7f21bb1ff700 1 Processor -- start 2023-10-06T05:26:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.259+0000 7f21bb1ff700 1 -- start start 2023-10-06T05:26:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.260+0000 7f21bb1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21b415ca40 0x7f21b4068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.260+0000 7f21bb1ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21b40c8f40 0x7f21b40adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.260+0000 7f21bb1ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b4068da0 con 0x7f21b406a300 2023-10-06T05:26:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.260+0000 7f21bb1ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b40cabc0 con 0x7f21b40c8f40 2023-10-06T05:26:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.260+0000 7f21bb1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b4068bb0 con 0x7f21b415ca40 2023-10-06T05:26:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.260+0000 7f21b979c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21b40c8f40 0x7f21b40adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.260+0000 7f21b979c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21b40c8f40 0x7f21b40adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55174/0 (socket says 172.21.15.31:55174) 2023-10-06T05:26:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.260+0000 7f21b979c700 1 -- 172.21.15.31:0/1486126062 learned_addr learned my addr 172.21.15.31:0/1486126062 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:08.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.260+0000 7f21b8f9b700 1 --2- 172.21.15.31:0/1486126062 >> v2:172.21.15.62:3300/0 conn(0x7f21b415ca40 0x7f21b4068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.260+0000 7f21b979c700 1 -- 172.21.15.31:0/1486126062 >> v2:172.21.15.62:3300/0 conn(0x7f21b415ca40 msgr2=0x7f21b4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:08.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.261+0000 7f21b979c700 1 --2- 172.21.15.31:0/1486126062 >> v2:172.21.15.62:3300/0 conn(0x7f21b415ca40 0x7f21b4068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.261+0000 7f21b979c700 1 -- 172.21.15.31:0/1486126062 >> v1:172.21.15.62:6789/0 conn(0x7f21b406a300 legacy=0x7f21b40c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.261+0000 7f21b979c700 1 -- 172.21.15.31:0/1486126062 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b4069080 con 0x7f21b40c8f40 2023-10-06T05:26:08.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.261+0000 7f21b979c700 1 --2- 172.21.15.31:0/1486126062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 0x7f21b40adde0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f21b4067f80 tx=0x7f21ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482e6f93a0860bc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.261+0000 7f21b37fe700 1 -- 172.21.15.31:0/1486126062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00e070 con 0x7f21b40c8f40 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.261+0000 7f21b37fe700 1 -- 172.21.15.31:0/1486126062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ac00ba40 con 0x7f21b40c8f40 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.261+0000 7f21b37fe700 1 -- 172.21.15.31:0/1486126062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac0092e0 con 0x7f21b40c8f40 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.262+0000 7f21bb1ff700 1 -- 172.21.15.31:0/1486126062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 msgr2=0x7f21b40adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.262+0000 7f21bb1ff700 1 --2- 172.21.15.31:0/1486126062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 0x7f21b40adde0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f21b4067f80 tx=0x7f21ac009840 comp rx=0 tx=0).stop 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.262+0000 7f21bb1ff700 1 -- 172.21.15.31:0/1486126062 shutdown_connections 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.262+0000 7f21bb1ff700 1 --2- 172.21.15.31:0/1486126062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 0x7f21b40adde0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.262+0000 7f21bb1ff700 1 --2- 172.21.15.31:0/1486126062 >> v2:172.21.15.62:3300/0 conn(0x7f21b415ca40 0x7f21b4068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.262+0000 7f21bb1ff700 1 -- 172.21.15.31:0/1486126062 >> 172.21.15.31:0/1486126062 conn(0x7f21b40ce290 msgr2=0x7f21b40ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.262+0000 7f21bb1ff700 1 -- 172.21.15.31:0/1486126062 shutdown_connections 2023-10-06T05:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.262+0000 7f21bb1ff700 1 -- 172.21.15.31:0/1486126062 wait complete. 2023-10-06T05:26:08.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.264+0000 7f21bb1ff700 1 Processor -- start 2023-10-06T05:26:08.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.264+0000 7f21bb1ff700 1 -- start start 2023-10-06T05:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.265+0000 7f21bb1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21b406a300 0x7f21b4169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.265+0000 7f21bb1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 0x7f21b40c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.265+0000 7f21bb1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21b415ca40 0x7f21b416c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21bb1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b4169760 con 0x7f21b40c8f40 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21bb1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b40b2590 con 0x7f21b406a300 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21bb1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41690e0 con 0x7f21b415ca40 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21b3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 0x7f21b40c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21b3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 0x7f21b40c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55180/0 (socket says 172.21.15.31:55180) 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21b3fff700 1 -- 172.21.15.31:0/2231718250 learned_addr learned my addr 172.21.15.31:0/2231718250 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21b8f9b700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21b406a300 0x7f21b4169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21b979c700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21b415ca40 0x7f21b416c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21b3fff700 1 -- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21b415ca40 msgr2=0x7f21b416c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21b3fff700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21b415ca40 0x7f21b416c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21b3fff700 1 -- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21b406a300 msgr2=0x7f21b4169b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21b3fff700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21b406a300 0x7f21b4169b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.266+0000 7f21b3fff700 1 -- 172.21.15.31:0/2231718250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac009ae0 con 0x7f21b40c8f40 2023-10-06T05:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.267+0000 7f21b3fff700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 0x7f21b40c69a0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f21a000c590 tx=0x7f21a000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.267+0000 7f21b0ff9700 1 -- 172.21.15.31:0/2231718250 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0011590 con 0x7f21b40c8f40 2023-10-06T05:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.267+0000 7f21b0ff9700 1 -- 172.21.15.31:0/2231718250 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a0011710 con 0x7f21b40c8f40 2023-10-06T05:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.267+0000 7f21bb1ff700 1 -- 172.21.15.31:0/2231718250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b40c6ee0 con 0x7f21b40c8f40 2023-10-06T05:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.267+0000 7f21b0ff9700 1 -- 172.21.15.31:0/2231718250 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0011ad0 con 0x7f21b40c8f40 2023-10-06T05:26:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.267+0000 7f21bb1ff700 1 -- 172.21.15.31:0/2231718250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b4169380 con 0x7f21b40c8f40 2023-10-06T05:26:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.268+0000 7f21bb1ff700 1 -- 172.21.15.31:0/2231718250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b41698a0 con 0x7f21b40c8f40 2023-10-06T05:26:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.269+0000 7f21b0ff9700 1 -- 172.21.15.31:0/2231718250 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f21a001b3b0 con 0x7f21b40c8f40 2023-10-06T05:26:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.270+0000 7f21b0ff9700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2198067830 0x7f2198069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.270+0000 7f21b0ff9700 1 -- 172.21.15.31:0/2231718250 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f21a00822a0 con 0x7f21b40c8f40 2023-10-06T05:26:08.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.270+0000 7f21b8f9b700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2198067830 0x7f2198069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.271+0000 7f21b8f9b700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2198067830 0x7f2198069ce0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f21a8006fd0 tx=0x7f21a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:08.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.274+0000 7f21b0ff9700 1 -- 172.21.15.31:0/2231718250 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21b41698a0 con 0x7f21b40c8f40 2023-10-06T05:26:08.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.558+0000 7f21bb1ff700 1 -- 172.21.15.31:0/2231718250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f21b41698a0 con 0x7f21b40c8f40 2023-10-06T05:26:08.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.563+0000 7f21b0ff9700 1 -- 172.21.15.31:0/2231718250 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7f21b41698a0 con 0x7f21b40c8f40 2023-10-06T05:26:08.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 -- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2198067830 msgr2=0x7f2198069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2198067830 0x7f2198069ce0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f21a8006fd0 tx=0x7f21a8008040 comp rx=0 tx=0).stop 2023-10-06T05:26:08.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 -- 172.21.15.31:0/2231718250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 msgr2=0x7f21b40c69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 0x7f21b40c69a0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f21a000c590 tx=0x7f21a000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 -- 172.21.15.31:0/2231718250 shutdown_connections 2023-10-06T05:26:08.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2198067830 0x7f2198069ce0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21b415ca40 0x7f21b416c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21b406a300 0x7f21b4169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 --2- 172.21.15.31:0/2231718250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21b40c8f40 0x7f21b40c69a0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 -- 172.21.15.31:0/2231718250 >> 172.21.15.31:0/2231718250 conn(0x7f21b40ce290 msgr2=0x7f21b40cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:08.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 -- 172.21.15.31:0/2231718250 shutdown_connections 2023-10-06T05:26:08.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.565+0000 7f21bb1ff700 1 -- 172.21.15.31:0/2231718250 wait complete. 2023-10-06T05:26:08.569 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-10-06T05:26:08.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.778+0000 7fcb3e9ff700 1 Processor -- start 2023-10-06T05:26:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.779+0000 7fcb3e9ff700 1 -- start start 2023-10-06T05:26:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.780+0000 7fcb3e9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcb38159500 0x7fcb380b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.780+0000 7fcb3e9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb3815d2d0 0x7fcb380cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.780+0000 7fcb3e9ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb380c3f40 con 0x7fcb381598d0 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.780+0000 7fcb3e9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb380b2860 con 0x7fcb3815d2d0 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.780+0000 7fcb3e9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3815d7d0 con 0x7fcb38159500 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.780+0000 7fcb377fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb3815d2d0 0x7fcb380cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.780+0000 7fcb377fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb3815d2d0 0x7fcb380cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55186/0 (socket says 172.21.15.31:55186) 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.780+0000 7fcb377fe700 1 -- 172.21.15.31:0/3457113804 learned_addr learned my addr 172.21.15.31:0/3457113804 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.780+0000 7fcb3cf9c700 1 --1- 172.21.15.31:0/3457113804 >> v1:172.21.15.31:6789/0 conn(0x7fcb381598d0 0x7fcb380aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:40838/0 (socket says 172.21.15.31:40838) 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.780+0000 7fcb37fff700 1 --2- 172.21.15.31:0/3457113804 >> v2:172.21.15.62:3300/0 conn(0x7fcb38159500 0x7fcb380b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.781+0000 7fcb377fe700 1 -- 172.21.15.31:0/3457113804 >> v2:172.21.15.62:3300/0 conn(0x7fcb38159500 msgr2=0x7fcb380b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.781+0000 7fcb377fe700 1 --2- 172.21.15.31:0/3457113804 >> v2:172.21.15.62:3300/0 conn(0x7fcb38159500 0x7fcb380b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.781+0000 7fcb377fe700 1 -- 172.21.15.31:0/3457113804 >> v1:172.21.15.31:6789/0 conn(0x7fcb381598d0 legacy=0x7fcb380aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.781+0000 7fcb377fe700 1 -- 172.21.15.31:0/3457113804 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380cb000 con 0x7fcb3815d2d0 2023-10-06T05:26:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.781+0000 7fcb36ffd700 1 -- 172.21.15.31:0/3457113804 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116702363 0 0) 0x7fcb380c3f40 con 0x7fcb381598d0 2023-10-06T05:26:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.781+0000 7fcb377fe700 1 --2- 172.21.15.31:0/3457113804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb3815d2d0 0x7fcb380cce00 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fcb38089840 tx=0x7fcb2c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa16e83ae388fa9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.781+0000 7fcb36ffd700 1 -- 172.21.15.31:0/3457113804 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c005c10 con 0x7fcb3815d2d0 2023-10-06T05:26:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.781+0000 7fcb36ffd700 1 -- 172.21.15.31:0/3457113804 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c005d70 con 0x7fcb3815d2d0 2023-10-06T05:26:08.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.781+0000 7fcb36ffd700 1 -- 172.21.15.31:0/3457113804 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c006260 con 0x7fcb3815d2d0 2023-10-06T05:26:08.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.782+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3457113804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb3815d2d0 msgr2=0x7fcb380cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.782+0000 7fcb3e9ff700 1 --2- 172.21.15.31:0/3457113804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb3815d2d0 0x7fcb380cce00 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fcb38089840 tx=0x7fcb2c009220 comp rx=0 tx=0).stop 2023-10-06T05:26:08.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.782+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3457113804 shutdown_connections 2023-10-06T05:26:08.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.782+0000 7fcb3e9ff700 1 --2- 172.21.15.31:0/3457113804 >> v2:172.21.15.62:3300/0 conn(0x7fcb38159500 0x7fcb380b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.782+0000 7fcb3e9ff700 1 --2- 172.21.15.31:0/3457113804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb3815d2d0 0x7fcb380cce00 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.782+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3457113804 >> 172.21.15.31:0/3457113804 conn(0x7fcb3806ec90 msgr2=0x7fcb3806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:08.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.782+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3457113804 shutdown_connections 2023-10-06T05:26:08.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.782+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3457113804 wait complete. 2023-10-06T05:26:08.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.784+0000 7fcb3e9ff700 1 Processor -- start 2023-10-06T05:26:08.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.784+0000 7fcb3e9ff700 1 -- start start 2023-10-06T05:26:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb3e9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb38159500 0x7fcb3816e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb3e9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb381598d0 0x7fcb3816a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb3e9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3815d2d0 0x7fcb380c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb3e9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3816b1c0 con 0x7fcb38159500 2023-10-06T05:26:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb3e9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3815b410 con 0x7fcb381598d0 2023-10-06T05:26:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb3e9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3816e700 con 0x7fcb3815d2d0 2023-10-06T05:26:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb38159500 0x7fcb3816e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb38159500 0x7fcb3816e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55194/0 (socket says 172.21.15.31:55194) 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb37fff700 1 -- 172.21.15.31:0/3536114859 learned_addr learned my addr 172.21.15.31:0/3536114859 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb377fe700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb381598d0 0x7fcb3816a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb3cf9c700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3815d2d0 0x7fcb380c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb37fff700 1 -- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3815d2d0 msgr2=0x7fcb380c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb37fff700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3815d2d0 0x7fcb380c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb37fff700 1 -- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb381598d0 msgr2=0x7fcb3816a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb37fff700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb381598d0 0x7fcb3816a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.785+0000 7fcb37fff700 1 -- 172.21.15.31:0/3536114859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c009480 con 0x7fcb38159500 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.786+0000 7fcb37fff700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb38159500 0x7fcb3816e1c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800ec80 tx=0x7fcb280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.786+0000 7fcb1ffff700 1 -- 172.21.15.31:0/3536114859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800c9d0 con 0x7fcb38159500 2023-10-06T05:26:08.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.786+0000 7fcb1ffff700 1 -- 172.21.15.31:0/3536114859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2800cb30 con 0x7fcb38159500 2023-10-06T05:26:08.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.786+0000 7fcb1ffff700 1 -- 172.21.15.31:0/3536114859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28013470 con 0x7fcb38159500 2023-10-06T05:26:08.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.786+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3536114859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3816ac00 con 0x7fcb38159500 2023-10-06T05:26:08.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.786+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3536114859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3816d9c0 con 0x7fcb38159500 2023-10-06T05:26:08.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.787+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3536114859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3804c380 con 0x7fcb38159500 2023-10-06T05:26:08.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.788+0000 7fcb1ffff700 1 -- 172.21.15.31:0/3536114859 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fcb28013820 con 0x7fcb38159500 2023-10-06T05:26:08.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.789+0000 7fcb1ffff700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb24067830 0x7fcb24069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:08.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.790+0000 7fcb1ffff700 1 -- 172.21.15.31:0/3536114859 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fcb28083410 con 0x7fcb38159500 2023-10-06T05:26:08.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.790+0000 7fcb377fe700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb24067830 0x7fcb24069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:08.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.790+0000 7fcb377fe700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb24067830 0x7fcb24069ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009260 tx=0x7fcb2c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:08.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:08.794+0000 7fcb1ffff700 1 -- 172.21.15.31:0/3536114859 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb28051e60 con 0x7fcb38159500 2023-10-06T05:26:09.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.044+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3536114859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb3815d0b0 con 0x7fcb38159500 2023-10-06T05:26:09.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.045+0000 7fcb1ffff700 1 -- 172.21.15.31:0/3536114859 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4607 (secure 0 0 0) 0x7fcb28017090 con 0x7fcb38159500 2023-10-06T05:26:09.047 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:09.047 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":112,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4242992570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4242992570},{"type":"v1","addr":"172.21.15.31:6835","nonce":4242992570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":7706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2405114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2405114362},{"type":"v1","addr":"172.21.15.62:6834","nonce":2405114362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2676480318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2676480318},{"type":"v1","addr":"172.21.15.62:6835","nonce":2676480318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:02.260523+0000","modified":"2023-10-06T05:26:06.149545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9249":{"gid":9249,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3001391649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3001391649},{"type":"v1","addr":"172.21.15.31:6834","nonce":3001391649}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:26:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.047+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb24067830 msgr2=0x7fcb24069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.047+0000 7fcb3e9ff700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb24067830 0x7fcb24069ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009260 tx=0x7fcb2c016000 comp rx=0 tx=0).stop 2023-10-06T05:26:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.047+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3536114859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb38159500 msgr2=0x7fcb3816e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.047+0000 7fcb3e9ff700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb38159500 0x7fcb3816e1c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800ec80 tx=0x7fcb280091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.047+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3536114859 shutdown_connections 2023-10-06T05:26:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.047+0000 7fcb3e9ff700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fcb24067830 0x7fcb24069ce0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.047+0000 7fcb3e9ff700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3815d2d0 0x7fcb380c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.047+0000 7fcb3e9ff700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb38159500 0x7fcb3816e1c0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.047+0000 7fcb3e9ff700 1 --2- 172.21.15.31:0/3536114859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb381598d0 0x7fcb3816a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.047+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3536114859 >> 172.21.15.31:0/3536114859 conn(0x7fcb3806ec90 msgr2=0x7fcb380d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.048+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3536114859 shutdown_connections 2023-10-06T05:26:09.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.048+0000 7fcb3e9ff700 1 -- 172.21.15.31:0/3536114859 wait complete. 2023-10-06T05:26:09.051 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 112 2023-10-06T05:26:09.469 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:26:02.260523+0000', 'modified': '2023-10-06T05:26:06.149545+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9249}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9249': {'gid': 9249, 'name': 'c', 'rank': 0, 'incarnation': 108, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6834/3001391649', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6832', 'nonce': 3001391649}, {'type': 'v1', 'addr': '172.21.15.31:6834', 'nonce': 3001391649}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:26:09.469 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:26:09.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T05:26:09.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.666+0000 7f04b77ff700 1 Processor -- start 2023-10-06T05:26:09.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.666+0000 7f04b77ff700 1 -- start start 2023-10-06T05:26:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b77ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f04b01595a0 0x7f04b0159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b77ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f04b00cc0d0 0x7f04b0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b77ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04b0068bb0 0x7f04b00c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b77ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b0069010 con 0x7f04b01595a0 2023-10-06T05:26:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b77ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b00c95b0 con 0x7f04b00cc0d0 2023-10-06T05:26:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b77ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00c9890 con 0x7f04b0068bb0 2023-10-06T05:26:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b559b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f04b01595a0 0x7f04b0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b559b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f04b01595a0 0x7f04b0159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55202/0 (socket says 172.21.15.31:55202) 2023-10-06T05:26:09.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b559b700 1 -- 172.21.15.31:0/999518204 learned_addr learned my addr 172.21.15.31:0/999518204 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:09.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b5d9c700 1 --2- 172.21.15.31:0/999518204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04b0068bb0 0x7f04b00c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/999518204 >> v2:172.21.15.62:3300/0 conn(0x7f04b00cc0d0 0x7f04b0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b559b700 1 -- 172.21.15.31:0/999518204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04b0068bb0 msgr2=0x7f04b00c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b559b700 1 --2- 172.21.15.31:0/999518204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04b0068bb0 0x7f04b00c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b559b700 1 -- 172.21.15.31:0/999518204 >> v2:172.21.15.62:3300/0 conn(0x7f04b00cc0d0 msgr2=0x7f04b0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b559b700 1 --2- 172.21.15.31:0/999518204 >> v2:172.21.15.62:3300/0 conn(0x7f04b00cc0d0 0x7f04b0068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b559b700 1 -- 172.21.15.31:0/999518204 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b00c9b70 con 0x7f04b01595a0 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.668+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/999518204 >> v2:172.21.15.62:3300/0 conn(0x7f04b00cc0d0 0x7f04b0068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.669+0000 7f04b559b700 1 --2- 172.21.15.31:0/999518204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b01595a0 0x7f04b0159970 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f04b006a360 tx=0x7f04ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fcc5e2f257bb45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.669+0000 7f04a7fff700 1 -- 172.21.15.31:0/999518204 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00b860 con 0x7f04b01595a0 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.669+0000 7f04a7fff700 1 -- 172.21.15.31:0/999518204 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ac00b9c0 con 0x7f04b01595a0 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.669+0000 7f04a7fff700 1 -- 172.21.15.31:0/999518204 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00bd60 con 0x7f04b01595a0 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.669+0000 7f04b77ff700 1 -- 172.21.15.31:0/999518204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b01595a0 msgr2=0x7f04b0159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.669+0000 7f04b77ff700 1 --2- 172.21.15.31:0/999518204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b01595a0 0x7f04b0159970 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f04b006a360 tx=0x7f04ac009250 comp rx=0 tx=0).stop 2023-10-06T05:26:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.670+0000 7f04b77ff700 1 -- 172.21.15.31:0/999518204 shutdown_connections 2023-10-06T05:26:09.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.670+0000 7f04b77ff700 1 --2- 172.21.15.31:0/999518204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04b0068bb0 0x7f04b00c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.670+0000 7f04b77ff700 1 --2- 172.21.15.31:0/999518204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b01595a0 0x7f04b0159970 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.670+0000 7f04b77ff700 1 --2- 172.21.15.31:0/999518204 >> v2:172.21.15.62:3300/0 conn(0x7f04b00cc0d0 0x7f04b0068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.670+0000 7f04b77ff700 1 -- 172.21.15.31:0/999518204 >> 172.21.15.31:0/999518204 conn(0x7f04b006b620 msgr2=0x7f04b006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:09.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.670+0000 7f04b77ff700 1 -- 172.21.15.31:0/999518204 shutdown_connections 2023-10-06T05:26:09.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.670+0000 7f04b77ff700 1 -- 172.21.15.31:0/999518204 wait complete. 2023-10-06T05:26:09.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.672+0000 7f04b77ff700 1 Processor -- start 2023-10-06T05:26:09.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.672+0000 7f04b77ff700 1 -- start start 2023-10-06T05:26:09.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.673+0000 7f04b77ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04b0068bb0 0x7f04b01560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.673+0000 7f04b77ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b00cc0d0 0x7f04b00b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.673+0000 7f04b77ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01595a0 0x7f04b01527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.673+0000 7f04b77ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00b26d0 con 0x7f04b00cc0d0 2023-10-06T05:26:09.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.673+0000 7f04b77ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00b1ac0 con 0x7f04b01595a0 2023-10-06T05:26:09.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.673+0000 7f04b77ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00b1da0 con 0x7f04b0068bb0 2023-10-06T05:26:09.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.673+0000 7f04b4d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b00cc0d0 0x7f04b00b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.673+0000 7f04b4d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b00cc0d0 0x7f04b00b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55210/0 (socket says 172.21.15.31:55210) 2023-10-06T05:26:09.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.673+0000 7f04b4d9a700 1 -- 172.21.15.31:0/4283685368 learned_addr learned my addr 172.21.15.31:0/4283685368 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:09.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04b559b700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04b0068bb0 0x7f04b01560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04b5d9c700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01595a0 0x7f04b01527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04b4d9a700 1 -- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04b0068bb0 msgr2=0x7f04b01560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04b0068bb0 0x7f04b01560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04b4d9a700 1 -- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01595a0 msgr2=0x7f04b01527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01595a0 0x7f04b01527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04b4d9a700 1 -- 172.21.15.31:0/4283685368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0094f0 con 0x7f04b00cc0d0 2023-10-06T05:26:09.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b00cc0d0 0x7f04b00b1090 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f04a000e8d0 tx=0x7f04a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:09.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04a57fa700 1 -- 172.21.15.31:0/4283685368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a000ee70 con 0x7f04b00cc0d0 2023-10-06T05:26:09.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04a57fa700 1 -- 172.21.15.31:0/4283685368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a000c5e0 con 0x7f04b00cc0d0 2023-10-06T05:26:09.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04b77ff700 1 -- 172.21.15.31:0/4283685368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b00b2080 con 0x7f04b00cc0d0 2023-10-06T05:26:09.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.674+0000 7f04a57fa700 1 -- 172.21.15.31:0/4283685368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a000c980 con 0x7f04b00cc0d0 2023-10-06T05:26:09.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.675+0000 7f04b77ff700 1 -- 172.21.15.31:0/4283685368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b01556f0 con 0x7f04b00cc0d0 2023-10-06T05:26:09.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.677+0000 7f04a57fa700 1 -- 172.21.15.31:0/4283685368 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f04a0013410 con 0x7f04b00cc0d0 2023-10-06T05:26:09.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.677+0000 7f04a57fa700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0494067830 0x7f0494069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.678+0000 7f04a57fa700 1 -- 172.21.15.31:0/4283685368 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f04a0082e90 con 0x7f04b00cc0d0 2023-10-06T05:26:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.678+0000 7f04b559b700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0494067830 0x7f0494069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.679+0000 7f04b559b700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0494067830 0x7f0494069ce0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f04b006a360 tx=0x7f04ac017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.679+0000 7f04c5e37b80 1 Processor -- start 2023-10-06T05:26:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.679+0000 7f04c5e37b80 1 -- start start 2023-10-06T05:26:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.680+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd73450 0x55e85fd73820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.680+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd7bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.680+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd7c4d0 0x55e85fd7c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.680+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85fc46310 con 0x55e85fd58f40 2023-10-06T05:26:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.680+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85fc83130 con 0x55e85fd7c4d0 2023-10-06T05:26:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.680+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e85fc80af0 con 0x55e85fd73450 2023-10-06T05:26:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.680+0000 7f04b4d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd7bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.680+0000 7f04b4d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd7bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55212/0 (socket says 172.21.15.31:55212) 2023-10-06T05:26:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.680+0000 7f04b4d9a700 1 -- 172.21.15.31:0/2557719840 learned_addr learned my addr 172.21.15.31:0/2557719840 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f04b5d9c700 1 --2- 172.21.15.31:0/2557719840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd7c4d0 0x55e85fd7c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f04b559b700 1 --2- 172.21.15.31:0/2557719840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd73450 0x55e85fd73820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f04b4d9a700 1 -- 172.21.15.31:0/2557719840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd73450 msgr2=0x55e85fd73820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/2557719840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd73450 0x55e85fd73820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f04b4d9a700 1 -- 172.21.15.31:0/2557719840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd7c4d0 msgr2=0x55e85fd7c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/2557719840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd7c4d0 0x55e85fd7c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f04b4d9a700 1 -- 172.21.15.31:0/2557719840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e85fa82790 con 0x55e85fd58f40 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/2557719840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd7bf90 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f04a0055530 tx=0x7f04a00554a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806ee24edddde579 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f049a7fc700 1 -- 172.21.15.31:0/2557719840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a005c540 con 0x55e85fd58f40 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f049a7fc700 1 -- 172.21.15.31:0/2557719840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a00536e0 con 0x55e85fd58f40 2023-10-06T05:26:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.681+0000 7f049a7fc700 1 -- 172.21.15.31:0/2557719840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a000cd70 con 0x55e85fd58f40 2023-10-06T05:26:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.682+0000 7f04c5e37b80 1 -- 172.21.15.31:0/2557719840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 msgr2=0x55e85fd7bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.682+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/2557719840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd7bf90 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f04a0055530 tx=0x7f04a00554a0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.682+0000 7f04c5e37b80 1 -- 172.21.15.31:0/2557719840 shutdown_connections 2023-10-06T05:26:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.682+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/2557719840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd7c4d0 0x55e85fd7c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.682+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/2557719840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd73450 0x55e85fd73820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.682+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/2557719840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd7bf90 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.682+0000 7f04c5e37b80 1 -- 172.21.15.31:0/2557719840 >> 172.21.15.31:0/2557719840 conn(0x55e85fd76880 msgr2=0x55e85fd76c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.682+0000 7f04c5e37b80 1 -- 172.21.15.31:0/2557719840 shutdown_connections 2023-10-06T05:26:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.682+0000 7f04c5e37b80 1 -- 172.21.15.31:0/2557719840 wait complete. 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.683+0000 7f04c5e37b80 1 -- start start 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.683+0000 7f04c5e37b80 1 Processor -- start 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd58f40 0x55e85fd50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd73450 0x55e85fd515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd7c4d0 0x55e85fd51b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85fc7c8e0 con 0x55e85fd73450 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85fc28980 con 0x55e85fd58f40 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e85fc28620 con 0x55e85fd7c4d0 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04c5e37b80 10 client.0 _ll_register_callbacks cb 0x55e85fa8ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04b4d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd73450 0x55e85fd515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04b4d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd73450 0x55e85fd515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55218/0 (socket says 172.21.15.31:55218) 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04b4d9a700 1 -- 172.21.15.31:0/1118866625 learned_addr learned my addr 172.21.15.31:0/1118866625 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04b559b700 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd58f40 0x55e85fd50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.684+0000 7f04b5d9c700 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd7c4d0 0x55e85fd51b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f04b4d9a700 1 -- 172.21.15.31:0/1118866625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd7c4d0 msgr2=0x55e85fd51b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd7c4d0 0x55e85fd51b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f04b4d9a700 1 -- 172.21.15.31:0/1118866625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd58f40 msgr2=0x55e85fd50960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd58f40 0x55e85fd50960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f04b4d9a700 1 -- 172.21.15.31:0/1118866625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b01556f0 con 0x55e85fd73450 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd73450 0x55e85fd515e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f04a0055530 tx=0x7f04a006d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a565c64563e355ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f048f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f048f7fe700 1 -- 172.21.15.31:0/1118866625 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0058870 con 0x55e85fd73450 2023-10-06T05:26:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f048f7fe700 1 -- 172.21.15.31:0/1118866625 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f04a005c540 con 0x55e85fd73450 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f04c5e37b80 1 -- 172.21.15.31:0/1118866625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e85fd20d60 con 0x55e85fd73450 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f048f7fe700 1 -- 172.21.15.31:0/1118866625 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04a006bdd0 con 0x55e85fd73450 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f04c5e37b80 10 client.9405 fetch_fsmap learned FSMap version 112 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.685+0000 7f04c5e37b80 1 -- 172.21.15.31:0/1118866625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55e85fb6f0a0 con 0x55e85fd73450 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.686+0000 7f048f7fe700 1 -- 172.21.15.31:0/1118866625 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e85fd20d60 con 0x55e85fd73450 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.686+0000 7f048f7fe700 1 -- 172.21.15.31:0/1118866625 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 112) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f04a000cae0 con 0x55e85fd73450 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.686+0000 7f04c5e37b80 10 client.9405 fetch_fsmap finished waiting for FSMap version 112 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.686+0000 7f04c5e37b80 10 client.9405 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.686+0000 7f04c5e37b80 20 client.9405 populate_metadata read hostname 'smithi031' 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.686+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.31:6832/3001391649,v1:172.21.15.31:6834/3001391649] conn(0x55e85fd53fb0 0x55e85fd74c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.686+0000 7f04c5e37b80 4 client.9405 mds_command: new command op to 9249 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:26:09.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.686+0000 7f04c5e37b80 1 -- 172.21.15.31:0/1118866625 --> [v2:172.21.15.31:6832/3001391649,v1:172.21.15.31:6834/3001391649] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e85fcfdb30 con 0x55e85fd53fb0 2023-10-06T05:26:09.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.686+0000 7f04b559b700 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.31:6832/3001391649,v1:172.21.15.31:6834/3001391649] conn(0x55e85fd53fb0 0x55e85fd74c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.687+0000 7f048f7fe700 10 client.9405 ms_handle_connect on v2:172.21.15.31:6832/3001391649 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.687+0000 7f04b559b700 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.31:6832/3001391649,v1:172.21.15.31:6834/3001391649] conn(0x55e85fd53fb0 0x55e85fd74c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5743be56e438ec0 server_cookie=5ff65523cac77170 in_seq=0 out_seq=0 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.687+0000 7f048f7fe700 1 -- 172.21.15.31:0/1118866625 <== mds.0 v2:172.21.15.31:6832/3001391649 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e85fcfdb30 con 0x55e85fd53fb0 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.687+0000 7f048f7fe700 10 client.9405 handle_command_reply: tid=0 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.687+0000 7f04c5e37b80 1 client.9405 shutdown 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 1 -- 172.21.15.31:0/1118866625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd73450 msgr2=0x55e85fd515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd73450 0x55e85fd515e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f04a0055530 tx=0x7f04a006d000 comp rx=0 tx=0).stop 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 1 -- 172.21.15.31:0/1118866625 shutdown_connections 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.31:6832/3001391649,v1:172.21.15.31:6834/3001391649] conn(0x55e85fd53fb0 0x55e85fd74c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd7c4d0 0x55e85fd51b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd58f40 0x55e85fd50960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/1118866625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd73450 0x55e85fd515e0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f048f7fe700 0 client.9405 ms_handle_reset on v2:172.21.15.31:6832/3001391649 2023-10-06T05:26:09.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 1 -- 172.21.15.31:0/1118866625 >> 172.21.15.31:0/1118866625 conn(0x55e85fd76880 msgr2=0x55e85fd28c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:09.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 1 -- 172.21.15.31:0/1118866625 shutdown_connections 2023-10-06T05:26:09.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 1 -- 172.21.15.31:0/1118866625 wait complete. 2023-10-06T05:26:09.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.688+0000 7f04c5e37b80 20 client.9405 trim_cache size 0 max 16384 2023-10-06T05:26:09.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.705+0000 7f04c5e37b80 1 Processor -- start 2023-10-06T05:26:09.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.706+0000 7f04c5e37b80 1 -- start start 2023-10-06T05:26:09.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.706+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd58f40 0x55e85fd50930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.706+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd73450 0x55e85fd7bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.706+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd7c4d0 0x55e85fd771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.706+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85fb141e0 con 0x55e85fd7c4d0 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.706+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85fd75370 con 0x55e85fd73450 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.706+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e85fd53370 con 0x55e85fd58f40 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b5d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd7c4d0 0x55e85fd771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b5d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd7c4d0 0x55e85fd771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55234/0 (socket says 172.21.15.31:55234) 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b5d9c700 1 -- 172.21.15.31:0/783264336 learned_addr learned my addr 172.21.15.31:0/783264336 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b559b700 1 --2- 172.21.15.31:0/783264336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd58f40 0x55e85fd50930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/783264336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd73450 0x55e85fd7bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b5d9c700 1 -- 172.21.15.31:0/783264336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd58f40 msgr2=0x55e85fd50930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b5d9c700 1 --2- 172.21.15.31:0/783264336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd58f40 0x55e85fd50930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b5d9c700 1 -- 172.21.15.31:0/783264336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd73450 msgr2=0x55e85fd7bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b5d9c700 1 --2- 172.21.15.31:0/783264336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd73450 0x55e85fd7bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b5d9c700 1 -- 172.21.15.31:0/783264336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e85fc4b3c0 con 0x55e85fd7c4d0 2023-10-06T05:26:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.707+0000 7f04b5d9c700 1 --2- 172.21.15.31:0/783264336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd7c4d0 0x55e85fd771d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009f70 tx=0x7f04a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effcecb2cc3fcd0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f049affd700 1 -- 172.21.15.31:0/783264336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8017440 con 0x55e85fd7c4d0 2023-10-06T05:26:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f049affd700 1 -- 172.21.15.31:0/783264336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a800f070 con 0x55e85fd7c4d0 2023-10-06T05:26:09.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f049affd700 1 -- 172.21.15.31:0/783264336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a80177d0 con 0x55e85fd7c4d0 2023-10-06T05:26:09.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f04c5e37b80 1 -- 172.21.15.31:0/783264336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd7c4d0 msgr2=0x55e85fd771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/783264336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd7c4d0 0x55e85fd771d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009f70 tx=0x7f04a800b040 comp rx=0 tx=0).stop 2023-10-06T05:26:09.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f04c5e37b80 1 -- 172.21.15.31:0/783264336 shutdown_connections 2023-10-06T05:26:09.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/783264336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd7c4d0 0x55e85fd771d0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/783264336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd58f40 0x55e85fd50930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/783264336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd73450 0x55e85fd7bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f04c5e37b80 1 -- 172.21.15.31:0/783264336 >> 172.21.15.31:0/783264336 conn(0x55e85fd76880 msgr2=0x55e85fd2f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f04c5e37b80 1 -- 172.21.15.31:0/783264336 shutdown_connections 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.708+0000 7f04c5e37b80 1 -- 172.21.15.31:0/783264336 wait complete. 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.709+0000 7f04c5e37b80 1 -- start start 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.709+0000 7f04c5e37b80 1 Processor -- start 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.710+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd51890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.710+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd73450 0x55e85fd7b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.710+0000 7f04c5e37b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd7c4d0 0x55e85fdd7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.710+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85fd7b570 con 0x55e85fd58f40 2023-10-06T05:26:09.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.710+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85fd292d0 con 0x55e85fd7c4d0 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.710+0000 7f04c5e37b80 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e85fd28c80 con 0x55e85fd73450 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.710+0000 7f04c5e37b80 10 client.0 _ll_register_callbacks cb 0x55e85fa8ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.710+0000 7f04b559b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd51890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.710+0000 7f04b559b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd51890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55244/0 (socket says 172.21.15.31:55244) 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.710+0000 7f04b559b700 1 -- 172.21.15.31:0/3030326084 learned_addr learned my addr 172.21.15.31:0/3030326084 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd73450 0x55e85fd7b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f04b5d9c700 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd7c4d0 0x55e85fdd7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f04b559b700 1 -- 172.21.15.31:0/3030326084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd73450 msgr2=0x55e85fd7b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f04b559b700 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd73450 0x55e85fd7b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f04b559b700 1 -- 172.21.15.31:0/3030326084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd7c4d0 msgr2=0x55e85fdd7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f04b559b700 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd7c4d0 0x55e85fdd7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f04b559b700 1 -- 172.21.15.31:0/3030326084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a8009cd0 con 0x55e85fd58f40 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f04b559b700 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd51890 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f04b006ee00 tx=0x7f04ac028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f157fab5c8e0863 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f048f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f048f7fe700 1 -- 172.21.15.31:0/3030326084 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac01f070 con 0x55e85fd58f40 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f048f7fe700 1 -- 172.21.15.31:0/3030326084 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f04ac035590 con 0x55e85fd58f40 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.711+0000 7f04c5e37b80 1 -- 172.21.15.31:0/3030326084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e85fd2a660 con 0x55e85fd58f40 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f048f7fe700 1 -- 172.21.15.31:0/3030326084 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04ac035960 con 0x55e85fd58f40 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f04c5e37b80 10 client.9411 fetch_fsmap learned FSMap version 112 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f04c5e37b80 1 -- 172.21.15.31:0/3030326084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55e85fae31d0 con 0x55e85fd58f40 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f048f7fe700 1 -- 172.21.15.31:0/3030326084 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e85fd2a660 con 0x55e85fd58f40 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f048f7fe700 1 -- 172.21.15.31:0/3030326084 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 112) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f04ac035ac0 con 0x55e85fd58f40 2023-10-06T05:26:09.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f04c5e37b80 10 client.9411 fetch_fsmap finished waiting for FSMap version 112 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f04c5e37b80 10 client.9411 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f04c5e37b80 20 client.9411 populate_metadata read hostname 'smithi031' 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.31:6832/3001391649,v1:172.21.15.31:6834/3001391649] conn(0x55e85fd54dd0 0x55e85fd551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f04c5e37b80 4 client.9411 mds_command: new command op to 9249 tid=0[{"prefix": "status"}] 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f04c5e37b80 1 -- 172.21.15.31:0/3030326084 --> [v2:172.21.15.31:6832/3001391649,v1:172.21.15.31:6834/3001391649] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f04a00536e0 con 0x55e85fd54dd0 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.712+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.31:6832/3001391649,v1:172.21.15.31:6834/3001391649] conn(0x55e85fd54dd0 0x55e85fd551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04b4d9a700 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.31:6832/3001391649,v1:172.21.15.31:6834/3001391649] conn(0x55e85fd54dd0 0x55e85fd551a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa6cb6c2052d92d server_cookie=3fb028d48dd19133 in_seq=0 out_seq=0 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f048f7fe700 10 client.9411 ms_handle_connect on v2:172.21.15.31:6832/3001391649 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f048f7fe700 1 -- 172.21.15.31:0/3030326084 <== mds.0 v2:172.21.15.31:6832/3001391649 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x7f04a00536e0 con 0x55e85fd54dd0 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f048f7fe700 10 client.9411 handle_command_reply: tid=0 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 client.9411 shutdown 2023-10-06T05:26:09.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 -- 172.21.15.31:0/3030326084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 msgr2=0x55e85fd51890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd51890 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f04b006ee00 tx=0x7f04ac028040 comp rx=0 tx=0).stop 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 -- 172.21.15.31:0/3030326084 shutdown_connections 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.31:6832/3001391649,v1:172.21.15.31:6834/3001391649] conn(0x55e85fd54dd0 0x55e85fd551a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e85fd7c4d0 0x55e85fdd7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e85fd58f40 0x55e85fd51890 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 --2- 172.21.15.31:0/3030326084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e85fd73450 0x55e85fd7b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f048f7fe700 0 client.9411 ms_handle_reset on v2:172.21.15.31:6832/3001391649 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 -- 172.21.15.31:0/3030326084 >> 172.21.15.31:0/3030326084 conn(0x55e85fd76880 msgr2=0x55e85fd51180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 -- 172.21.15.31:0/3030326084 shutdown_connections 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.713+0000 7f04c5e37b80 1 -- 172.21.15.31:0/3030326084 wait complete. 2023-10-06T05:26:09.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04c5e37b80 20 client.9411 trim_cache size 0 max 16384 2023-10-06T05:26:09.721 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:26:09.721 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "86bc8148-d401-41f1-bd7f-edc6b2417154", 2023-10-06T05:26:09.721 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-10-06T05:26:09.721 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 9249, 2023-10-06T05:26:09.721 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-10-06T05:26:09.721 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-10-06T05:26:09.721 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-10-06T05:26:09.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.5772127539999996, 2023-10-06T05:26:09.721 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 112, 2023-10-06T05:26:09.721 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 163, 2023-10-06T05:26:09.722 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 163, 2023-10-06T05:26:09.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.627451902000001 2023-10-06T05:26:09.722 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:26:09.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 -- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0494067830 msgr2=0x7f0494069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0494067830 0x7f0494069ce0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f04b006a360 tx=0x7f04ac017000 comp rx=0 tx=0).stop 2023-10-06T05:26:09.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 -- 172.21.15.31:0/4283685368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b00cc0d0 msgr2=0x7f04b00b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b00cc0d0 0x7f04b00b1090 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f04a000e8d0 tx=0x7f04a0008040 comp rx=0 tx=0).stop 2023-10-06T05:26:09.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 -- 172.21.15.31:0/4283685368 shutdown_connections 2023-10-06T05:26:09.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0494067830 0x7f0494069ce0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01595a0 0x7f04b01527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04b0068bb0 0x7f04b01560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 --2- 172.21.15.31:0/4283685368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04b00cc0d0 0x7f04b00b1090 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 -- 172.21.15.31:0/4283685368 >> 172.21.15.31:0/4283685368 conn(0x7f04b006b620 msgr2=0x7f04b00c3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:09.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 -- 172.21.15.31:0/4283685368 shutdown_connections 2023-10-06T05:26:09.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.714+0000 7f04b77ff700 1 -- 172.21.15.31:0/4283685368 wait complete. 2023-10-06T05:26:09.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:26:09.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.929+0000 7fd1f35ff700 1 Processor -- start 2023-10-06T05:26:09.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.929+0000 7fd1f35ff700 1 -- start start 2023-10-06T05:26:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.930+0000 7fd1f35ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec0cd510 0x7fd1ec0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.930+0000 7fd1f35ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd1ec15d2d0 0x7fd1ec0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.931+0000 7fd1f35ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0c3f40 con 0x7fd1ec159500 2023-10-06T05:26:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.931+0000 7fd1f35ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec0b2860 con 0x7fd1ec15d2d0 2023-10-06T05:26:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.931+0000 7fd1f35ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec15d7d0 con 0x7fd1ec0cd510 2023-10-06T05:26:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.931+0000 7fd1f1b9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd1ec15d2d0 0x7fd1ec0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.931+0000 7fd1f0b9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec0cd510 0x7fd1ec0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.931+0000 7fd1f1b9c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd1ec15d2d0 0x7fd1ec0aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:50786/0 (socket says 172.21.15.31:50786) 2023-10-06T05:26:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.931+0000 7fd1f0b9a700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec0cd510 0x7fd1ec0b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:43616/0 (socket says 172.21.15.31:43616) 2023-10-06T05:26:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.931+0000 7fd1f1b9c700 1 -- 172.21.15.31:0/3491131794 learned_addr learned my addr 172.21.15.31:0/3491131794 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.931+0000 7fd1f139b700 1 --1- 172.21.15.31:0/3491131794 >> v1:172.21.15.62:6789/0 conn(0x7fd1ec159500 0x7fd1ec0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:59116/0 (socket says 172.21.15.31:59116) 2023-10-06T05:26:09.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.932+0000 7fd1f0b9a700 1 -- 172.21.15.31:0/3491131794 >> v2:172.21.15.62:3300/0 conn(0x7fd1ec15d2d0 msgr2=0x7fd1ec0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:09.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.932+0000 7fd1f0b9a700 1 --2- 172.21.15.31:0/3491131794 >> v2:172.21.15.62:3300/0 conn(0x7fd1ec15d2d0 0x7fd1ec0aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.931+0000 7fd1e3fff700 1 -- 172.21.15.31:0/3491131794 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407296059 0 0) 0x7fd1ec0c3f40 con 0x7fd1ec159500 2023-10-06T05:26:09.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.932+0000 7fd1f0b9a700 1 -- 172.21.15.31:0/3491131794 >> v1:172.21.15.62:6789/0 conn(0x7fd1ec159500 legacy=0x7fd1ec0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.932+0000 7fd1f0b9a700 1 -- 172.21.15.31:0/3491131794 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0cb000 con 0x7fd1ec0cd510 2023-10-06T05:26:09.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.932+0000 7fd1f1b9c700 1 --2- 172.21.15.31:0/3491131794 >> v2:172.21.15.62:3300/0 conn(0x7fd1ec15d2d0 0x7fd1ec0aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:09.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.932+0000 7fd1f0b9a700 1 --2- 172.21.15.31:0/3491131794 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec0cd510 0x7fd1ec0b0ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0098d0 tx=0x7fd1dc009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1acec19c8cd769e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:09.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.932+0000 7fd1e3fff700 1 -- 172.21.15.31:0/3491131794 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc005c10 con 0x7fd1ec0cd510 2023-10-06T05:26:09.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.932+0000 7fd1e3fff700 1 -- 172.21.15.31:0/3491131794 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc005d70 con 0x7fd1ec0cd510 2023-10-06T05:26:09.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.932+0000 7fd1e3fff700 1 -- 172.21.15.31:0/3491131794 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc006280 con 0x7fd1ec0cd510 2023-10-06T05:26:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.933+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3491131794 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec0cd510 msgr2=0x7fd1ec0b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.933+0000 7fd1f35ff700 1 --2- 172.21.15.31:0/3491131794 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec0cd510 0x7fd1ec0b0ab0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0098d0 tx=0x7fd1dc009120 comp rx=0 tx=0).stop 2023-10-06T05:26:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.933+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3491131794 shutdown_connections 2023-10-06T05:26:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.933+0000 7fd1f35ff700 1 --2- 172.21.15.31:0/3491131794 >> v2:172.21.15.62:3300/0 conn(0x7fd1ec15d2d0 0x7fd1ec0aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.933+0000 7fd1f35ff700 1 --2- 172.21.15.31:0/3491131794 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec0cd510 0x7fd1ec0b0ab0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.933+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3491131794 >> 172.21.15.31:0/3491131794 conn(0x7fd1ec06ec90 msgr2=0x7fd1ec06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:09.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.933+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3491131794 shutdown_connections 2023-10-06T05:26:09.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.933+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3491131794 wait complete. 2023-10-06T05:26:09.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.935+0000 7fd1f35ff700 1 Processor -- start 2023-10-06T05:26:09.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.935+0000 7fd1f35ff700 1 -- start start 2023-10-06T05:26:09.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f35ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0cd510 0x7fd1ec16aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f35ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec159500 0x7fd1ec0b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f35ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ec15d2d0 0x7fd1ec0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f35ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0b61d0 con 0x7fd1ec0cd510 2023-10-06T05:26:09.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f35ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec16b110 con 0x7fd1ec15d2d0 2023-10-06T05:26:09.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f35ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0b3e40 con 0x7fd1ec159500 2023-10-06T05:26:09.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0cd510 0x7fd1ec16aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f139b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0cd510 0x7fd1ec16aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55258/0 (socket says 172.21.15.31:55258) 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f139b700 1 -- 172.21.15.31:0/3739750683 learned_addr learned my addr 172.21.15.31:0/3739750683 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f1b9c700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ec15d2d0 0x7fd1ec0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f0b9a700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec159500 0x7fd1ec0b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f139b700 1 -- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec159500 msgr2=0x7fd1ec0b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f139b700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec159500 0x7fd1ec0b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f139b700 1 -- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ec15d2d0 msgr2=0x7fd1ec0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f139b700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ec15d2d0 0x7fd1ec0d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.936+0000 7fd1f139b700 1 -- 172.21.15.31:0/3739750683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e4009ae0 con 0x7fd1ec0cd510 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.937+0000 7fd1f139b700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0cd510 0x7fd1ec16aa00 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80011e0 tx=0x7fd1e8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.937+0000 7fd1e17fa700 1 -- 172.21.15.31:0/3739750683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8007300 con 0x7fd1ec0cd510 2023-10-06T05:26:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.937+0000 7fd1e17fa700 1 -- 172.21.15.31:0/3739750683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e8007460 con 0x7fd1ec0cd510 2023-10-06T05:26:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.937+0000 7fd1e17fa700 1 -- 172.21.15.31:0/3739750683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8004830 con 0x7fd1ec0cd510 2023-10-06T05:26:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.937+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3739750683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc0093c0 con 0x7fd1ec0cd510 2023-10-06T05:26:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.937+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3739750683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec0b5f40 con 0x7fd1ec0cd510 2023-10-06T05:26:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.938+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3739750683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec16b2d0 con 0x7fd1ec0cd510 2023-10-06T05:26:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.939+0000 7fd1e17fa700 1 -- 172.21.15.31:0/3739750683 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fd1e8004be0 con 0x7fd1ec0cd510 2023-10-06T05:26:09.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.940+0000 7fd1e17fa700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd1d0067830 0x7fd1d0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:09.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.940+0000 7fd1e17fa700 1 -- 172.21.15.31:0/3739750683 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd1e80823d0 con 0x7fd1ec0cd510 2023-10-06T05:26:09.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.941+0000 7fd1f0b9a700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd1d0067830 0x7fd1d0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.941+0000 7fd1f0b9a700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd1d0067830 0x7fd1d0069ce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009160 tx=0x7fd1dc016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:09.944+0000 7fd1e17fa700 1 -- 172.21.15.31:0/3739750683 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1e8051090 con 0x7fd1ec0cd510 2023-10-06T05:26:10.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.195+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3739750683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd1ec04c380 con 0x7fd1ec0cd510 2023-10-06T05:26:10.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.195+0000 7fd1e17fa700 1 -- 172.21.15.31:0/3739750683 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd1e8018070 con 0x7fd1ec0cd510 2023-10-06T05:26:10.197 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:26:10.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.197+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd1d0067830 msgr2=0x7fd1d0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:10.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.197+0000 7fd1f35ff700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd1d0067830 0x7fd1d0069ce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009160 tx=0x7fd1dc016000 comp rx=0 tx=0).stop 2023-10-06T05:26:10.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.198+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3739750683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0cd510 msgr2=0x7fd1ec16aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:10.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.198+0000 7fd1f35ff700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0cd510 0x7fd1ec16aa00 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80011e0 tx=0x7fd1e8007e10 comp rx=0 tx=0).stop 2023-10-06T05:26:10.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.198+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3739750683 shutdown_connections 2023-10-06T05:26:10.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.198+0000 7fd1f35ff700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd1d0067830 0x7fd1d0069ce0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:10.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.198+0000 7fd1f35ff700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ec15d2d0 0x7fd1ec0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:10.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.198+0000 7fd1f35ff700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0cd510 0x7fd1ec16aa00 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:10.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.198+0000 7fd1f35ff700 1 --2- 172.21.15.31:0/3739750683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ec159500 0x7fd1ec0b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:10.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.198+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3739750683 >> 172.21.15.31:0/3739750683 conn(0x7fd1ec06ec90 msgr2=0x7fd1ec156320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:10.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.198+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3739750683 shutdown_connections 2023-10-06T05:26:10.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:10.198+0000 7fd1f35ff700 1 -- 172.21.15.31:0/3739750683 wait complete. 2023-10-06T05:26:10.216 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:26:10.217 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:26:10.217 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-10-06T05:26:10.217 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:26:10.217 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:10.217 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:26:10.217 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:26:10.217 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:26:10.217 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:10.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:10.218 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-10-06T05:26:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:26:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:26:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:26:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-10-06T05:26:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41744sec preferred_lft 41744sec 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::80b2:e2ff:fec8:5d50/64 scope link 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 1a:ea:47:02:1d:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:26:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::18ea:47ff:fe02:1d9d/64 scope link 2023-10-06T05:26:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:3b:7f:57:67:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:26:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c3b:7fff:fe57:6789/64 scope link 2023-10-06T05:26:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:10.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:10.268 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:26:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-10-06T05:26:10.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:10.377 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-10-06T05:26:10.427 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-10-06T05:26:10.427 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:26:10.427 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:26:10.428 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:26:10.428 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:26:10.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:10.429 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-10-06T05:26:10.429 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-10-06T05:26:10.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:26:10.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:26:10.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:26:10.429 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-10-06T05:26:10.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:10.603 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:26:10.649 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:26:10.649 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:26:10.650 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:10.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:10.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-10-06T05:26:10.722 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:10.722 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:10.769 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:26:10.769 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:10.769 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:10.855 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T05:26:10.855 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:26:10.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:26:10.956 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:26:10.956 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:26:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:26:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:26:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:26:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:26:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:26:10.987 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:26:10.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:10.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:11.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:26:11.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.254+0000 7f2b985ff700 1 Processor -- start 2023-10-06T05:26:11.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.254+0000 7f2b985ff700 1 -- start start 2023-10-06T05:26:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.255+0000 7f2b985ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2b900c7f30 0x7f2b90069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.255+0000 7f2b985ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b900ca5b0 con 0x7f2b900aee70 2023-10-06T05:26:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.255+0000 7f2b985ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b900af2f0 con 0x7f2b900c7b60 2023-10-06T05:26:11.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.255+0000 7f2b985ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9006f740 con 0x7f2b900c7f30 2023-10-06T05:26:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.255+0000 7f2b95b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2b900c7f30 0x7f2b90069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.255+0000 7f2b95b9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2b900c7f30 0x7f2b90069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55268/0 (socket says 172.21.15.31:55268) 2023-10-06T05:26:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.255+0000 7f2b95b9a700 1 -- 172.21.15.31:0/2539996204 learned_addr learned my addr 172.21.15.31:0/2539996204 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.256+0000 7f2b95b9a700 1 -- 172.21.15.31:0/2539996204 >> v1:172.21.15.62:6789/0 conn(0x7f2b900c7b60 legacy=0x7f2b9006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.256+0000 7f2b95b9a700 1 -- 172.21.15.31:0/2539996204 >> v1:172.21.15.31:6789/0 conn(0x7f2b900aee70 legacy=0x7f2b9013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.256+0000 7f2b95b9a700 1 -- 172.21.15.31:0/2539996204 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b900c9c80 con 0x7f2b900c7f30 2023-10-06T05:26:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.256+0000 7f2b95b9a700 1 --2- 172.21.15.31:0/2539996204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900c7f30 0x7f2b90069880 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f2b800099b0 tx=0x7f2b80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cf10b15968c087 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.256+0000 7f2b95399700 1 -- 172.21.15.31:0/2539996204 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8000b760 con 0x7f2b900c7f30 2023-10-06T05:26:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.256+0000 7f2b95399700 1 -- 172.21.15.31:0/2539996204 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8000b8c0 con 0x7f2b900c7f30 2023-10-06T05:26:11.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.256+0000 7f2b95399700 1 -- 172.21.15.31:0/2539996204 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8000bc40 con 0x7f2b900c7f30 2023-10-06T05:26:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.257+0000 7f2b985ff700 1 -- 172.21.15.31:0/2539996204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900c7f30 msgr2=0x7f2b90069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.257+0000 7f2b985ff700 1 --2- 172.21.15.31:0/2539996204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900c7f30 0x7f2b90069880 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f2b800099b0 tx=0x7f2b80009250 comp rx=0 tx=0).stop 2023-10-06T05:26:11.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.257+0000 7f2b985ff700 1 -- 172.21.15.31:0/2539996204 shutdown_connections 2023-10-06T05:26:11.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.257+0000 7f2b985ff700 1 --2- 172.21.15.31:0/2539996204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900c7f30 0x7f2b90069880 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.257+0000 7f2b985ff700 1 -- 172.21.15.31:0/2539996204 >> 172.21.15.31:0/2539996204 conn(0x7f2b900cc0b0 msgr2=0x7f2b900cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:11.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.257+0000 7f2b985ff700 1 -- 172.21.15.31:0/2539996204 shutdown_connections 2023-10-06T05:26:11.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.257+0000 7f2b985ff700 1 -- 172.21.15.31:0/2539996204 wait complete. 2023-10-06T05:26:11.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.259+0000 7f2b985ff700 1 Processor -- start 2023-10-06T05:26:11.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.259+0000 7f2b985ff700 1 -- start start 2023-10-06T05:26:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b985ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900aee70 0x7f2b900c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b985ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b900c7b60 0x7f2b90169dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b985ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b900c7f30 0x7f2b90168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b985ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900c63f0 con 0x7f2b900aee70 2023-10-06T05:26:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b985ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900c5840 con 0x7f2b900c7f30 2023-10-06T05:26:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b985ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900c5ac0 con 0x7f2b900c7b60 2023-10-06T05:26:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b9639b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900aee70 0x7f2b900c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b9639b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900aee70 0x7f2b900c3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55272/0 (socket says 172.21.15.31:55272) 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b9639b700 1 -- 172.21.15.31:0/1960880089 learned_addr learned my addr 172.21.15.31:0/1960880089 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b95b9a700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b900c7b60 0x7f2b90169dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b96b9c700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b900c7f30 0x7f2b90168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b9639b700 1 -- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b900c7b60 msgr2=0x7f2b90169dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b9639b700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b900c7b60 0x7f2b90169dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b9639b700 1 -- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b900c7f30 msgr2=0x7f2b90168fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b9639b700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b900c7f30 0x7f2b90168fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.260+0000 7f2b9639b700 1 -- 172.21.15.31:0/1960880089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b800094f0 con 0x7f2b900aee70 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.261+0000 7f2b9639b700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900aee70 0x7f2b900c3160 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00c5b0 tx=0x7f2b8c0020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.261+0000 7f2b867fc700 1 -- 172.21.15.31:0/1960880089 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c011590 con 0x7f2b900aee70 2023-10-06T05:26:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.261+0000 7f2b867fc700 1 -- 172.21.15.31:0/1960880089 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c0116f0 con 0x7f2b900aee70 2023-10-06T05:26:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.261+0000 7f2b867fc700 1 -- 172.21.15.31:0/1960880089 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c011a90 con 0x7f2b900aee70 2023-10-06T05:26:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.261+0000 7f2b985ff700 1 -- 172.21.15.31:0/1960880089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b900c5da0 con 0x7f2b900aee70 2023-10-06T05:26:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.261+0000 7f2b985ff700 1 -- 172.21.15.31:0/1960880089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b900c36a0 con 0x7f2b900aee70 2023-10-06T05:26:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.262+0000 7f2b985ff700 1 -- 172.21.15.31:0/1960880089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9004c380 con 0x7f2b900aee70 2023-10-06T05:26:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.263+0000 7f2b867fc700 1 -- 172.21.15.31:0/1960880089 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f2b8c012420 con 0x7f2b900aee70 2023-10-06T05:26:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.264+0000 7f2b867fc700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2b74067830 0x7f2b74069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.264+0000 7f2b867fc700 1 -- 172.21.15.31:0/1960880089 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f2b8c082940 con 0x7f2b900aee70 2023-10-06T05:26:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.264+0000 7f2b95b9a700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2b74067830 0x7f2b74069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.265+0000 7f2b95b9a700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2b74067830 0x7f2b74069ce0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2b90069dc0 tx=0x7f2b80016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.268+0000 7f2b867fc700 1 -- 172.21.15.31:0/1960880089 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b8c051580 con 0x7f2b900aee70 2023-10-06T05:26:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.513+0000 7f2b985ff700 1 -- 172.21.15.31:0/1960880089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2b900c9f60 con 0x7f2b900aee70 2023-10-06T05:26:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.513+0000 7f2b867fc700 1 -- 172.21.15.31:0/1960880089 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2b8c054ba0 con 0x7f2b900aee70 2023-10-06T05:26:11.515 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:26:11.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 -- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2b74067830 msgr2=0x7f2b74069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:11.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2b74067830 0x7f2b74069ce0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2b90069dc0 tx=0x7f2b80016000 comp rx=0 tx=0).stop 2023-10-06T05:26:11.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 -- 172.21.15.31:0/1960880089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900aee70 msgr2=0x7f2b900c3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:11.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900aee70 0x7f2b900c3160 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00c5b0 tx=0x7f2b8c0020e0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 -- 172.21.15.31:0/1960880089 shutdown_connections 2023-10-06T05:26:11.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2b74067830 0x7f2b74069ce0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b900c7f30 0x7f2b90168fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b900aee70 0x7f2b900c3160 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 --2- 172.21.15.31:0/1960880089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b900c7b60 0x7f2b90169dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:11.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 -- 172.21.15.31:0/1960880089 >> 172.21.15.31:0/1960880089 conn(0x7f2b900cc0b0 msgr2=0x7f2b900c1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:11.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.515+0000 7f2b985ff700 1 -- 172.21.15.31:0/1960880089 shutdown_connections 2023-10-06T05:26:11.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:11.516+0000 7f2b985ff700 1 -- 172.21.15.31:0/1960880089 wait complete. 2023-10-06T05:26:11.535 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:26:11.535 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:26:11.536 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-10-06T05:26:11.536 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T05:26:11.536 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T05:26:11.536 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:26:11.536 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:26:11.536 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:26:11.536 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:26:11.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:11.536 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-10-06T05:26:11.589 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:26:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:26:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:26:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-10-06T05:26:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:11.590 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-10-06T05:26:11.591 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-10-06T05:26:11.591 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:26:11.591 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:11.592 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:26:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T05:26:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41743sec preferred_lft 41743sec 2023-10-06T05:26:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2023-10-06T05:26:11.592 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:11.592 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:26:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:26:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:26:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::60dc:b7ff:fe37:9f7a/64 scope link 2023-10-06T05:26:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:11.593 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:11.593 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether aa:56:98:af:33:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:26:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::a856:98ff:feaf:33c2/64 scope link 2023-10-06T05:26:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:11.594 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:26:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 42:be:df:f4:b6:41 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:26:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::40be:dfff:fef4:b641/64 scope link 2023-10-06T05:26:11.594 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:26:11.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:11.595 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:26:11.667 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-10-06T05:26:11.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:11.671 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-10-06T05:26:11.721 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-10-06T05:26:11.722 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:26:11.722 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T05:26:11.722 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T05:26:11.722 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:26:11.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:11.722 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-10-06T05:26:11.722 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-10-06T05:26:11.722 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T05:26:11.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T05:26:11.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T05:26:11.723 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-10-06T05:26:11.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:11.894 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:26:11.940 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:26:11.940 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:26:11.941 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:26:11.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:11.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-10-06T05:26:12.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:12.013 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T05:26:12.059 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T05:26:12.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:26:12.059 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:26:12.144 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T05:26:12.144 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:26:12.144 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:26:12.285 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:26:12.285 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:26:12.312 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T05:26:12.312 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T05:26:12.312 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.62:3300,172.21.15.62:3301" 2023-10-06T05:26:12.312 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 1@86bc8148-d401-41f1-bd7f-edc6b2417154.cephfs=/ 2023-10-06T05:26:12.312 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.62:3300/172.21.15.62:3301" will pass to kernel 2023-10-06T05:26:12.317 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:26:12.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:12.317 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T05:26:12.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T05:26:12.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.597+0000 7ff3ab1ff700 1 Processor -- start 2023-10-06T05:26:12.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.597+0000 7ff3ab1ff700 1 -- start start 2023-10-06T05:26:12.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.598+0000 7ff3ab1ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3a4153950 0x7ff3a4153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.598+0000 7ff3ab1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 0x7ff3a415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.598+0000 7ff3ab1ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a415bb20 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.598+0000 7ff3ab1ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a415af10 con 0x7ff3a4153950 2023-10-06T05:26:12.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.598+0000 7ff3ab1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a415b1f0 con 0x7ff3a415d920 2023-10-06T05:26:12.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff3a40cd5d0 0x7ff3a415d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:40858/0 (socket says 172.21.15.31:40858) 2023-10-06T05:26:12.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a3fff700 1 -- 172.21.15.31:0/3332889817 learned_addr learned my addr 172.21.15.31:0/3332889817 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:12.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a979c700 1 --2- 172.21.15.31:0/3332889817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 0x7ff3a415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a8f9b700 1 --2- 172.21.15.31:0/3332889817 >> v2:172.21.15.62:3300/0 conn(0x7ff3a4153950 0x7ff3a4153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a37fe700 1 -- 172.21.15.31:0/3332889817 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701468758 0 0) 0x7ff3a415bb20 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a37fe700 1 -- 172.21.15.31:0/3332889817 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff394003570 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a979c700 1 -- 172.21.15.31:0/3332889817 >> v2:172.21.15.62:3300/0 conn(0x7ff3a4153950 msgr2=0x7ff3a4153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:12.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a979c700 1 --2- 172.21.15.31:0/3332889817 >> v2:172.21.15.62:3300/0 conn(0x7ff3a4153950 0x7ff3a4153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a979c700 1 -- 172.21.15.31:0/3332889817 >> v1:172.21.15.31:6789/0 conn(0x7ff3a40cd5d0 legacy=0x7ff3a415d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a979c700 1 -- 172.21.15.31:0/3332889817 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a415b4d0 con 0x7ff3a415d920 2023-10-06T05:26:12.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.599+0000 7ff3a8f9b700 1 --2- 172.21.15.31:0/3332889817 >> v2:172.21.15.62:3300/0 conn(0x7ff3a4153950 0x7ff3a4153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:12.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.600+0000 7ff3a979c700 1 --2- 172.21.15.31:0/3332889817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 0x7ff3a415bfb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009e60 tx=0x7ff39c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f7e9f61c62b9710 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:12.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.600+0000 7ff3a37fe700 1 -- 172.21.15.31:0/3332889817 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00e070 con 0x7ff3a415d920 2023-10-06T05:26:12.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.600+0000 7ff3a37fe700 1 -- 172.21.15.31:0/3332889817 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c00bab0 con 0x7ff3a415d920 2023-10-06T05:26:12.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.600+0000 7ff3a37fe700 1 -- 172.21.15.31:0/3332889817 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00be70 con 0x7ff3a415d920 2023-10-06T05:26:12.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.600+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/3332889817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 msgr2=0x7ff3a415bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.600+0000 7ff3ab1ff700 1 --2- 172.21.15.31:0/3332889817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 0x7ff3a415bfb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009e60 tx=0x7ff39c009780 comp rx=0 tx=0).stop 2023-10-06T05:26:12.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.600+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/3332889817 shutdown_connections 2023-10-06T05:26:12.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.600+0000 7ff3ab1ff700 1 --2- 172.21.15.31:0/3332889817 >> v2:172.21.15.62:3300/0 conn(0x7ff3a4153950 0x7ff3a4153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.600+0000 7ff3ab1ff700 1 --2- 172.21.15.31:0/3332889817 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 0x7ff3a415bfb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.600+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/3332889817 >> 172.21.15.31:0/3332889817 conn(0x7ff3a40681b0 msgr2=0x7ff3a40685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:12.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.601+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/3332889817 shutdown_connections 2023-10-06T05:26:12.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.601+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/3332889817 wait complete. 2023-10-06T05:26:12.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.602+0000 7ff3ab1ff700 1 Processor -- start 2023-10-06T05:26:12.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.602+0000 7ff3ab1ff700 1 -- start start 2023-10-06T05:26:12.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3ab1ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3a40cd5d0 0x7ff3a4151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3ab1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4153950 0x7ff3a40c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3ab1ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 0x7ff3a40c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3ab1ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40ca4b0 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3ab1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40ca020 con 0x7ff3a4153950 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3ab1ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c9410 con 0x7ff3a415d920 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3a8f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3a40cd5d0 0x7ff3a4151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3a8f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3a40cd5d0 0x7ff3a4151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55274/0 (socket says 172.21.15.31:55274) 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3a8f9b700 1 -- 172.21.15.31:0/651058726 learned_addr learned my addr 172.21.15.31:0/651058726 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3a979c700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 0x7ff3a40c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.603+0000 7ff3a3fff700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4153950 0x7ff3a40c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3a8f9b700 1 -- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 msgr2=0x7ff3a40c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3a8f9b700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 0x7ff3a40c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3a8f9b700 1 -- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4153950 msgr2=0x7ff3a40c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3a8f9b700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4153950 0x7ff3a40c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3a8f9b700 1 -- 172.21.15.31:0/651058726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3980093c0 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3a8f9b700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3a40cd5d0 0x7ff3a4151400 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7ff39800ec80 tx=0x7ff398009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:12.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3a0ff9700 1 -- 172.21.15.31:0/651058726 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398005a00 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3a0ff9700 1 -- 172.21.15.31:0/651058726 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff398005b60 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3a0ff9700 1 -- 172.21.15.31:0/651058726 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39800c9d0 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/651058726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39c009a20 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.604+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/651058726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a40c99d0 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.605+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/651058726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a40ca300 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.606+0000 7ff3a0ff9700 1 -- 172.21.15.31:0/651058726 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7ff39800cd80 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.607+0000 7ff3a0ff9700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff388067830 0x7ff388069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:12.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.608+0000 7ff3a0ff9700 1 -- 172.21.15.31:0/651058726 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff3980836b0 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.608+0000 7ff3a3fff700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff388067830 0x7ff388069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:12.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.608+0000 7ff3a3fff700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff388067830 0x7ff388069ce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40d1420 tx=0x7ff390009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:12.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.612+0000 7ff3a0ff9700 1 -- 172.21.15.31:0/651058726 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff398052100 con 0x7ff3a40cd5d0 2023-10-06T05:26:12.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:12.861+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/651058726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff3a404c3f0 con 0x7ff3a40cd5d0 2023-10-06T05:26:13.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.170+0000 7ff3a0ff9700 1 -- 172.21.15.31:0/651058726 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v113) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff3a404c3f0 con 0x7ff3a40cd5d0 2023-10-06T05:26:13.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.172+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff388067830 msgr2=0x7ff388069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.172+0000 7ff3ab1ff700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff388067830 0x7ff388069ce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40d1420 tx=0x7ff390009040 comp rx=0 tx=0).stop 2023-10-06T05:26:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.172+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/651058726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3a40cd5d0 msgr2=0x7ff3a4151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.172+0000 7ff3ab1ff700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3a40cd5d0 0x7ff3a4151400 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7ff39800ec80 tx=0x7ff398009120 comp rx=0 tx=0).stop 2023-10-06T05:26:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.172+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/651058726 shutdown_connections 2023-10-06T05:26:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.172+0000 7ff3ab1ff700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff388067830 0x7ff388069ce0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.172+0000 7ff3ab1ff700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3a415d920 0x7ff3a40c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.172+0000 7ff3ab1ff700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3a40cd5d0 0x7ff3a4151400 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.172+0000 7ff3ab1ff700 1 --2- 172.21.15.31:0/651058726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3a4153950 0x7ff3a40c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.172+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/651058726 >> 172.21.15.31:0/651058726 conn(0x7ff3a40681b0 msgr2=0x7ff3a41522d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:13.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.173+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/651058726 shutdown_connections 2023-10-06T05:26:13.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.173+0000 7ff3ab1ff700 1 -- 172.21.15.31:0/651058726 wait complete. 2023-10-06T05:26:13.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:26:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.389+0000 7ff3c69ff700 1 Processor -- start 2023-10-06T05:26:13.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.390+0000 7ff3c69ff700 1 -- start start 2023-10-06T05:26:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.391+0000 7ff3c69ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3c0159500 0x7ff3c00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.391+0000 7ff3c69ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c00cd510 0x7ff3c00aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.391+0000 7ff3c69ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c00c3f40 con 0x7ff3c015d2d0 2023-10-06T05:26:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.391+0000 7ff3c69ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c00b2860 con 0x7ff3c0159500 2023-10-06T05:26:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.391+0000 7ff3c69ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c015d7d0 con 0x7ff3c00cd510 2023-10-06T05:26:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.391+0000 7ff3bffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3c0159500 0x7ff3c00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.391+0000 7ff3bffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3c0159500 0x7ff3c00b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55286/0 (socket says 172.21.15.31:55286) 2023-10-06T05:26:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.391+0000 7ff3bffff700 1 -- 172.21.15.31:0/1731967153 learned_addr learned my addr 172.21.15.31:0/1731967153 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.391+0000 7ff3c4f9c700 1 --2- 172.21.15.31:0/1731967153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c00cd510 0x7ff3c00aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.392+0000 7ff3bffff700 1 -- 172.21.15.31:0/1731967153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c00cd510 msgr2=0x7ff3c00aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.392+0000 7ff3bffff700 1 --2- 172.21.15.31:0/1731967153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c00cd510 0x7ff3c00aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.392+0000 7ff3bffff700 1 -- 172.21.15.31:0/1731967153 >> v1:172.21.15.62:6789/0 conn(0x7ff3c015d2d0 legacy=0x7ff3c00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.392+0000 7ff3bffff700 1 -- 172.21.15.31:0/1731967153 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c00cb000 con 0x7ff3c0159500 2023-10-06T05:26:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.392+0000 7ff3bffff700 1 --2- 172.21.15.31:0/1731967153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0159500 0x7ff3c00b0ab0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00099b0 tx=0x7ff3b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37058054b843534f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.392+0000 7ff3beffd700 1 -- 172.21.15.31:0/1731967153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0009ea0 con 0x7ff3c0159500 2023-10-06T05:26:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.392+0000 7ff3beffd700 1 -- 172.21.15.31:0/1731967153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b000b860 con 0x7ff3c0159500 2023-10-06T05:26:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.392+0000 7ff3beffd700 1 -- 172.21.15.31:0/1731967153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b000bc00 con 0x7ff3c0159500 2023-10-06T05:26:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.393+0000 7ff3c69ff700 1 -- 172.21.15.31:0/1731967153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0159500 msgr2=0x7ff3c00b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.393+0000 7ff3c69ff700 1 --2- 172.21.15.31:0/1731967153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0159500 0x7ff3c00b0ab0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00099b0 tx=0x7ff3b0009250 comp rx=0 tx=0).stop 2023-10-06T05:26:13.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.393+0000 7ff3c69ff700 1 -- 172.21.15.31:0/1731967153 shutdown_connections 2023-10-06T05:26:13.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.393+0000 7ff3c69ff700 1 --2- 172.21.15.31:0/1731967153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c00cd510 0x7ff3c00aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.393+0000 7ff3c69ff700 1 --2- 172.21.15.31:0/1731967153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0159500 0x7ff3c00b0ab0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.393+0000 7ff3c69ff700 1 -- 172.21.15.31:0/1731967153 >> 172.21.15.31:0/1731967153 conn(0x7ff3c006ec90 msgr2=0x7ff3c006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:13.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.393+0000 7ff3c69ff700 1 -- 172.21.15.31:0/1731967153 shutdown_connections 2023-10-06T05:26:13.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.393+0000 7ff3c69ff700 1 -- 172.21.15.31:0/1731967153 wait complete. 2023-10-06T05:26:13.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.394+0000 7ff3c69ff700 1 Processor -- start 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.394+0000 7ff3c69ff700 1 -- start start 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.395+0000 7ff3c69ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd510 0x7ff3c00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.395+0000 7ff3c69ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0159500 0x7ff3c00b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.395+0000 7ff3c69ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c015d2d0 0x7ff3c00b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.395+0000 7ff3c69ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c016a3f0 con 0x7ff3c015d2d0 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.395+0000 7ff3c69ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c016a6d0 con 0x7ff3c00cd510 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.395+0000 7ff3c69ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c016a9b0 con 0x7ff3c0159500 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3c4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c015d2d0 0x7ff3c00b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3c4f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c015d2d0 0x7ff3c00b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55290/0 (socket says 172.21.15.31:55290) 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3c4f9c700 1 -- 172.21.15.31:0/420533971 learned_addr learned my addr 172.21.15.31:0/420533971 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3bffff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd510 0x7ff3c00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3bf7fe700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0159500 0x7ff3c00b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3c4f9c700 1 -- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0159500 msgr2=0x7ff3c00b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3c4f9c700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0159500 0x7ff3c00b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3c4f9c700 1 -- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd510 msgr2=0x7ff3c00ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3c4f9c700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd510 0x7ff3c00ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3c4f9c700 1 -- 172.21.15.31:0/420533971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b00094f0 con 0x7ff3c015d2d0 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3bffff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd510 0x7ff3c00ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3bf7fe700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0159500 0x7ff3c00b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.396+0000 7ff3c4f9c700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c015d2d0 0x7ff3c00b37d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009e90 tx=0x7ff3b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.397+0000 7ff3a7fff700 1 -- 172.21.15.31:0/420533971 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b800cc20 con 0x7ff3c015d2d0 2023-10-06T05:26:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.397+0000 7ff3a7fff700 1 -- 172.21.15.31:0/420533971 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b800f040 con 0x7ff3c015d2d0 2023-10-06T05:26:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.397+0000 7ff3c69ff700 1 -- 172.21.15.31:0/420533971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c016ac90 con 0x7ff3c015d2d0 2023-10-06T05:26:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.397+0000 7ff3a7fff700 1 -- 172.21.15.31:0/420533971 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8014480 con 0x7ff3c015d2d0 2023-10-06T05:26:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.397+0000 7ff3c69ff700 1 -- 172.21.15.31:0/420533971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c0169630 con 0x7ff3c015d2d0 2023-10-06T05:26:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.398+0000 7ff3c69ff700 1 -- 172.21.15.31:0/420533971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c004c380 con 0x7ff3c015d2d0 2023-10-06T05:26:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.399+0000 7ff3a7fff700 1 -- 172.21.15.31:0/420533971 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7ff3b8014830 con 0x7ff3c015d2d0 2023-10-06T05:26:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.400+0000 7ff3a7fff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3ac067830 0x7ff3ac069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.400+0000 7ff3a7fff700 1 -- 172.21.15.31:0/420533971 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ff3b8083850 con 0x7ff3c015d2d0 2023-10-06T05:26:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.401+0000 7ff3bffff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3ac067830 0x7ff3ac069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.401+0000 7ff3bffff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3ac067830 0x7ff3ac069ce0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00099b0 tx=0x7ff3b0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.405+0000 7ff3a7fff700 1 -- 172.21.15.31:0/420533971 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3b8055f80 con 0x7ff3c015d2d0 2023-10-06T05:26:13.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.635+0000 7ff3c69ff700 1 -- 172.21.15.31:0/420533971 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3c016a840 con 0x7ff3ac067830 2023-10-06T05:26:13.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.637+0000 7ff3a7fff700 1 -- 172.21.15.31:0/420533971 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff3c016a840 con 0x7ff3ac067830 2023-10-06T05:26:13.639 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-10-06T05:26:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-10-06T05:26:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-10-06T05:26:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-10-06T05:26:13.640 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-10-06T05:26:13.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 -- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3ac067830 msgr2=0x7ff3ac069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3ac067830 0x7ff3ac069ce0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00099b0 tx=0x7ff3b0017000 comp rx=0 tx=0).stop 2023-10-06T05:26:13.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 -- 172.21.15.31:0/420533971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c015d2d0 msgr2=0x7ff3c00b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c015d2d0 0x7ff3c00b37d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009e90 tx=0x7ff3b800b040 comp rx=0 tx=0).stop 2023-10-06T05:26:13.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 -- 172.21.15.31:0/420533971 shutdown_connections 2023-10-06T05:26:13.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff3ac067830 0x7ff3ac069ce0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c015d2d0 0x7ff3c00b37d0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3c00cd510 0x7ff3c00ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 --2- 172.21.15.31:0/420533971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3c0159500 0x7ff3c00b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 -- 172.21.15.31:0/420533971 >> 172.21.15.31:0/420533971 conn(0x7ff3c006ec90 msgr2=0x7ff3c0157ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:13.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 -- 172.21.15.31:0/420533971 shutdown_connections 2023-10-06T05:26:13.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.640+0000 7ff3c69ff700 1 -- 172.21.15.31:0/420533971 wait complete. 2023-10-06T05:26:13.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:26:13.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.856+0000 7fac35bff700 1 Processor -- start 2023-10-06T05:26:13.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.856+0000 7fac35bff700 1 -- start start 2023-10-06T05:26:13.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.857+0000 7fac35bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac30159560 0x7fac300cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.857+0000 7fac35bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac3015d2d0 con 0x7fac30159930 2023-10-06T05:26:13.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.857+0000 7fac35bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac3015d5b0 con 0x7fac300b2b80 2023-10-06T05:26:13.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.857+0000 7fac35bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3015d890 con 0x7fac30159560 2023-10-06T05:26:13.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.857+0000 7fac2f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac30159560 0x7fac300cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.857+0000 7fac2f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac30159560 0x7fac300cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55292/0 (socket says 172.21.15.31:55292) 2023-10-06T05:26:13.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.858+0000 7fac2f7fe700 1 -- 172.21.15.31:0/1593446895 learned_addr learned my addr 172.21.15.31:0/1593446895 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:13.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.857+0000 7fac2effd700 1 --1- 172.21.15.31:0/1593446895 >> v1:172.21.15.31:6789/0 conn(0x7fac30159930 0x7fac300b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:40864/0 (socket says 172.21.15.31:40864) 2023-10-06T05:26:13.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.858+0000 7fac2f7fe700 1 -- 172.21.15.31:0/1593446895 >> v1:172.21.15.62:6789/0 conn(0x7fac300b2b80 legacy=0x7fac300cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.858+0000 7fac2f7fe700 1 -- 172.21.15.31:0/1593446895 >> v1:172.21.15.31:6789/0 conn(0x7fac30159930 legacy=0x7fac300b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.858+0000 7fac2f7fe700 1 -- 172.21.15.31:0/1593446895 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3015db70 con 0x7fac30159560 2023-10-06T05:26:13.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.858+0000 7fac2e7fc700 1 -- 172.21.15.31:0/1593446895 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133047330 0 0) 0x7fac3015d2d0 con 0x7fac30159930 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.858+0000 7fac2f7fe700 1 --2- 172.21.15.31:0/1593446895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 0x7fac300cce70 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fac200099b0 tx=0x7fac20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac0a1bfee77c46f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.858+0000 7fac2e7fc700 1 -- 172.21.15.31:0/1593446895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000b860 con 0x7fac30159560 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.858+0000 7fac2e7fc700 1 -- 172.21.15.31:0/1593446895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2000b9c0 con 0x7fac30159560 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.858+0000 7fac2e7fc700 1 -- 172.21.15.31:0/1593446895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000bd60 con 0x7fac30159560 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.859+0000 7fac35bff700 1 -- 172.21.15.31:0/1593446895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 msgr2=0x7fac300cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.859+0000 7fac35bff700 1 --2- 172.21.15.31:0/1593446895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 0x7fac300cce70 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fac200099b0 tx=0x7fac20009250 comp rx=0 tx=0).stop 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.859+0000 7fac35bff700 1 -- 172.21.15.31:0/1593446895 shutdown_connections 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.859+0000 7fac35bff700 1 --2- 172.21.15.31:0/1593446895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 0x7fac300cce70 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.859+0000 7fac35bff700 1 -- 172.21.15.31:0/1593446895 >> 172.21.15.31:0/1593446895 conn(0x7fac3006ec90 msgr2=0x7fac3006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.859+0000 7fac35bff700 1 -- 172.21.15.31:0/1593446895 shutdown_connections 2023-10-06T05:26:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.859+0000 7fac35bff700 1 -- 172.21.15.31:0/1593446895 wait complete. 2023-10-06T05:26:13.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.860+0000 7fac35bff700 1 Processor -- start 2023-10-06T05:26:13.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.861+0000 7fac35bff700 1 -- start start 2023-10-06T05:26:13.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.861+0000 7fac35bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac300b2b80 0x7fac300c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac35bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 0x7fac300c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac35bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac30159930 0x7fac300c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac35bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3016d260 con 0x7fac30159560 2023-10-06T05:26:13.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac35bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300c7280 con 0x7fac300b2b80 2023-10-06T05:26:13.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac35bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac3016dac0 con 0x7fac30159930 2023-10-06T05:26:13.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 0x7fac300c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 0x7fac300c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55300/0 (socket says 172.21.15.31:55300) 2023-10-06T05:26:13.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2effd700 1 -- 172.21.15.31:0/2545824121 learned_addr learned my addr 172.21.15.31:0/2545824121 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:13.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2ffff700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac30159930 0x7fac300c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2f7fe700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac300b2b80 0x7fac300c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2effd700 1 -- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac30159930 msgr2=0x7fac300c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2effd700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac30159930 0x7fac300c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2effd700 1 -- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac300b2b80 msgr2=0x7fac300c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2effd700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac300b2b80 0x7fac300c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2effd700 1 -- 172.21.15.31:0/2545824121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200094f0 con 0x7fac30159560 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.862+0000 7fac2f7fe700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac300b2b80 0x7fac300c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.863+0000 7fac2effd700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 0x7fac300c6d40 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fac2400c560 tx=0x7fac24007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.863+0000 7fac177fe700 1 -- 172.21.15.31:0/2545824121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24005a70 con 0x7fac30159560 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.863+0000 7fac177fe700 1 -- 172.21.15.31:0/2545824121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac24005bd0 con 0x7fac30159560 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.863+0000 7fac35bff700 1 -- 172.21.15.31:0/2545824121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac300c4560 con 0x7fac30159560 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.863+0000 7fac177fe700 1 -- 172.21.15.31:0/2545824121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac240070d0 con 0x7fac30159560 2023-10-06T05:26:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.863+0000 7fac35bff700 1 -- 172.21.15.31:0/2545824121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3016d750 con 0x7fac30159560 2023-10-06T05:26:13.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.864+0000 7fac35bff700 1 -- 172.21.15.31:0/2545824121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3016dc50 con 0x7fac30159560 2023-10-06T05:26:13.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.865+0000 7fac177fe700 1 -- 172.21.15.31:0/2545824121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7fac240074c0 con 0x7fac30159560 2023-10-06T05:26:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.866+0000 7fac177fe700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac1c067830 0x7fac1c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.866+0000 7fac177fe700 1 -- 172.21.15.31:0/2545824121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fac24082400 con 0x7fac30159560 2023-10-06T05:26:13.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.867+0000 7fac2f7fe700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac1c067830 0x7fac1c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:13.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.867+0000 7fac2f7fe700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac1c067830 0x7fac1c069ce0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fac200099b0 tx=0x7fac20017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:13.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:13.871+0000 7fac177fe700 1 -- 172.21.15.31:0/2545824121 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac24051040 con 0x7fac30159560 2023-10-06T05:26:14.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.071+0000 7fac35bff700 1 -- 172.21.15.31:0/2545824121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fac3004c380 con 0x7fac30159560 2023-10-06T05:26:14.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.071+0000 7fac177fe700 1 -- 172.21.15.31:0/2545824121 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fac24017020 con 0x7fac30159560 2023-10-06T05:26:14.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.073+0000 7fac35bff700 1 -- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac1c067830 msgr2=0x7fac1c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.073+0000 7fac35bff700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac1c067830 0x7fac1c069ce0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fac200099b0 tx=0x7fac20017000 comp rx=0 tx=0).stop 2023-10-06T05:26:14.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.073+0000 7fac35bff700 1 -- 172.21.15.31:0/2545824121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 msgr2=0x7fac300c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.073+0000 7fac35bff700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 0x7fac300c6d40 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fac2400c560 tx=0x7fac24007f60 comp rx=0 tx=0).stop 2023-10-06T05:26:14.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.074+0000 7fac35bff700 1 -- 172.21.15.31:0/2545824121 shutdown_connections 2023-10-06T05:26:14.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.074+0000 7fac35bff700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fac1c067830 0x7fac1c069ce0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.074+0000 7fac35bff700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac30159930 0x7fac300c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.074+0000 7fac35bff700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac300b2b80 0x7fac300c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.074+0000 7fac35bff700 1 --2- 172.21.15.31:0/2545824121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac30159560 0x7fac300c6d40 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.074+0000 7fac35bff700 1 -- 172.21.15.31:0/2545824121 >> 172.21.15.31:0/2545824121 conn(0x7fac3006ec90 msgr2=0x7fac300b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.074+0000 7fac35bff700 1 -- 172.21.15.31:0/2545824121 shutdown_connections 2023-10-06T05:26:14.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.074+0000 7fac35bff700 1 -- 172.21.15.31:0/2545824121 wait complete. 2023-10-06T05:26:14.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:14.093 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/group_0000000000194988/subvolume_0000000000185018) 2023-10-06T05:26:14.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:14.172 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-10-06T05:26:14.220 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-10-06T05:26:14.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:14.221 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/group_0000000000194988/subvolume_0000000000185018) 2023-10-06T05:26:14.296 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-10-06T05:26:14.296 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/group_0000000000194988/subvolume_0000000000185018/.meta 2023-10-06T05:26:14.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000185018 group_0000000000194988 2023-10-06T05:26:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.608+0000 7f79159ff700 1 Processor -- start 2023-10-06T05:26:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.609+0000 7f79159ff700 1 -- start start 2023-10-06T05:26:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.610+0000 7f79159ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7910154cd0 0x7f79101583e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.610+0000 7f79159ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79101550a0 0x7f79100ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.610+0000 7f79159ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791006f640 con 0x7f791006ee10 2023-10-06T05:26:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.610+0000 7f79159ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f791006e610 con 0x7f7910154cd0 2023-10-06T05:26:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.610+0000 7f79159ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79100adf60 con 0x7f79101550a0 2023-10-06T05:26:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.610+0000 7f790effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7910154cd0 0x7f79101583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.610+0000 7f790effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7910154cd0 0x7f79101583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55314/0 (socket says 172.21.15.31:55314) 2023-10-06T05:26:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.610+0000 7f790effd700 1 -- 172.21.15.31:0/4283844119 learned_addr learned my addr 172.21.15.31:0/4283844119 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.610+0000 7f790e7fc700 1 --2- 172.21.15.31:0/4283844119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79101550a0 0x7f79100ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.611+0000 7f790effd700 1 -- 172.21.15.31:0/4283844119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79101550a0 msgr2=0x7f79100ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.611+0000 7f790effd700 1 --2- 172.21.15.31:0/4283844119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79101550a0 0x7f79100ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.611+0000 7f790effd700 1 -- 172.21.15.31:0/4283844119 >> v1:172.21.15.62:6789/0 conn(0x7f791006ee10 legacy=0x7f791006d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.611+0000 7f790effd700 1 -- 172.21.15.31:0/4283844119 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7910158920 con 0x7f7910154cd0 2023-10-06T05:26:14.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.611+0000 7f790effd700 1 --2- 172.21.15.31:0/4283844119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7910154cd0 0x7f79101583e0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f79000099a0 tx=0x7f7900009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb628c2b9839d61f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.611+0000 7f790dffb700 1 -- 172.21.15.31:0/4283844119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900009e90 con 0x7f7910154cd0 2023-10-06T05:26:14.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.611+0000 7f790dffb700 1 -- 172.21.15.31:0/4283844119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790000b860 con 0x7f7910154cd0 2023-10-06T05:26:14.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.611+0000 7f790dffb700 1 -- 172.21.15.31:0/4283844119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790000bc20 con 0x7f7910154cd0 2023-10-06T05:26:14.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.612+0000 7f79159ff700 1 -- 172.21.15.31:0/4283844119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7910154cd0 msgr2=0x7f79101583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.612+0000 7f79159ff700 1 --2- 172.21.15.31:0/4283844119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7910154cd0 0x7f79101583e0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f79000099a0 tx=0x7f7900009250 comp rx=0 tx=0).stop 2023-10-06T05:26:14.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.612+0000 7f79159ff700 1 -- 172.21.15.31:0/4283844119 shutdown_connections 2023-10-06T05:26:14.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.612+0000 7f79159ff700 1 --2- 172.21.15.31:0/4283844119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7910154cd0 0x7f79101583e0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.612+0000 7f79159ff700 1 --2- 172.21.15.31:0/4283844119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f79101550a0 0x7f79100ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.612+0000 7f79159ff700 1 -- 172.21.15.31:0/4283844119 >> 172.21.15.31:0/4283844119 conn(0x7f7910068030 msgr2=0x7f79100cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.612+0000 7f79159ff700 1 -- 172.21.15.31:0/4283844119 shutdown_connections 2023-10-06T05:26:14.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.612+0000 7f79159ff700 1 -- 172.21.15.31:0/4283844119 wait complete. 2023-10-06T05:26:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.613+0000 7f79159ff700 1 Processor -- start 2023-10-06T05:26:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.613+0000 7f79159ff700 1 -- start start 2023-10-06T05:26:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.614+0000 7f79159ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f791006ee10 0x7f7910150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.614+0000 7f79159ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7910154cd0 0x7f7910157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.614+0000 7f79159ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79101550a0 0x7f79101579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.614+0000 7f79159ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791015bb70 con 0x7f791006ee10 2023-10-06T05:26:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.614+0000 7f79159ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101591e0 con 0x7f79101550a0 2023-10-06T05:26:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.614+0000 7f79159ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7910157fb0 con 0x7f7910154cd0 2023-10-06T05:26:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f791006ee10 0x7f7910150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f791006ee10 0x7f7910150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55318/0 (socket says 172.21.15.31:55318) 2023-10-06T05:26:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790effd700 1 -- 172.21.15.31:0/3132394209 learned_addr learned my addr 172.21.15.31:0/3132394209 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790e7fc700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7910154cd0 0x7f7910157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790f7fe700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79101550a0 0x7f79101579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790effd700 1 -- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7910154cd0 msgr2=0x7f7910157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790effd700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7910154cd0 0x7f7910157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790effd700 1 -- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79101550a0 msgr2=0x7f79101579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790effd700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79101550a0 0x7f79101579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790effd700 1 -- 172.21.15.31:0/3132394209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7900009490 con 0x7f791006ee10 2023-10-06T05:26:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.615+0000 7f790effd700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f791006ee10 0x7f7910150ee0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f7900009290 tx=0x7f7900017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.616+0000 7f78f6ffd700 1 -- 172.21.15.31:0/3132394209 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790001f070 con 0x7f791006ee10 2023-10-06T05:26:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.616+0000 7f78f6ffd700 1 -- 172.21.15.31:0/3132394209 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790000b9d0 con 0x7f791006ee10 2023-10-06T05:26:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.616+0000 7f79159ff700 1 -- 172.21.15.31:0/3132394209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7910158290 con 0x7f791006ee10 2023-10-06T05:26:14.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.616+0000 7f78f6ffd700 1 -- 172.21.15.31:0/3132394209 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900011420 con 0x7f791006ee10 2023-10-06T05:26:14.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.616+0000 7f79159ff700 1 -- 172.21.15.31:0/3132394209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79101593f0 con 0x7f791006ee10 2023-10-06T05:26:14.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.617+0000 7f79159ff700 1 -- 172.21.15.31:0/3132394209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79101595e0 con 0x7f791006ee10 2023-10-06T05:26:14.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.618+0000 7f78f6ffd700 1 -- 172.21.15.31:0/3132394209 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f79000117f0 con 0x7f791006ee10 2023-10-06T05:26:14.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.619+0000 7f78f6ffd700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78fc067830 0x7f78fc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:14.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.619+0000 7f78f6ffd700 1 -- 172.21.15.31:0/3132394209 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f79000842b0 con 0x7f791006ee10 2023-10-06T05:26:14.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.619+0000 7f790e7fc700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78fc067830 0x7f78fc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:14.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.620+0000 7f790e7fc700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78fc067830 0x7f78fc069ce0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f791006d430 tx=0x7f7904008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:14.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.623+0000 7f78f6ffd700 1 -- 172.21.15.31:0/3132394209 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7900052ef0 con 0x7f791006ee10 2023-10-06T05:26:14.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.858+0000 7f79159ff700 1 -- 172.21.15.31:0/3132394209 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000185018", "group_name": "group_0000000000194988", "target": ["mon-mgr", ""]}) v1 -- 0x7f791004c380 con 0x7f78fc067830 2023-10-06T05:26:14.868 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:14.867+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:14.869 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:14.867+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:14.869 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:14.867+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:14.869 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:14.867+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:14.869 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:14.867+0000 7f2e4417d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:14.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.913+0000 7f78f6ffd700 1 -- 172.21.15.31:0/3132394209 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f791004c380 con 0x7f78fc067830 2023-10-06T05:26:14.915 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000194988/subvolume_0000000000185018 2023-10-06T05:26:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 -- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78fc067830 msgr2=0x7f78fc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78fc067830 0x7f78fc069ce0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f791006d430 tx=0x7f7904008040 comp rx=0 tx=0).stop 2023-10-06T05:26:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 -- 172.21.15.31:0/3132394209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f791006ee10 msgr2=0x7f7910150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f791006ee10 0x7f7910150ee0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f7900009290 tx=0x7f7900017000 comp rx=0 tx=0).stop 2023-10-06T05:26:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 -- 172.21.15.31:0/3132394209 shutdown_connections 2023-10-06T05:26:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f78fc067830 0x7f78fc069ce0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79101550a0 0x7f79101579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f791006ee10 0x7f7910150ee0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 --2- 172.21.15.31:0/3132394209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7910154cd0 0x7f7910157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 -- 172.21.15.31:0/3132394209 >> 172.21.15.31:0/3132394209 conn(0x7f7910068030 msgr2=0x7f7910151f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 -- 172.21.15.31:0/3132394209 shutdown_connections 2023-10-06T05:26:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:14.916+0000 7f79159ff700 1 -- 172.21.15.31:0/3132394209 wait complete. 2023-10-06T05:26:14.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000185018 group_0000000000194988 2023-10-06T05:26:15.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.133+0000 7f44273ff700 1 Processor -- start 2023-10-06T05:26:15.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.134+0000 7f44273ff700 1 -- start start 2023-10-06T05:26:15.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.135+0000 7f44273ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4420159cb0 0x7f44200caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.135+0000 7f44273ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44200cb440 con 0x7f44200cafd0 2023-10-06T05:26:15.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.135+0000 7f44273ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44200cb670 con 0x7f44200cf5f0 2023-10-06T05:26:15.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.135+0000 7f44273ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44200cd080 con 0x7f4420159cb0 2023-10-06T05:26:15.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.135+0000 7f442499a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f44200cafd0 0x7f442015a080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:40880/0 (socket says 172.21.15.31:40880) 2023-10-06T05:26:15.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.135+0000 7f442499a700 1 -- 172.21.15.31:0/464317017 learned_addr learned my addr 172.21.15.31:0/464317017 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:15.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.135+0000 7f442519b700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4420159cb0 0x7f44200caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.135+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124157357 0 0) 0x7f44200cb440 con 0x7f44200cafd0 2023-10-06T05:26:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440c003570 con 0x7f44200cafd0 2023-10-06T05:26:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903320114 0 0) 0x7f44200cb670 con 0x7f44200cf5f0 2023-10-06T05:26:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44200cb440 con 0x7f44200cf5f0 2023-10-06T05:26:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993589937 0 0) 0x7f440c003570 con 0x7f44200cafd0 2023-10-06T05:26:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4420159cb0 msgr2=0x7f44200caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4420159cb0 0x7f44200caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 >> v1:172.21.15.62:6789/0 conn(0x7f44200cf5f0 legacy=0x7f4420163270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200cb950 con 0x7f44200cafd0 2023-10-06T05:26:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f4410001470 con 0x7f44200cafd0 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 >> v1:172.21.15.31:6789/0 conn(0x7f44200cafd0 legacy=0x7f442015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c003c20 0x7f440c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4420159cb0 0x7f440c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 0x7f440c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200cd080 con 0x7f440c004d50 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c00bb40 con 0x7f440c003c20 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c00bd00 con 0x7f4420159cb0 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f442599c700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 0x7f440c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f442499a700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4420159cb0 0x7f440c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.136+0000 7f442519b700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c003c20 0x7f440c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.137+0000 7f442599c700 1 -- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4420159cb0 msgr2=0x7f440c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.137+0000 7f442599c700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4420159cb0 0x7f440c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.137+0000 7f442599c700 1 -- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c003c20 msgr2=0x7f440c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.137+0000 7f442599c700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c003c20 0x7f440c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.137+0000 7f442599c700 1 -- 172.21.15.31:0/464317017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c003570 con 0x7f440c004d50 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.137+0000 7f442599c700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 0x7f440c0051c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f4418002120 tx=0x7f44180017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2976fde5e052c451 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.137+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418001ab0 con 0x7f440c004d50 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.137+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441800c980 con 0x7f440c004d50 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.137+0000 7f4417fff700 1 -- 172.21.15.31:0/464317017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441800cd20 con 0x7f440c004d50 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.138+0000 7f44273ff700 1 -- 172.21.15.31:0/464317017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 msgr2=0x7f440c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.138+0000 7f44273ff700 1 --2- 172.21.15.31:0/464317017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 0x7f440c0051c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f4418002120 tx=0x7f44180017a0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.138+0000 7f442599c700 1 -- 172.21.15.31:0/464317017 reap_dead start 2023-10-06T05:26:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.138+0000 7f44273ff700 1 -- 172.21.15.31:0/464317017 shutdown_connections 2023-10-06T05:26:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.138+0000 7f44273ff700 1 -- 172.21.15.31:0/464317017 >> 172.21.15.31:0/464317017 conn(0x7f4420099710 msgr2=0x7f442006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.138+0000 7f44273ff700 1 -- 172.21.15.31:0/464317017 shutdown_connections 2023-10-06T05:26:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.138+0000 7f44273ff700 1 -- 172.21.15.31:0/464317017 wait complete. 2023-10-06T05:26:15.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.140+0000 7f44273ff700 1 Processor -- start 2023-10-06T05:26:15.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.140+0000 7f44273ff700 1 -- start start 2023-10-06T05:26:15.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.141+0000 7f44273ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 0x7f44201520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.141+0000 7f44273ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4420152610 0x7f44200c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f44273ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44200c41e0 0x7f44200c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f44273ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4420154f00 con 0x7f440c004d50 2023-10-06T05:26:15.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f44273ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200c4660 con 0x7f4420152610 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f44273ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442016a850 con 0x7f44200c41e0 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 0x7f44201520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442519b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 0x7f44201520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55346/0 (socket says 172.21.15.31:55346) 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442519b700 1 -- 172.21.15.31:0/1401119718 learned_addr learned my addr 172.21.15.31:0/1401119718 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442499a700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4420152610 0x7f44200c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442519b700 1 -- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44200c41e0 msgr2=0x7f44200c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442519b700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44200c41e0 0x7f44200c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442519b700 1 -- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4420152610 msgr2=0x7f44200c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442519b700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4420152610 0x7f44200c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442519b700 1 -- 172.21.15.31:0/1401119718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420169f40 con 0x7f440c004d50 2023-10-06T05:26:15.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442499a700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4420152610 0x7f44200c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f442519b700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 0x7f44201520d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f441c011b40 tx=0x7f441c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.142+0000 7f44157fa700 1 -- 172.21.15.31:0/1401119718 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c005c10 con 0x7f440c004d50 2023-10-06T05:26:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.143+0000 7f44157fa700 1 -- 172.21.15.31:0/1401119718 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c005d70 con 0x7f440c004d50 2023-10-06T05:26:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.143+0000 7f44157fa700 1 -- 172.21.15.31:0/1401119718 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c00fa90 con 0x7f440c004d50 2023-10-06T05:26:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.143+0000 7f44273ff700 1 -- 172.21.15.31:0/1401119718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44200c6980 con 0x7f440c004d50 2023-10-06T05:26:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.143+0000 7f44273ff700 1 -- 172.21.15.31:0/1401119718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442016a4e0 con 0x7f440c004d50 2023-10-06T05:26:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.144+0000 7f44273ff700 1 -- 172.21.15.31:0/1401119718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442004c380 con 0x7f440c004d50 2023-10-06T05:26:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.145+0000 7f44157fa700 1 -- 172.21.15.31:0/1401119718 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 85556+0+0 (secure 0 0 0) 0x7f441c006050 con 0x7f440c004d50 2023-10-06T05:26:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.145+0000 7f44157fa700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4404067830 0x7f4404069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.146+0000 7f44157fa700 1 -- 172.21.15.31:0/1401119718 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f441c083960 con 0x7f440c004d50 2023-10-06T05:26:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.146+0000 7f442499a700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4404067830 0x7f4404069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.147+0000 7f442499a700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4404067830 0x7f4404069ce0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f44100011e0 tx=0x7f4410009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.150+0000 7f44157fa700 1 -- 172.21.15.31:0/1401119718 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f441c0525a0 con 0x7f440c004d50 2023-10-06T05:26:15.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.393+0000 7f44273ff700 1 -- 172.21.15.31:0/1401119718 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000185018", "group_name": "group_0000000000194988", "target": ["mon-mgr", ""]}) v1 -- 0x7f442016a990 con 0x7f4404067830 2023-10-06T05:26:15.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.399+0000 7f44157fa700 1 -- 172.21.15.31:0/1401119718 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f4410000c80 con 0x7f4404067830 2023-10-06T05:26:15.401 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/group_0000000000194988/subvolume_0000000000185018 2023-10-06T05:26:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.401+0000 7f44273ff700 1 -- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4404067830 msgr2=0x7f4404069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.401+0000 7f44273ff700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4404067830 0x7f4404069ce0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f44100011e0 tx=0x7f4410009040 comp rx=0 tx=0).stop 2023-10-06T05:26:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.401+0000 7f44273ff700 1 -- 172.21.15.31:0/1401119718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 msgr2=0x7f44201520d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.401+0000 7f44273ff700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 0x7f44201520d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f441c011b40 tx=0x7f441c009120 comp rx=0 tx=0).stop 2023-10-06T05:26:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.402+0000 7f44273ff700 1 -- 172.21.15.31:0/1401119718 shutdown_connections 2023-10-06T05:26:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.402+0000 7f44273ff700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4404067830 0x7f4404069ce0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.402+0000 7f44273ff700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44200c41e0 0x7f44200c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.402+0000 7f44273ff700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f440c004d50 0x7f44201520d0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.402+0000 7f44273ff700 1 --2- 172.21.15.31:0/1401119718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4420152610 0x7f44200c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.402+0000 7f44273ff700 1 -- 172.21.15.31:0/1401119718 >> 172.21.15.31:0/1401119718 conn(0x7f4420099710 msgr2=0x7f4420157b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.402+0000 7f44273ff700 1 -- 172.21.15.31:0/1401119718 shutdown_connections 2023-10-06T05:26:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:15.402+0000 7f44273ff700 1 -- 172.21.15.31:0/1401119718 wait complete. 2023-10-06T05:26:15.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:15.421 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/493da0895e9e528f0f40c40701236011.meta) 2023-10-06T05:26:17.096 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-10-06T05:26:17.097 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-10-06T05:26:17.097 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-10-06T05:26:17.097 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/group_0000000000194988/subvolume_0000000000185018 2023-10-06T05:26:17.097 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-10-06T05:26:17.097 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:17.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000185018 group_0000000000194988 2023-10-06T05:26:17.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.310+0000 7f8e063ff700 1 Processor -- start 2023-10-06T05:26:17.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.311+0000 7f8e063ff700 1 -- start start 2023-10-06T05:26:17.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.312+0000 7f8e063ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e00142c20 0x7f8e000cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.312+0000 7f8e063ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8e000cb3e0 0x7f8e00068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.312+0000 7f8e063ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0015d210 con 0x7f8e0015a990 2023-10-06T05:26:17.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.312+0000 7f8e063ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e000cc810 con 0x7f8e000cb3e0 2023-10-06T05:26:17.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.312+0000 7f8e063ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0015d4b0 con 0x7f8e00142c20 2023-10-06T05:26:17.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.313+0000 7f8dfffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e00142c20 0x7f8e000cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.313+0000 7f8e0499c700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8e000cb3e0 0x7f8e00068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.313+0000 7f8dfffff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e00142c20 0x7f8e000cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.31:46714/0 (socket says 172.21.15.31:46714) 2023-10-06T05:26:17.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.313+0000 7f8dfffff700 1 -- 172.21.15.31:0/1160591039 learned_addr learned my addr 172.21.15.31:0/1160591039 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:17.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.313+0000 7f8dff7fe700 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8e0015a990 0x7f8e000cacd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.31:46190/0 (socket says 172.21.15.31:46190) 2023-10-06T05:26:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.313+0000 7f8e0499c700 1 -- 172.21.15.31:0/1160591039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e00142c20 msgr2=0x7f8e000cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.313+0000 7f8e0499c700 1 --2- 172.21.15.31:0/1160591039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e00142c20 0x7f8e000cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.313+0000 7f8e0499c700 1 -- 172.21.15.31:0/1160591039 >> v1:172.21.15.62:6789/0 conn(0x7f8e0015a990 legacy=0x7f8e000cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.313+0000 7f8e0499c700 1 -- 172.21.15.31:0/1160591039 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0015d790 con 0x7f8e000cb3e0 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.313+0000 7f8dfeffd700 1 -- 172.21.15.31:0/1160591039 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224698671 0 0) 0x7f8e0015d210 con 0x7f8e0015a990 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.314+0000 7f8e0499c700 1 --2- 172.21.15.31:0/1160591039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e000cb3e0 0x7f8e00068670 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009e60 tx=0x7f8df8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aebf7ecc815b7d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.314+0000 7f8dfeffd700 1 -- 172.21.15.31:0/1160591039 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df800e070 con 0x7f8e000cb3e0 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.314+0000 7f8dfeffd700 1 -- 172.21.15.31:0/1160591039 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df800bb60 con 0x7f8e000cb3e0 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.314+0000 7f8dfeffd700 1 -- 172.21.15.31:0/1160591039 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8012430 con 0x7f8e000cb3e0 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.314+0000 7f8e063ff700 1 -- 172.21.15.31:0/1160591039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e000cb3e0 msgr2=0x7f8e00068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.314+0000 7f8e063ff700 1 --2- 172.21.15.31:0/1160591039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e000cb3e0 0x7f8e00068670 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009e60 tx=0x7f8df8009780 comp rx=0 tx=0).stop 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.314+0000 7f8e063ff700 1 -- 172.21.15.31:0/1160591039 shutdown_connections 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.314+0000 7f8e063ff700 1 --2- 172.21.15.31:0/1160591039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e00142c20 0x7f8e000cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.314+0000 7f8e063ff700 1 --2- 172.21.15.31:0/1160591039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e000cb3e0 0x7f8e00068670 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.314+0000 7f8e063ff700 1 -- 172.21.15.31:0/1160591039 >> 172.21.15.31:0/1160591039 conn(0x7f8e000ce2d0 msgr2=0x7f8e000ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.315+0000 7f8e063ff700 1 -- 172.21.15.31:0/1160591039 shutdown_connections 2023-10-06T05:26:17.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.315+0000 7f8e063ff700 1 -- 172.21.15.31:0/1160591039 wait complete. 2023-10-06T05:26:17.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.316+0000 7f8e063ff700 1 Processor -- start 2023-10-06T05:26:17.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.316+0000 7f8e063ff700 1 -- start start 2023-10-06T05:26:17.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.317+0000 7f8e063ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000cb3e0 0x7f8e000adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.317+0000 7f8e063ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e00142c20 0x7f8e0013eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.317+0000 7f8e063ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e0015a990 0x7f8e000b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.317+0000 7f8e063ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000ae030 con 0x7f8e000cb3e0 2023-10-06T05:26:17.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.317+0000 7f8e063ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0013f3e0 con 0x7f8e00142c20 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.317+0000 7f8e063ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000ae2b0 con 0x7f8e0015a990 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.317+0000 7f8dfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000cb3e0 0x7f8e000adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.317+0000 7f8dfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000cb3e0 0x7f8e000adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46088/0 (socket says 172.21.15.31:46088) 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.317+0000 7f8dfffff700 1 -- 172.21.15.31:0/3015518353 learned_addr learned my addr 172.21.15.31:0/3015518353 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.317+0000 7f8dff7fe700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e00142c20 0x7f8e0013eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8e0499c700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e0015a990 0x7f8e000b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8dfffff700 1 -- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e0015a990 msgr2=0x7f8e000b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8dfffff700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e0015a990 0x7f8e000b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8dfffff700 1 -- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e00142c20 msgr2=0x7f8e0013eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8dfffff700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e00142c20 0x7f8e0013eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8dfffff700 1 -- 172.21.15.31:0/3015518353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df8009a20 con 0x7f8e000cb3e0 2023-10-06T05:26:17.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8dfffff700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000cb3e0 0x7f8e000adaf0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f8df000ea50 tx=0x7f8df0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8de7fff700 1 -- 172.21.15.31:0/3015518353 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df000c850 con 0x7f8e000cb3e0 2023-10-06T05:26:17.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8de7fff700 1 -- 172.21.15.31:0/3015518353 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df000c9b0 con 0x7f8e000cb3e0 2023-10-06T05:26:17.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8e063ff700 1 -- 172.21.15.31:0/3015518353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e000ae590 con 0x7f8e000cb3e0 2023-10-06T05:26:17.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8de7fff700 1 -- 172.21.15.31:0/3015518353 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df000cd30 con 0x7f8e000cb3e0 2023-10-06T05:26:17.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.318+0000 7f8e063ff700 1 -- 172.21.15.31:0/3015518353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e000aeb50 con 0x7f8e000cb3e0 2023-10-06T05:26:17.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.319+0000 7f8e063ff700 1 -- 172.21.15.31:0/3015518353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0004c380 con 0x7f8e000cb3e0 2023-10-06T05:26:17.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.320+0000 7f8de7fff700 1 -- 172.21.15.31:0/3015518353 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85607+0+0 (secure 0 0 0) 0x7f8df0013680 con 0x7f8e000cb3e0 2023-10-06T05:26:17.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.321+0000 7f8de7fff700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8dec0678c0 0x7f8dec069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.321+0000 7f8de7fff700 1 -- 172.21.15.31:0/3015518353 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f8df00831c0 con 0x7f8e000cb3e0 2023-10-06T05:26:17.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.322+0000 7f8dff7fe700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8dec0678c0 0x7f8dec069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.322+0000 7f8dff7fe700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8dec0678c0 0x7f8dec069d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8df4007fd0 tx=0x7f8df4007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.326+0000 7f8de7fff700 1 -- 172.21.15.31:0/3015518353 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8df0051e00 con 0x7f8e000cb3e0 2023-10-06T05:26:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.557+0000 7f8e063ff700 1 -- 172.21.15.31:0/3015518353 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000185018", "group_name": "group_0000000000194988", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e0015db20 con 0x7f8dec0678c0 2023-10-06T05:26:17.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.574+0000 7f8de7fff700 1 -- 172.21.15.31:0/3015518353 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8e0015db20 con 0x7f8dec0678c0 2023-10-06T05:26:17.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 -- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8dec0678c0 msgr2=0x7f8dec069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8dec0678c0 0x7f8dec069d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8df4007fd0 tx=0x7f8df4007df0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 -- 172.21.15.31:0/3015518353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000cb3e0 msgr2=0x7f8e000adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000cb3e0 0x7f8e000adaf0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f8df000ea50 tx=0x7f8df0009250 comp rx=0 tx=0).stop 2023-10-06T05:26:17.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 -- 172.21.15.31:0/3015518353 shutdown_connections 2023-10-06T05:26:17.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f8dec0678c0 0x7f8dec069d70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e0015a990 0x7f8e000b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e000cb3e0 0x7f8e000adaf0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 --2- 172.21.15.31:0/3015518353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e00142c20 0x7f8e0013eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 -- 172.21.15.31:0/3015518353 >> 172.21.15.31:0/3015518353 conn(0x7f8e000ce2d0 msgr2=0x7f8e0013fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 -- 172.21.15.31:0/3015518353 shutdown_connections 2023-10-06T05:26:17.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.577+0000 7f8e063ff700 1 -- 172.21.15.31:0/3015518353 wait complete. 2023-10-06T05:26:17.586 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:17.583+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:17.586 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:17.583+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:17.586 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:17.583+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:17.586 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:17.583+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:17.586 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:17.583+0000 7f2e47183700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:17.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:17.597 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T05:26:17.605 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:17.604+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:17.605 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:17.604+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:17.606 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:17.604+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:17.606 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:17.604+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:17.606 INFO:tasks.ceph.mgr.x.smithi062.stderr:2023-10-06T05:26:17.604+0000 7f2e45980700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:26:17.687 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-10-06T05:26:17.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000194988 2023-10-06T05:26:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.885+0000 7f45911ff700 1 Processor -- start 2023-10-06T05:26:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.885+0000 7f45911ff700 1 -- start start 2023-10-06T05:26:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.887+0000 7f45911ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c1573a0 0x7f458c0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.887+0000 7f45911ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f458c0ae860 0x7f458c1611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.887+0000 7f45911ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c158cf0 con 0x7f458c157770 2023-10-06T05:26:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.887+0000 7f45911ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c1580e0 con 0x7f458c0ae860 2023-10-06T05:26:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.887+0000 7f45911ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1583c0 con 0x7f458c1573a0 2023-10-06T05:26:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.887+0000 7f458a59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f458c157770 0x7f458c0b1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33520/0 (socket says 172.21.15.31:33520) 2023-10-06T05:26:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.887+0000 7f458a59c700 1 -- 172.21.15.31:0/923350441 learned_addr learned my addr 172.21.15.31:0/923350441 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:17.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.887+0000 7f458ad9d700 1 --2- 172.21.15.31:0/923350441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c1573a0 0x7f458c0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.887+0000 7f458b59e700 1 --2- 172.21.15.31:0/923350441 >> v2:172.21.15.62:3300/0 conn(0x7f458c0ae860 0x7f458c1611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.888+0000 7f4589d9b700 1 -- 172.21.15.31:0/923350441 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484225921 0 0) 0x7f458c158cf0 con 0x7f458c157770 2023-10-06T05:26:17.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.888+0000 7f4589d9b700 1 -- 172.21.15.31:0/923350441 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4574003570 con 0x7f458c157770 2023-10-06T05:26:17.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.888+0000 7f458ad9d700 1 -- 172.21.15.31:0/923350441 >> v2:172.21.15.62:3300/0 conn(0x7f458c0ae860 msgr2=0x7f458c1611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:17.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.888+0000 7f458ad9d700 1 --2- 172.21.15.31:0/923350441 >> v2:172.21.15.62:3300/0 conn(0x7f458c0ae860 0x7f458c1611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.888+0000 7f458ad9d700 1 -- 172.21.15.31:0/923350441 >> v1:172.21.15.31:6789/0 conn(0x7f458c157770 legacy=0x7f458c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.888+0000 7f458ad9d700 1 -- 172.21.15.31:0/923350441 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c1586a0 con 0x7f458c1573a0 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.888+0000 7f458b59e700 1 --2- 172.21.15.31:0/923350441 >> v2:172.21.15.62:3300/0 conn(0x7f458c0ae860 0x7f458c1611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.888+0000 7f458ad9d700 1 --2- 172.21.15.31:0/923350441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c1573a0 0x7f458c0acb70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f457c0098e0 tx=0x7f457c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a3db9b5ebe1963e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f4589d9b700 1 -- 172.21.15.31:0/923350441 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c011630 con 0x7f458c1573a0 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f4589d9b700 1 -- 172.21.15.31:0/923350441 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c011790 con 0x7f458c1573a0 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f4589d9b700 1 -- 172.21.15.31:0/923350441 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c011b30 con 0x7f458c1573a0 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f45911ff700 1 -- 172.21.15.31:0/923350441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c1573a0 msgr2=0x7f458c0acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f45911ff700 1 --2- 172.21.15.31:0/923350441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c1573a0 0x7f458c0acb70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f457c0098e0 tx=0x7f457c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f45911ff700 1 -- 172.21.15.31:0/923350441 shutdown_connections 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f45911ff700 1 --2- 172.21.15.31:0/923350441 >> v2:172.21.15.62:3300/0 conn(0x7f458c0ae860 0x7f458c1611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f45911ff700 1 --2- 172.21.15.31:0/923350441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c1573a0 0x7f458c0acb70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f45911ff700 1 -- 172.21.15.31:0/923350441 >> 172.21.15.31:0/923350441 conn(0x7f458c067560 msgr2=0x7f458c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f45911ff700 1 -- 172.21.15.31:0/923350441 shutdown_connections 2023-10-06T05:26:17.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.889+0000 7f45911ff700 1 -- 172.21.15.31:0/923350441 wait complete. 2023-10-06T05:26:17.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.891+0000 7f45911ff700 1 Processor -- start 2023-10-06T05:26:17.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.891+0000 7f45911ff700 1 -- start start 2023-10-06T05:26:17.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f45911ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f458c1573a0 0x7f458c14c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f45911ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c157770 0x7f458c0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f45911ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0cb660 0x7f458c14e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f45911ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0a98a0 con 0x7f458c0cb660 2023-10-06T05:26:17.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f45911ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0cbad0 con 0x7f458c1573a0 2023-10-06T05:26:17.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f45911ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c14d080 con 0x7f458c157770 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f458b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0cb660 0x7f458c14e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f458b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0cb660 0x7f458c14e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46098/0 (socket says 172.21.15.31:46098) 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f458b59e700 1 -- 172.21.15.31:0/4208037971 learned_addr learned my addr 172.21.15.31:0/4208037971 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f458ad9d700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f458c1573a0 0x7f458c14c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.892+0000 7f458a59c700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c157770 0x7f458c0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f458b59e700 1 -- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c157770 msgr2=0x7f458c0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f458b59e700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c157770 0x7f458c0cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f458b59e700 1 -- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f458c1573a0 msgr2=0x7f458c14c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f458b59e700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f458c1573a0 0x7f458c14c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f458b59e700 1 -- 172.21.15.31:0/4208037971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4584009a20 con 0x7f458c0cb660 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f458b59e700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0cb660 0x7f458c14e660 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f4584009f50 tx=0x7f458400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f4572ffd700 1 -- 172.21.15.31:0/4208037971 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400caf0 con 0x7f458c0cb660 2023-10-06T05:26:17.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f4572ffd700 1 -- 172.21.15.31:0/4208037971 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458400f040 con 0x7f458c0cb660 2023-10-06T05:26:17.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f45911ff700 1 -- 172.21.15.31:0/4208037971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457c009420 con 0x7f458c0cb660 2023-10-06T05:26:17.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f4572ffd700 1 -- 172.21.15.31:0/4208037971 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400ce80 con 0x7f458c0cb660 2023-10-06T05:26:17.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.893+0000 7f45911ff700 1 -- 172.21.15.31:0/4208037971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c14f6d0 con 0x7f458c0cb660 2023-10-06T05:26:17.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.894+0000 7f45911ff700 1 -- 172.21.15.31:0/4208037971 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c04c380 con 0x7f458c0cb660 2023-10-06T05:26:17.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.895+0000 7f4572ffd700 1 -- 172.21.15.31:0/4208037971 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85607+0+0 (secure 0 0 0) 0x7f4584014690 con 0x7f458c0cb660 2023-10-06T05:26:17.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.896+0000 7f4572ffd700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45780678c0 0x7f4578069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:17.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.896+0000 7f4572ffd700 1 -- 172.21.15.31:0/4208037971 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f4584083d30 con 0x7f458c0cb660 2023-10-06T05:26:17.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.897+0000 7f458ad9d700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45780678c0 0x7f4578069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:17.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.897+0000 7f458ad9d700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45780678c0 0x7f4578069d70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f457c0091e0 tx=0x7f457c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:17.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:17.900+0000 7f4572ffd700 1 -- 172.21.15.31:0/4208037971 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4584052770 con 0x7f458c0cb660 2023-10-06T05:26:18.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.130+0000 7f45911ff700 1 -- 172.21.15.31:0/4208037971 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "group_0000000000194988", "target": ["mon-mgr", ""]}) v1 -- 0x7f458c156110 con 0x7f45780678c0 2023-10-06T05:26:18.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.135+0000 7f4572ffd700 1 -- 172.21.15.31:0/4208037971 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f458c156110 con 0x7f45780678c0 2023-10-06T05:26:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 -- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45780678c0 msgr2=0x7f4578069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45780678c0 0x7f4578069d70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f457c0091e0 tx=0x7f457c017000 comp rx=0 tx=0).stop 2023-10-06T05:26:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 -- 172.21.15.31:0/4208037971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0cb660 msgr2=0x7f458c14e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0cb660 0x7f458c14e660 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f4584009f50 tx=0x7f458400b040 comp rx=0 tx=0).stop 2023-10-06T05:26:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 -- 172.21.15.31:0/4208037971 shutdown_connections 2023-10-06T05:26:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45780678c0 0x7f4578069d70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f458c0cb660 0x7f458c14e660 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f458c1573a0 0x7f458c14c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 --2- 172.21.15.31:0/4208037971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f458c157770 0x7f458c0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 -- 172.21.15.31:0/4208037971 >> 172.21.15.31:0/4208037971 conn(0x7f458c067560 msgr2=0x7f458c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:18.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 -- 172.21.15.31:0/4208037971 shutdown_connections 2023-10-06T05:26:18.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.138+0000 7f45911ff700 1 -- 172.21.15.31:0/4208037971 wait complete. 2023-10-06T05:26:18.157 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:26:18.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:26:18.259 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:26:18.370 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:26:18.371 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:26:18.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:18.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:26:18.483 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:26:18.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:18.483 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:26:18.532 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T05:26:18.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:26:18.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T05:26:18.627 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:26:18.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:18.627 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T05:26:18.674 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.870+0000 7fbf8c7ff700 1 Processor -- start 2023-10-06T05:26:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.871+0000 7fbf8c7ff700 1 -- start start 2023-10-06T05:26:18.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.872+0000 7fbf8c7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbf8413a1e0 0x7fbf840c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.872+0000 7fbf8c7ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf8413a5b0 0x7fbf8406d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.872+0000 7fbf8c7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf840c71b0 con 0x7fbf840c6c30 2023-10-06T05:26:18.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.872+0000 7fbf8c7ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf840c7490 con 0x7fbf8413a5b0 2023-10-06T05:26:18.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.872+0000 7fbf8c7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf840c7770 con 0x7fbf8413a1e0 2023-10-06T05:26:18.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.872+0000 7fbf89d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf8413a5b0 0x7fbf8406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.872+0000 7fbf89d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbf8413a5b0 0x7fbf8406d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46114/0 (socket says 172.21.15.31:46114) 2023-10-06T05:26:18.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.872+0000 7fbf89d9a700 1 -- 172.21.15.31:0/2430003178 learned_addr learned my addr 172.21.15.31:0/2430003178 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.872+0000 7fbf8a59b700 1 --2- 172.21.15.31:0/2430003178 >> v2:172.21.15.62:3300/0 conn(0x7fbf8413a1e0 0x7fbf840c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.873+0000 7fbf89d9a700 1 -- 172.21.15.31:0/2430003178 >> v2:172.21.15.62:3300/0 conn(0x7fbf8413a1e0 msgr2=0x7fbf840c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:18.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.873+0000 7fbf89d9a700 1 --2- 172.21.15.31:0/2430003178 >> v2:172.21.15.62:3300/0 conn(0x7fbf8413a1e0 0x7fbf840c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.873+0000 7fbf89d9a700 1 -- 172.21.15.31:0/2430003178 >> v1:172.21.15.62:6789/0 conn(0x7fbf840c6c30 legacy=0x7fbf8415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.873+0000 7fbf89d9a700 1 -- 172.21.15.31:0/2430003178 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf840c7a50 con 0x7fbf8413a5b0 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.873+0000 7fbf89d9a700 1 --2- 172.21.15.31:0/2430003178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8413a5b0 0x7fbf8406d440 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fbf840cbf90 tx=0x7fbf74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7bdb75511b4372 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.873+0000 7fbf89599700 1 -- 172.21.15.31:0/2430003178 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74009e70 con 0x7fbf8413a5b0 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.873+0000 7fbf89599700 1 -- 172.21.15.31:0/2430003178 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7400b860 con 0x7fbf8413a5b0 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.873+0000 7fbf89599700 1 -- 172.21.15.31:0/2430003178 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400bc00 con 0x7fbf8413a5b0 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.874+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/2430003178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8413a5b0 msgr2=0x7fbf8406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.874+0000 7fbf8c7ff700 1 --2- 172.21.15.31:0/2430003178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8413a5b0 0x7fbf8406d440 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fbf840cbf90 tx=0x7fbf74009250 comp rx=0 tx=0).stop 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.874+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/2430003178 shutdown_connections 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.874+0000 7fbf8c7ff700 1 --2- 172.21.15.31:0/2430003178 >> v2:172.21.15.62:3300/0 conn(0x7fbf8413a1e0 0x7fbf840c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.874+0000 7fbf8c7ff700 1 --2- 172.21.15.31:0/2430003178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8413a5b0 0x7fbf8406d440 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.874+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/2430003178 >> 172.21.15.31:0/2430003178 conn(0x7fbf840cc0b0 msgr2=0x7fbf840cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.874+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/2430003178 shutdown_connections 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.874+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/2430003178 wait complete. 2023-10-06T05:26:18.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.876+0000 7fbf8c7ff700 1 Processor -- start 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.876+0000 7fbf8c7ff700 1 -- start start 2023-10-06T05:26:18.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8c7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf840c6c30 0x7fbf841651c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8c7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf8413a1e0 0x7fbf84151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8c7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf8413a5b0 0x7fbf840c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8c7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8413bec0 con 0x7fbf840c6c30 2023-10-06T05:26:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8c7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8413c760 con 0x7fbf8413a1e0 2023-10-06T05:26:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8c7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf84165700 con 0x7fbf8413a5b0 2023-10-06T05:26:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8a59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf840c6c30 0x7fbf841651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8a59b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf840c6c30 0x7fbf841651c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46130/0 (socket says 172.21.15.31:46130) 2023-10-06T05:26:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8a59b700 1 -- 172.21.15.31:0/1485181348 learned_addr learned my addr 172.21.15.31:0/1485181348 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf89d9a700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf8413a1e0 0x7fbf84151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8ad9c700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf8413a5b0 0x7fbf840c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8a59b700 1 -- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf8413a5b0 msgr2=0x7fbf840c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8a59b700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf8413a5b0 0x7fbf840c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8a59b700 1 -- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf8413a1e0 msgr2=0x7fbf84151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8a59b700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf8413a1e0 0x7fbf84151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf8a59b700 1 -- 172.21.15.31:0/1485181348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf740094f0 con 0x7fbf840c6c30 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.878+0000 7fbf89d9a700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf8413a1e0 0x7fbf84151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.879+0000 7fbf8a59b700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf840c6c30 0x7fbf841651c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000ecb0 tx=0x7fbf80009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.879+0000 7fbf7a7fc700 1 -- 172.21.15.31:0/1485181348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8000c850 con 0x7fbf840c6c30 2023-10-06T05:26:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.879+0000 7fbf7a7fc700 1 -- 172.21.15.31:0/1485181348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8000c9b0 con 0x7fbf840c6c30 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.879+0000 7fbf7a7fc700 1 -- 172.21.15.31:0/1485181348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8000cd30 con 0x7fbf840c6c30 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.879+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/1485181348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf84168b00 con 0x7fbf840c6c30 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.879+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/1485181348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8413c3f0 con 0x7fbf840c6c30 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.880+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/1485181348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf840ccbd0 con 0x7fbf840c6c30 2023-10-06T05:26:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.881+0000 7fbf7a7fc700 1 -- 172.21.15.31:0/1485181348 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 85607+0+0 (secure 0 0 0) 0x7fbf80013680 con 0x7fbf840c6c30 2023-10-06T05:26:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.882+0000 7fbf7a7fc700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf680678f0 0x7fbf68069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.883+0000 7fbf7a7fc700 1 -- 172.21.15.31:0/1485181348 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fbf80083420 con 0x7fbf840c6c30 2023-10-06T05:26:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.883+0000 7fbf89d9a700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf680678f0 0x7fbf68069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.884+0000 7fbf89d9a700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf680678f0 0x7fbf68069da0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbf8413dba0 tx=0x7fbf74017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:18.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:18.887+0000 7fbf7a7fc700 1 -- 172.21.15.31:0/1485181348 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf80052060 con 0x7fbf840c6c30 2023-10-06T05:26:19.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.131+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/1485181348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf8404c380 con 0x7fbf840c6c30 2023-10-06T05:26:19.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.131+0000 7fbf7a7fc700 1 -- 172.21.15.31:0/1485181348 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4607 (secure 0 0 0) 0x7fbf80055680 con 0x7fbf840c6c30 2023-10-06T05:26:19.134 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:19.134 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":113,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4242992570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4242992570},{"type":"v1","addr":"172.21.15.31:6835","nonce":4242992570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":7706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2405114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2405114362},{"type":"v1","addr":"172.21.15.62:6834","nonce":2405114362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2676480318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2676480318},{"type":"v1","addr":"172.21.15.62:6835","nonce":2676480318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:02.260523+0000","modified":"2023-10-06T05:26:06.149545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9249":{"gid":9249,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3001391649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3001391649},{"type":"v1","addr":"172.21.15.31:6834","nonce":3001391649}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:26:19.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.133+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf680678f0 msgr2=0x7fbf68069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.133+0000 7fbf8c7ff700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf680678f0 0x7fbf68069da0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbf8413dba0 tx=0x7fbf74017000 comp rx=0 tx=0).stop 2023-10-06T05:26:19.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.133+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/1485181348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf840c6c30 msgr2=0x7fbf841651c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.133+0000 7fbf8c7ff700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf840c6c30 0x7fbf841651c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000ecb0 tx=0x7fbf80009120 comp rx=0 tx=0).stop 2023-10-06T05:26:19.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.133+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/1485181348 shutdown_connections 2023-10-06T05:26:19.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.133+0000 7fbf8c7ff700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fbf680678f0 0x7fbf68069da0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.133+0000 7fbf8c7ff700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf8413a5b0 0x7fbf840c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.133+0000 7fbf8c7ff700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf840c6c30 0x7fbf841651c0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.133+0000 7fbf8c7ff700 1 --2- 172.21.15.31:0/1485181348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf8413a1e0 0x7fbf84151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.133+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/1485181348 >> 172.21.15.31:0/1485181348 conn(0x7fbf840cc0b0 msgr2=0x7fbf84139a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.134+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/1485181348 shutdown_connections 2023-10-06T05:26:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.134+0000 7fbf8c7ff700 1 -- 172.21.15.31:0/1485181348 wait complete. 2023-10-06T05:26:19.138 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 113 2023-10-06T05:26:19.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:19.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.349+0000 7f110d3ff700 1 Processor -- start 2023-10-06T05:26:19.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.349+0000 7f110d3ff700 1 -- start start 2023-10-06T05:26:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.351+0000 7f110d3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11080c7b60 0x7f1108069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.351+0000 7f110d3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11080c7f30 0x7f11080aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.351+0000 7f110d3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11080ca5b0 con 0x7f110806ef50 2023-10-06T05:26:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.351+0000 7f110d3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110806f740 con 0x7f11080c7b60 2023-10-06T05:26:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.351+0000 7f110d3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11080c9c80 con 0x7f11080c7f30 2023-10-06T05:26:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.351+0000 7f1106ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11080c7b60 0x7f1108069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.351+0000 7f11077fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f110806ef50 0x7f110813f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33532/0 (socket says 172.21.15.31:33532) 2023-10-06T05:26:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.351+0000 7f11077fe700 1 -- 172.21.15.31:0/1091285879 learned_addr learned my addr 172.21.15.31:0/1091285879 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.351+0000 7f1106ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f11080c7b60 0x7f1108069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46138/0 (socket says 172.21.15.31:46138) 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.351+0000 7f11067fc700 1 --2- 172.21.15.31:0/1091285879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11080c7f30 0x7f11080aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.352+0000 7f1106ffd700 1 -- 172.21.15.31:0/1091285879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11080c7f30 msgr2=0x7f11080aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.352+0000 7f1106ffd700 1 --2- 172.21.15.31:0/1091285879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11080c7f30 0x7f11080aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.352+0000 7f1106ffd700 1 -- 172.21.15.31:0/1091285879 >> v1:172.21.15.31:6789/0 conn(0x7f110806ef50 legacy=0x7f110813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.352+0000 7f1106ffd700 1 -- 172.21.15.31:0/1091285879 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11080c9f60 con 0x7f11080c7b60 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.352+0000 7f1105ffb700 1 -- 172.21.15.31:0/1091285879 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206116736 0 0) 0x7f11080ca5b0 con 0x7f110806ef50 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.352+0000 7f1106ffd700 1 --2- 172.21.15.31:0/1091285879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 0x7f1108069880 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f10f800bb70 tx=0x7f10f800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab3fb8871c04b3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.352+0000 7f1105ffb700 1 -- 172.21.15.31:0/1091285879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8005c10 con 0x7f11080c7b60 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.352+0000 7f1105ffb700 1 -- 172.21.15.31:0/1091285879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f8005d70 con 0x7f11080c7b60 2023-10-06T05:26:19.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.352+0000 7f1105ffb700 1 -- 172.21.15.31:0/1091285879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f800db60 con 0x7f11080c7b60 2023-10-06T05:26:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.353+0000 7f110d3ff700 1 -- 172.21.15.31:0/1091285879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 msgr2=0x7f1108069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.353+0000 7f110d3ff700 1 --2- 172.21.15.31:0/1091285879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 0x7f1108069880 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f10f800bb70 tx=0x7f10f800b3e0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.353+0000 7f110d3ff700 1 -- 172.21.15.31:0/1091285879 shutdown_connections 2023-10-06T05:26:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.353+0000 7f110d3ff700 1 --2- 172.21.15.31:0/1091285879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 0x7f1108069880 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.353+0000 7f110d3ff700 1 --2- 172.21.15.31:0/1091285879 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11080c7f30 0x7f11080aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.353+0000 7f110d3ff700 1 -- 172.21.15.31:0/1091285879 >> 172.21.15.31:0/1091285879 conn(0x7f11080cc0b0 msgr2=0x7f11080cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.353+0000 7f110d3ff700 1 -- 172.21.15.31:0/1091285879 shutdown_connections 2023-10-06T05:26:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.353+0000 7f110d3ff700 1 -- 172.21.15.31:0/1091285879 wait complete. 2023-10-06T05:26:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.354+0000 7f110d3ff700 1 Processor -- start 2023-10-06T05:26:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.354+0000 7f110d3ff700 1 -- start start 2023-10-06T05:26:19.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.355+0000 7f110d3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f110806ef50 0x7f110813eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.355+0000 7f110d3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 0x7f11080af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.355+0000 7f110d3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11080c7f30 0x7f11080afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.355+0000 7f110d3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11080ae450 con 0x7f11080c7b60 2023-10-06T05:26:19.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.355+0000 7f110d3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110813f0b0 con 0x7f11080c7f30 2023-10-06T05:26:19.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.355+0000 7f110d3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11080ae170 con 0x7f110806ef50 2023-10-06T05:26:19.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11067fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 0x7f11080af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11067fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 0x7f11080af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46146/0 (socket says 172.21.15.31:46146) 2023-10-06T05:26:19.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11067fc700 1 -- 172.21.15.31:0/828533256 learned_addr learned my addr 172.21.15.31:0/828533256 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:19.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f1106ffd700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f110806ef50 0x7f110813eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11077fe700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11080c7f30 0x7f11080afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11067fc700 1 -- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f110806ef50 msgr2=0x7f110813eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11067fc700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f110806ef50 0x7f110813eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11067fc700 1 -- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11080c7f30 msgr2=0x7f11080afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11067fc700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11080c7f30 0x7f11080afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11067fc700 1 -- 172.21.15.31:0/828533256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f800b660 con 0x7f11080c7b60 2023-10-06T05:26:19.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11077fe700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11080c7f30 0x7f11080afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f11067fc700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 0x7f11080af930 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00ec50 tx=0x7f10fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.356+0000 7f10eeffd700 1 -- 172.21.15.31:0/828533256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00c850 con 0x7f11080c7b60 2023-10-06T05:26:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.357+0000 7f10eeffd700 1 -- 172.21.15.31:0/828533256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc00c9b0 con 0x7f11080c7b60 2023-10-06T05:26:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.357+0000 7f10eeffd700 1 -- 172.21.15.31:0/828533256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00cd70 con 0x7f11080c7b60 2023-10-06T05:26:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.357+0000 7f110d3ff700 1 -- 172.21.15.31:0/828533256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11081598b0 con 0x7f11080c7b60 2023-10-06T05:26:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.357+0000 7f110d3ff700 1 -- 172.21.15.31:0/828533256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1108159e70 con 0x7f11080c7b60 2023-10-06T05:26:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.358+0000 7f110d3ff700 1 -- 172.21.15.31:0/828533256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110815a1e0 con 0x7f11080c7b60 2023-10-06T05:26:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.359+0000 7f10eeffd700 1 -- 172.21.15.31:0/828533256 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f10fc013680 con 0x7f11080c7b60 2023-10-06T05:26:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.360+0000 7f10eeffd700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10f40679e0 0x7f10f4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.361+0000 7f10eeffd700 1 -- 172.21.15.31:0/828533256 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f10fc082ef0 con 0x7f11080c7b60 2023-10-06T05:26:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.361+0000 7f1106ffd700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10f40679e0 0x7f10f4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.362+0000 7f1106ffd700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10f40679e0 0x7f10f4069e90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b420 tx=0x7f10f801a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.364+0000 7f10eeffd700 1 -- 172.21.15.31:0/828533256 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10fc051b30 con 0x7f11080c7b60 2023-10-06T05:26:19.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.576+0000 7f110d3ff700 1 -- 172.21.15.31:0/828533256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1108159710 con 0x7f11080c7b60 2023-10-06T05:26:19.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.577+0000 7f10eeffd700 1 -- 172.21.15.31:0/828533256 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+16972 (secure 0 0 0) 0x7f1108159710 con 0x7f11080c7b60 2023-10-06T05:26:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:19.581 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":163,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:26:03.756620+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:00.480986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:01.238722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:19.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 -- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10f40679e0 msgr2=0x7f10f4069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10f40679e0 0x7f10f4069e90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b420 tx=0x7f10f801a000 comp rx=0 tx=0).stop 2023-10-06T05:26:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 -- 172.21.15.31:0/828533256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 msgr2=0x7f11080af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 0x7f11080af930 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00ec50 tx=0x7f10fc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 -- 172.21.15.31:0/828533256 shutdown_connections 2023-10-06T05:26:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f10f40679e0 0x7f10f4069e90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11080c7f30 0x7f11080afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f110806ef50 0x7f110813eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 --2- 172.21.15.31:0/828533256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11080c7b60 0x7f11080af930 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 -- 172.21.15.31:0/828533256 >> 172.21.15.31:0/828533256 conn(0x7f11080cc0b0 msgr2=0x7f11080caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 -- 172.21.15.31:0/828533256 shutdown_connections 2023-10-06T05:26:19.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.579+0000 7f110d3ff700 1 -- 172.21.15.31:0/828533256 wait complete. 2023-10-06T05:26:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T05:16:53.806841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T05:16:54.115786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:26:00.480986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:26:01.238722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:26:19.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:26:19.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.797+0000 7ffa427ff700 1 Processor -- start 2023-10-06T05:26:19.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.797+0000 7ffa427ff700 1 -- start start 2023-10-06T05:26:19.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.799+0000 7ffa427ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffa3c153d40 0x7ffa3c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.799+0000 7ffa427ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa3c0af890 0x7ffa3c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.799+0000 7ffa427ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa3c0b0e40 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.799+0000 7ffa427ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa3c0afc90 con 0x7ffa3c0af890 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.799+0000 7ffa427ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa3c0ad690 con 0x7ffa3c153d40 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.799+0000 7ffa40d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa3c0af890 0x7ffa3c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.799+0000 7ffa40d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffa3c0af890 0x7ffa3c0adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46150/0 (socket says 172.21.15.31:46150) 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.799+0000 7ffa40d9c700 1 -- 172.21.15.31:0/568668486 learned_addr learned my addr 172.21.15.31:0/568668486 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.799+0000 7ffa3bfff700 1 --2- 172.21.15.31:0/568668486 >> v2:172.21.15.62:3300/0 conn(0x7ffa3c153d40 0x7ffa3c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.800+0000 7ffa40d9c700 1 -- 172.21.15.31:0/568668486 >> v2:172.21.15.62:3300/0 conn(0x7ffa3c153d40 msgr2=0x7ffa3c0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.800+0000 7ffa40d9c700 1 --2- 172.21.15.31:0/568668486 >> v2:172.21.15.62:3300/0 conn(0x7ffa3c153d40 0x7ffa3c0b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.800+0000 7ffa40d9c700 1 -- 172.21.15.31:0/568668486 >> v1:172.21.15.62:6789/0 conn(0x7ffa3c0b09c0 legacy=0x7ffa3c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.800+0000 7ffa40d9c700 1 -- 172.21.15.31:0/568668486 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c0acd60 con 0x7ffa3c0af890 2023-10-06T05:26:19.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.800+0000 7ffa40d9c700 1 --2- 172.21.15.31:0/568668486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0af890 0x7ffa3c0adb20 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c06d1d0 tx=0x7ffa34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12df40a9b2c9d0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.800+0000 7ffa3affd700 1 -- 172.21.15.31:0/568668486 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400e070 con 0x7ffa3c0af890 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.800+0000 7ffa3affd700 1 -- 172.21.15.31:0/568668486 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3400ba50 con 0x7ffa3c0af890 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.800+0000 7ffa3affd700 1 -- 172.21.15.31:0/568668486 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa340092e0 con 0x7ffa3c0af890 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.801+0000 7ffa427ff700 1 -- 172.21.15.31:0/568668486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0af890 msgr2=0x7ffa3c0adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.801+0000 7ffa427ff700 1 --2- 172.21.15.31:0/568668486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0af890 0x7ffa3c0adb20 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c06d1d0 tx=0x7ffa34009840 comp rx=0 tx=0).stop 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.801+0000 7ffa427ff700 1 -- 172.21.15.31:0/568668486 shutdown_connections 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.801+0000 7ffa427ff700 1 --2- 172.21.15.31:0/568668486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0af890 0x7ffa3c0adb20 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.801+0000 7ffa427ff700 1 --2- 172.21.15.31:0/568668486 >> v2:172.21.15.62:3300/0 conn(0x7ffa3c153d40 0x7ffa3c0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.801+0000 7ffa427ff700 1 -- 172.21.15.31:0/568668486 >> 172.21.15.31:0/568668486 conn(0x7ffa3c067560 msgr2=0x7ffa3c067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.801+0000 7ffa427ff700 1 -- 172.21.15.31:0/568668486 shutdown_connections 2023-10-06T05:26:19.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.801+0000 7ffa427ff700 1 -- 172.21.15.31:0/568668486 wait complete. 2023-10-06T05:26:19.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.802+0000 7ffa427ff700 1 Processor -- start 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.803+0000 7ffa427ff700 1 -- start start 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.803+0000 7ffa427ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa3c0af890 0x7ffa3c15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.803+0000 7ffa427ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0b09c0 0x7ffa3c150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.803+0000 7ffa427ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa3c153d40 0x7ffa3c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.803+0000 7ffa427ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c15c950 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.803+0000 7ffa427ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c156df0 con 0x7ffa3c0af890 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.803+0000 7ffa427ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c1570d0 con 0x7ffa3c153d40 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0b09c0 0x7ffa3c150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0b09c0 0x7ffa3c150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46160/0 (socket says 172.21.15.31:46160) 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa3b7fe700 1 -- 172.21.15.31:0/3931154413 learned_addr learned my addr 172.21.15.31:0/3931154413 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa3bfff700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa3c0af890 0x7ffa3c15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa40d9c700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa3c153d40 0x7ffa3c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa3b7fe700 1 -- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa3c153d40 msgr2=0x7ffa3c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa3b7fe700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa3c153d40 0x7ffa3c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa3b7fe700 1 -- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa3c0af890 msgr2=0x7ffa3c15a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa3b7fe700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa3c0af890 0x7ffa3c15a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa3b7fe700 1 -- 172.21.15.31:0/3931154413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa34009ae0 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.804+0000 7ffa3b7fe700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0b09c0 0x7ffa3c150800 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000c590 tx=0x7ffa3000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.805+0000 7ffa23fff700 1 -- 172.21.15.31:0/3931154413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011590 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.805+0000 7ffa23fff700 1 -- 172.21.15.31:0/3931154413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa30011710 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.805+0000 7ffa23fff700 1 -- 172.21.15.31:0/3931154413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011ad0 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.805+0000 7ffa427ff700 1 -- 172.21.15.31:0/3931154413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3c1573b0 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.805+0000 7ffa427ff700 1 -- 172.21.15.31:0/3931154413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3c157740 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.806+0000 7ffa427ff700 1 -- 172.21.15.31:0/3931154413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3c04c380 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.807+0000 7ffa23fff700 1 -- 172.21.15.31:0/3931154413 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7ffa3001b3b0 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.808+0000 7ffa23fff700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa280679e0 0x7ffa28069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:19.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.808+0000 7ffa23fff700 1 -- 172.21.15.31:0/3931154413 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7ffa30082410 con 0x7ffa3c0b09c0 2023-10-06T05:26:19.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.809+0000 7ffa3bfff700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa280679e0 0x7ffa28069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:19.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.809+0000 7ffa3bfff700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa280679e0 0x7ffa28069e90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c006fd0 tx=0x7ffa2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:19.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:19.812+0000 7ffa23fff700 1 -- 172.21.15.31:0/3931154413 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa30050e20 con 0x7ffa3c0b09c0 2023-10-06T05:26:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.016+0000 7ffa427ff700 1 -- 172.21.15.31:0/3931154413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffa3c0b0e40 con 0x7ffa3c0b09c0 2023-10-06T05:26:20.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.016+0000 7ffa23fff700 1 -- 172.21.15.31:0/3931154413 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v163) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffa3c0b0e40 con 0x7ffa3c0b09c0 2023-10-06T05:26:20.018 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 8 2023-10-06T05:26:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.018+0000 7ffa427ff700 1 -- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa280679e0 msgr2=0x7ffa28069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.018+0000 7ffa427ff700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa280679e0 0x7ffa28069e90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c006fd0 tx=0x7ffa2c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.018+0000 7ffa427ff700 1 -- 172.21.15.31:0/3931154413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0b09c0 msgr2=0x7ffa3c150800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.018+0000 7ffa427ff700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0b09c0 0x7ffa3c150800 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000c590 tx=0x7ffa3000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.019+0000 7ffa427ff700 1 -- 172.21.15.31:0/3931154413 shutdown_connections 2023-10-06T05:26:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.019+0000 7ffa427ff700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ffa280679e0 0x7ffa28069e90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.019+0000 7ffa427ff700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa3c153d40 0x7ffa3c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.019+0000 7ffa427ff700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa3c0af890 0x7ffa3c15a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.019+0000 7ffa427ff700 1 --2- 172.21.15.31:0/3931154413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c0b09c0 0x7ffa3c150800 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.019+0000 7ffa427ff700 1 -- 172.21.15.31:0/3931154413 >> 172.21.15.31:0/3931154413 conn(0x7ffa3c067560 msgr2=0x7ffa3c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.019+0000 7ffa427ff700 1 -- 172.21.15.31:0/3931154413 shutdown_connections 2023-10-06T05:26:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.019+0000 7ffa427ff700 1 -- 172.21.15.31:0/3931154413 wait complete. 2023-10-06T05:26:20.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:26:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.238+0000 7f85347ff700 1 Processor -- start 2023-10-06T05:26:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.239+0000 7f85347ff700 1 -- start start 2023-10-06T05:26:20.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.240+0000 7f85347ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c0c8f40 0x7f852c068930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.240+0000 7f85347ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c0adb00 con 0x7f852c159540 2023-10-06T05:26:20.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.240+0000 7f85347ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c068f10 con 0x7f852c06a2a0 2023-10-06T05:26:20.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.240+0000 7f85347ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0691f0 con 0x7f852c0c8f40 2023-10-06T05:26:20.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.240+0000 7f853259b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f852c159540 0x7f852c0c8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33536/0 (socket says 172.21.15.31:33536) 2023-10-06T05:26:20.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.240+0000 7f853259b700 1 -- 172.21.15.31:0/880972264 learned_addr learned my addr 172.21.15.31:0/880972264 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:20.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.240+0000 7f8531d9a700 1 --2- 172.21.15.31:0/880972264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c0c8f40 0x7f852c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.241+0000 7f8531599700 1 -- 172.21.15.31:0/880972264 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350866484 0 0) 0x7f852c0adb00 con 0x7f852c159540 2023-10-06T05:26:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.241+0000 7f8531599700 1 -- 172.21.15.31:0/880972264 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8518003570 con 0x7f852c159540 2023-10-06T05:26:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.241+0000 7f8531599700 1 -- 172.21.15.31:0/880972264 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043653545 0 0) 0x7f852c068f10 con 0x7f852c06a2a0 2023-10-06T05:26:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.241+0000 7f8531599700 1 -- 172.21.15.31:0/880972264 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f852c0adb00 con 0x7f852c06a2a0 2023-10-06T05:26:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.241+0000 7f8531d9a700 1 -- 172.21.15.31:0/880972264 >> v1:172.21.15.62:6789/0 conn(0x7f852c06a2a0 legacy=0x7f852c0adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.241+0000 7f8531d9a700 1 -- 172.21.15.31:0/880972264 >> v1:172.21.15.31:6789/0 conn(0x7f852c159540 legacy=0x7f852c0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.241+0000 7f8531d9a700 1 -- 172.21.15.31:0/880972264 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0694d0 con 0x7f852c0c8f40 2023-10-06T05:26:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.241+0000 7f8531d9a700 1 --2- 172.21.15.31:0/880972264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c0c8f40 0x7f852c068930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f851c009910 tx=0x7f851c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f74565107cca780b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.241+0000 7f8531599700 1 -- 172.21.15.31:0/880972264 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c011630 con 0x7f852c0c8f40 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.242+0000 7f8531599700 1 -- 172.21.15.31:0/880972264 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c011790 con 0x7f852c0c8f40 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.242+0000 7f8531599700 1 -- 172.21.15.31:0/880972264 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c011b30 con 0x7f852c0c8f40 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.242+0000 7f85347ff700 1 -- 172.21.15.31:0/880972264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c0c8f40 msgr2=0x7f852c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.242+0000 7f85347ff700 1 --2- 172.21.15.31:0/880972264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c0c8f40 0x7f852c068930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f851c009910 tx=0x7f851c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.242+0000 7f85347ff700 1 -- 172.21.15.31:0/880972264 shutdown_connections 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.242+0000 7f85347ff700 1 --2- 172.21.15.31:0/880972264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c0c8f40 0x7f852c068930 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.242+0000 7f85347ff700 1 -- 172.21.15.31:0/880972264 >> 172.21.15.31:0/880972264 conn(0x7f852c0ce290 msgr2=0x7f852c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.242+0000 7f85347ff700 1 -- 172.21.15.31:0/880972264 shutdown_connections 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.242+0000 7f85347ff700 1 -- 172.21.15.31:0/880972264 wait complete. 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.244+0000 7f85347ff700 1 Processor -- start 2023-10-06T05:26:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.244+0000 7f85347ff700 1 -- start start 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f85347ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f852c06a2a0 0x7f852c0c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f85347ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f852c0c8f40 0x7f852c16a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f85347ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c159540 0x7f852c0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f85347ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0c4910 con 0x7f852c06a2a0 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f85347ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c16daa0 con 0x7f852c0c8f40 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f85347ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c16aa90 con 0x7f852c159540 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f853259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f852c06a2a0 0x7f852c0c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f853259b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f852c06a2a0 0x7f852c0c43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46164/0 (socket says 172.21.15.31:46164) 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f853259b700 1 -- 172.21.15.31:0/3401957648 learned_addr learned my addr 172.21.15.31:0/3401957648 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f8531d9a700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f852c0c8f40 0x7f852c16a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f8532d9c700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c159540 0x7f852c0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f853259b700 1 -- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c159540 msgr2=0x7f852c0caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f853259b700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c159540 0x7f852c0caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f853259b700 1 -- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f852c0c8f40 msgr2=0x7f852c16a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f853259b700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f852c0c8f40 0x7f852c16a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f853259b700 1 -- 172.21.15.31:0/3401957648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c009420 con 0x7f852c06a2a0 2023-10-06T05:26:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.245+0000 7f8532d9c700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c159540 0x7f852c0caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.246+0000 7f853259b700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f852c06a2a0 0x7f852c0c43d0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f852800c7c0 tx=0x7f8528007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.246+0000 7f85227fc700 1 -- 172.21.15.31:0/3401957648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528005c00 con 0x7f852c06a2a0 2023-10-06T05:26:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.246+0000 7f85227fc700 1 -- 172.21.15.31:0/3401957648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8528005d60 con 0x7f852c06a2a0 2023-10-06T05:26:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.246+0000 7f85227fc700 1 -- 172.21.15.31:0/3401957648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528007300 con 0x7f852c06a2a0 2023-10-06T05:26:20.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.246+0000 7f85347ff700 1 -- 172.21.15.31:0/3401957648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c16d1b0 con 0x7f852c06a2a0 2023-10-06T05:26:20.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.246+0000 7f85347ff700 1 -- 172.21.15.31:0/3401957648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c0b3e00 con 0x7f852c06a2a0 2023-10-06T05:26:20.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.247+0000 7f85347ff700 1 -- 172.21.15.31:0/3401957648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c16dd80 con 0x7f852c06a2a0 2023-10-06T05:26:20.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.248+0000 7f85227fc700 1 -- 172.21.15.31:0/3401957648 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f852801b3b0 con 0x7f852c06a2a0 2023-10-06T05:26:20.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.249+0000 7f85227fc700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85100679e0 0x7f8510069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.249+0000 7f85227fc700 1 -- 172.21.15.31:0/3401957648 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f85280826d0 con 0x7f852c06a2a0 2023-10-06T05:26:20.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.249+0000 7f8531d9a700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85100679e0 0x7f8510069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.250+0000 7f8531d9a700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85100679e0 0x7f8510069e90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f852c15ef70 tx=0x7f851c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.253+0000 7f85227fc700 1 -- 172.21.15.31:0/3401957648 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85280510e0 con 0x7f852c06a2a0 2023-10-06T05:26:20.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.455+0000 7f85347ff700 1 -- 172.21.15.31:0/3401957648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f852c16d340 con 0x7f852c06a2a0 2023-10-06T05:26:20.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.455+0000 7f85227fc700 1 -- 172.21.15.31:0/3401957648 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8528054700 con 0x7f852c06a2a0 2023-10-06T05:26:20.458 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-10-06T05:26:20.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.457+0000 7f85347ff700 1 -- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85100679e0 msgr2=0x7f8510069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.457+0000 7f85347ff700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85100679e0 0x7f8510069e90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f852c15ef70 tx=0x7f851c017000 comp rx=0 tx=0).stop 2023-10-06T05:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.457+0000 7f85347ff700 1 -- 172.21.15.31:0/3401957648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f852c06a2a0 msgr2=0x7f852c0c43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.457+0000 7f85347ff700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f852c06a2a0 0x7f852c0c43d0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f852800c7c0 tx=0x7f8528007ea0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.457+0000 7f85347ff700 1 -- 172.21.15.31:0/3401957648 shutdown_connections 2023-10-06T05:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.457+0000 7f85347ff700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f85100679e0 0x7f8510069e90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.457+0000 7f85347ff700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f852c159540 0x7f852c0caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.457+0000 7f85347ff700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f852c06a2a0 0x7f852c0c43d0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.457+0000 7f85347ff700 1 --2- 172.21.15.31:0/3401957648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f852c0c8f40 0x7f852c16a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.457+0000 7f85347ff700 1 -- 172.21.15.31:0/3401957648 >> 172.21.15.31:0/3401957648 conn(0x7f852c0ce290 msgr2=0x7f852c15b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:20.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.458+0000 7f85347ff700 1 -- 172.21.15.31:0/3401957648 shutdown_connections 2023-10-06T05:26:20.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.458+0000 7f85347ff700 1 -- 172.21.15.31:0/3401957648 wait complete. 2023-10-06T05:26:20.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:26:20.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.673+0000 7fd78b9ff700 1 Processor -- start 2023-10-06T05:26:20.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.674+0000 7fd78b9ff700 1 -- start start 2023-10-06T05:26:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.675+0000 7fd78b9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd78415ca40 0x7fd784068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.675+0000 7fd78b9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd7840c8f40 0x7fd7840adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.675+0000 7fd78b9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd784068da0 con 0x7fd78406a300 2023-10-06T05:26:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.675+0000 7fd78b9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7840cabc0 con 0x7fd78415ca40 2023-10-06T05:26:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.675+0000 7fd78b9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd784068bb0 con 0x7fd7840c8f40 2023-10-06T05:26:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.675+0000 7fd78979b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd78415ca40 0x7fd784068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.675+0000 7fd78979b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd78415ca40 0x7fd784068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46166/0 (socket says 172.21.15.31:46166) 2023-10-06T05:26:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.675+0000 7fd78979b700 1 -- 172.21.15.31:0/1272468392 learned_addr learned my addr 172.21.15.31:0/1272468392 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.675+0000 7fd789f9c700 1 --2- 172.21.15.31:0/1272468392 >> v2:172.21.15.62:3300/0 conn(0x7fd7840c8f40 0x7fd7840adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.676+0000 7fd78979b700 1 -- 172.21.15.31:0/1272468392 >> v2:172.21.15.62:3300/0 conn(0x7fd7840c8f40 msgr2=0x7fd7840adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.676+0000 7fd78979b700 1 --2- 172.21.15.31:0/1272468392 >> v2:172.21.15.62:3300/0 conn(0x7fd7840c8f40 0x7fd7840adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.676+0000 7fd78979b700 1 -- 172.21.15.31:0/1272468392 >> v1:172.21.15.62:6789/0 conn(0x7fd78406a300 legacy=0x7fd7840c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.676+0000 7fd78979b700 1 -- 172.21.15.31:0/1272468392 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd784069080 con 0x7fd78415ca40 2023-10-06T05:26:20.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.676+0000 7fd78979b700 1 --2- 172.21.15.31:0/1272468392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 0x7fd784068670 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fd7840c19b0 tx=0x7fd780009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922230d3bce0fab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.676+0000 7fd77bfff700 1 -- 172.21.15.31:0/1272468392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd780009ea0 con 0x7fd78415ca40 2023-10-06T05:26:20.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.676+0000 7fd77bfff700 1 -- 172.21.15.31:0/1272468392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78000b820 con 0x7fd78415ca40 2023-10-06T05:26:20.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.676+0000 7fd77bfff700 1 -- 172.21.15.31:0/1272468392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000bbc0 con 0x7fd78415ca40 2023-10-06T05:26:20.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.677+0000 7fd78b9ff700 1 -- 172.21.15.31:0/1272468392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 msgr2=0x7fd784068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.677+0000 7fd78b9ff700 1 --2- 172.21.15.31:0/1272468392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 0x7fd784068670 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fd7840c19b0 tx=0x7fd780009250 comp rx=0 tx=0).stop 2023-10-06T05:26:20.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.677+0000 7fd78b9ff700 1 -- 172.21.15.31:0/1272468392 shutdown_connections 2023-10-06T05:26:20.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.677+0000 7fd78b9ff700 1 --2- 172.21.15.31:0/1272468392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 0x7fd784068670 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.677+0000 7fd78b9ff700 1 --2- 172.21.15.31:0/1272468392 >> v2:172.21.15.62:3300/0 conn(0x7fd7840c8f40 0x7fd7840adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.677+0000 7fd78b9ff700 1 -- 172.21.15.31:0/1272468392 >> 172.21.15.31:0/1272468392 conn(0x7fd7840ce290 msgr2=0x7fd7840ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:20.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.677+0000 7fd78b9ff700 1 -- 172.21.15.31:0/1272468392 shutdown_connections 2023-10-06T05:26:20.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.677+0000 7fd78b9ff700 1 -- 172.21.15.31:0/1272468392 wait complete. 2023-10-06T05:26:20.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.678+0000 7fd78b9ff700 1 Processor -- start 2023-10-06T05:26:20.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.679+0000 7fd78b9ff700 1 -- start start 2023-10-06T05:26:20.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.679+0000 7fd78b9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78406a300 0x7fd784159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.679+0000 7fd78b9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840c8f40 0x7fd7840adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.679+0000 7fd78b9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 0x7fd784169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.679+0000 7fd78b9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840ad740 con 0x7fd78415ca40 2023-10-06T05:26:20.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.679+0000 7fd78b9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd784159fc0 con 0x7fd78406a300 2023-10-06T05:26:20.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.679+0000 7fd78b9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78416a780 con 0x7fd7840c8f40 2023-10-06T05:26:20.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd789f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 0x7fd784169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd789f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 0x7fd784169ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46174/0 (socket says 172.21.15.31:46174) 2023-10-06T05:26:20.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd789f9c700 1 -- 172.21.15.31:0/114546517 learned_addr learned my addr 172.21.15.31:0/114546517 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:20.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd78979b700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78406a300 0x7fd784159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd788f9a700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840c8f40 0x7fd7840adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd789f9c700 1 -- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840c8f40 msgr2=0x7fd7840adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd789f9c700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840c8f40 0x7fd7840adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd789f9c700 1 -- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78406a300 msgr2=0x7fd784159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd789f9c700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78406a300 0x7fd784159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd789f9c700 1 -- 172.21.15.31:0/114546517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7800094f0 con 0x7fd78415ca40 2023-10-06T05:26:20.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd788f9a700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840c8f40 0x7fd7840adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:20.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd78979b700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78406a300 0x7fd784159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:20.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.680+0000 7fd789f9c700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 0x7fd784169ba0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009ef0 tx=0x7fd77c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.681+0000 7fd7797fa700 1 -- 172.21.15.31:0/114546517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00cc50 con 0x7fd78415ca40 2023-10-06T05:26:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.681+0000 7fd7797fa700 1 -- 172.21.15.31:0/114546517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77c00f040 con 0x7fd78415ca40 2023-10-06T05:26:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.681+0000 7fd78b9ff700 1 -- 172.21.15.31:0/114546517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78416aa60 con 0x7fd78415ca40 2023-10-06T05:26:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.681+0000 7fd7797fa700 1 -- 172.21.15.31:0/114546517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c014480 con 0x7fd78415ca40 2023-10-06T05:26:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.681+0000 7fd78b9ff700 1 -- 172.21.15.31:0/114546517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7840c5580 con 0x7fd78415ca40 2023-10-06T05:26:20.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.682+0000 7fd78b9ff700 1 -- 172.21.15.31:0/114546517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78404c380 con 0x7fd78415ca40 2023-10-06T05:26:20.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.683+0000 7fd7797fa700 1 -- 172.21.15.31:0/114546517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fd77c014830 con 0x7fd78415ca40 2023-10-06T05:26:20.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.684+0000 7fd7797fa700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd7680679e0 0x7fd768069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:20.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.684+0000 7fd7797fa700 1 -- 172.21.15.31:0/114546517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7fd77c083f90 con 0x7fd78415ca40 2023-10-06T05:26:20.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.684+0000 7fd78979b700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd7680679e0 0x7fd768069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:20.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.685+0000 7fd78979b700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd7680679e0 0x7fd768069e90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd7840c19b0 tx=0x7fd780017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:20.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.688+0000 7fd7797fa700 1 -- 172.21.15.31:0/114546517 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd77c052900 con 0x7fd78415ca40 2023-10-06T05:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.890+0000 7fd78b9ff700 1 -- 172.21.15.31:0/114546517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7840ca1a0 con 0x7fd78415ca40 2023-10-06T05:26:20.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.890+0000 7fd7797fa700 1 -- 172.21.15.31:0/114546517 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v163) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd77c055f20 con 0x7fd78415ca40 2023-10-06T05:26:20.893 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-10-06T05:26:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 -- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd7680679e0 msgr2=0x7fd768069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd7680679e0 0x7fd768069e90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd7840c19b0 tx=0x7fd780017000 comp rx=0 tx=0).stop 2023-10-06T05:26:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 -- 172.21.15.31:0/114546517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 msgr2=0x7fd784169ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 0x7fd784169ba0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009ef0 tx=0x7fd77c00b040 comp rx=0 tx=0).stop 2023-10-06T05:26:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 -- 172.21.15.31:0/114546517 shutdown_connections 2023-10-06T05:26:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd7680679e0 0x7fd768069e90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd78415ca40 0x7fd784169ba0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78406a300 0x7fd784159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 --2- 172.21.15.31:0/114546517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840c8f40 0x7fd7840adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:20.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 -- 172.21.15.31:0/114546517 >> 172.21.15.31:0/114546517 conn(0x7fd7840ce290 msgr2=0x7fd7840b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:20.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 -- 172.21.15.31:0/114546517 shutdown_connections 2023-10-06T05:26:20.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:20.893+0000 7fd78b9ff700 1 -- 172.21.15.31:0/114546517 wait complete. 2023-10-06T05:26:20.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:26:21.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.110+0000 7f0ae4bff700 1 Processor -- start 2023-10-06T05:26:21.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.110+0000 7f0ae4bff700 1 -- start start 2023-10-06T05:26:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.111+0000 7f0ae4bff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ae00caeb0 0x7f0ae00cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.111+0000 7f0ae4bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae0069380 con 0x7f0ae00b10c0 2023-10-06T05:26:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.111+0000 7f0ae4bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae0068770 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.111+0000 7f0ae4bff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae0068a50 con 0x7f0ae00caeb0 2023-10-06T05:26:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.112+0000 7f0ade59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ae00caeb0 0x7f0ae00cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.112+0000 7f0addd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0ae00b10c0 0x7f0ae0159230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33548/0 (socket says 172.21.15.31:33548) 2023-10-06T05:26:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.112+0000 7f0addd9b700 1 -- 172.21.15.31:0/3195313445 learned_addr learned my addr 172.21.15.31:0/3195313445 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:21.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.112+0000 7f0ade59c700 1 -- 172.21.15.31:0/3195313445 >> v1:172.21.15.62:6789/0 conn(0x7f0ae00cc1d0 legacy=0x7f0ae00afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.112+0000 7f0ade59c700 1 -- 172.21.15.31:0/3195313445 >> v1:172.21.15.31:6789/0 conn(0x7f0ae00b10c0 legacy=0x7f0ae0159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.112+0000 7f0ade59c700 1 -- 172.21.15.31:0/3195313445 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0068d30 con 0x7f0ae00caeb0 2023-10-06T05:26:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.112+0000 7f0add59a700 1 -- 172.21.15.31:0/3195313445 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440184440 0 0) 0x7f0ae0069380 con 0x7f0ae00b10c0 2023-10-06T05:26:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.112+0000 7f0ade59c700 1 --2- 172.21.15.31:0/3195313445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00caeb0 0x7f0ae00cd0f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00098d0 tx=0x7f0ad0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7d352acc6920b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.113+0000 7f0add59a700 1 -- 172.21.15.31:0/3195313445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0009e70 con 0x7f0ae00caeb0 2023-10-06T05:26:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.113+0000 7f0add59a700 1 -- 172.21.15.31:0/3195313445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad0005c10 con 0x7f0ae00caeb0 2023-10-06T05:26:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.113+0000 7f0add59a700 1 -- 172.21.15.31:0/3195313445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000b650 con 0x7f0ae00caeb0 2023-10-06T05:26:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.113+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3195313445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00caeb0 msgr2=0x7f0ae00cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.113+0000 7f0ae4bff700 1 --2- 172.21.15.31:0/3195313445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00caeb0 0x7f0ae00cd0f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00098d0 tx=0x7f0ad0009120 comp rx=0 tx=0).stop 2023-10-06T05:26:21.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.113+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3195313445 shutdown_connections 2023-10-06T05:26:21.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.113+0000 7f0ae4bff700 1 --2- 172.21.15.31:0/3195313445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00caeb0 0x7f0ae00cd0f0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.113+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3195313445 >> 172.21.15.31:0/3195313445 conn(0x7f0ae0097550 msgr2=0x7f0ae00639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:21.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.113+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3195313445 shutdown_connections 2023-10-06T05:26:21.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.113+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3195313445 wait complete. 2023-10-06T05:26:21.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.115+0000 7f0ae4bff700 1 Processor -- start 2023-10-06T05:26:21.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.115+0000 7f0ae4bff700 1 -- start start 2023-10-06T05:26:21.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0ae4bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae00b10c0 0x7f0ae00c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0ae4bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae00caeb0 0x7f0ae015f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0ae4bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00cc1d0 0x7f0ae0151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0ae4bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0152bc0 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0ae4bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae015dd20 con 0x7f0ae00caeb0 2023-10-06T05:26:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0ae4bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae015e000 con 0x7f0ae00b10c0 2023-10-06T05:26:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0aded9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00cc1d0 0x7f0ae0151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0aded9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00cc1d0 0x7f0ae0151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46196/0 (socket says 172.21.15.31:46196) 2023-10-06T05:26:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0aded9d700 1 -- 172.21.15.31:0/3413886224 learned_addr learned my addr 172.21.15.31:0/3413886224 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:21.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0addd9b700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae00caeb0 0x7f0ae015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.116+0000 7f0ade59c700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae00b10c0 0x7f0ae00c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0aded9d700 1 -- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae00b10c0 msgr2=0x7f0ae00c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0aded9d700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae00b10c0 0x7f0ae00c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0aded9d700 1 -- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae00caeb0 msgr2=0x7f0ae015f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0aded9d700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae00caeb0 0x7f0ae015f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0aded9d700 1 -- 172.21.15.31:0/3413886224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad00093c0 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0aded9d700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00cc1d0 0x7f0ae0151710 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800c840 tx=0x7f0ad8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0ace7fc700 1 -- 172.21.15.31:0/3413886224 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8012630 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0ace7fc700 1 -- 172.21.15.31:0/3413886224 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad80127b0 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3413886224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae015e2e0 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0ace7fc700 1 -- 172.21.15.31:0/3413886224 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8012b70 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.117+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3413886224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae00c1dd0 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.118+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3413886224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae004c380 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.119+0000 7f0ace7fc700 1 -- 172.21.15.31:0/3413886224 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f0ad801a3a0 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.120+0000 7f0ace7fc700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ac00679e0 0x7f0ac0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.120+0000 7f0ace7fc700 1 -- 172.21.15.31:0/3413886224 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f0ad8083220 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.121+0000 7f0ade59c700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ac00679e0 0x7f0ac0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.121+0000 7f0ade59c700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ac00679e0 0x7f0ac0069e90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009160 tx=0x7f0ad0016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.124+0000 7f0ace7fc700 1 -- 172.21.15.31:0/3413886224 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ad8051c90 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.328+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3413886224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ae00c1fc0 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.328+0000 7f0ace7fc700 1 -- 172.21.15.31:0/3413886224 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v163) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0ad80552b0 con 0x7f0ae00cc1d0 2023-10-06T05:26:21.331 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-10-06T05:26:21.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.330+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ac00679e0 msgr2=0x7f0ac0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ac00679e0 0x7f0ac0069e90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009160 tx=0x7f0ad0016000 comp rx=0 tx=0).stop 2023-10-06T05:26:21.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3413886224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00cc1d0 msgr2=0x7f0ae0151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00cc1d0 0x7f0ae0151710 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800c840 tx=0x7f0ad8001670 comp rx=0 tx=0).stop 2023-10-06T05:26:21.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3413886224 shutdown_connections 2023-10-06T05:26:21.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f0ac00679e0 0x7f0ac0069e90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae00cc1d0 0x7f0ae0151710 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ae00b10c0 0x7f0ae00c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 --2- 172.21.15.31:0/3413886224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae00caeb0 0x7f0ae015f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3413886224 >> 172.21.15.31:0/3413886224 conn(0x7f0ae0097550 msgr2=0x7f0ae006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:21.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3413886224 shutdown_connections 2023-10-06T05:26:21.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.331+0000 7f0ae4bff700 1 -- 172.21.15.31:0/3413886224 wait complete. 2023-10-06T05:26:21.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:21.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.546+0000 7f21513ff700 1 Processor -- start 2023-10-06T05:26:21.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.546+0000 7f21513ff700 1 -- start start 2023-10-06T05:26:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.547+0000 7f21513ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f214c13a1e0 0x7f214c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.547+0000 7f21513ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c13a5b0 0x7f214c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.547+0000 7f21513ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214c0c71b0 con 0x7f214c0c6c30 2023-10-06T05:26:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.547+0000 7f21513ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214c0c7490 con 0x7f214c13a1e0 2023-10-06T05:26:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.547+0000 7f21513ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0c7770 con 0x7f214c13a5b0 2023-10-06T05:26:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f214b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f214c0c6c30 0x7f214c15afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33562/0 (socket says 172.21.15.31:33562) 2023-10-06T05:26:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f214b7fe700 1 -- 172.21.15.31:0/485061388 learned_addr learned my addr 172.21.15.31:0/485061388 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f214affd700 1 --2- 172.21.15.31:0/485061388 >> v2:172.21.15.62:3300/0 conn(0x7f214c13a1e0 0x7f214c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f214a7fc700 1 --2- 172.21.15.31:0/485061388 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c13a5b0 0x7f214c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f2149ffb700 1 -- 172.21.15.31:0/485061388 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358232989 0 0) 0x7f214c0c71b0 con 0x7f214c0c6c30 2023-10-06T05:26:21.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f2149ffb700 1 -- 172.21.15.31:0/485061388 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2134003570 con 0x7f214c0c6c30 2023-10-06T05:26:21.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f214affd700 1 -- 172.21.15.31:0/485061388 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c13a5b0 msgr2=0x7f214c06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:21.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f214affd700 1 --2- 172.21.15.31:0/485061388 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c13a5b0 0x7f214c06d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f214affd700 1 -- 172.21.15.31:0/485061388 >> v1:172.21.15.31:6789/0 conn(0x7f214c0c6c30 legacy=0x7f214c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f214affd700 1 -- 172.21.15.31:0/485061388 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c0c7a50 con 0x7f214c13a1e0 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.548+0000 7f214a7fc700 1 --2- 172.21.15.31:0/485061388 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c13a5b0 0x7f214c06d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.549+0000 7f214affd700 1 --2- 172.21.15.31:0/485061388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c13a1e0 0x7f214c0c66f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f213c0098d0 tx=0x7f213c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e8059e73f1d3d6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.549+0000 7f2149ffb700 1 -- 172.21.15.31:0/485061388 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c005c10 con 0x7f214c13a1e0 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.549+0000 7f2149ffb700 1 -- 172.21.15.31:0/485061388 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c005d70 con 0x7f214c13a1e0 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.549+0000 7f2149ffb700 1 -- 172.21.15.31:0/485061388 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c006260 con 0x7f214c13a1e0 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.549+0000 7f21513ff700 1 -- 172.21.15.31:0/485061388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c13a1e0 msgr2=0x7f214c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.549+0000 7f21513ff700 1 --2- 172.21.15.31:0/485061388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c13a1e0 0x7f214c0c66f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f213c0098d0 tx=0x7f213c009120 comp rx=0 tx=0).stop 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.550+0000 7f21513ff700 1 -- 172.21.15.31:0/485061388 shutdown_connections 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.550+0000 7f21513ff700 1 --2- 172.21.15.31:0/485061388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c13a1e0 0x7f214c0c66f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.550+0000 7f21513ff700 1 --2- 172.21.15.31:0/485061388 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c13a5b0 0x7f214c06d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.550+0000 7f21513ff700 1 -- 172.21.15.31:0/485061388 >> 172.21.15.31:0/485061388 conn(0x7f214c0cc0b0 msgr2=0x7f214c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:21.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.550+0000 7f21513ff700 1 -- 172.21.15.31:0/485061388 shutdown_connections 2023-10-06T05:26:21.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.550+0000 7f21513ff700 1 -- 172.21.15.31:0/485061388 wait complete. 2023-10-06T05:26:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.551+0000 7f21513ff700 1 Processor -- start 2023-10-06T05:26:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.551+0000 7f21513ff700 1 -- start start 2023-10-06T05:26:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.552+0000 7f21513ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c0c6c30 0x7f214c0cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.552+0000 7f21513ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c13a1e0 0x7f214c139a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.552+0000 7f21513ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f214c13a5b0 0x7f214c14f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.552+0000 7f21513ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0cb340 con 0x7f214c13a5b0 2023-10-06T05:26:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.552+0000 7f21513ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c156f50 con 0x7f214c13a1e0 2023-10-06T05:26:21.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.552+0000 7f21513ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214c156620 con 0x7f214c0c6c30 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.552+0000 7f214b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f214c13a5b0 0x7f214c14f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.552+0000 7f214b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f214c13a5b0 0x7f214c14f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46210/0 (socket says 172.21.15.31:46210) 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.552+0000 7f214b7fe700 1 -- 172.21.15.31:0/1258604547 learned_addr learned my addr 172.21.15.31:0/1258604547 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f214affd700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c0c6c30 0x7f214c0cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f214a7fc700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c13a1e0 0x7f214c139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f214b7fe700 1 -- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c0c6c30 msgr2=0x7f214c0cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f214b7fe700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c0c6c30 0x7f214c0cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f214b7fe700 1 -- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c13a1e0 msgr2=0x7f214c139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f214b7fe700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c13a1e0 0x7f214c139a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f214b7fe700 1 -- 172.21.15.31:0/1258604547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2140009420 con 0x7f214c13a5b0 2023-10-06T05:26:21.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f214b7fe700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f214c13a5b0 0x7f214c14f280 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f2144001460 tx=0x7f2144001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f2132ffd700 1 -- 172.21.15.31:0/1258604547 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214400ce80 con 0x7f214c13a5b0 2023-10-06T05:26:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f2132ffd700 1 -- 172.21.15.31:0/1258604547 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214400ab70 con 0x7f214c13a5b0 2023-10-06T05:26:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f2132ffd700 1 -- 172.21.15.31:0/1258604547 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21440070f0 con 0x7f214c13a5b0 2023-10-06T05:26:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.553+0000 7f21513ff700 1 -- 172.21.15.31:0/1258604547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c0093c0 con 0x7f214c13a5b0 2023-10-06T05:26:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.554+0000 7f21513ff700 1 -- 172.21.15.31:0/1258604547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214c156be0 con 0x7f214c13a5b0 2023-10-06T05:26:21.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.554+0000 7f21513ff700 1 -- 172.21.15.31:0/1258604547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214c1570e0 con 0x7f214c13a5b0 2023-10-06T05:26:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.555+0000 7f2132ffd700 1 -- 172.21.15.31:0/1258604547 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f2144007520 con 0x7f214c13a5b0 2023-10-06T05:26:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.557+0000 7f2132ffd700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21380679e0 0x7f2138069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.557+0000 7f2132ffd700 1 -- 172.21.15.31:0/1258604547 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f2144083410 con 0x7f214c13a5b0 2023-10-06T05:26:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.557+0000 7f214affd700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21380679e0 0x7f2138069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.558+0000 7f214affd700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21380679e0 0x7f2138069e90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f213c009160 tx=0x7f213c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:21.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.561+0000 7f2132ffd700 1 -- 172.21.15.31:0/1258604547 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21440520d0 con 0x7f214c13a5b0 2023-10-06T05:26:21.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.807+0000 7f21513ff700 1 -- 172.21.15.31:0/1258604547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f214c04c380 con 0x7f214c13a5b0 2023-10-06T05:26:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.808+0000 7f2132ffd700 1 -- 172.21.15.31:0/1258604547 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f21440556f0 con 0x7f214c13a5b0 2023-10-06T05:26:21.810 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:21.810 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":113,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4242992570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4242992570},{"type":"v1","addr":"172.21.15.31:6835","nonce":4242992570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":7706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2405114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2405114362},{"type":"v1","addr":"172.21.15.62:6834","nonce":2405114362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2676480318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2676480318},{"type":"v1","addr":"172.21.15.62:6835","nonce":2676480318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:02.260523+0000","modified":"2023-10-06T05:26:06.149545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9249":{"gid":9249,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3001391649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3001391649},{"type":"v1","addr":"172.21.15.31:6834","nonce":3001391649}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:26:21.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 -- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21380679e0 msgr2=0x7f2138069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21380679e0 0x7f2138069e90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f213c009160 tx=0x7f213c016000 comp rx=0 tx=0).stop 2023-10-06T05:26:21.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 -- 172.21.15.31:0/1258604547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f214c13a5b0 msgr2=0x7f214c14f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:21.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f214c13a5b0 0x7f214c14f280 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f2144001460 tx=0x7f2144001ae0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 -- 172.21.15.31:0/1258604547 shutdown_connections 2023-10-06T05:26:21.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f21380679e0 0x7f2138069e90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f214c13a5b0 0x7f214c14f280 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f214c0c6c30 0x7f214c0cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 --2- 172.21.15.31:0/1258604547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f214c13a1e0 0x7f214c139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:21.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 -- 172.21.15.31:0/1258604547 >> 172.21.15.31:0/1258604547 conn(0x7f214c0cc0b0 msgr2=0x7f214c06d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:21.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 -- 172.21.15.31:0/1258604547 shutdown_connections 2023-10-06T05:26:21.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:21.811+0000 7f21513ff700 1 -- 172.21.15.31:0/1258604547 wait complete. 2023-10-06T05:26:21.814 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 113 2023-10-06T05:26:21.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.025+0000 7f21327ff700 1 Processor -- start 2023-10-06T05:26:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.026+0000 7f21327ff700 1 -- start start 2023-10-06T05:26:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.027+0000 7f21327ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c1595a0 0x7f212c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.027+0000 7f21327ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f212c0cc0d0 0x7f212c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.027+0000 7f21327ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f212c069010 con 0x7f212c068bb0 2023-10-06T05:26:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.027+0000 7f21327ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212c0c95b0 con 0x7f212c0cc0d0 2023-10-06T05:26:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.027+0000 7f21327ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212c0c9890 con 0x7f212c1595a0 2023-10-06T05:26:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.027+0000 7f212b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f212c0cc0d0 0x7f212c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.027+0000 7f212b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f212c0cc0d0 0x7f212c068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46220/0 (socket says 172.21.15.31:46220) 2023-10-06T05:26:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.027+0000 7f212b7fe700 1 -- 172.21.15.31:0/384397992 learned_addr learned my addr 172.21.15.31:0/384397992 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.027+0000 7f212bfff700 1 --2- 172.21.15.31:0/384397992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c1595a0 0x7f212c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.028+0000 7f212b7fe700 1 -- 172.21.15.31:0/384397992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c1595a0 msgr2=0x7f212c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.028+0000 7f212b7fe700 1 --2- 172.21.15.31:0/384397992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c1595a0 0x7f212c159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.028+0000 7f212b7fe700 1 -- 172.21.15.31:0/384397992 >> v1:172.21.15.62:6789/0 conn(0x7f212c068bb0 legacy=0x7f212c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.028+0000 7f212b7fe700 1 -- 172.21.15.31:0/384397992 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c0c9b70 con 0x7f212c0cc0d0 2023-10-06T05:26:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.028+0000 7f212b7fe700 1 --2- 172.21.15.31:0/384397992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 0x7f212c068670 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f21200099b0 tx=0x7f2120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9667a69084347d17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.028+0000 7f212affd700 1 -- 172.21.15.31:0/384397992 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000b860 con 0x7f212c0cc0d0 2023-10-06T05:26:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.028+0000 7f212affd700 1 -- 172.21.15.31:0/384397992 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212000b9c0 con 0x7f212c0cc0d0 2023-10-06T05:26:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.028+0000 7f212affd700 1 -- 172.21.15.31:0/384397992 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000bd60 con 0x7f212c0cc0d0 2023-10-06T05:26:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.029+0000 7f21327ff700 1 -- 172.21.15.31:0/384397992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 msgr2=0x7f212c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.029+0000 7f21327ff700 1 --2- 172.21.15.31:0/384397992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 0x7f212c068670 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f21200099b0 tx=0x7f2120009250 comp rx=0 tx=0).stop 2023-10-06T05:26:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.029+0000 7f21327ff700 1 -- 172.21.15.31:0/384397992 shutdown_connections 2023-10-06T05:26:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.029+0000 7f21327ff700 1 --2- 172.21.15.31:0/384397992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c1595a0 0x7f212c159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.029+0000 7f21327ff700 1 --2- 172.21.15.31:0/384397992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 0x7f212c068670 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.029+0000 7f21327ff700 1 -- 172.21.15.31:0/384397992 >> 172.21.15.31:0/384397992 conn(0x7f212c06b620 msgr2=0x7f212c06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.029+0000 7f21327ff700 1 -- 172.21.15.31:0/384397992 shutdown_connections 2023-10-06T05:26:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.029+0000 7f21327ff700 1 -- 172.21.15.31:0/384397992 wait complete. 2023-10-06T05:26:22.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.031+0000 7f21327ff700 1 Processor -- start 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.031+0000 7f21327ff700 1 -- start start 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f21327ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f212c068bb0 0x7f212c0b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f21327ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 0x7f212c137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f21327ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c1595a0 0x7f212c138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f21327ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c138880 con 0x7f212c0cc0d0 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f21327ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c138ad0 con 0x7f212c068bb0 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f21327ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212c0b28a0 con 0x7f212c1595a0 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 0x7f212c137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f212bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f212c068bb0 0x7f212c0b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f212b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 0x7f212c137e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46226/0 (socket says 172.21.15.31:46226) 2023-10-06T05:26:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f212b7fe700 1 -- 172.21.15.31:0/2185728821 learned_addr learned my addr 172.21.15.31:0/2185728821 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f2130d9c700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c1595a0 0x7f212c138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f212b7fe700 1 -- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c1595a0 msgr2=0x7f212c138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f212b7fe700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c1595a0 0x7f212c138340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f212b7fe700 1 -- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f212c068bb0 msgr2=0x7f212c0b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f212b7fe700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f212c068bb0 0x7f212c0b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.032+0000 7f212b7fe700 1 -- 172.21.15.31:0/2185728821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21200094f0 con 0x7f212c0cc0d0 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.033+0000 7f212b7fe700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 0x7f212c137e00 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f212c06ee00 tx=0x7f2120017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.033+0000 7f2113fff700 1 -- 172.21.15.31:0/2185728821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120011e40 con 0x7f212c0cc0d0 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.033+0000 7f2113fff700 1 -- 172.21.15.31:0/2185728821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2120024440 con 0x7f212c0cc0d0 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.033+0000 7f21327ff700 1 -- 172.21.15.31:0/2185728821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212c0b1c90 con 0x7f212c0cc0d0 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.033+0000 7f2113fff700 1 -- 172.21.15.31:0/2185728821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120024800 con 0x7f212c0cc0d0 2023-10-06T05:26:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.033+0000 7f21327ff700 1 -- 172.21.15.31:0/2185728821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212c0b2250 con 0x7f212c0cc0d0 2023-10-06T05:26:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.034+0000 7f21327ff700 1 -- 172.21.15.31:0/2185728821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212c0b2a30 con 0x7f212c0cc0d0 2023-10-06T05:26:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.035+0000 7f2113fff700 1 -- 172.21.15.31:0/2185728821 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f212001a420 con 0x7f212c0cc0d0 2023-10-06T05:26:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.036+0000 7f2113fff700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2118067a10 0x7f2118069ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.036+0000 7f2113fff700 1 -- 172.21.15.31:0/2185728821 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f2120084b10 con 0x7f212c0cc0d0 2023-10-06T05:26:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.037+0000 7f212bfff700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2118067a10 0x7f2118069ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.037+0000 7f212bfff700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2118067a10 0x7f2118069ec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f211c006fd0 tx=0x7f211c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.040+0000 7f2113fff700 1 -- 172.21.15.31:0/2185728821 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2120053750 con 0x7f212c0cc0d0 2023-10-06T05:26:22.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.248+0000 7f21327ff700 1 -- 172.21.15.31:0/2185728821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f212c04c380 con 0x7f212c0cc0d0 2023-10-06T05:26:22.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.249+0000 7f2113fff700 1 -- 172.21.15.31:0/2185728821 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+16972 (secure 0 0 0) 0x7f2120056d70 con 0x7f212c0cc0d0 2023-10-06T05:26:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:22.251 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":163,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:26:03.756620+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:00.480986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:01.238722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:22.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 -- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2118067a10 msgr2=0x7f2118069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2118067a10 0x7f2118069ec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f211c006fd0 tx=0x7f211c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:22.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 -- 172.21.15.31:0/2185728821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 msgr2=0x7f212c137e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 0x7f212c137e00 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f212c06ee00 tx=0x7f2120017000 comp rx=0 tx=0).stop 2023-10-06T05:26:22.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 -- 172.21.15.31:0/2185728821 shutdown_connections 2023-10-06T05:26:22.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2118067a10 0x7f2118069ec0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c1595a0 0x7f212c138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f212c068bb0 0x7f212c0b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 --2- 172.21.15.31:0/2185728821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f212c0cc0d0 0x7f212c137e00 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 -- 172.21.15.31:0/2185728821 >> 172.21.15.31:0/2185728821 conn(0x7f212c06b620 msgr2=0x7f212c15b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:22.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 -- 172.21.15.31:0/2185728821 shutdown_connections 2023-10-06T05:26:22.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.251+0000 7f21327ff700 1 -- 172.21.15.31:0/2185728821 wait complete. 2023-10-06T05:26:22.271 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:26:22.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:22.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.465+0000 7f8049fff700 1 Processor -- start 2023-10-06T05:26:22.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.465+0000 7f8049fff700 1 -- start start 2023-10-06T05:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.467+0000 7f8049fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f804413a1e0 0x7f80440c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.467+0000 7f8049fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f804413a5b0 0x7f804406d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.467+0000 7f8049fff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440c71b0 con 0x7f80440c6c30 2023-10-06T05:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.467+0000 7f8049fff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440c7490 con 0x7f804413a1e0 2023-10-06T05:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.467+0000 7f8049fff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440c7770 con 0x7f804413a5b0 2023-10-06T05:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.467+0000 7f80437fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f804413a1e0 0x7f80440c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.467+0000 7f80437fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f804413a1e0 0x7f80440c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46232/0 (socket says 172.21.15.31:46232) 2023-10-06T05:26:22.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.467+0000 7f80437fe700 1 -- 172.21.15.31:0/525061395 learned_addr learned my addr 172.21.15.31:0/525061395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:22.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.467+0000 7f8042ffd700 1 --2- 172.21.15.31:0/525061395 >> v2:172.21.15.62:3300/0 conn(0x7f804413a5b0 0x7f804406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.467+0000 7f80437fe700 1 -- 172.21.15.31:0/525061395 >> v2:172.21.15.62:3300/0 conn(0x7f804413a5b0 msgr2=0x7f804406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:22.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.468+0000 7f80437fe700 1 --2- 172.21.15.31:0/525061395 >> v2:172.21.15.62:3300/0 conn(0x7f804413a5b0 0x7f804406d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.468+0000 7f80437fe700 1 -- 172.21.15.31:0/525061395 >> v1:172.21.15.62:6789/0 conn(0x7f80440c6c30 legacy=0x7f804415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:22.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.468+0000 7f80437fe700 1 -- 172.21.15.31:0/525061395 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440c7a50 con 0x7f804413a1e0 2023-10-06T05:26:22.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.468+0000 7f80437fe700 1 --2- 172.21.15.31:0/525061395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804413a1e0 0x7f80440c66f0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f804406d130 tx=0x7f8034009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50f69e919a98eda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.468+0000 7f80427fc700 1 -- 172.21.15.31:0/525061395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034005c10 con 0x7f804413a1e0 2023-10-06T05:26:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.468+0000 7f80427fc700 1 -- 172.21.15.31:0/525061395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034005d70 con 0x7f804413a1e0 2023-10-06T05:26:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.468+0000 7f80427fc700 1 -- 172.21.15.31:0/525061395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400b840 con 0x7f804413a1e0 2023-10-06T05:26:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.469+0000 7f8049fff700 1 -- 172.21.15.31:0/525061395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804413a1e0 msgr2=0x7f80440c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.469+0000 7f8049fff700 1 --2- 172.21.15.31:0/525061395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804413a1e0 0x7f80440c66f0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f804406d130 tx=0x7f8034009220 comp rx=0 tx=0).stop 2023-10-06T05:26:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.469+0000 7f8049fff700 1 -- 172.21.15.31:0/525061395 shutdown_connections 2023-10-06T05:26:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.469+0000 7f8049fff700 1 --2- 172.21.15.31:0/525061395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804413a1e0 0x7f80440c66f0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.469+0000 7f8049fff700 1 --2- 172.21.15.31:0/525061395 >> v2:172.21.15.62:3300/0 conn(0x7f804413a5b0 0x7f804406d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.469+0000 7f8049fff700 1 -- 172.21.15.31:0/525061395 >> 172.21.15.31:0/525061395 conn(0x7f80440cc0b0 msgr2=0x7f80440cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.469+0000 7f8049fff700 1 -- 172.21.15.31:0/525061395 shutdown_connections 2023-10-06T05:26:22.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.469+0000 7f8049fff700 1 -- 172.21.15.31:0/525061395 wait complete. 2023-10-06T05:26:22.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.470+0000 7f8049fff700 1 Processor -- start 2023-10-06T05:26:22.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.471+0000 7f8049fff700 1 -- start start 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.471+0000 7f8049fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80440c6c30 0x7f80440add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.471+0000 7f8049fff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804413a1e0 0x7f8044139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.471+0000 7f8049fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804413a5b0 0x7f80441533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f8049fff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044152f30 con 0x7f804413a5b0 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f8049fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440cbab0 con 0x7f80440c6c30 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f8049fff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8044152060 con 0x7f804413a1e0 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f8043fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804413a5b0 0x7f80441533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f80437fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80440c6c30 0x7f80440add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f80437fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80440c6c30 0x7f80440add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:42806/0 (socket says 172.21.15.31:42806) 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f80437fe700 1 -- 172.21.15.31:0/2163625591 learned_addr learned my addr 172.21.15.31:0/2163625591 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f8042ffd700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804413a1e0 0x7f8044139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f8043fff700 1 -- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804413a1e0 msgr2=0x7f8044139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f8043fff700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804413a1e0 0x7f8044139910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f8043fff700 1 -- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80440c6c30 msgr2=0x7f80440add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f8043fff700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80440c6c30 0x7f80440add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.472+0000 7f8043fff700 1 -- 172.21.15.31:0/2163625591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8034009480 con 0x7f804413a5b0 2023-10-06T05:26:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.473+0000 7f8043fff700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804413a5b0 0x7f80441533c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f803c00c810 tx=0x7f803c002620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.473+0000 7f802b7fe700 1 -- 172.21.15.31:0/2163625591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c012630 con 0x7f804413a5b0 2023-10-06T05:26:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.473+0000 7f802b7fe700 1 -- 172.21.15.31:0/2163625591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c0127b0 con 0x7f804413a5b0 2023-10-06T05:26:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.473+0000 7f802b7fe700 1 -- 172.21.15.31:0/2163625591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c012b70 con 0x7f804413a5b0 2023-10-06T05:26:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.473+0000 7f8049fff700 1 -- 172.21.15.31:0/2163625591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8044152320 con 0x7f804413a5b0 2023-10-06T05:26:22.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.473+0000 7f8049fff700 1 -- 172.21.15.31:0/2163625591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80441528e0 con 0x7f804413a5b0 2023-10-06T05:26:22.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.474+0000 7f8049fff700 1 -- 172.21.15.31:0/2163625591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80441530e0 con 0x7f804413a5b0 2023-10-06T05:26:22.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.475+0000 7f802b7fe700 1 -- 172.21.15.31:0/2163625591 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f803c01b400 con 0x7f804413a5b0 2023-10-06T05:26:22.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.476+0000 7f802b7fe700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f80300679e0 0x7f8030069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.476+0000 7f802b7fe700 1 -- 172.21.15.31:0/2163625591 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f803c0830f0 con 0x7f804413a5b0 2023-10-06T05:26:22.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.476+0000 7f80437fe700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f80300679e0 0x7f8030069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.477+0000 7f80437fe700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f80300679e0 0x7f8030069e90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8034007fd0 tx=0x7f8034015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.480+0000 7f802b7fe700 1 -- 172.21.15.31:0/2163625591 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80441530e0 con 0x7f804413a5b0 2023-10-06T05:26:22.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.729+0000 7f8049fff700 1 -- 172.21.15.31:0/2163625591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f804404c380 con 0x7f804413a5b0 2023-10-06T05:26:22.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.730+0000 7f802b7fe700 1 -- 172.21.15.31:0/2163625591 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f803c051d30 con 0x7f804413a5b0 2023-10-06T05:26:22.732 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:22.732 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":113,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4242992570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4242992570},{"type":"v1","addr":"172.21.15.31:6835","nonce":4242992570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":7706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2405114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2405114362},{"type":"v1","addr":"172.21.15.62:6834","nonce":2405114362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2676480318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2676480318},{"type":"v1","addr":"172.21.15.62:6835","nonce":2676480318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:02.260523+0000","modified":"2023-10-06T05:26:06.149545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9249":{"gid":9249,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3001391649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3001391649},{"type":"v1","addr":"172.21.15.31:6834","nonce":3001391649}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:26:22.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.732+0000 7f8049fff700 1 -- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f80300679e0 msgr2=0x7f8030069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.732+0000 7f8049fff700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f80300679e0 0x7f8030069e90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8034007fd0 tx=0x7f8034015000 comp rx=0 tx=0).stop 2023-10-06T05:26:22.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.732+0000 7f8049fff700 1 -- 172.21.15.31:0/2163625591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804413a5b0 msgr2=0x7f80441533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.732+0000 7f8049fff700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804413a5b0 0x7f80441533c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f803c00c810 tx=0x7f803c002620 comp rx=0 tx=0).stop 2023-10-06T05:26:22.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.733+0000 7f8049fff700 1 -- 172.21.15.31:0/2163625591 shutdown_connections 2023-10-06T05:26:22.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.733+0000 7f8049fff700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f80300679e0 0x7f8030069e90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.733+0000 7f8049fff700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804413a5b0 0x7f80441533c0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.733+0000 7f8049fff700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80440c6c30 0x7f80440add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.733+0000 7f8049fff700 1 --2- 172.21.15.31:0/2163625591 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f804413a1e0 0x7f8044139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.733+0000 7f8049fff700 1 -- 172.21.15.31:0/2163625591 >> 172.21.15.31:0/2163625591 conn(0x7f80440cc0b0 msgr2=0x7f80440c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:22.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.733+0000 7f8049fff700 1 -- 172.21.15.31:0/2163625591 shutdown_connections 2023-10-06T05:26:22.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.733+0000 7f8049fff700 1 -- 172.21.15.31:0/2163625591 wait complete. 2023-10-06T05:26:22.736 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 113 2023-10-06T05:26:22.752 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.945+0000 7f4b2adff700 1 Processor -- start 2023-10-06T05:26:22.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.946+0000 7f4b2adff700 1 -- start start 2023-10-06T05:26:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.947+0000 7f4b2adff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4b240c7b60 0x7f4b24069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.947+0000 7f4b2adff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b240aee70 0x7f4b2413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.947+0000 7f4b2adff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b240ca5b0 con 0x7f4b240c7f30 2023-10-06T05:26:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.947+0000 7f4b2adff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b240af2d0 con 0x7f4b240aee70 2023-10-06T05:26:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.947+0000 7f4b2adff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2406f740 con 0x7f4b240c7b60 2023-10-06T05:26:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.947+0000 7f4b2939c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b240aee70 0x7f4b2413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.947+0000 7f4b2939c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b240aee70 0x7f4b2413f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46238/0 (socket says 172.21.15.31:46238) 2023-10-06T05:26:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.947+0000 7f4b23fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4b240c7f30 0x7f4b2406ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33566/0 (socket says 172.21.15.31:33566) 2023-10-06T05:26:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.947+0000 7f4b2939c700 1 -- 172.21.15.31:0/4239923988 learned_addr learned my addr 172.21.15.31:0/4239923988 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.947+0000 7f4b28b9b700 1 --2- 172.21.15.31:0/4239923988 >> v2:172.21.15.62:3300/0 conn(0x7f4b240c7b60 0x7f4b24069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.948+0000 7f4b2939c700 1 -- 172.21.15.31:0/4239923988 >> v2:172.21.15.62:3300/0 conn(0x7f4b240c7b60 msgr2=0x7f4b24069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.948+0000 7f4b2939c700 1 --2- 172.21.15.31:0/4239923988 >> v2:172.21.15.62:3300/0 conn(0x7f4b240c7b60 0x7f4b24069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.948+0000 7f4b2939c700 1 -- 172.21.15.31:0/4239923988 >> v1:172.21.15.31:6789/0 conn(0x7f4b240c7f30 legacy=0x7f4b2406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.948+0000 7f4b2939c700 1 -- 172.21.15.31:0/4239923988 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b240c9c80 con 0x7f4b240aee70 2023-10-06T05:26:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.948+0000 7f4b237fe700 1 -- 172.21.15.31:0/4239923988 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456599882 0 0) 0x7f4b240ca5b0 con 0x7f4b240c7f30 2023-10-06T05:26:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.948+0000 7f4b2939c700 1 --2- 172.21.15.31:0/4239923988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240aee70 0x7f4b2413f690 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406fa80 tx=0x7f4b1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f184ab76341abbfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.948+0000 7f4b237fe700 1 -- 172.21.15.31:0/4239923988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00e070 con 0x7f4b240aee70 2023-10-06T05:26:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.948+0000 7f4b237fe700 1 -- 172.21.15.31:0/4239923988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c00ba50 con 0x7f4b240aee70 2023-10-06T05:26:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.948+0000 7f4b237fe700 1 -- 172.21.15.31:0/4239923988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0092e0 con 0x7f4b240aee70 2023-10-06T05:26:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.949+0000 7f4b2adff700 1 -- 172.21.15.31:0/4239923988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240aee70 msgr2=0x7f4b2413f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.949+0000 7f4b2adff700 1 --2- 172.21.15.31:0/4239923988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240aee70 0x7f4b2413f690 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406fa80 tx=0x7f4b1c009840 comp rx=0 tx=0).stop 2023-10-06T05:26:22.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.949+0000 7f4b2adff700 1 -- 172.21.15.31:0/4239923988 shutdown_connections 2023-10-06T05:26:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.949+0000 7f4b2adff700 1 --2- 172.21.15.31:0/4239923988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240aee70 0x7f4b2413f690 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.949+0000 7f4b2adff700 1 --2- 172.21.15.31:0/4239923988 >> v2:172.21.15.62:3300/0 conn(0x7f4b240c7b60 0x7f4b24069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.949+0000 7f4b2adff700 1 -- 172.21.15.31:0/4239923988 >> 172.21.15.31:0/4239923988 conn(0x7f4b240cc0b0 msgr2=0x7f4b240cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.949+0000 7f4b2adff700 1 -- 172.21.15.31:0/4239923988 shutdown_connections 2023-10-06T05:26:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.949+0000 7f4b2adff700 1 -- 172.21.15.31:0/4239923988 wait complete. 2023-10-06T05:26:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.950+0000 7f4b2adff700 1 Processor -- start 2023-10-06T05:26:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.950+0000 7f4b2adff700 1 -- start start 2023-10-06T05:26:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.951+0000 7f4b2adff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b240aee70 0x7f4b240ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.951+0000 7f4b2adff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240c7b60 0x7f4b24156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.951+0000 7f4b2adff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b240c7f30 0x7f4b241697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.951+0000 7f4b2adff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24138490 con 0x7f4b240c7b60 2023-10-06T05:26:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.951+0000 7f4b2adff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24138770 con 0x7f4b240aee70 2023-10-06T05:26:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.951+0000 7f4b2adff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24138a50 con 0x7f4b240c7f30 2023-10-06T05:26:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b23fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240c7b60 0x7f4b24156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b23fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240c7b60 0x7f4b24156020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46246/0 (socket says 172.21.15.31:46246) 2023-10-06T05:26:22.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b23fff700 1 -- 172.21.15.31:0/27421018 learned_addr learned my addr 172.21.15.31:0/27421018 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b28b9b700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b240aee70 0x7f4b240ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b2939c700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b240c7f30 0x7f4b241697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b23fff700 1 -- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b240c7f30 msgr2=0x7f4b241697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b23fff700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b240c7f30 0x7f4b241697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b23fff700 1 -- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b240aee70 msgr2=0x7f4b240ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b23fff700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b240aee70 0x7f4b240ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b23fff700 1 -- 172.21.15.31:0/27421018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c009ae0 con 0x7f4b240c7b60 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.952+0000 7f4b23fff700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240c7b60 0x7f4b24156020 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000c5b0 tx=0x7f4b10001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.953+0000 7f4b20ff9700 1 -- 172.21.15.31:0/27421018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10005a70 con 0x7f4b240c7b60 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.953+0000 7f4b20ff9700 1 -- 172.21.15.31:0/27421018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b10005bd0 con 0x7f4b240c7b60 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.953+0000 7f4b20ff9700 1 -- 172.21.15.31:0/27421018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b100070d0 con 0x7f4b240c7b60 2023-10-06T05:26:22.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.953+0000 7f4b2adff700 1 -- 172.21.15.31:0/27421018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b24138d30 con 0x7f4b240c7b60 2023-10-06T05:26:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.953+0000 7f4b2adff700 1 -- 172.21.15.31:0/27421018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b240ade20 con 0x7f4b240c7b60 2023-10-06T05:26:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.954+0000 7f4b2adff700 1 -- 172.21.15.31:0/27421018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b24139250 con 0x7f4b240c7b60 2023-10-06T05:26:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.955+0000 7f4b20ff9700 1 -- 172.21.15.31:0/27421018 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f4b10007480 con 0x7f4b240c7b60 2023-10-06T05:26:22.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.955+0000 7f4b20ff9700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b080679e0 0x7f4b08069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:22.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.956+0000 7f4b20ff9700 1 -- 172.21.15.31:0/27421018 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f4b10082540 con 0x7f4b240c7b60 2023-10-06T05:26:22.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.956+0000 7f4b28b9b700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b080679e0 0x7f4b08069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:22.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.956+0000 7f4b28b9b700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b080679e0 0x7f4b08069e90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4b18006aa0 tx=0x7f4b18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:22.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:22.960+0000 7f4b20ff9700 1 -- 172.21.15.31:0/27421018 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b24139250 con 0x7f4b240c7b60 2023-10-06T05:26:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.209+0000 7f4b2adff700 1 -- 172.21.15.31:0/27421018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b24139250 con 0x7f4b240c7b60 2023-10-06T05:26:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.210+0000 7f4b20ff9700 1 -- 172.21.15.31:0/27421018 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f4b24139250 con 0x7f4b240c7b60 2023-10-06T05:26:23.212 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:23.212 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":113,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4242992570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4242992570},{"type":"v1","addr":"172.21.15.31:6835","nonce":4242992570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":7706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2405114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2405114362},{"type":"v1","addr":"172.21.15.62:6834","nonce":2405114362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2676480318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2676480318},{"type":"v1","addr":"172.21.15.62:6835","nonce":2676480318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:26:02.260523+0000","modified":"2023-10-06T05:26:06.149545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9249":{"gid":9249,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/3001391649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3001391649},{"type":"v1","addr":"172.21.15.31:6834","nonce":3001391649}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T05:26:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 -- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b080679e0 msgr2=0x7f4b08069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b080679e0 0x7f4b08069e90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4b18006aa0 tx=0x7f4b18008040 comp rx=0 tx=0).stop 2023-10-06T05:26:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 -- 172.21.15.31:0/27421018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240c7b60 msgr2=0x7f4b24156020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240c7b60 0x7f4b24156020 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000c5b0 tx=0x7f4b10001040 comp rx=0 tx=0).stop 2023-10-06T05:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 -- 172.21.15.31:0/27421018 shutdown_connections 2023-10-06T05:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f4b080679e0 0x7f4b08069e90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b240c7f30 0x7f4b241697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b240aee70 0x7f4b240ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 --2- 172.21.15.31:0/27421018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b240c7b60 0x7f4b24156020 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 -- 172.21.15.31:0/27421018 >> 172.21.15.31:0/27421018 conn(0x7f4b240cc0b0 msgr2=0x7f4b240cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 -- 172.21.15.31:0/27421018 shutdown_connections 2023-10-06T05:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.212+0000 7f4b2adff700 1 -- 172.21.15.31:0/27421018 wait complete. 2023-10-06T05:26:23.216 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 113 2023-10-06T05:26:23.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:23.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.425+0000 7f3de4dff700 1 Processor -- start 2023-10-06T05:26:23.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.425+0000 7f3de4dff700 1 -- start start 2023-10-06T05:26:23.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.427+0000 7f3de4dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3de00c7f30 0x7f3de0069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.427+0000 7f3de4dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 0x7f3de013f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.427+0000 7f3de4dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de00ca5b0 con 0x7f3de00c7b60 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.427+0000 7f3de4dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de00af2f0 con 0x7f3de00c7f30 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.427+0000 7f3de4dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de006f740 con 0x7f3de00aee70 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.427+0000 7f3dddd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3de00c7f30 0x7f3de0069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.427+0000 7f3dde59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3de00c7b60 0x7f3de006ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33580/0 (socket says 172.21.15.31:33580) 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.427+0000 7f3dde59c700 1 -- 172.21.15.31:0/3548694949 learned_addr learned my addr 172.21.15.31:0/3548694949 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.427+0000 7f3dded9d700 1 --2- 172.21.15.31:0/3548694949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 0x7f3de013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.428+0000 7f3ddd59a700 1 -- 172.21.15.31:0/3548694949 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278605416 0 0) 0x7f3de00ca5b0 con 0x7f3de00c7b60 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.428+0000 7f3dddd9b700 1 -- 172.21.15.31:0/3548694949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 msgr2=0x7f3de013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.428+0000 7f3dddd9b700 1 --2- 172.21.15.31:0/3548694949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 0x7f3de013f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.428+0000 7f3dddd9b700 1 -- 172.21.15.31:0/3548694949 >> v1:172.21.15.31:6789/0 conn(0x7f3de00c7b60 legacy=0x7f3de006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.428+0000 7f3dddd9b700 1 -- 172.21.15.31:0/3548694949 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de00c9c80 con 0x7f3de00c7f30 2023-10-06T05:26:23.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.428+0000 7f3dddd9b700 1 --2- 172.21.15.31:0/3548694949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7f30 0x7f3de0069880 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40098e0 tx=0x7f3dd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caba4acf6e252eeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.428+0000 7f3ddd59a700 1 -- 172.21.15.31:0/3548694949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400b640 con 0x7f3de00c7f30 2023-10-06T05:26:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.428+0000 7f3ddd59a700 1 -- 172.21.15.31:0/3548694949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd400b7a0 con 0x7f3de00c7f30 2023-10-06T05:26:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.428+0000 7f3ddd59a700 1 -- 172.21.15.31:0/3548694949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400bb40 con 0x7f3de00c7f30 2023-10-06T05:26:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.429+0000 7f3de4dff700 1 -- 172.21.15.31:0/3548694949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7f30 msgr2=0x7f3de0069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.429+0000 7f3de4dff700 1 --2- 172.21.15.31:0/3548694949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7f30 0x7f3de0069880 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40098e0 tx=0x7f3dd40091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.429+0000 7f3de4dff700 1 -- 172.21.15.31:0/3548694949 shutdown_connections 2023-10-06T05:26:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.429+0000 7f3de4dff700 1 --2- 172.21.15.31:0/3548694949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 0x7f3de013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.429+0000 7f3de4dff700 1 --2- 172.21.15.31:0/3548694949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7f30 0x7f3de0069880 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.429+0000 7f3de4dff700 1 -- 172.21.15.31:0/3548694949 >> 172.21.15.31:0/3548694949 conn(0x7f3de00cc0b0 msgr2=0x7f3de00cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.429+0000 7f3de4dff700 1 -- 172.21.15.31:0/3548694949 shutdown_connections 2023-10-06T05:26:23.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.429+0000 7f3de4dff700 1 -- 172.21.15.31:0/3548694949 wait complete. 2023-10-06T05:26:23.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.430+0000 7f3de4dff700 1 Processor -- start 2023-10-06T05:26:23.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.431+0000 7f3de4dff700 1 -- start start 2023-10-06T05:26:23.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3de4dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 0x7f3de016dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3de4dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7b60 0x7f3de0169620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3de4dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de00c7f30 0x7f3de013c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3de4dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de016e1e0 con 0x7f3de00c7b60 2023-10-06T05:26:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3de4dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0169b60 con 0x7f3de00c7f30 2023-10-06T05:26:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3de4dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de015ed20 con 0x7f3de00aee70 2023-10-06T05:26:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dddd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7b60 0x7f3de0169620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dddd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7b60 0x7f3de0169620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46272/0 (socket says 172.21.15.31:46272) 2023-10-06T05:26:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dddd9b700 1 -- 172.21.15.31:0/2859834622 learned_addr learned my addr 172.21.15.31:0/2859834622 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dde59c700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 0x7f3de016dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dded9d700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de00c7f30 0x7f3de013c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dddd9b700 1 -- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 msgr2=0x7f3de016dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dddd9b700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 0x7f3de016dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dddd9b700 1 -- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de00c7f30 msgr2=0x7f3de013c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dddd9b700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de00c7f30 0x7f3de013c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dddd9b700 1 -- 172.21.15.31:0/2859834622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4009420 con 0x7f3de00c7b60 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dde59c700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 0x7f3de016dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.432+0000 7f3dded9d700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de00c7f30 0x7f3de013c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.433+0000 7f3dddd9b700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7b60 0x7f3de0169620 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40093f0 tx=0x7f3dd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.433+0000 7f3dce7fc700 1 -- 172.21.15.31:0/2859834622 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400bb20 con 0x7f3de00c7b60 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.433+0000 7f3dce7fc700 1 -- 172.21.15.31:0/2859834622 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd4011420 con 0x7f3de00c7b60 2023-10-06T05:26:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.433+0000 7f3de4dff700 1 -- 172.21.15.31:0/2859834622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de015dc80 con 0x7f3de00c7b60 2023-10-06T05:26:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.433+0000 7f3dce7fc700 1 -- 172.21.15.31:0/2859834622 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40117a0 con 0x7f3de00c7b60 2023-10-06T05:26:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.433+0000 7f3de4dff700 1 -- 172.21.15.31:0/2859834622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de015e240 con 0x7f3de00c7b60 2023-10-06T05:26:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.434+0000 7f3de4dff700 1 -- 172.21.15.31:0/2859834622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de004c380 con 0x7f3de00c7b60 2023-10-06T05:26:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.435+0000 7f3dce7fc700 1 -- 172.21.15.31:0/2859834622 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f3dd4011bd0 con 0x7f3de00c7b60 2023-10-06T05:26:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.436+0000 7f3dce7fc700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3dc00679e0 0x7f3dc0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.436+0000 7f3dce7fc700 1 -- 172.21.15.31:0/2859834622 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f3dd4084950 con 0x7f3de00c7b60 2023-10-06T05:26:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.437+0000 7f3dde59c700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3dc00679e0 0x7f3dc0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.438+0000 7f3dde59c700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3dc00679e0 0x7f3dc0069e90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0001360 tx=0x7f3dd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.441+0000 7f3dce7fc700 1 -- 172.21.15.31:0/2859834622 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3dd4053590 con 0x7f3de00c7b60 2023-10-06T05:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.641+0000 7f3de4dff700 1 -- 172.21.15.31:0/2859834622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3de00c9f60 con 0x7f3de00c7b60 2023-10-06T05:26:23.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.642+0000 7f3dce7fc700 1 -- 172.21.15.31:0/2859834622 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+16972 (secure 0 0 0) 0x7f3dd4056bb0 con 0x7f3de00c7b60 2023-10-06T05:26:23.644 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:23.645 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":163,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:26:03.756620+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:26:00.480986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T05:26:01.238722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 -- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3dc00679e0 msgr2=0x7f3dc0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3dc00679e0 0x7f3dc0069e90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0001360 tx=0x7f3dd0009040 comp rx=0 tx=0).stop 2023-10-06T05:26:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 -- 172.21.15.31:0/2859834622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7b60 msgr2=0x7f3de0169620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7b60 0x7f3de0169620 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40093f0 tx=0x7f3dd4017000 comp rx=0 tx=0).stop 2023-10-06T05:26:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 -- 172.21.15.31:0/2859834622 shutdown_connections 2023-10-06T05:26:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f3dc00679e0 0x7f3dc0069e90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3de00c7f30 0x7f3de013c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3de00aee70 0x7f3de016dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 --2- 172.21.15.31:0/2859834622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3de00c7b60 0x7f3de0169620 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 -- 172.21.15.31:0/2859834622 >> 172.21.15.31:0/2859834622 conn(0x7f3de00cc0b0 msgr2=0x7f3de013e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 -- 172.21.15.31:0/2859834622 shutdown_connections 2023-10-06T05:26:23.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.645+0000 7f3de4dff700 1 -- 172.21.15.31:0/2859834622 wait complete. 2023-10-06T05:26:23.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:26:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.858+0000 7f51203ff700 1 Processor -- start 2023-10-06T05:26:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.858+0000 7f51203ff700 1 -- start start 2023-10-06T05:26:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f51203ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f51181595a0 0x7f5118159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f51203ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51180cc0d0 0x7f5118068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f51203ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5118068bb0 0x7f51180c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f51203ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5118069010 con 0x7f51180cc0d0 2023-10-06T05:26:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f51203ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51180c95b0 con 0x7f51181595a0 2023-10-06T05:26:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f51203ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51180c9890 con 0x7f5118068bb0 2023-10-06T05:26:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f511d99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51180cc0d0 0x7f5118068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f511d99a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51180cc0d0 0x7f5118068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46276/0 (socket says 172.21.15.31:46276) 2023-10-06T05:26:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f511d99a700 1 -- 172.21.15.31:0/609528371 learned_addr learned my addr 172.21.15.31:0/609528371 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f511e99c700 1 --2- 172.21.15.31:0/609528371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5118068bb0 0x7f51180c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.860+0000 7f511e19b700 1 --2- 172.21.15.31:0/609528371 >> v2:172.21.15.62:3300/0 conn(0x7f51181595a0 0x7f5118159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.861+0000 7f511d99a700 1 -- 172.21.15.31:0/609528371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5118068bb0 msgr2=0x7f51180c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.861+0000 7f511d99a700 1 --2- 172.21.15.31:0/609528371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5118068bb0 0x7f51180c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.861+0000 7f511d99a700 1 -- 172.21.15.31:0/609528371 >> v2:172.21.15.62:3300/0 conn(0x7f51181595a0 msgr2=0x7f5118159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.861+0000 7f511d99a700 1 --2- 172.21.15.31:0/609528371 >> v2:172.21.15.62:3300/0 conn(0x7f51181595a0 0x7f5118159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.861+0000 7f511d99a700 1 -- 172.21.15.31:0/609528371 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180c9b70 con 0x7f51180cc0d0 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.861+0000 7f511d99a700 1 --2- 172.21.15.31:0/609528371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51180cc0d0 0x7f5118068670 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f511806c8d0 tx=0x7f5108009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afe78c2e55d0b7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.861+0000 7f511d199700 1 -- 172.21.15.31:0/609528371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510800b640 con 0x7f51180cc0d0 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.861+0000 7f511d199700 1 -- 172.21.15.31:0/609528371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510800b7a0 con 0x7f51180cc0d0 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.861+0000 7f511d199700 1 -- 172.21.15.31:0/609528371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510800bb60 con 0x7f51180cc0d0 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.862+0000 7f51203ff700 1 -- 172.21.15.31:0/609528371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51180cc0d0 msgr2=0x7f5118068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.862+0000 7f51203ff700 1 --2- 172.21.15.31:0/609528371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51180cc0d0 0x7f5118068670 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f511806c8d0 tx=0x7f5108009250 comp rx=0 tx=0).stop 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.862+0000 7f51203ff700 1 -- 172.21.15.31:0/609528371 shutdown_connections 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.862+0000 7f51203ff700 1 --2- 172.21.15.31:0/609528371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5118068bb0 0x7f51180c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.862+0000 7f51203ff700 1 --2- 172.21.15.31:0/609528371 >> v2:172.21.15.62:3300/0 conn(0x7f51181595a0 0x7f5118159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.862+0000 7f51203ff700 1 --2- 172.21.15.31:0/609528371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51180cc0d0 0x7f5118068670 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.862+0000 7f51203ff700 1 -- 172.21.15.31:0/609528371 >> 172.21.15.31:0/609528371 conn(0x7f511806b620 msgr2=0x7f511806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.862+0000 7f51203ff700 1 -- 172.21.15.31:0/609528371 shutdown_connections 2023-10-06T05:26:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.862+0000 7f51203ff700 1 -- 172.21.15.31:0/609528371 wait complete. 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.863+0000 7f51203ff700 1 Processor -- start 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.863+0000 7f51203ff700 1 -- start start 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.864+0000 7f51203ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5118068bb0 0x7f511814f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.864+0000 7f51203ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51180cc0d0 0x7f51180b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.864+0000 7f51203ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51181595a0 0x7f51180b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.864+0000 7f51203ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118155cd0 con 0x7f5118068bb0 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.864+0000 7f51203ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181550c0 con 0x7f51180cc0d0 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.864+0000 7f51203ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51181553a0 con 0x7f51181595a0 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5118068bb0 0x7f511814f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e19b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5118068bb0 0x7f511814f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46290/0 (socket says 172.21.15.31:46290) 2023-10-06T05:26:23.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e19b700 1 -- 172.21.15.31:0/1241233172 learned_addr learned my addr 172.21.15.31:0/1241233172 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511d99a700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51180cc0d0 0x7f51180b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e99c700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51181595a0 0x7f51180b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e19b700 1 -- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51181595a0 msgr2=0x7f51180b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e19b700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51181595a0 0x7f51180b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e19b700 1 -- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51180cc0d0 msgr2=0x7f51180b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e19b700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51180cc0d0 0x7f51180b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e19b700 1 -- 172.21.15.31:0/1241233172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080094f0 con 0x7f5118068bb0 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e99c700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51181595a0 0x7f51180b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.865+0000 7f511e19b700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5118068bb0 0x7f511814f1e0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f511400e930 tx=0x7f51140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.866+0000 7f510e7fc700 1 -- 172.21.15.31:0/1241233172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114013680 con 0x7f5118068bb0 2023-10-06T05:26:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.866+0000 7f510e7fc700 1 -- 172.21.15.31:0/1241233172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51140137e0 con 0x7f5118068bb0 2023-10-06T05:26:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.866+0000 7f510e7fc700 1 -- 172.21.15.31:0/1241233172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114013b80 con 0x7f5118068bb0 2023-10-06T05:26:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.866+0000 7f51203ff700 1 -- 172.21.15.31:0/1241233172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5118155680 con 0x7f5118068bb0 2023-10-06T05:26:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.866+0000 7f51203ff700 1 -- 172.21.15.31:0/1241233172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511814f720 con 0x7f5118068bb0 2023-10-06T05:26:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.867+0000 7f51203ff700 1 -- 172.21.15.31:0/1241233172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5118155b20 con 0x7f5118068bb0 2023-10-06T05:26:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.868+0000 7f510e7fc700 1 -- 172.21.15.31:0/1241233172 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f511400c460 con 0x7f5118068bb0 2023-10-06T05:26:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.868+0000 7f510e7fc700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f50fc0679e0 0x7f50fc069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.868+0000 7f510e7fc700 1 -- 172.21.15.31:0/1241233172 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6086+0+0 (secure 0 0 0) 0x7f5114083090 con 0x7f5118068bb0 2023-10-06T05:26:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.869+0000 7f511d99a700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f50fc0679e0 0x7f50fc069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.869+0000 7f511d99a700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f50fc0679e0 0x7f50fc069e90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f511806c8d0 tx=0x7f5108017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:23.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:23.873+0000 7f510e7fc700 1 -- 172.21.15.31:0/1241233172 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5114051cd0 con 0x7f5118068bb0 2023-10-06T05:26:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.120+0000 7f51203ff700 1 -- 172.21.15.31:0/1241233172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5118155fb0 con 0x7f5118068bb0 2023-10-06T05:26:24.122 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-10-06T05:26:24.120+0000 7f6507ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:26:24.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.128+0000 7f510e7fc700 1 -- 172.21.15.31:0/1241233172 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) v1 ==== 161+0+0 (secure 0 0 0) 0x7f51140552f0 con 0x7f5118068bb0 2023-10-06T05:26:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 -- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f50fc0679e0 msgr2=0x7f50fc069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f50fc0679e0 0x7f50fc069e90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f511806c8d0 tx=0x7f5108017000 comp rx=0 tx=0).stop 2023-10-06T05:26:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 -- 172.21.15.31:0/1241233172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5118068bb0 msgr2=0x7f511814f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5118068bb0 0x7f511814f1e0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f511400e930 tx=0x7f51140091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 -- 172.21.15.31:0/1241233172 shutdown_connections 2023-10-06T05:26:24.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f50fc0679e0 0x7f50fc069e90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51181595a0 0x7f51180b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5118068bb0 0x7f511814f1e0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 --2- 172.21.15.31:0/1241233172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51180cc0d0 0x7f51180b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 -- 172.21.15.31:0/1241233172 >> 172.21.15.31:0/1241233172 conn(0x7f511806b620 msgr2=0x7f5118157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:24.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 -- 172.21.15.31:0/1241233172 shutdown_connections 2023-10-06T05:26:24.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.130+0000 7f51203ff700 1 -- 172.21.15.31:0/1241233172 wait complete. 2023-10-06T05:26:24.135 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:26:24.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:26:24.190 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-10-06T05:26:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.352+0000 7ff6465ff700 1 Processor -- start 2023-10-06T05:26:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.352+0000 7ff6465ff700 1 -- start start 2023-10-06T05:26:24.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff6465ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff640155b40 0x7ff6400c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff6465ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff640158190 con 0x7ff640159c60 2023-10-06T05:26:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff6465ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff640157580 con 0x7ff640159890 2023-10-06T05:26:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff6465ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff640157860 con 0x7ff640155b40 2023-10-06T05:26:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff644b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff640155b40 0x7ff6400c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff63f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff640159c60 0x7ff640155430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33596/0 (socket says 172.21.15.31:33596) 2023-10-06T05:26:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff63f7fe700 1 -- 172.21.15.31:0/619448372 learned_addr learned my addr 172.21.15.31:0/619448372 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff644b9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff640155b40 0x7ff6400c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46306/0 (socket says 172.21.15.31:46306) 2023-10-06T05:26:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff63effd700 1 -- 172.21.15.31:0/619448372 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349624616 0 0) 0x7ff640158190 con 0x7ff640159c60 2023-10-06T05:26:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff644b9c700 1 -- 172.21.15.31:0/619448372 >> v1:172.21.15.62:6789/0 conn(0x7ff640159890 legacy=0x7ff640156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff644b9c700 1 -- 172.21.15.31:0/619448372 >> v1:172.21.15.31:6789/0 conn(0x7ff640159c60 legacy=0x7ff640155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.354+0000 7ff644b9c700 1 -- 172.21.15.31:0/619448372 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff640157b40 con 0x7ff640155b40 2023-10-06T05:26:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.355+0000 7ff644b9c700 1 --2- 172.21.15.31:0/619448372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640155b40 0x7ff6400c89e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ff638009e60 tx=0x7ff638009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9abea2b448910c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.355+0000 7ff63effd700 1 -- 172.21.15.31:0/619448372 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800bc30 con 0x7ff640155b40 2023-10-06T05:26:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.355+0000 7ff63effd700 1 -- 172.21.15.31:0/619448372 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63800bd90 con 0x7ff640155b40 2023-10-06T05:26:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.355+0000 7ff63effd700 1 -- 172.21.15.31:0/619448372 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638011650 con 0x7ff640155b40 2023-10-06T05:26:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.355+0000 7ff6465ff700 1 -- 172.21.15.31:0/619448372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640155b40 msgr2=0x7ff6400c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.355+0000 7ff6465ff700 1 --2- 172.21.15.31:0/619448372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640155b40 0x7ff6400c89e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ff638009e60 tx=0x7ff638009780 comp rx=0 tx=0).stop 2023-10-06T05:26:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.355+0000 7ff6465ff700 1 -- 172.21.15.31:0/619448372 shutdown_connections 2023-10-06T05:26:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.355+0000 7ff6465ff700 1 --2- 172.21.15.31:0/619448372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640155b40 0x7ff6400c89e0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.356+0000 7ff6465ff700 1 -- 172.21.15.31:0/619448372 >> 172.21.15.31:0/619448372 conn(0x7ff640067560 msgr2=0x7ff640067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.356+0000 7ff6465ff700 1 -- 172.21.15.31:0/619448372 shutdown_connections 2023-10-06T05:26:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.356+0000 7ff6465ff700 1 -- 172.21.15.31:0/619448372 wait complete. 2023-10-06T05:26:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.357+0000 7ff6465ff700 1 Processor -- start 2023-10-06T05:26:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.357+0000 7ff6465ff700 1 -- start start 2023-10-06T05:26:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.358+0000 7ff6465ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640155b40 0x7ff640168660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.358+0000 7ff6465ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640159890 0x7ff640152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.358+0000 7ff6465ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff640159c60 0x7ff640151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.358+0000 7ff6465ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640168c40 con 0x7ff640159890 2023-10-06T05:26:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.358+0000 7ff6465ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640168f20 con 0x7ff640155b40 2023-10-06T05:26:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.358+0000 7ff6465ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff640169200 con 0x7ff640159c60 2023-10-06T05:26:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640159890 0x7ff640152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640159890 0x7ff640152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46308/0 (socket says 172.21.15.31:46308) 2023-10-06T05:26:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63f7fe700 1 -- 172.21.15.31:0/1997636951 learned_addr learned my addr 172.21.15.31:0/1997636951 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63ffff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640155b40 0x7ff640168660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff644b9c700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff640159c60 0x7ff640151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63f7fe700 1 -- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff640159c60 msgr2=0x7ff640151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63f7fe700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff640159c60 0x7ff640151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63f7fe700 1 -- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640155b40 msgr2=0x7ff640168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63f7fe700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640155b40 0x7ff640168660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63f7fe700 1 -- 172.21.15.31:0/1997636951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638009a20 con 0x7ff640159890 2023-10-06T05:26:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63ffff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640155b40 0x7ff640168660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.359+0000 7ff63f7fe700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640159890 0x7ff640152490 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c7e0 tx=0x7ff634007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.360+0000 7ff627fff700 1 -- 172.21.15.31:0/1997636951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634005b30 con 0x7ff640159890 2023-10-06T05:26:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.360+0000 7ff627fff700 1 -- 172.21.15.31:0/1997636951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff634005c90 con 0x7ff640159890 2023-10-06T05:26:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.360+0000 7ff6465ff700 1 -- 172.21.15.31:0/1997636951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6401694e0 con 0x7ff640159890 2023-10-06T05:26:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.360+0000 7ff627fff700 1 -- 172.21.15.31:0/1997636951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634007300 con 0x7ff640159890 2023-10-06T05:26:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.360+0000 7ff6465ff700 1 -- 172.21.15.31:0/1997636951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6401529d0 con 0x7ff640159890 2023-10-06T05:26:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.361+0000 7ff6465ff700 1 -- 172.21.15.31:0/1997636951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6400c88a0 con 0x7ff640159890 2023-10-06T05:26:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.362+0000 7ff627fff700 1 -- 172.21.15.31:0/1997636951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7ff63401b400 con 0x7ff640159890 2023-10-06T05:26:24.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.363+0000 7ff627fff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff62c0679e0 0x7ff62c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:24.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.363+0000 7ff627fff700 1 -- 172.21.15.31:0/1997636951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6172+0+0 (secure 0 0 0) 0x7ff634082a30 con 0x7ff640159890 2023-10-06T05:26:24.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.363+0000 7ff63ffff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff62c0679e0 0x7ff62c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:24.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.364+0000 7ff63ffff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff62c0679e0 0x7ff62c069e90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff630001840 tx=0x7ff630009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:24.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.367+0000 7ff627fff700 1 -- 172.21.15.31:0/1997636951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6400c88a0 con 0x7ff640159890 2023-10-06T05:26:24.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:24.613+0000 7ff6465ff700 1 -- 172.21.15.31:0/1997636951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff6400c88a0 con 0x7ff640159890 2023-10-06T05:26:25.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.127+0000 7ff627fff700 1 -- 172.21.15.31:0/1997636951 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v115) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff6400c88a0 con 0x7ff640159890 2023-10-06T05:26:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 -- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff62c0679e0 msgr2=0x7ff62c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff62c0679e0 0x7ff62c069e90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff630001840 tx=0x7ff630009040 comp rx=0 tx=0).stop 2023-10-06T05:26:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 -- 172.21.15.31:0/1997636951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640159890 msgr2=0x7ff640152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640159890 0x7ff640152490 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c7e0 tx=0x7ff634007df0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 -- 172.21.15.31:0/1997636951 shutdown_connections 2023-10-06T05:26:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff62c0679e0 0x7ff62c069e90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff640159c60 0x7ff640151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640155b40 0x7ff640168660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 --2- 172.21.15.31:0/1997636951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff640159890 0x7ff640152490 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 -- 172.21.15.31:0/1997636951 >> 172.21.15.31:0/1997636951 conn(0x7ff640067560 msgr2=0x7ff640150860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 -- 172.21.15.31:0/1997636951 shutdown_connections 2023-10-06T05:26:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.131+0000 7ff6465ff700 1 -- 172.21.15.31:0/1997636951 wait complete. 2023-10-06T05:26:25.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:26:25.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.399+0000 7f03aabff700 1 Processor -- start 2023-10-06T05:26:25.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.399+0000 7f03aabff700 1 -- start start 2023-10-06T05:26:25.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.400+0000 7f03aabff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a406e300 0x7f03a4159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.400+0000 7f03aabff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a4159fe0 con 0x7f03a4159ba0 2023-10-06T05:26:25.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.400+0000 7f03aabff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a40c72b0 con 0x7f03a406df30 2023-10-06T05:26:25.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.400+0000 7f03aabff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40aba60 con 0x7f03a406e300 2023-10-06T05:26:25.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.401+0000 7f03a919c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f03a4159ba0 0x7f03a4152cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33600/0 (socket says 172.21.15.31:33600) 2023-10-06T05:26:25.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.401+0000 7f03a919c700 1 -- 172.21.15.31:0/3783832937 learned_addr learned my addr 172.21.15.31:0/3783832937 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:25.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.401+0000 7f03a3fff700 1 --2- 172.21.15.31:0/3783832937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a406e300 0x7f03a4159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.401+0000 7f03a37fe700 1 -- 172.21.15.31:0/3783832937 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102284274 0 0) 0x7f03a4159fe0 con 0x7f03a4159ba0 2023-10-06T05:26:25.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.401+0000 7f03a37fe700 1 -- 172.21.15.31:0/3783832937 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0394004680 con 0x7f03a4159ba0 2023-10-06T05:26:25.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.401+0000 7f03a37fe700 1 -- 172.21.15.31:0/3783832937 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036069984 0 0) 0x7f03a40c72b0 con 0x7f03a406df30 2023-10-06T05:26:25.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.401+0000 7f03a37fe700 1 -- 172.21.15.31:0/3783832937 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a4159fe0 con 0x7f03a406df30 2023-10-06T05:26:25.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.401+0000 7f03a3fff700 1 -- 172.21.15.31:0/3783832937 >> v1:172.21.15.62:6789/0 conn(0x7f03a406df30 legacy=0x7f03a406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.401+0000 7f03a3fff700 1 -- 172.21.15.31:0/3783832937 >> v1:172.21.15.31:6789/0 conn(0x7f03a4159ba0 legacy=0x7f03a4152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.402+0000 7f03a3fff700 1 -- 172.21.15.31:0/3783832937 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a40c7590 con 0x7f03a406e300 2023-10-06T05:26:25.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.402+0000 7f03a3fff700 1 --2- 172.21.15.31:0/3783832937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a406e300 0x7f03a4159660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0390009910 tx=0x7f03900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67da4dbdbae4240c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.402+0000 7f03a37fe700 1 -- 172.21.15.31:0/3783832937 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390011630 con 0x7f03a406e300 2023-10-06T05:26:25.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.402+0000 7f03a37fe700 1 -- 172.21.15.31:0/3783832937 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390011790 con 0x7f03a406e300 2023-10-06T05:26:25.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.402+0000 7f03a37fe700 1 -- 172.21.15.31:0/3783832937 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390011b30 con 0x7f03a406e300 2023-10-06T05:26:25.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.403+0000 7f03aabff700 1 -- 172.21.15.31:0/3783832937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a406e300 msgr2=0x7f03a4159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.403+0000 7f03aabff700 1 --2- 172.21.15.31:0/3783832937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a406e300 0x7f03a4159660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0390009910 tx=0x7f03900091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.403+0000 7f03aabff700 1 -- 172.21.15.31:0/3783832937 shutdown_connections 2023-10-06T05:26:25.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.403+0000 7f03aabff700 1 --2- 172.21.15.31:0/3783832937 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a406e300 0x7f03a4159660 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.403+0000 7f03aabff700 1 -- 172.21.15.31:0/3783832937 >> 172.21.15.31:0/3783832937 conn(0x7f03a4068030 msgr2=0x7f03a40cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:25.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.403+0000 7f03aabff700 1 -- 172.21.15.31:0/3783832937 shutdown_connections 2023-10-06T05:26:25.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.403+0000 7f03aabff700 1 -- 172.21.15.31:0/3783832937 wait complete. 2023-10-06T05:26:25.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.405+0000 7f03aabff700 1 Processor -- start 2023-10-06T05:26:25.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.405+0000 7f03aabff700 1 -- start start 2023-10-06T05:26:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03aabff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a406df30 0x7f03a40cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03aabff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03a406e300 0x7f03a4169d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03aabff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a4159ba0 0x7f03a40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03aabff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40ca0f0 con 0x7f03a406e300 2023-10-06T05:26:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03aabff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40c1820 con 0x7f03a406df30 2023-10-06T05:26:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03aabff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40ca320 con 0x7f03a4159ba0 2023-10-06T05:26:25.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03a406e300 0x7f03a4169d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03a406e300 0x7f03a4169d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46314/0 (socket says 172.21.15.31:46314) 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03a3fff700 1 -- 172.21.15.31:0/885894227 learned_addr learned my addr 172.21.15.31:0/885894227 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03a919c700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a4159ba0 0x7f03a40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03a899b700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a406df30 0x7f03a40cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03a3fff700 1 -- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a4159ba0 msgr2=0x7f03a40c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.406+0000 7f03a3fff700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a4159ba0 0x7f03a40c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.407+0000 7f03a3fff700 1 -- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a406df30 msgr2=0x7f03a40cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.407+0000 7f03a3fff700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a406df30 0x7f03a40cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.407+0000 7f03a3fff700 1 -- 172.21.15.31:0/885894227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390009420 con 0x7f03a406e300 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.407+0000 7f03a3fff700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03a406e300 0x7f03a4169d00 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f0390009f00 tx=0x7f0390017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.407+0000 7f03a0ff9700 1 -- 172.21.15.31:0/885894227 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039001f070 con 0x7f03a406e300 2023-10-06T05:26:25.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.407+0000 7f03a0ff9700 1 -- 172.21.15.31:0/885894227 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390002f70 con 0x7f03a406e300 2023-10-06T05:26:25.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.407+0000 7f03aabff700 1 -- 172.21.15.31:0/885894227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a40ca600 con 0x7f03a406e300 2023-10-06T05:26:25.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.407+0000 7f03a0ff9700 1 -- 172.21.15.31:0/885894227 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390011900 con 0x7f03a406e300 2023-10-06T05:26:25.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.407+0000 7f03aabff700 1 -- 172.21.15.31:0/885894227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a40c1440 con 0x7f03a406e300 2023-10-06T05:26:25.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.408+0000 7f03aabff700 1 -- 172.21.15.31:0/885894227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a40c19b0 con 0x7f03a406e300 2023-10-06T05:26:25.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.409+0000 7f03a0ff9700 1 -- 172.21.15.31:0/885894227 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f0390023070 con 0x7f03a406e300 2023-10-06T05:26:25.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.410+0000 7f03a0ff9700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03880679e0 0x7f0388069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:25.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.410+0000 7f03a0ff9700 1 -- 172.21.15.31:0/885894227 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f0390084370 con 0x7f03a406e300 2023-10-06T05:26:25.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.411+0000 7f03a899b700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03880679e0 0x7f0388069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:25.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.411+0000 7f03a899b700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03880679e0 0x7f0388069e90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0398003ae0 tx=0x7f0398007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.415+0000 7f03a0ff9700 1 -- 172.21.15.31:0/885894227 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0390052de0 con 0x7f03a406e300 2023-10-06T05:26:25.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:25.618+0000 7f03aabff700 1 -- 172.21.15.31:0/885894227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03a404c380 con 0x7f03a406e300 2023-10-06T05:26:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.247+0000 7f03a0ff9700 1 -- 172.21.15.31:0/885894227 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v165) v1 ==== 181+0+0 (secure 0 0 0) 0x7f03900150d0 con 0x7f03a406e300 2023-10-06T05:26:26.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.251+0000 7f03aabff700 1 -- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03880679e0 msgr2=0x7f0388069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.251+0000 7f03aabff700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03880679e0 0x7f0388069e90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0398003ae0 tx=0x7f0398007d40 comp rx=0 tx=0).stop 2023-10-06T05:26:26.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.251+0000 7f03aabff700 1 -- 172.21.15.31:0/885894227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03a406e300 msgr2=0x7f03a4169d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.251+0000 7f03aabff700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03a406e300 0x7f03a4169d00 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f0390009f00 tx=0x7f0390017000 comp rx=0 tx=0).stop 2023-10-06T05:26:26.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.251+0000 7f03aabff700 1 -- 172.21.15.31:0/885894227 shutdown_connections 2023-10-06T05:26:26.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.251+0000 7f03aabff700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f03880679e0 0x7f0388069e90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.251+0000 7f03aabff700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03a4159ba0 0x7f03a40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.251+0000 7f03aabff700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03a406df30 0x7f03a40cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.251+0000 7f03aabff700 1 --2- 172.21.15.31:0/885894227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03a406e300 0x7f03a4169d00 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.251+0000 7f03aabff700 1 -- 172.21.15.31:0/885894227 >> 172.21.15.31:0/885894227 conn(0x7f03a4068030 msgr2=0x7f03a40ade50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:26.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.252+0000 7f03aabff700 1 -- 172.21.15.31:0/885894227 shutdown_connections 2023-10-06T05:26:26.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.252+0000 7f03aabff700 1 -- 172.21.15.31:0/885894227 wait complete. 2023-10-06T05:26:26.255 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:26:26.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:26:26.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.474+0000 7f2cb01ff700 1 Processor -- start 2023-10-06T05:26:26.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.475+0000 7f2cb01ff700 1 -- start start 2023-10-06T05:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.476+0000 7f2cb01ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ca806df30 0x7f2ca806d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.476+0000 7f2cb01ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ca806d830 0x7f2ca8159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.476+0000 7f2cb01ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca80c72b0 con 0x7f2ca806e300 2023-10-06T05:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.476+0000 7f2cb01ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca80aba60 con 0x7f2ca806df30 2023-10-06T05:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.476+0000 7f2cb01ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80c7590 con 0x7f2ca806d830 2023-10-06T05:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.476+0000 7f2cadf9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ca806df30 0x7f2ca806d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.476+0000 7f2cadf9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ca806df30 0x7f2ca806d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46326/0 (socket says 172.21.15.31:46326) 2023-10-06T05:26:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.476+0000 7f2cadf9b700 1 -- 172.21.15.31:0/3442311677 learned_addr learned my addr 172.21.15.31:0/3442311677 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:26.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.476+0000 7f2cad79a700 1 --2- 172.21.15.31:0/3442311677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ca806d830 0x7f2ca8159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.476+0000 7f2cadf9b700 1 -- 172.21.15.31:0/3442311677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ca806d830 msgr2=0x7f2ca8159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:26.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.477+0000 7f2cadf9b700 1 --2- 172.21.15.31:0/3442311677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ca806d830 0x7f2ca8159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.477+0000 7f2cadf9b700 1 -- 172.21.15.31:0/3442311677 >> v1:172.21.15.62:6789/0 conn(0x7f2ca806e300 legacy=0x7f2ca8152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.477+0000 7f2cadf9b700 1 -- 172.21.15.31:0/3442311677 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80c7870 con 0x7f2ca806df30 2023-10-06T05:26:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.477+0000 7f2cadf9b700 1 --2- 172.21.15.31:0/3442311677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806df30 0x7f2ca806d2f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400ab30 tx=0x7f2ca400a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1fb28c3787e3b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.477+0000 7f2cacf99700 1 -- 172.21.15.31:0/3442311677 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4005c10 con 0x7f2ca806df30 2023-10-06T05:26:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.477+0000 7f2cacf99700 1 -- 172.21.15.31:0/3442311677 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca4005d70 con 0x7f2ca806df30 2023-10-06T05:26:26.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.477+0000 7f2cacf99700 1 -- 172.21.15.31:0/3442311677 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400ca60 con 0x7f2ca806df30 2023-10-06T05:26:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.478+0000 7f2cb01ff700 1 -- 172.21.15.31:0/3442311677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806df30 msgr2=0x7f2ca806d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.478+0000 7f2cb01ff700 1 --2- 172.21.15.31:0/3442311677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806df30 0x7f2ca806d2f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400ab30 tx=0x7f2ca400a3c0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.478+0000 7f2cb01ff700 1 -- 172.21.15.31:0/3442311677 shutdown_connections 2023-10-06T05:26:26.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.478+0000 7f2cb01ff700 1 --2- 172.21.15.31:0/3442311677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806df30 0x7f2ca806d2f0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.478+0000 7f2cb01ff700 1 --2- 172.21.15.31:0/3442311677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ca806d830 0x7f2ca8159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.478+0000 7f2cb01ff700 1 -- 172.21.15.31:0/3442311677 >> 172.21.15.31:0/3442311677 conn(0x7f2ca8068030 msgr2=0x7f2ca80cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.478+0000 7f2cb01ff700 1 -- 172.21.15.31:0/3442311677 shutdown_connections 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.478+0000 7f2cb01ff700 1 -- 172.21.15.31:0/3442311677 wait complete. 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.479+0000 7f2cb01ff700 1 Processor -- start 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.480+0000 7f2cb01ff700 1 -- start start 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cb01ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806d830 0x7f2ca80bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cb01ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca806df30 0x7f2ca8169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cb01ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ca806e300 0x7f2ca816a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cb01ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca816a970 con 0x7f2ca806d830 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cb01ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca816ac50 con 0x7f2ca806df30 2023-10-06T05:26:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cb01ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80c49f0 con 0x7f2ca806e300 2023-10-06T05:26:26.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cadf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806d830 0x7f2ca80bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cadf9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806d830 0x7f2ca80bfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46330/0 (socket says 172.21.15.31:46330) 2023-10-06T05:26:26.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cadf9b700 1 -- 172.21.15.31:0/439812118 learned_addr learned my addr 172.21.15.31:0/439812118 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:26.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cae79c700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ca806e300 0x7f2ca816a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cad79a700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca806df30 0x7f2ca8169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cadf9b700 1 -- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ca806e300 msgr2=0x7f2ca816a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cadf9b700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ca806e300 0x7f2ca816a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cadf9b700 1 -- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca806df30 msgr2=0x7f2ca8169e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:26.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cadf9b700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca806df30 0x7f2ca8169e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:26.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.481+0000 7f2cadf9b700 1 -- 172.21.15.31:0/439812118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca400a620 con 0x7f2ca806d830 2023-10-06T05:26:26.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.482+0000 7f2cadf9b700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806d830 0x7f2ca80bfae0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400c4d0 tx=0x7f2ca4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:26.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.482+0000 7f2c9dffb700 1 -- 172.21.15.31:0/439812118 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca401f070 con 0x7f2ca806d830 2023-10-06T05:26:26.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.482+0000 7f2c9dffb700 1 -- 172.21.15.31:0/439812118 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca400ca40 con 0x7f2ca806d830 2023-10-06T05:26:26.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.482+0000 7f2cb01ff700 1 -- 172.21.15.31:0/439812118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca80c4cd0 con 0x7f2ca806d830 2023-10-06T05:26:26.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.482+0000 7f2c9dffb700 1 -- 172.21.15.31:0/439812118 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4023640 con 0x7f2ca806d830 2023-10-06T05:26:26.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.482+0000 7f2cb01ff700 1 -- 172.21.15.31:0/439812118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca81547f0 con 0x7f2ca806d830 2023-10-06T05:26:26.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.483+0000 7f2cb01ff700 1 -- 172.21.15.31:0/439812118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca80c4ee0 con 0x7f2ca806d830 2023-10-06T05:26:26.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.484+0000 7f2c9dffb700 1 -- 172.21.15.31:0/439812118 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f2ca4005ee0 con 0x7f2ca806d830 2023-10-06T05:26:26.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.485+0000 7f2c9dffb700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c8c0679e0 0x7f2c8c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.485+0000 7f2c9dffb700 1 -- 172.21.15.31:0/439812118 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f2ca4084730 con 0x7f2ca806d830 2023-10-06T05:26:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.485+0000 7f2cad79a700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c8c0679e0 0x7f2c8c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.486+0000 7f2cad79a700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c8c0679e0 0x7f2c8c069e90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80c18b0 tx=0x7f2c98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.489+0000 7f2c9dffb700 1 -- 172.21.15.31:0/439812118 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ca4053140 con 0x7f2ca806d830 2023-10-06T05:26:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:26.694+0000 7f2cb01ff700 1 -- 172.21.15.31:0/439812118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2ca804c380 con 0x7f2ca806d830 2023-10-06T05:26:27.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.250+0000 7f2c9dffb700 1 -- 172.21.15.31:0/439812118 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v166) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2ca4056760 con 0x7f2ca806d830 2023-10-06T05:26:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.255+0000 7f2cb01ff700 1 -- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c8c0679e0 msgr2=0x7f2c8c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.255+0000 7f2cb01ff700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c8c0679e0 0x7f2c8c069e90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80c18b0 tx=0x7f2c98008040 comp rx=0 tx=0).stop 2023-10-06T05:26:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.255+0000 7f2cb01ff700 1 -- 172.21.15.31:0/439812118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806d830 msgr2=0x7f2ca80bfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.255+0000 7f2cb01ff700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806d830 0x7f2ca80bfae0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400c4d0 tx=0x7f2ca4019000 comp rx=0 tx=0).stop 2023-10-06T05:26:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.257+0000 7f2cb01ff700 1 -- 172.21.15.31:0/439812118 shutdown_connections 2023-10-06T05:26:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.257+0000 7f2cb01ff700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2c8c0679e0 0x7f2c8c069e90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.257+0000 7f2cb01ff700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ca806e300 0x7f2ca816a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.257+0000 7f2cb01ff700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ca806d830 0x7f2ca80bfae0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.257+0000 7f2cb01ff700 1 --2- 172.21.15.31:0/439812118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca806df30 0x7f2ca8169e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.257+0000 7f2cb01ff700 1 -- 172.21.15.31:0/439812118 >> 172.21.15.31:0/439812118 conn(0x7f2ca8068030 msgr2=0x7f2ca80a9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.258+0000 7f2cb01ff700 1 -- 172.21.15.31:0/439812118 shutdown_connections 2023-10-06T05:26:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.258+0000 7f2cb01ff700 1 -- 172.21.15.31:0/439812118 wait complete. 2023-10-06T05:26:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-10-06T05:26:27.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:26:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.478+0000 7f6b371ff700 1 Processor -- start 2023-10-06T05:26:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.478+0000 7f6b371ff700 1 -- start start 2023-10-06T05:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.479+0000 7f6b371ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b30153d40 0x7f6b300b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.479+0000 7f6b371ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b300b09c0 0x7f6b300af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.479+0000 7f6b371ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6b300af6c0 0x7f6b30159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.479+0000 7f6b371ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b300ad690 con 0x7f6b300b09c0 2023-10-06T05:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.479+0000 7f6b371ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b300b0e30 con 0x7f6b300af6c0 2023-10-06T05:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.479+0000 7f6b371ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300acd60 con 0x7f6b30153d40 2023-10-06T05:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b2ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b300b09c0 0x7f6b300af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b2ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b300b09c0 0x7f6b300af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59302/0 (socket says 172.21.15.31:59302) 2023-10-06T05:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b2ffff700 1 -- 172.21.15.31:0/2074942200 learned_addr learned my addr 172.21.15.31:0/2074942200 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b3579c700 1 --2- 172.21.15.31:0/2074942200 >> v2:172.21.15.62:3300/0 conn(0x7f6b300af6c0 0x7f6b30159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b34f9b700 1 --2- 172.21.15.31:0/2074942200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b30153d40 0x7f6b300b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b2ffff700 1 -- 172.21.15.31:0/2074942200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b30153d40 msgr2=0x7f6b300b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b2ffff700 1 --2- 172.21.15.31:0/2074942200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b30153d40 0x7f6b300b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b2ffff700 1 -- 172.21.15.31:0/2074942200 >> v2:172.21.15.62:3300/0 conn(0x7f6b300af6c0 msgr2=0x7f6b30159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:27.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b2ffff700 1 --2- 172.21.15.31:0/2074942200 >> v2:172.21.15.62:3300/0 conn(0x7f6b300af6c0 0x7f6b30159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b2ffff700 1 -- 172.21.15.31:0/2074942200 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b300ad040 con 0x7f6b300b09c0 2023-10-06T05:26:27.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.480+0000 7f6b2ffff700 1 --2- 172.21.15.31:0/2074942200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300b09c0 0x7f6b300af180 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6b3006d180 tx=0x7f6b1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51c7ebd0f1ab44f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.481+0000 7f6b2f7fe700 1 -- 172.21.15.31:0/2074942200 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00b860 con 0x7f6b300b09c0 2023-10-06T05:26:27.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.481+0000 7f6b2f7fe700 1 -- 172.21.15.31:0/2074942200 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c00b9c0 con 0x7f6b300b09c0 2023-10-06T05:26:27.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.481+0000 7f6b2f7fe700 1 -- 172.21.15.31:0/2074942200 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00bd60 con 0x7f6b300b09c0 2023-10-06T05:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.481+0000 7f6b371ff700 1 -- 172.21.15.31:0/2074942200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300b09c0 msgr2=0x7f6b300af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.481+0000 7f6b371ff700 1 --2- 172.21.15.31:0/2074942200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300b09c0 0x7f6b300af180 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6b3006d180 tx=0x7f6b1c009250 comp rx=0 tx=0).stop 2023-10-06T05:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.481+0000 7f6b371ff700 1 -- 172.21.15.31:0/2074942200 shutdown_connections 2023-10-06T05:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.481+0000 7f6b371ff700 1 --2- 172.21.15.31:0/2074942200 >> v2:172.21.15.62:3300/0 conn(0x7f6b300af6c0 0x7f6b30159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.481+0000 7f6b371ff700 1 --2- 172.21.15.31:0/2074942200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b30153d40 0x7f6b300b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.481+0000 7f6b371ff700 1 --2- 172.21.15.31:0/2074942200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300b09c0 0x7f6b300af180 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.481+0000 7f6b371ff700 1 -- 172.21.15.31:0/2074942200 >> 172.21.15.31:0/2074942200 conn(0x7f6b30067560 msgr2=0x7f6b30067960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.482+0000 7f6b371ff700 1 -- 172.21.15.31:0/2074942200 shutdown_connections 2023-10-06T05:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.482+0000 7f6b371ff700 1 -- 172.21.15.31:0/2074942200 wait complete. 2023-10-06T05:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.483+0000 7f6b371ff700 1 Processor -- start 2023-10-06T05:26:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.483+0000 7f6b371ff700 1 -- start start 2023-10-06T05:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.484+0000 7f6b371ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300af6c0 0x7f6b300c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.484+0000 7f6b371ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b300b09c0 0x7f6b30150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.484+0000 7f6b371ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b30153d40 0x7f6b300c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.484+0000 7f6b371ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300c2070 con 0x7f6b300af6c0 2023-10-06T05:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.484+0000 7f6b371ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300c16a0 con 0x7f6b30153d40 2023-10-06T05:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.484+0000 7f6b371ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3016a910 con 0x7f6b300b09c0 2023-10-06T05:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.484+0000 7f6b34f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300af6c0 0x7f6b300c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.484+0000 7f6b34f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300af6c0 0x7f6b300c1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59316/0 (socket says 172.21.15.31:59316) 2023-10-06T05:26:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.484+0000 7f6b34f9b700 1 -- 172.21.15.31:0/2171144305 learned_addr learned my addr 172.21.15.31:0/2171144305 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b2ffff700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b300b09c0 0x7f6b30150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b3579c700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b30153d40 0x7f6b300c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b34f9b700 1 -- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b300b09c0 msgr2=0x7f6b30150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b34f9b700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b300b09c0 0x7f6b30150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b34f9b700 1 -- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b30153d40 msgr2=0x7f6b300c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b34f9b700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b30153d40 0x7f6b300c27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b34f9b700 1 -- 172.21.15.31:0/2171144305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0094f0 con 0x7f6b300af6c0 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b34f9b700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300af6c0 0x7f6b300c1b30 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400d890 tx=0x7f6b24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b2cff9700 1 -- 172.21.15.31:0/2171144305 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400b600 con 0x7f6b300af6c0 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b2cff9700 1 -- 172.21.15.31:0/2171144305 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400b760 con 0x7f6b300af6c0 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b2cff9700 1 -- 172.21.15.31:0/2171144305 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400bae0 con 0x7f6b300af6c0 2023-10-06T05:26:27.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b371ff700 1 -- 172.21.15.31:0/2171144305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b300c1050 con 0x7f6b300af6c0 2023-10-06T05:26:27.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.485+0000 7f6b371ff700 1 -- 172.21.15.31:0/2171144305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b301511a0 con 0x7f6b300af6c0 2023-10-06T05:26:27.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.486+0000 7f6b371ff700 1 -- 172.21.15.31:0/2171144305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3004c380 con 0x7f6b300af6c0 2023-10-06T05:26:27.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.487+0000 7f6b2cff9700 1 -- 172.21.15.31:0/2171144305 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f6b2401e460 con 0x7f6b300af6c0 2023-10-06T05:26:27.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.488+0000 7f6b2cff9700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b140679e0 0x7f6b14069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.488+0000 7f6b2cff9700 1 -- 172.21.15.31:0/2171144305 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f6b240835d0 con 0x7f6b300af6c0 2023-10-06T05:26:27.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.489+0000 7f6b2ffff700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b140679e0 0x7f6b14069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.490+0000 7f6b2ffff700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b140679e0 0x7f6b14069e90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6b3006d180 tx=0x7f6b1c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.492+0000 7f6b2cff9700 1 -- 172.21.15.31:0/2171144305 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b2408a070 con 0x7f6b300af6c0 2023-10-06T05:26:27.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.693+0000 7f6b371ff700 1 -- 172.21.15.31:0/2171144305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6b300ad320 con 0x7f6b300af6c0 2023-10-06T05:26:27.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.697+0000 7f6b2cff9700 1 -- 172.21.15.31:0/2171144305 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6b24059e60 con 0x7f6b300af6c0 2023-10-06T05:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.699+0000 7f6b371ff700 1 -- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b140679e0 msgr2=0x7f6b14069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.699+0000 7f6b371ff700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b140679e0 0x7f6b14069e90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6b3006d180 tx=0x7f6b1c017000 comp rx=0 tx=0).stop 2023-10-06T05:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.699+0000 7f6b371ff700 1 -- 172.21.15.31:0/2171144305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300af6c0 msgr2=0x7f6b300c1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.699+0000 7f6b371ff700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300af6c0 0x7f6b300c1b30 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400d890 tx=0x7f6b24008040 comp rx=0 tx=0).stop 2023-10-06T05:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.699+0000 7f6b371ff700 1 -- 172.21.15.31:0/2171144305 shutdown_connections 2023-10-06T05:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.699+0000 7f6b371ff700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f6b140679e0 0x7f6b14069e90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.699+0000 7f6b371ff700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b30153d40 0x7f6b300c27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.699+0000 7f6b371ff700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b300af6c0 0x7f6b300c1b30 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.699+0000 7f6b371ff700 1 --2- 172.21.15.31:0/2171144305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b300b09c0 0x7f6b30150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.699+0000 7f6b371ff700 1 -- 172.21.15.31:0/2171144305 >> 172.21.15.31:0/2171144305 conn(0x7f6b30067560 msgr2=0x7f6b3006d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.700+0000 7f6b371ff700 1 -- 172.21.15.31:0/2171144305 shutdown_connections 2023-10-06T05:26:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.700+0000 7f6b371ff700 1 -- 172.21.15.31:0/2171144305 wait complete. 2023-10-06T05:26:27.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2023-10-06T05:26:27.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.917+0000 7f0580dff700 1 Processor -- start 2023-10-06T05:26:27.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.917+0000 7f0580dff700 1 -- start start 2023-10-06T05:26:27.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.918+0000 7f0580dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f057c0c7b60 0x7f057c069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.918+0000 7f0580dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f057c0c7f30 0x7f057c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.918+0000 7f0580dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057c06ef50 0x7f057c13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.918+0000 7f0580dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057c0ca5b0 con 0x7f057c0c7b60 2023-10-06T05:26:27.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.918+0000 7f0580dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057c06f740 con 0x7f057c0c7f30 2023-10-06T05:26:27.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.918+0000 7f0580dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057c0c9c80 con 0x7f057c06ef50 2023-10-06T05:26:27.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.918+0000 7f057a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f057c0c7b60 0x7f057c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.919+0000 7f057a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f057c0c7b60 0x7f057c069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59332/0 (socket says 172.21.15.31:59332) 2023-10-06T05:26:27.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.919+0000 7f057a59c700 1 -- 172.21.15.31:0/1021264807 learned_addr learned my addr 172.21.15.31:0/1021264807 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:27.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.919+0000 7f0579d9b700 1 --2- 172.21.15.31:0/1021264807 >> v2:172.21.15.62:3300/0 conn(0x7f057c0c7f30 0x7f057c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.919+0000 7f057ad9d700 1 --2- 172.21.15.31:0/1021264807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057c06ef50 0x7f057c13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.919+0000 7f057a59c700 1 -- 172.21.15.31:0/1021264807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057c06ef50 msgr2=0x7f057c13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:27.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.919+0000 7f057a59c700 1 --2- 172.21.15.31:0/1021264807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057c06ef50 0x7f057c13f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.919+0000 7f057a59c700 1 -- 172.21.15.31:0/1021264807 >> v2:172.21.15.62:3300/0 conn(0x7f057c0c7f30 msgr2=0x7f057c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:27.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.919+0000 7f057a59c700 1 --2- 172.21.15.31:0/1021264807 >> v2:172.21.15.62:3300/0 conn(0x7f057c0c7f30 0x7f057c0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.919+0000 7f057a59c700 1 -- 172.21.15.31:0/1021264807 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c0c9f60 con 0x7f057c0c7b60 2023-10-06T05:26:27.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.919+0000 7f057a59c700 1 --2- 172.21.15.31:0/1021264807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 0x7f057c069880 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f057c06c990 tx=0x7f056c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51866f5b5bd95d2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f057959a700 1 -- 172.21.15.31:0/1021264807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c005c10 con 0x7f057c0c7b60 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f057959a700 1 -- 172.21.15.31:0/1021264807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c005d70 con 0x7f057c0c7b60 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f057959a700 1 -- 172.21.15.31:0/1021264807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c00b840 con 0x7f057c0c7b60 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f0580dff700 1 -- 172.21.15.31:0/1021264807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 msgr2=0x7f057c069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f0580dff700 1 --2- 172.21.15.31:0/1021264807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 0x7f057c069880 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f057c06c990 tx=0x7f056c009220 comp rx=0 tx=0).stop 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f0580dff700 1 -- 172.21.15.31:0/1021264807 shutdown_connections 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f0580dff700 1 --2- 172.21.15.31:0/1021264807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057c06ef50 0x7f057c13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f0580dff700 1 --2- 172.21.15.31:0/1021264807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 0x7f057c069880 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f0580dff700 1 --2- 172.21.15.31:0/1021264807 >> v2:172.21.15.62:3300/0 conn(0x7f057c0c7f30 0x7f057c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f0580dff700 1 -- 172.21.15.31:0/1021264807 >> 172.21.15.31:0/1021264807 conn(0x7f057c0cc0b0 msgr2=0x7f057c0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f0580dff700 1 -- 172.21.15.31:0/1021264807 shutdown_connections 2023-10-06T05:26:27.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.920+0000 7f0580dff700 1 -- 172.21.15.31:0/1021264807 wait complete. 2023-10-06T05:26:27.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.922+0000 7f0580dff700 1 Processor -- start 2023-10-06T05:26:27.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.922+0000 7f0580dff700 1 -- start start 2023-10-06T05:26:27.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.923+0000 7f0580dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057c06ef50 0x7f057c0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.923+0000 7f0580dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 0x7f057c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.923+0000 7f0580dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057c0c7f30 0x7f057c13eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.923+0000 7f0580dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c0ae0e0 con 0x7f057c0c7b60 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.923+0000 7f0580dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c16de20 con 0x7f057c06ef50 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.923+0000 7f0580dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057c0a9d40 con 0x7f057c0c7f30 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.923+0000 7f0579d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 0x7f057c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.923+0000 7f0579d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 0x7f057c159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59340/0 (socket says 172.21.15.31:59340) 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.923+0000 7f0579d9b700 1 -- 172.21.15.31:0/2160479129 learned_addr learned my addr 172.21.15.31:0/2160479129 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f057ad9d700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057c0c7f30 0x7f057c13eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f057a59c700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057c06ef50 0x7f057c0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f0579d9b700 1 -- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057c0c7f30 msgr2=0x7f057c13eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f0579d9b700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057c0c7f30 0x7f057c13eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f0579d9b700 1 -- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057c06ef50 msgr2=0x7f057c0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f0579d9b700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057c06ef50 0x7f057c0ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f0579d9b700 1 -- 172.21.15.31:0/2160479129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c009480 con 0x7f057c0c7b60 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f057a59c700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057c06ef50 0x7f057c0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f0579d9b700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 0x7f057c159ee0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f057000d890 tx=0x7f0570008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f056a7fc700 1 -- 172.21.15.31:0/2160479129 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05700125f0 con 0x7f057c0c7b60 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f056a7fc700 1 -- 172.21.15.31:0/2160479129 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0570012770 con 0x7f057c0c7b60 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f0580dff700 1 -- 172.21.15.31:0/2160479129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c0adb20 con 0x7f057c0c7b60 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f056a7fc700 1 -- 172.21.15.31:0/2160479129 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570012b30 con 0x7f057c0c7b60 2023-10-06T05:26:27.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.924+0000 7f0580dff700 1 -- 172.21.15.31:0/2160479129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c0a99d0 con 0x7f057c0c7b60 2023-10-06T05:26:27.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.925+0000 7f0580dff700 1 -- 172.21.15.31:0/2160479129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c04c380 con 0x7f057c0c7b60 2023-10-06T05:26:27.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.926+0000 7f056a7fc700 1 -- 172.21.15.31:0/2160479129 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f057001e460 con 0x7f057c0c7b60 2023-10-06T05:26:27.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.928+0000 7f056a7fc700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f055c0679e0 0x7f055c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:27.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.928+0000 7f056a7fc700 1 -- 172.21.15.31:0/2160479129 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f0570083630 con 0x7f057c0c7b60 2023-10-06T05:26:27.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.929+0000 7f057a59c700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f055c0679e0 0x7f055c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:27.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.929+0000 7f057a59c700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f055c0679e0 0x7f055c069e90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f057c159040 tx=0x7f056c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:27.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:27.932+0000 7f056a7fc700 1 -- 172.21.15.31:0/2160479129 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0570089030 con 0x7f057c0c7b60 2023-10-06T05:26:28.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.219+0000 7f0580dff700 1 -- 172.21.15.31:0/2160479129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f057c0a9e80 con 0x7f057c0c7b60 2023-10-06T05:26:28.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.256+0000 7f056a7fc700 1 -- 172.21.15.31:0/2160479129 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v516) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0570055d00 con 0x7f057c0c7b60 2023-10-06T05:26:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 -- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f055c0679e0 msgr2=0x7f055c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f055c0679e0 0x7f055c069e90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f057c159040 tx=0x7f056c016000 comp rx=0 tx=0).stop 2023-10-06T05:26:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 -- 172.21.15.31:0/2160479129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 msgr2=0x7f057c159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 0x7f057c159ee0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f057000d890 tx=0x7f0570008040 comp rx=0 tx=0).stop 2023-10-06T05:26:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 -- 172.21.15.31:0/2160479129 shutdown_connections 2023-10-06T05:26:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f055c0679e0 0x7f055c069e90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f057c0c7f30 0x7f057c13eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f057c06ef50 0x7f057c0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 --2- 172.21.15.31:0/2160479129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f057c0c7b60 0x7f057c159ee0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 -- 172.21.15.31:0/2160479129 >> 172.21.15.31:0/2160479129 conn(0x7f057c0cc0b0 msgr2=0x7f057c0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 -- 172.21.15.31:0/2160479129 shutdown_connections 2023-10-06T05:26:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:28.258+0000 7f0580dff700 1 -- 172.21.15.31:0/2160479129 wait complete. 2023-10-06T05:26:28.278 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2023-10-06T05:26:28.278 INFO:tasks.cephfs_test_runner: 2023-10-06T05:26:28.278 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T05:26:28.278 INFO:tasks.cephfs_test_runner:Ran 11 tests in 528.186s 2023-10-06T05:26:28.278 INFO:tasks.cephfs_test_runner: 2023-10-06T05:26:28.278 INFO:tasks.cephfs_test_runner:OK 2023-10-06T05:26:28.279 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T05:26:28.291 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T05:26:28.355 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T05:26:28.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:28.355 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:26:28.402 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:26:28.403 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:26:28.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:28.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:26:28.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:28.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:26:28.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:28.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-10-06T05:26:28.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:28.669 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-10-06T05:26:28.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:28.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-10-06T05:26:28.796 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-10-06T05:26:28.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:26:28.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:28.799 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:26:28.850 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:26:28.851 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:26:28.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:28.852 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:26:28.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:28.926 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:26:29.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:29.005 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2023-10-06T05:26:29.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:29.136 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-10-06T05:26:29.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:26:29.188 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2023-10-06T05:26:29.265 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2023-10-06T05:26:29.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:26:29.268 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T05:26:29.277 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T05:26:29.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:29.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.475+0000 7f99c41ff700 1 Processor -- start 2023-10-06T05:26:29.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.475+0000 7f99c41ff700 1 -- start start 2023-10-06T05:26:29.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.476+0000 7f99c41ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7b60 0x7f99bc069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.476+0000 7f99c41ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f99bc0c7f30 0x7f99bc0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.476+0000 7f99c41ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f99bc06ef50 0x7f99bc13f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.476+0000 7f99c41ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99bc0ca5b0 con 0x7f99bc0c7f30 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.476+0000 7f99c41ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99bc06f740 con 0x7f99bc06ef50 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.476+0000 7f99c41ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc0c9c80 con 0x7f99bc0c7b60 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f99bc0c7f30 0x7f99bc0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f99bc0c7f30 0x7f99bc0aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59344/0 (socket says 172.21.15.31:59344) 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c179a700 1 -- 172.21.15.31:0/1079621458 learned_addr learned my addr 172.21.15.31:0/1079621458 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c1f9b700 1 --2- 172.21.15.31:0/1079621458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7b60 0x7f99bc069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c279c700 1 --2- 172.21.15.31:0/1079621458 >> v2:172.21.15.62:3300/0 conn(0x7f99bc06ef50 0x7f99bc13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c179a700 1 -- 172.21.15.31:0/1079621458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7b60 msgr2=0x7f99bc069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c179a700 1 --2- 172.21.15.31:0/1079621458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7b60 0x7f99bc069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c179a700 1 -- 172.21.15.31:0/1079621458 >> v2:172.21.15.62:3300/0 conn(0x7f99bc06ef50 msgr2=0x7f99bc13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c179a700 1 --2- 172.21.15.31:0/1079621458 >> v2:172.21.15.62:3300/0 conn(0x7f99bc06ef50 0x7f99bc13f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c179a700 1 -- 172.21.15.31:0/1079621458 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc0c9f60 con 0x7f99bc0c7f30 2023-10-06T05:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c279c700 1 --2- 172.21.15.31:0/1079621458 >> v2:172.21.15.62:3300/0 conn(0x7f99bc06ef50 0x7f99bc13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.477+0000 7f99c179a700 1 --2- 172.21.15.31:0/1079621458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc0c7f30 0x7f99bc0aee70 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0cbca0 tx=0x7f99ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a1248c0e4a9564 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c0f99700 1 -- 172.21.15.31:0/1079621458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac009ea0 con 0x7f99bc0c7f30 2023-10-06T05:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c0f99700 1 -- 172.21.15.31:0/1079621458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ac00b820 con 0x7f99bc0c7f30 2023-10-06T05:26:29.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c0f99700 1 -- 172.21.15.31:0/1079621458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00bbc0 con 0x7f99bc0c7f30 2023-10-06T05:26:29.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c41ff700 1 -- 172.21.15.31:0/1079621458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc0c7f30 msgr2=0x7f99bc0aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c41ff700 1 --2- 172.21.15.31:0/1079621458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc0c7f30 0x7f99bc0aee70 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0cbca0 tx=0x7f99ac009250 comp rx=0 tx=0).stop 2023-10-06T05:26:29.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c41ff700 1 -- 172.21.15.31:0/1079621458 shutdown_connections 2023-10-06T05:26:29.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c41ff700 1 --2- 172.21.15.31:0/1079621458 >> v2:172.21.15.62:3300/0 conn(0x7f99bc06ef50 0x7f99bc13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c41ff700 1 --2- 172.21.15.31:0/1079621458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7b60 0x7f99bc069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c41ff700 1 --2- 172.21.15.31:0/1079621458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc0c7f30 0x7f99bc0aee70 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c41ff700 1 -- 172.21.15.31:0/1079621458 >> 172.21.15.31:0/1079621458 conn(0x7f99bc0cc0b0 msgr2=0x7f99bc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:29.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c41ff700 1 -- 172.21.15.31:0/1079621458 shutdown_connections 2023-10-06T05:26:29.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.478+0000 7f99c41ff700 1 -- 172.21.15.31:0/1079621458 wait complete. 2023-10-06T05:26:29.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.480+0000 7f99c41ff700 1 Processor -- start 2023-10-06T05:26:29.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.480+0000 7f99c41ff700 1 -- start start 2023-10-06T05:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.481+0000 7f99c41ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc06ef50 0x7f99bc169f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.481+0000 7f99c41ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99bc0c7b60 0x7f99bc156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.481+0000 7f99c41ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7f30 0x7f99bc157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.481+0000 7f99c41ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc16aef0 con 0x7f99bc06ef50 2023-10-06T05:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.481+0000 7f99c41ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc0c5430 con 0x7f99bc0c7b60 2023-10-06T05:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.481+0000 7f99c41ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc16a650 con 0x7f99bc0c7f30 2023-10-06T05:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.481+0000 7f99c1f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc06ef50 0x7f99bc169f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.481+0000 7f99c1f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc06ef50 0x7f99bc169f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59358/0 (socket says 172.21.15.31:59358) 2023-10-06T05:26:29.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.481+0000 7f99c1f9b700 1 -- 172.21.15.31:0/1370201349 learned_addr learned my addr 172.21.15.31:0/1370201349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c279c700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7f30 0x7f99bc157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c179a700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99bc0c7b60 0x7f99bc156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c1f9b700 1 -- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7f30 msgr2=0x7f99bc157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c1f9b700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7f30 0x7f99bc157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c1f9b700 1 -- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99bc0c7b60 msgr2=0x7f99bc156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c1f9b700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99bc0c7b60 0x7f99bc156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c1f9b700 1 -- 172.21.15.31:0/1370201349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0094f0 con 0x7f99bc06ef50 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c279c700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7f30 0x7f99bc157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c179a700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99bc0c7b60 0x7f99bc156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c1f9b700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc06ef50 0x7f99bc169f50 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f99b800ec80 tx=0x7f99b8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:29.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99b1ffb700 1 -- 172.21.15.31:0/1370201349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800c9d0 con 0x7f99bc06ef50 2023-10-06T05:26:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99b1ffb700 1 -- 172.21.15.31:0/1370201349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b800cb30 con 0x7f99bc06ef50 2023-10-06T05:26:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99b1ffb700 1 -- 172.21.15.31:0/1370201349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8013470 con 0x7f99bc06ef50 2023-10-06T05:26:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c41ff700 1 -- 172.21.15.31:0/1370201349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99bc16a930 con 0x7f99bc06ef50 2023-10-06T05:26:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.482+0000 7f99c41ff700 1 -- 172.21.15.31:0/1370201349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99bc16d6f0 con 0x7f99bc06ef50 2023-10-06T05:26:29.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.483+0000 7f99c41ff700 1 -- 172.21.15.31:0/1370201349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99bc16d550 con 0x7f99bc06ef50 2023-10-06T05:26:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.484+0000 7f99b1ffb700 1 -- 172.21.15.31:0/1370201349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f99b8013820 con 0x7f99bc06ef50 2023-10-06T05:26:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.485+0000 7f99b1ffb700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99a00679e0 0x7f99a0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.485+0000 7f99b1ffb700 1 -- 172.21.15.31:0/1370201349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f99b8083660 con 0x7f99bc06ef50 2023-10-06T05:26:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.486+0000 7f99c179a700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99a00679e0 0x7f99a0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.486+0000 7f99c179a700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99a00679e0 0x7f99a0069e90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0cbca0 tx=0x7f99ac017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:29.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.489+0000 7f99b1ffb700 1 -- 172.21.15.31:0/1370201349 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99bc16d550 con 0x7f99bc06ef50 2023-10-06T05:26:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.693+0000 7f99c41ff700 1 -- 172.21.15.31:0/1370201349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99bc16d550 con 0x7f99bc06ef50 2023-10-06T05:26:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.694+0000 7f99b1ffb700 1 -- 172.21.15.31:0/1370201349 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+13881 (secure 0 0 0) 0x7f99bc16d550 con 0x7f99bc06ef50 2023-10-06T05:26:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:29.696 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":166,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:26:27.243619+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6834/3001391649":"2023-10-07T05:26:24.121507+0000","172.21.15.31:6832/3001391649":"2023-10-07T05:26:24.121507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:29.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 -- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99a00679e0 msgr2=0x7f99a0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99a00679e0 0x7f99a0069e90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0cbca0 tx=0x7f99ac017000 comp rx=0 tx=0).stop 2023-10-06T05:26:29.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 -- 172.21.15.31:0/1370201349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc06ef50 msgr2=0x7f99bc169f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc06ef50 0x7f99bc169f50 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f99b800ec80 tx=0x7f99b8009120 comp rx=0 tx=0).stop 2023-10-06T05:26:29.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 -- 172.21.15.31:0/1370201349 shutdown_connections 2023-10-06T05:26:29.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f99a00679e0 0x7f99a0069e90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f99bc0c7f30 0x7f99bc157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99bc06ef50 0x7f99bc169f50 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 --2- 172.21.15.31:0/1370201349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99bc0c7b60 0x7f99bc156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 -- 172.21.15.31:0/1370201349 >> 172.21.15.31:0/1370201349 conn(0x7f99bc0cc0b0 msgr2=0x7f99bc13ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:29.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 -- 172.21.15.31:0/1370201349 shutdown_connections 2023-10-06T05:26:29.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.696+0000 7f99c41ff700 1 -- 172.21.15.31:0/1370201349 wait complete. 2023-10-06T05:26:29.716 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T05:26:29.716 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:26:29.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.909+0000 7f41ddbff700 1 Processor -- start 2023-10-06T05:26:29.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.909+0000 7f41ddbff700 1 -- start start 2023-10-06T05:26:29.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41ddbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41d813a1e0 0x7f41d80c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41ddbff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f41d813a5b0 0x7f41d806d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41ddbff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41d80c6c30 0x7f41d806d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41ddbff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d80c71b0 con 0x7f41d80c6c30 2023-10-06T05:26:29.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41ddbff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d80c7490 con 0x7f41d813a5b0 2023-10-06T05:26:29.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41ddbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80c7770 con 0x7f41d813a1e0 2023-10-06T05:26:29.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41d7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41d80c6c30 0x7f41d806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41d7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f41d80c6c30 0x7f41d806d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59368/0 (socket says 172.21.15.31:59368) 2023-10-06T05:26:29.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41d7fff700 1 -- 172.21.15.31:0/170474223 learned_addr learned my addr 172.21.15.31:0/170474223 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:29.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41d6ffd700 1 --2- 172.21.15.31:0/170474223 >> v2:172.21.15.62:3300/0 conn(0x7f41d813a5b0 0x7f41d806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.911+0000 7f41d77fe700 1 --2- 172.21.15.31:0/170474223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41d813a1e0 0x7f41d80c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.912+0000 7f41d7fff700 1 -- 172.21.15.31:0/170474223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41d813a1e0 msgr2=0x7f41d80c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:29.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.912+0000 7f41d7fff700 1 --2- 172.21.15.31:0/170474223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41d813a1e0 0x7f41d80c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.912+0000 7f41d7fff700 1 -- 172.21.15.31:0/170474223 >> v2:172.21.15.62:3300/0 conn(0x7f41d813a5b0 msgr2=0x7f41d806d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:29.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.912+0000 7f41d7fff700 1 --2- 172.21.15.31:0/170474223 >> v2:172.21.15.62:3300/0 conn(0x7f41d813a5b0 0x7f41d806d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.912+0000 7f41d7fff700 1 -- 172.21.15.31:0/170474223 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80c7a50 con 0x7f41d80c6c30 2023-10-06T05:26:29.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.912+0000 7f41d7fff700 1 --2- 172.21.15.31:0/170474223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d80c6c30 0x7f41d806d980 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f41d80cbef0 tx=0x7f41d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a3e823e92c346b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:29.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.912+0000 7f41d67fc700 1 -- 172.21.15.31:0/170474223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000e070 con 0x7f41d80c6c30 2023-10-06T05:26:29.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.912+0000 7f41d67fc700 1 -- 172.21.15.31:0/170474223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d000b970 con 0x7f41d80c6c30 2023-10-06T05:26:29.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.912+0000 7f41d67fc700 1 -- 172.21.15.31:0/170474223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d00092e0 con 0x7f41d80c6c30 2023-10-06T05:26:29.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.913+0000 7f41ddbff700 1 -- 172.21.15.31:0/170474223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d80c6c30 msgr2=0x7f41d806d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.913+0000 7f41ddbff700 1 --2- 172.21.15.31:0/170474223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d80c6c30 0x7f41d806d980 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f41d80cbef0 tx=0x7f41d0009840 comp rx=0 tx=0).stop 2023-10-06T05:26:29.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.913+0000 7f41ddbff700 1 -- 172.21.15.31:0/170474223 shutdown_connections 2023-10-06T05:26:29.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.913+0000 7f41ddbff700 1 --2- 172.21.15.31:0/170474223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d80c6c30 0x7f41d806d980 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.913+0000 7f41ddbff700 1 --2- 172.21.15.31:0/170474223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41d813a1e0 0x7f41d80c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.913+0000 7f41ddbff700 1 --2- 172.21.15.31:0/170474223 >> v2:172.21.15.62:3300/0 conn(0x7f41d813a5b0 0x7f41d806d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.913+0000 7f41ddbff700 1 -- 172.21.15.31:0/170474223 >> 172.21.15.31:0/170474223 conn(0x7f41d80cc0b0 msgr2=0x7f41d80cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:29.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.913+0000 7f41ddbff700 1 -- 172.21.15.31:0/170474223 shutdown_connections 2023-10-06T05:26:29.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.913+0000 7f41ddbff700 1 -- 172.21.15.31:0/170474223 wait complete. 2023-10-06T05:26:29.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.914+0000 7f41ddbff700 1 Processor -- start 2023-10-06T05:26:29.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.914+0000 7f41ddbff700 1 -- start start 2023-10-06T05:26:29.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.915+0000 7f41ddbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d80c6c30 0x7f41d813ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.915+0000 7f41ddbff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d813a1e0 0x7f41d81387a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.915+0000 7f41ddbff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41d813a5b0 0x7f41d8138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.915+0000 7f41ddbff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d8154d40 con 0x7f41d813a1e0 2023-10-06T05:26:29.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.915+0000 7f41ddbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d816a8b0 con 0x7f41d80c6c30 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.915+0000 7f41ddbff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d8139220 con 0x7f41d813a5b0 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d813a1e0 0x7f41d81387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d813a1e0 0x7f41d81387a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59376/0 (socket says 172.21.15.31:59376) 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d6ffd700 1 -- 172.21.15.31:0/3092929702 learned_addr learned my addr 172.21.15.31:0/3092929702 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d7fff700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41d813a5b0 0x7f41d8138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d77fe700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d80c6c30 0x7f41d813ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d6ffd700 1 -- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41d813a5b0 msgr2=0x7f41d8138ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d6ffd700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41d813a5b0 0x7f41d8138ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d6ffd700 1 -- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d80c6c30 msgr2=0x7f41d813ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d6ffd700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d80c6c30 0x7f41d813ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d6ffd700 1 -- 172.21.15.31:0/3092929702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0009ae0 con 0x7f41d813a1e0 2023-10-06T05:26:29.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.916+0000 7f41d6ffd700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d813a1e0 0x7f41d81387a0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e930 tx=0x7f41cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:29.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.917+0000 7f41bf7fe700 1 -- 172.21.15.31:0/3092929702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc009510 con 0x7f41d813a1e0 2023-10-06T05:26:29.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.917+0000 7f41bf7fe700 1 -- 172.21.15.31:0/3092929702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc00c850 con 0x7f41d813a1e0 2023-10-06T05:26:29.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.917+0000 7f41ddbff700 1 -- 172.21.15.31:0/3092929702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d80cabd0 con 0x7f41d813a1e0 2023-10-06T05:26:29.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.917+0000 7f41bf7fe700 1 -- 172.21.15.31:0/3092929702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00cc10 con 0x7f41d813a1e0 2023-10-06T05:26:29.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.917+0000 7f41ddbff700 1 -- 172.21.15.31:0/3092929702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d80cb410 con 0x7f41d813a1e0 2023-10-06T05:26:29.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.918+0000 7f41ddbff700 1 -- 172.21.15.31:0/3092929702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d804c380 con 0x7f41d813a1e0 2023-10-06T05:26:29.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.919+0000 7f41bf7fe700 1 -- 172.21.15.31:0/3092929702 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f41cc013470 con 0x7f41d813a1e0 2023-10-06T05:26:29.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.920+0000 7f41bf7fe700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f41c40679e0 0x7f41c4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:29.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.920+0000 7f41bf7fe700 1 -- 172.21.15.31:0/3092929702 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f41cc083030 con 0x7f41d813a1e0 2023-10-06T05:26:29.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.920+0000 7f41d77fe700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f41c40679e0 0x7f41c4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:29.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.921+0000 7f41d77fe700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f41c40679e0 0x7f41c4069e90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f41d80b1bb0 tx=0x7f41c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:29.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:29.924+0000 7f41bf7fe700 1 -- 172.21.15.31:0/3092929702 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41cc051cd0 con 0x7f41d813a1e0 2023-10-06T05:26:30.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.126+0000 7f41ddbff700 1 -- 172.21.15.31:0/3092929702 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41d80c7d30 con 0x7f41c40679e0 2023-10-06T05:26:30.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.130+0000 7f41bf7fe700 1 -- 172.21.15.31:0/3092929702 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72887 (secure 0 0 0) 0x7f41d80c7d30 con 0x7f41c40679e0 2023-10-06T05:26:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:30.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.132+0000 7f41ddbff700 1 -- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f41c40679e0 msgr2=0x7f41c4069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.132+0000 7f41ddbff700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f41c40679e0 0x7f41c4069e90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f41d80b1bb0 tx=0x7f41c8008040 comp rx=0 tx=0).stop 2023-10-06T05:26:30.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.132+0000 7f41ddbff700 1 -- 172.21.15.31:0/3092929702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d813a1e0 msgr2=0x7f41d81387a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.132+0000 7f41ddbff700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d813a1e0 0x7f41d81387a0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e930 tx=0x7f41cc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.132+0000 7f41ddbff700 1 -- 172.21.15.31:0/3092929702 shutdown_connections 2023-10-06T05:26:30.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.132+0000 7f41ddbff700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f41c40679e0 0x7f41c4069e90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.132+0000 7f41ddbff700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41d813a5b0 0x7f41d8138ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.132+0000 7f41ddbff700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d80c6c30 0x7f41d813ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.132+0000 7f41ddbff700 1 --2- 172.21.15.31:0/3092929702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41d813a1e0 0x7f41d81387a0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.132+0000 7f41ddbff700 1 -- 172.21.15.31:0/3092929702 >> 172.21.15.31:0/3092929702 conn(0x7f41d80cc0b0 msgr2=0x7f41d80afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.133+0000 7f41ddbff700 1 -- 172.21.15.31:0/3092929702 shutdown_connections 2023-10-06T05:26:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.133+0000 7f41ddbff700 1 -- 172.21.15.31:0/3092929702 wait complete. 2023-10-06T05:26:30.137 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:26:30.152 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2023-10-06T05:26:29.533653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428620,"kb_used_data":17548,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546100,"statfs":{"total":773094113280,"available":770607206400,"internally_reserved":0,"allocated":17969152,"data_stored":9057058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-55322,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":-76,"num_write_kb":-84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":127,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.896726+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:26:03.896726+0000","last_peered":"2023-10-06T05:26:03.896726+0000","last_clean":"2023-10-06T05:26:03.896726+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:26:03.896726+0000","last_undegraded":"2023-10-06T05:26:03.896726+0000","last_fullsized":"2023-10-06T05:26:03.896726+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":127,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895874+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:26:03.895874+0000","last_peered":"2023-10-06T05:26:03.895874+0000","last_clean":"2023-10-06T05:26:03.895874+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:26:03.895874+0000","last_undegraded":"2023-10-06T05:26:03.895874+0000","last_fullsized":"2023-10-06T05:26:03.895874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.764860+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:26:03.764860+0000","last_peered":"2023-10-06T05:26:03.764860+0000","last_clean":"2023-10-06T05:26:03.764860+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:26:03.764860+0000","last_undegraded":"2023-10-06T05:26:03.764860+0000","last_fullsized":"2023-10-06T05:26:03.764860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895445+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:26:03.895445+0000","last_peered":"2023-10-06T05:26:03.895445+0000","last_clean":"2023-10-06T05:26:03.895445+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:26:03.895445+0000","last_undegraded":"2023-10-06T05:26:03.895445+0000","last_fullsized":"2023-10-06T05:26:03.895445+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":126,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895669+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:26:03.895669+0000","last_peered":"2023-10-06T05:26:03.895669+0000","last_clean":"2023-10-06T05:26:03.895669+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:26:03.895669+0000","last_undegraded":"2023-10-06T05:26:03.895669+0000","last_fullsized":"2023-10-06T05:26:03.895669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":315,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895351+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:26:03.895351+0000","last_peered":"2023-10-06T05:26:03.895351+0000","last_clean":"2023-10-06T05:26:03.895351+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:26:03.895351+0000","last_undegraded":"2023-10-06T05:26:03.895351+0000","last_fullsized":"2023-10-06T05:26:03.895351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":141,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.760960+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:26:03.760960+0000","last_peered":"2023-10-06T05:26:03.760960+0000","last_clean":"2023-10-06T05:26:03.760960+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:26:03.760960+0000","last_undegraded":"2023-10-06T05:26:03.760960+0000","last_fullsized":"2023-10-06T05:26:03.760960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":158,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-06T05:26:24.128127+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:26:24.128127+0000","last_peered":"2023-10-06T05:26:24.128127+0000","last_clean":"2023-10-06T05:26:24.128127+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:26:24.128127+0000","last_undegraded":"2023-10-06T05:26:24.128127+0000","last_fullsized":"2023-10-06T05:26:24.128127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":144,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-06T05:26:24.127950+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:26:24.127950+0000","last_peered":"2023-10-06T05:26:24.127950+0000","last_clean":"2023-10-06T05:26:24.127950+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:26:24.127950+0000","last_undegraded":"2023-10-06T05:26:24.127950+0000","last_fullsized":"2023-10-06T05:26:24.127950+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304060,"kb_used_data":2676,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067780,"statfs":{"total":96636764160,"available":96325406720,"internally_reserved":0,"allocated":2740224,"data_stored":1619189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68999999999999995,"15min":0.68899999999999995},"min":{"1min":0.47699999999999998,"5min":0.375,"15min":0.36099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98399999999999999,"5min":1.081,"15min":1.081},"last":0.78600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.46300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.873,"5min":0.999,"15min":1.0109999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89400000000000002,"5min":0.91700000000000004,"15min":1.018},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68400000000000005,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.41699999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.056},"last":0.622}]},{"osd":3,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.44800000000000001},"max":{"1min":1.002,"5min":1.149,"15min":1.149},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0249999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.27,"15min":1.27},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96699999999999997,"5min":0.97899999999999998,"15min":0.999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.80000000000000004},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.84999999999999998},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.1040000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79800000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.441},"max":{"1min":1.0600000000000001,"5min":1.248,"15min":1.248},"last":0.73599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303484,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068356,"statfs":{"total":96636764160,"available":96325996544,"internally_reserved":0,"allocated":2150400,"data_stored":1029608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.38500000000000001,"15min":0.32300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.8500000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66300000000000003,"15min":0.69399999999999995},"min":{"1min":0.46400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.77700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.45100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93100000000000005,"5min":1.177,"15min":1.9279999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85299999999999998,"5min":1.073,"15min":1.4239999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82799999999999996,"5min":1.093,"15min":1.369},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0549999999999999,"5min":1.085,"15min":1.891},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.51000000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99399999999999999,"5min":1.018,"15min":1.347},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.59999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96699999999999997,"5min":0.98699999999999999,"15min":1},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.753},"min":{"1min":0.46700000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1060000000000001,"5min":1.1359999999999999,"15min":1.9670000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79000000000000004,"15min":0.80500000000000005},"min":{"1min":0.49399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0600000000000001,"5min":1.278,"15min":1.4510000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.46700000000000003,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":1.03,"5min":1.0680000000000001,"15min":1.415},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1439999999999999,"5min":1.2350000000000001,"15min":2},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.60299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.071,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.628},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.60999999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0880000000000001,"5min":1.133,"15min":1.46},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303988,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067852,"statfs":{"total":96636764160,"available":96325480448,"internally_reserved":0,"allocated":2666496,"data_stored":1558844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.75800000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45200000000000001,"15min":0.48299999999999998},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53000000000000003,"5min":0.60099999999999998,"15min":0.82099999999999995},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"max":{"1min":0.58899999999999997,"5min":0.71399999999999997,"15min":0.83799999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.32400000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.753,"15min":0.78700000000000003},"last":0.624}]},{"osd":3,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.59199999999999997,"5min":0.78400000000000003,"15min":0.92400000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68799999999999994,"5min":0.68999999999999995,"15min":0.88600000000000001},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.78300000000000003},"min":{"1min":0.60899999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89900000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.754},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.55600000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91700000000000004,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71899999999999997,"15min":0.74299999999999999},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.97599999999999998,"5min":1.169,"15min":1.169},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.80400000000000005},"min":{"1min":0.56699999999999995,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":0.86199999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.85399999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71299999999999997,"15min":0.74399999999999999},"min":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88800000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.79400000000000004},"min":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.86799999999999999,"5min":1.387,"15min":1.387},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77000000000000002,"15min":0.80800000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.50600000000000001},"max":{"1min":0.93500000000000005,"5min":1.244,"15min":1.244},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78300000000000003,"15min":0.81599999999999995},"min":{"1min":0.58699999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":0.88800000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.90600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303412,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068428,"statfs":{"total":96636764160,"available":96326070272,"internally_reserved":0,"allocated":2076672,"data_stored":968566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41099999999999998,"15min":0.42699999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.84499999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.47699999999999998},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59699999999999998,"5min":0.755,"15min":1.042},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.629,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48499999999999999,"15min":0.50600000000000001},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.58199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76800000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.41599999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.90500000000000003},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.74399999999999999},"min":{"1min":0.61499999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":1.1379999999999999,"15min":1.242},"last":1.3260000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.76100000000000001},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":0.97299999999999998,"15min":1.198},"last":1.006}]},{"osd":5,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.26},"last":1.3009999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72499999999999998,"15min":0.76300000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.48099999999999998},"max":{"1min":0.874,"5min":1.1299999999999999,"15min":1.163},"last":1.038}]},{"osd":6,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89200000000000002,"5min":1.105,"15min":1.323},"last":1.1859999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.78300000000000003},"min":{"1min":0.58099999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.3100000000000001},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76100000000000001,"15min":0.81699999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.52400000000000002},"max":{"1min":0.92200000000000004,"5min":1.056,"15min":1.264},"last":1.115},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.80400000000000005},"min":{"1min":0.57599999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93500000000000005,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":1.262}]}]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303412,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068428,"statfs":{"total":96636764160,"available":96326070272,"internally_reserved":0,"allocated":2076672,"data_stored":968566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41499999999999998,"15min":0.42599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56699999999999995,"5min":0.64600000000000002,"15min":0.77800000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44700000000000001,"15min":0.47499999999999998},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.94099999999999995},"last":0.47099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.28299999999999997,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.71699999999999997,"15min":0.88700000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75600000000000001,"5min":0.80600000000000005,"15min":0.93600000000000005},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.502,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70899999999999996,"5min":0.77300000000000002,"15min":0.78900000000000003},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70899999999999996,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":1.022,"15min":1.1299999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71999999999999997,"15min":0.77300000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":0.96899999999999997,"5min":1.085,"15min":1.1699999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.76400000000000001},"min":{"1min":0.53200000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.96799999999999997,"5min":1.042,"15min":1.099},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.78500000000000003},"min":{"1min":0.53700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.163},"last":1.093}]},{"osd":6,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.754,"15min":0.81100000000000005},"min":{"1min":0.61099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94699999999999995,"5min":1.056,"15min":1.21},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.024,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77000000000000002,"15min":0.83399999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0660000000000001,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":1.2150000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303420,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068420,"statfs":{"total":96636764160,"available":96326062080,"internally_reserved":0,"allocated":2084864,"data_stored":968602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.435,"15min":0.45100000000000001},"min":{"1min":0.35199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70799999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.504},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.76200000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.36699999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.65300000000000002,"5min":0.999,"15min":0.999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.98999999999999999,"15min":1.014},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.752,"15min":0.79400000000000004},"min":{"1min":0.52400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.88300000000000001,"5min":1.1220000000000001,"15min":1.2669999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78900000000000003,"15min":0.81899999999999995},"min":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.54200000000000004},"max":{"1min":0.93200000000000005,"5min":1.113,"15min":1.3540000000000001},"last":0.78700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97199999999999998,"5min":1.093,"15min":1.204},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.81399999999999995},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98499999999999999,"5min":1.208,"15min":1.329},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.79400000000000004},"min":{"1min":0.52300000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.96099999999999997,"5min":1.0880000000000001,"15min":1.484},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.82099999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.2809999999999999,"15min":1.629},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89500000000000002,"5min":1.1379999999999999,"15min":1.5620000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79500000000000004,"15min":0.81999999999999995},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.325},"last":0.79600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303424,"kb_used_data":2040,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068416,"statfs":{"total":96636764160,"available":96326057984,"internally_reserved":0,"allocated":2088960,"data_stored":974848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.67300000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35599999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.77},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68400000000000005,"15min":0.72099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":1.0209999999999999,"15min":1.6419999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1899999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.48399999999999999,"5min":0.46700000000000003,"15min":0.39800000000000002},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72599999999999998,"15min":0.78400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.42199999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.93400000000000005},"min":{"1min":0.58599999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":7.742},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.079},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71499999999999997,"15min":0.73999999999999999},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.181},"last":0.82999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.72399999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.2669999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.436},"max":{"1min":0.98799999999999999,"5min":1.093,"15min":1.171},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.51300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0249999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.871},{"interface":"front","average":{"1min":0.751,"5min":0.78000000000000003,"15min":0.82999999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0609999999999999,"5min":1.403,"15min":1.571},"last":0.80100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303420,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068420,"statfs":{"total":96636764160,"available":96326062080,"internally_reserved":0,"allocated":2084864,"data_stored":968835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63900000000000001,"15min":0.85099999999999998},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.98099999999999998,"5min":0.98199999999999998,"15min":9.2780000000000005},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65400000000000003,"15min":0.66600000000000004},"min":{"1min":0.40699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":1.0449999999999999,"15min":1.0469999999999999},"last":0.93899999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66700000000000004,"15min":0.875},"min":{"1min":0.437,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":1.0329999999999999,"15min":9.25},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.91900000000000004},"min":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0549999999999999,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.96999999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81000000000000005,"5min":1.052,"15min":10.384},"last":1.002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70999999999999996,"15min":0.92500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":1.0309999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.95399999999999996},"min":{"1min":0.52800000000000002,"5min":0.502,"15min":0.495},"max":{"1min":0.879,"5min":1.76,"15min":10.205},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.46100000000000002},"max":{"1min":0.92600000000000005,"5min":1.097,"15min":1.292},"last":0.85599999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.069,"15min":1.069},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0409999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.86799999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.755},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77700000000000002,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.298},"last":0.88600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95899999999999996,"5min":1.556,"15min":1.556},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.79900000000000004},"min":{"1min":0.47999999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.91800000000000004,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.95299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:26:30.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:26:30.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.351+0000 7f2b039ff700 1 Processor -- start 2023-10-06T05:26:30.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.352+0000 7f2b039ff700 1 -- start start 2023-10-06T05:26:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.353+0000 7f2b039ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2afc159430 0x7f2afc068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.353+0000 7f2b039ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2afc067560 0x7f2afc069690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.353+0000 7f2b039ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2afc068650 con 0x7f2afc159800 2023-10-06T05:26:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.353+0000 7f2b039ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc067a40 con 0x7f2afc067560 2023-10-06T05:26:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.353+0000 7f2b039ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc067d20 con 0x7f2afc159430 2023-10-06T05:26:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.353+0000 7f2b00f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2afc067560 0x7f2afc069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.354+0000 7f2b00f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2afc067560 0x7f2afc069690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59382/0 (socket says 172.21.15.31:59382) 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.353+0000 7f2b01f9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2afc159800 0x7f2afc0c6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56888/0 (socket says 172.21.15.31:56888) 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.354+0000 7f2b00f9a700 1 -- 172.21.15.31:0/968929017 learned_addr learned my addr 172.21.15.31:0/968929017 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.354+0000 7f2b0179b700 1 --2- 172.21.15.31:0/968929017 >> v2:172.21.15.62:3300/0 conn(0x7f2afc159430 0x7f2afc068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.354+0000 7f2b00f9a700 1 -- 172.21.15.31:0/968929017 >> v2:172.21.15.62:3300/0 conn(0x7f2afc159430 msgr2=0x7f2afc068ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.354+0000 7f2b00f9a700 1 --2- 172.21.15.31:0/968929017 >> v2:172.21.15.62:3300/0 conn(0x7f2afc159430 0x7f2afc068ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.354+0000 7f2b00f9a700 1 -- 172.21.15.31:0/968929017 >> v1:172.21.15.31:6789/0 conn(0x7f2afc159800 legacy=0x7f2afc0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.354+0000 7f2b00f9a700 1 -- 172.21.15.31:0/968929017 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc068000 con 0x7f2afc067560 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.354+0000 7f2b00f9a700 1 --2- 172.21.15.31:0/968929017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc067560 0x7f2afc069690 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f2afc06a2f0 tx=0x7f2aec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed2c017013a1c93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.354+0000 7f2af3fff700 1 -- 172.21.15.31:0/968929017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00b860 con 0x7f2afc067560 2023-10-06T05:26:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.355+0000 7f2af3fff700 1 -- 172.21.15.31:0/968929017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec00b9c0 con 0x7f2afc067560 2023-10-06T05:26:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.355+0000 7f2af3fff700 1 -- 172.21.15.31:0/968929017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00bd60 con 0x7f2afc067560 2023-10-06T05:26:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.355+0000 7f2b039ff700 1 -- 172.21.15.31:0/968929017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc067560 msgr2=0x7f2afc069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.355+0000 7f2b039ff700 1 --2- 172.21.15.31:0/968929017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc067560 0x7f2afc069690 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f2afc06a2f0 tx=0x7f2aec009250 comp rx=0 tx=0).stop 2023-10-06T05:26:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.355+0000 7f2b039ff700 1 -- 172.21.15.31:0/968929017 shutdown_connections 2023-10-06T05:26:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.355+0000 7f2b039ff700 1 --2- 172.21.15.31:0/968929017 >> v2:172.21.15.62:3300/0 conn(0x7f2afc159430 0x7f2afc068ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.355+0000 7f2b039ff700 1 --2- 172.21.15.31:0/968929017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc067560 0x7f2afc069690 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.355+0000 7f2b039ff700 1 -- 172.21.15.31:0/968929017 >> 172.21.15.31:0/968929017 conn(0x7f2afc0c9320 msgr2=0x7f2afc0c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.355+0000 7f2b039ff700 1 -- 172.21.15.31:0/968929017 shutdown_connections 2023-10-06T05:26:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.355+0000 7f2b039ff700 1 -- 172.21.15.31:0/968929017 wait complete. 2023-10-06T05:26:30.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.357+0000 7f2b039ff700 1 Processor -- start 2023-10-06T05:26:30.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.357+0000 7f2b039ff700 1 -- start start 2023-10-06T05:26:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b039ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2afc067560 0x7f2afc0c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b039ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc159430 0x7f2afc15bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b039ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc159800 0x7f2afc0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b039ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc15c750 con 0x7f2afc159800 2023-10-06T05:26:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b039ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc15ca30 con 0x7f2afc159430 2023-10-06T05:26:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b039ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc15cd10 con 0x7f2afc067560 2023-10-06T05:26:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b01f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc159800 0x7f2afc0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b01f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc159800 0x7f2afc0c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59396/0 (socket says 172.21.15.31:59396) 2023-10-06T05:26:30.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b01f9c700 1 -- 172.21.15.31:0/636460944 learned_addr learned my addr 172.21.15.31:0/636460944 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b0179b700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2afc067560 0x7f2afc0c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.358+0000 7f2b00f9a700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc159430 0x7f2afc15bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2b01f9c700 1 -- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2afc067560 msgr2=0x7f2afc0c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2b01f9c700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2afc067560 0x7f2afc0c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2b01f9c700 1 -- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc159430 msgr2=0x7f2afc15bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2b01f9c700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc159430 0x7f2afc15bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2b01f9c700 1 -- 172.21.15.31:0/636460944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0094f0 con 0x7f2afc159800 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2b01f9c700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc159800 0x7f2afc0c7c00 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f2af400cc70 tx=0x7f2af4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2af17fa700 1 -- 172.21.15.31:0/636460944 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4005df0 con 0x7f2afc159800 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2af17fa700 1 -- 172.21.15.31:0/636460944 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af400a640 con 0x7f2afc159800 2023-10-06T05:26:30.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2b039ff700 1 -- 172.21.15.31:0/636460944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc15cff0 con 0x7f2afc159800 2023-10-06T05:26:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2af17fa700 1 -- 172.21.15.31:0/636460944 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af400a9c0 con 0x7f2afc159800 2023-10-06T05:26:30.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.359+0000 7f2b039ff700 1 -- 172.21.15.31:0/636460944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc0c6670 con 0x7f2afc159800 2023-10-06T05:26:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.360+0000 7f2b039ff700 1 -- 172.21.15.31:0/636460944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc15d510 con 0x7f2afc159800 2023-10-06T05:26:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.361+0000 7f2af17fa700 1 -- 172.21.15.31:0/636460944 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f2af40070f0 con 0x7f2afc159800 2023-10-06T05:26:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.362+0000 7f2af17fa700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2ae00679e0 0x7f2ae0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.362+0000 7f2af17fa700 1 -- 172.21.15.31:0/636460944 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f2af4082e60 con 0x7f2afc159800 2023-10-06T05:26:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.363+0000 7f2b0179b700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2ae00679e0 0x7f2ae0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.363+0000 7f2b0179b700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2ae00679e0 0x7f2ae0069e90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2af8006fd0 tx=0x7f2af8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.366+0000 7f2af17fa700 1 -- 172.21.15.31:0/636460944 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2afc15d510 con 0x7f2afc159800 2023-10-06T05:26:30.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.565+0000 7f2b039ff700 1 -- 172.21.15.31:0/636460944 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2afc15d510 con 0x7f2ae00679e0 2023-10-06T05:26:30.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.569+0000 7f2af17fa700 1 -- 172.21.15.31:0/636460944 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72887 (secure 0 0 0) 0x7f2afc15d510 con 0x7f2ae00679e0 2023-10-06T05:26:30.572 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:30.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.571+0000 7f2b039ff700 1 -- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2ae00679e0 msgr2=0x7f2ae0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.571+0000 7f2b039ff700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2ae00679e0 0x7f2ae0069e90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2af8006fd0 tx=0x7f2af8008040 comp rx=0 tx=0).stop 2023-10-06T05:26:30.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.571+0000 7f2b039ff700 1 -- 172.21.15.31:0/636460944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc159800 msgr2=0x7f2afc0c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.571+0000 7f2b039ff700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc159800 0x7f2afc0c7c00 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f2af400cc70 tx=0x7f2af4001a40 comp rx=0 tx=0).stop 2023-10-06T05:26:30.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.571+0000 7f2b039ff700 1 -- 172.21.15.31:0/636460944 shutdown_connections 2023-10-06T05:26:30.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.571+0000 7f2b039ff700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f2ae00679e0 0x7f2ae0069e90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.571+0000 7f2b039ff700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2afc159800 0x7f2afc0c7c00 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.571+0000 7f2b039ff700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2afc067560 0x7f2afc0c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.571+0000 7f2b039ff700 1 --2- 172.21.15.31:0/636460944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc159430 0x7f2afc15bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.571+0000 7f2b039ff700 1 -- 172.21.15.31:0/636460944 >> 172.21.15.31:0/636460944 conn(0x7f2afc0c9320 msgr2=0x7f2afc0c1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.572+0000 7f2b039ff700 1 -- 172.21.15.31:0/636460944 shutdown_connections 2023-10-06T05:26:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.572+0000 7f2b039ff700 1 -- 172.21.15.31:0/636460944 wait complete. 2023-10-06T05:26:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:26:30.591 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2023-10-06T05:26:29.533653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428620,"kb_used_data":17548,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546100,"statfs":{"total":773094113280,"available":770607206400,"internally_reserved":0,"allocated":17969152,"data_stored":9057058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-55322,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":-76,"num_write_kb":-84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":127,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.896726+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:26:03.896726+0000","last_peered":"2023-10-06T05:26:03.896726+0000","last_clean":"2023-10-06T05:26:03.896726+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:26:03.896726+0000","last_undegraded":"2023-10-06T05:26:03.896726+0000","last_fullsized":"2023-10-06T05:26:03.896726+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":127,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895874+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:26:03.895874+0000","last_peered":"2023-10-06T05:26:03.895874+0000","last_clean":"2023-10-06T05:26:03.895874+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:26:03.895874+0000","last_undegraded":"2023-10-06T05:26:03.895874+0000","last_fullsized":"2023-10-06T05:26:03.895874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.764860+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:26:03.764860+0000","last_peered":"2023-10-06T05:26:03.764860+0000","last_clean":"2023-10-06T05:26:03.764860+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:26:03.764860+0000","last_undegraded":"2023-10-06T05:26:03.764860+0000","last_fullsized":"2023-10-06T05:26:03.764860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895445+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:26:03.895445+0000","last_peered":"2023-10-06T05:26:03.895445+0000","last_clean":"2023-10-06T05:26:03.895445+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:26:03.895445+0000","last_undegraded":"2023-10-06T05:26:03.895445+0000","last_fullsized":"2023-10-06T05:26:03.895445+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":126,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895669+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:26:03.895669+0000","last_peered":"2023-10-06T05:26:03.895669+0000","last_clean":"2023-10-06T05:26:03.895669+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:26:03.895669+0000","last_undegraded":"2023-10-06T05:26:03.895669+0000","last_fullsized":"2023-10-06T05:26:03.895669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":315,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895351+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:26:03.895351+0000","last_peered":"2023-10-06T05:26:03.895351+0000","last_clean":"2023-10-06T05:26:03.895351+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:26:03.895351+0000","last_undegraded":"2023-10-06T05:26:03.895351+0000","last_fullsized":"2023-10-06T05:26:03.895351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":141,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.760960+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:26:03.760960+0000","last_peered":"2023-10-06T05:26:03.760960+0000","last_clean":"2023-10-06T05:26:03.760960+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:26:03.760960+0000","last_undegraded":"2023-10-06T05:26:03.760960+0000","last_fullsized":"2023-10-06T05:26:03.760960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":158,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-06T05:26:24.128127+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:26:24.128127+0000","last_peered":"2023-10-06T05:26:24.128127+0000","last_clean":"2023-10-06T05:26:24.128127+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:26:24.128127+0000","last_undegraded":"2023-10-06T05:26:24.128127+0000","last_fullsized":"2023-10-06T05:26:24.128127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":144,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-06T05:26:24.127950+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:26:24.127950+0000","last_peered":"2023-10-06T05:26:24.127950+0000","last_clean":"2023-10-06T05:26:24.127950+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:26:24.127950+0000","last_undegraded":"2023-10-06T05:26:24.127950+0000","last_fullsized":"2023-10-06T05:26:24.127950+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304060,"kb_used_data":2676,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067780,"statfs":{"total":96636764160,"available":96325406720,"internally_reserved":0,"allocated":2740224,"data_stored":1619189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68999999999999995,"15min":0.68899999999999995},"min":{"1min":0.47699999999999998,"5min":0.375,"15min":0.36099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98399999999999999,"5min":1.081,"15min":1.081},"last":0.78600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.46300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.873,"5min":0.999,"15min":1.0109999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89400000000000002,"5min":0.91700000000000004,"15min":1.018},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68400000000000005,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.41699999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.056},"last":0.622}]},{"osd":3,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.44800000000000001},"max":{"1min":1.002,"5min":1.149,"15min":1.149},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0249999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.27,"15min":1.27},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96699999999999997,"5min":0.97899999999999998,"15min":0.999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.80000000000000004},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.84999999999999998},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.1040000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79800000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.441},"max":{"1min":1.0600000000000001,"5min":1.248,"15min":1.248},"last":0.73599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303484,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068356,"statfs":{"total":96636764160,"available":96325996544,"internally_reserved":0,"allocated":2150400,"data_stored":1029608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.38500000000000001,"15min":0.32300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.8500000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66300000000000003,"15min":0.69399999999999995},"min":{"1min":0.46400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.77700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.45100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93100000000000005,"5min":1.177,"15min":1.9279999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85299999999999998,"5min":1.073,"15min":1.4239999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82799999999999996,"5min":1.093,"15min":1.369},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0549999999999999,"5min":1.085,"15min":1.891},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.51000000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99399999999999999,"5min":1.018,"15min":1.347},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.59999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96699999999999997,"5min":0.98699999999999999,"15min":1},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.753},"min":{"1min":0.46700000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1060000000000001,"5min":1.1359999999999999,"15min":1.9670000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79000000000000004,"15min":0.80500000000000005},"min":{"1min":0.49399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0600000000000001,"5min":1.278,"15min":1.4510000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.46700000000000003,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":1.03,"5min":1.0680000000000001,"15min":1.415},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1439999999999999,"5min":1.2350000000000001,"15min":2},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.60299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.071,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.628},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.60999999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0880000000000001,"5min":1.133,"15min":1.46},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303988,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067852,"statfs":{"total":96636764160,"available":96325480448,"internally_reserved":0,"allocated":2666496,"data_stored":1558844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.75800000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45200000000000001,"15min":0.48299999999999998},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53000000000000003,"5min":0.60099999999999998,"15min":0.82099999999999995},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"max":{"1min":0.58899999999999997,"5min":0.71399999999999997,"15min":0.83799999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.32400000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.753,"15min":0.78700000000000003},"last":0.624}]},{"osd":3,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.59199999999999997,"5min":0.78400000000000003,"15min":0.92400000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68799999999999994,"5min":0.68999999999999995,"15min":0.88600000000000001},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.78300000000000003},"min":{"1min":0.60899999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89900000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.754},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.55600000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91700000000000004,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71899999999999997,"15min":0.74299999999999999},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.97599999999999998,"5min":1.169,"15min":1.169},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.80400000000000005},"min":{"1min":0.56699999999999995,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":0.86199999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.85399999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71299999999999997,"15min":0.74399999999999999},"min":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88800000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.79400000000000004},"min":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.86799999999999999,"5min":1.387,"15min":1.387},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77000000000000002,"15min":0.80800000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.50600000000000001},"max":{"1min":0.93500000000000005,"5min":1.244,"15min":1.244},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78300000000000003,"15min":0.81599999999999995},"min":{"1min":0.58699999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":0.88800000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.90600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303412,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068428,"statfs":{"total":96636764160,"available":96326070272,"internally_reserved":0,"allocated":2076672,"data_stored":968566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41099999999999998,"15min":0.42699999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.84499999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.47699999999999998},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59699999999999998,"5min":0.755,"15min":1.042},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.629,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48499999999999999,"15min":0.50600000000000001},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.58199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76800000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.41599999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.90500000000000003},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.74399999999999999},"min":{"1min":0.61499999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":1.1379999999999999,"15min":1.242},"last":1.3260000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.76100000000000001},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":0.97299999999999998,"15min":1.198},"last":1.006}]},{"osd":5,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.26},"last":1.3009999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72499999999999998,"15min":0.76300000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.48099999999999998},"max":{"1min":0.874,"5min":1.1299999999999999,"15min":1.163},"last":1.038}]},{"osd":6,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89200000000000002,"5min":1.105,"15min":1.323},"last":1.1859999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.78300000000000003},"min":{"1min":0.58099999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.3100000000000001},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76100000000000001,"15min":0.81699999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.52400000000000002},"max":{"1min":0.92200000000000004,"5min":1.056,"15min":1.264},"last":1.115},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.80400000000000005},"min":{"1min":0.57599999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93500000000000005,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":1.262}]}]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303412,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068428,"statfs":{"total":96636764160,"available":96326070272,"internally_reserved":0,"allocated":2076672,"data_stored":968566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41499999999999998,"15min":0.42599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56699999999999995,"5min":0.64600000000000002,"15min":0.77800000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44700000000000001,"15min":0.47499999999999998},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.94099999999999995},"last":0.47099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.28299999999999997,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.71699999999999997,"15min":0.88700000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75600000000000001,"5min":0.80600000000000005,"15min":0.93600000000000005},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.502,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70899999999999996,"5min":0.77300000000000002,"15min":0.78900000000000003},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70899999999999996,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":1.022,"15min":1.1299999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71999999999999997,"15min":0.77300000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":0.96899999999999997,"5min":1.085,"15min":1.1699999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.76400000000000001},"min":{"1min":0.53200000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.96799999999999997,"5min":1.042,"15min":1.099},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.78500000000000003},"min":{"1min":0.53700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.163},"last":1.093}]},{"osd":6,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.754,"15min":0.81100000000000005},"min":{"1min":0.61099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94699999999999995,"5min":1.056,"15min":1.21},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.024,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77000000000000002,"15min":0.83399999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0660000000000001,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":1.2150000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303420,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068420,"statfs":{"total":96636764160,"available":96326062080,"internally_reserved":0,"allocated":2084864,"data_stored":968602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.435,"15min":0.45100000000000001},"min":{"1min":0.35199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70799999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.504},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.76200000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.36699999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.65300000000000002,"5min":0.999,"15min":0.999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.98999999999999999,"15min":1.014},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.752,"15min":0.79400000000000004},"min":{"1min":0.52400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.88300000000000001,"5min":1.1220000000000001,"15min":1.2669999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78900000000000003,"15min":0.81899999999999995},"min":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.54200000000000004},"max":{"1min":0.93200000000000005,"5min":1.113,"15min":1.3540000000000001},"last":0.78700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97199999999999998,"5min":1.093,"15min":1.204},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.81399999999999995},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98499999999999999,"5min":1.208,"15min":1.329},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.79400000000000004},"min":{"1min":0.52300000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.96099999999999997,"5min":1.0880000000000001,"15min":1.484},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.82099999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.2809999999999999,"15min":1.629},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89500000000000002,"5min":1.1379999999999999,"15min":1.5620000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79500000000000004,"15min":0.81999999999999995},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.325},"last":0.79600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303424,"kb_used_data":2040,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068416,"statfs":{"total":96636764160,"available":96326057984,"internally_reserved":0,"allocated":2088960,"data_stored":974848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.67300000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35599999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.77},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68400000000000005,"15min":0.72099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":1.0209999999999999,"15min":1.6419999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1899999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.48399999999999999,"5min":0.46700000000000003,"15min":0.39800000000000002},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72599999999999998,"15min":0.78400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.42199999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.93400000000000005},"min":{"1min":0.58599999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":7.742},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.079},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71499999999999997,"15min":0.73999999999999999},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.181},"last":0.82999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.72399999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.2669999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.436},"max":{"1min":0.98799999999999999,"5min":1.093,"15min":1.171},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.51300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0249999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.871},{"interface":"front","average":{"1min":0.751,"5min":0.78000000000000003,"15min":0.82999999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0609999999999999,"5min":1.403,"15min":1.571},"last":0.80100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303420,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068420,"statfs":{"total":96636764160,"available":96326062080,"internally_reserved":0,"allocated":2084864,"data_stored":968835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63900000000000001,"15min":0.85099999999999998},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.98099999999999998,"5min":0.98199999999999998,"15min":9.2780000000000005},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65400000000000003,"15min":0.66600000000000004},"min":{"1min":0.40699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":1.0449999999999999,"15min":1.0469999999999999},"last":0.93899999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66700000000000004,"15min":0.875},"min":{"1min":0.437,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":1.0329999999999999,"15min":9.25},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.91900000000000004},"min":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0549999999999999,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.96999999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81000000000000005,"5min":1.052,"15min":10.384},"last":1.002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70999999999999996,"15min":0.92500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":1.0309999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.95399999999999996},"min":{"1min":0.52800000000000002,"5min":0.502,"15min":0.495},"max":{"1min":0.879,"5min":1.76,"15min":10.205},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.46100000000000002},"max":{"1min":0.92600000000000005,"5min":1.097,"15min":1.292},"last":0.85599999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.069,"15min":1.069},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0409999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.86799999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.755},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77700000000000002,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.298},"last":0.88600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95899999999999996,"5min":1.556,"15min":1.556},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.79900000000000004},"min":{"1min":0.47999999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.91800000000000004,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.95299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:26:30.593 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T05:26:30.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:26:30.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.787+0000 7ff6d3bff700 1 Processor -- start 2023-10-06T05:26:30.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.787+0000 7ff6d3bff700 1 -- start start 2023-10-06T05:26:30.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.788+0000 7ff6d3bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 0x7ff6cc159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.788+0000 7ff6d3bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc0c95b0 con 0x7ff6cc068d80 2023-10-06T05:26:30.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.788+0000 7ff6d3bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc0c9890 con 0x7ff6cc1595a0 2023-10-06T05:26:30.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.788+0000 7ff6d3bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0c9b70 con 0x7ff6cc0c9040 2023-10-06T05:26:30.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.789+0000 7ff6d119a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff6cc068d80 0x7ff6cc0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56902/0 (socket says 172.21.15.31:56902) 2023-10-06T05:26:30.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.789+0000 7ff6d119a700 1 -- 172.21.15.31:0/552188577 learned_addr learned my addr 172.21.15.31:0/552188577 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:30.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.789+0000 7ff6d219c700 1 --2- 172.21.15.31:0/552188577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 0x7ff6cc159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.789+0000 7ff6d0999700 1 -- 172.21.15.31:0/552188577 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555425806 0 0) 0x7ff6cc0c95b0 con 0x7ff6cc068d80 2023-10-06T05:26:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.789+0000 7ff6d0999700 1 -- 172.21.15.31:0/552188577 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6b8003570 con 0x7ff6cc068d80 2023-10-06T05:26:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.789+0000 7ff6d0999700 1 -- 172.21.15.31:0/552188577 <== mon.1 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893864937 0 0) 0x7ff6cc0c9890 con 0x7ff6cc1595a0 2023-10-06T05:26:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.789+0000 7ff6d0999700 1 -- 172.21.15.31:0/552188577 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6cc0c95b0 con 0x7ff6cc1595a0 2023-10-06T05:26:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.789+0000 7ff6d219c700 1 -- 172.21.15.31:0/552188577 >> v1:172.21.15.62:6789/0 conn(0x7ff6cc1595a0 legacy=0x7ff6cc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.789+0000 7ff6d219c700 1 -- 172.21.15.31:0/552188577 >> v1:172.21.15.31:6789/0 conn(0x7ff6cc068d80 legacy=0x7ff6cc0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.789+0000 7ff6d219c700 1 -- 172.21.15.31:0/552188577 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc0ca130 con 0x7ff6cc0c9040 2023-10-06T05:26:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.790+0000 7ff6d219c700 1 --2- 172.21.15.31:0/552188577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 0x7ff6cc159970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009ec0 tx=0x7ff6c4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be40ae0f97c750f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.790+0000 7ff6d0999700 1 -- 172.21.15.31:0/552188577 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c400e070 con 0x7ff6cc0c9040 2023-10-06T05:26:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.790+0000 7ff6d0999700 1 -- 172.21.15.31:0/552188577 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c400bb40 con 0x7ff6cc0c9040 2023-10-06T05:26:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.790+0000 7ff6d0999700 1 -- 172.21.15.31:0/552188577 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4012430 con 0x7ff6cc0c9040 2023-10-06T05:26:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.790+0000 7ff6d3bff700 1 -- 172.21.15.31:0/552188577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 msgr2=0x7ff6cc159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.790+0000 7ff6d3bff700 1 --2- 172.21.15.31:0/552188577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 0x7ff6cc159970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009ec0 tx=0x7ff6c4009780 comp rx=0 tx=0).stop 2023-10-06T05:26:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.790+0000 7ff6d3bff700 1 -- 172.21.15.31:0/552188577 shutdown_connections 2023-10-06T05:26:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.790+0000 7ff6d3bff700 1 --2- 172.21.15.31:0/552188577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 0x7ff6cc159970 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.790+0000 7ff6d3bff700 1 -- 172.21.15.31:0/552188577 >> 172.21.15.31:0/552188577 conn(0x7ff6cc06b620 msgr2=0x7ff6cc06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.791+0000 7ff6d3bff700 1 -- 172.21.15.31:0/552188577 shutdown_connections 2023-10-06T05:26:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.791+0000 7ff6d3bff700 1 -- 172.21.15.31:0/552188577 wait complete. 2023-10-06T05:26:30.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.792+0000 7ff6d3bff700 1 Processor -- start 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.792+0000 7ff6d3bff700 1 -- start start 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.793+0000 7ff6d3bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6cc068d80 0x7ff6cc0c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.793+0000 7ff6d3bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 0x7ff6cc155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.793+0000 7ff6d3bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6cc1595a0 0x7ff6cc153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.793+0000 7ff6d3bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0c6640 con 0x7ff6cc068d80 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.793+0000 7ff6d3bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0c6920 con 0x7ff6cc1595a0 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.793+0000 7ff6d3bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0c6c00 con 0x7ff6cc0c9040 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.793+0000 7ff6d199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6cc068d80 0x7ff6cc0c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.793+0000 7ff6d199b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6cc068d80 0x7ff6cc0c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59404/0 (socket says 172.21.15.31:59404) 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.793+0000 7ff6d199b700 1 -- 172.21.15.31:0/172762264 learned_addr learned my addr 172.21.15.31:0/172762264 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:30.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6d119a700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 0x7ff6cc155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6d219c700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6cc1595a0 0x7ff6cc153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6d199b700 1 -- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 msgr2=0x7ff6cc155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6d199b700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 0x7ff6cc155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6d199b700 1 -- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6cc1595a0 msgr2=0x7ff6cc153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6d199b700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6cc1595a0 0x7ff6cc153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6d199b700 1 -- 172.21.15.31:0/172762264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c4009a20 con 0x7ff6cc068d80 2023-10-06T05:26:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6d119a700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 0x7ff6cc155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6d199b700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6cc068d80 0x7ff6cc0c5db0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800c580 tx=0x7ff6c8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6c1ffb700 1 -- 172.21.15.31:0/172762264 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8010510 con 0x7ff6cc068d80 2023-10-06T05:26:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6c1ffb700 1 -- 172.21.15.31:0/172762264 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c8010670 con 0x7ff6cc068d80 2023-10-06T05:26:30.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6c1ffb700 1 -- 172.21.15.31:0/172762264 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8010a30 con 0x7ff6cc068d80 2023-10-06T05:26:30.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.794+0000 7ff6d3bff700 1 -- 172.21.15.31:0/172762264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc0c6ee0 con 0x7ff6cc068d80 2023-10-06T05:26:30.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.795+0000 7ff6d3bff700 1 -- 172.21.15.31:0/172762264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc0c62f0 con 0x7ff6cc068d80 2023-10-06T05:26:30.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.796+0000 7ff6d3bff700 1 -- 172.21.15.31:0/172762264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc04c380 con 0x7ff6cc068d80 2023-10-06T05:26:30.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.796+0000 7ff6c1ffb700 1 -- 172.21.15.31:0/172762264 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7ff6c801b400 con 0x7ff6cc068d80 2023-10-06T05:26:30.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.798+0000 7ff6c1ffb700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6b00679e0 0x7ff6b0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:30.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.798+0000 7ff6c1ffb700 1 -- 172.21.15.31:0/172762264 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7ff6c8082ba0 con 0x7ff6cc068d80 2023-10-06T05:26:30.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.798+0000 7ff6d119a700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6b00679e0 0x7ff6b0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:30.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.799+0000 7ff6d119a700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6b00679e0 0x7ff6b0069e90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc15f9a0 tx=0x7ff6bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.802+0000 7ff6c1ffb700 1 -- 172.21.15.31:0/172762264 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6c80517a0 con 0x7ff6cc068d80 2023-10-06T05:26:31.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:30.998+0000 7ff6d3bff700 1 -- 172.21.15.31:0/172762264 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff6cc0b40c0 con 0x7ff6b00679e0 2023-10-06T05:26:31.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.002+0000 7ff6c1ffb700 1 -- 172.21.15.31:0/172762264 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72887 (secure 0 0 0) 0x7ff6cc0b40c0 con 0x7ff6b00679e0 2023-10-06T05:26:31.005 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:31.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.004+0000 7ff6d3bff700 1 -- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6b00679e0 msgr2=0x7ff6b0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:31.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.004+0000 7ff6d3bff700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6b00679e0 0x7ff6b0069e90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc15f9a0 tx=0x7ff6bc009040 comp rx=0 tx=0).stop 2023-10-06T05:26:31.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.004+0000 7ff6d3bff700 1 -- 172.21.15.31:0/172762264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6cc068d80 msgr2=0x7ff6cc0c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:31.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.004+0000 7ff6d3bff700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6cc068d80 0x7ff6cc0c5db0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800c580 tx=0x7ff6c8007e10 comp rx=0 tx=0).stop 2023-10-06T05:26:31.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.005+0000 7ff6d3bff700 1 -- 172.21.15.31:0/172762264 shutdown_connections 2023-10-06T05:26:31.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.005+0000 7ff6d3bff700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7ff6b00679e0 0x7ff6b0069e90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.005+0000 7ff6d3bff700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6cc1595a0 0x7ff6cc153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.005+0000 7ff6d3bff700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6cc068d80 0x7ff6cc0c5db0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.005+0000 7ff6d3bff700 1 --2- 172.21.15.31:0/172762264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6cc0c9040 0x7ff6cc155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.005+0000 7ff6d3bff700 1 -- 172.21.15.31:0/172762264 >> 172.21.15.31:0/172762264 conn(0x7ff6cc06b620 msgr2=0x7ff6cc0ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:31.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.005+0000 7ff6d3bff700 1 -- 172.21.15.31:0/172762264 shutdown_connections 2023-10-06T05:26:31.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.005+0000 7ff6d3bff700 1 -- 172.21.15.31:0/172762264 wait complete. 2023-10-06T05:26:31.008 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:26:31.024 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2023-10-06T05:26:29.533653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428620,"kb_used_data":17548,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546100,"statfs":{"total":773094113280,"available":770607206400,"internally_reserved":0,"allocated":17969152,"data_stored":9057058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-55322,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":-76,"num_write_kb":-84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":127,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.896726+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:26:03.896726+0000","last_peered":"2023-10-06T05:26:03.896726+0000","last_clean":"2023-10-06T05:26:03.896726+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:26:03.896726+0000","last_undegraded":"2023-10-06T05:26:03.896726+0000","last_fullsized":"2023-10-06T05:26:03.896726+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":127,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895874+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:26:03.895874+0000","last_peered":"2023-10-06T05:26:03.895874+0000","last_clean":"2023-10-06T05:26:03.895874+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:26:03.895874+0000","last_undegraded":"2023-10-06T05:26:03.895874+0000","last_fullsized":"2023-10-06T05:26:03.895874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.764860+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:26:03.764860+0000","last_peered":"2023-10-06T05:26:03.764860+0000","last_clean":"2023-10-06T05:26:03.764860+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:26:03.764860+0000","last_undegraded":"2023-10-06T05:26:03.764860+0000","last_fullsized":"2023-10-06T05:26:03.764860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895445+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:26:03.895445+0000","last_peered":"2023-10-06T05:26:03.895445+0000","last_clean":"2023-10-06T05:26:03.895445+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:26:03.895445+0000","last_undegraded":"2023-10-06T05:26:03.895445+0000","last_fullsized":"2023-10-06T05:26:03.895445+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":126,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895669+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:26:03.895669+0000","last_peered":"2023-10-06T05:26:03.895669+0000","last_clean":"2023-10-06T05:26:03.895669+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:26:03.895669+0000","last_undegraded":"2023-10-06T05:26:03.895669+0000","last_fullsized":"2023-10-06T05:26:03.895669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":315,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.895351+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:26:03.895351+0000","last_peered":"2023-10-06T05:26:03.895351+0000","last_clean":"2023-10-06T05:26:03.895351+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:26:03.895351+0000","last_undegraded":"2023-10-06T05:26:03.895351+0000","last_fullsized":"2023-10-06T05:26:03.895351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":141,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T05:26:03.760960+0000","last_change":"2023-10-06T05:16:55.390688+0000","last_active":"2023-10-06T05:26:03.760960+0000","last_peered":"2023-10-06T05:26:03.760960+0000","last_clean":"2023-10-06T05:26:03.760960+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:26:03.760960+0000","last_undegraded":"2023-10-06T05:26:03.760960+0000","last_fullsized":"2023-10-06T05:26:03.760960+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":158,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-06T05:26:24.128127+0000","last_change":"2023-10-06T05:16:55.391768+0000","last_active":"2023-10-06T05:26:24.128127+0000","last_peered":"2023-10-06T05:26:24.128127+0000","last_clean":"2023-10-06T05:26:24.128127+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:26:24.128127+0000","last_undegraded":"2023-10-06T05:26:24.128127+0000","last_fullsized":"2023-10-06T05:26:24.128127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":144,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-06T05:26:24.127950+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:26:24.127950+0000","last_peered":"2023-10-06T05:26:24.127950+0000","last_clean":"2023-10-06T05:26:24.127950+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:26:24.127950+0000","last_undegraded":"2023-10-06T05:26:24.127950+0000","last_fullsized":"2023-10-06T05:26:24.127950+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:45.599793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304060,"kb_used_data":2676,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067780,"statfs":{"total":96636764160,"available":96325406720,"internally_reserved":0,"allocated":2740224,"data_stored":1619189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68999999999999995,"15min":0.68899999999999995},"min":{"1min":0.47699999999999998,"5min":0.375,"15min":0.36099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98399999999999999,"5min":1.081,"15min":1.081},"last":0.78600000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.46300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.873,"5min":0.999,"15min":1.0109999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89400000000000002,"5min":0.91700000000000004,"15min":1.018},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68400000000000005,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.41699999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.056},"last":0.622}]},{"osd":3,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.44800000000000001},"max":{"1min":1.002,"5min":1.149,"15min":1.149},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0249999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.27,"15min":1.27},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96699999999999997,"5min":0.97899999999999998,"15min":0.999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.80000000000000004},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.84999999999999998},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.1040000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79800000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.441},"max":{"1min":1.0600000000000001,"5min":1.248,"15min":1.248},"last":0.73599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303484,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068356,"statfs":{"total":96636764160,"available":96325996544,"internally_reserved":0,"allocated":2150400,"data_stored":1029608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.38500000000000001,"15min":0.32300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.8500000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66300000000000003,"15min":0.69399999999999995},"min":{"1min":0.46400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.77700000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.45100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93100000000000005,"5min":1.177,"15min":1.9279999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85299999999999998,"5min":1.073,"15min":1.4239999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82799999999999996,"5min":1.093,"15min":1.369},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0549999999999999,"5min":1.085,"15min":1.891},"last":0.83299999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.51000000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99399999999999999,"5min":1.018,"15min":1.347},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.59999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96699999999999997,"5min":0.98699999999999999,"15min":1},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.753},"min":{"1min":0.46700000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1060000000000001,"5min":1.1359999999999999,"15min":1.9670000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79000000000000004,"15min":0.80500000000000005},"min":{"1min":0.49399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0600000000000001,"5min":1.278,"15min":1.4510000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.46700000000000003,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":1.03,"5min":1.0680000000000001,"15min":1.415},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1439999999999999,"5min":1.2350000000000001,"15min":2},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.60299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.071,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.628},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.60999999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0880000000000001,"5min":1.133,"15min":1.46},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303988,"kb_used_data":2604,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067852,"statfs":{"total":96636764160,"available":96325480448,"internally_reserved":0,"allocated":2666496,"data_stored":1558844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.75800000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45200000000000001,"15min":0.48299999999999998},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53000000000000003,"5min":0.60099999999999998,"15min":0.82099999999999995},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"max":{"1min":0.58899999999999997,"5min":0.71399999999999997,"15min":0.83799999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.32400000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.753,"15min":0.78700000000000003},"last":0.624}]},{"osd":3,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.59199999999999997,"5min":0.78400000000000003,"15min":0.92400000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68799999999999994,"5min":0.68999999999999995,"15min":0.88600000000000001},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.78300000000000003},"min":{"1min":0.60899999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89900000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.754},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.55600000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91700000000000004,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71899999999999997,"15min":0.74299999999999999},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.97599999999999998,"5min":1.169,"15min":1.169},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.80400000000000005},"min":{"1min":0.56699999999999995,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":0.86199999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.85399999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71299999999999997,"15min":0.74399999999999999},"min":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88800000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.79400000000000004},"min":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.86799999999999999,"5min":1.387,"15min":1.387},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77000000000000002,"15min":0.80800000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.50600000000000001},"max":{"1min":0.93500000000000005,"5min":1.244,"15min":1.244},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78300000000000003,"15min":0.81599999999999995},"min":{"1min":0.58699999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":0.88800000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.90600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303412,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068428,"statfs":{"total":96636764160,"available":96326070272,"internally_reserved":0,"allocated":2076672,"data_stored":968566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41099999999999998,"15min":0.42699999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.84499999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.47699999999999998},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59699999999999998,"5min":0.755,"15min":1.042},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.629,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48499999999999999,"15min":0.50600000000000001},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.58199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76800000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.41599999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.90500000000000003},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.74399999999999999},"min":{"1min":0.61499999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":1.1379999999999999,"15min":1.242},"last":1.3260000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.76100000000000001},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":0.97299999999999998,"15min":1.198},"last":1.006}]},{"osd":5,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.26},"last":1.3009999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72499999999999998,"15min":0.76300000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.48099999999999998},"max":{"1min":0.874,"5min":1.1299999999999999,"15min":1.163},"last":1.038}]},{"osd":6,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89200000000000002,"5min":1.105,"15min":1.323},"last":1.1859999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.78300000000000003},"min":{"1min":0.58099999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.3100000000000001},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76100000000000001,"15min":0.81699999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.52400000000000002},"max":{"1min":0.92200000000000004,"5min":1.056,"15min":1.264},"last":1.115},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.80400000000000005},"min":{"1min":0.57599999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93500000000000005,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":1.262}]}]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303412,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068428,"statfs":{"total":96636764160,"available":96326070272,"internally_reserved":0,"allocated":2076672,"data_stored":968566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41499999999999998,"15min":0.42599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56699999999999995,"5min":0.64600000000000002,"15min":0.77800000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44700000000000001,"15min":0.47499999999999998},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.94099999999999995},"last":0.47099999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.28299999999999997,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.71699999999999997,"15min":0.88700000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75600000000000001,"5min":0.80600000000000005,"15min":0.93600000000000005},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.502,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70899999999999996,"5min":0.77300000000000002,"15min":0.78900000000000003},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70899999999999996,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":1.022,"15min":1.1299999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71999999999999997,"15min":0.77300000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":0.96899999999999997,"5min":1.085,"15min":1.1699999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.76400000000000001},"min":{"1min":0.53200000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.96799999999999997,"5min":1.042,"15min":1.099},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.78500000000000003},"min":{"1min":0.53700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.163},"last":1.093}]},{"osd":6,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.754,"15min":0.81100000000000005},"min":{"1min":0.61099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94699999999999995,"5min":1.056,"15min":1.21},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.024,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77000000000000002,"15min":0.83399999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0660000000000001,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":1.2150000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303420,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068420,"statfs":{"total":96636764160,"available":96326062080,"internally_reserved":0,"allocated":2084864,"data_stored":968602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.435,"15min":0.45100000000000001},"min":{"1min":0.35199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70799999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.504},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.76200000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.36699999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.65300000000000002,"5min":0.999,"15min":0.999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.98999999999999999,"15min":1.014},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.752,"15min":0.79400000000000004},"min":{"1min":0.52400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.88300000000000001,"5min":1.1220000000000001,"15min":1.2669999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78900000000000003,"15min":0.81899999999999995},"min":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.54200000000000004},"max":{"1min":0.93200000000000005,"5min":1.113,"15min":1.3540000000000001},"last":0.78700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97199999999999998,"5min":1.093,"15min":1.204},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.81399999999999995},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98499999999999999,"5min":1.208,"15min":1.329},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.79400000000000004},"min":{"1min":0.52300000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.96099999999999997,"5min":1.0880000000000001,"15min":1.484},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.82099999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.2809999999999999,"15min":1.629},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89500000000000002,"5min":1.1379999999999999,"15min":1.5620000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79500000000000004,"15min":0.81999999999999995},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.325},"last":0.79600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303424,"kb_used_data":2040,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068416,"statfs":{"total":96636764160,"available":96326057984,"internally_reserved":0,"allocated":2088960,"data_stored":974848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.67300000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35599999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.77},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68400000000000005,"15min":0.72099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":1.0209999999999999,"15min":1.6419999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1899999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.48399999999999999,"5min":0.46700000000000003,"15min":0.39800000000000002},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72599999999999998,"15min":0.78400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.42199999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.93400000000000005},"min":{"1min":0.58599999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":7.742},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.079},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71499999999999997,"15min":0.73999999999999999},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.181},"last":0.82999999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.72399999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.2669999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.436},"max":{"1min":0.98799999999999999,"5min":1.093,"15min":1.171},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.51300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0249999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.871},{"interface":"front","average":{"1min":0.751,"5min":0.78000000000000003,"15min":0.82999999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0609999999999999,"5min":1.403,"15min":1.571},"last":0.80100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303420,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068420,"statfs":{"total":96636764160,"available":96326062080,"internally_reserved":0,"allocated":2084864,"data_stored":968835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63900000000000001,"15min":0.85099999999999998},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.98099999999999998,"5min":0.98199999999999998,"15min":9.2780000000000005},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65400000000000003,"15min":0.66600000000000004},"min":{"1min":0.40699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":1.0449999999999999,"15min":1.0469999999999999},"last":0.93899999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66700000000000004,"15min":0.875},"min":{"1min":0.437,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":1.0329999999999999,"15min":9.25},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.91900000000000004},"min":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0549999999999999,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.96999999999999997}]},{"osd":2,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81000000000000005,"5min":1.052,"15min":10.384},"last":1.002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70999999999999996,"15min":0.92500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":1.0309999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.95399999999999996},"min":{"1min":0.52800000000000002,"5min":0.502,"15min":0.495},"max":{"1min":0.879,"5min":1.76,"15min":10.205},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.46100000000000002},"max":{"1min":0.92600000000000005,"5min":1.097,"15min":1.292},"last":0.85599999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.069,"15min":1.069},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0409999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.86799999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.755},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77700000000000002,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.298},"last":0.88600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95899999999999996,"5min":1.556,"15min":1.556},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.79900000000000004},"min":{"1min":0.47999999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.91800000000000004,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.95299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:26:31.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:26:31.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.220+0000 7f4797dff700 1 Processor -- start 2023-10-06T05:26:31.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.220+0000 7f4797dff700 1 -- start start 2023-10-06T05:26:31.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4797dff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f47900c7b60 0x7f4790069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:31.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4797dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47900c7f30 0x7f47900aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:31.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4797dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f479006ef50 0x7f479013f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:31.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4797dff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47900ca5b0 con 0x7f47900c7b60 2023-10-06T05:26:31.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4797dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479006f740 con 0x7f479006ef50 2023-10-06T05:26:31.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4797dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47900c9c80 con 0x7f47900c7f30 2023-10-06T05:26:31.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4795b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f47900c7b60 0x7f4790069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4795b9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f47900c7b60 0x7f4790069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59416/0 (socket says 172.21.15.31:59416) 2023-10-06T05:26:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4795b9b700 1 -- 172.21.15.31:0/3432066742 learned_addr learned my addr 172.21.15.31:0/3432066742 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f479639c700 1 --2- 172.21.15.31:0/3432066742 >> v2:172.21.15.62:3300/0 conn(0x7f479006ef50 0x7f479013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4795b9b700 1 -- 172.21.15.31:0/3432066742 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47900c7f30 msgr2=0x7f47900aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4795b9b700 1 --2- 172.21.15.31:0/3432066742 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47900c7f30 0x7f47900aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4795b9b700 1 -- 172.21.15.31:0/3432066742 >> v2:172.21.15.62:3300/0 conn(0x7f479006ef50 msgr2=0x7f479013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4795b9b700 1 --2- 172.21.15.31:0/3432066742 >> v2:172.21.15.62:3300/0 conn(0x7f479006ef50 0x7f479013f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f4795b9b700 1 -- 172.21.15.31:0/3432066742 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47900c9f60 con 0x7f47900c7b60 2023-10-06T05:26:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.222+0000 7f479639c700 1 --2- 172.21.15.31:0/3432066742 >> v2:172.21.15.62:3300/0 conn(0x7f479006ef50 0x7f479013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.223+0000 7f4795b9b700 1 --2- 172.21.15.31:0/3432066742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 0x7f4790069880 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f47900cbca0 tx=0x7f478c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeea8cdf15e8684 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.223+0000 7f4794b99700 1 -- 172.21.15.31:0/3432066742 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c005c10 con 0x7f47900c7b60 2023-10-06T05:26:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.223+0000 7f4794b99700 1 -- 172.21.15.31:0/3432066742 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c005d70 con 0x7f47900c7b60 2023-10-06T05:26:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.223+0000 7f4794b99700 1 -- 172.21.15.31:0/3432066742 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c00b880 con 0x7f47900c7b60 2023-10-06T05:26:31.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.223+0000 7f4797dff700 1 -- 172.21.15.31:0/3432066742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 msgr2=0x7f4790069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:31.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.223+0000 7f4797dff700 1 --2- 172.21.15.31:0/3432066742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 0x7f4790069880 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f47900cbca0 tx=0x7f478c009220 comp rx=0 tx=0).stop 2023-10-06T05:26:31.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.224+0000 7f4797dff700 1 -- 172.21.15.31:0/3432066742 shutdown_connections 2023-10-06T05:26:31.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.224+0000 7f4797dff700 1 --2- 172.21.15.31:0/3432066742 >> v2:172.21.15.62:3300/0 conn(0x7f479006ef50 0x7f479013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.224+0000 7f4797dff700 1 --2- 172.21.15.31:0/3432066742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 0x7f4790069880 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.224+0000 7f4797dff700 1 --2- 172.21.15.31:0/3432066742 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47900c7f30 0x7f47900aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.224+0000 7f4797dff700 1 -- 172.21.15.31:0/3432066742 >> 172.21.15.31:0/3432066742 conn(0x7f47900cc0b0 msgr2=0x7f47900cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:31.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.224+0000 7f4797dff700 1 -- 172.21.15.31:0/3432066742 shutdown_connections 2023-10-06T05:26:31.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.224+0000 7f4797dff700 1 -- 172.21.15.31:0/3432066742 wait complete. 2023-10-06T05:26:31.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.226+0000 7f4797dff700 1 Processor -- start 2023-10-06T05:26:31.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.226+0000 7f4797dff700 1 -- start start 2023-10-06T05:26:31.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.227+0000 7f4797dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f479006ef50 0x7f47900c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:31.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.227+0000 7f4797dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 0x7f47900b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:31.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.227+0000 7f4797dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47900c7f30 0x7f47900af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:31.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.227+0000 7f4797dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47900c64c0 con 0x7f47900c7b60 2023-10-06T05:26:31.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.227+0000 7f4797dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47901569c0 con 0x7f47900c7f30 2023-10-06T05:26:31.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.227+0000 7f4797dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4790156ca0 con 0x7f479006ef50 2023-10-06T05:26:31.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f479539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 0x7f47900b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:31.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f479539a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 0x7f47900b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59420/0 (socket says 172.21.15.31:59420) 2023-10-06T05:26:31.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f479539a700 1 -- 172.21.15.31:0/1777503911 learned_addr learned my addr 172.21.15.31:0/1777503911 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:31.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f4795b9b700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f479006ef50 0x7f47900c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:31.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f479639c700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47900c7f30 0x7f47900af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:31.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f479539a700 1 -- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f479006ef50 msgr2=0x7f47900c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:31.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f479539a700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f479006ef50 0x7f47900c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f479539a700 1 -- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47900c7f30 msgr2=0x7f47900af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:31.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f479539a700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47900c7f30 0x7f47900af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f479539a700 1 -- 172.21.15.31:0/1777503911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c009480 con 0x7f47900c7b60 2023-10-06T05:26:31.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f479539a700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 0x7f47900b0ff0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f478000b5e0 tx=0x7f47800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:31.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.228+0000 7f4785ffb700 1 -- 172.21.15.31:0/1777503911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478000f510 con 0x7f47900c7b60 2023-10-06T05:26:31.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.229+0000 7f4785ffb700 1 -- 172.21.15.31:0/1777503911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478000f690 con 0x7f47900c7b60 2023-10-06T05:26:31.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.229+0000 7f4785ffb700 1 -- 172.21.15.31:0/1777503911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478000fa50 con 0x7f47900c7b60 2023-10-06T05:26:31.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.229+0000 7f4797dff700 1 -- 172.21.15.31:0/1777503911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4790156f80 con 0x7f47900c7b60 2023-10-06T05:26:31.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.229+0000 7f4797dff700 1 -- 172.21.15.31:0/1777503911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47900a9940 con 0x7f47900c7b60 2023-10-06T05:26:31.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.230+0000 7f4797dff700 1 -- 172.21.15.31:0/1777503911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479004c380 con 0x7f47900c7b60 2023-10-06T05:26:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.231+0000 7f4785ffb700 1 -- 172.21.15.31:0/1777503911 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f478001a400 con 0x7f47900c7b60 2023-10-06T05:26:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.231+0000 7f4785ffb700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f47740679e0 0x7f4774069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.232+0000 7f4785ffb700 1 -- 172.21.15.31:0/1777503911 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f4780082950 con 0x7f47900c7b60 2023-10-06T05:26:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.232+0000 7f4795b9b700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f47740679e0 0x7f4774069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.233+0000 7f4795b9b700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f47740679e0 0x7f4774069e90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f478c007fd0 tx=0x7f478c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:31.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.236+0000 7f4785ffb700 1 -- 172.21.15.31:0/1777503911 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4780051670 con 0x7f47900c7b60 2023-10-06T05:26:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.437+0000 7f4797dff700 1 -- 172.21.15.31:0/1777503911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47900b19c0 con 0x7f47900c7b60 2023-10-06T05:26:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.438+0000 7f4785ffb700 1 -- 172.21.15.31:0/1777503911 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+13881 (secure 0 0 0) 0x7f4780054c90 con 0x7f47900c7b60 2023-10-06T05:26:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:31.440 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":166,"fsid":"86bc8148-d401-41f1-bd7f-edc6b2417154","created":"2023-10-06T05:16:41.135424+0000","modified":"2023-10-06T05:26:27.243619+0000","last_up_change":"2023-10-06T05:16:53.258246+0000","last_in_change":"2023-10-06T05:16:44.603355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T05:16:53.806841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T05:16:54.115786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5ce3e1c-3a1c-41d0-add7-c2c866710e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":114853},{"type":"v1","addr":"172.21.15.31:6801","nonce":114853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":114853},{"type":"v1","addr":"172.21.15.31:6803","nonce":114853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":114853},{"type":"v1","addr":"172.21.15.31:6807","nonce":114853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":114853},{"type":"v1","addr":"172.21.15.31:6805","nonce":114853}]},"public_addr":"172.21.15.31:6801/114853","cluster_addr":"172.21.15.31:6803/114853","heartbeat_back_addr":"172.21.15.31:6807/114853","heartbeat_front_addr":"172.21.15.31:6805/114853","state":["exists","up"]},{"osd":1,"uuid":"171d8e7f-7775-4532-a010-d9e0a9c28b12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":114859},{"type":"v1","addr":"172.21.15.31:6817","nonce":114859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":114859},{"type":"v1","addr":"172.21.15.31:6819","nonce":114859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":114859},{"type":"v1","addr":"172.21.15.31:6824","nonce":114859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":114859},{"type":"v1","addr":"172.21.15.31:6821","nonce":114859}]},"public_addr":"172.21.15.31:6817/114859","cluster_addr":"172.21.15.31:6819/114859","heartbeat_back_addr":"172.21.15.31:6824/114859","heartbeat_front_addr":"172.21.15.31:6821/114859","state":["exists","up"]},{"osd":2,"uuid":"128ab23f-d6cc-48db-bd7f-fc80834fcf18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":114858},{"type":"v1","addr":"172.21.15.31:6809","nonce":114858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":114858},{"type":"v1","addr":"172.21.15.31:6811","nonce":114858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":114858},{"type":"v1","addr":"172.21.15.31:6815","nonce":114858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":114858},{"type":"v1","addr":"172.21.15.31:6813","nonce":114858}]},"public_addr":"172.21.15.31:6809/114858","cluster_addr":"172.21.15.31:6811/114858","heartbeat_back_addr":"172.21.15.31:6815/114858","heartbeat_front_addr":"172.21.15.31:6813/114858","state":["exists","up"]},{"osd":3,"uuid":"a7b8fafe-138c-48fa-93bf-86376c93ee02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6823","nonce":114865},{"type":"v1","addr":"172.21.15.31:6825","nonce":114865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":114865},{"type":"v1","addr":"172.21.15.31:6827","nonce":114865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":114865},{"type":"v1","addr":"172.21.15.31:6831","nonce":114865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":114865},{"type":"v1","addr":"172.21.15.31:6829","nonce":114865}]},"public_addr":"172.21.15.31:6825/114865","cluster_addr":"172.21.15.31:6827/114865","heartbeat_back_addr":"172.21.15.31:6831/114865","heartbeat_front_addr":"172.21.15.31:6829/114865","state":["exists","up"]},{"osd":4,"uuid":"19501bab-f99b-4ea0-9a75-455fd46c87ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":115118},{"type":"v1","addr":"172.21.15.62:6801","nonce":115118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":115118},{"type":"v1","addr":"172.21.15.62:6803","nonce":115118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":115118},{"type":"v1","addr":"172.21.15.62:6807","nonce":115118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":115118},{"type":"v1","addr":"172.21.15.62:6805","nonce":115118}]},"public_addr":"172.21.15.62:6801/115118","cluster_addr":"172.21.15.62:6803/115118","heartbeat_back_addr":"172.21.15.62:6807/115118","heartbeat_front_addr":"172.21.15.62:6805/115118","state":["exists","up"]},{"osd":5,"uuid":"370a0c4f-8324-451d-b90f-607c426a9c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":115134},{"type":"v1","addr":"172.21.15.62:6817","nonce":115134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":115134},{"type":"v1","addr":"172.21.15.62:6819","nonce":115134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":115134},{"type":"v1","addr":"172.21.15.62:6826","nonce":115134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":115134},{"type":"v1","addr":"172.21.15.62:6822","nonce":115134}]},"public_addr":"172.21.15.62:6817/115134","cluster_addr":"172.21.15.62:6819/115134","heartbeat_back_addr":"172.21.15.62:6826/115134","heartbeat_front_addr":"172.21.15.62:6822/115134","state":["exists","up"]},{"osd":6,"uuid":"ad324207-ac27-4cab-ab02-1c0b5fa7de64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6821","nonce":115137},{"type":"v1","addr":"172.21.15.62:6823","nonce":115137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6825","nonce":115137},{"type":"v1","addr":"172.21.15.62:6827","nonce":115137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":115137},{"type":"v1","addr":"172.21.15.62:6831","nonce":115137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":115137},{"type":"v1","addr":"172.21.15.62:6829","nonce":115137}]},"public_addr":"172.21.15.62:6823/115137","cluster_addr":"172.21.15.62:6827/115137","heartbeat_back_addr":"172.21.15.62:6831/115137","heartbeat_front_addr":"172.21.15.62:6829/115137","state":["exists","up"]},{"osd":7,"uuid":"44fd2503-f934-4417-85de-16dcfb16d465","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":115119},{"type":"v1","addr":"172.21.15.62:6809","nonce":115119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":115119},{"type":"v1","addr":"172.21.15.62:6811","nonce":115119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":115119},{"type":"v1","addr":"172.21.15.62:6815","nonce":115119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":115119},{"type":"v1","addr":"172.21.15.62:6813","nonce":115119}]},"public_addr":"172.21.15.62:6809/115119","cluster_addr":"172.21.15.62:6811/115119","heartbeat_back_addr":"172.21.15.62:6815/115119","heartbeat_front_addr":"172.21.15.62:6813/115119","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.900308+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.939040+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.949270+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.003214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.901298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.961107+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:50.006977+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T05:16:49.940217+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6834/3001391649":"2023-10-07T05:26:24.121507+0000","172.21.15.31:6832/3001391649":"2023-10-07T05:26:24.121507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:26:31.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.440+0000 7f4797dff700 1 -- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f47740679e0 msgr2=0x7f4774069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:31.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.440+0000 7f4797dff700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f47740679e0 0x7f4774069e90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f478c007fd0 tx=0x7f478c016000 comp rx=0 tx=0).stop 2023-10-06T05:26:31.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.440+0000 7f4797dff700 1 -- 172.21.15.31:0/1777503911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 msgr2=0x7f47900b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:31.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.440+0000 7f4797dff700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 0x7f47900b0ff0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f478000b5e0 tx=0x7f47800091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.440+0000 7f4797dff700 1 -- 172.21.15.31:0/1777503911 shutdown_connections 2023-10-06T05:26:31.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.440+0000 7f4797dff700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f47740679e0 0x7f4774069e90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.440+0000 7f4797dff700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47900c7f30 0x7f47900af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.440+0000 7f4797dff700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f479006ef50 0x7f47900c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.440+0000 7f4797dff700 1 --2- 172.21.15.31:0/1777503911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47900c7b60 0x7f47900b0ff0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.440+0000 7f4797dff700 1 -- 172.21.15.31:0/1777503911 >> 172.21.15.31:0/1777503911 conn(0x7f47900cc0b0 msgr2=0x7f4790154b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.441+0000 7f4797dff700 1 -- 172.21.15.31:0/1777503911 shutdown_connections 2023-10-06T05:26:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:31.441+0000 7f4797dff700 1 -- 172.21.15.31:0/1777503911 wait complete. 2023-10-06T05:26:32.460 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T05:26:32.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:26:32.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:32.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:26:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.678+0000 7fb6723ff700 1 Processor -- start 2023-10-06T05:26:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.678+0000 7fb6723ff700 1 -- start start 2023-10-06T05:26:32.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.680+0000 7fb6723ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb66c159410 0x7fb66c0c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.680+0000 7fb6723ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb66c06b4c0 0x7fb66c1597e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.680+0000 7fb6723ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66c069750 con 0x7fb66c0c73c0 2023-10-06T05:26:32.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.680+0000 7fb6723ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66c0c7860 con 0x7fb66c159410 2023-10-06T05:26:32.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.680+0000 7fb6723ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66c069a30 con 0x7fb66c06b4c0 2023-10-06T05:26:32.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.680+0000 7fb66bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb66c159410 0x7fb66c0c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.680+0000 7fb66b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb66c0c73c0 0x7fb66c06adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56918/0 (socket says 172.21.15.31:56918) 2023-10-06T05:26:32.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.680+0000 7fb66bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb66c159410 0x7fb66c0c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59426/0 (socket says 172.21.15.31:59426) 2023-10-06T05:26:32.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.680+0000 7fb66b7fe700 1 -- 172.21.15.31:0/2394613553 learned_addr learned my addr 172.21.15.31:0/2394613553 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:32.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.681+0000 7fb67099c700 1 --2- 172.21.15.31:0/2394613553 >> v2:172.21.15.62:3300/0 conn(0x7fb66c06b4c0 0x7fb66c1597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.681+0000 7fb66bfff700 1 -- 172.21.15.31:0/2394613553 >> v2:172.21.15.62:3300/0 conn(0x7fb66c06b4c0 msgr2=0x7fb66c1597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:32.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.681+0000 7fb66bfff700 1 --2- 172.21.15.31:0/2394613553 >> v2:172.21.15.62:3300/0 conn(0x7fb66c06b4c0 0x7fb66c1597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.681+0000 7fb66bfff700 1 -- 172.21.15.31:0/2394613553 >> v1:172.21.15.31:6789/0 conn(0x7fb66c0c73c0 legacy=0x7fb66c06adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.681+0000 7fb66bfff700 1 -- 172.21.15.31:0/2394613553 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c069d10 con 0x7fb66c159410 2023-10-06T05:26:32.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.681+0000 7fb66bfff700 1 --2- 172.21.15.31:0/2394613553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb66c159410 0x7fb66c0c6e80 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0c6e10 tx=0x7fb65c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2293a3040b18d1ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:32.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.681+0000 7fb66affd700 1 -- 172.21.15.31:0/2394613553 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c00b6b0 con 0x7fb66c159410 2023-10-06T05:26:32.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.681+0000 7fb66affd700 1 -- 172.21.15.31:0/2394613553 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65c00b810 con 0x7fb66c159410 2023-10-06T05:26:32.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.681+0000 7fb66affd700 1 -- 172.21.15.31:0/2394613553 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c00bbb0 con 0x7fb66c159410 2023-10-06T05:26:32.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.682+0000 7fb6723ff700 1 -- 172.21.15.31:0/2394613553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb66c159410 msgr2=0x7fb66c0c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.682+0000 7fb6723ff700 1 --2- 172.21.15.31:0/2394613553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb66c159410 0x7fb66c0c6e80 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0c6e10 tx=0x7fb65c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.682+0000 7fb6723ff700 1 -- 172.21.15.31:0/2394613553 shutdown_connections 2023-10-06T05:26:32.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.682+0000 7fb6723ff700 1 --2- 172.21.15.31:0/2394613553 >> v2:172.21.15.62:3300/0 conn(0x7fb66c06b4c0 0x7fb66c1597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.682+0000 7fb6723ff700 1 --2- 172.21.15.31:0/2394613553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb66c159410 0x7fb66c0c6e80 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.682+0000 7fb6723ff700 1 -- 172.21.15.31:0/2394613553 >> 172.21.15.31:0/2394613553 conn(0x7fb66c0c9350 msgr2=0x7fb66c0c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:32.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.682+0000 7fb6723ff700 1 -- 172.21.15.31:0/2394613553 shutdown_connections 2023-10-06T05:26:32.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.682+0000 7fb6723ff700 1 -- 172.21.15.31:0/2394613553 wait complete. 2023-10-06T05:26:32.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.683+0000 7fb6723ff700 1 Processor -- start 2023-10-06T05:26:32.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.684+0000 7fb6723ff700 1 -- start start 2023-10-06T05:26:32.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.684+0000 7fb6723ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb66c06b4c0 0x7fb66c0ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.684+0000 7fb6723ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb66c0c73c0 0x7fb66c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb6723ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb66c159410 0x7fb66c0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb6723ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c159880 con 0x7fb66c0c73c0 2023-10-06T05:26:32.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb6723ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c1520c0 con 0x7fb66c159410 2023-10-06T05:26:32.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb6723ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c159b20 con 0x7fb66c06b4c0 2023-10-06T05:26:32.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb66b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb66c0c73c0 0x7fb66c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb66b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb66c0c73c0 0x7fb66c15bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59432/0 (socket says 172.21.15.31:59432) 2023-10-06T05:26:32.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb66b7fe700 1 -- 172.21.15.31:0/1883821600 learned_addr learned my addr 172.21.15.31:0/1883821600 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb67099c700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb66c159410 0x7fb66c0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb66bfff700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb66c06b4c0 0x7fb66c0ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb66b7fe700 1 -- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb66c06b4c0 msgr2=0x7fb66c0ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb66b7fe700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb66c06b4c0 0x7fb66c0ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb66b7fe700 1 -- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb66c159410 msgr2=0x7fb66c0b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb66b7fe700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb66c159410 0x7fb66c0b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb66b7fe700 1 -- 172.21.15.31:0/1883821600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c009420 con 0x7fb66c0c73c0 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb66bfff700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb66c06b4c0 0x7fb66c0ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.685+0000 7fb67099c700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb66c159410 0x7fb66c0b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.686+0000 7fb66b7fe700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb66c0c73c0 0x7fb66c15bea0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c7c0 tx=0x7fb660007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.686+0000 7fb653fff700 1 -- 172.21.15.31:0/1883821600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660005b30 con 0x7fb66c0c73c0 2023-10-06T05:26:32.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.686+0000 7fb653fff700 1 -- 172.21.15.31:0/1883821600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb660005cb0 con 0x7fb66c0c73c0 2023-10-06T05:26:32.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.686+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66c159e00 con 0x7fb66c0c73c0 2023-10-06T05:26:32.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.686+0000 7fb653fff700 1 -- 172.21.15.31:0/1883821600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660004a80 con 0x7fb66c0c73c0 2023-10-06T05:26:32.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.686+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66c0b1b10 con 0x7fb66c0c73c0 2023-10-06T05:26:32.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.687+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb66c04c380 con 0x7fb66c0c73c0 2023-10-06T05:26:32.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.688+0000 7fb653fff700 1 -- 172.21.15.31:0/1883821600 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fb6600070f0 con 0x7fb66c0c73c0 2023-10-06T05:26:32.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.689+0000 7fb653fff700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb6580679e0 0x7fb658069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.689+0000 7fb653fff700 1 -- 172.21.15.31:0/1883821600 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fb6600825d0 con 0x7fb66c0c73c0 2023-10-06T05:26:32.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.689+0000 7fb653fff700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] conn(0x7fb65806de50 0x7fb658070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.689+0000 7fb653fff700 1 -- 172.21.15.31:0/1883821600 --> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb658070950 con 0x7fb65806de50 2023-10-06T05:26:32.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.689+0000 7fb653fff700 1 -- 172.21.15.31:0/1883821600 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6600872b0 con 0x7fb66c0c73c0 2023-10-06T05:26:32.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.689+0000 7fb66bfff700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb6580679e0 0x7fb658069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.689+0000 7fb67099c700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] conn(0x7fb65806de50 0x7fb658070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.690+0000 7fb67099c700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] conn(0x7fb65806de50 0x7fb658070280 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:32.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.690+0000 7fb66bfff700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb6580679e0 0x7fb658069e90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0093f0 tx=0x7fb65c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:32.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.690+0000 7fb653fff700 1 -- 172.21.15.31:0/1883821600 <== osd.0 v2:172.21.15.31:6800/114853 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb658070950 con 0x7fb65806de50 2023-10-06T05:26:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.705+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 --> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb66c063620 con 0x7fb65806de50 2023-10-06T05:26:32.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.705+0000 7fb653fff700 1 -- 172.21.15.31:0/1883821600 <== osd.0 v2:172.21.15.31:6800/114853 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb66c063620 con 0x7fb65806de50 2023-10-06T05:26:32.707 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:26:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:26:32.708 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:26:32.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 >> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] conn(0x7fb65806de50 msgr2=0x7fb658070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb6723ff700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.31:6800/114853,v1:172.21.15.31:6801/114853] conn(0x7fb65806de50 0x7fb658070280 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb6580679e0 msgr2=0x7fb658069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb6723ff700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb6580679e0 0x7fb658069e90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0093f0 tx=0x7fb65c017000 comp rx=0 tx=0).stop 2023-10-06T05:26:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb66c0c73c0 msgr2=0x7fb66c15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb6723ff700 1 --2- 172.21.15.31:0/1883821600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb66c0c73c0 0x7fb66c15bea0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c7c0 tx=0x7fb660007f70 comp rx=0 tx=0).stop 2023-10-06T05:26:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb67099c700 1 -- 172.21.15.31:0/1883821600 reap_dead start 2023-10-06T05:26:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 shutdown_connections 2023-10-06T05:26:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 >> 172.21.15.31:0/1883821600 conn(0x7fb66c0c9350 msgr2=0x7fb66c158730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 shutdown_connections 2023-10-06T05:26:32.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.706+0000 7fb6723ff700 1 -- 172.21.15.31:0/1883821600 wait complete. 2023-10-06T05:26:32.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T05:26:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.959+0000 7f08f33ff700 1 Processor -- start 2023-10-06T05:26:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.959+0000 7f08f33ff700 1 -- start start 2023-10-06T05:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.961+0000 7f08f33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec06a570 0x7f08ec162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.961+0000 7f08f33ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec0c71b0 con 0x7f08ec13a5b0 2023-10-06T05:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.961+0000 7f08f33ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec0c7490 con 0x7f08ec13a1e0 2023-10-06T05:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.961+0000 7f08f33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c7770 con 0x7f08ec06a570 2023-10-06T05:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.961+0000 7f08f099a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f08ec13a5b0 0x7f08ec06d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56920/0 (socket says 172.21.15.31:56920) 2023-10-06T05:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.961+0000 7f08f099a700 1 -- 172.21.15.31:0/1548962196 learned_addr learned my addr 172.21.15.31:0/1548962196 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.961+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157960209 0 0) 0x7f08ec0c71b0 con 0x7f08ec13a5b0 2023-10-06T05:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.961+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d8003570 con 0x7f08ec13a5b0 2023-10-06T05:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.961+0000 7f08f199c700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec06a570 0x7f08ec162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 144093162 0 0) 0x7f08d8003570 con 0x7f08ec13a5b0 2023-10-06T05:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec06a570 msgr2=0x7f08ec162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec06a570 0x7f08ec162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 >> v1:172.21.15.62:6789/0 conn(0x7f08ec13a1e0 legacy=0x7f08ec0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec0c7a50 con 0x7f08ec13a5b0 2023-10-06T05:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f08dc001470 con 0x7f08ec13a5b0 2023-10-06T05:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 >> v1:172.21.15.31:6789/0 conn(0x7f08ec13a5b0 legacy=0x7f08ec06d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec06a570 0x7f08d8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 0x7f08d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d8004d50 0x7f08d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c7770 con 0x7f08d8004060 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d800bc20 con 0x7f08d8004d50 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d800bde0 con 0x7f08ec06a570 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08f099a700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 0x7f08d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08f119b700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec06a570 0x7f08d8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.962+0000 7f08f199c700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d8004d50 0x7f08d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.963+0000 7f08f099a700 1 -- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec06a570 msgr2=0x7f08d8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.963+0000 7f08f099a700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec06a570 0x7f08d8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.963+0000 7f08f099a700 1 -- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d8004d50 msgr2=0x7f08d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.963+0000 7f08f099a700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d8004d50 0x7f08d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.963+0000 7f08f099a700 1 -- 172.21.15.31:0/1548962196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d8003570 con 0x7f08d8004060 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.963+0000 7f08f099a700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 0x7f08d8004810 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c2a0 tx=0x7f08dc001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303f44d8528564df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.963+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00cb70 con 0x7f08d8004060 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.963+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc00ccf0 con 0x7f08d8004060 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.963+0000 7f08e3fff700 1 -- 172.21.15.31:0/1548962196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc005b70 con 0x7f08d8004060 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.964+0000 7f08f33ff700 1 -- 172.21.15.31:0/1548962196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 msgr2=0x7f08d8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.964+0000 7f08f33ff700 1 --2- 172.21.15.31:0/1548962196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 0x7f08d8004810 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c2a0 tx=0x7f08dc001d80 comp rx=0 tx=0).stop 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.964+0000 7f08f199c700 1 -- 172.21.15.31:0/1548962196 reap_dead start 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.964+0000 7f08f33ff700 1 -- 172.21.15.31:0/1548962196 shutdown_connections 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.964+0000 7f08f33ff700 1 -- 172.21.15.31:0/1548962196 >> 172.21.15.31:0/1548962196 conn(0x7f08ec0cc0b0 msgr2=0x7f08ec0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.964+0000 7f08f33ff700 1 -- 172.21.15.31:0/1548962196 shutdown_connections 2023-10-06T05:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.964+0000 7f08f33ff700 1 -- 172.21.15.31:0/1548962196 wait complete. 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.965+0000 7f08f33ff700 1 Processor -- start 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.965+0000 7f08f33ff700 1 -- start start 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.966+0000 7f08f33ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 0x7f08ec13bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.966+0000 7f08f33ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08ec13c1b0 0x7f08ec0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.966+0000 7f08f33ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec0b0a70 0x7f08ec0c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.966+0000 7f08f33ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0b0ee0 con 0x7f08d8004060 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.966+0000 7f08f33ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec16a8f0 con 0x7f08ec13c1b0 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.966+0000 7f08f33ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec16a610 con 0x7f08ec0b0a70 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 0x7f08ec13bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f119b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 0x7f08ec13bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59456/0 (socket says 172.21.15.31:59456) 2023-10-06T05:26:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f119b700 1 -- 172.21.15.31:0/3576195000 learned_addr learned my addr 172.21.15.31:0/3576195000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f099a700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08ec13c1b0 0x7f08ec0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f199c700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec0b0a70 0x7f08ec0c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f119b700 1 -- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec0b0a70 msgr2=0x7f08ec0c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f119b700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec0b0a70 0x7f08ec0c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f119b700 1 -- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08ec13c1b0 msgr2=0x7f08ec0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f119b700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08ec13c1b0 0x7f08ec0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f119b700 1 -- 172.21.15.31:0/3576195000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec13b190 con 0x7f08d8004060 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.967+0000 7f08f119b700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 0x7f08ec13bc70 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f08e8002a20 tx=0x7f08e8007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.968+0000 7f08e17fa700 1 -- 172.21.15.31:0/3576195000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8002e60 con 0x7f08d8004060 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.968+0000 7f08e17fa700 1 -- 172.21.15.31:0/3576195000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e800f850 con 0x7f08d8004060 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.968+0000 7f08e17fa700 1 -- 172.21.15.31:0/3576195000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800fbf0 con 0x7f08d8004060 2023-10-06T05:26:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.968+0000 7f08f33ff700 1 -- 172.21.15.31:0/3576195000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec13b470 con 0x7f08d8004060 2023-10-06T05:26:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.968+0000 7f08f33ff700 1 -- 172.21.15.31:0/3576195000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec0ca460 con 0x7f08d8004060 2023-10-06T05:26:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.969+0000 7f08f33ff700 1 -- 172.21.15.31:0/3576195000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec04c380 con 0x7f08d8004060 2023-10-06T05:26:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.970+0000 7f08e17fa700 1 -- 172.21.15.31:0/3576195000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f08e801c400 con 0x7f08d8004060 2023-10-06T05:26:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.971+0000 7f08e17fa700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f08d00679e0 0x7f08d0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.971+0000 7f08e17fa700 1 -- 172.21.15.31:0/3576195000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f08e8083360 con 0x7f08d8004060 2023-10-06T05:26:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.971+0000 7f08f099a700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f08d00679e0 0x7f08d0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.972+0000 7f08f099a700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f08d00679e0 0x7f08d0069e90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f08dc007c00 tx=0x7f08dc015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:32.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:32.975+0000 7f08e17fa700 1 -- 172.21.15.31:0/3576195000 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08e8052050 con 0x7f08d8004060 2023-10-06T05:26:33.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.177+0000 7f08f33ff700 1 -- 172.21.15.31:0/3576195000 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f08ec0c7d30 con 0x7f08d00679e0 2023-10-06T05:26:33.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.178+0000 7f08e17fa700 1 -- 172.21.15.31:0/3576195000 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f08ec0c7d30 con 0x7f08d00679e0 2023-10-06T05:26:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.180+0000 7f08f33ff700 1 -- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f08d00679e0 msgr2=0x7f08d0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.180+0000 7f08f33ff700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f08d00679e0 0x7f08d0069e90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f08dc007c00 tx=0x7f08dc015000 comp rx=0 tx=0).stop 2023-10-06T05:26:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.181+0000 7f08f33ff700 1 -- 172.21.15.31:0/3576195000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 msgr2=0x7f08ec13bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.181+0000 7f08f33ff700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 0x7f08ec13bc70 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f08e8002a20 tx=0x7f08e8007d80 comp rx=0 tx=0).stop 2023-10-06T05:26:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.181+0000 7f08f33ff700 1 -- 172.21.15.31:0/3576195000 shutdown_connections 2023-10-06T05:26:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.181+0000 7f08f33ff700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f08d00679e0 0x7f08d0069e90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.181+0000 7f08f33ff700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08ec0b0a70 0x7f08ec0c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.181+0000 7f08f33ff700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d8004060 0x7f08ec13bc70 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.181+0000 7f08f33ff700 1 --2- 172.21.15.31:0/3576195000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08ec13c1b0 0x7f08ec0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.181+0000 7f08f33ff700 1 -- 172.21.15.31:0/3576195000 >> 172.21.15.31:0/3576195000 conn(0x7f08ec0cc0b0 msgr2=0x7f08ec156bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.181+0000 7f08f33ff700 1 -- 172.21.15.31:0/3576195000 shutdown_connections 2023-10-06T05:26:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.181+0000 7f08f33ff700 1 -- 172.21.15.31:0/3576195000 wait complete. 2023-10-06T05:26:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T05:26:33.200 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T05:26:33.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:26:33.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.399+0000 7fad141ff700 1 Processor -- start 2023-10-06T05:26:33.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.399+0000 7fad141ff700 1 -- start start 2023-10-06T05:26:33.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.400+0000 7fad141ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad0c06e300 0x7fad0c159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.400+0000 7fad141ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c159fe0 con 0x7fad0c06df30 2023-10-06T05:26:33.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.400+0000 7fad141ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0c72b0 con 0x7fad0c159ba0 2023-10-06T05:26:33.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.400+0000 7fad141ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c0aba60 con 0x7fad0c06e300 2023-10-06T05:26:33.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.400+0000 7fad1179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad0c06e300 0x7fad0c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.400+0000 7fad1179a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad0c06e300 0x7fad0c159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59458/0 (socket says 172.21.15.31:59458) 2023-10-06T05:26:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.400+0000 7fad1179a700 1 -- 172.21.15.31:0/544683303 learned_addr learned my addr 172.21.15.31:0/544683303 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.400+0000 7fad11f9b700 1 --1- 172.21.15.31:0/544683303 >> v1:172.21.15.31:6789/0 conn(0x7fad0c06df30 0x7fad0c06d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56926/0 (socket says 172.21.15.31:56926) 2023-10-06T05:26:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.401+0000 7fad1179a700 1 -- 172.21.15.31:0/544683303 >> v1:172.21.15.62:6789/0 conn(0x7fad0c159ba0 legacy=0x7fad0c152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.401+0000 7fad1179a700 1 -- 172.21.15.31:0/544683303 >> v1:172.21.15.31:6789/0 conn(0x7fad0c06df30 legacy=0x7fad0c06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.401+0000 7fad1179a700 1 -- 172.21.15.31:0/544683303 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0c7590 con 0x7fad0c06e300 2023-10-06T05:26:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.401+0000 7fad1179a700 1 --2- 172.21.15.31:0/544683303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c06e300 0x7fad0c159660 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7facfc00bc50 tx=0x7facfc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728cec4f39aabb3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.401+0000 7fad10f99700 1 -- 172.21.15.31:0/544683303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc00be60 con 0x7fad0c06e300 2023-10-06T05:26:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.401+0000 7fad10f99700 1 -- 172.21.15.31:0/544683303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc00de00 con 0x7fad0c06e300 2023-10-06T05:26:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.401+0000 7fad10f99700 1 -- 172.21.15.31:0/544683303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0196b0 con 0x7fad0c06e300 2023-10-06T05:26:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.402+0000 7fad141ff700 1 -- 172.21.15.31:0/544683303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c06e300 msgr2=0x7fad0c159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.402+0000 7fad141ff700 1 --2- 172.21.15.31:0/544683303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c06e300 0x7fad0c159660 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7facfc00bc50 tx=0x7facfc00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.402+0000 7fad141ff700 1 -- 172.21.15.31:0/544683303 shutdown_connections 2023-10-06T05:26:33.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.402+0000 7fad141ff700 1 --2- 172.21.15.31:0/544683303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c06e300 0x7fad0c159660 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.402+0000 7fad141ff700 1 -- 172.21.15.31:0/544683303 >> 172.21.15.31:0/544683303 conn(0x7fad0c068030 msgr2=0x7fad0c0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:33.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.402+0000 7fad141ff700 1 -- 172.21.15.31:0/544683303 shutdown_connections 2023-10-06T05:26:33.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.402+0000 7fad141ff700 1 -- 172.21.15.31:0/544683303 wait complete. 2023-10-06T05:26:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.404+0000 7fad141ff700 1 Processor -- start 2023-10-06T05:26:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.404+0000 7fad141ff700 1 -- start start 2023-10-06T05:26:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad141ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad0c06df30 0x7fad0c151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad141ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06e300 0x7fad0c0c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad141ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c159ba0 0x7fad0c0c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad141ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0acd70 con 0x7fad0c159ba0 2023-10-06T05:26:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad141ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0c2360 con 0x7fad0c06df30 2023-10-06T05:26:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad141ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0c27e0 con 0x7fad0c06e300 2023-10-06T05:26:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad1279c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c159ba0 0x7fad0c0c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad1279c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c159ba0 0x7fad0c0c1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59464/0 (socket says 172.21.15.31:59464) 2023-10-06T05:26:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad1279c700 1 -- 172.21.15.31:0/1833917607 learned_addr learned my addr 172.21.15.31:0/1833917607 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad1179a700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06e300 0x7fad0c0c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad11f9b700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad0c06df30 0x7fad0c151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad1279c700 1 -- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06e300 msgr2=0x7fad0c0c18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad1279c700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06e300 0x7fad0c0c18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad1279c700 1 -- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad0c06df30 msgr2=0x7fad0c151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad1279c700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad0c06df30 0x7fad0c151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad1279c700 1 -- 172.21.15.31:0/1833917607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc00b790 con 0x7fad0c159ba0 2023-10-06T05:26:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.406+0000 7fad1179a700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad0c06e300 0x7fad0c0c18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:33.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.407+0000 7fad1279c700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c159ba0 0x7fad0c0c1e20 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fad0400c870 tx=0x7fad04001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.407+0000 7fad01ffb700 1 -- 172.21.15.31:0/1833917607 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04012630 con 0x7fad0c159ba0 2023-10-06T05:26:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.407+0000 7fad01ffb700 1 -- 172.21.15.31:0/1833917607 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad040127b0 con 0x7fad0c159ba0 2023-10-06T05:26:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.407+0000 7fad01ffb700 1 -- 172.21.15.31:0/1833917607 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04012b70 con 0x7fad0c159ba0 2023-10-06T05:26:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.407+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c0ac1d0 con 0x7fad0c159ba0 2023-10-06T05:26:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.407+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c0ac4f0 con 0x7fad0c159ba0 2023-10-06T05:26:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.408+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad0c04c380 con 0x7fad0c159ba0 2023-10-06T05:26:33.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.409+0000 7fad01ffb700 1 -- 172.21.15.31:0/1833917607 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fad04003030 con 0x7fad0c159ba0 2023-10-06T05:26:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.410+0000 7fad01ffb700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7facf00679e0 0x7facf0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.410+0000 7fad01ffb700 1 -- 172.21.15.31:0/1833917607 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fad040830d0 con 0x7fad0c159ba0 2023-10-06T05:26:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.410+0000 7fad01ffb700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] conn(0x7facf006de50 0x7facf0070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.410+0000 7fad01ffb700 1 -- 172.21.15.31:0/1833917607 --> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facf0070950 con 0x7facf006de50 2023-10-06T05:26:33.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.410+0000 7fad01ffb700 1 -- 172.21.15.31:0/1833917607 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad04087db0 con 0x7fad0c159ba0 2023-10-06T05:26:33.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.410+0000 7fad1179a700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] conn(0x7facf006de50 0x7facf0070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.410+0000 7fad11f9b700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7facf00679e0 0x7facf0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.411+0000 7fad1179a700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] conn(0x7facf006de50 0x7facf0070280 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:33.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.411+0000 7fad11f9b700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7facf00679e0 0x7facf0069e90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fad08002830 tx=0x7fad08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:33.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.412+0000 7fad01ffb700 1 -- 172.21.15.31:0/1833917607 <== osd.1 v2:172.21.15.31:6816/114859 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7facf0070950 con 0x7facf006de50 2023-10-06T05:26:33.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.426+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 --> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fad0c0c7870 con 0x7facf006de50 2023-10-06T05:26:33.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad01ffb700 1 -- 172.21.15.31:0/1833917607 <== osd.1 v2:172.21.15.31:6816/114859 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fad0c0c7870 con 0x7facf006de50 2023-10-06T05:26:33.429 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:26:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:26:33.430 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:26:33.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 >> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] conn(0x7facf006de50 msgr2=0x7facf0070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad141ff700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.31:6816/114859,v1:172.21.15.31:6817/114859] conn(0x7facf006de50 0x7facf0070280 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7facf00679e0 msgr2=0x7facf0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad141ff700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7facf00679e0 0x7facf0069e90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fad08002830 tx=0x7fad08009040 comp rx=0 tx=0).stop 2023-10-06T05:26:33.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c159ba0 msgr2=0x7fad0c0c1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad141ff700 1 --2- 172.21.15.31:0/1833917607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad0c159ba0 0x7fad0c0c1e20 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fad0400c870 tx=0x7fad04001670 comp rx=0 tx=0).stop 2023-10-06T05:26:33.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad1279c700 1 -- 172.21.15.31:0/1833917607 reap_dead start 2023-10-06T05:26:33.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 shutdown_connections 2023-10-06T05:26:33.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 >> 172.21.15.31:0/1833917607 conn(0x7fad0c068030 msgr2=0x7fad0c0c93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:33.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.427+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 shutdown_connections 2023-10-06T05:26:33.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.428+0000 7fad141ff700 1 -- 172.21.15.31:0/1833917607 wait complete. 2023-10-06T05:26:33.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T05:26:33.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.642+0000 7fc5219ff700 1 Processor -- start 2023-10-06T05:26:33.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.642+0000 7fc5219ff700 1 -- start start 2023-10-06T05:26:33.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.643+0000 7fc5219ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc51c0cd510 0x7fc51c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.643+0000 7fc5219ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51c0c3f40 con 0x7fc51c15d2d0 2023-10-06T05:26:33.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.643+0000 7fc5219ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51c0cb000 con 0x7fc51c159500 2023-10-06T05:26:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.643+0000 7fc5219ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51c0b2860 con 0x7fc51c0cd510 2023-10-06T05:26:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.644+0000 7fc51b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc51c15d2d0 0x7fc51c0aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56934/0 (socket says 172.21.15.31:56934) 2023-10-06T05:26:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.644+0000 7fc51b7fe700 1 -- 172.21.15.31:0/1231508819 learned_addr learned my addr 172.21.15.31:0/1231508819 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:33.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.644+0000 7fc51a7fc700 1 --2- 172.21.15.31:0/1231508819 >> v2:172.21.15.62:3300/0 conn(0x7fc51c0cd510 0x7fc51c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.644+0000 7fc519ffb700 1 -- 172.21.15.31:0/1231508819 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433093553 0 0) 0x7fc51c0c3f40 con 0x7fc51c15d2d0 2023-10-06T05:26:33.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.644+0000 7fc519ffb700 1 -- 172.21.15.31:0/1231508819 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc504003570 con 0x7fc51c15d2d0 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.644+0000 7fc51a7fc700 1 -- 172.21.15.31:0/1231508819 >> v1:172.21.15.62:6789/0 conn(0x7fc51c159500 legacy=0x7fc51c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.644+0000 7fc51a7fc700 1 -- 172.21.15.31:0/1231508819 >> v1:172.21.15.31:6789/0 conn(0x7fc51c15d2d0 legacy=0x7fc51c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.644+0000 7fc51a7fc700 1 -- 172.21.15.31:0/1231508819 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c15d7d0 con 0x7fc51c0cd510 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.645+0000 7fc51a7fc700 1 --2- 172.21.15.31:0/1231508819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c0cd510 0x7fc51c0b0ab0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc5100098d0 tx=0x7fc510009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f14485b74e3f5729 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.645+0000 7fc519ffb700 1 -- 172.21.15.31:0/1231508819 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510005c10 con 0x7fc51c0cd510 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.645+0000 7fc519ffb700 1 -- 172.21.15.31:0/1231508819 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc510005d70 con 0x7fc51c0cd510 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.645+0000 7fc519ffb700 1 -- 172.21.15.31:0/1231508819 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510006280 con 0x7fc51c0cd510 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.645+0000 7fc5219ff700 1 -- 172.21.15.31:0/1231508819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c0cd510 msgr2=0x7fc51c0b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.645+0000 7fc5219ff700 1 --2- 172.21.15.31:0/1231508819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c0cd510 0x7fc51c0b0ab0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc5100098d0 tx=0x7fc510009120 comp rx=0 tx=0).stop 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.646+0000 7fc5219ff700 1 -- 172.21.15.31:0/1231508819 shutdown_connections 2023-10-06T05:26:33.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.646+0000 7fc5219ff700 1 --2- 172.21.15.31:0/1231508819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c0cd510 0x7fc51c0b0ab0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.646+0000 7fc5219ff700 1 -- 172.21.15.31:0/1231508819 >> 172.21.15.31:0/1231508819 conn(0x7fc51c06ec90 msgr2=0x7fc51c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:33.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.646+0000 7fc5219ff700 1 -- 172.21.15.31:0/1231508819 shutdown_connections 2023-10-06T05:26:33.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.646+0000 7fc5219ff700 1 -- 172.21.15.31:0/1231508819 wait complete. 2023-10-06T05:26:33.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.647+0000 7fc5219ff700 1 Processor -- start 2023-10-06T05:26:33.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.647+0000 7fc5219ff700 1 -- start start 2023-10-06T05:26:33.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.648+0000 7fc5219ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c0cd510 0x7fc51c155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.648+0000 7fc5219ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc51c159500 0x7fc51c15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.648+0000 7fc5219ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c15d2d0 0x7fc51c0c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.648+0000 7fc5219ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c16a5a0 con 0x7fc51c15d2d0 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.648+0000 7fc5219ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c169cb0 con 0x7fc51c0cd510 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.648+0000 7fc5219ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c15b8e0 con 0x7fc51c159500 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.648+0000 7fc51b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c15d2d0 0x7fc51c0c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.648+0000 7fc51b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c15d2d0 0x7fc51c0c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59472/0 (socket says 172.21.15.31:59472) 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.648+0000 7fc51b7fe700 1 -- 172.21.15.31:0/3125252750 learned_addr learned my addr 172.21.15.31:0/3125252750 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc51a7fc700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc51c159500 0x7fc51c15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc51affd700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c0cd510 0x7fc51c155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc51b7fe700 1 -- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc51c159500 msgr2=0x7fc51c15af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc51b7fe700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc51c159500 0x7fc51c15af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc51b7fe700 1 -- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c0cd510 msgr2=0x7fc51c155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc51b7fe700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c0cd510 0x7fc51c155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc51b7fe700 1 -- 172.21.15.31:0/3125252750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5100093c0 con 0x7fc51c15d2d0 2023-10-06T05:26:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc51b7fe700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c15d2d0 0x7fc51c0c5d10 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fc514001460 tx=0x7fc514001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc502ffd700 1 -- 172.21.15.31:0/3125252750 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51400a950 con 0x7fc51c15d2d0 2023-10-06T05:26:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc502ffd700 1 -- 172.21.15.31:0/3125252750 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51400aab0 con 0x7fc51c15d2d0 2023-10-06T05:26:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc5219ff700 1 -- 172.21.15.31:0/3125252750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c169f50 con 0x7fc51c15d2d0 2023-10-06T05:26:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.649+0000 7fc502ffd700 1 -- 172.21.15.31:0/3125252750 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51400ae30 con 0x7fc51c15d2d0 2023-10-06T05:26:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.650+0000 7fc5219ff700 1 -- 172.21.15.31:0/3125252750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c16aa30 con 0x7fc51c15d2d0 2023-10-06T05:26:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.651+0000 7fc5219ff700 1 -- 172.21.15.31:0/3125252750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c04c380 con 0x7fc51c15d2d0 2023-10-06T05:26:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.652+0000 7fc502ffd700 1 -- 172.21.15.31:0/3125252750 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fc514007300 con 0x7fc51c15d2d0 2023-10-06T05:26:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.653+0000 7fc502ffd700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc5080679e0 0x7fc508069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:33.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.653+0000 7fc502ffd700 1 -- 172.21.15.31:0/3125252750 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fc5140832d0 con 0x7fc51c15d2d0 2023-10-06T05:26:33.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.653+0000 7fc51affd700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc5080679e0 0x7fc508069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:33.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.654+0000 7fc51affd700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc5080679e0 0x7fc508069e90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc50c003860 tx=0x7fc50c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:33.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.657+0000 7fc502ffd700 1 -- 172.21.15.31:0/3125252750 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc514051ff0 con 0x7fc51c15d2d0 2023-10-06T05:26:33.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.862+0000 7fc5219ff700 1 -- 172.21.15.31:0/3125252750 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc51c0cade0 con 0x7fc5080679e0 2023-10-06T05:26:33.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.863+0000 7fc502ffd700 1 -- 172.21.15.31:0/3125252750 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc51c0cade0 con 0x7fc5080679e0 2023-10-06T05:26:33.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 -- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc5080679e0 msgr2=0x7fc508069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc5080679e0 0x7fc508069e90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc50c003860 tx=0x7fc50c009040 comp rx=0 tx=0).stop 2023-10-06T05:26:33.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 -- 172.21.15.31:0/3125252750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c15d2d0 msgr2=0x7fc51c0c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:33.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c15d2d0 0x7fc51c0c5d10 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fc514001460 tx=0x7fc514001a40 comp rx=0 tx=0).stop 2023-10-06T05:26:33.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 -- 172.21.15.31:0/3125252750 shutdown_connections 2023-10-06T05:26:33.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc5080679e0 0x7fc508069e90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc51c15d2d0 0x7fc51c0c5d10 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc51c0cd510 0x7fc51c155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 --2- 172.21.15.31:0/3125252750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc51c159500 0x7fc51c15af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:33.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 -- 172.21.15.31:0/3125252750 >> 172.21.15.31:0/3125252750 conn(0x7fc51c06ec90 msgr2=0x7fc51c0cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:33.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 -- 172.21.15.31:0/3125252750 shutdown_connections 2023-10-06T05:26:33.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:33.865+0000 7fc5219ff700 1 -- 172.21.15.31:0/3125252750 wait complete. 2023-10-06T05:26:33.868 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T05:26:33.885 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T05:26:33.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:26:34.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.082+0000 7fb94fdff700 1 Processor -- start 2023-10-06T05:26:34.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.082+0000 7fb94fdff700 1 -- start start 2023-10-06T05:26:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.084+0000 7fb94fdff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb94815d920 0x7fb9481645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.084+0000 7fb94fdff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94815cfc0 con 0x7fb948153d20 2023-10-06T05:26:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.084+0000 7fb94fdff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94815bb20 con 0x7fb948153950 2023-10-06T05:26:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.084+0000 7fb94fdff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb948154170 con 0x7fb94815d920 2023-10-06T05:26:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.084+0000 7fb94e39c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb94815d920 0x7fb9481645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.084+0000 7fb94d39a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb948153d20 0x7fb94815d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56938/0 (socket says 172.21.15.31:56938) 2023-10-06T05:26:34.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.084+0000 7fb94e39c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb94815d920 0x7fb9481645b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59482/0 (socket says 172.21.15.31:59482) 2023-10-06T05:26:34.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.084+0000 7fb94d39a700 1 -- 172.21.15.31:0/3462160202 learned_addr learned my addr 172.21.15.31:0/3462160202 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.085+0000 7fb94e39c700 1 -- 172.21.15.31:0/3462160202 >> v1:172.21.15.62:6789/0 conn(0x7fb948153950 legacy=0x7fb9480cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.085+0000 7fb94cb99700 1 -- 172.21.15.31:0/3462160202 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160639169 0 0) 0x7fb94815cfc0 con 0x7fb948153d20 2023-10-06T05:26:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.085+0000 7fb94e39c700 1 -- 172.21.15.31:0/3462160202 >> v1:172.21.15.31:6789/0 conn(0x7fb948153d20 legacy=0x7fb94815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.085+0000 7fb94e39c700 1 -- 172.21.15.31:0/3462160202 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94815af10 con 0x7fb94815d920 2023-10-06T05:26:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.085+0000 7fb94e39c700 1 --2- 172.21.15.31:0/3462160202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb94815d920 0x7fb9481645b0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fb940009e60 tx=0x7fb940009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36e5d95b7ce4923 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.085+0000 7fb94cb99700 1 -- 172.21.15.31:0/3462160202 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94000e070 con 0x7fb94815d920 2023-10-06T05:26:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.085+0000 7fb94cb99700 1 -- 172.21.15.31:0/3462160202 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94000bc30 con 0x7fb94815d920 2023-10-06T05:26:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.085+0000 7fb94cb99700 1 -- 172.21.15.31:0/3462160202 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940012430 con 0x7fb94815d920 2023-10-06T05:26:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.086+0000 7fb94fdff700 1 -- 172.21.15.31:0/3462160202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb94815d920 msgr2=0x7fb9481645b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.086+0000 7fb94fdff700 1 --2- 172.21.15.31:0/3462160202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb94815d920 0x7fb9481645b0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fb940009e60 tx=0x7fb940009780 comp rx=0 tx=0).stop 2023-10-06T05:26:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.086+0000 7fb94fdff700 1 -- 172.21.15.31:0/3462160202 shutdown_connections 2023-10-06T05:26:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.086+0000 7fb94fdff700 1 --2- 172.21.15.31:0/3462160202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb94815d920 0x7fb9481645b0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.086+0000 7fb94fdff700 1 -- 172.21.15.31:0/3462160202 >> 172.21.15.31:0/3462160202 conn(0x7fb9480681b0 msgr2=0x7fb9480685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.086+0000 7fb94fdff700 1 -- 172.21.15.31:0/3462160202 shutdown_connections 2023-10-06T05:26:34.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.086+0000 7fb94fdff700 1 -- 172.21.15.31:0/3462160202 wait complete. 2023-10-06T05:26:34.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.087+0000 7fb94fdff700 1 Processor -- start 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.087+0000 7fb94fdff700 1 -- start start 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.088+0000 7fb94fdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb948153950 0x7fb9481693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.088+0000 7fb94fdff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb948153d20 0x7fb9480b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.088+0000 7fb94fdff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb94815d920 0x7fb9480b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.088+0000 7fb94fdff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94814ee20 con 0x7fb948153d20 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.088+0000 7fb94fdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94814c470 con 0x7fb948153950 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.088+0000 7fb94fdff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94814be60 con 0x7fb94815d920 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94d39a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb948153d20 0x7fb9480b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94d39a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb948153d20 0x7fb9480b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59484/0 (socket says 172.21.15.31:59484) 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94d39a700 1 -- 172.21.15.31:0/3460396699 learned_addr learned my addr 172.21.15.31:0/3460396699 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:34.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94e39c700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb94815d920 0x7fb9480b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94db9b700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb948153950 0x7fb9481693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94d39a700 1 -- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb94815d920 msgr2=0x7fb9480b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94d39a700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb94815d920 0x7fb9480b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94d39a700 1 -- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb948153950 msgr2=0x7fb9481693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94d39a700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb948153950 0x7fb9481693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94d39a700 1 -- 172.21.15.31:0/3460396699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb940009a20 con 0x7fb948153d20 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb94d39a700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb948153d20 0x7fb9480b3240 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c5b0 tx=0x7fb938001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb93dffb700 1 -- 172.21.15.31:0/3460396699 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938005b30 con 0x7fb948153d20 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb93dffb700 1 -- 172.21.15.31:0/3460396699 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb938005c90 con 0x7fb948153d20 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.089+0000 7fb93dffb700 1 -- 172.21.15.31:0/3460396699 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938007300 con 0x7fb948153d20 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.090+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb948150940 con 0x7fb948153d20 2023-10-06T05:26:34.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.090+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb948150c20 con 0x7fb948153d20 2023-10-06T05:26:34.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.090+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb94804c380 con 0x7fb948153d20 2023-10-06T05:26:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.091+0000 7fb93dffb700 1 -- 172.21.15.31:0/3460396699 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fb93801b3b0 con 0x7fb948153d20 2023-10-06T05:26:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.092+0000 7fb93dffb700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb92c0679e0 0x7fb92c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.092+0000 7fb93dffb700 1 -- 172.21.15.31:0/3460396699 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fb938082690 con 0x7fb948153d20 2023-10-06T05:26:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.093+0000 7fb93dffb700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] conn(0x7fb92c06de50 0x7fb92c070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.093+0000 7fb93dffb700 1 -- 172.21.15.31:0/3460396699 --> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb92c070950 con 0x7fb92c06de50 2023-10-06T05:26:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.093+0000 7fb93dffb700 1 -- 172.21.15.31:0/3460396699 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb938087370 con 0x7fb948153d20 2023-10-06T05:26:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.093+0000 7fb94db9b700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb92c0679e0 0x7fb92c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.093+0000 7fb94e39c700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] conn(0x7fb92c06de50 0x7fb92c070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.093+0000 7fb94e39c700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] conn(0x7fb92c06de50 0x7fb92c070280 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.094+0000 7fb94db9b700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb92c0679e0 0x7fb92c069e90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb9440037f0 tx=0x7fb944009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.094+0000 7fb93dffb700 1 -- 172.21.15.31:0/3460396699 <== osd.2 v2:172.21.15.31:6808/114858 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb92c070950 con 0x7fb92c06de50 2023-10-06T05:26:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.109+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 --> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb94815b1f0 con 0x7fb92c06de50 2023-10-06T05:26:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.109+0000 7fb93dffb700 1 -- 172.21.15.31:0/3460396699 <== osd.2 v2:172.21.15.31:6808/114858 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb94815b1f0 con 0x7fb92c06de50 2023-10-06T05:26:34.111 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:26:34.111 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:26:34.111 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:26:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.109+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 >> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] conn(0x7fb92c06de50 msgr2=0x7fb92c070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.109+0000 7fb94fdff700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.31:6808/114858,v1:172.21.15.31:6809/114858] conn(0x7fb92c06de50 0x7fb92c070280 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.110+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb92c0679e0 msgr2=0x7fb92c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.110+0000 7fb94fdff700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb92c0679e0 0x7fb92c069e90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb9440037f0 tx=0x7fb944009040 comp rx=0 tx=0).stop 2023-10-06T05:26:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.110+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb948153d20 msgr2=0x7fb9480b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.110+0000 7fb94fdff700 1 --2- 172.21.15.31:0/3460396699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb948153d20 0x7fb9480b3240 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c5b0 tx=0x7fb938001070 comp rx=0 tx=0).stop 2023-10-06T05:26:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.110+0000 7fb94e39c700 1 -- 172.21.15.31:0/3460396699 reap_dead start 2023-10-06T05:26:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.110+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 shutdown_connections 2023-10-06T05:26:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.110+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 >> 172.21.15.31:0/3460396699 conn(0x7fb9480681b0 msgr2=0x7fb9480b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.110+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 shutdown_connections 2023-10-06T05:26:34.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.110+0000 7fb94fdff700 1 -- 172.21.15.31:0/3460396699 wait complete. 2023-10-06T05:26:34.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T05:26:34.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.324+0000 7fdbf27ff700 1 Processor -- start 2023-10-06T05:26:34.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.324+0000 7fdbf27ff700 1 -- start start 2023-10-06T05:26:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.325+0000 7fdbf27ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdbec068d80 0x7fdbec159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.325+0000 7fdbf27ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec0c95b0 con 0x7fdbec0cc0d0 2023-10-06T05:26:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.325+0000 7fdbf27ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec0c9890 con 0x7fdbec1595a0 2023-10-06T05:26:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.325+0000 7fdbf27ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbec0c9b70 con 0x7fdbec068d80 2023-10-06T05:26:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.325+0000 7fdbf0d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdbec0cc0d0 0x7fdbec0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56942/0 (socket says 172.21.15.31:56942) 2023-10-06T05:26:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.325+0000 7fdbf0d9c700 1 -- 172.21.15.31:0/1641175227 learned_addr learned my addr 172.21.15.31:0/1641175227 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.325+0000 7fdbeb7fe700 1 --2- 172.21.15.31:0/1641175227 >> v2:172.21.15.62:3300/0 conn(0x7fdbec068d80 0x7fdbec159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.326+0000 7fdbeaffd700 1 -- 172.21.15.31:0/1641175227 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505603169 0 0) 0x7fdbec0c95b0 con 0x7fdbec0cc0d0 2023-10-06T05:26:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.326+0000 7fdbeaffd700 1 -- 172.21.15.31:0/1641175227 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd4003570 con 0x7fdbec0cc0d0 2023-10-06T05:26:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.326+0000 7fdbeb7fe700 1 -- 172.21.15.31:0/1641175227 >> v1:172.21.15.62:6789/0 conn(0x7fdbec1595a0 legacy=0x7fdbec068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.326+0000 7fdbeb7fe700 1 -- 172.21.15.31:0/1641175227 >> v1:172.21.15.31:6789/0 conn(0x7fdbec0cc0d0 legacy=0x7fdbec0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.326+0000 7fdbeb7fe700 1 -- 172.21.15.31:0/1641175227 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec0ca130 con 0x7fdbec068d80 2023-10-06T05:26:34.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.326+0000 7fdbeb7fe700 1 --2- 172.21.15.31:0/1641175227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbec068d80 0x7fdbec159970 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009910 tx=0x7fdbe00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82c85d7d55c44ed3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.327+0000 7fdbeaffd700 1 -- 172.21.15.31:0/1641175227 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000b640 con 0x7fdbec068d80 2023-10-06T05:26:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.327+0000 7fdbeaffd700 1 -- 172.21.15.31:0/1641175227 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe000b7a0 con 0x7fdbec068d80 2023-10-06T05:26:34.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.327+0000 7fdbeaffd700 1 -- 172.21.15.31:0/1641175227 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000bb20 con 0x7fdbec068d80 2023-10-06T05:26:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.327+0000 7fdbf27ff700 1 -- 172.21.15.31:0/1641175227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbec068d80 msgr2=0x7fdbec159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.327+0000 7fdbf27ff700 1 --2- 172.21.15.31:0/1641175227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbec068d80 0x7fdbec159970 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009910 tx=0x7fdbe00091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.327+0000 7fdbf27ff700 1 -- 172.21.15.31:0/1641175227 shutdown_connections 2023-10-06T05:26:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.327+0000 7fdbf27ff700 1 --2- 172.21.15.31:0/1641175227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbec068d80 0x7fdbec159970 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.327+0000 7fdbf27ff700 1 -- 172.21.15.31:0/1641175227 >> 172.21.15.31:0/1641175227 conn(0x7fdbec06b620 msgr2=0x7fdbec06ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.327+0000 7fdbf27ff700 1 -- 172.21.15.31:0/1641175227 shutdown_connections 2023-10-06T05:26:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.327+0000 7fdbf27ff700 1 -- 172.21.15.31:0/1641175227 wait complete. 2023-10-06T05:26:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.329+0000 7fdbf27ff700 1 Processor -- start 2023-10-06T05:26:34.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.329+0000 7fdbf27ff700 1 -- start start 2023-10-06T05:26:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf27ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbec068d80 0x7fdbec14ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf27ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbec0cc0d0 0x7fdbec14e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf27ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbec1595a0 0x7fdbec152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf27ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec14f540 con 0x7fdbec1595a0 2023-10-06T05:26:34.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf27ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0b1d40 con 0x7fdbec0cc0d0 2023-10-06T05:26:34.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf27ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0b1130 con 0x7fdbec068d80 2023-10-06T05:26:34.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf0d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbec1595a0 0x7fdbec152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf0d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbec1595a0 0x7fdbec152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59490/0 (socket says 172.21.15.31:59490) 2023-10-06T05:26:34.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf0d9c700 1 -- 172.21.15.31:0/3955153711 learned_addr learned my addr 172.21.15.31:0/3955153711 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:34.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbeb7fe700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbec0cc0d0 0x7fdbec14e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbebfff700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbec068d80 0x7fdbec14ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf0d9c700 1 -- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbec068d80 msgr2=0x7fdbec14ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf0d9c700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbec068d80 0x7fdbec14ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf0d9c700 1 -- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbec0cc0d0 msgr2=0x7fdbec14e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf0d9c700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbec0cc0d0 0x7fdbec14e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.330+0000 7fdbf0d9c700 1 -- 172.21.15.31:0/3955153711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe0009420 con 0x7fdbec1595a0 2023-10-06T05:26:34.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.331+0000 7fdbeb7fe700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbec0cc0d0 0x7fdbec14e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:34.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.331+0000 7fdbf0d9c700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbec1595a0 0x7fdbec152490 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4001460 tx=0x7fdbe4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.331+0000 7fdbd3fff700 1 -- 172.21.15.31:0/3955153711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe400ce80 con 0x7fdbec1595a0 2023-10-06T05:26:34.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.331+0000 7fdbd3fff700 1 -- 172.21.15.31:0/3955153711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe400ab70 con 0x7fdbec1595a0 2023-10-06T05:26:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.331+0000 7fdbd3fff700 1 -- 172.21.15.31:0/3955153711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40070f0 con 0x7fdbec1595a0 2023-10-06T05:26:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.331+0000 7fdbf27ff700 1 -- 172.21.15.31:0/3955153711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec0b1410 con 0x7fdbec1595a0 2023-10-06T05:26:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.331+0000 7fdbf27ff700 1 -- 172.21.15.31:0/3955153711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec0b19d0 con 0x7fdbec1595a0 2023-10-06T05:26:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.332+0000 7fdbf27ff700 1 -- 172.21.15.31:0/3955153711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbec0b1ed0 con 0x7fdbec1595a0 2023-10-06T05:26:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.333+0000 7fdbd3fff700 1 -- 172.21.15.31:0/3955153711 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fdbe4007520 con 0x7fdbec1595a0 2023-10-06T05:26:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.334+0000 7fdbd3fff700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdbd80679e0 0x7fdbd8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.334+0000 7fdbd3fff700 1 -- 172.21.15.31:0/3955153711 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fdbe40833b0 con 0x7fdbec1595a0 2023-10-06T05:26:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.335+0000 7fdbebfff700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdbd80679e0 0x7fdbd8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.335+0000 7fdbebfff700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdbd80679e0 0x7fdbd8069e90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc000c00 tx=0x7fdbdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.338+0000 7fdbd3fff700 1 -- 172.21.15.31:0/3955153711 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbe408c080 con 0x7fdbec1595a0 2023-10-06T05:26:34.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.540+0000 7fdbf27ff700 1 -- 172.21.15.31:0/3955153711 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbec14ea10 con 0x7fdbd80679e0 2023-10-06T05:26:34.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.541+0000 7fdbd3fff700 1 -- 172.21.15.31:0/3955153711 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdbec14ea10 con 0x7fdbd80679e0 2023-10-06T05:26:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 -- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdbd80679e0 msgr2=0x7fdbd8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdbd80679e0 0x7fdbd8069e90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc000c00 tx=0x7fdbdc009040 comp rx=0 tx=0).stop 2023-10-06T05:26:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 -- 172.21.15.31:0/3955153711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbec1595a0 msgr2=0x7fdbec152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbec1595a0 0x7fdbec152490 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4001460 tx=0x7fdbe4001ae0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 -- 172.21.15.31:0/3955153711 shutdown_connections 2023-10-06T05:26:34.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fdbd80679e0 0x7fdbd8069e90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbec1595a0 0x7fdbec152490 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbec068d80 0x7fdbec14ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 --2- 172.21.15.31:0/3955153711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbec0cc0d0 0x7fdbec14e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 -- 172.21.15.31:0/3955153711 >> 172.21.15.31:0/3955153711 conn(0x7fdbec06b620 msgr2=0x7fdbec157450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 -- 172.21.15.31:0/3955153711 shutdown_connections 2023-10-06T05:26:34.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.543+0000 7fdbf27ff700 1 -- 172.21.15.31:0/3955153711 wait complete. 2023-10-06T05:26:34.548 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T05:26:34.562 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T05:26:34.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:26:34.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.760+0000 7ff01b3ff700 1 Processor -- start 2023-10-06T05:26:34.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.760+0000 7ff01b3ff700 1 -- start start 2023-10-06T05:26:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01b3ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff014142c20 0x7ff0140cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01b3ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01415d210 con 0x7ff0140cb3e0 2023-10-06T05:26:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01b3ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0140cc810 con 0x7ff01415a990 2023-10-06T05:26:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01b3ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01415d4b0 con 0x7ff014142c20 2023-10-06T05:26:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01919b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff014142c20 0x7ff0140cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01919b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff014142c20 0x7ff0140cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59506/0 (socket says 172.21.15.31:59506) 2023-10-06T05:26:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01919b700 1 -- 172.21.15.31:0/1442619650 learned_addr learned my addr 172.21.15.31:0/1442619650 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01999c700 1 --1- 172.21.15.31:0/1442619650 >> v1:172.21.15.31:6789/0 conn(0x7ff0140cb3e0 0x7ff014068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56944/0 (socket says 172.21.15.31:56944) 2023-10-06T05:26:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01919b700 1 -- 172.21.15.31:0/1442619650 >> v1:172.21.15.62:6789/0 conn(0x7ff01415a990 legacy=0x7ff0140cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01919b700 1 -- 172.21.15.31:0/1442619650 >> v1:172.21.15.31:6789/0 conn(0x7ff0140cb3e0 legacy=0x7ff014068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff01919b700 1 -- 172.21.15.31:0/1442619650 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01415d790 con 0x7ff014142c20 2023-10-06T05:26:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.762+0000 7ff00bfff700 1 -- 172.21.15.31:0/1442619650 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850523357 0 0) 0x7ff01415d210 con 0x7ff0140cb3e0 2023-10-06T05:26:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.763+0000 7ff01919b700 1 --2- 172.21.15.31:0/1442619650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff014142c20 0x7ff0140cbea0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff0100099b0 tx=0x7ff010009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db21ed34b4342196 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.763+0000 7ff00bfff700 1 -- 172.21.15.31:0/1442619650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01000b820 con 0x7ff014142c20 2023-10-06T05:26:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.763+0000 7ff00bfff700 1 -- 172.21.15.31:0/1442619650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01000b980 con 0x7ff014142c20 2023-10-06T05:26:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.763+0000 7ff00bfff700 1 -- 172.21.15.31:0/1442619650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01000bd20 con 0x7ff014142c20 2023-10-06T05:26:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.763+0000 7ff01b3ff700 1 -- 172.21.15.31:0/1442619650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff014142c20 msgr2=0x7ff0140cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.763+0000 7ff01b3ff700 1 --2- 172.21.15.31:0/1442619650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff014142c20 0x7ff0140cbea0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff0100099b0 tx=0x7ff010009250 comp rx=0 tx=0).stop 2023-10-06T05:26:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.764+0000 7ff01b3ff700 1 -- 172.21.15.31:0/1442619650 shutdown_connections 2023-10-06T05:26:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.764+0000 7ff01b3ff700 1 --2- 172.21.15.31:0/1442619650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff014142c20 0x7ff0140cbea0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.764+0000 7ff01b3ff700 1 -- 172.21.15.31:0/1442619650 >> 172.21.15.31:0/1442619650 conn(0x7ff0140ce2d0 msgr2=0x7ff0140ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.764+0000 7ff01b3ff700 1 -- 172.21.15.31:0/1442619650 shutdown_connections 2023-10-06T05:26:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.764+0000 7ff01b3ff700 1 -- 172.21.15.31:0/1442619650 wait complete. 2023-10-06T05:26:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.765+0000 7ff01b3ff700 1 Processor -- start 2023-10-06T05:26:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.765+0000 7ff01b3ff700 1 -- start start 2023-10-06T05:26:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.766+0000 7ff01b3ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0140cb3e0 0x7ff01416dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.766+0000 7ff01b3ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff014142c20 0x7ff0140b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.766+0000 7ff01b3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01415a990 0x7ff014141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.766+0000 7ff01b3ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0140b38e0 con 0x7ff0140cb3e0 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.766+0000 7ff01b3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0140b1580 con 0x7ff01415a990 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.766+0000 7ff01b3ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0140b4080 con 0x7ff014142c20 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0140cb3e0 0x7ff01416dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01919b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0140cb3e0 0x7ff01416dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59510/0 (socket says 172.21.15.31:59510) 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01919b700 1 -- 172.21.15.31:0/3623944680 learned_addr learned my addr 172.21.15.31:0/3623944680 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01899a700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff014142c20 0x7ff0140b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01999c700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01415a990 0x7ff014141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01919b700 1 -- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff014142c20 msgr2=0x7ff0140b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01919b700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff014142c20 0x7ff0140b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01919b700 1 -- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01415a990 msgr2=0x7ff014141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01919b700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01415a990 0x7ff014141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01919b700 1 -- 172.21.15.31:0/3623944680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0100094f0 con 0x7ff0140cb3e0 2023-10-06T05:26:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01899a700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff014142c20 0x7ff0140b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01999c700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01415a990 0x7ff014141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01919b700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0140cb3e0 0x7ff01416dd90 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff0100094c0 tx=0x7ff010016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff0097fa700 1 -- 172.21.15.31:0/3623944680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01000bc40 con 0x7ff0140cb3e0 2023-10-06T05:26:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff0097fa700 1 -- 172.21.15.31:0/3623944680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff010020070 con 0x7ff0140cb3e0 2023-10-06T05:26:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.767+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0140b3320 con 0x7ff0140cb3e0 2023-10-06T05:26:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.768+0000 7ff0097fa700 1 -- 172.21.15.31:0/3623944680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01001b6a0 con 0x7ff0140cb3e0 2023-10-06T05:26:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.768+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0140b1210 con 0x7ff0140cb3e0 2023-10-06T05:26:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.768+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff0140b1070 con 0x7ff0140cb3e0 2023-10-06T05:26:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.770+0000 7ff0097fa700 1 -- 172.21.15.31:0/3623944680 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7ff010022030 con 0x7ff0140cb3e0 2023-10-06T05:26:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.770+0000 7ff0097fa700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feff80679e0 0x7feff8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.770+0000 7ff0097fa700 1 -- 172.21.15.31:0/3623944680 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7ff010085050 con 0x7ff0140cb3e0 2023-10-06T05:26:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.771+0000 7ff0097fa700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] conn(0x7feff806de50 0x7feff8070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.771+0000 7ff0097fa700 1 -- 172.21.15.31:0/3623944680 --> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feff8070950 con 0x7feff806de50 2023-10-06T05:26:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.771+0000 7ff0097fa700 1 -- 172.21.15.31:0/3623944680 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff010089a00 con 0x7ff0140cb3e0 2023-10-06T05:26:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.771+0000 7ff01899a700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feff80679e0 0x7feff8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.771+0000 7ff01999c700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] conn(0x7feff806de50 0x7feff8070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.772+0000 7ff01999c700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] conn(0x7feff806de50 0x7feff8070280 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.772+0000 7ff01899a700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feff80679e0 0x7feff8069e90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff0140ca810 tx=0x7ff004008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:34.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.772+0000 7ff0097fa700 1 -- 172.21.15.31:0/3623944680 <== osd.3 v2:172.21.15.31:6823/114865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7feff8070950 con 0x7feff806de50 2023-10-06T05:26:34.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.787+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 --> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff0140cdee0 con 0x7feff806de50 2023-10-06T05:26:34.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff0097fa700 1 -- 172.21.15.31:0/3623944680 <== osd.3 v2:172.21.15.31:6823/114865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff0140cdee0 con 0x7feff806de50 2023-10-06T05:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:26:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 >> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] conn(0x7feff806de50 msgr2=0x7feff8070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff01b3ff700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.31:6823/114865,v1:172.21.15.31:6825/114865] conn(0x7feff806de50 0x7feff8070280 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feff80679e0 msgr2=0x7feff8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff01b3ff700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7feff80679e0 0x7feff8069e90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff0140ca810 tx=0x7ff004008040 comp rx=0 tx=0).stop 2023-10-06T05:26:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0140cb3e0 msgr2=0x7ff01416dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff01b3ff700 1 --2- 172.21.15.31:0/3623944680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0140cb3e0 0x7ff01416dd90 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff0100094c0 tx=0x7ff010016000 comp rx=0 tx=0).stop 2023-10-06T05:26:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff01999c700 1 -- 172.21.15.31:0/3623944680 reap_dead start 2023-10-06T05:26:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 shutdown_connections 2023-10-06T05:26:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 >> 172.21.15.31:0/3623944680 conn(0x7ff0140ce2d0 msgr2=0x7ff0140c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:34.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.788+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 shutdown_connections 2023-10-06T05:26:34.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:34.789+0000 7ff01b3ff700 1 -- 172.21.15.31:0/3623944680 wait complete. 2023-10-06T05:26:34.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T05:26:35.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.001+0000 7f3788dff700 1 Processor -- start 2023-10-06T05:26:35.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.002+0000 7f3788dff700 1 -- start start 2023-10-06T05:26:35.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.003+0000 7f3788dff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f378415d2d0 0x7f37840b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.003+0000 7f3788dff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37840c3f40 con 0x7f37840cd510 2023-10-06T05:26:35.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.003+0000 7f3788dff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37840cb000 con 0x7f3784159500 2023-10-06T05:26:35.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.003+0000 7f3788dff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37840b2860 con 0x7f378415d2d0 2023-10-06T05:26:35.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.003+0000 7f3781d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f37840cd510 0x7f37840aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56960/0 (socket says 172.21.15.31:56960) 2023-10-06T05:26:35.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.003+0000 7f3781d9b700 1 -- 172.21.15.31:0/2482545658 learned_addr learned my addr 172.21.15.31:0/2482545658 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:35.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.003+0000 7f3782d9d700 1 --2- 172.21.15.31:0/2482545658 >> v2:172.21.15.62:3300/0 conn(0x7f378415d2d0 0x7f37840b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.003+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354604537 0 0) 0x7f37840c3f40 con 0x7f37840cd510 2023-10-06T05:26:35.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.003+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f376c003570 con 0x7f37840cd510 2023-10-06T05:26:35.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3253832163 0 0) 0x7f376c003570 con 0x7f37840cd510 2023-10-06T05:26:35.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 >> v2:172.21.15.62:3300/0 conn(0x7f378415d2d0 msgr2=0x7f37840b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:35.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 --2- 172.21.15.31:0/2482545658 >> v2:172.21.15.62:3300/0 conn(0x7f378415d2d0 0x7f37840b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 >> v1:172.21.15.62:6789/0 conn(0x7f3784159500 legacy=0x7f37840cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378415d7d0 con 0x7f37840cd510 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7f37780015f0 con 0x7f37840cd510 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f3782d9d700 1 --2- 172.21.15.31:0/2482545658 >> v2:172.21.15.62:3300/0 conn(0x7f378415d2d0 0x7f37840b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 >> v1:172.21.15.31:6789/0 conn(0x7f37840cd510 legacy=0x7f37840aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c003a60 0x7f376c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f376c004810 0x7f376c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376c0051c0 0x7f376c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37840b2860 con 0x7f376c0051c0 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c00c0e0 con 0x7f376c003a60 2023-10-06T05:26:35.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376c00c2a0 con 0x7f376c004810 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.004+0000 7f3782d9d700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376c0051c0 0x7f376c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f3781d9b700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f376c004810 0x7f376c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f378259c700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c003a60 0x7f376c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f3782d9d700 1 -- 172.21.15.31:0/2482545658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f376c004810 msgr2=0x7f376c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f3782d9d700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f376c004810 0x7f376c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f3782d9d700 1 -- 172.21.15.31:0/2482545658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c003a60 msgr2=0x7f376c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f3782d9d700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c003a60 0x7f376c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f3782d9d700 1 -- 172.21.15.31:0/2482545658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c003570 con 0x7f376c0051c0 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f3781d9b700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f376c004810 0x7f376c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f378259c700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c003a60 0x7f376c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:35.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f3782d9d700 1 -- 172.21.15.31:0/2482545658 reap_dead start 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f3782d9d700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376c0051c0 0x7f376c00ba60 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f377c00e3c0 tx=0x7f377c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90224f2eb8720e54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c009c50 con 0x7f376c0051c0 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377c00f040 con 0x7f376c0051c0 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.005+0000 7f378159a700 1 -- 172.21.15.31:0/2482545658 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c00eea0 con 0x7f376c0051c0 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.006+0000 7f3788dff700 1 -- 172.21.15.31:0/2482545658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376c0051c0 msgr2=0x7f376c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.006+0000 7f3788dff700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376c0051c0 0x7f376c00ba60 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f377c00e3c0 tx=0x7f377c00b040 comp rx=0 tx=0).stop 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.006+0000 7f3788dff700 1 -- 172.21.15.31:0/2482545658 shutdown_connections 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.006+0000 7f3788dff700 1 --2- 172.21.15.31:0/2482545658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f376c0051c0 0x7f376c00ba60 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.006+0000 7f3788dff700 1 -- 172.21.15.31:0/2482545658 >> 172.21.15.31:0/2482545658 conn(0x7f378406ec90 msgr2=0x7f378406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.006+0000 7f3788dff700 1 -- 172.21.15.31:0/2482545658 shutdown_connections 2023-10-06T05:26:35.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.006+0000 7f3788dff700 1 -- 172.21.15.31:0/2482545658 wait complete. 2023-10-06T05:26:35.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.007+0000 7f3788dff700 1 Processor -- start 2023-10-06T05:26:35.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.008+0000 7f3788dff700 1 -- start start 2023-10-06T05:26:35.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.008+0000 7f3788dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c0051c0 0x7f3784152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.008+0000 7f3788dff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37841564d0 0x7f378415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.008+0000 7f3788dff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3784150550 0x7f37841509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.008+0000 7f3788dff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784150f10 con 0x7f37841564d0 2023-10-06T05:26:35.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.008+0000 7f3788dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784153580 con 0x7f376c0051c0 2023-10-06T05:26:35.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.008+0000 7f3788dff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37840b3980 con 0x7f3784150550 2023-10-06T05:26:35.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f3781d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37841564d0 0x7f378415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f3781d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37841564d0 0x7f378415f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59528/0 (socket says 172.21.15.31:59528) 2023-10-06T05:26:35.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f3781d9b700 1 -- 172.21.15.31:0/570715523 learned_addr learned my addr 172.21.15.31:0/570715523 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f378259c700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c0051c0 0x7f3784152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f3782d9d700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3784150550 0x7f37841509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f3781d9b700 1 -- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3784150550 msgr2=0x7f37841509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f3781d9b700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3784150550 0x7f37841509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f3781d9b700 1 -- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c0051c0 msgr2=0x7f3784152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f3781d9b700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c0051c0 0x7f3784152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f3781d9b700 1 -- 172.21.15.31:0/570715523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784153000 con 0x7f37841564d0 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f378259c700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c0051c0 0x7f3784152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.009+0000 7f3781d9b700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37841564d0 0x7f378415f5d0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f377800d830 tx=0x7f3778009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.010+0000 7f37727fc700 1 -- 172.21.15.31:0/570715523 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3778001e70 con 0x7f37841564d0 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.010+0000 7f37727fc700 1 -- 172.21.15.31:0/570715523 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37780070d0 con 0x7f37841564d0 2023-10-06T05:26:35.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.010+0000 7f37727fc700 1 -- 172.21.15.31:0/570715523 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3778007470 con 0x7f37841564d0 2023-10-06T05:26:35.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.010+0000 7f3788dff700 1 -- 172.21.15.31:0/570715523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37841532a0 con 0x7f37841564d0 2023-10-06T05:26:35.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.010+0000 7f3788dff700 1 -- 172.21.15.31:0/570715523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37840b3610 con 0x7f37841564d0 2023-10-06T05:26:35.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.011+0000 7f3788dff700 1 -- 172.21.15.31:0/570715523 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378404c380 con 0x7f37841564d0 2023-10-06T05:26:35.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.012+0000 7f37727fc700 1 -- 172.21.15.31:0/570715523 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f37780057f0 con 0x7f37841564d0 2023-10-06T05:26:35.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.013+0000 7f37727fc700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f37640679e0 0x7f3764069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.013+0000 7f37727fc700 1 -- 172.21.15.31:0/570715523 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f3778085530 con 0x7f37841564d0 2023-10-06T05:26:35.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.013+0000 7f378259c700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f37640679e0 0x7f3764069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.014+0000 7f378259c700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f37640679e0 0x7f3764069e90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3774003930 tx=0x7f3774009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.017+0000 7f37727fc700 1 -- 172.21.15.31:0/570715523 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3778051570 con 0x7f37841564d0 2023-10-06T05:26:35.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.220+0000 7f3788dff700 1 -- 172.21.15.31:0/570715523 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f37840b3ac0 con 0x7f37640679e0 2023-10-06T05:26:35.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.221+0000 7f37727fc700 1 -- 172.21.15.31:0/570715523 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f37840b3ac0 con 0x7f37640679e0 2023-10-06T05:26:35.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 -- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f37640679e0 msgr2=0x7f3764069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f37640679e0 0x7f3764069e90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3774003930 tx=0x7f3774009040 comp rx=0 tx=0).stop 2023-10-06T05:26:35.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 -- 172.21.15.31:0/570715523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37841564d0 msgr2=0x7f378415f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37841564d0 0x7f378415f5d0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f377800d830 tx=0x7f3778009040 comp rx=0 tx=0).stop 2023-10-06T05:26:35.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 -- 172.21.15.31:0/570715523 shutdown_connections 2023-10-06T05:26:35.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f37640679e0 0x7f3764069e90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3784150550 0x7f37841509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376c0051c0 0x7f3784152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 --2- 172.21.15.31:0/570715523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37841564d0 0x7f378415f5d0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 -- 172.21.15.31:0/570715523 >> 172.21.15.31:0/570715523 conn(0x7f378406ec90 msgr2=0x7f37840ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:35.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 -- 172.21.15.31:0/570715523 shutdown_connections 2023-10-06T05:26:35.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.223+0000 7f3788dff700 1 -- 172.21.15.31:0/570715523 wait complete. 2023-10-06T05:26:35.226 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T05:26:35.242 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T05:26:35.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:26:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.442+0000 7f8025bff700 1 Processor -- start 2023-10-06T05:26:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.442+0000 7f8025bff700 1 -- start start 2023-10-06T05:26:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.443+0000 7f8025bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8020159cb0 0x7f80200caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.443+0000 7f8025bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 0x7f802015a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.443+0000 7f8025bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80200cb420 con 0x7f802015a5c0 2023-10-06T05:26:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.443+0000 7f8025bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200cd080 con 0x7f8020159cb0 2023-10-06T05:26:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.443+0000 7f8025bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80200cb670 con 0x7f80200cafd0 2023-10-06T05:26:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.443+0000 7f801ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f802015a5c0 0x7f80200ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56970/0 (socket says 172.21.15.31:56970) 2023-10-06T05:26:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.443+0000 7f801ffff700 1 -- 172.21.15.31:0/238154723 learned_addr learned my addr 172.21.15.31:0/238154723 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:35.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.443+0000 7f801effd700 1 --2- 172.21.15.31:0/238154723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 0x7f802015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.443+0000 7f801f7fe700 1 --2- 172.21.15.31:0/238154723 >> v2:172.21.15.62:3300/0 conn(0x7f8020159cb0 0x7f80200caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.444+0000 7f801e7fc700 1 -- 172.21.15.31:0/238154723 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370043965 0 0) 0x7f80200cb420 con 0x7f802015a5c0 2023-10-06T05:26:35.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.444+0000 7f801e7fc700 1 -- 172.21.15.31:0/238154723 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8008003570 con 0x7f802015a5c0 2023-10-06T05:26:35.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.444+0000 7f801effd700 1 -- 172.21.15.31:0/238154723 >> v2:172.21.15.62:3300/0 conn(0x7f8020159cb0 msgr2=0x7f80200caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:35.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.444+0000 7f801effd700 1 --2- 172.21.15.31:0/238154723 >> v2:172.21.15.62:3300/0 conn(0x7f8020159cb0 0x7f80200caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.444+0000 7f801effd700 1 -- 172.21.15.31:0/238154723 >> v1:172.21.15.31:6789/0 conn(0x7f802015a5c0 legacy=0x7f80200ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.444+0000 7f801effd700 1 -- 172.21.15.31:0/238154723 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80200cb950 con 0x7f80200cafd0 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.444+0000 7f801effd700 1 --2- 172.21.15.31:0/238154723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 0x7f802015a080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f80140098e0 tx=0x7f80140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7da36a06f0d984 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f801e7fc700 1 -- 172.21.15.31:0/238154723 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014011630 con 0x7f80200cafd0 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f801e7fc700 1 -- 172.21.15.31:0/238154723 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014011790 con 0x7f80200cafd0 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f801e7fc700 1 -- 172.21.15.31:0/238154723 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014011b30 con 0x7f80200cafd0 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f8025bff700 1 -- 172.21.15.31:0/238154723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 msgr2=0x7f802015a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f8025bff700 1 --2- 172.21.15.31:0/238154723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 0x7f802015a080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f80140098e0 tx=0x7f80140091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f8025bff700 1 -- 172.21.15.31:0/238154723 shutdown_connections 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f8025bff700 1 --2- 172.21.15.31:0/238154723 >> v2:172.21.15.62:3300/0 conn(0x7f8020159cb0 0x7f80200caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f8025bff700 1 --2- 172.21.15.31:0/238154723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 0x7f802015a080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f8025bff700 1 -- 172.21.15.31:0/238154723 >> 172.21.15.31:0/238154723 conn(0x7f8020099710 msgr2=0x7f802006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:35.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f8025bff700 1 -- 172.21.15.31:0/238154723 shutdown_connections 2023-10-06T05:26:35.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.445+0000 7f8025bff700 1 -- 172.21.15.31:0/238154723 wait complete. 2023-10-06T05:26:35.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.447+0000 7f8025bff700 1 Processor -- start 2023-10-06T05:26:35.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.447+0000 7f8025bff700 1 -- start start 2023-10-06T05:26:35.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f8025bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 0x7f8020145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f8025bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8020159cb0 0x7f80200c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f8025bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802015a5c0 0x7f80200693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f8025bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80200c53b0 con 0x7f8020159cb0 2023-10-06T05:26:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f8025bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80200c47a0 con 0x7f802015a5c0 2023-10-06T05:26:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f8025bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80200c4a80 con 0x7f80200cafd0 2023-10-06T05:26:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f801effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8020159cb0 0x7f80200c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f801effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8020159cb0 0x7f80200c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59534/0 (socket says 172.21.15.31:59534) 2023-10-06T05:26:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f801effd700 1 -- 172.21.15.31:0/3636706117 learned_addr learned my addr 172.21.15.31:0/3636706117 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f801f7fe700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 0x7f8020145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f801ffff700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802015a5c0 0x7f80200693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f801effd700 1 -- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 msgr2=0x7f8020145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f801effd700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 0x7f8020145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f801effd700 1 -- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802015a5c0 msgr2=0x7f80200693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f801effd700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802015a5c0 0x7f80200693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.448+0000 7f801effd700 1 -- 172.21.15.31:0/3636706117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8014009420 con 0x7f8020159cb0 2023-10-06T05:26:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.449+0000 7f801f7fe700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80200cafd0 0x7f8020145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.449+0000 7f801ffff700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802015a5c0 0x7f80200693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.449+0000 7f801effd700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8020159cb0 0x7f80200c3f30 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f80200cb7b0 tx=0x7f8014017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.449+0000 7f80077fe700 1 -- 172.21.15.31:0/3636706117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014011b10 con 0x7f8020159cb0 2023-10-06T05:26:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.449+0000 7f80077fe700 1 -- 172.21.15.31:0/3636706117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80140253f0 con 0x7f8020159cb0 2023-10-06T05:26:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.449+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80200c4d60 con 0x7f8020159cb0 2023-10-06T05:26:35.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.449+0000 7f80077fe700 1 -- 172.21.15.31:0/3636706117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014025790 con 0x7f8020159cb0 2023-10-06T05:26:35.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.449+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8020145d90 con 0x7f8020159cb0 2023-10-06T05:26:35.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.450+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f802004c380 con 0x7f8020159cb0 2023-10-06T05:26:35.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.451+0000 7f80077fe700 1 -- 172.21.15.31:0/3636706117 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f8014025b80 con 0x7f8020159cb0 2023-10-06T05:26:35.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.452+0000 7f80077fe700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f800c0679e0 0x7f800c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.452+0000 7f80077fe700 1 -- 172.21.15.31:0/3636706117 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f8014085380 con 0x7f8020159cb0 2023-10-06T05:26:35.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.452+0000 7f80077fe700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] conn(0x7f800c06de50 0x7f800c070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.452+0000 7f80077fe700 1 -- 172.21.15.31:0/3636706117 --> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f800c070950 con 0x7f800c06de50 2023-10-06T05:26:35.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.452+0000 7f80077fe700 1 -- 172.21.15.31:0/3636706117 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8014087e50 con 0x7f8020159cb0 2023-10-06T05:26:35.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.453+0000 7f801f7fe700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f800c0679e0 0x7f800c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.453+0000 7f801ffff700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] conn(0x7f800c06de50 0x7f800c070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.453+0000 7f801f7fe700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f800c0679e0 0x7f800c069e90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8010009390 tx=0x7f8010008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.453+0000 7f801ffff700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] conn(0x7f800c06de50 0x7f800c070280 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.454+0000 7f80077fe700 1 -- 172.21.15.31:0/3636706117 <== osd.4 v2:172.21.15.62:6800/115118 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f800c070950 con 0x7f800c06de50 2023-10-06T05:26:35.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.469+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 --> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f80200cbc30 con 0x7f800c06de50 2023-10-06T05:26:35.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f80077fe700 1 -- 172.21.15.31:0/3636706117 <== osd.4 v2:172.21.15.62:6800/115118 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f80200cbc30 con 0x7f800c06de50 2023-10-06T05:26:35.471 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:26:35.472 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:26:35.472 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:26:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] conn(0x7f800c06de50 msgr2=0x7f800c070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f8025bff700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:6800/115118,v1:172.21.15.62:6801/115118] conn(0x7f800c06de50 0x7f800c070280 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f800c0679e0 msgr2=0x7f800c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f8025bff700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f800c0679e0 0x7f800c069e90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8010009390 tx=0x7f8010008040 comp rx=0 tx=0).stop 2023-10-06T05:26:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8020159cb0 msgr2=0x7f80200c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f8025bff700 1 --2- 172.21.15.31:0/3636706117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8020159cb0 0x7f80200c3f30 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f80200cb7b0 tx=0x7f8014017000 comp rx=0 tx=0).stop 2023-10-06T05:26:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f801ffff700 1 -- 172.21.15.31:0/3636706117 reap_dead start 2023-10-06T05:26:35.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 shutdown_connections 2023-10-06T05:26:35.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 >> 172.21.15.31:0/3636706117 conn(0x7f8020099710 msgr2=0x7f80200cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:35.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 shutdown_connections 2023-10-06T05:26:35.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.470+0000 7f8025bff700 1 -- 172.21.15.31:0/3636706117 wait complete. 2023-10-06T05:26:35.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T05:26:35.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.684+0000 7fc935bff700 1 Processor -- start 2023-10-06T05:26:35.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.685+0000 7fc935bff700 1 -- start start 2023-10-06T05:26:35.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.686+0000 7fc935bff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc930159500 0x7fc9300b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.686+0000 7fc935bff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9300c3f40 con 0x7fc93015d2d0 2023-10-06T05:26:35.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.686+0000 7fc935bff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9300cb000 con 0x7fc9300cd510 2023-10-06T05:26:35.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.686+0000 7fc935bff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9300b2860 con 0x7fc930159500 2023-10-06T05:26:35.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.686+0000 7fc92effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc93015d2d0 0x7fc9300cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56972/0 (socket says 172.21.15.31:56972) 2023-10-06T05:26:35.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.686+0000 7fc92effd700 1 -- 172.21.15.31:0/42193525 learned_addr learned my addr 172.21.15.31:0/42193525 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:35.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.686+0000 7fc92f7fe700 1 --2- 172.21.15.31:0/42193525 >> v2:172.21.15.62:3300/0 conn(0x7fc930159500 0x7fc9300b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.687+0000 7fc92e7fc700 1 -- 172.21.15.31:0/42193525 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099476996 0 0) 0x7fc9300c3f40 con 0x7fc93015d2d0 2023-10-06T05:26:35.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.687+0000 7fc92e7fc700 1 -- 172.21.15.31:0/42193525 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc918003570 con 0x7fc93015d2d0 2023-10-06T05:26:35.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.687+0000 7fc92f7fe700 1 -- 172.21.15.31:0/42193525 >> v1:172.21.15.62:6789/0 conn(0x7fc9300cd510 legacy=0x7fc9300aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.687+0000 7fc92f7fe700 1 -- 172.21.15.31:0/42193525 >> v1:172.21.15.31:6789/0 conn(0x7fc93015d2d0 legacy=0x7fc9300cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.687+0000 7fc92e7fc700 1 -- 172.21.15.31:0/42193525 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3623598889 0 0) 0x7fc918003570 con 0x7fc93015d2d0 2023-10-06T05:26:35.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.687+0000 7fc92f7fe700 1 -- 172.21.15.31:0/42193525 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93015d7d0 con 0x7fc930159500 2023-10-06T05:26:35.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.687+0000 7fc92f7fe700 1 --2- 172.21.15.31:0/42193525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc930159500 0x7fc9300b0ab0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098e0 tx=0x7fc9200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b25e66bfddfa5c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.688+0000 7fc92e7fc700 1 -- 172.21.15.31:0/42193525 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240015f0 con 0x7fc930159500 2023-10-06T05:26:35.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.688+0000 7fc92e7fc700 1 -- 172.21.15.31:0/42193525 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc920011420 con 0x7fc930159500 2023-10-06T05:26:35.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.688+0000 7fc92e7fc700 1 -- 172.21.15.31:0/42193525 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9200117a0 con 0x7fc930159500 2023-10-06T05:26:35.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.688+0000 7fc935bff700 1 -- 172.21.15.31:0/42193525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc930159500 msgr2=0x7fc9300b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.688+0000 7fc935bff700 1 --2- 172.21.15.31:0/42193525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc930159500 0x7fc9300b0ab0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098e0 tx=0x7fc9200091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.688+0000 7fc935bff700 1 -- 172.21.15.31:0/42193525 shutdown_connections 2023-10-06T05:26:35.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.688+0000 7fc935bff700 1 --2- 172.21.15.31:0/42193525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc930159500 0x7fc9300b0ab0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.688+0000 7fc935bff700 1 -- 172.21.15.31:0/42193525 >> 172.21.15.31:0/42193525 conn(0x7fc93006ec90 msgr2=0x7fc93006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:35.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.688+0000 7fc935bff700 1 -- 172.21.15.31:0/42193525 shutdown_connections 2023-10-06T05:26:35.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.688+0000 7fc935bff700 1 -- 172.21.15.31:0/42193525 wait complete. 2023-10-06T05:26:35.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.690+0000 7fc935bff700 1 Processor -- start 2023-10-06T05:26:35.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.690+0000 7fc935bff700 1 -- start start 2023-10-06T05:26:35.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.691+0000 7fc935bff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9300cd510 0x7fc93015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.691+0000 7fc935bff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc930159500 0x7fc930158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.691+0000 7fc935bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc93015d2d0 0x7fc9300b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.691+0000 7fc935bff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93016a900 con 0x7fc9300cd510 2023-10-06T05:26:35.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.691+0000 7fc935bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93016abe0 con 0x7fc93015d2d0 2023-10-06T05:26:35.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.691+0000 7fc935bff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93016aec0 con 0x7fc930159500 2023-10-06T05:26:35.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.691+0000 7fc92f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9300cd510 0x7fc93015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.691+0000 7fc92f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9300cd510 0x7fc93015a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59540/0 (socket says 172.21.15.31:59540) 2023-10-06T05:26:35.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.691+0000 7fc92f7fe700 1 -- 172.21.15.31:0/2774956844 learned_addr learned my addr 172.21.15.31:0/2774956844 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:35.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc92f7fe700 1 -- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc930159500 msgr2=0x7fc930158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc92f7fe700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc930159500 0x7fc930158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc92f7fe700 1 -- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc93015d2d0 msgr2=0x7fc9300b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc92ffff700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc93015d2d0 0x7fc9300b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc92f7fe700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc93015d2d0 0x7fc9300b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc92f7fe700 1 -- 172.21.15.31:0/2774956844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc920009420 con 0x7fc9300cd510 2023-10-06T05:26:35.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc92ffff700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc93015d2d0 0x7fc9300b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:26:35.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc92f7fe700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9300cd510 0x7fc93015a990 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fc920009fd0 tx=0x7fc920016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc9177fe700 1 -- 172.21.15.31:0/2774956844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92001cce0 con 0x7fc9300cd510 2023-10-06T05:26:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc9177fe700 1 -- 172.21.15.31:0/2774956844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92001ce60 con 0x7fc9300cd510 2023-10-06T05:26:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc9177fe700 1 -- 172.21.15.31:0/2774956844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92001a680 con 0x7fc9300cd510 2023-10-06T05:26:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.692+0000 7fc935bff700 1 -- 172.21.15.31:0/2774956844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93016b1a0 con 0x7fc9300cd510 2023-10-06T05:26:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.693+0000 7fc935bff700 1 -- 172.21.15.31:0/2774956844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc930158d60 con 0x7fc9300cd510 2023-10-06T05:26:35.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.694+0000 7fc935bff700 1 -- 172.21.15.31:0/2774956844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9300c6b60 con 0x7fc9300cd510 2023-10-06T05:26:35.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.694+0000 7fc9177fe700 1 -- 172.21.15.31:0/2774956844 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fc92001aab0 con 0x7fc9300cd510 2023-10-06T05:26:35.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.695+0000 7fc9177fe700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc91c0679e0 0x7fc91c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:35.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.695+0000 7fc9177fe700 1 -- 172.21.15.31:0/2774956844 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fc920084be0 con 0x7fc9300cd510 2023-10-06T05:26:35.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.696+0000 7fc92effd700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc91c0679e0 0x7fc91c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:35.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.697+0000 7fc92effd700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc91c0679e0 0x7fc91c069e90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc924002ee0 tx=0x7fc924002e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:35.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.700+0000 7fc9177fe700 1 -- 172.21.15.31:0/2774956844 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc920053570 con 0x7fc9300cd510 2023-10-06T05:26:35.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.904+0000 7fc935bff700 1 -- 172.21.15.31:0/2774956844 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc93016ad50 con 0x7fc91c0679e0 2023-10-06T05:26:35.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.906+0000 7fc9177fe700 1 -- 172.21.15.31:0/2774956844 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc93016ad50 con 0x7fc91c0679e0 2023-10-06T05:26:35.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 -- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc91c0679e0 msgr2=0x7fc91c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc91c0679e0 0x7fc91c069e90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc924002ee0 tx=0x7fc924002e10 comp rx=0 tx=0).stop 2023-10-06T05:26:35.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 -- 172.21.15.31:0/2774956844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9300cd510 msgr2=0x7fc93015a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:35.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9300cd510 0x7fc93015a990 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fc920009fd0 tx=0x7fc920016000 comp rx=0 tx=0).stop 2023-10-06T05:26:35.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 -- 172.21.15.31:0/2774956844 shutdown_connections 2023-10-06T05:26:35.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc91c0679e0 0x7fc91c069e90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc93015d2d0 0x7fc9300b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9300cd510 0x7fc93015a990 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 --2- 172.21.15.31:0/2774956844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc930159500 0x7fc930158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:35.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 -- 172.21.15.31:0/2774956844 >> 172.21.15.31:0/2774956844 conn(0x7fc93006ec90 msgr2=0x7fc9300b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:35.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 -- 172.21.15.31:0/2774956844 shutdown_connections 2023-10-06T05:26:35.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:35.907+0000 7fc935bff700 1 -- 172.21.15.31:0/2774956844 wait complete. 2023-10-06T05:26:35.911 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T05:26:35.927 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T05:26:35.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:26:36.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.124+0000 7efc9f5ff700 1 Processor -- start 2023-10-06T05:26:36.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.125+0000 7efc9f5ff700 1 -- start start 2023-10-06T05:26:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.126+0000 7efc9f5ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efc98153d20 0x7efc9815d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.126+0000 7efc9f5ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc9815dbb0 con 0x7efc98153950 2023-10-06T05:26:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.126+0000 7efc9f5ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc9815bb20 con 0x7efc9815d750 2023-10-06T05:26:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.126+0000 7efc9f5ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc98154170 con 0x7efc98153d20 2023-10-06T05:26:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.126+0000 7efc9d39b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efc98153950 0x7efc980cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56976/0 (socket says 172.21.15.31:56976) 2023-10-06T05:26:36.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.126+0000 7efc9d39b700 1 -- 172.21.15.31:0/2558676825 learned_addr learned my addr 172.21.15.31:0/2558676825 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:36.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.126+0000 7efc9cb9a700 1 --2- 172.21.15.31:0/2558676825 >> v2:172.21.15.62:3300/0 conn(0x7efc98153d20 0x7efc9815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019683165 0 0) 0x7efc94001290 con 0x7efc98153950 2023-10-06T05:26:36.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc84003570 con 0x7efc98153950 2023-10-06T05:26:36.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 382563369 0 0) 0x7efc84003570 con 0x7efc98153950 2023-10-06T05:26:36.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 >> v2:172.21.15.62:3300/0 conn(0x7efc98153d20 msgr2=0x7efc9815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 --2- 172.21.15.31:0/2558676825 >> v2:172.21.15.62:3300/0 conn(0x7efc98153d20 0x7efc9815d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 >> v1:172.21.15.62:6789/0 conn(0x7efc9815d750 legacy=0x7efc9815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9815af10 con 0x7efc98153950 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 108782472 0 0) 0x7efc94001780 con 0x7efc98153950 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 >> v1:172.21.15.31:6789/0 conn(0x7efc98153950 legacy=0x7efc980cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc84003a60 0x7efc84003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84004810 0x7efc84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc840051c0 0x7efc8400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8400c130 con 0x7efc84004810 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8400c2a0 con 0x7efc840051c0 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8400c460 con 0x7efc84003a60 2023-10-06T05:26:36.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc9cb9a700 1 --2- 172.21.15.31:0/2558676825 >> v2:172.21.15.62:3300/0 conn(0x7efc98153d20 0x7efc9815d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.127+0000 7efc9cb9a700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84004810 0x7efc84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc9db9c700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc840051c0 0x7efc8400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc9d39b700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc84003a60 0x7efc84003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc9cb9a700 1 -- 172.21.15.31:0/2558676825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc84003a60 msgr2=0x7efc84003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc9cb9a700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc84003a60 0x7efc84003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc9cb9a700 1 -- 172.21.15.31:0/2558676825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc840051c0 msgr2=0x7efc8400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc9cb9a700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc840051c0 0x7efc8400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc9cb9a700 1 -- 172.21.15.31:0/2558676825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc84003570 con 0x7efc84004810 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc9db9c700 1 -- 172.21.15.31:0/2558676825 reap_dead start 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc9cb9a700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84004810 0x7efc84004c80 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7efc88009db0 tx=0x7efc880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31826a880c61ab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88009740 con 0x7efc84004810 2023-10-06T05:26:36.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8800e920 con 0x7efc84004810 2023-10-06T05:26:36.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.128+0000 7efc8ffff700 1 -- 172.21.15.31:0/2558676825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8800eca0 con 0x7efc84004810 2023-10-06T05:26:36.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.129+0000 7efc9f5ff700 1 -- 172.21.15.31:0/2558676825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84004810 msgr2=0x7efc84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.129+0000 7efc9f5ff700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84004810 0x7efc84004c80 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7efc88009db0 tx=0x7efc880091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.129+0000 7efc9f5ff700 1 -- 172.21.15.31:0/2558676825 shutdown_connections 2023-10-06T05:26:36.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.129+0000 7efc9f5ff700 1 --2- 172.21.15.31:0/2558676825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84004810 0x7efc84004c80 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.129+0000 7efc9f5ff700 1 -- 172.21.15.31:0/2558676825 >> 172.21.15.31:0/2558676825 conn(0x7efc980681b0 msgr2=0x7efc980685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.129+0000 7efc9f5ff700 1 -- 172.21.15.31:0/2558676825 shutdown_connections 2023-10-06T05:26:36.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.129+0000 7efc9f5ff700 1 -- 172.21.15.31:0/2558676825 wait complete. 2023-10-06T05:26:36.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.130+0000 7efc9f5ff700 1 Processor -- start 2023-10-06T05:26:36.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.131+0000 7efc9f5ff700 1 -- start start 2023-10-06T05:26:36.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.131+0000 7efc9f5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc84004810 0x7efc980c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.131+0000 7efc9f5ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc9815a300 0x7efc98169b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.131+0000 7efc9f5ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc9816a050 0x7efc9816a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.131+0000 7efc9f5ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9816aa30 con 0x7efc9816a050 2023-10-06T05:26:36.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.131+0000 7efc9f5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9814e8a0 con 0x7efc84004810 2023-10-06T05:26:36.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.131+0000 7efc9f5ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc980ca3e0 con 0x7efc9815a300 2023-10-06T05:26:36.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9db9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc9816a050 0x7efc9816a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9db9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc9816a050 0x7efc9816a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59568/0 (socket says 172.21.15.31:59568) 2023-10-06T05:26:36.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9db9c700 1 -- 172.21.15.31:0/1508064107 learned_addr learned my addr 172.21.15.31:0/1508064107 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9d39b700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc84004810 0x7efc980c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9cb9a700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc9815a300 0x7efc98169b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9db9c700 1 -- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc9815a300 msgr2=0x7efc98169b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9db9c700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc9815a300 0x7efc98169b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9db9c700 1 -- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc84004810 msgr2=0x7efc980c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9db9c700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc84004810 0x7efc980c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9db9c700 1 -- 172.21.15.31:0/1508064107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc980c9e20 con 0x7efc9816a050 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9d39b700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc84004810 0x7efc980c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.132+0000 7efc9db9c700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc9816a050 0x7efc9816a4f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7efc98069080 tx=0x7efc90001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.133+0000 7efc8d7fa700 1 -- 172.21.15.31:0/1508064107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90006db0 con 0x7efc9816a050 2023-10-06T05:26:36.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.133+0000 7efc8d7fa700 1 -- 172.21.15.31:0/1508064107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9000c5a0 con 0x7efc9816a050 2023-10-06T05:26:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.133+0000 7efc8d7fa700 1 -- 172.21.15.31:0/1508064107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9000c960 con 0x7efc9816a050 2023-10-06T05:26:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.133+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc980ca100 con 0x7efc9816a050 2023-10-06T05:26:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.133+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9816ace0 con 0x7efc9816a050 2023-10-06T05:26:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.134+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efc9804c380 con 0x7efc9816a050 2023-10-06T05:26:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.135+0000 7efc8d7fa700 1 -- 172.21.15.31:0/1508064107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7efc9000cd50 con 0x7efc9816a050 2023-10-06T05:26:36.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.136+0000 7efc8d7fa700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efc7c0679e0 0x7efc7c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.136+0000 7efc8d7fa700 1 -- 172.21.15.31:0/1508064107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7efc90082a60 con 0x7efc9816a050 2023-10-06T05:26:36.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.136+0000 7efc8d7fa700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] conn(0x7efc7c06de50 0x7efc7c070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.136+0000 7efc8d7fa700 1 -- 172.21.15.31:0/1508064107 --> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc7c070950 con 0x7efc7c06de50 2023-10-06T05:26:36.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.136+0000 7efc8d7fa700 1 -- 172.21.15.31:0/1508064107 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc90087390 con 0x7efc9816a050 2023-10-06T05:26:36.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.136+0000 7efc9d39b700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efc7c0679e0 0x7efc7c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.136+0000 7efc9cb9a700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] conn(0x7efc7c06de50 0x7efc7c070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.137+0000 7efc9d39b700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efc7c0679e0 0x7efc7c069e90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efc94000ef0 tx=0x7efc94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.137+0000 7efc9cb9a700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] conn(0x7efc7c06de50 0x7efc7c070280 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.138+0000 7efc8d7fa700 1 -- 172.21.15.31:0/1508064107 <== osd.5 v2:172.21.15.62:6816/115134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7efc7c070950 con 0x7efc7c06de50 2023-10-06T05:26:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.153+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 --> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efc9815b1f0 con 0x7efc7c06de50 2023-10-06T05:26:36.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc8d7fa700 1 -- 172.21.15.31:0/1508064107 <== osd.5 v2:172.21.15.62:6816/115134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efc9815b1f0 con 0x7efc7c06de50 2023-10-06T05:26:36.155 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:26:36.156 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:26:36.156 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:26:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] conn(0x7efc7c06de50 msgr2=0x7efc7c070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9f5ff700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:6816/115134,v1:172.21.15.62:6817/115134] conn(0x7efc7c06de50 0x7efc7c070280 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efc7c0679e0 msgr2=0x7efc7c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9f5ff700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7efc7c0679e0 0x7efc7c069e90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efc94000ef0 tx=0x7efc94009040 comp rx=0 tx=0).stop 2023-10-06T05:26:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc9816a050 msgr2=0x7efc9816a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9f5ff700 1 --2- 172.21.15.31:0/1508064107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc9816a050 0x7efc9816a4f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7efc98069080 tx=0x7efc90001640 comp rx=0 tx=0).stop 2023-10-06T05:26:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9db9c700 1 -- 172.21.15.31:0/1508064107 reap_dead start 2023-10-06T05:26:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 shutdown_connections 2023-10-06T05:26:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 >> 172.21.15.31:0/1508064107 conn(0x7efc980681b0 msgr2=0x7efc980cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 shutdown_connections 2023-10-06T05:26:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.154+0000 7efc9f5ff700 1 -- 172.21.15.31:0/1508064107 wait complete. 2023-10-06T05:26:36.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T05:26:36.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.369+0000 7f5d623ff700 1 Processor -- start 2023-10-06T05:26:36.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.369+0000 7f5d623ff700 1 -- start start 2023-10-06T05:26:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.370+0000 7f5d623ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d5c0cdfe0 0x7f5d5c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.370+0000 7f5d623ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d5c0ccc80 0x7f5d5c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.370+0000 7f5d623ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5c0cf280 con 0x7f5d5c159580 2023-10-06T05:26:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.370+0000 7f5d623ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c0ce480 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.370+0000 7f5d623ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c0ce670 con 0x7f5d5c0cdfe0 2023-10-06T05:26:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.370+0000 7f5d5bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5d5c159580 0x7f5d5c0cb9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56992/0 (socket says 172.21.15.31:56992) 2023-10-06T05:26:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.370+0000 7f5d5bfff700 1 -- 172.21.15.31:0/4193877248 learned_addr learned my addr 172.21.15.31:0/4193877248 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:36.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.370+0000 7f5d6099c700 1 --2- 172.21.15.31:0/4193877248 >> v2:172.21.15.31:3300/0 conn(0x7f5d5c0ccc80 0x7f5d5c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.370+0000 7f5d5b7fe700 1 --2- 172.21.15.31:0/4193877248 >> v2:172.21.15.62:3300/0 conn(0x7f5d5c0cdfe0 0x7f5d5c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.371+0000 7f5d6099c700 1 -- 172.21.15.31:0/4193877248 >> v2:172.21.15.62:3300/0 conn(0x7f5d5c0cdfe0 msgr2=0x7f5d5c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.371+0000 7f5d6099c700 1 --2- 172.21.15.31:0/4193877248 >> v2:172.21.15.62:3300/0 conn(0x7f5d5c0cdfe0 0x7f5d5c0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.371+0000 7f5d5affd700 1 -- 172.21.15.31:0/4193877248 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877575099 0 0) 0x7f5d5c0cf280 con 0x7f5d5c159580 2023-10-06T05:26:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.371+0000 7f5d6099c700 1 -- 172.21.15.31:0/4193877248 >> v1:172.21.15.31:6789/0 conn(0x7f5d5c159580 legacy=0x7f5d5c0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.371+0000 7f5d6099c700 1 -- 172.21.15.31:0/4193877248 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c0ce950 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.371+0000 7f5d6099c700 1 --2- 172.21.15.31:0/4193877248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 0x7f5d5c0cd080 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c06fc50 tx=0x7f5d54009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aab3ba735070676 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.371+0000 7f5d5affd700 1 -- 172.21.15.31:0/4193877248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400e070 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.371+0000 7f5d5affd700 1 -- 172.21.15.31:0/4193877248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5400bc30 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.371+0000 7f5d5affd700 1 -- 172.21.15.31:0/4193877248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54012430 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.372+0000 7f5d623ff700 1 -- 172.21.15.31:0/4193877248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 msgr2=0x7f5d5c0cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.372+0000 7f5d623ff700 1 --2- 172.21.15.31:0/4193877248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 0x7f5d5c0cd080 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c06fc50 tx=0x7f5d54009780 comp rx=0 tx=0).stop 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.372+0000 7f5d623ff700 1 -- 172.21.15.31:0/4193877248 shutdown_connections 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.372+0000 7f5d623ff700 1 --2- 172.21.15.31:0/4193877248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 0x7f5d5c0cd080 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.372+0000 7f5d623ff700 1 --2- 172.21.15.31:0/4193877248 >> v2:172.21.15.62:3300/0 conn(0x7f5d5c0cdfe0 0x7f5d5c0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.372+0000 7f5d623ff700 1 -- 172.21.15.31:0/4193877248 >> 172.21.15.31:0/4193877248 conn(0x7f5d5c099710 msgr2=0x7f5d5c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.372+0000 7f5d623ff700 1 -- 172.21.15.31:0/4193877248 shutdown_connections 2023-10-06T05:26:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.372+0000 7f5d623ff700 1 -- 172.21.15.31:0/4193877248 wait complete. 2023-10-06T05:26:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.373+0000 7f5d623ff700 1 Processor -- start 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.373+0000 7f5d623ff700 1 -- start start 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.374+0000 7f5d623ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 0x7f5d5c15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.374+0000 7f5d623ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d5c0cdfe0 0x7f5d5c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.374+0000 7f5d623ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d5c159580 0x7f5d5c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.374+0000 7f5d623ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c15dd30 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.374+0000 7f5d623ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0b1ea0 con 0x7f5d5c0cdfe0 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.374+0000 7f5d623ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0b20f0 con 0x7f5d5c159580 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.374+0000 7f5d5bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 0x7f5d5c15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d5bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 0x7f5d5c15cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59600/0 (socket says 172.21.15.31:59600) 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d5bfff700 1 -- 172.21.15.31:0/2430466408 learned_addr learned my addr 172.21.15.31:0/2430466408 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:36.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d5b7fe700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d5c0cdfe0 0x7f5d5c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d6099c700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d5c159580 0x7f5d5c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d5bfff700 1 -- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d5c159580 msgr2=0x7f5d5c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d5bfff700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d5c159580 0x7f5d5c15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d5bfff700 1 -- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d5c0cdfe0 msgr2=0x7f5d5c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d5bfff700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d5c0cdfe0 0x7f5d5c15d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d5bfff700 1 -- 172.21.15.31:0/2430466408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d54009a20 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d5b7fe700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d5c0cdfe0 0x7f5d5c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d5bfff700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 0x7f5d5c15cd70 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00c5e0 tx=0x7f5d4c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d43fff700 1 -- 172.21.15.31:0/2430466408 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0072e0 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d43fff700 1 -- 172.21.15.31:0/2430466408 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c007440 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.375+0000 7f5d623ff700 1 -- 172.21.15.31:0/2430466408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c0b23d0 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.376+0000 7f5d43fff700 1 -- 172.21.15.31:0/2430466408 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0057f0 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.376+0000 7f5d623ff700 1 -- 172.21.15.31:0/2430466408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c0b2760 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.376+0000 7f5d623ff700 1 -- 172.21.15.31:0/2430466408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c04c380 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.377+0000 7f5d43fff700 1 -- 172.21.15.31:0/2430466408 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f5d4c005c00 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.378+0000 7f5d43fff700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d480679e0 0x7f5d48069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.378+0000 7f5d43fff700 1 -- 172.21.15.31:0/2430466408 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f5d4c082690 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.379+0000 7f5d5b7fe700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d480679e0 0x7f5d48069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.380+0000 7f5d5b7fe700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d480679e0 0x7f5d48069e90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0c7430 tx=0x7f5d50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.383+0000 7f5d43fff700 1 -- 172.21.15.31:0/2430466408 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d4c051100 con 0x7f5d5c0ccc80 2023-10-06T05:26:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.584+0000 7f5d623ff700 1 -- 172.21.15.31:0/2430466408 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d5c0cecc0 con 0x7f5d480679e0 2023-10-06T05:26:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.585+0000 7f5d43fff700 1 -- 172.21.15.31:0/2430466408 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5d5c0cecc0 con 0x7f5d480679e0 2023-10-06T05:26:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 -- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d480679e0 msgr2=0x7f5d48069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d480679e0 0x7f5d48069e90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0c7430 tx=0x7f5d50008040 comp rx=0 tx=0).stop 2023-10-06T05:26:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 -- 172.21.15.31:0/2430466408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 msgr2=0x7f5d5c15cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 0x7f5d5c15cd70 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00c5e0 tx=0x7f5d4c007e80 comp rx=0 tx=0).stop 2023-10-06T05:26:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 -- 172.21.15.31:0/2430466408 shutdown_connections 2023-10-06T05:26:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f5d480679e0 0x7f5d48069e90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d5c159580 0x7f5d5c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c0ccc80 0x7f5d5c15cd70 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 --2- 172.21.15.31:0/2430466408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d5c0cdfe0 0x7f5d5c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 -- 172.21.15.31:0/2430466408 >> 172.21.15.31:0/2430466408 conn(0x7f5d5c099710 msgr2=0x7f5d5c0ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 -- 172.21.15.31:0/2430466408 shutdown_connections 2023-10-06T05:26:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.587+0000 7f5d623ff700 1 -- 172.21.15.31:0/2430466408 wait complete. 2023-10-06T05:26:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T05:26:36.606 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T05:26:36.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:26:36.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.802+0000 7fb4d87ff700 1 Processor -- start 2023-10-06T05:26:36.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.802+0000 7fb4d87ff700 1 -- start start 2023-10-06T05:26:36.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d87ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4d006df30 0x7fb4d006d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d87ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d00c72b0 con 0x7fb4d006d830 2023-10-06T05:26:36.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d87ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d00c7590 con 0x7fb4d0159d70 2023-10-06T05:26:36.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d87ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d00aba60 con 0x7fb4d006df30 2023-10-06T05:26:36.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d659b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4d006df30 0x7fb4d006d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d659b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4d006df30 0x7fb4d006d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59602/0 (socket says 172.21.15.31:59602) 2023-10-06T05:26:36.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d5d9a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb4d006d830 0x7fb4d0159660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56998/0 (socket says 172.21.15.31:56998) 2023-10-06T05:26:36.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d659b700 1 -- 172.21.15.31:0/3063746300 learned_addr learned my addr 172.21.15.31:0/3063746300 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:36.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d659b700 1 -- 172.21.15.31:0/3063746300 >> v1:172.21.15.62:6789/0 conn(0x7fb4d0159d70 legacy=0x7fb4d0152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d659b700 1 -- 172.21.15.31:0/3063746300 >> v1:172.21.15.31:6789/0 conn(0x7fb4d006d830 legacy=0x7fb4d0159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.804+0000 7fb4d659b700 1 -- 172.21.15.31:0/3063746300 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d00c7870 con 0x7fb4d006df30 2023-10-06T05:26:36.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.805+0000 7fb4d659b700 1 --2- 172.21.15.31:0/3063746300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d006df30 0x7fb4d006d2f0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00bc60 tx=0x7fb4cc00b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74085e28529bb079 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.805+0000 7fb4d5599700 1 -- 172.21.15.31:0/3063746300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00be70 con 0x7fb4d006df30 2023-10-06T05:26:36.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.805+0000 7fb4d5599700 1 -- 172.21.15.31:0/3063746300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc00ddc0 con 0x7fb4d006df30 2023-10-06T05:26:36.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.805+0000 7fb4d5599700 1 -- 172.21.15.31:0/3063746300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc005c10 con 0x7fb4d006df30 2023-10-06T05:26:36.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.805+0000 7fb4d87ff700 1 -- 172.21.15.31:0/3063746300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d006df30 msgr2=0x7fb4d006d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.805+0000 7fb4d87ff700 1 --2- 172.21.15.31:0/3063746300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d006df30 0x7fb4d006d2f0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00bc60 tx=0x7fb4cc00b4c0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.806+0000 7fb4d87ff700 1 -- 172.21.15.31:0/3063746300 shutdown_connections 2023-10-06T05:26:36.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.806+0000 7fb4d87ff700 1 --2- 172.21.15.31:0/3063746300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d006df30 0x7fb4d006d2f0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.806+0000 7fb4d87ff700 1 -- 172.21.15.31:0/3063746300 >> 172.21.15.31:0/3063746300 conn(0x7fb4d0068030 msgr2=0x7fb4d00cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.806+0000 7fb4d87ff700 1 -- 172.21.15.31:0/3063746300 shutdown_connections 2023-10-06T05:26:36.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.806+0000 7fb4d87ff700 1 -- 172.21.15.31:0/3063746300 wait complete. 2023-10-06T05:26:36.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.807+0000 7fb4d87ff700 1 Processor -- start 2023-10-06T05:26:36.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.807+0000 7fb4d87ff700 1 -- start start 2023-10-06T05:26:36.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.808+0000 7fb4d87ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d006d830 0x7fb4d0165040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.808+0000 7fb4d87ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d006df30 0x7fb4d00c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.808+0000 7fb4d87ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4d0159d70 0x7fb4d00bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.808+0000 7fb4d87ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0165580 con 0x7fb4d006d830 2023-10-06T05:26:36.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.808+0000 7fb4d87ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0151b20 con 0x7fb4d006df30 2023-10-06T05:26:36.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.808+0000 7fb4d87ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0151e00 con 0x7fb4d0159d70 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.808+0000 7fb4d659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d006d830 0x7fb4d0165040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.808+0000 7fb4d659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d006d830 0x7fb4d0165040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59612/0 (socket says 172.21.15.31:59612) 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.808+0000 7fb4d659b700 1 -- 172.21.15.31:0/2245528676 learned_addr learned my addr 172.21.15.31:0/2245528676 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.809+0000 7fb4d6d9c700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4d0159d70 0x7fb4d00bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.809+0000 7fb4d5d9a700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d006df30 0x7fb4d00c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.809+0000 7fb4d659b700 1 -- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4d0159d70 msgr2=0x7fb4d00bfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.809+0000 7fb4d659b700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4d0159d70 0x7fb4d00bfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.809+0000 7fb4d659b700 1 -- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d006df30 msgr2=0x7fb4d00c2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.809+0000 7fb4d659b700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d006df30 0x7fb4d00c2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.809+0000 7fb4d659b700 1 -- 172.21.15.31:0/2245528676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc00b720 con 0x7fb4d006d830 2023-10-06T05:26:36.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.809+0000 7fb4d659b700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d006d830 0x7fb4d0165040 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc014000 tx=0x7fb4cc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.809+0000 7fb4c67fc700 1 -- 172.21.15.31:0/2245528676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc005c10 con 0x7fb4d006d830 2023-10-06T05:26:36.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.810+0000 7fb4c67fc700 1 -- 172.21.15.31:0/2245528676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc01d070 con 0x7fb4d006d830 2023-10-06T05:26:36.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.810+0000 7fb4c67fc700 1 -- 172.21.15.31:0/2245528676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0106e0 con 0x7fb4d006d830 2023-10-06T05:26:36.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.810+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d01520e0 con 0x7fb4d006d830 2023-10-06T05:26:36.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.810+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d0151710 con 0x7fb4d006d830 2023-10-06T05:26:36.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.811+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb4d0151900 con 0x7fb4d006d830 2023-10-06T05:26:36.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.812+0000 7fb4c67fc700 1 -- 172.21.15.31:0/2245528676 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fb4cc024440 con 0x7fb4d006d830 2023-10-06T05:26:36.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.812+0000 7fb4c67fc700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4b40679e0 0x7fb4b4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.813+0000 7fb4c67fc700 1 -- 172.21.15.31:0/2245528676 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fb4cc014030 con 0x7fb4d006d830 2023-10-06T05:26:36.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.813+0000 7fb4c67fc700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] conn(0x7fb4b406de50 0x7fb4b4070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:36.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.813+0000 7fb4c67fc700 1 -- 172.21.15.31:0/2245528676 --> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4b4070950 con 0x7fb4b406de50 2023-10-06T05:26:36.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.813+0000 7fb4c67fc700 1 -- 172.21.15.31:0/2245528676 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4cc08b870 con 0x7fb4d006d830 2023-10-06T05:26:36.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.813+0000 7fb4d5d9a700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4b40679e0 0x7fb4b4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.813+0000 7fb4d6d9c700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] conn(0x7fb4b406de50 0x7fb4b4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:36.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.814+0000 7fb4d5d9a700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4b40679e0 0x7fb4b4069e90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0153b40 tx=0x7fb4c0007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.814+0000 7fb4d6d9c700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] conn(0x7fb4b406de50 0x7fb4b4070280 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:36.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.815+0000 7fb4c67fc700 1 -- 172.21.15.31:0/2245528676 <== osd.6 v2:172.21.15.62:6821/115137 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb4b4070950 con 0x7fb4b406de50 2023-10-06T05:26:36.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.829+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 --> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb4d004c380 con 0x7fb4b406de50 2023-10-06T05:26:36.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.830+0000 7fb4c67fc700 1 -- 172.21.15.31:0/2245528676 <== osd.6 v2:172.21.15.62:6821/115137 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb4d004c380 con 0x7fb4b406de50 2023-10-06T05:26:36.834 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:26:36.834 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:26:36.834 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:26:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] conn(0x7fb4b406de50 msgr2=0x7fb4b4070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d87ff700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:6821/115137,v1:172.21.15.62:6823/115137] conn(0x7fb4b406de50 0x7fb4b4070280 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4b40679e0 msgr2=0x7fb4b4069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d87ff700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fb4b40679e0 0x7fb4b4069e90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0153b40 tx=0x7fb4c0007eb0 comp rx=0 tx=0).stop 2023-10-06T05:26:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d006d830 msgr2=0x7fb4d0165040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d87ff700 1 --2- 172.21.15.31:0/2245528676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4d006d830 0x7fb4d0165040 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc014000 tx=0x7fb4cc01a000 comp rx=0 tx=0).stop 2023-10-06T05:26:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d6d9c700 1 -- 172.21.15.31:0/2245528676 reap_dead start 2023-10-06T05:26:36.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 shutdown_connections 2023-10-06T05:26:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 >> 172.21.15.31:0/2245528676 conn(0x7fb4d0068030 msgr2=0x7fb4d00cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 shutdown_connections 2023-10-06T05:26:36.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:36.831+0000 7fb4d87ff700 1 -- 172.21.15.31:0/2245528676 wait complete. 2023-10-06T05:26:36.849 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T05:26:37.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.044+0000 7f18cb9ff700 1 Processor -- start 2023-10-06T05:26:37.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.044+0000 7f18cb9ff700 1 -- start start 2023-10-06T05:26:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.046+0000 7f18cb9ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18c40c7f30 0x7f18c4069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.046+0000 7f18cb9ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18c40aee70 0x7f18c413f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.046+0000 7f18cb9ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c40ca5b0 con 0x7f18c40c7b60 2023-10-06T05:26:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.046+0000 7f18cb9ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c40af2f0 con 0x7f18c40c7f30 2023-10-06T05:26:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.046+0000 7f18cb9ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c406f740 con 0x7f18c40aee70 2023-10-06T05:26:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.046+0000 7f18c8f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18c40c7f30 0x7f18c4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.046+0000 7f18c8f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18c40c7f30 0x7f18c4069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35944/0 (socket says 172.21.15.31:35944) 2023-10-06T05:26:37.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.046+0000 7f18c8f9a700 1 -- 172.21.15.31:0/4099199718 learned_addr learned my addr 172.21.15.31:0/4099199718 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:37.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.046+0000 7f18c9f9c700 1 --2- 172.21.15.31:0/4099199718 >> v2:172.21.15.62:3300/0 conn(0x7f18c40aee70 0x7f18c413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.047+0000 7f18c8f9a700 1 -- 172.21.15.31:0/4099199718 >> v2:172.21.15.62:3300/0 conn(0x7f18c40aee70 msgr2=0x7f18c413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:37.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.047+0000 7f18c8f9a700 1 --2- 172.21.15.31:0/4099199718 >> v2:172.21.15.62:3300/0 conn(0x7f18c40aee70 0x7f18c413f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.047+0000 7f18c8f9a700 1 -- 172.21.15.31:0/4099199718 >> v1:172.21.15.62:6789/0 conn(0x7f18c40c7b60 legacy=0x7f18c406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.047+0000 7f18c8f9a700 1 -- 172.21.15.31:0/4099199718 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c40c9c80 con 0x7f18c40c7f30 2023-10-06T05:26:37.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.047+0000 7f18c8f9a700 1 --2- 172.21.15.31:0/4099199718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 0x7f18c4069880 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f18c406d6e0 tx=0x7f18b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ade6794d38c90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:37.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.047+0000 7f18bbfff700 1 -- 172.21.15.31:0/4099199718 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4009e70 con 0x7f18c40c7f30 2023-10-06T05:26:37.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.047+0000 7f18bbfff700 1 -- 172.21.15.31:0/4099199718 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b400b860 con 0x7f18c40c7f30 2023-10-06T05:26:37.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.047+0000 7f18bbfff700 1 -- 172.21.15.31:0/4099199718 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b400bc00 con 0x7f18c40c7f30 2023-10-06T05:26:37.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.048+0000 7f18cb9ff700 1 -- 172.21.15.31:0/4099199718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 msgr2=0x7f18c4069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.048+0000 7f18cb9ff700 1 --2- 172.21.15.31:0/4099199718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 0x7f18c4069880 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f18c406d6e0 tx=0x7f18b4009250 comp rx=0 tx=0).stop 2023-10-06T05:26:37.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.048+0000 7f18cb9ff700 1 -- 172.21.15.31:0/4099199718 shutdown_connections 2023-10-06T05:26:37.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.048+0000 7f18cb9ff700 1 --2- 172.21.15.31:0/4099199718 >> v2:172.21.15.62:3300/0 conn(0x7f18c40aee70 0x7f18c413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.048+0000 7f18cb9ff700 1 --2- 172.21.15.31:0/4099199718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 0x7f18c4069880 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.048+0000 7f18cb9ff700 1 -- 172.21.15.31:0/4099199718 >> 172.21.15.31:0/4099199718 conn(0x7f18c40cc0b0 msgr2=0x7f18c40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:37.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.048+0000 7f18cb9ff700 1 -- 172.21.15.31:0/4099199718 shutdown_connections 2023-10-06T05:26:37.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.048+0000 7f18cb9ff700 1 -- 172.21.15.31:0/4099199718 wait complete. 2023-10-06T05:26:37.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.049+0000 7f18cb9ff700 1 Processor -- start 2023-10-06T05:26:37.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.049+0000 7f18cb9ff700 1 -- start start 2023-10-06T05:26:37.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.050+0000 7f18cb9ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c40aee70 0x7f18c40c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.050+0000 7f18cb9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c40c7b60 0x7f18c415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.050+0000 7f18cb9ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 0x7f18c415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.050+0000 7f18cb9ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c415c6f0 con 0x7f18c40c7f30 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.050+0000 7f18cb9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c415a9a0 con 0x7f18c40c7b60 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.050+0000 7f18cb9ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c415b7e0 con 0x7f18c40aee70 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c9f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 0x7f18c415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c9f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 0x7f18c415c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35960/0 (socket says 172.21.15.31:35960) 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c9f9c700 1 -- 172.21.15.31:0/1275423795 learned_addr learned my addr 172.21.15.31:0/1275423795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c8f9a700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c40c7b60 0x7f18c415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c979b700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c40aee70 0x7f18c40c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c9f9c700 1 -- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c40aee70 msgr2=0x7f18c40c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c9f9c700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c40aee70 0x7f18c40c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c9f9c700 1 -- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c40c7b60 msgr2=0x7f18c415bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c9f9c700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c40c7b60 0x7f18c415bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c9f9c700 1 -- 172.21.15.31:0/1275423795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b40094f0 con 0x7f18c40c7f30 2023-10-06T05:26:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c8f9a700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c40c7b60 0x7f18c415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c979b700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c40aee70 0x7f18c40c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.051+0000 7f18c9f9c700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 0x7f18c415c1b0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009e90 tx=0x7f18bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.052+0000 7f18b97fa700 1 -- 172.21.15.31:0/1275423795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00cc20 con 0x7f18c40c7f30 2023-10-06T05:26:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.052+0000 7f18b97fa700 1 -- 172.21.15.31:0/1275423795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc00f040 con 0x7f18c40c7f30 2023-10-06T05:26:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.052+0000 7f18b97fa700 1 -- 172.21.15.31:0/1275423795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc014480 con 0x7f18c40c7f30 2023-10-06T05:26:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.052+0000 7f18cb9ff700 1 -- 172.21.15.31:0/1275423795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c415abd0 con 0x7f18c40c7f30 2023-10-06T05:26:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.052+0000 7f18cb9ff700 1 -- 172.21.15.31:0/1275423795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c415b190 con 0x7f18c40c7f30 2023-10-06T05:26:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.053+0000 7f18cb9ff700 1 -- 172.21.15.31:0/1275423795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c415b950 con 0x7f18c40c7f30 2023-10-06T05:26:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.054+0000 7f18b97fa700 1 -- 172.21.15.31:0/1275423795 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f18bc0147f0 con 0x7f18c40c7f30 2023-10-06T05:26:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.055+0000 7f18b97fa700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f18a80679e0 0x7f18a8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.055+0000 7f18b97fa700 1 -- 172.21.15.31:0/1275423795 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f18bc083e30 con 0x7f18c40c7f30 2023-10-06T05:26:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.055+0000 7f18c979b700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f18a80679e0 0x7f18a8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.056+0000 7f18c979b700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f18a80679e0 0x7f18a8069e90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f18c416ce10 tx=0x7f18c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.059+0000 7f18b97fa700 1 -- 172.21.15.31:0/1275423795 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18bc052800 con 0x7f18c40c7f30 2023-10-06T05:26:37.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.265+0000 7f18cb9ff700 1 -- 172.21.15.31:0/1275423795 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f18c404c380 con 0x7f18a80679e0 2023-10-06T05:26:37.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.265+0000 7f18b97fa700 1 -- 172.21.15.31:0/1275423795 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f18c404c380 con 0x7f18a80679e0 2023-10-06T05:26:37.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 -- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f18a80679e0 msgr2=0x7f18a8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f18a80679e0 0x7f18a8069e90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f18c416ce10 tx=0x7f18c0009040 comp rx=0 tx=0).stop 2023-10-06T05:26:37.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 -- 172.21.15.31:0/1275423795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 msgr2=0x7f18c415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 0x7f18c415c1b0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009e90 tx=0x7f18bc00b040 comp rx=0 tx=0).stop 2023-10-06T05:26:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 -- 172.21.15.31:0/1275423795 shutdown_connections 2023-10-06T05:26:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f18a80679e0 0x7f18a8069e90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c40c7f30 0x7f18c415c1b0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18c40aee70 0x7f18c40c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 --2- 172.21.15.31:0/1275423795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18c40c7b60 0x7f18c415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 -- 172.21.15.31:0/1275423795 >> 172.21.15.31:0/1275423795 conn(0x7f18c40cc0b0 msgr2=0x7f18c40afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 -- 172.21.15.31:0/1275423795 shutdown_connections 2023-10-06T05:26:37.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.268+0000 7f18cb9ff700 1 -- 172.21.15.31:0/1275423795 wait complete. 2023-10-06T05:26:37.272 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T05:26:37.287 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T05:26:37.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:26:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.484+0000 7fc2151ff700 1 Processor -- start 2023-10-06T05:26:37.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.484+0000 7fc2151ff700 1 -- start start 2023-10-06T05:26:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.486+0000 7fc2151ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc21015a990 0x7fc2100cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.486+0000 7fc2151ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2100cc5b0 con 0x7fc210142c20 2023-10-06T05:26:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.486+0000 7fc2151ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21015d210 con 0x7fc210142ff0 2023-10-06T05:26:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.486+0000 7fc2151ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2100cc810 con 0x7fc21015a990 2023-10-06T05:26:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.486+0000 7fc20ed9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc210142c20 0x7fc2100cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45178/0 (socket says 172.21.15.31:45178) 2023-10-06T05:26:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.486+0000 7fc20ed9d700 1 -- 172.21.15.31:0/2033151504 learned_addr learned my addr 172.21.15.31:0/2033151504 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-10-06T05:26:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.486+0000 7fc20e59c700 1 --2- 172.21.15.31:0/2033151504 >> v2:172.21.15.62:3300/0 conn(0x7fc21015a990 0x7fc2100cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.486+0000 7fc20dd9b700 1 -- 172.21.15.31:0/2033151504 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752549000 0 0) 0x7fc2100cc5b0 con 0x7fc210142c20 2023-10-06T05:26:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.486+0000 7fc20dd9b700 1 -- 172.21.15.31:0/2033151504 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f8003570 con 0x7fc210142c20 2023-10-06T05:26:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.487+0000 7fc20e59c700 1 -- 172.21.15.31:0/2033151504 >> v1:172.21.15.62:6789/0 conn(0x7fc210142ff0 legacy=0x7fc210068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.487+0000 7fc20dd9b700 1 -- 172.21.15.31:0/2033151504 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1147513235 0 0) 0x7fc1f8003570 con 0x7fc210142c20 2023-10-06T05:26:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.487+0000 7fc20e59c700 1 -- 172.21.15.31:0/2033151504 >> v1:172.21.15.31:6789/0 conn(0x7fc210142c20 legacy=0x7fc2100cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.487+0000 7fc20e59c700 1 -- 172.21.15.31:0/2033151504 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21015d4b0 con 0x7fc21015a990 2023-10-06T05:26:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.487+0000 7fc20e59c700 1 --2- 172.21.15.31:0/2033151504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc21015a990 0x7fc2100cacd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc2040098e0 tx=0x7fc2040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4282f02dbac20079 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.487+0000 7fc20dd9b700 1 -- 172.21.15.31:0/2033151504 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200001470 con 0x7fc21015a990 2023-10-06T05:26:37.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.487+0000 7fc20dd9b700 1 -- 172.21.15.31:0/2033151504 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc204011420 con 0x7fc21015a990 2023-10-06T05:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.487+0000 7fc20dd9b700 1 -- 172.21.15.31:0/2033151504 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2040117a0 con 0x7fc21015a990 2023-10-06T05:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.488+0000 7fc2151ff700 1 -- 172.21.15.31:0/2033151504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc21015a990 msgr2=0x7fc2100cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.488+0000 7fc2151ff700 1 --2- 172.21.15.31:0/2033151504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc21015a990 0x7fc2100cacd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc2040098e0 tx=0x7fc2040091a0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.488+0000 7fc2151ff700 1 -- 172.21.15.31:0/2033151504 shutdown_connections 2023-10-06T05:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.488+0000 7fc2151ff700 1 --2- 172.21.15.31:0/2033151504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc21015a990 0x7fc2100cacd0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.488+0000 7fc2151ff700 1 -- 172.21.15.31:0/2033151504 >> 172.21.15.31:0/2033151504 conn(0x7fc2100ce2d0 msgr2=0x7fc2100ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.488+0000 7fc2151ff700 1 -- 172.21.15.31:0/2033151504 shutdown_connections 2023-10-06T05:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.488+0000 7fc2151ff700 1 -- 172.21.15.31:0/2033151504 wait complete. 2023-10-06T05:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.489+0000 7fc2151ff700 1 Processor -- start 2023-10-06T05:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.490+0000 7fc2151ff700 1 -- start start 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc2151ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc210142c20 0x7fc21013efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc2151ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc210142ff0 0x7fc2100b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc2151ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc21015a990 0x7fc21015f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc2151ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21013f4e0 con 0x7fc21015a990 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc2151ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100aef40 con 0x7fc210142ff0 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc2151ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100ae330 con 0x7fc210142c20 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc20f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc21015a990 0x7fc21015f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc20ed9d700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc210142c20 0x7fc21013efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc20e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc210142ff0 0x7fc2100b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc20f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc21015a990 0x7fc21015f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35968/0 (socket says 172.21.15.31:35968) 2023-10-06T05:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc20f59e700 1 -- 172.21.15.31:0/3688527480 learned_addr learned my addr 172.21.15.31:0/3688527480 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc20f59e700 1 -- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc210142c20 msgr2=0x7fc21013efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc20f59e700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc210142c20 0x7fc21013efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc20f59e700 1 -- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc210142ff0 msgr2=0x7fc2100b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc20f59e700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc210142ff0 0x7fc2100b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.491+0000 7fc20f59e700 1 -- 172.21.15.31:0/3688527480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc204009420 con 0x7fc21015a990 2023-10-06T05:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.492+0000 7fc20f59e700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc21015a990 0x7fc21015f290 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fc20800c980 tx=0x7fc208001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.492+0000 7fc1f6ffd700 1 -- 172.21.15.31:0/3688527480 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208012630 con 0x7fc21015a990 2023-10-06T05:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.492+0000 7fc1f6ffd700 1 -- 172.21.15.31:0/3688527480 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208012790 con 0x7fc21015a990 2023-10-06T05:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.492+0000 7fc1f6ffd700 1 -- 172.21.15.31:0/3688527480 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208012b50 con 0x7fc21015a990 2023-10-06T05:26:37.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.492+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2100ae610 con 0x7fc21015a990 2023-10-06T05:26:37.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.492+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2100aebd0 con 0x7fc21015a990 2023-10-06T05:26:37.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.493+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc21004c380 con 0x7fc21015a990 2023-10-06T05:26:37.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.494+0000 7fc1f6ffd700 1 -- 172.21.15.31:0/3688527480 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fc208003030 con 0x7fc21015a990 2023-10-06T05:26:37.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.495+0000 7fc1f6ffd700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc1fc0679e0 0x7fc1fc069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.495+0000 7fc1f6ffd700 1 -- 172.21.15.31:0/3688527480 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fc2080830d0 con 0x7fc21015a990 2023-10-06T05:26:37.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.496+0000 7fc1f6ffd700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] conn(0x7fc1fc06de50 0x7fc1fc070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.496+0000 7fc1f6ffd700 1 -- 172.21.15.31:0/3688527480 --> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1fc070950 con 0x7fc1fc06de50 2023-10-06T05:26:37.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.496+0000 7fc1f6ffd700 1 -- 172.21.15.31:0/3688527480 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc208087db0 con 0x7fc21015a990 2023-10-06T05:26:37.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.496+0000 7fc20ed9d700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc1fc0679e0 0x7fc1fc069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.496+0000 7fc20e59c700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] conn(0x7fc1fc06de50 0x7fc1fc070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.497+0000 7fc20ed9d700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc1fc0679e0 0x7fc1fc069e90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc200004dc0 tx=0x7fc200000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:37.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.497+0000 7fc20e59c700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] conn(0x7fc1fc06de50 0x7fc1fc070280 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:37.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.498+0000 7fc1f6ffd700 1 -- 172.21.15.31:0/3688527480 <== osd.7 v2:172.21.15.62:6808/115119 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fc1fc070950 con 0x7fc1fc06de50 2023-10-06T05:26:37.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.512+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 --> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc21015d790 con 0x7fc1fc06de50 2023-10-06T05:26:37.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.513+0000 7fc1f6ffd700 1 -- 172.21.15.31:0/3688527480 <== osd.7 v2:172.21.15.62:6808/115119 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc21015d790 con 0x7fc1fc06de50 2023-10-06T05:26:37.515 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-10-06T05:26:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:26:37.515 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-10-06T05:26:37.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.513+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] conn(0x7fc1fc06de50 msgr2=0x7fc1fc070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.513+0000 7fc2151ff700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:6808/115119,v1:172.21.15.62:6809/115119] conn(0x7fc1fc06de50 0x7fc1fc070280 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.513+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc1fc0679e0 msgr2=0x7fc1fc069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.513+0000 7fc2151ff700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fc1fc0679e0 0x7fc1fc069e90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc200004dc0 tx=0x7fc200000bc0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.513+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc21015a990 msgr2=0x7fc21015f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.513+0000 7fc2151ff700 1 --2- 172.21.15.31:0/3688527480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc21015a990 0x7fc21015f290 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fc20800c980 tx=0x7fc208001770 comp rx=0 tx=0).stop 2023-10-06T05:26:37.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.513+0000 7fc20f59e700 1 -- 172.21.15.31:0/3688527480 reap_dead start 2023-10-06T05:26:37.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.514+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 shutdown_connections 2023-10-06T05:26:37.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.514+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 >> 172.21.15.31:0/3688527480 conn(0x7fc2100ce2d0 msgr2=0x7fc2100ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:37.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.514+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 shutdown_connections 2023-10-06T05:26:37.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.514+0000 7fc2151ff700 1 -- 172.21.15.31:0/3688527480 wait complete. 2023-10-06T05:26:37.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T05:26:37.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.726+0000 7fd8479ff700 1 Processor -- start 2023-10-06T05:26:37.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.726+0000 7fd8479ff700 1 -- start start 2023-10-06T05:26:37.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd8479ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84013a1e0 0x7fd8400c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd8479ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd84013a5b0 0x7fd84006d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd8479ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8400c6c30 0x7fd84006d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd8479ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8400c71b0 con 0x7fd84013a5b0 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd8479ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8400c7490 con 0x7fd8400c6c30 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd8479ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8400c7770 con 0x7fd84013a1e0 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd844f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd84013a5b0 0x7fd84006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd844f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd84013a5b0 0x7fd84006d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35982/0 (socket says 172.21.15.31:35982) 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd844f9a700 1 -- 172.21.15.31:0/3938341822 learned_addr learned my addr 172.21.15.31:0/3938341822 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd845f9c700 1 --2- 172.21.15.31:0/3938341822 >> v2:172.21.15.62:3300/0 conn(0x7fd8400c6c30 0x7fd84006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd84579b700 1 --2- 172.21.15.31:0/3938341822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84013a1e0 0x7fd8400c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd844f9a700 1 -- 172.21.15.31:0/3938341822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84013a1e0 msgr2=0x7fd8400c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:37.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd844f9a700 1 --2- 172.21.15.31:0/3938341822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84013a1e0 0x7fd8400c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd844f9a700 1 -- 172.21.15.31:0/3938341822 >> v2:172.21.15.62:3300/0 conn(0x7fd8400c6c30 msgr2=0x7fd84006d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:37.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.728+0000 7fd844f9a700 1 --2- 172.21.15.31:0/3938341822 >> v2:172.21.15.62:3300/0 conn(0x7fd8400c6c30 0x7fd84006d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.729+0000 7fd844f9a700 1 -- 172.21.15.31:0/3938341822 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8400c7a50 con 0x7fd84013a5b0 2023-10-06T05:26:37.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.729+0000 7fd844f9a700 1 --2- 172.21.15.31:0/3938341822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd84013a5b0 0x7fd84006d440 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fd84006b530 tx=0x7fd830009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ac92f7211839bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:37.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.729+0000 7fd837fff700 1 -- 172.21.15.31:0/3938341822 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830009e70 con 0x7fd84013a5b0 2023-10-06T05:26:37.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.729+0000 7fd837fff700 1 -- 172.21.15.31:0/3938341822 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83000b860 con 0x7fd84013a5b0 2023-10-06T05:26:37.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.729+0000 7fd837fff700 1 -- 172.21.15.31:0/3938341822 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83000bc00 con 0x7fd84013a5b0 2023-10-06T05:26:37.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.730+0000 7fd8479ff700 1 -- 172.21.15.31:0/3938341822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd84013a5b0 msgr2=0x7fd84006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.730+0000 7fd8479ff700 1 --2- 172.21.15.31:0/3938341822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd84013a5b0 0x7fd84006d440 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fd84006b530 tx=0x7fd830009250 comp rx=0 tx=0).stop 2023-10-06T05:26:37.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.730+0000 7fd8479ff700 1 -- 172.21.15.31:0/3938341822 shutdown_connections 2023-10-06T05:26:37.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.730+0000 7fd8479ff700 1 --2- 172.21.15.31:0/3938341822 >> v2:172.21.15.62:3300/0 conn(0x7fd8400c6c30 0x7fd84006d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.730+0000 7fd8479ff700 1 --2- 172.21.15.31:0/3938341822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84013a1e0 0x7fd8400c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.730+0000 7fd8479ff700 1 --2- 172.21.15.31:0/3938341822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd84013a5b0 0x7fd84006d440 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.730+0000 7fd8479ff700 1 -- 172.21.15.31:0/3938341822 >> 172.21.15.31:0/3938341822 conn(0x7fd8400cc0b0 msgr2=0x7fd8400cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:37.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.730+0000 7fd8479ff700 1 -- 172.21.15.31:0/3938341822 shutdown_connections 2023-10-06T05:26:37.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.730+0000 7fd8479ff700 1 -- 172.21.15.31:0/3938341822 wait complete. 2023-10-06T05:26:37.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.731+0000 7fd8479ff700 1 Processor -- start 2023-10-06T05:26:37.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.731+0000 7fd8479ff700 1 -- start start 2023-10-06T05:26:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.732+0000 7fd8479ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8400c6c30 0x7fd84014fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.732+0000 7fd8479ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd84013a1e0 0x7fd840150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.732+0000 7fd8479ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84013a5b0 0x7fd840151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.732+0000 7fd8479ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84013c460 con 0x7fd8400c6c30 2023-10-06T05:26:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.732+0000 7fd8479ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84013bb30 con 0x7fd84013a1e0 2023-10-06T05:26:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.732+0000 7fd8479ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84013be10 con 0x7fd84013a5b0 2023-10-06T05:26:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd84579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8400c6c30 0x7fd84014fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd84579b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8400c6c30 0x7fd84014fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35984/0 (socket says 172.21.15.31:35984) 2023-10-06T05:26:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd84579b700 1 -- 172.21.15.31:0/2275620212 learned_addr learned my addr 172.21.15.31:0/2275620212 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd844f9a700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd84013a1e0 0x7fd840150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd845f9c700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84013a5b0 0x7fd840151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd84579b700 1 -- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84013a5b0 msgr2=0x7fd840151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd84579b700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84013a5b0 0x7fd840151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd84579b700 1 -- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd84013a1e0 msgr2=0x7fd840150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd84579b700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd84013a1e0 0x7fd840150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd84579b700 1 -- 172.21.15.31:0/2275620212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8300094f0 con 0x7fd8400c6c30 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.733+0000 7fd84579b700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8400c6c30 0x7fd84014fe90 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00d810 tx=0x7fd83c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.734+0000 7fd8357fa700 1 -- 172.21.15.31:0/2275620212 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c0125f0 con 0x7fd8400c6c30 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.734+0000 7fd8357fa700 1 -- 172.21.15.31:0/2275620212 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c012750 con 0x7fd8400c6c30 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.734+0000 7fd8357fa700 1 -- 172.21.15.31:0/2275620212 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c012b10 con 0x7fd8400c6c30 2023-10-06T05:26:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.734+0000 7fd8479ff700 1 -- 172.21.15.31:0/2275620212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84013c0f0 con 0x7fd8400c6c30 2023-10-06T05:26:37.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.734+0000 7fd8479ff700 1 -- 172.21.15.31:0/2275620212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd840150610 con 0x7fd8400c6c30 2023-10-06T05:26:37.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.735+0000 7fd8479ff700 1 -- 172.21.15.31:0/2275620212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84004c380 con 0x7fd8400c6c30 2023-10-06T05:26:37.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.736+0000 7fd8357fa700 1 -- 172.21.15.31:0/2275620212 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7fd83c01e460 con 0x7fd8400c6c30 2023-10-06T05:26:37.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.736+0000 7fd8357fa700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd8240679e0 0x7fd824069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:37.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.737+0000 7fd8357fa700 1 -- 172.21.15.31:0/2275620212 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fd83c083450 con 0x7fd8400c6c30 2023-10-06T05:26:37.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.737+0000 7fd844f9a700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd8240679e0 0x7fd824069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:37.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.738+0000 7fd844f9a700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd8240679e0 0x7fd824069e90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd84006b530 tx=0x7fd830017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:37.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.740+0000 7fd8357fa700 1 -- 172.21.15.31:0/2275620212 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd83c052090 con 0x7fd8400c6c30 2023-10-06T05:26:37.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.941+0000 7fd8479ff700 1 -- 172.21.15.31:0/2275620212 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd84013c5a0 con 0x7fd8240679e0 2023-10-06T05:26:37.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.942+0000 7fd8357fa700 1 -- 172.21.15.31:0/2275620212 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd830000c50 con 0x7fd8240679e0 2023-10-06T05:26:37.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 -- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd8240679e0 msgr2=0x7fd824069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd8240679e0 0x7fd824069e90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd84006b530 tx=0x7fd830017000 comp rx=0 tx=0).stop 2023-10-06T05:26:37.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 -- 172.21.15.31:0/2275620212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8400c6c30 msgr2=0x7fd84014fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:37.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8400c6c30 0x7fd84014fe90 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00d810 tx=0x7fd83c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:37.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 -- 172.21.15.31:0/2275620212 shutdown_connections 2023-10-06T05:26:37.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7fd8240679e0 0x7fd824069e90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd84013a5b0 0x7fd840151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8400c6c30 0x7fd84014fe90 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 --2- 172.21.15.31:0/2275620212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd84013a1e0 0x7fd840150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:37.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 -- 172.21.15.31:0/2275620212 >> 172.21.15.31:0/2275620212 conn(0x7fd8400cc0b0 msgr2=0x7fd840139500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:37.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 -- 172.21.15.31:0/2275620212 shutdown_connections 2023-10-06T05:26:37.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:37.945+0000 7fd8479ff700 1 -- 172.21.15.31:0/2275620212 wait complete. 2023-10-06T05:26:37.948 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T05:26:37.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:26:38.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.161+0000 7f394e7ff700 1 Processor -- start 2023-10-06T05:26:38.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.162+0000 7f394e7ff700 1 -- start start 2023-10-06T05:26:38.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.163+0000 7f394e7ff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39480c7b60 0x7f3948069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.163+0000 7f394e7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7f30 0x7f39480aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.163+0000 7f394e7ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39480ca5b0 con 0x7f394806ef50 2023-10-06T05:26:38.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.163+0000 7f394e7ff700 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394806f740 con 0x7f39480c7b60 2023-10-06T05:26:38.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.163+0000 7f394e7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39480c9c80 con 0x7f39480c7f30 2023-10-06T05:26:38.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.163+0000 7f3947fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39480c7b60 0x7f3948069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.163+0000 7f3947fff700 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39480c7b60 0x7f3948069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.31:35662/0 (socket says 172.21.15.31:35662) 2023-10-06T05:26:38.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.163+0000 7f39477fe700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7f30 0x7f39480aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.163+0000 7f3947fff700 1 -- 172.21.15.31:0/1640071719 learned_addr learned my addr 172.21.15.31:0/1640071719 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:38.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.164+0000 7f3947fff700 1 -- 172.21.15.31:0/1640071719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7f30 msgr2=0x7f39480aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:38.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.164+0000 7f3947fff700 1 --2- 172.21.15.31:0/1640071719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7f30 0x7f39480aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.164+0000 7f3947fff700 1 -- 172.21.15.31:0/1640071719 >> v1:172.21.15.62:6789/0 conn(0x7f394806ef50 legacy=0x7f394813f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.164+0000 7f3947fff700 1 -- 172.21.15.31:0/1640071719 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39480c9f60 con 0x7f39480c7b60 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.164+0000 7f39477fe700 1 --2- 172.21.15.31:0/1640071719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7f30 0x7f39480aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.164+0000 7f3947fff700 1 --2- 172.21.15.31:0/1640071719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39480c7b60 0x7f3948069880 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3938009990 tx=0x7f3938009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7905350fa5aa4b02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.164+0000 7f3946ffd700 1 -- 172.21.15.31:0/1640071719 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938009e80 con 0x7f39480c7b60 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.165+0000 7f3946ffd700 1 -- 172.21.15.31:0/1640071719 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938005c10 con 0x7f39480c7b60 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.165+0000 7f3946ffd700 1 -- 172.21.15.31:0/1640071719 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800b650 con 0x7f39480c7b60 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.165+0000 7f394e7ff700 1 -- 172.21.15.31:0/1640071719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39480c7b60 msgr2=0x7f3948069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.165+0000 7f394e7ff700 1 --2- 172.21.15.31:0/1640071719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39480c7b60 0x7f3948069880 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3938009990 tx=0x7f3938009220 comp rx=0 tx=0).stop 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.165+0000 7f394e7ff700 1 -- 172.21.15.31:0/1640071719 shutdown_connections 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.165+0000 7f394e7ff700 1 --2- 172.21.15.31:0/1640071719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39480c7b60 0x7f3948069880 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.165+0000 7f394e7ff700 1 --2- 172.21.15.31:0/1640071719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7f30 0x7f39480aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.165+0000 7f394e7ff700 1 -- 172.21.15.31:0/1640071719 >> 172.21.15.31:0/1640071719 conn(0x7f39480cc0b0 msgr2=0x7f39480cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:38.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.165+0000 7f394e7ff700 1 -- 172.21.15.31:0/1640071719 shutdown_connections 2023-10-06T05:26:38.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.165+0000 7f394e7ff700 1 -- 172.21.15.31:0/1640071719 wait complete. 2023-10-06T05:26:38.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.167+0000 7f394e7ff700 1 Processor -- start 2023-10-06T05:26:38.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.167+0000 7f394e7ff700 1 -- start start 2023-10-06T05:26:38.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f394e7ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f394806ef50 0x7f394816a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f394e7ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7b60 0x7f39480c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f394e7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39480c7f30 0x7f39481565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f394e7ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394816a6b0 con 0x7f394806ef50 2023-10-06T05:26:38.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f394e7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39480c3810 con 0x7f39480c7f30 2023-10-06T05:26:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f394e7ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39480c3200 con 0x7f39480c7b60 2023-10-06T05:26:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f3947fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f394806ef50 0x7f394816a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f3947fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f394806ef50 0x7f394816a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35992/0 (socket says 172.21.15.31:35992) 2023-10-06T05:26:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f3947fff700 1 -- 172.21.15.31:0/548199373 learned_addr learned my addr 172.21.15.31:0/548199373 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f39477fe700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7b60 0x7f39480c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f394cd9c700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39480c7f30 0x7f39481565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.168+0000 7f3947fff700 1 -- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7b60 msgr2=0x7f39480c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.169+0000 7f3947fff700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7b60 0x7f39480c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.169+0000 7f3947fff700 1 -- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39480c7f30 msgr2=0x7f39481565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.169+0000 7f3947fff700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39480c7f30 0x7f39481565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.169+0000 7f3947fff700 1 -- 172.21.15.31:0/548199373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c009420 con 0x7f394806ef50 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.169+0000 7f3947fff700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f394806ef50 0x7f394816a170 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f393800b3c0 tx=0x7f3938016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.169+0000 7f392ffff700 1 -- 172.21.15.31:0/548199373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938011ae0 con 0x7f394806ef50 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.169+0000 7f392ffff700 1 -- 172.21.15.31:0/548199373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938020070 con 0x7f394806ef50 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.169+0000 7f392ffff700 1 -- 172.21.15.31:0/548199373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938011e70 con 0x7f394806ef50 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.169+0000 7f394e7ff700 1 -- 172.21.15.31:0/548199373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3938009480 con 0x7f394806ef50 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.169+0000 7f394e7ff700 1 -- 172.21.15.31:0/548199373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39480c34a0 con 0x7f394806ef50 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.170+0000 7f394e7ff700 1 -- 172.21.15.31:0/548199373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394804c380 con 0x7f394806ef50 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.171+0000 7f392ffff700 1 -- 172.21.15.31:0/548199373 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f3938022030 con 0x7f394806ef50 2023-10-06T05:26:38.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.172+0000 7f392ffff700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f39340679e0 0x7f3934069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:38.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.172+0000 7f392ffff700 1 -- 172.21.15.31:0/548199373 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f3938084d40 con 0x7f394806ef50 2023-10-06T05:26:38.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.172+0000 7f39477fe700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f39340679e0 0x7f3934069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:38.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.173+0000 7f39477fe700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f39340679e0 0x7f3934069e90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f393c0093f0 tx=0x7f393c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:38.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.176+0000 7f392ffff700 1 -- 172.21.15.31:0/548199373 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3938053750 con 0x7f394806ef50 2023-10-06T05:26:38.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.371+0000 7f394e7ff700 1 -- 172.21.15.31:0/548199373 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39480cab70 con 0x7f39340679e0 2023-10-06T05:26:38.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.375+0000 7f392ffff700 1 -- 172.21.15.31:0/548199373 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72799 (secure 0 0 0) 0x7f39480cab70 con 0x7f39340679e0 2023-10-06T05:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:38.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 -- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f39340679e0 msgr2=0x7f3934069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:38.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f39340679e0 0x7f3934069e90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f393c0093f0 tx=0x7f393c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:38.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 -- 172.21.15.31:0/548199373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f394806ef50 msgr2=0x7f394816a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:38.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f394806ef50 0x7f394816a170 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f393800b3c0 tx=0x7f3938016000 comp rx=0 tx=0).stop 2023-10-06T05:26:38.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 -- 172.21.15.31:0/548199373 shutdown_connections 2023-10-06T05:26:38.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f39340679e0 0x7f3934069e90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39480c7f30 0x7f39481565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f394806ef50 0x7f394816a170 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 --2- 172.21.15.31:0/548199373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39480c7b60 0x7f39480c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:38.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 -- 172.21.15.31:0/548199373 >> 172.21.15.31:0/548199373 conn(0x7f39480cc0b0 msgr2=0x7f394806e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:38.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 -- 172.21.15.31:0/548199373 shutdown_connections 2023-10-06T05:26:38.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:38.377+0000 7f394e7ff700 1 -- 172.21.15.31:0/548199373 wait complete. 2023-10-06T05:26:38.380 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2023-10-06T05:26:37.535352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428680,"kb_used_data":17608,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546040,"statfs":{"total":773094113280,"available":770607144960,"internally_reserved":0,"allocated":18030592,"data_stored":9026992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.284794"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":129,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T05:26:26.838946+0000","last_change":"2023-10-06T05:16:55.392137+0000","last_active":"2023-10-06T05:26:26.838946+0000","last_peered":"2023-10-06T05:26:26.838946+0000","last_clean":"2023-10-06T05:26:26.838946+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:26:26.838946+0000","last_undegraded":"2023-10-06T05:26:26.838946+0000","last_fullsized":"2023-10-06T05:26:26.838946+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":129,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T05:26:26.837862+0000","last_change":"2023-10-06T05:16:55.820587+0000","last_active":"2023-10-06T05:26:26.837862+0000","last_peered":"2023-10-06T05:26:26.837862+0000","last_clean":"2023-10-06T05:26:26.837862+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:26:26.837862+0000","last_undegraded":"2023-10-06T05:26:26.837862+0000","last_fullsized":"2023-10-06T05:26:26.837862+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-06T05:26:24.764648+0000","last_change":"2023-10-06T05:16:55.820637+0000","last_active":"2023-10-06T05:26:24.764648+0000","last_peered":"2023-10-06T05:26:24.764648+0000","last_clean":"2023-10-06T05:26:24.764648+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:26:24.764648+0000","last_undegraded":"2023-10-06T05:26:24.764648+0000","last_fullsized":"2023-10-06T05:26:24.764648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:03.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":128,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T05:26:27.174892+0000","last_change":"2023-10-06T05:16:55.820491+0000","last_active":"2023-10-06T05:26:27.174892+0000","last_peered":"2023-10-06T05:26:27.174892+0000","last_clean":"2023-10-06T05:26:27.174892+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:26:27.174892+0000","last_undegraded":"2023-10-06T05:26:27.174892+0000","last_fullsized":"2023-10-06T05:26:27.174892+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:56.029674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":128,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T05:26:27.174959+0000","last_change":"2023-10-06T05:16:55.820496+0000","last_active":"2023-10-06T05:26:27.174959+0000","last_peered":"2023-10-06T05:26:27.174959+0000","last_clean":"2023-10-06T05:26:27.174959+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:26:27.174959+0000","last_undegraded":"2023-10-06T05:26:27.174959+0000","last_fullsized":"2023-10-06T05:26:27.174959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:57.964243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"72'128","reported_seq":317,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T05:26:27.174780+0000","last_change":"2023-10-06T05:16:55.392880+0000","last_active":"2023-10-06T05:26:27.174780+0000","last_peered":"2023-10-06T05:26:27.174780+0000","last_clean":"2023-10-06T05:26:27.174780+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:26:27.174780+0000","last_undegraded":"2023-10-06T05:26:27.174780+0000","last_fullsized":"2023-10-06T05:26:27.174780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:47.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":154,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:34.953394+0000","last_change":"2023-10-06T05:26:34.953368+0000","last_active":"2023-10-06T05:26:34.953394+0000","last_peered":"2023-10-06T05:26:34.953394+0000","last_clean":"2023-10-06T05:26:34.953394+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:26:34.953394+0000","last_undegraded":"2023-10-06T05:26:34.953394+0000","last_fullsized":"2023-10-06T05:26:34.953394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:34.953281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:34.953281+0000","last_clean_scrub_stamp":"2023-10-06T05:26:34.953281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:38.499866+0000","scrub_duration":0.0023260400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":170,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:36.253461+0000","last_change":"2023-10-06T05:26:36.253437+0000","last_active":"2023-10-06T05:26:36.253461+0000","last_peered":"2023-10-06T05:26:36.253461+0000","last_clean":"2023-10-06T05:26:36.253461+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:26:36.253461+0000","last_undegraded":"2023-10-06T05:26:36.253461+0000","last_fullsized":"2023-10-06T05:26:36.253461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:36.253357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:36.253357+0000","last_clean_scrub_stamp":"2023-10-06T05:26:36.253357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:51.646685+0000","scrub_duration":0.002185513,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":146,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:27.253088+0000","last_change":"2023-10-06T05:16:55.892234+0000","last_active":"2023-10-06T05:26:27.253088+0000","last_peered":"2023-10-06T05:26:27.253088+0000","last_clean":"2023-10-06T05:26:27.253088+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:26:27.253088+0000","last_undegraded":"2023-10-06T05:26:27.253088+0000","last_fullsized":"2023-10-06T05:26:27.253088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:16:54.371947+0000","last_clean_scrub_stamp":"2023-10-06T05:16:54.371947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304020,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067820,"statfs":{"total":96636764160,"available":96325447680,"internally_reserved":0,"allocated":2699264,"data_stored":1571150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68999999999999995,"15min":0.68899999999999995},"min":{"1min":0.47699999999999998,"5min":0.375,"15min":0.36099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98399999999999999,"5min":1.081,"15min":1.081},"last":0.71199999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.46300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.873,"5min":0.999,"15min":1.0109999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.73299999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89400000000000002,"5min":0.91700000000000004,"15min":1.018},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68400000000000005,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.41699999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.056},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.44800000000000001},"max":{"1min":1.002,"5min":1.149,"15min":1.149},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.66300000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0249999999999999},"last":0.878},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.27,"15min":1.27},"last":0.80700000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96699999999999997,"5min":0.97899999999999998,"15min":0.999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.80000000000000004},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.84999999999999998},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.1040000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79800000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.441},"max":{"1min":1.0600000000000001,"5min":1.248,"15min":1.248},"last":0.75}]}]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.38500000000000001,"15min":0.32300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.8500000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66300000000000003,"15min":0.69399999999999995},"min":{"1min":0.46400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.63300000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.45100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93100000000000005,"5min":1.177,"15min":1.9279999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85299999999999998,"5min":1.073,"15min":1.4239999999999999},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82799999999999996,"5min":1.093,"15min":1.369},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0549999999999999,"5min":1.085,"15min":1.891},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.51000000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99399999999999999,"5min":1.018,"15min":1.347},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.59999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96699999999999997,"5min":0.98699999999999999,"15min":1},"last":0.79500000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.753},"min":{"1min":0.46700000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1060000000000001,"5min":1.1359999999999999,"15min":1.9670000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79000000000000004,"15min":0.80500000000000005},"min":{"1min":0.49399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0600000000000001,"5min":1.278,"15min":1.4510000000000001},"last":0.81899999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.46700000000000003,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":1.03,"5min":1.0680000000000001,"15min":1.415},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1439999999999999,"5min":1.2350000000000001,"15min":2},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.60299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.071,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.877},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.60999999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0880000000000001,"5min":1.133,"15min":1.46},"last":1.0600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304020,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067820,"statfs":{"total":96636764160,"available":96325447680,"internally_reserved":0,"allocated":2699264,"data_stored":1571150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.75800000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45200000000000001,"15min":0.48299999999999998},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53000000000000003,"5min":0.60099999999999998,"15min":0.82099999999999995},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"max":{"1min":0.58899999999999997,"5min":0.71399999999999997,"15min":0.83799999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.32400000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.753,"15min":0.78700000000000003},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.59199999999999997,"5min":0.78400000000000003,"15min":0.92400000000000004},"last":0.502},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68799999999999994,"5min":0.68999999999999995,"15min":0.88600000000000001},"last":0.48599999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.78300000000000003},"min":{"1min":0.60899999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89900000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.623},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.55600000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91700000000000004,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71899999999999997,"15min":0.74299999999999999},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.97599999999999998,"5min":1.169,"15min":1.169},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.80400000000000005},"min":{"1min":0.56699999999999995,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":0.86199999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71299999999999997,"15min":0.74399999999999999},"min":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88800000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.79400000000000004},"min":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.86799999999999999,"5min":1.387,"15min":1.387},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77000000000000002,"15min":0.80800000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.50600000000000001},"max":{"1min":0.93500000000000005,"5min":1.244,"15min":1.244},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78300000000000003,"15min":0.81599999999999995},"min":{"1min":0.58699999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":0.88800000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.76700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41099999999999998,"15min":0.42699999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.84499999999999997},"last":0.375},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.47699999999999998},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59699999999999998,"5min":0.755,"15min":1.042},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.629,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48499999999999999,"15min":0.50600000000000001},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.58199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.36099999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76800000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.41599999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.90500000000000003},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.74399999999999999},"min":{"1min":0.61499999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":1.1379999999999999,"15min":1.242},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.76100000000000001},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":0.97299999999999998,"15min":1.198},"last":0.78700000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.26},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72499999999999998,"15min":0.76300000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.48099999999999998},"max":{"1min":0.874,"5min":1.1299999999999999,"15min":1.163},"last":0.755}]},{"osd":6,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89200000000000002,"5min":1.105,"15min":1.323},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.78300000000000003},"min":{"1min":0.58099999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.3100000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76100000000000001,"15min":0.81699999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.52400000000000002},"max":{"1min":0.92200000000000004,"5min":1.056,"15min":1.264},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.80400000000000005},"min":{"1min":0.57599999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93500000000000005,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41499999999999998,"15min":0.42599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56699999999999995,"5min":0.64600000000000002,"15min":0.77800000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44700000000000001,"15min":0.47499999999999998},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.94099999999999995},"last":0.371}]},{"osd":1,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.28299999999999997,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.71699999999999997,"15min":0.88700000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75600000000000001,"5min":0.80600000000000005,"15min":0.93600000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.502,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70899999999999996,"5min":0.77300000000000002,"15min":0.78900000000000003},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70899999999999996,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":1.022,"15min":1.1299999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71999999999999997,"15min":0.77300000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":0.96899999999999997,"5min":1.085,"15min":1.1699999999999999},"last":0.86399999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.76400000000000001},"min":{"1min":0.53200000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.96799999999999997,"5min":1.042,"15min":1.099},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.78500000000000003},"min":{"1min":0.53700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.163},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.754,"15min":0.81100000000000005},"min":{"1min":0.61099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94699999999999995,"5min":1.056,"15min":1.21},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.024,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77000000000000002,"15min":0.83399999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0660000000000001,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":0.84699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.435,"15min":0.45100000000000001},"min":{"1min":0.35199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70799999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.76200000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.505}]},{"osd":1,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.65300000000000002,"5min":0.999,"15min":0.999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.98999999999999999,"15min":1.014},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.752,"15min":0.79400000000000004},"min":{"1min":0.52400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.88300000000000001,"5min":1.1220000000000001,"15min":1.2669999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78900000000000003,"15min":0.81899999999999995},"min":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.54200000000000004},"max":{"1min":0.93200000000000005,"5min":1.113,"15min":1.3540000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97199999999999998,"5min":1.093,"15min":1.204},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.81399999999999995},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98499999999999999,"5min":1.208,"15min":1.329},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.79400000000000004},"min":{"1min":0.52300000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.96099999999999997,"5min":1.0880000000000001,"15min":1.484},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.82099999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.2809999999999999,"15min":1.629},"last":0.61599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89500000000000002,"5min":1.1379999999999999,"15min":1.5620000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79500000000000004,"15min":0.81999999999999995},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.325},"last":0.82399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.67300000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35599999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.77},"last":0.70199999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68400000000000005,"15min":0.72099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":1.0209999999999999,"15min":1.6419999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1899999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.48399999999999999,"5min":0.46700000000000003,"15min":0.39800000000000002},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.628},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72599999999999998,"15min":0.78400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.42199999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.93400000000000005},"min":{"1min":0.58599999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":7.742},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.079},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71499999999999997,"15min":0.73999999999999999},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.181},"last":0.80700000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.72399999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.2669999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.436},"max":{"1min":0.98799999999999999,"5min":1.093,"15min":1.171},"last":0.85199999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.51300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0249999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.78000000000000003,"15min":0.82999999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0609999999999999,"5min":1.403,"15min":1.571},"last":0.96799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63900000000000001,"15min":0.85099999999999998},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.98099999999999998,"5min":0.98199999999999998,"15min":9.2780000000000005},"last":0.374},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65400000000000003,"15min":0.66600000000000004},"min":{"1min":0.40699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":1.0449999999999999,"15min":1.0469999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66700000000000004,"15min":0.875},"min":{"1min":0.437,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":1.0329999999999999,"15min":9.25},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.91900000000000004},"min":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0549999999999999,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81000000000000005,"5min":1.052,"15min":10.384},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70999999999999996,"15min":0.92500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.95399999999999996},"min":{"1min":0.52800000000000002,"5min":0.502,"15min":0.495},"max":{"1min":0.879,"5min":1.76,"15min":10.205},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.46100000000000002},"max":{"1min":0.92600000000000005,"5min":1.097,"15min":1.292},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.069,"15min":1.069},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0409999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77700000000000002,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.298},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95899999999999996,"5min":1.556,"15min":1.556},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.79900000000000004},"min":{"1min":0.47999999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.91800000000000004,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:26:38.401 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-10-06T05:16:54.371947+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=16, tm_sec=54, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:26:38.401 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-10-06T05:16:54.371947+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=16, tm_sec=54, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:26:38.401 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-10-06T05:16:54.371947+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=16, tm_sec=54, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:26:38.401 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T05:16:54.371947+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=16, tm_sec=54, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:26:38.401 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-10-06T05:16:54.371947+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=16, tm_sec=54, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:26:38.402 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T05:16:54.371947+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=16, tm_sec=54, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:26:38.402 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-10-06T05:16:54.371947+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=16, tm_sec=54, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=26, tm_sec=31, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:26:38.402 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T05:26:58.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:26:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.609+0000 7f7b777ff700 1 Processor -- start 2023-10-06T05:26:58.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.609+0000 7f7b777ff700 1 -- start start 2023-10-06T05:26:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.610+0000 7f7b777ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700c7b60 0x7f7b70069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.610+0000 7f7b777ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b700aee70 0x7f7b7013f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.610+0000 7f7b777ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b700ca5b0 con 0x7f7b700c7f30 2023-10-06T05:26:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.610+0000 7f7b777ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b700af2d0 con 0x7f7b700aee70 2023-10-06T05:26:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.610+0000 7f7b777ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7006f740 con 0x7f7b700c7b60 2023-10-06T05:26:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.610+0000 7f7b75d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b700aee70 0x7f7b7013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.611+0000 7f7b75d9c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7b700aee70 0x7f7b7013f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52430/0 (socket says 172.21.15.31:52430) 2023-10-06T05:26:58.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.611+0000 7f7b75d9c700 1 -- 172.21.15.31:0/2475663758 learned_addr learned my addr 172.21.15.31:0/2475663758 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.611+0000 7f7b7559b700 1 --2- 172.21.15.31:0/2475663758 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700c7b60 0x7f7b70069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.611+0000 7f7b75d9c700 1 -- 172.21.15.31:0/2475663758 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700c7b60 msgr2=0x7f7b70069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.611+0000 7f7b75d9c700 1 --2- 172.21.15.31:0/2475663758 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700c7b60 0x7f7b70069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.611+0000 7f7b75d9c700 1 -- 172.21.15.31:0/2475663758 >> v1:172.21.15.62:6789/0 conn(0x7f7b700c7f30 legacy=0x7f7b7006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.611+0000 7f7b75d9c700 1 -- 172.21.15.31:0/2475663758 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b700c9c80 con 0x7f7b700aee70 2023-10-06T05:26:58.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.611+0000 7f7b75d9c700 1 --2- 172.21.15.31:0/2475663758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700aee70 0x7f7b7013f690 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009f20 tx=0x7f7b68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9031efdebc309144 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:58.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.611+0000 7f7b67fff700 1 -- 172.21.15.31:0/2475663758 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800e070 con 0x7f7b700aee70 2023-10-06T05:26:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.612+0000 7f7b67fff700 1 -- 172.21.15.31:0/2475663758 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6800ba80 con 0x7f7b700aee70 2023-10-06T05:26:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.612+0000 7f7b67fff700 1 -- 172.21.15.31:0/2475663758 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680092e0 con 0x7f7b700aee70 2023-10-06T05:26:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.612+0000 7f7b777ff700 1 -- 172.21.15.31:0/2475663758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700aee70 msgr2=0x7f7b7013f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.612+0000 7f7b777ff700 1 --2- 172.21.15.31:0/2475663758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700aee70 0x7f7b7013f690 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009f20 tx=0x7f7b68009840 comp rx=0 tx=0).stop 2023-10-06T05:26:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.612+0000 7f7b777ff700 1 -- 172.21.15.31:0/2475663758 shutdown_connections 2023-10-06T05:26:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.612+0000 7f7b777ff700 1 --2- 172.21.15.31:0/2475663758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700aee70 0x7f7b7013f690 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.612+0000 7f7b777ff700 1 --2- 172.21.15.31:0/2475663758 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700c7b60 0x7f7b70069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.612+0000 7f7b777ff700 1 -- 172.21.15.31:0/2475663758 >> 172.21.15.31:0/2475663758 conn(0x7f7b700cc0b0 msgr2=0x7f7b700cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:58.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.612+0000 7f7b777ff700 1 -- 172.21.15.31:0/2475663758 shutdown_connections 2023-10-06T05:26:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.612+0000 7f7b777ff700 1 -- 172.21.15.31:0/2475663758 wait complete. 2023-10-06T05:26:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.614+0000 7f7b777ff700 1 Processor -- start 2023-10-06T05:26:58.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.614+0000 7f7b777ff700 1 -- start start 2023-10-06T05:26:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b777ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700aee70 0x7f7b700ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b777ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700c7b60 0x7f7b70130a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b777ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b700c7f30 0x7f7b700c1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b777ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700aeaa0 con 0x7f7b700c7b60 2023-10-06T05:26:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b777ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700c43d0 con 0x7f7b700c7f30 2023-10-06T05:26:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b777ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b70130f60 con 0x7f7b700aee70 2023-10-06T05:26:58.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b74d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700c7b60 0x7f7b70130a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b74d9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700c7b60 0x7f7b70130a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52432/0 (socket says 172.21.15.31:52432) 2023-10-06T05:26:58.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b74d9a700 1 -- 172.21.15.31:0/2621904702 learned_addr learned my addr 172.21.15.31:0/2621904702 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:58.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b7559b700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700aee70 0x7f7b700ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.615+0000 7f7b75d9c700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b700c7f30 0x7f7b700c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b74d9a700 1 -- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700aee70 msgr2=0x7f7b700ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b74d9a700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700aee70 0x7f7b700ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b74d9a700 1 -- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b700c7f30 msgr2=0x7f7b700c1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b74d9a700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b700c7f30 0x7f7b700c1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b74d9a700 1 -- 172.21.15.31:0/2621904702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b68009ae0 con 0x7f7b700c7b60 2023-10-06T05:26:58.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b7559b700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700aee70 0x7f7b700ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:58.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b74d9a700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700c7b60 0x7f7b70130a20 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c4e0 tx=0x7f7b6000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:58.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b657fa700 1 -- 172.21.15.31:0/2621904702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60011590 con 0x7f7b700c7b60 2023-10-06T05:26:58.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b657fa700 1 -- 172.21.15.31:0/2621904702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b60011710 con 0x7f7b700c7b60 2023-10-06T05:26:58.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b777ff700 1 -- 172.21.15.31:0/2621904702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7013d080 con 0x7f7b700c7b60 2023-10-06T05:26:58.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b657fa700 1 -- 172.21.15.31:0/2621904702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60011ad0 con 0x7f7b700c7b60 2023-10-06T05:26:58.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.616+0000 7f7b777ff700 1 -- 172.21.15.31:0/2621904702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b70169be0 con 0x7f7b700c7b60 2023-10-06T05:26:58.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.617+0000 7f7b777ff700 1 -- 172.21.15.31:0/2621904702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7004c380 con 0x7f7b700c7b60 2023-10-06T05:26:58.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.618+0000 7f7b657fa700 1 -- 172.21.15.31:0/2621904702 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f7b6001b3b0 con 0x7f7b700c7b60 2023-10-06T05:26:58.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.619+0000 7f7b657fa700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b540679e0 0x7f7b54069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:58.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.619+0000 7f7b657fa700 1 -- 172.21.15.31:0/2621904702 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f7b60082690 con 0x7f7b700c7b60 2023-10-06T05:26:58.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.620+0000 7f7b7559b700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b540679e0 0x7f7b54069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:58.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.620+0000 7f7b7559b700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b540679e0 0x7f7b54069e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7b700abcf0 tx=0x7f7b6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:58.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.624+0000 7f7b657fa700 1 -- 172.21.15.31:0/2621904702 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b600510a0 con 0x7f7b700c7b60 2023-10-06T05:26:58.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.824+0000 7f7b777ff700 1 -- 172.21.15.31:0/2621904702 --> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b700c45a0 con 0x7f7b540679e0 2023-10-06T05:26:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.827+0000 7f7b657fa700 1 -- 172.21.15.31:0/2621904702 <== mgr.4101 v2:172.21.15.62:6836/114414 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73036 (secure 0 0 0) 0x7f7b6c00f790 con 0x7f7b540679e0 2023-10-06T05:26:58.830 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.829+0000 7f7b777ff700 1 -- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b540679e0 msgr2=0x7f7b54069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.829+0000 7f7b777ff700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b540679e0 0x7f7b54069e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7b700abcf0 tx=0x7f7b6c008040 comp rx=0 tx=0).stop 2023-10-06T05:26:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.829+0000 7f7b777ff700 1 -- 172.21.15.31:0/2621904702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700c7b60 msgr2=0x7f7b70130a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.829+0000 7f7b777ff700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700c7b60 0x7f7b70130a20 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c4e0 tx=0x7f7b6000c8a0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.829+0000 7f7b777ff700 1 -- 172.21.15.31:0/2621904702 shutdown_connections 2023-10-06T05:26:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.829+0000 7f7b777ff700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f7b540679e0 0x7f7b54069e90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.829+0000 7f7b777ff700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b700c7f30 0x7f7b700c1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.829+0000 7f7b777ff700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b700aee70 0x7f7b700ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.829+0000 7f7b777ff700 1 --2- 172.21.15.31:0/2621904702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b700c7b60 0x7f7b70130a20 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.830+0000 7f7b777ff700 1 -- 172.21.15.31:0/2621904702 >> 172.21.15.31:0/2621904702 conn(0x7f7b700cc0b0 msgr2=0x7f7b700b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.830+0000 7f7b777ff700 1 -- 172.21.15.31:0/2621904702 shutdown_connections 2023-10-06T05:26:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:58.830+0000 7f7b777ff700 1 -- 172.21.15.31:0/2621904702 wait complete. 2023-10-06T05:26:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-10-06T05:26:58.848 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2023-10-06T05:26:57.540110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428680,"kb_used_data":17608,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546040,"statfs":{"total":773094113280,"available":770607144960,"internally_reserved":0,"allocated":18030592,"data_stored":9026992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002741"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":140,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:41.918604+0000","last_change":"2023-10-06T05:26:41.918570+0000","last_active":"2023-10-06T05:26:41.918604+0000","last_peered":"2023-10-06T05:26:41.918604+0000","last_clean":"2023-10-06T05:26:41.918604+0000","last_became_active":"2023-10-06T05:16:55.391778+0000","last_became_peered":"2023-10-06T05:16:55.391778+0000","last_unstale":"2023-10-06T05:26:41.918604+0000","last_undegraded":"2023-10-06T05:26:41.918604+0000","last_fullsized":"2023-10-06T05:26:41.918604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:41.918455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:41.918455+0000","last_clean_scrub_stamp":"2023-10-06T05:26:41.918455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:54:33.972363+0000","scrub_duration":0.0028251470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":140,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:42.882585+0000","last_change":"2023-10-06T05:26:42.882544+0000","last_active":"2023-10-06T05:26:42.882585+0000","last_peered":"2023-10-06T05:26:42.882585+0000","last_clean":"2023-10-06T05:26:42.882585+0000","last_became_active":"2023-10-06T05:16:55.820305+0000","last_became_peered":"2023-10-06T05:16:55.820305+0000","last_unstale":"2023-10-06T05:26:42.882585+0000","last_undegraded":"2023-10-06T05:26:42.882585+0000","last_fullsized":"2023-10-06T05:26:42.882585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:42.882440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:42.882440+0000","last_clean_scrub_stamp":"2023-10-06T05:26:42.882440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:17.335345+0000","scrub_duration":0.0027566180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":137,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:38.658978+0000","last_change":"2023-10-06T05:26:38.658944+0000","last_active":"2023-10-06T05:26:38.658978+0000","last_peered":"2023-10-06T05:26:38.658978+0000","last_clean":"2023-10-06T05:26:38.658978+0000","last_became_active":"2023-10-06T05:16:55.820308+0000","last_became_peered":"2023-10-06T05:16:55.820308+0000","last_unstale":"2023-10-06T05:26:38.658978+0000","last_undegraded":"2023-10-06T05:26:38.658978+0000","last_fullsized":"2023-10-06T05:26:38.658978+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:38.658844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:38.658844+0000","last_clean_scrub_stamp":"2023-10-06T05:26:38.658844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:57.230882+0000","scrub_duration":0.0022007720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":139,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:38.078419+0000","last_change":"2023-10-06T05:26:38.078370+0000","last_active":"2023-10-06T05:26:38.078419+0000","last_peered":"2023-10-06T05:26:38.078419+0000","last_clean":"2023-10-06T05:26:38.078419+0000","last_became_active":"2023-10-06T05:16:55.820230+0000","last_became_peered":"2023-10-06T05:16:55.820230+0000","last_unstale":"2023-10-06T05:26:38.078419+0000","last_undegraded":"2023-10-06T05:26:38.078419+0000","last_fullsized":"2023-10-06T05:26:38.078419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:38.078248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:38.078248+0000","last_clean_scrub_stamp":"2023-10-06T05:26:38.078248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:05.163720+0000","scrub_duration":0.001866513,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":139,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:44.064010+0000","last_change":"2023-10-06T05:26:44.063976+0000","last_active":"2023-10-06T05:26:44.064010+0000","last_peered":"2023-10-06T05:26:44.064010+0000","last_clean":"2023-10-06T05:26:44.064010+0000","last_became_active":"2023-10-06T05:16:55.820209+0000","last_became_peered":"2023-10-06T05:16:55.820209+0000","last_unstale":"2023-10-06T05:26:44.064010+0000","last_undegraded":"2023-10-06T05:26:44.064010+0000","last_fullsized":"2023-10-06T05:26:44.064010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:44.063865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:44.063865+0000","last_clean_scrub_stamp":"2023-10-06T05:26:44.063865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:27:59.374991+0000","scrub_duration":0.0023779309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"166'130","reported_seq":330,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:45.039560+0000","last_change":"2023-10-06T05:26:45.039528+0000","last_active":"2023-10-06T05:26:45.039560+0000","last_peered":"2023-10-06T05:26:45.039560+0000","last_clean":"2023-10-06T05:26:45.039560+0000","last_became_active":"2023-10-06T05:16:55.392531+0000","last_became_peered":"2023-10-06T05:16:55.392531+0000","last_unstale":"2023-10-06T05:26:45.039560+0000","last_undegraded":"2023-10-06T05:26:45.039560+0000","last_fullsized":"2023-10-06T05:26:45.039560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"166'130","last_scrub_stamp":"2023-10-06T05:26:45.039432+0000","last_deep_scrub":"166'130","last_deep_scrub_stamp":"2023-10-06T05:26:45.039432+0000","last_clean_scrub_stamp":"2023-10-06T05:26:45.039432+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:30.032991+0000","scrub_duration":0.0062306480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":154,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:34.953394+0000","last_change":"2023-10-06T05:26:34.953368+0000","last_active":"2023-10-06T05:26:34.953394+0000","last_peered":"2023-10-06T05:26:34.953394+0000","last_clean":"2023-10-06T05:26:34.953394+0000","last_became_active":"2023-10-06T05:16:55.390405+0000","last_became_peered":"2023-10-06T05:16:55.390405+0000","last_unstale":"2023-10-06T05:26:34.953394+0000","last_undegraded":"2023-10-06T05:26:34.953394+0000","last_fullsized":"2023-10-06T05:26:34.953394+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:34.953281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:34.953281+0000","last_clean_scrub_stamp":"2023-10-06T05:26:34.953281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:38.499866+0000","scrub_duration":0.0023260400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":170,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:36.253461+0000","last_change":"2023-10-06T05:26:36.253437+0000","last_active":"2023-10-06T05:26:36.253461+0000","last_peered":"2023-10-06T05:26:36.253461+0000","last_clean":"2023-10-06T05:26:36.253461+0000","last_became_active":"2023-10-06T05:16:55.391402+0000","last_became_peered":"2023-10-06T05:16:55.391402+0000","last_unstale":"2023-10-06T05:26:36.253461+0000","last_undegraded":"2023-10-06T05:26:36.253461+0000","last_fullsized":"2023-10-06T05:26:36.253461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:36.253357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:36.253357+0000","last_clean_scrub_stamp":"2023-10-06T05:26:36.253357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:51.646685+0000","scrub_duration":0.002185513,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":156,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T05:26:37.327987+0000","last_change":"2023-10-06T05:26:37.327941+0000","last_active":"2023-10-06T05:26:37.327987+0000","last_peered":"2023-10-06T05:26:37.327987+0000","last_clean":"2023-10-06T05:26:37.327987+0000","last_became_active":"2023-10-06T05:16:55.891991+0000","last_became_peered":"2023-10-06T05:16:55.891991+0000","last_unstale":"2023-10-06T05:26:37.327987+0000","last_undegraded":"2023-10-06T05:26:37.327987+0000","last_fullsized":"2023-10-06T05:26:37.327987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:26:37.327831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:26:37.327831+0000","last_clean_scrub_stamp":"2023-10-06T05:26:37.327831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:40.123685+0000","scrub_duration":0.002086105,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304020,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067820,"statfs":{"total":96636764160,"available":96325447680,"internally_reserved":0,"allocated":2699264,"data_stored":1571150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68999999999999995,"15min":0.68899999999999995},"min":{"1min":0.47699999999999998,"5min":0.375,"15min":0.36099999999999999},"max":{"1min":0.96099999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98399999999999999,"5min":1.081,"15min":1.081},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.72299999999999998},"min":{"1min":0.46300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.873,"5min":0.999,"15min":1.0109999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.72899999999999998},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81899999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89400000000000002,"5min":0.91700000000000004,"15min":1.018},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68400000000000005,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.41699999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.056},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.70299999999999996,"15min":0.73399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.44800000000000001},"max":{"1min":1.002,"5min":1.149,"15min":1.149},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.77900000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0249999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41499999999999998},"max":{"1min":0.92700000000000005,"5min":1.27,"15min":1.27},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96699999999999997,"5min":0.97899999999999998,"15min":0.999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.80000000000000004},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.80300000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.84999999999999998},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.1040000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79800000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.441},"max":{"1min":1.0600000000000001,"5min":1.248,"15min":1.248},"last":0.751}]}]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.38500000000000001,"15min":0.32300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.8500000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66300000000000003,"15min":0.69399999999999995},"min":{"1min":0.46400000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":1.2170000000000001,"15min":2.0760000000000001},"last":0.74399999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.45100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93100000000000005,"5min":1.177,"15min":1.9279999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85299999999999998,"5min":1.073,"15min":1.4239999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82799999999999996,"5min":1.093,"15min":1.369},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0549999999999999,"5min":1.085,"15min":1.891},"last":0.94999999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.51000000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99399999999999999,"5min":1.018,"15min":1.347},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.59999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.96699999999999997,"5min":0.98699999999999999,"15min":1},"last":0.91200000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71299999999999997,"15min":0.753},"min":{"1min":0.46700000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1060000000000001,"5min":1.1359999999999999,"15min":1.9670000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79000000000000004,"15min":0.80500000000000005},"min":{"1min":0.49399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0600000000000001,"5min":1.278,"15min":1.4510000000000001},"last":0.97999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.46700000000000003,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":1.03,"5min":1.0680000000000001,"15min":1.415},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.79700000000000004},"min":{"1min":0.59699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1439999999999999,"5min":1.2350000000000001,"15min":2},"last":0.83899999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:26:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.60299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.071,"5min":1.0760000000000001,"15min":1.4790000000000001},"last":0.876},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.60999999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0880000000000001,"5min":1.133,"15min":1.46},"last":0.85899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304020,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067820,"statfs":{"total":96636764160,"available":96325447680,"internally_reserved":0,"allocated":2699264,"data_stored":1571150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42399999999999999,"15min":0.435},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.75800000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45200000000000001,"15min":0.48299999999999998},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53000000000000003,"5min":0.60099999999999998,"15min":0.82099999999999995},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"max":{"1min":0.58899999999999997,"5min":0.71399999999999997,"15min":0.83799999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.32400000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.753,"15min":0.78700000000000003},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.59199999999999997,"5min":0.78400000000000003,"15min":0.92400000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68799999999999994,"5min":0.68999999999999995,"15min":0.88600000000000001},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.78300000000000003},"min":{"1min":0.60899999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89900000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.016},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.55600000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91700000000000004,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.96899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71899999999999997,"15min":0.74299999999999999},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.97599999999999998,"5min":1.169,"15min":1.169},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.80400000000000005},"min":{"1min":0.56699999999999995,"5min":0.5,"15min":0.48599999999999999},"max":{"1min":0.86199999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.71299999999999997,"15min":0.74399999999999999},"min":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88800000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.79400000000000004},"min":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.86799999999999999,"5min":1.387,"15min":1.387},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77000000000000002,"15min":0.80800000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.50600000000000001},"max":{"1min":0.93500000000000005,"5min":1.244,"15min":1.244},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78300000000000003,"15min":0.81599999999999995},"min":{"1min":0.58699999999999997,"5min":0.55300000000000005,"15min":0.52200000000000002},"max":{"1min":0.88800000000000001,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.89500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574973,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41099999999999998,"15min":0.42699999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.84499999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.47699999999999998},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59699999999999998,"5min":0.755,"15min":1.042},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.629,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48499999999999999,"15min":0.50600000000000001},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.58199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76800000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.41599999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.90500000000000003},"last":0.56599999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70899999999999996,"15min":0.74399999999999999},"min":{"1min":0.61499999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":1.1379999999999999,"15min":1.242},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.76100000000000001},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":0.97299999999999998,"15min":1.198},"last":0.878}]},{"osd":5,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72599999999999998,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.26},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72499999999999998,"15min":0.76300000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.48099999999999998},"max":{"1min":0.874,"5min":1.1299999999999999,"15min":1.163},"last":0.78100000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.78200000000000003},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89200000000000002,"5min":1.105,"15min":1.323},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.78300000000000003},"min":{"1min":0.58099999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.3100000000000001},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76100000000000001,"15min":0.81699999999999995},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.52400000000000002},"max":{"1min":0.92200000000000004,"5min":1.056,"15min":1.264},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.80400000000000005},"min":{"1min":0.57599999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93500000000000005,"5min":1.1719999999999999,"15min":1.3680000000000001},"last":0.91000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41499999999999998,"15min":0.42599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56699999999999995,"5min":0.64600000000000002,"15min":0.77800000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44700000000000001,"15min":0.47499999999999998},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.57599999999999996,"5min":0.69099999999999995,"15min":0.94099999999999995},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.28299999999999997,"5min":0.27400000000000002,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.71699999999999997,"15min":0.88700000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.78900000000000003,"15min":0.80300000000000005},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75600000000000001,"5min":0.80600000000000005,"15min":0.93600000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.502,"15min":0.52800000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70899999999999996,"5min":0.77300000000000002,"15min":0.78900000000000003},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70899999999999996,"15min":0.74399999999999999},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":1.022,"15min":1.1299999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71999999999999997,"15min":0.77300000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":0.96899999999999997,"5min":1.085,"15min":1.1699999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.76400000000000001},"min":{"1min":0.53200000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.96799999999999997,"5min":1.042,"15min":1.099},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.78500000000000003},"min":{"1min":0.53700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.163},"last":0.75700000000000001}]},{"osd":6,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.754,"15min":0.81100000000000005},"min":{"1min":0.61099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94699999999999995,"5min":1.056,"15min":1.21},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74299999999999999,"15min":0.79500000000000004},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.77600000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.024,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77000000000000002,"15min":0.83399999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0660000000000001,"5min":1.1080000000000001,"15min":1.1439999999999999},"last":0.81299999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.435,"15min":0.45100000000000001},"min":{"1min":0.35199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70799999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.307,"15min":0.28399999999999997},"max":{"1min":0.76200000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.63200000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.52100000000000002},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.65300000000000002,"5min":0.999,"15min":0.999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.98999999999999999,"15min":1.014},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.752,"15min":0.79400000000000004},"min":{"1min":0.52400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.88300000000000001,"5min":1.1220000000000001,"15min":1.2669999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78900000000000003,"15min":0.81899999999999995},"min":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.54200000000000004},"max":{"1min":0.93200000000000005,"5min":1.113,"15min":1.3540000000000001},"last":0.85199999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97199999999999998,"5min":1.093,"15min":1.204},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76600000000000001,"15min":0.81399999999999995},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98499999999999999,"5min":1.208,"15min":1.329},"last":0.89600000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.79400000000000004},"min":{"1min":0.52300000000000002,"5min":0.501,"15min":0.501},"max":{"1min":0.96099999999999997,"5min":1.0880000000000001,"15min":1.484},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.82099999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52000000000000002},"max":{"1min":1.0389999999999999,"5min":1.2809999999999999,"15min":1.629},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89500000000000002,"5min":1.1379999999999999,"15min":1.5620000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79500000000000004,"15min":0.81999999999999995},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.53600000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.325},"last":0.753}]}]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.67300000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35599999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.77},"last":0.92800000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68400000000000005,"15min":0.72099999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":1.0209999999999999,"15min":1.6419999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1899999999999999},"last":0.95299999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.48399999999999999,"5min":0.46700000000000003,"15min":0.39800000000000002},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72599999999999998,"15min":0.78400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.42199999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":2.6709999999999998},"last":0.89100000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.93400000000000005},"min":{"1min":0.58599999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":7.742},"last":0.84999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.079},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71499999999999997,"15min":0.73999999999999999},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.181},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69899999999999995,"15min":0.72399999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.2669999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.436},"max":{"1min":0.98799999999999999,"5min":1.093,"15min":1.171},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.81399999999999995},"min":{"1min":0.51300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0249999999999999,"5min":1.1899999999999999,"15min":1.228},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.78000000000000003,"15min":0.82999999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0609999999999999,"5min":1.403,"15min":1.571},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303440,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068400,"statfs":{"total":96636764160,"available":96326041600,"internally_reserved":0,"allocated":2105344,"data_stored":980782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.63900000000000001,"15min":0.85099999999999998},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36299999999999999},"max":{"1min":0.98099999999999998,"5min":0.98199999999999998,"15min":9.2780000000000005},"last":0.623},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65400000000000003,"15min":0.66600000000000004},"min":{"1min":0.40699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":1.0449999999999999,"15min":1.0469999999999999},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66700000000000004,"15min":0.875},"min":{"1min":0.437,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":1.0329999999999999,"15min":9.25},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.91900000000000004},"min":{"1min":0.42599999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0549999999999999,"5min":1.6910000000000001,"15min":10.108000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81000000000000005,"5min":1.052,"15min":10.384},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70999999999999996,"15min":0.92500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.1539999999999999,"15min":9.1699999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.95399999999999996},"min":{"1min":0.52800000000000002,"5min":0.502,"15min":0.495},"max":{"1min":0.879,"5min":1.76,"15min":10.205},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.46100000000000002},"max":{"1min":0.92600000000000005,"5min":1.097,"15min":1.292},"last":0.72399999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.069,"15min":1.069},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0409999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.53200000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.85599999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77700000000000002,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.298},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:26:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77800000000000002,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95899999999999996,"5min":1.556,"15min":1.556},"last":0.752},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76800000000000002,"15min":0.79900000000000004},"min":{"1min":0.47999999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.91800000000000004,"5min":1.1479999999999999,"15min":1.2809999999999999},"last":0.84099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:26:58.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T05:26:59.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.039+0000 7f56d67ff700 1 Processor -- start 2023-10-06T05:26:59.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.039+0000 7f56d67ff700 1 -- start start 2023-10-06T05:26:59.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56d67ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56d015ca40 0x7f56d0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56d67ff700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d0068da0 con 0x7f56d00c8f40 2023-10-06T05:26:59.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56d67ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d0069080 con 0x7f56d006a300 2023-10-06T05:26:59.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56d67ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d00cabc0 con 0x7f56d015ca40 2023-10-06T05:26:59.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56cffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56d015ca40 0x7f56d0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56cffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56d015ca40 0x7f56d0068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52434/0 (socket says 172.21.15.31:52434) 2023-10-06T05:26:59.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56d4d9c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f56d00c8f40 0x7f56d00adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54388/0 (socket says 172.21.15.31:54388) 2023-10-06T05:26:59.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56cffff700 1 -- 172.21.15.31:0/1488759135 learned_addr learned my addr 172.21.15.31:0/1488759135 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:59.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56ceffd700 1 -- 172.21.15.31:0/1488759135 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179474147 0 0) 0x7f56d0068da0 con 0x7f56d00c8f40 2023-10-06T05:26:59.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56ceffd700 1 -- 172.21.15.31:0/1488759135 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b8003570 con 0x7f56d00c8f40 2023-10-06T05:26:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56cffff700 1 -- 172.21.15.31:0/1488759135 >> v1:172.21.15.62:6789/0 conn(0x7f56d006a300 legacy=0x7f56d00c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56cffff700 1 -- 172.21.15.31:0/1488759135 >> v1:172.21.15.31:6789/0 conn(0x7f56d00c8f40 legacy=0x7f56d00adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.041+0000 7f56cffff700 1 -- 172.21.15.31:0/1488759135 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d0069360 con 0x7f56d015ca40 2023-10-06T05:26:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.042+0000 7f56cffff700 1 --2- 172.21.15.31:0/1488759135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d015ca40 0x7f56d0068670 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f56c00099b0 tx=0x7f56c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70822af4219d022e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.042+0000 7f56ceffd700 1 -- 172.21.15.31:0/1488759135 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0009ea0 con 0x7f56d015ca40 2023-10-06T05:26:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.042+0000 7f56ceffd700 1 -- 172.21.15.31:0/1488759135 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c000b860 con 0x7f56d015ca40 2023-10-06T05:26:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.042+0000 7f56ceffd700 1 -- 172.21.15.31:0/1488759135 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c000bc00 con 0x7f56d015ca40 2023-10-06T05:26:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.042+0000 7f56d67ff700 1 -- 172.21.15.31:0/1488759135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d015ca40 msgr2=0x7f56d0068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.042+0000 7f56d67ff700 1 --2- 172.21.15.31:0/1488759135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d015ca40 0x7f56d0068670 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f56c00099b0 tx=0x7f56c0009250 comp rx=0 tx=0).stop 2023-10-06T05:26:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.043+0000 7f56d67ff700 1 -- 172.21.15.31:0/1488759135 shutdown_connections 2023-10-06T05:26:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.043+0000 7f56d67ff700 1 --2- 172.21.15.31:0/1488759135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d015ca40 0x7f56d0068670 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.043+0000 7f56d67ff700 1 -- 172.21.15.31:0/1488759135 >> 172.21.15.31:0/1488759135 conn(0x7f56d00ce290 msgr2=0x7f56d00ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.043+0000 7f56d67ff700 1 -- 172.21.15.31:0/1488759135 shutdown_connections 2023-10-06T05:26:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.043+0000 7f56d67ff700 1 -- 172.21.15.31:0/1488759135 wait complete. 2023-10-06T05:26:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.044+0000 7f56d67ff700 1 Processor -- start 2023-10-06T05:26:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.044+0000 7f56d67ff700 1 -- start start 2023-10-06T05:26:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.045+0000 7f56d67ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d006a300 0x7f56d0159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.045+0000 7f56d67ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00c8f40 0x7f56d00c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.045+0000 7f56d67ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56d015ca40 0x7f56d00c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.045+0000 7f56d67ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0159d70 con 0x7f56d006a300 2023-10-06T05:26:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.045+0000 7f56d67ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00ae0f0 con 0x7f56d00c8f40 2023-10-06T05:26:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.045+0000 7f56d67ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00c7180 con 0x7f56d015ca40 2023-10-06T05:26:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.045+0000 7f56cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d006a300 0x7f56d0159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.045+0000 7f56cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d006a300 0x7f56d0159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52448/0 (socket says 172.21.15.31:52448) 2023-10-06T05:26:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.045+0000 7f56cffff700 1 -- 172.21.15.31:0/2398457160 learned_addr learned my addr 172.21.15.31:0/2398457160 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56d4d9c700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56d015ca40 0x7f56d00c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56cf7fe700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00c8f40 0x7f56d00c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56cffff700 1 -- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56d015ca40 msgr2=0x7f56d00c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56cffff700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56d015ca40 0x7f56d00c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56cffff700 1 -- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00c8f40 msgr2=0x7f56d00c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56cffff700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00c8f40 0x7f56d00c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56cffff700 1 -- 172.21.15.31:0/2398457160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c00094f0 con 0x7f56d006a300 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56cf7fe700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00c8f40 0x7f56d00c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56d4d9c700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56d015ca40 0x7f56d00c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56cffff700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d006a300 0x7f56d0159830 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f56c0006f50 tx=0x7f56c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56b7fff700 1 -- 172.21.15.31:0/2398457160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c000bbe0 con 0x7f56d006a300 2023-10-06T05:26:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56b7fff700 1 -- 172.21.15.31:0/2398457160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c0011420 con 0x7f56d006a300 2023-10-06T05:26:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56b7fff700 1 -- 172.21.15.31:0/2398457160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c00117c0 con 0x7f56d006a300 2023-10-06T05:26:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.046+0000 7f56d67ff700 1 -- 172.21.15.31:0/2398457160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d00c6bc0 con 0x7f56d006a300 2023-10-06T05:26:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.047+0000 7f56d67ff700 1 -- 172.21.15.31:0/2398457160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d00add80 con 0x7f56d006a300 2023-10-06T05:26:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.047+0000 7f56d67ff700 1 -- 172.21.15.31:0/2398457160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d004c380 con 0x7f56d006a300 2023-10-06T05:26:59.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.049+0000 7f56b7fff700 1 -- 172.21.15.31:0/2398457160 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f56c0011bb0 con 0x7f56d006a300 2023-10-06T05:26:59.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.049+0000 7f56b7fff700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f56bc0679e0 0x7f56bc069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.049+0000 7f56b7fff700 1 -- 172.21.15.31:0/2398457160 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f56c0084a30 con 0x7f56d006a300 2023-10-06T05:26:59.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.050+0000 7f56cf7fe700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f56bc0679e0 0x7f56bc069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.050+0000 7f56cf7fe700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f56bc0679e0 0x7f56bc069e90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f56c4007fd0 tx=0x7f56c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.053+0000 7f56b7fff700 1 -- 172.21.15.31:0/2398457160 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56c0053670 con 0x7f56d006a300 2023-10-06T05:26:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.257+0000 7f56d67ff700 1 -- 172.21.15.31:0/2398457160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f56d00adf10 con 0x7f56d006a300 2023-10-06T05:26:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.261+0000 7f56b7fff700 1 -- 172.21.15.31:0/2398457160 <== mon.0 v2:172.21.15.31:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f56c0053670 con 0x7f56d006a300 2023-10-06T05:26:59.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.261+0000 7f56b7fff700 1 -- 172.21.15.31:0/2398457160 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v32)=0 v32) v1 ==== 123+0+0 (secure 0 0 0) 0x7f56d00adf10 con 0x7f56d006a300 2023-10-06T05:26:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 -- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f56bc0679e0 msgr2=0x7f56bc069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f56bc0679e0 0x7f56bc069e90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f56c4007fd0 tx=0x7f56c4009040 comp rx=0 tx=0).stop 2023-10-06T05:26:59.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 -- 172.21.15.31:0/2398457160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d006a300 msgr2=0x7f56d0159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d006a300 0x7f56d0159830 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f56c0006f50 tx=0x7f56c0017000 comp rx=0 tx=0).stop 2023-10-06T05:26:59.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 -- 172.21.15.31:0/2398457160 shutdown_connections 2023-10-06T05:26:59.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f56bc0679e0 0x7f56bc069e90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56d015ca40 0x7f56d00c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56d006a300 0x7f56d0159830 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 --2- 172.21.15.31:0/2398457160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00c8f40 0x7f56d00c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 -- 172.21.15.31:0/2398457160 >> 172.21.15.31:0/2398457160 conn(0x7f56d00ce290 msgr2=0x7f56d015aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:59.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 -- 172.21.15.31:0/2398457160 shutdown_connections 2023-10-06T05:26:59.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.264+0000 7f56d67ff700 1 -- 172.21.15.31:0/2398457160 wait complete. 2023-10-06T05:26:59.283 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:26:59.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:26:59.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.477+0000 7f45c79ff700 1 Processor -- start 2023-10-06T05:26:59.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.477+0000 7f45c79ff700 1 -- start start 2023-10-06T05:26:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.478+0000 7f45c79ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45c01596d0 0x7f45c0159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.478+0000 7f45c79ff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45c00c6f80 0x7f45c006a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.478+0000 7f45c79ff700 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c0067f70 con 0x7f45c00c16e0 2023-10-06T05:26:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.478+0000 7f45c79ff700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c006e040 con 0x7f45c00c6f80 2023-10-06T05:26:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.478+0000 7f45c79ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00681c0 con 0x7f45c01596d0 2023-10-06T05:26:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.478+0000 7f45c4f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45c00c6f80 0x7f45c006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.478+0000 7f45c4f9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45c00c6f80 0x7f45c006a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52464/0 (socket says 172.21.15.31:52464) 2023-10-06T05:26:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.478+0000 7f45c4f9a700 1 -- 172.21.15.31:0/3372234899 learned_addr learned my addr 172.21.15.31:0/3372234899 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.479+0000 7f45c579b700 1 --2- 172.21.15.31:0/3372234899 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45c01596d0 0x7f45c0159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.479+0000 7f45c4f9a700 1 -- 172.21.15.31:0/3372234899 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45c01596d0 msgr2=0x7f45c0159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.479+0000 7f45c4f9a700 1 --2- 172.21.15.31:0/3372234899 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45c01596d0 0x7f45c0159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.479+0000 7f45c4f9a700 1 -- 172.21.15.31:0/3372234899 >> v1:172.21.15.62:6789/0 conn(0x7f45c00c16e0 legacy=0x7f45c015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:26:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.479+0000 7f45c4f9a700 1 -- 172.21.15.31:0/3372234899 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c00684a0 con 0x7f45c00c6f80 2023-10-06T05:26:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.479+0000 7f45c4f9a700 1 --2- 172.21.15.31:0/3372234899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c00c6f80 0x7f45c006a930 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f45b00099b0 tx=0x7f45b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bdfe53aa12e987 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.479+0000 7f45b7fff700 1 -- 172.21.15.31:0/3372234899 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000b820 con 0x7f45c00c6f80 2023-10-06T05:26:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.479+0000 7f45b7fff700 1 -- 172.21.15.31:0/3372234899 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f45b000b980 con 0x7f45c00c6f80 2023-10-06T05:26:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.479+0000 7f45b7fff700 1 -- 172.21.15.31:0/3372234899 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000bd40 con 0x7f45c00c6f80 2023-10-06T05:26:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.480+0000 7f45c79ff700 1 -- 172.21.15.31:0/3372234899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c00c6f80 msgr2=0x7f45c006a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.480+0000 7f45c79ff700 1 --2- 172.21.15.31:0/3372234899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c00c6f80 0x7f45c006a930 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f45b00099b0 tx=0x7f45b0009250 comp rx=0 tx=0).stop 2023-10-06T05:26:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.480+0000 7f45c79ff700 1 -- 172.21.15.31:0/3372234899 shutdown_connections 2023-10-06T05:26:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.480+0000 7f45c79ff700 1 --2- 172.21.15.31:0/3372234899 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45c01596d0 0x7f45c0159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.480+0000 7f45c79ff700 1 --2- 172.21.15.31:0/3372234899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c00c6f80 0x7f45c006a930 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.480+0000 7f45c79ff700 1 -- 172.21.15.31:0/3372234899 >> 172.21.15.31:0/3372234899 conn(0x7f45c00c9e80 msgr2=0x7f45c006a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.480+0000 7f45c79ff700 1 -- 172.21.15.31:0/3372234899 shutdown_connections 2023-10-06T05:26:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.480+0000 7f45c79ff700 1 -- 172.21.15.31:0/3372234899 wait complete. 2023-10-06T05:26:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.481+0000 7f45c79ff700 1 Processor -- start 2023-10-06T05:26:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.482+0000 7f45c79ff700 1 -- start start 2023-10-06T05:26:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.482+0000 7f45c79ff700 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45c00c16e0 0x7f45c0146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c79ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c00c6f80 0x7f45c00c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c79ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c01596d0 0x7f45c00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c79ff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00c50c0 con 0x7f45c01596d0 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c79ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00c44b0 con 0x7f45c00c6f80 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c79ff700 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00c4790 con 0x7f45c00c16e0 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c5f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c01596d0 0x7f45c00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c5f9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c01596d0 0x7f45c00b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52480/0 (socket says 172.21.15.31:52480) 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c5f9c700 1 -- 172.21.15.31:0/2238001163 learned_addr learned my addr 172.21.15.31:0/2238001163 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c4f9a700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c00c6f80 0x7f45c00c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c579b700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45c00c16e0 0x7f45c0146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c5f9c700 1 -- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45c00c16e0 msgr2=0x7f45c0146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c5f9c700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45c00c16e0 0x7f45c0146390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c5f9c700 1 -- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c00c6f80 msgr2=0x7f45c00c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c5f9c700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c00c6f80 0x7f45c00c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.483+0000 7f45c5f9c700 1 -- 172.21.15.31:0/2238001163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b00094f0 con 0x7f45c01596d0 2023-10-06T05:26:59.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.484+0000 7f45c5f9c700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c01596d0 0x7f45c00b1b40 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f45b800bc30 tx=0x7f45b8009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.484+0000 7f45b57fa700 1 -- 172.21.15.31:0/2238001163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b800c9b0 con 0x7f45c01596d0 2023-10-06T05:26:59.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.484+0000 7f45b57fa700 1 -- 172.21.15.31:0/2238001163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f45b800cb50 con 0x7f45c01596d0 2023-10-06T05:26:59.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.484+0000 7f45c79ff700 1 -- 172.21.15.31:0/2238001163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c00c4a70 con 0x7f45c01596d0 2023-10-06T05:26:59.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.484+0000 7f45b57fa700 1 -- 172.21.15.31:0/2238001163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8014410 con 0x7f45c01596d0 2023-10-06T05:26:59.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.484+0000 7f45c79ff700 1 -- 172.21.15.31:0/2238001163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c01468d0 con 0x7f45c01596d0 2023-10-06T05:26:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.485+0000 7f45c79ff700 1 -- 172.21.15.31:0/2238001163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c00c5230 con 0x7f45c01596d0 2023-10-06T05:26:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.486+0000 7f45b57fa700 1 -- 172.21.15.31:0/2238001163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 85709+0+0 (secure 0 0 0) 0x7f45b8014800 con 0x7f45c01596d0 2023-10-06T05:26:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.487+0000 7f45b57fa700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45a40679e0 0x7f45a4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:26:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.487+0000 7f45b57fa700 1 -- 172.21.15.31:0/2238001163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5320+0+0 (secure 0 0 0) 0x7f45b80831f0 con 0x7f45c01596d0 2023-10-06T05:26:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.487+0000 7f45c579b700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45a40679e0 0x7f45a4069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:26:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.488+0000 7f45c579b700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45a40679e0 0x7f45a4069e90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f45bc006fd0 tx=0x7f45bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:26:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.491+0000 7f45b57fa700 1 -- 172.21.15.31:0/2238001163 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45b804c5a0 con 0x7f45c01596d0 2023-10-06T05:26:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.733+0000 7f45c79ff700 1 -- 172.21.15.31:0/2238001163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45c004c380 con 0x7f45c01596d0 2023-10-06T05:26:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.733+0000 7f45b57fa700 1 -- 172.21.15.31:0/2238001163 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+3534 (secure 0 0 0) 0x7f45b8051c80 con 0x7f45c01596d0 2023-10-06T05:26:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-10-06T05:26:59.735 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":115,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4242992570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4242992570},{"type":"v1","addr":"172.21.15.31:6835","nonce":4242992570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":7706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2405114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2405114362},{"type":"v1","addr":"172.21.15.62:6834","nonce":2405114362}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8926,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2676480318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2676480318},{"type":"v1","addr":"172.21.15.62:6835","nonce":2676480318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":9579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/3293883161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3293883161},{"type":"v1","addr":"172.21.15.31:6834","nonce":3293883161}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115}],"filesystems":[]} 2023-10-06T05:26:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.735+0000 7f45c79ff700 1 -- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45a40679e0 msgr2=0x7f45a4069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.735+0000 7f45c79ff700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45a40679e0 0x7f45a4069e90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f45bc006fd0 tx=0x7f45bc008040 comp rx=0 tx=0).stop 2023-10-06T05:26:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.735+0000 7f45c79ff700 1 -- 172.21.15.31:0/2238001163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c01596d0 msgr2=0x7f45c00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:26:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.735+0000 7f45c79ff700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c01596d0 0x7f45c00b1b40 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f45b800bc30 tx=0x7f45b8009820 comp rx=0 tx=0).stop 2023-10-06T05:26:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.735+0000 7f45c79ff700 1 -- 172.21.15.31:0/2238001163 shutdown_connections 2023-10-06T05:26:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.735+0000 7f45c79ff700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:6836/114414,v1:172.21.15.62:6837/114414] conn(0x7f45a40679e0 0x7f45a4069e90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.735+0000 7f45c79ff700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45c01596d0 0x7f45c00b1b40 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.735+0000 7f45c79ff700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f45c00c16e0 0x7f45c0146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.735+0000 7f45c79ff700 1 --2- 172.21.15.31:0/2238001163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c00c6f80 0x7f45c00c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:26:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.735+0000 7f45c79ff700 1 -- 172.21.15.31:0/2238001163 >> 172.21.15.31:0/2238001163 conn(0x7f45c00c9e80 msgr2=0x7f45c0067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:26:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.736+0000 7f45c79ff700 1 -- 172.21.15.31:0/2238001163 shutdown_connections 2023-10-06T05:26:59.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-10-06T05:26:59.736+0000 7f45c79ff700 1 -- 172.21.15.31:0/2238001163 wait complete. 2023-10-06T05:26:59.739 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 115 2023-10-06T05:26:59.755 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T05:26:59.755 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T05:26:59.755 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:26:59.755 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:26:59.768 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:26:59.769 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:26:59.769 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:26:59.820 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:26:59.821 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:26:59.821 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:26:59.836 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:26:59.836 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:26:59.836 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:26:59.892 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:26:59.892 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T05:26:59.893 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T05:26:59.893 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:26:59.933 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T05:26:59.933 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T05:26:59.933 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.014 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T05:27:00.015 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T05:27:00.015 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.094 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T05:27:00.094 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T05:27:00.094 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.175 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T05:27:00.175 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T05:27:00.175 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.223 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T05:27:00.223 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T05:27:00.223 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.280 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T05:27:00.281 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T05:27:00.281 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.337 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T05:27:00.338 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T05:27:00.338 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.386 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T05:27:00.386 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T05:27:00.386 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T05:27:00.386 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.447 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T05:27:00.447 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T05:27:00.447 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.514 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T05:27:00.514 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T05:27:00.515 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T05:27:00.515 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.515 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-10-06T05:27:00.535 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T05:27:00.535 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T05:27:00.535 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.553 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T05:27:00.553 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T05:27:00.553 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:27:00.613 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T05:27:00.613 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T05:27:00.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T05:27:00.702 INFO:teuthology.orchestra.run.smithi031.stdout:1696569422.4212732 mon.a (mon.0) 137 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T05:27:00.703 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T05:27:00.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T05:27:00.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T05:27:00.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T05:27:00.972 INFO:teuthology.orchestra.run.smithi031.stdout:1696569422.4212732 mon.a (mon.0) 137 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T05:27:00.973 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:27:00.974 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T05:27:01.261 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:27:01.262 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T05:27:01.497 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:27:01.498 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T05:27:01.712 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:27:01.713 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T05:27:01.924 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:27:01.925 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T05:27:02.223 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:27:02.223 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T05:27:02.432 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:27:02.432 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T05:27:02.637 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:27:02.637 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T05:27:02.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:02.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:02.861 INFO:tasks.ceph:Archiving mon data... 2023-10-06T05:27:02.862 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320/data/mon.a.tgz 2023-10-06T05:27:02.863 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2023-10-06T05:27:02.911 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.xPJ8Jcmmoh 2023-10-06T05:27:02.911 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.xPJ8Jcmmoh 2023-10-06T05:27:03.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.xPJ8Jcmmoh 2023-10-06T05:27:03.296 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.xPJ8Jcmmoh is 1MB 2023-10-06T05:27:03.405 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.xPJ8Jcmmoh 2023-10-06T05:27:03.451 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320/data/mon.b.tgz 2023-10-06T05:27:03.451 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2023-10-06T05:27:03.500 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.95JyeaGzNI 2023-10-06T05:27:03.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.95JyeaGzNI 2023-10-06T05:27:03.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.95JyeaGzNI 2023-10-06T05:27:03.903 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.95JyeaGzNI is 1MB 2023-10-06T05:27:04.011 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.95JyeaGzNI 2023-10-06T05:27:04.057 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320/data/mon.c.tgz 2023-10-06T05:27:04.058 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2023-10-06T05:27:04.143 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.9Zl76F4Pi6 2023-10-06T05:27:04.144 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.9Zl76F4Pi6 2023-10-06T05:27:04.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.9Zl76F4Pi6 2023-10-06T05:27:04.566 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.9Zl76F4Pi6 is 1MB 2023-10-06T05:27:04.675 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.9Zl76F4Pi6 2023-10-06T05:27:04.721 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T05:27:04.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T05:27:04.726 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T05:27:04.832 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T05:27:04.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T05:27:04.837 DEBUG:teuthology.orchestra.run.smithi062:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T05:27:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T05:27:04.921 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T05:27:04.938 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T05:27:04.940 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320/remote/smithi031/crash 2023-10-06T05:27:04.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T05:27:05.010 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320/remote/smithi062/crash 2023-10-06T05:27:05.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T05:27:05.080 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T05:27:05.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:05.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:05.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:05.253 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T05:27:05.325 INFO:tasks.ceph:Compressing logs... 2023-10-06T05:27:05.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:27:05.330 DEBUG:teuthology.orchestra.run.smithi062:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:27:05.515 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-10-06T05:27:10.539 INFO:tasks.ceph:Archiving logs... 2023-10-06T05:27:10.540 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320/remote/smithi031/log 2023-10-06T05:27:10.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T05:27:11.745 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320/remote/smithi062/log 2023-10-06T05:27:11.746 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T05:27:12.406 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T05:27:12.415 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T05:27:12.415 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T05:27:12.420 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T05:27:12.550 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T05:27:12.560 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T05:27:12.560 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:27:12.565 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:27:12.606 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2023-10-06T05:27:12.608 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2023-10-06T05:27:12.614 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:27:12.614 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2023-10-06T05:27:12.614 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +112us[ +111us] +/- 71ms 2023-10-06T05:27:12.614 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -374us[ -375us] +/- 76ms 2023-10-06T05:27:12.614 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +249us[ +249us] +/- 82ms 2023-10-06T05:27:12.614 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:27:12.616 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:27:12.617 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2023-10-06T05:27:12.617 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 +126us[ +126us] +/- 71ms 2023-10-06T05:27:12.617 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv02.front.sepia.ceph.com 3 6 377 13 -347us[ -332us] +/- 76ms 2023-10-06T05:27:12.617 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +260us[ +276us] +/- 82ms 2023-10-06T05:27:12.617 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:27:12.618 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T05:27:12.627 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T05:27:12.628 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T05:27:12.754 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T05:27:12.861 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T05:27:13.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:27:13.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:13.108 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:27:13.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:13.185 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T05:27:13.195 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T05:27:13.205 INFO:teuthology.task.internal:Duration was 1501.594300 seconds 2023-10-06T05:27:13.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T05:27:13.214 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T05:27:13.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T05:27:13.219 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T05:27:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:27:13.337 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:27:13.606 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T05:27:13.606 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2023-10-06T05:27:13.606 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T05:27:13.669 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2023-10-06T05:27:13.669 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T05:27:13.731 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T05:27:13.731 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:27:13.736 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:27:13.822 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T05:27:13.822 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T05:27:13.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T05:27:13.961 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T05:27:14.194 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T05:27:14.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T05:27:14.297 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T05:27:15.384 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T05:27:15.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:27:15.936 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:27:16.058 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2023-10-06T05:27:16.134 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2023-10-06T05:27:16.213 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:27:16.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:16.368 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:27:16.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:27:16.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T05:27:16.424 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T05:27:16.424 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320/remote/smithi031 2023-10-06T05:27:16.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T05:27:16.504 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320/remote/smithi062 2023-10-06T05:27:16.519 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T05:27:16.608 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T05:27:16.608 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T05:27:16.613 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T05:27:16.659 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T05:27:16.842 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T05:27:16.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T05:27:16.886 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T05:27:16.886 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T05:27:16.890 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T05:27:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 05:27 /home/ubuntu/cephtest 2023-10-06T05:27:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 05:27 /home/ubuntu/cephtest 2023-10-06T05:27:16.940 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T05:27:16.953 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T05:27:17.028 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T05:27:17.056 DEBUG:teuthology.nuke:shortname: smithi031 2023-10-06T05:27:17.057 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T05:27:17.080 DEBUG:teuthology.nuke:shortname: smithi062 2023-10-06T05:27:17.080 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T05:27:17.083 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:52:21.445791', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOCtRlN3Pn051NZ34WfUmcvDmyDYi1mMnnAnxhB2BJY7wk7jHyiXEIm5o3wKOQLH9VdtEhJwxUx5MB1PDqcrXk='} 2023-10-06T05:27:17.109 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412320', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:52:21.447283', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsWIedGVlr9JwI163R6gtU8pVRhzz0ug/T52ijS54Zev/RwmSYO1NjHrhaiYHJrMWM5FMNsEeqjICUBTlGjX1Q='} 2023-10-06T05:27:17.129 INFO:teuthology.orchestra.console.smithi031:Power off 2023-10-06T05:27:17.129 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T05:27:17.155 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2023-10-06T05:27:17.156 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:27:17.174 INFO:teuthology.orchestra.console.smithi062:Power off 2023-10-06T05:27:17.295 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T05:27:17.307 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-10-06T05:27:17.318 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2023-10-06T05:27:17.608 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:27:17.636 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-10-06T05:27:21.308 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:27:21.418 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-10-06T05:27:21.647 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:27:21.757 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2023-10-06T05:27:25.422 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:27:26.098 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:27:26.200 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-10-06T05:27:26.310 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2023-10-06T05:27:26.411 INFO:teuthology.orchestra.console.smithi062:Power off completed 2023-10-06T05:27:30.311 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:27:30.421 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-10-06T05:27:30.522 INFO:teuthology.orchestra.console.smithi031:Power off completed 2023-10-06T05:27:30.627 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c client.1-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} duration: 1501.594300031662 failure_reason: '"1696569422.4212732 mon.a (mon.0) 137 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T05:27:30.627 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T05:27:30.708 INFO:teuthology.run:FAIL